Report

Our testing framework employs a systematic 5-tier image spectrum designed to identify the precise failure points of deepfake detection systems. We create a controlled progression from verified authentic images to obvious AI-generated content, with calibrated distortions at each level. This approach isolates specific vulnerabilities rather than providing generic pass/fail results.

Image Selection & Creation Process

• Tier 1: Verified real images from editorial sources (Reuters, government accounts)
• Tier 2: Real images with single distortions (compression, noise, blur)
• Tier 3: Real images with compound processing (social media filters, format cycles)
• Tier 4: Real images with adversarial attacks (multi-compression, frequency manipulation)
• Tier 5: Pure AI-generated images from latest models (Nano Banana, Runways, face-swap)

Each image undergoes complete metadata scrubbing and neutral file naming to prevent detection bias. Our Python-based processing pipeline uses Pillow, OpenCV, and scikit-image libraries for reproducible distortions. Every transformation is logged with parameters, hashes, and execution times for complete auditability.

Testing Protocol

• 50 images per audit distributed across all tiers
• Systematic demographic and quality balancing
• Hash verification for processing integrity

This methodology reveals not just whether systems fail, but exactly why and where they break down.