ammarsapru/AI_OR_NOT_CLASSIFICATION

AI vs Real Image Detector — 3-Branch Ensemble

Full write-up, methodology, and deployment code: https://github.com/ammarsapru/ai-detector

Binary classifier that predicts whether an image is a real photograph or AI-generated. It fuses three complementary views of an image:

• CLIP ViT-L-14 (frozen) + trainable adapter — semantic plausibility (768-d)
• EfficientNet-B3 (fine-tuned) — local texture / spatial artifacts (1536-d)
• FFT-CNN (custom, on the Fourier spectrum) — frequency-domain fingerprints left by generator upsampling (512-d)

The three feature vectors (2816-d) are concatenated and passed to a fusion MLP (2816 → 512 → 128 → 2).

• Input: RGB image, resized to 224×224
• Output: 2 logits → softmax. Label map: 0 = real, 1 = ai
• Params: ~317.9M total, ~13.9M trainable

Full architecture write-up (per-branch design rationale): MODEL_ARCHITECTURE.md.

How to use

This is a custom nn.Module, so AutoModel / pipeline() will not load it. Download model.py + best_model.pt from this repo and:

import torch, torch.nn.functional as F
import torchvision.transforms as T
from PIL import Image
from huggingface_hub import hf_hub_download
from model import load_detector  # model.py from this repo

repo = "YOUR-USERNAME/ai-detector"
ckpt = hf_hub_download(repo, "best_model.pt")
device = "cuda" if torch.cuda.is_available() else "cpu"
model, cfg = load_detector(ckpt, device=device)

tf = T.Compose([T.Resize((224, 224)), T.ToTensor()])
img = Image.open("example.jpg").convert("RGB")
with torch.no_grad():
    probs = F.softmax(model(tf(img).unsqueeze(0).to(device)), dim=1)[0]
print({"real": float(probs[0]), "ai": float(probs[1])})

Training data

• AI images: Tiny-GenImage — 8 generators (Midjourney, SD v1.4/v1.5, SDXL, DALL·E 2/3, VQDM, Wukong)
• Real images: Tiny-ImageNet
• Balanced 14,000 real / 14,000 AI; 85/15 train/val split; images resized to 224×224.

Results

In-distribution: 98.5% accuracy / 0.985 macro-F1 (98% real recall, 99% AI recall), n=4,200, 3 epochs.

Out-of-distribution testing revealed an asymmetric generalization profile — both classes have a genuine strength, with one narrow gap:

Probe	Class	Condition	Result	Provenance
AIGC-Detection-Benchmark, standard (n=900, 17 gen.)	AI	full/variable native res, no resize trick	94.0% recall	recovered chat transcript
AIGC-Detection-Benchmark, standard (n=900, 17 gen.)	real	full/variable native res, no resize trick	8.0% recall	recovered chat transcript
Tiny-ImageNet valid, held-out (n=200)	real	native 64×64	97.0% recall	saved classification report
AIGC-Detection-Benchmark, forced to 64×64 (n=900)	AI	forced downscale-then-upscale	68.0% recall	saved classification report
AIGC-Detection-Benchmark, forced to 64×64 (n=900)	real	forced downscale-then-upscale	36.0% recall	saved classification report

AI-image detection already generalizes well to full-resolution external images (94.0% recall, no resize trick needed). Real-image detection performs extremely well within its native 64×64 resolution domain (97.0% recall on a held-out split never seen in training) — essentially matching in-distribution performance. The gap is narrow: that native-resolution strength doesn't yet extend to full-resolution photos (8.0%). Forcing inputs to 64×64 before inference is not a fix: it partially helps real (8.0%→36.0%) but hurts AI (94.0%→68.0%), a net loss. Full methodology, mechanism, and per-generator breakdown: see report.pdf in this repo, or the technical report on GitHub.

Limitations & intended use

• Real-image detection does not generalize to full-resolution photos (8.0% recall) — the central limitation. Works well on natively low-res real photos (97.0%), not on full-resolution ones. Traces to the real-image training source (Tiny-ImageNet) being natively 64×64 — not an architectural flaw, since the AI branch does not show the same failure. See report.pdf §6.
• Do not force inputs to 64×64 before inference as a workaround — it trades a partial real-recall gain (8.0%→36.0%) for a larger AI-recall loss (94.0%→68.0%), net negative.
• One data point (8.0% real / 94.0% AI recall) is recovered from a preserved chat transcript, not the notebook's saved output — the evaluation cell was edited in place to produce the forced-resize version, overwriting the original run's output. See report.pdf §5.1/§7.
• Distribution note: real and AI images came from different source datasets, so in-distribution numbers may be optimistic.
• Cross-generator generalization has not been properly validated — an intended train/test generator exclusion in the OOD benchmark code was not actually enforced (see report.pdf §5.3). Treat any apparent generalization in this report as provisional.
• Intended for research / educational use in synthetic-media detection, not as a sole authority for high-stakes decisions.

Training config

AdamW (per-branch LRs: CLIP adapter 1e-5, EfficientNet 3e-5, FFT 1e-4, head 1e-4), CosineAnnealingLR, grad-clip 1.0, weight decay 1e-4, CrossEntropy loss, augmentation (flip, color jitter, grayscale, blur).