pi-r2 β€” DETR for document layout detection (loc_beyond_words)

Fine-tuned DETR (ResNet-50) for detecting layout regions in historical newspaper page scans, trained on biglam/loc_beyond_words (2,846 train / 712 validation images, 7 classes).

Model / license

Classes (7)

Photograph, Illustration, Map, Comics/Cartoon, Editorial Cartoon, Headline, Advertisement

Validation results (COCO-style metrics on the held-out validation split, 712 images)

Metric Value
AP @[IoU .50:.95] 0.423
AP @IoU .50 0.566
AP @IoU .75 0.486
AP small 0.050
AP medium 0.250
AP large 0.453
AR max=1 0.286
AR max=10 0.574
AR max=100 0.625

Per-class AP

Class AP
Photograph n/a
Illustration n/a
Map n/a
Comics/Cartoon n/a
Editorial Cartoon n/a
Headline n/a
Advertisement n/a

Note: AP_small/AP_medium/AP_large are computed on the resized evaluation inputs, not original pixel areas; treat small-object numbers cautiously.

Training details

  • Recipe: standard DETR fine-tuning (AdamW, lr=1e-4, backbone lr=1e-5, wd=1e-4, grad clip 0.1, fp16, linear warmup/decay, batch size 2 with grad accumulation 2).
  • Input: images resized to shortest edge 480px / longest edge 800px (aspect kept), RGB.
  • Epochs: up to 15 with early stopping (patience 3 on AP@.5) and validation every 4 epochs.
  • Hardware: 1x NVIDIA T4 (16GB) via Hugging Face Jobs (t4-small).
  • Cost: < $5 total GPU compute.

Quickstart

from transformers import DetrImageProcessor, DetrForObjectDetection
from PIL import Image
import torch

processor = DetrImageProcessor.from_pretrained("harness-race/pi-r2")
model = DetrForObjectDetection.from_pretrained("harness-race/pi-r2")

image = Image.open("page.png").convert("RGB")
inputs = processor(images=image, return_tensors="pt")
with torch.no_grad():
    outputs = model(**inputs)

results = processor.post_process_object_detection(
    outputs, threshold=0.5, target_sizes=[(image.height, image.width)])[0]
for score, label, box in zip(results["scores"], results["labels"], results["boxes"]):
    print(model.config.id2label[label.item()], round(score.item(), 3), [round(v, 1) for v in box.tolist()])

Files

  • model.safetensors, config.json, preprocessor_config.json β€” fine-tuned model + processor
  • val_metrics.json β€” full COCO validation metrics (incl. per-class AP)
  • figures/ β€” sample predictions on the validation set
  • train_detr.py, requirements.txt β€” training code for reproducibility

Limitations

  • Trained on grayscale newspaper scans; color documents may perform worse.
  • Classes are heavily imbalanced (Headline/Advertisement dominate; Map/Editorial Cartoon are rare).
  • Small text-heavy regions may be missed at 480–800px input resolution.
Downloads last month
12
Safetensors
Model size
41.6M params
Tensor type
F32
Β·
Inference Providers NEW
This model isn't deployed by any Inference Provider. πŸ™‹ Ask for provider support

Model tree for harness-race/pi-r2

Finetuned
(873)
this model

Dataset used to train harness-race/pi-r2