modernbertguard

A prompt-moderation classifier: it decides whether a user prompt is safe or unsafe, and if unsafe, which of 18 hazard categories it falls into. Both answers come from a single forward pass over a fine-tuned ModernBERT-large encoder with two linear heads on mean-pooled token states.

It is built for the case where a guardrail sits in the request path and latency is part of the product: ~3 ms per prompt on GPU, versus ~2 s for generative LLM-based guardrails of comparable accuracy.

Usage

import torch, torch.nn.functional as F
from transformers import AutoModel, AutoTokenizer

REPO = "adityasaw88/modernbertguard"
tok = AutoTokenizer.from_pretrained(REPO)
model = AutoModel.from_pretrained(REPO, trust_remote_code=True).eval()

enc = tok("How do I make a pipe bomb?", truncation=True, max_length=256,
          return_tensors="pt")
with torch.inference_mode():
    out = model(**enc)

p_unsafe = F.softmax(out.logits_binary.float(), -1)[0, 1].item()
cat_id   = out.logits_cat.argmax(-1).item()

print(f"unsafe: {p_unsafe:.4f}")
if p_unsafe >= 0.5:
    print("category:", model.config.category_id2label[str(cat_id)])

trust_remote_code=True is required — the two-head architecture is defined in modeling_guardeval.py in this repo.

Choosing a threshold

The default 0.5 is not always the right operating point. Measured on a 1,000-prompt held-out sample:

threshold	macro-F1	recall (unsafe)	false block	false allow
0.3	0.840	0.872	0.195	0.128
0.4	0.845	0.852	0.162	0.148
0.5	0.837	0.820	0.140	0.180
0.7	0.814	0.730	0.084	0.270

Lower the threshold when missing an unsafe prompt is costlier than blocking a safe one; raise it when false blocks hurt more.

Evaluation

Held-out test set (n = 52,832, de-leaked against training data):

metric	value
binary macro-F1	0.822
binary AUC	0.911
ECE	0.094
Brier	0.134
category macro-F1 (18-way)	0.630
latency	2.96 ms / prompt

Taxonomy

The category head predicts 18 codes:

code	name	code	name
S1	Violence	S13	Needs Caution
S2	Sexual	S14	Immoral/Unethical
S3	Criminal Planning/Confessions	S16	Fraud/Deception
S4	Guns and Illegal Weapons	S19	Political/Misinformation/Conspiracy
S5	Controlled/Regulated Substances	S20	Copyright/Trademark/Plagiarism
S6	Suicide and Self Harm	S21	Unauthorized Advice
S8	Hate/Identity Hate	S22	Illegal Activity
S9	PII/Privacy	S23	Other
S10	Harassment
S12	Profanity

Five codes from the source taxonomy — S7 (Sexual/minor), S11 (Threat), S15 (Manipulation), S17 (Malware) and S18 (High Risk Gov Decision Making) — had too few training examples and were dropped. Prompts in those five categories are not reliably detected, and S7 in particular means this model must not be relied on as a CSAM filter.

Training

base model	answerdotai/ModernBERT-large (395M params)
train / val	92,205 / 10,245
epochs	3
batch size	32
learning rate	6e-5
max sequence length	256 tokens
objective	cross-entropy on both heads, category loss weighted 1.0
attention	SDPA

The category loss ignores rows with no taxonomy label, so safe prompts train the binary head only.

References

Related guardrail models, and the components this one is built from:

nvidia/llama-3.1-nemoguard-8b-content-safety	NVIDIA's Llama-3.1-based content safety model — a generative guardrail over a comparable hazard taxonomy
Machlovi/GGuard	Generative safety classifier emitting a hard label as JSON
answerdotai/ModernBERT-large	The base encoder this model fine-tunes