Transformers documentation
Model structure rules
Model structure rules
Transformers enforces a set of static rules on every modeling_*.py, modular_*.py, and configuration_*.py file. The mlinter package provides the checker engine, and the repository keeps its active rule set in utils/rules.toml. That local TOML lets us enable, disable, or tweak rules quickly without waiting for a new transformers-mlinter release.
These are the expected model conventions for adding or changing modeling code. They keep the codebase consistent and ensure compatibility with features like pipeline parallelism, device maps, and weight tying.
Running the checker
make typing runs mlinter alongside the ty type checker through the repo wrapper, so it picks up utils/rules.toml. Run the same wrapper directly with the following commands.
python utils/check_modeling_structure.py # check all modeling files
python utils/check_modeling_structure.py --changed-only # check only files changed vs origin/main
python utils/check_modeling_structure.py --list-rules # list all rules and their enabled status
python utils/check_modeling_structure.py --rule TRF001 # show built-in docs for a specific ruleThe --changed-only flag is the fastest option during development. It only checks the files you’ve modified relative to the main branch. If you invoke mlinter directly instead of the wrapper, pass --rules-toml utils/rules.toml so local overrides are applied.
Fixing a violation
When a rule violation is detected, the error looks like this:
src/transformers/models/acme/modeling_acme.py:18: TRF013: AcmeModel.__init__ does not call self.post_init().Use the rule ID to look up the fix in the rules reference. TRF013 is triggered when a PreTrainedModel subclass doesn’t call self.post_init(). That method performs essential finalization steps, and omitting it causes runtime bugs.
class AcmeModel(AcmePreTrainedModel):
def __init__(self, config):
super().__init__(config)
self.layers = nn.ModuleList(
[AcmeDecoderLayer(config) for _ in range(config.num_hidden_layers)]
)
+ self.post_init()Rules reference
Each rule below lists what it enforces and a diff showing the fix. Run python utils/check_modeling_structure.py --rule TRF001 to see the built-in docs for any rule with the repo’s current rule set.
TRF001
Checks that <Model>PreTrainedModel’s config_class is named <Model>Config. A mismatch can break loading, auto classes and developer expectations.
class AcmePreTrainedModel(PreTrainedModel):
- config_class = WileConfig
+ config_class = AcmeConfigTRF002
Checks base_model_prefix is a non-empty, whitespace-free string literal. Invalid prefixes can break weight-loading key mapping and base-model access.
class AcmePreTrainedModel(PreTrainedModel):
- base_model_prefix = ""
+ base_model_prefix = "model"TRF003
Flags the old if not return_dict: return (x,) pattern in forward. Manual return_dict branching is verbose and easy to get wrong. Let @capture_output or @can_return_tuple do it.
-def forward(self, x, return_dict=None):
- if not return_dict:
- return (x,)
- return AcmeModelOutput(last_hidden_state=x)
+@can_return_tuple
+def forward(self, x):
+ return AcmeModelOutput(last_hidden_state=x)TRF004
Checks that no model class defines a tie_weights method. Overriding tie_weights breaks loading, device_map computation and saving. Declare tied weights in the _tied_weights_keys class attribute.
-def tie_weights(self):
- self.lm_head.weight = self.emb.weight
+class AcmeForCausalLM(AcmePreTrainedModel):
+ _tied_weights_keys = ["lm_head.weight"]TRF005
Checks the shape of _no_split_modules when present. Malformed values can break device-map partitioning and sharding.
-_no_split_modules = [SomeLayerClass, ""]
+_no_split_modules = ["AcmeDecoderLayer", "AcmeAttention"]TRF006
Checks that cache arguments in a forward signature are used in the body. An unused cache argument suggests incomplete caching support and inconsistent API behavior.
def forward(self, x, past_key_values=None, use_cache=False):
+ if use_cache:
+ ...
return xTRF007
Checks for self attribute assignments after self.post_init() in init. Mutating model structure after post_init bypasses its initialization and finalization work.
def __init__(self, config):
...
- self.post_init()
- self.proj = nn.Linear(...)
+ self.proj = nn.Linear(...)
+ self.post_init()TRF008
Checks that add_start_docstrings on a model class gets a non-empty argument. An empty argument produces unclear, low-quality generated API docs.
-@add_start_docstrings("")
+@add_start_docstrings("The Acme model.")
class AcmeModel(AcmePreTrainedModel):
...TRF009
Flags imports into another model’s package from a model’s shipped implementation files: modeling*, configuration, processing_, imageprocessing, videoprocessing, featureextraction, tokenization_ and generation*.py. Three forms count: package path (from transformers.models.other.modeling_other import X), relative path (from ..other.configuration_other import X) and public API (from transformers import OtherModel) — for the last the owning directory is inferred from the class name and confirmed against the classes it defines, so an unresolvable name is left alone. Out of scope: `modular.py(building on another model is its purpose, and the converter flattens those imports away),convert_.py,init.pyand files underauto. Allowed targets: the model's own directory,autoandtimm_wrapper`. One model, one definition: model A’s behavior lives in its own files, and a change to model B must not silently change model A — for every file kind, not just modeling. Reach sub-configs through AutoConfig and CONFIG_MAPPING, and use a modular file to build on another model’s code.
-from transformers.models.llama.modeling_llama import LlamaAttention
-from transformers import CLIPTextModelWithProjection
+# Keep implementation local to this model's own files.
+# To build on another model, write modular_acme.py; to reuse a snippet,
+# copy it with a # Copied from comment.TRF010
Checks that direct PreTrainedConfig/PretrainedConfig subclasses in configuration*.py and modular*.py carry @strict(accept_kwargs=True). Without it a new config misses the repo’s runtime type-validation contract and drifts from the dataclass-based config standard.
+@strict(accept_kwargs=True)
class AcmeConfig(PreTrainedConfig):
...TRF011
In forward() of PreTrainedModel subclasses, flags submodule attribute accesses torch.nn.Identity would not have: on loop variables over self.layers, and self.<submodule>.<attr> where <attr> is not a standard nn.Module attribute. Pipeline parallelism may replace any submodule with torch.nn.Identity, so reading a custom attribute (e.g. decoder_layer.attention_type) off it raises AttributeError at runtime. Read per-layer metadata from self.config.
def forward(self, ...):
- for decoder_layer in self.layers:
+ for i, decoder_layer in enumerate(self.layers):
hidden_states = decoder_layer(
hidden_states,
- attention_mask=causal_mask_mapping[decoder_layer.attention_type],
+ attention_mask=causal_mask_mapping[self.config.layer_types[i]],
)TRF012
Flags in-place ops (.normal(), .zero(), …) on module weights inside _init_weights. Parameters carry internal flags tracking whether they still need re-initialization; in-place ops bypass them. Use the init primitives.
+from transformers import initialization as init
+
def _init_weights(self, module):
- module.weight.normal_(mean=0.0, std=0.02)
+ init.normal_(module.weight, mean=0.0, std=0.02)TRF013
Checks that every PreTrainedModel subclass defining init calls self.post_init(). In modular files super().init() counts, since it propagates the parent’s post_init. post_init does essential finalization (weight init, gradient checkpointing setup, …); skipping it causes subtle runtime bugs.
class AcmeModel(AcmePreTrainedModel):
def __init__(self, config):
super().__init__(config)
self.layers = nn.ModuleList(...)
+ self.post_init()TRF014
Flags trust_remote_code used or passed (e.g. as a kwarg) in native model integration files. trust_remote_code loads arbitrary code, including binaries — a power feature for users, not something a native integration may depend on, since remote code cannot be reviewed or maintained in transformers.
class AcmeModel(AcmePreTrainedModel):
def __init__(self, config):
super().__init__(config)
- self.model = AutoModel.from_pretrained(..., trust_remote_code=True)
+ self.model = AutoModel.from_pretrained(...)TRF015
When a PreTrainedModel subclass sets a non-empty _tied_weights_keys, checks the companion configuration file for a tie_word_embeddings field. Without it users cannot control weight tying: the model ties unconditionally, breaking serialization round-trips and fine-tuning with untied heads.
# configuration_foo.py
@strict(accept_kwargs=True)
class FooConfig(PreTrainedConfig):
hidden_size: int = 768
+ tie_word_embeddings: bool = TrueTRF016
When an imageprocessing.py or videoprocessing.py class declares boolean do_* attributes (do_resize, do_rescale, do_normalize, do_convert_rgb, …) and overrides preprocess() or _preprocess(), checks each flag is still consumed there: referenced directly, delegated via super().preprocess/_preprocess(…, **kwargs), or — image processors only — forwarded through _preprocess_image_like_inputs/_prepare_image_like_inputs. do_sample_frames is exempt: the base preprocess() consumes it before _preprocess() runs. A do_X the override never references is dead: setting do_X=False has no effect, the operation runs anyway, and per-call overrides silently break.
class AcmeVideoProcessor(BaseVideoProcessor):
do_resize = True
do_normalize = True
def _preprocess(
self,
videos,
+ do_resize: bool,
+ do_normalize: bool,
size,
image_mean,
image_std,
**kwargs,
):
for video in videos:
- video = self.resize(video, size=size)
- video = self.normalize(video, image_mean, image_std)
+ if do_resize:
+ video = self.resize(video, size=size)
+ if do_normalize:
+ video = self.normalize(video, image_mean, image_std)TRF017
On classes carrying both @auto_docstring and @dataclass, checks @auto_docstring comes first. Decorators apply bottom-up, so @dataclass on top runs @auto_docstring first, on a class with no synthesized init yet: it then modifies the parent’s init.doc instead of the subclass’s.
-@dataclass
@auto_docstring(
custom_intro="""
Output type of `AcmeForPreTraining`.
"""
)
+@dataclass
class AcmeForPreTrainingOutput(ModelOutput):
...TRF018
Checks that every PreTrainedModel subclass overriding _init_weights(self, module, ...) chains up via super()._init_weights(...). Modular files also accept the sentinels PreTrainedModel._init_weights(self, module), PreTrainedModel._init_weights(module) and raise AttributeError(...). For a deliberate full override, suppress with # trf-ignore: TRF018 above the method. The base _init_weights covers the standard module types (Linear, Embedding, LayerNorm, RotaryEmbedding, …). Skipping it leaves every submodule the override misses uninitialized — which passes tests and surfaces much later as a subtle weight-init bug (cf. https://github.com/huggingface/transformers/pull/45597).
from ... import initialization as init
def _init_weights(self, module):
+ super()._init_weights(module)
if isinstance(module, AcmeCustomLayer):
- module.gate.data.zero_()
+ init.zeros_(module.gate)TRF019
Flags a non-empty _defaults on *ProcessorKwargs TypedDicts in processing_*.py. Models released before the cutoff date are grandfathered. Hardcoded _defaults scatter processor configuration across Python source, are awkward to override from config, and bloat the code. In processor_config.json on the hub they travel with the checkpoint and can be updated without a code change.
class Gemma4ProcessorKwargs(ProcessingKwargs, total=False):
- _defaults = {
- "text_kwargs": {"padding": False},
- "images_kwargs": {"return_tensors": "pt"},
- }
images_kwargs: Gemma4ImageProcessorKwargsTRF020
In model directories whose configuration declares kv_lora_rank (Multi-head Latent Attention), checks the attention class owning the KV LoRA expansion projection (kv_b_proj, or any nn.Linear(config.kv_lora_rank, ...)): the expansion must live in a dedicated method (e.g. expand_kv) that forward() calls, not inline. In modular files a method inherited from an imported base counts. External backends (vLLM/SGLang) override the expansion to consume the compressed KV cache directly. Inlined in forward() there is nothing to override, so the backend must materialize the full cache — losing the memory savings MLA exists for.
+ def expand_kv(self, k_nope, k_pe):
+ key_shape = (*k_nope.shape[:-1], -1, self.qk_nope_head_dim + self.v_head_dim)
+ k_nope = self.kv_b_proj(k_nope).view(key_shape).transpose(1, 2)
+ k_nope, value_states = torch.split(k_nope, [self.qk_nope_head_dim, self.v_head_dim], dim=-1)
+ k_pe = k_pe.expand(*k_nope.shape[:-1], -1)
+ key_states = torch.cat((k_nope, k_pe), dim=-1)
+ return key_states, value_states
+
def forward(self, hidden_states, ...):
...
- k_nope = self.kv_b_proj(k_pass).view(key_shape).transpose(1, 2)
- k_nope, value_states = torch.split(k_nope, [self.qk_nope_head_dim, self.v_head_dim], dim=-1)
- k_pe = k_rot.expand(*k_nope.shape[:-1], -1)
- key_states = torch.cat((k_nope, k_pe), dim=-1)
+ key_states, value_states = self.expand_kv(k_pass, k_rot)TRF021
In modeling*.py and modular.py, flags torch.tensor(<value>, ..., device=<non-cpu>) where <value> provably resolves to a Python scalar — from numeric literals and arithmetic on them, torch.finfo/iinfo fields, scalar-returning builtins and math. calls, locals bound exactly once, self.<attr> assigned in the class body, and config fields annotated int/float/bool in the companion configuration file (following attribute_map). Anything that may also be a sequence (eos_token_id: int | list[int] | None) or that cannot be resolved is left alone. init, _init_weights, post_init and post_init are exempt: they never run inside a capture region. torch.tensor(<python scalar>, device=<accelerator>) materialises the value on the host then copies it to the device; CUDA graph capture forbids that copy, so the model cannot be captured. torch.full((), <value>, dtype=…, device=…) fills the same 0-d tensor on-device with a capturable kernel and no synchronisation.
def get_placeholder_mask(self, input_ids, inputs_embeds):
special_image_mask = (
inputs_embeds
== self.get_input_embeddings()(
- torch.tensor(self.config.image_token_id, dtype=torch.long, device=inputs_embeds.device)
+ torch.full((), self.config.image_token_id, dtype=torch.long, device=inputs_embeds.device)
)
).all(-1)TRF022
Checks that every string in a _no_split_modules list in modeling*.py or modular*.py names a class defined in that file, imported into it, or defined by a sibling module of the same model directory. Complements TRF005, which only checks the value’s shape. device_map matches these strings against module.__class__.__name__ at runtime, so a stale or misspelled name matches nothing and is silently ignored — the module it should keep together can still be split across devices. Delete entries naming another model’s classes rather than correcting them: post_init already collects _no_split_modules from child submodels.
class VideoLlavaPreTrainedModel(PreTrainedModel):
- _no_split_modules = ["VideoLlavaVisionAttention"]TRF023
In configuration*.py and modular.py, flags `Configfields named after an upstream paper's abbreviation instead of the canonical name: d_model/n_embd -> hidden_size, d_ff/d_inner/ffn_dim/ffn_hidden_size/expansion_ratio -> intermediate_size, d_head -> head_dim, n_head/n_heads -> num_attention_heads, n_layer/n_layers/num_blocks -> num_hidden_layers. Fields are read from the class body and from __init__/__post_init__ assignments and defaults. Ambiguous but idiomatic names (num_heads, num_layers, embed_dim, mlp_ratio) are not flagged, and models added before cutoff_date keep theirs. Everything that reads a model's shape -- device_map planning, tensor/pipeline parallel plans, quantization, PEFT, attention-backend selection,attribute_mapconsumers -- looks up the canonical names, so a config spelling the same quantityd_model` silently opts out of all of them. Map the checkpoint’s own spelling in the conversion script.
@strict(accept_kwargs=True)
class AcmeConfig(PreTrainedConfig):
- d_model: int = 1024
- d_ff: int = 4096
- n_heads: int = 16
- n_layers: int = 24
+ hidden_size: int = 1024
+ intermediate_size: int = 4096
+ num_attention_heads: int = 16
+ num_hidden_layers: int = 24TRF024
In modeling*.py and modular.py, flags an integer literal greater than 8 in a dimension argument of a torch.nn constructor (Linear, Embedding, LayerNorm, RMSNorm, GroupNorm, BatchNorm, InstanceNorm, Convd, ConvTranspose*d, Bilinear, MultiheadAttention), positional or by keyword (in_features, out_features, in_channels, out_channels, num_embeddings, embedding_dim, embed_dim, normalized_shape, num_channels, hidden_size). Operator-shape arguments (kernel_size, stride, padding, num_groups) are ignored; literals up to 8 keep scalar heads, binary classifiers and RGB channel counts clean. Models added before cutoff_date are exempt. A hardcoded width pins the module to one checkpoint size: the same architecture at another scale loads with a shape mismatch, and from_pretrained cannot say which value is wrong because no config field points at it. It also splits the source of truth, so editing the config no longer changes the model that gets built.
class AcmeAtomEmbedding(nn.Module):
def __init__(self, config):
super().__init__()
- self.proj = nn.Linear(768, 3072, bias=False)
- self.norm = nn.LayerNorm(3072)
+ self.proj = nn.Linear(config.hidden_size, config.intermediate_size, bias=False)
+ self.norm = nn.LayerNorm(config.intermediate_size)TRF025
In modeling*.py and modular.py, flags calls to a mask factory (the maskingutils entry points create_causal_mask, create_bidirectional_mask, create_sliding_window_causal_mask, create_chunked_causal_mask, create_masks_for_generate, and any create_mask helper) inside a class that does not inherit from PreTrainedModel — plain nn.Module blocks such as layers, attention modules and encoders. Mask construction is O(sequence length squared) work that does not vary per layer, so building it in the layer pays that cost once per layer, and each layer then owns its own mask, so the attention backends can no longer be handed a single prepared one. Build it once in the model and pass it down.
class AcmeLayer(nn.Module):
def forward(self, hidden_states, attention_mask=None, **kwargs):
- attention_mask = create_causal_mask(
- config=self.config, input_embeds=hidden_states, attention_mask=attention_mask, ...
- )
return self.self_attn(hidden_states, attention_mask, **kwargs)
class AcmeModel(AcmePreTrainedModel):
def forward(self, input_ids=None, attention_mask=None, **kwargs):
+ causal_mask = create_causal_mask(
+ config=self.config, input_embeds=inputs_embeds, attention_mask=attention_mask, ...
+ )
for layer in self.layers:
- hidden_states = layer(hidden_states, attention_mask, **kwargs)
+ hidden_states = layer(hidden_states, causal_mask, **kwargs)TRF026
In modeling*.py and modular*.py, flags a non-PreTrainedModel class that defines only init and forward, assigns exactly one self.<attr> in init, and whose forward body is exactly return self.<attr>(...) for it (a leading docstring is ignored). Any other method, extra attribute, statement before the return, or super() call in forward means the class does work of its own. The wrapper adds a level to every weight name, to _no_split_modules, parallelism plans and every conversion mapping, while computing nothing — and readers have to open one more class to find that out. PreTrainedModel subclasses are exempt: they exist for from_pretrained and the auto classes even when forward only delegates.
-class AcmeAtomTransformer(nn.Module):
- def __init__(self, config):
- super().__init__()
- self.encoder = AcmeEncoder(config)
-
- def forward(self, hidden_states, **kwargs):
- return self.encoder(hidden_states, **kwargs)
-
class AcmeModel(AcmePreTrainedModel):
def __init__(self, config):
super().__init__(config)
- self.atom_transformer = AcmeAtomTransformer(config)
+ self.encoder = AcmeEncoder(config)TRF027
Flags any assert statement in modeling*.py, modular.py and configuration_.py. python -O strips asserts, so a shape or config check written as one silently disappears in optimised runs. An assert also gives a bare AssertionError, where a ValueError can name the offending value and say what to do about it.
def forward(self, hidden_states):
- assert hidden_states.dim() == 3
+ if hidden_states.dim() != 3:
+ raise ValueError(f"Expected a 3D tensor, got shape {tuple(hidden_states.shape)}.")TRF028
Checks the first 25 lines of modeling*.py, modular.py, configuration_.py, processing*.py, image_processing.py and videoprocessing.py for a Licensed under the <name> License line followed by every clause of the standard warranty paragraph, from You may obtain a copy of the License at through limitations under the License.. Lines are flattened and lowercased first, so wrapping and comment style do not matter. The license name and copyright line are not checked: they vary per model. A file shipped without the header leaves its provenance ambiguous, and adding it later means touching an already-released file. Matching only Apache License would let through the defects that actually occur: a paragraph truncated mid-way, a header that stops after the URL, or one mangled by a bad search-and-replace.
+# Copyright 2026 The HuggingFace Team. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# ...
"""PyTorch Acme model."""TRF029
In modeling*.py and modular*.py, flags an __init__ taking config alongside an argument whose name is unambiguously a config field (hidden_size, num_attention_heads, intermediate_size, head_dim, num_hidden_layers, embed_dim, dropout, eps, patch_size, rope_theta, …). A parameter optional with a None default is exempt: that is an override, not a second source of truth, and it is how one MLP class serves both the dense and the expert width of a MoE model. A hardcoded default such as hidden_size: int = 1024 is not — it wins over the config whenever the caller passes nothing. kosmos2 is allowlisted: its doc page is not derivable from the directory name, so the cutoff cannot grandfather it. The same number now has two sources of truth and the caller picks the winner, so editing the config no longer changes the model that gets built. It also pushes architecture knowledge out to every call site, where it does not belong.
class AcmeAttention(nn.Module):
- def __init__(self, config, embed_dim, num_heads, dropout):
+ def __init__(self, config, layer_idx=None):
super().__init__()
- self.embed_dim = embed_dim
- self.num_heads = num_heads
+ self.embed_dim = config.hidden_size
+ self.num_heads = config.num_attention_heads
class AcmeMLP(nn.Module):
# an optional override is fine: omitting it reads the config
def __init__(self, config, intermediate_size=None):
super().__init__()
self.intermediate_size = intermediate_size or config.intermediate_sizeTRF030
In modeling*.py and modular*.py, flags attribute chains rooted at config or self.config three or more levels deep. config.hidden_size (one hop) and config.text_config.hidden_size (two hops, the normal sub-config access) are fine. One violation per line. A module that walks config.diffusion_config.atom_encoder_config.hidden_size is coupled to the whole config hierarchy rather than its own slice, so it cannot be reused, tested or given a different sub-config. Pass the relevant sub-config down and the chain collapses to one hop.
class AcmeAtomEncoder(nn.Module):
def __init__(self, config):
super().__init__()
- self.norm = AcmeLayerNorm(config.diffusion_config.atom_encoder_config.hidden_size)
+ self.norm = AcmeLayerNorm(config.hidden_size)TRF031
In modeling*.py and modular*.py, flags a top-level @dataclass whose bases include nothing ending in Output, unless it has two or more mandatory fields — those are internal argument bundles, which ModelOutput rejects at runtime. A plain output dataclass does not index like a tuple, does not survive return_dict=False, and is invisible to @auto_docstring, so its fields never reach the generated API docs. ModelOutput gets all three for free.
@auto_docstring
@dataclass
-class AcmeStructureOutput:
+class AcmeStructureOutput(ModelOutput):
positions: torch.Tensor
confidence: Optional[torch.Tensor] = NoneTRF032
In modeling*.py and modular*.py, flags maskedfill, masked_fill, full, full_like and new_full called with a negated numeric literal of magnitude 1e3 or more. A hardcoded -1e9 overflows to -inf in float16 and is nowhere near the smallest value in float32, so the same mask behaves differently per dtype and can produce NaNs after softmax. torch.finfo(dtype).min is the smallest representable value in whatever dtype is actually running.
-attention_scores = attention_scores.masked_fill(~mask, -1e9)
+attention_scores = attention_scores.masked_fill(~mask, torch.finfo(attention_scores.dtype).min)TRF033
In modeling*.py and modular*.py, flags methods whose name starts with set_, except the PreTrainedModel contract methods set_input_embeddings, set_output_embeddings, set_decoder, set_encoder, set_attn_implementation and set_default_language. A setter makes behaviour depend on call order: the value is not in the config, so it is not saved, not restored by from_pretrained, and not visible to device-map or parallelism planning. Users have to know to call it, and forgetting is silent.
class AcmeTriangleAttention(nn.Module):
- def set_chunk_size(self, chunk_size):
- self.chunk_size = chunk_size
+ def __init__(self, config):
+ super().__init__()
+ self.chunk_size = config.chunk_sizeTRF034
In modeling*.py and modular.py, flags a locally-defined `Layer/Blockclass instantiated in annn.ModuleList(…)that does not reachGradientCheckpointingLayerthrough its base chain; modular files follow relative imports into sibling models, and unresolved chains are inconclusive. Out of scope: a model that never setssupports_gradient_checkpointing = True, which raises fromgradient_checkpointing_enable()instead of skipping a layer; a layer holdingnn.BatchNorm/nn.InstanceNorm*, whose statistics would be recomputed twice; and a stack that is not the model's token mixer, shown by an attention, modulation, mixer or SSM module assigned asself.x = Y(…).gradient_checkpointing_enable()` wraps a layer only if it is a GradientCheckpointingLayer. A plain nn.Module on the trunk is skipped silently, so training looks checkpointed while still allocating full activations, and the OOM surfaces far from the cause. Elsewhere — a conv backbone, a decode head — the trade is the author’s call, so the rule stays out.
-class AcmeDecoderLayer(nn.Module):
+class AcmeDecoderLayer(GradientCheckpointingLayer):
def __init__(self, config, layer_idx):
super().__init__()TRF035
Flags # noqa comments, with or without codes, in modeling*.py, modular.py and configuration_.py. In a modular file F401, F821 and F822 are accepted: it is a generation source that deliberately does not define every name it uses, so ruff’s undefined-name family fires on correct code — __all__ entries the converter fills in, classes living in the parent model, imports kept to be re-exported. A # noqa naming only those codes is skipped; one naming anything else is reported on the codes that are left. A bare # noqa is always reported; in a modular file the message asks for the code rather than a rewrite. Model files are ordinary code held to the repo’s lint rules, so a suppression means the underlying issue was left in place — and a bare # noqa also hides every future violation on that line. Fix the code instead.
-from ...modeling_utils import PreTrainedModel # noqa: F401
+from ...modeling_utils import PreTrainedModelTRF036
In modeling*.py and modular*.py, flags any nn.Sequential(...) construction. Sequential names its children by position, so weights land at mlp.0.weight and mlp.2.weight: the conversion mapping, _tied_weights_keys and every parallelism plan then reference indices, and inserting a layer renames everything after it. It also hides the forward, so the dtype casts and residuals between steps are invisible where they happen.
- self.mlp = nn.Sequential(
- nn.Linear(config.hidden_size, config.intermediate_size),
- nn.GELU(),
- nn.Linear(config.intermediate_size, config.hidden_size),
- )
+ self.fc1 = nn.Linear(config.hidden_size, config.intermediate_size)
+ self.act = ACT2FN[config.hidden_act]
+ self.fc2 = nn.Linear(config.intermediate_size, config.hidden_size)TRF037
In modeling*.py and modular*.py, flags calls to einsum, reporting the equation when it is a literal. Disabled by default: einsum is occasionally the clearest way to express a contraction, so this is opt-in rather than a hard convention. An einsum equation encodes the shapes in a notation the reader has to decode, and a dynamically built equation hides which contraction runs at all. Expand it into explicit matmul/transpose operations.
- pair_bias = torch.einsum("bqhc,bkhc->bhqk", query_states, key_states)
+ pair_bias = query_states.permute(0, 2, 1, 3) @ key_states.permute(0, 2, 3, 1)TRF038
Checks that every modeling*.py, processing.py, imageprocessing.py, videoprocessing.py, featureextraction.py and tokenization*.py file has a matching tests/models/<model>/test.py (modelingacme.py -> tests/models/acme/test_modeling_acme.py). Exempt: configuration.py, covered by ConfigTester in the companion testmodeling.py, and tokenization_utils.py, a helper covered through its tokenizer; tokenization<name>_fast.py maps to its slow counterpart’s test file. A modular*.py can define several families at once, so its classes are routed by name suffix — XxxModel/XxxPreTrainedModel/XxxFor<Task> modeling, XxxImageProcessor(Fast) image processing, XxxProcessor processing, XxxVideoProcessor video processing, XxxFeatureExtractor feature extraction, XxxTokenizer(Fast) tokenization, XxxConfig skipped — and one violation is reported per missing test file. # trf-ignore: TRF038 is not supported; use allowlist_models so an exemption is visible in review. A source file with no test file has no regression coverage: a broken forward pass, a bad conversion mapping, or a tokenizer that drops a special token can land and stay broken indefinitely. A dummy config with random weights, or a small hand-written vocabulary, is enough to exercise any of them.
src/transformers/models/acme/modeling_acme.py
+tests/models/acme/test_modeling_acme.py
src/transformers/models/acme/tokenization_acme.py
+tests/models/acme/test_tokenization_acme.pyTRF039
Finds if is_*_available(): import ... blocks (including combinations such as is_vision_available() and is_torch_available()) and flags the import when the name is referenced nowhere else in the file, including inside string type hints and all. ruff’s unused-import check does not clean these up: the import is reachable, so the block looks fine on its own. Once a refactor no longer needs PIL.Image, torch, etc., the guarded import lingers as dead weight and a misleading signal about the file’s real dependencies.
if is_vision_available():
- from PIL import ImageTRF040
In modeling*.py and modular*.py, flags methods decorated with both @capture_outputs and @can_return_tuple. Complements TRF003, which covers manual return_dict branching in forward(). Both decorators pop return_dict, so only the outermost one sees the true value. @capture_outputs already handles the to_tuple conversion, making @can_return_tuple redundant.
-@can_return_tuple
@merge_with_config_defaults
@capture_outputs
@auto_docstring
def forward(self, x):
return AcmeModelOutput(last_hidden_state=x)TRF041
In modeling*.py and modular.py, flags every if/elif and conditional expression whose condition reads a `config.orself.config.attribute without a# CODEPATH:comment, accepted on the branch line or in the contiguous comment block directly above it. Any config attribute counts, not just boolean flags. Exempt by structure:X if X is not None else fallback, where the tested field is itself one of the results --getattr(config, x, default)spelled long, which cannot fork the graph; and a guard, anifwith noelsewhose body only raises or only warns/logs, since one side aborts and nothing diverges past it. Merely mentioning None does not qualify:config.visionconfig is not Nonestill owes a note. Exempt by field: framework plumbing that gates no checkpoint divergence --problem_typeselecting a loss,hidden_actlooking up an activation,num_labels,use_cache,is_decoder, the special token ids, thesummaryhead settings; the full list isDEFAULT_EXEMPT_ATTRIBUTESinmlinter/trf041.py, extended per project byignored_attributes = […]on the rule table. A model exempts one of its own fields file-wide with a module-level# trf-ignore: TRF041 config.scale_embedding, config.auxiliary_lossat column 0 (self.config.x,config.xandxare the same field). It must name at least one field -- a bare# trf-ignore: TRF041keeps its per-line meaning -- and a condition is skipped only when every field it reads is exempt. A config-gated branch is a second architecture in the same file, and the code cannot say whether both halves are still reachable -- which is how dead experimental branches survive for releases. The rule does not forbid the branch: like Rust's// SAFETY:`, it asks for the checkpoints taking each side to be written down next to it. A branch nobody can name one for is a branch to delete.
+ # CODEPATH: ESMC-6B ships pre-normalised embeddings, the 300M/600M checkpoints do not.
if config.use_embedding_norm:
hidden_states = self.embedding_norm(hidden_states)
- if config.msa_encoder_enabled:
- hidden_states = self.msa_encoder(hidden_states)
+ # no released checkpoint sets msa_encoder_enabled -> branch removedTRF042
In tests/models//testtokenization.py, checks the file defines a collected test class inheriting TokenizerTesterMixin. Only classes the runner collects count — a TestCase base, or the *Test naming convention when the base is another model’s test class — so helper-only files are skipped, and a helper mixing in the suite does not satisfy the rule for a real test class. Inheritance is followed through bases in the same file and into another model’s tokenizer test imported by name; an unresolvable base never counts. Reported on the first test class that does not run the suite. auto is allowlisted: test_tokenization_auto.py tests AutoTokenizer resolution, not a tokenizer. TokenizerTesterMixin is where encode/decode round-tripping, padding and truncation, special-token handling, added-token persistence and save/load equivalence are actually checked. A test that only asserts a few hand-written token id lists passes while the tokenizer is broken in every one of them, and still looks tested in review.
-class AcmeTokenizationTest(unittest.TestCase):
+class AcmeTokenizationTest(TokenizerTesterMixin, unittest.TestCase):
+ tokenizer_class = AcmeTokenizer
+ test_slow_tokenizer = TrueTRF043
Flags a declared position_ids parameter in the forward signature of classes whose name ends in Attention. position_ids is consumed downstream by flash-attention padding-free training and must flow through kwargs. Naming it in the signature swallows it before the attention interface can read it; the llama standard passes position_embeddings pluskwargs.
class AcmeAttention(nn.Module):
def forward(
self,
hidden_states,
position_embeddings,
attention_mask=None,
- position_ids=None,
**kwargs: Unpack[TransformersKwargs],
):TRF044
Flags a parameter named cacheposition on any function in modeling.py and modular_.py. cache_position was removed from all models in v5. Reintroducing it (usually copied from pre-v5 sources) threads a dead argument through every layer; the cache update is past_key_values.update(key_states, value_states, self.layer_idx), with no position threading.
def forward(
self,
hidden_states,
past_key_values=None,
- cache_position=None,
**kwargs,
):
- key_states, value_states = past_key_values.update(key_states, value_states, self.layer_idx, cache_position)
+ key_states, value_states = past_key_values.update(key_states, value_states, self.layer_idx)TRF045
Flags the legacy outputattentions, output_hidden_states and return_dict parameters in forward signatures in modeling.py and modular_.py. Models contributed before the cutoff date are exempt. The decorator stack owns output control: @captureoutputs resolves the output* flags against the config and records tensors via _can_record_outputs, and @can_return_tuple handles return_dict. Declaring them in the signature reintroduces manual flag threading that drifts from the decorators.
+@capture_outputs
def forward(
self,
input_ids,
- output_attentions=None,
- output_hidden_states=None,
- return_dict=None,
**kwargs: Unpack[TransformersKwargs],
):TRF046
Flags assignments to self attributes in forward methods in modeling*.py and modular*.py. State written during forward breaks batching, torch.compile, and reasoning about the module. Carried state is passed explicitly (cache objects, the generate loop); values that depend only on config or static shapes belong in init.
def forward(self, hidden_states):
- self.sequence_length = hidden_states.shape[1]
- embeddings = self.compute_embeddings(self.sequence_length)
+ embeddings = self.compute_embeddings(hidden_states.shape[1])TRF047
Flags assignments to self attributes in preprocess, preprocess, call and post_process* methods in image_processing.py and videoprocessing.py. A processor carrying state between calls breaks preprocess-many-then-postprocess batching: the second preprocess overwrites the state the first postprocess needs. Return the value or pass it through the method chain.
def _preprocess(self, images, **kwargs):
- self.original_sizes = [image.shape[-2:] for image in images]
+ original_sizes = [image.shape[-2:] for image in images]
...
+ return BatchFeature(data={"pixel_values": pixel_values, "original_sizes": original_sizes})TRF048
Flags list/tuple/set literals in class-level _tied_weights_keys declarations. v5 changed _tied_weights_keys to a dict mapping each tied target parameter to its source. The list form no longer says which parameter is the source, so tying, device_map computation and saving misbehave silently.
class AcmeForCausalLM(AcmePreTrainedModel):
- _tied_weights_keys = ["lm_head.weight"]
+ _tied_weights_keys = {"lm_head.weight": "model.embed_tokens.weight"}TRF049
Flags init calls in init methods in modeling*.py and modular.py: nn.init. / init.* primitives and in-place initializers on own parameters (self.weight.data.normal_()). Models instantiate on the meta device, so tensor values written in init are discarded before loading: a parameter initialized only there holds random content when fine-tuning from scratch or after a meta-device reload. Allocate with torch.empty in init, initialize in _init_weights.
class AcmeEmbeddings(nn.Module):
def __init__(self, config):
super().__init__()
self.position_embedding = nn.Parameter(torch.empty(config.num_positions, config.hidden_size))
- nn.init.trunc_normal_(self.position_embedding, std=config.initializer_range)
class AcmePreTrainedModel(PreTrainedModel):
def _init_weights(self, module):
super()._init_weights(module)
+ if isinstance(module, AcmeEmbeddings):
+ init.trunc_normal_(module.position_embedding, std=self.config.initializer_range)TRF050
Flags calls to a *RotaryEmbedding class in the init of classes whose name ends in Attention. The Model owns a single rotary_emb, builds inv_freq once, and passes cos/sin down as position_embeddings. One rotary module per attention layer duplicates buffers, recomputes frequencies per layer, and diverges from the contract that attention receives position_embeddings.
class AcmeAttention(nn.Module):
def __init__(self, config, layer_idx):
super().__init__()
- self.rotary_emb = AcmeRotaryEmbedding(config)
class AcmeModel(AcmePreTrainedModel):
def __init__(self, config):
super().__init__(config)
+ self.rotary_emb = AcmeRotaryEmbedding(config)TRF051
Flags comparisons against a attn_implementation attribute in modeling.py and modular_.py. Backend dispatch belongs to ALL_ATTENTION_FUNCTIONS.get_interface, and backend-conditional tensor munging (padding, reshaping) belongs in the shared wrappers under integrations/. Inline branching keeps the model body kernel-aware and breaks when new backends register.
-if self.config._attn_implementation == "flash_attention_2":
- attn_output = flash_path(query_states, key_states, value_states)
-else:
- attn_output = eager_path(query_states, key_states, value_states)
+attention_interface = ALL_ATTENTION_FUNCTIONS.get_interface(self.config._attn_implementation, eager_attention_forward)
+attn_output, attn_weights = attention_interface(self, query_states, key_states, value_states, ...)TRF052
Flags module-level assignments to names ending in ATTENTION_CLASSES in modeling.py and modular_.py. Per-backend attention classes picked from a dict are the pre-interface idiom: near-identical classes drift apart, and hub attention kernels registered into ALL_ATTENTION_FUNCTIONS never reach them. One attention class dispatching through the interface replaces the dict; do not propagate it from a legacy parent.
-ACME_ATTENTION_CLASSES = {
- "eager": AcmeAttention,
- "flash_attention_2": AcmeFlashAttention2,
- "sdpa": AcmeSdpaAttention,
-}
+class AcmeAttention(nn.Module):
+ def forward(self, ...):
+ attention_interface = ALL_ATTENTION_FUNCTIONS.get_interface(self.config._attn_implementation, eager_attention_forward)TRF053
In modeling*.py and modular*.py, flags assignments that build shiftlogits/shift_labels (and shifted variants) by slicing, as in labels[…, 1:]. Receiving already-shifted labels (shift_labels = kwargs.pop(“shift_labels”, labels)) is the correct idiom and is not flagged. self.loss_function shifts labels itself, so pre-shifting in modeling code trains on doubly-shifted targets or forces a bespoke loss path. Decoder-only models pass raw labels and let the loss shift them. Encoder-decoder models are the mirror case: their labels are already shifted by the prepended decoder start token, so they pass shift_labels=labels to stop the loss shifting again.
if labels is not None:
- shift_logits = logits[..., :-1, :].contiguous()
- shift_labels = labels[..., 1:].contiguous()
- loss = nn.functional.cross_entropy(shift_logits.view(-1, self.config.vocab_size), shift_labels.view(-1))
+ # decoder-only: labels are unshifted, the loss shifts them
+ loss = self.loss_function(logits=logits, labels=labels, vocab_size=self.config.vocab_size)
+ # encoder-decoder: labels are already shifted, hand them over as shift_labels
+ loss = self.loss_function(logits=logits, labels=labels, shift_labels=labels, vocab_size=self.config.vocab_size)TRF055
Flags config = SomeConfig as a class attribute on PreTrainedModel subclasses in modeling*.py and modular*.py. PreTrainedModel.__init_subclass__ derives config_class from a config annotation, via inspect.get_annotations(cls). An assignment creates a stray class attribute that call does not report, so the subclass silently keeps the parent’s config_class. A bare annotation has no runtime value, creates no attribute, and is picked up correctly.
class Gemma4VisionModel(Gemma4PreTrainedModel):
"""The Gemma 4 Vision Encoder."""
- config = Gemma4VisionConfig
+ config: Gemma4VisionConfigTRF056
In modeling*.py and modular*.py, flags .item() and .tolist() calls inside any forward. A .tolist() feeding the split-size argument of split(...) is exempt: torch.split needs Python ints. Both calls read a tensor back to the host, which dynamo cannot trace, so the graph breaks.
- for grid, item in zip(grid_thw.tolist(), split_items):
- _, height, width = grid
- merged.append(self.patch_merger(item, size=(height, width)))
+ for grid, item in zip(grid_thw, split_items):
+ merged.append(self.patch_merger(item, size=(grid[1], grid[2])))TRF057
Checks @auto_docstring on public PreTrainedModel subclasses (<Model>PreTrainedModel, <Model>Model, <Model>For<Task>, backbones), PreTrainedConfig subclasses, ModelOutput subclasses, image processors and ProcessorMixin subclasses, and on their public methods forward, get_image_features, get_video_features, get_audio_features, get_text_features, preprocess and __call__. A modular_*.py is checked against the files generated from it. Without it a class ships with no intro and no parameter documentation, and a method with no argument documentation, no Returns section and no usage example — all of which then have to be hand-written per model instead of coming from auto_docstring.py.
+@auto_docstring
@dataclass
class AcmeModelOutputWithPast(ModelOutput):
logits: torch.FloatTensor | None = None
+@auto_docstring
class AcmeForConditionalGeneration(AcmePreTrainedModel):
+ @auto_docstring
def forward(self, input_ids, pixel_values=None, **kwargs):
...TRF058
In modeling*.py and modular*.py, flags register_buffer("<name>", ...) calls whose buffer name is a string literal, on any receiver (self, or another module such as layer.mamba). A computed name — a variable or f-string, e.g. one buffer per layer inside a loop — has no attribute-assignment equivalent and is exempt. Since torch>=2.5 nn.Buffer registers a buffer through plain attribute assignment, like nn.Parameter. A buffer created by a method call only exists as a side effect of running __init__, so a modular file that wants to tweak one has to redefine the whole __init__. Assigned as an attribute, it can be inherited and overridden on its own.
- self.register_buffer("inv_freq", inv_freq, persistent=False)
- self.register_buffer(
- "position_ids", torch.arange(config.max_position_embeddings).expand((1, -1)), persistent=False
- )
+ self.inv_freq = nn.Buffer(inv_freq, persistent=False)
+ self.position_ids = nn.Buffer(torch.arange(config.max_position_embeddings).expand((1, -1)), persistent=False)TRF059
For model directories whose tensor-parallel plan assigns moe_tp_experts, checks that routed *Experts classes take hidden states, top-k indices and top-k routing weights as the first three positional forward arguments. Aliases such as selected_experts and routing_weights are accepted. MoeExpertsParallel applies a gradient transform to positional argument 3. A different signature silently applies it to the wrong tensor or skips it.
class AcmeExperts(nn.Module):
- def forward(self, hidden_states):
+ def forward(self, hidden_states, top_k_index, top_k_weights):
...Suppressing violations
If you need to suppress a rule violation, use one of the two options below.
Inline suppression
Add a # trf-ignore: RULE_ID comment on the violating line. Include an explanation so reviewers understand why the suppression is justified.
# trf-ignore: TRF011 — mask is derived from self.config, not the layer
hidden_states = layer(hidden_states, attention_mask=mask_from_config)Don’t use trf-ignore to silence violations that should be fixed in the code.
allowlist_models
For models with legacy code that can’t be fixed immediately, add the model’s directory name to the relevant rule’s allowlist_models list in the mlinter rules.toml.
[rules.TRF004]
allowlist_models = ["existing_model", "your_model_name"]