diegoquinteiro/SmolLM2-135M-Observable

SmolLM2-135M Observable

This is a weight-only Q4 ONNX export of HuggingFaceTB/SmolLM2-135M for an interactive course laboratory. It exposes the intermediate tensors needed to inspect one forward pass in a browser.

The model is a base next-token predictor. It is not an instruction-following or chat model.

Inputs

The sequence inputs use int64 tensors with shape [batch, sequence]:

• input_ids
• attention_mask

query_index is an int64 tensor with shape [batch]. It selects the token position whose layer update will be decomposed.

The browser laboratory uses a batch size of one and short sequences.

Outputs

The graph returns 152 tensors:

• next_token_logits, with shape [batch, 49152];
• hidden_state_00 through hidden_state_30, with shape [batch, sequence, 576];
• attention_01 through attention_30, with shape [batch, 9, sequence];
• attention_contribution_01 through attention_contribution_30, with shape [batch, sequence, 576];
• attention_output_01 through attention_output_30, with shape [batch, 576];
• mlp_output_01 through mlp_output_30, with shape [batch, 576].

hidden_state_00 contains the input embeddings. The following hidden states contain the raw residual stream after each transformer layer, before the final RMSNorm.

For a query token i and source token j, each contribution is:

cᵢ⟵ⱼ = Wₒ concatₕ(aᵢⱼʰ vⱼʰ)

The source contributions sum to attention_output. Adding that result to the previous residual state gives the post-attention state. Adding mlp_output gives the next residual state. The manifest records numerical checks for both identities.

Quantization

The published graph uses 4-bit weight-only quantization with asymmetric 32-value blocks. Activations and outputs remain float32. The graph uses the ONNX Runtime MatMulNBits operator.

The export script compares the ONNX result with the original PyTorch model. The included manifest records the validation prompt, tensor shapes and numerical comparison for the published file.

Intended use

This artifact supports a browser laboratory that visualizes:

• next-token probabilities;
• token IDs and token boundaries;
• attention weights by layer and head;
• additive attention contributions for one selected token;
• the MLP update for that token;
• a two-dimensional projection of residual states across layers.

The two-dimensional coordinates are calculated by the browser. They are not stored in this model.

Limitations

• The Q4 graph does not reproduce the float32 model exactly.
• The graph does not use a KV cache and recomputes the whole sequence when the selected token changes.
• Small base Models produce weak and sometimes incoherent continuations.
• Attention weights show values calculated inside the model. They do not establish a causal explanation for an output.
• This export is meant for teaching and inspection, not production inference.

Source

The export script and browser laboratory were created for the Engenharia Assistida por IA course. Hugging Face published the original SmolLM2 weights and architecture under the Apache 2.0 license.