crane-30b

crane-30b is a CRANE merge: it is produced by merging two Qwen3-30B-A3B checkpoints (an Instruct base and a Thinking donor) with the CRANE method — it is not trained or fine-tuned from scratch. CRANE (Constrained Reasoning Injection for Code Agents via Nullspace Editing) injects reasoning ability from the Thinking donor into the tool-disciplined Instruct / code base while preserving the base model's output format and tool-calling behavior.

Project page: https://rpi-nsl.github.io/CRANE/ · Code: github.com/rpi-nsl/CRANE

Note: this is the CRANE weight-merging method for code agents. It is unrelated to the similarly-named "CRANE: Reasoning with constrained LLM generation" (arXiv 2502.09061), despite the shared acronym.

How it was made (CRANE)

CRANE is a training-free, parameter-editing weight merge that injects reasoning ability from a "Thinking" donor into a tool-disciplined Instruct / code base, while constraining the edit so the base model's output format and tool-calling behavior are preserved. It treats the Thinking − Instruct delta $\delta ={\theta }_{\text{think}}-{\theta }_{\text{inst}}\backslash delta\; =\; \backslash theta\_\{\backslash text\{think\}\}\; -\; \backslash theta\_\{\backslash text\{inst\}\}$ as a pool of candidate reasoning edits, and applies three composable stages per layer $ll$ and parameter component $cc$:

$${\theta }_{\text{merged}}^{(l,c)}={\theta }_{\text{inst}}^{(l,c)}+\underset{\text{Stage 3}}{\underbrace{{\mathrm{\Pi }}_{\tau ,q(l,c)}^{\text{GSP}}}}\text{\hspace{0.17em}⁣}(\alpha \cdot \underset{\text{Stage 2}}{\underbrace{S_{\text{CTG}}(c,l)}}\cdot \underset{\text{Stage 1}}{\underbrace{T\left({\delta }^{(l,c)}\right)}})\backslash theta\_\{\backslash text\{merged\}\}^\{(l,c)\}\; =\; \backslash theta\_\{\backslash text\{inst\}\}^\{(l,c)\}\; +\; \backslash underbrace\{\backslash Pi\_\{\backslash tau,\backslash ,q(l,c)\}^\{\backslash text\{GSP\}\}\}\_\{\backslash text\{Stage\; 3\}\}\backslash !\backslash Big(\; \backslash alpha\; \backslash cdot\; \backslash underbrace\{S\_\{\backslash text\{CTG\}\}(c,l)\}\_\{\backslash text\{Stage\; 2\}\}\; \backslash cdot\; \backslash underbrace\{T\backslash big(\backslash delta^\{(l,c)\}\backslash big)\}\_\{\backslash text\{Stage\; 1\}\}\; \backslash Big)$$

Three small calibration sets drive the stages — ${\mathcal{D}}_R\backslash mathcal\{D\}\_R$ (reasoning transfer), ${\mathcal{D}}_A\backslash mathcal\{D\}\_A$ (agent-behavior / tool-use preservation), and ${\mathcal{D}}_F\backslash mathcal\{D\}\_F$ (format preservation):

• Stage 1 — Magnitude thresholding $T(\delta )T(\backslash delta)$. A deterministic median-magnitude threshold keeps only the larger (top-half) delta coordinates and rescales them by 2, discarding low-confidence noise.
• Stage 2 — Conservative Taylor Gate $S_{\text{CTG}}S\_\{\backslash text\{CTG\}\}$. From a signed, direction-aware score $s_K(j)=-g_{K,j}{\delta }_{j}s\_K(j)\; =\; -g\_\{K,j\}\backslash ,\backslash delta\_j$ per calibration loss, CTG keeps the positive part of the per-coordinate minimum over the reasoning and agent-behavior objectives, $p_j=[\min \{s_R(j),s_A(j)\}{]}_{+}p\_j\; =\; [\backslash min\backslash \{s\_R(j),\; s\_A(j)\backslash \}]\_+$ — rewarding a coordinate only when the edit helps both. These aggregate into the per-component, per-layer coefficient $S_{\text{CTG}}(c,l)S\_\{\backslash text\{CTG\}\}(c,l)$, scaled by the single global merge strength $\alpha \backslash alpha$.
• Stage 3 — Graduated Sigmoidal Projection (GSP). From the SVD of format-critical Instruct activations $H_q=U_q{\mathrm{\Sigma }}_q{V}_q^{\mathrm{\top }}H\_q\; =\; U\_q\backslash Sigma\_q\; V\_q^\{\backslash top\}$, a smooth sigmoidal weight ${\mathbf{w}}_q\backslash mathbf\{w\}\_q$ (set by singular amplitude and threshold $\tau \backslash tau$) gives the projector ${\mathrm{\Pi }}_{\tau ,q}^{\text{GSP}}({\mathrm{\Delta }}_q)={\mathrm{\Delta }}_q-{\mathrm{\Delta }}_q{V}_q\mathrm{diag}({\mathbf{w}}_q)V_q^{\mathrm{\top }}\backslash Pi\_\{\backslash tau,q\}^\{\backslash text\{GSP\}\}(\backslash Delta\_q)\; =\; \backslash Delta\_q\; -\; \backslash Delta\_q\; V\_q\; \backslash operatorname\{diag\}(\backslash mathbf\{w\}\_q)\; V\_q^\{\backslash top\}$, attenuating high-amplitude format directions so reasoning is injected without perturbing chat-template tokens, tool-call delimiters, or JSON/schema structure.

The result is a merge that gains planning / reflection / recovery reasoning while keeping the base agent's compact, tool-call-disciplined behavior — the entire merge is a closed-form edit of the Instruct weights, with no fine-tuning.

This checkpoint's recipe

This checkpoint merges Qwen/Qwen3-30B-A3B-Instruct-2507 (base) and Qwen/Qwen3-30B-A3B-Thinking-2507 (donor) with:

• Global injection strength — $\alpha =0.25\backslash alpha\; =\; 0.25$, multiplied by the per-component CTG coefficients, so the Thinking delta is added at roughly a quarter strength.
• Per-layer / per-component gating — attention, expert (FFN), norm, and router components each get their own $S_{\text{CTG}}(c,l)S\_\{\backslash text\{CTG\}\}(c,l)$ coefficient, varying by layer index rather than a single flat scalar.
• GSP projector — a freshly rebuilt Qwen3-30B graduated-sigmoidal projector (sigmoid threshold $\tau =0.03\backslash tau\; =\; 0.03$) protects the format / tool-call subspace before injection.

Architecture

The merge preserves the standard Qwen3-30B-A3B (MoE) topology unchanged:

Property	Value
model_type	qwen3_moe
Architecture class	Qwen3MoeForCausalLM
Total params	~30B
Active params	~3B
hidden_size	2048
num_hidden_layers	48
num_experts	128
num_experts_per_tok	8
num_attention_heads	32
num_key_value_heads	4
head_dim	128
moe_intermediate_size	768
max_position_embeddings	262144
vocab_size	151936
dtype	bfloat16
rope_theta	10000000

A config_1m.json is also included for the extended long-context variant: it keeps the same rope_scaling (null) and max_position_embeddings (262144), but adds a dual_chunk_attention_config (dual chunk attention, original_max_position_embeddings = 131072 + sparse-attention settings) for longer-context inference.

Usage

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "zmzfpc/crane-30b"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
)

messages = [
    {"role": "user", "content": "Write a Python function that returns the nth Fibonacci number."},
]

inputs = tokenizer.apply_chat_template(
    messages,
    add_generation_prompt=True,
    return_tensors="pt",
).to(model.device)

outputs = model.generate(inputs, max_new_tokens=512)
print(tokenizer.decode(outputs[0][inputs.shape[-1]:], skip_special_tokens=True))

Requires a recent transformers with Qwen3-MoE support (transformers >= 4.51).

Citation / attribution

If you use this model or the CRANE method, please cite:

@misc{zhu2026crane,
  title        = {CRANE: Constrained Reasoning Injection for Code Agents via Nullspace Editing},
  author       = {Zhu, Mingzhi and Merler, Michele and Pavuluri, Raju and Patterson, Stacy},
  year         = {2026},
  eprint       = {2605.14084},
  archivePrefix= {arXiv},
  primaryClass = {cs.SE},
  url          = {https://arxiv.org/abs/2605.14084}
}

Project page: https://rpi-nsl.github.io/CRANE/ · Code: github.com/rpi-nsl/CRANE

Base models — built from two Apache-2.0 checkpoints:

• Qwen/Qwen3-30B-A3B-Instruct-2507 (base / backbone)
• Qwen/Qwen3-30B-A3B-Thinking-2507 (reasoning donor)

License: Apache-2.0 (consistent with both base models and the CRANE code).