mando2222/vibethinker-3b-P150

VibeThinker-3B — Tenstorrent Blackhole (vLLM bundle)

A tt-kernel v4 vLLM bundle that runs WeiboAI/VibeThinker-3B — a Qwen2.5-3B-family dense reasoning model (Qwen2ForCausalLM) — on a single Tenstorrent Blackhole p150a through the Tenstorrent vLLM plugin. It ships no kernel cache and no weights: kernels JIT-compile at first-run warmup and weights are pulled from the base repo. This is the generic tt_transformers path — the registration references the upstream built-in QwenForCausalLM, so no custom adapter is shipped.

Serve it with tt (recommended)

tt serve --backend kernel mando2222/vibethinker-3b-P150

That's the whole bringup. The tt CLI provisions the entire bare-metal serving stack for you — there is no tt-metal to build by hand. From the bundle's manifest it:

1. resolves the declared platform.ttnn range (>=0.72,<0.73) to a released ttnn wheel and installs it, plus the matching sfpi JIT toolchain (so kernel compilation uses the toolchain that ttnn version expects, not whatever is on the host);
2. installs the Tenstorrent vLLM fork + plugin (CUDA-free) and tt-kernel;
3. composes all of the above into a reusable serving environment under $TT_DATA_DIR/envs, keyed by version so multiple models share one install rather than duplicating it;
4. hands off to tt-kernel, which selects that environment and launches the server.

First run JIT-compiles kernels and downloads the weights from WeiboAI/VibeThinker-3B; later runs reuse both. The result is an OpenAI-compatible server on :8000 (model id WeiboAI/VibeThinker-3B, aliases vibethinker-3b, vibethinker-3b-P150).

Manage the composed environments with tt env list / tt env gc.

Requirements

• A Tenstorrent Blackhole card (this bundle targets a single p150a) with the driver loaded.
• Linux x86_64 (ttnn/sfpi ship for that platform only).
• The tt CLI. No hand-built tt-metal, no Docker, and no scripts/install.sh step — tt owns provisioning.

Using the model

Native 131072 context; up to 32 concurrent requests (max_num_seqs=32). The model emits long <think>…</think> chain-of-thought — use generous max_tokens and its recommended sampling (temperature 0.6 / top_p 0.95 / top_k 40). bf8 weights + bf8 KV, traced decode, host sampling.

curl http://localhost:8000/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{"model":"vibethinker-3b","messages":[{"role":"user","content":"Prove that sqrt(2) is irrational."}],"max_tokens":4096,"temperature":0.6,"top_p":0.95}'

What's in this bundle

A single tt_kernel_manifest.json (tt-kernel schema v4, unified model+manifest). It declares the platform envelope (platform.ttnn), the serving runtime (runtime.kind = vllm), the plugin entrypoint (Qwen2ForCausalLM → models.tt_transformers.tt.generator_vllm:QwenForCausalLM), the launch recipe, and the weights reference — from which tt-kernel renders the plugin's vllm_metadata.json on pull. No precompiled kernels, no weights.

Advanced: serve without tt

You can still serve directly with tt-kernel if you manage the tt-metal + vLLM stack yourself (see the tt-kernel docs and tt-kernel instances). The tt path above automates exactly that setup.