Instructions to use armanibadboy/WAL-Ternary-27B-RC1 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use armanibadboy/WAL-Ternary-27B-RC1 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="armanibadboy/WAL-Ternary-27B-RC1")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("armanibadboy/WAL-Ternary-27B-RC1")
model = AutoModelForCausalLM.from_pretrained("armanibadboy/WAL-Ternary-27B-RC1", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use armanibadboy/WAL-Ternary-27B-RC1 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "armanibadboy/WAL-Ternary-27B-RC1"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "armanibadboy/WAL-Ternary-27B-RC1",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/armanibadboy/WAL-Ternary-27B-RC1

SGLang

How to use armanibadboy/WAL-Ternary-27B-RC1 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "armanibadboy/WAL-Ternary-27B-RC1" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "armanibadboy/WAL-Ternary-27B-RC1",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "armanibadboy/WAL-Ternary-27B-RC1" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "armanibadboy/WAL-Ternary-27B-RC1",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use armanibadboy/WAL-Ternary-27B-RC1 with Docker Model Runner:
```
docker model run hf.co/armanibadboy/WAL-Ternary-27B-RC1
```

WAL-Ternary-27B-RC1

The model is the proof. The pipeline is the product.

WAL-Ternary-27B-RC1 is an independently constructed, language-only ultra-low-bit checkpoint derived from Qwen3.6-27B. It is the second full-scale demonstration of the WAL approach after WAL-Ternary-8B.

We are not claiming the best 27B ternary model. We are publishing an open, auditable path for constructing, physically packing, directly executing, and measuring one without large-scale quantization-aware retraining.

Русский · Қазақша · Model files on Hugging Face

New here? Follow Start here. To rebuild or port the method, use the step-by-step reproduction guide.

Release status

Item	Status
Physical autonomous checkpoint	PASS
496/496 Transformer-body matrices packed	PASS
Reference reload	PASS
Direct-packed full-model execution	PASS
Persistent dense BF16/INT8 body	0 bytes
Canonical quality backend	`reference-exact`
Experimental performance backend	`cuda-fast`
Production maturity	Not yet
CPU / Apple Silicon backend	Not yet

RC1 is a research release. It generates coherent text and preserves substantial functionality, but its knowledge quality and runtime speed remain below the trained low-bit frontier.

Physical model

Property	Value
Language parameters	26,895,998,464
Canonical checkpoint bytes	8,319,902,374
Full-model BPW	2.474688533
Compression relative to BF16	about 6.47×
Body matrices	496
WALB2 bundles	152
Configured context	262,144 tokens
Runtime validation completed through	4,096 tokens
Manifest SHA-256	`383c0603bce54a2072db0bc329c4e5b5dab512cb1d5672f255348bedfbe7b077`

The reported BPW covers the mandatory physical checkpoint payload. Repository documentation and the derived hardware cache are not counted in model BPW.

Representation

Each selected body matrix is approximated as:

W ≈ T3 + sparse + WALB2

T3: strict {-1, 0, +1} base, stored as seekable losslessly compressed 2-bit symbols.
Sparse lane: role- and depth-dependent k0/k2/k4/k8 corrections per group of 128 weights.
WALB2: two packed binary low-rank residual paths on 152 causally selected matrices.
Embedding: 75% INT3 rows and 25% causally selected INT4 rows.
LM head: packed INT5.
Small tensors: BF16.
Body scales: a documented 24-bit float storage format decoded to FP32 in the derived compute cache. This does not make the model “FP24”; its large weight bodies remain ternary hybrid.

See docs/FORMAT.md for the storage ABI.

Measured quality

All public RC1 baseline values below were measured through the direct-packed reference-exact runtime, not a persistent materialized BF16 body.

Benchmark	RC1
C4 perplexity	15.6639
WikiText-2 perplexity	10.9453
MMLU-Redux micro	60.98%
MMLU-Redux macro	61.14%
MMLU-Redux questions	2,722

These are baseline numbers, not a claim of parity with Qwen BF16 or Bonsai. The full generative and same-harness competitor battery is incomplete.

Measured runtime

Current research backend, batch 1 on an NVIDIA H200:

Metric	Result
Decode	3.22 tok/s
Short prefill	19.48 tok/s
Short-run peak VRAM	10.97 GB
4K-context peak VRAM	11.65 GB
Persistent packed model payload in VRAM	about 10.54 GB
Persistent dense body	0 bytes

These numbers establish direct execution and memory behavior. They are not a production-speed claim. The released backend is validated for NVIDIA Hopper SM90; other GPUs may compile but are not certified.

Install

The cleanest environment is Python 3.11 with a recent CUDA-enabled PyTorch.

git clone https://github.com/AubakirovArman/WAL-Ternary-27B.git
cd WAL-Ternary-27B
python -m venv .venv
source .venv/bin/activate
pip install --upgrade pip
pip install -e .
wal27 doctor

PyTorch wheels are platform-specific. If pip install -e . selects the wrong wheel, install the correct CUDA build of PyTorch first, then repeat the command.

Download and verify

The CLI downloads the Hugging Face repository automatically:

wal27 inspect armanibadboy/WAL-Ternary-27B-RC1
wal27 inspect armanibadboy/WAL-Ternary-27B-RC1 --verify

--verify reads and hashes the complete 8.32 GB payload.

Prepare the hardware cache

The canonical archive is compact and portable. Before inference, build a parent-attested SM90 compute cache:

wal27 prepare armanibadboy/WAL-Ternary-27B-RC1 \
  --cache /data/wal27-rc1-sm90-cache

The derived cache is approximately 10.54 GB. It keeps ternary and binary codes packed, expands 24-bit scales to FP32 and sparse positions to the layout used by the current kernels, and is excluded from canonical BPW. A cache whose parent manifest does not match RC1 is rejected.

Generate

Correctness-oriented direct-packed mode:

wal27 generate armanibadboy/WAL-Ternary-27B-RC1 \
  --cache /data/wal27-rc1-sm90-cache \
  --mode exact \
  --prompt "Explain ternary neural networks in simple terms." \
  --max-new-tokens 128

Experimental faster arithmetic:

wal27 generate armanibadboy/WAL-Ternary-27B-RC1 \
  --cache /data/wal27-rc1-sm90-cache \
  --mode fast \
  --prompt "Қазақстандағы жасанды интеллект туралы қысқаша жаз." \
  --max-new-tokens 128

exact is the canonical quality backend. fast passed strong diagnostic teacher-forced parity but has not completed the final preregistered generative acceptance protocol.

Repository map

src/wal27/                 public codec, cache builder and CUDA runtime
docs/                      concise format/runtime/parity reports
research/designs/          complete 27B design and result journal
research/experiments/      archival experiment implementations
research/lib/              shared public T3/WALB2 research code
reproduction/              machine-readable RC1 phase map
results/                   immutable summary receipts

The archival research scripts intentionally preserve historical experiment names and some machine-local default paths. The trilingual reproduction guides explain how to adapt them and which gates must remain frozen. They document the exact campaign; they are not the supported user CLI. Use wal27 for the released model.

What the campaign established

Full-body ternary hybrid conversion scales from 8B to 27B.
Uniform sparse density is inferior to causal role/depth allocation.
WALB2 helps when it is fitted to the residual of the chosen sparse base and allocated selectively.
Endpoint precision must be calibrated against the low-bit body.
Small aggregate calibration fields can improve average language and knowledge metrics while still breaking individual confident decisions.
Linearized safety does not guarantee safety after low-precision folding.
A physical low-bit checkpoint is not enough: direct-packed runtime parity is part of the model evidence.

Negative results are retained in the research ledger instead of being erased.

Comparison policy

Bonsai is an important trained low-bit reference and currently represents a stronger endpoint in quality, size, and runtime maturity. No Bonsai tensors were used. WAL explores a different regime: rapidly converting an existing BF16 checkpoint with limited calibration/fitting and exposing the entire process.

Do not compare vendor-reported and WAL scores as if they used one harness. A same-harness BF16/IQ2/Binary-Bonsai/Ternary-Bonsai comparison remains open.

Limitations

Research release; API and cache ABI may change.
H200/SM90 is the only validated GPU target.
No 27B CPU or Apple Silicon backend yet.
Decode and prefill are not production optimized.
Long-context numerical behavior is validated through 4K, not the configured 262K maximum.
MMLU remains materially below the desired trained low-bit frontier.
cuda-fast is not yet the canonical quality evaluator.
This is the language checkpoint only; vision and MTP training head are absent.

Citation

@software{aubakirov2026wal27,
  author = {Arman Aubakirov},
  title = {WAL-Ternary-27B: an open ultra-low-bit conversion and runtime laboratory},
  year = {2026},
  url = {https://github.com/AubakirovArman/WAL-Ternary-27B}
}

License

Apache License 2.0. See LICENSE and NOTICE. Users remain responsible for the source-model license, dataset licenses, and applicable law.

Downloads last month: -

Model tree for armanibadboy/WAL-Ternary-27B-RC1

Base model

Qwen/Qwen3.6-27B

Finetuned

(341)

this model