Instructions to use PrimeIntellect/DeepSeek-V4.1-Flash-bf16 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use PrimeIntellect/DeepSeek-V4.1-Flash-bf16 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="PrimeIntellect/DeepSeek-V4.1-Flash-bf16")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("PrimeIntellect/DeepSeek-V4.1-Flash-bf16", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use PrimeIntellect/DeepSeek-V4.1-Flash-bf16 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "PrimeIntellect/DeepSeek-V4.1-Flash-bf16"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "PrimeIntellect/DeepSeek-V4.1-Flash-bf16",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/PrimeIntellect/DeepSeek-V4.1-Flash-bf16

SGLang

How to use PrimeIntellect/DeepSeek-V4.1-Flash-bf16 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "PrimeIntellect/DeepSeek-V4.1-Flash-bf16" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "PrimeIntellect/DeepSeek-V4.1-Flash-bf16",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "PrimeIntellect/DeepSeek-V4.1-Flash-bf16" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "PrimeIntellect/DeepSeek-V4.1-Flash-bf16",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use PrimeIntellect/DeepSeek-V4.1-Flash-bf16 with Docker Model Runner:
```
docker model run hf.co/PrimeIntellect/DeepSeek-V4.1-Flash-bf16
```

DeepSeek-V4.1-Flash-bf16

A bfloat16 re-export of deepseek-ai/DeepSeek-V4.1-Flash under the published tensor names. The dense linears (fp8 e4m3, ue8m0 scales over 32x32 blocks) and the routed experts (MXFP4, ue8m0 scales over 1x32 groups) are dequantized to plain bfloat16. Every fp8 / fp4 value times its power-of-two scale is representable in bfloat16, so the dequantization is exact. 48 safetensors shards, 1.34 TB.

Differences from upstream

quantization_config removed from config.json. An engine that reads it allocates fp8 and MXFP4 parameters, which bfloat16 weights cannot fill.
The two engram tables (layers.1.engram.embed and layers.14.engram.embed, ~98B parameters each) are not dequantized: they keep the upstream fp8 e4m3 weight and 1x32 ue8m0 .scale, because vLLM's engram is fp8-only. In bfloat16 they would add ~197 GB.
Everything else is copied unchanged, including the vision tower, the MTP / DSpark weights, the tokenizer, encoding/ and inference/.
Not everything is bfloat16: mHC, attention-sink and bias tensors stay fp32. The export holds 48,107 bf16, 387 fp32, 2 fp8 e4m3 and 2 e8m0 tensors.

Use

vLLM: serve it without quantization_config; pass quantization = "fp8" (online per-block fp8) to quantize the bf16 weights at load.
prime-rl: use it as model.name. The trainer dequantizes the engram tables and renames to its own layout on first load.

License

MIT, inherited from upstream. The upstream LICENSE is included verbatim.

Downloads last month: 190

Safetensors

Model size

763B params

Tensor type

BF16

F32

F8_E4M3

Model tree for PrimeIntellect/DeepSeek-V4.1-Flash-bf16

Base model

deepseek-ai/DeepSeek-V4.1-Flash

Finetuned

(25)

this model