Instructions to use LLMWildling/gemma4-52b-a6b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use LLMWildling/gemma4-52b-a6b with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="LLMWildling/gemma4-52b-a6b")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForCausalLM

processor = AutoProcessor.from_pretrained("LLMWildling/gemma4-52b-a6b")
model = AutoModelForCausalLM.from_pretrained("LLMWildling/gemma4-52b-a6b")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use LLMWildling/gemma4-52b-a6b with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "LLMWildling/gemma4-52b-a6b"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "LLMWildling/gemma4-52b-a6b",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/LLMWildling/gemma4-52b-a6b

SGLang

How to use LLMWildling/gemma4-52b-a6b with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "LLMWildling/gemma4-52b-a6b" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "LLMWildling/gemma4-52b-a6b",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "LLMWildling/gemma4-52b-a6b" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "LLMWildling/gemma4-52b-a6b",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use LLMWildling/gemma4-52b-a6b with Docker Model Runner:
```
docker model run hf.co/LLMWildling/gemma4-52b-a6b
```

Gemma4-52B-A6B

gemma4-52b-a6b is a production MoE expansion of unsloth/Gemma-4-26B-A4B-it. It was built with a two-GPU local training setup and expanded with additional specialist capacity for software engineering, repository reasoning, agentic workflows, and general instruction following.

This is a full model checkpoint, not a LoRA adapter.

Model Details

Base model: unsloth/Gemma-4-26B-A4B-it
Architecture: Gemma4 MoE
Total logical parameters: approximately 50.6B
Active parameters: approximately 6B
Expert layout: 128 base experts + 128 added specialist experts
Native context length: 256k tokens
Validated local serve window: 131k tokens
Primary focus: SWE, code/repository analysis, agentic workflows, and reasoning
Recommended temperature: 0.5 to 0.7

Serving

Use the included Gemma4 chat template. Thinking mode is intended to be enabled.

CUDA_VISIBLE_DEVICES=0 \
vllm serve . \
  --served-model-name gemma4-52b-a6b \
  --host 0.0.0.0 \
  --port 23333 \
  --dtype bfloat16 \
  --tensor-parallel-size 1 \
  --max-model-len 131072 \
  --gpu-memory-utilization 0.88 \
  --trust-remote-code \
  --reasoning-parser gemma4 \
  --tool-call-parser gemma4 \
  --enable-auto-tool-choice \
  --chat-template ./chat_template.jinja \
  --default-chat-template-kwargs '{"enable_thinking": true}'

OpenAI-compatible endpoint:

http://localhost:23333/v1

Notes

This release is intended for production evaluation of the expanded Gemma4 MoE line. Feedback is welcome through the model discussion page or the usual Hugging Face repository feedback channels.

Downloads last month: 84

Safetensors

Model size

28B params

Tensor type

F32

BF16

F8_E4M3