Quatfit Mini
Gemma 4–Based 8B Multimodal Model · Up to 4× Faster Inference · 131K Context
Base Model
Quatfit Mini is derived from Google Gemma 4 and preserves the original multimodal transformer architecture.
Quatfit AI Research extends the base model through:
- Supervised instruction tuning
- Alignment optimization
- Deployment optimization
- GGUF optimization
- Speculative decoding support
- Optimized inference pipeline
- Consumer hardware optimization
Model Summary
| Property | Value |
|---|---|
| Parameters | 8B |
| Base Architecture | Google Gemma 4 |
| Model Type | Decoder-only Multimodal Transformer |
| Context Length | 131,072 tokens |
| Precision | BF16 |
| Vocabulary | 262K |
| Languages | English, Hindi, Multilingual |
| Modalities | Text, Image, Audio |
Intended Uses
Primary Use Cases
- Agentic AI
- Coding Assistant
- Visual Question Answering
- OCR
- Diagram Understanding
- Audio Understanding
- Long-context reasoning
- Research Copilot
- Productivity Automation
- Tool Calling
- API Development
Out-of-Scope
- Medical diagnosis
- Legal advice
- High-risk decision making
- Enterprise-scale software engineering
- Repository-scale code generation
- Competitive programming
Installation
pip install git+https://github.com/Jatinverma0786/transformers.git
After the upstream Hugging Face Transformers integration is merged:
pip install "transformers[torch]"
Loading the Model
import torch
from transformers import (
AutoProcessor,
AutoModelForImageTextToText,
)
model = AutoModelForImageTextToText.from_pretrained(
"Quatfit/Quatfit-Mini",
torch_dtype="auto",
device_map="auto"
)
processor = AutoProcessor.from_pretrained(
"Quatfit/Quatfit-Mini"
)
Text Generation
messages = [
{
"role": "user",
"content": "Write a Python implementation of binary search."
}
]
inputs = processor.apply_chat_template(
messages,
tokenize=True,
return_tensors="pt"
)
outputs = model.generate(
**inputs,
max_new_tokens=512
)
print(processor.decode(outputs[0]))
Image Understanding
messages = [
{
"role": "user",
"content": [
{
"type":"text",
"text":"Describe this image."
},
{
"type":"image",
"image":"image.png"
}
]
}
]
inputs = processor.apply_chat_template(
messages,
tokenize=True,
return_tensors="pt"
)
outputs = model.generate(
**inputs,
max_new_tokens=512
)
print(processor.decode(outputs[0]))
Long Context Example
text = open("document.txt").read()
messages = [
{
"role":"user",
"content":f"Summarize:\n\n{text}"
}
]
inputs = processor.apply_chat_template(
messages,
tokenize=True,
return_tensors="pt"
)
outputs = model.generate(
**inputs,
max_new_tokens=1024
)
print(processor.decode(outputs[0]))
Performance
| Configuration | Relative Speed | VRAM |
|---|---|---|
| Gemma 4 Reference | 1× | ~16 GB |
| Quatfit Mini BF16 | 2.5× | ~16 GB |
| + Speculative Decoding | 3.9× | ~16 GB |
| GGUF Q4_K_M | 4.1× | ~5 GB |
Benchmark Scores
| Domain | Accuracy |
|---|---|
| Overall | 89.1% |
| CLI | 95.0% |
| Exams | 93.3% |
| Coding | 92.5% |
| Agentic Tasks | 92.5% |
| Science | 91.7% |
| Finance | 90.0% |
| Security | 90.0% |
| Social Intelligence | 90.0% |
| Reasoning | 88.9% |
| Expert Knowledge | 83.8% |
| Mathematics | 81.3% |
Architecture
Quatfit Mini preserves the Google Gemma 4 multimodal architecture while introducing Quatfit-specific optimizations for inference and deployment.
Foundation
| Component | Description |
|---|---|
| Base Model | Google Gemma 4 |
| Architecture | Dense Decoder-only Transformer |
| Text Backbone | Gemma 4 |
| Vision Backbone | Gemma 4 Vision Transformer |
| Audio Backbone | Gemma 4 Conformer |
| Context Length | 131,072 |
| Precision | BF16 |
Text Decoder
| Component | Value |
|---|---|
| Layers | 42 |
| Hidden Size | 2560 |
| Attention Heads | 8 |
| KV Heads | 2 (Grouped Query Attention) |
| Feed Forward | GeGLU |
| Rotary Embeddings | Yes |
| RMSNorm | Yes |
| Flash Attention | Flash Attention 3 |
Vision Encoder
- Gemma 4 Vision Transformer
- Patch Size 16×16
- Pan & Scan Support
- Native Visual Embeddings
Audio Encoder
- Gemma 4 Conformer
- Streaming Compatible
- Causal Chunk Attention
Quatfit Optimizations
Quatfit Mini extends Gemma 4 with deployment-focused optimizations including:
- Optimized GGUF conversion
- Faster llama.cpp inference
- Speculative Decoding
- Flash Attention 3
- Sliding Window Attention
- Grouped Query Attention
- KV Cache Sharing
- Consumer GPU optimization
- Memory-efficient inference
- Quantization-aware deployment
Training
Quatfit Mini is built upon the pretrained Google Gemma 4 model and further optimized by Quatfit AI Research through post-training techniques including:
- Supervised Fine-Tuning (SFT)
- Preference Alignment
- Instruction Tuning
- Multimodal Alignment
- Inference Optimization
The underlying foundation model retains the original Gemma 4 pretraining while Quatfit contributes additional post-training and deployment optimizations.
Cross Platform Support
| Quantization | VRAM | Platforms |
|---|---|---|
| Q4_K_M | ~5 GB | Ollama, LM Studio, llama.cpp, Jan, Open WebUI |
| Q5_K_M | ~6 GB | Ollama, LM Studio, llama.cpp |
| Q6_K | ~7 GB | llama.cpp |
| Q8_0 | ~9 GB | Full Precision |
Hardware
Recommended
- NVIDIA RTX 3090
- RTX 4090
- RTX 6000 Ada
- A100
- H100
Minimum
- GPU with ~6 GB VRAM (Q4 GGUF)
- CPU inference through llama.cpp
Responsible AI
Quatfit Mini may generate inaccurate, biased, or inappropriate outputs.
For production deployments:
- Verify critical information
- Apply RAG for factual grounding
- Use application-level safety filters
- Keep human oversight for high-risk domains
Citation
@article{quatfitmini2026,
title={Quatfit Mini: A Gemma 4-Based Multimodal Model Optimized for Efficient Inference},
author={Quatfit AI Research},
year={2026}
}
License
Quatfit Non-Commercial License v1.
Commercial licensing is available through Quatfit AI Research.
- Downloads last month
- 597