Instructions to use norecyc/lastbox-gemma4-e2b-sft-v3 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use norecyc/lastbox-gemma4-e2b-sft-v3 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="norecyc/lastbox-gemma4-e2b-sft-v3")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForImageTextToText

processor = AutoProcessor.from_pretrained("norecyc/lastbox-gemma4-e2b-sft-v3")
model = AutoModelForImageTextToText.from_pretrained("norecyc/lastbox-gemma4-e2b-sft-v3")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

llama-cpp-python

How to use norecyc/lastbox-gemma4-e2b-sft-v3 with llama-cpp-python:

# !pip install llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
	repo_id="norecyc/lastbox-gemma4-e2b-sft-v3",
	filename="lastbox-gemma4-e2b-v3-bf16.gguf",
)

llm.create_chat_completion(
	messages = [
		{
			"role": "user",
			"content": "What is the capital of France?"
		}
	]
)

Notebooks
Google Colab
Kaggle
Local Apps

llama.cpp

How to use norecyc/lastbox-gemma4-e2b-sft-v3 with llama.cpp:

Install from brew

brew install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf norecyc/lastbox-gemma4-e2b-sft-v3:BF16
# Run inference directly in the terminal:
llama-cli -hf norecyc/lastbox-gemma4-e2b-sft-v3:BF16

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf norecyc/lastbox-gemma4-e2b-sft-v3:BF16
# Run inference directly in the terminal:
llama-cli -hf norecyc/lastbox-gemma4-e2b-sft-v3:BF16

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf norecyc/lastbox-gemma4-e2b-sft-v3:BF16
# Run inference directly in the terminal:
./llama-cli -hf norecyc/lastbox-gemma4-e2b-sft-v3:BF16

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf norecyc/lastbox-gemma4-e2b-sft-v3:BF16
# Run inference directly in the terminal:
./build/bin/llama-cli -hf norecyc/lastbox-gemma4-e2b-sft-v3:BF16

Use Docker

docker model run hf.co/norecyc/lastbox-gemma4-e2b-sft-v3:BF16

LM Studio
Jan

vLLM

How to use norecyc/lastbox-gemma4-e2b-sft-v3 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "norecyc/lastbox-gemma4-e2b-sft-v3"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "norecyc/lastbox-gemma4-e2b-sft-v3",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/norecyc/lastbox-gemma4-e2b-sft-v3:BF16

SGLang

How to use norecyc/lastbox-gemma4-e2b-sft-v3 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "norecyc/lastbox-gemma4-e2b-sft-v3" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "norecyc/lastbox-gemma4-e2b-sft-v3",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "norecyc/lastbox-gemma4-e2b-sft-v3" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "norecyc/lastbox-gemma4-e2b-sft-v3",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Ollama
How to use norecyc/lastbox-gemma4-e2b-sft-v3 with Ollama:
```
ollama run hf.co/norecyc/lastbox-gemma4-e2b-sft-v3:BF16
```

Unsloth Studio new

How to use norecyc/lastbox-gemma4-e2b-sft-v3 with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for norecyc/lastbox-gemma4-e2b-sft-v3 to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for norecyc/lastbox-gemma4-e2b-sft-v3 to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for norecyc/lastbox-gemma4-e2b-sft-v3 to start chatting

Docker Model Runner
How to use norecyc/lastbox-gemma4-e2b-sft-v3 with Docker Model Runner:
```
docker model run hf.co/norecyc/lastbox-gemma4-e2b-sft-v3:BF16
```

Lemonade

How to use norecyc/lastbox-gemma4-e2b-sft-v3 with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull norecyc/lastbox-gemma4-e2b-sft-v3:BF16

Run and chat with the model

lemonade run user.lastbox-gemma4-e2b-sft-v3-BF16

List all available models

lemonade list

LastBox — Gemma 4 E2B fine-tuned for offline survival assistance (v3 SFT, submission snapshot)

Tiny offline survival assistant in a Pelican case. Gemma 4 E2B fine-tuned with Unsloth, quantized to Q4_K_M, served by llama.cpp on an 8 GB Raspberry Pi 5 over a LoRa 868 MHz mesh radio. ~700 ms first token, 6–7 tok/s sustained on ARM CPU.

This is the v3 SFT snapshot submitted to the Kaggle "Gemma 4 Good" Hackathon 2026 (deadline 2026-05-18 23:59 UTC). For the post-deadline v6 SFT-warmup checkpoint with 72 % tool-emission and 38× agentic_score lift, see norecyc/lastbox-gemma4-e2b-v6-toolprior.

GitHub: https://github.com/agentsmill/lastbox (tag v1-submission)
Live site: https://agentsmill.github.io/lastbox/

What it does

Survival Q&A — text-only first aid, bushcraft, navigation, power, hazards.
Optical triage — paired with mmproj-F16.gguf (SigLIP vision, unmodified upstream) it describes plants, wounds, terrain from the RPi camera.
Mesh-radio relay — every reply destined for the radio is hard-capped at 150 bytes UTF-8 so it fits a single LoRa packet at the legal duty cycle.

Files

File	Size	Purpose
`lastbox-gemma4-e2b-v3-q4_k_m.gguf`	3.2 GB	Quantized inference weights — drop into `llama-server`
`lastbox-gemma4-e2b-v3-bf16.gguf`	8.7 GB	bf16 weights (source for quantization)
`model.safetensors`	9.6 GB	Merged HF format
`tokenizer.json` + config + chat template	–	Required for inference
`lora/adapter_model.safetensors`	49 MB	LoRA adapter (for reproducible re-merge against base)

Quickstart — llama.cpp server

docker run --rm -p 11436:8080 \
  -v $(pwd):/models:ro \
  ghcr.io/ggml-org/llama.cpp:server \
  -m /models/lastbox-gemma4-e2b-v3-q4_k_m.gguf \
  --host 0.0.0.0 --port 8080 \
  --ctx-size 4096 --threads 4 --parallel 1 \
  --jinja --chat-template-file /models/chat_template.jinja \
  --no-mmap --mlock

Training recipe

base:               unsloth/gemma-4-E2B-it
framework:          Unsloth FastModel + TRL SFTTrainer
LoRA:               r=8, alpha=8, dropout=0, bias="none"
target modules:     language layers + attention + MLP, no vision
chat template:      gemma-4 (no-thinking) — suppresses CoT preamble
hardware:           1× NVIDIA GB10 (Grace Blackwell, 121 GB unified, CUDA 13)
data:               1 034 EN survival dialogs (train) + 114 (val)
data generation:    Kimi K2.5 teacher via OpenRouter, ~$1.10 total
epochs:             2 (v3 no-think); 3 (v2 thinking, deprecated)
lr:                 2e-4, cosine, warmup 10 steps
batch / grad accum: 4 / 4
training time:      27 min on GB10
final train loss:   0.08, eval loss plateau ~2.64 on 114-dialog held-out

Eval (golden_en, n=25 held-out, streaming SSE methodology)

Metric	v3 (this checkpoint)
response_quality	0.506
byte_compliance	0.48
format_ok hybrid	0.52
persona_ok	0.52
tool_emission_rate	~0 %
median first-token	~700 ms (warm)
sustained throughput	6.4–7 tok/s

Known limitations of this v3 snapshot:

tool_emission_rate is ~0% — the SFT optimised on tool-call traces but the eval-time prompt was missing the tool-definitions JSON block from the training format. Fixed in v6 along with a targeted SFT warmup on tool-only pairs.
Streaming-SSE eval lost 12/25 dialogs to client-side disconnects, so the 0.52 floor on byte/format/persona is actually 13/25 completion ratio, not a quality issue. Non-streaming eval shows every completed dialog passes.

Dataset

Training and eval data: norecyc/lastbox-survival-dialogues

License

Model weights are subject to the Gemma Terms of Use. Training scripts and LoRA adapter weights are Apache 2.0 (see GitHub repo).

Citation

@misc{lastbox_gemma4_v3_2026,
  title  = {LastBox v3: Gemma 4 E2B fine-tuned for offline survival assistance on Raspberry Pi 5},
  author = {Mateusz Pawelczuk},
  year   = {2026},
  url    = {https://huggingface.co/norecyc/lastbox-gemma4-e2b-sft-v3},
  note   = {Kaggle Gemma 4 Good Hackathon 2026 submission}
}

Downloads last month: 515

Safetensors

Model size

5B params

Tensor type

BF16

Model tree for norecyc/lastbox-gemma4-e2b-sft-v3

Base model

google/gemma-4-E2B

Finetuned

google/gemma-4-E2B-it

Finetuned

unsloth/gemma-4-E2B-it

Adapter

(30)

this model

Adapters

1 model

norecyc
/

lastbox-gemma4-e2b-sft-v3