Sommerfugl-31B

Sommerfugl ("butterfly") is a family of Norwegian language models trained by oolabs.no. Built with Gemma: this model is google/gemma-4-31B-it finetuned with a parameter-efficient LoRA (122M trainable parameters, merged into the weights) on 45,000 quality-gated synthetic Norwegian instruction rows — grounded in real Norwegian text (NCC, Wikipedia, ordbøkene) and rigorously decontaminated against all 19 Norwegian evaluation datasets (sentence-level blocklist + 10-gram overlap index).

Why

Google's instruction tuning of Gemma 4 damages its Norwegian: on the National Library of Norway's leaderboard, gemma-4-31B-it ranks in the top 3 on eleven categories but collapses to ~rank 48 on Norwegian language knowledge and ~rank 51 on Norwegian summarization. Sommerfugl-31B repairs both while retaining the base's strengths.

Results

Evaluated under the National Library's published leaderboard protocol (lm-evaluation-harness + NorEval; cloze MC, chat template, best over prompt versions × {0,5}-shot, greedy decoding). Gemma column: NB's own leaderboard row. Sommerfugl and Borealis 2 (NbAiLab's newest flagship, 26B-A4B + 47.6B-word CPT + SFT, evaluated by us under the identical protocol):

task (metric)	gemma-4-31B-it	Sommerfugl-31B	Borealis 2 preview
Language knowledge: NoCoLA (acc)	0.828	0.859	0.834
Language knowledge: NCB (acc)	0.755	0.825	0.774
Summaries: NorSumm nob / nno (bleu)	4.08 / 3.37	7.39 / 7.00	6.62 / 5.15
Summarize on request (bleu)	0.98	4.61	1.75
Rewrite (bleu)	0.03	3.66	0.11
Idioms nob / nno (fscore)	0.085 / 0.141	0.442 / 0.541	0.121 / 0.160
Grammar correction (exact match)	0.294	0.402	0.342
Reading comp.: NorQuAD (f1)	0.445	0.748	0.799
Translation en↔nb/nn (bleu, 4-dir avg)	57.2	59.1	58.1
Belebele (acc)	0.938	0.938	0.904
OpenbookQA / CommonsenseQA (acc)	0.968 / 0.860	0.957 / 0.830	0.955 / 0.808
NoReC sentiment (acc)	0.910	0.923	0.918
TruthfulQA mc nob / nno (acc)	0.850 / 0.930	0.664 / 0.772	0.811 / 0.842
MMLU English (acc)	0.831	0.831	0.550
MMLU Norwegian (acc)	0.859	0.824	—

Head-to-head vs Borealis 2 across all 21 protocol-complete rows: 18–3.

Known limitations (reported deliberately)

• TruthfulQA (mc) regresses vs the base -it (−0.19): supervised finetuning erodes part of the base's RLHF truthfulness calibration. Under investigation for v2.
• MMLU-nb −0.035 vs the base -it.
• Not evaluated: NB's unpublished internal tasks (translated ARC/GSM8K/IFEval/GoldenSwag, safety sets — ~8% of their suite) and the heavy 5-shot MC variants; exclusions applied identically to every model we compare.
• We publish no overall/aggregate score: that number belongs to the National Library's own evaluation pipeline.

Training data

Synthetic instruction data generated locally with unsloth/gemma-4-26B-A4B-it as teacher, transforming real Norwegian sources (never free-form generation): press-register summaries from NCC news, rule-based grammar-correction pairs, idiom completions crawled from ordbøkene (ord.uib.no), judged abstention/false-premise QA, extractive QA, nb→nn translation. Every row passed decontamination against all evaluation datasets. Identity data: ~300 rows.

Total training compute for this model: ≈ $47 of rented H200 time.