Solar-Open2-250B-GGUF

Built with Solar

Unofficial community GGUF quantizations of upstage/Solar-Open2-250B. Not affiliated with, sponsored by, or endorsed by Upstage. This is an independent derivative work.

Available quantizations

• Q4_K_M (~141 GB) — Solar-Open2-250B-Q4_K_M-*-of-*.gguf. The 128-wide KDA gate projections (ssm_f_b / ssm_g_b) fall back q4_K→q5_0, as is normal for Kimi-family linear-attention models.
• Q8_0 (~250 GB) — Solar-Open2-250B-Q8_0-*-of-*.gguf. Near-lossless; no fallbacks (Q8_0's 32-wide blocks quantize the 128-wide gates cleanly).

Both are split into <50 GB shards; point llama.cpp at the first shard of a set and it loads the rest automatically.

Modification notice: these files were modified from the original — converted to GGUF and quantized from the original bf16 weights.

⚠️ Requires a patched llama.cpp

Solar-Open2 is a hybrid MoE (KDA linear attention + gated GQA + DeepSeek-V3-style MoE) and its architecture is not in upstream llama.cpp. To run these GGUFs you must build llama.cpp from the fork that adds the solar-open2 architecture:

https://github.com/llamaraspberryrabbit/llama.cpp/tree/add-solar-open2

⚠️ This fork is AI-created and will not be upstreamed

That fork — the HF→GGUF converter, the C++ inference graph, the reasoning-parser patch, and these quantizations — was created by AI (Claude Opus 4.8). It has not been numerically validated against Upstage's reference implementation, and no pull request will be opened against upstream llama.cpp. Treat it as an experimental, community-grade artifact, and review the code yourself before relying on it — you are responsible for anything you run.

It has been verified to load and generate coherent, factually correct output on a CUDA build, and reasoning is parsed into reasoning_content — but see the caveat above.

Usage

# reasoning model — a repeat penalty is recommended
llama-cli -m Solar-Open2-250B-Q4_K_M-00001-of-000NN.gguf \
    -ngl 99 -c 8192 --jinja --reasoning-format deepseek \
    --repeat-penalty 1.2 -cnv

llama.cpp loads the shard set automatically when you point it at the first shard.

License

Built with Solar. Licensed under the Upstage Solar License (see the LICENSE file in this repo, included per Section 4(e)). The original Solar-Open2 model is © Upstage Co., Ltd. Per the license, this derivative's name begins with "Solar" and displays "Built with Solar." Use of the "Solar" name does not imply any endorsement by or official relationship with Upstage.