cstr/Sidon-GGUF

Sidon GGUF (quant ladder)

GGUF conversions of SaruLab Sidon v0.1, a multilingual speech-restoration model that removes noise and reverberation and restores speech bandwidth (16 kHz in → 48 kHz out). Packaged for native, no-PyTorch inference with CrispASR.

The model is the first eight layers of w2v-BERT 2.0 with the Sidon LoRA adapter merged, followed by Sidon's continuous DAC decoder. The SeamlessM4T log-mel window and filter bank are embedded in the GGUF. Port by @KevinAHM (CrispASR PR #283); this repo adds the quantization ladder.

Files

File	Precision	Size	Notes
sidon-v0.1-f16.gguf	FP16 weights, FP32 norms/biases	470 MiB	Reference. Byte-identical to KevinAHM/Sidon-GGUF.
sidon-v0.1-q8_0.gguf	Q8_0 predictor matmuls	320 MiB	Highest-fidelity quant.
sidon-v0.1-q6_k.gguf	Q6_K predictor matmuls	281 MiB	Balanced.
sidon-v0.1-q4_k.gguf	Q4_K predictor matmuls	240 MiB	Smallest. Intelligible, but see fidelity note.

Only the w2v-BERT predictor's large linear weights are quantized; the DAC decoder convolutions and all norms/biases/Snake alpha parameters are kept at their original precision.

Re-quantized 2026-09-02. Two tensors that the earlier ladder quantized are now kept at source precision: the per-layer relative-position table (...self_attn.distance_embedding.weight, 73x64) and the input feature projection (predictor.feature_projection.projection.weight, 160x1024). Neither has a 256-aligned row, so under --q4_k both had silently fallen back to legacy Q4_0 rather than a k-quant — the worst place in this model to spend precision, since the position table biases every attention score in all eight layers.

This also fixes CrispASR issue #416, where the quantized models produced a full-length file of silence on Vulkan while f16 was fine. The position table is the only quantized weight in the model that is broadcast across the attention heads, and ggml routes a quantized weight through its MMQ/Q8_1 pipeline only on devices advertising integer dot product — so the failure appeared on GPUs reporting int dot: 1 (e.g. GTX 1660 SUPER) and not on CPU or on devices without it. Keeping the table unquantized removes it from that path, so these files work on affected GPUs with existing CrispASR builds; CrispASR >= the next release also fixes it at runtime for previously downloaded files.

Cost of the change: +282 KB on the 240 MiB Q4_K file (+0.12 %).

Fidelity

Validated by ASR round-trip (Whisper base.en) on the restored output — the metric that matters for a restoration model, since output-waveform correlation understates perceptual quality:

Quant	ASR round-trip (JFK clip)	Output-waveform corr. vs F16
F16	WER 0 (exact transcript)	1.000
Q8_0	WER 0	r ≈ 0.99, SNR ≈ 16 dB
Q4_K	WER 0	r ≈ 0.61, SNR ≈ 1 dB

All quants restore fully intelligible speech (WER 0). Q4_K's low waveform correlation means its fine restored detail diverges from F16 even though intelligibility is preserved; prefer Q8_0 or Q6_K when output fidelity to the reference matters, and Q4_K when size is the priority.

Validation against the original model

The GGUF port was checked against the upstream SaruLab TorchScript modules (feature_extractor_cpu.pt + decoder_cpu.pt) on the JFK clip:

Stage	Metric	Result
Predictor handoff (w2v-BERT)	cosine, our F16 vs upstream F32	0.998
End-to-end 48 kHz output	Pearson r	0.945
ASR round-trip (Whisper base.en)	transcription	identical

The predictor reproduces the original almost exactly (0.998); the end-to-end drop to 0.945 is the continuous-DAC decoder amplifying F16 weight-rounding through its Snake activations, not a port error — the restored speech is functionally identical (same transcription). The reference intermediates used for this check are published here as sidon-ref.gguf (input_16k, predictor_feats, output_48k) with sidon-ref-48k.wav; regenerate with tools/reference_backends/sidon_ref_dump.py and re-run our side with CRISPASR_SIDON_DUMP_HANDOFF=<path>.

CrispASR usage

crispasr --s2s -m sidon-v0.1-q8_0.gguf -f input.wav --s2s-output restored-48khz.wav

CrispASR auto-detects the sidon architecture from GGUF metadata. The S2S interface processes a complete clip and returns a complete restored clip (not streaming). GPU: --gpu-backend cuda / --gpu-backend vulkan.

Long inputs: the predictor uses O(T²) self-attention (~50 feature frames/sec). Inputs are capped at ~60 s (3000 frames) by default and fail cleanly past that; raise CRISPASR_SIDON_MAX_FRAMES if you have the memory.

Conversion

# F16 from upstream weights:
python models/convert-sidon-to-gguf.py --base w2v-bert-2.0 --sidon sidon_raw_weight --out sidon-v0.1-f16.gguf
# quants:
crispasr-quantize sidon-v0.1-f16.gguf sidon-v0.1-q4_k.gguf q4_k