Random Output — Workaround & Best Practices
Problem: When running PiCo-1B in GGUF format, the model outputs random/gibberish text instead of coherent completions.
Root Causes Identified:
Base model vs. Chat mode — PiCo-1B is a base (pre-trained) model, not an instruction-tuned chat model. Running it in a standard "Chat" or "Instruct" interface causes it to panic and generate incoherent output. Switch to Text Completion / Notebook mode in your frontend (Ollama, LM Studio, etc.).
Aggressive quantization — At 1B parameters, the model is already very small. Heavy compression like Q4_K_M can corrupt weights and degrade output quality significantly.
Recommended Fix:
Use the full (unquantized) weights or bigger compressed model versions for stable output.
Ensure your local inference software (Ollama, LM Studio, llama.cpp, etc.) is updated to the latest version.
Always run in Text Completion mode, not Chat/Instruct mode.
Confirmed Working: A community member verified this resolved the issue on modest hardware, with stable tokens/sec even at large context windows.
If you're experiencing similar issues or have found other workarounds, please share below! 🙏