rovdetection/code-1b-chat-v2-gguf

code-1b-chat-v2 GGUF

GGUF quantization of rovdetection/code-1b-chat-v2.

Available files

File	Quant	Size	Use case
code-1b-chat-v2-Q4_K_M.gguf	Q4_K_M	~700 MB	Recommended — best quality/speed

Usage with llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
    repo_id="rovdetection/code-1b-chat-v2-gguf",
    filename="code-1b-chat-v2-Q4_K_M.gguf",
    n_ctx=2048,
)

SYSTEM = "Below is an instruction that describes a coding task. Write a response that appropriately completes the request."
prompt = f"{SYSTEM}\n\n### Instruction:\nWrite a fibonacci function.\n\n### Response:\n"

out = llm(prompt, max_tokens=200, temperature=0.7, stop=["### Instruction:"])
print(out["choices"][0]["text"])