NexAI-v1

A locally-deployable, CPU-optimized instruction-following language model for software development assistance

Developed by Anoop Singh, founder of NextMatrix

---

Table of Contents

1. Model Overview
2. Model Details
3. Intended Use
4. How to Get Started
5. Training Details
6. Evaluation and Known Limitations
7. Technical Specifications
8. License
9. Citation

---

Model Overview

NexAI-v1 is a 7-billion-parameter causal language model fine-tuned for coding assistance and instruction-following, built on top of Qwen2.5-7B-Instruct using QLoRA and distributed as a quantized GGUF artifact for CPU inference.

The model targets a specific deployment gap: most capable open-weight coding assistants assume GPU access at inference time. NexAI-v1 is packaged specifically to run on consumer CPU hardware at usable speeds via llama.cpp and llama-cpp-python, making it suitable for local, offline, or privacy-sensitive development environments where sending code to a third-party API is not an option.

Model Details

Developed by	Anoop Singh, founder of NextMatrix
Model type	Decoder-only causal language model
Base model	Qwen/Qwen2.5-7B-Instruct
Fine-tuning method	QLoRA (4-bit NF4 base, LoRA adapters merged into full-precision weights)
Parameters	7.61B
Distributed format	GGUF
Quantization	Q4_K_M (~4.9 bits per weight)
Artifact size	~4.5 GB
Context length	32,768 tokens (inherited from base model)
Languages	English, Hindi / Hinglish
License	Apache 2.0

Intended Use

In scope:

• Local, offline coding assistance and code explanation
• Retrieval-augmented generation (RAG) workflows where the model is given retrieved context and asked to answer grounded in that context
• Prototyping and evaluation of small, CPU-deployable instruction-following models
• Educational use: understanding QLoRA fine-tuning and GGUF deployment pipelines

Out of scope:

• Medical, legal, or financial advice
• Any application requiring guaranteed factual accuracy without a human review step
• High-throughput production serving (this build is optimized for local single-user CPU inference, not scaled deployment)

How to Get Started

Python (llama-cpp-python)

pip install llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
    repo_id="Anoopsingh53/nexai-v1",
    filename="nexai-v1-Q4_K_M.gguf",
    n_ctx=4096,
    n_threads=4,
    n_gpu_layers=0,  # CPU-only inference
    verbose=False,
)

response = llm.create_chat_completion(
    messages=[
        {"role": "system", "content": "You are NexAI, a helpful coding assistant."},
        {"role": "user", "content": "Write a Python function to check if a number is prime."},
    ],
    max_tokens=512,
)

print(response["choices"][0]["message"]["content"])

llama.cpp CLI

./llama-cli \
  -m nexai-v1-Q4_K_M.gguf \
  -p "<|im_start|>system\nYou are NexAI, a helpful assistant.<|im_end|>\n<|im_start|>user\nHello!<|im_end|>\n<|im_start|>assistant\n" \
  -n 256 \
  -c 4096

Training Details

Method	QLoRA: 4-bit NF4 quantized base model, LoRA adapters trained then merged into full fp16 weights
Optimizer	AdamW, 8-bit (paged_adamw_8bit)
Learning rate	2e-4
LoRA rank / alpha / dropout	r=16 / alpha=32 / dropout=0.05
LoRA target modules	q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
Hardware	Single NVIDIA Tesla T4 (Kaggle)

Data status: the current release was fine-tuned on a small demonstration dataset used to validate the training-to-deployment pipeline end-to-end. It should be treated as an early checkpoint rather than a model tuned on a large, diverse instruction corpus. A larger and more diverse fine-tuning dataset is planned for a future release; this section will be updated with dataset size and composition at that time.

Evaluation and Known Limitations

No formal benchmark evaluation (e.g. HumanEval, MBPP, MT-Bench) has been run on this checkpoint yet. Until benchmark results are published here, treat any quality claims about this specific fine-tune as unverified — the underlying Qwen2.5-7B-Instruct base model's published benchmarks do not automatically transfer to this fine-tune.

Known limitations:

• Can produce incorrect or outdated information, particularly outside its training distribution
• Generated code should be reviewed and tested before use, like any LLM output
• Quantization (Q4_K_M) trades a small amount of accuracy for size and speed relative to the full-precision merged model
• Not evaluated for safety-critical or adversarial-input robustness

Technical Specifications

Architecture	Qwen2.5 dense transformer
Tokenizer	Qwen2.5 tokenizer (BPE)
Inference format	GGUF, compatible with llama.cpp, llama-cpp-python, LM Studio, Ollama (via GGUF import), and other GGUF-compatible runtimes
Minimum recommended RAM	8 GB system RAM for comfortable CPU inference

License

Released under the Apache 2.0 License. The Qwen2.5 base model weights remain subject to Alibaba's original license terms; see the base model card for details.

Citation

If you reference this model, please cite:

@misc{nexai-v1,
  author = {Singh, Anoop},
  title = {NexAI-v1: A CPU-Optimized Instruction-Following Model},
  year = {2026},
  publisher = {Hugging Face},
  howpublished = {\url{https://huggingface.co/Anoopsingh53/nexai-v1}}
}

---

Author: Anoop Singh — NextMatrix Base model: Qwen/Qwen2.5-7B-Instruct