Qwen2.5-Coder-7B-Agentic-CoT-LoRA

This model is a LoRA fine-tune of Qwen/Qwen2.5-Coder-7B-Instruct.

Training Details

This model was fine-tuned on AlicanKiraz0/Agentic-Chain-of-Thought-Coding-SFT-Dataset, a high-quality supervised fine-tuning dataset for training agentic coding assistants with Chain-of-Thought reasoning capabilities.

Dataset Information

The training dataset was created by processing and distilling ~20GB of GitHub crawl data using Minimax-M2 to generate structured, reasoning-rich coding examples. Each sample demonstrates systematic problem-solving with explicit tool usage patterns.

Dataset features:

• Task descriptions with clear coding objectives
• Context and background information
• Strategic planning breakdowns (3-8 steps)
• Chain-of-Thought reasoning explanations
• Tool action sequences (editor, bash, python, browser)
• Implementation summaries

Training Configuration

• Base Model: Qwen/Qwen2.5-Coder-7B-Instruct
• Method: LoRA (Low-Rank Adaptation)
• LoRA Rank: 64
• LoRA Alpha: 128
• Target Modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
• Epochs: 15
• Learning Rate: 2e-4
• Scheduler: Cosine
• Max Sequence Length: 4096
• Precision: BF16

Usage

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base_model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-Coder-7B-Instruct")
model = PeftModel.from_pretrained(base_model, "GhostScientist/Qwen2.5-Coder-7B-Agentic-CoT-LoRA")
tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen2.5-Coder-7B-Instruct")

messages = [
    {"role": "system", "content": "You are an expert agentic coding assistant that solves complex programming tasks through systematic reasoning and tool usage."},
    {"role": "user", "content": "Build a REST API with authentication..."}
]

text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(text, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=2048)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Intended Use

This model is designed for:

• Agentic coding tasks requiring systematic reasoning
• Chain-of-thought problem decomposition
• Tool-use planning and execution
• Complex multi-step coding challenges

Limitations

• Fine-tuned on synthetically generated data
• English-only
• May inherit biases from base model and training data