| --- |
| language: |
| - tr |
| - en |
| - de |
| - es |
| - fr |
| - ru |
| - zh |
| - ja |
| - ko |
| license: mit |
| tags: |
| - turkish |
| - türkiye |
| - ai |
| - lamapi |
| - next-codex |
| - coder |
| - codex |
| - text-generation |
| - open-source |
| - 30b |
| - moe |
| - mixture-of-experts |
| - code-generation |
| - coding |
| - llm |
| - transformer |
| - artificial-intelligence |
| pipeline_tag: text-generation |
| datasets: |
| - mlabonne/FineTome-100k |
| - google/code_x_glue_ct_code_to_text |
| - bigcode/the-stack-v2 |
| - neulab/agent-data-collection |
| - openai/gsm8k |
| - princeton-nlp/SWE-bench_Verified |
| - microsoft/orca-math-word-problems-200k |
| - qwedsacf/competition_math |
| - hotpotqa/hotpot_qa |
| - wics/strategy-qa |
| - glaiveai/glaive-function-calling-v2 |
| - Anthropic/hh-rlhf |
| - ccdv/cnn_dailymail |
| - allenai/ai2_arc |
| - allenai/sciq |
| - google-research-datasets/mbpp |
| - openai/openai_humaneval |
| - allenai/openbookqa |
| - baber/piqa |
| - allenai/winogrande |
| - Rowan/hellaswag |
| - allenai/social_i_qa |
| - databricks/databricks-dolly-15k |
| - truthfulqa/truthful_qa |
| - HuggingFaceH4/ultrachat_200k |
| - OpenAssistant/oasst1 |
| - iamtarun/python_code_instructions_18k_alpaca |
| - nickrosh/Evol-Instruct-Code-80k-v1 |
| - arcee-ai/agent-data |
| - GreenerPastures/All-Your-Base-Full |
| - FreedomIntelligence/Socratic |
| - qihoo360/Light-R1-SFTData |
| - dongguanting/ARPO-SFT-54K |
| library_name: transformers |
| --- |
| |
|  |
|
|
| # 💻 Next-Codex (L846MoE) |
|
|
| ### Code your future with our models. |
|
|
| [](https://opensource.org/licenses/MIT) |
| []() |
| [](https://huggingface.co/Lamapi/next-codex) |
| [](https://discord.gg/XgH4EpyPD2) |
|
|
| --- |
|
|
| ## 📖 Overview |
|
|
| **Next-Codex** is a high-performance, specialized **Mixture-of-Experts (MoE)** Large Language Model designed specifically for code generation, debugging, and software engineering tasks. |
|
|
| Unlike traditional dense models, **Next-Codex** utilizes a sparse architecture with **30 Billion total parameters**, but only activates **3 Billion parameters per token**. This unique design allows it to deliver the deep reasoning capabilities of a massive model while maintaining the ultra-low latency and inference cost of a lightweight 3B model. It is fine-tuned on a massive corpus of code across 20+ programming languages, making it the most efficient coding assistant in its class. |
|
|
| --- |
|
|
| ## ⚡ Highlights |
|
|
| - 🇹🇷 **Türkiye’s First Specialized MoE Coding Model:** Designed for speed and precision. |
| - 🚀 **Hyper-Efficient Inference:** Runs with **3B active parameters**, enabling deployment on consumer GPUs (e.g., RTX 3090/4090). |
| - 💻 **SOTA Coding Performance:** Surpasses Claude Sonnet 4 and rivals o3-High in Python & JavaScript benchmarks. |
| - 🌍 **Polyglot Programming:** Master-level proficiency in Python, JS/TS, Rust, Go, C++, SQL, and Swift. |
| - 🧠 **Context-Aware Debugging:** Excellent at understanding large codebases and suggesting architectural improvements. |
| - 🏢 **Production Ready:** Optimized for autocomplete, unit test generation, and docstring creation. |
|
|
| --- |
|
|
| ## 📊 Benchmark Performance (Coding & Logic) |
|
|
| **Next-Codex** achieves state-of-the-art results among open-weights coding models, balancing extreme efficiency with high accuracy. |
|
|
| Benchmarks are being conducted... |
| --- |
|
|
| ## 🚀 Installation & Usage |
|
|
| **Note:** Due to the MoE architecture, this model is memory efficient. You can run it comfortably on 24GB VRAM GPUs (4-bit quantization highly recommended for lower VRAM). |
|
|
| ``` |
| !pip install unsloth transformers |
| ``` |
|
|
| ```python |
| from unsloth import FastLanguageModel |
| |
| # Load the MoE Model |
| model, tokenizer = FastLanguageModel.from_pretrained( |
| "Lamapi/next-codex", |
| load_in_4bit = True, # Optimized for 24GB VRAM |
| ) |
| |
| messages = [ |
| {"role": "system", "content": "You are Next-Codex, an expert software engineer and AI coding assistant."}, |
| {"role" : "user", "content" : "Write a highly optimized Rust function to calculate the Fibonacci sequence using memoization."} |
| ] |
| |
| text = tokenizer.apply_chat_template( |
| messages, |
| tokenize = False, |
| add_generation_prompt = True |
| ) |
| |
| from transformers import TextStreamer |
| _ = model.generate( |
| **tokenizer(text, return_tensors = "pt").to("cuda"), |
| max_new_tokens = 2048, |
| temperature = 0.2, # Lower temperature for code precision |
| top_p = 0.95, |
| streamer = TextStreamer(tokenizer, skip_prompt = True), |
| ) |
| ``` |
|
|
| --- |
|
|
| ## 🧩 Key Features |
|
|
| | Feature | Description | |
| | :--- | :--- | |
| | 🔀 **Smart Routing (MoE)** | Dynamically routes tokens to the best "expert" layers, activating only 3B params for speed. | |
| | 🛠️ **Full-Stack Mastery** | Trained on frontend (React, Vue), backend (Django, Spring), and systems (C, Rust) code. | |
| | 🇹🇷 **Code Support** | Exceptional ability to understand Turkish variable names and comments in legacy codebases. | |
| | 🐞 **Deep Debugging** | Analyzes stack traces and logic errors to provide instant fixes. | |
| | 📝 **Docstring & Testing** | Automatically generates Javadoc, PyDoc, and Unit Tests (Pytest/Jest). | |
| | 🔒 **Secure Coding** | Aligned to avoid common vulnerabilities (SQLi, XSS) in generated code. | |
|
|
| --- |
|
|
| ## 📐 Model Specifications |
|
|
| | Specification | Details | |
| | :--- | :--- | |
| | **Architecture** | Mixture of Experts (MoE) Transformer | |
| | **Total Parameters** | 30 Billion | |
| | **Active Parameters** | 3 Billion (per token) | |
| | **Context Window** | 32k Tokens | |
| | **Experts** | 8 Experts (Top-2 Routing) | |
| | **Training Data** | 1T+ Tokens of Code (The Stack v2, GitHub, Synthetic) | |
| | **Quantization** | GGUF, AWQ, GPTQ supported | |
|
|
| --- |
|
|
| ## 🎯 Ideal Use Cases |
|
|
| * **IDE Autocomplete Plugins** — Low latency makes it perfect for "Copilot" style completions. |
| * **Legacy Code Refactoring** — Converting outdated code to modern standards (e.g., Java 8 to Java 21). |
| * **SQL Generation** — Text-to-SQL for complex data analytics. |
| * **Turkish/English Development** — Teams working in bilingual environments. |
| * **Algorithm Optimization** — Reducing time complexity of existing functions. |
|
|
| --- |
|
|
| ## 📄 License |
|
|
| Licensed under the **MIT License** — free for commercial and non-commercial use. |
|
|
| --- |
|
|
| ## 📞 Contact & Support |
|
|
| * 📧 **Email:** [lamapicontact@gmail.com](mailto:lamapicontact@gmail.com) |
| * 🤗 **HuggingFace:** [Lamapi](https://huggingface.co/Lamapi) |
|
|
| --- |
|
|
| > **Next-Codex** — Smart as a giant, fast as a lightweight. The future of coding is MoE. |
|
|
| [](https://huggingface.co/Lamapi) |
