README.md

---
title: ThingsAI
type: org
tags:
- slm
- llm
- pytorch
- amdl
- math
- code
---


# Welcome to ThingsAI. Building highly efficient, logic-driven Small Language Models that run anywhere.

## Our Models

* **Quark-135M**
  A lightweight bilingual (Italian + English) language model with 135M parameters. Features GQA, SwiGLU, RMSNorm, and RoPE. Trained on 50B+ tokens of curated data.
* **Quark-270M**
  Our scaled small model featuring 270M parameters, 32 layers, 768 hidden dimensions, and a 65K vocabulary. Designed for extended bilingual capabilities.
* **Quark-Math-Code (~36M)**
  Our ultra-compact, deep-thin architecture (~36M parameters, 14 layers, 65K vocabulary) engineered specifically for STEM, coding, and mathematical reasoning. Actively pre-training on a 5B token target with a hardened Chain-of-Thought (CoT), OpenWebMath, and pure-code mix.
* **Quark-Mod**
  A multi-label moderation model covering 9 categories for safe AI deployment: toxic, severe_toxic, obscene, threat, insult, identity_hate, cyberbullying, hate_speech, offensive.

## What We Focus On

* **Hyper-Efficient Architectures:** Mastering the sub-1B parameter space using GQA, Grouped-Query Attention, and deep-thin layer scaling.
* **Embedded Chain-of-Thought (CoT):** Hardcoding step-by-step reasoning tokens into the pre-training phase of tiny models to punch far above their weight class in logic benchmarks.
* **Bilingual & Specialty Data:** Multi-source streaming pipelines fusing Italian, English, high-density mathematics, and code.
* **Open-Source & Real-World Deployable:** Everything from weights to datasets is open. Tailored to achieve massive throughput on consumer GPUs and edge hardware.

## Resources

* **Quark-135M-Bilingual:** Our flagship general-purpose bilingual model.
* **Quark-Mod:** Multi-label content moderation for production pipelines.
* **HuggingFace Community:** All our released models, tokenizers, and custom datasets.
* **GitHub Open Source:** Training scripts, custom multi-source streaming iterators, and deployment tools.