README.md

metadata

language:
  - en
license: apache-2.0
tags:
  - recursive-language-model
  - multimodal
  - self-automated
  - pytorch
  - safetensors
  - ancient-ai
model_type: ancient_ai
pipeline_tag: text-generation

Ancient.AI.V — Recursive Language Model

Architecture: Recursive Language Model (RLM) Not a Large Language Model — a fundamentally different architecture built from scratch.

Property	Value
Parameters	1.147B
Context Window	64,000 tokens
Layers	24
Hidden Size	2,048
Attention Heads	16 (GQA, 8 KV heads)
FFN Dimension	8,192
Vocab Size	64,000
Activation	SwiGLU
Position Encoding	YaRN-extended RoPE (base 500k, scale 8×)
Weight Format	safetensors
Precision	bfloat16 (fine-tune target)

---

What Makes It Different From an LLM

Standard LLMs run one forward pass: input → output.

Ancient.AI.V runs a Recursive Outer Loop: the model refines its own output recursion_depth times per call, with a learned halting gate that stops early when confident. This is the core of the Recursive Language Model paradigm.

---

Integrated Self-Automated (SA) Modules

All 17 SA modules operate simultaneously within each decoder layer as parallel residual paths — not sequential post-processing steps.

Module	Implementation
SA Meta-Learning	Per-sample fast-weight delta generation (learned MAML inner loop)
SA Reinforcement Learning	Per-token value estimation + policy gate (actor-critic in forward pass)
SA Continual Learning	EWC-inspired importance weighting from initial representations
SA Adaptive Learning	Learned depth-gating; tokens can exit processing early
SA Rewriting	Cross-attention from current → earlier hidden states (in-context revision)
SA NLP	Bigram/trigram convolutions + semantic role projection
SA Problem Solving	Multi-step latent chain-of-thought scratchpad (3 internal steps)
SA Innovation	Novelty-promoting repulsion in embedding space
SA Debugging	Anomaly detection + learned correction on hidden state norms
SA Long/Short-Term Memory	512 persistent learnable memory slots with read/write gating
SA Recursive Seed Learning	Compress → refine → expand self-representation cycle
SA Self-Evaluation & Reward	Per-token reward MLP; plugs directly into PPO/GRPO fine-tuning
SA Goal & Constraint Engine	Learned goal embedding cross-attends to steer generation
SA Memory Consolidation	Bidirectional GRU trace encoder with hippocampal replay
SA Introspection Interface	Uncertainty + confidence mapping over hidden states
SA Recursive Outer Loop	Post-stack self-refinement with learned halting
SA Conversational Intelligence	Dialogue state tracker (turn, topic shift, emotion, formality)

---

Multimodal Support

Native encoders for all four modalities, fused before the decoder stack:

• Text — BPE tokenizer, 64k vocab
• Image — ViT-style patch encoder (16×16 patches, up to 224×224)
• Audio — Whisper-style mel-spectrogram encoder (80 mel bins)
• Video — Frame-by-frame ViT + temporal self-attention

---

Training / Fine-Tuning

This checkpoint contains randomly initialized weights — it is an architecture shell ready for fine-tuning.

Recommended fine-tuning approaches:

• SFT (Supervised Fine-Tuning) with causal LM loss
• RLHF/PPO — plug training reward into the SASelfEvaluation reward head
• GRPO — the sa_eval reward signal is already shaped for group-relative optimization
• LoRA / QLoRA — compatible with standard PEFT adapters

Training the self-reward head jointly with SFT gives Ancient.AI.V self-improvement capability without a separate reward model.

---

Usage

# AutoTokenizer available after fine-tuning with a trained tokenizer
from ancient_ai import AncientConfig, AncientAIV  # after registering custom class
import torch

cfg   = AncientConfig()
model = AncientAIV(cfg)
# Load weights:
# model = AncientAIV.from_pretrained("GODsStrongestSoldier/Ancient.AI.V")

tokenizer = AutoTokenizer.from_pretrained("GODsStrongestSoldier/Ancient.AI.V")
input_ids = tokenizer("Hello Ancient.AI", return_tensors="pt").input_ids

generated = model.generate_text(input_ids, max_new=200, temperature=0.8)
print(tokenizer.decode(generated[0]))

---

Architecture Citation

Ancient.AI.V — Recursive Language Model (RLM)
Author: GODsStrongestSoldier
Year: 2025
Architecture: Custom RLM with 17 integrated SA modules
Repo: https://huggingface.co/GODsStrongestSoldier/Ancient.AI.V

---

License

Apache 2.0 — free for research and commercial fine-tuning.