README.md

---
library_name: transformers
pipeline_tag: token-classification
tags:
  - name-splitting
  - ner
  - modernbert
  - names
language:
  - es
  - en
  - pt
license: mit
model-index:
  - name: tori2-bilineal
    results:
      - task:
          type: token-classification
          name: Name Splitting (bilineal)
        dataset:
          type: custom
          name: Bilineal eval split (MX/CO/ES/PE/CL)
        metrics:
          - type: f1
            value: 0.9948
            name: F1
          - type: precision
            value: 0.9948
            name: Precision
          - type: recall
            value: 0.9949
            name: Recall
  - name: tori2-unilineal
    results:
      - task:
          type: token-classification
          name: Name Splitting (unilineal)
        dataset:
          type: custom
          name: Unilineal eval split (AR/US/BR/PT)
        metrics:
          - type: f1
            value: 0.9927
            name: F1
          - type: precision
            value: 0.9927
            name: Precision
          - type: recall
            value: 0.9927
            name: Recall
---

# Tori v2 — Name Splitter

ModernBERT-base (149M params) fine-tuned for splitting full name strings into
**forenames** and **surnames** using BIO token classification.

## Evaluation Results

| Variant | F1 | Precision | Recall | Eval Dataset |
|---------|---:|----------:|-------:|--------------|
| **bilineal** (default) | 0.9948 | 0.9948 | 0.9949 | MX/CO/ES/PE/CL names |
| **unilineal** | 0.9927 | 0.9927 | 0.9927 | AR/US/BR/PT names |

Metrics are entity-level (seqeval) — a name span is only correct if all its tokens match.

## Variants

| Variant | Countries | Surname Pattern | Subfolder |
|---------|-----------|-----------------|-----------|
| **bilineal** (default) | MX, CO, ES, PE, CL | Double surname (paternal + maternal) | `/` (root) |
| **unilineal** | AR, US, BR, PT | Single surname | `unilineal/` |

## Usage

```python
from tori.inference import load_pipeline, split_name

# Default: bilineal model (double-surname countries)
pipe = load_pipeline("ittailup/tori2")
result = split_name(pipe, "Juan Carlos García López")
print(result.forenames)  # ['Juan', 'Carlos']
print(result.surnames)   # ['García', 'López']

# Unilineal model (single-surname countries)
pipe = load_pipeline("ittailup/tori2", variant="unilineal")
result = split_name(pipe, "John Michael Smith")
print(result.forenames)  # ['John', 'Michael']
print(result.surnames)   # ['Smith']
```

## Labels

- `O` — Outside any name entity
- `B-forenames` — Beginning of forename
- `I-forenames` — Inside forename (continuation)
- `B-surnames` — Beginning of surname
- `I-surnames` — Inside surname (continuation)

## Important: Custom Aggregation Required

This model uses ModernBERT's GPT-style BPE tokenizer (Ġ prefix), which is
**not compatible** with HuggingFace's built-in `aggregation_strategy="simple"`.
Use the `tori.inference` module which handles subword aggregation correctly,
or use `aggregation_strategy="none"` and aggregate tokens yourself using
character offsets.

## Training

- **Base model**: [answerdotai/ModernBERT-base](https://huggingface.co/answerdotai/ModernBERT-base) (149M params)
- **Training data**: [philipperemy/name-dataset](https://github.com/philipperemy/name-dataset), Mexico SEP, RENAPER (AR), datos.gob.ar
- **Batch size**: 256 effective (128 x 2 gradient accumulation)
- **Learning rate**: 5e-5, cosine schedule with 10% warmup
- **Epochs**: 3
- **Precision**: bf16
- **Hardware**: NVIDIA A10G (AWS g5.xlarge)