🍐 PEAR: Pairwise Evaluation for Automatic Relative Scoring in Machine Translation

Model variant

This repository contains the PEAR checkpoint trained without Knowledge Distillation (KD).

Installation

Install the PEAR inference toolkit from the GitHub repository:

git clone https://github.com/prosho-97/pear.git
cd pear
pip install .

Quick start: pairwise QE scoring

import pear

metric = pear.load_metric("pear")  # resolves to this Hugging Face model

scores = pear.score_pairwise(
    metric,
    sources=["The cat is on the mat."],
    translations_a=["El gato está en la alfombra."],
    translations_b=["El gato está en el mapa."],
    batch_size=16,
    gpus=1,
    progress_bar=True,
)

Positive scores prefer translations_a; negative scores prefer translations_b.

To score both candidate orders:

scores = pear.score_pairwise(
    metric,
    sources=["The cat is on the mat."],
    translations_a=["El gato está en la alfombra."],
    translations_b=["El gato está en el mapa."],
    mode="both",
)
# {"forward": [...], "reverse": [...]}

Reference-anchored PEAR

PEAR can also be used with a human reference, or any other anchor translation, as the second candidate:

scores = pear.score_reference_anchored(
    metric,
    sources=["The cat is on the mat."],
    translations=["El gato está en la alfombra."],
    references=["El gato está sobre la alfombra."],
    batch_size=16,
)

As in pairwise QE scoring, mode="both" is available for both-order reference-anchored inference.

PEAR for MBR decoding

from pear.mbr import pear_utility_matrix, select_mbr_hypothesis

metric = pear.load_metric("pear")
source = "Questa è una traduzione molto buona."
hypotheses = [
    "This is a good translation.",
    "This is a very good translation.",
    "This is a bad translation.",
]

utility = pear_utility_matrix(metric, source, hypotheses, mode="half", batch_size=16)
index, expected_utility = select_mbr_hypothesis(utility)
print(hypotheses[index], expected_utility)

Use mode="full" for all off-diagonal ordered pairs, or mode="half" to score only one triangular half and fill the opposite direction by PEAR antisymmetry.

CLI examples

Pairwise TSV input must contain src, mt_0, and mt_1 columns:

pear score --model pear --input pairs.tsv --output scored.tsv --batch-size 16
pear score --hf-model Prosho/pear --input pairs.tsv --output scored.tsv --batch-size 16

Reference-anchored TSV input must contain src, mt, and ref columns:

pear score --model pear --mode reference --input refs.tsv --output scored.tsv --batch-size 16

MBR input JSONL rows must contain src and hypotheses:

pear mbr --model pear --input nbest.jsonl --output selected.jsonl --utility half --batch-size 16

Citation

If you use this model, please cite the PEAR paper:

@inproceedings{proietti-etal-2026-pear,
    title = "{PEAR}: Pairwise Evaluation for Automatic Relative Scoring in Machine Translation",
    author = "Proietti, Lorenzo  and
      Grundkiewicz, Roman  and
      Post, Matt",
    editor = "Liakata, Maria  and
      Moreira, Viviane P.  and
      Zhang, Jiajun  and
      Jurgens, David",
    booktitle = "Proceedings of the 64th Annual Meeting of the {A}ssociation for {C}omputational {L}inguistics (Volume 1: Long Papers)",
    month = jul,
    year = "2026",
    address = "San Diego, California, United States",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2026.acl-long.1953/",
    doi = "10.18653/v1/2026.acl-long.1953",
    pages = "42189--42207",
    ISBN = "979-8-89176-390-6"
}

Links

• Paper: https://aclanthology.org/2026.acl-long.1953/
• Code: https://github.com/prosho-97/pear
• Collection: https://huggingface.co/collections/Prosho/pear
• PEAR-XL: https://huggingface.co/Prosho/pear-xl