lfm2.5-350M-datause-prwp

LoRA SFT of LiquidAI/LFM2.5-350M for data-mention extraction: emit the data-bearing phrases in a text as compact JSON (data_mentions with data_mention/specificity_type).

Training

• base model: LiquidAI/LFM2.5-350M
• dataset: rafmacalaba/data-use-mention-sft
• epochs: 3
• learning rate: 0.0002
• LoRA: r=16 alpha=32 dropout=0.05
• completion-only masking (loss on assistant JSON turn)

Evaluation (holdout, n=9079)

Jaccard entity-level matching: acronym-aware span clustering + Hungarian optimal bipartite match (match thr=0.5), F0.5 primary. Aligned with rafmacalaba/gliner_datause_extended. The holdout is rafmacalaba/data-use-mention-sft, so numbers are not directly comparable to the GLiNER model's data-use-mentions-extended holdout.

label	tp	fp	fn	precision	recall	f0.5	f1
overall	9888	2537	2414	0.7958	0.8038	0.7974	0.7998

Per-label

label	tp	fp	fn	precision	recall	f0.5	f1
descriptive	4034	1930	1762	0.6764	0.6960	0.6802	0.6861
named	4183	1281	1175	0.7656	0.7807	0.7685	0.7731
vague	488	539	662	0.4752	0.4243	0.4641	0.4483

Per-corpus

label	tp	fp	fn	precision	recall	f0.5	f1
prwp	9888	2537	2414	0.7958	0.8038	0.7974	0.7998

Per-origin

label	tp	fp	fn	precision	recall	f0.5	f1
general_prwp	9888	2537	2414	0.7958	0.8038	0.7974	0.7998

Sample predictions (holdout)

gold	predicted
{"data_mentions":[{"data_mention":"administrative data","specificity_type":"descriptive"}]}	{"data_mentions":[{"data_mention":"administrative data","specificity_type":"descriptive"}]}
{"data_mentions":[{"data_mention":"World Development Indicators","specificity_type":"named"},{"data_mention":"CPHS","spe	{"data_mentions":[{"data_mention":"World Development Indicators","specificity_type":"named"},{"data_mention":"CPHS","spe
{"data_mentions":[{"data_mention":"ES data","specificity_type":"named"}]}	{"data_mentions":[{"data_mention":"ES data","specificity_type":"named"}]}
{"data_mentions":[]}	{"data_mentions":[{"data_mention":"EUROMOD I4.0+","specificity_type":"named"}]}
{"data_mentions":[{"data_mention":"World Development Indicators","specificity_type":"named"},{"data_mention":"IMF World	{"data_mentions":[{"data_mention":"World Development Indicators","specificity_type":"named"},{"data_mention":"IMF World
{"data_mentions":[]}	{"data_mentions":[]}
{"data_mentions":[{"data_mention":"UNESCO-provided data","specificity_type":"descriptive"}]}	{"data_mentions":[{"data_mention":"PIRLS","specificity_type":"named"},{"data_mention":"UNESCO-provided data","specificit
{"data_mentions":[{"data_mention":"IHPS dataset","specificity_type":"named"},{"data_mention":"IHS4 data","specificity_ty	{"data_mentions":[{"data_mention":"IHS4 dataset","specificity_type":"named"},{"data_mention":"IHS4 data","specificity_ty
{"data_mentions":[{"data_mention":"HFSSS","specificity_type":"named"}]}	{"data_mentions":[{"data_mention":"HFSSS","specificity_type":"named"}]}
{"data_mentions":[{"data_mention":"HFCS data","specificity_type":"named"}]}	{"data_mentions":[{"data_mention":"HFCS data","specificity_type":"named"}]}
{"data_mentions":[{"data_mention":"data on patents and publications","specificity_type":"descriptive"}]}	{"data_mentions":[{"data_mention":"data on patents and publications","specificity_type":"descriptive"}]}
{"data_mentions":[{"data_mention":"UN COMTRADE Statistics","specificity_type":"named"}]}	{"data_mentions":[{"data_mention":"UN COMTRADE Statistics","specificity_type":"named"}]}