espnet/geolid_combined_shared_trainable

ESPnet2 GeoLID: combined_shared_trainable

This is a normal ESPnet language identification model using the stock S3prlFrontend and ESPnetLIDModel (model: espnet) with optional lang2vec conditioning. GeoLID wraps SSL layers 32, 36, 40 and 44 with lang2vec prediction and residual conditioning. The model code is maintained in ESPnet; no checkpoint-local Python modeling files or trust_remote_code=True are needed.

The checkpoint was converted, not retrained. local/convert_checkpoint.py renamed the original parameter keys and removed checked duplicate aliases. All learned tensor values are unchanged. The auxiliary linear head now belongs to the model; classification uses the standard aamsoftmax_sc_topk loss. Runtime uses current ESPnet pooling and normalization, without legacy compatibility code. Measured accuracy of this converted checkpoint is reported separately from historical paper results below.

Checkpoint provenance and evaluation

All combined-model paper results—the six test-set accuracies and Macro Avg.—were obtained with the earlier epoch-24 checkpoint (48,000 training batches). The downloadable weights are the later epoch-31 checkpoint (62,000 batches). Conversion preserved the learned tensor values.

The six archived evaluations started on June 2, 2025, between 00:34 and 02:37 CDT and all finished by 02:55. The first training log records the epoch-24 best update at 20:01 on June 1; the resumed log records the epoch-31 best update only at 11:22 on June 2. The released weights have 41 BatchNorm counters equal to 62,000. Thus every historical combined column, including Macro Avg., predates the released checkpoint. This corrects the paper's attribution of those results to the 62k checkpoint.

The epoch-24 weights have not been recovered and the old inference logs contain no weight hash. The epoch attribution follows the archived training/evaluation chronology; the historical scores have not been independently rerun.

Checkpoint / result source	VoxLingua107 dev	Babel	FLEURS	ML-SUPERB 2.0 dev	ML-SUPERB 2.0 dialect	VoxPopuli	Macro Avg.
Historical paper — epoch 24 (48k)	94.4	95.4	97.7	88.6	86.8	99.0	93.7
Released converted — epoch 31 (62k), batch size 1	93.29	—	97.51	88.69	85.26	98.98	—

Babel remains unevaluated: the licensed dev audio and original complete manifest are unavailable in this evaluation environment. Its released-model cell and the six-set Macro Avg. are left blank. No paper score is substituted, and no five-set average is presented as the six-set Macro Avg.

Each result is utterance-level accuracy on the complete listed split. Macro Avg. is the unweighted mean of the six dataset accuracies, calculated from integer correct/total counts before rounding. Historical paper values are retained as reported.

The new runs use the unmodified espnet2.bin.lid_inference CLI at ESPnet revision 07dc3256fbe6ab542a260314f9e3d9a0bdc9de9a: NVIDIA H200, GPU float32, batch size 1 per forward pass, 8 workers per inference process, full-length 16 kHz mono waveforms, --use_preprocessor false, --fix_duration false, and inputs sorted by utterance ID. FLEURS uses the CLI's standard multiprocessing mode across four GPUs; the other sets use one GPU. Each process still classifies one utterance per forward pass. PyTorch 2.11.0+cu128, Transformers 5.14.1 and espnet-s3prl 2026.5.14 match the earlier VoxLingua107 run.

VoxPopuli uses its official test partition (16,991 utterances), not its 18,636-utterance validation partition. The archived report's 178 error IDs, reference labels and every per-language accuracy match this test manifest. FLEURS uses the full 77,810-utterance official test set. ML-SUPERB 2.0 uses the shared recipe's language normalization and excludes its 283 Norwegian dev rows, leaving 23,609 dev and 7,095 dialect-dev utterances; these match the archived error IDs and per-language rates. No further model-inventory filtering or audio truncation is applied.

For FLEURS, all 4,000 predictions saved by the initial single-GPU run exactly match the four-process run. This checks the parallel scheduling change while keeping batch size 1.

New evaluation report, reproduction command and per-set evidence provide complete predictions, reference labels, inference logs and metadata for each completed test set.

The published weights' earlier legacy-reference accuracy on VoxLingua107 dev was 93.35% (batch size 4). This evaluates the released epoch-31 weights through the legacy implementation; it is separate from the epoch-24 paper result. Runtime and batching differ from the current measurements.

The previous VoxLingua107 evaluation, metadata, predictions and log retain their original revision 301372e4da311d2a3d9574a5a43b3037ae6e67c1. The new VoxLingua107 run again produced 1501/1609 correct, with all 1,609 predictions unchanged. Earlier loading and compatibility checks remain at their original revisions in validation.json.

Inference batch size matters: S3PRL frame alignment and ECAPA's temporal convolutions and unmasked SE averaging can change features before masked normalization/final pooling. All new results here use batch size 1.

Installation

Install the upstream ESPnet revision containing the merged GeoLID recipe:

pip install 'espnet @ git+https://github.com/espnet/espnet.git@c9fde840770adc76100b781044abe160e38124bd'
pip install espnet-s3prl transformers espnet-model-zoo soundfile

S3PRL's hf_wav2vec2_custom upstream imports Transformers; espnet-s3prl does not install it automatically. With an ESPnet tools checkout, the equivalent installers are make s3prl.done transformers.done in tools/. The lang2vec package is used only by recipe target preparation and is not needed for inference.

Inference

import soundfile as sf
import torch
from espnet_model_zoo.downloader import ModelDownloader
from espnet2.tasks.lid import LIDTask

files = ModelDownloader().download_and_unpack("espnet/geolid_combined_shared_trainable")
model, args = LIDTask.build_model_from_file(
    files["train_config"], files["model_file"], device="cpu"
)
model.eval()
audio, sample_rate = sf.read("speech.wav", dtype="float32")
assert sample_rate == 16000 and audio.ndim == 1
with torch.inference_mode():
    embeddings, prediction = model(
        torch.from_numpy(audio)[None],
        torch.tensor([len(audio)]),
        extract_embd=True,
    )
with open(args.lang2utt) as stream:
    languages = [line.split()[0] for line in stream]
print(languages[prediction.item()])

The backbone is initialized through the stock hf_wav2vec2_custom upstream from facebook/mms-1b, then the complete converted checkpoint loads the trained SSL, conditioning and classifier parameters. This repository contains 157 language classes. languages.txt preserves classifier-index order recovered from the original model card and the original sorted recipe label construction; it is not a copy of the missing original training utterance manifest.

conversion.json records the source checkpoint revision and label provenance. validation.json preserves the historical strict-loading and layout-migration checks, and separately records the complete evaluation after model integration. evaluation.json and evaluation_predictions.jsonl retain the earlier VoxLingua107 evaluation. The new measurements and complete per-set evidence are under evaluations/20261027/. The metadata targets are used only for training; inference uses speech and predicts its own conditioning vectors. The old training logs and figures retained under exp_* describe the original training run. Earlier checkpoint layouts remain available in repository history.

License

Derived from facebook/mms-1b, these model weights are licensed CC-BY-NC-4.0, including the non-commercial restriction. The previous CC-BY-4.0 metadata was incorrect and has been corrected.

Citation

@inproceedings{wang2025geolid,
  title={Geolocation-Aware Robust Spoken Language Identification},
  author={Qingzheng Wang and Hye-jin Shim and Jiancheng Sun and Shinji Watanabe},
  booktitle={2025 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU)},
  year={2025},
  organization={IEEE},
  url={https://arxiv.org/abs/2508.17148}
}

Recipe and evaluation results