README.md

metadata

language: en
library_name: transformers
pipeline_tag: automatic-speech-recognition
tags:
  - automatic-speech-recognition
  - speech-to-text
  - asr
  - speech
  - english
  - qwen3
  - audio
  - reinforcement-learning
datasets:
  - openslr/librispeech_asr
  - speechcolab/gigaspeech
  - mozilla-foundation/common_voice_17_0
  - facebook/voxpopuli
  - LIUM/tedlium
  - edinburghcstr/ami
  - anton-l/earnings22
  - kensho/spgispeech
metrics:
  - wer
model-index:
  - name: Musci-ASR-2.4B
    results:
      - task:
          type: automatic-speech-recognition
        dataset:
          name: Open ASR Leaderboard
          type: hf-audio/esb-datasets-test-only-sorted
        metrics:
          - type: wer
            value: 5.44
            name: Average WER
license: apache-2.0

Musci-ASR-2.4B

Musci-ASR-2.4B is an English speech-to-text model that pairs a Qwen3-1.7B-base language-model backbone with a Qwen3-Omni-MoE audio encoder. A gated-MLP adapter projects audio features into the language-model embedding space. The model is trained on public English ASR corpora and fine-tuned with reinforcement learning on the Open ASR Leaderboard training splits.

The model has approximately 2.4B parameters and is distributed as a single bfloat16 safetensors shard of approximately 4.84 GB.

Model Details

• Developed by: Musci Research
• Model type: Automatic Speech Recognition / speech-to-text model
• Language: English
• License: Apache-2.0
• Library: Transformers
• Backbone: Qwen3-1.7B-base, 28 layers, hidden size 2048
• Audio encoder: Qwen3-Omni-MoE audio encoder
• Adapter: Gated-MLP adapter, hidden size 8192
• Parameter size: approximately 2.4B
• Checkpoint format: bfloat16 safetensors

Intended Use

This model is intended for English automatic speech recognition, including transcription of English speech audio for research and evaluation purposes.

Inference

import librosa
import torch
from huggingface_hub import hf_hub_download
from transformers import AutoModelForCausalLM, AutoTokenizer
from transformers.dynamic_module_utils import get_class_from_dynamic_module

REPO = "Musci-research/Musci-ASR-2.4B"
DEVICE = "cuda:0"

model = AutoModelForCausalLM.from_pretrained(
    REPO, torch_dtype=torch.bfloat16, trust_remote_code=True
).to(DEVICE).eval()
tokenizer = AutoTokenizer.from_pretrained(REPO, trust_remote_code=True)

MusciProcessor = get_class_from_dynamic_module("processing_Musci.MusciProcessor", REPO)
MelConfig = get_class_from_dynamic_module("processing_Musci.MelConfig", REPO)

mel_cfg = MelConfig(
    mel_sr=16000,
    mel_dim=128,
    mel_n_fft=400,
    mel_hop_length=160,
)
processor = MusciProcessor(tokenizer, config=mel_cfg, enable_time_marker=False)
processor.load_template(hf_hub_download(REPO, "chat_template_default.py"))

waveform, _ = librosa.load("your_audio.wav", sr=16000)
inputs = processor(audio=waveform, return_tensors="pt").to(DEVICE)
inputs["audio_data"] = inputs["audio_data"].to(model.dtype)

with torch.no_grad():
    out_ids = model.generate(
        **inputs,
        max_new_tokens=512,
        do_sample=False,
        num_beams=1,
        use_cache=True,
        eos_token_id=[processor.end_token_id],
    )

new_ids = out_ids[:, inputs["input_ids"].shape[1]:]
transcript = processor.batch_decode(new_ids, skip_special_tokens=True)[0].strip()
print(transcript)

Audio Frontend

• Sample rate: 16 kHz
• Features: Whisper log-mel filterbank
• Mel bins: 128
• FFT size: 400
• Hop length: 160

Training

The model was trained on public English ASR corpora and fine-tuned with reinforcement learning on the Open ASR Leaderboard training splits.

Limitations

The model is designed for English ASR. It may perform worse on non-English speech, heavy accents, noisy recordings, overlapping speakers, far-field audio, domain-specific terminology, or audio conditions that differ significantly from the training and evaluation data. The output should be manually reviewed before use in high-stakes settings.

Citation

@misc{musci_asr_2025,
  title        = {{Musci-ASR-2.4B}},
  author       = {{Musci Research}},
  year         = {2025},
  howpublished = {\url{https://huggingface.co/Musci-research/Musci-ASR-2.4B}}
}

License

This model is released under the Apache-2.0 license.