mlboydaisuke/Kokoro-82M-ExecuTorch

Kokoro-82M — ExecuTorch (text to speech, 54 voices)

Phonemes in, a waveform out, in one .pte with two methods. StyleTTS2-shaped: a 12-layer phoneme BERT and a duration predictor decide how long each sound lasts, and an iSTFTNet vocoder turns the stretched features into 24 kHz audio. A single 256-dimensional style vector picks the voice and colours the prosody.

predict  input_ids (1, N) int64, ref_s (1, 256) fp32, speed (1) fp32
             ->  d (1, 640, N), t_en (1, 512, N), duration (N) int64
vocode   d, t_en, aln (1, N, F) fp32, ref_s
             ->  waveform (F * 600) fp32 @ 24 kHz

• File: kokoro_82m_xnnpack_fp32.pte — 325.4 MB, two methods
• Source: hexgrad/Kokoro-82M — 81.8M parameters
• License: apache-2.0
• Voices: 54, shipped separately as voices/*.pt in the source repo
• Languages: the graph is language-blind — it takes phoneme ids. Verified here in English and Japanese; the voice pack also carries Spanish, French, Hindi, Italian, Portuguese and Chinese voices, which are not verified on this card.

Both axes — phonemes and frames — are dynamic, and both are exact. Nothing is padded, nothing is stretched, and there is no ladder of fixed-size methods. That took some doing; see below.

Running it

1. Text to phonemes, outside the graph. Kokoro's vocabulary is IPA, and getting there is misaki (or espeak-ng) — a lexicon and a G2P model, not arithmetic. Same line this shelf takes with E5's prefix and Whisper's mel: the recipe is here, the graph takes what the recipe produces.

from misaki import en
ps, _ = en.G2P(trf=False, british=False)(text)
ids = [vocab[c] for c in ps if c in vocab]          # config.json's 178-entry vocab
input_ids = torch.LongTensor([[0, *ids, 0]])        # wrapped in the boundary token

For Japanese it is misaki.ja.JAG2P() and a j* voice, and nothing else changes — the ids go into the same graph. Install misaki[ja] with unidic-lite: unidic's dictionary is a separate 250 MB download, and fugashi prefers unidic when both are present, which leaves you with an empty dicdir and an error about mecabrc.

2. Pick the style row by phoneme count. The voice pack is 510 rows and the row is pack[len(ps) - 1] — indexed by the length of the phoneme string, which is what upstream's own pipeline does, and not by the number of ids, which is smaller whenever the string holds a character the vocabulary does not have.

3. Run predict, which returns the features and one duration per phoneme.

4. Build the alignment matrix yourself, at exactly sum(duration) frames. Upstream builds it with repeat_interleave, whose width is the sum of the durations — the model's own output deciding the shape of its next input, which no single graph can express. That is the only reason this is two methods rather than one. The same matrix is comparisons only:

ends = torch.cumsum(duration, 0)
starts = ends - duration
frame = torch.arange(int(ends[-1]))                 # exactly sum(duration)
aln = ((frame[None, :] >= starts[:, None]) &
       (frame[None, :] < ends[:, None])).float()[None]

Give it exactly sum(duration) frames. Not more — see the next section.

5. Run vocode. Out comes 600 * F samples at 24 kHz. speed above 1 speaks faster; it divides the durations before they are rounded.

Do not pad either axis

Both axes are dynamic, so there is no window to pad into — but it is worth saying why the file is built that way, because the obvious fixed-window design does not work here and the damage does not show up in a transcript.

axis	what forbids padding	measured
phonemes	five bidirectional LSTMs — state flows in from the padding	speaking rate moves up to 19%
frames	a bidirectional LSTM and AdaIN1d, which is InstanceNorm over time	log-mel 0.18–0.86 against a 0.04 noise floor

The frame axis is the surprising one. AdaIN1d normalises over time, so one extra frame changes the statistics the entire signal is divided by. Padding to the next 16-frame rung, appending 256 frames, and padding out to 1024 all land far outside what the model does to itself, and it is not a level change — taking out one global gain factor leaves the distance where it was.

Padding with spaces rather than zeros roughly halves the damage on the phoneme axis, and a recogniser transcribes every padded arm correctly. That is exactly why the gate here is not a recogniser alone.

The LSTMs are rolled, not unrolled

nn.LSTM will not export with a dynamic sequence axis: torch.export pins it to whatever it was traced at. The reason is that to_edge unrolls the recurrence — this model's predict graph is 1238 ATen nodes at any length, and 1651 + 108 per phoneme in edge dialect.

That makes a ladder of fixed-length methods look like the only option, and then makes the ladder impossible. The XNNPACK partitioner is superlinear in node count and cuts an unrolled LSTM into hundreds of tiny delegates — 383 partitions at 32 phonemes — so lowering one method costs:

phonemes	edge nodes	lowering
8	1651	33 s
16	2515	63 s
32	4243	153 s
128	14611	~19 min, extrapolated

A rung per phoneme count from 8 to 128 is upwards of 16 hours, and the frame axis would need its own ladder on top of that.

A scan higher-order op keeps the loop rolled. ExecuTorch lowers it, the runtime runs it, and the sequence axis stays dynamic. On this model's own LSTM shape — 640 in, 256 hidden each way, 128 steps:

	build	edge nodes	delegates	128 steps
nn.LSTM, one fixed length	59.2 s	3366	131	5.72 ms
rolled scan, any length	3.1 s	49	3	16.65 ms

Three times the runtime for one LSTM, against a build that finishes and a file that takes any length. Kokoro has six of them. The whole file now builds in about two minutes.

Verification

Ten sentences through misaki — five English on af_heart, five Japanese on jf_alpha — each one synthesised by the .pte and by the unmodified upstream model in eager. Three gates, because no single one works here:

Durations must match exactly — they are integers, they decide the rhythm, and they come out of the half of the model that has no noise in it. 10 of 10 exact, both languages.

Waveform correlation is not usable. The vocoder's excitation carries Gaussian noise and a random initial phase, so the eager model does not reproduce itself: two runs of the same input correlate 0.9948. A correlation gate here measures the noise.

Log-mel distance against the model's own floor is the gate that works. Measure the distance between two eager runs, then between eager and the .pte, and ask whether the second is the first:

	log-mel vs eager	eager's own floor	ratio
worst of ten	0.0474	0.0455	1.04x
best of ten	0.0393	0.0408	0.96x

Below and above 1.0 across the five, which is what "indistinguishable from running it again" looks like — and it moves a few percent between runs, because the noise source is in both arms. For scale, one extra frame of padding shows up at 4.4x, and the fp16 build below at 73x.

Transcripts, through Qwen3-ASR, scored against eager's own transcript rather than against the sentence — what is under test is the conversion, and a recogniser choosing a different kanji is not the file's doing. CER 0.0000 on nine of ten.

The tenth is worth writing down, because it is the recogniser and not the model:

この電車は東京駅に止まりますか   (does this train stop at Tokyo Station)
eager  ...東京駅に泊まりますか    (stay overnight)      CER 0.067
pte    ...東京駅に停まりますか    (stop)

Same reading, different kanji — and eager disagrees with itself here: four runs of the identical input gave 泊 once and 停 three times, and the .pte did the same. The vocoder's noise is enough to tip a near-tie in the recogniser. Durations are identical and log-mel is 1.02x the floor on this clip, so the two files are as close as eager is to itself. An ASR-only gate would have recorded this as a defect.

Speed

3.25 s of audio (50 phonemes, 130 frames) on an M-series laptop, host CPU:

	ms
predict	56.6
vocode	441.4
total	498.0 — 6.5x faster than real time
the same utterance in eager PyTorch	225.5

Measured on a machine that was busy, so read these as a floor rather than a number to quote. Eager is faster here and that is expected: it reaches Accelerate's LSTM and convolution kernels, while about half the graph runs on portable kernels (predict 49.6% of ops delegated, vocode 54.8%). No device numbers yet.

What is not in this file

fp16 was built and withdrawn. It fails three ways at once, and the first one is not about ExecuTorch at all:

	fp32 (this file)	fp16 (withdrawn)
size	325.4 MB	278.7 MB — a 14% saving, not the 50% the weights imply
worst log-mel / noise floor	0.99x	73x
worst CER	0.0000	1.0000 — nothing intelligible
3.25 s of audio	498 ms	2104 ms

Halving this model breaks it in eager PyTorch, before any export. At 96 phonemes the vocoder returns nan; at 24 it survives but sits at 2.8x the noise floor. Keeping the 73 data-statistic norm layers in fp32 — the usual fix for InstanceNorm overflow — does not save it, so the overflow is not only in the norms. One duration in 96 also flips, which is a rounding tie rather than a numerical failure.

The size is the least of it but worth knowing: the weights do halve to 163.4 MB, and the file is still 278.7 MB, because the XNNPACK delegate carries its own copy of the weights it takes.

No int8 build. Measured, not assumed: dynamic int8 quantises nn.Linear only, and Kokoro is 66.9% Conv1d with just 15.9% of its weights in Linear layers, so it would touch a sixth of the file. Static int8 would reach the convolutions, but a vocoder's quality under quantisation has to be measured per task rather than declared, and that has not been done here.

The vocoder is partitioned without XNNPACK's PermuteConfig. A permute inside the decoder feeds two consumers outside its partition, and the delegate's output list then carries that one node twice, which XNNPACK rejects with "Output node ... is already in the inputs ... pass through arguments". It is the partition boundary that is wrong, not the graph: the same graph lowers the moment permutes are not eligible to be one.