ElevenLabs
xAI
Fish Audio
InworldSee what a listener saw
This is the screen a listener works on. Play the clip, read the transcript, and answer one question for every marked span — how the words were said, and whether anything went wrong. Try it yourself: the options below are live.
1I made coffee this morning, and the kids left early, because 2the bus came at seven fifteen. Then the clinic called, and 3the doctor said Dad's test came back fine, and then 4I called Aunt May, 5she's coming in June, and we booked the dinner.
A real clip from the run — 5 style spans, 16 seconds.
About TTS Inline Steerability
A text-to-speech model is increasingly asked to do more than read a sentence. It is asked to laugh in one place, sigh in another, and drop to a whisper for six words before returning to a normal voice. Those instructions are written into the text itself as inline tags.
The question this leaderboard answers is simple: when a model is given an inline tag, does it perform that tag — the right one, in the right place, without damaging the rest of the sentence?
How it works
- A sentence is written once, in a vendor-neutral notation, carrying tags at planned
positions — a point tag such as
{laugh}, or a span such as{whisper}…{/whisper}. - An adapter rewrites each tag into the exact string that vendor documents. No model is given a substitute for a tag it does not support, and no sentence is written for a particular model.
- Every model generates the same sentences. Five different people listen to each clip, independently, and for each tag say what they actually heard — choosing from the six sounds, or nothing there, read out as words, garbled, or another sound. Nobody hears the same clip twice.
- A tag scores a point when at least three of the five listeners agreed it was right. One person’s opinion never decides a tag.
Events and styles
An event is a sound at one point in the text: laugh, chuckle, giggle, sigh, breath, cry. A style is a delivery held across a run of words: whisper, soft, loud, slow, fast. Sentences come in three sets, each ranked on its own, because a model that is good at one is routinely poor at another.
What the controls do
Four switches above the table. Each one changes which answers are counted, never how they were graded.
| Control | Setting | What it means |
|---|---|---|
| Trackwhich sentences are scored | All sets | All three sets together. |
Events | Sentences carrying six sound tags and nothing else. | |
Styles | Sentences carrying five style spans and nothing else. | |
Mixed | Sentences carrying both — six sounds and five spans in one sentence, eleven tags in all. The hardest set for every model here. | |
| Scoringhow strictly an answer is marked | Exact tag | The listener must name the exact sound. Told {chuckle}, only “chuckle” counts — “laugh” and “giggle” are wrong. |
Same family | A neighbouring sound counts. Laughter = laugh, chuckle, giggle. Exhale = sigh, breath. Cry stands alone, so its score never changes. Style spans score identically in both modes, because a whisper is not a near-miss for “fast”. Nothing there, read out as words and garbled are wrong in both. | |
| Textwhat the words around the tag are doing | All | Every sentence, both kinds below. |
Neutral | The words agree with the tag — a {chuckle} placed on something that is actually funny. | |
Fights the tag | The words pull the other way — a {chuckle} on “there is nothing funny about this”. It tests whether the model follows the instruction it was given or the mood of the sentence. Half the sentences in the run are written this way. | |
| Voice / Language | Female / Male | Which voice generated the clip. Voices were fixed before any audio was heard. |
US English | The only language in this run. |
What each column means
| Column | What it tells you |
|---|---|
| Rank1 (1–2) | The position, then the range that position could plausibly take. 1 (1–2) means the model could honestly be first or second; 5 (5–5) means its place is not in doubt. Two models sharing a rank number are statistically tied — the difference between them could be chance. |
| Tag Accuracy | The headline. Out of every individual tag the model was given, the share it performed correctly. Higher is better. |
| 95% CI | How far the score could move if the eval were run again on a fresh set of sentences. The dot is the score, the bar is the range. Where two bars overlap heavily, the gap between those models is not yet evidence of anything — which is why they share a rank. |
| Task Success Rate | A much harder test: the share of sentences in which every single tag was correct. One tag wrong anywhere and the whole sentence fails. The letter under the number says which set it covers — (E) Events, (S) Styles, (M) Mixed, or (all sets) for every sentence pooled together. Pick a set in Track to see that set on its own. The sets are not equally hard: a sentence carries 6 tags in Events, 5 in Styles and 11 in Mixed, so a Mixed sentence is far harder to get whole. |
| Tier | The plain-English version of the score. Reliable — performs the tag correctly at least five times in six, so a product can depend on it. Inconsistent — between half and five-in-six. Unreliable — below half. |
| Tags judged | How much evidence sits behind the row. More tags means a narrower confidence interval. |
| Per-tag accuracy | The same tag-accuracy figure broken down one tag at a time, so you can see which specific instructions a model handles. Which tags appear follows the Track and Scoring switches: six sounds on Events, five styles on Styles, all eleven elsewhere, or the three sound families in same-family scoring. Green is 60% or above, amber 35–60%, red below 35%. |
The eleven tags
A tag only enters this list when at least three vendors document it. Emotion styles such as angry or sad are left out of this version, because judging them needs an opinion about emotion rather than a yes-or-no about a sound.
| Tag | Type | What it asks the model to do |
|---|---|---|
{laugh} | Event | Full, voiced laughter with several pulses. |
{chuckle} | Event | A short, soft, mostly closed-mouth laugh. |
{giggle} | Event | A light, quick, higher-pitched laugh. |
{sigh} | Event | A long audible exhale carrying emotion — tired, relieved, resigned. |
{breath} | Event | An audible breath: catching your breath, or a breath before going on. |
{cry} | Event | A short burst of weeping. |
{whisper}…{/whisper} | Style | An unvoiced whisper across the span. |
{soft}…{/soft} | Style | Quiet but still voiced. |
{loud}…{/loud} | Style | Raised volume, not a scream. |
{slow}…{/slow} | Style | A noticeably slower speaking rate. |
{fast}…{/fast} | Style | A noticeably faster speaking rate. |
A model that does not document a tag is never sent its sentences and never given a substitute, so it is not ranked on a set where it is missing tags. The two Gemini models document all six event sounds as inline tags, but document no way to hold a style across a chosen run of words: Google’s documented method is a style setting that covers a whole turn, and its own guidance for a change mid-sentence is to split the script into separate turns. Our style spans sit inside one continuous sentence, so these models appear on the Events set alone.
The hardest tags
Accuracy on each tag, averaged over every model that was sent it. The dashed line on the event bars marks 10%: what a listener would score by guessing, since the event question offers ten options.
Sounds, ranked scoring
Styles, ranked scoring
Sounds, exact (not ranked)
Cry is the broken one at 25% even under the ranked measure, and it is the only sound with no family to fall back on. The right-hand column shows what happens when the exact word is demanded instead of the family: every sound collapses, which is the disagreement between listeners showing up as a score.
Results
Model-and-tag pairs now good enough to use
29 of 78
One listener and exact-wording scoring put this at 1 of 78. Five listeners and family scoring put it at 29. 4 models reach the Reliable tier on the Events set.
The top two cannot be separated
74.0% vs 73.3%
Inworld TTS-2 and Eleven v4 are statistically tied for first on tag accuracy across all five rounds (McNemar, p ≥ 0.05). They are not tied on the tracks: Inworld leads Styles, Eleven v4 leads Events.
The two leaders are not interchangeable
99% vs 24%
Eleven v4 performs whisper at 99% and Inworld at 24%. On slow it is the other way round, 27% against 92%. Which one is right for you depends on which tags you send.
Exact wording still cannot be ranked
0.65
Five listeners agree on which exact sound they heard only 0.65 of the way to perfect agreement, still below the 0.667 needed. Laughter can be measured; which kind of laughter cannot.
Tag accuracy by set
The share of individual tags performed correctly.
| Model | Events | Styles | Mixed | All sets |
|---|---|---|---|---|
| Inworld TTS-2 | 78.5% | 69.2% | 73.7% | 74.0% |
| Eleven v4 | 90.8% | 53.8% | 72.5% | 73.3% |
| Gemini 3.8 Flash TTS | 59.7% | – | – | 59.7% |
| Fish Drama 3 | 84.0% | 31.4% | 58.6% | 59.4% |
| Gemini 3.8 Flash-Lite TTS | 59.2% | – | – | 59.2% |
| Grok TTS | 87.8% | 17.8% | 49.7% | 52.9% |
| Resemble Ultra | 85.0% | 18.0% | 48.3% | 51.4% |
| Fish S2.1 Pro | 35.0% | 0.8% | 17.2% | 18.3% |
Task Success Rate by set
The share of sentences where every tag was correct. Reported per set and never pooled, because a sentence carries 6 tags in Events, 5 in Styles and 11 in Mixed.
| Model | Events | Styles | Mixed |
|---|---|---|---|
| Inworld TTS-2 | 2.0% | 9.0% | 0.0% |
| Eleven v4 | 51.0% | 5.0% | 2.0% |
| Gemini 3.8 Flash TTS | 0.0% | – | – |
| Fish Drama 3 | 33.0% | 0.0% | 0.0% |
| Gemini 3.8 Flash-Lite TTS | 0.0% | – | – |
| Grok TTS | 43.0% | 0.0% | 0.0% |
| Resemble Ultra | 26.0% | 0.0% | 0.0% |
| Fish S2.1 Pro | 0.0% | 0.0% | 0.0% |
Supporting measures
None of these affect the ranking.
| Model | Clean speech | Words match | Score | Tags with all five rounds |
|---|---|---|---|---|
| Inworld TTS-2 | 97.3% | 99.3% | 74.0% | 94% |
| Eleven v4 | 96.6% | 100.0% | 73.3% | 97% |
| Gemini 3.8 Flash TTS | 90.9% | 97.0% | 59.7% | 98% |
| Fish Drama 3 | 98.7% | 100.0% | 59.4% | 95% |
| Gemini 3.8 Flash-Lite TTS | 96.0% | 100.0% | 59.2% | 99% |
| Grok TTS | 99.3% | 98.0% | 52.9% | 94% |
| Resemble Ultra | 97.7% | 98.3% | 51.4% | 95% |
| Fish S2.1 Pro | 96.3% | 99.3% | 18.3% | 92% |
Appendix
Showing: US English · All setsEnglish only. Every clip generated once per model from the same script, with voices fixed before any audio was heard.
71,172 listener judgements on 14,400 tags over 2,000 sentences by 250 listeners over 5 rounds · job 1635 · last updated 2026-11-06.