Whisper vs Qwen3-ASR vs R2T2 — WebGPU benchmark

Speech recognition and translation models, measured

Speech recognition and translation models that run in the browser, measured on one graphics card and one CPU: error rate, speed, download size and licence.

A reference page. Downloaded models run in your browser. Gemini Nano and Chrome Translator stay on this device. Chrome Speech Recognition's default mode sends audio to Google's servers.

Measured by ByteScope, 2026-09-27: FLEURS test split, 150 utterances per language, file mode. GPU: GTX 1060 6GB without shader-f16. CPU: WebAssembly build, 5 cores. Chrome 154.0.8037.57.

Speech recognition with a graphics card (WebGPU)

WebGPU build with fp32 activations — the build a GPU without shader-f16 gets.

Model	Download	English WER	Chinese CER	Chinese CER, digits ignored	Japanese CER	RTF en / zh / ja
Whisper large (v3-turbo)	767.5 MB	4.8%	7.5%	7.2%	5.6%	0.386 / 0.315 / 0.329
Confucius4-R2T2	2,183.5 MB	6.1%	8.2%	5.9%	6.9%	0.189 / 0.153 / 0.181
Qwen3-ASR 1.7B	2,183.5 MB	6.9%	7.7%	5.9%	7.4%	0.193 / 0.154 / 0.190
Qwen3-ASR 0.6B	848.9 MB	8.1%	8.0%	6.3%	9.8%	0.102 / 0.083 / 0.099
Whisper small	594.4 MB	6.8%	11.7%	11.5%	13.4%	0.207 / 0.227 / 0.220
Whisper base	214.5 MB	10.5%	20.7%	20.7%	27.2%	0.133 / 0.160 / 0.163

Error rates in %, lower is better. RTF = processing time ÷ audio length; below 1 is faster than real time.

GPU RTFs were measured while a CPU batch ran alongside and are upper bounds. Whisper large-v3-turbo measured again on the same graphics card after the CPU batch finished: 0.260 / 0.263 / 0.241.

Speech recognition without a graphics card (CPU)

WebAssembly build on 5 CPU cores.

Model	Utterances	Download	English WER	Chinese CER	Japanese CER	RTF en / zh / ja
Qwen3-ASR 1.7B	30	2,183.5 MB	6.5%	5.7%	6.5%	1.143 / 1.074 / 1.169
Qwen3-ASR 0.6B	150	848.9 MB	8.2%	7.5%	9.2%	0.543 / 0.456 / 0.523
Whisper small	150	418.1 MB	6.8%	11.6%	13.4%	1.601 / 1.333 / 1.166
Whisper base	150	103.4 MB	10.6%	19.5%	26.0%	0.382 / 0.339 / 0.328

Error rates in %, lower is better. RTF = processing time ÷ audio length; below 1 is faster than real time.

Qwen3-ASR 1.7B ran only the first 30 utterances per language on the CPU; on those, its English and Chinese error counts matched its GPU run exactly.

Translation — blind judging

Hy-MT2 1.8B vs Chrome Translator: 280 FLORES / Tatoeba lines, 5 directions, blind MQM judging, 2026-09-27.

Hy-MT2 1.8B preferred	Chrome Translator preferred	Tie	Total
139	66	75	280

Major errors: Hy-MT2 42, Chrome Translator 61.

Hy-MT2 1.8B vs Gemini Nano: 559 Tatoeba / FLORES lines, 6 directions between en, zh and ja, blind, 2026-09-28.

Direction	Hy-MT2 1.8B preferred	Gemini Nano preferred	Tie	Nano returned nothing (counted in Hy-MT2 wins)
ja→en	22	18	59	0
ja→zh-Hant	44	14	42	9
zh→en	27	5	68	0
zh→ja	45	5	32	3
en→zh-Hant	26	12	40	0
en→ja	36	9	55	0
Total	200	63	296	12

Tie rule: a difference that is only mainland-versus-Taiwan wording, a proper-noun transliteration or simplified characters counts as a tie.

Firefox Translations (Bergamot), Chrome Translator and Hy-MT2 1.8B on the same 300 lines, 6 directions: one blind judge scored every translation 0–4, 2026-09-28. Mean score, higher is better.

Direction	Firefox Translations (Bergamot), mean	Chrome Translator, mean	Hy-MT2 1.8B, mean	Firefox Translations (Bergamot) vs Chrome Translator: lines better / worse
en→zh-Hant	2.94	3.26	3.52	7 / 18
zh→en	3.46	3.08	3.64	18 / 7
ja→zh-Hant	2.96	2.82	3.40	17 / 14
en→ja	3.14	3.10	—	14 / 11
ja→en	3.48	3.28	—	14 / 8
zh→ja	3.12	2.86	—	18 / 9

— = Hy-MT2 1.8B was not run in that direction. One judge, 50 lines per direction. A difference that is only mainland-versus-Taiwan wording, a transliteration or simplified characters was not marked down.

The models

From the model registry the tools load from. Names are the models' own.

Model	Task	Download	Runs on	Languages	Licence
Confucius4-R2T2	Speech recognition	2,183.5 MB	WebGPU, or the CPU (WebAssembly)	Full: zh, en Partial: fr, de, it, ja, ko, pt, ru, es, ar	NetEase Youdao Model Use License Agreement Licence notice: Any modifications made to the original model in this Derivative Work are not endorsed, warranted, or guaranteed by the original right-holder of the original model, and the original right-holder disclaims all liability related to this Derivative Work.
Qwen3-ASR 1.7B	Speech recognition	2,183.5 MB	WebGPU, or the CPU (WebAssembly)	Full: zh, en other languages partial	Apache-2.0
Qwen3-ASR 0.6B	Speech recognition	848.9 MB	WebGPU, or the CPU (WebAssembly)	Full: zh, en other languages partial	Apache-2.0
Whisper large (v3-turbo)	Speech recognition	767.5 MB (WebGPU) 571.9 MB (WebGPU + shader-f16, CPU)	WebGPU, or the CPU (WebAssembly)	Full: en, zh, ja, ko, es, fr, de, it, pt, ru other languages partial	MIT
Whisper small	Speech recognition	594.4 MB (WebGPU) 418.1 MB (WebGPU + shader-f16, CPU)	WebGPU, or the CPU (WebAssembly)	Full: en, zh, ja, ko, es, fr, de, it, pt, ru other languages partial	MIT
Whisper base	Speech recognition	214.5 MB (WebGPU) 103.4 MB (WebGPU + shader-f16, CPU)	WebGPU, or the CPU (WebAssembly)	Full: en, zh, ja, ko, es, fr, de, it, pt, ru other languages partial	MIT
Chrome Speech Recognition Default mode sends audio to Google's servers.	Speech recognition	Nothing from ByteScope — Chrome manages its own model	Built into Chrome	Depends on Chrome's language packs	Google Chrome
Gemini Nano	Speech recognition	Nothing from ByteScope — Chrome manages its own model	Built into Chrome	Full: de, en, es, fr, ja Partial: zh other languages partial	Google Generative AI Prohibited Use Policy
Hy-MT2 1.8B	Translation	1,300.6 MB	WebGPU only	Full: zh, yue, en, ja, ko, fr, de, es, pt, ru, it, ar, th, vi, tr other languages partial	Apache-2.0
Chrome Translator	Translation	Nothing from ByteScope — Chrome manages its own model	Built into Chrome	Depends on Chrome's language packs	Google Chrome
Firefox Translations (Bergamot)	Translation	365.3 MB	The CPU (WebAssembly) only	Full: en, zh, zh-Hant, ja	MPL-2.0
Gemini Nano	Translation	Nothing from ByteScope — Chrome manages its own model	Built into Chrome	Full: de, en, es, fr, ja Partial: zh other languages partial	Google Generative AI Prohibited Use Policy
Qwen3-ForcedAligner 0.6B	Word timing	1,847.6 MB	WebGPU, or the CPU (WebAssembly)	Full: zh, en, ja Partial: yue, fr, de, it, pt, ru, es	Apache-2.0

Vendor claims (not comparable)

Figures published by the model makers, on their own test sets and hardware. Not measured by ByteScope.

Model	Metric	Value	Test set	Condition
Confucius4-R2T2	WER	2.13% vs WhisperRT 3.71%	LibriSpeech test-clean	160 ms streaming chunks source
Confucius4-R2T2	CER	5.87% vs Qwen3-ASR-1.7B (streaming) 8.81%	WenetSpeech test-net	160 ms streaming chunks source
Qwen3-ASR 1.7B	CER	2.71% vs Whisper large-v3 5.06%	AISHELL-2 test	offline source
Qwen3-ASR 1.7B	WER	1.63% vs Whisper large-v3 1.51%	LibriSpeech test-clean	offline source
Qwen3-ASR 1.7B	RTF	0.215	41 s Japanese clip	WebGPU, file mode, the qwen-asr author's machine (4.66x realtime) source
Qwen3-ASR 1.7B	RTF	0.84	1 s streaming chunks	WebGPU, streaming, the qwen-asr author's machine (1.19x realtime) source

About these measurements

How these numbers were measured

Every figure in the tables was measured by ByteScope in Chrome, through the same code the subtitle and caption tools run. Speech recognition was scored on the FLEURS test split, 150 utterances per language, one file each: word error rate (WER) for English, character error rate (CER) for Chinese and Japanese, lower is better; Chinese is compared after converting both sides to Simplified characters. The GPU run used a GTX 1060 6GB, a card without shader-f16; the CPU run used the WebAssembly build with Chrome limited to 5 cores. RTF is processing time divided by audio length, so anything below one finishes faster than the audio plays. These speeds were measured on this one machine.

Which speech model is the most accurate

With a graphics card, Whisper large-v3-turbo is the most accurate for English (4.8% WER) and Japanese (5.6% CER), and in Japanese the gap to every other model is larger than the noise. For Chinese, turbo (7.5%), Qwen3-ASR 1.7B (7.7%), Qwen3-ASR 0.6B (8.0%) and Confucius4-R2T2 (8.2%) are statistically tied under strict scoring. Ignore how numbers are written and Qwen3-ASR 1.7B and R2T2 drop to 5.9%, clearly ahead of turbo's 7.2%: the Qwen family writes numbers in Chinese characters, while the FLEURS references use digits. Whisper small and base trail far behind in Chinese and Japanese.

Without a graphics card

On the CPU, Qwen3-ASR 0.6B is the pick for Chinese (7.5% CER) and Japanese (9.2%), and it finishes faster than the audio at RTF 0.456–0.543. For English, Whisper small (6.8% WER) and Qwen3-ASR 0.6B (8.2%) are tied within the noise, but Whisper small runs at RTF 1.601 on the CPU, so it takes longer than the recording itself. Qwen3-ASR 1.7B beats 0.6B in English and Japanese on the CPU as well, but at RTF 1.143 it is slower than real time, which is why only its first 30 utterances per language were run there. Chrome Speech Recognition and Gemini Nano need neither a graphics card nor a download from ByteScope; they were not part of this measurement.

Translation

Hy-MT2 1.8B was judged blind against Chrome's built-in Translator on 280 FLORES and Tatoeba lines in 5 directions: Hy-MT2 was preferred 139 times, Chrome Translator 66 times, and 75 were ties, with 42 major errors against 61. Against Gemini Nano, on 559 lines covering every direction between English, Chinese and Japanese, the score was 200 to 63 with 296 ties, and Hy-MT2 came out ahead in all six directions — narrowly only from Japanese to English, 22 to 18. In that round a difference that was only mainland-versus-Taiwan wording, a transliterated proper noun or simplified characters counted as a tie, and both systems' Traditional Chinese output went through the same correction pass first. Hy-MT2 needs WebGPU; Chrome Translator and Gemini Nano come with Chrome. Firefox Translations (Bergamot) was judged in a separate round, on the same lines as Chrome Translator and Hy-MT2: it scored above Chrome Translator in five of the six directions between English, Chinese and Japanese, and below it only from English to Traditional Chinese. It runs on the CPU, so when Hy-MT2 cannot run, the tools recommend it for those five directions.

Limits of this comparison

One graphics card, one machine. The GTX 1060 has no shader-f16, so the f16 builds — which make the Whisper downloads smaller — were not measured. The GPU speed runs overlapped a CPU batch, so the GPU RTFs are upper bounds. Whisper large-v3-turbo was measured again on the same graphics card after that CPU batch finished: 0.260, 0.263 and 0.241. Everything here is file mode; live captioning, where voice detection cuts the audio into sentences, was not measured. The Gemini Nano translation round had a single judge. The figures model vendors publish are listed in their own table and marked as vendor claims: they come from other test sets and other hardware and cannot be compared with these tables.

Does anything leave the browser

No audio or text is uploaded by any of these models, with one labelled exception. Whisper, Qwen3-ASR, Confucius4-R2T2, Hy-MT2 and Qwen3-ForcedAligner are downloaded once and then run in your tab; Gemini Nano and Chrome Translator are Chrome's own on-device models. Chrome Speech Recognition in its default mode sends the audio to Google's servers, and its engine card in every tool says so.

Frequently asked questions

Which speech recognition model is the most accurate for Chinese?

Under strict scoring four models are tied with a graphics card: Whisper large-v3-turbo 7.5%, Qwen3-ASR 1.7B 7.7%, Qwen3-ASR 0.6B 8.0% and Confucius4-R2T2 8.2% CER on 150 FLEURS utterances. If the way numbers are written is ignored, Qwen3-ASR 1.7B and R2T2 lead at 5.9%. Without a graphics card, Qwen3-ASR 0.6B is the practical choice at 7.5%.

Which model is the most accurate for English and Japanese?

Whisper large-v3-turbo: 4.8% WER in English and 5.6% CER in Japanese with a graphics card. In Japanese it beats every other model beyond the noise; in English, Confucius4-R2T2 at 6.1% is within the noise of it. On the CPU, English is a tie within the noise: Whisper small 6.8% WER on 150 utterances, Qwen3-ASR 1.7B 6.9% (the GPU run's 150-utterance figure; the 30-utterance CPU run was 6.5%), and Qwen3-ASR 0.6B 8.2%. Whisper small runs slower than the audio.

Which models work without a graphics card?

Every speech recognition model here, and every translation model except Hy-MT2 1.8B, which needs WebGPU. Whisper and Qwen3-ASR have a WebAssembly build for the CPU, and Chrome Speech Recognition, Gemini Nano and Chrome Translator are built into Chrome. On the CPU, Qwen3-ASR 0.6B (RTF 0.456–0.543) and Whisper base (0.328–0.382) finish faster than the audio; Whisper small and Qwen3-ASR 1.7B do not. Confucius4-R2T2 is the exception in practice: without WebGPU there is no word-by-word streaming, only sentence-by-sentence or whole files, and on the CPU it runs slower than the audio. Word-by-word live captions need a graphics card.

How big are the downloads?

For the WebGPU build measured here: Whisper base 214.5 MB, Whisper small 594.4 MB, Whisper large-v3-turbo 767.5 MB (the CPU and f16 builds are smaller: 103.4, 418.1 and 571.9 MB), Qwen3-ASR 0.6B 848.9 MB, Qwen3-ASR 1.7B and Confucius4-R2T2 2,183.5 MB each, Hy-MT2 1.8B 1,300.6 MB and Qwen3-ForcedAligner 0.6B 1,847.6 MB, in decimal megabytes. Each downloads once, only after you confirm, and is shared by every ByteScope tool. Chrome Speech Recognition, Gemini Nano and Chrome Translator download nothing from ByteScope.

Does my audio leave the browser?

No, with one labelled exception. Every downloaded model runs in your tab, and Gemini Nano and Chrome Translator are Chrome's on-device models. Chrome Speech Recognition in its default mode sends the audio to Google's servers; its engine card says so, and any other model avoids it.

Is Confucius4-R2T2 better than Whisper?

It depends on the language. With a graphics card, R2T2 scored 6.1% WER in English against Whisper large-v3-turbo's 4.8%, within the noise; 8.2% against 7.5% CER in Chinese, a tie; and 6.9% against 5.6% in Japanese, where turbo is clearly better. Ignoring how numbers are written, R2T2's Chinese drops to 5.9% and beats turbo's 7.2%. R2T2 is faster on this card (RTF 0.153–0.189 against 0.315–0.386) and streams words while you speak in Live Captions, but it downloads 2,183.5 MB against 767.5 MB.

Why are the vendors' published error rates lower than these?

They are measured on other test sets and other hardware. The Qwen3-ASR repository reports 1.63% WER on LibriSpeech test-clean and 2.71% CER on AISHELL-2 for 1.7B; the Confucius4-R2T2 model card reports 2.13% WER on LibriSpeech test-clean. Those are vendor claims, kept in their own table. ByteScope runs quantised builds in a browser on FLEURS, so the two sets of numbers cannot be compared directly.

Which translation model should I use?

With a working graphics card, Hy-MT2 1.8B: judged blind ahead of Chrome Translator 139 to 66 (75 ties) and of Gemini Nano 200 to 63 (296 ties), and it is what the tools recommend when WebGPU works. Without one, Firefox Translations (Bergamot) runs on the CPU and scored above Chrome Translator in five of the six directions between English, Chinese and Japanese, so the tools recommend it there; from English to Traditional Chinese Chrome Translator scored higher and stays the default. Chrome Translator needs no download from ByteScope, and Gemini Nano covers pairs it lacks.