transformers.js, ONNX Runtime Web and tfjs: I benchmarked all three browser ML runtimes with real numbers
I built a benchmark to measure transformers.js, ONNX Runtime Web and tfjs running the same sentence-embedding task, plus an identical MLP on the two runtimes that can actually run it. WASM beat WebGPU on both tasks, tfjs's backend choice mattered more than which runtime I picked, and along the way I found a backend that would not even start by default on this machine.
"Which browser ML runtime is fastest" is a question that gets answered by ear a lot and measured rarely, and almost never with the same model on every side. I built a reproducible benchmark to actually measure it: transformers.js, raw ONNX Runtime Web and tfjs, in Chrome, on this machine, the same day. Two tasks, not one -- and the reason for two is, to me, the most important result of the whole thing.
The first task is sentence embeddings with quantized MiniLM -- the same model I already use in the lab's RAG demo and in this site's search. transformers.js and raw ONNX Runtime Web both run the exact `model_quantized.onnx` file, with the same tokenizer, so the only real difference between those two rows is how each one invokes inference. But tfjs has no MiniLM port. Rather than leave it out of the table, I let it run the Universal Sentence Encoder instead (a different model: 512 dimensions instead of 384, an averaging network instead of a transformer) and I say so plainly in the README: that row is not an isolated runtime comparison, it's "what the most common embedding job actually costs on tfjs," with a model asterisk stapled on top.
To get one genuinely clean comparison, I built a second task: a tiny MLP with byte-identical weights on the two runtimes that can actually run it. A Python script generates `Linear(384,256) -> ReLU -> Linear(256,128) -> ReLU -> Linear(128,64)` with a fixed seed (139,712 parameters) and exports it twice: to JSON so tfjs can build the model at runtime, and to an ONNX graph for ONNX Runtime Web. transformers.js does not appear in this table -- it is a Hugging Face model-loading layer, not a generic tensor runtime, and forcing it to run a made-up MLP would have meant fighting the tool instead of measuring it. I checked the weights three separate ways: the ONNX build script validates the compiled graph against a reference fixture using onnxruntime itself, a pure-JavaScript test re-runs the forward pass by hand, and the first five output values from every actual runtime match that same reference to the fifth decimal place.
The harness is two build-free HTML pages, served with `python3 -m http.server`. Every combination was measured across 3 fresh browser sessions -- cache disabled in each one, so every session genuinely re-downloads the model -- with 10 warmup passes and 50 measured ones for batch-1 latency, plus 3 and 10 for batch-8. I pinned `numThreads` to 1 on both ONNX-Runtime-Web-backed runtimes so the result would not depend on whether the server sends the `Cross-Origin-Opener-Policy` headers threaded WASM needs -- `python3 -m http.server` does not send them, and I preferred a number that reproduces on any static server over a faster one that's fragile.
The first snag was small and real: the tfjs Universal Sentence Encoder package points at TF Hub by default, and TF Hub doesn't exist anymore. `tfhub.dev` now redirects to Kaggle Models, so the package's default loader downloads a Kaggle HTML page instead of a model. The fix was routing around that dead default straight to the checkpoint still hosted at `storage.googleapis.com/tfjs-models`, 28 MB and very much alive. It isn't a finding about ML runtimes, it's a finding about how fast a 2021 library's defaults rot -- and I left it documented right in the runner's own code, because it cost me half an hour to work out.
The second snag was more serious: WebGPU simply would not start on this machine, despite a real RTX 3050 sitting right there. `navigator.gpu.requestAdapter()` returned `null` in a plain Chrome window, and `chrome://gpu` explained why: Vulkan showed as "Disabled" even with the NVIDIA Vulkan ICD installed and WebGL correctly using the real GPU the whole time. The fix was launching Chrome by hand with `--enable-features=Vulkan --use-vulkan=native`, which is what produced every WebGPU number in this benchmark. Open the harness in a default Chrome on Linux and the WebGPU option will throw "Failed to get GPU adapter" -- and that is part of the result, not a bug in the harness.
With all of that sorted, the central finding: WASM beat WebGPU on both tasks, every single time, by a lot. For MiniLM, WASM ran 12 to 12.2 ms against WebGPU's 158-173 ms -- 13 to 14 times faster. For the tiny MLP the gap is even more extreme: 0.10 ms on WASM against 4.50 ms on WebGPU, 45 times. None of this is a WebGPU failure -- it's well documented in both runtimes' communities that WebGPU only pays off once a model or a batch is large enough that the fixed per-dispatch cost to the GPU stops dominating. Batch-8 narrows the gap (2.9x instead of 13x for MiniLM) but WASM still wins -- nothing in this benchmark's model sizes was big enough to flip that story.
tfjs's own backend choice mattered more than which runtime I picked. Inside tfjs alone, WebGL versus CPU is an 18x latency gap for the Universal Sentence Encoder (15.95 ms vs 291.75 ms) and a 76x gap in batch throughput (305 vs 4 items/s) -- bigger than the gap between any two *different* runtimes in either table. If a real app falls back to tfjs's CPU backend without noticing (easy to do: `tf.setBackend` isn't always called explicitly, and WebGL initialization can fail silently on a machine without a GPU), that slowdown dwarfs any decision about which runtime to use in the first place.
Cold-load time and per-call latency point in opposite directions for tfjs. The Universal Sentence Encoder loads in about 228 ms, 4 to 6 times faster than MiniLM through either ONNX Runtime Web path (1,000-1,400 ms) -- and that gap is real, not a network effect, since both models are vendored locally and the time is dominated by parsing and session construction, not download. But at per-call latency, with WASM as the fastest backend measured, ONNX Runtime Web still wins (12 ms vs 16 ms for tfjs+WebGL). An app that builds one session and reuses it many times lives off that second number; one that embeds a single query and discards the session lives off the first.
All of it -- the full methodology, the results table with all 3 raw sessions per combination, a "What this does not tell you" section listing every limit I found (one machine, the model asterisk on tfjs, the WebGPU that needed Vulkan forced on, approximate memory that only counts the JS heap) and the exact scripts to reproduce it -- is in browser-runtimes-bench. I did not build this to decide "the best runtime": I built it because that question, asked the way it usually is, has no honest answer without first saying what was held constant and what wasn't. If you came from the browser ML guide, this is the other half: not just how a model gets served in the browser, but what actually happens underneath when three different runtimes run it.
