Running AI Models in the Browser: What We Learned Shipping On-Device Models

Updated · by Hakeem and Ahmad

Small AI models now run well enough in an ordinary browser tab to power real features, as long as each one is a few megabytes, downloads only when someone asks for it, and is tested in every browser engine, because speed varies tenfold between them. In November 2026 we added on-device models to six tools on My Gadgets: background removal, face finding for passport photos and blurring, OCR for images and scanned PDFs, and Arabic diacritization. No photo, page or text is sent to a server for any of them. This post records what we used, what we measured, what we dropped, and the practical problems we hit. It is written for developers weighing the same choice.

What did we ship?

Feature Model Licence Model size Runtime
AI cutout in Remove Background U²-Net small (u2netp) Apache-2.0 4.57 MB ONNX Runtime Web 1.19.2 (~2.5 MB)
Auto-frame in Passport & Visa Photo Resizer, Find faces in Blur Image UltraFace RFB-320 MIT 1.27 MB ONNX Runtime Web 1.19.2
Image to Text (OCR) and scanned pages in Extract Text from PDF Tesseract LSTM, tessdata_best (integer) for English and Arabic Apache-2.0 2.95 MB (eng) + 1.66 MB (ara), gzipped Tesseract.js 5.1.1 (~1.4 MB engine)
Add tashkeel in Arabic Tashkeel ichkil 1.0.0 MIT ~5 MB ONNX Runtime Web 1.19.2

Model files are served from our own domain, each next to its licence and a short README naming its source; the runtimes are pinned to exact versions on jsDelivr. The ONNX Runtime download is shared, so a visitor who has used one ONNX feature does not pay for it again.

How fast are they, per browser?

All numbers come from Playwright on a desktop machine, in Chromium, Firefox and WebKit. We have not yet put numbers on real phones.

Task Chromium / WebKit Firefox
Background cutout (u2netp, 320 × 320 input) 2.5–3.7 s ~32 s
Face finding (UltraFace, 320 × 240 input) 40–160 ms a picture, after a 1.5–2 s first session ~300 ms
OCR, one picture 0.3–2 s 4–6 s
Tashkeel, 400-character paragraph 0.1–0.9 s (range across all three engines) (same range)

The Firefox gap for the cutout is the striking one: about ten times slower for the same model and the same single-threaded WebAssembly backend. We have not pinned down why. Instead of hiding it, the tool says before you start that it can take up to a minute in Firefox or on an older phone.

Everything runs single-threaded. Multi-threaded WebAssembly needs SharedArrayBuffer, which needs the page to be cross-origin isolated (COOP and COEP headers). The site does not send those headers, so we set numThreads = 1 and accepted the speed.

How accurate are they?

We tested each model against something we could check, rather than trusting its model card alone:

  • u2netp: IoU 0.986 on a synthetic disc over noisy stripes, the same in all three engines; on a real portrait the background corners were cleared and the face and body kept.
  • UltraFace: one face found on a portrait and three of three on the Apollo 11 crew photo (both public-domain NASA pictures), in all three engines.
  • Tesseract: character error rate on computer-drawn text of 0.5–2.3% for Arabic and 0–0.5% for English, rising to 9.9% for English shrunk to 0.4×. Real phone photos, handwriting and calligraphic fonts are worse, and the tool page says so.
  • ichkil: all 29 of the model's own reference vectors matched exactly in the browser, in all three engines. Its published error on its test set (Fadel et al., 2019, mostly classical books) is 2.8% of letters, and we measured 3.2% on 500 random lines of it in the browser. On modern news writing (the WikiNews test set) we measured 11.8%: about one word in three had at least one wrong mark, mostly a wrong short vowel. The page now gives both numbers.

These are small tests, not benchmarks. Their job was to prove that the model runs correctly in each browser and does what it claims on typical inputs, and to give the tool pages honest numbers. For what the accuracy means to users, see Why Does OCR Get My Text Wrong? and AI Background Removal: When It Works, When It Fails.

What did we drop, and why?

  • MediaPipe face detection. The 0.10.14 bundle we wanted returned 404 on jsDelivr, and its WebAssembly is 9.4 MB, against 1.27 MB for UltraFace plus a runtime we were already loading.
  • MODNet (portrait matting, Apache-2.0, 6.6 MB quantised). Fine on people, but its IoU was only 0.81–0.86 on our non-person test. Background removal is used for products and pets as much as portraits, so u2netp covers more cases.
  • BiRefNet and RMBG. Better cutouts, especially hair, but 40–170 MB. RMBG is also not licensed for commercial use, and a site that shows ads is commercial.
  • CATT for tashkeel (Apache-2.0, 21.6 MB as int8). It was the other candidate; we took ichkil, at about a quarter of the size, once it matched its reference outputs in all three engines.

The pattern: we take the smallest model that does the job on the inputs people actually bring, with a licence that plainly allows a commercial site. A model the visitor waits two minutes to download is a model they never see working.

What went wrong along the way?

WebKit traps now and then while creating a session. Its WebAssembly engine sometimes throws "Out of bounds memory access" when ONNX Runtime creates an inference session. A second attempt runs cleanly, so the shared loader retries once. We had already seen the same behaviour with qpdf.

Gzip on top of gzip. Tesseract's language data ships as .traineddata.gz, and Tesseract.js unpacks it itself. If the web server adds Content-Encoding: gzip, the browser unpacks it first and Tesseract.js then fails on data that is no longer gzipped. Our nginx serves /assets/models/ as plain bytes with gzip off and a month's cache; a new model gets a new file name rather than a cache-busting query.

JSON that the server would not serve. ichkil's character vocabulary lives in a config.json, and our nginx does not serve .json files, so the 40-entry vocabulary is written into the tool's script instead.

One run at a time. An ONNX session cannot run twice at once, and a user can click twice. The loader queues runs per model and creates each session once; a failed download clears the session so a later click can try again.

Long input. ichkil accepts about 1,024 characters at a time, so text is cut into pieces of at most 1,000 characters at sentence ends, then at spaces, and joined back exactly. Tesseract slows sharply on very large photos without reading better, so pictures are scaled to at most 3,000 pixels on the longer side.

How do we keep the no-upload promise?

The model only ever reads pixels or characters from the page; there is no request that carries the user's data. Concretely:

  1. Nothing is downloaded until the user presses the AI option, and the option states the download size first (about 4 MB for face finding, 7 MB for the cutout, 3–6 MB for OCR depending on language, 8 MB for tashkeel).
  2. The model and runtime downloads contain only code and weights. The photo is drawn to a canvas, turned into a tensor in memory, and the result is drawn back to a canvas.
  3. Our end-to-end tests record every request a page makes. The tashkeel test fails if anything other than a GET leaves the page while it runs, and the site-wide privacy tests check that opened files and typed text never appear in a request. The image AI tests check results, not requests, so extending that check to them is on our list.
  4. The privacy policy lists every tool that downloads a model, its size and where it comes from.

Would we do it again?

Yes, with the same rules: a few megabytes at most, a licence that clearly allows commercial use, loaded only on demand with the size stated, validated in all three engines before it is built, and described with measured numbers rather than adjectives. The next candidates we are looking at, such as speech recognition and translation, are larger, and each will have to clear the same bar on phones before it ships.

Quick answers

Does the model run offline? After the first download the browser normally keeps the files in its cache, but we do not guarantee offline use; the first use always needs a connection.

Why the WebAssembly backend and not WebGPU? WebAssembly runs in all three engines, and these models are small enough that it is fast enough outside Firefox. WebGPU could be faster where it is available; we have not tested it yet.

Can I see the model files? Yes. Each lives under /assets/models/<name>/ on the site with its licence and source.

What to do next

Try the AI cutout in Remove Background or OCR in Image to Text, with your browser's network panel open: you will see the model download once, and nothing go out with your file.

Try the tool

Remove the background from a photo with an on-device AI cutout, or remove a plain single-colour background, and download a transparent PNG.

Open Remove Background →

More guides for Image Tools

Browse all Image Tools →