Offline OCR Pipeline on Android

Direct answer

An OCR pipeline should preserve stage boundaries: acquire an image, verify orientation and crop, select a recognizer, obtain structured text regions, let the user correct uncertainty, and pass only the reviewed fields to later processing. Dedicated text-recognition APIs return structured elements such as blocks, lines, or tokens; a multimodal model may instead return an interpretation in a prompt-defined format. These are not interchangeable outputs and should not be scored with the same acceptance rules. Offline capability depends on the exact library and artifact path. Google ML Kit documents both bundled and Google Play Services-based options; the unbundled option may need a model download before the first result, while the bundled option includes its model at app build time. A privacy or offline statement therefore must name the selected implementation and test its setup as well as inference. A clean-install test in airplane mode before model availability is different from a warm test after a resource has already been downloaded. Do not say “OCR is offline” because a model is described as on-device without verifying downloads, telemetry, backups, and exports. Image preparation is a first-order part of the pipeline. CameraX provides rotation metadata and an ImageAnalysis path; release ImageProxy on completion even if recognition fails. For a file import, treat the URI and permission lifetime carefully, decode only the needed image, and record any crop, resize, or orientation transform. Official ML Kit guidance notes that focus, resolution, and text size affect recognition and suggests the caller should recapture when quality is insufficient. Do not invent an accuracy percentage or guarantee all scripts are recognized. Structure the output contract to match the user task. A receipt flow might need merchant, date, total, and line items; a menu translation flow needs the original text and a separate translation result. Preserve source image boundaries and confidence or region metadata only if the selected library actually provides it. Never silently map a model guess into a trusted field. Allow the user to review, edit, reject, or copy recognized text. Keep source image and extracted text retention policies separate. Compare three architectures before committing: a dedicated OCR recognizer, a multimodal model asked to transcribe visible text, or a hybrid that uses OCR for region extraction and a separate model for translation or explanation. Each has different output structure, model-delivery, first-run, and verification requirements. Do not infer Cove Photo’s exact OCR accuracy from the code path; the fact card describes its structured detectedText field and export actions, not a benchmark. A testable implementation sequence is: 1. Define the text task and output schema. Decide which fields are required, which are suggestions, how users correct them, and which source image or region each field came from. 2. Pin input preparation: camera or file URI, rotation, crop, resolution, color format, and resource ownership. Add tests for rotated, blurred, low-light, cropped, and empty images without claiming a numerical accuracy threshold. 3. Verify the recognizer delivery path from clean installation through first use. For any dynamic model download, show its state and test offline behavior before and after availability. 4. Validate structured results, present uncertainty, and let users correct or reject output before actions such as saving or sharing. 5. Test cleanup: close camera frames, image buffers, recognizer sessions, temporary files, and any task owner on success, cancellation, and exception. The result should tell a reviewer exactly which pipeline produced each value and what remains user-verified. It is a method, not a model leaderboard.

A receipt image feeds an offline expense draft

A user photographs a receipt and expects an expense draft containing recognizable text, not an automatically trusted financial record. The Android team must decide whether a dedicated OCR API, a multimodal model, or a staged combination supplies the draft.

Decision

Keep image acquisition, recognition, interpretation, and user approval as separate stages, and verify the chosen model-delivery path in the network condition the feature promises.

Expected outcome

The user can inspect and correct extracted fields while the test report preserves the source, transformation, result structure, and offline boundary.

Evidence to collect

The claim becomes reviewable only when the following evidence is attached to the same artifact and test run:

• Reference images and text
• Preprocessing revision
• Failure classes

Task quality needs a versioned corpus and acceptance rules. Fluent sample output is not an accuracy result.

Implementation workflow

1. Define an editable output schema. List required fields, optional suggestions, source region, correction action, and rejection behavior. Keep detected text separate from translated or inferred values and do not mark a model-produced amount as verified financial data.
2. Control input orientation and lifetime. Record whether input came from CameraX or a document URI, carry the correct rotation and crop, and bound resolution to the task. Close ImageProxy and buffers after processing on success and failure.
3. Verify the exact recognizer and model delivery. Identify whether the recognizer is bundled or downloaded through a service, expose readiness, and test a fresh installation both offline and after required assets are available. Do not infer offline setup from a previous warm run.
4. Review, validate, and export deliberately. Check the returned structure and any available region metadata, show low-quality or missing fields, allow correction or rejection, then test copy and export separately. Apply retention and deletion rules to the source image and extracted text independently.

Keep each transition observable. A failure should identify the stage, artifact, runtime, and recovery action without logging private user content.

Acceptance criteria

• Every saved field can be traced to recognized text, a model interpretation, or an explicit user edit.
• The tested offline claim specifies whether a required model download is already present or still needed.
• Images, camera frames, recognizer resources, and temporary buffers are released on success, cancellation, and failure.

Decision table

Failure patterns to prevent

• Silent text correction
• Mixing extraction and interpretation

Also prevent silent fallback, unpinned artifacts, missing cancellation, and conclusions that combine unlike configurations. Store unsuccessful runs alongside successful ones.

Minimum reproducibility record

Release checklist

• ☐ The primary query is answered without an unsupported number.
• ☐ Every artifact and runtime is pinned.
• ☐ The representative task and failure policy are explicit.
• ☐ Lifecycle, cancellation, cleanup, and fallback are tested.
• ☐ User-content and network boundaries are documented.
• ☐ Result wording applies only to the recorded configuration.
• ☐ The page links to raw method or evidence when results are added.

Sources and related evidence

• ML Kit text recognition
• Google for Developers — ML Kit Text Recognition on Android
• Android Developers — CameraX image analysis
• Google for Developers — ML Kit Text Recognition v2

Chinese deployment and troubleshooting content is organized in the .奇连 AI 端侧专题.

Frequently asked questions