Offline Form Extraction on Android

Direct answer

Reliable form extraction is a staged workflow, not a single model prompt. Keep the original image or document as the evidence source; capture a versioned OCR result; map literal spans into typed candidate fields; validate each candidate; show uncertainty and provenance; then let the user correct or reject the result before saving or exporting. `OCR-DEDICATED-01` describes text-recognition output as structured blocks, lines, and elements rather than one undifferentiated string. `COVE-OCR-01` documents a Cove Photo path where visible text is represented separately as a structured field and can be copied or exported. Those facts support a pipeline distinction; they do not establish that a particular form, field, language, or image is recognized accurately. The first important boundary is literal recognition versus interpretation. OCR can return text and location-like structure, while a downstream parser may infer that one span is a date, amount, address, or identifier. Preserve both the raw recognized text and the mapped value, with links between them. Do not silently normalize leading zeros, punctuation, date order, decimal separators, or identifiers. If an expected field is absent, retain “missing” rather than inventing a value. If two spans could fill the same field, keep the ambiguity visible instead of choosing the most plausible answer without a rule. Represent extraction as a typed, versioned result. A field can include the candidate value, source span, confidence category only if the implementation actually defines it, validation outcome, and review state. Avoid fabricated numeric confidence scores. A schema should distinguish accepted, needs-review, missing, rejected, and unsupported values. Unknown fields should be preserved or rejected according to an explicit version policy, not silently discarded. Record OCR engine/configuration, image preprocessing revision, parser version, and schema version so later changes can be attributed; do not turn one successful fixture into an accuracy statement. Design the human review around the source. Selecting a field should highlight or reveal its corresponding text and, where useful, image region. Show transformations such as trimming whitespace or applying a known date format. The user should be able to restore the literal value, correct it, leave it empty, or cancel the whole extraction. For consequential data—such as a payment reference, medicine label, or account identifier—make confirmation explicit and do not present model interpretation as verified fact. An export should include only fields the user reviewed or clearly mark fields that remain provisional. Test with a versioned synthetic corpus covering clear values, low contrast, skew, handwriting only if that modality is in scope, multiple columns, duplicate labels, missing values, ambiguous dates, punctuation, leading zeros, and unexpected text. Define expected properties field by field, not just a pleasing final form. Run separate checks for image capture/import, rotation and preprocessing, OCR structure, field mapping, validation, review editing, save, export, and deletion. Keep source images and derived text retention policies explicit; forms frequently contain personal data. Use synthetic fixtures in automation and avoid logging raw form contents. Failure should be recoverable: retain the source if the user chose to keep it, mark partial extraction, allow correction, and avoid overwriting an existing record with unreviewed guesses. Only make claims supported by the exact pipeline and test set. “The app can extract these fields from this fixture set” is bounded; “it reads any form accurately offline” is not. If a dedicated OCR library and a local language model are combined, report their roles separately. OCR output is evidence of recognized text, not proof that a downstream semantic field is correct. This separation makes the feature easier to test, lets users correct errors, and keeps unsupported accuracy or compatibility claims out of the product copy.

A user extracts invoice fields before approving an expense

A receipt image contains a merchant name, date, total, and a reference number. OCR returns several lines, while a parser has to map them into fields and some values could be ambiguous.

Decision

Keep the literal recognized spans beside proposed field values, flag ambiguity, and require user review before writing the expense record or exporting it.

Expected outcome

The saved record reflects confirmed values, preserves corrections, and does not silently invent missing identifiers or normalize uncertain numbers.

Evidence to collect

The claim becomes reviewable only when the following evidence is attached to the same artifact and test run:

• Reference forms
• Field-level expected output
• Missing and ambiguous cases

Task quality needs a versioned corpus and acceptance rules. Fluent sample output is not an accuracy result.

Implementation workflow

1. Version the source and OCR stage. Record the input fixture identity, orientation/preprocessing revision, OCR library configuration, and structured result. Keep raw recognized spans separate from any semantic interpretation.
2. Map spans to typed candidates. Use explicit field rules for date, amount, and identifiers. Retain the source span for every candidate and distinguish missing, ambiguous, invalid, and unsupported values.
3. Validate without silent correction. Check allowed format and required fields deterministically. Do not repair leading zeros, currency separators, identifiers, or date order unless the rule is explicit and the transformation is displayed.
4. Build source-linked human review. Let the user inspect each candidate with its OCR text or image region, correct it, leave it empty, or cancel. Require an explicit confirmation for fields that affect the saved record.
5. Test persistence and privacy paths. Use synthetic forms to test save, edit, export, deletion, lifecycle recovery, and logs. Confirm that only reviewed values are exported by default and that raw form content is not sent to telemetry.

Keep each transition observable. A failure should identify the stage, artifact, runtime, and recovery action without logging private user content.

Acceptance criteria

• Every extracted candidate links to a literal OCR span and has an explicit state; missing or ambiguous input is never silently fabricated.
• The user can correct, reject, or leave fields blank, and unreviewed candidates are not represented as verified facts in save or export.
• A versioned synthetic fixture suite covers transformations, failure, persistence, deletion, and privacy-sensitive diagnostic output.

Decision table

Failure patterns to prevent

• Auto-correcting identifiers
• No review for high-impact fields

Also prevent silent fallback, unpinned artifacts, missing cancellation, and conclusions that combine unlike configurations. Store unsuccessful runs alongside successful ones.

Minimum reproducibility record

Release checklist

• ☐ The primary query is answered without an unsupported number.
• ☐ Every artifact and runtime is pinned.
• ☐ The representative task and failure policy are explicit.
• ☐ Lifecycle, cancellation, cleanup, and fallback are tested.
• ☐ User-content and network boundaries are documented.
• ☐ Result wording applies only to the recorded configuration.
• ☐ The page links to raw method or evidence when results are added.

Sources and related evidence

• ML Kit text recognition
• Google ML Kit text recognition v2
• Android CameraX image analysis
• Android security and privacy overview

Chinese deployment and troubleshooting content is organized in the .奇连 AI 端侧专题.

Frequently asked questions