Offline Photo and Conversation Translation Workflow

Direct answer

A photo-to-conversation workflow should be tested as coordinated but distinct tasks. The photo path acquires an image, extracts visible text, and presents a translation for review; the spoken path captures a new turn, recognizes speech, translates it, and returns the result. A shared screen or “conversation” label does not prove the pipelines share an offline engine or preserve the right source. `COVE-OCR-01` documents a structured visible-text field and copy/export behavior in the checked Cove Photo code. `COVE-ASR-01` describes an audio transcription path in the repository. These facts support separate records; they do not establish that the two product modes are integrated or that a photo translation can automatically enter speech dialogue. Define a session state contract before testing. Give each photo, OCR output, translated text, audio turn, transcript, and correction its own identifier and source pointer. When the user chooses to continue from a photo result into conversation, record the explicit handoff: did the user copy a reviewed sentence, select a region, or start a new spoken task? Do not silently convert every OCR result into microphone input. Keep image evidence, recognized text, generated description, and translation as distinct data types. The original image should remain available for correction if the product's retention behavior permits; a local inference claim does not determine storage or network behavior. Use a synthetic or authorized image fixture with visible, checked text, then use a separate consented spoken fixture for the next turn. Validate image orientation, crop, source text, uncertain spans, and target translation. For speech, label the transcript and direction independently. Test names, numbers, negation, a blank or unreadable photo region, and a spoken follow-up referring to “that line.” Verify that the follow-up uses the selected reviewed text rather than an unreviewed model description. If source provenance cannot be preserved, expose that limitation rather than imply the conversation is grounded in the photo. Exercise cross-mode interruptions: cancel photo processing, navigate away, revoke camera or microphone access, rotate or recreate the activity, lose connectivity, and resume from history. The app should not attach a stale photo result to a new voice turn or expose the wrong transcript. Test history and export separately: which source image, OCR text, spoken transcript, translation, and user correction are retained? Verify deletion and copy actions per content type. CameraX image frames require lifecycle cleanup when that pipeline uses ImageAnalysis; Android audio capture has its own permission and lifecycle rules. Do not infer that closing one pipeline clears the other pipeline's data. Offline testing needs a declared starting state. Record whether OCR libraries or models, translation assets, voice recognizers, and TTS voices were installed or cached. Test each mode offline after setup, then test setup itself as a different scenario. Recognizer offline preference can be ignored, and voice availability can be separately constrained. Observe network requests using supported inspection plus code review; do not equate airplane mode with proof that every stage is local. If photo analysis works but spoken recognition does not, present a partial capability state rather than an all-or-nothing badge. The report includes build, artifact/runtime per mode, image/audio fixtures, handoff action, source identifiers, network state, user review, history behavior, cancellation, and all failures. Keep results stage-specific and avoid a single end-to-end score without a defined rubric. A passing fixture does not establish that every photo, language, voice, or phone works offline. This guide is about preserving traceability and user control when a session changes modality, not claiming any unreleased feature or universal translation quality.

A traveler reads a sign and then asks a spoken follow-up

The user captures a sign, reviews visible text and its translation, then asks a spoken question about one line. The app has separate image and audio state owners. This test targets explicit cross-mode handoff and provenance, not photo-only translation readiness.

Decision

Require an explicit handoff from the reviewed line to the spoken task, preserve source references, and test each mode and network dependency separately.

Expected outcome

The report shows whether the follow-up is grounded in selected reviewed text and whether image/audio state remains separate through history and cancellation.

Evidence to collect

The claim becomes reviewable only when the following evidence is attached to the same artifact and test run:

• Image, text, and audio fixtures
• Cross-mode state contract
• Airplane-mode task and recovery results

Task quality needs a versioned corpus and acceptance rules. Fluent sample output is not an accuracy result.

Implementation workflow

1. Define cross-mode state and provenance. Assign identifiers to image, OCR, translation, audio turn, transcript, and correction. Specify which user action links one item to the next.
2. Validate the image path first. Use a checked fixture and inspect orientation, crop, literal text, uncertain spans, translation, and user correction before using it in a later step.
3. Start speech as an explicit new task. Capture a consented follow-up, record language direction and transcript separately, and verify whether it references a reviewed selected line rather than a generated description.
4. Inject lifecycle and permission failures. Cancel, navigate, revoke camera/microphone access, recreate the screen, lose connectivity, and resume history. Ensure stale image or transcript state is not attached to a new turn.
5. Audit retention and offline routes. Record setup/cached assets for OCR, translation, recognition, and playback. Test each mode after setup offline and inspect supported traffic plus code for unobserved clients.

Keep each transition observable. A failure should identify the stage, artifact, runtime, and recovery action without logging private user content.

Acceptance criteria

• Image source, recognized text, translation, spoken transcript, and user correction remain distinguishable with explicit provenance across a handoff.
• Cancellation, navigation, permission revocation, and process recreation do not attach stale image/audio data to another task.
• Offline statements are stage-specific and scoped to recorded asset setup, build, and network observations; no universal mode or device claim is made.

Decision table

Failure patterns to prevent

• Sharing hidden state between modalities
• Calling photo description literal translation

Also prevent silent fallback, unpinned artifacts, missing cancellation, and conclusions that combine unlike configurations. Store unsuccessful runs alongside successful ones.

Minimum reproducibility record

Release checklist

• ☐ The primary query is answered without an unsupported number.
• ☐ Every artifact and runtime is pinned.
• ☐ The representative task and failure policy are explicit.
• ☐ Lifecycle, cancellation, cleanup, and fallback are tested.
• ☐ User-content and network boundaries are documented.
• ☐ Result wording applies only to the recorded configuration.
• ☐ The page links to raw method or evidence when results are added.

Sources and related evidence

• ML Kit text recognition
• Cove Travel voice translation
• CameraX image analysis
• Android audio capture overview
• Google ML Kit text recognition

Chinese deployment and troubleshooting content is organized in the .奇连 AI 端侧专题.

Frequently asked questions