Offline Two-Speaker Translation Test Plan

Direct answer

A two-speaker translation test must verify who spoke, what was captured, which language direction was used, and how each result was presented. It is not enough to feed two short phrases to a model and call the result a conversation. `COVE-ASR-01` documents the current shared inference path for audio transcription, and `COVE-ASR-02` documents sequential segments with visible placeholders when a segment fails. Neither fact says that the current application automatically identifies speakers or performs diarization. Treat speaker identity as a test label supplied by the fixture or user, not a capability to assume. Prepare paired turns with a checked transcript and direction annotation for every turn. Mark intended speaker, source language, target language, turn order, expected display grouping, and whether playback is part of the scenario. Include overlap, interruption, silence, a speaker who changes language, repeated short words, names, and a turn that should be rejected as unclear. Keep conversational content synthetic or consented. A language choice in UI does not establish that capture or transcription works in that language; record each stage's own evidence. Keep three records separate: captured audio/segment identity, recognized source words, and target translation. If the app merges segments, preserve the mapping from each merged span to its source turn. A missing recognition segment can make the translated output appear assigned to the other person. Likewise, a fluent translation may still reverse meaning or attach a reply to the wrong speaker. If the implementation does not expose speaker labels, the test report should say the labels were manually annotated and should not imply the application inferred them. Test interruption and task state, not just ideal alternating turns. Pause or cancel capture, deny or revoke microphone permission, background the activity, stop recording, and simulate model or parser failure. The UI should make a partial turn visible, avoid sending it as a complete sentence, and let the user retry or edit before sharing. Android's capture documentation and `AudioRecord` reference describe platform-level acquisition interfaces, but they do not define the product's conversation semantics; those belong in the app's own acceptance contract. Use a fixed fixture set for repeated runs. Record build and artifact identity, runtime, audio format, input route, language direction for every turn, network state, segment boundaries, user corrections, cancellation, and final disposition. A target-side reference should be reviewed independently. Evaluate turn ownership, omission, negation, number preservation, order, and whether unclear speech is marked uncertain. Do not invent accuracy scores or claim the test predicts performance in a real crowded conversation. Preserve failures and disagreement rather than selecting only clean examples. The final report should answer practical questions: can the tested build keep two manually labeled turns in order; do partial failures remain attributable; is each translated text associated with the intended turn; can the user correct it; and what happens on interruption? It should state unsupported capabilities and the exact environment tested. A successful test for alternating synthetic turns proves only that recorded workflow, not automatic speaker recognition, every language pair, continuous streaming, or safe use in a high-stakes exchange.

Two people alternate short directions in a controlled test

A reviewer tests a conversation screen with one speaker asking a question and another replying. The application exposes audio and text, but automatic speaker identification is not established. The team must distinguish manually supplied turn labels from behavior the build actually performs.

Decision

Label speakers and language direction in the fixture, preserve one record per turn, and test whether the product keeps ordering and partial failures visible.

Expected outcome

The report distinguishes manually supplied turn labels from application behavior and identifies whether any stage loses or misattributes content.

Evidence to collect

The claim becomes reviewable only when the following evidence is attached to the same artifact and test run:

• Two-speaker audio fixtures
• Turn and language labels
• Ordered output and failure record

Task quality needs a versioned corpus and acceptance rules. Fluent sample output is not an accuracy result.

Implementation workflow

1. Define turn labels and expected presentation. For each turn set a stable speaker ID, source/target direction, order, transcript reference, expected translation property, and whether playback is included. Reuse the same labels across repetitions so output changes can be attributed to the run rather than the annotator.
2. Capture a controlled fixture set. Use consented or synthetic voices. Include alternating turns, overlap, interruption, silence, mixed language only if in scope, short words, names, numbers, and unclear input.
3. Trace audio through each stage. Record captured segments, recognized text, translation, display grouping, and optional playback separately. Keep segment-to-turn mapping for every speaker through any merge or retry.
4. Exercise cancellation and lifecycle changes. Test stop, cancel, permission denial/revocation, backgrounding, process recreation, and recognizer/model/parser failure. Verify partial turns are not presented as complete, and confirm a new turn starts only after state returns to idle.
5. Review task properties without inventing a score. Compare turn order, attribution, omissions, negation, numbers, and uncertainty against reviewed references. Keep failures and corrections in the report for every fixture.

Keep each transition observable. A failure should identify the stage, artifact, runtime, and recovery action without logging private user content.

Acceptance criteria

• Each fixture turn has explicit speaker and language labels that are not misrepresented as automatic speaker recognition.
• Partial, cancelled, missing, or failed turns remain visible and attributable; the app does not silently attach content to the other speaker.
• The report records build, artifact/runtime, route, direction, fixture, and limitations and makes no unmeasured language or conversation-quality claim.

Decision table

Failure patterns to prevent

• Merging speakers silently
• Testing isolated phrases only

Also prevent silent fallback, unpinned artifacts, missing cancellation, and conclusions that combine unlike configurations. Store unsuccessful runs alongside successful ones.

Minimum reproducibility record

Release checklist

• ☐ The primary query is answered without an unsupported number.
• ☐ Every artifact and runtime is pinned.
• ☐ The representative task and failure policy are explicit.
• ☐ Lifecycle, cancellation, cleanup, and fallback are tested.
• ☐ User-content and network boundaries are documented.
• ☐ Result wording applies only to the recorded configuration.
• ☐ The page links to raw method or evidence when results are added.

Sources and related evidence

• Cove Travel voice translation
• Cove benchmark method
• Android audio capture overview
• Android AudioRecord reference
• Android audio focus overview

Chinese deployment and troubleshooting content is organized in the .奇连 AI 端侧专题.

Frequently asked questions