Repeated Token Loops in Local AI Output

Direct answer

A repeated-token loop is an output symptom with several possible causes. First determine whether the repetition exists in the runtime’s completed response or appears only after the application assembles stream events. Then verify the exact prompt and conversation history sent to the runtime, artifact and tokenizer identity, template, decoding configuration, and stop/finalization state. A repeated phrase can be generated by the model, included twice in the prompt, appended twice by a collector, or replayed from restored UI state. Those cases require different fixes; turning temperature or another sampling value without a controlled trace can hide rather than locate the issue. Use a versioned synthetic fixture with a short prompt that previously exposed the pattern, plus a clean single-turn control and a multi-turn case. Freeze artifact, runtime, prompt template, decoding settings, app build, and stream collector behavior. Record event sequence numbers if exposed, prompt-history entries, each chunk, terminal event, and final parsed output. If the runtime only returns a final string, note that chunk-level evidence is unavailable. Compare raw prompt length and content without storing real user conversations. Never interpret repeated visible words as a universal model defect from one example. Inspect the prompt history first. An application may append the latest user turn to a conversation that already contains it, restore a prior stream after navigation, or retry a request with stale state. Compare the exact rendered conversation against the intended sequence, including role boundaries and turn endings. Then examine stream lifecycle: one request ID should have one collector owner, partial text should be appended once, and a completed or cancelled request should not emit into a later task. A screen observer restarting does not necessarily mean the native generation restarted or stopped. Only after input and app delivery are ruled out should the team test decoding or model behavior. Change one documented decoding parameter at a time on the same fixture, run repeated trials, and report all outputs rather than the best one. A prompt or stop-token change is a separate experiment and must be pinned to the exact model format. Do not prescribe a universal temperature, repetition penalty, or stopping token unless the selected runtime and model documentation support it. Include termination reason and cancellation behavior in the record. This is a test method, not a performance or model-quality result. The repository benchmark fact requires raw device and configuration records before quantitative claims. The goal is to label the first boundary where repetition appears, preserve a regression fixture, and provide a clear user recovery such as stop, retry with a clean conversation, or edit the prompt. Keep private conversation text out of routine diagnostics; synthetic fixtures and redacted stage metadata are sufficient for most reproductions.

The final sentence appears twice after returning to a chat screen

A user sends one prompt and sees the model repeat its last phrase after navigating away and back during streaming. The team does not know whether the native runtime generated the phrase twice or the screen observer appended an earlier chunk again.

Decision

Use request IDs and ordered stream traces with a fixed synthetic prompt; compare runtime completion, app accumulation, and restored conversation state separately.

Expected outcome

The fault is assigned to generation, prompt history, stream collection, or restoration, and a regression test catches the same boundary without recording a real conversation.

Evidence to collect

The claim becomes reviewable only when the following evidence is attached to the same artifact and test run:

• Pinned configuration
• Repeated-output fixture
• Cancellation and recovery result

A release claim is valid only for the recorded configuration and acceptance set. Generalization requires additional records.

Implementation workflow

1. Freeze the complete generation configuration. Record artifact and tokenizer identity, runtime revision, prompt/template, conversation history, decode settings, stop configuration, app build, and test fixture. Keep one request per run and assign it an ID.
2. Capture prompt history and rendered input. Compare the intended turns with the exact rendered sequence immediately before runtime submission. Look for duplicated latest user turns, stale history restoration, or role boundaries inserted twice; redact any private content.
3. Trace stream events and finalization. When the API exposes chunks, record request ID, sequence, chunk identity, cancellation, completion, and terminal state. Assert each chunk is consumed once and old collectors cannot write into a later task. If only a final result is exposed, mark intermediate events unavailable.
4. Separate app duplication from generated repetition. Compare raw runtime completion with the final UI text. Repeat the same synthetic fixture in a single-turn control and after screen recreation. Check replayed UI state and retry paths before changing sampling settings.
5. Run one-variable model experiments. Only after prompt and collector behavior are correct, change one documented decoding or stop rule and repeat the full fixture set. Keep every output and termination reason; do not publish a universal setting from a favorable sample.

Keep each transition observable. A failure should identify the stage, artifact, runtime, and recovery action without logging private user content.

Acceptance criteria

• The first boundary where repeated text appears is identifiable, or explicitly marked unobservable.
• Exact artifact, prompt history, runtime, decoding, stop configuration, app build, and request identity are recorded.
• Stream chunks or completed output are not appended twice after navigation, cancellation, retry, or state restoration.
• Any decoding change is an isolated experiment and does not become a universal fix claim without broader raw evidence.

Decision table

Failure patterns to prevent

• Publishing a universal decoding fix
• Dropping loop failures from evaluation

Also prevent silent fallback, unpinned artifacts, missing cancellation, and conclusions that combine unlike configurations. Store unsuccessful runs alongside successful ones.

Minimum reproducibility record

Release checklist

• ☐ The primary query is answered without an unsupported number.
• ☐ Every artifact and runtime is pinned.
• ☐ The representative task and failure policy are explicit.
• ☐ Lifecycle, cancellation, cleanup, and fallback are tested.
• ☐ User-content and network boundaries are documented.
• ☐ Result wording applies only to the recorded configuration.
• ☐ The page links to raw method or evidence when results are added.

Sources and related evidence

• Cove benchmark method
• LiteRT-LM official Kotlin conversation and streaming API guide
• Google Gemma chatbot conversation history guide

Chinese deployment and troubleshooting content is organized in the .奇连 AI 端侧专题.

Frequently asked questions