Direct answer
Diagnose apparent prompt truncation by comparing the exact text at each boundary in the request pipeline. Start with the source input the user supplied, then capture the normalized application input, rendered chat template, tokenizer or runtime input representation if the selected API exposes it, configured output budget, and the response that the application actually receives. Look for the first boundary at which expected material is missing. A short answer is not by itself proof that the prompt was truncated: the model may prioritize differently, the prompt may be ambiguous, output generation may stop early, or the application parser may discard content. Conversely, seeing the full text in a log does not prove that the runtime consumed every token. Treat the exact model artifact, template, tokenizer, runtime, decoding configuration, and input fixture as a single reproducibility record. LiteRT-LM is its own runtime and packaging environment, so do not transplant tokenizer assumptions or formatting instructions from GGUF, MediaPipe, or a different model guide simply because the task sounds similar. If the runtime does not expose token counts or the final serialized request, say that the exact truncation boundary cannot be observed through this integration. Do not invent a context-window size from a model-family page or use character count as a token count. Build a boundary corpus with short, normal, and deliberately long inputs; include markers at the beginning, middle, and end, and a required-fact checklist. Test one change at a time: template, source normalization, context allocation, output budget, then parser. Record the full prompt only in controlled test data that contains no private user content. In production diagnostics, use redacted length and stage metadata rather than storing user prompts or generated answers. For each run, define whether the app clips source text, truncates conversation history, rejects oversize input, or asks the user to shorten it. Silent clipping is difficult to explain and can create dangerous omissions for tasks where preserving source facts matters. The repository has no complete real-device benchmark dataset, and its memory facts caution that runtime buffers and token-related working state are not inferable from model download size alone. Therefore this article provides a diagnosis protocol, not a claim about any specific device’s maximum context, memory capacity, or response quality. The reliable outcome is a traceable classification of which layer dropped or transformed content and a regression fixture that catches the same boundary after future changes.
The generated summary repeatedly misses the final paragraph
A user pastes a long document into an Android app. The output is fluent but omits a required fact near the end. The app recently changed the prompt template and now needs to identify whether text was clipped before inference, displaced by template content, stopped by an output limit, or lost during parsing.
Decision
Compare exact pipeline boundaries with a marker-based fixed fixture, changing one layer at a time. Do not label the symptom “context limit” until the missing boundary is observed.
Expected outcome
The team can point to a reproducible transformation or classify the cause as not observable through the current runtime integration, then add a regression check without retaining private documents.
Evidence to collect
The claim becomes reviewable only when the following evidence is attached to the same artifact and test run:
- Rendered prompt fixtures
- Tokenizer and context policy
- Boundary, truncation, and user-visible behavior
A release claim is valid only for the recorded configuration and acceptance set. Generalization requires additional records.
Implementation workflow
- Freeze the execution identity. Record the exact artifact/revision, runtime package and version, chat template, tokenizer identity if known, generation configuration, app build, and input fixture. Do not compare traces if multiple variables changed between them.
- Add boundary markers and an acceptance checklist. Create a non-sensitive synthetic input with unique markers near the beginning, middle, and end plus several required facts. Define what constitutes preservation, clipping, and acceptable summarization before looking at generated answers.
- Trace each observable request layer. Capture source text, application normalization result, rendered template, and runtime-serialized or tokenized input only when the API exposes them. Record the output budget and stop condition. Mark hidden layers as unobservable instead of estimating them from character count.
- Isolate transformation from model behavior. If a marker is absent before the runtime boundary, investigate app clipping or template assembly. If all input is visible but generation ends early, examine stop and output-budget behavior. If output contains the fact but the UI omits it, inspect parsing and rendering. Repeat the fixed fixture to distinguish deterministic plumbing from model variability.
- Choose and test explicit product behavior. Select a user-visible policy for oversized inputs—reject with an explanation, split using a documented method, or summarize only after disclosing the scope. Add unit fixtures for each boundary and keep private production content out of telemetry and logs.
Keep each transition observable. A failure should identify the stage, artifact, runtime, and recovery action without logging private user content.
Acceptance criteria
- The team can identify the last pipeline stage where all required markers are observed, or explicitly documents that the native integration hides that boundary.
- Character count is not presented as a token count, and no context limit is asserted without evidence for the exact artifact and runtime.
- A fixed synthetic corpus detects clipping, output stopping, and parser loss as separate failure classes.
- Production diagnostics do not store raw private prompts or generated text merely to troubleshoot truncation.
Decision table
| Dimension | Choice or evidence | How to interpret it |
|---|---|---|
| Source/app boundary | Compare supplied input with normalized input | Finds app-side clipping, normalization, or input-field limits before template construction. |
| Template/runtime boundary | Inspect serialized request or tokenizer output only if exposed | Can show template expansion or runtime input transformation; hidden boundaries must remain explicitly unknown. |
| Output/UI boundary | Compare raw response, stop state, parser result, and rendered view | Separates generation stopping from application-side filtering or display loss. |
Failure patterns to prevent
- Blaming model quality before inspecting input
- Truncating source text silently
Also prevent silent fallback, unpinned artifacts, missing cancellation, and conclusions that combine unlike configurations. Store unsuccessful runs alongside successful ones.
Minimum reproducibility record
| Layer | Record |
|---|---|
| Device | Manufacturer, model, chipset, RAM class, operating-system build |
| Software | Application version and git commit |
| Model | Family, variant, revision, format, file length, hash, quantization |
| Runtime | Name, revision, requested backend, observed backend evidence |
| Workload | Fixture revision, input hash, prompt hash, output policy |
| Outcome | Completed, failed, cancelled, fallback, and privacy-safe diagnostics |
Release checklist
- ☐ The primary query is answered without an unsupported number.
- ☐ Every artifact and runtime is pinned.
- ☐ The representative task and failure policy are explicit.
- ☐ Lifecycle, cancellation, cleanup, and fallback are tested.
- ☐ User-content and network boundaries are documented.
- ☐ Result wording applies only to the recorded configuration.
- ☐ The page links to raw method or evidence when results are added.
Sources and related evidence
Chinese deployment and troubleshooting content is organized in the .奇连 AI 端侧专题.
Frequently asked questions
Does a short answer mean the model hit its context limit?
No. The model may have summarized selectively, the input may be ambiguous, generation may have stopped, or the app may have filtered output. Compare observable request and response boundaries before assigning a cause.
Can I estimate token usage by dividing characters by a constant?
Do not treat that as an exact token count. Tokenization depends on the artifact and tokenizer. Use an official exposed tokenizer or runtime measurement when available; otherwise mark the count as unknown.
Should production logs contain the complete prompt to diagnose clipping?
Usually no. Reproduce the issue with synthetic fixtures, and use redacted stage/length metadata in production. Raw prompts and outputs can contain private content and should not be retained by default for debugging convenience.