Manage Multi-Turn State with LiteRT-LM

Direct answer

A multi-turn chat has at least three kinds of state: durable conversation history owned by the application, transient UI state owned by the screen, and runtime conversation/session state owned by the inference engine. Treating these as one object creates subtle bugs: a rotation can start a new conversation, a retry can append the same user turn twice, or a new chat can inherit messages from the previous one. LiteRT-LM's official documentation describes a stateful conversation abstraction around the model runtime, while Android state guidance distinguishes UI retention and durable persistence. The application should make that ownership boundary explicit without assuming a runtime session is a database. Define a conversation identity before sending the first turn. The application can persist a conversation ID, ordered user-visible messages, timestamps, and a state such as active, closed, or archived. Runtime state can be reconstructed from that record according to the selected model and prompt policy. Do not persist native pointers or internal caches as chat history. If the runtime maintains hidden context that cannot be serialized, document what it means to create a fresh conversation and what portion of the visible history is supplied again. The exact replay policy must be tested for the runtime revision in use. Handle each turn as a transaction. Assign a turn ID, mark it pending, and append its accepted user message once. Generated output can be partial, complete, rejected, or cancelled. Only commit a final assistant message when the operation reaches the declared terminal state. If the UI is recreated during streaming, the new screen observes the existing turn ID and does not send it again. If the process dies, recover the durable messages, mark any incomplete turn, create runtime state again if supported, and tell the user whether to retry. Never silently treat a partial generation as a completed assistant message. Reset and cancellation are different operations. Cancelling a turn stops the current work but may leave the conversation available for another turn; resetting a conversation intentionally clears or replaces its context. A new conversation should receive a new identity and fresh runtime state. Deleting the visible message list while retaining the old native session can leak previous context into later answers. Conversely, closing an engine during a screen disposal can terminate work still owned by the conversation coordinator. Specify the lifetime of the engine, conversation, and each turn separately. Build tests that make ordering visible. Send a known sequence such as user A, assistant A, user B, then assert the second turn belongs to the intended conversation and that reset creates an isolated context. Inject cancellation during a streaming turn, recreate the screen, and verify the turn ID is not submitted twice. Start two conversation IDs and deliberately interleave their scheduling; prove that state does not cross. A fake runtime can test application ownership and event order, while a runtime-backed test verifies the supported behavior of the pinned library. Do not treat a fake as proof of model quality. Persist the minimum needed for recovery. Large transcripts may belong in a database; bundles and saved-state mechanisms should carry small IDs and cursor state. Apply retention and deletion policy to history separately from runtime cleanup. If the feature is local-only, do not add a remote conversation store just to simplify synchronization. Avoid logging message text to diagnose state leakage; log conversation and turn IDs, state transitions, model revision, and sanitized failure categories instead. The repository facts identify a LiteRT-LM runtime path and explicit Cove lifecycle management, but they do not prove a particular app's multi-turn implementation or any release-specific API behavior. Use the official docs for the exact library revision and test on the project artifact. Report whether state persistence, recreation, reset, and cancellation were verified; do not claim every conversation survives process death. A clean state ownership model makes failures traceable and avoids cross-chat leakage even when the model output itself is nondeterministic.

A user starts a second chat after cancelling a streamed reply

The first local conversation has a pending streamed answer. The user cancels it, opens a new chat, and the screen is recreated before the next turn finishes. A delayed native callback may arrive afterward.

Decision

Give the new chat a fresh conversation ID and runtime context; keep the first turn cancelled and prevent delayed callbacks from entering the new transcript.

Expected outcome

Messages remain associated with the correct chat, the cancelled answer is not falsely committed, and runtime cleanup follows explicit ownership.

Evidence to collect

The claim becomes reviewable only when the following evidence is attached to the same artifact and test run:

• Conversation-state schema
• Rendered prompt fixtures
• Reset and process-recreation tests

A runtime comparison is publishable only when artifact identity, build configuration, selected backend, task fixture, and failures are recorded.

Implementation workflow

1. Draw the three state owners. Document app-persisted messages and conversation IDs, screen-only controls and scroll state, and runtime conversation/session resources. Identify which layer is authoritative for every transition.
2. Define identity and turn transactions. Assign one conversation ID and unique turn IDs. Persist accepted user messages once, model partial output separately, and commit a final response only after the operation reaches its terminal state.
3. Specify reset, cancel, close, and retry. Treat turn cancellation as distinct from conversation reset. Define engine and session lifetimes and how a retry receives new identity without retaining stale hidden context.
4. Test ordering, recreation, and isolation. Replay a known sequence, cancel a streamed turn, rotate the screen, and interleave two conversation IDs. Assert no duplicate submit, stale callback, or cross-chat message occurs.
5. Recover only durable state after process loss. Rebuild runtime state from supported durable messages, mark unfinished turns incomplete, and require a deliberate retry. Do not serialize native handles or promise invisible continuation.

Keep each transition observable. A failure should identify the stage, artifact, runtime, and recovery action without logging private user content.

Acceptance criteria

• Each user-visible conversation and turn has a stable identity that survives screen recreation without duplicate submission.
• Cancellation, reset, close, and retry produce distinct states and do not carry hidden context into a new conversation.
• Interleaved conversation tests prove messages and delayed callbacks remain bound to the correct conversation ID.
• Process recovery uses supported durable messages, labels incomplete turns, and does not persist native pointers.

Decision table

Failure patterns to prevent

• Treating native state as durable history
• Mixing conversations after cancellation

Also prevent silent fallback, unpinned artifacts, missing cancellation, and conclusions that combine unlike configurations. Store unsuccessful runs alongside successful ones.

Minimum reproducibility record

Release checklist

• ☐ The primary query is answered without an unsupported number.
• ☐ Every artifact and runtime is pinned.
• ☐ The representative task and failure policy are explicit.
• ☐ Lifecycle, cancellation, cleanup, and fallback are tested.
• ☐ User-content and network boundaries are documented.
• ☐ Result wording applies only to the recorded configuration.
• ☐ The page links to raw method or evidence when results are added.

Sources and related evidence

• LiteRT-LM repository
• LiteRT-LM Kotlin API and conversation lifecycle
• Android activity state changes and retained state

Chinese deployment and troubleshooting content is organized in the .奇连 AI 端侧专题.

Frequently asked questions