Session Timeout and Cleanup for Local AI

Direct answer

A session timeout and a request deadline solve different problems. An idle-session time-to-live decides how long a conversation owner may retain a native session after useful activity stops. A per-request deadline bounds one inference attempt that is already running. Expiring the former can close or recreate conversation state; reaching the latter requests that the active operation stop and then requires a terminal outcome. Combining both into one timer makes user-visible behavior and cleanup difficult to reason about, especially when a new turn begins near the expiry boundary. Assign each timer to the layer that owns the state. A conversation or session coordinator can own the idle timer because it sees accepted turns, user close actions, app lifecycle changes, and session identity. The inference operation owner can apply a request deadline because it knows when native work began and which terminal callback belongs to that request. A screen should not own a conversation lifetime merely because it displays the transcript, and a UI Flow collector cancelling does not by itself establish that native work stopped. If an application wrapper exposes only a cold Flow, inspect its actual cancellation behavior before promising a stop action. LiteRT-LM at the cited upstream commit exposes Conversation.cancelProcess. Its source says this call is a no-op when there is no active inference and throws if the Conversation is not alive. The same source maps native cancellation status 1 to CancellationException, and contains a comment that rollback of internal state is still a pending feature request. These details are version-specific. They do not prove the resolved Android dependency in a project includes the method, that an app-facing engine interface exposes it, that cancellation restores a reusable conversation, or that KV/cache memory has been released. Treat cancellation as a state transition with acknowledgement, not as a Boolean that the UI flips. Record request identity, cancellation cause, the native terminal callback or exception, and whether cleanup completed. If a deadline races with normal completion, associate the final event with the correct request and commit it once. A return from cancelProcess only reports that the cancellation call returned; it does not prove native inference has reached a terminal state. Wait for the terminal callback or exception for that request and verify no operation remains in flight. Only then, if session ownership and the resolved API permit, may the coordinator run a supported close-and-recreate sequence. Never force-close an active or still-cleaning Conversation. Keep user cancel, idle TTL expiry, and request deadline as distinguishable reasons. Kotlin coroutine timeouts cancel the coroutine block cooperatively; they do not automatically terminate arbitrary native or blocking work. A timeout wrapper is therefore only one signal in the policy. The application must propagate cancellation through an actual runtime API where available, continue observing completion, and prevent a replacement request from racing into a session that is still executing or cleaning up. If the resolved runtime lacks a cancellation method, represent the operation as stopping until its terminal callback arrives. A close-and-recreate path is allowed only after the matching request is terminal, no work is in flight, and the session owner confirms that the resolved API supports that sequence. Do not use close as a force-stop or claim immediate resource release because a timer fired. Define reuse rules for each terminal state. A request that completes normally may leave the conversation available for another turn if the API and application state permit it. After cancellation, deadline, or native exception, wait for the matching request terminal state and confirm no in-flight operation remains before the owner decides whether to reuse or, when supported, close and recreate. An idle-expired session can be recreated from application-owned history only according to the product’s replay policy; do not assume native hidden state can be serialized. In this repository, the app-facing AiInferenceEngine interface exposes Flow-returning task methods but no cancel or close method, while the concrete InferenceEngine has its own close path. Do not quietly imply that current ViewModels can invoke a native session API through that interface. Memory cleanup must be measured separately from control flow. A closed conversation or completed cancellation is not evidence that every native allocation or KV cache returned to the operating system. If resource release matters, run a repeatable workload on a named build, artifact, runtime, and device, and observe available memory diagnostics after cleanup and subsequent work. The repository fact supports using Android Studio Memory Profiler for observations; it does not supply a universal session TTL, memory threshold, or guaranteed release result. Report the method and the measured outcome only when raw observations exist. A robust policy writes down which event resets idle time, whether the timer pauses during an active request, how close and app backgrounding interact, what the user sees while stopping, and which outcomes permit reuse. Use a monotonic clock for elapsed time rather than wall-clock timestamps, attach a generation ID so an old timer cannot close a newer session, and cancel the timer when ownership ends. Keep transcript retention independent from native session cleanup. Tests should cover timer races, active work at expiry, user close, request timeout, cancellation acknowledgement, recreation, and memory observation without baking an unsupported default duration into product guidance.

An idle chat expires while a request deadline is shorter

A chat screen starts a long-running local answer. Separately, the product wants to release an inactive native Conversation after a period with no accepted user turns. The request may finish, be cancelled by the user, exceed its own deadline, or still be cleaning up when the idle timer fires. A lifecycle owner must not let an old timer close a newer conversation.

Decision

Use distinct session and request state machines, bind each timer to a session or request generation, and wait for a native terminal outcome before deciding whether the same Conversation may be reused.

Expected outcome

The UI can distinguish idle expiry from request cancellation or timeout. Reuse occurs only when the resolved API and a version-specific test establish it; otherwise the coordinator recreates the session and records cleanup as observed or unknown.

Evidence to collect

The claim becomes reviewable only when the following evidence is attached to the same artifact and test run:

• Ownership timeline
• Timeout and resume test
• Cleanup verification

A runtime comparison is publishable only when artifact identity, build configuration, selected backend, task fixture, and failures are recorded.

Implementation workflow

1. Name session and request lifetimes independently. Write separate states for idle, active request, stopping, completed, expired, closed, and recreated conversation state. Give the session coordinator ownership of idle expiry and the request owner control of per-call deadlines.
2. Check the resolved runtime and wrapper contracts. Resolve the actual Gradle component and inspect the available Conversation methods and app-facing adapter. Do not infer a cancel or close method from an upstream commit, and do not assume Flow collection cancellation reaches native inference.
3. Define terminal events and reuse decisions. Map normal completion, user cancel, deadline, native error, idle expiry, and close to distinct outcomes. A cancelProcess return only reports that the call returned; require the matching request terminal callback or exception and verify no operation remains in flight before any reuse or owner-approved close-and-recreate sequence. Never force-close an active Conversation.
4. Make timer races generation-safe. Use an elapsed-time source and a session generation ID. Reset idle time only for specified accepted activity, define whether active work suspends expiry, and ensure an old scheduled timer cannot close a replacement Conversation.
5. Separate control acknowledgement from memory evidence. Record cancellation cause and terminal state separately from memory observations. If resource release is a requirement, measure before and after a repeatable workload with the exact artifact, runtime, build, and device rather than treating close or callback completion as proof of cache release.
6. Exercise lifecycle and timeout boundaries. Test expiry with no active request, expiry during active work, request timeout near normal completion, user cancel, closed-session cancel, retry, app backgrounding, and session recreation. Verify transcript retention is independent from native session cleanup.

Keep each transition observable. A failure should identify the stage, artifact, runtime, and recovery action without logging private user content.

Acceptance criteria

• Idle-session expiry and per-request deadlines have separate owners, reset conditions, reasons, and user-visible states.
• A new request cannot reuse or close/recreate a Conversation until the prior request reaches its matching terminal outcome and no work is in flight; reuse and close behavior must be supported by the resolved version and session owner.
• Cancellation acknowledgement and memory release are reported as different evidence; no universal TTL, immediate native stop, or guaranteed KV-cache release is asserted.

Decision table

Failure patterns to prevent

• Closing active inference abruptly
• Keeping temporary media indefinitely

Also prevent silent fallback, unpinned artifacts, missing cancellation, and conclusions that combine unlike configurations. Store unsuccessful runs alongside successful ones.

Minimum reproducibility record

Release checklist

• ☐ The primary query is answered without an unsupported number.
• ☐ Every artifact and runtime is pinned.
• ☐ The representative task and failure policy are explicit.
• ☐ Lifecycle, cancellation, cleanup, and fallback are tested.
• ☐ User-content and network boundaries are documented.
• ☐ Result wording applies only to the recorded configuration.
• ☐ The page links to raw method or evidence when results are added.

Sources and related evidence

• Android Memory Profiler
• LiteRT-LM Conversation.kt at the pinned upstream commit: cancelProcess behavior and status mapping
• LiteRT-LM Kotlin getting-started guide at the same pinned commit
• Kotlin kotlinx.coroutines: withTimeout cancellation and cooperative behavior
• Android Developers: Android Studio Memory Profiler

Chinese deployment and troubleshooting content is organized in the .奇连 AI 端侧专题.

Frequently asked questions