Direct answer
Keep model loading and inference away from Android's main thread, but do not solve responsiveness by launching arbitrary background jobs from each button press. Define one application-owned execution boundary that owns the native runtime, states whether requests are serialized or queued, and communicates bounded progress and completion to UI state. Android’s threading guidance explains that the main thread dispatches input and drawing work and should not be blocked by long operations. It also warns that UI objects are not thread-safe. This leads to two separate rules: native or expensive work belongs behind an appropriate worker boundary, while rendering state is delivered back through a safe, lifecycle-aware mechanism. Thread placement alone does not establish concurrency safety. A native engine may have mutable session state, shared memory, or non-reentrant calls. Unless the selected runtime documentation and implementation prove parallel calls are supported for the exact configuration, start with a single owner and explicit serialization. A coroutine cancellation request is also not proof that a blocking native call stopped. The adapter must define how cancellation is observed, whether the call can be interrupted, when cleanup completes, and when a subsequent request may begin. Until cleanup is confirmed, a “cancelled” UI label must not allow a second request to race the first native operation. The current Cove ModelManager serializes model loading. Treat that source fact as a property of model loading, not a blanket guarantee that every inference path is thread-safe. Use separate lifecycle and concurrency tests for load, inference, close, screen recreation, and failure. Keep runtime handles out of Compose state and avoid tying a process-level engine to a screen object that can be destroyed. A ViewModel may own observable task state, while a longer-lived repository or coordinator owns the native resource; exact ownership should match the app’s intended task lifetime. Build tests around event order, not just final text. Submit a request, recreate the screen, cancel, close, issue another request, and assert that there is no duplicate engine, stale collector, main-thread violation, or out-of-order result. Include errors and timeouts as first-class states. Measure responsiveness only with a defined method and raw run records; this guide does not claim a particular speed improvement. The objective is preserving responsiveness and resource correctness on tested configurations, not claiming universal performance.
A generate button freezes the screen during the first model load
The first user action triggers model initialization and a native inference call. A loading spinner is shown, but taps stop responding; after navigation away and back, the same task may be submitted again. The team needs a threading contract that handles ownership, cancellation, and repeated requests.
Decision
Move work behind one runtime-owning coordinator, serialize unless concurrency is explicitly supported, and expose task state independently from the screen lifecycle.
Expected outcome
The interface remains able to render state changes, duplicate requests are governed by policy, and cancellation has a testable completion boundary.
Evidence to collect
The claim becomes reviewable only when the following evidence is attached to the same artifact and test run:
- Main-thread policy check
- Cancellation trace
- UI responsiveness test
A runtime comparison is publishable only when artifact identity, build configuration, selected backend, task fixture, and failures are recorded.
Implementation workflow
- Map ownership before choosing a dispatcher. Draw which object owns the engine, conversation/session, request queue, and UI state. Select a worker execution context based on the runtime’s blocking behavior, while ensuring no Compose or View object crosses onto that worker. Record which owner survives ordinary navigation and which is intentionally closed.
- Specify concurrency and back-pressure. Choose a policy such as one active request with a visible queued or rejected state. Do not create an unbounded coroutine per click. If the engine is shared, define which requests may share a session and how conversation identity is isolated. Treat parallel inference as unsupported until exact runtime evidence says otherwise.
- Define cancellation as a protocol. Write down when cancellation is requested, how the native adapter observes it, what partial output is retained, when buffers and sessions are released, and which signal means a new request is safe. If the native call cannot be interrupted, report cancellation as pending cleanup rather than claiming it stopped immediately.
- Bridge results to lifecycle-aware UI state. Publish immutable state updates through a stream or state holder collected by the visible screen. Do not update UI objects from the worker. Make a screen collector’s cancellation distinct from cancellation of the underlying application-owned task.
- Test races and responsiveness boundaries. Exercise double tap, navigation, recreation, cancellation during load and inference, engine close, error, timeout, and next-request start. Assert order and ownership. Capture main-thread diagnostics and defined responsiveness traces; do not convert a single timing run into a device-family performance promise.
Keep each transition observable. A failure should identify the stage, artifact, runtime, and recovery action without logging private user content.
Acceptance criteria
- No long model load or inference call executes in a UI callback or blocks rendering; all UI state changes return through a thread-safe main-thread mechanism.
- The coordinator has a documented request concurrency policy and never launches unbounded native work from repeated taps.
- Cancellation is not reported as fully complete until the adapter’s documented stop or cleanup boundary has been observed.
- Lifecycle tests prove that screen recreation neither creates duplicate runtime ownership nor accidentally cancels work intended to outlive the screen.
Decision table
| Dimension | Choice or evidence | How to interpret it |
|---|---|---|
| Launch work from every Composable | Avoid | Composition and recomposition are not stable native-resource ownership boundaries. |
| One serialized runtime coordinator | Default until proven otherwise | Makes ownership, ordering, cancellation, and recovery observable and testable. |
| Parallel native requests | Enable only with exact runtime support and race tests | May improve throughput in some designs but can corrupt shared session state if assumed. |
Failure patterns to prevent
- Blocking a Composable
- Spawning unbounded inference jobs
Also prevent silent fallback, unpinned artifacts, missing cancellation, and conclusions that combine unlike configurations. Store unsuccessful runs alongside successful ones.
Minimum reproducibility record
| Layer | Record |
|---|---|
| Device | Manufacturer, model, chipset, RAM class, operating-system build |
| Software | Application version and git commit |
| Model | Family, variant, revision, format, file length, hash, quantization |
| Runtime | Name, revision, requested backend, observed backend evidence |
| Workload | Fixture revision, input hash, prompt hash, output policy |
| Outcome | Completed, failed, cancelled, fallback, and privacy-safe diagnostics |
Release checklist
- ☐ The primary query is answered without an unsupported number.
- ☐ Every artifact and runtime is pinned.
- ☐ The representative task and failure policy are explicit.
- ☐ Lifecycle, cancellation, cleanup, and fallback are tested.
- ☐ User-content and network boundaries are documented.
- ☐ Result wording applies only to the recorded configuration.
- ☐ The page links to raw method or evidence when results are added.
Sources and related evidence
Chinese deployment and troubleshooting content is organized in the .奇连 AI 端侧专题.
Frequently asked questions
Is moving inference to Dispatchers.IO enough?
No. A dispatcher can keep a blocking call off the UI thread, but it does not define native ownership, concurrency, cancellation, cleanup, or result ordering. Those need an application-level contract and tests.
Does cancelling a coroutine stop native inference immediately?
Not necessarily. Cancellation is cooperative at the coroutine boundary; a blocking native call may need a separate supported interruption mechanism. Keep cancellation pending until the adapter confirms the operation has stopped or cleanup has completed.
Should a ViewModel own the native engine?
It depends on the intended task lifetime and runtime contract. A ViewModel can expose UI task state, while a repository or coordinator may need to own a resource that should survive screen recreation. Document and test the chosen lifetime.