Design an Android Native Inference Interface

Direct answer

A native inference interface should describe the application task and lifecycle, not leak one vendor’s engine objects into UI code. Define a small contract for initialization, one request or stream, cancellation, and close; represent progress and failures with app-owned types; keep model-path and backend configuration in an adapter; and test the contract with a fake before connecting a native runtime. The interface does not make two runtimes interchangeable. It gives the app a controlled seam where different adapters can be evaluated against the same acceptance checks without moving native session types into a ViewModel or screen. The boundary should make ownership and concurrency explicit. Decide whether one process-level coordinator owns a single engine, whether conversations are isolated, and which component is allowed to close native state. For most application screens, a ViewModel should own task state and call a repository or runtime coordinator; a Composable should render state and send user actions. Avoid creating engines in UI constructors or recomposition. Native initialization can be slow, so expose an initializing state and do not block the main thread. A stream should have defined behavior when its collector disappears: either the request remains owned and can be resumed, or cancellation propagates to native work and resources are released. Use typed errors rather than a single string such as “AI failed.” File not found, artifact mismatch, engine initialization, unsupported operation, cancellation, malformed response, and out-of-memory require different recovery and user guidance. Store technical details in a privacy-safe diagnostic record, but keep personal prompt text out unless the user explicitly chooses to share it. Define output parsing outside the native adapter if product logic owns the schema; keep token-generation concerns and task validation separate. The interface should also resist accidental concurrency. A single native runtime may not support concurrent use of the same conversation or engine. Serialize requests when the runtime contract requires it; if independent sessions are supported, prove the behavior rather than assuming it. Every operation should carry a request identifier so that a cancelled or older response cannot overwrite a newer screen state. On process recreation, rebuild the runtime from durable artifact identity and task metadata, not from an in-memory native handle. Persist only the small amount of user work needed to resume the task, and keep derived output governed by the app’s privacy and deletion rules. A practical contract often has states such as unavailable, initializing, ready, running, cancelling, failed, and closed. It may return a Flow for incremental events, but the event model should distinguish partial text, validated final output, tool request, and failure. The application adapter then translates each runtime’s API to this contract. A fake adapter can test UI behavior, lifecycle changes, cancellation races, empty output, malformed structured results, and fallback without loading a model. A real adapter must still be tested with a pinned artifact and runtime. Do not advertise a shared interface as proof of equivalent quality, speed, format support, or hardware acceleration. A LiteRT-LM adapter and a llama.cpp adapter may differ in artifact format, available APIs, supported operations, and failure behavior. The seam is useful only if it preserves those differences as explicit capability and evidence fields rather than hiding them behind a lowest-common-denominator “works” boolean.

Replacing a runtime without rewriting every screen

An Android app has one Compose screen and a ViewModel that directly imports a native engine class. A team wants to compare a second runtime, add cancellation, and test process recreation, but changing the UI for each experiment has made failures hard to isolate.

Decision

Introduce an application-owned inference contract and two adapters only where their artifact and feature support are verified. Keep UI task state independent from native objects.

Expected outcome

The same UI contract can be tested with a fake and then with each real adapter, while unsupported runtime differences remain visible and documented.

Evidence to collect

The claim becomes reviewable only when the following evidence is attached to the same artifact and test run:

• Interface contract tests
• Fake engine tests
• Runtime adapter tests

A runtime comparison is publishable only when artifact identity, build configuration, selected backend, task fixture, and failures are recorded.

Implementation workflow

1. Write the task contract first. List the user request, input type, accepted output properties, progress events, cancellation behavior, and safe failure state. Keep feature validation outside the runtime interface unless the runtime itself owns that specific constraint.
2. Define owner and serialization rules. Choose which repository or process coordinator creates and closes the engine, whether requests are serialized, and how a request identifier prevents stale responses from updating a newer screen. Document what happens when a collector disappears.
3. Build a fake before native adapters. Use a fake implementation to test initializing, streaming, cancellation, timeout or failure, malformed output, process recreation, and view-model state. The fake must expose delayed and out-of-order events so the UI is not tested only against instant happy paths.
4. Adapt runtimes without flattening differences. Map each native library into the same app-owned states, but keep artifact format, supported backend, and unsupported task capability as explicit metadata. Run the same acceptance fixture on each adapter only when that artifact and operation are supported.

Keep each transition observable. A failure should identify the stage, artifact, runtime, and recovery action without logging private user content.

Acceptance criteria

• No Compose screen or ViewModel stores a native engine or conversation object.
• Every request has a clear owner, cancellation path, close rule, and stale-result guard.
• A fake adapter covers lifecycle and error cases; each real adapter has an artifact-specific test record.

Decision table

Failure patterns to prevent

• Runtime types in UI state
• Unspecified concurrency

Also prevent silent fallback, unpinned artifacts, missing cancellation, and conclusions that combine unlike configurations. Store unsuccessful runs alongside successful ones.

Minimum reproducibility record

Release checklist

• ☐ The primary query is answered without an unsupported number.
• ☐ Every artifact and runtime is pinned.
• ☐ The representative task and failure policy are explicit.
• ☐ Lifecycle, cancellation, cleanup, and fallback are tested.
• ☐ User-content and network boundaries are documented.
• ☐ Result wording applies only to the recorded configuration.
• ☐ The page links to raw method or evidence when results are added.

Sources and related evidence

• LiteRT-LM repository
• llama.cpp repository
• Google AI Edge LiteRT-LM Kotlin API
• Android Developers app architecture guide
• Android Developers activity lifecycle

Chinese deployment and troubleshooting content is organized in the .奇连 AI 端侧专题.

Frequently asked questions