Runtime Health Checks for Local AI

Direct answer

A useful on-device AI health check is a sequence of separately named observations, not a green light inferred from a filename. Check that the expected artifact is present; verify whatever trusted identity data the application actually has; initialize the selected runtime; record the result of backend setup without confusing a requested backend with an observed one; then run a small, representative task and validate its output. Each stage needs its own outcome and recovery path. A file can exist and still be incomplete, mismatched, unreadable, or unsupported. A runtime can initialize while a task later fails. A task can return text that does not satisfy the feature contract. Keeping these distinctions visible makes a diagnostic actionable instead of reassuring for the wrong reason. This distinction is especially important when model delivery and native inference are maintained by separate components. LiteRT-LM is a runtime with its own packaging and API surface; a model-family name alone does not establish that an artifact belongs to a particular runtime. In the current Cove code, ModelManager represents not-downloaded, downloading, downloaded, loading, ready, and error states, serializes loading, and removes the artifact after a load failure before returning to not-downloaded. The Gemma 4 E2B SHA-256 field in the current model configuration is empty. Therefore, do not describe that configured artifact as hash-verified, and do not let a “ready” badge silently imply cryptographic provenance that the application does not hold. Design the probe around a real user-visible task but make it safe to repeat. Use a small fixed fixture rather than private user material, establish expected properties before running, and record a classified failure if a stage fails. The check should be bounded: do not run a costly generation on every launch just to paint a status icon, and do not automatically retry an error indefinitely. Separate a lightweight inventory check from an explicit diagnostic run. Show users what has been checked, when it was checked, and what remains unknown. If hardware-backend identity is not exposed by the chosen runtime, say “backend not confirmed” rather than guessing from a device label or configuration preference. A release report should name the exact app build, artifact revision or source, runtime revision, selected backend request, observed backend evidence if available, test fixture, and task result. It should state device and OS only for the run actually performed. It should not generalize to all phones, all variants, or all future updates. The purpose of the health check is controlled diagnosis and honest readiness—not a benchmark, compatibility database, or certification.

A settings screen says the model is ready, but no task has been tried

A user installed an application update and sees a model file in app storage. The prior process may have been killed during load, the app configuration may have changed, and support needs to know whether the artifact, runtime, and actual feature work independently. The team has no new device-wide benchmark to rely on.

Decision

Replace a single readiness boolean with a staged, bounded diagnostic. Keep file presence, artifact verification, runtime initialization, backend observation, and task acceptance separate.

Expected outcome

The user and support reviewer can see the last confirmed stage and a safe next action, without the interface claiming unsupported compatibility or hash verification.

Evidence to collect

The claim becomes reviewable only when the following evidence is attached to the same artifact and test run:

• Artifact verification
• Runtime load probe
• Representative task result

A runtime comparison is publishable only when artifact identity, build configuration, selected backend, task fixture, and failures are recorded.

Implementation workflow

The review records and compares three forms of evidence: confirming that an artifact verification step has run, that a runtime load probe has succeeded, and that a representative task result has been obtained. Each item represents a different stage of the check sequence, and aligning them side by side clarifies where confidence is established and where further evidence would still be required. (54 words)

1. Inventory the exact inputs. Record the app build, runtime package and resolved version, configured model identifier, storage location, and any expected artifact length or digest that is genuinely present in trusted configuration. If a digest field is empty, record “digest unavailable”; do not derive trust from the filename, download completion flag, or model-family label.
2. Check the artifact without loading it. Confirm that the expected file is accessible to the app and compare its length with a trusted manifest only when such a manifest exists. Distinguish absent, partial, unreadable, and identity-mismatch outcomes. Keep this inventory probe cheap and avoid parsing or mapping large model data merely to render a settings page.
3. Initialize one owned runtime instance. Ask the application-level runtime owner to initialize the selected engine once, outside UI composition and the main thread. Record the returned state and preserve the error stage. Do not start overlapping load attempts while cleanup from an earlier attempt is still running.
4. Run a bounded representative task. Use a fixed, non-sensitive input tied to the feature and define the expected output properties first. Capture completion, rejection, timeout, or parser failure distinctly. If the runtime can identify its actual backend, record that evidence; if not, retain only the requested setting and mark execution as unconfirmed.
5. Review recovery and user-facing meaning. For each failure, offer a relevant next action such as retry after a user-initiated repair, select another documented artifact, or contact support with redacted stage metadata. Re-run only after state is clean. Label the result with its timestamp and scope, and explain that one successful task is not a universal device-support guarantee.

Acceptance criteria

• The health result names the last completed stage and never equates file presence, a downloaded state, or runtime initialization with task acceptance.
• A missing or unverified digest is disclosed as unavailable; the interface does not claim the current Cove artifact is hash-verified.
• The diagnostic is bounded, uses a fixed non-sensitive fixture, and does not launch duplicate native sessions or loop retries.
• Any compatibility statement is scoped to the exact recorded app, artifact, runtime, device, OS, and task run.

Decision table

Failure patterns to prevent

• Calling file existence healthy
• Running a costly task at every launch

Two pitfalls commonly distort these assessments. Declaring the system healthy simply because a file existence check passes ignores every layer that follows, while running a costly task at every launch imposes an unreasonable burden on routine startup. Release language should therefore be limited strictly to what the recorded evidence directly supports, without extrapolation. (54 words)

Minimum reproducibility record

Release checklist

• ☐ The primary query is answered without an unsupported number.
• ☐ Every artifact and runtime is pinned.
• ☐ The representative task and failure policy are explicit.
• ☐ Lifecycle, cancellation, cleanup, and fallback are tested.
• ☐ User-content and network boundaries are documented.
• ☐ Result wording applies only to the recorded configuration.
• ☐ The page links to raw method or evidence when results are added.

Sources and related evidence

• LiteRT-LM official Kotlin API and getting-started documentation
• Android activity and process lifecycle

Chinese deployment and troubleshooting content is organized in the .奇连 AI 端侧专题.

Frequently asked questions