On-Device Intent Classification with Safe Fallback

Direct answer

An on-device intent classifier should select from a versioned, finite taxonomy and be allowed to return “unknown,” “ambiguous,” or “needs clarification.” It should not turn arbitrary generated text into an executable command. `FUNCTION-01` describes a mobile function-calling workflow, while the host-validation fact `GEMMA-FUNCTION-HOST-VALIDATION-2026-01` states that model-generated calls are proposals the app must parse and validate. For a classifier, the same separation is useful: the model proposes a label; application code checks whether it is known, permitted in the current screen, and sufficiently specified before any next step. First define the problem as a routing contract. Name each intent in user language, list representative positive examples, near-neighbor examples, counterexamples, and phrases that should not match. Keep classification separate from action execution. For example, a “search local notes” label can request a UI transition; it should not imply permission to read every file or send content remotely. If the classifier triggers a consequential action, use a separate explicit user confirmation and business rule. Do not allow the model to create new labels at runtime. Constrain the response format and validate it in deterministic code. A schema can require one label identifier plus optional short rationale only if that rationale is safe and useful. Prefer not storing rationale if it can repeat private input. Reject unknown fields or define a forward-compatible version policy. If the runtime returns prose, parse only an exact allowlisted token after normalization rules have been reviewed. Never use substring matching where one label could contain another. Do not treat a fluent explanation as evidence that the selected label is correct. Ambiguity is a product state, not an exception to hide. When the user prompt fits multiple classes, return clarification options or a neutral fallback. For empty input, unsupported language, malformed output, timeout, or model unavailable, retain the original request and show a recoverable message. Do not silently choose the most permissive class. If user wording is outside the taxonomy, do not feed it into an unconstrained “execute this” function. Keep any local action capability in a host-owned allowlist and use typed, minimal arguments. Evaluate using a frozen fixture set with labels and decision rules. Include hard negatives, overlapping intents, short vague text, spelling errors if in scope, multilingual inputs only where supported, prompt-injection attempts, and malformed model responses. Measure per-class confusion and the rate of unknown/clarification states from actual runs; do not invent a score or publish a benchmark until raw records and a documented method exist. Change one variable at a time—taxonomy, prompt, artifact, runtime, or parser—so failures can be attributed. Keep privacy-safe fixtures and do not copy real private messages into logs. Acceptance should combine correct routing with safe rejection. A classifier that gets many easy examples right but executes an ambiguous or unsupported label is not acceptable. Test authorization independently: even a valid intent label cannot grant a permission, consent to network transfer, or bypass user confirmation. This is especially important when local inference is combined with a network route; the route policy must be application-owned and transparent. The result is a narrow classifier that can be improved with evidence, while unknown cases remain visible and user-controlled rather than being disguised as certainty.

A voice note assistant routes four supported commands

A local assistant recognizes requests to summarize, search saved notes, create a draft, or ask a question. Short messages may fit more than one class, and the app can also open a remote help page.

Decision

Use fixed labels plus an unknown/clarify result, validate the label in host code, and keep remote navigation or data transfer behind separate user action.

Expected outcome

The app routes only validated supported cases and asks a question or does nothing safely when wording is ambiguous, malformed, or unsupported.

Evidence to collect

The claim becomes reviewable only when the following evidence is attached to the same artifact and test run:

• Intent taxonomy
• Confusion fixtures
• Unknown-intent behavior

Task quality needs a versioned corpus and acceptance rules. Fluent sample output is not an accuracy result.

Implementation workflow

1. Freeze a finite taxonomy. Write label definitions, inclusion and exclusion examples, neighbors, and unsupported cases. Give the taxonomy a version and do not accept model-created labels.
2. Separate proposal from permission and action. Treat the classifier response as an untrusted proposal. Application code checks known label, current UI context, required fields, and policy; sensitive actions still require their own consent or confirmation.
3. Validate exact output deterministically. Parse a small schema, reject malformed and unknown values, and avoid overlapping substring rules. Preserve original input for the user, not as diagnostic payload.
4. Design unknown and clarify outcomes. Handle ambiguous, empty, unsupported, timeout, model unavailable, and malformed cases with a visible recovery path; do not default to the most permissive intent.
5. Evaluate frozen fixtures and failures. Run positive, near-neighbor, hard-negative, injection, and malformed-output fixtures. Record confusion and rejection outcomes for the exact artifact and prompt without inventing scores.

Keep each transition observable. A failure should identify the stage, artifact, runtime, and recovery action without logging private user content.

Acceptance criteria

• Every accepted label exists in the versioned host taxonomy and is valid for the current feature context.
• Ambiguous, unknown, malformed, and unsupported inputs are rejected or clarified without triggering a remote route or consequential action.
• A frozen fixture report records artifact, prompt, parser, all outcomes, and privacy-safe test inputs; no unsupported universal accuracy claim is made.

Decision table

Failure patterns to prevent

• Executing free-form labels
• Hiding low-confidence cases

Also prevent silent fallback, unpinned artifacts, missing cancellation, and conclusions that combine unlike configurations. Store unsuccessful runs alongside successful ones.

Minimum reproducibility record

Release checklist

• ☐ The primary query is answered without an unsupported number.
• ☐ Every artifact and runtime is pinned.
• ☐ The representative task and failure policy are explicit.
• ☐ Lifecycle, cancellation, cleanup, and fallback are tested.
• ☐ User-content and network boundaries are documented.
• ☐ Result wording applies only to the recorded configuration.
• ☐ The page links to raw method or evidence when results are added.

Sources and related evidence

• FunctionGemma mobile actions
• Cove benchmark method
• Google Gemma function calling guide
• Google FunctionGemma model card
• Android app architecture guide

Chinese deployment and troubleshooting content is organized in the .奇连 AI 端侧专题.

Frequently asked questions