Validate Structured Output from a Local Model

Direct answer

Structured output is still model output and should be treated as untrusted input until parsed and validated by the host application. A prompt that requests JSON does not guarantee valid JSON, required fields, correct types, safe values, or the absence of extra text. Parse with a strict, bounded decoder; validate the result against an application-owned schema; apply domain rules against current state; and expose a correction or failure path instead of silently coercing unknown values. Do not deserialize arbitrary model text directly into privileged classes or use it to invoke methods by name. Keep syntax validation separate from semantic validation. A response can be valid JSON while an amount is negative, a date is outside the task range, a translation is empty, or an enum value is unknown. Define required and optional fields, size bounds, allowed enums, list limits, and what happens when a field is absent. Use versioned schemas so saved outputs can be reviewed after the model prompt or runtime changes. If the model emits a partial stream, do not persist it as a completed object until the parser confirms the final boundary. A robust result pipeline has stages: raw output capture in memory, syntax parse, schema check, business-rule validation, user review where needed, and persistence. Each stage should return a typed result that distinguishes incomplete, invalid, accepted, corrected, and rejected data. Preserve the original input separately from a normalized version when the user needs to compare them, but follow a retention rule and avoid writing raw private content into logs. When validation fails, report the field and category rather than echoing the entire sensitive response. Models can include prose before or after a structured block, repeat keys, emit duplicate objects, or use unexpected numeric formats. Decide whether the parser rejects these cases or a narrowly defined normalization layer handles them. Never use permissive regex extraction to rescue an arbitrary function call or hide malformed boundaries. If the runtime offers constrained decoding for a specific artifact, verify the exact documented scope and still validate the parsed object; generation constraints do not replace application authorization or domain checks. Test with both expected and hostile fixtures: valid minimal object, missing field, null, wrong scalar type, out-of-range value, unknown enum, extra nested field, oversized list, malformed escape, partial stream, repeated object, and instruction-like text in a string. Also test process recreation and schema migration for saved outputs. Keep acceptance criteria based on task properties, not whether the output merely parses. A translation may parse but omit a sentence; an extracted field may be syntactically valid but unsupported by the source. When structured data triggers an action, add a second gate. Schema-valid tool arguments are not user authorization. Check the active request, current permissions, app policy, user confirmation, and idempotency before executing a handler. This boundary matters even when inference is local. Implementation protocol: 1. Define a versioned schema with required keys, types, limits, and explicit unknown or missing-field behavior. 2. Parse in a bounded stage; reject malformed or oversized data and do not repair errors with ambiguous string slicing. 3. Validate semantic constraints against the source and current task, then present uncertain or consequential fields for user review. 4. Persist only after validation and according to the app’s data-retention policy; keep raw output out of routine logs. 5. Test malformed, adversarial, partial, duplicate, and migration fixtures before changing the schema or prompt. The result is a reliable data boundary around a probabilistic model, not a guarantee that the model always follows the requested format.

A model returns an expense draft in JSON form

An offline assistant extracts merchant, date, and total from a user-provided receipt image. The prompt requests a JSON object, but the system must prevent missing values or plausible-looking guesses from being silently saved as verified expense data.

Decision

Parse into a narrow versioned schema, check each field against available source evidence, and require user review for uncertain or consequential values.

Expected outcome

The app stores a validated draft with provenance or rejects it clearly; syntactically valid output alone never bypasses review.

Evidence to collect

The claim becomes reviewable only when the following evidence is attached to the same artifact and test run:

• Schema tests
• Malformed-output fixtures
• Fallback behavior

Task quality needs a versioned corpus and acceptance rules. Fluent sample output is not an accuracy result.

Implementation workflow

1. Define a bounded schema. Specify required fields, nullable rules, enum values, list-size limits, and maximum string lengths. Version the schema with the task and avoid mapping arbitrary keys into privileged object types.
2. Parse strictly and classify errors. Reject malformed structure, partial streams, duplicate or ambiguous records, oversized payloads, and unsupported types. Return an error category and field path instead of echoing the full private response.
3. Validate meaning against evidence. Check ranges, date policy, source presence, and whether the source image or recognized text supports each field. Keep a suggestion distinct from a verified value and let the user correct it.
4. Gate persistence and actions. Save only after validation and review according to the retention policy. If output can trigger a tool, separately check authorization, permission, confirmation, request identity, and duplicate execution.

Keep each transition observable. A failure should identify the stage, artifact, runtime, and recovery action without logging private user content.

Acceptance criteria

• Every persisted field has a defined type, range, and source or user-review path.
• Malformed, partial, extra-field, wrong-type, and oversized outputs are rejected or handled by a documented narrow rule.
• No structured result invokes an action or becomes trusted data solely because it parsed successfully.

Decision table

Failure patterns to prevent

• Regex-only trust
• Guessing missing fields

Also prevent silent fallback, unpinned artifacts, missing cancellation, and conclusions that combine unlike configurations. Store unsuccessful runs alongside successful ones.

Minimum reproducibility record

Release checklist

• ☐ The primary query is answered without an unsupported number.
• ☐ Every artifact and runtime is pinned.
• ☐ The representative task and failure policy are explicit.
• ☐ Lifecycle, cancellation, cleanup, and fallback are tested.
• ☐ User-content and network boundaries are documented.
• ☐ Result wording applies only to the recorded configuration.
• ☐ The page links to raw method or evidence when results are added.

Sources and related evidence

• FunctionGemma mobile actions
• Google AI for Developers — function calling with Gemma 4
• Android Developers — unsafe deserialization risks
• Android Developers — app architecture guide

Chinese deployment and troubleshooting content is organized in the .奇连 AI 端侧专题.

Frequently asked questions