Direct answer
Baseline Profiles and Macrobenchmark operate on application code paths and user-visible journeys; they do not directly prove that a model artifact is compatible or that inference became faster. Android recommends Macrobenchmark for larger flows such as app startup and UI interaction, and uses Baseline Profile generation to identify code paths for ahead-of-time compilation. For a local-AI app, the relevant journey may navigate to a feature, expose model state, or show a result screen. The model runtime's own load and inference boundaries must remain separately identified in the test record. Begin by writing the journey in user terms. For example, “open the note feature, see whether the runtime is ready, submit a fixed local task, and render the accepted result” contains app startup, UI navigation, model readiness, inference, and output presentation. Decide which of these the profile test is meant to capture. If it is intended to capture navigation only, use a fixture or state that avoids loading or running inference, and say so. If model loading is included, record that as a separate phase rather than attributing the entire app journey to a profile. Keep build variants honest. Android's Macrobenchmark guidance asks teams to benchmark a target configured close to release and distinguishes the benchmark test module from the app under test. Record the exact app variant, profile state, compilation mode, runtime build, model artifact identity, device, and test revision. Do not compare a debuggable local build with an optimized release build and call the result a profile effect. Do not assume a profile artifact was installed just because a generation task succeeded; verify the release packaging and the conditions under which the benchmark runs. Control state between repetitions. The journey should have a deterministic starting screen and declared model readiness. If the task is not part of the measured interaction, precondition it consistently and exclude its duration from the measured phase. If inference is part of the scenario, pin the input and define which output event terminates the task. Ensure a test does not accidentally run network download, artifact verification, cold engine initialization, or model warmup in one variant but not the other. Those are valid product paths, but they answer different questions. Read the trace and result artifacts alongside the number. Macrobenchmark can emit structured results and traces; preserve them with the test revision and build identity. Inspect whether time is spent in launch, composition, navigation, main-thread blocking, model initialization, or inference. A profile may change execution behavior for included code paths; it cannot resolve a missing library, invalid model, unsupported operation, or privacy issue. Keep conclusion language tied to the measured layer. Test both profile-enabled and comparison conditions using the prescribed harness. Do not use an emulator result as proof for a physical device or treat one run as universal. If the project cannot configure the proper benchmark module or release-like variant, report that limitation rather than substituting an ad hoc wall-clock timer. The purpose of the article is a reliable test design, not a claim that Cove currently ships a Baseline Profile or receives a specific speedup. Finally, watch for user-visible correctness while evaluating performance. The app must not display “ready” early, duplicate a request, or omit a failure just to make a journey appear smoother. Keep a companion functional assertion for the same route, so an improved trace cannot conceal a lost result or an inaccessible action. Store the profile revision, journey definition, observed status, and trace reference; do not place prompt text or media payloads into the benchmark report.
A Baseline Profile is being evaluated for a model-assisted note flow
The benchmark starts the app, navigates to a note feature, waits on model readiness, submits a fixed input, and displays a result. The team needs to know whether its measure represents UI code paths or inference.
Decision
Split the user journey into named phases, state which phase the profile targets, and preserve release-like build and trace evidence.
Expected outcome
A reviewer can interpret the UI/profile result without attributing model-load or inference time to the wrong layer.
Evidence to collect
The claim becomes reviewable only when the following evidence is attached to the same artifact and test run:
- Profile and release identity
- Cold-start and navigation fixtures
- Model-state-specific outcome records
A release claim is valid only for the recorded configuration and acceptance set. Generalization requires additional records.
Implementation workflow
- Write the target journey and measured layer. Describe the path from launch through navigation and result display. State whether it measures shell rendering, model readiness, inference, or a defined subset.
- Use the proper benchmark and build variants. Configure the Macrobenchmark target close to release, identify the test module and profile state, and record compilation mode. Do not compare debug and release as though only the profile changed.
- Reset the same starting conditions. Prepare the same screen and artifact state for each run. If model initialization or inference is excluded, hold it consistently outside the measured interval; if included, make its boundary explicit.
- Inspect traces and functional outcomes. Review launch, composition, navigation, main-thread work, model readiness, and inference spans. Pair trace review with an assertion that the task result and error state remain correct.
- Report scope, not a broad speed claim. Bind the result to app variant, profile revision, device, OS, journey, runtime, and artifact. Note unavailable traces or setup limitations; do not infer model compatibility or universal gain.
Keep each transition observable. A failure should identify the stage, artifact, runtime, and recovery action without logging private user content.
Acceptance criteria
- The measured journey states whether it includes app startup, model readiness, inference, and result rendering.
- The profile test and target app build identities are recorded with a release-like configuration and declared compilation mode.
- Trace review distinguishes UI work from native model initialization and inference; functional output assertions remain enabled.
- Published conclusions are limited to the actual test device, build, journey, profile, and measurement setup.
Decision table
| Dimension | Choice or evidence | How to interpret it |
|---|---|---|
| Macrobenchmark UI journey | Measure startup, navigation, and rendering boundaries | Useful for user-visible app flows; does not alone measure model quality. |
| Model-specific runtime test | Measure artifact load and inference with named events | Answers runtime questions; should not be labeled a Baseline Profile result. |
| Manual stopwatch timing | Use only as an exploratory check with explicit limits | Hard to isolate phases or repeat reliably; not a substitute for traceable benchmark setup. |
Failure patterns to prevent
- Calling profile compilation model acceleration
- Benchmarking debug builds only
Also prevent silent fallback, unpinned artifacts, missing cancellation, and conclusions that combine unlike configurations. Store unsuccessful runs alongside successful ones.
Minimum reproducibility record
| Layer | Record |
|---|---|
| Device | Manufacturer, model, chipset, RAM class, operating-system build |
| Software | Application version and git commit |
| Model | Family, variant, revision, format, file length, hash, quantization |
| Runtime | Name, revision, requested backend, observed backend evidence |
| Workload | Fixture revision, input hash, prompt hash, output policy |
| Outcome | Completed, failed, cancelled, fallback, and privacy-safe diagnostics |
Release checklist
- ☐ The primary query is answered without an unsupported number.
- ☐ Every artifact and runtime is pinned.
- ☐ The representative task and failure policy are explicit.
- ☐ Lifecycle, cancellation, cleanup, and fallback are tested.
- ☐ User-content and network boundaries are documented.
- ☐ Result wording applies only to the recorded configuration.
- ☐ The page links to raw method or evidence when results are added.
Sources and related evidence
Chinese deployment and troubleshooting content is organized in the .奇连 AI 端侧专题.
Frequently asked questions
Does a Baseline Profile speed up model inference?
Do not assume that. It targets included application code paths; runtime inference needs its own artifact, backend, workload, and measurement evidence.
Can I compare a debug build with a profiled release build?
Not as a profile-only comparison. Build type and optimization differ; use the benchmark guidance and record matching release-like conditions for both builds.
Should model initialization be included in startup metrics?
Only when that is the declared question. Record it as a distinct event so UI startup, engine load, and task output are not conflated.