Measure Garbage-Collection Pressure in On-Device AI

Direct answer

Garbage-collection pressure is one possible contributor to pauses in a local-AI pipeline, but a GC event near inference does not prove that the model caused the pause. Track managed allocations, collection events, task timestamps, and native/process memory on one timeline. Android’s memory documentation describes ART’s managed heap and generational collection; it also explains that total process memory spans Java/Kotlin, native, graphics, shared, and other areas. LiteRT-LM or another native runtime may allocate outside the managed heap, so a quiet Java heap does not mean the process used little memory, and a GC event is not automatically the cause of a slow response. Choose a representative fixed workload and record artifact/runtime identity, app build, device and OS, input fixture, media preprocessing, and lifecycle. Capture baseline, task start, allocations before and during the task, GC timestamps or profiler events, completion, cancellation, and cleanup. Use Android Studio Memory Profiler or another named platform tool. Keep the metric explicit: Java/Kotlin heap allocation, native heap, PSS, RSS, or GC-event count are not interchangeable. If the tool cannot observe native allocations from the runtime, state the gap rather than estimating them from file length. Design experiments to separate allocation churn from persistent retention. Compare repeated identical tasks with the same media and prompt; track whether temporary strings, wrappers, copied buffers, or UI state grow on each run. Then vary one factor such as image preprocessing, transcript length, or result history. A rise in managed allocations that is later collected differs from native allocation that stays resident, an intentional cache, or an object retained by a reference. Do not “fix” GC pressure by calling System.gc() in production or clearing user-visible history without a product decision. Correlate pauses with task behavior. If the UI stutters but no relevant GC event aligns with the pause, inspect main-thread work and native blocking separately. If GC events coincide with allocation bursts, identify allocating call sites and object lifetimes using profiler allocation recording. If memory grows while managed heap stabilizes, inspect native allocations, bitmaps, mappings/resident pages, and shared memory. If a task is cancelled, confirm both managed wrappers and native resources follow their owners’ cleanup contracts. Report only the tested scope. The repository has no complete real-device benchmark dataset, so this article defines a diagnostic protocol, not a GC pause threshold, model memory figure, or latency claim. Preserve failed and interrupted runs, fixture versions, raw profiler captures, and environment notes. A useful conclusion states whether GC was observed near a specific phase, which allocation sites changed, what remains unobservable, and which recovery tests passed.

A screen pauses during repeated image analysis, but model memory is unclear

A local vision app processes the same synthetic image repeatedly. Users report pauses and the managed-heap graph rises and falls, but native runtime buffers and decoded bitmap copies have not been measured.

Decision

Correlate GC events, managed allocation sites, native/process metrics, and task boundaries before attributing a pause to garbage collection or inference.

Expected outcome

The team separates transient allocation churn, retained managed objects, native memory, and main-thread blocking as different hypotheses.

Evidence to collect

The claim becomes reviewable only when the following evidence is attached to the same artifact and test run:

• Allocation trace
• GC event timeline
• Task latency sequence

A device badge must come from an exact device, OS, artifact, runtime, backend, and workload record, not parameter-count intuition.

Implementation workflow

Connect the allocation trace with the GC event timeline and the task latency sequence in order to record managed allocations, observe collection events, and compare them against observed task execution times, keeping the recorded evidence aligned across these items rather than extending interpretation into areas outside the prompt.

1. Pin task and measurement environment. Record app, artifact, runtime, backend, device/OS, synthetic input, preprocessing, task sequence, and lifecycle. Use the same fixture across repeated runs and retain failures.
2. Capture managed and process metrics by name. Use Android Studio Memory Profiler or a documented tool to record Java/Kotlin allocations, GC events, and available native/process metrics. Keep units and metric sources separate rather than merging them into one RAM value.
3. Align GC and task events. Timestamp request submission, preprocessing, inference, streaming, rendering, cancellation, and cleanup alongside collection events. Determine whether the pause aligns with GC, main-thread work, or a blocking native call.
4. Find allocation churn versus retention. Record allocation sites and object lifetimes for wrappers, strings, bitmap copies, and UI state. Repeat one task and check whether objects are reclaimed, intentionally cached, or retained; inspect native memory separately.
5. Change one allocation source and regress. Vary one factor such as image size, result-history retention, or conversion strategy while keeping model task fixed. Add a regression assertion for bounded ownership and cleanup; do not call System.gc() or erase user data as a shortcut.

Acceptance criteria

• GC events, managed heap, native/process memory, and task timing are recorded as distinct measurements.
• Pause attribution is supported by an aligned trace or explicitly left unconfirmed; temporal proximity alone is not proof.
• Repeated-task tests distinguish collectible churn, retained references, native allocation, and intentional cache behavior.
• No universal GC threshold, RAM requirement, or inference-latency claim is published without raw device records.

Decision table

Failure patterns to prevent

• Blaming every pause on the model
• Ignoring native memory

Restate the pitfalls of blaming every pause on the model and ignoring native memory, limiting release wording to the recorded evidence, with the noted boundary identifiers MEMORY-01 and BENCHMARK-01 marking the scope of what has been observed without introducing additional technical nouns, behaviors, or mechanisms beyond what is explicitly supplied.

Minimum reproducibility record

Release checklist

• ☐ The primary query is answered without an unsupported number.
• ☐ Every artifact and runtime is pinned.
• ☐ The representative task and failure policy are explicit.
• ☐ Lifecycle, cancellation, cleanup, and fallback are tested.
• ☐ User-content and network boundaries are documented.
• ☐ Result wording applies only to the recorded configuration.
• ☐ The page links to raw method or evidence when results are added.

Sources and related evidence

• Cove benchmark method
• Android memory management and garbage collection overview
• Android Studio Memory Profiler

Chinese deployment and troubleshooting content is organized in the .奇连 AI 端侧专题.

Frequently asked questions