Direct answer
“Qwen 3.5 2B GGUF performance on mobile” cannot be answered by model name alone. The official Qwen 3.5-2B repository and a third-party GGUF conversion are not interchangeable artifacts: conversion choices, quantization, tokenizer files, chat template, multimodal support, runtime build, and device backend can all change what is actually being tested. A speed result from a desktop, a different model revision, or another converter does not establish phone behavior. Begin by pinning the upstream model revision and the exact converted GGUF files, including their source, hashes, and quantization. Then run a fixed task on one named device and preserve all runs, including load failure, CPU fallback, out-of-memory, cancellation, and sustained degradation. This page publishes no Qwen mobile speed or device-support result. It provides a method for producing a result that another engineer could reproduce and audit.
Scenario: choosing a local Qwen artifact for an Android prototype
An engineer finds an official Qwen 3.5-2B model card and several community GGUF files, then sees a fast desktop demo. The prototype must decide whether a specific converted file and runtime can complete a bounded task on a phone without thermal or memory assumptions being hidden.
Decision
Treat every conversion and runtime combination as its own candidate. Select one exact artifact, device, Android build, and task fixture; set acceptance conditions before measuring; and keep results separate if any of those variables changes.
Expected outcome
The team gets a scoped record: what artifact was opened, what backend actually executed, which task completed, what the full resource trace looked like, and which errors remain. The conclusion applies only to that recorded combination.
Evidence to collect
The claim becomes reviewable only when the following evidence is attached to the same artifact and test run:
- Official model revision and converted-artifact provenance
- Runtime, backend, device, and workload identity
- Ordered cold, warm, sustained, failure, and recovery records
A release claim is valid only for the recorded configuration and acceptance set. Generalization requires additional records.
Implementation workflow
- Separate the official checkpoint from the GGUF conversion. Record the upstream Qwen model repository and immutable commit first. Then record the exact converter project, converter revision, conversion settings, quantization type, output file list, sizes, and cryptographic digests. Do not label a community-converted file as an official Qwen GGUF release unless the official source actually distributes it. Keep the tokenizer, vocabulary, and chat-template files paired with the converted artifact.
- Verify that the chosen runtime opened the intended model. Capture the runtime name and revision, native library build, Android ABI, load result, and any backend initialization messages that can be observed. Distinguish a requested accelerator from evidence that it ran the workload. Save the first error and fallback path instead of silently retrying another backend and reporting only the successful attempt.
- Define a phone workload before timing it. Use a versioned, privacy-safe prompt fixture that resembles the intended product task. Specify input length, output policy, stop conditions, decoding settings, and whether image or video input is involved. If the conversion or runtime path is text-only, do not generalize its results to a multimodal task. Keep the same fixture and settings across repeated runs.
- Measure distinct lifecycle phases. Separate artifact verification, cold model load, first inference, warm inference, and a repeated workload. State the timing start and end events for each measurement. Record memory before and after load, peak during the task, thermal signals available on the device, cancellation, fallback, and cleanup. A single fast completion does not establish sustained behavior or battery impact.
- Retain every run and compare like with like. Store each attempt as a separate record tied to the same artifact, runtime, phone, OS build, fixture, and settings. Include failures, truncated outputs, incomplete loads, and cancelled tasks. When comparing two quantizations, change only the artifact and preserve the remaining configuration. Do not average away failure modes or rank candidates using file size alone.
- Publish a bounded result, not a universal phone claim. Report the tested artifact provenance, conversion, quantization, device, software, runtime, observed backend evidence, workload, metric definitions, and full-run distribution. Say which tasks were not evaluated. Use wording such as “completed this test on this configuration,” not “runs on Android phones” or “fast on mobile.” A broader support claim needs more devices and representative results.
Keep each transition observable. A failure should identify the stage, artifact, runtime, and recovery action without logging private user content.
Acceptance criteria
- The official upstream checkpoint and the converted GGUF artifact have separate provenance and pinned identities.
- Backend claims rely on observed runtime evidence, not the device chipset or requested setting.
- Cold load, representative task, repeated workload, resource observations, fallback, and failures are recorded separately.
- Every conclusion is limited to the tested artifact, runtime build, device, software, fixture, and settings.
Decision table
| Dimension | Choice or evidence | How to interpret it |
|---|---|---|
| Model identity | Official Qwen checkpoint plus a pinned GGUF conversion record | A family label alone does not identify the bytes or conversion tested. |
| Runtime path | Runtime build and observed backend evidence | A requested GPU or NPU is not proof that the workload used it. |
| Workload | Fixed prompt, context policy, output limit, and modality | Different tasks and input lengths produce incomparable runs. |
| Mobile evidence | Cold, warm, repeated, failure, memory, and thermal records on the target phone | A desktop demo or first successful run cannot establish sustained phone support. |
Failure patterns to prevent
- Calling the upstream Safetensors checkpoint a GGUF release
- Copying desktop timings to phones
- Publishing results without the conversion and task details
Also prevent silent fallback, unpinned artifacts, missing cancellation, and conclusions that combine unlike configurations. Store unsuccessful runs alongside successful ones.
Minimum reproducibility record
| Layer | Record |
|---|---|
| Device | Manufacturer, model, chipset, RAM class, operating-system build |
| Software | Application version and git commit |
| Model | Family, variant, revision, format, file length, hash, quantization |
| Runtime | Name, revision, requested backend, observed backend evidence |
| Workload | Fixture revision, input hash, prompt hash, output policy |
| Outcome | Completed, failed, cancelled, fallback, and privacy-safe diagnostics |
Release checklist
- ☐ The primary query is answered without an unsupported number.
- ☐ Every artifact and runtime is pinned.
- ☐ The representative task and failure policy are explicit.
- ☐ Lifecycle, cancellation, cleanup, and fallback are tested.
- ☐ User-content and network boundaries are documented.
- ☐ Result wording applies only to the recorded configuration.
- ☐ The page links to raw method or evidence when results are added.
Sources and related evidence
- llama.cpp repository
- Cove benchmark method
- Official Qwen/Qwen3.5-2B model repository and files
- Official llama.cpp project and GGUF runtime documentation
- Hugging Face Transformers Qwen3.5 model documentation
Chinese deployment and troubleshooting content is organized in the .奇连 AI 端侧专题.
Frequently asked questions
Does the official Qwen 3.5-2B model card prove mobile GGUF speed?
No. The upstream model card describes the upstream model artifact, not every community conversion, mobile runtime, or phone. Mobile performance requires records for the exact GGUF files, conversion, runtime, device, backend, workload, and measurement boundaries.
Can I use a desktop benchmark to estimate phone performance?
A desktop result can motivate a test, but it is not a mobile result. Phones have different memory limits, thermal behavior, operating-system scheduling, and available runtime backends. Measure on the actual device class and preserve all attempts and failures.
What does “2B” tell me about phone compatibility?
A parameter-count label does not establish artifact size, runtime memory, context capacity, supported modalities, or task quality. Check the exact converted file and measure the product workload. Publish compatibility only within the tested configuration and evidence scope.
Should I compare GGUF quantizations by speed alone?
No. Compare the same task and settings while checking output acceptance, memory, load behavior, fallback, sustained operation, and failures. A faster candidate that no longer meets the task’s acceptance criteria is not an equivalent replacement.