Running Muse Glimmer 30B on Strix Halo: a real multimodal check

The first report on this Muse Glimmer 30B deployment was incomplete. It measured a real text-generation speedup, but the running service had no multimodal projector. That meant it could not demonstrate vision. This replacement tests the actual image path on an AMD Strix Halo system, keeps the earlier DFlash result in its proper text-only lane, and shows the failures that changed the setup.

The missing piece was the projector

The previous Q4 DFlash profile used the model's embedded chat template, but did not pass --mmproj. A chat response from that service was valid text inference, not vision evidence. The corrected managed profile loads Unsloth's matching mmproj-kquant.gguf, enables --jinja, uses a 16K context, Flash Attention, Q8 KV cache, and one server slot on Vulkan0.

The target GGUF SHA-256 is 82bece30…a409fd38; the projector SHA-256 is f48b4523…f7c00c6. llama.cpp reported the projector as a loaded multimodal model before accepting requests. The service is managed by Home Modal on Fedora rather than started as an orphan process.

flowchart LR
  A[Public image] --> B[OpenAI chat content parts]
  B --> C[llama.cpp + matching mmproj]
  C --> D[Muse template emits image patch]
  D --> E[Structured output]
  B -. same prompt, no image .-> F[Withheld control]
  style C fill:#FBF9F6,stroke:#B8431B,color:#1D1B18
  style E fill:#FBF9F6,stroke:#3F7D4E,color:#3F7D4E
  style F fill:#FBF9F6,stroke:#9A6B18,color:#1D1B18

A vision test with a control

The test image is the Golden Gate Bridge image used in the Muse Glimmer localization reference. The prompt requested a box on a normalized 0–1000 grid. The image-present request returned a broad bridge box; the same prompt without an image returned the all-zero fallback. The withheld response is retained specifically so it cannot be mistaken for a grounding success.

# image present
{"x_min": 0, "y_min": 300, "x_max": 900, "y_max": 520}

# same request, image withheld
{"x_min": 0, "y_min": 0, "x_max": 0, "y_max": 0}

This is a coarse localization demonstration, not a benchmark score. The image-present call processed 496 prompt tokens at 117.64 tokens/s and generated 155 tokens at 13.79 tokens/s according to the server timing fields. External image fetching initially stalled on the Fedora worker, and a malformed local base64 payload was rejected with HTTP 400; embedding a verified local image fixed both transport variables before the recorded request was made.

Medical-themed, but not a medical claim

A second visual check used Wikimedia Commons' CC0 lateral chest-radiograph teaching image. The only question was its source-documented projection: lateral or frontal. The output was lateral. No diagnosis, normality assessment, pathology question, treatment question, patient image, or clinical metric was involved.

One correctly named projection on a public teaching image does not validate a medical model. It is only evidence that this local multimodal path can carry an image through the projector and template to a constrained non-diagnostic response. That boundary matters for any local-first medical workflow: inputs must be licensed or de-identified, outputs must stay auditable, and model behavior must not be promoted to patient-care evidence.

Code generation was compiled, not admired

The previous post showed a visibly incomplete completion. This run asked for one TypeScript function: return the longest strictly increasing contiguous run, or zero for an empty list. The model returned code only; the exact generated file passed strict TypeScript checking with tsc --noEmit.

export function longestIncreasingRun(values: number[]): number {
  if (values.length === 0) return 0;
  let maxLen: number = 1;
  let curLen: number = 1;
  for (let i: number = 1; i < values.length; i++) {
    curLen = values[i] > values[i - 1] ? curLen + 1 : 1;
    if (curLen > maxLen) maxLen = curLen;
  }
  return maxLen;
}

Where DFlash fits

DFlash remains the text-generation optimization result, not a claim about the vision request above. In the earlier controlled Q4 test on this same Strix Halo host, a 256-token target-only run measured 13.98 generated tokens/s. The matching DFlash run measured 21.23 tokens/s, and the final verification measured 24.51 tokens/s, with the trained 15-token draft window. Those runs were isolated because two full 30B models on unified memory invalidate a throughput comparison.

The existing DFlash service remains running on port 8300. Replacing it with a projector-enabled DFlash variant requires a controlled maintenance window; launching a second full model merely to claim a number would repeat the original mistake. The Q4 vision service therefore remains a distinct, managed profile on port 8310.

What is now established

• The matching Q4 GGUF, projector, embedded template, and Vulkan service are live on the Fedora Strix Halo worker.
• Image-present and image-withheld requests diverge in the expected direction for the bridge grounding prompt.
• A CC0, non-identifying teaching image produced its source-documented projection only; this is not clinical validation.
• Generated TypeScript passed strict type checking.
• DFlash has a separate, measured text-only throughput result; it is not being conflated with vision throughput.