Directing a folktale with MiniMax H3: location plates, one-generation multishot, and a sparse-attention check

Three weeks of MiniMax H3 renders of a Luo folktale kept failing the same way: every clip passed inspection alone, and the assembled cut fell apart — characters drifted between shots, the same garden rendered as four different places, a spear grew a second head. What fixed it was not a better prompt. It was a formulation change (locations became first-class assets, action beats became one generation instead of five), a gate that judges the assembled film instead of the shots, and a stack of mechanisms proven by single-variable A/Bs. Along the way: verbatim recreations of three community workflows, and a measured 6.4–8.0× attention speedup on consumer GPUs from a sparse-attention kernel that turned out to need no porting at all.

Everything below carries its artifact, config, and date. The working session behind this post is this Claude Code conversation; render receipts live in the repo as JSON alongside each number.

Gate the film, not the shots

Version 17 shipped with every take individually verified by a per-shot VLM gate — and the director rejected it in one sentence. Per-shot gates cannot see causality, geography, or pacing. The replacement judges the assembled cut in three layers: ffmpeg scene detection for mechanical flow findings (overstays, flash cuts), one neutral description per shot from a vision model, then a text-only judge over the ordered descriptions for story logic and named key moments.

Two hard-won constraints shaped that design. First, the per-frame judge had a measured polarity bug: asked a negative compound, it would describe a passing frame and output FAIL — one recorded verdict wrote "two distinct people stand separately with their own clothes" as evidence for a failure. Judging the ordered descriptions as text sidesteps the bug entirely. Second, a sampling gate verifies states, not events: a 0.4-second spear release evades three sample points per shot, so the gate kept scoring a visible throw as "missing." The fix was editorial, not statistical — cover fast events with multiple shots so they span seconds, which is what a film editor would have said in the first place.

v18 gate: elephant FAILED every cross-shot consistency pair
          (different tusks, ears, face); locations desert->savanna->hut
v20 gate: all pairs PASS; the only cross-strike difference the VLM
          names is the lodged spear -- which is the story

A place is a first-class asset

The residual failure after character references pinned identity was location drift. A prompt line like "same savanna" is too underdetermined; the sampler invents a new place per shot. The practitioner convergence — visible in the Patchwright breakdown, Max Thomas's Sidewalk Syndicate case study, and Corridor's older anime pipeline — is that a location gets the same treatment as a character: one approved master plate, every shot background derived from it, never re-described. Consistency comes from shared upstream evidence, not from asking a stateless video model to remember the previous shot.

In the series compiler this became a locations: registry: a shot declaring a location gets its plate appended to the reference sockets as the final addressed picture, and an unaddressed plate is a construction-time refusal — an unnamed reference steers nothing. The first render under the plate law was the first cut where the elephant raids the exact garden the brothers stand in and the demand happens in.

v22 — the cut built under the plate law. Garden raid, brothers, taking, throw, flee, demand.

The whole film as a strip — one world, one palette, two brothers, one spear.

Multishot in one generation

A shared community workflow (a five-shot 90s-anime skateboard sequence) exposed the biggest coherence lever of the whole project, and the graph turned out to be nothing special — plain reference-to-video with a turbo LoRA. The technique is entirely in the prompt: timed [Shot N] Xs to Ys sections inside one generation. Shots generated in one context share identity, palette, and place by construction — the thing weeks of clip-by-clip assembly had been approximating.

The first probe on the folktale's action block carried draw → throw → spear in flight → strike with wound → trumpeting → flee past the huts through the model's own internal cuts, with one man, one spear, and one garden throughout — and the best character rendering of the project. The probe's sibling seed failed usefully too: its first shot opened on the character sheet's cream background, re-confirming that cast sheets leak their backgrounds and must be rendered on-palette. The technique is now a compiler feature: a shot may declare multishot: sections and the compiler emits H3's native schema — world and invariants preamble, timed sections, soundscape tail.

The probe's throw beat at full res: release → flight → lodged shaft with wound → trumpet.

References are the whole game — the girl, the lane, and the stand, generated once and addressed in every shot.

Three laws with receipts

Recreating shared workflows verbatim

Confidence in a stack comes from reproducing other people's results byte-for-byte before adapting them. Three community workflows were rerun with their exact prompts, seeds, samplers, and resolutions: the fake speedpaint timelapse (three chained parts: line art, flat color, finishing — the artwork strictly monotonic, the same hands throughout), the time-period-shift transformation (faithful except its fade-to-black-and-white, which stayed color — logged as the one deviation), and the anime skater above. The recreations also surfaced two schema elements the folktale pipeline had never used: H3's structured caption keys (integrated_multimodal_description / timestamped detailed_description / overall_soundscape / non_diegetic_music) and inline <d>[English] ...</d> dialogue tags — the likely reason earlier plain-prose speech attempts came out garbled.

The three-part speedpaint chained end to end — line art → flat color → finishing.

The schema is worth reading once in full — this is the speedpaint's second part, abridged at the timestamps but with its structure intact:

integrated_multimodal_description: [Shot 1] One unbroken LOCKED-OFF overhead shot ...
THE ARTWORK ONLY EVER ADVANCES: every inked line and every finished mark is permanent,
each new pass lands on top of the one before, and a finished area looks identical in
every later frame.

detailed_description:
At 00:00.300 ... his hand uncaps the marker and lays the first flat tone ...
At 00:03.500 the hair takes colour lock by lock, platinum at the crown fading to soft
pink at the tips, each lock one flat even tone inside its ink outline.
At 00:06.500 the jacket takes glossy black ...
At 00:09.000 her huge eyes take deep luminous teal, one flat tone each.
At 00:11.000 his hand is ALREADY DARTING toward a second capped marker, mid-reach at
the final frame.

overall_soundscape: starts with a marker's first squeak on paper, then steady rapid
marker strokes — the tool sounding exactly as used, sped into dense flurries ...

non_diegetic_music: one continuous mellow lo-fi instrumental groove, constant and
unbroken from the first frame to the very last frame.

Note the three moves the schema makes: monotonicity stated as law ("ONLY EVER ADVANCES"), sub-second event timestamps, and every part ending mid-action so the next has a clean cut point. All three ported into the folktale's compiler.

speedpaint p1  1024x768 x 362f  82,176 tokens  20 steps  res_multistep
               RTX 4090, int8 UNet, encoder int8 on CPU
               95.0 s/step wall (31.7 min) -- completed, no OOM
speedpaint p2  1024x768 x 277f  ~68k tokens    83.0 s/step (chained)
time-shift     1024x768 x 243f  ~55k tokens    RTX 3090 -- completed

The render pipeline previously refused anything above 17k video tokens because nothing had been measured there. These runs were the measurement: the >17k regime completes on 24 GB cards — on a much steeper seconds-per-step slope, which is exactly what the next section is about.

The sparse-attention kernel that needed no port

Hao AI Lab's FastH3 preview pairs DMD2 distillation (49 → 4 diffusion calls) with Video Sparse Attention — 90% of video-to-video attention tiles dropped — quoting up to 14.38× on a B200. The headline reads Blackwell-only, and the first-pass assessment here said so too. That assessment was wrong, and the correction came from reading the dispatcher instead of the marketing: the FastVideo kernel package ships a Triton implementation of VSA block-sparse attention in every wheel, and on non-Hopper/Blackwell hardware the Triton path is the default. FastH3's 64-token blocks are exactly what that path supports. No porting required — only measurement.

# settings -- reproduction block
hardware   RTX 4090 24GB / RTX 3090 24GB (one process, one GPU each)
software   fastvideo-kernel wheel (pypi), torch 2.11.0+cu130, python 3.12
workload   bf16, B=1, H=24, D=128; 90% block sparsity
           (keep top 10% of 64-token KV blocks per query block)
protocol   3 warmup + 10 timed iterations, median reported (warm floor)
bench      scripts/vsa_bench.py in the project repo

# speed -- dense SDPA vs VSA Triton, same tensors
                       seq=16,884        seq=82,176
RTX 3090   dense       50.19 ms          1196.12 ms
           vsa_triton   7.70 ms   6.52x   188.18 ms   6.36x
RTX 4090   dense       23.44 ms           620.03 ms
           vsa_triton   3.43 ms   6.84x    77.66 ms   7.98x

The two sequence lengths are the project's real regimes: 16,884 tokens is the draft canvas, 82,176 is the speedpaint scale where dense attention costs 95 s/step end to end. At that scale dense attention is roughly half the step time, so an integrated VSA render projects to about 2× end-to-end — largest exactly where the pipeline is slowest. Two honest gates before any production claim: sparse attention needs the VSA-trained adapter weights (running it against base weights degrades quality — a same-seed with/without A/B that produces identical frames means the adapter silently failed to load, not that it made no difference), and the preview is text-to-video only, so the reference-conditioned family this project leans on waits for the announced variants. Kernel-level numbers are not end-to-end numbers; the integration measurement comes next.

Sources

Community work this session recreated, adopted, or measured — in roughly the order it entered the pipeline: the H3 ControlNet-Union writeup, the TTM cut-and-drag paper, the VACE paper, H3 GuideMaster (source of the 17-frame native guide grid), the wushu action LoRA, the H3 latent upscaler, the lightx2v turbo-8 768p release discussion, silveroxides' experimental merges, the De-Rope anti-smearing nodes with their example workflow (whose prompt also documents H3's video-editing retention schema), raylight (multi-GPU ComfyUI; no H3 support yet — watch item), the r/comfyui consistency thread, the r/StableDiffusion spatial-consistency thread, and the FastH3 work by Hao AI Lab with weights at FastVideo on Hugging Face. Cast and location assets in the folktale project use CC-BY sources with attribution recorded in the repo.

Next: wiring the Triton VSA path under the render graph for a measured end-to-end A/B, the De-Rope second pass on the action block, dialogue retested with proper tags and Ugandan-English voice cards, and the ship-quality re-render of the locked shot specs. The lesson ledger grew by four laws this session, and two of them came from renders that failed — which is the ledger working as intended.