Backporting Apple USB4 into an Asahi kernel, and what broke on the way
No Asahi branch has both a working GPU and Apple USB4. Backporting the host controller down into 7.1.13 got two Macs talking over a direct cable at 11.9 Gbit/s and 0.254 ms — after four bugs, one of them upstream and one of them mine.
No Asahi Linux branch has both a working GPU and Apple USB4. The 7.1.x
branches build DRM_ASAHI and have no host Thunderbolt controller; the 7.2
branch has the controller and marks the GPU driver depends on BROKEN.
Backporting the controller down into 7.1.13 got a Mac Studio and a MacBook Pro
talking over a direct USB4 cable at 11.9 Gbit/s and 0.254 ms round-trip
— 17.7x lower latency than the same two machines over the LAN. Four bugs stood
in the way. Only the first was a missing backport; one was mine, one was a gap
in the older kernel, and one is an upstream ordering bug that nobody had hit
because the code path had never been exercised.
Test bench
Two Apple Silicon machines and one cable. There is no PCIe link table here because there is no PCIe in the path — the equivalent evidence is the USB4 lane training state and the negotiated generation, which the controller firmware reports over its RTKit syslog.
The link state, read from the running system rather than from the cable spec. Both lanes trained; a retimer enumerated; the peer identified itself. The controller firmware version matters because it is signed Apple firmware loaded at link-up, not something the kernel supplies:
The chain that has to complete
Bringing up Apple's Thunderbolt host controller is not one driver probing. It is a relay: the Type-C power-delivery controller notices the cable has negotiated USB4, and hands that fact out-of-band to a switch that the Thunderbolt driver registered. Only that callback starts the ACIO complex, which boots a coprocessor, which creates the host interface, which allocates the domain. Every stage below was a separate failure this session.
Bug 1 — the provider and the framework, but no consumer
After the backport compiled, everything looked complete and nothing happened.
All six ACIO blocks bound. The host-interface driver registered. The
deferred-probe list was empty. The device tree carried every property. Zero
oopses. And /sys/bus/thunderbolt/devices stayed empty.
The answer was in the kerneldoc of the function that never ran: "Called by the Type-C port driver to forward the cable details out-of-band to the switch handler." The 7.1.13 tree had the provider that registers a switch and the framework that implements the plumbing, and nothing that called it. A grep for the symbol across the whole tree matched four files, all of them definitions:
The missing piece is in the power-delivery driver: a stored switch reference
and six call sites that fire on cable transitions. Porting it made the link
come up — and wedged the machine at 7.26 seconds into boot, before systemd
arms the watchdog, with no ssh and no SysRq, costing a physical power-cycle.
Resolving the switch reference inside probe() walks a
connector-to-controller device link that the device tree already flags as a
dependency cycle. Moving the resolution to the first cable event — a
workqueue, post-boot, where the 60-second watchdog still covers the machine —
removed the risk entirely, and made the whole feature switchable at runtime
instead of at boot.
Bug 2 — a shortcut coming due, and the address that identified it
Upstream 7.2 replaced struct pci_dev *pdev with struct device *dev in the
host-interface struct, because a platform controller has no PCI device at
all. That change cascades to 92 call sites. I added dev alongside pdev
instead and left the call sites alone. That held right up until a non-PCI
controller actually probed:
The useful detail is the fault address. It is 0x350, not 0 — because
&nullpdev->dev is not NULL, it is offsetof(struct pci_dev, dev) plus
whatever member the code then reads. A small non-zero fault address in a
driver that was just ported from PCI to platform is a signature, not a
mystery. The same signature appeared a second time one layer out, at 0xd0,
in the networking module — because the fix had covered the Thunderbolt driver
directory but not the inline helper in the public header that modules use.
The 86 references split cleanly: 61 want a struct device * and are pure
substitutions; 25 are genuine PCI operations — pci_read_config_dword,
pcie_find_root_port, msix_enabled — confined to files a platform
controller never enters. Only the 61 were rewritten.
Bug 3 — the DROM is in the device tree, and nothing read it
With the crash fixed, the domain allocated, the rings started, and the software connection manager talked to the hardware — it enumerated DisplayPort input resources on two ports, which is only possible over a working control channel. Then it stopped:
The device tree was not at fault. Every host-interface node carried a real
76-byte apple,thunderbolt-drom with a distinct UID. Apple Silicon has no
DROM EEPROM — the boot loader supplies it as a property — and 7.1.13's EEPROM
code had no path to read one, so the host router's descriptor stayed NULL. Two
small functions and one branch. It only compiles after bug 2 is fixed, because
it reaches the controller through the field that fix introduced.
Bug 4 — an upstream ordering bug in a path nobody had exercised
The domain came up, both lanes trained, the peer Mac identified itself — and the network driver could not allocate a transmit ring:
-1 is "allocate any free hop". The generic ring allocator requested the ring
interrupt before allocating the hop ID, while Apple's controller looks its
interrupt up as tx_irqs[ring->hop]. With the hop still -1, that is an
out-of-bounds read, and then a request for whatever it found.
This one is not a backport artefact. The donor tree has the identical ordering and the identical indexing — the PCI controller hands out MSI-X vectors and never indexes by hop, so it is indifferent, and the Apple ring-interrupt path had simply never been run with the network driver. Swapping the two calls (and unwinding the hop on failure) is safe for both.
Results
Settings: MTU 9000 both ends, direct cable, no other traffic on the interfaces. Throughput is a warm floor from repeated 5-second runs — three per configuration, reported as the observed range, not a first-run pair. Latency is 200 ICMP samples at a 10 ms interval.
| path | min (ms) | avg (ms) | max (ms) | stddev (ms) |
|---|---|---|---|---|
| over USB4 | 0.160 | 0.254 | 0.386 | 0.046 |
| same host over the LAN | 3.556 | 4.487 | 8.771 | 0.919 |
That is 17.7x lower latency, 20x tighter jitter than the same two machines over the LAN.
| direction | streams | observed range (Gbit/s) |
|---|---|---|
| B -> A | 1 | 11.0 - 11.9 |
| B -> A | 4 | 11.4 - 11.8 |
| B -> A | 8 | 11.0 - 11.5 |
| A -> B | 4 | 10.5 - 11.7 |
Symmetric, and stream count does not matter: one stream is as fast as eight. That rules out a TCP window or a single-core send path and puts the ceiling in the link or the driver.
A defect the throughput number hides
MTU 9000 is configured and accepted on both ends, and TCP runs at full speed. But any packet above roughly 8198 bytes that is not segmented by the stack is dropped completely:
The cutoff is consistent with the transport's 4096-byte frames — two frames plus headers. TCP never notices because segmentation offload produces segments below the ceiling, which is exactly what makes this worth writing down: a throughput benchmark will not find it, and anything sending large unsegmented datagrams over the link will black-hole silently.
Two traps that cost more time than the bugs
A failed network ring leaves a cross-domain properties request stuck forever,
and that jams the workqueue system. The cascade reaches grub2-editenv, which
lands in uninterruptible sleep — so the command that arms a one-shot boot
entry silently fails, and systemctl reboot does nothing at all. Two
"reboots" appeared to succeed while uptime kept climbing and I kept testing a
kernel I thought I had replaced. Verify a reboot with uptime -s, never with
"the machine answered ssh again".
Separately, and not caused by any of this: the stock distribution kernel takes a display-driver oops during boot that kills a udev worker inside the module loader. Whatever driver happens to be loading at that moment never appears. One boot loses both network interfaces, the next loses half of USB, the next is fine — which reads for days as flaky hardware. Measured across boots: three USB root hubs instead of six, and the external disk holding the kernel build tree simply absent. Blacklisting the display driver gives six root hubs and zero oopses, at the cost of the desktop compositor only. Do not attribute that class of failure to your own kernel without booting a stock one first.
Where it stands
| Track | Status |
|---|---|
| USB4 host-to-host | Up. 11.9 Gbit/s, 0.254 ms, symmetric, zero oopses through a full benchmark run. |
| GPU on same kernel | Intact — the whole point of backporting down rather than forward. |
| Link efficiency | 30% of 40 Gbit/s. Open. Needs the cost decomposed, not guessed at. |
| RDMA | Not yet on this host — no verbs provider built for this kernel. Proven on a different pair over the same transport. |
The backport is driven by an idempotent script rather than a patch series, with every hunk anchored on verbatim text and asserted to match exactly once — it refuses rather than guesses when a tree does not match. That mattered more than expected: the source moved under it constantly, and a step that half-applies to a kernel you then boot is a much worse outcome than one that stops and says why.
Sources: the AsahiLinux/linux branches (the GPU driver lives on 7.1.x; the USB4 host controller on the 7.2 work-in-progress branch), and the Asahi Linux project's reverse-engineering of the ACIO and ATC PHY blocks, without which none of this exists.
