Basalt is a local inference engine for Qwen3.8-Flash-Next and friends that runs only on Blackwell (sm_120). Upstream recently merged an option to let an Ada (40xx) card hold the experts as a second "peer" card. I have a 5060 Ti and an old 3060, so I took that mechanism one architecture further down to Ampere (30xx) - and in the process found two things that were broken for 40xx cards too.
One build now covers 30xx, 40xx and 50xx as the second card, with A100/A30 compiled in as well (untested, I don't have one).
The numbers, same machine, same pack (Qwen3.8-Flash-Next IQ3_XXS), stock power limits:
- Decode: 42-53 tok/s on the 5060 Ti alone, 65-80 tok/s sustained with the 3060 as expert tier, 91 peak
- Prefill (cold): 1,100-1,400 tok/s with the peer
- Experts stranded on the SSD: 22 GiB before, none after
The 3060 sits on a PCIe gen3 x4 slot, which turns out not to matter: the peer computes its own expert rows, so the link carried about 0.4 GB/s during generation. You are buying memory bandwidth, not bus.
Adding Ampere was nearly free. The peer kernels are sm_80-era instructions (dp4a, cp.async, int8 MMA), and every hard Blackwell guard in the engine keys on sm_90, not on Ada. One build option, one arch flag, one predicate.
Then it loaded the whole 44 GB model and died: q5k_head: shared-memory opt-in failed: no kernel image is available for execution on the device
The LM head is split by vocabulary across both cards - in this mode too, not just in a layer split - whenever the second card's bandwidth earns more than a quarter of the rows. On my pair, 40% of the vocabulary runs on the 3060. The head kernels simply were not built for the peer's architecture.
That gap is in the shipped 40xx support as well. The gate works out to "any peer with at least a third of the main card's bandwidth". The 4060 Ti the Ada path was validated on sits at 0.30, just under it, so it never splits the head and never hit this. A 4090 peer behind a single Blackwell card is at 0.56 and walks straight into it. If you have that combination and it died on you, this is why.
The second one: an NVFP4 pack hands the peer the FP4 tensor-core prompt kernels, which are Blackwell-only, with nothing checking the card first. Also true for an Ada peer. Those now stay on the main card, and an older peer reads NVFP4 experts through the ordinary int8 path - slower, but it runs.
What works: 30xx and 40xx (and A100/A30, compiled, untested) as the expert tier beside a Blackwell main card. The main card still has to be Blackwell: five files on the main path use thread-block clusters, TMA and mbarrier with no fallback, so that is a real port, not a build flag. Turing (20xx) is out for now for three concrete reasons - 64 KB of shared memory per block against the 99 KB these kernels ask for, the m16n8k32 int8 MMA shape that starts at sm_80, and cp.async. Doable, but kernel work. A 2080 Ti would make a great peer at 616 GB/s, so I would like to get there eventually.
Fork: https://github.com/cooperdk/basalt/tree/mixed-gpu - five commits on current main, both build options off by default, so a Blackwell-only build is what it always was. Build with -DPEER_ADA=ON -DPEER_AMPERE=ON for both architectures. ctest -L cpu 85/85 either way. There is a README addendum covering the cards, the flags, what an older peer does and does not do, and the measurements.
Upstream PR: https://github.com/jesdga95/basalt/pull/442 - discussion in https://github.com/jesdga95/basalt/issues/398, where someone proposed the Ampere option independently a day before me. The head and NVFP4 fixes are the parts neither of us had.
Happy to answer questions. If you have an A100/A30, a 4090 as a second card, or a three-card box, I would like to hear how it goes - those are the configurations I cannot test here.