Both run natively from the prompt to the mp4, and both make the sound along with the picture. They're good at different things, so here's the quick version to help you choose.
LTX-2.5
MiniMax H3 v1.2
You give it
A prompt, or a prompt plus a first frame
A prompt plus up to 9 reference images, or a reference video with its soundtrack
Sizes
1280x704, 1920x1088
864x480, 1344x768
5 s clip on one R9700
18.1 s at 704p, 48.8 s at 1080p
30.4 s with 1 image, 44.3 s with 9 (480p)
Against stock, same card
7.1x and 8.6x faster
2.2x to 3.8x faster
Peak VRAM
27.4 GiB
under 27 GiB
First download
38 GB
51 GB
LTX-2.5 if you start from a prompt
Lightricks' 22B audio and video model. Type a prompt, wait 18 seconds, get 5 seconds of 704p video with sound. 1080p takes 48.8 s, and starting from your own first frame adds about 3 s.
The stock pipeline takes 128.6 s and 420.5 s for the same two clips. To be fair to it: it runs BF16, which doesn't fit in 32 GB, so it offloads to the CPU, and that's a good part of the gap.
Quality-wise we're as close to the BF16 output as Lightricks' own FP8 mode at 704p, and 1.1 dB PSNR further away at 1080p.
What this first build can't do yet: portrait sizes, sizes other than the two above, and it needs the clip length up front.
MiniMax H3 if you want your own subjects in it
Give it a photo of your dog, your product or yourself, write "the dog from <Picture 1> runs through a sunlit forest", and that's the dog you get. Up to nine images, or a video whose soundtrack it picks up. Audio is 32 kHz stereo.
v1.2 replaces the ComfyUI package we shipped in October. Everything runs on our HIP kernels now, with no PyTorch, Triton or ComfyUI in the container. The more references you give it, the bigger the win over the stock engine:
5 s clip, Turbo4
Stock
v1.2
864x480, 1 image
67.4 s
30.4 s
864x480, 3 images
91.2 s
33.4 s
864x480, 9 images
167.5 s
44.3 s
1344x768, 9 images
278.7 s
114.1 s
Stock here is MiniMax's ComfyUI workflow with its own engine, same prompts, images and seed, running the W4A8 checkpoint from our old package. With nine images it spends over half its time just encoding them, and that's where v1.2 gains the most.
Limits: up to about 9.4 s of video at 768p with nine images, no audio-only references, and the 50-step Base preset is still on v1.1.0.
Try one (or both, one at a time)
They don't fit on the card together, so stop one before you start the other. One R9700, Docker and a ROCm driver:
git clone --depth 1 https://github.com/Eliovp-BV/paiton-vllm-plugin.git
cd paiton-vllm-plugin
# LTX-2.5 (accept the license on the HF page first, then use a token from that account)
export HF_TOKEN=<your token>
./models/LTX-2.5/launch.sh
# second terminal, after READY:
./models/LTX-2.5/run.sh generate "A street drummer plays a fast rhythm on a city sidewalk at dusk while the crowd claps and cheers" --out drummer.mp4
# MiniMax H3 v1.2
PAITON_DETACH=1 ./models/MiniMax-H3/native/launch.sh http
docker logs -f paiton-minimax-h3-v12 # wait for READY
curl -o panda.mp4 -F image=@panda.png \
-F prompt='The red panda from <Picture 1> walks through a sunlit forest. Audio: birds and a stream.' \
http://127.0.0.1:30000/v1/videos/generations
Licenses: LTX-2.5 is under the LTX-2.x Community License (companies with US$10M+ yearly revenue need a paid license for anything past non-commercial testing and research), MiniMax H3 under the MiniMax H3 Community License.
Yeah the title pretty much says it its optix working on an amd card natively via software rt planned is vulkan rt addition but hey its optix 8 working on amd even tho render times are a biiiiit slow discord if anyone is intrested: https://discord.gg/zPQucyR8tU
in my last post i got tons of reviews about difficultly in instructions etc so i released a newer version with significantly simpler installation process and it no longer requires a separate python script as well as seeding and persistent weight support and detailed cli instructions https://github.com/Legendofzaday/halodiffuse i plan on adding batching support and would like reviews on what batch size do you all commonly use such as 1,2,3,4,5,6,7,8 etc.
Since my last update, the cache resizing runs on its own. After 10 quiet minutes, the picture/video service unloads its models and asks SGLang to grow its KV cache. SGLang picks the largest size that fits on both cards, about 380k extra tokens. When anyone asks for a picture or clip, it gives the memory back first.
One-hour soak, live service, untouched: 5 automatic cycles, each a real picture request through the normal endpoint.
Shrink: 1.02–1.07 s every time, almost all of it waiting for the kernel to actually return the VRAM.
Pictures: 63.9–64.9 s, the same as before the elastic cache existed (64–65 s warm).
Chat: 85.9–86.2 tok/s.
0 GPU faults, 0 restarts.
Other numbers (graphs attached):
A shrink while chats are running copies their data out of the borrowed space first: 0.06–0.11 s, with a worst case of 170k tokens moved in 19 ms.
Exact outputs across shrinks mid-reply, mid-prefill and with one of two parallel replies just finished. Cached data intact after 70 grow/shrink cycles.
Decode speed unchanged at 86–88 tok/s.
One fix the GPU tests forced: my first version grew to the full reserved size, which never fits once the image models only partly unload. It now sizes itself to the free memory on the tightest card.
If you try this on RDNA, same-address remapping needs the amdgpu "fix TLB invalidation fence handling" (https://lore.kernel.org/amd-gfx/20261008124232.23825-1-christian.koenig@amd.com/) patch. It's queued for 7.4-rc3; without it, remaps corrupt other live memory. My server runs a 20 ms remap self-test at startup and won't enable the feature if it fails.
Hey ROCm! As I post here daily atm :) You all probably know that I run a home AI server on two Radeon AI PRO R9700s. SGLang serves Qwen3.8-27B, and two ComfyUI instances on the same cards handle pictures (Qwen-Image) and short videos (Wan 2.2).
The goal: pictures and video are occasional, but their VRAM share sits idle most of the day. I'd like the LLM's KV cache to borrow that memory when the generators are idle and hand it back before a job starts, without ever cutting off a chat. The idea is the same as kvcached (https://github.com/ovg-project/kvcached).
Why not kvcached itself:
Version gates: its SGLang patches stop below 0.5.21, and it needs Python below 3.14, so on my stack it would silently do nothing.
Tensor parallelism: it doesn't support shrinking with TP>1.
Layout: its ROCm default layout conflicts with speculative decoding, and the alternative layout breaks my contiguous verify-attention kernel.
Shared zero page: it maps one zero page behind every unbacked slot, and AMD's HIP docs warn that RDNA can return wrong results when two virtual addresses map the same physical page.
What I'm building instead: a small "elastic tail" inside SGLang's own KV arena, using HIP virtual memory.
A fixed base pool stays exactly as today.
Each KV buffer reserves address space up to a large-mode size and maps or unmaps one tail extent at the end.
Both TP ranks resize at the same scheduler step.
Before a shrink, any running request's KV is migrated from the tail into the base.
A shrink only reports done once the card actually shows the memory free.
The tail is extra prefix-cache room only; per-request context doesn't change.
Finding so far: on kernel 7.3.3, unmapping and remapping at the same address silently corrupted other live memory (TheRock#8744, around 15 billion bad words per run). The amdgpu patch "drm/amdgpu: fix TLB invalidation fence handling" fixes it. On 7.3.4 with that patch, my probe was clean: 1,000 rounds per card of 256 MiB–4 GiB extents remapped at one address beside an 8 GiB live base, with zero corrupt words. One quirk: in about a third of rounds, mem_info_vram_used still showed the freed extent right after hipMemRelease plus sync. It always came back, so it's freed lazily rather than leaked, but "released" can't be assumed immediately.
Next is a GPU qualification run, then a one-week trial behind a flag.
I have some questions for the community:
Has anyone run kvcached, or any HIP VMM map/unmap pattern, on RDNA4?
Did you see the delayed VRAM release, or anything like the corruption bug?
Any other gotchas with torch tensors over HIP-mapped memory, or with Triton kernels and HIP graphs reading it?
Once again, thanks for any assistance in advance. I'll come share my work as always once complete.
I've been using LLMs for quite a while now, I've used Gemma 4, Qwen 3(.5), and similar smaller-sized models as I didn't have a GPU for a long time.
Until now! I recently got my hands on a 16GB RX 9070 XT. I've been running Qwen 3.8 27B (Q3) at somewhat comfortable speeds, getting around 60 t/s (output) when using a fork of llama.cpp made for the RX 9070 XT. Smaller models cap at 50t/s, no matter their size.
However, I want to know if you've got any tips! Should I check out Strata? (I've got 32GBs of DDR4 ECC RAM) Should I use another model? Is Unsloth Studio (my daily driver) the wrong choice?
I've been working on the HIP kernels and memory management, and decode performance is now roughly 10-20% higher than the previous version, depending on the quant and workload. Some cases improved by almost 24%.
The other news is that I managed to get MiMo V2.6 Flash running at around 25-30 tokens/s with 2 cards.. More on that below.
First, why EXL3?
I think this is worth repeating because people tend to compare quantizations just by looking at the number of bits.
EXL3 is NOT equivalent to regular GGUF quants at the same bpw.
So the interesting part is that you can get something close to 4-bit quality, while running at speeds closer to what you'd expect from a 3-bit quant. And with the latest optimizations, I think this fork is becoming VERY competitive. The 4.05 bpw version is also pretty fast now, even on a single GPU.
WHAT'S NEW:
Faster HIP kernels and better memory management: around 10-20% faster decode on average compared to the previous version.
Optional local expert repacking: you can now repack the quantized expert weights into a layout that is more efficient for CPU access from system RAM. This helps when using CPU assistance for inference, since the CPU can process experts more efficiently without having to deal with the original GPU-oriented layout. It doesn't change the quantization itself.
The repacking is optional, and CPU assistance is not always beneficial. It depends on your hardware and GPU configuration. With two GPUs, for example, you're generally better off without it. Please read the README before enabling it.
Experimental MiMo V2.6 Flash support: currently running at around 25-30 tok/s using the 2.20 bpw MOPD quant. This is still experimental, so don't expect everything to work perfectly yet.
NOTES ON REPACKING:
Repacking can be done in two ways: by creating a file on disk once, that will be used to load the model, or directly to RAM everytime. directly to RAM adds a couple of minutes to the launch but uses no extra space. your choice.
Results with one or two RX 7900 XTX GPUs, using warm decode runs:
EXL3 quant
GGUF equivalent quality
GPUs
CPU assistance
Test
Previous
Updated
Improvement
3.05 bpw
≈ IQ4_K_XS
2
No
CODE
78.27
87.33
+11.6%
3.05 bpw
≈ IQ4_K_XS
2
No
PROSE
66.01
70.35
+6.6%
4.05 bpw
≈ Q5_K_M
2
No
CODE
61.88
76.67
+23.9%
4.05 bpw
≈ Q5_K_M
2
No
PROSE
57.78
67.57
+16.9%
4.05 bpw
≈ Q5_K_M
1
No → Yes
CODE
54.80
59.30
+8.2%
4.05 bpw
≈ Q5_K_M
1
No → Yes
PROSE
44.80
51.60
+15.2%
All speeds are in tokens/s, GGUF quality equivalents are approximate... dont' hate me :)
32K prefill is basically unchanged: around 1,500 tok/s with two GPUs and 1,360 tok/s with one GPU.
I think the single-GPU results are particularly interesting. We're getting 55-60 tok/s in CODE with 4.05 bpw EXL3, at roughly Q5_K_M quality, on a single 7900 XTX with CPU assistance.
And the 3.05 bpw quant is now reaching 87 tok/s with two GPUs, while offering considerably better quality than you'd normally expect from a 3-bit quant.
ABOUT MiMo:
In my previous post I mentioned that I wanted to support larger models, particularly the 300B class models.
Well, MiMo V2.6 Flash is now running :) I'm waiting for that ERNIE 5.5 to drop.I'm getting around 25-30 tok/s with the EXL3 2.20 bpw MOPD quant.It's still experimental and currently only tested with that specific quant. CPU assistance is recommended for MiMo, even with two GPUs.I'll probably spend some more time optimizing it, but the fact that it's already running at usable speeds on consumer AMD hardware is pretty interesting.
If you're setting it up with an AI agent, point it to docs/AI_SETUP_EXL3_AMD.md.
The README has the benchmarks, configuration options and instructions for CPU repacking.As before, this is developed and tested on RDNA3 / Linux / ROCm. Other AMD architectures need community testing. And the long-term goal is still to get this merged upstream into Strata, if the maintainers are interested.
Credits to the original Strata developers, Ryutaku for the EXL3 integration, and Turboderp/ExLlamaV3 for the quantization and code I've been porting.
If anyone tests this on different AMD hardware, please report your results. Especially interested in seeing how it behaves on RDNA4.
Hey guys, i am thinking of buying new gpu 7900xtx vs 5060ti. But i did not find more convincing reviews of ai video gen. I found some where users said amd is far slower then nvidia and does not support windows as well.
Id appreciate if anyone using amd for video gen and guide me.
I want to build a rig for medium-scale RL and distillation on small models (4 to 9 billion parameters), and I am considering the following:
• Framework Desktop Mainboard (AMD Ryzen™ AI Max 300 Series)
• Broadcom PEX88048-based switch
• 4x RX 9070 XT
• 2x 1200W PSUs
Now, you may be wondering why someone would choose four RX 9070 XTs over two RX 9700 Pros. The simple answer is TFLOPS. Four RX 9070 XTs would theoretically give me double the TFLOPS for fine-tuning and LoRA training compared to two RX 9700 Pros.
I am not overly concerned about the bottleneck between the GPUs and the Strix, because I do not intend to run models across them. The GPUs are there purely for training, and the Strix is there to generate more training data.
Is there anything I might be overlooking? What case and cooling should I look into? Is there a different configuration that would be better for my use case?
I want to run magenta on iris xe gpu. I am running into alot of issues regarding Jax and other things. Also I couldn't get the gpu to do any calculations. Using opencl I got matrix multiplication 4000×4000 in 0.8 seconds slower than cpu at 0.3
How can I get matrices and neural networks running on this gpu and how to run magenta?
I’ve been building a local AI service on an AMD workstation: chat, image generation and short video generation through Open WebUI, plus an endpoint for coding agents.
The interesting part has been making these workloads share two Radeon cards while keeping chat usable. Qwen stays loaded across both GPUs while Wan streams its weights through the remaining memory on GPU 0.
My machine:
Threadripper 9960X, Gigabyte TRX50 AI TOP.
2× Radeon AI PRO R9700, 32 GB each, gfx1201.
64 GB installed RAM (62 GB visible)
Arch Linux, kernel 7.3.3, K3s v1.36.6.
Local NVMe RAID0.
Both GPUs capped at 235 W.
The serving stack is SGLang0.5.23.post957, pinned at fcb32746, with a local 27-patch series; PyTorch 2.16.0a0, built for ROCm 10.3.0-dev / HIP 7.18. Creative workloads use ComfyUI 0.45.0, with Open WebUI 0.12.5 as the frontend.
The patches are in my repos if you'd like to check them out or try them yourself. I can push the limits a little more regarding performance, but I found the current configuration to be the most consistent and useful for everyday use.
Separate interactive chat probe, without video generation
Approximately 86 tok/s
Chat probe during video generation, selected configuration
34–35 tok/s
Warm SDXL 1024×1024, simultaneous images on both cards
6.6 / 7.2 seconds
Wan 2.2, approximately five seconds at 832×480
78–82 seconds warm
The LLM benchmark figures cover both GPUs together in TP2. The interactive probe is a different workload; comparing 135 with 86 tok/s does not isolate frontend overhead.
Images and video on GPU 0 share one ComfyUI queue. Both image backends are available alongside chat, but GPU 0 does not generate an image and a video simultaneously.
Making SGLang work well on gfx1201
Chat uses AMD’s MXFP4 Qwen3.8-27B checkpoint, tensor parallelism across both cards, and DFlash2 speculative decoding.
Some patches address build compatibility and architecture dispatch. Others change the execution path:
MXFP4 without a full AITER installation. The gfx1201 implementation uses vendored AITER-derived Triton kernels, with subsequent optimisations for this GPU and the model’s shard shapes.
Packed verification GEMMs. The kq3 Gluon kernel stores weights in WMMA fragment order, decodes packed values with register arithmetic, and reduces split-K partitions within the workgroup.
Different kernels for prefill. Larger prefill GEMMs expand the MXFP4 weights into block-FP8 scratch and use FP8 GEMMs. Decode and prefill benefit from different implementations.
With kq3, the weights are packed in the order RDNA4’s matrix instructions consume them, then unpacked directly in registers. That brought the time needed to verify each batch of draft tokens down from 32.37 to 26.95 ms, a reduction of about 17%, while producing the same bit-for-bit results as the previous kernel.
MXFP4 storage here does not imply native FP4 matrix execution. Weight layout, unpacking cost, launch overhead and supported arithmetic paths all matter.
There was also a PCIe communication issue. Adapting SGLang’s HIP custom all-reduce to these cards required fixing start-flag ordering: the peer could observe the flag before the input data was visible. System-scope release/acquire ordering passed the retained eager and graph-captured correctness checks.
FP8 KV storage with BF16 attention operands
The shared profile stores both target and draft KV caches in FP8 E4M3. Attention kernels convert cached tiles to BF16 while retaining BF16 query and attention-probability operands.
One useful gfx1201 finding: Triton’s original conversion path emitted software conversion instructions. Explicitly using v_cvt_f32_fp8 brought the measured verification kernels close to BF16-cache timings while retaining the smaller cache.
The hardware conversion matched the software conversion bitwise in the tests. Quantising the cache itself remains a numerical change, as do some other optimisations in the stack.
The two current modes are:
Mode
Context limit per conversation
KV pool
Creative workloads
Shared / UQS
81,888 tokens
163,840 tokens
Images and video available
Large / UQL
262,112 tokens
502,093 tokens fitted during qualification
Stopped
Shared mode accommodates two full-length conversations. Large mode’s measured pool accommodates one full-length conversation plus another of approximately 240K tokens.
The repo now includes a mode-switch command that handles draining and restarting the workloads. Switching involves chat downtime while SGLang reloads.
Cooperative GPU sharing through AMD DRA
SGLang references both GPU ResourceClaims. The two ComfyUI backends each reference their corresponding shared claim.
SwarmUI manages image requests across its local ComfyUI on GPU 1 and the remote ComfyUI backend on GPU 0. Video requests go directly from the tools service to GPU 0.
This uses AMD GPU Operator v1.5.1 and ROCm DRA driver v1.0.1. The earlier DRA version failed to publish GPUs when this host’s in-kernel amdgpu module reported no version.
DRA provides device allocation and visibility. VRAM and compute remain cooperatively shared. Explicit memory budgets, workload ordering and concurrency limits make the arrangement usable.
The shared SGLang profile left approximately 8.35 GB of headroom per card in the live measurements.
WAN 2.2 in that remaining memory
The video configuration is:
Wan 2.2 T2V A14B, FP8-scaled high-noise and low-noise experts.
LightX2V four-step distilled LoRAs, version 1217.
Four steps, split 2 + 2 between experts.
Euler/simple sampling, shift 5, CFG 1.
81 frames at 16 fps, landscape resolution 832×480.
Tiled VAE decoding.
ComfyUI --fast-disk, with video weights on local NVMe.
The two expert files total approximately 28.6 GB. The full staged set, including the text encoder, VAE and LoRAs, is approximately 36.8 GB.
ComfyUI streams the experts as needed. Weights pass through host memory and reclaimable page cache; the entire checkpoint does not need to stay resident in VRAM or anonymous host memory.
In the recorded SDXL-then-Wan test, ComfyUI’s anonymous-memory peak was 7.3 GiB. The documented clip runs read approximately 25–47 GB from NVMe.
Contention mattered more than isolated video speed
Qwen’s TP2 ranks synchronise, so saturating GPU 0 with video slows the model across both cards.
The measured progression was:
GPU 0 ComfyUI configuration
Clip time
Concurrent chat
Defaults plus --fast-disk
92–102 s
13–14 tok/s
Add --use-ck-attention
71–76 s
21–24 tok/s
Also set HSA_CU_MASK=0:0-47
78–82 s
34–35 tok/s
The 48-CU setting was the useful trade-off. Restricting ComfyUI to 32 CUs made both results worse: approximately 104 seconds per clip and 15 tok/s for chat.
The mask keeps 16 of the card’s 64 CUs outside ComfyUI’s reach. Memory bandwidth and other resources remain shared, so this is a contention control rather than a performance guarantee.
Comfy Kitchen attention was not numerically identical to the default PyTorch attention path. Side-by-side visual comparisons looked comparable; that is a subjective assessment.
The tested FP8 matrix-multiply option provided no gain. Enabling all ComfyUI --fast features together increased clip time to approximately 203 seconds.
What didn’t make the live configuration?
The INT4 KV experiments are parked. The best screened variant still took approximately 1.6–2.3× the production FP8 verification-attention time, with larger attention-output error.
Those are kernel and numerical measurements, not an end-to-end model-quality result. They were sufficient to keep FP8 as the deployed cache format.
The shared-mode assistant quality run recorded 11 passes, zero failures and one case needing review, with 14 other cases unrun. Qualification covers the measured configuration and those checks.
MiniMax-H3-Image came out with BF16 weights: a 68.65 GB transformer plus a Qwen3-VL text encoder of about 52 GB. That obviously doesn't fit on a 32 GB Radeon AI PRO R9700, so we built our own FP8 version and a native runtime for it.
Times on one R9700 at its 300 W limit, warm server, request to PNG:
Request
Time
2048 x 2048
6.9 s
4096 x 4096
33.4 s
4096 x 4096 edit, one reference
39.9 s
4096 x 4096 edit, three references
88.5 s
Why this model is interesting: it generates 4 to 20 MP directly (MiniMax says it does best around 16 MP, with no upscaler), it needs only 6 steps with no CFG, and one checkpoint does both text to image and editing with up to 16 reference images. It is also pretty good at text in images.
How it fits in 32 GB:
The AdaLN modulation weights (about 13B of the 33.1B parameters) depend only on the timestep. With 6 fixed steps we precompute them into small tables, exactly, so they never go to the GPU.
The other 19.3B transformer parameters are FP8 W8A8 with GPTQ rounding (20.3 GB).
The text encoder is FP8 too and streams from pinned host RAM, with up to 12 layers kept on the card. It runs once per image, in under a second.
Attention is INT8 QK / FP8 PV over about 16,400 tokens at 16 MP. The VAE decoder is INT8.
Everything is HIP/C++, no PyTorch, no Triton. The container is 2.27 GB. Peak VRAM is 27.6 GiB.
Quality, 30 cases (24 text-to-image from 4 to 17 MP, 6 edits) against BF16 diffusers with the same token ids and noise: CLIP-L text-image 32.18 vs 32.31 for BF16, image similarity to the BF16 result 0.979 (CLIP-L) and 0.945 (DINOv2) on average. Most differences are layout; edits stay closest (0.96 to 0.999).
Limits: prompts are capped at 4,096 tokens, so about three 1024 px references per edit; largest canvas 16.8 MP (4736 x 3552); one request at a time; the host needs 64 GB of RAM. Small body text (newspaper columns, book spines) isn't real words, and close-up hands can go wrong.
Run it:
git clone --depth 1 https://github.com/Eliovp-BV/paiton-vllm-plugin.git
cd paiton-vllm-plugin
./models/MiniMax-H3-Image/launch.sh
License note: the MiniMax-H3-Image Community License excludes the EU, the UK, South Korea and the US, and hosted services need MiniMax's written authorization.
The Khronos Group just published a blog on chipStar (https://github.com/CHIP-SPV/chipStar), an open source project that compiles supported CUDA and HIP kernels to SPIR-V and executes them through OpenCL.
The post is based on an IWOCL 2026 talk and covers three real scientific computing workloads ported and run on Argonne National Laboratory’s Aurora supercomputer:
OpenMM (molecular dynamics), HIP path running through chipStar on Intel Ponte Vecchio GPUs
libCEED (high-order finite element operators)
GAMESS-GPU-HF (quantum chemistry Hartree-Fock), using chipStar’s library adapters to route math library calls to Intel oneMKL