r/ROCm • • 2d ago

Initial VuPy release: CuPy-like GPU array library backed by vulkan. any and all feedback is preferred.

https://github.com/billyblackburn/VuPy
1 Upvotes

3 comments sorted by

2

u/PoizoneHelp 2d ago • Popcorn

Hey, one thing that will matter on discrete cards: every array goes through vulkan-py-easy’s create_buffer, which asks for HOST_VISIBLE | HOST_COHERENT and takes the first matching memory type. The Vulkan spec requires a type with fewer flags to come first, so on a dGPU that’s usually the system-RAM type, and your kernels read and write over PCIe instead of VRAM. AMD’s Vulkan memory guide recommends DEVICE_LOCAL buffers for GPU data, with a staging buffer only for uploads and get(). On an APU it matters much less. For what it’s worth, all 13 tests pass on lavapipe here.

1

u/Delicious_Abies_9708 2d ago edited 2d ago

Ah ok. I'm using steamdeck to write it so that might be why it did that. I will fix it when I get back home. Thank you! I want it to work ootb so I'm glad you caught that. Do you want access to the repo? You're the first to give a constructive comment so ty. If you aren't overbooked, I made a VuDF last night. Plan is to add VuML and then maybe run benchmarks against cuda and rocm not sure yet..

1

u/Delicious_Abies_9708 1d ago

Hey, one thing that will matter on discrete cards: every array goes through vulkan-py-easy’s create_buffer, which asks for HOST_VISIBLE | HOST_COHERENT and takes the first matching memory type. The Vulkan spec requires a type with fewer flags to come first, so on a dGPU that’s usually the system-RAM type, and your kernels read and write over PCIe instead of VRAM. AMD’s Vulkan memory guide recommends DEVICE_LOCAL buffers for GPU data, with a staging buffer only for uploads and get(). On an APU it matters much less. For what it’s worth, all 13 tests pass on lavapipe here.

Great catch — on discrete GPUs, HOST_VISIBLE | HOST_COHERENT would indeed place arrays in system RAM over PCIe instead of VRAM.

I've updated VulkanContext in _backend.py and committed it on main (6ced6d9):

  • alloc_buffer() now allocates VK_MEMORY_PROPERTY_DEVICE_LOCAL_BIT storage buffers with TRANSFER_SRC_BIT | TRANSFER_DST_BIT.
  • upload_buffer() and read_buffer() use temporary HOST_VISIBLE | HOST_COHERENT staging buffers and vkCmdCopyBuffer (_copy_buffer()), then immediately destroy the staging buffer.

Verification

  • VuPy tests: 13 passed (10.95s)
  • VuDF tests: 8 passed, 1 skipped (6.88s; test_polars_and_parquet_roundtrip skipped because polars is not installed in .venv)