Hi Francesco, On 2026-10-23 10:09, Francesco Poli wrote:
Does this also mean that I cannot (currently) use this integrated GPU for GPGPU computations? I haven't yet tried...
You can use your integrated GPU for GPGPU computations, though it may be more difficult. You have to build the code yourself rather than relying on Debian's prebuilt packages. That means no use of the rocblas, rocsparse, rocsolver, or rocrand packages. The rocfft and hipfft packages are purely based on run-time compilation, so they probably work. Similarly, header-only libraries like rocprim, rocthrust, and hipcub might be usable.
When libraries are built for the HIP language, the GPU code generated by the compiler is /native machine code/. This is unlike other GPU computational frameworks like Vulkan, OpenGL, or OpenCL, where application binaries generally either depend on run-time compilation (RTC) of shader sources, or ship an intermediate representation of the shaders (such as SPIR-V) and then JIT to native GPU code on the user's machine.
The HIP toolchain and runtime in Debian Testing and Unstable can build and run GPGPU programs and libraries for gfx90c. However, the prebuilt library and program packages included in Debian are not built for gfx90c. If the prebuilt libggml0-backend-hip and librocblas4 packages contained gfx90c machine code, then they would run on your system.
In any case, there are two main reasons why we don't build the Debian packages for gfx90c:
1. Upstream libraries like rocblas include a lot of compile-time and run-time if/else cases that branch on the detected gfx architecture, but the code is not written to handle gfx90c (as it is not on their official support list). To successfully build, we'd need to develop and maintain patches providing fallback code paths. I'd like to do that, but my time is limited.
2. The AMD ROCm packages are enormous on disk and they take an extremely long time to compile. Both size-on-disk and time-to-compile scale linearly with the number of architectures you build for. We're building for less than half of the potential architectures we could target but we're already running up against various limits in our tools and infrastructure.
Your guess seems to be confirmed by the actual numbers. It seems that the Vulkan backend performs better.
Thank you for sharing. It's always nice to see actual benchmarks to confirm my suspicions.
Also, if you have anything to comment about my comparison of Vulkan and CPU backends (why is `pp512` performance almost double, while `tg128` is only very slightly better), I would be very interested in hearing your take on that...
I'm afraid that I don't know enough to do anything but speculate. My suspicion would be that token generation gets little benefit from the APU hardware because it is memory-bound and it cannot be significantly optimized for that reason. Prompt processing, on the other hand, is compute-bound and there's more room for the Vulkan implementation to distinguish itself by making more efficient use of the APU hardware. However, this is nothing more than a guess.
Sincerely, Cory Bloor
