control: severity -1 wishlist
Hi Francesco,
On 2026-10-22 12:23, Francesco Poli (frx) wrote:
$ llama-bench -hf MaziyarPanahi/Mistral-7B-Instruct-v0.3-GGUF
ggml_cuda_init: found 1 ROCm devices (Total VRAM: 13977 MiB):
Device 0: AMD Radeon Graphics, gfx90c:xnack+ (0x90c), VMM: no, Wave Size:
64, VRAM: 13977 MiB
load_backend: loaded ROCm backend from
/usr/lib/x86_64-linux-gnu/ggml/backends0/libggml-hip.so
ggml_vulkan: Found 1 Vulkan devices:
ggml_vulkan: 0 = AMD Radeon Graphics (RADV RENOIR) (radv) | uma: 1 | fp16: 1 |
bf16: 0 | fp4: 0 | warp size: 64 | shared memory: 65536 | int dot: 0 | matrix
cores: none
load_backend: loaded Vulkan backend from
/usr/lib/x86_64-linux-gnu/ggml/backends0/libggml-vulkan.so
load_backend: loaded CPU backend from
/usr/lib/x86_64-linux-gnu/ggml/backends0/libggml-cpu-haswell.so
| model | size | params | backend | ngl |
test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: |
--------------: | -------------------: |
ROCm error: invalid device function
current device: 0, in function ggml_cuda_kernel_launch at
/build/reproducible-path/ggml-0.27.0/src/ggml-cuda/common.cuh:1720
hipGetLastError()
/build/reproducible-path/ggml-0.27.0/src/ggml-cuda/ggml-cuda.cu:109: ROCm error
The "invalid device function" error that you're seeing is the current
ROCm way of saying "the library you're trying to use was not built for
your GPU." The gfx90c architecture is not currently on the list of GPUs
supported by ROCm math libraries in Unstable or Forky, because the
"fallback architecture" patch we were using to enable gfx90c in ROCm 5
could not easily be carried over into ROCm 6 or later. This was
mentioned briefly in the NEWS for rocm-hipamd 6.4.3-1 [1]. There is a
suggested workaround on the Supported GPU wiki page [2] (which also
reminds me that the wiki has not yet been updated for Forky or current
Unstable). For posterity, I'll repeat the suggestion here: try setting
`export HSA_OVERRIDE_GFX_VERSION=9.0.0` in your environment to use the
code objects built for gfx900 on your gfx90c GPU.
I'm hoping we'll be able to get gfx90c working without needing those
sorts of hacks for Forky, but I cannot make any hard promises. All I can
say is that I will do my best. However, I'd be surprised if the HIP
backend performs better than the vulkan implementation on gfx90c even if
I get it working. I don't think upstream has done much rocblas kernel
tuning for gfx90c (if any), but tuned kernels are vital for achieving
optimal performance on the ROCm BLAS stack.
Sincerely,
Cory Bloor
[1]:
https://salsa.debian.org/rocm-team/rocm-hipamd/-/blob/debian/6.4.3-1/debian/NEWS?ref_type=tags
[2]:
https://salsa.debian.org/rocm-team/community/team-project/-/wikis/Supported-GPU-list#gfx90c
