guoyuhong opened a new pull request, #440: URL: https://github.com/apache/tvm-ffi/pull/440
I'm using SGLang, which relies on TVM-FFI, on a machine equipped with an NVIDIA A10 GPU. We encountered the following error: <img width="2864" height="2180" alt="image" src="https://github.com/user-attachments/assets/d014ed35-7940-44b1-bf36-6950a9d6d14f"; /> This issue is commonly observed on NVIDIA Ampere-generation GPUs (e.g., A10, A100) — see related discussion: https://github.com/sgl-project/sglang/issues/18108. The root cause is that older NVIDIA drivers (commonly deployed on Ampere systems) do not support the compute_cap query field in nvidia-smi. As a result, _get_cuda_target fails when trying to auto-detect the CUDA compute capability. <img width="862" height="66" alt="image" src="https://github.com/user-attachments/assets/66af252d-baeb-48f7-af1a-539e27d62899"; /> To address this, we fall back to querying the GPU name via: ```bash nvidia-smi --query-gpu=name --format=csv,noheader ``` <img width="728" height="96" alt="image" src="https://github.com/user-attachments/assets/102c58f4-69ba-4649-ad06-17faaf686699"; /> and then map known Ampere GPU names (e.g., "NVIDIA A10") to their corresponding compute capabilities (e.g., 8.6). This change enables robust GPU detection on Ampere devices with legacy drivers while maintaining backward compatibility. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
