| Issue |
244078
|
| Summary |
[fuzzer-found] [NVPTX] Crash lowering atomic fadd on a byval kernel parameter (Windows, 23.1.4jl)
|
| Labels |
new issue
|
| Assignees |
|
| Reporter |
minansys
|
## Summary
An independent CUDA kernel crashes the NVPTX backend when a field of a by-value
kernel argument is updated with a floating-point atomic. No Enzyme plugin or
automatic differentiation calls are needed to reproduce this.
Emitting device LLVM IR succeeds, and `opt -passes=verify` accepts it. Compiling
that same IR with standalone `llc` crashes in Infer address spaces. The CUDA
source compilation crashes in CodeGen Prepare. Adding
`-mllvm -nvptx-early-byval-copy=1` makes the CUDA compilation succeed.
This was tested with a Julia-distributed LLVM binary, not a stock LLVM release.
It has not been established whether this affects stock LLVM or other host
platforms. The exact defective transformation has not been isolated.
## Environment
- Host: Windows x64, Visual Studio Build Tools x64 environment.
- Compiler: `clang version 23.1.4jl`, Julia LLVM distribution.
- Reported source commit: `3d876d0b0d8b4ac77362bd55cd030de355e83a9e`.
- Compiler distribution's default target: `x86_64-w64-windows-gnu`.
- Explicit host target used below: `x86_64-pc-windows-msvc`.
- CUDA Toolkit: 12.8.
- Device target: `sm_120`, PTX `+ptx87`.
- No Enzyme DLL or other compiler plugin is loaded in the reproduction.
## CUDA Reproducer
Save as `NvptxByvalRepro.cu`. This intentionally uses Clang's `_Atomic` extension
and language-level atomic builtins, rather than assuming CUDA `atomicAdd`
supports thread-local storage. It is a compile-only compiler testcase.
```cuda
#include <cuda_runtime.h>
using dsize_t = unsigned int;
struct Operation {
float* values_;
float scale_;
};
struct ShadowOperation {
float* values_;
_Atomic(float) scale_;
};
extern "C" __global__ void ByvalKernel(dsize_t count, Operation primal,
ShadowOperation shadow) {
const dsize_t index = blockIdx.x * blockDim.x + threadIdx.x;
if (index >= count || primal.values_ == nullptr ||
shadow.values_ == nullptr) {
return;
}
const float value = primal.values_[index];
primal.values_[index] = primal.scale_ * value;
const float gradient = shadow.values_[index];
__c11_atomic_fetch_add(&shadow.scale_, gradient * value, __ATOMIC_RELAXED);
shadow.values_[index] =
__c11_atomic_load(&shadow.scale_, __ATOMIC_RELAXED) * primal.scale_;
}
```
Run from an x64 Visual Studio Developer Command Prompt with this LLVM's `bin`
directory on `PATH`:
```bat
clang++.exe --target=x86_64-pc-windows-msvc --cuda-path="C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.8" --cuda-gpu-arch=sm_120 --cuda-device-only -std=c++17 -O3 -S -fno-crash-diagnostics NvptxByvalRepro.cu -o baseline.ptx
```
Expected: compile successfully or emit a supported diagnostic, not crash.
Actual: exit 1, reporting `0xC0000005` during CodeGen Prepare on `ByvalKernel`.
## Backend-Only Reproduction
Emit IR without the early-copy workaround, verify it, and compile it directly:
```bat
clang++.exe --target=x86_64-pc-windows-msvc --cuda-path="C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.8" --cuda-gpu-arch=sm_120 --cuda-device-only -std=c++17 -O3 -S -emit-llvm NvptxByvalRepro.cu -o byval.ll
opt.exe -passes=verify -disable-output byval.ll
llc.exe -O3 -mcpu=sm_120 -mattr=+ptx87 byval.ll -o backend.ptx
```
The first two commands exit 0. `llc` exits -1073741819 (`0xC0000005`).
Relevant stack frames, with local filesystem paths and addresses omitted:
```text
Running pass 'Infer address spaces' on function '@ByvalKernel'
Exception Code: 0xC0000005
llvm::GetElementPtrInst::Create
InferAddressSpacesImpl::rewriteWithNewAddressSpaces
InferAddressSpacesImpl::run
InferAddressSpaces::runOnFunction
llvm::FPPassManager::runOnFunction
llvm::FPPassManager::runOnModule
llvm::legacy::PassManagerImpl::run
```
<details>
<summary>Verified LLVM IR (only the source filename/path is sanitized)</summary>
```llvm
source_filename = "NvptxByvalRepro.cu"
target datalayout = "e-p6:32:32-i64:64-i128:128-i256:256-v16:16-v32:32-n16:32:64"
target triple = "nvptx64-nvidia-cuda"
%struct.Operation = type { ptr, float }
%struct.ShadowOperation = type { ptr, float }
define dso_local ptx_kernel void @ByvalKernel(i32 noundef %0, ptr nofree noundef readonly byval(%struct.Operation) align 8 captures(none) %1, ptr nofree noundef byval(%struct.ShadowOperation) align 8 captures(none) %2) local_unnamed_addr #0 {
%4 = tail call noundef i32 @llvm.nvvm.read.ptx.sreg.ctaid.x()
%5 = tail call noundef i32 @llvm.nvvm.read.ptx.sreg.ntid.x()
%6 = mul i32 %4, %5
%7 = tail call noundef i32 @llvm.nvvm.read.ptx.sreg.tid.x()
%8 = add i32 %6, %7
%9 = icmp uge i32 %8, %0
%10 = load ptr, ptr %1, align 8
%11 = icmp eq ptr %10, null
%12 = select i1 %9, i1 true, i1 %11
%13 = load ptr, ptr %2, align 8
%14 = icmp eq ptr %13, null
%15 = select i1 %12, i1 true, i1 %14
br i1 %15, label %30, label %16
16:
%17 = zext i32 %8 to i64
%18 = getelementptr inbounds nuw [4 x i8], ptr %10, i64 %17
%19 = load float, ptr %18, align 4
%20 = getelementptr inbounds nuw i8, ptr %1, i64 8
%21 = load float, ptr %20, align 8
%22 = fmul contract float %19, %21
store float %22, ptr %18, align 4
%23 = getelementptr inbounds nuw [4 x i8], ptr %13, i64 %17
%24 = load float, ptr %23, align 4
%25 = getelementptr inbounds nuw i8, ptr %2, i64 8
%26 = fmul contract float %19, %24
%27 = atomicrmw fadd ptr %25, float %26 monotonic, align 8
%28 = load atomic float, ptr %25 monotonic, align 8
%29 = fmul contract float %21, %28
store float %29, ptr %23, align 4
br label %30
30:
ret void
}
declare noundef range(i32 0, 2147483647) i32 @llvm.nvvm.read.ptx.sreg.ctaid.x() #1
declare noundef range(i32 1, 1025) i32 @llvm.nvvm.read.ptx.sreg.ntid.x() #1
declare noundef range(i32 0, 1024) i32 @llvm.nvvm.read.ptx.sreg.tid.x() #1
attributes #0 = { mustprogress noinline norecurse nounwind willreturn memory(readwrite, inaccessiblemem: none, target_mem: none) "frame-pointer"="all" "no-trapping-math"="true" "stack-protector-buffer-size"="8" "target-cpu"="sm_120" "target-features"="+ptx87,+sm_120" "uniform-work-group-size" }
attributes #1 = { mustprogress nocallback nofree nosync nounwind speculatable willreturn memory(none) }
!llvm.module.flags = !{!0, !1, !2, !3}
!llvm.ident = !{!4, !5}
!nvvmir.version = !{!6}
!0 = !{i32 2, !"SDK Version", [2 x i32] [i32 12, i32 8]}
!1 = !{i32 1, !"wchar_size", i32 2}
!2 = !{i32 4, !"nvvm-reflect-ftz", i32 0}
!3 = !{i32 7, !"frame-pointer", i32 2}
!4 = !{!"clang version 23.1.4jl (/sharedcache/clones/llvm-project.git-5a9787eb535c2edc5dea030cc221c1d60f38c9f42344f410e425ea2139e233aa 3d876d0b0d8b4ac77362bd55cd030de355e83a9e)"}
!5 = !{!"clang version 3.8.0 "}
!6 = !{i32 2, i32 0}
```
</details>
## Workaround and Controls
Adding `-mllvm -nvptx-early-byval-copy=1` to the CUDA compilation command makes
it exit 0 while preserving `-O3` and `sm_120`. This is not a claim that the
underlying LLVM defect has been fixed or that runtime behavior was validated.
Removing the atomic operation made a small control case compile. Volatile-only
and global-memory-atomic controls also compiled. These observations suggest an
interaction with the writable by-value parameter and atomic lowering, but do
not identify the exact defective LLVM source operation.
Only compile-only reproductions were run. No GPU execution, runtime correctness
claim, Linux comparison, stock-LLVM comparison, or regression range is available.
## AI-Assistance Disclosure
The reduced testcase and this report were prepared with GitHub Copilot while
investigating a real CUDA compilation failure. No automated fuzzer was used.
The `[fuzzer-found]` title prefix follows LLVM's current bug-reporting guidance
for AI-generated testcases. All reported compiler outcomes come from local
compiler invocations; untested platforms and distributions are identified above.
_______________________________________________
llvm-bugs mailing list
[email protected]
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-bugs