Issue 244078
Summary [fuzzer-found] [NVPTX] Crash lowering atomic fadd on a byval kernel parameter (Windows, 23.1.4jl)
Labels new issue
Assignees
Reporter minansys
    ## Summary

An independent CUDA kernel crashes the NVPTX backend when a field of a by-value
kernel argument is updated with a floating-point atomic. No Enzyme plugin or
automatic differentiation calls are needed to reproduce this.

Emitting device LLVM IR succeeds, and `opt -passes=verify` accepts it. Compiling
that same IR with standalone `llc` crashes in Infer address spaces. The CUDA
source compilation crashes in CodeGen Prepare. Adding
`-mllvm -nvptx-early-byval-copy=1` makes the CUDA compilation succeed.

This was tested with a Julia-distributed LLVM binary, not a stock LLVM release.
It has not been established whether this affects stock LLVM or other host
platforms. The exact defective transformation has not been isolated.

## Environment

- Host: Windows x64, Visual Studio Build Tools x64 environment.
- Compiler: `clang version 23.1.4jl`, Julia LLVM distribution.
- Reported source commit: `3d876d0b0d8b4ac77362bd55cd030de355e83a9e`.
- Compiler distribution's default target: `x86_64-w64-windows-gnu`.
- Explicit host target used below: `x86_64-pc-windows-msvc`.
- CUDA Toolkit: 12.8.
- Device target: `sm_120`, PTX `+ptx87`.
- No Enzyme DLL or other compiler plugin is loaded in the reproduction.

## CUDA Reproducer

Save as `NvptxByvalRepro.cu`. This intentionally uses Clang's `_Atomic` extension
and language-level atomic builtins, rather than assuming CUDA `atomicAdd`
supports thread-local storage. It is a compile-only compiler testcase.

```cuda
#include <cuda_runtime.h>

using dsize_t = unsigned int;

struct Operation {
 float* values_;
    float scale_;
};

struct ShadowOperation {
 float* values_;
    _Atomic(float) scale_;
};

extern "C" __global__ void ByvalKernel(dsize_t count, Operation primal,
 ShadowOperation shadow) {
    const dsize_t index = blockIdx.x * blockDim.x + threadIdx.x;
    if (index >= count || primal.values_ == nullptr ||
        shadow.values_ == nullptr) {
        return;
    }

 const float value = primal.values_[index];
    primal.values_[index] = primal.scale_ * value;
    const float gradient = shadow.values_[index];
 __c11_atomic_fetch_add(&shadow.scale_, gradient * value, __ATOMIC_RELAXED);
    shadow.values_[index] =
 __c11_atomic_load(&shadow.scale_, __ATOMIC_RELAXED) * primal.scale_;
}
```

Run from an x64 Visual Studio Developer Command Prompt with this LLVM's `bin`
directory on `PATH`:

```bat
clang++.exe --target=x86_64-pc-windows-msvc --cuda-path="C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.8" --cuda-gpu-arch=sm_120 --cuda-device-only -std=c++17 -O3 -S -fno-crash-diagnostics NvptxByvalRepro.cu -o baseline.ptx
```

Expected: compile successfully or emit a supported diagnostic, not crash.

Actual: exit 1, reporting `0xC0000005` during CodeGen Prepare on `ByvalKernel`.

## Backend-Only Reproduction

Emit IR without the early-copy workaround, verify it, and compile it directly:

```bat
clang++.exe --target=x86_64-pc-windows-msvc --cuda-path="C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.8" --cuda-gpu-arch=sm_120 --cuda-device-only -std=c++17 -O3 -S -emit-llvm NvptxByvalRepro.cu -o byval.ll
opt.exe -passes=verify -disable-output byval.ll
llc.exe -O3 -mcpu=sm_120 -mattr=+ptx87 byval.ll -o backend.ptx
```

The first two commands exit 0. `llc` exits -1073741819 (`0xC0000005`).
Relevant stack frames, with local filesystem paths and addresses omitted:

```text
Running pass 'Infer address spaces' on function '@ByvalKernel'
Exception Code: 0xC0000005
llvm::GetElementPtrInst::Create
InferAddressSpacesImpl::rewriteWithNewAddressSpaces
InferAddressSpacesImpl::run
InferAddressSpaces::runOnFunction
llvm::FPPassManager::runOnFunction
llvm::FPPassManager::runOnModule
llvm::legacy::PassManagerImpl::run
```

<details>
<summary>Verified LLVM IR (only the source filename/path is sanitized)</summary>

```llvm
source_filename = "NvptxByvalRepro.cu"
target datalayout = "e-p6:32:32-i64:64-i128:128-i256:256-v16:16-v32:32-n16:32:64"
target triple = "nvptx64-nvidia-cuda"

%struct.Operation = type { ptr, float }
%struct.ShadowOperation = type { ptr, float }

define dso_local ptx_kernel void @ByvalKernel(i32 noundef %0, ptr nofree noundef readonly byval(%struct.Operation) align 8 captures(none) %1, ptr nofree noundef byval(%struct.ShadowOperation) align 8 captures(none) %2) local_unnamed_addr #0 {
  %4 = tail call noundef i32 @llvm.nvvm.read.ptx.sreg.ctaid.x()
  %5 = tail call noundef i32 @llvm.nvvm.read.ptx.sreg.ntid.x()
  %6 = mul i32 %4, %5
  %7 = tail call noundef i32 @llvm.nvvm.read.ptx.sreg.tid.x()
  %8 = add i32 %6, %7
  %9 = icmp uge i32 %8, %0
  %10 = load ptr, ptr %1, align 8
 %11 = icmp eq ptr %10, null
  %12 = select i1 %9, i1 true, i1 %11
  %13 = load ptr, ptr %2, align 8
  %14 = icmp eq ptr %13, null
  %15 = select i1 %12, i1 true, i1 %14
  br i1 %15, label %30, label %16

16:
  %17 = zext i32 %8 to i64
  %18 = getelementptr inbounds nuw [4 x i8], ptr %10, i64 %17
  %19 = load float, ptr %18, align 4
  %20 = getelementptr inbounds nuw i8, ptr %1, i64 8
  %21 = load float, ptr %20, align 8
  %22 = fmul contract float %19, %21
  store float %22, ptr %18, align 4
  %23 = getelementptr inbounds nuw [4 x i8], ptr %13, i64 %17
  %24 = load float, ptr %23, align 4
  %25 = getelementptr inbounds nuw i8, ptr %2, i64 8
  %26 = fmul contract float %19, %24
  %27 = atomicrmw fadd ptr %25, float %26 monotonic, align 8
  %28 = load atomic float, ptr %25 monotonic, align 8
 %29 = fmul contract float %21, %28
  store float %29, ptr %23, align 4
  br label %30

30:
  ret void
}

declare noundef range(i32 0, 2147483647) i32 @llvm.nvvm.read.ptx.sreg.ctaid.x() #1
declare noundef range(i32 1, 1025) i32 @llvm.nvvm.read.ptx.sreg.ntid.x() #1
declare noundef range(i32 0, 1024) i32 @llvm.nvvm.read.ptx.sreg.tid.x() #1

attributes #0 = { mustprogress noinline norecurse nounwind willreturn memory(readwrite, inaccessiblemem: none, target_mem: none) "frame-pointer"="all" "no-trapping-math"="true" "stack-protector-buffer-size"="8" "target-cpu"="sm_120" "target-features"="+ptx87,+sm_120" "uniform-work-group-size" }
attributes #1 = { mustprogress nocallback nofree nosync nounwind speculatable willreturn memory(none) }

!llvm.module.flags = !{!0, !1, !2, !3}
!llvm.ident = !{!4, !5}
!nvvmir.version = !{!6}

!0 = !{i32 2, !"SDK Version", [2 x i32] [i32 12, i32 8]}
!1 = !{i32 1, !"wchar_size", i32 2}
!2 = !{i32 4, !"nvvm-reflect-ftz", i32 0}
!3 = !{i32 7, !"frame-pointer", i32 2}
!4 = !{!"clang version 23.1.4jl (/sharedcache/clones/llvm-project.git-5a9787eb535c2edc5dea030cc221c1d60f38c9f42344f410e425ea2139e233aa 3d876d0b0d8b4ac77362bd55cd030de355e83a9e)"}
!5 = !{!"clang version 3.8.0 "}
!6 = !{i32 2, i32 0}
```

</details>

## Workaround and Controls

Adding `-mllvm -nvptx-early-byval-copy=1` to the CUDA compilation command makes
it exit 0 while preserving `-O3` and `sm_120`. This is not a claim that the
underlying LLVM defect has been fixed or that runtime behavior was validated.

Removing the atomic operation made a small control case compile. Volatile-only
and global-memory-atomic controls also compiled. These observations suggest an
interaction with the writable by-value parameter and atomic lowering, but do
not identify the exact defective LLVM source operation.

Only compile-only reproductions were run. No GPU execution, runtime correctness
claim, Linux comparison, stock-LLVM comparison, or regression range is available.

## AI-Assistance Disclosure

The reduced testcase and this report were prepared with GitHub Copilot while
investigating a real CUDA compilation failure. No automated fuzzer was used.
The `[fuzzer-found]` title prefix follows LLVM's current bug-reporting guidance
for AI-generated testcases. All reported compiler outcomes come from local
compiler invocations; untested platforms and distributions are identified above.
_______________________________________________
llvm-bugs mailing list
[email protected]
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-bugs

Reply via email to