Issue 241283
Summary [clang][X86-64] MEMORY-class vector return emitted direct; sret demotion breaks va_arg (wrong code at -O0)
Labels clang
Assignees
Reporter jeffchen006
    A variadic function returns a 128-byte vector (MEMORY class). Clang emits a direct return and assumes 6 free GPRs. The back end demotes the return to a hidden sret pointer in RDI, so all GPR arguments shift by one, and `va_arg(ap, S)` reads the wrong slots. At -O1+ the return value is removed by IPO and both sides agree again.

**Reproducer** (`repro.c`)
```c
/* f reads its vararg s = {6, 7} and prints it.  Correct output: 6 7
   The 128-byte vector return value is MEMORY class; clang passes it "direct",
 the back end then adds a hidden return pointer in RDI, and clang's va_arg
 counts GPRs without it, so s is read one register off. */
#include <stdarg.h>
#include <stdio.h>
typedef int V __attribute__((vector_size(128)));
typedef struct { long a, b; } S;
__attribute__((noinline)) static V f(long p1, long p2, long p3, long p4, ...) {
  va_list ap;
  va_start(ap, p4);
  S s = va_arg(ap, S);
 va_end(ap);
  printf("%ld %ld\n", s.a, s.b);
  return (V){0};
}
int main(void) {
  S s = {6, 7};
  f(0, 0, 0, 0, s);
  return 0;
}
```

**Commands and output** (clang version 24.0.0git (https://github.com/llvm/llvm-project.git 8207e9143575), Target: x86_64-unknown-linux-gnu):
```console
$ clang -O0 repro.c -o o0
$ ./o0
7 0
$ clang -O3 repro.c -o o3
$ ./o3
6 7
```
Expected (as `-O3`): `6 7`. `-O0` prints `7 0`.

**Why the expected output is right**
The call passes exactly one `S` vararg and `va_arg` reads one `S`. UBSan/ASan/MSan at -O0 report nothing.

**How the -O0/-O3 difference arises**
Here -O0 is the wrong one. At -O1+ deadargelim/IPSCCP make the internal function return void, the sret demotion goes away, and the caller and callee agree again (the bug is hidden, not fixed).

**Potential Root cause**
`clang/lib/CodeGen/Targets/X86.cpp`, `X86_64ABIInfo::classifyReturnType`: a vector of more than 32 bytes is MEMORY (here a 128-byte vector), so it calls `getIndirectReturnResult`, which returns `ABIArgInfo::getDirect()` for a non-aggregate type (only `_BitInt` gets a real sret). `computeInfo` decrements `FreeIntRegs` only for indirect returns. The back end cannot return this vector in registers (CanLowerReturn fails) and demotes it to sret in RDI. Clang's `va_arg(S)` then reads one register off.

**Possible Proposed fix**
Return MEMORY-class vectors with an explicit sret, like `_BitInt`:
```diff
 ABIArgInfo X86_64ABIInfo::getIndirectReturnResult(QualType Ty) const {
   if (!isAggregateTypeForABI(Ty)) {
     ...
-    if (Ty->isBitIntType())
+ if (Ty->isBitIntType() || Ty->isVectorType())
       return getNaturalAlignIndirect(Ty, getDataLayout().getAllocaAddrSpace());
```
This changes the ABI for MEMORY-class vector returns that the back end now returns in registers (see also #42798); it can be restricted to the vectors that the back end demotes anyway.

_______________________________________________
llvm-bugs mailing list
[email protected]
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-bugs

Reply via email to