| Issue |
240874
|
| Summary |
[X86][RegisterCoalescer] `<2 x double>` loses its upper half across a call after the coalescer routes it through an `FR64` vreg
|
| Labels |
new issue
|
| Assignees |
|
| Reporter |
gbaraldi
|
`llc` miscompiles the function below on x86-64. A `<2 x double>` that is live
across a call is spilled with an 8-byte `movsd` and reloaded with an 8-byte
`movsd`, so its upper element comes back as `0.0`.
It reproduces with plain `llc -O2` and no target features on 16.0.6,
18.1.3, 19.1.1, 20.1.2, 21.1.8, 22.1.8 and main (cdda042f213c).
`-join-liveintervals=false` and `-O0` give correct code.
## Reproducer
`repro.ll`:
```llvm
target triple = "x86_64-unknown-linux-gnu"
declare void @clobber()
define <2 x double> @f(<2 x double> %x, double %t0, double %s0, i1 %c1, i1 %c2, i1 %c3, ptr %p, ptr %q) {
entry:
%v0 = insertelement <2 x double> poison, double %s0, i64 0
store <2 x double> %v0, ptr %q
br i1 %c1, label %lo, label %exit
call:
br i1 %c3, label %exit, label %ret2
lo:
%x0 = extractelement <2 x double> %x, i64 0
br i1 %c2, label %call, label %ret1
ret1:
store double %x0, ptr %p
ret <2 x double> zeroinitializer
ret2:
ret <2 x double> zeroinitializer
exit:
%t = phi double [ %t0, %entry ], [ %x0, %call ]
%s = phi double [ %s0, %entry ], [ %x0, %call ]
%v = phi <2 x double> [ %v0, %entry ], [ %x, %call ]
store double %t, ptr %p
%p1 = getelementptr double, ptr %p, i64 1
store double %s, ptr %p1
ret <2 x double> %v
}
```
On the path `entry -> lo -> call -> exit` the funct
unchanged.
`main.c`:
```c
#include <emmintrin.h>
#include <stdio.h>
__m128d f(__m128d x, double t0, double s0, _Bool c1, _Bool c2, _Bool c3,
double *p, __m128d *q);
void clobber(void) {}
int main(void) {
double p[2];
__m128d q;
/* entry -> lo -> call -> exit: the result must b
__m128d r = f(_mm_set_pd(2.0, 1.0), 3.0, 4.0, 1, 1, 1, p, &q);
printf("f returned <%g, %g>, expected <1, 2>\n", r[0], r[1]);
return !(r[0] == 1.0 && r[1] == 2.0);
}
```
```
$ llc -O2 repro.ll -filetype=obj -o repro.o && clang main.c repro.o && ./a.out
f returned <1, 0>, expected <1, 2>
$ llc -O2 -join-liveintervals=false repro.ll -filetype=obj -o repro.o && clang main.c repro.o && ./a.out
f returned <1, 2>, expected <1, 2>
```
`llc -O2 repro.ll` on main, with `%x` in `xmm0`:
```asm
# %bb.4: # %call
pushq %rbx
subq $16, %rsp
movq %rcx, 8(%rsp) # 8
movl %edx, %ebx
movsd %xmm0, (%rsp) # 8-byte Spill <- only x[0] is saved
callq clobber@PLT
movsd (%rsp), %xmm1 # 8
# xmm1 = mem[0],zero
movq 8(%rsp), %rcx # 8-byte Reload
testb $1, %bl
movaps %xmm1, %xmm2
leaq 16(%rsp), %rsp
popq %rbx
je .LBB0_3
.LBB0_5: # %exit
movsd %xmm1, (%rcx)
movsd %xmm2, 8(%rcx)
movaps %xmm2, %xmm0 # <- returned as the vector: <x[0], 0.0>
retq
```
Some not very sloppy AI output ahead, but sounds reasonable to my non backend eyes.
## What happens
The MIR going into the register coalescer is fine (xmm copies only):
```
bb.0.entry:
%7:fr64 = COPY $xmm2 ; s0
%6:fr64 = COPY $xmm1 ; t0
%5:vr128 = COPY $xmm0 ; x
%0:vr128 = COPY %7 ; v0 = scalar_to_vector(s0)
...
%20:fr64 = COPY %6 ; t
%21:fr64 = COPY %7 ; s
%22:vr128 = COPY %0 ; v
bb.1.call:
CALL64pcrel32 @clobber
%20:fr64 = COPY %1 ; t = x[0]
%21:fr64 = COPY %1 ; s = x[0]
%22:vr128 = COPY %5 ; v = x, all 128 bi
bb.2.lo:
%1:fr64 = COPY %5 ; x0 = x[0]
```
The coalescer then does the following:
1. It joins `%1` into `%20`. `%20:fr64` now has two
`entry` and `COPY %5` in `lo`. Joining `%20` wit
with interference, so `%20` stays a separate `fr64` register.
2. It joins `%0`, `%7` and `%21` into one `vr128` r
in `entry`.
3. It joins that register with `%22` through `%22:v
In `call`, both registers are defined while the other is live:
```
%21:vr128 = COPY %20:fr64 ; where %20:fr64 =
%22:vr128 = COPY %5:vr128
```
`JoinVals::analyzeValue` asks `valuesIdentical` about the two.
`followCopyChain` walks `%21 <- %20 <- %5` and `%22 <- %5`, reaches the same
def of `%5` on both sides, and reports the value
coalescer resolves the conflict as `CR_Erase` and deletes
`%22:vr128 = COPY %5:vr128`.
```
228B %22:vr128 = COPY %21:vr128
Considering merging to VR128 with %21 in %2
RHS = %21 [96r,256B:2)[348r,384B:1)[624B,676r:0) 0@624B-phi 1@348r 2@96r
LHS = %22 [228r,256B:2)[356r,384B:156r 2@228r
merge %22:1@356r into %21:1@348r --> @348r
merge %22:2@228r into %21:2@96r --> @96r
merge %21:0@624B into %22:0@624B --> @624B
erased: 356r %22:vr128 = COPY %5
erased: 228r %22:vr128 = COPY %21:vr128
```
After the pass the vector reaches `%22` only throug
and live across the call:
```
bb.1.call:
CALL64pcrel32 @clobber
%22:vr128 = COPY %20 ; v = x, but throug
bb.2.lo:
%20:fr64 = COPY %5 ; last use of %5
bb.5.exit:
$xmm0 = COPY %22
```
`FR64` has a 64-bit spill size, so the allocator sa
`movsd`, and the upper half of `x` is gone.
`followCopyChain` looks through any full `COPY` between virtual registers
(`MI->isFullCopy()`) without looking at register cl
`FR64` and `VR128` contain the same physical registers, and isel uses plain
`COPY`s between them for `extractelement ..., 0` and `scalar_to_vector`. A
`vr128 -> fr64 -> vr128` chain therefore is not val
`fr64` link can be spilled, but `valuesIdentical` treats it as one.
One possible direction (not tested): have `followCopyChain` stop at a copy
whose source register class is narrower than the cl
being joined to (`CP.getNewRC()`), so that a value is never proven identical
through an intermediate register that cannot hold all of it.
## MIR-only reproducer
The same join, reduced by hand to the coalescer inp
`coalescer.mir`:
```
# RUN: llc -mtriple=x86_64-unknown-linux-gnu -run-pass=register-coalescer -verify-coalescing -o - %s
--- |
target triple = "x86_64-unknown-linux-gnu"
declare void @clobber()
define <2 x double> @f(<2 x double> %x, double %t, double %s, i32 %c, ptr %p) {
ret <2 x double> %x
}
...
---
name: f
tracksRegLiveness: true
frameInfo:
adjustsStack: true
hasCalls: true
body: |
bb.0:
successors: %bb.1, %bb.2
liveins: $xmm0, $xmm1, $xmm2, $edi, $rsi
%0:vr128 = COPY $xmm0 ; x, a full <2 x double
%1:fr64 = COPY $xmm1 ; t, a scalar
%2:fr64 = COPY $xmm2 ; s, a scalar
%3:vr128 = COPY %2 ; v = scalar_to_vector(
%4:gr32 = COPY $edi
%5:gr64 = COPY $rsi
TEST32rr %4, %4, implicit-def $eflags
JCC_1 %bb.2, 4, implicit killed $eflags
bb.1:
successors: %bb.2
%1:fr64 = COPY %0 ; t = x[0]
ADJCALLSTACKDOWN64 0, 0, 0, implicit-def dead $s, implicit-def dead $ssp, implicit $rsp, implicit $ssp
CALL64pcrel32 @clobber, csr_64, implicit $rsp, rsp, implicit-def $ssp
ADJCALLSTACKUP64 0, 0, implicit-def dead $rsp, plicit-def dead $ssp, implicit $rsp, implicit $ssp
%2:fr64 = COPY %1 ; s = t (= x[0])
%3:vr128 = COPY %0 ; v = x (all 128 b
bb.2:
MOVSDmr %5, 1, $noreg, 0, $noreg, %1
MOVSDmr %5, 1, $noreg, 8, $noreg, %2
$xmm0 = COPY %3
RET 0, $xmm0
...
```
`llc -run-pass=register-coalescer coalescer.mir -o -` erases
`%3:vr128 = COPY %0` and leaves `bb.1` as:
```
bb.1:
%1:fr64 = COPY %0
ADJCALLSTACKDOWN64 ...
CALL64pcrel32 @clobber, csr_64, ...
ADJCALLSTACKUP64 ...
%3:vr128 = COPY %1
```
`llc -start-before=register-coalescer coalescer.mir
```asm
movsd %xmm3, 8(%rsp) # 8
callq clobber
movsd 8(%rsp), %xmm1 # 8-byte Reload
# xmm1 = me
movaps %xmm1, %xmm0
```
Linked against a `main` that calls
`f(_mm_set_pd(2.0, 1.0), 3.0, 4.0, 1, p)`, it returns `<1, 0>` on 18.1.3,
20.1.2, 22.1.8 and main, and `<1, 2>` with `-join-liveintervals=false`.
## Origin
Found through a Julia miscompile, https://github.com/JuliaLang/julia/issues/64827
(Julia 1.12 with LLVM 18.1.7 and Julia 1.13 with LL
that code the vector is a loop-carried `NTuple{2,Float64}` and the `fr64`
register is a loop phi holding its first element. T
only miscompiled by `llc` up to 20.x, because 21 an
different MIR for it before the coalescer. Feeding
MIR of that function to the 22.1.8 or main coalesce
code.
_______________________________________________
llvm-bugs mailing list
[email protected]
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-bugs