https://github.com/vpykhtin updated https://github.com/llvm/llvm-project/pull/219230
>From 4ac9b563698da5b71c2729b373a2ad4034b91b64 Mon Sep 17 00:00:00 2001 From: Valery Pykhtin <[email protected]> Date: Wed, 26 Aug 2026 17:57:52 +0000 Subject: [PATCH 1/3] [AMDGPU] Verify VGPR tuple alignment from the operand register class The machine verifier decided VGPR tuple alignment with isProperlyAlignedRC(), which inspects only the register's own class. Alignment is not really a property of the register in isolation: whether a 64-bit tuple must be even-aligned depends on the operand it feeds, and on mixed-alignment targets the same register class can be required to be aligned in one operand and exempt in another. Inspecting only the register also conflates alignment with unrelated problems - a register that is simply the wrong bank or size for the operand came out as "requires even aligned vector registers" as well. Make the operand's register class the source of truth instead: a register is misaligned only when it does not satisfy the operand's class but its even-aligned same-bank/width equivalent (SIRegisterInfo::getAlignedEquivalentRC) would. A register that fits neither is a genuine class or bank mismatch and is left to the illegal-register and sub-register checks. So an AGPR in a VGPR|SGPR (VS_64) operand is now reported as an illegal register, and a wrong-size register (e.g. a 64-bit VGPR in a 128-bit MFMA source) or an invalid sub-register index is reported by those checks alone, no longer doubled up as an "even aligned" error. This drops the redundant diagnostics in tests. Deriving the requirement from the operand class also lets several special cases go away. The RegClass == -1 early-out is hoisted so the operand class is always valid, and the V_MOV_B64_PSEUDO / AV_MOV_B64_IMM_PSEUDO / spill exemptions are dropped: those operands use unaligned register classes (VReg_64, AV_64, and the spill classes), which every register already satisfies, so the comparison never flags them. Inline-asm operands (RegClass == -1) are no longer alignment-checked, matching the prior FIXME that they were never meaningfully verified. The same reasoning removes the DS_GWS-specific alignment check: on subtargets that require aligned VGPRs the DS_GWS data0 operand has the AV_64_Align2 register class, so the operand-class check above already diagnoses its alignment and the separate check is redundant. The image vaddr operand is a plain VGPR_32 whose class cannot encode even-alignment, so its dedicated position check is kept. Co-Authored-By: Claude <[email protected]> --- llvm/lib/Target/AMDGPU/SIInstrInfo.cpp | 70 +++++++++---------- llvm/lib/Target/AMDGPU/SIRegisterInfo.cpp | 15 ++++ llvm/lib/Target/AMDGPU/SIRegisterInfo.h | 10 +++ .../CodeGen/AMDGPU/verify-ds-gws-align.mir | 10 --- .../AMDGPU/verify-gfx90a-aligned-vgprs.mir | 10 --- ...ported-subreg-index-aligned-vgpr-check.mir | 4 -- ...rted-unaligned-vgpr-check-vsrc-operand.mir | 2 - 7 files changed, 57 insertions(+), 64 deletions(-) diff --git a/llvm/lib/Target/AMDGPU/SIInstrInfo.cpp b/llvm/lib/Target/AMDGPU/SIInstrInfo.cpp index 29a22a324447e..329b9440de0d0 100644 --- a/llvm/lib/Target/AMDGPU/SIInstrInfo.cpp +++ b/llvm/lib/Target/AMDGPU/SIInstrInfo.cpp @@ -5512,44 +5512,48 @@ bool SIInstrInfo::verifyInstruction(const MachineInstr &MI, break; } + // Operands without a fixed register class (RegClass == -1), such as inline + // asm operands, are not verified here. + if (RegClass == -1) + continue; + if (!MO.isReg()) continue; Register Reg = MO.getReg(); if (!Reg) continue; - // FIXME: Ideally we would have separate instruction definitions with the - // aligned register constraint. - // FIXME: We do not verify inline asm operands, but custom inline asm - // verification is broken anyway - if (ST.needsAlignedVGPRs() && Opcode != AMDGPU::AV_MOV_B64_IMM_PSEUDO && - Opcode != AMDGPU::V_MOV_B64_PSEUDO && !isSpill(MI)) { - const TargetRegisterClass *RC = RI.getRegClassForReg(MRI, Reg); - if (RI.hasVectorRegisters(RC) && MO.getSubReg()) { - if (const TargetRegisterClass *SubRC = - RI.getSubRegisterClass(RC, MO.getSubReg())) { - RC = RI.getCompatibleSubRegClass(RC, SubRC, MO.getSubReg()); - if (RC) - RC = SubRC; + const TargetRegisterClass *OpRC = RI.getRegClass(RegClass); + + if (ST.needsAlignedVGPRs()) { + const TargetRegisterClass *RegRC = RI.getRegClassForReg(MRI, Reg); + if (RI.hasVectorRegisters(RegRC)) { + if (MO.getSubReg()) { + // Narrow to the sub-register's class. getSubRegisterClass already + // accounts for the sub-register index's alignment within the tuple + // (an odd-aligned slice yields an unaligned class, caught below); a + // null result means an invalid sub-register index and is left to the + // sub-register check. + RegRC = RI.getSubRegisterClass(RegRC, MO.getSubReg()); + } + // Flag an alignment-only mismatch: the register does not satisfy the + // operand's class, but its even-aligned same-bank/width equivalent + // would. A bank or size mismatch fails even when aligned, so it is left + // to the illegal-register / sub-register checks. + if (RegRC && !OpRC->hasSubClassEq(RegRC)) { + const TargetRegisterClass *AlignedRegRC = + RI.getAlignedEquivalentRC(RegRC); + if (AlignedRegRC && OpRC->hasSubClassEq(AlignedRegRC)) { + ErrInfo = "Subtarget requires even aligned vector registers"; + return false; + } } - } - - // Check that this is the aligned version of the class. - if (!RC || !RI.isProperlyAlignedRC(*RC)) { - ErrInfo = "Subtarget requires even aligned vector registers"; - return false; } } - if (RegClass != -1) { - if (Reg.isVirtual()) - continue; - - const TargetRegisterClass *RC = RI.getRegClass(RegClass); - if (!RC->contains(Reg)) { - ErrInfo = "Operand has incorrect register class."; - return false; - } + if (Reg.isPhysical() && !OpRC->contains(Reg)) { + ErrInfo = "Operand has incorrect register class."; + return false; } } @@ -6096,16 +6100,6 @@ bool SIInstrInfo::verifyInstruction(const MachineInstr &MI, !(RI.getChannelFromSubReg(Op->getSubReg()) & 1); }; - if (Opcode == AMDGPU::DS_GWS_INIT || Opcode == AMDGPU::DS_GWS_SEMA_BR || - Opcode == AMDGPU::DS_GWS_BARRIER) { - - if (!isAlignedReg(AMDGPU::OpName::data0)) { - ErrInfo = "Subtarget requires even aligned vector registers " - "for DS_GWS instructions"; - return false; - } - } - if (isMIMG(MI)) { if (!isAlignedReg(AMDGPU::OpName::vaddr)) { ErrInfo = "Subtarget requires even aligned vector registers " diff --git a/llvm/lib/Target/AMDGPU/SIRegisterInfo.cpp b/llvm/lib/Target/AMDGPU/SIRegisterInfo.cpp index f570aa4e2f6fb..bf257c521cbd6 100644 --- a/llvm/lib/Target/AMDGPU/SIRegisterInfo.cpp +++ b/llvm/lib/Target/AMDGPU/SIRegisterInfo.cpp @@ -3935,6 +3935,21 @@ SIRegisterInfo::getDefaultVectorSuperClassForBitWidth(unsigned BitWidth) const { : getVGPRClassForBitWidth(BitWidth); } +const TargetRegisterClass * +SIRegisterInfo::getAlignedEquivalentRC(const TargetRegisterClass *RC) const { + assert(RC->isAllocatable() && + "expected an allocatable register class; non-allocatable VS_* operand " + "classes cannot be a register's class"); + unsigned Width = getRegSizeInBits(*RC); + if (isVGPRClass(RC)) + return getVGPRClassForBitWidth(Width); + if (isAGPRClass(RC)) + return getAGPRClassForBitWidth(Width); + if (isVectorSuperClass(RC)) + return getVectorSuperClassForBitWidth(Width); + return nullptr; +} + const TargetRegisterClass * SIRegisterInfo::getSGPRClassForBitWidth(unsigned BitWidth) { if (BitWidth == 16 || BitWidth == 32) diff --git a/llvm/lib/Target/AMDGPU/SIRegisterInfo.h b/llvm/lib/Target/AMDGPU/SIRegisterInfo.h index 296389ed88915..482e1d1003a3a 100644 --- a/llvm/lib/Target/AMDGPU/SIRegisterInfo.h +++ b/llvm/lib/Target/AMDGPU/SIRegisterInfo.h @@ -226,6 +226,16 @@ class SIRegisterInfo final : public AMDGPUGenRegisterInfo { const TargetRegisterClass * getDefaultVectorSuperClassForBitWidth(unsigned BitWidth) const; + /// \returns the even-aligned register class with the same register bank and + /// width as \p RC (its aligned "canonical" class), or null if \p RC has no + /// vector registers. Only meaningful on subtargets that require aligned + /// VGPRs, where the per-bitwidth getters return the aligned variant. \p RC + /// must be an allocatable register's class; the non-allocatable VS_* operand + /// classes are intentionally not handled. + LLVM_READONLY + const TargetRegisterClass * + getAlignedEquivalentRC(const TargetRegisterClass *RC) const; + LLVM_READONLY static const TargetRegisterClass *getSGPRClassForBitWidth(unsigned BitWidth); diff --git a/llvm/test/CodeGen/AMDGPU/verify-ds-gws-align.mir b/llvm/test/CodeGen/AMDGPU/verify-ds-gws-align.mir index 8a7fc2a3af057..feffe3d27e451 100644 --- a/llvm/test/CodeGen/AMDGPU/verify-ds-gws-align.mir +++ b/llvm/test/CodeGen/AMDGPU/verify-ds-gws-align.mir @@ -10,29 +10,21 @@ body: | bb.0: ; A 32-bit sub-register does not fit the 64-bit AV_64_Align2 data0 operand. %0:areg_128_align2 = IMPLICIT_DEF - ; GFX90A-ERR: *** Bad machine code: Subtarget requires even aligned vector registers for DS_GWS instructions *** - ; GFX90A-ERR: - instruction: DS_GWS_INIT killed %0.sub1:areg_128_align2 ; GFX90A-ERR: *** Bad machine code: Illegal virtual register for instruction *** ; GFX90A-ERR: - instruction: DS_GWS_INIT killed %0.sub1:areg_128_align2 ; GFX90A-ERR: AReg_128_Align2.sub1 cannot be used for AV_64_Align2 operands. DS_GWS_INIT killed %0.sub1, 0, implicit $m0, implicit $exec :: (store (s32) into custom "GWSResource") %0:areg_128_align2 = IMPLICIT_DEF - ; GFX90A-ERR: *** Bad machine code: Subtarget requires even aligned vector registers for DS_GWS instructions *** - ; GFX90A-ERR: - instruction: DS_GWS_INIT killed %0.sub3:areg_128_align2 ; GFX90A-ERR: *** Bad machine code: Illegal virtual register for instruction *** ; GFX90A-ERR: - instruction: DS_GWS_INIT killed %0.sub3:areg_128_align2 ; GFX90A-ERR: AReg_128_Align2.sub3 cannot be used for AV_64_Align2 operands. DS_GWS_INIT killed %0.sub3, 0, implicit $m0, implicit $exec :: (store (s32) into custom "GWSResource") %1:vreg_64_align2 = IMPLICIT_DEF - ; GFX90A-ERR: *** Bad machine code: Subtarget requires even aligned vector registers for DS_GWS instructions *** - ; GFX90A-ERR: - instruction: DS_GWS_SEMA_BR killed %1.sub1:vreg_64_align2 ; GFX90A-ERR: *** Bad machine code: Illegal virtual register for instruction *** ; GFX90A-ERR: - instruction: DS_GWS_SEMA_BR killed %1.sub1:vreg_64_align2 ; GFX90A-ERR: VReg_64_Align2.sub1 cannot be used for AV_64_Align2 operands. DS_GWS_SEMA_BR killed %1.sub1, 0, implicit $m0, implicit $exec :: (store (s32) into custom "GWSResource") %2:vreg_64 = IMPLICIT_DEF - ; GFX90A-ERR: *** Bad machine code: Subtarget requires even aligned vector registers for DS_GWS instructions *** - ; GFX90A-ERR: - instruction: DS_GWS_BARRIER killed %2.sub0:vreg_64 ; GFX90A-ERR: *** Bad machine code: Illegal virtual register for instruction *** ; GFX90A-ERR: - instruction: DS_GWS_BARRIER killed %2.sub0:vreg_64 ; GFX90A-ERR: VReg_64.sub0 cannot be used for AV_64_Align2 operands. @@ -40,8 +32,6 @@ body: | ; A whole 32-bit register is the wrong size for the 64-bit operand. %3:vgpr_32 = IMPLICIT_DEF - ; GFX90A-ERR: *** Bad machine code: Subtarget requires even aligned vector registers for DS_GWS instructions *** - ; GFX90A-ERR: - instruction: DS_GWS_INIT killed %3:vgpr_32 ; GFX90A-ERR: *** Bad machine code: Illegal virtual register for instruction *** ; GFX90A-ERR: - instruction: DS_GWS_INIT killed %3:vgpr_32 ; GFX90A-ERR: Expected a AV_64_Align2 register, but got a VGPR_32 register diff --git a/llvm/test/CodeGen/AMDGPU/verify-gfx90a-aligned-vgprs.mir b/llvm/test/CodeGen/AMDGPU/verify-gfx90a-aligned-vgprs.mir index f8f8197cbc005..8cec088873a94 100644 --- a/llvm/test/CodeGen/AMDGPU/verify-gfx90a-aligned-vgprs.mir +++ b/llvm/test/CodeGen/AMDGPU/verify-gfx90a-aligned-vgprs.mir @@ -193,16 +193,12 @@ body: | $vgpr0_vgpr1 = V_PK_ADD_F32 0, %13, 11, %12, 0, 0, 0, 0, 0, implicit $mode, implicit $exec ; src %14 is an AGPR - a bank mismatch, not an alignment problem - ; CHECK: *** Bad machine code: Subtarget requires even aligned vector registers *** - ; CHECK: - instruction: $vgpr0_vgpr1 = V_PK_ADD_F32 0, %13:vreg_64_align2, 11, %14.sub1_sub2:areg_96_align2 ; CHECK: *** Bad machine code: Illegal virtual register for instruction *** ; CHECK: - instruction: $vgpr0_vgpr1 = V_PK_ADD_F32 0, %13:vreg_64_align2, 11, %14.sub1_sub2:areg_96_align2 ; CHECK: AReg_96_Align2.sub1_sub2 cannot be used for VS_64_Align2 operands. $vgpr0_vgpr1 = V_PK_ADD_F32 0, %13, 11, %14.sub1_sub2, 0, 0, 0, 0, 0, implicit $mode, implicit $exec ; src %14 is an AGPR - a bank mismatch, not an alignment problem - ; CHECK: *** Bad machine code: Subtarget requires even aligned vector registers *** - ; CHECK: - instruction: $vgpr0_vgpr1 = V_PK_ADD_F32 0, %14.sub1_sub2:areg_96_align2, 11, %13:vreg_64_align2 ; CHECK: *** Bad machine code: Illegal virtual register for instruction *** ; CHECK: - instruction: $vgpr0_vgpr1 = V_PK_ADD_F32 0, %14.sub1_sub2:areg_96_align2, 11, %13:vreg_64_align2 ; CHECK: AReg_96_Align2.sub1_sub2 cannot be used for VS_64_Align2 operands. @@ -219,16 +215,12 @@ body: | $vgpr0_vgpr1 = V_PK_MUL_F32 0, %13, 11, %12, 0, 0, 0, 0, 0, implicit $mode, implicit $exec ; src %14 is an AGPR - a bank mismatch, not an alignment problem - ; CHECK: *** Bad machine code: Subtarget requires even aligned vector registers *** - ; CHECK: - instruction: $vgpr0_vgpr1 = V_PK_MUL_F32 0, %13:vreg_64_align2, 11, %14.sub1_sub2:areg_96_align2 ; CHECK: *** Bad machine code: Illegal virtual register for instruction *** ; CHECK: - instruction: $vgpr0_vgpr1 = V_PK_MUL_F32 0, %13:vreg_64_align2, 11, %14.sub1_sub2:areg_96_align2 ; CHECK: AReg_96_Align2.sub1_sub2 cannot be used for VS_64_Align2 operands. $vgpr0_vgpr1 = V_PK_MUL_F32 0, %13, 11, %14.sub1_sub2, 0, 0, 0, 0, 0, implicit $mode, implicit $exec ; src %14 is an AGPR - a bank mismatch, not an alignment problem - ; CHECK: *** Bad machine code: Subtarget requires even aligned vector registers *** - ; CHECK: - instruction: $vgpr0_vgpr1 = V_PK_MUL_F32 0, %14.sub1_sub2:areg_96_align2, 11, %13:vreg_64_align2 ; CHECK: *** Bad machine code: Illegal virtual register for instruction *** ; CHECK: - instruction: $vgpr0_vgpr1 = V_PK_MUL_F32 0, %14.sub1_sub2:areg_96_align2, 11, %13:vreg_64_align2 ; CHECK: AReg_96_Align2.sub1_sub2 cannot be used for VS_64_Align2 operands. @@ -249,8 +241,6 @@ body: | $vgpr0_vgpr1 = nofpexcept V_PK_FMA_F32 8, %13, 8, %12, 11, %14.sub0_sub1, 0, 0, 0, 0, 0, implicit $mode, implicit $exec ; src %14 is an AGPR - a bank mismatch (both %13 sources are aligned) - ; CHECK: *** Bad machine code: Subtarget requires even aligned vector registers *** - ; CHECK: - instruction: $vgpr0_vgpr1 = nofpexcept V_PK_FMA_F32 8, %13:vreg_64_align2, 8, %13:vreg_64_align2, 11, %14.sub1_sub2:areg_96_align2 ; CHECK: *** Bad machine code: Illegal virtual register for instruction *** ; CHECK: - instruction: $vgpr0_vgpr1 = nofpexcept V_PK_FMA_F32 8, %13:vreg_64_align2, 8, %13:vreg_64_align2, 11, %14.sub1_sub2:areg_96_align2 ; CHECK: AReg_96_Align2.sub1_sub2 cannot be used for VS_64_Align2 operands. diff --git a/llvm/test/MachineVerifier/AMDGPU/unsupported-subreg-index-aligned-vgpr-check.mir b/llvm/test/MachineVerifier/AMDGPU/unsupported-subreg-index-aligned-vgpr-check.mir index 5b1aaa6337261..6ddf0f62a6f1c 100644 --- a/llvm/test/MachineVerifier/AMDGPU/unsupported-subreg-index-aligned-vgpr-check.mir +++ b/llvm/test/MachineVerifier/AMDGPU/unsupported-subreg-index-aligned-vgpr-check.mir @@ -21,10 +21,6 @@ body: | ; CHECK-NEXT: - operand 1: %1.sub16_sub17_sub18_sub19:vreg_512_align2 ; CHECK-NEXT: Register class VReg_512_Align2 does not support subreg index sub16_sub17_sub18_sub19 - ; CHECK: *** Bad machine code: Subtarget requires even aligned vector registers *** - ; CHECK-NEXT: - function: uses_invalid_subregister_for_regclass - ; CHECK-NEXT: - basic block: %bb.0 - ; CHECK-NEXT: - instruction: GLOBAL_STORE_DWORDX4_SADDR %0:vgpr_32, %2.sub16_sub17_sub18_sub19:vreg_512, undef $sgpr8_sgpr9, 80, 0, implicit $exec :: (store (s128), addrspace 1) GLOBAL_STORE_DWORDX4_SADDR %0, %1.sub16_sub17_sub18_sub19, undef $sgpr8_sgpr9, 80, 0, implicit $exec :: (store (s128), addrspace 1) ; Test with unaligned class diff --git a/llvm/test/MachineVerifier/AMDGPU/unsupported-unaligned-vgpr-check-vsrc-operand.mir b/llvm/test/MachineVerifier/AMDGPU/unsupported-unaligned-vgpr-check-vsrc-operand.mir index c982b1a35d44a..3df4ce13fc15d 100644 --- a/llvm/test/MachineVerifier/AMDGPU/unsupported-unaligned-vgpr-check-vsrc-operand.mir +++ b/llvm/test/MachineVerifier/AMDGPU/unsupported-unaligned-vgpr-check-vsrc-operand.mir @@ -23,12 +23,10 @@ body: | %1:vgpr_32 = V_MOV_B32_e32 1, implicit $exec %2:vreg_64 = IMPLICIT_DEF - ; CHECK: *** Bad machine code: Subtarget requires even aligned vector registers *** ; CHECK: *** Bad machine code: Illegal virtual register for instruction *** %3:areg_128_align2 = V_MFMA_F32_4X4X1F32_e64 %0, %1, %2, 0, 0, 0, implicit $mode, implicit $exec %4:vreg_64 = IMPLICIT_DEF - ; CHECK: *** Bad machine code: Subtarget requires even aligned vector registers *** ; CHECK: *** Bad machine code: Illegal virtual register for instruction *** %5:vreg_128_align2 = V_MFMA_F32_4X4X1F32_vgprcd_e64 %0, %1, %4, 0, 0, 0, implicit $mode, implicit $exec ... >From eb16d6773ac4044bd4d901b82755ac6ba2499b96 Mon Sep 17 00:00:00 2001 From: Valery Pykhtin <[email protected]> Date: Mon, 31 Aug 2026 13:06:03 +0000 Subject: [PATCH 2/3] [AMDGPU] Match the AsmParser's operand-side check and share its helper In the upcoming PR AsmParser performs the same alignment check but differently: it tests whether a concrete physreg, which carries no regclass, is in the relaxed operand's regclass. Change the verifier to work the same way, reusing AMDGPUBaseInfo::getUnalignedEquivalentRC. Co-Authored-By: Claude <[email protected]> --- llvm/lib/Target/AMDGPU/SIInstrInfo.cpp | 13 +-- llvm/lib/Target/AMDGPU/SIRegisterInfo.cpp | 15 ---- llvm/lib/Target/AMDGPU/SIRegisterInfo.h | 10 --- .../Target/AMDGPU/Utils/AMDGPUBaseInfo.cpp | 87 +++++++++++++++++++ llvm/lib/Target/AMDGPU/Utils/AMDGPUBaseInfo.h | 5 ++ 5 files changed, 99 insertions(+), 31 deletions(-) diff --git a/llvm/lib/Target/AMDGPU/SIInstrInfo.cpp b/llvm/lib/Target/AMDGPU/SIInstrInfo.cpp index 329b9440de0d0..998edf2bec73e 100644 --- a/llvm/lib/Target/AMDGPU/SIInstrInfo.cpp +++ b/llvm/lib/Target/AMDGPU/SIInstrInfo.cpp @@ -5537,13 +5537,14 @@ bool SIInstrInfo::verifyInstruction(const MachineInstr &MI, RegRC = RI.getSubRegisterClass(RegRC, MO.getSubReg()); } // Flag an alignment-only mismatch: the register does not satisfy the - // operand's class, but its even-aligned same-bank/width equivalent - // would. A bank or size mismatch fails even when aligned, so it is left - // to the illegal-register / sub-register checks. + // operand's class, but does satisfy it with the alignment requirement + // relaxed (the operand class's unaligned equivalent). A bank or size + // mismatch fails even when relaxed, so it is left to the + // illegal-register / sub-register checks. if (RegRC && !OpRC->hasSubClassEq(RegRC)) { - const TargetRegisterClass *AlignedRegRC = - RI.getAlignedEquivalentRC(RegRC); - if (AlignedRegRC && OpRC->hasSubClassEq(AlignedRegRC)) { + int UnalignedRC = AMDGPU::getUnalignedEquivalentRC(RegClass); + if (UnalignedRC >= 0 && + RI.getRegClass(UnalignedRC)->hasSubClassEq(RegRC)) { ErrInfo = "Subtarget requires even aligned vector registers"; return false; } diff --git a/llvm/lib/Target/AMDGPU/SIRegisterInfo.cpp b/llvm/lib/Target/AMDGPU/SIRegisterInfo.cpp index bf257c521cbd6..f570aa4e2f6fb 100644 --- a/llvm/lib/Target/AMDGPU/SIRegisterInfo.cpp +++ b/llvm/lib/Target/AMDGPU/SIRegisterInfo.cpp @@ -3935,21 +3935,6 @@ SIRegisterInfo::getDefaultVectorSuperClassForBitWidth(unsigned BitWidth) const { : getVGPRClassForBitWidth(BitWidth); } -const TargetRegisterClass * -SIRegisterInfo::getAlignedEquivalentRC(const TargetRegisterClass *RC) const { - assert(RC->isAllocatable() && - "expected an allocatable register class; non-allocatable VS_* operand " - "classes cannot be a register's class"); - unsigned Width = getRegSizeInBits(*RC); - if (isVGPRClass(RC)) - return getVGPRClassForBitWidth(Width); - if (isAGPRClass(RC)) - return getAGPRClassForBitWidth(Width); - if (isVectorSuperClass(RC)) - return getVectorSuperClassForBitWidth(Width); - return nullptr; -} - const TargetRegisterClass * SIRegisterInfo::getSGPRClassForBitWidth(unsigned BitWidth) { if (BitWidth == 16 || BitWidth == 32) diff --git a/llvm/lib/Target/AMDGPU/SIRegisterInfo.h b/llvm/lib/Target/AMDGPU/SIRegisterInfo.h index 482e1d1003a3a..296389ed88915 100644 --- a/llvm/lib/Target/AMDGPU/SIRegisterInfo.h +++ b/llvm/lib/Target/AMDGPU/SIRegisterInfo.h @@ -226,16 +226,6 @@ class SIRegisterInfo final : public AMDGPUGenRegisterInfo { const TargetRegisterClass * getDefaultVectorSuperClassForBitWidth(unsigned BitWidth) const; - /// \returns the even-aligned register class with the same register bank and - /// width as \p RC (its aligned "canonical" class), or null if \p RC has no - /// vector registers. Only meaningful on subtargets that require aligned - /// VGPRs, where the per-bitwidth getters return the aligned variant. \p RC - /// must be an allocatable register's class; the non-allocatable VS_* operand - /// classes are intentionally not handled. - LLVM_READONLY - const TargetRegisterClass * - getAlignedEquivalentRC(const TargetRegisterClass *RC) const; - LLVM_READONLY static const TargetRegisterClass *getSGPRClassForBitWidth(unsigned BitWidth); diff --git a/llvm/lib/Target/AMDGPU/Utils/AMDGPUBaseInfo.cpp b/llvm/lib/Target/AMDGPU/Utils/AMDGPUBaseInfo.cpp index 0a6e98777953c..96e21d51a6da7 100644 --- a/llvm/lib/Target/AMDGPU/Utils/AMDGPUBaseInfo.cpp +++ b/llvm/lib/Target/AMDGPU/Utils/AMDGPUBaseInfo.cpp @@ -2949,6 +2949,93 @@ unsigned getRegBitWidth(const MCRegisterClass &RC) { return getRegBitWidth(RC.getID()); } +int getUnalignedEquivalentRC(unsigned RCID) { + switch (RCID) { + case AMDGPU::VReg_64_Align2RegClassID: + return AMDGPU::VReg_64RegClassID; + case AMDGPU::VReg_96_Align2RegClassID: + return AMDGPU::VReg_96RegClassID; + case AMDGPU::VReg_128_Align2RegClassID: + return AMDGPU::VReg_128RegClassID; + case AMDGPU::VReg_160_Align2RegClassID: + return AMDGPU::VReg_160RegClassID; + case AMDGPU::VReg_192_Align2RegClassID: + return AMDGPU::VReg_192RegClassID; + case AMDGPU::VReg_224_Align2RegClassID: + return AMDGPU::VReg_224RegClassID; + case AMDGPU::VReg_256_Align2RegClassID: + return AMDGPU::VReg_256RegClassID; + case AMDGPU::VReg_288_Align2RegClassID: + return AMDGPU::VReg_288RegClassID; + case AMDGPU::VReg_320_Align2RegClassID: + return AMDGPU::VReg_320RegClassID; + case AMDGPU::VReg_352_Align2RegClassID: + return AMDGPU::VReg_352RegClassID; + case AMDGPU::VReg_384_Align2RegClassID: + return AMDGPU::VReg_384RegClassID; + case AMDGPU::VReg_512_Align2RegClassID: + return AMDGPU::VReg_512RegClassID; + case AMDGPU::VReg_1024_Align2RegClassID: + return AMDGPU::VReg_1024RegClassID; + case AMDGPU::AReg_64_Align2RegClassID: + return AMDGPU::AReg_64RegClassID; + case AMDGPU::AReg_96_Align2RegClassID: + return AMDGPU::AReg_96RegClassID; + case AMDGPU::AReg_128_Align2RegClassID: + return AMDGPU::AReg_128RegClassID; + case AMDGPU::AReg_160_Align2RegClassID: + return AMDGPU::AReg_160RegClassID; + case AMDGPU::AReg_192_Align2RegClassID: + return AMDGPU::AReg_192RegClassID; + case AMDGPU::AReg_224_Align2RegClassID: + return AMDGPU::AReg_224RegClassID; + case AMDGPU::AReg_256_Align2RegClassID: + return AMDGPU::AReg_256RegClassID; + case AMDGPU::AReg_288_Align2RegClassID: + return AMDGPU::AReg_288RegClassID; + case AMDGPU::AReg_320_Align2RegClassID: + return AMDGPU::AReg_320RegClassID; + case AMDGPU::AReg_352_Align2RegClassID: + return AMDGPU::AReg_352RegClassID; + case AMDGPU::AReg_384_Align2RegClassID: + return AMDGPU::AReg_384RegClassID; + case AMDGPU::AReg_512_Align2RegClassID: + return AMDGPU::AReg_512RegClassID; + case AMDGPU::AReg_1024_Align2RegClassID: + return AMDGPU::AReg_1024RegClassID; + case AMDGPU::AV_64_Align2RegClassID: + return AMDGPU::AV_64RegClassID; + case AMDGPU::AV_96_Align2RegClassID: + return AMDGPU::AV_96RegClassID; + case AMDGPU::AV_128_Align2RegClassID: + return AMDGPU::AV_128RegClassID; + case AMDGPU::AV_160_Align2RegClassID: + return AMDGPU::AV_160RegClassID; + case AMDGPU::AV_192_Align2RegClassID: + return AMDGPU::AV_192RegClassID; + case AMDGPU::AV_224_Align2RegClassID: + return AMDGPU::AV_224RegClassID; + case AMDGPU::AV_256_Align2RegClassID: + return AMDGPU::AV_256RegClassID; + case AMDGPU::AV_288_Align2RegClassID: + return AMDGPU::AV_288RegClassID; + case AMDGPU::AV_320_Align2RegClassID: + return AMDGPU::AV_320RegClassID; + case AMDGPU::AV_352_Align2RegClassID: + return AMDGPU::AV_352RegClassID; + case AMDGPU::AV_384_Align2RegClassID: + return AMDGPU::AV_384RegClassID; + case AMDGPU::AV_512_Align2RegClassID: + return AMDGPU::AV_512RegClassID; + case AMDGPU::AV_1024_Align2RegClassID: + return AMDGPU::AV_1024RegClassID; + case AMDGPU::VS_64_Align2RegClassID: + return AMDGPU::VS_64RegClassID; + default: + return -1; + } +} + bool isInlinableLiteral64(int64_t Literal, bool HasInv2Pi) { if (isInlinableIntLiteral(Literal)) return true; diff --git a/llvm/lib/Target/AMDGPU/Utils/AMDGPUBaseInfo.h b/llvm/lib/Target/AMDGPU/Utils/AMDGPUBaseInfo.h index 44661a99da3e1..29eeb8854f524 100644 --- a/llvm/lib/Target/AMDGPU/Utils/AMDGPUBaseInfo.h +++ b/llvm/lib/Target/AMDGPU/Utils/AMDGPUBaseInfo.h @@ -1561,6 +1561,11 @@ unsigned getRegBitWidth(unsigned RCID); /// Get the size in bits of a register from the register class \p RC. unsigned getRegBitWidth(const MCRegisterClass &RC); +/// Return the register class equivalent to the even-aligned VGPR/AGPR/AV tuple +/// register class \p RCID with the alignment requirement relaxed, or -1 if +/// \p RCID has no such equivalent. +int getUnalignedEquivalentRC(unsigned RCID); + LLVM_READNONE inline unsigned getOperandSize(const MCOperandInfo &OpInfo) { switch (OpInfo.OperandType) { >From 90acd6cf5007a22e9e8639e0bc26d7b51e0111c8 Mon Sep 17 00:00:00 2001 From: Valery Pykhtin <[email protected]> Date: Wed, 9 Sep 2026 12:45:28 +0000 Subject: [PATCH 3/3] [AMDGPU] Drop the redundant operand register-class verifier check The generic MachineVerifier already rejects a register operand whose class does not fit the instruction's operand class, for both virtual (hasSuperClassEq / getMatchingSuperRegClass) and physical (contains) registers, using the HwMode-resolved operand class. On subtargets that require aligned VGPRs that class is the even-aligned one, so the generic check catches a misaligned tuple on its own. Remove SIInstrInfo::verifyInstruction's own alignment and physical register-class checks and the now-unused getUnalignedEquivalentRC helper. Misaligned tuples now report the generic "Illegal virtual/physical register for instruction" instead of "Subtarget requires even aligned vector registers"; a bad physical class reports the same instead of "Operand has incorrect register class". --- llvm/lib/Target/AMDGPU/SIInstrInfo.cpp | 46 ---------- .../Target/AMDGPU/Utils/AMDGPUBaseInfo.cpp | 87 ------------------- llvm/lib/Target/AMDGPU/Utils/AMDGPUBaseInfo.h | 5 -- .../CodeGen/AMDGPU/verify-ds-gws-align.mir | 14 --- .../AMDGPU/verify-gfx90a-aligned-vgprs.mir | 50 ----------- llvm/test/CodeGen/AMDGPU/verify-image.mir | 4 - .../CodeGen/AMDGPU/verify-vimage-vsample.mir | 4 - ...rted-unaligned-vgpr-check-vsrc-operand.mir | 3 - 8 files changed, 213 deletions(-) diff --git a/llvm/lib/Target/AMDGPU/SIInstrInfo.cpp b/llvm/lib/Target/AMDGPU/SIInstrInfo.cpp index 998edf2bec73e..3ec1d39456ac1 100644 --- a/llvm/lib/Target/AMDGPU/SIInstrInfo.cpp +++ b/llvm/lib/Target/AMDGPU/SIInstrInfo.cpp @@ -5425,7 +5425,6 @@ bool SIInstrInfo::verifyInstruction(const MachineInstr &MI, } const MCOperandInfo &OpInfo = Desc.operands()[i]; - int16_t RegClass = getOpRegClassID(OpInfo); switch (OpInfo.OperandType) { case MCOI::OPERAND_REGISTER: @@ -5511,51 +5510,6 @@ bool SIInstrInfo::verifyInstruction(const MachineInstr &MI, continue; break; } - - // Operands without a fixed register class (RegClass == -1), such as inline - // asm operands, are not verified here. - if (RegClass == -1) - continue; - - if (!MO.isReg()) - continue; - Register Reg = MO.getReg(); - if (!Reg) - continue; - - const TargetRegisterClass *OpRC = RI.getRegClass(RegClass); - - if (ST.needsAlignedVGPRs()) { - const TargetRegisterClass *RegRC = RI.getRegClassForReg(MRI, Reg); - if (RI.hasVectorRegisters(RegRC)) { - if (MO.getSubReg()) { - // Narrow to the sub-register's class. getSubRegisterClass already - // accounts for the sub-register index's alignment within the tuple - // (an odd-aligned slice yields an unaligned class, caught below); a - // null result means an invalid sub-register index and is left to the - // sub-register check. - RegRC = RI.getSubRegisterClass(RegRC, MO.getSubReg()); - } - // Flag an alignment-only mismatch: the register does not satisfy the - // operand's class, but does satisfy it with the alignment requirement - // relaxed (the operand class's unaligned equivalent). A bank or size - // mismatch fails even when relaxed, so it is left to the - // illegal-register / sub-register checks. - if (RegRC && !OpRC->hasSubClassEq(RegRC)) { - int UnalignedRC = AMDGPU::getUnalignedEquivalentRC(RegClass); - if (UnalignedRC >= 0 && - RI.getRegClass(UnalignedRC)->hasSubClassEq(RegRC)) { - ErrInfo = "Subtarget requires even aligned vector registers"; - return false; - } - } - } - } - - if (Reg.isPhysical() && !OpRC->contains(Reg)) { - ErrInfo = "Operand has incorrect register class."; - return false; - } } // Verify SDWA diff --git a/llvm/lib/Target/AMDGPU/Utils/AMDGPUBaseInfo.cpp b/llvm/lib/Target/AMDGPU/Utils/AMDGPUBaseInfo.cpp index 96e21d51a6da7..0a6e98777953c 100644 --- a/llvm/lib/Target/AMDGPU/Utils/AMDGPUBaseInfo.cpp +++ b/llvm/lib/Target/AMDGPU/Utils/AMDGPUBaseInfo.cpp @@ -2949,93 +2949,6 @@ unsigned getRegBitWidth(const MCRegisterClass &RC) { return getRegBitWidth(RC.getID()); } -int getUnalignedEquivalentRC(unsigned RCID) { - switch (RCID) { - case AMDGPU::VReg_64_Align2RegClassID: - return AMDGPU::VReg_64RegClassID; - case AMDGPU::VReg_96_Align2RegClassID: - return AMDGPU::VReg_96RegClassID; - case AMDGPU::VReg_128_Align2RegClassID: - return AMDGPU::VReg_128RegClassID; - case AMDGPU::VReg_160_Align2RegClassID: - return AMDGPU::VReg_160RegClassID; - case AMDGPU::VReg_192_Align2RegClassID: - return AMDGPU::VReg_192RegClassID; - case AMDGPU::VReg_224_Align2RegClassID: - return AMDGPU::VReg_224RegClassID; - case AMDGPU::VReg_256_Align2RegClassID: - return AMDGPU::VReg_256RegClassID; - case AMDGPU::VReg_288_Align2RegClassID: - return AMDGPU::VReg_288RegClassID; - case AMDGPU::VReg_320_Align2RegClassID: - return AMDGPU::VReg_320RegClassID; - case AMDGPU::VReg_352_Align2RegClassID: - return AMDGPU::VReg_352RegClassID; - case AMDGPU::VReg_384_Align2RegClassID: - return AMDGPU::VReg_384RegClassID; - case AMDGPU::VReg_512_Align2RegClassID: - return AMDGPU::VReg_512RegClassID; - case AMDGPU::VReg_1024_Align2RegClassID: - return AMDGPU::VReg_1024RegClassID; - case AMDGPU::AReg_64_Align2RegClassID: - return AMDGPU::AReg_64RegClassID; - case AMDGPU::AReg_96_Align2RegClassID: - return AMDGPU::AReg_96RegClassID; - case AMDGPU::AReg_128_Align2RegClassID: - return AMDGPU::AReg_128RegClassID; - case AMDGPU::AReg_160_Align2RegClassID: - return AMDGPU::AReg_160RegClassID; - case AMDGPU::AReg_192_Align2RegClassID: - return AMDGPU::AReg_192RegClassID; - case AMDGPU::AReg_224_Align2RegClassID: - return AMDGPU::AReg_224RegClassID; - case AMDGPU::AReg_256_Align2RegClassID: - return AMDGPU::AReg_256RegClassID; - case AMDGPU::AReg_288_Align2RegClassID: - return AMDGPU::AReg_288RegClassID; - case AMDGPU::AReg_320_Align2RegClassID: - return AMDGPU::AReg_320RegClassID; - case AMDGPU::AReg_352_Align2RegClassID: - return AMDGPU::AReg_352RegClassID; - case AMDGPU::AReg_384_Align2RegClassID: - return AMDGPU::AReg_384RegClassID; - case AMDGPU::AReg_512_Align2RegClassID: - return AMDGPU::AReg_512RegClassID; - case AMDGPU::AReg_1024_Align2RegClassID: - return AMDGPU::AReg_1024RegClassID; - case AMDGPU::AV_64_Align2RegClassID: - return AMDGPU::AV_64RegClassID; - case AMDGPU::AV_96_Align2RegClassID: - return AMDGPU::AV_96RegClassID; - case AMDGPU::AV_128_Align2RegClassID: - return AMDGPU::AV_128RegClassID; - case AMDGPU::AV_160_Align2RegClassID: - return AMDGPU::AV_160RegClassID; - case AMDGPU::AV_192_Align2RegClassID: - return AMDGPU::AV_192RegClassID; - case AMDGPU::AV_224_Align2RegClassID: - return AMDGPU::AV_224RegClassID; - case AMDGPU::AV_256_Align2RegClassID: - return AMDGPU::AV_256RegClassID; - case AMDGPU::AV_288_Align2RegClassID: - return AMDGPU::AV_288RegClassID; - case AMDGPU::AV_320_Align2RegClassID: - return AMDGPU::AV_320RegClassID; - case AMDGPU::AV_352_Align2RegClassID: - return AMDGPU::AV_352RegClassID; - case AMDGPU::AV_384_Align2RegClassID: - return AMDGPU::AV_384RegClassID; - case AMDGPU::AV_512_Align2RegClassID: - return AMDGPU::AV_512RegClassID; - case AMDGPU::AV_1024_Align2RegClassID: - return AMDGPU::AV_1024RegClassID; - case AMDGPU::VS_64_Align2RegClassID: - return AMDGPU::VS_64RegClassID; - default: - return -1; - } -} - bool isInlinableLiteral64(int64_t Literal, bool HasInv2Pi) { if (isInlinableIntLiteral(Literal)) return true; diff --git a/llvm/lib/Target/AMDGPU/Utils/AMDGPUBaseInfo.h b/llvm/lib/Target/AMDGPU/Utils/AMDGPUBaseInfo.h index 29eeb8854f524..44661a99da3e1 100644 --- a/llvm/lib/Target/AMDGPU/Utils/AMDGPUBaseInfo.h +++ b/llvm/lib/Target/AMDGPU/Utils/AMDGPUBaseInfo.h @@ -1561,11 +1561,6 @@ unsigned getRegBitWidth(unsigned RCID); /// Get the size in bits of a register from the register class \p RC. unsigned getRegBitWidth(const MCRegisterClass &RC); -/// Return the register class equivalent to the even-aligned VGPR/AGPR/AV tuple -/// register class \p RCID with the alignment requirement relaxed, or -1 if -/// \p RCID has no such equivalent. -int getUnalignedEquivalentRC(unsigned RCID); - LLVM_READNONE inline unsigned getOperandSize(const MCOperandInfo &OpInfo) { switch (OpInfo.OperandType) { diff --git a/llvm/test/CodeGen/AMDGPU/verify-ds-gws-align.mir b/llvm/test/CodeGen/AMDGPU/verify-ds-gws-align.mir index feffe3d27e451..d38822a4a8550 100644 --- a/llvm/test/CodeGen/AMDGPU/verify-ds-gws-align.mir +++ b/llvm/test/CodeGen/AMDGPU/verify-ds-gws-align.mir @@ -39,51 +39,37 @@ body: | ; Physical single 32-bit registers are the wrong class for the operand. $vgpr1 = IMPLICIT_DEF - ; GFX90A-ERR: *** Bad machine code: Operand has incorrect register class. *** - ; GFX90A-ERR: - instruction: DS_GWS_INIT $vgpr1, ; GFX90A-ERR: *** Bad machine code: Illegal physical register for instruction *** ; GFX90A-ERR: $vgpr1 is not a AV_64_Align2 register. DS_GWS_INIT $vgpr1, 0, implicit $m0, implicit $exec :: (store (s32) into custom "GWSResource") $agpr1 = IMPLICIT_DEF - ; GFX90A-ERR: *** Bad machine code: Operand has incorrect register class. *** - ; GFX90A-ERR: - instruction: DS_GWS_INIT $agpr1, ; GFX90A-ERR: *** Bad machine code: Illegal physical register for instruction *** ; GFX90A-ERR: $agpr1 is not a AV_64_Align2 register. DS_GWS_INIT $agpr1, 0, implicit $m0, implicit $exec :: (store (s32) into custom "GWSResource") ; Odd-aligned physical 64-bit tuples are a genuine alignment error. $vgpr3_vgpr4 = IMPLICIT_DEF - ; GFX90A-ERR: *** Bad machine code: Subtarget requires even aligned vector registers *** - ; GFX90A-ERR: - instruction: DS_GWS_INIT $vgpr1_vgpr2 ; GFX90A-ERR: *** Bad machine code: Illegal physical register for instruction *** ; GFX90A-ERR: $vgpr1_vgpr2 is not a AV_64_Align2 register. DS_GWS_INIT $vgpr1_vgpr2, 0, implicit $m0, implicit $exec :: (store (s32) into custom "GWSResource") $agpr3_agpr4 = IMPLICIT_DEF - ; GFX90A-ERR: *** Bad machine code: Subtarget requires even aligned vector registers *** - ; GFX90A-ERR: - instruction: DS_GWS_INIT $agpr3_agpr4 ; GFX90A-ERR: *** Bad machine code: Illegal physical register for instruction *** ; GFX90A-ERR: $agpr3_agpr4 is not a AV_64_Align2 register. DS_GWS_INIT $agpr3_agpr4, 0, implicit $m0, implicit $exec :: (store (s32) into custom "GWSResource") ; Unaligned virtual 64-bit tuples are a genuine alignment error. %4:vreg_64 = IMPLICIT_DEF - ; GFX90A-ERR: *** Bad machine code: Subtarget requires even aligned vector registers *** - ; GFX90A-ERR: - instruction: DS_GWS_INIT %4:vreg_64 ; GFX90A-ERR: *** Bad machine code: Illegal virtual register for instruction *** ; GFX90A-ERR: Expected a AV_64_Align2 register, but got a VReg_64 register DS_GWS_INIT %4, 0, implicit $m0, implicit $exec :: (store (s32) into custom "GWSResource") %5:areg_64 = IMPLICIT_DEF - ; GFX90A-ERR: *** Bad machine code: Subtarget requires even aligned vector registers *** - ; GFX90A-ERR: - instruction: DS_GWS_INIT %5:areg_64 ; GFX90A-ERR: *** Bad machine code: Illegal virtual register for instruction *** ; GFX90A-ERR: Expected a AV_64_Align2 register, but got a AReg_64 register DS_GWS_INIT %5, 0, implicit $m0, implicit $exec :: (store (s32) into custom "GWSResource") %6:av_64 = IMPLICIT_DEF - ; GFX90A-ERR: *** Bad machine code: Subtarget requires even aligned vector registers *** - ; GFX90A-ERR: - instruction: DS_GWS_INIT %6:av_64 ; GFX90A-ERR: *** Bad machine code: Illegal virtual register for instruction *** ; GFX90A-ERR: Expected a AV_64_Align2 register, but got a AV_64 register DS_GWS_INIT %6, 0, implicit $m0, implicit $exec :: (store (s32) into custom "GWSResource") diff --git a/llvm/test/CodeGen/AMDGPU/verify-gfx90a-aligned-vgprs.mir b/llvm/test/CodeGen/AMDGPU/verify-gfx90a-aligned-vgprs.mir index 8cec088873a94..fa0433369b585 100644 --- a/llvm/test/CodeGen/AMDGPU/verify-gfx90a-aligned-vgprs.mir +++ b/llvm/test/CodeGen/AMDGPU/verify-gfx90a-aligned-vgprs.mir @@ -63,91 +63,59 @@ body: | ; Check virtual register uses: an unaligned tuple is a misalignment, and its ; class is also wrong for the aligned operand. - ; CHECK: *** Bad machine code: Subtarget requires even aligned vector registers *** - ; CHECK: - instruction: GLOBAL_STORE_DWORDX2 %0:vreg_64_align2, %1:vreg_64 ; CHECK: *** Bad machine code: Illegal virtual register for instruction *** ; CHECK: Expected a AV_64_Align2 register, but got a VReg_64 register GLOBAL_STORE_DWORDX2 %0, %1, 0, 0, implicit $exec - ; CHECK: *** Bad machine code: Subtarget requires even aligned vector registers *** - ; CHECK: - instruction: GLOBAL_STORE_DWORDX3 %0:vreg_64_align2, %2:vreg_96 ; CHECK: *** Bad machine code: Illegal virtual register for instruction *** ; CHECK: Expected a AV_96_Align2 register, but got a VReg_96 register GLOBAL_STORE_DWORDX3 %0, %2, 0, 0, implicit $exec - ; CHECK: *** Bad machine code: Subtarget requires even aligned vector registers *** - ; CHECK: - instruction: GLOBAL_STORE_DWORDX4 %0:vreg_64_align2, %3:vreg_128 ; CHECK: *** Bad machine code: Illegal virtual register for instruction *** ; CHECK: Expected a AV_128_Align2 register, but got a VReg_128 register GLOBAL_STORE_DWORDX4 %0, %3, 0, 0, implicit $exec ; Check virtual registers with subregisters. - ; CHECK: *** Bad machine code: Subtarget requires even aligned vector registers *** - ; CHECK: - instruction: GLOBAL_STORE_DWORDX2 %0:vreg_64_align2, %3.sub0_sub1:vreg_128 ; CHECK: *** Bad machine code: Illegal virtual register for instruction *** ; CHECK: VReg_128.sub0_sub1 cannot be used for AV_64_Align2 operands. GLOBAL_STORE_DWORDX2 %0, %3.sub0_sub1, 0, 0, implicit $exec - ; CHECK: *** Bad machine code: Subtarget requires even aligned vector registers *** - ; CHECK: - instruction: GLOBAL_STORE_DWORDX2 %0:vreg_64_align2, %3.sub2_sub3:vreg_128 ; CHECK: *** Bad machine code: Illegal virtual register for instruction *** ; CHECK: VReg_128.sub2_sub3 cannot be used for AV_64_Align2 operands. GLOBAL_STORE_DWORDX2 %0, %3.sub2_sub3, 0, 0, implicit $exec - ; CHECK: *** Bad machine code: Subtarget requires even aligned vector registers *** - ; CHECK: - instruction: GLOBAL_STORE_DWORDX2 %0:vreg_64_align2, %3.sub1_sub2:vreg_128 ; CHECK: *** Bad machine code: Illegal virtual register for instruction *** ; CHECK: VReg_128.sub1_sub2 cannot be used for AV_64_Align2 operands. GLOBAL_STORE_DWORDX2 %0, %3.sub1_sub2, 0, 0, implicit $exec - ; CHECK: *** Bad machine code: Subtarget requires even aligned vector registers *** - ; CHECK: - instruction: GLOBAL_STORE_DWORDX2 %0:vreg_64_align2, %5.sub1_sub2:vreg_128_align2 ; CHECK: *** Bad machine code: Illegal virtual register for instruction *** ; CHECK: VReg_128_Align2.sub1_sub2 cannot be used for AV_64_Align2 operands. GLOBAL_STORE_DWORDX2 %0, %5.sub1_sub2, 0, 0, implicit $exec ; Check physical register uses. - ; CHECK: *** Bad machine code: Subtarget requires even aligned vector registers *** - ; CHECK: - instruction: GLOBAL_STORE_DWORDX2 $vgpr0_vgpr1, $vgpr3_vgpr4 ; CHECK: *** Bad machine code: Illegal physical register for instruction *** ; CHECK: $vgpr3_vgpr4 is not a AV_64_Align2 register. GLOBAL_STORE_DWORDX2 $vgpr0_vgpr1, $vgpr3_vgpr4, 0, 0, implicit $exec - ; CHECK: *** Bad machine code: Subtarget requires even aligned vector registers *** - ; CHECK: - instruction: GLOBAL_STORE_DWORDX3 $vgpr0_vgpr1, $vgpr3_vgpr4_vgpr5 ; CHECK: *** Bad machine code: Illegal physical register for instruction *** ; CHECK: $vgpr3_vgpr4_vgpr5 is not a AV_96_Align2 register. GLOBAL_STORE_DWORDX3 $vgpr0_vgpr1, $vgpr3_vgpr4_vgpr5, 0, 0, implicit $exec - ; CHECK: *** Bad machine code: Subtarget requires even aligned vector registers *** - ; CHECK: - instruction: GLOBAL_STORE_DWORDX4 $vgpr0_vgpr1, $vgpr3_vgpr4_vgpr5_vgpr6 ; CHECK: *** Bad machine code: Illegal physical register for instruction *** ; CHECK: $vgpr3_vgpr4_vgpr5_vgpr6 is not a AV_128_Align2 register. GLOBAL_STORE_DWORDX4 $vgpr0_vgpr1, $vgpr3_vgpr4_vgpr5_vgpr6, 0, 0, implicit $exec ; Check virtual register defs. - ; CHECK: *** Bad machine code: Subtarget requires even aligned vector registers *** - ; CHECK: - instruction: %6:vreg_64 = GLOBAL_LOAD_DWORDX2 %0:vreg_64_align2 ; CHECK: *** Bad machine code: Illegal virtual register for instruction *** ; CHECK: Expected a AV_64_Align2 register, but got a VReg_64 register %6:vreg_64 = GLOBAL_LOAD_DWORDX2 %0, 0, 0, implicit $exec - ; CHECK: *** Bad machine code: Subtarget requires even aligned vector registers *** - ; CHECK: - instruction: %7:vreg_96 = GLOBAL_LOAD_DWORDX3 %0:vreg_64_align2 ; CHECK: *** Bad machine code: Illegal virtual register for instruction *** ; CHECK: Expected a AV_96_Align2 register, but got a VReg_96 register %7:vreg_96 = GLOBAL_LOAD_DWORDX3 %0, 0, 0, implicit $exec - ; CHECK: *** Bad machine code: Subtarget requires even aligned vector registers *** - ; CHECK: - instruction: %8:vreg_128 = GLOBAL_LOAD_DWORDX4 %0:vreg_64_align2 ; CHECK: *** Bad machine code: Illegal virtual register for instruction *** ; CHECK: Expected a AV_128_Align2 register, but got a VReg_128 register %8:vreg_128 = GLOBAL_LOAD_DWORDX4 %0, 0, 0, implicit $exec ; Check physical register defs. - ; CHECK: *** Bad machine code: Subtarget requires even aligned vector registers *** - ; CHECK: - instruction: $vgpr1_vgpr2 = GLOBAL_LOAD_DWORDX2 %0:vreg_64_align2 ; CHECK: *** Bad machine code: Illegal physical register for instruction *** ; CHECK: $vgpr1_vgpr2 is not a AV_64_Align2 register. $vgpr1_vgpr2 = GLOBAL_LOAD_DWORDX2 %0, 0, 0, implicit $exec - ; CHECK: *** Bad machine code: Subtarget requires even aligned vector registers *** - ; CHECK: - instruction: $vgpr1_vgpr2_vgpr3 = GLOBAL_LOAD_DWORDX3 %0:vreg_64_align2 ; CHECK: *** Bad machine code: Illegal physical register for instruction *** ; CHECK: $vgpr1_vgpr2_vgpr3 is not a AV_96_Align2 register. $vgpr1_vgpr2_vgpr3 = GLOBAL_LOAD_DWORDX3 %0, 0, 0, implicit $exec - ; CHECK: *** Bad machine code: Subtarget requires even aligned vector registers *** - ; CHECK: - instruction: $vgpr1_vgpr2_vgpr3_vgpr4 = GLOBAL_LOAD_DWORDX4 %0:vreg_64_align2 ; CHECK: *** Bad machine code: Illegal physical register for instruction *** ; CHECK: $vgpr1_vgpr2_vgpr3_vgpr4 is not a AV_128_Align2 register. $vgpr1_vgpr2_vgpr3_vgpr4 = GLOBAL_LOAD_DWORDX4 %0, 0, 0, implicit $exec @@ -156,13 +124,9 @@ body: | %9:vgpr_32 = IMPLICIT_DEF %10:areg_64 = IMPLICIT_DEF %11:areg_128_align2 = IMPLICIT_DEF - ; CHECK: *** Bad machine code: Subtarget requires even aligned vector registers *** - ; CHECK: - instruction: DS_WRITE_B64_gfx9 %9:vgpr_32, %10:areg_64 ; CHECK: *** Bad machine code: Illegal virtual register for instruction *** ; CHECK: Expected a AV_64_Align2 register, but got a AReg_64 register DS_WRITE_B64_gfx9 %9, %10, 0, 0, implicit $exec - ; CHECK: *** Bad machine code: Subtarget requires even aligned vector registers *** - ; CHECK: - instruction: DS_WRITE_B64_gfx9 %9:vgpr_32, %11.sub1_sub2:areg_128_align2 ; CHECK: *** Bad machine code: Illegal virtual register for instruction *** ; CHECK: AReg_128_Align2.sub1_sub2 cannot be used for AV_64_Align2 operands. DS_WRITE_B64_gfx9 %9, %11.sub1_sub2, 0, 0, implicit $exec @@ -178,18 +142,12 @@ body: | %14:areg_96_align2 = IMPLICIT_DEF ; odd def $vgpr3_vgpr4 (v3) is a misaligned VGPR pair - ; CHECK: *** Bad machine code: Subtarget requires even aligned vector registers *** - ; CHECK: - instruction: $vgpr3_vgpr4 = V_PK_MOV_B32 $vgpr3_vgpr4 = V_PK_MOV_B32 8, 0, 8, 0, 0, 0, 0, 0, 0, implicit $exec ; src %12 is an unaligned VGPR pair - ; CHECK: *** Bad machine code: Subtarget requires even aligned vector registers *** - ; CHECK: - instruction: $vgpr0_vgpr1 = V_PK_ADD_F32 0, %12:vreg_64, 11, %13:vreg_64_align2 $vgpr0_vgpr1 = V_PK_ADD_F32 0, %12, 11, %13, 0, 0, 0, 0, implicit $mode, implicit $exec ; src %12 is an unaligned VGPR pair - ; CHECK: *** Bad machine code: Subtarget requires even aligned vector registers *** - ; CHECK: - instruction: $vgpr0_vgpr1 = V_PK_ADD_F32 0, %13:vreg_64_align2, 11, %12:vreg_64 $vgpr0_vgpr1 = V_PK_ADD_F32 0, %13, 11, %12, 0, 0, 0, 0, 0, implicit $mode, implicit $exec ; src %14 is an AGPR - a bank mismatch, not an alignment problem @@ -205,13 +163,9 @@ body: | $vgpr0_vgpr1 = V_PK_ADD_F32 0, %14.sub1_sub2, 11, %13, 0, 0, 0, 0, implicit $mode, implicit $exec ; src %12 is an unaligned VGPR pair - ; CHECK: *** Bad machine code: Subtarget requires even aligned vector registers *** - ; CHECK: - instruction: $vgpr0_vgpr1 = V_PK_MUL_F32 0, %12:vreg_64, 11, %13:vreg_64_align2 $vgpr0_vgpr1 = V_PK_MUL_F32 0, %12, 11, %13, 0, 0, 0, 0, implicit $mode, implicit $exec ; src %12 is an unaligned VGPR pair - ; CHECK: *** Bad machine code: Subtarget requires even aligned vector registers *** - ; CHECK: - instruction: $vgpr0_vgpr1 = V_PK_MUL_F32 0, %13:vreg_64_align2, 11, %12:vreg_64 $vgpr0_vgpr1 = V_PK_MUL_F32 0, %13, 11, %12, 0, 0, 0, 0, 0, implicit $mode, implicit $exec ; src %14 is an AGPR - a bank mismatch, not an alignment problem @@ -227,15 +181,11 @@ body: | $vgpr0_vgpr1 = V_PK_MUL_F32 0, %14.sub1_sub2, 11, %13, 0, 0, 0, 0, implicit $mode, implicit $exec ; src %12 is an unaligned VGPR pair (even aligned); src %14 is an AGPR (bank mismatch) - ; CHECK: *** Bad machine code: Subtarget requires even aligned vector registers *** - ; CHECK: - instruction: $vgpr0_vgpr1 = nofpexcept V_PK_FMA_F32 8, %12:vreg_64, 8, %13:vreg_64_align2, 11, %14.sub0_sub1:areg_96_align2 ; CHECK: *** Bad machine code: Illegal virtual register for instruction *** ; CHECK: AReg_96_Align2.sub0_sub1 cannot be used for VS_64_Align2 operands. $vgpr0_vgpr1 = nofpexcept V_PK_FMA_F32 8, %12, 8, %13, 11, %14.sub0_sub1, 0, 0, 0, 0, 0, implicit $mode, implicit $exec ; src %12 is an unaligned VGPR pair (even aligned); src %14 is an AGPR (bank mismatch) - ; CHECK: *** Bad machine code: Subtarget requires even aligned vector registers *** - ; CHECK: - instruction: $vgpr0_vgpr1 = nofpexcept V_PK_FMA_F32 8, %13:vreg_64_align2, 8, %12:vreg_64, 11, %14.sub0_sub1:areg_96_align2 ; CHECK: *** Bad machine code: Illegal virtual register for instruction *** ; CHECK: AReg_96_Align2.sub0_sub1 cannot be used for VS_64_Align2 operands. $vgpr0_vgpr1 = nofpexcept V_PK_FMA_F32 8, %13, 8, %12, 11, %14.sub0_sub1, 0, 0, 0, 0, 0, implicit $mode, implicit $exec diff --git a/llvm/test/CodeGen/AMDGPU/verify-image.mir b/llvm/test/CodeGen/AMDGPU/verify-image.mir index 0d1cec5170cb5..2e9fc6f1a858f 100644 --- a/llvm/test/CodeGen/AMDGPU/verify-image.mir +++ b/llvm/test/CodeGen/AMDGPU/verify-image.mir @@ -7,8 +7,6 @@ body: | bb.0: liveins: $sgpr0_sgpr1_sgpr2_sgpr3_sgpr4_sgpr5_sgpr6_sgpr7, $sgpr8_sgpr9_sgpr10_sgpr11, $vgpr0, $vgpr1, $vgpr2, $vgpr3, $vgpr4, $vgpr5, $vgpr6, $vgpr7, $vgpr8, $vgpr9 - ; GFX11-ERR: *** Bad machine code: Operand has incorrect register class. *** - ; GFX11-ERR: - instruction: renamable $vgpr10 = IMAGE_SAMPLE_D_V1_V9_nsa_gfx11 renamable $vgpr1, renamable $vgpr0, renamable $vgpr2, renamable $vgpr3, renamable $vgpr4_vgpr5_vgpr6_vgpr7, renamable $sgpr0_sgpr1_sgpr2_sgpr3_sgpr4_sgpr5_sgpr6_sgpr7, renamable $sgpr8_sgpr9_sgpr10_sgpr11, 1, 2, 0, 0, 0, 0, 0, 0, 0, implicit $exec :: (dereferenceable load (s128), addrspace 7) ; GFX11-ERR: *** Bad machine code: Illegal physical register for instruction *** ; GFX11-ERR: - instruction: renamable $vgpr10 = IMAGE_SAMPLE_D_V1_V9_nsa_gfx11 renamable $vgpr1, renamable $vgpr0, renamable $vgpr2, renamable $vgpr3, renamable $vgpr4_vgpr5_vgpr6_vgpr7, renamable $sgpr0_sgpr1_sgpr2_sgpr3_sgpr4_sgpr5_sgpr6_sgpr7, renamable $sgpr8_sgpr9_sgpr10_sgpr11, 1, 2, 0, 0, 0, 0, 0, 0, 0, implicit $exec :: (dereferenceable load (s128), addrspace 7) ; GFX11-ERR: - operand 5: renamable $vgpr4_vgpr5_vgpr6_vgpr7 @@ -16,8 +14,6 @@ body: | renamable $vgpr10 = IMAGE_SAMPLE_D_V1_V9_nsa_gfx11 renamable $vgpr1, renamable $vgpr0, renamable $vgpr2, renamable $vgpr3, renamable $vgpr4_vgpr5_vgpr6_vgpr7, renamable $sgpr0_sgpr1_sgpr2_sgpr3_sgpr4_sgpr5_sgpr6_sgpr7, renamable $sgpr8_sgpr9_sgpr10_sgpr11, 1, 2, 0, 0, 0, 0, 0, 0, 0, implicit $exec :: (dereferenceable load (s128), addrspace 7) - ; GFX11-ERR: *** Bad machine code: Operand has incorrect register class. *** - ; GFX11-ERR: - instruction: renamable $vgpr11 = IMAGE_SAMPLE_D_V1_V9_nsa_gfx11 renamable $vgpr1, renamable $vgpr0, renamable $vgpr2, renamable $vgpr3, renamable $vgpr4_vgpr5_vgpr6_vgpr7_vgpr8_vgpr9, renamable $sgpr0_sgpr1_sgpr2_sgpr3_sgpr4_sgpr5_sgpr6_sgpr7, renamable $sgpr8_sgpr9_sgpr10_sgpr11, 1, 2, 0, 0, 0, 0, 0, 0, 0, implicit $exec :: (dereferenceable load (s128), addrspace 7) ; GFX11-ERR: *** Bad machine code: Illegal physical register for instruction *** ; GFX11-ERR: - instruction: renamable $vgpr11 = IMAGE_SAMPLE_D_V1_V9_nsa_gfx11 renamable $vgpr1, renamable $vgpr0, renamable $vgpr2, renamable $vgpr3, renamable $vgpr4_vgpr5_vgpr6_vgpr7_vgpr8_vgpr9, renamable $sgpr0_sgpr1_sgpr2_sgpr3_sgpr4_sgpr5_sgpr6_sgpr7, renamable $sgpr8_sgpr9_sgpr10_sgpr11, 1, 2, 0, 0, 0, 0, 0, 0, 0, implicit $exec :: (dereferenceable load (s128), addrspace 7) ; GFX11-ERR: - operand 5: renamable $vgpr4_vgpr5_vgpr6_vgpr7_vgpr8_vgpr9 diff --git a/llvm/test/CodeGen/AMDGPU/verify-vimage-vsample.mir b/llvm/test/CodeGen/AMDGPU/verify-vimage-vsample.mir index 6c3860e48a850..c1c3cd60a6829 100644 --- a/llvm/test/CodeGen/AMDGPU/verify-vimage-vsample.mir +++ b/llvm/test/CodeGen/AMDGPU/verify-vimage-vsample.mir @@ -34,16 +34,12 @@ body: | ; GFX12-ERR: - instruction: renamable $vgpr26 = IMAGE_SAMPLE_V1_V1_gfx12 killed renamable $vgpr0, killed renamable $sgpr0_sgpr1_sgpr2_sgpr3_sgpr4_sgpr5_sgpr6_sgpr7, killed renamable $sgpr8_sgpr9_sgpr10_sgpr11, 15, 0, 0, 0, 0, 0, 0, 0, -1, implicit $exec :: (dereferenceable load (s128), addrspace 7) renamable $vgpr26 = IMAGE_SAMPLE_V1_V1_gfx12 killed renamable $vgpr0, killed renamable $sgpr0_sgpr1_sgpr2_sgpr3_sgpr4_sgpr5_sgpr6_sgpr7, killed renamable $sgpr8_sgpr9_sgpr10_sgpr11, 15, 0, 0, 0, 0, 0, 0, 0, -1, implicit $exec :: (dereferenceable load (s128), addrspace 7) - ; GFX12-ERR: *** Bad machine code: Operand has incorrect register class. *** - ; GFX12-ERR: - instruction: renamable $vgpr27 = IMAGE_SAMPLE_D_V1_V9_gfx12 renamable $vgpr0, renamable $vgpr1, renamable $vgpr2, renamable $vgpr3_vgpr4_vgpr5_vgpr6_vgpr7, renamable $sgpr0_sgpr1_sgpr2_sgpr3_sgpr4_sgpr5_sgpr6_sgpr7, renamable $sgpr8_sgpr9_sgpr10_sgpr11, 1, 2, 0, 0, 0, 0, 0, 0, 0, implicit $exec :: (dereferenceable load (s128), addrspace 7) ; GFX12-ERR: *** Bad machine code: Illegal physical register for instruction *** ; GFX12-ERR: - instruction: renamable $vgpr27 = IMAGE_SAMPLE_D_V1_V9_gfx12 renamable $vgpr0, renamable $vgpr1, renamable $vgpr2, renamable $vgpr3_vgpr4_vgpr5_vgpr6_vgpr7, renamable $sgpr0_sgpr1_sgpr2_sgpr3_sgpr4_sgpr5_sgpr6_sgpr7, renamable $sgpr8_sgpr9_sgpr10_sgpr11, 1, 2, 0, 0, 0, 0, 0, 0, 0, implicit $exec :: (dereferenceable load (s128), addrspace 7) ; GFX12-ERR: - operand 4: renamable $vgpr3_vgpr4_vgpr5_vgpr6_vgpr7 ; GFX12-ERR: $vgpr3_vgpr4_vgpr5_vgpr6_vgpr7 is not a VReg_192 register. renamable $vgpr27 = IMAGE_SAMPLE_D_V1_V9_gfx12 renamable $vgpr0, renamable $vgpr1, renamable $vgpr2, renamable $vgpr3_vgpr4_vgpr5_vgpr6_vgpr7, renamable $sgpr0_sgpr1_sgpr2_sgpr3_sgpr4_sgpr5_sgpr6_sgpr7, renamable $sgpr8_sgpr9_sgpr10_sgpr11, 1, 2, 0, 0, 0, 0, 0, 0, 0, implicit $exec :: (dereferenceable load (s128), addrspace 7) - ; GFX12-ERR: *** Bad machine code: Operand has incorrect register class. *** - ; GFX12-ERR: - instruction: renamable $vgpr28 = IMAGE_SAMPLE_D_V1_V9_gfx12 renamable $vgpr0, renamable $vgpr1, renamable $vgpr2, renamable $vgpr3_vgpr4_vgpr5_vgpr6_vgpr7_vgpr8_vgpr9, renamable $sgpr0_sgpr1_sgpr2_sgpr3_sgpr4_sgpr5_sgpr6_sgpr7, renamable $sgpr8_sgpr9_sgpr10_sgpr11, 1, 2, 0, 0, 0, 0, 0, 0, 0, implicit $exec :: (dereferenceable load (s128), addrspace 7) ; GFX12-ERR: *** Bad machine code: Illegal physical register for instruction *** ; GFX12-ERR: - instruction: renamable $vgpr28 = IMAGE_SAMPLE_D_V1_V9_gfx12 renamable $vgpr0, renamable $vgpr1, renamable $vgpr2, renamable $vgpr3_vgpr4_vgpr5_vgpr6_vgpr7_vgpr8_vgpr9, renamable $sgpr0_sgpr1_sgpr2_sgpr3_sgpr4_sgpr5_sgpr6_sgpr7, renamable $sgpr8_sgpr9_sgpr10_sgpr11, 1, 2, 0, 0, 0, 0, 0, 0, 0, implicit $exec :: (dereferenceable load (s128), addrspace 7) ; GFX12-ERR: - operand 4: renamable $vgpr3_vgpr4_vgpr5_vgpr6_vgpr7_vgpr8_vgpr9 diff --git a/llvm/test/MachineVerifier/AMDGPU/unsupported-unaligned-vgpr-check-vsrc-operand.mir b/llvm/test/MachineVerifier/AMDGPU/unsupported-unaligned-vgpr-check-vsrc-operand.mir index 3df4ce13fc15d..52d79ef9f1084 100644 --- a/llvm/test/MachineVerifier/AMDGPU/unsupported-unaligned-vgpr-check-vsrc-operand.mir +++ b/llvm/test/MachineVerifier/AMDGPU/unsupported-unaligned-vgpr-check-vsrc-operand.mir @@ -9,12 +9,9 @@ body: | bb.0: liveins: $vgpr1_vgpr2 - ; CHECK: *** Bad machine code: Subtarget requires even aligned vector registers *** - ; CHECK: - instruction: $vcc = V_CMP_NE_U64_e64 0, $vgpr1_vgpr2, implicit $exec ; CHECK: *** Bad machine code: Illegal physical register for instruction *** $vcc = V_CMP_NE_U64_e64 0, $vgpr1_vgpr2, implicit $exec - ; CHECK: *** Bad machine code: Subtarget requires even aligned vector registers *** ; CHECK: *** Bad machine code: Illegal physical register for instruction *** ; CHECK: - instruction: V_CMP_NE_U64_e32 0, $vgpr1_vgpr2, implicit-def $vcc, implicit $exec V_CMP_NE_U64_e32 0, $vgpr1_vgpr2, implicit-def $vcc, implicit $exec _______________________________________________ llvm-branch-commits mailing list [email protected] https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits
