https://github.com/Pierre-vh updated https://github.com/llvm/llvm-project/pull/209748
>From 6285d9ef27f1f386b9c365e63d2600a4d431fa26 Mon Sep 17 00:00:00 2001 From: pvanhout <[email protected]> Date: Tue, 16 Jun 2026 10:21:06 +0200 Subject: [PATCH 1/4] [AMDGPU] Add synthetic apertures and use them for barriers Define what a synthetic aperture is, and adjust the barrier AS to use this new system. This makes the barrier AS even safer to use as now we can use all 32 bits of it without ever risking hitting a valid address of any kind (LDS or outside LDS). --- llvm/docs/AMDGPUUsage.rst | 49 +++++- llvm/include/llvm/Support/AMDGPUAddrSpace.h | 5 - .../lib/Target/AMDGPU/AMDGPULegalizerInfo.cpp | 44 +++-- llvm/lib/Target/AMDGPU/AMDGPULegalizerInfo.h | 4 + llvm/lib/Target/AMDGPU/AMDGPUMemoryUtils.cpp | 9 ++ llvm/lib/Target/AMDGPU/AMDGPUMemoryUtils.h | 4 + llvm/lib/Target/AMDGPU/SIDefines.h | 12 ++ llvm/lib/Target/AMDGPU/SIISelLowering.cpp | 35 ++-- llvm/lib/Target/AMDGPU/SIISelLowering.h | 2 + .../CodeGen/AMDGPU/addrspacecast-barrier.ll | 153 ++++++++---------- 10 files changed, 183 insertions(+), 134 deletions(-) diff --git a/llvm/docs/AMDGPUUsage.rst b/llvm/docs/AMDGPUUsage.rst index 8e9199bc17bcc..1827508e43fd2 100644 --- a/llvm/docs/AMDGPUUsage.rst +++ b/llvm/docs/AMDGPUUsage.rst @@ -1194,6 +1194,9 @@ supported for the ``amdgcn`` target. A global address space address has the same value when used as a flat address so no conversion is needed. + See also :ref:`synthetic apertures<amdgpu-synthetic-apertures>` which exist + in the generic address space. + **Global and Constant** The global and constant address spaces both use global virtual addresses, which are the same virtual address space used by the CPU. However, some @@ -1374,7 +1377,9 @@ supported for the ``amdgcn`` target. **Barrier** This address space represents barrier IDs (introduced in GFX12) as addresses. - It does not map directly to any addressable memory, thus pointers into this address space: + It does not map directly to any addressable memory and is implemented using + :ref:`synthetic apertures<amdgpu-synthetic-apertures>`, thus pointers into + this address space: * Never alias with any other pointers outside this address space. * Cannot be dereferenced. @@ -1388,10 +1393,6 @@ supported for the ``amdgcn`` target. The ``NULL`` pointer (as a constant) can be consumed by some intrinsics and corresponds to the NULL named barrier. - These pointers do not have a corresponding hardware aperture but safe round-tripping - through the generic address space is still possible. Attempting to dereference a - generic pointer derived from a barrier pointer is undefined behavior. - **Streamout Registers** Dedicated registers used by the GS NGG Streamout Instructions. The register file is modelled as a memory in a distinct address space because it is indexed @@ -1399,6 +1400,44 @@ supported for the ``amdgcn`` target. accesses affect LGKMcnt. This is an internal address space used only by the compiler. Do not use this address space for IR pointers. +.. _amdgpu-synthetic-apertures: + +Synthetic Apertures +~~~~~~~~~~~~~~~~~~~ + +*Synthetic apertures* are defined that enable safe roundtrips of pointers +from special address spaces through the generic address space. Attempting to +dereference generic pointers obtained in this way (using e.g. `load` or +`store`) has undefined behavior. The following synthetic apertures are defined: + +.. table:: AMDGPU Synthetic Apertures + :name: amdgpu-synthetic-apertures-table + :widths: 40 20 40 + + ============ ====== =============================================================== + Name Number Corresponding :ref:`Address Space<amdgpu-address-spaces-table>` + ============ ====== =============================================================== + BARRIER 1 Barrier + ============ ====== =============================================================== + +Note that the address size of an address spaces implemented via synthetic apertures +can only be 32 bits wide or less. The full width of the source pointer is usable and +preserved when converting it from/to the generic address space. + +Converting a pointer to generic (64 bits) using synthetic apertures is done as follows: + + * The value of the source pointer (32 bits) becomes the lower 32 bits of the generic pointer. + * The upper 32 bits are a bitwise ``OR`` of: + + * The upper 32 bits of the LDS segment aperture. + * The synthetic aperture number. + +The conversion back to the original address space can simply be done by discarding the +upper 32 bits of the generic pointer. + +As the LDS aperture is defined by its 16 most significant bits, we can theoretically +support up to ``2 << 16`` synthetic apertures safely. + .. _amdgpu-memory-scopes: Memory Scopes diff --git a/llvm/include/llvm/Support/AMDGPUAddrSpace.h b/llvm/include/llvm/Support/AMDGPUAddrSpace.h index d72ba0a1415c0..1824c1bcc2e89 100644 --- a/llvm/include/llvm/Support/AMDGPUAddrSpace.h +++ b/llvm/include/llvm/Support/AMDGPUAddrSpace.h @@ -91,11 +91,6 @@ enum : unsigned { // Some places use this if the address space can't be determined. UNKNOWN_ADDRESS_SPACE = ~0u, }; - -/// The BARRIER AS does not have an aperture in HW, so when converting -/// BARRIER addresses from/to generic, we represent them as LDS addresses -/// offset by a large amount so they can never alias with real LDS memory. -static constexpr unsigned BarrierAddrLDSOffset = 0x802000u; } // end namespace AMDGPUAS namespace AMDGPU { diff --git a/llvm/lib/Target/AMDGPU/AMDGPULegalizerInfo.cpp b/llvm/lib/Target/AMDGPU/AMDGPULegalizerInfo.cpp index 609e5b2bbeb09..af66ab91362d8 100644 --- a/llvm/lib/Target/AMDGPU/AMDGPULegalizerInfo.cpp +++ b/llvm/lib/Target/AMDGPU/AMDGPULegalizerInfo.cpp @@ -2425,10 +2425,27 @@ bool AMDGPULegalizerInfo::legalizeCustom( llvm_unreachable("expected switch to return"); } -Register AMDGPULegalizerInfo::getSegmentAperture( - unsigned AS, - MachineRegisterInfo &MRI, - MachineIRBuilder &B) const { +Register AMDGPULegalizerInfo::getSegmentAperture(unsigned AS, + MachineRegisterInfo &MRI, + MachineIRBuilder &B) const { + unsigned BaseAS = AS; + unsigned SANum = AMDGPU::tryGetSyntheticApertureNumber(AS); + if (SANum != AMDGPU::SyntheticAperture::None) + BaseAS = AMDGPUAS::LOCAL_ADDRESS; + + Register Aperture = getBaseSegmentAperture(BaseAS, MRI, B); + + if (SANum != AMDGPU::SyntheticAperture::None) { + const LLT S32 = LLT::scalar(32); + auto Tag = B.buildConstant(S32, SANum); + return B.buildOr(S32, Aperture, Tag).getReg(0); + } + + return Aperture; +} + +Register AMDGPULegalizerInfo::getBaseSegmentAperture( + unsigned AS, MachineRegisterInfo &MRI, MachineIRBuilder &B) const { MachineFunction &MF = B.getMF(); const GCNSubtarget &ST = MF.getSubtarget<GCNSubtarget>(); const LLT I32 = LLT::integer(32); @@ -2583,17 +2600,6 @@ bool AMDGPULegalizerInfo::legalizeAddrSpaceCast( return B.buildIntToPtr(Dst, Sub).getReg(0); } - if (DestAS == AMDGPUAS::BARRIER) { - // flat -> barrier: extract the low 32 bits, then sub the barrier AS - // offset. - Register LoBits = B.buildExtract(S32, Src, 0).getReg(0); - Register Sub = - B.buildSub(S32, LoBits, - B.buildConstant(S32, AMDGPUAS::BarrierAddrLDSOffset)) - .getReg(0); - return B.buildIntToPtr(Dst, Sub).getReg(0); - } - return B.buildExtract(Dst, Src, 0).getReg(0); }; @@ -2664,14 +2670,6 @@ bool AMDGPULegalizerInfo::legalizeAddrSpaceCast( if (!ApertureReg.isValid()) return false; - if (SrcAS == AMDGPUAS::BARRIER) { - // barrier -> flat: add the barrier AS offset - SrcAsInt = - B.buildAdd(S32, SrcAsInt, - B.buildConstant(S32, AMDGPUAS::BarrierAddrLDSOffset)) - .getReg(0); - } - // TODO: Should we allow mismatched types but matching sizes in merges to // avoid the ptrtoint? return B.buildMergeLikeInstr(Dst, {SrcAsInt, ApertureReg}).getReg(0); diff --git a/llvm/lib/Target/AMDGPU/AMDGPULegalizerInfo.h b/llvm/lib/Target/AMDGPU/AMDGPULegalizerInfo.h index 89819fe990f5a..30fd16930e6ae 100644 --- a/llvm/lib/Target/AMDGPU/AMDGPULegalizerInfo.h +++ b/llvm/lib/Target/AMDGPU/AMDGPULegalizerInfo.h @@ -264,6 +264,10 @@ class AMDGPULegalizerInfo final : public LegalizerInfo { bool legalizeIntrinsic(LegalizerHelper &Helper, MachineInstr &MI) const override; + +private: + Register getBaseSegmentAperture(unsigned AS, MachineRegisterInfo &MRI, + MachineIRBuilder &B) const; }; } // End llvm namespace. #endif diff --git a/llvm/lib/Target/AMDGPU/AMDGPUMemoryUtils.cpp b/llvm/lib/Target/AMDGPU/AMDGPUMemoryUtils.cpp index 30e2a380820b3..5b1a5f6d4e3db 100644 --- a/llvm/lib/Target/AMDGPU/AMDGPUMemoryUtils.cpp +++ b/llvm/lib/Target/AMDGPU/AMDGPUMemoryUtils.cpp @@ -32,6 +32,15 @@ Align getAlign(const DataLayout &DL, const GlobalVariable *GV) { GV->getValueType()); } +unsigned tryGetSyntheticApertureNumber(unsigned AS) { + switch (AS) { + case AMDGPUAS::BARRIER: + return SyntheticAperture::BARRIER; + default: + return SyntheticAperture::None; + } +} + void copyMetadataForWidenedLoad(LoadInst &Dest, const LoadInst &Source) { SmallVector<std::pair<unsigned, MDNode *>, 8> MD; Source.getAllMetadata(MD); diff --git a/llvm/lib/Target/AMDGPU/AMDGPUMemoryUtils.h b/llvm/lib/Target/AMDGPU/AMDGPUMemoryUtils.h index 93fee16594e69..791ca0f7b779c 100644 --- a/llvm/lib/Target/AMDGPU/AMDGPUMemoryUtils.h +++ b/llvm/lib/Target/AMDGPU/AMDGPUMemoryUtils.h @@ -37,6 +37,10 @@ using VariableFunctionMap = DenseMap<GlobalVariable *, DenseSet<Function *>>; Align getAlign(const DataLayout &DL, const GlobalVariable *GV); +// Get the synthetic aperture number for the given address space, or None (0) +// if the address space does not have one. +unsigned tryGetSyntheticApertureNumber(unsigned AS); + // Copy metadata onto a load widened to read a superset of Source's bytes. Only // value-independent metadata is copied; metadata describing the loaded value // (!range, !noundef, !nofpclass, !tbaa, ...) is dropped. diff --git a/llvm/lib/Target/AMDGPU/SIDefines.h b/llvm/lib/Target/AMDGPU/SIDefines.h index ceffe78ba2676..223b916da20cf 100644 --- a/llvm/lib/Target/AMDGPU/SIDefines.h +++ b/llvm/lib/Target/AMDGPU/SIDefines.h @@ -1356,6 +1356,18 @@ enum Type { }; } // namespace Barrier + +namespace SyntheticAperture { +/// Synthetic aperture numbers. +/// +/// NOTE: This is also documented in AMDGPUUsage. +enum SyntheticAperture { + None = 0, + + BARRIER = 1, +}; +} // namespace SyntheticAperture + } // namespace AMDGPU // clang-format off diff --git a/llvm/lib/Target/AMDGPU/SIISelLowering.cpp b/llvm/lib/Target/AMDGPU/SIISelLowering.cpp index b3b8ee8959647..0359aa67985c6 100644 --- a/llvm/lib/Target/AMDGPU/SIISelLowering.cpp +++ b/llvm/lib/Target/AMDGPU/SIISelLowering.cpp @@ -9325,6 +9325,23 @@ SDValue SITargetLowering::LowerINLINEASM(SDValue Op, SelectionDAG &DAG) const { SDValue SITargetLowering::getSegmentAperture(unsigned AS, const SDLoc &DL, SelectionDAG &DAG) const { + unsigned BaseAS = AS; + unsigned SANum = AMDGPU::tryGetSyntheticApertureNumber(AS); + if (SANum != AMDGPU::SyntheticAperture::None) + BaseAS = AMDGPUAS::LOCAL_ADDRESS; + + SDValue Aperture = getBaseSegmentAperture(BaseAS, DL, DAG); + + if (SANum != AMDGPU::SyntheticAperture::None) { + SDValue Tag = DAG.getConstant(SANum, DL, MVT::i32); + return DAG.getNode(ISD::OR, DL, MVT::i32, Aperture, Tag); + } + + return Aperture; +} + +SDValue SITargetLowering::getBaseSegmentAperture(unsigned AS, const SDLoc &DL, + SelectionDAG &DAG) const { const bool IsLDS = (AS == AMDGPUAS::LOCAL_ADDRESS || AS == AMDGPUAS::BARRIER); if (Subtarget->hasApertureRegs()) { @@ -9441,11 +9458,6 @@ SDValue SITargetLowering::lowerADDRSPACECAST(SDValue Op, DAG.getRegister(AMDGPU::SRC_FLAT_SCRATCH_BASE_LO, MVT::i32)), 0); Ptr = DAG.getNode(ISD::SUB, SL, MVT::i32, Ptr, FlatScratchBaseLo); - } else if (DestAS == AMDGPUAS::BARRIER) { - // flat -> barrier: sub the barrier AS offset. - Ptr = DAG.getNode( - ISD::SUB, SL, MVT::i32, Ptr, - DAG.getConstant(AMDGPUAS::BarrierAddrLDSOffset, SL, MVT::i32)); } if (IsNonNull || isKnownNonNull(Op, DAG, TM, SrcAS)) @@ -9496,18 +9508,7 @@ SDValue SITargetLowering::lowerADDRSPACECAST(SDValue Op, } else { SDValue Aperture = getSegmentAperture(SrcAS, SL, DAG); - if (SrcAS == AMDGPUAS::BARRIER) { - // barrier -> flat: add the barrier AS offset. - SDValue SrcOffset = DAG.getNode( - ISD::ADD, SL, MVT::i32, Src, - DAG.getConstant(AMDGPUAS::BarrierAddrLDSOffset, SL, MVT::i32)); - CvtPtr = DAG.getNode(ISD::BUILD_VECTOR, SL, MVT::v2i32, SrcOffset, - Aperture); - } else { - CvtPtr = - DAG.getNode(ISD::BUILD_VECTOR, SL, MVT::v2i32, Src, Aperture); - } - + CvtPtr = DAG.getNode(ISD::BUILD_VECTOR, SL, MVT::v2i32, Src, Aperture); CvtPtr = DAG.getNode(ISD::BITCAST, SL, MVT::i64, CvtPtr); } diff --git a/llvm/lib/Target/AMDGPU/SIISelLowering.h b/llvm/lib/Target/AMDGPU/SIISelLowering.h index 86653fe6920c5..4289f15839ebc 100644 --- a/llvm/lib/Target/AMDGPU/SIISelLowering.h +++ b/llvm/lib/Target/AMDGPU/SIISelLowering.h @@ -175,6 +175,8 @@ class SITargetLowering final : public AMDGPUTargetLowering { SDValue lowerXMULO(SDValue Op, SelectionDAG &DAG) const; SDValue lowerXMUL_LOHI(SDValue Op, SelectionDAG &DAG) const; + SDValue getBaseSegmentAperture(unsigned AS, const SDLoc &DL, + SelectionDAG &DAG) const; SDValue getSegmentAperture(unsigned AS, const SDLoc &DL, SelectionDAG &DAG) const; diff --git a/llvm/test/CodeGen/AMDGPU/addrspacecast-barrier.ll b/llvm/test/CodeGen/AMDGPU/addrspacecast-barrier.ll index 0318d3afde1b8..2e65d365b63ed 100644 --- a/llvm/test/CodeGen/AMDGPU/addrspacecast-barrier.ll +++ b/llvm/test/CodeGen/AMDGPU/addrspacecast-barrier.ll @@ -19,11 +19,11 @@ define amdgpu_kernel void @barrier_to_generic(ptr addrspace(15) %bar, ptr %out) ; GFX942-SDAG-NEXT: s_mov_b64 s[0:1], src_shared_base ; GFX942-SDAG-NEXT: s_load_dword s0, s[4:5], 0x0 ; GFX942-SDAG-NEXT: s_load_dwordx2 s[2:3], s[4:5], 0x8 +; GFX942-SDAG-NEXT: s_or_b32 s1, s1, 1 ; GFX942-SDAG-NEXT: s_waitcnt lgkmcnt(0) -; GFX942-SDAG-NEXT: s_add_i32 s4, s0, 0x802000 ; GFX942-SDAG-NEXT: s_cmp_lg_u32 s0, 0 -; GFX942-SDAG-NEXT: s_cselect_b32 s0, s4, 0 ; GFX942-SDAG-NEXT: s_cselect_b32 s1, s1, 0 +; GFX942-SDAG-NEXT: s_cselect_b32 s0, s0, 0 ; GFX942-SDAG-NEXT: v_mov_b32_e32 v2, s0 ; GFX942-SDAG-NEXT: v_mov_b32_e32 v3, s1 ; GFX942-SDAG-NEXT: v_mov_b64_e32 v[0:1], s[2:3] @@ -32,12 +32,12 @@ define amdgpu_kernel void @barrier_to_generic(ptr addrspace(15) %bar, ptr %out) ; ; GFX942-GISEL-LABEL: barrier_to_generic: ; GFX942-GISEL: ; %bb.0: -; GFX942-GISEL-NEXT: s_load_dword s6, s[4:5], 0x0 -; GFX942-GISEL-NEXT: s_load_dwordx2 s[2:3], s[4:5], 0x8 ; GFX942-GISEL-NEXT: s_mov_b64 s[0:1], src_shared_base +; GFX942-GISEL-NEXT: s_load_dword s0, s[4:5], 0x0 +; GFX942-GISEL-NEXT: s_load_dwordx2 s[2:3], s[4:5], 0x8 +; GFX942-GISEL-NEXT: s_or_b32 s1, s1, 1 ; GFX942-GISEL-NEXT: s_waitcnt lgkmcnt(0) -; GFX942-GISEL-NEXT: s_add_u32 s0, s6, 0x802000 -; GFX942-GISEL-NEXT: s_cmp_lg_u32 s6, 0 +; GFX942-GISEL-NEXT: s_cmp_lg_u32 s0, 0 ; GFX942-GISEL-NEXT: s_cselect_b64 s[0:1], s[0:1], 0 ; GFX942-GISEL-NEXT: v_mov_b64_e32 v[0:1], s[0:1] ; GFX942-GISEL-NEXT: v_mov_b64_e32 v[2:3], s[2:3] @@ -54,11 +54,11 @@ define amdgpu_kernel void @barrier_to_generic(ptr addrspace(15) %bar, ptr %out) ; GFX1030-SDAG-NEXT: s_clause 0x1 ; GFX1030-SDAG-NEXT: s_load_dword s0, s[8:9], 0x0 ; GFX1030-SDAG-NEXT: s_load_dwordx2 s[2:3], s[8:9], 0x8 +; GFX1030-SDAG-NEXT: s_or_b32 s1, s1, 1 ; GFX1030-SDAG-NEXT: s_waitcnt lgkmcnt(0) -; GFX1030-SDAG-NEXT: s_add_i32 s4, s0, 0x802000 ; GFX1030-SDAG-NEXT: s_cmp_lg_u32 s0, 0 ; GFX1030-SDAG-NEXT: v_mov_b32_e32 v2, s2 -; GFX1030-SDAG-NEXT: s_cselect_b32 s0, s4, 0 +; GFX1030-SDAG-NEXT: s_cselect_b32 s0, s0, 0 ; GFX1030-SDAG-NEXT: s_cselect_b32 s1, s1, 0 ; GFX1030-SDAG-NEXT: v_mov_b32_e32 v0, s0 ; GFX1030-SDAG-NEXT: v_mov_b32_e32 v1, s1 @@ -72,13 +72,13 @@ define amdgpu_kernel void @barrier_to_generic(ptr addrspace(15) %bar, ptr %out) ; GFX1030-GISEL-NEXT: s_addc_u32 s13, s13, 0 ; GFX1030-GISEL-NEXT: s_setreg_b32 hwreg(HW_REG_FLAT_SCR_LO), s12 ; GFX1030-GISEL-NEXT: s_setreg_b32 hwreg(HW_REG_FLAT_SCR_HI), s13 +; GFX1030-GISEL-NEXT: s_mov_b64 s[0:1], src_shared_base ; GFX1030-GISEL-NEXT: s_clause 0x1 -; GFX1030-GISEL-NEXT: s_load_dword s4, s[8:9], 0x0 +; GFX1030-GISEL-NEXT: s_load_dword s0, s[8:9], 0x0 ; GFX1030-GISEL-NEXT: s_load_dwordx2 s[2:3], s[8:9], 0x8 -; GFX1030-GISEL-NEXT: s_mov_b64 s[0:1], src_shared_base +; GFX1030-GISEL-NEXT: s_or_b32 s1, s1, 1 ; GFX1030-GISEL-NEXT: s_waitcnt lgkmcnt(0) -; GFX1030-GISEL-NEXT: s_add_u32 s0, s4, 0x802000 -; GFX1030-GISEL-NEXT: s_cmp_lg_u32 s4, 0 +; GFX1030-GISEL-NEXT: s_cmp_lg_u32 s0, 0 ; GFX1030-GISEL-NEXT: v_mov_b32_e32 v2, s2 ; GFX1030-GISEL-NEXT: s_cselect_b64 s[0:1], s[0:1], 0 ; GFX1030-GISEL-NEXT: v_mov_b32_e32 v3, s3 @@ -93,11 +93,11 @@ define amdgpu_kernel void @barrier_to_generic(ptr addrspace(15) %bar, ptr %out) ; GFX1200-SDAG-NEXT: s_clause 0x1 ; GFX1200-SDAG-NEXT: s_load_b32 s0, s[4:5], 0x0 ; GFX1200-SDAG-NEXT: s_load_b64 s[2:3], s[4:5], 0x8 +; GFX1200-SDAG-NEXT: s_or_b32 s1, s1, 1 ; GFX1200-SDAG-NEXT: s_wait_kmcnt 0x0 -; GFX1200-SDAG-NEXT: s_add_co_i32 s4, s0, 0x802000 ; GFX1200-SDAG-NEXT: s_cmp_lg_u32 s0, 0 ; GFX1200-SDAG-NEXT: v_dual_mov_b32 v2, s2 :: v_dual_mov_b32 v3, s3 -; GFX1200-SDAG-NEXT: s_cselect_b32 s0, s4, 0 +; GFX1200-SDAG-NEXT: s_cselect_b32 s0, s0, 0 ; GFX1200-SDAG-NEXT: s_cselect_b32 s1, s1, 0 ; GFX1200-SDAG-NEXT: s_delay_alu instid0(SALU_CYCLE_1) ; GFX1200-SDAG-NEXT: v_dual_mov_b32 v0, s0 :: v_dual_mov_b32 v1, s1 @@ -106,13 +106,13 @@ define amdgpu_kernel void @barrier_to_generic(ptr addrspace(15) %bar, ptr %out) ; ; GFX1200-GISEL-LABEL: barrier_to_generic: ; GFX1200-GISEL: ; %bb.0: +; GFX1200-GISEL-NEXT: s_mov_b64 s[0:1], src_shared_base ; GFX1200-GISEL-NEXT: s_clause 0x1 -; GFX1200-GISEL-NEXT: s_load_b32 s6, s[4:5], 0x0 +; GFX1200-GISEL-NEXT: s_load_b32 s0, s[4:5], 0x0 ; GFX1200-GISEL-NEXT: s_load_b64 s[2:3], s[4:5], 0x8 -; GFX1200-GISEL-NEXT: s_mov_b64 s[0:1], src_shared_base +; GFX1200-GISEL-NEXT: s_or_b32 s1, s1, 1 ; GFX1200-GISEL-NEXT: s_wait_kmcnt 0x0 -; GFX1200-GISEL-NEXT: s_add_co_u32 s0, s6, 0x802000 -; GFX1200-GISEL-NEXT: s_cmp_lg_u32 s6, 0 +; GFX1200-GISEL-NEXT: s_cmp_lg_u32 s0, 0 ; GFX1200-GISEL-NEXT: v_dual_mov_b32 v2, s2 :: v_dual_mov_b32 v3, s3 ; GFX1200-GISEL-NEXT: s_cselect_b64 s[0:1], s[0:1], 0 ; GFX1200-GISEL-NEXT: s_delay_alu instid0(SALU_CYCLE_1) @@ -129,10 +129,10 @@ define amdgpu_kernel void @barrier_to_generic(ptr addrspace(15) %bar, ptr %out) ; GFX1250-SDAG-NEXT: s_clause 0x1 ; GFX1250-SDAG-NEXT: s_load_b32 s0, s[4:5], 0x0 nv ; GFX1250-SDAG-NEXT: s_load_b64 s[2:3], s[4:5], 0x8 nv +; GFX1250-SDAG-NEXT: s_or_b32 s1, s1, 1 ; GFX1250-SDAG-NEXT: s_wait_kmcnt 0x0 -; GFX1250-SDAG-NEXT: s_add_co_i32 s4, s0, 0x802000 ; GFX1250-SDAG-NEXT: s_cmp_lg_u32 s0, 0 -; GFX1250-SDAG-NEXT: s_cselect_b32 s0, s4, 0 +; GFX1250-SDAG-NEXT: s_cselect_b32 s0, s0, 0 ; GFX1250-SDAG-NEXT: s_cselect_b32 s1, s1, 0 ; GFX1250-SDAG-NEXT: v_dual_mov_b32 v2, 0 :: v_dual_mov_b32 v0, s0 ; GFX1250-SDAG-NEXT: v_mov_b32_e32 v1, s1 @@ -144,14 +144,14 @@ define amdgpu_kernel void @barrier_to_generic(ptr addrspace(15) %bar, ptr %out) ; GFX1250-GISEL-NEXT: global_wb ; GFX1250-GISEL-NEXT: v_nop ; GFX1250-GISEL-NEXT: s_setreg_imm32_b32 hwreg(HW_REG_WAVE_MODE, 25, 1), 1 ; msbs: dst=0 src0=0 src1=0 src2=0 +; GFX1250-GISEL-NEXT: s_mov_b64 s[0:1], src_shared_base ; GFX1250-GISEL-NEXT: s_clause 0x1 -; GFX1250-GISEL-NEXT: s_load_b32 s6, s[4:5], 0x0 nv +; GFX1250-GISEL-NEXT: s_load_b32 s0, s[4:5], 0x0 nv ; GFX1250-GISEL-NEXT: s_load_b64 s[2:3], s[4:5], 0x8 nv -; GFX1250-GISEL-NEXT: s_mov_b64 s[0:1], src_shared_base +; GFX1250-GISEL-NEXT: s_or_b32 s1, s1, 1 ; GFX1250-GISEL-NEXT: v_mov_b32_e32 v2, 0 ; GFX1250-GISEL-NEXT: s_wait_kmcnt 0x0 -; GFX1250-GISEL-NEXT: s_add_co_u32 s0, s6, 0x802000 -; GFX1250-GISEL-NEXT: s_cmp_lg_u32 s6, 0 +; GFX1250-GISEL-NEXT: s_cmp_lg_u32 s0, 0 ; GFX1250-GISEL-NEXT: s_cselect_b64 s[0:1], s[0:1], 0 ; GFX1250-GISEL-NEXT: s_delay_alu instid0(SALU_CYCLE_1) ; GFX1250-GISEL-NEXT: v_mov_b64_e32 v[0:1], s[0:1] @@ -167,18 +167,20 @@ define amdgpu_kernel void @barrier_gv_to_generic(ptr %out) { ; GFX942-SDAG: ; %bb.0: ; GFX942-SDAG-NEXT: s_load_dwordx2 s[2:3], s[4:5], 0x0 ; GFX942-SDAG-NEXT: s_mov_b64 s[0:1], src_shared_base -; GFX942-SDAG-NEXT: v_mov_b32_e32 v0, 0x802001 -; GFX942-SDAG-NEXT: v_mov_b32_e32 v1, s1 +; GFX942-SDAG-NEXT: s_or_b32 s0, s1, 1 +; GFX942-SDAG-NEXT: v_mov_b32_e32 v2, 1 +; GFX942-SDAG-NEXT: v_mov_b32_e32 v3, s0 ; GFX942-SDAG-NEXT: s_waitcnt lgkmcnt(0) -; GFX942-SDAG-NEXT: v_mov_b64_e32 v[2:3], s[2:3] -; GFX942-SDAG-NEXT: flat_store_dwordx2 v[2:3], v[0:1] +; GFX942-SDAG-NEXT: v_mov_b64_e32 v[0:1], s[2:3] +; GFX942-SDAG-NEXT: flat_store_dwordx2 v[0:1], v[2:3] ; GFX942-SDAG-NEXT: s_endpgm ; ; GFX942-GISEL-LABEL: barrier_gv_to_generic: ; GFX942-GISEL: ; %bb.0: ; GFX942-GISEL-NEXT: s_load_dwordx2 s[2:3], s[4:5], 0x0 ; GFX942-GISEL-NEXT: s_mov_b64 s[0:1], src_shared_base -; GFX942-GISEL-NEXT: s_mov_b32 s0, 0x802001 +; GFX942-GISEL-NEXT: s_mov_b32 s0, 1 +; GFX942-GISEL-NEXT: s_or_b32 s1, s1, 1 ; GFX942-GISEL-NEXT: v_mov_b64_e32 v[0:1], s[0:1] ; GFX942-GISEL-NEXT: s_waitcnt lgkmcnt(0) ; GFX942-GISEL-NEXT: v_mov_b64_e32 v[2:3], s[2:3] @@ -193,8 +195,9 @@ define amdgpu_kernel void @barrier_gv_to_generic(ptr %out) { ; GFX1030-SDAG-NEXT: s_setreg_b32 hwreg(HW_REG_FLAT_SCR_HI), s13 ; GFX1030-SDAG-NEXT: s_load_dwordx2 s[2:3], s[8:9], 0x0 ; GFX1030-SDAG-NEXT: s_mov_b64 s[0:1], src_shared_base -; GFX1030-SDAG-NEXT: v_mov_b32_e32 v0, 0x802001 -; GFX1030-SDAG-NEXT: v_mov_b32_e32 v1, s1 +; GFX1030-SDAG-NEXT: v_mov_b32_e32 v0, 1 +; GFX1030-SDAG-NEXT: s_or_b32 s0, s1, 1 +; GFX1030-SDAG-NEXT: v_mov_b32_e32 v1, s0 ; GFX1030-SDAG-NEXT: s_waitcnt lgkmcnt(0) ; GFX1030-SDAG-NEXT: v_mov_b32_e32 v2, s2 ; GFX1030-SDAG-NEXT: v_mov_b32_e32 v3, s3 @@ -209,7 +212,9 @@ define amdgpu_kernel void @barrier_gv_to_generic(ptr %out) { ; GFX1030-GISEL-NEXT: s_setreg_b32 hwreg(HW_REG_FLAT_SCR_HI), s13 ; GFX1030-GISEL-NEXT: s_load_dwordx2 s[2:3], s[8:9], 0x0 ; GFX1030-GISEL-NEXT: s_mov_b64 s[0:1], src_shared_base -; GFX1030-GISEL-NEXT: s_mov_b32 s0, 0x802001 +; GFX1030-GISEL-NEXT: s_mov_b32 s0, 1 +; GFX1030-GISEL-NEXT: s_or_b32 s1, s1, 1 +; GFX1030-GISEL-NEXT: v_mov_b32_e32 v0, s0 ; GFX1030-GISEL-NEXT: v_mov_b32_e32 v1, s1 ; GFX1030-GISEL-NEXT: v_mov_b32_e32 v0, s0 ; GFX1030-GISEL-NEXT: s_waitcnt lgkmcnt(0) @@ -222,8 +227,9 @@ define amdgpu_kernel void @barrier_gv_to_generic(ptr %out) { ; GFX1200-SDAG: ; %bb.0: ; GFX1200-SDAG-NEXT: s_load_b64 s[2:3], s[4:5], 0x0 ; GFX1200-SDAG-NEXT: s_mov_b64 s[0:1], src_shared_base -; GFX1200-SDAG-NEXT: s_delay_alu instid0(SALU_CYCLE_1) -; GFX1200-SDAG-NEXT: v_dual_mov_b32 v0, 0x802001 :: v_dual_mov_b32 v1, s1 +; GFX1200-SDAG-NEXT: s_delay_alu instid0(SALU_CYCLE_1) | instskip(NEXT) | instid1(SALU_CYCLE_1) +; GFX1200-SDAG-NEXT: s_or_b32 s0, s1, 1 +; GFX1200-SDAG-NEXT: v_dual_mov_b32 v0, 1 :: v_dual_mov_b32 v1, s0 ; GFX1200-SDAG-NEXT: s_wait_kmcnt 0x0 ; GFX1200-SDAG-NEXT: v_dual_mov_b32 v2, s2 :: v_dual_mov_b32 v3, s3 ; GFX1200-SDAG-NEXT: flat_store_b64 v[2:3], v[0:1] @@ -233,7 +239,8 @@ define amdgpu_kernel void @barrier_gv_to_generic(ptr %out) { ; GFX1200-GISEL: ; %bb.0: ; GFX1200-GISEL-NEXT: s_load_b64 s[2:3], s[4:5], 0x0 ; GFX1200-GISEL-NEXT: s_mov_b64 s[0:1], src_shared_base -; GFX1200-GISEL-NEXT: s_mov_b32 s0, 0x802001 +; GFX1200-GISEL-NEXT: s_mov_b32 s0, 1 +; GFX1200-GISEL-NEXT: s_or_b32 s1, s1, 1 ; GFX1200-GISEL-NEXT: s_delay_alu instid0(SALU_CYCLE_1) ; GFX1200-GISEL-NEXT: v_dual_mov_b32 v1, s1 :: v_dual_mov_b32 v0, s0 ; GFX1200-GISEL-NEXT: s_wait_kmcnt 0x0 @@ -248,9 +255,10 @@ define amdgpu_kernel void @barrier_gv_to_generic(ptr %out) { ; GFX1250-SDAG-NEXT: s_setreg_imm32_b32 hwreg(HW_REG_WAVE_MODE, 25, 1), 1 ; msbs: dst=0 src0=0 src1=0 src2=0 ; GFX1250-SDAG-NEXT: s_load_b64 s[2:3], s[4:5], 0x0 nv ; GFX1250-SDAG-NEXT: s_mov_b64 s[0:1], src_shared_base +; GFX1250-SDAG-NEXT: v_dual_mov_b32 v2, 0 :: v_dual_mov_b32 v0, 1 +; GFX1250-SDAG-NEXT: s_or_b32 s0, s1, 1 ; GFX1250-SDAG-NEXT: s_delay_alu instid0(SALU_CYCLE_1) -; GFX1250-SDAG-NEXT: v_dual_mov_b32 v2, 0 :: v_dual_mov_b32 v1, s1 -; GFX1250-SDAG-NEXT: v_mov_b32_e32 v0, 0x802001 +; GFX1250-SDAG-NEXT: v_mov_b32_e32 v1, s0 ; GFX1250-SDAG-NEXT: s_wait_kmcnt 0x0 ; GFX1250-SDAG-NEXT: flat_store_b64 v2, v[0:1], s[2:3] ; GFX1250-SDAG-NEXT: s_endpgm @@ -262,7 +270,8 @@ define amdgpu_kernel void @barrier_gv_to_generic(ptr %out) { ; GFX1250-GISEL-NEXT: s_setreg_imm32_b32 hwreg(HW_REG_WAVE_MODE, 25, 1), 1 ; msbs: dst=0 src0=0 src1=0 src2=0 ; GFX1250-GISEL-NEXT: s_load_b64 s[2:3], s[4:5], 0x0 nv ; GFX1250-GISEL-NEXT: s_mov_b64 s[0:1], src_shared_base -; GFX1250-GISEL-NEXT: s_mov_b32 s0, 0x802001 +; GFX1250-GISEL-NEXT: s_mov_b32 s0, 1 +; GFX1250-GISEL-NEXT: s_or_b32 s1, s1, 1 ; GFX1250-GISEL-NEXT: v_mov_b32_e32 v2, 0 ; GFX1250-GISEL-NEXT: v_mov_b64_e32 v[0:1], s[0:1] ; GFX1250-GISEL-NEXT: s_wait_kmcnt 0x0 @@ -354,10 +363,9 @@ define amdgpu_kernel void @generic_to_barrier(ptr %generic, ptr %out) { ; GFX942-SDAG: ; %bb.0: ; GFX942-SDAG-NEXT: s_load_dwordx4 s[0:3], s[4:5], 0x0 ; GFX942-SDAG-NEXT: s_waitcnt lgkmcnt(0) -; GFX942-SDAG-NEXT: v_mov_b32_e32 v0, s2 -; GFX942-SDAG-NEXT: s_add_i32 s2, s0, 0xff7fe000 ; GFX942-SDAG-NEXT: s_cmp_lg_u64 s[0:1], 0 -; GFX942-SDAG-NEXT: s_cselect_b32 s0, s2, 0 +; GFX942-SDAG-NEXT: s_cselect_b32 s0, s0, 0 +; GFX942-SDAG-NEXT: v_mov_b32_e32 v0, s2 ; GFX942-SDAG-NEXT: v_mov_b32_e32 v1, s3 ; GFX942-SDAG-NEXT: v_mov_b32_e32 v2, s0 ; GFX942-SDAG-NEXT: flat_store_dword v[0:1], v2 @@ -367,56 +375,36 @@ define amdgpu_kernel void @generic_to_barrier(ptr %generic, ptr %out) { ; GFX942-GISEL: ; %bb.0: ; GFX942-GISEL-NEXT: s_load_dwordx4 s[0:3], s[4:5], 0x0 ; GFX942-GISEL-NEXT: s_waitcnt lgkmcnt(0) -; GFX942-GISEL-NEXT: s_add_u32 s4, s0, 0xff7fe000 ; GFX942-GISEL-NEXT: s_cmp_lg_u64 s[0:1], 0 -; GFX942-GISEL-NEXT: s_cselect_b32 s0, s4, 0 +; GFX942-GISEL-NEXT: s_cselect_b32 s0, s0, 0 ; GFX942-GISEL-NEXT: v_mov_b32_e32 v2, s0 ; GFX942-GISEL-NEXT: v_mov_b64_e32 v[0:1], s[2:3] ; GFX942-GISEL-NEXT: flat_store_dword v[0:1], v2 ; GFX942-GISEL-NEXT: s_endpgm ; -; GFX1030-SDAG-LABEL: generic_to_barrier: -; GFX1030-SDAG: ; %bb.0: -; GFX1030-SDAG-NEXT: s_add_u32 s12, s12, s17 -; GFX1030-SDAG-NEXT: s_addc_u32 s13, s13, 0 -; GFX1030-SDAG-NEXT: s_setreg_b32 hwreg(HW_REG_FLAT_SCR_LO), s12 -; GFX1030-SDAG-NEXT: s_setreg_b32 hwreg(HW_REG_FLAT_SCR_HI), s13 -; GFX1030-SDAG-NEXT: s_load_dwordx4 s[0:3], s[8:9], 0x0 -; GFX1030-SDAG-NEXT: s_waitcnt lgkmcnt(0) -; GFX1030-SDAG-NEXT: s_add_i32 s4, s0, 0xff7fe000 -; GFX1030-SDAG-NEXT: s_cmp_lg_u64 s[0:1], 0 -; GFX1030-SDAG-NEXT: v_mov_b32_e32 v0, s2 -; GFX1030-SDAG-NEXT: s_cselect_b32 s0, s4, 0 -; GFX1030-SDAG-NEXT: v_mov_b32_e32 v1, s3 -; GFX1030-SDAG-NEXT: v_mov_b32_e32 v2, s0 -; GFX1030-SDAG-NEXT: flat_store_dword v[0:1], v2 -; GFX1030-SDAG-NEXT: s_endpgm -; -; GFX1030-GISEL-LABEL: generic_to_barrier: -; GFX1030-GISEL: ; %bb.0: -; GFX1030-GISEL-NEXT: s_add_u32 s12, s12, s17 -; GFX1030-GISEL-NEXT: s_addc_u32 s13, s13, 0 -; GFX1030-GISEL-NEXT: s_setreg_b32 hwreg(HW_REG_FLAT_SCR_LO), s12 -; GFX1030-GISEL-NEXT: s_setreg_b32 hwreg(HW_REG_FLAT_SCR_HI), s13 -; GFX1030-GISEL-NEXT: s_load_dwordx4 s[0:3], s[8:9], 0x0 -; GFX1030-GISEL-NEXT: s_waitcnt lgkmcnt(0) -; GFX1030-GISEL-NEXT: s_add_u32 s4, s0, 0xff7fe000 -; GFX1030-GISEL-NEXT: s_cmp_lg_u64 s[0:1], 0 -; GFX1030-GISEL-NEXT: v_mov_b32_e32 v0, s2 -; GFX1030-GISEL-NEXT: s_cselect_b32 s0, s4, 0 -; GFX1030-GISEL-NEXT: v_mov_b32_e32 v1, s3 -; GFX1030-GISEL-NEXT: v_mov_b32_e32 v2, s0 -; GFX1030-GISEL-NEXT: flat_store_dword v[0:1], v2 -; GFX1030-GISEL-NEXT: s_endpgm +; GFX1030-LABEL: generic_to_barrier: +; GFX1030: ; %bb.0: +; GFX1030-NEXT: s_add_u32 s12, s12, s17 +; GFX1030-NEXT: s_addc_u32 s13, s13, 0 +; GFX1030-NEXT: s_setreg_b32 hwreg(HW_REG_FLAT_SCR_LO), s12 +; GFX1030-NEXT: s_setreg_b32 hwreg(HW_REG_FLAT_SCR_HI), s13 +; GFX1030-NEXT: s_load_dwordx4 s[0:3], s[8:9], 0x0 +; GFX1030-NEXT: s_waitcnt lgkmcnt(0) +; GFX1030-NEXT: s_cmp_lg_u64 s[0:1], 0 +; GFX1030-NEXT: v_mov_b32_e32 v0, s2 +; GFX1030-NEXT: s_cselect_b32 s0, s0, 0 +; GFX1030-NEXT: v_mov_b32_e32 v1, s3 +; GFX1030-NEXT: v_mov_b32_e32 v2, s0 +; GFX1030-NEXT: flat_store_dword v[0:1], v2 +; GFX1030-NEXT: s_endpgm ; ; GFX1200-SDAG-LABEL: generic_to_barrier: ; GFX1200-SDAG: ; %bb.0: ; GFX1200-SDAG-NEXT: s_load_b128 s[0:3], s[4:5], 0x0 ; GFX1200-SDAG-NEXT: s_wait_kmcnt 0x0 -; GFX1200-SDAG-NEXT: s_add_co_i32 s4, s0, 0xff7fe000 ; GFX1200-SDAG-NEXT: s_cmp_lg_u64 s[0:1], 0 ; GFX1200-SDAG-NEXT: v_dual_mov_b32 v0, s2 :: v_dual_mov_b32 v1, s3 -; GFX1200-SDAG-NEXT: s_cselect_b32 s0, s4, 0 +; GFX1200-SDAG-NEXT: s_cselect_b32 s0, s0, 0 ; GFX1200-SDAG-NEXT: s_delay_alu instid0(SALU_CYCLE_1) ; GFX1200-SDAG-NEXT: v_mov_b32_e32 v2, s0 ; GFX1200-SDAG-NEXT: flat_store_b32 v[0:1], v2 @@ -426,10 +414,9 @@ define amdgpu_kernel void @generic_to_barrier(ptr %generic, ptr %out) { ; GFX1200-GISEL: ; %bb.0: ; GFX1200-GISEL-NEXT: s_load_b128 s[0:3], s[4:5], 0x0 ; GFX1200-GISEL-NEXT: s_wait_kmcnt 0x0 -; GFX1200-GISEL-NEXT: s_add_co_u32 s4, s0, 0xff7fe000 ; GFX1200-GISEL-NEXT: s_cmp_lg_u64 s[0:1], 0 ; GFX1200-GISEL-NEXT: v_mov_b32_e32 v0, s2 -; GFX1200-GISEL-NEXT: s_cselect_b32 s0, s4, 0 +; GFX1200-GISEL-NEXT: s_cselect_b32 s0, s0, 0 ; GFX1200-GISEL-NEXT: s_delay_alu instid0(SALU_CYCLE_1) ; GFX1200-GISEL-NEXT: v_dual_mov_b32 v1, s3 :: v_dual_mov_b32 v2, s0 ; GFX1200-GISEL-NEXT: flat_store_b32 v[0:1], v2 @@ -442,9 +429,8 @@ define amdgpu_kernel void @generic_to_barrier(ptr %generic, ptr %out) { ; GFX1250-SDAG-NEXT: s_setreg_imm32_b32 hwreg(HW_REG_WAVE_MODE, 25, 1), 1 ; msbs: dst=0 src0=0 src1=0 src2=0 ; GFX1250-SDAG-NEXT: s_load_b128 s[0:3], s[4:5], 0x0 nv ; GFX1250-SDAG-NEXT: s_wait_kmcnt 0x0 -; GFX1250-SDAG-NEXT: s_add_co_i32 s4, s0, 0xff7fe000 ; GFX1250-SDAG-NEXT: s_cmp_lg_u64 s[0:1], 0 -; GFX1250-SDAG-NEXT: s_cselect_b32 s0, s4, 0 +; GFX1250-SDAG-NEXT: s_cselect_b32 s0, s0, 0 ; GFX1250-SDAG-NEXT: s_delay_alu instid0(SALU_CYCLE_1) ; GFX1250-SDAG-NEXT: v_dual_mov_b32 v0, 0 :: v_dual_mov_b32 v1, s0 ; GFX1250-SDAG-NEXT: flat_store_b32 v0, v1, s[2:3] @@ -458,9 +444,8 @@ define amdgpu_kernel void @generic_to_barrier(ptr %generic, ptr %out) { ; GFX1250-GISEL-NEXT: s_load_b128 s[0:3], s[4:5], 0x0 nv ; GFX1250-GISEL-NEXT: v_mov_b32_e32 v1, 0 ; GFX1250-GISEL-NEXT: s_wait_kmcnt 0x0 -; GFX1250-GISEL-NEXT: s_add_co_u32 s4, s0, 0xff7fe000 ; GFX1250-GISEL-NEXT: s_cmp_lg_u64 s[0:1], 0 -; GFX1250-GISEL-NEXT: s_cselect_b32 s0, s4, 0 +; GFX1250-GISEL-NEXT: s_cselect_b32 s0, s0, 0 ; GFX1250-GISEL-NEXT: s_delay_alu instid0(SALU_CYCLE_1) ; GFX1250-GISEL-NEXT: v_mov_b32_e32 v0, s0 ; GFX1250-GISEL-NEXT: flat_store_b32 v1, v0, s[2:3] >From 5b355da46e7746133133c5d573f4bef681ef41f7 Mon Sep 17 00:00:00 2001 From: pvanhout <[email protected]> Date: Tue, 16 Jun 2026 15:39:21 +0200 Subject: [PATCH 2/4] Update docs --- llvm/docs/AMDGPUUsage.rst | 33 ++++++++++++++++++++------------- 1 file changed, 20 insertions(+), 13 deletions(-) diff --git a/llvm/docs/AMDGPUUsage.rst b/llvm/docs/AMDGPUUsage.rst index 1827508e43fd2..160548eb72ce1 100644 --- a/llvm/docs/AMDGPUUsage.rst +++ b/llvm/docs/AMDGPUUsage.rst @@ -1408,21 +1408,23 @@ Synthetic Apertures *Synthetic apertures* are defined that enable safe roundtrips of pointers from special address spaces through the generic address space. Attempting to dereference generic pointers obtained in this way (using e.g. `load` or -`store`) has undefined behavior. The following synthetic apertures are defined: +`store`) has undefined behavior. -.. table:: AMDGPU Synthetic Apertures - :name: amdgpu-synthetic-apertures-table - :widths: 40 20 40 +The address size of an address spaces that use synthetic apertures can only be 32 bits +wide or less. The full width of the source pointer is usable and preserved when converting it +from/to the generic address space. + +The following synthetic apertures are defined: - ============ ====== =============================================================== - Name Number Corresponding :ref:`Address Space<amdgpu-address-spaces-table>` - ============ ====== =============================================================== - BARRIER 1 Barrier - ============ ====== =============================================================== +.. table:: AMDGPU Synthetic Apertures + :name: amdgpu-synthetic-apertures-table + :widths: 30 10 30 30 -Note that the address size of an address spaces implemented via synthetic apertures -can only be 32 bits wide or less. The full width of the source pointer is usable and -preserved when converting it from/to the generic address space. + ============ ======= ============== ================================================================ + Name Number Mask Corresponding :ref:`Address Space<amdgpu-address-spaces-table>` + ============ ======= ============== ================================================================ + BARRIER 1 ``0x00000001`` Barrier + ============ ======= ============== ================================================================ Converting a pointer to generic (64 bits) using synthetic apertures is done as follows: @@ -1430,7 +1432,12 @@ Converting a pointer to generic (64 bits) using synthetic apertures is done as f * The upper 32 bits are a bitwise ``OR`` of: * The upper 32 bits of the LDS segment aperture. - * The synthetic aperture number. + + * **NOTE:** The lower 48 bits of the LDS segment aperture are expected to be zeroes. + + * The relevant mask in the :ref:`above table<amdgpu-synthetic-apertures-table>`. + + * **NOTE:** The upper 16 bits of the mask are expected to be zeroes. The conversion back to the original address space can simply be done by discarding the upper 32 bits of the generic pointer. >From 5362e8de3a81ca344741584be9fe16f4ce2e68c8 Mon Sep 17 00:00:00 2001 From: pvanhout <[email protected]> Date: Fri, 19 Jun 2026 10:40:08 +0200 Subject: [PATCH 3/4] Comments --- llvm/lib/Target/AMDGPU/AMDGPULegalizerInfo.cpp | 2 +- llvm/lib/Target/AMDGPU/AMDGPUMemoryUtils.cpp | 2 +- llvm/lib/Target/AMDGPU/AMDGPUMemoryUtils.h | 2 +- llvm/lib/Target/AMDGPU/SIDefines.h | 7 ++++--- llvm/lib/Target/AMDGPU/SIISelLowering.cpp | 2 +- llvm/test/CodeGen/AMDGPU/addrspacecast-barrier.ll | 4 +--- 6 files changed, 9 insertions(+), 10 deletions(-) diff --git a/llvm/lib/Target/AMDGPU/AMDGPULegalizerInfo.cpp b/llvm/lib/Target/AMDGPU/AMDGPULegalizerInfo.cpp index af66ab91362d8..4292ecaa6821e 100644 --- a/llvm/lib/Target/AMDGPU/AMDGPULegalizerInfo.cpp +++ b/llvm/lib/Target/AMDGPU/AMDGPULegalizerInfo.cpp @@ -2429,7 +2429,7 @@ Register AMDGPULegalizerInfo::getSegmentAperture(unsigned AS, MachineRegisterInfo &MRI, MachineIRBuilder &B) const { unsigned BaseAS = AS; - unsigned SANum = AMDGPU::tryGetSyntheticApertureNumber(AS); + unsigned SANum = AMDGPU::getSyntheticApertureNumber(AS); if (SANum != AMDGPU::SyntheticAperture::None) BaseAS = AMDGPUAS::LOCAL_ADDRESS; diff --git a/llvm/lib/Target/AMDGPU/AMDGPUMemoryUtils.cpp b/llvm/lib/Target/AMDGPU/AMDGPUMemoryUtils.cpp index 5b1a5f6d4e3db..f35e4e012e2a2 100644 --- a/llvm/lib/Target/AMDGPU/AMDGPUMemoryUtils.cpp +++ b/llvm/lib/Target/AMDGPU/AMDGPUMemoryUtils.cpp @@ -32,7 +32,7 @@ Align getAlign(const DataLayout &DL, const GlobalVariable *GV) { GV->getValueType()); } -unsigned tryGetSyntheticApertureNumber(unsigned AS) { +unsigned getSyntheticApertureNumber(unsigned AS) { switch (AS) { case AMDGPUAS::BARRIER: return SyntheticAperture::BARRIER; diff --git a/llvm/lib/Target/AMDGPU/AMDGPUMemoryUtils.h b/llvm/lib/Target/AMDGPU/AMDGPUMemoryUtils.h index 791ca0f7b779c..365605c796fdd 100644 --- a/llvm/lib/Target/AMDGPU/AMDGPUMemoryUtils.h +++ b/llvm/lib/Target/AMDGPU/AMDGPUMemoryUtils.h @@ -39,7 +39,7 @@ Align getAlign(const DataLayout &DL, const GlobalVariable *GV); // Get the synthetic aperture number for the given address space, or None (0) // if the address space does not have one. -unsigned tryGetSyntheticApertureNumber(unsigned AS); +unsigned getSyntheticApertureNumber(unsigned AS); // Copy metadata onto a load widened to read a superset of Source's bytes. Only // value-independent metadata is copied; metadata describing the loaded value diff --git a/llvm/lib/Target/AMDGPU/SIDefines.h b/llvm/lib/Target/AMDGPU/SIDefines.h index 223b916da20cf..cdb2530a53d78 100644 --- a/llvm/lib/Target/AMDGPU/SIDefines.h +++ b/llvm/lib/Target/AMDGPU/SIDefines.h @@ -1358,13 +1358,14 @@ enum Type { } // namespace Barrier namespace SyntheticAperture { -/// Synthetic aperture numbers. +/// Synthetic aperture numbers. This is OR'd with the high +/// bits of the LDS aperture pointer. /// /// NOTE: This is also documented in AMDGPUUsage. enum SyntheticAperture { - None = 0, + None = 0x00000000, - BARRIER = 1, + BARRIER = 0x00000001, }; } // namespace SyntheticAperture diff --git a/llvm/lib/Target/AMDGPU/SIISelLowering.cpp b/llvm/lib/Target/AMDGPU/SIISelLowering.cpp index 0359aa67985c6..c02dead05508a 100644 --- a/llvm/lib/Target/AMDGPU/SIISelLowering.cpp +++ b/llvm/lib/Target/AMDGPU/SIISelLowering.cpp @@ -9326,7 +9326,7 @@ SDValue SITargetLowering::LowerINLINEASM(SDValue Op, SelectionDAG &DAG) const { SDValue SITargetLowering::getSegmentAperture(unsigned AS, const SDLoc &DL, SelectionDAG &DAG) const { unsigned BaseAS = AS; - unsigned SANum = AMDGPU::tryGetSyntheticApertureNumber(AS); + unsigned SANum = AMDGPU::getSyntheticApertureNumber(AS); if (SANum != AMDGPU::SyntheticAperture::None) BaseAS = AMDGPUAS::LOCAL_ADDRESS; diff --git a/llvm/test/CodeGen/AMDGPU/addrspacecast-barrier.ll b/llvm/test/CodeGen/AMDGPU/addrspacecast-barrier.ll index 2e65d365b63ed..4f9dec008819c 100644 --- a/llvm/test/CodeGen/AMDGPU/addrspacecast-barrier.ll +++ b/llvm/test/CodeGen/AMDGPU/addrspacecast-barrier.ll @@ -216,7 +216,6 @@ define amdgpu_kernel void @barrier_gv_to_generic(ptr %out) { ; GFX1030-GISEL-NEXT: s_or_b32 s1, s1, 1 ; GFX1030-GISEL-NEXT: v_mov_b32_e32 v0, s0 ; GFX1030-GISEL-NEXT: v_mov_b32_e32 v1, s1 -; GFX1030-GISEL-NEXT: v_mov_b32_e32 v0, s0 ; GFX1030-GISEL-NEXT: s_waitcnt lgkmcnt(0) ; GFX1030-GISEL-NEXT: v_mov_b32_e32 v2, s2 ; GFX1030-GISEL-NEXT: v_mov_b32_e32 v3, s3 @@ -242,7 +241,7 @@ define amdgpu_kernel void @barrier_gv_to_generic(ptr %out) { ; GFX1200-GISEL-NEXT: s_mov_b32 s0, 1 ; GFX1200-GISEL-NEXT: s_or_b32 s1, s1, 1 ; GFX1200-GISEL-NEXT: s_delay_alu instid0(SALU_CYCLE_1) -; GFX1200-GISEL-NEXT: v_dual_mov_b32 v1, s1 :: v_dual_mov_b32 v0, s0 +; GFX1200-GISEL-NEXT: v_dual_mov_b32 v0, s0 :: v_dual_mov_b32 v1, s1 ; GFX1200-GISEL-NEXT: s_wait_kmcnt 0x0 ; GFX1200-GISEL-NEXT: v_dual_mov_b32 v2, s2 :: v_dual_mov_b32 v3, s3 ; GFX1200-GISEL-NEXT: flat_store_b64 v[2:3], v[0:1] @@ -455,5 +454,4 @@ define amdgpu_kernel void @generic_to_barrier(ptr %generic, ptr %out) { ret void } ;; NOTE: These prefixes are unused and the list is autogenerated. Do not add tests below this line: -; GFX1030: {{.*}} ; GFX1200: {{.*}} >From 58b385b63a141f6c0bf94f667ea415ddc41c9324 Mon Sep 17 00:00:00 2001 From: pvanhout <[email protected]> Date: Wed, 22 Jul 2026 10:11:24 +0200 Subject: [PATCH 4/4] Comments --- llvm/docs/AMDGPUUsage.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/llvm/docs/AMDGPUUsage.rst b/llvm/docs/AMDGPUUsage.rst index 160548eb72ce1..a227d1dc4da89 100644 --- a/llvm/docs/AMDGPUUsage.rst +++ b/llvm/docs/AMDGPUUsage.rst @@ -1443,7 +1443,7 @@ The conversion back to the original address space can simply be done by discardi upper 32 bits of the generic pointer. As the LDS aperture is defined by its 16 most significant bits, we can theoretically -support up to ``2 << 16`` synthetic apertures safely. +support up to ``(1 << 16) - 1`` synthetic apertures safely. .. _amdgpu-memory-scopes: _______________________________________________ llvm-branch-commits mailing list [email protected] https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits
