[llvm-branch-commits] [llvm] 3b5d36e - [Verifier] disable llvm.experimental.noalias.scope.decl dominance check.
Author: Jeroen Dobbelaere Date: 2021-01-25T16:43:08+01:00 New Revision: 3b5d36ece21f9baf96d82944b0165cb352443bee URL: https://github.com/llvm/llvm-project/commit/3b5d36ece21f9baf96d82944b0165cb352443bee DIFF: https://github.com/llvm/llvm-project/commit/3b5d36ece21f9baf96d82944b0165cb352443bee.diff LOG: [Verifier] disable llvm.experimental.noalias.scope.decl dominance check. This was enabled in https://reviews.llvm.org/D95335 but it breaks the stage2 fuchsia build (See http://lab.llvm.org:8011/#/builders/98/builds/4105/steps/9/logs/stdio) Added: Modified: llvm/lib/IR/Verifier.cpp Removed: diff --git a/llvm/lib/IR/Verifier.cpp b/llvm/lib/IR/Verifier.cpp index a05c3ed200fd..2b12e656c45e 100644 --- a/llvm/lib/IR/Verifier.cpp +++ b/llvm/lib/IR/Verifier.cpp @@ -116,7 +116,7 @@ using namespace llvm; static cl::opt VerifyNoAliasScopeDomination( -"verify-noalias-scope-decl-dom", cl::Hidden, cl::init(true), +"verify-noalias-scope-decl-dom", cl::Hidden, cl::init(false), cl::desc("Ensure that llvm.experimental.noalias.scope.decl for identical " "scopes are not dominating")); ___ llvm-branch-commits mailing list llvm-branch-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits
[llvm-branch-commits] [llvm] 6e530a3 - [Verifier] enable and limit llvm.experimental.noalias.scope.decl dominance checking
Author: Jeroen Dobbelaere Date: 2021-01-25T16:19:12+01:00 New Revision: 6e530a3dac0c41608bac30f12d59fa3cbca48c4a URL: https://github.com/llvm/llvm-project/commit/6e530a3dac0c41608bac30f12d59fa3cbca48c4a DIFF: https://github.com/llvm/llvm-project/commit/6e530a3dac0c41608bac30f12d59fa3cbca48c4a.diff LOG: [Verifier] enable and limit llvm.experimental.noalias.scope.decl dominance checking Checking the llvm.experimental.noalias.scope.decl dominance can be worstcase O(N^2). Limit the dominance check to N=32. Reviewed By: fhahn Differential Revision: https://reviews.llvm.org/D95335 Added: Modified: llvm/lib/IR/Verifier.cpp Removed: diff --git a/llvm/lib/IR/Verifier.cpp b/llvm/lib/IR/Verifier.cpp index 8d960770313b..a05c3ed200fd 100644 --- a/llvm/lib/IR/Verifier.cpp +++ b/llvm/lib/IR/Verifier.cpp @@ -116,7 +116,7 @@ using namespace llvm; static cl::opt VerifyNoAliasScopeDomination( -"verify-noalias-scope-decl-dom", cl::Hidden, cl::init(false), +"verify-noalias-scope-decl-dom", cl::Hidden, cl::init(true), cl::desc("Ensure that llvm.experimental.noalias.scope.decl for identical " "scopes are not dominating")); @@ -5587,18 +5587,17 @@ void Verifier::verifyNoAliasScopeDecl() { } while (ItNext != NoAliasScopeDecls.end() && GetScope(*ItNext) == CurScope); -// [ItCurrent, ItNext[ represents the declarations for the same scope. -// Ensure they are not dominating each other -for (auto *I : llvm::make_range(ItCurrent, ItNext)) { - for (auto *J : llvm::make_range(ItCurrent, ItNext)) { -if (I != J) { - Assert(!DT.dominates(I, J), - "llvm.experimental.noalias.scope.decl dominates another one " - "with the same scope", - I); -} - } -} +// [ItCurrent, ItNext) represents the declarations for the same scope. +// Ensure they are not dominating each other.. but only if it is not too +// expensive. +if (ItNext - ItCurrent < 32) + for (auto *I : llvm::make_range(ItCurrent, ItNext)) +for (auto *J : llvm::make_range(ItCurrent, ItNext)) + if (I != J) +Assert(!DT.dominates(I, J), + "llvm.experimental.noalias.scope.decl dominates another one " + "with the same scope", + I); ItCurrent = ItNext; } } ___ llvm-branch-commits mailing list llvm-branch-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits
[llvm-branch-commits] [llvm] dcc7706 - [InstCombine] Remove unused llvm.experimental.noalias.scope.decl
Author: Jeroen Dobbelaere Date: 2021-01-24T13:55:50+01:00 New Revision: dcc7706fcf2438b92d6f619e63c5db4880042ed2 URL: https://github.com/llvm/llvm-project/commit/dcc7706fcf2438b92d6f619e63c5db4880042ed2 DIFF: https://github.com/llvm/llvm-project/commit/dcc7706fcf2438b92d6f619e63c5db4880042ed2.diff LOG: [InstCombine] Remove unused llvm.experimental.noalias.scope.decl A @llvm.experimental.noalias.scope.decl is only useful if there is !alias.scope and !noalias metadata that uses the declared scope. When that is not the case for at least one of the two, the intrinsic call can as well be removed. Reviewed By: nikic Differential Revision: https://reviews.llvm.org/D95141 Added: llvm/test/Transforms/InstCombine/noalias-scope-decl.ll Modified: llvm/lib/Transforms/InstCombine/InstructionCombining.cpp llvm/test/Transforms/Coroutines/coro-retcon-resume-values.ll llvm/test/Transforms/Coroutines/coro-retcon-value.ll llvm/test/Transforms/Coroutines/coro-retcon.ll llvm/test/Transforms/Coroutines/ex2.ll llvm/test/Transforms/Coroutines/ex3.ll llvm/test/Transforms/Coroutines/ex4.ll llvm/test/Transforms/PhaseOrdering/inlining-alignment-assumptions.ll llvm/test/Transforms/PhaseOrdering/instcombine-sroa-inttoptr.ll Removed: diff --git a/llvm/lib/Transforms/InstCombine/InstructionCombining.cpp b/llvm/lib/Transforms/InstCombine/InstructionCombining.cpp index 2f8a80a89992..518e909e8ab4 100644 --- a/llvm/lib/Transforms/InstCombine/InstructionCombining.cpp +++ b/llvm/lib/Transforms/InstCombine/InstructionCombining.cpp @@ -3766,6 +3766,55 @@ bool InstCombinerImpl::run() { return MadeIRChange; } +// Track the scopes used by !alias.scope and !noalias. In a function, a +// @llvm.experimental.noalias.scope.decl is only useful if that scope is used +// by both sets. If not, the declaration of the scope can be safely omitted. +// The MDNode of the scope can be omitted as well for the instructions that are +// part of this function. We do not do that at this point, as this might become +// too time consuming to do. +class AliasScopeTracker { + SmallPtrSet UsedAliasScopesAndLists; + SmallPtrSet UsedNoAliasScopesAndLists; + +public: + void analyse(Instruction *I) { +// This seems to be faster than checking 'mayReadOrWriteMemory()'. +if (!I->hasMetadataOtherThanDebugLoc()) + return; + +auto Track = [](Metadata *ScopeList, auto ) { + const auto *MDScopeList = dyn_cast_or_null(ScopeList); + if (!MDScopeList || !Container.insert(MDScopeList).second) +return; + for (auto : MDScopeList->operands()) +if (auto *MDScope = dyn_cast(MDOperand)) + Container.insert(MDScope); +}; + +Track(I->getMetadata(LLVMContext::MD_alias_scope), UsedAliasScopesAndLists); +Track(I->getMetadata(LLVMContext::MD_noalias), UsedNoAliasScopesAndLists); + } + + bool isNoAliasScopeDeclDead(Instruction *Inst) { +NoAliasScopeDeclInst *Decl = dyn_cast(Inst); +if (!Decl) + return false; + +assert(Decl->use_empty() && + "llvm.experimental.noalias.scope.decl in use ?"); +const MDNode *MDSL = Decl->getScopeList(); +assert(MDSL->getNumOperands() == 1 && + "llvm.experimental.noalias.scope should refer to a single scope"); +auto = MDSL->getOperand(0); +if (auto *MD = dyn_cast(MDOperand)) + return !UsedAliasScopesAndLists.contains(MD) || + !UsedNoAliasScopesAndLists.contains(MD); + +// Not an MDNode ? throw away. +return true; + } +}; + /// Populate the IC worklist from a function, by walking it in depth-first /// order and adding all reachable code to the worklist. /// @@ -3784,6 +3833,7 @@ static bool prepareICWorklistFromFunction(Function , const DataLayout , SmallVector InstrsForInstCombineWorklist; DenseMap FoldedConstants; + AliasScopeTracker SeenAliasScopes; do { BasicBlock *BB = Worklist.pop_back_val(); @@ -3830,8 +3880,10 @@ static bool prepareICWorklistFromFunction(Function , const DataLayout , // Skip processing debug intrinsics in InstCombine. Processing these call instructions // consumes non-trivial amount of time and provides no value for the optimization. - if (!isa(Inst)) + if (!isa(Inst)) { InstrsForInstCombineWorklist.push_back(Inst); +SeenAliasScopes.analyse(Inst); + } } // Recursively visit successors. If this is a branch or switch on a @@ -3879,7 +3931,8 @@ static bool prepareICWorklistFromFunction(Function , const DataLayout , for (Instruction *Inst : reverse(InstrsForInstCombineWorklist)) { // DCE instruction if trivially dead. As we iterate in reverse program // order here, we will clean up whole chains of dead instructions. -if (isInstructionTriviallyDead(Inst, TLI)) { +if (isInstructionTriviallyDead(Inst, TLI) || +
[llvm-branch-commits] [llvm] 659c7bc - [LoopRotate] Use llvm.experimental.noalias.scope.decl for duplicating noalias metadata as needed
Author: Jeroen Dobbelaere Date: 2021-01-24T13:53:13+01:00 New Revision: 659c7bcde62e96c84f157b1d4ac4f320c56089a1 URL: https://github.com/llvm/llvm-project/commit/659c7bcde62e96c84f157b1d4ac4f320c56089a1 DIFF: https://github.com/llvm/llvm-project/commit/659c7bcde62e96c84f157b1d4ac4f320c56089a1.diff LOG: [LoopRotate] Use llvm.experimental.noalias.scope.decl for duplicating noalias metadata as needed Similar to D92887, LoopRotation also needs duplicate the noalias scopes when rotating a `@llvm.experimental.noalias.scope.decl` across a block boundary. This is based on the version from the Full Restrict paches (D68511). The problem it fixes also showed up in Transforms/Coroutines/ex5.ll after D93040 (when enabling strict checking with -verify-noalias-scope-decl-dom). Reviewed By: nikic Differential Revision: https://reviews.llvm.org/D94306 Added: llvm/test/Transforms/LoopRotate/noalias.ll Modified: llvm/include/llvm/Transforms/Utils/Cloning.h llvm/lib/Transforms/Utils/CloneFunction.cpp llvm/lib/Transforms/Utils/LoopRotationUtils.cpp Removed: diff --git a/llvm/include/llvm/Transforms/Utils/Cloning.h b/llvm/include/llvm/Transforms/Utils/Cloning.h index e43c43cf76e7..16062fb2f5f5 100644 --- a/llvm/include/llvm/Transforms/Utils/Cloning.h +++ b/llvm/include/llvm/Transforms/Utils/Cloning.h @@ -302,6 +302,13 @@ void adaptNoAliasScopes( void cloneAndAdaptNoAliasScopes(ArrayRef NoAliasDeclScopes, ArrayRef NewBlocks, LLVMContext , StringRef Ext); + +/// Clone the specified noalias decl scopes. Then adapt all instructions in the +/// [IStart, IEnd] (IEnd included !) range to the cloned versions. 'Ext' will be +/// added to the duplicate scope names. +void cloneAndAdaptNoAliasScopes(ArrayRef NoAliasDeclScopes, +Instruction *IStart, Instruction *IEnd, +LLVMContext , StringRef Ext); } // end namespace llvm #endif // LLVM_TRANSFORMS_UTILS_CLONING_H diff --git a/llvm/lib/Transforms/Utils/CloneFunction.cpp b/llvm/lib/Transforms/Utils/CloneFunction.cpp index 3ff1e59f8eb1..ac474fbac7b3 100644 --- a/llvm/lib/Transforms/Utils/CloneFunction.cpp +++ b/llvm/lib/Transforms/Utils/CloneFunction.cpp @@ -968,6 +968,28 @@ void llvm::cloneAndAdaptNoAliasScopes( adaptNoAliasScopes(, ClonedScopes, ClonedMVScopes, Context); } +void llvm::cloneAndAdaptNoAliasScopes( +ArrayRef NoAliasDeclScopes, Instruction *IStart, +Instruction *IEnd, LLVMContext , StringRef Ext) { + if (NoAliasDeclScopes.empty()) +return; + + DenseMap ClonedScopes; + DenseMap ClonedMVScopes; + LLVM_DEBUG(dbgs() << "cloneAndAdaptNoAliasScopes: cloning " +<< NoAliasDeclScopes.size() << " node(s)\n"); + + cloneNoAliasScopes(NoAliasDeclScopes, ClonedScopes, ClonedMVScopes, Ext, + Context); + // Identify instructions using metadata that needs adaptation + assert(IStart->getParent() == IEnd->getParent() && " diff erent basic block ?"); + auto ItStart = IStart->getIterator(); + auto ItEnd = IEnd->getIterator(); + ++ItEnd; // IEnd is included, increment ItEnd to get the end of the range + for (auto : llvm::make_range(ItStart, ItEnd)) +adaptNoAliasScopes(, ClonedScopes, ClonedMVScopes, Context); +} + void llvm::identifyNoAliasScopesToClone( ArrayRef BBs, SmallVectorImpl ) { diff --git a/llvm/lib/Transforms/Utils/LoopRotationUtils.cpp b/llvm/lib/Transforms/Utils/LoopRotationUtils.cpp index 850170960937..8192092822aa 100644 --- a/llvm/lib/Transforms/Utils/LoopRotationUtils.cpp +++ b/llvm/lib/Transforms/Utils/LoopRotationUtils.cpp @@ -35,6 +35,7 @@ #include "llvm/Support/Debug.h" #include "llvm/Support/raw_ostream.h" #include "llvm/Transforms/Utils/BasicBlockUtils.h" +#include "llvm/Transforms/Utils/Cloning.h" #include "llvm/Transforms/Utils/Local.h" #include "llvm/Transforms/Utils/LoopUtils.h" #include "llvm/Transforms/Utils/SSAUpdater.h" @@ -400,6 +401,14 @@ bool LoopRotate::rotateLoop(Loop *L, bool SimplifiedLatch) { break; } +// Remember the local noalias scope declarations in the header. After the +// rotation, they must be duplicated and the scope must be cloned. This +// avoids unwanted interaction across iterations. +SmallVector NoAliasDeclInstructions; +for (Instruction : *OrigHeader) + if (auto *Decl = dyn_cast()) +NoAliasDeclInstructions.push_back(Decl); + while (I != E) { Instruction *Inst = &*I++; @@ -460,6 +469,70 @@ bool LoopRotate::rotateLoop(Loop *L, bool SimplifiedLatch) { } } +if (!NoAliasDeclInstructions.empty()) { + // There are noalias scope declarations: + // (general): + // Original:OrigPre { OrigHeader NewHeader ... Latch } + // after: (OrigPre+OrigHeader') { NewHeader ... Latch
[llvm-branch-commits] [llvm] 7746296 - [LoopUnroll] Use llvm.experimental.noalias.scope.decl for duplicating noalias metadata as needed
Author: Jeroen Dobbelaere Date: 2021-01-24T13:48:20+01:00 New Revision: 774629641bf32503353a179e98aaa3ef055d6870 URL: https://github.com/llvm/llvm-project/commit/774629641bf32503353a179e98aaa3ef055d6870 DIFF: https://github.com/llvm/llvm-project/commit/774629641bf32503353a179e98aaa3ef055d6870.diff LOG: [LoopUnroll] Use llvm.experimental.noalias.scope.decl for duplicating noalias metadata as needed This is a fix for https://bugs.llvm.org/show_bug.cgi?id=39282. Compared to D90104, this version is based on part of the full restrict patched (D68484) and uses the `@llvm.experimental.noalias.scope.decl` intrinsic to track the location where !noalias and !alias.scope scopes have been introduced. This allows us to only duplicate the scopes that are really needed. Notes: - it also includes changes and tests from D90104 Reviewed By: nikic Differential Revision: https://reviews.llvm.org/D92887 Added: llvm/test/Transforms/LoopUnroll/noalias.ll Modified: llvm/include/llvm/IR/Metadata.h llvm/include/llvm/Transforms/Utils/Cloning.h llvm/lib/Transforms/Utils/CloneFunction.cpp llvm/lib/Transforms/Utils/LoopUnroll.cpp llvm/test/Transforms/PhaseOrdering/pr39282.ll Removed: diff --git a/llvm/include/llvm/IR/Metadata.h b/llvm/include/llvm/IR/Metadata.h index 33b92c3c90da..0b87416befe9 100644 --- a/llvm/include/llvm/IR/Metadata.h +++ b/llvm/include/llvm/IR/Metadata.h @@ -1208,6 +1208,12 @@ class AliasScopeNode { return nullptr; return dyn_cast_or_null(Node->getOperand(1)); } + StringRef getName() const { +if (Node->getNumOperands() > 2) + if (MDString *N = dyn_cast_or_null(Node->getOperand(2))) +return N->getString(); +return StringRef(); + } }; /// Typed iterator through MDNode operands. diff --git a/llvm/include/llvm/Transforms/Utils/Cloning.h b/llvm/include/llvm/Transforms/Utils/Cloning.h index dffb7801bc8e..e43c43cf76e7 100644 --- a/llvm/include/llvm/Transforms/Utils/Cloning.h +++ b/llvm/include/llvm/Transforms/Utils/Cloning.h @@ -268,6 +268,40 @@ void updateProfileCallee( Function *Callee, int64_t entryDelta, const ValueMap *VMap = nullptr); +/// Find the 'llvm.experimental.noalias.scope.decl' intrinsics in the specified +/// basic blocks and extract their scope. These are candidates for duplication +/// when cloning. +void identifyNoAliasScopesToClone( +ArrayRef BBs, +SmallVectorImpl ); + +/// Duplicate the specified list of noalias decl scopes. +/// The 'Ext' string is added as an extension to the name. +/// Afterwards, the ClonedMVScopes contains a mapping of the original MV onto +/// the cloned version. +/// The ClonedScopes contains the mapping of the original scope MDNode onto the +/// cloned scope. +/// Be aware that the cloned scopes are still part of the original scope domain. +void cloneNoAliasScopes( +ArrayRef NoAliasDeclScopes, +DenseMap , +DenseMap , +StringRef Ext, LLVMContext ); + +/// Adapt the metadata for the specified instruction according to the +/// provided mapping. This is normally used after cloning an instruction, when +/// some noalias scopes needed to be cloned. +void adaptNoAliasScopes( +llvm::Instruction *I, const DenseMap , +const DenseMap , +LLVMContext ); + +/// Clone the specified noalias decl scopes. Then adapt all instructions in the +/// NewBlocks basicblocks to the cloned versions. +/// 'Ext' will be added to the duplicate scope names. +void cloneAndAdaptNoAliasScopes(ArrayRef NoAliasDeclScopes, +ArrayRef NewBlocks, +LLVMContext , StringRef Ext); } // end namespace llvm #endif // LLVM_TRANSFORMS_UTILS_CLONING_H diff --git a/llvm/lib/Transforms/Utils/CloneFunction.cpp b/llvm/lib/Transforms/Utils/CloneFunction.cpp index 5193b2da26fe..3ff1e59f8eb1 100644 --- a/llvm/lib/Transforms/Utils/CloneFunction.cpp +++ b/llvm/lib/Transforms/Utils/CloneFunction.cpp @@ -27,6 +27,7 @@ #include "llvm/IR/Instructions.h" #include "llvm/IR/IntrinsicInst.h" #include "llvm/IR/LLVMContext.h" +#include "llvm/IR/MDBuilder.h" #include "llvm/IR/Metadata.h" #include "llvm/IR/Module.h" #include "llvm/Transforms/Utils/BasicBlockUtils.h" @@ -36,6 +37,8 @@ #include using namespace llvm; +#define DEBUG_TYPE "clone-function" + /// See comments in Cloning.h. BasicBlock *llvm::CloneBasicBlock(const BasicBlock *BB, ValueToValueMapTy , const Twine , Function *F, @@ -881,3 +884,96 @@ BasicBlock *llvm::DuplicateInstructionsInSplitBetween( return NewBB; } + +void llvm::cloneNoAliasScopes( +ArrayRef NoAliasDeclScopes, +DenseMap , +DenseMap , +StringRef Ext, LLVMContext ) { + MDBuilder MDB(Context); + + for (auto *MV : NoAliasDeclScopes) { +SmallVector ScopeList; +for (auto : cast(MV->getMetadata())->operands()) { + if (MDNode *MD =
[llvm-branch-commits] [llvm] 2b9a834 - [InlineFunction] Use llvm.experimental.noalias.scope.decl for noalias arguments.
Author: Jeroen Dobbelaere Date: 2021-01-23T12:10:57+01:00 New Revision: 2b9a834c43cb1f93d33958c14b695896bb4e9c1e URL: https://github.com/llvm/llvm-project/commit/2b9a834c43cb1f93d33958c14b695896bb4e9c1e DIFF: https://github.com/llvm/llvm-project/commit/2b9a834c43cb1f93d33958c14b695896bb4e9c1e.diff LOG: [InlineFunction] Use llvm.experimental.noalias.scope.decl for noalias arguments. Insert a llvm.experimental.noalias.scope.decl intrinsic that identifies where a noalias argument was inlined. This patch includes some refactorings from D90104. Reviewed By: nikic Differential Revision: https://reviews.llvm.org/D93040 Added: llvm/test/Transforms/Inline/noalias-calls2.ll Modified: clang/test/CodeGen/aarch64-ls64.c llvm/lib/Transforms/Utils/InlineFunction.cpp llvm/test/Transforms/Coroutines/ArgAddr.ll llvm/test/Transforms/Coroutines/coro-retcon-resume-values.ll llvm/test/Transforms/Coroutines/coro-retcon-value.ll llvm/test/Transforms/Coroutines/coro-retcon.ll llvm/test/Transforms/Coroutines/ex2.ll llvm/test/Transforms/Coroutines/ex3.ll llvm/test/Transforms/Coroutines/ex4.ll llvm/test/Transforms/Inline/launder.invariant.group.ll llvm/test/Transforms/Inline/noalias-calls-always.ll llvm/test/Transforms/Inline/noalias-calls.ll llvm/test/Transforms/Inline/noalias-cs.ll llvm/test/Transforms/Inline/noalias.ll llvm/test/Transforms/Inline/noalias2.ll llvm/test/Transforms/PhaseOrdering/inlining-alignment-assumptions.ll llvm/test/Transforms/PhaseOrdering/instcombine-sroa-inttoptr.ll llvm/test/Transforms/PhaseOrdering/pr39282.ll Removed: diff --git a/clang/test/CodeGen/aarch64-ls64.c b/clang/test/CodeGen/aarch64-ls64.c index 77e4b41fbd58..17fce0094ac9 100644 --- a/clang/test/CodeGen/aarch64-ls64.c +++ b/clang/test/CodeGen/aarch64-ls64.c @@ -21,6 +21,7 @@ uint64_t status; // CHECK-NEXT:[[__ADDR_ADDR_I:%.*]] = alloca i8*, align 8 // CHECK-NEXT:[[REF_TMP:%.*]] = alloca [[STRUCT_DATA512_T:%.*]], align 8 // CHECK-NEXT:[[TMP0:%.*]] = load i8*, i8** @addr, align 8 +// CHECK-NEXT:call void @llvm.experimental.noalias.scope.decl(metadata !6) // CHECK-NEXT:store i8* [[TMP0]], i8** [[__ADDR_ADDR_I]], align 8, !noalias !6 // CHECK-NEXT:[[TMP1:%.*]] = load i8*, i8** [[__ADDR_ADDR_I]], align 8, !noalias !6 // CHECK-NEXT:[[VAL_I:%.*]] = getelementptr inbounds [[STRUCT_DATA512_T]], %struct.data512_t* [[REF_TMP]], i32 0, i32 0 diff --git a/llvm/lib/Transforms/Utils/InlineFunction.cpp b/llvm/lib/Transforms/Utils/InlineFunction.cpp index 65bf6a3b32dc..abdd2b2361b7 100644 --- a/llvm/lib/Transforms/Utils/InlineFunction.cpp +++ b/llvm/lib/Transforms/Utils/InlineFunction.cpp @@ -79,6 +79,12 @@ EnableNoAliasConversion("enable-noalias-to-md-conversion", cl::init(true), cl::Hidden, cl::desc("Convert noalias attributes to metadata during inlining.")); +static cl::opt +UseNoAliasIntrinsic("use-noalias-intrinsic-during-inlining", cl::Hidden, +cl::ZeroOrMore, cl::init(true), +cl::desc("Use the llvm.experimental.noalias.scope.decl " + "intrinsic during inlining.")); + // Disabled by default, because the added alignment assumptions may increase // compile-time and block optimizations. This option is not suitable for use // with frontends that emit comprehensive parameter alignment annotations. @@ -821,91 +827,119 @@ static void PropagateCallSiteMetadata(CallBase , ValueToValueMapTy ) { } } -/// When inlining a function that contains noalias scope metadata, -/// this metadata needs to be cloned so that the inlined blocks -/// have diff erent "unique scopes" at every call site. Were this not done, then -/// aliasing scopes from a function inlined into a caller multiple times could -/// not be diff erentiated (and this would lead to miscompiles because the -/// non-aliasing property communicated by the metadata could have -/// call-site-specific control dependencies). -static void CloneAliasScopeMetadata(CallBase , ValueToValueMapTy ) { - const Function *CalledFunc = CB.getCalledFunction(); +/// Utility for cloning !noalias and !alias.scope metadata. When a code region +/// using scoped alias metadata is inlined, the aliasing relationships may not +/// hold between the two version. It is necessary to create a deep clone of the +/// metadata, putting the two versions in separate scope domains. +class ScopedAliasMetadataDeepCloner { + using MetadataMap = DenseMap; SetVector MD; - - // Note: We could only clone the metadata if it is already used in the - // caller. I'm omitting that check here because it might confuse - // inter-procedural alias analysis passes. We can revisit this if it becomes - // an efficiency or overhead problem. - - for (const BasicBlock : *CalledFunc) -for (const Instruction : I) { - if (const
[llvm-branch-commits] [llvm] d8ffaa9 - [NFC] cleanup noalias2.ll test
Author: Jeroen Dobbelaere Date: 2021-01-19T20:47:02+01:00 New Revision: d8ffaa9f7234d8bf40682763373ab060d14adf22 URL: https://github.com/llvm/llvm-project/commit/d8ffaa9f7234d8bf40682763373ab060d14adf22 DIFF: https://github.com/llvm/llvm-project/commit/d8ffaa9f7234d8bf40682763373ab060d14adf22.diff LOG: [NFC] cleanup noalias2.ll test D75825 and D75828 modified llvm/test/Transforms/Inline/noalias2.ll to handle llvm.assume. The checking though was broken. The NO_ASSUME has been replaced by a normal CHECK; the ASSUME rules were never triggered and have been removed. The test checks have been regenerated. Reviewed By: nikic Differential Revision: https://reviews.llvm.org/D94978 Added: Modified: llvm/test/Transforms/Inline/noalias2.ll Removed: diff --git a/llvm/test/Transforms/Inline/noalias2.ll b/llvm/test/Transforms/Inline/noalias2.ll index 59809379c2e5..8732cb538730 100644 --- a/llvm/test/Transforms/Inline/noalias2.ll +++ b/llvm/test/Transforms/Inline/noalias2.ll @@ -1,5 +1,5 @@ ; NOTE: Assertions have been autogenerated by utils/update_test_checks.py UTC_ARGS: --function-signature -; RUN: opt -inline -enable-noalias-to-md-conversion -S < %s | FileCheck %s --check-prefixes=CHECK,NO_ASSUME +; RUN: opt -inline -enable-noalias-to-md-conversion -S < %s | FileCheck %s ; RUN: opt -inline -enable-noalias-to-md-conversion --enable-knowledge-retention -S < %s | FileCheck %s target datalayout = "e-p:64:64:64-i1:8:8-i8:8:8-i16:16:16-i32:32:32-i64:64:64-f32:32:32-f64:64:64-v64:64:64-v128:128:128-a0:0:64-s0:64:64-f80:128:128-n8:16:32:64-S128" @@ -7,19 +7,13 @@ target triple = "x86_64-unknown-linux-gnu" define void @hello(float* noalias nocapture %a, float* noalias nocapture readonly %c) #0 { ; CHECK-LABEL: define {{[^@]+}}@hello -; CHECK-SAME: (float* noalias nocapture [[A:%.*]], float* noalias nocapture readonly [[C:%.*]]) #0 +; CHECK-SAME: (float* noalias nocapture [[A:%.*]], float* noalias nocapture readonly [[C:%.*]]) [[ATTR0:#.*]] { ; CHECK-NEXT: entry: ; CHECK-NEXT:[[TMP0:%.*]] = load float, float* [[C]], align 4 ; CHECK-NEXT:[[ARRAYIDX:%.*]] = getelementptr inbounds float, float* [[A]], i64 5 ; CHECK-NEXT:store float [[TMP0]], float* [[ARRAYIDX]], align 4 ; CHECK-NEXT:ret void ; -; ASSUME-LABEL: @hello( -; ASSUME-NEXT: entry: -; ASSUME-NEXT:[[TMP0:%.*]] = load float, float* [[C:%.*]], align 4 -; ASSUME-NEXT:[[ARRAYIDX:%.*]] = getelementptr inbounds float, float* [[A:%.*]], i64 5 -; ASSUME-NEXT:store float [[TMP0]], float* [[ARRAYIDX]], align 4 -; ASSUME-NEXT:ret void entry: %0 = load float, float* %c, align 4 %arrayidx = getelementptr inbounds float, float* %a, i64 5 @@ -29,7 +23,7 @@ entry: define void @foo(float* noalias nocapture %a, float* noalias nocapture readonly %c) #0 { ; CHECK-LABEL: define {{[^@]+}}@foo -; CHECK-SAME: (float* noalias nocapture [[A:%.*]], float* noalias nocapture readonly [[C:%.*]]) #0 +; CHECK-SAME: (float* noalias nocapture [[A:%.*]], float* noalias nocapture readonly [[C:%.*]]) [[ATTR0]] { ; CHECK-NEXT: entry: ; CHECK-NEXT:[[TMP0:%.*]] = load float, float* [[C]], align 4, !alias.scope !0, !noalias !3 ; CHECK-NEXT:[[ARRAYIDX_I:%.*]] = getelementptr inbounds float, float* [[A]], i64 5 @@ -39,16 +33,6 @@ define void @foo(float* noalias nocapture %a, float* noalias nocapture readonly ; CHECK-NEXT:store float [[TMP1]], float* [[ARRAYIDX]], align 4 ; CHECK-NEXT:ret void ; -; ASSUME-LABEL: @foo( -; ASSUME-NEXT: entry: -; ASSUME-NEXT:call void @llvm.assume(i1 true) [ "noalias"(float* [[A:%.*]]), "noalias"(float* [[C:%.*]]) ] -; ASSUME-NEXT:[[TMP0:%.*]] = load float, float* [[C]], align 4, !alias.scope !0, !noalias !3 -; ASSUME-NEXT:[[ARRAYIDX_I:%.*]] = getelementptr inbounds float, float* [[A]], i64 5 -; ASSUME-NEXT:store float [[TMP0]], float* [[ARRAYIDX_I]], align 4, !alias.scope !3, !noalias !0 -; ASSUME-NEXT:[[TMP1:%.*]] = load float, float* [[C]], align 4 -; ASSUME-NEXT:[[ARRAYIDX:%.*]] = getelementptr inbounds float, float* [[A]], i64 7 -; ASSUME-NEXT:store float [[TMP1]], float* [[ARRAYIDX]], align 4 -; ASSUME-NEXT:ret void entry: tail call void @hello(float* %a, float* %c) %0 = load float, float* %c, align 4 @@ -59,7 +43,7 @@ entry: define void @hello2(float* noalias nocapture %a, float* noalias nocapture %b, float* nocapture readonly %c) #0 { ; CHECK-LABEL: define {{[^@]+}}@hello2 -; CHECK-SAME: (float* noalias nocapture [[A:%.*]], float* noalias nocapture [[B:%.*]], float* nocapture readonly [[C:%.*]]) #0 +; CHECK-SAME: (float* noalias nocapture [[A:%.*]], float* noalias nocapture [[B:%.*]], float* nocapture readonly [[C:%.*]]) [[ATTR0]] { ; CHECK-NEXT: entry: ; CHECK-NEXT:[[TMP0:%.*]] = load float, float* [[C]], align 4 ; CHECK-NEXT:[[ARRAYIDX:%.*]] = getelementptr inbounds float, float* [[A]], i64 6 @@ -81,7
[llvm-branch-commits] [llvm] 121cac0 - [noalias.decl] Look through llvm.experimental.noalias.scope.decl
Author: Jeroen Dobbelaere Date: 2021-01-19T20:09:42+01:00 New Revision: 121cac01e8f8afe6ed2bb0b8ffe92f323776a716 URL: https://github.com/llvm/llvm-project/commit/121cac01e8f8afe6ed2bb0b8ffe92f323776a716 DIFF: https://github.com/llvm/llvm-project/commit/121cac01e8f8afe6ed2bb0b8ffe92f323776a716.diff LOG: [noalias.decl] Look through llvm.experimental.noalias.scope.decl Just like llvm.assume, there are a lot of cases where we can just ignore llvm.experimental.noalias.scope.decl. Reviewed By: nikic Differential Revision: https://reviews.llvm.org/D93042 Added: llvm/test/Analysis/BasicAA/noalias-scope-decl.ll llvm/test/Analysis/MemorySSA/noalias-scope-decl.ll llvm/test/Transforms/EarlyCSE/noalias-scope-decl.ll llvm/test/Transforms/LoopVectorize/noalias-scope-decl.ll Modified: llvm/include/llvm/Analysis/TargetTransformInfoImpl.h llvm/include/llvm/Analysis/VectorUtils.h llvm/lib/Analysis/AliasSetTracker.cpp llvm/lib/Analysis/MemorySSA.cpp llvm/lib/Analysis/ValueTracking.cpp llvm/lib/Analysis/VectorUtils.cpp llvm/lib/CodeGen/Analysis.cpp llvm/lib/Transforms/Scalar/EarlyCSE.cpp llvm/lib/Transforms/Vectorize/LoopVectorizationLegality.cpp llvm/lib/Transforms/Vectorize/LoopVectorize.cpp llvm/test/Analysis/AliasSet/intrinsics.ll llvm/test/Analysis/CostModel/X86/free-intrinsics.ll llvm/test/Analysis/CostModel/free-intrinsics-datalayout.ll llvm/test/Analysis/CostModel/free-intrinsics-no_info.ll Removed: diff --git a/llvm/include/llvm/Analysis/TargetTransformInfoImpl.h b/llvm/include/llvm/Analysis/TargetTransformInfoImpl.h index 47de99b02d97..1e9201430168 100644 --- a/llvm/include/llvm/Analysis/TargetTransformInfoImpl.h +++ b/llvm/include/llvm/Analysis/TargetTransformInfoImpl.h @@ -557,6 +557,7 @@ class TargetTransformInfoImplBase { case Intrinsic::is_constant: case Intrinsic::lifetime_start: case Intrinsic::lifetime_end: +case Intrinsic::experimental_noalias_scope_decl: case Intrinsic::objectsize: case Intrinsic::ptr_annotation: case Intrinsic::var_annotation: diff --git a/llvm/include/llvm/Analysis/VectorUtils.h b/llvm/include/llvm/Analysis/VectorUtils.h index ce3cb22dcd6e..26cb0e456ed4 100644 --- a/llvm/include/llvm/Analysis/VectorUtils.h +++ b/llvm/include/llvm/Analysis/VectorUtils.h @@ -304,7 +304,7 @@ typedef unsigned ID; /// the incoming type is void, we return void. If the EC represents a /// scalar, we return the scalar type. inline Type *ToVectorTy(Type *Scalar, ElementCount EC) { - if (Scalar->isVoidTy() || EC.isScalar()) + if (Scalar->isVoidTy() || Scalar->isMetadataTy() || EC.isScalar()) return Scalar; return VectorType::get(Scalar, EC); } diff --git a/llvm/lib/Analysis/AliasSetTracker.cpp b/llvm/lib/Analysis/AliasSetTracker.cpp index 9b21fbf4c9b7..486b4d99dfae 100644 --- a/llvm/lib/Analysis/AliasSetTracker.cpp +++ b/llvm/lib/Analysis/AliasSetTracker.cpp @@ -438,6 +438,7 @@ void AliasSetTracker::addUnknown(Instruction *Inst) { break; // FIXME: Add lifetime/invariant intrinsics (See: PR30807). case Intrinsic::assume: +case Intrinsic::experimental_noalias_scope_decl: case Intrinsic::sideeffect: case Intrinsic::pseudoprobe: return; diff --git a/llvm/lib/Analysis/MemorySSA.cpp b/llvm/lib/Analysis/MemorySSA.cpp index f0d27e0b2c6b..52dca7d378e1 100644 --- a/llvm/lib/Analysis/MemorySSA.cpp +++ b/llvm/lib/Analysis/MemorySSA.cpp @@ -285,6 +285,7 @@ instructionClobbersQuery(const MemoryDef *MD, const MemoryLocation , case Intrinsic::invariant_start: case Intrinsic::invariant_end: case Intrinsic::assume: +case Intrinsic::experimental_noalias_scope_decl: return {false, NoAlias}; case Intrinsic::dbg_addr: case Intrinsic::dbg_declare: @@ -1767,9 +1768,15 @@ MemoryUseOrDef *MemorySSA::createNewAccess(Instruction *I, // dependencies here. // FIXME: Replace this special casing with a more accurate modelling of // assume's control dependency. - if (IntrinsicInst *II = dyn_cast(I)) -if (II->getIntrinsicID() == Intrinsic::assume) + if (IntrinsicInst *II = dyn_cast(I)) { +switch (II->getIntrinsicID()) { +default: + break; +case Intrinsic::assume: +case Intrinsic::experimental_noalias_scope_decl: return nullptr; +} + } // Using a nonstandard AA pipelines might leave us with unexpected modref // results for I, so add a check to not model instructions that may not read diff --git a/llvm/lib/Analysis/ValueTracking.cpp b/llvm/lib/Analysis/ValueTracking.cpp index 7f8f101d42af..4f0c7057089b 100644 --- a/llvm/lib/Analysis/ValueTracking.cpp +++ b/llvm/lib/Analysis/ValueTracking.cpp @@ -536,6 +536,7 @@ bool llvm::isAssumeLikeIntrinsic(const Instruction *I) { case Intrinsic::invariant_end: case Intrinsic::lifetime_start: case Intrinsic::lifetime_end: +
[llvm-branch-commits] [llvm] 668827b - Introduce llvm.noalias.decl intrinsic
Author: Jeroen Dobbelaere Date: 2021-01-16T09:20:45+01:00 New Revision: 668827b6485664dbcf6caa2756fe2f6579ab1885 URL: https://github.com/llvm/llvm-project/commit/668827b6485664dbcf6caa2756fe2f6579ab1885 DIFF: https://github.com/llvm/llvm-project/commit/668827b6485664dbcf6caa2756fe2f6579ab1885.diff LOG: Introduce llvm.noalias.decl intrinsic The ``llvm.experimental.noalias.scope.decl`` intrinsic identifies where a noalias scope is declared. When the intrinsic is duplicated, a decision must also be made about the scope: depending on the reason of the duplication, the scope might need to be duplicated as well. Reviewed By: nikic, jdoerfert Differential Revision: https://reviews.llvm.org/D93039 Added: llvm/test/Verifier/noalias_scope_decl.ll Modified: llvm/docs/LangRef.rst llvm/include/llvm/IR/IRBuilder.h llvm/include/llvm/IR/Intrinsics.h llvm/include/llvm/IR/Intrinsics.td llvm/lib/CodeGen/IntrinsicLowering.cpp llvm/lib/CodeGen/SelectionDAG/FastISel.cpp llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp llvm/lib/IR/IRBuilder.cpp llvm/lib/IR/Verifier.cpp Removed: diff --git a/llvm/docs/LangRef.rst b/llvm/docs/LangRef.rst index a6f6e8281a72..ccf1feb420eb 100644 --- a/llvm/docs/LangRef.rst +++ b/llvm/docs/LangRef.rst @@ -19603,6 +19603,82 @@ Semantics: This function returns the same values as the libm ``trunc`` functions would and handles error conditions in the same way. +.. _int_experimental_noalias_scope_decl: + +'``llvm.experimental.noalias.scope.decl``' Intrinsic + + +Syntax: +""" + + +:: + + declare void @llvm.experimental.noalias.scope.decl(metadata !id.scope.list) + +Overview: +" + +The ``llvm.experimental.noalias.scope.decl`` intrinsic identifies where a +noalias scope is declared. When the intrinsic is duplicated, a decision must +also be made about the scope: depending on the reason of the duplication, +the scope might need to be duplicated as well. + + +Arguments: +"" + +The ``!id.scope.list`` argument is metadata that is a list of ``noalias`` +metadata references. The format is identical to that required for ``noalias`` +metadata. This list must have exactly one element. + +Semantics: +"" + +The ``llvm.experimental.noalias.scope.decl`` intrinsic identifies where a +noalias scope is declared. When the intrinsic is duplicated, a decision must +also be made about the scope: depending on the reason of the duplication, +the scope might need to be duplicated as well. + +For example, when the intrinsic is used inside a loop body, and that loop is +unrolled, the associated noalias scope must also be duplicated. Otherwise, the +noalias property it signifies would spill across loop iterations, whereas it +was only valid within a single iteration. + +.. code-block:: llvm + + ; This examples shows two possible positions for noalias.decl and how they impact the semantics: + ; If it is outside the loop (Version 1), then %a and %b are noalias across *all* iterations. + ; If it is inside the loop (Version 2), then %a and %b are noalias only within *one* iteration. + declare void @decl_in_loop(i8* %a.base, i8* %b.base) { + entry: +; call void @llvm.experimental.noalias.scope.decl(metadata !2) ; Version 1: noalias decl outside loop +br label %loop + + loop: +%a = phi i8* [ %a.base, %entry ], [ %a.inc, %loop ] +%b = phi i8* [ %b.base, %entry ], [ %b.inc, %loop ] +; call void @llvm.experimental.noalias.scope.decl(metadata !2) ; Version 2: noalias decl inside loop +%val = load i8, i8* %a, !alias.scope !2 +store i8 %val, i8* %b, !noalias !2 +%a.inc = getelementptr inbounds i8, i8* %a, i64 1 +%b.inc = getelementptr inbounds i8, i8* %b, i64 1 +%cond = call i1 @cond() +br i1 %cond, label %loop, label %exit + + exit: +ret void + } + + !0 = !{!0} ; domain + !1 = !{!1, !0} ; scope + !2 = !{!1} ; scope list + +Multiple calls to `@llvm.experimental.noalias.scope.decl` for the same scope +are possible, but one should never dominate another. Violations are pointed out +by the verifier as they indicate a problem in either a transformation pass or +the input. + Floating Point Environment Manipulation intrinsics -- diff --git a/llvm/include/llvm/IR/IRBuilder.h b/llvm/include/llvm/IR/IRBuilder.h index c9074abe88c2..9cefc9aa764c 100644 --- a/llvm/include/llvm/IR/IRBuilder.h +++ b/llvm/include/llvm/IR/IRBuilder.h @@ -858,6 +858,13 @@ class IRBuilderBase { CallInst *CreateAssumption(Value *Cond, ArrayRef OpBundles = llvm::None); + /// Create a llvm.experimental.noalias.scope.decl intrinsic call. + Instruction *CreateNoAliasScopeDeclaration(Value *Scope); + Instruction *CreateNoAliasScopeDeclaration(MDNode *ScopeTag) { +return
[llvm-branch-commits] [llvm] bb72adc - [NFC] Use correct ssa.copy spelling when referring to the intrinsic
Author: Jeroen Dobbelaere Date: 2021-01-13T20:43:14+01:00 New Revision: bb72adcaee7db0877e1cecb29d414003bf19ce02 URL: https://github.com/llvm/llvm-project/commit/bb72adcaee7db0877e1cecb29d414003bf19ce02 DIFF: https://github.com/llvm/llvm-project/commit/bb72adcaee7db0877e1cecb29d414003bf19ce02.diff LOG: [NFC] Use correct ssa.copy spelling when referring to the intrinsic Split out from D91250. Fixes wrong ssa_copy naming. Reviewed By: fhahn Differential Revision: https://reviews.llvm.org/D94310 Added: Modified: llvm/docs/LangRef.rst Removed: diff --git a/llvm/docs/LangRef.rst b/llvm/docs/LangRef.rst index 854c72191da2..5a4c652c3603 100644 --- a/llvm/docs/LangRef.rst +++ b/llvm/docs/LangRef.rst @@ -20063,7 +20063,7 @@ optimizer. .. _int_ssa_copy: -'``llvm.ssa_copy``' Intrinsic +'``llvm.ssa.copy``' Intrinsic ^^^ Syntax: @@ -20071,7 +20071,7 @@ Syntax: :: - declare type @llvm.ssa_copy(type %operand) returned(1) readnone + declare type @llvm.ssa.copy(type %operand) returned(1) readnone Arguments: "" @@ -20081,7 +20081,7 @@ The first argument is an operand which is used as the returned value. Overview: "" -The ``llvm.ssa_copy`` intrinsic can be used to attach information to +The ``llvm.ssa.copy`` intrinsic can be used to attach information to operations by copying them and giving them new names. For example, the PredicateInfo utility uses it to build Extended SSA form, and attach various forms of information to operands that dominate specific ___ llvm-branch-commits mailing list llvm-branch-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits