Re: r367829 - [CrossTU][NFCI] Refactor loadExternalAST function
There was yet another uninit read:
Testing: 0 .. 10.. 20.. 30.. 40.. 50.. 60.. 70.. 80..
FAIL: Clang-Unit ::
CrossTU/./CrossTUTests/CrossTranslationUnit.RespectsLoadThreshold (13891 of
15323)
TEST 'Clang-Unit ::
CrossTU/./CrossTUTests/CrossTranslationUnit.RespectsLoadThreshold' FAILED
Note: Google Test filter = CrossTranslationUnit.RespectsLoadThreshold
[==] Running 1 test from 1 test case.
[--] Global test environment set-up.
[--] 1 test from CrossTranslationUnit
[ RUN ] CrossTranslationUnit.RespectsLoadThreshold
==8561==WARNING: MemorySanitizer: use-of-uninitialized-value
#0 0x1010708 in ~LoadGuard
/b/sanitizer-x86_64-linux-bootstrap-msan/build/llvm/tools/clang/include/clang/CrossTU/CrossTranslationUnit.h:288:11
#1 0x1010708 in
clang::cross_tu::CrossTranslationUnitContext::loadExternalAST(llvm::StringRef,
llvm::StringRef, llvm::StringRef, bool)
/b/sanitizer-x86_64-linux-bootstrap-msan/build/llvm/tools/clang/lib/CrossTU/CrossTranslationUnit.cpp:499:1
#2 0x100babc in llvm::Expected
clang::cross_tu::CrossTranslationUnitContext::getCrossTUDefinitionImpl(clang::FunctionDecl
const*, llvm::StringRef, llvm::StringRef, bool)
/b/sanitizer-x86_64-linux-bootstrap-msan/build/llvm/tools/clang/lib/CrossTU/CrossTranslationUnit.cpp:241:7
#3 0x100b476 in
clang::cross_tu::CrossTranslationUnitContext::getCrossTUDefinition(clang::FunctionDecl
const*, llvm::StringRef, llvm::StringRef, bool)
/b/sanitizer-x86_64-linux-bootstrap-msan/build/llvm/tools/clang/lib/CrossTU/CrossTranslationUnit.cpp:307:10
I tried fixing that one in r367912. Please verify that that's what you
meant as well.
On Mon, Aug 5, 2019 at 11:22 AM Nico Weber wrote:
> The problem is probably this part from the diff:
>
> - unsigned NumASTLoaded{0u};
> +
> + /// The number successfully loaded ASTs. Used to indicate, and - with
> the
> + /// appropriate threshold value - limit the memory usage of the
> + /// CrossTranslationUnitContext.
> + unsigned NumASTLoaded;
>
>
> i.e. you removed the initialization of NumASTLoaded. Was there a reason
> for that? I've put it back for now in r367875, but please verify that the
> code now does what you intended it to do – maybe you were planning to
> initialize this somewhere else.
>
> On Mon, Aug 5, 2019 at 10:03 AM Nico Weber wrote:
>
>> The msan bot doesn't like this, it reports an uninitialized read a
>> t clang/lib/CrossTU/CrossTranslationUnit.cpp :
>>
>>
>> http://lab.llvm.org:8011/builders/sanitizer-x86_64-linux-fast/builds/34087/steps/check-clang%20msan/logs/stdio
>>
>>
>> Testing: 0
>> FAIL: Clang :: Analysis/ctu-unknown-parts-in-triples.cpp (492 of 15321)
>> TEST 'Clang ::
>> Analysis/ctu-unknown-parts-in-triples.cpp' FAILED
>> Script:
>> --
>> : 'RUN: at line 4'; rm -rf
>> /b/sanitizer-x86_64-linux-fast/build/llvm_build_msan/tools/clang/test/Analysis/Output/ctu-unknown-parts-in-triples.cpp.tmp
>> && mkdir
>> /b/sanitizer-x86_64-linux-fast/build/llvm_build_msan/tools/clang/test/Analysis/Output/ctu-unknown-parts-in-triples.cpp.tmp
>> : 'RUN: at line 5'; mkdir -p
>> /b/sanitizer-x86_64-linux-fast/build/llvm_build_msan/tools/clang/test/Analysis/Output/ctu-unknown-parts-in-triples.cpp.tmp/ctudir
>> : 'RUN: at line 6';
>> /b/sanitizer-x86_64-linux-fast/build/llvm_build_msan/bin/clang -cc1
>> -internal-isystem
>> /b/sanitizer-x86_64-linux-fast/build/llvm_build_msan/lib/clang/10.0.0/include
>> -nostdsysteminc -triple x86_64-pc-linux-gnu-emit-pch -o
>> /b/sanitizer-x86_64-linux-fast/build/llvm_build_msan/tools/clang/test/Analysis/Output/ctu-unknown-parts-in-triples.cpp.tmp/ctudir/ctu-other.cpp.ast
>> /b/sanitizer-x86_64-linux-fast/build/llvm/tools/clang/test/Analysis/Inputs/ctu-other.cpp
>> : 'RUN: at line 8'; cp
>> /b/sanitizer-x86_64-linux-fast/build/llvm/tools/clang/test/Analysis/Inputs/ctu-other.cpp.externalDefMap.txt
>> /b/sanitizer-x86_64-linux-fast/build/llvm_build_msan/tools/clang/test/Analysis/Output/ctu-unknown-parts-in-triples.cpp.tmp/ctudir/externalDefMap.txt
>> : 'RUN: at line 9';
>> /b/sanitizer-x86_64-linux-fast/build/llvm_build_msan/bin/clang -cc1
>> -internal-isystem
>> /b/sanitizer-x86_64-linux-fast/build/llvm_build_msan/lib/clang/10.0.0/include
>> -nostdsysteminc -analyze -analyzer-constraints=range -triple
>> x86_64-unknown-linux-gnu-analyzer-checker=core,debug.ExprInspection
>> -analyzer-config experimental-enable-naive-ctu-analysis=true
>> -analyzer-config
>> ctu-dir=/b/sanitizer-x86_64-linux-fast/build/llvm_build_msan/tools/clang/test/Analysis/Output/ctu-unknown-parts-in-triples.cpp.tmp/ctudir
>>-Werror=ctu-verify
>> /b/sanitizer-x86_64-linux-fast/build/llvm/tools/clang/test/Analysis/ctu-unknown-parts-in-triples.cpp
>> --
>> Exit Code: 77
>>
>> Command Output (stderr):
>> --
>> ==5072==WARNING: MemorySanitizer: use-of-uninitialized-value
>> #0 0xb05c3c4 in
>>
Re: r367829 - [CrossTU][NFCI] Refactor loadExternalAST function
The problem is probably this part from the diff:
- unsigned NumASTLoaded{0u};
+
+ /// The number successfully loaded ASTs. Used to indicate, and - with
the
+ /// appropriate threshold value - limit the memory usage of the
+ /// CrossTranslationUnitContext.
+ unsigned NumASTLoaded;
i.e. you removed the initialization of NumASTLoaded. Was there a reason for
that? I've put it back for now in r367875, but please verify that the code
now does what you intended it to do – maybe you were planning to initialize
this somewhere else.
On Mon, Aug 5, 2019 at 10:03 AM Nico Weber wrote:
> The msan bot doesn't like this, it reports an uninitialized read a
> t clang/lib/CrossTU/CrossTranslationUnit.cpp :
>
>
> http://lab.llvm.org:8011/builders/sanitizer-x86_64-linux-fast/builds/34087/steps/check-clang%20msan/logs/stdio
>
>
> Testing: 0
> FAIL: Clang :: Analysis/ctu-unknown-parts-in-triples.cpp (492 of 15321)
> TEST 'Clang ::
> Analysis/ctu-unknown-parts-in-triples.cpp' FAILED
> Script:
> --
> : 'RUN: at line 4'; rm -rf
> /b/sanitizer-x86_64-linux-fast/build/llvm_build_msan/tools/clang/test/Analysis/Output/ctu-unknown-parts-in-triples.cpp.tmp
> && mkdir
> /b/sanitizer-x86_64-linux-fast/build/llvm_build_msan/tools/clang/test/Analysis/Output/ctu-unknown-parts-in-triples.cpp.tmp
> : 'RUN: at line 5'; mkdir -p
> /b/sanitizer-x86_64-linux-fast/build/llvm_build_msan/tools/clang/test/Analysis/Output/ctu-unknown-parts-in-triples.cpp.tmp/ctudir
> : 'RUN: at line 6';
> /b/sanitizer-x86_64-linux-fast/build/llvm_build_msan/bin/clang -cc1
> -internal-isystem
> /b/sanitizer-x86_64-linux-fast/build/llvm_build_msan/lib/clang/10.0.0/include
> -nostdsysteminc -triple x86_64-pc-linux-gnu-emit-pch -o
> /b/sanitizer-x86_64-linux-fast/build/llvm_build_msan/tools/clang/test/Analysis/Output/ctu-unknown-parts-in-triples.cpp.tmp/ctudir/ctu-other.cpp.ast
> /b/sanitizer-x86_64-linux-fast/build/llvm/tools/clang/test/Analysis/Inputs/ctu-other.cpp
> : 'RUN: at line 8'; cp
> /b/sanitizer-x86_64-linux-fast/build/llvm/tools/clang/test/Analysis/Inputs/ctu-other.cpp.externalDefMap.txt
> /b/sanitizer-x86_64-linux-fast/build/llvm_build_msan/tools/clang/test/Analysis/Output/ctu-unknown-parts-in-triples.cpp.tmp/ctudir/externalDefMap.txt
> : 'RUN: at line 9';
> /b/sanitizer-x86_64-linux-fast/build/llvm_build_msan/bin/clang -cc1
> -internal-isystem
> /b/sanitizer-x86_64-linux-fast/build/llvm_build_msan/lib/clang/10.0.0/include
> -nostdsysteminc -analyze -analyzer-constraints=range -triple
> x86_64-unknown-linux-gnu-analyzer-checker=core,debug.ExprInspection
> -analyzer-config experimental-enable-naive-ctu-analysis=true
> -analyzer-config
> ctu-dir=/b/sanitizer-x86_64-linux-fast/build/llvm_build_msan/tools/clang/test/Analysis/Output/ctu-unknown-parts-in-triples.cpp.tmp/ctudir
>-Werror=ctu-verify
> /b/sanitizer-x86_64-linux-fast/build/llvm/tools/clang/test/Analysis/ctu-unknown-parts-in-triples.cpp
> --
> Exit Code: 77
>
> Command Output (stderr):
> --
> ==5072==WARNING: MemorySanitizer: use-of-uninitialized-value
> #0 0xb05c3c4 in
> clang::cross_tu::CrossTranslationUnitContext::loadExternalAST(llvm::StringRef,
> llvm::StringRef, llvm::StringRef, bool)
> /b/sanitizer-x86_64-linux-fast/build/llvm/tools/clang/lib/CrossTU/CrossTranslationUnit.cpp:467:7
> #1 0xb053a98 in llvm::Expected
> clang::cross_tu::CrossTranslationUnitContext::getCrossTUDefinitionImpl(clang::FunctionDecl
> const*, llvm::StringRef, llvm::StringRef, bool)
> /b/sanitizer-x86_64-linux-fast/build/llvm/tools/clang/lib/CrossTU/CrossTranslationUnit.cpp:241:7
> #2 0xb053466 in
> clang::cross_tu::CrossTranslationUnitContext::getCrossTUDefinition(clang::FunctionDecl
> const*, llvm::StringRef, llvm::StringRef, bool)
> /b/sanitizer-x86_64-linux-fast/build/llvm/tools/clang/lib/CrossTU/CrossTranslationUnit.cpp:307:10
> #3 0xadb69f5 in clang::ento::AnyFunctionCall::getRuntimeDefinition()
> const
> /b/sanitizer-x86_64-linux-fast/build/llvm/tools/clang/lib/StaticAnalyzer/Core/CallEvent.cpp:575:14
>
> On Mon, Aug 5, 2019 at 7:05 AM Endre Fulop via cfe-commits <
> cfe-commits@lists.llvm.org> wrote:
>
>> Author: gamesh411
>> Date: Mon Aug 5 04:06:41 2019
>> New Revision: 367829
>>
>> URL: http://llvm.org/viewvc/llvm-project?rev=367829=rev
>> Log:
>> [CrossTU][NFCI] Refactor loadExternalAST function
>>
>> Summary:
>> Refactor loadExternalAST method of CrossTranslationUnitContext in order to
>> reduce maintenance burden and so that features are easier to add in the
>> future.
>>
>> Reviewers: martong
>>
>> Subscribers: rnkovacs, dkrupp, Szelethus, cfe-commits
>>
>> Tags: #clang
>>
>> Differential Revision: https://reviews.llvm.org/D64753
>>
>> Modified:
>> cfe/trunk/include/clang/CrossTU/CrossTranslationUnit.h
>> cfe/trunk/lib/CrossTU/CrossTranslationUnit.cpp
>>
>> Modified: cfe/trunk/include/clang/CrossTU/CrossTranslationUnit.h
>> URL:
>>
Re: r367829 - [CrossTU][NFCI] Refactor loadExternalAST function
The msan bot doesn't like this, it reports an uninitialized read a t clang/lib/CrossTU/CrossTranslationUnit.cpp : http://lab.llvm.org:8011/builders/sanitizer-x86_64-linux-fast/builds/34087/steps/check-clang%20msan/logs/stdio Testing: 0 FAIL: Clang :: Analysis/ctu-unknown-parts-in-triples.cpp (492 of 15321) TEST 'Clang :: Analysis/ctu-unknown-parts-in-triples.cpp' FAILED Script: -- : 'RUN: at line 4'; rm -rf /b/sanitizer-x86_64-linux-fast/build/llvm_build_msan/tools/clang/test/Analysis/Output/ctu-unknown-parts-in-triples.cpp.tmp && mkdir /b/sanitizer-x86_64-linux-fast/build/llvm_build_msan/tools/clang/test/Analysis/Output/ctu-unknown-parts-in-triples.cpp.tmp : 'RUN: at line 5'; mkdir -p /b/sanitizer-x86_64-linux-fast/build/llvm_build_msan/tools/clang/test/Analysis/Output/ctu-unknown-parts-in-triples.cpp.tmp/ctudir : 'RUN: at line 6'; /b/sanitizer-x86_64-linux-fast/build/llvm_build_msan/bin/clang -cc1 -internal-isystem /b/sanitizer-x86_64-linux-fast/build/llvm_build_msan/lib/clang/10.0.0/include -nostdsysteminc -triple x86_64-pc-linux-gnu-emit-pch -o /b/sanitizer-x86_64-linux-fast/build/llvm_build_msan/tools/clang/test/Analysis/Output/ctu-unknown-parts-in-triples.cpp.tmp/ctudir/ctu-other.cpp.ast /b/sanitizer-x86_64-linux-fast/build/llvm/tools/clang/test/Analysis/Inputs/ctu-other.cpp : 'RUN: at line 8'; cp /b/sanitizer-x86_64-linux-fast/build/llvm/tools/clang/test/Analysis/Inputs/ctu-other.cpp.externalDefMap.txt /b/sanitizer-x86_64-linux-fast/build/llvm_build_msan/tools/clang/test/Analysis/Output/ctu-unknown-parts-in-triples.cpp.tmp/ctudir/externalDefMap.txt : 'RUN: at line 9'; /b/sanitizer-x86_64-linux-fast/build/llvm_build_msan/bin/clang -cc1 -internal-isystem /b/sanitizer-x86_64-linux-fast/build/llvm_build_msan/lib/clang/10.0.0/include -nostdsysteminc -analyze -analyzer-constraints=range -triple x86_64-unknown-linux-gnu-analyzer-checker=core,debug.ExprInspection -analyzer-config experimental-enable-naive-ctu-analysis=true -analyzer-config ctu-dir=/b/sanitizer-x86_64-linux-fast/build/llvm_build_msan/tools/clang/test/Analysis/Output/ctu-unknown-parts-in-triples.cpp.tmp/ctudir -Werror=ctu-verify /b/sanitizer-x86_64-linux-fast/build/llvm/tools/clang/test/Analysis/ctu-unknown-parts-in-triples.cpp -- Exit Code: 77 Command Output (stderr): -- ==5072==WARNING: MemorySanitizer: use-of-uninitialized-value #0 0xb05c3c4 in clang::cross_tu::CrossTranslationUnitContext::loadExternalAST(llvm::StringRef, llvm::StringRef, llvm::StringRef, bool) /b/sanitizer-x86_64-linux-fast/build/llvm/tools/clang/lib/CrossTU/CrossTranslationUnit.cpp:467:7 #1 0xb053a98 in llvm::Expected clang::cross_tu::CrossTranslationUnitContext::getCrossTUDefinitionImpl(clang::FunctionDecl const*, llvm::StringRef, llvm::StringRef, bool) /b/sanitizer-x86_64-linux-fast/build/llvm/tools/clang/lib/CrossTU/CrossTranslationUnit.cpp:241:7 #2 0xb053466 in clang::cross_tu::CrossTranslationUnitContext::getCrossTUDefinition(clang::FunctionDecl const*, llvm::StringRef, llvm::StringRef, bool) /b/sanitizer-x86_64-linux-fast/build/llvm/tools/clang/lib/CrossTU/CrossTranslationUnit.cpp:307:10 #3 0xadb69f5 in clang::ento::AnyFunctionCall::getRuntimeDefinition() const /b/sanitizer-x86_64-linux-fast/build/llvm/tools/clang/lib/StaticAnalyzer/Core/CallEvent.cpp:575:14 On Mon, Aug 5, 2019 at 7:05 AM Endre Fulop via cfe-commits < cfe-commits@lists.llvm.org> wrote: > Author: gamesh411 > Date: Mon Aug 5 04:06:41 2019 > New Revision: 367829 > > URL: http://llvm.org/viewvc/llvm-project?rev=367829=rev > Log: > [CrossTU][NFCI] Refactor loadExternalAST function > > Summary: > Refactor loadExternalAST method of CrossTranslationUnitContext in order to > reduce maintenance burden and so that features are easier to add in the > future. > > Reviewers: martong > > Subscribers: rnkovacs, dkrupp, Szelethus, cfe-commits > > Tags: #clang > > Differential Revision: https://reviews.llvm.org/D64753 > > Modified: > cfe/trunk/include/clang/CrossTU/CrossTranslationUnit.h > cfe/trunk/lib/CrossTU/CrossTranslationUnit.cpp > > Modified: cfe/trunk/include/clang/CrossTU/CrossTranslationUnit.h > URL: > http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/CrossTU/CrossTranslationUnit.h?rev=367829=367828=367829=diff > > == > --- cfe/trunk/include/clang/CrossTU/CrossTranslationUnit.h (original) > +++ cfe/trunk/include/clang/CrossTU/CrossTranslationUnit.h Mon Aug 5 > 04:06:41 2019 > @@ -192,11 +192,11 @@ private: >template >llvm::Expected importDefinitionImpl(const T *D, ASTUnit > *Unit); > > - llvm::StringMap> FileASTUnitMap; > - llvm::StringMap NameASTUnitMap; > - llvm::StringMap NameFileMap; > - llvm::DenseMap> > - ASTUnitImporterMap; > + using ImporterMapTy = > + llvm::DenseMap>; > + > + ImporterMapTy ASTUnitImporterMap; > + >CompilerInstance >ASTContext >std::shared_ptr
