[PATCH] D33045: [libclang] Avoid more stats than necessary for reparse.
nik abandoned this revision. nik added a comment. I do not have time to work on this right now, Abandoning. https://reviews.llvm.org/D33045 ___ cfe-commits mailing list cfe-commits@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[PATCH] D33045: [libclang] Avoid more stats than necessary for reparse.
nik updated this revision to Diff 99740.
https://reviews.llvm.org/D33045
Files:
include/clang/Frontend/ASTUnit.h
lib/Frontend/ASTUnit.cpp
tools/libclang/CIndexCodeCompletion.cpp
Index: tools/libclang/CIndexCodeCompletion.cpp
===
--- tools/libclang/CIndexCodeCompletion.cpp
+++ tools/libclang/CIndexCodeCompletion.cpp
@@ -680,6 +680,7 @@
}
// Parse the resulting source file to find code-completion results.
+ AST->recreateFileManager();
AllocatedCXCodeCompleteResults *Results = new AllocatedCXCodeCompleteResults(
>getFileManager());
Results->Results = nullptr;
Index: lib/Frontend/ASTUnit.cpp
===
--- lib/Frontend/ASTUnit.cpp
+++ lib/Frontend/ASTUnit.cpp
@@ -241,6 +241,10 @@
this->PP = std::move(PP);
}
+void ASTUnit::recreateFileManager() {
+FileMgr = new FileManager(FileMgr->getFileSystemOpts());
+}
+
/// \brief Determine the set of code-completion contexts in which this
/// declaration should be shown.
static unsigned getDeclShowContexts(const NamedDecl *ND,
@@ -1311,6 +1315,10 @@
/// this routine will determine if it is still valid and, if so, avoid
/// rebuilding the precompiled preamble.
///
+/// The caller is required to recreate the FileMgr before calling this routine
+/// to ensure a clean stat cache and on the other hand that the stats done here
+/// can be reused for e.g. Reparse().
+///
/// \param AllowRebuild When true (the default), this routine is
/// allowed to rebuild the precompiled preamble if it is found to be
/// out-of-date.
@@ -1368,59 +1376,58 @@
if (AnyFileChanged)
break;
-vfs::Status Status;
-if (FileMgr->getNoncachedStatValue(R.second, Status)) {
+const FileEntry *fileEntry = FileMgr->getFile(R.second);
+if (!fileEntry) {
// If we can't stat the file we're remapping to, assume that something
// horrible happened.
AnyFileChanged = true;
break;
}
-OverriddenFiles[Status.getUniqueID()] = PreambleFileHash::createForFile(
-Status.getSize(),
-llvm::sys::toTimeT(Status.getLastModificationTime()));
+OverriddenFiles[fileEntry->getUniqueID()] = PreambleFileHash::createForFile(
+fileEntry->getSize(),
+fileEntry->getModificationTime());
}
for (const auto : PreprocessorOpts.RemappedFileBuffers) {
if (AnyFileChanged)
break;
-vfs::Status Status;
-if (FileMgr->getNoncachedStatValue(RB.first, Status)) {
+const FileEntry *fileEntry = FileMgr->getFile(RB.first);
+if (!fileEntry) {
AnyFileChanged = true;
break;
}
-OverriddenFiles[Status.getUniqueID()] =
+OverriddenFiles[fileEntry->getUniqueID()] =
PreambleFileHash::createForMemoryBuffer(RB.second);
}
// Check whether anything has changed.
for (llvm::StringMap::iterator
F = FilesInPreamble.begin(), FEnd = FilesInPreamble.end();
!AnyFileChanged && F != FEnd;
++F) {
-vfs::Status Status;
-if (FileMgr->getNoncachedStatValue(F->first(), Status)) {
+const FileEntry *fileEntry = FileMgr->getFile(F->first());
+if (!fileEntry) {
// If we can't stat the file, assume that something horrible happened.
AnyFileChanged = true;
break;
}
std::map::iterator Overridden
- = OverriddenFiles.find(Status.getUniqueID());
+ = OverriddenFiles.find(fileEntry->getUniqueID());
if (Overridden != OverriddenFiles.end()) {
// This file was remapped; check whether the newly-mapped file
// matches up with the previous mapping.
if (Overridden->second != F->second)
AnyFileChanged = true;
continue;
}
-
+
// The file was not remapped; check whether it has changed on disk.
-if (Status.getSize() != uint64_t(F->second.Size) ||
-llvm::sys::toTimeT(Status.getLastModificationTime()) !=
-F->second.ModTime)
+if (fileEntry->getSize() != F->second.Size ||
+fileEntry->getModificationTime() != F->second.ModTime)
AnyFileChanged = true;
}
@@ -1873,6 +1880,8 @@
getDiagnostics().Reset();
ProcessWarningOptions(getDiagnostics(), Invocation->getDiagnosticOpts());
+ recreateFileManager();
+
std::unique_ptr OverrideMainBuffer;
if (PrecompilePreambleAfterNParses > 0) {
PreambleRebuildCounter = PrecompilePreambleAfterNParses;
@@ -2040,15 +2049,16 @@
RemappedFile.second);
}
+ recreateFileManager();
+
// If we have a preamble file lying around, or if we might try to
//
[PATCH] D33045: [libclang] Avoid more stats than necessary for reparse.
nik added a comment. In https://reviews.llvm.org/D33045#759436, @ilya-biryukov wrote: > Are there any other callers to getMainBufferWithPrecompiledPreamble? Yes. Huch, right... don't know why I didn't adapted those. Done now. > Maybe they cause LibclangReparseTest.ReparseWithModule to fail? So it looks like. Actually that one become flaky now and the other failing ones are: Failing Tests (74): Clang-Unit :: AST/ASTTests/ImportExpr.ImportAtomicExpr Clang-Unit :: AST/ASTTests/ImportExpr.ImportBinaryConditionalOperator Clang-Unit :: AST/ASTTests/ImportExpr.ImportCXXNullPtrLiteralExpr Clang-Unit :: AST/ASTTests/ImportExpr.ImportCXXThisExpr Clang-Unit :: AST/ASTTests/ImportExpr.ImportCompoundLiteralExpr Clang-Unit :: AST/ASTTests/ImportExpr.ImportConditionalOperator Clang-Unit :: AST/ASTTests/ImportExpr.ImportDesignatedInitExpr Clang-Unit :: AST/ASTTests/ImportExpr.ImportFloatinglLiteralExpr Clang-Unit :: AST/ASTTests/ImportExpr.ImportGNUNullExpr Clang-Unit :: AST/ASTTests/ImportExpr.ImportInitListExpr Clang-Unit :: AST/ASTTests/ImportExpr.ImportLabelDeclAndAddrLabelExpr Clang-Unit :: AST/ASTTests/ImportExpr.ImportParenListExpr Clang-Unit :: AST/ASTTests/ImportExpr.ImportPredefinedExpr Clang-Unit :: AST/ASTTests/ImportExpr.ImportStmtExpr Clang-Unit :: AST/ASTTests/ImportExpr.ImportStringLiteral Clang-Unit :: AST/ASTTests/ImportExpr.ImportVAArgExpr Clang-Unit :: AST/ASTTests/ImportType.ImportAtomicType Clang-Unit :: AST/ASTTests/RecursiveASTVisitor.NoPostOrderTraversal Clang-Unit :: AST/ASTTests/RecursiveASTVisitor.PostOrderTraversal Clang-Unit :: ASTMatchers/ASTMatchersTests/AstMatcherPMacro.Works Clang-Unit :: ASTMatchers/ASTMatchersTests/AstPolymorphicMatcherPMacro.Works Clang-Unit :: ASTMatchers/ASTMatchersTests/EachOf.BehavesLikeAnyOfUnlessBothMatch Clang-Unit :: ASTMatchers/ASTMatchersTests/EachOf.TriggersForEachMatch Clang-Unit :: ASTMatchers/ASTMatchersTests/EqualsBoundNodeMatcher.FiltersMatchedCombinations Clang-Unit :: ASTMatchers/ASTMatchersTests/EqualsBoundNodeMatcher.UnlessDescendantsOfAncestorsMatch Clang-Unit :: ASTMatchers/ASTMatchersTests/EqualsBoundNodeMatcher.UsingForEachDescendant Clang-Unit :: ASTMatchers/ASTMatchersTests/FindAll.BindsDescendantNodeOnMatch Clang-Unit :: ASTMatchers/ASTMatchersTests/FindAll.BindsNodeAndDescendantNodesOnOneMatch Clang-Unit :: ASTMatchers/ASTMatchersTests/FindAll.BindsNodeOnMatch Clang-Unit :: ASTMatchers/ASTMatchersTests/ForEach.BindsMultipleNodes Clang-Unit :: ASTMatchers/ASTMatchersTests/ForEach.BindsOneNode Clang-Unit :: ASTMatchers/ASTMatchersTests/ForEach.BindsRecursiveCombinations Clang-Unit :: ASTMatchers/ASTMatchersTests/ForEachArgumentWithParam.HandlesBoundNodesForNonMatches Clang-Unit :: ASTMatchers/ASTMatchersTests/ForEachArgumentWithParam.MatchesCXXMemberCallExpr Clang-Unit :: ASTMatchers/ASTMatchersTests/ForEachArgumentWithParam.MatchesCallExpr Clang-Unit :: ASTMatchers/ASTMatchersTests/ForEachArgumentWithParam.MatchesConstructExpr Clang-Unit :: ASTMatchers/ASTMatchersTests/ForEachDescendant.BindsCombinations Clang-Unit :: ASTMatchers/ASTMatchersTests/ForEachDescendant.BindsCorrectNodes Clang-Unit :: ASTMatchers/ASTMatchersTests/ForEachDescendant.BindsMultipleNodes Clang-Unit :: ASTMatchers/ASTMatchersTests/ForEachDescendant.BindsOneNode Clang-Unit :: ASTMatchers/ASTMatchersTests/ForEachDescendant.BindsRecursiveCombinations Clang-Unit :: ASTMatchers/ASTMatchersTests/ForEachDescendant.NestedForEachDescendant Clang-Unit :: ASTMatchers/ASTMatchersTests/Has.DoesNotDeleteBindings Clang-Unit :: ASTMatchers/ASTMatchersTests/Has.MatchesChildrenOfTypes Clang-Unit :: ASTMatchers/ASTMatchersTests/HasAncestor.BindsCombinationsWithHasDescendant Clang-Unit :: ASTMatchers/ASTMatchersTests/HasAncestor.BindsRecursiveCombinations Clang-Unit :: ASTMatchers/ASTMatchersTests/HasAncestor.MatchesClosestAncestor Clang-Unit :: ASTMatchers/ASTMatchersTests/HasDescendant.MatchesDescendantTypes Clang-Unit :: ASTMatchers/ASTMatchersTests/HasDescendant.MatchesDescendantsOfTypes Clang-Unit :: ASTMatchers/ASTMatchersTests/IsEqualTo.MatchesNodesByIdentity Clang-Unit :: ASTMatchers/ASTMatchersTests/LoopingMatchers.DoNotOverwritePreviousMatchResultOnFailure Clang-Unit :: ASTMatchers/ASTMatchersTests/MatchFinder.CanMatchDeclarationsRecursively Clang-Unit :: ASTMatchers/ASTMatchersTests/MatchFinder.CanMatchSingleNodesRecursively Clang-Unit :: ASTMatchers/ASTMatchersTests/MatchFinder.CanMatchStatementsRecursively Clang-Unit :: ASTMatchers/ASTMatchersTests/MatchFinder.InterceptsEndOfTranslationUnit Clang-Unit :: ASTMatchers/ASTMatchersTests/MatchFinder.InterceptsStartOfTranslationUnit Clang-Unit ::
[PATCH] D33045: [libclang] Avoid more stats than necessary for reparse.
ilya-biryukov added a comment.
Sorry, didn't get a chance to look into it very thoroughly yet.
Are there any other callers to getMainBufferWithPrecompiledPreamble?
Maybe they cause LibclangReparseTest.ReparseWithModule to fail?
Comment at: lib/Frontend/ASTUnit.cpp:1395
+const FileEntry *fileEntry = FileMgr->getFile(R.second);
+if (!fileEntry) {
// If we can't stat the file we're remapping to, assume that
something
Are we relying on the caller to create new FileMgr before calling
getMainBufferWithPrecompiledPreamble?
Otherwise we'll hit a cached entry and possibly miss an update to the file?
This deserves a comment if that's the case.
Comment at: lib/Frontend/ASTUnit.cpp:2074
// Clear out the diagnostics state.
- FileMgr.reset();
getDiagnostics().Reset();
Parse method used to recreate FileMgr from CompilerInvocation, because it was
reset() at this point.
Won't something break because we're reusing the FileMgr now?
Maybe the code creating FileMgr in Parse should be deleted?
https://reviews.llvm.org/D33045
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[PATCH] D33045: [libclang] Avoid more stats than necessary for reparse.
nik added a comment. Ping :) https://reviews.llvm.org/D33045 ___ cfe-commits mailing list cfe-commits@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[PATCH] D33045: [libclang] Avoid more stats than necessary for reparse.
nik added a comment. Hmm, looks like libclang/libclangTests/LibclangReparseTest.ReparseWithModule is flaky? FAIL: Clang-Unit :: libclang/libclangTests/LibclangReparseTest.ReparseWithModule (10699 of 10701) - TEST 'Clang-Unit :: libclang/libclangTests/LibclangReparseTest.ReparseWithModule' FAILED Note: Google Test filter = LibclangReparseTest.ReparseWithModule [==] Running 1 test from 1 test case. [--] Global test environment set-up. [--] 1 test from LibclangReparseTest [ RUN ] LibclangReparseTest.ReparseWithModule LIBCLANG FATAL ERROR: Unexpected end of file Testing Time: 61.30s Failing Tests (1): Clang-Unit :: libclang/libclangTests/LibclangReparseTest.ReparseWithModule Expected Passes: 10621 Expected Failures : 19 Unsupported Tests : 60 Unexpected Failures: 1 tools/clang/test/CMakeFiles/check-clang.dir/build.make:57: recipe for target 'tools/clang/test/CMakeFiles/check-clang' failed make[3]: *** [tools/clang/test/CMakeFiles/check-clang] Error 1 CMakeFiles/Makefile2:33822: recipe for target 'tools/clang/test/CMakeFiles/check-clang.dir/all' failed make[2]: *** [tools/clang/test/CMakeFiles/check-clang.dir/all] Error 2 CMakeFiles/Makefile2:33829: recipe for target 'tools/clang/test/CMakeFiles/check-clang.dir/rule' failed make[1]: *** [tools/clang/test/CMakeFiles/check-clang.dir/rule] Error 2 Makefile:7932: recipe for target 'check-clang' failed make: *** [check-clang] Error 2 zsh: exit 2 make check-clang https://reviews.llvm.org/D33045 ___ cfe-commits mailing list cfe-commits@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
