[PATCH] D145843: [clangd] Add option to always insert headers with <> instead of ""

[Kadir Cetinkaya via Phabricator via cfe-commits]

kadircet added a comment.

sorry but I am not sure what's the value proposed by this patch in its current 
form. in https://github.com/clangd/clangd/issues/1247 and other places we've 
discussed this, i believe the sentiment was towards providing a config option 
that'll let people customize header insertion style in combination with a 
directory filter, e.g:

  Style:
IncludeInsertion:
Directory: foo/
Delimeter: <
 ... More IncludeInsertion customizations.

Any specific reasons for going with the hard-coded approach? As I don't think 
we can accept all this extra code to make sure it works for a "handful" 
projects/users (in the wild I haven't seen many projects that strictly use `<` 
includes). The use case you want can still be accommodated by such a generic 
approach, you can have `Directory: /` or we can even treat absence of 
`Directory` field as an "always" matcher.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D145843/new/

https://reviews.llvm.org/D145843

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D146417: [clangd] Fix AddUsing in the face of typo-correction

[Kadir Cetinkaya via Phabricator via cfe-commits]

This revision was landed with ongoing or failed builds.
This revision was automatically updated to reflect the committed changes.
Closed by commit rG6f23fee4ef98: [clangd] Fix AddUsing in the face of 
typo-correction (authored by kadircet).

Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D146417/new/

https://reviews.llvm.org/D146417

Files:
  clang-tools-extra/clangd/refactor/tweaks/AddUsing.cpp
  clang-tools-extra/clangd/unittests/tweaks/AddUsingTests.cpp

Index: clang-tools-extra/clangd/unittests/tweaks/AddUsingTests.cpp
===
--- clang-tools-extra/clangd/unittests/tweaks/AddUsingTests.cpp
+++ clang-tools-extra/clangd/unittests/tweaks/AddUsingTests.cpp
@@ -8,8 +8,11 @@
 
 #include "Config.h"
 #include "TweakTesting.h"
+#include "llvm/ADT/StringMap.h"
+#include "llvm/ADT/StringRef.h"
 #include "gmock/gmock.h"
 #include "gtest/gtest.h"
+#include 
 
 namespace clang {
 namespace clangd {
@@ -30,7 +33,7 @@
 void oo() {}
 template class tt {};
 namespace two {
-enum ee {};
+enum ee { ee_enum_value };
 void ff() {}
 class cc {
 public:
@@ -64,9 +67,6 @@
   EXPECT_UNAVAILABLE(Header + "void fun() { ::ban::fo^o(); }");
   EXPECT_AVAILABLE(Header + "void fun() { banana::fo^o(); }");
 
-  // Do not offer code action on typo-corrections.
-  EXPECT_UNAVAILABLE(Header + "/*error-ok*/c^c C;");
-
   // NestedNameSpecifier, but no namespace.
   EXPECT_UNAVAILABLE(Header + "class Foo {}; class F^oo foo;");
 
@@ -466,7 +466,37 @@
 using one::vec;
 
 vec foo;
-)cpp"}};
+)cpp"},
+  // Typo correction.
+  {R"cpp(
+// error-ok
+#include "test.hpp"
+c^c C;
+)cpp",
+   R"cpp(
+// error-ok
+#include "test.hpp"
+using one::two::cc;
+
+cc C;
+)cpp"},
+  {R"cpp(
+// error-ok
+#include "test.hpp"
+void foo() {
+  switch(one::two::ee{}) { case two::ee_^one:break; }
+}
+)cpp",
+   R"cpp(
+// error-ok
+#include "test.hpp"
+using one::two::ee_one;
+
+void foo() {
+  switch(one::two::ee{}) { case ee_one:break; }
+}
+)cpp"},
+  };
   llvm::StringMap EditedFiles;
   for (const auto  : Cases) {
 ExtraFiles["test.hpp"] = R"cpp(
Index: clang-tools-extra/clangd/refactor/tweaks/AddUsing.cpp
===
--- clang-tools-extra/clangd/refactor/tweaks/AddUsing.cpp
+++ clang-tools-extra/clangd/refactor/tweaks/AddUsing.cpp
@@ -8,10 +8,25 @@
 
 #include "AST.h"
 #include "Config.h"
+#include "SourceCode.h"
 #include "refactor/Tweak.h"
 #include "support/Logger.h"
 #include "clang/AST/Decl.h"
+#include "clang/AST/Expr.h"
+#include "clang/AST/NestedNameSpecifier.h"
 #include "clang/AST/RecursiveASTVisitor.h"
+#include "clang/AST/Type.h"
+#include "clang/AST/TypeLoc.h"
+#include "clang/Basic/LLVM.h"
+#include "clang/Basic/SourceLocation.h"
+#include "clang/Tooling/Core/Replacement.h"
+#include "clang/Tooling/Syntax/Tokens.h"
+#include "llvm/ADT/StringRef.h"
+#include "llvm/Support/FormatVariadic.h"
+#include "llvm/Support/raw_ostream.h"
+#include 
+#include 
+#include 
 
 namespace clang {
 namespace clangd {
@@ -45,8 +60,12 @@
   // All of the following are set by prepare().
   // The qualifier to remove.
   NestedNameSpecifierLoc QualifierToRemove;
-  // The name following QualifierToRemove.
-  llvm::StringRef Name;
+  // Qualified name to use when spelling the using declaration. This might be
+  // different than SpelledQualifier in presence of error correction.
+  std::string QualifierToSpell;
+  // The name and qualifier as spelled in the code.
+  llvm::StringRef SpelledQualifier;
+  llvm::StringRef SpelledName;
   // If valid, the insertion point for "using" statement must come after this.
   // This is relevant when the type is defined in the main file, to make sure
   // the type/function is already defined at the point where "using" is added.
@@ -56,7 +75,7 @@
 
 std::string AddUsing::title() const {
   return std::string(llvm::formatv(
-  "Add using-declaration for {0} and remove qualifier", Name));
+  "Add using-declaration for {0} and remove qualifier", SpelledName));
 }
 
 // Locates all "using" statements relevant to SelectionDeclContext.
@@ -269,36 +288,23 @@
   if (Node == nullptr)
 return false;
 
+  SourceRange SpelledNameRange;
   if (auto *D = Node->ASTNode.get()) {
 if (auto *II = D->getDecl()->getIdentifier()) {
   QualifierToRemove = D->getQualifierLoc();
-  Name = II->getName();
+  SpelledNameRange = D->getSourceRange();
   MustInsertAfterLoc = D->getDecl()->getBeginLoc();
 }
   } else if (auto *T = Node->ASTNode.get()) {
 if (auto E = T->getAs()) {
   QualifierToRemove = E.getQualifierLoc();
-  if (!QualifierToRemove)
-return false;
 
-  auto NameRange = E.getSourceRange();
+  SpelledNameRange = E.getSourceRange();
   if (auto T = E.getNamedTypeLoc().getAs()) {
 // Remove the template arguments from the name.
-

[PATCH] D146406: [IncludeCleaner][clangd] Mark umbrella headers as users of private

[Kadir Cetinkaya via Phabricator via cfe-commits]

kadircet added inline comments.



Comment at: clang-tools-extra/include-cleaner/lib/Analysis.cpp:100
+  // Check if main file is the public interface for a private header. If so
+  // we shouldn't diagnose it as unused.
+  if (auto PHeader = PI->getPublic(I.Resolved); !PHeader.empty()) {

hokein wrote:
> The beahvior (no showing the diagnostic) seems reasonable to me as we can 
> infer that the included header is supposed to be exported by the main file. 
> Just explore this a bit more.
> 
> The sad bit is that we start diverging from the classical IWYU tool (I have 
> check it, for this case, it gives an unused-include unless you add an export 
> pragma).
> 
> I think the main cause is that we're missing the `// IWYU pragma: export`, we 
> should still want to recommend users to add the pragma export, right?
> 
> My only concern is that without the `export` pragma, whether the include is 
> used can't be reason about by just "reading" the main-file content by human, 
> e.g. for the case below, there is no usage of the `private.h` (previously the 
> trailing `// IWYU pragma: export` comment is considered a usage), we have to 
> open the private.h and find the private mapping to verify the usage.
> 
> ```
> // public.h
> 
> #include "private.h"
> ```
> 
> It seems better to provide an `adding //IWYU pragma: export` FIXIT.
> 
> The sad bit is that we start diverging from the classical IWYU tool (I have 
> check it, for this case, it gives an unused-include unless you add an export 
> pragma).

we're not providing the same behavior with IWYU tool, it's nice to be 
compatible with it and this change won't break that compatibility (i.e. if IWYU 
tool drops this include, we're not suggesting to insert or while IWYU tool is 
suggesting to insert we're not suggesting to remove). so i don't think this 
argument applies here.

> I think the main cause is that we're missing the // IWYU pragma: export, we 
> should still want to recommend users to add the pragma export, right?

i don't see the reason why. i think we're actually losing value by enforcing 
people to annotate headers in both places. if the library provider already put 
a pragma in the included header, marking the umbrella header as the private 
header, what's the extra value we're getting by making them also annotate the 
public header with a bunch of export pragmas? i feel like this goes against the 
nature of letting tooling automate things as much as possible.

> My only concern is that without the export pragma, whether the include is 
> used can't be reason about by just "reading" the main-file content by human, 
> e.g. for the case below, there is no usage of the private.h (previously the 
> trailing // IWYU pragma: export comment is considered a usage), we have to 
> open the private.h and find the private mapping to verify the usage.

this definitely makes sense, and having an explicit IWYU export/keep pragma on 
these includes are definitely nice but I believe in the common case, because 
private-public matching is 1:1, the reason why a private header is included 
inside a public header is quite obvious hence absence of these pragmas won't 
really cause confusion, to the contrary forcing people to introduce them will 
actually create frustration.

WDYT?


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D146406/new/

https://reviews.llvm.org/D146406

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D146279: [clangd] Extend CollectMainFileMacros.

[Kadir Cetinkaya via Phabricator via cfe-commits]

kadircet accepted this revision.
kadircet added a comment.
This revision is now accepted and ready to land.

thanks!




Comment at: clang-tools-extra/clangd/CollectMacros.h:48
 public:
-  explicit CollectMainFileMacros(const SourceManager , MainFileMacros )
-  : SM(SM), Out(Out) {}
+  explicit CollectMainFileMacros(const SourceManager ,
+ const Preprocessor , MainFileMacros )

nit: you can get SM out of PP, no need to pass both.



Comment at: clang-tools-extra/clangd/CollectMacros.h:70
 
   void Ifdef(SourceLocation Loc, const Token ,
  const MacroDefinition ) override {

nit: can you actually move macro bodies out-of-line ? it's unfortunate that we 
need to build half of the project everytime we touch this header.



Comment at: clang-tools-extra/clangd/ParsedAST.cpp:615
+  std::make_unique(
+  Clang->getSourceManager(), Clang->getPreprocessor(), Macros));
 

nit: we seem to be calling `Clang->getPreprocessor()` in many places, maybe 
extract it into a variable instead?


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D146279/new/

https://reviews.llvm.org/D146279

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D146417: [clangd] Fix AddUsing in the face of typo-correction

[Kadir Cetinkaya via Phabricator via cfe-commits]

kadircet created this revision.
kadircet added a reviewer: hokein.
Herald added a subscriber: arphaman.
Herald added a project: All.
kadircet requested review of this revision.
Herald added subscribers: cfe-commits, MaskRay, ilya-biryukov.
Herald added a project: clang-tools-extra.

Fixes https://github.com/clangd/clangd/issues/559


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D146417

Files:
  clang-tools-extra/clangd/refactor/tweaks/AddUsing.cpp
  clang-tools-extra/clangd/unittests/tweaks/AddUsingTests.cpp

Index: clang-tools-extra/clangd/unittests/tweaks/AddUsingTests.cpp
===
--- clang-tools-extra/clangd/unittests/tweaks/AddUsingTests.cpp
+++ clang-tools-extra/clangd/unittests/tweaks/AddUsingTests.cpp
@@ -8,8 +8,11 @@
 
 #include "Config.h"
 #include "TweakTesting.h"
+#include "llvm/ADT/StringMap.h"
+#include "llvm/ADT/StringRef.h"
 #include "gmock/gmock.h"
 #include "gtest/gtest.h"
+#include 
 
 namespace clang {
 namespace clangd {
@@ -30,7 +33,7 @@
 void oo() {}
 template class tt {};
 namespace two {
-enum ee {};
+enum ee { ee_enum_value };
 void ff() {}
 class cc {
 public:
@@ -64,9 +67,6 @@
   EXPECT_UNAVAILABLE(Header + "void fun() { ::ban::fo^o(); }");
   EXPECT_AVAILABLE(Header + "void fun() { banana::fo^o(); }");
 
-  // Do not offer code action on typo-corrections.
-  EXPECT_UNAVAILABLE(Header + "/*error-ok*/c^c C;");
-
   // NestedNameSpecifier, but no namespace.
   EXPECT_UNAVAILABLE(Header + "class Foo {}; class F^oo foo;");
 
@@ -466,7 +466,37 @@
 using one::vec;
 
 vec foo;
-)cpp"}};
+)cpp"},
+  // Typo correction.
+  {R"cpp(
+// error-ok
+#include "test.hpp"
+c^c C;
+)cpp",
+   R"cpp(
+// error-ok
+#include "test.hpp"
+using one::two::cc;
+
+cc C;
+)cpp"},
+  {R"cpp(
+// error-ok
+#include "test.hpp"
+void foo() {
+  switch(one::two::ee{}) { case two::ee_^one:break; }
+}
+)cpp",
+   R"cpp(
+// error-ok
+#include "test.hpp"
+using one::two::ee_one;
+
+void foo() {
+  switch(one::two::ee{}) { case ee_one:break; }
+}
+)cpp"},
+  };
   llvm::StringMap EditedFiles;
   for (const auto  : Cases) {
 ExtraFiles["test.hpp"] = R"cpp(
Index: clang-tools-extra/clangd/refactor/tweaks/AddUsing.cpp
===
--- clang-tools-extra/clangd/refactor/tweaks/AddUsing.cpp
+++ clang-tools-extra/clangd/refactor/tweaks/AddUsing.cpp
@@ -8,10 +8,25 @@
 
 #include "AST.h"
 #include "Config.h"
+#include "SourceCode.h"
 #include "refactor/Tweak.h"
 #include "support/Logger.h"
 #include "clang/AST/Decl.h"
+#include "clang/AST/Expr.h"
+#include "clang/AST/NestedNameSpecifier.h"
 #include "clang/AST/RecursiveASTVisitor.h"
+#include "clang/AST/Type.h"
+#include "clang/AST/TypeLoc.h"
+#include "clang/Basic/LLVM.h"
+#include "clang/Basic/SourceLocation.h"
+#include "clang/Tooling/Core/Replacement.h"
+#include "clang/Tooling/Syntax/Tokens.h"
+#include "llvm/ADT/StringRef.h"
+#include "llvm/Support/FormatVariadic.h"
+#include "llvm/Support/raw_ostream.h"
+#include 
+#include 
+#include 
 
 namespace clang {
 namespace clangd {
@@ -45,8 +60,12 @@
   // All of the following are set by prepare().
   // The qualifier to remove.
   NestedNameSpecifierLoc QualifierToRemove;
-  // The name following QualifierToRemove.
-  llvm::StringRef Name;
+  // Qualified name to use when spelling the using declaration. This might be
+  // different than SpelledQualifier in presence of error correction.
+  std::string QualifierToSpell;
+  // The name and qualifier as spelled in the code.
+  llvm::StringRef SpelledQualifier;
+  llvm::StringRef SpelledName;
   // If valid, the insertion point for "using" statement must come after this.
   // This is relevant when the type is defined in the main file, to make sure
   // the type/function is already defined at the point where "using" is added.
@@ -56,7 +75,7 @@
 
 std::string AddUsing::title() const {
   return std::string(llvm::formatv(
-  "Add using-declaration for {0} and remove qualifier", Name));
+  "Add using-declaration for {0} and remove qualifier", SpelledName));
 }
 
 // Locates all "using" statements relevant to SelectionDeclContext.
@@ -269,36 +288,23 @@
   if (Node == nullptr)
 return false;
 
+  SourceRange SpelledNameRange;
   if (auto *D = Node->ASTNode.get()) {
 if (auto *II = D->getDecl()->getIdentifier()) {
   QualifierToRemove = D->getQualifierLoc();
-  Name = II->getName();
+  SpelledNameRange = D->getSourceRange();
   MustInsertAfterLoc = D->getDecl()->getBeginLoc();
 }
   } else if (auto *T = Node->ASTNode.get()) {
 if (auto E = T->getAs()) {
   QualifierToRemove = E.getQualifierLoc();
-  if (!QualifierToRemove)
-return false;
 
-  auto NameRange = E.getSourceRange();
+  SpelledNameRange = E.getSourceRange();
   if (auto T = E.getNamedTypeLoc().getAs()) {
 // Remove the template arguments from the name.
-

[PATCH] D146406: [IncludeCleaner][clangd] Mark umbrella headers as users of private

[Kadir Cetinkaya via Phabricator via cfe-commits]

kadircet created this revision.
kadircet added a reviewer: hokein.
Herald added a subscriber: arphaman.
Herald added a project: All.
kadircet requested review of this revision.
Herald added subscribers: cfe-commits, MaskRay, ilya-biryukov.
Herald added projects: clang, clang-tools-extra.

Private headers inside umbrella files shouldn't be marked as unused.


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D146406

Files:
  clang-tools-extra/clangd/IncludeCleaner.cpp
  clang-tools-extra/clangd/unittests/IncludeCleanerTests.cpp
  clang-tools-extra/include-cleaner/lib/Analysis.cpp
  clang-tools-extra/include-cleaner/unittests/AnalysisTest.cpp
  clang/include/clang/Testing/TestAST.h
  clang/lib/Testing/TestAST.cpp

Index: clang/lib/Testing/TestAST.cpp
===
--- clang/lib/Testing/TestAST.cpp
+++ clang/lib/Testing/TestAST.cpp
@@ -16,6 +16,7 @@
 #include "llvm/Support/VirtualFileSystem.h"
 
 #include "gtest/gtest.h"
+#include 
 
 namespace clang {
 namespace {
@@ -91,7 +92,9 @@
 Argv.push_back(S.c_str());
   for (const auto  : In.ExtraArgs)
 Argv.push_back(S.c_str());
-  std::string Filename = getFilenameForTesting(In.Language).str();
+  std::string Filename = In.FileName;
+  if (Filename.empty())
+Filename = getFilenameForTesting(In.Language).str();
   Argv.push_back(Filename.c_str());
   Clang->setInvocation(std::make_unique());
   if (!CompilerInvocation::CreateFromArgs(Clang->getInvocation(), Argv,
Index: clang/include/clang/Testing/TestAST.h
===
--- clang/include/clang/Testing/TestAST.h
+++ clang/include/clang/Testing/TestAST.h
@@ -49,6 +49,9 @@
   /// Keys are plain filenames ("foo.h"), values are file content.
   llvm::StringMap ExtraFiles = {};
 
+  /// Filename to use for translation unit. A default will be used when empty.
+  std::string FileName;
+
   /// By default, error diagnostics during parsing are reported as gtest errors.
   /// To suppress this, set ErrorOK or include "error-ok" in a comment in Code.
   /// In either case, all diagnostics appear in TestAST::diagnostics().
Index: clang-tools-extra/include-cleaner/unittests/AnalysisTest.cpp
===
--- clang-tools-extra/include-cleaner/unittests/AnalysisTest.cpp
+++ clang-tools-extra/include-cleaner/unittests/AnalysisTest.cpp
@@ -212,17 +212,38 @@
 return std::make_unique(PP, PI);
   };
 
-  TestAST AST(Inputs);
-  auto Decls = AST.context().getTranslationUnitDecl()->decls();
-  auto Results =
-  analyze(std::vector{Decls.begin(), Decls.end()},
-  PP.MacroReferences, PP.Includes, , AST.sourceManager(),
-  AST.preprocessor().getHeaderSearchInfo());
-
-  const Include *B = PP.Includes.atLine(3);
-  ASSERT_EQ(B->Spelled, "b.h");
-  EXPECT_THAT(Results.Missing, ElementsAre("\"c.h\""));
-  EXPECT_THAT(Results.Unused, ElementsAre(B));
+  {
+TestAST AST(Inputs);
+auto Decls = AST.context().getTranslationUnitDecl()->decls();
+auto Results =
+analyze(std::vector{Decls.begin(), Decls.end()},
+PP.MacroReferences, PP.Includes, , AST.sourceManager(),
+AST.preprocessor().getHeaderSearchInfo());
+
+const Include *B = PP.Includes.atLine(3);
+ASSERT_EQ(B->Spelled, "b.h");
+EXPECT_THAT(Results.Missing, ElementsAre("\"c.h\""));
+EXPECT_THAT(Results.Unused, ElementsAre(B));
+  }
+
+  // Check that umbrella header uses private include.
+  {
+Inputs.Code = R"cpp(
+#include "private.h"
+)cpp";
+Inputs.ExtraFiles["private.h"] =
+guard("// IWYU pragma: private, include \"public.h\"");
+Inputs.FileName = "public.h";
+PP.Includes = {};
+PI = {};
+TestAST AST(Inputs);
+auto Results = analyze({}, {}, PP.Includes, , AST.sourceManager(),
+   AST.preprocessor().getHeaderSearchInfo());
+
+const Include *Private = PP.Includes.atLine(2);
+ASSERT_EQ(Private->Spelled, "private.h");
+EXPECT_THAT(Results.Unused, testing::IsEmpty());
+  }
 }
 
 TEST(FixIncludes, Basic) {
Index: clang-tools-extra/include-cleaner/lib/Analysis.cpp
===
--- clang-tools-extra/include-cleaner/lib/Analysis.cpp
+++ clang-tools-extra/include-cleaner/lib/Analysis.cpp
@@ -90,9 +90,25 @@
});
 
   AnalysisResults Results;
-  for (const Include  : Inc.all())
-if (!Used.contains() && PI && !PI->shouldKeep(I.Line))
-  Results.Unused.push_back();
+  for (const Include  : Inc.all()) {
+if (Used.contains())
+  continue;
+if (PI) {
+  if (PI->shouldKeep(I.Line))
+continue;
+  // Check if main file is the public interface for a private header. If so
+  // we shouldn't diagnose it as unused.
+  if (auto PHeader = PI->getPublic(I.Resolved); !PHeader.empty()) {
+PHeader = PHeader.trim("<>\"");
+// Since most 

[PATCH] D143755: [clang-format] Add a space between an overloaded operator and '>'

[Kadir Cetinkaya via Phabricator via cfe-commits]

kadircet added a comment.

Hi @owenpan, this seems to be crashing for:

  struct Foo { operator enum foo{} };

with stack trace:

  $ ~/repos/llvm/build/bin/clang-format format_crash.cc --dry-run
  clang-format: 
/usr/local/google/home/kadircet/repos/llvm/clang/lib/Format/TokenAnnotator.cpp:1229:
 bool clang::format::(anonymous namespace)::AnnotatingParser::consumeToken(): 
Assertion `CurrentToken' failed.
  PLEASE submit a bug report to https://github.com/llvm/llvm-project/issues/ 
and include the crash backtrace.
  Stack dump:
  0.  Program arguments: 
/usr/local/google/home/kadircet/repos/llvm/build/bin/clang-format 
format_crash.cc --dry-run
   #0 0x00397687 llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) 
/usr/local/google/home/kadircet/repos/llvm/llvm/lib/Support/Unix/Signals.inc:567:13
   #1 0x0039583e llvm::sys::RunSignalHandlers() 
/usr/local/google/home/kadircet/repos/llvm/llvm/lib/Support/Signals.cpp:105:18
   #2 0x00397faa SignalHandler(int) 
/usr/local/google/home/kadircet/repos/llvm/llvm/lib/Support/Unix/Signals.inc:412:1
   #3 0x7f49d145af90 (/lib/x86_64-linux-gnu/libc.so.6+0x3bf90)
   #4 0x7f49d14a9ccc __pthread_kill_implementation 
./nptl/pthread_kill.c:44:76
   #5 0x7f49d145aef2 raise ./signal/../sysdeps/posix/raise.c:27:6
   #6 0x7f49d1445472 abort ./stdlib/abort.c:81:7
   #7 0x7f49d1445395 _nl_load_domain ./intl/loadmsgcat.c:1177:9
   #8 0x7f49d1453df2 (/lib/x86_64-linux-gnu/libc.so.6+0x34df2)
   #9 0x0043a7c1 parseBrace 
/usr/local/google/home/kadircet/repos/llvm/clang/lib/Format/TokenAnnotator.cpp:867:9
  #10 0x0043a7c1 clang::format::(anonymous 
namespace)::AnnotatingParser::consumeToken() 
/usr/local/google/home/kadircet/repos/llvm/clang/lib/Format/TokenAnnotator.cpp:1170:12
  #11 0x0042e1f8 clang::format::(anonymous 
namespace)::AnnotatingParser::parseLine() 
/usr/local/google/home/kadircet/repos/llvm/clang/lib/Format/TokenAnnotator.cpp:1558:11
  #12 0x0042dc37 
clang::format::TokenAnnotator::annotate(clang::format::AnnotatedLine&) 
/usr/local/google/home/kadircet/repos/llvm/clang/lib/Format/TokenAnnotator.cpp:2837:13
  #13 0x0041b910 clang::format::TokenAnalyzer::process(bool) 
/usr/local/google/home/kadircet/repos/llvm/clang/lib/Format/TokenAnalyzer.cpp:0:19
  #14 0x003d0576 ~TokenAnalyzer 
/usr/local/google/home/kadircet/repos/llvm/clang/lib/Format/TokenAnalyzer.h:88:7
  #15 0x003d0576 operator() 
/usr/local/google/home/kadircet/repos/llvm/clang/lib/Format/Format.cpp:3492:9
  #16 0x003d0576 __invoke_impl, (lambda at 
/usr/local/google/home/kadircet/repos/llvm/clang/lib/Format/Format.cpp:3491:27) 
&, const clang::format::Environment &> 
/usr/bin/../lib/gcc/x86_64-linux-gnu/12/../../../../include/c++/12/bits/invoke.h:61:14
  #17 0x003d0576 __invoke_r, (lambda at 
/usr/local/google/home/kadircet/repos/llvm/clang/lib/Format/Format.cpp:3491:27) 
&, const clang::format::Environment &> 
/usr/bin/../lib/gcc/x86_64-linux-gnu/12/../../../../include/c++/12/bits/invoke.h:114:9
  #18 0x003d0576 
std::_Function_handler 
(clang::format::Environment const&), 
clang::format::internal::reformat(clang::format::FormatStyle const&, 
llvm::StringRef, llvm::ArrayRef, unsigned int, unsigned 
int, unsigned int, llvm::StringRef, 
clang::format::FormattingAttemptStatus*)::$_8>::_M_invoke(std::_Any_data 
const&, clang::format::Environment const&) 
/usr/bin/../lib/gcc/x86_64-linux-gnu/12/../../../../include/c++/12/bits/std_function.h:290:9
  #19 0x003b8a18 _M_is_engaged 
/usr/bin/../lib/gcc/x86_64-linux-gnu/12/../../../../include/c++/12/optional:471:58
  #20 0x003b8a18 operator bool 
/usr/bin/../lib/gcc/x86_64-linux-gnu/12/../../../../include/c++/12/optional:985:22
  #21 0x003b8a18 
clang::format::internal::reformat(clang::format::FormatStyle const&, 
llvm::StringRef, llvm::ArrayRef, unsigned int, unsigned 
int, unsigned int, llvm::StringRef, clang::format::FormattingAttemptStatus*) 
/usr/local/google/home/kadircet/repos/llvm/clang/lib/Format/Format.cpp:3533:9
  #22 0x003b9c73 _Rb_tree_impl 
/usr/bin/../lib/gcc/x86_64-linux-gnu/12/../../../../include/c++/12/bits/stl_tree.h:687:12
  #23 0x003b9c73 _Rb_tree 
/usr/bin/../lib/gcc/x86_64-linux-gnu/12/../../../../include/c++/12/bits/stl_tree.h:954:7
  #24 0x003b9c73 set 
/usr/bin/../lib/gcc/x86_64-linux-gnu/12/../../../../include/c++/12/bits/stl_set.h:231:7
  #25 0x003b9c73 Replacements 
/usr/local/google/home/kadircet/repos/llvm/clang/include/clang/Tooling/Core/Replacement.h:212:7
  #26 0x003b9c73 clang::format::reformat(clang::format::FormatStyle 
const&, llvm::StringRef, llvm::ArrayRef, 
llvm::StringRef, clang::format::FormattingAttemptStatus*) 
/usr/local/google/home/kadircet/repos/llvm/clang/lib/Format/Format.cpp:3557:10
  #27 0x0033da7f clang::format::format(llvm::StringRef) 

[PATCH] D146202: [clang] Fix a UsingTemplate regression after 3e78fa860235431323aaf08c8fa922d75a7cfffa

[Kadir Cetinkaya via Phabricator via cfe-commits]

kadircet accepted this revision.
kadircet added a subscriber: ChuanqiXu.
kadircet added a comment.
This revision is now accepted and ready to land.

FWIW, I believe this patch does the right thing by marking the 
`DeducedTemplateSpecializationType` as `using`. It's explicitly introduced into 
the global namespace through the using decl, and even before 
3e78fa860235431323aaf08c8fa922d75a7cfffa 
 we 
weren't marking them as such.

But I am still not sure about the implications of this in the modules world. So 
adding @ChuanqiXu as a reviewer in case he has more insights.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D146202/new/

https://reviews.llvm.org/D146202

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D146116: [clangd] Respect WantDiags when emitting diags from possibly stale preambles

[Kadir Cetinkaya via Phabricator via cfe-commits]

This revision was automatically updated to reflect the committed changes.
Closed by commit rG9e8bac748064: [clangd] Respect WantDiags when emitting diags 
from possibly stale preambles (authored by kadircet).

Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D146116/new/

https://reviews.llvm.org/D146116

Files:
  clang-tools-extra/clangd/TUScheduler.cpp
  clang-tools-extra/clangd/unittests/TUSchedulerTests.cpp


Index: clang-tools-extra/clangd/unittests/TUSchedulerTests.cpp
===
--- clang-tools-extra/clangd/unittests/TUSchedulerTests.cpp
+++ clang-tools-extra/clangd/unittests/TUSchedulerTests.cpp
@@ -1309,6 +1309,13 @@
 
   // Make sure that we have eventual consistency.
   EXPECT_THAT(Collector.diagVersions().back(), Pair(PI.Version, PI.Version));
+
+  // Check that WantDiagnostics::No doesn't emit any diags.
+  PI.Version = "4";
+  PI.Contents = "#define FOO\n" + PI.Version;
+  S.update(File, PI, WantDiagnostics::No);
+  S.blockUntilIdle(timeoutSeconds(5));
+  EXPECT_THAT(Collector.diagVersions().back(), Pair("3", "3"));
 }
 
 // If a header file is missing from the CDB (or inferred using heuristics), and
Index: clang-tools-extra/clangd/TUScheduler.cpp
===
--- clang-tools-extra/clangd/TUScheduler.cpp
+++ clang-tools-extra/clangd/TUScheduler.cpp
@@ -948,7 +948,8 @@
 // rebuild. Newly built preamble cannot emit diagnostics before this call
 // finishes (ast callbacks are called from astpeer thread), hence we
 // gurantee eventual consistency.
-if (LatestPreamble && Config::current().Diagnostics.AllowStalePreamble)
+if (LatestPreamble && WantDiags != WantDiagnostics::No &&
+Config::current().Diagnostics.AllowStalePreamble)
   generateDiagnostics(std::move(Invocation), std::move(Inputs),
   std::move(CompilerInvocationDiags));
 


Index: clang-tools-extra/clangd/unittests/TUSchedulerTests.cpp
===
--- clang-tools-extra/clangd/unittests/TUSchedulerTests.cpp
+++ clang-tools-extra/clangd/unittests/TUSchedulerTests.cpp
@@ -1309,6 +1309,13 @@
 
   // Make sure that we have eventual consistency.
   EXPECT_THAT(Collector.diagVersions().back(), Pair(PI.Version, PI.Version));
+
+  // Check that WantDiagnostics::No doesn't emit any diags.
+  PI.Version = "4";
+  PI.Contents = "#define FOO\n" + PI.Version;
+  S.update(File, PI, WantDiagnostics::No);
+  S.blockUntilIdle(timeoutSeconds(5));
+  EXPECT_THAT(Collector.diagVersions().back(), Pair("3", "3"));
 }
 
 // If a header file is missing from the CDB (or inferred using heuristics), and
Index: clang-tools-extra/clangd/TUScheduler.cpp
===
--- clang-tools-extra/clangd/TUScheduler.cpp
+++ clang-tools-extra/clangd/TUScheduler.cpp
@@ -948,7 +948,8 @@
 // rebuild. Newly built preamble cannot emit diagnostics before this call
 // finishes (ast callbacks are called from astpeer thread), hence we
 // gurantee eventual consistency.
-if (LatestPreamble && Config::current().Diagnostics.AllowStalePreamble)
+if (LatestPreamble && WantDiags != WantDiagnostics::No &&
+Config::current().Diagnostics.AllowStalePreamble)
   generateDiagnostics(std::move(Invocation), std::move(Inputs),
   std::move(CompilerInvocationDiags));
 
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D146116: [clangd] Respect WantDiags when emitting diags from possibly stale preambles

[Kadir Cetinkaya via Phabricator via cfe-commits]

kadircet created this revision.
kadircet added a reviewer: hokein.
Herald added subscribers: arphaman, javed.absar.
Herald added a project: All.
kadircet requested review of this revision.
Herald added subscribers: cfe-commits, MaskRay, ilya-biryukov.
Herald added a project: clang-tools-extra.

Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D146116

Files:
  clang-tools-extra/clangd/TUScheduler.cpp
  clang-tools-extra/clangd/unittests/TUSchedulerTests.cpp


Index: clang-tools-extra/clangd/unittests/TUSchedulerTests.cpp
===
--- clang-tools-extra/clangd/unittests/TUSchedulerTests.cpp
+++ clang-tools-extra/clangd/unittests/TUSchedulerTests.cpp
@@ -1309,6 +1309,13 @@
 
   // Make sure that we have eventual consistency.
   EXPECT_THAT(Collector.diagVersions().back(), Pair(PI.Version, PI.Version));
+
+  // Check that WantDiagnostics::No doesn't emit any diags.
+  PI.Version = "4";
+  PI.Contents = "#define FOO\n" + PI.Version;
+  S.update(File, PI, WantDiagnostics::No);
+  S.blockUntilIdle(timeoutSeconds(5));
+  EXPECT_THAT(Collector.diagVersions().back(), Pair("3", "3"));
 }
 
 // If a header file is missing from the CDB (or inferred using heuristics), and
Index: clang-tools-extra/clangd/TUScheduler.cpp
===
--- clang-tools-extra/clangd/TUScheduler.cpp
+++ clang-tools-extra/clangd/TUScheduler.cpp
@@ -948,7 +948,8 @@
 // rebuild. Newly built preamble cannot emit diagnostics before this call
 // finishes (ast callbacks are called from astpeer thread), hence we
 // gurantee eventual consistency.
-if (LatestPreamble && Config::current().Diagnostics.AllowStalePreamble)
+if (LatestPreamble && WantDiags != WantDiagnostics::No &&
+Config::current().Diagnostics.AllowStalePreamble)
   generateDiagnostics(std::move(Invocation), std::move(Inputs),
   std::move(CompilerInvocationDiags));
 


Index: clang-tools-extra/clangd/unittests/TUSchedulerTests.cpp
===
--- clang-tools-extra/clangd/unittests/TUSchedulerTests.cpp
+++ clang-tools-extra/clangd/unittests/TUSchedulerTests.cpp
@@ -1309,6 +1309,13 @@
 
   // Make sure that we have eventual consistency.
   EXPECT_THAT(Collector.diagVersions().back(), Pair(PI.Version, PI.Version));
+
+  // Check that WantDiagnostics::No doesn't emit any diags.
+  PI.Version = "4";
+  PI.Contents = "#define FOO\n" + PI.Version;
+  S.update(File, PI, WantDiagnostics::No);
+  S.blockUntilIdle(timeoutSeconds(5));
+  EXPECT_THAT(Collector.diagVersions().back(), Pair("3", "3"));
 }
 
 // If a header file is missing from the CDB (or inferred using heuristics), and
Index: clang-tools-extra/clangd/TUScheduler.cpp
===
--- clang-tools-extra/clangd/TUScheduler.cpp
+++ clang-tools-extra/clangd/TUScheduler.cpp
@@ -948,7 +948,8 @@
 // rebuild. Newly built preamble cannot emit diagnostics before this call
 // finishes (ast callbacks are called from astpeer thread), hence we
 // gurantee eventual consistency.
-if (LatestPreamble && Config::current().Diagnostics.AllowStalePreamble)
+if (LatestPreamble && WantDiags != WantDiagnostics::No &&
+Config::current().Diagnostics.AllowStalePreamble)
   generateDiagnostics(std::move(Invocation), std::move(Inputs),
   std::move(CompilerInvocationDiags));
 
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D146028: [clangd] Patch main file macros in preamble

[Kadir Cetinkaya via Phabricator via cfe-commits]

This revision was automatically updated to reflect the committed changes.
Closed by commit rG82c8bf8fcc91: [clangd] Patch main file macros in preamble 
(authored by kadircet).

Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D146028/new/

https://reviews.llvm.org/D146028

Files:
  clang-tools-extra/clangd/Preamble.cpp
  clang-tools-extra/clangd/Preamble.h
  clang-tools-extra/clangd/unittests/PreambleTests.cpp


Index: clang-tools-extra/clangd/unittests/PreambleTests.cpp
===
--- clang-tools-extra/clangd/unittests/PreambleTests.cpp
+++ clang-tools-extra/clangd/unittests/PreambleTests.cpp
@@ -30,7 +30,6 @@
 #include "llvm/Support/MemoryBuffer.h"
 #include "llvm/Support/ScopedPrinter.h"
 #include "llvm/Support/VirtualFileSystem.h"
-#include "llvm/Testing/Support/SupportHelpers.h"
 #include "gmock/gmock.h"
 #include "gtest/gtest-matchers.h"
 #include "gtest/gtest.h"
@@ -855,11 +854,12 @@
 ]]
 #pragma $y[[mark YY
 ]]
+#define BAZ
 
 #endif)cpp");
 auto AST = createPatchedAST(Code.code(), NewCode.code());
-// FIXME: Macros and marks have locations that need to be patched.
-EXPECT_THAT(AST->getMacros().Names, IsEmpty());
+EXPECT_THAT(AST->getMacros().Names.keys(),
+UnorderedElementsAreArray({"FOO", "BAR", "BAZ"}));
 EXPECT_THAT(AST->getMarks(),
 UnorderedElementsAre(Mark(NewCode.range("x"), " XX"),
  Mark(NewCode.range("y"), " YY")));
Index: clang-tools-extra/clangd/Preamble.h
===
--- clang-tools-extra/clangd/Preamble.h
+++ clang-tools-extra/clangd/Preamble.h
@@ -164,7 +164,7 @@
   static constexpr llvm::StringLiteral HeaderName = "__preamble_patch__.h";
 
   llvm::ArrayRef marks() const;
-  MainFileMacros mainFileMacros() const;
+  const MainFileMacros () const;
 
 private:
   static PreamblePatch create(llvm::StringRef FileName,
@@ -183,6 +183,7 @@
   PreambleBounds ModifiedBounds = {0, false};
   const PreambleData *Baseline = nullptr;
   std::vector PatchedMarks;
+  MainFileMacros PatchedMacros;
 };
 
 } // namespace clangd
Index: clang-tools-extra/clangd/Preamble.cpp
===
--- clang-tools-extra/clangd/Preamble.cpp
+++ clang-tools-extra/clangd/Preamble.cpp
@@ -325,6 +325,7 @@
   std::vector Lines;
   PreambleBounds Bounds = {0, false};
   std::vector Marks;
+  MainFileMacros Macros;
 };
 
 /// Scans the preprocessor directives in the preamble section of the file by
@@ -373,14 +374,15 @@
   if (!Action.BeginSourceFile(*Clang, Clang->getFrontendOpts().Inputs[0]))
 return error("failed BeginSourceFile");
   Preprocessor  = Clang->getPreprocessor();
+  const auto  = PP.getSourceManager();
   IncludeStructure Includes;
   Includes.collect(*Clang);
   ScannedPreamble SP;
   SP.Bounds = Bounds;
   PP.addPPCallbacks(
   std::make_unique(PP, SP.TextualDirectives));
-  PP.addPPCallbacks(
-  collectPragmaMarksCallback(PP.getSourceManager(), SP.Marks));
+  PP.addPPCallbacks(collectPragmaMarksCallback(SM, SP.Marks));
+  PP.addPPCallbacks(std::make_unique(SM, SP.Macros));
   if (llvm::Error Err = Action.Execute())
 return std::move(Err);
   Action.EndSourceFile();
@@ -859,6 +861,7 @@
 
   PP.PatchedDiags = patchDiags(Baseline.Diags, *BaselineScan, *ModifiedScan);
   PP.PatchedMarks = std::move(ModifiedScan->Marks);
+  PP.PatchedMacros = std::move(ModifiedScan->Macros);
   dlog("Created preamble patch: {0}", Patch.str());
   Patch.flush();
   return PP;
@@ -915,11 +918,10 @@
   return PatchedMarks;
 }
 
-MainFileMacros PreamblePatch::mainFileMacros() const {
+const MainFileMacros ::mainFileMacros() const {
   if (PatchContents.empty())
 return Baseline->Macros;
-  // FIXME: Patch main file macros.
-  return MainFileMacros();
+  return PatchedMacros;
 }
 } // namespace clangd
 } // namespace clang


Index: clang-tools-extra/clangd/unittests/PreambleTests.cpp
===
--- clang-tools-extra/clangd/unittests/PreambleTests.cpp
+++ clang-tools-extra/clangd/unittests/PreambleTests.cpp
@@ -30,7 +30,6 @@
 #include "llvm/Support/MemoryBuffer.h"
 #include "llvm/Support/ScopedPrinter.h"
 #include "llvm/Support/VirtualFileSystem.h"
-#include "llvm/Testing/Support/SupportHelpers.h"
 #include "gmock/gmock.h"
 #include "gtest/gtest-matchers.h"
 #include "gtest/gtest.h"
@@ -855,11 +854,12 @@
 ]]
 #pragma $y[[mark YY
 ]]
+#define BAZ
 
 #endif)cpp");
 auto AST = createPatchedAST(Code.code(), NewCode.code());
-// FIXME: Macros and marks have locations that need to be patched.
-EXPECT_THAT(AST->getMacros().Names, IsEmpty());
+EXPECT_THAT(AST->getMacros().Names.keys(),
+UnorderedElementsAreArray({"FOO", "BAR", "BAZ"}));
 EXPECT_THAT(AST->getMarks(),
 UnorderedElementsAre(Mark(NewCode.range("x"), " 

[PATCH] D146026: [clangd] Patch PragmaMarks in preamble section of the file

[Kadir Cetinkaya via Phabricator via cfe-commits]

This revision was automatically updated to reflect the committed changes.
Closed by commit rG9c888120e3c8: [clangd] Patch PragmaMarks in preamble section 
of the file (authored by kadircet).

Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D146026/new/

https://reviews.llvm.org/D146026

Files:
  clang-tools-extra/clangd/Preamble.cpp
  clang-tools-extra/clangd/Preamble.h
  clang-tools-extra/clangd/unittests/PreambleTests.cpp

Index: clang-tools-extra/clangd/unittests/PreambleTests.cpp
===
--- clang-tools-extra/clangd/unittests/PreambleTests.cpp
+++ clang-tools-extra/clangd/unittests/PreambleTests.cpp
@@ -829,6 +829,10 @@
   }
 }
 
+MATCHER_P2(Mark, Range, Text, "") {
+  return std::tie(arg.Rng, arg.Trivia) == std::tie(Range, Text);
+}
+
 TEST(PreamblePatch, MacroAndMarkHandling) {
   Config Cfg;
   Cfg.Diagnostics.AllowStalePreamble = true;
@@ -847,13 +851,18 @@
 #ifndef FOO
 #define FOO
 #define BAR
-#pragma mark XX
+#pragma $x[[mark XX
+]]
+#pragma $y[[mark YY
+]]
 
 #endif)cpp");
 auto AST = createPatchedAST(Code.code(), NewCode.code());
 // FIXME: Macros and marks have locations that need to be patched.
 EXPECT_THAT(AST->getMacros().Names, IsEmpty());
-EXPECT_THAT(AST->getMarks(), IsEmpty());
+EXPECT_THAT(AST->getMarks(),
+UnorderedElementsAre(Mark(NewCode.range("x"), " XX"),
+ Mark(NewCode.range("y"), " YY")));
   }
 }
 
Index: clang-tools-extra/clangd/Preamble.h
===
--- clang-tools-extra/clangd/Preamble.h
+++ clang-tools-extra/clangd/Preamble.h
@@ -182,6 +182,7 @@
   std::vector PatchedDiags;
   PreambleBounds ModifiedBounds = {0, false};
   const PreambleData *Baseline = nullptr;
+  std::vector PatchedMarks;
 };
 
 } // namespace clangd
Index: clang-tools-extra/clangd/Preamble.cpp
===
--- clang-tools-extra/clangd/Preamble.cpp
+++ clang-tools-extra/clangd/Preamble.cpp
@@ -15,7 +15,9 @@
 #include "Protocol.h"
 #include "SourceCode.h"
 #include "clang-include-cleaner/Record.h"
+#include "index/CanonicalIncludes.h"
 #include "support/Logger.h"
+#include "support/Path.h"
 #include "support/ThreadsafeFS.h"
 #include "support/Trace.h"
 #include "clang/AST/DeclTemplate.h"
@@ -27,6 +29,7 @@
 #include "clang/Basic/TokenKinds.h"
 #include "clang/Frontend/CompilerInvocation.h"
 #include "clang/Frontend/FrontendActions.h"
+#include "clang/Frontend/PrecompiledPreamble.h"
 #include "clang/Lex/HeaderSearch.h"
 #include "clang/Lex/Lexer.h"
 #include "clang/Lex/PPCallbacks.h"
@@ -42,6 +45,7 @@
 #include "llvm/ADT/StringExtras.h"
 #include "llvm/ADT/StringMap.h"
 #include "llvm/ADT/StringRef.h"
+#include "llvm/Support/Casting.h"
 #include "llvm/Support/Error.h"
 #include "llvm/Support/ErrorHandling.h"
 #include "llvm/Support/FormatVariadic.h"
@@ -50,10 +54,12 @@
 #include "llvm/Support/VirtualFileSystem.h"
 #include "llvm/Support/raw_ostream.h"
 #include 
+#include 
 #include 
 #include 
 #include 
 #include 
+#include 
 #include 
 #include 
 
@@ -318,6 +324,7 @@
   // Literal lines of the preamble contents.
   std::vector Lines;
   PreambleBounds Bounds = {0, false};
+  std::vector Marks;
 };
 
 /// Scans the preprocessor directives in the preamble section of the file by
@@ -372,6 +379,8 @@
   SP.Bounds = Bounds;
   PP.addPPCallbacks(
   std::make_unique(PP, SP.TextualDirectives));
+  PP.addPPCallbacks(
+  collectPragmaMarksCallback(PP.getSourceManager(), SP.Marks));
   if (llvm::Error Err = Action.Execute())
 return std::move(Err);
   Action.EndSourceFile();
@@ -849,6 +858,7 @@
   }
 
   PP.PatchedDiags = patchDiags(Baseline.Diags, *BaselineScan, *ModifiedScan);
+  PP.PatchedMarks = std::move(ModifiedScan->Marks);
   dlog("Created preamble patch: {0}", Patch.str());
   Patch.flush();
   return PP;
@@ -902,8 +912,7 @@
 llvm::ArrayRef PreamblePatch::marks() const {
   if (PatchContents.empty())
 return Baseline->Marks;
-  // FIXME: Patch pragma marks.
-  return {};
+  return PatchedMarks;
 }
 
 MainFileMacros PreamblePatch::mainFileMacros() const {
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D146028: [clangd] Patch main file macros in preamble

[Kadir Cetinkaya via Phabricator via cfe-commits]

kadircet updated this revision to Diff 505093.
kadircet marked an inline comment as done.
kadircet added a comment.

- Rebase
- Prevent unnecessary copies


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D146028/new/

https://reviews.llvm.org/D146028

Files:
  clang-tools-extra/clangd/Preamble.cpp
  clang-tools-extra/clangd/Preamble.h
  clang-tools-extra/clangd/unittests/PreambleTests.cpp


Index: clang-tools-extra/clangd/unittests/PreambleTests.cpp
===
--- clang-tools-extra/clangd/unittests/PreambleTests.cpp
+++ clang-tools-extra/clangd/unittests/PreambleTests.cpp
@@ -30,7 +30,6 @@
 #include "llvm/Support/MemoryBuffer.h"
 #include "llvm/Support/ScopedPrinter.h"
 #include "llvm/Support/VirtualFileSystem.h"
-#include "llvm/Testing/Support/SupportHelpers.h"
 #include "gmock/gmock.h"
 #include "gtest/gtest-matchers.h"
 #include "gtest/gtest.h"
@@ -855,11 +854,12 @@
 ]]
 #pragma $y[[mark YY
 ]]
+#define BAZ
 
 #endif)cpp");
 auto AST = createPatchedAST(Code.code(), NewCode.code());
-// FIXME: Macros and marks have locations that need to be patched.
-EXPECT_THAT(AST->getMacros().Names, IsEmpty());
+EXPECT_THAT(AST->getMacros().Names.keys(),
+UnorderedElementsAreArray({"FOO", "BAR", "BAZ"}));
 EXPECT_THAT(AST->getMarks(),
 UnorderedElementsAre(Mark(NewCode.range("x"), " XX"),
  Mark(NewCode.range("y"), " YY")));
Index: clang-tools-extra/clangd/Preamble.h
===
--- clang-tools-extra/clangd/Preamble.h
+++ clang-tools-extra/clangd/Preamble.h
@@ -164,7 +164,7 @@
   static constexpr llvm::StringLiteral HeaderName = "__preamble_patch__.h";
 
   llvm::ArrayRef marks() const;
-  MainFileMacros mainFileMacros() const;
+  const MainFileMacros () const;
 
 private:
   static PreamblePatch create(llvm::StringRef FileName,
@@ -183,6 +183,7 @@
   PreambleBounds ModifiedBounds = {0, false};
   const PreambleData *Baseline = nullptr;
   std::vector PatchedMarks;
+  MainFileMacros PatchedMacros;
 };
 
 } // namespace clangd
Index: clang-tools-extra/clangd/Preamble.cpp
===
--- clang-tools-extra/clangd/Preamble.cpp
+++ clang-tools-extra/clangd/Preamble.cpp
@@ -325,6 +325,7 @@
   std::vector Lines;
   PreambleBounds Bounds = {0, false};
   std::vector Marks;
+  MainFileMacros Macros;
 };
 
 /// Scans the preprocessor directives in the preamble section of the file by
@@ -373,14 +374,15 @@
   if (!Action.BeginSourceFile(*Clang, Clang->getFrontendOpts().Inputs[0]))
 return error("failed BeginSourceFile");
   Preprocessor  = Clang->getPreprocessor();
+  const auto  = PP.getSourceManager();
   IncludeStructure Includes;
   Includes.collect(*Clang);
   ScannedPreamble SP;
   SP.Bounds = Bounds;
   PP.addPPCallbacks(
   std::make_unique(PP, SP.TextualDirectives));
-  PP.addPPCallbacks(
-  collectPragmaMarksCallback(PP.getSourceManager(), SP.Marks));
+  PP.addPPCallbacks(collectPragmaMarksCallback(SM, SP.Marks));
+  PP.addPPCallbacks(std::make_unique(SM, SP.Macros));
   if (llvm::Error Err = Action.Execute())
 return std::move(Err);
   Action.EndSourceFile();
@@ -859,6 +861,7 @@
 
   PP.PatchedDiags = patchDiags(Baseline.Diags, *BaselineScan, *ModifiedScan);
   PP.PatchedMarks = std::move(ModifiedScan->Marks);
+  PP.PatchedMacros = std::move(ModifiedScan->Macros);
   dlog("Created preamble patch: {0}", Patch.str());
   Patch.flush();
   return PP;
@@ -915,11 +918,10 @@
   return PatchedMarks;
 }
 
-MainFileMacros PreamblePatch::mainFileMacros() const {
+const MainFileMacros ::mainFileMacros() const {
   if (PatchContents.empty())
 return Baseline->Macros;
-  // FIXME: Patch main file macros.
-  return MainFileMacros();
+  return PatchedMacros;
 }
 } // namespace clangd
 } // namespace clang


Index: clang-tools-extra/clangd/unittests/PreambleTests.cpp
===
--- clang-tools-extra/clangd/unittests/PreambleTests.cpp
+++ clang-tools-extra/clangd/unittests/PreambleTests.cpp
@@ -30,7 +30,6 @@
 #include "llvm/Support/MemoryBuffer.h"
 #include "llvm/Support/ScopedPrinter.h"
 #include "llvm/Support/VirtualFileSystem.h"
-#include "llvm/Testing/Support/SupportHelpers.h"
 #include "gmock/gmock.h"
 #include "gtest/gtest-matchers.h"
 #include "gtest/gtest.h"
@@ -855,11 +854,12 @@
 ]]
 #pragma $y[[mark YY
 ]]
+#define BAZ
 
 #endif)cpp");
 auto AST = createPatchedAST(Code.code(), NewCode.code());
-// FIXME: Macros and marks have locations that need to be patched.
-EXPECT_THAT(AST->getMacros().Names, IsEmpty());
+EXPECT_THAT(AST->getMacros().Names.keys(),
+UnorderedElementsAreArray({"FOO", "BAR", "BAZ"}));
 EXPECT_THAT(AST->getMarks(),
 UnorderedElementsAre(Mark(NewCode.range("x"), " XX"),
 

[PATCH] D146026: [clangd] Patch PragmaMarks in preamble section of the file

[Kadir Cetinkaya via Phabricator via cfe-commits]

kadircet added inline comments.



Comment at: clang-tools-extra/clangd/Preamble.cpp:576
+
+std::vector getPragmaMarks(const ScannedPreamble ) {
+  std::vector Marks;

hokein wrote:
> any reason not calling `collectPragmaMarksCallback` just like the 
> https://reviews.llvm.org/D146028 for macros? is it for simplicity?
> 
> The pragma mark might be simple enough to parse ourselves, but it doesn't 
> handle well for some trivial/corner cases like (well we probably don't care 
> about these cases)
> 
> ```
> #pragma mark \
>abc
> ```
yeah it was for not running it unnecessarily, but you're right I guess it's not 
worth the discrepancy. using `collectPragmaMarksCallback` instead.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D146026/new/

https://reviews.llvm.org/D146026

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D146026: [clangd] Patch PragmaMarks in preamble section of the file

[Kadir Cetinkaya via Phabricator via cfe-commits]

kadircet updated this revision to Diff 505091.
kadircet marked an inline comment as done.
kadircet added a comment.

- Use collectPragmaMarksCallback


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D146026/new/

https://reviews.llvm.org/D146026

Files:
  clang-tools-extra/clangd/Preamble.cpp
  clang-tools-extra/clangd/Preamble.h
  clang-tools-extra/clangd/unittests/PreambleTests.cpp

Index: clang-tools-extra/clangd/unittests/PreambleTests.cpp
===
--- clang-tools-extra/clangd/unittests/PreambleTests.cpp
+++ clang-tools-extra/clangd/unittests/PreambleTests.cpp
@@ -829,6 +829,10 @@
   }
 }
 
+MATCHER_P2(Mark, Range, Text, "") {
+  return std::tie(arg.Rng, arg.Trivia) == std::tie(Range, Text);
+}
+
 TEST(PreamblePatch, MacroAndMarkHandling) {
   Config Cfg;
   Cfg.Diagnostics.AllowStalePreamble = true;
@@ -847,13 +851,18 @@
 #ifndef FOO
 #define FOO
 #define BAR
-#pragma mark XX
+#pragma $x[[mark XX
+]]
+#pragma $y[[mark YY
+]]
 
 #endif)cpp");
 auto AST = createPatchedAST(Code.code(), NewCode.code());
 // FIXME: Macros and marks have locations that need to be patched.
 EXPECT_THAT(AST->getMacros().Names, IsEmpty());
-EXPECT_THAT(AST->getMarks(), IsEmpty());
+EXPECT_THAT(AST->getMarks(),
+UnorderedElementsAre(Mark(NewCode.range("x"), " XX"),
+ Mark(NewCode.range("y"), " YY")));
   }
 }
 
Index: clang-tools-extra/clangd/Preamble.h
===
--- clang-tools-extra/clangd/Preamble.h
+++ clang-tools-extra/clangd/Preamble.h
@@ -182,6 +182,7 @@
   std::vector PatchedDiags;
   PreambleBounds ModifiedBounds = {0, false};
   const PreambleData *Baseline = nullptr;
+  std::vector PatchedMarks;
 };
 
 } // namespace clangd
Index: clang-tools-extra/clangd/Preamble.cpp
===
--- clang-tools-extra/clangd/Preamble.cpp
+++ clang-tools-extra/clangd/Preamble.cpp
@@ -15,7 +15,9 @@
 #include "Protocol.h"
 #include "SourceCode.h"
 #include "clang-include-cleaner/Record.h"
+#include "index/CanonicalIncludes.h"
 #include "support/Logger.h"
+#include "support/Path.h"
 #include "support/ThreadsafeFS.h"
 #include "support/Trace.h"
 #include "clang/AST/DeclTemplate.h"
@@ -27,6 +29,7 @@
 #include "clang/Basic/TokenKinds.h"
 #include "clang/Frontend/CompilerInvocation.h"
 #include "clang/Frontend/FrontendActions.h"
+#include "clang/Frontend/PrecompiledPreamble.h"
 #include "clang/Lex/HeaderSearch.h"
 #include "clang/Lex/Lexer.h"
 #include "clang/Lex/PPCallbacks.h"
@@ -42,6 +45,7 @@
 #include "llvm/ADT/StringExtras.h"
 #include "llvm/ADT/StringMap.h"
 #include "llvm/ADT/StringRef.h"
+#include "llvm/Support/Casting.h"
 #include "llvm/Support/Error.h"
 #include "llvm/Support/ErrorHandling.h"
 #include "llvm/Support/FormatVariadic.h"
@@ -50,10 +54,12 @@
 #include "llvm/Support/VirtualFileSystem.h"
 #include "llvm/Support/raw_ostream.h"
 #include 
+#include 
 #include 
 #include 
 #include 
 #include 
+#include 
 #include 
 #include 
 
@@ -318,6 +324,7 @@
   // Literal lines of the preamble contents.
   std::vector Lines;
   PreambleBounds Bounds = {0, false};
+  std::vector Marks;
 };
 
 /// Scans the preprocessor directives in the preamble section of the file by
@@ -372,6 +379,8 @@
   SP.Bounds = Bounds;
   PP.addPPCallbacks(
   std::make_unique(PP, SP.TextualDirectives));
+  PP.addPPCallbacks(
+  collectPragmaMarksCallback(PP.getSourceManager(), SP.Marks));
   if (llvm::Error Err = Action.Execute())
 return std::move(Err);
   Action.EndSourceFile();
@@ -849,6 +858,7 @@
   }
 
   PP.PatchedDiags = patchDiags(Baseline.Diags, *BaselineScan, *ModifiedScan);
+  PP.PatchedMarks = std::move(ModifiedScan->Marks);
   dlog("Created preamble patch: {0}", Patch.str());
   Patch.flush();
   return PP;
@@ -902,8 +912,7 @@
 llvm::ArrayRef PreamblePatch::marks() const {
   if (PatchContents.empty())
 return Baseline->Marks;
-  // FIXME: Patch pragma marks.
-  return {};
+  return PatchedMarks;
 }
 
 MainFileMacros PreamblePatch::mainFileMacros() const {
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D146024: [clangd] Drop stale macro and mark ranges

[Kadir Cetinkaya via Phabricator via cfe-commits]

This revision was automatically updated to reflect the committed changes.
kadircet marked an inline comment as done.
Closed by commit rGc11c2f5f6548: [clangd] Drop stale macro and mark ranges 
(authored by kadircet).

Changed prior to commit:
  https://reviews.llvm.org/D146024?vs=504990=505046#toc

Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D146024/new/

https://reviews.llvm.org/D146024

Files:
  clang-tools-extra/clangd/ParsedAST.cpp
  clang-tools-extra/clangd/Preamble.cpp
  clang-tools-extra/clangd/Preamble.h
  clang-tools-extra/clangd/unittests/PreambleTests.cpp

Index: clang-tools-extra/clangd/unittests/PreambleTests.cpp
===
--- clang-tools-extra/clangd/unittests/PreambleTests.cpp
+++ clang-tools-extra/clangd/unittests/PreambleTests.cpp
@@ -19,6 +19,7 @@
 #include "TestFS.h"
 #include "TestTU.h"
 #include "XRefs.h"
+#include "support/Context.h"
 #include "clang/Basic/SourceManager.h"
 #include "clang/Format/Format.h"
 #include "clang/Frontend/FrontendActions.h"
@@ -36,6 +37,7 @@
 #include 
 #include 
 #include 
+#include 
 #include 
 
 using testing::AllOf;
@@ -826,6 +828,35 @@
 EXPECT_THAT(*AST->getDiagnostics(), IsEmpty());
   }
 }
+
+TEST(PreamblePatch, MacroAndMarkHandling) {
+  Config Cfg;
+  Cfg.Diagnostics.AllowStalePreamble = true;
+  WithContextValue WithCfg(Config::Key, std::move(Cfg));
+
+  {
+Annotations Code(R"cpp(
+#ifndef FOO
+#define FOO
+// Some comments
+#pragma mark XX
+#define BAR
+
+#endif)cpp");
+Annotations NewCode(R"cpp(
+#ifndef FOO
+#define FOO
+#define BAR
+#pragma mark XX
+
+#endif)cpp");
+auto AST = createPatchedAST(Code.code(), NewCode.code());
+// FIXME: Macros and marks have locations that need to be patched.
+EXPECT_THAT(AST->getMacros().Names, IsEmpty());
+EXPECT_THAT(AST->getMarks(), IsEmpty());
+  }
+}
+
 } // namespace
 } // namespace clangd
 } // namespace clang
Index: clang-tools-extra/clangd/Preamble.h
===
--- clang-tools-extra/clangd/Preamble.h
+++ clang-tools-extra/clangd/Preamble.h
@@ -163,6 +163,9 @@
 
   static constexpr llvm::StringLiteral HeaderName = "__preamble_patch__.h";
 
+  llvm::ArrayRef marks() const;
+  MainFileMacros mainFileMacros() const;
+
 private:
   static PreamblePatch create(llvm::StringRef FileName,
   const ParseInputs ,
@@ -178,6 +181,7 @@
   // Diags that were attached to a line preserved in Modified contents.
   std::vector PatchedDiags;
   PreambleBounds ModifiedBounds = {0, false};
+  const PreambleData *Baseline = nullptr;
 };
 
 } // namespace clangd
Index: clang-tools-extra/clangd/Preamble.cpp
===
--- clang-tools-extra/clangd/Preamble.cpp
+++ clang-tools-extra/clangd/Preamble.cpp
@@ -7,6 +7,7 @@
 //===--===//
 
 #include "Preamble.h"
+#include "CollectMacros.h"
 #include "Compiler.h"
 #include "Config.h"
 #include "Diagnostics.h"
@@ -765,6 +766,7 @@
 return PreamblePatch::unmodified(Baseline);
 
   PreamblePatch PP;
+  PP.Baseline = 
   // This shouldn't coincide with any real file name.
   llvm::SmallString<128> PatchName;
   llvm::sys::path::append(PatchName, llvm::sys::path::parent_path(FileName),
@@ -886,6 +888,7 @@
 
 PreamblePatch PreamblePatch::unmodified(const PreambleData ) {
   PreamblePatch PP;
+  PP.Baseline = 
   PP.PreambleIncludes = Preamble.Includes.MainFileIncludes;
   PP.ModifiedBounds = Preamble.Preamble.getBounds();
   PP.PatchedDiags = Preamble.Diags;
@@ -896,5 +899,18 @@
   return PatchContents.empty() ||
  Config::current().Diagnostics.AllowStalePreamble;
 }
+llvm::ArrayRef PreamblePatch::marks() const {
+  if (PatchContents.empty())
+return Baseline->Marks;
+  // FIXME: Patch pragma marks.
+  return {};
+}
+
+MainFileMacros PreamblePatch::mainFileMacros() const {
+  if (PatchContents.empty())
+return Baseline->Macros;
+  // FIXME: Patch main file macros.
+  return MainFileMacros();
+}
 } // namespace clangd
 } // namespace clang
Index: clang-tools-extra/clangd/ParsedAST.cpp
===
--- clang-tools-extra/clangd/ParsedAST.cpp
+++ clang-tools-extra/clangd/ParsedAST.cpp
@@ -605,16 +605,15 @@
   // Copy over the macros in the preamble region of the main file, and combine
   // with non-preamble macros below.
   MainFileMacros Macros;
-  if (Preamble)
-Macros = Preamble->Macros;
+  std::vector Marks;
+  if (Preamble) {
+Macros = Patch->mainFileMacros();
+Marks = Patch->marks();
+  }
   Clang->getPreprocessor().addPPCallbacks(
   std::make_unique(Clang->getSourceManager(),
   Macros));
 
-  std::vector Marks;
-  // FIXME: We need to patch the marks for stale preambles.
-  if (Preamble)
-Marks = Preamble->Marks;
 

[PATCH] D146021: [Tooling/Inclusion] Index more sub std namespace symbols.

[Kadir Cetinkaya via Phabricator via cfe-commits]

kadircet accepted this revision.
kadircet added a comment.
This revision is now accepted and ready to land.

thanks!


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D146021/new/

https://reviews.llvm.org/D146021

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D146028: [clangd] Patch main file macros in preamble

[Kadir Cetinkaya via Phabricator via cfe-commits]

kadircet created this revision.
Herald added a subscriber: arphaman.
Herald added a project: All.
kadircet requested review of this revision.
Herald added subscribers: cfe-commits, MaskRay, ilya-biryukov.
Herald added a project: clang-tools-extra.

Depends on D146026 
Fixes https://github.com/clangd/clangd/issues/1537.


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D146028

Files:
  clang-tools-extra/clangd/Preamble.cpp
  clang-tools-extra/clangd/Preamble.h
  clang-tools-extra/clangd/unittests/PreambleTests.cpp


Index: clang-tools-extra/clangd/unittests/PreambleTests.cpp
===
--- clang-tools-extra/clangd/unittests/PreambleTests.cpp
+++ clang-tools-extra/clangd/unittests/PreambleTests.cpp
@@ -31,7 +31,6 @@
 #include "llvm/Support/MemoryBuffer.h"
 #include "llvm/Support/ScopedPrinter.h"
 #include "llvm/Support/VirtualFileSystem.h"
-#include "llvm/Testing/Support/SupportHelpers.h"
 #include "gmock/gmock.h"
 #include "gtest/gtest-matchers.h"
 #include "gtest/gtest.h"
@@ -856,11 +855,12 @@
 ]]
 #pragma $y[[mark YY
 ]]
+#define BAZ
 
 #endif)cpp");
 auto AST = createPatchedAST(Code.code(), NewCode.code());
-// FIXME: Macros and marks have locations that need to be patched.
-EXPECT_THAT(AST->getMacros().Names, IsEmpty());
+EXPECT_THAT(AST->getMacros().Names.keys(),
+UnorderedElementsAreArray({"FOO", "BAR", "BAZ"}));
 EXPECT_THAT(AST->getMarks(),
 UnorderedElementsAre(Mark(NewCode.range("x"), " XX"),
  Mark(NewCode.range("y"), " YY")));
Index: clang-tools-extra/clangd/Preamble.h
===
--- clang-tools-extra/clangd/Preamble.h
+++ clang-tools-extra/clangd/Preamble.h
@@ -183,6 +183,7 @@
   PreambleBounds ModifiedBounds = {0, false};
   const PreambleData *Baseline = nullptr;
   std::vector PatchedMarks;
+  MainFileMacros PatchedMacros;
 };
 
 } // namespace clangd
Index: clang-tools-extra/clangd/Preamble.cpp
===
--- clang-tools-extra/clangd/Preamble.cpp
+++ clang-tools-extra/clangd/Preamble.cpp
@@ -324,6 +324,7 @@
   // Literal lines of the preamble contents.
   std::vector Lines;
   PreambleBounds Bounds = {0, false};
+  MainFileMacros Macros;
 };
 
 /// Scans the preprocessor directives in the preamble section of the file by
@@ -378,6 +379,8 @@
   SP.Bounds = Bounds;
   PP.addPPCallbacks(
   std::make_unique(PP, SP.TextualDirectives));
+  PP.addPPCallbacks(std::make_unique(
+  PP.getSourceManager(), SP.Macros));
   if (llvm::Error Err = Action.Execute())
 return std::move(Err);
   Action.EndSourceFile();
@@ -879,6 +882,7 @@
 
   PP.PatchedDiags = patchDiags(Baseline.Diags, *BaselineScan, *ModifiedScan);
   PP.PatchedMarks = getPragmaMarks(*ModifiedScan);
+  PP.PatchedMacros = std::move(ModifiedScan->Macros);
   dlog("Created preamble patch: {0}", Patch.str());
   Patch.flush();
   return PP;
@@ -938,8 +942,7 @@
 MainFileMacros PreamblePatch::mainFileMacros() const {
   if (PatchContents.empty())
 return Baseline->Macros;
-  // FIXME: Patch main file macros.
-  return MainFileMacros();
+  return PatchedMacros;
 }
 } // namespace clangd
 } // namespace clang


Index: clang-tools-extra/clangd/unittests/PreambleTests.cpp
===
--- clang-tools-extra/clangd/unittests/PreambleTests.cpp
+++ clang-tools-extra/clangd/unittests/PreambleTests.cpp
@@ -31,7 +31,6 @@
 #include "llvm/Support/MemoryBuffer.h"
 #include "llvm/Support/ScopedPrinter.h"
 #include "llvm/Support/VirtualFileSystem.h"
-#include "llvm/Testing/Support/SupportHelpers.h"
 #include "gmock/gmock.h"
 #include "gtest/gtest-matchers.h"
 #include "gtest/gtest.h"
@@ -856,11 +855,12 @@
 ]]
 #pragma $y[[mark YY
 ]]
+#define BAZ
 
 #endif)cpp");
 auto AST = createPatchedAST(Code.code(), NewCode.code());
-// FIXME: Macros and marks have locations that need to be patched.
-EXPECT_THAT(AST->getMacros().Names, IsEmpty());
+EXPECT_THAT(AST->getMacros().Names.keys(),
+UnorderedElementsAreArray({"FOO", "BAR", "BAZ"}));
 EXPECT_THAT(AST->getMarks(),
 UnorderedElementsAre(Mark(NewCode.range("x"), " XX"),
  Mark(NewCode.range("y"), " YY")));
Index: clang-tools-extra/clangd/Preamble.h
===
--- clang-tools-extra/clangd/Preamble.h
+++ clang-tools-extra/clangd/Preamble.h
@@ -183,6 +183,7 @@
   PreambleBounds ModifiedBounds = {0, false};
   const PreambleData *Baseline = nullptr;
   std::vector PatchedMarks;
+  MainFileMacros PatchedMacros;
 };
 
 } // namespace clangd
Index: clang-tools-extra/clangd/Preamble.cpp
===
--- clang-tools-extra/clangd/Preamble.cpp
+++ 

[PATCH] D146026: [clangd] Patch PragmaMarks in preamble section of the file

[Kadir Cetinkaya via Phabricator via cfe-commits]

kadircet created this revision.
kadircet added a reviewer: hokein.
Herald added a subscriber: arphaman.
Herald added a project: All.
kadircet requested review of this revision.
Herald added subscribers: cfe-commits, MaskRay, ilya-biryukov.
Herald added a project: clang-tools-extra.

Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D146026

Files:
  clang-tools-extra/clangd/Preamble.cpp
  clang-tools-extra/clangd/Preamble.h
  clang-tools-extra/clangd/unittests/PreambleTests.cpp

Index: clang-tools-extra/clangd/unittests/PreambleTests.cpp
===
--- clang-tools-extra/clangd/unittests/PreambleTests.cpp
+++ clang-tools-extra/clangd/unittests/PreambleTests.cpp
@@ -830,6 +830,10 @@
   }
 }
 
+MATCHER_P2(Mark, Range, Text, "") {
+  return std::tie(arg.Rng, arg.Trivia) == std::tie(Range, Text);
+}
+
 TEST(PreamblePatch, MacroAndMarkHandling) {
   Config Cfg;
   Cfg.Diagnostics.AllowStalePreamble = true;
@@ -848,13 +852,18 @@
 #ifndef FOO
 #define FOO
 #define BAR
-#pragma mark XX
+#pragma $x[[mark XX
+]]
+#pragma $y[[mark YY
+]]
 
 #endif)cpp");
 auto AST = createPatchedAST(Code.code(), NewCode.code());
 // FIXME: Macros and marks have locations that need to be patched.
 EXPECT_THAT(AST->getMacros().Names, IsEmpty());
-EXPECT_THAT(AST->getMarks(), IsEmpty());
+EXPECT_THAT(AST->getMarks(),
+UnorderedElementsAre(Mark(NewCode.range("x"), " XX"),
+ Mark(NewCode.range("y"), " YY")));
   }
 }
 
Index: clang-tools-extra/clangd/Preamble.h
===
--- clang-tools-extra/clangd/Preamble.h
+++ clang-tools-extra/clangd/Preamble.h
@@ -182,6 +182,7 @@
   std::vector PatchedDiags;
   PreambleBounds ModifiedBounds = {0, false};
   const PreambleData *Baseline = nullptr;
+  std::vector PatchedMarks;
 };
 
 } // namespace clangd
Index: clang-tools-extra/clangd/Preamble.cpp
===
--- clang-tools-extra/clangd/Preamble.cpp
+++ clang-tools-extra/clangd/Preamble.cpp
@@ -15,7 +15,9 @@
 #include "Protocol.h"
 #include "SourceCode.h"
 #include "clang-include-cleaner/Record.h"
+#include "index/CanonicalIncludes.h"
 #include "support/Logger.h"
+#include "support/Path.h"
 #include "support/ThreadsafeFS.h"
 #include "support/Trace.h"
 #include "clang/AST/DeclTemplate.h"
@@ -27,6 +29,7 @@
 #include "clang/Basic/TokenKinds.h"
 #include "clang/Frontend/CompilerInvocation.h"
 #include "clang/Frontend/FrontendActions.h"
+#include "clang/Frontend/PrecompiledPreamble.h"
 #include "clang/Lex/HeaderSearch.h"
 #include "clang/Lex/Lexer.h"
 #include "clang/Lex/PPCallbacks.h"
@@ -42,6 +45,7 @@
 #include "llvm/ADT/StringExtras.h"
 #include "llvm/ADT/StringMap.h"
 #include "llvm/ADT/StringRef.h"
+#include "llvm/Support/Casting.h"
 #include "llvm/Support/Error.h"
 #include "llvm/Support/ErrorHandling.h"
 #include "llvm/Support/FormatVariadic.h"
@@ -50,10 +54,12 @@
 #include "llvm/Support/VirtualFileSystem.h"
 #include "llvm/Support/raw_ostream.h"
 #include 
+#include 
 #include 
 #include 
 #include 
 #include 
+#include 
 #include 
 #include 
 
@@ -566,6 +572,29 @@
 return NewD;
   }
 };
+
+std::vector getPragmaMarks(const ScannedPreamble ) {
+  std::vector Marks;
+  for (size_t I = 0, E = Modified.Lines.size(); I != E; ++I) {
+llvm::StringRef Line = Modified.Lines[I].ltrim();
+size_t MarkOffset = Line.find("mark");
+if (MarkOffset == Line.npos)
+  continue;
+if (!Line.consume_front("#pragma"))
+  continue;
+Line = Line.ltrim();
+if (!Line.consume_front("mark"))
+  continue;
+PragmaMark  = Marks.emplace_back();
+PM.Trivia = Line.str();
+PM.Rng.start.line = I;
+// We are only skipping whitespaces and `#pragma` when calculating this
+// offset, hence no need to care about utf-16.
+PM.Rng.start.character = MarkOffset;
+PM.Rng.end.line = I + 1;
+  }
+  return Marks;
+}
 } // namespace
 
 std::shared_ptr
@@ -849,6 +878,7 @@
   }
 
   PP.PatchedDiags = patchDiags(Baseline.Diags, *BaselineScan, *ModifiedScan);
+  PP.PatchedMarks = getPragmaMarks(*ModifiedScan);
   dlog("Created preamble patch: {0}", Patch.str());
   Patch.flush();
   return PP;
@@ -902,8 +932,7 @@
 llvm::ArrayRef PreamblePatch::marks() const {
   if (PatchContents.empty())
 return Baseline->Marks;
-  // FIXME: Patch pragma marks.
-  return {};
+  return PatchedMarks;
 }
 
 MainFileMacros PreamblePatch::mainFileMacros() const {
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D146024: [clangd] Drop stale macro and mark ranges

[Kadir Cetinkaya via Phabricator via cfe-commits]

kadircet created this revision.
kadircet added a reviewer: hokein.
Herald added a subscriber: arphaman.
Herald added a project: All.
kadircet requested review of this revision.
Herald added subscribers: cfe-commits, MaskRay, ilya-biryukov.
Herald added a project: clang-tools-extra.

I'll follow up with patching of those ranges, this is to stop bleeding
mentioned in https://github.com/clangd/clangd/issues/1537.


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D146024

Files:
  clang-tools-extra/clangd/ParsedAST.cpp
  clang-tools-extra/clangd/Preamble.cpp
  clang-tools-extra/clangd/Preamble.h
  clang-tools-extra/clangd/unittests/PreambleTests.cpp

Index: clang-tools-extra/clangd/unittests/PreambleTests.cpp
===
--- clang-tools-extra/clangd/unittests/PreambleTests.cpp
+++ clang-tools-extra/clangd/unittests/PreambleTests.cpp
@@ -19,6 +19,8 @@
 #include "TestFS.h"
 #include "TestTU.h"
 #include "XRefs.h"
+#include "index/FileIndex.h"
+#include "support/Context.h"
 #include "clang/Basic/SourceManager.h"
 #include "clang/Format/Format.h"
 #include "clang/Frontend/FrontendActions.h"
@@ -36,6 +38,7 @@
 #include 
 #include 
 #include 
+#include 
 #include 
 
 using testing::AllOf;
@@ -826,6 +829,35 @@
 EXPECT_THAT(*AST->getDiagnostics(), IsEmpty());
   }
 }
+
+TEST(PreamblePatch, MacroAndMarkHandling) {
+  Config Cfg;
+  Cfg.Diagnostics.AllowStalePreamble = true;
+  WithContextValue WithCfg(Config::Key, std::move(Cfg));
+
+  {
+Annotations Code(R"cpp(
+#ifndef FOO
+#define FOO
+// Some comments
+#pragma mark XX
+#define BAR
+
+#endif)cpp");
+Annotations NewCode(R"cpp(
+#ifndef FOO
+#define FOO
+#define BAR
+#pragma mark XX
+
+#endif)cpp");
+auto AST = createPatchedAST(Code.code(), NewCode.code());
+// FIXME: Macros and marks have locations that need to be patched.
+EXPECT_THAT(AST->getMacros().Names, IsEmpty());
+EXPECT_THAT(AST->getMarks(), IsEmpty());
+  }
+}
+
 } // namespace
 } // namespace clangd
 } // namespace clang
Index: clang-tools-extra/clangd/Preamble.h
===
--- clang-tools-extra/clangd/Preamble.h
+++ clang-tools-extra/clangd/Preamble.h
@@ -163,6 +163,9 @@
 
   static constexpr llvm::StringLiteral HeaderName = "__preamble_patch__.h";
 
+  llvm::ArrayRef marks() const;
+  MainFileMacros mainFileMacros() const;
+
 private:
   static PreamblePatch create(llvm::StringRef FileName,
   const ParseInputs ,
@@ -178,6 +181,7 @@
   // Diags that were attached to a line preserved in Modified contents.
   std::vector PatchedDiags;
   PreambleBounds ModifiedBounds = {0, false};
+  const PreambleData *Baseline = nullptr;
 };
 
 } // namespace clangd
Index: clang-tools-extra/clangd/Preamble.cpp
===
--- clang-tools-extra/clangd/Preamble.cpp
+++ clang-tools-extra/clangd/Preamble.cpp
@@ -7,6 +7,7 @@
 //===--===//
 
 #include "Preamble.h"
+#include "CollectMacros.h"
 #include "Compiler.h"
 #include "Config.h"
 #include "Diagnostics.h"
@@ -765,6 +766,7 @@
 return PreamblePatch::unmodified(Baseline);
 
   PreamblePatch PP;
+  PP.Baseline = 
   // This shouldn't coincide with any real file name.
   llvm::SmallString<128> PatchName;
   llvm::sys::path::append(PatchName, llvm::sys::path::parent_path(FileName),
@@ -886,6 +888,7 @@
 
 PreamblePatch PreamblePatch::unmodified(const PreambleData ) {
   PreamblePatch PP;
+  PP.Baseline = 
   PP.PreambleIncludes = Preamble.Includes.MainFileIncludes;
   PP.ModifiedBounds = Preamble.Preamble.getBounds();
   PP.PatchedDiags = Preamble.Diags;
@@ -896,5 +899,18 @@
   return PatchContents.empty() ||
  Config::current().Diagnostics.AllowStalePreamble;
 }
+llvm::ArrayRef PreamblePatch::marks() const {
+  if (PatchContents.empty())
+return Baseline->Marks;
+  // FIXME: Patch pragma marks.
+  return {};
+}
+
+MainFileMacros PreamblePatch::mainFileMacros() const {
+  if (PatchContents.empty())
+return Baseline->Macros;
+  // FIXME: Patch main file macros.
+  return MainFileMacros();
+}
 } // namespace clangd
 } // namespace clang
Index: clang-tools-extra/clangd/ParsedAST.cpp
===
--- clang-tools-extra/clangd/ParsedAST.cpp
+++ clang-tools-extra/clangd/ParsedAST.cpp
@@ -605,16 +605,15 @@
   // Copy over the macros in the preamble region of the main file, and combine
   // with non-preamble macros below.
   MainFileMacros Macros;
-  if (Preamble)
-Macros = Preamble->Macros;
+  std::vector Marks;
+  if (Preamble) {
+Macros = Patch->mainFileMacros();
+Marks = Patch->marks();
+  }
   Clang->getPreprocessor().addPPCallbacks(
   std::make_unique(Clang->getSourceManager(),
   Macros));
 
-  std::vector Marks;
-  // FIXME: We need to patch 

[PATCH] D145921: [clangd] Add missing unittests to build graph

[Kadir Cetinkaya via Phabricator via cfe-commits]

This revision was landed with ongoing or failed builds.
This revision was automatically updated to reflect the committed changes.
Closed by commit rGe26dad0a661e: [clangd] Add missing unittests to build graph 
(authored by kadircet).

Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D145921/new/

https://reviews.llvm.org/D145921

Files:
  clang-tools-extra/clangd/refactor/tweaks/SpecialMembers.cpp
  clang-tools-extra/clangd/unittests/CMakeLists.txt
  clang-tools-extra/clangd/unittests/support/FileCacheTests.cpp
  clang-tools-extra/clangd/unittests/tweaks/SpecialMembersTests.cpp

Index: clang-tools-extra/clangd/unittests/tweaks/SpecialMembersTests.cpp
===
--- clang-tools-extra/clangd/unittests/tweaks/SpecialMembersTests.cpp
+++ clang-tools-extra/clangd/unittests/tweaks/SpecialMembersTests.cpp
@@ -7,8 +7,6 @@
 //===--===//
 
 #include "TweakTesting.h"
-#include "gmock/gmock-matchers.h"
-#include "gmock/gmock.h"
 #include "gtest/gtest.h"
 
 namespace clang {
@@ -27,17 +25,17 @@
  "S =(S&&); S =(const S&);"
  "};");
 
-  const char *Output = R"cpp(struct S{S(const S &) = default;
-  S(S &&) = default;
-  S =(const S &) = default;
-  S =(S &&) = default;
+  const char *Output = R"cpp(struct S{S(const S&) = default;
+S(S&&) = default;
+S =(const S&) = default;
+S =(S&&) = default;
 };)cpp";
   EXPECT_EQ(apply("struct ^S{};"), Output);
 
-  Output = R"cpp(struct S{S(const S &) = default;
-S(S &&) = default;
-S =(const S &) = delete;
-S =(S &&) = delete;
+  Output = R"cpp(struct S{S(const S&) = default;
+S(S&&) = default;
+S =(const S&) = delete;
+S =(S&&) = delete;
 int& ref;};)cpp";
   EXPECT_EQ(apply("struct ^S{int& ref;};"), Output);
 }
Index: clang-tools-extra/clangd/unittests/support/FileCacheTests.cpp
===
--- clang-tools-extra/clangd/unittests/support/FileCacheTests.cpp
+++ clang-tools-extra/clangd/unittests/support/FileCacheTests.cpp
@@ -11,9 +11,9 @@
 #include "TestFS.h"
 #include "gmock/gmock.h"
 #include "gtest/gtest.h"
-#include 
 #include 
 #include 
+#include 
 
 namespace clang {
 namespace clangd {
@@ -34,10 +34,10 @@
   FS.Files.erase(testPath("foo.cc"));
   }
 
-  std::string get(std::chrono::steady_clock::time_point FreshTime,
-  bool ExpectParse) const {
+  std::pair
+  get(std::chrono::steady_clock::time_point FreshTime) const {
 bool GotParse = false;
-bool GotRead;
+bool GotRead = false;
 std::string Result;
 read(
 FS, FreshTime,
@@ -49,12 +49,14 @@
   GotRead = true;
   Result = Value;
 });
-EXPECT_EQ(GotParse, ExpectParse);
 EXPECT_TRUE(GotRead);
-return Result;
+return {Result, GotParse};
   }
 };
 
+MATCHER_P(Parsed, Value, "") { return arg.second && arg.first == Value; }
+MATCHER_P(Cached, Value, "") { return !arg.second && arg.first == Value; }
+
 TEST(FileCacheTest, Invalidation) {
   TestCache C;
 
@@ -62,20 +64,20 @@
   auto MustBeFresh = StaleOK + std::chrono::hours(1);
 
   C.setContents("a");
-  EXPECT_EQ("a", C.get(StaleOK, /*ExpectParse=*/true)) << "Parsed first time";
-  EXPECT_EQ("a", C.get(StaleOK, /*ExpectParse=*/false)) << "Cached (time)";
-  EXPECT_EQ("a", C.get(MustBeFresh, /*ExpectParse=*/false)) << "Cached (stat)";
+  EXPECT_THAT(C.get(StaleOK), Parsed("a")) << "Parsed first time";
+  EXPECT_THAT(C.get(StaleOK), Cached("a")) << "Cached (time)";
+  EXPECT_THAT(C.get(MustBeFresh), Cached("a")) << "Cached (stat)";
   C.setContents("bb");
-  EXPECT_EQ("a", C.get(StaleOK, /*ExpectParse=*/false)) << "Cached (time)";
-  EXPECT_EQ("bb", C.get(MustBeFresh, /*ExpectParse=*/true)) << "Size changed";
-  EXPECT_EQ("bb", C.get(MustBeFresh, /*ExpectParse=*/true)) << "Cached (stat)";
+  EXPECT_THAT(C.get(StaleOK), Cached("a")) << "Cached (time)";
+  EXPECT_THAT(C.get(MustBeFresh), Parsed("bb")) << "Size changed";
+  EXPECT_THAT(C.get(MustBeFresh), Cached("bb")) << "Cached (stat)";
   C.setContents(nullptr);
-  EXPECT_EQ("bb", C.get(StaleOK, /*ExpectParse=*/false)) << "Cached (time)";
-  EXPECT_EQ("", C.get(MustBeFresh, /*ExpectParse=*/true)) << "Stat failed";
-  EXPECT_EQ("", C.get(MustBeFresh, /*ExpectParse=*/false)) << "Cached (404)";
+  EXPECT_THAT(C.get(StaleOK), Cached("bb")) << "Cached (time)";
+  EXPECT_THAT(C.get(MustBeFresh), Parsed("")) << "stat failed";
+  EXPECT_THAT(C.get(MustBeFresh), Cached("")) << "Cached (404)";
   C.setContents("bb"); // Match the previous stat values!
-  EXPECT_EQ("", C.get(StaleOK, /*ExpectParse=*/false)) << "Cached (time)";
-  EXPECT_EQ("bb", C.get(MustBeFresh, /*ExpectParse=*/true)) << "Size changed";
+  EXPECT_THAT(C.get(StaleOK), Cached("")) << "Cached (time)";
+  EXPECT_THAT(C.get(MustBeFresh), Parsed("bb")) << "Size changed";
 }
 
 } // namespace
Index: 

[PATCH] D143436: [clangd] Move standard options adaptor to CommandMangler

[Kadir Cetinkaya via Phabricator via cfe-commits]

kadircet accepted this revision.
kadircet added a comment.
This revision is now accepted and ready to land.

thanks, lgtm!




Comment at: clang-tools-extra/clangd/CompileCommands.cpp:199
+  // use/change working directory, which ExpandResponseFiles doesn't).
+  FS = llvm::vfs::getRealFileSystem();
+}

DmitryPolukhin wrote:
> kadircet wrote:
> > getting real filesystem is cheap, no need to make this part of the state, 
> > just inline it into `tooling::addExpandedResponseFiles` call.
> We need version of `tooling::addExpandedResponseFiles` with FS as an argument 
> for testing at least, see 
> https://github.com/llvm/llvm-project/blob/main/clang/unittests/Tooling/CompilationDatabaseTest.cpp#L968
>  switching it to using real filesystem IMHO makes things worse. So moved this 
> code to the call site.
> So moved this code to the call site.

I was also trying to say that. so all good.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D143436/new/

https://reviews.llvm.org/D143436

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D145921: [clangd] Add missing unittests to build graph

[Kadir Cetinkaya via Phabricator via cfe-commits]

kadircet marked an inline comment as done.
kadircet added inline comments.



Comment at: clang-tools-extra/clangd/unittests/tweaks/SpecialMembersTests.cpp:21
   EXPECT_UNAVAILABLE("struct S { ^ };");
-  EXPECT_UNAVAILABLE("union ^U {};");
+  EXPECT_AVAILABLE("union ^U {};");
   EXPECT_AVAILABLE("struct ^S { S(const S&); S(S&&); };");

hokein wrote:
> the old behavior (UNAVAILABLE) seems more reasonable to me, I have a fix in 
> https://reviews.llvm.org/D145922.
i don't see much reason for disabling on unions, but don't have a strong 
preference.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D145921/new/

https://reviews.llvm.org/D145921

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D145921: [clangd] Add missing unittests to build graph

[Kadir Cetinkaya via Phabricator via cfe-commits]

kadircet updated this revision to Diff 504597.
kadircet marked an inline comment as done.
kadircet added a comment.

- Disable special members tweaks on unions


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D145921/new/

https://reviews.llvm.org/D145921

Files:
  clang-tools-extra/clangd/refactor/tweaks/SpecialMembers.cpp
  clang-tools-extra/clangd/unittests/CMakeLists.txt
  clang-tools-extra/clangd/unittests/support/FileCacheTests.cpp
  clang-tools-extra/clangd/unittests/tweaks/SpecialMembersTests.cpp

Index: clang-tools-extra/clangd/unittests/tweaks/SpecialMembersTests.cpp
===
--- clang-tools-extra/clangd/unittests/tweaks/SpecialMembersTests.cpp
+++ clang-tools-extra/clangd/unittests/tweaks/SpecialMembersTests.cpp
@@ -7,8 +7,6 @@
 //===--===//
 
 #include "TweakTesting.h"
-#include "gmock/gmock-matchers.h"
-#include "gmock/gmock.h"
 #include "gtest/gtest.h"
 
 namespace clang {
@@ -27,17 +25,17 @@
  "S =(S&&); S =(const S&);"
  "};");
 
-  const char *Output = R"cpp(struct S{S(const S &) = default;
-  S(S &&) = default;
-  S =(const S &) = default;
-  S =(S &&) = default;
+  const char *Output = R"cpp(struct S{S(const S&) = default;
+S(S&&) = default;
+S =(const S&) = default;
+S =(S&&) = default;
 };)cpp";
   EXPECT_EQ(apply("struct ^S{};"), Output);
 
-  Output = R"cpp(struct S{S(const S &) = default;
-S(S &&) = default;
-S =(const S &) = delete;
-S =(S &&) = delete;
+  Output = R"cpp(struct S{S(const S&) = default;
+S(S&&) = default;
+S =(const S&) = delete;
+S =(S&&) = delete;
 int& ref;};)cpp";
   EXPECT_EQ(apply("struct ^S{int& ref;};"), Output);
 }
Index: clang-tools-extra/clangd/unittests/support/FileCacheTests.cpp
===
--- clang-tools-extra/clangd/unittests/support/FileCacheTests.cpp
+++ clang-tools-extra/clangd/unittests/support/FileCacheTests.cpp
@@ -11,9 +11,9 @@
 #include "TestFS.h"
 #include "gmock/gmock.h"
 #include "gtest/gtest.h"
-#include 
 #include 
 #include 
+#include 
 
 namespace clang {
 namespace clangd {
@@ -34,10 +34,10 @@
   FS.Files.erase(testPath("foo.cc"));
   }
 
-  std::string get(std::chrono::steady_clock::time_point FreshTime,
-  bool ExpectParse) const {
+  std::pair
+  get(std::chrono::steady_clock::time_point FreshTime) const {
 bool GotParse = false;
-bool GotRead;
+bool GotRead = false;
 std::string Result;
 read(
 FS, FreshTime,
@@ -49,12 +49,14 @@
   GotRead = true;
   Result = Value;
 });
-EXPECT_EQ(GotParse, ExpectParse);
 EXPECT_TRUE(GotRead);
-return Result;
+return {Result, GotParse};
   }
 };
 
+MATCHER_P(Parsed, Value, "") { return arg.second && arg.first == Value; }
+MATCHER_P(Cached, Value, "") { return !arg.second && arg.first == Value; }
+
 TEST(FileCacheTest, Invalidation) {
   TestCache C;
 
@@ -62,20 +64,20 @@
   auto MustBeFresh = StaleOK + std::chrono::hours(1);
 
   C.setContents("a");
-  EXPECT_EQ("a", C.get(StaleOK, /*ExpectParse=*/true)) << "Parsed first time";
-  EXPECT_EQ("a", C.get(StaleOK, /*ExpectParse=*/false)) << "Cached (time)";
-  EXPECT_EQ("a", C.get(MustBeFresh, /*ExpectParse=*/false)) << "Cached (stat)";
+  EXPECT_THAT(C.get(StaleOK), Parsed("a")) << "Parsed first time";
+  EXPECT_THAT(C.get(StaleOK), Cached("a")) << "Cached (time)";
+  EXPECT_THAT(C.get(MustBeFresh), Cached("a")) << "Cached (stat)";
   C.setContents("bb");
-  EXPECT_EQ("a", C.get(StaleOK, /*ExpectParse=*/false)) << "Cached (time)";
-  EXPECT_EQ("bb", C.get(MustBeFresh, /*ExpectParse=*/true)) << "Size changed";
-  EXPECT_EQ("bb", C.get(MustBeFresh, /*ExpectParse=*/true)) << "Cached (stat)";
+  EXPECT_THAT(C.get(StaleOK), Cached("a")) << "Cached (time)";
+  EXPECT_THAT(C.get(MustBeFresh), Parsed("bb")) << "Size changed";
+  EXPECT_THAT(C.get(MustBeFresh), Cached("bb")) << "Cached (stat)";
   C.setContents(nullptr);
-  EXPECT_EQ("bb", C.get(StaleOK, /*ExpectParse=*/false)) << "Cached (time)";
-  EXPECT_EQ("", C.get(MustBeFresh, /*ExpectParse=*/true)) << "Stat failed";
-  EXPECT_EQ("", C.get(MustBeFresh, /*ExpectParse=*/false)) << "Cached (404)";
+  EXPECT_THAT(C.get(StaleOK), Cached("bb")) << "Cached (time)";
+  EXPECT_THAT(C.get(MustBeFresh), Parsed("")) << "stat failed";
+  EXPECT_THAT(C.get(MustBeFresh), Cached("")) << "Cached (404)";
   C.setContents("bb"); // Match the previous stat values!
-  EXPECT_EQ("", C.get(StaleOK, /*ExpectParse=*/false)) << "Cached (time)";
-  EXPECT_EQ("bb", C.get(MustBeFresh, /*ExpectParse=*/true)) << "Size changed";
+  EXPECT_THAT(C.get(StaleOK), Cached("")) << "Cached (time)";
+  EXPECT_THAT(C.get(MustBeFresh), Parsed("bb")) << "Size changed";
 }
 
 } // namespace
Index: clang-tools-extra/clangd/unittests/CMakeLists.txt

[PATCH] D127184: [clangd] Add to header map

[Kadir Cetinkaya via Phabricator via cfe-commits]

kadircet added a comment.

we can also handle them through the stdlib symbol mappings, see 
https://github.com/llvm/llvm-project/issues/61373


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D127184/new/

https://reviews.llvm.org/D127184

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D145921: [clangd] Add missing unittests to build graph

[Kadir Cetinkaya via Phabricator via cfe-commits]

kadircet created this revision.
kadircet added reviewers: sammccall, hokein.
Herald added a subscriber: arphaman.
Herald added a project: All.
kadircet requested review of this revision.
Herald added subscribers: cfe-commits, MaskRay, ilya-biryukov.
Herald added a project: clang-tools-extra.

Also fix tests


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D145921

Files:
  clang-tools-extra/clangd/refactor/tweaks/SpecialMembers.cpp
  clang-tools-extra/clangd/unittests/CMakeLists.txt
  clang-tools-extra/clangd/unittests/support/FileCacheTests.cpp
  clang-tools-extra/clangd/unittests/tweaks/SpecialMembersTests.cpp

Index: clang-tools-extra/clangd/unittests/tweaks/SpecialMembersTests.cpp
===
--- clang-tools-extra/clangd/unittests/tweaks/SpecialMembersTests.cpp
+++ clang-tools-extra/clangd/unittests/tweaks/SpecialMembersTests.cpp
@@ -7,8 +7,6 @@
 //===--===//
 
 #include "TweakTesting.h"
-#include "gmock/gmock-matchers.h"
-#include "gmock/gmock.h"
 #include "gtest/gtest.h"
 
 namespace clang {
@@ -20,24 +18,24 @@
 TEST_F(SpecialMembersTest, Test) {
   EXPECT_AVAILABLE("struct ^S {};");
   EXPECT_UNAVAILABLE("struct S { ^ };");
-  EXPECT_UNAVAILABLE("union ^U {};");
+  EXPECT_AVAILABLE("union ^U {};");
   EXPECT_AVAILABLE("struct ^S { S(const S&); S(S&&); };");
   EXPECT_UNAVAILABLE("struct ^S {"
  "S(const S&); S(S&&);"
  "S =(S&&); S =(const S&);"
  "};");
 
-  const char *Output = R"cpp(struct S{S(const S &) = default;
-  S(S &&) = default;
-  S =(const S &) = default;
-  S =(S &&) = default;
+  const char *Output = R"cpp(struct S{S(const S&) = default;
+S(S&&) = default;
+S =(const S&) = default;
+S =(S&&) = default;
 };)cpp";
   EXPECT_EQ(apply("struct ^S{};"), Output);
 
-  Output = R"cpp(struct S{S(const S &) = default;
-S(S &&) = default;
-S =(const S &) = delete;
-S =(S &&) = delete;
+  Output = R"cpp(struct S{S(const S&) = default;
+S(S&&) = default;
+S =(const S&) = delete;
+S =(S&&) = delete;
 int& ref;};)cpp";
   EXPECT_EQ(apply("struct ^S{int& ref;};"), Output);
 }
Index: clang-tools-extra/clangd/unittests/support/FileCacheTests.cpp
===
--- clang-tools-extra/clangd/unittests/support/FileCacheTests.cpp
+++ clang-tools-extra/clangd/unittests/support/FileCacheTests.cpp
@@ -11,9 +11,9 @@
 #include "TestFS.h"
 #include "gmock/gmock.h"
 #include "gtest/gtest.h"
-#include 
 #include 
 #include 
+#include 
 
 namespace clang {
 namespace clangd {
@@ -34,10 +34,10 @@
   FS.Files.erase(testPath("foo.cc"));
   }
 
-  std::string get(std::chrono::steady_clock::time_point FreshTime,
-  bool ExpectParse) const {
+  std::pair
+  get(std::chrono::steady_clock::time_point FreshTime) const {
 bool GotParse = false;
-bool GotRead;
+bool GotRead = false;
 std::string Result;
 read(
 FS, FreshTime,
@@ -49,12 +49,14 @@
   GotRead = true;
   Result = Value;
 });
-EXPECT_EQ(GotParse, ExpectParse);
 EXPECT_TRUE(GotRead);
-return Result;
+return {Result, GotParse};
   }
 };
 
+MATCHER_P(Parsed, Value, "") { return arg.second && arg.first == Value; }
+MATCHER_P(Cached, Value, "") { return !arg.second && arg.first == Value; }
+
 TEST(FileCacheTest, Invalidation) {
   TestCache C;
 
@@ -62,20 +64,20 @@
   auto MustBeFresh = StaleOK + std::chrono::hours(1);
 
   C.setContents("a");
-  EXPECT_EQ("a", C.get(StaleOK, /*ExpectParse=*/true)) << "Parsed first time";
-  EXPECT_EQ("a", C.get(StaleOK, /*ExpectParse=*/false)) << "Cached (time)";
-  EXPECT_EQ("a", C.get(MustBeFresh, /*ExpectParse=*/false)) << "Cached (stat)";
+  EXPECT_THAT(C.get(StaleOK), Parsed("a")) << "Parsed first time";
+  EXPECT_THAT(C.get(StaleOK), Cached("a")) << "Cached (time)";
+  EXPECT_THAT(C.get(MustBeFresh), Cached("a")) << "Cached (stat)";
   C.setContents("bb");
-  EXPECT_EQ("a", C.get(StaleOK, /*ExpectParse=*/false)) << "Cached (time)";
-  EXPECT_EQ("bb", C.get(MustBeFresh, /*ExpectParse=*/true)) << "Size changed";
-  EXPECT_EQ("bb", C.get(MustBeFresh, /*ExpectParse=*/true)) << "Cached (stat)";
+  EXPECT_THAT(C.get(StaleOK), Cached("a")) << "Cached (time)";
+  EXPECT_THAT(C.get(MustBeFresh), Parsed("bb")) << "Size changed";
+  EXPECT_THAT(C.get(MustBeFresh), Cached("bb")) << "Cached (stat)";
   C.setContents(nullptr);
-  EXPECT_EQ("bb", C.get(StaleOK, /*ExpectParse=*/false)) << "Cached (time)";
-  EXPECT_EQ("", C.get(MustBeFresh, /*ExpectParse=*/true)) << "Stat failed";
-  EXPECT_EQ("", C.get(MustBeFresh, /*ExpectParse=*/false)) << "Cached (404)";
+  EXPECT_THAT(C.get(StaleOK), Cached("bb")) << "Cached (time)";
+  EXPECT_THAT(C.get(MustBeFresh), Parsed("")) << "stat failed";
+  EXPECT_THAT(C.get(MustBeFresh), Cached("")) << "Cached (404)";
   C.setContents("bb"); // Match the previous stat values!

[PATCH] D145773: [clangd] UnusedIncludes: Strict config now uses the include-cleaner-library implementation.

[Kadir Cetinkaya via Phabricator via cfe-commits]

kadircet accepted this revision.
kadircet added inline comments.
This revision is now accepted and ready to land.



Comment at: clang-tools-extra/clangd/ConfigCompile.cpp:438
+ .map("Experiment",
+  Config::IncludesPolicy::Strict) // for backward
+  // compatibility

hokein wrote:
> kadircet wrote:
> > i think we should at least be emitting a diagnostics to encourage people 
> > for moving back to strict, so what about something like:
> > ```
> > if (F.UnusuedIncludes) {
> >   auto Val = compileEnum; // only for Strict and None
> >   if (!Val && **F.UnusedIncludes == "Experiment") {
> >diag(Warning, "Experiment is deprecated for UnusedIncludes, use 
> > Strict instead.", F.UnusedIncludes.Range);
> >Val = Config::IncludesPolicy::Strict;
> >   }
> > }
> > ```
> I thought it was not worth a diagnostic because this flag was introduced 
> recently, and we have never advertised it to open-source users.
> 
> But the flag is in the recent clangd16 release, so it probably justifies the 
> value. 
> 
> BTW, looks like we forgot to update the release notes for clangd16, 
> https://github.com/llvm/llvm-project/blob/release/16.x/clang-tools-extra/docs/ReleaseNotes.rst#improvements-to-clangd
>  is empty.
> 
> But the flag is in the recent clangd16 release, so it probably justifies the 
> value.

right.

> BTW, looks like we forgot to update the release notes for clangd16, 
> https://github.com/llvm/llvm-project/blob/release/16.x/clang-tools-extra/docs/ReleaseNotes.rst#improvements-to-clangd
>  is empty.

yeah, i've aslo noticed that will take a look. but regarding this feature, I 
don't think we should be advertising `Experiment` there, right?



Comment at: clang-tools-extra/clangd/IncludeCleaner.cpp:769
-  Cfg.Diagnostics.UnusedIncludes == Config::IncludesPolicy::Strict
-  ? computeUnusedIncludes(AST)
-  : Findings.UnusedIncludes,

hokein wrote:
> kadircet wrote:
> > can you also delete `computeUnusedIncludes` and its friends (also from the 
> > tests)?
> this is in plan, but in a separate patch, https://reviews.llvm.org/D145776
as mentioned on that patch, i think it's better to land them in single step, to 
make sure we can revert a single patch if we want the old behavior rather than 
multiple.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D145773/new/

https://reviews.llvm.org/D145773

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D145776: [clangd] Remove the classical clangd-own unsued-include implementation.

[Kadir Cetinkaya via Phabricator via cfe-commits]

kadircet accepted this revision.
kadircet added a comment.
This revision is now accepted and ready to land.

i'd still merge this with the previous patch, as all of this is dead code after 
config option deletion. so it'd be better to just revert a single patch if we 
want to restore the old behavior, rather than two.




Comment at: clang-tools-extra/clangd/unittests/IncludeCleanerTests.cpp:720
 
 TEST(IncludeCleaner, RecursiveInclusion) {
   TestTU TU;

i don't think this test is meaningful for our current usage of the 
include-cleaner library, feel free to drop it.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D145776/new/

https://reviews.llvm.org/D145776

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D145773: [clangd] UnusedIncludes: Strict config now uses the include-cleaner-library implementation.

[Kadir Cetinkaya via Phabricator via cfe-commits]

kadircet added inline comments.



Comment at: clang-tools-extra/clangd/ConfigCompile.cpp:438
+ .map("Experiment",
+  Config::IncludesPolicy::Strict) // for backward
+  // compatibility

i think we should at least be emitting a diagnostics to encourage people for 
moving back to strict, so what about something like:
```
if (F.UnusuedIncludes) {
  auto Val = compileEnum; // only for Strict and None
  if (!Val && **F.UnusedIncludes == "Experiment") {
   diag(Warning, "Experiment is deprecated for UnusedIncludes, use Strict 
instead.", F.UnusedIncludes.Range);
   Val = Config::IncludesPolicy::Strict;
  }
}
```



Comment at: clang-tools-extra/clangd/IncludeCleaner.cpp:769
-  Cfg.Diagnostics.UnusedIncludes == Config::IncludesPolicy::Strict
-  ? computeUnusedIncludes(AST)
-  : Findings.UnusedIncludes,

can you also delete `computeUnusedIncludes` and its friends (also from the 
tests)?


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D145773/new/

https://reviews.llvm.org/D145773

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D143436: [clangd] Move standard options adaptor to CommandMangler

[Kadir Cetinkaya via Phabricator via cfe-commits]

kadircet added a comment.

> @kadircet it is obvious that there is something in this diff that causes this 
> hesitancy in accepting it. I'm ready to keep iterating on the solution but I 
> need a clue what needs the improvement. Please comment.

sorry for the late reply, i was on vacation and just started catching up with 
things. this looks great, thanks!




Comment at: clang-tools-extra/clangd/CompileCommands.cpp:199
+  // use/change working directory, which ExpandResponseFiles doesn't).
+  FS = llvm::vfs::getRealFileSystem();
+}

getting real filesystem is cheap, no need to make this part of the state, just 
inline it into `tooling::addExpandedResponseFiles` call.



Comment at: clang-tools-extra/clangd/CompileCommands.cpp:222
+  tooling::addExpandedResponseFiles(Cmd, Command.Directory, Tokenizer, *FS);
+  tooling::addTargetAndModeForProgramName(Cmd, Cmd.front());
   auto  = clang::driver::getDriverOptTable();

nridge wrote:
> DmitryPolukhin wrote:
> > nridge wrote:
> > > DmitryPolukhin wrote:
> > > > nridge wrote:
> > > > > nridge wrote:
> > > > > > The target needs to be added **after** the call to 
> > > > > > `SystemIncludeExtractor` later in this function (this is what 
> > > > > > D138546 is trying to fix). The reason is that 
> > > > > > `SystemIncludeExtractor` includes any `--target` flag in the 
> > > > > > compiler driver being queried for system includes, which may be 
> > > > > > gcc, which does not support `--target`.
> > > > > (I guess we could make that change separately in D138546, but if 
> > > > > we're changing the place where the `--target` is added in this patch, 
> > > > > I figure we might as well move it directly to the desired place.)
> > > > I think there are order problems here:
> > > > - we need `--driver-mode=cl` injected here to make check on line #229 
> > > > work as expected
> > > > - if we don't inject it driver mode here, command line edits won't see 
> > > > the change; see how I modified test 
> > > > clangd/unittests/CompileCommandsTests.cpp line #203, with D138546 edits 
> > > > were not applied properly to driver mode
> > > > 
> > > > I'll double check how it works on Windows but I expect that moving it 
> > > > after SystemIncludeExtractor will break proper driver mode detection.
> > > > I think there are order problems here:
> > > > - we need `--driver-mode=cl` injected here to make check on line #229 
> > > > work as expected
> > > 
> > > Looking at the [line in 
> > > question](https://searchfox.org/llvm/rev/0cbb8ec030e23c0e13331b5d54155def8c901b36/clang-tools-extra/clangd/CompileCommands.cpp#213),
> > >  it calls `driver::getDriverMode()` which [falls back on extracting the 
> > > driver 
> > > mode](https://searchfox.org/llvm/rev/0cbb8ec030e23c0e13331b5d54155def8c901b36/clang/lib/Driver/Driver.cpp#6407)
> > >  from the program name anyways -- so I think that should be fine.
> > > 
> > > > - if we don't inject it driver mode here, command line edits won't see 
> > > > the change; see how I modified test 
> > > > clangd/unittests/CompileCommandsTests.cpp line #203, with D138546 edits 
> > > > were not applied properly to driver mode
> > > 
> > > I'm not following the motivation for this test. Is there any real-world 
> > > use case which requires the command line edits seeing the `--driver-mode` 
> > > parameter?
> > > I'm not following the motivation for this test. Is there any real-world 
> > > use case which requires the command line edits seeing the `--driver-mode` 
> > > parameter?
> > 
> > @nridge, it will break existing behaviour when user was able to remove 
> > these inserted options like this `CompileFlags: {Remove: 
> > ['--driver-mode=*', '--target=*']}`. If I move call of 
> > `addTargetAndModeForProgramName` after `SystemIncludeExtractor`r, inserted 
> > options will stay in list. User may only override it with 
> > `CompileFlags.Compiler` but it will override all compilers. I don't know is 
> > it real problem or not so if you think I should still move it after 
> > `SystemIncludeExtractor`, I'll do it.
> > @nridge, it will break existing behaviour when user was able to remove 
> > these inserted options like this `CompileFlags: {Remove: 
> > ['--driver-mode=*', '--target=*']}`.
> 
> Thanks. I'll leave the final word to @kadircet here but in my opinion, being 
> able to remove those flags via `Remove` is not important; as you say, the 
> user can change the `Compiler` to influence the driver mode if needed.
agreed. `addTargetAndModeForProgramName` will add these flags only when they're 
not present. deleting --driver-mode or --target, without providing a fixed 
replacement doesn't sound like a real use case.



Comment at: 
clang-tools-extra/clangd/test/Inputs/did-change-configuration-params.args:1
+-Wpedantic

you can test `expandResponseFile` behaviour through unittests as well, see 

[PATCH] D145553: [Tooling/Inclusion] Add missing placerholder _1 symbols.

[Kadir Cetinkaya via Phabricator via cfe-commits]

kadircet accepted this revision.
kadircet added inline comments.
This revision is now accepted and ready to land.



Comment at: clang/lib/Tooling/Inclusions/Stdlib/StdSpecialSymbolMap.inc:260
+// text, which are not handled by the script.
+// N is an implementation-defined number, 10 should be enough in practice.
+SYMBOL(_1, std::placeholders::, )

libstdc++ actually has _N=29 (don't know why not 30, as it starts from 1 
actually :D) you mind listing it all the way till 29?


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D145553/new/

https://reviews.llvm.org/D145553

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D143496: [clangd] Add support for missing includes analysis.

[Kadir Cetinkaya via Phabricator via cfe-commits]

kadircet accepted this revision.
kadircet added a comment.
This revision is now accepted and ready to land.

thanks for bearing with me, let's ship it!




Comment at: clang-tools-extra/clangd/IncludeCleaner.cpp:456
+  tooling::IncludeStyle IncludeStyle;
+  auto DefaultStyle = format::getStyle(
+  format::DefaultFormatStyle, AST.tuPath(), format::DefaultFallbackStyle,

nit: this could be shorter with
```
auto FileStyle = format::getStyle(..);
if (!FileStyle) {
  elog("...");
  FileStyle = format::getLLVMStyle();
}
tooling::HeaderIncludes HeaderIncludes(AST.tuPath(), Code, 
FileStyle->IncludeStyle);
```



Comment at: clang-tools-extra/clangd/unittests/IncludeCleanerTests.cpp:448
+  auto Range = MainFile.range("b");
+  auto Start = positionToOffset(MainFile.code(), Range.start);
+  EXPECT_FALSE(Start.takeError());

nit:
```
size_t Start = llvm::cantFail(positionToOffset(MainFile.code(), Range.start));
size_t End = llvm::cantFail(positionToOffset(MainFile.code(), Range.end));
```

no need for `EXPECT_FALSE(..takeError())`s as `llvm::cantFail` will fail (no 
pun intended :P), `static_cast`s are also redundant



Comment at: clang-tools-extra/clangd/unittests/IncludeCleanerTests.cpp:458
+  EXPECT_THAT(Findings.MissingIncludes, ElementsAre(BInfo));
+  EXPECT_TRUE(BDecl);
+}

it'd be better to `ASSERT_TRUE(BDecl);` right after the `for loop`, as rest of 
the code will crash (and even trigger undefined behavior because we're 
dereferencing nullptr in failure case).

difference between `ASSERT_X` and `EXPECT_X` macros are, the former will stop 
execution of the particular test (hence we'll never trigger a nullptr deref 
with ASSERT_TRUE), whereas the latter just prints the failure, but doesn't 
abort the execution of test (hence helps print multiple failures at once, when 
they're non fatal).


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D143496/new/

https://reviews.llvm.org/D143496

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D143496: [clangd] Add support for missing includes analysis.

[Kadir Cetinkaya via Phabricator via cfe-commits]

kadircet added inline comments.



Comment at: clang-tools-extra/clangd/IncludeCleaner.cpp:456
+  auto IncludeStyle =
+  clang::format::getLLVMStyle().IncludeStyle;
+  auto DefaultStyle = clang::format::getStyle(

creating a copy of LLVM style unnecessarily all the time is not really great, 
can you move this into the failure case instead?

also you can drop the `clang::` here and elsewhere, as this code is already 
part of `clang::` namespace.



Comment at: clang-tools-extra/clangd/IncludeCleaner.cpp:457
+  clang::format::getLLVMStyle().IncludeStyle;
+  auto DefaultStyle = clang::format::getStyle(
+  clang::format::DefaultFormatStyle, MainFile->getName(),

as mentioned above we also need to make sure we're passing the relevant VFS 
instance inside the source manager, rather than using the real file system (as 
some clients rely on the VFS).



Comment at: clang-tools-extra/clangd/IncludeCleaner.cpp:458
+  auto DefaultStyle = clang::format::getStyle(
+  clang::format::DefaultFormatStyle, MainFile->getName(),
+  clang::format::DefaultFallbackStyle);

s/MainFile->getName()/AST.tuPath()/

to be consistent with other places.



Comment at: clang-tools-extra/clangd/IncludeCleaner.cpp:460
+  clang::format::DefaultFallbackStyle);
+  if (!DefaultStyle.takeError()) {
+IncludeStyle = DefaultStyle->IncludeStyle;

can you also `elog` this error? as it should be rare and when this goes wrong, 
having this mentioned in the logs are really useful for debugging (since the 
failure is actually outside of clangd, it usually means a malformed config file 
somewhere)



Comment at: clang-tools-extra/clangd/unittests/IncludeCleanerTests.cpp:426
+void foo() {
+  ^b();
+})cpp");

nit: instead of using a point, can you use a range here instead (i.e. `[[b]]`)? 
afterwards you can have a `FileRange` pointing at both offsets, rather than 
relying on the length of the identifier.



Comment at: clang-tools-extra/clangd/unittests/IncludeCleanerTests.cpp:446
+BDecl = CandidateDecl;
+include_cleaner::Symbol B{*D};
+auto Offset = positionToOffset(MainFile.code(), MainFile.point());

rest of the code here doesn't really belong to the for loop, can you take them 
out?


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D143496/new/

https://reviews.llvm.org/D143496

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D145228: [clangd] Add clangd headers to install targets

[Kadir Cetinkaya via Phabricator via cfe-commits]

kadircet requested changes to this revision.
kadircet added a comment.
This revision now requires changes to proceed.

i agree with Sam's concerns here. clangd isn't designed to be consumed as a 
library, but rather as a binary through LSP. increasing surface are here and 
letting people build applications on top of clangd internals would create extra 
maintenance burden that we're not equipped to support.

do you have any specific use case that's pending on this change, or is this a 
change that might allow things in the future? if you have a specific need it 
would be better if you can share it directly and we can look for other 
solutions instead.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D145228/new/

https://reviews.llvm.org/D145228

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D143496: [clangd] Add support for missing includes analysis.

[Kadir Cetinkaya via Phabricator via cfe-commits]

kadircet added a comment.

thanks, looks great!




Comment at: clang-tools-extra/clangd/IncludeCleaner.cpp:451
+  tooling::HeaderIncludes HeaderIncludes(AST.tuPath(), Code,
+ tooling::IncludeStyle{});
+  const SourceManager  = AST.getSourceManager();

we should respect the style configurations (sorry for missing this in previous 
iterations).

you can get the relevant style with: `clang::format::getStyle`, which has an 
[IncludeStyle](https://github.com/llvm/llvm-project/blob/main/clang/include/clang/Format/Format.h#L2188).
 in case the `getStyle` fails, we should fallback to 
`clang::format::getLLVMStyle` as we do in other places. you can get at the 
relevant VFS instance through sourcemanager.



Comment at: clang-tools-extra/clangd/IncludeCleaner.cpp:712
+  for (auto *Inc : ConvertedIncludes.match(H)) {
+if (Pragmas == nullptr || 
Pragmas->getPublic(Inc->Resolved).empty())
+  Satisfied = true;

you can directly use `!Pragmas->isPrivate(Inc->Resolved)` here, instead of 
getpublic



Comment at: clang-tools-extra/clangd/IncludeCleaner.cpp:712
+  for (auto *Inc : ConvertedIncludes.match(H)) {
+if (Pragmas == nullptr || 
Pragmas->getPublic(Inc->Resolved).empty())
+  Satisfied = true;

kadircet wrote:
> you can directly use `!Pragmas->isPrivate(Inc->Resolved)` here, instead of 
> getpublic
this check seems to be new. what's the reason for rejecting private providers? 
I can see that we might want to be conservative by not inserting private 
providers, but treating symbols as unsatisfied when a private provider is 
**already** included doesn't feel right. e.g. the code being analyzed might be 
allowed to depend on this private header, because it's also part of the 
library, or it's the public header that's exposing this private header. in such 
a scenario we shouldn't try to insert the public header again. is there a more 
concrete issue this code is trying to address?



Comment at: clang-tools-extra/clangd/unittests/IncludeCleanerTests.cpp:443
+std::string Name = 
llvm::dyn_cast(D)->getQualifiedNameAsString();
+if (Name != "b") {
+  continue;

nit: braces



Comment at: clang-tools-extra/clangd/unittests/IncludeCleanerTests.cpp:456
+
+// TODO(bakalova) examples with std::vector and IWYU private pragma still 
needed
+TEST(IncludeCleaner, GenerateMissingHeaderDiags) {

i think the example for `std::vector` is solid, and `IWYU pragma private` needs 
a little adjustment.



Comment at: clang-tools-extra/clangd/unittests/IncludeCleanerTests.cpp:469
+#include "header.h"
+#include "private.h"
+$insert_f[[]]$insert_vector[[]]

we should include private.h through some indirection (not public.h) to check 
`IWYU pragma private` spellings are respected.



Comment at: clang-tools-extra/clangd/unittests/IncludeCleanerTests.cpp:475
+
+  ns::$d[[Bar]] bar;
+  bar.d();

name this range as `bar` instead of `d`?



Comment at: clang-tools-extra/clangd/unittests/IncludeCleanerTests.cpp:479
+
+  buzz(); 
+

could you add a comment here saying this shouldn't be diagnosed?



Comment at: clang-tools-extra/clangd/unittests/IncludeCleanerTests.cpp:449
+include_cleaner::Symbol B{*D};
+syntax::FileRange BRange{SM, B.declaration().getBeginLoc(), 1};
+include_cleaner::Header Header{*SM.getFileManager().getFile("b.h")};

VitaNuo wrote:
> kadircet wrote:
> > this is pointing at the declaration inside `b.h` not to the reference 
> > inside the main file. are you sure this test passes?
> Yes, all the tests pass. 
> `D` is a `Decl` from the main file, otherwise it wouldn't have passed the 
> safeguard ` if 
> (!SM.isWrittenInMainFile(SM.getExpansionLoc(D->getLocation( continue;` 
> above.
this is passing because `bool BDeclFound;` is uninitialized above, if you set 
it to `bool BDeclFound = false;` you should see the test fail.

there's no declaration for `b` inside the main file, it's declared in `b.h` and 
**referenced** inside the main file. you still need to search for the decl 
(without the constraint of being written in main file), use it to build an 
include_cleaner::Symbol, and use a `clangd::Annotation` range for the range of 
the reference.

it might be easer to write this as:
```
const NamedDecl* B = nullptr;
for (...) {
 ...
 B = D;
}
ASSERT_TRUE(B);
// build expected diagnostic info based on B and check that it's equal to what 
we've produced
```


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D143496/new/

https://reviews.llvm.org/D143496

___
cfe-commits mailing list
cfe-commits@lists.llvm.org

[PATCH] D145365: [Tooling/Inclusion] Add missing index_sequence symbols.

[Kadir Cetinkaya via Phabricator via cfe-commits]

kadircet accepted this revision.
kadircet added inline comments.
This revision is now accepted and ready to land.



Comment at: clang/lib/Tooling/Inclusions/Stdlib/StdSpecialSymbolMap.inc:252
+// Remove them when the cppreference offline archive catches up.
+SYMBOL(index_sequence, std::, )
+SYMBOL(index_sequence_for, std::, )

hokein wrote:
> kadircet wrote:
> > there's also `integer_sequence` see https://eel.is/c++draft/utility.syn
> The integer_sequence has been indexed in the symbol page, it is already in 
> `StdSymbolMap.inc`, 
> https://github.com/llvm/llvm-project/blob/main/clang/lib/Tooling/Inclusions/Stdlib/StdSymbolMap.inc#L1576
oops, thanks!


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D145365/new/

https://reviews.llvm.org/D145365

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D145365: [Tooling/Inclusion] Add missing index_sequence symbols.

[Kadir Cetinkaya via Phabricator via cfe-commits]

kadircet added inline comments.



Comment at: clang/lib/Tooling/Inclusions/Stdlib/StdSpecialSymbolMap.inc:252
+// Remove them when the cppreference offline archive catches up.
+SYMBOL(index_sequence, std::, )
+SYMBOL(index_sequence_for, std::, )

there's also `integer_sequence` see https://eel.is/c++draft/utility.syn


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D145365/new/

https://reviews.llvm.org/D145365

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D145364: [include-cleaner] Fix a crash on non-identifier-name symbols.

[Kadir Cetinkaya via Phabricator via cfe-commits]

kadircet accepted this revision.
kadircet added a comment.
This revision is now accepted and ready to land.

thanks, LGTM!




Comment at: clang-tools-extra/include-cleaner/test/nocrash.cpp:1
+// RUN: clang-include-cleaner %s --
+

hokein wrote:
> I have considered to add a unittest in `FindHeadersTest`, but that is not 
> trivial to do, the unittest is written based on the assumption that the 
> captured symbol name is a simple identifier (we use the it to find the symbol 
> based on a given text name).
> 
> Other ideas are welcome.
> 
i think this LG, any place that needs to test these APIs requires handles to 
the decls. So we either need to have a test that does complicated AST 
traversals to reach a particular node or test things through an integration 
test, something like this or a unittest on top of walkUsed/analyze. I don't 
think former is worth for a crash test and having a unittest based on 
walkUsed/analyze for checking a crash inside headersForSymbol is more confusing.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D145364/new/

https://reviews.llvm.org/D145364

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D144721: [Tooling/Inclusion] Add the generic abs symbol to the table.

[Kadir Cetinkaya via Phabricator via cfe-commits]

kadircet accepted this revision.
kadircet added a comment.
This revision is now accepted and ready to land.

oh right, i thought these were also provided by multiple headers, but looks 
like they're only provided by `cstdlib`, thanks!


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D144721/new/

https://reviews.llvm.org/D144721

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D144721: [Tooling/Inclusion] Add the generic abs symbol to the table.

[Kadir Cetinkaya via Phabricator via cfe-commits]

kadircet added a comment.

this fixes the `abs` in the mentioned issue, but leaves the friends out :( 
those are specifically `std::labs, std::llabs, std::imaxabs`


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D144721/new/

https://reviews.llvm.org/D144721

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D143496: [clangd] Add support for missing includes analysis.

[Kadir Cetinkaya via Phabricator via cfe-commits]

kadircet added a comment.

thanks! looks amazing, we're missing a little bit of test coverage though




Comment at: clang-tools-extra/clangd/IncludeCleaner.cpp:280
 
+bool isFilteredByConfig(const Config , llvm::StringRef HeaderSpelling) {
+  // Convert the path to Unix slashes and try to match against the filter.

s/HeaderSpelling/HeaderPath



Comment at: clang-tools-extra/clangd/IncludeCleaner.cpp:282
+  // Convert the path to Unix slashes and try to match against the filter.
+  llvm::SmallString<64> Path(HeaderSpelling);
+  llvm::sys::path::native(Path, llvm::sys::path::Style::posix);

s/Path/NormalizedPath



Comment at: clang-tools-extra/clangd/IncludeCleaner.cpp:414
+
+std::string findResolvedPath(const include_cleaner::Header ) {
+  std::string ResolvedPath;

what about just `resolvedPath`, if you'd rather keep the verb, i think `get` 
makes more sense than `find`. we're not really searching anything.



Comment at: clang-tools-extra/clangd/IncludeCleaner.cpp:418
+  case include_cleaner::Header::Physical:
+ResolvedPath = SymProvider.physical()->tryGetRealPathName();
+break;

nit: you can directly `return  SymProvider.physical()->tryGetRealPathName();` 
(same for other 2 cases) and have an `llvm_unreachable("Unknown symbol kind");` 
after the switch statement.



Comment at: clang-tools-extra/clangd/IncludeCleaner.cpp:421
+  case include_cleaner::Header::Standard:
+ResolvedPath = SymProvider.standard().name();
+break;

in this and the next case we need to trim `<>"`



Comment at: clang-tools-extra/clangd/IncludeCleaner.cpp:430
+
+std::string findSymbolName(const include_cleaner::Symbol ) {
+  std::string SymbolName;

same as above, either just `symbolName` or `get`



Comment at: clang-tools-extra/clangd/IncludeCleaner.cpp:434
+  case include_cleaner::Symbol::Macro:
+SymbolName = Sym.macro().Name->getName();
+break;

again you can just return here and below



Comment at: clang-tools-extra/clangd/IncludeCleaner.cpp:434
+  case include_cleaner::Symbol::Macro:
+SymbolName = Sym.macro().Name->getName();
+break;

kadircet wrote:
> again you can just return here and below
`getName` is a StringRef, and unfortunately there are some platforms (like 
darwin) that don't support implicit conversion from stringrefs to std::string. 
so can you call `.str()` explicitly in the end?



Comment at: clang-tools-extra/clangd/IncludeCleaner.cpp:754
+  IncludeCleanerFindings Findings;
+  if (Cfg.Diagnostics.MissingIncludes != Config::IncludesPolicy::None ||
+  Cfg.Diagnostics.UnusedIncludes != Config::IncludesPolicy::None) {

i think for now this should be
```
if (Cfg.Diagnostics.MissingIncludes == Config::IncludesPolicy::Strict ||
  Cfg.Diagnostics.UnusedIncludes == Config::IncludesPolicy::Experiment) {
```

otherwise we'll run both legacy and new analysis for `UnusedIncludes == Strict`



Comment at: clang-tools-extra/clangd/unittests/IncludeCleanerTests.cpp:449
+include_cleaner::Symbol B{*D};
+syntax::FileRange BRange{SM, B.declaration().getBeginLoc(), 1};
+include_cleaner::Header Header{*SM.getFileManager().getFile("b.h")};

this is pointing at the declaration inside `b.h` not to the reference inside 
the main file. are you sure this test passes?



Comment at: clang-tools-extra/clangd/unittests/IncludeCleanerTests.cpp:469
+  $d[[d]]();
+  $f[[f]]();
+})cpp");

can you also add a reference (and declaration) for std::vector, and have an 
IWYU private pragma in one of the headers to test code paths that spell 
verbatim and standard headers? also having some diagnostic suppressed via 
`IgnoreHeaders` is important to check



Comment at: clang-tools-extra/clangd/unittests/IncludeCleanerTests.cpp:481
+  TU.AdditionalFiles["system/e.h"] = guard("#include ");
+  TU.AdditionalFiles["system/f.h"] = guard("void f();");
+  TU.ExtraArgs.push_back("-isystem" + testPath("system"));

can you make one of these names qualified? e.g. `namespace ns { struct Bar { 
void f(); }; }`


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D143496/new/

https://reviews.llvm.org/D143496

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D144713: [Tooling/Includsion] Add the missing NULL symbol to the table.

[Kadir Cetinkaya via Phabricator via cfe-commits]

kadircet accepted this revision.
kadircet added a comment.
This revision is now accepted and ready to land.

thanks!


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D144713/new/

https://reviews.llvm.org/D144713

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D144713: [Tooling/Includsion] Add the missing NULL symbol to the table.

[Kadir Cetinkaya via Phabricator via cfe-commits]

kadircet added inline comments.



Comment at: clang/lib/Tooling/Inclusions/Stdlib/StdSpecialSymbolMap.inc:242
+// Macros
+SYMBOL(NULL, , )
+SYMBOL(NULL, , )

we use `None` for global namespace rather than leaving empty. let's keep it 
that way?


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D144713/new/

https://reviews.llvm.org/D144713

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D144646: [Tooling][Inclusions] Add c-header and global namespace alternatives for size_t

[Kadir Cetinkaya via Phabricator via cfe-commits]

This revision was landed with ongoing or failed builds.
This revision was automatically updated to reflect the committed changes.
Closed by commit rG385c8cd3cd66: [Tooling][Inclusions] Add c-header and global 
namespace alternatives for size_t (authored by kadircet).

Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D144646/new/

https://reviews.llvm.org/D144646

Files:
  clang/lib/Tooling/Inclusions/Stdlib/StdSpecialSymbolMap.inc


Index: clang/lib/Tooling/Inclusions/Stdlib/StdSpecialSymbolMap.inc
===
--- clang/lib/Tooling/Inclusions/Stdlib/StdSpecialSymbolMap.inc
+++ clang/lib/Tooling/Inclusions/Stdlib/StdSpecialSymbolMap.inc
@@ -22,6 +22,20 @@
 SYMBOL(size_t, std::, )
 SYMBOL(size_t, std::, )
 SYMBOL(size_t, std::, )
+SYMBOL(size_t, None, )
+SYMBOL(size_t, None, )
+SYMBOL(size_t, None, )
+SYMBOL(size_t, None, )
+SYMBOL(size_t, None, )
+SYMBOL(size_t, None, )
+SYMBOL(size_t, None, )
+SYMBOL(size_t, None, )
+SYMBOL(size_t, None, )
+SYMBOL(size_t, None, )
+SYMBOL(size_t, None, )
+SYMBOL(size_t, None, )
+SYMBOL(size_t, None, )
+SYMBOL(size_t, None, )
 SYMBOL(unwrap_ref_decay, std::, )
 SYMBOL(unwrap_ref_decay, std::, )
 SYMBOL(unwrap_reference, std::, )


Index: clang/lib/Tooling/Inclusions/Stdlib/StdSpecialSymbolMap.inc
===
--- clang/lib/Tooling/Inclusions/Stdlib/StdSpecialSymbolMap.inc
+++ clang/lib/Tooling/Inclusions/Stdlib/StdSpecialSymbolMap.inc
@@ -22,6 +22,20 @@
 SYMBOL(size_t, std::, )
 SYMBOL(size_t, std::, )
 SYMBOL(size_t, std::, )
+SYMBOL(size_t, None, )
+SYMBOL(size_t, None, )
+SYMBOL(size_t, None, )
+SYMBOL(size_t, None, )
+SYMBOL(size_t, None, )
+SYMBOL(size_t, None, )
+SYMBOL(size_t, None, )
+SYMBOL(size_t, None, )
+SYMBOL(size_t, None, )
+SYMBOL(size_t, None, )
+SYMBOL(size_t, None, )
+SYMBOL(size_t, None, )
+SYMBOL(size_t, None, )
+SYMBOL(size_t, None, )
 SYMBOL(unwrap_ref_decay, std::, )
 SYMBOL(unwrap_ref_decay, std::, )
 SYMBOL(unwrap_reference, std::, )
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D144708: [clangd] Fix UB in scanPreamble

[Kadir Cetinkaya via Phabricator via cfe-commits]

This revision was landed with ongoing or failed builds.
This revision was automatically updated to reflect the committed changes.
kadircet marked an inline comment as done.
Closed by commit rGf393e1f6b3b4: [clangd] Fix UB in scanPreamble (authored by 
kadircet).

Changed prior to commit:
  https://reviews.llvm.org/D144708?vs=500101=500112#toc

Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D144708/new/

https://reviews.llvm.org/D144708

Files:
  clang-tools-extra/clangd/Preamble.cpp


Index: clang-tools-extra/clangd/Preamble.cpp
===
--- clang-tools-extra/clangd/Preamble.cpp
+++ clang-tools-extra/clangd/Preamble.cpp
@@ -332,6 +332,8 @@
   EmptyFS FS;
   // Build and run Preprocessor over the preamble.
   ParseInputs PI;
+  // Memory buffers below expect null-terminated && non-null strings. So make
+  // sure to always use PI.Contents!
   PI.Contents = Contents.str();
   PI.TFS = 
   PI.CompileCommand = Cmd;
@@ -345,8 +347,8 @@
   // twice. However, it's important to precisely follow the preamble bounds 
used
   // elsewhere.
   auto Bounds = ComputePreambleBounds(*CI->getLangOpts(), *ContentsBuffer, 0);
-  auto PreambleContents =
-  llvm::MemoryBuffer::getMemBufferCopy(Contents.substr(0, Bounds.Size));
+  auto PreambleContents = llvm::MemoryBuffer::getMemBufferCopy(
+  llvm::StringRef(PI.Contents).take_front(Bounds.Size));
   auto Clang = prepareCompilerInstance(
   std::move(CI), nullptr, std::move(PreambleContents),
   // Provide an empty FS to prevent preprocessor from performing IO. This
@@ -739,9 +741,8 @@
   //   whole preamble, which is terribly slow.
   // - If scanning for Modified fails, cannot figure out newly added ones so
   //   there's nothing to do but generate an empty patch.
-  auto BaselineScan = scanPreamble(
-  // Contents needs to be null-terminated.
-  Baseline.Preamble.getContents(), Modified.CompileCommand);
+  auto BaselineScan =
+  scanPreamble(Baseline.Preamble.getContents(), Modified.CompileCommand);
   if (!BaselineScan) {
 elog("Failed to scan baseline of {0}: {1}", FileName,
  BaselineScan.takeError());


Index: clang-tools-extra/clangd/Preamble.cpp
===
--- clang-tools-extra/clangd/Preamble.cpp
+++ clang-tools-extra/clangd/Preamble.cpp
@@ -332,6 +332,8 @@
   EmptyFS FS;
   // Build and run Preprocessor over the preamble.
   ParseInputs PI;
+  // Memory buffers below expect null-terminated && non-null strings. So make
+  // sure to always use PI.Contents!
   PI.Contents = Contents.str();
   PI.TFS = 
   PI.CompileCommand = Cmd;
@@ -345,8 +347,8 @@
   // twice. However, it's important to precisely follow the preamble bounds used
   // elsewhere.
   auto Bounds = ComputePreambleBounds(*CI->getLangOpts(), *ContentsBuffer, 0);
-  auto PreambleContents =
-  llvm::MemoryBuffer::getMemBufferCopy(Contents.substr(0, Bounds.Size));
+  auto PreambleContents = llvm::MemoryBuffer::getMemBufferCopy(
+  llvm::StringRef(PI.Contents).take_front(Bounds.Size));
   auto Clang = prepareCompilerInstance(
   std::move(CI), nullptr, std::move(PreambleContents),
   // Provide an empty FS to prevent preprocessor from performing IO. This
@@ -739,9 +741,8 @@
   //   whole preamble, which is terribly slow.
   // - If scanning for Modified fails, cannot figure out newly added ones so
   //   there's nothing to do but generate an empty patch.
-  auto BaselineScan = scanPreamble(
-  // Contents needs to be null-terminated.
-  Baseline.Preamble.getContents(), Modified.CompileCommand);
+  auto BaselineScan =
+  scanPreamble(Baseline.Preamble.getContents(), Modified.CompileCommand);
   if (!BaselineScan) {
 elog("Failed to scan baseline of {0}: {1}", FileName,
  BaselineScan.takeError());
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D144708: [clangd] Fix UB in scanPreamble

[Kadir Cetinkaya via Phabricator via cfe-commits]

kadircet marked an inline comment as done.
kadircet added inline comments.



Comment at: clang-tools-extra/clangd/Preamble.cpp:351
   auto PreambleContents =
-  llvm::MemoryBuffer::getMemBufferCopy(Contents.substr(0, Bounds.Size));
+  llvm::MemoryBuffer::getMemBufferCopy(PI.Contents.substr(0, Bounds.Size));
   auto Clang = prepareCompilerInstance(

hokein wrote:
> Looks like we have an extra cost here -- the std::string.substr will 
> construct a new string everytime, we could save it by using 
> `llvm::StringRef(PI.Contents).substr(0, Bounds.Size)`.
thanks!


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D144708/new/

https://reviews.llvm.org/D144708

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D143096: [clangd] Provide patched diagnostics with preamble patch

[Kadir Cetinkaya via Phabricator via cfe-commits]

kadircet added a comment.

thanks for the ping! see the discussion in 
https://reviews.llvm.org/D142890#4149679


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D143096/new/

https://reviews.llvm.org/D143096

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D142890: [clangd] Add config option for fast diagnostics mode

[Kadir Cetinkaya via Phabricator via cfe-commits]

kadircet added a comment.

In D142890#4149181 , @vitalybuka 
wrote:

> One of your patches likely introduced UB 
> https://lab.llvm.org/buildbot/#/builders/85/builds/14558
> Can you please take a look?

Thanks a lot Vitaly and sorry for not noticing this myself :(
This sounds to me like a bad consequence between an empty stringref and 
memorybuffers. I've put together https://reviews.llvm.org/D144706 to make sure 
we can prevent this from happening in the library, but getting it reviewed will 
probably take time.
In the meanwhile I've sent out https://reviews.llvm.org/D144708 to stop the 
bleeding.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D142890/new/

https://reviews.llvm.org/D142890

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D144708: [clangd] Fix UB in scanPreamble

[Kadir Cetinkaya via Phabricator via cfe-commits]

kadircet created this revision.
kadircet added a reviewer: hokein.
Herald added a subscriber: arphaman.
Herald added a project: All.
kadircet requested review of this revision.
Herald added subscribers: cfe-commits, MaskRay, ilya-biryukov.
Herald added a project: clang-tools-extra.

getMemBufferCopy triggers an UB when it receives a default constructed
StringRef. Make sure that we're always passing the null-terminated string
created in ParseInputs throughout the scanPreamble.


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D144708

Files:
  clang-tools-extra/clangd/Preamble.cpp


Index: clang-tools-extra/clangd/Preamble.cpp
===
--- clang-tools-extra/clangd/Preamble.cpp
+++ clang-tools-extra/clangd/Preamble.cpp
@@ -332,6 +332,8 @@
   EmptyFS FS;
   // Build and run Preprocessor over the preamble.
   ParseInputs PI;
+  // Memory buffers below expect null-terminated && non-null strings. So make
+  // sure to always use PI.Contents!
   PI.Contents = Contents.str();
   PI.TFS = 
   PI.CompileCommand = Cmd;
@@ -346,7 +348,7 @@
   // elsewhere.
   auto Bounds = ComputePreambleBounds(*CI->getLangOpts(), *ContentsBuffer, 0);
   auto PreambleContents =
-  llvm::MemoryBuffer::getMemBufferCopy(Contents.substr(0, Bounds.Size));
+  llvm::MemoryBuffer::getMemBufferCopy(PI.Contents.substr(0, Bounds.Size));
   auto Clang = prepareCompilerInstance(
   std::move(CI), nullptr, std::move(PreambleContents),
   // Provide an empty FS to prevent preprocessor from performing IO. This
@@ -371,7 +373,7 @@
 return std::move(Err);
   Action.EndSourceFile();
   SP.Includes = std::move(Includes.MainFileIncludes);
-  llvm::append_range(SP.Lines, llvm::split(Contents, "\n"));
+  llvm::append_range(SP.Lines, llvm::split(PI.Contents, "\n"));
   return SP;
 }
 
@@ -739,9 +741,8 @@
   //   whole preamble, which is terribly slow.
   // - If scanning for Modified fails, cannot figure out newly added ones so
   //   there's nothing to do but generate an empty patch.
-  auto BaselineScan = scanPreamble(
-  // Contents needs to be null-terminated.
-  Baseline.Preamble.getContents(), Modified.CompileCommand);
+  auto BaselineScan =
+  scanPreamble(Baseline.Preamble.getContents(), Modified.CompileCommand);
   if (!BaselineScan) {
 elog("Failed to scan baseline of {0}: {1}", FileName,
  BaselineScan.takeError());


Index: clang-tools-extra/clangd/Preamble.cpp
===
--- clang-tools-extra/clangd/Preamble.cpp
+++ clang-tools-extra/clangd/Preamble.cpp
@@ -332,6 +332,8 @@
   EmptyFS FS;
   // Build and run Preprocessor over the preamble.
   ParseInputs PI;
+  // Memory buffers below expect null-terminated && non-null strings. So make
+  // sure to always use PI.Contents!
   PI.Contents = Contents.str();
   PI.TFS = 
   PI.CompileCommand = Cmd;
@@ -346,7 +348,7 @@
   // elsewhere.
   auto Bounds = ComputePreambleBounds(*CI->getLangOpts(), *ContentsBuffer, 0);
   auto PreambleContents =
-  llvm::MemoryBuffer::getMemBufferCopy(Contents.substr(0, Bounds.Size));
+  llvm::MemoryBuffer::getMemBufferCopy(PI.Contents.substr(0, Bounds.Size));
   auto Clang = prepareCompilerInstance(
   std::move(CI), nullptr, std::move(PreambleContents),
   // Provide an empty FS to prevent preprocessor from performing IO. This
@@ -371,7 +373,7 @@
 return std::move(Err);
   Action.EndSourceFile();
   SP.Includes = std::move(Includes.MainFileIncludes);
-  llvm::append_range(SP.Lines, llvm::split(Contents, "\n"));
+  llvm::append_range(SP.Lines, llvm::split(PI.Contents, "\n"));
   return SP;
 }
 
@@ -739,9 +741,8 @@
   //   whole preamble, which is terribly slow.
   // - If scanning for Modified fails, cannot figure out newly added ones so
   //   there's nothing to do but generate an empty patch.
-  auto BaselineScan = scanPreamble(
-  // Contents needs to be null-terminated.
-  Baseline.Preamble.getContents(), Modified.CompileCommand);
+  auto BaselineScan =
+  scanPreamble(Baseline.Preamble.getContents(), Modified.CompileCommand);
   if (!BaselineScan) {
 elog("Failed to scan baseline of {0}: {1}", FileName,
  BaselineScan.takeError());
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D144646: [Tooling][Inclusions] Add c-header and global namespace alternatives for size_t

[Kadir Cetinkaya via Phabricator via cfe-commits]

kadircet created this revision.
kadircet added a reviewer: hokein.
Herald added a project: All.
kadircet requested review of this revision.
Herald added a project: clang.
Herald added a subscriber: cfe-commits.

Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D144646

Files:
  clang/lib/Tooling/Inclusions/Stdlib/StdSpecialSymbolMap.inc


Index: clang/lib/Tooling/Inclusions/Stdlib/StdSpecialSymbolMap.inc
===
--- clang/lib/Tooling/Inclusions/Stdlib/StdSpecialSymbolMap.inc
+++ clang/lib/Tooling/Inclusions/Stdlib/StdSpecialSymbolMap.inc
@@ -22,6 +22,20 @@
 SYMBOL(size_t, std::, )
 SYMBOL(size_t, std::, )
 SYMBOL(size_t, std::, )
+SYMBOL(size_t, None, )
+SYMBOL(size_t, None, )
+SYMBOL(size_t, None, )
+SYMBOL(size_t, None, )
+SYMBOL(size_t, None, )
+SYMBOL(size_t, None, )
+SYMBOL(size_t, None, )
+SYMBOL(size_t, None, )
+SYMBOL(size_t, None, )
+SYMBOL(size_t, None, )
+SYMBOL(size_t, None, )
+SYMBOL(size_t, None, )
+SYMBOL(size_t, None, )
+SYMBOL(size_t, None, )
 SYMBOL(unwrap_ref_decay, std::, )
 SYMBOL(unwrap_ref_decay, std::, )
 SYMBOL(unwrap_reference, std::, )


Index: clang/lib/Tooling/Inclusions/Stdlib/StdSpecialSymbolMap.inc
===
--- clang/lib/Tooling/Inclusions/Stdlib/StdSpecialSymbolMap.inc
+++ clang/lib/Tooling/Inclusions/Stdlib/StdSpecialSymbolMap.inc
@@ -22,6 +22,20 @@
 SYMBOL(size_t, std::, )
 SYMBOL(size_t, std::, )
 SYMBOL(size_t, std::, )
+SYMBOL(size_t, None, )
+SYMBOL(size_t, None, )
+SYMBOL(size_t, None, )
+SYMBOL(size_t, None, )
+SYMBOL(size_t, None, )
+SYMBOL(size_t, None, )
+SYMBOL(size_t, None, )
+SYMBOL(size_t, None, )
+SYMBOL(size_t, None, )
+SYMBOL(size_t, None, )
+SYMBOL(size_t, None, )
+SYMBOL(size_t, None, )
+SYMBOL(size_t, None, )
+SYMBOL(size_t, None, )
 SYMBOL(unwrap_ref_decay, std::, )
 SYMBOL(unwrap_ref_decay, std::, )
 SYMBOL(unwrap_reference, std::, )
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D144641: [clangd] Set diag data before emitting

[Kadir Cetinkaya via Phabricator via cfe-commits]

This revision was automatically updated to reflect the committed changes.
Closed by commit rG981e3a35a145: [clangd] Set diag data before emitting 
(authored by kadircet).

Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D144641/new/

https://reviews.llvm.org/D144641

Files:
  clang-tools-extra/clangd/Diagnostics.cpp
  clang-tools-extra/clangd/unittests/DiagnosticsTests.cpp


Index: clang-tools-extra/clangd/unittests/DiagnosticsTests.cpp
===
--- clang-tools-extra/clangd/unittests/DiagnosticsTests.cpp
+++ clang-tools-extra/clangd/unittests/DiagnosticsTests.cpp
@@ -23,6 +23,7 @@
 #include "clang/Basic/Diagnostic.h"
 #include "clang/Basic/DiagnosticSema.h"
 #include "llvm/ADT/StringRef.h"
+#include "llvm/Support/JSON.h"
 #include "llvm/Support/ScopedPrinter.h"
 #include "llvm/Support/TargetSelect.h"
 #include "gmock/gmock.h"
@@ -1012,6 +1013,7 @@
   D.Severity = DiagnosticsEngine::Error;
   D.File = "foo/bar/main.cpp";
   D.AbsFile = std::string(MainFile.file());
+  D.OpaqueData["test"] = "bar";
 
   clangd::Note NoteInMain;
   NoteInMain.Message = "declared somewhere in the main file";
@@ -1050,6 +1052,7 @@
 ../foo/baz/header.h:10:11:
 note: declared somewhere in the header file)";
   MainLSP.tags = {DiagnosticTag::Unnecessary};
+  MainLSP.data = D.OpaqueData;
 
   clangd::Diagnostic NoteInMainLSP;
   NoteInMainLSP.range = NoteInMain.Range;
Index: clang-tools-extra/clangd/Diagnostics.cpp
===
--- clang-tools-extra/clangd/Diagnostics.cpp
+++ clang-tools-extra/clangd/Diagnostics.cpp
@@ -526,6 +526,9 @@
 }
   }
   Main.tags = D.Tags;
+  // FIXME: Get rid of the copies here by taking in a mutable clangd::Diag.
+  for (auto  : D.OpaqueData)
+Main.data.insert({Entry.first, Entry.second});
   OutFn(std::move(Main), D.Fixes);
 
   // If we didn't emit the notes as relatedLocations, emit separate diagnostics
@@ -540,10 +543,6 @@
   Res.message = noteMessage(D, Note, Opts);
   OutFn(std::move(Res), llvm::ArrayRef());
 }
-
-  // FIXME: Get rid of the copies here by taking in a mutable clangd::Diag.
-  for (auto  : D.OpaqueData)
-Main.data.insert({Entry.first, Entry.second});
 }
 
 int getSeverity(DiagnosticsEngine::Level L) {


Index: clang-tools-extra/clangd/unittests/DiagnosticsTests.cpp
===
--- clang-tools-extra/clangd/unittests/DiagnosticsTests.cpp
+++ clang-tools-extra/clangd/unittests/DiagnosticsTests.cpp
@@ -23,6 +23,7 @@
 #include "clang/Basic/Diagnostic.h"
 #include "clang/Basic/DiagnosticSema.h"
 #include "llvm/ADT/StringRef.h"
+#include "llvm/Support/JSON.h"
 #include "llvm/Support/ScopedPrinter.h"
 #include "llvm/Support/TargetSelect.h"
 #include "gmock/gmock.h"
@@ -1012,6 +1013,7 @@
   D.Severity = DiagnosticsEngine::Error;
   D.File = "foo/bar/main.cpp";
   D.AbsFile = std::string(MainFile.file());
+  D.OpaqueData["test"] = "bar";
 
   clangd::Note NoteInMain;
   NoteInMain.Message = "declared somewhere in the main file";
@@ -1050,6 +1052,7 @@
 ../foo/baz/header.h:10:11:
 note: declared somewhere in the header file)";
   MainLSP.tags = {DiagnosticTag::Unnecessary};
+  MainLSP.data = D.OpaqueData;
 
   clangd::Diagnostic NoteInMainLSP;
   NoteInMainLSP.range = NoteInMain.Range;
Index: clang-tools-extra/clangd/Diagnostics.cpp
===
--- clang-tools-extra/clangd/Diagnostics.cpp
+++ clang-tools-extra/clangd/Diagnostics.cpp
@@ -526,6 +526,9 @@
 }
   }
   Main.tags = D.Tags;
+  // FIXME: Get rid of the copies here by taking in a mutable clangd::Diag.
+  for (auto  : D.OpaqueData)
+Main.data.insert({Entry.first, Entry.second});
   OutFn(std::move(Main), D.Fixes);
 
   // If we didn't emit the notes as relatedLocations, emit separate diagnostics
@@ -540,10 +543,6 @@
   Res.message = noteMessage(D, Note, Opts);
   OutFn(std::move(Res), llvm::ArrayRef());
 }
-
-  // FIXME: Get rid of the copies here by taking in a mutable clangd::Diag.
-  for (auto  : D.OpaqueData)
-Main.data.insert({Entry.first, Entry.second});
 }
 
 int getSeverity(DiagnosticsEngine::Level L) {
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D144641: [clangd] Set diag data before emitting

[Kadir Cetinkaya via Phabricator via cfe-commits]

kadircet created this revision.
kadircet added a reviewer: hokein.
Herald added a subscriber: arphaman.
Herald added a project: All.
kadircet requested review of this revision.
Herald added subscribers: cfe-commits, MaskRay, ilya-biryukov.
Herald added a project: clang-tools-extra.

Also fixes a benign use-after-free.


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D144641

Files:
  clang-tools-extra/clangd/Diagnostics.cpp
  clang-tools-extra/clangd/unittests/DiagnosticsTests.cpp


Index: clang-tools-extra/clangd/unittests/DiagnosticsTests.cpp
===
--- clang-tools-extra/clangd/unittests/DiagnosticsTests.cpp
+++ clang-tools-extra/clangd/unittests/DiagnosticsTests.cpp
@@ -23,6 +23,7 @@
 #include "clang/Basic/Diagnostic.h"
 #include "clang/Basic/DiagnosticSema.h"
 #include "llvm/ADT/StringRef.h"
+#include "llvm/Support/JSON.h"
 #include "llvm/Support/ScopedPrinter.h"
 #include "llvm/Support/TargetSelect.h"
 #include "gmock/gmock.h"
@@ -1012,6 +1013,7 @@
   D.Severity = DiagnosticsEngine::Error;
   D.File = "foo/bar/main.cpp";
   D.AbsFile = std::string(MainFile.file());
+  D.OpaqueData["test"] = "bar";
 
   clangd::Note NoteInMain;
   NoteInMain.Message = "declared somewhere in the main file";
@@ -1050,6 +1052,7 @@
 ../foo/baz/header.h:10:11:
 note: declared somewhere in the header file)";
   MainLSP.tags = {DiagnosticTag::Unnecessary};
+  MainLSP.data = D.OpaqueData;
 
   clangd::Diagnostic NoteInMainLSP;
   NoteInMainLSP.range = NoteInMain.Range;
Index: clang-tools-extra/clangd/Diagnostics.cpp
===
--- clang-tools-extra/clangd/Diagnostics.cpp
+++ clang-tools-extra/clangd/Diagnostics.cpp
@@ -526,6 +526,9 @@
 }
   }
   Main.tags = D.Tags;
+  // FIXME: Get rid of the copies here by taking in a mutable clangd::Diag.
+  for (auto  : D.OpaqueData)
+Main.data.insert({Entry.first, Entry.second});
   OutFn(std::move(Main), D.Fixes);
 
   // If we didn't emit the notes as relatedLocations, emit separate diagnostics
@@ -540,10 +543,6 @@
   Res.message = noteMessage(D, Note, Opts);
   OutFn(std::move(Res), llvm::ArrayRef());
 }
-
-  // FIXME: Get rid of the copies here by taking in a mutable clangd::Diag.
-  for (auto  : D.OpaqueData)
-Main.data.insert({Entry.first, Entry.second});
 }
 
 int getSeverity(DiagnosticsEngine::Level L) {


Index: clang-tools-extra/clangd/unittests/DiagnosticsTests.cpp
===
--- clang-tools-extra/clangd/unittests/DiagnosticsTests.cpp
+++ clang-tools-extra/clangd/unittests/DiagnosticsTests.cpp
@@ -23,6 +23,7 @@
 #include "clang/Basic/Diagnostic.h"
 #include "clang/Basic/DiagnosticSema.h"
 #include "llvm/ADT/StringRef.h"
+#include "llvm/Support/JSON.h"
 #include "llvm/Support/ScopedPrinter.h"
 #include "llvm/Support/TargetSelect.h"
 #include "gmock/gmock.h"
@@ -1012,6 +1013,7 @@
   D.Severity = DiagnosticsEngine::Error;
   D.File = "foo/bar/main.cpp";
   D.AbsFile = std::string(MainFile.file());
+  D.OpaqueData["test"] = "bar";
 
   clangd::Note NoteInMain;
   NoteInMain.Message = "declared somewhere in the main file";
@@ -1050,6 +1052,7 @@
 ../foo/baz/header.h:10:11:
 note: declared somewhere in the header file)";
   MainLSP.tags = {DiagnosticTag::Unnecessary};
+  MainLSP.data = D.OpaqueData;
 
   clangd::Diagnostic NoteInMainLSP;
   NoteInMainLSP.range = NoteInMain.Range;
Index: clang-tools-extra/clangd/Diagnostics.cpp
===
--- clang-tools-extra/clangd/Diagnostics.cpp
+++ clang-tools-extra/clangd/Diagnostics.cpp
@@ -526,6 +526,9 @@
 }
   }
   Main.tags = D.Tags;
+  // FIXME: Get rid of the copies here by taking in a mutable clangd::Diag.
+  for (auto  : D.OpaqueData)
+Main.data.insert({Entry.first, Entry.second});
   OutFn(std::move(Main), D.Fixes);
 
   // If we didn't emit the notes as relatedLocations, emit separate diagnostics
@@ -540,10 +543,6 @@
   Res.message = noteMessage(D, Note, Opts);
   OutFn(std::move(Res), llvm::ArrayRef());
 }
-
-  // FIXME: Get rid of the copies here by taking in a mutable clangd::Diag.
-  for (auto  : D.OpaqueData)
-Main.data.insert({Entry.first, Entry.second});
 }
 
 int getSeverity(DiagnosticsEngine::Level L) {
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D144599: [clangd/index/remote]NFC: Adapt code to newer grpc/protobuf versions

[Kadir Cetinkaya via Phabricator via cfe-commits]

kadircet accepted this revision.
kadircet added a comment.
This revision is now accepted and ready to land.

thanks, indeed the protobuf version used by grpc-1.36.3 has these available, 
https://github.com/protocolbuffers/protobuf/blob/19fb89416f3fdc2d6668f3738f444885575285bc/src/google/protobuf/stubs/status.h.

if you don't mind me asking, are you deliberately building remote-index 
components? i.e. do you have an internal remote-index server/deployment ? 
Asking as it'd be great to know that we've adoption here, outside of ourselves.

OTOH, if you're not actually using it, this shouldn't be built unless 
`-DCLANGD_ENABLE_REMOTE=1 ` is part of the build config, hence we might have a 
bug in the build configuration and I'd like to make sure it's fixed.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D144599/new/

https://reviews.llvm.org/D144599

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D144582: [include-cleaner] Always treat constructor calls as implicit

[Kadir Cetinkaya via Phabricator via cfe-commits]

This revision was automatically updated to reflect the committed changes.
Closed by commit rG279b4985ed4f: [include-cleaner] Always treat constructor 
calls as implicit (authored by kadircet).

Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D144582/new/

https://reviews.llvm.org/D144582

Files:
  clang-tools-extra/include-cleaner/lib/WalkAST.cpp
  clang-tools-extra/include-cleaner/unittests/WalkASTTest.cpp


Index: clang-tools-extra/include-cleaner/unittests/WalkASTTest.cpp
===
--- clang-tools-extra/include-cleaner/unittests/WalkASTTest.cpp
+++ clang-tools-extra/include-cleaner/unittests/WalkASTTest.cpp
@@ -111,6 +111,9 @@
   testWalk("struct $explicit^S {};", "^S *y;");
   testWalk("enum $explicit^E {};", "^E *y;");
   testWalk("struct $explicit^S { static int x; };", "int y = ^S::x;");
+  // One explicit call from the TypeLoc in constructor spelling, another
+  // implicit reference through the constructor call.
+  testWalk("struct $explicit^$implicit^S { static int x; };", "auto y = 
^S();");
 }
 
 TEST(WalkAST, Alias) {
@@ -241,7 +244,7 @@
 TEST(WalkAST, ConstructExprs) {
   testWalk("struct $implicit^S {};", "S ^t;");
   testWalk("struct $implicit^S { S(); };", "S ^t;");
-  testWalk("struct $explicit^S { S(int); };", "S ^t(42);");
+  testWalk("struct $implicit^S { S(int); };", "S ^t(42);");
   testWalk("struct $implicit^S { S(int); };", "S t = ^42;");
   testWalk("namespace ns { struct S{}; } using ns::$implicit^S;", "S ^t;");
 }
Index: clang-tools-extra/include-cleaner/lib/WalkAST.cpp
===
--- clang-tools-extra/include-cleaner/lib/WalkAST.cpp
+++ clang-tools-extra/include-cleaner/lib/WalkAST.cpp
@@ -102,9 +102,12 @@
   }
 
   bool VisitCXXConstructExpr(CXXConstructExpr *E) {
+// Always treat consturctor calls as implicit. We'll have an explicit
+// reference for the constructor calls that mention the type-name (through
+// TypeLocs). This reference only matters for cases where there's no
+// explicit syntax at all or there're only braces.
 report(E->getLocation(), getMemberProvider(E->getType()),
-   E->getParenOrBraceRange().isValid() ? RefType::Explicit
-   : RefType::Implicit);
+   RefType::Implicit);
 return true;
   }
 


Index: clang-tools-extra/include-cleaner/unittests/WalkASTTest.cpp
===
--- clang-tools-extra/include-cleaner/unittests/WalkASTTest.cpp
+++ clang-tools-extra/include-cleaner/unittests/WalkASTTest.cpp
@@ -111,6 +111,9 @@
   testWalk("struct $explicit^S {};", "^S *y;");
   testWalk("enum $explicit^E {};", "^E *y;");
   testWalk("struct $explicit^S { static int x; };", "int y = ^S::x;");
+  // One explicit call from the TypeLoc in constructor spelling, another
+  // implicit reference through the constructor call.
+  testWalk("struct $explicit^$implicit^S { static int x; };", "auto y = ^S();");
 }
 
 TEST(WalkAST, Alias) {
@@ -241,7 +244,7 @@
 TEST(WalkAST, ConstructExprs) {
   testWalk("struct $implicit^S {};", "S ^t;");
   testWalk("struct $implicit^S { S(); };", "S ^t;");
-  testWalk("struct $explicit^S { S(int); };", "S ^t(42);");
+  testWalk("struct $implicit^S { S(int); };", "S ^t(42);");
   testWalk("struct $implicit^S { S(int); };", "S t = ^42;");
   testWalk("namespace ns { struct S{}; } using ns::$implicit^S;", "S ^t;");
 }
Index: clang-tools-extra/include-cleaner/lib/WalkAST.cpp
===
--- clang-tools-extra/include-cleaner/lib/WalkAST.cpp
+++ clang-tools-extra/include-cleaner/lib/WalkAST.cpp
@@ -102,9 +102,12 @@
   }
 
   bool VisitCXXConstructExpr(CXXConstructExpr *E) {
+// Always treat consturctor calls as implicit. We'll have an explicit
+// reference for the constructor calls that mention the type-name (through
+// TypeLocs). This reference only matters for cases where there's no
+// explicit syntax at all or there're only braces.
 report(E->getLocation(), getMemberProvider(E->getType()),
-   E->getParenOrBraceRange().isValid() ? RefType::Explicit
-   : RefType::Implicit);
+   RefType::Implicit);
 return true;
   }
 
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D143496: [clangd] Add support for missing includes analysis.

[Kadir Cetinkaya via Phabricator via cfe-commits]

kadircet added inline comments.



Comment at: clang-tools-extra/clangd/IncludeCleaner.cpp:291
   }
   for (auto  : Cfg.Diagnostics.Includes.IgnoreHeader) {
 // Convert the path to Unix slashes and try to match against the filter.

can you also change the logic here to use `isFilteredByConfig` (we need the 
`native` call inside `isFilteredByConfig` as well to make sure it works on 
windows)



Comment at: clang-tools-extra/clangd/IncludeCleaner.cpp:373
+include_cleaner::Header Provider) {
+  std::string SpelledHeader;
+  if (Provider.kind() == include_cleaner::Header::Physical) {

nit: you can directly define `SpelledHeader` at line 377



Comment at: clang-tools-extra/clangd/IncludeCleaner.cpp:408
+
+bool isFilteredByConfig(const Config , llvm::StringRef HeaderSpelling) {
+  for (auto  : Cfg.Diagnostics.Includes.IgnoreHeader) {

this shouldn't be spelling, it should be the resolved path of the include.



Comment at: clang-tools-extra/clangd/IncludeCleaner.cpp:433
+if (isFilteredByConfig(Cfg, Spelling)) {
+  dlog("{0} header is filtered out by the configuration", Spelling);
+  continue;

can you prefix this with `IncludeCleaner: ` and rather say `not diagnosing 
missing include {0}, filtered by config` to add a bit more context about what 
specific interaction the log is coming from



Comment at: clang-tools-extra/clangd/IncludeCleaner.cpp:461
+ SymbolWithMissingInclude.SymRefRange.endOffset())};
+D.Fixes.emplace_back();
+D.Fixes.back().Message = "#include " + Spelling;

nit:
```
auto  = D.Fixes.emplace_back();
F.Message = ...;
F.Edits.push_back(replacementToEdit(...));
```



Comment at: clang-tools-extra/clangd/IncludeCleaner.cpp:465
+D.Fixes.back().Edits.emplace_back(std::move(Edit));
+D.InsideMainFile = true;
+  }

nit: it'd put this next to `D.File` above



Comment at: clang-tools-extra/clangd/IncludeCleaner.cpp:695
+auto Range = syntax::Token::range(
+AST.getSourceManager(), (*SpelledForExpanded)[0],
+(*SpelledForExpanded)[(*SpelledForExpanded).size() - 1]);

`auto Range = syntax::Token::range(SM, SpelledForExpanded->front(), 
SpelledForExpanded->back());`



Comment at: clang-tools-extra/clangd/IncludeCleaner.cpp:697
+(*SpelledForExpanded)[(*SpelledForExpanded).size() - 1]);
+std::string SymbolName;
+switch (Ref.Target.kind()) {

as mentioned elsewhere, i think we should delay this symbol name spelling to 
diagnostic generation. to make sure core analysis we perform don't do work that 
might not get re-used (e.g. if we're not going to diagnose missing includes, or 
in the future when we don't care about spelling of all the symbols)



Comment at: clang-tools-extra/clangd/IncludeCleaner.cpp:712
+  getUnused(AST, Used, /*ReferencedPublicHeaders*/ {});
+  return {std::move(UnusedIncludes), std::move(MissingIncludes)};
 }

nit: I think logically it makes more sense for us to return set of `Used` 
includes here, and let the interaction that issues unused include diagnostics 
to derive this information from the set of used includes, and change the the 
missingincludes to a `vector< tuple >` (not only the 
unsatisfied ones) would represent the analysis better and make it more usable 
in the future (i.e. when we want to augment Hover responses, we can't re-use 
all the logic in here, we really need to implement another call to `walkUsed` 
because the analysis we get out of this call won't contain information for 
`satisfied` symbols.

no need to do it now though, we can perform that kind of refactoring as we're 
adding the features too (or maybe it'll actually look neater to just have 
another call in those features rather than try and re-use the logic here)



Comment at: clang-tools-extra/clangd/IncludeCleaner.cpp:359
+TransformedInc.Angled = WrittenRef.starts_with("<");
+auto FE = SM.getFileManager().getFile(Inc.Resolved);
+if (!FE)

VitaNuo wrote:
> kadircet wrote:
> > unfortunately `getFile` returns an `llvm::Expected` which requires explicit 
> > error handling (or it'll trigger a crash). you can simply `elog` the issue:
> > ```
> > if (!FE) {
> >   elog("IncludeCleaner: Failed to get an entry for resolved path {0}: {1}", 
> > Inc.Resolved, FE.takeError());
> >   continue;
> > }
> > ```
> It returns `llvm::ErrorOr`, if I am not mistaken. There was explicit error 
> handling already (`if (!FE) continue` below), just without the `elog`. Are 
> you trying to say it will crash without the logging? Not sure that's feasible 
> :)
> Added the logging.
> It returns llvm::ErrorOr, if I am not mistaken. 

Ah you're right, I confused it with 

[PATCH] D144579: [include-cleaner] Check macros against stdlib database

[Kadir Cetinkaya via Phabricator via cfe-commits]

This revision was automatically updated to reflect the committed changes.
Closed by commit rG961e32c587cb: [include-cleaner] Check macros against stdlib 
database (authored by kadircet).

Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D144579/new/

https://reviews.llvm.org/D144579

Files:
  clang-tools-extra/include-cleaner/lib/LocateSymbol.cpp
  clang-tools-extra/include-cleaner/unittests/LocateSymbolTest.cpp


Index: clang-tools-extra/include-cleaner/unittests/LocateSymbolTest.cpp
===
--- clang-tools-extra/include-cleaner/unittests/LocateSymbolTest.cpp
+++ clang-tools-extra/include-cleaner/unittests/LocateSymbolTest.cpp
@@ -122,9 +122,17 @@
 }
 
 TEST(LocateSymbol, Stdlib) {
-  LocateExample Test("namespace std { struct vector; }");
-  EXPECT_THAT(locateSymbol(Test.findDecl("vector")),
-  ElementsAre(*tooling::stdlib::Symbol::named("std::", "vector")));
+  {
+LocateExample Test("namespace std { struct vector; }");
+EXPECT_THAT(
+locateSymbol(Test.findDecl("vector")),
+ElementsAre(*tooling::stdlib::Symbol::named("std::", "vector")));
+  }
+  {
+LocateExample Test("#define assert(x)\nvoid foo() { assert(true); }");
+EXPECT_THAT(locateSymbol(Test.findMacro("assert")),
+ElementsAre(*tooling::stdlib::Symbol::named("", "assert")));
+  }
 }
 
 TEST(LocateSymbol, Macros) {
Index: clang-tools-extra/include-cleaner/lib/LocateSymbol.cpp
===
--- clang-tools-extra/include-cleaner/lib/LocateSymbol.cpp
+++ clang-tools-extra/include-cleaner/lib/LocateSymbol.cpp
@@ -7,6 +7,7 @@
 
//===--===//
 
 #include "AnalysisInternal.h"
+#include "clang-include-cleaner/Types.h"
 #include "clang/AST/Decl.h"
 #include "clang/AST/DeclBase.h"
 #include "clang/AST/DeclCXX.h"
@@ -53,6 +54,12 @@
   return Result;
 }
 
+std::vector> locateMacro(const Macro ) {
+  // FIXME: Should we also provide physical locations?
+  if (auto SS = tooling::stdlib::Symbol::named("", M.Name->getName()))
+return {{*SS, Hints::CompleteSymbol}};
+  return {{M.Definition, Hints::CompleteSymbol}};
+}
 } // namespace
 
 std::vector> locateSymbol(const Symbol ) {
@@ -60,7 +67,7 @@
   case Symbol::Declaration:
 return locateDecl(S.declaration());
   case Symbol::Macro:
-return {{S.macro().Definition, Hints::CompleteSymbol}};
+return locateMacro(S.macro());
   }
   llvm_unreachable("Unknown Symbol::Kind enum");
 }


Index: clang-tools-extra/include-cleaner/unittests/LocateSymbolTest.cpp
===
--- clang-tools-extra/include-cleaner/unittests/LocateSymbolTest.cpp
+++ clang-tools-extra/include-cleaner/unittests/LocateSymbolTest.cpp
@@ -122,9 +122,17 @@
 }
 
 TEST(LocateSymbol, Stdlib) {
-  LocateExample Test("namespace std { struct vector; }");
-  EXPECT_THAT(locateSymbol(Test.findDecl("vector")),
-  ElementsAre(*tooling::stdlib::Symbol::named("std::", "vector")));
+  {
+LocateExample Test("namespace std { struct vector; }");
+EXPECT_THAT(
+locateSymbol(Test.findDecl("vector")),
+ElementsAre(*tooling::stdlib::Symbol::named("std::", "vector")));
+  }
+  {
+LocateExample Test("#define assert(x)\nvoid foo() { assert(true); }");
+EXPECT_THAT(locateSymbol(Test.findMacro("assert")),
+ElementsAre(*tooling::stdlib::Symbol::named("", "assert")));
+  }
 }
 
 TEST(LocateSymbol, Macros) {
Index: clang-tools-extra/include-cleaner/lib/LocateSymbol.cpp
===
--- clang-tools-extra/include-cleaner/lib/LocateSymbol.cpp
+++ clang-tools-extra/include-cleaner/lib/LocateSymbol.cpp
@@ -7,6 +7,7 @@
 //===--===//
 
 #include "AnalysisInternal.h"
+#include "clang-include-cleaner/Types.h"
 #include "clang/AST/Decl.h"
 #include "clang/AST/DeclBase.h"
 #include "clang/AST/DeclCXX.h"
@@ -53,6 +54,12 @@
   return Result;
 }
 
+std::vector> locateMacro(const Macro ) {
+  // FIXME: Should we also provide physical locations?
+  if (auto SS = tooling::stdlib::Symbol::named("", M.Name->getName()))
+return {{*SS, Hints::CompleteSymbol}};
+  return {{M.Definition, Hints::CompleteSymbol}};
+}
 } // namespace
 
 std::vector> locateSymbol(const Symbol ) {
@@ -60,7 +67,7 @@
   case Symbol::Declaration:
 return locateDecl(S.declaration());
   case Symbol::Macro:
-return {{S.macro().Definition, Hints::CompleteSymbol}};
+return locateMacro(S.macro());
   }
   llvm_unreachable("Unknown Symbol::Kind enum");
 }
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D144582: [include-cleaner] Always treat constructor calls as implicit

[Kadir Cetinkaya via Phabricator via cfe-commits]

kadircet created this revision.
kadircet added a reviewer: hokein.
Herald added a project: All.
kadircet requested review of this revision.
Herald added a project: clang-tools-extra.
Herald added a subscriber: cfe-commits.

Treating constructor calls when the type name isn't explicitly spelled
can cause spurious results, so turn them into implicit references.
This doesn't change the behaviour for constructor calls that explicitly spell
the type name, as we should see a reference through the typeloc.

Fixes https://github.com/llvm/llvm-project/issues/60812


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D144582

Files:
  clang-tools-extra/include-cleaner/lib/WalkAST.cpp
  clang-tools-extra/include-cleaner/unittests/WalkASTTest.cpp


Index: clang-tools-extra/include-cleaner/unittests/WalkASTTest.cpp
===
--- clang-tools-extra/include-cleaner/unittests/WalkASTTest.cpp
+++ clang-tools-extra/include-cleaner/unittests/WalkASTTest.cpp
@@ -111,6 +111,9 @@
   testWalk("struct $explicit^S {};", "^S *y;");
   testWalk("enum $explicit^E {};", "^E *y;");
   testWalk("struct $explicit^S { static int x; };", "int y = ^S::x;");
+  // One explicit call from the TypeLoc in constructor spelling, another
+  // implicit reference through the constructor call.
+  testWalk("struct $explicit^$implicit^S { static int x; };", "auto y = 
^S();");
 }
 
 TEST(WalkAST, Alias) {
@@ -241,7 +244,7 @@
 TEST(WalkAST, ConstructExprs) {
   testWalk("struct $implicit^S {};", "S ^t;");
   testWalk("struct $implicit^S { S(); };", "S ^t;");
-  testWalk("struct $explicit^S { S(int); };", "S ^t(42);");
+  testWalk("struct $implicit^S { S(int); };", "S ^t(42);");
   testWalk("struct $implicit^S { S(int); };", "S t = ^42;");
   testWalk("namespace ns { struct S{}; } using ns::$implicit^S;", "S ^t;");
 }
Index: clang-tools-extra/include-cleaner/lib/WalkAST.cpp
===
--- clang-tools-extra/include-cleaner/lib/WalkAST.cpp
+++ clang-tools-extra/include-cleaner/lib/WalkAST.cpp
@@ -102,9 +102,12 @@
   }
 
   bool VisitCXXConstructExpr(CXXConstructExpr *E) {
+// Always treat consturctor calls as implicit. We'll have an explicit
+// reference for the constructor calls that mention the type-name (through
+// TypeLocs). This reference only matters for cases where there's no
+// explicit syntax at all or there're only braces.
 report(E->getLocation(), getMemberProvider(E->getType()),
-   E->getParenOrBraceRange().isValid() ? RefType::Explicit
-   : RefType::Implicit);
+   RefType::Implicit);
 return true;
   }
 


Index: clang-tools-extra/include-cleaner/unittests/WalkASTTest.cpp
===
--- clang-tools-extra/include-cleaner/unittests/WalkASTTest.cpp
+++ clang-tools-extra/include-cleaner/unittests/WalkASTTest.cpp
@@ -111,6 +111,9 @@
   testWalk("struct $explicit^S {};", "^S *y;");
   testWalk("enum $explicit^E {};", "^E *y;");
   testWalk("struct $explicit^S { static int x; };", "int y = ^S::x;");
+  // One explicit call from the TypeLoc in constructor spelling, another
+  // implicit reference through the constructor call.
+  testWalk("struct $explicit^$implicit^S { static int x; };", "auto y = ^S();");
 }
 
 TEST(WalkAST, Alias) {
@@ -241,7 +244,7 @@
 TEST(WalkAST, ConstructExprs) {
   testWalk("struct $implicit^S {};", "S ^t;");
   testWalk("struct $implicit^S { S(); };", "S ^t;");
-  testWalk("struct $explicit^S { S(int); };", "S ^t(42);");
+  testWalk("struct $implicit^S { S(int); };", "S ^t(42);");
   testWalk("struct $implicit^S { S(int); };", "S t = ^42;");
   testWalk("namespace ns { struct S{}; } using ns::$implicit^S;", "S ^t;");
 }
Index: clang-tools-extra/include-cleaner/lib/WalkAST.cpp
===
--- clang-tools-extra/include-cleaner/lib/WalkAST.cpp
+++ clang-tools-extra/include-cleaner/lib/WalkAST.cpp
@@ -102,9 +102,12 @@
   }
 
   bool VisitCXXConstructExpr(CXXConstructExpr *E) {
+// Always treat consturctor calls as implicit. We'll have an explicit
+// reference for the constructor calls that mention the type-name (through
+// TypeLocs). This reference only matters for cases where there's no
+// explicit syntax at all or there're only braces.
 report(E->getLocation(), getMemberProvider(E->getType()),
-   E->getParenOrBraceRange().isValid() ? RefType::Explicit
-   : RefType::Implicit);
+   RefType::Implicit);
 return true;
   }
 
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D144579: [include-cleaner] Check macros against stdlib database

[Kadir Cetinkaya via Phabricator via cfe-commits]

kadircet created this revision.
kadircet added a reviewer: hokein.
Herald added a project: All.
kadircet requested review of this revision.
Herald added a project: clang-tools-extra.
Herald added a subscriber: cfe-commits.

Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D144579

Files:
  clang-tools-extra/include-cleaner/lib/LocateSymbol.cpp
  clang-tools-extra/include-cleaner/unittests/LocateSymbolTest.cpp


Index: clang-tools-extra/include-cleaner/unittests/LocateSymbolTest.cpp
===
--- clang-tools-extra/include-cleaner/unittests/LocateSymbolTest.cpp
+++ clang-tools-extra/include-cleaner/unittests/LocateSymbolTest.cpp
@@ -122,9 +122,17 @@
 }
 
 TEST(LocateSymbol, Stdlib) {
-  LocateExample Test("namespace std { struct vector; }");
-  EXPECT_THAT(locateSymbol(Test.findDecl("vector")),
-  ElementsAre(*tooling::stdlib::Symbol::named("std::", "vector")));
+  {
+LocateExample Test("namespace std { struct vector; }");
+EXPECT_THAT(
+locateSymbol(Test.findDecl("vector")),
+ElementsAre(*tooling::stdlib::Symbol::named("std::", "vector")));
+  }
+  {
+LocateExample Test("#define assert(x)\nvoid foo() { assert(true); }");
+EXPECT_THAT(locateSymbol(Test.findMacro("assert")),
+ElementsAre(*tooling::stdlib::Symbol::named("", "assert")));
+  }
 }
 
 TEST(LocateSymbol, Macros) {
Index: clang-tools-extra/include-cleaner/lib/LocateSymbol.cpp
===
--- clang-tools-extra/include-cleaner/lib/LocateSymbol.cpp
+++ clang-tools-extra/include-cleaner/lib/LocateSymbol.cpp
@@ -7,6 +7,7 @@
 
//===--===//
 
 #include "AnalysisInternal.h"
+#include "clang-include-cleaner/Types.h"
 #include "clang/AST/Decl.h"
 #include "clang/AST/DeclBase.h"
 #include "clang/AST/DeclCXX.h"
@@ -53,6 +54,12 @@
   return Result;
 }
 
+std::vector> locateMacro(const Macro ) {
+  // FIXME: Should we also provide physical locations?
+  if (auto SS = tooling::stdlib::Symbol::named("", M.Name->getName()))
+return {{*SS, Hints::CompleteSymbol}};
+  return {{M.Definition, Hints::CompleteSymbol}};
+}
 } // namespace
 
 std::vector> locateSymbol(const Symbol ) {
@@ -60,7 +67,7 @@
   case Symbol::Declaration:
 return locateDecl(S.declaration());
   case Symbol::Macro:
-return {{S.macro().Definition, Hints::CompleteSymbol}};
+return locateMacro(S.macro());
   }
   llvm_unreachable("Unknown Symbol::Kind enum");
 }


Index: clang-tools-extra/include-cleaner/unittests/LocateSymbolTest.cpp
===
--- clang-tools-extra/include-cleaner/unittests/LocateSymbolTest.cpp
+++ clang-tools-extra/include-cleaner/unittests/LocateSymbolTest.cpp
@@ -122,9 +122,17 @@
 }
 
 TEST(LocateSymbol, Stdlib) {
-  LocateExample Test("namespace std { struct vector; }");
-  EXPECT_THAT(locateSymbol(Test.findDecl("vector")),
-  ElementsAre(*tooling::stdlib::Symbol::named("std::", "vector")));
+  {
+LocateExample Test("namespace std { struct vector; }");
+EXPECT_THAT(
+locateSymbol(Test.findDecl("vector")),
+ElementsAre(*tooling::stdlib::Symbol::named("std::", "vector")));
+  }
+  {
+LocateExample Test("#define assert(x)\nvoid foo() { assert(true); }");
+EXPECT_THAT(locateSymbol(Test.findMacro("assert")),
+ElementsAre(*tooling::stdlib::Symbol::named("", "assert")));
+  }
 }
 
 TEST(LocateSymbol, Macros) {
Index: clang-tools-extra/include-cleaner/lib/LocateSymbol.cpp
===
--- clang-tools-extra/include-cleaner/lib/LocateSymbol.cpp
+++ clang-tools-extra/include-cleaner/lib/LocateSymbol.cpp
@@ -7,6 +7,7 @@
 //===--===//
 
 #include "AnalysisInternal.h"
+#include "clang-include-cleaner/Types.h"
 #include "clang/AST/Decl.h"
 #include "clang/AST/DeclBase.h"
 #include "clang/AST/DeclCXX.h"
@@ -53,6 +54,12 @@
   return Result;
 }
 
+std::vector> locateMacro(const Macro ) {
+  // FIXME: Should we also provide physical locations?
+  if (auto SS = tooling::stdlib::Symbol::named("", M.Name->getName()))
+return {{*SS, Hints::CompleteSymbol}};
+  return {{M.Definition, Hints::CompleteSymbol}};
+}
 } // namespace
 
 std::vector> locateSymbol(const Symbol ) {
@@ -60,7 +67,7 @@
   case Symbol::Declaration:
 return locateDecl(S.declaration());
   case Symbol::Macro:
-return {{S.macro().Definition, Hints::CompleteSymbol}};
+return locateMacro(S.macro());
   }
   llvm_unreachable("Unknown Symbol::Kind enum");
 }
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D144456: [clangd] Publish diagnostics with stale preambles

[Kadir Cetinkaya via Phabricator via cfe-commits]

This revision was landed with ongoing or failed builds.
This revision was automatically updated to reflect the committed changes.
Closed by commit rG465ee9bfb26d: [clangd] Publish diagnostics with stale 
preambles (authored by kadircet).

Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D144456/new/

https://reviews.llvm.org/D144456

Files:
  clang-tools-extra/clangd/TUScheduler.cpp
  clang-tools-extra/clangd/unittests/TUSchedulerTests.cpp

Index: clang-tools-extra/clangd/unittests/TUSchedulerTests.cpp
===
--- clang-tools-extra/clangd/unittests/TUSchedulerTests.cpp
+++ clang-tools-extra/clangd/unittests/TUSchedulerTests.cpp
@@ -8,6 +8,8 @@
 
 #include "Annotations.h"
 #include "ClangdServer.h"
+#include "Compiler.h"
+#include "Config.h"
 #include "Diagnostics.h"
 #include "GlobalCompilationDatabase.h"
 #include "Matchers.h"
@@ -21,7 +23,6 @@
 #include "support/Path.h"
 #include "support/TestTracer.h"
 #include "support/Threading.h"
-#include "support/ThreadsafeFS.h"
 #include "clang/Basic/DiagnosticDriver.h"
 #include "llvm/ADT/ArrayRef.h"
 #include "llvm/ADT/FunctionExtras.h"
@@ -31,19 +32,23 @@
 #include "llvm/ADT/StringRef.h"
 #include "gmock/gmock.h"
 #include "gtest/gtest.h"
-#include 
 #include 
 #include 
+#include 
 #include 
+#include 
 #include 
+#include 
 #include 
 #include 
 #include 
+#include 
 
 namespace clang {
 namespace clangd {
 namespace {
 
+using ::testing::_;
 using ::testing::AllOf;
 using ::testing::AnyOf;
 using ::testing::Contains;
@@ -1194,6 +1199,118 @@
   EXPECT_EQ(PreamblePublishCount, 2);
 }
 
+TEST_F(TUSchedulerTests, PublishWithStalePreamble) {
+  // Callbacks that blocks the preamble thread after the first preamble is
+  // built and stores preamble/main-file versions for diagnostics released.
+  class BlockPreambleThread : public ParsingCallbacks {
+  public:
+using DiagsCB = std::function;
+BlockPreambleThread(Notification , DiagsCB CB)
+: UnblockPreamble(UnblockPreamble), CB(std::move(CB)) {}
+
+void onPreambleAST(PathRef Path, llvm::StringRef Version,
+   const CompilerInvocation &, ASTContext ,
+   Preprocessor &, const CanonicalIncludes &) override {
+  if (BuildBefore)
+ASSERT_TRUE(UnblockPreamble.wait(timeoutSeconds(5)))
+<< "Expected notification";
+  BuildBefore = true;
+}
+
+void onMainAST(PathRef File, ParsedAST , PublishFn Publish) override {
+  CB(AST);
+}
+
+void onFailedAST(PathRef File, llvm::StringRef Version,
+ std::vector Diags, PublishFn Publish) override {
+  ADD_FAILURE() << "Received failed ast for: " << File << " with version "
+<< Version << '\n';
+}
+
+  private:
+bool BuildBefore = false;
+Notification 
+std::function CB;
+  };
+
+  // Helpers for issuing blocking update requests on a TUScheduler, whose
+  // onMainAST callback would call onDiagnostics.
+  class DiagCollector {
+  public:
+void onDiagnostics(ParsedAST ) {
+  std::scoped_lock Lock(DiagMu);
+  DiagVersions.emplace_back(
+  std::make_pair(AST.preambleVersion()->str(), AST.version().str()));
+  DiagsReceived.notify_all();
+}
+
+std::pair
+waitForNewDiags(TUScheduler , PathRef File, ParseInputs PI) {
+  std::unique_lock Lock(DiagMu);
+  // Perform the update under the lock to make sure it isn't handled until
+  // we're waiting for it.
+  S.update(File, std::move(PI), WantDiagnostics::Auto);
+  size_t OldSize = DiagVersions.size();
+  bool ReceivedDiags = DiagsReceived.wait_for(
+  Lock, std::chrono::seconds(5),
+  [this, OldSize] { return OldSize + 1 == DiagVersions.size(); });
+  if (!ReceivedDiags) {
+ADD_FAILURE() << "Timed out waiting for diags";
+return {"invalid", "version"};
+  }
+  return DiagVersions.back();
+}
+
+std::vector> diagVersions() {
+  std::scoped_lock Lock(DiagMu);
+  return DiagVersions;
+}
+
+  private:
+std::condition_variable DiagsReceived;
+std::mutex DiagMu;
+std::vector>
+DiagVersions;
+  };
+
+  Config Cfg;
+  Cfg.Diagnostics.AllowStalePreamble = true;
+  WithContextValue WithCfg(Config::Key, std::move(Cfg));
+
+  DiagCollector Collector;
+  Notification UnblockPreamble;
+  auto DiagCallbacks = std::make_unique(
+  UnblockPreamble,
+  [](ParsedAST ) { Collector.onDiagnostics(AST); });
+  TUScheduler S(CDB, optsForTest(), std::move(DiagCallbacks));
+  Path File = testPath("foo.cpp");
+  auto BlockForDiags = [&](ParseInputs PI) {
+return Collector.waitForNewDiags(S, File, std::move(PI));
+  };
+
+  // Build first preamble.
+  auto PI = getInputs(File, "");
+  PI.Version = PI.Contents = "1";
+  ASSERT_THAT(BlockForDiags(PI), testing::Pair("1", "1"));
+
+  // Now preamble thread is blocked, so rest of the requests sees only the
+  

[PATCH] D143096: [clangd] Provide patched diagnostics with preamble patch

[Kadir Cetinkaya via Phabricator via cfe-commits]

This revision was automatically updated to reflect the committed changes.
Closed by commit rG8c2a12f7f9b6: [clangd] Provide patched diagnostics with 
preamble patch (authored by kadircet).

Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D143096/new/

https://reviews.llvm.org/D143096

Files:
  clang-tools-extra/clangd/ParsedAST.cpp
  clang-tools-extra/clangd/Preamble.cpp
  clang-tools-extra/clangd/Preamble.h
  clang-tools-extra/clangd/unittests/PreambleTests.cpp

Index: clang-tools-extra/clangd/unittests/PreambleTests.cpp
===
--- clang-tools-extra/clangd/unittests/PreambleTests.cpp
+++ clang-tools-extra/clangd/unittests/PreambleTests.cpp
@@ -723,9 +723,8 @@
 #define BAR
 #include [[]])");
 auto AST = createPatchedAST(Code.code(), NewCode.code());
-// FIXME: We should point at the correct coordinates in NewCode.
 EXPECT_THAT(*AST->getDiagnostics(),
-ElementsAre(Diag(Code.range(), "pp_file_not_found")));
+ElementsAre(Diag(NewCode.range(), "pp_file_not_found")));
   }
   {
 // Check with removals from preamble.
@@ -734,18 +733,15 @@
 #include [[]])");
 Annotations NewCode("#include [[]]");
 auto AST = createPatchedAST(Code.code(), NewCode.code());
-// FIXME: We should point at the correct coordinates in NewCode.
 EXPECT_THAT(*AST->getDiagnostics(),
-ElementsAre(Diag(Code.range(), "pp_file_not_found")));
+ElementsAre(Diag(NewCode.range(), "pp_file_not_found")));
   }
   {
 // Drop line with diags.
 Annotations Code("#include [[]]");
 Annotations NewCode("#define BAR\n#define BAZ\n");
 auto AST = createPatchedAST(Code.code(), NewCode.code());
-// FIXME: No diagnostics.
-EXPECT_THAT(*AST->getDiagnostics(),
-ElementsAre(Diag(Code.range(), "pp_file_not_found")));
+EXPECT_THAT(*AST->getDiagnostics(), IsEmpty());
   }
   {
 // Picks closest line in case of multiple alternatives.
@@ -756,18 +752,79 @@
 #define BAR
 #include )");
 auto AST = createPatchedAST(Code.code(), NewCode.code());
-// FIXME: We should point at the correct coordinates in NewCode.
 EXPECT_THAT(*AST->getDiagnostics(),
-ElementsAre(Diag(Code.range(), "pp_file_not_found")));
+ElementsAre(Diag(NewCode.range(), "pp_file_not_found")));
   }
   {
 // Drop diag if line spelling has changed.
 Annotations Code("#include [[]]");
 Annotations NewCode(" # include ");
 auto AST = createPatchedAST(Code.code(), NewCode.code());
-// FIXME: No diags.
+EXPECT_THAT(*AST->getDiagnostics(), IsEmpty());
+  }
+  {
+// Multiple lines.
+Annotations Code(R"(
+#define BAR
+#include [[]])");
+Annotations NewCode(R"(#include [[]])");
+auto AST = createPatchedAST(Code.code(), NewCode.code());
 EXPECT_THAT(*AST->getDiagnostics(),
-ElementsAre(Diag(Code.range(), "pp_file_not_found")));
+ElementsAre(Diag(NewCode.range(), "pp_file_not_found")));
+  }
+  {
+// Multiple lines with change.
+Annotations Code(R"(
+#define BAR
+#include 
+#include [[]])");
+Annotations NewCode(R"(#include )");
+auto AST = createPatchedAST(Code.code(), NewCode.code());
+EXPECT_THAT(*AST->getDiagnostics(), IsEmpty());
+  }
+  {
+// Preserves notes.
+Annotations Code(R"(
+#define $note[[BAR]] 1
+#define $main[[BAR]] 2)");
+Annotations NewCode(R"(
+#define BAZ 0
+#define $note[[BAR]] 1
+#define BAZ 0
+#define $main[[BAR]] 2)");
+auto AST = createPatchedAST(Code.code(), NewCode.code());
+EXPECT_THAT(
+*AST->getDiagnostics(),
+ElementsAre(AllOf(Diag(NewCode.range("main"), "-Wmacro-redefined"),
+  withNote(Diag(NewCode.range("note"));
+  }
+  {
+// Preserves diag without note.
+Annotations Code(R"(
+#define $note[[BAR]] 1
+#define $main[[BAR]] 2)");
+Annotations NewCode(R"(
+#define $main[[BAR]] 2)");
+auto AST = createPatchedAST(Code.code(), NewCode.code());
+EXPECT_THAT(
+*AST->getDiagnostics(),
+ElementsAre(AllOf(Diag(NewCode.range("main"), "-Wmacro-redefined"),
+  Field(::Notes, IsEmpty();
+  }
+  {
+// Make sure orphaned notes are not promoted to diags.
+Annotations Code(R"(
+#define $note[[BAR]] 1
+#define $main[[BAR]] 2)");
+Annotations NewCode(R"(
+#define BAZ 0
+#define BAR 1)");
+auto AST = createPatchedAST(Code.code(), NewCode.code());
+EXPECT_THAT(*AST->getDiagnostics(), IsEmpty());
   }
 }
 } // namespace
Index: clang-tools-extra/clangd/Preamble.h
===
--- clang-tools-extra/clangd/Preamble.h
+++ clang-tools-extra/clangd/Preamble.h
@@ -34,6 +34,7 @@
 #include "clang/Frontend/PrecompiledPreamble.h"
 #include "clang/Lex/Lexer.h"
 #include "clang/Tooling/CompilationDatabase.h"
+#include "llvm/ADT/ArrayRef.h"
 

[PATCH] D143095: [clangd] Respect preamble-patch when handling diags

[Kadir Cetinkaya via Phabricator via cfe-commits]

This revision was automatically updated to reflect the committed changes.
Closed by commit rG909cd1f9a893: [clangd] Respect preamble-patch when handling 
diags (authored by kadircet).

Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D143095/new/

https://reviews.llvm.org/D143095

Files:
  clang-tools-extra/clangd/Diagnostics.cpp
  clang-tools-extra/clangd/Preamble.cpp
  clang-tools-extra/clangd/Preamble.h
  clang-tools-extra/clangd/SourceCode.cpp
  clang-tools-extra/clangd/SourceCode.h
  clang-tools-extra/clangd/unittests/PreambleTests.cpp

Index: clang-tools-extra/clangd/unittests/PreambleTests.cpp
===
--- clang-tools-extra/clangd/unittests/PreambleTests.cpp
+++ clang-tools-extra/clangd/unittests/PreambleTests.cpp
@@ -41,6 +41,7 @@
 using testing::AllOf;
 using testing::Contains;
 using testing::ElementsAre;
+using testing::ElementsAreArray;
 using testing::Eq;
 using testing::Field;
 using testing::HasSubstr;
@@ -705,10 +706,8 @@
 auto AST = createPatchedAST(Code.code(), NewCode.code(), AdditionalFiles);
 EXPECT_THAT(
 *AST->getDiagnostics(),
-ElementsAre(AllOf(
-Diag(NewCode.range("foo2"), "-Wmacro-redefined"),
-// FIXME: This should be translated into main file.
-withNote(Field(::File, HasSubstr("_preamble_patch_"));
+ElementsAre(AllOf(Diag(NewCode.range("foo2"), "-Wmacro-redefined"),
+  withNote(Diag(NewCode.range("foo1"));
   }
 }
 
Index: clang-tools-extra/clangd/SourceCode.h
===
--- clang-tools-extra/clangd/SourceCode.h
+++ clang-tools-extra/clangd/SourceCode.h
@@ -333,6 +333,10 @@
  (isUppercase(Name[1]) || Name[1] == '_');
 }
 
+/// Translates locations inside preamble patch to their main-file equivalent
+/// using presumed locations. Returns \p Loc if it isn't inside preamble patch.
+SourceLocation translatePreamblePatchLocation(SourceLocation Loc,
+  const SourceManager );
 } // namespace clangd
 } // namespace clang
 #endif
Index: clang-tools-extra/clangd/SourceCode.cpp
===
--- clang-tools-extra/clangd/SourceCode.cpp
+++ clang-tools-extra/clangd/SourceCode.cpp
@@ -796,6 +796,12 @@
   return Results;
 }
 
+// Checks whether \p FileName is a valid spelling of main file.
+bool isMainFile(llvm::StringRef FileName, const SourceManager ) {
+  auto FE = SM.getFileManager().getFile(FileName);
+  return FE && *FE == SM.getFileEntryForID(SM.getMainFileID());
+}
+
 } // namespace
 
 std::vector visibleNamespaces(llvm::StringRef Code,
@@ -1219,5 +1225,24 @@
   return SM.getBufferData(FID).startswith(ProtoHeaderComment);
 }
 
+SourceLocation translatePreamblePatchLocation(SourceLocation Loc,
+  const SourceManager ) {
+  auto DefFile = SM.getFileID(Loc);
+  if (auto FE = SM.getFileEntryRefForID(DefFile)) {
+auto IncludeLoc = SM.getIncludeLoc(DefFile);
+// Preamble patch is included inside the builtin file.
+if (IncludeLoc.isValid() && SM.isWrittenInBuiltinFile(IncludeLoc) &&
+FE->getName().endswith(PreamblePatch::HeaderName)) {
+  auto Presumed = SM.getPresumedLoc(Loc);
+  // Check that line directive is pointing at main file.
+  if (Presumed.isValid() && Presumed.getFileID().isInvalid() &&
+  isMainFile(Presumed.getFilename(), SM)) {
+Loc = SM.translateLineCol(SM.getMainFileID(), Presumed.getLine(),
+  Presumed.getColumn());
+  }
+}
+  }
+  return Loc;
+}
 } // namespace clangd
 } // namespace clang
Index: clang-tools-extra/clangd/Preamble.h
===
--- clang-tools-extra/clangd/Preamble.h
+++ clang-tools-extra/clangd/Preamble.h
@@ -157,6 +157,8 @@
   /// Whether diagnostics generated using this patch are trustable.
   bool preserveDiagnostics() const;
 
+  static constexpr llvm::StringLiteral HeaderName = "__preamble_patch__.h";
+
 private:
   static PreamblePatch create(llvm::StringRef FileName,
   const ParseInputs ,
@@ -172,11 +174,6 @@
   PreambleBounds ModifiedBounds = {0, false};
 };
 
-/// Translates locations inside preamble patch to their main-file equivalent
-/// using presumed locations. Returns \p Loc if it isn't inside preamble patch.
-SourceLocation translatePreamblePatchLocation(SourceLocation Loc,
-  const SourceManager );
-
 } // namespace clangd
 } // namespace clang
 
Index: clang-tools-extra/clangd/Preamble.cpp
===
--- clang-tools-extra/clangd/Preamble.cpp
+++ clang-tools-extra/clangd/Preamble.cpp
@@ -53,7 +53,6 @@
 namespace clang {
 namespace clangd {
 namespace {
-constexpr 

[PATCH] D142890: [clangd] Add config option for fast diagnostics mode

[Kadir Cetinkaya via Phabricator via cfe-commits]

This revision was automatically updated to reflect the committed changes.
kadircet marked an inline comment as done.
Closed by commit rG7177a237b68f: [clangd] Add config option for fast 
diagnostics mode (authored by kadircet).

Changed prior to commit:
  https://reviews.llvm.org/D142890?vs=495889=499485#toc

Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D142890/new/

https://reviews.llvm.org/D142890

Files:
  clang-tools-extra/clangd/Config.h
  clang-tools-extra/clangd/ConfigCompile.cpp
  clang-tools-extra/clangd/ConfigFragment.h
  clang-tools-extra/clangd/ConfigYAML.cpp
  clang-tools-extra/clangd/Preamble.cpp
  clang-tools-extra/clangd/Preamble.h
  clang-tools-extra/clangd/unittests/ConfigCompileTests.cpp
  clang-tools-extra/clangd/unittests/ConfigYAMLTests.cpp
  clang-tools-extra/clangd/unittests/IncludeCleanerTests.cpp
  clang-tools-extra/clangd/unittests/PreambleTests.cpp

Index: clang-tools-extra/clangd/unittests/PreambleTests.cpp
===
--- clang-tools-extra/clangd/unittests/PreambleTests.cpp
+++ clang-tools-extra/clangd/unittests/PreambleTests.cpp
@@ -8,9 +8,13 @@
 
 #include "Annotations.h"
 #include "Compiler.h"
+#include "Config.h"
+#include "Diagnostics.h"
 #include "Headers.h"
 #include "Hover.h"
+#include "ParsedAST.h"
 #include "Preamble.h"
+#include "Protocol.h"
 #include "SourceCode.h"
 #include "TestFS.h"
 #include "TestTU.h"
@@ -19,10 +23,13 @@
 #include "clang/Format/Format.h"
 #include "clang/Frontend/FrontendActions.h"
 #include "clang/Frontend/PrecompiledPreamble.h"
+#include "llvm/ADT/StringMap.h"
 #include "llvm/ADT/StringRef.h"
 #include "llvm/Support/Error.h"
 #include "llvm/Support/MemoryBuffer.h"
+#include "llvm/Support/ScopedPrinter.h"
 #include "llvm/Support/VirtualFileSystem.h"
+#include "llvm/Testing/Support/SupportHelpers.h"
 #include "gmock/gmock.h"
 #include "gtest/gtest-matchers.h"
 #include "gtest/gtest.h"
@@ -31,10 +38,18 @@
 #include 
 #include 
 
+using testing::AllOf;
 using testing::Contains;
+using testing::ElementsAre;
+using testing::Eq;
 using testing::Field;
+using testing::HasSubstr;
+using testing::IsEmpty;
 using testing::Matcher;
 using testing::MatchesRegex;
+using testing::Not;
+using testing::UnorderedElementsAre;
+using testing::UnorderedElementsAreArray;
 
 namespace clang {
 namespace clangd {
@@ -204,9 +219,12 @@
   Field(::FileKind, SrcMgr::CharacteristicKind::C_User;
 }
 
-std::optional createPatchedAST(llvm::StringRef Baseline,
-  llvm::StringRef Modified) {
-  auto BaselinePreamble = TestTU::withCode(Baseline).preamble();
+std::optional
+createPatchedAST(llvm::StringRef Baseline, llvm::StringRef Modified,
+ llvm::StringMap AdditionalFiles = {}) {
+  auto TU = TestTU::withCode(Baseline);
+  TU.AdditionalFiles = std::move(AdditionalFiles);
+  auto BaselinePreamble = TU.preamble();
   if (!BaselinePreamble) {
 ADD_FAILURE() << "Failed to build baseline preamble";
 return std::nullopt;
@@ -214,7 +232,7 @@
 
   IgnoreDiagnostics Diags;
   MockFS FS;
-  auto TU = TestTU::withCode(Modified);
+  TU.Code = Modified.str();
   auto CI = buildCompilerInvocation(TU.inputs(FS), Diags);
   if (!CI) {
 ADD_FAILURE() << "Failed to build compiler invocation";
@@ -599,6 +617,161 @@
 TU.inputs(FS), *BaselinePreamble);
   EXPECT_TRUE(PP.text().empty());
 }
+
+::testing::Matcher
+withNote(::testing::Matcher NoteMatcher) {
+  return Field(::Notes, ElementsAre(NoteMatcher));
+}
+MATCHER_P(Diag, Range, "Diag at " + llvm::to_string(Range)) {
+  return arg.Range == Range;
+}
+MATCHER_P2(Diag, Range, Name,
+   "Diag at " + llvm::to_string(Range) + " = [" + Name + "]") {
+  return arg.Range == Range && arg.Name == Name;
+}
+
+TEST(PreamblePatch, DiagnosticsFromMainASTAreInRightPlace) {
+  Config Cfg;
+  Cfg.Diagnostics.AllowStalePreamble = true;
+  WithContextValue WithCfg(Config::Key, std::move(Cfg));
+
+  {
+Annotations Code("#define FOO");
+// Check with removals from preamble.
+Annotations NewCode("[[x]];/* error-ok */");
+auto AST = createPatchedAST(Code.code(), NewCode.code());
+EXPECT_THAT(*AST->getDiagnostics(),
+ElementsAre(Diag(NewCode.range(), "missing_type_specifier")));
+  }
+  {
+// Check with additions to preamble.
+Annotations Code("#define FOO");
+Annotations NewCode(R"(
+#define FOO
+#define BAR
+[[x]];/* error-ok */)");
+auto AST = createPatchedAST(Code.code(), NewCode.code());
+EXPECT_THAT(*AST->getDiagnostics(),
+ElementsAre(Diag(NewCode.range(), "missing_type_specifier")));
+  }
+}
+
+TEST(PreamblePatch, DiagnosticsToPreamble) {
+  Config Cfg;
+  Cfg.Diagnostics.AllowStalePreamble = true;
+  Cfg.Diagnostics.UnusedIncludes = Config::UnusedIncludesPolicy::Strict;
+  WithContextValue WithCfg(Config::Key, std::move(Cfg));
+
+  llvm::StringMap 

[PATCH] D143093: [clangd] #undef macros inside preamble patch

[Kadir Cetinkaya via Phabricator via cfe-commits]

This revision was automatically updated to reflect the committed changes.
Closed by commit rG75ae784e8f49: [clangd] #undef macros inside preamble patch 
(authored by kadircet).

Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D143093/new/

https://reviews.llvm.org/D143093

Files:
  clang-tools-extra/clangd/Preamble.cpp
  clang-tools-extra/clangd/unittests/PreambleTests.cpp

Index: clang-tools-extra/clangd/unittests/PreambleTests.cpp
===
--- clang-tools-extra/clangd/unittests/PreambleTests.cpp
+++ clang-tools-extra/clangd/unittests/PreambleTests.cpp
@@ -288,6 +288,7 @@
 #define BAR
 [[BAR]])cpp",
   R"cpp(#line 0 ".*main.cpp"
+#undef BAR
 #line 2
 #define BAR
 )cpp",
@@ -299,6 +300,7 @@
 
 [[BAR]])cpp",
   R"cpp(#line 0 ".*main.cpp"
+#undef BAR
 #line 2
 #define BAR
 )cpp",
@@ -310,6 +312,7 @@
 BAR
 [[BAR]])cpp",
   R"cpp(#line 0 ".*main.cpp"
+#undef BAR
 #line 3
 #define BAR
 )cpp",
@@ -342,8 +345,10 @@
   )cpp");
 
   llvm::StringLiteral ExpectedPatch(R"cpp(#line 0 ".*main.cpp"
+#undef BAR
 #line 2
 #define BAR\(X, Y\) X Y
+#undef BAR
 #line 3
 #define BAR\(X\) X
 )cpp");
@@ -693,23 +698,17 @@
   {
 Annotations Code("#define [[FOO]] 1\n");
 // Check ranges for notes.
-Annotations NewCode(R"(#define $barxyz[[BARXYZ]] 1
+Annotations NewCode(R"(#define BARXYZ 1
 #define $foo1[[FOO]] 1
 void foo();
 #define $foo2[[FOO]] 2)");
 auto AST = createPatchedAST(Code.code(), NewCode.code(), AdditionalFiles);
 EXPECT_THAT(
 *AST->getDiagnostics(),
-ElementsAre(
-// FIXME: This diagnostics shouldn't exist. It's emitted from the
-// preamble patch to the stale location inside preamble.
-AllOf(Field(::Name, Eq("-Wmacro-redefined")),
-  Field(::File, HasSubstr("_preamble_patch_")),
-  withNote(Diag(NewCode.range("barxyz",
-AllOf(
-Diag(NewCode.range("foo2"), "-Wmacro-redefined"),
-// FIXME: This should be translated into main file.
-withNote(Field(::File, HasSubstr("_preamble_patch_"));
+ElementsAre(AllOf(
+Diag(NewCode.range("foo2"), "-Wmacro-redefined"),
+// FIXME: This should be translated into main file.
+withNote(Field(::File, HasSubstr("_preamble_patch_"));
   }
 }
 
Index: clang-tools-extra/clangd/Preamble.cpp
===
--- clang-tools-extra/clangd/Preamble.cpp
+++ clang-tools-extra/clangd/Preamble.cpp
@@ -213,6 +213,9 @@
   // Full text that's representing the directive, including the `#`.
   std::string Text;
   unsigned Offset;
+  tok::PPKeywordKind Directive = tok::PPKeywordKind::pp_not_keyword;
+  // Name of the macro being defined in the case of a #define directive.
+  std::string MacroName;
 
   bool operator==(const TextualPPDirective ) const {
 return std::tie(DirectiveLine, Offset, Text) ==
@@ -283,6 +286,8 @@
   return;
 TextualDirectives.emplace_back();
 TextualPPDirective  = TextualDirectives.back();
+TD.Directive = tok::pp_define;
+TD.MacroName = MacroNameTok.getIdentifierInfo()->getName().str();
 
 const auto *MI = MD->getMacroInfo();
 TD.Text =
@@ -560,8 +565,8 @@
 
   if (BuiltPreamble) {
 log("Built preamble of size {0} for file {1} version {2} in {3} seconds",
- BuiltPreamble->getSize(), FileName, Inputs.Version,
- PreambleTimer.getTime());
+BuiltPreamble->getSize(), FileName, Inputs.Version,
+PreambleTimer.getTime());
 std::vector Diags = PreambleDiagnostics.take();
 auto Result = std::make_shared(std::move(*BuiltPreamble));
 Result->Version = Inputs.Version;
@@ -724,6 +729,10 @@
 // reduce complexity. The former might cause problems because scanning is
 // imprecise and might pick directives from disabled regions.
 for (const auto  : ModifiedScan->TextualDirectives) {
+  // Introduce an #undef directive before #defines to suppress any
+  // re-definition warnings.
+  if (TD.Directive == tok::pp_define)
+Patch << "#undef " << TD.MacroName << '\n';
   Patch << "#line " << TD.DirectiveLine << '\n';
   Patch << TD.Text << '\n';
 }
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D144484: [Tooling/Inclusion] Handle std::get symbol.

[Kadir Cetinkaya via Phabricator via cfe-commits]

kadircet accepted this revision.
kadircet added a comment.
This revision is now accepted and ready to land.

thanks!


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D144484/new/

https://reviews.llvm.org/D144484

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D143509: Move the BySpelling map to IncludeStructure.

[Kadir Cetinkaya via Phabricator via cfe-commits]

kadircet accepted this revision.
kadircet added a comment.
This revision is now accepted and ready to land.

thanks (both for the patch and for bearing with me)!


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D143509/new/

https://reviews.llvm.org/D143509

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D143509: Move the BySpelling map to IncludeStructure.

[Kadir Cetinkaya via Phabricator via cfe-commits]

kadircet added inline comments.



Comment at: clang-tools-extra/clangd/Headers.cpp:303
+  llvm::SmallVector Includes;
+  auto It = MainFileIncludesBySpelling.find(Spelling);
+  if (It == MainFileIncludesBySpelling.end())

VitaNuo wrote:
> kadircet wrote:
> > nit:
> > ```
> > llvm::SmallVector Includes;
> > for (auto Idx : MainFileIncludesBySpelling.lookup(Spelling))
> >   Includes.push_back([Idx]);
> > return Includes;
> > ```
> But `lookup` will return 0 if it doesn't find the spelling in the map 
> (default value of `int`). And at the same time 0 is a valid index. Isn't that 
> just wrong?
`MainFileIncludesBySpelling` is a map with values of type vector, not int. 
hence it'll be an empty vector, not `0`.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D143509/new/

https://reviews.llvm.org/D143509

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D143509: Move the BySpelling map to IncludeStructure.

[Kadir Cetinkaya via Phabricator via cfe-commits]

kadircet added inline comments.



Comment at: clang-tools-extra/clangd/Headers.cpp:303
+  llvm::SmallVector Includes;
+  auto It = MainFileIncludesBySpelling.find(Spelling);
+  if (It == MainFileIncludesBySpelling.end())

nit:
```
llvm::SmallVector Includes;
for (auto Idx : MainFileIncludesBySpelling.lookup(Spelling))
  Includes.push_back([Idx]);
return Includes;
```



Comment at: clang-tools-extra/clangd/Headers.h:167
+  // Spelling should include brackets or quotes, e.g. .
+  llvm::SmallVector
+  mainFileIncludesWithSpelling(llvm::StringRef Spelling) const {

kadircet wrote:
> we're still returning just the `HeaderID`, the suggestion was to return 
> `llvm::SmallVector` so that applications can work with other 
> information available in the `Inclusion`. we also won't have any requirements 
> around include being resolved that way. the application should figure out 
> what to do if `HeaderID` is missing.
> 
> also can you move this function body to source file instead?
almost, but not right. we're returning copies of `Inclusion`s here, not 
pointers to `Inclusion`s inside the main file. the suggested signature was 
`llvm::SmallVector` any reason for returning by value? 



Comment at: clang-tools-extra/clangd/IncludeCleaner.cpp:516
+ Includes.mainFileIncludesWithSpelling(H.verbatim())) {
+  std::optional HeaderOpt = Inc.HeaderID;
+  if (HeaderOpt.has_value()) {

nit:
```
if(!Inc.HeaderID.has_value())
  continue;
Used.insert(static_cast(*Inc.HeaderID));
```


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D143509/new/

https://reviews.llvm.org/D143509

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D144456: [clangd] Publish diagnostics with stale preambles

[Kadir Cetinkaya via Phabricator via cfe-commits]

kadircet updated this revision to Diff 499224.
kadircet marked 7 inline comments as done.
kadircet added a comment.

- Notify preamble peer before building an AST, to concurrently build fresh 
preamble & AST.
- Refactoring for tests


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D144456/new/

https://reviews.llvm.org/D144456

Files:
  clang-tools-extra/clangd/TUScheduler.cpp
  clang-tools-extra/clangd/unittests/TUSchedulerTests.cpp

Index: clang-tools-extra/clangd/unittests/TUSchedulerTests.cpp
===
--- clang-tools-extra/clangd/unittests/TUSchedulerTests.cpp
+++ clang-tools-extra/clangd/unittests/TUSchedulerTests.cpp
@@ -8,6 +8,8 @@
 
 #include "Annotations.h"
 #include "ClangdServer.h"
+#include "Compiler.h"
+#include "Config.h"
 #include "Diagnostics.h"
 #include "GlobalCompilationDatabase.h"
 #include "Matchers.h"
@@ -21,7 +23,6 @@
 #include "support/Path.h"
 #include "support/TestTracer.h"
 #include "support/Threading.h"
-#include "support/ThreadsafeFS.h"
 #include "clang/Basic/DiagnosticDriver.h"
 #include "llvm/ADT/ArrayRef.h"
 #include "llvm/ADT/FunctionExtras.h"
@@ -31,19 +32,23 @@
 #include "llvm/ADT/StringRef.h"
 #include "gmock/gmock.h"
 #include "gtest/gtest.h"
-#include 
 #include 
 #include 
+#include 
 #include 
+#include 
 #include 
+#include 
 #include 
 #include 
 #include 
+#include 
 
 namespace clang {
 namespace clangd {
 namespace {
 
+using ::testing::_;
 using ::testing::AllOf;
 using ::testing::AnyOf;
 using ::testing::Contains;
@@ -1194,6 +1199,118 @@
   EXPECT_EQ(PreamblePublishCount, 2);
 }
 
+TEST_F(TUSchedulerTests, PublishWithStalePreamble) {
+  // Callbacks that blocks the preamble thread after the first preamble is
+  // built and stores preamble/main-file versions for diagnostics released.
+  class BlockPreambleThread : public ParsingCallbacks {
+  public:
+using DiagsCB = std::function;
+BlockPreambleThread(Notification , DiagsCB CB)
+: UnblockPreamble(UnblockPreamble), CB(std::move(CB)) {}
+
+void onPreambleAST(PathRef Path, llvm::StringRef Version,
+   const CompilerInvocation &, ASTContext ,
+   Preprocessor &, const CanonicalIncludes &) override {
+  if (BuildBefore)
+ASSERT_TRUE(UnblockPreamble.wait(timeoutSeconds(5)))
+<< "Expected notification";
+  BuildBefore = true;
+}
+
+void onMainAST(PathRef File, ParsedAST , PublishFn Publish) override {
+  CB(AST);
+}
+
+void onFailedAST(PathRef File, llvm::StringRef Version,
+ std::vector Diags, PublishFn Publish) override {
+  ADD_FAILURE() << "Received failed ast for: " << File << " with version "
+<< Version << '\n';
+}
+
+  private:
+bool BuildBefore = false;
+Notification 
+std::function CB;
+  };
+
+  // Helpers for issuing blocking update requests on a TUScheduler, whose
+  // onMainAST callback would call onDiagnostics.
+  class DiagCollector {
+  public:
+void onDiagnostics(ParsedAST ) {
+  std::scoped_lock Lock(DiagMu);
+  DiagVersions.emplace_back(
+  std::make_pair(AST.preambleVersion()->str(), AST.version().str()));
+  DiagsReceived.notify_all();
+}
+
+std::pair
+waitForNewDiags(TUScheduler , PathRef File, ParseInputs PI) {
+  std::unique_lock Lock(DiagMu);
+  // Perform the update under the lock to make sure it isn't handled until
+  // we're waiting for it.
+  S.update(File, std::move(PI), WantDiagnostics::Auto);
+  size_t OldSize = DiagVersions.size();
+  bool ReceivedDiags = DiagsReceived.wait_for(
+  Lock, std::chrono::seconds(5),
+  [this, OldSize] { return OldSize + 1 == DiagVersions.size(); });
+  if (!ReceivedDiags) {
+ADD_FAILURE() << "Timed out waiting for diags";
+return {"invalid", "version"};
+  }
+  return DiagVersions.back();
+}
+
+std::vector> diagVersions() {
+  std::scoped_lock Lock(DiagMu);
+  return DiagVersions;
+}
+
+  private:
+std::condition_variable DiagsReceived;
+std::mutex DiagMu;
+std::vector>
+DiagVersions;
+  };
+
+  Config Cfg;
+  Cfg.Diagnostics.AllowStalePreamble = true;
+  WithContextValue WithCfg(Config::Key, std::move(Cfg));
+
+  DiagCollector Collector;
+  Notification UnblockPreamble;
+  auto DiagCallbacks = std::make_unique(
+  UnblockPreamble,
+  [](ParsedAST ) { Collector.onDiagnostics(AST); });
+  TUScheduler S(CDB, optsForTest(), std::move(DiagCallbacks));
+  Path File = testPath("foo.cpp");
+  auto BlockForDiags = [&](ParseInputs PI) {
+return Collector.waitForNewDiags(S, File, std::move(PI));
+  };
+
+  // Build first preamble.
+  auto PI = getInputs(File, "");
+  PI.Version = PI.Contents = "1";
+  ASSERT_THAT(BlockForDiags(PI), testing::Pair("1", "1"));
+
+  // Now preamble thread is blocked, so rest of the requests sees only the
+  // 

[PATCH] D143496: [clangd] Add support for missing includes analysis.

[Kadir Cetinkaya via Phabricator via cfe-commits]

kadircet added inline comments.



Comment at: clang-tools-extra/clangd/IncludeCleaner.cpp:359
+TransformedInc.Angled = WrittenRef.starts_with("<");
+auto FE = SM.getFileManager().getFile(Inc.Resolved);
+if (!FE)

unfortunately `getFile` returns an `llvm::Expected` which requires explicit 
error handling (or it'll trigger a crash). you can simply `elog` the issue:
```
if (!FE) {
  elog("IncludeCleaner: Failed to get an entry for resolved path {0}: {1}", 
Inc.Resolved, FE.takeError());
  continue;
}
```



Comment at: clang-tools-extra/clangd/IncludeCleaner.cpp:519
+std::vector
+collectMacroReferences(ParsedAST ) {
+  const auto  = AST.getSourceManager();

this should also be either static or put into anon namespace



Comment at: clang-tools-extra/clangd/IncludeCleaner.cpp:554
+
+std::vector generateMissingIncludeDiagnostics(
+ParsedAST , llvm::SmallVector MissingIncludes,

`generateMissingIncludeDiagnostics` should also be either static or put into 
anon namespace



Comment at: clang-tools-extra/clangd/IncludeCleaner.cpp:555
+std::vector generateMissingIncludeDiagnostics(
+ParsedAST , llvm::SmallVector MissingIncludes,
+llvm::StringRef Code) {

i think it's better to use an `llvm::ArrayRef` instead of `llvm::SmallVector` 
for diag infos here, we don't need to copy.



Comment at: clang-tools-extra/clangd/IncludeCleaner.cpp:563
+  for (const auto  : MissingIncludes) {
+std::string Spelling =
+spellHeader(AST, MainFile, SymbolWithMissingInclude.Providers.front());

we've got `Config::Diagnostics::Includes::IgnoreHeader` that disables 
include-cleaner analysis on headers that match a pattern. we should be 
respecting those config options here too.



Comment at: clang-tools-extra/clangd/IncludeCleaner.cpp:582
+HeaderRef.trim("\"<>"), Angled, tooling::IncludeDirective::Include);
+if (!Replacement.has_value())
   continue;

could you add a comment here, as this is subtle, something like `We might 
suggest insertion of an existing include in edge cases, e.g. include is present 
in a PP-disabled region, or spelling of the header turns out to be the same as 
one of the unresolved includes in the main file`



Comment at: clang-tools-extra/clangd/IncludeCleaner.cpp:589
+D.InsideMainFile = true;
+Result.push_back(D);
+  }

nit: `Result.push_back(std::move(D))`, as `D` has lots of strings in it, it 
might be expensive to copy it around. Even better if you construct the Diag in 
place via `Diag  = Result.emplace_back()`, you can achieve this by moving all 
the logic that might bail out (e.g. replacement generation) to the top of the 
loop, and start forming the diagnostic only after we're sure that there'll be 
one.



Comment at: clang-tools-extra/clangd/IncludeCleaner.cpp:595
+std::vector
+generateUnusedIncludeDiagnostics(PathRef FileName,
+ std::vector UnusedIncludes,

this also needs to be static or put into anon namespace



Comment at: clang-tools-extra/clangd/IncludeCleaner.cpp:596
+generateUnusedIncludeDiagnostics(PathRef FileName,
+ std::vector UnusedIncludes,
+ llvm::StringRef Code) {

no need to copy the vector by taking a `std::vector` here, you can take an 
`llvm::ArrayRef` instead.



Comment at: clang-tools-extra/clangd/IncludeCleaner.cpp:623
 D.InsideMainFile = true;
-Result.push_back(std::move(D));
+Result.push_back(D);
+  }

can you restore `std::move`?



Comment at: clang-tools-extra/clangd/IncludeCleaner.cpp:632
+  include_cleaner::Includes ConvertedIncludes =
+  convertIncludes(AST.getSourceManager(), Includes.MainFileIncludes);
+  const FileEntry *MainFile = SM.getFileEntryForID(SM.getMainFileID());

s/AST.getSourceManager()/SM



Comment at: clang-tools-extra/clangd/IncludeCleaner.cpp:639
+  llvm::DenseSet Used;
+  include_cleaner::walkUsed(
+  AST.getLocalTopLevelDecls(), /*MacroRefs=*/Macros,

can you introduce a `trace::Span` wrapping the call to `walkUsed` with name 
`IncludeCleanerAnalysis` so that we can collect some stats about latency here?



Comment at: clang-tools-extra/clangd/IncludeCleaner.cpp:651
+  }
+  for (auto * : ConvertedIncludes.match(H)) {
+Satisfied = true;

nit: drop either `*` or `&` (preferably `&`), having a reference vs a pointer 
doesn't make any differences performance wise, but creates a confusion (as we 
don't realy need a reference to a pointer here)



Comment at: clang-tools-extra/clangd/IncludeCleaner.cpp:661
+const 

[PATCH] D144456: [clangd] Publish diagnostics with stale preambles

[Kadir Cetinkaya via Phabricator via cfe-commits]

kadircet created this revision.
kadircet added a reviewer: sammccall.
Herald added subscribers: arphaman, javed.absar.
Herald added a project: All.
kadircet requested review of this revision.
Herald added subscribers: cfe-commits, MaskRay, ilya-biryukov.
Herald added a project: clang-tools-extra.

This patch achieves this by building an AST and invoking main file
callbacks on each update, in addition to preamble updates.

It means we might have some extra AST builds now (e.g. if an update was
with a stale preamble and there were no reads on it, we would only build
an AST once we had the fresh preamble. Now we'll build 2, once with the
stale preamble and another with the fresh one, but we'll have one more
diagnostics cycle in between.).

This patch preserves forward progress of diagnostics by always using the
latest main file contents when emitting diagnostics after preamble
builds. It also guarantees eventual consistency:

- if an update doesn't invalidate preamble, we'll emit diagnostics with fresh 
preamble already.
- if an update invalidates preamble, we'll first emit diagnostics with stale 
contents, and then once the preamble build finishes it'll emit diagnostics (as 
preamble has changed) with newest version.

This has implications on parsing callbacks, as previously onMainAST
callback was called at most once, now it can be called up to 2 times.
All of the existing clients can already deal with callback firing
multiple times.


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D144456

Files:
  clang-tools-extra/clangd/TUScheduler.cpp
  clang-tools-extra/clangd/unittests/TUSchedulerTests.cpp

Index: clang-tools-extra/clangd/unittests/TUSchedulerTests.cpp
===
--- clang-tools-extra/clangd/unittests/TUSchedulerTests.cpp
+++ clang-tools-extra/clangd/unittests/TUSchedulerTests.cpp
@@ -8,6 +8,8 @@
 
 #include "Annotations.h"
 #include "ClangdServer.h"
+#include "Compiler.h"
+#include "Config.h"
 #include "Diagnostics.h"
 #include "GlobalCompilationDatabase.h"
 #include "Matchers.h"
@@ -21,7 +23,6 @@
 #include "support/Path.h"
 #include "support/TestTracer.h"
 #include "support/Threading.h"
-#include "support/ThreadsafeFS.h"
 #include "clang/Basic/DiagnosticDriver.h"
 #include "llvm/ADT/ArrayRef.h"
 #include "llvm/ADT/FunctionExtras.h"
@@ -31,19 +32,22 @@
 #include "llvm/ADT/StringRef.h"
 #include "gmock/gmock.h"
 #include "gtest/gtest.h"
-#include 
 #include 
 #include 
+#include 
 #include 
 #include 
+#include 
 #include 
 #include 
 #include 
+#include 
 
 namespace clang {
 namespace clangd {
 namespace {
 
+using ::testing::_;
 using ::testing::AllOf;
 using ::testing::AnyOf;
 using ::testing::Contains;
@@ -1194,6 +1198,96 @@
   EXPECT_EQ(PreamblePublishCount, 2);
 }
 
+TEST_F(TUSchedulerTests, PublishWithStalePreamble) {
+  class BlockPreambleThread : public ParsingCallbacks {
+  public:
+BlockPreambleThread(Notification ) : N(N) {}
+void onPreambleAST(PathRef Path, llvm::StringRef Version,
+   const CompilerInvocation &, ASTContext ,
+   Preprocessor &, const CanonicalIncludes &) override {
+  if (BuildBefore)
+ASSERT_TRUE(N.wait(timeoutSeconds(5))) << "Expected notification";
+  BuildBefore = true;
+}
+void onMainAST(PathRef File, ParsedAST , PublishFn Publish) override {
+  std::scoped_lock Lock(DiagMu);
+  DiagVersions.emplace_back(
+  std::make_pair(AST.preambleVersion()->str(), AST.version().str()));
+  DiagsReceived.notify_all();
+}
+
+void onFailedAST(PathRef File, llvm::StringRef Version,
+ std::vector Diags, PublishFn Publish) override {
+  ADD_FAILURE() << "Received failed ast for: " << File << " with version "
+<< Version << '\n';
+}
+
+std::optional>
+WaitForNewDiags(TUScheduler , PathRef File, ParseInputs PI) {
+  std::unique_lock Lock(DiagMu);
+  // Perform the update under the lock to make sure it isn't handled until
+  // we're waiting for it.
+  S.update(File, std::move(PI), WantDiagnostics::Auto);
+  size_t OldSize = DiagVersions.size();
+  bool ReceivedDiags =
+  DiagsReceived.wait_for(Lock, std::chrono::seconds(5), [&] {
+return OldSize + 1 == DiagVersions.size();
+  });
+  if (!ReceivedDiags)
+return std::nullopt;
+  return DiagVersions.back();
+}
+
+std::vector> diagVersions() {
+  std::scoped_lock Lock(DiagMu);
+  return DiagVersions;
+}
+
+  private:
+bool BuildBefore = false;
+Notification 
+std::mutex DiagMu;
+std::condition_variable DiagsReceived;
+std::vector>
+DiagVersions;
+  };
+
+  Config Cfg;
+  Cfg.Diagnostics.AllowStalePreamble = true;
+  WithContextValue WithCfg(Config::Key, std::move(Cfg));
+
+  Notification BlockPreamble;
+  auto DiagCallbacks = std::make_unique(BlockPreamble);
+  auto DCPtr = DiagCallbacks.get();
+  

[PATCH] D143096: [clangd] Provide patched diagnostics with preamble patch

[Kadir Cetinkaya via Phabricator via cfe-commits]

kadircet added a comment.

thanks for the review!




Comment at: clang-tools-extra/clangd/Preamble.cpp:509
+  // Check if AlternateLine matches all lines in the range.
+  if (llvm::any_of(
+  llvm::zip_equal(RangeContents,

sammccall wrote:
> why not just `if (RangeContents != CurrentLines.slice(...))`? I feel like I'm 
> missing something subtle :-)
> 
> Also if you replace the `slice(a, b)` with `drop_front(a).take_front(b)` then 
> you don't need the `CurrentEnd > CurrentLines.size()` check
> why not just if (RangeContents != CurrentLines.slice(...))? I feel like I'm 
> missing something subtle :-)

no that's absolutely right, i guess i had a brain fart  


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D143096/new/

https://reviews.llvm.org/D143096

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D143096: [clangd] Provide patched diagnostics with preamble patch

[Kadir Cetinkaya via Phabricator via cfe-commits]

kadircet updated this revision to Diff 498351.
kadircet marked 4 inline comments as done.
kadircet added a comment.

- Use direct comparison of ArrayRefs
- optional for closest
- loop over lookup results rather than through iterator


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D143096/new/

https://reviews.llvm.org/D143096

Files:
  clang-tools-extra/clangd/ParsedAST.cpp
  clang-tools-extra/clangd/Preamble.cpp
  clang-tools-extra/clangd/Preamble.h
  clang-tools-extra/clangd/unittests/PreambleTests.cpp

Index: clang-tools-extra/clangd/unittests/PreambleTests.cpp
===
--- clang-tools-extra/clangd/unittests/PreambleTests.cpp
+++ clang-tools-extra/clangd/unittests/PreambleTests.cpp
@@ -723,9 +723,8 @@
 #define BAR
 #include [[]])");
 auto AST = createPatchedAST(Code.code(), NewCode.code());
-// FIXME: We should point at the correct coordinates in NewCode.
 EXPECT_THAT(*AST->getDiagnostics(),
-ElementsAre(Diag(Code.range(), "pp_file_not_found")));
+ElementsAre(Diag(NewCode.range(), "pp_file_not_found")));
   }
   {
 // Check with removals from preamble.
@@ -734,18 +733,15 @@
 #include [[]])");
 Annotations NewCode("#include [[]]");
 auto AST = createPatchedAST(Code.code(), NewCode.code());
-// FIXME: We should point at the correct coordinates in NewCode.
 EXPECT_THAT(*AST->getDiagnostics(),
-ElementsAre(Diag(Code.range(), "pp_file_not_found")));
+ElementsAre(Diag(NewCode.range(), "pp_file_not_found")));
   }
   {
 // Drop line with diags.
 Annotations Code("#include [[]]");
 Annotations NewCode("#define BAR\n#define BAZ\n");
 auto AST = createPatchedAST(Code.code(), NewCode.code());
-// FIXME: No diagnostics.
-EXPECT_THAT(*AST->getDiagnostics(),
-ElementsAre(Diag(Code.range(), "pp_file_not_found")));
+EXPECT_THAT(*AST->getDiagnostics(), IsEmpty());
   }
   {
 // Picks closest line in case of multiple alternatives.
@@ -756,18 +752,79 @@
 #define BAR
 #include )");
 auto AST = createPatchedAST(Code.code(), NewCode.code());
-// FIXME: We should point at the correct coordinates in NewCode.
 EXPECT_THAT(*AST->getDiagnostics(),
-ElementsAre(Diag(Code.range(), "pp_file_not_found")));
+ElementsAre(Diag(NewCode.range(), "pp_file_not_found")));
   }
   {
 // Drop diag if line spelling has changed.
 Annotations Code("#include [[]]");
 Annotations NewCode(" # include ");
 auto AST = createPatchedAST(Code.code(), NewCode.code());
-// FIXME: No diags.
+EXPECT_THAT(*AST->getDiagnostics(), IsEmpty());
+  }
+  {
+// Multiple lines.
+Annotations Code(R"(
+#define BAR
+#include [[]])");
+Annotations NewCode(R"(#include [[]])");
+auto AST = createPatchedAST(Code.code(), NewCode.code());
 EXPECT_THAT(*AST->getDiagnostics(),
-ElementsAre(Diag(Code.range(), "pp_file_not_found")));
+ElementsAre(Diag(NewCode.range(), "pp_file_not_found")));
+  }
+  {
+// Multiple lines with change.
+Annotations Code(R"(
+#define BAR
+#include 
+#include [[]])");
+Annotations NewCode(R"(#include )");
+auto AST = createPatchedAST(Code.code(), NewCode.code());
+EXPECT_THAT(*AST->getDiagnostics(), IsEmpty());
+  }
+  {
+// Preserves notes.
+Annotations Code(R"(
+#define $note[[BAR]] 1
+#define $main[[BAR]] 2)");
+Annotations NewCode(R"(
+#define BAZ 0
+#define $note[[BAR]] 1
+#define BAZ 0
+#define $main[[BAR]] 2)");
+auto AST = createPatchedAST(Code.code(), NewCode.code());
+EXPECT_THAT(
+*AST->getDiagnostics(),
+ElementsAre(AllOf(Diag(NewCode.range("main"), "-Wmacro-redefined"),
+  withNote(Diag(NewCode.range("note"));
+  }
+  {
+// Preserves diag without note.
+Annotations Code(R"(
+#define $note[[BAR]] 1
+#define $main[[BAR]] 2)");
+Annotations NewCode(R"(
+#define $main[[BAR]] 2)");
+auto AST = createPatchedAST(Code.code(), NewCode.code());
+EXPECT_THAT(
+*AST->getDiagnostics(),
+ElementsAre(AllOf(Diag(NewCode.range("main"), "-Wmacro-redefined"),
+  Field(::Notes, IsEmpty();
+  }
+  {
+// Make sure orphaned notes are not promoted to diags.
+Annotations Code(R"(
+#define $note[[BAR]] 1
+#define $main[[BAR]] 2)");
+Annotations NewCode(R"(
+#define BAZ 0
+#define BAR 1)");
+auto AST = createPatchedAST(Code.code(), NewCode.code());
+EXPECT_THAT(*AST->getDiagnostics(), IsEmpty());
   }
 }
 } // namespace
Index: clang-tools-extra/clangd/Preamble.h
===
--- clang-tools-extra/clangd/Preamble.h
+++ clang-tools-extra/clangd/Preamble.h
@@ -34,6 +34,7 @@
 #include "clang/Frontend/PrecompiledPreamble.h"
 #include "clang/Lex/Lexer.h"
 #include 

[PATCH] D143096: [clangd] Provide patched diagnostics with preamble patch

[Kadir Cetinkaya via Phabricator via cfe-commits]

kadircet added inline comments.



Comment at: clang-tools-extra/clangd/Preamble.cpp:625
+// same spelling.
+static std::vector patchDiags(llvm::ArrayRef BaselineDiags,
+const ScannedPreamble ,

sammccall wrote:
> I think this function is too long with too many local lambdas, consider 
> converting it to a class instead (call me old-fashioned!)
well it's gonna be an equally long class, but sure :D



Comment at: clang-tools-extra/clangd/Preamble.cpp:645
+auto ModifiedStartIt =
+ModifiedContentsToLine.find(BaselineScan.Lines[BaselineStart]);
+if (ModifiedStartIt == ModifiedContentsToLine.end())

sammccall wrote:
> we're assuming that the ranges are within the preamble and everyone agrees 
> about the bounds. If not, BaselineStart may not be a valid index into 
> BaselineScan.Lines
> 
> This assumption seems broadly reasonable, but is *very much* not locally 
> enforced. I'd suggest a defensive check or at least an assert.
oops, thanks! both here and also the multi-ine check below.



Comment at: clang-tools-extra/clangd/Preamble.cpp:648
+  return std::nullopt;
+int Closest = ModifiedStartIt->second.front();
+for (auto AlternateLine : ModifiedStartIt->second) {

sammccall wrote:
> this doesn't look right: you're first deciding which possible starting point 
> is closest, and then deciding whether it matches. So a range that matches can 
> be masked by a range where only the first line matches, if the latter is 
> closer.
i thought because this would be rare it's fine to not do that, but the same 
applies in the other direction as well :D
moved the content based matching inside the loop to consider a line as 
alternative only after it matches the contents.



Comment at: clang-tools-extra/clangd/Preamble.cpp:706
+Diag NewDiag;
+// Copy all fields but Notes, Fixes, Name and Tags.
+static_cast(NewDiag) = static_cast(D);

sammccall wrote:
> reasoning about the fields you're *not* rewriting feels fragile. (Here and in 
> TranslateFix).
> 
> Consider copying the whole object and mutating in place (`bool 
> TranslateDiag(Diag&)` together with `erase_if`)
well i am a little worried about copying around notes/fixes just to drop them 
again (which can have quite some strings). but probably rare enough in practice 
to not matter.



Comment at: clang-tools-extra/clangd/unittests/PreambleTests.cpp:820
+  {
+// Note is also dropped if diag is gone.
+Annotations Code(R"(

sammccall wrote:
> i'm confused about what this comment is getting at - a note without a diag is 
> not even representable
well, it is to make sure we're not promoting an orphaned note to a main diag. 
updated the comment but happy to drop if you think it's confusing rather than 
being helpful.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D143096/new/

https://reviews.llvm.org/D143096

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D143096: [clangd] Provide patched diagnostics with preamble patch

[Kadir Cetinkaya via Phabricator via cfe-commits]

kadircet updated this revision to Diff 498308.
kadircet marked 6 inline comments as done.
kadircet added a comment.

- Move patch translation logic to a separate class
- Perform eager copies and use `bool translateX(X&)` type of APIs instead of 
returning optionals.
- Perform multi line content check for each alternative, not just the closest 
match.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D143096/new/

https://reviews.llvm.org/D143096

Files:
  clang-tools-extra/clangd/ParsedAST.cpp
  clang-tools-extra/clangd/Preamble.cpp
  clang-tools-extra/clangd/Preamble.h
  clang-tools-extra/clangd/unittests/PreambleTests.cpp

Index: clang-tools-extra/clangd/unittests/PreambleTests.cpp
===
--- clang-tools-extra/clangd/unittests/PreambleTests.cpp
+++ clang-tools-extra/clangd/unittests/PreambleTests.cpp
@@ -723,9 +723,8 @@
 #define BAR
 #include [[]])");
 auto AST = createPatchedAST(Code.code(), NewCode.code());
-// FIXME: We should point at the correct coordinates in NewCode.
 EXPECT_THAT(*AST->getDiagnostics(),
-ElementsAre(Diag(Code.range(), "pp_file_not_found")));
+ElementsAre(Diag(NewCode.range(), "pp_file_not_found")));
   }
   {
 // Check with removals from preamble.
@@ -734,18 +733,15 @@
 #include [[]])");
 Annotations NewCode("#include [[]]");
 auto AST = createPatchedAST(Code.code(), NewCode.code());
-// FIXME: We should point at the correct coordinates in NewCode.
 EXPECT_THAT(*AST->getDiagnostics(),
-ElementsAre(Diag(Code.range(), "pp_file_not_found")));
+ElementsAre(Diag(NewCode.range(), "pp_file_not_found")));
   }
   {
 // Drop line with diags.
 Annotations Code("#include [[]]");
 Annotations NewCode("#define BAR\n#define BAZ\n");
 auto AST = createPatchedAST(Code.code(), NewCode.code());
-// FIXME: No diagnostics.
-EXPECT_THAT(*AST->getDiagnostics(),
-ElementsAre(Diag(Code.range(), "pp_file_not_found")));
+EXPECT_THAT(*AST->getDiagnostics(), IsEmpty());
   }
   {
 // Picks closest line in case of multiple alternatives.
@@ -756,18 +752,79 @@
 #define BAR
 #include )");
 auto AST = createPatchedAST(Code.code(), NewCode.code());
-// FIXME: We should point at the correct coordinates in NewCode.
 EXPECT_THAT(*AST->getDiagnostics(),
-ElementsAre(Diag(Code.range(), "pp_file_not_found")));
+ElementsAre(Diag(NewCode.range(), "pp_file_not_found")));
   }
   {
 // Drop diag if line spelling has changed.
 Annotations Code("#include [[]]");
 Annotations NewCode(" # include ");
 auto AST = createPatchedAST(Code.code(), NewCode.code());
-// FIXME: No diags.
+EXPECT_THAT(*AST->getDiagnostics(), IsEmpty());
+  }
+  {
+// Multiple lines.
+Annotations Code(R"(
+#define BAR
+#include [[]])");
+Annotations NewCode(R"(#include [[]])");
+auto AST = createPatchedAST(Code.code(), NewCode.code());
 EXPECT_THAT(*AST->getDiagnostics(),
-ElementsAre(Diag(Code.range(), "pp_file_not_found")));
+ElementsAre(Diag(NewCode.range(), "pp_file_not_found")));
+  }
+  {
+// Multiple lines with change.
+Annotations Code(R"(
+#define BAR
+#include 
+#include [[]])");
+Annotations NewCode(R"(#include )");
+auto AST = createPatchedAST(Code.code(), NewCode.code());
+EXPECT_THAT(*AST->getDiagnostics(), IsEmpty());
+  }
+  {
+// Preserves notes.
+Annotations Code(R"(
+#define $note[[BAR]] 1
+#define $main[[BAR]] 2)");
+Annotations NewCode(R"(
+#define BAZ 0
+#define $note[[BAR]] 1
+#define BAZ 0
+#define $main[[BAR]] 2)");
+auto AST = createPatchedAST(Code.code(), NewCode.code());
+EXPECT_THAT(
+*AST->getDiagnostics(),
+ElementsAre(AllOf(Diag(NewCode.range("main"), "-Wmacro-redefined"),
+  withNote(Diag(NewCode.range("note"));
+  }
+  {
+// Preserves diag without note.
+Annotations Code(R"(
+#define $note[[BAR]] 1
+#define $main[[BAR]] 2)");
+Annotations NewCode(R"(
+#define $main[[BAR]] 2)");
+auto AST = createPatchedAST(Code.code(), NewCode.code());
+EXPECT_THAT(
+*AST->getDiagnostics(),
+ElementsAre(AllOf(Diag(NewCode.range("main"), "-Wmacro-redefined"),
+  Field(::Notes, IsEmpty();
+  }
+  {
+// Make sure orphaned notes are not promoted to diags.
+Annotations Code(R"(
+#define $note[[BAR]] 1
+#define $main[[BAR]] 2)");
+Annotations NewCode(R"(
+#define BAZ 0
+#define BAR 1)");
+auto AST = createPatchedAST(Code.code(), NewCode.code());
+EXPECT_THAT(*AST->getDiagnostics(), IsEmpty());
   }
 }
 } // namespace
Index: clang-tools-extra/clangd/Preamble.h
===
--- clang-tools-extra/clangd/Preamble.h
+++ clang-tools-extra/clangd/Preamble.h
@@ -34,6 

[PATCH] D143509: Move the BySpelling map to IncludeStructure.

[Kadir Cetinkaya via Phabricator via cfe-commits]

kadircet added inline comments.



Comment at: clang-tools-extra/clangd/Headers.h:167
+  // Spelling should include brackets or quotes, e.g. .
+  llvm::SmallVector
+  mainFileIncludesWithSpelling(llvm::StringRef Spelling) const {

we're still returning just the `HeaderID`, the suggestion was to return 
`llvm::SmallVector` so that applications can work with other 
information available in the `Inclusion`. we also won't have any requirements 
around include being resolved that way. the application should figure out what 
to do if `HeaderID` is missing.

also can you move this function body to source file instead?


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D143509/new/

https://reviews.llvm.org/D143509

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D143496: [clangd] Add support for missing includes analysis.

[Kadir Cetinkaya via Phabricator via cfe-commits]

kadircet added a comment.

Thanks! Details mostly looks good, but i think we had different ideas about how 
the final diagnostics should look like. Let's try to clear that up a bit. My 
suggested proposal is:

- Having a main diagnostic for each symbol that doesn't have a satisfying 
include in the main file, attached to the first use of the symbol, with message 
` providing 'ns::Foo' is not directly included`, with severity warning
- Having notes attached to this main diagnostic, on rest of the references of 
the same symbol inside the main file, with message `Also used here`, with 
severity info
- Having a single fix attached to main diagnostic inserting the first provider.

I believe in the future the main diagnostic message should change to `No header 
providing 'ns::Foo' is directly included`, and we should have multiple fixes 
attached. But there's value in explicitly spelling the header name when there's 
only one fix to get rid of indirections.

This means:

  + we'll emit different diagnostics for each unsatisfied symbol even if 
they're satisfied by the same provider.
  + the user will be able to see usage sites for each symbol as a whole (as 
we're attaching them as notes), and if need be can make a decision around 
dropping the dependency or introducing a forward decl.
  + we won't "butcher" the diagnostic panel with 100 duplicated diagnostics all 
talking about the same thing.
  - in certain editors, like vscode, related information doesn't show up as 
extra decorations in code. so if the user ignores diagnostic panel and the 
first usage of a symbol is not visible. they'll likely miss the finding.

WDYT about this approach? any other alternatives you've in mind? I think it 
might make sense to get others opinion's here as well. Especially around the 
diagnostic message, as usually whatever is convenient to a single person isn't 
necessarily clear/obvious for every one :D




Comment at: clang-tools-extra/clangd/IncludeCleaner.cpp:516
+
+  std::vector Macros =
+  collectMacroReferences(AST);

VitaNuo wrote:
> kadircet wrote:
> > 
> Why would you use `auto` here? The return type is not obvious from the 
> function call.
> 
> The style guide says: "types that you and your reviewer experience as 
> unnecessary clutter will very often provide useful information to others. For 
> example, you can assume that the return type of `make_unique()` is 
> obvious, but the return type of `MyWidgetFactory()` probably isn't." 
> (http://go/cstyle#Type_deduction)
well, feel free to keep it. but at least, inside clangd codebase, we use auto 
frequently (maybe more than we should). especially in cases like this where a 
declaration would otherwise fit a single line and because the name of the 
initializer implies this will be a container of macro references and any extra 
details about the container is probably irrelevant.



Comment at: clang-tools-extra/clangd/IncludeCleaner.cpp:547
+convertIncludes(const SourceManager ,
+std::vector MainFileIncludes) {
+  include_cleaner::Includes Includes;

VitaNuo wrote:
> kadircet wrote:
> > you can just pass an `llvm::ArrayRef` to prevent a copy
> By preventing a copy, do you mean that the construction of 
> `llvm::ArrayRef` will only copy a pointer to the data rather than 
> the whole vector? AFAIU `const std::vector&` should be even better 
> then, no copies involved. CMIIW.
well from performance-wise they're pretty close, passing by const ref doesn't 
mean you don't do any copies, it'll still require address of the entity to be 
signalled somehow, which is a pointer copy. passing an arrayref implies copying 
a pointer to data and the size (so it's slightly worse in that regard). but it 
can represent any chunk of contiguous memory, the data source doesn't need to 
be a vector. moreover you can easily pass a slice of a vector rather than the 
whole vector etc.

the performance implications rarely matters in practice, and the abstraction it 
provides on the interfaces is usually a benefit, e.g. if we were to change 
underlying type from vector to llvm::SmallVector, the APIs would still work and 
you don't need to think about any concrete types. hence we tend to prefer 
having ArrayRef on API boundaries whenever possible.



Comment at: clang-tools-extra/clangd/IncludeCleaner.cpp:512
+include_cleaner::Includes
+convertIncludes(const SourceManager ,
+const std::vector ) {

since this is a local symbol, either mark it as `static` or move it to 
anonymous namespace above so that it won't be visible to other translation 
units.



Comment at: clang-tools-extra/clangd/IncludeCleaner.cpp:513
+convertIncludes(const SourceManager ,
+const std::vector ) {
+  include_cleaner::Includes Includes;

nit: you can use `llvm::ArrayRef` instead of a `const std::vector &`. 
`ArrayRef` is a trivially 

[PATCH] D143906: [include-cleaner] Better support ambiguous std symbols

[Kadir Cetinkaya via Phabricator via cfe-commits]

kadircet accepted this revision.
kadircet added a comment.
This revision is now accepted and ready to land.

thanks!




Comment at: clang-tools-extra/include-cleaner/lib/FindHeaders.cpp:85
 
+Hints isPublicHeader(const FileEntry *FE, const PragmaIncludes *PI) {
+  return (PI->isPrivate(FE) || !PI->isSelfContained(FE)) ? Hints::None

we actually need to nullcheck for PI here, defaulting to PublicHeader when it 
isn't present.



Comment at: clang-tools-extra/include-cleaner/lib/FindHeaders.cpp:86
+Hints isPublicHeader(const FileEntry *FE, const PragmaIncludes *PI) {
+  return (PI->isPrivate(FE) || !PI->isSelfContained(FE)) ? Hints::None
+ : Hints::PublicHeader;

nit: maybe unwrap this? e.g:
```
if (PI->isPrivate(..) ... ) return Hints::None;
return Hints::PublicHeader;
```



Comment at: clang-tools-extra/include-cleaner/lib/FindHeaders.cpp:95
+Results.emplace_back(H, Hints::PublicHeader);
+for (const auto *Export : PI->getExporters(H, SM.getFileManager()))
+  Results.emplace_back(Header(Export), isPublicHeader(Export, PI));

we need to nullcheck for `PI` here



Comment at: clang-tools-extra/include-cleaner/lib/FindHeaders.cpp:134
+  }
+  return hintedHeadersForStdHeaders(Headers, SM, PI);
+}

can you also wrap this in `applyHints(..., Hints::CompleteSymbol)`, similar to 
what `locateSymbol` does for rest of the stdlib symbols


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D143906/new/

https://reviews.llvm.org/D143906

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D142836: [clangd] Add symbol mappings for `std::experimental::filesystem`

[Kadir Cetinkaya via Phabricator via cfe-commits]

kadircet accepted this revision.
kadircet added a comment.
This revision is now accepted and ready to land.

thanks!




Comment at: clang/lib/Tooling/Inclusions/Stdlib/StdTsSymbolMap.inc:6
+// whatever.
+// clang-format off
+SYMBOL(absolute, std::experimental::filesystem::, )

zyounan wrote:
> kadircet wrote:
> > zyounan wrote:
> > > kadircet wrote:
> > > > can you strip clang-format pragmas to be similar to other mapping files.
> > > I'd love to if possible, but for some reason clang-format would add extra 
> > > spaces before and after the slash, ``, which 
> > > looks ugly and I don't expect it right.
> > i don't follow, why do you **need** to run clang-format on this file? 
> > symbols are already ordered
> Specifically, it is `git clang-format`. The pipeline would fail if this 
> command does modify some files IIRC.
not sure what workflow you're using but clang-format is not a mandatory part of 
the workflow.
recommended way is updating/uploading patches using arcanist, whose config only 
contains files with one of the `cc|h|cpp` extensions, see 
https://github.com/llvm/llvm-project/blob/main/.arclint.

anyways, happy to land this for you if it's not easy to disable clang-format 
linting on your workflow for whatever reason.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D142836/new/

https://reviews.llvm.org/D142836

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D143906: [include-cleaner] Better support ambiguous std symbols

[Kadir Cetinkaya via Phabricator via cfe-commits]

kadircet added inline comments.



Comment at: clang-tools-extra/include-cleaner/lib/FindHeaders.cpp:104
+  }
+  if (FName == "remove") {
+if (FD->getNumParams() == 1)

nit: `else if`



Comment at: clang-tools-extra/include-cleaner/lib/FindHeaders.cpp:110
+  // remove(ForwardIt first, ForwardIt last, const T& value);
+  return {Header("")};
+  }

we also need to account for exporters, similar to `findHeaders`. might be worth 
lifting that branch to a free-function.



Comment at: clang-tools-extra/include-cleaner/lib/FindHeaders.cpp:174
const PragmaIncludes *PI) {
+  if (auto Headers = specialStandardSymbols(S); !Headers.empty())
+return Headers;

rather than doing this here and bailing out early, i think we should just 
augment the `Headers` below, after `locateSymbol` and `findHeaders` is run.

that way we'll make sure rest of the logic applies in the future without 
special casing (e.g. we've some fixmes for treating implementation locations 
for stdlib symbols as providers, in such a scenario this early exits would 
create non-uniformity).
it also implies we'll need to think about hints to attach here, which seems 
consistent with rest of the logic again (as we still assign hints to headers 
derived from stdlib recognizer).

so what about an alternative with a signature like: 
`llvm::SmallVector> headersForSpecialSymbols(S, SM, PI);` then 
we can just initialize Headers with a call to it?


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D143906/new/

https://reviews.llvm.org/D143906

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D142836: [clangd] Add symbol mappings for `std::experimental::filesystem`

[Kadir Cetinkaya via Phabricator via cfe-commits]

kadircet added inline comments.



Comment at: clang/lib/Tooling/Inclusions/Stdlib/StdTsSymbolMap.inc:6
+// whatever.
+// clang-format off
+SYMBOL(absolute, std::experimental::filesystem::, )

zyounan wrote:
> kadircet wrote:
> > can you strip clang-format pragmas to be similar to other mapping files.
> I'd love to if possible, but for some reason clang-format would add extra 
> spaces before and after the slash, ``, which looks 
> ugly and I don't expect it right.
i don't follow, why do you **need** to run clang-format on this file? symbols 
are already ordered


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D142836/new/

https://reviews.llvm.org/D142836

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D143112: [clang] Support parsing comments without ASTContext

[Kadir Cetinkaya via Phabricator via cfe-commits]

kadircet added a comment.

Thanks, this looks like a good start. I left some comments about the details of 
implementation, as for placing and APIs in general, i think it's useful to see 
how things will be built on top inside clangd to make the right call here.




Comment at: clang/include/clang/AST/RawCommentList.h:159
+  /// Parse the \p Comment, storing the result in \p Allocator
+  static comments::FullComment *parse(llvm::StringRef Comment,
+  llvm::BumpPtrAllocator ,

tom-anders wrote:
> Not sure if this is the right place for this? Could be moved to `FullComment` 
> or made a free function instead?
i think this would make more sense inside clangd as a free function when we're 
processing comments. it's nice to get to re-use some Lexer/Sema/Parser creation 
logic, but i don't think it's big enough to justify a new public interfaces 
that's solely used by clangd.
if we can figure out more patterns that we can migrate (the unittest in this 
patch is a good example, but still just a unittest and doesn't benefit much 
from code reuse) we can consider lifting this logic up to a common place. the 
other users should hopefully make it easier to decide where this code should 
live.



Comment at: clang/lib/AST/RawCommentList.cpp:227
+  
SourceMgr.getLocForStartOfFile(SourceMgr.getMainFileID()),
+  Allocator, SourceMgr.getDiagnostics(), Traits);
+}

tom-anders wrote:
> I wasn't sure if it's worth making the `Diagnostics` parameter optional here 
> - Our `SourceMgr` already has it anyway, does it impact performance 
> significantly to use it, even though we don't need it (yet)?
as things stand, Lexer, Sema and Parser expect a non-null diagnosticsengine so 
we have to pass something (i guess that's what you meant by making it optional).

the performance implications comes from two places:
1- extra analysis being performed in the face of malformed comments to produce 
"useful" diagnostics
2- creation cost of diagnostic itself, as all the data about ranges, decls etc. 
are copied around when constructing a diagnostic.

i don't know how costly the first action is today, it's unfortunately the case 
that would require a lot of code changes, if it's happening quite often in 
practice (especially when we're in codebases without proper doxygen comments)
the latter is somewhat easier to address by providing our own "mock" 
diagnostics engine, that'll just drop all of these extra diagnostic data on the 
floor instead of copying around.

i think taking a look at the codebase for the first (searching for `Diag(...) 
<<` patterns in Comment{Lexer,Sema,Parser}.cpp should give some ideas) and 
assessing whether we're performing any extra complicated analysis would be a 
good start. i would expect us not to, and if that's the case i think we can 
leave it at that and don't try to make code changes.
the latter we can address easily and we should.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D143112/new/

https://reviews.llvm.org/D143112

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D142836: [clangd] Add symbol mappings for `std::experimental::filesystem`

[Kadir Cetinkaya via Phabricator via cfe-commits]

kadircet added a comment.

thanks a lot! mostly LG couple more nits




Comment at: clang/lib/Tooling/Inclusions/Stdlib/StdTsSymbolMap.inc:1
+// These symbols are exported from N4100[fs.filesystem.synopsis], the final
+// draft for experimental filesystem. Note that not all of these symbols were

i think comment is a little too verbose. can you just say:
```
These are derived from N4100[fs.filesystem.synopsis], final draft for 
experimental filesystem.
```

There's no need for mentioning that these became the standard in C++17 or being 
a cornerstone for stdlib implementations. As they won't necessarily be true for 
other technical specs nor if we were adding this pre-c++17 standardisation. But 
we'd still like to have these widely adapted symbols included in the mapping to 
make sure we're not generating false negatives.



Comment at: clang/lib/Tooling/Inclusions/Stdlib/StdTsSymbolMap.inc:6
+// whatever.
+// clang-format off
+SYMBOL(absolute, std::experimental::filesystem::, )

can you strip clang-format pragmas to be similar to other mapping files.



Comment at: clang/unittests/Tooling/StandardLibraryTest.cpp:83
+TEST(StdlibTest, Experimental) {
+  EXPECT_TRUE(
+  stdlib::Header::named("", stdlib::Lang::CXX));

another EXPECT_FALSE with `Lang::C` would also be important to make sure we're 
not adding these for C mappings by mistake.



Comment at: clang/unittests/Tooling/StandardLibraryTest.cpp:91
+  EXPECT_EQ(Symbol->name(), "path");
+  EXPECT_EQ(Symbol->qualifiedName(), "std::experimental::filesystem::path");
+

can you also check for `Symbol->headers()`



Comment at: clang/unittests/Tooling/StandardLibraryTest.cpp:93
+
+  // system_complete is replaced by std::filesystem::absolute
+  Symbol = stdlib::Symbol::named("std::filesystem::", "system_complete");

i don't think there's much point in asserting these "meta" details about the 
mapping (same for the test below).


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D142836/new/

https://reviews.llvm.org/D142836

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D143640: [Tooling/Inclusion] Support overload symbols in the stdlib.

[Kadir Cetinkaya via Phabricator via cfe-commits]

kadircet added a comment.

In D143640#4122603 , @hokein wrote:

> Thanks for the comment, putting more thoughts.
>
> In D143640#4121998 , @kadircet 
> wrote:
>
>> All this complexity for handling only 4 symbols feels wrong. is this the 
>> whole set? are there any other symbols that fall under this class? can we 
>> disambiguate all of them solely based on number of parameters?
>
> No, these 4 symbols is not the whole set, there are other symbols (e.g. 
> `isspace` from ``, or ``), but these are not important, IMO.

I agree them being not important. But I think having such a complex 
infrastructure for just 4 symbols and making it more complex in the future when 
we want to handle these just sounds like a big negative.

> The number of parameters is the most critical information to disambiguate -- 
> there are some exceptions, e.g. `std::get()` has a few 1-parameter overloads 
> for array, vector, etc. We will add more information in the `SymbolName` 
> structure for disambiguation in the future.

Makes sense. but maybe there's a way that involves not relying on parameter 
count at all and something completely different. that's another reason why I 
was asking for other symbols that are causing problems today.

>> Downsides:
>>
>> - We are relying heavily on number of parameters in certain cases now
>
> Yeah, I think this is expected if we want to do disambiguations.
>
>> - We only have that information available for symbols with variants
>
> My intention of this information is internal, and merely used for 
> disambiguation. And we're not exposing them to public APIs.

I can see that purpose, but having those symbols available internally without 
any way for users to poke at them sounds like a shortcoming of the design 
that'll never address. Hence I was trying to avoid that.

>> - We're creating a discrepancy in Symbol::all() vs Symbol::named() views of 
>> the world
>
> Oh, this is not intended, we can remove this discrepancy, and make 
> `Symbol::all()` and `Symbol::named()` aligned.

I don't think removing that discrepancy by also dropping these symbols from 
::all is the right trade-off though. As they're still available as a `Symbol` 
through Recognizer. I think the more appropriate thing would require us to 
change `::named` to return multiple symbols, with enough details for 
disambiguation if needed.

>> - In addition to generator and hand curated list, now there's a 3rd place to 
>> augment symbol mapping
>
> Right, this bit is untweaked in this current patch (there is a FIXME), we can 
> merge it with the hand-curated list.
>
>> This complexity both in terms of mental load when introducing new symbols, 
>> and also possible discrepancies that can be encountered when using the 
>> public APIs makes me question if we're doing the sensible thing that'll 
>> generalize for the future variant symbols, or just trying to patch up a 
>> solution for couple cases we know to be not working.
>> I'd suggest:
>> 1- doing a little bit more design work here and at least getting rid of the 
>> burden on the public APIs and a way to introduce these symbols through hand 
>> curated lists (e.g. we should be returning all alternatives in 
>> Symbol::named, with enough details for the caller to have a chance to 
>> disambiguate and have maybe new syntax in raw symbol map files to indicate 
>> this variant situation).
>
> OK, looks like we have some different ideas on the design here. I'm not sure 
> returning all alternatives in `Symbol::named` and exposing all information 
> for disambiguation is a good idea (I'm also not sure it is an important 
> usecase). This seems complicating the public APIs too much.

I think we agree on them being not important so far. But I don't see why having 
a discrepancy between `Symbol`s received through `Recognizer` and 
`Symbol::named` is preferred instead of having a unified view between the two 
(which we already have right now, by excluding variants from both).
I was suggesting preserving this invariant because it's easier to reason about 
the library when we only have a single view rather than 2. Could you elaborate 
a little on why you think "exposing all information for disambiguation" is not 
a good idea? The public interfaces stays ~the same. We'll be having these extra 
information in `Symbol`, if we can find a meaningful set.

> My original design is
>
> - full disambiguation is *only* performed in the `Recognizer` API where the 
> input is the AST node
> - other qualified-name-based APIs `Symbol::named` remains the same (or best 
> effort returning the symbol with most common header)
>
> I share your concern about the current approach, it is not perfect, but I 
> think this is moving towards a right direction, the tradeoff seems OK to me 
> (rather than putting the special-case handling to *every* application, we 
> isolate it in one place).

I agree that sharing this logic with 

[PATCH] D143436: [clangd] Apply standard adaptors to CDBs pushed from LSP

[Kadir Cetinkaya via Phabricator via cfe-commits]

kadircet added a comment.

> There is a discrepancy between how clangd processes CDB loaded from JSON file 
> on disk and pushed via LSP.

That's actually because we model compile commands pushed via LSP as a "fixed 
compilation database" rather than a json compilation database (you can check 
the code in `parseFixed`, near `parseJson`).
The reason behind the discrepancy between fixed and json compilation databases 
mostly arises from the fact that the former is written by people specifically 
to be used by clang-tooling, whereas the latter is usually provided by build 
system and doesn't necessarily have the same concerns as clang-tooling hence 
requires certain modifications.

That being said, the two particular transformations (response files & 
target/mode inference) seem like outliers when it comes to such 
transformations. These are handled by clang-driver binary, outside of the 
driver library, in 
https://github.com/llvm/llvm-project/blob/main/clang/tools/driver/driver.cpp#L426
 and 
https://github.com/llvm/llvm-project/blob/main/clang/tools/driver/driver.cpp#L534.
 Therefore all the other tools that wants to make use of clang infra tries to 
simulate this logic, at different layers with slight differences (at the bare 
minimum, as the logic isn't shared and people mostly care about the driver, 
logic in other places just gets out-of-date).
I believe the right thing to do here is putting all that arg preparation logic 
into driver library itself, so that all code paths that tries to create a 
compiler invocation can have the same behaviour. Unfortunately this is likely 
too hard to pull at this stage, due to people relying on almost every 
implementation detail of these interfaces.
So a middle ground would probably involve, moving that logic inside driver 
binary to a common library, so that future users can at least directly use it 
rather than trying to come up with their own interpretation (or trying to chose 
from existing N patterns) and we'll get rid of the issues that result from 
logic in this place and its duplicates getting out-of-sync. This way we don't 
need to migrate all the applications that want to create a compiler invocation 
to a new API and can only migrate clangd (CommandMangler is I believe the right 
layer for these purposes. as it handles all "string"-like modifications on the 
compile flags today).

Does that make sense? Is it something you'd like to propose a patch for? 
Because as things stand I think we're just making the matter worse here by 
introducing some new code paths that're trying to emulate the logic in driver 
today and will get out of sync at some point.




Comment at: clang-tools-extra/clangd/ClangdLSPServer.cpp:1350
+->getCompileCommands(File)[0]);
 if (Old != New) {
   CDB->setCompileCommand(File, std::move(New));

it looks like this comparison is no longer valid.

`OverlayCDB::getCompileCommand` will apply more modifications to the stored 
flags than just expanding response files and inferring targets. I believe the 
intent of this interaction was to treat compile flags from the LSP client as-is 
without any modifications (as I explained more in my main comment).

No action needed right now, just thinking out-loud. I think the proper thing to 
do here is apply these "common driver transformations" here, and store/use the 
flags as is going forward inside clangd, without extra mangling.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D143436/new/

https://reviews.llvm.org/D143436

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D143260: [clangd] Add semantic token for labels

[Kadir Cetinkaya via Phabricator via cfe-commits]

kadircet added a comment.

thanks for the patch, but could you elaborate a little bit on "why this is 
useful for clients".
in theory semantic highlighting tries to provide annotations for tokens that 
are hard to disambiguate by a syntax highlighter due to need for context.
at hindsight i can't see why `goto X;` and `X:` is not enough for clients to 
implement this without any need for semantic analysis. are there contexts where 
this kind of syntactical match is not enough?
moreover there are other label-like constructs that we're not handling, e.g. 
access specifiers and switch cases. any particular reason for not handling them 
as part of "label" highlights if we were to handle label-decls (the argument 
above applies to this case too though, I think these can be inferred without 
any semantic analysis)?


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D143260/new/

https://reviews.llvm.org/D143260

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D142836: [clangd] Add symbol mappings for `std::experimental::filesystem`

[Kadir Cetinkaya via Phabricator via cfe-commits]

kadircet added a comment.

Thanks for updating this, as I mentioned in 
https://reviews.llvm.org/D143319#4115186, we should put these symbols into 
their own symbol map. ATM `StdSymbolMap.inc` is still generated by an automated 
tool and shouldn't be modified by hand.
So can you rather put these symbols into 
`llvm/llvm-project/clang/lib/Tooling/Inclusions/Stdlib/StdTsSymbolMap.inc` ? 
you also need to rebase your branch, we've moved these mappings from `include` 
directory to `lib`. they're implementation details now, and not public 
interfaces.
After putting it into a new file, you'll also need to include it in 
https://github.com/llvm/llvm-project/blob/main/clang/lib/Tooling/Inclusions/Stdlib/StandardLibrary.cpp#L65.

As for tests, rather than using clangd, can you introduce tests into 
StandardLibrary instead in 
https://github.com/llvm/llvm-project/blob/main/clang/unittests/Tooling/StandardLibraryTest.cpp
 ?


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D142836/new/

https://reviews.llvm.org/D142836

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D143096: [clangd] Provide patched diagnostics with preamble patch

[Kadir Cetinkaya via Phabricator via cfe-commits]

kadircet updated this revision to Diff 496907.
kadircet edited the summary of this revision.
kadircet added a comment.

- Update commit message


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D143096/new/

https://reviews.llvm.org/D143096

Files:
  clang-tools-extra/clangd/ParsedAST.cpp
  clang-tools-extra/clangd/Preamble.cpp
  clang-tools-extra/clangd/Preamble.h
  clang-tools-extra/clangd/unittests/PreambleTests.cpp

Index: clang-tools-extra/clangd/unittests/PreambleTests.cpp
===
--- clang-tools-extra/clangd/unittests/PreambleTests.cpp
+++ clang-tools-extra/clangd/unittests/PreambleTests.cpp
@@ -724,9 +724,8 @@
 #define BAR
 #include [[]])");
 auto AST = createPatchedAST(Code.code(), NewCode.code());
-// FIXME: We should point at the correct coordinates in NewCode.
 EXPECT_THAT(*AST->getDiagnostics(),
-ElementsAre(Diag(Code.range(), "pp_file_not_found")));
+ElementsAre(Diag(NewCode.range(), "pp_file_not_found")));
   }
   {
 // Check with removals from preamble.
@@ -735,18 +734,15 @@
 #include [[]])");
 Annotations NewCode("#include [[]]");
 auto AST = createPatchedAST(Code.code(), NewCode.code());
-// FIXME: We should point at the correct coordinates in NewCode.
 EXPECT_THAT(*AST->getDiagnostics(),
-ElementsAre(Diag(Code.range(), "pp_file_not_found")));
+ElementsAre(Diag(NewCode.range(), "pp_file_not_found")));
   }
   {
 // Drop line with diags.
 Annotations Code("#include [[]]");
 Annotations NewCode("#define BAR\n#define BAZ\n");
 auto AST = createPatchedAST(Code.code(), NewCode.code());
-// FIXME: No diagnostics.
-EXPECT_THAT(*AST->getDiagnostics(),
-ElementsAre(Diag(Code.range(), "pp_file_not_found")));
+EXPECT_THAT(*AST->getDiagnostics(), IsEmpty());
   }
   {
 // Picks closest line in case of multiple alternatives.
@@ -757,18 +753,79 @@
 #define BAR
 #include )");
 auto AST = createPatchedAST(Code.code(), NewCode.code());
-// FIXME: We should point at the correct coordinates in NewCode.
 EXPECT_THAT(*AST->getDiagnostics(),
-ElementsAre(Diag(Code.range(), "pp_file_not_found")));
+ElementsAre(Diag(NewCode.range(), "pp_file_not_found")));
   }
   {
 // Drop diag if line spelling has changed.
 Annotations Code("#include [[]]");
 Annotations NewCode(" # include ");
 auto AST = createPatchedAST(Code.code(), NewCode.code());
-// FIXME: No diags.
+EXPECT_THAT(*AST->getDiagnostics(), IsEmpty());
+  }
+  {
+// Multiple lines.
+Annotations Code(R"(
+#define BAR
+#include [[]])");
+Annotations NewCode(R"(#include [[]])");
+auto AST = createPatchedAST(Code.code(), NewCode.code());
 EXPECT_THAT(*AST->getDiagnostics(),
-ElementsAre(Diag(Code.range(), "pp_file_not_found")));
+ElementsAre(Diag(NewCode.range(), "pp_file_not_found")));
+  }
+  {
+// Multiple lines with change.
+Annotations Code(R"(
+#define BAR
+#include 
+#include [[]])");
+Annotations NewCode(R"(#include )");
+auto AST = createPatchedAST(Code.code(), NewCode.code());
+EXPECT_THAT(*AST->getDiagnostics(), IsEmpty());
+  }
+  {
+// Preserves notes.
+Annotations Code(R"(
+#define $note[[BAR]] 1
+#define $main[[BAR]] 2)");
+Annotations NewCode(R"(
+#define BAZ 0
+#define $note[[BAR]] 1
+#define BAZ 0
+#define $main[[BAR]] 2)");
+auto AST = createPatchedAST(Code.code(), NewCode.code());
+EXPECT_THAT(
+*AST->getDiagnostics(),
+ElementsAre(AllOf(Diag(NewCode.range("main"), "-Wmacro-redefined"),
+  withNote(Diag(NewCode.range("note"));
+  }
+  {
+// Preserves diag without note.
+Annotations Code(R"(
+#define $note[[BAR]] 1
+#define $main[[BAR]] 2)");
+Annotations NewCode(R"(
+#define $main[[BAR]] 2)");
+auto AST = createPatchedAST(Code.code(), NewCode.code());
+EXPECT_THAT(
+*AST->getDiagnostics(),
+ElementsAre(AllOf(Diag(NewCode.range("main"), "-Wmacro-redefined"),
+  Field(::Notes, IsEmpty();
+  }
+  {
+// Note is also dropped if diag is gone.
+Annotations Code(R"(
+#define $note[[BAR]] 1
+#define $main[[BAR]] 2)");
+Annotations NewCode(R"(
+#define BAZ 0
+#define BAR 1)");
+auto AST = createPatchedAST(Code.code(), NewCode.code());
+EXPECT_THAT(*AST->getDiagnostics(), IsEmpty());
   }
 }
 } // namespace
Index: clang-tools-extra/clangd/Preamble.h
===
--- clang-tools-extra/clangd/Preamble.h
+++ clang-tools-extra/clangd/Preamble.h
@@ -34,6 +34,7 @@
 #include "clang/Frontend/PrecompiledPreamble.h"
 #include "clang/Lex/Lexer.h"
 #include "clang/Tooling/CompilationDatabase.h"
+#include "llvm/ADT/ArrayRef.h"
 #include "llvm/ADT/StringRef.h"
 
 #include 
@@ -157,6 

[PATCH] D143096: [clangd] Provide patched diagnostics with preamble patch

[Kadir Cetinkaya via Phabricator via cfe-commits]

kadircet updated this revision to Diff 496905.
kadircet added a comment.

- Reflow comments


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D143096/new/

https://reviews.llvm.org/D143096

Files:
  clang-tools-extra/clangd/ParsedAST.cpp
  clang-tools-extra/clangd/Preamble.cpp
  clang-tools-extra/clangd/Preamble.h
  clang-tools-extra/clangd/unittests/PreambleTests.cpp

Index: clang-tools-extra/clangd/unittests/PreambleTests.cpp
===
--- clang-tools-extra/clangd/unittests/PreambleTests.cpp
+++ clang-tools-extra/clangd/unittests/PreambleTests.cpp
@@ -724,9 +724,8 @@
 #define BAR
 #include [[]])");
 auto AST = createPatchedAST(Code.code(), NewCode.code());
-// FIXME: We should point at the correct coordinates in NewCode.
 EXPECT_THAT(*AST->getDiagnostics(),
-ElementsAre(Diag(Code.range(), "pp_file_not_found")));
+ElementsAre(Diag(NewCode.range(), "pp_file_not_found")));
   }
   {
 // Check with removals from preamble.
@@ -735,18 +734,15 @@
 #include [[]])");
 Annotations NewCode("#include [[]]");
 auto AST = createPatchedAST(Code.code(), NewCode.code());
-// FIXME: We should point at the correct coordinates in NewCode.
 EXPECT_THAT(*AST->getDiagnostics(),
-ElementsAre(Diag(Code.range(), "pp_file_not_found")));
+ElementsAre(Diag(NewCode.range(), "pp_file_not_found")));
   }
   {
 // Drop line with diags.
 Annotations Code("#include [[]]");
 Annotations NewCode("#define BAR\n#define BAZ\n");
 auto AST = createPatchedAST(Code.code(), NewCode.code());
-// FIXME: No diagnostics.
-EXPECT_THAT(*AST->getDiagnostics(),
-ElementsAre(Diag(Code.range(), "pp_file_not_found")));
+EXPECT_THAT(*AST->getDiagnostics(), IsEmpty());
   }
   {
 // Picks closest line in case of multiple alternatives.
@@ -757,18 +753,79 @@
 #define BAR
 #include )");
 auto AST = createPatchedAST(Code.code(), NewCode.code());
-// FIXME: We should point at the correct coordinates in NewCode.
 EXPECT_THAT(*AST->getDiagnostics(),
-ElementsAre(Diag(Code.range(), "pp_file_not_found")));
+ElementsAre(Diag(NewCode.range(), "pp_file_not_found")));
   }
   {
 // Drop diag if line spelling has changed.
 Annotations Code("#include [[]]");
 Annotations NewCode(" # include ");
 auto AST = createPatchedAST(Code.code(), NewCode.code());
-// FIXME: No diags.
+EXPECT_THAT(*AST->getDiagnostics(), IsEmpty());
+  }
+  {
+// Multiple lines.
+Annotations Code(R"(
+#define BAR
+#include [[]])");
+Annotations NewCode(R"(#include [[]])");
+auto AST = createPatchedAST(Code.code(), NewCode.code());
 EXPECT_THAT(*AST->getDiagnostics(),
-ElementsAre(Diag(Code.range(), "pp_file_not_found")));
+ElementsAre(Diag(NewCode.range(), "pp_file_not_found")));
+  }
+  {
+// Multiple lines with change.
+Annotations Code(R"(
+#define BAR
+#include 
+#include [[]])");
+Annotations NewCode(R"(#include )");
+auto AST = createPatchedAST(Code.code(), NewCode.code());
+EXPECT_THAT(*AST->getDiagnostics(), IsEmpty());
+  }
+  {
+// Preserves notes.
+Annotations Code(R"(
+#define $note[[BAR]] 1
+#define $main[[BAR]] 2)");
+Annotations NewCode(R"(
+#define BAZ 0
+#define $note[[BAR]] 1
+#define BAZ 0
+#define $main[[BAR]] 2)");
+auto AST = createPatchedAST(Code.code(), NewCode.code());
+EXPECT_THAT(
+*AST->getDiagnostics(),
+ElementsAre(AllOf(Diag(NewCode.range("main"), "-Wmacro-redefined"),
+  withNote(Diag(NewCode.range("note"));
+  }
+  {
+// Preserves diag without note.
+Annotations Code(R"(
+#define $note[[BAR]] 1
+#define $main[[BAR]] 2)");
+Annotations NewCode(R"(
+#define $main[[BAR]] 2)");
+auto AST = createPatchedAST(Code.code(), NewCode.code());
+EXPECT_THAT(
+*AST->getDiagnostics(),
+ElementsAre(AllOf(Diag(NewCode.range("main"), "-Wmacro-redefined"),
+  Field(::Notes, IsEmpty();
+  }
+  {
+// Note is also dropped if diag is gone.
+Annotations Code(R"(
+#define $note[[BAR]] 1
+#define $main[[BAR]] 2)");
+Annotations NewCode(R"(
+#define BAZ 0
+#define BAR 1)");
+auto AST = createPatchedAST(Code.code(), NewCode.code());
+EXPECT_THAT(*AST->getDiagnostics(), IsEmpty());
   }
 }
 } // namespace
Index: clang-tools-extra/clangd/Preamble.h
===
--- clang-tools-extra/clangd/Preamble.h
+++ clang-tools-extra/clangd/Preamble.h
@@ -34,6 +34,7 @@
 #include "clang/Frontend/PrecompiledPreamble.h"
 #include "clang/Lex/Lexer.h"
 #include "clang/Tooling/CompilationDatabase.h"
+#include "llvm/ADT/ArrayRef.h"
 #include "llvm/ADT/StringRef.h"
 
 #include 
@@ -157,6 +158,8 @@
   /// Whether diagnostics generated using 

[PATCH] D143096: [clangd] Provide patched diagnostics with preamble patch

[Kadir Cetinkaya via Phabricator via cfe-commits]

kadircet updated this revision to Diff 496900.
kadircet marked 12 inline comments as done.
kadircet added a comment.

- Change patching logic to iterate over diags instead of preamble lines
- Change translation logic to:
  - Preserve a diagnostic if its range can be translated.
  - Preserve all the notes whose range can be translated.
  - Preserve all the fixes whose edit ranges can be translated.
- Drop restrictions on ranges being a single line, make sure contents for all 
the lines are matched between modified and basline contents.
- Add couple new tests to demonstrate what's preserved.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D143096/new/

https://reviews.llvm.org/D143096

Files:
  clang-tools-extra/clangd/ParsedAST.cpp
  clang-tools-extra/clangd/Preamble.cpp
  clang-tools-extra/clangd/Preamble.h
  clang-tools-extra/clangd/unittests/PreambleTests.cpp

Index: clang-tools-extra/clangd/unittests/PreambleTests.cpp
===
--- clang-tools-extra/clangd/unittests/PreambleTests.cpp
+++ clang-tools-extra/clangd/unittests/PreambleTests.cpp
@@ -724,9 +724,8 @@
 #define BAR
 #include [[]])");
 auto AST = createPatchedAST(Code.code(), NewCode.code());
-// FIXME: We should point at the correct coordinates in NewCode.
 EXPECT_THAT(*AST->getDiagnostics(),
-ElementsAre(Diag(Code.range(), "pp_file_not_found")));
+ElementsAre(Diag(NewCode.range(), "pp_file_not_found")));
   }
   {
 // Check with removals from preamble.
@@ -735,18 +734,15 @@
 #include [[]])");
 Annotations NewCode("#include [[]]");
 auto AST = createPatchedAST(Code.code(), NewCode.code());
-// FIXME: We should point at the correct coordinates in NewCode.
 EXPECT_THAT(*AST->getDiagnostics(),
-ElementsAre(Diag(Code.range(), "pp_file_not_found")));
+ElementsAre(Diag(NewCode.range(), "pp_file_not_found")));
   }
   {
 // Drop line with diags.
 Annotations Code("#include [[]]");
 Annotations NewCode("#define BAR\n#define BAZ\n");
 auto AST = createPatchedAST(Code.code(), NewCode.code());
-// FIXME: No diagnostics.
-EXPECT_THAT(*AST->getDiagnostics(),
-ElementsAre(Diag(Code.range(), "pp_file_not_found")));
+EXPECT_THAT(*AST->getDiagnostics(), IsEmpty());
   }
   {
 // Picks closest line in case of multiple alternatives.
@@ -757,18 +753,79 @@
 #define BAR
 #include )");
 auto AST = createPatchedAST(Code.code(), NewCode.code());
-// FIXME: We should point at the correct coordinates in NewCode.
 EXPECT_THAT(*AST->getDiagnostics(),
-ElementsAre(Diag(Code.range(), "pp_file_not_found")));
+ElementsAre(Diag(NewCode.range(), "pp_file_not_found")));
   }
   {
 // Drop diag if line spelling has changed.
 Annotations Code("#include [[]]");
 Annotations NewCode(" # include ");
 auto AST = createPatchedAST(Code.code(), NewCode.code());
-// FIXME: No diags.
+EXPECT_THAT(*AST->getDiagnostics(), IsEmpty());
+  }
+  {
+// Multiple lines.
+Annotations Code(R"(
+#define BAR
+#include [[]])");
+Annotations NewCode(R"(#include [[]])");
+auto AST = createPatchedAST(Code.code(), NewCode.code());
 EXPECT_THAT(*AST->getDiagnostics(),
-ElementsAre(Diag(Code.range(), "pp_file_not_found")));
+ElementsAre(Diag(NewCode.range(), "pp_file_not_found")));
+  }
+  {
+// Multiple lines with change.
+Annotations Code(R"(
+#define BAR
+#include 
+#include [[]])");
+Annotations NewCode(R"(#include )");
+auto AST = createPatchedAST(Code.code(), NewCode.code());
+EXPECT_THAT(*AST->getDiagnostics(), IsEmpty());
+  }
+  {
+// Preserves notes.
+Annotations Code(R"(
+#define $note[[BAR]] 1
+#define $main[[BAR]] 2)");
+Annotations NewCode(R"(
+#define BAZ 0
+#define $note[[BAR]] 1
+#define BAZ 0
+#define $main[[BAR]] 2)");
+auto AST = createPatchedAST(Code.code(), NewCode.code());
+EXPECT_THAT(
+*AST->getDiagnostics(),
+ElementsAre(AllOf(Diag(NewCode.range("main"), "-Wmacro-redefined"),
+  withNote(Diag(NewCode.range("note"));
+  }
+  {
+// Preserves diag without note.
+Annotations Code(R"(
+#define $note[[BAR]] 1
+#define $main[[BAR]] 2)");
+Annotations NewCode(R"(
+#define $main[[BAR]] 2)");
+auto AST = createPatchedAST(Code.code(), NewCode.code());
+EXPECT_THAT(
+*AST->getDiagnostics(),
+ElementsAre(AllOf(Diag(NewCode.range("main"), "-Wmacro-redefined"),
+  Field(::Notes, IsEmpty();
+  }
+  {
+// Note is also dropped if diag is gone.
+Annotations Code(R"(
+#define $note[[BAR]] 1
+#define $main[[BAR]] 2)");
+Annotations NewCode(R"(
+#define BAZ 0
+#define BAR 1)");
+auto AST = createPatchedAST(Code.code(), NewCode.code());
+EXPECT_THAT(*AST->getDiagnostics(), 

[PATCH] D143597: [clangd] Drop includes from disabled PP regions in preamble patch

[Kadir Cetinkaya via Phabricator via cfe-commits]

This revision was automatically updated to reflect the committed changes.
Closed by commit rG19659b5f0dd1: [clangd] Drop includes from disabled PP 
regions in preamble patch (authored by kadircet).

Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D143597/new/

https://reviews.llvm.org/D143597

Files:
  clang-tools-extra/clangd/Preamble.cpp
  clang-tools-extra/clangd/unittests/PreambleTests.cpp


Index: clang-tools-extra/clangd/unittests/PreambleTests.cpp
===
--- clang-tools-extra/clangd/unittests/PreambleTests.cpp
+++ clang-tools-extra/clangd/unittests/PreambleTests.cpp
@@ -170,6 +170,9 @@
   auto TU = TestTU::withCode(R"cpp(
 #include "a.h" // IWYU pragma: keep
 #include "c.h"
+#ifdef FOO
+#include "d.h"
+#endif
   )cpp");
   TU.AdditionalFiles["a.h"] = "#include \"b.h\"";
   TU.AdditionalFiles["b.h"] = "";
@@ -178,10 +181,14 @@
   auto BaselinePreamble = buildPreamble(
   TU.Filename, *buildCompilerInvocation(PI, Diags), PI, true, nullptr);
   // We drop c.h from modified and add a new header. Since the latter is 
patched
-  // we should only get a.h in preamble includes.
+  // we should only get a.h in preamble includes. d.h shouldn't be part of the
+  // preamble, as it's coming from a disabled region.
   TU.Code = R"cpp(
 #include "a.h"
 #include "b.h"
+#ifdef FOO
+#include "d.h"
+#endif
   )cpp";
   auto PP = PreamblePatch::createFullPatch(testPath(TU.Filename), 
TU.inputs(FS),
*BaselinePreamble);
Index: clang-tools-extra/clangd/Preamble.cpp
===
--- clang-tools-extra/clangd/Preamble.cpp
+++ clang-tools-extra/clangd/Preamble.cpp
@@ -685,13 +685,16 @@
   // Include already present in the baseline preamble. Set resolved path 
and
   // put into preamble includes.
   if (It != ExistingIncludes.end()) {
-auto  = PP.PreambleIncludes.emplace_back();
-// Copy everything from existing include, apart from the location, when
-// it's coming from baseline preamble.
-if (It->second)
+if (It->second) {
+  // If this header is included in an active region of the baseline
+  // preamble, preserve it.
+  auto  = PP.PreambleIncludes.emplace_back();
+  // Copy everything from existing include, apart from the location,
+  // when it's coming from baseline preamble.
   PatchedInc = *It->second;
-PatchedInc.HashLine = Inc.HashLine;
-PatchedInc.HashOffset = Inc.HashOffset;
+  PatchedInc.HashLine = Inc.HashLine;
+  PatchedInc.HashOffset = Inc.HashOffset;
+}
 continue;
   }
   // Include is new in the modified preamble. Inject it into the patch and


Index: clang-tools-extra/clangd/unittests/PreambleTests.cpp
===
--- clang-tools-extra/clangd/unittests/PreambleTests.cpp
+++ clang-tools-extra/clangd/unittests/PreambleTests.cpp
@@ -170,6 +170,9 @@
   auto TU = TestTU::withCode(R"cpp(
 #include "a.h" // IWYU pragma: keep
 #include "c.h"
+#ifdef FOO
+#include "d.h"
+#endif
   )cpp");
   TU.AdditionalFiles["a.h"] = "#include \"b.h\"";
   TU.AdditionalFiles["b.h"] = "";
@@ -178,10 +181,14 @@
   auto BaselinePreamble = buildPreamble(
   TU.Filename, *buildCompilerInvocation(PI, Diags), PI, true, nullptr);
   // We drop c.h from modified and add a new header. Since the latter is patched
-  // we should only get a.h in preamble includes.
+  // we should only get a.h in preamble includes. d.h shouldn't be part of the
+  // preamble, as it's coming from a disabled region.
   TU.Code = R"cpp(
 #include "a.h"
 #include "b.h"
+#ifdef FOO
+#include "d.h"
+#endif
   )cpp";
   auto PP = PreamblePatch::createFullPatch(testPath(TU.Filename), TU.inputs(FS),
*BaselinePreamble);
Index: clang-tools-extra/clangd/Preamble.cpp
===
--- clang-tools-extra/clangd/Preamble.cpp
+++ clang-tools-extra/clangd/Preamble.cpp
@@ -685,13 +685,16 @@
   // Include already present in the baseline preamble. Set resolved path and
   // put into preamble includes.
   if (It != ExistingIncludes.end()) {
-auto  = PP.PreambleIncludes.emplace_back();
-// Copy everything from existing include, apart from the location, when
-// it's coming from baseline preamble.
-if (It->second)
+if (It->second) {
+  // If this header is included in an active region of the baseline
+  // preamble, preserve it.
+  auto  = PP.PreambleIncludes.emplace_back();
+  // Copy everything from existing include, apart from the location,
+  // when it's coming from baseline preamble.
   

[PATCH] D143197: [clangd] Fix bugs in main-file include patching for stale preambles

[Kadir Cetinkaya via Phabricator via cfe-commits]

This revision was automatically updated to reflect the committed changes.
Closed by commit rGfae01d175a29: [clangd] Fix bugs in main-file include 
patching for stale preambles (authored by kadircet).

Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D143197/new/

https://reviews.llvm.org/D143197

Files:
  clang-tools-extra/clangd/Preamble.cpp
  clang-tools-extra/clangd/unittests/PreambleTests.cpp

Index: clang-tools-extra/clangd/unittests/PreambleTests.cpp
===
--- clang-tools-extra/clangd/unittests/PreambleTests.cpp
+++ clang-tools-extra/clangd/unittests/PreambleTests.cpp
@@ -15,6 +15,7 @@
 #include "TestFS.h"
 #include "TestTU.h"
 #include "XRefs.h"
+#include "clang/Basic/SourceManager.h"
 #include "clang/Format/Format.h"
 #include "clang/Frontend/FrontendActions.h"
 #include "clang/Frontend/PrecompiledPreamble.h"
@@ -23,6 +24,7 @@
 #include "llvm/Support/MemoryBuffer.h"
 #include "llvm/Support/VirtualFileSystem.h"
 #include "gmock/gmock.h"
+#include "gtest/gtest-matchers.h"
 #include "gtest/gtest.h"
 #include 
 #include 
@@ -166,7 +168,7 @@
   MockFS FS;
   IgnoreDiagnostics Diags;
   auto TU = TestTU::withCode(R"cpp(
-#include "a.h"
+#include "a.h" // IWYU pragma: keep
 #include "c.h"
   )cpp");
   TU.AdditionalFiles["a.h"] = "#include \"b.h\"";
@@ -185,9 +187,14 @@
*BaselinePreamble);
   // Only a.h should exists in the preamble, as c.h has been dropped and b.h was
   // newly introduced.
-  EXPECT_THAT(PP.preambleIncludes(),
-  ElementsAre(AllOf(Field(::Written, "\"a.h\""),
-Field(::Resolved, testPath("a.h");
+  EXPECT_THAT(
+  PP.preambleIncludes(),
+  ElementsAre(AllOf(
+  Field(::Written, "\"a.h\""),
+  Field(::Resolved, testPath("a.h")),
+  Field(::HeaderID, testing::Not(testing::Eq(std::nullopt))),
+  Field(::BehindPragmaKeep, true),
+  Field(::FileKind, SrcMgr::CharacteristicKind::C_User;
 }
 
 std::optional createPatchedAST(llvm::StringRef Baseline,
@@ -225,6 +232,26 @@
   .str();
 }
 
+TEST(PreamblePatchTest, IncludesArePreserved) {
+  llvm::StringLiteral Baseline = R"(//error-ok
+#include 
+#include 
+)";
+  llvm::StringLiteral Modified = R"(//error-ok
+#include 
+#include 
+#define FOO)";
+
+  auto Includes = createPatchedAST(Baseline, Modified.str())
+  ->getIncludeStructure()
+  .MainFileIncludes;
+  EXPECT_TRUE(!Includes.empty());
+  EXPECT_EQ(Includes, TestTU::withCode(Baseline)
+  .build()
+  .getIncludeStructure()
+  .MainFileIncludes);
+}
+
 TEST(PreamblePatchTest, Define) {
   // BAR should be defined while parsing the AST.
   struct {
Index: clang-tools-extra/clangd/Preamble.cpp
===
--- clang-tools-extra/clangd/Preamble.cpp
+++ clang-tools-extra/clangd/Preamble.cpp
@@ -671,10 +671,10 @@
 // We are only interested in newly added includes, record the ones in
 // Baseline for exclusion.
 llvm::DenseMap,
-   /*Resolved=*/llvm::StringRef>
+   const Inclusion *>
 ExistingIncludes;
 for (const auto  : Baseline.Includes.MainFileIncludes)
-  ExistingIncludes[{Inc.Directive, Inc.Written}] = Inc.Resolved;
+  ExistingIncludes[{Inc.Directive, Inc.Written}] = 
 // There might be includes coming from disabled regions, record these for
 // exclusion too. note that we don't have resolved paths for those.
 for (const auto  : BaselineScan->Includes)
@@ -685,8 +685,13 @@
   // Include already present in the baseline preamble. Set resolved path and
   // put into preamble includes.
   if (It != ExistingIncludes.end()) {
-Inc.Resolved = It->second.str();
-PP.PreambleIncludes.push_back(Inc);
+auto  = PP.PreambleIncludes.emplace_back();
+// Copy everything from existing include, apart from the location, when
+// it's coming from baseline preamble.
+if (It->second)
+  PatchedInc = *It->second;
+PatchedInc.HashLine = Inc.HashLine;
+PatchedInc.HashOffset = Inc.HashOffset;
 continue;
   }
   // Include is new in the modified preamble. Inject it into the patch and
@@ -696,6 +701,11 @@
   Patch << llvm::formatv(
   "#{0} {1}\n", spellingForIncDirective(Inc.Directive), Inc.Written);
 }
+  } else {
+// Make sure we have the full set of includes available even when we're not
+// patching. As these are used by features we provide afterwards like hover,
+// go-to-def or include-cleaner when preamble is stale.
+PP.PreambleIncludes = Baseline.Includes.MainFileIncludes;
   }
 
   if (DirectivesChanged) {
___
cfe-commits mailing list

[PATCH] D143640: [Tooling/Inclusion] Support overload symbols in the stdlib.

[Kadir Cetinkaya via Phabricator via cfe-commits]

kadircet added a comment.

All this complexity for handling only 4 symbols feels wrong. is this the whole 
set? are there any other symbols that fall under this class? can we 
disambiguate all of them solely based on number of parameters?
Downsides:

- We are relying heavily on number of parameters in certain cases now
- We only have that information available for symbols with variants
- We're creating a discrepancy in Symbol::all() vs Symbol::named() views of the 
world
- In addition to generator and hand curated list, now there's a 3rd place to 
augment symbol mapping

This complexity both in terms of mental load when introducing new symbols, and 
also possible discrepancies that can be encountered when using the public APIs 
makes me question if we're doing the sensible thing that'll generalize for the 
future variant symbols, or just trying to patch up a solution for couple cases 
we know to be not working.
I'd suggest:
1- doing a little bit more design work here and at least getting rid of the 
burden on the public APIs and a way to introduce these symbols through hand 
curated lists (e.g. we should be returning all alternatives in Symbol::named, 
with enough details for the caller to have a chance to disambiguate and have 
maybe new syntax in raw symbol map files to indicate this variant situation).
2- or handling this in include-cleaner only, similar to how we do it in clangd 
and other applications, until we encounter more problematic symbols. as 
handling this in the general case seems hard (especially in the absence of an 
AST), and we haven't seen enough signals for symbols other than `std::move` to 
cause trouble.

i feel like 2nd alternative gets us a long way with minimal effort. WDYT?




Comment at: clang/lib/Tooling/Inclusions/Stdlib/StandardLibrary.cpp:131
[](const SymbolHeaderMapping::SymbolName ) {
  return S.qualifiedName() == QName;
}) &&

i think we'd now trigger this assert if a variant is provided by multiple 
headers


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D143640/new/

https://reviews.llvm.org/D143640

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D143509: Move the BySpelling map to IncludeStructure.

[Kadir Cetinkaya via Phabricator via cfe-commits]

kadircet added a comment.

oh forgot to mention, could you also please add some tests into 
llvm-project/clang-tools-extra/clangd/unittests/HeadersTests.cpp ?


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D143509/new/

https://reviews.llvm.org/D143509

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D143509: Move the BySpelling map to IncludeStructure.

[Kadir Cetinkaya via Phabricator via cfe-commits]

kadircet added inline comments.



Comment at: clang-tools-extra/clangd/Headers.cpp:76
+Out->MainFileIncludesBySpelling.try_emplace(Inc.Written)
+.first->second.push_back(static_cast(*Inc.HeaderID));
   }

right now we're only storing "resolved" includes from the main file and not 
all, this is creating a discrepancy between the view one gets through 
`MainFileIncludes` and through this map.
in addition to that only storing `HeaderID` gets the job done for 
IncludeCleaner, but won't really help anyone that wants to match main file 
includes apart from that (there's no easy way to go from a `HeaderID` to an 
`Inclusion`).

so instead of storing the `HeaderID` in the map values, we can actually store 
indexes into `MainFileIncludes`. later on during the lookup, we can build a 
`SmallVector` that contains pointers to the relevant includes. 
WDYT?



Comment at: clang-tools-extra/clangd/Headers.h:166
+  llvm::SmallVector
+  lookupHeaderIDsBySpelling(llvm::StringRef Spelling) const {
+return MainFileIncludesBySpelling.lookup(Spelling);

what about renaming `lookupHeaderIDsBySpelling` to 
`mainFileIncludesWithSpelling`. with a comment explaining `Returns includes 
inside the main file with the given Spelling. Spelling should include brackets 
or quotes, e.g. `.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D143509/new/

https://reviews.llvm.org/D143509

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D143638: [clangd] Move function body to out-of-line: unnamed class method incorrect moving

[Kadir Cetinkaya via Phabricator via cfe-commits]

kadircet accepted this revision.
kadircet added inline comments.
This revision is now accepted and ready to land.



Comment at: clang-tools-extra/clangd/unittests/tweaks/DefineOutlineTests.cpp:89
+  // Not available on methods of unnamed classes.
+  EXPECT_UNAVAILABLE(R"cpp(
+struct Foo {

nit: i think you can group all these 3 tests.

```
// Not available if any of the parent context is unnamed, as we can't spell it.
EXPECT_UNAVAILABLE(R"cpp(
struct Foo {
  struct { 
 void b^ar() {}
 struct Baz { void b^ar() {} };
  } Bar;
};
namespace {
  struct Foo { void f^oo() {} };
}
)cpp");
```


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D143638/new/

https://reviews.llvm.org/D143638

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D143638: [clangd] Move function body to out-of-line: unnamed class method incorrect moving

[Kadir Cetinkaya via Phabricator via cfe-commits]

kadircet added inline comments.



Comment at: clang-tools-extra/clangd/refactor/tweaks/DefineOutline.cpp:395
 
 // Bail out in templated classes, as it is hard to spell the class name, 
i.e
 // if the template parameter is unnamed.

could you move this comment closer to the `isTempated` if statement below?



Comment at: clang-tools-extra/clangd/refactor/tweaks/DefineOutline.cpp:401
+
+  // The refactoring is meaningless for unnamed classes.
+  const auto *Parent = MD->getParent();

not just classes, but also for cases with unnamed namespaces.

could you change this to look like:
```
for(DeclContext *DC = MD->getParent(); DC; DC = DC->getParent()) {
  if (auto *ND = llvm::dyn_cast(DC)) {
 if(ND->getDeclName().isEmpty())
return false;
  }
}
```



Comment at: clang-tools-extra/clangd/unittests/tweaks/DefineOutlineTests.cpp:97
+  // nesting.
+  EXPECT_UNAVAILABLE(R"cpp(
+struct Foo {

can you also have a test inside an unnamed namespace?


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D143638/new/

https://reviews.llvm.org/D143638

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D143214: [include-mapping] Add C-compatibility symbol entries.

[Kadir Cetinkaya via Phabricator via cfe-commits]

kadircet accepted this revision.
kadircet added a comment.

thanks, lgtm again


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D143214/new/

https://reviews.llvm.org/D143214

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D143319: [clangd] Support iwyu pragma: no_include

[Kadir Cetinkaya via Phabricator via cfe-commits]

kadircet added a comment.

thanks a lot for the patch!

we're migrating IWYU related functionality in clangd to make use of 
include-cleaner library nowadays. so can you actually move the IWYU no_include 
pragma handling logic into 
https://github.com/llvm/llvm-project/blob/main/clang-tools-extra/include-cleaner/lib/Record.cpp?
This won't have affect on clangd regarding the particular problem you're facing 
immediately, but we're going to merge the logic inside include-cleaner and 
clangd soon and such discrepancies should disappear then.

in the meanwhile regarding the particular problem you're facing for 
experimental symbols. sorry for shutting down D142836 
, but that actually made us re-evaluate the 
approach of hand-curated lists and we've decided to introduce such mappings. so 
if you can revamp D142836  and put 
`TsStdSymbolMap.inc` in 
https://github.com/llvm/llvm-project/tree/main/clang/lib/Tooling/Inclusions/Stdlib/StdTsSymbolMap.inc
 we can land it and make sure it work as intended inside clangd, even before we 
merge the IWYU pragma handling logic. Happy to do that myself if you don't feel 
like doing it, sorry for creating some confusion here :(


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D143319/new/

https://reviews.llvm.org/D143319

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D143214: [include-mapping] Add C-compatibility symbol entries.

[Kadir Cetinkaya via Phabricator via cfe-commits]

kadircet added inline comments.



Comment at: clang/lib/Tooling/Inclusions/Stdlib/StandardLibrary.cpp:218
+  if (!D) // global scope.
+return getMappingPerLang(L)->NamespaceSymbols->lookup("");
   auto It = NamespaceCache.find(D);

oh i actually missed the behaviour change here in the previous version of the 
code and it seems very subtle.
can we change this function to take in a `DeclContext*` instead and terminate 
traversal inside the callsite at `TUDecl` and only treat `TUDecl` as global 
scope here?


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D143214/new/

https://reviews.llvm.org/D143214

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D143496: [clangd] Add support for missing includes analysis.

[Kadir Cetinkaya via Phabricator via cfe-commits]

kadircet added inline comments.



Comment at: clang-tools-extra/clangd/Config.h:91
 
+  enum class MissingIncludesPolicy {
+/// Diagnose missing includes.

rather than duplicating, what about renaming `UnusedIncludesPolicy` to 
`IncludesPolicy` and use it for both `UnusedIncludes` and `MissingIncludes` 
options below?



Comment at: clang-tools-extra/clangd/ConfigFragment.h:224
 
-/// Controls how clangd will correct "unnecessary #include directives.
+/// Controls how clangd will correct "unnecessary" #include directives.
 /// clangd can warn if a header is `#include`d but not used, and suggest

nit: i'd rather drop the quotes completely to match your description of 
MissingIncludes below.



Comment at: clang-tools-extra/clangd/ConfigFragment.h:238
 
+/// Controls how clangd handles missing #include directives.
+/// clangd can warn if a header for a symbol is not `#include`d (missing),





Comment at: clang-tools-extra/clangd/ConfigFragment.h:239-240
+/// Controls how clangd handles missing #include directives.
+/// clangd can warn if a header for a symbol is not `#include`d (missing),
+/// and suggest adding it.
+///





Comment at: clang-tools-extra/clangd/ConfigFragment.h:242
+///
+/// Strict means a header is missing if it is not *directly #include'd.
+/// The file might still compile if the header is included transitively.





Comment at: clang-tools-extra/clangd/IncludeCleaner.cpp:516
+
+  std::vector Macros =
+  collectMacroReferences(AST);





Comment at: clang-tools-extra/clangd/IncludeCleaner.cpp:547
+convertIncludes(const SourceManager ,
+std::vector MainFileIncludes) {
+  include_cleaner::Includes Includes;

you can just pass an `llvm::ArrayRef` to prevent a copy



Comment at: clang-tools-extra/clangd/IncludeCleaner.cpp:550
+  for (const Inclusion  : MainFileIncludes) {
+llvm::ErrorOr ResolvedOrError =
+SM.getFileManager().getFile(Inc.Resolved);

you can re-write this as:
```
include_cleaner::Include TransformedInc;
TransformedInc.Spelled = Inc.Written.trim("\"<>");
TransformedInc.HashLocation = SM.getComposedLoc(SM.getMainFileID(), 
Inc.HashOffset); // we should actually convert this from a SourceLocation to 
offset in include_cleaner::Include as well
TransformedInc.Line = Inc.HashLine;
TransformedInc.Angled = WrittenRef.starts_with("<");
if(auto FE = SM.getFileManager().getFile(Inc.Resolved))
  TransformedInc.Resolved = *FE;
Includes.add(std::move(TransformedInc));
```



Comment at: clang-tools-extra/clangd/IncludeCleaner.cpp:570
+computeMissingIncludes(ParsedAST ) {
+  std::vector Macros =
+  collectMacroReferences(AST);

nit: `auto Macros = ..`



Comment at: clang-tools-extra/clangd/IncludeCleaner.cpp:572
+  collectMacroReferences(AST);
+  std::vector MainFileIncludes =
+  AST.getIncludeStructure().MainFileIncludes;

no need for copying to vector here, `const auto& MainFileIncludes = ...`



Comment at: clang-tools-extra/clangd/IncludeCleaner.cpp:577
+  convertIncludes(AST.getSourceManager(), MainFileIncludes);
+  std::string FileName =
+  AST.getSourceManager()

you can use `AST.tuPath()`



Comment at: clang-tools-extra/clangd/IncludeCleaner.cpp:582
+  .str();
+  if (FileName.find("foo") != std::string::npos) {
+vlog("Include cleaner includes: {0}", IncludeCleanerIncludes.all().size());

looks like debugging artifact?



Comment at: clang-tools-extra/clangd/IncludeCleaner.cpp:593
+  llvm::ArrayRef Providers) {
+bool Satisfied = false;
+for (const include_cleaner::Header  : Providers) {

nit: you can check whether `Ref.RT` is `Explicit` at the top, and bail out 
early.



Comment at: clang-tools-extra/clangd/IncludeCleaner.cpp:597
+  H.physical() == MainFile) {
+Satisfied = true;
+  }

nit: you can just `break` after satisfying the include (same below)



Comment at: clang-tools-extra/clangd/IncludeCleaner.cpp:605
+Ref.RT == include_cleaner::RefType::Explicit) {
+  std::string SpelledHeader = include_cleaner::spellHeader(
+  Providers.front(), AST.getPreprocessor().getHeaderSearchInfo(),

clangd has some header spelling customizations. so we should actually be doing 
this through 
`URI::includeSpelling(URI::create(getCanonicalPath(Providers.front().physical(),
 SM)))` first, and fall back to `spellHeader` if it fails for physical header 
providers to make sure we're consistent.

this is used by $EMPLOYER$'s