[PATCH] D85750: [SyntaxTree] Unbox operators into tokens for nodes generated from `CXXOperatorCallExpr`

[Eduardo Caldas via Phabricator via cfe-commits]

eduucaldas added inline comments.



Comment at: clang/lib/Tooling/Syntax/BuildTree.cpp:1024
+  // representation of built-in and user-defined operators.
+  if (child->getBeginLoc() == S->getOperatorLoc())
+continue;

gribozavr2 wrote:
> eduucaldas wrote:
> > Here we want to check if this child is just a DeclRefExpr to an operator - 
> > as opposed to an operand.
> > Is comparing SourceLocation generally safe?
> > Is there a better way of figuring out if the child DeclRefExpr refers to an 
> > operator?
> Comparing SourceLocations should work, but you're right that it is a 
> questionable pattern.
> 
> Iterating only over `S->arguments()` instead of all children could work here.
Very good tip!


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D85750/new/

https://reviews.llvm.org/D85750

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D85750: [SyntaxTree] Unbox operators into tokens for nodes generated from `CXXOperatorCallExpr`

[Eduardo Caldas via Phabricator via cfe-commits]

eduucaldas updated this revision to Diff 284988.
eduucaldas marked 3 inline comments as done.
eduucaldas added a comment.

use arguments instead of children


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D85750/new/

https://reviews.llvm.org/D85750

Files:
  clang/lib/Tooling/Syntax/BuildTree.cpp
  clang/unittests/Tooling/Syntax/TreeTest.cpp

Index: clang/unittests/Tooling/Syntax/TreeTest.cpp
===
--- clang/unittests/Tooling/Syntax/TreeTest.cpp
+++ clang/unittests/Tooling/Syntax/TreeTest.cpp
@@ -2592,9 +2592,7 @@
 | | |-IdExpression
 | | | `-UnqualifiedId
 | | |   `-x
-| | |-IdExpression
-| | | `-UnqualifiedId
-| | |   `-=
+| | |-=
 | | `-IdExpression
 | |   `-UnqualifiedId
 | | `-y
@@ -2605,9 +2603,7 @@
 | | | `-IdExpression
 | | |   `-UnqualifiedId
 | | | `-x
-| | |-IdExpression
-| | | `-UnqualifiedId
-| | |   `-+
+| | |-+
 | | `-IdExpression
 | |   `-UnqualifiedId
 | | `-y
@@ -2617,9 +2613,7 @@
 | | |-IdExpression
 | | | `-UnqualifiedId
 | | |   `-x
-| | |-IdExpression
-| | | `-UnqualifiedId
-| | |   `-<
+| | |-<
 | | `-IdExpression
 | |   `-UnqualifiedId
 | | `-y
@@ -2629,9 +2623,7 @@
 | | |-IdExpression
 | | | `-UnqualifiedId
 | | |   `-x
-| | |-IdExpression
-| | | `-UnqualifiedId
-| | |   `-<<
+| | |-<<
 | | `-IdExpression
 | |   `-UnqualifiedId
 | | `-y
@@ -2641,9 +2633,7 @@
 | | |-IdExpression
 | | | `-UnqualifiedId
 | | |   `-x
-| | |-IdExpression
-| | | `-UnqualifiedId
-| | |   `-,
+| | |-,
 | | `-IdExpression
 | |   `-UnqualifiedId
 | | `-y
@@ -2730,27 +2720,21 @@
 |-{
 |-ExpressionStatement
 | |-PrefixUnaryOperatorExpression
-| | |-IdExpression
-| | | `-UnqualifiedId
-| | |   `-++
+| | |-++
 | | `-IdExpression
 | |   `-UnqualifiedId
 | | `-x
 | `-;
 |-ExpressionStatement
 | |-PrefixUnaryOperatorExpression
-| | |-IdExpression
-| | | `-UnqualifiedId
-| | |   `-!
+| | |-!
 | | `-IdExpression
 | |   `-UnqualifiedId
 | | `-x
 | `-;
 |-ExpressionStatement
 | |-PrefixUnaryOperatorExpression
-| | |-IdExpression
-| | | `-UnqualifiedId
-| | |   `-&
+| | |-&
 | | `-IdExpression
 | |   `-UnqualifiedId
 | | `-x
@@ -2809,9 +2793,7 @@
 | | |-IdExpression
 | | | `-UnqualifiedId
 | | |   `-x
-| | `-IdExpression
-| |   `-UnqualifiedId
-| | `-++
+| | `-++
 | `-;
 `-}
 )txt"));
Index: clang/lib/Tooling/Syntax/BuildTree.cpp
===
--- clang/lib/Tooling/Syntax/BuildTree.cpp
+++ clang/lib/Tooling/Syntax/BuildTree.cpp
@@ -1007,23 +1007,26 @@
   }
 
   bool TraverseCXXOperatorCallExpr(CXXOperatorCallExpr *S) {
-if (getOperatorNodeKind(*S) ==
-syntax::NodeKind::PostfixUnaryOperatorExpression) {
+// To construct a syntax tree of the same shape for calls to built-in and
+// user-defined operators, ignore the `DeclRefExpr` that refers to the
+// operator and treat it as a simple token. Do that by traversing
+// arguments instead of children.
+for (auto *child : S->arguments()) {
   // A postfix unary operator is declared as taking two operands. The
   // second operand is used to distinguish from its prefix counterpart. In
   // the semantic AST this "phantom" operand is represented as a
   // `IntegerLiteral` with invalid `SourceLocation`. We skip visiting this
   // operand because it does not correspond to anything written in source
-  // code
-  for (auto *child : S->children()) {
-if (child->getSourceRange().isInvalid())
-  continue;
-if (!TraverseStmt(child))
-  return false;
+  // code.
+  if (child->getSourceRange().isInvalid()) {
+assert(getOperatorNodeKind(*S) ==
+   syntax::NodeKind::PostfixUnaryOperatorExpression);
+continue;
   }
-  return WalkUpFromCXXOperatorCallExpr(S);
-} else
-  return RecursiveASTVisitor::TraverseCXXOperatorCallExpr(S);
+  if (!TraverseStmt(child))
+return false;
+}
+return WalkUpFromCXXOperatorCallExpr(S);
   }
 
   bool WalkUpFromCXXOperatorCallExpr(CXXOperatorCallExpr *S) {
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D85750: [SyntaxTree] Unbox operators into tokens for nodes generated from `CXXOperatorCallExpr`

[Eduardo Caldas via Phabricator via cfe-commits]

eduucaldas added a reviewer: gribozavr2.
eduucaldas added inline comments.



Comment at: clang/lib/Tooling/Syntax/BuildTree.cpp:1024
+  // representation of built-in and user-defined operators.
+  if (child->getBeginLoc() == S->getOperatorLoc())
+continue;

Here we want to check if this child is just a DeclRefExpr to an operator - as 
opposed to an operand.
Is comparing SourceLocation generally safe?
Is there a better way of figuring out if the child DeclRefExpr refers to an 
operator?


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D85750/new/

https://reviews.llvm.org/D85750

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D85750: [SyntaxTree] Unbox operators into tokens for nodes generated from `CXXOperatorCallExpr`

[Eduardo Caldas via Phabricator via cfe-commits]

eduucaldas updated this revision to Diff 284794.
eduucaldas added a comment.

Add explanation comments


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D85750/new/

https://reviews.llvm.org/D85750

Files:
  clang/lib/Tooling/Syntax/BuildTree.cpp
  clang/unittests/Tooling/Syntax/TreeTest.cpp

Index: clang/unittests/Tooling/Syntax/TreeTest.cpp
===
--- clang/unittests/Tooling/Syntax/TreeTest.cpp
+++ clang/unittests/Tooling/Syntax/TreeTest.cpp
@@ -2592,9 +2592,7 @@
 | | |-IdExpression
 | | | `-UnqualifiedId
 | | |   `-x
-| | |-IdExpression
-| | | `-UnqualifiedId
-| | |   `-=
+| | |-=
 | | `-IdExpression
 | |   `-UnqualifiedId
 | | `-y
@@ -2605,9 +2603,7 @@
 | | | `-IdExpression
 | | |   `-UnqualifiedId
 | | | `-x
-| | |-IdExpression
-| | | `-UnqualifiedId
-| | |   `-+
+| | |-+
 | | `-IdExpression
 | |   `-UnqualifiedId
 | | `-y
@@ -2617,9 +2613,7 @@
 | | |-IdExpression
 | | | `-UnqualifiedId
 | | |   `-x
-| | |-IdExpression
-| | | `-UnqualifiedId
-| | |   `-<
+| | |-<
 | | `-IdExpression
 | |   `-UnqualifiedId
 | | `-y
@@ -2629,9 +2623,7 @@
 | | |-IdExpression
 | | | `-UnqualifiedId
 | | |   `-x
-| | |-IdExpression
-| | | `-UnqualifiedId
-| | |   `-<<
+| | |-<<
 | | `-IdExpression
 | |   `-UnqualifiedId
 | | `-y
@@ -2641,9 +2633,7 @@
 | | |-IdExpression
 | | | `-UnqualifiedId
 | | |   `-x
-| | |-IdExpression
-| | | `-UnqualifiedId
-| | |   `-,
+| | |-,
 | | `-IdExpression
 | |   `-UnqualifiedId
 | | `-y
@@ -2730,27 +2720,21 @@
 |-{
 |-ExpressionStatement
 | |-PrefixUnaryOperatorExpression
-| | |-IdExpression
-| | | `-UnqualifiedId
-| | |   `-++
+| | |-++
 | | `-IdExpression
 | |   `-UnqualifiedId
 | | `-x
 | `-;
 |-ExpressionStatement
 | |-PrefixUnaryOperatorExpression
-| | |-IdExpression
-| | | `-UnqualifiedId
-| | |   `-!
+| | |-!
 | | `-IdExpression
 | |   `-UnqualifiedId
 | | `-x
 | `-;
 |-ExpressionStatement
 | |-PrefixUnaryOperatorExpression
-| | |-IdExpression
-| | | `-UnqualifiedId
-| | |   `-&
+| | |-&
 | | `-IdExpression
 | |   `-UnqualifiedId
 | | `-x
@@ -2809,9 +2793,7 @@
 | | |-IdExpression
 | | | `-UnqualifiedId
 | | |   `-x
-| | `-IdExpression
-| |   `-UnqualifiedId
-| | `-++
+| | `-++
 | `-;
 `-}
 )txt"));
Index: clang/lib/Tooling/Syntax/BuildTree.cpp
===
--- clang/lib/Tooling/Syntax/BuildTree.cpp
+++ clang/lib/Tooling/Syntax/BuildTree.cpp
@@ -1007,23 +1007,26 @@
   }
 
   bool TraverseCXXOperatorCallExpr(CXXOperatorCallExpr *S) {
-if (getOperatorNodeKind(*S) ==
-syntax::NodeKind::PostfixUnaryOperatorExpression) {
+for (auto *child : S->children()) {
   // A postfix unary operator is declared as taking two operands. The
   // second operand is used to distinguish from its prefix counterpart. In
   // the semantic AST this "phantom" operand is represented as a
   // `IntegerLiteral` with invalid `SourceLocation`. We skip visiting this
   // operand because it does not correspond to anything written in source
   // code
-  for (auto *child : S->children()) {
-if (child->getSourceRange().isInvalid())
-  continue;
-if (!TraverseStmt(child))
-  return false;
+  if (child->getSourceRange().isInvalid()) {
+assert(getOperatorNodeKind(*S) ==
+   syntax::NodeKind::PostfixUnaryOperatorExpression);
+continue;
   }
-  return WalkUpFromCXXOperatorCallExpr(S);
-} else
-  return RecursiveASTVisitor::TraverseCXXOperatorCallExpr(S);
+  // An operator is treated as a simple token. This allows for an uniform
+  // representation of built-in and user-defined operators.
+  if (child->getBeginLoc() == S->getOperatorLoc())
+continue;
+  if (!TraverseStmt(child))
+return false;
+}
+return WalkUpFromCXXOperatorCallExpr(S);
   }
 
   bool WalkUpFromCXXOperatorCallExpr(CXXOperatorCallExpr *S) {
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D85750: [SyntaxTree] Unbox operators into tokens for nodes generated from `CXXOperatorCallExpr`

[Eduardo Caldas via Phabricator via cfe-commits]

eduucaldas created this revision.
Herald added a project: clang.
Herald added a subscriber: cfe-commits.
eduucaldas requested review of this revision.

For an user define `<`, `x < y` would yield the syntax tree:

  BinaryOperatorExpression
  |-IdExpression
  | `-UnqualifiedId
  |   `-x
  |-IdExpression
  | `-UnqualifiedId
  |   `-<
  `-IdExpression
`-UnqualifiedId
  `-y

But there is no syntatic difference at call site between call site or
built-in `<`. As such they should generate the same syntax tree, namely:

  BinaryOperatorExpression
  |-IdExpression
  | `-UnqualifiedId
  |   `-x
  |-<
  `-IdExpression
`-UnqualifiedId
  `-y


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D85750

Files:
  clang/lib/Tooling/Syntax/BuildTree.cpp
  clang/unittests/Tooling/Syntax/TreeTest.cpp

Index: clang/unittests/Tooling/Syntax/TreeTest.cpp
===
--- clang/unittests/Tooling/Syntax/TreeTest.cpp
+++ clang/unittests/Tooling/Syntax/TreeTest.cpp
@@ -2592,9 +2592,7 @@
 | | |-IdExpression
 | | | `-UnqualifiedId
 | | |   `-x
-| | |-IdExpression
-| | | `-UnqualifiedId
-| | |   `-=
+| | |-=
 | | `-IdExpression
 | |   `-UnqualifiedId
 | | `-y
@@ -2605,9 +2603,7 @@
 | | | `-IdExpression
 | | |   `-UnqualifiedId
 | | | `-x
-| | |-IdExpression
-| | | `-UnqualifiedId
-| | |   `-+
+| | |-+
 | | `-IdExpression
 | |   `-UnqualifiedId
 | | `-y
@@ -2617,9 +2613,7 @@
 | | |-IdExpression
 | | | `-UnqualifiedId
 | | |   `-x
-| | |-IdExpression
-| | | `-UnqualifiedId
-| | |   `-<
+| | |-<
 | | `-IdExpression
 | |   `-UnqualifiedId
 | | `-y
@@ -2629,9 +2623,7 @@
 | | |-IdExpression
 | | | `-UnqualifiedId
 | | |   `-x
-| | |-IdExpression
-| | | `-UnqualifiedId
-| | |   `-<<
+| | |-<<
 | | `-IdExpression
 | |   `-UnqualifiedId
 | | `-y
@@ -2641,9 +2633,7 @@
 | | |-IdExpression
 | | | `-UnqualifiedId
 | | |   `-x
-| | |-IdExpression
-| | | `-UnqualifiedId
-| | |   `-,
+| | |-,
 | | `-IdExpression
 | |   `-UnqualifiedId
 | | `-y
@@ -2730,27 +2720,21 @@
 |-{
 |-ExpressionStatement
 | |-PrefixUnaryOperatorExpression
-| | |-IdExpression
-| | | `-UnqualifiedId
-| | |   `-++
+| | |-++
 | | `-IdExpression
 | |   `-UnqualifiedId
 | | `-x
 | `-;
 |-ExpressionStatement
 | |-PrefixUnaryOperatorExpression
-| | |-IdExpression
-| | | `-UnqualifiedId
-| | |   `-!
+| | |-!
 | | `-IdExpression
 | |   `-UnqualifiedId
 | | `-x
 | `-;
 |-ExpressionStatement
 | |-PrefixUnaryOperatorExpression
-| | |-IdExpression
-| | | `-UnqualifiedId
-| | |   `-&
+| | |-&
 | | `-IdExpression
 | |   `-UnqualifiedId
 | | `-x
@@ -2809,9 +2793,7 @@
 | | |-IdExpression
 | | | `-UnqualifiedId
 | | |   `-x
-| | `-IdExpression
-| |   `-UnqualifiedId
-| | `-++
+| | `-++
 | `-;
 `-}
 )txt"));
Index: clang/lib/Tooling/Syntax/BuildTree.cpp
===
--- clang/lib/Tooling/Syntax/BuildTree.cpp
+++ clang/lib/Tooling/Syntax/BuildTree.cpp
@@ -15,6 +15,7 @@
 #include "clang/AST/ExprCXX.h"
 #include "clang/AST/RecursiveASTVisitor.h"
 #include "clang/AST/Stmt.h"
+#include "clang/AST/StmtIterator.h"
 #include "clang/AST/TypeLoc.h"
 #include "clang/AST/TypeLocVisitor.h"
 #include "clang/Basic/LLVM.h"
@@ -1007,23 +1008,24 @@
   }
 
   bool TraverseCXXOperatorCallExpr(CXXOperatorCallExpr *S) {
-if (getOperatorNodeKind(*S) ==
-syntax::NodeKind::PostfixUnaryOperatorExpression) {
+for (auto *child : S->children()) {
   // A postfix unary operator is declared as taking two operands. The
   // second operand is used to distinguish from its prefix counterpart. In
   // the semantic AST this "phantom" operand is represented as a
   // `IntegerLiteral` with invalid `SourceLocation`. We skip visiting this
   // operand because it does not correspond to anything written in source
   // code
-  for (auto *child : S->children()) {
-if (child->getSourceRange().isInvalid())
-  continue;
-if (!TraverseStmt(child))
-  return false;
+  if (child->getSourceRange().isInvalid()) {
+assert(getOperatorNodeKind(*S) ==
+   syntax::NodeKind::PostfixUnaryOperatorExpression);
+continue;
   }
-  return WalkUpFromCXXOperatorCallExpr(S);
-} else
-  return RecursiveASTVisitor::TraverseCXXOperatorCallExpr(S);
+  if (child->getBeginLoc() == S->getOperatorLoc())
+continue;
+  if (!TraverseStmt(child))
+return false;
+}
+return WalkUpFromCXXOperatorCallExpr(S);
   }
 
   bool WalkUpFromCXXOperatorCallExpr(CXXOperatorCallExpr *S) {

[PATCH] D85713: [SyntaxTree]: Use Annotations in tests to reduce noise

[Eduardo Caldas via Phabricator via cfe-commits]

eduucaldas updated this revision to Diff 284678.
eduucaldas added a comment.

Use annotations to reduce noise of expression tests


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D85713/new/

https://reviews.llvm.org/D85713

Files:
  clang/unittests/Tooling/Syntax/TreeTest.cpp

Index: clang/unittests/Tooling/Syntax/TreeTest.cpp
===
--- clang/unittests/Tooling/Syntax/TreeTest.cpp
+++ clang/unittests/Tooling/Syntax/TreeTest.cpp
@@ -168,6 +168,35 @@
 return ::testing::AssertionSuccess();
   }
 
+  ::testing::AssertionResult
+  treeDumpEqualOnAnnotations(StringRef CodeWithAnnotations,
+ ArrayRef TreeDumps) {
+SCOPED_TRACE(llvm::join(GetParam().getCommandLineArgs(), " "));
+
+auto AnnotatedCode = llvm::Annotations(CodeWithAnnotations);
+auto *Root = buildTree(AnnotatedCode.code(), GetParam());
+
+if (Diags->getClient()->getNumErrors() != 0) {
+  return ::testing::AssertionFailure()
+ << "Source file has syntax errors, they were printed to the test "
+"log";
+}
+
+auto AnnotatedRanges = AnnotatedCode.ranges();
+assert(AnnotatedRanges.size() == TreeDumps.size());
+for (auto i = 0ul; i < AnnotatedRanges.size(); i++) {
+  auto *AnnotatedNode = nodeByRange(AnnotatedRanges[i], Root);
+  assert(AnnotatedNode);
+  auto AnnotatedNodeDump =
+  std::string(StringRef(AnnotatedNode->dump(*Arena)).trim());
+  // EXPECT_EQ shows the diff between the two strings if they are different.
+  EXPECT_EQ(TreeDumps[i].trim().str(), AnnotatedNodeDump);
+  if (AnnotatedNodeDump != TreeDumps[i].trim().str())
+return ::testing::AssertionFailure();
+}
+return ::testing::AssertionSuccess();
+  }
+
   // Adds a file to the test VFS.
   void addFile(StringRef Path, StringRef Contents) {
 if (!FS->addFile(Path, time_t(),
@@ -632,7 +661,7 @@
 )txt"));
 }
 
-TEST_P(SyntaxTreeTest, UnqualifiedId) {
+TEST_P(SyntaxTreeTest, UnqualifiedIdDecl) {
   if (!GetParam().isCXX()) {
 return;
   }
@@ -645,14 +674,7 @@
 };
 template
 void f(T&);
-void test(X x) {
-  x;  // identifier
-  operator+(x, x);// operator-function-id
-  f(x);// template-id
-  // TODO: Expose `id-expression` from `MemberExpr`
-  x.operator int();   // conversion-funtion-id
-  x.~X(); // ~type-name
-}
+void test(X x);
 )cpp",
   R"txt(
 *: TranslationUnit
@@ -722,72 +744,100 @@
   |   | `-SimpleDeclarator
   |   |   `-x
   |   `-)
-  `-CompoundStatement
-|-{
-|-ExpressionStatement
-| |-IdExpression
-| | `-UnqualifiedId
-| |   `-x
-| `-;
-|-ExpressionStatement
-| |-UnknownExpression
-| | |-IdExpression
-| | | `-UnqualifiedId
-| | |   |-operator
-| | |   `-+
-| | |-(
-| | |-IdExpression
-| | | `-UnqualifiedId
-| | |   `-x
-| | |-,
-| | |-IdExpression
-| | | `-UnqualifiedId
-| | |   `-x
-| | `-)
-| `-;
-|-ExpressionStatement
-| |-UnknownExpression
-| | |-IdExpression
-| | | `-UnqualifiedId
-| | |   |-f
-| | |   |-<
-| | |   |-X
-| | |   `->
-| | |-(
-| | |-IdExpression
-| | | `-UnqualifiedId
-| | |   `-x
-| | `-)
-| `-;
-|-ExpressionStatement
-| |-UnknownExpression
-| | |-UnknownExpression
-| | | |-IdExpression
-| | | | `-UnqualifiedId
-| | | |   `-x
-| | | |-.
-| | | |-operator
-| | | `-int
-| | |-(
-| | `-)
-| `-;
-|-ExpressionStatement
-| |-UnknownExpression
-| | |-UnknownExpression
-| | | |-IdExpression
-| | | | `-UnqualifiedId
-| | | |   `-x
-| | | |-.
-| | | |-~
-| | | `-X
-| | |-(
-| | `-)
-| `-;
-`-}
+  `-;
 )txt"));
 }
 
-TEST_P(SyntaxTreeTest, UnqualifiedIdCxx11OrLater) {
+TEST_P(SyntaxTreeTest, UnqualifiedId) {
+  if (!GetParam().isCXX()) {
+return;
+  }
+  EXPECT_TRUE(treeDumpEqualOnAnnotations(
+  R"cpp(
+struct X {
+  // TODO: Expose `id-expression` from `Declarator`
+  friend X operator+(const X&, const X&);
+  operator int();
+};
+template
+void f(T&);
+void test(X x) 
+[[{
+  x;  // identifier
+  operator+(x, x);// operator-function-id
+  f(x);// template-id
+  // TODO: Expose `id-expression` from `MemberExpr`
+  x.operator int();   // conversion-funtion-id
+  x.~X(); // ~type-name
+}]]
+)cpp",
+  {R"txt(
+CompoundStatement
+|-{
+|-ExpressionStatement
+| |-IdExpression
+| | `-UnqualifiedId
+| |   `-x
+| `-;
+|-ExpressionStatement
+| |-UnknownExpression
+| | |-IdExpression
+| | | `-UnqualifiedId
+| | |   |-operator
+| | |   `-+
+| | |-(
+| | |-IdExpression
+| | | `-UnqualifiedId
+| | |   `-x
+| | |-,
+| | |-IdExpression
+| | | `-UnqualifiedId
+| | |   `-x
+| | `-)
+| `-;
+|-ExpressionStatement
+| |-UnknownExpression
+| | |-IdExpression

[PATCH] D85713: [SyntaxTree]: Use Annotations in tests to reduce noise

[Eduardo Caldas via Phabricator via cfe-commits]

eduucaldas added a reviewer: gribozavr2.
eduucaldas added a comment.

A proposition, upon review I'll change other tests.

One concern is that we might lose coverage while reducing noise. But I'll take 
a look into that with calm when changing the tests.




Comment at: clang/unittests/Tooling/Syntax/TreeTest.cpp:186-187
+auto AnnotatedRanges = AnnotatedCode.ranges();
+assert(AnnotatedRanges.size() == TreeDumps.size());
+for (auto i = 0u; i < AnnotatedRanges.size(); i++) {
+  auto *AnnotatedNode = nodeByRange(AnnotatedRanges[i], Root);

I just wanted to do a for( auto [range, dump]& : zip(AnnotatedRanges,TreeDumps))

Is indexed loop the way to go in C++? 
And also I used `0u` here because we make a comparison to 
`std::vector::size_type`, is there a less error-prone way of writing those loop 
indexes?


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D85713/new/

https://reviews.llvm.org/D85713

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D85713: [SyntaxTree]: Use Annotations in tests to reduce noise

[Eduardo Caldas via Phabricator via cfe-commits]

eduucaldas updated this revision to Diff 284610.
eduucaldas added a comment.

Use unsigned long in the loop index


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D85713/new/

https://reviews.llvm.org/D85713

Files:
  clang/unittests/Tooling/Syntax/TreeTest.cpp

Index: clang/unittests/Tooling/Syntax/TreeTest.cpp
===
--- clang/unittests/Tooling/Syntax/TreeTest.cpp
+++ clang/unittests/Tooling/Syntax/TreeTest.cpp
@@ -168,6 +168,35 @@
 return ::testing::AssertionSuccess();
   }
 
+  ::testing::AssertionResult
+  treeDumpEqualOnAnnotations(StringRef CodeWithAnnotations,
+ ArrayRef TreeDumps) {
+SCOPED_TRACE(llvm::join(GetParam().getCommandLineArgs(), " "));
+
+auto AnnotatedCode = llvm::Annotations(CodeWithAnnotations);
+auto *Root = buildTree(AnnotatedCode.code(), GetParam());
+
+if (Diags->getClient()->getNumErrors() != 0) {
+  return ::testing::AssertionFailure()
+ << "Source file has syntax errors, they were printed to the test "
+"log";
+}
+
+auto AnnotatedRanges = AnnotatedCode.ranges();
+assert(AnnotatedRanges.size() == TreeDumps.size());
+for (auto i = 0ul; i < AnnotatedRanges.size(); i++) {
+  auto *AnnotatedNode = nodeByRange(AnnotatedRanges[i], Root);
+  assert(AnnotatedNode);
+  auto AnnotatedNodeDump =
+  std::string(StringRef(AnnotatedNode->dump(*Arena)).trim());
+  // EXPECT_EQ shows the diff between the two strings if they are different.
+  EXPECT_EQ(TreeDumps[i].trim().str(), AnnotatedNodeDump);
+  if (AnnotatedNodeDump != TreeDumps[i].trim().str())
+return ::testing::AssertionFailure();
+}
+return ::testing::AssertionSuccess();
+  }
+
   // Adds a file to the test VFS.
   void addFile(StringRef Path, StringRef Contents) {
 if (!FS->addFile(Path, time_t(),
@@ -214,34 +243,23 @@
 };
 
 TEST_P(SyntaxTreeTest, Simple) {
-  EXPECT_TRUE(treeDumpEqual(
+  EXPECT_TRUE(treeDumpEqualOnAnnotations(
   R"cpp(
 int main() {}
-void foo() {}
+[[void foo() {}]]
 )cpp",
-  R"txt(
-*: TranslationUnit
-|-SimpleDeclaration
-| |-int
-| |-SimpleDeclarator
-| | |-main
-| | `-ParametersAndQualifiers
-| |   |-(
-| |   `-)
-| `-CompoundStatement
-|   |-{
-|   `-}
-`-SimpleDeclaration
-  |-void
-  |-SimpleDeclarator
-  | |-foo
-  | `-ParametersAndQualifiers
-  |   |-(
-  |   `-)
-  `-CompoundStatement
-|-{
-`-}
-)txt"));
+  {R"txt(
+SimpleDeclaration
+|-void
+|-SimpleDeclarator
+| |-foo
+| `-ParametersAndQualifiers
+|   |-(
+|   `-)
+`-CompoundStatement
+  |-{
+  `-}
+)txt"}));
 }
 
 TEST_P(SyntaxTreeTest, SimpleVariable) {
@@ -871,7 +889,7 @@
   if (!GetParam().isCXX()) {
 return;
   }
-  EXPECT_TRUE(treeDumpEqual(
+  EXPECT_TRUE(treeDumpEqualOnAnnotations(
   R"cpp(
 namespace n {
   struct S {
@@ -889,194 +907,71 @@
   };
 };
 void test() {
-  :: // global-namespace-specifier
-  n::// namespace-specifier
-  S::// type-name-specifier
-  template ST:: // type-template-instantiation-specifier
-  f();
-
-  n::// namespace-specifier
-  S::// type-name-specifier
-  ST::  // type-template-instantiation-specifier
-  f();
-
-  ST:: // type-name-specifier
-  S::   // type-name-specifier
-  f();
-
-  ST:: // type-name-specifier
-  S::   // type-name-specifier
-  template f();
+[[::n::S::template ST::]]f();
+
+[[n::S::ST::]]f();
+
+[[ST::S::]]f();
+
+[[ST::S::]]template f();
 }
 )cpp",
-  R"txt(
-*: TranslationUnit
-|-NamespaceDefinition
-| |-namespace
-| |-n
-| |-{
-| |-SimpleDeclaration
-| | |-struct
-| | |-S
-| | |-{
-| | |-TemplateDeclaration
-| | | |-template
-| | | |-<
-| | | |-UnknownDeclaration
-| | | | |-typename
-| | | | `-T
-| | | |->
-| | | `-SimpleDeclaration
-| | |   |-struct
-| | |   |-ST
-| | |   |-{
-| | |   |-SimpleDeclaration
-| | |   | |-static
-| | |   | |-void
-| | |   | |-SimpleDeclarator
-| | |   | | |-f
-| | |   | | `-ParametersAndQualifiers
-| | |   | |   |-(
-| | |   | |   `-)
-| | |   | `-;
-| | |   |-}
-| | |   `-;
-| | |-}
-| | `-;
-| `-}
-|-TemplateDeclaration
+  {R"txt(
+NestedNameSpecifier
+|-::
+|-IdentifierNameSpecifier
+| `-n
+|-::
+|-IdentifierNameSpecifier
+| `-S
+|-::
+|-SimpleTemplateNameSpecifier
 | |-template
+| |-ST
 | |-<
-| |-UnknownDeclaration
-| | |-typename
-| | `-T
-| |->
-| `-SimpleDeclaration
-|   |-struct
-|   |-ST
-|   |-{
-|   |-SimpleDeclaration
-|   | |-struct
-|   | |-S
-|   | |-{
-|   | |-TemplateDeclaration
-|   | | |-template
-|   | | |-<
-|   | | |-UnknownDeclaration
-|   | | | |-typename
-|   | | | `-U
-|   | | |->
-|   | | `-SimpleDeclaration
-|   | |   |-static
-|   | |   |-U
-|   | |   |-SimpleDeclarator
-|   | |   | |-f
-|   | |   | `-ParametersAndQualifiers
-|   | |   |   |-(
-|   | |   |   `-)
-|   | |   `-;
-|   | |-}
-|   | `-;
-|   |-}
-|   `-;

[PATCH] D85713: [SyntaxTree]: Use Annotations in tests to reduce noise

[Eduardo Caldas via Phabricator via cfe-commits]

eduucaldas created this revision.
Herald added a project: clang.
Herald added a subscriber: cfe-commits.
eduucaldas requested review of this revision.

Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D85713

Files:
  clang/unittests/Tooling/Syntax/TreeTest.cpp

Index: clang/unittests/Tooling/Syntax/TreeTest.cpp
===
--- clang/unittests/Tooling/Syntax/TreeTest.cpp
+++ clang/unittests/Tooling/Syntax/TreeTest.cpp
@@ -168,6 +168,35 @@
 return ::testing::AssertionSuccess();
   }
 
+  ::testing::AssertionResult
+  treeDumpEqualOnAnnotations(StringRef CodeWithAnnotations,
+ ArrayRef TreeDumps) {
+SCOPED_TRACE(llvm::join(GetParam().getCommandLineArgs(), " "));
+
+auto AnnotatedCode = llvm::Annotations(CodeWithAnnotations);
+auto *Root = buildTree(AnnotatedCode.code(), GetParam());
+
+if (Diags->getClient()->getNumErrors() != 0) {
+  return ::testing::AssertionFailure()
+ << "Source file has syntax errors, they were printed to the test "
+"log";
+}
+
+auto AnnotatedRanges = AnnotatedCode.ranges();
+assert(AnnotatedRanges.size() == TreeDumps.size());
+for (auto i = 0u; i < AnnotatedRanges.size(); i++) {
+  auto *AnnotatedNode = nodeByRange(AnnotatedRanges[i], Root);
+  assert(AnnotatedNode);
+  auto AnnotatedNodeDump =
+  std::string(StringRef(AnnotatedNode->dump(*Arena)).trim());
+  // EXPECT_EQ shows the diff between the two strings if they are different.
+  EXPECT_EQ(TreeDumps[i].trim().str(), AnnotatedNodeDump);
+  if (AnnotatedNodeDump != TreeDumps[i].trim().str())
+return ::testing::AssertionFailure();
+}
+return ::testing::AssertionSuccess();
+  }
+
   // Adds a file to the test VFS.
   void addFile(StringRef Path, StringRef Contents) {
 if (!FS->addFile(Path, time_t(),
@@ -214,34 +243,23 @@
 };
 
 TEST_P(SyntaxTreeTest, Simple) {
-  EXPECT_TRUE(treeDumpEqual(
+  EXPECT_TRUE(treeDumpEqualOnAnnotations(
   R"cpp(
 int main() {}
-void foo() {}
+[[void foo() {}]]
 )cpp",
-  R"txt(
-*: TranslationUnit
-|-SimpleDeclaration
-| |-int
-| |-SimpleDeclarator
-| | |-main
-| | `-ParametersAndQualifiers
-| |   |-(
-| |   `-)
-| `-CompoundStatement
-|   |-{
-|   `-}
-`-SimpleDeclaration
-  |-void
-  |-SimpleDeclarator
-  | |-foo
-  | `-ParametersAndQualifiers
-  |   |-(
-  |   `-)
-  `-CompoundStatement
-|-{
-`-}
-)txt"));
+  {R"txt(
+SimpleDeclaration
+|-void
+|-SimpleDeclarator
+| |-foo
+| `-ParametersAndQualifiers
+|   |-(
+|   `-)
+`-CompoundStatement
+  |-{
+  `-}
+)txt"}));
 }
 
 TEST_P(SyntaxTreeTest, SimpleVariable) {
@@ -871,7 +889,7 @@
   if (!GetParam().isCXX()) {
 return;
   }
-  EXPECT_TRUE(treeDumpEqual(
+  EXPECT_TRUE(treeDumpEqualOnAnnotations(
   R"cpp(
 namespace n {
   struct S {
@@ -889,194 +907,71 @@
   };
 };
 void test() {
-  :: // global-namespace-specifier
-  n::// namespace-specifier
-  S::// type-name-specifier
-  template ST:: // type-template-instantiation-specifier
-  f();
-
-  n::// namespace-specifier
-  S::// type-name-specifier
-  ST::  // type-template-instantiation-specifier
-  f();
-
-  ST:: // type-name-specifier
-  S::   // type-name-specifier
-  f();
-
-  ST:: // type-name-specifier
-  S::   // type-name-specifier
-  template f();
+[[::n::S::template ST::]]f();
+
+[[n::S::ST::]]f();
+
+[[ST::S::]]f();
+
+[[ST::S::]]template f();
 }
 )cpp",
-  R"txt(
-*: TranslationUnit
-|-NamespaceDefinition
-| |-namespace
-| |-n
-| |-{
-| |-SimpleDeclaration
-| | |-struct
-| | |-S
-| | |-{
-| | |-TemplateDeclaration
-| | | |-template
-| | | |-<
-| | | |-UnknownDeclaration
-| | | | |-typename
-| | | | `-T
-| | | |->
-| | | `-SimpleDeclaration
-| | |   |-struct
-| | |   |-ST
-| | |   |-{
-| | |   |-SimpleDeclaration
-| | |   | |-static
-| | |   | |-void
-| | |   | |-SimpleDeclarator
-| | |   | | |-f
-| | |   | | `-ParametersAndQualifiers
-| | |   | |   |-(
-| | |   | |   `-)
-| | |   | `-;
-| | |   |-}
-| | |   `-;
-| | |-}
-| | `-;
-| `-}
-|-TemplateDeclaration
+  {R"txt(
+NestedNameSpecifier
+|-::
+|-IdentifierNameSpecifier
+| `-n
+|-::
+|-IdentifierNameSpecifier
+| `-S
+|-::
+|-SimpleTemplateNameSpecifier
 | |-template
+| |-ST
 | |-<
-| |-UnknownDeclaration
-| | |-typename
-| | `-T
-| |->
-| `-SimpleDeclaration
-|   |-struct
-|   |-ST
-|   |-{
-|   |-SimpleDeclaration
-|   | |-struct
-|   | |-S
-|   | |-{
-|   | |-TemplateDeclaration
-|   | | |-template
-|   | | |-<
-|   | | |-UnknownDeclaration
-|   | | | |-typename
-|   | | | `-U
-|   | | |->
-|   | | `-SimpleDeclaration
-|   | |   |-static
-|   | |   |-U
-|   | |   |-SimpleDeclarator
-|   | |   | |-f
-|   | |   | `-ParametersAndQualifiers
-|   | |   |   |-(
-|   | |   |   `-)
-|   | |   `-;
-|   | |-}
-|   | `-;
-|   |-}
-|   `-;
-`-SimpleDeclaration
-  |-void

[PATCH] D85439: [SyntaxTree] Expand support for `NestedNameSpecifier`

[Eduardo Caldas via Phabricator via cfe-commits]

eduucaldas updated this revision to Diff 284387.
eduucaldas marked 6 inline comments as done and 2 inline comments as done.
eduucaldas added a comment.

Answer comments


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D85439/new/

https://reviews.llvm.org/D85439

Files:
  clang/include/clang/AST/NestedNameSpecifier.h
  clang/lib/Tooling/Syntax/BuildTree.cpp
  clang/unittests/Tooling/Syntax/TreeTest.cpp

Index: clang/unittests/Tooling/Syntax/TreeTest.cpp
===
--- clang/unittests/Tooling/Syntax/TreeTest.cpp
+++ clang/unittests/Tooling/Syntax/TreeTest.cpp
@@ -2973,7 +2973,8 @@
 `-UsingNamespaceDirective
   |-using
   |-namespace
-  |-::
+  |-NestedNameSpecifier
+  | `-::
   |-ns
   `-;
 )txt"));
@@ -3002,8 +3003,10 @@
 | `-}
 `-UsingDeclaration
   |-using
-  |-ns
-  |-::
+  |-NestedNameSpecifier
+  | |-IdentifierNameSpecifier
+  | | `-ns
+  | `-::
   |-a
   `-;
 )txt"));
@@ -3207,11 +3210,13 @@
   |->
   `-SimpleDeclaration
 |-struct
-|-X
-|-<
-|-T
-|->
-|-::
+|-NestedNameSpecifier
+| |-SimpleTemplateNameSpecifier
+| | |-X
+| | |-<
+| | |-T
+| | `->
+| `-::
 |-Y
 |-{
 |-}
@@ -3245,15 +3250,19 @@
 |-{
 |-UsingDeclaration
 | |-using
-| |-T
-| |-::
+| |-NestedNameSpecifier
+| | |-IdentifierNameSpecifier
+| | | `-T
+| | `-::
 | |-foo
 | `-;
 |-UsingDeclaration
 | |-using
 | |-typename
-| |-T
-| |-::
+| |-NestedNameSpecifier
+| | |-IdentifierNameSpecifier
+| | | `-T
+| | `-::
 | |-bar
 | `-;
 |-}
Index: clang/lib/Tooling/Syntax/BuildTree.cpp
===
--- clang/lib/Tooling/Syntax/BuildTree.cpp
+++ clang/lib/Tooling/Syntax/BuildTree.cpp
@@ -241,10 +241,24 @@
 assert(Added && "mapping added twice");
   }
 
+  void add(NestedNameSpecifierLoc From, syntax::Tree *To) {
+assert(To != nullptr);
+assert(From.hasQualifier());
+
+bool Added = NNSNodes.insert({From, To}).second;
+(void)Added;
+assert(Added && "mapping added twice");
+  }
+
   syntax::Tree *find(ASTPtr P) const { return Nodes.lookup(P); }
 
+  syntax::Tree *find(NestedNameSpecifierLoc P) const {
+return NNSNodes.lookup(P);
+  }
+
 private:
   llvm::DenseMap Nodes;
+  llvm::DenseMap NNSNodes;
 };
 } // namespace
 
@@ -281,16 +295,20 @@
 if (From)
   Mapping.add(From, New);
   }
+
   void foldNode(ArrayRef Range, syntax::Tree *New, TypeLoc L) {
 // FIXME: add mapping for TypeLocs
 foldNode(Range, New, nullptr);
   }
 
-  void foldNode(ArrayRef Range, syntax::Tree *New,
-NestedNameSpecifierLoc L) {
-// FIXME: add mapping for NestedNameSpecifierLoc
-foldNode(Range, New, nullptr);
+  void foldNode(llvm::ArrayRef Range, syntax::Tree *New,
+NestedNameSpecifierLoc From) {
+assert(New);
+Pending.foldChildren(Arena, Range, New);
+if (From)
+  Mapping.add(From, New);
   }
+
   /// Notifies that we should not consume trailing semicolon when computing
   /// token range of \p D.
   void noticeDeclWithoutSemicolon(Decl *D);
@@ -312,6 +330,8 @@
   void markChild(syntax::Node *N, NodeRole R);
   /// Set role for the syntax node matching \p N.
   void markChild(ASTPtr N, NodeRole R);
+  /// Set role for the syntax node matching \p N.
+  void markChild(NestedNameSpecifierLoc N, NodeRole R);
 
   /// Finish building the tree and consume the root node.
   syntax::TranslationUnit *finalize() && {
@@ -744,45 +764,18 @@
 return true;
   }
 
-  syntax::NameSpecifier *BuildNameSpecifier(const NestedNameSpecifier ) {
-switch (NNS.getKind()) {
-case NestedNameSpecifier::Global:
-  return new (allocator()) syntax::GlobalNameSpecifier;
-case NestedNameSpecifier::Namespace:
-case NestedNameSpecifier::NamespaceAlias:
-case NestedNameSpecifier::Identifier:
-  return new (allocator()) syntax::IdentifierNameSpecifier;
-case NestedNameSpecifier::TypeSpecWithTemplate:
-  return new (allocator()) syntax::SimpleTemplateNameSpecifier;
-case NestedNameSpecifier::TypeSpec: {
-  const auto *NNSType = NNS.getAsType();
-  assert(NNSType);
-  if (isa(NNSType))
-return new (allocator()) syntax::DecltypeNameSpecifier;
-  if (isa(
-  NNSType))
-return new (allocator()) syntax::SimpleTemplateNameSpecifier;
-  return new (allocator()) syntax::IdentifierNameSpecifier;
-}
-case NestedNameSpecifier::Super:
-  // FIXME: Support Microsoft's __super
-  llvm::report_fatal_error("We don't yet support the __super specifier",
-   true);
-}
-llvm_unreachable("Unhandled NestedNameSpecifier::SpecifierKind enum");
-  }
-
   // FIXME: Fix `NestedNameSpecifierLoc::getLocalSourceRange` for the
   // `DependentTemplateSpecializationType` case.
-  /// Given a 

[PATCH] D85439: [SyntaxTree] Expand support for `NestedNameSpecifier`

[Eduardo Caldas via Phabricator via cfe-commits]

eduucaldas added a comment.

There are some refinements to do, to generate a complete syntax tree. Namely 
tag decltype-name-specifier and simple-template-specifier children with roles, 
to allow for accessors.




Comment at: clang/lib/Tooling/Syntax/BuildTree.cpp:838-842
+  // TODO: Build `SimpleTemplateNameSpecifier` children and implement
+  // accessors to them.
+  // Be aware, we cannot do that simply by calling `TraverseTypeLoc`,
+  // some `TypeLoc`s have inside them the previous name specifier and
+  // we want to treat them independently.

Let's generate the syntax tree for the nested-name-specifier:
```
T::template U::X::
```

`T` and `X` are just identifier-name-specifiers. They are easy.

`template U` however is a simple-template-name-specifier. 
```
simple-template-name-specifier :
  template_opt simple-template-id
```
We could generate it by traversing the corresponding semantic node. But this 
node is a `DependentTemplateSpecializationTypeLoc` and it covers `T::template 
U`. The traversal would then cover `T::` and thus generate a crash.

As such, we should treat simple-template-name-specifier in a less generic way.



Comment at: clang/lib/Tooling/Syntax/BuildTree.cpp:849-850
+  const auto TL = NNSLoc.getTypeLoc().castAs();
+  if (!RecursiveASTVisitor::TraverseDecltypeTypeLoc(TL))
+return nullptr;
+  auto *NS = new (allocator()) syntax::DecltypeNameSpecifier;

eduucaldas wrote:
> Since we are overriding the TraverseNestedNameSpecifierLoc that previously 
> fired TraverseTypeLoc we fire it when building a NameSpecifier.
As opposed to simple-template-name-specifier, we can here use a normal 
traversal to build the decltype-name-specifier because this specifier doesn't 
cross boundaries of name specifiers


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D85439/new/

https://reviews.llvm.org/D85439

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D85440: [SyntaxTree] Implement `NNS` using the `List` base API

[Eduardo Caldas via Phabricator via cfe-commits]

eduucaldas updated this revision to Diff 284344.
eduucaldas added a comment.

lint


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D85440/new/

https://reviews.llvm.org/D85440

Files:
  clang/include/clang/Tooling/Syntax/Nodes.h
  clang/lib/Tooling/Syntax/BuildTree.cpp
  clang/lib/Tooling/Syntax/Nodes.cpp
  clang/lib/Tooling/Syntax/Tree.cpp

Index: clang/lib/Tooling/Syntax/Tree.cpp
===
--- clang/lib/Tooling/Syntax/Tree.cpp
+++ clang/lib/Tooling/Syntax/Tree.cpp
@@ -357,21 +357,32 @@
   return children;
 }
 
-// The methods below can't be implemented without information about the derived
-// list. These methods will be implemented by switching on the derived list's
-// `NodeKind`
-
 clang::tok::TokenKind syntax::List::getDelimiterTokenKind() {
-  llvm_unreachable("There are no subclasses of List, thus "
-   "getDelimiterTokenKind() cannot be called");
+  switch (this->kind()) {
+  case NodeKind::NestedNameSpecifier:
+return clang::tok::coloncolon;
+  default:
+llvm_unreachable("This is not a subclass of List, thus "
+ "getDelimiterTokenKind() cannot be called");
+  }
 }
 
 syntax::List::TerminationKind syntax::List::getTerminationKind() {
-  llvm_unreachable("There are no subclasses of List, thus getTerminationKind() "
-   "cannot be called");
+  switch (this->kind()) {
+  case NodeKind::NestedNameSpecifier:
+return TerminationKind::Terminated;
+  default:
+llvm_unreachable("This is not a subclass of List, thus "
+ "getTerminationKind() cannot be called");
+  }
 }
 
 bool syntax::List::canBeEmpty() {
-  llvm_unreachable(
-  "There are no subclasses of List, thus canBeEmpty() cannot be called");
+  switch (this->kind()) {
+  case NodeKind::NestedNameSpecifier:
+return false;
+  default:
+llvm_unreachable("This is not a subclass of List, thus canBeEmpty() "
+ "cannot be called");
+  }
 }
Index: clang/lib/Tooling/Syntax/Nodes.cpp
===
--- clang/lib/Tooling/Syntax/Nodes.cpp
+++ clang/lib/Tooling/Syntax/Nodes.cpp
@@ -200,30 +200,32 @@
 return OS << "IdExpression_id";
   case syntax::NodeRole::IdExpression_qualifier:
 return OS << "IdExpression_qualifier";
-  case syntax::NodeRole::NestedNameSpecifier_specifier:
-return OS << "NestedNameSpecifier_specifier";
-  case syntax::NodeRole::NestedNameSpecifier_delimiter:
-return OS << "NestedNameSpecifier_delimiter";
   case syntax::NodeRole::ParenExpression_subExpression:
 return OS << "ParenExpression_subExpression";
   }
   llvm_unreachable("invalid role");
 }
 
-std::vector syntax::NestedNameSpecifier::delimiters() {
-  std::vector Children;
-  for (auto *C = firstChild(); C; C = C->nextSibling()) {
-assert(C->role() == syntax::NodeRole::NestedNameSpecifier_delimiter);
-Children.push_back(llvm::cast(C));
+// We could have an interator in list to not pay memory costs of temporary
+// vector
+std::vector syntax::NestedNameSpecifier::specifiers() {
+  auto specifiersAsNodes = getElementsAsNodes();
+  std::vector Children;
+  for (const auto  : specifiersAsNodes) {
+Children.push_back(llvm::cast(element));
   }
   return Children;
 }
 
-std::vector syntax::NestedNameSpecifier::specifiers() {
-  std::vector Children;
-  for (auto *C = firstChild(); C; C = C->nextSibling()) {
-assert(C->role() == syntax::NodeRole::NestedNameSpecifier_specifier);
-Children.push_back(cast(C));
+std::vector>
+syntax::NestedNameSpecifier::specifiersAndDoubleColons() {
+  auto specifiersAsNodesAndDoubleColons = getElementsAsNodesAndDelimiters();
+  std::vector>
+  Children;
+  for (const auto  : specifiersAsNodesAndDoubleColons) {
+Children.push_back(
+{llvm::cast(specifierAndDoubleColon.element),
+ specifierAndDoubleColon.delimiter});
   }
   return Children;
 }
Index: clang/lib/Tooling/Syntax/BuildTree.cpp
===
--- clang/lib/Tooling/Syntax/BuildTree.cpp
+++ clang/lib/Tooling/Syntax/BuildTree.cpp
@@ -809,9 +809,8 @@
   if (!isa(NS))
 Builder.foldNode(Builder.getRange(getLocalSourceRange(it)).drop_back(),
  NS, it);
-  Builder.markChild(NS, syntax::NodeRole::NestedNameSpecifier_specifier);
-  Builder.markChildToken(it.getEndLoc(),
- syntax::NodeRole::NestedNameSpecifier_delimiter);
+  Builder.markChild(NS, syntax::NodeRole::List_element);
+  Builder.markChildToken(it.getEndLoc(), syntax::NodeRole::List_delimiter);
 }
 auto *NNS = new (allocator()) syntax::NestedNameSpecifier;
 Builder.foldNode(Builder.getRange(QualifierLoc.getSourceRange()), NNS,
Index: clang/include/clang/Tooling/Syntax/Nodes.h
===
--- 

[PATCH] D85440: [SyntaxTree] Implement `NNS` using the `List` base API

[Eduardo Caldas via Phabricator via cfe-commits]

eduucaldas updated this revision to Diff 284342.
eduucaldas added a comment.

Answer comments


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D85440/new/

https://reviews.llvm.org/D85440

Files:
  clang/include/clang/Tooling/Syntax/Nodes.h
  clang/lib/Tooling/Syntax/BuildTree.cpp
  clang/lib/Tooling/Syntax/Nodes.cpp
  clang/lib/Tooling/Syntax/Tree.cpp

Index: clang/lib/Tooling/Syntax/Tree.cpp
===
--- clang/lib/Tooling/Syntax/Tree.cpp
+++ clang/lib/Tooling/Syntax/Tree.cpp
@@ -357,21 +357,32 @@
   return children;
 }
 
-// The methods below can't be implemented without information about the derived
-// list. These methods will be implemented by switching on the derived list's
-// `NodeKind`
-
 clang::tok::TokenKind syntax::List::getDelimiterTokenKind() {
-  llvm_unreachable("There are no subclasses of List, thus "
-   "getDelimiterTokenKind() cannot be called");
+  switch (this->kind()) {
+  case NodeKind::NestedNameSpecifier:
+return clang::tok::coloncolon;
+  default:
+llvm_unreachable("This is not a subclass of List, thus "
+ "getDelimiterTokenKind() cannot be called");
+  }
 }
 
 syntax::List::TerminationKind syntax::List::getTerminationKind() {
-  llvm_unreachable("There are no subclasses of List, thus getTerminationKind() "
-   "cannot be called");
+  switch (this->kind()) {
+  case NodeKind::NestedNameSpecifier:
+return TerminationKind::Terminated;
+  default:
+llvm_unreachable("This is not a subclass of List, thus "
+ "getTerminationKind() cannot be called");
+  }
 }
 
 bool syntax::List::canBeEmpty() {
-  llvm_unreachable(
-  "There are no subclasses of List, thus canBeEmpty() cannot be called");
+  switch (this->kind()) {
+  case NodeKind::NestedNameSpecifier:
+return false;
+  default:
+llvm_unreachable("This is not a subclass of List, thus canBeEmpty() "
+ "cannot be called");
+  }
 }
Index: clang/lib/Tooling/Syntax/Nodes.cpp
===
--- clang/lib/Tooling/Syntax/Nodes.cpp
+++ clang/lib/Tooling/Syntax/Nodes.cpp
@@ -200,30 +200,32 @@
 return OS << "IdExpression_id";
   case syntax::NodeRole::IdExpression_qualifier:
 return OS << "IdExpression_qualifier";
-  case syntax::NodeRole::NestedNameSpecifier_specifier:
-return OS << "NestedNameSpecifier_specifier";
-  case syntax::NodeRole::NestedNameSpecifier_delimiter:
-return OS << "NestedNameSpecifier_delimiter";
   case syntax::NodeRole::ParenExpression_subExpression:
 return OS << "ParenExpression_subExpression";
   }
   llvm_unreachable("invalid role");
 }
 
-std::vector syntax::NestedNameSpecifier::delimiters() {
-  std::vector Children;
-  for (auto *C = firstChild(); C; C = C->nextSibling()) {
-assert(C->role() == syntax::NodeRole::NestedNameSpecifier_delimiter);
-Children.push_back(llvm::cast(C));
+// We could have an interator in list to not pay memory costs of temporary
+// vector
+std::vector syntax::NestedNameSpecifier::specifiers() {
+  auto specifiersAsNodes = getElementsAsNodes();
+  std::vector Children;
+  for (const auto  : specifiersAsNodes) {
+Children.push_back(llvm::cast(element));
   }
   return Children;
 }
 
-std::vector syntax::NestedNameSpecifier::specifiers() {
-  std::vector Children;
-  for (auto *C = firstChild(); C; C = C->nextSibling()) {
-assert(C->role() == syntax::NodeRole::NestedNameSpecifier_specifier);
-Children.push_back(cast(C));
+std::vector>
+syntax::NestedNameSpecifier::specifiersAndDoubleColons() {
+  auto specifiersAsNodesAndDoubleColons = getElementsAsNodesAndDelimiters();
+  std::vector>
+  Children;
+  for (const auto  : specifiersAsNodesAndDoubleColons) {
+Children.push_back(
+{llvm::cast(specifierAndDoubleColon.element),
+ specifierAndDoubleColon.delimiter});
   }
   return Children;
 }
Index: clang/lib/Tooling/Syntax/BuildTree.cpp
===
--- clang/lib/Tooling/Syntax/BuildTree.cpp
+++ clang/lib/Tooling/Syntax/BuildTree.cpp
@@ -809,9 +809,8 @@
   if (!isa(NS))
 Builder.foldNode(Builder.getRange(getLocalSourceRange(it)).drop_back(),
  NS, it);
-  Builder.markChild(NS, syntax::NodeRole::NestedNameSpecifier_specifier);
-  Builder.markChildToken(it.getEndLoc(),
- syntax::NodeRole::NestedNameSpecifier_delimiter);
+  Builder.markChild(NS, syntax::NodeRole::List_element);
+  Builder.markChildToken(it.getEndLoc(), syntax::NodeRole::List_delimiter);
 }
 auto *NNS = new (allocator()) syntax::NestedNameSpecifier;
 Builder.foldNode(Builder.getRange(QualifierLoc.getSourceRange()), NNS,
Index: clang/include/clang/Tooling/Syntax/Nodes.h
===
--- 

[PATCH] D85440: [SyntaxTree] Implement `NNS` using the `List` base API

[Eduardo Caldas via Phabricator via cfe-commits]

eduucaldas updated this revision to Diff 284324.
eduucaldas added a comment.

Implement `canBeEmpty`, `getDelimiterToken`, `getTerminationKind` for 
`NestedNameSpecifier`


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D85440/new/

https://reviews.llvm.org/D85440

Files:
  clang/include/clang/Tooling/Syntax/Nodes.h
  clang/lib/Tooling/Syntax/BuildTree.cpp
  clang/lib/Tooling/Syntax/Nodes.cpp
  clang/lib/Tooling/Syntax/Tree.cpp

Index: clang/lib/Tooling/Syntax/Tree.cpp
===
--- clang/lib/Tooling/Syntax/Tree.cpp
+++ clang/lib/Tooling/Syntax/Tree.cpp
@@ -357,21 +357,38 @@
   return children;
 }
 
-// The methods below can't be implemented without information about the derived
-// list. These methods will be implemented by switching on the derived list's
-// `NodeKind`
-
 clang::tok::TokenKind syntax::List::getDelimiterTokenKind() {
-  llvm_unreachable("There are no subclasses of List, thus "
-   "getDelimiterTokenKind() cannot be called");
+  switch (this->kind()) {
+  case NodeKind::NestedNameSpecifier: {
+return clang::tok::coloncolon;
+  }
+  default: {
+llvm_unreachable("This is not a subclass of List, thus "
+ "getDelimiterTokenKind() cannot be called");
+  }
+  }
 }
 
 syntax::List::TerminationKind syntax::List::getTerminationKind() {
-  llvm_unreachable("There are no subclasses of List, thus getTerminationKind() "
-   "cannot be called");
+  switch (this->kind()) {
+  case NodeKind::NestedNameSpecifier: {
+return TerminationKind::Terminated;
+  }
+  default: {
+llvm_unreachable("This is not a subclass of List, thus "
+ "getTerminationKind() cannot be called");
+  }
+  }
 }
 
 bool syntax::List::canBeEmpty() {
-  llvm_unreachable(
-  "There are no subclasses of List, thus canBeEmpty() cannot be called");
+  switch (this->kind()) {
+  case NodeKind::NestedNameSpecifier: {
+return false;
+  }
+  default: {
+llvm_unreachable("This is not a subclass of List, thus canBeEmpty() "
+ "cannot be called");
+  }
+  }
 }
Index: clang/lib/Tooling/Syntax/Nodes.cpp
===
--- clang/lib/Tooling/Syntax/Nodes.cpp
+++ clang/lib/Tooling/Syntax/Nodes.cpp
@@ -200,30 +200,32 @@
 return OS << "IdExpression_id";
   case syntax::NodeRole::IdExpression_qualifier:
 return OS << "IdExpression_qualifier";
-  case syntax::NodeRole::NestedNameSpecifier_specifier:
-return OS << "NestedNameSpecifier_specifier";
-  case syntax::NodeRole::NestedNameSpecifier_delimiter:
-return OS << "NestedNameSpecifier_delimiter";
   case syntax::NodeRole::ParenExpression_subExpression:
 return OS << "ParenExpression_subExpression";
   }
   llvm_unreachable("invalid role");
 }
 
-std::vector syntax::NestedNameSpecifier::delimiters() {
-  std::vector Children;
-  for (auto *C = firstChild(); C; C = C->nextSibling()) {
-assert(C->role() == syntax::NodeRole::NestedNameSpecifier_delimiter);
-Children.push_back(llvm::cast(C));
+// We could have an interator in list to not pay memory costs of temporary
+// vector
+std::vector syntax::NestedNameSpecifier::specifiers() {
+  auto specifiersAsNodes = getElementsAsNodes();
+  std::vector Children;
+  for (const auto  : specifiersAsNodes) {
+Children.push_back(llvm::cast(element));
   }
   return Children;
 }
 
-std::vector syntax::NestedNameSpecifier::specifiers() {
-  std::vector Children;
-  for (auto *C = firstChild(); C; C = C->nextSibling()) {
-assert(C->role() == syntax::NodeRole::NestedNameSpecifier_specifier);
-Children.push_back(cast(C));
+std::vector>
+syntax::NestedNameSpecifier::specifiersAndDoubleColons() {
+  auto specifiersAsNodesAndDoubleColons = getElementsAsNodesAndDelimiters();
+  std::vector>
+  Children;
+  for (const auto  : specifiersAsNodesAndDoubleColons) {
+Children.push_back(
+{llvm::cast(specifierAndDoubleColon.element),
+ specifierAndDoubleColon.delimiter});
   }
   return Children;
 }
Index: clang/lib/Tooling/Syntax/BuildTree.cpp
===
--- clang/lib/Tooling/Syntax/BuildTree.cpp
+++ clang/lib/Tooling/Syntax/BuildTree.cpp
@@ -809,9 +809,8 @@
   if (!isa(NS))
 Builder.foldNode(Builder.getRange(getLocalSourceRange(it)).drop_back(),
  NS, it);
-  Builder.markChild(NS, syntax::NodeRole::NestedNameSpecifier_specifier);
-  Builder.markChildToken(it.getEndLoc(),
- syntax::NodeRole::NestedNameSpecifier_delimiter);
+  Builder.markChild(NS, syntax::NodeRole::List_element);
+  Builder.markChildToken(it.getEndLoc(), syntax::NodeRole::List_delimiter);
 }
 auto *NNS = new (allocator()) syntax::NestedNameSpecifier;
 Builder.foldNode(Builder.getRange(QualifierLoc.getSourceRange()), NNS,
Index: 

[PATCH] D85295: [SyntaxTree] Implement the List construct.

[Eduardo Caldas via Phabricator via cfe-commits]

This revision was landed with ongoing or failed builds.
This revision was automatically updated to reflect the committed changes.
Closed by commit rGa90c78ac5261: [SyntaxTree] Implement the List construct. 
(authored by eduucaldas).

Changed prior to commit:
  https://reviews.llvm.org/D85295?vs=284292=284304#toc

Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D85295/new/

https://reviews.llvm.org/D85295

Files:
  clang/include/clang/Tooling/Syntax/Nodes.h
  clang/include/clang/Tooling/Syntax/Tree.h
  clang/lib/Tooling/Syntax/Nodes.cpp
  clang/lib/Tooling/Syntax/Tree.cpp

Index: clang/lib/Tooling/Syntax/Tree.cpp
===
--- clang/lib/Tooling/Syntax/Tree.cpp
+++ clang/lib/Tooling/Syntax/Tree.cpp
@@ -268,3 +268,110 @@
   }
   return nullptr;
 }
+
+std::vector>
+syntax::List::getElementsAsNodesAndDelimiters() {
+  if (!firstChild())
+return {};
+
+  auto children = std::vector>();
+  syntax::Node *elementWithoutDelimiter = nullptr;
+  for (auto *C = firstChild(); C; C = C->nextSibling()) {
+switch (C->role()) {
+case syntax::NodeRole::List_element: {
+  if (elementWithoutDelimiter) {
+children.push_back({elementWithoutDelimiter, nullptr});
+  }
+  elementWithoutDelimiter = C;
+  break;
+}
+case syntax::NodeRole::List_delimiter: {
+  children.push_back({elementWithoutDelimiter, cast(C)});
+  elementWithoutDelimiter = nullptr;
+  break;
+}
+default:
+  llvm_unreachable(
+  "A list can have only elements and delimiters as children.");
+}
+  }
+
+  switch (getTerminationKind()) {
+  case syntax::List::TerminationKind::Separated: {
+children.push_back({elementWithoutDelimiter, nullptr});
+break;
+  }
+  case syntax::List::TerminationKind::Terminated:
+  case syntax::List::TerminationKind::MaybeTerminated: {
+if (elementWithoutDelimiter) {
+  children.push_back({elementWithoutDelimiter, nullptr});
+}
+break;
+  }
+  }
+
+  return children;
+}
+
+// Almost the same implementation of `getElementsAsNodesAndDelimiters` but
+// ignoring delimiters
+std::vector syntax::List::getElementsAsNodes() {
+  if (!firstChild())
+return {};
+
+  auto children = std::vector();
+  syntax::Node *elementWithoutDelimiter = nullptr;
+  for (auto *C = firstChild(); C; C = C->nextSibling()) {
+switch (C->role()) {
+case syntax::NodeRole::List_element: {
+  if (elementWithoutDelimiter) {
+children.push_back(elementWithoutDelimiter);
+  }
+  elementWithoutDelimiter = C;
+  break;
+}
+case syntax::NodeRole::List_delimiter: {
+  children.push_back(elementWithoutDelimiter);
+  elementWithoutDelimiter = nullptr;
+  break;
+}
+default:
+  llvm_unreachable("A list has only elements or delimiters.");
+}
+  }
+
+  switch (getTerminationKind()) {
+  case syntax::List::TerminationKind::Separated: {
+children.push_back(elementWithoutDelimiter);
+break;
+  }
+  case syntax::List::TerminationKind::Terminated:
+  case syntax::List::TerminationKind::MaybeTerminated: {
+if (elementWithoutDelimiter) {
+  children.push_back(elementWithoutDelimiter);
+}
+break;
+  }
+  }
+
+  return children;
+}
+
+// The methods below can't be implemented without information about the derived
+// list. These methods will be implemented by switching on the derived list's
+// `NodeKind`
+
+clang::tok::TokenKind syntax::List::getDelimiterTokenKind() {
+  llvm_unreachable("There are no subclasses of List, thus "
+   "getDelimiterTokenKind() cannot be called");
+}
+
+syntax::List::TerminationKind syntax::List::getTerminationKind() {
+  llvm_unreachable("There are no subclasses of List, thus getTerminationKind() "
+   "cannot be called");
+}
+
+bool syntax::List::canBeEmpty() {
+  llvm_unreachable(
+  "There are no subclasses of List, thus canBeEmpty() cannot be called");
+}
Index: clang/lib/Tooling/Syntax/Nodes.cpp
===
--- clang/lib/Tooling/Syntax/Nodes.cpp
+++ clang/lib/Tooling/Syntax/Nodes.cpp
@@ -152,6 +152,10 @@
 return OS << "TemplateKeyword";
   case syntax::NodeRole::BodyStatement:
 return OS << "BodyStatement";
+  case syntax::NodeRole::List_element:
+return OS << "List_element";
+  case syntax::NodeRole::List_delimiter:
+return OS << "List_delimiter";
   case syntax::NodeRole::CaseStatement_value:
 return OS << "CaseStatement_value";
   case syntax::NodeRole::IfStatement_thenStatement:
Index: clang/include/clang/Tooling/Syntax/Tree.h
===
--- clang/include/clang/Tooling/Syntax/Tree.h
+++ clang/include/clang/Tooling/Syntax/Tree.h
@@ -191,6 +191,59 @@
   Node *FirstChild = nullptr;
 };
 
+/// A list of Elements separated or terminated by a fixed token.
+///
+/// This type models the following 

[PATCH] D85295: [SyntaxTree] Implement the List construct.

[Eduardo Caldas via Phabricator via cfe-commits]

eduucaldas added inline comments.



Comment at: clang/include/clang/Tooling/Syntax/Tree.h:249
+
+  /// Return whether *under valid code* the list can be empty.
+  ///

gribozavr2 wrote:
> "Whether this list can be empty in syntactically and semantically correct 
> code.
> 
> This list may be empty when the source code has errors even if canBeEmpty() 
> returns true."
"...even if `canBeEmpty()` returns **false**"



Comment at: clang/lib/Tooling/Syntax/Tree.cpp:367
+clang::tok::TokenKind syntax::List::getDelimiterTokenKind() {
+  llvm_unreachable("A list can have only elements and delimiters as 
children.");
+}

gribozavr2 wrote:
> I think a better error message would say that there are no subclasses of 
> List, so this method can't be called.
Very sorry about that, this error was a copy-paste, I forgot to edit it


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D85295/new/

https://reviews.llvm.org/D85295

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D85295: [SyntaxTree] Implement the List construct.

[Eduardo Caldas via Phabricator via cfe-commits]

eduucaldas updated this revision to Diff 284292.
eduucaldas marked 3 inline comments as done.
eduucaldas added a comment.

Answer code-review


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D85295/new/

https://reviews.llvm.org/D85295

Files:
  clang/include/clang/Tooling/Syntax/Nodes.h
  clang/include/clang/Tooling/Syntax/Tree.h
  clang/lib/Tooling/Syntax/Nodes.cpp
  clang/lib/Tooling/Syntax/Tree.cpp

Index: clang/lib/Tooling/Syntax/Tree.cpp
===
--- clang/lib/Tooling/Syntax/Tree.cpp
+++ clang/lib/Tooling/Syntax/Tree.cpp
@@ -268,3 +268,110 @@
   }
   return nullptr;
 }
+
+std::vector>
+syntax::List::getElementsAsNodesAndDelimiters() {
+  if (!firstChild())
+return {};
+
+  auto children = std::vector>();
+  syntax::Node *elementWithoutDelimiter = nullptr;
+  for (auto *C = firstChild(); C; C = C->nextSibling()) {
+switch (C->role()) {
+case syntax::NodeRole::List_element: {
+  if (elementWithoutDelimiter) {
+children.push_back({elementWithoutDelimiter, nullptr});
+  }
+  elementWithoutDelimiter = C;
+  break;
+}
+case syntax::NodeRole::List_delimiter: {
+  children.push_back({elementWithoutDelimiter, cast(C)});
+  elementWithoutDelimiter = nullptr;
+  break;
+}
+default:
+  llvm_unreachable(
+  "A list can have only elements and delimiters as children.");
+}
+  }
+
+  switch (getTerminationKind()) {
+  case syntax::List::TerminationKind::Separated: {
+children.push_back({elementWithoutDelimiter, nullptr});
+break;
+  }
+  case syntax::List::TerminationKind::Terminated:
+  case syntax::List::TerminationKind::MaybeTerminated: {
+if (elementWithoutDelimiter) {
+  children.push_back({elementWithoutDelimiter, nullptr});
+}
+break;
+  }
+  }
+
+  return children;
+}
+
+// Almost the same implementation of `getElementsAsNodesAndDelimiters` but
+// ignoring delimiters
+std::vector syntax::List::getElementsAsNodes() {
+  if (!firstChild())
+return {};
+
+  auto children = std::vector();
+  syntax::Node *elementWithoutDelimiter = nullptr;
+  for (auto *C = firstChild(); C; C = C->nextSibling()) {
+switch (C->role()) {
+case syntax::NodeRole::List_element: {
+  if (elementWithoutDelimiter) {
+children.push_back(elementWithoutDelimiter);
+  }
+  elementWithoutDelimiter = C;
+  break;
+}
+case syntax::NodeRole::List_delimiter: {
+  children.push_back(elementWithoutDelimiter);
+  elementWithoutDelimiter = nullptr;
+  break;
+}
+default:
+  llvm_unreachable("A list has only elements or delimiters.");
+}
+  }
+
+  switch (getTerminationKind()) {
+  case syntax::List::TerminationKind::Separated: {
+children.push_back(elementWithoutDelimiter);
+break;
+  }
+  case syntax::List::TerminationKind::Terminated:
+  case syntax::List::TerminationKind::MaybeTerminated: {
+if (elementWithoutDelimiter) {
+  children.push_back(elementWithoutDelimiter);
+}
+break;
+  }
+  }
+
+  return children;
+}
+
+// The methods below can't be implemented without information about the derived
+// list. These methods will be implemented by switching on the derived list's
+// `NodeKind`
+
+clang::tok::TokenKind syntax::List::getDelimiterTokenKind() {
+  llvm_unreachable("There are no subclasses of List, thus "
+   "getDelimiterTokenKind() cannot be called");
+}
+
+syntax::List::TerminationKind syntax::List::getTerminationKind() {
+  llvm_unreachable("There are no subclasses of List, thus getTerminationKind() "
+   "cannot be called");
+}
+
+bool syntax::List::canBeEmpty() {
+  llvm_unreachable(
+  "There are no subclasses of List, thus canBeEmpty() cannot be called");
+}
Index: clang/lib/Tooling/Syntax/Nodes.cpp
===
--- clang/lib/Tooling/Syntax/Nodes.cpp
+++ clang/lib/Tooling/Syntax/Nodes.cpp
@@ -144,6 +144,10 @@
 return OS << "ExternKeyword";
   case syntax::NodeRole::BodyStatement:
 return OS << "BodyStatement";
+  case syntax::NodeRole::List_element:
+return OS << "List_element";
+  case syntax::NodeRole::List_delimiter:
+return OS << "List_delimiter";
   case syntax::NodeRole::CaseStatement_value:
 return OS << "CaseStatement_value";
   case syntax::NodeRole::IfStatement_thenStatement:
Index: clang/include/clang/Tooling/Syntax/Tree.h
===
--- clang/include/clang/Tooling/Syntax/Tree.h
+++ clang/include/clang/Tooling/Syntax/Tree.h
@@ -191,6 +191,59 @@
   Node *FirstChild = nullptr;
 };
 
+/// A list of Elements separated or terminated by a fixed token.
+///
+/// This type models the following grammar construct:
+/// delimited-list(element, delimiter, termination, canBeEmpty)
+class List : public Tree {
+public:
+  template  struct ElementAndDelimiter {
+

[PATCH] D84348: [SyntaxTree] Use simplified grammar rule for `NestedNameSpecifier` grammar nodes

[Eduardo Caldas via Phabricator via cfe-commits]

eduucaldas closed this revision.
eduucaldas added a comment.

commit hash 8abb5fb68f81b0e42d824bf080b1cef9a61559d6 



Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D84348/new/

https://reviews.llvm.org/D84348

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D85439: [SyntaxTree] Expand support for `NestedNameSpecifier`

[Eduardo Caldas via Phabricator via cfe-commits]

eduucaldas marked 2 inline comments as done.
eduucaldas added a comment.

Now NNS are based on inheritance!




Comment at: clang/lib/Tooling/Syntax/BuildTree.cpp:849-850
+  const auto TL = NNSLoc.getTypeLoc().castAs();
+  if (!RecursiveASTVisitor::TraverseDecltypeTypeLoc(TL))
+return nullptr;
+  auto *NS = new (allocator()) syntax::DecltypeNameSpecifier;

Since we are overriding the TraverseNestedNameSpecifierLoc that previously 
fired TraverseTypeLoc we fire it when building a NameSpecifier.



Comment at: clang/unittests/Tooling/Syntax/TreeTest.cpp:1235
 | | | | |   |-(
-| | | | |   |-IdExpression
-| | | | |   | `-UnqualifiedId
-| | | | |   |   `-s
+| | | | |   |-s
 | | | | |   `-)

eduucaldas wrote:
> standard `TraverseNestedNameSpecifierLoc` fired `TraverseTypeLoc`, once we 
> override it we lost this. This will be fixed when refining the Node for 
> `DecltypeSpecifier`
This is no longer an issue, because we now fire TraverseTypeLoc in the case of 
DecltypeNameSpecifier


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D85439/new/

https://reviews.llvm.org/D85439

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D85439: [SyntaxTree] Expand support for `NestedNameSpecifier`

[Eduardo Caldas via Phabricator via cfe-commits]

eduucaldas updated this revision to Diff 284285.
eduucaldas added a comment.

.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D85439/new/

https://reviews.llvm.org/D85439

Files:
  clang/include/clang/AST/NestedNameSpecifier.h
  clang/lib/Tooling/Syntax/BuildTree.cpp
  clang/unittests/Tooling/Syntax/TreeTest.cpp

Index: clang/unittests/Tooling/Syntax/TreeTest.cpp
===
--- clang/unittests/Tooling/Syntax/TreeTest.cpp
+++ clang/unittests/Tooling/Syntax/TreeTest.cpp
@@ -2973,7 +2973,8 @@
 `-UsingNamespaceDirective
   |-using
   |-namespace
-  |-::
+  |-NestedNameSpecifier
+  | `-::
   |-ns
   `-;
 )txt"));
@@ -3002,8 +3003,10 @@
 | `-}
 `-UsingDeclaration
   |-using
-  |-ns
-  |-::
+  |-NestedNameSpecifier
+  | |-IdentifierNameSpecifier
+  | | `-ns
+  | `-::
   |-a
   `-;
 )txt"));
@@ -3207,11 +3210,13 @@
   |->
   `-SimpleDeclaration
 |-struct
-|-X
-|-<
-|-T
-|->
-|-::
+|-NestedNameSpecifier
+| |-SimpleTemplateNameSpecifier
+| | |-X
+| | |-<
+| | |-T
+| | `->
+| `-::
 |-Y
 |-{
 |-}
@@ -3245,15 +3250,19 @@
 |-{
 |-UsingDeclaration
 | |-using
-| |-T
-| |-::
+| |-NestedNameSpecifier
+| | |-IdentifierNameSpecifier
+| | | `-T
+| | `-::
 | |-foo
 | `-;
 |-UsingDeclaration
 | |-using
 | |-typename
-| |-T
-| |-::
+| |-NestedNameSpecifier
+| | |-IdentifierNameSpecifier
+| | | `-T
+| | `-::
 | |-bar
 | `-;
 |-}
Index: clang/lib/Tooling/Syntax/BuildTree.cpp
===
--- clang/lib/Tooling/Syntax/BuildTree.cpp
+++ clang/lib/Tooling/Syntax/BuildTree.cpp
@@ -241,10 +241,24 @@
 assert(Added && "mapping added twice");
   }
 
+  void add(NestedNameSpecifierLoc From, syntax::Tree *To) {
+assert(To != nullptr);
+assert(From.hasQualifier());
+
+bool Added = NNSNodes.insert({From, To}).second;
+(void)Added;
+assert(Added && "mapping added twice");
+  }
+
   syntax::Tree *find(ASTPtr P) const { return Nodes.lookup(P); }
 
+  syntax::Tree *find(NestedNameSpecifierLoc P) const {
+return NNSNodes.lookup(P);
+  }
+
 private:
   llvm::DenseMap Nodes;
+  llvm::DenseMap NNSNodes;
 };
 } // namespace
 
@@ -281,16 +295,20 @@
 if (From)
   Mapping.add(From, New);
   }
+
   void foldNode(ArrayRef Range, syntax::Tree *New, TypeLoc L) {
 // FIXME: add mapping for TypeLocs
 foldNode(Range, New, nullptr);
   }
 
-  void foldNode(ArrayRef Range, syntax::Tree *New,
-NestedNameSpecifierLoc L) {
-// FIXME: add mapping for NestedNameSpecifierLoc
-foldNode(Range, New, nullptr);
+  void foldNode(llvm::ArrayRef Range, syntax::Tree *New,
+NestedNameSpecifierLoc From) {
+assert(New);
+Pending.foldChildren(Arena, Range, New);
+if (From)
+  Mapping.add(From, New);
   }
+
   /// Notifies that we should not consume trailing semicolon when computing
   /// token range of \p D.
   void noticeDeclWithoutSemicolon(Decl *D);
@@ -312,6 +330,8 @@
   void markChild(syntax::Node *N, NodeRole R);
   /// Set role for the syntax node matching \p N.
   void markChild(ASTPtr N, NodeRole R);
+  /// Set role for the syntax node matching \p N.
+  void markChild(NestedNameSpecifierLoc N, NodeRole R);
 
   /// Finish building the tree and consume the root node.
   syntax::TranslationUnit *finalize() && {
@@ -744,45 +764,18 @@
 return true;
   }
 
-  syntax::NameSpecifier *BuildNameSpecifier(const NestedNameSpecifier ) {
-switch (NNS.getKind()) {
-case NestedNameSpecifier::Global:
-  return new (allocator()) syntax::GlobalNameSpecifier;
-case NestedNameSpecifier::Namespace:
-case NestedNameSpecifier::NamespaceAlias:
-case NestedNameSpecifier::Identifier:
-  return new (allocator()) syntax::IdentifierNameSpecifier;
-case NestedNameSpecifier::TypeSpecWithTemplate:
-  return new (allocator()) syntax::SimpleTemplateNameSpecifier;
-case NestedNameSpecifier::TypeSpec: {
-  const auto *NNSType = NNS.getAsType();
-  assert(NNSType);
-  if (isa(NNSType))
-return new (allocator()) syntax::DecltypeNameSpecifier;
-  if (isa(
-  NNSType))
-return new (allocator()) syntax::SimpleTemplateNameSpecifier;
-  return new (allocator()) syntax::IdentifierNameSpecifier;
-}
-case NestedNameSpecifier::Super:
-  // FIXME: Support Microsoft's __super
-  llvm::report_fatal_error("We don't yet support the __super specifier",
-   true);
-}
-llvm_unreachable("Unhandled NestedNameSpecifier::SpecifierKind enum");
-  }
-
   // FIXME: Fix `NestedNameSpecifierLoc::getLocalSourceRange` for the
   // `DependentTemplateSpecializationType` case.
-  /// Given a nested-name-specifier return the range for the last name specifier
+  /// Given a 

[PATCH] D85439: [SyntaxTree] Expand support for `NestedNameSpecifier`

[Eduardo Caldas via Phabricator via cfe-commits]

eduucaldas updated this revision to Diff 284284.
eduucaldas added a comment.

Move to using inheritance


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D85439/new/

https://reviews.llvm.org/D85439

Files:
  clang/include/clang/AST/NestedNameSpecifier.h
  clang/lib/Tooling/Syntax/BuildTree.cpp
  clang/unittests/Tooling/Syntax/TreeTest.cpp

Index: clang/unittests/Tooling/Syntax/TreeTest.cpp
===
--- clang/unittests/Tooling/Syntax/TreeTest.cpp
+++ clang/unittests/Tooling/Syntax/TreeTest.cpp
@@ -2973,7 +2973,8 @@
 `-UsingNamespaceDirective
   |-using
   |-namespace
-  |-::
+  |-NestedNameSpecifier
+  | `-::
   |-ns
   `-;
 )txt"));
@@ -3002,8 +3003,10 @@
 | `-}
 `-UsingDeclaration
   |-using
-  |-ns
-  |-::
+  |-NestedNameSpecifier
+  | |-IdentifierNameSpecifier
+  | | `-ns
+  | `-::
   |-a
   `-;
 )txt"));
@@ -3207,11 +3210,13 @@
   |->
   `-SimpleDeclaration
 |-struct
-|-X
-|-<
-|-T
-|->
-|-::
+|-NestedNameSpecifier
+| |-SimpleTemplateNameSpecifier
+| | |-X
+| | |-<
+| | |-T
+| | `->
+| `-::
 |-Y
 |-{
 |-}
@@ -3245,15 +3250,19 @@
 |-{
 |-UsingDeclaration
 | |-using
-| |-T
-| |-::
+| |-NestedNameSpecifier
+| | |-IdentifierNameSpecifier
+| | | `-T
+| | `-::
 | |-foo
 | `-;
 |-UsingDeclaration
 | |-using
 | |-typename
-| |-T
-| |-::
+| |-NestedNameSpecifier
+| | |-IdentifierNameSpecifier
+| | | `-T
+| | `-::
 | |-bar
 | `-;
 |-}
Index: clang/lib/Tooling/Syntax/BuildTree.cpp
===
--- clang/lib/Tooling/Syntax/BuildTree.cpp
+++ clang/lib/Tooling/Syntax/BuildTree.cpp
@@ -241,10 +241,24 @@
 assert(Added && "mapping added twice");
   }
 
+  void add(NestedNameSpecifierLoc From, syntax::Tree *To) {
+assert(To != nullptr);
+assert(From.hasQualifier());
+
+bool Added = NNSNodes.insert({From, To}).second;
+(void)Added;
+assert(Added && "mapping added twice");
+  }
+
   syntax::Tree *find(ASTPtr P) const { return Nodes.lookup(P); }
 
+  syntax::Tree *find(NestedNameSpecifierLoc P) const {
+return NNSNodes.lookup(P);
+  }
+
 private:
   llvm::DenseMap Nodes;
+  llvm::DenseMap NNSNodes;
 };
 } // namespace
 
@@ -281,16 +295,20 @@
 if (From)
   Mapping.add(From, New);
   }
+
   void foldNode(ArrayRef Range, syntax::Tree *New, TypeLoc L) {
 // FIXME: add mapping for TypeLocs
 foldNode(Range, New, nullptr);
   }
 
-  void foldNode(ArrayRef Range, syntax::Tree *New,
-NestedNameSpecifierLoc L) {
-// FIXME: add mapping for NestedNameSpecifierLoc
-foldNode(Range, New, nullptr);
+  void foldNode(llvm::ArrayRef Range, syntax::Tree *New,
+NestedNameSpecifierLoc From) {
+assert(New);
+Pending.foldChildren(Arena, Range, New);
+if (From)
+  Mapping.add(From, New);
   }
+
   /// Notifies that we should not consume trailing semicolon when computing
   /// token range of \p D.
   void noticeDeclWithoutSemicolon(Decl *D);
@@ -312,6 +330,8 @@
   void markChild(syntax::Node *N, NodeRole R);
   /// Set role for the syntax node matching \p N.
   void markChild(ASTPtr N, NodeRole R);
+  /// Set role for the syntax node matching \p N.
+  void markChild(NestedNameSpecifierLoc N, NodeRole R);
 
   /// Finish building the tree and consume the root node.
   syntax::TranslationUnit *finalize() && {
@@ -798,31 +818,31 @@
 return SR;
   }
 
-  syntax::NestedNameSpecifier *
-  BuildNestedNameSpecifier(const NestedNameSpecifierLoc ) {
+  // To build syntax tree nodes for NestedNameSpecifierLoc we override Traverse
+  // instead of WalkUpFrom because we want to traverse the children ourselves
+  // and build a list instead of a nested tree of name specifier prefixes.
+  bool TraverseNestedNameSpecifierLoc(NestedNameSpecifierLoc QualifierLoc) {
 if (!QualifierLoc)
-  return nullptr;
+  return true;
 for (auto it = QualifierLoc; it; it = it.getPrefix()) {
-  assert(it.hasQualifier());
   auto *NS = BuildNameSpecifier(*it.getNestedNameSpecifier());
   assert(NS);
   if (!isa(NS))
 Builder.foldNode(Builder.getRange(getLocalSourceRange(it)).drop_back(),
- NS, it);
+ NS, nullptr);
   Builder.markChild(NS, syntax::NodeRole::NestedNameSpecifier_specifier);
   Builder.markChildToken(it.getEndLoc(),
  syntax::NodeRole::NestedNameSpecifier_delimiter);
 }
-auto *NNS = new (allocator()) syntax::NestedNameSpecifier;
-Builder.foldNode(Builder.getRange(QualifierLoc.getSourceRange()), NNS,
+Builder.foldNode(Builder.getRange(QualifierLoc.getSourceRange()),
+ new (allocator()) syntax::NestedNameSpecifier,
  QualifierLoc);
-return NNS;
+return true;
   }
 
   bool 

[PATCH] D84348: WIP: Add complete id-expression support to syntax trees

[Eduardo Caldas via Phabricator via cfe-commits]

eduucaldas updated this revision to Diff 283989.
eduucaldas added a comment.

Last version sent upstream


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D84348/new/

https://reviews.llvm.org/D84348

Files:
  clang/include/clang/Tooling/Syntax/Nodes.h
  clang/lib/Tooling/Syntax/BuildTree.cpp
  clang/lib/Tooling/Syntax/Nodes.cpp
  clang/unittests/Tooling/Syntax/TreeTest.cpp

Index: clang/unittests/Tooling/Syntax/TreeTest.cpp
===
--- clang/unittests/Tooling/Syntax/TreeTest.cpp
+++ clang/unittests/Tooling/Syntax/TreeTest.cpp
@@ -873,24 +873,47 @@
   }
   EXPECT_TRUE(treeDumpEqual(
   R"cpp(
-namespace a {
+namespace n {
   struct S {
 template
-static T f(){}
+struct ST {
+  static void f();
+};
   };
 }
+template
+struct ST {
+  struct S {
+template
+static U f();
+  };
+};
 void test() {
-  ::  // global-namespace-specifier
-  a:: // namespace-specifier
-  S:: // type-name-specifier
+  :: // global-namespace-specifier
+  n::// namespace-specifier
+  S::// type-name-specifier
+  template ST:: // type-template-instantiation-specifier
+  f();
+
+  n::// namespace-specifier
+  S::// type-name-specifier
+  ST::  // type-template-instantiation-specifier
+  f();
+
+  ST:: // type-name-specifier
+  S::   // type-name-specifier
   f();
+
+  ST:: // type-name-specifier
+  S::   // type-name-specifier
+  template f();
 }
 )cpp",
   R"txt(
 *: TranslationUnit
 |-NamespaceDefinition
 | |-namespace
-| |-a
+| |-n
 | |-{
 | |-SimpleDeclaration
 | | |-struct
@@ -904,19 +927,58 @@
 | | | | `-T
 | | | |->
 | | | `-SimpleDeclaration
-| | |   |-static
-| | |   |-T
-| | |   |-SimpleDeclarator
-| | |   | |-f
-| | |   | `-ParametersAndQualifiers
-| | |   |   |-(
-| | |   |   `-)
-| | |   `-CompoundStatement
-| | | |-{
-| | | `-}
+| | |   |-struct
+| | |   |-ST
+| | |   |-{
+| | |   |-SimpleDeclaration
+| | |   | |-static
+| | |   | |-void
+| | |   | |-SimpleDeclarator
+| | |   | | |-f
+| | |   | | `-ParametersAndQualifiers
+| | |   | |   |-(
+| | |   | |   `-)
+| | |   | `-;
+| | |   |-}
+| | |   `-;
 | | |-}
 | | `-;
 | `-}
+|-TemplateDeclaration
+| |-template
+| |-<
+| |-UnknownDeclaration
+| | |-typename
+| | `-T
+| |->
+| `-SimpleDeclaration
+|   |-struct
+|   |-ST
+|   |-{
+|   |-SimpleDeclaration
+|   | |-struct
+|   | |-S
+|   | |-{
+|   | |-TemplateDeclaration
+|   | | |-template
+|   | | |-<
+|   | | |-UnknownDeclaration
+|   | | | |-typename
+|   | | | `-U
+|   | | |->
+|   | | `-SimpleDeclaration
+|   | |   |-static
+|   | |   |-U
+|   | |   |-SimpleDeclarator
+|   | |   | |-f
+|   | |   | `-ParametersAndQualifiers
+|   | |   |   |-(
+|   | |   |   `-)
+|   | |   `-;
+|   | |-}
+|   | `-;
+|   |-}
+|   `-;
 `-SimpleDeclaration
   |-void
   |-SimpleDeclarator
@@ -930,14 +992,81 @@
 | |-UnknownExpression
 | | |-IdExpression
 | | | |-NestedNameSpecifier
-| | | | |-NameSpecifier
-| | | | | `-::
-| | | | |-NameSpecifier
-| | | | | |-a
-| | | | | `-::
-| | | | `-NameSpecifier
-| | | |   |-S
-| | | |   `-::
+| | | | |-::
+| | | | |-IdentifierNameSpecifier
+| | | | | `-n
+| | | | |-::
+| | | | |-IdentifierNameSpecifier
+| | | | | `-S
+| | | | |-::
+| | | | |-SimpleTemplateNameSpecifier
+| | | | | |-template
+| | | | | |-ST
+| | | | | |-<
+| | | | | |-int
+| | | | | `->
+| | | | `-::
+| | | `-UnqualifiedId
+| | |   `-f
+| | |-(
+| | `-)
+| `-;
+|-ExpressionStatement
+| |-UnknownExpression
+| | |-IdExpression
+| | | |-NestedNameSpecifier
+| | | | |-IdentifierNameSpecifier
+| | | | | `-n
+| | | | |-::
+| | | | |-IdentifierNameSpecifier
+| | | | | `-S
+| | | | |-::
+| | | | |-SimpleTemplateNameSpecifier
+| | | | | |-ST
+| | | | | |-<
+| | | | | |-int
+| | | | | `->
+| | | | `-::
+| | | `-UnqualifiedId
+| | |   `-f
+| | |-(
+| | `-)
+| `-;
+|-ExpressionStatement
+| |-UnknownExpression
+| | |-IdExpression
+| | | |-NestedNameSpecifier
+| | | | |-SimpleTemplateNameSpecifier
+| | | | | |-ST
+| | | | | |-<
+| | | | | |-int
+| | | | | `->
+| | | | |-::
+| | | | |-IdentifierNameSpecifier
+| | | | | `-S
+| | | | `-::
+| | | `-UnqualifiedId
+| | |   |-f
+| | |   |-<
+| | |   |-int
+| | |   `->
+| | |-(
+| | `-)
+| `-;
+|-ExpressionStatement
+| |-UnknownExpression
+| | |-IdExpression
+| | | |-NestedNameSpecifier
+| | | | |-SimpleTemplateNameSpecifier
+| | | | | |-ST
+| | | | | |-<
+| | | | | |-int
+| | | | | `->
+| | | | |-::
+| | | | |-IdentifierNameSpecifier
+| | | | | `-S
+| | | | `-::
+| | 

[PATCH] D84781: [SyntaxTree] Use PointerUnion instead of inheritance for alternative clauses in NNS

[Eduardo Caldas via Phabricator via cfe-commits]

eduucaldas updated this revision to Diff 283876.
eduucaldas added a comment.

Remove Fix crash on name specifier commit from patch


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D84781/new/

https://reviews.llvm.org/D84781

Files:
  clang/include/clang/Tooling/Syntax/Nodes.h
  clang/include/clang/Tooling/Syntax/Tree.h
  clang/lib/Tooling/Syntax/BuildTree.cpp
  clang/lib/Tooling/Syntax/Nodes.cpp
  clang/lib/Tooling/Syntax/Tree.cpp
  clang/unittests/Tooling/Syntax/TreeTest.cpp

Index: clang/unittests/Tooling/Syntax/TreeTest.cpp
===
--- clang/unittests/Tooling/Syntax/TreeTest.cpp
+++ clang/unittests/Tooling/Syntax/TreeTest.cpp
@@ -993,18 +993,19 @@
 | | |-IdExpression
 | | | |-NestedNameSpecifier
 | | | | |-::
-| | | | |-IdentifierNameSpecifier
+| | | | |-NameSpecifier
 | | | | | `-n
 | | | | |-::
-| | | | |-IdentifierNameSpecifier
+| | | | |-NameSpecifier
 | | | | | `-S
 | | | | |-::
-| | | | |-SimpleTemplateNameSpecifier
-| | | | | |-template
-| | | | | |-ST
-| | | | | |-<
-| | | | | |-int
-| | | | | `->
+| | | | |-NameSpecifier
+| | | | | `-SimpleTemplateSpecifier
+| | | | |   |-template
+| | | | |   |-ST
+| | | | |   |-<
+| | | | |   |-int
+| | | | |   `->
 | | | | `-::
 | | | `-UnqualifiedId
 | | |   `-f
@@ -1015,17 +1016,18 @@
 | |-UnknownExpression
 | | |-IdExpression
 | | | |-NestedNameSpecifier
-| | | | |-IdentifierNameSpecifier
+| | | | |-NameSpecifier
 | | | | | `-n
 | | | | |-::
-| | | | |-IdentifierNameSpecifier
+| | | | |-NameSpecifier
 | | | | | `-S
 | | | | |-::
-| | | | |-SimpleTemplateNameSpecifier
-| | | | | |-ST
-| | | | | |-<
-| | | | | |-int
-| | | | | `->
+| | | | |-NameSpecifier
+| | | | | `-SimpleTemplateSpecifier
+| | | | |   |-ST
+| | | | |   |-<
+| | | | |   |-int
+| | | | |   `->
 | | | | `-::
 | | | `-UnqualifiedId
 | | |   `-f
@@ -1036,13 +1038,14 @@
 | |-UnknownExpression
 | | |-IdExpression
 | | | |-NestedNameSpecifier
-| | | | |-SimpleTemplateNameSpecifier
-| | | | | |-ST
-| | | | | |-<
-| | | | | |-int
-| | | | | `->
+| | | | |-NameSpecifier
+| | | | | `-SimpleTemplateSpecifier
+| | | | |   |-ST
+| | | | |   |-<
+| | | | |   |-int
+| | | | |   `->
 | | | | |-::
-| | | | |-IdentifierNameSpecifier
+| | | | |-NameSpecifier
 | | | | | `-S
 | | | | `-::
 | | | `-UnqualifiedId
@@ -1057,13 +1060,14 @@
 | |-UnknownExpression
 | | |-IdExpression
 | | | |-NestedNameSpecifier
-| | | | |-SimpleTemplateNameSpecifier
-| | | | | |-ST
-| | | | | |-<
-| | | | | |-int
-| | | | | `->
+| | | | |-NameSpecifier
+| | | | | `-SimpleTemplateSpecifier
+| | | | |   |-ST
+| | | | |   |-<
+| | | | |   |-int
+| | | | |   `->
 | | | | |-::
-| | | | |-IdentifierNameSpecifier
+| | | | |-NameSpecifier
 | | | | | `-S
 | | | | `-::
 | | | |-template
@@ -1121,15 +1125,16 @@
   | |-UnknownExpression
   | | |-IdExpression
   | | | |-NestedNameSpecifier
-  | | | | |-IdentifierNameSpecifier
+  | | | | |-NameSpecifier
   | | | | | `-T
   | | | | |-::
-  | | | | |-SimpleTemplateNameSpecifier
-  | | | | | |-template
-  | | | | | |-U
-  | | | | | |-<
-  | | | | | |-int
-  | | | | | `->
+  | | | | |-NameSpecifier
+  | | | | | `-SimpleTemplateSpecifier
+  | | | | |   |-template
+  | | | | |   |-U
+  | | | | |   |-<
+  | | | | |   |-int
+  | | | | |   `->
   | | | | `-::
   | | | `-UnqualifiedId
   | | |   `-f
@@ -1140,10 +1145,10 @@
   | |-UnknownExpression
   | | |-IdExpression
   | | | |-NestedNameSpecifier
-  | | | | |-IdentifierNameSpecifier
+  | | | | |-NameSpecifier
   | | | | | `-T
   | | | | |-::
-  | | | | |-IdentifierNameSpecifier
+  | | | | |-NameSpecifier
   | | | | | `-U
   | | | | `-::
   | | | `-UnqualifiedId
@@ -1155,7 +1160,7 @@
   | |-UnknownExpression
   | | |-IdExpression
   | | | |-NestedNameSpecifier
-  | | | | |-IdentifierNameSpecifier
+  | | | | |-NameSpecifier
   | | | | | `-T
   | | | | `-::
   | | | |-template
@@ -1222,13 +1227,14 @@
 | |-UnknownExpression
 | | |-IdExpression
 | | | |-NestedNameSpecifier
-| | | | |-DecltypeNameSpecifier
-| | | | | |-decltype
-| | | | | |-(
-| | | | | |-IdExpression
-| | | | | | `-UnqualifiedId
-| | | | | |   `-s
-| | | | | `-)
+| | | | |-NameSpecifier
+| | | | | `-DecltypeSpecifier
+| | | | |   |-decltype
+| | | | |   |-(
+| | | | |   |-IdExpression
+| | | | |   | `-UnqualifiedId
+| | | | |   |   `-s
+| | | | |   `-)
 | | | | `-::
 | | | 

[PATCH] D84348: WIP: Add complete id-expression support to syntax trees

[Eduardo Caldas via Phabricator via cfe-commits]

eduucaldas updated this revision to Diff 283875.
eduucaldas added a comment.

rebase to add this commit from a further patch

- [SyntaxTree] Fix crash on name specifier.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D84348/new/

https://reviews.llvm.org/D84348

Files:
  clang/include/clang/Tooling/Syntax/Nodes.h
  clang/lib/Tooling/Syntax/BuildTree.cpp
  clang/lib/Tooling/Syntax/Nodes.cpp
  clang/unittests/Tooling/Syntax/TreeTest.cpp

Index: clang/unittests/Tooling/Syntax/TreeTest.cpp
===
--- clang/unittests/Tooling/Syntax/TreeTest.cpp
+++ clang/unittests/Tooling/Syntax/TreeTest.cpp
@@ -873,24 +873,47 @@
   }
   EXPECT_TRUE(treeDumpEqual(
   R"cpp(
-namespace a {
+namespace n {
   struct S {
 template
-static T f(){}
+struct ST {
+  static void f();
+};
   };
 }
+template
+struct ST {
+  struct S {
+template
+static U f();
+  };
+};
 void test() {
-  ::  // global-namespace-specifier
-  a:: // namespace-specifier
-  S:: // type-name-specifier
+  :: // global-namespace-specifier
+  n::// namespace-specifier
+  S::// type-name-specifier
+  template ST:: // type-template-instantiation-specifier
+  f();
+
+  n::// namespace-specifier
+  S::// type-name-specifier
+  ST::  // type-template-instantiation-specifier
+  f();
+
+  ST:: // type-name-specifier
+  S::   // type-name-specifier
   f();
+
+  ST:: // type-name-specifier
+  S::   // type-name-specifier
+  template f();
 }
 )cpp",
   R"txt(
 *: TranslationUnit
 |-NamespaceDefinition
 | |-namespace
-| |-a
+| |-n
 | |-{
 | |-SimpleDeclaration
 | | |-struct
@@ -904,19 +927,58 @@
 | | | | `-T
 | | | |->
 | | | `-SimpleDeclaration
-| | |   |-static
-| | |   |-T
-| | |   |-SimpleDeclarator
-| | |   | |-f
-| | |   | `-ParametersAndQualifiers
-| | |   |   |-(
-| | |   |   `-)
-| | |   `-CompoundStatement
-| | | |-{
-| | | `-}
+| | |   |-struct
+| | |   |-ST
+| | |   |-{
+| | |   |-SimpleDeclaration
+| | |   | |-static
+| | |   | |-void
+| | |   | |-SimpleDeclarator
+| | |   | | |-f
+| | |   | | `-ParametersAndQualifiers
+| | |   | |   |-(
+| | |   | |   `-)
+| | |   | `-;
+| | |   |-}
+| | |   `-;
 | | |-}
 | | `-;
 | `-}
+|-TemplateDeclaration
+| |-template
+| |-<
+| |-UnknownDeclaration
+| | |-typename
+| | `-T
+| |->
+| `-SimpleDeclaration
+|   |-struct
+|   |-ST
+|   |-{
+|   |-SimpleDeclaration
+|   | |-struct
+|   | |-S
+|   | |-{
+|   | |-TemplateDeclaration
+|   | | |-template
+|   | | |-<
+|   | | |-UnknownDeclaration
+|   | | | |-typename
+|   | | | `-U
+|   | | |->
+|   | | `-SimpleDeclaration
+|   | |   |-static
+|   | |   |-U
+|   | |   |-SimpleDeclarator
+|   | |   | |-f
+|   | |   | `-ParametersAndQualifiers
+|   | |   |   |-(
+|   | |   |   `-)
+|   | |   `-;
+|   | |-}
+|   | `-;
+|   |-}
+|   `-;
 `-SimpleDeclaration
   |-void
   |-SimpleDeclarator
@@ -930,14 +992,81 @@
 | |-UnknownExpression
 | | |-IdExpression
 | | | |-NestedNameSpecifier
-| | | | |-NameSpecifier
-| | | | | `-::
-| | | | |-NameSpecifier
-| | | | | |-a
-| | | | | `-::
-| | | | `-NameSpecifier
-| | | |   |-S
-| | | |   `-::
+| | | | |-::
+| | | | |-IdentifierNameSpecifier
+| | | | | `-n
+| | | | |-::
+| | | | |-IdentifierNameSpecifier
+| | | | | `-S
+| | | | |-::
+| | | | |-SimpleTemplateNameSpecifier
+| | | | | |-template
+| | | | | |-ST
+| | | | | |-<
+| | | | | |-int
+| | | | | `->
+| | | | `-::
+| | | `-UnqualifiedId
+| | |   `-f
+| | |-(
+| | `-)
+| `-;
+|-ExpressionStatement
+| |-UnknownExpression
+| | |-IdExpression
+| | | |-NestedNameSpecifier
+| | | | |-IdentifierNameSpecifier
+| | | | | `-n
+| | | | |-::
+| | | | |-IdentifierNameSpecifier
+| | | | | `-S
+| | | | |-::
+| | | | |-SimpleTemplateNameSpecifier
+| | | | | |-ST
+| | | | | |-<
+| | | | | |-int
+| | | | | `->
+| | | | `-::
+| | | `-UnqualifiedId
+| | |   `-f
+| | |-(
+| | `-)
+| `-;
+|-ExpressionStatement
+| |-UnknownExpression
+| | |-IdExpression
+| | | |-NestedNameSpecifier
+| | | | |-SimpleTemplateNameSpecifier
+| | | | | |-ST
+| | | | | |-<
+| | | | | |-int
+| | | | | `->
+| | | | |-::
+| | | | |-IdentifierNameSpecifier
+| | | | | `-S
+| | | | `-::
+| | | `-UnqualifiedId
+| | |   |-f
+| | |   |-<
+| | |   |-int
+| | |   `->
+| | |-(
+| | `-)
+| `-;
+|-ExpressionStatement
+| |-UnknownExpression
+| | |-IdExpression
+| | | |-NestedNameSpecifier
+| | | | |-SimpleTemplateNameSpecifier
+| | | | | |-ST
+| | | | | |-<
+| | | | | |-int
+| | | | | `->
+| | | | |-::
+| | | | 

[PATCH] D85427: [SyntaxTree][NFC] remove redundant namespace-specifiers

[Eduardo Caldas via Phabricator via cfe-commits]

This revision was landed with ongoing or failed builds.
This revision was automatically updated to reflect the committed changes.
Closed by commit rGba41a0f7339c: [SyntaxTree][NFC] remove redundant 
namespace-specifiers (authored by eduucaldas).

Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D85427/new/

https://reviews.llvm.org/D85427

Files:
  clang/include/clang/Tooling/Syntax/Nodes.h
  clang/include/clang/Tooling/Syntax/Tree.h
  clang/lib/Tooling/Syntax/BuildTree.cpp
  clang/lib/Tooling/Syntax/Nodes.cpp
  clang/lib/Tooling/Syntax/Tree.cpp
  clang/unittests/Tooling/Syntax/TreeTest.cpp

Index: clang/unittests/Tooling/Syntax/TreeTest.cpp
===
--- clang/unittests/Tooling/Syntax/TreeTest.cpp
+++ clang/unittests/Tooling/Syntax/TreeTest.cpp
@@ -40,7 +40,7 @@
 using namespace clang;
 
 namespace {
-static llvm::ArrayRef tokens(syntax::Node *N) {
+static ArrayRef tokens(syntax::Node *N) {
   assert(N->isOriginal() && "tokens of modified nodes are not well-defined");
   if (auto *L = dyn_cast(N))
 return llvm::makeArrayRef(L->token(), 1);
@@ -53,7 +53,7 @@
public ::testing::WithParamInterface {
 protected:
   // Build a syntax tree for the code.
-  syntax::TranslationUnit *buildTree(llvm::StringRef Code,
+  syntax::TranslationUnit *buildTree(StringRef Code,
  const TestClangConfig ) {
 // FIXME: this code is almost the identical to the one in TokensTest. Share
 //it.
@@ -169,7 +169,7 @@
   }
 
   // Adds a file to the test VFS.
-  void addFile(llvm::StringRef Path, llvm::StringRef Contents) {
+  void addFile(StringRef Path, StringRef Contents) {
 if (!FS->addFile(Path, time_t(),
  llvm::MemoryBuffer::getMemBufferCopy(Contents))) {
   ADD_FAILURE() << "could not add a file to VFS: " << Path;
@@ -179,7 +179,7 @@
   /// Finds the deepest node in the tree that covers exactly \p R.
   /// FIXME: implement this efficiently and move to public syntax tree API.
   syntax::Node *nodeByRange(llvm::Annotations::Range R, syntax::Node *Root) {
-llvm::ArrayRef Toks = tokens(Root);
+ArrayRef Toks = tokens(Root);
 
 if (Toks.front().location().isFileID() &&
 Toks.back().location().isFileID() &&
@@ -198,15 +198,14 @@
   }
 
   // Data fields.
-  llvm::IntrusiveRefCntPtr DiagOpts =
-  new DiagnosticOptions();
-  llvm::IntrusiveRefCntPtr Diags =
+  IntrusiveRefCntPtr DiagOpts = new DiagnosticOptions();
+  IntrusiveRefCntPtr Diags =
   new DiagnosticsEngine(new DiagnosticIDs, DiagOpts.get());
   IntrusiveRefCntPtr FS =
   new llvm::vfs::InMemoryFileSystem;
-  llvm::IntrusiveRefCntPtr FileMgr =
+  IntrusiveRefCntPtr FileMgr =
   new FileManager(FileSystemOptions(), FS);
-  llvm::IntrusiveRefCntPtr SourceMgr =
+  IntrusiveRefCntPtr SourceMgr =
   new SourceManager(*Diags, *FileMgr);
   std::shared_ptr Invocation;
   // Set after calling buildTree().
Index: clang/lib/Tooling/Syntax/Tree.cpp
===
--- clang/lib/Tooling/Syntax/Tree.cpp
+++ clang/lib/Tooling/Syntax/Tree.cpp
@@ -36,11 +36,9 @@
  const TokenBuffer )
 : SourceMgr(SourceMgr), LangOpts(LangOpts), Tokens(Tokens) {}
 
-const clang::syntax::TokenBuffer ::Arena::tokenBuffer() const {
-  return Tokens;
-}
+const syntax::TokenBuffer ::Arena::tokenBuffer() const { return Tokens; }
 
-std::pair>
+std::pair>
 syntax::Arena::lexBuffer(std::unique_ptr Input) {
   auto FID = SourceMgr.createFileID(std::move(Input));
   auto It = ExtraTokens.try_emplace(FID, tokenize(FID, SourceMgr, LangOpts));
@@ -135,7 +133,7 @@
 }
 
 namespace {
-static void dumpTokens(llvm::raw_ostream , ArrayRef Tokens,
+static void dumpTokens(raw_ostream , ArrayRef Tokens,
const SourceManager ) {
   assert(!Tokens.empty());
   bool First = true;
@@ -153,7 +151,7 @@
   }
 }
 
-static void dumpTree(llvm::raw_ostream , const syntax::Node *N,
+static void dumpTree(raw_ostream , const syntax::Node *N,
  const syntax::Arena , std::vector IndentMask) {
   std::string Marks;
   if (!N->isOriginal())
@@ -165,13 +163,13 @@
   if (!Marks.empty())
 OS << Marks << ": ";
 
-  if (auto *L = llvm::dyn_cast(N)) {
+  if (auto *L = dyn_cast(N)) {
 dumpTokens(OS, *L->token(), A.sourceManager());
 OS << "\n";
 return;
   }
 
-  auto *T = llvm::cast(N);
+  auto *T = cast(N);
   OS << T->kind() << "\n";
 
   for (auto It = T->firstChild(); It != nullptr; It = It->nextSibling()) {
@@ -205,7 +203,7 @@
   std::string Storage;
   llvm::raw_string_ostream OS(Storage);
   traverse(this, [&](const syntax::Node *N) {
-auto *L = llvm::dyn_cast(N);
+auto *L = dyn_cast(N);
 if (!L)
   return;
 ::dumpTokens(OS, *L->token(), A.sourceManager());
Index: clang/lib/Tooling/Syntax/Nodes.cpp

[PATCH] D84348: WIP: Add complete id-expression support to syntax trees

[Eduardo Caldas via Phabricator via cfe-commits]

eduucaldas added inline comments.



Comment at: clang/lib/Tooling/Syntax/BuildTree.cpp:789
+// Remove "::" from the `SourceRange`
+SR.setEnd(SR.getEnd().getLocWithOffset(-1));
+

Newbie mistake. Corrected in latter commit


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D84348/new/

https://reviews.llvm.org/D84348

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D85295: [SyntaxTree] Implement the List construct.

[Eduardo Caldas via Phabricator via cfe-commits]

eduucaldas added inline comments.



Comment at: clang/include/clang/Tooling/Syntax/Tree.h:247
+
+  TerminationKind getTerminationKind();
+

gribozavr2 wrote:
> I just realized that the rest of the syntax tree API does not use the `get~` 
> prefix. However, most of Clang does, as far as I know. Which way should we 
> repaint?
Let's go with Clang! I can make this change in another patch


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D85295/new/

https://reviews.llvm.org/D85295

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D85427: [SyntaxTree][NFC] remove redundant namespace-specifiers

[Eduardo Caldas via Phabricator via cfe-commits]

eduucaldas updated this revision to Diff 283659.
eduucaldas added a comment.

removed namespace specifiers `llvm::`


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D85427/new/

https://reviews.llvm.org/D85427

Files:
  clang/include/clang/Tooling/Syntax/Nodes.h
  clang/include/clang/Tooling/Syntax/Tree.h
  clang/lib/Tooling/Syntax/BuildTree.cpp
  clang/lib/Tooling/Syntax/Nodes.cpp
  clang/lib/Tooling/Syntax/Tree.cpp
  clang/unittests/Tooling/Syntax/TreeTest.cpp

Index: clang/unittests/Tooling/Syntax/TreeTest.cpp
===
--- clang/unittests/Tooling/Syntax/TreeTest.cpp
+++ clang/unittests/Tooling/Syntax/TreeTest.cpp
@@ -40,7 +40,7 @@
 using namespace clang;
 
 namespace {
-static llvm::ArrayRef tokens(syntax::Node *N) {
+static ArrayRef tokens(syntax::Node *N) {
   assert(N->isOriginal() && "tokens of modified nodes are not well-defined");
   if (auto *L = dyn_cast(N))
 return llvm::makeArrayRef(L->token(), 1);
@@ -53,7 +53,7 @@
public ::testing::WithParamInterface {
 protected:
   // Build a syntax tree for the code.
-  syntax::TranslationUnit *buildTree(llvm::StringRef Code,
+  syntax::TranslationUnit *buildTree(StringRef Code,
  const TestClangConfig ) {
 // FIXME: this code is almost the identical to the one in TokensTest. Share
 //it.
@@ -169,7 +169,7 @@
   }
 
   // Adds a file to the test VFS.
-  void addFile(llvm::StringRef Path, llvm::StringRef Contents) {
+  void addFile(StringRef Path, StringRef Contents) {
 if (!FS->addFile(Path, time_t(),
  llvm::MemoryBuffer::getMemBufferCopy(Contents))) {
   ADD_FAILURE() << "could not add a file to VFS: " << Path;
@@ -179,7 +179,7 @@
   /// Finds the deepest node in the tree that covers exactly \p R.
   /// FIXME: implement this efficiently and move to public syntax tree API.
   syntax::Node *nodeByRange(llvm::Annotations::Range R, syntax::Node *Root) {
-llvm::ArrayRef Toks = tokens(Root);
+ArrayRef Toks = tokens(Root);
 
 if (Toks.front().location().isFileID() &&
 Toks.back().location().isFileID() &&
@@ -198,15 +198,14 @@
   }
 
   // Data fields.
-  llvm::IntrusiveRefCntPtr DiagOpts =
-  new DiagnosticOptions();
-  llvm::IntrusiveRefCntPtr Diags =
+  IntrusiveRefCntPtr DiagOpts = new DiagnosticOptions();
+  IntrusiveRefCntPtr Diags =
   new DiagnosticsEngine(new DiagnosticIDs, DiagOpts.get());
   IntrusiveRefCntPtr FS =
   new llvm::vfs::InMemoryFileSystem;
-  llvm::IntrusiveRefCntPtr FileMgr =
+  IntrusiveRefCntPtr FileMgr =
   new FileManager(FileSystemOptions(), FS);
-  llvm::IntrusiveRefCntPtr SourceMgr =
+  IntrusiveRefCntPtr SourceMgr =
   new SourceManager(*Diags, *FileMgr);
   std::shared_ptr Invocation;
   // Set after calling buildTree().
Index: clang/lib/Tooling/Syntax/Tree.cpp
===
--- clang/lib/Tooling/Syntax/Tree.cpp
+++ clang/lib/Tooling/Syntax/Tree.cpp
@@ -36,11 +36,9 @@
  const TokenBuffer )
 : SourceMgr(SourceMgr), LangOpts(LangOpts), Tokens(Tokens) {}
 
-const clang::syntax::TokenBuffer ::Arena::tokenBuffer() const {
-  return Tokens;
-}
+const syntax::TokenBuffer ::Arena::tokenBuffer() const { return Tokens; }
 
-std::pair>
+std::pair>
 syntax::Arena::lexBuffer(std::unique_ptr Input) {
   auto FID = SourceMgr.createFileID(std::move(Input));
   auto It = ExtraTokens.try_emplace(FID, tokenize(FID, SourceMgr, LangOpts));
@@ -135,7 +133,7 @@
 }
 
 namespace {
-static void dumpTokens(llvm::raw_ostream , ArrayRef Tokens,
+static void dumpTokens(raw_ostream , ArrayRef Tokens,
const SourceManager ) {
   assert(!Tokens.empty());
   bool First = true;
@@ -153,7 +151,7 @@
   }
 }
 
-static void dumpTree(llvm::raw_ostream , const syntax::Node *N,
+static void dumpTree(raw_ostream , const syntax::Node *N,
  const syntax::Arena , std::vector IndentMask) {
   std::string Marks;
   if (!N->isOriginal())
@@ -165,13 +163,13 @@
   if (!Marks.empty())
 OS << Marks << ": ";
 
-  if (auto *L = llvm::dyn_cast(N)) {
+  if (auto *L = dyn_cast(N)) {
 dumpTokens(OS, *L->token(), A.sourceManager());
 OS << "\n";
 return;
   }
 
-  auto *T = llvm::cast(N);
+  auto *T = cast(N);
   OS << T->kind() << "\n";
 
   for (auto It = T->firstChild(); It != nullptr; It = It->nextSibling()) {
@@ -205,7 +203,7 @@
   std::string Storage;
   llvm::raw_string_ostream OS(Storage);
   traverse(this, [&](const syntax::Node *N) {
-auto *L = llvm::dyn_cast(N);
+auto *L = dyn_cast(N);
 if (!L)
   return;
 ::dumpTokens(OS, *L->token(), A.sourceManager());
Index: clang/lib/Tooling/Syntax/Nodes.cpp
===
--- clang/lib/Tooling/Syntax/Nodes.cpp
+++ clang/lib/Tooling/Syntax/Nodes.cpp
@@ -10,7 +10,7 @@
 
 using 

[PATCH] D84781: [SyntaxTree] Use PointerUnion instead of inheritance for alternative clauses in NNS

[Eduardo Caldas via Phabricator via cfe-commits]

eduucaldas added inline comments.



Comment at: clang/lib/Tooling/Syntax/BuildTree.cpp:788-789
-
-// Remove "::" from the `SourceRange`
-SR.setEnd(SR.getEnd().getLocWithOffset(-1));
 

Newbie mistake causing the crash



Comment at: clang/lib/Tooling/Syntax/BuildTree.cpp:815
+  Builder.markChild(new (allocator()) syntax::EmptyNode,
+syntax::NodeRole::Unknown);
+  return new (allocator()) syntax::NameSpecifier;

I mark `NodeRole` as `Unknown` here, this would need fixing in the future. But 
should we make roles that  mimic the `NodeKind` of the alternative?



Comment at: clang/unittests/Tooling/Syntax/TreeTest.cpp:1004-1009
+| | | | | `-SimpleTemplateSpecifier
+| | | | |   |-template
+| | | | |   |-ST
+| | | | |   |-<
+| | | | |   |-int
+| | | | |   `->

Here notice the additional level of nesting


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D84781/new/

https://reviews.llvm.org/D84781

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D85440: [SyntaxTree] Implement `NNS` using the `List` base API

[Eduardo Caldas via Phabricator via cfe-commits]

eduucaldas updated this revision to Diff 283643.
eduucaldas added a comment.

.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D85440/new/

https://reviews.llvm.org/D85440

Files:
  clang/include/clang/Tooling/Syntax/Nodes.h
  clang/lib/Tooling/Syntax/BuildTree.cpp
  clang/lib/Tooling/Syntax/Nodes.cpp

Index: clang/lib/Tooling/Syntax/Nodes.cpp
===
--- clang/lib/Tooling/Syntax/Nodes.cpp
+++ clang/lib/Tooling/Syntax/Nodes.cpp
@@ -200,10 +200,6 @@
 return OS << "IdExpression_id";
   case syntax::NodeRole::IdExpression_qualifier:
 return OS << "IdExpression_qualifier";
-  case syntax::NodeRole::NestedNameSpecifier_specifier:
-return OS << "NestedNameSpecifier_specifier";
-  case syntax::NodeRole::NestedNameSpecifier_delimiter:
-return OS << "NestedNameSpecifier_delimiter";
   case syntax::NodeRole::ParenExpression_subExpression:
 return OS << "ParenExpression_subExpression";
   }
@@ -219,23 +215,29 @@
   syntax::SimpleTemplateSpecifier *>::getFromOpaqueValue(firstChild());
 }
 
-std::vector syntax::NestedNameSpecifier::delimiters() {
-  std::vector Children;
-  for (auto *C = firstChild(); C; C = C->nextSibling()) {
-assert(C->role() == syntax::NodeRole::NestedNameSpecifier_delimiter);
-Children.push_back(llvm::cast(C));
+// We could have an interator in list to not pay memory costs of temporary
+// vector
+std::vector syntax::NestedNameSpecifier::specifiers() {
+  auto specifiersAsNodes = getElementsAsNodes();
+  std::vector Children;
+  for (const auto  : specifiersAsNodes) {
+Children.push_back(llvm::cast(element));
   }
   return Children;
 }
 
-std::vector syntax::NestedNameSpecifier::specifiers() {
-  std::vector Children;
-  for (auto *C = firstChild(); C; C = C->nextSibling()) {
-assert(C->role() == syntax::NodeRole::NestedNameSpecifier_specifier);
-Children.push_back(llvm::cast(C));
+std::vector>
+syntax::NestedNameSpecifier::specifiersAndDoubleColons() {
+  auto specifiersAsNodesAndDoubleColons = getElementsAsNodesAndDelimiters();
+  std::vector>
+  Children;
+  for (const auto  : specifiersAsNodesAndDoubleColons) {
+Children.push_back(
+{llvm::cast(specifierAndDoubleColon.element),
+ specifierAndDoubleColon.delimiter});
   }
   return Children;
-}
+};
 
 syntax::NestedNameSpecifier *syntax::IdExpression::qualifier() {
   return llvm::cast_or_null(
Index: clang/lib/Tooling/Syntax/BuildTree.cpp
===
--- clang/lib/Tooling/Syntax/BuildTree.cpp
+++ clang/lib/Tooling/Syntax/BuildTree.cpp
@@ -895,9 +895,8 @@
 for (auto it = QualifierLoc; it; it = it.getPrefix()) {
   auto *NS = BuildNameSpecifier(it);
   assert(NS);
-  Builder.markChild(NS, syntax::NodeRole::NestedNameSpecifier_specifier);
-  Builder.markChildToken(it.getEndLoc(),
- syntax::NodeRole::NestedNameSpecifier_delimiter);
+  Builder.markChild(NS, syntax::NodeRole::List_element);
+  Builder.markChildToken(it.getEndLoc(), syntax::NodeRole::List_delimiter);
 }
 Builder.foldNode(Builder.getRange(QualifierLoc.getSourceRange()),
  new (allocator()) syntax::NestedNameSpecifier,
Index: clang/include/clang/Tooling/Syntax/Nodes.h
===
--- clang/include/clang/Tooling/Syntax/Nodes.h
+++ clang/include/clang/Tooling/Syntax/Nodes.h
@@ -174,8 +174,6 @@
   ParametersAndQualifiers_trailingReturn,
   IdExpression_id,
   IdExpression_qualifier,
-  NestedNameSpecifier_specifier,
-  NestedNameSpecifier_delimiter,
   ParenExpression_subExpression
 };
 /// For debugging purposes.
@@ -235,14 +233,15 @@
 
 /// Models a `nested-name-specifier`. C++ [expr.prim.id.qual]
 /// e.g. the `std::vector::` in `std::vector::size`.
-class NestedNameSpecifier final : public Tree {
+class NestedNameSpecifier final : public List {
 public:
-  NestedNameSpecifier() : Tree(NodeKind::NestedNameSpecifier) {}
+  NestedNameSpecifier() : List(NodeKind::NestedNameSpecifier) {}
   static bool classof(const Node *N) {
 return N->kind() <= NodeKind::NestedNameSpecifier;
   }
   std::vector specifiers();
-  std::vector delimiters();
+  std::vector>
+  specifiersAndDoubleColons();
 };
 
 /// Models an `unqualified-id`. C++ [expr.prim.id.unqual]
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D85440: [SyntaxTree] Implement `NNS` using the `List` base API

[Eduardo Caldas via Phabricator via cfe-commits]

eduucaldas updated this revision to Diff 283638.
eduucaldas added a comment.

Clean List specific code


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D85440/new/

https://reviews.llvm.org/D85440

Files:
  clang/include/clang/Tooling/Syntax/Nodes.h
  clang/lib/Tooling/Syntax/BuildTree.cpp
  clang/lib/Tooling/Syntax/Nodes.cpp

Index: clang/lib/Tooling/Syntax/Nodes.cpp
===
--- clang/lib/Tooling/Syntax/Nodes.cpp
+++ clang/lib/Tooling/Syntax/Nodes.cpp
@@ -200,10 +200,6 @@
 return OS << "IdExpression_id";
   case syntax::NodeRole::IdExpression_qualifier:
 return OS << "IdExpression_qualifier";
-  case syntax::NodeRole::NestedNameSpecifier_specifier:
-return OS << "NestedNameSpecifier_specifier";
-  case syntax::NodeRole::NestedNameSpecifier_delimiter:
-return OS << "NestedNameSpecifier_delimiter";
   case syntax::NodeRole::ParenExpression_subExpression:
 return OS << "ParenExpression_subExpression";
   }
@@ -219,23 +215,29 @@
   syntax::SimpleTemplateSpecifier *>::getFromOpaqueValue(firstChild());
 }
 
-std::vector syntax::NestedNameSpecifier::delimiters() {
-  std::vector Children;
-  for (auto *C = firstChild(); C; C = C->nextSibling()) {
-assert(C->role() == syntax::NodeRole::NestedNameSpecifier_delimiter);
-Children.push_back(llvm::cast(C));
+// We could have an interator in list to not pay memory costs of temporary
+// vector
+std::vector syntax::NestedNameSpecifier::specifiers() {
+  auto specifiersAsNodes = getElementsAsNodes();
+  std::vector Children;
+  for (const auto  : specifiersAsNodes) {
+Children.push_back(llvm::cast(element));
   }
   return Children;
 }
 
-std::vector syntax::NestedNameSpecifier::specifiers() {
-  std::vector Children;
-  for (auto *C = firstChild(); C; C = C->nextSibling()) {
-assert(C->role() == syntax::NodeRole::NestedNameSpecifier_specifier);
-Children.push_back(llvm::cast(C));
+std::vector>
+syntax::NestedNameSpecifier::specifiersAndDoubleColons() {
+  auto specifiersAsNodesAndDoubleColons = getElementsAsNodesAndDelimiters();
+  std::vector>
+  Children;
+  for (const auto  : specifiersAsNodesAndDoubleColons) {
+Children.push_back(
+{llvm::cast(specifierAndDoubleColon.element),
+ specifierAndDoubleColon.delimiter});
   }
   return Children;
-}
+};
 
 syntax::NestedNameSpecifier *syntax::IdExpression::qualifier() {
   return llvm::cast_or_null(
Index: clang/lib/Tooling/Syntax/BuildTree.cpp
===
--- clang/lib/Tooling/Syntax/BuildTree.cpp
+++ clang/lib/Tooling/Syntax/BuildTree.cpp
@@ -895,9 +895,8 @@
 for (auto it = QualifierLoc; it; it = it.getPrefix()) {
   auto *NS = BuildNameSpecifier(it);
   assert(NS);
-  Builder.markChild(NS, syntax::NodeRole::NestedNameSpecifier_specifier);
-  Builder.markChildToken(it.getEndLoc(),
- syntax::NodeRole::NestedNameSpecifier_delimiter);
+  Builder.markChild(NS, syntax::NodeRole::List_element);
+  Builder.markChildToken(it.getEndLoc(), syntax::NodeRole::List_delimiter);
 }
 Builder.foldNode(Builder.getRange(QualifierLoc.getSourceRange()),
  new (allocator()) syntax::NestedNameSpecifier,
Index: clang/include/clang/Tooling/Syntax/Nodes.h
===
--- clang/include/clang/Tooling/Syntax/Nodes.h
+++ clang/include/clang/Tooling/Syntax/Nodes.h
@@ -174,8 +174,6 @@
   ParametersAndQualifiers_trailingReturn,
   IdExpression_id,
   IdExpression_qualifier,
-  NestedNameSpecifier_specifier,
-  NestedNameSpecifier_delimiter,
   ParenExpression_subExpression
 };
 /// For debugging purposes.
@@ -235,14 +233,15 @@
 
 /// Models a `nested-name-specifier`. C++ [expr.prim.id.qual]
 /// e.g. the `std::vector::` in `std::vector::size`.
-class NestedNameSpecifier final : public Tree {
+class NestedNameSpecifier final : public List {
 public:
-  NestedNameSpecifier() : Tree(NodeKind::NestedNameSpecifier) {}
+  NestedNameSpecifier() : List(NodeKind::NestedNameSpecifier) {}
   static bool classof(const Node *N) {
 return N->kind() <= NodeKind::NestedNameSpecifier;
   }
   std::vector specifiers();
-  std::vector delimiters();
+  std::vector>
+  specifiersAndDoubleColons();
 };
 
 /// Models an `unqualified-id`. C++ [expr.prim.id.unqual]
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D85439: [SyntaxTree] Expand support for `NestedNameSpecifier`

[Eduardo Caldas via Phabricator via cfe-commits]

eduucaldas added a reviewer: gribozavr2.
eduucaldas added inline comments.



Comment at: clang/lib/Tooling/Syntax/BuildTree.cpp:227-254
+namespace llvm {
+template <> struct DenseMapInfo {
+  using FirstInfo = DenseMapInfo;
+  using SecondInfo = DenseMapInfo;
+
+  static inline NestedNameSpecifierLoc getEmptyKey() {
+return NestedNameSpecifierLoc(FirstInfo::getEmptyKey(),

Inpired on the definition of: 
* `template struct DenseMapInfo`
* `template struct DenseMapInfo>`

Please tell me if this is all silly.



Comment at: clang/unittests/Tooling/Syntax/TreeTest.cpp:1235
 | | | | |   |-(
-| | | | |   |-IdExpression
-| | | | |   | `-UnqualifiedId
-| | | | |   |   `-s
+| | | | |   |-s
 | | | | |   `-)

standard `TraverseNestedNameSpecifierLoc` fired `TraverseTypeLoc`, once we 
override it we lost this. This will be fixed when refining the Node for 
`DecltypeSpecifier`


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D85439/new/

https://reviews.llvm.org/D85439

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D85440: [SyntaxTree] Implement `NNS` using the `List` base API

[Eduardo Caldas via Phabricator via cfe-commits]

eduucaldas created this revision.
Herald added a project: clang.
Herald added a subscriber: cfe-commits.
eduucaldas requested review of this revision.

Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D85440

Files:
  clang/include/clang/Tooling/Syntax/Nodes.h
  clang/include/clang/Tooling/Syntax/Tree.h
  clang/lib/Tooling/Syntax/BuildTree.cpp
  clang/lib/Tooling/Syntax/Nodes.cpp

Index: clang/lib/Tooling/Syntax/Nodes.cpp
===
--- clang/lib/Tooling/Syntax/Nodes.cpp
+++ clang/lib/Tooling/Syntax/Nodes.cpp
@@ -200,10 +200,6 @@
 return OS << "IdExpression_id";
   case syntax::NodeRole::IdExpression_qualifier:
 return OS << "IdExpression_qualifier";
-  case syntax::NodeRole::NestedNameSpecifier_specifier:
-return OS << "NestedNameSpecifier_specifier";
-  case syntax::NodeRole::NestedNameSpecifier_delimiter:
-return OS << "NestedNameSpecifier_delimiter";
   case syntax::NodeRole::ParenExpression_subExpression:
 return OS << "ParenExpression_subExpression";
   }
@@ -219,23 +215,29 @@
   syntax::SimpleTemplateSpecifier *>::getFromOpaqueValue(firstChild());
 }
 
-std::vector syntax::NestedNameSpecifier::delimiters() {
-  std::vector Children;
-  for (auto *C = firstChild(); C; C = C->nextSibling()) {
-assert(C->role() == syntax::NodeRole::NestedNameSpecifier_delimiter);
-Children.push_back(llvm::cast(C));
+// We could have an interator in list to not pay memory costs of temporary
+// vector
+std::vector syntax::NestedNameSpecifier::specifiers() {
+  auto specifiersAsNodes = getElementsAsNodes();
+  std::vector Children;
+  for (const auto  : specifiersAsNodes) {
+Children.push_back(llvm::cast(element));
   }
   return Children;
 }
 
-std::vector syntax::NestedNameSpecifier::specifiers() {
-  std::vector Children;
-  for (auto *C = firstChild(); C; C = C->nextSibling()) {
-assert(C->role() == syntax::NodeRole::NestedNameSpecifier_specifier);
-Children.push_back(llvm::cast(C));
+std::vector>
+syntax::NestedNameSpecifier::specifiersAndDoubleColons() {
+  auto specifiersAsNodesAndDoubleColons = getElementsAsNodesAndDelimiters();
+  std::vector>
+  Children;
+  for (const auto  : specifiersAsNodesAndDoubleColons) {
+Children.push_back(
+{llvm::cast(specifierAndDoubleColon.element),
+ specifierAndDoubleColon.delimiter});
   }
   return Children;
-}
+};
 
 syntax::NestedNameSpecifier *syntax::IdExpression::qualifier() {
   return llvm::cast_or_null(
Index: clang/lib/Tooling/Syntax/BuildTree.cpp
===
--- clang/lib/Tooling/Syntax/BuildTree.cpp
+++ clang/lib/Tooling/Syntax/BuildTree.cpp
@@ -895,9 +895,8 @@
 for (auto it = QualifierLoc; it; it = it.getPrefix()) {
   auto *NS = BuildNameSpecifier(it);
   assert(NS);
-  Builder.markChild(NS, syntax::NodeRole::NestedNameSpecifier_specifier);
-  Builder.markChildToken(it.getEndLoc(),
- syntax::NodeRole::NestedNameSpecifier_delimiter);
+  Builder.markChild(NS, syntax::NodeRole::List_element);
+  Builder.markChildToken(it.getEndLoc(), syntax::NodeRole::List_delimiter);
 }
 Builder.foldNode(Builder.getRange(QualifierLoc.getSourceRange()),
  new (allocator()) syntax::NestedNameSpecifier,
Index: clang/include/clang/Tooling/Syntax/Tree.h
===
--- clang/include/clang/Tooling/Syntax/Tree.h
+++ clang/include/clang/Tooling/Syntax/Tree.h
@@ -214,6 +214,7 @@
 /// canBeEmpty() returning `true`
 /// getDelimiterTokenKind() returning `,`
 class List : public Tree {
+public:
   template  struct ElementAndDelimiter {
 Element *element;
 Leaf *delimiter;
@@ -225,6 +226,7 @@
 Separated,
   };
 
+  using Tree::Tree;
   /// Returns the elements and corresponding delimiters. Missing elements
   /// and delimiters are represented as null pointers.
   ///
Index: clang/include/clang/Tooling/Syntax/Nodes.h
===
--- clang/include/clang/Tooling/Syntax/Nodes.h
+++ clang/include/clang/Tooling/Syntax/Nodes.h
@@ -174,8 +174,6 @@
   ParametersAndQualifiers_trailingReturn,
   IdExpression_id,
   IdExpression_qualifier,
-  NestedNameSpecifier_specifier,
-  NestedNameSpecifier_delimiter,
   ParenExpression_subExpression
 };
 /// For debugging purposes.
@@ -235,14 +233,15 @@
 
 /// Models a `nested-name-specifier`. C++ [expr.prim.id.qual]
 /// e.g. the `std::vector::` in `std::vector::size`.
-class NestedNameSpecifier final : public Tree {
+class NestedNameSpecifier final : public List {
 public:
-  NestedNameSpecifier() : Tree(NodeKind::NestedNameSpecifier) {}
+  NestedNameSpecifier() : List(NodeKind::NestedNameSpecifier) {}
   static bool classof(const Node *N) {
 return N->kind() <= NodeKind::NestedNameSpecifier;
   }
   std::vector specifiers();
-  std::vector delimiters();
+  

[PATCH] D85439: [SyntaxTree] Expand support for `NestedNameSpecifier`

[Eduardo Caldas via Phabricator via cfe-commits]

eduucaldas created this revision.
Herald added a project: clang.
Herald added a subscriber: cfe-commits.
eduucaldas requested review of this revision.

We want NestedNameSpecifier syntax nodes to be generally supported, not
only for `DeclRefExpr` and `DependentScopedDeclRefExpr`.

To achieve this we:

- Use the `RecursiveASTVisitor`'s API to traverse

`NestedNameSpecifierLoc`s and automatically create its syntax nodes

- Add links from the `NestedNameSpecifierLoc`s to their syntax nodes.

In this way, from any semantic construct that has a `NestedNameSpecifier`,
we implicitly generate its syntax node via RAV and we can easily access
this syntax node via the links we added.


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D85439

Files:
  clang/lib/Tooling/Syntax/BuildTree.cpp
  clang/unittests/Tooling/Syntax/TreeTest.cpp

Index: clang/unittests/Tooling/Syntax/TreeTest.cpp
===
--- clang/unittests/Tooling/Syntax/TreeTest.cpp
+++ clang/unittests/Tooling/Syntax/TreeTest.cpp
@@ -1232,9 +1232,7 @@
 | | | | | `-DecltypeSpecifier
 | | | | |   |-decltype
 | | | | |   |-(
-| | | | |   |-IdExpression
-| | | | |   | `-UnqualifiedId
-| | | | |   |   `-s
+| | | | |   |-s
 | | | | |   `-)
 | | | | `-::
 | | | `-UnqualifiedId
@@ -2980,7 +2978,8 @@
 `-UsingNamespaceDirective
   |-using
   |-namespace
-  |-::
+  |-NestedNameSpecifier
+  | `-::
   |-ns
   `-;
 )txt"));
@@ -3009,8 +3008,10 @@
 | `-}
 `-UsingDeclaration
   |-using
-  |-ns
-  |-::
+  |-NestedNameSpecifier
+  | |-NameSpecifier
+  | | `-ns
+  | `-::
   |-a
   `-;
 )txt"));
@@ -3214,11 +3215,14 @@
   |->
   `-SimpleDeclaration
 |-struct
-|-X
-|-<
-|-T
-|->
-|-::
+|-NestedNameSpecifier
+| |-NameSpecifier
+| | `-SimpleTemplateSpecifier
+| |   |-X
+| |   |-<
+| |   |-T
+| |   `->
+| `-::
 |-Y
 |-{
 |-}
@@ -3252,15 +3256,19 @@
 |-{
 |-UsingDeclaration
 | |-using
-| |-T
-| |-::
+| |-NestedNameSpecifier
+| | |-NameSpecifier
+| | | `-T
+| | `-::
 | |-foo
 | `-;
 |-UsingDeclaration
 | |-using
 | |-typename
-| |-T
-| |-::
+| |-NestedNameSpecifier
+| | |-NameSpecifier
+| | | `-T
+| | `-::
 | |-bar
 | `-;
 |-}
Index: clang/lib/Tooling/Syntax/BuildTree.cpp
===
--- clang/lib/Tooling/Syntax/BuildTree.cpp
+++ clang/lib/Tooling/Syntax/BuildTree.cpp
@@ -224,6 +224,34 @@
   return SourceRange(Start, End);
 }
 
+namespace llvm {
+template <> struct DenseMapInfo {
+  using FirstInfo = DenseMapInfo;
+  using SecondInfo = DenseMapInfo;
+
+  static inline NestedNameSpecifierLoc getEmptyKey() {
+return NestedNameSpecifierLoc(FirstInfo::getEmptyKey(),
+  SecondInfo::getEmptyKey());
+  }
+
+  static inline NestedNameSpecifierLoc getTombstoneKey() {
+return NestedNameSpecifierLoc(FirstInfo::getTombstoneKey(),
+  SecondInfo::getTombstoneKey());
+  }
+
+  static unsigned getHashValue(const clang::NestedNameSpecifierLoc ) {
+return detail::combineHashValue(
+FirstInfo::getHashValue(PairVal.getNestedNameSpecifier()),
+SecondInfo::getHashValue(PairVal.getOpaqueData()));
+  }
+
+  static bool isEqual(const NestedNameSpecifierLoc ,
+  const NestedNameSpecifierLoc ) {
+return LHS == RHS;
+  }
+};
+} // namespace llvm
+
 namespace {
 /// All AST hierarchy roots that can be represented as pointers.
 using ASTPtr = llvm::PointerUnion;
@@ -243,8 +271,22 @@
 
   syntax::Tree *find(ASTPtr P) const { return Nodes.lookup(P); }
 
+  void add(NestedNameSpecifierLoc From, syntax::Tree *To) {
+assert(To != nullptr);
+assert(From.hasQualifier());
+
+bool Added = NNSNodes.insert({From, To}).second;
+(void)Added;
+assert(Added && "mapping added twice");
+  }
+
+  syntax::Tree *find(NestedNameSpecifierLoc P) const {
+return NNSNodes.lookup(P);
+  }
+
 private:
   llvm::DenseMap Nodes;
+  llvm::DenseMap NNSNodes;
 };
 } // namespace
 
@@ -289,9 +331,11 @@
   }
 
   void foldNode(llvm::ArrayRef Range, syntax::Tree *New,
-NestedNameSpecifierLoc L) {
-// FIXME: add mapping for NestedNameSpecifierLoc
-foldNode(Range, New, nullptr);
+NestedNameSpecifierLoc From) {
+assert(New);
+Pending.foldChildren(Arena, Range, New);
+if (From)
+  Mapping.add(From, New);
   }
   /// Notifies that we should not consume trailing semicolon when computing
   /// token range of \p D.
@@ -315,6 +359,9 @@
   /// Set role for the syntax node matching \p N.
   void markChild(ASTPtr N, NodeRole R);
 
+  /// Set role for the syntax node matching \p N.
+  void markChild(NestedNameSpecifierLoc N, NodeRole R);
+
   /// Finish building the tree and consume the root node.
   syntax::TranslationUnit *finalize() && {

[PATCH] D84781: Use PointerUnion instead of inheritance for alternative clauses in NNS

[Eduardo Caldas via Phabricator via cfe-commits]

eduucaldas updated this revision to Diff 283627.
eduucaldas added a comment.

- [SyntaxTree] Fix crash on name specifier.

This diff revision is based on https://reviews.llvm.org/D84348


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D84781/new/

https://reviews.llvm.org/D84781

Files:
  clang/include/clang/Tooling/Syntax/Nodes.h
  clang/include/clang/Tooling/Syntax/Tree.h
  clang/lib/Tooling/Syntax/BuildTree.cpp
  clang/lib/Tooling/Syntax/Nodes.cpp
  clang/lib/Tooling/Syntax/Tree.cpp
  clang/unittests/Tooling/Syntax/TreeTest.cpp

Index: clang/unittests/Tooling/Syntax/TreeTest.cpp
===
--- clang/unittests/Tooling/Syntax/TreeTest.cpp
+++ clang/unittests/Tooling/Syntax/TreeTest.cpp
@@ -994,18 +994,19 @@
 | | |-IdExpression
 | | | |-NestedNameSpecifier
 | | | | |-::
-| | | | |-IdentifierNameSpecifier
+| | | | |-NameSpecifier
 | | | | | `-n
 | | | | |-::
-| | | | |-IdentifierNameSpecifier
+| | | | |-NameSpecifier
 | | | | | `-S
 | | | | |-::
-| | | | |-SimpleTemplateNameSpecifier
-| | | | | |-template
-| | | | | |-ST
-| | | | | |-<
-| | | | | |-int
-| | | | | `->
+| | | | |-NameSpecifier
+| | | | | `-SimpleTemplateSpecifier
+| | | | |   |-template
+| | | | |   |-ST
+| | | | |   |-<
+| | | | |   |-int
+| | | | |   `->
 | | | | `-::
 | | | `-UnqualifiedId
 | | |   `-f
@@ -1016,17 +1017,18 @@
 | |-UnknownExpression
 | | |-IdExpression
 | | | |-NestedNameSpecifier
-| | | | |-IdentifierNameSpecifier
+| | | | |-NameSpecifier
 | | | | | `-n
 | | | | |-::
-| | | | |-IdentifierNameSpecifier
+| | | | |-NameSpecifier
 | | | | | `-S
 | | | | |-::
-| | | | |-SimpleTemplateNameSpecifier
-| | | | | |-ST
-| | | | | |-<
-| | | | | |-int
-| | | | | `->
+| | | | |-NameSpecifier
+| | | | | `-SimpleTemplateSpecifier
+| | | | |   |-ST
+| | | | |   |-<
+| | | | |   |-int
+| | | | |   `->
 | | | | `-::
 | | | `-UnqualifiedId
 | | |   `-f
@@ -1037,13 +1039,14 @@
 | |-UnknownExpression
 | | |-IdExpression
 | | | |-NestedNameSpecifier
-| | | | |-SimpleTemplateNameSpecifier
-| | | | | |-ST
-| | | | | |-<
-| | | | | |-int
-| | | | | `->
+| | | | |-NameSpecifier
+| | | | | `-SimpleTemplateSpecifier
+| | | | |   |-ST
+| | | | |   |-<
+| | | | |   |-int
+| | | | |   `->
 | | | | |-::
-| | | | |-IdentifierNameSpecifier
+| | | | |-NameSpecifier
 | | | | | `-S
 | | | | `-::
 | | | `-UnqualifiedId
@@ -1058,13 +1061,14 @@
 | |-UnknownExpression
 | | |-IdExpression
 | | | |-NestedNameSpecifier
-| | | | |-SimpleTemplateNameSpecifier
-| | | | | |-ST
-| | | | | |-<
-| | | | | |-int
-| | | | | `->
+| | | | |-NameSpecifier
+| | | | | `-SimpleTemplateSpecifier
+| | | | |   |-ST
+| | | | |   |-<
+| | | | |   |-int
+| | | | |   `->
 | | | | |-::
-| | | | |-IdentifierNameSpecifier
+| | | | |-NameSpecifier
 | | | | | `-S
 | | | | `-::
 | | | |-template
@@ -1122,15 +1126,16 @@
   | |-UnknownExpression
   | | |-IdExpression
   | | | |-NestedNameSpecifier
-  | | | | |-IdentifierNameSpecifier
+  | | | | |-NameSpecifier
   | | | | | `-T
   | | | | |-::
-  | | | | |-SimpleTemplateNameSpecifier
-  | | | | | |-template
-  | | | | | |-U
-  | | | | | |-<
-  | | | | | |-int
-  | | | | | `->
+  | | | | |-NameSpecifier
+  | | | | | `-SimpleTemplateSpecifier
+  | | | | |   |-template
+  | | | | |   |-U
+  | | | | |   |-<
+  | | | | |   |-int
+  | | | | |   `->
   | | | | `-::
   | | | `-UnqualifiedId
   | | |   `-f
@@ -1141,10 +1146,10 @@
   | |-UnknownExpression
   | | |-IdExpression
   | | | |-NestedNameSpecifier
-  | | | | |-IdentifierNameSpecifier
+  | | | | |-NameSpecifier
   | | | | | `-T
   | | | | |-::
-  | | | | |-IdentifierNameSpecifier
+  | | | | |-NameSpecifier
   | | | | | `-U
   | | | | `-::
   | | | `-UnqualifiedId
@@ -1156,7 +1161,7 @@
   | |-UnknownExpression
   | | |-IdExpression
   | | | |-NestedNameSpecifier
-  | | | | |-IdentifierNameSpecifier
+  | | | | |-NameSpecifier
   | | | | | `-T
   | | | | `-::
   | | | |-template
@@ -1223,13 +1228,14 @@
 | |-UnknownExpression
 | | |-IdExpression
 | | | |-NestedNameSpecifier
-| | | | |-DecltypeNameSpecifier
-| | | | | |-decltype
-| | | | | |-(
-| | | | | |-IdExpression
-| | | | | | `-UnqualifiedId
-| | | | | |   `-s
-| | | | | `-)
+| | | | |-NameSpecifier
+| | | | | `-DecltypeSpecifier
+| | | | |   |-decltype
+| | | | |   |-(
+| | | | |   |-IdExpression
+| | | | |   | `-UnqualifiedId
+| | | | |   |   `-s

[PATCH] D84348: WIP: Add complete id-expression support to syntax trees

[Eduardo Caldas via Phabricator via cfe-commits]

eduucaldas updated this revision to Diff 283622.
eduucaldas added a comment.

- Update comments to reflect change in API.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D84348/new/

https://reviews.llvm.org/D84348

Files:
  clang/include/clang/Tooling/Syntax/Nodes.h
  clang/lib/Tooling/Syntax/BuildTree.cpp
  clang/lib/Tooling/Syntax/Nodes.cpp
  clang/unittests/Tooling/Syntax/TreeTest.cpp

Index: clang/unittests/Tooling/Syntax/TreeTest.cpp
===
--- clang/unittests/Tooling/Syntax/TreeTest.cpp
+++ clang/unittests/Tooling/Syntax/TreeTest.cpp
@@ -874,24 +874,47 @@
   }
   EXPECT_TRUE(treeDumpEqual(
   R"cpp(
-namespace a {
+namespace n {
   struct S {
 template
-static T f(){}
+struct ST {
+  static void f();
+};
   };
 }
+template
+struct ST {
+  struct S {
+template
+static U f();
+  };
+};
 void test() {
-  ::  // global-namespace-specifier
-  a:: // namespace-specifier
-  S:: // type-name-specifier
+  :: // global-namespace-specifier
+  n::// namespace-specifier
+  S::// type-name-specifier
+  template ST:: // type-template-instantiation-specifier
+  f();
+
+  n::// namespace-specifier
+  S::// type-name-specifier
+  ST::  // type-template-instantiation-specifier
+  f();
+
+  ST:: // type-name-specifier
+  S::   // type-name-specifier
   f();
+
+  ST:: // type-name-specifier
+  S::   // type-name-specifier
+  template f();
 }
 )cpp",
   R"txt(
 *: TranslationUnit
 |-NamespaceDefinition
 | |-namespace
-| |-a
+| |-n
 | |-{
 | |-SimpleDeclaration
 | | |-struct
@@ -905,19 +928,58 @@
 | | | | `-T
 | | | |->
 | | | `-SimpleDeclaration
-| | |   |-static
-| | |   |-T
-| | |   |-SimpleDeclarator
-| | |   | |-f
-| | |   | `-ParametersAndQualifiers
-| | |   |   |-(
-| | |   |   `-)
-| | |   `-CompoundStatement
-| | | |-{
-| | | `-}
+| | |   |-struct
+| | |   |-ST
+| | |   |-{
+| | |   |-SimpleDeclaration
+| | |   | |-static
+| | |   | |-void
+| | |   | |-SimpleDeclarator
+| | |   | | |-f
+| | |   | | `-ParametersAndQualifiers
+| | |   | |   |-(
+| | |   | |   `-)
+| | |   | `-;
+| | |   |-}
+| | |   `-;
 | | |-}
 | | `-;
 | `-}
+|-TemplateDeclaration
+| |-template
+| |-<
+| |-UnknownDeclaration
+| | |-typename
+| | `-T
+| |->
+| `-SimpleDeclaration
+|   |-struct
+|   |-ST
+|   |-{
+|   |-SimpleDeclaration
+|   | |-struct
+|   | |-S
+|   | |-{
+|   | |-TemplateDeclaration
+|   | | |-template
+|   | | |-<
+|   | | |-UnknownDeclaration
+|   | | | |-typename
+|   | | | `-U
+|   | | |->
+|   | | `-SimpleDeclaration
+|   | |   |-static
+|   | |   |-U
+|   | |   |-SimpleDeclarator
+|   | |   | |-f
+|   | |   | `-ParametersAndQualifiers
+|   | |   |   |-(
+|   | |   |   `-)
+|   | |   `-;
+|   | |-}
+|   | `-;
+|   |-}
+|   `-;
 `-SimpleDeclaration
   |-void
   |-SimpleDeclarator
@@ -931,14 +993,81 @@
 | |-UnknownExpression
 | | |-IdExpression
 | | | |-NestedNameSpecifier
-| | | | |-NameSpecifier
-| | | | | `-::
-| | | | |-NameSpecifier
-| | | | | |-a
-| | | | | `-::
-| | | | `-NameSpecifier
-| | | |   |-S
-| | | |   `-::
+| | | | |-::
+| | | | |-IdentifierNameSpecifier
+| | | | | `-n
+| | | | |-::
+| | | | |-IdentifierNameSpecifier
+| | | | | `-S
+| | | | |-::
+| | | | |-SimpleTemplateNameSpecifier
+| | | | | |-template
+| | | | | |-ST
+| | | | | |-<
+| | | | | |-int
+| | | | | `->
+| | | | `-::
+| | | `-UnqualifiedId
+| | |   `-f
+| | |-(
+| | `-)
+| `-;
+|-ExpressionStatement
+| |-UnknownExpression
+| | |-IdExpression
+| | | |-NestedNameSpecifier
+| | | | |-IdentifierNameSpecifier
+| | | | | `-n
+| | | | |-::
+| | | | |-IdentifierNameSpecifier
+| | | | | `-S
+| | | | |-::
+| | | | |-SimpleTemplateNameSpecifier
+| | | | | |-ST
+| | | | | |-<
+| | | | | |-int
+| | | | | `->
+| | | | `-::
+| | | `-UnqualifiedId
+| | |   `-f
+| | |-(
+| | `-)
+| `-;
+|-ExpressionStatement
+| |-UnknownExpression
+| | |-IdExpression
+| | | |-NestedNameSpecifier
+| | | | |-SimpleTemplateNameSpecifier
+| | | | | |-ST
+| | | | | |-<
+| | | | | |-int
+| | | | | `->
+| | | | |-::
+| | | | |-IdentifierNameSpecifier
+| | | | | `-S
+| | | | `-::
+| | | `-UnqualifiedId
+| | |   |-f
+| | |   |-<
+| | |   |-int
+| | |   `->
+| | |-(
+| | `-)
+| `-;
+|-ExpressionStatement
+| |-UnknownExpression
+| | |-IdExpression
+| | | |-NestedNameSpecifier
+| | | | |-SimpleTemplateNameSpecifier
+| | | | | |-ST
+| | | | | |-<
+| | | | | |-int
+| | | | | `->
+| | | | |-::
+| | | | |-IdentifierNameSpecifier
+| | | | | `-S
+| | 

[PATCH] D85427: [SyntaxTree][NFC] remove redundant namespace-specifiers

[Eduardo Caldas via Phabricator via cfe-commits]

eduucaldas created this revision.
Herald added a project: clang.
Herald added a subscriber: cfe-commits.
eduucaldas requested review of this revision.

Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D85427

Files:
  clang/include/clang/Tooling/Syntax/Nodes.h
  clang/include/clang/Tooling/Syntax/Tree.h
  clang/lib/Tooling/Syntax/BuildTree.cpp
  clang/lib/Tooling/Syntax/Tree.cpp

Index: clang/lib/Tooling/Syntax/Tree.cpp
===
--- clang/lib/Tooling/Syntax/Tree.cpp
+++ clang/lib/Tooling/Syntax/Tree.cpp
@@ -36,9 +36,7 @@
  const TokenBuffer )
 : SourceMgr(SourceMgr), LangOpts(LangOpts), Tokens(Tokens) {}
 
-const clang::syntax::TokenBuffer ::Arena::tokenBuffer() const {
-  return Tokens;
-}
+const syntax::TokenBuffer ::Arena::tokenBuffer() const { return Tokens; }
 
 std::pair>
 syntax::Arena::lexBuffer(std::unique_ptr Input) {
Index: clang/lib/Tooling/Syntax/BuildTree.cpp
===
--- clang/lib/Tooling/Syntax/BuildTree.cpp
+++ clang/lib/Tooling/Syntax/BuildTree.cpp
@@ -45,7 +45,7 @@
 using namespace clang;
 
 LLVM_ATTRIBUTE_UNUSED
-static bool isImplicitExpr(clang::Expr *E) { return E->IgnoreImplicit() != E; }
+static bool isImplicitExpr(Expr *E) { return E->IgnoreImplicit() != E; }
 
 namespace {
 /// Get start location of the Declarator from the TypeLoc.
@@ -384,7 +384,7 @@
   }
 
   llvm::ArrayRef getDeclarationRange(Decl *D) {
-llvm::ArrayRef Tokens;
+llvm::ArrayRef Tokens;
 // We want to drop the template parameters for specializations.
 if (const auto *S = llvm::dyn_cast(D))
   Tokens = getRange(S->TypeDecl::getBeginLoc(), S->getEndLoc());
@@ -722,16 +722,16 @@
   syntax::UserDefinedLiteralExpression *
   buildUserDefinedLiteral(UserDefinedLiteral *S) {
 switch (S->getLiteralOperatorKind()) {
-case clang::UserDefinedLiteral::LOK_Integer:
+case UserDefinedLiteral::LOK_Integer:
   return new (allocator()) syntax::IntegerUserDefinedLiteralExpression;
-case clang::UserDefinedLiteral::LOK_Floating:
+case UserDefinedLiteral::LOK_Floating:
   return new (allocator()) syntax::FloatUserDefinedLiteralExpression;
-case clang::UserDefinedLiteral::LOK_Character:
+case UserDefinedLiteral::LOK_Character:
   return new (allocator()) syntax::CharUserDefinedLiteralExpression;
-case clang::UserDefinedLiteral::LOK_String:
+case UserDefinedLiteral::LOK_String:
   return new (allocator()) syntax::StringUserDefinedLiteralExpression;
-case clang::UserDefinedLiteral::LOK_Raw:
-case clang::UserDefinedLiteral::LOK_Template:
+case UserDefinedLiteral::LOK_Raw:
+case UserDefinedLiteral::LOK_Template:
   // For raw literal operator and numeric literal operator template we
   // cannot get the type of the operand in the semantic AST. We get this
   // information from the token. As integer and floating point have the same
Index: clang/include/clang/Tooling/Syntax/Tree.h
===
--- clang/include/clang/Tooling/Syntax/Tree.h
+++ clang/include/clang/Tooling/Syntax/Tree.h
@@ -50,7 +50,7 @@
   /// Add \p Buffer to the underlying source manager, tokenize it and store the
   /// resulting tokens. Useful when there is a need to materialize tokens that
   /// were not written in user code.
-  std::pair>
+  std::pair>
   lexBuffer(std::unique_ptr Buffer);
 
 private:
@@ -58,7 +58,7 @@
   const LangOptions 
   const TokenBuffer 
   /// IDs and storage for additional tokenized files.
-  llvm::DenseMap> ExtraTokens;
+  llvm::DenseMap> ExtraTokens;
   /// Keeps all the allocated nodes and their intermediate data structures.
   llvm::BumpPtrAllocator Allocator;
 };
@@ -139,13 +139,13 @@
 /// A leaf node points to a single token inside the expanded token stream.
 class Leaf final : public Node {
 public:
-  Leaf(const syntax::Token *T);
+  Leaf(const Token *T);
   static bool classof(const Node *N);
 
-  const syntax::Token *token() const { return Tok; }
+  const Token *token() const { return Tok; }
 
 private:
-  const syntax::Token *Tok;
+  const Token *Tok;
 };
 
 /// A node that has children and represents a syntactic language construct.
@@ -167,7 +167,7 @@
 
 protected:
   /// Find the first node with a corresponding role.
-  syntax::Node *findChild(NodeRole R);
+  Node *findChild(NodeRole R);
 
 private:
   /// Prepend \p Child to the list of children and and sets the parent pointer.
Index: clang/include/clang/Tooling/Syntax/Nodes.h
===
--- clang/include/clang/Tooling/Syntax/Nodes.h
+++ clang/include/clang/Tooling/Syntax/Nodes.h
@@ -212,7 +212,7 @@
   static bool classof(const Node *N) {
 return N->kind() <= NodeKind::NestedNameSpecifier;
   }
-  std::vector specifiers();
+  std::vector specifiers();
 };
 
 /// Models an `unqualified-id`. C++ 

[PATCH] D85295: [SyntaxTree] Implement the List construct.

[Eduardo Caldas via Phabricator via cfe-commits]

eduucaldas updated this revision to Diff 283536.
eduucaldas marked 9 inline comments as done.
eduucaldas added a comment.

Answer comments

non-delimited-lists need further discussion


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D85295/new/

https://reviews.llvm.org/D85295

Files:
  clang/include/clang/Tooling/Syntax/Nodes.h
  clang/include/clang/Tooling/Syntax/Tree.h
  clang/lib/Tooling/Syntax/Nodes.cpp
  clang/lib/Tooling/Syntax/Tree.cpp

Index: clang/lib/Tooling/Syntax/Tree.cpp
===
--- clang/lib/Tooling/Syntax/Tree.cpp
+++ clang/lib/Tooling/Syntax/Tree.cpp
@@ -270,3 +270,107 @@
   }
   return nullptr;
 }
+
+std::vector>
+syntax::List::getElementsAsNodesAndDelimiters() {
+  if (!firstChild())
+return {};
+
+  auto children = std::vector>();
+  syntax::Node *elementWithoutDelimiter = nullptr;
+  for (auto *C = firstChild(); C; C = C->nextSibling()) {
+switch (C->role()) {
+case syntax::NodeRole::List_element: {
+  if (elementWithoutDelimiter) {
+children.push_back({elementWithoutDelimiter, nullptr});
+  }
+  elementWithoutDelimiter = C;
+  break;
+}
+case syntax::NodeRole::List_delimiter: {
+  children.push_back({elementWithoutDelimiter, cast(C)});
+  elementWithoutDelimiter = nullptr;
+  break;
+}
+default:
+  llvm_unreachable(
+  "A list can have only elements and delimiters as children.");
+}
+  }
+
+  switch (getTerminationKind()) {
+  case syntax::List::TerminationKind::Separated: {
+children.push_back({elementWithoutDelimiter, nullptr});
+break;
+  }
+  case syntax::List::TerminationKind::Terminated:
+  case syntax::List::TerminationKind::MaybeTerminated: {
+if (elementWithoutDelimiter) {
+  children.push_back({elementWithoutDelimiter, nullptr});
+}
+break;
+  }
+  }
+
+  return children;
+}
+
+// Almost the same implementation of `getElementsAsNodesAndDelimiters` but
+// ignoring delimiters
+std::vector syntax::List::getElementsAsNodes() {
+  if (!firstChild())
+return {};
+
+  auto children = std::vector();
+  syntax::Node *elementWithoutDelimiter = nullptr;
+  for (auto *C = firstChild(); C; C = C->nextSibling()) {
+switch (C->role()) {
+case syntax::NodeRole::List_element: {
+  if (elementWithoutDelimiter) {
+children.push_back(elementWithoutDelimiter);
+  }
+  elementWithoutDelimiter = C;
+  break;
+}
+case syntax::NodeRole::List_delimiter: {
+  children.push_back(elementWithoutDelimiter);
+  elementWithoutDelimiter = nullptr;
+  break;
+}
+default:
+  llvm_unreachable("A list has only elements or delimiters.");
+}
+  }
+
+  switch (getTerminationKind()) {
+  case syntax::List::TerminationKind::Separated: {
+children.push_back(elementWithoutDelimiter);
+break;
+  }
+  case syntax::List::TerminationKind::Terminated:
+  case syntax::List::TerminationKind::MaybeTerminated: {
+if (elementWithoutDelimiter) {
+  children.push_back(elementWithoutDelimiter);
+}
+break;
+  }
+  }
+
+  return children;
+}
+
+// The methods below can't be implemented without information about the derived
+// list. These methods will be implemented by switching on the derived list's
+// `NodeKind`
+
+clang::tok::TokenKind syntax::List::getDelimiterTokenKind() {
+  llvm_unreachable("A list can have only elements and delimiters as children.");
+}
+
+syntax::List::TerminationKind syntax::List::getTerminationKind() {
+  llvm_unreachable("A list can have only elements and delimiters as children.");
+}
+
+bool syntax::List::canBeEmpty() {
+  llvm_unreachable("A list can have only elements and delimiters as children.");
+}
Index: clang/lib/Tooling/Syntax/Nodes.cpp
===
--- clang/lib/Tooling/Syntax/Nodes.cpp
+++ clang/lib/Tooling/Syntax/Nodes.cpp
@@ -144,6 +144,10 @@
 return OS << "ExternKeyword";
   case syntax::NodeRole::BodyStatement:
 return OS << "BodyStatement";
+  case syntax::NodeRole::List_element:
+return OS << "List_element";
+  case syntax::NodeRole::List_delimiter:
+return OS << "List_delimiter";
   case syntax::NodeRole::CaseStatement_value:
 return OS << "CaseStatement_value";
   case syntax::NodeRole::IfStatement_thenStatement:
Index: clang/include/clang/Tooling/Syntax/Tree.h
===
--- clang/include/clang/Tooling/Syntax/Tree.h
+++ clang/include/clang/Tooling/Syntax/Tree.h
@@ -191,6 +191,68 @@
   Node *FirstChild = nullptr;
 };
 
+/// A Tree that represents a syntactic list of elements.
+///
+/// We try to model with this type the artificial grammar-construct:
+/// delimited-list(element, delimiter, termination, canBeEmpty)
+///
+/// For example, from C++ [dcl.decl]:
+/// init-declarator-list:
+///   init-declarator
+///   init-declarator-list , init-declarator
+/// 

[PATCH] D85295: [SyntaxTree] Implement the List construct.

[Eduardo Caldas via Phabricator via cfe-commits]

eduucaldas added inline comments.



Comment at: clang/include/clang/Tooling/Syntax/Tree.h:197
+  MaybeTerminated,
+  Separated,
+};

gribozavr2 wrote:
> Add a "WithoutDelimiters" case as well?
I think we might want to treat non-delimited-list in another way. as many of 
the member functions stop making sense. 
`getElementsAsNodesAndDelimiters` 
`getDelimiterTokenKind`
`getTerminationKind`



Comment at: clang/include/clang/Tooling/Syntax/Tree.h:229
+  /// elements to empty or one-element lists.
+  clang::tok::TokenKind getDelimiterTokenKind();
+

gribozavr2 wrote:
> Should we change this function to return optional to allow representing lists 
> that don't have any delimiters?
Cf, previous comment on non-delimited-lists



Comment at: clang/lib/Tooling/Syntax/Tree.cpp:277
+  if (!firstChild()) {
+assert(canBeEmpty());
+return {};

gribozavr2 wrote:
> This assert is only correct for valid code. When a syntax tree represents 
> invalid code, any list can be empty.
Yeah, of course, we had already discussed that verifying those things in the 
casa of valid code would be the task of the verifier.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D85295/new/

https://reviews.llvm.org/D85295

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D85316: [SyntaxTree] Proposition of new tree dump

[Eduardo Caldas via Phabricator via cfe-commits]

eduucaldas added inline comments.



Comment at: clang/unittests/Tooling/Syntax/TreeTest.cpp:3363
+| |-'if' IntroducerKeyword I
+| |-'(' I
+| |-BinaryOperatorExpression I

Some points to make a decision. 
What should be the order of these extra information?
I like the order now, Most frequent in the left.

More decriptive markers?
We could choose to have more descriptive markers
`I` -> `unmodifieable`
`M`-> `not backed by source code` / `synthesized`

Ambiguity.
Since we choose to not dump a role for `NodeRole::Unknown` we might have some 
ambiguity between NodeRoles and markers. We can avoid that by surrounding 
markers, or making them lower-case. or even by just leaving one character 
markers




Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D85316/new/

https://reviews.llvm.org/D85316

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D85316: [SyntaxTree] Proposition of new tree dump

[Eduardo Caldas via Phabricator via cfe-commits]

eduucaldas updated this revision to Diff 283513.
eduucaldas added a comment.

Reflect comments


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D85316/new/

https://reviews.llvm.org/D85316

Files:
  clang/unittests/Tooling/Syntax/TreeTest.cpp


Index: clang/unittests/Tooling/Syntax/TreeTest.cpp
===
--- clang/unittests/Tooling/Syntax/TreeTest.cpp
+++ clang/unittests/Tooling/Syntax/TreeTest.cpp
@@ -3348,34 +3348,34 @@
   HALF_IF HALF_IF_2 else {}
 })cpp",
   R"txt(
-*: TranslationUnit
+TranslationUnit Detached
 `-SimpleDeclaration
-  |-void
-  |-SimpleDeclarator
-  | |-test
+  |-'void'
+  |-SimpleDeclarator SimpleDeclaration_declarator
+  | |-'test'
   | `-ParametersAndQualifiers
-  |   |-(
-  |   `-)
+  |   |-'(' OpenParen
+  |   `-')' CloseParen
   `-CompoundStatement
-|-{
-|-IfStatement
-| |-I: if
-| |-I: (
-| |-I: BinaryOperatorExpression
-| | |-I: IntegerLiteralExpression
-| | | `-I: 1
-| | |-I: +
-| | `-I: IntegerLiteralExpression
-| |   `-I: 1
-| |-I: )
-| |-I: CompoundStatement
-| | |-I: {
-| | `-I: }
-| |-else
-| `-CompoundStatement
-|   |-{
-|   `-}
-`-}
+|-'{' OpenParen
+|-IfStatement CompoundStatement_statement
+| |-'if' IntroducerKeyword I
+| |-'(' I
+| |-BinaryOperatorExpression I
+| | |-IntegerLiteralExpression BinaryOperatorExpression_leftHandSide I
+| | | `-'1' LiteralToken I
+| | |-'+' OperatorExpression_operatorToken I
+| | `-IntegerLiteralExpression BinaryOperatorExpression_rightHandSide I
+| |   `-'1' LiteralToken I
+| |-')' I
+| |-CompoundStatement IfStatement_thenStatement I
+| | |-'{' OpenParen I
+| | `-'}' CloseParen I
+| |-'else' IfStatement_elseKeyword
+| `-CompoundStatement IfStatement_elseStatement
+|   |-'{' OpenParen
+|   `-'}' CloseParen
+`-'}' CloseParen
 )txt"));
 }
 


Index: clang/unittests/Tooling/Syntax/TreeTest.cpp
===
--- clang/unittests/Tooling/Syntax/TreeTest.cpp
+++ clang/unittests/Tooling/Syntax/TreeTest.cpp
@@ -3348,34 +3348,34 @@
   HALF_IF HALF_IF_2 else {}
 })cpp",
   R"txt(
-*: TranslationUnit
+TranslationUnit Detached
 `-SimpleDeclaration
-  |-void
-  |-SimpleDeclarator
-  | |-test
+  |-'void'
+  |-SimpleDeclarator SimpleDeclaration_declarator
+  | |-'test'
   | `-ParametersAndQualifiers
-  |   |-(
-  |   `-)
+  |   |-'(' OpenParen
+  |   `-')' CloseParen
   `-CompoundStatement
-|-{
-|-IfStatement
-| |-I: if
-| |-I: (
-| |-I: BinaryOperatorExpression
-| | |-I: IntegerLiteralExpression
-| | | `-I: 1
-| | |-I: +
-| | `-I: IntegerLiteralExpression
-| |   `-I: 1
-| |-I: )
-| |-I: CompoundStatement
-| | |-I: {
-| | `-I: }
-| |-else
-| `-CompoundStatement
-|   |-{
-|   `-}
-`-}
+|-'{' OpenParen
+|-IfStatement CompoundStatement_statement
+| |-'if' IntroducerKeyword I
+| |-'(' I
+| |-BinaryOperatorExpression I
+| | |-IntegerLiteralExpression BinaryOperatorExpression_leftHandSide I
+| | | `-'1' LiteralToken I
+| | |-'+' OperatorExpression_operatorToken I
+| | `-IntegerLiteralExpression BinaryOperatorExpression_rightHandSide I
+| |   `-'1' LiteralToken I
+| |-')' I
+| |-CompoundStatement IfStatement_thenStatement I
+| | |-'{' OpenParen I
+| | `-'}' CloseParen I
+| |-'else' IfStatement_elseKeyword
+| `-CompoundStatement IfStatement_elseStatement
+|   |-'{' OpenParen
+|   `-'}' CloseParen
+`-'}' CloseParen
 )txt"));
 }
 
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D85330: [SyntaxTree] Extend the syntax tree dump

[Eduardo Caldas via Phabricator via cfe-commits]

eduucaldas updated this revision to Diff 283506.
eduucaldas added a comment.

Answer comment.

Please suggest more descriptive markers.
Options I thought of:
`I` -> `unmodifiable`
`M` -> `not backed by source code` / `synthesized`


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D85330/new/

https://reviews.llvm.org/D85330

Files:
  clang/include/clang/Tooling/Syntax/Tree.h
  clang/lib/Tooling/Syntax/BuildTree.cpp
  clang/lib/Tooling/Syntax/Tree.cpp
  clang/unittests/Tooling/Syntax/TreeTest.cpp

Index: clang/unittests/Tooling/Syntax/TreeTest.cpp
===
--- clang/unittests/Tooling/Syntax/TreeTest.cpp
+++ clang/unittests/Tooling/Syntax/TreeTest.cpp
@@ -159,7 +159,8 @@
  << "Source file has syntax errors, they were printed to the test "
 "log";
 }
-std::string Actual = std::string(StringRef(Root->dump(*Arena)).trim());
+std::string Actual =
+std::string(StringRef(Root->dump(Arena->sourceManager())).trim());
 // EXPECT_EQ shows the diff between the two strings if they are different.
 EXPECT_EQ(Tree.trim().str(), Actual);
 if (Actual != Tree.trim().str()) {
Index: clang/lib/Tooling/Syntax/Tree.cpp
===
--- clang/lib/Tooling/Syntax/Tree.cpp
+++ clang/lib/Tooling/Syntax/Tree.cpp
@@ -135,46 +135,45 @@
 }
 
 namespace {
-static void dumpTokens(llvm::raw_ostream , ArrayRef Tokens,
-   const SourceManager ) {
-  assert(!Tokens.empty());
-  bool First = true;
-  for (const auto  : Tokens) {
-if (!First)
-  OS << " ";
-else
-  First = false;
-// Handle 'eof' separately, calling text() on it produces an empty string.
-if (T.kind() == tok::eof) {
-  OS << "";
-  continue;
-}
-OS << T.text(SM);
-  }
+static void dumpLeaf(llvm::raw_ostream , const syntax::Leaf *L,
+ const SourceManager ) {
+  assert(L);
+  const auto *Token = L->token();
+  assert(Token);
+  // Handle 'eof' separately, calling text() on it produces an empty string.
+  if (Token->kind() == tok::eof)
+OS << "";
+  else
+OS << Token->text(SM);
 }
 
-static void dumpTree(llvm::raw_ostream , const syntax::Node *N,
- const syntax::Arena , std::vector IndentMask) {
-  std::string Marks;
-  if (!N->isOriginal())
-Marks += "M";
-  if (N->role() == syntax::NodeRole::Detached)
-Marks += "*"; // FIXME: find a nice way to print other roles.
-  if (!N->canModify())
-Marks += "I";
-  if (!Marks.empty())
-OS << Marks << ": ";
-
+static void dumpNode(llvm::raw_ostream , const syntax::Node *N,
+ const SourceManager , std::vector IndentMask) {
+  auto dumpExtraInfo = [](const syntax::Node *N) {
+if (N->role() != syntax::NodeRole::Unknown)
+  OS << " " << N->role();
+if (!N->isOriginal())
+  OS << " M";
+if (!N->canModify())
+  OS << " I";
+  };
+
+  assert(N);
   if (auto *L = llvm::dyn_cast(N)) {
-dumpTokens(OS, *L->token(), A.sourceManager());
+OS << "'";
+dumpLeaf(OS, L, SM);
+OS << "'";
+dumpExtraInfo(N);
 OS << "\n";
 return;
   }
 
   auto *T = llvm::cast(N);
-  OS << T->kind() << "\n";
+  OS << T->kind();
+  dumpExtraInfo(N);
+  OS << "\n";
 
-  for (auto It = T->firstChild(); It != nullptr; It = It->nextSibling()) {
+  for (const auto *It = T->firstChild(); It; It = It->nextSibling()) {
 for (bool Filled : IndentMask) {
   if (Filled)
 OS << "| ";
@@ -188,28 +187,27 @@
   OS << "|-";
   IndentMask.push_back(true);
 }
-dumpTree(OS, It, A, IndentMask);
+dumpNode(OS, It, SM, IndentMask);
 IndentMask.pop_back();
   }
 }
 } // namespace
 
-std::string syntax::Node::dump(const Arena ) const {
+std::string syntax::Node::dump(const SourceManager ) const {
   std::string Str;
   llvm::raw_string_ostream OS(Str);
-  dumpTree(OS, this, A, /*IndentMask=*/{});
+  dumpNode(OS, this, SM, /*IndentMask=*/{});
   return std::move(OS.str());
 }
 
-std::string syntax::Node::dumpTokens(const Arena ) const {
+std::string syntax::Node::dumpTokens(const SourceManager ) const {
   std::string Storage;
   llvm::raw_string_ostream OS(Storage);
   traverse(this, [&](const syntax::Node *N) {
-auto *L = llvm::dyn_cast(N);
-if (!L)
-  return;
-::dumpTokens(OS, *L->token(), A.sourceManager());
-OS << " ";
+if (auto *L = llvm::dyn_cast(N)) {
+  dumpLeaf(OS, L, SM);
+  OS << " ";
+}
   });
   return OS.str();
 }
Index: clang/lib/Tooling/Syntax/BuildTree.cpp
===
--- clang/lib/Tooling/Syntax/BuildTree.cpp
+++ clang/lib/Tooling/Syntax/BuildTree.cpp
@@ -526,7 +526,7 @@
 R += std::string(llvm::formatv(
 "- '{0}' covers '{1}'+{2} tokens\n", It->second->kind(),
 It->first->text(A.sourceManager()), CoveredTokens));
-  

[PATCH] D85330: [SyntaxTree] Extend the syntax tree dump

[Eduardo Caldas via Phabricator via cfe-commits]

eduucaldas updated this revision to Diff 283305.
eduucaldas added a comment.

Rollback strOfLeaf


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D85330/new/

https://reviews.llvm.org/D85330

Files:
  clang/include/clang/Tooling/Syntax/Tree.h
  clang/lib/Tooling/Syntax/BuildTree.cpp
  clang/lib/Tooling/Syntax/Tree.cpp
  clang/unittests/Tooling/Syntax/TreeTest.cpp

Index: clang/unittests/Tooling/Syntax/TreeTest.cpp
===
--- clang/unittests/Tooling/Syntax/TreeTest.cpp
+++ clang/unittests/Tooling/Syntax/TreeTest.cpp
@@ -159,7 +159,8 @@
  << "Source file has syntax errors, they were printed to the test "
 "log";
 }
-std::string Actual = std::string(StringRef(Root->dump(*Arena)).trim());
+std::string Actual =
+std::string(StringRef(Root->dump(Arena->sourceManager())).trim());
 // EXPECT_EQ shows the diff between the two strings if they are different.
 EXPECT_EQ(Tree.trim().str(), Actual);
 if (Actual != Tree.trim().str()) {
Index: clang/lib/Tooling/Syntax/Tree.cpp
===
--- clang/lib/Tooling/Syntax/Tree.cpp
+++ clang/lib/Tooling/Syntax/Tree.cpp
@@ -135,46 +135,46 @@
 }
 
 namespace {
-static void dumpTokens(llvm::raw_ostream , ArrayRef Tokens,
-   const SourceManager ) {
-  assert(!Tokens.empty());
-  bool First = true;
-  for (const auto  : Tokens) {
-if (!First)
-  OS << " ";
-else
-  First = false;
-// Handle 'eof' separately, calling text() on it produces an empty string.
-if (T.kind() == tok::eof) {
-  OS << "";
-  continue;
-}
-OS << T.text(SM);
-  }
+static void dumpLeaf(llvm::raw_ostream , const syntax::Leaf *L,
+ const SourceManager ) {
+  assert(L);
+  const auto *Token = L->token();
+  assert(Token);
+  // Handle 'eof' separately, calling text() on it produces an empty string.
+  if (Token->kind() == tok::eof)
+OS << "";
+  else
+OS << Token->text(SM);
 }
 
-static void dumpTree(llvm::raw_ostream , const syntax::Node *N,
- const syntax::Arena , std::vector IndentMask) {
+static void dumpNode(llvm::raw_ostream , const syntax::Node *N,
+ const SourceManager , std::vector IndentMask) {
   std::string Marks;
   if (!N->isOriginal())
 Marks += "M";
-  if (N->role() == syntax::NodeRole::Detached)
-Marks += "*"; // FIXME: find a nice way to print other roles.
   if (!N->canModify())
 Marks += "I";
   if (!Marks.empty())
 OS << Marks << ": ";
 
   if (auto *L = llvm::dyn_cast(N)) {
-dumpTokens(OS, *L->token(), A.sourceManager());
+OS << "'";
+dumpLeaf(OS, L, SM);
+OS << "'";
+if (L->role() != syntax::NodeRole::Unknown)
+  OS << " " << L->role();
 OS << "\n";
 return;
   }
 
   auto *T = llvm::cast(N);
-  OS << T->kind() << "\n";
+  OS << T->kind();
+  if (T->role() != syntax::NodeRole::Unknown)
+OS << " " << T->role();
+  OS << "\n";
 
-  for (auto It = T->firstChild(); It != nullptr; It = It->nextSibling()) {
+  for (const auto *It = T->firstChild(); It != nullptr;
+   It = It->nextSibling()) {
 for (bool Filled : IndentMask) {
   if (Filled)
 OS << "| ";
@@ -188,28 +188,27 @@
   OS << "|-";
   IndentMask.push_back(true);
 }
-dumpTree(OS, It, A, IndentMask);
+dumpNode(OS, It, SM, IndentMask);
 IndentMask.pop_back();
   }
 }
 } // namespace
 
-std::string syntax::Node::dump(const Arena ) const {
+std::string syntax::Node::dump(const SourceManager ) const {
   std::string Str;
   llvm::raw_string_ostream OS(Str);
-  dumpTree(OS, this, A, /*IndentMask=*/{});
+  dumpNode(OS, this, SM, /*IndentMask=*/{});
   return std::move(OS.str());
 }
 
-std::string syntax::Node::dumpTokens(const Arena ) const {
+std::string syntax::Node::dumpTokens(const SourceManager ) const {
   std::string Storage;
   llvm::raw_string_ostream OS(Storage);
   traverse(this, [&](const syntax::Node *N) {
-auto *L = llvm::dyn_cast(N);
-if (!L)
-  return;
-::dumpTokens(OS, *L->token(), A.sourceManager());
-OS << " ";
+if (auto *L = llvm::dyn_cast(N)) {
+  dumpLeaf(OS, L, SM);
+  OS << " ";
+}
   });
   return OS.str();
 }
Index: clang/lib/Tooling/Syntax/BuildTree.cpp
===
--- clang/lib/Tooling/Syntax/BuildTree.cpp
+++ clang/lib/Tooling/Syntax/BuildTree.cpp
@@ -526,7 +526,7 @@
 R += std::string(llvm::formatv(
 "- '{0}' covers '{1}'+{2} tokens\n", It->second->kind(),
 It->first->text(A.sourceManager()), CoveredTokens));
-R += It->second->dump(A);
+R += It->second->dump(A.sourceManager());
   }
   return R;
 }
Index: clang/include/clang/Tooling/Syntax/Tree.h
===
--- 

[PATCH] D85316: [SyntaxTree] Proposition of new tree dump

[Eduardo Caldas via Phabricator via cfe-commits]

eduucaldas updated this revision to Diff 283298.
eduucaldas added a comment.

Answering comments


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D85316/new/

https://reviews.llvm.org/D85316

Files:
  clang/unittests/Tooling/Syntax/TreeTest.cpp


Index: clang/unittests/Tooling/Syntax/TreeTest.cpp
===
--- clang/unittests/Tooling/Syntax/TreeTest.cpp
+++ clang/unittests/Tooling/Syntax/TreeTest.cpp
@@ -308,45 +308,45 @@
 }
 )cpp",
   R"txt(
-*: TranslationUnit
+TranslationUnit Detached
 `-SimpleDeclaration
-  |-int
-  |-SimpleDeclarator
-  | |-main
+  |-'int'
+  |-SimpleDeclarator SimpleDeclaration_declarator
+  | |-'main'
   | `-ParametersAndQualifiers
-  |   |-(
-  |   `-)
+  |   |-'(' OpenParen
+  |   `-')' CloseParen
   `-CompoundStatement
-|-{
-|-IfStatement
-| |-if
-| |-(
+|-'{' OpenParen
+|-IfStatement CompoundStatement_statement
+| |-'if' IntroducerKeyword
+| |-'('
 | |-IntegerLiteralExpression
-| | `-1
-| |-)
-| `-CompoundStatement
-|   |-{
-|   `-}
-|-IfStatement
-| |-if
-| |-(
+| | `-'1' LiteralToken
+| |-')'
+| `-CompoundStatement IfStatement_thenStatement
+|   |-'{' OpenParen
+|   `-'}' CloseParen
+|-IfStatement CompoundStatement_statement
+| |-'if' IntroducerKeyword
+| |-'('
 | |-IntegerLiteralExpression
-| | `-1
-| |-)
-| |-CompoundStatement
-| | |-{
-| | `-}
-| |-else
-| `-IfStatement
-|   |-if
-|   |-(
+| | `-'1' LiteralToken
+| |-')'
+| |-CompoundStatement IfStatement_thenStatement
+| | |-'{' OpenParen
+| | `-'}' CloseParen
+| |-'else' IfStatement_elseKeyword
+| `-IfStatement IfStatement_elseStatement
+|   |-'if' IntroducerKeyword
+|   |-'('
 |   |-IntegerLiteralExpression
-|   | `-0
-|   |-)
-|   `-CompoundStatement
-| |-{
-| `-}
-`-}
+|   | `-'0' LiteralToken
+|   |-')'
+|   `-CompoundStatement IfStatement_thenStatement
+| |-'{' OpenParen
+| `-'}' CloseParen
+`-'}' CloseParen
 )txt"));
 }
 


Index: clang/unittests/Tooling/Syntax/TreeTest.cpp
===
--- clang/unittests/Tooling/Syntax/TreeTest.cpp
+++ clang/unittests/Tooling/Syntax/TreeTest.cpp
@@ -308,45 +308,45 @@
 }
 )cpp",
   R"txt(
-*: TranslationUnit
+TranslationUnit Detached
 `-SimpleDeclaration
-  |-int
-  |-SimpleDeclarator
-  | |-main
+  |-'int'
+  |-SimpleDeclarator SimpleDeclaration_declarator
+  | |-'main'
   | `-ParametersAndQualifiers
-  |   |-(
-  |   `-)
+  |   |-'(' OpenParen
+  |   `-')' CloseParen
   `-CompoundStatement
-|-{
-|-IfStatement
-| |-if
-| |-(
+|-'{' OpenParen
+|-IfStatement CompoundStatement_statement
+| |-'if' IntroducerKeyword
+| |-'('
 | |-IntegerLiteralExpression
-| | `-1
-| |-)
-| `-CompoundStatement
-|   |-{
-|   `-}
-|-IfStatement
-| |-if
-| |-(
+| | `-'1' LiteralToken
+| |-')'
+| `-CompoundStatement IfStatement_thenStatement
+|   |-'{' OpenParen
+|   `-'}' CloseParen
+|-IfStatement CompoundStatement_statement
+| |-'if' IntroducerKeyword
+| |-'('
 | |-IntegerLiteralExpression
-| | `-1
-| |-)
-| |-CompoundStatement
-| | |-{
-| | `-}
-| |-else
-| `-IfStatement
-|   |-if
-|   |-(
+| | `-'1' LiteralToken
+| |-')'
+| |-CompoundStatement IfStatement_thenStatement
+| | |-'{' OpenParen
+| | `-'}' CloseParen
+| |-'else' IfStatement_elseKeyword
+| `-IfStatement IfStatement_elseStatement
+|   |-'if' IntroducerKeyword
+|   |-'('
 |   |-IntegerLiteralExpression
-|   | `-0
-|   |-)
-|   `-CompoundStatement
-| |-{
-| `-}
-`-}
+|   | `-'0' LiteralToken
+|   |-')'
+|   `-CompoundStatement IfStatement_thenStatement
+| |-'{' OpenParen
+| `-'}' CloseParen
+`-'}' CloseParen
 )txt"));
 }
 
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D85330: [SyntaxTree] Extend the syntax tree dump

[Eduardo Caldas via Phabricator via cfe-commits]

eduucaldas created this revision.
Herald added a project: clang.
Herald added a subscriber: cfe-commits.
eduucaldas requested review of this revision.

Functional changes in the dump:

- Surround Leaf tokens with `'`
- Append `Node` dumps  with `NodeRole` information, except for unknown roles

Non-functional changes:

- `::dumpTokens(llvm::raw_ostream, ArrayRef, const SourceManager 
)` always received as parameter a `syntax::Token *` pointing to 
`Leaf::token()`. Changed the function to `strOfLeaf(syntax::Leaf *, const 
SourceManager&)`
- `dumpTree` acted on a Node, rename to `dumpNode`

TODO:
Adapt all the tree dumps in `TreeTest.cpp` to this new format.


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D85330

Files:
  clang/include/clang/Tooling/Syntax/Tree.h
  clang/lib/Tooling/Syntax/BuildTree.cpp
  clang/lib/Tooling/Syntax/Tree.cpp
  clang/unittests/Tooling/Syntax/TreeTest.cpp

Index: clang/unittests/Tooling/Syntax/TreeTest.cpp
===
--- clang/unittests/Tooling/Syntax/TreeTest.cpp
+++ clang/unittests/Tooling/Syntax/TreeTest.cpp
@@ -159,7 +159,8 @@
  << "Source file has syntax errors, they were printed to the test "
 "log";
 }
-std::string Actual = std::string(StringRef(Root->dump(*Arena)).trim());
+std::string Actual =
+std::string(StringRef(Root->dump(Arena->sourceManager())).trim());
 // EXPECT_EQ shows the diff between the two strings if they are different.
 EXPECT_EQ(Tree.trim().str(), Actual);
 if (Actual != Tree.trim().str()) {
Index: clang/lib/Tooling/Syntax/Tree.cpp
===
--- clang/lib/Tooling/Syntax/Tree.cpp
+++ clang/lib/Tooling/Syntax/Tree.cpp
@@ -135,46 +135,43 @@
 }
 
 namespace {
-static void dumpTokens(llvm::raw_ostream , ArrayRef Tokens,
-   const SourceManager ) {
-  assert(!Tokens.empty());
-  bool First = true;
-  for (const auto  : Tokens) {
-if (!First)
-  OS << " ";
-else
-  First = false;
-// Handle 'eof' separately, calling text() on it produces an empty string.
-if (T.kind() == tok::eof) {
-  OS << "";
-  continue;
-}
-OS << T.text(SM);
-  }
+static llvm::StringRef strOfLeaf(const syntax::Leaf *L,
+ const SourceManager ) {
+  assert(L);
+  const auto *Token = L->token();
+  assert(Token);
+  // Handle 'eof' separately, calling text() on it produces an empty string.
+  if (Token->kind() == tok::eof)
+return "";
+  return Token->text(SM);
 }
 
-static void dumpTree(llvm::raw_ostream , const syntax::Node *N,
- const syntax::Arena , std::vector IndentMask) {
+static void dumpNode(llvm::raw_ostream , const syntax::Node *N,
+ const SourceManager , std::vector IndentMask) {
   std::string Marks;
   if (!N->isOriginal())
 Marks += "M";
-  if (N->role() == syntax::NodeRole::Detached)
-Marks += "*"; // FIXME: find a nice way to print other roles.
   if (!N->canModify())
 Marks += "I";
   if (!Marks.empty())
 OS << Marks << ": ";
 
   if (auto *L = llvm::dyn_cast(N)) {
-dumpTokens(OS, *L->token(), A.sourceManager());
+OS << "'" << strOfLeaf(L, SM) << "'";
+if (L->role() != syntax::NodeRole::Unknown)
+  OS << " " << L->role();
 OS << "\n";
 return;
   }
 
   auto *T = llvm::cast(N);
-  OS << T->kind() << "\n";
+  OS << T->kind();
+  if (T->role() != syntax::NodeRole::Unknown)
+OS << " " << T->role();
+  OS << "\n";
 
-  for (auto It = T->firstChild(); It != nullptr; It = It->nextSibling()) {
+  for (const auto *It = T->firstChild(); It != nullptr;
+   It = It->nextSibling()) {
 for (bool Filled : IndentMask) {
   if (Filled)
 OS << "| ";
@@ -188,28 +185,26 @@
   OS << "|-";
   IndentMask.push_back(true);
 }
-dumpTree(OS, It, A, IndentMask);
+dumpNode(OS, It, SM, IndentMask);
 IndentMask.pop_back();
   }
 }
 } // namespace
 
-std::string syntax::Node::dump(const Arena ) const {
+std::string syntax::Node::dump(const SourceManager ) const {
   std::string Str;
   llvm::raw_string_ostream OS(Str);
-  dumpTree(OS, this, A, /*IndentMask=*/{});
+  dumpNode(OS, this, SM, /*IndentMask=*/{});
   return std::move(OS.str());
 }
 
-std::string syntax::Node::dumpTokens(const Arena ) const {
+std::string syntax::Node::dumpTokens(const SourceManager ) const {
   std::string Storage;
   llvm::raw_string_ostream OS(Storage);
   traverse(this, [&](const syntax::Node *N) {
-auto *L = llvm::dyn_cast(N);
-if (!L)
-  return;
-::dumpTokens(OS, *L->token(), A.sourceManager());
-OS << " ";
+if (auto *L = llvm::dyn_cast(N)) {
+  OS << strOfLeaf(L, SM) << " ";
+}
   });
   return OS.str();
 }
Index: clang/lib/Tooling/Syntax/BuildTree.cpp
===
--- clang/lib/Tooling/Syntax/BuildTree.cpp
+++ 

[PATCH] D85316: [SyntaxTree] Proposition of new tree dump

[Eduardo Caldas via Phabricator via cfe-commits]

eduucaldas created this revision.
Herald added a project: clang.
Herald added a subscriber: cfe-commits.
eduucaldas requested review of this revision.

Some key choices to highlight:

- Surround Tokens with "''"
- Do not print `UnknownRole`, to reduce noise
- Surround Roles with "<", to clarify the difference in meaning


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D85316

Files:
  clang/unittests/Tooling/Syntax/TreeTest.cpp


Index: clang/unittests/Tooling/Syntax/TreeTest.cpp
===
--- clang/unittests/Tooling/Syntax/TreeTest.cpp
+++ clang/unittests/Tooling/Syntax/TreeTest.cpp
@@ -307,45 +307,45 @@
 }
 )cpp",
   R"txt(
-*: TranslationUnit
+TranslationUnit   <*>
 `-SimpleDeclaration
-  |-int
-  |-SimpleDeclarator
-  | |-main
+  |-'int'
+  |-SimpleDeclarator   
+  | |-'main'
   | `-ParametersAndQualifiers
-  |   |-(
-  |   `-)
+  |   |-'('   
+  |   `-')'   
   `-CompoundStatement
-|-{
-|-IfStatement
-| |-if
-| |-(
+|-'{'   
+|-IfStatement   
+| |-'if'   
+| |-'('
 | |-IntegerLiteralExpression
-| | `-1
-| |-)
-| `-CompoundStatement
-|   |-{
-|   `-}
-|-IfStatement
-| |-if
-| |-(
+| | `-'1'   
+| |-')'
+| `-CompoundStatement   
+|   |-'{'   
+|   `-'}'   
+|-IfStatement   
+| |-'if'   
+| |-'('
 | |-IntegerLiteralExpression
-| | `-1
-| |-)
-| |-CompoundStatement
-| | |-{
-| | `-}
-| |-else
-| `-IfStatement
-|   |-if
-|   |-(
+| | `-'1'   
+| |-')'
+| |-CompoundStatement   
+| | |-'{'   
+| | `-'}'   
+| |-'else'   
+| `-IfStatement   
+|   |-'if'   
+|   |-'('
 |   |-IntegerLiteralExpression
-|   | `-0
-|   |-)
-|   `-CompoundStatement
-| |-{
-| `-}
-`-}
+|   | `-'0'   
+|   |-')'
+|   `-CompoundStatement   
+| |-'{'   
+| `-'}'   
+`-'}'   
 )txt"));
 }
 


Index: clang/unittests/Tooling/Syntax/TreeTest.cpp
===
--- clang/unittests/Tooling/Syntax/TreeTest.cpp
+++ clang/unittests/Tooling/Syntax/TreeTest.cpp
@@ -307,45 +307,45 @@
 }
 )cpp",
   R"txt(
-*: TranslationUnit
+TranslationUnit   <*>
 `-SimpleDeclaration
-  |-int
-  |-SimpleDeclarator
-  | |-main
+  |-'int'
+  |-SimpleDeclarator   
+  | |-'main'
   | `-ParametersAndQualifiers
-  |   |-(
-  |   `-)
+  |   |-'('   
+  |   `-')'   
   `-CompoundStatement
-|-{
-|-IfStatement
-| |-if
-| |-(
+|-'{'   
+|-IfStatement   
+| |-'if'   
+| |-'('
 | |-IntegerLiteralExpression
-| | `-1
-| |-)
-| `-CompoundStatement
-|   |-{
-|   `-}
-|-IfStatement
-| |-if
-| |-(
+| | `-'1'   
+| |-')'
+| `-CompoundStatement   
+|   |-'{'   
+|   `-'}'   
+|-IfStatement   
+| |-'if'   
+| |-'('
 | |-IntegerLiteralExpression
-| | `-1
-| |-)
-| |-CompoundStatement
-| | |-{
-| | `-}
-| |-else
-| `-IfStatement
-|   |-if
-|   |-(
+| | `-'1'   
+| |-')'
+| |-CompoundStatement   
+| | |-'{'   
+| | `-'}'   
+| |-'else'   
+| `-IfStatement   
+|   |-'if'   
+|   |-'('
 |   |-IntegerLiteralExpression
-|   | `-0
-|   |-)
-|   `-CompoundStatement
-| |-{
-| `-}
-`-}
+|   | `-'0'   
+|   |-')'
+|   `-CompoundStatement   
+| |-'{'   
+| `-'}'   
+`-'}'   
 )txt"));
 }
 
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D85305: [SyntaxTree] Remove dead code on dump functions

[Eduardo Caldas via Phabricator via cfe-commits]

eduucaldas created this revision.
Herald added a project: clang.
Herald added a subscriber: cfe-commits.
eduucaldas requested review of this revision.

- `Node::dumpTokens` was never used.
- `::dumpTokens(llvm::raw_ostream, ArrayRef, const

SourceManager )` was used twice, once by `Node::dumpTokens`.
Additionally it always received as parameter a `syntax::Token *` pointing to 
one token only, instead of an `ArrayRef`

I removed the first and inlined the simplified version of the second in its 
only caller


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D85305

Files:
  clang/include/clang/Tooling/Syntax/Tree.h
  clang/lib/Tooling/Syntax/Tree.cpp


Index: clang/lib/Tooling/Syntax/Tree.cpp
===
--- clang/lib/Tooling/Syntax/Tree.cpp
+++ clang/lib/Tooling/Syntax/Tree.cpp
@@ -135,24 +135,6 @@
 }
 
 namespace {
-static void dumpTokens(llvm::raw_ostream , ArrayRef Tokens,
-   const SourceManager ) {
-  assert(!Tokens.empty());
-  bool First = true;
-  for (const auto  : Tokens) {
-if (!First)
-  OS << " ";
-else
-  First = false;
-// Handle 'eof' separately, calling text() on it produces an empty string.
-if (T.kind() == tok::eof) {
-  OS << "";
-  continue;
-}
-OS << T.text(SM);
-  }
-}
-
 static void dumpTree(llvm::raw_ostream , const syntax::Node *N,
  const syntax::Arena , std::vector IndentMask) {
   std::string Marks;
@@ -166,7 +148,13 @@
 OS << Marks << ": ";
 
   if (auto *L = llvm::dyn_cast(N)) {
-dumpTokens(OS, *L->token(), A.sourceManager());
+auto *Token = L->token();
+assert(Token);
+// Handle 'eof' separately, calling text() on it produces an empty string.
+if (Token->kind() == tok::eof)
+  OS << "";
+else
+  OS << Token->text(A.sourceManager());
 OS << "\n";
 return;
   }
@@ -174,7 +162,8 @@
   auto *T = llvm::cast(N);
   OS << T->kind() << "\n";
 
-  for (auto It = T->firstChild(); It != nullptr; It = It->nextSibling()) {
+  for (const auto *It = T->firstChild(); It != nullptr;
+   It = It->nextSibling()) {
 for (bool Filled : IndentMask) {
   if (Filled)
 OS << "| ";
@@ -201,19 +190,6 @@
   return std::move(OS.str());
 }
 
-std::string syntax::Node::dumpTokens(const Arena ) const {
-  std::string Storage;
-  llvm::raw_string_ostream OS(Storage);
-  traverse(this, [&](const syntax::Node *N) {
-auto *L = llvm::dyn_cast(N);
-if (!L)
-  return;
-::dumpTokens(OS, *L->token(), A.sourceManager());
-OS << " ";
-  });
-  return OS.str();
-}
-
 void syntax::Node::assertInvariants() const {
 #ifndef NDEBUG
   if (isDetached())
Index: clang/include/clang/Tooling/Syntax/Tree.h
===
--- clang/include/clang/Tooling/Syntax/Tree.h
+++ clang/include/clang/Tooling/Syntax/Tree.h
@@ -107,8 +107,6 @@
 
   /// Dumps the structure of a subtree. For debugging and testing purposes.
   std::string dump(const Arena ) const;
-  /// Dumps the tokens forming this subtree.
-  std::string dumpTokens(const Arena ) const;
 
   /// Asserts invariants on this node of the tree and its immediate children.
   /// Will not recurse into the subtree. No-op if NDEBUG is set.


Index: clang/lib/Tooling/Syntax/Tree.cpp
===
--- clang/lib/Tooling/Syntax/Tree.cpp
+++ clang/lib/Tooling/Syntax/Tree.cpp
@@ -135,24 +135,6 @@
 }
 
 namespace {
-static void dumpTokens(llvm::raw_ostream , ArrayRef Tokens,
-   const SourceManager ) {
-  assert(!Tokens.empty());
-  bool First = true;
-  for (const auto  : Tokens) {
-if (!First)
-  OS << " ";
-else
-  First = false;
-// Handle 'eof' separately, calling text() on it produces an empty string.
-if (T.kind() == tok::eof) {
-  OS << "";
-  continue;
-}
-OS << T.text(SM);
-  }
-}
-
 static void dumpTree(llvm::raw_ostream , const syntax::Node *N,
  const syntax::Arena , std::vector IndentMask) {
   std::string Marks;
@@ -166,7 +148,13 @@
 OS << Marks << ": ";
 
   if (auto *L = llvm::dyn_cast(N)) {
-dumpTokens(OS, *L->token(), A.sourceManager());
+auto *Token = L->token();
+assert(Token);
+// Handle 'eof' separately, calling text() on it produces an empty string.
+if (Token->kind() == tok::eof)
+  OS << "";
+else
+  OS << Token->text(A.sourceManager());
 OS << "\n";
 return;
   }
@@ -174,7 +162,8 @@
   auto *T = llvm::cast(N);
   OS << T->kind() << "\n";
 
-  for (auto It = T->firstChild(); It != nullptr; It = It->nextSibling()) {
+  for (const auto *It = T->firstChild(); It != nullptr;
+   It = It->nextSibling()) {
 for (bool Filled : IndentMask) {
   if (Filled)
 OS << "| ";
@@ -201,19 +190,6 @@
   return std::move(OS.str());
 }
 
-std::string syntax::Node::dumpTokens(const Arena ) const {
-  

[PATCH] D85295: [SyntaxTree] Implement the List construct.

[Eduardo Caldas via Phabricator via cfe-commits]

eduucaldas added a reviewer: gribozavr2.
eduucaldas added inline comments.



Comment at: clang/include/clang/Tooling/Syntax/Tree.h:223
+
+  // These can't be implemented with the information we have!
+

As this is a base List, we don't  have the necessary information tom implement 
the following methods. I chose to give them default values, check their 
definition


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D85295/new/

https://reviews.llvm.org/D85295

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D85295: [SyntaxTree] Implement the List construct.

[Eduardo Caldas via Phabricator via cfe-commits]

eduucaldas created this revision.
Herald added a project: clang.
Herald added a subscriber: cfe-commits.
eduucaldas requested review of this revision.

We defined a List construct to help with the implementation of list-like
grammar rules. This is a first implementation of this API.


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D85295

Files:
  clang/include/clang/Tooling/Syntax/Nodes.h
  clang/include/clang/Tooling/Syntax/Tree.h
  clang/lib/Tooling/Syntax/Nodes.cpp
  clang/lib/Tooling/Syntax/Tree.cpp

Index: clang/lib/Tooling/Syntax/Tree.cpp
===
--- clang/lib/Tooling/Syntax/Tree.cpp
+++ clang/lib/Tooling/Syntax/Tree.cpp
@@ -270,3 +270,104 @@
   }
   return nullptr;
 }
+
+std::vector>
+syntax::List::getElementsAsNodesAndDelimiters() {
+  if (!firstChild()) {
+assert(canBeEmpty());
+return {};
+  }
+  auto children = std::vector>();
+  syntax::Node *elementWithoutDelimiter = nullptr;
+  for (auto *C = firstChild(); C; C = C->nextSibling()) {
+switch (C->role()) {
+case syntax::NodeRole::List_element: {
+  if (elementWithoutDelimiter) {
+children.push_back({elementWithoutDelimiter, nullptr});
+  }
+  elementWithoutDelimiter = C;
+  break;
+}
+case syntax::NodeRole::List_delimiter: {
+  children.push_back(
+  {elementWithoutDelimiter, llvm::cast(C)});
+  elementWithoutDelimiter = nullptr;
+  break;
+}
+default:
+  llvm_unreachable("A list has only elements or delimiters.");
+}
+  }
+
+  switch (getTerminationKind()) {
+  case syntax::TerminationKind::Separated: {
+children.push_back({elementWithoutDelimiter, nullptr});
+break;
+  }
+  case syntax::TerminationKind::Terminated:
+  case syntax::TerminationKind::MaybeTerminated: {
+if (elementWithoutDelimiter) {
+  children.push_back({elementWithoutDelimiter, nullptr});
+}
+break;
+  }
+  }
+
+  return children;
+}
+
+// Almost the same implementation of `getElementsAsNodesAndDelimiters` but
+// ignoring delimiters
+std::vector syntax::List::getElementsAsNodes() {
+  if (!firstChild()) {
+assert(canBeEmpty());
+return {};
+  }
+  auto children = std::vector();
+  syntax::Node *elementWithoutDelimiter = nullptr;
+  for (auto *C = firstChild(); C; C = C->nextSibling()) {
+switch (C->role()) {
+case syntax::NodeRole::List_element: {
+  if (elementWithoutDelimiter) {
+children.push_back(elementWithoutDelimiter);
+  }
+  elementWithoutDelimiter = C;
+  break;
+}
+case syntax::NodeRole::List_delimiter: {
+  children.push_back(elementWithoutDelimiter);
+  elementWithoutDelimiter = nullptr;
+  break;
+}
+default:
+  llvm_unreachable("A list has only elements or delimiters.");
+}
+  }
+
+  switch (getTerminationKind()) {
+  case syntax::TerminationKind::Separated: {
+children.push_back(elementWithoutDelimiter);
+break;
+  }
+  case syntax::TerminationKind::Terminated:
+  case syntax::TerminationKind::MaybeTerminated: {
+if (elementWithoutDelimiter) {
+  children.push_back(elementWithoutDelimiter);
+}
+break;
+  }
+  }
+
+  return children;
+}
+
+// The methods below can't be implemented without additional information we
+// have. Should we give them default permissive values?
+
+clang::tok::TokenKind syntax::List::getDelimiterTokenKind() {
+  return clang::tok::TokenKind::unknown;
+}
+syntax::TerminationKind syntax::List::getTerminationKind() {
+  return syntax::TerminationKind::MaybeTerminated;
+}
+bool syntax::List::canBeEmpty() { return true; }
Index: clang/lib/Tooling/Syntax/Nodes.cpp
===
--- clang/lib/Tooling/Syntax/Nodes.cpp
+++ clang/lib/Tooling/Syntax/Nodes.cpp
@@ -144,6 +144,10 @@
 return OS << "ExternKeyword";
   case syntax::NodeRole::BodyStatement:
 return OS << "BodyStatement";
+  case syntax::NodeRole::List_element:
+return OS << "List_element";
+  case syntax::NodeRole::List_delimiter:
+return OS << "List_delimiter";
   case syntax::NodeRole::CaseStatement_value:
 return OS << "CaseStatement_value";
   case syntax::NodeRole::IfStatement_thenStatement:
Index: clang/include/clang/Tooling/Syntax/Tree.h
===
--- clang/include/clang/Tooling/Syntax/Tree.h
+++ clang/include/clang/Tooling/Syntax/Tree.h
@@ -191,6 +191,47 @@
   Node *FirstChild = nullptr;
 };
 
+enum TerminationKind {
+  Terminated,
+  MaybeTerminated,
+  Separated,
+};
+
+class List : public Tree {
+  template  struct ElementAndDelimiter {
+Element *element;
+Leaf *delimiter;
+  };
+
+  /// Returns the elements and corresponding delimiters. Missing elements
+  /// and delimiters are represented as null pointers.
+  ///
+  /// For example, in a separated list:
+  /// "a, b, c" <=> [("a", ","), ("b", ","), ("c", null)]
+  /// "a, , c" <=> [("a", ","), (null, 

[PATCH] D85185: [SyntaxTree] Add test coverage for `->*` operator

[Eduardo Caldas via Phabricator via cfe-commits]

This revision was landed with ongoing or failed builds.
This revision was automatically updated to reflect the committed changes.
Closed by commit rGc5cdc3e801ad: [SyntaxTree] Add test coverage for `-*` 
operator (authored by eduucaldas).

Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D85185/new/

https://reviews.llvm.org/D85185

Files:
  clang/unittests/Tooling/Syntax/TreeTest.cpp


Index: clang/unittests/Tooling/Syntax/TreeTest.cpp
===
--- clang/unittests/Tooling/Syntax/TreeTest.cpp
+++ clang/unittests/Tooling/Syntax/TreeTest.cpp
@@ -2329,16 +2329,17 @@
   friend bool operator<(const X&, const X&);
   friend X operator<<(X&, const X&);
   X operator,(X&);
-  // TODO: Fix crash on member function pointer and add a test for `->*`
-  // TODO: Unbox operators in syntax tree. 
+  X operator->*(int);
+  // TODO: Unbox operators in syntax tree.
   // Represent operators by `+` instead of `IdExpression-UnqualifiedId-+`
 };
-void test(X x, X y) {
+void test(X x, X y, X* xp, int X::* pmi) {
   x = y;
   x + y;
   x < y;
   x << y;
   x, y;
+  xp->*pmi;
 }
 )cpp",
   R"txt(
@@ -2437,6 +2438,17 @@
 | | |   |   `-&
 | | |   `-)
 | | `-;
+| |-SimpleDeclaration
+| | |-X
+| | |-SimpleDeclarator
+| | | |-operator
+| | | |-->*
+| | | `-ParametersAndQualifiers
+| | |   |-(
+| | |   |-SimpleDeclaration
+| | |   | `-int
+| | |   `-)
+| | `-;
 | |-}
 | `-;
 `-SimpleDeclaration
@@ -2454,6 +2466,21 @@
   |   | |-X
   |   | `-SimpleDeclarator
   |   |   `-y
+  |   |-,
+  |   |-SimpleDeclaration
+  |   | |-X
+  |   | `-SimpleDeclarator
+  |   |   |-*
+  |   |   `-xp
+  |   |-,
+  |   |-SimpleDeclaration
+  |   | |-int
+  |   | `-SimpleDeclarator
+  |   |   |-MemberPointer
+  |   |   | |-X
+  |   |   | |-::
+  |   |   | `-*
+  |   |   `-pmi
   |   `-)
   `-CompoundStatement
 |-{
@@ -2518,6 +2545,16 @@
 | |   `-UnqualifiedId
 | | `-y
 | `-;
+|-ExpressionStatement
+| |-BinaryOperatorExpression
+| | |-IdExpression
+| | | `-UnqualifiedId
+| | |   `-xp
+| | |-->*
+| | `-IdExpression
+| |   `-UnqualifiedId
+| | `-pmi
+| `-;
 `-}
 )txt"));
 }


Index: clang/unittests/Tooling/Syntax/TreeTest.cpp
===
--- clang/unittests/Tooling/Syntax/TreeTest.cpp
+++ clang/unittests/Tooling/Syntax/TreeTest.cpp
@@ -2329,16 +2329,17 @@
   friend bool operator<(const X&, const X&);
   friend X operator<<(X&, const X&);
   X operator,(X&);
-  // TODO: Fix crash on member function pointer and add a test for `->*`
-  // TODO: Unbox operators in syntax tree. 
+  X operator->*(int);
+  // TODO: Unbox operators in syntax tree.
   // Represent operators by `+` instead of `IdExpression-UnqualifiedId-+`
 };
-void test(X x, X y) {
+void test(X x, X y, X* xp, int X::* pmi) {
   x = y;
   x + y;
   x < y;
   x << y;
   x, y;
+  xp->*pmi;
 }
 )cpp",
   R"txt(
@@ -2437,6 +2438,17 @@
 | | |   |   `-&
 | | |   `-)
 | | `-;
+| |-SimpleDeclaration
+| | |-X
+| | |-SimpleDeclarator
+| | | |-operator
+| | | |-->*
+| | | `-ParametersAndQualifiers
+| | |   |-(
+| | |   |-SimpleDeclaration
+| | |   | `-int
+| | |   `-)
+| | `-;
 | |-}
 | `-;
 `-SimpleDeclaration
@@ -2454,6 +2466,21 @@
   |   | |-X
   |   | `-SimpleDeclarator
   |   |   `-y
+  |   |-,
+  |   |-SimpleDeclaration
+  |   | |-X
+  |   | `-SimpleDeclarator
+  |   |   |-*
+  |   |   `-xp
+  |   |-,
+  |   |-SimpleDeclaration
+  |   | |-int
+  |   | `-SimpleDeclarator
+  |   |   |-MemberPointer
+  |   |   | |-X
+  |   |   | |-::
+  |   |   | `-*
+  |   |   `-pmi
   |   `-)
   `-CompoundStatement
 |-{
@@ -2518,6 +2545,16 @@
 | |   `-UnqualifiedId
 | | `-y
 | `-;
+|-ExpressionStatement
+| |-BinaryOperatorExpression
+| | |-IdExpression
+| | | `-UnqualifiedId
+| | |   `-xp
+| | |-->*
+| | `-IdExpression
+| |   `-UnqualifiedId
+| | `-pmi
+| `-;
 `-}
 )txt"));
 }
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D85185: [SyntaxTree] Add test coverage for `->*` operator

[Eduardo Caldas via Phabricator via cfe-commits]

eduucaldas updated this revision to Diff 283144.
eduucaldas added a comment.

Answer comments


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D85185/new/

https://reviews.llvm.org/D85185

Files:
  clang/unittests/Tooling/Syntax/TreeTest.cpp


Index: clang/unittests/Tooling/Syntax/TreeTest.cpp
===
--- clang/unittests/Tooling/Syntax/TreeTest.cpp
+++ clang/unittests/Tooling/Syntax/TreeTest.cpp
@@ -2322,16 +2322,17 @@
   friend bool operator<(const X&, const X&);
   friend X operator<<(X&, const X&);
   X operator,(X&);
-  // TODO: Fix crash on member function pointer and add a test for `->*`
-  // TODO: Unbox operators in syntax tree. 
+  X operator->*(int);
+  // TODO: Unbox operators in syntax tree.
   // Represent operators by `+` instead of `IdExpression-UnqualifiedId-+`
 };
-void test(X x, X y) {
+void test(X x, X y, X* xp, int X::* pmi) {
   x = y;
   x + y;
   x < y;
   x << y;
   x, y;
+  xp->*pmi;
 }
 )cpp",
   R"txt(
@@ -2430,6 +2431,17 @@
 | | |   |   `-&
 | | |   `-)
 | | `-;
+| |-SimpleDeclaration
+| | |-X
+| | |-SimpleDeclarator
+| | | |-operator
+| | | |-->*
+| | | `-ParametersAndQualifiers
+| | |   |-(
+| | |   |-SimpleDeclaration
+| | |   | `-int
+| | |   `-)
+| | `-;
 | |-}
 | `-;
 `-SimpleDeclaration
@@ -2447,6 +2459,21 @@
   |   | |-X
   |   | `-SimpleDeclarator
   |   |   `-y
+  |   |-,
+  |   |-SimpleDeclaration
+  |   | |-X
+  |   | `-SimpleDeclarator
+  |   |   |-*
+  |   |   `-xp
+  |   |-,
+  |   |-SimpleDeclaration
+  |   | |-int
+  |   | `-SimpleDeclarator
+  |   |   |-MemberPointer
+  |   |   | |-X
+  |   |   | |-::
+  |   |   | `-*
+  |   |   `-pmi
   |   `-)
   `-CompoundStatement
 |-{
@@ -2511,6 +2538,16 @@
 | |   `-UnqualifiedId
 | | `-y
 | `-;
+|-ExpressionStatement
+| |-BinaryOperatorExpression
+| | |-IdExpression
+| | | `-UnqualifiedId
+| | |   `-xp
+| | |-->*
+| | `-IdExpression
+| |   `-UnqualifiedId
+| | `-pmi
+| `-;
 `-}
 )txt"));
 }


Index: clang/unittests/Tooling/Syntax/TreeTest.cpp
===
--- clang/unittests/Tooling/Syntax/TreeTest.cpp
+++ clang/unittests/Tooling/Syntax/TreeTest.cpp
@@ -2322,16 +2322,17 @@
   friend bool operator<(const X&, const X&);
   friend X operator<<(X&, const X&);
   X operator,(X&);
-  // TODO: Fix crash on member function pointer and add a test for `->*`
-  // TODO: Unbox operators in syntax tree. 
+  X operator->*(int);
+  // TODO: Unbox operators in syntax tree.
   // Represent operators by `+` instead of `IdExpression-UnqualifiedId-+`
 };
-void test(X x, X y) {
+void test(X x, X y, X* xp, int X::* pmi) {
   x = y;
   x + y;
   x < y;
   x << y;
   x, y;
+  xp->*pmi;
 }
 )cpp",
   R"txt(
@@ -2430,6 +2431,17 @@
 | | |   |   `-&
 | | |   `-)
 | | `-;
+| |-SimpleDeclaration
+| | |-X
+| | |-SimpleDeclarator
+| | | |-operator
+| | | |-->*
+| | | `-ParametersAndQualifiers
+| | |   |-(
+| | |   |-SimpleDeclaration
+| | |   | `-int
+| | |   `-)
+| | `-;
 | |-}
 | `-;
 `-SimpleDeclaration
@@ -2447,6 +2459,21 @@
   |   | |-X
   |   | `-SimpleDeclarator
   |   |   `-y
+  |   |-,
+  |   |-SimpleDeclaration
+  |   | |-X
+  |   | `-SimpleDeclarator
+  |   |   |-*
+  |   |   `-xp
+  |   |-,
+  |   |-SimpleDeclaration
+  |   | |-int
+  |   | `-SimpleDeclarator
+  |   |   |-MemberPointer
+  |   |   | |-X
+  |   |   | |-::
+  |   |   | `-*
+  |   |   `-pmi
   |   `-)
   `-CompoundStatement
 |-{
@@ -2511,6 +2538,16 @@
 | |   `-UnqualifiedId
 | | `-y
 | `-;
+|-ExpressionStatement
+| |-BinaryOperatorExpression
+| | |-IdExpression
+| | | `-UnqualifiedId
+| | |   `-xp
+| | |-->*
+| | `-IdExpression
+| |   `-UnqualifiedId
+| | `-pmi
+| `-;
 `-}
 )txt"));
 }
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D85146: [SyntaxTree] Fix crash on pointer to member function

[Eduardo Caldas via Phabricator via cfe-commits]

This revision was automatically updated to reflect the committed changes.
Closed by commit rG8ce15f7eeb12: [SyntaxTree] Fix crash on pointer to member 
function (authored by eduucaldas).

Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D85146/new/

https://reviews.llvm.org/D85146

Files:
  clang/lib/Tooling/Syntax/BuildTree.cpp
  clang/unittests/Tooling/Syntax/TreeTest.cpp

Index: clang/unittests/Tooling/Syntax/TreeTest.cpp
===
--- clang/unittests/Tooling/Syntax/TreeTest.cpp
+++ clang/unittests/Tooling/Syntax/TreeTest.cpp
@@ -4074,6 +4074,99 @@
 )txt"));
 }
 
+TEST_P(SyntaxTreeTest, MemberFunctionPointer) {
+  if (!GetParam().isCXX()) {
+return;
+  }
+  EXPECT_TRUE(treeDumpEqual(
+  R"cpp(
+struct X {
+  struct Y {};
+};
+void (X::*xp)();
+void (X::**xpp)(const int*);
+// FIXME: Generate the right syntax tree for this type,
+// i.e. create a syntax node for the outer member pointer
+void (X::Y::*xyp)(const int*, char);
+)cpp",
+  R"txt(
+*: TranslationUnit
+|-SimpleDeclaration
+| |-struct
+| |-X
+| |-{
+| |-SimpleDeclaration
+| | |-struct
+| | |-Y
+| | |-{
+| | |-}
+| | `-;
+| |-}
+| `-;
+|-SimpleDeclaration
+| |-void
+| |-SimpleDeclarator
+| | |-ParenDeclarator
+| | | |-(
+| | | |-MemberPointer
+| | | | |-X
+| | | | |-::
+| | | | `-*
+| | | |-xp
+| | | `-)
+| | `-ParametersAndQualifiers
+| |   |-(
+| |   `-)
+| `-;
+|-SimpleDeclaration
+| |-void
+| |-SimpleDeclarator
+| | |-ParenDeclarator
+| | | |-(
+| | | |-MemberPointer
+| | | | |-X
+| | | | |-::
+| | | | `-*
+| | | |-*
+| | | |-xpp
+| | | `-)
+| | `-ParametersAndQualifiers
+| |   |-(
+| |   |-SimpleDeclaration
+| |   | |-const
+| |   | |-int
+| |   | `-SimpleDeclarator
+| |   |   `-*
+| |   `-)
+| `-;
+`-SimpleDeclaration
+  |-void
+  |-SimpleDeclarator
+  | |-ParenDeclarator
+  | | |-(
+  | | |-X
+  | | |-::
+  | | |-MemberPointer
+  | | | |-Y
+  | | | |-::
+  | | | `-*
+  | | |-xyp
+  | | `-)
+  | `-ParametersAndQualifiers
+  |   |-(
+  |   |-SimpleDeclaration
+  |   | |-const
+  |   | |-int
+  |   | `-SimpleDeclarator
+  |   |   `-*
+  |   |-,
+  |   |-SimpleDeclaration
+  |   | `-char
+  |   `-)
+  `-;
+)txt"));
+}
+
 TEST_P(SyntaxTreeTest, ComplexDeclarator) {
   EXPECT_TRUE(treeDumpEqual(
   R"cpp(
Index: clang/lib/Tooling/Syntax/BuildTree.cpp
===
--- clang/lib/Tooling/Syntax/BuildTree.cpp
+++ clang/lib/Tooling/Syntax/BuildTree.cpp
@@ -939,6 +939,8 @@
 return true;
   }
 
+  // FIXME: Deleting the `TraverseParenTypeLoc` override doesn't change test
+  // results. Find test coverage or remove it.
   bool TraverseParenTypeLoc(ParenTypeLoc L) {
 // We reverse order of traversal to get the proper syntax structure.
 if (!WalkUpFromParenTypeLoc(L))
@@ -987,6 +989,16 @@
 return WalkUpFromFunctionTypeLoc(L);
   }
 
+  bool TraverseMemberPointerTypeLoc(MemberPointerTypeLoc L) {
+// In the source code "void (Y::*mp)()" `MemberPointerTypeLoc` corresponds
+// to "Y::*" but it points to a `ParenTypeLoc` that corresponds to
+// "(Y::*mp)" We thus reverse the order of traversal to get the proper
+// syntax structure.
+if (!WalkUpFromMemberPointerTypeLoc(L))
+  return false;
+return TraverseTypeLoc(L.getPointeeLoc());
+  }
+
   bool WalkUpFromMemberPointerTypeLoc(MemberPointerTypeLoc L) {
 auto SR = L.getLocalSourceRange();
 Builder.foldNode(Builder.getRange(SR),
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D85186: [SyntaxTree] Add support for `LiteralExpression`

[Eduardo Caldas via Phabricator via cfe-commits]

This revision was landed with ongoing or failed builds.
This revision was automatically updated to reflect the committed changes.
Closed by commit rG860cbbdd6b84: [SyntaxTree] Add support for 
`LiteralExpression` (authored by eduucaldas).

Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D85186/new/

https://reviews.llvm.org/D85186

Files:
  clang/include/clang/Tooling/Syntax/Nodes.h
  clang/lib/Tooling/Syntax/Nodes.cpp

Index: clang/lib/Tooling/Syntax/Nodes.cpp
===
--- clang/lib/Tooling/Syntax/Nodes.cpp
+++ clang/lib/Tooling/Syntax/Nodes.cpp
@@ -230,37 +230,7 @@
   findChild(syntax::NodeRole::CloseParen));
 }
 
-syntax::Leaf *syntax::IntegerLiteralExpression::literalToken() {
-  return llvm::cast_or_null(
-  findChild(syntax::NodeRole::LiteralToken));
-}
-
-syntax::Leaf *syntax::CharacterLiteralExpression::literalToken() {
-  return llvm::cast_or_null(
-  findChild(syntax::NodeRole::LiteralToken));
-}
-
-syntax::Leaf *syntax::FloatingLiteralExpression::literalToken() {
-  return llvm::cast_or_null(
-  findChild(syntax::NodeRole::LiteralToken));
-}
-
-syntax::Leaf *syntax::StringLiteralExpression::literalToken() {
-  return llvm::cast_or_null(
-  findChild(syntax::NodeRole::LiteralToken));
-}
-
-syntax::Leaf *syntax::BoolLiteralExpression::literalToken() {
-  return llvm::cast_or_null(
-  findChild(syntax::NodeRole::LiteralToken));
-}
-
-syntax::Leaf *syntax::CxxNullPtrExpression::nullPtrKeyword() {
-  return llvm::cast_or_null(
-  findChild(syntax::NodeRole::LiteralToken));
-}
-
-syntax::Leaf *syntax::UserDefinedLiteralExpression::literalToken() {
+syntax::Leaf *syntax::LiteralExpression::literalToken() {
   return llvm::cast_or_null(
   findChild(syntax::NodeRole::LiteralToken));
 }
Index: clang/include/clang/Tooling/Syntax/Nodes.h
===
--- clang/include/clang/Tooling/Syntax/Nodes.h
+++ clang/include/clang/Tooling/Syntax/Nodes.h
@@ -267,66 +267,82 @@
   syntax::Leaf *closeParen();
 };
 
+/// Expression for literals. C++ [lex.literal]
+class LiteralExpression : public Expression {
+public:
+  LiteralExpression(NodeKind K) : Expression(K) {}
+  static bool classof(const Node *N) {
+return N->kind() == NodeKind::IntegerLiteralExpression ||
+   N->kind() == NodeKind::CharacterLiteralExpression ||
+   N->kind() == NodeKind::FloatingLiteralExpression ||
+   N->kind() == NodeKind::StringLiteralExpression ||
+   N->kind() == NodeKind::BoolLiteralExpression ||
+   N->kind() == NodeKind::CxxNullPtrExpression ||
+   N->kind() == NodeKind::IntegerUserDefinedLiteralExpression ||
+   N->kind() == NodeKind::FloatUserDefinedLiteralExpression ||
+   N->kind() == NodeKind::CharUserDefinedLiteralExpression ||
+   N->kind() == NodeKind::StringUserDefinedLiteralExpression;
+  }
+  syntax::Leaf *literalToken();
+};
+
 /// Expression for integer literals. C++ [lex.icon]
-class IntegerLiteralExpression final : public Expression {
+class IntegerLiteralExpression final : public LiteralExpression {
 public:
-  IntegerLiteralExpression() : Expression(NodeKind::IntegerLiteralExpression) {}
+  IntegerLiteralExpression()
+  : LiteralExpression(NodeKind::IntegerLiteralExpression) {}
   static bool classof(const Node *N) {
 return N->kind() == NodeKind::IntegerLiteralExpression;
   }
-  syntax::Leaf *literalToken();
 };
 
 /// Expression for character literals. C++ [lex.ccon]
-class CharacterLiteralExpression final : public Expression {
+class CharacterLiteralExpression final : public LiteralExpression {
 public:
   CharacterLiteralExpression()
-  : Expression(NodeKind::CharacterLiteralExpression) {}
+  : LiteralExpression(NodeKind::CharacterLiteralExpression) {}
   static bool classof(const Node *N) {
 return N->kind() == NodeKind::CharacterLiteralExpression;
   }
-  syntax::Leaf *literalToken();
 };
 
 /// Expression for floating-point literals. C++ [lex.fcon]
-class FloatingLiteralExpression final : public Expression {
+class FloatingLiteralExpression final : public LiteralExpression {
 public:
   FloatingLiteralExpression()
-  : Expression(NodeKind::FloatingLiteralExpression) {}
+  : LiteralExpression(NodeKind::FloatingLiteralExpression) {}
   static bool classof(const Node *N) {
 return N->kind() == NodeKind::FloatingLiteralExpression;
   }
-  syntax::Leaf *literalToken();
 };
 
 /// Expression for string-literals. C++ [lex.string]
-class StringLiteralExpression final : public Expression {
+class StringLiteralExpression final : public LiteralExpression {
 public:
-  StringLiteralExpression() : Expression(NodeKind::StringLiteralExpression) {}
+  StringLiteralExpression()
+  : LiteralExpression(NodeKind::StringLiteralExpression) {}
   static bool classof(const Node *N) {
 return N->kind() == 

[PATCH] D85186: [SyntaxTree] Add support for `LiteralExpression`

[Eduardo Caldas via Phabricator via cfe-commits]

eduucaldas created this revision.
Herald added a project: clang.
Herald added a subscriber: cfe-commits.
eduucaldas requested review of this revision.

We use inheritance to model the grammar's disjunction rule:
literal:

  integer-literal
  character-literal
  floating-point-literal
  string-literal
  boolean-literal
  pointer-literal
  user-defined-literal


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D85186

Files:
  clang/include/clang/Tooling/Syntax/Nodes.h
  clang/lib/Tooling/Syntax/Nodes.cpp

Index: clang/lib/Tooling/Syntax/Nodes.cpp
===
--- clang/lib/Tooling/Syntax/Nodes.cpp
+++ clang/lib/Tooling/Syntax/Nodes.cpp
@@ -230,37 +230,7 @@
   findChild(syntax::NodeRole::CloseParen));
 }
 
-syntax::Leaf *syntax::IntegerLiteralExpression::literalToken() {
-  return llvm::cast_or_null(
-  findChild(syntax::NodeRole::LiteralToken));
-}
-
-syntax::Leaf *syntax::CharacterLiteralExpression::literalToken() {
-  return llvm::cast_or_null(
-  findChild(syntax::NodeRole::LiteralToken));
-}
-
-syntax::Leaf *syntax::FloatingLiteralExpression::literalToken() {
-  return llvm::cast_or_null(
-  findChild(syntax::NodeRole::LiteralToken));
-}
-
-syntax::Leaf *syntax::StringLiteralExpression::literalToken() {
-  return llvm::cast_or_null(
-  findChild(syntax::NodeRole::LiteralToken));
-}
-
-syntax::Leaf *syntax::BoolLiteralExpression::literalToken() {
-  return llvm::cast_or_null(
-  findChild(syntax::NodeRole::LiteralToken));
-}
-
-syntax::Leaf *syntax::CxxNullPtrExpression::nullPtrKeyword() {
-  return llvm::cast_or_null(
-  findChild(syntax::NodeRole::LiteralToken));
-}
-
-syntax::Leaf *syntax::UserDefinedLiteralExpression::literalToken() {
+syntax::Leaf *syntax::LiteralExpression::literalToken() {
   return llvm::cast_or_null(
   findChild(syntax::NodeRole::LiteralToken));
 }
Index: clang/include/clang/Tooling/Syntax/Nodes.h
===
--- clang/include/clang/Tooling/Syntax/Nodes.h
+++ clang/include/clang/Tooling/Syntax/Nodes.h
@@ -267,66 +267,82 @@
   syntax::Leaf *closeParen();
 };
 
+/// Expression for literals. C++ [lex.literal]
+class LiteralExpression : public Expression {
+public:
+  LiteralExpression(NodeKind K) : Expression(K) {}
+  static bool classof(const Node *N) {
+return N->kind() == NodeKind::IntegerLiteralExpression ||
+   N->kind() == NodeKind::CharacterLiteralExpression ||
+   N->kind() == NodeKind::FloatingLiteralExpression ||
+   N->kind() == NodeKind::StringLiteralExpression ||
+   N->kind() == NodeKind::BoolLiteralExpression ||
+   N->kind() == NodeKind::CxxNullPtrExpression ||
+   N->kind() == NodeKind::IntegerUserDefinedLiteralExpression ||
+   N->kind() == NodeKind::FloatUserDefinedLiteralExpression ||
+   N->kind() == NodeKind::CharUserDefinedLiteralExpression ||
+   N->kind() == NodeKind::StringUserDefinedLiteralExpression;
+  }
+  syntax::Leaf *literalToken();
+};
+
 /// Expression for integer literals. C++ [lex.icon]
-class IntegerLiteralExpression final : public Expression {
+class IntegerLiteralExpression final : public LiteralExpression {
 public:
-  IntegerLiteralExpression() : Expression(NodeKind::IntegerLiteralExpression) {}
+  IntegerLiteralExpression()
+  : LiteralExpression(NodeKind::IntegerLiteralExpression) {}
   static bool classof(const Node *N) {
 return N->kind() == NodeKind::IntegerLiteralExpression;
   }
-  syntax::Leaf *literalToken();
 };
 
 /// Expression for character literals. C++ [lex.ccon]
-class CharacterLiteralExpression final : public Expression {
+class CharacterLiteralExpression final : public LiteralExpression {
 public:
   CharacterLiteralExpression()
-  : Expression(NodeKind::CharacterLiteralExpression) {}
+  : LiteralExpression(NodeKind::CharacterLiteralExpression) {}
   static bool classof(const Node *N) {
 return N->kind() == NodeKind::CharacterLiteralExpression;
   }
-  syntax::Leaf *literalToken();
 };
 
 /// Expression for floating-point literals. C++ [lex.fcon]
-class FloatingLiteralExpression final : public Expression {
+class FloatingLiteralExpression final : public LiteralExpression {
 public:
   FloatingLiteralExpression()
-  : Expression(NodeKind::FloatingLiteralExpression) {}
+  : LiteralExpression(NodeKind::FloatingLiteralExpression) {}
   static bool classof(const Node *N) {
 return N->kind() == NodeKind::FloatingLiteralExpression;
   }
-  syntax::Leaf *literalToken();
 };
 
 /// Expression for string-literals. C++ [lex.string]
-class StringLiteralExpression final : public Expression {
+class StringLiteralExpression final : public LiteralExpression {
 public:
-  StringLiteralExpression() : Expression(NodeKind::StringLiteralExpression) {}
+  StringLiteralExpression()
+  : LiteralExpression(NodeKind::StringLiteralExpression) {}
   static bool classof(const Node *N) 

[PATCH] D85185: [SyntaxTree] Add test coverage for `->*` operator

[Eduardo Caldas via Phabricator via cfe-commits]

eduucaldas created this revision.
Herald added a project: clang.
Herald added a subscriber: cfe-commits.
eduucaldas requested review of this revision.

This was the last binary operator that we supported but didn't have any
test coverage. The recent fix in a crash in member pointers allowed us
to add this test.


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D85185

Files:
  clang/unittests/Tooling/Syntax/TreeTest.cpp


Index: clang/unittests/Tooling/Syntax/TreeTest.cpp
===
--- clang/unittests/Tooling/Syntax/TreeTest.cpp
+++ clang/unittests/Tooling/Syntax/TreeTest.cpp
@@ -2322,16 +2322,18 @@
   friend bool operator<(const X&, const X&);
   friend X operator<<(X&, const X&);
   X operator,(X&);
-  // TODO: Fix crash on member function pointer and add a test for `->*`
-  // TODO: Unbox operators in syntax tree. 
+  X operator->*(int);
+  // TODO: Unbox operators in syntax tree.
   // Represent operators by `+` instead of `IdExpression-UnqualifiedId-+`
 };
-void test(X x, X y) {
+int X::*pmi;
+void test(X x, X y, X* xp) {
   x = y;
   x + y;
   x < y;
   x << y;
   x, y;
+  xp->*pmi;
 }
 )cpp",
   R"txt(
@@ -2430,8 +2432,28 @@
 | | |   |   `-&
 | | |   `-)
 | | `-;
+| |-SimpleDeclaration
+| | |-X
+| | |-SimpleDeclarator
+| | | |-operator
+| | | |-->*
+| | | `-ParametersAndQualifiers
+| | |   |-(
+| | |   |-SimpleDeclaration
+| | |   | `-int
+| | |   `-)
+| | `-;
 | |-}
 | `-;
+|-SimpleDeclaration
+| |-int
+| |-SimpleDeclarator
+| | |-MemberPointer
+| | | |-X
+| | | |-::
+| | | `-*
+| | `-pmi
+| `-;
 `-SimpleDeclaration
   |-void
   |-SimpleDeclarator
@@ -2447,6 +2469,12 @@
   |   | |-X
   |   | `-SimpleDeclarator
   |   |   `-y
+  |   |-,
+  |   |-SimpleDeclaration
+  |   | |-X
+  |   | `-SimpleDeclarator
+  |   |   |-*
+  |   |   `-xp
   |   `-)
   `-CompoundStatement
 |-{
@@ -2511,6 +2539,16 @@
 | |   `-UnqualifiedId
 | | `-y
 | `-;
+|-ExpressionStatement
+| |-BinaryOperatorExpression
+| | |-IdExpression
+| | | `-UnqualifiedId
+| | |   `-xp
+| | |-->*
+| | `-IdExpression
+| |   `-UnqualifiedId
+| | `-pmi
+| `-;
 `-}
 )txt"));
 }


Index: clang/unittests/Tooling/Syntax/TreeTest.cpp
===
--- clang/unittests/Tooling/Syntax/TreeTest.cpp
+++ clang/unittests/Tooling/Syntax/TreeTest.cpp
@@ -2322,16 +2322,18 @@
   friend bool operator<(const X&, const X&);
   friend X operator<<(X&, const X&);
   X operator,(X&);
-  // TODO: Fix crash on member function pointer and add a test for `->*`
-  // TODO: Unbox operators in syntax tree. 
+  X operator->*(int);
+  // TODO: Unbox operators in syntax tree.
   // Represent operators by `+` instead of `IdExpression-UnqualifiedId-+`
 };
-void test(X x, X y) {
+int X::*pmi;
+void test(X x, X y, X* xp) {
   x = y;
   x + y;
   x < y;
   x << y;
   x, y;
+  xp->*pmi;
 }
 )cpp",
   R"txt(
@@ -2430,8 +2432,28 @@
 | | |   |   `-&
 | | |   `-)
 | | `-;
+| |-SimpleDeclaration
+| | |-X
+| | |-SimpleDeclarator
+| | | |-operator
+| | | |-->*
+| | | `-ParametersAndQualifiers
+| | |   |-(
+| | |   |-SimpleDeclaration
+| | |   | `-int
+| | |   `-)
+| | `-;
 | |-}
 | `-;
+|-SimpleDeclaration
+| |-int
+| |-SimpleDeclarator
+| | |-MemberPointer
+| | | |-X
+| | | |-::
+| | | `-*
+| | `-pmi
+| `-;
 `-SimpleDeclaration
   |-void
   |-SimpleDeclarator
@@ -2447,6 +2469,12 @@
   |   | |-X
   |   | `-SimpleDeclarator
   |   |   `-y
+  |   |-,
+  |   |-SimpleDeclaration
+  |   | |-X
+  |   | `-SimpleDeclarator
+  |   |   |-*
+  |   |   `-xp
   |   `-)
   `-CompoundStatement
 |-{
@@ -2511,6 +2539,16 @@
 | |   `-UnqualifiedId
 | | `-y
 | `-;
+|-ExpressionStatement
+| |-BinaryOperatorExpression
+| | |-IdExpression
+| | | `-UnqualifiedId
+| | |   `-xp
+| | |-->*
+| | `-IdExpression
+| |   `-UnqualifiedId
+| | `-pmi
+| `-;
 `-}
 )txt"));
 }
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D85146: [SyntaxTree] Fix crash on pointer to member function

[Eduardo Caldas via Phabricator via cfe-commits]

eduucaldas updated this revision to Diff 282814.
eduucaldas marked 2 inline comments as done.
eduucaldas added a comment.

answer comments


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D85146/new/

https://reviews.llvm.org/D85146

Files:
  clang/lib/Tooling/Syntax/BuildTree.cpp
  clang/unittests/Tooling/Syntax/TreeTest.cpp

Index: clang/unittests/Tooling/Syntax/TreeTest.cpp
===
--- clang/unittests/Tooling/Syntax/TreeTest.cpp
+++ clang/unittests/Tooling/Syntax/TreeTest.cpp
@@ -4067,6 +4067,99 @@
 )txt"));
 }
 
+TEST_P(SyntaxTreeTest, MemberFunctionPointer) {
+  if (!GetParam().isCXX()) {
+return;
+  }
+  EXPECT_TRUE(treeDumpEqual(
+  R"cpp(
+struct X {
+  struct Y {};
+};
+void (X::*xp)();
+void (X::**xpp)(const int*);
+// FIXME: Generate the right syntax tree for this type,
+// i.e. create a syntax node for the outer member pointer
+void (X::Y::*xyp)(const int*, char);
+)cpp",
+  R"txt(
+*: TranslationUnit
+|-SimpleDeclaration
+| |-struct
+| |-X
+| |-{
+| |-SimpleDeclaration
+| | |-struct
+| | |-Y
+| | |-{
+| | |-}
+| | `-;
+| |-}
+| `-;
+|-SimpleDeclaration
+| |-void
+| |-SimpleDeclarator
+| | |-ParenDeclarator
+| | | |-(
+| | | |-MemberPointer
+| | | | |-X
+| | | | |-::
+| | | | `-*
+| | | |-xp
+| | | `-)
+| | `-ParametersAndQualifiers
+| |   |-(
+| |   `-)
+| `-;
+|-SimpleDeclaration
+| |-void
+| |-SimpleDeclarator
+| | |-ParenDeclarator
+| | | |-(
+| | | |-MemberPointer
+| | | | |-X
+| | | | |-::
+| | | | `-*
+| | | |-*
+| | | |-xpp
+| | | `-)
+| | `-ParametersAndQualifiers
+| |   |-(
+| |   |-SimpleDeclaration
+| |   | |-const
+| |   | |-int
+| |   | `-SimpleDeclarator
+| |   |   `-*
+| |   `-)
+| `-;
+`-SimpleDeclaration
+  |-void
+  |-SimpleDeclarator
+  | |-ParenDeclarator
+  | | |-(
+  | | |-X
+  | | |-::
+  | | |-MemberPointer
+  | | | |-Y
+  | | | |-::
+  | | | `-*
+  | | |-xyp
+  | | `-)
+  | `-ParametersAndQualifiers
+  |   |-(
+  |   |-SimpleDeclaration
+  |   | |-const
+  |   | |-int
+  |   | `-SimpleDeclarator
+  |   |   `-*
+  |   |-,
+  |   |-SimpleDeclaration
+  |   | `-char
+  |   `-)
+  `-;
+)txt"));
+}
+
 TEST_P(SyntaxTreeTest, ComplexDeclarator) {
   EXPECT_TRUE(treeDumpEqual(
   R"cpp(
Index: clang/lib/Tooling/Syntax/BuildTree.cpp
===
--- clang/lib/Tooling/Syntax/BuildTree.cpp
+++ clang/lib/Tooling/Syntax/BuildTree.cpp
@@ -939,6 +939,8 @@
 return true;
   }
 
+  // FIXME: Deleting the `TraverseParenTypeLoc` override doesn't change test
+  // results. Find test coverage or remove it.
   bool TraverseParenTypeLoc(ParenTypeLoc L) {
 // We reverse order of traversal to get the proper syntax structure.
 if (!WalkUpFromParenTypeLoc(L))
@@ -987,6 +989,16 @@
 return WalkUpFromFunctionTypeLoc(L);
   }
 
+  bool TraverseMemberPointerTypeLoc(MemberPointerTypeLoc L) {
+// In the source code "void (Y::*mp)()" `MemberPointerTypeLoc` corresponds
+// to "Y::*" but it points to a `ParenTypeLoc` that corresponds to
+// "(Y::*mp)" We thus reverse the order of traversal to get the proper
+// syntax structure.
+if (!WalkUpFromMemberPointerTypeLoc(L))
+  return false;
+return TraverseTypeLoc(L.getPointeeLoc());
+  }
+
   bool WalkUpFromMemberPointerTypeLoc(MemberPointerTypeLoc L) {
 auto SR = L.getLocalSourceRange();
 Builder.foldNode(Builder.getRange(SR),
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D85146: [SyntaxTree] Fix crash on pointer to member function

[Eduardo Caldas via Phabricator via cfe-commits]

eduucaldas added a comment.

Additionally deleting TraverseParenTypeLoc changes nothing in the tests. I can 
also investigate that, I can try to come up with a test that makes this 
Traverse override relevant and if I'm not able, I would remove this override.




Comment at: clang/unittests/Tooling/Syntax/TreeTest.cpp:4080-4081
+void (X::*xp)();
+// FIXME: Generate the right syntax tree for this type,
+// i.e. create a syntax node for the outer member pointer
+void (X::Y::**xyp)(const int*, char);

I spent 15 minutes investigating that, if you think I should go further I will.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D85146/new/

https://reviews.llvm.org/D85146

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D85146: [SyntaxTree] Fix crash on pointer to member function

[Eduardo Caldas via Phabricator via cfe-commits]

eduucaldas created this revision.
Herald added a project: clang.
Herald added a subscriber: cfe-commits.
eduucaldas requested review of this revision.

Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D85146

Files:
  clang/lib/Tooling/Syntax/BuildTree.cpp
  clang/unittests/Tooling/Syntax/TreeTest.cpp

Index: clang/unittests/Tooling/Syntax/TreeTest.cpp
===
--- clang/unittests/Tooling/Syntax/TreeTest.cpp
+++ clang/unittests/Tooling/Syntax/TreeTest.cpp
@@ -4067,6 +4067,78 @@
 )txt"));
 }
 
+TEST_P(SyntaxTreeTest, MemberFunctionPointer) {
+  if (!GetParam().isCXX()) {
+return;
+  }
+  EXPECT_TRUE(treeDumpEqual(
+  R"cpp(
+struct X{
+  struct Y{};
+};
+void (X::*xp)();
+// FIXME: Generate the right syntax tree for this type,
+// i.e. create a syntax node for the outer member pointer
+void (X::Y::**xyp)(const int*, char);
+)cpp",
+  R"txt(
+*: TranslationUnit
+|-SimpleDeclaration
+| |-struct
+| |-X
+| |-{
+| |-SimpleDeclaration
+| | |-struct
+| | |-Y
+| | |-{
+| | |-}
+| | `-;
+| |-}
+| `-;
+|-SimpleDeclaration
+| |-void
+| |-SimpleDeclarator
+| | |-ParenDeclarator
+| | | |-(
+| | | |-MemberPointer
+| | | | |-X
+| | | | |-::
+| | | | `-*
+| | | |-xp
+| | | `-)
+| | `-ParametersAndQualifiers
+| |   |-(
+| |   `-)
+| `-;
+`-SimpleDeclaration
+  |-void
+  |-SimpleDeclarator
+  | |-ParenDeclarator
+  | | |-(
+  | | |-X
+  | | |-::
+  | | |-MemberPointer
+  | | | |-Y
+  | | | |-::
+  | | | `-*
+  | | |-*
+  | | |-xyp
+  | | `-)
+  | `-ParametersAndQualifiers
+  |   |-(
+  |   |-SimpleDeclaration
+  |   | |-const
+  |   | |-int
+  |   | `-SimpleDeclarator
+  |   |   `-*
+  |   |-,
+  |   |-SimpleDeclaration
+  |   | `-char
+  |   `-)
+  `-;
+)txt"));
+}
+
 TEST_P(SyntaxTreeTest, ComplexDeclarator) {
   EXPECT_TRUE(treeDumpEqual(
   R"cpp(
Index: clang/lib/Tooling/Syntax/BuildTree.cpp
===
--- clang/lib/Tooling/Syntax/BuildTree.cpp
+++ clang/lib/Tooling/Syntax/BuildTree.cpp
@@ -987,6 +987,16 @@
 return WalkUpFromFunctionTypeLoc(L);
   }
 
+  bool TraverseMemberPointerTypeLoc(MemberPointerTypeLoc L) {
+// In the source code "void (Y::*mp)()" `MemberPointerTypeLoc` corresponds
+// to "Y::*" but it points to a `ParenTypeLoc` that corresponds to
+// "(Y::*mp)" We thus reverse the order of traversal to get the proper
+// syntax structure.
+if (!WalkUpFromMemberPointerTypeLoc(L))
+  return false;
+return TraverseTypeLoc(L.getPointeeLoc());
+  }
+
   bool WalkUpFromMemberPointerTypeLoc(MemberPointerTypeLoc L) {
 auto SR = L.getLocalSourceRange();
 Builder.foldNode(Builder.getRange(SR),
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D84781: Use PointerUnion instead of inheritance for alternative clauses in NNS

[Eduardo Caldas via Phabricator via cfe-commits]

eduucaldas created this revision.
Herald added a project: clang.
Herald added a subscriber: cfe-commits.
eduucaldas requested review of this revision.

Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D84781

Files:
  clang/include/clang/Tooling/Syntax/Nodes.h
  clang/include/clang/Tooling/Syntax/Tree.h
  clang/lib/Tooling/Syntax/BuildTree.cpp
  clang/lib/Tooling/Syntax/Nodes.cpp
  clang/lib/Tooling/Syntax/Tree.cpp
  clang/unittests/Tooling/Syntax/TreeTest.cpp

Index: clang/unittests/Tooling/Syntax/TreeTest.cpp
===
--- clang/unittests/Tooling/Syntax/TreeTest.cpp
+++ clang/unittests/Tooling/Syntax/TreeTest.cpp
@@ -987,18 +987,19 @@
 | | |-IdExpression
 | | | |-NestedNameSpecifier
 | | | | |-::
-| | | | |-IdentifierNameSpecifier
+| | | | |-NameSpecifier
 | | | | | `-n
 | | | | |-::
-| | | | |-IdentifierNameSpecifier
+| | | | |-NameSpecifier
 | | | | | `-S
 | | | | |-::
-| | | | |-SimpleTemplateNameSpecifier
-| | | | | |-template
-| | | | | |-ST
-| | | | | |-<
-| | | | | |-int
-| | | | | `->
+| | | | |-NameSpecifier
+| | | | | `-SimpleTemplateSpecifier
+| | | | |   |-template
+| | | | |   |-ST
+| | | | |   |-<
+| | | | |   |-int
+| | | | |   `->
 | | | | `-::
 | | | `-UnqualifiedId
 | | |   `-f
@@ -1009,17 +1010,18 @@
 | |-UnknownExpression
 | | |-IdExpression
 | | | |-NestedNameSpecifier
-| | | | |-IdentifierNameSpecifier
+| | | | |-NameSpecifier
 | | | | | `-n
 | | | | |-::
-| | | | |-IdentifierNameSpecifier
+| | | | |-NameSpecifier
 | | | | | `-S
 | | | | |-::
-| | | | |-SimpleTemplateNameSpecifier
-| | | | | |-ST
-| | | | | |-<
-| | | | | |-int
-| | | | | `->
+| | | | |-NameSpecifier
+| | | | | `-SimpleTemplateSpecifier
+| | | | |   |-ST
+| | | | |   |-<
+| | | | |   |-int
+| | | | |   `->
 | | | | `-::
 | | | `-UnqualifiedId
 | | |   `-f
@@ -1030,13 +1032,14 @@
 | |-UnknownExpression
 | | |-IdExpression
 | | | |-NestedNameSpecifier
-| | | | |-SimpleTemplateNameSpecifier
-| | | | | |-ST
-| | | | | |-<
-| | | | | |-int
-| | | | | `->
+| | | | |-NameSpecifier
+| | | | | `-SimpleTemplateSpecifier
+| | | | |   |-ST
+| | | | |   |-<
+| | | | |   |-int
+| | | | |   `->
 | | | | |-::
-| | | | |-IdentifierNameSpecifier
+| | | | |-NameSpecifier
 | | | | | `-S
 | | | | `-::
 | | | `-UnqualifiedId
@@ -1051,13 +1054,14 @@
 | |-UnknownExpression
 | | |-IdExpression
 | | | |-NestedNameSpecifier
-| | | | |-SimpleTemplateNameSpecifier
-| | | | | |-ST
-| | | | | |-<
-| | | | | |-int
-| | | | | `->
+| | | | |-NameSpecifier
+| | | | | `-SimpleTemplateSpecifier
+| | | | |   |-ST
+| | | | |   |-<
+| | | | |   |-int
+| | | | |   `->
 | | | | |-::
-| | | | |-IdentifierNameSpecifier
+| | | | |-NameSpecifier
 | | | | | `-S
 | | | | `-::
 | | | |-template
@@ -1115,15 +1119,16 @@
   | |-UnknownExpression
   | | |-IdExpression
   | | | |-NestedNameSpecifier
-  | | | | |-IdentifierNameSpecifier
+  | | | | |-NameSpecifier
   | | | | | `-T
   | | | | |-::
-  | | | | |-SimpleTemplateNameSpecifier
-  | | | | | |-template
-  | | | | | |-U
-  | | | | | |-<
-  | | | | | |-int
-  | | | | | `->
+  | | | | |-NameSpecifier
+  | | | | | `-SimpleTemplateSpecifier
+  | | | | |   |-template
+  | | | | |   |-U
+  | | | | |   |-<
+  | | | | |   |-int
+  | | | | |   `->
   | | | | `-::
   | | | `-UnqualifiedId
   | | |   `-f
@@ -1134,10 +1139,10 @@
   | |-UnknownExpression
   | | |-IdExpression
   | | | |-NestedNameSpecifier
-  | | | | |-IdentifierNameSpecifier
+  | | | | |-NameSpecifier
   | | | | | `-T
   | | | | |-::
-  | | | | |-IdentifierNameSpecifier
+  | | | | |-NameSpecifier
   | | | | | `-U
   | | | | `-::
   | | | `-UnqualifiedId
@@ -1149,7 +1154,7 @@
   | |-UnknownExpression
   | | |-IdExpression
   | | | |-NestedNameSpecifier
-  | | | | |-IdentifierNameSpecifier
+  | | | | |-NameSpecifier
   | | | | | `-T
   | | | | `-::
   | | | |-template
@@ -1216,13 +1221,14 @@
 | |-UnknownExpression
 | | |-IdExpression
 | | | |-NestedNameSpecifier
-| | | | |-DecltypeNameSpecifier
-| | | | | |-decltype
-| | | | | |-(
-| | | | | |-IdExpression
-| | | | | | `-UnqualifiedId
-| | | | | |   `-s
-| | | | | `-)
+| | | | |-NameSpecifier
+| | | | | `-DecltypeSpecifier
+| | | | |   |-decltype
+| | | | |   |-(
+| | | | |   |-IdExpression
+| | | | |   | `-UnqualifiedId
+| | | | |   |   `-s
+| | | | |   `-)
 | | | | `-::
 | | | `-UnqualifiedId
 | | |   `-f
Index: 

[PATCH] D84348: WIP: Add complete id-expression support to syntax trees

[Eduardo Caldas via Phabricator via cfe-commits]

eduucaldas marked 2 inline comments as done.
eduucaldas added inline comments.



Comment at: clang/lib/Tooling/Syntax/BuildTree.cpp:834
+  // FIXME: I feel like this could be upstreamed.
+  SourceRange getUnqualifiedIdSourceRange(DeclRefExpr *S) {
+if (S->hasExplicitTemplateArgs())

gribozavr2 wrote:
> WDYM by "upstream"?
I meant to put that logic under the DeclRefExpr node, instead of here. But I 
found a way of writing this logic in a simpler way :). So I just inlined it!


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D84348/new/

https://reviews.llvm.org/D84348



___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D84348: WIP: Add complete id-expression support to syntax trees

[Eduardo Caldas via Phabricator via cfe-commits]

eduucaldas updated this revision to Diff 280899.
eduucaldas marked 9 inline comments as done.
eduucaldas added a comment.

- Answer code review
- Simpler logic for `getUnqualifiedIdSourceRange` and inline it
- Remove ambiguously named variable NNS


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D84348/new/

https://reviews.llvm.org/D84348

Files:
  clang/include/clang/Tooling/Syntax/Nodes.h
  clang/lib/Tooling/Syntax/BuildTree.cpp
  clang/lib/Tooling/Syntax/Nodes.cpp
  clang/unittests/Tooling/Syntax/TreeTest.cpp

Index: clang/unittests/Tooling/Syntax/TreeTest.cpp
===
--- clang/unittests/Tooling/Syntax/TreeTest.cpp
+++ clang/unittests/Tooling/Syntax/TreeTest.cpp
@@ -867,24 +867,47 @@
   }
   EXPECT_TRUE(treeDumpEqual(
   R"cpp(
-namespace a {
+namespace n {
   struct S {
 template
-static T f(){}
+struct ST {
+  static void f();
+};
   };
 }
+template
+struct ST {
+  struct S {
+template
+static U f();
+  };
+};
 void test() {
-  ::  // global-namespace-specifier
-  a:: // namespace-specifier
-  S:: // type-name-specifier
+  :: // global-namespace-specifier
+  n::// namespace-specifier
+  S::// type-name-specifier
+  template ST:: // type-template-instantiation-specifier
+  f();
+
+  n::// namespace-specifier
+  S::// type-name-specifier
+  ST::  // type-template-instantiation-specifier
+  f();
+
+  ST:: // type-name-specifier
+  S::   // type-name-specifier
   f();
+
+  ST:: // type-name-specifier
+  S::   // type-name-specifier
+  template f();
 }
 )cpp",
   R"txt(
 *: TranslationUnit
 |-NamespaceDefinition
 | |-namespace
-| |-a
+| |-n
 | |-{
 | |-SimpleDeclaration
 | | |-struct
@@ -898,19 +921,58 @@
 | | | | `-T
 | | | |->
 | | | `-SimpleDeclaration
-| | |   |-static
-| | |   |-T
-| | |   |-SimpleDeclarator
-| | |   | |-f
-| | |   | `-ParametersAndQualifiers
-| | |   |   |-(
-| | |   |   `-)
-| | |   `-CompoundStatement
-| | | |-{
-| | | `-}
+| | |   |-struct
+| | |   |-ST
+| | |   |-{
+| | |   |-SimpleDeclaration
+| | |   | |-static
+| | |   | |-void
+| | |   | |-SimpleDeclarator
+| | |   | | |-f
+| | |   | | `-ParametersAndQualifiers
+| | |   | |   |-(
+| | |   | |   `-)
+| | |   | `-;
+| | |   |-}
+| | |   `-;
 | | |-}
 | | `-;
 | `-}
+|-TemplateDeclaration
+| |-template
+| |-<
+| |-UnknownDeclaration
+| | |-typename
+| | `-T
+| |->
+| `-SimpleDeclaration
+|   |-struct
+|   |-ST
+|   |-{
+|   |-SimpleDeclaration
+|   | |-struct
+|   | |-S
+|   | |-{
+|   | |-TemplateDeclaration
+|   | | |-template
+|   | | |-<
+|   | | |-UnknownDeclaration
+|   | | | |-typename
+|   | | | `-U
+|   | | |->
+|   | | `-SimpleDeclaration
+|   | |   |-static
+|   | |   |-U
+|   | |   |-SimpleDeclarator
+|   | |   | |-f
+|   | |   | `-ParametersAndQualifiers
+|   | |   |   |-(
+|   | |   |   `-)
+|   | |   `-;
+|   | |-}
+|   | `-;
+|   |-}
+|   `-;
 `-SimpleDeclaration
   |-void
   |-SimpleDeclarator
@@ -924,14 +986,81 @@
 | |-UnknownExpression
 | | |-IdExpression
 | | | |-NestedNameSpecifier
-| | | | |-NameSpecifier
-| | | | | `-::
-| | | | |-NameSpecifier
-| | | | | |-a
-| | | | | `-::
-| | | | `-NameSpecifier
-| | | |   |-S
-| | | |   `-::
+| | | | |-::
+| | | | |-IdentifierNameSpecifier
+| | | | | `-n
+| | | | |-::
+| | | | |-IdentifierNameSpecifier
+| | | | | `-S
+| | | | |-::
+| | | | |-SimpleTemplateNameSpecifier
+| | | | | |-template
+| | | | | |-ST
+| | | | | |-<
+| | | | | |-int
+| | | | | `->
+| | | | `-::
+| | | `-UnqualifiedId
+| | |   `-f
+| | |-(
+| | `-)
+| `-;
+|-ExpressionStatement
+| |-UnknownExpression
+| | |-IdExpression
+| | | |-NestedNameSpecifier
+| | | | |-IdentifierNameSpecifier
+| | | | | `-n
+| | | | |-::
+| | | | |-IdentifierNameSpecifier
+| | | | | `-S
+| | | | |-::
+| | | | |-SimpleTemplateNameSpecifier
+| | | | | |-ST
+| | | | | |-<
+| | | | | |-int
+| | | | | `->
+| | | | `-::
+| | | `-UnqualifiedId
+| | |   `-f
+| | |-(
+| | `-)
+| `-;
+|-ExpressionStatement
+| |-UnknownExpression
+| | |-IdExpression
+| | | |-NestedNameSpecifier
+| | | | |-SimpleTemplateNameSpecifier
+| | | | | |-ST
+| | | | | |-<
+| | | | | |-int
+| | | | | `->
+| | | | |-::
+| | | | |-IdentifierNameSpecifier
+| | | | | `-S
+| | | | `-::
+| | | `-UnqualifiedId
+| | |   |-f
+| | |   |-<
+| | |   |-int
+| | |   `->
+| | |-(
+| | `-)
+| `-;
+|-ExpressionStatement
+| |-UnknownExpression
+| | |-IdExpression
+| | | |-NestedNameSpecifier
+| | | | |-SimpleTemplateNameSpecifier
+| | | | | |-ST
+| | | | | |-<

[PATCH] D84348: WIP: Add complete id-expression support to syntax trees

[Eduardo Caldas via Phabricator via cfe-commits]

eduucaldas updated this revision to Diff 280511.
eduucaldas added a comment.

- Remove UnknownNameSpecifier, answer to comments


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D84348/new/

https://reviews.llvm.org/D84348

Files:
  clang/include/clang/Tooling/Syntax/Nodes.h
  clang/lib/Tooling/Syntax/BuildTree.cpp
  clang/lib/Tooling/Syntax/Nodes.cpp
  clang/unittests/Tooling/Syntax/TreeTest.cpp

Index: clang/unittests/Tooling/Syntax/TreeTest.cpp
===
--- clang/unittests/Tooling/Syntax/TreeTest.cpp
+++ clang/unittests/Tooling/Syntax/TreeTest.cpp
@@ -867,24 +867,47 @@
   }
   EXPECT_TRUE(treeDumpEqual(
   R"cpp(
-namespace a {
+namespace n {
   struct S {
 template
-static T f(){}
+struct TS {
+  static void f();
+};
   };
 }
+template
+struct TS {
+  struct S {
+template
+static U f();
+  };
+};
 void test() {
-  ::  // global-namespace-specifier
-  a:: // namespace-specifier
-  S:: // type-name-specifier
+  :: // global-namespace-specifier
+  n::// namespace-specifier
+  S::// type-name-specifier
+  template TS:: // type-template-instantiation-specifier
+  f();
+
+  n::// namespace-specifier
+  S::// type-name-specifier
+  TS::  // type-template-instantiation-specifier
+  f();
+
+  TS:: // type-name-specifier
+  S::   // type-name-specifier
   f();
+
+  TS:: // type-name-specifier
+  S::   // type-name-specifier
+  template f();
 }
 )cpp",
   R"txt(
 *: TranslationUnit
 |-NamespaceDefinition
 | |-namespace
-| |-a
+| |-n
 | |-{
 | |-SimpleDeclaration
 | | |-struct
@@ -898,19 +921,58 @@
 | | | | `-T
 | | | |->
 | | | `-SimpleDeclaration
-| | |   |-static
-| | |   |-T
-| | |   |-SimpleDeclarator
-| | |   | |-f
-| | |   | `-ParametersAndQualifiers
-| | |   |   |-(
-| | |   |   `-)
-| | |   `-CompoundStatement
-| | | |-{
-| | | `-}
+| | |   |-struct
+| | |   |-TS
+| | |   |-{
+| | |   |-SimpleDeclaration
+| | |   | |-static
+| | |   | |-void
+| | |   | |-SimpleDeclarator
+| | |   | | |-f
+| | |   | | `-ParametersAndQualifiers
+| | |   | |   |-(
+| | |   | |   `-)
+| | |   | `-;
+| | |   |-}
+| | |   `-;
 | | |-}
 | | `-;
 | `-}
+|-TemplateDeclaration
+| |-template
+| |-<
+| |-UnknownDeclaration
+| | |-typename
+| | `-T
+| |->
+| `-SimpleDeclaration
+|   |-struct
+|   |-TS
+|   |-{
+|   |-SimpleDeclaration
+|   | |-struct
+|   | |-S
+|   | |-{
+|   | |-TemplateDeclaration
+|   | | |-template
+|   | | |-<
+|   | | |-UnknownDeclaration
+|   | | | |-typename
+|   | | | `-U
+|   | | |->
+|   | | `-SimpleDeclaration
+|   | |   |-static
+|   | |   |-U
+|   | |   |-SimpleDeclarator
+|   | |   | |-f
+|   | |   | `-ParametersAndQualifiers
+|   | |   |   |-(
+|   | |   |   `-)
+|   | |   `-;
+|   | |-}
+|   | `-;
+|   |-}
+|   `-;
 `-SimpleDeclaration
   |-void
   |-SimpleDeclarator
@@ -924,14 +986,81 @@
 | |-UnknownExpression
 | | |-IdExpression
 | | | |-NestedNameSpecifier
-| | | | |-NameSpecifier
-| | | | | `-::
-| | | | |-NameSpecifier
-| | | | | |-a
-| | | | | `-::
-| | | | `-NameSpecifier
-| | | |   |-S
-| | | |   `-::
+| | | | |-::
+| | | | |-IdentifierNameSpecifier
+| | | | | `-n
+| | | | |-::
+| | | | |-IdentifierNameSpecifier
+| | | | | `-S
+| | | | |-::
+| | | | |-SimpleTemplateNameSpecifier
+| | | | | |-template
+| | | | | |-TS
+| | | | | |-<
+| | | | | |-int
+| | | | | `->
+| | | | `-::
+| | | `-UnqualifiedId
+| | |   `-f
+| | |-(
+| | `-)
+| `-;
+|-ExpressionStatement
+| |-UnknownExpression
+| | |-IdExpression
+| | | |-NestedNameSpecifier
+| | | | |-IdentifierNameSpecifier
+| | | | | `-n
+| | | | |-::
+| | | | |-IdentifierNameSpecifier
+| | | | | `-S
+| | | | |-::
+| | | | |-SimpleTemplateNameSpecifier
+| | | | | |-TS
+| | | | | |-<
+| | | | | |-int
+| | | | | `->
+| | | | `-::
+| | | `-UnqualifiedId
+| | |   `-f
+| | |-(
+| | `-)
+| `-;
+|-ExpressionStatement
+| |-UnknownExpression
+| | |-IdExpression
+| | | |-NestedNameSpecifier
+| | | | |-SimpleTemplateNameSpecifier
+| | | | | |-TS
+| | | | | |-<
+| | | | | |-int
+| | | | | `->
+| | | | |-::
+| | | | |-IdentifierNameSpecifier
+| | | | | `-S
+| | | | `-::
+| | | `-UnqualifiedId
+| | |   |-f
+| | |   |-<
+| | |   |-int
+| | |   `->
+| | |-(
+| | `-)
+| `-;
+|-ExpressionStatement
+| |-UnknownExpression
+| | |-IdExpression
+| | | |-NestedNameSpecifier
+| | | | |-SimpleTemplateNameSpecifier
+| | | | | |-TS
+| | | | | |-<
+| | | | | |-int
+| | | | | `->
+| | | | |-::
+| | | | |-IdentifierNameSpecifier
+| | | | | `-S
+   

[PATCH] D84348: WIP: Add complete id-expression support to syntax trees

[Eduardo Caldas via Phabricator via cfe-commits]

eduucaldas updated this revision to Diff 280497.
eduucaldas added a comment.

- Improve getLocalSourceRange
- nested-name-specifier is now a ::-separated list of name-specifiers


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D84348/new/

https://reviews.llvm.org/D84348

Files:
  clang/include/clang/Tooling/Syntax/Nodes.h
  clang/lib/Tooling/Syntax/BuildTree.cpp
  clang/lib/Tooling/Syntax/Nodes.cpp
  clang/unittests/Tooling/Syntax/TreeTest.cpp

Index: clang/unittests/Tooling/Syntax/TreeTest.cpp
===
--- clang/unittests/Tooling/Syntax/TreeTest.cpp
+++ clang/unittests/Tooling/Syntax/TreeTest.cpp
@@ -867,24 +867,47 @@
   }
   EXPECT_TRUE(treeDumpEqual(
   R"cpp(
-namespace a {
+namespace n {
   struct S {
 template
-static T f(){}
+struct TS {
+  static void f();
+};
   };
 }
+template
+struct TS {
+  struct S {
+template
+static U f();
+  };
+};
 void test() {
-  ::  // global-namespace-specifier
-  a:: // namespace-specifier
-  S:: // type-name-specifier
+  :: // global-namespace-specifier
+  n::// namespace-specifier
+  S::// type-name-specifier
+  template TS:: // type-template-instantiation-specifier
+  f();
+
+  n::// namespace-specifier
+  S::// type-name-specifier
+  TS::  // type-template-instantiation-specifier
+  f();
+
+  TS:: // type-name-specifier
+  S::   // type-name-specifier
   f();
+
+  TS:: // type-name-specifier
+  S::   // type-name-specifier
+  template f();
 }
 )cpp",
   R"txt(
 *: TranslationUnit
 |-NamespaceDefinition
 | |-namespace
-| |-a
+| |-n
 | |-{
 | |-SimpleDeclaration
 | | |-struct
@@ -898,19 +921,58 @@
 | | | | `-T
 | | | |->
 | | | `-SimpleDeclaration
-| | |   |-static
-| | |   |-T
-| | |   |-SimpleDeclarator
-| | |   | |-f
-| | |   | `-ParametersAndQualifiers
-| | |   |   |-(
-| | |   |   `-)
-| | |   `-CompoundStatement
-| | | |-{
-| | | `-}
+| | |   |-struct
+| | |   |-TS
+| | |   |-{
+| | |   |-SimpleDeclaration
+| | |   | |-static
+| | |   | |-void
+| | |   | |-SimpleDeclarator
+| | |   | | |-f
+| | |   | | `-ParametersAndQualifiers
+| | |   | |   |-(
+| | |   | |   `-)
+| | |   | `-;
+| | |   |-}
+| | |   `-;
 | | |-}
 | | `-;
 | `-}
+|-TemplateDeclaration
+| |-template
+| |-<
+| |-UnknownDeclaration
+| | |-typename
+| | `-T
+| |->
+| `-SimpleDeclaration
+|   |-struct
+|   |-TS
+|   |-{
+|   |-SimpleDeclaration
+|   | |-struct
+|   | |-S
+|   | |-{
+|   | |-TemplateDeclaration
+|   | | |-template
+|   | | |-<
+|   | | |-UnknownDeclaration
+|   | | | |-typename
+|   | | | `-U
+|   | | |->
+|   | | `-SimpleDeclaration
+|   | |   |-static
+|   | |   |-U
+|   | |   |-SimpleDeclarator
+|   | |   | |-f
+|   | |   | `-ParametersAndQualifiers
+|   | |   |   |-(
+|   | |   |   `-)
+|   | |   `-;
+|   | |-}
+|   | `-;
+|   |-}
+|   `-;
 `-SimpleDeclaration
   |-void
   |-SimpleDeclarator
@@ -924,14 +986,81 @@
 | |-UnknownExpression
 | | |-IdExpression
 | | | |-NestedNameSpecifier
-| | | | |-NameSpecifier
-| | | | | `-::
-| | | | |-NameSpecifier
-| | | | | |-a
-| | | | | `-::
-| | | | `-NameSpecifier
-| | | |   |-S
-| | | |   `-::
+| | | | |-::
+| | | | |-IdentifierNameSpecifier
+| | | | | `-n
+| | | | |-::
+| | | | |-IdentifierNameSpecifier
+| | | | | `-S
+| | | | |-::
+| | | | |-SimpleTemplateNameSpecifier
+| | | | | |-template
+| | | | | |-TS
+| | | | | |-<
+| | | | | |-int
+| | | | | `->
+| | | | `-::
+| | | `-UnqualifiedId
+| | |   `-f
+| | |-(
+| | `-)
+| `-;
+|-ExpressionStatement
+| |-UnknownExpression
+| | |-IdExpression
+| | | |-NestedNameSpecifier
+| | | | |-IdentifierNameSpecifier
+| | | | | `-n
+| | | | |-::
+| | | | |-IdentifierNameSpecifier
+| | | | | `-S
+| | | | |-::
+| | | | |-SimpleTemplateNameSpecifier
+| | | | | |-TS
+| | | | | |-<
+| | | | | |-int
+| | | | | `->
+| | | | `-::
+| | | `-UnqualifiedId
+| | |   `-f
+| | |-(
+| | `-)
+| `-;
+|-ExpressionStatement
+| |-UnknownExpression
+| | |-IdExpression
+| | | |-NestedNameSpecifier
+| | | | |-SimpleTemplateNameSpecifier
+| | | | | |-TS
+| | | | | |-<
+| | | | | |-int
+| | | | | `->
+| | | | |-::
+| | | | |-IdentifierNameSpecifier
+| | | | | `-S
+| | | | `-::
+| | | `-UnqualifiedId
+| | |   |-f
+| | |   |-<
+| | |   |-int
+| | |   `->
+| | |-(
+| | `-)
+| `-;
+|-ExpressionStatement
+| |-UnknownExpression
+| | |-IdExpression
+| | | |-NestedNameSpecifier
+| | | | |-SimpleTemplateNameSpecifier
+| | | | | |-TS
+| | | | | |-<
+| | | | | |-int
+| | | | | `->
+| | | | |-::
+| | | 

[PATCH] D84348: WIP: Add complete id-expression support to syntax trees

[Eduardo Caldas via Phabricator via cfe-commits]

eduucaldas updated this revision to Diff 280216.
eduucaldas added a comment.

Update API to new nested-name-specifier grammar rule


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D84348/new/

https://reviews.llvm.org/D84348

Files:
  clang/include/clang/Tooling/Syntax/Nodes.h
  clang/lib/Tooling/Syntax/BuildTree.cpp
  clang/lib/Tooling/Syntax/Nodes.cpp
  clang/unittests/Tooling/Syntax/TreeTest.cpp

Index: clang/unittests/Tooling/Syntax/TreeTest.cpp
===
--- clang/unittests/Tooling/Syntax/TreeTest.cpp
+++ clang/unittests/Tooling/Syntax/TreeTest.cpp
@@ -988,13 +988,13 @@
 | | | |-NestedNameSpecifier
 | | | | |-GlobalNameSpecifier
 | | | | | `-::
-| | | | |-NamespaceNameSpecifier
+| | | | |-IdentifierNameSpecifier
 | | | | | |-n
 | | | | | `-::
-| | | | |-TypeNameSpecifier
+| | | | |-IdentifierNameSpecifier
 | | | | | |-S
 | | | | | `-::
-| | | | `-TypeWithTemplateNameSpecifier
+| | | | `-SimpleTemplateNameSpecifier
 | | | |   |-template
 | | | |   |-TS
 | | | |   |-<
@@ -1010,13 +1010,13 @@
 | |-UnknownExpression
 | | |-IdExpression
 | | | |-NestedNameSpecifier
-| | | | |-NamespaceNameSpecifier
+| | | | |-IdentifierNameSpecifier
 | | | | | |-n
 | | | | | `-::
-| | | | |-TypeNameSpecifier
+| | | | |-IdentifierNameSpecifier
 | | | | | |-S
 | | | | | `-::
-| | | | `-TypeNameSpecifier
+| | | | `-SimpleTemplateNameSpecifier
 | | | |   |-TS
 | | | |   |-<
 | | | |   |-int
@@ -1031,13 +1031,13 @@
 | |-UnknownExpression
 | | |-IdExpression
 | | | |-NestedNameSpecifier
-| | | | |-TypeNameSpecifier
+| | | | |-SimpleTemplateNameSpecifier
 | | | | | |-TS
 | | | | | |-<
 | | | | | |-int
 | | | | | |->
 | | | | | `-::
-| | | | `-TypeNameSpecifier
+| | | | `-IdentifierNameSpecifier
 | | | |   |-S
 | | | |   `-::
 | | | `-UnqualifiedId
@@ -1052,13 +1052,13 @@
 | |-UnknownExpression
 | | |-IdExpression
 | | | |-NestedNameSpecifier
-| | | | |-TypeNameSpecifier
+| | | | |-SimpleTemplateNameSpecifier
 | | | | | |-TS
 | | | | | |-<
 | | | | | |-int
 | | | | | |->
 | | | | | `-::
-| | | | `-TypeNameSpecifier
+| | | | `-IdentifierNameSpecifier
 | | | |   |-S
 | | | |   `-::
 | | | |-template
@@ -1116,10 +1116,10 @@
   | |-UnknownExpression
   | | |-IdExpression
   | | | |-NestedNameSpecifier
-  | | | | |-TypeNameSpecifier
+  | | | | |-IdentifierNameSpecifier
   | | | | | |-T
   | | | | | `-::
-  | | | | `-TypeWithTemplateNameSpecifier
+  | | | | `-SimpleTemplateNameSpecifier
   | | | |   |-template
   | | | |   |-U
   | | | |   |-<
@@ -1135,7 +1135,7 @@
   | |-UnknownExpression
   | | |-IdExpression
   | | | |-NestedNameSpecifier
-  | | | | |-TypeNameSpecifier
+  | | | | |-IdentifierNameSpecifier
   | | | | | |-T
   | | | | | `-::
   | | | | `-IdentifierNameSpecifier
@@ -1150,7 +1150,7 @@
   | |-UnknownExpression
   | | |-IdExpression
   | | | |-NestedNameSpecifier
-  | | | | `-TypeNameSpecifier
+  | | | | `-IdentifierNameSpecifier
   | | | |   |-T
   | | | |   `-::
   | | | |-template
@@ -1217,7 +1217,7 @@
 | |-UnknownExpression
 | | |-IdExpression
 | | | |-NestedNameSpecifier
-| | | | `-TypeNameSpecifier
+| | | | `-DecltypeNameSpecifier
 | | | |   |-decltype
 | | | |   |-(
 | | | |   |-IdExpression
Index: clang/lib/Tooling/Syntax/Nodes.cpp
===
--- clang/lib/Tooling/Syntax/Nodes.cpp
+++ clang/lib/Tooling/Syntax/Nodes.cpp
@@ -118,16 +118,12 @@
 return OS << "MemberPointer";
   case NodeKind::GlobalNameSpecifier:
 return OS << "GlobalNameSpecifier";
-  case NodeKind::NamespaceNameSpecifier:
-return OS << "NamespaceNameSpecifier";
-  case NodeKind::TypeNameSpecifier:
-return OS << "TypeNameSpecifier";
-  case NodeKind::UnknownNameSpecifier:
-return OS << "UnknownNameSpecifier";
+  case NodeKind::DecltypeNameSpecifier:
+return OS << "DecltypeNameSpecifier";
   case NodeKind::IdentifierNameSpecifier:
 return OS << "IdentifierNameSpecifier";
-  case NodeKind::TypeWithTemplateNameSpecifier:
-return OS << "TypeWithTemplateNameSpecifier";
+  case NodeKind::SimpleTemplateNameSpecifier:
+return OS << "SimpleTemplateNameSpecifier";
   case NodeKind::NestedNameSpecifier:
 return OS << "NestedNameSpecifier";
   }
Index: clang/lib/Tooling/Syntax/BuildTree.cpp
===
--- clang/lib/Tooling/Syntax/BuildTree.cpp
+++ clang/lib/Tooling/Syntax/BuildTree.cpp
@@ -744,23 +744,37 @@
 return true;
   }
 
-  syntax::NameSpecifier *
-  BuildNameSpecifier(NestedNameSpecifier::SpecifierKind K) {
-

[PATCH] D84348: WIP: Add complete id-expression support to syntax trees

[Eduardo Caldas via Phabricator via cfe-commits]

eduucaldas added a reviewer: gribozavr2.
eduucaldas marked 6 inline comments as done.
eduucaldas added inline comments.



Comment at: clang/include/clang/Tooling/Syntax/Nodes.h:101-106
+  UnknownNameSpecifier,
+  GlobalNameSpecifier,
+  NamespaceNameSpecifier,
+  TypeNameSpecifier,
+  IdentifierNameSpecifier,
+  TypeWithTemplateNameSpecifier

All of this is ephemeral, we are gonna have just one Name Specifier with a 
PointerUnion.



Comment at: clang/include/clang/Tooling/Syntax/Nodes.h:206-276
+class NameSpecifier : public Tree {
 public:
-  NameSpecifier() : Tree(NodeKind::NameSpecifier) {}
+  NameSpecifier(NodeKind K) : Tree(K) {}
   static bool classof(const Node *N) {
-return N->kind() == NodeKind::NameSpecifier;
+return N->kind() == NodeKind::GlobalNameSpecifier ||
+   N->kind() == NodeKind::TypeNameSpecifier ||
+   N->kind() == NodeKind::NamespaceNameSpecifier ||

(bis) All of this is temporary, we are gonna have just one Name Specifier with 
a PointerUnion.



Comment at: clang/lib/Tooling/Syntax/BuildTree.cpp:768-769
-// Get `UnqualifiedId` from `DeclRefExpr`.
-// FIXME: Extract this logic so that it can be used by `MemberExpr`,
-// and other semantic constructs, now it is tied to `DeclRefExpr`.
-if (!S->hasExplicitTemplateArgs()) {

We extracted the logic into `getUnqualifiedSourceRange`



Comment at: clang/lib/Tooling/Syntax/BuildTree.cpp:831-850
+  // FIXME: Same logic as DeclRefExpr. How to DRY?
+  SourceRange getUnqualifiedIdSourceRange(DependentScopeDeclRefExpr *S) {
+if (S->hasExplicitTemplateArgs())
+  return SourceRange(S->getNameInfo().getBeginLoc(), S->getRAngleLoc());
+return S->getNameInfo().getSourceRange();
+  }
+

This is the same logic as `DeclRefExpr`! Exacly the same code!
`DependentScopeDeclRefExpr` is important because it enables `T::template S::`



Comment at: clang/lib/Tooling/Syntax/Nodes.cpp:119-130
+  case NodeKind::GlobalNameSpecifier:
+return OS << "GlobalNameSpecifier";
+  case NodeKind::NamespaceNameSpecifier:
+return OS << "NamespaceNameSpecifier";
+  case NodeKind::TypeNameSpecifier:
+return OS << "TypeNameSpecifier";
+  case NodeKind::UnknownNameSpecifier:

(bis) All of this is temporary, we are gonna have just one Name Specifier with 
a PointerUnion.



Comment at: clang/unittests/Tooling/Syntax/TreeTest.cpp:958-964
-struct X {
-  template static void f();
-  template
-  struct Y {
-static void f();
-  };
-};

I noticed that we don't need that as everything is dependent on the template 
anyways


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D84348/new/

https://reviews.llvm.org/D84348



___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D84348: WIP: Add complete id-expression support to syntax trees

[Eduardo Caldas via Phabricator via cfe-commits]

eduucaldas created this revision.
Herald added a project: clang.
Herald added a subscriber: cfe-commits.

Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D84348

Files:
  clang/include/clang/Tooling/Syntax/Nodes.h
  clang/lib/Tooling/Syntax/BuildTree.cpp
  clang/lib/Tooling/Syntax/Nodes.cpp
  clang/unittests/Tooling/Syntax/TreeTest.cpp

Index: clang/unittests/Tooling/Syntax/TreeTest.cpp
===
--- clang/unittests/Tooling/Syntax/TreeTest.cpp
+++ clang/unittests/Tooling/Syntax/TreeTest.cpp
@@ -867,24 +867,47 @@
   }
   EXPECT_TRUE(treeDumpEqual(
   R"cpp(
-namespace a {
+namespace n {
   struct S {
 template
-static T f(){}
+struct TS {
+  static void f();
+};
   };
 }
+template
+struct TS {
+  struct S {
+template
+static U f();
+  };
+};
 void test() {
-  ::  // global-namespace-specifier
-  a:: // namespace-specifier
-  S:: // type-name-specifier
+  :: // global-namespace-specifier
+  n::// namespace-specifier
+  S::// type-name-specifier
+  template TS:: // type-template-instantiation-specifier
+  f();
+
+  n::// namespace-specifier
+  S::// type-name-specifier
+  TS::  // type-template-instantiation-specifier
+  f();
+
+  TS:: // type-name-specifier
+  S::   // type-name-specifier
   f();
+
+  TS:: // type-name-specifier
+  S::   // type-name-specifier
+  template f();
 }
 )cpp",
   R"txt(
 *: TranslationUnit
 |-NamespaceDefinition
 | |-namespace
-| |-a
+| |-n
 | |-{
 | |-SimpleDeclaration
 | | |-struct
@@ -898,19 +921,58 @@
 | | | | `-T
 | | | |->
 | | | `-SimpleDeclaration
-| | |   |-static
-| | |   |-T
-| | |   |-SimpleDeclarator
-| | |   | |-f
-| | |   | `-ParametersAndQualifiers
-| | |   |   |-(
-| | |   |   `-)
-| | |   `-CompoundStatement
-| | | |-{
-| | | `-}
+| | |   |-struct
+| | |   |-TS
+| | |   |-{
+| | |   |-SimpleDeclaration
+| | |   | |-static
+| | |   | |-void
+| | |   | |-SimpleDeclarator
+| | |   | | |-f
+| | |   | | `-ParametersAndQualifiers
+| | |   | |   |-(
+| | |   | |   `-)
+| | |   | `-;
+| | |   |-}
+| | |   `-;
 | | |-}
 | | `-;
 | `-}
+|-TemplateDeclaration
+| |-template
+| |-<
+| |-UnknownDeclaration
+| | |-typename
+| | `-T
+| |->
+| `-SimpleDeclaration
+|   |-struct
+|   |-TS
+|   |-{
+|   |-SimpleDeclaration
+|   | |-struct
+|   | |-S
+|   | |-{
+|   | |-TemplateDeclaration
+|   | | |-template
+|   | | |-<
+|   | | |-UnknownDeclaration
+|   | | | |-typename
+|   | | | `-U
+|   | | |->
+|   | | `-SimpleDeclaration
+|   | |   |-static
+|   | |   |-U
+|   | |   |-SimpleDeclarator
+|   | |   | |-f
+|   | |   | `-ParametersAndQualifiers
+|   | |   |   |-(
+|   | |   |   `-)
+|   | |   `-;
+|   | |-}
+|   | `-;
+|   |-}
+|   `-;
 `-SimpleDeclaration
   |-void
   |-SimpleDeclarator
@@ -924,14 +986,82 @@
 | |-UnknownExpression
 | | |-IdExpression
 | | | |-NestedNameSpecifier
-| | | | |-NameSpecifier
+| | | | |-GlobalNameSpecifier
+| | | | | `-::
+| | | | |-NamespaceNameSpecifier
+| | | | | |-n
+| | | | | `-::
+| | | | |-TypeNameSpecifier
+| | | | | |-S
+| | | | | `-::
+| | | | `-TypeWithTemplateNameSpecifier
+| | | |   |-template
+| | | |   |-TS
+| | | |   |-<
+| | | |   |-int
+| | | |   |->
+| | | |   `-::
+| | | `-UnqualifiedId
+| | |   `-f
+| | |-(
+| | `-)
+| `-;
+|-ExpressionStatement
+| |-UnknownExpression
+| | |-IdExpression
+| | | |-NestedNameSpecifier
+| | | | |-NamespaceNameSpecifier
+| | | | | |-n
+| | | | | `-::
+| | | | |-TypeNameSpecifier
+| | | | | |-S
+| | | | | `-::
+| | | | `-TypeNameSpecifier
+| | | |   |-TS
+| | | |   |-<
+| | | |   |-int
+| | | |   |->
+| | | |   `-::
+| | | `-UnqualifiedId
+| | |   `-f
+| | |-(
+| | `-)
+| `-;
+|-ExpressionStatement
+| |-UnknownExpression
+| | |-IdExpression
+| | | |-NestedNameSpecifier
+| | | | |-TypeNameSpecifier
+| | | | | |-TS
+| | | | | |-<
+| | | | | |-int
+| | | | | |->
 | | | | | `-::
-| | | | |-NameSpecifier
-| | | | | |-a
+| | | | `-TypeNameSpecifier
+| | | |   |-S
+| | | |   `-::
+| | | `-UnqualifiedId
+| | |   |-f
+| | |   |-<
+| | |   |-int
+| | |   `->
+| | |-(
+| | `-)
+| `-;
+|-ExpressionStatement
+| |-UnknownExpression
+| | |-IdExpression
+| | | |-NestedNameSpecifier
+| | | | |-TypeNameSpecifier
+| | | | | |-TS
+| | | | | |-<
+| | | | | |-int
+| | | | | |->
 | | | | | `-::
-| | | | `-NameSpecifier
+| | | | `-TypeNameSpecifier
 | | | |   |-S
 | | | |   `-::
+| | | |-template
 | | | `-UnqualifiedId
 | | |   |-f
 | | |   |-<
@@ -944,7 +1074,7 @@
 )txt"));
 }
 
-TEST_P(SyntaxTreeTest, 

[PATCH] D82157: Fix crash on `user defined literals`

[Eduardo Caldas via Phabricator via cfe-commits]

This revision was automatically updated to reflect the committed changes.
Closed by commit rGf33c2c27a8d4: Fix crash on `user defined literals` (authored 
by eduucaldas).

Changed prior to commit:
  https://reviews.llvm.org/D82157?vs=277060=277071#toc

Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D82157/new/

https://reviews.llvm.org/D82157

Files:
  clang/include/clang/Tooling/Syntax/Nodes.h
  clang/lib/Tooling/Syntax/BuildTree.cpp
  clang/lib/Tooling/Syntax/Nodes.cpp
  clang/unittests/Tooling/Syntax/TreeTest.cpp

Index: clang/unittests/Tooling/Syntax/TreeTest.cpp
===
--- clang/unittests/Tooling/Syntax/TreeTest.cpp
+++ clang/unittests/Tooling/Syntax/TreeTest.cpp
@@ -1184,20 +1184,108 @@
 )txt"));
 }
 
-TEST_P(SyntaxTreeTest, IntegerLiteral) {
+TEST_P(SyntaxTreeTest, UserDefinedLiteral) {
+  if (!GetParam().isCXX11OrLater()) {
+return;
+  }
   EXPECT_TRUE(treeDumpEqual(
   R"cpp(
+unsigned operator "" _i(unsigned long long);
+unsigned operator "" _f(long double);
+unsigned operator "" _c(char);
+
+unsigned operator "" _r(const char*); // raw-literal operator
+
+template 
+unsigned operator "" _t();// numeric literal operator template
+
 void test() {
-  12;
-  12u;
-  12l;
-  12ul;
-  014;
-  0XC;
+  12_i;  // call: operator "" _i(12uLL)  | kind: integer
+  1.2_f; // call: operator "" _f(1.2L)   | kind: float
+  '2'_c; // call: operator "" _c('2')| kind: char
+
+  // TODO: Generate `FloatUserDefinedLiteralExpression` and
+  // `IntegerUserDefinedLiteralExpression` instead of
+  // `UnknownUserDefinedLiteralExpression`. See `getUserDefinedLiteralKind`
+  12_r;  // call: operator "" _r("12")   | kind: integer
+  1.2_r; // call: operator "" _i("1.2")  | kind: float
+  12_t;  // call: operator<'1', '2'> "" _x() | kind: integer
+  1.2_t; // call: operator<'1', '2'> "" _x() | kind: float
 }
-)cpp",
+)cpp",
   R"txt(
 *: TranslationUnit
+|-SimpleDeclaration
+| |-unsigned
+| |-SimpleDeclarator
+| | |-operator
+| | |-""
+| | |-_i
+| | `-ParametersAndQualifiers
+| |   |-(
+| |   |-SimpleDeclaration
+| |   | |-unsigned
+| |   | |-long
+| |   | `-long
+| |   `-)
+| `-;
+|-SimpleDeclaration
+| |-unsigned
+| |-SimpleDeclarator
+| | |-operator
+| | |-""
+| | |-_f
+| | `-ParametersAndQualifiers
+| |   |-(
+| |   |-SimpleDeclaration
+| |   | |-long
+| |   | `-double
+| |   `-)
+| `-;
+|-SimpleDeclaration
+| |-unsigned
+| |-SimpleDeclarator
+| | |-operator
+| | |-""
+| | |-_c
+| | `-ParametersAndQualifiers
+| |   |-(
+| |   |-SimpleDeclaration
+| |   | `-char
+| |   `-)
+| `-;
+|-SimpleDeclaration
+| |-unsigned
+| |-SimpleDeclarator
+| | |-operator
+| | |-""
+| | |-_r
+| | `-ParametersAndQualifiers
+| |   |-(
+| |   |-SimpleDeclaration
+| |   | |-const
+| |   | |-char
+| |   | `-SimpleDeclarator
+| |   |   `-*
+| |   `-)
+| `-;
+|-TemplateDeclaration
+| |-template
+| |-<
+| |-SimpleDeclaration
+| | `-char
+| |-...
+| |->
+| `-SimpleDeclaration
+|   |-unsigned
+|   |-SimpleDeclarator
+|   | |-operator
+|   | |-""
+|   | |-_t
+|   | `-ParametersAndQualifiers
+|   |   |-(
+|   |   `-)
+|   `-;
 `-SimpleDeclaration
   |-void
   |-SimpleDeclarator
@@ -1208,28 +1296,95 @@
   `-CompoundStatement
 |-{
 |-ExpressionStatement
-| |-IntegerLiteralExpression
-| | `-12
+| |-IntegerUserDefinedLiteralExpression
+| | `-12_i
 | `-;
 |-ExpressionStatement
-| |-IntegerLiteralExpression
-| | `-12u
+| |-FloatUserDefinedLiteralExpression
+| | `-1.2_f
 | `-;
 |-ExpressionStatement
-| |-IntegerLiteralExpression
-| | `-12l
+| |-CharUserDefinedLiteralExpression
+| | `-'2'_c
 | `-;
 |-ExpressionStatement
-| |-IntegerLiteralExpression
-| | `-12ul
+| |-UnknownUserDefinedLiteralExpression
+| | `-12_r
 | `-;
 |-ExpressionStatement
-| |-IntegerLiteralExpression
-| | `-014
+| |-UnknownUserDefinedLiteralExpression
+| | `-1.2_r
 | `-;
 |-ExpressionStatement
-| |-IntegerLiteralExpression
-| | `-0XC
+| |-UnknownUserDefinedLiteralExpression
+| | `-12_t
+| `-;
+|-ExpressionStatement
+| |-UnknownUserDefinedLiteralExpression
+| | `-1.2_t
+| `-;
+`-}
+)txt"));
+}
+
+TEST_P(SyntaxTreeTest, UserDefinedLiteralString) {
+  if (!GetParam().isCXX11OrLater()) {
+return;
+  }
+  EXPECT_TRUE(treeDumpEqual(
+  R"cpp(
+typedef decltype(sizeof(void *)) size_t;
+unsigned operator "" _s(const char*, size_t);
+void test() {
+  "12"_s;// call: operator "" _s("12")   | kind: string
+}
+)cpp",
+  R"txt(
+*: TranslationUnit
+|-SimpleDeclaration
+| |-typedef
+| |-decltype
+| |-(
+| |-UnknownExpression
+| | |-sizeof
+| | |-(
+| | |-void
+| | |-*
+| | `-)
+| |-)
+| |-SimpleDeclarator
+| | `-size_t
+| `-;
+|-SimpleDeclaration
+| |-unsigned
+| |-SimpleDeclarator
+| | |-operator
+| | |-""
+| | |-_s
+| | `-ParametersAndQualifiers
+| |   |-(
+| |  

[PATCH] D82157: Fix crash on `user defined literals`

[Eduardo Caldas via Phabricator via cfe-commits]

eduucaldas updated this revision to Diff 277060.
eduucaldas marked an inline comment as done.
eduucaldas added a comment.

Add assert


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D82157/new/

https://reviews.llvm.org/D82157

Files:
  clang/include/clang/Tooling/Syntax/Nodes.h
  clang/lib/Tooling/Syntax/BuildTree.cpp
  clang/lib/Tooling/Syntax/Nodes.cpp
  clang/unittests/Tooling/Syntax/TreeTest.cpp

Index: clang/unittests/Tooling/Syntax/TreeTest.cpp
===
--- clang/unittests/Tooling/Syntax/TreeTest.cpp
+++ clang/unittests/Tooling/Syntax/TreeTest.cpp
@@ -1184,20 +1184,139 @@
 )txt"));
 }
 
-TEST_P(SyntaxTreeTest, IntegerLiteral) {
+TEST_P(SyntaxTreeTest, UserDefinedLiteral) {
+  if (!GetParam().isCXX11OrLater()) {
+return;
+  }
   EXPECT_TRUE(treeDumpEqual(
   R"cpp(
+typedef decltype(sizeof(void *)) size_t;
+
+unsigned operator "" _i(unsigned long long);
+unsigned operator "" _f(long double);
+unsigned operator "" _c(char);
+unsigned operator "" _s(const char*, size_t);
+unsigned operator "" _r(const char*);
+template 
+unsigned operator "" _t();
+
 void test() {
-  12;
-  12u;
-  12l;
-  12ul;
-  014;
-  0XC;
+  12_i;   // call: operator "" _i(12uLL)  | kind: integer
+  1.2_f;  // call: operator "" _f(1.2L)   | kind: float
+  '2'_c;  // call: operator "" _c('2')| kind: char
+  "12"_s; // call: operator "" _s("12")   | kind: string
+
+  12_r;   // call: operator "" _r("12")   | kind: integer
+  1.2_r;  // call: operator "" _i("1.2")  | kind: float
+  12_t;   // call: operator<'1', '2'> "" _x() | kind: integer
+  1.2_t;  // call: operator<'1', '2'> "" _x() | kind: float
 }
-)cpp",
+)cpp",
   R"txt(
 *: TranslationUnit
+|-SimpleDeclaration
+| |-typedef
+| |-decltype
+| |-(
+| |-UnknownExpression
+| | |-sizeof
+| | |-(
+| | |-void
+| | |-*
+| | `-)
+| |-)
+| |-SimpleDeclarator
+| | `-size_t
+| `-;
+|-SimpleDeclaration
+| |-unsigned
+| |-SimpleDeclarator
+| | |-operator
+| | |-""
+| | |-_i
+| | `-ParametersAndQualifiers
+| |   |-(
+| |   |-SimpleDeclaration
+| |   | |-unsigned
+| |   | |-long
+| |   | `-long
+| |   `-)
+| `-;
+|-SimpleDeclaration
+| |-unsigned
+| |-SimpleDeclarator
+| | |-operator
+| | |-""
+| | |-_f
+| | `-ParametersAndQualifiers
+| |   |-(
+| |   |-SimpleDeclaration
+| |   | |-long
+| |   | `-double
+| |   `-)
+| `-;
+|-SimpleDeclaration
+| |-unsigned
+| |-SimpleDeclarator
+| | |-operator
+| | |-""
+| | |-_c
+| | `-ParametersAndQualifiers
+| |   |-(
+| |   |-SimpleDeclaration
+| |   | `-char
+| |   `-)
+| `-;
+|-SimpleDeclaration
+| |-unsigned
+| |-SimpleDeclarator
+| | |-operator
+| | |-""
+| | |-_s
+| | `-ParametersAndQualifiers
+| |   |-(
+| |   |-SimpleDeclaration
+| |   | |-const
+| |   | |-char
+| |   | `-SimpleDeclarator
+| |   |   `-*
+| |   |-,
+| |   |-SimpleDeclaration
+| |   | `-size_t
+| |   `-)
+| `-;
+|-SimpleDeclaration
+| |-unsigned
+| |-SimpleDeclarator
+| | |-operator
+| | |-""
+| | |-_r
+| | `-ParametersAndQualifiers
+| |   |-(
+| |   |-SimpleDeclaration
+| |   | |-const
+| |   | |-char
+| |   | `-SimpleDeclarator
+| |   |   `-*
+| |   `-)
+| `-;
+|-TemplateDeclaration
+| |-template
+| |-<
+| |-SimpleDeclaration
+| | `-char
+| |-...
+| |->
+| `-SimpleDeclaration
+|   |-unsigned
+|   |-SimpleDeclarator
+|   | |-operator
+|   | |-""
+|   | |-_t
+|   | `-ParametersAndQualifiers
+|   |   |-(
+|   |   `-)
+|   `-;
 `-SimpleDeclaration
   |-void
   |-SimpleDeclarator
@@ -1208,28 +1327,36 @@
   `-CompoundStatement
 |-{
 |-ExpressionStatement
-| |-IntegerLiteralExpression
-| | `-12
+| |-IntegerUserDefinedLiteralExpression
+| | `-12_i
 | `-;
 |-ExpressionStatement
-| |-IntegerLiteralExpression
-| | `-12u
+| |-FloatUserDefinedLiteralExpression
+| | `-1.2_f
 | `-;
 |-ExpressionStatement
-| |-IntegerLiteralExpression
-| | `-12l
+| |-CharUserDefinedLiteralExpression
+| | `-'2'_c
 | `-;
 |-ExpressionStatement
-| |-IntegerLiteralExpression
-| | `-12ul
+| |-StringUserDefinedLiteralExpression
+| | `-"12"_s
 | `-;
 |-ExpressionStatement
-| |-IntegerLiteralExpression
-| | `-014
+| |-IntegerUserDefinedLiteralExpression
+| | `-12_r
 | `-;
 |-ExpressionStatement
-| |-IntegerLiteralExpression
-| | `-0XC
+| |-FloatUserDefinedLiteralExpression
+| | `-1.2_r
+| `-;
+|-ExpressionStatement
+| |-IntegerUserDefinedLiteralExpression
+| | `-12_t
+| `-;
+|-ExpressionStatement
+| |-FloatUserDefinedLiteralExpression
+| | `-1.2_t
 | `-;
 `-}
 )txt"));
Index: clang/lib/Tooling/Syntax/Nodes.cpp
===
--- clang/lib/Tooling/Syntax/Nodes.cpp
+++ clang/lib/Tooling/Syntax/Nodes.cpp
@@ -32,6 +32,14 @@
 return OS << "BoolLiteralExpression";
   case NodeKind::CxxNullPtrExpression:
 return OS << 

[PATCH] D82157: Fix crash on `user defined literals`

[Eduardo Caldas via Phabricator via cfe-commits]

eduucaldas updated this revision to Diff 276960.
eduucaldas added a comment.

- Add comment explaining complication on LOK_Raw and LOK_Template
- Use FileRange::text instead of Lexer::getSpelling


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D82157/new/

https://reviews.llvm.org/D82157

Files:
  clang/include/clang/Tooling/Syntax/Nodes.h
  clang/lib/Tooling/Syntax/BuildTree.cpp
  clang/lib/Tooling/Syntax/Nodes.cpp
  clang/unittests/Tooling/Syntax/TreeTest.cpp

Index: clang/unittests/Tooling/Syntax/TreeTest.cpp
===
--- clang/unittests/Tooling/Syntax/TreeTest.cpp
+++ clang/unittests/Tooling/Syntax/TreeTest.cpp
@@ -1184,20 +1184,139 @@
 )txt"));
 }
 
-TEST_P(SyntaxTreeTest, IntegerLiteral) {
+TEST_P(SyntaxTreeTest, UserDefinedLiteral) {
+  if (!GetParam().isCXX11OrLater()) {
+return;
+  }
   EXPECT_TRUE(treeDumpEqual(
   R"cpp(
+typedef decltype(sizeof(void *)) size_t;
+
+unsigned operator "" _i(unsigned long long);
+unsigned operator "" _f(long double);
+unsigned operator "" _c(char);
+unsigned operator "" _s(const char*, size_t);
+unsigned operator "" _r(const char*);
+template 
+unsigned operator "" _t();
+
 void test() {
-  12;
-  12u;
-  12l;
-  12ul;
-  014;
-  0XC;
+  12_i;   // call: operator "" _i(12uLL)  | kind: integer
+  1.2_f;  // call: operator "" _f(1.2L)   | kind: float
+  '2'_c;  // call: operator "" _c('2')| kind: char
+  "12"_s; // call: operator "" _s("12")   | kind: string
+
+  12_r;   // call: operator "" _r("12")   | kind: integer
+  1.2_r;  // call: operator "" _i("1.2")  | kind: float
+  12_t;   // call: operator<'1', '2'> "" _x() | kind: integer
+  1.2_t;  // call: operator<'1', '2'> "" _x() | kind: float
 }
-)cpp",
+)cpp",
   R"txt(
 *: TranslationUnit
+|-SimpleDeclaration
+| |-typedef
+| |-decltype
+| |-(
+| |-UnknownExpression
+| | |-sizeof
+| | |-(
+| | |-void
+| | |-*
+| | `-)
+| |-)
+| |-SimpleDeclarator
+| | `-size_t
+| `-;
+|-SimpleDeclaration
+| |-unsigned
+| |-SimpleDeclarator
+| | |-operator
+| | |-""
+| | |-_i
+| | `-ParametersAndQualifiers
+| |   |-(
+| |   |-SimpleDeclaration
+| |   | |-unsigned
+| |   | |-long
+| |   | `-long
+| |   `-)
+| `-;
+|-SimpleDeclaration
+| |-unsigned
+| |-SimpleDeclarator
+| | |-operator
+| | |-""
+| | |-_f
+| | `-ParametersAndQualifiers
+| |   |-(
+| |   |-SimpleDeclaration
+| |   | |-long
+| |   | `-double
+| |   `-)
+| `-;
+|-SimpleDeclaration
+| |-unsigned
+| |-SimpleDeclarator
+| | |-operator
+| | |-""
+| | |-_c
+| | `-ParametersAndQualifiers
+| |   |-(
+| |   |-SimpleDeclaration
+| |   | `-char
+| |   `-)
+| `-;
+|-SimpleDeclaration
+| |-unsigned
+| |-SimpleDeclarator
+| | |-operator
+| | |-""
+| | |-_s
+| | `-ParametersAndQualifiers
+| |   |-(
+| |   |-SimpleDeclaration
+| |   | |-const
+| |   | |-char
+| |   | `-SimpleDeclarator
+| |   |   `-*
+| |   |-,
+| |   |-SimpleDeclaration
+| |   | `-size_t
+| |   `-)
+| `-;
+|-SimpleDeclaration
+| |-unsigned
+| |-SimpleDeclarator
+| | |-operator
+| | |-""
+| | |-_r
+| | `-ParametersAndQualifiers
+| |   |-(
+| |   |-SimpleDeclaration
+| |   | |-const
+| |   | |-char
+| |   | `-SimpleDeclarator
+| |   |   `-*
+| |   `-)
+| `-;
+|-TemplateDeclaration
+| |-template
+| |-<
+| |-SimpleDeclaration
+| | `-char
+| |-...
+| |->
+| `-SimpleDeclaration
+|   |-unsigned
+|   |-SimpleDeclarator
+|   | |-operator
+|   | |-""
+|   | |-_t
+|   | `-ParametersAndQualifiers
+|   |   |-(
+|   |   `-)
+|   `-;
 `-SimpleDeclaration
   |-void
   |-SimpleDeclarator
@@ -1208,28 +1327,36 @@
   `-CompoundStatement
 |-{
 |-ExpressionStatement
-| |-IntegerLiteralExpression
-| | `-12
+| |-IntegerUserDefinedLiteralExpression
+| | `-12_i
 | `-;
 |-ExpressionStatement
-| |-IntegerLiteralExpression
-| | `-12u
+| |-FloatUserDefinedLiteralExpression
+| | `-1.2_f
 | `-;
 |-ExpressionStatement
-| |-IntegerLiteralExpression
-| | `-12l
+| |-CharUserDefinedLiteralExpression
+| | `-'2'_c
 | `-;
 |-ExpressionStatement
-| |-IntegerLiteralExpression
-| | `-12ul
+| |-StringUserDefinedLiteralExpression
+| | `-"12"_s
 | `-;
 |-ExpressionStatement
-| |-IntegerLiteralExpression
-| | `-014
+| |-IntegerUserDefinedLiteralExpression
+| | `-12_r
 | `-;
 |-ExpressionStatement
-| |-IntegerLiteralExpression
-| | `-0XC
+| |-FloatUserDefinedLiteralExpression
+| | `-1.2_r
+| `-;
+|-ExpressionStatement
+| |-IntegerUserDefinedLiteralExpression
+| | `-12_t
+| `-;
+|-ExpressionStatement
+| |-FloatUserDefinedLiteralExpression
+| | `-1.2_t
 | `-;
 `-}
 )txt"));
Index: clang/lib/Tooling/Syntax/Nodes.cpp
===
--- clang/lib/Tooling/Syntax/Nodes.cpp
+++ clang/lib/Tooling/Syntax/Nodes.cpp
@@ -32,6 +32,14 @@
 return OS << "BoolLiteralExpression";
   case 

[PATCH] D82157: Fix crash on `user defined literals`

[Eduardo Caldas via Phabricator via cfe-commits]

eduucaldas marked 2 inline comments as done.
eduucaldas added inline comments.



Comment at: clang/lib/Tooling/Syntax/BuildTree.cpp:723
+  syntax::UserDefinedLiteralExpression *
+  buildUserDefinedLiteral(UserDefinedLiteral *S) {
+switch (S->getLiteralOperatorKind()) {

b or B? Should I use camelCase or TitleCase?




Comment at: clang/lib/Tooling/Syntax/BuildTree.cpp:735-751
+  auto TokLoc = S->getBeginLoc();
+  auto buffer = SmallVector();
+  bool invalidSpelling = false;
+  auto TokSpelling =
+  Lexer::getSpelling(TokLoc, buffer, Context.getSourceManager(),
+ Context.getLangOpts(), );
+  assert(!invalidSpelling);

I'll write a comment explaining why this crazyness. Promise not to rant about 
the lexer that doesn't distinguish between float and integer.



Comment at: clang/lib/Tooling/Syntax/BuildTree.cpp:738-740
+  auto TokSpelling =
+  Lexer::getSpelling(TokLoc, buffer, Context.getSourceManager(),
+ Context.getLangOpts(), );

This is exactly the function used by the parser to get the `TokSpelling`. see: 
llvm-project/clang/lib/Sema/SemaExpr.cpp:3633


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D82157/new/

https://reviews.llvm.org/D82157



___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D82157: Fix crash on `user defined literals`

[Eduardo Caldas via Phabricator via cfe-commits]

eduucaldas added inline comments.



Comment at: clang/unittests/Tooling/Syntax/TreeTest.cpp:1210
+  12_t;  // call: operator<'1', '2'> "" _x() | kind: integer
+  1.2_t; // call: operator<'1', '2'> "" _x() | kind: float
 }

gribozavr2 wrote:
> call -> calls? (as in, "this expression calls ...")
it is a noun here, as kind is.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D82157/new/

https://reviews.llvm.org/D82157



___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D82157: Fix crash on `user defined literals`

[Eduardo Caldas via Phabricator via cfe-commits]

eduucaldas updated this revision to Diff 276919.
eduucaldas marked 9 inline comments as done.
eduucaldas added a comment.

Use right function to get TokSpelling, answer comments


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D82157/new/

https://reviews.llvm.org/D82157

Files:
  clang/include/clang/Tooling/Syntax/Nodes.h
  clang/lib/Tooling/Syntax/BuildTree.cpp
  clang/lib/Tooling/Syntax/Nodes.cpp
  clang/unittests/Tooling/Syntax/TreeTest.cpp

Index: clang/unittests/Tooling/Syntax/TreeTest.cpp
===
--- clang/unittests/Tooling/Syntax/TreeTest.cpp
+++ clang/unittests/Tooling/Syntax/TreeTest.cpp
@@ -1184,20 +1184,139 @@
 )txt"));
 }
 
-TEST_P(SyntaxTreeTest, IntegerLiteral) {
+TEST_P(SyntaxTreeTest, UserDefinedLiteral) {
+  if (!GetParam().isCXX11OrLater()) {
+return;
+  }
   EXPECT_TRUE(treeDumpEqual(
   R"cpp(
+typedef decltype(sizeof(void *)) size_t;
+
+unsigned operator "" _i(unsigned long long);
+unsigned operator "" _f(long double);
+unsigned operator "" _c(char);
+unsigned operator "" _s(const char*, size_t);
+unsigned operator "" _r(const char*);
+template 
+unsigned operator "" _t();
+
 void test() {
-  12;
-  12u;
-  12l;
-  12ul;
-  014;
-  0XC;
+  12_i;   // call: operator "" _i(12uLL)  | kind: integer
+  1.2_f;  // call: operator "" _f(1.2L)   | kind: float
+  '2'_c;  // call: operator "" _c('2')| kind: char
+  "12"_s; // call: operator "" _s("12")   | kind: string
+
+  12_r;   // call: operator "" _r("12")   | kind: integer
+  1.2_r;  // call: operator "" _i("1.2")  | kind: float
+  12_t;   // call: operator<'1', '2'> "" _x() | kind: integer
+  1.2_t;  // call: operator<'1', '2'> "" _x() | kind: float
 }
-)cpp",
+)cpp",
   R"txt(
 *: TranslationUnit
+|-SimpleDeclaration
+| |-typedef
+| |-decltype
+| |-(
+| |-UnknownExpression
+| | |-sizeof
+| | |-(
+| | |-void
+| | |-*
+| | `-)
+| |-)
+| |-SimpleDeclarator
+| | `-size_t
+| `-;
+|-SimpleDeclaration
+| |-unsigned
+| |-SimpleDeclarator
+| | |-operator
+| | |-""
+| | |-_i
+| | `-ParametersAndQualifiers
+| |   |-(
+| |   |-SimpleDeclaration
+| |   | |-unsigned
+| |   | |-long
+| |   | `-long
+| |   `-)
+| `-;
+|-SimpleDeclaration
+| |-unsigned
+| |-SimpleDeclarator
+| | |-operator
+| | |-""
+| | |-_f
+| | `-ParametersAndQualifiers
+| |   |-(
+| |   |-SimpleDeclaration
+| |   | |-long
+| |   | `-double
+| |   `-)
+| `-;
+|-SimpleDeclaration
+| |-unsigned
+| |-SimpleDeclarator
+| | |-operator
+| | |-""
+| | |-_c
+| | `-ParametersAndQualifiers
+| |   |-(
+| |   |-SimpleDeclaration
+| |   | `-char
+| |   `-)
+| `-;
+|-SimpleDeclaration
+| |-unsigned
+| |-SimpleDeclarator
+| | |-operator
+| | |-""
+| | |-_s
+| | `-ParametersAndQualifiers
+| |   |-(
+| |   |-SimpleDeclaration
+| |   | |-const
+| |   | |-char
+| |   | `-SimpleDeclarator
+| |   |   `-*
+| |   |-,
+| |   |-SimpleDeclaration
+| |   | `-size_t
+| |   `-)
+| `-;
+|-SimpleDeclaration
+| |-unsigned
+| |-SimpleDeclarator
+| | |-operator
+| | |-""
+| | |-_r
+| | `-ParametersAndQualifiers
+| |   |-(
+| |   |-SimpleDeclaration
+| |   | |-const
+| |   | |-char
+| |   | `-SimpleDeclarator
+| |   |   `-*
+| |   `-)
+| `-;
+|-TemplateDeclaration
+| |-template
+| |-<
+| |-SimpleDeclaration
+| | `-char
+| |-...
+| |->
+| `-SimpleDeclaration
+|   |-unsigned
+|   |-SimpleDeclarator
+|   | |-operator
+|   | |-""
+|   | |-_t
+|   | `-ParametersAndQualifiers
+|   |   |-(
+|   |   `-)
+|   `-;
 `-SimpleDeclaration
   |-void
   |-SimpleDeclarator
@@ -1208,28 +1327,36 @@
   `-CompoundStatement
 |-{
 |-ExpressionStatement
-| |-IntegerLiteralExpression
-| | `-12
+| |-IntegerUserDefinedLiteralExpression
+| | `-12_i
 | `-;
 |-ExpressionStatement
-| |-IntegerLiteralExpression
-| | `-12u
+| |-FloatUserDefinedLiteralExpression
+| | `-1.2_f
 | `-;
 |-ExpressionStatement
-| |-IntegerLiteralExpression
-| | `-12l
+| |-CharUserDefinedLiteralExpression
+| | `-'2'_c
 | `-;
 |-ExpressionStatement
-| |-IntegerLiteralExpression
-| | `-12ul
+| |-StringUserDefinedLiteralExpression
+| | `-"12"_s
 | `-;
 |-ExpressionStatement
-| |-IntegerLiteralExpression
-| | `-014
+| |-IntegerUserDefinedLiteralExpression
+| | `-12_r
 | `-;
 |-ExpressionStatement
-| |-IntegerLiteralExpression
-| | `-0XC
+| |-FloatUserDefinedLiteralExpression
+| | `-1.2_r
+| `-;
+|-ExpressionStatement
+| |-IntegerUserDefinedLiteralExpression
+| | `-12_t
+| `-;
+|-ExpressionStatement
+| |-FloatUserDefinedLiteralExpression
+| | `-1.2_t
 | `-;
 `-}
 )txt"));
Index: clang/lib/Tooling/Syntax/Nodes.cpp
===
--- clang/lib/Tooling/Syntax/Nodes.cpp
+++ clang/lib/Tooling/Syntax/Nodes.cpp
@@ -32,6 +32,14 @@
 return OS << "BoolLiteralExpression";
   case 

[PATCH] D82157: Fix crash on `user defined literals`

[Eduardo Caldas via Phabricator via cfe-commits]

eduucaldas updated this revision to Diff 276772.
eduucaldas added a comment.

Nothing


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D82157/new/

https://reviews.llvm.org/D82157

Files:
  clang/include/clang/Tooling/Syntax/Nodes.h
  clang/lib/Tooling/Syntax/BuildTree.cpp
  clang/lib/Tooling/Syntax/Nodes.cpp
  clang/unittests/Tooling/Syntax/TreeTest.cpp

Index: clang/unittests/Tooling/Syntax/TreeTest.cpp
===
--- clang/unittests/Tooling/Syntax/TreeTest.cpp
+++ clang/unittests/Tooling/Syntax/TreeTest.cpp
@@ -1184,20 +1184,105 @@
 )txt"));
 }
 
-TEST_P(SyntaxTreeTest, IntegerLiteral) {
+TEST_P(SyntaxTreeTest, UserDefinedLiteral) {
+  if (!GetParam().isCXX11OrLater()) {
+return;
+  }
   EXPECT_TRUE(treeDumpEqual(
   R"cpp(
+unsigned operator "" _i(unsigned long long);
+unsigned operator "" _f(long double);
+unsigned operator "" _c(char);
+
+unsigned operator "" _r(const char*); // raw-literal operator
+
+template 
+unsigned operator "" _t();// numeric literal operator template
+
 void test() {
-  12;
-  12u;
-  12l;
-  12ul;
-  014;
-  0XC;
+  12_i;  // call: operator "" _i(12uLL)  | kind: integer
+  1.2_f; // call: operator "" _f(1.2L)   | kind: float
+  '2'_c; // call: operator "" _c('2')| kind: char
+
+  12_r;  // call: operator "" _r("12")   | kind: integer
+  1.2_r; // call: operator "" _i("1.2")  | kind: float
+  12_t;  // call: operator<'1', '2'> "" _x() | kind: integer
+  1.2_t; // call: operator<'1', '2'> "" _x() | kind: float
 }
-)cpp",
+)cpp",
   R"txt(
 *: TranslationUnit
+|-SimpleDeclaration
+| |-unsigned
+| |-SimpleDeclarator
+| | |-operator
+| | |-""
+| | |-_i
+| | `-ParametersAndQualifiers
+| |   |-(
+| |   |-SimpleDeclaration
+| |   | |-unsigned
+| |   | |-long
+| |   | `-long
+| |   `-)
+| `-;
+|-SimpleDeclaration
+| |-unsigned
+| |-SimpleDeclarator
+| | |-operator
+| | |-""
+| | |-_f
+| | `-ParametersAndQualifiers
+| |   |-(
+| |   |-SimpleDeclaration
+| |   | |-long
+| |   | `-double
+| |   `-)
+| `-;
+|-SimpleDeclaration
+| |-unsigned
+| |-SimpleDeclarator
+| | |-operator
+| | |-""
+| | |-_c
+| | `-ParametersAndQualifiers
+| |   |-(
+| |   |-SimpleDeclaration
+| |   | `-char
+| |   `-)
+| `-;
+|-SimpleDeclaration
+| |-unsigned
+| |-SimpleDeclarator
+| | |-operator
+| | |-""
+| | |-_r
+| | `-ParametersAndQualifiers
+| |   |-(
+| |   |-SimpleDeclaration
+| |   | |-const
+| |   | |-char
+| |   | `-SimpleDeclarator
+| |   |   `-*
+| |   `-)
+| `-;
+|-TemplateDeclaration
+| |-template
+| |-<
+| |-SimpleDeclaration
+| | `-char
+| |-...
+| |->
+| `-SimpleDeclaration
+|   |-unsigned
+|   |-SimpleDeclarator
+|   | |-operator
+|   | |-""
+|   | |-_t
+|   | `-ParametersAndQualifiers
+|   |   |-(
+|   |   `-)
+|   `-;
 `-SimpleDeclaration
   |-void
   |-SimpleDeclarator
@@ -1208,28 +1293,95 @@
   `-CompoundStatement
 |-{
 |-ExpressionStatement
-| |-IntegerLiteralExpression
-| | `-12
+| |-IntegerUserDefinedLiteralExpression
+| | `-12_i
 | `-;
 |-ExpressionStatement
-| |-IntegerLiteralExpression
-| | `-12u
+| |-FloatUserDefinedLiteralExpression
+| | `-1.2_f
 | `-;
 |-ExpressionStatement
-| |-IntegerLiteralExpression
-| | `-12l
+| |-CharUserDefinedLiteralExpression
+| | `-'2'_c
 | `-;
 |-ExpressionStatement
-| |-IntegerLiteralExpression
-| | `-12ul
+| |-IntegerUserDefinedLiteralExpression
+| | `-12_r
 | `-;
 |-ExpressionStatement
-| |-IntegerLiteralExpression
-| | `-014
+| |-FloatUserDefinedLiteralExpression
+| | `-1.2_r
 | `-;
 |-ExpressionStatement
-| |-IntegerLiteralExpression
-| | `-0XC
+| |-IntegerUserDefinedLiteralExpression
+| | `-12_t
+| `-;
+|-ExpressionStatement
+| |-FloatUserDefinedLiteralExpression
+| | `-1.2_t
+| `-;
+`-}
+)txt"));
+}
+
+TEST_P(SyntaxTreeTest, UserDefinedLiteralString) {
+  if (!GetParam().isCXX11OrLater()) {
+return;
+  }
+  EXPECT_TRUE(treeDumpEqual(
+  R"cpp(
+typedef decltype(sizeof(void *)) size_t;
+unsigned operator "" _s(const char*, size_t);
+void test() {
+  "12"_s;// call: operator "" _s("12")   | kind: string
+}
+)cpp",
+  R"txt(
+*: TranslationUnit
+|-SimpleDeclaration
+| |-typedef
+| |-decltype
+| |-(
+| |-UnknownExpression
+| | |-sizeof
+| | |-(
+| | |-void
+| | |-*
+| | `-)
+| |-)
+| |-SimpleDeclarator
+| | `-size_t
+| `-;
+|-SimpleDeclaration
+| |-unsigned
+| |-SimpleDeclarator
+| | |-operator
+| | |-""
+| | |-_s
+| | `-ParametersAndQualifiers
+| |   |-(
+| |   |-SimpleDeclaration
+| |   | |-const
+| |   | |-char
+| |   | `-SimpleDeclarator
+| |   |   `-*
+| |   |-,
+| |   |-SimpleDeclaration
+| |   | `-size_t
+| |   `-)
+| `-;
+`-SimpleDeclaration
+  |-void
+  |-SimpleDeclarator
+  | |-test
+  | `-ParametersAndQualifiers
+  |   |-(
+  |   `-)
+  `-CompoundStatement
+|-{
+|-ExpressionStatement
+| 

[PATCH] D82157: Fix crash on `user defined literals`

[Eduardo Caldas via Phabricator via cfe-commits]

eduucaldas marked an inline comment as done.
eduucaldas added inline comments.



Comment at: clang/lib/Tooling/Syntax/BuildTree.cpp:736
+  auto TokSpelling =
+  Builder.findToken(TokLoc)->text(Context.getSourceManager());
+  auto Literal = NumericLiteralParser{TokSpelling,

Not sure if text is the right thing to get a `TokenSpelling` from a 
`syntax::Token`


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D82157/new/

https://reviews.llvm.org/D82157



___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D82157: Fix crash on `user defined literals`

[Eduardo Caldas via Phabricator via cfe-commits]

eduucaldas updated this revision to Diff 276771.
eduucaldas added a comment.

nothing


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D82157/new/

https://reviews.llvm.org/D82157

Files:
  clang/include/clang/Tooling/Syntax/Nodes.h
  clang/lib/Tooling/Syntax/BuildTree.cpp
  clang/lib/Tooling/Syntax/Nodes.cpp
  clang/unittests/Tooling/Syntax/TreeTest.cpp

Index: clang/unittests/Tooling/Syntax/TreeTest.cpp
===
--- clang/unittests/Tooling/Syntax/TreeTest.cpp
+++ clang/unittests/Tooling/Syntax/TreeTest.cpp
@@ -1204,9 +1204,6 @@
   1.2_f; // call: operator "" _f(1.2L)   | kind: float
   '2'_c; // call: operator "" _c('2')| kind: char
 
-  // TODO: Generate `FloatUserDefinedLiteralExpression` and
-  // `IntegerUserDefinedLiteralExpression` instead of
-  // `UnknownUserDefinedLiteralExpression`. See `getUserDefinedLiteralKind`
   12_r;  // call: operator "" _r("12")   | kind: integer
   1.2_r; // call: operator "" _i("1.2")  | kind: float
   12_t;  // call: operator<'1', '2'> "" _x() | kind: integer
@@ -1308,19 +1305,19 @@
 | | `-'2'_c
 | `-;
 |-ExpressionStatement
-| |-UnknownUserDefinedLiteralExpression
+| |-IntegerUserDefinedLiteralExpression
 | | `-12_r
 | `-;
 |-ExpressionStatement
-| |-UnknownUserDefinedLiteralExpression
+| |-FloatUserDefinedLiteralExpression
 | | `-1.2_r
 | `-;
 |-ExpressionStatement
-| |-UnknownUserDefinedLiteralExpression
+| |-IntegerUserDefinedLiteralExpression
 | | `-12_t
 | `-;
 |-ExpressionStatement
-| |-UnknownUserDefinedLiteralExpression
+| |-FloatUserDefinedLiteralExpression
 | | `-1.2_t
 | `-;
 `-}
Index: clang/lib/Tooling/Syntax/Nodes.cpp
===
--- clang/lib/Tooling/Syntax/Nodes.cpp
+++ clang/lib/Tooling/Syntax/Nodes.cpp
@@ -32,8 +32,6 @@
 return OS << "BoolLiteralExpression";
   case NodeKind::CxxNullPtrExpression:
 return OS << "CxxNullPtrExpression";
-  case NodeKind::UnknownUserDefinedLiteralExpression:
-return OS << "UnknownUserDefinedLiteralExpression";
   case NodeKind::IntegerUserDefinedLiteralExpression:
 return OS << "IntegerUserDefinedLiteralExpression";
   case NodeKind::FloatUserDefinedLiteralExpression:
Index: clang/lib/Tooling/Syntax/BuildTree.cpp
===
--- clang/lib/Tooling/Syntax/BuildTree.cpp
+++ clang/lib/Tooling/Syntax/BuildTree.cpp
@@ -23,6 +23,7 @@
 #include "clang/Basic/Specifiers.h"
 #include "clang/Basic/TokenKinds.h"
 #include "clang/Lex/Lexer.h"
+#include "clang/Lex/LiteralSupport.h"
 #include "clang/Tooling/Syntax/Nodes.h"
 #include "clang/Tooling/Syntax/Tokens.h"
 #include "clang/Tooling/Syntax/Tree.h"
@@ -552,8 +553,8 @@
 namespace {
 class BuildTreeVisitor : public RecursiveASTVisitor {
 public:
-  explicit BuildTreeVisitor(ASTContext , syntax::TreeBuilder )
-  : Builder(Builder), LangOpts(Ctx.getLangOpts()) {}
+  explicit BuildTreeVisitor(ASTContext , syntax::TreeBuilder )
+  : Builder(Builder), Context(Context) {}
 
   bool shouldTraversePostOrder() const { return true; }
 
@@ -730,9 +731,19 @@
   return syntax::NodeKind::StringUserDefinedLiteralExpression;
 case clang::UserDefinedLiteral::LOK_Raw:
 case clang::UserDefinedLiteral::LOK_Template:
-  // FIXME: Apply `NumericLiteralParser` to the underlying token to deduce
-  // the right UDL kind. That would require a `Preprocessor` though.
-  return syntax::NodeKind::UnknownUserDefinedLiteralExpression;
+  auto TokLoc = S->getBeginLoc();
+  auto TokSpelling =
+  Builder.findToken(TokLoc)->text(Context.getSourceManager());
+  auto Literal = NumericLiteralParser{TokSpelling,
+  TokLoc,
+  Context.getSourceManager(),
+  Context.getLangOpts(),
+  Context.getTargetInfo(),
+  Context.getDiagnostics()};
+  if (Literal.isIntegerLiteral())
+return syntax::NodeKind::IntegerUserDefinedLiteralExpression;
+  else
+return syntax::NodeKind::FloatUserDefinedLiteralExpression;
 }
   }
 
@@ -1262,7 +1273,7 @@
   llvm::BumpPtrAllocator () { return Builder.allocator(); }
 
   syntax::TreeBuilder 
-  const LangOptions 
+  const ASTContext 
 };
 } // namespace
 
Index: clang/include/clang/Tooling/Syntax/Nodes.h
===
--- clang/include/clang/Tooling/Syntax/Nodes.h
+++ clang/include/clang/Tooling/Syntax/Nodes.h
@@ -50,7 +50,6 @@
   StringLiteralExpression,
   BoolLiteralExpression,
   CxxNullPtrExpression,
-  UnknownUserDefinedLiteralExpression,
   IntegerUserDefinedLiteralExpression,
   FloatUserDefinedLiteralExpression,
   

[PATCH] D82157: Fix crash on `user defined literals`

[Eduardo Caldas via Phabricator via cfe-commits]

eduucaldas updated this revision to Diff 276770.
eduucaldas added a comment.

Add support for integer and floating UDL even for raw literal operator and 
numeric literal operator template


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D82157/new/

https://reviews.llvm.org/D82157

Files:
  clang/include/clang/AST/Expr.h
  clang/include/clang/Tooling/Syntax/Nodes.h
  clang/lib/Tooling/Syntax/BuildTree.cpp
  clang/lib/Tooling/Syntax/Nodes.cpp
  clang/unittests/Tooling/Syntax/TreeTest.cpp

Index: clang/unittests/Tooling/Syntax/TreeTest.cpp
===
--- clang/unittests/Tooling/Syntax/TreeTest.cpp
+++ clang/unittests/Tooling/Syntax/TreeTest.cpp
@@ -1204,9 +1204,6 @@
   1.2_f; // call: operator "" _f(1.2L)   | kind: float
   '2'_c; // call: operator "" _c('2')| kind: char
 
-  // TODO: Generate `FloatUserDefinedLiteralExpression` and
-  // `IntegerUserDefinedLiteralExpression` instead of
-  // `UnknownUserDefinedLiteralExpression`. See `getUserDefinedLiteralKind`
   12_r;  // call: operator "" _r("12")   | kind: integer
   1.2_r; // call: operator "" _i("1.2")  | kind: float
   12_t;  // call: operator<'1', '2'> "" _x() | kind: integer
@@ -1308,19 +1305,19 @@
 | | `-'2'_c
 | `-;
 |-ExpressionStatement
-| |-UnknownUserDefinedLiteralExpression
+| |-IntegerUserDefinedLiteralExpression
 | | `-12_r
 | `-;
 |-ExpressionStatement
-| |-UnknownUserDefinedLiteralExpression
+| |-FloatUserDefinedLiteralExpression
 | | `-1.2_r
 | `-;
 |-ExpressionStatement
-| |-UnknownUserDefinedLiteralExpression
+| |-IntegerUserDefinedLiteralExpression
 | | `-12_t
 | `-;
 |-ExpressionStatement
-| |-UnknownUserDefinedLiteralExpression
+| |-FloatUserDefinedLiteralExpression
 | | `-1.2_t
 | `-;
 `-}
Index: clang/lib/Tooling/Syntax/Nodes.cpp
===
--- clang/lib/Tooling/Syntax/Nodes.cpp
+++ clang/lib/Tooling/Syntax/Nodes.cpp
@@ -32,8 +32,6 @@
 return OS << "BoolLiteralExpression";
   case NodeKind::CxxNullPtrExpression:
 return OS << "CxxNullPtrExpression";
-  case NodeKind::UnknownUserDefinedLiteralExpression:
-return OS << "UnknownUserDefinedLiteralExpression";
   case NodeKind::IntegerUserDefinedLiteralExpression:
 return OS << "IntegerUserDefinedLiteralExpression";
   case NodeKind::FloatUserDefinedLiteralExpression:
Index: clang/lib/Tooling/Syntax/BuildTree.cpp
===
--- clang/lib/Tooling/Syntax/BuildTree.cpp
+++ clang/lib/Tooling/Syntax/BuildTree.cpp
@@ -23,6 +23,7 @@
 #include "clang/Basic/Specifiers.h"
 #include "clang/Basic/TokenKinds.h"
 #include "clang/Lex/Lexer.h"
+#include "clang/Lex/LiteralSupport.h"
 #include "clang/Tooling/Syntax/Nodes.h"
 #include "clang/Tooling/Syntax/Tokens.h"
 #include "clang/Tooling/Syntax/Tree.h"
@@ -552,8 +553,8 @@
 namespace {
 class BuildTreeVisitor : public RecursiveASTVisitor {
 public:
-  explicit BuildTreeVisitor(ASTContext , syntax::TreeBuilder )
-  : Builder(Builder), LangOpts(Ctx.getLangOpts()) {}
+  explicit BuildTreeVisitor(ASTContext , syntax::TreeBuilder )
+  : Builder(Builder), Context(Context) {}
 
   bool shouldTraversePostOrder() const { return true; }
 
@@ -730,9 +731,19 @@
   return syntax::NodeKind::StringUserDefinedLiteralExpression;
 case clang::UserDefinedLiteral::LOK_Raw:
 case clang::UserDefinedLiteral::LOK_Template:
-  // FIXME: Apply `NumericLiteralParser` to the underlying token to deduce
-  // the right UDL kind. That would require a `Preprocessor` though.
-  return syntax::NodeKind::UnknownUserDefinedLiteralExpression;
+  auto TokLoc = S->getBeginLoc();
+  auto TokSpelling =
+  Builder.findToken(TokLoc)->text(Context.getSourceManager());
+  auto Literal = NumericLiteralParser{TokSpelling,
+  TokLoc,
+  Context.getSourceManager(),
+  Context.getLangOpts(),
+  Context.getTargetInfo(),
+  Context.getDiagnostics()};
+  if (Literal.isIntegerLiteral())
+return syntax::NodeKind::IntegerUserDefinedLiteralExpression;
+  else
+return syntax::NodeKind::FloatUserDefinedLiteralExpression;
 }
   }
 
@@ -1262,7 +1273,7 @@
   llvm::BumpPtrAllocator () { return Builder.allocator(); }
 
   syntax::TreeBuilder 
-  const LangOptions 
+  const ASTContext 
 };
 } // namespace
 
Index: clang/include/clang/Tooling/Syntax/Nodes.h
===
--- clang/include/clang/Tooling/Syntax/Nodes.h
+++ clang/include/clang/Tooling/Syntax/Nodes.h
@@ -50,7 +50,6 @@
   StringLiteralExpression,
   BoolLiteralExpression,
   

[PATCH] D83480: Refactored NumericLiteralParser to not require a Preprocessor

[Eduardo Caldas via Phabricator via cfe-commits]

eduucaldas accepted this revision.
eduucaldas added a comment.
This revision is now accepted and ready to land.

Additionally, consider applying a similar change to `CharLiteralParser`, for 
consistency.

All the comments are just suggestions. Feel free to land this without answering 
them.




Comment at: clang/include/clang/Lex/LiteralSupport.h:57-59
-  NumericLiteralParser(StringRef TokSpelling,
-   SourceLocation TokLoc,
-   Preprocessor );

We don't need to remove this constructor, we can keep the same signature and 
make it call the new constructor. The same is done for `StringLiteralParser`.

That would allow some callers that don't care much about the implementation 
details to just use the simpler to write version.



Comment at: clang/lib/Lex/LiteralSupport.cpp:766
+  !isValidUDSuffix(LangOpts, StringRef(s, ThisTokEnd - s))) {
+Diags.Report(AdvanceToTokenCharacter(TokLoc, s - ThisTokBegin),
+ diag::err_invalid_digit)

How about just using `Lexer::AdvanceToTokenCharacter`? 


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D83480/new/

https://reviews.llvm.org/D83480



___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D82157: Fix crash on `user defined literals`

[Eduardo Caldas via Phabricator via cfe-commits]

eduucaldas updated this revision to Diff 276681.
eduucaldas added a comment.

Document proposed solution for treating raw literal operators and numeric 
template operators


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D82157/new/

https://reviews.llvm.org/D82157

Files:
  clang/include/clang/Tooling/Syntax/Nodes.h
  clang/lib/Tooling/Syntax/BuildTree.cpp
  clang/lib/Tooling/Syntax/Nodes.cpp
  clang/unittests/Tooling/Syntax/TreeTest.cpp

Index: clang/unittests/Tooling/Syntax/TreeTest.cpp
===
--- clang/unittests/Tooling/Syntax/TreeTest.cpp
+++ clang/unittests/Tooling/Syntax/TreeTest.cpp
@@ -1184,20 +1184,108 @@
 )txt"));
 }
 
-TEST_P(SyntaxTreeTest, IntegerLiteral) {
+TEST_P(SyntaxTreeTest, UserDefinedLiteral) {
+  if (!GetParam().isCXX11OrLater()) {
+return;
+  }
   EXPECT_TRUE(treeDumpEqual(
   R"cpp(
+unsigned operator "" _i(unsigned long long);
+unsigned operator "" _f(long double);
+unsigned operator "" _c(char);
+
+unsigned operator "" _r(const char*); // raw-literal operator
+
+template 
+unsigned operator "" _t();// numeric literal operator template
+
 void test() {
-  12;
-  12u;
-  12l;
-  12ul;
-  014;
-  0XC;
+  12_i;  // call: operator "" _i(12uLL)  | kind: integer
+  1.2_f; // call: operator "" _f(1.2L)   | kind: float
+  '2'_c; // call: operator "" _c('2')| kind: char
+
+  // TODO: Generate `FloatUserDefinedLiteralExpression` and
+  // `IntegerUserDefinedLiteralExpression` instead of
+  // `UnknownUserDefinedLiteralExpression`. See `getUserDefinedLiteralKind`
+  12_r;  // call: operator "" _r("12")   | kind: integer
+  1.2_r; // call: operator "" _i("1.2")  | kind: float
+  12_t;  // call: operator<'1', '2'> "" _x() | kind: integer
+  1.2_t; // call: operator<'1', '2'> "" _x() | kind: float
 }
-)cpp",
+)cpp",
   R"txt(
 *: TranslationUnit
+|-SimpleDeclaration
+| |-unsigned
+| |-SimpleDeclarator
+| | |-operator
+| | |-""
+| | |-_i
+| | `-ParametersAndQualifiers
+| |   |-(
+| |   |-SimpleDeclaration
+| |   | |-unsigned
+| |   | |-long
+| |   | `-long
+| |   `-)
+| `-;
+|-SimpleDeclaration
+| |-unsigned
+| |-SimpleDeclarator
+| | |-operator
+| | |-""
+| | |-_f
+| | `-ParametersAndQualifiers
+| |   |-(
+| |   |-SimpleDeclaration
+| |   | |-long
+| |   | `-double
+| |   `-)
+| `-;
+|-SimpleDeclaration
+| |-unsigned
+| |-SimpleDeclarator
+| | |-operator
+| | |-""
+| | |-_c
+| | `-ParametersAndQualifiers
+| |   |-(
+| |   |-SimpleDeclaration
+| |   | `-char
+| |   `-)
+| `-;
+|-SimpleDeclaration
+| |-unsigned
+| |-SimpleDeclarator
+| | |-operator
+| | |-""
+| | |-_r
+| | `-ParametersAndQualifiers
+| |   |-(
+| |   |-SimpleDeclaration
+| |   | |-const
+| |   | |-char
+| |   | `-SimpleDeclarator
+| |   |   `-*
+| |   `-)
+| `-;
+|-TemplateDeclaration
+| |-template
+| |-<
+| |-SimpleDeclaration
+| | `-char
+| |-...
+| |->
+| `-SimpleDeclaration
+|   |-unsigned
+|   |-SimpleDeclarator
+|   | |-operator
+|   | |-""
+|   | |-_t
+|   | `-ParametersAndQualifiers
+|   |   |-(
+|   |   `-)
+|   `-;
 `-SimpleDeclaration
   |-void
   |-SimpleDeclarator
@@ -1208,28 +1296,95 @@
   `-CompoundStatement
 |-{
 |-ExpressionStatement
-| |-IntegerLiteralExpression
-| | `-12
+| |-IntegerUserDefinedLiteralExpression
+| | `-12_i
 | `-;
 |-ExpressionStatement
-| |-IntegerLiteralExpression
-| | `-12u
+| |-FloatUserDefinedLiteralExpression
+| | `-1.2_f
 | `-;
 |-ExpressionStatement
-| |-IntegerLiteralExpression
-| | `-12l
+| |-CharUserDefinedLiteralExpression
+| | `-'2'_c
 | `-;
 |-ExpressionStatement
-| |-IntegerLiteralExpression
-| | `-12ul
+| |-UnknownUserDefinedLiteralExpression
+| | `-12_r
 | `-;
 |-ExpressionStatement
-| |-IntegerLiteralExpression
-| | `-014
+| |-UnknownUserDefinedLiteralExpression
+| | `-1.2_r
 | `-;
 |-ExpressionStatement
-| |-IntegerLiteralExpression
-| | `-0XC
+| |-UnknownUserDefinedLiteralExpression
+| | `-12_t
+| `-;
+|-ExpressionStatement
+| |-UnknownUserDefinedLiteralExpression
+| | `-1.2_t
+| `-;
+`-}
+)txt"));
+}
+
+TEST_P(SyntaxTreeTest, UserDefinedLiteralString) {
+  if (!GetParam().isCXX11OrLater()) {
+return;
+  }
+  EXPECT_TRUE(treeDumpEqual(
+  R"cpp(
+typedef decltype(sizeof(void *)) size_t;
+unsigned operator "" _s(const char*, size_t);
+void test() {
+  "12"_s;// call: operator "" _s("12")   | kind: string
+}
+)cpp",
+  R"txt(
+*: TranslationUnit
+|-SimpleDeclaration
+| |-typedef
+| |-decltype
+| |-(
+| |-UnknownExpression
+| | |-sizeof
+| | |-(
+| | |-void
+| | |-*
+| | `-)
+| |-)
+| |-SimpleDeclarator
+| | `-size_t
+| `-;
+|-SimpleDeclaration
+| |-unsigned
+| |-SimpleDeclarator
+| | |-operator
+| | |-""
+| | |-_s
+| | `-ParametersAndQualifiers
+| |   |-(
+| |   |-SimpleDeclaration
+| |   | |-const
+| |   | |-char
+| |   | 

[PATCH] D82157: Fix crash on `user defined literals`

[Eduardo Caldas via Phabricator via cfe-commits]

eduucaldas updated this revision to Diff 276678.
eduucaldas added a comment.

Polishing patch


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D82157/new/

https://reviews.llvm.org/D82157

Files:
  clang/include/clang/Tooling/Syntax/Nodes.h
  clang/lib/Tooling/Syntax/BuildTree.cpp
  clang/lib/Tooling/Syntax/Nodes.cpp
  clang/unittests/Tooling/Syntax/TreeTest.cpp

Index: clang/unittests/Tooling/Syntax/TreeTest.cpp
===
--- clang/unittests/Tooling/Syntax/TreeTest.cpp
+++ clang/unittests/Tooling/Syntax/TreeTest.cpp
@@ -1184,20 +1184,106 @@
 )txt"));
 }
 
-TEST_P(SyntaxTreeTest, IntegerLiteral) {
+TEST_P(SyntaxTreeTest, UserDefinedLiteral) {
+  if (!GetParam().isCXX11OrLater()) {
+return;
+  }
   EXPECT_TRUE(treeDumpEqual(
   R"cpp(
+unsigned operator "" _i(unsigned long long);
+unsigned operator "" _f(long double);
+unsigned operator "" _c(char);
+
+unsigned operator "" _r(const char*); // raw-literal operator
+
+template 
+unsigned operator "" _t();// numeric literal operator template
+
 void test() {
-  12;
-  12u;
-  12l;
-  12ul;
-  014;
-  0XC;
+  12_i;  // call: operator "" _i(12uLL)  | kind: integer
+  1.2_f; // call: operator "" _f(1.2L)   | kind: float
+  '2'_c; // call: operator "" _c('2')| kind: char
+
+  // PROBLEM: How to discover the kind of user-defined-literal from the AST?
+  12_r;  // call: operator "" _r("12")   | kind: integer
+  1.2_r; // call: operator "" _i("1.2")  | kind: float
+  12_t;  // call: operator<'1', '2'> "" _x() | kind: integer
+  1.2_t; // call: operator<'1', '2'> "" _x() | kind: float
 }
-)cpp",
+)cpp",
   R"txt(
 *: TranslationUnit
+|-SimpleDeclaration
+| |-unsigned
+| |-SimpleDeclarator
+| | |-operator
+| | |-""
+| | |-_i
+| | `-ParametersAndQualifiers
+| |   |-(
+| |   |-SimpleDeclaration
+| |   | |-unsigned
+| |   | |-long
+| |   | `-long
+| |   `-)
+| `-;
+|-SimpleDeclaration
+| |-unsigned
+| |-SimpleDeclarator
+| | |-operator
+| | |-""
+| | |-_f
+| | `-ParametersAndQualifiers
+| |   |-(
+| |   |-SimpleDeclaration
+| |   | |-long
+| |   | `-double
+| |   `-)
+| `-;
+|-SimpleDeclaration
+| |-unsigned
+| |-SimpleDeclarator
+| | |-operator
+| | |-""
+| | |-_c
+| | `-ParametersAndQualifiers
+| |   |-(
+| |   |-SimpleDeclaration
+| |   | `-char
+| |   `-)
+| `-;
+|-SimpleDeclaration
+| |-unsigned
+| |-SimpleDeclarator
+| | |-operator
+| | |-""
+| | |-_r
+| | `-ParametersAndQualifiers
+| |   |-(
+| |   |-SimpleDeclaration
+| |   | |-const
+| |   | |-char
+| |   | `-SimpleDeclarator
+| |   |   `-*
+| |   `-)
+| `-;
+|-TemplateDeclaration
+| |-template
+| |-<
+| |-SimpleDeclaration
+| | `-char
+| |-...
+| |->
+| `-SimpleDeclaration
+|   |-unsigned
+|   |-SimpleDeclarator
+|   | |-operator
+|   | |-""
+|   | |-_t
+|   | `-ParametersAndQualifiers
+|   |   |-(
+|   |   `-)
+|   `-;
 `-SimpleDeclaration
   |-void
   |-SimpleDeclarator
@@ -1208,28 +1294,95 @@
   `-CompoundStatement
 |-{
 |-ExpressionStatement
-| |-IntegerLiteralExpression
-| | `-12
+| |-IntegerUserDefinedLiteralExpression
+| | `-12_i
 | `-;
 |-ExpressionStatement
-| |-IntegerLiteralExpression
-| | `-12u
+| |-FloatUserDefinedLiteralExpression
+| | `-1.2_f
 | `-;
 |-ExpressionStatement
-| |-IntegerLiteralExpression
-| | `-12l
+| |-CharUserDefinedLiteralExpression
+| | `-'2'_c
 | `-;
 |-ExpressionStatement
-| |-IntegerLiteralExpression
-| | `-12ul
+| |-UnknownUserDefinedLiteralExpression
+| | `-12_r
 | `-;
 |-ExpressionStatement
-| |-IntegerLiteralExpression
-| | `-014
+| |-UnknownUserDefinedLiteralExpression
+| | `-1.2_r
 | `-;
 |-ExpressionStatement
-| |-IntegerLiteralExpression
-| | `-0XC
+| |-UnknownUserDefinedLiteralExpression
+| | `-12_t
+| `-;
+|-ExpressionStatement
+| |-UnknownUserDefinedLiteralExpression
+| | `-1.2_t
+| `-;
+`-}
+)txt"));
+}
+
+TEST_P(SyntaxTreeTest, UserDefinedLiteralString) {
+  if (!GetParam().isCXX11OrLater()) {
+return;
+  }
+  EXPECT_TRUE(treeDumpEqual(
+  R"cpp(
+typedef decltype(sizeof(void *)) size_t;
+unsigned operator "" _s(const char*, size_t);
+void test() {
+  "12"_s;// call: operator "" _s("12")   | kind: string
+}
+)cpp",
+  R"txt(
+*: TranslationUnit
+|-SimpleDeclaration
+| |-typedef
+| |-decltype
+| |-(
+| |-UnknownExpression
+| | |-sizeof
+| | |-(
+| | |-void
+| | |-*
+| | `-)
+| |-)
+| |-SimpleDeclarator
+| | `-size_t
+| `-;
+|-SimpleDeclaration
+| |-unsigned
+| |-SimpleDeclarator
+| | |-operator
+| | |-""
+| | |-_s
+| | `-ParametersAndQualifiers
+| |   |-(
+| |   |-SimpleDeclaration
+| |   | |-const
+| |   | |-char
+| |   | `-SimpleDeclarator
+| |   |   `-*
+| |   |-,
+| |   |-SimpleDeclaration
+| |   | `-size_t
+| |   `-)
+| `-;
+`-SimpleDeclaration
+  |-void
+  |-SimpleDeclarator
+  | |-test
+  | `-ParametersAndQualifiers
+  | 

[PATCH] D82157: Fix crash on `user defined literals`

[Eduardo Caldas via Phabricator via cfe-commits]

eduucaldas added inline comments.



Comment at: clang/lib/Tooling/Syntax/BuildTree.cpp:634
+// `SourceLocation`s. As a result one of these nodes has a valid
+// `SourceLocation` that doesn't point to a token.
+//

gribozavr2 wrote:
> "The semantic AST node for has child nodes that reference two source 
> locations, the location of the beginning of the token (`1`), and the location 
> of the beginning of the UDL suffix (`_`). The UDL suffix location does not 
> point to the beginning of a token, so we can't represent the UDL suffix as a 
> separate syntax tree node."
I changed your comment slightly, WDYT


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D82157/new/

https://reviews.llvm.org/D82157



___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D82157: Fix crash on `user defined literals`

[Eduardo Caldas via Phabricator via cfe-commits]

eduucaldas marked 4 inline comments as done.
eduucaldas added inline comments.



Comment at: clang/include/clang/Tooling/Syntax/Nodes.h:347-357
+/// Expression for an unkonwn user-defined literal. C++ [lex.ext]
+class UnknownUserDefinedLiteralExpression final
+: public UserDefinedLiteralExpression {
+public:
+  UnknownUserDefinedLiteralExpression()
+  : UserDefinedLiteralExpression(
+NodeKind::UnknownUserDefinedLiteralExpression) {}

This is gonna be removed once we are able to distinguish between integer and 
float on raw and template UDL



Comment at: clang/lib/Tooling/Syntax/BuildTree.cpp:218-219
 auto InitializerEnd = Initializer.getEnd();
-assert(SM.isBeforeInTranslationUnit(End, InitializerEnd) || End == 
InitializerEnd);
+assert(SM.isBeforeInTranslationUnit(End, InitializerEnd) ||
+   End == InitializerEnd);
 End = InitializerEnd;

I know it adds noise, but this is the only place where the formatting doesn't 
obey the llvm clang format.



Comment at: clang/lib/Tooling/Syntax/BuildTree.cpp:732
+case clang::UserDefinedLiteral::LOK_Template:
+  return syntax::NodeKind::UnknownUserDefinedLiteralExpression;
+}

Here we need logic to determine which kind of UDL, if float or integer



Comment at: clang/unittests/Tooling/Syntax/TreeTest.cpp:1309-1310
 |-ExpressionStatement
-| |-IntegerLiteralExpression
-| | `-12ul
+| |-UnknownUserDefinedLiteralExpression
+| | `-12_r
 | `-;

For raw and template UDL, we don't know yet how to get the information about 
integer of float


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D82157/new/

https://reviews.llvm.org/D82157



___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D82157: Fix crash on `user defined literals`

[Eduardo Caldas via Phabricator via cfe-commits]

eduucaldas updated this revision to Diff 276638.
eduucaldas added a comment.

workaround size_t


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D82157/new/

https://reviews.llvm.org/D82157

Files:
  clang/include/clang/Testing/TestClangConfig.h
  clang/include/clang/Tooling/Syntax/Nodes.h
  clang/lib/Tooling/Syntax/BuildTree.cpp
  clang/lib/Tooling/Syntax/Nodes.cpp
  clang/unittests/Tooling/Syntax/TreeTest.cpp

Index: clang/unittests/Tooling/Syntax/TreeTest.cpp
===
--- clang/unittests/Tooling/Syntax/TreeTest.cpp
+++ clang/unittests/Tooling/Syntax/TreeTest.cpp
@@ -1184,20 +1184,106 @@
 )txt"));
 }
 
-TEST_P(SyntaxTreeTest, IntegerLiteral) {
+TEST_P(SyntaxTreeTest, UserDefinedLiteral) {
+  if (!GetParam().isCXX11OrLater()) {
+return;
+  }
   EXPECT_TRUE(treeDumpEqual(
   R"cpp(
+unsigned operator "" _i(unsigned long long);
+unsigned operator "" _f(long double);
+unsigned operator "" _c(char);
+
+unsigned operator "" _r(const char*); // raw-literal operator
+
+template 
+unsigned operator "" _t();// numeric literal operator template
+
 void test() {
-  12;
-  12u;
-  12l;
-  12ul;
-  014;
-  0XC;
+  12_i;  // call: operator "" _i(12uLL)  | kind: integer
+  1.2_f; // call: operator "" _f(1.2L)   | kind: float
+  '2'_c; // call: operator "" _c('2')| kind: char
+
+  // PROBLEM: How to discover the kind of user-defined-literal from the AST?
+  12_r;  // call: operator "" _r("12")   | kind: integer
+  1.2_r; // call: operator "" _i("1.2")  | kind: float
+  12_t;  // call: operator<'1', '2'> "" _x() | kind: integer
+  1.2_t; // call: operator<'1', '2'> "" _x() | kind: float
 }
-)cpp",
+)cpp",
   R"txt(
 *: TranslationUnit
+|-SimpleDeclaration
+| |-unsigned
+| |-SimpleDeclarator
+| | |-operator
+| | |-""
+| | |-_i
+| | `-ParametersAndQualifiers
+| |   |-(
+| |   |-SimpleDeclaration
+| |   | |-unsigned
+| |   | |-long
+| |   | `-long
+| |   `-)
+| `-;
+|-SimpleDeclaration
+| |-unsigned
+| |-SimpleDeclarator
+| | |-operator
+| | |-""
+| | |-_f
+| | `-ParametersAndQualifiers
+| |   |-(
+| |   |-SimpleDeclaration
+| |   | |-long
+| |   | `-double
+| |   `-)
+| `-;
+|-SimpleDeclaration
+| |-unsigned
+| |-SimpleDeclarator
+| | |-operator
+| | |-""
+| | |-_c
+| | `-ParametersAndQualifiers
+| |   |-(
+| |   |-SimpleDeclaration
+| |   | `-char
+| |   `-)
+| `-;
+|-SimpleDeclaration
+| |-unsigned
+| |-SimpleDeclarator
+| | |-operator
+| | |-""
+| | |-_r
+| | `-ParametersAndQualifiers
+| |   |-(
+| |   |-SimpleDeclaration
+| |   | |-const
+| |   | |-char
+| |   | `-SimpleDeclarator
+| |   |   `-*
+| |   `-)
+| `-;
+|-TemplateDeclaration
+| |-template
+| |-<
+| |-SimpleDeclaration
+| | `-char
+| |-...
+| |->
+| `-SimpleDeclaration
+|   |-unsigned
+|   |-SimpleDeclarator
+|   | |-operator
+|   | |-""
+|   | |-_t
+|   | `-ParametersAndQualifiers
+|   |   |-(
+|   |   `-)
+|   `-;
 `-SimpleDeclaration
   |-void
   |-SimpleDeclarator
@@ -1208,28 +1294,95 @@
   `-CompoundStatement
 |-{
 |-ExpressionStatement
-| |-IntegerLiteralExpression
-| | `-12
+| |-IntegerUserDefinedLiteralExpression
+| | `-12_i
 | `-;
 |-ExpressionStatement
-| |-IntegerLiteralExpression
-| | `-12u
+| |-FloatUserDefinedLiteralExpression
+| | `-1.2_f
 | `-;
 |-ExpressionStatement
-| |-IntegerLiteralExpression
-| | `-12l
+| |-CharUserDefinedLiteralExpression
+| | `-'2'_c
 | `-;
 |-ExpressionStatement
-| |-IntegerLiteralExpression
-| | `-12ul
+| |-UnknownUserDefinedLiteralExpression
+| | `-12_r
 | `-;
 |-ExpressionStatement
-| |-IntegerLiteralExpression
-| | `-014
+| |-UnknownUserDefinedLiteralExpression
+| | `-1.2_r
 | `-;
 |-ExpressionStatement
-| |-IntegerLiteralExpression
-| | `-0XC
+| |-UnknownUserDefinedLiteralExpression
+| | `-12_t
+| `-;
+|-ExpressionStatement
+| |-UnknownUserDefinedLiteralExpression
+| | `-1.2_t
+| `-;
+`-}
+)txt"));
+}
+
+TEST_P(SyntaxTreeTest, UserDefinedLiteralString) {
+  if (!GetParam().isCXX11OrLater()) {
+return;
+  }
+  EXPECT_TRUE(treeDumpEqual(
+  R"cpp(
+typedef decltype(sizeof(void *)) size_t;
+unsigned operator "" _s(const char*, size_t);
+void test() {
+  "12"_s;// call: operator "" _s("12")   | kind: string
+}
+)cpp",
+  R"txt(
+*: TranslationUnit
+|-SimpleDeclaration
+| |-typedef
+| |-decltype
+| |-(
+| |-UnknownExpression
+| | |-sizeof
+| | |-(
+| | |-void
+| | |-*
+| | `-)
+| |-)
+| |-SimpleDeclarator
+| | `-size_t
+| `-;
+|-SimpleDeclaration
+| |-unsigned
+| |-SimpleDeclarator
+| | |-operator
+| | |-""
+| | |-_s
+| | `-ParametersAndQualifiers
+| |   |-(
+| |   |-SimpleDeclaration
+| |   | |-const
+| |   | |-char
+| |   | `-SimpleDeclarator
+| |   |   `-*
+| |   |-,
+| |   |-SimpleDeclaration
+| |   | `-size_t
+| |   `-)
+| `-;
+`-SimpleDeclaration
+  |-void
+  

[PATCH] D82157: Fix crash on `user defined literals`

[Eduardo Caldas via Phabricator via cfe-commits]

eduucaldas updated this revision to Diff 276637.
eduucaldas marked 4 inline comments as done.
eduucaldas added a comment.

Add support for {Integer,Float,Char,String}UserDefinedLiteral


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D82157/new/

https://reviews.llvm.org/D82157

Files:
  clang/include/clang/Testing/TestClangConfig.h
  clang/include/clang/Tooling/Syntax/Nodes.h
  clang/lib/Tooling/Syntax/BuildTree.cpp
  clang/lib/Tooling/Syntax/Nodes.cpp
  clang/unittests/Tooling/Syntax/TreeTest.cpp

Index: clang/unittests/Tooling/Syntax/TreeTest.cpp
===
--- clang/unittests/Tooling/Syntax/TreeTest.cpp
+++ clang/unittests/Tooling/Syntax/TreeTest.cpp
@@ -1184,20 +1184,106 @@
 )txt"));
 }
 
-TEST_P(SyntaxTreeTest, IntegerLiteral) {
+TEST_P(SyntaxTreeTest, UserDefinedLiteral) {
+  if (!GetParam().isCXX11OrLater()) {
+return;
+  }
   EXPECT_TRUE(treeDumpEqual(
   R"cpp(
+unsigned operator "" _i(unsigned long long);
+unsigned operator "" _f(long double);
+unsigned operator "" _c(char);
+
+unsigned operator "" _r(const char*); // raw-literal operator
+
+template 
+unsigned operator "" _t();// numeric literal operator template
+
 void test() {
-  12;
-  12u;
-  12l;
-  12ul;
-  014;
-  0XC;
+  12_i;  // call: operator "" _i(12uLL)  | kind: integer
+  1.2_f; // call: operator "" _f(1.2L)   | kind: float
+  '2'_c; // call: operator "" _c('2')| kind: char
+
+  // PROBLEM: How to discover the kind of user-defined-literal from the AST?
+  12_r;  // call: operator "" _r("12")   | kind: integer
+  1.2_r; // call: operator "" _i("1.2")  | kind: float
+  12_t;  // call: operator<'1', '2'> "" _x() | kind: integer
+  1.2_t; // call: operator<'1', '2'> "" _x() | kind: float
 }
-)cpp",
+)cpp",
   R"txt(
 *: TranslationUnit
+|-SimpleDeclaration
+| |-unsigned
+| |-SimpleDeclarator
+| | |-operator
+| | |-""
+| | |-_i
+| | `-ParametersAndQualifiers
+| |   |-(
+| |   |-SimpleDeclaration
+| |   | |-unsigned
+| |   | |-long
+| |   | `-long
+| |   `-)
+| `-;
+|-SimpleDeclaration
+| |-unsigned
+| |-SimpleDeclarator
+| | |-operator
+| | |-""
+| | |-_f
+| | `-ParametersAndQualifiers
+| |   |-(
+| |   |-SimpleDeclaration
+| |   | |-long
+| |   | `-double
+| |   `-)
+| `-;
+|-SimpleDeclaration
+| |-unsigned
+| |-SimpleDeclarator
+| | |-operator
+| | |-""
+| | |-_c
+| | `-ParametersAndQualifiers
+| |   |-(
+| |   |-SimpleDeclaration
+| |   | `-char
+| |   `-)
+| `-;
+|-SimpleDeclaration
+| |-unsigned
+| |-SimpleDeclarator
+| | |-operator
+| | |-""
+| | |-_r
+| | `-ParametersAndQualifiers
+| |   |-(
+| |   |-SimpleDeclaration
+| |   | |-const
+| |   | |-char
+| |   | `-SimpleDeclarator
+| |   |   `-*
+| |   `-)
+| `-;
+|-TemplateDeclaration
+| |-template
+| |-<
+| |-SimpleDeclaration
+| | `-char
+| |-...
+| |->
+| `-SimpleDeclaration
+|   |-unsigned
+|   |-SimpleDeclarator
+|   | |-operator
+|   | |-""
+|   | |-_t
+|   | `-ParametersAndQualifiers
+|   |   |-(
+|   |   `-)
+|   `-;
 `-SimpleDeclaration
   |-void
   |-SimpleDeclarator
@@ -1208,28 +1294,93 @@
   `-CompoundStatement
 |-{
 |-ExpressionStatement
-| |-IntegerLiteralExpression
-| | `-12
+| |-IntegerUserDefinedLiteralExpression
+| | `-12_i
 | `-;
 |-ExpressionStatement
-| |-IntegerLiteralExpression
-| | `-12u
+| |-FloatUserDefinedLiteralExpression
+| | `-1.2_f
 | `-;
 |-ExpressionStatement
-| |-IntegerLiteralExpression
-| | `-12l
+| |-CharUserDefinedLiteralExpression
+| | `-'2'_c
 | `-;
 |-ExpressionStatement
-| |-IntegerLiteralExpression
-| | `-12ul
+| |-UnknownUserDefinedLiteralExpression
+| | `-12_r
 | `-;
 |-ExpressionStatement
-| |-IntegerLiteralExpression
-| | `-014
+| |-UnknownUserDefinedLiteralExpression
+| | `-1.2_r
 | `-;
 |-ExpressionStatement
-| |-IntegerLiteralExpression
-| | `-0XC
+| |-UnknownUserDefinedLiteralExpression
+| | `-12_t
+| `-;
+|-ExpressionStatement
+| |-UnknownUserDefinedLiteralExpression
+| | `-1.2_t
+| `-;
+`-}
+)txt"));
+}
+
+TEST_P(SyntaxTreeTest, UserDefinedLiteralString) {
+  if (!GetParam().isCXX11OrLater()) {
+return;
+  }
+  EXPECT_TRUE(treeDumpEqual(
+  R"cpp(
+typedef __SIZE_TYPE__ size_t;
+unsigned operator "" _s(const char*, size_t);
+void test() {
+  "12"_s;// call: operator "" _s("12")   | kind: string
+}
+)cpp",
+  std::string(R"txt(
+*: TranslationUnit
+|-SimpleDeclaration
+| |-typedef)txt") +
+  (!GetParam().isLinux() ? R"txt(
+| |-I: long)txt"
+ : "") +
+  R"txt(
+| |-I: long
+| |-I: unsigned
+| |-I: int
+| |-SimpleDeclarator
+| | `-size_t
+| `-;
+|-SimpleDeclaration
+| |-unsigned
+| |-SimpleDeclarator
+| | |-operator
+| | |-""
+| | |-_s
+| | `-ParametersAndQualifiers
+| |   |-(
+| |   |-SimpleDeclaration
+| |   | |-const
+| |   | |-char
+| |   | 

[PATCH] D82954: Fix crash on overloaded postfix unary operators due to invalid SourceLocation

[Eduardo Caldas via Phabricator via cfe-commits]

This revision was not accepted when it landed; it landed in state "Needs 
Review".
This revision was automatically updated to reflect the committed changes.
Closed by commit rGea8bba7e8d0d: Fix crash on overloaded postfix unary 
operators due to invalid sloc (authored by eduucaldas).

Changed prior to commit:
  https://reviews.llvm.org/D82954?vs=275318=275709#toc

Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D82954/new/

https://reviews.llvm.org/D82954

Files:
  clang/lib/Tooling/Syntax/BuildTree.cpp
  clang/unittests/Tooling/Syntax/TreeTest.cpp

Index: clang/unittests/Tooling/Syntax/TreeTest.cpp
===
--- clang/unittests/Tooling/Syntax/TreeTest.cpp
+++ clang/unittests/Tooling/Syntax/TreeTest.cpp
@@ -2193,11 +2193,18 @@
   X& operator=(const X&);
   friend X operator+(X, const X&);
   friend bool operator<(const X&, const X&);
+  friend X operator<<(X&, const X&);
+  X operator,(X&);
+  // TODO: Fix crash on member function pointer and add a test for `->*`
+  // TODO: Unbox operators in syntax tree. 
+  // Represent operators by `+` instead of `IdExpression-UnqualifiedId-+`
 };
 void test(X x, X y) {
   x = y;
   x + y;
   x < y;
+  x << y;
+  x, y;
 }
 )cpp",
   R"txt(
@@ -2262,6 +2269,40 @@
 | |   |   |   `-&
 | |   |   `-)
 | |   `-;
+| |-UnknownDeclaration
+| | `-SimpleDeclaration
+| |   |-friend
+| |   |-X
+| |   |-SimpleDeclarator
+| |   | |-operator
+| |   | |-<<
+| |   | `-ParametersAndQualifiers
+| |   |   |-(
+| |   |   |-SimpleDeclaration
+| |   |   | |-X
+| |   |   | `-SimpleDeclarator
+| |   |   |   `-&
+| |   |   |-,
+| |   |   |-SimpleDeclaration
+| |   |   | |-const
+| |   |   | |-X
+| |   |   | `-SimpleDeclarator
+| |   |   |   `-&
+| |   |   `-)
+| |   `-;
+| |-SimpleDeclaration
+| | |-X
+| | |-SimpleDeclarator
+| | | |-operator
+| | | |-,
+| | | `-ParametersAndQualifiers
+| | |   |-(
+| | |   |-SimpleDeclaration
+| | |   | |-X
+| | |   | `-SimpleDeclarator
+| | |   |   `-&
+| | |   `-)
+| | `-;
 | |-}
 | `-;
 `-SimpleDeclaration
@@ -2319,6 +2360,185 @@
 | |   `-UnqualifiedId
 | | `-y
 | `-;
+|-ExpressionStatement
+| |-BinaryOperatorExpression
+| | |-IdExpression
+| | | `-UnqualifiedId
+| | |   `-x
+| | |-IdExpression
+| | | `-UnqualifiedId
+| | |   `-<<
+| | `-IdExpression
+| |   `-UnqualifiedId
+| | `-y
+| `-;
+|-ExpressionStatement
+| |-BinaryOperatorExpression
+| | |-IdExpression
+| | | `-UnqualifiedId
+| | |   `-x
+| | |-IdExpression
+| | | `-UnqualifiedId
+| | |   `-,
+| | `-IdExpression
+| |   `-UnqualifiedId
+| | `-y
+| `-;
+`-}
+)txt"));
+}
+
+TEST_P(SyntaxTreeTest, UserDefinedUnaryPrefixOperator) {
+  if (!GetParam().isCXX()) {
+return;
+  }
+  EXPECT_TRUE(treeDumpEqual(
+  R"cpp(
+struct X {
+  X operator++();
+  bool operator!();
+  X* operator&();
+};
+void test(X x) {
+  ++x;
+  !x;
+  
+}
+)cpp",
+  R"txt(
+*: TranslationUnit
+|-SimpleDeclaration
+| |-struct
+| |-X
+| |-{
+| |-SimpleDeclaration
+| | |-X
+| | |-SimpleDeclarator
+| | | |-operator
+| | | |-++
+| | | `-ParametersAndQualifiers
+| | |   |-(
+| | |   `-)
+| | `-;
+| |-SimpleDeclaration
+| | |-bool
+| | |-SimpleDeclarator
+| | | |-operator
+| | | |-!
+| | | `-ParametersAndQualifiers
+| | |   |-(
+| | |   `-)
+| | `-;
+| |-SimpleDeclaration
+| | |-X
+| | |-SimpleDeclarator
+| | | |-*
+| | | |-operator
+| | | |-&
+| | | `-ParametersAndQualifiers
+| | |   |-(
+| | |   `-)
+| | `-;
+| |-}
+| `-;
+`-SimpleDeclaration
+  |-void
+  |-SimpleDeclarator
+  | |-test
+  | `-ParametersAndQualifiers
+  |   |-(
+  |   |-SimpleDeclaration
+  |   | |-X
+  |   | `-SimpleDeclarator
+  |   |   `-x
+  |   `-)
+  `-CompoundStatement
+|-{
+|-ExpressionStatement
+| |-PrefixUnaryOperatorExpression
+| | |-IdExpression
+| | | `-UnqualifiedId
+| | |   `-++
+| | `-IdExpression
+| |   `-UnqualifiedId
+| | `-x
+| `-;
+|-ExpressionStatement
+| |-PrefixUnaryOperatorExpression
+| | |-IdExpression
+| | | `-UnqualifiedId
+| | |   `-!
+| | `-IdExpression
+| |   `-UnqualifiedId
+| | `-x
+| `-;
+|-ExpressionStatement
+| |-PrefixUnaryOperatorExpression
+| | |-IdExpression
+| | | `-UnqualifiedId
+| | |   `-&
+| | `-IdExpression
+| |   `-UnqualifiedId
+| | `-x
+| `-;
+`-}
+)txt"));
+}
+
+TEST_P(SyntaxTreeTest, UserDefinedUnaryPostfixOperator) {
+  if (!GetParam().isCXX()) {
+return;
+  }
+  EXPECT_TRUE(treeDumpEqual(
+  R"cpp(
+struct X {
+  X operator++(int);
+};
+void test(X x) {
+  x++;
+}
+)cpp",
+  R"txt(
+*: TranslationUnit
+|-SimpleDeclaration
+| |-struct
+| |-X
+| |-{
+| |-SimpleDeclaration
+| | |-X
+| | |-SimpleDeclarator
+| | | |-operator
+| | | |-++
+| | | `-ParametersAndQualifiers
+| | |   |-(
+| | |   |-SimpleDeclaration
+| | |   | `-int
+| | |   `-)
+| | `-;

[PATCH] D82157: Fix crash on `user defined literals`

[Eduardo Caldas via Phabricator via cfe-commits]

eduucaldas marked 2 inline comments as done.
eduucaldas added inline comments.



Comment at: clang/unittests/Tooling/Syntax/TreeTest.cpp:1265
+| |-UserDefinedLiteralExpression
+| | `-12_w
+| `-;

gribozavr2 wrote:
> It looks somewhat weird to me that integer and floating point literals end up 
> with the same syntax tree node type. WDYT about making different nodes for 
> different literals (integer, floating-point, string, character)?
Makes sense. Let's follow the [[ 
https://eel.is/c++draft/lex.ext#nt:user-defined-literal | grammar ]] then.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D82157/new/

https://reviews.llvm.org/D82157



___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D82954: Fix crash on overloaded postfix unary operators due to invalid SourceLocation

[Eduardo Caldas via Phabricator via cfe-commits]

This revision was automatically updated to reflect the committed changes.
Closed by commit rGea8bba7e8d0d: Fix crash on overloaded postfix unary 
operators due to invalid sloc (authored by eduucaldas).

Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D82954/new/

https://reviews.llvm.org/D82954

Files:
  clang/lib/Tooling/Syntax/BuildTree.cpp
  clang/unittests/Tooling/Syntax/TreeTest.cpp

Index: clang/unittests/Tooling/Syntax/TreeTest.cpp
===
--- clang/unittests/Tooling/Syntax/TreeTest.cpp
+++ clang/unittests/Tooling/Syntax/TreeTest.cpp
@@ -2193,11 +2193,18 @@
   X& operator=(const X&);
   friend X operator+(X, const X&);
   friend bool operator<(const X&, const X&);
+  friend X operator<<(X&, const X&);
+  X operator,(X&);
+  // TODO: Fix crash on member function pointer and add a test for `->*`
+  // TODO: Unbox operators in syntax tree. 
+  // Represent operators by `+` instead of `IdExpression-UnqualifiedId-+`
 };
 void test(X x, X y) {
   x = y;
   x + y;
   x < y;
+  x << y;
+  x, y;
 }
 )cpp",
   R"txt(
@@ -2262,6 +2269,40 @@
 | |   |   |   `-&
 | |   |   `-)
 | |   `-;
+| |-UnknownDeclaration
+| | `-SimpleDeclaration
+| |   |-friend
+| |   |-X
+| |   |-SimpleDeclarator
+| |   | |-operator
+| |   | |-<<
+| |   | `-ParametersAndQualifiers
+| |   |   |-(
+| |   |   |-SimpleDeclaration
+| |   |   | |-X
+| |   |   | `-SimpleDeclarator
+| |   |   |   `-&
+| |   |   |-,
+| |   |   |-SimpleDeclaration
+| |   |   | |-const
+| |   |   | |-X
+| |   |   | `-SimpleDeclarator
+| |   |   |   `-&
+| |   |   `-)
+| |   `-;
+| |-SimpleDeclaration
+| | |-X
+| | |-SimpleDeclarator
+| | | |-operator
+| | | |-,
+| | | `-ParametersAndQualifiers
+| | |   |-(
+| | |   |-SimpleDeclaration
+| | |   | |-X
+| | |   | `-SimpleDeclarator
+| | |   |   `-&
+| | |   `-)
+| | `-;
 | |-}
 | `-;
 `-SimpleDeclaration
@@ -2319,6 +2360,185 @@
 | |   `-UnqualifiedId
 | | `-y
 | `-;
+|-ExpressionStatement
+| |-BinaryOperatorExpression
+| | |-IdExpression
+| | | `-UnqualifiedId
+| | |   `-x
+| | |-IdExpression
+| | | `-UnqualifiedId
+| | |   `-<<
+| | `-IdExpression
+| |   `-UnqualifiedId
+| | `-y
+| `-;
+|-ExpressionStatement
+| |-BinaryOperatorExpression
+| | |-IdExpression
+| | | `-UnqualifiedId
+| | |   `-x
+| | |-IdExpression
+| | | `-UnqualifiedId
+| | |   `-,
+| | `-IdExpression
+| |   `-UnqualifiedId
+| | `-y
+| `-;
+`-}
+)txt"));
+}
+
+TEST_P(SyntaxTreeTest, UserDefinedUnaryPrefixOperator) {
+  if (!GetParam().isCXX()) {
+return;
+  }
+  EXPECT_TRUE(treeDumpEqual(
+  R"cpp(
+struct X {
+  X operator++();
+  bool operator!();
+  X* operator&();
+};
+void test(X x) {
+  ++x;
+  !x;
+  
+}
+)cpp",
+  R"txt(
+*: TranslationUnit
+|-SimpleDeclaration
+| |-struct
+| |-X
+| |-{
+| |-SimpleDeclaration
+| | |-X
+| | |-SimpleDeclarator
+| | | |-operator
+| | | |-++
+| | | `-ParametersAndQualifiers
+| | |   |-(
+| | |   `-)
+| | `-;
+| |-SimpleDeclaration
+| | |-bool
+| | |-SimpleDeclarator
+| | | |-operator
+| | | |-!
+| | | `-ParametersAndQualifiers
+| | |   |-(
+| | |   `-)
+| | `-;
+| |-SimpleDeclaration
+| | |-X
+| | |-SimpleDeclarator
+| | | |-*
+| | | |-operator
+| | | |-&
+| | | `-ParametersAndQualifiers
+| | |   |-(
+| | |   `-)
+| | `-;
+| |-}
+| `-;
+`-SimpleDeclaration
+  |-void
+  |-SimpleDeclarator
+  | |-test
+  | `-ParametersAndQualifiers
+  |   |-(
+  |   |-SimpleDeclaration
+  |   | |-X
+  |   | `-SimpleDeclarator
+  |   |   `-x
+  |   `-)
+  `-CompoundStatement
+|-{
+|-ExpressionStatement
+| |-PrefixUnaryOperatorExpression
+| | |-IdExpression
+| | | `-UnqualifiedId
+| | |   `-++
+| | `-IdExpression
+| |   `-UnqualifiedId
+| | `-x
+| `-;
+|-ExpressionStatement
+| |-PrefixUnaryOperatorExpression
+| | |-IdExpression
+| | | `-UnqualifiedId
+| | |   `-!
+| | `-IdExpression
+| |   `-UnqualifiedId
+| | `-x
+| `-;
+|-ExpressionStatement
+| |-PrefixUnaryOperatorExpression
+| | |-IdExpression
+| | | `-UnqualifiedId
+| | |   `-&
+| | `-IdExpression
+| |   `-UnqualifiedId
+| | `-x
+| `-;
+`-}
+)txt"));
+}
+
+TEST_P(SyntaxTreeTest, UserDefinedUnaryPostfixOperator) {
+  if (!GetParam().isCXX()) {
+return;
+  }
+  EXPECT_TRUE(treeDumpEqual(
+  R"cpp(
+struct X {
+  X operator++(int);
+};
+void test(X x) {
+  x++;
+}
+)cpp",
+  R"txt(
+*: TranslationUnit
+|-SimpleDeclaration
+| |-struct
+| |-X
+| |-{
+| |-SimpleDeclaration
+| | |-X
+| | |-SimpleDeclarator
+| | | |-operator
+| | | |-++
+| | | `-ParametersAndQualifiers
+| | |   |-(
+| | |   |-SimpleDeclaration
+| | |   | `-int
+| | |   `-)
+| | `-;
+| |-}
+| `-;
+`-SimpleDeclaration
+  |-void
+  |-SimpleDeclarator
+  | |-test
+  | `-ParametersAndQualifiers
+  |   |-(
+  |   |-SimpleDeclaration
+  |   | |-X
+  | 

[PATCH] D82954: Fix crash on overloaded postfix unary operators due to invalid SourceLocation

[Eduardo Caldas via Phabricator via cfe-commits]

eduucaldas updated this revision to Diff 276411.
eduucaldas marked 2 inline comments as done.
eduucaldas added a comment.

minor fix comments


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D82954/new/

https://reviews.llvm.org/D82954

Files:
  clang/lib/Tooling/Syntax/BuildTree.cpp
  clang/unittests/Tooling/Syntax/TreeTest.cpp

Index: clang/unittests/Tooling/Syntax/TreeTest.cpp
===
--- clang/unittests/Tooling/Syntax/TreeTest.cpp
+++ clang/unittests/Tooling/Syntax/TreeTest.cpp
@@ -2193,11 +2193,18 @@
   X& operator=(const X&);
   friend X operator+(X, const X&);
   friend bool operator<(const X&, const X&);
+  friend X operator<<(X&, const X&);
+  X operator,(X&);
+  // TODO: Fix crash on member function pointer and add a test for `->*`
+  // TODO: Unbox operators in syntax tree. 
+  // Represent operators by `+` instead of `IdExpression-UnqualifiedId-+`
 };
 void test(X x, X y) {
   x = y;
   x + y;
   x < y;
+  x << y;
+  x, y;
 }
 )cpp",
   R"txt(
@@ -2262,6 +2269,40 @@
 | |   |   |   `-&
 | |   |   `-)
 | |   `-;
+| |-UnknownDeclaration
+| | `-SimpleDeclaration
+| |   |-friend
+| |   |-X
+| |   |-SimpleDeclarator
+| |   | |-operator
+| |   | |-<<
+| |   | `-ParametersAndQualifiers
+| |   |   |-(
+| |   |   |-SimpleDeclaration
+| |   |   | |-X
+| |   |   | `-SimpleDeclarator
+| |   |   |   `-&
+| |   |   |-,
+| |   |   |-SimpleDeclaration
+| |   |   | |-const
+| |   |   | |-X
+| |   |   | `-SimpleDeclarator
+| |   |   |   `-&
+| |   |   `-)
+| |   `-;
+| |-SimpleDeclaration
+| | |-X
+| | |-SimpleDeclarator
+| | | |-operator
+| | | |-,
+| | | `-ParametersAndQualifiers
+| | |   |-(
+| | |   |-SimpleDeclaration
+| | |   | |-X
+| | |   | `-SimpleDeclarator
+| | |   |   `-&
+| | |   `-)
+| | `-;
 | |-}
 | `-;
 `-SimpleDeclaration
@@ -2319,6 +2360,185 @@
 | |   `-UnqualifiedId
 | | `-y
 | `-;
+|-ExpressionStatement
+| |-BinaryOperatorExpression
+| | |-IdExpression
+| | | `-UnqualifiedId
+| | |   `-x
+| | |-IdExpression
+| | | `-UnqualifiedId
+| | |   `-<<
+| | `-IdExpression
+| |   `-UnqualifiedId
+| | `-y
+| `-;
+|-ExpressionStatement
+| |-BinaryOperatorExpression
+| | |-IdExpression
+| | | `-UnqualifiedId
+| | |   `-x
+| | |-IdExpression
+| | | `-UnqualifiedId
+| | |   `-,
+| | `-IdExpression
+| |   `-UnqualifiedId
+| | `-y
+| `-;
+`-}
+)txt"));
+}
+
+TEST_P(SyntaxTreeTest, UserDefinedUnaryPrefixOperator) {
+  if (!GetParam().isCXX()) {
+return;
+  }
+  EXPECT_TRUE(treeDumpEqual(
+  R"cpp(
+struct X {
+  X operator++();
+  bool operator!();
+  X* operator&();
+};
+void test(X x) {
+  ++x;
+  !x;
+  
+}
+)cpp",
+  R"txt(
+*: TranslationUnit
+|-SimpleDeclaration
+| |-struct
+| |-X
+| |-{
+| |-SimpleDeclaration
+| | |-X
+| | |-SimpleDeclarator
+| | | |-operator
+| | | |-++
+| | | `-ParametersAndQualifiers
+| | |   |-(
+| | |   `-)
+| | `-;
+| |-SimpleDeclaration
+| | |-bool
+| | |-SimpleDeclarator
+| | | |-operator
+| | | |-!
+| | | `-ParametersAndQualifiers
+| | |   |-(
+| | |   `-)
+| | `-;
+| |-SimpleDeclaration
+| | |-X
+| | |-SimpleDeclarator
+| | | |-*
+| | | |-operator
+| | | |-&
+| | | `-ParametersAndQualifiers
+| | |   |-(
+| | |   `-)
+| | `-;
+| |-}
+| `-;
+`-SimpleDeclaration
+  |-void
+  |-SimpleDeclarator
+  | |-test
+  | `-ParametersAndQualifiers
+  |   |-(
+  |   |-SimpleDeclaration
+  |   | |-X
+  |   | `-SimpleDeclarator
+  |   |   `-x
+  |   `-)
+  `-CompoundStatement
+|-{
+|-ExpressionStatement
+| |-PrefixUnaryOperatorExpression
+| | |-IdExpression
+| | | `-UnqualifiedId
+| | |   `-++
+| | `-IdExpression
+| |   `-UnqualifiedId
+| | `-x
+| `-;
+|-ExpressionStatement
+| |-PrefixUnaryOperatorExpression
+| | |-IdExpression
+| | | `-UnqualifiedId
+| | |   `-!
+| | `-IdExpression
+| |   `-UnqualifiedId
+| | `-x
+| `-;
+|-ExpressionStatement
+| |-PrefixUnaryOperatorExpression
+| | |-IdExpression
+| | | `-UnqualifiedId
+| | |   `-&
+| | `-IdExpression
+| |   `-UnqualifiedId
+| | `-x
+| `-;
+`-}
+)txt"));
+}
+
+TEST_P(SyntaxTreeTest, UserDefinedUnaryPostfixOperator) {
+  if (!GetParam().isCXX()) {
+return;
+  }
+  EXPECT_TRUE(treeDumpEqual(
+  R"cpp(
+struct X {
+  X operator++(int);
+};
+void test(X x) {
+  x++;
+}
+)cpp",
+  R"txt(
+*: TranslationUnit
+|-SimpleDeclaration
+| |-struct
+| |-X
+| |-{
+| |-SimpleDeclaration
+| | |-X
+| | |-SimpleDeclarator
+| | | |-operator
+| | | |-++
+| | | `-ParametersAndQualifiers
+| | |   |-(
+| | |   |-SimpleDeclaration
+| | |   | `-int
+| | |   `-)
+| | `-;
+| |-}
+| `-;
+`-SimpleDeclaration
+  |-void
+  |-SimpleDeclarator
+  | |-test
+  | `-ParametersAndQualifiers
+  |   |-(
+  |   |-SimpleDeclaration
+  |   | |-X
+  |   | `-SimpleDeclarator
+  |   |   `-x
+  |   `-)
+  

[PATCH] D82954: Fix crash on overloaded postfix unary operators due to invalid SourceLocation

[Eduardo Caldas via Phabricator via cfe-commits]

eduucaldas updated this revision to Diff 276408.
eduucaldas marked an inline comment as done.
eduucaldas added a comment.

Switch to `TraverseCXXOperatorCallExpr`


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D82954/new/

https://reviews.llvm.org/D82954

Files:
  clang/lib/Tooling/Syntax/BuildTree.cpp
  clang/unittests/Tooling/Syntax/TreeTest.cpp

Index: clang/unittests/Tooling/Syntax/TreeTest.cpp
===
--- clang/unittests/Tooling/Syntax/TreeTest.cpp
+++ clang/unittests/Tooling/Syntax/TreeTest.cpp
@@ -2193,11 +2193,18 @@
   X& operator=(const X&);
   friend X operator+(X, const X&);
   friend bool operator<(const X&, const X&);
+  friend X operator<<(X&, const X&);
+  X operator,(X&);
+  // TODO: Fix crash on member function pointer and add a test for `->*`
+  // TODO: Unbox operators in syntax tree. 
+  // Represent operators by `+` instead of `IdExpression-UnqualifiedId-+`
 };
 void test(X x, X y) {
   x = y;
   x + y;
   x < y;
+  x << y;
+  x, y;
 }
 )cpp",
   R"txt(
@@ -2262,6 +2269,40 @@
 | |   |   |   `-&
 | |   |   `-)
 | |   `-;
+| |-UnknownDeclaration
+| | `-SimpleDeclaration
+| |   |-friend
+| |   |-X
+| |   |-SimpleDeclarator
+| |   | |-operator
+| |   | |-<<
+| |   | `-ParametersAndQualifiers
+| |   |   |-(
+| |   |   |-SimpleDeclaration
+| |   |   | |-X
+| |   |   | `-SimpleDeclarator
+| |   |   |   `-&
+| |   |   |-,
+| |   |   |-SimpleDeclaration
+| |   |   | |-const
+| |   |   | |-X
+| |   |   | `-SimpleDeclarator
+| |   |   |   `-&
+| |   |   `-)
+| |   `-;
+| |-SimpleDeclaration
+| | |-X
+| | |-SimpleDeclarator
+| | | |-operator
+| | | |-,
+| | | `-ParametersAndQualifiers
+| | |   |-(
+| | |   |-SimpleDeclaration
+| | |   | |-X
+| | |   | `-SimpleDeclarator
+| | |   |   `-&
+| | |   `-)
+| | `-;
 | |-}
 | `-;
 `-SimpleDeclaration
@@ -2319,6 +2360,185 @@
 | |   `-UnqualifiedId
 | | `-y
 | `-;
+|-ExpressionStatement
+| |-BinaryOperatorExpression
+| | |-IdExpression
+| | | `-UnqualifiedId
+| | |   `-x
+| | |-IdExpression
+| | | `-UnqualifiedId
+| | |   `-<<
+| | `-IdExpression
+| |   `-UnqualifiedId
+| | `-y
+| `-;
+|-ExpressionStatement
+| |-BinaryOperatorExpression
+| | |-IdExpression
+| | | `-UnqualifiedId
+| | |   `-x
+| | |-IdExpression
+| | | `-UnqualifiedId
+| | |   `-,
+| | `-IdExpression
+| |   `-UnqualifiedId
+| | `-y
+| `-;
+`-}
+)txt"));
+}
+
+TEST_P(SyntaxTreeTest, UserDefinedUnaryPrefixOperator) {
+  if (!GetParam().isCXX()) {
+return;
+  }
+  EXPECT_TRUE(treeDumpEqual(
+  R"cpp(
+struct X {
+  X operator++();
+  bool operator!();
+  X* operator&();
+};
+void test(X x) {
+  ++x;
+  !x;
+  
+}
+)cpp",
+  R"txt(
+*: TranslationUnit
+|-SimpleDeclaration
+| |-struct
+| |-X
+| |-{
+| |-SimpleDeclaration
+| | |-X
+| | |-SimpleDeclarator
+| | | |-operator
+| | | |-++
+| | | `-ParametersAndQualifiers
+| | |   |-(
+| | |   `-)
+| | `-;
+| |-SimpleDeclaration
+| | |-bool
+| | |-SimpleDeclarator
+| | | |-operator
+| | | |-!
+| | | `-ParametersAndQualifiers
+| | |   |-(
+| | |   `-)
+| | `-;
+| |-SimpleDeclaration
+| | |-X
+| | |-SimpleDeclarator
+| | | |-*
+| | | |-operator
+| | | |-&
+| | | `-ParametersAndQualifiers
+| | |   |-(
+| | |   `-)
+| | `-;
+| |-}
+| `-;
+`-SimpleDeclaration
+  |-void
+  |-SimpleDeclarator
+  | |-test
+  | `-ParametersAndQualifiers
+  |   |-(
+  |   |-SimpleDeclaration
+  |   | |-X
+  |   | `-SimpleDeclarator
+  |   |   `-x
+  |   `-)
+  `-CompoundStatement
+|-{
+|-ExpressionStatement
+| |-PrefixUnaryOperatorExpression
+| | |-IdExpression
+| | | `-UnqualifiedId
+| | |   `-++
+| | `-IdExpression
+| |   `-UnqualifiedId
+| | `-x
+| `-;
+|-ExpressionStatement
+| |-PrefixUnaryOperatorExpression
+| | |-IdExpression
+| | | `-UnqualifiedId
+| | |   `-!
+| | `-IdExpression
+| |   `-UnqualifiedId
+| | `-x
+| `-;
+|-ExpressionStatement
+| |-PrefixUnaryOperatorExpression
+| | |-IdExpression
+| | | `-UnqualifiedId
+| | |   `-&
+| | `-IdExpression
+| |   `-UnqualifiedId
+| | `-x
+| `-;
+`-}
+)txt"));
+}
+
+TEST_P(SyntaxTreeTest, UserDefinedUnaryPostfixOperator) {
+  if (!GetParam().isCXX()) {
+return;
+  }
+  EXPECT_TRUE(treeDumpEqual(
+  R"cpp(
+struct X {
+  X operator++(int);
+};
+void test(X x) {
+  x++;
+}
+)cpp",
+  R"txt(
+*: TranslationUnit
+|-SimpleDeclaration
+| |-struct
+| |-X
+| |-{
+| |-SimpleDeclaration
+| | |-X
+| | |-SimpleDeclarator
+| | | |-operator
+| | | |-++
+| | | `-ParametersAndQualifiers
+| | |   |-(
+| | |   |-SimpleDeclaration
+| | |   | `-int
+| | |   `-)
+| | `-;
+| |-}
+| `-;
+`-SimpleDeclaration
+  |-void
+  |-SimpleDeclarator
+  | |-test
+  | `-ParametersAndQualifiers
+  |   |-(
+  |   |-SimpleDeclaration
+  |   | |-X
+  |   | `-SimpleDeclarator
+  |   |   `-x

[PATCH] D82954: Fix crash on overloaded postfix unary operators due to invalid SourceLocation

[Eduardo Caldas via Phabricator via cfe-commits]

eduucaldas marked 2 inline comments as done.
eduucaldas added inline comments.



Comment at: clang/unittests/Tooling/Syntax/TreeTest.cpp:2192
   R"cpp(
+class osstream {};
 struct X {

gribozavr2 wrote:
> eduucaldas wrote:
> > gribozavr2 wrote:
> > > I don't think we need a separate class to show the left shift operator. 
> > > The declaration below can be:
> > > 
> > > ```
> > >   friend X operator<<(X&, const X&);
> > > ```
> > If we don't bother much about "realistic" operator declarations we could 
> > drop all the `friend` and declare every operator in their most concise 
> > form. WDYT
> I think we shouldn't try to make tests realistic in terms of function names 
> etc., but we should try to cover as many different AST shapes as possible. In 
> the case of binary operators, we have three cases -- free function, friend 
> function, member function, that all generate slightly different ASTs, so I 
> believe we should try to cover them all.
But those all regard the operators declaration. 
Here we test the operator call expression - `CXXOperatorCallExpr`.

I think we should test friend function declarations when we add support for 
them in the tree, and then we add tests for declaration of friend operators, 
friend member functions and whatnot.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D82954/new/

https://reviews.llvm.org/D82954



___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D82157: Fix crash on `user defined literals`

[Eduardo Caldas via Phabricator via cfe-commits]

eduucaldas updated this revision to Diff 276361.
eduucaldas added a comment.

Reflect fix on RecursiveASTVisitor


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D82157/new/

https://reviews.llvm.org/D82157

Files:
  clang/include/clang/Tooling/Syntax/Nodes.h
  clang/lib/Tooling/Syntax/BuildTree.cpp
  clang/lib/Tooling/Syntax/Nodes.cpp
  clang/unittests/Tooling/Syntax/TreeTest.cpp

Index: clang/unittests/Tooling/Syntax/TreeTest.cpp
===
--- clang/unittests/Tooling/Syntax/TreeTest.cpp
+++ clang/unittests/Tooling/Syntax/TreeTest.cpp
@@ -1184,6 +1184,93 @@
 )txt"));
 }
 
+TEST_P(SyntaxTreeTest, UserDefinedLiteral) {
+  if (!GetParam().isCXX11OrLater()) {
+return;
+  }
+  EXPECT_TRUE(treeDumpEqual(
+  R"cpp(
+long double operator "" _w(long double);
+unsigned operator "" _w(const char*);
+template  unsigned operator "" _x();
+int main() {
+1.2_w; // calls operator "" _w(1.2L)
+12_w;  // calls operator "" _w("12")
+12_x;  // calls operator<'1', '2'> "" _x()
+}
+)cpp",
+  R"txt(
+*: TranslationUnit
+|-SimpleDeclaration
+| |-long
+| |-double
+| |-SimpleDeclarator
+| | |-operator
+| | |-""
+| | |-_w
+| | `-ParametersAndQualifiers
+| |   |-(
+| |   |-SimpleDeclaration
+| |   | |-long
+| |   | `-double
+| |   `-)
+| `-;
+|-SimpleDeclaration
+| |-unsigned
+| |-SimpleDeclarator
+| | |-operator
+| | |-""
+| | |-_w
+| | `-ParametersAndQualifiers
+| |   |-(
+| |   |-SimpleDeclaration
+| |   | |-const
+| |   | |-char
+| |   | `-SimpleDeclarator
+| |   |   `-*
+| |   `-)
+| `-;
+|-TemplateDeclaration
+| |-template
+| |-<
+| |-SimpleDeclaration
+| | `-char
+| |-...
+| |->
+| `-SimpleDeclaration
+|   |-unsigned
+|   |-SimpleDeclarator
+|   | |-operator
+|   | |-""
+|   | |-_x
+|   | `-ParametersAndQualifiers
+|   |   |-(
+|   |   `-)
+|   `-;
+`-SimpleDeclaration
+  |-int
+  |-SimpleDeclarator
+  | |-main
+  | `-ParametersAndQualifiers
+  |   |-(
+  |   `-)
+  `-CompoundStatement
+|-{
+|-ExpressionStatement
+| |-UserDefinedLiteralExpression
+| | `-1.2_w
+| `-;
+|-ExpressionStatement
+| |-UserDefinedLiteralExpression
+| | `-12_w
+| `-;
+|-ExpressionStatement
+| |-UserDefinedLiteralExpression
+| | `-12_x
+| `-;
+`-}
+)txt"));
+}
 TEST_P(SyntaxTreeTest, IntegerLiteral) {
   EXPECT_TRUE(treeDumpEqual(
   R"cpp(
Index: clang/lib/Tooling/Syntax/Nodes.cpp
===
--- clang/lib/Tooling/Syntax/Nodes.cpp
+++ clang/lib/Tooling/Syntax/Nodes.cpp
@@ -32,6 +32,8 @@
 return OS << "BoolLiteralExpression";
   case NodeKind::CxxNullPtrExpression:
 return OS << "CxxNullPtrExpression";
+  case NodeKind::UserDefinedLiteralExpression:
+return OS << "UserDefinedLiteralExpression";
   case NodeKind::PrefixUnaryOperatorExpression:
 return OS << "PrefixUnaryOperatorExpression";
   case NodeKind::PostfixUnaryOperatorExpression:
@@ -252,6 +254,11 @@
   findChild(syntax::NodeRole::LiteralToken));
 }
 
+syntax::Leaf *syntax::UserDefinedLiteralExpression::literalToken() {
+  return llvm::cast_or_null(
+  findChild(syntax::NodeRole::LiteralToken));
+}
+
 syntax::Expression *syntax::BinaryOperatorExpression::lhs() {
   return llvm::cast_or_null(
   findChild(syntax::NodeRole::BinaryOperatorExpression_leftHandSide));
Index: clang/lib/Tooling/Syntax/BuildTree.cpp
===
--- clang/lib/Tooling/Syntax/BuildTree.cpp
+++ clang/lib/Tooling/Syntax/BuildTree.cpp
@@ -627,6 +627,26 @@
 return NNS;
   }
 
+  bool TraverseUserDefinedLiteral(UserDefinedLiteral *S) {
+// The user-defined literal `1.2_w` corresponds to *one* token. The semantic
+// node for it however may have two children nodes, both with valid
+// `SourceLocation`s. As a result one of these nodes has a valid
+// `SourceLocation` that doesn't point to a token.
+//
+// If we traverse the children of a user-defined literal, we then arrive to
+// a semantic node that doesn't have a token, and that breaks an invariant
+// of the syntax tree. For that reason we skip traversing user-defined
+// literal children.
+
+return WalkUpFromUserDefinedLiteral(S);
+  }
+
+  bool WalkUpFromUserDefinedLiteral(UserDefinedLiteral *S) {
+Builder.markChildToken(S->getBeginLoc(), syntax::NodeRole::LiteralToken);
+Builder.foldNode(Builder.getExprRange(S),
+ new (allocator()) syntax::UserDefinedLiteralExpression, S);
+return true;
+  }
   bool WalkUpFromDeclRefExpr(DeclRefExpr *S) {
 if (auto *NNS = BuildNestedNameSpecifier(S->getQualifierLoc()))
   Builder.markChild(NNS, syntax::NodeRole::IdExpression_qualifier);
Index: clang/include/clang/Tooling/Syntax/Nodes.h
===
--- clang/include/clang/Tooling/Syntax/Nodes.h
+++ 

[PATCH] D82954: Fix crash on overloaded postfix unary operators due to invalid SourceLocation

[Eduardo Caldas via Phabricator via cfe-commits]

eduucaldas updated this revision to Diff 276344.
eduucaldas marked 4 inline comments as done.
eduucaldas added a comment.

answering minor comments


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D82954/new/

https://reviews.llvm.org/D82954

Files:
  clang/lib/Tooling/Syntax/BuildTree.cpp
  clang/unittests/Tooling/Syntax/TreeTest.cpp

Index: clang/unittests/Tooling/Syntax/TreeTest.cpp
===
--- clang/unittests/Tooling/Syntax/TreeTest.cpp
+++ clang/unittests/Tooling/Syntax/TreeTest.cpp
@@ -2193,11 +2193,18 @@
   X& operator=(const X&);
   friend X operator+(X, const X&);
   friend bool operator<(const X&, const X&);
+  friend X operator<<(X&, const X&);
+  X operator,(X&);
+  // TODO: Fix crash on member function pointer and add a test for `->*`
+  // TODO: Unbox operators in syntax tree. 
+  // Represent operators by `+` instead of `IdExpression-UnqualifiedId-+`
 };
 void test(X x, X y) {
   x = y;
   x + y;
   x < y;
+  x << y;
+  x, y;
 }
 )cpp",
   R"txt(
@@ -2262,6 +2269,40 @@
 | |   |   |   `-&
 | |   |   `-)
 | |   `-;
+| |-UnknownDeclaration
+| | `-SimpleDeclaration
+| |   |-friend
+| |   |-X
+| |   |-SimpleDeclarator
+| |   | |-operator
+| |   | |-<<
+| |   | `-ParametersAndQualifiers
+| |   |   |-(
+| |   |   |-SimpleDeclaration
+| |   |   | |-X
+| |   |   | `-SimpleDeclarator
+| |   |   |   `-&
+| |   |   |-,
+| |   |   |-SimpleDeclaration
+| |   |   | |-const
+| |   |   | |-X
+| |   |   | `-SimpleDeclarator
+| |   |   |   `-&
+| |   |   `-)
+| |   `-;
+| |-SimpleDeclaration
+| | |-X
+| | |-SimpleDeclarator
+| | | |-operator
+| | | |-,
+| | | `-ParametersAndQualifiers
+| | |   |-(
+| | |   |-SimpleDeclaration
+| | |   | |-X
+| | |   | `-SimpleDeclarator
+| | |   |   `-&
+| | |   `-)
+| | `-;
 | |-}
 | `-;
 `-SimpleDeclaration
@@ -2319,6 +2360,185 @@
 | |   `-UnqualifiedId
 | | `-y
 | `-;
+|-ExpressionStatement
+| |-BinaryOperatorExpression
+| | |-IdExpression
+| | | `-UnqualifiedId
+| | |   `-x
+| | |-IdExpression
+| | | `-UnqualifiedId
+| | |   `-<<
+| | `-IdExpression
+| |   `-UnqualifiedId
+| | `-y
+| `-;
+|-ExpressionStatement
+| |-BinaryOperatorExpression
+| | |-IdExpression
+| | | `-UnqualifiedId
+| | |   `-x
+| | |-IdExpression
+| | | `-UnqualifiedId
+| | |   `-,
+| | `-IdExpression
+| |   `-UnqualifiedId
+| | `-y
+| `-;
+`-}
+)txt"));
+}
+
+TEST_P(SyntaxTreeTest, UserDefinedUnaryPrefixOperator) {
+  if (!GetParam().isCXX()) {
+return;
+  }
+  EXPECT_TRUE(treeDumpEqual(
+  R"cpp(
+struct X {
+  X operator++();
+  bool operator!();
+  X* operator&();
+};
+void test(X x) {
+  ++x;
+  !x;
+  
+}
+)cpp",
+  R"txt(
+*: TranslationUnit
+|-SimpleDeclaration
+| |-struct
+| |-X
+| |-{
+| |-SimpleDeclaration
+| | |-X
+| | |-SimpleDeclarator
+| | | |-operator
+| | | |-++
+| | | `-ParametersAndQualifiers
+| | |   |-(
+| | |   `-)
+| | `-;
+| |-SimpleDeclaration
+| | |-bool
+| | |-SimpleDeclarator
+| | | |-operator
+| | | |-!
+| | | `-ParametersAndQualifiers
+| | |   |-(
+| | |   `-)
+| | `-;
+| |-SimpleDeclaration
+| | |-X
+| | |-SimpleDeclarator
+| | | |-*
+| | | |-operator
+| | | |-&
+| | | `-ParametersAndQualifiers
+| | |   |-(
+| | |   `-)
+| | `-;
+| |-}
+| `-;
+`-SimpleDeclaration
+  |-void
+  |-SimpleDeclarator
+  | |-test
+  | `-ParametersAndQualifiers
+  |   |-(
+  |   |-SimpleDeclaration
+  |   | |-X
+  |   | `-SimpleDeclarator
+  |   |   `-x
+  |   `-)
+  `-CompoundStatement
+|-{
+|-ExpressionStatement
+| |-PrefixUnaryOperatorExpression
+| | |-IdExpression
+| | | `-UnqualifiedId
+| | |   `-++
+| | `-IdExpression
+| |   `-UnqualifiedId
+| | `-x
+| `-;
+|-ExpressionStatement
+| |-PrefixUnaryOperatorExpression
+| | |-IdExpression
+| | | `-UnqualifiedId
+| | |   `-!
+| | `-IdExpression
+| |   `-UnqualifiedId
+| | `-x
+| `-;
+|-ExpressionStatement
+| |-PrefixUnaryOperatorExpression
+| | |-IdExpression
+| | | `-UnqualifiedId
+| | |   `-&
+| | `-IdExpression
+| |   `-UnqualifiedId
+| | `-x
+| `-;
+`-}
+)txt"));
+}
+
+TEST_P(SyntaxTreeTest, UserDefinedUnaryPostfixOperator) {
+  if (!GetParam().isCXX()) {
+return;
+  }
+  EXPECT_TRUE(treeDumpEqual(
+  R"cpp(
+struct X {
+  X operator++(int);
+};
+void test(X x) {
+  x++;
+}
+)cpp",
+  R"txt(
+*: TranslationUnit
+|-SimpleDeclaration
+| |-struct
+| |-X
+| |-{
+| |-SimpleDeclaration
+| | |-X
+| | |-SimpleDeclarator
+| | | |-operator
+| | | |-++
+| | | `-ParametersAndQualifiers
+| | |   |-(
+| | |   |-SimpleDeclaration
+| | |   | `-int
+| | |   `-)
+| | `-;
+| |-}
+| `-;
+`-SimpleDeclaration
+  |-void
+  |-SimpleDeclarator
+  | |-test
+  | `-ParametersAndQualifiers
+  |   |-(
+  |   |-SimpleDeclaration
+  |   | |-X
+  |   | `-SimpleDeclarator
+  |   |   `-x
+  |   `-)
+  

[PATCH] D82954: Fix crash on overloaded postfix unary operators due to invalid SourceLocation

[Eduardo Caldas via Phabricator via cfe-commits]

eduucaldas added inline comments.



Comment at: clang/unittests/Tooling/Syntax/TreeTest.cpp:2192
   R"cpp(
+class osstream {};
 struct X {

gribozavr2 wrote:
> I don't think we need a separate class to show the left shift operator. The 
> declaration below can be:
> 
> ```
>   friend X operator<<(X&, const X&);
> ```
If we don't bother much about "realistic" operator declarations we could drop 
all the `friend` and declare every operator in their most concise form. WDYT


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D82954/new/

https://reviews.llvm.org/D82954



___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D82954: Fix crash on overloaded postfix unary operators due to invalid SourceLocation

[Eduardo Caldas via Phabricator via cfe-commits]

eduucaldas updated this revision to Diff 275942.
eduucaldas added a comment.

Unifying user defined binary operator tests


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D82954/new/

https://reviews.llvm.org/D82954

Files:
  clang/lib/Tooling/Syntax/BuildTree.cpp
  clang/unittests/Tooling/Syntax/TreeTest.cpp

Index: clang/unittests/Tooling/Syntax/TreeTest.cpp
===
--- clang/unittests/Tooling/Syntax/TreeTest.cpp
+++ clang/unittests/Tooling/Syntax/TreeTest.cpp
@@ -2189,20 +2189,32 @@
   }
   EXPECT_TRUE(treeDumpEqual(
   R"cpp(
+class osstream {};
 struct X {
   X& operator=(const X&);
   friend X operator+(X, const X&);
   friend bool operator<(const X&, const X&);
+  friend osstream operator<<(osstream&, const X&);
+  X operator,(X&);
+  // TODO: Test for `->*`. Fix crash before
 };
-void test(X x, X y) {
+void test(X x, X y, osstream out) {
   x = y;
   x + y;
   x < y;
+  out << x;
+  x, y;
 }
 )cpp",
   R"txt(
 *: TranslationUnit
 |-SimpleDeclaration
+| |-class
+| |-osstream
+| |-{
+| |-}
+| `-;
+|-SimpleDeclaration
 | |-struct
 | |-X
 | |-{
@@ -2262,6 +2274,40 @@
 | |   |   |   `-&
 | |   |   `-)
 | |   `-;
+| |-UnknownDeclaration
+| | `-SimpleDeclaration
+| |   |-friend
+| |   |-osstream
+| |   |-SimpleDeclarator
+| |   | |-operator
+| |   | |-<<
+| |   | `-ParametersAndQualifiers
+| |   |   |-(
+| |   |   |-SimpleDeclaration
+| |   |   | |-osstream
+| |   |   | `-SimpleDeclarator
+| |   |   |   `-&
+| |   |   |-,
+| |   |   |-SimpleDeclaration
+| |   |   | |-const
+| |   |   | |-X
+| |   |   | `-SimpleDeclarator
+| |   |   |   `-&
+| |   |   `-)
+| |   `-;
+| |-SimpleDeclaration
+| | |-X
+| | |-SimpleDeclarator
+| | | |-operator
+| | | |-,
+| | | `-ParametersAndQualifiers
+| | |   |-(
+| | |   |-SimpleDeclaration
+| | |   | |-X
+| | |   | `-SimpleDeclarator
+| | |   |   `-&
+| | |   `-)
+| | `-;
 | |-}
 | `-;
 `-SimpleDeclaration
@@ -2279,6 +2325,11 @@
   |   | |-X
   |   | `-SimpleDeclarator
   |   |   `-y
+  |   |-,
+  |   |-SimpleDeclaration
+  |   | |-osstream
+  |   | `-SimpleDeclarator
+  |   |   `-out
   |   `-)
   `-CompoundStatement
 |-{
@@ -2319,6 +2370,185 @@
 | |   `-UnqualifiedId
 | | `-y
 | `-;
+|-ExpressionStatement
+| |-BinaryOperatorExpression
+| | |-IdExpression
+| | | `-UnqualifiedId
+| | |   `-out
+| | |-IdExpression
+| | | `-UnqualifiedId
+| | |   `-<<
+| | `-IdExpression
+| |   `-UnqualifiedId
+| | `-x
+| `-;
+|-ExpressionStatement
+| |-BinaryOperatorExpression
+| | |-IdExpression
+| | | `-UnqualifiedId
+| | |   `-x
+| | |-IdExpression
+| | | `-UnqualifiedId
+| | |   `-,
+| | `-IdExpression
+| |   `-UnqualifiedId
+| | `-y
+| `-;
+`-}
+)txt"));
+}
+
+TEST_P(SyntaxTreeTest, UserDefinedUnaryPrefixOperator) {
+  if (!GetParam().isCXX()) {
+return;
+  }
+  EXPECT_TRUE(treeDumpEqual(
+  R"cpp(
+struct X {
+  X operator++();
+  bool operator!();
+  X* operator&();
+};
+void test(X x) {
+  ++x;
+  !x;
+  
+}
+)cpp",
+  R"txt(
+*: TranslationUnit
+|-SimpleDeclaration
+| |-struct
+| |-X
+| |-{
+| |-SimpleDeclaration
+| | |-X
+| | |-SimpleDeclarator
+| | | |-operator
+| | | |-++
+| | | `-ParametersAndQualifiers
+| | |   |-(
+| | |   `-)
+| | `-;
+| |-SimpleDeclaration
+| | |-bool
+| | |-SimpleDeclarator
+| | | |-operator
+| | | |-!
+| | | `-ParametersAndQualifiers
+| | |   |-(
+| | |   `-)
+| | `-;
+| |-SimpleDeclaration
+| | |-X
+| | |-SimpleDeclarator
+| | | |-*
+| | | |-operator
+| | | |-&
+| | | `-ParametersAndQualifiers
+| | |   |-(
+| | |   `-)
+| | `-;
+| |-}
+| `-;
+`-SimpleDeclaration
+  |-void
+  |-SimpleDeclarator
+  | |-test
+  | `-ParametersAndQualifiers
+  |   |-(
+  |   |-SimpleDeclaration
+  |   | |-X
+  |   | `-SimpleDeclarator
+  |   |   `-x
+  |   `-)
+  `-CompoundStatement
+|-{
+|-ExpressionStatement
+| |-PrefixUnaryOperatorExpression
+| | |-IdExpression
+| | | `-UnqualifiedId
+| | |   `-++
+| | `-IdExpression
+| |   `-UnqualifiedId
+| | `-x
+| `-;
+|-ExpressionStatement
+| |-PrefixUnaryOperatorExpression
+| | |-IdExpression
+| | | `-UnqualifiedId
+| | |   `-!
+| | `-IdExpression
+| |   `-UnqualifiedId
+| | `-x
+| `-;
+|-ExpressionStatement
+| |-PrefixUnaryOperatorExpression
+| | |-IdExpression
+| | | `-UnqualifiedId
+| | |   `-&
+| | `-IdExpression
+| |   `-UnqualifiedId
+| | `-x
+| `-;
+`-}
+)txt"));
+}
+
+TEST_P(SyntaxTreeTest, UserDefinedUnaryPostfixOperator) {
+  if (!GetParam().isCXX()) {
+return;
+  }
+  EXPECT_TRUE(treeDumpEqual(
+  R"cpp(
+struct X {
+  X operator++(int);
+};
+void test(X x) {
+  x++;
+}
+)cpp",
+  R"txt(
+*: TranslationUnit
+|-SimpleDeclaration
+| |-struct
+| |-X
+| |-{
+| |-SimpleDeclaration
+| | |-X
+| | |-SimpleDeclarator
+| | | |-operator
+| | | 

[PATCH] D82954: Fix crash on overloaded postfix unary operators due to invalid SourceLocation

[Eduardo Caldas via Phabricator via cfe-commits]

eduucaldas added a comment.






Comment at: clang/lib/Tooling/Syntax/BuildTree.cpp:741
   bool WalkUpFromIntegerLiteral(IntegerLiteral *S) {
+if (S->getLocation().isInvalid())
+  return true;

gribozavr2 wrote:
> WDYT about overriding `TraverseCXXOperatorCallExpr`, so that we don't even 
> visit the synthetic argument of the postfix unary `++`? I would prefer to not 
> introduce blanket "if invalid then ignore" checks in the code.
>>! In D82954#2125300, @eduucaldas wrote:
> [[ https://godbolt.org/z/CWVEJ2 | Code that reproduces the crash ]]
> Notice that when using a postfix unary operator ++ one argument is introduced 
> to differ it from its prefix counterpart, but that argument is not used. This 
> phantom argument shows up in the ClangAST in the form of an `IntegerLiteral` 
> with `invalid sloc`. This invalid sloc in a valid AST node was causing the 
> SyntaxTree generation to crash.
> We can address this problems at two different levels:
> 1. At the `TraverseCXXOperatorCallExpr`, by overriding it and replacing the 
> children iteration to not iterate over this phantom argument.
> 2. At the `WalkUpFromIntegerLiteral`, by skipping the visitation of the 
> phantom node.
> We preferred the latter.
> 1. Cons: We would have to duplicate the implementation of RecursiveASTVisitor 
> in BuildTree, to handle this subtle change in traversal. That would add code 
> complexity.
> 2. Cons: We are handling a problem of `CXXOperatorCallExpr` in 
> `IntegerLiteral`. 
> 
> We chose the latter as, anyways, if an AST node has an invalid sloc, it was 
> either a problem with parsing or the node is supposed to be ignored

I've explained my reasoning in my first comment for this patch. But as it was a 
long time ago, I guess it got lost, even by me.
 
I'll sketch how the Traverse solution would look like, to be able to give more 
concrete arguments.



Comment at: clang/unittests/Tooling/Syntax/TreeTest.cpp:2376
+| | |   `-x
+| | `-IdExpression
+| |   `-UnqualifiedId

gribozavr2 wrote:
> I'm not sure about this part where `++` is wrapped in IdExpression -- 
> shouldn't the syntax tree look identical to a builtin postfix `++` (see 
> another test in this file)?
This comes back to a discussion we had a month ago about operators ( `+`, `!`, 
etc)
**Question**: Should we represent operators (built-in or overloaded) in the 
syntax tree uniformly? If so in which way?
**Context**: The ClangAST stores built-in operators as mere tokens, whereas 
overloaded operators are represented as a `DeclRefExpr`. That makes a lot of 
sense for the ClangAST, as we might refer back to the declaration of the 
overloaded operator, but the declaration of built-in operator doesn't exist.
**Conclusion**: Have the same representation for both types of operators. I 
have implemented the "unboxed" representation of overloaded operators, i.e. 
just storing their token in the syntax tree. I have not committed that, but I 
can do it just after this patch.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D82954/new/

https://reviews.llvm.org/D82954



___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D82954: Fix crash on overloaded postfix unary operators due to invalid SourceLocation

[Eduardo Caldas via Phabricator via cfe-commits]

eduucaldas updated this revision to Diff 275940.
eduucaldas marked 9 inline comments as done.
eduucaldas added a comment.

Answering comments


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D82954/new/

https://reviews.llvm.org/D82954

Files:
  clang/lib/Tooling/Syntax/BuildTree.cpp
  clang/unittests/Tooling/Syntax/TreeTest.cpp

Index: clang/unittests/Tooling/Syntax/TreeTest.cpp
===
--- clang/unittests/Tooling/Syntax/TreeTest.cpp
+++ clang/unittests/Tooling/Syntax/TreeTest.cpp
@@ -2189,20 +2189,29 @@
   }
   EXPECT_TRUE(treeDumpEqual(
   R"cpp(
+class osstream {};
 struct X {
   X& operator=(const X&);
   friend X operator+(X, const X&);
   friend bool operator<(const X&, const X&);
+  friend osstream operator<<(osstream&, const X&);
 };
-void test(X x, X y) {
+void test(X x, X y, osstream out) {
   x = y;
   x + y;
   x < y;
+  out << x;
 }
 )cpp",
   R"txt(
 *: TranslationUnit
 |-SimpleDeclaration
+| |-class
+| |-osstream
+| |-{
+| |-}
+| `-;
+|-SimpleDeclaration
 | |-struct
 | |-X
 | |-{
@@ -2262,6 +2271,27 @@
 | |   |   |   `-&
 | |   |   `-)
 | |   `-;
+| |-UnknownDeclaration
+| | `-SimpleDeclaration
+| |   |-friend
+| |   |-osstream
+| |   |-SimpleDeclarator
+| |   | |-operator
+| |   | |-<<
+| |   | `-ParametersAndQualifiers
+| |   |   |-(
+| |   |   |-SimpleDeclaration
+| |   |   | |-osstream
+| |   |   | `-SimpleDeclarator
+| |   |   |   `-&
+| |   |   |-,
+| |   |   |-SimpleDeclaration
+| |   |   | |-const
+| |   |   | |-X
+| |   |   | `-SimpleDeclarator
+| |   |   |   `-&
+| |   |   `-)
+| |   `-;
 | |-}
 | `-;
 `-SimpleDeclaration
@@ -2279,6 +2309,11 @@
   |   | |-X
   |   | `-SimpleDeclarator
   |   |   `-y
+  |   |-,
+  |   |-SimpleDeclaration
+  |   | |-osstream
+  |   | `-SimpleDeclarator
+  |   |   `-out
   |   `-)
   `-CompoundStatement
 |-{
@@ -2319,6 +2354,242 @@
 | |   `-UnqualifiedId
 | | `-y
 | `-;
+|-ExpressionStatement
+| |-BinaryOperatorExpression
+| | |-IdExpression
+| | | `-UnqualifiedId
+| | |   `-out
+| | |-IdExpression
+| | | `-UnqualifiedId
+| | |   `-<<
+| | `-IdExpression
+| |   `-UnqualifiedId
+| | `-x
+| `-;
+`-}
+)txt"));
+}
+
+TEST_P(SyntaxTreeTest, UserDefinedRareBinaryOperators) {
+  if (!GetParam().isCXX()) {
+return;
+  }
+  EXPECT_TRUE(treeDumpEqual(
+  R"cpp(
+struct X {
+  X operator,(X&);
+};
+void test(X x, X y) {
+  x, y;
+  // TODO: Test for `->*`. That introduced a crash
+}
+)cpp",
+  R"txt(
+*: TranslationUnit
+|-SimpleDeclaration
+| |-struct
+| |-X
+| |-{
+| |-SimpleDeclaration
+| | |-X
+| | |-SimpleDeclarator
+| | | |-operator
+| | | |-,
+| | | `-ParametersAndQualifiers
+| | |   |-(
+| | |   |-SimpleDeclaration
+| | |   | |-X
+| | |   | `-SimpleDeclarator
+| | |   |   `-&
+| | |   `-)
+| | `-;
+| |-}
+| `-;
+`-SimpleDeclaration
+  |-void
+  |-SimpleDeclarator
+  | |-test
+  | `-ParametersAndQualifiers
+  |   |-(
+  |   |-SimpleDeclaration
+  |   | |-X
+  |   | `-SimpleDeclarator
+  |   |   `-x
+  |   |-,
+  |   |-SimpleDeclaration
+  |   | |-X
+  |   | `-SimpleDeclarator
+  |   |   `-y
+  |   `-)
+  `-CompoundStatement
+|-{
+|-ExpressionStatement
+| |-BinaryOperatorExpression
+| | |-IdExpression
+| | | `-UnqualifiedId
+| | |   `-x
+| | |-IdExpression
+| | | `-UnqualifiedId
+| | |   `-,
+| | `-IdExpression
+| |   `-UnqualifiedId
+| | `-y
+| `-;
+`-}
+)txt"));
+}
+
+TEST_P(SyntaxTreeTest, UserDefinedUnaryPrefixOperator) {
+  if (!GetParam().isCXX()) {
+return;
+  }
+  EXPECT_TRUE(treeDumpEqual(
+  R"cpp(
+struct X {
+  X operator++();
+  bool operator!();
+  X* operator&();
+};
+void test(X x) {
+  ++x;
+  !x;
+  
+}
+)cpp",
+  R"txt(
+*: TranslationUnit
+|-SimpleDeclaration
+| |-struct
+| |-X
+| |-{
+| |-SimpleDeclaration
+| | |-X
+| | |-SimpleDeclarator
+| | | |-operator
+| | | |-++
+| | | `-ParametersAndQualifiers
+| | |   |-(
+| | |   `-)
+| | `-;
+| |-SimpleDeclaration
+| | |-bool
+| | |-SimpleDeclarator
+| | | |-operator
+| | | |-!
+| | | `-ParametersAndQualifiers
+| | |   |-(
+| | |   `-)
+| | `-;
+| |-SimpleDeclaration
+| | |-X
+| | |-SimpleDeclarator
+| | | |-*
+| | | |-operator
+| | | |-&
+| | | `-ParametersAndQualifiers
+| | |   |-(
+| | |   `-)
+| | `-;
+| |-}
+| `-;
+`-SimpleDeclaration
+  |-void
+  |-SimpleDeclarator
+  | |-test
+  | `-ParametersAndQualifiers
+  |   |-(
+  |   |-SimpleDeclaration
+  |   | |-X
+  |   | `-SimpleDeclarator
+  |   |   `-x
+  |   `-)
+  `-CompoundStatement
+|-{
+|-ExpressionStatement
+| |-PrefixUnaryOperatorExpression
+| | |-IdExpression
+| | | `-UnqualifiedId
+| | |   `-++
+| | `-IdExpression
+| |   `-UnqualifiedId
+| | `-x
+| `-;
+|-ExpressionStatement
+| |-PrefixUnaryOperatorExpression
+| | |-IdExpression
+| | | `-UnqualifiedId
+| | |   `-!
+| | `-IdExpression
+| |   

[PATCH] D82954: Fix crash on overloaded postfix unary operators due to invalid SourceLocation

[Eduardo Caldas via Phabricator via cfe-commits]

eduucaldas updated this revision to Diff 275930.
eduucaldas added a comment.

`->*` and `,` are binary operators.
Unit tests covering most of the operator kinds


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D82954/new/

https://reviews.llvm.org/D82954

Files:
  clang/lib/Tooling/Syntax/BuildTree.cpp
  clang/unittests/Tooling/Syntax/TreeTest.cpp

Index: clang/unittests/Tooling/Syntax/TreeTest.cpp
===
--- clang/unittests/Tooling/Syntax/TreeTest.cpp
+++ clang/unittests/Tooling/Syntax/TreeTest.cpp
@@ -2189,20 +2189,29 @@
   }
   EXPECT_TRUE(treeDumpEqual(
   R"cpp(
+class osstream {};
 struct X {
   X& operator=(const X&);
   friend X operator+(X, const X&);
   friend bool operator<(const X&, const X&);
+  friend osstream operator<<(osstream&, const X&);
 };
-void test(X x, X y) {
+void test(X x, X y, osstream out) {
   x = y;
   x + y;
   x < y;
+  out << x;
 }
 )cpp",
   R"txt(
 *: TranslationUnit
 |-SimpleDeclaration
+| |-class
+| |-osstream
+| |-{
+| |-}
+| `-;
+|-SimpleDeclaration
 | |-struct
 | |-X
 | |-{
@@ -2262,6 +2271,27 @@
 | |   |   |   `-&
 | |   |   `-)
 | |   `-;
+| |-UnknownDeclaration
+| | `-SimpleDeclaration
+| |   |-friend
+| |   |-osstream
+| |   |-SimpleDeclarator
+| |   | |-operator
+| |   | |-<<
+| |   | `-ParametersAndQualifiers
+| |   |   |-(
+| |   |   |-SimpleDeclaration
+| |   |   | |-osstream
+| |   |   | `-SimpleDeclarator
+| |   |   |   `-&
+| |   |   |-,
+| |   |   |-SimpleDeclaration
+| |   |   | |-const
+| |   |   | |-X
+| |   |   | `-SimpleDeclarator
+| |   |   |   `-&
+| |   |   `-)
+| |   `-;
 | |-}
 | `-;
 `-SimpleDeclaration
@@ -2279,6 +2309,11 @@
   |   | |-X
   |   | `-SimpleDeclarator
   |   |   `-y
+  |   |-,
+  |   |-SimpleDeclaration
+  |   | |-osstream
+  |   | `-SimpleDeclarator
+  |   |   `-out
   |   `-)
   `-CompoundStatement
 |-{
@@ -2319,6 +2354,242 @@
 | |   `-UnqualifiedId
 | | `-y
 | `-;
+|-ExpressionStatement
+| |-BinaryOperatorExpression
+| | |-IdExpression
+| | | `-UnqualifiedId
+| | |   `-out
+| | |-IdExpression
+| | | `-UnqualifiedId
+| | |   `-<<
+| | `-IdExpression
+| |   `-UnqualifiedId
+| | `-x
+| `-;
+`-}
+)txt"));
+}
+
+TEST_P(SyntaxTreeTest, UserDefinedRareBinaryOperators) {
+  if (!GetParam().isCXX()) {
+return;
+  }
+  EXPECT_TRUE(treeDumpEqual(
+  R"cpp(
+struct X {
+  X operator,(X&);
+};
+void test(X x, X y) {
+  x, y;
+  // TODO: Test for `->*`. That introduced a crash
+}
+)cpp",
+  R"txt(
+*: TranslationUnit
+|-SimpleDeclaration
+| |-struct
+| |-X
+| |-{
+| |-SimpleDeclaration
+| | |-X
+| | |-SimpleDeclarator
+| | | |-operator
+| | | |-,
+| | | `-ParametersAndQualifiers
+| | |   |-(
+| | |   |-SimpleDeclaration
+| | |   | |-X
+| | |   | `-SimpleDeclarator
+| | |   |   `-&
+| | |   `-)
+| | `-;
+| |-}
+| `-;
+`-SimpleDeclaration
+  |-void
+  |-SimpleDeclarator
+  | |-test
+  | `-ParametersAndQualifiers
+  |   |-(
+  |   |-SimpleDeclaration
+  |   | |-X
+  |   | `-SimpleDeclarator
+  |   |   `-x
+  |   |-,
+  |   |-SimpleDeclaration
+  |   | |-X
+  |   | `-SimpleDeclarator
+  |   |   `-y
+  |   `-)
+  `-CompoundStatement
+|-{
+|-ExpressionStatement
+| |-BinaryOperatorExpression
+| | |-IdExpression
+| | | `-UnqualifiedId
+| | |   `-x
+| | |-IdExpression
+| | | `-UnqualifiedId
+| | |   `-,
+| | `-IdExpression
+| |   `-UnqualifiedId
+| | `-y
+| `-;
+`-}
+)txt"));
+}
+
+TEST_P(SyntaxTreeTest, UserDefinedUnaryPrefixOperator) {
+  if (!GetParam().isCXX()) {
+return;
+  }
+  EXPECT_TRUE(treeDumpEqual(
+  R"cpp(
+struct X {
+  X operator++();
+  bool operator!();
+  X* operator&();
+};
+void test(X x) {
+  ++x;
+  !x;
+  
+}
+)cpp",
+  R"txt(
+*: TranslationUnit
+|-SimpleDeclaration
+| |-struct
+| |-X
+| |-{
+| |-SimpleDeclaration
+| | |-X
+| | |-SimpleDeclarator
+| | | |-operator
+| | | |-++
+| | | `-ParametersAndQualifiers
+| | |   |-(
+| | |   `-)
+| | `-;
+| |-SimpleDeclaration
+| | |-bool
+| | |-SimpleDeclarator
+| | | |-operator
+| | | |-!
+| | | `-ParametersAndQualifiers
+| | |   |-(
+| | |   `-)
+| | `-;
+| |-SimpleDeclaration
+| | |-X
+| | |-SimpleDeclarator
+| | | |-*
+| | | |-operator
+| | | |-&
+| | | `-ParametersAndQualifiers
+| | |   |-(
+| | |   `-)
+| | `-;
+| |-}
+| `-;
+`-SimpleDeclaration
+  |-void
+  |-SimpleDeclarator
+  | |-test
+  | `-ParametersAndQualifiers
+  |   |-(
+  |   |-SimpleDeclaration
+  |   | |-X
+  |   | `-SimpleDeclarator
+  |   |   `-x
+  |   `-)
+  `-CompoundStatement
+|-{
+|-ExpressionStatement
+| |-PrefixUnaryOperatorExpression
+| | |-IdExpression
+| | | `-UnqualifiedId
+| | |   `-++
+| | `-IdExpression
+| |   `-UnqualifiedId
+| | `-x
+| `-;
+|-ExpressionStatement
+| |-PrefixUnaryOperatorExpression
+| | |-IdExpression
+| | | `-UnqualifiedId
+| | |   `-!
+| | 

[PATCH] D82954: Fix crash on overloaded postfix unary operators due to invalid SourceLocation

[Eduardo Caldas via Phabricator via cfe-commits]

eduucaldas added inline comments.



Comment at: clang/lib/Tooling/Syntax/BuildTree.cpp:188
+  case OO_Comma:
+  case OO_ArrowStar:
+return syntax::NodeKind::UnknownExpression;

Actually arrow star is treated like a normal binary operator in the [[ 
https://eel.is/c++draft/expr.compound#expr.mptr.oper | grammar ]]


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D82954/new/

https://reviews.llvm.org/D82954



___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D82954: Fix crash on overloaded postfix unary operators due to invalid sloc

[Eduardo Caldas via Phabricator via cfe-commits]

eduucaldas added inline comments.



Comment at: clang/lib/Tooling/Syntax/BuildTree.cpp:118
 
+syntax::NodeKind getOperatorNodeKind(const CXXOperatorCallExpr ) {
+  switch (E.getOperator()) {

# Where to put this logic? 
The pro of having this function inside `BuildTree.cpp` is that it is closer to 
it's *only* usage. And also for now `BuildTree.cpp` agglomerates everything 
that takes a semantic AST as an input, so it would be coherent.

Another option is to put it in `Nodes`, as it might serve as documentation for 
`*OperatorExpression`s. 
For example, it would document that the semantic node `CXXOperatorCallExpr` can 
also generate the syntax node `BinaryOperatorExpression`, which was previously 
only generated by a semantic `BinaryOperator`

Another option is to add put this function as a lambda inside 
`WalkUpFromCXXOperatorCallExpr` as probably it will only be used there. 


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D82954/new/

https://reviews.llvm.org/D82954



___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D82937: Fix `isInfixBinaryOp` that returned true for postfix ++

[Eduardo Caldas via Phabricator via cfe-commits]

eduucaldas updated this revision to Diff 275325.
eduucaldas added a comment.

Remove review formatting noise


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D82937/new/

https://reviews.llvm.org/D82937

Files:
  clang/include/clang/AST/ExprCXX.h
  clang/lib/AST/ExprCXX.cpp
  clang/unittests/Tooling/CMakeLists.txt
  clang/unittests/Tooling/CXXOperatorCallExprTest.cpp

Index: clang/unittests/Tooling/CXXOperatorCallExprTest.cpp
===
--- /dev/null
+++ clang/unittests/Tooling/CXXOperatorCallExprTest.cpp
@@ -0,0 +1,77 @@
+//===- unittests/Tooling/CXXOperatorCallExprTest.cpp --===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===--===//
+//
+// Unit tests for the predicates in CXXOperatorCallExpr.
+//
+//===--===//
+
+#include "TestVisitor.h"
+#include "gtest/gtest.h"
+
+namespace clang {
+namespace {
+
+TEST(CXXOperatorPredicatesTest, InfixBinaryOp) {
+  const std::string Code = R"cpp(
+  struct X{
+friend X operator+(X, X);
+  };
+  void test(X x){
+x + x;
+  }
+  )cpp";
+
+  struct Visitor : TestVisitor {
+bool VisitCXXOperatorCallExpr(CXXOperatorCallExpr *E) {
+  EXPECT_TRUE(E->isInfixBinaryOp());
+  return true;
+}
+  } visitor;
+  visitor.runOver(Code);
+}
+
+TEST(CXXOperatorPredicatesTest, CallLikeOp) {
+  const std::string Code = R"cpp(
+  struct X{
+int operator[](int idx);
+  };
+  void test(X x){
+x[1];
+  }
+  )cpp";
+  struct Visitor : TestVisitor {
+bool VisitCXXOperatorCallExpr(CXXOperatorCallExpr *E) {
+  EXPECT_FALSE(E->isInfixBinaryOp());
+  return true;
+}
+  } visitor;
+
+  visitor.runOver(Code);
+}
+
+TEST(CXXOperatorPredicatesTest, PostfixUnaryOp) {
+  const std::string Code = R"cpp(
+  struct X{
+X operator++(int);
+  };
+  void test(X x){
+x++;
+  }
+  )cpp";
+
+  struct Visitor : TestVisitor {
+bool VisitCXXOperatorCallExpr(CXXOperatorCallExpr *E) {
+  EXPECT_FALSE(E->isInfixBinaryOp());
+  return true;
+}
+  } visitor;
+
+  visitor.runOver(Code);
+}
+} // namespace
+} // namespace clang
Index: clang/unittests/Tooling/CMakeLists.txt
===
--- clang/unittests/Tooling/CMakeLists.txt
+++ clang/unittests/Tooling/CMakeLists.txt
@@ -18,6 +18,7 @@
   CastExprTest.cpp
   CommentHandlerTest.cpp
   CompilationDatabaseTest.cpp
+  CXXOperatorCallExprTest.cpp
   DependencyScannerTest.cpp
   DiagnosticsYamlTest.cpp
   ExecutionTest.cpp
Index: clang/lib/AST/ExprCXX.cpp
===
--- clang/lib/AST/ExprCXX.cpp
+++ clang/lib/AST/ExprCXX.cpp
@@ -46,15 +46,18 @@
 //===--===//
 
 bool CXXOperatorCallExpr::isInfixBinaryOp() const {
-  // An infix binary operator is any operator with two arguments other than
-  // operator() and operator[]. Note that none of these operators can have
-  // default arguments, so it suffices to check the number of argument
-  // expressions.
   if (getNumArgs() != 2)
 return false;
 
   switch (getOperator()) {
-  case OO_Call: case OO_Subscript:
+  // operator() may have two arguments, but it's not a binary operator
+  case OO_Call:
+  // operator[] takes two arguments but it's not infix
+  case OO_Subscript:
+  // Postfix unary operators (++ and --) take 2 arguments to differ from their
+  // prefix counterparts
+  case OO_PlusPlus:
+  case OO_MinusMinus:
 return false;
   default:
 return true;
Index: clang/include/clang/AST/ExprCXX.h
===
--- clang/include/clang/AST/ExprCXX.h
+++ clang/include/clang/AST/ExprCXX.h
@@ -139,7 +139,6 @@
   }
   bool isComparisonOp() const { return isComparisonOp(getOperator()); }
 
-  /// Is this written as an infix binary operator?
   bool isInfixBinaryOp() const;
 
   /// Returns the location of the operator symbol in the expression.
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D82937: Fix `isInfixBinaryOp` that returned true for postfix ++

[Eduardo Caldas via Phabricator via cfe-commits]

eduucaldas updated this revision to Diff 275317.
eduucaldas added a comment.

Nits


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D82937/new/

https://reviews.llvm.org/D82937

Files:
  clang/include/clang/AST/ExprCXX.h
  clang/lib/AST/ExprCXX.cpp
  clang/unittests/Tooling/CMakeLists.txt
  clang/unittests/Tooling/CXXOperatorCallExprTest.cpp

Index: clang/unittests/Tooling/CXXOperatorCallExprTest.cpp
===
--- /dev/null
+++ clang/unittests/Tooling/CXXOperatorCallExprTest.cpp
@@ -0,0 +1,77 @@
+//===- unittests/Tooling/CXXOperatorCallExprTest.cpp --===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===--===//
+//
+// Unit tests for the predicates in CXXOperatorCallExpr.
+//
+//===--===//
+
+#include "TestVisitor.h"
+#include "gtest/gtest.h"
+
+namespace clang {
+namespace {
+
+TEST(CXXOperatorPredicatesTest, InfixBinaryOp) {
+  const std::string Code = R"cpp(
+  struct X{
+friend X operator+(X, X);
+  };
+  void test(X x){
+x + x;
+  }
+  )cpp";
+
+  struct Visitor : TestVisitor {
+bool VisitCXXOperatorCallExpr(CXXOperatorCallExpr *E) {
+  EXPECT_TRUE(E->isInfixBinaryOp());
+  return true;
+}
+  } visitor;
+  visitor.runOver(Code);
+}
+
+TEST(CXXOperatorPredicatesTest, CallLikeOp) {
+  const std::string Code = R"cpp(
+  struct X{
+int operator[](int idx);
+  };
+  void test(X x){
+x[1];
+  }
+  )cpp";
+  struct Visitor : TestVisitor {
+bool VisitCXXOperatorCallExpr(CXXOperatorCallExpr *E) {
+  EXPECT_FALSE(E->isInfixBinaryOp());
+  return true;
+}
+  } visitor;
+
+  visitor.runOver(Code);
+}
+
+TEST(CXXOperatorPredicatesTest, PostfixUnaryOp) {
+  const std::string Code = R"cpp(
+  struct X{
+X operator++(int);
+  };
+  void test(X x){
+x++;
+  }
+  )cpp";
+
+  struct Visitor : TestVisitor {
+bool VisitCXXOperatorCallExpr(CXXOperatorCallExpr *E) {
+  EXPECT_FALSE(E->isInfixBinaryOp());
+  return true;
+}
+  } visitor;
+
+  visitor.runOver(Code);
+}
+} // namespace
+} // namespace clang
Index: clang/unittests/Tooling/CMakeLists.txt
===
--- clang/unittests/Tooling/CMakeLists.txt
+++ clang/unittests/Tooling/CMakeLists.txt
@@ -18,6 +18,7 @@
   CastExprTest.cpp
   CommentHandlerTest.cpp
   CompilationDatabaseTest.cpp
+  CXXOperatorCallExprTest.cpp
   DependencyScannerTest.cpp
   DiagnosticsYamlTest.cpp
   ExecutionTest.cpp
Index: clang/lib/AST/ExprCXX.cpp
===
--- clang/lib/AST/ExprCXX.cpp
+++ clang/lib/AST/ExprCXX.cpp
@@ -46,15 +46,17 @@
 //===--===//
 
 bool CXXOperatorCallExpr::isInfixBinaryOp() const {
-  // An infix binary operator is any operator with two arguments other than
-  // operator() and operator[]. Note that none of these operators can have
-  // default arguments, so it suffices to check the number of argument
-  // expressions.
   if (getNumArgs() != 2)
 return false;
-
   switch (getOperator()) {
-  case OO_Call: case OO_Subscript:
+  // operator() may have two arguments, but it's not a binary operator
+  case OO_Call:
+  // operator[] takes two arguments but it's not infix
+  case OO_Subscript:
+  // Postfix unary operators (++ and --) take 2 arguments to differ from their
+  // prefix counterparts
+  case OO_PlusPlus:
+  case OO_MinusMinus:
 return false;
   default:
 return true;
Index: clang/include/clang/AST/ExprCXX.h
===
--- clang/include/clang/AST/ExprCXX.h
+++ clang/include/clang/AST/ExprCXX.h
@@ -121,6 +121,7 @@
Opc == OO_GreaterGreaterEqual || Opc == OO_AmpEqual ||
Opc == OO_CaretEqual || Opc == OO_PipeEqual;
   }
+
   bool isAssignmentOp() const { return isAssignmentOp(getOperator()); }
 
   static bool isComparisonOp(OverloadedOperatorKind Opc) {
@@ -137,9 +138,9 @@
   return false;
 }
   }
+
   bool isComparisonOp() const { return isComparisonOp(getOperator()); }
 
-  /// Is this written as an infix binary operator?
   bool isInfixBinaryOp() const;
 
   /// Returns the location of the operator symbol in the expression.
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits