[PATCH] D82312: Add `CharLiteral` to SyntaxTree

2020-06-26 Thread Dmitri Gribenko via Phabricator via cfe-commits 
gribozavr2 added a comment.

@vvereschaka Sorry about that. It looks like a bug in MSVC. I implemented a 
workaround in https://reviews.llvm.org/D82636 -- please review if you have time.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D82312/new/

https://reviews.llvm.org/D82312



___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D82312: Add `CharLiteral` to SyntaxTree

2020-06-26 Thread Eduardo Caldas via Phabricator via cfe-commits 
eduucaldas added a comment.

In D82312#2116028 , @vvereschaka wrote:

> Hello Eduardo,
>
> sorry, but one of your these commits break the clang unit tests on the 
> following builders:
>
> - http://lab.llvm.org:8011/builders/llvm-clang-x86_64-win-fast/builds/18313
> - http://lab.llvm.org:8011/builders/llvm-clang-win-x-aarch64/builds/256
> - http://lab.llvm.org:8011/builders/llvm-clang-win-x-armv7l/builds/96
>
>   with the following errors: ``` FAILED: 
> tools/clang/unittests/Tooling/Syntax/CMakeFiles/SyntaxTests.dir/TreeTest.cpp.obj
>  
> C:\PROGRA~2\MICROS~1\2017\COMMUN~1\VC\Tools\MSVC\1416~1.270\bin\Hostx64\x64\cl.exe
>   /nologo /TP -DGTEST_HAS_RTTI=0 -DGTEST_HAS_TR1_TUPLE=0 -DGTEST_LANG_CXX11=1 
> -DUNICODE -D_CRT_NONSTDC_NO_DEPRECATE -D_CRT_NONSTDC_NO_WARNINGS 
> -D_CRT_SECURE_NO_DEPRECATE -D_CRT_SECURE_NO_WARNINGS -D_GNU_SOURCE 
> -D_HAS_EXCEPTIONS=0 -D_SCL_SECURE_NO_DEPRECATE -D_SCL_SECURE_NO_WARNINGS 
> -D_UNICODE -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS 
> -D__STDC_LIMIT_MACROS -Itools\clang\unittests\Tooling\Syntax 
> -IC:\buildbot\as-builder-3\llvm-clang-x86_64-win-fast\llvm-project\clang\unittests\Tooling\Syntax
>  
> -IC:\buildbot\as-builder-3\llvm-clang-x86_64-win-fast\llvm-project\clang\include
>  -Itools\clang\include -Iinclude 
> -IC:\buildbot\as-builder-3\llvm-clang-x86_64-win-fast\llvm-project\llvm\include
>  
> -IC:\buildbot\as-builder-3\llvm-clang-x86_64-win-fast\llvm-project\llvm\utils\unittest\googletest\include
>  
> -IC:\buildbot\as-builder-3\llvm-clang-x86_64-win-fast\llvm-project\llvm\utils\unittest\googlemock\include
>  /DWIN32 /D_WINDOWS   /Zc:inline /Zc:strictStrings /Oi /Zc:rvalueCast /W4 
>  -wd4141 -wd4146 -wd4244 -wd4267 -wd4291 -wd4351 
> -wd4456 -wd4457 -wd4458 -wd4459 -wd4503 -wd4624 -wd4722 -wd4100 -wd4127 
> -wd4512 -wd4505 -wd4610 -wd4510 -wd4702 -wd4245 -wd4706 -wd4310 -wd4701 
> -wd4703 -wd4389 -wd4611 -wd4805 -wd4204 -wd4577 -wd4091 -wd4592 -wd4319 
> -wd4709 -wd4324 -w14062 -we4238 /Gw /MD /O2 
>  /Ob2 /DNDEBUG/EHs-c- /GR- 
> -std:c++14 /showIncludes 
> /Fotools\clang\unittests\Tooling\Syntax\CMakeFiles\SyntaxTests.dir\TreeTest.cpp.obj
>  /Fdtools\clang\unittests\Tooling\Syntax\CMakeFiles\SyntaxTests.dir\ /FS -c 
> C:\buildbot\as-builder-3\llvm-clang-x86_64-win-fast\llvm-project\clang\unittests\Tooling\Syntax\TreeTest.cpp
>  
> C:\buildbot\as-builder-3\llvm-clang-x86_64-win-fast\llvm-project\clang\unittests\Tooling\Syntax\TreeTest.cpp(1665):
>  error C2017: illegal escape sequence 
> C:\buildbot\as-builder-3\llvm-clang-x86_64-win-fast\llvm-project\clang\unittests\Tooling\Syntax\TreeTest.cpp(1665):
>  error C2146: syntax error: missing ')' before identifier 'n' 
> C:\buildbot\as-builder-3\llvm-clang-x86_64-win-fast\llvm-project\clang\unittests\Tooling\Syntax\TreeTest.cpp(1639):
>  error C2660: 'testing::internal::GetBoolAssertionFailureMessage': function 
> does not take 2 arguments 
> C:\buildbot\as-builder-3\llvm-clang-x86_64-win-fast\llvm-project\llvm\utils\unittest\googletest\include\gtest/internal/gtest-internal.h(226):
>  note: see declaration of 'testing::internal::GetBoolAssertionFailureMessage' 
> C:\buildbot\as-builder-3\llvm-clang-x86_64-win-fast\llvm-project\clang\unittests\Tooling\Syntax\TreeTest.cpp(1639):
>  error C2440: '': cannot convert from 'initializer list' 
> to 'testing::internal::AssertHelper' 
> C:\buildbot\as-builder-3\llvm-clang-x86_64-win-fast\llvm-project\clang\unittests\Tooling\Syntax\TreeTest.cpp(1665):
>  note: No constructor could take the source type, or constructor overload 
> resolution was ambiguous 
> C:\buildbot\as-builder-3\llvm-clang-x86_64-win-fast\llvm-project\clang\unittests\Tooling\Syntax\TreeTest.cpp(1665):
>  error C2146: syntax error: missing ';' before identifier 'n' 
> C:\buildbot\as-builder-3\llvm-clang-x86_64-win-fast\llvm-project\clang\unittests\Tooling\Syntax\TreeTest.cpp(1665):
>  error C2059: syntax error: ')' 
> C:\buildbot\as-builder-3\llvm-clang-x86_64-win-fast\llvm-project\clang\unittests\Tooling\Syntax\TreeTest.cpp(1639):
>  error C2065: 'n': undeclared identifier 
> C:\buildbot\as-builder-3\llvm-clang-x86_64-win-fast\llvm-project\clang\unittests\Tooling\Syntax\TreeTest.cpp(1665):
>  error C2146: syntax error: missing ';' before identifier 'SyntaxTree' 
> C:\buildbot\as-builder-3\llvm-clang-x86_64-win-fast\llvm-project\clang\unittests\Tooling\Syntax\TreeTest.cpp(1639):
>  error C2065: 'SyntaxTree': undeclared identifier 
> C:\buildbot\as-builder-3\llvm-clang-x86_64-win-fast\llvm-project\clang\unittests\Tooling\Syntax\TreeTest.cpp(1639):
>  error C2143: syntax error: missing ';' before 'string' ``` see more details 
> here: 
> http://lab.llvm.org:8011/builders/llvm-clang-x86_64-win-fast/builds/18313/steps/test-check-clang-unit/logs/stdio
>
>   I mean the following commits: *221d7bbe49cceb0e408f0f46d9f8371e6c9fee2c 
>

[PATCH] D82312: Add `CharLiteral` to SyntaxTree

2020-06-25 Thread Vlad Vereschaka via Phabricator via cfe-commits 
vvereschaka added a comment.

Hello Eduardo,

sorry, but one of your these commits break the clang unit tests on the 
following builders:

- http://lab.llvm.org:8011/builders/llvm-clang-x86_64-win-fast/builds/18313
- http://lab.llvm.org:8011/builders/llvm-clang-win-x-aarch64/builds/256
- http://lab.llvm.org:8011/builders/llvm-clang-win-x-armv7l/builds/96

with the following errors:

  FAILED: 
tools/clang/unittests/Tooling/Syntax/CMakeFiles/SyntaxTests.dir/TreeTest.cpp.obj
 
  
C:\PROGRA~2\MICROS~1\2017\COMMUN~1\VC\Tools\MSVC\1416~1.270\bin\Hostx64\x64\cl.exe
  /nologo /TP -DGTEST_HAS_RTTI=0 -DGTEST_HAS_TR1_TUPLE=0 -DGTEST_LANG_CXX11=1 
-DUNICODE -D_CRT_NONSTDC_NO_DEPRECATE -D_CRT_NONSTDC_NO_WARNINGS 
-D_CRT_SECURE_NO_DEPRECATE -D_CRT_SECURE_NO_WARNINGS -D_GNU_SOURCE 
-D_HAS_EXCEPTIONS=0 -D_SCL_SECURE_NO_DEPRECATE -D_SCL_SECURE_NO_WARNINGS 
-D_UNICODE -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS 
-D__STDC_LIMIT_MACROS -Itools\clang\unittests\Tooling\Syntax 
-IC:\buildbot\as-builder-3\llvm-clang-x86_64-win-fast\llvm-project\clang\unittests\Tooling\Syntax
 
-IC:\buildbot\as-builder-3\llvm-clang-x86_64-win-fast\llvm-project\clang\include
 -Itools\clang\include -Iinclude 
-IC:\buildbot\as-builder-3\llvm-clang-x86_64-win-fast\llvm-project\llvm\include 
-IC:\buildbot\as-builder-3\llvm-clang-x86_64-win-fast\llvm-project\llvm\utils\unittest\googletest\include
 
-IC:\buildbot\as-builder-3\llvm-clang-x86_64-win-fast\llvm-project\llvm\utils\unittest\googlemock\include
 /DWIN32 /D_WINDOWS   /Zc:inline /Zc:strictStrings /Oi /Zc:rvalueCast /W4 
-wd4141 -wd4146 -wd4244 -wd4267 -wd4291 -wd4351 -wd4456 -wd4457 -wd4458 -wd4459 
-wd4503 -wd4624 -wd4722 -wd4100 -wd4127 -wd4512 -wd4505 -wd4610 -wd4510 -wd4702 
-wd4245 -wd4706 -wd4310 -wd4701 -wd4703 -wd4389 -wd4611 -wd4805 -wd4204 -wd4577 
-wd4091 -wd4592 -wd4319 -wd4709 -wd4324 -w14062 -we4238 /Gw /MD /O2 /Ob2 
/DNDEBUG/EHs-c- /GR- -std:c++14 /showIncludes 
/Fotools\clang\unittests\Tooling\Syntax\CMakeFiles\SyntaxTests.dir\TreeTest.cpp.obj
 /Fdtools\clang\unittests\Tooling\Syntax\CMakeFiles\SyntaxTests.dir\ /FS -c 
C:\buildbot\as-builder-3\llvm-clang-x86_64-win-fast\llvm-project\clang\unittests\Tooling\Syntax\TreeTest.cpp
  
C:\buildbot\as-builder-3\llvm-clang-x86_64-win-fast\llvm-project\clang\unittests\Tooling\Syntax\TreeTest.cpp(1665):
 error C2017: illegal escape sequence
  
C:\buildbot\as-builder-3\llvm-clang-x86_64-win-fast\llvm-project\clang\unittests\Tooling\Syntax\TreeTest.cpp(1665):
 error C2146: syntax error: missing ')' before identifier 'n'
  
C:\buildbot\as-builder-3\llvm-clang-x86_64-win-fast\llvm-project\clang\unittests\Tooling\Syntax\TreeTest.cpp(1639):
 error C2660: 'testing::internal::GetBoolAssertionFailureMessage': function 
does not take 2 arguments
  
C:\buildbot\as-builder-3\llvm-clang-x86_64-win-fast\llvm-project\llvm\utils\unittest\googletest\include\gtest/internal/gtest-internal.h(226):
 note: see declaration of 'testing::internal::GetBoolAssertionFailureMessage'
  
C:\buildbot\as-builder-3\llvm-clang-x86_64-win-fast\llvm-project\clang\unittests\Tooling\Syntax\TreeTest.cpp(1639):
 error C2440: '': cannot convert from 'initializer list' 
to 'testing::internal::AssertHelper'
  
C:\buildbot\as-builder-3\llvm-clang-x86_64-win-fast\llvm-project\clang\unittests\Tooling\Syntax\TreeTest.cpp(1665):
 note: No constructor could take the source type, or constructor overload 
resolution was ambiguous
  
C:\buildbot\as-builder-3\llvm-clang-x86_64-win-fast\llvm-project\clang\unittests\Tooling\Syntax\TreeTest.cpp(1665):
 error C2146: syntax error: missing ';' before identifier 'n'
  
C:\buildbot\as-builder-3\llvm-clang-x86_64-win-fast\llvm-project\clang\unittests\Tooling\Syntax\TreeTest.cpp(1665):
 error C2059: syntax error: ')'
  
C:\buildbot\as-builder-3\llvm-clang-x86_64-win-fast\llvm-project\clang\unittests\Tooling\Syntax\TreeTest.cpp(1639):
 error C2065: 'n': undeclared identifier
  
C:\buildbot\as-builder-3\llvm-clang-x86_64-win-fast\llvm-project\clang\unittests\Tooling\Syntax\TreeTest.cpp(1665):
 error C2146: syntax error: missing ';' before identifier 'SyntaxTree'
  
C:\buildbot\as-builder-3\llvm-clang-x86_64-win-fast\llvm-project\clang\unittests\Tooling\Syntax\TreeTest.cpp(1639):
 error C2065: 'SyntaxTree': undeclared identifier
  
C:\buildbot\as-builder-3\llvm-clang-x86_64-win-fast\llvm-project\clang\unittests\Tooling\Syntax\TreeTest.cpp(1639):
 error C2143: syntax error: missing ';' before 'string'

see more details here: 
http://lab.llvm.org:8011/builders/llvm-clang-x86_64-win-fast/builds/18313/steps/test-check-clang-unit/logs/stdio

I mean the following commits:
*221d7bbe49cceb0e408f0f46d9f8371e6c9fee2c 
* 
https://reviews.llvm.org/D82312
*466e8b7ea6e162d48cac42ccda210bdeb11080e3 
* 
https://reviews.llvm.org/D82360
*7b404b6d003181e990f53d27866ee98d5151c4f3

[PATCH] D82312: Add `CharLiteral` to SyntaxTree

2020-06-25 Thread Eduardo Caldas via Phabricator via cfe-commits 
This revision was automatically updated to reflect the committed changes.
Closed by commit rG221d7bbe49cc: Add `CharLiteral` to SyntaxTree (authored by 
eduucaldas).

Changed prior to commit:
  https://reviews.llvm.org/D82312?vs=273343&id=273436#toc

Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D82312/new/

https://reviews.llvm.org/D82312

Files:
  clang/include/clang/Tooling/Syntax/Nodes.h
  clang/lib/Tooling/Syntax/BuildTree.cpp
  clang/lib/Tooling/Syntax/Nodes.cpp
  clang/unittests/Tooling/Syntax/TreeTest.cpp

Index: clang/unittests/Tooling/Syntax/TreeTest.cpp
===
--- clang/unittests/Tooling/Syntax/TreeTest.cpp
+++ clang/unittests/Tooling/Syntax/TreeTest.cpp
@@ -73,6 +73,10 @@
 return Language == Lang_C89 || Language == Lang_C99;
   }
 
+  bool isCXX17OrLater() const {
+return Language == Lang_CXX17 || Language == Lang_CXX20;
+  }
+
   bool supportsCXXDynamicExceptionSpecification() const {
 return Language == Lang_CXX03 || Language == Lang_CXX11 ||
Language == Lang_CXX14;
@@ -1232,6 +1236,135 @@
 )txt"));
 }
 
+TEST_P(SyntaxTreeTest, CharacterLiteral) {
+  EXPECT_TRUE(treeDumpEqual(
+  R"cpp(
+void test() {
+  'a';
+  '\n';
+  '\x20';
+  '\0';
+  L'a';
+  L'α';
+}
+)cpp",
+  R"txt(
+*: TranslationUnit
+`-SimpleDeclaration
+  |-void
+  |-SimpleDeclarator
+  | |-test
+  | `-ParametersAndQualifiers
+  |   |-(
+  |   `-)
+  `-CompoundStatement
+|-{
+|-ExpressionStatement
+| |-CharacterLiteralExpression
+| | `-'a'
+| `-;
+|-ExpressionStatement
+| |-CharacterLiteralExpression
+| | `-'\n'
+| `-;
+|-ExpressionStatement
+| |-CharacterLiteralExpression
+| | `-'\x20'
+| `-;
+|-ExpressionStatement
+| |-CharacterLiteralExpression
+| | `-'\0'
+| `-;
+|-ExpressionStatement
+| |-CharacterLiteralExpression
+| | `-L'a'
+| `-;
+|-ExpressionStatement
+| |-CharacterLiteralExpression
+| | `-L'α'
+| `-;
+`-}
+)txt"));
+}
+
+TEST_P(SyntaxTreeTest, CharacterLiteralUtf) {
+  if (!GetParam().isCXX11OrLater()) {
+return;
+  }
+  EXPECT_TRUE(treeDumpEqual(
+  R"cpp(
+void test() {
+  u'a';
+  u'構';
+  U'a';
+  U'🌲';
+}
+)cpp",
+  R"txt(
+*: TranslationUnit
+`-SimpleDeclaration
+  |-void
+  |-SimpleDeclarator
+  | |-test
+  | `-ParametersAndQualifiers
+  |   |-(
+  |   `-)
+  `-CompoundStatement
+|-{
+|-ExpressionStatement
+| |-CharacterLiteralExpression
+| | `-u'a'
+| `-;
+|-ExpressionStatement
+| |-CharacterLiteralExpression
+| | `-u'構'
+| `-;
+|-ExpressionStatement
+| |-CharacterLiteralExpression
+| | `-U'a'
+| `-;
+|-ExpressionStatement
+| |-CharacterLiteralExpression
+| | `-U'🌲'
+| `-;
+`-}
+)txt"));
+}
+
+TEST_P(SyntaxTreeTest, CharacterLiteralUtf8) {
+  if (!GetParam().isCXX17OrLater()) {
+return;
+  }
+  EXPECT_TRUE(treeDumpEqual(
+  R"cpp(
+void test() {
+  u8'a';
+  u8'\x7f';
+}
+)cpp",
+  R"txt(
+*: TranslationUnit
+`-SimpleDeclaration
+  |-void
+  |-SimpleDeclarator
+  | |-test
+  | `-ParametersAndQualifiers
+  |   |-(
+  |   `-)
+  `-CompoundStatement
+|-{
+|-ExpressionStatement
+| |-CharacterLiteralExpression
+| | `-u8'a'
+| `-;
+|-ExpressionStatement
+| |-CharacterLiteralExpression
+| | `-u8'\x7f'
+| `-;
+`-}
+)txt"));
+}
+
 TEST_P(SyntaxTreeTest, BoolLiteral) {
   if (GetParam().hasBoolType()) {
 return;
Index: clang/lib/Tooling/Syntax/Nodes.cpp
===
--- clang/lib/Tooling/Syntax/Nodes.cpp
+++ clang/lib/Tooling/Syntax/Nodes.cpp
@@ -24,6 +24,8 @@
 return OS << "IntegerLiteralExpression";
   case NodeKind::BoolLiteralExpression:
 return OS << "BoolLiteralExpression";
+  case NodeKind::CharacterLiteralExpression:
+return OS << "CharacterLiteralExpression";
   case NodeKind::PrefixUnaryOperatorExpression:
 return OS << "PrefixUnaryOperatorExpression";
   case NodeKind::PostfixUnaryOperatorExpression:
@@ -207,6 +209,11 @@
   findChild(syntax::NodeRole::LiteralToken));
 }
 
+syntax::Leaf *syntax::CharacterLiteralExpression::literalToken() {
+  return llvm::cast_or_null(
+  findChild(syntax::NodeRole::LiteralToken));
+}
+
 syntax::Leaf *syntax::CxxNullPtrExpression::nullPtrKeyword() {
   return llvm::cast_or_null(
   findChild(syntax::NodeRole::LiteralToken));
Index: clang/lib/Tooling/Syntax/BuildTree.cpp
===
--- clang/lib/Tooling/Syntax/BuildTree.cpp
+++ clang/lib/Tooling/Syntax/BuildTree.cpp
@@ -661,6 +661,13 @@
 return true;
   }
 
+  bool WalkUpFromCharacterLiteral(CharacterLiteral *S) {
+Builder.markChildToken(S->getLocation(), syntax::NodeRole::LiteralToken);
+Builder.foldNode(Builder.getExprRange(S),
+ new (allocator()) syntax::CharacterLiteralExpression

[PATCH] D82312: Add `CharLiteral` to SyntaxTree

2020-06-25 Thread Eduardo Caldas via Phabricator via cfe-commits 
eduucaldas updated this revision to Diff 273343.
eduucaldas marked 3 inline comments as done.
eduucaldas added a comment.

Add tests for unicode characters


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D82312/new/

https://reviews.llvm.org/D82312

Files:
  clang/include/clang/Tooling/Syntax/Nodes.h
  clang/lib/Tooling/Syntax/BuildTree.cpp
  clang/lib/Tooling/Syntax/Nodes.cpp
  clang/unittests/Tooling/Syntax/TreeTest.cpp

Index: clang/unittests/Tooling/Syntax/TreeTest.cpp
===
--- clang/unittests/Tooling/Syntax/TreeTest.cpp
+++ clang/unittests/Tooling/Syntax/TreeTest.cpp
@@ -69,6 +69,10 @@
Language == Lang_CXX20;
   }
 
+  bool isCXX17OrLater() const {
+return Language == Lang_CXX17 || Language == Lang_CXX20;
+  }
+
   bool supportsCXXDynamicExceptionSpecification() const {
 return Language == Lang_CXX03 || Language == Lang_CXX11 ||
Language == Lang_CXX14;
@@ -1228,6 +1232,135 @@
 )txt"));
 }
 
+TEST_P(SyntaxTreeTest, CharacterLiteral) {
+  EXPECT_TRUE(treeDumpEqual(
+  R"cpp(
+void test() {
+  'a';
+  '\n';
+  '\x20';
+  '\0';
+  L'a';
+  L'α';
+}
+)cpp",
+  R"txt(
+*: TranslationUnit
+`-SimpleDeclaration
+  |-void
+  |-SimpleDeclarator
+  | |-test
+  | `-ParametersAndQualifiers
+  |   |-(
+  |   `-)
+  `-CompoundStatement
+|-{
+|-ExpressionStatement
+| |-CharacterLiteralExpression
+| | `-'a'
+| `-;
+|-ExpressionStatement
+| |-CharacterLiteralExpression
+| | `-'\n'
+| `-;
+|-ExpressionStatement
+| |-CharacterLiteralExpression
+| | `-'\x20'
+| `-;
+|-ExpressionStatement
+| |-CharacterLiteralExpression
+| | `-'\0'
+| `-;
+|-ExpressionStatement
+| |-CharacterLiteralExpression
+| | `-L'a'
+| `-;
+|-ExpressionStatement
+| |-CharacterLiteralExpression
+| | `-L'α'
+| `-;
+`-}
+)txt"));
+}
+
+TEST_P(SyntaxTreeTest, CharacterLiteralUtf) {
+  if (!GetParam().isCXX11OrLater()) {
+return;
+  }
+  EXPECT_TRUE(treeDumpEqual(
+  R"cpp(
+void test() {
+  u'a';
+  u'構';
+  U'a';
+  U'🌲';
+}
+)cpp",
+  R"txt(
+*: TranslationUnit
+`-SimpleDeclaration
+  |-void
+  |-SimpleDeclarator
+  | |-test
+  | `-ParametersAndQualifiers
+  |   |-(
+  |   `-)
+  `-CompoundStatement
+|-{
+|-ExpressionStatement
+| |-CharacterLiteralExpression
+| | `-u'a'
+| `-;
+|-ExpressionStatement
+| |-CharacterLiteralExpression
+| | `-u'構'
+| `-;
+|-ExpressionStatement
+| |-CharacterLiteralExpression
+| | `-U'a'
+| `-;
+|-ExpressionStatement
+| |-CharacterLiteralExpression
+| | `-U'🌲'
+| `-;
+`-}
+)txt"));
+}
+
+TEST_P(SyntaxTreeTest, CharacterLiteralUtf8) {
+  if (!GetParam().isCXX17OrLater()) {
+return;
+  }
+  EXPECT_TRUE(treeDumpEqual(
+  R"cpp(
+void test() {
+  u8'a';
+  u8'\x7f';
+}
+)cpp",
+  R"txt(
+*: TranslationUnit
+`-SimpleDeclaration
+  |-void
+  |-SimpleDeclarator
+  | |-test
+  | `-ParametersAndQualifiers
+  |   |-(
+  |   `-)
+  `-CompoundStatement
+|-{
+|-ExpressionStatement
+| |-CharacterLiteralExpression
+| | `-u8'a'
+| `-;
+|-ExpressionStatement
+| |-CharacterLiteralExpression
+| | `-u8'\x7f'
+| `-;
+`-}
+)txt"));
+}
+
 TEST_P(SyntaxTreeTest, IntegerLiteral) {
   EXPECT_TRUE(treeDumpEqual(
   R"cpp(
Index: clang/lib/Tooling/Syntax/Nodes.cpp
===
--- clang/lib/Tooling/Syntax/Nodes.cpp
+++ clang/lib/Tooling/Syntax/Nodes.cpp
@@ -22,6 +22,8 @@
 return OS << "CxxNullPtrExpression";
   case NodeKind::IntegerLiteralExpression:
 return OS << "IntegerLiteralExpression";
+  case NodeKind::CharacterLiteralExpression:
+return OS << "CharacterLiteralExpression";
   case NodeKind::PrefixUnaryOperatorExpression:
 return OS << "PrefixUnaryOperatorExpression";
   case NodeKind::PostfixUnaryOperatorExpression:
@@ -200,6 +202,11 @@
   findChild(syntax::NodeRole::LiteralToken));
 }
 
+syntax::Leaf *syntax::CharacterLiteralExpression::literalToken() {
+  return llvm::cast_or_null(
+  findChild(syntax::NodeRole::LiteralToken));
+}
+
 syntax::Leaf *syntax::CxxNullPtrExpression::nullPtrKeyword() {
   return llvm::cast_or_null(
   findChild(syntax::NodeRole::LiteralToken));
Index: clang/lib/Tooling/Syntax/BuildTree.cpp
===
--- clang/lib/Tooling/Syntax/BuildTree.cpp
+++ clang/lib/Tooling/Syntax/BuildTree.cpp
@@ -654,6 +654,13 @@
 return true;
   }
 
+  bool WalkUpFromCharacterLiteral(CharacterLiteral *S) {
+Builder.markChildToken(S->getLocation(), syntax::NodeRole::LiteralToken);
+Builder.foldNode(Builder.getExprRange(S),
+ new (allocator()) syntax::CharacterLiteralExpression, S);
+return true;
+  }
+
   bool WalkUpFromCXXNullPtrLiteralExpr(CXXNullPtrLiteralExpr *S) {
 Builder.ma

[PATCH] D82312: Add `CharLiteral` to SyntaxTree

2020-06-24 Thread Dmitri Gribenko via Phabricator via cfe-commits 
gribozavr2 accepted this revision.
gribozavr2 added a comment.
This revision is now accepted and ready to land.

LGTM after new tests are added.




Comment at: clang/unittests/Tooling/Syntax/TreeTest.cpp:1240
+  'a';
+  L'a';
+}

Could you add tests for escape sequences (`'\n'`, `'\x20'` etc.), and non-ASCII 
characters, here, and below in Unicode literals?



Comment at: clang/unittests/Tooling/Syntax/TreeTest.cpp:1266
+
+TEST_P(SyntaxTreeTest, CharacterLiteralCxx11) {
+  if (!GetParam().isCXX11OrLater()) {

eduucaldas wrote:
> Rename to `CharacterLiteralUtf`?
Either name is good, I think.



Comment at: clang/unittests/Tooling/Syntax/TreeTest.cpp:1300
+
+TEST_P(SyntaxTreeTest, CharacterLiteralCxx17) {
+  if (!GetParam().isCXX17OrLater()) {

eduucaldas wrote:
> Rename to `CharacterLiteralUtf8`?
Either name is good, I think.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D82312/new/

https://reviews.llvm.org/D82312



___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D82312: Add `CharLiteral` to SyntaxTree

2020-06-23 Thread Eduardo Caldas via Phabricator via cfe-commits 
eduucaldas marked 2 inline comments as done.
eduucaldas added inline comments.



Comment at: clang/unittests/Tooling/Syntax/TreeTest.cpp:1266
+
+TEST_P(SyntaxTreeTest, CharacterLiteralCxx11) {
+  if (!GetParam().isCXX11OrLater()) {

Rename to `CharacterLiteralUtf`?



Comment at: clang/unittests/Tooling/Syntax/TreeTest.cpp:1300
+
+TEST_P(SyntaxTreeTest, CharacterLiteralCxx17) {
+  if (!GetParam().isCXX17OrLater()) {

Rename to `CharacterLiteralUtf8`?


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D82312/new/

https://reviews.llvm.org/D82312



___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D82312: Add `CharLiteral` to SyntaxTree

2020-06-22 Thread Eduardo Caldas via Phabricator via cfe-commits 
eduucaldas created this revision.
Herald added a project: clang.
Herald added a subscriber: cfe-commits.
eduucaldas added a reviewer: gribozavr2.

Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D82312

Files:
  clang/include/clang/Tooling/Syntax/Nodes.h
  clang/lib/Tooling/Syntax/BuildTree.cpp
  clang/lib/Tooling/Syntax/Nodes.cpp
  clang/unittests/Tooling/Syntax/TreeTest.cpp

Index: clang/unittests/Tooling/Syntax/TreeTest.cpp
===
--- clang/unittests/Tooling/Syntax/TreeTest.cpp
+++ clang/unittests/Tooling/Syntax/TreeTest.cpp
@@ -69,6 +69,10 @@
Language == Lang_CXX20;
   }
 
+  bool isCXX17OrLater() const {
+return Language == Lang_CXX17 || Language == Lang_CXX20;
+  }
+
   bool supportsCXXDynamicExceptionSpecification() const {
 return Language == Lang_CXX03 || Language == Lang_CXX11 ||
Language == Lang_CXX14;
@@ -1228,6 +1232,100 @@
 )txt"));
 }
 
+TEST_P(SyntaxTreeTest, CharacterLiteral) {
+  EXPECT_TRUE(treeDumpEqual(
+  R"cpp(
+void test() {
+  'a';
+  L'a';
+}
+)cpp",
+  R"txt(
+*: TranslationUnit
+`-SimpleDeclaration
+  |-void
+  |-SimpleDeclarator
+  | |-test
+  | `-ParametersAndQualifiers
+  |   |-(
+  |   `-)
+  `-CompoundStatement
+|-{
+|-ExpressionStatement
+| |-CharacterLiteralExpression
+| | `-'a'
+| `-;
+|-ExpressionStatement
+| |-CharacterLiteralExpression
+| | `-L'a'
+| `-;
+`-}
+)txt"));
+}
+
+TEST_P(SyntaxTreeTest, CharacterLiteralCxx11) {
+  if (!GetParam().isCXX11OrLater()) {
+return;
+  }
+  EXPECT_TRUE(treeDumpEqual(
+  R"cpp(
+void test() {
+  u'a';
+  U'a';
+}
+)cpp",
+  R"txt(
+*: TranslationUnit
+`-SimpleDeclaration
+  |-void
+  |-SimpleDeclarator
+  | |-test
+  | `-ParametersAndQualifiers
+  |   |-(
+  |   `-)
+  `-CompoundStatement
+|-{
+|-ExpressionStatement
+| |-CharacterLiteralExpression
+| | `-u'a'
+| `-;
+|-ExpressionStatement
+| |-CharacterLiteralExpression
+| | `-U'a'
+| `-;
+`-}
+)txt"));
+}
+
+TEST_P(SyntaxTreeTest, CharacterLiteralCxx17) {
+  if (!GetParam().isCXX17OrLater()) {
+return;
+  }
+  EXPECT_TRUE(treeDumpEqual(
+  R"cpp(
+void test() {
+  u8'a';
+}
+)cpp",
+  R"txt(
+*: TranslationUnit
+`-SimpleDeclaration
+  |-void
+  |-SimpleDeclarator
+  | |-test
+  | `-ParametersAndQualifiers
+  |   |-(
+  |   `-)
+  `-CompoundStatement
+|-{
+|-ExpressionStatement
+| |-CharacterLiteralExpression
+| | `-u8'a'
+| `-;
+`-}
+)txt"));
+}
+
 TEST_P(SyntaxTreeTest, IntegerLiteral) {
   EXPECT_TRUE(treeDumpEqual(
   R"cpp(
Index: clang/lib/Tooling/Syntax/Nodes.cpp
===
--- clang/lib/Tooling/Syntax/Nodes.cpp
+++ clang/lib/Tooling/Syntax/Nodes.cpp
@@ -22,6 +22,8 @@
 return OS << "CxxNullPtrExpression";
   case NodeKind::IntegerLiteralExpression:
 return OS << "IntegerLiteralExpression";
+  case NodeKind::CharacterLiteralExpression:
+return OS << "CharacterLiteralExpression";
   case NodeKind::PrefixUnaryOperatorExpression:
 return OS << "PrefixUnaryOperatorExpression";
   case NodeKind::PostfixUnaryOperatorExpression:
@@ -200,6 +202,11 @@
   findChild(syntax::NodeRole::LiteralToken));
 }
 
+syntax::Leaf *syntax::CharacterLiteralExpression::literalToken() {
+  return llvm::cast_or_null(
+  findChild(syntax::NodeRole::LiteralToken));
+}
+
 syntax::Leaf *syntax::CxxNullPtrExpression::nullPtrKeyword() {
   return llvm::cast_or_null(
   findChild(syntax::NodeRole::LiteralToken));
Index: clang/lib/Tooling/Syntax/BuildTree.cpp
===
--- clang/lib/Tooling/Syntax/BuildTree.cpp
+++ clang/lib/Tooling/Syntax/BuildTree.cpp
@@ -654,6 +654,13 @@
 return true;
   }
 
+  bool WalkUpFromCharacterLiteral(CharacterLiteral *S) {
+Builder.markChildToken(S->getLocation(), syntax::NodeRole::LiteralToken);
+Builder.foldNode(Builder.getExprRange(S),
+ new (allocator()) syntax::CharacterLiteralExpression, S);
+return true;
+  }
+
   bool WalkUpFromCXXNullPtrLiteralExpr(CXXNullPtrLiteralExpr *S) {
 Builder.markChildToken(S->getLocation(), syntax::NodeRole::LiteralToken);
 Builder.foldNode(Builder.getExprRange(S),
Index: clang/include/clang/Tooling/Syntax/Nodes.h
===
--- clang/include/clang/Tooling/Syntax/Nodes.h
+++ clang/include/clang/Tooling/Syntax/Nodes.h
@@ -45,6 +45,7 @@
   BinaryOperatorExpression,
   CxxNullPtrExpression,
   IntegerLiteralExpression,
+  CharacterLiteralExpression,
   IdExpression,
 
   // Statements.
@@ -254,6 +255,17 @@
   syntax::Leaf *nullPtrKeyword();
 };
 
+/// Expression for character literals. C++ [lex.ccon]
+class CharacterLiteralExpression final : public Expression {
+public:
+  CharacterLiteralExpression()
+  : Expression(NodeKind::CharacterLiteralExpression) {}
+  sta