Re: [blink-dev] Intent to Ship: Tabbed web apps

[Brett Wilson]

To follow up, there's a nice shout out at Google I/O about this feature and
also mentions that Figma is trying it:

https://youtu.be/KFeuEMAaKfM?si=pr71c4hfxhI5RbT0=1039

On Sun, May 12, 2024 at 6:08 PM Matt Giuca  wrote:

> Thanks everyone. This is now enabled on ToT in 126.
>
> On Fri, 10 May 2024 at 17:59, TAMURA, Kent  wrote:
>
>> LGTM3.
>>
>>
>> On Fri, May 10, 2024 at 2:41 AM Mike Taylor 
>> wrote:
>>
>>> LGTM2
>>> On 5/9/24 10:48 AM, Chris Harrelson wrote:
>>>
>>> Thanks for these clarifications.
>>>
>>> LGTM1
>>>
>>> On Wed, May 8, 2024 at 6:05 PM Daniel Murphy 
>>> wrote:
>>>
>>>> We don't have an objection to this feature existing - it's actually
>>>> currently in our backlog. But it is very low in our priority list. While we
>>>> can review patches, we will not be able to dedicate resources for
>>>> consultation or maintenance.
>>>>
>>>> On Wed, May 8, 2024 at 9:19 AM Brett Wilson 
>>>> wrote:
>>>>
>>>>> On Wed, May 8, 2024 at 8:39 AM Alex Russell 
>>>>> wrote:
>>>>>
>>>>>> I'm happy for this to be CrOS first, but would like to unpack Brett's
>>>>>> statement above a bit. If we (MSFT) were to polish this up for Windows,
>>>>>> would patches for that be accepted?
>>>>>>
>>>>>
>>>>> I can't speak for the browser team. But my current understanding is
>>>>> that the browser team likes this in principle but doesn't prioritize it
>>>>> high enough to work on it right now (this is a correction from my previous
>>>>> assertion). So I'm pretty sure patches enabling this for other platforms
>>>>> would be accepted.
>>>>>
>>>>> Brett
>>>>> --
>>>>> You received this message because you are subscribed to the Google
>>>>> Groups "blink-dev" group.
>>>>> To unsubscribe from this group and stop receiving emails from it, send
>>>>> an email to blink-dev+unsubscr...@chromium.org.
>>>>> To view this discussion on the web visit
>>>>> https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CABiGVV-Zeyv3Rez%2BPQ%2B%2BfW4ihpRCwnnGN2HNxOyXTA7_uWehzw%40mail.gmail.com
>>>>> <https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CABiGVV-Zeyv3Rez%2BPQ%2B%2BfW4ihpRCwnnGN2HNxOyXTA7_uWehzw%40mail.gmail.com?utm_medium=email_source=footer>
>>>>> .
>>>>>
>>>> --
>>>> You received this message because you are subscribed to the Google
>>>> Groups "blink-dev" group.
>>>> To unsubscribe from this group and stop receiving emails from it, send
>>>> an email to blink-dev+unsubscr...@chromium.org.
>>>> To view this discussion on the web visit
>>>> https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CA%2B4qT32O-zoM4tarHQvoHkmYt%2B%3Dc5iOiPdkueMk%2BhUe7mkYU%2BA%40mail.gmail.com
>>>> <https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CA%2B4qT32O-zoM4tarHQvoHkmYt%2B%3Dc5iOiPdkueMk%2BhUe7mkYU%2BA%40mail.gmail.com?utm_medium=email_source=footer>
>>>> .
>>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "blink-dev" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to blink-dev+unsubscr...@chromium.org.
>>> To view this discussion on the web visit
>>> https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAOMQ%2Bw_QmtkQeDUW%2BEbWtQC4ghvpHFk-Uf65YqYjPMjEQEwwCg%40mail.gmail.com
>>> <https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAOMQ%2Bw_QmtkQeDUW%2BEbWtQC4ghvpHFk-Uf65YqYjPMjEQEwwCg%40mail.gmail.com?utm_medium=email_source=footer>
>>> .
>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "blink-dev" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to blink-dev+unsubscr...@chromium.org.
>>> To view this discussion on the web visit
>>> https://groups.google.com/a/chromium.org/d/msgid/blink-dev/201d6c2d-be84-4d6b-9c4b-b1c9f89b0bf8%40chromium.org
>>> <https://groups.google.com/a/chromium.org/d/msgid/blink-dev/201d6c2d-be84-4d6b-9c4b-b1c9f89b0bf8%40chromium.org?utm_medium=email_source=footer>
>>> .
>>>
>>
>>
>> --
>> TAMURA Kent
>> Software Engineer, Google
>>
>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to blink-dev+unsubscr...@chromium.org.
To view this discussion on the web visit 
https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CABiGVV90nyjCkO8zvKA3g7VjCBfbBfqxP_yj9jjeE-0eN-pQYA%40mail.gmail.com.

Re: [blink-dev] Intent to Ship: Tabbed web apps

[Brett Wilson]

On Wed, May 8, 2024 at 8:39 AM Alex Russell 
wrote:

> I'm happy for this to be CrOS first, but would like to unpack Brett's
> statement above a bit. If we (MSFT) were to polish this up for Windows,
> would patches for that be accepted?
>

I can't speak for the browser team. But my current understanding is that
the browser team likes this in principle but doesn't prioritize it high
enough to work on it right now (this is a correction from my previous
assertion). So I'm pretty sure patches enabling this for other platforms
would be accepted.

Brett

-- 
You received this message because you are subscribed to the Google Groups 
"blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to blink-dev+unsubscr...@chromium.org.
To view this discussion on the web visit 
https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CABiGVV-Zeyv3Rez%2BPQ%2B%2BfW4ihpRCwnnGN2HNxOyXTA7_uWehzw%40mail.gmail.com.

Re: [blink-dev] Intent to Ship: Tabbed web apps

[Brett Wilson]

On Mon, May 6, 2024 at 3:02 AM Yoav Weiss (@Shopify) 
wrote:

>
>
> On Fri, May 3, 2024 at 7:28 PM Brett Wilson  wrote:
>
>> Contact emailsbre...@chromium.org, alancut...@chromium.org,
>> mgi...@chromium.org, loubr...@google.com
>>
>> Explainer
>> https://github.com/WICG/manifest-incubations/blob/gh-pages/tabbed-mode-explainer.md
>>
>> Specificationhttps://wicg.github.io/manifest-incubations/#dfn-tabbed
>>
>> Summary
>>
>> Allow web app windows to have a tab strip. This adds a new display mode
>> "tabbed" and a new manifest field to allow customizations to the tab strip.
>>
>>
>> Blink componentBlink>AppManifest
>> <https://bugs.chromium.org/p/chromium/issues/list?q=component:Blink%3EAppManifest>
>>
>> TAG reviewhttps://github.com/w3ctag/design-reviews/issues/841
>>
>> TAG review statusIssues addressed
>>
>> Chromium Trial NameWebAppTabStrip
>>
>> Link to origin trial feedback summary
>> https://github.com/WICG/manifest-incubations/issues
>>
>> Origin Trial documentation link
>> https://github.com/WICG/manifest-incubations/blob/gh-pages/tabbed-mode-explainer.md
>>
>> Risks
>>
>>
>> Interoperability and Compatibility
>>
>>
>>
>> *Gecko*: Defer (https://github.com/mozilla/standards-positions/issues/811
>> )
>>
>> *WebKit*: No signal (
>> https://github.com/WebKit/standards-positions/issues/195)
>>
>> *Web developers*: Positive (https://github.com/w3c/manifest/issues/737)
>>
>> *Other signals*:
>>
>> WebView application risks
>>
>> Does this intent deprecate or change behavior of existing APIs, such that
>> it has potentially high risk for Android WebView-based applications?
>>
>> N/A
>>
>>
>> Debuggability
>>
>> chrome://web-app-internals can be used for debugging, and the new
>> manifest field could also be added to the DevTools Application pane.
>>
>>
>> Will this feature be supported on all six Blink platforms (Windows, Mac,
>> Linux, ChromeOS, Android, and Android WebView)?No
>>
>> The origin trial is available on ChromeOS only. Support for other desktop
>> platforms is planned.
>>
>>
>> Is this feature fully tested by web-platform-tests
>> <https://chromium.googlesource.com/chromium/src/+/main/docs/testing/web_platform_tests.md>
>> ?Yes
>>
>>
>> https://github.com/web-platform-tests/wpt/tree/master/appmanifest/display-override-member
>>
>>
>> Flag name on chrome://flagschrome://flags/#enable-desktop-pwas-tab-strip
>>
>> Finch feature nameDesktopPWAsTabStrip
>>
>> Requires code in //chrome?True
>>
>> Tracking bughttps://issuetracker.google.com/issues/40598974
>>
>> Launch bughttps://launch.corp.google.com/launch/4253814
>>
>> MeasurementLaunch.WebAppDisplayMode: Tabbed
>>
>> Availability expectationFeature is available only on Chrome-on-ChromeOS
>> for the foreseeable future.
>>
>
> This seems a bit contradictory with "Support for other desktop platforms
> is planned" above. Are there plans for support beyond CrOS?
>
>>
>> <https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CABiGVV9MstA8bLmUTLkkfTjeYK8bb7fkhyKL_OMt_d7UzavRTA%40mail.gmail.com?utm_medium=email_source=footer>
>
>
Sorry, the first part was a mistake. Chrome team has requested this not be
on other platforms now.

Brett

-- 
You received this message because you are subscribed to the Google Groups 
"blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to blink-dev+unsubscr...@chromium.org.
To view this discussion on the web visit 
https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CABiGVV8N3PxBDPFGkE5M_-g22qWnQGhNq3sd%2BDxCgCmCN%3DS4Xg%40mail.gmail.com.

[blink-dev] Intent to Ship: Tabbed web apps

[Brett Wilson]

Contact emailsbre...@chromium.org, alancut...@chromium.org,
mgi...@chromium.org, loubr...@google.com

Explainer
https://github.com/WICG/manifest-incubations/blob/gh-pages/tabbed-mode-explainer.md

Specificationhttps://wicg.github.io/manifest-incubations/#dfn-tabbed

Summary

Allow web app windows to have a tab strip. This adds a new display mode
"tabbed" and a new manifest field to allow customizations to the tab strip.


Blink componentBlink>AppManifest


TAG reviewhttps://github.com/w3ctag/design-reviews/issues/841

TAG review statusIssues addressed

Chromium Trial NameWebAppTabStrip

Link to origin trial feedback summary
https://github.com/WICG/manifest-incubations/issues

Origin Trial documentation link
https://github.com/WICG/manifest-incubations/blob/gh-pages/tabbed-mode-explainer.md

Risks


Interoperability and Compatibility



*Gecko*: Defer (https://github.com/mozilla/standards-positions/issues/811)

*WebKit*: No signal (
https://github.com/WebKit/standards-positions/issues/195)

*Web developers*: Positive (https://github.com/w3c/manifest/issues/737)

*Other signals*:

WebView application risks

Does this intent deprecate or change behavior of existing APIs, such that
it has potentially high risk for Android WebView-based applications?

N/A


Debuggability

chrome://web-app-internals can be used for debugging, and the new manifest
field could also be added to the DevTools Application pane.


Will this feature be supported on all six Blink platforms (Windows, Mac,
Linux, ChromeOS, Android, and Android WebView)?No

The origin trial is available on ChromeOS only. Support for other desktop
platforms is planned.


Is this feature fully tested by web-platform-tests

?Yes

https://github.com/web-platform-tests/wpt/tree/master/appmanifest/display-override-member


Flag name on chrome://flagschrome://flags/#enable-desktop-pwas-tab-strip

Finch feature nameDesktopPWAsTabStrip

Requires code in //chrome?True

Tracking bughttps://issuetracker.google.com/issues/40598974

Launch bughttps://launch.corp.google.com/launch/4253814

MeasurementLaunch.WebAppDisplayMode: Tabbed

Availability expectationFeature is available only on Chrome-on-ChromeOS for
the foreseeable future.

Adoption expectationFeature is used by specific partner(s) to provide
functionality within 12 months of launch in Chrome. May be of interest to a
handful of PWA authors primarily in the productivity space.

Adoption planWorking with a small number of partners directly.

Non-OSS dependencies

Does the feature depend on any code or APIs outside the Chromium open
source repository and its open-source dependencies to function?
N/A

Sample links
https://paint-rightful-patch.glitch.me

Estimated milestones
Shipping on desktop 126
Origin trial desktop first 118
Origin trial desktop last 126
Origin trial extension 1 end milestone 126

Anticipated spec changes

Open questions about a feature may be a source of future web compat or
interop issues. Please list open issues (e.g. links to known github issues
in the project for the feature specification) whose resolution may
introduce web compat/interop risk (e.g., changing to naming or structure of
the API in a non-backward-compatible way).
Chromium implementation currently does not parse string-form URL patterns
as required by the spec. Marked "at risk". (
https://github.com/WICG/manifest-incubations/issues/97)

Link to entry on the Chrome Platform Status
https://chromestatus.com/feature/5128143454076928?gate=6176288199409664

Links to previous Intent discussionsIntent to prototype:
https://groups.google.com/a/chromium.org/g/blink-dev/c/IvfIkjvQYuY/m/cixwOyEeAAAJ
Intent
to Experiment:
https://groups.google.com/a/chromium.org/g/blink-dev/c/m16m2TEq-NM/m/0Bc10numCgAJ
Intent to Extend Experiment 1:
https://groups.google.com/a/chromium.org/g/blink-dev/c/5aRDL-E9olQ/m/Pb7ECdcpAAAJ
Intent to Ship:
https://groups.google.com/a/chromium.org/g/blink-dev/c/5aRDL-E9olQ/m/Pb7ECdcpAAAJ


This intent message was generated by Chrome Platform Status
.

-- 
You received this message because you are subscribed to the Google Groups 
"blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to blink-dev+unsubscr...@chromium.org.
To view this discussion on the web visit 
https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CABiGVV9MstA8bLmUTLkkfTjeYK8bb7fkhyKL_OMt_d7UzavRTA%40mail.gmail.com.

Re: [blink-dev] Intent to Continue Experimenting: Tabbed Web Apps

[Brett Wilson]

On Wed, Apr 10, 2024 at 8:29 AM Alex Russell 
wrote:

> Great to hear there's support for the feature.
>
> 118-126 w/o breaking changes is pushing things. Is it practical to ship
> inside the 123-126 window? How close are we?\
>

Hi Alex,

Sorry for the delay, we've been trying to figure out the timing for this
feature since there are a bunch of moving parts. We're going to try to get
this into M126 (soon) and are working on the finishing touches. Look for
communication in the coming weeks.

The main reason is that M126 is the ChromeOS Long-Term Support ("LTS")
release used by many companies and schools. The date-based origin trials
interact particularly poorly with the LTS releases, and because this
feature is triggered by a manifest, it can't really be feature-detected for
a fallback like you would normally do for a web trial. As a result, the
options are (1) land in M126, (2) extend the origin trial until M126 LTS
ends in 2025, or (3) inflict brokenness on LTS users.

To answer Stefan's questions:

   - I think this would be a nice feature on all platforms as well, but the
   browser team has not shown interest outside of ChromeOS for now.
   - The thinking behind the lack of favicons is that they would normally
   all be the same which is a bit weird.
   - The active/inactive split should be inherited from the general browser
   tab strip behavior. We have promised not to diverge the presentation from
   the default tab strip to keep the maintenance burden on the team lower
   while re-using the browser tab strip code.

Brett


> On Tuesday, April 9, 2024 at 9:07:37 AM UTC-7 Stefan Peter wrote:
>
>> Hi, I think it would help to extend the Open Trial to Windows and Mac as
>> well. We're really like that feature it fits perfectly our strategy and the
>> behaviour of or App (having mutliple remote session to computers active in
>> different tabs)
>>
>> I have only minor problems with it, like that there are no favicons in
>> the tabs or that the visible split between inactive tabs is not so well
>> visible. But otherwise can't wait to have it as a non experimental feature.
>> :)
>>
>> Domenic Denicola schrieb am Mittwoch, 13. März 2024 um 03:45:58 UTC+1:
>>
>>> LGTM to extend. The recent spec progress looks significant, and other
>>> areas are in good shape.
>>>
>>> On Wed, Mar 13, 2024 at 11:36 AM Matt Giuca  wrote:
>>>
>> The original trial was from 118 to 123, inclusive.

 We want to extend it so it continues running from 124 to 126, inclusive.

 On Fri, 8 Mar 2024 at 16:52, Domenic Denicola 
 wrote:

>>> For what milestones do you want to extend the experiment for?
>
> On Thu, Mar 7, 2024 at 3:39 PM Matt Giuca  wrote:
>
>> Contact emailsloub...@google.com, gle...@chromium.org,
>> mgi...@chromium.org, bre...@chromium.org
>>
>
>>
>> Explainer
>> https://github.com/WICG/manifest-incubations/blob/gh-pages/tabbed-mode-explainer.md
>>
>> Specificationhttps://github.com/WICG/manifest-incubations/pull/95
>>  (draft)
>>
>> Summary
>>
>> Allow web app windows to have a tab strip. This adds a new display
>> mode "tabbed" and a new manifest field to allow customizations to the tab
>> strip.
>>
>> Following a 6-milestone origin trial, we would like to continue
>> experimenting as the feature team is not ready to commit to shipping, and
>> gathered very little data from the initial experiment (as partners we 
>> have
>> lined up have not yet started their experiment).
>>
>> Per the Blink policy, we have made substantial progress in these five
>> areas:
>>
>>- Draft spec: https://github.com/WICG/manifest-incubations/pull/95
>>- TAG review: https://github.com/w3ctag/design-reviews/issues/841
>>(closed with "unsatisfied"; I have sent a follow-up comment).
>>- Signals requests:
>>   - WebKit:
>>   https://github.com/WebKit/standards-positions/issues/195
>>   (ignored)
>>   - Mozilla:
>>   https://github.com/mozilla/standards-positions/issues/811
>>   (closed as not interested in any of "these sorts of features")
>>- Outreach for feedback from spec community:
>>https://developer.chrome.com/docs/capabilities/tabbed-application-mode
>>- WPT tests:
>>
>> https://github.com/web-platform-tests/wpt/blob/master/appmanifest/display-override-member/display-override-member-media-feature-tabbed-manual.tentative.html
>>
>> Blink componentBlink>AppManifest
>> 
>>
>> TAG reviewhttps://github.com/w3ctag/design-reviews/issues/841
>>
>> TAG review statusClosed ("Unsatisfied")
>>
>> Chromium Trial NameWebAppTabStrip
>>
>> Origin trial feedback summary
>> Only a small number of signups, most look like individuals wanting to
>> experiment. A handful of 

[PATCH] D139154: [clang-doc] Add template support

[Brett Wilson via Phabricator via cfe-commits]

This revision was automatically updated to reflect the committed changes.
Closed by commit rG4a68babd9973: [clang-doc] Add template support. (authored by 
brettw).

Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D139154/new/

https://reviews.llvm.org/D139154

Files:
  clang-tools-extra/clang-doc/BitcodeReader.cpp
  clang-tools-extra/clang-doc/BitcodeWriter.cpp
  clang-tools-extra/clang-doc/BitcodeWriter.h
  clang-tools-extra/clang-doc/Representation.cpp
  clang-tools-extra/clang-doc/Representation.h
  clang-tools-extra/clang-doc/Serialize.cpp
  clang-tools-extra/clang-doc/YAMLGenerator.cpp
  clang-tools-extra/test/clang-doc/single-file-public.cpp
  clang-tools-extra/test/clang-doc/single-file.cpp
  clang-tools-extra/test/clang-doc/templates.cpp
  clang-tools-extra/unittests/clang-doc/HTMLGeneratorTest.cpp
  clang-tools-extra/unittests/clang-doc/SerializeTest.cpp
  clang-tools-extra/unittests/clang-doc/YAMLGeneratorTest.cpp

Index: clang-tools-extra/unittests/clang-doc/YAMLGeneratorTest.cpp
===
--- clang-tools-extra/unittests/clang-doc/YAMLGeneratorTest.cpp
+++ clang-tools-extra/unittests/clang-doc/YAMLGeneratorTest.cpp
@@ -28,10 +28,11 @@
   I.Path = "path/to/A";
   I.Namespace.emplace_back(EmptySID, "A", InfoType::IT_namespace);
 
-  I.Children.Namespaces.emplace_back(EmptySID, "ChildNamespace",
- InfoType::IT_namespace,
- "path/to/A/Namespace");
+  I.Children.Namespaces.emplace_back(
+  EmptySID, "ChildNamespace", InfoType::IT_namespace,
+  "path::to::A::Namespace::ChildNamespace", "path/to/A/Namespace");
   I.Children.Records.emplace_back(EmptySID, "ChildStruct", InfoType::IT_record,
+  "path::to::A::Namespace::ChildStruct",
   "path/to/A/Namespace");
   I.Children.Functions.emplace_back();
   I.Children.Functions.back().Name = "OneFunction";
@@ -53,13 +54,16 @@
 Namespace:
   - Type:Namespace
 Name:'A'
+QualName:'A'
 ChildNamespaces:
   - Type:Namespace
 Name:'ChildNamespace'
+QualName:'path::to::A::Namespace::ChildNamespace'
 Path:'path/to/A/Namespace'
 ChildRecords:
   - Type:Record
 Name:'ChildStruct'
+QualName:'path::to::A::Namespace::ChildStruct'
 Path:'path/to/A/Namespace'
 ChildFunctions:
   - USR: ''
@@ -83,8 +87,7 @@
   I.DefLoc = Location(10, llvm::SmallString<16>{"test.cpp"});
   I.Loc.emplace_back(12, llvm::SmallString<16>{"test.cpp"});
 
-  I.Members.emplace_back(TypeInfo("int", "path/to/int"), "X",
- AccessSpecifier::AS_private);
+  I.Members.emplace_back(TypeInfo("int"), "X", AccessSpecifier::AS_private);
 
   // Member documentation.
   CommentInfo TopComment;
@@ -103,15 +106,15 @@
AccessSpecifier::AS_public, true);
   I.Bases.back().Children.Functions.emplace_back();
   I.Bases.back().Children.Functions.back().Name = "InheritedFunctionOne";
-  I.Bases.back().Members.emplace_back(TypeInfo("int", "path/to/int"), "N",
+  I.Bases.back().Members.emplace_back(TypeInfo("int"), "N",
   AccessSpecifier::AS_private);
   // F is in the global namespace
   I.Parents.emplace_back(EmptySID, "F", InfoType::IT_record, "");
   I.VirtualParents.emplace_back(EmptySID, "G", InfoType::IT_record,
-"path/to/G");
+"path::to::G::G", "path/to/G");
 
   I.Children.Records.emplace_back(EmptySID, "ChildStruct", InfoType::IT_record,
-  "path/to/A/r");
+  "path::to::A::r::ChildStruct", "path/to/A/r");
   I.Children.Functions.emplace_back();
   I.Children.Functions.back().Name = "OneFunction";
   I.Children.Enums.emplace_back();
@@ -131,6 +134,7 @@
 Namespace:
   - Type:Namespace
 Name:'A'
+QualName:'A'
 DefLocation:
   LineNumber:  10
   Filename:'test.cpp'
@@ -142,7 +146,7 @@
 Members:
   - Type:
   Name:'int'
-  Path:'path/to/int'
+  QualName:'int'
 Name:'X'
 Access:  Private
 Description:
@@ -161,7 +165,7 @@
 Members:
   - Type:
   Name:'int'
-  Path:'path/to/int'
+  QualName:'int'
 Name:'N'
 Access:  Private
 ChildFunctions:
@@ -178,10 +182,12 @@
 VirtualParents:
   - Type:Record
 Name:'G'
+QualName:'path::to::G::G'
 Path:'path/to/G'
 ChildRecords:
   - Type:Record
 Name:'ChildStruct'
+QualName:'path::to::A::r::ChildStruct'
 

[clang-tools-extra] 4a68bab - [clang-doc] Add template support.

[Brett Wilson via cfe-commits]

Author: Brett Wilson
Date: 2022-12-08T08:02:02-08:00
New Revision: 4a68babd9973f043fd3e40f159fbb990880606a6

URL: 
https://github.com/llvm/llvm-project/commit/4a68babd9973f043fd3e40f159fbb990880606a6
DIFF: 
https://github.com/llvm/llvm-project/commit/4a68babd9973f043fd3e40f159fbb990880606a6.diff

LOG: [clang-doc] Add template support.

Reads template information from the AST and adds template parameters and
specialization information to the corresponding clang-doc structures.

Add a "QualName" to the Reference struct which includes the full
qualified type name. The Reference object represents a link in the
HTML/MD generators so is based on the unqualified name. But this does
not encode C-V qualifiers or template information that decorate the
name. The new QualName member encodes all of this information and also
makes it easier for the generators or downsteam YAML consumers to
generate the full name (before they had to process the "Path").

In test code that was changed, remove made-up paths to built-in types
like "int". In addition to slightnly cleaning up the code, these types
do not have paths in real execution, and generating incorrect references
to nonexistant data may complicate future changes in the generators.

Convert llvm::Optional to std::optional (YAML library requires this for
the new usage, and this makes everything consistent according to the
llvm::Optional -> std::optional transition).

Differential Revision: https://reviews.llvm.org/D139154

Added: 
clang-tools-extra/test/clang-doc/templates.cpp

Modified: 
clang-tools-extra/clang-doc/BitcodeReader.cpp
clang-tools-extra/clang-doc/BitcodeWriter.cpp
clang-tools-extra/clang-doc/BitcodeWriter.h
clang-tools-extra/clang-doc/Representation.cpp
clang-tools-extra/clang-doc/Representation.h
clang-tools-extra/clang-doc/Serialize.cpp
clang-tools-extra/clang-doc/YAMLGenerator.cpp
clang-tools-extra/test/clang-doc/single-file-public.cpp
clang-tools-extra/test/clang-doc/single-file.cpp
clang-tools-extra/unittests/clang-doc/HTMLGeneratorTest.cpp
clang-tools-extra/unittests/clang-doc/SerializeTest.cpp
clang-tools-extra/unittests/clang-doc/YAMLGeneratorTest.cpp

Removed: 




diff  --git a/clang-tools-extra/clang-doc/BitcodeReader.cpp 
b/clang-tools-extra/clang-doc/BitcodeReader.cpp
index c6928f60f3970..9ac60fa73a782 100644
--- a/clang-tools-extra/clang-doc/BitcodeReader.cpp
+++ b/clang-tools-extra/clang-doc/BitcodeReader.cpp
@@ -350,6 +350,8 @@ llvm::Error parseRecord(const Record , unsigned ID, 
llvm::StringRef Blob,
 return decodeRecord(R, I->USR, Blob);
   case REFERENCE_NAME:
 return decodeRecord(R, I->Name, Blob);
+  case REFERENCE_QUAL_NAME:
+return decodeRecord(R, I->QualName, Blob);
   case REFERENCE_TYPE:
 return decodeRecord(R, I->RefType, Blob);
   case REFERENCE_PATH:
@@ -362,6 +364,29 @@ llvm::Error parseRecord(const Record , unsigned ID, 
llvm::StringRef Blob,
   }
 }
 
+llvm::Error parseRecord(const Record , unsigned ID, llvm::StringRef Blob,
+TemplateInfo *I) {
+  // Currently there are no child records of TemplateInfo (only child blocks).
+  return llvm::createStringError(llvm::inconvertibleErrorCode(),
+ "invalid field for TemplateParamInfo");
+}
+
+llvm::Error parseRecord(const Record , unsigned ID, llvm::StringRef Blob,
+TemplateSpecializationInfo *I) {
+  if (ID == TEMPLATE_SPECIALIZATION_OF)
+return decodeRecord(R, I->SpecializationOf, Blob);
+  return llvm::createStringError(llvm::inconvertibleErrorCode(),
+ "invalid field for TemplateParamInfo");
+}
+
+llvm::Error parseRecord(const Record , unsigned ID, llvm::StringRef Blob,
+TemplateParamInfo *I) {
+  if (ID == TEMPLATE_PARAM_CONTENTS)
+return decodeRecord(R, I->Contents, Blob);
+  return llvm::createStringError(llvm::inconvertibleErrorCode(),
+ "invalid field for TemplateParamInfo");
+}
+
 template  llvm::Expected getCommentInfo(T I) {
   return llvm::createStringError(llvm::inconvertibleErrorCode(),
  "invalid type cannot contain CommentInfo");
@@ -594,6 +619,45 @@ template <> void addChild(BaseRecordInfo *I, FunctionInfo 
&) {
   I->Children.Functions.emplace_back(std::move(R));
 }
 
+// TemplateParam children. These go into either a TemplateInfo (for template
+// parameters) or TemplateSpecializationInfo (for the specialization's
+// parameters).
+template  void addTemplateParam(T I, TemplateParamInfo &) {
+  llvm::errs() << "invalid container for template parameter";
+  exit(1);
+}
+template <> void addTemplateParam(TemplateInfo *I, TemplateParamInfo &) {
+  I->Params.emplace_back(std::mov

[PATCH] D139154: [clang-doc] Add template support

[Brett Wilson via Phabricator via cfe-commits]

brettw updated this revision to Diff 480979.
brettw added a comment.

New patch with llvm::Optional converted to std::optional (previously collided 
with a refactor).


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D139154/new/

https://reviews.llvm.org/D139154

Files:
  clang-tools-extra/clang-doc/BitcodeReader.cpp
  clang-tools-extra/clang-doc/BitcodeWriter.cpp
  clang-tools-extra/clang-doc/BitcodeWriter.h
  clang-tools-extra/clang-doc/Representation.cpp
  clang-tools-extra/clang-doc/Representation.h
  clang-tools-extra/clang-doc/Serialize.cpp
  clang-tools-extra/clang-doc/YAMLGenerator.cpp
  clang-tools-extra/test/clang-doc/single-file-public.cpp
  clang-tools-extra/test/clang-doc/single-file.cpp
  clang-tools-extra/test/clang-doc/templates.cpp
  clang-tools-extra/unittests/clang-doc/HTMLGeneratorTest.cpp
  clang-tools-extra/unittests/clang-doc/SerializeTest.cpp
  clang-tools-extra/unittests/clang-doc/YAMLGeneratorTest.cpp

Index: clang-tools-extra/unittests/clang-doc/YAMLGeneratorTest.cpp
===
--- clang-tools-extra/unittests/clang-doc/YAMLGeneratorTest.cpp
+++ clang-tools-extra/unittests/clang-doc/YAMLGeneratorTest.cpp
@@ -28,10 +28,11 @@
   I.Path = "path/to/A";
   I.Namespace.emplace_back(EmptySID, "A", InfoType::IT_namespace);
 
-  I.Children.Namespaces.emplace_back(EmptySID, "ChildNamespace",
- InfoType::IT_namespace,
- "path/to/A/Namespace");
+  I.Children.Namespaces.emplace_back(
+  EmptySID, "ChildNamespace", InfoType::IT_namespace,
+  "path::to::A::Namespace::ChildNamespace", "path/to/A/Namespace");
   I.Children.Records.emplace_back(EmptySID, "ChildStruct", InfoType::IT_record,
+  "path::to::A::Namespace::ChildStruct",
   "path/to/A/Namespace");
   I.Children.Functions.emplace_back();
   I.Children.Functions.back().Name = "OneFunction";
@@ -53,13 +54,16 @@
 Namespace:
   - Type:Namespace
 Name:'A'
+QualName:'A'
 ChildNamespaces:
   - Type:Namespace
 Name:'ChildNamespace'
+QualName:'path::to::A::Namespace::ChildNamespace'
 Path:'path/to/A/Namespace'
 ChildRecords:
   - Type:Record
 Name:'ChildStruct'
+QualName:'path::to::A::Namespace::ChildStruct'
 Path:'path/to/A/Namespace'
 ChildFunctions:
   - USR: ''
@@ -83,8 +87,7 @@
   I.DefLoc = Location(10, llvm::SmallString<16>{"test.cpp"});
   I.Loc.emplace_back(12, llvm::SmallString<16>{"test.cpp"});
 
-  I.Members.emplace_back(TypeInfo("int", "path/to/int"), "X",
- AccessSpecifier::AS_private);
+  I.Members.emplace_back(TypeInfo("int"), "X", AccessSpecifier::AS_private);
 
   // Member documentation.
   CommentInfo TopComment;
@@ -103,15 +106,15 @@
AccessSpecifier::AS_public, true);
   I.Bases.back().Children.Functions.emplace_back();
   I.Bases.back().Children.Functions.back().Name = "InheritedFunctionOne";
-  I.Bases.back().Members.emplace_back(TypeInfo("int", "path/to/int"), "N",
+  I.Bases.back().Members.emplace_back(TypeInfo("int"), "N",
   AccessSpecifier::AS_private);
   // F is in the global namespace
   I.Parents.emplace_back(EmptySID, "F", InfoType::IT_record, "");
   I.VirtualParents.emplace_back(EmptySID, "G", InfoType::IT_record,
-"path/to/G");
+"path::to::G::G", "path/to/G");
 
   I.Children.Records.emplace_back(EmptySID, "ChildStruct", InfoType::IT_record,
-  "path/to/A/r");
+  "path::to::A::r::ChildStruct", "path/to/A/r");
   I.Children.Functions.emplace_back();
   I.Children.Functions.back().Name = "OneFunction";
   I.Children.Enums.emplace_back();
@@ -131,6 +134,7 @@
 Namespace:
   - Type:Namespace
 Name:'A'
+QualName:'A'
 DefLocation:
   LineNumber:  10
   Filename:'test.cpp'
@@ -142,7 +146,7 @@
 Members:
   - Type:
   Name:'int'
-  Path:'path/to/int'
+  QualName:'int'
 Name:'X'
 Access:  Private
 Description:
@@ -161,7 +165,7 @@
 Members:
   - Type:
   Name:'int'
-  Path:'path/to/int'
+  QualName:'int'
 Name:'N'
 Access:  Private
 ChildFunctions:
@@ -178,10 +182,12 @@
 VirtualParents:
   - Type:Record
 Name:'G'
+QualName:'path::to::G::G'
 Path:'path/to/G'
 ChildRecords:
   - Type:Record
 Name:'ChildStruct'
+QualName:'path::to::A::r::ChildStruct'
 Path:'path/to/A/r'
 

[clang-tools-extra] 91b38c6 - Revert "[clang-doc] Add template support."

[Brett Wilson via cfe-commits]

Author: Brett Wilson
Date: 2022-12-07T10:22:51-08:00
New Revision: 91b38c6aaddefabad2a4c959ea3865e356761ed5

URL: 
https://github.com/llvm/llvm-project/commit/91b38c6aaddefabad2a4c959ea3865e356761ed5
DIFF: 
https://github.com/llvm/llvm-project/commit/91b38c6aaddefabad2a4c959ea3865e356761ed5.diff

LOG: Revert "[clang-doc] Add template support."

Causes a build failure in YAML specializations.

This reverts commit 0f6dbb5f164662c3e6a167a89e7a89f07c60e32b.

Added: 


Modified: 
clang-tools-extra/clang-doc/BitcodeReader.cpp
clang-tools-extra/clang-doc/BitcodeWriter.cpp
clang-tools-extra/clang-doc/BitcodeWriter.h
clang-tools-extra/clang-doc/Representation.cpp
clang-tools-extra/clang-doc/Representation.h
clang-tools-extra/clang-doc/Serialize.cpp
clang-tools-extra/clang-doc/YAMLGenerator.cpp
clang-tools-extra/test/clang-doc/single-file-public.cpp
clang-tools-extra/test/clang-doc/single-file.cpp
clang-tools-extra/unittests/clang-doc/HTMLGeneratorTest.cpp
clang-tools-extra/unittests/clang-doc/SerializeTest.cpp
clang-tools-extra/unittests/clang-doc/YAMLGeneratorTest.cpp

Removed: 
clang-tools-extra/test/clang-doc/templates.cpp



diff  --git a/clang-tools-extra/clang-doc/BitcodeReader.cpp 
b/clang-tools-extra/clang-doc/BitcodeReader.cpp
index 3fb85056cee8f..072011448330e 100644
--- a/clang-tools-extra/clang-doc/BitcodeReader.cpp
+++ b/clang-tools-extra/clang-doc/BitcodeReader.cpp
@@ -351,8 +351,6 @@ llvm::Error parseRecord(const Record , unsigned ID, 
llvm::StringRef Blob,
 return decodeRecord(R, I->USR, Blob);
   case REFERENCE_NAME:
 return decodeRecord(R, I->Name, Blob);
-  case REFERENCE_QUAL_NAME:
-return decodeRecord(R, I->QualName, Blob);
   case REFERENCE_TYPE:
 return decodeRecord(R, I->RefType, Blob);
   case REFERENCE_PATH:
@@ -365,29 +363,6 @@ llvm::Error parseRecord(const Record , unsigned ID, 
llvm::StringRef Blob,
   }
 }
 
-llvm::Error parseRecord(const Record , unsigned ID, llvm::StringRef Blob,
-TemplateInfo *I) {
-  // Currently there are no child records of TemplateInfo (only child blocks).
-  return llvm::createStringError(llvm::inconvertibleErrorCode(),
- "invalid field for TemplateParamInfo");
-}
-
-llvm::Error parseRecord(const Record , unsigned ID, llvm::StringRef Blob,
-TemplateSpecializationInfo *I) {
-  if (ID == TEMPLATE_SPECIALIZATION_OF)
-return decodeRecord(R, I->SpecializationOf, Blob);
-  return llvm::createStringError(llvm::inconvertibleErrorCode(),
- "invalid field for TemplateParamInfo");
-}
-
-llvm::Error parseRecord(const Record , unsigned ID, llvm::StringRef Blob,
-TemplateParamInfo *I) {
-  if (ID == TEMPLATE_PARAM_CONTENTS)
-return decodeRecord(R, I->Contents, Blob);
-  return llvm::createStringError(llvm::inconvertibleErrorCode(),
- "invalid field for TemplateParamInfo");
-}
-
 template  llvm::Expected getCommentInfo(T I) {
   return llvm::createStringError(llvm::inconvertibleErrorCode(),
  "invalid type cannot contain CommentInfo");
@@ -620,45 +595,6 @@ template <> void addChild(BaseRecordInfo *I, FunctionInfo 
&) {
   I->Children.Functions.emplace_back(std::move(R));
 }
 
-// TemplateParam children. These go into either a TemplateInfo (for template
-// parameters) or TemplateSpecializationInfo (for the specialization's
-// parameters).
-template  void addTemplateParam(T I, TemplateParamInfo &) {
-  llvm::errs() << "invalid container for template parameter";
-  exit(1);
-}
-template <> void addTemplateParam(TemplateInfo *I, TemplateParamInfo &) {
-  I->Params.emplace_back(std::move(P));
-}
-template <>
-void addTemplateParam(TemplateSpecializationInfo *I, TemplateParamInfo &) {
-  I->Params.emplace_back(std::move(P));
-}
-
-// Template info. These apply to either records or functions.
-template  void addTemplate(T I, TemplateInfo &) {
-  llvm::errs() << "invalid container for template info";
-  exit(1);
-}
-template <> void addTemplate(RecordInfo *I, TemplateInfo &) {
-  I->Template.emplace(std::move(P));
-}
-template <> void addTemplate(FunctionInfo *I, TemplateInfo &) {
-  I->Template.emplace(std::move(P));
-}
-
-// Template specializations go only into template records.
-template 
-void addTemplateSpecialization(T I, TemplateSpecializationInfo &) {
-  llvm::errs() << "invalid container for template specialization info";
-  exit(1);
-}
-template <>
-void addTemplateSpecialization(TemplateInfo *I,
-   TemplateSpecializationInfo &) {
-  I->Specialization.emplace(std::move(TSI));
-}
-
 /

[PATCH] D139154: [clang-doc] Add template support

[Brett Wilson via Phabricator via cfe-commits]

This revision was automatically updated to reflect the committed changes.
Closed by commit rG0f6dbb5f1646: [clang-doc] Add template support. (authored by 
brettw).

Changed prior to commit:
  https://reviews.llvm.org/D139154?vs=480560=480960#toc

Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D139154/new/

https://reviews.llvm.org/D139154

Files:
  clang-tools-extra/clang-doc/BitcodeReader.cpp
  clang-tools-extra/clang-doc/BitcodeWriter.cpp
  clang-tools-extra/clang-doc/BitcodeWriter.h
  clang-tools-extra/clang-doc/Representation.cpp
  clang-tools-extra/clang-doc/Representation.h
  clang-tools-extra/clang-doc/Serialize.cpp
  clang-tools-extra/clang-doc/YAMLGenerator.cpp
  clang-tools-extra/test/clang-doc/single-file-public.cpp
  clang-tools-extra/test/clang-doc/single-file.cpp
  clang-tools-extra/test/clang-doc/templates.cpp
  clang-tools-extra/unittests/clang-doc/HTMLGeneratorTest.cpp
  clang-tools-extra/unittests/clang-doc/SerializeTest.cpp
  clang-tools-extra/unittests/clang-doc/YAMLGeneratorTest.cpp

Index: clang-tools-extra/unittests/clang-doc/YAMLGeneratorTest.cpp
===
--- clang-tools-extra/unittests/clang-doc/YAMLGeneratorTest.cpp
+++ clang-tools-extra/unittests/clang-doc/YAMLGeneratorTest.cpp
@@ -28,10 +28,11 @@
   I.Path = "path/to/A";
   I.Namespace.emplace_back(EmptySID, "A", InfoType::IT_namespace);
 
-  I.Children.Namespaces.emplace_back(EmptySID, "ChildNamespace",
- InfoType::IT_namespace,
- "path/to/A/Namespace");
+  I.Children.Namespaces.emplace_back(
+  EmptySID, "ChildNamespace", InfoType::IT_namespace,
+  "path::to::A::Namespace::ChildNamespace", "path/to/A/Namespace");
   I.Children.Records.emplace_back(EmptySID, "ChildStruct", InfoType::IT_record,
+  "path::to::A::Namespace::ChildStruct",
   "path/to/A/Namespace");
   I.Children.Functions.emplace_back();
   I.Children.Functions.back().Name = "OneFunction";
@@ -53,13 +54,16 @@
 Namespace:
   - Type:Namespace
 Name:'A'
+QualName:'A'
 ChildNamespaces:
   - Type:Namespace
 Name:'ChildNamespace'
+QualName:'path::to::A::Namespace::ChildNamespace'
 Path:'path/to/A/Namespace'
 ChildRecords:
   - Type:Record
 Name:'ChildStruct'
+QualName:'path::to::A::Namespace::ChildStruct'
 Path:'path/to/A/Namespace'
 ChildFunctions:
   - USR: ''
@@ -83,8 +87,7 @@
   I.DefLoc = Location(10, llvm::SmallString<16>{"test.cpp"});
   I.Loc.emplace_back(12, llvm::SmallString<16>{"test.cpp"});
 
-  I.Members.emplace_back(TypeInfo("int", "path/to/int"), "X",
- AccessSpecifier::AS_private);
+  I.Members.emplace_back(TypeInfo("int"), "X", AccessSpecifier::AS_private);
 
   // Member documentation.
   CommentInfo TopComment;
@@ -103,15 +106,15 @@
AccessSpecifier::AS_public, true);
   I.Bases.back().Children.Functions.emplace_back();
   I.Bases.back().Children.Functions.back().Name = "InheritedFunctionOne";
-  I.Bases.back().Members.emplace_back(TypeInfo("int", "path/to/int"), "N",
+  I.Bases.back().Members.emplace_back(TypeInfo("int"), "N",
   AccessSpecifier::AS_private);
   // F is in the global namespace
   I.Parents.emplace_back(EmptySID, "F", InfoType::IT_record, "");
   I.VirtualParents.emplace_back(EmptySID, "G", InfoType::IT_record,
-"path/to/G");
+"path::to::G::G", "path/to/G");
 
   I.Children.Records.emplace_back(EmptySID, "ChildStruct", InfoType::IT_record,
-  "path/to/A/r");
+  "path::to::A::r::ChildStruct", "path/to/A/r");
   I.Children.Functions.emplace_back();
   I.Children.Functions.back().Name = "OneFunction";
   I.Children.Enums.emplace_back();
@@ -131,6 +134,7 @@
 Namespace:
   - Type:Namespace
 Name:'A'
+QualName:'A'
 DefLocation:
   LineNumber:  10
   Filename:'test.cpp'
@@ -142,7 +146,7 @@
 Members:
   - Type:
   Name:'int'
-  Path:'path/to/int'
+  QualName:'int'
 Name:'X'
 Access:  Private
 Description:
@@ -161,7 +165,7 @@
 Members:
   - Type:
   Name:'int'
-  Path:'path/to/int'
+  QualName:'int'
 Name:'N'
 Access:  Private
 ChildFunctions:
@@ -178,10 +182,12 @@
 VirtualParents:
   - Type:Record
 Name:'G'
+QualName:'path::to::G::G'
 Path:'path/to/G'
 ChildRecords:
   - Type:Record
 Name:  

[clang-tools-extra] 0f6dbb5 - [clang-doc] Add template support.

[Brett Wilson via cfe-commits]

Author: Brett Wilson
Date: 2022-12-07T09:48:13-08:00
New Revision: 0f6dbb5f164662c3e6a167a89e7a89f07c60e32b

URL: 
https://github.com/llvm/llvm-project/commit/0f6dbb5f164662c3e6a167a89e7a89f07c60e32b
DIFF: 
https://github.com/llvm/llvm-project/commit/0f6dbb5f164662c3e6a167a89e7a89f07c60e32b.diff

LOG: [clang-doc] Add template support.

Reads template information from the AST and adds template parameters and
specialization information to the corresponding clang-doc structures.

Add a "QualName" to the Reference struct which includes the full
qualified type name. The Reference object represents a link in the
HTML/MD generators so is based on the unqualified name. But this does
not encode C-V qualifiers or template information that decorate the
name. The new QualName member encodes all of this information and also
makes it easier for the generators or downsteam YAML consumers to
generate the full name (before they had to process the "Path").

In test code that was changed, remove made-up paths to built-in types
like "int". In addition to slightnly cleaning up the code, these types
do not have paths in real execution, and generating incorrect references
to nonexistant data may complicate future changes in the generators.

Differential Revision: https://reviews.llvm.org/D139154

Added: 
clang-tools-extra/test/clang-doc/templates.cpp

Modified: 
clang-tools-extra/clang-doc/BitcodeReader.cpp
clang-tools-extra/clang-doc/BitcodeWriter.cpp
clang-tools-extra/clang-doc/BitcodeWriter.h
clang-tools-extra/clang-doc/Representation.cpp
clang-tools-extra/clang-doc/Representation.h
clang-tools-extra/clang-doc/Serialize.cpp
clang-tools-extra/clang-doc/YAMLGenerator.cpp
clang-tools-extra/test/clang-doc/single-file-public.cpp
clang-tools-extra/test/clang-doc/single-file.cpp
clang-tools-extra/unittests/clang-doc/HTMLGeneratorTest.cpp
clang-tools-extra/unittests/clang-doc/SerializeTest.cpp
clang-tools-extra/unittests/clang-doc/YAMLGeneratorTest.cpp

Removed: 




diff  --git a/clang-tools-extra/clang-doc/BitcodeReader.cpp 
b/clang-tools-extra/clang-doc/BitcodeReader.cpp
index 072011448330e..3fb85056cee8f 100644
--- a/clang-tools-extra/clang-doc/BitcodeReader.cpp
+++ b/clang-tools-extra/clang-doc/BitcodeReader.cpp
@@ -351,6 +351,8 @@ llvm::Error parseRecord(const Record , unsigned ID, 
llvm::StringRef Blob,
 return decodeRecord(R, I->USR, Blob);
   case REFERENCE_NAME:
 return decodeRecord(R, I->Name, Blob);
+  case REFERENCE_QUAL_NAME:
+return decodeRecord(R, I->QualName, Blob);
   case REFERENCE_TYPE:
 return decodeRecord(R, I->RefType, Blob);
   case REFERENCE_PATH:
@@ -363,6 +365,29 @@ llvm::Error parseRecord(const Record , unsigned ID, 
llvm::StringRef Blob,
   }
 }
 
+llvm::Error parseRecord(const Record , unsigned ID, llvm::StringRef Blob,
+TemplateInfo *I) {
+  // Currently there are no child records of TemplateInfo (only child blocks).
+  return llvm::createStringError(llvm::inconvertibleErrorCode(),
+ "invalid field for TemplateParamInfo");
+}
+
+llvm::Error parseRecord(const Record , unsigned ID, llvm::StringRef Blob,
+TemplateSpecializationInfo *I) {
+  if (ID == TEMPLATE_SPECIALIZATION_OF)
+return decodeRecord(R, I->SpecializationOf, Blob);
+  return llvm::createStringError(llvm::inconvertibleErrorCode(),
+ "invalid field for TemplateParamInfo");
+}
+
+llvm::Error parseRecord(const Record , unsigned ID, llvm::StringRef Blob,
+TemplateParamInfo *I) {
+  if (ID == TEMPLATE_PARAM_CONTENTS)
+return decodeRecord(R, I->Contents, Blob);
+  return llvm::createStringError(llvm::inconvertibleErrorCode(),
+ "invalid field for TemplateParamInfo");
+}
+
 template  llvm::Expected getCommentInfo(T I) {
   return llvm::createStringError(llvm::inconvertibleErrorCode(),
  "invalid type cannot contain CommentInfo");
@@ -595,6 +620,45 @@ template <> void addChild(BaseRecordInfo *I, FunctionInfo 
&) {
   I->Children.Functions.emplace_back(std::move(R));
 }
 
+// TemplateParam children. These go into either a TemplateInfo (for template
+// parameters) or TemplateSpecializationInfo (for the specialization's
+// parameters).
+template  void addTemplateParam(T I, TemplateParamInfo &) {
+  llvm::errs() << "invalid container for template parameter";
+  exit(1);
+}
+template <> void addTemplateParam(TemplateInfo *I, TemplateParamInfo &) {
+  I->Params.emplace_back(std::move(P));
+}
+template <>
+void addTemplateParam(TemplateSpecializationInfo *I, TemplateParamInfo &) {
+  I->Params.emplace_back(std::move(P));
+}
+
+// Template info. The

[PATCH] D139154: [clang-doc] Add template support

[Brett Wilson via Phabricator via cfe-commits]

brettw added inline comments.



Comment at: clang-tools-extra/clang-doc/Representation.h:119-123
+  // This variant (that takes no qualified name parameter) uses the Name as the
+  // QualName (very useful in unit tests to reduce verbosity). This can't use
+  // the empty string as a default because the global namespace uses
+  // "GlobalNamespace" as the name, but should have an empty QualName
+  // (corresponding to the name in code).

paulkirth wrote:
> Does this imply that its illegal to use `""` as the name with this 
> constructor? if so, it probably requires at least an assert.
No, I meant something else, clarified.



Comment at: clang-tools-extra/clang-doc/Representation.h:210-212
+  // The literal contents of the code for everything that specifies this
+  // template parameter. This will include all components, so for a template
+  // declaration it could be "typename T = int".

paulkirth wrote:
> I think this is equivalent. Do you mean this contains //all// instantiations 
> of the parameter across the codebase? or for a single declaration?
Clarified


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D139154/new/

https://reviews.llvm.org/D139154

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D139154: [clang-doc] Add template support

[Brett Wilson via Phabricator via cfe-commits]

brettw updated this revision to Diff 480560.
brettw marked 5 inline comments as done.

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D139154/new/

https://reviews.llvm.org/D139154

Files:
  clang-tools-extra/clang-doc/BitcodeReader.cpp
  clang-tools-extra/clang-doc/BitcodeWriter.cpp
  clang-tools-extra/clang-doc/BitcodeWriter.h
  clang-tools-extra/clang-doc/Representation.cpp
  clang-tools-extra/clang-doc/Representation.h
  clang-tools-extra/clang-doc/Serialize.cpp
  clang-tools-extra/clang-doc/YAMLGenerator.cpp
  clang-tools-extra/test/clang-doc/single-file-public.cpp
  clang-tools-extra/test/clang-doc/single-file.cpp
  clang-tools-extra/test/clang-doc/templates.cpp
  clang-tools-extra/unittests/clang-doc/HTMLGeneratorTest.cpp
  clang-tools-extra/unittests/clang-doc/SerializeTest.cpp
  clang-tools-extra/unittests/clang-doc/YAMLGeneratorTest.cpp

Index: clang-tools-extra/unittests/clang-doc/YAMLGeneratorTest.cpp
===
--- clang-tools-extra/unittests/clang-doc/YAMLGeneratorTest.cpp
+++ clang-tools-extra/unittests/clang-doc/YAMLGeneratorTest.cpp
@@ -28,10 +28,11 @@
   I.Path = "path/to/A";
   I.Namespace.emplace_back(EmptySID, "A", InfoType::IT_namespace);
 
-  I.Children.Namespaces.emplace_back(EmptySID, "ChildNamespace",
- InfoType::IT_namespace,
- "path/to/A/Namespace");
+  I.Children.Namespaces.emplace_back(
+  EmptySID, "ChildNamespace", InfoType::IT_namespace,
+  "path::to::A::Namespace::ChildNamespace", "path/to/A/Namespace");
   I.Children.Records.emplace_back(EmptySID, "ChildStruct", InfoType::IT_record,
+  "path::to::A::Namespace::ChildStruct",
   "path/to/A/Namespace");
   I.Children.Functions.emplace_back();
   I.Children.Functions.back().Name = "OneFunction";
@@ -53,13 +54,16 @@
 Namespace:
   - Type:Namespace
 Name:'A'
+QualName:'A'
 ChildNamespaces:
   - Type:Namespace
 Name:'ChildNamespace'
+QualName:'path::to::A::Namespace::ChildNamespace'
 Path:'path/to/A/Namespace'
 ChildRecords:
   - Type:Record
 Name:'ChildStruct'
+QualName:'path::to::A::Namespace::ChildStruct'
 Path:'path/to/A/Namespace'
 ChildFunctions:
   - USR: ''
@@ -83,8 +87,7 @@
   I.DefLoc = Location(10, llvm::SmallString<16>{"test.cpp"});
   I.Loc.emplace_back(12, llvm::SmallString<16>{"test.cpp"});
 
-  I.Members.emplace_back(TypeInfo("int", "path/to/int"), "X",
- AccessSpecifier::AS_private);
+  I.Members.emplace_back(TypeInfo("int"), "X", AccessSpecifier::AS_private);
 
   // Member documentation.
   CommentInfo TopComment;
@@ -103,15 +106,15 @@
AccessSpecifier::AS_public, true);
   I.Bases.back().Children.Functions.emplace_back();
   I.Bases.back().Children.Functions.back().Name = "InheritedFunctionOne";
-  I.Bases.back().Members.emplace_back(TypeInfo("int", "path/to/int"), "N",
+  I.Bases.back().Members.emplace_back(TypeInfo("int"), "N",
   AccessSpecifier::AS_private);
   // F is in the global namespace
   I.Parents.emplace_back(EmptySID, "F", InfoType::IT_record, "");
   I.VirtualParents.emplace_back(EmptySID, "G", InfoType::IT_record,
-"path/to/G");
+"path::to::G::G", "path/to/G");
 
   I.Children.Records.emplace_back(EmptySID, "ChildStruct", InfoType::IT_record,
-  "path/to/A/r");
+  "path::to::A::r::ChildStruct", "path/to/A/r");
   I.Children.Functions.emplace_back();
   I.Children.Functions.back().Name = "OneFunction";
   I.Children.Enums.emplace_back();
@@ -131,6 +134,7 @@
 Namespace:
   - Type:Namespace
 Name:'A'
+QualName:'A'
 DefLocation:
   LineNumber:  10
   Filename:'test.cpp'
@@ -142,7 +146,7 @@
 Members:
   - Type:
   Name:'int'
-  Path:'path/to/int'
+  QualName:'int'
 Name:'X'
 Access:  Private
 Description:
@@ -161,7 +165,7 @@
 Members:
   - Type:
   Name:'int'
-  Path:'path/to/int'
+  QualName:'int'
 Name:'N'
 Access:  Private
 ChildFunctions:
@@ -178,10 +182,12 @@
 VirtualParents:
   - Type:Record
 Name:'G'
+QualName:'path::to::G::G'
 Path:'path/to/G'
 ChildRecords:
   - Type:Record
 Name:'ChildStruct'
+QualName:'path::to::A::r::ChildStruct'
 Path:'path/to/A/r'
 ChildFunctions:
   - USR: ''
@@ 

[PATCH] D139154: [clang-doc] Add template support

[Brett Wilson via Phabricator via cfe-commits]

brettw marked an inline comment as done.
brettw added a comment.

New patch up.




Comment at: clang-tools-extra/test/clang-doc/single-file.cpp:19
 // CHECK-NEXT:DefLocation:
-// CHECK-NEXT:  LineNumber:  [[@LINE-8]]
+// CHECK-NEXT:  LineNumber:  11
 // CHECK-NEXT:  Filename:'{{.*}}

I changed the existing test from relative line numbers which don't make sense 
given the structure of this (where the code is mostly static at the top, and 
the YAML is at the bottom which occasionally changes).



Comment at: clang-tools-extra/unittests/clang-doc/SerializeTest.cpp:631
 
+TEST(SerializeTests, emitFunctionTemplate) {
+  EmittedInfoList Infos;

paulkirth wrote:
> I may have missed these cases in the test file, so if they exist just ignore 
> me.
> 
> -  a template w/ no concrete types e.g., `template void 
> GetFoo(T);` This seems like a very common case, so it would be good to have a 
> test explicitly for it. 
> - a nested template, both where the inner template type is known, and where 
> it is not.
> 
I added the nested template example to the unit test. I also added a template 
parameter pack to the e2e test.


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D139154/new/

https://reviews.llvm.org/D139154

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D139154: [clang-doc] Add template support

[Brett Wilson via Phabricator via cfe-commits]

brettw updated this revision to Diff 480536.
brettw marked an inline comment as done.

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D139154/new/

https://reviews.llvm.org/D139154

Files:
  clang-tools-extra/clang-doc/BitcodeReader.cpp
  clang-tools-extra/clang-doc/BitcodeWriter.cpp
  clang-tools-extra/clang-doc/BitcodeWriter.h
  clang-tools-extra/clang-doc/Representation.cpp
  clang-tools-extra/clang-doc/Representation.h
  clang-tools-extra/clang-doc/Serialize.cpp
  clang-tools-extra/clang-doc/YAMLGenerator.cpp
  clang-tools-extra/test/clang-doc/single-file-public.cpp
  clang-tools-extra/test/clang-doc/single-file.cpp
  clang-tools-extra/test/clang-doc/templates.cpp
  clang-tools-extra/unittests/clang-doc/HTMLGeneratorTest.cpp
  clang-tools-extra/unittests/clang-doc/SerializeTest.cpp
  clang-tools-extra/unittests/clang-doc/YAMLGeneratorTest.cpp

Index: clang-tools-extra/unittests/clang-doc/YAMLGeneratorTest.cpp
===
--- clang-tools-extra/unittests/clang-doc/YAMLGeneratorTest.cpp
+++ clang-tools-extra/unittests/clang-doc/YAMLGeneratorTest.cpp
@@ -28,10 +28,11 @@
   I.Path = "path/to/A";
   I.Namespace.emplace_back(EmptySID, "A", InfoType::IT_namespace);
 
-  I.Children.Namespaces.emplace_back(EmptySID, "ChildNamespace",
- InfoType::IT_namespace,
- "path/to/A/Namespace");
+  I.Children.Namespaces.emplace_back(
+  EmptySID, "ChildNamespace", InfoType::IT_namespace,
+  "path::to::A::Namespace::ChildNamespace", "path/to/A/Namespace");
   I.Children.Records.emplace_back(EmptySID, "ChildStruct", InfoType::IT_record,
+  "path::to::A::Namespace::ChildStruct",
   "path/to/A/Namespace");
   I.Children.Functions.emplace_back();
   I.Children.Functions.back().Name = "OneFunction";
@@ -53,13 +54,16 @@
 Namespace:
   - Type:Namespace
 Name:'A'
+QualName:'A'
 ChildNamespaces:
   - Type:Namespace
 Name:'ChildNamespace'
+QualName:'path::to::A::Namespace::ChildNamespace'
 Path:'path/to/A/Namespace'
 ChildRecords:
   - Type:Record
 Name:'ChildStruct'
+QualName:'path::to::A::Namespace::ChildStruct'
 Path:'path/to/A/Namespace'
 ChildFunctions:
   - USR: ''
@@ -83,8 +87,7 @@
   I.DefLoc = Location(10, llvm::SmallString<16>{"test.cpp"});
   I.Loc.emplace_back(12, llvm::SmallString<16>{"test.cpp"});
 
-  I.Members.emplace_back(TypeInfo("int", "path/to/int"), "X",
- AccessSpecifier::AS_private);
+  I.Members.emplace_back(TypeInfo("int"), "X", AccessSpecifier::AS_private);
 
   // Member documentation.
   CommentInfo TopComment;
@@ -103,15 +106,15 @@
AccessSpecifier::AS_public, true);
   I.Bases.back().Children.Functions.emplace_back();
   I.Bases.back().Children.Functions.back().Name = "InheritedFunctionOne";
-  I.Bases.back().Members.emplace_back(TypeInfo("int", "path/to/int"), "N",
+  I.Bases.back().Members.emplace_back(TypeInfo("int"), "N",
   AccessSpecifier::AS_private);
   // F is in the global namespace
   I.Parents.emplace_back(EmptySID, "F", InfoType::IT_record, "");
   I.VirtualParents.emplace_back(EmptySID, "G", InfoType::IT_record,
-"path/to/G");
+"path::to::G::G", "path/to/G");
 
   I.Children.Records.emplace_back(EmptySID, "ChildStruct", InfoType::IT_record,
-  "path/to/A/r");
+  "path::to::A::r::ChildStruct", "path/to/A/r");
   I.Children.Functions.emplace_back();
   I.Children.Functions.back().Name = "OneFunction";
   I.Children.Enums.emplace_back();
@@ -131,6 +134,7 @@
 Namespace:
   - Type:Namespace
 Name:'A'
+QualName:'A'
 DefLocation:
   LineNumber:  10
   Filename:'test.cpp'
@@ -142,7 +146,7 @@
 Members:
   - Type:
   Name:'int'
-  Path:'path/to/int'
+  QualName:'int'
 Name:'X'
 Access:  Private
 Description:
@@ -161,7 +165,7 @@
 Members:
   - Type:
   Name:'int'
-  Path:'path/to/int'
+  QualName:'int'
 Name:'N'
 Access:  Private
 ChildFunctions:
@@ -178,10 +182,12 @@
 VirtualParents:
   - Type:Record
 Name:'G'
+QualName:'path::to::G::G'
 Path:'path/to/G'
 ChildRecords:
   - Type:Record
 Name:'ChildStruct'
+QualName:'path::to::A::r::ChildStruct'
 Path:'path/to/A/r'
 ChildFunctions:
   - USR: ''
@@ 

[PATCH] D139154: [clang-doc] Add template support

[Brett Wilson via Phabricator via cfe-commits]

brettw updated this revision to Diff 480251.
brettw added a reviewer: paulkirth.

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D139154/new/

https://reviews.llvm.org/D139154

Files:
  clang-tools-extra/clang-doc/BitcodeReader.cpp
  clang-tools-extra/clang-doc/BitcodeWriter.cpp
  clang-tools-extra/clang-doc/BitcodeWriter.h
  clang-tools-extra/clang-doc/Representation.cpp
  clang-tools-extra/clang-doc/Representation.h
  clang-tools-extra/clang-doc/Serialize.cpp
  clang-tools-extra/clang-doc/YAMLGenerator.cpp
  clang-tools-extra/unittests/clang-doc/HTMLGeneratorTest.cpp
  clang-tools-extra/unittests/clang-doc/SerializeTest.cpp
  clang-tools-extra/unittests/clang-doc/YAMLGeneratorTest.cpp

Index: clang-tools-extra/unittests/clang-doc/YAMLGeneratorTest.cpp
===
--- clang-tools-extra/unittests/clang-doc/YAMLGeneratorTest.cpp
+++ clang-tools-extra/unittests/clang-doc/YAMLGeneratorTest.cpp
@@ -28,10 +28,11 @@
   I.Path = "path/to/A";
   I.Namespace.emplace_back(EmptySID, "A", InfoType::IT_namespace);
 
-  I.Children.Namespaces.emplace_back(EmptySID, "ChildNamespace",
- InfoType::IT_namespace,
- "path/to/A/Namespace");
+  I.Children.Namespaces.emplace_back(
+  EmptySID, "ChildNamespace", InfoType::IT_namespace,
+  "path::to::A::Namespace::ChildNamespace", "path/to/A/Namespace");
   I.Children.Records.emplace_back(EmptySID, "ChildStruct", InfoType::IT_record,
+  "path::to::A::Namespace::ChildStruct",
   "path/to/A/Namespace");
   I.Children.Functions.emplace_back();
   I.Children.Functions.back().Name = "OneFunction";
@@ -53,13 +54,16 @@
 Namespace:
   - Type:Namespace
 Name:'A'
+QualName:'A'
 ChildNamespaces:
   - Type:Namespace
 Name:'ChildNamespace'
+QualName:'path::to::A::Namespace::ChildNamespace'
 Path:'path/to/A/Namespace'
 ChildRecords:
   - Type:Record
 Name:'ChildStruct'
+QualName:'path::to::A::Namespace::ChildStruct'
 Path:'path/to/A/Namespace'
 ChildFunctions:
   - USR: ''
@@ -83,8 +87,7 @@
   I.DefLoc = Location(10, llvm::SmallString<16>{"test.cpp"});
   I.Loc.emplace_back(12, llvm::SmallString<16>{"test.cpp"});
 
-  I.Members.emplace_back(TypeInfo("int", "path/to/int"), "X",
- AccessSpecifier::AS_private);
+  I.Members.emplace_back(TypeInfo("int"), "X", AccessSpecifier::AS_private);
 
   // Member documentation.
   CommentInfo TopComment;
@@ -103,15 +106,15 @@
AccessSpecifier::AS_public, true);
   I.Bases.back().Children.Functions.emplace_back();
   I.Bases.back().Children.Functions.back().Name = "InheritedFunctionOne";
-  I.Bases.back().Members.emplace_back(TypeInfo("int", "path/to/int"), "N",
+  I.Bases.back().Members.emplace_back(TypeInfo("int"), "N",
   AccessSpecifier::AS_private);
   // F is in the global namespace
   I.Parents.emplace_back(EmptySID, "F", InfoType::IT_record, "");
   I.VirtualParents.emplace_back(EmptySID, "G", InfoType::IT_record,
-"path/to/G");
+"path::to::G::G", "path/to/G");
 
   I.Children.Records.emplace_back(EmptySID, "ChildStruct", InfoType::IT_record,
-  "path/to/A/r");
+  "path::to::A::r::ChildStruct", "path/to/A/r");
   I.Children.Functions.emplace_back();
   I.Children.Functions.back().Name = "OneFunction";
   I.Children.Enums.emplace_back();
@@ -131,6 +134,7 @@
 Namespace:
   - Type:Namespace
 Name:'A'
+QualName:'A'
 DefLocation:
   LineNumber:  10
   Filename:'test.cpp'
@@ -142,7 +146,7 @@
 Members:
   - Type:
   Name:'int'
-  Path:'path/to/int'
+  QualName:'int'
 Name:'X'
 Access:  Private
 Description:
@@ -161,7 +165,7 @@
 Members:
   - Type:
   Name:'int'
-  Path:'path/to/int'
+  QualName:'int'
 Name:'N'
 Access:  Private
 ChildFunctions:
@@ -178,10 +182,12 @@
 VirtualParents:
   - Type:Record
 Name:'G'
+QualName:'path::to::G::G'
 Path:'path/to/G'
 ChildRecords:
   - Type:Record
 Name:'ChildStruct'
+QualName:'path::to::A::r::ChildStruct'
 Path:'path/to/A/r'
 ChildFunctions:
   - USR: ''
@@ -206,10 +212,9 @@
 
   I.Access = AccessSpecifier::AS_none;
 
-  I.ReturnType = TypeInfo(
-  Reference(EmptySID, "void", InfoType::IT_default, "path/to/void"));
-  

[PATCH] D139154: [clang-doc] Add template support

[Brett Wilson via Phabricator via cfe-commits]

brettw updated this revision to Diff 479772.

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D139154/new/

https://reviews.llvm.org/D139154

Files:
  clang-tools-extra/clang-doc/BitcodeReader.cpp
  clang-tools-extra/clang-doc/BitcodeWriter.cpp
  clang-tools-extra/clang-doc/BitcodeWriter.h
  clang-tools-extra/clang-doc/Representation.cpp
  clang-tools-extra/clang-doc/Representation.h
  clang-tools-extra/clang-doc/Serialize.cpp
  clang-tools-extra/clang-doc/YAMLGenerator.cpp

Index: clang-tools-extra/clang-doc/YAMLGenerator.cpp
===
--- clang-tools-extra/clang-doc/YAMLGenerator.cpp
+++ clang-tools-extra/clang-doc/YAMLGenerator.cpp
@@ -9,6 +9,7 @@
 //===--===//
 
 #include "Generators.h"
+#include "Representation.h"
 #include "llvm/Support/YAMLTraits.h"
 #include "llvm/Support/raw_ostream.h"
 
@@ -23,6 +24,7 @@
 LLVM_YAML_IS_SEQUENCE_VECTOR(FunctionInfo)
 LLVM_YAML_IS_SEQUENCE_VECTOR(EnumInfo)
 LLVM_YAML_IS_SEQUENCE_VECTOR(EnumValueInfo)
+LLVM_YAML_IS_SEQUENCE_VECTOR(TemplateParamInfo)
 LLVM_YAML_IS_SEQUENCE_VECTOR(TypedefInfo)
 LLVM_YAML_IS_SEQUENCE_VECTOR(BaseRecordInfo)
 LLVM_YAML_IS_SEQUENCE_VECTOR(std::unique_ptr)
@@ -142,6 +144,7 @@
   IO.mapOptional("ChildFunctions", I.Children.Functions);
   IO.mapOptional("ChildEnums", I.Children.Enums);
   IO.mapOptional("ChildTypedefs", I.Children.Typedefs);
+  IO.mapOptional("Template", I.Template);
 }
 
 static void CommentInfoMapping(IO , CommentInfo ) {
@@ -174,6 +177,7 @@
   static void mapping(IO , Reference ) {
 IO.mapOptional("Type", Ref.RefType, InfoType::IT_default);
 IO.mapOptional("Name", Ref.Name, SmallString<16>());
+IO.mapOptional("QualName", Ref.QualName, SmallString<16>());
 IO.mapOptional("USR", Ref.USR, SymbolID());
 IO.mapOptional("Path", Ref.Path, SmallString<128>());
   }
@@ -267,6 +271,28 @@
 // the AS that shouldn't be part of the output. Even though AS_public is the
 // default in the struct, it should be displayed in the YAML output.
 IO.mapOptional("Access", I.Access, clang::AccessSpecifier::AS_none);
+IO.mapOptional("Template", I.Template);
+  }
+};
+
+template <> struct MappingTraits {
+  static void mapping(IO , TemplateParamInfo ) {
+IO.mapOptional("Contents", I.Contents);
+  }
+};
+
+template <> struct MappingTraits {
+  static void mapping(IO , TemplateSpecializationInfo ) {
+IO.mapOptional("SpecializationOf", I.SpecializationOf);
+IO.mapOptional("Params", I.Params);
+  }
+};
+
+template <> struct MappingTraits {
+  static void mapping(IO , TemplateInfo ) {
+IO.mapOptional("Params", I.Params);
+IO.mapOptional("Specialization", I.Specialization,
+   Optional());
   }
 };
 
Index: clang-tools-extra/clang-doc/Serialize.cpp
===
--- clang-tools-extra/clang-doc/Serialize.cpp
+++ clang-tools-extra/clang-doc/Serialize.cpp
@@ -239,7 +239,7 @@
 TypeInfo getTypeInfoForType(const QualType ) {
   const TagDecl *TD = getTagDeclForType(T);
   if (!TD)
-return TypeInfo(Reference(SymbolID(), T.getAsString()));
+return TypeInfo(Reference(SymbolID(), T.getAsString(), T.getAsString()));
 
   InfoType IT;
   if (dyn_cast(TD)) {
@@ -249,7 +249,7 @@
   } else {
 IT = InfoType::IT_default;
   }
-  return TypeInfo(Reference(getUSRForDecl(TD), TD->getNameAsString(), IT,
+  return TypeInfo(Reference(getUSRForDecl(TD), TD->getNameAsString(), T.getAsString(), IT,
 getInfoRelativePath(TD)));
 }
 
@@ -280,12 +280,12 @@
 //
 // See MakeAndInsertIntoParent().
 static void InsertChild(ScopeChildren , const NamespaceInfo ) {
-  Scope.Namespaces.emplace_back(Info.USR, Info.Name, InfoType::IT_namespace,
+  Scope.Namespaces.emplace_back(Info.USR, Info.Name, Info.Name, InfoType::IT_namespace,
 getInfoRelativePath(Info.Namespace));
 }
 
 static void InsertChild(ScopeChildren , const RecordInfo ) {
-  Scope.Records.emplace_back(Info.USR, Info.Name, InfoType::IT_record,
+  Scope.Records.emplace_back(Info.USR, Info.Name, Info.Name, InfoType::IT_record,
  getInfoRelativePath(Info.Namespace));
 }
 
@@ -405,10 +405,7 @@
   for (const ParmVarDecl *P : D->parameters()) {
 FieldTypeInfo  = I.Params.emplace_back(
 getTypeInfoForType(P->getOriginalType()), P->getNameAsString());
-
-if (const Expr *DefaultArg = P->getDefaultArg()) {
-  FieldInfo.DefaultValue = getSourceCode(D, DefaultArg->getSourceRange());
-}
+FieldInfo.DefaultValue = getSourceCode(D, P->getDefaultArgRange());
   }
 }
 
@@ -424,9 +421,10 @@
 if (const auto *Ty = B.getType()->getAs()) {
   const TemplateDecl *D = Ty->getTemplateName().getAsTemplateDecl();
   I.Parents.emplace_back(getUSRForDecl(D), B.getType().getAsString(),
+  B.getType().getAsString(),
  

[PATCH] D139154: [clang-doc] Add template support

[Brett Wilson via Phabricator via cfe-commits]

brettw created this revision.
brettw added a project: clang-tools-extra.
Herald added a project: All.
brettw requested review of this revision.
Herald added a subscriber: cfe-commits.

Decodes the template parameters for both functions and records.

For template specializations, records a reference to the main record that the 
specialization is of, and also the parameters that the code supplied to that 
template.


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D139154

Files:
  clang-tools-extra/clang-doc/BitcodeReader.cpp
  clang-tools-extra/clang-doc/BitcodeWriter.cpp
  clang-tools-extra/clang-doc/BitcodeWriter.h
  clang-tools-extra/clang-doc/Representation.cpp
  clang-tools-extra/clang-doc/Representation.h
  clang-tools-extra/clang-doc/Serialize.cpp
  clang-tools-extra/clang-doc/YAMLGenerator.cpp

Index: clang-tools-extra/clang-doc/YAMLGenerator.cpp
===
--- clang-tools-extra/clang-doc/YAMLGenerator.cpp
+++ clang-tools-extra/clang-doc/YAMLGenerator.cpp
@@ -9,6 +9,7 @@
 //===--===//
 
 #include "Generators.h"
+#include "Representation.h"
 #include "llvm/Support/YAMLTraits.h"
 #include "llvm/Support/raw_ostream.h"
 
@@ -23,6 +24,7 @@
 LLVM_YAML_IS_SEQUENCE_VECTOR(FunctionInfo)
 LLVM_YAML_IS_SEQUENCE_VECTOR(EnumInfo)
 LLVM_YAML_IS_SEQUENCE_VECTOR(EnumValueInfo)
+LLVM_YAML_IS_SEQUENCE_VECTOR(TemplateParamInfo)
 LLVM_YAML_IS_SEQUENCE_VECTOR(TypedefInfo)
 LLVM_YAML_IS_SEQUENCE_VECTOR(BaseRecordInfo)
 LLVM_YAML_IS_SEQUENCE_VECTOR(std::unique_ptr)
@@ -142,6 +144,7 @@
   IO.mapOptional("ChildFunctions", I.Children.Functions);
   IO.mapOptional("ChildEnums", I.Children.Enums);
   IO.mapOptional("ChildTypedefs", I.Children.Typedefs);
+  IO.mapOptional("Template", I.Template);
 }
 
 static void CommentInfoMapping(IO , CommentInfo ) {
@@ -267,6 +270,28 @@
 // the AS that shouldn't be part of the output. Even though AS_public is the
 // default in the struct, it should be displayed in the YAML output.
 IO.mapOptional("Access", I.Access, clang::AccessSpecifier::AS_none);
+IO.mapOptional("Template", I.Template);
+  }
+};
+
+template <> struct MappingTraits {
+  static void mapping(IO , TemplateParamInfo ) {
+IO.mapOptional("Contents", I.Contents);
+  }
+};
+
+template <> struct MappingTraits {
+  static void mapping(IO , TemplateSpecializationInfo ) {
+IO.mapOptional("SpecializationOf", I.SpecializationOf);
+IO.mapOptional("Params", I.Params);
+  }
+};
+
+template <> struct MappingTraits {
+  static void mapping(IO , TemplateInfo ) {
+IO.mapOptional("Params", I.Params);
+IO.mapOptional("Specialization", I.Specialization,
+   Optional());
   }
 };
 
Index: clang-tools-extra/clang-doc/Serialize.cpp
===
--- clang-tools-extra/clang-doc/Serialize.cpp
+++ clang-tools-extra/clang-doc/Serialize.cpp
@@ -249,7 +249,8 @@
   } else {
 IT = InfoType::IT_default;
   }
-  return TypeInfo(Reference(getUSRForDecl(TD), TD->getNameAsString(), IT,
+  return TypeInfo(Reference(getUSRForDecl(TD),
+/*TD->getNameAsString()*/ T.getAsString(), IT,
 getInfoRelativePath(TD)));
 }
 
@@ -405,10 +406,7 @@
   for (const ParmVarDecl *P : D->parameters()) {
 FieldTypeInfo  = I.Params.emplace_back(
 getTypeInfoForType(P->getOriginalType()), P->getNameAsString());
-
-if (const Expr *DefaultArg = P->getDefaultArg()) {
-  FieldInfo.DefaultValue = getSourceCode(D, DefaultArg->getSourceRange());
-}
+FieldInfo.DefaultValue = getSourceCode(D, P->getDefaultArgRange());
   }
 }
 
@@ -476,6 +474,30 @@
 InfoType::IT_namespace);
 }
 
+void PopulateTemplateParameters(llvm::Optional ,
+const clang::Decl *D) {
+  if (const TemplateParameterList *ParamList =
+  D->getDescribedTemplateParams()) {
+if (!TemplateInfo) {
+  TemplateInfo.emplace();
+}
+for (const NamedDecl *ND : *ParamList) {
+  TemplateInfo->Params.emplace_back(
+  getSourceCode(ND, ND->getSourceRange()));
+}
+  }
+}
+
+TemplateParamInfo TemplateArgumentToInfo(const clang::Decl *D,
+ const TemplateArgument ) {
+  // The TemplateArgument's pretty printing handles all the normal cases
+  // well enough for our requirements.
+  std::string Str;
+  llvm::raw_string_ostream Stream(Str);
+  Arg.print(PrintingPolicy(D->getLangOpts()), Stream, false);
+  return TemplateParamInfo(Str);
+}
+
 template 
 static void populateInfo(Info , const T *D, const FullComment *C,
  bool ) {
@@ -508,6 +530,26 @@
  IsInAnonymousNamespace);
   I.ReturnType = getTypeInfoForType(D->getReturnType());
   parseParameters(I, D);
+
+  PopulateTemplateParameters(I.Template, D);
+
+  // Handle function template 

[PATCH] D138961: [clang-doc] Fix warnings about lock_guard.

[Brett Wilson via Phabricator via cfe-commits]

brettw created this revision.
brettw added a project: clang-tools-extra.
Herald added a subscriber: arphaman.
Herald added a project: All.
brettw requested review of this revision.
Herald added a subscriber: cfe-commits.

Fixes a warning about a potentially unsupported template argument deduction by 
explicitly specifying the template type in std::lock_guard.


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D138961

Files:
  clang-tools-extra/clang-doc/tool/ClangDocMain.cpp


Index: clang-tools-extra/clang-doc/tool/ClangDocMain.cpp
===
--- clang-tools-extra/clang-doc/tool/ClangDocMain.cpp
+++ clang-tools-extra/clang-doc/tool/ClangDocMain.cpp
@@ -264,13 +264,13 @@
 
   // Add a reference to this Info in the Index
   {
-std::lock_guard Guard(IndexMutex);
+std::lock_guard Guard(IndexMutex);
 clang::doc::Generator::addInfoToIndex(CDCtx.Idx, Reduced.get().get());
   }
 
   // Save in the result map (needs a lock due to threaded access).
   {
-std::lock_guard Guard(USRToInfoMutex);
+std::lock_guard Guard(USRToInfoMutex);
 USRToInfo[Group.getKey()] = std::move(Reduced.get());
   }
 });


Index: clang-tools-extra/clang-doc/tool/ClangDocMain.cpp
===
--- clang-tools-extra/clang-doc/tool/ClangDocMain.cpp
+++ clang-tools-extra/clang-doc/tool/ClangDocMain.cpp
@@ -264,13 +264,13 @@
 
   // Add a reference to this Info in the Index
   {
-std::lock_guard Guard(IndexMutex);
+std::lock_guard Guard(IndexMutex);
 clang::doc::Generator::addInfoToIndex(CDCtx.Idx, Reduced.get().get());
   }
 
   // Save in the result map (needs a lock due to threaded access).
   {
-std::lock_guard Guard(USRToInfoMutex);
+std::lock_guard Guard(USRToInfoMutex);
 USRToInfo[Group.getKey()] = std::move(Reduced.get());
   }
 });
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D138073: [clang-doc] Move file layout to the generators.

[Brett Wilson via Phabricator via cfe-commits]

This revision was automatically updated to reflect the committed changes.
Closed by commit rG7b8c7e02122a: [clang-doc] Move file layout to the 
generators. (authored by brettw).

Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D138073/new/

https://reviews.llvm.org/D138073

Files:
  clang-tools-extra/clang-doc/Generators.h
  clang-tools-extra/clang-doc/HTMLGenerator.cpp
  clang-tools-extra/clang-doc/MDGenerator.cpp
  clang-tools-extra/clang-doc/YAMLGenerator.cpp
  clang-tools-extra/clang-doc/tool/ClangDocMain.cpp
  clang-tools-extra/test/clang-doc/single-file-public.cpp
  clang-tools-extra/test/clang-doc/single-file.cpp

Index: clang-tools-extra/test/clang-doc/single-file.cpp
===
--- clang-tools-extra/test/clang-doc/single-file.cpp
+++ clang-tools-extra/test/clang-doc/single-file.cpp
@@ -3,7 +3,7 @@
 // RUN: echo "" > %t/compile_flags.txt
 // RUN: cp "%s" "%t/test.cpp"
 // RUN: clang-doc --doxygen --executor=standalone -p %t %t/test.cpp -output=%t/docs
-// RUN: cat %t/docs/GlobalNamespace/index.yaml | FileCheck %s --check-prefix=CHECK
+// RUN: cat %t/docs/index.yaml | FileCheck %s --check-prefix=CHECK
 // RUN: rm -rf %t
 
 void function(int x);
Index: clang-tools-extra/test/clang-doc/single-file-public.cpp
===
--- clang-tools-extra/test/clang-doc/single-file-public.cpp
+++ clang-tools-extra/test/clang-doc/single-file-public.cpp
@@ -3,7 +3,10 @@
 // RUN: echo "" > %t/compile_flags.txt
 // RUN: cp "%s" "%t/test.cpp"
 // RUN: clang-doc --doxygen --public --executor=standalone -p %t %t/test.cpp -output=%t/docs
-// RUN: cat %t/docs/GlobalNamespace/Record.yaml | FileCheck %s --check-prefix=CHECK
+//   This produces two files, index.yaml and one for the record named by its USR
+//   (which we don't know in advance). This checks the record file by searching
+//   for a name with a 40-char USR name.
+// RUN: find %t/docs -regex ".*/[0-9A-F]*.yaml" -exec cat {} ";" | FileCheck %s --check-prefix=CHECK
 // RUN: rm -rf %t
 
 class Record {
@@ -19,7 +22,7 @@
 void Record::function_public() {}
 
 // CHECK: ---
-// CHECK-NEXT: USR: '{{[0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z]}}'
+// CHECK-NEXT: USR: '{{([0-9A-F]{40})}}'
 // CHECK-NEXT: Name:'Record'
 // CHECK-NEXT: Path:'GlobalNamespace'
 // CHECK-NEXT: Namespace:
@@ -30,12 +33,12 @@
 // CHECK-NEXT:   Filename:'{{.*}}'
 // CHECK-NEXT: TagType: Class
 // CHECK-NEXT: ChildFunctions:
-// CHECK-NEXT:   - USR: '{{[0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z]}}'
+// CHECK-NEXT:   - USR: '{{([0-9A-F]{40})}}'
 // CHECK-NEXT: Name:'function_public'
 // CHECK-NEXT: Namespace:
 // CHECK-NEXT:   - Type:Record
 // CHECK-NEXT: Name:'Record'
-// CHECK-NEXT: USR: '{{[0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z]}}'
+// CHECK-NEXT: USR: '{{([0-9A-F]{40})}}'
 // CHECK-NEXT:   - Type:Namespace
 // CHECK-NEXT: Name:'GlobalNamespace'
 // CHECK-NEXT: DefLocation:
@@ -48,7 +51,7 @@
 // CHECK-NEXT: Parent:
 // CHECK-NEXT: Type:Record
 // CHECK-NEXT: Name:'Record'
-// CHECK-NEXT: USR: '{{[0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z]}}'
+// CHECK-NEXT: USR: '{{([0-9A-F]{40})}}'
 // CHECK-NEXT: ReturnType:
 // CHECK-NEXT:   Type:
 // CHECK-NEXT: Name:'void'
Index: clang-tools-extra/clang-doc/tool/ClangDocMain.cpp
===
--- clang-tools-extra/clang-doc/tool/ClangDocMain.cpp
+++ clang-tools-extra/clang-doc/tool/ClangDocMain.cpp
@@ -43,6 +43,7 @@
 #include 

[clang-tools-extra] 7b8c7e0 - [clang-doc] Move file layout to the generators.

[Brett Wilson via cfe-commits]

Author: Brett Wilson
Date: 2022-11-29T14:40:58-08:00
New Revision: 7b8c7e02122a2ea392b371e3e39b405bc98146de

URL: 
https://github.com/llvm/llvm-project/commit/7b8c7e02122a2ea392b371e3e39b405bc98146de
DIFF: 
https://github.com/llvm/llvm-project/commit/7b8c7e02122a2ea392b371e3e39b405bc98146de.diff

LOG: [clang-doc] Move file layout to the generators.

Previously file naming and directory layout was handled on a per Info
object basis by ClangDocMain and the generators blindly wrote to the
files given. This means all generators must use the same file layout and
caused problems where multiple objects mapped to the same file. The
object collision problem happens most easily with template
specializations because the template parameters are not part of the
"name".

This patch moves the responsibility for output file organization to the
generators. Currently HTML and MD use the same structure as before. But
they now collect all objects that map to a given file and combine them,
avoiding the corruption problems.

Converts the YAML generator to naming files based on USR in one
directory. This is easier for downstream tools to manage and avoids the
naming problems with template specializations. Since this change
requires backward-incompatible output changes to referenced files anyway
(since each one is now an array), this is a good time to introduce this
change.

Differential Revision: https://reviews.llvm.org/D138073

Added: 


Modified: 
clang-tools-extra/clang-doc/Generators.h
clang-tools-extra/clang-doc/HTMLGenerator.cpp
clang-tools-extra/clang-doc/MDGenerator.cpp
clang-tools-extra/clang-doc/YAMLGenerator.cpp
clang-tools-extra/clang-doc/tool/ClangDocMain.cpp
clang-tools-extra/test/clang-doc/single-file-public.cpp
clang-tools-extra/test/clang-doc/single-file.cpp

Removed: 




diff  --git a/clang-tools-extra/clang-doc/Generators.h 
b/clang-tools-extra/clang-doc/Generators.h
index 89c6b34c4384..ba0ef64d3d0f 100644
--- a/clang-tools-extra/clang-doc/Generators.h
+++ b/clang-tools-extra/clang-doc/Generators.h
@@ -25,15 +25,23 @@ class Generator {
 public:
   virtual ~Generator() = default;
 
-  // Write out the decl info in the specified format.
-  virtual llvm::Error generateDocForInfo(Info *I, llvm::raw_ostream ,
- const ClangDocContext ) = 0;
+  // Write out the decl info for the objects in the given map in the specified
+  // format.
+  virtual llvm::Error
+  generateDocs(StringRef RootDir,
+   llvm::StringMap> Infos,
+   const ClangDocContext ) = 0;
+
   // This function writes a file with the index previously constructed.
   // It can be overwritten by any of the inherited generators.
   // If the override method wants to run this it should call
   // Generator::createResources(CDCtx);
   virtual llvm::Error createResources(ClangDocContext );
 
+  // Write out one specific decl info to the destination stream.
+  virtual llvm::Error generateDocForInfo(Info *I, llvm::raw_ostream ,
+ const ClangDocContext ) = 0;
+
   static void addInfoToIndex(Index , const doc::Info *Info);
 };
 

diff  --git a/clang-tools-extra/clang-doc/HTMLGenerator.cpp 
b/clang-tools-extra/clang-doc/HTMLGenerator.cpp
index 6f1d45551bc7..11a78ba60e35 100644
--- a/clang-tools-extra/clang-doc/HTMLGenerator.cpp
+++ b/clang-tools-extra/clang-doc/HTMLGenerator.cpp
@@ -11,6 +11,7 @@
 #include "clang/Basic/Version.h"
 #include "llvm/ADT/StringExtras.h"
 #include "llvm/ADT/StringRef.h"
+#include "llvm/ADT/StringSet.h"
 #include "llvm/Support/FileSystem.h"
 #include "llvm/Support/JSON.h"
 #include "llvm/Support/Path.h"
@@ -839,13 +840,69 @@ class HTMLGenerator : public Generator {
 public:
   static const char *Format;
 
+  llvm::Error generateDocs(StringRef RootDir,
+   llvm::StringMap> Infos,
+   const ClangDocContext ) override;
+  llvm::Error createResources(ClangDocContext ) override;
   llvm::Error generateDocForInfo(Info *I, llvm::raw_ostream ,
  const ClangDocContext ) override;
-  llvm::Error createResources(ClangDocContext ) override;
 };
 
 const char *HTMLGenerator::Format = "html";
 
+llvm::Error
+HTMLGenerator::generateDocs(StringRef RootDir,
+llvm::StringMap> Infos,
+const ClangDocContext ) {
+  // Track which directories we already tried to create.
+  llvm::StringSet<> CreatedDirs;
+
+  // Collect all output by file name and create the nexessary directories.
+  llvm::StringMap> FileToInfos;
+  for (const auto  : Infos) {
+doc::Info *Info = Group.getValue().get();
+
+llvm::SmallString<128> Path;
+llvm::sys::path::native(RootDir, Path);
+llvm::sys::path::a

[PATCH] D138073: [clang-doc] Move file layout to the generators.

[Brett Wilson via Phabricator via cfe-commits]

brettw updated this revision to Diff 478683.

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D138073/new/

https://reviews.llvm.org/D138073

Files:
  clang-tools-extra/clang-doc/Generators.h
  clang-tools-extra/clang-doc/HTMLGenerator.cpp
  clang-tools-extra/clang-doc/MDGenerator.cpp
  clang-tools-extra/clang-doc/YAMLGenerator.cpp
  clang-tools-extra/clang-doc/tool/ClangDocMain.cpp
  clang-tools-extra/test/clang-doc/single-file-public.cpp
  clang-tools-extra/test/clang-doc/single-file.cpp

Index: clang-tools-extra/test/clang-doc/single-file.cpp
===
--- clang-tools-extra/test/clang-doc/single-file.cpp
+++ clang-tools-extra/test/clang-doc/single-file.cpp
@@ -3,7 +3,7 @@
 // RUN: echo "" > %t/compile_flags.txt
 // RUN: cp "%s" "%t/test.cpp"
 // RUN: clang-doc --doxygen --executor=standalone -p %t %t/test.cpp -output=%t/docs
-// RUN: cat %t/docs/GlobalNamespace/index.yaml | FileCheck %s --check-prefix=CHECK
+// RUN: cat %t/docs/index.yaml | FileCheck %s --check-prefix=CHECK
 // RUN: rm -rf %t
 
 void function(int x);
Index: clang-tools-extra/test/clang-doc/single-file-public.cpp
===
--- clang-tools-extra/test/clang-doc/single-file-public.cpp
+++ clang-tools-extra/test/clang-doc/single-file-public.cpp
@@ -3,7 +3,10 @@
 // RUN: echo "" > %t/compile_flags.txt
 // RUN: cp "%s" "%t/test.cpp"
 // RUN: clang-doc --doxygen --public --executor=standalone -p %t %t/test.cpp -output=%t/docs
-// RUN: cat %t/docs/GlobalNamespace/Record.yaml | FileCheck %s --check-prefix=CHECK
+//   This produces two files, index.yaml and one for the record named by its USR
+//   (which we don't know in advance). This checks the record file by searching
+//   for a name with a 40-char USR name.
+// RUN: find %t/docs -regex ".*/[0-9A-F]*.yaml" -exec cat {} ";" | FileCheck %s --check-prefix=CHECK
 // RUN: rm -rf %t
 
 class Record {
@@ -19,7 +22,7 @@
 void Record::function_public() {}
 
 // CHECK: ---
-// CHECK-NEXT: USR: '{{[0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z]}}'
+// CHECK-NEXT: USR: '{{([0-9A-F]{40})}}'
 // CHECK-NEXT: Name:'Record'
 // CHECK-NEXT: Path:'GlobalNamespace'
 // CHECK-NEXT: Namespace:
@@ -30,12 +33,12 @@
 // CHECK-NEXT:   Filename:'{{.*}}'
 // CHECK-NEXT: TagType: Class
 // CHECK-NEXT: ChildFunctions:
-// CHECK-NEXT:   - USR: '{{[0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z]}}'
+// CHECK-NEXT:   - USR: '{{([0-9A-F]{40})}}'
 // CHECK-NEXT: Name:'function_public'
 // CHECK-NEXT: Namespace:
 // CHECK-NEXT:   - Type:Record
 // CHECK-NEXT: Name:'Record'
-// CHECK-NEXT: USR: '{{[0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z]}}'
+// CHECK-NEXT: USR: '{{([0-9A-F]{40})}}'
 // CHECK-NEXT:   - Type:Namespace
 // CHECK-NEXT: Name:'GlobalNamespace'
 // CHECK-NEXT: DefLocation:
@@ -48,7 +51,7 @@
 // CHECK-NEXT: Parent:
 // CHECK-NEXT: Type:Record
 // CHECK-NEXT: Name:'Record'
-// CHECK-NEXT: USR: '{{[0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z]}}'
+// CHECK-NEXT: USR: '{{([0-9A-F]{40})}}'
 // CHECK-NEXT: ReturnType:
 // CHECK-NEXT:   Type:
 // CHECK-NEXT: Name:'void'
Index: clang-tools-extra/clang-doc/tool/ClangDocMain.cpp
===
--- clang-tools-extra/clang-doc/tool/ClangDocMain.cpp
+++ clang-tools-extra/clang-doc/tool/ClangDocMain.cpp
@@ -43,6 +43,7 @@
 #include "llvm/Support/ThreadPool.h"
 #include "llvm/Support/raw_ostream.h"
 #include 
+#include 
 #include 
 
 using namespace clang::ast_matchers;
@@ -130,54 +131,6 @@
   return 

[PATCH] D138073: [clang-doc] Move file layout to the generators.

[Brett Wilson via Phabricator via cfe-commits]

This revision was automatically updated to reflect the committed changes.
Closed by commit rGf8a469fc5727: [clang-doc] Move file layout to the 
generators. (authored by brettw).

Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D138073/new/

https://reviews.llvm.org/D138073

Files:
  clang-tools-extra/clang-doc/Generators.h
  clang-tools-extra/clang-doc/HTMLGenerator.cpp
  clang-tools-extra/clang-doc/MDGenerator.cpp
  clang-tools-extra/clang-doc/YAMLGenerator.cpp
  clang-tools-extra/clang-doc/tool/ClangDocMain.cpp
  clang-tools-extra/test/clang-doc/single-file-public.cpp
  clang-tools-extra/test/clang-doc/single-file.cpp

Index: clang-tools-extra/test/clang-doc/single-file.cpp
===
--- clang-tools-extra/test/clang-doc/single-file.cpp
+++ clang-tools-extra/test/clang-doc/single-file.cpp
@@ -3,7 +3,7 @@
 // RUN: echo "" > %t/compile_flags.txt
 // RUN: cp "%s" "%t/test.cpp"
 // RUN: clang-doc --doxygen --executor=standalone -p %t %t/test.cpp -output=%t/docs
-// RUN: cat %t/docs/GlobalNamespace/index.yaml | FileCheck %s --check-prefix=CHECK
+// RUN: cat %t/docs/index.yaml | FileCheck %s --check-prefix=CHECK
 // RUN: rm -rf %t
 
 void function(int x);
Index: clang-tools-extra/test/clang-doc/single-file-public.cpp
===
--- clang-tools-extra/test/clang-doc/single-file-public.cpp
+++ clang-tools-extra/test/clang-doc/single-file-public.cpp
@@ -3,7 +3,10 @@
 // RUN: echo "" > %t/compile_flags.txt
 // RUN: cp "%s" "%t/test.cpp"
 // RUN: clang-doc --doxygen --public --executor=standalone -p %t %t/test.cpp -output=%t/docs
-// RUN: cat %t/docs/GlobalNamespace/Record.yaml | FileCheck %s --check-prefix=CHECK
+//   This produces two files, index.yaml and one for the record named by its USR
+//   (which we don't know in advance). This checks the record file by searching
+//   for a name with a 40-char USR name.
+// RUN: find %t -regextype sed -regex ".*[0-9A-F]\{40\}.yaml" -exec cat {} ";" | FileCheck %s --check-prefix=CHECK
 // RUN: rm -rf %t
 
 class Record {
@@ -19,7 +22,7 @@
 void Record::function_public() {}
 
 // CHECK: ---
-// CHECK-NEXT: USR: '{{[0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z]}}'
+// CHECK-NEXT: USR: '{{([0-9A-F]{40})}}'
 // CHECK-NEXT: Name:'Record'
 // CHECK-NEXT: Path:'GlobalNamespace'
 // CHECK-NEXT: Namespace:
@@ -30,12 +33,12 @@
 // CHECK-NEXT:   Filename:'{{.*}}'
 // CHECK-NEXT: TagType: Class
 // CHECK-NEXT: ChildFunctions:
-// CHECK-NEXT:   - USR: '{{[0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z]}}'
+// CHECK-NEXT:   - USR: '{{([0-9A-F]{40})}}'
 // CHECK-NEXT: Name:'function_public'
 // CHECK-NEXT: Namespace:
 // CHECK-NEXT:   - Type:Record
 // CHECK-NEXT: Name:'Record'
-// CHECK-NEXT: USR: '{{[0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z]}}'
+// CHECK-NEXT: USR: '{{([0-9A-F]{40})}}'
 // CHECK-NEXT:   - Type:Namespace
 // CHECK-NEXT: Name:'GlobalNamespace'
 // CHECK-NEXT: DefLocation:
@@ -48,7 +51,7 @@
 // CHECK-NEXT: Parent:
 // CHECK-NEXT: Type:Record
 // CHECK-NEXT: Name:'Record'
-// CHECK-NEXT: USR: '{{[0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z]}}'
+// CHECK-NEXT: USR: '{{([0-9A-F]{40})}}'
 // CHECK-NEXT: ReturnType:
 // CHECK-NEXT:   Type:
 // CHECK-NEXT: Name:'void'
Index: clang-tools-extra/clang-doc/tool/ClangDocMain.cpp
===
--- clang-tools-extra/clang-doc/tool/ClangDocMain.cpp
+++ clang-tools-extra/clang-doc/tool/ClangDocMain.cpp
@@ -43,6 +43,7 @@
 

[clang-tools-extra] f8a469f - [clang-doc] Move file layout to the generators.

[Brett Wilson via cfe-commits]

Author: Brett Wilson
Date: 2022-11-22T10:27:29-08:00
New Revision: f8a469fc572778d05b72f34a772082cf3abd3cda

URL: 
https://github.com/llvm/llvm-project/commit/f8a469fc572778d05b72f34a772082cf3abd3cda
DIFF: 
https://github.com/llvm/llvm-project/commit/f8a469fc572778d05b72f34a772082cf3abd3cda.diff

LOG: [clang-doc] Move file layout to the generators.

Previously file naming and directory layout was handled on a per Info
object basis by ClangDocMain and the generators blindly wrote to the
files given. This means all generators must use the same file layout and
caused problems where multiple objects mapped to the same file. The
object collision problem happens most easily with template
specializations because the template parameters are not part of the
"name".

This patch moves the responsibility for output file organization to the
generators. Currently HTML and MD use the same structure as before. But
they now collect all objects that map to a given file and combine them,
avoiding the corruption problems.

Converts the YAML generator to naming files based on USR in one
directory. This is easier for downstream tools to manage and avoids
the naming problems with template specializations. Since this change
requires backward-incompatible output changes to referenced files anyway
(since each one is now an array), this is a good time to introduce this
change.

Differential Revision: https://reviews.llvm.org/D138073

Added: 


Modified: 
clang-tools-extra/clang-doc/Generators.h
clang-tools-extra/clang-doc/HTMLGenerator.cpp
clang-tools-extra/clang-doc/MDGenerator.cpp
clang-tools-extra/clang-doc/YAMLGenerator.cpp
clang-tools-extra/clang-doc/tool/ClangDocMain.cpp
clang-tools-extra/test/clang-doc/single-file-public.cpp
clang-tools-extra/test/clang-doc/single-file.cpp

Removed: 




diff  --git a/clang-tools-extra/clang-doc/Generators.h 
b/clang-tools-extra/clang-doc/Generators.h
index 89c6b34c43844..ba0ef64d3d0f5 100644
--- a/clang-tools-extra/clang-doc/Generators.h
+++ b/clang-tools-extra/clang-doc/Generators.h
@@ -25,15 +25,23 @@ class Generator {
 public:
   virtual ~Generator() = default;
 
-  // Write out the decl info in the specified format.
-  virtual llvm::Error generateDocForInfo(Info *I, llvm::raw_ostream ,
- const ClangDocContext ) = 0;
+  // Write out the decl info for the objects in the given map in the specified
+  // format.
+  virtual llvm::Error
+  generateDocs(StringRef RootDir,
+   llvm::StringMap> Infos,
+   const ClangDocContext ) = 0;
+
   // This function writes a file with the index previously constructed.
   // It can be overwritten by any of the inherited generators.
   // If the override method wants to run this it should call
   // Generator::createResources(CDCtx);
   virtual llvm::Error createResources(ClangDocContext );
 
+  // Write out one specific decl info to the destination stream.
+  virtual llvm::Error generateDocForInfo(Info *I, llvm::raw_ostream ,
+ const ClangDocContext ) = 0;
+
   static void addInfoToIndex(Index , const doc::Info *Info);
 };
 

diff  --git a/clang-tools-extra/clang-doc/HTMLGenerator.cpp 
b/clang-tools-extra/clang-doc/HTMLGenerator.cpp
index 6f1d45551bc78..11a78ba60e35c 100644
--- a/clang-tools-extra/clang-doc/HTMLGenerator.cpp
+++ b/clang-tools-extra/clang-doc/HTMLGenerator.cpp
@@ -11,6 +11,7 @@
 #include "clang/Basic/Version.h"
 #include "llvm/ADT/StringExtras.h"
 #include "llvm/ADT/StringRef.h"
+#include "llvm/ADT/StringSet.h"
 #include "llvm/Support/FileSystem.h"
 #include "llvm/Support/JSON.h"
 #include "llvm/Support/Path.h"
@@ -839,13 +840,69 @@ class HTMLGenerator : public Generator {
 public:
   static const char *Format;
 
+  llvm::Error generateDocs(StringRef RootDir,
+   llvm::StringMap> Infos,
+   const ClangDocContext ) override;
+  llvm::Error createResources(ClangDocContext ) override;
   llvm::Error generateDocForInfo(Info *I, llvm::raw_ostream ,
  const ClangDocContext ) override;
-  llvm::Error createResources(ClangDocContext ) override;
 };
 
 const char *HTMLGenerator::Format = "html";
 
+llvm::Error
+HTMLGenerator::generateDocs(StringRef RootDir,
+llvm::StringMap> Infos,
+const ClangDocContext ) {
+  // Track which directories we already tried to create.
+  llvm::StringSet<> CreatedDirs;
+
+  // Collect all output by file name and create the nexessary directories.
+  llvm::StringMap> FileToInfos;
+  for (const auto  : Infos) {
+doc::Info *Info = Group.getValue().get();
+
+llvm::SmallString<128> Path;
+llvm::sys::path::native(RootDir, Path);
+llvm::sys::path::a

[PATCH] D138073: [clang-doc] Move file layout to the generators.

[Brett Wilson via Phabricator via cfe-commits]

brettw added inline comments.



Comment at: clang-tools-extra/clang-doc/HTMLGenerator.cpp:890-892
+// TODO If there are multiple Infos for this file name (for example,
+// template specializations), this will generate multiple complete web 
pages
+// (with  and , etc.) concatenated together. This generator

paulkirth wrote:
> brettw wrote:
> > paulkirth wrote:
> > > brettw wrote:
> > > > paulkirth wrote:
> > > > > So does this patch make HTML unusable? Currently, if there is a 
> > > > > conflict, the file gets replaced, right? 
> > > > > 
> > > > > I read this as we're going from a situation where we lost data, but 
> > > > > still had valid HTML pages to one where we have complete, but 
> > > > > incorrect HTML, is that correct?
> > > > > 
> > > > > Also, why does that happen under template specialization? USRs are 
> > > > > SHA1 hashes of the symbol. I would expect that hashing the symbol 
> > > > > would be unambiguous and unique, given C++'s ODR. 
> > > > > 
> > > > > That last point made me wonder if an alternate way to handle this 
> > > > > without using the hash would be to use the mangled names?
> > > > I think this makes HTML strictly better (though not ideal).
> > > > 
> > > > Previously if there was a collision, they got written over the top of 
> > > > each other without truncation (not sure why). So if the longer page was 
> > > > written second, you got lucky and you get a complete valid page and 
> > > > just lost the shorter struct's definition. If you got unlucky the 
> > > > shorter page was written second and you get a corrupted file with the 
> > > > shorter content, followed by the end of the longer content peeking out 
> > > > from the end.
> > > > 
> > > > With this change, you get both content concatenated without corruption, 
> > > > you just have duplicate doctype and title tags in between them. 
> > > > Browsers should generally handle this so the page should be usable.
> > > > 
> > > > Fixing this requires a refactor of the HTML generator such that the 
> > > > concept of writing a struct is separate from writing a page. Even if I 
> > > > was going to work on the HTML generator, I would probably want to do 
> > > > this in a separate patch anyway.
> > > > 
> > > > The markdown generator does the same, but markdown doesn't have all of 
> > > > the headers stuff that shouldn't be in the middle of the file, you just 
> > > > get a new `# ...` for the second section and it should be reasonable.
> > > > 
> > > > The HTML and MD generators name files according to the `Name` of the 
> > > > structure. USR is not involved. So any time to things in the same 
> > > > namespace have the same name, you'll get a collision. This happens most 
> > > > commonly with template specialization because the names are identical 
> > > > (the name doesn't account for the template parameters).
> > > > 
> > > > All of this complexity is why I changed the YAML output to use USR 
> > > > which avoids the problem. I didn't want to make this change for the 
> > > > HTML and MD generators because it will make ugly file names that the 
> > > > user will see, and you probably want the template specializations to be 
> > > > in the same file anyway.
> > > > I think this makes HTML strictly better (though not ideal).
> > > > 
> > > > Previously if there was a collision, they got written over the top of 
> > > > each other without truncation (not sure why). So if the longer page was 
> > > > written second, you got lucky and you get a complete valid page and 
> > > > just lost the shorter struct's definition. If you got unlucky the 
> > > > shorter page was written second and you get a corrupted file with the 
> > > > shorter content, followed by the end of the longer content peeking out 
> > > > from the end.
> > > > 
> > > > With this change, you get both content concatenated without corruption, 
> > > > you just have duplicate doctype and title tags in between them. 
> > > > Browsers should generally handle this so the page should be usable.
> > > > 
> > > 
> > > Thanks for the explanation. My concern was that this change made HTML 
> > > unusable, if it is still valid (if not ideal) then I don't think there is 
> > > an issue.
> > > 
> > > > Fixing this requires a refactor of the HTML generator such that the 
> > > > concept of writing a struct is separate from writing a page. Even if I 
> > > > was going to work on the HTML generator, I would probably want to do 
> > > > this in a separate patch anyway.
> > > > 
> > > 
> > > No arguments re splitting up work. Can you file an issue to track this on 
> > > github and reference it in the TODO? That should help make sure this gets 
> > > addressed.
> > > 
> > > > The markdown generator does the same, but markdown doesn't have all of 
> > > > the headers stuff that shouldn't be in the middle of the file, you just 
> > > > get a new `# ...` for the second section and it should be reasonable.
> > > > 
> > > > The 

[PATCH] D138073: [clang-doc] Move file layout to the generators.

[Brett Wilson via Phabricator via cfe-commits]

brettw updated this revision to Diff 476955.
brettw marked 2 inline comments as done.

Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D138073/new/

https://reviews.llvm.org/D138073

Files:
  clang-tools-extra/clang-doc/Generators.h
  clang-tools-extra/clang-doc/HTMLGenerator.cpp
  clang-tools-extra/clang-doc/MDGenerator.cpp
  clang-tools-extra/clang-doc/YAMLGenerator.cpp
  clang-tools-extra/clang-doc/tool/ClangDocMain.cpp
  clang-tools-extra/test/clang-doc/single-file-public.cpp
  clang-tools-extra/test/clang-doc/single-file.cpp

Index: clang-tools-extra/test/clang-doc/single-file.cpp
===
--- clang-tools-extra/test/clang-doc/single-file.cpp
+++ clang-tools-extra/test/clang-doc/single-file.cpp
@@ -3,7 +3,7 @@
 // RUN: echo "" > %t/compile_flags.txt
 // RUN: cp "%s" "%t/test.cpp"
 // RUN: clang-doc --doxygen --executor=standalone -p %t %t/test.cpp -output=%t/docs
-// RUN: cat %t/docs/GlobalNamespace/index.yaml | FileCheck %s --check-prefix=CHECK
+// RUN: cat %t/docs/index.yaml | FileCheck %s --check-prefix=CHECK
 // RUN: rm -rf %t
 
 void function(int x);
Index: clang-tools-extra/test/clang-doc/single-file-public.cpp
===
--- clang-tools-extra/test/clang-doc/single-file-public.cpp
+++ clang-tools-extra/test/clang-doc/single-file-public.cpp
@@ -3,7 +3,10 @@
 // RUN: echo "" > %t/compile_flags.txt
 // RUN: cp "%s" "%t/test.cpp"
 // RUN: clang-doc --doxygen --public --executor=standalone -p %t %t/test.cpp -output=%t/docs
-// RUN: cat %t/docs/GlobalNamespace/Record.yaml | FileCheck %s --check-prefix=CHECK
+//   This produces two files, index.yaml and one for the record named by its USR
+//   (which we don't know in advance). This checks the record file by searching
+//   for a name with a 40-char USR name.
+// RUN: find %t -regextype sed -regex ".*[0-9A-F]\{40\}.yaml" -exec cat {} ";" | FileCheck %s --check-prefix=CHECK
 // RUN: rm -rf %t
 
 class Record {
@@ -19,7 +22,7 @@
 void Record::function_public() {}
 
 // CHECK: ---
-// CHECK-NEXT: USR: '{{[0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z]}}'
+// CHECK-NEXT: USR: '{{([0-9A-F]{40})}}'
 // CHECK-NEXT: Name:'Record'
 // CHECK-NEXT: Path:'GlobalNamespace'
 // CHECK-NEXT: Namespace:
@@ -30,12 +33,12 @@
 // CHECK-NEXT:   Filename:'{{.*}}'
 // CHECK-NEXT: TagType: Class
 // CHECK-NEXT: ChildFunctions:
-// CHECK-NEXT:   - USR: '{{[0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z]}}'
+// CHECK-NEXT:   - USR: '{{([0-9A-F]{40})}}'
 // CHECK-NEXT: Name:'function_public'
 // CHECK-NEXT: Namespace:
 // CHECK-NEXT:   - Type:Record
 // CHECK-NEXT: Name:'Record'
-// CHECK-NEXT: USR: '{{[0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z]}}'
+// CHECK-NEXT: USR: '{{([0-9A-F]{40})}}'
 // CHECK-NEXT:   - Type:Namespace
 // CHECK-NEXT: Name:'GlobalNamespace'
 // CHECK-NEXT: DefLocation:
@@ -48,7 +51,7 @@
 // CHECK-NEXT: Parent:
 // CHECK-NEXT: Type:Record
 // CHECK-NEXT: Name:'Record'
-// CHECK-NEXT: USR: '{{[0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z]}}'
+// CHECK-NEXT: USR: '{{([0-9A-F]{40})}}'
 // CHECK-NEXT: ReturnType:
 // CHECK-NEXT:   Type:
 // CHECK-NEXT: Name:'void'
Index: clang-tools-extra/clang-doc/tool/ClangDocMain.cpp
===
--- clang-tools-extra/clang-doc/tool/ClangDocMain.cpp
+++ clang-tools-extra/clang-doc/tool/ClangDocMain.cpp
@@ -43,6 +43,7 @@
 #include "llvm/Support/ThreadPool.h"
 #include "llvm/Support/raw_ostream.h"
 #include 

[PATCH] D138073: [clang-doc] Move file layout to the generators.

[Brett Wilson via Phabricator via cfe-commits]

brettw added inline comments.



Comment at: clang-tools-extra/clang-doc/HTMLGenerator.cpp:890-892
+// TODO If there are multiple Infos for this file name (for example,
+// template specializations), this will generate multiple complete web 
pages
+// (with  and , etc.) concatenated together. This generator

paulkirth wrote:
> brettw wrote:
> > paulkirth wrote:
> > > So does this patch make HTML unusable? Currently, if there is a conflict, 
> > > the file gets replaced, right? 
> > > 
> > > I read this as we're going from a situation where we lost data, but still 
> > > had valid HTML pages to one where we have complete, but incorrect HTML, 
> > > is that correct?
> > > 
> > > Also, why does that happen under template specialization? USRs are SHA1 
> > > hashes of the symbol. I would expect that hashing the symbol would be 
> > > unambiguous and unique, given C++'s ODR. 
> > > 
> > > That last point made me wonder if an alternate way to handle this without 
> > > using the hash would be to use the mangled names?
> > I think this makes HTML strictly better (though not ideal).
> > 
> > Previously if there was a collision, they got written over the top of each 
> > other without truncation (not sure why). So if the longer page was written 
> > second, you got lucky and you get a complete valid page and just lost the 
> > shorter struct's definition. If you got unlucky the shorter page was 
> > written second and you get a corrupted file with the shorter content, 
> > followed by the end of the longer content peeking out from the end.
> > 
> > With this change, you get both content concatenated without corruption, you 
> > just have duplicate doctype and title tags in between them. Browsers should 
> > generally handle this so the page should be usable.
> > 
> > Fixing this requires a refactor of the HTML generator such that the concept 
> > of writing a struct is separate from writing a page. Even if I was going to 
> > work on the HTML generator, I would probably want to do this in a separate 
> > patch anyway.
> > 
> > The markdown generator does the same, but markdown doesn't have all of the 
> > headers stuff that shouldn't be in the middle of the file, you just get a 
> > new `# ...` for the second section and it should be reasonable.
> > 
> > The HTML and MD generators name files according to the `Name` of the 
> > structure. USR is not involved. So any time to things in the same namespace 
> > have the same name, you'll get a collision. This happens most commonly with 
> > template specialization because the names are identical (the name doesn't 
> > account for the template parameters).
> > 
> > All of this complexity is why I changed the YAML output to use USR which 
> > avoids the problem. I didn't want to make this change for the HTML and MD 
> > generators because it will make ugly file names that the user will see, and 
> > you probably want the template specializations to be in the same file 
> > anyway.
> > I think this makes HTML strictly better (though not ideal).
> > 
> > Previously if there was a collision, they got written over the top of each 
> > other without truncation (not sure why). So if the longer page was written 
> > second, you got lucky and you get a complete valid page and just lost the 
> > shorter struct's definition. If you got unlucky the shorter page was 
> > written second and you get a corrupted file with the shorter content, 
> > followed by the end of the longer content peeking out from the end.
> > 
> > With this change, you get both content concatenated without corruption, you 
> > just have duplicate doctype and title tags in between them. Browsers should 
> > generally handle this so the page should be usable.
> > 
> 
> Thanks for the explanation. My concern was that this change made HTML 
> unusable, if it is still valid (if not ideal) then I don't think there is an 
> issue.
> 
> > Fixing this requires a refactor of the HTML generator such that the concept 
> > of writing a struct is separate from writing a page. Even if I was going to 
> > work on the HTML generator, I would probably want to do this in a separate 
> > patch anyway.
> > 
> 
> No arguments re splitting up work. Can you file an issue to track this on 
> github and reference it in the TODO? That should help make sure this gets 
> addressed.
> 
> > The markdown generator does the same, but markdown doesn't have all of the 
> > headers stuff that shouldn't be in the middle of the file, you just get a 
> > new `# ...` for the second section and it should be reasonable.
> > 
> > The HTML and MD generators name files according to the `Name` of the 
> > structure. USR is not involved. So any time to things in the same namespace 
> > have the same name, you'll get a collision. This happens most commonly with 
> > template specialization because the names are identical (the name doesn't 
> > account for the template parameters).
> > 
> 
> I think there should be 

[PATCH] D138073: [clang-doc] Move file layout to the generators.

[Brett Wilson via Phabricator via cfe-commits]

brettw updated this revision to Diff 476612.
brettw marked an inline comment as done.

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D138073/new/

https://reviews.llvm.org/D138073

Files:
  clang-tools-extra/clang-doc/Generators.h
  clang-tools-extra/clang-doc/HTMLGenerator.cpp
  clang-tools-extra/clang-doc/MDGenerator.cpp
  clang-tools-extra/clang-doc/YAMLGenerator.cpp
  clang-tools-extra/clang-doc/tool/ClangDocMain.cpp
  clang-tools-extra/test/clang-doc/single-file-public.cpp
  clang-tools-extra/test/clang-doc/single-file.cpp

Index: clang-tools-extra/test/clang-doc/single-file.cpp
===
--- clang-tools-extra/test/clang-doc/single-file.cpp
+++ clang-tools-extra/test/clang-doc/single-file.cpp
@@ -3,7 +3,7 @@
 // RUN: echo "" > %t/compile_flags.txt
 // RUN: cp "%s" "%t/test.cpp"
 // RUN: clang-doc --doxygen --executor=standalone -p %t %t/test.cpp -output=%t/docs
-// RUN: cat %t/docs/GlobalNamespace/index.yaml | FileCheck %s --check-prefix=CHECK
+// RUN: cat %t/docs/index.yaml | FileCheck %s --check-prefix=CHECK
 // RUN: rm -rf %t
 
 void function(int x);
Index: clang-tools-extra/test/clang-doc/single-file-public.cpp
===
--- clang-tools-extra/test/clang-doc/single-file-public.cpp
+++ clang-tools-extra/test/clang-doc/single-file-public.cpp
@@ -3,7 +3,10 @@
 // RUN: echo "" > %t/compile_flags.txt
 // RUN: cp "%s" "%t/test.cpp"
 // RUN: clang-doc --doxygen --public --executor=standalone -p %t %t/test.cpp -output=%t/docs
-// RUN: cat %t/docs/GlobalNamespace/Record.yaml | FileCheck %s --check-prefix=CHECK
+//   This produces two files, index.yaml and one for the record named by its USR
+//   (which we don't know in advance). This checks the record file by searching
+//   for a name with a 40-char USR name.
+// RUN: find %t -regex ".*[0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z][0-9A-Z].*" -exec cat {} \; | FileCheck %s --check-prefix=CHECK
 // RUN: rm -rf %t
 
 class Record {
Index: clang-tools-extra/clang-doc/tool/ClangDocMain.cpp
===
--- clang-tools-extra/clang-doc/tool/ClangDocMain.cpp
+++ clang-tools-extra/clang-doc/tool/ClangDocMain.cpp
@@ -43,6 +43,7 @@
 #include "llvm/Support/ThreadPool.h"
 #include "llvm/Support/raw_ostream.h"
 #include 
+#include 
 #include 
 
 using namespace clang::ast_matchers;
@@ -130,54 +131,6 @@
   return llvm::sys::fs::getMainExecutable(Argv0, MainAddr);
 }
 
-bool CreateDirectory(const Twine , bool ClearDirectory = false) {
-  std::error_code OK;
-  llvm::SmallString<128> DocsRootPath;
-  if (ClearDirectory) {
-std::error_code RemoveStatus = llvm::sys::fs::remove_directories(DirName);
-if (RemoveStatus != OK) {
-  llvm::errs() << "Unable to remove existing documentation directory for "
-   << DirName << ".\n";
-  return true;
-}
-  }
-  std::error_code DirectoryStatus = llvm::sys::fs::create_directories(DirName);
-  if (DirectoryStatus != OK) {
-llvm::errs() << "Unable to create documentation directories.\n";
-return true;
-  }
-  return false;
-}
-
-// A function to extract the appropriate file name for a given info's
-// documentation. The path returned is a composite of the output directory, the
-// info's relative path and name and the extension. The relative path should
-// have been constructed in the serialization phase.
-//
-// Example: Given the below, the  path for class C will be
-// /A/B/C.
-//
-// namespace A {
-// namespace B {
-//
-// class C {};
-//
-// }
-// }
-llvm::Expected> getInfoOutputFile(StringRef Root,
- StringRef RelativePath,
- StringRef Name,
- StringRef Ext) {
-  llvm::SmallString<128> Path;
-  llvm::sys::path::native(Root, Path);
-  llvm::sys::path::append(Path, RelativePath);
-  if (CreateDirectory(Path))
-return llvm::createStringError(llvm::inconvertibleErrorCode(),
-   "failed to create directory");
-  llvm::sys::path::append(Path, Name + Ext);
-  return Path;
-}
-
 int main(int argc, const char **argv) {
   llvm::sys::PrintStackTraceOnErrorSignal(argv[0]);
   std::error_code OK;
@@ -274,6 +227,11 @@
 R.first->second.emplace_back(Value);
   });
 
+  // Collects all Infos according to their unique USR value. This map is added
+  // to from the thread pool below and is protected by the USRToInfoMutex.
+  llvm::sys::Mutex USRToInfoMutex;
+  llvm::StringMap> USRToInfo;
+
   // First reducing phase (reduce all decls into one info per 

[PATCH] D138073: [clang-doc] Move file layout to the generators.

[Brett Wilson via Phabricator via cfe-commits]

brettw added inline comments.



Comment at: clang-tools-extra/clang-doc/HTMLGenerator.cpp:890-892
+// TODO If there are multiple Infos for this file name (for example,
+// template specializations), this will generate multiple complete web 
pages
+// (with  and , etc.) concatenated together. This generator

paulkirth wrote:
> So does this patch make HTML unusable? Currently, if there is a conflict, the 
> file gets replaced, right? 
> 
> I read this as we're going from a situation where we lost data, but still had 
> valid HTML pages to one where we have complete, but incorrect HTML, is that 
> correct?
> 
> Also, why does that happen under template specialization? USRs are SHA1 
> hashes of the symbol. I would expect that hashing the symbol would be 
> unambiguous and unique, given C++'s ODR. 
> 
> That last point made me wonder if an alternate way to handle this without 
> using the hash would be to use the mangled names?
I think this makes HTML strictly better (though not ideal).

Previously if there was a collision, they got written over the top of each 
other without truncation (not sure why). So if the longer page was written 
second, you got lucky and you get a complete valid page and just lost the 
shorter struct's definition. If you got unlucky the shorter page was written 
second and you get a corrupted file with the shorter content, followed by the 
end of the longer content peeking out from the end.

With this change, you get both content concatenated without corruption, you 
just have duplicate doctype and title tags in between them. Browsers should 
generally handle this so the page should be usable.

Fixing this requires a refactor of the HTML generator such that the concept of 
writing a struct is separate from writing a page. Even if I was going to work 
on the HTML generator, I would probably want to do this in a separate patch 
anyway.

The markdown generator does the same, but markdown doesn't have all of the 
headers stuff that shouldn't be in the middle of the file, you just get a new 
`# ...` for the second section and it should be reasonable.

The HTML and MD generators name files according to the `Name` of the structure. 
USR is not involved. So any time to things in the same namespace have the same 
name, you'll get a collision. This happens most commonly with template 
specialization because the names are identical (the name doesn't account for 
the template parameters).

All of this complexity is why I changed the YAML output to use USR which avoids 
the problem. I didn't want to make this change for the HTML and MD generators 
because it will make ugly file names that the user will see, and you probably 
want the template specializations to be in the same file anyway.



Comment at: clang-tools-extra/test/clang-doc/single-file-public.cpp:6
 // RUN: clang-doc --doxygen --public --executor=standalone -p %t %t/test.cpp 
-output=%t/docs
-// RUN: cat %t/docs/GlobalNamespace/Record.yaml | FileCheck %s 
--check-prefix=CHECK
+// RUN: cat %t/docs/`ls %t/docs | grep -v index.yaml` | FileCheck %s 
--check-prefix=CHECK
 // RUN: rm -rf %t

paulkirth wrote:
> Its OK to use `ls` and `grep` in the runline, but you cannot use the backtick 
> syntax. It will cause problems on other platforms and I can't find any other 
> usage of that in our codebase. TBH I'm a bit surprised this works, since 
> `lit` has some interesting behavior around RUN. 
> 
> I think you can just use `cat < ls %t/docs | grep -v index.yaml` instead. I 
> haven't tried it, but that is certainly a more typical method of doing this 
> in a lit test. Using `find` may also work, but I only found one use in 
> `llvm/test/ExecutionEngine/MCJIT/load-object-a.ll`.
Your example doesn't work (it's trying to load a file named "ls") but I found 
another solution using xargs (which is hopefully supported on all of the 
platforms).


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D138073/new/

https://reviews.llvm.org/D138073

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D138073: [clang-doc] Move file layout to the generators.

[Brett Wilson via Phabricator via cfe-commits]

brettw updated this revision to Diff 476183.
brettw marked 3 inline comments as done.

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D138073/new/

https://reviews.llvm.org/D138073

Files:
  clang-tools-extra/clang-doc/Generators.h
  clang-tools-extra/clang-doc/HTMLGenerator.cpp
  clang-tools-extra/clang-doc/MDGenerator.cpp
  clang-tools-extra/clang-doc/YAMLGenerator.cpp
  clang-tools-extra/clang-doc/tool/ClangDocMain.cpp
  clang-tools-extra/test/clang-doc/single-file-public.cpp
  clang-tools-extra/test/clang-doc/single-file.cpp

Index: clang-tools-extra/test/clang-doc/single-file.cpp
===
--- clang-tools-extra/test/clang-doc/single-file.cpp
+++ clang-tools-extra/test/clang-doc/single-file.cpp
@@ -3,7 +3,7 @@
 // RUN: echo "" > %t/compile_flags.txt
 // RUN: cp "%s" "%t/test.cpp"
 // RUN: clang-doc --doxygen --executor=standalone -p %t %t/test.cpp -output=%t/docs
-// RUN: cat %t/docs/GlobalNamespace/index.yaml | FileCheck %s --check-prefix=CHECK
+// RUN: cat %t/docs/index.yaml | FileCheck %s --check-prefix=CHECK
 // RUN: rm -rf %t
 
 void function(int x);
Index: clang-tools-extra/test/clang-doc/single-file-public.cpp
===
--- clang-tools-extra/test/clang-doc/single-file-public.cpp
+++ clang-tools-extra/test/clang-doc/single-file-public.cpp
@@ -3,7 +3,9 @@
 // RUN: echo "" > %t/compile_flags.txt
 // RUN: cp "%s" "%t/test.cpp"
 // RUN: clang-doc --doxygen --public --executor=standalone -p %t %t/test.cpp -output=%t/docs
-// RUN: cat %t/docs/GlobalNamespace/Record.yaml | FileCheck %s --check-prefix=CHECK
+//   This produces two files, index.yaml and one for the record named by its USR
+//   (which we don't know in advance). This checks the record file.
+// RUN: ls %t/docs | grep -v index.yaml | xargs -IYAML_RECORD_FILE cat %t/docs/YAML_RECORD_FILE | FileCheck %s --check-prefix=CHECK
 // RUN: rm -rf %t
 
 class Record {
Index: clang-tools-extra/clang-doc/tool/ClangDocMain.cpp
===
--- clang-tools-extra/clang-doc/tool/ClangDocMain.cpp
+++ clang-tools-extra/clang-doc/tool/ClangDocMain.cpp
@@ -43,6 +43,7 @@
 #include "llvm/Support/ThreadPool.h"
 #include "llvm/Support/raw_ostream.h"
 #include 
+#include 
 #include 
 
 using namespace clang::ast_matchers;
@@ -130,54 +131,6 @@
   return llvm::sys::fs::getMainExecutable(Argv0, MainAddr);
 }
 
-bool CreateDirectory(const Twine , bool ClearDirectory = false) {
-  std::error_code OK;
-  llvm::SmallString<128> DocsRootPath;
-  if (ClearDirectory) {
-std::error_code RemoveStatus = llvm::sys::fs::remove_directories(DirName);
-if (RemoveStatus != OK) {
-  llvm::errs() << "Unable to remove existing documentation directory for "
-   << DirName << ".\n";
-  return true;
-}
-  }
-  std::error_code DirectoryStatus = llvm::sys::fs::create_directories(DirName);
-  if (DirectoryStatus != OK) {
-llvm::errs() << "Unable to create documentation directories.\n";
-return true;
-  }
-  return false;
-}
-
-// A function to extract the appropriate file name for a given info's
-// documentation. The path returned is a composite of the output directory, the
-// info's relative path and name and the extension. The relative path should
-// have been constructed in the serialization phase.
-//
-// Example: Given the below, the  path for class C will be
-// /A/B/C.
-//
-// namespace A {
-// namespace B {
-//
-// class C {};
-//
-// }
-// }
-llvm::Expected> getInfoOutputFile(StringRef Root,
- StringRef RelativePath,
- StringRef Name,
- StringRef Ext) {
-  llvm::SmallString<128> Path;
-  llvm::sys::path::native(Root, Path);
-  llvm::sys::path::append(Path, RelativePath);
-  if (CreateDirectory(Path))
-return llvm::createStringError(llvm::inconvertibleErrorCode(),
-   "failed to create directory");
-  llvm::sys::path::append(Path, Name + Ext);
-  return Path;
-}
-
 int main(int argc, const char **argv) {
   llvm::sys::PrintStackTraceOnErrorSignal(argv[0]);
   std::error_code OK;
@@ -274,6 +227,11 @@
 R.first->second.emplace_back(Value);
   });
 
+  // Collects all Infos according to their unique USR value. This map is added
+  // to from the thread pool below and is protected by the USRToInfoMutex.
+  llvm::sys::Mutex USRToInfoMutex;
+  llvm::StringMap> USRToInfo;
+
   // First reducing phase (reduce all decls into one info per decl).
   llvm::outs() << "Reducing " << USRToBitcode.size() << " infos...\n";
   std::atomic Error;
@@ -304,31 +262,17 @@
 return;
   }
 
-  doc::Info *I = Reduced.get().get();
-  auto InfoPath =
-  getInfoOutputFile(OutDirectory, I->getRelativeFilePath(""),
-

[PATCH] D138073: [clang-doc] Move file layout to the generators.

[Brett Wilson via Phabricator via cfe-commits]

brettw updated this revision to Diff 475869.
brettw added a comment.

Updated the lit tests.


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D138073/new/

https://reviews.llvm.org/D138073

Files:
  clang-tools-extra/clang-doc/Generators.h
  clang-tools-extra/clang-doc/HTMLGenerator.cpp
  clang-tools-extra/clang-doc/MDGenerator.cpp
  clang-tools-extra/clang-doc/YAMLGenerator.cpp
  clang-tools-extra/clang-doc/tool/ClangDocMain.cpp
  clang-tools-extra/test/clang-doc/single-file-public.cpp
  clang-tools-extra/test/clang-doc/single-file.cpp

Index: clang-tools-extra/test/clang-doc/single-file.cpp
===
--- clang-tools-extra/test/clang-doc/single-file.cpp
+++ clang-tools-extra/test/clang-doc/single-file.cpp
@@ -3,7 +3,7 @@
 // RUN: echo "" > %t/compile_flags.txt
 // RUN: cp "%s" "%t/test.cpp"
 // RUN: clang-doc --doxygen --executor=standalone -p %t %t/test.cpp -output=%t/docs
-// RUN: cat %t/docs/GlobalNamespace/index.yaml | FileCheck %s --check-prefix=CHECK
+// RUN: cat %t/docs/index.yaml | FileCheck %s --check-prefix=CHECK
 // RUN: rm -rf %t
 
 void function(int x);
Index: clang-tools-extra/test/clang-doc/single-file-public.cpp
===
--- clang-tools-extra/test/clang-doc/single-file-public.cpp
+++ clang-tools-extra/test/clang-doc/single-file-public.cpp
@@ -3,7 +3,7 @@
 // RUN: echo "" > %t/compile_flags.txt
 // RUN: cp "%s" "%t/test.cpp"
 // RUN: clang-doc --doxygen --public --executor=standalone -p %t %t/test.cpp -output=%t/docs
-// RUN: cat %t/docs/GlobalNamespace/Record.yaml | FileCheck %s --check-prefix=CHECK
+// RUN: cat %t/docs/`ls %t/docs | grep -v index.yaml` | FileCheck %s --check-prefix=CHECK
 // RUN: rm -rf %t
 
 class Record {
Index: clang-tools-extra/clang-doc/tool/ClangDocMain.cpp
===
--- clang-tools-extra/clang-doc/tool/ClangDocMain.cpp
+++ clang-tools-extra/clang-doc/tool/ClangDocMain.cpp
@@ -43,6 +43,7 @@
 #include "llvm/Support/ThreadPool.h"
 #include "llvm/Support/raw_ostream.h"
 #include 
+#include 
 #include 
 
 using namespace clang::ast_matchers;
@@ -130,54 +131,6 @@
   return llvm::sys::fs::getMainExecutable(Argv0, MainAddr);
 }
 
-bool CreateDirectory(const Twine , bool ClearDirectory = false) {
-  std::error_code OK;
-  llvm::SmallString<128> DocsRootPath;
-  if (ClearDirectory) {
-std::error_code RemoveStatus = llvm::sys::fs::remove_directories(DirName);
-if (RemoveStatus != OK) {
-  llvm::errs() << "Unable to remove existing documentation directory for "
-   << DirName << ".\n";
-  return true;
-}
-  }
-  std::error_code DirectoryStatus = llvm::sys::fs::create_directories(DirName);
-  if (DirectoryStatus != OK) {
-llvm::errs() << "Unable to create documentation directories.\n";
-return true;
-  }
-  return false;
-}
-
-// A function to extract the appropriate file name for a given info's
-// documentation. The path returned is a composite of the output directory, the
-// info's relative path and name and the extension. The relative path should
-// have been constructed in the serialization phase.
-//
-// Example: Given the below, the  path for class C will be
-// /A/B/C.
-//
-// namespace A {
-// namespace B {
-//
-// class C {};
-//
-// }
-// }
-llvm::Expected> getInfoOutputFile(StringRef Root,
- StringRef RelativePath,
- StringRef Name,
- StringRef Ext) {
-  llvm::SmallString<128> Path;
-  llvm::sys::path::native(Root, Path);
-  llvm::sys::path::append(Path, RelativePath);
-  if (CreateDirectory(Path))
-return llvm::createStringError(llvm::inconvertibleErrorCode(),
-   "failed to create directory");
-  llvm::sys::path::append(Path, Name + Ext);
-  return Path;
-}
-
 int main(int argc, const char **argv) {
   llvm::sys::PrintStackTraceOnErrorSignal(argv[0]);
   std::error_code OK;
@@ -274,6 +227,11 @@
 R.first->second.emplace_back(Value);
   });
 
+  // Collects all Infos according to their unique USR value. This map is added
+  // to from the thread pool below and is protected by the USRToInfoMutex.
+  llvm::sys::Mutex USRToInfoMutex;
+  llvm::StringMap> USRToInfo;
+
   // First reducing phase (reduce all decls into one info per decl).
   llvm::outs() << "Reducing " << USRToBitcode.size() << " infos...\n";
   std::atomic Error;
@@ -304,31 +262,17 @@
 return;
   }
 
-  doc::Info *I = Reduced.get().get();
-  auto InfoPath =
-  getInfoOutputFile(OutDirectory, I->getRelativeFilePath(""),
-I->getFileBaseName(), "." + Format);
-  if (!InfoPath) {
-llvm::errs() << toString(InfoPath.takeError()) << "\n";
-Error = true;
-return;
-  }
-  std::error_code 

[PATCH] D138073: [clang-doc] Move file layout to the generators.

[Brett Wilson via Phabricator via cfe-commits]

brettw added a comment.

If you're curious, you can see the simplification the YAML output format change 
makes in the consuming code:
https://fuchsia-review.git.corp.google.com/c/fuchsia/+/760570/2/tools/cppdocgen/clangdoc/clangdoc.go
Since it allows us to remove some special-casing code for loading things in the 
global namespace and for anonymous records.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D138073/new/

https://reviews.llvm.org/D138073

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D138073: [clang-doc] Move file layout to the generators.

[Brett Wilson via Phabricator via cfe-commits]

brettw created this revision.
brettw added a reviewer: paulkirth.
brettw added a project: clang-tools-extra.
Herald added subscribers: kadircet, arphaman.
Herald added a project: All.
brettw requested review of this revision.
Herald added subscribers: cfe-commits, ilya-biryukov.

Previously file naming and directory layout was handled on a per Info object 
basis by ClangDocMain and the generators blindly wrote to the files given. This 
means all generators must use the same file layout and caused problems where 
multiple objects mapped to the same file. The object collision problem happens 
most easily with template specializations because the template parameters are 
not part of the "name".

  

This patch moves the responsibility for output file organization to the 
generators. Currently HTML and MD use the same structure as before. But they 
now collect all objects that map to a given file and combine them, avoiding the 
corruption problems.

  

Converts the YAML generator to naming files based on USR in one directory. This 
is easier for downstream tools to manage and avoids the naming problems with 
template specializations. Since this change requires backward-incompatible 
output changes to referenced files anyway (since each one is now an array), 
this is a good time to introduce this change.


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D138073

Files:
  clang-tools-extra/clang-doc/Generators.h
  clang-tools-extra/clang-doc/HTMLGenerator.cpp
  clang-tools-extra/clang-doc/MDGenerator.cpp
  clang-tools-extra/clang-doc/YAMLGenerator.cpp
  clang-tools-extra/clang-doc/tool/ClangDocMain.cpp

Index: clang-tools-extra/clang-doc/tool/ClangDocMain.cpp
===
--- clang-tools-extra/clang-doc/tool/ClangDocMain.cpp
+++ clang-tools-extra/clang-doc/tool/ClangDocMain.cpp
@@ -43,6 +43,7 @@
 #include "llvm/Support/ThreadPool.h"
 #include "llvm/Support/raw_ostream.h"
 #include 
+#include 
 #include 
 
 using namespace clang::ast_matchers;
@@ -130,54 +131,6 @@
   return llvm::sys::fs::getMainExecutable(Argv0, MainAddr);
 }
 
-bool CreateDirectory(const Twine , bool ClearDirectory = false) {
-  std::error_code OK;
-  llvm::SmallString<128> DocsRootPath;
-  if (ClearDirectory) {
-std::error_code RemoveStatus = llvm::sys::fs::remove_directories(DirName);
-if (RemoveStatus != OK) {
-  llvm::errs() << "Unable to remove existing documentation directory for "
-   << DirName << ".\n";
-  return true;
-}
-  }
-  std::error_code DirectoryStatus = llvm::sys::fs::create_directories(DirName);
-  if (DirectoryStatus != OK) {
-llvm::errs() << "Unable to create documentation directories.\n";
-return true;
-  }
-  return false;
-}
-
-// A function to extract the appropriate file name for a given info's
-// documentation. The path returned is a composite of the output directory, the
-// info's relative path and name and the extension. The relative path should
-// have been constructed in the serialization phase.
-//
-// Example: Given the below, the  path for class C will be
-// /A/B/C.
-//
-// namespace A {
-// namespace B {
-//
-// class C {};
-//
-// }
-// }
-llvm::Expected> getInfoOutputFile(StringRef Root,
- StringRef RelativePath,
- StringRef Name,
- StringRef Ext) {
-  llvm::SmallString<128> Path;
-  llvm::sys::path::native(Root, Path);
-  llvm::sys::path::append(Path, RelativePath);
-  if (CreateDirectory(Path))
-return llvm::createStringError(llvm::inconvertibleErrorCode(),
-   "failed to create directory");
-  llvm::sys::path::append(Path, Name + Ext);
-  return Path;
-}
-
 int main(int argc, const char **argv) {
   llvm::sys::PrintStackTraceOnErrorSignal(argv[0]);
   std::error_code OK;
@@ -274,6 +227,11 @@
 R.first->second.emplace_back(Value);
   });
 
+  // Collects all Infos according to their unique USR value. This map is added
+  // to from the thread pool below and is protected by the USRToInfoMutex.
+  llvm::sys::Mutex USRToInfoMutex;
+  llvm::StringMap> USRToInfo;
+
   // First reducing phase (reduce all decls into one info per decl).
   llvm::outs() << "Reducing " << USRToBitcode.size() << " infos...\n";
   std::atomic Error;
@@ -304,31 +262,17 @@
 return;
   }
 
-  doc::Info *I = Reduced.get().get();
-  auto InfoPath =
-  getInfoOutputFile(OutDirectory, I->getRelativeFilePath(""),
-I->getFileBaseName(), "." + Format);
-  if (!InfoPath) {
-llvm::errs() << toString(InfoPath.takeError()) << "\n";
-Error = true;
-return;
-  }
-  std::error_code FileErr;
-  llvm::raw_fd_ostream InfoOS(InfoPath.get(), FileErr,
-  llvm::sys::fs::OF_None);
-  if 

[PATCH] D136894: Add clang-doc readme

[Brett Wilson via Phabricator via cfe-commits]

brettw abandoned this revision.
brettw added a comment.

Ok thanks.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D136894/new/

https://reviews.llvm.org/D136894

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D136894: Add clang-doc readme

[Brett Wilson via Phabricator via cfe-commits]

brettw added a comment.

I definitely wasn't intending to write "formal documentation" for clang-doc. I 
was trying to write some notes for a future contributor as I'm winding down my 
work here, and it seems nobody else is working on it or has any knowledge about 
it. Is there a place in LLVM for this type of documentation? I can rename the 
file to "notes" or something if you think that would help make things more 
clear.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D136894/new/

https://reviews.llvm.org/D136894

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D136894: Add clang-doc readme

[Brett Wilson via Phabricator via cfe-commits]

brettw created this revision.
brettw added a reviewer: paulkirth.
brettw added a project: clang-tools-extra.
Herald added a project: All.
brettw requested review of this revision.
Herald added a subscriber: cfe-commits.

Adds a readme to clang-doc with an overview of the structure of the code.


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D136894

Files:
  clang-tools-extra/clang-doc/README.txt


Index: clang-tools-extra/clang-doc/README.txt
===
--- /dev/null
+++ clang-tools-extra/clang-doc/README.txt
@@ -0,0 +1,60 @@
+==
+How clang-doc works internally
+==
+
+Clang-doc uses the "tooling" library which can run the compiler. It can take 
the files directly or
+it can extract the file list and the commands corresponding to each from a 
compile_commands.json
+file.
+
+The tooling library spins up threads and parses the compilation units in 
parallel. Clang-doc
+registers a callback to run on the AST of each unit.
+
+When the AST is known, the MapASTVisitor in Mapper.cpp is run on the AST. It 
has callbacks for the
+main AST nodes that clang-doc cares about. This is a very simple object that 
mostly calls into
+Serialize.cpp to generate the "representation" of the code. These are the 
various *Info structures
+like FunctionInfo and NamespaceInfo defined in Representation.h that 
correspond to each element of
+the code that might be documented.
+
+The representation from each execution thread is serialized to bitcode using 
BitcodeWriter.cpp. This
+is a custom bitcode defined in BitcodeWriter.h and is NOT regular LLVM bitcode 
IR. Watch out: the
+"Serialize.cpp" file is not related to this step though the name may imply it.
+
+These bitcode representations are then passed back to the main thread and 
deserialized back to a
+new copy of the representation in BitcodeReader.cpp. This round-trip through 
bitcode is used only to
+get the data out of the AST visitor (which is expected to return a byte 
stream) and is never saved
+or used for any other purpose. This round-trip adds significant complexity and 
we should consider
+passing the Representation object hierarchy back to the main thread 
out-of-band without
+serialization and deleting the bitcode representation.
+
+After deserialization, the various representation objects from each thread are 
merged/reduced into a
+single structure. This is necessary both to collect everything and to merge 
the declarations of
+things (of which there may be many) with the actual implementation. Many 
fields on a record can't be
+merged in the abstract (for example, a boolean field on two structures with 
two different values has
+no clear merged result). But this question can be resolved by thinking about 
the items as
+definitions and declarations and picking the value which would be present on 
the definition (usually
+the more specific or non-default value).
+
+After merging the final representation is passed to the output generator which 
writes the final
+files.
+
+==
+To add a new field
+==
+
+To add a new bit of information to the documentation that you extract from the 
AST:
+
+ 1. Add it to the appropriate *Info structure in Representation.h.
+
+ 2. Extract the information from the AST by adding to the code in 
Serialize.cpp and possibly
+Mapper.cpp. Save it to the Info structure used above.
+
+ 3. Write bitcode serialization code in BitcodeWriter.cpp for the new field.
+
+ 4. Write bitcode deserialization code in BitcodeReader.cpp.
+
+ 5. Add merging code in the merge() function for your Info structure (see 
above for advice).
+
+ 6. Update the backend(s) to use the new data.
+
+Note! Steps 3, 4, and especially 5 are easy to forget. Without all of these, 
the code may look like
+it works but you will only get default values out at the end.


Index: clang-tools-extra/clang-doc/README.txt
===
--- /dev/null
+++ clang-tools-extra/clang-doc/README.txt
@@ -0,0 +1,60 @@
+==
+How clang-doc works internally
+==
+
+Clang-doc uses the "tooling" library which can run the compiler. It can take the files directly or
+it can extract the file list and the commands corresponding to each from a compile_commands.json
+file.
+
+The tooling library spins up threads and parses the compilation units in parallel. Clang-doc
+registers a callback to run on the AST of each unit.
+
+When the AST is known, the MapASTVisitor in Mapper.cpp is run on the AST. It has callbacks for the
+main AST nodes that clang-doc cares about. This is a very simple object that mostly calls into
+Serialize.cpp to generate the "representation" of the code. These are the various *Info structures
+like FunctionInfo and NamespaceInfo defined in Representation.h that correspond to each element of
+the code that might be 

[PATCH] D136638: [clang-doc] Fix typedef/using output.

[Brett Wilson via Phabricator via cfe-commits]

This revision was landed with ongoing or failed builds.
This revision was automatically updated to reflect the committed changes.
Closed by commit rG7231c9966e52: [clang-doc] Fix typedef/using output. 
(authored by brettw).

Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D136638/new/

https://reviews.llvm.org/D136638

Files:
  clang-tools-extra/clang-doc/BitcodeReader.cpp
  clang-tools-extra/clang-doc/BitcodeWriter.cpp
  clang-tools-extra/clang-doc/Representation.h
  clang-tools-extra/unittests/clang-doc/BitcodeTest.cpp

Index: clang-tools-extra/unittests/clang-doc/BitcodeTest.cpp
===
--- clang-tools-extra/unittests/clang-doc/BitcodeTest.cpp
+++ clang-tools-extra/unittests/clang-doc/BitcodeTest.cpp
@@ -183,11 +183,33 @@
   I.Underlying = TypeInfo("unsigned");
   I.IsUsing = true;
 
+  CommentInfo Top;
+  Top.Kind = "FullComment";
+
+  Top.Children.emplace_back(std::make_unique());
+  CommentInfo *BlankLine = Top.Children.back().get();
+  BlankLine->Kind = "ParagraphComment";
+  BlankLine->Children.emplace_back(std::make_unique());
+  BlankLine->Children.back()->Kind = "TextComment";
+
+  I.Description.emplace_back(std::move(Top));
+
   std::string WriteResult = writeInfo();
   EXPECT_TRUE(WriteResult.size() > 0);
   std::vector> ReadResults = readInfo(WriteResult, 1);
 
   CheckTypedefInfo(, InfoAsTypedef(ReadResults[0].get()));
+
+  // Check one with no IsUsing set, no description, and no definition location.
+  TypedefInfo I2;
+  I2.Name = "SomethingElse";
+  I2.IsUsing = false;
+  I2.Underlying = TypeInfo("int");
+
+  WriteResult = writeInfo();
+  EXPECT_TRUE(WriteResult.size() > 0);
+  ReadResults = readInfo(WriteResult, 1);
+  CheckTypedefInfo(, InfoAsTypedef(ReadResults[0].get()));
 }
 
 TEST(SerializeTest, emitInfoWithCommentBitcode) {
Index: clang-tools-extra/clang-doc/Representation.h
===
--- clang-tools-extra/clang-doc/Representation.h
+++ clang-tools-extra/clang-doc/Representation.h
@@ -365,7 +365,7 @@
   //   using MyVector = std::vector
   // False means it's a C-style typedef:
   //   typedef std::vector MyVector;
-  bool IsUsing;
+  bool IsUsing = false;
 };
 
 struct BaseRecordInfo : public RecordInfo {
Index: clang-tools-extra/clang-doc/BitcodeWriter.cpp
===
--- clang-tools-extra/clang-doc/BitcodeWriter.cpp
+++ clang-tools-extra/clang-doc/BitcodeWriter.cpp
@@ -432,6 +432,8 @@
   emitRecord(T.Name, TYPEDEF_NAME);
   for (const auto  : T.Namespace)
 emitBlock(N, FieldId::F_namespace);
+  for (const auto  : T.Description)
+emitBlock(CI);
   if (T.DefLoc)
 emitRecord(*T.DefLoc, TYPEDEF_DEFLOCATION);
   emitRecord(T.IsUsing, TYPEDEF_IS_USING);
Index: clang-tools-extra/clang-doc/BitcodeReader.cpp
===
--- clang-tools-extra/clang-doc/BitcodeReader.cpp
+++ clang-tools-extra/clang-doc/BitcodeReader.cpp
@@ -368,28 +368,27 @@
 }
 
 template <> llvm::Expected getCommentInfo(FunctionInfo *I) {
-  I->Description.emplace_back();
-  return >Description.back();
+  return >Description.emplace_back();
 }
 
 template <> llvm::Expected getCommentInfo(NamespaceInfo *I) {
-  I->Description.emplace_back();
-  return >Description.back();
+  return >Description.emplace_back();
 }
 
 template <> llvm::Expected getCommentInfo(RecordInfo *I) {
-  I->Description.emplace_back();
-  return >Description.back();
+  return >Description.emplace_back();
 }
 
 template <> llvm::Expected getCommentInfo(MemberTypeInfo *I) {
-  I->Description.emplace_back();
-  return >Description.back();
+  return >Description.emplace_back();
 }
 
 template <> llvm::Expected getCommentInfo(EnumInfo *I) {
-  I->Description.emplace_back();
-  return >Description.back();
+  return >Description.emplace_back();
+}
+
+template <> llvm::Expected getCommentInfo(TypedefInfo *I) {
+  return >Description.emplace_back();
 }
 
 template <> llvm::Expected getCommentInfo(CommentInfo *I) {
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang-tools-extra] 7231c99 - [clang-doc] Fix typedef/using output.

[Brett Wilson via cfe-commits]

Author: Brett Wilson
Date: 2022-10-25T13:47:24-07:00
New Revision: 7231c9966e523a6fa20aa63b9a9245aff49cf881

URL: 
https://github.com/llvm/llvm-project/commit/7231c9966e523a6fa20aa63b9a9245aff49cf881
DIFF: 
https://github.com/llvm/llvm-project/commit/7231c9966e523a6fa20aa63b9a9245aff49cf881.diff

LOG: [clang-doc] Fix typedef/using output.

Provides an initializer for the TypedefInfo.IsUsing member. Previously
this member was uninitialized and would produce random output.

Adds the Description (code comments) to the bitcode reader/writer.
Previously the typedef/using descriptions were lost during the bitcode
round-trip. Adds a test for this.

Differential Revision: https://reviews.llvm.org/D136638

Added: 


Modified: 
clang-tools-extra/clang-doc/BitcodeReader.cpp
clang-tools-extra/clang-doc/BitcodeWriter.cpp
clang-tools-extra/clang-doc/Representation.h
clang-tools-extra/unittests/clang-doc/BitcodeTest.cpp

Removed: 




diff  --git a/clang-tools-extra/clang-doc/BitcodeReader.cpp 
b/clang-tools-extra/clang-doc/BitcodeReader.cpp
index 8e1db35532a05..524b82d9fcc53 100644
--- a/clang-tools-extra/clang-doc/BitcodeReader.cpp
+++ b/clang-tools-extra/clang-doc/BitcodeReader.cpp
@@ -368,28 +368,27 @@ template  llvm::Expected 
getCommentInfo(T I) {
 }
 
 template <> llvm::Expected getCommentInfo(FunctionInfo *I) {
-  I->Description.emplace_back();
-  return >Description.back();
+  return >Description.emplace_back();
 }
 
 template <> llvm::Expected getCommentInfo(NamespaceInfo *I) {
-  I->Description.emplace_back();
-  return >Description.back();
+  return >Description.emplace_back();
 }
 
 template <> llvm::Expected getCommentInfo(RecordInfo *I) {
-  I->Description.emplace_back();
-  return >Description.back();
+  return >Description.emplace_back();
 }
 
 template <> llvm::Expected getCommentInfo(MemberTypeInfo *I) {
-  I->Description.emplace_back();
-  return >Description.back();
+  return >Description.emplace_back();
 }
 
 template <> llvm::Expected getCommentInfo(EnumInfo *I) {
-  I->Description.emplace_back();
-  return >Description.back();
+  return >Description.emplace_back();
+}
+
+template <> llvm::Expected getCommentInfo(TypedefInfo *I) {
+  return >Description.emplace_back();
 }
 
 template <> llvm::Expected getCommentInfo(CommentInfo *I) {

diff  --git a/clang-tools-extra/clang-doc/BitcodeWriter.cpp 
b/clang-tools-extra/clang-doc/BitcodeWriter.cpp
index 7768f4b06519b..bb0698a5a4028 100644
--- a/clang-tools-extra/clang-doc/BitcodeWriter.cpp
+++ b/clang-tools-extra/clang-doc/BitcodeWriter.cpp
@@ -432,6 +432,8 @@ void ClangDocBitcodeWriter::emitBlock(const TypedefInfo ) 
{
   emitRecord(T.Name, TYPEDEF_NAME);
   for (const auto  : T.Namespace)
 emitBlock(N, FieldId::F_namespace);
+  for (const auto  : T.Description)
+emitBlock(CI);
   if (T.DefLoc)
 emitRecord(*T.DefLoc, TYPEDEF_DEFLOCATION);
   emitRecord(T.IsUsing, TYPEDEF_IS_USING);

diff  --git a/clang-tools-extra/clang-doc/Representation.h 
b/clang-tools-extra/clang-doc/Representation.h
index d416306704611..7d76844462885 100644
--- a/clang-tools-extra/clang-doc/Representation.h
+++ b/clang-tools-extra/clang-doc/Representation.h
@@ -365,7 +365,7 @@ struct TypedefInfo : public SymbolInfo {
   //   using MyVector = std::vector
   // False means it's a C-style typedef:
   //   typedef std::vector MyVector;
-  bool IsUsing;
+  bool IsUsing = false;
 };
 
 struct BaseRecordInfo : public RecordInfo {

diff  --git a/clang-tools-extra/unittests/clang-doc/BitcodeTest.cpp 
b/clang-tools-extra/unittests/clang-doc/BitcodeTest.cpp
index db6569419d579..53e84b1814453 100644
--- a/clang-tools-extra/unittests/clang-doc/BitcodeTest.cpp
+++ b/clang-tools-extra/unittests/clang-doc/BitcodeTest.cpp
@@ -183,11 +183,33 @@ TEST(BitcodeTest, emitTypedefInfoBitcode) {
   I.Underlying = TypeInfo("unsigned");
   I.IsUsing = true;
 
+  CommentInfo Top;
+  Top.Kind = "FullComment";
+
+  Top.Children.emplace_back(std::make_unique());
+  CommentInfo *BlankLine = Top.Children.back().get();
+  BlankLine->Kind = "ParagraphComment";
+  BlankLine->Children.emplace_back(std::make_unique());
+  BlankLine->Children.back()->Kind = "TextComment";
+
+  I.Description.emplace_back(std::move(Top));
+
   std::string WriteResult = writeInfo();
   EXPECT_TRUE(WriteResult.size() > 0);
   std::vector> ReadResults = readInfo(WriteResult, 1);
 
   CheckTypedefInfo(, InfoAsTypedef(ReadResults[0].get()));
+
+  // Check one with no IsUsing set, no description, and no definition location.
+  TypedefInfo I2;
+  I2.Name = "SomethingElse";
+  I2.IsUsing = false;
+  I2.Underlying = TypeInfo("int");
+
+  WriteResult = writeInfo();
+  EXPECT_TRUE(WriteResult.size() > 0);
+  ReadResults = readInfo(WriteResult, 1);
+  CheckTypedef

[PATCH] D136638: [clang-doc] Fix typedef/using output.

[Brett Wilson via Phabricator via cfe-commits]

brettw updated this revision to Diff 470598.

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D136638/new/

https://reviews.llvm.org/D136638

Files:
  clang-tools-extra/clang-doc/BitcodeReader.cpp
  clang-tools-extra/clang-doc/BitcodeWriter.cpp
  clang-tools-extra/clang-doc/Representation.h
  clang-tools-extra/unittests/clang-doc/BitcodeTest.cpp

Index: clang-tools-extra/unittests/clang-doc/BitcodeTest.cpp
===
--- clang-tools-extra/unittests/clang-doc/BitcodeTest.cpp
+++ clang-tools-extra/unittests/clang-doc/BitcodeTest.cpp
@@ -183,11 +183,33 @@
   I.Underlying = TypeInfo("unsigned");
   I.IsUsing = true;
 
+  CommentInfo Top;
+  Top.Kind = "FullComment";
+
+  Top.Children.emplace_back(std::make_unique());
+  CommentInfo *BlankLine = Top.Children.back().get();
+  BlankLine->Kind = "ParagraphComment";
+  BlankLine->Children.emplace_back(std::make_unique());
+  BlankLine->Children.back()->Kind = "TextComment";
+
+  I.Description.emplace_back(std::move(Top));
+
   std::string WriteResult = writeInfo();
   EXPECT_TRUE(WriteResult.size() > 0);
   std::vector> ReadResults = readInfo(WriteResult, 1);
 
   CheckTypedefInfo(, InfoAsTypedef(ReadResults[0].get()));
+
+  // Check one with no IsUsing set, no description, and no definition location.
+  TypedefInfo I2;
+  I2.Name = "SomethingElse";
+  I2.IsUsing = false;
+  I2.Underlying = TypeInfo("int");
+
+  WriteResult = writeInfo();
+  EXPECT_TRUE(WriteResult.size() > 0);
+  ReadResults = readInfo(WriteResult, 1);
+  CheckTypedefInfo(, InfoAsTypedef(ReadResults[0].get()));
 }
 
 TEST(SerializeTest, emitInfoWithCommentBitcode) {
Index: clang-tools-extra/clang-doc/Representation.h
===
--- clang-tools-extra/clang-doc/Representation.h
+++ clang-tools-extra/clang-doc/Representation.h
@@ -365,7 +365,7 @@
   //   using MyVector = std::vector
   // False means it's a C-style typedef:
   //   typedef std::vector MyVector;
-  bool IsUsing;
+  bool IsUsing = false;
 };
 
 struct BaseRecordInfo : public RecordInfo {
Index: clang-tools-extra/clang-doc/BitcodeWriter.cpp
===
--- clang-tools-extra/clang-doc/BitcodeWriter.cpp
+++ clang-tools-extra/clang-doc/BitcodeWriter.cpp
@@ -432,6 +432,8 @@
   emitRecord(T.Name, TYPEDEF_NAME);
   for (const auto  : T.Namespace)
 emitBlock(N, FieldId::F_namespace);
+  for (const auto  : T.Description)
+emitBlock(CI);
   if (T.DefLoc)
 emitRecord(*T.DefLoc, TYPEDEF_DEFLOCATION);
   emitRecord(T.IsUsing, TYPEDEF_IS_USING);
Index: clang-tools-extra/clang-doc/BitcodeReader.cpp
===
--- clang-tools-extra/clang-doc/BitcodeReader.cpp
+++ clang-tools-extra/clang-doc/BitcodeReader.cpp
@@ -368,28 +368,27 @@
 }
 
 template <> llvm::Expected getCommentInfo(FunctionInfo *I) {
-  I->Description.emplace_back();
-  return >Description.back();
+  return >Description.emplace_back();
 }
 
 template <> llvm::Expected getCommentInfo(NamespaceInfo *I) {
-  I->Description.emplace_back();
-  return >Description.back();
+  return >Description.emplace_back();
 }
 
 template <> llvm::Expected getCommentInfo(RecordInfo *I) {
-  I->Description.emplace_back();
-  return >Description.back();
+  return >Description.emplace_back();
 }
 
 template <> llvm::Expected getCommentInfo(MemberTypeInfo *I) {
-  I->Description.emplace_back();
-  return >Description.back();
+  return >Description.emplace_back();
 }
 
 template <> llvm::Expected getCommentInfo(EnumInfo *I) {
-  I->Description.emplace_back();
-  return >Description.back();
+  return >Description.emplace_back();
+}
+
+template <> llvm::Expected getCommentInfo(TypedefInfo *I) {
+  return >Description.emplace_back();
 }
 
 template <> llvm::Expected getCommentInfo(CommentInfo *I) {
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D136638: [clang-doc] Fix typedef/using output.

[Brett Wilson via Phabricator via cfe-commits]

brettw created this revision.
brettw added a reviewer: paulkirth.
brettw added a project: clang-tools-extra.
Herald added a project: All.
brettw requested review of this revision.
Herald added a subscriber: cfe-commits.

Provides an initializer for the TypedefInfo.IsUsing member. Previously this 
member was uninitialized and would produce random output.

  

Adds the Description (code comments) to the bitcode reader/writer. Previously 
the typedef/using descriptions were lost during the bitcode round-trip. Adds a 
test for this.


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D136638

Files:
  clang-tools-extra/clang-doc/BitcodeReader.cpp
  clang-tools-extra/clang-doc/BitcodeWriter.cpp
  clang-tools-extra/clang-doc/Representation.h
  clang-tools-extra/unittests/clang-doc/BitcodeTest.cpp


Index: clang-tools-extra/unittests/clang-doc/BitcodeTest.cpp
===
--- clang-tools-extra/unittests/clang-doc/BitcodeTest.cpp
+++ clang-tools-extra/unittests/clang-doc/BitcodeTest.cpp
@@ -183,6 +183,17 @@
   I.Underlying = TypeInfo("unsigned");
   I.IsUsing = true;
 
+  CommentInfo Top;
+  Top.Kind = "FullComment";
+
+  Top.Children.emplace_back(std::make_unique());
+  CommentInfo *BlankLine = Top.Children.back().get();
+  BlankLine->Kind = "ParagraphComment";
+  BlankLine->Children.emplace_back(std::make_unique());
+  BlankLine->Children.back()->Kind = "TextComment";
+
+  I.Description.emplace_back(std::move(Top));
+
   std::string WriteResult = writeInfo();
   EXPECT_TRUE(WriteResult.size() > 0);
   std::vector> ReadResults = readInfo(WriteResult, 1);
Index: clang-tools-extra/clang-doc/Representation.h
===
--- clang-tools-extra/clang-doc/Representation.h
+++ clang-tools-extra/clang-doc/Representation.h
@@ -365,7 +365,7 @@
   //   using MyVector = std::vector
   // False means it's a C-style typedef:
   //   typedef std::vector MyVector;
-  bool IsUsing;
+  bool IsUsing = false;
 };
 
 struct BaseRecordInfo : public RecordInfo {
Index: clang-tools-extra/clang-doc/BitcodeWriter.cpp
===
--- clang-tools-extra/clang-doc/BitcodeWriter.cpp
+++ clang-tools-extra/clang-doc/BitcodeWriter.cpp
@@ -432,6 +432,8 @@
   emitRecord(T.Name, TYPEDEF_NAME);
   for (const auto  : T.Namespace)
 emitBlock(N, FieldId::F_namespace);
+  for (const auto  : T.Description)
+emitBlock(CI);
   if (T.DefLoc)
 emitRecord(*T.DefLoc, TYPEDEF_DEFLOCATION);
   emitRecord(T.IsUsing, TYPEDEF_IS_USING);
Index: clang-tools-extra/clang-doc/BitcodeReader.cpp
===
--- clang-tools-extra/clang-doc/BitcodeReader.cpp
+++ clang-tools-extra/clang-doc/BitcodeReader.cpp
@@ -392,6 +392,11 @@
   return >Description.back();
 }
 
+template <> llvm::Expected getCommentInfo(TypedefInfo *I) {
+  I->Description.emplace_back();
+  return >Description.back();
+}
+
 template <> llvm::Expected getCommentInfo(CommentInfo *I) {
   I->Children.emplace_back(std::make_unique());
   return I->Children.back().get();


Index: clang-tools-extra/unittests/clang-doc/BitcodeTest.cpp
===
--- clang-tools-extra/unittests/clang-doc/BitcodeTest.cpp
+++ clang-tools-extra/unittests/clang-doc/BitcodeTest.cpp
@@ -183,6 +183,17 @@
   I.Underlying = TypeInfo("unsigned");
   I.IsUsing = true;
 
+  CommentInfo Top;
+  Top.Kind = "FullComment";
+
+  Top.Children.emplace_back(std::make_unique());
+  CommentInfo *BlankLine = Top.Children.back().get();
+  BlankLine->Kind = "ParagraphComment";
+  BlankLine->Children.emplace_back(std::make_unique());
+  BlankLine->Children.back()->Kind = "TextComment";
+
+  I.Description.emplace_back(std::move(Top));
+
   std::string WriteResult = writeInfo();
   EXPECT_TRUE(WriteResult.size() > 0);
   std::vector> ReadResults = readInfo(WriteResult, 1);
Index: clang-tools-extra/clang-doc/Representation.h
===
--- clang-tools-extra/clang-doc/Representation.h
+++ clang-tools-extra/clang-doc/Representation.h
@@ -365,7 +365,7 @@
   //   using MyVector = std::vector
   // False means it's a C-style typedef:
   //   typedef std::vector MyVector;
-  bool IsUsing;
+  bool IsUsing = false;
 };
 
 struct BaseRecordInfo : public RecordInfo {
Index: clang-tools-extra/clang-doc/BitcodeWriter.cpp
===
--- clang-tools-extra/clang-doc/BitcodeWriter.cpp
+++ clang-tools-extra/clang-doc/BitcodeWriter.cpp
@@ -432,6 +432,8 @@
   emitRecord(T.Name, TYPEDEF_NAME);
   for (const auto  : T.Namespace)
 emitBlock(N, FieldId::F_namespace);
+  for (const auto  : T.Description)
+emitBlock(CI);
   if (T.DefLoc)
 emitRecord(*T.DefLoc, TYPEDEF_DEFLOCATION);
   emitRecord(T.IsUsing, TYPEDEF_IS_USING);
Index: 

[clang-tools-extra] 21fb70c - [clang-doc] Add typedef/using information.

[Brett Wilson via cfe-commits]

Author: Brett Wilson
Date: 2022-10-14T14:59:29-07:00
New Revision: 21fb70c6ab3b25fe8f5f8384714a9a359b4b5a54

URL: 
https://github.com/llvm/llvm-project/commit/21fb70c6ab3b25fe8f5f8384714a9a359b4b5a54
DIFF: 
https://github.com/llvm/llvm-project/commit/21fb70c6ab3b25fe8f5f8384714a9a359b4b5a54.diff

LOG: [clang-doc] Add typedef/using information.

Read typedef and "using" type alias declarations and serialize into the
internal structures. Emit this information in the YAML output. The HTML
and MD generators are unchanged.

Separate out the logic to create the parent namespace or record object
and insert the newly created child into it. This logic was previously
duplicated for every "info" type and is now shared.

To help this, a struct containing the child vectors was separated out so
children can be added generically and without having too many templates.

A small change was made to populateParentNamespaces() to allow using
types that aren't themselves DeclContexts (typedefs are the first
example of this).

Differential Revision: https://reviews.llvm.org/D134371

Added: 


Modified: 
clang-tools-extra/clang-doc/BitcodeReader.cpp
clang-tools-extra/clang-doc/BitcodeWriter.cpp
clang-tools-extra/clang-doc/BitcodeWriter.h
clang-tools-extra/clang-doc/HTMLGenerator.cpp
clang-tools-extra/clang-doc/MDGenerator.cpp
clang-tools-extra/clang-doc/Mapper.cpp
clang-tools-extra/clang-doc/Mapper.h
clang-tools-extra/clang-doc/Representation.cpp
clang-tools-extra/clang-doc/Representation.h
clang-tools-extra/clang-doc/Serialize.cpp
clang-tools-extra/clang-doc/Serialize.h
clang-tools-extra/clang-doc/YAMLGenerator.cpp
clang-tools-extra/unittests/clang-doc/BitcodeTest.cpp
clang-tools-extra/unittests/clang-doc/ClangDocTest.cpp
clang-tools-extra/unittests/clang-doc/ClangDocTest.h
clang-tools-extra/unittests/clang-doc/HTMLGeneratorTest.cpp
clang-tools-extra/unittests/clang-doc/MDGeneratorTest.cpp
clang-tools-extra/unittests/clang-doc/MergeTest.cpp
clang-tools-extra/unittests/clang-doc/SerializeTest.cpp
clang-tools-extra/unittests/clang-doc/YAMLGeneratorTest.cpp

Removed: 




diff  --git a/clang-tools-extra/clang-doc/BitcodeReader.cpp 
b/clang-tools-extra/clang-doc/BitcodeReader.cpp
index 027272691d03..8e1db35532a0 100644
--- a/clang-tools-extra/clang-doc/BitcodeReader.cpp
+++ b/clang-tools-extra/clang-doc/BitcodeReader.cpp
@@ -24,12 +24,6 @@ llvm::Error decodeRecord(const Record , 
llvm::SmallVectorImpl ,
   return llvm::Error::success();
 }
 
-llvm::Error decodeRecord(const Record , std::string ,
- llvm::StringRef Blob) {
-  Field.assign(Blob.begin(), Blob.end());
-  return llvm::Error::success();
-}
-
 llvm::Error decodeRecord(const Record , SymbolID ,
  llvm::StringRef Blob) {
   if (R[0] != BitCodeConstants::USRHashSize)
@@ -104,6 +98,7 @@ llvm::Error decodeRecord(const Record , InfoType ,
   case InfoType::IT_function:
   case InfoType::IT_default:
   case InfoType::IT_enum:
+  case InfoType::IT_typedef:
 Field = IT;
 return llvm::Error::success();
   }
@@ -233,6 +228,23 @@ llvm::Error parseRecord(const Record , unsigned ID, 
llvm::StringRef Blob,
   }
 }
 
+llvm::Error parseRecord(const Record , unsigned ID, llvm::StringRef Blob,
+TypedefInfo *I) {
+  switch (ID) {
+  case TYPEDEF_USR:
+return decodeRecord(R, I->USR, Blob);
+  case TYPEDEF_NAME:
+return decodeRecord(R, I->Name, Blob);
+  case TYPEDEF_DEFLOCATION:
+return decodeRecord(R, I->DefLoc, Blob);
+  case TYPEDEF_IS_USING:
+return decodeRecord(R, I->IsUsing, Blob);
+  default:
+return llvm::createStringError(llvm::inconvertibleErrorCode(),
+   "invalid field for TypedefInfo");
+  }
+}
+
 llvm::Error parseRecord(const Record , unsigned ID, llvm::StringRef Blob,
 EnumValueInfo *I) {
   switch (ID) {
@@ -424,6 +436,11 @@ template <> llvm::Error addTypeInfo(EnumInfo *I, TypeInfo 
&) {
   return llvm::Error::success();
 }
 
+template <> llvm::Error addTypeInfo(TypedefInfo *I, TypeInfo &) {
+  I->Underlying = std::move(T);
+  return llvm::Error::success();
+}
+
 template  llvm::Error addReference(T I, Reference &, FieldId F) {
   return llvm::createStringError(llvm::inconvertibleErrorCode(),
  "invalid type cannot contain Reference");
@@ -475,6 +492,17 @@ template <> llvm::Error addReference(EnumInfo *I, 
Reference &, FieldId F) {
   }
 }
 
+template <> llvm::Error addReference(TypedefInfo *I, Reference &, FieldId F) 
{
+  switch (F) {
+  case FieldId::F_namespace:
+I->Namespace.emplace_back(std::move(R));
+return llvm::Error::success();
+  default:
+return llvm::createStringError(llvm::inconvertibleErrorCod

[PATCH] D134371: [clang-doc] Add typedef/using information.

[Brett Wilson via Phabricator via cfe-commits]

This revision was automatically updated to reflect the committed changes.
Closed by commit rG21fb70c6ab3b: [clang-doc] Add typedef/using information. 
(authored by brettw).

Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D134371/new/

https://reviews.llvm.org/D134371

Files:
  clang-tools-extra/clang-doc/BitcodeReader.cpp
  clang-tools-extra/clang-doc/BitcodeWriter.cpp
  clang-tools-extra/clang-doc/BitcodeWriter.h
  clang-tools-extra/clang-doc/HTMLGenerator.cpp
  clang-tools-extra/clang-doc/MDGenerator.cpp
  clang-tools-extra/clang-doc/Mapper.cpp
  clang-tools-extra/clang-doc/Mapper.h
  clang-tools-extra/clang-doc/Representation.cpp
  clang-tools-extra/clang-doc/Representation.h
  clang-tools-extra/clang-doc/Serialize.cpp
  clang-tools-extra/clang-doc/Serialize.h
  clang-tools-extra/clang-doc/YAMLGenerator.cpp
  clang-tools-extra/unittests/clang-doc/BitcodeTest.cpp
  clang-tools-extra/unittests/clang-doc/ClangDocTest.cpp
  clang-tools-extra/unittests/clang-doc/ClangDocTest.h
  clang-tools-extra/unittests/clang-doc/HTMLGeneratorTest.cpp
  clang-tools-extra/unittests/clang-doc/MDGeneratorTest.cpp
  clang-tools-extra/unittests/clang-doc/MergeTest.cpp
  clang-tools-extra/unittests/clang-doc/SerializeTest.cpp
  clang-tools-extra/unittests/clang-doc/YAMLGeneratorTest.cpp

Index: clang-tools-extra/unittests/clang-doc/YAMLGeneratorTest.cpp
===
--- clang-tools-extra/unittests/clang-doc/YAMLGeneratorTest.cpp
+++ clang-tools-extra/unittests/clang-doc/YAMLGeneratorTest.cpp
@@ -28,15 +28,16 @@
   I.Path = "path/to/A";
   I.Namespace.emplace_back(EmptySID, "A", InfoType::IT_namespace);
 
-  I.ChildNamespaces.emplace_back(EmptySID, "ChildNamespace",
- InfoType::IT_namespace, "path/to/A/Namespace");
-  I.ChildRecords.emplace_back(EmptySID, "ChildStruct", InfoType::IT_record,
-  "path/to/A/Namespace");
-  I.ChildFunctions.emplace_back();
-  I.ChildFunctions.back().Name = "OneFunction";
-  I.ChildFunctions.back().Access = AccessSpecifier::AS_none;
-  I.ChildEnums.emplace_back();
-  I.ChildEnums.back().Name = "OneEnum";
+  I.Children.Namespaces.emplace_back(EmptySID, "ChildNamespace",
+ InfoType::IT_namespace,
+ "path/to/A/Namespace");
+  I.Children.Records.emplace_back(EmptySID, "ChildStruct", InfoType::IT_record,
+  "path/to/A/Namespace");
+  I.Children.Functions.emplace_back();
+  I.Children.Functions.back().Name = "OneFunction";
+  I.Children.Functions.back().Access = AccessSpecifier::AS_none;
+  I.Children.Enums.emplace_back();
+  I.Children.Enums.back().Name = "OneEnum";
 
   auto G = getYAMLGenerator();
   assert(G);
@@ -100,8 +101,8 @@
   I.TagType = TagTypeKind::TTK_Class;
   I.Bases.emplace_back(EmptySID, "F", "path/to/F", true,
AccessSpecifier::AS_public, true);
-  I.Bases.back().ChildFunctions.emplace_back();
-  I.Bases.back().ChildFunctions.back().Name = "InheritedFunctionOne";
+  I.Bases.back().Children.Functions.emplace_back();
+  I.Bases.back().Children.Functions.back().Name = "InheritedFunctionOne";
   I.Bases.back().Members.emplace_back(TypeInfo("int", "path/to/int"), "N",
   AccessSpecifier::AS_private);
   // F is in the global namespace
@@ -109,12 +110,12 @@
   I.VirtualParents.emplace_back(EmptySID, "G", InfoType::IT_record,
 "path/to/G");
 
-  I.ChildRecords.emplace_back(EmptySID, "ChildStruct", InfoType::IT_record,
-  "path/to/A/r");
-  I.ChildFunctions.emplace_back();
-  I.ChildFunctions.back().Name = "OneFunction";
-  I.ChildEnums.emplace_back();
-  I.ChildEnums.back().Name = "OneEnum";
+  I.Children.Records.emplace_back(EmptySID, "ChildStruct", InfoType::IT_record,
+  "path/to/A/r");
+  I.Children.Functions.emplace_back();
+  I.Children.Functions.back().Name = "OneFunction";
+  I.Children.Enums.emplace_back();
+  I.Children.Enums.back().Name = "OneEnum";
 
   auto G = getYAMLGenerator();
   assert(G);
@@ -330,6 +331,30 @@
   EXPECT_EQ(Expected, Actual.str());
 }
 
+TEST(YAMLGeneratorTest, enumTypedefYAML) {
+  TypedefInfo I;
+  I.Name = "MyUsing";
+  I.Underlying = TypeInfo("int");
+  I.IsUsing = true;
+
+  auto G = getYAMLGenerator();
+  assert(G);
+  std::string Buffer;
+  llvm::raw_string_ostream Actual(Buffer);
+  auto Err = G->generateDocForInfo(, Actual, ClangDocContext());
+  assert(!Err);
+  std::string Expected =
+  R"raw(---
+USR: ''
+Name:'MyUsing'
+Underlying:
+  Name:'int'
+IsUsing: true
+...
+)raw";
+  EXPECT_EQ(Expected, Actual.str());
+}
+
 TEST(YAMLGeneratorTest, emitCommentYAML) {
   FunctionInfo I;
   I.Name = "f";
Index: 

[PATCH] D134371: [clang-doc] Add typedef/using information.

[Brett Wilson via Phabricator via cfe-commits]

brettw updated this revision to Diff 467895.

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D134371/new/

https://reviews.llvm.org/D134371

Files:
  clang-tools-extra/clang-doc/BitcodeReader.cpp
  clang-tools-extra/clang-doc/BitcodeWriter.cpp
  clang-tools-extra/clang-doc/BitcodeWriter.h
  clang-tools-extra/clang-doc/HTMLGenerator.cpp
  clang-tools-extra/clang-doc/MDGenerator.cpp
  clang-tools-extra/clang-doc/Mapper.cpp
  clang-tools-extra/clang-doc/Mapper.h
  clang-tools-extra/clang-doc/Representation.cpp
  clang-tools-extra/clang-doc/Representation.h
  clang-tools-extra/clang-doc/Serialize.cpp
  clang-tools-extra/clang-doc/Serialize.h
  clang-tools-extra/clang-doc/YAMLGenerator.cpp
  clang-tools-extra/unittests/clang-doc/BitcodeTest.cpp
  clang-tools-extra/unittests/clang-doc/ClangDocTest.cpp
  clang-tools-extra/unittests/clang-doc/ClangDocTest.h
  clang-tools-extra/unittests/clang-doc/HTMLGeneratorTest.cpp
  clang-tools-extra/unittests/clang-doc/MDGeneratorTest.cpp
  clang-tools-extra/unittests/clang-doc/MergeTest.cpp
  clang-tools-extra/unittests/clang-doc/SerializeTest.cpp
  clang-tools-extra/unittests/clang-doc/YAMLGeneratorTest.cpp

Index: clang-tools-extra/unittests/clang-doc/YAMLGeneratorTest.cpp
===
--- clang-tools-extra/unittests/clang-doc/YAMLGeneratorTest.cpp
+++ clang-tools-extra/unittests/clang-doc/YAMLGeneratorTest.cpp
@@ -28,15 +28,16 @@
   I.Path = "path/to/A";
   I.Namespace.emplace_back(EmptySID, "A", InfoType::IT_namespace);
 
-  I.ChildNamespaces.emplace_back(EmptySID, "ChildNamespace",
- InfoType::IT_namespace, "path/to/A/Namespace");
-  I.ChildRecords.emplace_back(EmptySID, "ChildStruct", InfoType::IT_record,
-  "path/to/A/Namespace");
-  I.ChildFunctions.emplace_back();
-  I.ChildFunctions.back().Name = "OneFunction";
-  I.ChildFunctions.back().Access = AccessSpecifier::AS_none;
-  I.ChildEnums.emplace_back();
-  I.ChildEnums.back().Name = "OneEnum";
+  I.Children.Namespaces.emplace_back(EmptySID, "ChildNamespace",
+ InfoType::IT_namespace,
+ "path/to/A/Namespace");
+  I.Children.Records.emplace_back(EmptySID, "ChildStruct", InfoType::IT_record,
+  "path/to/A/Namespace");
+  I.Children.Functions.emplace_back();
+  I.Children.Functions.back().Name = "OneFunction";
+  I.Children.Functions.back().Access = AccessSpecifier::AS_none;
+  I.Children.Enums.emplace_back();
+  I.Children.Enums.back().Name = "OneEnum";
 
   auto G = getYAMLGenerator();
   assert(G);
@@ -100,8 +101,8 @@
   I.TagType = TagTypeKind::TTK_Class;
   I.Bases.emplace_back(EmptySID, "F", "path/to/F", true,
AccessSpecifier::AS_public, true);
-  I.Bases.back().ChildFunctions.emplace_back();
-  I.Bases.back().ChildFunctions.back().Name = "InheritedFunctionOne";
+  I.Bases.back().Children.Functions.emplace_back();
+  I.Bases.back().Children.Functions.back().Name = "InheritedFunctionOne";
   I.Bases.back().Members.emplace_back(TypeInfo("int", "path/to/int"), "N",
   AccessSpecifier::AS_private);
   // F is in the global namespace
@@ -109,12 +110,12 @@
   I.VirtualParents.emplace_back(EmptySID, "G", InfoType::IT_record,
 "path/to/G");
 
-  I.ChildRecords.emplace_back(EmptySID, "ChildStruct", InfoType::IT_record,
-  "path/to/A/r");
-  I.ChildFunctions.emplace_back();
-  I.ChildFunctions.back().Name = "OneFunction";
-  I.ChildEnums.emplace_back();
-  I.ChildEnums.back().Name = "OneEnum";
+  I.Children.Records.emplace_back(EmptySID, "ChildStruct", InfoType::IT_record,
+  "path/to/A/r");
+  I.Children.Functions.emplace_back();
+  I.Children.Functions.back().Name = "OneFunction";
+  I.Children.Enums.emplace_back();
+  I.Children.Enums.back().Name = "OneEnum";
 
   auto G = getYAMLGenerator();
   assert(G);
@@ -330,6 +331,30 @@
   EXPECT_EQ(Expected, Actual.str());
 }
 
+TEST(YAMLGeneratorTest, enumTypedefYAML) {
+  TypedefInfo I;
+  I.Name = "MyUsing";
+  I.Underlying = TypeInfo("int");
+  I.IsUsing = true;
+
+  auto G = getYAMLGenerator();
+  assert(G);
+  std::string Buffer;
+  llvm::raw_string_ostream Actual(Buffer);
+  auto Err = G->generateDocForInfo(, Actual, ClangDocContext());
+  assert(!Err);
+  std::string Expected =
+  R"raw(---
+USR: ''
+Name:'MyUsing'
+Underlying:
+  Name:'int'
+IsUsing: true
+...
+)raw";
+  EXPECT_EQ(Expected, Actual.str());
+}
+
 TEST(YAMLGeneratorTest, emitCommentYAML) {
   FunctionInfo I;
   I.Name = "f";
Index: clang-tools-extra/unittests/clang-doc/SerializeTest.cpp
===
--- clang-tools-extra/unittests/clang-doc/SerializeTest.cpp
+++ 

[PATCH] D134371: [clang-doc] Add typedef/using information.

[Brett Wilson via Phabricator via cfe-commits]

brettw updated this revision to Diff 467894.
brettw added a comment.

Rebase


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D134371/new/

https://reviews.llvm.org/D134371

Files:
  clang-tools-extra/clang-doc/BitcodeReader.cpp
  clang-tools-extra/clang-doc/BitcodeWriter.cpp
  clang-tools-extra/clang-doc/BitcodeWriter.h
  clang-tools-extra/clang-doc/HTMLGenerator.cpp
  clang-tools-extra/clang-doc/MDGenerator.cpp
  clang-tools-extra/clang-doc/Mapper.cpp
  clang-tools-extra/clang-doc/Mapper.h
  clang-tools-extra/clang-doc/Representation.cpp
  clang-tools-extra/clang-doc/Representation.h
  clang-tools-extra/clang-doc/Serialize.cpp
  clang-tools-extra/clang-doc/Serialize.h
  clang-tools-extra/clang-doc/YAMLGenerator.cpp
  clang-tools-extra/unittests/clang-doc/BitcodeTest.cpp
  clang-tools-extra/unittests/clang-doc/ClangDocTest.cpp
  clang-tools-extra/unittests/clang-doc/ClangDocTest.h
  clang-tools-extra/unittests/clang-doc/HTMLGeneratorTest.cpp
  clang-tools-extra/unittests/clang-doc/MDGeneratorTest.cpp
  clang-tools-extra/unittests/clang-doc/MergeTest.cpp
  clang-tools-extra/unittests/clang-doc/SerializeTest.cpp
  clang-tools-extra/unittests/clang-doc/YAMLGeneratorTest.cpp

Index: clang-tools-extra/unittests/clang-doc/YAMLGeneratorTest.cpp
===
--- clang-tools-extra/unittests/clang-doc/YAMLGeneratorTest.cpp
+++ clang-tools-extra/unittests/clang-doc/YAMLGeneratorTest.cpp
@@ -28,15 +28,16 @@
   I.Path = "path/to/A";
   I.Namespace.emplace_back(EmptySID, "A", InfoType::IT_namespace);
 
-  I.ChildNamespaces.emplace_back(EmptySID, "ChildNamespace",
- InfoType::IT_namespace, "path/to/A/Namespace");
-  I.ChildRecords.emplace_back(EmptySID, "ChildStruct", InfoType::IT_record,
-  "path/to/A/Namespace");
-  I.ChildFunctions.emplace_back();
-  I.ChildFunctions.back().Name = "OneFunction";
-  I.ChildFunctions.back().Access = AccessSpecifier::AS_none;
-  I.ChildEnums.emplace_back();
-  I.ChildEnums.back().Name = "OneEnum";
+  I.Children.Namespaces.emplace_back(EmptySID, "ChildNamespace",
+ InfoType::IT_namespace,
+ "path/to/A/Namespace");
+  I.Children.Records.emplace_back(EmptySID, "ChildStruct", InfoType::IT_record,
+  "path/to/A/Namespace");
+  I.Children.Functions.emplace_back();
+  I.Children.Functions.back().Name = "OneFunction";
+  I.Children.Functions.back().Access = AccessSpecifier::AS_none;
+  I.Children.Enums.emplace_back();
+  I.Children.Enums.back().Name = "OneEnum";
 
   auto G = getYAMLGenerator();
   assert(G);
@@ -100,8 +101,8 @@
   I.TagType = TagTypeKind::TTK_Class;
   I.Bases.emplace_back(EmptySID, "F", "path/to/F", true,
AccessSpecifier::AS_public, true);
-  I.Bases.back().ChildFunctions.emplace_back();
-  I.Bases.back().ChildFunctions.back().Name = "InheritedFunctionOne";
+  I.Bases.back().Children.Functions.emplace_back();
+  I.Bases.back().Children.Functions.back().Name = "InheritedFunctionOne";
   I.Bases.back().Members.emplace_back(TypeInfo("int", "path/to/int"), "N",
   AccessSpecifier::AS_private);
   // F is in the global namespace
@@ -109,12 +110,12 @@
   I.VirtualParents.emplace_back(EmptySID, "G", InfoType::IT_record,
 "path/to/G");
 
-  I.ChildRecords.emplace_back(EmptySID, "ChildStruct", InfoType::IT_record,
-  "path/to/A/r");
-  I.ChildFunctions.emplace_back();
-  I.ChildFunctions.back().Name = "OneFunction";
-  I.ChildEnums.emplace_back();
-  I.ChildEnums.back().Name = "OneEnum";
+  I.Children.Records.emplace_back(EmptySID, "ChildStruct", InfoType::IT_record,
+  "path/to/A/r");
+  I.Children.Functions.emplace_back();
+  I.Children.Functions.back().Name = "OneFunction";
+  I.Children.Enums.emplace_back();
+  I.Children.Enums.back().Name = "OneEnum";
 
   auto G = getYAMLGenerator();
   assert(G);
@@ -330,6 +331,30 @@
   EXPECT_EQ(Expected, Actual.str());
 }
 
+TEST(YAMLGeneratorTest, enumTypedefYAML) {
+  TypedefInfo I;
+  I.Name = "MyUsing";
+  I.Underlying = TypeInfo("int");
+  I.IsUsing = true;
+
+  auto G = getYAMLGenerator();
+  assert(G);
+  std::string Buffer;
+  llvm::raw_string_ostream Actual(Buffer);
+  auto Err = G->generateDocForInfo(, Actual, ClangDocContext());
+  assert(!Err);
+  std::string Expected =
+  R"raw(---
+USR: ''
+Name:'MyUsing'
+Underlying:
+  Name:'int'
+IsUsing: true
+...
+)raw";
+  EXPECT_EQ(Expected, Actual.str());
+}
+
 TEST(YAMLGeneratorTest, emitCommentYAML) {
   FunctionInfo I;
   I.Name = "f";
Index: clang-tools-extra/unittests/clang-doc/SerializeTest.cpp
===
--- clang-tools-extra/unittests/clang-doc/SerializeTest.cpp
+++ 

[PATCH] D134371: [clang-doc] Add typedef/using information.

[Brett Wilson via Phabricator via cfe-commits]

brettw reopened this revision.
brettw added a comment.
This revision is now accepted and ready to land.

New patch up to fix warnings that caused the revert.


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D134371/new/

https://reviews.llvm.org/D134371

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D134371: [clang-doc] Add typedef/using information.

[Brett Wilson via Phabricator via cfe-commits]

brettw updated this revision to Diff 466858.
brettw added a comment.

Fixed -Wall issues.


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D134371/new/

https://reviews.llvm.org/D134371

Files:
  clang-tools-extra/clang-doc/BitcodeReader.cpp
  clang-tools-extra/clang-doc/BitcodeWriter.cpp
  clang-tools-extra/clang-doc/BitcodeWriter.h
  clang-tools-extra/clang-doc/HTMLGenerator.cpp
  clang-tools-extra/clang-doc/MDGenerator.cpp
  clang-tools-extra/clang-doc/Mapper.cpp
  clang-tools-extra/clang-doc/Mapper.h
  clang-tools-extra/clang-doc/Representation.cpp
  clang-tools-extra/clang-doc/Representation.h
  clang-tools-extra/clang-doc/Serialize.cpp
  clang-tools-extra/clang-doc/Serialize.h
  clang-tools-extra/clang-doc/YAMLGenerator.cpp
  clang-tools-extra/unittests/clang-doc/BitcodeTest.cpp
  clang-tools-extra/unittests/clang-doc/ClangDocTest.cpp
  clang-tools-extra/unittests/clang-doc/ClangDocTest.h
  clang-tools-extra/unittests/clang-doc/HTMLGeneratorTest.cpp
  clang-tools-extra/unittests/clang-doc/MDGeneratorTest.cpp
  clang-tools-extra/unittests/clang-doc/MergeTest.cpp
  clang-tools-extra/unittests/clang-doc/SerializeTest.cpp
  clang-tools-extra/unittests/clang-doc/YAMLGeneratorTest.cpp

Index: clang-tools-extra/unittests/clang-doc/YAMLGeneratorTest.cpp
===
--- clang-tools-extra/unittests/clang-doc/YAMLGeneratorTest.cpp
+++ clang-tools-extra/unittests/clang-doc/YAMLGeneratorTest.cpp
@@ -28,15 +28,16 @@
   I.Path = "path/to/A";
   I.Namespace.emplace_back(EmptySID, "A", InfoType::IT_namespace);
 
-  I.ChildNamespaces.emplace_back(EmptySID, "ChildNamespace",
- InfoType::IT_namespace, "path/to/A/Namespace");
-  I.ChildRecords.emplace_back(EmptySID, "ChildStruct", InfoType::IT_record,
-  "path/to/A/Namespace");
-  I.ChildFunctions.emplace_back();
-  I.ChildFunctions.back().Name = "OneFunction";
-  I.ChildFunctions.back().Access = AccessSpecifier::AS_none;
-  I.ChildEnums.emplace_back();
-  I.ChildEnums.back().Name = "OneEnum";
+  I.Children.Namespaces.emplace_back(EmptySID, "ChildNamespace",
+ InfoType::IT_namespace,
+ "path/to/A/Namespace");
+  I.Children.Records.emplace_back(EmptySID, "ChildStruct", InfoType::IT_record,
+  "path/to/A/Namespace");
+  I.Children.Functions.emplace_back();
+  I.Children.Functions.back().Name = "OneFunction";
+  I.Children.Functions.back().Access = AccessSpecifier::AS_none;
+  I.Children.Enums.emplace_back();
+  I.Children.Enums.back().Name = "OneEnum";
 
   auto G = getYAMLGenerator();
   assert(G);
@@ -100,8 +101,8 @@
   I.TagType = TagTypeKind::TTK_Class;
   I.Bases.emplace_back(EmptySID, "F", "path/to/F", true,
AccessSpecifier::AS_public, true);
-  I.Bases.back().ChildFunctions.emplace_back();
-  I.Bases.back().ChildFunctions.back().Name = "InheritedFunctionOne";
+  I.Bases.back().Children.Functions.emplace_back();
+  I.Bases.back().Children.Functions.back().Name = "InheritedFunctionOne";
   I.Bases.back().Members.emplace_back(TypeInfo("int", "path/to/int"), "N",
   AccessSpecifier::AS_private);
   // F is in the global namespace
@@ -109,12 +110,12 @@
   I.VirtualParents.emplace_back(EmptySID, "G", InfoType::IT_record,
 "path/to/G");
 
-  I.ChildRecords.emplace_back(EmptySID, "ChildStruct", InfoType::IT_record,
-  "path/to/A/r");
-  I.ChildFunctions.emplace_back();
-  I.ChildFunctions.back().Name = "OneFunction";
-  I.ChildEnums.emplace_back();
-  I.ChildEnums.back().Name = "OneEnum";
+  I.Children.Records.emplace_back(EmptySID, "ChildStruct", InfoType::IT_record,
+  "path/to/A/r");
+  I.Children.Functions.emplace_back();
+  I.Children.Functions.back().Name = "OneFunction";
+  I.Children.Enums.emplace_back();
+  I.Children.Enums.back().Name = "OneEnum";
 
   auto G = getYAMLGenerator();
   assert(G);
@@ -330,6 +331,30 @@
   EXPECT_EQ(Expected, Actual.str());
 }
 
+TEST(YAMLGeneratorTest, enumTypedefYAML) {
+  TypedefInfo I;
+  I.Name = "MyUsing";
+  I.Underlying = TypeInfo("int");
+  I.IsUsing = true;
+
+  auto G = getYAMLGenerator();
+  assert(G);
+  std::string Buffer;
+  llvm::raw_string_ostream Actual(Buffer);
+  auto Err = G->generateDocForInfo(, Actual, ClangDocContext());
+  assert(!Err);
+  std::string Expected =
+  R"raw(---
+USR: ''
+Name:'MyUsing'
+Underlying:
+  Name:'int'
+IsUsing: true
+...
+)raw";
+  EXPECT_EQ(Expected, Actual.str());
+}
+
 TEST(YAMLGeneratorTest, emitCommentYAML) {
   FunctionInfo I;
   I.Name = "f";
Index: clang-tools-extra/unittests/clang-doc/SerializeTest.cpp
===
--- 

[PATCH] D134235: [clang-doc] Clean up *Info constructors.

[Brett Wilson via Phabricator via cfe-commits]

brettw added inline comments.



Comment at: clang-tools-extra/clang-doc/BitcodeReader.cpp:336
 Reference *I, FieldId ) {
   switch (ID) {
   case REFERENCE_USR:

haowei wrote:
> This block ID came from L582, which read from a bitcode file when 
> `BI_REFERENCE_BLOCK_ID` is encountered. I am not familiar with the content of 
> an arbitrary bitcode file. But after your patch, if this block ID is 
> encountered, it will result in an error (a behavior change).
> 
> It looks like `REFERENCE_IS_IN_GLOBAL_NAMESPACE` is only defined in in 
> clang-doc and not part of LLVM's bitcode header. So it looks like it is not a 
> standardized ID to me. Could you confirm it?
> 
> If `REFERENCE_IS_IN_GLOBAL_NAMESPACE` only presents in bitcode file generated 
> from BitcodeWriter from clang-doc and we are not expecting to provide 
> compatibility to an arbitrary bitcode generated from elsewhere, I am OK with 
> deleting this field (I still don't think this is the right approach to do but 
> I don't have a strong opinion on this either). 
Correct, this whole bitcode system is used only to communicate within clang-doc 
between threads (!) (I think it should be entirely removed but I don't have the 
time for that). It is never serialized to a file and is not shared with any 
other component.


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D134235/new/

https://reviews.llvm.org/D134235

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D134235: [clang-doc] Clean up *Info constructors.

[Brett Wilson via Phabricator via cfe-commits]

brettw added a comment.

I can guarantee that nobody uses this for anything useful because it doesn't 
work well enough. For example, before I started changing it, requesting HTML 
output crashed on startup and many of the most basic syntactic things were 
broken. My previous patch changed enums from being a list of dictionaries which 
is much more backwards incompatible than this. I don't think we need to worry 
about that.


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D134235/new/

https://reviews.llvm.org/D134235

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

Re: [PATCH] D134235: [clang-doc] Clean up *Info constructors.

[Brett Wilson via cfe-commits]

On Mon, Sep 26, 2022 at 1:42 PM Haowei Wu via Phabricator <
revi...@reviews.llvm.org> wrote:

> haowei added a comment.
>
> @brettw Thanks for the clarification. I was oncall for buildgardener last
> week and got quite busy due to breakages.
>
> Thanks for your clarification, that is really helpful to understand the
> your intent. While I still don't see the way these constructors are
> implemented is an issue. I don't have a strong opinions on this and OK with
> your refactors.
>
> "semantically equivalent" means these test code should have same
> input/output, performance same routines of task and use same amount of
> memory. Before your change, these typeinfo's symbolID in the test are
> always pointing to a constant EmptySID, after your change, some of these
> typeinfo points to an empty symbol ID in allocated in memory when typeinfo
> is created. While the output is the same, they are not equivalent. For
> refactor changes, I would prefer to avoid introducing such changes.
>
> I have one question about deleting the IsInGlobalNamespace. I understand
> it is not well implemented and has a bug in its constructor, but I don't
> think we should simply delete it just because it is buggy and not well
> used. It is part of C++ and commonly used. Do you have any plan to add it
> back after all the refactor work finishes?
>

I guess I'll ask you: what does this boolean mean and why do we need it?
This flag was only ever derived from the "path" which is just a member in
the same class. Users already have the full namespace information as far as
I can tell.

Brett
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D134235: [clang-doc] Clean up *Info constructors.

[Brett Wilson via Phabricator via cfe-commits]

brettw added inline comments.



Comment at: clang-tools-extra/clang-doc/Representation.h:202
+   bool IsFileInRootDir = false)
   : LineNumber(LineNumber), Filename(std::move(Filename)),
 IsFileInRootDir(IsFileInRootDir) {}

paulkirth wrote:
> do we still want to use `std::move` here now that its a StringRef? I don't 
> think we would do that w/ `const char*`, which is what `StringRef` boils down 
> to. It should be passed by value everywhere.
> 
> see 
> https://llvm.org/docs/ProgrammersManual.html#passing-strings-the-stringref-and-twine-classes,
>  
> https://llvm.org/docs/ProgrammersManual.html#the-stringref-class 
> https://llvm.org/docs/ProgrammersManual.html#string-like-containers
The move is unnecessary and I removed it.


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D134235/new/

https://reviews.llvm.org/D134235

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D134371: [clang-doc] Add typedef/using information.

[Brett Wilson via Phabricator via cfe-commits]

brettw added inline comments.



Comment at: clang-tools-extra/clang-doc/BitcodeReader.cpp:540
 }
-
 template <> void addChild(NamespaceInfo *I, EnumInfo &) {

paulkirth wrote:
> nit: can we avoid unrelated changes to  whitespace here and elsewhere in the 
> patch?
I did this on purpose. There were 8 variants of AddChild and I added two more. 
It became difficult to find the right one so I grouped them with comments for 
each group. This may be difficult to see in the side-by-side but it's much 
nicer in an editor.



Comment at: clang-tools-extra/clang-doc/Representation.h:280
 // Info for namespaces.
-struct NamespaceInfo : public Info {
+struct NamespaceInfo : public Info, public ScopeHasChildren {
   NamespaceInfo(SymbolID USR = SymbolID(), StringRef Name = StringRef(),

paulkirth wrote:
> It seems a bit odd to use inheritance here on a type w/ public fields, and no 
> methods. do you think using composition would improve the situation?
> 
>  I'm fine w/ it if we think this is the simpler/more maintainable solution, 
> but given that we don't have any methods in this case I'm unsure if there's 
> much benefit.
I don't have a strong opinion. I changed to composition.



Comment at: clang-tools-extra/clang-doc/Serialize.cpp:633
+  // both the parent and the record itself.
+  return {std::move(I), MakeAndInsertIntoParent(*I)};
 }

paulkirth wrote:
> Won't this deref the moved from `unique_ptr`? 
> 
> https://en.cppreference.com/w/cpp/language/eval_order
> > In list-initialization, every value computation and side effect of a given 
> > initializer clause is sequenced before every value computation and side 
> > effect associated with any initializer clause that follows it in the 
> > brace-enclosed comma-separated list of initializers.
> 
> Under that reading, the move //should// always happen first. If it happens to 
> still work, I think it's just luck in the implementation.
Thanks, nice catch.


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D134371/new/

https://reviews.llvm.org/D134371

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D134371: [clang-doc] Add typedef/using information.

[Brett Wilson via Phabricator via cfe-commits]

brettw updated this revision to Diff 462313.

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D134371/new/

https://reviews.llvm.org/D134371

Files:
  clang-tools-extra/clang-doc/BitcodeReader.cpp
  clang-tools-extra/clang-doc/BitcodeWriter.cpp
  clang-tools-extra/clang-doc/BitcodeWriter.h
  clang-tools-extra/clang-doc/HTMLGenerator.cpp
  clang-tools-extra/clang-doc/MDGenerator.cpp
  clang-tools-extra/clang-doc/Mapper.cpp
  clang-tools-extra/clang-doc/Mapper.h
  clang-tools-extra/clang-doc/Representation.cpp
  clang-tools-extra/clang-doc/Representation.h
  clang-tools-extra/clang-doc/Serialize.cpp
  clang-tools-extra/clang-doc/Serialize.h
  clang-tools-extra/clang-doc/YAMLGenerator.cpp
  clang-tools-extra/unittests/clang-doc/BitcodeTest.cpp
  clang-tools-extra/unittests/clang-doc/ClangDocTest.cpp
  clang-tools-extra/unittests/clang-doc/ClangDocTest.h
  clang-tools-extra/unittests/clang-doc/HTMLGeneratorTest.cpp
  clang-tools-extra/unittests/clang-doc/MDGeneratorTest.cpp
  clang-tools-extra/unittests/clang-doc/MergeTest.cpp
  clang-tools-extra/unittests/clang-doc/SerializeTest.cpp
  clang-tools-extra/unittests/clang-doc/YAMLGeneratorTest.cpp

Index: clang-tools-extra/unittests/clang-doc/YAMLGeneratorTest.cpp
===
--- clang-tools-extra/unittests/clang-doc/YAMLGeneratorTest.cpp
+++ clang-tools-extra/unittests/clang-doc/YAMLGeneratorTest.cpp
@@ -28,15 +28,16 @@
   I.Path = "path/to/A";
   I.Namespace.emplace_back(EmptySID, "A", InfoType::IT_namespace);
 
-  I.ChildNamespaces.emplace_back(EmptySID, "ChildNamespace",
- InfoType::IT_namespace, "path/to/A/Namespace");
-  I.ChildRecords.emplace_back(EmptySID, "ChildStruct", InfoType::IT_record,
-  "path/to/A/Namespace");
-  I.ChildFunctions.emplace_back();
-  I.ChildFunctions.back().Name = "OneFunction";
-  I.ChildFunctions.back().Access = AccessSpecifier::AS_none;
-  I.ChildEnums.emplace_back();
-  I.ChildEnums.back().Name = "OneEnum";
+  I.Children.Namespaces.emplace_back(EmptySID, "ChildNamespace",
+ InfoType::IT_namespace,
+ "path/to/A/Namespace");
+  I.Children.Records.emplace_back(EmptySID, "ChildStruct", InfoType::IT_record,
+  "path/to/A/Namespace");
+  I.Children.Functions.emplace_back();
+  I.Children.Functions.back().Name = "OneFunction";
+  I.Children.Functions.back().Access = AccessSpecifier::AS_none;
+  I.Children.Enums.emplace_back();
+  I.Children.Enums.back().Name = "OneEnum";
 
   auto G = getYAMLGenerator();
   assert(G);
@@ -100,8 +101,8 @@
   I.TagType = TagTypeKind::TTK_Class;
   I.Bases.emplace_back(EmptySID, "F", "path/to/F", true,
AccessSpecifier::AS_public, true);
-  I.Bases.back().ChildFunctions.emplace_back();
-  I.Bases.back().ChildFunctions.back().Name = "InheritedFunctionOne";
+  I.Bases.back().Children.Functions.emplace_back();
+  I.Bases.back().Children.Functions.back().Name = "InheritedFunctionOne";
   I.Bases.back().Members.emplace_back(TypeInfo("int", "path/to/int"), "N",
   AccessSpecifier::AS_private);
   // F is in the global namespace
@@ -109,12 +110,12 @@
   I.VirtualParents.emplace_back(EmptySID, "G", InfoType::IT_record,
 "path/to/G");
 
-  I.ChildRecords.emplace_back(EmptySID, "ChildStruct", InfoType::IT_record,
-  "path/to/A/r");
-  I.ChildFunctions.emplace_back();
-  I.ChildFunctions.back().Name = "OneFunction";
-  I.ChildEnums.emplace_back();
-  I.ChildEnums.back().Name = "OneEnum";
+  I.Children.Records.emplace_back(EmptySID, "ChildStruct", InfoType::IT_record,
+  "path/to/A/r");
+  I.Children.Functions.emplace_back();
+  I.Children.Functions.back().Name = "OneFunction";
+  I.Children.Enums.emplace_back();
+  I.Children.Enums.back().Name = "OneEnum";
 
   auto G = getYAMLGenerator();
   assert(G);
@@ -330,6 +331,30 @@
   EXPECT_EQ(Expected, Actual.str());
 }
 
+TEST(YAMLGeneratorTest, enumTypedefYAML) {
+  TypedefInfo I;
+  I.Name = "MyUsing";
+  I.Underlying = TypeInfo("int");
+  I.IsUsing = true;
+
+  auto G = getYAMLGenerator();
+  assert(G);
+  std::string Buffer;
+  llvm::raw_string_ostream Actual(Buffer);
+  auto Err = G->generateDocForInfo(, Actual, ClangDocContext());
+  assert(!Err);
+  std::string Expected =
+  R"raw(---
+USR: ''
+Name:'MyUsing'
+Underlying:
+  Name:'int'
+IsUsing: true
+...
+)raw";
+  EXPECT_EQ(Expected, Actual.str());
+}
+
 TEST(YAMLGeneratorTest, emitCommentYAML) {
   FunctionInfo I;
   I.Name = "f";
Index: clang-tools-extra/unittests/clang-doc/SerializeTest.cpp
===
--- clang-tools-extra/unittests/clang-doc/SerializeTest.cpp
+++ 

[PATCH] D134235: [clang-doc] Clean up *Info constructors.

[Brett Wilson via Phabricator via cfe-commits]

brettw updated this revision to Diff 462281.

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D134235/new/

https://reviews.llvm.org/D134235

Files:
  clang-tools-extra/clang-doc/BitcodeReader.cpp
  clang-tools-extra/clang-doc/BitcodeWriter.cpp
  clang-tools-extra/clang-doc/BitcodeWriter.h
  clang-tools-extra/clang-doc/HTMLGenerator.cpp
  clang-tools-extra/clang-doc/Representation.cpp
  clang-tools-extra/clang-doc/Representation.h
  clang-tools-extra/clang-doc/Serialize.cpp
  clang-tools-extra/clang-doc/YAMLGenerator.cpp
  clang-tools-extra/unittests/clang-doc/BitcodeTest.cpp
  clang-tools-extra/unittests/clang-doc/GeneratorTest.cpp
  clang-tools-extra/unittests/clang-doc/HTMLGeneratorTest.cpp
  clang-tools-extra/unittests/clang-doc/MDGeneratorTest.cpp
  clang-tools-extra/unittests/clang-doc/MergeTest.cpp
  clang-tools-extra/unittests/clang-doc/SerializeTest.cpp
  clang-tools-extra/unittests/clang-doc/YAMLGeneratorTest.cpp

Index: clang-tools-extra/unittests/clang-doc/YAMLGeneratorTest.cpp
===
--- clang-tools-extra/unittests/clang-doc/YAMLGeneratorTest.cpp
+++ clang-tools-extra/unittests/clang-doc/YAMLGeneratorTest.cpp
@@ -82,7 +82,7 @@
   I.DefLoc = Location(10, llvm::SmallString<16>{"test.cpp"});
   I.Loc.emplace_back(12, llvm::SmallString<16>{"test.cpp"});
 
-  I.Members.emplace_back("int", "path/to/int", "X",
+  I.Members.emplace_back(TypeInfo("int", "path/to/int"), "X",
  AccessSpecifier::AS_private);
 
   // Member documentation.
@@ -102,7 +102,7 @@
AccessSpecifier::AS_public, true);
   I.Bases.back().ChildFunctions.emplace_back();
   I.Bases.back().ChildFunctions.back().Name = "InheritedFunctionOne";
-  I.Bases.back().Members.emplace_back("int", "path/to/int", "N",
+  I.Bases.back().Members.emplace_back(TypeInfo("int", "path/to/int"), "N",
   AccessSpecifier::AS_private);
   // F is in the global namespace
   I.Parents.emplace_back(EmptySID, "F", InfoType::IT_record, "");
@@ -174,7 +174,6 @@
 Parents:
   - Type:Record
 Name:'F'
-IsInGlobalNamespace: true
 VirtualParents:
   - Type:Record
 Name:'G'
@@ -206,10 +205,10 @@
 
   I.Access = AccessSpecifier::AS_none;
 
-  I.ReturnType =
-  TypeInfo(EmptySID, "void", InfoType::IT_default, "path/to/void");
-  I.Params.emplace_back("int", "path/to/int", "P");
-  I.Params.emplace_back("double", "path/to/double", "D");
+  I.ReturnType = TypeInfo(
+  Reference(EmptySID, "void", InfoType::IT_default, "path/to/void"));
+  I.Params.emplace_back(TypeInfo("int", "path/to/int"), "P");
+  I.Params.emplace_back(TypeInfo("double", "path/to/double"), "D");
   I.Params.back().DefaultValue = "2.0 * M_PI";
   I.IsMethod = true;
   I.Parent = Reference(EmptySID, "Parent", InfoType::IT_record);
@@ -335,9 +334,9 @@
   FunctionInfo I;
   I.Name = "f";
   I.DefLoc = Location(10, llvm::SmallString<16>{"test.cpp"});
-  I.ReturnType = TypeInfo(EmptySID, "void", InfoType::IT_default);
-  I.Params.emplace_back("int", "I");
-  I.Params.emplace_back("int", "J");
+  I.ReturnType = TypeInfo("void");
+  I.Params.emplace_back(TypeInfo("int"), "I");
+  I.Params.emplace_back(TypeInfo("int"), "J");
   I.Access = AccessSpecifier::AS_none;
 
   CommentInfo Top;
Index: clang-tools-extra/unittests/clang-doc/SerializeTest.cpp
===
--- clang-tools-extra/unittests/clang-doc/SerializeTest.cpp
+++ clang-tools-extra/unittests/clang-doc/SerializeTest.cpp
@@ -119,7 +119,7 @@
   NamespaceInfo ExpectedBWithFunction(EmptySID);
   FunctionInfo F;
   F.Name = "f";
-  F.ReturnType = TypeInfo(EmptySID, "void", InfoType::IT_default);
+  F.ReturnType = TypeInfo("void");
   F.DefLoc = Location(0, llvm::SmallString<16>{"test.cpp"});
   F.Namespace.emplace_back(EmptySID, "B", InfoType::IT_namespace);
   F.Namespace.emplace_back(EmptySID, "A", InfoType::IT_namespace);
@@ -165,7 +165,8 @@
InfoType::IT_namespace);
   ExpectedE.TagType = TagTypeKind::TTK_Class;
   ExpectedE.DefLoc = Location(0, llvm::SmallString<16>{"test.cpp"});
-  ExpectedE.Members.emplace_back("int", "value", AccessSpecifier::AS_public);
+  ExpectedE.Members.emplace_back(TypeInfo("int"), "value",
+ AccessSpecifier::AS_public);
   // TODO the data member should have the docstring on it:
   //ExpectedE.Members.back().Description.push_back(MakeOneLineCommentInfo(" Some docs"));
   CheckRecordInfo(, E);
@@ -175,7 +176,7 @@
   FunctionInfo EConstructor;
   EConstructor.Name = "E";
   EConstructor.Parent = Reference(EmptySID, "E", InfoType::IT_record);
-  EConstructor.ReturnType = TypeInfo(EmptySID, "void", InfoType::IT_default);
+  EConstructor.ReturnType = TypeInfo("void");
   EConstructor.DefLoc = Location(0, llvm::SmallString<16>{"test.cpp"});
   EConstructor.Namespace.emplace_back(EmptySID, "E", 

[PATCH] D134235: [clang-doc] Clean up *Info constructors.

[Brett Wilson via Phabricator via cfe-commits]

brettw marked an inline comment as done.
brettw added a comment.

If you have more questions, I can schedule a meeting.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D134235/new/

https://reviews.llvm.org/D134235

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D134235: [clang-doc] Clean up *Info constructors.

[Brett Wilson via Phabricator via cfe-commits]

brettw marked an inline comment as done.
brettw added a comment.

First, for the mechanical part of the change, this code implement multiple 
constructors:

  Info() = default;
  Info(InfoType IT) : IT(IT) {}
  Info(InfoType IT, SymbolID USR) : USR(USR), IT(IT) {}
  Info(InfoType IT, SymbolID USR, StringRef Name)
      : USR(USR), IT(IT), Name(Name) {}
  Info(InfoType IT, SymbolID USR, StringRef Name, StringRef Path)

is not common practice. This is an extremely verbose and error-prone way of 
implementing default arguments. It seems obvious to me that the one constructor 
that implements the exact same behavior is much more clear. Most of this patch 
is doing this mechanical change for these objects.

There are a few cases in this patch where I removed some constructors that took 
some "other" subset of parameters.

  Reference() = default;
  Reference(llvm::StringRef Name) : Name(Name) {}
  Reference(llvm::StringRef Name, StringRef Path)
      : Name(Name), Path(Path), IsInGlobalNamespace(Path.empty()) {}
  Reference(SymbolID USR, StringRef Name, InfoType IT)
      : USR(USR), Name(Name), RefType(IT) {}
  Reference(SymbolID USR, StringRef Name, InfoType IT, StringRef Path)
      : USR(USR), Name(Name), RefType(IT), Path(Path),
        IsInGlobalNamespace(Path.empty()) {}

Working on this code, I found the number of constructors with different 
parameters to make things more difficult to follow. It's harder to figure out 
what exactly a line of code is doing when there are so many variants you have 
to match against. It's also more difficult to figure out how to create one of 
these objects when you have to read so many variants to see what to call. This 
mental overhead may not be obvious when just looking at a diff, especially the 
test code where you may see an extra empty SID parameter inserted here and 
there. This new one tries to have a single constructor with default arguments 
from the right, with the exception of the "TypeInfo" that creates a single 
named type with no SymbolID because that's by far the most common variant in 
tests.

With this Reference constructor in particular, all of these variants hide a 
bug. If I didn't call that bug out in the change description, would you have 
seen it? Having a standard constructor makes this much easier to follow. 
Changing several callers to manually specify an empty SymbolID() seems like a 
small price to pay (I think this is the only case where any calls get "more 
complicated."

I tried to standardize on a single way to construct these objects. As an 
example, the TypeInfo is just a wrapper around a Reference. Reference has many 
constructors, not all of which are a subset of the others. Does this mean 
TypeInfo should get these same 5 constructor variants? The author said no, they 
picked 3 of the variants and implemented TypeInfo constructors, and used 
different names for some of the parameters which makes it even harder to 
follow. Continuing down the chain, FieldTypeInfo derives from TypeInfo, and 
MemberTypeInfo derives from FieldTypeInfo. Does this mean they should get the 
same 5 variants, plus initializers for their own values? I would say no. If 
these derived classes needed to specify the Reference information, they should 
take a Reference as the first parameter (they don't all need to do this today).

There's a problem with the current object model. The FieldTypeInfo and 
MemberTypeInfo derive from TypeInfo. But I think that this is wrong. The 
FieldTypeInfo isn't the type information for a field, it's really a "Field" and 
a MemberTypeInfo isn't the type info for a member, it's the "Member" itself. 
Conceptually, FieldTypeInfo should be a FieldInfo and have a struct `TypeInfo 
Type;` rather than using an "is a" relationship and deriving from a type. Given 
that, this object naming and constructor "should" be:

  MemberInfo(TypeInfo Type, StringRef Name, AccessSpecifier Access = Public)

This patch gets us closer to this, where we still have the weird "is a" 
derivation but the constructors match what I think they should be:

  MemberTypeInfo() = default;
  MemberTypeInfo(const TypeInfo , StringRef Name, AccessSpecifier Access)

The parameters that define the implementation details of how the type is 
represented don't belong on this class: if the caller wants to create a type, 
they should create it using one of the appropriate TypeInfo constructors rather 
than each variant being duplicated here.




Comment at: clang-tools-extra/clang-doc/Representation.h:165
-  : Type(Type, Field, IT) {}
-  TypeInfo(SymbolID Type, StringRef Field, InfoType IT, StringRef Path)
-  : Type(Type, Field, IT, Path) {}

haowei wrote:
> The clean up here prevents setting a customized SymbolID for TypeInfo, which 
> becomes a problem in the HTMLGeneratorTest, in which TypeInfo is constructed 
> with an constant EmptySID. While the outcome is the same, it is not 
> semantically equivalent.  
It does not prevent 

[PATCH] D134371: [clang-doc] Add typedef/using information.

[Brett Wilson via Phabricator via cfe-commits]

brettw created this revision.
brettw added a reviewer: paulkirth.
brettw added a project: clang-tools-extra.
Herald added a subscriber: arphaman.
Herald added a project: All.
brettw requested review of this revision.
Herald added a subscriber: cfe-commits.

Read typedef and "using" type alias declarations and serialize into the 
internal structures. Emit this information in the YAML output. The HTML and MD 
generators are unchanged.

Separate out the logic to create the parent namespace or record object and 
insert the newly created child into it. This logic was previously duplicated 
for every "info" type and is now shared.

To help this, a struct containing the child vectors was separated out so 
children can be added generically and without having too many templates.

A small change was made to populateParentNamespaces() to allow using types that 
aren't themselves DeclContexts (typedefs are the first example of this).


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D134371

Files:
  clang-tools-extra/clang-doc/BitcodeReader.cpp
  clang-tools-extra/clang-doc/BitcodeWriter.cpp
  clang-tools-extra/clang-doc/BitcodeWriter.h
  clang-tools-extra/clang-doc/Mapper.cpp
  clang-tools-extra/clang-doc/Mapper.h
  clang-tools-extra/clang-doc/Representation.cpp
  clang-tools-extra/clang-doc/Representation.h
  clang-tools-extra/clang-doc/Serialize.cpp
  clang-tools-extra/clang-doc/Serialize.h
  clang-tools-extra/clang-doc/YAMLGenerator.cpp
  clang-tools-extra/unittests/clang-doc/BitcodeTest.cpp
  clang-tools-extra/unittests/clang-doc/ClangDocTest.cpp
  clang-tools-extra/unittests/clang-doc/ClangDocTest.h
  clang-tools-extra/unittests/clang-doc/SerializeTest.cpp
  clang-tools-extra/unittests/clang-doc/YAMLGeneratorTest.cpp

Index: clang-tools-extra/unittests/clang-doc/YAMLGeneratorTest.cpp
===
--- clang-tools-extra/unittests/clang-doc/YAMLGeneratorTest.cpp
+++ clang-tools-extra/unittests/clang-doc/YAMLGeneratorTest.cpp
@@ -330,6 +330,30 @@
   EXPECT_EQ(Expected, Actual.str());
 }
 
+TEST(YAMLGeneratorTest, enumTypedefYAML) {
+  TypedefInfo I;
+  I.Name = "MyUsing";
+  I.Underlying = TypeInfo("int");
+  I.IsUsing = true;
+
+  auto G = getYAMLGenerator();
+  assert(G);
+  std::string Buffer;
+  llvm::raw_string_ostream Actual(Buffer);
+  auto Err = G->generateDocForInfo(, Actual, ClangDocContext());
+  assert(!Err);
+  std::string Expected =
+  R"raw(---
+USR: ''
+Name:'MyUsing'
+Underlying:
+  Name:'int'
+IsUsing: true
+...
+)raw";
+  EXPECT_EQ(Expected, Actual.str());
+}
+
 TEST(YAMLGeneratorTest, emitCommentYAML) {
   FunctionInfo I;
   I.Name = "f";
Index: clang-tools-extra/unittests/clang-doc/SerializeTest.cpp
===
--- clang-tools-extra/unittests/clang-doc/SerializeTest.cpp
+++ clang-tools-extra/unittests/clang-doc/SerializeTest.cpp
@@ -59,6 +59,10 @@
   bool VisitRecordDecl(const RecordDecl *D) { return mapDecl(D); }
 
   bool VisitEnumDecl(const EnumDecl *D) { return mapDecl(D); }
+
+  bool VisitTypedefDecl(const TypedefDecl *D) { return mapDecl(D); }
+
+  bool VisitTypeAliasDecl(const TypeAliasDecl *D) { return mapDecl(D); }
 };
 
 void ExtractInfosFromCode(StringRef Code, size_t NumExpectedInfos, bool Public,
@@ -591,5 +595,32 @@
   CheckNamespaceInfo(, ParentB);
 }
 
+TEST(SerializeTests, emitTypedefs) {
+  EmittedInfoList Infos;
+  ExtractInfosFromCode("typedef int MyInt; using MyDouble = double;", 2,
+   /*Public=*/false, Infos);
+
+  // First info will be the global namespace with the typedef in it.
+  NamespaceInfo *GlobalNS1 = InfoAsNamespace(Infos[0].get());
+  ASSERT_EQ(1u, GlobalNS1->ChildTypedefs.size());
+
+  const TypedefInfo  = GlobalNS1->ChildTypedefs[0];
+  EXPECT_EQ("MyInt", FirstTD.Name);
+  EXPECT_FALSE(FirstTD.IsUsing);
+  EXPECT_EQ("int", FirstTD.Underlying.Type.Name);
+
+  // The second will be another global namespace with the using in it (the
+  // global namespace is duplicated because the items haven't been merged at the
+  // serialization phase of processing).
+  NamespaceInfo *GlobalNS2 = InfoAsNamespace(Infos[1].get());
+  ASSERT_EQ(1u, GlobalNS2->ChildTypedefs.size());
+
+  // Second is the "using" typedef.
+  const TypedefInfo  = GlobalNS2->ChildTypedefs[0];
+  EXPECT_EQ("MyDouble", SecondTD.Name);
+  EXPECT_TRUE(SecondTD.IsUsing);
+  EXPECT_EQ("double", SecondTD.Underlying.Type.Name);
+}
+
 } // namespace doc
 } // end namespace clang
Index: clang-tools-extra/unittests/clang-doc/ClangDocTest.h
===
--- clang-tools-extra/unittests/clang-doc/ClangDocTest.h
+++ clang-tools-extra/unittests/clang-doc/ClangDocTest.h
@@ -27,6 +27,7 @@
 RecordInfo *InfoAsRecord(Info *I);
 FunctionInfo *InfoAsFunction(Info *I);
 EnumInfo *InfoAsEnum(Info *I);
+TypedefInfo *InfoAsTypedef(Info 

[PATCH] D134235: [clang-doc] Clean up *Info constructors.

[Brett Wilson via Phabricator via cfe-commits]

brettw created this revision.
Herald added a project: All.
brettw requested review of this revision.
Herald added a project: clang-tools-extra.
Herald added a subscriber: cfe-commits.

The *Info object (for the copy of the AST") constructors had many duplicated 
variants. Many of the variants seemed to be in an attempt to avoid default 
arguments. But default arguments are not prohibited and using them allows most 
of the variants to be removed which improves readability.

Remove the IsInGlobalNamespace flag on a Reference. This is set when the path 
is empty, and only read once in the HTML generator with the identical 
condition. The constructor cleanup exposed a problem where this was set to 
false when the constructor with no path was used, but true when the path was 
set to empty.

There should be no observable change with the exception that 
IsInGlobalNamespace is no longer emitted in YAML.


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D134235

Files:
  clang-tools-extra/clang-doc/BitcodeReader.cpp
  clang-tools-extra/clang-doc/BitcodeWriter.cpp
  clang-tools-extra/clang-doc/BitcodeWriter.h
  clang-tools-extra/clang-doc/HTMLGenerator.cpp
  clang-tools-extra/clang-doc/Representation.cpp
  clang-tools-extra/clang-doc/Representation.h
  clang-tools-extra/clang-doc/Serialize.cpp
  clang-tools-extra/clang-doc/YAMLGenerator.cpp
  clang-tools-extra/unittests/clang-doc/BitcodeTest.cpp
  clang-tools-extra/unittests/clang-doc/GeneratorTest.cpp
  clang-tools-extra/unittests/clang-doc/HTMLGeneratorTest.cpp
  clang-tools-extra/unittests/clang-doc/MDGeneratorTest.cpp
  clang-tools-extra/unittests/clang-doc/MergeTest.cpp
  clang-tools-extra/unittests/clang-doc/SerializeTest.cpp
  clang-tools-extra/unittests/clang-doc/YAMLGeneratorTest.cpp

Index: clang-tools-extra/unittests/clang-doc/YAMLGeneratorTest.cpp
===
--- clang-tools-extra/unittests/clang-doc/YAMLGeneratorTest.cpp
+++ clang-tools-extra/unittests/clang-doc/YAMLGeneratorTest.cpp
@@ -82,7 +82,7 @@
   I.DefLoc = Location(10, llvm::SmallString<16>{"test.cpp"});
   I.Loc.emplace_back(12, llvm::SmallString<16>{"test.cpp"});
 
-  I.Members.emplace_back("int", "path/to/int", "X",
+  I.Members.emplace_back(TypeInfo("int", "path/to/int"), "X",
  AccessSpecifier::AS_private);
 
   // Member documentation.
@@ -102,7 +102,7 @@
AccessSpecifier::AS_public, true);
   I.Bases.back().ChildFunctions.emplace_back();
   I.Bases.back().ChildFunctions.back().Name = "InheritedFunctionOne";
-  I.Bases.back().Members.emplace_back("int", "path/to/int", "N",
+  I.Bases.back().Members.emplace_back(TypeInfo("int", "path/to/int"), "N",
   AccessSpecifier::AS_private);
   // F is in the global namespace
   I.Parents.emplace_back(EmptySID, "F", InfoType::IT_record, "");
@@ -174,7 +174,6 @@
 Parents:
   - Type:Record
 Name:'F'
-IsInGlobalNamespace: true
 VirtualParents:
   - Type:Record
 Name:'G'
@@ -206,10 +205,10 @@
 
   I.Access = AccessSpecifier::AS_none;
 
-  I.ReturnType =
-  TypeInfo(EmptySID, "void", InfoType::IT_default, "path/to/void");
-  I.Params.emplace_back("int", "path/to/int", "P");
-  I.Params.emplace_back("double", "path/to/double", "D");
+  I.ReturnType = TypeInfo(
+  Reference(EmptySID, "void", InfoType::IT_default, "path/to/void"));
+  I.Params.emplace_back(TypeInfo("int", "path/to/int"), "P");
+  I.Params.emplace_back(TypeInfo("double", "path/to/double"), "D");
   I.Params.back().DefaultValue = "2.0 * M_PI";
   I.IsMethod = true;
   I.Parent = Reference(EmptySID, "Parent", InfoType::IT_record);
@@ -335,9 +334,9 @@
   FunctionInfo I;
   I.Name = "f";
   I.DefLoc = Location(10, llvm::SmallString<16>{"test.cpp"});
-  I.ReturnType = TypeInfo(EmptySID, "void", InfoType::IT_default);
-  I.Params.emplace_back("int", "I");
-  I.Params.emplace_back("int", "J");
+  I.ReturnType = TypeInfo("void");
+  I.Params.emplace_back(TypeInfo("int"), "I");
+  I.Params.emplace_back(TypeInfo("int"), "J");
   I.Access = AccessSpecifier::AS_none;
 
   CommentInfo Top;
Index: clang-tools-extra/unittests/clang-doc/SerializeTest.cpp
===
--- clang-tools-extra/unittests/clang-doc/SerializeTest.cpp
+++ clang-tools-extra/unittests/clang-doc/SerializeTest.cpp
@@ -119,7 +119,7 @@
   NamespaceInfo ExpectedBWithFunction(EmptySID);
   FunctionInfo F;
   F.Name = "f";
-  F.ReturnType = TypeInfo(EmptySID, "void", InfoType::IT_default);
+  F.ReturnType = TypeInfo("void");
   F.DefLoc = Location(0, llvm::SmallString<16>{"test.cpp"});
   F.Namespace.emplace_back(EmptySID, "B", InfoType::IT_namespace);
   F.Namespace.emplace_back(EmptySID, "A", InfoType::IT_namespace);
@@ -165,7 +165,8 @@
InfoType::IT_namespace);
   ExpectedE.TagType = TagTypeKind::TTK_Class;
   ExpectedE.DefLoc = 

[PATCH] D134225: [clang-doc] Centralize TypeInfo creation.

[Brett Wilson via Phabricator via cfe-commits]

brettw updated this revision to Diff 461397.
brettw marked an inline comment as done.
brettw added a comment.

Comment addressed


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D134225/new/

https://reviews.llvm.org/D134225

Files:
  clang-tools-extra/clang-doc/Representation.h
  clang-tools-extra/clang-doc/Serialize.cpp

Index: clang-tools-extra/clang-doc/Serialize.cpp
===
--- clang-tools-extra/clang-doc/Serialize.cpp
+++ clang-tools-extra/clang-doc/Serialize.cpp
@@ -224,12 +224,35 @@
   return hashUSR(USR);
 }
 
-static RecordDecl *getDeclForType(const QualType ) {
+static TagDecl *getTagDeclForType(const QualType ) {
+  if (const TagDecl *D = T->getAsTagDecl())
+return D->getDefinition();
+  return nullptr;
+}
+
+static RecordDecl *getRecordDeclForType(const QualType ) {
   if (const RecordDecl *D = T->getAsRecordDecl())
 return D->getDefinition();
   return nullptr;
 }
 
+TypeInfo getTypeInfoForType(const QualType ) {
+  const TagDecl *TD = getTagDeclForType(T);
+  if (!TD)
+return TypeInfo(Reference(T.getAsString()));
+
+  InfoType IT;
+  if (dyn_cast(TD)) {
+IT = InfoType::IT_enum;
+  } else if (dyn_cast(TD)) {
+IT = InfoType::IT_record;
+  } else {
+IT = InfoType::IT_default;
+  }
+  return TypeInfo(Reference(getUSRForDecl(TD), TD->getNameAsString(), IT,
+getInfoRelativePath(TD)));
+}
+
 static bool isPublic(const clang::AccessSpecifier AS,
  const clang::Linkage Link) {
   if (AS == clang::AccessSpecifier::AS_private)
@@ -286,28 +309,14 @@
   for (const FieldDecl *F : D->fields()) {
 if (!shouldSerializeInfo(PublicOnly, /*IsInAnonymousNamespace=*/false, F))
   continue;
-if (const auto *T = getDeclForType(F->getTypeSourceInfo()->getType())) {
-  // Use getAccessUnsafe so that we just get the default AS_none if it's not
-  // valid, as opposed to an assert.
-  if (const auto *N = dyn_cast(T)) {
-I.Members.emplace_back(
-getUSRForDecl(T), N->getNameAsString(), InfoType::IT_enum,
-getInfoRelativePath(N), F->getNameAsString(),
-getFinalAccessSpecifier(Access, N->getAccessUnsafe()));
-continue;
-  } else if (const auto *N = dyn_cast(T)) {
-I.Members.emplace_back(
-getUSRForDecl(T), N->getNameAsString(), InfoType::IT_record,
-getInfoRelativePath(N), F->getNameAsString(),
-getFinalAccessSpecifier(Access, N->getAccessUnsafe()));
-continue;
-  }
-}
 
-auto& member = I.Members.emplace_back(
-F->getTypeSourceInfo()->getType().getAsString(), F->getNameAsString(),
+// Use getAccessUnsafe so that we just get the default AS_none if it's not
+// valid, as opposed to an assert.
+MemberTypeInfo  = I.Members.emplace_back(
+getTypeInfoForType(F->getTypeSourceInfo()->getType()),
+F->getNameAsString(),
 getFinalAccessSpecifier(Access, F->getAccessUnsafe()));
-populateMemberTypeInfo(member, F);
+populateMemberTypeInfo(NewMember, F);
   }
 }
 
@@ -325,27 +334,11 @@
 
 static void parseParameters(FunctionInfo , const FunctionDecl *D) {
   for (const ParmVarDecl *P : D->parameters()) {
-FieldTypeInfo *FieldInfo = nullptr;
-if (const auto *T = getDeclForType(P->getOriginalType())) {
-  if (const auto *N = dyn_cast(T)) {
-FieldInfo = _back(
-getUSRForDecl(N), N->getNameAsString(), InfoType::IT_enum,
-getInfoRelativePath(N), P->getNameAsString());
-  } else if (const auto *N = dyn_cast(T)) {
-FieldInfo = _back(
-getUSRForDecl(N), N->getNameAsString(), InfoType::IT_record,
-getInfoRelativePath(N), P->getNameAsString());
-  }
-  // Otherwise fall through to the default case below.
-}
-
-if (!FieldInfo) {
-  FieldInfo = _back(P->getOriginalType().getAsString(),
- P->getNameAsString());
-}
+FieldTypeInfo  = I.Params.emplace_back(
+getTypeInfoForType(P->getOriginalType()), P->getNameAsString());
 
 if (const Expr *DefaultArg = P->getDefaultArg()) {
-  FieldInfo->DefaultValue = getSourceCode(D, DefaultArg->getSourceRange());
+  FieldInfo.DefaultValue = getSourceCode(D, DefaultArg->getSourceRange());
 }
   }
 }
@@ -363,14 +356,14 @@
   const TemplateDecl *D = Ty->getTemplateName().getAsTemplateDecl();
   I.Parents.emplace_back(getUSRForDecl(D), B.getType().getAsString(),
  InfoType::IT_record);
-} else if (const RecordDecl *P = getDeclForType(B.getType()))
+} else if (const RecordDecl *P = getRecordDeclForType(B.getType()))
   I.Parents.emplace_back(getUSRForDecl(P), P->getNameAsString(),
  InfoType::IT_record, getInfoRelativePath(P));
 else
   I.Parents.emplace_back(B.getType().getAsString());
   }
   for (const CXXBaseSpecifier  : D->vbases()) {
- 

[PATCH] D134055: [clang-doc] Add support for explicitly typed enums

[Brett Wilson via Phabricator via cfe-commits]

brettw updated this revision to Diff 461366.
brettw added a comment.

Clang-formatted


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D134055/new/

https://reviews.llvm.org/D134055

Files:
  clang-tools-extra/clang-doc/BitcodeReader.cpp
  clang-tools-extra/clang-doc/BitcodeWriter.cpp
  clang-tools-extra/clang-doc/BitcodeWriter.h
  clang-tools-extra/clang-doc/HTMLGenerator.cpp
  clang-tools-extra/clang-doc/MDGenerator.cpp
  clang-tools-extra/clang-doc/Representation.h
  clang-tools-extra/clang-doc/Serialize.cpp
  clang-tools-extra/clang-doc/YAMLGenerator.cpp
  clang-tools-extra/unittests/clang-doc/SerializeTest.cpp
  clang-tools-extra/unittests/clang-doc/YAMLGeneratorTest.cpp

Index: clang-tools-extra/unittests/clang-doc/YAMLGeneratorTest.cpp
===
--- clang-tools-extra/unittests/clang-doc/YAMLGeneratorTest.cpp
+++ clang-tools-extra/unittests/clang-doc/YAMLGeneratorTest.cpp
@@ -256,7 +256,11 @@
   EXPECT_EQ(Expected, Actual.str());
 }
 
-TEST(YAMLGeneratorTest, emitEnumYAML) {
+// Tests the equivalent of:
+// namespace A {
+// enum e { X };
+// }
+TEST(YAMLGeneratorTest, emitSimpleEnumYAML) {
   EnumInfo I;
   I.Name = "e";
   I.Namespace.emplace_back(EmptySID, "A", InfoType::IT_namespace);
@@ -265,7 +269,7 @@
   I.Loc.emplace_back(12, llvm::SmallString<16>{"test.cpp"});
 
   I.Members.emplace_back("X");
-  I.Scoped = true;
+  I.Scoped = false;
 
   auto G = getYAMLGenerator();
   assert(G);
@@ -286,9 +290,42 @@
 Location:
   - LineNumber:  12
 Filename:'test.cpp'
+Members:
+  - Name:'X'
+Value:   '0'
+...
+)raw";
+  EXPECT_EQ(Expected, Actual.str());
+}
+
+// Tests the equivalent of:
+// enum class e : short { X = FOO_BAR + 2 };
+TEST(YAMLGeneratorTest, enumTypedScopedEnumYAML) {
+  EnumInfo I;
+  I.Name = "e";
+
+  I.Members.emplace_back("X", "-9876", "FOO_BAR + 2");
+  I.Scoped = true;
+  I.BaseType = TypeInfo("short");
+
+  auto G = getYAMLGenerator();
+  assert(G);
+  std::string Buffer;
+  llvm::raw_string_ostream Actual(Buffer);
+  auto Err = G->generateDocForInfo(, Actual, ClangDocContext());
+  assert(!Err);
+  std::string Expected =
+  R"raw(---
+USR: ''
+Name:'e'
 Scoped:  true
+BaseType:
+  Type:
+Name:'short'
 Members:
-  - 'X'
+  - Name:'X'
+Value:   '-9876'
+Expr:'FOO_BAR + 2'
 ...
 )raw";
   EXPECT_EQ(Expected, Actual.str());
Index: clang-tools-extra/unittests/clang-doc/SerializeTest.cpp
===
--- clang-tools-extra/unittests/clang-doc/SerializeTest.cpp
+++ clang-tools-extra/unittests/clang-doc/SerializeTest.cpp
@@ -266,8 +266,8 @@
   EnumInfo E;
   E.Name = "E";
   E.DefLoc = Location(0, llvm::SmallString<16>{"test.cpp"});
-  E.Members.emplace_back("X");
-  E.Members.emplace_back("Y");
+  E.Members.emplace_back("X", "0");
+  E.Members.emplace_back("Y", "1");
   ExpectedNamespaceWithEnum.ChildEnums.emplace_back(std::move(E));
   CheckNamespaceInfo(, NamespaceWithEnum);
 
@@ -277,8 +277,8 @@
   G.Name = "G";
   G.Scoped = true;
   G.DefLoc = Location(0, llvm::SmallString<16>{"test.cpp"});
-  G.Members.emplace_back("A");
-  G.Members.emplace_back("B");
+  G.Members.emplace_back("A", "0");
+  G.Members.emplace_back("B", "1");
   ExpectedNamespaceWithScopedEnum.ChildEnums.emplace_back(std::move(G));
   CheckNamespaceInfo(, NamespaceWithScopedEnum);
 }
Index: clang-tools-extra/clang-doc/YAMLGenerator.cpp
===
--- clang-tools-extra/clang-doc/YAMLGenerator.cpp
+++ clang-tools-extra/clang-doc/YAMLGenerator.cpp
@@ -14,6 +14,7 @@
 
 using namespace clang::doc;
 
+// These define YAML traits for decoding the listed values within a vector.
 LLVM_YAML_IS_SEQUENCE_VECTOR(FieldTypeInfo)
 LLVM_YAML_IS_SEQUENCE_VECTOR(MemberTypeInfo)
 LLVM_YAML_IS_SEQUENCE_VECTOR(Reference)
@@ -21,6 +22,7 @@
 LLVM_YAML_IS_SEQUENCE_VECTOR(CommentInfo)
 LLVM_YAML_IS_SEQUENCE_VECTOR(FunctionInfo)
 LLVM_YAML_IS_SEQUENCE_VECTOR(EnumInfo)
+LLVM_YAML_IS_SEQUENCE_VECTOR(EnumValueInfo)
 LLVM_YAML_IS_SEQUENCE_VECTOR(BaseRecordInfo)
 LLVM_YAML_IS_SEQUENCE_VECTOR(std::unique_ptr)
 LLVM_YAML_IS_SEQUENCE_VECTOR(llvm::SmallString<16>)
@@ -226,10 +228,19 @@
   }
 };
 
+template <> struct MappingTraits {
+  static void mapping(IO , EnumValueInfo ) {
+IO.mapOptional("Name", I.Name);
+IO.mapOptional("Value", I.Value);
+IO.mapOptional("Expr", I.ValueExpr, SmallString<16>());
+  }
+};
+
 template <> struct MappingTraits {
   static void mapping(IO , EnumInfo ) {
 SymbolInfoMapping(IO, I);
 IO.mapOptional("Scoped", I.Scoped, false);
+IO.mapOptional("BaseType", I.BaseType);
 IO.mapOptional("Members", I.Members);
   }
 };
Index: clang-tools-extra/clang-doc/Serialize.cpp
===
--- 

[PATCH] D134225: [clang-doc] Centralize TypeInfo creation.

[Brett Wilson via Phabricator via cfe-commits]

brettw created this revision.
brettw added a reviewer: paulkirth.
brettw added a project: clang-tools-extra.
Herald added a project: All.
brettw requested review of this revision.
Herald added a subscriber: cfe-commits.

Several different places in the code had similar computations for the 
parameters that were eventually passed to the TypeInfo constructor.

  

This centralizes that code in one function, and allows passing TypeInfo to the 
various other *Info structures that need it.

  

Remove some "auto" types and replace with the real type for getting 
declarations. This was making some duplicate checking difficult to see.


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D134225

Files:
  clang-tools-extra/clang-doc/Representation.h
  clang-tools-extra/clang-doc/Serialize.cpp

Index: clang-tools-extra/clang-doc/Serialize.cpp
===
--- clang-tools-extra/clang-doc/Serialize.cpp
+++ clang-tools-extra/clang-doc/Serialize.cpp
@@ -224,12 +224,34 @@
   return hashUSR(USR);
 }
 
-static RecordDecl *getDeclForType(const QualType ) {
+static TagDecl *getTagDeclForType(const QualType ) {
+  if (const TagDecl *D = T->getAsTagDecl())
+return D->getDefinition();
+  return nullptr;
+}
+
+static RecordDecl *getRecordDeclForType(const QualType ) {
   if (const RecordDecl *D = T->getAsRecordDecl())
 return D->getDefinition();
   return nullptr;
 }
 
+TypeInfo getTypeInfoForType(const QualType ) {
+  if (const TagDecl *TD = getTagDeclForType(T)) {
+InfoType IT;
+if (dyn_cast(TD)) {
+  IT = InfoType::IT_enum;
+} else if (dyn_cast(TD)) {
+  IT = InfoType::IT_record;
+} else {
+  IT = InfoType::IT_default;
+}
+return TypeInfo(Reference(getUSRForDecl(TD), TD->getNameAsString(),
+  IT, getInfoRelativePath(TD)));
+  }
+  return TypeInfo(Reference(T.getAsString()));
+}
+
 static bool isPublic(const clang::AccessSpecifier AS,
  const clang::Linkage Link) {
   if (AS == clang::AccessSpecifier::AS_private)
@@ -286,28 +308,14 @@
   for (const FieldDecl *F : D->fields()) {
 if (!shouldSerializeInfo(PublicOnly, /*IsInAnonymousNamespace=*/false, F))
   continue;
-if (const auto *T = getDeclForType(F->getTypeSourceInfo()->getType())) {
-  // Use getAccessUnsafe so that we just get the default AS_none if it's not
-  // valid, as opposed to an assert.
-  if (const auto *N = dyn_cast(T)) {
-I.Members.emplace_back(
-getUSRForDecl(T), N->getNameAsString(), InfoType::IT_enum,
-getInfoRelativePath(N), F->getNameAsString(),
-getFinalAccessSpecifier(Access, N->getAccessUnsafe()));
-continue;
-  } else if (const auto *N = dyn_cast(T)) {
-I.Members.emplace_back(
-getUSRForDecl(T), N->getNameAsString(), InfoType::IT_record,
-getInfoRelativePath(N), F->getNameAsString(),
-getFinalAccessSpecifier(Access, N->getAccessUnsafe()));
-continue;
-  }
-}
 
-auto& member = I.Members.emplace_back(
-F->getTypeSourceInfo()->getType().getAsString(), F->getNameAsString(),
+// Use getAccessUnsafe so that we just get the default AS_none if it's not
+// valid, as opposed to an assert.
+MemberTypeInfo  = I.Members.emplace_back(
+getTypeInfoForType(F->getTypeSourceInfo()->getType()),
+F->getNameAsString(),
 getFinalAccessSpecifier(Access, F->getAccessUnsafe()));
-populateMemberTypeInfo(member, F);
+populateMemberTypeInfo(NewMember, F);
   }
 }
 
@@ -325,27 +333,12 @@
 
 static void parseParameters(FunctionInfo , const FunctionDecl *D) {
   for (const ParmVarDecl *P : D->parameters()) {
-FieldTypeInfo *FieldInfo = nullptr;
-if (const auto *T = getDeclForType(P->getOriginalType())) {
-  if (const auto *N = dyn_cast(T)) {
-FieldInfo = _back(
-getUSRForDecl(N), N->getNameAsString(), InfoType::IT_enum,
-getInfoRelativePath(N), P->getNameAsString());
-  } else if (const auto *N = dyn_cast(T)) {
-FieldInfo = _back(
-getUSRForDecl(N), N->getNameAsString(), InfoType::IT_record,
-getInfoRelativePath(N), P->getNameAsString());
-  }
-  // Otherwise fall through to the default case below.
-}
-
-if (!FieldInfo) {
-  FieldInfo = _back(P->getOriginalType().getAsString(),
- P->getNameAsString());
-}
+FieldTypeInfo& FieldInfo = I.Params.emplace_back(
+getTypeInfoForType(P->getOriginalType()),
+P->getNameAsString());
 
 if (const Expr *DefaultArg = P->getDefaultArg()) {
-  FieldInfo->DefaultValue = getSourceCode(D, DefaultArg->getSourceRange());
+  FieldInfo.DefaultValue = getSourceCode(D, DefaultArg->getSourceRange());
 }
   }
 }
@@ -363,14 +356,14 @@
   const TemplateDecl *D = Ty->getTemplateName().getAsTemplateDecl();
   

[PATCH] D134055: [clang-doc] Add support for explicitly typed enums

[Brett Wilson via Phabricator via cfe-commits]

brettw added inline comments.



Comment at: clang-tools-extra/clang-doc/BitcodeReader.cpp:53
 
+llvm::Error decodeRecord(const Record , llvm::APSInt , llvm::StringRef 
Blob) {
+  auto ByteWidth = R[0];

paulkirth wrote:
> brettw wrote:
> > paulkirth wrote:
> > > do you need to do all of this? APSInt already supports to/from string 
> > > methods, as well as converting to/from integers. can you use that here 
> > > and in the writer to avoid some complexity?
> > I don't think converting to an integer is a good idea because people 
> > sometimes use min/max values for enum values and since this could be signed 
> > or unsigned, it gets kind of complicated.
> > 
> > Serializing a number as a string to bitcode also seemed wrong to me.
> > 
> > The simplest thing would be to store the value as a string in the 
> > EnumValueInfo and then always treat this as a string from then on. If you 
> > want it simplified, I think that's the thing to do. But I thought you would 
> > want the numeric value stored in clang-doc's "ast" because some backends 
> > may want to treat this as a number, check its signed/unsignedness, etc. I 
> > happy to change this if you want.
> Those are fair points, and I think I misread/misunderstood a bit of what's 
> going on here.
> 
> As for encoding/decoding integers,  BitcodeWriter already has integer 
> support, so if you need to convert to/from those, it should already work, 
> right?
> 
> Regardless, you may want to look at the bitcode reader/writer in llvm to see 
> how they serialize/deserialize APInt, as their implementation seems a bit 
> more straightforward IMO.
> 
> https://github.com/llvm/llvm-project/blob/9050a59c6678789831b7286b8b68d18b966c4694/llvm/lib/Bitcode/Writer/BitcodeWriter.cpp#L1636
> https://github.com/llvm/llvm-project/blob/main/llvm/lib/Bitcode/Writer/BitcodeWriter.cpp#L2520
> https://github.com/llvm/llvm-project/blob/9050a59c6678789831b7286b8b68d18b966c4694/llvm/lib/Bitcode/Reader/BitcodeReader.cpp#L2843
> https://github.com/llvm/llvm-project/blob/9050a59c6678789831b7286b8b68d18b966c4694/llvm/lib/Bitcode/Reader/BitcodeReader.cpp#L2743
I just converted everything to a string which allowed most of this code to be 
deleted. I don't think any theoretical benefit of keeping the APSInt is worth 
the complexity.



Comment at: clang-tools-extra/clang-doc/BitcodeReader.cpp:71
+
+  llvm::SmallVector AsWords;
+  AsWords.resize(WordWidth);

paulkirth wrote:
> You can avoid the resize w/ SmallVector(Size) constructor, right?
This is now deleted.



Comment at: clang-tools-extra/clang-doc/Representation.h:425
+  // constant. This will be empty for implicit enumeration values.
+  std::string ValueExpr;
+};

paulkirth wrote:
> Sorry to nitpick, but SmallString was the correct choice to use in this type. 
> We avoid its use as a return value, because it tends to be brittle, and stops 
> us from assigning into arbitrary sized SmallStrings.  In the struct, it's a 
> reasonable choice, especially if you expect most uses to be small.  for these 
> expressions, I expect most to either be the number itself, or some shift 
> operation, so SmallString<16> was probably more than sufficient.
In the common case there will be will be empty, so std::string actually seems 
more efficient. Also, I personally think the current quantity of SmallString in 
this code is over-optimization.


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D134055/new/

https://reviews.llvm.org/D134055

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D134055: [clang-doc] Add support for explicitly typed enums

[Brett Wilson via Phabricator via cfe-commits]

brettw updated this revision to Diff 461259.
brettw marked an inline comment as done.

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D134055/new/

https://reviews.llvm.org/D134055

Files:
  clang-tools-extra/clang-doc/BitcodeReader.cpp
  clang-tools-extra/clang-doc/BitcodeWriter.cpp
  clang-tools-extra/clang-doc/BitcodeWriter.h
  clang-tools-extra/clang-doc/HTMLGenerator.cpp
  clang-tools-extra/clang-doc/MDGenerator.cpp
  clang-tools-extra/clang-doc/Representation.h
  clang-tools-extra/clang-doc/Serialize.cpp
  clang-tools-extra/clang-doc/YAMLGenerator.cpp
  clang-tools-extra/unittests/clang-doc/SerializeTest.cpp
  clang-tools-extra/unittests/clang-doc/YAMLGeneratorTest.cpp

Index: clang-tools-extra/unittests/clang-doc/YAMLGeneratorTest.cpp
===
--- clang-tools-extra/unittests/clang-doc/YAMLGeneratorTest.cpp
+++ clang-tools-extra/unittests/clang-doc/YAMLGeneratorTest.cpp
@@ -256,7 +256,11 @@
   EXPECT_EQ(Expected, Actual.str());
 }
 
-TEST(YAMLGeneratorTest, emitEnumYAML) {
+// Tests the equivalent of:
+// namespace A {
+// enum e { X };
+// }
+TEST(YAMLGeneratorTest, emitSimpleEnumYAML) {
   EnumInfo I;
   I.Name = "e";
   I.Namespace.emplace_back(EmptySID, "A", InfoType::IT_namespace);
@@ -265,7 +269,7 @@
   I.Loc.emplace_back(12, llvm::SmallString<16>{"test.cpp"});
 
   I.Members.emplace_back("X");
-  I.Scoped = true;
+  I.Scoped = false;
 
   auto G = getYAMLGenerator();
   assert(G);
@@ -286,9 +290,42 @@
 Location:
   - LineNumber:  12
 Filename:'test.cpp'
+Members:
+  - Name:'X'
+Value:   '0'
+...
+)raw";
+  EXPECT_EQ(Expected, Actual.str());
+}
+
+// Tests the equivalent of:
+// enum class e : short { X = FOO_BAR + 2 };
+TEST(YAMLGeneratorTest, enumTypedScopedEnumYAML) {
+  EnumInfo I;
+  I.Name = "e";
+
+  I.Members.emplace_back("X", "-9876", "FOO_BAR + 2");
+  I.Scoped = true;
+  I.BaseType = TypeInfo("short");
+
+  auto G = getYAMLGenerator();
+  assert(G);
+  std::string Buffer;
+  llvm::raw_string_ostream Actual(Buffer);
+  auto Err = G->generateDocForInfo(, Actual, ClangDocContext());
+  assert(!Err);
+  std::string Expected =
+  R"raw(---
+USR: ''
+Name:'e'
 Scoped:  true
+BaseType:
+  Type:
+Name:'short'
 Members:
-  - 'X'
+  - Name:'X'
+Value:   '-9876'
+Expr:'FOO_BAR + 2'
 ...
 )raw";
   EXPECT_EQ(Expected, Actual.str());
Index: clang-tools-extra/unittests/clang-doc/SerializeTest.cpp
===
--- clang-tools-extra/unittests/clang-doc/SerializeTest.cpp
+++ clang-tools-extra/unittests/clang-doc/SerializeTest.cpp
@@ -266,8 +266,8 @@
   EnumInfo E;
   E.Name = "E";
   E.DefLoc = Location(0, llvm::SmallString<16>{"test.cpp"});
-  E.Members.emplace_back("X");
-  E.Members.emplace_back("Y");
+  E.Members.emplace_back("X", "0");
+  E.Members.emplace_back("Y", "1");
   ExpectedNamespaceWithEnum.ChildEnums.emplace_back(std::move(E));
   CheckNamespaceInfo(, NamespaceWithEnum);
 
@@ -277,8 +277,8 @@
   G.Name = "G";
   G.Scoped = true;
   G.DefLoc = Location(0, llvm::SmallString<16>{"test.cpp"});
-  G.Members.emplace_back("A");
-  G.Members.emplace_back("B");
+  G.Members.emplace_back("A", "0");
+  G.Members.emplace_back("B", "1");
   ExpectedNamespaceWithScopedEnum.ChildEnums.emplace_back(std::move(G));
   CheckNamespaceInfo(, NamespaceWithScopedEnum);
 }
Index: clang-tools-extra/clang-doc/YAMLGenerator.cpp
===
--- clang-tools-extra/clang-doc/YAMLGenerator.cpp
+++ clang-tools-extra/clang-doc/YAMLGenerator.cpp
@@ -14,6 +14,7 @@
 
 using namespace clang::doc;
 
+// These define YAML traits for decoding the listed values within a vector.
 LLVM_YAML_IS_SEQUENCE_VECTOR(FieldTypeInfo)
 LLVM_YAML_IS_SEQUENCE_VECTOR(MemberTypeInfo)
 LLVM_YAML_IS_SEQUENCE_VECTOR(Reference)
@@ -21,6 +22,7 @@
 LLVM_YAML_IS_SEQUENCE_VECTOR(CommentInfo)
 LLVM_YAML_IS_SEQUENCE_VECTOR(FunctionInfo)
 LLVM_YAML_IS_SEQUENCE_VECTOR(EnumInfo)
+LLVM_YAML_IS_SEQUENCE_VECTOR(EnumValueInfo)
 LLVM_YAML_IS_SEQUENCE_VECTOR(BaseRecordInfo)
 LLVM_YAML_IS_SEQUENCE_VECTOR(std::unique_ptr)
 LLVM_YAML_IS_SEQUENCE_VECTOR(llvm::SmallString<16>)
@@ -226,10 +228,19 @@
   }
 };
 
+template <> struct MappingTraits {
+  static void mapping(IO , EnumValueInfo ) {
+IO.mapOptional("Name", I.Name);
+IO.mapOptional("Value", I.Value);
+IO.mapOptional("Expr", I.ValueExpr, SmallString<16>());
+  }
+};
+
 template <> struct MappingTraits {
   static void mapping(IO , EnumInfo ) {
 SymbolInfoMapping(IO, I);
 IO.mapOptional("Scoped", I.Scoped, false);
+IO.mapOptional("BaseType", I.BaseType);
 IO.mapOptional("Members", I.Members);
   }
 };
Index: clang-tools-extra/clang-doc/Serialize.cpp
===
--- 

[PATCH] D134055: [clang-doc] Export enum type and value information.

[Brett Wilson via Phabricator via cfe-commits]

brettw added inline comments.



Comment at: clang-tools-extra/clang-doc/BitcodeReader.cpp:53
 
+llvm::Error decodeRecord(const Record , llvm::APSInt , llvm::StringRef 
Blob) {
+  auto ByteWidth = R[0];

paulkirth wrote:
> do you need to do all of this? APSInt already supports to/from string 
> methods, as well as converting to/from integers. can you use that here and in 
> the writer to avoid some complexity?
I don't think converting to an integer is a good idea because people sometimes 
use min/max values for enum values and since this could be signed or unsigned, 
it gets kind of complicated.

Serializing a number as a string to bitcode also seemed wrong to me.

The simplest thing would be to store the value as a string in the EnumValueInfo 
and then always treat this as a string from then on. If you want it simplified, 
I think that's the thing to do. But I thought you would want the numeric value 
stored in clang-doc's "ast" because some backends may want to treat this as a 
number, check its signed/unsignedness, etc. I happy to change this if you want.



Comment at: clang-tools-extra/clang-doc/BitcodeReader.cpp:221
 return decodeRecord(R, I->Loc, Blob);
-  case ENUM_MEMBER:
-return decodeRecord(R, I->Members, Blob);

paulkirth wrote:
> Don't you still need this? if it's now unused, we should probably remove it 
> from the RecordId enum too.
I think this is only used for properties and not the child members, so it isn't 
used. I removed the enum.



Comment at: clang-tools-extra/clang-doc/Serialize.cpp:185
 
+SmallString<128> getSourceCode(const Decl* D, const SourceRange& R) {
+  return Lexer::getSourceText(

paulkirth wrote:
> We normally try to avoid returning small string by value. see 
> https://llvm.org/docs/ProgrammersManual.html#llvm-adt-smallstring-h
> 
> Normally in the case that we need to return a string from a method, we just 
> use std::string. In other cases, it may be more appropriate to use an output 
> parameter instead of a return.
I was just copying some other code in this file, I changed this to std::string.



Comment at: clang-tools-extra/clang-doc/Serialize.cpp:316
+  for (const EnumConstantDecl *E : D->enumerators()) {
+SmallString<128> ValueExpr;
+if (const Expr* InitExpr = E->getInitExpr())

paulkirth wrote:
> why was 128 chosen? Aren't we storing it into a `SmallString<16>` in 
> `EnumValueInfo`? is there some external reason that we expect this to be the 
> right size? 
> 
> Do you have an idea for how long these are likely to be? if we expect them to 
> be large or completely unpredictable, it may make more sense to use a 
> `std::string` and avoid wasting stack space.
> 
>  
I changed this and the EnumValueInfo struct to std::string. I think the usage 
of SmallString in these records is over the top but I was trying to copy the 
existing style.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D134055/new/

https://reviews.llvm.org/D134055

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D134055: [clang-doc] Export enum type and value information.

[Brett Wilson via Phabricator via cfe-commits]

brettw updated this revision to Diff 460882.
brettw marked 2 inline comments as done.
brettw retitled this revision from "[clang-doc] Export more enum information" 
to "[clang-doc] Export enum type and value information.".
brettw edited the summary of this revision.

Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D134055/new/

https://reviews.llvm.org/D134055

Files:
  clang-tools-extra/clang-doc/BitcodeReader.cpp
  clang-tools-extra/clang-doc/BitcodeWriter.cpp
  clang-tools-extra/clang-doc/BitcodeWriter.h
  clang-tools-extra/clang-doc/HTMLGenerator.cpp
  clang-tools-extra/clang-doc/MDGenerator.cpp
  clang-tools-extra/clang-doc/Representation.h
  clang-tools-extra/clang-doc/Serialize.cpp
  clang-tools-extra/clang-doc/YAMLGenerator.cpp
  clang-tools-extra/unittests/clang-doc/SerializeTest.cpp
  clang-tools-extra/unittests/clang-doc/YAMLGeneratorTest.cpp

Index: clang-tools-extra/unittests/clang-doc/YAMLGeneratorTest.cpp
===
--- clang-tools-extra/unittests/clang-doc/YAMLGeneratorTest.cpp
+++ clang-tools-extra/unittests/clang-doc/YAMLGeneratorTest.cpp
@@ -256,7 +256,11 @@
   EXPECT_EQ(Expected, Actual.str());
 }
 
-TEST(YAMLGeneratorTest, emitEnumYAML) {
+// Tests the equivalent of:
+// namespace A {
+// enum e { X };
+// }
+TEST(YAMLGeneratorTest, emitSimpleEnumYAML) {
   EnumInfo I;
   I.Name = "e";
   I.Namespace.emplace_back(EmptySID, "A", InfoType::IT_namespace);
@@ -265,7 +269,7 @@
   I.Loc.emplace_back(12, llvm::SmallString<16>{"test.cpp"});
 
   I.Members.emplace_back("X");
-  I.Scoped = true;
+  I.Scoped = false;
 
   auto G = getYAMLGenerator();
   assert(G);
@@ -286,9 +290,42 @@
 Location:
   - LineNumber:  12
 Filename:'test.cpp'
+Members:
+  - Name:'X'
+Value:   '0'
+...
+)raw";
+  EXPECT_EQ(Expected, Actual.str());
+}
+
+// Tests the equivalent of:
+// enum class e : short { X = FOO_BAR + 2 };
+TEST(YAMLGeneratorTest, enumTypedScopedEnumYAML) {
+  EnumInfo I;
+  I.Name = "e";
+
+  I.Members.emplace_back("X", llvm::APSInt::get(-9876), "FOO_BAR + 2");
+  I.Scoped = true;
+  I.BaseType = TypeInfo("short");
+
+  auto G = getYAMLGenerator();
+  assert(G);
+  std::string Buffer;
+  llvm::raw_string_ostream Actual(Buffer);
+  auto Err = G->generateDocForInfo(, Actual, ClangDocContext());
+  assert(!Err);
+  std::string Expected =
+  R"raw(---
+USR: ''
+Name:'e'
 Scoped:  true
+BaseType:
+  Type:
+Name:'short'
 Members:
-  - 'X'
+  - Name:'X'
+Value:   '-9876'
+Expr:'FOO_BAR + 2'
 ...
 )raw";
   EXPECT_EQ(Expected, Actual.str());
Index: clang-tools-extra/unittests/clang-doc/SerializeTest.cpp
===
--- clang-tools-extra/unittests/clang-doc/SerializeTest.cpp
+++ clang-tools-extra/unittests/clang-doc/SerializeTest.cpp
@@ -266,8 +266,8 @@
   EnumInfo E;
   E.Name = "E";
   E.DefLoc = Location(0, llvm::SmallString<16>{"test.cpp"});
-  E.Members.emplace_back("X");
-  E.Members.emplace_back("Y");
+  E.Members.emplace_back("X", llvm::APSInt::get(0));
+  E.Members.emplace_back("Y", llvm::APSInt::get(1));
   ExpectedNamespaceWithEnum.ChildEnums.emplace_back(std::move(E));
   CheckNamespaceInfo(, NamespaceWithEnum);
 
@@ -277,8 +277,8 @@
   G.Name = "G";
   G.Scoped = true;
   G.DefLoc = Location(0, llvm::SmallString<16>{"test.cpp"});
-  G.Members.emplace_back("A");
-  G.Members.emplace_back("B");
+  G.Members.emplace_back("A", llvm::APSInt::get(0));
+  G.Members.emplace_back("B", llvm::APSInt::get(1));
   ExpectedNamespaceWithScopedEnum.ChildEnums.emplace_back(std::move(G));
   CheckNamespaceInfo(, NamespaceWithScopedEnum);
 }
Index: clang-tools-extra/clang-doc/YAMLGenerator.cpp
===
--- clang-tools-extra/clang-doc/YAMLGenerator.cpp
+++ clang-tools-extra/clang-doc/YAMLGenerator.cpp
@@ -14,6 +14,7 @@
 
 using namespace clang::doc;
 
+// These define YAML traits for decoding the listed values within a vector.
 LLVM_YAML_IS_SEQUENCE_VECTOR(FieldTypeInfo)
 LLVM_YAML_IS_SEQUENCE_VECTOR(MemberTypeInfo)
 LLVM_YAML_IS_SEQUENCE_VECTOR(Reference)
@@ -21,6 +22,7 @@
 LLVM_YAML_IS_SEQUENCE_VECTOR(CommentInfo)
 LLVM_YAML_IS_SEQUENCE_VECTOR(FunctionInfo)
 LLVM_YAML_IS_SEQUENCE_VECTOR(EnumInfo)
+LLVM_YAML_IS_SEQUENCE_VECTOR(EnumValueInfo)
 LLVM_YAML_IS_SEQUENCE_VECTOR(BaseRecordInfo)
 LLVM_YAML_IS_SEQUENCE_VECTOR(std::unique_ptr)
 LLVM_YAML_IS_SEQUENCE_VECTOR(llvm::SmallString<16>)
@@ -226,10 +228,22 @@
   }
 };
 
+template <> struct MappingTraits {
+  static void mapping(IO , EnumValueInfo ) {
+IO.mapOptional("Name", I.Name);
+
+SmallString<16> ValueStr;
+I.Value.toString(ValueStr);
+IO.mapOptional("Value", ValueStr);
+IO.mapOptional("Expr", I.ValueExpr, std::string());
+  }
+};
+
 template <> struct 

[PATCH] D134055: [clang-doc] Export more enum information

[Brett Wilson via Phabricator via cfe-commits]

brettw created this revision.
brettw added a reviewer: paulkirth.
brettw added a project: clang-tools-extra.
Herald added a project: All.
brettw requested review of this revision.
Herald added a subscriber: cfe-commits.

Add support for explicitly typed enums:

  enum Foo : unsigned { ... };

to the internal representation and to the YAML output.

  

Add support for getting the value of an enum constant, as well as accessing the 
original expression that produced it. This changes the YAML output of enums 
from an array of strings for the enum members to an array of dictionaries. 
These dictionaries now report the name, value, and original expression.

  

The markdown and HTML outputs are unchanged, they still output the name from 
the new enhanced internal schema.


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D134055

Files:
  clang-tools-extra/clang-doc/BitcodeReader.cpp
  clang-tools-extra/clang-doc/BitcodeWriter.cpp
  clang-tools-extra/clang-doc/BitcodeWriter.h
  clang-tools-extra/clang-doc/HTMLGenerator.cpp
  clang-tools-extra/clang-doc/MDGenerator.cpp
  clang-tools-extra/clang-doc/Representation.h
  clang-tools-extra/clang-doc/Serialize.cpp
  clang-tools-extra/clang-doc/YAMLGenerator.cpp
  clang-tools-extra/unittests/clang-doc/SerializeTest.cpp
  clang-tools-extra/unittests/clang-doc/YAMLGeneratorTest.cpp

Index: clang-tools-extra/unittests/clang-doc/YAMLGeneratorTest.cpp
===
--- clang-tools-extra/unittests/clang-doc/YAMLGeneratorTest.cpp
+++ clang-tools-extra/unittests/clang-doc/YAMLGeneratorTest.cpp
@@ -256,7 +256,11 @@
   EXPECT_EQ(Expected, Actual.str());
 }
 
-TEST(YAMLGeneratorTest, emitEnumYAML) {
+// Tests the equivalent of:
+// namespace A {
+// enum e { X };
+// }
+TEST(YAMLGeneratorTest, emitSimpleEnumYAML) {
   EnumInfo I;
   I.Name = "e";
   I.Namespace.emplace_back(EmptySID, "A", InfoType::IT_namespace);
@@ -265,7 +269,7 @@
   I.Loc.emplace_back(12, llvm::SmallString<16>{"test.cpp"});
 
   I.Members.emplace_back("X");
-  I.Scoped = true;
+  I.Scoped = false;
 
   auto G = getYAMLGenerator();
   assert(G);
@@ -286,9 +290,42 @@
 Location:
   - LineNumber:  12
 Filename:'test.cpp'
+Members:
+  - Name:'X'
+Value:   '0'
+...
+)raw";
+  EXPECT_EQ(Expected, Actual.str());
+}
+
+// Tests the equivalent of:
+// enum class e : short { X = FOO_BAR + 2 };
+TEST(YAMLGeneratorTest, enumTypedScopedEnumYAML) {
+  EnumInfo I;
+  I.Name = "e";
+
+  I.Members.emplace_back("X", llvm::APSInt::get(-9876), "FOO_BAR + 2");
+  I.Scoped = true;
+  I.BaseType = TypeInfo("short");
+
+  auto G = getYAMLGenerator();
+  assert(G);
+  std::string Buffer;
+  llvm::raw_string_ostream Actual(Buffer);
+  auto Err = G->generateDocForInfo(, Actual, ClangDocContext());
+  assert(!Err);
+  std::string Expected =
+  R"raw(---
+USR: ''
+Name:'e'
 Scoped:  true
+BaseType:
+  Type:
+Name:'short'
 Members:
-  - 'X'
+  - Name:'X'
+Value:   '-9876'
+Expr:'FOO_BAR + 2'
 ...
 )raw";
   EXPECT_EQ(Expected, Actual.str());
Index: clang-tools-extra/unittests/clang-doc/SerializeTest.cpp
===
--- clang-tools-extra/unittests/clang-doc/SerializeTest.cpp
+++ clang-tools-extra/unittests/clang-doc/SerializeTest.cpp
@@ -266,8 +266,8 @@
   EnumInfo E;
   E.Name = "E";
   E.DefLoc = Location(0, llvm::SmallString<16>{"test.cpp"});
-  E.Members.emplace_back("X");
-  E.Members.emplace_back("Y");
+  E.Members.emplace_back("X", llvm::APSInt::get(0));
+  E.Members.emplace_back("Y", llvm::APSInt::get(1));
   ExpectedNamespaceWithEnum.ChildEnums.emplace_back(std::move(E));
   CheckNamespaceInfo(, NamespaceWithEnum);
 
@@ -277,8 +277,8 @@
   G.Name = "G";
   G.Scoped = true;
   G.DefLoc = Location(0, llvm::SmallString<16>{"test.cpp"});
-  G.Members.emplace_back("A");
-  G.Members.emplace_back("B");
+  G.Members.emplace_back("A", llvm::APSInt::get(0));
+  G.Members.emplace_back("B", llvm::APSInt::get(1));
   ExpectedNamespaceWithScopedEnum.ChildEnums.emplace_back(std::move(G));
   CheckNamespaceInfo(, NamespaceWithScopedEnum);
 }
Index: clang-tools-extra/clang-doc/YAMLGenerator.cpp
===
--- clang-tools-extra/clang-doc/YAMLGenerator.cpp
+++ clang-tools-extra/clang-doc/YAMLGenerator.cpp
@@ -14,6 +14,7 @@
 
 using namespace clang::doc;
 
+// These define YAML traits for decoding the listed values within a vector.
 LLVM_YAML_IS_SEQUENCE_VECTOR(FieldTypeInfo)
 LLVM_YAML_IS_SEQUENCE_VECTOR(MemberTypeInfo)
 LLVM_YAML_IS_SEQUENCE_VECTOR(Reference)
@@ -21,6 +22,7 @@
 LLVM_YAML_IS_SEQUENCE_VECTOR(CommentInfo)
 LLVM_YAML_IS_SEQUENCE_VECTOR(FunctionInfo)
 LLVM_YAML_IS_SEQUENCE_VECTOR(EnumInfo)
+LLVM_YAML_IS_SEQUENCE_VECTOR(EnumValueInfo)
 LLVM_YAML_IS_SEQUENCE_VECTOR(BaseRecordInfo)
 

[PATCH] D133732: [clang-doc] Support default args for functions.

[Brett Wilson via Phabricator via cfe-commits]

brettw updated this revision to Diff 460538.
brettw marked 4 inline comments as done.
brettw added a comment.

There is no usage change for this. It just adds another field to the YAML.


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D133732/new/

https://reviews.llvm.org/D133732

Files:
  clang-tools-extra/clang-doc/BitcodeReader.cpp
  clang-tools-extra/clang-doc/BitcodeWriter.cpp
  clang-tools-extra/clang-doc/BitcodeWriter.h
  clang-tools-extra/clang-doc/Representation.h
  clang-tools-extra/clang-doc/Serialize.cpp
  clang-tools-extra/clang-doc/YAMLGenerator.cpp
  clang-tools-extra/unittests/clang-doc/SerializeTest.cpp
  clang-tools-extra/unittests/clang-doc/YAMLGeneratorTest.cpp

Index: clang-tools-extra/unittests/clang-doc/YAMLGeneratorTest.cpp
===
--- clang-tools-extra/unittests/clang-doc/YAMLGeneratorTest.cpp
+++ clang-tools-extra/unittests/clang-doc/YAMLGeneratorTest.cpp
@@ -209,6 +209,8 @@
   I.ReturnType =
   TypeInfo(EmptySID, "void", InfoType::IT_default, "path/to/void");
   I.Params.emplace_back("int", "path/to/int", "P");
+  I.Params.emplace_back("double", "path/to/double", "D");
+  I.Params.back().DefaultValue = "2.0 * M_PI";
   I.IsMethod = true;
   I.Parent = Reference(EmptySID, "Parent", InfoType::IT_record);
 
@@ -240,6 +242,11 @@
   Name:'int'
   Path:'path/to/int'
 Name:'P'
+  - Type:
+  Name:'double'
+  Path:'path/to/double'
+Name:'D'
+DefaultValue:'2.0 * M_PI'
 ReturnType:
   Type:
 Name:'void'
Index: clang-tools-extra/unittests/clang-doc/SerializeTest.cpp
===
--- clang-tools-extra/unittests/clang-doc/SerializeTest.cpp
+++ clang-tools-extra/unittests/clang-doc/SerializeTest.cpp
@@ -511,7 +511,7 @@
   std::vector Args;
   Args.push_back("-fmodules-ts");
   ExtractInfosFromCodeWithArgs(R"raw(export module M;
-int moduleFunction(int x);
+int moduleFunction(int x, double d = 3.2 - 1.0);
 static int staticModuleFunction(int x);
 export double exportedModuleFunction(double y);)raw",
2, /*Public=*/true, Infos, Args);
@@ -523,6 +523,8 @@
   F.ReturnType = TypeInfo(EmptySID, "int", InfoType::IT_default);
   F.Loc.emplace_back(0, llvm::SmallString<16>{"test.cpp"});
   F.Params.emplace_back("int", "x");
+  F.Params.emplace_back("double", "d");
+  F.Params.back().DefaultValue = "3.2 - 1.0";
   F.Access = AccessSpecifier::AS_none;
   ExpectedBWithFunction.ChildFunctions.emplace_back(std::move(F));
   CheckNamespaceInfo(, BWithFunction);
Index: clang-tools-extra/clang-doc/YAMLGenerator.cpp
===
--- clang-tools-extra/clang-doc/YAMLGenerator.cpp
+++ clang-tools-extra/clang-doc/YAMLGenerator.cpp
@@ -109,6 +109,7 @@
 static void FieldTypeInfoMapping(IO , FieldTypeInfo ) {
   TypeInfoMapping(IO, I);
   IO.mapOptional("Name", I.Name, SmallString<16>());
+  IO.mapOptional("DefaultValue", I.DefaultValue, SmallString<16>());
 }
 
 static void InfoMapping(IO , Info ) {
@@ -183,6 +184,7 @@
   static void mapping(IO , FieldTypeInfo ) {
 TypeInfoMapping(IO, I);
 IO.mapOptional("Name", I.Name, SmallString<16>());
+IO.mapOptional("DefaultValue", I.DefaultValue, SmallString<16>());
   }
 };
 
Index: clang-tools-extra/clang-doc/Serialize.cpp
===
--- clang-tools-extra/clang-doc/Serialize.cpp
+++ clang-tools-extra/clang-doc/Serialize.cpp
@@ -10,6 +10,7 @@
 #include "BitcodeWriter.h"
 #include "clang/AST/Comment.h"
 #include "clang/Index/USRGeneration.h"
+#include "clang/Lex/Lexer.h"
 #include "llvm/ADT/Hashing.h"
 #include "llvm/ADT/StringExtras.h"
 #include "llvm/Support/SHA1.h"
@@ -310,21 +311,31 @@
 
 static void parseParameters(FunctionInfo , const FunctionDecl *D) {
   for (const ParmVarDecl *P : D->parameters()) {
+FieldTypeInfo *FieldInfo = nullptr;
 if (const auto *T = getDeclForType(P->getOriginalType())) {
   if (const auto *N = dyn_cast(T)) {
-I.Params.emplace_back(getUSRForDecl(N), N->getNameAsString(),
-  InfoType::IT_enum, getInfoRelativePath(N),
-  P->getNameAsString());
-continue;
+FieldInfo = _back(
+getUSRForDecl(N), N->getNameAsString(), InfoType::IT_enum,
+getInfoRelativePath(N), P->getNameAsString());
   } else if (const auto *N = dyn_cast(T)) {
-I.Params.emplace_back(getUSRForDecl(N), N->getNameAsString(),
-  InfoType::IT_record, getInfoRelativePath(N),
-  P->getNameAsString());
-continue;
+FieldInfo = _back(
+getUSRForDecl(N), N->getNameAsString(), InfoType::IT_record,
+getInfoRelativePath(N), P->getNameAsString());
   }
+  // Otherwise fall 

[PATCH] D133732: [clang-doc] Support default args for functions.

[Brett Wilson via Phabricator via cfe-commits]

brettw created this revision.
brettw added a reviewer: paulkirth.
brettw added a project: clang-tools-extra.
Herald added a project: All.
brettw requested review of this revision.
Herald added a subscriber: cfe-commits.

Adds support for default arguments in the internal representation and reads 
these values from the source. Implements writing these values to YAML but does 
not implement this for the HTML or markdown outputs.


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D133732

Files:
  clang-tools-extra/clang-doc/BitcodeReader.cpp
  clang-tools-extra/clang-doc/BitcodeWriter.cpp
  clang-tools-extra/clang-doc/BitcodeWriter.h
  clang-tools-extra/clang-doc/Representation.h
  clang-tools-extra/clang-doc/Serialize.cpp
  clang-tools-extra/clang-doc/YAMLGenerator.cpp
  clang-tools-extra/unittests/clang-doc/SerializeTest.cpp
  clang-tools-extra/unittests/clang-doc/YAMLGeneratorTest.cpp

Index: clang-tools-extra/unittests/clang-doc/YAMLGeneratorTest.cpp
===
--- clang-tools-extra/unittests/clang-doc/YAMLGeneratorTest.cpp
+++ clang-tools-extra/unittests/clang-doc/YAMLGeneratorTest.cpp
@@ -209,6 +209,8 @@
   I.ReturnType =
   TypeInfo(EmptySID, "void", InfoType::IT_default, "path/to/void");
   I.Params.emplace_back("int", "path/to/int", "P");
+  I.Params.emplace_back("double", "path/to/double", "D");
+  I.Params.back().DefaultValue = "2.0 * M_PI";
   I.IsMethod = true;
   I.Parent = Reference(EmptySID, "Parent", InfoType::IT_record);
 
@@ -240,6 +242,11 @@
   Name:'int'
   Path:'path/to/int'
 Name:'P'
+  - Type:
+  Name:'double'
+  Path:'path/to/double'
+Name:'D'
+DefaultValue:'2.0 * M_PI'
 ReturnType:
   Type:
 Name:'void'
Index: clang-tools-extra/unittests/clang-doc/SerializeTest.cpp
===
--- clang-tools-extra/unittests/clang-doc/SerializeTest.cpp
+++ clang-tools-extra/unittests/clang-doc/SerializeTest.cpp
@@ -511,7 +511,7 @@
   std::vector Args;
   Args.push_back("-fmodules-ts");
   ExtractInfosFromCodeWithArgs(R"raw(export module M;
-int moduleFunction(int x);
+int moduleFunction(int x, double d = 3.2 - 1.0);
 static int staticModuleFunction(int x);
 export double exportedModuleFunction(double y);)raw",
2, /*Public=*/true, Infos, Args);
@@ -523,6 +523,8 @@
   F.ReturnType = TypeInfo(EmptySID, "int", InfoType::IT_default);
   F.Loc.emplace_back(0, llvm::SmallString<16>{"test.cpp"});
   F.Params.emplace_back("int", "x");
+  F.Params.emplace_back("double", "d");
+  F.Params.back().DefaultValue = "3.2 - 1.0";
   F.Access = AccessSpecifier::AS_none;
   ExpectedBWithFunction.ChildFunctions.emplace_back(std::move(F));
   CheckNamespaceInfo(, BWithFunction);
Index: clang-tools-extra/clang-doc/YAMLGenerator.cpp
===
--- clang-tools-extra/clang-doc/YAMLGenerator.cpp
+++ clang-tools-extra/clang-doc/YAMLGenerator.cpp
@@ -109,6 +109,7 @@
 static void FieldTypeInfoMapping(IO , FieldTypeInfo ) {
   TypeInfoMapping(IO, I);
   IO.mapOptional("Name", I.Name, SmallString<16>());
+  IO.mapOptional("DefaultValue", I.DefaultValue, SmallString<16>());
 }
 
 static void InfoMapping(IO , Info ) {
@@ -183,6 +184,7 @@
   static void mapping(IO , FieldTypeInfo ) {
 TypeInfoMapping(IO, I);
 IO.mapOptional("Name", I.Name, SmallString<16>());
+IO.mapOptional("DefaultValue", I.DefaultValue, SmallString<16>());
   }
 };
 
Index: clang-tools-extra/clang-doc/Serialize.cpp
===
--- clang-tools-extra/clang-doc/Serialize.cpp
+++ clang-tools-extra/clang-doc/Serialize.cpp
@@ -10,6 +10,7 @@
 #include "BitcodeWriter.h"
 #include "clang/AST/Comment.h"
 #include "clang/Index/USRGeneration.h"
+#include "clang/Lex/Lexer.h"
 #include "llvm/ADT/Hashing.h"
 #include "llvm/ADT/StringExtras.h"
 #include "llvm/Support/SHA1.h"
@@ -296,7 +297,7 @@
   }
 }
 
-auto& member = I.Members.emplace_back(
+auto  = I.Members.emplace_back(
 F->getTypeSourceInfo()->getType().getAsString(), F->getNameAsString(),
 getFinalAccessSpecifier(Access, F->getAccessUnsafe()));
 populateMemberTypeInfo(member, F);
@@ -310,21 +311,31 @@
 
 static void parseParameters(FunctionInfo , const FunctionDecl *D) {
   for (const ParmVarDecl *P : D->parameters()) {
+FieldTypeInfo *FieldInfo = nullptr;
 if (const auto *T = getDeclForType(P->getOriginalType())) {
   if (const auto *N = dyn_cast(T)) {
-I.Params.emplace_back(getUSRForDecl(N), N->getNameAsString(),
-  InfoType::IT_enum, getInfoRelativePath(N),
-  P->getNameAsString());
-continue;
+FieldInfo = _back(
+getUSRForDecl(N), N->getNameAsString(), InfoType::IT_enum,
+   

[PATCH] D131739: [clang-doc] Always emit the TagType for RecordInfo

[Brett Wilson via Phabricator via cfe-commits]

brettw added a comment.

In D131739#3719634 , @paulkirth wrote:

> Can you update the summary to reflect what's in this change now + add a tag 
> for clang-doc?
>
> `[clang-doc] Always emit the TagType for RecordInfo`
>
> I can land this and D131793  once the 
> summaries are in order.

Done


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D131739/new/

https://reviews.llvm.org/D131739

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D131793: [clang-doc] Fix assert on startup

[Brett Wilson via Phabricator via cfe-commits]

brettw added a comment.

In D131793#3719619 , @paulkirth wrote:

> This is LGTM, but can you add a tag to the summary title/commit message? ie. 
> `[clang-doc] Fix assert on startup in clang-doc`

Done


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D131793/new/

https://reviews.llvm.org/D131793

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D131739: Fix some clang-doc issues.

[Brett Wilson via Phabricator via cfe-commits]

brettw updated this revision to Diff 452227.
brettw edited the summary of this revision.
brettw added a comment.

Startup assert fix broken out into https://reviews.llvm.org/D131793


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D131739/new/

https://reviews.llvm.org/D131739

Files:
  clang-tools-extra/clang-doc/Representation.cpp
  clang-tools-extra/clang-doc/Representation.h
  clang-tools-extra/clang-doc/YAMLGenerator.cpp
  clang-tools-extra/unittests/clang-doc/MergeTest.cpp
  clang-tools-extra/unittests/clang-doc/YAMLGeneratorTest.cpp


Index: clang-tools-extra/unittests/clang-doc/YAMLGeneratorTest.cpp
===
--- clang-tools-extra/unittests/clang-doc/YAMLGeneratorTest.cpp
+++ clang-tools-extra/unittests/clang-doc/YAMLGeneratorTest.cpp
@@ -76,6 +76,7 @@
   RecordInfo I;
   I.Name = "r";
   I.Path = "path/to/A";
+  I.IsTypeDef = true;
   I.Namespace.emplace_back(EmptySID, "A", InfoType::IT_namespace);
 
   I.DefLoc = Location(10, llvm::SmallString<16>{"test.cpp"});
@@ -136,6 +137,7 @@
   - LineNumber:  12
 Filename:'test.cpp'
 TagType: Class
+IsTypeDef:   true
 Members:
   - Type:
   Name:'int'
@@ -154,6 +156,7 @@
   - USR: ''
 Name:'F'
 Path:'path/to/F'
+TagType: Struct
 Members:
   - Type:
   Name:'int'
Index: clang-tools-extra/unittests/clang-doc/MergeTest.cpp
===
--- clang-tools-extra/unittests/clang-doc/MergeTest.cpp
+++ clang-tools-extra/unittests/clang-doc/MergeTest.cpp
@@ -78,6 +78,7 @@
 TEST(MergeTest, mergeRecordInfos) {
   RecordInfo One;
   One.Name = "r";
+  One.IsTypeDef = true;
   One.Namespace.emplace_back(EmptySID, "A", InfoType::IT_namespace);
 
   One.DefLoc = Location(10, llvm::SmallString<16>{"test.cpp"});
@@ -119,6 +120,7 @@
 
   auto Expected = std::make_unique();
   Expected->Name = "r";
+  Expected->IsTypeDef = true;
   Expected->Namespace.emplace_back(EmptySID, "A", InfoType::IT_namespace);
 
   Expected->DefLoc = Location(10, llvm::SmallString<16>{"test.cpp"});
Index: clang-tools-extra/clang-doc/YAMLGenerator.cpp
===
--- clang-tools-extra/clang-doc/YAMLGenerator.cpp
+++ clang-tools-extra/clang-doc/YAMLGenerator.cpp
@@ -127,7 +127,8 @@
 
 static void RecordInfoMapping(IO , RecordInfo ) {
   SymbolInfoMapping(IO, I);
-  IO.mapOptional("TagType", I.TagType, clang::TagTypeKind::TTK_Struct);
+  IO.mapOptional("TagType", I.TagType);
+  IO.mapOptional("IsTypeDef", I.IsTypeDef, false);
   IO.mapOptional("Members", I.Members);
   IO.mapOptional("Bases", I.Bases);
   IO.mapOptional("Parents", I.Parents, llvm::SmallVector());
Index: clang-tools-extra/clang-doc/Representation.h
===
--- clang-tools-extra/clang-doc/Representation.h
+++ clang-tools-extra/clang-doc/Representation.h
@@ -348,10 +348,15 @@
 
   void merge(RecordInfo &);
 
-  TagTypeKind TagType = TagTypeKind::TTK_Struct; // Type of this record
- // (struct, class, union,
- // interface).
-  bool IsTypeDef = false; // Indicates if record was declared using typedef
+  // Type of this record (struct, class, union, interface).
+  TagTypeKind TagType = TagTypeKind::TTK_Struct;
+
+  // Indicates if the record was declared using a typedef. Things like 
anonymous
+  // structs in a typedef:
+  //   typedef struct { ... } foo_t;
+  // are converted into records with the typedef as the Name + this flag set.
+  bool IsTypeDef = false;
+
   llvm::SmallVector
   Members; // List of info about record 
members.
   llvm::SmallVector Parents; // List of base/parent records
Index: clang-tools-extra/clang-doc/Representation.cpp
===
--- clang-tools-extra/clang-doc/Representation.cpp
+++ clang-tools-extra/clang-doc/Representation.cpp
@@ -222,6 +222,7 @@
   assert(mergeable(Other));
   if (!TagType)
 TagType = Other.TagType;
+  IsTypeDef = IsTypeDef || Other.IsTypeDef;
   if (Members.empty())
 Members = std::move(Other.Members);
   if (Bases.empty())


Index: clang-tools-extra/unittests/clang-doc/YAMLGeneratorTest.cpp
===
--- clang-tools-extra/unittests/clang-doc/YAMLGeneratorTest.cpp
+++ clang-tools-extra/unittests/clang-doc/YAMLGeneratorTest.cpp
@@ -76,6 +76,7 @@
   RecordInfo I;
   I.Name = "r";
   I.Path = "path/to/A";
+  I.IsTypeDef = true;
   I.Namespace.emplace_back(EmptySID, "A", InfoType::IT_namespace);
 
   I.DefLoc = Location(10, llvm::SmallString<16>{"test.cpp"});
@@ -136,6 +137,7 @@
   - LineNumber:  12
 Filename:'test.cpp'
 TagType: 

[PATCH] D131793: Fix assert on startup in clang-doc

[Brett Wilson via Phabricator via cfe-commits]

brettw created this revision.
brettw added a reviewer: paulkirth.
brettw added a project: clang-tools-extra.
Herald added subscribers: usaxena95, arphaman.
Herald added a project: All.
brettw requested review of this revision.
Herald added a subscriber: cfe-commits.

When using `clang-doc --format=html` it will crash on startup because of an 
assertion doing a self-assignment of a `SmallString`. This patch removes the 
self-assignment by creating an intermediate copy.


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D131793

Files:
  clang-tools-extra/clang-doc/tool/ClangDocMain.cpp


Index: clang-tools-extra/clang-doc/tool/ClangDocMain.cpp
===
--- clang-tools-extra/clang-doc/tool/ClangDocMain.cpp
+++ clang-tools-extra/clang-doc/tool/ClangDocMain.cpp
@@ -231,9 +231,10 @@
   if (Format == "html") {
 void *MainAddr = (void *)(intptr_t)GetExecutablePath;
 std::string ClangDocPath = GetExecutablePath(argv[0], MainAddr);
+llvm::SmallString<128> NativeClangDocPath;
+llvm::sys::path::native(ClangDocPath, NativeClangDocPath);
 llvm::SmallString<128> AssetsPath;
-llvm::sys::path::native(ClangDocPath, AssetsPath);
-AssetsPath = llvm::sys::path::parent_path(AssetsPath);
+AssetsPath = llvm::sys::path::parent_path(NativeClangDocPath);
 llvm::sys::path::append(AssetsPath, "..", "share", "clang");
 llvm::SmallString<128> DefaultStylesheet;
 llvm::sys::path::native(AssetsPath, DefaultStylesheet);


Index: clang-tools-extra/clang-doc/tool/ClangDocMain.cpp
===
--- clang-tools-extra/clang-doc/tool/ClangDocMain.cpp
+++ clang-tools-extra/clang-doc/tool/ClangDocMain.cpp
@@ -231,9 +231,10 @@
   if (Format == "html") {
 void *MainAddr = (void *)(intptr_t)GetExecutablePath;
 std::string ClangDocPath = GetExecutablePath(argv[0], MainAddr);
+llvm::SmallString<128> NativeClangDocPath;
+llvm::sys::path::native(ClangDocPath, NativeClangDocPath);
 llvm::SmallString<128> AssetsPath;
-llvm::sys::path::native(ClangDocPath, AssetsPath);
-AssetsPath = llvm::sys::path::parent_path(AssetsPath);
+AssetsPath = llvm::sys::path::parent_path(NativeClangDocPath);
 llvm::sys::path::append(AssetsPath, "..", "share", "clang");
 llvm::SmallString<128> DefaultStylesheet;
 llvm::sys::path::native(AssetsPath, DefaultStylesheet);
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D131739: Fix some clang-doc issues.

[Brett Wilson via Phabricator via cfe-commits]

brettw created this revision.
brettw added a reviewer: paulkirth.
brettw added a project: clang-tools-extra.
Herald added subscribers: usaxena95, arphaman.
Herald added a project: All.
brettw requested review of this revision.
Herald added a subscriber: cfe-commits.

Always emit the TagType for RecordInfo in YAML output. Previously this omitted 
the type for "struct", considering it the default. But records in C++ don't 
really have a default type so always emitting this is more clear.

  

Fix crash on startup when generating HTML code. SmallString (surprisingly) 
can't handle self-assignment which was happening because parent_path returns a 
string piece into its argument. This avoids the self-assignment by using 
another intermediate string.

  

Emit IsTypeDef in YAML. Previously this existed only in the Representation but 
was never written. Additionally, adds IsTypeDef to the record merge operation 
which was clearing it (all RecordInfo structures are merged with am empty 
RecordInfo during the reduce phase).


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D131739

Files:
  clang-tools-extra/clang-doc/Representation.cpp
  clang-tools-extra/clang-doc/Representation.h
  clang-tools-extra/clang-doc/YAMLGenerator.cpp
  clang-tools-extra/clang-doc/tool/ClangDocMain.cpp
  clang-tools-extra/unittests/clang-doc/MergeTest.cpp
  clang-tools-extra/unittests/clang-doc/YAMLGeneratorTest.cpp

Index: clang-tools-extra/unittests/clang-doc/YAMLGeneratorTest.cpp
===
--- clang-tools-extra/unittests/clang-doc/YAMLGeneratorTest.cpp
+++ clang-tools-extra/unittests/clang-doc/YAMLGeneratorTest.cpp
@@ -76,6 +76,7 @@
   RecordInfo I;
   I.Name = "r";
   I.Path = "path/to/A";
+  I.IsTypeDef = true;
   I.Namespace.emplace_back(EmptySID, "A", InfoType::IT_namespace);
 
   I.DefLoc = Location(10, llvm::SmallString<16>{"test.cpp"});
@@ -136,6 +137,7 @@
   - LineNumber:  12
 Filename:'test.cpp'
 TagType: Class
+IsTypeDef:   true
 Members:
   - Type:
   Name:'int'
@@ -154,6 +156,7 @@
   - USR: ''
 Name:'F'
 Path:'path/to/F'
+TagType: Struct
 Members:
   - Type:
   Name:'int'
Index: clang-tools-extra/unittests/clang-doc/MergeTest.cpp
===
--- clang-tools-extra/unittests/clang-doc/MergeTest.cpp
+++ clang-tools-extra/unittests/clang-doc/MergeTest.cpp
@@ -78,6 +78,7 @@
 TEST(MergeTest, mergeRecordInfos) {
   RecordInfo One;
   One.Name = "r";
+  One.IsTypeDef = true;
   One.Namespace.emplace_back(EmptySID, "A", InfoType::IT_namespace);
 
   One.DefLoc = Location(10, llvm::SmallString<16>{"test.cpp"});
@@ -119,6 +120,7 @@
 
   auto Expected = std::make_unique();
   Expected->Name = "r";
+  Expected->IsTypeDef = true;
   Expected->Namespace.emplace_back(EmptySID, "A", InfoType::IT_namespace);
 
   Expected->DefLoc = Location(10, llvm::SmallString<16>{"test.cpp"});
Index: clang-tools-extra/clang-doc/tool/ClangDocMain.cpp
===
--- clang-tools-extra/clang-doc/tool/ClangDocMain.cpp
+++ clang-tools-extra/clang-doc/tool/ClangDocMain.cpp
@@ -231,9 +231,10 @@
   if (Format == "html") {
 void *MainAddr = (void *)(intptr_t)GetExecutablePath;
 std::string ClangDocPath = GetExecutablePath(argv[0], MainAddr);
+llvm::SmallString<128> NativeClangDocPath;
+llvm::sys::path::native(ClangDocPath, NativeClangDocPath);
 llvm::SmallString<128> AssetsPath;
-llvm::sys::path::native(ClangDocPath, AssetsPath);
-AssetsPath = llvm::sys::path::parent_path(AssetsPath);
+AssetsPath = llvm::sys::path::parent_path(NativeClangDocPath);
 llvm::sys::path::append(AssetsPath, "..", "share", "clang");
 llvm::SmallString<128> DefaultStylesheet;
 llvm::sys::path::native(AssetsPath, DefaultStylesheet);
Index: clang-tools-extra/clang-doc/YAMLGenerator.cpp
===
--- clang-tools-extra/clang-doc/YAMLGenerator.cpp
+++ clang-tools-extra/clang-doc/YAMLGenerator.cpp
@@ -127,7 +127,8 @@
 
 static void RecordInfoMapping(IO , RecordInfo ) {
   SymbolInfoMapping(IO, I);
-  IO.mapOptional("TagType", I.TagType, clang::TagTypeKind::TTK_Struct);
+  IO.mapOptional("TagType", I.TagType);
+  IO.mapOptional("IsTypeDef", I.IsTypeDef, false);
   IO.mapOptional("Members", I.Members);
   IO.mapOptional("Bases", I.Bases);
   IO.mapOptional("Parents", I.Parents, llvm::SmallVector());
Index: clang-tools-extra/clang-doc/Representation.h
===
--- clang-tools-extra/clang-doc/Representation.h
+++ clang-tools-extra/clang-doc/Representation.h
@@ -348,10 +348,15 @@
 
   void merge(RecordInfo &);
 
-  TagTypeKind TagType = TagTypeKind::TTK_Struct; // Type of this record

[PATCH] D131298: [clang-doc] Read docstrings for record members

[Brett Wilson via Phabricator via cfe-commits]

brettw updated this revision to Diff 451882.
brettw marked 2 inline comments as done.
brettw added a comment.

New patch up with requested changes.


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D131298/new/

https://reviews.llvm.org/D131298

Files:
  clang-tools-extra/clang-doc/BitcodeReader.cpp
  clang-tools-extra/clang-doc/BitcodeWriter.cpp
  clang-tools-extra/clang-doc/Representation.h
  clang-tools-extra/clang-doc/Serialize.cpp
  clang-tools-extra/clang-doc/YAMLGenerator.cpp
  clang-tools-extra/unittests/clang-doc/BitcodeTest.cpp
  clang-tools-extra/unittests/clang-doc/ClangDocTest.cpp
  clang-tools-extra/unittests/clang-doc/SerializeTest.cpp
  clang-tools-extra/unittests/clang-doc/YAMLGeneratorTest.cpp

Index: clang-tools-extra/unittests/clang-doc/YAMLGeneratorTest.cpp
===
--- clang-tools-extra/unittests/clang-doc/YAMLGeneratorTest.cpp
+++ clang-tools-extra/unittests/clang-doc/YAMLGeneratorTest.cpp
@@ -83,6 +83,19 @@
 
   I.Members.emplace_back("int", "path/to/int", "X",
  AccessSpecifier::AS_private);
+
+  // Member documentation.
+  CommentInfo TopComment;
+  TopComment.Kind = "FullComment";
+  TopComment.Children.emplace_back(std::make_unique());
+  CommentInfo *Brief = TopComment.Children.back().get();
+  Brief->Kind = "ParagraphComment";
+  Brief->Children.emplace_back(std::make_unique());
+  Brief->Children.back()->Kind = "TextComment";
+  Brief->Children.back()->Name = "ParagraphComment";
+  Brief->Children.back()->Text = "Value of the thing.";
+  I.Members.back().Description.push_back(std::move(TopComment));
+
   I.TagType = TagTypeKind::TTK_Class;
   I.Bases.emplace_back(EmptySID, "F", "path/to/F", true,
AccessSpecifier::AS_public, true);
@@ -129,6 +142,14 @@
   Path:'path/to/int'
 Name:'X'
 Access:  Private
+Description:
+  - Kind:'FullComment'
+Children:
+  - Kind:'ParagraphComment'
+Children:
+  - Kind:'TextComment'
+Text:'Value of the thing.'
+Name:'ParagraphComment'
 Bases:
   - USR: ''
 Name:'F'
Index: clang-tools-extra/unittests/clang-doc/SerializeTest.cpp
===
--- clang-tools-extra/unittests/clang-doc/SerializeTest.cpp
+++ clang-tools-extra/unittests/clang-doc/SerializeTest.cpp
@@ -80,6 +80,26 @@
   ASSERT_EQ(NumExpectedInfos, EmittedInfos.size());
 }
 
+// Constructs a comment definition as the parser would for one comment line.
+/* TODO uncomment this when the missing comment is fixed in emitRecordInfo and
+   the code that calls this is re-enabled.
+CommentInfo MakeOneLineCommentInfo(const std::string ) {
+  CommentInfo TopComment;
+  TopComment.Kind = "FullComment";
+  TopComment.Children.emplace_back(std::make_unique());
+
+  CommentInfo *Brief = TopComment.Children.back().get();
+  Brief->Kind = "ParagraphComment";
+
+  Brief->Children.emplace_back(std::make_unique());
+  Brief->Children.back()->Kind = "TextComment";
+  Brief->Children.back()->Name = "ParagraphComment";
+  Brief->Children.back()->Text = Text;
+
+  return TopComment;
+}
+*/
+
 // Test serialization of namespace declarations.
 TEST(SerializeTest, emitNamespaceInfo) {
   EmittedInfoList Infos;
@@ -124,6 +144,9 @@
   ExtractInfosFromCode(R"raw(class E {
 public:
   E() {}
+
+  // Some docs.
+  int value;
 protected:
   void ProtectedMethod();
 };
@@ -142,6 +165,9 @@
InfoType::IT_namespace);
   ExpectedE.TagType = TagTypeKind::TTK_Class;
   ExpectedE.DefLoc = Location(0, llvm::SmallString<16>{"test.cpp"});
+  ExpectedE.Members.emplace_back("int", "value", AccessSpecifier::AS_public);
+  // TODO the data member should have the docstring on it:
+  //ExpectedE.Members.back().Description.push_back(MakeOneLineCommentInfo(" Some docs"));
   CheckRecordInfo(, E);
 
   RecordInfo *RecordWithEConstructor = InfoAsRecord(Infos[2].get());
Index: clang-tools-extra/unittests/clang-doc/ClangDocTest.cpp
===
--- clang-tools-extra/unittests/clang-doc/ClangDocTest.cpp
+++ clang-tools-extra/unittests/clang-doc/ClangDocTest.cpp
@@ -34,7 +34,12 @@
   return static_cast(I);
 }
 
-void CheckCommentInfo(CommentInfo , CommentInfo ) {
+void CheckCommentInfo(const std::vector ,
+  const std::vector );
+void CheckCommentInfo(const std::vector> ,
+  const std::vector> );
+
+void CheckCommentInfo(const CommentInfo , const CommentInfo ) {
   EXPECT_EQ(Expected.Kind, Actual.Kind);
   EXPECT_EQ(Expected.Text, Actual.Text);
   EXPECT_EQ(Expected.Name, Actual.Name);
@@ -56,9 +61,21 @@
   for (size_t Idx = 0; Idx < Actual.Args.size(); ++Idx)
 EXPECT_EQ(Expected.Args[Idx], 

[PATCH] D131298: [clang-doc] Read docstrings for record members

[Brett Wilson via Phabricator via cfe-commits]

brettw added inline comments.



Comment at: clang-tools-extra/unittests/clang-doc/SerializeTest.cpp:166
+  ExpectedE.Members.emplace_back("int", "value", AccessSpecifier::AS_public);
+  //ExpectedE.Members.back().Description.push_back(MakeOneLineCommentInfo(" 
Some docs"));
   CheckRecordInfo(, E);

paulkirth wrote:
> brettw wrote:
> > I spent quite some time trying to get this to work and I'm stuck. If I copy 
> > the contents of the raw string above into a file and run clang-doc on it, I 
> > get a comment in the YAML output. But in this test I never get a comment 
> > and I don't know how this parsing environment is different than the "real" 
> > one.
> > 
> > I'm very confused by the result of `ExtractInfosFromCode` function in the 
> > first place. What are the 1st and 6th item which are never checked? Why 
> > does it emit two duplicate records for the `E` struct, one (Infos[2]) with 
> > a data member and no function and one (Infos[3]) with no data member and a 
> > function in it?
> > 
> > Any suggestions here would be appreciated.
> So I also wanted to understand why. `MakeOneLinecCmmentIInfo()` is not used 
> elsewhere in clang-doc, and I don't think it was ever tested.
> 
> Looking at the body it doesn't use it's input parameter at all. 
> 
> There may also be an issue w/ how it constructs the `CommentInfo`.
> 
> Leaving this as a TODO is fine. I don't think this is something we're going 
> to solve in this patch, but if you can also add a TODO to `MakeOneLine 
> CommentInfo`, it would be appreciated.
The problem isn't with MakeOneLineCommentInfo (I fixed the parameter usage) but 
rather that the records generated in the test has no comments at all. The 
current test code (with the line commented out) expects that the comment is 
completely missing, which is the only way the test will pass now.

Since nobody else uses it, I commented it out and referenced the calling code.


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D131298/new/

https://reviews.llvm.org/D131298

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D131298: [clang-doc] Read docstrings for record members

[Brett Wilson via Phabricator via cfe-commits]

brettw updated this revision to Diff 451632.
brettw added a comment.

New patch up.


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D131298/new/

https://reviews.llvm.org/D131298

Files:
  clang-tools-extra/clang-doc/BitcodeReader.cpp
  clang-tools-extra/clang-doc/BitcodeWriter.cpp
  clang-tools-extra/clang-doc/Representation.h
  clang-tools-extra/clang-doc/Serialize.cpp
  clang-tools-extra/clang-doc/YAMLGenerator.cpp
  clang-tools-extra/unittests/clang-doc/BitcodeTest.cpp
  clang-tools-extra/unittests/clang-doc/ClangDocTest.cpp
  clang-tools-extra/unittests/clang-doc/SerializeTest.cpp
  clang-tools-extra/unittests/clang-doc/YAMLGeneratorTest.cpp

Index: clang-tools-extra/unittests/clang-doc/YAMLGeneratorTest.cpp
===
--- clang-tools-extra/unittests/clang-doc/YAMLGeneratorTest.cpp
+++ clang-tools-extra/unittests/clang-doc/YAMLGeneratorTest.cpp
@@ -83,6 +83,19 @@
 
   I.Members.emplace_back("int", "path/to/int", "X",
  AccessSpecifier::AS_private);
+
+  // Member documentation.
+  CommentInfo TopComment;
+  TopComment.Kind = "FullComment";
+  TopComment.Children.emplace_back(std::make_unique());
+  CommentInfo *Brief = TopComment.Children.back().get();
+  Brief->Kind = "ParagraphComment";
+  Brief->Children.emplace_back(std::make_unique());
+  Brief->Children.back()->Kind = "TextComment";
+  Brief->Children.back()->Name = "ParagraphComment";
+  Brief->Children.back()->Text = "Value of the thing.";
+  I.Members.back().Description.push_back(std::move(TopComment));
+
   I.TagType = TagTypeKind::TTK_Class;
   I.Bases.emplace_back(EmptySID, "F", "path/to/F", true,
AccessSpecifier::AS_public, true);
@@ -129,6 +142,14 @@
   Path:'path/to/int'
 Name:'X'
 Access:  Private
+Description:
+  - Kind:'FullComment'
+Children:
+  - Kind:'ParagraphComment'
+Children:
+  - Kind:'TextComment'
+Text:'Value of the thing.'
+Name:'ParagraphComment'
 Bases:
   - USR: ''
 Name:'F'
Index: clang-tools-extra/unittests/clang-doc/SerializeTest.cpp
===
--- clang-tools-extra/unittests/clang-doc/SerializeTest.cpp
+++ clang-tools-extra/unittests/clang-doc/SerializeTest.cpp
@@ -80,6 +80,26 @@
   ASSERT_EQ(NumExpectedInfos, EmittedInfos.size());
 }
 
+// Constructs a comment definition as the parser would for one comment line.
+/* TODO uncomment this when the missing comment is fixed in emitRecordInfo and
+   the code that calls this is re-enabled.
+CommentInfo MakeOneLineCommentInfo(const std::string ) {
+  CommentInfo TopComment;
+  TopComment.Kind = "FullComment";
+  TopComment.Children.emplace_back(std::make_unique());
+
+  CommentInfo *Brief = TopComment.Children.back().get();
+  Brief->Kind = "ParagraphComment";
+
+  Brief->Children.emplace_back(std::make_unique());
+  Brief->Children.back()->Kind = "TextComment";
+  Brief->Children.back()->Name = "ParagraphComment";
+  Brief->Children.back()->Text = Text;
+
+  return TopComment;
+}
+*/
+
 // Test serialization of namespace declarations.
 TEST(SerializeTest, emitNamespaceInfo) {
   EmittedInfoList Infos;
@@ -124,6 +144,9 @@
   ExtractInfosFromCode(R"raw(class E {
 public:
   E() {}
+
+  // Some docs.
+  int value;
 protected:
   void ProtectedMethod();
 };
@@ -142,6 +165,9 @@
InfoType::IT_namespace);
   ExpectedE.TagType = TagTypeKind::TTK_Class;
   ExpectedE.DefLoc = Location(0, llvm::SmallString<16>{"test.cpp"});
+  ExpectedE.Members.emplace_back("int", "value", AccessSpecifier::AS_public);
+  // TODO the data member should have the docstring on it:
+  //ExpectedE.Members.back().Description.push_back(MakeOneLineCommentInfo(" Some docs"));
   CheckRecordInfo(, E);
 
   RecordInfo *RecordWithEConstructor = InfoAsRecord(Infos[2].get());
Index: clang-tools-extra/unittests/clang-doc/ClangDocTest.cpp
===
--- clang-tools-extra/unittests/clang-doc/ClangDocTest.cpp
+++ clang-tools-extra/unittests/clang-doc/ClangDocTest.cpp
@@ -34,7 +34,12 @@
   return static_cast(I);
 }
 
-void CheckCommentInfo(CommentInfo , CommentInfo ) {
+void CheckCommentInfo(const std::vector ,
+  const std::vector );
+void CheckCommentInfo(const std::vector> ,
+  const std::vector> );
+
+void CheckCommentInfo(const CommentInfo , const CommentInfo ) {
   EXPECT_EQ(Expected.Kind, Actual.Kind);
   EXPECT_EQ(Expected.Text, Actual.Text);
   EXPECT_EQ(Expected.Name, Actual.Name);
@@ -56,9 +61,21 @@
   for (size_t Idx = 0; Idx < Actual.Args.size(); ++Idx)
 EXPECT_EQ(Expected.Args[Idx], Actual.Args[Idx]);
 
-  ASSERT_EQ(Expected.Children.size(), 

[PATCH] D131298: [clang-doc] Read docstrings for record members

[Brett Wilson via Phabricator via cfe-commits]

brettw updated this revision to Diff 451299.
brettw marked an inline comment as done.
brettw added a comment.

I updated the comment test that's not currently working with a TODO.

There are some open questions in the code review about some bigger questions 
about this tool. Since this has been virtually abandoned for several years, I 
don't think anybody knows these answers and we're not going to figure out 
everything for this patch.

I think this patch makes things incrementally better and I'd like to land 
something soon so I can work on the next missing feature. I think over time we 
can build up knowledge about this tool and make it better on a wider scale. Can 
you please talk another look?


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D131298/new/

https://reviews.llvm.org/D131298

Files:
  clang-tools-extra/clang-doc/BitcodeReader.cpp
  clang-tools-extra/clang-doc/BitcodeWriter.cpp
  clang-tools-extra/clang-doc/Representation.h
  clang-tools-extra/clang-doc/Serialize.cpp
  clang-tools-extra/clang-doc/YAMLGenerator.cpp
  clang-tools-extra/unittests/clang-doc/BitcodeTest.cpp
  clang-tools-extra/unittests/clang-doc/ClangDocTest.cpp
  clang-tools-extra/unittests/clang-doc/SerializeTest.cpp
  clang-tools-extra/unittests/clang-doc/YAMLGeneratorTest.cpp

Index: clang-tools-extra/unittests/clang-doc/YAMLGeneratorTest.cpp
===
--- clang-tools-extra/unittests/clang-doc/YAMLGeneratorTest.cpp
+++ clang-tools-extra/unittests/clang-doc/YAMLGeneratorTest.cpp
@@ -83,6 +83,19 @@
 
   I.Members.emplace_back("int", "path/to/int", "X",
  AccessSpecifier::AS_private);
+
+  // Member documentation.
+  CommentInfo TopComment;
+  TopComment.Kind = "FullComment";
+  TopComment.Children.emplace_back(std::make_unique());
+  CommentInfo *Brief = TopComment.Children.back().get();
+  Brief->Kind = "ParagraphComment";
+  Brief->Children.emplace_back(std::make_unique());
+  Brief->Children.back()->Kind = "TextComment";
+  Brief->Children.back()->Name = "ParagraphComment";
+  Brief->Children.back()->Text = "Value of the thing.";
+  I.Members.back().Description.push_back(std::move(TopComment));
+
   I.TagType = TagTypeKind::TTK_Class;
   I.Bases.emplace_back(EmptySID, "F", "path/to/F", true,
AccessSpecifier::AS_public, true);
@@ -129,6 +142,14 @@
   Path:'path/to/int'
 Name:'X'
 Access:  Private
+Description:
+  - Kind:'FullComment'
+Children:
+  - Kind:'ParagraphComment'
+Children:
+  - Kind:'TextComment'
+Text:'Value of the thing.'
+Name:'ParagraphComment'
 Bases:
   - USR: ''
 Name:'F'
Index: clang-tools-extra/unittests/clang-doc/SerializeTest.cpp
===
--- clang-tools-extra/unittests/clang-doc/SerializeTest.cpp
+++ clang-tools-extra/unittests/clang-doc/SerializeTest.cpp
@@ -80,6 +80,23 @@
   ASSERT_EQ(NumExpectedInfos, EmittedInfos.size());
 }
 
+// Constructs a comment definition as the parser would for one comment line.
+CommentInfo MakeOneLineCommentInfo(const std::string ) {
+  CommentInfo TopComment;
+  TopComment.Kind = "FullComment";
+  TopComment.Children.emplace_back(std::make_unique());
+
+  CommentInfo *Brief = TopComment.Children.back().get();
+  Brief->Kind = "ParagraphComment";
+
+  Brief->Children.emplace_back(std::make_unique());
+  Brief->Children.back()->Kind = "TextComment";
+  Brief->Children.back()->Name = "ParagraphComment";
+  Brief->Children.back()->Text = "Value of the thing.";
+
+  return TopComment;
+}
+
 // Test serialization of namespace declarations.
 TEST(SerializeTest, emitNamespaceInfo) {
   EmittedInfoList Infos;
@@ -124,6 +141,9 @@
   ExtractInfosFromCode(R"raw(class E {
 public:
   E() {}
+
+  // Some docs.
+  int value;
 protected:
   void ProtectedMethod();
 };
@@ -142,6 +162,9 @@
InfoType::IT_namespace);
   ExpectedE.TagType = TagTypeKind::TTK_Class;
   ExpectedE.DefLoc = Location(0, llvm::SmallString<16>{"test.cpp"});
+  ExpectedE.Members.emplace_back("int", "value", AccessSpecifier::AS_public);
+  // TODO the data member should have the docstring on it:
+  //ExpectedE.Members.back().Description.push_back(MakeOneLineCommentInfo(" Some docs"));
   CheckRecordInfo(, E);
 
   RecordInfo *RecordWithEConstructor = InfoAsRecord(Infos[2].get());
Index: clang-tools-extra/unittests/clang-doc/ClangDocTest.cpp
===
--- clang-tools-extra/unittests/clang-doc/ClangDocTest.cpp
+++ clang-tools-extra/unittests/clang-doc/ClangDocTest.cpp
@@ -34,7 +34,12 @@
   return static_cast(I);
 }
 
-void CheckCommentInfo(CommentInfo , CommentInfo ) {
+void CheckCommentInfo(const std::vector ,

[PATCH] D131298: [clang-doc] Read docstrings for record members

[Brett Wilson via Phabricator via cfe-commits]

brettw marked 2 inline comments as done.
brettw added inline comments.



Comment at: clang-tools-extra/clang-doc/Serialize.cpp:439
+static void populateMemberTypeInfo(MemberTypeInfo , const FieldDecl *D) {
+  ASTContext& Context = D->getASTContext();
+  RawComment *Comment = Context.getRawCommentForDeclNoCache(D);

paulkirth wrote:
> brettw wrote:
> > paulkirth wrote:
> > > I think we need to at least assert that `D != nullptr`. While there is 
> > > only one caller now, its likely there may be more in the future. Plus 
> > > `nullptr` is deref is never fun to debug.  
> > > 
> > > If getting a `nullptr` is expected to be a common input, then an early 
> > > return is also fine.
> > > 
> > > If you prefer, a few places in the codebase do both to maintain correct 
> > > behavior when asserts are disabled. Often they have a form along these 
> > > lines:
> > > 
> > > ```
> > > if(!D){
> > >   assert(false && "Some error message");
> > >   return;
> > > }
> > > ```
> > > 
> > > Any of these would be fine, and I don't have a preference between them.
> > > 
> > I'm fine adding an assert to unblock the review.
> > 
> > But clang-doc is full of pointers and none of them are asserted on. Is this 
> > one special in some way?  To me, it seems weird to assert on this one 
> > random pointer. Is there some policy about this?
> > 
> > I follow the opposite theory that asserts on null pointers all over the 
> > place clutter the code and random null parameters are one of the easiest 
> > types of crashes to debug.
> There's nothing special regarding this particular pointer, but such checks 
> are common in other parts of LLVM.
> 
> We use asserts liberally throughout the LLVM codebase 
> (https://llvm.org/docs/CodingStandards.html#assert-liberally) and asserting 
> that a pointer is valid is common. 
> 
> The other thing to keep in mind is that debug builds of LLVM are > 20GB, and 
> enabling asserts is a more desirable choice for most of our developers. That 
> probably isn't as bad for clang-doc, but I'd rather err on the side of 
> caution.
> 
> While I agree that right now it may be a bit strange to have a single assert, 
>  hopefully overtime the surrounding code will start to use such checks in a 
> way that is more consistent with the rest of the codebase.
> 
> But like I said, an early return is also fine IMO.
> 
Done


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D131298/new/

https://reviews.llvm.org/D131298

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D131298: [clang-doc] Read docstrings for record members

[Brett Wilson via Phabricator via cfe-commits]

brettw updated this revision to Diff 451001.
brettw marked an inline comment as done.

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D131298/new/

https://reviews.llvm.org/D131298

Files:
  clang-tools-extra/clang-doc/BitcodeReader.cpp
  clang-tools-extra/clang-doc/BitcodeWriter.cpp
  clang-tools-extra/clang-doc/Representation.h
  clang-tools-extra/clang-doc/Serialize.cpp
  clang-tools-extra/clang-doc/YAMLGenerator.cpp
  clang-tools-extra/unittests/clang-doc/BitcodeTest.cpp
  clang-tools-extra/unittests/clang-doc/ClangDocTest.cpp
  clang-tools-extra/unittests/clang-doc/SerializeTest.cpp
  clang-tools-extra/unittests/clang-doc/YAMLGeneratorTest.cpp

Index: clang-tools-extra/unittests/clang-doc/YAMLGeneratorTest.cpp
===
--- clang-tools-extra/unittests/clang-doc/YAMLGeneratorTest.cpp
+++ clang-tools-extra/unittests/clang-doc/YAMLGeneratorTest.cpp
@@ -83,6 +83,19 @@
 
   I.Members.emplace_back("int", "path/to/int", "X",
  AccessSpecifier::AS_private);
+
+  // Member documentation.
+  CommentInfo TopComment;
+  TopComment.Kind = "FullComment";
+  TopComment.Children.emplace_back(std::make_unique());
+  CommentInfo *Brief = TopComment.Children.back().get();
+  Brief->Kind = "ParagraphComment";
+  Brief->Children.emplace_back(std::make_unique());
+  Brief->Children.back()->Kind = "TextComment";
+  Brief->Children.back()->Name = "ParagraphComment";
+  Brief->Children.back()->Text = "Value of the thing.";
+  I.Members.back().Description.push_back(std::move(TopComment));
+
   I.TagType = TagTypeKind::TTK_Class;
   I.Bases.emplace_back(EmptySID, "F", "path/to/F", true,
AccessSpecifier::AS_public, true);
@@ -129,6 +142,14 @@
   Path:'path/to/int'
 Name:'X'
 Access:  Private
+Description:
+  - Kind:'FullComment'
+Children:
+  - Kind:'ParagraphComment'
+Children:
+  - Kind:'TextComment'
+Text:'Value of the thing.'
+Name:'ParagraphComment'
 Bases:
   - USR: ''
 Name:'F'
Index: clang-tools-extra/unittests/clang-doc/SerializeTest.cpp
===
--- clang-tools-extra/unittests/clang-doc/SerializeTest.cpp
+++ clang-tools-extra/unittests/clang-doc/SerializeTest.cpp
@@ -80,6 +80,23 @@
   ASSERT_EQ(NumExpectedInfos, EmittedInfos.size());
 }
 
+// Constructs a comment definition as the parser would for one comment line.
+CommentInfo MakeOneLineCommentInfo(const std::string ) {
+  CommentInfo TopComment;
+  TopComment.Kind = "FullComment";
+  TopComment.Children.emplace_back(std::make_unique());
+
+  CommentInfo *Brief = TopComment.Children.back().get();
+  Brief->Kind = "ParagraphComment";
+
+  Brief->Children.emplace_back(std::make_unique());
+  Brief->Children.back()->Kind = "TextComment";
+  Brief->Children.back()->Name = "ParagraphComment";
+  Brief->Children.back()->Text = "Value of the thing.";
+
+  return TopComment;
+}
+
 // Test serialization of namespace declarations.
 TEST(SerializeTest, emitNamespaceInfo) {
   EmittedInfoList Infos;
@@ -124,6 +141,9 @@
   ExtractInfosFromCode(R"raw(class E {
 public:
   E() {}
+
+  // Some docs.
+  int value;
 protected:
   void ProtectedMethod();
 };
@@ -142,6 +162,8 @@
InfoType::IT_namespace);
   ExpectedE.TagType = TagTypeKind::TTK_Class;
   ExpectedE.DefLoc = Location(0, llvm::SmallString<16>{"test.cpp"});
+  ExpectedE.Members.emplace_back("int", "value", AccessSpecifier::AS_public);
+  //ExpectedE.Members.back().Description.push_back(MakeOneLineCommentInfo(" Some docs"));
   CheckRecordInfo(, E);
 
   RecordInfo *RecordWithEConstructor = InfoAsRecord(Infos[2].get());
Index: clang-tools-extra/unittests/clang-doc/ClangDocTest.cpp
===
--- clang-tools-extra/unittests/clang-doc/ClangDocTest.cpp
+++ clang-tools-extra/unittests/clang-doc/ClangDocTest.cpp
@@ -34,7 +34,12 @@
   return static_cast(I);
 }
 
-void CheckCommentInfo(CommentInfo , CommentInfo ) {
+void CheckCommentInfo(const std::vector ,
+  const std::vector );
+void CheckCommentInfo(const std::vector> ,
+  const std::vector> );
+
+void CheckCommentInfo(const CommentInfo , const CommentInfo ) {
   EXPECT_EQ(Expected.Kind, Actual.Kind);
   EXPECT_EQ(Expected.Text, Actual.Text);
   EXPECT_EQ(Expected.Name, Actual.Name);
@@ -56,9 +61,21 @@
   for (size_t Idx = 0; Idx < Actual.Args.size(); ++Idx)
 EXPECT_EQ(Expected.Args[Idx], Actual.Args[Idx]);
 
-  ASSERT_EQ(Expected.Children.size(), Actual.Children.size());
-  for (size_t Idx = 0; Idx < Actual.Children.size(); ++Idx)
-CheckCommentInfo(*Expected.Children[Idx], *Actual.Children[Idx]);
+  

[PATCH] D131298: [clang-doc] Read docstrings for record members

[Brett Wilson via Phabricator via cfe-commits]

brettw added a comment.






Comment at: clang-tools-extra/clang-doc/BitcodeReader.cpp:354
+template <> llvm::Expected getCommentInfo(MemberTypeInfo *I) {
+  I->Description.emplace_back();
+  return >Description.back();

paulkirth wrote:
> So, I see that this uses the same pattern as in other `getCommentInfo(...)` 
> API's, but I'm a bit confused by this.
> 
> Do we always grow the vector whenever `getCommentInfo(...)` is  called? I 
> would expect a `get...` function to be `const` and just return data. This 
> grows the `Description` vector on every call, which seems like an odd choice. 
> The pattern is also pervasive in BitcodeReader.cpp.
> 
> @phosek is this intentional? If `Description` exceeds capacity and reallocs 
> on `emplace_back`, then any reference it had returned would be invalidated, 
> right? Or am I missing something here re: the various `*TypeInfos` and how 
> clang-doc uses them?
You are correct that this always grows. These functions are only called once 
and that's when readSubBlock gets a block ID. The comment is filled into the 
returned array.

My question is why the comments on the info structs are arrays. I have only 
ever seen one top-level comment of type "FullComment" which can have many 
children.



Comment at: clang-tools-extra/clang-doc/Serialize.cpp:439
+static void populateMemberTypeInfo(MemberTypeInfo , const FieldDecl *D) {
+  ASTContext& Context = D->getASTContext();
+  RawComment *Comment = Context.getRawCommentForDeclNoCache(D);

paulkirth wrote:
> I think we need to at least assert that `D != nullptr`. While there is only 
> one caller now, its likely there may be more in the future. Plus `nullptr` is 
> deref is never fun to debug.  
> 
> If getting a `nullptr` is expected to be a common input, then an early return 
> is also fine.
> 
> If you prefer, a few places in the codebase do both to maintain correct 
> behavior when asserts are disabled. Often they have a form along these lines:
> 
> ```
> if(!D){
>   assert(false && "Some error message");
>   return;
> }
> ```
> 
> Any of these would be fine, and I don't have a preference between them.
> 
I'm fine adding an assert to unblock the review.

But clang-doc is full of pointers and none of them are asserted on. Is this one 
special in some way?  To me, it seems weird to assert on this one random 
pointer. Is there some policy about this?

I follow the opposite theory that asserts on null pointers all over the place 
clutter the code and random null parameters are one of the easiest types of 
crashes to debug.



Comment at: clang-tools-extra/clang-doc/Serialize.cpp:440
+  ASTContext& Context = D->getASTContext();
+  RawComment *Comment = Context.getRawCommentForDeclNoCache(D);
+  if (!Comment)

paulkirth wrote:
> If the goal is to get the FullComment, then doesn't `getCommentForDecl(...) ` 
> handle that? This also seems like an area where we'd want to use caching if 
> we could.
I have no idea how this works, I copied this snipped from 
MapASTVisitor::getComment in Mapper.cpp



Comment at: clang-tools-extra/clang-doc/YAMLGenerator.cpp:195
 IO.mapOptional("Access", I.Access, clang::AccessSpecifier::AS_none);
+IO.mapOptional("Description", I.Description);
   }

paulkirth wrote:
> Can we get a comment describing this?
I don't think this needs a comment, a substantial part of this file is just 
adding these mappings between YAML keys and field names. The one above it has a 
comment only because of the non-obvious default value. If you feel strongly, 
please let me know what the comment should say.



Comment at: clang-tools-extra/unittests/clang-doc/BitcodeTest.cpp:291
+/*
+TEST(BitcodeTest, emitMemberTypeInfo) {
+  MemberTypeInfo I;

Whoops, I'll delete this.



Comment at: clang-tools-extra/unittests/clang-doc/SerializeTest.cpp:166
+  ExpectedE.Members.emplace_back("int", "value", AccessSpecifier::AS_public);
+  //ExpectedE.Members.back().Description.push_back(MakeOneLineCommentInfo(" 
Some docs"));
   CheckRecordInfo(, E);

I spent quite some time trying to get this to work and I'm stuck. If I copy the 
contents of the raw string above into a file and run clang-doc on it, I get a 
comment in the YAML output. But in this test I never get a comment and I don't 
know how this parsing environment is different than the "real" one.

I'm very confused by the result of `ExtractInfosFromCode` function in the first 
place. What are the 1st and 6th item which are never checked? Why does it emit 
two duplicate records for the `E` struct, one (Infos[2]) with a data member and 
no function and one (Infos[3]) with no data member and a function in it?

Any suggestions here would be appreciated.


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D131298/new/

https://reviews.llvm.org/D131298

[PATCH] D131298: [clang-doc] Read docstrings for record members

[Brett Wilson via Phabricator via cfe-commits]

brettw updated this revision to Diff 450976.

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D131298/new/

https://reviews.llvm.org/D131298

Files:
  clang-tools-extra/clang-doc/BitcodeReader.cpp
  clang-tools-extra/clang-doc/BitcodeWriter.cpp
  clang-tools-extra/clang-doc/Representation.h
  clang-tools-extra/clang-doc/Serialize.cpp
  clang-tools-extra/clang-doc/YAMLGenerator.cpp
  clang-tools-extra/unittests/clang-doc/BitcodeTest.cpp
  clang-tools-extra/unittests/clang-doc/ClangDocTest.cpp
  clang-tools-extra/unittests/clang-doc/SerializeTest.cpp
  clang-tools-extra/unittests/clang-doc/YAMLGeneratorTest.cpp

Index: clang-tools-extra/unittests/clang-doc/YAMLGeneratorTest.cpp
===
--- clang-tools-extra/unittests/clang-doc/YAMLGeneratorTest.cpp
+++ clang-tools-extra/unittests/clang-doc/YAMLGeneratorTest.cpp
@@ -83,6 +83,19 @@
 
   I.Members.emplace_back("int", "path/to/int", "X",
  AccessSpecifier::AS_private);
+
+  // Member documentation.
+  CommentInfo TopComment;
+  TopComment.Kind = "FullComment";
+  TopComment.Children.emplace_back(std::make_unique());
+  CommentInfo *Brief = TopComment.Children.back().get();
+  Brief->Kind = "ParagraphComment";
+  Brief->Children.emplace_back(std::make_unique());
+  Brief->Children.back()->Kind = "TextComment";
+  Brief->Children.back()->Name = "ParagraphComment";
+  Brief->Children.back()->Text = "Value of the thing.";
+  I.Members.back().Description.push_back(std::move(TopComment));
+
   I.TagType = TagTypeKind::TTK_Class;
   I.Bases.emplace_back(EmptySID, "F", "path/to/F", true,
AccessSpecifier::AS_public, true);
@@ -129,6 +142,14 @@
   Path:'path/to/int'
 Name:'X'
 Access:  Private
+Description:
+  - Kind:'FullComment'
+Children:
+  - Kind:'ParagraphComment'
+Children:
+  - Kind:'TextComment'
+Text:'Value of the thing.'
+Name:'ParagraphComment'
 Bases:
   - USR: ''
 Name:'F'
Index: clang-tools-extra/unittests/clang-doc/SerializeTest.cpp
===
--- clang-tools-extra/unittests/clang-doc/SerializeTest.cpp
+++ clang-tools-extra/unittests/clang-doc/SerializeTest.cpp
@@ -80,6 +80,23 @@
   ASSERT_EQ(NumExpectedInfos, EmittedInfos.size());
 }
 
+// Constructs a comment definition as the parser would for one comment line.
+CommentInfo MakeOneLineCommentInfo(const std::string ) {
+  CommentInfo TopComment;
+  TopComment.Kind = "FullComment";
+  TopComment.Children.emplace_back(std::make_unique());
+
+  CommentInfo *Brief = TopComment.Children.back().get();
+  Brief->Kind = "ParagraphComment";
+
+  Brief->Children.emplace_back(std::make_unique());
+  Brief->Children.back()->Kind = "TextComment";
+  Brief->Children.back()->Name = "ParagraphComment";
+  Brief->Children.back()->Text = "Value of the thing.";
+
+  return TopComment;
+}
+
 // Test serialization of namespace declarations.
 TEST(SerializeTest, emitNamespaceInfo) {
   EmittedInfoList Infos;
@@ -124,6 +141,9 @@
   ExtractInfosFromCode(R"raw(class E {
 public:
   E() {}
+
+  // Some docs.
+  int value;
 protected:
   void ProtectedMethod();
 };
@@ -142,6 +162,8 @@
InfoType::IT_namespace);
   ExpectedE.TagType = TagTypeKind::TTK_Class;
   ExpectedE.DefLoc = Location(0, llvm::SmallString<16>{"test.cpp"});
+  ExpectedE.Members.emplace_back("int", "value", AccessSpecifier::AS_public);
+  //ExpectedE.Members.back().Description.push_back(MakeOneLineCommentInfo(" Some docs"));
   CheckRecordInfo(, E);
 
   RecordInfo *RecordWithEConstructor = InfoAsRecord(Infos[2].get());
@@ -172,6 +194,7 @@
 InfoType::IT_namespace);
   Method.Access = AccessSpecifier::AS_protected;
   Method.IsMethod = true;
+
   ExpectedRecordWithMethod.ChildFunctions.emplace_back(std::move(Method));
   CheckRecordInfo(, RecordWithMethod);
 
Index: clang-tools-extra/unittests/clang-doc/ClangDocTest.cpp
===
--- clang-tools-extra/unittests/clang-doc/ClangDocTest.cpp
+++ clang-tools-extra/unittests/clang-doc/ClangDocTest.cpp
@@ -34,7 +34,12 @@
   return static_cast(I);
 }
 
-void CheckCommentInfo(CommentInfo , CommentInfo ) {
+void CheckCommentInfo(const std::vector ,
+  const std::vector );
+void CheckCommentInfo(const std::vector> ,
+  const std::vector> );
+
+void CheckCommentInfo(const CommentInfo , const CommentInfo ) {
   EXPECT_EQ(Expected.Kind, Actual.Kind);
   EXPECT_EQ(Expected.Text, Actual.Text);
   EXPECT_EQ(Expected.Name, Actual.Name);
@@ -56,9 +61,21 @@
   for (size_t Idx = 0; Idx < Actual.Args.size(); ++Idx)
 EXPECT_EQ(Expected.Args[Idx], 

[PATCH] D131298: [clang-doc] Read docstrings for record members

[Brett Wilson via Phabricator via cfe-commits]

brettw updated this revision to Diff 450967.
brettw added a comment.
Herald added subscribers: usaxena95, arphaman.

Added some tests (with one open question).


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D131298/new/

https://reviews.llvm.org/D131298

Files:
  clang-tools-extra/clang-doc/BitcodeReader.cpp
  clang-tools-extra/clang-doc/BitcodeWriter.cpp
  clang-tools-extra/clang-doc/Representation.h
  clang-tools-extra/clang-doc/Serialize.cpp
  clang-tools-extra/clang-doc/YAMLGenerator.cpp
  clang-tools-extra/unittests/clang-doc/BitcodeTest.cpp
  clang-tools-extra/unittests/clang-doc/ClangDocTest.cpp
  clang-tools-extra/unittests/clang-doc/SerializeTest.cpp
  clang-tools-extra/unittests/clang-doc/YAMLGeneratorTest.cpp

Index: clang-tools-extra/unittests/clang-doc/YAMLGeneratorTest.cpp
===
--- clang-tools-extra/unittests/clang-doc/YAMLGeneratorTest.cpp
+++ clang-tools-extra/unittests/clang-doc/YAMLGeneratorTest.cpp
@@ -83,6 +83,19 @@
 
   I.Members.emplace_back("int", "path/to/int", "X",
  AccessSpecifier::AS_private);
+
+  // Member documentation.
+  CommentInfo TopComment;
+  TopComment.Kind = "FullComment";
+  TopComment.Children.emplace_back(std::make_unique());
+  CommentInfo *Brief = TopComment.Children.back().get();
+  Brief->Kind = "ParagraphComment";
+  Brief->Children.emplace_back(std::make_unique());
+  Brief->Children.back()->Kind = "TextComment";
+  Brief->Children.back()->Name = "ParagraphComment";
+  Brief->Children.back()->Text = "Value of the thing.";
+  I.Members.back().Description.push_back(std::move(TopComment));
+
   I.TagType = TagTypeKind::TTK_Class;
   I.Bases.emplace_back(EmptySID, "F", "path/to/F", true,
AccessSpecifier::AS_public, true);
@@ -129,6 +142,14 @@
   Path:'path/to/int'
 Name:'X'
 Access:  Private
+Description:
+  - Kind:'FullComment'
+Children:
+  - Kind:'ParagraphComment'
+Children:
+  - Kind:'TextComment'
+Text:'Value of the thing.'
+Name:'ParagraphComment'
 Bases:
   - USR: ''
 Name:'F'
Index: clang-tools-extra/unittests/clang-doc/SerializeTest.cpp
===
--- clang-tools-extra/unittests/clang-doc/SerializeTest.cpp
+++ clang-tools-extra/unittests/clang-doc/SerializeTest.cpp
@@ -80,6 +80,23 @@
   ASSERT_EQ(NumExpectedInfos, EmittedInfos.size());
 }
 
+// Constructs a comment definition as the parser would for one comment line.
+CommentInfo MakeOneLineCommentInfo(const std::string ) {
+  CommentInfo TopComment;
+  TopComment.Kind = "FullComment";
+  TopComment.Children.emplace_back(std::make_unique());
+
+  CommentInfo *Brief = TopComment.Children.back().get();
+  Brief->Kind = "ParagraphComment";
+
+  Brief->Children.emplace_back(std::make_unique());
+  Brief->Children.back()->Kind = "TextComment";
+  Brief->Children.back()->Name = "ParagraphComment";
+  Brief->Children.back()->Text = "Value of the thing.";
+
+  return TopComment;
+}
+
 // Test serialization of namespace declarations.
 TEST(SerializeTest, emitNamespaceInfo) {
   EmittedInfoList Infos;
@@ -124,6 +141,9 @@
   ExtractInfosFromCode(R"raw(class E {
 public:
   E() {}
+
+  // Some docs.
+  int value;
 protected:
   void ProtectedMethod();
 };
@@ -142,6 +162,8 @@
InfoType::IT_namespace);
   ExpectedE.TagType = TagTypeKind::TTK_Class;
   ExpectedE.DefLoc = Location(0, llvm::SmallString<16>{"test.cpp"});
+  ExpectedE.Members.emplace_back("int", "value", AccessSpecifier::AS_public);
+  //ExpectedE.Members.back().Description.push_back(MakeOneLineCommentInfo(" Some docs"));
   CheckRecordInfo(, E);
 
   RecordInfo *RecordWithEConstructor = InfoAsRecord(Infos[2].get());
@@ -172,6 +194,7 @@
 InfoType::IT_namespace);
   Method.Access = AccessSpecifier::AS_protected;
   Method.IsMethod = true;
+
   ExpectedRecordWithMethod.ChildFunctions.emplace_back(std::move(Method));
   CheckRecordInfo(, RecordWithMethod);
 
Index: clang-tools-extra/unittests/clang-doc/ClangDocTest.cpp
===
--- clang-tools-extra/unittests/clang-doc/ClangDocTest.cpp
+++ clang-tools-extra/unittests/clang-doc/ClangDocTest.cpp
@@ -34,7 +34,12 @@
   return static_cast(I);
 }
 
-void CheckCommentInfo(CommentInfo , CommentInfo ) {
+void CheckCommentInfo(const std::vector ,
+  const std::vector );
+void CheckCommentInfo(const std::vector> ,
+  const std::vector> );
+
+void CheckCommentInfo(const CommentInfo , const CommentInfo ) {
   EXPECT_EQ(Expected.Kind, Actual.Kind);
   EXPECT_EQ(Expected.Text, Actual.Text);
   EXPECT_EQ(Expected.Name, Actual.Name);
@@ 

[PATCH] D131298: [clang-doc] Read docstrings for record members

[Brett Wilson via Phabricator via cfe-commits]

brettw created this revision.
brettw added a reviewer: phosek.
brettw added a project: clang-tools-extra.
Herald added a project: All.
brettw requested review of this revision.
Herald added a subscriber: cfe-commits.

Struct/class data members did not have the comments associated with them. This 
adds that information to the MemberTypeInfo class and emits it in the YAML. 
This does not update the frontends yet.


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D131298

Files:
  clang-tools-extra/clang-doc/BitcodeReader.cpp
  clang-tools-extra/clang-doc/BitcodeWriter.cpp
  clang-tools-extra/clang-doc/Representation.h
  clang-tools-extra/clang-doc/Serialize.cpp
  clang-tools-extra/clang-doc/YAMLGenerator.cpp


Index: clang-tools-extra/clang-doc/YAMLGenerator.cpp
===
--- clang-tools-extra/clang-doc/YAMLGenerator.cpp
+++ clang-tools-extra/clang-doc/YAMLGenerator.cpp
@@ -192,6 +192,7 @@
 // the AS that shouldn't be part of the output. Even though AS_public is 
the
 // default in the struct, it should be displayed in the YAML output.
 IO.mapOptional("Access", I.Access, clang::AccessSpecifier::AS_none);
+IO.mapOptional("Description", I.Description);
   }
 };
 
Index: clang-tools-extra/clang-doc/Serialize.cpp
===
--- clang-tools-extra/clang-doc/Serialize.cpp
+++ clang-tools-extra/clang-doc/Serialize.cpp
@@ -29,6 +29,8 @@
 populateParentNamespaces(llvm::SmallVector ,
  const T *D, bool );
 
+static void populateMemberTypeInfo(MemberTypeInfo , const FieldDecl *D);
+
 // A function to extract the appropriate relative path for a given info's
 // documentation. The path returned is a composite of the parent namespaces.
 //
@@ -293,9 +295,11 @@
 continue;
   }
 }
-I.Members.emplace_back(
+
+auto& member = I.Members.emplace_back(
 F->getTypeSourceInfo()->getType().getAsString(), F->getNameAsString(),
 getFinalAccessSpecifier(Access, F->getAccessUnsafe()));
+populateMemberTypeInfo(member, F);
   }
 }
 
@@ -431,6 +435,19 @@
   parseParameters(I, D);
 }
 
+static void populateMemberTypeInfo(MemberTypeInfo , const FieldDecl *D) {
+  ASTContext& Context = D->getASTContext();
+  RawComment *Comment = Context.getRawCommentForDeclNoCache(D);
+  if (!Comment)
+return;
+
+  Comment->setAttached();
+  if (comments::FullComment* fc = Comment->parse(Context, nullptr, D)) {
+I.Description.emplace_back();
+parseFullComment(fc, I.Description.back());
+  }
+}
+
 static void
 parseBases(RecordInfo , const CXXRecordDecl *D, bool IsFileInRootDir,
bool PublicOnly, bool IsParent,
Index: clang-tools-extra/clang-doc/Representation.h
===
--- clang-tools-extra/clang-doc/Representation.h
+++ clang-tools-extra/clang-doc/Representation.h
@@ -210,6 +210,8 @@
   // with value 0 to be used as the default.
   // (AS_public = 0, AS_protected = 1, AS_private = 2, AS_none = 3)
   AccessSpecifier Access = AccessSpecifier::AS_public;
+
+  std::vector Description; // Comment description of this field.
 };
 
 struct Location {
Index: clang-tools-extra/clang-doc/BitcodeWriter.cpp
===
--- clang-tools-extra/clang-doc/BitcodeWriter.cpp
+++ clang-tools-extra/clang-doc/BitcodeWriter.cpp
@@ -426,6 +426,8 @@
   emitBlock(T.Type, FieldId::F_type);
   emitRecord(T.Name, MEMBER_TYPE_NAME);
   emitRecord(T.Access, MEMBER_TYPE_ACCESS);
+  for (const auto  : T.Description)
+emitBlock(CI);
 }
 
 void ClangDocBitcodeWriter::emitBlock(const CommentInfo ) {
Index: clang-tools-extra/clang-doc/BitcodeReader.cpp
===
--- clang-tools-extra/clang-doc/BitcodeReader.cpp
+++ clang-tools-extra/clang-doc/BitcodeReader.cpp
@@ -350,6 +350,11 @@
   return >Description.back();
 }
 
+template <> llvm::Expected getCommentInfo(MemberTypeInfo *I) {
+  I->Description.emplace_back();
+  return >Description.back();
+}
+
 template <> llvm::Expected getCommentInfo(EnumInfo *I) {
   I->Description.emplace_back();
   return >Description.back();


Index: clang-tools-extra/clang-doc/YAMLGenerator.cpp
===
--- clang-tools-extra/clang-doc/YAMLGenerator.cpp
+++ clang-tools-extra/clang-doc/YAMLGenerator.cpp
@@ -192,6 +192,7 @@
 // the AS that shouldn't be part of the output. Even though AS_public is the
 // default in the struct, it should be displayed in the YAML output.
 IO.mapOptional("Access", I.Access, clang::AccessSpecifier::AS_none);
+IO.mapOptional("Description", I.Description);
   }
 };
 
Index: clang-tools-extra/clang-doc/Serialize.cpp
===
--- clang-tools-extra/clang-doc/Serialize.cpp
+++ 

Re: [webkit-dev] Changing the implementation of KURL

[Brett Wilson]

On Sat, Jan 28, 2012 at 12:19 AM, Adam Barth aba...@webkit.org wrote:
 On Fri, Jan 27, 2012 at 9:44 PM, Darin Fisher da...@chromium.org wrote:
 On Fri, Jan 27, 2012 at 2:39 AM, Adam Barth aba...@webkit.org wrote:
 On Fri, Jan 27, 2012 at 1:49 AM, Maciej Stachowiak m...@apple.com wrote:
  That said, this plan was based on the premise that Chromium folks were
  willing to cooperate with the unforking effort, and would be happy to
  use a
  WebKit-integrated URL library based on GoogleURL. If that is no longer
  the
  case, then certainly we should not proceed on a false premise.

 I've been talking a bit with Benjamin about this topic off-list.  I'm
 hopeful that with some careful attention to dependencies and
 interfaces, Chromium will be able to use WTFURL in place of GoogleURL.

 I still think it is a bit backwards for Chromium's network stack to depend
 on WebKit,
 but I remain open minded about this.  I'm curious how it will work out.

 The general approach I had in mind was to view WTFURL as a separate
 library that just happens to be hosted at svn.webkit.org.  That
 requires some careful managing of dependencies but it seems worth
 trying.

I don't really want to reopen this debate, but how is that different
than checking GoogleURL into webkit? Maciej was saying that this would
impose an unacceptable burden on WebKit developers. I'm wondering what
his specific complaints about burden are and whether they might also
apply to a standalone WTFURL class.

For example, if Maciej is concerned about coding style, we can easily
fix that either way and it's a non-issue. If the concern is
familiarity, I'm not sure I see the difference between Adam
reimplementing WTFURL and using some existing code since they're
both new code from the perspective of average WebKit contributors.

I'm asking because I suspect Maciej's main concern is control over the
library and dependencies and the ability to easily make changes
necessary for Apple and WebKit (I would be worried about this, too).
From my perspective, however, this might imply that he would want the
ability in the future to make modifications and add dependencies that
would be nonstarters with respect to Chromium's requirements. The
normal answer (which I agree with) when some random port wants to do
something weird and be able to compile without, for example,
JavaScript, is that WebKit doesn't support these one-off use cases,
take the whole thing and we should agree that this reasoning wouldn't
apply to dependencies on the URL component.

I think there was some agreement last year when the first part of this
project happened. If we restart it, I would want to again make sure
that everybody really understands what our dependency requirements are
and agree that we're not going to be changing them because it's better
for the WebKit community (c.f. Joe Mason's request to use all of
WTF). I also worry that this will be a 9 month project for Adam, as
the current partially done parsing thing took longer than anybody
would have liked. Personally I would prefer Adam spend time fixing my
security bugs :) So I want to be explicit on what Apple perceives as
the benefit of Adam writes a bunch of new well-tested code with no
dependencies vs. copy existing well-tested code with no dependencies
into third_party and possibly reformat so we can make sure there are
no surprises later.

That said, I am quite supportive of you unification work for the
benefit of WebKit!

Brett
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev

Re: [webkit-dev] Changing the implementation of KURL

[Brett Wilson]

On Sat, Jan 28, 2012 at 4:07 PM, Maciej Stachowiak m...@apple.com wrote:

 On Jan 28, 2012, at 9:24 AM, Brett Wilson wrote:

 On Sat, Jan 28, 2012 at 12:19 AM, Adam Barth aba...@webkit.org wrote:
 On Fri, Jan 27, 2012 at 9:44 PM, Darin Fisher da...@chromium.org wrote:
 On Fri, Jan 27, 2012 at 2:39 AM, Adam Barth aba...@webkit.org wrote:
 On Fri, Jan 27, 2012 at 1:49 AM, Maciej Stachowiak m...@apple.com wrote:
 That said, this plan was based on the premise that Chromium folks were
 willing to cooperate with the unforking effort, and would be happy to
 use a
 WebKit-integrated URL library based on GoogleURL. If that is no longer
 the
 case, then certainly we should not proceed on a false premise.

 I've been talking a bit with Benjamin about this topic off-list.  I'm
 hopeful that with some careful attention to dependencies and
 interfaces, Chromium will be able to use WTFURL in place of GoogleURL.

 I still think it is a bit backwards for Chromium's network stack to depend
 on WebKit,
 but I remain open minded about this.  I'm curious how it will work out.

 The general approach I had in mind was to view WTFURL as a separate
 library that just happens to be hosted at svn.webkit.org.  That
 requires some careful managing of dependencies but it seems worth
 trying.

 I don't really want to reopen this debate, but how is that different
 than checking GoogleURL into webkit? Maciej was saying that this would
 impose an unacceptable burden on WebKit developers. I'm wondering what
 his specific complaints about burden are and whether they might also
 apply to a standalone WTFURL class.

 For example, if Maciej is concerned about coding style, we can easily
 fix that either way and it's a non-issue. If the concern is
 familiarity, I'm not sure I see the difference between Adam
 reimplementing WTFURL and using some existing code since they're
 both new code from the perspective of average WebKit contributors.

 I don't think anyone is expecting to implement new URL code from scratch. The 
 proposals that were put forward are, as I understand them:

 (1) Check in GoogleURL as-is into the ThirdParty directory as an external 
 dependency; future maintenance would be done via the canonical GoogleURL 
 repository.
 (2) Check in GoogleURL as a proper part of WebKit, adapting its coding style 
 and wrapping it in a way that works well with other WebKit/WTF data types.

 The issue is not new code or not, but the ability to maintain it going 
 forward.

That's great, thanks.

 I'm asking because I suspect Maciej's main concern is control over the
 library and dependencies and the ability to easily make changes
 necessary for Apple and WebKit (I would be worried about this, too).

 That's exactly it.

 From my perspective, however, this might imply that he would want the
 ability in the future to make modifications and add dependencies that
 would be nonstarters with respect to Chromium's requirements. The
 normal answer (which I agree with) when some random port wants to do
 something weird and be able to compile without, for example,
 JavaScript, is that WebKit doesn't support these one-off use cases,
 take the whole thing and we should agree that this reasoning wouldn't
 apply to dependencies on the URL component.

 I think there was some agreement last year when the first part of this
 project happened. If we restart it, I would want to again make sure
 that everybody really understands what our dependency requirements are
 and agree that we're not going to be changing them because it's better
 for the WebKit community (c.f. Joe Mason's request to use all of
 WTF). I also worry that this will be a 9 month project for Adam, as
 the current partially done parsing thing took longer than anybody
 would have liked. Personally I would prefer Adam spend time fixing my
 security bugs :) So I want to be explicit on what Apple perceives as
 the benefit of Adam writes a bunch of new well-tested code with no
 dependencies vs. copy existing well-tested code with no dependencies
 into third_party and possibly reformat so we can make sure there are
 no surprises later.

 I think that, at this point, Benjamin is volunteering to do the bulk of work 
 and Adam is offering to help. The code would be more along the lines of 
 reformatting and refactoring than write[ing] a bunch of new code as I 
 understand it.

 I understand your concern about wanting to keept his code free of unwanted 
 dependencies. However, we do already have a pretty hard boundary that WTF 
 can't depend on anything higher in the stack. We could make WTFURL even more 
 restricted if that was necessary or helpful. However, I think most of us who 
 do not work on Chromium are not clear on what exactly the dependency 
 constraints are. It would be helpful if you or Adam or someone else could 
 document them.

 Let's take some specific examples. Would using WTF::Vector inside the 
 implementation (not necessarily at the API boundary, just internally) be 
 acceptable

Re: [webkit-dev] Changing the implementation of KURL

[Brett Wilson]

On Sat, Jan 28, 2012 at 4:48 PM, Brett Wilson bre...@chromium.org wrote:
 On Sat, Jan 28, 2012 at 4:07 PM, Maciej Stachowiak m...@apple.com wrote:

 On Jan 28, 2012, at 9:24 AM, Brett Wilson wrote:

 On Sat, Jan 28, 2012 at 12:19 AM, Adam Barth aba...@webkit.org wrote:
 On Fri, Jan 27, 2012 at 9:44 PM, Darin Fisher da...@chromium.org wrote:
 On Fri, Jan 27, 2012 at 2:39 AM, Adam Barth aba...@webkit.org wrote:
 On Fri, Jan 27, 2012 at 1:49 AM, Maciej Stachowiak m...@apple.com 
 wrote:
 That said, this plan was based on the premise that Chromium folks were
 willing to cooperate with the unforking effort, and would be happy to
 use a
 WebKit-integrated URL library based on GoogleURL. If that is no longer
 the
 case, then certainly we should not proceed on a false premise.

 I've been talking a bit with Benjamin about this topic off-list.  I'm
 hopeful that with some careful attention to dependencies and
 interfaces, Chromium will be able to use WTFURL in place of GoogleURL.

 I still think it is a bit backwards for Chromium's network stack to depend
 on WebKit,
 but I remain open minded about this.  I'm curious how it will work out.

 The general approach I had in mind was to view WTFURL as a separate
 library that just happens to be hosted at svn.webkit.org.  That
 requires some careful managing of dependencies but it seems worth
 trying.

 I don't really want to reopen this debate, but how is that different
 than checking GoogleURL into webkit? Maciej was saying that this would
 impose an unacceptable burden on WebKit developers. I'm wondering what
 his specific complaints about burden are and whether they might also
 apply to a standalone WTFURL class.

 For example, if Maciej is concerned about coding style, we can easily
 fix that either way and it's a non-issue. If the concern is
 familiarity, I'm not sure I see the difference between Adam
 reimplementing WTFURL and using some existing code since they're
 both new code from the perspective of average WebKit contributors.

 I don't think anyone is expecting to implement new URL code from scratch. 
 The proposals that were put forward are, as I understand them:

 (1) Check in GoogleURL as-is into the ThirdParty directory as an external 
 dependency; future maintenance would be done via the canonical GoogleURL 
 repository.
 (2) Check in GoogleURL as a proper part of WebKit, adapting its coding style 
 and wrapping it in a way that works well with other WebKit/WTF data types.

 The issue is not new code or not, but the ability to maintain it going 
 forward.

 That's great, thanks.

 I'm asking because I suspect Maciej's main concern is control over the
 library and dependencies and the ability to easily make changes
 necessary for Apple and WebKit (I would be worried about this, too).

 That's exactly it.

 From my perspective, however, this might imply that he would want the
 ability in the future to make modifications and add dependencies that
 would be nonstarters with respect to Chromium's requirements. The
 normal answer (which I agree with) when some random port wants to do
 something weird and be able to compile without, for example,
 JavaScript, is that WebKit doesn't support these one-off use cases,
 take the whole thing and we should agree that this reasoning wouldn't
 apply to dependencies on the URL component.

 I think there was some agreement last year when the first part of this
 project happened. If we restart it, I would want to again make sure
 that everybody really understands what our dependency requirements are
 and agree that we're not going to be changing them because it's better
 for the WebKit community (c.f. Joe Mason's request to use all of
 WTF). I also worry that this will be a 9 month project for Adam, as
 the current partially done parsing thing took longer than anybody
 would have liked. Personally I would prefer Adam spend time fixing my
 security bugs :) So I want to be explicit on what Apple perceives as
 the benefit of Adam writes a bunch of new well-tested code with no
 dependencies vs. copy existing well-tested code with no dependencies
 into third_party and possibly reformat so we can make sure there are
 no surprises later.

 I think that, at this point, Benjamin is volunteering to do the bulk of work 
 and Adam is offering to help. The code would be more along the lines of 
 reformatting and refactoring than write[ing] a bunch of new code as I 
 understand it.

 I understand your concern about wanting to keept his code free of unwanted 
 dependencies. However, we do already have a pretty hard boundary that WTF 
 can't depend on anything higher in the stack. We could make WTFURL even more 
 restricted if that was necessary or helpful. However, I think most of us who 
 do not work on Chromium are not clear on what exactly the dependency 
 constraints are. It would be helpful if you or Adam or someone else could 
 document them.

 Let's take some specific examples. Would using WTF::Vector inside the 
 implementation

Re: [webkit-dev] Changing the implementation of KURL

[Brett Wilson]

On Sat, Jan 28, 2012 at 5:28 PM, Joe Mason jma...@rim.com wrote:
 -Original Message-
 From: webkit-dev-boun...@lists.webkit.org [mailto:webkit-dev-
 boun...@lists.webkit.org] On Behalf Of Maciej Stachowiak
 Sent: Saturday, January 28, 2012 7:08 PM
 To: Brett Wilson
 Cc: Benjamin Poulain; WebKit Development
 Subject: Re: [webkit-dev] Changing the implementation of KURL

 Let's take some specific examples. Would using WTF::Vector inside the
 implementation (not necessarily at the API boundary, just internally)
 be acceptable? Or would it be required to use C arrays or std::vector?
 Would using WTF's ASSERT family of macros be acceptable, or should some
 other form of asserts be used? The are examples I can think of where
 dependencies could simply be added in the course of trying to get the
 code to be in WebKit style.

 Another big potential dependency would be use of Platform.h and the 
 ENABLE/USE/etc system of macros - I could easily see a new feature including 
 special processing for a new URL scheme, similar to the way file: url's have 
 slightly different parsing rules than http: urls.  In this case we'd want to 
 wrap the special handling in ENABLE(feature).  (Arguably the url class should 
 be flexible enough that you don't have to hardcode special handling for a 
 scheme, though.)

I don't see that as being much of an issue. Google Safe Browsing can
always write their own Platform.h that does what they need.

Brett
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev

Re: [webkit-dev] no URL validation checking in KURLGoogle?

[Brett Wilson]

On Fri, Nov 11, 2011 at 12:29 AM, Huang, Deqing deqing.hu...@intel.com wrote:
 Hi,

 It seems that KURLGoogle is not validating url, it simply marks url as
 valid.

 I tested an invalid url http://a a/ in Chrome, it tried to open it and
 failed.
 In FireFox for this kind of url it will pop out a dialog saying The URL is
 not valid and cannot be loaded.

 Should url be validated in KURLGoogle too? Or any concerns not adding it?

This mailing list is not the correct place for this. If you think you
have a bug, you should file one.

I question why you think the problem is KURLGoogle and why you think
it is not validating the URL. If you step through the code for this
URL, you will find that this is not the case.

Brett
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev

Re: [webkit-dev] Adding ENABLE_BATTERY_STATUS to WebCore

[Brett Wilson]

On Wed, Jun 15, 2011 at 9:30 AM, Holger Freyther ze...@selfish.org wrote:
 On 06/15/2011 06:11 PM, laszlo.1.gom...@nokia.com wrote:
 Hi,



 The use-case for us is to enable content developers to implement rudimentary 
 power management (e.g. to stop expensive operations on the page, perhaps 
 save state). I'm not sure if this API is really meant for accurately 
 reporting all the possible power management states of the system as Anssi 
 pointed out.

 Okay, point on complexity taken. My question is what if you want to add
 complexity, is there something in the event that prevents that (I have no idea
 about DOM compatibility issues)? Don't get me wrong I think having more device
 support is great.

 My other complain was, it is too simple. E.g. 'isPlugged' has no guarantee
 that the battery is getting charged. Is this a problem?

Why would a web page care about whether the battery is being charged
when the device is plugged in?

Brett
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev

Re: [webkit-dev] Correct relative URL behavior

[Brett Wilson]

On Wed, May 4, 2011 at 2:49 AM, Maciej Stachowiak m...@apple.com wrote:

 On May 4, 2011, at 2:33 AM, Xianzhu Wang wrote:

 Hi,
 KURL's relative URL behavior is different between Chromium and non-Chromium
 ports, because Chromium ports use KURLGoogle.cpp instead of KURL.cpp.
 In KURL(base, relative), when base is a not hierarchical, WebKit's
 KURL::string() returns relative, while Chromium's returns an empty string.
 The behavior of Chromium
 causes https://bugs.webkit.org/show_bug.cgi?id=55643 (Chromium unnecessarily
 creates SVGImage when an SVG document contains images).
 I'm wondering which behavior is correct. Assuming WebKit's KURL behavior is
 correct, my first patch is to change KURLGoogle.cpp to match KURL.cpp, which
 affects the result
 of fast/url/relative.html. platform/chromium/fast/url/relative-expected.txt
 contains all PASSs, while fast/url/relative-expected.txt contains 8 expected
 FAILs, 2 of which relate to my question:

 I don't know if KURL's behavior is correct, but I believe Chromium's
 behavior of resolving to an empty string URL in this case is not matched by
 any other browser. So I am pretty sure KURLGoogle is incorrect.

I agree that the Google behavior should be changed when resolving
against a non-hierarchical base, although I think there are some cases
where I think KURL is also incorrect here (I don't remember
specifics). Adam is correct that there's a fair amount of ambiguity,
so it's difficult to declare either one correct for some cases.

 FAIL canonicalize('baz.html') should be . Was baz.html.
 FAIL canonicalize(':foo') should be . Was :foo.
 My questions are:
 1. Are all the PASS expectations of fast/url/relative.html correct? If yes,
 we should file a bug to track the failures of KURL.
 2. With my patch, Chromium will produce the above two FAILs for
 fast/url/relative.html. Is this acceptable to be rebaselined?

 Adam made the test expectations all have PASS matching google-url behavior,
 without analysis of what is actually correct. I think it may be misleading
 for the tests to say PASS/FAIL at all at this point, but I would say the
 test should be rebaselined.

Note that the fix would be in the google-url project rather than in
the KURLGoogle.cpp. The change will likely be several weeks of work.
This hasn't been fixed yet because we've not seen it affect real
sites.

Brett
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev

[rrd-users] Once a day updates incorrect by timezone

[Brett Wilson]

Hi All,

Im monitoring my solar power installation and logging it into rrdtool then
graphing the output.
*http://powermonitoring.zzl.org/*

Its all working well apart from the daily updates which run at midnight each
night (local time) and pull the KwH produced today and the Time online today
from the inverter. The bottom graph from the webpage is the one im talking
about.

My rrd database is setup as follows:-
--start N
-s 86400
DS:KWH_Today:GAUGE:86500:U:U
DS:INV_Online:GAUGE:86500:U:U
RRA:AVERAGE:0.5:1:365

Then im updating as follows at 23:59 local time (GMT +10):-
rrdtool update Xantrex_Daily_Totals.rrd N:somenumber:somenumber

From what I have found In the forums I need to replace the N with something
else for the timestamp to ensure that its put in for the correct day however
am unsure what to put in.

Any help would be appreciated.

Cheers
Brett
___
rrd-users mailing list
rrd-users@lists.oetiker.ch
https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users

[v8-dev] Re: [Chrome-team] Opera 10.60 alpha 1 released

[Brett Wilson]

On Mon, May 31, 2010 at 10:23 AM, Peter Kasting pkast...@google.com wrote:
 Announcement: http://my.opera.com/desktopteam/blog/2010/05/31/opera-10-60-alpha-1
 According to various blogs, a few of the major changes are that this build
 scores dramatically higher (50-100%) than Opera 10.50 on Peacekeeper, and
 that per-tab Aero Peek has been disabled by default, for the same reason we
 disabled it: it's horrible for anyone with very many tabs.

The thumbnail service I wrote for ChromeOS should be perfect for this
feature. The main limitation is that it won't show thumbnails for tabs
that have never been rendered, which really bothers some people.

Brett

-- 
v8-dev mailing list
v8-dev@googlegroups.com
http://groups.google.com/group/v8-dev

Re: [chromium-dev] Interesting performance characteristics of http://a.qoid.us/google.html

[Brett Wilson]

On Mon, Jan 11, 2010 at 9:04 PM, Chris Evans cev...@chromium.org wrote:
 Hi,
 I found this interesting: http://a.qoid.us/google.html
 Because it's clearly very non-JavaScript focused (just a CSS rotation) and
 yet the browsers exhibit very different performance characteristics.
 All on my underpowered Windows laptop,
 Chrome 3  4: smooth. Very nice anti-aliased accurate rendering.
 Safari 4: not quite as smooth; less fps but still not bad. Text / lines more
 jagged and text wobbles during the rotation.
 The other browsers do not support the CSS used / required for this demo.
 There was a similar version, but using SVG animation to do something very
 similar. (It's currently offline). Firefox and Opera were able to attempt
 the rotation of this one but both performed much worse that Safari and
 Chrome.
 Anyone know why Chrome, and to a certain extent Safari, are so much better
 on this non-JS test? Is the Skia library underpinning Chrome superior?
 There's also a difference between Chrome Windows and Chrome Linux. Whilst
 Chrome Linux performs smoothly, the text is clearly jiggling around and even
 getting more / less bold throughout the rotation cycle. Doesn't do that on
 Windows. Any ideas?

I imagine the problems on Linux are the same as they were on Windows,
where the underlying graphics layer wants to place glyphs aligned on
pixel boundaries. This gives rounding problems and the letters jump
around. I spent several weeks making the rotations and transparency on
Windows text and form controls look good.

Brett
-- 
Chromium Developers mailing list: chromium-dev@googlegroups.com 
View archives, change email options, or unsubscribe: 
http://groups.google.com/group/chromium-dev

Re: [chromium-dev] Interesting performance characteristics of http://a.qoid.us/google.html

[Brett Wilson]

On Tue, Jan 12, 2010 at 11:33 AM, Evan Martin e...@chromium.org wrote:
 On Tue, Jan 12, 2010 at 10:17 AM, Brett Wilson bre...@chromium.org wrote:
 I imagine the problems on Linux are the same as they were on Windows,
 where the underlying graphics layer wants to place glyphs aligned on
 pixel boundaries. This gives rounding problems and the letters jump
 around. I spent several weeks making the rotations and transparency on
 Windows text and form controls look good.

 Can you elaborate a bit on what you did?  (I write this as maybe the
 person who will need to fight with this at some point, but practically
 I wonder if it'll eventually fall on you.  :P)

This is the code in SkiaFontWin.cpp. That's non-ideal, and it would be
better to have Skia get the outlines itself and use the Skia path. I
don't actually know where the rounding problems occur on Linux. Skia
should be fine with it, but its probably in the way it interacts with
FreeType (maybe a glyph cache that requires being pixel-aligned?)

Brett
-- 
Chromium Developers mailing list: chromium-dev@googlegroups.com 
View archives, change email options, or unsubscribe: 
http://groups.google.com/group/chromium-dev

Re: [chromium-dev] How can I tell if a Renderer process finishes loading a page

[Brett Wilson]

On Mon, Jan 11, 2010 at 12:59 AM, hap 497 hap...@gmail.com wrote:
 Thank you.

 But when I load www.cnn.com, I see something strange:
 First the throbber is spinning, and then the cnn fave icon is shown
 and then the throbber is spinning again, and then fave icon is shown.
 And then the fave icon stays.

 Can you please help me understand under what situation will cause this
 to happen? I don't think it is caused by frame set. This is because I
 tried http://java.sun.com/j2se/1.5.0/docs/api/index.html, the throbber
 stops after each frame in frame set is loaded.

I don't think anybody is able to answer this type of question off the
top of their head.

RenderView::didStartLoading / didStopLoading correspond to the
throbber starting and stopping in the renderer, I'd trace back form
there.

Brett
-- 
Chromium Developers mailing list: chromium-dev@googlegroups.com 
View archives, change email options, or unsubscribe: 
http://groups.google.com/group/chromium-dev

Re: [chromium-dev] Splitting off some pieces of chrome.gyp...

[Brett Wilson]

On Mon, Dec 7, 2009 at 1:43 PM, Bradley Nelson bradnel...@google.com wrote:
 Hello All,
 Last week I re-landed a change to split off parts of chrome.gyp into .gypi's
 in the same directory.
 I had done something similar a couple weeks back, but took it out because
 concern was raised about merge conflicts in m4.
 I put it back in because I got the all clear on m4.
 The goal is to reduce contention on chrome.gyp which has gotten quite large
 - 7000+ lines.

I just tried to add a file to chrome.gyp and I couldn't find where it
should go. It took me a minute to remember this thread.

I'm concerned our build system is getting out of control. I can not
explain in simple sentences where anything should go. Even if I'm
using my visual tool and find the right project, the only real way to
find where a file should go is to grep for files nearby. .gyp files
already had this property where you can't find where things should go
in them and you needed to search. Now it seems like we've added an
additional layer of you don't know which file its in, either.

How much does this solve contention? It seems like actual conflicts
for changes will be the same as before. What contention cases are
there that splitting everything out helps a lot?

Thanks
Brett

PS Thanks for working on the build system, I don't mean to be non-appreciative.

-- 
Chromium Developers mailing list: chromium-dev@googlegroups.com 
View archives, change email options, or unsubscribe: 
http://groups.google.com/group/chromium-dev

Re: [chromium-dev] Thoughts on // NOLINT?

[Brett Wilson]

On Wed, Dec 9, 2009 at 4:24 PM, John Abd-El-Malek j...@chromium.org wrote:
 btw I searched the code, almost all the instances are in code from different
 repositories, like v8, gtest, gmock.  I counted only 17 instances in
 Chrome's code.


Most of the Chrome NOLINTs look like the're around ifdefs, where the
ifdef code sometimes mean that a comma or a semicolon goes on the line
after the ifdef. We should be working to remove these anyway since the
ifdefs are super ugly, and I'm not sure the NOLINT comment actually
makes them worse. Some of these may not be practical or desirable to
remove, though.

So I don't think I see a big problem with the way NOLINT is used in
Chrome. Looking through V8 I don't see a huge problem either.

Some NOLINT calls weren't clear to me why the linter complained. I
suggest that NOLINT comments be documented. In some places they
already are. Then reviewers will know to argue when they see something
untoward, whereas just // NOLINT isn't alwasy clear about what the
problem is and whether they should complain. This will also make
NOLINTs more painful to add since you have to justify why you're
adding it, which will hopefully decrease its usage.

Brett

-- 
Chromium Developers mailing list: chromium-dev@googlegroups.com 
View archives, change email options, or unsubscribe: 
http://groups.google.com/group/chromium-dev

[chromium-dev] Re: Changes to using threads in the browser process

[Brett Wilson]

On Tue, Nov 3, 2009 at 6:01 AM, Mike Pinkerton pinker...@chromium.org wrote:

 Can you put this information, as well as some examples for proper
 usage, on dev.chromium.org. I doubt this stuff is very well documented
 from the perspective of someone trying to use it, and this looks like
 a great start!!!

The place to put it is here:
http://dev.chromium.org/developers/design-documents/threading

Brett

--~--~-~--~~~---~--~~
Chromium Developers mailing list: chromium-dev@googlegroups.com 
View archives, change email options, or unsubscribe: 
http://groups.google.com/group/chromium-dev
-~--~~~~--~~--~--~---

[chromium-dev] Re: Tab Thumbnails and Aero Peek (of Windows 7)

[Brett Wilson]

2009/10/23 Hironori Bono (坊野 博典) hb...@google.com:
 Hi Brett,

 Thank you so much for noticing this. I'm integrating the
 ThumbnailGenertor class into my prototype now. :)
 By the way, I'm a little wondering if there is a function that changes
 the width and the height of a thumbnail image generated by the
 ThumbnailGenerator class at run time. If not, I would like to hear
 there is a plan to add it.
 When Windows 7 shows tab thumbnails, it automatically calculates the
 width and the height of a thumbnail image so that it can show all
 thumbnails in one line. (When we open too many tabs and Windows 7
 cannot show their thumbnails, it shows a list of tab titles.) In
 brief, we cannot assume the width or the height of a thumbnail as
 constant values on Windows 7.
 So, it may be better for the ThumbnailGenerator class to have an
 interface that changes the width and the height of its thumbnails. (I
 has added code that resizes the thumbnails retrieved from the
 ThumbnailGenerator object and sends the resized thumbnails to Windows
 7. So, I don't mind it at all.)

There isn't a way to do this. The reason is that the thumbnail
generator precalculates thumbnails in advance for many pages (for
example, when we're discarding the backing store). This means we have
to know basically when we start Chrome what the max thumbnail size
will be. It uses power-of-two downsampling to be as fast as possible,
so the actual thumbnails aren't guaranteed to be any particular size.

I would just use the default size for now. If it doesn't look OK, we
can add a future enhancement to request the size, so the ones it
generates on-demand (the ones where there is a backing store) can have
a larger size if needed.

Brett

--~--~-~--~~~---~--~~
Chromium Developers mailing list: chromium-dev@googlegroups.com 
View archives, change email options, or unsubscribe: 
http://groups.google.com/group/chromium-dev
-~--~~~~--~~--~--~---

[chromium-dev] Re: Tab Thumbnails and Aero Peek (of Windows 7)

[Brett Wilson]

Hi Hironori,

Thanks for researching this topic. On generating thumbnails, I wrote
chrome/browser/tab_contents/thumbnail_generator which tries to get the
most up-to-date thumbnail images for all tabs (except ones that
haven't ever been shown) with minimal performance overhead. The code
hasn't been used much in practice yet, but seems to basically work.

It was written for a tab switcher, and the intent is that it will
provide thumbnails for all services in the future (including the new
tab page and the Windows 7 features) for all platforms. So I would
definitely use that, and we should work on fixing any bugs or
limitations that pop up.

It's defaulted to off for most builds because it has a runtime
overhead and its not currently used. If you want to turn it on for
your build, it's started in the constructor for TabContents.

Brett

--~--~-~--~~~---~--~~
Chromium Developers mailing list: chromium-dev@googlegroups.com 
View archives, change email options, or unsubscribe: 
http://groups.google.com/group/chromium-dev
-~--~~~~--~~--~--~---

[chromium-dev] Re: Spellchecker and memory-mapped dicts

[Brett Wilson]

On Thu, Oct 22, 2009 at 1:39 PM, Evan Stade est...@chromium.org wrote:

 Hi all,

 At its last meeting the jank task force discussed improving
 responsiveness of the spellchecker but we didn't come to a solid
 conclusion so I thought I'd bring it up here to see if anyone else has
 opinions. The main concern is that we don't block the IO thread on
 file access. To this end, I recently moved initialization of the
 spellchecker from the IO thread to the file thread. However, instead
 of reading in the spellchecker dictionary in one solid chunk, we
 memory-map it. Then later we check individual words on the IO thread,
 which will be slow since the dictionary starts off effectively
 completely paged out. The proposal is that we read in the dictionary
 at spellchecker intialization instead of memory mapping it.

 Memory mapping pros:
 - possibly uses less overall memory, depending on the structure of the
 dictionary and the usage pattern of the user.
 - strikeloading the dictionary doesn't block for a long
 time/strike this one no longer occurs either way due to my recent
 refactoring

 Reading it all at once pros:
 - costly disk accesses are kept to the file thread (excepting future
 memory paging)
 - overall disk access time is probably lower (since we can read in the
 dict in one chunk)

 For reference, the English dictionary is about 500K, and most
 dictionaries are under 2 megs, some (such as Hungarian) are much
 higher, but no dictionary is over 10 megs.

 Opinions?

I've thought about this some (I wrote the memory map thing there now).

History of the spellchecker:
v1 : Per-process Hunspell storage (lots of memory duplicated in each
renderer, expensive to load).
v2 : Browser-process Hunspell storage (lots of memory, expensive to
load, only occurs once)
v3 : Browser-process memmap (less memory, cheap to load, only occurs once).

I would like to consider moving hunspell back to the renderer so we
can avoid sync IPCs and blocking the I/O thread on spellchecking.
Spellchecking isn't fast (especially suggestions) even when everything
is in memory, so it always sucks to have it block the I/O thread. Now
that it can be memmapped, each renderer can memmap its own image of
the data.

This doesn't help on Mac where we want to use the system spellchecker.
There would also be some amount of duplication since there are certain
tables that are initialized once at the beginning (I don't think its
that big, though).

I would suggest first making the current histograms in the
spellchecker.cc file UMA (currently they're debug-only local ones) so
we can see how much blocking we're getting from Hunspell in the field.

Brett

--~--~-~--~~~---~--~~
Chromium Developers mailing list: chromium-dev@googlegroups.com 
View archives, change email options, or unsubscribe: 
http://groups.google.com/group/chromium-dev
-~--~~~~--~~--~--~---