[clang] [llvm] Update documentation and release notes for llvm-profgen COFF support (PR #84864)

[via cfe-commits]

github-actions[bot] wrote:



@tcreech-intel Congratulations on having your first Pull Request (PR) merged 
into the LLVM Project!

Your changes will be combined with recent changes from other authors, then 
tested
by our [build bots](https://lab.llvm.org/buildbot/). If there is a problem with 
a build, you may recieve a report in an email or a comment on this PR.

Please check whether problems have been caused by your change specifically, as
the builds can include changes from many authors. It is not uncommon for your
change to be included in a build that fails due to someone else's changes, or
infrastructure issues.

How to do this, and the rest of the post-merge process, is covered in detail 
[here](https://llvm.org/docs/MyFirstTypoFix.html#myfirsttypofix-issues-after-landing-your-pr).

If your change does cause a problem, it may be reverted, or you can revert it 
yourself.
This is a normal part of [LLVM 
development](https://llvm.org/docs/DeveloperPolicy.html#patch-reversion-policy).
 You can fix your changes and open a new PR to merge them again.

If you don't get any reports, no action is required from you. Your changes are 
working as expected, well done!


https://github.com/llvm/llvm-project/pull/84864
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [llvm] Update documentation and release notes for llvm-profgen COFF support (PR #84864)

[Wei Xiao via cfe-commits]

https://github.com/williamweixiao closed 
https://github.com/llvm/llvm-project/pull/84864
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [llvm] Update documentation and release notes for llvm-profgen COFF support (PR #84864)

[Tim Creech via cfe-commits]

tcreech-intel wrote:

> please resolve the conflicts.

Thanks for the review, @williamweixiao. I've resolved the conflicts in 2312821.

https://github.com/llvm/llvm-project/pull/84864
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [llvm] Update documentation and release notes for llvm-profgen COFF support (PR #84864)

[Tim Creech via cfe-commits]

https://github.com/tcreech-intel updated 
https://github.com/llvm/llvm-project/pull/84864

>From 4dc108d0d290ee5fd6a73c029c051fdb2215d00a Mon Sep 17 00:00:00 2001
From: Tim Creech 
Date: Mon, 11 Mar 2024 22:35:59 -0400
Subject: [PATCH 1/5] Update documentation and release notes for llvm-profgen
 COFF support

This change:
- Updates the existing Clang User's Manual section on SPGO so that it
  describes how to use llvm-profgen to perform SPGO on Windows. This is
  new functionality implemented in #83972.
- Fixes a minor typo in the existing llvm-profgen invocation example.
- Adds an LLVM release note on this new functionality in llvm-profgen.
---
 clang/docs/UsersManual.rst | 47 +++---
 llvm/docs/ReleaseNotes.rst |  5 
 2 files changed, 44 insertions(+), 8 deletions(-)

diff --git a/clang/docs/UsersManual.rst b/clang/docs/UsersManual.rst
index 7391e4cf3a9aeb..9cf313c3727125 100644
--- a/clang/docs/UsersManual.rst
+++ b/clang/docs/UsersManual.rst
@@ -2410,20 +2410,35 @@ usual build cycle when using sample profilers for 
optimization:
 
 1. Build the code with source line table information. You can use all the
usual build flags that you always build your application with. The only
-   requirement is that you add ``-gline-tables-only`` or ``-g`` to the
-   command line. This is important for the profiler to be able to map
-   instructions back to source line locations.
+   requirement is that DWARF debug info including source line information is
+   generated. This DWARF information is important for the profiler to be able
+   to map instructions back to source line locations.
+
+   On Linux, ``-g`` or just ``-gline-tables-only`` is sufficient:
 
.. code-block:: console
 
  $ clang++ -O2 -gline-tables-only code.cc -o code
 
+   It is also possible to include DWARF in Windows binaries:
+
+   .. code-block:: console
+
+ $ clang-cl -O2 -gdwarf -gline-tables-only coff-profile.cpp -fuse-ld=lld 
-link -debug:dwarf
+
 2. Run the executable under a sampling profiler. The specific profiler
you use does not really matter, as long as its output can be converted
-   into the format that the LLVM optimizer understands. Currently, there
-   exists a conversion tool for the Linux Perf profiler
-   (https://perf.wiki.kernel.org/), so these examples assume that you
-   are using Linux Perf to profile your code.
+   into the format that the LLVM optimizer understands.
+
+   Two such profilers are the the Linux Perf profiler
+   (https://perf.wiki.kernel.org/) and Intel's Sampling Enabling Product (SEP),
+   available as part of `Intel VTune
+   
`_.
+
+   The LLVM tool ``llvm-profgen`` can convert output of either Perf or SEP. An
+   external tool, AutoFDO, also supports Linux Perf output.
+
+   When using Perf:
 
.. code-block:: console
 
@@ -2434,6 +2449,15 @@ usual build cycle when using sample profilers for 
optimization:
it provides better call information, which improves the accuracy of
the profile data.
 
+   When using SEP:
+
+   .. code-block:: console
+
+ $ sep -start -ec BR_INST_RETIRED.NEAR_TAKEN:precise=yes:pdir -lbr 
no_filter:usr -perf-script ip,brstack -app ./code
+
+   This produces a ``perf.data.script`` output which can be used with
+   ``llvm-profgen``'s ``--perfscript`` input option.
+
 3. Convert the collected profile data to LLVM's sample profile format.
This is currently supported via the AutoFDO converter ``create_llvm_prof``.
It is available at https://github.com/google/autofdo. Once built and
@@ -2454,7 +2478,14 @@ usual build cycle when using sample profilers for 
optimization:
 
.. code-block:: console
 
- $ llvm-profgen --binary=./code --output=code.prof--perfdata=perf.data
+ $ llvm-profgen --binary=./code --output=code.prof --perfdata=perf.data
+
+   When using SEP the output is in the textual format corresponding to
+   `llvm-profgen --perfscript`. For example:
+
+   .. code-block:: console
+
+ $ llvm-profgen --binary=./code --output=code.prof 
--perfscript=perf.data.script
 
 
 4. Build the code again using the collected profile. This step feeds
diff --git a/llvm/docs/ReleaseNotes.rst b/llvm/docs/ReleaseNotes.rst
index b34a5f31c5eb0a..c2bbc647bc18e6 100644
--- a/llvm/docs/ReleaseNotes.rst
+++ b/llvm/docs/ReleaseNotes.rst
@@ -157,6 +157,11 @@ Changes to the LLVM tools
   ``--set-symbols-visibility`` options for ELF input to change the
   visibility of symbols.
 
+* llvm-profgen now supports COFF+DWARF binaries. This enables Sample-based PGO
+  on Windows using Intel VTune's SEP. For details on usage, see the `end-user
+  documentation for SPGO
+  `_.
+
 Changes to LLDB
 -
 

>From 53f4c5dc84d71fd4efa5384818ecfc3401a0e7f6 Mon Sep 17 00:00:00 2001
From: Tim Creech 
Date: Tue, 12 Mar 2024 09:14:27 -0400

[clang] [llvm] Update documentation and release notes for llvm-profgen COFF support (PR #84864)

[Wei Xiao via cfe-commits]

williamweixiao wrote:

please resolve the conflicts.

https://github.com/llvm/llvm-project/pull/84864
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [llvm] Update documentation and release notes for llvm-profgen COFF support (PR #84864)

[Wei Xiao via cfe-commits]

https://github.com/williamweixiao approved this pull request.


https://github.com/llvm/llvm-project/pull/84864
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [llvm] Update documentation and release notes for llvm-profgen COFF support (PR #84864)

[Haohai Wen via cfe-commits]

https://github.com/HaohaiWen approved this pull request.

LGTM

https://github.com/llvm/llvm-project/pull/84864
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [llvm] Update documentation and release notes for llvm-profgen COFF support (PR #84864)

[Tim Creech via cfe-commits]

@@ -2454,7 +2481,14 @@ usual build cycle when using sample profilers for 
optimization:
 
.. code-block:: console
 
- $ llvm-profgen --binary=./code --output=code.prof--perfdata=perf.data
+ $ llvm-profgen --binary=./code --output=code.prof --perfdata=perf.data
+
+   When using SEP the output is in the textual format corresponding to
+   ``llvm-profgen --perfscript``. For example:
+
+   .. code-block:: console
+
+ $ llvm-profgen --binary=./code --output=code.prof 
--perfscript=perf.data.script

tcreech-intel wrote:

Thanks. Fixed.

https://github.com/llvm/llvm-project/pull/84864
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [llvm] Update documentation and release notes for llvm-profgen COFF support (PR #84864)

[Tim Creech via cfe-commits]

https://github.com/tcreech-intel updated 
https://github.com/llvm/llvm-project/pull/84864

>From 4dc108d0d290ee5fd6a73c029c051fdb2215d00a Mon Sep 17 00:00:00 2001
From: Tim Creech 
Date: Mon, 11 Mar 2024 22:35:59 -0400
Subject: [PATCH 1/5] Update documentation and release notes for llvm-profgen
 COFF support

This change:
- Updates the existing Clang User's Manual section on SPGO so that it
  describes how to use llvm-profgen to perform SPGO on Windows. This is
  new functionality implemented in #83972.
- Fixes a minor typo in the existing llvm-profgen invocation example.
- Adds an LLVM release note on this new functionality in llvm-profgen.
---
 clang/docs/UsersManual.rst | 47 +++---
 llvm/docs/ReleaseNotes.rst |  5 
 2 files changed, 44 insertions(+), 8 deletions(-)

diff --git a/clang/docs/UsersManual.rst b/clang/docs/UsersManual.rst
index 7391e4cf3a9aeb..9cf313c3727125 100644
--- a/clang/docs/UsersManual.rst
+++ b/clang/docs/UsersManual.rst
@@ -2410,20 +2410,35 @@ usual build cycle when using sample profilers for 
optimization:
 
 1. Build the code with source line table information. You can use all the
usual build flags that you always build your application with. The only
-   requirement is that you add ``-gline-tables-only`` or ``-g`` to the
-   command line. This is important for the profiler to be able to map
-   instructions back to source line locations.
+   requirement is that DWARF debug info including source line information is
+   generated. This DWARF information is important for the profiler to be able
+   to map instructions back to source line locations.
+
+   On Linux, ``-g`` or just ``-gline-tables-only`` is sufficient:
 
.. code-block:: console
 
  $ clang++ -O2 -gline-tables-only code.cc -o code
 
+   It is also possible to include DWARF in Windows binaries:
+
+   .. code-block:: console
+
+ $ clang-cl -O2 -gdwarf -gline-tables-only coff-profile.cpp -fuse-ld=lld 
-link -debug:dwarf
+
 2. Run the executable under a sampling profiler. The specific profiler
you use does not really matter, as long as its output can be converted
-   into the format that the LLVM optimizer understands. Currently, there
-   exists a conversion tool for the Linux Perf profiler
-   (https://perf.wiki.kernel.org/), so these examples assume that you
-   are using Linux Perf to profile your code.
+   into the format that the LLVM optimizer understands.
+
+   Two such profilers are the the Linux Perf profiler
+   (https://perf.wiki.kernel.org/) and Intel's Sampling Enabling Product (SEP),
+   available as part of `Intel VTune
+   
`_.
+
+   The LLVM tool ``llvm-profgen`` can convert output of either Perf or SEP. An
+   external tool, AutoFDO, also supports Linux Perf output.
+
+   When using Perf:
 
.. code-block:: console
 
@@ -2434,6 +2449,15 @@ usual build cycle when using sample profilers for 
optimization:
it provides better call information, which improves the accuracy of
the profile data.
 
+   When using SEP:
+
+   .. code-block:: console
+
+ $ sep -start -ec BR_INST_RETIRED.NEAR_TAKEN:precise=yes:pdir -lbr 
no_filter:usr -perf-script ip,brstack -app ./code
+
+   This produces a ``perf.data.script`` output which can be used with
+   ``llvm-profgen``'s ``--perfscript`` input option.
+
 3. Convert the collected profile data to LLVM's sample profile format.
This is currently supported via the AutoFDO converter ``create_llvm_prof``.
It is available at https://github.com/google/autofdo. Once built and
@@ -2454,7 +2478,14 @@ usual build cycle when using sample profilers for 
optimization:
 
.. code-block:: console
 
- $ llvm-profgen --binary=./code --output=code.prof--perfdata=perf.data
+ $ llvm-profgen --binary=./code --output=code.prof --perfdata=perf.data
+
+   When using SEP the output is in the textual format corresponding to
+   `llvm-profgen --perfscript`. For example:
+
+   .. code-block:: console
+
+ $ llvm-profgen --binary=./code --output=code.prof 
--perfscript=perf.data.script
 
 
 4. Build the code again using the collected profile. This step feeds
diff --git a/llvm/docs/ReleaseNotes.rst b/llvm/docs/ReleaseNotes.rst
index b34a5f31c5eb0a..c2bbc647bc18e6 100644
--- a/llvm/docs/ReleaseNotes.rst
+++ b/llvm/docs/ReleaseNotes.rst
@@ -157,6 +157,11 @@ Changes to the LLVM tools
   ``--set-symbols-visibility`` options for ELF input to change the
   visibility of symbols.
 
+* llvm-profgen now supports COFF+DWARF binaries. This enables Sample-based PGO
+  on Windows using Intel VTune's SEP. For details on usage, see the `end-user
+  documentation for SPGO
+  `_.
+
 Changes to LLDB
 -
 

>From 53f4c5dc84d71fd4efa5384818ecfc3401a0e7f6 Mon Sep 17 00:00:00 2001
From: Tim Creech 
Date: Tue, 12 Mar 2024 09:14:27 -0400

[clang] [llvm] Update documentation and release notes for llvm-profgen COFF support (PR #84864)

[Haohai Wen via cfe-commits]

@@ -2454,7 +2481,14 @@ usual build cycle when using sample profilers for 
optimization:
 
.. code-block:: console
 
- $ llvm-profgen --binary=./code --output=code.prof--perfdata=perf.data
+ $ llvm-profgen --binary=./code --output=code.prof --perfdata=perf.data
+
+   When using SEP the output is in the textual format corresponding to
+   ``llvm-profgen --perfscript``. For example:
+
+   .. code-block:: console
+
+ $ llvm-profgen --binary=./code --output=code.prof 
--perfscript=perf.data.script

HaohaiWen wrote:

code.perf.data.script

https://github.com/llvm/llvm-project/pull/84864
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [llvm] Update documentation and release notes for llvm-profgen COFF support (PR #84864)

[via cfe-commits]

llvmbot wrote:




@llvm/pr-subscribers-clang

Author: Tim Creech (tcreech-intel)


Changes

This change:
- Updates the existing Clang User's Manual section on SPGO so that it describes 
how to use llvm-profgen to perform SPGO on Windows. This is new functionality 
implemented in #83972.
- Fixes a minor typo in the existing llvm-profgen invocation example.
- Adds an LLVM release note on this new functionality in llvm-profgen.

---
Full diff: https://github.com/llvm/llvm-project/pull/84864.diff


2 Files Affected:

- (modified) clang/docs/UsersManual.rst (+47-13) 
- (modified) llvm/docs/ReleaseNotes.rst (+5) 


``diff
diff --git a/clang/docs/UsersManual.rst b/clang/docs/UsersManual.rst
index 7391e4cf3a9aeb..b83e6deac75e3f 100644
--- a/clang/docs/UsersManual.rst
+++ b/clang/docs/UsersManual.rst
@@ -2410,20 +2410,39 @@ usual build cycle when using sample profilers for 
optimization:
 
 1. Build the code with source line table information. You can use all the
usual build flags that you always build your application with. The only
-   requirement is that you add ``-gline-tables-only`` or ``-g`` to the
-   command line. This is important for the profiler to be able to map
-   instructions back to source line locations.
+   requirement is that DWARF debug info including source line information is
+   generated. This DWARF information is important for the profiler to be able
+   to map instructions back to source line locations.
+
+   On Linux, ``-g`` or just ``-gline-tables-only`` is sufficient:
 
.. code-block:: console
 
  $ clang++ -O2 -gline-tables-only code.cc -o code
 
+   While MSVC-style targets default to CodeView debug information, DWARF debug
+   information is required to generate source-level LLVM profiles. Use
+   ``-gdwarf`` to include DWARF debug information:
+
+   .. code-block:: console
+
+ $ clang-cl -O2 -gdwarf -gline-tables-only coff-profile.cpp -fuse-ld=lld 
-link -debug:dwarf
+
 2. Run the executable under a sampling profiler. The specific profiler
you use does not really matter, as long as its output can be converted
-   into the format that the LLVM optimizer understands. Currently, there
-   exists a conversion tool for the Linux Perf profiler
-   (https://perf.wiki.kernel.org/), so these examples assume that you
-   are using Linux Perf to profile your code.
+   into the format that the LLVM optimizer understands.
+
+   Two such profilers are the the Linux Perf profiler
+   (https://perf.wiki.kernel.org/) and Intel's Sampling Enabling Product (SEP),
+   available as part of `Intel VTune
+   
`_.
+   While Perf is Linux-specific, SEP can be used on Linux, Windows, and 
FreeBSD.
+
+   The LLVM tool ``llvm-profgen`` can convert output of either Perf or SEP. An
+   external project, `AutoFDO `_, also
+   provides a ``create_llvm_prof`` tool which supports Linux Perf output.
+
+   When using Perf:
 
.. code-block:: console
 
@@ -2434,11 +2453,19 @@ usual build cycle when using sample profilers for 
optimization:
it provides better call information, which improves the accuracy of
the profile data.
 
-3. Convert the collected profile data to LLVM's sample profile format.
-   This is currently supported via the AutoFDO converter ``create_llvm_prof``.
-   It is available at https://github.com/google/autofdo. Once built and
-   installed, you can convert the ``perf.data`` file to LLVM using
-   the command:
+   When using SEP:
+
+   .. code-block:: console
+
+ $ sep -start -out code.tb7 -ec 
BR_INST_RETIRED.NEAR_TAKEN:precise=yes:pdir -lbr no_filter:usr -perf-script 
brstack -app ./code
+
+   This produces a ``code.perf.data.script`` output which can be used with
+   ``llvm-profgen``'s ``--perfscript`` input option.
+
+3. Convert the collected profile data to LLVM's sample profile format. This is
+   currently supported via the `AutoFDO `_
+   converter ``create_llvm_prof``. Once built and installed, you can convert
+   the ``perf.data`` file to LLVM using the command:
 
.. code-block:: console
 
@@ -2454,7 +2481,14 @@ usual build cycle when using sample profilers for 
optimization:
 
.. code-block:: console
 
- $ llvm-profgen --binary=./code --output=code.prof--perfdata=perf.data
+ $ llvm-profgen --binary=./code --output=code.prof --perfdata=perf.data
+
+   When using SEP the output is in the textual format corresponding to
+   ``llvm-profgen --perfscript``. For example:
+
+   .. code-block:: console
+
+ $ llvm-profgen --binary=./code --output=code.prof 
--perfscript=perf.data.script
 
 
 4. Build the code again using the collected profile. This step feeds
diff --git a/llvm/docs/ReleaseNotes.rst b/llvm/docs/ReleaseNotes.rst
index b34a5f31c5eb0a..c2bbc647bc18e6 100644
--- a/llvm/docs/ReleaseNotes.rst
+++ b/llvm/docs/ReleaseNotes.rst
@@ -157,6 +157,11 @@ Changes to 

[clang] [llvm] Update documentation and release notes for llvm-profgen COFF support (PR #84864)

[Tim Creech via cfe-commits]

https://github.com/tcreech-intel ready_for_review 
https://github.com/llvm/llvm-project/pull/84864
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [llvm] Update documentation and release notes for llvm-profgen COFF support (PR #84864)

[Tim Creech via cfe-commits]

@@ -2410,20 +2410,35 @@ usual build cycle when using sample profilers for 
optimization:
 
 1. Build the code with source line table information. You can use all the
usual build flags that you always build your application with. The only
-   requirement is that you add ``-gline-tables-only`` or ``-g`` to the
-   command line. This is important for the profiler to be able to map
-   instructions back to source line locations.
+   requirement is that DWARF debug info including source line information is
+   generated. This DWARF information is important for the profiler to be able
+   to map instructions back to source line locations.
+
+   On Linux, ``-g`` or just ``-gline-tables-only`` is sufficient:
 
.. code-block:: console
 
  $ clang++ -O2 -gline-tables-only code.cc -o code
 
+   It is also possible to include DWARF in Windows binaries:

tcreech-intel wrote:

I've expanded on this a little in 
[a5e879c](https://github.com/llvm/llvm-project/pull/84864/commits/a5e879ce5016fee9cf3109bb9fc7785c396ac509).

https://github.com/llvm/llvm-project/pull/84864
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [llvm] Update documentation and release notes for llvm-profgen COFF support (PR #84864)

[Tim Creech via cfe-commits]

@@ -2410,20 +2410,35 @@ usual build cycle when using sample profilers for 
optimization:
 
 1. Build the code with source line table information. You can use all the
usual build flags that you always build your application with. The only
-   requirement is that you add ``-gline-tables-only`` or ``-g`` to the
-   command line. This is important for the profiler to be able to map
-   instructions back to source line locations.
+   requirement is that DWARF debug info including source line information is
+   generated. This DWARF information is important for the profiler to be able
+   to map instructions back to source line locations.
+
+   On Linux, ``-g`` or just ``-gline-tables-only`` is sufficient:
 
.. code-block:: console
 
  $ clang++ -O2 -gline-tables-only code.cc -o code
 
+   It is also possible to include DWARF in Windows binaries:
+
+   .. code-block:: console
+
+ $ clang-cl -O2 -gdwarf -gline-tables-only coff-profile.cpp -fuse-ld=lld 
-link -debug:dwarf
+
 2. Run the executable under a sampling profiler. The specific profiler
you use does not really matter, as long as its output can be converted
-   into the format that the LLVM optimizer understands. Currently, there
-   exists a conversion tool for the Linux Perf profiler
-   (https://perf.wiki.kernel.org/), so these examples assume that you
-   are using Linux Perf to profile your code.
+   into the format that the LLVM optimizer understands.
+
+   Two such profilers are the the Linux Perf profiler
+   (https://perf.wiki.kernel.org/) and Intel's Sampling Enabling Product (SEP),
+   available as part of `Intel VTune
+   
`_.
+
+   The LLVM tool ``llvm-profgen`` can convert output of either Perf or SEP. An
+   external tool, AutoFDO, also supports Linux Perf output.

tcreech-intel wrote:

Yes, probably a good idea to include a link here now that it's the first 
mention of AutoFDO. Fixed in 
[a5e879c](https://github.com/llvm/llvm-project/pull/84864/commits/a5e879ce5016fee9cf3109bb9fc7785c396ac509).

https://github.com/llvm/llvm-project/pull/84864
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [llvm] Update documentation and release notes for llvm-profgen COFF support (PR #84864)

[Tim Creech via cfe-commits]

@@ -2410,20 +2410,35 @@ usual build cycle when using sample profilers for 
optimization:
 
 1. Build the code with source line table information. You can use all the
usual build flags that you always build your application with. The only
-   requirement is that you add ``-gline-tables-only`` or ``-g`` to the
-   command line. This is important for the profiler to be able to map
-   instructions back to source line locations.
+   requirement is that DWARF debug info including source line information is
+   generated. This DWARF information is important for the profiler to be able
+   to map instructions back to source line locations.
+
+   On Linux, ``-g`` or just ``-gline-tables-only`` is sufficient:
 
.. code-block:: console
 
  $ clang++ -O2 -gline-tables-only code.cc -o code
 
+   It is also possible to include DWARF in Windows binaries:
+
+   .. code-block:: console
+
+ $ clang-cl -O2 -gdwarf -gline-tables-only coff-profile.cpp -fuse-ld=lld 
-link -debug:dwarf
+
 2. Run the executable under a sampling profiler. The specific profiler
you use does not really matter, as long as its output can be converted
-   into the format that the LLVM optimizer understands. Currently, there
-   exists a conversion tool for the Linux Perf profiler
-   (https://perf.wiki.kernel.org/), so these examples assume that you
-   are using Linux Perf to profile your code.
+   into the format that the LLVM optimizer understands.
+
+   Two such profilers are the the Linux Perf profiler
+   (https://perf.wiki.kernel.org/) and Intel's Sampling Enabling Product (SEP),

tcreech-intel wrote:

Thanks -- I've made this more clear in 
[a5e879c](https://github.com/llvm/llvm-project/pull/84864/commits/a5e879ce5016fee9cf3109bb9fc7785c396ac509).

https://github.com/llvm/llvm-project/pull/84864
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [llvm] Update documentation and release notes for llvm-profgen COFF support (PR #84864)

[Tim Creech via cfe-commits]

@@ -2434,6 +2449,15 @@ usual build cycle when using sample profilers for 
optimization:
it provides better call information, which improves the accuracy of
the profile data.
 
+   When using SEP:
+
+   .. code-block:: console
+
+ $ sep -start -ec BR_INST_RETIRED.NEAR_TAKEN:precise=yes:pdir -lbr 
no_filter:usr -perf-script ip,brstack -app ./code

tcreech-intel wrote:

Fixed in 
[53f4c5d](https://github.com/llvm/llvm-project/pull/84864/commits/53f4c5dc84d71fd4efa5384818ecfc3401a0e7f6).

https://github.com/llvm/llvm-project/pull/84864
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [llvm] Update documentation and release notes for llvm-profgen COFF support (PR #84864)

[Tim Creech via cfe-commits]

https://github.com/tcreech-intel updated 
https://github.com/llvm/llvm-project/pull/84864

>From 4dc108d0d290ee5fd6a73c029c051fdb2215d00a Mon Sep 17 00:00:00 2001
From: Tim Creech 
Date: Mon, 11 Mar 2024 22:35:59 -0400
Subject: [PATCH 1/4] Update documentation and release notes for llvm-profgen
 COFF support

This change:
- Updates the existing Clang User's Manual section on SPGO so that it
  describes how to use llvm-profgen to perform SPGO on Windows. This is
  new functionality implemented in #83972.
- Fixes a minor typo in the existing llvm-profgen invocation example.
- Adds an LLVM release note on this new functionality in llvm-profgen.
---
 clang/docs/UsersManual.rst | 47 +++---
 llvm/docs/ReleaseNotes.rst |  5 
 2 files changed, 44 insertions(+), 8 deletions(-)

diff --git a/clang/docs/UsersManual.rst b/clang/docs/UsersManual.rst
index 7391e4cf3a9aeb..9cf313c3727125 100644
--- a/clang/docs/UsersManual.rst
+++ b/clang/docs/UsersManual.rst
@@ -2410,20 +2410,35 @@ usual build cycle when using sample profilers for 
optimization:
 
 1. Build the code with source line table information. You can use all the
usual build flags that you always build your application with. The only
-   requirement is that you add ``-gline-tables-only`` or ``-g`` to the
-   command line. This is important for the profiler to be able to map
-   instructions back to source line locations.
+   requirement is that DWARF debug info including source line information is
+   generated. This DWARF information is important for the profiler to be able
+   to map instructions back to source line locations.
+
+   On Linux, ``-g`` or just ``-gline-tables-only`` is sufficient:
 
.. code-block:: console
 
  $ clang++ -O2 -gline-tables-only code.cc -o code
 
+   It is also possible to include DWARF in Windows binaries:
+
+   .. code-block:: console
+
+ $ clang-cl -O2 -gdwarf -gline-tables-only coff-profile.cpp -fuse-ld=lld 
-link -debug:dwarf
+
 2. Run the executable under a sampling profiler. The specific profiler
you use does not really matter, as long as its output can be converted
-   into the format that the LLVM optimizer understands. Currently, there
-   exists a conversion tool for the Linux Perf profiler
-   (https://perf.wiki.kernel.org/), so these examples assume that you
-   are using Linux Perf to profile your code.
+   into the format that the LLVM optimizer understands.
+
+   Two such profilers are the the Linux Perf profiler
+   (https://perf.wiki.kernel.org/) and Intel's Sampling Enabling Product (SEP),
+   available as part of `Intel VTune
+   
`_.
+
+   The LLVM tool ``llvm-profgen`` can convert output of either Perf or SEP. An
+   external tool, AutoFDO, also supports Linux Perf output.
+
+   When using Perf:
 
.. code-block:: console
 
@@ -2434,6 +2449,15 @@ usual build cycle when using sample profilers for 
optimization:
it provides better call information, which improves the accuracy of
the profile data.
 
+   When using SEP:
+
+   .. code-block:: console
+
+ $ sep -start -ec BR_INST_RETIRED.NEAR_TAKEN:precise=yes:pdir -lbr 
no_filter:usr -perf-script ip,brstack -app ./code
+
+   This produces a ``perf.data.script`` output which can be used with
+   ``llvm-profgen``'s ``--perfscript`` input option.
+
 3. Convert the collected profile data to LLVM's sample profile format.
This is currently supported via the AutoFDO converter ``create_llvm_prof``.
It is available at https://github.com/google/autofdo. Once built and
@@ -2454,7 +2478,14 @@ usual build cycle when using sample profilers for 
optimization:
 
.. code-block:: console
 
- $ llvm-profgen --binary=./code --output=code.prof--perfdata=perf.data
+ $ llvm-profgen --binary=./code --output=code.prof --perfdata=perf.data
+
+   When using SEP the output is in the textual format corresponding to
+   `llvm-profgen --perfscript`. For example:
+
+   .. code-block:: console
+
+ $ llvm-profgen --binary=./code --output=code.prof 
--perfscript=perf.data.script
 
 
 4. Build the code again using the collected profile. This step feeds
diff --git a/llvm/docs/ReleaseNotes.rst b/llvm/docs/ReleaseNotes.rst
index b34a5f31c5eb0a..c2bbc647bc18e6 100644
--- a/llvm/docs/ReleaseNotes.rst
+++ b/llvm/docs/ReleaseNotes.rst
@@ -157,6 +157,11 @@ Changes to the LLVM tools
   ``--set-symbols-visibility`` options for ELF input to change the
   visibility of symbols.
 
+* llvm-profgen now supports COFF+DWARF binaries. This enables Sample-based PGO
+  on Windows using Intel VTune's SEP. For details on usage, see the `end-user
+  documentation for SPGO
+  `_.
+
 Changes to LLDB
 -
 

>From 53f4c5dc84d71fd4efa5384818ecfc3401a0e7f6 Mon Sep 17 00:00:00 2001
From: Tim Creech 
Date: Tue, 12 Mar 2024 09:14:27 -0400

[clang] [llvm] Update documentation and release notes for llvm-profgen COFF support (PR #84864)

[Tim Creech via cfe-commits]

@@ -2434,6 +2449,15 @@ usual build cycle when using sample profilers for 
optimization:
it provides better call information, which improves the accuracy of
the profile data.
 
+   When using SEP:
+
+   .. code-block:: console
+
+ $ sep -start -ec BR_INST_RETIRED.NEAR_TAKEN:precise=yes:pdir -lbr 
no_filter:usr -perf-script ip,brstack -app ./code

tcreech-intel wrote:

I had omitted `-out` for the sake of brevity, but maybe this is confusing. I'll 
update it and name a specific perf.data.script file below.

https://github.com/llvm/llvm-project/pull/84864
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [llvm] Update documentation and release notes for llvm-profgen COFF support (PR #84864)

[Martin Storsjö via cfe-commits]

@@ -2410,20 +2410,35 @@ usual build cycle when using sample profilers for 
optimization:
 
 1. Build the code with source line table information. You can use all the
usual build flags that you always build your application with. The only
-   requirement is that you add ``-gline-tables-only`` or ``-g`` to the
-   command line. This is important for the profiler to be able to map
-   instructions back to source line locations.
+   requirement is that DWARF debug info including source line information is
+   generated. This DWARF information is important for the profiler to be able
+   to map instructions back to source line locations.
+
+   On Linux, ``-g`` or just ``-gline-tables-only`` is sufficient:
 
.. code-block:: console
 
  $ clang++ -O2 -gline-tables-only code.cc -o code
 
+   It is also possible to include DWARF in Windows binaries:

mstorsjo wrote:

Codeview is the standard in MSVC style environments, while DWARF is the 
standard in MinGW too.

https://github.com/llvm/llvm-project/pull/84864
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [llvm] Update documentation and release notes for llvm-profgen COFF support (PR #84864)

[Tobias Hieta via cfe-commits]

@@ -2410,20 +2410,35 @@ usual build cycle when using sample profilers for 
optimization:
 
 1. Build the code with source line table information. You can use all the
usual build flags that you always build your application with. The only
-   requirement is that you add ``-gline-tables-only`` or ``-g`` to the
-   command line. This is important for the profiler to be able to map
-   instructions back to source line locations.
+   requirement is that DWARF debug info including source line information is
+   generated. This DWARF information is important for the profiler to be able
+   to map instructions back to source line locations.
+
+   On Linux, ``-g`` or just ``-gline-tables-only`` is sufficient:
 
.. code-block:: console
 
  $ clang++ -O2 -gline-tables-only code.cc -o code
 
+   It is also possible to include DWARF in Windows binaries:
+
+   .. code-block:: console
+
+ $ clang-cl -O2 -gdwarf -gline-tables-only coff-profile.cpp -fuse-ld=lld 
-link -debug:dwarf
+
 2. Run the executable under a sampling profiler. The specific profiler
you use does not really matter, as long as its output can be converted
-   into the format that the LLVM optimizer understands. Currently, there
-   exists a conversion tool for the Linux Perf profiler
-   (https://perf.wiki.kernel.org/), so these examples assume that you
-   are using Linux Perf to profile your code.
+   into the format that the LLVM optimizer understands.
+
+   Two such profilers are the the Linux Perf profiler
+   (https://perf.wiki.kernel.org/) and Intel's Sampling Enabling Product (SEP),
+   available as part of `Intel VTune
+   
`_.
+
+   The LLVM tool ``llvm-profgen`` can convert output of either Perf or SEP. An
+   external tool, AutoFDO, also supports Linux Perf output.

tru wrote:

maybe link to AutoFDO or explain what it's used for here?

https://github.com/llvm/llvm-project/pull/84864
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [llvm] Update documentation and release notes for llvm-profgen COFF support (PR #84864)

[Tobias Hieta via cfe-commits]

@@ -2410,20 +2410,35 @@ usual build cycle when using sample profilers for 
optimization:
 
 1. Build the code with source line table information. You can use all the
usual build flags that you always build your application with. The only
-   requirement is that you add ``-gline-tables-only`` or ``-g`` to the
-   command line. This is important for the profiler to be able to map
-   instructions back to source line locations.
+   requirement is that DWARF debug info including source line information is
+   generated. This DWARF information is important for the profiler to be able
+   to map instructions back to source line locations.
+
+   On Linux, ``-g`` or just ``-gline-tables-only`` is sufficient:
 
.. code-block:: console
 
  $ clang++ -O2 -gline-tables-only code.cc -o code
 
+   It is also possible to include DWARF in Windows binaries:
+
+   .. code-block:: console
+
+ $ clang-cl -O2 -gdwarf -gline-tables-only coff-profile.cpp -fuse-ld=lld 
-link -debug:dwarf
+
 2. Run the executable under a sampling profiler. The specific profiler
you use does not really matter, as long as its output can be converted
-   into the format that the LLVM optimizer understands. Currently, there
-   exists a conversion tool for the Linux Perf profiler
-   (https://perf.wiki.kernel.org/), so these examples assume that you
-   are using Linux Perf to profile your code.
+   into the format that the LLVM optimizer understands.
+
+   Two such profilers are the the Linux Perf profiler
+   (https://perf.wiki.kernel.org/) and Intel's Sampling Enabling Product (SEP),

tru wrote:

I think we can make it clear that perf is only on Linux while SEP is Linux and 
Windows.

https://github.com/llvm/llvm-project/pull/84864
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [llvm] Update documentation and release notes for llvm-profgen COFF support (PR #84864)

[Tobias Hieta via cfe-commits]

@@ -2410,20 +2410,35 @@ usual build cycle when using sample profilers for 
optimization:
 
 1. Build the code with source line table information. You can use all the
usual build flags that you always build your application with. The only
-   requirement is that you add ``-gline-tables-only`` or ``-g`` to the
-   command line. This is important for the profiler to be able to map
-   instructions back to source line locations.
+   requirement is that DWARF debug info including source line information is
+   generated. This DWARF information is important for the profiler to be able
+   to map instructions back to source line locations.
+
+   On Linux, ``-g`` or just ``-gline-tables-only`` is sufficient:
 
.. code-block:: console
 
  $ clang++ -O2 -gline-tables-only code.cc -o code
 
+   It is also possible to include DWARF in Windows binaries:

tru wrote:

Maybe this should be worded that while codeview is the standard on Windows, you 
can use DWARF instead.

https://github.com/llvm/llvm-project/pull/84864
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [llvm] Update documentation and release notes for llvm-profgen COFF support (PR #84864)

[Tobias Hieta via cfe-commits]

https://github.com/tru commented:

Thanks for working on documentation!

https://github.com/llvm/llvm-project/pull/84864
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [llvm] Update documentation and release notes for llvm-profgen COFF support (PR #84864)

[Tobias Hieta via cfe-commits]

https://github.com/tru edited https://github.com/llvm/llvm-project/pull/84864
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [llvm] Update documentation and release notes for llvm-profgen COFF support (PR #84864)

[Haohai Wen via cfe-commits]

@@ -2434,6 +2449,15 @@ usual build cycle when using sample profilers for 
optimization:
it provides better call information, which improves the accuracy of
the profile data.
 
+   When using SEP:
+
+   .. code-block:: console
+
+ $ sep -start -ec BR_INST_RETIRED.NEAR_TAKEN:precise=yes:pdir -lbr 
no_filter:usr -perf-script ip,brstack -app ./code

HaohaiWen wrote:

I think we should remove ip on output perf script and just leave `-perf-script 
brstack`

https://github.com/llvm/llvm-project/pull/84864
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [llvm] Update documentation and release notes for llvm-profgen COFF support (PR #84864)

[Haohai Wen via cfe-commits]

@@ -2434,6 +2449,15 @@ usual build cycle when using sample profilers for 
optimization:
it provides better call information, which improves the accuracy of
the profile data.
 
+   When using SEP:
+
+   .. code-block:: console
+
+ $ sep -start -ec BR_INST_RETIRED.NEAR_TAKEN:precise=yes:pdir -lbr 
no_filter:usr -perf-script ip,brstack -app ./code
+
+   This produces a ``perf.data.script`` output which can be used with

HaohaiWen wrote:

Should change to code.perf.data.script if specify -out code.tb7

https://github.com/llvm/llvm-project/pull/84864
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [llvm] Update documentation and release notes for llvm-profgen COFF support (PR #84864)

[Haohai Wen via cfe-commits]

@@ -2434,6 +2449,15 @@ usual build cycle when using sample profilers for 
optimization:
it provides better call information, which improves the accuracy of
the profile data.
 
+   When using SEP:
+
+   .. code-block:: console
+
+ $ sep -start -ec BR_INST_RETIRED.NEAR_TAKEN:precise=yes:pdir -lbr 
no_filter:usr -perf-script ip,brstack -app ./code

HaohaiWen wrote:

Should we add `-out code.tb7 to sep'? Otherwise sep will generate a temp name 
(tbs17644.perf.data.script in my env).
e.g.
```
sep -start -out code.tb7 -ec BR_INST_RETIRED.NEAR_TAKEN:precise=yes:pdir -lbr 
no_filter:usr -perf-script brstack -app ./code
```

https://github.com/llvm/llvm-project/pull/84864
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [llvm] Update documentation and release notes for llvm-profgen COFF support (PR #84864)

[via cfe-commits]

github-actions[bot] wrote:



Thank you for submitting a Pull Request (PR) to the LLVM Project!

This PR will be automatically labeled and the relevant teams will be
notified.

If you wish to, you can add reviewers by using the "Reviewers" section on this 
page.

If this is not working for you, it is probably because you do not have write
permissions for the repository. In which case you can instead tag reviewers by
name in a comment by using `@` followed by their GitHub username.

If you have received no comments on your PR for a week, you can request a review
by "ping"ing the PR by adding a comment “Ping”. The common courtesy "ping" rate
is once a week. Please remember that you are asking for valuable time from 
other developers.

If you have further questions, they may be answered by the [LLVM GitHub User 
Guide](https://llvm.org/docs/GitHub.html).

You can also ask questions in a comment on this PR, on the [LLVM 
Discord](https://discord.com/invite/xS7Z362) or on the 
[forums](https://discourse.llvm.org/).

https://github.com/llvm/llvm-project/pull/84864
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [llvm] Update documentation and release notes for llvm-profgen COFF support (PR #84864)

[Tim Creech via cfe-commits]

https://github.com/tcreech-intel created 
https://github.com/llvm/llvm-project/pull/84864

This change:
- Updates the existing Clang User's Manual section on SPGO so that it describes 
how to use llvm-profgen to perform SPGO on Windows. This is new functionality 
implemented in #83972.
- Fixes a minor typo in the existing llvm-profgen invocation example.
- Adds an LLVM release note on this new functionality in llvm-profgen.

>From 4dc108d0d290ee5fd6a73c029c051fdb2215d00a Mon Sep 17 00:00:00 2001
From: Tim Creech 
Date: Mon, 11 Mar 2024 22:35:59 -0400
Subject: [PATCH] Update documentation and release notes for llvm-profgen COFF
 support

This change:
- Updates the existing Clang User's Manual section on SPGO so that it
  describes how to use llvm-profgen to perform SPGO on Windows. This is
  new functionality implemented in #83972.
- Fixes a minor typo in the existing llvm-profgen invocation example.
- Adds an LLVM release note on this new functionality in llvm-profgen.
---
 clang/docs/UsersManual.rst | 47 +++---
 llvm/docs/ReleaseNotes.rst |  5 
 2 files changed, 44 insertions(+), 8 deletions(-)

diff --git a/clang/docs/UsersManual.rst b/clang/docs/UsersManual.rst
index 7391e4cf3a9aeb..9cf313c3727125 100644
--- a/clang/docs/UsersManual.rst
+++ b/clang/docs/UsersManual.rst
@@ -2410,20 +2410,35 @@ usual build cycle when using sample profilers for 
optimization:
 
 1. Build the code with source line table information. You can use all the
usual build flags that you always build your application with. The only
-   requirement is that you add ``-gline-tables-only`` or ``-g`` to the
-   command line. This is important for the profiler to be able to map
-   instructions back to source line locations.
+   requirement is that DWARF debug info including source line information is
+   generated. This DWARF information is important for the profiler to be able
+   to map instructions back to source line locations.
+
+   On Linux, ``-g`` or just ``-gline-tables-only`` is sufficient:
 
.. code-block:: console
 
  $ clang++ -O2 -gline-tables-only code.cc -o code
 
+   It is also possible to include DWARF in Windows binaries:
+
+   .. code-block:: console
+
+ $ clang-cl -O2 -gdwarf -gline-tables-only coff-profile.cpp -fuse-ld=lld 
-link -debug:dwarf
+
 2. Run the executable under a sampling profiler. The specific profiler
you use does not really matter, as long as its output can be converted
-   into the format that the LLVM optimizer understands. Currently, there
-   exists a conversion tool for the Linux Perf profiler
-   (https://perf.wiki.kernel.org/), so these examples assume that you
-   are using Linux Perf to profile your code.
+   into the format that the LLVM optimizer understands.
+
+   Two such profilers are the the Linux Perf profiler
+   (https://perf.wiki.kernel.org/) and Intel's Sampling Enabling Product (SEP),
+   available as part of `Intel VTune
+   
`_.
+
+   The LLVM tool ``llvm-profgen`` can convert output of either Perf or SEP. An
+   external tool, AutoFDO, also supports Linux Perf output.
+
+   When using Perf:
 
.. code-block:: console
 
@@ -2434,6 +2449,15 @@ usual build cycle when using sample profilers for 
optimization:
it provides better call information, which improves the accuracy of
the profile data.
 
+   When using SEP:
+
+   .. code-block:: console
+
+ $ sep -start -ec BR_INST_RETIRED.NEAR_TAKEN:precise=yes:pdir -lbr 
no_filter:usr -perf-script ip,brstack -app ./code
+
+   This produces a ``perf.data.script`` output which can be used with
+   ``llvm-profgen``'s ``--perfscript`` input option.
+
 3. Convert the collected profile data to LLVM's sample profile format.
This is currently supported via the AutoFDO converter ``create_llvm_prof``.
It is available at https://github.com/google/autofdo. Once built and
@@ -2454,7 +2478,14 @@ usual build cycle when using sample profilers for 
optimization:
 
.. code-block:: console
 
- $ llvm-profgen --binary=./code --output=code.prof--perfdata=perf.data
+ $ llvm-profgen --binary=./code --output=code.prof --perfdata=perf.data
+
+   When using SEP the output is in the textual format corresponding to
+   `llvm-profgen --perfscript`. For example:
+
+   .. code-block:: console
+
+ $ llvm-profgen --binary=./code --output=code.prof 
--perfscript=perf.data.script
 
 
 4. Build the code again using the collected profile. This step feeds
diff --git a/llvm/docs/ReleaseNotes.rst b/llvm/docs/ReleaseNotes.rst
index b34a5f31c5eb0a..c2bbc647bc18e6 100644
--- a/llvm/docs/ReleaseNotes.rst
+++ b/llvm/docs/ReleaseNotes.rst
@@ -157,6 +157,11 @@ Changes to the LLVM tools
   ``--set-symbols-visibility`` options for ELF input to change the
   visibility of symbols.
 
+* llvm-profgen now supports COFF+DWARF binaries. This enables Sample-based PGO
+  on Windows using Intel VTune's