[clang] [Clang][CMake] Support perf, LBR, and Instrument CLANG_BOLT options (PR #69133)

2024-01-29 Thread Amir Ayupov via cfe-commits 

aaupov wrote:

Final update:
- [clang-hip-vega20](https://lab.llvm.org/buildbot/#/builders/165): green 
https://lab.llvm.org/buildbot/#/builders/165/builds/48052

https://github.com/llvm/llvm-project/pull/69133
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [Clang][CMake] Support perf, LBR, and Instrument CLANG_BOLT options (PR #69133)

2024-01-29 Thread Amir Ayupov via cfe-commits 

aaupov wrote:

Status update:
- [sanitizer-windows](https://lab.llvm.org/buildbot/#/builders/127): back to 
green starting with https://lab.llvm.org/buildbot/#/builders/127/builds/61505
- 
[libc-aarch64-ubuntu-fullbuild-dbg](https://lab.llvm.org/buildbot/#/builders/223):
 green https://lab.llvm.org/buildbot/#/builders/223/builds/35300
- 
[libc-x86_64-debian-gcc-fullbuild-dbg](https://lab.llvm.org/buildbot/#/builders/250):
 green https://lab.llvm.org/buildbot/#/builders/250/builds/17643
- 
[libc-x86_64-debian-dbg-runtimes-build](https://lab.llvm.org/buildbot/#/builders/225):
 green https://lab.llvm.org/buildbot/#/builders/225/builds/30352
- 
[libc-x86_64-debian-fullbuild-dbg](https://lab.llvm.org/buildbot/#/builders/163):
 green https://lab.llvm.org/buildbot/#/builders/163/builds/50708
- [clang-hip-vega20](https://lab.llvm.org/buildbot/#/builders/165): red


https://github.com/llvm/llvm-project/pull/69133
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [Clang][CMake] Support perf, LBR, and Instrument CLANG_BOLT options (PR #69133)

2024-01-28 Thread Amir Ayupov via cfe-commits 

aaupov wrote:

This issue with incremental builds is explicitly mentioned in 
[HowToAddABuilder](https://llvm.org/docs/HowToAddABuilder.html):

> Use CCache and NOT incremental builds
Using ccache materially improves average build times. Incremental builds can be 
slightly faster, but introduce the risk of build corruption due to e.g. state 
changes, etc… At this point, the recommendation is not to use incremental 
builds and instead use ccache as the latter captures the majority of the 
benefit with less risk of false positives.

I've sent an email to builders owners. Still, sorry about landing it on a 
weekend. I expected that builders are configured according to the guidelines so 
the issues will clear automatically. Apologies for those depending on these 
broken builders.

https://github.com/llvm/llvm-project/pull/69133
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [Clang][CMake] Support perf, LBR, and Instrument CLANG_BOLT options (PR #69133)

2024-01-28 Thread Amir Ayupov via cfe-commits 

aaupov wrote:

Status of the builders that reported breakage:
- [reverse-iteration](https://lab.llvm.org/buildbot/#/builders/54): 
https://lab.llvm.org/buildbot/#/builders/54/builds/8671 broken, 
https://lab.llvm.org/buildbot/#/builders/54/builds/8672 green, breakage was 
unrelated (unresolved tests)
- 
[sanitizer-aarch64-linux-bootstrap-msan](https://lab.llvm.org/buildbot/#/builders/237)
 - annotated builder: https://lab.llvm.org/buildbot/#/builders/237/builds/6271 
broken, https://lab.llvm.org/buildbot/#/builders/237/builds/6272 green after 
cleanup step
- 
[sanitizer-x86_64-linux-bootstrap-ubsan](https://lab.llvm.org/buildbot/#/builders/85)
 - annotated builder: https://lab.llvm.org/buildbot/#/builders/85/builds/21896 
broken, https://lab.llvm.org/buildbot/#/builders/85/builds/21897 green after 
cleanup step
- 
[sanitizer-aarch64-linux-bootstrap-ubsan](https://lab.llvm.org/buildbot/#/builders/238)
 - annotated builder: https://lab.llvm.org/buildbot/#/builders/238/builds/7702 
broken, https://lab.llvm.org/buildbot/#/builders/238/builds/7703 green after 
cleanup
- [sanitizer-x86_64-linux](https://lab.llvm.org/buildbot/#/builders/37) - 
annotated builder: https://lab.llvm.org/buildbot/#/builders/37/builds/30519 
broken, https://lab.llvm.org/buildbot/#/builders/37/builds/30521 green after 
two builds
- 
[sanitizer-aarch64-linux-bootstrap-asan](https://lab.llvm.org/buildbot/#/builders/239)
 - annotated builder, https://lab.llvm.org/buildbot/#/builders/239/builds/5498 
broken due to merge-fdata target (related), 
https://lab.llvm.org/buildbot/#/builders/239/builds/5499 broken due to some 
other unrelated cause. Cleanup step resolved cmake issue.
- [premerge-monolithic-windows](https://lab.llvm.org/buildbot/#/builders/271): 
not annotated builder, https://lab.llvm.org/buildbot/#/builders/271/builds/3444 
- broken as clean-build-dir step failed, 
https://lab.llvm.org/buildbot/#/builders/271/builds/3446 - green after 
successful cleanup.
- 
[libc-x86_64-debian-fullbuild-dbg](https://lab.llvm.org/buildbot/#/builders/163)
 - annotated builder: https://lab.llvm.org/buildbot/#/builders/163/builds/50669 
- broken, still broken because of a **missing** cleanup step.
- 
[libc-x86_64-debian-dbg-runtimes-build](https://lab.llvm.org/buildbot/#/builders/225)
 - annotated builder, https://lab.llvm.org/buildbot/#/builders/225/builds/30309 
broken, missing cleanup step.
- [clang-hip-vega20](https://lab.llvm.org/buildbot/#/builders/165) - annotated 
builder, https://lab.llvm.org/buildbot/#/builders/165/builds/48023 broken, 
missing cleanup step.
- 
[libc-x86_64-debian-gcc-fullbuild-dbg](https://lab.llvm.org/buildbot/#/builders/250)
 - annotated builder, https://lab.llvm.org/buildbot/#/builders/250/builds/17603 
broken, missing cleanup step.
- 
[libc-aarch64-ubuntu-fullbuild-dbg](https://lab.llvm.org/buildbot/#/builders/223)
 - annotated builder, https://lab.llvm.org/buildbot/#/builders/223/builds/35258 
broken, missing cleanup step.
- [sanitizer-windows](https://lab.llvm.org/buildbot/#/builders/127) - annotated 
builder, https://lab.llvm.org/buildbot/#/builders/127/builds/61496 broken, 
missing cleanup step.

In short, this diff is a red herring for the reported breakages. The real cause 
is annotated builders not handling build dir cleanup properly. Still, I'm 
reaching out to all of the broken buildbot owners asking to clean the build 
directory manually.

https://github.com/llvm/llvm-project/pull/69133
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [Clang][CMake] Support perf, LBR, and Instrument CLANG_BOLT options (PR #69133)

2024-01-28 Thread via cfe-commits 

lntue wrote:

> These are all annotated builders. I tried triggering rebuilds with clean 
> build dir through buildbot UI but it didn't have an effect. I don't think 
> it's a reasonable expectation that every change must configure cleanly from 
> an existing build directory.

I'm sorry but I also don't think it is reasonable to re-landing a change that 
is known for breaking builders during the weekends, when not many people are 
around to react to the breakage.

https://github.com/llvm/llvm-project/pull/69133
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [Clang][CMake] Support perf, LBR, and Instrument CLANG_BOLT options (PR #69133)

2024-01-28 Thread Jan Patrick Lehr via cfe-commits 

jplehr wrote:

Hi @aaupov I think this did break the AMD Hip build bot (another annotated 
builder).
Are you looking into the potential issue?

https://github.com/llvm/llvm-project/pull/69133
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [Clang][CMake] Support perf, LBR, and Instrument CLANG_BOLT options (PR #69133)

2024-01-27 Thread Amir Ayupov via cfe-commits 

aaupov wrote:

These are all annotated builders. I tried triggering rebuilds with clean build 
dir through buildbot UI but it didn't have an effect. I don't think it's a 
reasonable expectation that every change must configure cleanly from an 
existing build directory.

https://github.com/llvm/llvm-project/pull/69133
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [Clang][CMake] Support perf, LBR, and Instrument CLANG_BOLT options (PR #69133)

2024-01-22 Thread Amir Ayupov via cfe-commits 

aaupov wrote:

Reverted. Strangely I couldn't repro from 
https://lab.llvm.org/buildbot/#/builders/225/builds/29950/ the failure locally. 
But will get to the bottom of it.

https://github.com/llvm/llvm-project/pull/69133
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [Clang][CMake] Support perf, LBR, and Instrument CLANG_BOLT options (PR #69133)

2024-01-22 Thread Nick Desaulniers via cfe-commits 

nickdesaulniers wrote:

I think the libc buildbots are now failing as a result of this change. @aaupov 
can you PTAL?

- https://lab.llvm.org/buildbot/#/builders/223/builds/34823
- https://lab.llvm.org/buildbot/#/builders/225/builds/29950

https://github.com/llvm/llvm-project/pull/69133
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [Clang][CMake] Support perf, LBR, and Instrument CLANG_BOLT options (PR #69133)

2024-01-22 Thread David Spickett via cfe-commits 


@@ -850,23 +850,38 @@ if (CLANG_ENABLE_BOOTSTRAP)
   endforeach()
 endif()
 
-if (CLANG_BOLT_INSTRUMENT AND NOT LLVM_BUILD_INSTRUMENTED)
+set(CLANG_BOLT "INSTRUMENT" CACHE STRING "Apply BOLT optimization to Clang. \

DavidSpickett wrote:

Wild guess, this defaults to `INSTRUMENT`, but it should default to some `OFF` 
value?

https://github.com/llvm/llvm-project/pull/69133
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [Clang][CMake] Support perf, LBR, and Instrument CLANG_BOLT options (PR #69133)

2024-01-22 Thread David Spickett via cfe-commits 

DavidSpickett wrote:

https://lab.llvm.org/buildbot/#/builders/245/builds/19468

```
CMake Error at 
/home/tcwg-buildbot/worker/clang-armv8-quick/llvm/clang/utils/perf-training/CMakeLists.txt:105
 (add_custom_target):
  Error evaluating generator expression:
$
  No target "merge-fdata"
```

This particular bot is clang+llvm and only the ARM target, but we've seen it on 
our AArch64 bots too. 

https://github.com/llvm/llvm-project/pull/69133
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [Clang][CMake] Support perf, LBR, and Instrument CLANG_BOLT options (PR #69133)

2024-01-22 Thread David Spickett via cfe-commits 

DavidSpickett wrote:

I think because bots don't always clean their cmake files, revert incoming.

https://github.com/llvm/llvm-project/pull/69133
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [Clang][CMake] Support perf, LBR, and Instrument CLANG_BOLT options (PR #69133)

2024-01-21 Thread via cfe-commits 

dyung wrote:

@aaupov I believe this change is somehow causing cmake failures on several bots 
I oversee, although I cannot explain why it passed when building your commit. 
For example:

https://lab.llvm.org/buildbot/#/builders/139/builds/57628
```
CMake Error at 
/home/buildbot/buildbot-root/llvm-clang-x86_64-sie-ubuntu-fast/llvm-project/clang/utils/perf-training/CMakeLists.txt:105
 (add_custom_target):
  Error evaluating generator expression:
$
  No target "merge-fdata"
```
Any idea what might be going wrong here?

https://github.com/llvm/llvm-project/pull/69133
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [Clang][CMake] Support perf, LBR, and Instrument CLANG_BOLT options (PR #69133)

2024-01-21 Thread Amir Ayupov via cfe-commits 

https://github.com/aaupov closed https://github.com/llvm/llvm-project/pull/69133
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [Clang][CMake] Support perf, LBR, and Instrument CLANG_BOLT options (PR #69133)

2024-01-21 Thread Petr Hosek via cfe-commits 

https://github.com/petrhosek approved this pull request.


https://github.com/llvm/llvm-project/pull/69133
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [Clang][CMake] Support perf, LBR, and Instrument CLANG_BOLT options (PR #69133)

2023-11-07 Thread Amir Ayupov via cfe-commits 

https://github.com/aaupov updated 
https://github.com/llvm/llvm-project/pull/69133

>From 1ca9ce9ef3e90e3d24aea3945ae2d5129682621d Mon Sep 17 00:00:00 2001
From: Amir Aupov 
Date: Mon, 16 Oct 2023 01:08:28 +0200
Subject: [PATCH 1/3] [Clang][CMake] Support perf, LBR, and Instrument
 CLANG_BOLT options

Split up and refactor CLANG_BOLT_INSTRUMENT into support for
perf no-LBR and perf with LBR profiling modes.

Differential Revision: https://reviews.llvm.org/D143617
---
 clang/CMakeLists.txt  | 44 -
 clang/cmake/caches/BOLT.cmake |  2 +-
 clang/utils/perf-training/CMakeLists.txt  | 29 -
 clang/utils/perf-training/bolt.lit.cfg| 53 +---
 .../utils/perf-training/bolt.lit.site.cfg.in  |  2 +
 clang/utils/perf-training/perf-helper.py  | 63 +++
 6 files changed, 167 insertions(+), 26 deletions(-)

diff --git a/clang/CMakeLists.txt b/clang/CMakeLists.txt
index 9b52c58be41e7f7..8f64d95cc394ffe 100644
--- a/clang/CMakeLists.txt
+++ b/clang/CMakeLists.txt
@@ -850,23 +850,38 @@ if (CLANG_ENABLE_BOOTSTRAP)
   endforeach()
 endif()
 
-if (CLANG_BOLT_INSTRUMENT AND NOT LLVM_BUILD_INSTRUMENTED)
+set(CLANG_BOLT "INSTRUMENT" CACHE STRING "Apply BOLT optimization to Clang. \
+  May be specified as Instrument or Perf or LBR to use a particular profiling \
+  mechanism.")
+string(TOUPPER "${CLANG_BOLT}" uppercase_CLANG_BOLT)
+
+if (CLANG_BOLT AND NOT LLVM_BUILD_INSTRUMENTED)
   set(CLANG_PATH ${LLVM_RUNTIME_OUTPUT_INTDIR}/clang)
-  set(CLANG_INSTRUMENTED ${CLANG_PATH}-bolt.inst)
+  set(CLANG_INSTRUMENTED 
${LLVM_RUNTIME_OUTPUT_INTDIR}/${CLANG_BOLT_INSTRUMENTED})
   set(BOLT_FDATA ${CMAKE_CURRENT_BINARY_DIR}/utils/perf-training/prof.fdata)
 
-  # Instrument clang with BOLT
-  add_custom_target(clang-instrumented
-DEPENDS ${CLANG_INSTRUMENTED}
-  )
-  add_custom_command(OUTPUT ${CLANG_INSTRUMENTED}
-DEPENDS clang llvm-bolt
-COMMAND llvm-bolt ${CLANG_PATH} -o ${CLANG_INSTRUMENTED}
-  -instrument --instrumentation-file-append-pid
-  --instrumentation-file=${BOLT_FDATA}
-COMMENT "Instrumenting clang binary with BOLT"
-VERBATIM
-  )
+  # Pass extra flag in no-LBR mode
+  if (uppercase_CLANG_BOLT STREQUAL "PERF")
+set(BOLT_NO_LBR "-nl")
+  endif()
+
+  if (uppercase_CLANG_BOLT STREQUAL "INSTRUMENT")
+# Instrument clang with BOLT
+add_custom_target(clang-instrumented
+  DEPENDS ${CLANG_INSTRUMENTED}
+)
+add_custom_command(OUTPUT ${CLANG_INSTRUMENTED}
+  DEPENDS clang llvm-bolt
+  COMMAND llvm-bolt ${CLANG_PATH} -o ${CLANG_INSTRUMENTED}
+-instrument --instrumentation-file-append-pid
+--instrumentation-file=${BOLT_FDATA}
+  COMMENT "Instrumenting clang binary with BOLT"
+  VERBATIM
+)
+add_custom_target(clang-bolt-training-deps DEPENDS clang-instrumented)
+  else() # perf or LBR
+add_custom_target(clang-bolt-training-deps DEPENDS clang)
+  endif()
 
   # Optimize original (pre-bolt) Clang using the collected profile
   set(CLANG_OPTIMIZED ${CMAKE_CURRENT_BINARY_DIR}/clang.bolt)
@@ -880,6 +895,7 @@ if (CLANG_BOLT_INSTRUMENT AND NOT LLVM_BUILD_INSTRUMENTED)
   -data ${BOLT_FDATA}
   -reorder-blocks=ext-tsp -reorder-functions=hfsort+ -split-functions
   -split-all-cold -split-eh -dyno-stats -icf=1 -use-gnu-stack
+  ${BOLT_NO_LBR}
 COMMAND ${CMAKE_COMMAND} -E rename ${CLANG_OPTIMIZED} $
 COMMENT "Optimizing Clang with BOLT"
 VERBATIM
diff --git a/clang/cmake/caches/BOLT.cmake b/clang/cmake/caches/BOLT.cmake
index 0442f73e5426ac7..eba2346b2f4ca12 100644
--- a/clang/cmake/caches/BOLT.cmake
+++ b/clang/cmake/caches/BOLT.cmake
@@ -1,5 +1,5 @@
 set(CMAKE_BUILD_TYPE Release CACHE STRING "")
-set(CLANG_BOLT_INSTRUMENT ON CACHE BOOL "")
+set(CLANG_BOLT "INSTRUMENT" CACHE STRING "")
 set(CMAKE_EXE_LINKER_FLAGS "-Wl,--emit-relocs,-znow" CACHE STRING "")
 
 set(LLVM_ENABLE_PROJECTS "bolt;clang" CACHE STRING "")
diff --git a/clang/utils/perf-training/CMakeLists.txt 
b/clang/utils/perf-training/CMakeLists.txt
index c6d51863fb1b5c2..48fbee62a8636d1 100644
--- a/clang/utils/perf-training/CMakeLists.txt
+++ b/clang/utils/perf-training/CMakeLists.txt
@@ -62,7 +62,9 @@ if(APPLE AND DTRACE AND NOT LLVM_TOOL_LLVM_DRIVER_BUILD)
 DEPENDS generate-dtrace-logs)
 endif()
 
-if(CLANG_BOLT_INSTRUMENT AND NOT LLVM_BUILD_INSTRUMENTED)
+if(CLANG_BOLT AND NOT LLVM_BUILD_INSTRUMENTED)
+  set(CLANG_BOLT_INSTRUMENTED "clang-bolt.inst" CACHE STRING
+"Name of BOLT-instrumented Clang binary")
   configure_lit_site_cfg(
 ${CMAKE_CURRENT_SOURCE_DIR}/bolt.lit.site.cfg.in
 ${CMAKE_CURRENT_BINARY_DIR}/bolt-fdata/lit.site.cfg
@@ -71,16 +73,37 @@ if(CLANG_BOLT_INSTRUMENT AND NOT LLVM_BUILD_INSTRUMENTED)
   add_lit_testsuite(generate-bolt-fdata "Generating BOLT profile for Clang"
 ${CMAKE_CURRENT_BINARY_DIR}/bolt-fdata/
 EXCLUDE_FROM_CHECK_ALL
-DEPENDS clang-instrumented clear-bolt-fdata
+DEPENDS clang-bolt-training-deps

[clang] [Clang][CMake] Support perf, LBR, and Instrument CLANG_BOLT options (PR #69133)

2023-11-07 Thread Amir Ayupov via cfe-commits 

https://github.com/aaupov updated 
https://github.com/llvm/llvm-project/pull/69133

>From 96e644279ccec1970c42cca89c05aac186b872e6 Mon Sep 17 00:00:00 2001
From: Amir Aupov 
Date: Mon, 16 Oct 2023 01:08:28 +0200
Subject: [PATCH 1/3] [Clang][CMake] Support perf, LBR, and Instrument
 CLANG_BOLT options

Split up and refactor CLANG_BOLT_INSTRUMENT into support for
perf no-LBR and perf with LBR profiling modes.

Differential Revision: https://reviews.llvm.org/D143617
---
 clang/CMakeLists.txt  | 44 -
 clang/cmake/caches/BOLT.cmake |  2 +-
 clang/utils/perf-training/CMakeLists.txt  | 29 -
 clang/utils/perf-training/bolt.lit.cfg| 53 +---
 .../utils/perf-training/bolt.lit.site.cfg.in  |  2 +
 clang/utils/perf-training/perf-helper.py  | 63 +++
 6 files changed, 167 insertions(+), 26 deletions(-)

diff --git a/clang/CMakeLists.txt b/clang/CMakeLists.txt
index 9b52c58be41e7f7..8f64d95cc394ffe 100644
--- a/clang/CMakeLists.txt
+++ b/clang/CMakeLists.txt
@@ -850,23 +850,38 @@ if (CLANG_ENABLE_BOOTSTRAP)
   endforeach()
 endif()
 
-if (CLANG_BOLT_INSTRUMENT AND NOT LLVM_BUILD_INSTRUMENTED)
+set(CLANG_BOLT "INSTRUMENT" CACHE STRING "Apply BOLT optimization to Clang. \
+  May be specified as Instrument or Perf or LBR to use a particular profiling \
+  mechanism.")
+string(TOUPPER "${CLANG_BOLT}" uppercase_CLANG_BOLT)
+
+if (CLANG_BOLT AND NOT LLVM_BUILD_INSTRUMENTED)
   set(CLANG_PATH ${LLVM_RUNTIME_OUTPUT_INTDIR}/clang)
-  set(CLANG_INSTRUMENTED ${CLANG_PATH}-bolt.inst)
+  set(CLANG_INSTRUMENTED 
${LLVM_RUNTIME_OUTPUT_INTDIR}/${CLANG_BOLT_INSTRUMENTED})
   set(BOLT_FDATA ${CMAKE_CURRENT_BINARY_DIR}/utils/perf-training/prof.fdata)
 
-  # Instrument clang with BOLT
-  add_custom_target(clang-instrumented
-DEPENDS ${CLANG_INSTRUMENTED}
-  )
-  add_custom_command(OUTPUT ${CLANG_INSTRUMENTED}
-DEPENDS clang llvm-bolt
-COMMAND llvm-bolt ${CLANG_PATH} -o ${CLANG_INSTRUMENTED}
-  -instrument --instrumentation-file-append-pid
-  --instrumentation-file=${BOLT_FDATA}
-COMMENT "Instrumenting clang binary with BOLT"
-VERBATIM
-  )
+  # Pass extra flag in no-LBR mode
+  if (uppercase_CLANG_BOLT STREQUAL "PERF")
+set(BOLT_NO_LBR "-nl")
+  endif()
+
+  if (uppercase_CLANG_BOLT STREQUAL "INSTRUMENT")
+# Instrument clang with BOLT
+add_custom_target(clang-instrumented
+  DEPENDS ${CLANG_INSTRUMENTED}
+)
+add_custom_command(OUTPUT ${CLANG_INSTRUMENTED}
+  DEPENDS clang llvm-bolt
+  COMMAND llvm-bolt ${CLANG_PATH} -o ${CLANG_INSTRUMENTED}
+-instrument --instrumentation-file-append-pid
+--instrumentation-file=${BOLT_FDATA}
+  COMMENT "Instrumenting clang binary with BOLT"
+  VERBATIM
+)
+add_custom_target(clang-bolt-training-deps DEPENDS clang-instrumented)
+  else() # perf or LBR
+add_custom_target(clang-bolt-training-deps DEPENDS clang)
+  endif()
 
   # Optimize original (pre-bolt) Clang using the collected profile
   set(CLANG_OPTIMIZED ${CMAKE_CURRENT_BINARY_DIR}/clang.bolt)
@@ -880,6 +895,7 @@ if (CLANG_BOLT_INSTRUMENT AND NOT LLVM_BUILD_INSTRUMENTED)
   -data ${BOLT_FDATA}
   -reorder-blocks=ext-tsp -reorder-functions=hfsort+ -split-functions
   -split-all-cold -split-eh -dyno-stats -icf=1 -use-gnu-stack
+  ${BOLT_NO_LBR}
 COMMAND ${CMAKE_COMMAND} -E rename ${CLANG_OPTIMIZED} $
 COMMENT "Optimizing Clang with BOLT"
 VERBATIM
diff --git a/clang/cmake/caches/BOLT.cmake b/clang/cmake/caches/BOLT.cmake
index 0442f73e5426ac7..eba2346b2f4ca12 100644
--- a/clang/cmake/caches/BOLT.cmake
+++ b/clang/cmake/caches/BOLT.cmake
@@ -1,5 +1,5 @@
 set(CMAKE_BUILD_TYPE Release CACHE STRING "")
-set(CLANG_BOLT_INSTRUMENT ON CACHE BOOL "")
+set(CLANG_BOLT "INSTRUMENT" CACHE STRING "")
 set(CMAKE_EXE_LINKER_FLAGS "-Wl,--emit-relocs,-znow" CACHE STRING "")
 
 set(LLVM_ENABLE_PROJECTS "bolt;clang" CACHE STRING "")
diff --git a/clang/utils/perf-training/CMakeLists.txt 
b/clang/utils/perf-training/CMakeLists.txt
index c6d51863fb1b5c2..48fbee62a8636d1 100644
--- a/clang/utils/perf-training/CMakeLists.txt
+++ b/clang/utils/perf-training/CMakeLists.txt
@@ -62,7 +62,9 @@ if(APPLE AND DTRACE AND NOT LLVM_TOOL_LLVM_DRIVER_BUILD)
 DEPENDS generate-dtrace-logs)
 endif()
 
-if(CLANG_BOLT_INSTRUMENT AND NOT LLVM_BUILD_INSTRUMENTED)
+if(CLANG_BOLT AND NOT LLVM_BUILD_INSTRUMENTED)
+  set(CLANG_BOLT_INSTRUMENTED "clang-bolt.inst" CACHE STRING
+"Name of BOLT-instrumented Clang binary")
   configure_lit_site_cfg(
 ${CMAKE_CURRENT_SOURCE_DIR}/bolt.lit.site.cfg.in
 ${CMAKE_CURRENT_BINARY_DIR}/bolt-fdata/lit.site.cfg
@@ -71,16 +73,37 @@ if(CLANG_BOLT_INSTRUMENT AND NOT LLVM_BUILD_INSTRUMENTED)
   add_lit_testsuite(generate-bolt-fdata "Generating BOLT profile for Clang"
 ${CMAKE_CURRENT_BINARY_DIR}/bolt-fdata/
 EXCLUDE_FROM_CHECK_ALL
-DEPENDS clang-instrumented clear-bolt-fdata
+DEPENDS clang-bolt-training-deps

[clang] [Clang][CMake] Support perf, LBR, and Instrument CLANG_BOLT options (PR #69133)

2023-11-07 Thread Amir Ayupov via cfe-commits 

aaupov wrote:

@petrhosek – thank you for a review!

https://github.com/llvm/llvm-project/pull/69133
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [Clang][CMake] Support perf, LBR, and Instrument CLANG_BOLT options (PR #69133)

2023-11-07 Thread Amir Ayupov via cfe-commits 

https://github.com/aaupov updated 
https://github.com/llvm/llvm-project/pull/69133

>From 96e644279ccec1970c42cca89c05aac186b872e6 Mon Sep 17 00:00:00 2001
From: Amir Aupov 
Date: Mon, 16 Oct 2023 01:08:28 +0200
Subject: [PATCH 1/2] [Clang][CMake] Support perf, LBR, and Instrument
 CLANG_BOLT options

Split up and refactor CLANG_BOLT_INSTRUMENT into support for
perf no-LBR and perf with LBR profiling modes.

Differential Revision: https://reviews.llvm.org/D143617
---
 clang/CMakeLists.txt  | 44 -
 clang/cmake/caches/BOLT.cmake |  2 +-
 clang/utils/perf-training/CMakeLists.txt  | 29 -
 clang/utils/perf-training/bolt.lit.cfg| 53 +---
 .../utils/perf-training/bolt.lit.site.cfg.in  |  2 +
 clang/utils/perf-training/perf-helper.py  | 63 +++
 6 files changed, 167 insertions(+), 26 deletions(-)

diff --git a/clang/CMakeLists.txt b/clang/CMakeLists.txt
index 9b52c58be41e7f7..8f64d95cc394ffe 100644
--- a/clang/CMakeLists.txt
+++ b/clang/CMakeLists.txt
@@ -850,23 +850,38 @@ if (CLANG_ENABLE_BOOTSTRAP)
   endforeach()
 endif()
 
-if (CLANG_BOLT_INSTRUMENT AND NOT LLVM_BUILD_INSTRUMENTED)
+set(CLANG_BOLT "INSTRUMENT" CACHE STRING "Apply BOLT optimization to Clang. \
+  May be specified as Instrument or Perf or LBR to use a particular profiling \
+  mechanism.")
+string(TOUPPER "${CLANG_BOLT}" uppercase_CLANG_BOLT)
+
+if (CLANG_BOLT AND NOT LLVM_BUILD_INSTRUMENTED)
   set(CLANG_PATH ${LLVM_RUNTIME_OUTPUT_INTDIR}/clang)
-  set(CLANG_INSTRUMENTED ${CLANG_PATH}-bolt.inst)
+  set(CLANG_INSTRUMENTED 
${LLVM_RUNTIME_OUTPUT_INTDIR}/${CLANG_BOLT_INSTRUMENTED})
   set(BOLT_FDATA ${CMAKE_CURRENT_BINARY_DIR}/utils/perf-training/prof.fdata)
 
-  # Instrument clang with BOLT
-  add_custom_target(clang-instrumented
-DEPENDS ${CLANG_INSTRUMENTED}
-  )
-  add_custom_command(OUTPUT ${CLANG_INSTRUMENTED}
-DEPENDS clang llvm-bolt
-COMMAND llvm-bolt ${CLANG_PATH} -o ${CLANG_INSTRUMENTED}
-  -instrument --instrumentation-file-append-pid
-  --instrumentation-file=${BOLT_FDATA}
-COMMENT "Instrumenting clang binary with BOLT"
-VERBATIM
-  )
+  # Pass extra flag in no-LBR mode
+  if (uppercase_CLANG_BOLT STREQUAL "PERF")
+set(BOLT_NO_LBR "-nl")
+  endif()
+
+  if (uppercase_CLANG_BOLT STREQUAL "INSTRUMENT")
+# Instrument clang with BOLT
+add_custom_target(clang-instrumented
+  DEPENDS ${CLANG_INSTRUMENTED}
+)
+add_custom_command(OUTPUT ${CLANG_INSTRUMENTED}
+  DEPENDS clang llvm-bolt
+  COMMAND llvm-bolt ${CLANG_PATH} -o ${CLANG_INSTRUMENTED}
+-instrument --instrumentation-file-append-pid
+--instrumentation-file=${BOLT_FDATA}
+  COMMENT "Instrumenting clang binary with BOLT"
+  VERBATIM
+)
+add_custom_target(clang-bolt-training-deps DEPENDS clang-instrumented)
+  else() # perf or LBR
+add_custom_target(clang-bolt-training-deps DEPENDS clang)
+  endif()
 
   # Optimize original (pre-bolt) Clang using the collected profile
   set(CLANG_OPTIMIZED ${CMAKE_CURRENT_BINARY_DIR}/clang.bolt)
@@ -880,6 +895,7 @@ if (CLANG_BOLT_INSTRUMENT AND NOT LLVM_BUILD_INSTRUMENTED)
   -data ${BOLT_FDATA}
   -reorder-blocks=ext-tsp -reorder-functions=hfsort+ -split-functions
   -split-all-cold -split-eh -dyno-stats -icf=1 -use-gnu-stack
+  ${BOLT_NO_LBR}
 COMMAND ${CMAKE_COMMAND} -E rename ${CLANG_OPTIMIZED} $
 COMMENT "Optimizing Clang with BOLT"
 VERBATIM
diff --git a/clang/cmake/caches/BOLT.cmake b/clang/cmake/caches/BOLT.cmake
index 0442f73e5426ac7..eba2346b2f4ca12 100644
--- a/clang/cmake/caches/BOLT.cmake
+++ b/clang/cmake/caches/BOLT.cmake
@@ -1,5 +1,5 @@
 set(CMAKE_BUILD_TYPE Release CACHE STRING "")
-set(CLANG_BOLT_INSTRUMENT ON CACHE BOOL "")
+set(CLANG_BOLT "INSTRUMENT" CACHE STRING "")
 set(CMAKE_EXE_LINKER_FLAGS "-Wl,--emit-relocs,-znow" CACHE STRING "")
 
 set(LLVM_ENABLE_PROJECTS "bolt;clang" CACHE STRING "")
diff --git a/clang/utils/perf-training/CMakeLists.txt 
b/clang/utils/perf-training/CMakeLists.txt
index c6d51863fb1b5c2..48fbee62a8636d1 100644
--- a/clang/utils/perf-training/CMakeLists.txt
+++ b/clang/utils/perf-training/CMakeLists.txt
@@ -62,7 +62,9 @@ if(APPLE AND DTRACE AND NOT LLVM_TOOL_LLVM_DRIVER_BUILD)
 DEPENDS generate-dtrace-logs)
 endif()
 
-if(CLANG_BOLT_INSTRUMENT AND NOT LLVM_BUILD_INSTRUMENTED)
+if(CLANG_BOLT AND NOT LLVM_BUILD_INSTRUMENTED)
+  set(CLANG_BOLT_INSTRUMENTED "clang-bolt.inst" CACHE STRING
+"Name of BOLT-instrumented Clang binary")
   configure_lit_site_cfg(
 ${CMAKE_CURRENT_SOURCE_DIR}/bolt.lit.site.cfg.in
 ${CMAKE_CURRENT_BINARY_DIR}/bolt-fdata/lit.site.cfg
@@ -71,16 +73,37 @@ if(CLANG_BOLT_INSTRUMENT AND NOT LLVM_BUILD_INSTRUMENTED)
   add_lit_testsuite(generate-bolt-fdata "Generating BOLT profile for Clang"
 ${CMAKE_CURRENT_BINARY_DIR}/bolt-fdata/
 EXCLUDE_FROM_CHECK_ALL
-DEPENDS clang-instrumented clear-bolt-fdata
+DEPENDS clang-bolt-training-deps

[clang] [Clang][CMake] Support perf, LBR, and Instrument CLANG_BOLT options (PR #69133)

2023-11-06 Thread Petr Hosek via cfe-commits 


@@ -850,23 +850,38 @@ if (CLANG_ENABLE_BOOTSTRAP)
   endforeach()
 endif()
 
-if (CLANG_BOLT_INSTRUMENT AND NOT LLVM_BUILD_INSTRUMENTED)
+set(CLANG_BOLT "INSTRUMENT" CACHE STRING "Apply BOLT optimization to Clang. \
+  May be specified as Instrument or Perf or LBR to use a particular profiling \
+  mechanism.")
+string(TOUPPER "${CLANG_BOLT}" uppercase_CLANG_BOLT)

petrhosek wrote:

This is a super minor nit, but you can reuse the same variable (it wouldn't 
actually change the cached value, the non-cached variable is going to shadow 
the cached one which is fine in this case).

https://github.com/llvm/llvm-project/pull/69133
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [Clang][CMake] Support perf, LBR, and Instrument CLANG_BOLT options (PR #69133)

2023-11-06 Thread Petr Hosek via cfe-commits 


@@ -67,6 +67,67 @@ def merge_fdata(args):
 return 0
 
 
+def perf(args):
+parser = argparse.ArgumentParser(
+prog="perf-helper perf", description="perf wrapper for BOLT profile 
collection"
+)
+parser.add_argument(
+"--lbr", action="store_true", help="Use perf with branch stacks"
+)
+parser.add_argument("cmd", nargs="*", help="")
+
+# Use python's arg parser to handle all leading option arguments, but pass
+# everything else through to perf
+first_cmd = next(arg for arg in args if not arg.startswith("--"))

petrhosek wrote:

You can use `parser.add_argument('cmd', nargs=argparse.REMAINDER)` to let 
`argparse` handle this for you.

https://github.com/llvm/llvm-project/pull/69133
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [Clang][CMake] Support perf, LBR, and Instrument CLANG_BOLT options (PR #69133)

2023-11-06 Thread Petr Hosek via cfe-commits 

https://github.com/petrhosek edited 
https://github.com/llvm/llvm-project/pull/69133
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [Clang][CMake] Support perf, LBR, and Instrument CLANG_BOLT options (PR #69133)

2023-11-06 Thread Petr Hosek via cfe-commits 


@@ -6,15 +6,52 @@ import lit.util
 import os
 import subprocess
 
-config.clang = os.path.realpath(lit.util.which('clang-bolt.inst', 
config.clang_tools_dir)).replace('\\', '/')
+clang_binary = "clang"
+perf_wrapper = ""
+if config.clang_bolt_mode.lower() == "instrument":
+clang_binary = config.clang_bolt_name
+else:  # perf or LBR
+perf_wrapper = "%s %s/perf-helper.py perf" % (

petrhosek wrote:

Please use Python f-string.

https://github.com/llvm/llvm-project/pull/69133
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [Clang][CMake] Support perf, LBR, and Instrument CLANG_BOLT options (PR #69133)

2023-10-15 Thread Amir Ayupov via cfe-commits 

https://github.com/aaupov updated 
https://github.com/llvm/llvm-project/pull/69133

>From 96e644279ccec1970c42cca89c05aac186b872e6 Mon Sep 17 00:00:00 2001
From: Amir Aupov 
Date: Mon, 16 Oct 2023 01:08:28 +0200
Subject: [PATCH] [Clang][CMake] Support perf, LBR, and Instrument CLANG_BOLT
 options

Split up and refactor CLANG_BOLT_INSTRUMENT into support for
perf no-LBR and perf with LBR profiling modes.

Differential Revision: https://reviews.llvm.org/D143617
---
 clang/CMakeLists.txt  | 44 -
 clang/cmake/caches/BOLT.cmake |  2 +-
 clang/utils/perf-training/CMakeLists.txt  | 29 -
 clang/utils/perf-training/bolt.lit.cfg| 53 +---
 .../utils/perf-training/bolt.lit.site.cfg.in  |  2 +
 clang/utils/perf-training/perf-helper.py  | 63 +++
 6 files changed, 167 insertions(+), 26 deletions(-)

diff --git a/clang/CMakeLists.txt b/clang/CMakeLists.txt
index 9b52c58be41e7f7..8f64d95cc394ffe 100644
--- a/clang/CMakeLists.txt
+++ b/clang/CMakeLists.txt
@@ -850,23 +850,38 @@ if (CLANG_ENABLE_BOOTSTRAP)
   endforeach()
 endif()
 
-if (CLANG_BOLT_INSTRUMENT AND NOT LLVM_BUILD_INSTRUMENTED)
+set(CLANG_BOLT "INSTRUMENT" CACHE STRING "Apply BOLT optimization to Clang. \
+  May be specified as Instrument or Perf or LBR to use a particular profiling \
+  mechanism.")
+string(TOUPPER "${CLANG_BOLT}" uppercase_CLANG_BOLT)
+
+if (CLANG_BOLT AND NOT LLVM_BUILD_INSTRUMENTED)
   set(CLANG_PATH ${LLVM_RUNTIME_OUTPUT_INTDIR}/clang)
-  set(CLANG_INSTRUMENTED ${CLANG_PATH}-bolt.inst)
+  set(CLANG_INSTRUMENTED 
${LLVM_RUNTIME_OUTPUT_INTDIR}/${CLANG_BOLT_INSTRUMENTED})
   set(BOLT_FDATA ${CMAKE_CURRENT_BINARY_DIR}/utils/perf-training/prof.fdata)
 
-  # Instrument clang with BOLT
-  add_custom_target(clang-instrumented
-DEPENDS ${CLANG_INSTRUMENTED}
-  )
-  add_custom_command(OUTPUT ${CLANG_INSTRUMENTED}
-DEPENDS clang llvm-bolt
-COMMAND llvm-bolt ${CLANG_PATH} -o ${CLANG_INSTRUMENTED}
-  -instrument --instrumentation-file-append-pid
-  --instrumentation-file=${BOLT_FDATA}
-COMMENT "Instrumenting clang binary with BOLT"
-VERBATIM
-  )
+  # Pass extra flag in no-LBR mode
+  if (uppercase_CLANG_BOLT STREQUAL "PERF")
+set(BOLT_NO_LBR "-nl")
+  endif()
+
+  if (uppercase_CLANG_BOLT STREQUAL "INSTRUMENT")
+# Instrument clang with BOLT
+add_custom_target(clang-instrumented
+  DEPENDS ${CLANG_INSTRUMENTED}
+)
+add_custom_command(OUTPUT ${CLANG_INSTRUMENTED}
+  DEPENDS clang llvm-bolt
+  COMMAND llvm-bolt ${CLANG_PATH} -o ${CLANG_INSTRUMENTED}
+-instrument --instrumentation-file-append-pid
+--instrumentation-file=${BOLT_FDATA}
+  COMMENT "Instrumenting clang binary with BOLT"
+  VERBATIM
+)
+add_custom_target(clang-bolt-training-deps DEPENDS clang-instrumented)
+  else() # perf or LBR
+add_custom_target(clang-bolt-training-deps DEPENDS clang)
+  endif()
 
   # Optimize original (pre-bolt) Clang using the collected profile
   set(CLANG_OPTIMIZED ${CMAKE_CURRENT_BINARY_DIR}/clang.bolt)
@@ -880,6 +895,7 @@ if (CLANG_BOLT_INSTRUMENT AND NOT LLVM_BUILD_INSTRUMENTED)
   -data ${BOLT_FDATA}
   -reorder-blocks=ext-tsp -reorder-functions=hfsort+ -split-functions
   -split-all-cold -split-eh -dyno-stats -icf=1 -use-gnu-stack
+  ${BOLT_NO_LBR}
 COMMAND ${CMAKE_COMMAND} -E rename ${CLANG_OPTIMIZED} $
 COMMENT "Optimizing Clang with BOLT"
 VERBATIM
diff --git a/clang/cmake/caches/BOLT.cmake b/clang/cmake/caches/BOLT.cmake
index 0442f73e5426ac7..eba2346b2f4ca12 100644
--- a/clang/cmake/caches/BOLT.cmake
+++ b/clang/cmake/caches/BOLT.cmake
@@ -1,5 +1,5 @@
 set(CMAKE_BUILD_TYPE Release CACHE STRING "")
-set(CLANG_BOLT_INSTRUMENT ON CACHE BOOL "")
+set(CLANG_BOLT "INSTRUMENT" CACHE STRING "")
 set(CMAKE_EXE_LINKER_FLAGS "-Wl,--emit-relocs,-znow" CACHE STRING "")
 
 set(LLVM_ENABLE_PROJECTS "bolt;clang" CACHE STRING "")
diff --git a/clang/utils/perf-training/CMakeLists.txt 
b/clang/utils/perf-training/CMakeLists.txt
index c6d51863fb1b5c2..48fbee62a8636d1 100644
--- a/clang/utils/perf-training/CMakeLists.txt
+++ b/clang/utils/perf-training/CMakeLists.txt
@@ -62,7 +62,9 @@ if(APPLE AND DTRACE AND NOT LLVM_TOOL_LLVM_DRIVER_BUILD)
 DEPENDS generate-dtrace-logs)
 endif()
 
-if(CLANG_BOLT_INSTRUMENT AND NOT LLVM_BUILD_INSTRUMENTED)
+if(CLANG_BOLT AND NOT LLVM_BUILD_INSTRUMENTED)
+  set(CLANG_BOLT_INSTRUMENTED "clang-bolt.inst" CACHE STRING
+"Name of BOLT-instrumented Clang binary")
   configure_lit_site_cfg(
 ${CMAKE_CURRENT_SOURCE_DIR}/bolt.lit.site.cfg.in
 ${CMAKE_CURRENT_BINARY_DIR}/bolt-fdata/lit.site.cfg
@@ -71,16 +73,37 @@ if(CLANG_BOLT_INSTRUMENT AND NOT LLVM_BUILD_INSTRUMENTED)
   add_lit_testsuite(generate-bolt-fdata "Generating BOLT profile for Clang"
 ${CMAKE_CURRENT_BINARY_DIR}/bolt-fdata/
 EXCLUDE_FROM_CHECK_ALL
-DEPENDS clang-instrumented clear-bolt-fdata
+DEPENDS clang-bolt-training-deps

[clang] [Clang][CMake] Support perf, LBR, and Instrument CLANG_BOLT options (PR #69133)

2023-10-15 Thread via cfe-commits 

github-actions[bot] wrote:




:warning: Python code formatter, darker found issues in your code. :warning:



You can test this locally with the following command:


``bash
darker --check --diff -r 
7f881a2abe2c3eceeae0272fc41ba0a237770450..3987088f1b6f6b895f7563ad70c97074b0d4ded0
 clang/utils/perf-training/perf-helper.py
``





View the diff from darker here.


``diff
--- perf-helper.py  2023-10-15 23:09:26.00 +
+++ perf-helper.py  2023-10-15 23:14:14.499925 +
@@ -109,13 +109,11 @@
 description="perf2bolt conversion wrapper for perf.data files",
 )
 parser.add_argument("bolt", help="Path to llvm-bolt")
 parser.add_argument("path", help="Path containing perf.data files")
 parser.add_argument("binary", help="Input binary")
-parser.add_argument(
-"--lbr", action="store_true", help="Use LBR perf2bolt mode"
-)
+parser.add_argument("--lbr", action="store_true", help="Use LBR perf2bolt 
mode")
 opts = parser.parse_args(args)
 
 p2b_args = [
 opts.bolt,
 opts.binary,

``




https://github.com/llvm/llvm-project/pull/69133
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [Clang][CMake] Support perf, LBR, and Instrument CLANG_BOLT options (PR #69133)

2023-10-15 Thread via cfe-commits 

llvmbot wrote:




@llvm/pr-subscribers-clang

Author: Amir Ayupov (aaupov)


Changes

Split up and refactor CLANG_BOLT_INSTRUMENT into support for
perf no-LBR and perf with LBR profiling modes.

Differential Revision: https://reviews.llvm.org/D143617


---
Full diff: https://github.com/llvm/llvm-project/pull/69133.diff


6 Files Affected:

- (modified) clang/CMakeLists.txt (+30-14) 
- (modified) clang/cmake/caches/BOLT.cmake (+1-1) 
- (modified) clang/utils/perf-training/CMakeLists.txt (+26-3) 
- (modified) clang/utils/perf-training/bolt.lit.cfg (+45-8) 
- (modified) clang/utils/perf-training/bolt.lit.site.cfg.in (+2) 
- (modified) clang/utils/perf-training/perf-helper.py (+65) 


``diff
diff --git a/clang/CMakeLists.txt b/clang/CMakeLists.txt
index 9b52c58be41e7f7..8f64d95cc394ffe 100644
--- a/clang/CMakeLists.txt
+++ b/clang/CMakeLists.txt
@@ -850,23 +850,38 @@ if (CLANG_ENABLE_BOOTSTRAP)
   endforeach()
 endif()
 
-if (CLANG_BOLT_INSTRUMENT AND NOT LLVM_BUILD_INSTRUMENTED)
+set(CLANG_BOLT "INSTRUMENT" CACHE STRING "Apply BOLT optimization to Clang. \
+  May be specified as Instrument or Perf or LBR to use a particular profiling \
+  mechanism.")
+string(TOUPPER "${CLANG_BOLT}" uppercase_CLANG_BOLT)
+
+if (CLANG_BOLT AND NOT LLVM_BUILD_INSTRUMENTED)
   set(CLANG_PATH ${LLVM_RUNTIME_OUTPUT_INTDIR}/clang)
-  set(CLANG_INSTRUMENTED ${CLANG_PATH}-bolt.inst)
+  set(CLANG_INSTRUMENTED 
${LLVM_RUNTIME_OUTPUT_INTDIR}/${CLANG_BOLT_INSTRUMENTED})
   set(BOLT_FDATA ${CMAKE_CURRENT_BINARY_DIR}/utils/perf-training/prof.fdata)
 
-  # Instrument clang with BOLT
-  add_custom_target(clang-instrumented
-DEPENDS ${CLANG_INSTRUMENTED}
-  )
-  add_custom_command(OUTPUT ${CLANG_INSTRUMENTED}
-DEPENDS clang llvm-bolt
-COMMAND llvm-bolt ${CLANG_PATH} -o ${CLANG_INSTRUMENTED}
-  -instrument --instrumentation-file-append-pid
-  --instrumentation-file=${BOLT_FDATA}
-COMMENT "Instrumenting clang binary with BOLT"
-VERBATIM
-  )
+  # Pass extra flag in no-LBR mode
+  if (uppercase_CLANG_BOLT STREQUAL "PERF")
+set(BOLT_NO_LBR "-nl")
+  endif()
+
+  if (uppercase_CLANG_BOLT STREQUAL "INSTRUMENT")
+# Instrument clang with BOLT
+add_custom_target(clang-instrumented
+  DEPENDS ${CLANG_INSTRUMENTED}
+)
+add_custom_command(OUTPUT ${CLANG_INSTRUMENTED}
+  DEPENDS clang llvm-bolt
+  COMMAND llvm-bolt ${CLANG_PATH} -o ${CLANG_INSTRUMENTED}
+-instrument --instrumentation-file-append-pid
+--instrumentation-file=${BOLT_FDATA}
+  COMMENT "Instrumenting clang binary with BOLT"
+  VERBATIM
+)
+add_custom_target(clang-bolt-training-deps DEPENDS clang-instrumented)
+  else() # perf or LBR
+add_custom_target(clang-bolt-training-deps DEPENDS clang)
+  endif()
 
   # Optimize original (pre-bolt) Clang using the collected profile
   set(CLANG_OPTIMIZED ${CMAKE_CURRENT_BINARY_DIR}/clang.bolt)
@@ -880,6 +895,7 @@ if (CLANG_BOLT_INSTRUMENT AND NOT LLVM_BUILD_INSTRUMENTED)
   -data ${BOLT_FDATA}
   -reorder-blocks=ext-tsp -reorder-functions=hfsort+ -split-functions
   -split-all-cold -split-eh -dyno-stats -icf=1 -use-gnu-stack
+  ${BOLT_NO_LBR}
 COMMAND ${CMAKE_COMMAND} -E rename ${CLANG_OPTIMIZED} $
 COMMENT "Optimizing Clang with BOLT"
 VERBATIM
diff --git a/clang/cmake/caches/BOLT.cmake b/clang/cmake/caches/BOLT.cmake
index 0442f73e5426ac7..eba2346b2f4ca12 100644
--- a/clang/cmake/caches/BOLT.cmake
+++ b/clang/cmake/caches/BOLT.cmake
@@ -1,5 +1,5 @@
 set(CMAKE_BUILD_TYPE Release CACHE STRING "")
-set(CLANG_BOLT_INSTRUMENT ON CACHE BOOL "")
+set(CLANG_BOLT "INSTRUMENT" CACHE STRING "")
 set(CMAKE_EXE_LINKER_FLAGS "-Wl,--emit-relocs,-znow" CACHE STRING "")
 
 set(LLVM_ENABLE_PROJECTS "bolt;clang" CACHE STRING "")
diff --git a/clang/utils/perf-training/CMakeLists.txt 
b/clang/utils/perf-training/CMakeLists.txt
index c6d51863fb1b5c2..48fbee62a8636d1 100644
--- a/clang/utils/perf-training/CMakeLists.txt
+++ b/clang/utils/perf-training/CMakeLists.txt
@@ -62,7 +62,9 @@ if(APPLE AND DTRACE AND NOT LLVM_TOOL_LLVM_DRIVER_BUILD)
 DEPENDS generate-dtrace-logs)
 endif()
 
-if(CLANG_BOLT_INSTRUMENT AND NOT LLVM_BUILD_INSTRUMENTED)
+if(CLANG_BOLT AND NOT LLVM_BUILD_INSTRUMENTED)
+  set(CLANG_BOLT_INSTRUMENTED "clang-bolt.inst" CACHE STRING
+"Name of BOLT-instrumented Clang binary")
   configure_lit_site_cfg(
 ${CMAKE_CURRENT_SOURCE_DIR}/bolt.lit.site.cfg.in
 ${CMAKE_CURRENT_BINARY_DIR}/bolt-fdata/lit.site.cfg
@@ -71,16 +73,37 @@ if(CLANG_BOLT_INSTRUMENT AND NOT LLVM_BUILD_INSTRUMENTED)
   add_lit_testsuite(generate-bolt-fdata "Generating BOLT profile for Clang"
 ${CMAKE_CURRENT_BINARY_DIR}/bolt-fdata/
 EXCLUDE_FROM_CHECK_ALL
-DEPENDS clang-instrumented clear-bolt-fdata
+DEPENDS clang-bolt-training-deps clear-bolt-fdata clear-perf-data
 )
 
   add_custom_target(clear-bolt-fdata
 COMMAND "${Python3_EXECUTABLE}" ${CMAKE_CURRENT_SOURCE_DIR}/perf-helper.py 
clean ${CMAKE_CURRENT_BINARY_DIR} fdata

[clang] [Clang][CMake] Support perf, LBR, and Instrument CLANG_BOLT options (PR #69133)

2023-10-15 Thread Amir Ayupov via cfe-commits 

https://github.com/aaupov created 
https://github.com/llvm/llvm-project/pull/69133

Split up and refactor CLANG_BOLT_INSTRUMENT into support for
perf no-LBR and perf with LBR profiling modes.

Differential Revision: https://reviews.llvm.org/D143617


>From 3987088f1b6f6b895f7563ad70c97074b0d4ded0 Mon Sep 17 00:00:00 2001
From: Amir Aupov 
Date: Mon, 16 Oct 2023 01:08:28 +0200
Subject: [PATCH] [Clang][CMake] Support perf, LBR, and Instrument CLANG_BOLT
 options

Split up and refactor CLANG_BOLT_INSTRUMENT into support for
perf no-LBR and perf with LBR profiling modes.

Differential Revision: https://reviews.llvm.org/D143617
---
 clang/CMakeLists.txt  | 44 +
 clang/cmake/caches/BOLT.cmake |  2 +-
 clang/utils/perf-training/CMakeLists.txt  | 29 -
 clang/utils/perf-training/bolt.lit.cfg| 53 ---
 .../utils/perf-training/bolt.lit.site.cfg.in  |  2 +
 clang/utils/perf-training/perf-helper.py  | 65 +++
 6 files changed, 169 insertions(+), 26 deletions(-)

diff --git a/clang/CMakeLists.txt b/clang/CMakeLists.txt
index 9b52c58be41e7f7..8f64d95cc394ffe 100644
--- a/clang/CMakeLists.txt
+++ b/clang/CMakeLists.txt
@@ -850,23 +850,38 @@ if (CLANG_ENABLE_BOOTSTRAP)
   endforeach()
 endif()
 
-if (CLANG_BOLT_INSTRUMENT AND NOT LLVM_BUILD_INSTRUMENTED)
+set(CLANG_BOLT "INSTRUMENT" CACHE STRING "Apply BOLT optimization to Clang. \
+  May be specified as Instrument or Perf or LBR to use a particular profiling \
+  mechanism.")
+string(TOUPPER "${CLANG_BOLT}" uppercase_CLANG_BOLT)
+
+if (CLANG_BOLT AND NOT LLVM_BUILD_INSTRUMENTED)
   set(CLANG_PATH ${LLVM_RUNTIME_OUTPUT_INTDIR}/clang)
-  set(CLANG_INSTRUMENTED ${CLANG_PATH}-bolt.inst)
+  set(CLANG_INSTRUMENTED 
${LLVM_RUNTIME_OUTPUT_INTDIR}/${CLANG_BOLT_INSTRUMENTED})
   set(BOLT_FDATA ${CMAKE_CURRENT_BINARY_DIR}/utils/perf-training/prof.fdata)
 
-  # Instrument clang with BOLT
-  add_custom_target(clang-instrumented
-DEPENDS ${CLANG_INSTRUMENTED}
-  )
-  add_custom_command(OUTPUT ${CLANG_INSTRUMENTED}
-DEPENDS clang llvm-bolt
-COMMAND llvm-bolt ${CLANG_PATH} -o ${CLANG_INSTRUMENTED}
-  -instrument --instrumentation-file-append-pid
-  --instrumentation-file=${BOLT_FDATA}
-COMMENT "Instrumenting clang binary with BOLT"
-VERBATIM
-  )
+  # Pass extra flag in no-LBR mode
+  if (uppercase_CLANG_BOLT STREQUAL "PERF")
+set(BOLT_NO_LBR "-nl")
+  endif()
+
+  if (uppercase_CLANG_BOLT STREQUAL "INSTRUMENT")
+# Instrument clang with BOLT
+add_custom_target(clang-instrumented
+  DEPENDS ${CLANG_INSTRUMENTED}
+)
+add_custom_command(OUTPUT ${CLANG_INSTRUMENTED}
+  DEPENDS clang llvm-bolt
+  COMMAND llvm-bolt ${CLANG_PATH} -o ${CLANG_INSTRUMENTED}
+-instrument --instrumentation-file-append-pid
+--instrumentation-file=${BOLT_FDATA}
+  COMMENT "Instrumenting clang binary with BOLT"
+  VERBATIM
+)
+add_custom_target(clang-bolt-training-deps DEPENDS clang-instrumented)
+  else() # perf or LBR
+add_custom_target(clang-bolt-training-deps DEPENDS clang)
+  endif()
 
   # Optimize original (pre-bolt) Clang using the collected profile
   set(CLANG_OPTIMIZED ${CMAKE_CURRENT_BINARY_DIR}/clang.bolt)
@@ -880,6 +895,7 @@ if (CLANG_BOLT_INSTRUMENT AND NOT LLVM_BUILD_INSTRUMENTED)
   -data ${BOLT_FDATA}
   -reorder-blocks=ext-tsp -reorder-functions=hfsort+ -split-functions
   -split-all-cold -split-eh -dyno-stats -icf=1 -use-gnu-stack
+  ${BOLT_NO_LBR}
 COMMAND ${CMAKE_COMMAND} -E rename ${CLANG_OPTIMIZED} $
 COMMENT "Optimizing Clang with BOLT"
 VERBATIM
diff --git a/clang/cmake/caches/BOLT.cmake b/clang/cmake/caches/BOLT.cmake
index 0442f73e5426ac7..eba2346b2f4ca12 100644
--- a/clang/cmake/caches/BOLT.cmake
+++ b/clang/cmake/caches/BOLT.cmake
@@ -1,5 +1,5 @@
 set(CMAKE_BUILD_TYPE Release CACHE STRING "")
-set(CLANG_BOLT_INSTRUMENT ON CACHE BOOL "")
+set(CLANG_BOLT "INSTRUMENT" CACHE STRING "")
 set(CMAKE_EXE_LINKER_FLAGS "-Wl,--emit-relocs,-znow" CACHE STRING "")
 
 set(LLVM_ENABLE_PROJECTS "bolt;clang" CACHE STRING "")
diff --git a/clang/utils/perf-training/CMakeLists.txt 
b/clang/utils/perf-training/CMakeLists.txt
index c6d51863fb1b5c2..48fbee62a8636d1 100644
--- a/clang/utils/perf-training/CMakeLists.txt
+++ b/clang/utils/perf-training/CMakeLists.txt
@@ -62,7 +62,9 @@ if(APPLE AND DTRACE AND NOT LLVM_TOOL_LLVM_DRIVER_BUILD)
 DEPENDS generate-dtrace-logs)
 endif()
 
-if(CLANG_BOLT_INSTRUMENT AND NOT LLVM_BUILD_INSTRUMENTED)
+if(CLANG_BOLT AND NOT LLVM_BUILD_INSTRUMENTED)
+  set(CLANG_BOLT_INSTRUMENTED "clang-bolt.inst" CACHE STRING
+"Name of BOLT-instrumented Clang binary")
   configure_lit_site_cfg(
 ${CMAKE_CURRENT_SOURCE_DIR}/bolt.lit.site.cfg.in
 ${CMAKE_CURRENT_BINARY_DIR}/bolt-fdata/lit.site.cfg
@@ -71,16 +73,37 @@ if(CLANG_BOLT_INSTRUMENT AND NOT LLVM_BUILD_INSTRUMENTED)
   add_lit_testsuite(generate-bolt-fdata "Generating BOLT profile for Clang"