[llvm-branch-commits] [runtimes] Allow building against an installed LLVM tree (PR #86209)

2024-06-25 Thread Louis Dionne via llvm-branch-commits 

https://github.com/ldionne approved this pull request.

LGTM but I think @petrhosek should also have a look.

https://github.com/llvm/llvm-project/pull/86209
___
llvm-branch-commits mailing list
llvm-branch-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits

[llvm-branch-commits] [runtimes] Allow building against an installed LLVM tree (PR #86209)

2024-04-12 Thread Alexander Richardson via llvm-branch-commits 


@@ -218,6 +218,22 @@ foreach(entry ${runtimes})
 endforeach()
 
 if(LLVM_INCLUDE_TESTS)
+  # Add lit if needed before adding any runtimes since their CMake tests
+  # configuration might depend on lit being present.
+  if (NOT HAVE_LLVM_LIT)
+# If built by manually invoking cmake on this directory, we don't have
+# llvm-lit. If invoked via llvm/runtimes, the toplevel llvm cmake
+# invocation already generated the llvm-lit script.
+set(LLVM_LIT_OUTPUT_DIR ${CMAKE_CURRENT_BINARY_DIR}/bin)
+add_subdirectory(${LLVM_MAIN_SRC_DIR}/utils/llvm-lit
+ ${CMAKE_CURRENT_BINARY_DIR}/llvm-lit)
+# Ensure that the testsuites use the local lit rather than
+# LLVM_INSTALL_DIR/bin/llvm-lit (which may not exist if LLVM_BINARY_DIR
+# points at an installed LLVM tree rather than a build tree.
+get_llvm_lit_path(_base_dir _file_name)
+set(LLVM_EXTERNAL_LIT "${_base_dir}/${_file_name}" CACHE STRING "Command 
used to spawn lit" FORCE)

arichardson wrote:

Sorry for the delay here, was waiting to land the dependent PRs. Updated now.

https://github.com/llvm/llvm-project/pull/86209
___
llvm-branch-commits mailing list
llvm-branch-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits

[llvm-branch-commits] [runtimes] Allow building against an installed LLVM tree (PR #86209)

2024-04-12 Thread Alexander Richardson via llvm-branch-commits 

https://github.com/arichardson updated 
https://github.com/llvm/llvm-project/pull/86209


___
llvm-branch-commits mailing list
llvm-branch-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits

[llvm-branch-commits] [runtimes] Allow building against an installed LLVM tree (PR #86209)

2024-04-12 Thread Alexander Richardson via llvm-branch-commits 

https://github.com/arichardson updated 
https://github.com/llvm/llvm-project/pull/86209


___
llvm-branch-commits mailing list
llvm-branch-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits

[llvm-branch-commits] [runtimes] Allow building against an installed LLVM tree (PR #86209)

2024-03-26 Thread Louis Dionne via llvm-branch-commits 


@@ -218,6 +218,22 @@ foreach(entry ${runtimes})
 endforeach()
 
 if(LLVM_INCLUDE_TESTS)
+  # Add lit if needed before adding any runtimes since their CMake tests
+  # configuration might depend on lit being present.
+  if (NOT HAVE_LLVM_LIT)
+# If built by manually invoking cmake on this directory, we don't have
+# llvm-lit. If invoked via llvm/runtimes, the toplevel llvm cmake
+# invocation already generated the llvm-lit script.
+set(LLVM_LIT_OUTPUT_DIR ${CMAKE_CURRENT_BINARY_DIR}/bin)
+add_subdirectory(${LLVM_MAIN_SRC_DIR}/utils/llvm-lit
+ ${CMAKE_CURRENT_BINARY_DIR}/llvm-lit)
+# Ensure that the testsuites use the local lit rather than
+# LLVM_INSTALL_DIR/bin/llvm-lit (which may not exist if LLVM_BINARY_DIR
+# points at an installed LLVM tree rather than a build tree.
+get_llvm_lit_path(_base_dir _file_name)
+set(LLVM_EXTERNAL_LIT "${_base_dir}/${_file_name}" CACHE STRING "Command 
used to spawn lit" FORCE)

ldionne wrote:

Ah. I missed the distinction between `llvm/runtimes` and just `runtimes`. 
This could be made clearer by saying something like

```
If built with the runtimes build (rooted at runtimes/CMakeLists.txt), we don't 
have
llvm-lit. If built with the bootstrapping build (rooted at 
llvm/CMakeLists.txt), the
top-level llvm CMake invocation already generated the llvm-lit script.
```

https://github.com/llvm/llvm-project/pull/86209
___
llvm-branch-commits mailing list
llvm-branch-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits

[llvm-branch-commits] [runtimes] Allow building against an installed LLVM tree (PR #86209)

2024-03-26 Thread Alexander Richardson via llvm-branch-commits 


@@ -218,6 +218,22 @@ foreach(entry ${runtimes})
 endforeach()
 
 if(LLVM_INCLUDE_TESTS)
+  # Add lit if needed before adding any runtimes since their CMake tests
+  # configuration might depend on lit being present.
+  if (NOT HAVE_LLVM_LIT)
+# If built by manually invoking cmake on this directory, we don't have
+# llvm-lit. If invoked via llvm/runtimes, the toplevel llvm cmake
+# invocation already generated the llvm-lit script.
+set(LLVM_LIT_OUTPUT_DIR ${CMAKE_CURRENT_BINARY_DIR}/bin)
+add_subdirectory(${LLVM_MAIN_SRC_DIR}/utils/llvm-lit
+ ${CMAKE_CURRENT_BINARY_DIR}/llvm-lit)
+# Ensure that the testsuites use the local lit rather than
+# LLVM_INSTALL_DIR/bin/llvm-lit (which may not exist if LLVM_BINARY_DIR
+# points at an installed LLVM tree rather than a build tree.
+get_llvm_lit_path(_base_dir _file_name)
+set(LLVM_EXTERNAL_LIT "${_base_dir}/${_file_name}" CACHE STRING "Command 
used to spawn lit" FORCE)

arichardson wrote:

This should be explained by the existing comment:

> If built by manually invoking cmake on this directory, we don't have
> llvm-lit. If invoked via llvm/runtimes, the toplevel llvm cmake
> invocation already generated the llvm-lit script.

https://github.com/llvm/llvm-project/pull/86209
___
llvm-branch-commits mailing list
llvm-branch-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits

[llvm-branch-commits] [runtimes] Allow building against an installed LLVM tree (PR #86209)

2024-03-26 Thread Louis Dionne via llvm-branch-commits 


@@ -218,6 +218,22 @@ foreach(entry ${runtimes})
 endforeach()
 
 if(LLVM_INCLUDE_TESTS)
+  # Add lit if needed before adding any runtimes since their CMake tests
+  # configuration might depend on lit being present.
+  if (NOT HAVE_LLVM_LIT)
+# If built by manually invoking cmake on this directory, we don't have
+# llvm-lit. If invoked via llvm/runtimes, the toplevel llvm cmake
+# invocation already generated the llvm-lit script.
+set(LLVM_LIT_OUTPUT_DIR ${CMAKE_CURRENT_BINARY_DIR}/bin)
+add_subdirectory(${LLVM_MAIN_SRC_DIR}/utils/llvm-lit
+ ${CMAKE_CURRENT_BINARY_DIR}/llvm-lit)
+# Ensure that the testsuites use the local lit rather than
+# LLVM_INSTALL_DIR/bin/llvm-lit (which may not exist if LLVM_BINARY_DIR
+# points at an installed LLVM tree rather than a build tree.
+get_llvm_lit_path(_base_dir _file_name)
+set(LLVM_EXTERNAL_LIT "${_base_dir}/${_file_name}" CACHE STRING "Command 
used to spawn lit" FORCE)

ldionne wrote:

I don't understand. We *must* have `lit` available since we are in the 
`runtimes/CMakeLists.txt`? I actually don't understand why the `if (NOT 
HAVE_LLVM_LIT)` branch is ever taken. Can you shed some light on this?

https://github.com/llvm/llvm-project/pull/86209
___
llvm-branch-commits mailing list
llvm-branch-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits

[llvm-branch-commits] [runtimes] Allow building against an installed LLVM tree (PR #86209)

2024-03-21 Thread via llvm-branch-commits 

llvmbot wrote:




@llvm/pr-subscribers-compiler-rt-sanitizer

Author: Alexander Richardson (arichardson)


Changes

I am currently trying to test the LLVM runtimes (including compiler-rt)
against an installed LLVM tree rather than a build tree (since that is
no longer available). Currently, the runtimes build of compiler-rt assumes
that LLVM_BINARY_DIR is writable since it uses configure_file() to write
there during the CMake configure stage. Instead, generate this file inside
CMAKE_CURRENT_BINARY_DIR, which will match LLVM_BINARY_DIR when invoked
from llvm/runtimes/CMakeLists.txt.

I also needed to make a minor change to the hwasan tests: hwasan_symbolize
was previously found in the LLVM_BINARY_DIR, but since it is generated as
part of the compiler-rt build it is now inside the CMake build directory
instead. I fixed this by passing the output directory to lit as
config.compiler_rt_bindir and using llvm_config.add_tool_substitutions().

For testing that we no longer write to the LLVM install directory as
part of testing or configuration, I created a read-only bind mount and
configured the runtimes builds as follows:
```
$ sudo mount --bind --read-only ~/llvm-install /tmp/upstream-llvm-readonly
$ cmake -DCMAKE_BUILD_TYPE=Debug \
  -DCMAKE_C_COMPILER=/tmp/upstream-llvm-readonly/bin/clang \
  -DCMAKE_CXX_COMPILER=/tmp/upstream-llvm-readonly/bin/clang++ \
  -DLLVM_INCLUDE_TESTS=TRUE -DLLVM_ENABLE_ASSERTIONS=TRUE \
  -DCOMPILER_RT_INCLUDE_TESTS=TRUE -DCOMPILER_RT_DEBUG=OFF \
  -DLLVM_ENABLE_RUNTIMES=compiler-rt \
  -DLLVM_BINARY_DIR=/tmp/upstream-llvm-readonly \
  -G Ninja -S ~/upstream-llvm-project/runtimes \
  -B ~/upstream-llvm-project/runtimes/cmake-build-debug-llvm-git
```


---
Full diff: https://github.com/llvm/llvm-project/pull/86209.diff


4 Files Affected:

- (modified) compiler-rt/cmake/Modules/AddCompilerRT.cmake (+1) 
- (modified) compiler-rt/test/hwasan/lit.cfg.py (+7) 
- (modified) compiler-rt/test/lit.common.configured.in (+1) 
- (modified) runtimes/CMakeLists.txt (+18-10) 


``diff
diff --git a/compiler-rt/cmake/Modules/AddCompilerRT.cmake 
b/compiler-rt/cmake/Modules/AddCompilerRT.cmake
index 567b123b7abcac..4c3752d14a4789 100644
--- a/compiler-rt/cmake/Modules/AddCompilerRT.cmake
+++ b/compiler-rt/cmake/Modules/AddCompilerRT.cmake
@@ -769,6 +769,7 @@ function(configure_compiler_rt_lit_site_cfg input output)
 
   string(REPLACE ${CMAKE_CFG_INTDIR} ${LLVM_BUILD_MODE} 
COMPILER_RT_RESOLVED_TEST_COMPILER ${COMPILER_RT_TEST_COMPILER})
   string(REPLACE ${CMAKE_CFG_INTDIR} ${LLVM_BUILD_MODE} 
COMPILER_RT_RESOLVED_OUTPUT_DIR ${COMPILER_RT_OUTPUT_DIR})
+  string(REPLACE ${CMAKE_CFG_INTDIR} ${LLVM_BUILD_MODE} 
COMPILER_RT_RESOLVED_EXEC_OUTPUT_DIR ${COMPILER_RT_EXEC_OUTPUT_DIR})
   string(REPLACE ${CMAKE_CFG_INTDIR} ${LLVM_BUILD_MODE} 
COMPILER_RT_RESOLVED_LIBRARY_OUTPUT_DIR ${output_dir})
 
   configure_lit_site_cfg(${input} ${output})
diff --git a/compiler-rt/test/hwasan/lit.cfg.py 
b/compiler-rt/test/hwasan/lit.cfg.py
index 594f3294a84ac1..96cc2f477ad445 100644
--- a/compiler-rt/test/hwasan/lit.cfg.py
+++ b/compiler-rt/test/hwasan/lit.cfg.py
@@ -2,6 +2,9 @@
 
 import os
 
+from lit.llvm import llvm_config
+from lit.llvm.subst import ToolSubst, FindTool
+
 # Setup config name.
 config.name = "HWAddressSanitizer" + getattr(config, "name_suffix", "default")
 
@@ -74,6 +77,10 @@ def build_invocation(compile_flags):
 ("%env_hwasan_opts=", "env HWASAN_OPTIONS=" + default_hwasan_opts_str)
 )
 
+# Ensure that we can use hwasan_symbolize from the expected location
+llvm_config.add_tool_substitutions([ToolSubst("hwasan_symbolize", 
unresolved="fatal")],
+   search_dirs=[config.compiler_rt_bindir])
+
 # Default test suffixes.
 config.suffixes = [".c", ".cpp"]
 
diff --git a/compiler-rt/test/lit.common.configured.in 
b/compiler-rt/test/lit.common.configured.in
index 8955efa3eafd18..e146e847aa00e6 100644
--- a/compiler-rt/test/lit.common.configured.in
+++ b/compiler-rt/test/lit.common.configured.in
@@ -28,6 +28,7 @@ set_default("python_executable", "@Python3_EXECUTABLE@")
 set_default("compiler_rt_debug", @COMPILER_RT_DEBUG_PYBOOL@)
 set_default("compiler_rt_intercept_libdispatch", 
@COMPILER_RT_INTERCEPT_LIBDISPATCH_PYBOOL@)
 set_default("compiler_rt_output_dir", "@COMPILER_RT_RESOLVED_OUTPUT_DIR@")
+set_default("compiler_rt_bindir", "@COMPILER_RT_RESOLVED_EXEC_OUTPUT_DIR@")
 set_default("compiler_rt_libdir", "@COMPILER_RT_RESOLVED_LIBRARY_OUTPUT_DIR@")
 set_default("emulator", "@COMPILER_RT_EMULATOR@")
 set_default("asan_shadow_scale", "@COMPILER_RT_ASAN_SHADOW_SCALE@")
diff --git a/runtimes/CMakeLists.txt b/runtimes/CMakeLists.txt
index 6f24fbcccec955..3195a293c0adfa 100644
--- a/runtimes/CMakeLists.txt
+++ b/runtimes/CMakeLists.txt
@@ -218,6 +218,22 @@ foreach(entry ${runtimes})
 endforeach()
 
 if(LLVM_INCLUDE_TESTS)
+  # Add lit if needed before adding any runtimes since their CMake tests
+  # configuration might depend on lit being present.
+  if (NOT

[llvm-branch-commits] [runtimes] Allow building against an installed LLVM tree (PR #86209)

2024-03-21 Thread Alexander Richardson via llvm-branch-commits 

https://github.com/arichardson created 
https://github.com/llvm/llvm-project/pull/86209

I am currently trying to test the LLVM runtimes (including compiler-rt)
against an installed LLVM tree rather than a build tree (since that is
no longer available). Currently, the runtimes build of compiler-rt assumes
that LLVM_BINARY_DIR is writable since it uses configure_file() to write
there during the CMake configure stage. Instead, generate this file inside
CMAKE_CURRENT_BINARY_DIR, which will match LLVM_BINARY_DIR when invoked
from llvm/runtimes/CMakeLists.txt.

I also needed to make a minor change to the hwasan tests: hwasan_symbolize
was previously found in the LLVM_BINARY_DIR, but since it is generated as
part of the compiler-rt build it is now inside the CMake build directory
instead. I fixed this by passing the output directory to lit as
config.compiler_rt_bindir and using llvm_config.add_tool_substitutions().

For testing that we no longer write to the LLVM install directory as
part of testing or configuration, I created a read-only bind mount and
configured the runtimes builds as follows:
```
$ sudo mount --bind --read-only ~/llvm-install /tmp/upstream-llvm-readonly
$ cmake -DCMAKE_BUILD_TYPE=Debug \
  -DCMAKE_C_COMPILER=/tmp/upstream-llvm-readonly/bin/clang \
  -DCMAKE_CXX_COMPILER=/tmp/upstream-llvm-readonly/bin/clang++ \
  -DLLVM_INCLUDE_TESTS=TRUE -DLLVM_ENABLE_ASSERTIONS=TRUE \
  -DCOMPILER_RT_INCLUDE_TESTS=TRUE -DCOMPILER_RT_DEBUG=OFF \
  -DLLVM_ENABLE_RUNTIMES=compiler-rt \
  -DLLVM_BINARY_DIR=/tmp/upstream-llvm-readonly \
  -G Ninja -S ~/upstream-llvm-project/runtimes \
  -B ~/upstream-llvm-project/runtimes/cmake-build-debug-llvm-git
```



___
llvm-branch-commits mailing list
llvm-branch-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits