[clang] [libc] [llvm] [openmp] [libc] Rework the GPU build to be a regular target (PR #81921)
jhuber6 wrote: @vvereschaka Should be fixed now. https://github.com/llvm/llvm-project/pull/81921 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [libc] [llvm] [openmp] [libc] Rework the GPU build to be a regular target (PR #81921)
jhuber6 wrote: > @jhuber6 , looks like these changes break the following builds > > * https://lab.llvm.org/buildbot/#/builders/235/builds/5630 > > * https://lab.llvm.org/buildbot/#/builders/232/builds/19808 > > > there are a lot of CMake error messages started with > > ``` > CMake Error at cmake/modules/AddLLVM.cmake:631 (set_target_properties): > set_target_properties called with incorrect number of arguments. > Call Stack (most recent call first): > cmake/modules/AddLLVM.cmake:854 (llvm_add_library) > lib/Transforms/Hello/CMakeLists.txt:13 (add_llvm_library) > -- Targeting X86 > -- Targeting NVPTX > CMake Error at CMakeLists.txt:1213 (add_subdirectory): > add_subdirectory given source "/unittest" which is not an existing > directory. > CMake Error at tools/llvm-config/CMakeLists.txt:54 (string): > string sub-command REGEX, mode MATCH needs at least 5 arguments total to > command. > ... > ``` > > would you take care of it? I'll look into it, my guess is that I used `LLVM_DEFAULT_TARGET_TRIPLE` instead of one of the runtime ones or something. https://github.com/llvm/llvm-project/pull/81921 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [libc] [llvm] [openmp] [libc] Rework the GPU build to be a regular target (PR #81921)
vvereschaka wrote: @jhuber6 , looks like these changes break the following builds * https://lab.llvm.org/buildbot/#/builders/235/builds/5630 * https://lab.llvm.org/buildbot/#/builders/232/builds/19808 there are a lot of CMake error messages started with ``` CMake Error at cmake/modules/AddLLVM.cmake:631 (set_target_properties): set_target_properties called with incorrect number of arguments. Call Stack (most recent call first): cmake/modules/AddLLVM.cmake:854 (llvm_add_library) lib/Transforms/Hello/CMakeLists.txt:13 (add_llvm_library) -- Targeting X86 -- Targeting NVPTX CMake Error at CMakeLists.txt:1213 (add_subdirectory): add_subdirectory given source "/unittest" which is not an existing directory. CMake Error at tools/llvm-config/CMakeLists.txt:54 (string): string sub-command REGEX, mode MATCH needs at least 5 arguments total to command. ... ``` would you take care of it? https://github.com/llvm/llvm-project/pull/81921 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [libc] [llvm] [openmp] [libc] Rework the GPU build to be a regular target (PR #81921)
https://github.com/jhuber6 closed https://github.com/llvm/llvm-project/pull/81921 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [libc] [llvm] [openmp] [libc] Rework the GPU build to be a regular target (PR #81921)
https://github.com/jhuber6 updated https://github.com/llvm/llvm-project/pull/81921 >From c118c0b82cf47b36460479fd920325dedc7a6c79 Mon Sep 17 00:00:00 2001 From: Joseph Huber Date: Tue, 13 Feb 2024 21:08:02 -0600 Subject: [PATCH] [libc] Rework the GPU build to be a regular target Summary: This is a massive patch because it reworks the entire build and everything that depends on it. This is not split up because various bots would fail otherwise. I will attempt to describe the necessary changes here. This patch completely reworks how the GPU build is built and targeted. Previously, we used a standard runtimes build and handled both NVPTX and AMDGPU in a single build via multi-targeting. This added a lot of divergence in the build system and prevented us from doing various things like building for the CPU / GPU at the same time, or exporting the startup libraries or running tests without a full rebuild. The new appraoch is to handle the GPU builds as strict cross-compiling runtimes. The first step required https://github.com/llvm/llvm-project/pull/81557 to allow the `LIBC` target to build for the GPU without touching the other targets. This means that the GPU uses all the same handling as the other builds in `libc`. The new expected way to build the GPU libc is with `LLVM_LIBC_RUNTIME_TARGETS=amdgcn-amd-amdhsa;nvptx64-nvidia-cuda`. The second step was reworking how we generated the embedded GPU library by moving it into the library install step. Where we previously had one `libcgpu.a` we now have `libcgpu-amdgpu.a` and `libcgpu-nvptx.a`. This patch includes the necessary clang / OpenMP changes to make that not break the bots when this lands. We unfortunately still require that the NVPTX target has an `internal` target for tests. This is because the NVPTX target needs to do LTO for the provided version (The offloading toolchain can handle it) but cannot use it for the native toolchain which is used for making tests. This approach is vastly suprerior in every way, allowing us to treat the GPU as a standard cross-compiling target. We can now install the GPU utilities to do things like use the offload tests and other fun things. Depends on https://github.com/llvm/llvm-project/pull/81557 --- clang/lib/Driver/ToolChains/CommonArgs.cpp| 37 +- clang/test/Driver/openmp-offload-gpu.c| 20 +- libc/CMakeLists.txt | 20 +- .../cmake/modules/LLVMLibCArchitectures.cmake | 28 +- libc/cmake/modules/LLVMLibCCheckMPFR.cmake| 2 +- .../modules/LLVMLibCCompileOptionRules.cmake | 76 +--- libc/cmake/modules/LLVMLibCHeaderRules.cmake | 2 +- libc/cmake/modules/LLVMLibCLibraryRules.cmake | 141 +-- libc/cmake/modules/LLVMLibCObjectRules.cmake | 348 -- libc/cmake/modules/LLVMLibCTestRules.cmake| 47 ++- .../modules/prepare_libc_gpu_build.cmake | 108 ++ libc/docs/gpu/using.rst | 33 +- libc/include/CMakeLists.txt | 6 +- libc/lib/CMakeLists.txt | 35 +- libc/src/__support/File/CMakeLists.txt| 2 +- libc/src/__support/GPU/CMakeLists.txt | 2 +- libc/src/__support/OSUtil/CMakeLists.txt | 2 +- libc/src/__support/RPC/CMakeLists.txt | 2 +- libc/src/math/CMakeLists.txt | 16 +- libc/src/math/gpu/vendor/CMakeLists.txt | 1 - libc/src/stdio/CMakeLists.txt | 2 +- libc/src/stdlib/CMakeLists.txt| 4 +- libc/src/string/CMakeLists.txt| 12 +- libc/startup/gpu/CMakeLists.txt | 35 +- libc/startup/gpu/amdgpu/CMakeLists.txt| 13 - libc/startup/gpu/nvptx/CMakeLists.txt | 9 - libc/test/CMakeLists.txt | 6 +- libc/test/IntegrationTest/CMakeLists.txt | 16 - libc/test/UnitTest/CMakeLists.txt | 2 +- libc/test/src/__support/CMakeLists.txt| 49 +-- libc/test/src/__support/CPP/CMakeLists.txt| 2 +- libc/test/src/__support/File/CMakeLists.txt | 2 +- libc/test/src/errno/CMakeLists.txt| 2 +- libc/test/src/math/CMakeLists.txt | 20 +- libc/test/src/math/smoke/CMakeLists.txt | 8 +- libc/test/src/stdio/CMakeLists.txt| 2 +- libc/test/src/stdlib/CMakeLists.txt | 6 +- libc/test/utils/UnitTest/CMakeLists.txt | 2 +- libc/utils/CMakeLists.txt | 2 +- libc/utils/MPFRWrapper/CMakeLists.txt | 2 +- libc/utils/gpu/CMakeLists.txt | 4 +- libc/utils/gpu/loader/CMakeLists.txt | 48 ++- libc/utils/gpu/loader/amdgpu/CMakeLists.txt | 6 +- libc/utils/gpu/loader/nvptx/CMakeLists.txt| 10 +- libc/utils/gpu/server/CMakeLists.txt | 9 + llvm/CMakeLists.txt | 4 +- llvm/cmake/modules/HandleLLVMOptions.cmake| 7 + llvm/runtimes/CMakeLists.txt | 11 +- openmp/libomptarget/CMakeLists.txt| 9
[clang] [libc] [llvm] [openmp] [libc] Rework the GPU build to be a regular target (PR #81921)
https://github.com/jhuber6 updated https://github.com/llvm/llvm-project/pull/81921 >From 575390d65fd35729e855823e38dfd28f7a15debd Mon Sep 17 00:00:00 2001 From: Joseph Huber Date: Tue, 13 Feb 2024 21:08:02 -0600 Subject: [PATCH] [libc] Rework the GPU build to be a regular target Summary: This is a massive patch because it reworks the entire build and everything that depends on it. This is not split up because various bots would fail otherwise. I will attempt to describe the necessary changes here. This patch completely reworks how the GPU build is built and targeted. Previously, we used a standard runtimes build and handled both NVPTX and AMDGPU in a single build via multi-targeting. This added a lot of divergence in the build system and prevented us from doing various things like building for the CPU / GPU at the same time, or exporting the startup libraries or running tests without a full rebuild. The new appraoch is to handle the GPU builds as strict cross-compiling runtimes. The first step required https://github.com/llvm/llvm-project/pull/81557 to allow the `LIBC` target to build for the GPU without touching the other targets. This means that the GPU uses all the same handling as the other builds in `libc`. The new expected way to build the GPU libc is with `LLVM_LIBC_RUNTIME_TARGETS=amdgcn-amd-amdhsa;nvptx64-nvidia-cuda`. The second step was reworking how we generated the embedded GPU library by moving it into the library install step. Where we previously had one `libcgpu.a` we now have `libcgpu-amdgpu.a` and `libcgpu-nvptx.a`. This patch includes the necessary clang / OpenMP changes to make that not break the bots when this lands. We unfortunately still require that the NVPTX target has an `internal` target for tests. This is because the NVPTX target needs to do LTO for the provided version (The offloading toolchain can handle it) but cannot use it for the native toolchain which is used for making tests. This approach is vastly suprerior in every way, allowing us to treat the GPU as a standard cross-compiling target. We can now install the GPU utilities to do things like use the offload tests and other fun things. Depends on https://github.com/llvm/llvm-project/pull/81557 --- clang/lib/Driver/ToolChains/CommonArgs.cpp| 37 +- clang/test/Driver/openmp-offload-gpu.c| 14 +- libc/CMakeLists.txt | 20 +- .../cmake/modules/LLVMLibCArchitectures.cmake | 28 +- libc/cmake/modules/LLVMLibCCheckMPFR.cmake| 2 +- .../modules/LLVMLibCCompileOptionRules.cmake | 76 +--- libc/cmake/modules/LLVMLibCHeaderRules.cmake | 2 +- libc/cmake/modules/LLVMLibCLibraryRules.cmake | 141 +-- libc/cmake/modules/LLVMLibCObjectRules.cmake | 348 -- libc/cmake/modules/LLVMLibCTestRules.cmake| 47 ++- .../modules/prepare_libc_gpu_build.cmake | 108 ++ libc/docs/gpu/using.rst | 33 +- libc/include/CMakeLists.txt | 6 +- libc/lib/CMakeLists.txt | 35 +- libc/src/__support/File/CMakeLists.txt| 2 +- libc/src/__support/GPU/CMakeLists.txt | 2 +- libc/src/__support/OSUtil/CMakeLists.txt | 2 +- libc/src/__support/RPC/CMakeLists.txt | 2 +- libc/src/math/CMakeLists.txt | 16 +- libc/src/math/gpu/vendor/CMakeLists.txt | 1 - libc/src/stdio/CMakeLists.txt | 2 +- libc/src/stdlib/CMakeLists.txt| 4 +- libc/src/string/CMakeLists.txt| 12 +- libc/startup/gpu/CMakeLists.txt | 35 +- libc/startup/gpu/amdgpu/CMakeLists.txt| 13 - libc/startup/gpu/nvptx/CMakeLists.txt | 9 - libc/test/CMakeLists.txt | 6 +- libc/test/IntegrationTest/CMakeLists.txt | 16 - libc/test/UnitTest/CMakeLists.txt | 2 +- libc/test/src/__support/CMakeLists.txt| 49 +-- libc/test/src/__support/CPP/CMakeLists.txt| 2 +- libc/test/src/__support/File/CMakeLists.txt | 2 +- libc/test/src/errno/CMakeLists.txt| 2 +- libc/test/src/math/CMakeLists.txt | 20 +- libc/test/src/math/smoke/CMakeLists.txt | 8 +- libc/test/src/stdio/CMakeLists.txt| 2 +- libc/test/src/stdlib/CMakeLists.txt | 6 +- libc/test/utils/UnitTest/CMakeLists.txt | 2 +- libc/utils/CMakeLists.txt | 2 +- libc/utils/MPFRWrapper/CMakeLists.txt | 2 +- libc/utils/gpu/CMakeLists.txt | 4 +- libc/utils/gpu/loader/CMakeLists.txt | 48 ++- libc/utils/gpu/loader/amdgpu/CMakeLists.txt | 6 +- libc/utils/gpu/loader/nvptx/CMakeLists.txt| 10 +- libc/utils/gpu/server/CMakeLists.txt | 9 + llvm/CMakeLists.txt | 4 +- llvm/cmake/modules/HandleLLVMOptions.cmake| 7 + llvm/runtimes/CMakeLists.txt | 11 +- openmp/libomptarget/CMakeLists.txt| 9
[clang] [libc] [llvm] [openmp] [libc] Rework the GPU build to be a regular target (PR #81921)
https://github.com/jhuber6 updated https://github.com/llvm/llvm-project/pull/81921 >From 2cf6f184e2e8a6abc31e0dfb19c706569357597d Mon Sep 17 00:00:00 2001 From: Joseph Huber Date: Tue, 13 Feb 2024 21:08:02 -0600 Subject: [PATCH] [libc] Rework the GPU build to be a regular target Summary: This is a massive patch because it reworks the entire build and everything that depends on it. This is not split up because various bots would fail otherwise. I will attempt to describe the necessary changes here. This patch completely reworks how the GPU build is built and targeted. Previously, we used a standard runtimes build and handled both NVPTX and AMDGPU in a single build via multi-targeting. This added a lot of divergence in the build system and prevented us from doing various things like building for the CPU / GPU at the same time, or exporting the startup libraries or running tests without a full rebuild. The new appraoch is to handle the GPU builds as strict cross-compiling runtimes. The first step required https://github.com/llvm/llvm-project/pull/81557 to allow the `LIBC` target to build for the GPU without touching the other targets. This means that the GPU uses all the same handling as the other builds in `libc`. The new expected way to build the GPU libc is with `LLVM_LIBC_RUNTIME_TARGETS=amdgcn-amd-amdhsa;nvptx64-nvidia-cuda`. The second step was reworking how we generated the embedded GPU library by moving it into the library install step. Where we previously had one `libcgpu.a` we now have `libcgpu-amdgpu.a` and `libcgpu-nvptx.a`. This patch includes the necessary clang / OpenMP changes to make that not break the bots when this lands. We unfortunately still require that the NVPTX target has an `internal` target for tests. This is because the NVPTX target needs to do LTO for the provided version (The offloading toolchain can handle it) but cannot use it for the native toolchain which is used for making tests. This approach is vastly suprerior in every way, allowing us to treat the GPU as a standard cross-compiling target. We can now install the GPU utilities to do things like use the offload tests and other fun things. Depends on https://github.com/llvm/llvm-project/pull/81557 --- clang/lib/Driver/ToolChains/CommonArgs.cpp| 37 +- clang/test/Driver/openmp-offload-gpu.c| 14 +- libc/CMakeLists.txt | 20 +- .../cmake/modules/LLVMLibCArchitectures.cmake | 28 +- libc/cmake/modules/LLVMLibCCheckMPFR.cmake| 2 +- .../modules/LLVMLibCCompileOptionRules.cmake | 76 +--- libc/cmake/modules/LLVMLibCHeaderRules.cmake | 2 +- libc/cmake/modules/LLVMLibCLibraryRules.cmake | 141 +-- libc/cmake/modules/LLVMLibCObjectRules.cmake | 348 -- libc/cmake/modules/LLVMLibCTestRules.cmake| 47 ++- .../modules/prepare_libc_gpu_build.cmake | 108 ++ libc/include/CMakeLists.txt | 6 +- libc/lib/CMakeLists.txt | 35 +- libc/src/__support/File/CMakeLists.txt| 2 +- libc/src/__support/GPU/CMakeLists.txt | 2 +- libc/src/__support/OSUtil/CMakeLists.txt | 2 +- libc/src/__support/RPC/CMakeLists.txt | 2 +- libc/src/math/CMakeLists.txt | 16 +- libc/src/math/gpu/vendor/CMakeLists.txt | 1 - libc/src/stdio/CMakeLists.txt | 2 +- libc/src/stdlib/CMakeLists.txt| 4 +- libc/src/string/CMakeLists.txt| 12 +- libc/startup/gpu/CMakeLists.txt | 35 +- libc/startup/gpu/amdgpu/CMakeLists.txt| 13 - libc/startup/gpu/nvptx/CMakeLists.txt | 9 - libc/test/CMakeLists.txt | 6 +- libc/test/IntegrationTest/CMakeLists.txt | 16 - libc/test/UnitTest/CMakeLists.txt | 2 +- libc/test/src/__support/CMakeLists.txt| 49 +-- libc/test/src/__support/CPP/CMakeLists.txt| 2 +- libc/test/src/__support/File/CMakeLists.txt | 2 +- libc/test/src/errno/CMakeLists.txt| 2 +- libc/test/src/math/CMakeLists.txt | 20 +- libc/test/src/math/smoke/CMakeLists.txt | 8 +- libc/test/src/stdio/CMakeLists.txt| 2 +- libc/test/src/stdlib/CMakeLists.txt | 6 +- libc/test/utils/UnitTest/CMakeLists.txt | 2 +- libc/utils/CMakeLists.txt | 2 +- libc/utils/MPFRWrapper/CMakeLists.txt | 2 +- libc/utils/gpu/CMakeLists.txt | 4 +- libc/utils/gpu/loader/CMakeLists.txt | 48 ++- libc/utils/gpu/loader/amdgpu/CMakeLists.txt | 6 +- libc/utils/gpu/loader/nvptx/CMakeLists.txt| 10 +- libc/utils/gpu/server/CMakeLists.txt | 9 + llvm/CMakeLists.txt | 4 +- llvm/cmake/modules/HandleLLVMOptions.cmake| 7 + llvm/runtimes/CMakeLists.txt | 11 +- openmp/libomptarget/CMakeLists.txt| 9 +- .../plugins-nextgen/common/CMakeLists.txt | 6
[clang] [libc] [llvm] [openmp] [libc] Rework the GPU build to be a regular target (PR #81921)
https://github.com/JonChesterfield dismissed https://github.com/llvm/llvm-project/pull/81921 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [libc] [llvm] [openmp] [libc] Rework the GPU build to be a regular target (PR #81921)
https://github.com/JonChesterfield commented: Stalled on https://github.com/llvm/llvm-project/pull/81557, trying to remove the approve mark as otherwise i'll forget about this https://github.com/llvm/llvm-project/pull/81921 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [libc] [llvm] [openmp] [libc] Rework the GPU build to be a regular target (PR #81921)
https://github.com/JonChesterfield approved this pull request. https://github.com/llvm/llvm-project/pull/81921 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [libc] [llvm] [openmp] [libc] Rework the GPU build to be a regular target (PR #81921)
JonChesterfield wrote: OK, worked through this patch now. The noise is substantial but it's an improvement on what we have - the overall impression is that the cmake was originally very special cased for GPUs and now treats them very similarly to other targets, with some careful footwork around compiling the loaders for the host. This is also a step towards making the GPU parts of the cmake easier to edit for non-GPU people which is a strong win. Thank you Michael and JP. Solid effort working through this Joseph. Let's ship it. https://github.com/llvm/llvm-project/pull/81921 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [libc] [llvm] [openmp] [libc] Rework the GPU build to be a regular target (PR #81921)
https://github.com/jhuber6 updated https://github.com/llvm/llvm-project/pull/81921 >From a396fe930db6c3fb20dc4f7918736e54d21cb24b Mon Sep 17 00:00:00 2001 From: Joseph Huber Date: Tue, 13 Feb 2024 21:08:02 -0600 Subject: [PATCH] [libc] Rework the GPU build to be a regular target Summary: This is a massive patch because it reworks the entire build and everything that depends on it. This is not split up because various bots would fail otherwise. I will attempt to describe the necessary changes here. This patch completely reworks how the GPU build is built and targeted. Previously, we used a standard runtimes build and handled both NVPTX and AMDGPU in a single build via multi-targeting. This added a lot of divergence in the build system and prevented us from doing various things like building for the CPU / GPU at the same time, or exporting the startup libraries or running tests without a full rebuild. The new appraoch is to handle the GPU builds as strict cross-compiling runtimes. The first step required https://github.com/llvm/llvm-project/pull/81557 to allow the `LIBC` target to build for the GPU without touching the other targets. This means that the GPU uses all the same handling as the other builds in `libc`. The new expected way to build the GPU libc is with `LLVM_LIBC_RUNTIME_TARGETS=amdgcn-amd-amdhsa;nvptx64-nvidia-cuda`. The second step was reworking how we generated the embedded GPU library by moving it into the library install step. Where we previously had one `libcgpu.a` we now have `libcgpu-amdgpu.a` and `libcgpu-nvptx.a`. This patch includes the necessary clang / OpenMP changes to make that not break the bots when this lands. We unfortunately still require that the NVPTX target has an `internal` target for tests. This is because the NVPTX target needs to do LTO for the provided version (The offloading toolchain can handle it) but cannot use it for the native toolchain which is used for making tests. This approach is vastly suprerior in every way, allowing us to treat the GPU as a standard cross-compiling target. We can now install the GPU utilities to do things like use the offload tests and other fun things. Depends on https://github.com/llvm/llvm-project/pull/81557 --- clang/lib/Driver/ToolChains/CommonArgs.cpp| 37 +- clang/test/Driver/openmp-offload-gpu.c| 14 +- libc/CMakeLists.txt | 20 +- .../cmake/modules/LLVMLibCArchitectures.cmake | 28 +- libc/cmake/modules/LLVMLibCCheckMPFR.cmake| 2 +- .../modules/LLVMLibCCompileOptionRules.cmake | 76 +--- libc/cmake/modules/LLVMLibCHeaderRules.cmake | 2 +- libc/cmake/modules/LLVMLibCLibraryRules.cmake | 141 +-- libc/cmake/modules/LLVMLibCObjectRules.cmake | 348 -- libc/cmake/modules/LLVMLibCTestRules.cmake| 47 ++- .../modules/prepare_libc_gpu_build.cmake | 108 ++ libc/include/CMakeLists.txt | 6 +- libc/lib/CMakeLists.txt | 42 ++- libc/src/__support/File/CMakeLists.txt| 2 +- libc/src/__support/GPU/CMakeLists.txt | 2 +- libc/src/__support/OSUtil/CMakeLists.txt | 2 +- libc/src/__support/RPC/CMakeLists.txt | 2 +- libc/src/math/CMakeLists.txt | 16 +- libc/src/math/gpu/vendor/CMakeLists.txt | 1 - libc/src/stdio/CMakeLists.txt | 2 +- libc/src/stdlib/CMakeLists.txt| 4 +- libc/src/string/CMakeLists.txt| 12 +- libc/startup/gpu/CMakeLists.txt | 35 +- libc/startup/gpu/amdgpu/CMakeLists.txt| 13 - libc/startup/gpu/nvptx/CMakeLists.txt | 9 - libc/test/CMakeLists.txt | 6 +- libc/test/IntegrationTest/CMakeLists.txt | 16 - libc/test/UnitTest/CMakeLists.txt | 2 +- libc/test/src/__support/CMakeLists.txt| 49 +-- libc/test/src/__support/CPP/CMakeLists.txt| 2 +- libc/test/src/__support/File/CMakeLists.txt | 2 +- libc/test/src/errno/CMakeLists.txt| 2 +- libc/test/src/math/CMakeLists.txt | 20 +- libc/test/src/math/smoke/CMakeLists.txt | 8 +- libc/test/src/stdio/CMakeLists.txt| 2 +- libc/test/src/stdlib/CMakeLists.txt | 6 +- libc/test/utils/UnitTest/CMakeLists.txt | 2 +- libc/utils/CMakeLists.txt | 2 +- libc/utils/MPFRWrapper/CMakeLists.txt | 2 +- libc/utils/gpu/CMakeLists.txt | 4 +- libc/utils/gpu/loader/CMakeLists.txt | 48 ++- libc/utils/gpu/loader/amdgpu/CMakeLists.txt | 6 +- libc/utils/gpu/loader/nvptx/CMakeLists.txt| 10 +- libc/utils/gpu/server/CMakeLists.txt | 9 + llvm/CMakeLists.txt | 8 + llvm/cmake/modules/HandleLLVMOptions.cmake| 7 + llvm/runtimes/CMakeLists.txt | 10 +- openmp/libomptarget/CMakeLists.txt| 9 +- .../plugins-nextgen/common/CMakeLists.txt | 6
[clang] [libc] [llvm] [openmp] [libc] Rework the GPU build to be a regular target (PR #81921)
JonChesterfield wrote: One large patch may be necessary - is it also necessary to interleave reordering files with changing the contents? It makes the GUI diff tool we're using here essentially useless. If the moving code between files and factoring into functions was a separate commit we'd have a much better chance of seeing what you've changed vs what you've moved. https://github.com/llvm/llvm-project/pull/81921 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [libc] [llvm] [openmp] [libc] Rework the GPU build to be a regular target (PR #81921)
https://github.com/jhuber6 edited https://github.com/llvm/llvm-project/pull/81921 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [libc] [llvm] [openmp] [libc] Rework the GPU build to be a regular target (PR #81921)
@@ -50,31 +50,9 @@ function(collect_object_file_deps target result)
endif()
endfunction(collect_object_file_deps)
-# A rule to build a library from a collection of entrypoint objects.
-# Usage:
-# add_entrypoint_library(
-# DEPENDS
-# )
-#
-# NOTE: If one wants an entrypoint to be available in a library, then they will
-# have to list the entrypoint target explicitly in the DEPENDS list. Implicit
-# entrypoint dependencies will not be added to the library.
-function(add_entrypoint_library target_name)
- cmake_parse_arguments(
-"ENTRYPOINT_LIBRARY"
-"" # No optional arguments
-"" # No single value arguments
-"DEPENDS" # Multi-value arguments
-${ARGN}
- )
- if(NOT ENTRYPOINT_LIBRARY_DEPENDS)
-message(FATAL_ERROR "'add_entrypoint_library' target requires a DEPENDS
list "
-"of 'add_entrypoint_object' targets.")
- endif()
-
- get_fq_deps_list(fq_deps_list ${ENTRYPOINT_LIBRARY_DEPENDS})
+function(get_all_object_file_deps result fq_deps_list)
jhuber6 wrote:
It used to be simpler, but I had to rebase it on top of other changes in the
`libc`. I could potentially make a simpler patch beforehand.
https://github.com/llvm/llvm-project/pull/81921
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [libc] [llvm] [openmp] [libc] Rework the GPU build to be a regular target (PR #81921)
@@ -50,31 +50,9 @@ function(collect_object_file_deps target result)
endif()
endfunction(collect_object_file_deps)
-# A rule to build a library from a collection of entrypoint objects.
-# Usage:
-# add_entrypoint_library(
-# DEPENDS
-# )
-#
-# NOTE: If one wants an entrypoint to be available in a library, then they will
-# have to list the entrypoint target explicitly in the DEPENDS list. Implicit
-# entrypoint dependencies will not be added to the library.
-function(add_entrypoint_library target_name)
- cmake_parse_arguments(
-"ENTRYPOINT_LIBRARY"
-"" # No optional arguments
-"" # No single value arguments
-"DEPENDS" # Multi-value arguments
-${ARGN}
- )
- if(NOT ENTRYPOINT_LIBRARY_DEPENDS)
-message(FATAL_ERROR "'add_entrypoint_library' target requires a DEPENDS
list "
-"of 'add_entrypoint_object' targets.")
- endif()
-
- get_fq_deps_list(fq_deps_list ${ENTRYPOINT_LIBRARY_DEPENDS})
+function(get_all_object_file_deps result fq_deps_list)
JonChesterfield wrote:
This looks like factoring some existing code into a function - if you landed
that refactor without changing what the code does, I think it would make this
diff much more legible as the current and new code could align
https://github.com/llvm/llvm-project/pull/81921
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [libc] [llvm] [openmp] [libc] Rework the GPU build to be a regular target (PR #81921)
https://github.com/jhuber6 updated https://github.com/llvm/llvm-project/pull/81921 >From f3013086f60f2a78c12887cf1736455e8fb1911b Mon Sep 17 00:00:00 2001 From: Joseph Huber Date: Tue, 13 Feb 2024 21:08:02 -0600 Subject: [PATCH] [libc] Rework the GPU build to be a regular target Summary: This is a massive patch because it reworks the entire build and everything that depends on it. This is not split up because various bots would fail otherwise. I will attempt to describe the necessary changes here. This patch completely reworks how the GPU build is built and targeted. Previously, we used a standard runtimes build and handled both NVPTX and AMDGPU in a single build via multi-targeting. This added a lot of divergence in the build system and prevented us from doing various things like building for the CPU / GPU at the same time, or exporting the startup libraries or running tests without a full rebuild. The new appraoch is to handle the GPU builds as strict cross-compiling runtimes. The first step required https://github.com/llvm/llvm-project/pull/81557 to allow the `LIBC` target to build for the GPU without touching the other targets. This means that the GPU uses all the same handling as the other builds in `libc`. The new expected way to build the GPU libc is with `LLVM_LIBC_RUNTIME_TARGETS=amdgcn-amd-amdhsa;nvptx64-nvidia-cuda`. The second step was reworking how we generated the embedded GPU library by moving it into the library install step. Where we previously had one `libcgpu.a` we now have `libcgpu-amdgpu.a` and `libcgpu-nvptx.a`. This patch includes the necessary clang / OpenMP changes to make that not break the bots when this lands. We unfortunately still require that the NVPTX target has an `internal` target for tests. This is because the NVPTX target needs to do LTO for the provided version (The offloading toolchain can handle it) but cannot use it for the native toolchain which is used for making tests. This approach is vastly suprerior in every way, allowing us to treat the GPU as a standard cross-compiling target. We can now install the GPU utilities to do things like use the offload tests and other fun things. Depends on https://github.com/llvm/llvm-project/pull/81557 --- clang/lib/Driver/ToolChains/CommonArgs.cpp| 37 +- clang/test/Driver/openmp-offload-gpu.c| 14 +- libc/CMakeLists.txt | 20 +- .../cmake/modules/LLVMLibCArchitectures.cmake | 28 +- libc/cmake/modules/LLVMLibCCheckMPFR.cmake| 2 +- .../modules/LLVMLibCCompileOptionRules.cmake | 105 ++ libc/cmake/modules/LLVMLibCHeaderRules.cmake | 2 +- libc/cmake/modules/LLVMLibCLibraryRules.cmake | 141 +-- libc/cmake/modules/LLVMLibCObjectRules.cmake | 348 -- libc/cmake/modules/LLVMLibCTestRules.cmake| 68 ++-- .../modules/prepare_libc_gpu_build.cmake | 108 ++ libc/include/CMakeLists.txt | 6 +- libc/lib/CMakeLists.txt | 42 ++- libc/src/__support/File/CMakeLists.txt| 2 +- libc/src/__support/GPU/CMakeLists.txt | 2 +- libc/src/__support/OSUtil/CMakeLists.txt | 2 +- libc/src/__support/RPC/CMakeLists.txt | 2 +- libc/src/math/CMakeLists.txt | 16 +- libc/src/math/gpu/vendor/CMakeLists.txt | 1 - libc/src/stdio/CMakeLists.txt | 2 +- libc/src/stdlib/CMakeLists.txt| 4 +- libc/src/string/CMakeLists.txt| 12 +- libc/startup/gpu/CMakeLists.txt | 35 +- libc/startup/gpu/amdgpu/CMakeLists.txt| 13 - libc/startup/gpu/nvptx/CMakeLists.txt | 9 - libc/test/CMakeLists.txt | 6 +- libc/test/IntegrationTest/CMakeLists.txt | 16 - libc/test/UnitTest/CMakeLists.txt | 6 +- libc/test/src/__support/CMakeLists.txt| 49 +-- libc/test/src/__support/CPP/CMakeLists.txt| 2 +- libc/test/src/__support/File/CMakeLists.txt | 2 +- libc/test/src/errno/CMakeLists.txt| 2 +- libc/test/src/math/CMakeLists.txt | 20 +- libc/test/src/math/smoke/CMakeLists.txt | 8 +- libc/test/src/stdio/CMakeLists.txt| 2 +- libc/test/src/stdlib/CMakeLists.txt | 6 +- libc/test/utils/UnitTest/CMakeLists.txt | 2 +- libc/utils/CMakeLists.txt | 2 +- libc/utils/MPFRWrapper/CMakeLists.txt | 2 +- libc/utils/gpu/CMakeLists.txt | 4 +- libc/utils/gpu/loader/CMakeLists.txt | 48 ++- libc/utils/gpu/loader/amdgpu/CMakeLists.txt | 6 +- libc/utils/gpu/loader/nvptx/CMakeLists.txt| 10 +- libc/utils/gpu/server/CMakeLists.txt | 9 + llvm/CMakeLists.txt | 8 + llvm/cmake/modules/HandleLLVMOptions.cmake| 7 + llvm/runtimes/CMakeLists.txt | 10 +- openmp/libomptarget/CMakeLists.txt| 9 +- .../plugins-nextgen/common/CMakeLists.txt |
[clang] [libc] [llvm] [openmp] [libc] Rework the GPU build to be a regular target (PR #81921)
https://github.com/jhuber6 updated https://github.com/llvm/llvm-project/pull/81921 >From 8727a9631480deac9d9df386ed26dfcd35914a13 Mon Sep 17 00:00:00 2001 From: Joseph Huber Date: Tue, 13 Feb 2024 21:08:02 -0600 Subject: [PATCH] [libc] Rework the GPU build to be a regular target Summary: This is a massive patch because it reworks the entire build and everything that depends on it. This is not split up because various bots would fail otherwise. I will attempt to describe the necessary changes here. This patch completely reworks how the GPU build is built and targeted. Previously, we used a standard runtimes build and handled both NVPTX and AMDGPU in a single build via multi-targeting. This added a lot of divergence in the build system and prevented us from doing various things like building for the CPU / GPU at the same time, or exporting the startup libraries or running tests without a full rebuild. The new appraoch is to handle the GPU builds as strict cross-compiling runtimes. The first step required https://github.com/llvm/llvm-project/pull/81557 to allow the `LIBC` target to build for the GPU without touching the other targets. This means that the GPU uses all the same handling as the other builds in `libc`. The new expected way to build the GPU libc is with `LLVM_LIBC_RUNTIME_TARGETS=amdgcn-amd-amdhsa;nvptx64-nvidia-cuda`. The second step was reworking how we generated the embedded GPU library by moving it into the library install step. Where we previously had one `libcgpu.a` we now have `libcgpu-amdgpu.a` and `libcgpu-nvptx.a`. This patch includes the necessary clang / OpenMP changes to make that not break the bots when this lands. We unfortunately still require that the NVPTX target has an `internal` target for tests. This is because the NVPTX target needs to do LTO for the provided version (The offloading toolchain can handle it) but cannot use it for the native toolchain which is used for making tests. This approach is vastly suprerior in every way, allowing us to treat the GPU as a standard cross-compiling target. We can now install the GPU utilities to do things like use the offload tests and other fun things. Depends on https://github.com/llvm/llvm-project/pull/81557 --- clang/lib/Driver/ToolChains/CommonArgs.cpp| 37 +- clang/test/Driver/openmp-offload-gpu.c| 14 +- libc/CMakeLists.txt | 20 +- .../cmake/modules/LLVMLibCArchitectures.cmake | 28 +- libc/cmake/modules/LLVMLibCCheckMPFR.cmake| 2 +- .../modules/LLVMLibCCompileOptionRules.cmake | 105 ++ libc/cmake/modules/LLVMLibCHeaderRules.cmake | 2 +- libc/cmake/modules/LLVMLibCLibraryRules.cmake | 141 +-- libc/cmake/modules/LLVMLibCObjectRules.cmake | 348 -- libc/cmake/modules/LLVMLibCTestRules.cmake| 68 ++-- .../modules/prepare_libc_gpu_build.cmake | 108 ++ libc/include/CMakeLists.txt | 6 +- libc/lib/CMakeLists.txt | 42 ++- libc/src/__support/File/CMakeLists.txt| 2 +- libc/src/__support/GPU/CMakeLists.txt | 2 +- libc/src/__support/OSUtil/CMakeLists.txt | 2 +- libc/src/__support/RPC/CMakeLists.txt | 2 +- libc/src/math/CMakeLists.txt | 16 +- libc/src/math/gpu/vendor/CMakeLists.txt | 1 - libc/src/stdio/CMakeLists.txt | 2 +- libc/src/stdlib/CMakeLists.txt| 4 +- libc/src/string/CMakeLists.txt| 12 +- libc/startup/gpu/CMakeLists.txt | 35 +- libc/startup/gpu/amdgpu/CMakeLists.txt| 13 - libc/startup/gpu/nvptx/CMakeLists.txt | 9 - libc/test/CMakeLists.txt | 6 +- libc/test/IntegrationTest/CMakeLists.txt | 16 - libc/test/UnitTest/CMakeLists.txt | 6 +- libc/test/src/__support/CMakeLists.txt| 49 +-- libc/test/src/__support/CPP/CMakeLists.txt| 2 +- libc/test/src/__support/File/CMakeLists.txt | 2 +- libc/test/src/errno/CMakeLists.txt| 2 +- libc/test/src/math/CMakeLists.txt | 20 +- libc/test/src/math/smoke/CMakeLists.txt | 8 +- libc/test/src/stdio/CMakeLists.txt| 2 +- libc/test/src/stdlib/CMakeLists.txt | 6 +- libc/test/utils/UnitTest/CMakeLists.txt | 2 +- libc/utils/CMakeLists.txt | 2 +- libc/utils/MPFRWrapper/CMakeLists.txt | 2 +- libc/utils/gpu/CMakeLists.txt | 4 +- libc/utils/gpu/loader/CMakeLists.txt | 48 ++- libc/utils/gpu/loader/amdgpu/CMakeLists.txt | 6 +- libc/utils/gpu/loader/nvptx/CMakeLists.txt| 10 +- libc/utils/gpu/server/CMakeLists.txt | 9 + llvm/CMakeLists.txt | 8 + llvm/cmake/modules/HandleLLVMOptions.cmake| 7 + llvm/runtimes/CMakeLists.txt | 10 +- openmp/libomptarget/CMakeLists.txt| 9 +- .../plugins-nextgen/common/CMakeLists.txt |
[clang] [libc] [llvm] [openmp] [libc] Rework the GPU build to be a regular target (PR #81921)
https://github.com/jhuber6 updated https://github.com/llvm/llvm-project/pull/81921 >From 85f7218baa72307699b48bffa3da4005597ec719 Mon Sep 17 00:00:00 2001 From: Joseph Huber Date: Tue, 13 Feb 2024 21:08:02 -0600 Subject: [PATCH] [libc] Rework the GPU build to be a regular target Summary: This is a massive patch because it reworks the entire build and everything that depends on it. This is not split up because various bots would fail otherwise. I will attempt to describe the necessary changes here. This patch completely reworks how the GPU build is built and targeted. Previously, we used a standard runtimes build and handled both NVPTX and AMDGPU in a single build via multi-targeting. This added a lot of divergence in the build system and prevented us from doing various things like building for the CPU / GPU at the same time, or exporting the startup libraries or running tests without a full rebuild. The new appraoch is to handle the GPU builds as strict cross-compiling runtimes. The first step required https://github.com/llvm/llvm-project/pull/81557 to allow the `LIBC` target to build for the GPU without touching the other targets. This means that the GPU uses all the same handling as the other builds in `libc`. The new expected way to build the GPU libc is with `LLVM_LIBC_RUNTIME_TARGETS=amdgcn-amd-amdhsa;nvptx64-nvidia-cuda`. The second step was reworking how we generated the embedded GPU library by moving it into the library install step. Where we previously had one `libcgpu.a` we now have `libcgpu-amdgpu.a` and `libcgpu-nvptx.a`. This patch includes the necessary clang / OpenMP changes to make that not break the bots when this lands. We unfortunately still require that the NVPTX target has an `internal` target for tests. This is because the NVPTX target needs to do LTO for the provided version (The offloading toolchain can handle it) but cannot use it for the native toolchain which is used for making tests. This approach is vastly suprerior in every way, allowing us to treat the GPU as a standard cross-compiling target. We can now install the GPU utilities to do things like use the offload tests and other fun things. Depends on https://github.com/llvm/llvm-project/pull/81557 --- clang/lib/Driver/ToolChains/CommonArgs.cpp| 37 +- clang/test/Driver/openmp-offload-gpu.c| 14 +- libc/CMakeLists.txt | 20 +- .../cmake/modules/LLVMLibCArchitectures.cmake | 28 +- libc/cmake/modules/LLVMLibCCheckMPFR.cmake| 2 +- .../modules/LLVMLibCCompileOptionRules.cmake | 104 ++ libc/cmake/modules/LLVMLibCHeaderRules.cmake | 2 +- libc/cmake/modules/LLVMLibCLibraryRules.cmake | 141 +-- libc/cmake/modules/LLVMLibCObjectRules.cmake | 348 -- libc/cmake/modules/LLVMLibCTestRules.cmake| 67 ++-- .../modules/prepare_libc_gpu_build.cmake | 108 ++ libc/include/CMakeLists.txt | 6 +- libc/lib/CMakeLists.txt | 42 ++- libc/src/__support/File/CMakeLists.txt| 2 +- libc/src/__support/GPU/CMakeLists.txt | 2 +- libc/src/__support/OSUtil/CMakeLists.txt | 2 +- libc/src/__support/RPC/CMakeLists.txt | 2 +- libc/src/math/CMakeLists.txt | 16 +- libc/src/math/gpu/vendor/CMakeLists.txt | 1 - libc/src/stdio/CMakeLists.txt | 2 +- libc/src/stdlib/CMakeLists.txt| 4 +- libc/src/string/CMakeLists.txt| 12 +- libc/startup/gpu/CMakeLists.txt | 35 +- libc/startup/gpu/amdgpu/CMakeLists.txt| 13 - libc/startup/gpu/nvptx/CMakeLists.txt | 9 - libc/test/CMakeLists.txt | 6 +- libc/test/IntegrationTest/CMakeLists.txt | 16 - libc/test/UnitTest/CMakeLists.txt | 6 +- libc/test/src/__support/CMakeLists.txt| 49 +-- libc/test/src/__support/CPP/CMakeLists.txt| 2 +- libc/test/src/__support/File/CMakeLists.txt | 2 +- libc/test/src/errno/CMakeLists.txt| 2 +- libc/test/src/math/CMakeLists.txt | 20 +- libc/test/src/math/smoke/CMakeLists.txt | 8 +- libc/test/src/stdio/CMakeLists.txt| 2 +- libc/test/src/stdlib/CMakeLists.txt | 6 +- libc/test/utils/UnitTest/CMakeLists.txt | 2 +- libc/utils/CMakeLists.txt | 2 +- libc/utils/MPFRWrapper/CMakeLists.txt | 2 +- libc/utils/gpu/CMakeLists.txt | 4 +- libc/utils/gpu/loader/CMakeLists.txt | 48 ++- libc/utils/gpu/loader/amdgpu/CMakeLists.txt | 6 +- libc/utils/gpu/loader/nvptx/CMakeLists.txt| 10 +- libc/utils/gpu/server/CMakeLists.txt | 9 + llvm/CMakeLists.txt | 8 + llvm/cmake/modules/HandleLLVMOptions.cmake| 7 + llvm/runtimes/CMakeLists.txt | 10 +- openmp/libomptarget/CMakeLists.txt| 9 +- .../plugins-nextgen/common/CMakeLists.txt |
[clang] [libc] [llvm] [openmp] [libc] Rework the GPU build to be a regular target (PR #81921)
https://github.com/jplehr commented: I looked at the changes and from the little I understand CMake they seem ok. I added one nit. Maybe @saiislam can have a look as well. https://github.com/llvm/llvm-project/pull/81921 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [libc] [llvm] [openmp] [libc] Rework the GPU build to be a regular target (PR #81921)
https://github.com/jplehr edited https://github.com/llvm/llvm-project/pull/81921 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [libc] [llvm] [openmp] [libc] Rework the GPU build to be a regular target (PR #81921)
@@ -102,6 +80,121 @@ function(add_entrypoint_library target_name)
list(APPEND all_deps ${entrypoint_target})
endforeach(dep)
list(REMOVE_DUPLICATES all_deps)
+ set(${result} ${all_deps} PARENT_SCOPE)
+endfunction()
+
+# A rule to build a library from a collection of entrypoint objects and bundle
+# it into a GPU fatbinary. Usage is the same as 'add_entrypoint_library'.
+# Usage:
+# add_gpu_entrypoint_library(
+# DEPENDS
+# )
+function(add_gpu_entrypoint_library target_name)
+ cmake_parse_arguments(
+"ENTRYPOINT_LIBRARY"
+"" # No optional arguments
+"" # No single value arguments
+"DEPENDS" # Multi-value arguments
+${ARGN}
+ )
+ if(NOT ENTRYPOINT_LIBRARY_DEPENDS)
+message(FATAL_ERROR "'add_entrypoint_library' target requires a DEPENDS
list "
+"of 'add_entrypoint_object' targets.")
+ endif()
+
+ get_fq_deps_list(fq_deps_list ${ENTRYPOINT_LIBRARY_DEPENDS})
+ get_all_object_file_deps(all_deps "${fq_deps_list}")
+
+ # The GPU 'libc' needs to be exported in a format that can be linked with
+ # offloading langauges like OpenMP or CUDA. This wraps every GPU object into
a
+ # fat binary and adds them to a static library.
+ set(objects "")
+ foreach(dep IN LISTS all_deps)
+set(object
$<$,${dep}>:$>)
+string(FIND ${dep} "." last_dot_loc REVERSE)
+math(EXPR name_loc "${last_dot_loc} + 1")
+string(SUBSTRING ${dep} ${name_loc} -1 name)
+if(LIBC_TARGET_ARCHITECTURE_IS_NVPTX)
+ set(prefix
--image=arch=generic,triple=nvptx64-nvidia-cuda,feature=+ptx63)
+else()
jplehr wrote:
Other places do `elseif(LIBC_TARGET_ARCHITECTURE_IS_AMDGPU)`. Maybe here as
well for consistency?
https://github.com/llvm/llvm-project/pull/81921
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [libc] [llvm] [openmp] [libc] Rework the GPU build to be a regular target (PR #81921)
jhuber6 wrote: Tested this one a few machines and it works as expected after some final tweaks. FYI @jplehr and @Artem-B, this will change the CMake configuration required for building and testing on the build bots. The new expected way to test each one respectively would be the following ``` -DLLVM_LIBC_RUNTIME_TARGETS=amdgcn-amd-amdhsa;nvptx64-nvidia-cuda ninja check-libc-amdgcn-amd-amdhsa ninja check-libc-nvptx64-nvidia-cuda ``` I think specifically there are two CMake issues that might make this annoying. I would like to be able to just do `ninja check-libc` and have it check all of them, but for the bots it's likely enough to just check specifically which one we're interested in. https://github.com/llvm/llvm-project/pull/81921 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [libc] [llvm] [openmp] [libc] Rework the GPU build to be a regular target (PR #81921)
https://github.com/jhuber6 updated https://github.com/llvm/llvm-project/pull/81921 >From 63e4205a9f5cc3ea8a4ce0730b01d78b6c9bde42 Mon Sep 17 00:00:00 2001 From: Joseph Huber Date: Tue, 13 Feb 2024 21:08:02 -0600 Subject: [PATCH] [libc] Rework the GPU build to be a regular target Summary: This is a massive patch because it reworks the entire build and everything that depends on it. This is not split up because various bots would fail otherwise. I will attempt to describe the necessary changes here. This patch completely reworks how the GPU build is built and targeted. Previously, we used a standard runtimes build and handled both NVPTX and AMDGPU in a single build via multi-targeting. This added a lot of divergence in the build system and prevented us from doing various things like building for the CPU / GPU at the same time, or exporting the startup libraries or running tests without a full rebuild. The new appraoch is to handle the GPU builds as strict cross-compiling runtimes. The first step required https://github.com/llvm/llvm-project/pull/81557 to allow the `LIBC` target to build for the GPU without touching the other targets. This means that the GPU uses all the same handling as the other builds in `libc`. The new expected way to build the GPU libc is with `LLVM_LIBC_RUNTIME_TARGETS=amdgcn-amd-amdhsa;nvptx64-nvidia-cuda`. The second step was reworking how we generated the embedded GPU library by moving it into the library install step. Where we previously had one `libcgpu.a` we now have `libcgpu-amdgpu.a` and `libcgpu-nvptx.a`. This patch includes the necessary clang / OpenMP changes to make that not break the bots when this lands. We unfortunately still require that the NVPTX target has an `internal` target for tests. This is because the NVPTX target needs to do LTO for the provided version (The offloading toolchain can handle it) but cannot use it for the native toolchain which is used for making tests. This approach is vastly suprerior in every way, allowing us to treat the GPU as a standard cross-compiling target. We can now install the GPU utilities to do things like use the offload tests and other fun things. Depends on https://github.com/llvm/llvm-project/pull/81557 --- clang/lib/Driver/ToolChains/CommonArgs.cpp| 37 +- clang/test/Driver/openmp-offload-gpu.c| 14 +- libc/CMakeLists.txt | 20 +- .../cmake/modules/LLVMLibCArchitectures.cmake | 28 +- libc/cmake/modules/LLVMLibCCheckMPFR.cmake| 2 +- .../modules/LLVMLibCCompileOptionRules.cmake | 104 ++ libc/cmake/modules/LLVMLibCHeaderRules.cmake | 2 +- libc/cmake/modules/LLVMLibCLibraryRules.cmake | 141 +-- libc/cmake/modules/LLVMLibCObjectRules.cmake | 348 -- libc/cmake/modules/LLVMLibCTestRules.cmake| 67 ++-- .../modules/prepare_libc_gpu_build.cmake | 99 ++--- libc/include/CMakeLists.txt | 6 +- libc/lib/CMakeLists.txt | 42 ++- libc/src/__support/File/CMakeLists.txt| 2 +- libc/src/__support/GPU/CMakeLists.txt | 2 +- libc/src/__support/OSUtil/CMakeLists.txt | 2 +- libc/src/__support/RPC/CMakeLists.txt | 2 +- libc/src/math/CMakeLists.txt | 16 +- libc/src/math/gpu/vendor/CMakeLists.txt | 1 - libc/src/stdio/CMakeLists.txt | 2 +- libc/src/stdlib/CMakeLists.txt| 4 +- libc/src/string/CMakeLists.txt| 12 +- libc/startup/gpu/CMakeLists.txt | 35 +- libc/startup/gpu/amdgpu/CMakeLists.txt| 13 - libc/startup/gpu/nvptx/CMakeLists.txt | 9 - libc/test/CMakeLists.txt | 6 +- libc/test/IntegrationTest/CMakeLists.txt | 16 - libc/test/UnitTest/CMakeLists.txt | 6 +- libc/test/src/__support/CMakeLists.txt| 49 +-- libc/test/src/__support/CPP/CMakeLists.txt| 2 +- libc/test/src/__support/File/CMakeLists.txt | 2 +- libc/test/src/errno/CMakeLists.txt| 2 +- libc/test/src/math/CMakeLists.txt | 20 +- libc/test/src/math/smoke/CMakeLists.txt | 8 +- libc/test/src/stdio/CMakeLists.txt| 2 +- libc/test/src/stdlib/CMakeLists.txt | 6 +- libc/test/utils/UnitTest/CMakeLists.txt | 2 +- libc/utils/CMakeLists.txt | 2 +- libc/utils/MPFRWrapper/CMakeLists.txt | 2 +- libc/utils/gpu/CMakeLists.txt | 4 +- libc/utils/gpu/loader/CMakeLists.txt | 48 ++- libc/utils/gpu/loader/amdgpu/CMakeLists.txt | 6 +- libc/utils/gpu/loader/nvptx/CMakeLists.txt| 10 +- libc/utils/gpu/server/CMakeLists.txt | 9 + llvm/CMakeLists.txt | 8 + llvm/cmake/modules/HandleLLVMOptions.cmake| 7 + llvm/runtimes/CMakeLists.txt | 6 +- openmp/libomptarget/CMakeLists.txt| 9 +- .../plugins-nextgen/common/CMakeLists.txt |
[clang] [libc] [llvm] [openmp] [libc] Rework the GPU build to be a regular target (PR #81921)
https://github.com/jhuber6 updated https://github.com/llvm/llvm-project/pull/81921 >From c4758c8663307708e5ac653a8692e595f4b1f4cc Mon Sep 17 00:00:00 2001 From: Joseph Huber Date: Tue, 13 Feb 2024 21:08:02 -0600 Subject: [PATCH] [libc] Rework the GPU build to be a regular target Summary: This is a massive patch because it reworks the entire build and everything that depends on it. This is not split up because various bots would fail otherwise. I will attempt to describe the necessary changes here. This patch completely reworks how the GPU build is built and targeted. Previously, we used a standard runtimes build and handled both NVPTX and AMDGPU in a single build via multi-targeting. This added a lot of divergence in the build system and prevented us from doing various things like building for the CPU / GPU at the same time, or exporting the startup libraries or running tests without a full rebuild. The new appraoch is to handle the GPU builds as strict cross-compiling runtimes. The first step required https://github.com/llvm/llvm-project/pull/81557 to allow the `LIBC` target to build for the GPU without touching the other targets. This means that the GPU uses all the same handling as the other builds in `libc`. The new expected way to build the GPU libc is with `LLVM_LIBC_RUNTIME_TARGETS=amdgcn-amd-amdhsa;nvptx64-nvidia-cuda`. The second step was reworking how we generated the embedded GPU library by moving it into the library install step. Where we previously had one `libcgpu.a` we now have `libcgpu-amdgpu.a` and `libcgpu-nvptx.a`. This patch includes the necessary clang / OpenMP changes to make that not break the bots when this lands. We unfortunately still require that the NVPTX target has an `internal` target for tests. This is because the NVPTX target needs to do LTO for the provided version (The offloading toolchain can handle it) but cannot use it for the native toolchain which is used for making tests. This approach is vastly suprerior in every way, allowing us to treat the GPU as a standard cross-compiling target. We can now install the GPU utilities to do things like use the offload tests and other fun things. Depends on https://github.com/llvm/llvm-project/pull/81557 --- clang/lib/Driver/ToolChains/CommonArgs.cpp| 37 +- clang/test/Driver/openmp-offload-gpu.c| 14 +- libc/CMakeLists.txt | 20 +- .../cmake/modules/LLVMLibCArchitectures.cmake | 28 +- libc/cmake/modules/LLVMLibCCheckMPFR.cmake| 2 +- .../modules/LLVMLibCCompileOptionRules.cmake | 104 ++ libc/cmake/modules/LLVMLibCHeaderRules.cmake | 2 +- libc/cmake/modules/LLVMLibCLibraryRules.cmake | 141 ++-- libc/cmake/modules/LLVMLibCObjectRules.cmake | 329 -- libc/cmake/modules/LLVMLibCTestRules.cmake| 67 ++-- .../modules/prepare_libc_gpu_build.cmake | 102 ++ libc/include/CMakeLists.txt | 6 +- libc/lib/CMakeLists.txt | 42 ++- libc/src/__support/File/CMakeLists.txt| 2 +- libc/src/__support/GPU/CMakeLists.txt | 2 +- libc/src/__support/OSUtil/CMakeLists.txt | 2 +- libc/src/__support/RPC/CMakeLists.txt | 2 +- libc/src/math/CMakeLists.txt | 16 +- libc/src/math/gpu/vendor/CMakeLists.txt | 1 - libc/src/stdio/CMakeLists.txt | 2 +- libc/src/stdlib/CMakeLists.txt| 4 +- libc/src/string/CMakeLists.txt| 12 +- libc/startup/gpu/CMakeLists.txt | 35 +- libc/startup/gpu/amdgpu/CMakeLists.txt| 12 - libc/startup/gpu/nvptx/CMakeLists.txt | 11 +- libc/test/CMakeLists.txt | 6 +- libc/test/IntegrationTest/CMakeLists.txt | 16 - libc/test/UnitTest/CMakeLists.txt | 6 +- libc/test/src/__support/CMakeLists.txt| 49 +-- libc/test/src/__support/CPP/CMakeLists.txt| 2 +- libc/test/src/__support/File/CMakeLists.txt | 2 +- libc/test/src/errno/CMakeLists.txt| 2 +- libc/test/src/math/CMakeLists.txt | 20 +- libc/test/src/math/smoke/CMakeLists.txt | 8 +- libc/test/src/stdio/CMakeLists.txt| 2 +- libc/test/src/stdlib/CMakeLists.txt | 6 +- libc/test/utils/UnitTest/CMakeLists.txt | 2 +- libc/utils/CMakeLists.txt | 2 +- libc/utils/MPFRWrapper/CMakeLists.txt | 2 +- libc/utils/gpu/CMakeLists.txt | 4 +- libc/utils/gpu/loader/CMakeLists.txt | 48 ++- libc/utils/gpu/loader/amdgpu/CMakeLists.txt | 6 +- libc/utils/gpu/loader/nvptx/CMakeLists.txt| 10 +- libc/utils/gpu/server/CMakeLists.txt | 9 + llvm/CMakeLists.txt | 8 + llvm/cmake/modules/HandleLLVMOptions.cmake| 7 + llvm/runtimes/CMakeLists.txt | 6 +- openmp/libomptarget/CMakeLists.txt| 9 +- .../plugins-nextgen/common/CMakeLists.txt
[clang] [libc] [llvm] [openmp] [libc] Rework the GPU build to be a regular target (PR #81921)
@@ -135,86 +147,20 @@ function(_get_common_test_compile_options output_var
flags)
# list(APPEND compile_options "-Wglobal-constructors")
# endif()
endif()
- if (LIBC_TARGET_ARCHITECTURE_IS_GPU)
-# TODO: Set these flags
-# list(APPEND compile_options "-nogpulib")
-# list(APPEND compile_options "-fvisibility=hidden")
-# list(APPEND compile_options "-fconvergent-functions")
-
-# # Manually disable all standard include paths and include the resource
-# # directory to prevent system headers from being included.
-# list(APPEND compile_options "-isystem${COMPILER_RESOURCE_DIR}/include")
-# list(APPEND compile_options "-nostdinc")
- endif()
set(${output_var} ${compile_options} PARENT_SCOPE)
endfunction()
-
-# Obtains NVPTX specific arguments for compilation.
-# The PTX feature is primarily based on the CUDA toolchain version. We want to
-# be able to target NVPTX without an existing CUDA installation, so we need to
-# set this manually. This simply sets the PTX feature to the minimum required
-# for the features we wish to use on that target. The minimum PTX features used
-# here roughly corresponds to the CUDA 9.0 release.
-# Adjust as needed for desired PTX features.
-function(get_nvptx_compile_options output_var gpu_arch)
- set(nvptx_options "")
- list(APPEND nvptx_options "-march=${gpu_arch}")
- list(APPEND nvptx_options "-Wno-unknown-cuda-version")
- list(APPEND nvptx_options "SHELL:-mllvm -nvptx-emit-init-fini-kernel=false")
- if(${gpu_arch} STREQUAL "sm_35")
-list(APPEND nvptx_options "--cuda-feature=+ptx63")
- elseif(${gpu_arch} STREQUAL "sm_37")
-list(APPEND nvptx_options "--cuda-feature=+ptx63")
- elseif(${gpu_arch} STREQUAL "sm_50")
-list(APPEND nvptx_options "--cuda-feature=+ptx63")
- elseif(${gpu_arch} STREQUAL "sm_52")
-list(APPEND nvptx_options "--cuda-feature=+ptx63")
- elseif(${gpu_arch} STREQUAL "sm_53")
-list(APPEND nvptx_options "--cuda-feature=+ptx63")
- elseif(${gpu_arch} STREQUAL "sm_60")
-list(APPEND nvptx_options "--cuda-feature=+ptx63")
- elseif(${gpu_arch} STREQUAL "sm_61")
-list(APPEND nvptx_options "--cuda-feature=+ptx63")
- elseif(${gpu_arch} STREQUAL "sm_62")
-list(APPEND nvptx_options "--cuda-feature=+ptx63")
- elseif(${gpu_arch} STREQUAL "sm_70")
-list(APPEND nvptx_options "--cuda-feature=+ptx63")
- elseif(${gpu_arch} STREQUAL "sm_72")
-list(APPEND nvptx_options "--cuda-feature=+ptx63")
- elseif(${gpu_arch} STREQUAL "sm_75")
-list(APPEND nvptx_options "--cuda-feature=+ptx63")
- elseif(${gpu_arch} STREQUAL "sm_80")
-list(APPEND nvptx_options "--cuda-feature=+ptx72")
- elseif(${gpu_arch} STREQUAL "sm_86")
-list(APPEND nvptx_options "--cuda-feature=+ptx72")
- elseif(${gpu_arch} STREQUAL "sm_89")
-list(APPEND nvptx_options "--cuda-feature=+ptx72")
- elseif(${gpu_arch} STREQUAL "sm_90")
-list(APPEND nvptx_options "--cuda-feature=+ptx72")
- else()
-message(FATAL_ERROR "Unknown Nvidia GPU architecture '${gpu_arch}'")
+function(_get_hermetic_test_compile_options output_var flags)
jhuber6 wrote:
I was also going to fold this into the integration test arguments in a later
patch, since we have the same flags there. The integration tests are marked as
deprecated but I think we should keep them around as it's nice to have tests
that don't include the really heavy unit testing framework. It's the only way I
can do multi-threaded tests on the GPU.
https://github.com/llvm/llvm-project/pull/81921
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [libc] [llvm] [openmp] [libc] Rework the GPU build to be a regular target (PR #81921)
@@ -135,86 +147,20 @@ function(_get_common_test_compile_options output_var
flags)
# list(APPEND compile_options "-Wglobal-constructors")
# endif()
endif()
- if (LIBC_TARGET_ARCHITECTURE_IS_GPU)
-# TODO: Set these flags
-# list(APPEND compile_options "-nogpulib")
-# list(APPEND compile_options "-fvisibility=hidden")
-# list(APPEND compile_options "-fconvergent-functions")
-
-# # Manually disable all standard include paths and include the resource
-# # directory to prevent system headers from being included.
-# list(APPEND compile_options "-isystem${COMPILER_RESOURCE_DIR}/include")
-# list(APPEND compile_options "-nostdinc")
- endif()
set(${output_var} ${compile_options} PARENT_SCOPE)
endfunction()
-
-# Obtains NVPTX specific arguments for compilation.
-# The PTX feature is primarily based on the CUDA toolchain version. We want to
-# be able to target NVPTX without an existing CUDA installation, so we need to
-# set this manually. This simply sets the PTX feature to the minimum required
-# for the features we wish to use on that target. The minimum PTX features used
-# here roughly corresponds to the CUDA 9.0 release.
-# Adjust as needed for desired PTX features.
-function(get_nvptx_compile_options output_var gpu_arch)
- set(nvptx_options "")
- list(APPEND nvptx_options "-march=${gpu_arch}")
- list(APPEND nvptx_options "-Wno-unknown-cuda-version")
- list(APPEND nvptx_options "SHELL:-mllvm -nvptx-emit-init-fini-kernel=false")
- if(${gpu_arch} STREQUAL "sm_35")
-list(APPEND nvptx_options "--cuda-feature=+ptx63")
- elseif(${gpu_arch} STREQUAL "sm_37")
-list(APPEND nvptx_options "--cuda-feature=+ptx63")
- elseif(${gpu_arch} STREQUAL "sm_50")
-list(APPEND nvptx_options "--cuda-feature=+ptx63")
- elseif(${gpu_arch} STREQUAL "sm_52")
-list(APPEND nvptx_options "--cuda-feature=+ptx63")
- elseif(${gpu_arch} STREQUAL "sm_53")
-list(APPEND nvptx_options "--cuda-feature=+ptx63")
- elseif(${gpu_arch} STREQUAL "sm_60")
-list(APPEND nvptx_options "--cuda-feature=+ptx63")
- elseif(${gpu_arch} STREQUAL "sm_61")
-list(APPEND nvptx_options "--cuda-feature=+ptx63")
- elseif(${gpu_arch} STREQUAL "sm_62")
-list(APPEND nvptx_options "--cuda-feature=+ptx63")
- elseif(${gpu_arch} STREQUAL "sm_70")
-list(APPEND nvptx_options "--cuda-feature=+ptx63")
- elseif(${gpu_arch} STREQUAL "sm_72")
-list(APPEND nvptx_options "--cuda-feature=+ptx63")
- elseif(${gpu_arch} STREQUAL "sm_75")
-list(APPEND nvptx_options "--cuda-feature=+ptx63")
- elseif(${gpu_arch} STREQUAL "sm_80")
-list(APPEND nvptx_options "--cuda-feature=+ptx72")
- elseif(${gpu_arch} STREQUAL "sm_86")
-list(APPEND nvptx_options "--cuda-feature=+ptx72")
- elseif(${gpu_arch} STREQUAL "sm_89")
-list(APPEND nvptx_options "--cuda-feature=+ptx72")
- elseif(${gpu_arch} STREQUAL "sm_90")
-list(APPEND nvptx_options "--cuda-feature=+ptx72")
- else()
-message(FATAL_ERROR "Unknown Nvidia GPU architecture '${gpu_arch}'")
+function(_get_hermetic_test_compile_options output_var flags)
jhuber6 wrote:
Out of scope, I was just rebasing on top of the recent changes so it worked.
That can be fixed in a follow-up.
https://github.com/llvm/llvm-project/pull/81921
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [libc] [llvm] [openmp] [libc] Rework the GPU build to be a regular target (PR #81921)
@@ -135,86 +147,20 @@ function(_get_common_test_compile_options output_var
flags)
# list(APPEND compile_options "-Wglobal-constructors")
# endif()
endif()
- if (LIBC_TARGET_ARCHITECTURE_IS_GPU)
-# TODO: Set these flags
-# list(APPEND compile_options "-nogpulib")
-# list(APPEND compile_options "-fvisibility=hidden")
-# list(APPEND compile_options "-fconvergent-functions")
-
-# # Manually disable all standard include paths and include the resource
-# # directory to prevent system headers from being included.
-# list(APPEND compile_options "-isystem${COMPILER_RESOURCE_DIR}/include")
-# list(APPEND compile_options "-nostdinc")
- endif()
set(${output_var} ${compile_options} PARENT_SCOPE)
endfunction()
-
-# Obtains NVPTX specific arguments for compilation.
-# The PTX feature is primarily based on the CUDA toolchain version. We want to
-# be able to target NVPTX without an existing CUDA installation, so we need to
-# set this manually. This simply sets the PTX feature to the minimum required
-# for the features we wish to use on that target. The minimum PTX features used
-# here roughly corresponds to the CUDA 9.0 release.
-# Adjust as needed for desired PTX features.
-function(get_nvptx_compile_options output_var gpu_arch)
- set(nvptx_options "")
- list(APPEND nvptx_options "-march=${gpu_arch}")
- list(APPEND nvptx_options "-Wno-unknown-cuda-version")
- list(APPEND nvptx_options "SHELL:-mllvm -nvptx-emit-init-fini-kernel=false")
- if(${gpu_arch} STREQUAL "sm_35")
-list(APPEND nvptx_options "--cuda-feature=+ptx63")
- elseif(${gpu_arch} STREQUAL "sm_37")
-list(APPEND nvptx_options "--cuda-feature=+ptx63")
- elseif(${gpu_arch} STREQUAL "sm_50")
-list(APPEND nvptx_options "--cuda-feature=+ptx63")
- elseif(${gpu_arch} STREQUAL "sm_52")
-list(APPEND nvptx_options "--cuda-feature=+ptx63")
- elseif(${gpu_arch} STREQUAL "sm_53")
-list(APPEND nvptx_options "--cuda-feature=+ptx63")
- elseif(${gpu_arch} STREQUAL "sm_60")
-list(APPEND nvptx_options "--cuda-feature=+ptx63")
- elseif(${gpu_arch} STREQUAL "sm_61")
-list(APPEND nvptx_options "--cuda-feature=+ptx63")
- elseif(${gpu_arch} STREQUAL "sm_62")
-list(APPEND nvptx_options "--cuda-feature=+ptx63")
- elseif(${gpu_arch} STREQUAL "sm_70")
-list(APPEND nvptx_options "--cuda-feature=+ptx63")
- elseif(${gpu_arch} STREQUAL "sm_72")
-list(APPEND nvptx_options "--cuda-feature=+ptx63")
- elseif(${gpu_arch} STREQUAL "sm_75")
-list(APPEND nvptx_options "--cuda-feature=+ptx63")
- elseif(${gpu_arch} STREQUAL "sm_80")
-list(APPEND nvptx_options "--cuda-feature=+ptx72")
- elseif(${gpu_arch} STREQUAL "sm_86")
-list(APPEND nvptx_options "--cuda-feature=+ptx72")
- elseif(${gpu_arch} STREQUAL "sm_89")
-list(APPEND nvptx_options "--cuda-feature=+ptx72")
- elseif(${gpu_arch} STREQUAL "sm_90")
-list(APPEND nvptx_options "--cuda-feature=+ptx72")
- else()
-message(FATAL_ERROR "Unknown Nvidia GPU architecture '${gpu_arch}'")
+function(_get_hermetic_test_compile_options output_var flags)
michaelrj-google wrote:
if you have the `flags` argument you should probably call the
`_get_compile_options_from_flags` function in this as well.
https://github.com/llvm/llvm-project/pull/81921
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [libc] [llvm] [openmp] [libc] Rework the GPU build to be a regular target (PR #81921)
https://github.com/jhuber6 updated https://github.com/llvm/llvm-project/pull/81921 >From 3c4a7ea70941fbf3c8a47c0715423ae38cc25a68 Mon Sep 17 00:00:00 2001 From: Joseph Huber Date: Tue, 13 Feb 2024 21:08:02 -0600 Subject: [PATCH] [libc] Rework the GPU build to be a regular target Summary: This is a massive patch because it reworks the entire build and everything that depends on it. This is not split up because various bots would fail otherwise. I will attempt to describe the necessary changes here. This patch completely reworks how the GPU build is built and targeted. Previously, we used a standard runtimes build and handled both NVPTX and AMDGPU in a single build via multi-targeting. This added a lot of divergence in the build system and prevented us from doing various things like building for the CPU / GPU at the same time, or exporting the startup libraries or running tests without a full rebuild. The new appraoch is to handle the GPU builds as strict cross-compiling runtimes. The first step required https://github.com/llvm/llvm-project/pull/81557 to allow the `LIBC` target to build for the GPU without touching the other targets. This means that the GPU uses all the same handling as the other builds in `libc`. The new expected way to build the GPU libc is with `LLVM_LIBC_RUNTIME_TARGETS=amdgcn-amd-amdhsa;nvptx64-nvidia-cuda`. The second step was reworking how we generated the embedded GPU library by moving it into the library install step. Where we previously had one `libcgpu.a` we now have `libcgpu-amdgpu.a` and `libcgpu-nvptx.a`. This patch includes the necessary clang / OpenMP changes to make that not break the bots when this lands. We unfortunately still require that the NVPTX target has an `internal` target for tests. This is because the NVPTX target needs to do LTO for the provided version (The offloading toolchain can handle it) but cannot use it for the native toolchain which is used for making tests. This approach is vastly suprerior in every way, allowing us to treat the GPU as a standard cross-compiling target. We can now install the GPU utilities to do things like use the offload tests and other fun things. Depends on https://github.com/llvm/llvm-project/pull/81557 --- clang/lib/Driver/ToolChains/CommonArgs.cpp| 37 +- clang/test/Driver/openmp-offload-gpu.c| 14 +- libc/CMakeLists.txt | 20 +- .../cmake/modules/LLVMLibCArchitectures.cmake | 28 +- libc/cmake/modules/LLVMLibCCheckMPFR.cmake| 2 +- .../modules/LLVMLibCCompileOptionRules.cmake | 104 ++ libc/cmake/modules/LLVMLibCHeaderRules.cmake | 2 +- libc/cmake/modules/LLVMLibCLibraryRules.cmake | 141 ++-- libc/cmake/modules/LLVMLibCObjectRules.cmake | 329 -- libc/cmake/modules/LLVMLibCTestRules.cmake| 67 ++-- .../modules/prepare_libc_gpu_build.cmake | 102 ++ libc/include/CMakeLists.txt | 6 +- libc/lib/CMakeLists.txt | 42 ++- libc/src/__support/File/CMakeLists.txt| 2 +- libc/src/__support/GPU/CMakeLists.txt | 2 +- libc/src/__support/OSUtil/CMakeLists.txt | 2 +- libc/src/__support/RPC/CMakeLists.txt | 2 +- libc/src/math/CMakeLists.txt | 16 +- libc/src/math/gpu/vendor/CMakeLists.txt | 1 - libc/src/stdio/CMakeLists.txt | 2 +- libc/src/stdlib/CMakeLists.txt| 4 +- libc/src/string/CMakeLists.txt| 12 +- libc/startup/gpu/CMakeLists.txt | 35 +- libc/startup/gpu/amdgpu/CMakeLists.txt| 12 - libc/startup/gpu/nvptx/CMakeLists.txt | 11 +- libc/test/CMakeLists.txt | 6 +- libc/test/IntegrationTest/CMakeLists.txt | 16 - libc/test/UnitTest/CMakeLists.txt | 6 +- libc/test/src/__support/CMakeLists.txt| 49 +-- libc/test/src/__support/CPP/CMakeLists.txt| 2 +- libc/test/src/__support/File/CMakeLists.txt | 2 +- libc/test/src/errno/CMakeLists.txt| 2 +- libc/test/src/math/CMakeLists.txt | 20 +- libc/test/src/math/smoke/CMakeLists.txt | 8 +- libc/test/src/stdio/CMakeLists.txt| 2 +- libc/test/src/stdlib/CMakeLists.txt | 6 +- libc/test/utils/UnitTest/CMakeLists.txt | 2 +- libc/utils/CMakeLists.txt | 2 +- libc/utils/MPFRWrapper/CMakeLists.txt | 2 +- libc/utils/gpu/CMakeLists.txt | 4 +- libc/utils/gpu/loader/CMakeLists.txt | 42 ++- libc/utils/gpu/loader/amdgpu/CMakeLists.txt | 6 +- libc/utils/gpu/loader/nvptx/CMakeLists.txt| 10 +- libc/utils/gpu/server/CMakeLists.txt | 9 + llvm/CMakeLists.txt | 8 + llvm/cmake/modules/HandleLLVMOptions.cmake| 7 + llvm/runtimes/CMakeLists.txt | 6 +- openmp/libomptarget/CMakeLists.txt| 9 +- .../plugins-nextgen/common/CMakeLists.txt
[clang] [libc] [llvm] [openmp] [libc] Rework the GPU build to be a regular target (PR #81921)
https://github.com/jhuber6 updated https://github.com/llvm/llvm-project/pull/81921 >From d0f782f4db249f6be08dba5060ee403974c95fdf Mon Sep 17 00:00:00 2001 From: Joseph Huber Date: Tue, 13 Feb 2024 21:08:02 -0600 Subject: [PATCH] [libc] Rework the GPU build to be a regular target Summary: This is a massive patch because it reworks the entire build and everything that depends on it. This is not split up because various bots would fail otherwise. I will attempt to describe the necessary changes here. This patch completely reworks how the GPU build is built and targeted. Previously, we used a standard runtimes build and handled both NVPTX and AMDGPU in a single build via multi-targeting. This added a lot of divergence in the build system and prevented us from doing various things like building for the CPU / GPU at the same time, or exporting the startup libraries or running tests without a full rebuild. The new appraoch is to handle the GPU builds as strict cross-compiling runtimes. The first step required https://github.com/llvm/llvm-project/pull/81557 to allow the `LIBC` target to build for the GPU without touching the other targets. This means that the GPU uses all the same handling as the other builds in `libc`. The new expected way to build the GPU libc is with `LLVM_LIBC_RUNTIME_TARGETS=amdgcn-amd-amdhsa;nvptx64-nvidia-cuda`. The second step was reworking how we generated the embedded GPU library by moving it into the library install step. Where we previously had one `libcgpu.a` we now have `libcgpu-amdgpu.a` and `libcgpu-nvptx.a`. This patch includes the necessary clang / OpenMP changes to make that not break the bots when this lands. We unfortunately still require that the NVPTX target has an `internal` target for tests. This is because the NVPTX target needs to do LTO for the provided version (The offloading toolchain can handle it) but cannot use it for the native toolchain which is used for making tests. This approach is vastly suprerior in every way, allowing us to treat the GPU as a standard cross-compiling target. We can now install the GPU utilities to do things like use the offload tests and other fun things. Depends on https://github.com/llvm/llvm-project/pull/81557 --- clang/lib/Driver/ToolChains/CommonArgs.cpp| 37 +- clang/test/Driver/openmp-offload-gpu.c| 14 +- libc/CMakeLists.txt | 20 +- .../cmake/modules/LLVMLibCArchitectures.cmake | 28 +- libc/cmake/modules/LLVMLibCCheckMPFR.cmake| 2 +- .../modules/LLVMLibCCompileOptionRules.cmake | 104 ++--- libc/cmake/modules/LLVMLibCHeaderRules.cmake | 2 +- libc/cmake/modules/LLVMLibCLibraryRules.cmake | 141 +- libc/cmake/modules/LLVMLibCObjectRules.cmake | 416 +++--- libc/cmake/modules/LLVMLibCTestRules.cmake| 67 ++- .../modules/prepare_libc_gpu_build.cmake | 102 ++--- libc/include/CMakeLists.txt | 6 +- libc/lib/CMakeLists.txt | 42 +- libc/src/__support/File/CMakeLists.txt| 2 +- libc/src/__support/GPU/CMakeLists.txt | 2 +- libc/src/__support/OSUtil/CMakeLists.txt | 2 +- libc/src/__support/RPC/CMakeLists.txt | 2 +- libc/src/math/CMakeLists.txt | 16 +- libc/src/math/gpu/vendor/CMakeLists.txt | 1 - libc/src/stdio/CMakeLists.txt | 2 +- libc/src/stdlib/CMakeLists.txt| 4 +- libc/src/string/CMakeLists.txt| 12 +- libc/startup/gpu/CMakeLists.txt | 35 +- libc/startup/gpu/amdgpu/CMakeLists.txt| 12 - libc/startup/gpu/nvptx/CMakeLists.txt | 11 +- libc/test/CMakeLists.txt | 6 +- libc/test/IntegrationTest/CMakeLists.txt | 16 - libc/test/UnitTest/CMakeLists.txt | 6 +- libc/test/src/__support/CMakeLists.txt| 49 ++- libc/test/src/__support/CPP/CMakeLists.txt| 2 +- libc/test/src/__support/File/CMakeLists.txt | 2 +- libc/test/src/errno/CMakeLists.txt| 2 +- libc/test/src/math/CMakeLists.txt | 20 +- libc/test/src/math/smoke/CMakeLists.txt | 8 +- libc/test/src/stdio/CMakeLists.txt| 2 +- libc/test/src/stdlib/CMakeLists.txt | 6 +- libc/test/utils/UnitTest/CMakeLists.txt | 2 +- libc/utils/CMakeLists.txt | 2 +- libc/utils/MPFRWrapper/CMakeLists.txt | 2 +- libc/utils/gpu/CMakeLists.txt | 4 +- libc/utils/gpu/loader/CMakeLists.txt | 42 +- libc/utils/gpu/loader/amdgpu/CMakeLists.txt | 6 +- libc/utils/gpu/loader/nvptx/CMakeLists.txt| 10 +- libc/utils/gpu/server/CMakeLists.txt | 9 + llvm/CMakeLists.txt | 8 + llvm/cmake/modules/HandleLLVMOptions.cmake| 7 + llvm/runtimes/CMakeLists.txt | 6 +- openmp/libomptarget/CMakeLists.txt| 9 +- .../plugins-nextgen/common/CMakeLists.txt | 6
[clang] [libc] [llvm] [openmp] [libc] Rework the GPU build to be a regular target (PR #81921)
https://github.com/jhuber6 updated https://github.com/llvm/llvm-project/pull/81921 >From caf0ee274f353b6adb23c455121ec2102c260de0 Mon Sep 17 00:00:00 2001 From: Joseph Huber Date: Tue, 13 Feb 2024 21:08:02 -0600 Subject: [PATCH] [libc] Rework the GPU build to be a regular target Summary: This is a massive patch because it reworks the entire build and everything that depends on it. This is not split up because various bots would fail otherwise. I will attempt to describe the necessary changes here. This patch completely reworks how the GPU build is built and targeted. Previously, we used a standard runtimes build and handled both NVPTX and AMDGPU in a single build via multi-targeting. This added a lot of divergence in the build system and prevented us from doing various things like building for the CPU / GPU at the same time, or exporting the startup libraries or running tests without a full rebuild. The new appraoch is to handle the GPU builds as strict cross-compiling runtimes. The first step required https://github.com/llvm/llvm-project/pull/81557 to allow the `LIBC` target to build for the GPU without touching the other targets. This means that the GPU uses all the same handling as the other builds in `libc`. The new expected way to build the GPU libc is with `LLVM_LIBC_RUNTIME_TARGETS=amdgcn-amd-amdhsa;nvptx64-nvidia-cuda`. The second step was reworking how we generated the embedded GPU library by moving it into the library install step. Where we previously had one `libcgpu.a` we now have `libcgpu-amdgpu.a` and `libcgpu-nvptx.a`. This patch includes the necessary clang / OpenMP changes to make that not break the bots when this lands. We unfortunately still require that the NVPTX target has an `internal` target for tests. This is because the NVPTX target needs to do LTO for the provided version (The offloading toolchain can handle it) but cannot use it for the native toolchain which is used for making tests. This approach is vastly suprerior in every way, allowing us to treat the GPU as a standard cross-compiling target. We can now install the GPU utilities to do things like use the offload tests and other fun things. Depends on https://github.com/llvm/llvm-project/pull/81557 --- clang/lib/Driver/ToolChains/CommonArgs.cpp| 37 +- clang/test/Driver/openmp-offload-gpu.c| 14 +- libc/CMakeLists.txt | 20 +- .../cmake/modules/LLVMLibCArchitectures.cmake | 28 +- libc/cmake/modules/LLVMLibCCheckMPFR.cmake| 2 +- .../modules/LLVMLibCCompileOptionRules.cmake | 104 ++--- libc/cmake/modules/LLVMLibCHeaderRules.cmake | 2 +- libc/cmake/modules/LLVMLibCLibraryRules.cmake | 141 +- libc/cmake/modules/LLVMLibCObjectRules.cmake | 416 +++--- libc/cmake/modules/LLVMLibCTestRules.cmake| 67 ++- .../modules/prepare_libc_gpu_build.cmake | 102 ++--- libc/include/CMakeLists.txt | 6 +- libc/lib/CMakeLists.txt | 43 +- libc/src/__support/File/CMakeLists.txt| 2 +- libc/src/__support/GPU/CMakeLists.txt | 2 +- libc/src/__support/OSUtil/CMakeLists.txt | 2 +- libc/src/__support/RPC/CMakeLists.txt | 2 +- libc/src/math/CMakeLists.txt | 16 +- libc/src/math/gpu/vendor/CMakeLists.txt | 1 - libc/src/stdio/CMakeLists.txt | 2 +- libc/src/stdlib/CMakeLists.txt| 4 +- libc/src/string/CMakeLists.txt| 12 +- libc/startup/gpu/CMakeLists.txt | 35 +- libc/startup/gpu/amdgpu/CMakeLists.txt| 12 - libc/startup/gpu/nvptx/CMakeLists.txt | 11 +- libc/test/CMakeLists.txt | 6 +- libc/test/IntegrationTest/CMakeLists.txt | 16 - libc/test/UnitTest/CMakeLists.txt | 6 +- libc/test/src/__support/CMakeLists.txt| 49 ++- libc/test/src/__support/CPP/CMakeLists.txt| 2 +- libc/test/src/__support/File/CMakeLists.txt | 2 +- libc/test/src/errno/CMakeLists.txt| 2 +- libc/test/src/math/CMakeLists.txt | 20 +- libc/test/src/math/smoke/CMakeLists.txt | 8 +- libc/test/src/stdio/CMakeLists.txt| 2 +- libc/test/src/stdlib/CMakeLists.txt | 6 +- libc/test/utils/UnitTest/CMakeLists.txt | 2 +- libc/utils/CMakeLists.txt | 2 +- libc/utils/MPFRWrapper/CMakeLists.txt | 2 +- libc/utils/gpu/CMakeLists.txt | 4 +- libc/utils/gpu/loader/CMakeLists.txt | 42 +- libc/utils/gpu/loader/amdgpu/CMakeLists.txt | 6 +- libc/utils/gpu/loader/nvptx/CMakeLists.txt| 10 +- libc/utils/gpu/server/CMakeLists.txt | 9 + llvm/CMakeLists.txt | 8 + llvm/cmake/modules/HandleLLVMOptions.cmake| 7 + llvm/runtimes/CMakeLists.txt | 6 +- openmp/libomptarget/CMakeLists.txt| 9 +- .../plugins-nextgen/common/CMakeLists.txt | 6
[clang] [libc] [llvm] [openmp] [libc] Rework the GPU build to be a regular target (PR #81921)
@@ -1,12 +1,9 @@
set(libc_archive_targets "")
+ list(APPEND added_archive_targets ${archive_1})
jhuber6 wrote:
Don't know how that got there, I'll fix it.
https://github.com/llvm/llvm-project/pull/81921
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [libc] [llvm] [openmp] [libc] Rework the GPU build to be a regular target (PR #81921)
@@ -533,15 +542,14 @@ endfunction(add_integration_test)
set(LIBC_HERMETIC_TEST_COMPILE_OPTIONS ${LIBC_COMPILE_OPTIONS_DEFAULT}
michaelrj-google wrote:
it should be just the part here and the part relating to compile options in
libc object rules
https://github.com/llvm/llvm-project/pull/81921
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [libc] [llvm] [openmp] [libc] Rework the GPU build to be a regular target (PR #81921)
@@ -533,15 +542,14 @@ endfunction(add_integration_test)
set(LIBC_HERMETIC_TEST_COMPILE_OPTIONS ${LIBC_COMPILE_OPTIONS_DEFAULT}
jhuber6 wrote:
Yeah there's a lot of logic that's moved and broken after rebasing. Trying to
figure out what's changed.
https://github.com/llvm/llvm-project/pull/81921
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [libc] [llvm] [openmp] [libc] Rework the GPU build to be a regular target (PR #81921)
@@ -533,15 +542,14 @@ endfunction(add_integration_test)
set(LIBC_HERMETIC_TEST_COMPILE_OPTIONS ${LIBC_COMPILE_OPTIONS_DEFAULT}
michaelrj-google wrote:
this section was moved into `LLVMLibcCCompileOptionRules.cmake` in
https://github.com/llvm/llvm-project/pull/81917
https://github.com/llvm/llvm-project/pull/81921
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [libc] [llvm] [openmp] [libc] Rework the GPU build to be a regular target (PR #81921)
@@ -1,12 +1,9 @@
set(libc_archive_targets "")
+ list(APPEND added_archive_targets ${archive_1})
michaelrj-google wrote:
I don't think this is in the right place since `archive_1` is defined below.
https://github.com/llvm/llvm-project/pull/81921
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [libc] [llvm] [openmp] [libc] Rework the GPU build to be a regular target (PR #81921)
https://github.com/michaelrj-google commented: Overall LGTM from the libc side https://github.com/llvm/llvm-project/pull/81921 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [libc] [llvm] [openmp] [libc] Rework the GPU build to be a regular target (PR #81921)
https://github.com/michaelrj-google edited https://github.com/llvm/llvm-project/pull/81921 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [libc] [llvm] [openmp] [libc] Rework the GPU build to be a regular target (PR #81921)
https://github.com/jhuber6 updated https://github.com/llvm/llvm-project/pull/81921 >From 613402be5f027c7f5494513772d0f17dd046a3e8 Mon Sep 17 00:00:00 2001 From: Joseph Huber Date: Tue, 13 Feb 2024 21:08:02 -0600 Subject: [PATCH] [libc] Rework the GPU build to be a regular target Summary: This is a massive patch because it reworks the entire build and everything that depends on it. This is not split up because various bots would fail otherwise. I will attempt to describe the necessary changes here. This patch completely reworks how the GPU build is built and targeted. Previously, we used a standard runtimes build and handled both NVPTX and AMDGPU in a single build via multi-targeting. This added a lot of divergence in the build system and prevented us from doing various things like building for the CPU / GPU at the same time, or exporting the startup libraries or running tests without a full rebuild. The new appraoch is to handle the GPU builds as strict cross-compiling runtimes. The first step required https://github.com/llvm/llvm-project/pull/81557 to allow the `LIBC` target to build for the GPU without touching the other targets. This means that the GPU uses all the same handling as the other builds in `libc`. The new expected way to build the GPU libc is with `LLVM_LIBC_RUNTIME_TARGETS=amdgcn-amd-amdhsa;nvptx64-nvidia-cuda`. The second step was reworking how we generated the embedded GPU library by moving it into the library install step. Where we previously had one `libcgpu.a` we now have `libcgpu-amdgpu.a` and `libcgpu-nvptx.a`. This patch includes the necessary clang / OpenMP changes to make that not break the bots when this lands. We unfortunately still require that the NVPTX target has an `internal` target for tests. This is because the NVPTX target needs to do LTO for the provided version (The offloading toolchain can handle it) but cannot use it for the native toolchain which is used for making tests. This approach is vastly suprerior in every way, allowing us to treat the GPU as a standard cross-compiling target. We can now install the GPU utilities to do things like use the offload tests and other fun things. Depends on https://github.com/llvm/llvm-project/pull/81557 --- clang/lib/Driver/ToolChains/CommonArgs.cpp| 37 +- clang/test/Driver/openmp-offload-gpu.c| 14 +- libc/CMakeLists.txt | 20 +- .../cmake/modules/LLVMLibCArchitectures.cmake | 28 +- libc/cmake/modules/LLVMLibCCheckMPFR.cmake| 2 +- libc/cmake/modules/LLVMLibCHeaderRules.cmake | 2 +- libc/cmake/modules/LLVMLibCLibraryRules.cmake | 141 +-- libc/cmake/modules/LLVMLibCObjectRules.cmake | 395 -- libc/cmake/modules/LLVMLibCTestRules.cmake| 71 ++-- .../modules/prepare_libc_gpu_build.cmake | 102 ++--- libc/include/CMakeLists.txt | 6 +- libc/lib/CMakeLists.txt | 43 +- libc/src/__support/File/CMakeLists.txt| 2 +- libc/src/__support/GPU/CMakeLists.txt | 2 +- libc/src/__support/OSUtil/CMakeLists.txt | 2 +- libc/src/__support/RPC/CMakeLists.txt | 2 +- libc/src/math/CMakeLists.txt | 16 +- libc/src/math/gpu/vendor/CMakeLists.txt | 1 - libc/src/stdio/CMakeLists.txt | 2 +- libc/src/stdlib/CMakeLists.txt| 4 +- libc/src/string/CMakeLists.txt| 12 +- libc/startup/gpu/CMakeLists.txt | 35 +- libc/startup/gpu/amdgpu/CMakeLists.txt| 12 - libc/startup/gpu/nvptx/CMakeLists.txt | 11 +- libc/test/CMakeLists.txt | 6 +- libc/test/IntegrationTest/CMakeLists.txt | 16 - libc/test/UnitTest/CMakeLists.txt | 2 +- libc/test/src/__support/CMakeLists.txt| 49 ++- libc/test/src/__support/CPP/CMakeLists.txt| 2 +- libc/test/src/__support/File/CMakeLists.txt | 2 +- libc/test/src/errno/CMakeLists.txt| 2 +- libc/test/src/math/CMakeLists.txt | 20 +- libc/test/src/math/smoke/CMakeLists.txt | 8 +- libc/test/src/stdio/CMakeLists.txt| 2 +- libc/test/src/stdlib/CMakeLists.txt | 6 +- libc/test/utils/UnitTest/CMakeLists.txt | 2 +- libc/utils/CMakeLists.txt | 2 +- libc/utils/MPFRWrapper/CMakeLists.txt | 2 +- libc/utils/gpu/CMakeLists.txt | 4 +- libc/utils/gpu/loader/CMakeLists.txt | 42 +- libc/utils/gpu/loader/amdgpu/CMakeLists.txt | 6 +- libc/utils/gpu/loader/nvptx/CMakeLists.txt| 10 +- libc/utils/gpu/server/CMakeLists.txt | 9 + llvm/CMakeLists.txt | 8 + llvm/cmake/modules/HandleLLVMOptions.cmake| 7 + llvm/runtimes/CMakeLists.txt | 6 +- openmp/libomptarget/CMakeLists.txt| 9 +- .../plugins-nextgen/common/CMakeLists.txt | 6 +- .../plugins-nextgen/common/src/RPC.cpp| 3 +-
[clang] [libc] [llvm] [openmp] [libc] Rework the GPU build to be a regular target (PR #81921)
https://github.com/jhuber6 edited https://github.com/llvm/llvm-project/pull/81921 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [libc] [llvm] [openmp] [libc] Rework the GPU build to be a regular target (PR #81921)
https://github.com/jhuber6 updated https://github.com/llvm/llvm-project/pull/81921 >From 5fdaa384ebc962429950b79098dee0581c74f4f3 Mon Sep 17 00:00:00 2001 From: Joseph Huber Date: Tue, 13 Feb 2024 21:08:02 -0600 Subject: [PATCH] [libc] Rework the GPU build to be a regular target Summary: This is a massive patch because it reworks the entire build and everything that depends on it. This is not split up because various bots would fail otherwise. I will attempt to describe the necessary changes here. This patch completely reworks how the GPU build is built and targeted. Previously, we used a standard runtimes build and handled both NVPTX and AMDGPU in a single build via multi-targeting. This added a lot of divergence in the build system and prevented us from doing various things like building for the CPU / GPU at the same time, or exporting the startup libraries or running tests without a full rebuild. The new appraoch is to handle the GPU builds as strict cross-compiling runtimes. The first step required https://github.com/llvm/llvm-project/pull/81557 to allow the `LIBC` target to build for the GPU without touching the other targets. This means that the GPU uses all the same handling as the other builds in `libc`. The new expected way to build the GPU libc is with `LLVM_LIBC_RUNTIME_TARGETS=amdgcn-amd-amdhsa;nvptx64-nvidia-cuda`. The second step was reworking how we generated the embedded GPU library by moving it into the library install step. Where we previously had one `libcgpu.a` we now have `libcgpu-amdgpu.a` and `libcgpu-nvptx.a`. This patch includes the necessary clang / OpenMP changes to make that not break the bots when this lands. We unfortunately still require that the NVPTX target has an `internal` target for tests. This is because the NVPTX target needs to do LTO for the provided version (The offloading toolchain can handle it) but cannot use it for the native toolchain which is used for making tests. This approach is vastly suprerior in every way, allowing us to treat the GPU as a standard cross-compiling target. We can now install the GPU utilities to do things like use the offload tests and other fun things. Depends on https://github.com/llvm/llvm-project/pull/81557 --- clang/lib/Driver/ToolChains/CommonArgs.cpp| 37 +- clang/test/Driver/openmp-offload-gpu.c| 14 +- libc/CMakeLists.txt | 20 +- .../cmake/modules/LLVMLibCArchitectures.cmake | 28 +- libc/cmake/modules/LLVMLibCCheckMPFR.cmake| 2 +- libc/cmake/modules/LLVMLibCHeaderRules.cmake | 2 +- libc/cmake/modules/LLVMLibCLibraryRules.cmake | 141 +-- libc/cmake/modules/LLVMLibCObjectRules.cmake | 395 -- libc/cmake/modules/LLVMLibCTestRules.cmake| 71 ++-- .../modules/prepare_libc_gpu_build.cmake | 102 ++--- libc/include/CMakeLists.txt | 6 +- libc/lib/CMakeLists.txt | 43 +- libc/src/__support/File/CMakeLists.txt| 2 +- libc/src/__support/GPU/CMakeLists.txt | 2 +- libc/src/__support/OSUtil/CMakeLists.txt | 2 +- libc/src/__support/RPC/CMakeLists.txt | 2 +- libc/src/math/CMakeLists.txt | 16 +- libc/src/math/gpu/vendor/CMakeLists.txt | 1 - libc/src/stdio/CMakeLists.txt | 2 +- libc/src/stdlib/CMakeLists.txt| 4 +- libc/src/string/CMakeLists.txt| 12 +- libc/startup/gpu/CMakeLists.txt | 35 +- libc/startup/gpu/amdgpu/CMakeLists.txt| 12 - libc/startup/gpu/nvptx/CMakeLists.txt | 11 +- libc/test/CMakeLists.txt | 6 +- libc/test/IntegrationTest/CMakeLists.txt | 16 - libc/test/UnitTest/CMakeLists.txt | 2 +- libc/test/src/__support/CMakeLists.txt| 49 ++- libc/test/src/__support/CPP/CMakeLists.txt| 2 +- libc/test/src/__support/File/CMakeLists.txt | 2 +- libc/test/src/errno/CMakeLists.txt| 2 +- libc/test/src/math/CMakeLists.txt | 20 +- libc/test/src/math/smoke/CMakeLists.txt | 8 +- libc/test/src/stdio/CMakeLists.txt| 2 +- libc/test/src/stdlib/CMakeLists.txt | 6 +- libc/test/utils/UnitTest/CMakeLists.txt | 2 +- libc/utils/CMakeLists.txt | 2 +- libc/utils/MPFRWrapper/CMakeLists.txt | 2 +- libc/utils/gpu/CMakeLists.txt | 4 +- libc/utils/gpu/loader/CMakeLists.txt | 42 +- libc/utils/gpu/loader/amdgpu/CMakeLists.txt | 6 +- libc/utils/gpu/loader/nvptx/CMakeLists.txt| 10 +- libc/utils/gpu/server/CMakeLists.txt | 9 + llvm/CMakeLists.txt | 8 + llvm/cmake/modules/HandleLLVMOptions.cmake| 7 + llvm/runtimes/CMakeLists.txt | 6 +- openmp/libomptarget/CMakeLists.txt| 9 +- .../plugins-nextgen/common/CMakeLists.txt | 6 +- .../plugins-nextgen/common/src/RPC.cpp| 3 +-
[clang] [libc] [llvm] [openmp] [libc] Rework the GPU build to be a regular target (PR #81921)
llvmbot wrote: @llvm/pr-subscribers-backend-amdgpu Author: Joseph Huber (jhuber6) Changes Summary: This is a massive patch because it reworks the entire build and everything that depends on it. This is not split up because various bots would fail otherwise. I will attempt to describe the necessary changes here. This patch completely reworks how the GPU build is built and targeted. Previously, we used a standard runtimes build and handled both NVPTX and AMDGPU in a single build via multi-targeting. This added a lot of divergence in the build system and prevented us from doing various things like building for the CPU / GPU at the same time, or exporting the startup libraries or running tests without a full rebuild. The new appraoch is to handle the GPU builds as strict cross-compiling runtimes. The first step required https://github.com/llvm/llvm-project/pull/81557 to allow the `LIBC` target to build for the GPU without touching the other targets. This means that the GPU uses all the same handling as the other builds in `libc`. The new expected way to build the GPU libc is with `LLVM_LIBC_RUNTIME_TARGETS=amdgcn-amd-amdhsa;nvptx64-nvidia-cuda`. The second step was reworking how we generated the embedded GPU library by moving it into the library install step. Where we previously had one `libcgpu.a` we now have `libcgpu-amdgpu.a` and `libcgpu-nvptx.a`. This patch includes the necessary clang / OpenMP changes to make that not break the bots when this lands. We unfortunately still require that the NVPTX target has an `internal` target for tests. This is because the NVPTX target needs to do LTO for the provided version (The offloading toolchain can handle it) but cannot use it for the native toolchain which is used for making tests. This approach is vastly suprerior in every way, allowing us to treat the GPU as a standard cross-compiling target. We can now install the GPU utilities to do things like use the offload tests and other fun things. Depends on https://github.com/llvm/llvm-project/pull/81557 --- Patch is 93.74 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/81921.diff 50 Files Affected: - (modified) clang/lib/Driver/ToolChains/CommonArgs.cpp (+63-31) - (modified) clang/test/Driver/openmp-offload-gpu.c (+11-3) - (modified) libc/CMakeLists.txt (+10-10) - (modified) libc/cmake/modules/LLVMLibCArchitectures.cmake (+16-12) - (modified) libc/cmake/modules/LLVMLibCCheckMPFR.cmake (+1-1) - (modified) libc/cmake/modules/LLVMLibCHeaderRules.cmake (+1-1) - (modified) libc/cmake/modules/LLVMLibCLibraryRules.cmake (+117-24) - (modified) libc/cmake/modules/LLVMLibCObjectRules.cmake (+78-317) - (modified) libc/cmake/modules/LLVMLibCTestRules.cmake (+44-27) - (modified) libc/cmake/modules/prepare_libc_gpu_build.cmake (+36-66) - (modified) libc/include/CMakeLists.txt (+3-3) - (modified) libc/lib/CMakeLists.txt (+41-6) - (modified) libc/src/__support/File/CMakeLists.txt (+1-1) - (modified) libc/src/__support/GPU/CMakeLists.txt (+1-1) - (modified) libc/src/__support/OSUtil/CMakeLists.txt (+1-1) - (modified) libc/src/__support/RPC/CMakeLists.txt (+1-1) - (modified) libc/src/math/CMakeLists.txt (+14-2) - (modified) libc/src/math/gpu/vendor/CMakeLists.txt (-1) - (modified) libc/src/stdio/CMakeLists.txt (+1-1) - (modified) libc/src/stdlib/CMakeLists.txt (+2-2) - (modified) libc/src/string/CMakeLists.txt (+6-6) - (modified) libc/startup/gpu/CMakeLists.txt (+13-22) - (modified) libc/startup/gpu/amdgpu/CMakeLists.txt (-12) - (modified) libc/startup/gpu/nvptx/CMakeLists.txt (+2-9) - (modified) libc/test/CMakeLists.txt (+3-3) - (modified) libc/test/IntegrationTest/CMakeLists.txt (-16) - (modified) libc/test/UnitTest/CMakeLists.txt (+1-1) - (modified) libc/test/src/__support/CMakeLists.txt (+26-23) - (modified) libc/test/src/__support/CPP/CMakeLists.txt (+1-1) - (modified) libc/test/src/__support/File/CMakeLists.txt (+1-1) - (modified) libc/test/src/errno/CMakeLists.txt (+1-1) - (modified) libc/test/src/math/CMakeLists.txt (+12-8) - (modified) libc/test/src/math/smoke/CMakeLists.txt (+4-4) - (modified) libc/test/src/stdio/CMakeLists.txt (+1-1) - (modified) libc/test/src/stdlib/CMakeLists.txt (+3-3) - (modified) libc/test/utils/UnitTest/CMakeLists.txt (+1-1) - (modified) libc/utils/CMakeLists.txt (+1-1) - (modified) libc/utils/MPFRWrapper/CMakeLists.txt (+1-1) - (modified) libc/utils/gpu/CMakeLists.txt (+3-1) - (modified) libc/utils/gpu/loader/CMakeLists.txt (+26-16) - (modified) libc/utils/gpu/loader/amdgpu/CMakeLists.txt (+3-3) - (modified) libc/utils/gpu/loader/nvptx/CMakeLists.txt (+5-5) - (modified) libc/utils/gpu/server/CMakeLists.txt (+9) - (modified) llvm/CMakeLists.txt (+8) - (modified) llvm/cmake/modules/HandleLLVMOptions.cmake (+7) - (modified) llvm/runtimes/CMakeLists.txt (+3-3) - (modified) openmp/libomptarget/CMakeLists.txt (+1-8) - (modified) openmp/libomptarget/plugins-nextgen/common/CMakeLists.txt (+5-1)
[clang] [libc] [llvm] [openmp] [libc] Rework the GPU build to be a regular target (PR #81921)
https://github.com/jhuber6 created https://github.com/llvm/llvm-project/pull/81921 Summary: This is a massive patch because it reworks the entire build and everything that depends on it. This is not split up because various bots would fail otherwise. I will attempt to describe the necessary changes here. This patch completely reworks how the GPU build is built and targeted. Previously, we used a standard runtimes build and handled both NVPTX and AMDGPU in a single build via multi-targeting. This added a lot of divergence in the build system and prevented us from doing various things like building for the CPU / GPU at the same time, or exporting the startup libraries or running tests without a full rebuild. The new appraoch is to handle the GPU builds as strict cross-compiling runtimes. The first step required https://github.com/llvm/llvm-project/pull/81557 to allow the `LIBC` target to build for the GPU without touching the other targets. This means that the GPU uses all the same handling as the other builds in `libc`. The new expected way to build the GPU libc is with `LLVM_LIBC_RUNTIME_TARGETS=amdgcn-amd-amdhsa;nvptx64-nvidia-cuda`. The second step was reworking how we generated the embedded GPU library by moving it into the library install step. Where we previously had one `libcgpu.a` we now have `libcgpu-amdgpu.a` and `libcgpu-nvptx.a`. This patch includes the necessary clang / OpenMP changes to make that not break the bots when this lands. We unfortunately still require that the NVPTX target has an `internal` target for tests. This is because the NVPTX target needs to do LTO for the provided version (The offloading toolchain can handle it) but cannot use it for the native toolchain which is used for making tests. This approach is vastly suprerior in every way, allowing us to treat the GPU as a standard cross-compiling target. We can now install the GPU utilities to do things like use the offload tests and other fun things. Depends on https://github.com/llvm/llvm-project/pull/81557 >From 2f71ab85f56da2eb5d5a86bbee6479231a7129e5 Mon Sep 17 00:00:00 2001 From: Joseph Huber Date: Tue, 13 Feb 2024 21:08:02 -0600 Subject: [PATCH] [libc] Rework the GPU build to be a regular target Summary: This is a massive patch because it reworks the entire build and everything that depends on it. This is not split up because various bots would fail otherwise. I will attempt to describe the necessary changes here. This patch completely reworks how the GPU build is built and targeted. Previously, we used a standard runtimes build and handled both NVPTX and AMDGPU in a single build via multi-targeting. This added a lot of divergence in the build system and prevented us from doing various things like building for the CPU / GPU at the same time, or exporting the startup libraries or running tests without a full rebuild. The new appraoch is to handle the GPU builds as strict cross-compiling runtimes. The first step required https://github.com/llvm/llvm-project/pull/81557 to allow the `LIBC` target to build for the GPU without touching the other targets. This means that the GPU uses all the same handling as the other builds in `libc`. The new expected way to build the GPU libc is with `LLVM_LIBC_RUNTIME_TARGETS=amdgcn-amd-amdhsa;nvptx64-nvidia-cuda`. The second step was reworking how we generated the embedded GPU library by moving it into the library install step. Where we previously had one `libcgpu.a` we now have `libcgpu-amdgpu.a` and `libcgpu-nvptx.a`. This patch includes the necessary clang / OpenMP changes to make that not break the bots when this lands. We unfortunately still require that the NVPTX target has an `internal` target for tests. This is because the NVPTX target needs to do LTO for the provided version (The offloading toolchain can handle it) but cannot use it for the native toolchain which is used for making tests. This approach is vastly suprerior in every way, allowing us to treat the GPU as a standard cross-compiling target. We can now install the GPU utilities to do things like use the offload tests and other fun things. Depends on https://github.com/llvm/llvm-project/pull/81557 --- clang/lib/Driver/ToolChains/CommonArgs.cpp| 94 +++-- clang/test/Driver/openmp-offload-gpu.c| 14 +- libc/CMakeLists.txt | 20 +- .../cmake/modules/LLVMLibCArchitectures.cmake | 28 +- libc/cmake/modules/LLVMLibCCheckMPFR.cmake| 2 +- libc/cmake/modules/LLVMLibCHeaderRules.cmake | 2 +- libc/cmake/modules/LLVMLibCLibraryRules.cmake | 141 +-- libc/cmake/modules/LLVMLibCObjectRules.cmake | 395 -- libc/cmake/modules/LLVMLibCTestRules.cmake| 71 ++-- .../modules/prepare_libc_gpu_build.cmake | 102 ++--- libc/include/CMakeLists.txt | 6 +- libc/lib/CMakeLists.txt | 47 ++- libc/src/__support/File/CMakeLists.txt| 2 +- libc/src/__support/GPU/CMakeLists.txt | 2 +-
