https://github.com/jhuber6 created https://github.com/llvm/llvm-project/pull/202382
This reverts commit 7389aa2ef380ca2b64d8fa34b633e5d2a4efef0a. This reverts commit 8aafa50c7a2dfb8ca1d5cdf8980f7f2d259779f5. >From c62cc5e5ac09ba9df24ac8ceb5e7ad200b0a397d Mon Sep 17 00:00:00 2001 From: Joseph Huber <[email protected]> Date: Mon, 8 Jun 2026 11:12:16 -0500 Subject: [PATCH] Revert "[clang][sycl][nvlink] Share static library linking in Frontend/Offloading (#201253)" This reverts commit 7389aa2ef380ca2b64d8fa34b633e5d2a4efef0a. This reverts commit 8aafa50c7a2dfb8ca1d5cdf8980f7f2d259779f5. --- clang/docs/ClangSYCLLinker.rst | 80 +---- .../OffloadTools/clang-sycl-linker/basic.ll | 104 ++---- .../OffloadTools/clang-sycl-linker/link.ll | 120 +------ .../clang-sycl-linker/split-mode.ll | 6 +- .../OffloadTools/clang-sycl-linker/triple.ll | 2 - .../tools/clang-nvlink-wrapper/CMakeLists.txt | 1 - .../ClangNVLinkWrapper.cpp | 287 ++++++++++++++--- .../clang-sycl-linker/ClangSYCLLinker.cpp | 175 ++++++---- clang/tools/clang-sycl-linker/SYCLLinkOpts.td | 21 +- .../llvm/Frontend/Offloading/ArchiveLinker.h | 116 ------- .../lib/Frontend/Offloading/ArchiveLinker.cpp | 301 ------------------ llvm/lib/Frontend/Offloading/CMakeLists.txt | 1 - 12 files changed, 408 insertions(+), 806 deletions(-) delete mode 100644 llvm/include/llvm/Frontend/Offloading/ArchiveLinker.h delete mode 100644 llvm/lib/Frontend/Offloading/ArchiveLinker.cpp diff --git a/clang/docs/ClangSYCLLinker.rst b/clang/docs/ClangSYCLLinker.rst index 7cf5c05eb5a0a..c28c9fefaace3 100644 --- a/clang/docs/ClangSYCLLinker.rst +++ b/clang/docs/ClangSYCLLinker.rst @@ -50,10 +50,7 @@ be passed down to downstream AOT compilation tools like 'ocloc' and 'opencl-aot' -help-hidden Display all available options -help Display available options (--help-hidden for more) -L <dir> Add <dir> to the library search path - -l <libname> Search for library <libname> - --whole-archive Include all archive members in the link - --no-whole-archive Only include archive members that resolve undefined symbols (default) - -u <symbol> Force undefined symbol during linking + --bc-library <name> Add LLVM bitcode library <name> (with extension) to the link. A relative <name> is resolved against -L paths; an absolute path is taken as-is. --module-split-mode=<mode> Module split mode: 'source' (default), 'kernel', or 'none' --ocloc-options=<value> Options passed to ocloc for Intel GPU AOT compilation --opencl-aot-options=<value> Options passed to opencl-aot for Intel CPU AOT compilation @@ -64,58 +61,8 @@ be passed down to downstream AOT compilation tools like 'ocloc' and 'opencl-aot' -v Print verbose information -spirv-dump-device-code=<dir> Directory to dump SPIR-V IR code into -Library Linking -=============== - -Device bitcode libraries can be packaged into archive libraries (``.a`` files) -using ``llvm-ar`` and linked using the ``-l`` option: - -.. code-block:: console - - llvm-ar rc libdevice.a func1.bc func2.bc func3.bc - clang-sycl-linker input.bc -l device -L /path/to/libs - -The linker supports standard archive library search semantics: - -* ``-l <name>`` searches for ``lib<name>.a`` in the directories specified by ``-L`` -* ``-l :<exact-name>`` searches for the exact filename in the ``-L`` paths -* Absolute paths can be passed as positional arguments: ``clang-sycl-linker input.bc /path/to/libdevice.a`` - -By default, archive linking is **lazy** - only archive members (individual ``.bc`` files) -that resolve undefined symbols are extracted and linked. This happens at file -granularity: if any symbol in a ``.bc`` file is needed, all symbols in that file -are included. The linker uses a symbol-driven fixed-point algorithm: it -repeatedly scans archives to extract members that resolve currently undefined -symbols until no more extractions occur. - -To force extraction of all archive members regardless of symbol resolution, use -``--whole-archive``: - -.. code-block:: console - - clang-sycl-linker input.bc --whole-archive -l device --no-whole-archive -l other - -The ``-u <symbol>`` option can be used to force a symbol to be undefined, which -can trigger extraction of archive members that define that symbol: - -.. code-block:: console - - clang-sycl-linker input.bc -u my_init_function -l device - -Implementation --------------- - -Archive linking in ``clang-sycl-linker`` is implemented using the shared -``llvm::offloading::resolveArchiveMembers()`` API from -``llvm/lib/Frontend/Offloading/ArchiveLinker.cpp``. This same infrastructure is -also used by ``clang-nvlink-wrapper``, ensuring consistent archive linking -semantics across offloading tools. - -Examples -======== - -Basic Usage ------------ +Example +======= This tool is intended to be invoked when targeting any of the target offloading toolchains. When the --sycl-link option is passed to the clang driver, the @@ -127,24 +74,3 @@ generate the final executable. .. code-block:: console clang-sycl-linker --triple spirv64 --arch bmg_g21 input.bc - -Linking with Device Libraries ------------------------------- - -To link device bitcode libraries, first package them into archive files: - -.. code-block:: console - - # Create device library archives - llvm-ar rc libmath.a sin.bc cos.bc tan.bc - llvm-ar rc libutils.a helper1.bc helper2.bc - - # Link with lazy loading (only needed members extracted) - clang-sycl-linker --triple spirv64 kernel.bc -l math -l utils -L /path/to/libs -o kernel.spv - - # Force all members to be included from libmath.a - clang-sycl-linker --triple spirv64 kernel.bc --whole-archive -l math --no-whole-archive -l utils -L /path/to/libs -o kernel.spv - - # Use exact archive filename or absolute path - clang-sycl-linker --triple spirv64 kernel.bc -l :libmath.a -L /path/to/libs -o kernel.spv - clang-sycl-linker --triple spirv64 kernel.bc /absolute/path/libmath.a -o kernel.spv diff --git a/clang/test/OffloadTools/clang-sycl-linker/basic.ll b/clang/test/OffloadTools/clang-sycl-linker/basic.ll index 2b001dc0e1c35..bd65a35bd8384 100644 --- a/clang/test/OffloadTools/clang-sycl-linker/basic.ll +++ b/clang/test/OffloadTools/clang-sycl-linker/basic.ll @@ -22,100 +22,76 @@ ; ; Test non-existent input file ; RUN: not clang-sycl-linker %t-missing.bc -o %t.out 2>&1 | FileCheck %s --check-prefix=MISSING -; MISSING: input file not found: '{{.*}}-missing.bc' +; MISSING: Input file '{{.*}}-missing.bc' does not exist ; ; Test the dry run of a simple case to link two input files. ; Test that IMG_SPIRV image kind is set for non-AOT compilation. ; RUN: clang-sycl-linker --dry-run -v --module-split-mode=none %t/input1.bc %t/input2.bc -o %t/spirv.out 2>&1 \ ; RUN: | FileCheck %s --check-prefix=SIMPLE-FO -; SIMPLE-FO: link: inputs: {{.*}}.bc, {{.*}}.bc output: [[LLVMLINKOUT:.*]].bc +; SIMPLE-FO: link: inputs: {{.*}}.bc, {{.*}}.bc libfiles: output: [[LLVMLINKOUT:.*]].bc ; SIMPLE-FO-NEXT: LLVM backend: input: [[LLVMLINKOUT]].bc, output: {{.*}}_0.spv ; SIMPLE-FO-NEXT: sycl-bundle: image kind: spv, triple: spirv64, arch: {{$}} ; SIMPLE-FO-NOT: {{.+}} ; -; Test the dry run of a simple case with device library archive specified using --whole-archive. +; Test the dry run of a simple case with device library files specified. ; RUN: mkdir -p %t/libs -; RUN: llvm-as %t/lib1.ll -o %t/libs/lib1.bc -; RUN: llvm-as %t/lib2.ll -o %t/libs/lib2.bc -; RUN: rm -f %t/libs/libdevice.a -; RUN: llvm-ar rc %t/libs/libdevice.a %t/libs/lib1.bc %t/libs/lib2.bc -; RUN: clang-sycl-linker --dry-run -v --module-split-mode=none %t/input1.bc %t/input2.bc --library-path=%t/libs --whole-archive -l device -o a.spv 2>&1 \ +; RUN: touch %t/libs/lib1.bc +; RUN: touch %t/libs/lib2.bc +; RUN: clang-sycl-linker --dry-run -v --module-split-mode=none %t/input1.bc %t/input2.bc --library-path=%t/libs --bc-library lib1.bc --bc-library lib2.bc -o a.spv 2>&1 \ ; RUN: | FileCheck %s --check-prefix=DEVLIBS -; DEVLIBS: link: inputs: {{.*}}.bc, {{.*}}.bc, {{.*}}libdevice.a(lib1.bc), {{.*}}libdevice.a(lib2.bc) output: [[LLVMLINKOUT:.*]].bc +; DEVLIBS: link: inputs: {{.*}}.bc libfiles: {{.*}}lib1.bc, {{.*}}lib2.bc output: [[LLVMLINKOUT:.*]].bc ; DEVLIBS-NEXT: LLVM backend: input: [[LLVMLINKOUT]].bc, output: a_0.spv ; DEVLIBS-NEXT: sycl-bundle: image kind: spv, triple: spirv64, arch: {{$}} ; DEVLIBS-NOT: {{.+}} ; -; Test -L short form (joined) and -l with archive using --whole-archive. -; RUN: clang-sycl-linker --dry-run -v --module-split-mode=none %t/input1.bc -L%t/libs --whole-archive -l device -o a.spv 2>&1 \ +; Test -L short form (joined) and --bc-library= joined form. +; RUN: clang-sycl-linker --dry-run -v --module-split-mode=none %t/input1.bc -L%t/libs --bc-library=lib1.bc -o a.spv 2>&1 \ ; RUN: | FileCheck %s --check-prefix=DEVLIBS-SHORT -; DEVLIBS-SHORT: link: inputs: {{.*}}.bc, {{.*}}libdevice.a(lib1.bc), {{.*}}libdevice.a(lib2.bc) output: {{.*}}.bc +; DEVLIBS-SHORT: link: inputs: {{.*}}.bc libfiles: {{.*}}libs{{[/\\]}}lib1.bc output: {{.*}}.bc ; -; Test that search continues past the first -L when the library is not found there. libdevice.a exists only in %t/libs (the second -L). +; Test that search continues past the first -L when the library is not found there. lib1.bc exists only in %t/libs (the second -L). ; RUN: mkdir -p %t/empty -; RUN: clang-sycl-linker --dry-run -v --module-split-mode=none %t/input1.bc -L %t/empty -L %t/libs --whole-archive -l device -o a.spv 2>&1 \ +; RUN: clang-sycl-linker --dry-run -v --module-split-mode=none %t/input1.bc -L %t/empty -L %t/libs --bc-library lib1.bc -o a.spv 2>&1 \ ; RUN: | FileCheck %s --check-prefix=DEVLIBS-FALLTHROUGH -; DEVLIBS-FALLTHROUGH: link: inputs: {{.*}}.bc, {{.*}}libdevice.a(lib1.bc), {{.*}}libdevice.a(lib2.bc) output: {{.*}}.bc +; DEVLIBS-FALLTHROUGH: link: inputs: {{.*}}.bc libfiles: {{.*}}libs{{[/\\]}}lib1.bc output: {{.*}}.bc +; +; Test that -L paths are searched in order: when the same name exists in multiple -L dirs, the first one wins. +; RUN: mkdir -p %t/libs2 +; RUN: touch %t/libs/shadow.bc %t/libs2/shadow.bc +; RUN: clang-sycl-linker --dry-run -v --module-split-mode=none %t/input1.bc -L %t/libs2 -L %t/libs --bc-library shadow.bc -o a.spv 2>&1 \ +; RUN: | FileCheck %s --check-prefix=DEVLIBS-ORDER +; DEVLIBS-ORDER: link: inputs: {{.*}}.bc libfiles: {{.*}}libs2{{[/\\]}}shadow.bc output: {{.*}}.bc ; ; Test a simple case with a random file (not bitcode) as input. ; RUN: touch %t/dummy.o ; RUN: not clang-sycl-linker %t/dummy.o -o a.spv 2>&1 \ ; RUN: | FileCheck %s --check-prefix=FILETYPEERROR -; FILETYPEERROR: Unsupported file type: '{{.*}}dummy.o' -; -; Test that unsupported file type error includes buffer identifier when found inside an archive. -; Create an archive containing an unsupported file (text file instead of bitcode). -; RUN: echo "not bitcode" > %t/invalid.txt -; RUN: rm -f %t/libinvalid.a -; RUN: llvm-ar rc %t/libinvalid.a %t/invalid.txt -; RUN: not clang-sycl-linker --dry-run %t/input1.bc -L %t --whole-archive -l invalid -o a.spv 2>&1 \ -; RUN: | FileCheck %s --check-prefix=ARCHIVE-INVALID-MEMBER -; ARCHIVE-INVALID-MEMBER: Unsupported file type: '{{.*}}libinvalid.a(invalid.txt)' -; -; Test mixed archive: valid bitcode member + invalid member. -; The error should clearly identify which member is invalid. -; RUN: rm -f %t/libmixed.a -; RUN: llvm-ar rc %t/libmixed.a %t/libs/lib1.bc %t/invalid.txt -; RUN: not clang-sycl-linker --dry-run %t/input1.bc -L %t --whole-archive -l mixed -o a.spv 2>&1 \ -; RUN: | FileCheck %s --check-prefix=ARCHIVE-MIXED-INVALID -; ARCHIVE-MIXED-INVALID: Unsupported file type: '{{.*}}libmixed.a(invalid.txt)' +; FILETYPEERROR: Unsupported file type ; ; Test to see if device library related errors are emitted. -; RUN: not clang-sycl-linker --dry-run %t/input1.bc %t/input2.bc --library-path=%t/libs -l device -l nonexistent -o a.spv 2>&1 \ +; RUN: not clang-sycl-linker --dry-run %t/input1.bc %t/input2.bc --library-path=%t/libs --bc-library lib1.bc --bc-library lib2.bc --bc-library lib3.bc -o a.spv 2>&1 \ ; RUN: | FileCheck %s --check-prefix=DEVLIBSERR -; DEVLIBSERR: unable to find library -lnonexistent +; DEVLIBSERR: '{{.*}}lib3.bc' library file not found ; -; Test that there is no implicit CWD search: a bare library name without any -L +; Test that there is no implicit CWD search: a bare bitcode name without any -L ; must fail to resolve, even if a same-named file exists in the CWD. -; RUN: cd %t && not clang-sycl-linker --dry-run input1.bc -l mixed -o a.spv 2>&1 \ +; RUN: cd %t && not clang-sycl-linker --dry-run input1.bc --bc-library input1.bc -o a.spv 2>&1 \ ; RUN: | FileCheck %s --check-prefix=NO-CWD-SEARCH -; NO-CWD-SEARCH: unable to find library -lmixed +; NO-CWD-SEARCH: 'input1.bc' library file not found ; ; Test that a directory matching the requested name is not accepted as a library: -; %t/libs is a directory created above; resolving -l:libs against -L %t -; would detect it's a directory and error with the filename in the message. -; RUN: not clang-sycl-linker --dry-run %t/input1.bc -L %t -l :libs -o a.spv 2>&1 \ +; %t/libs is a directory created above; resolving --bc-library libs against -L %t +; would otherwise pick it up and fail later with a confusing bitcode-reader error. +; RUN: not clang-sycl-linker --dry-run %t/input1.bc -L %t --bc-library libs -o a.spv 2>&1 \ ; RUN: | FileCheck %s --check-prefix=NO-DIR-AS-LIB -; NO-DIR-AS-LIB: '{{.*}}libs': Is a directory -; -; Test that providing only an empty archive results in "No input files could be resolved" error -; RUN: rm -f %t/empty.a -; RUN: llvm-ar rc %t/empty.a -; RUN: not clang-sycl-linker --dry-run --whole-archive %t/empty.a -o a.spv 2>&1 \ -; RUN: | FileCheck %s --check-prefix=NO-RESOLVED-INPUT -; NO-RESOLVED-INPUT: No input files could be resolved -; -; Test that providing only a lazy archive with no extracted members results in "No input files could be resolved" error -; RUN: not clang-sycl-linker --dry-run %t/libs/libdevice.a -o a.spv 2>&1 \ -; RUN: | FileCheck %s --check-prefix=NO-RESOLVED-LAZY -; NO-RESOLVED-LAZY: No input files could be resolved +; NO-DIR-AS-LIB: 'libs' library file not found ; ; Test AOT compilation for an Intel GPU. ; Test that IMG_Object image kind is set for AOT compilation (Intel GPU). ; RUN: clang-sycl-linker --dry-run -v --module-split-mode=none -arch=bmg_g21 %t/input1.bc %t/input2.bc -o %t/aot-gpu.out 2>&1 \ ; RUN: --ocloc-options="-a -b" \ ; RUN: | FileCheck %s --check-prefix=AOT-INTEL-GPU -; AOT-INTEL-GPU: link: inputs: {{.*}}.bc, {{.*}}.bc output: [[LLVMLINKOUT:.*]].bc +; AOT-INTEL-GPU: link: inputs: {{.*}}.bc, {{.*}}.bc libfiles: output: [[LLVMLINKOUT:.*]].bc ; AOT-INTEL-GPU-NEXT: LLVM backend: input: [[LLVMLINKOUT]].bc, output: [[SPIRVTRANSLATIONOUT:.*]]_0.spv ; AOT-INTEL-GPU-NEXT: "{{.*}}ocloc{{.*}}" {{.*}}-device bmg_g21 -a -b {{.*}}-output [[SPIRVTRANSLATIONOUT]]_0.out -file [[SPIRVTRANSLATIONOUT]]_0.spv ; AOT-INTEL-GPU-NEXT: sycl-bundle: image kind: o, triple: spirv64, arch: bmg_g21 @@ -126,7 +102,7 @@ ; RUN: clang-sycl-linker --dry-run -v --module-split-mode=none -arch=graniterapids %t/input1.bc %t/input2.bc -o %t/aot-cpu.out 2>&1 \ ; RUN: --opencl-aot-options="-a -b" \ ; RUN: | FileCheck %s --check-prefix=AOT-INTEL-CPU -; AOT-INTEL-CPU: link: inputs: {{.*}}.bc, {{.*}}.bc output: [[LLVMLINKOUT:.*]].bc +; AOT-INTEL-CPU: link: inputs: {{.*}}.bc, {{.*}}.bc libfiles: output: [[LLVMLINKOUT:.*]].bc ; AOT-INTEL-CPU-NEXT: LLVM backend: input: [[LLVMLINKOUT]].bc, output: [[SPIRVTRANSLATIONOUT:.*]]_0.spv ; AOT-INTEL-CPU-NEXT: "{{.*}}opencl-aot{{.*}}" {{.*}}--device=cpu -a -b {{.*}}-o [[SPIRVTRANSLATIONOUT]]_0.out [[SPIRVTRANSLATIONOUT]]_0.spv ; AOT-INTEL-CPU-NEXT: sycl-bundle: image kind: o, triple: spirv64, arch: graniterapids @@ -188,19 +164,3 @@ target triple = "spirv64" define spir_func i32 @helper() { ret i32 0 } - -;--- lib1.ll -target datalayout = "e-i64:64-v16:16-v24:32-v32:32-v48:64-v96:128-v192:256-v256:256-v512:512-v1024:1024-n8:16:32:64-G1" -target triple = "spirv64" - -define spir_func i32 @lib1_func() { - ret i32 1 -} - -;--- lib2.ll -target datalayout = "e-i64:64-v16:16-v24:32-v32:32-v48:64-v96:128-v192:256-v256:256-v512:512-v1024:1024-n8:16:32:64-G1" -target triple = "spirv64" - -define spir_func i32 @lib2_func() { - ret i32 2 -} diff --git a/clang/test/OffloadTools/clang-sycl-linker/link.ll b/clang/test/OffloadTools/clang-sycl-linker/link.ll index 381f8ec618a1b..4114f0a3f3fb1 100644 --- a/clang/test/OffloadTools/clang-sycl-linker/link.ll +++ b/clang/test/OffloadTools/clang-sycl-linker/link.ll @@ -7,11 +7,6 @@ ; RUN: llvm-as %t/bar.ll -o %t/bar.bc ; RUN: llvm-as %t/baz.ll -o %t/baz.bc ; RUN: llvm-as %t/libfoo.ll -o %t/libfoo.bc -; RUN: llvm-as %t/addFive.ll -o %t/addFive.bc -; RUN: llvm-as %t/unusedFunc.ll -o %t/unusedFunc.bc -; RUN: rm -f %t/libfoo.a %t/libdevice.a -; RUN: llvm-ar rc %t/libfoo.a %t/libfoo.bc -; RUN: llvm-ar rc %t/libdevice.a %t/addFive.bc %t/unusedFunc.bc ; ; Test linking two input files. ; RUN: clang-sycl-linker %t/foo.bc %t/bar.bc --dry-run -o a.spv --print-linked-module 2>&1 \ @@ -27,101 +22,18 @@ ; RUN: | FileCheck %s --check-prefix=CHECK-MULTIPLE-DEFS ; CHECK-MULTIPLE-DEFS: error: Linking globals named {{.*}}bar_func1{{.*}} symbol multiply defined! ; -; Test lazy linking with an archive library: only needed members are extracted. -; foo.bc references addFive, so addFive.bc is extracted from libdevice.a. -; unusedFunc.bc is not needed, so it should NOT be extracted. -; RUN: clang-sycl-linker %t/foo.bc %t/bar.bc -l device -L %t --dry-run -o a.spv --print-linked-module 2>&1 \ -; RUN: | FileCheck %s --check-prefix=CHECK-LAZY-LINK -; CHECK-LAZY-LINK: define {{.*}}foo_func1{{.*}} -; CHECK-LAZY-LINK: define {{.*}}foo_func2{{.*}} -; CHECK-LAZY-LINK: define {{.*}}bar_func1{{.*}} -; CHECK-LAZY-LINK: define {{.*}}addFive{{.*}} -; CHECK-LAZY-LINK-NOT: define {{.*}}unusedFunc{{.*}} -; -; Test linking with an archive library file using -l:libname.a syntax. -; Archive linking extracts members at file granularity, so all functions in libfoo.bc are included. -; RUN: clang-sycl-linker %t/foo.bc %t/bar.bc -l :libfoo.a -L %t --dry-run -o a.spv --print-linked-module 2>&1 \ +; Test linking with a BC library file resolved through -L. +; RUN: clang-sycl-linker %t/foo.bc %t/bar.bc --bc-library libfoo.bc -L %t --dry-run -o a.spv --print-linked-module 2>&1 \ ; RUN: | FileCheck %s --check-prefix=CHECK-DEVICE-LIB ; CHECK-DEVICE-LIB: define {{.*}}foo_func1{{.*}} ; CHECK-DEVICE-LIB: define {{.*}}foo_func2{{.*}} ; CHECK-DEVICE-LIB: define {{.*}}bar_func1{{.*}} ; CHECK-DEVICE-LIB: define {{.*}}addFive{{.*}} -; CHECK-DEVICE-LIB: define {{.*}}unusedFunc{{.*}} -; -; Test that an absolute path as a positional argument still performs lazy member extraction. -; libdevice.a has two members (addFive.bc and unusedFunc.bc). -; Since foo.bc needs addFive, only addFive.bc member is extracted; unusedFunc.bc is not. -; RUN: clang-sycl-linker %t/foo.bc %t/bar.bc %t/libdevice.a --dry-run -o a.spv --print-linked-module 2>&1 \ -; RUN: | FileCheck %s --check-prefix=CHECK-DEVICE-LIB-POS -; CHECK-DEVICE-LIB-POS: define {{.*}}foo_func1{{.*}} -; CHECK-DEVICE-LIB-POS: define {{.*}}foo_func2{{.*}} -; CHECK-DEVICE-LIB-POS: define {{.*}}bar_func1{{.*}} -; CHECK-DEVICE-LIB-POS: define {{.*}}addFive{{.*}} -; CHECK-DEVICE-LIB-POS-NOT: define {{.*}}unusedFunc{{.*}} -; -; Test that -L paths are searched in order: when the same name exists in multiple -L dirs, the first one wins. -; RUN: mkdir -p %t/libs1 %t/libs2 -; RUN: rm -f %t/libs1/libshadow.a %t/libs2/libshadow.a -; RUN: llvm-ar rc %t/libs1/libshadow.a %t/addFive.bc -; RUN: llvm-ar rc %t/libs2/libshadow.a %t/unusedFunc.bc -; RUN: clang-sycl-linker %t/foo.bc -L %t/libs2 -L %t/libs1 --whole-archive -l shadow --dry-run -o a.spv --print-linked-module 2>&1 \ -; RUN: | FileCheck %s --check-prefix=CHECK-LIB-ORDER -; CHECK-LIB-ORDER: define {{.*}}unusedFunc -; CHECK-LIB-ORDER-NOT: define {{.*}}addFive -; -; Test that -u forces extraction of an otherwise-unreferenced archive member. -; Without -u, unusedFunc is not extracted. With -u unusedFunc, it is pulled in. -; RUN: clang-sycl-linker %t/bar.bc %t/libdevice.a --dry-run -o a.spv --print-linked-module 2>&1 \ -; RUN: | FileCheck %s --check-prefix=CHECK-NO-FORCE-UNDEF -; CHECK-NO-FORCE-UNDEF: define {{.*}}bar_func1{{.*}} -; CHECK-NO-FORCE-UNDEF-NOT: define {{.*}}unusedFunc{{.*}} -; CHECK-NO-FORCE-UNDEF-NOT: define {{.*}}addFive{{.*}} -; -; RUN: clang-sycl-linker %t/bar.bc %t/libdevice.a -u unusedFunc --dry-run -o a.spv --print-linked-module 2>&1 \ -; RUN: | FileCheck %s --check-prefix=CHECK-FORCE-UNDEF -; CHECK-FORCE-UNDEF: define {{.*}}bar_func1{{.*}} -; CHECK-FORCE-UNDEF: define {{.*}}unusedFunc{{.*}} -; CHECK-FORCE-UNDEF-NOT: define {{.*}}addFive{{.*}} -; -; Test that multiple -u flags work correctly and extract all specified members. -; RUN: clang-sycl-linker %t/bar.bc %t/libdevice.a -u unusedFunc -u addFive --dry-run -o a.spv --print-linked-module 2>&1 \ -; RUN: | FileCheck %s --check-prefix=CHECK-MULTI-UNDEF -; CHECK-MULTI-UNDEF: define {{.*}}bar_func1{{.*}} -; CHECK-MULTI-UNDEF: define {{.*}}addFive{{.*}} -; CHECK-MULTI-UNDEF: define {{.*}}unusedFunc{{.*}} -; -; Test that -u works correctly with -l library syntax (not just positional archives). -; RUN: clang-sycl-linker %t/bar.bc -l device -L %t -u unusedFunc --dry-run -o a.spv --print-linked-module 2>&1 \ -; RUN: | FileCheck %s --check-prefix=CHECK-UNDEF-WITH-L -; CHECK-UNDEF-WITH-L: define {{.*}}bar_func1{{.*}} -; CHECK-UNDEF-WITH-L: define {{.*}}unusedFunc{{.*}} -; CHECK-UNDEF-WITH-L-NOT: define {{.*}}addFive{{.*}} +; CHECK-DEVICE-LIB-NOT: define {{.*}}unusedFunc{{.*}} ; -; Test that -u combined with actual references works correctly (both should be extracted). -; foo.bc references addFive, and -u forces unusedFunc. -; RUN: clang-sycl-linker %t/foo.bc %t/bar.bc %t/libdevice.a -u unusedFunc --dry-run -o a.spv --print-linked-module 2>&1 \ -; RUN: | FileCheck %s --check-prefix=CHECK-UNDEF-PLUS-REF -; CHECK-UNDEF-PLUS-REF: define {{.*}}foo_func1{{.*}} -; CHECK-UNDEF-PLUS-REF: define {{.*}}bar_func1{{.*}} -; CHECK-UNDEF-PLUS-REF: define {{.*}}addFive{{.*}} -; CHECK-UNDEF-PLUS-REF: define {{.*}}unusedFunc{{.*}} -; -; Regression test: -u symbol should remain undefined until resolved by archive member. -; This test verifies the fix for the bug where forced-undefined entries were overwritten -; before ResolvesReference was computed, making -u ineffective. -; RUN: clang-sycl-linker %t/bar.bc -u addFive %t/libdevice.a --dry-run -o a.spv --print-linked-module 2>&1 \ -; RUN: | FileCheck %s --check-prefix=CHECK-UNDEF-REMAINS -; CHECK-UNDEF-REMAINS: define {{.*}}bar_func1{{.*}} -; CHECK-UNDEF-REMAINS: define {{.*}}addFive{{.*}} -; CHECK-UNDEF-REMAINS-NOT: define {{.*}}unusedFunc{{.*}} -; -; Test -u with archive processed BEFORE the symbol table has been populated by regular inputs. -; This specifically tests that the forced-undefined placeholder survives initial processing. -; RUN: clang-sycl-linker -u addFive %t/libdevice.a %t/bar.bc --dry-run -o a.spv --print-linked-module 2>&1 \ -; RUN: | FileCheck %s --check-prefix=CHECK-UNDEF-FIRST -; CHECK-UNDEF-FIRST: define {{.*}}addFive{{.*}} -; CHECK-UNDEF-FIRST: define {{.*}}bar_func1{{.*}} -; CHECK-UNDEF-FIRST-NOT: define {{.*}}unusedFunc{{.*}} +; Test that an absolute path to --bc-library is taken as-is, with no -L required. +; RUN: clang-sycl-linker %t/foo.bc %t/bar.bc --bc-library %t/libfoo.bc --dry-run -o a.spv --print-linked-module 2>&1 \ +; RUN: | FileCheck %s --check-prefix=CHECK-DEVICE-LIB ;--- foo.ll target datalayout = "e-i64:64-v16:16-v24:32-v32:32-v48:64-v96:128-v192:256-v256:256-v512:512-v1024:1024-n8:16:32:64-G1" @@ -181,23 +93,3 @@ entry: %res = mul nsw i32 %a, 5 ret i32 %res } - -;--- addFive.ll -target datalayout = "e-i64:64-v16:16-v24:32-v32:32-v48:64-v96:128-v192:256-v256:256-v512:512-v1024:1024-n8:16:32:64-G1" -target triple = "spirv64" - -define spir_func i32 @addFive(i32 %a) { -entry: - %res = add nsw i32 %a, 5 - ret i32 %res -} - -;--- unusedFunc.ll -target datalayout = "e-i64:64-v16:16-v24:32-v32:32-v48:64-v96:128-v192:256-v256:256-v512:512-v1024:1024-n8:16:32:64-G1" -target triple = "spirv64" - -define spir_func i32 @unusedFunc(i32 %a) { -entry: - %res = mul nsw i32 %a, 5 - ret i32 %res -} diff --git a/clang/test/OffloadTools/clang-sycl-linker/split-mode.ll b/clang/test/OffloadTools/clang-sycl-linker/split-mode.ll index 2def1e6d4d066..d10dbacf259fe 100644 --- a/clang/test/OffloadTools/clang-sycl-linker/split-mode.ll +++ b/clang/test/OffloadTools/clang-sycl-linker/split-mode.ll @@ -12,7 +12,7 @@ ; Test the split mode ("none"): kernels from different TUs are not split into separate images. ; RUN: clang-sycl-linker --dry-run -v --module-split-mode=none %t.bc -o %t-none.out 2>&1 \ ; RUN: | FileCheck %s --check-prefix=SPLIT-NONE -; SPLIT-NONE: link: inputs: {{.*}}.bc output: [[LLVMLINKOUT:.*]].bc +; SPLIT-NONE: link: inputs: {{.*}}.bc libfiles: output: [[LLVMLINKOUT:.*]].bc ; SPLIT-NONE-NEXT: LLVM backend: input: [[LLVMLINKOUT]].bc, output: {{.*}}_0.spv ; SPLIT-NONE-NEXT: sycl-bundle: image kind: spv, triple: spirv64, arch: {{$}} ; SPLIT-NONE-NOT: {{.+}} @@ -20,7 +20,7 @@ ; Test the split mode ("kernel"): each SPIR_KERNEL function produces its own device image. ; RUN: clang-sycl-linker --dry-run -v --module-split-mode=kernel %t.bc -o %t-split-kernel.out 2>&1 \ ; RUN: | FileCheck %s --check-prefix=SPLIT-KERNEL -; SPLIT-KERNEL: link: inputs: {{.*}}.bc output: [[LLVMLINKOUT:.*]].bc +; SPLIT-KERNEL: link: inputs: {{.*}}.bc libfiles: output: [[LLVMLINKOUT:.*]].bc ; SPLIT-KERNEL-NEXT: sycl-module-split: input: [[LLVMLINKOUT]].bc, mode: kernel ; SPLIT-KERNEL-NEXT: [[SPLIT0:.*]].bc [kernel_c ] ; SPLIT-KERNEL-NEXT: [[SPLIT1:.*]].bc [kernel_b ] @@ -43,7 +43,7 @@ ; Test per-TU split ('source' explicitly provided) ; RUN: clang-sycl-linker --dry-run -v --module-split-mode=source %t.bc -o %t-src.out 2>&1 \ ; RUN: | FileCheck %s --check-prefix=SPLIT-SRC -; SPLIT-SRC: link: inputs: {{.*}}.bc output: [[LLVMLINKOUT:.*]].bc +; SPLIT-SRC: link: inputs: {{.*}}.bc libfiles: output: [[LLVMLINKOUT:.*]].bc ; SPLIT-SRC-NEXT: sycl-module-split: input: [[LLVMLINKOUT]].bc, mode: source ; SPLIT-SRC-NEXT: [[S0:.*]].bc [kernel_b kernel_c ] ; SPLIT-SRC-NEXT: [[S1:.*]].bc [kernel_a ] diff --git a/clang/test/OffloadTools/clang-sycl-linker/triple.ll b/clang/test/OffloadTools/clang-sycl-linker/triple.ll index 022a43fb34db2..222930987ce16 100644 --- a/clang/test/OffloadTools/clang-sycl-linker/triple.ll +++ b/clang/test/OffloadTools/clang-sycl-linker/triple.ll @@ -63,8 +63,6 @@ define spir_kernel void @kernel_c() #0 { attributes #0 = { "sycl-module-id"="TU3.cpp" } ;--- no-triple.ll -target datalayout = "e-i64:64-v16:16-v24:32-v32:32-v48:64-v96:128-v192:256-v256:256-v512:512-v1024:1024-n8:16:32:64-G1" - define spir_kernel void @kernel_d() #0 { ret void } diff --git a/clang/tools/clang-nvlink-wrapper/CMakeLists.txt b/clang/tools/clang-nvlink-wrapper/CMakeLists.txt index 8df5e4294755f..846fa952ba58d 100644 --- a/clang/tools/clang-nvlink-wrapper/CMakeLists.txt +++ b/clang/tools/clang-nvlink-wrapper/CMakeLists.txt @@ -3,7 +3,6 @@ set(LLVM_LINK_COMPONENTS BitWriter Core BinaryFormat - FrontendOffloading MC Target TransformUtils diff --git a/clang/tools/clang-nvlink-wrapper/ClangNVLinkWrapper.cpp b/clang/tools/clang-nvlink-wrapper/ClangNVLinkWrapper.cpp index 98cd5daafc9a0..70178568f76c6 100644 --- a/clang/tools/clang-nvlink-wrapper/ClangNVLinkWrapper.cpp +++ b/clang/tools/clang-nvlink-wrapper/ClangNVLinkWrapper.cpp @@ -20,7 +20,6 @@ #include "llvm/BinaryFormat/Magic.h" #include "llvm/Bitcode/BitcodeWriter.h" #include "llvm/CodeGen/CommandFlags.h" -#include "llvm/Frontend/Offloading/ArchiveLinker.h" #include "llvm/IR/DiagnosticPrinter.h" #include "llvm/LTO/LTO.h" #include "llvm/Object/Archive.h" @@ -207,6 +206,47 @@ Expected<std::string> findProgram(const ArgList &Args, StringRef Name, return *Path; } +std::optional<std::string> findFile(StringRef Dir, StringRef Root, + const Twine &Name) { + SmallString<128> Path; + if (Dir.starts_with("=")) + sys::path::append(Path, Root, Dir.substr(1), Name); + else + sys::path::append(Path, Dir, Name); + + if (sys::fs::exists(Path)) + return static_cast<std::string>(Path); + return std::nullopt; +} + +std::optional<std::string> +findFromSearchPaths(StringRef Name, StringRef Root, + ArrayRef<StringRef> SearchPaths) { + for (StringRef Dir : SearchPaths) + if (std::optional<std::string> File = findFile(Dir, Root, Name)) + return File; + return std::nullopt; +} + +std::optional<std::string> +searchLibraryBaseName(StringRef Name, StringRef Root, + ArrayRef<StringRef> SearchPaths) { + for (StringRef Dir : SearchPaths) + if (std::optional<std::string> File = + findFile(Dir, Root, "lib" + Name + ".a")) + return File; + return std::nullopt; +} + +/// Search for static libraries in the linker's library path given input like +/// `-lfoo` or `-l:libfoo.a`. +std::optional<std::string> searchLibrary(StringRef Input, StringRef Root, + ArrayRef<StringRef> SearchPaths) { + if (Input.starts_with(":")) + return findFromSearchPaths(Input.drop_front(), Root, SearchPaths); + return searchLibraryBaseName(Input, Root, SearchPaths); +} + void printCommands(ArrayRef<StringRef> CmdArgs) { if (CmdArgs.empty()) return; @@ -215,6 +255,49 @@ void printCommands(ArrayRef<StringRef> CmdArgs) { errs() << join(std::next(CmdArgs.begin()), CmdArgs.end(), " ") << "\n"; } +/// A minimum symbol interface that provides the necessary information to +/// extract archive members and resolve LTO symbols. +struct Symbol { + enum Flags { + None = 0, + Undefined = 1 << 0, + Weak = 1 << 1, + }; + + Symbol() : File(), Flags(None), UsedInRegularObj(false) {} + Symbol(Symbol::Flags Flags) : File(), Flags(Flags), UsedInRegularObj(true) {} + + Symbol(MemoryBufferRef File, const irsymtab::Reader::SymbolRef Sym) + : File(File), Flags(0), UsedInRegularObj(false) { + if (Sym.isUndefined()) + Flags |= Undefined; + if (Sym.isWeak()) + Flags |= Weak; + } + + Symbol(MemoryBufferRef File, const SymbolRef Sym) + : File(File), Flags(0), UsedInRegularObj(false) { + auto FlagsOrErr = Sym.getFlags(); + if (!FlagsOrErr) + reportError(FlagsOrErr.takeError()); + if (*FlagsOrErr & SymbolRef::SF_Undefined) + Flags |= Undefined; + if (*FlagsOrErr & SymbolRef::SF_Weak) + Flags |= Weak; + + auto NameOrErr = Sym.getName(); + if (!NameOrErr) + reportError(NameOrErr.takeError()); + } + + bool isWeak() const { return Flags & Weak; } + bool isUndefined() const { return Flags & Undefined; } + + MemoryBufferRef File; + uint32_t Flags; + bool UsedInRegularObj; +}; + Expected<StringRef> runPTXAs(StringRef File, const ArgList &Args) { SmallVector<StringRef, 1> SearchPaths; if (Arg *A = Args.getLastArg(OPT_cuda_path_EQ)) @@ -330,10 +413,97 @@ Expected<std::unique_ptr<lto::LTO>> createLTO(const ArgList &Args) { return std::make_unique<lto::LTO>(std::move(Conf), Backend, Partitions, Kind); } +Expected<bool> getSymbolsFromBitcode(MemoryBufferRef Buffer, + StringMap<Symbol> &SymTab, bool IsLazy) { + Expected<IRSymtabFile> IRSymtabOrErr = readIRSymtab(Buffer); + if (!IRSymtabOrErr) + return IRSymtabOrErr.takeError(); + bool Extracted = !IsLazy; + StringMap<Symbol> PendingSymbols; + for (unsigned I = 0; I != IRSymtabOrErr->Mods.size(); ++I) { + for (const auto &IRSym : IRSymtabOrErr->TheReader.module_symbols(I)) { + if (IRSym.isFormatSpecific() || !IRSym.isGlobal()) + continue; + + Symbol &OldSym = !SymTab.count(IRSym.getName()) && IsLazy + ? PendingSymbols[IRSym.getName()] + : SymTab[IRSym.getName()]; + Symbol Sym = Symbol(Buffer, IRSym); + if (OldSym.File.getBuffer().empty()) + OldSym = Sym; + + bool ResolvesReference = + !Sym.isUndefined() && + (OldSym.isUndefined() || (OldSym.isWeak() && !Sym.isWeak())) && + !(OldSym.isWeak() && OldSym.isUndefined() && IsLazy); + Extracted |= ResolvesReference; + + Sym.UsedInRegularObj = OldSym.UsedInRegularObj; + if (ResolvesReference) + OldSym = Sym; + } + } + if (Extracted) + for (const auto &[Name, Symbol] : PendingSymbols) + SymTab[Name] = Symbol; + return Extracted; +} + +Expected<bool> getSymbolsFromObject(ObjectFile &ObjFile, + StringMap<Symbol> &SymTab, bool IsLazy) { + bool Extracted = !IsLazy; + StringMap<Symbol> PendingSymbols; + for (SymbolRef ObjSym : ObjFile.symbols()) { + auto NameOrErr = ObjSym.getName(); + if (!NameOrErr) + return NameOrErr.takeError(); + + Symbol &OldSym = !SymTab.count(*NameOrErr) && IsLazy + ? PendingSymbols[*NameOrErr] + : SymTab[*NameOrErr]; + Symbol Sym = Symbol(ObjFile.getMemoryBufferRef(), ObjSym); + if (OldSym.File.getBuffer().empty()) + OldSym = Sym; + + bool ResolvesReference = OldSym.isUndefined() && !Sym.isUndefined() && + (!OldSym.isWeak() || !IsLazy); + Extracted |= ResolvesReference; + + if (ResolvesReference) + OldSym = Sym; + OldSym.UsedInRegularObj = true; + } + if (Extracted) + for (const auto &[Name, Symbol] : PendingSymbols) + SymTab[Name] = Symbol; + return Extracted; +} + +Expected<bool> getSymbols(MemoryBufferRef Buffer, StringMap<Symbol> &SymTab, + bool IsLazy) { + switch (identify_magic(Buffer.getBuffer())) { + case file_magic::bitcode: { + return getSymbolsFromBitcode(Buffer, SymTab, IsLazy); + } + case file_magic::elf_relocatable: { + Expected<std::unique_ptr<ObjectFile>> ObjFile = + ObjectFile::createObjectFile(Buffer); + if (!ObjFile) + return ObjFile.takeError(); + return getSymbolsFromObject(**ObjFile, SymTab, IsLazy); + } + default: + return createStringError("Unsupported file type"); + } +} + Expected<SmallVector<StringRef>> getInput(const ArgList &Args) { - // Build input descriptors for the archive resolver - SmallVector<offloading::InputDesc> InputDescs; + SmallVector<StringRef> LibraryPaths; + for (const opt::Arg *Arg : Args.filtered(OPT_library_path)) + LibraryPaths.push_back(Arg->getValue()); + bool WholeArchive = false; + SmallVector<std::pair<std::unique_ptr<MemoryBuffer>, bool>> InputFiles; for (const opt::Arg *Arg : Args.filtered( OPT_INPUT, OPT_library, OPT_whole_archive, OPT_no_whole_archive)) { if (Arg->getOption().matches(OPT_whole_archive) || @@ -342,43 +512,84 @@ Expected<SmallVector<StringRef>> getInput(const ArgList &Args) { continue; } - offloading::InputDesc Desc; - Desc.Value = Arg->getValue(); - Desc.InputKind = Arg->getOption().matches(OPT_library) - ? offloading::InputDesc::Kind::Library - : offloading::InputDesc::Kind::File; - Desc.WholeArchive = WholeArchive; - InputDescs.push_back(Desc); + std::optional<std::string> Filename = + Arg->getOption().matches(OPT_library) + ? searchLibrary(Arg->getValue(), /*Root=*/"", LibraryPaths) + : std::string(Arg->getValue()); + + if (!Filename && Arg->getOption().matches(OPT_library)) + return createStringError("unable to find library -l%s", Arg->getValue()); + + if (!Filename || !sys::fs::exists(*Filename) || + sys::fs::is_directory(*Filename)) + continue; + + ErrorOr<std::unique_ptr<MemoryBuffer>> BufferOrErr = + MemoryBuffer::getFileOrSTDIN(*Filename); + if (std::error_code EC = BufferOrErr.getError()) + return createFileError(*Filename, EC); + + MemoryBufferRef Buffer = **BufferOrErr; + switch (identify_magic(Buffer.getBuffer())) { + case file_magic::bitcode: + case file_magic::elf_relocatable: + InputFiles.emplace_back(std::move(*BufferOrErr), /*IsLazy=*/false); + break; + case file_magic::archive: { + Expected<std::unique_ptr<object::Archive>> LibFile = + object::Archive::create(Buffer); + if (!LibFile) + return LibFile.takeError(); + Error Err = Error::success(); + for (auto Child : (*LibFile)->children(Err)) { + auto ChildBufferOrErr = Child.getMemoryBufferRef(); + if (!ChildBufferOrErr) + return ChildBufferOrErr.takeError(); + std::unique_ptr<MemoryBuffer> ChildBuffer = + MemoryBuffer::getMemBufferCopy( + ChildBufferOrErr->getBuffer(), + ChildBufferOrErr->getBufferIdentifier()); + InputFiles.emplace_back(std::move(ChildBuffer), !WholeArchive); + } + if (Err) + return Err; + break; + } + default: + return createStringError("Unsupported file type"); + } } - // Gather search paths and forced undefined symbols - SmallVector<StringRef> LibraryPaths; - for (const opt::Arg *Arg : Args.filtered(OPT_library_path)) - LibraryPaths.push_back(Arg->getValue()); + bool Extracted = true; + StringMap<Symbol> SymTab; + for (auto &Sym : Args.getAllArgValues(OPT_u)) + SymTab[Sym] = Symbol(Symbol::Undefined); + SmallVector<std::unique_ptr<MemoryBuffer>> LinkerInput; + while (Extracted) { + Extracted = false; + for (auto &[Input, IsLazy] : InputFiles) { + if (!Input) + continue; + + if (hasFatBinary(Args, *Input)) { + LinkerInput.emplace_back(std::move(Input)); + continue; + } + + // Archive members only extract if they define needed symbols. We will + // re-scan all the inputs if any files were extracted for the link job. + Expected<bool> ExtractOrErr = getSymbols(*Input, SymTab, IsLazy); + if (!ExtractOrErr) + return ExtractOrErr.takeError(); + + Extracted |= *ExtractOrErr; + if (!*ExtractOrErr) + continue; - std::vector<std::string> ForcedUndefStorage = Args.getAllArgValues(OPT_u); - SmallVector<StringRef> ForcedUndefs(ForcedUndefStorage.begin(), - ForcedUndefStorage.end()); - - // The device code we are linking targets NVPTX. Any other ELF object is a - // host "fat binary" that should be forwarded without symbol scanning. The - // --assume-device-object flag (under --dry-run) overrides this and treats - // every input as device code, so disable detection by passing no archs. - SmallVector<Triple::ArchType> DeviceArchs; - if (!(Args.hasArg(OPT_dry_run) && Args.hasArg(OPT_assume_device_object))) - DeviceArchs = {Triple::nvptx, Triple::nvptx64}; - - // Resolve archive members. - Expected<offloading::ResolvedInputs> ResolvedOrErr = - offloading::resolveArchiveMembers(InputDescs, LibraryPaths, ForcedUndefs, - "", DeviceArchs); - if (!ResolvedOrErr) - return ResolvedOrErr.takeError(); - - offloading::ResolvedInputs &Resolved = *ResolvedOrErr; - SmallVector<std::unique_ptr<MemoryBuffer>> LinkerInput = - std::move(Resolved.Buffers); - StringMap<offloading::Symbol> &SymTab = Resolved.SymTab; + LinkerInput.emplace_back(std::move(Input)); + } + } + InputFiles.clear(); // Extract any bitcode files to be passed to the LTO pipeline. SmallVector<std::unique_ptr<MemoryBuffer>> BitcodeFiles; @@ -405,7 +616,7 @@ Expected<SmallVector<StringRef>> getInput(const ArgList &Args) { size_t Idx = 0; for (auto &Sym : Symbols) { lto::SymbolResolution &Res = Resolutions[Idx++]; - offloading::Symbol ObjSym = SymTab[Sym.getName()]; + Symbol ObjSym = SymTab[Sym.getName()]; // We will use this as the prevailing symbol in LTO if it is not // undefined and it is from the file that contained the canonical // definition. diff --git a/clang/tools/clang-sycl-linker/ClangSYCLLinker.cpp b/clang/tools/clang-sycl-linker/ClangSYCLLinker.cpp index b2326f7488ab0..e5e092c4737ec 100644 --- a/clang/tools/clang-sycl-linker/ClangSYCLLinker.cpp +++ b/clang/tools/clang-sycl-linker/ClangSYCLLinker.cpp @@ -18,13 +18,11 @@ #include "clang/Basic/OffloadArch.h" #include "clang/Basic/Version.h" -#include "llvm/ADT/STLExtras.h" #include "llvm/ADT/StringExtras.h" #include "llvm/ADT/StringSwitch.h" #include "llvm/BinaryFormat/Magic.h" #include "llvm/Bitcode/BitcodeWriter.h" #include "llvm/CodeGen/CommandFlags.h" -#include "llvm/Frontend/Offloading/ArchiveLinker.h" #include "llvm/Frontend/Offloading/Utility.h" #include "llvm/IR/DiagnosticPrinter.h" #include "llvm/IR/LLVMContext.h" @@ -189,49 +187,84 @@ static Error executeCommands(StringRef ExecutablePath, return Error::success(); } -static Expected<SmallVector<std::unique_ptr<MemoryBuffer>>> -getInput(const ArgList &Args) { - // Build input descriptors for the shared archive resolver - SmallVector<offloading::InputDesc> InputDescs; - bool WholeArchive = false; - for (const opt::Arg *Arg : Args.filtered( - OPT_INPUT, OPT_library, OPT_whole_archive, OPT_no_whole_archive)) { - if (Arg->getOption().matches(OPT_whole_archive) || - Arg->getOption().matches(OPT_no_whole_archive)) { - WholeArchive = Arg->getOption().matches(OPT_whole_archive); - continue; - } - - offloading::InputDesc Desc; - Desc.Value = Arg->getValue(); - Desc.InputKind = Arg->getOption().matches(OPT_library) - ? offloading::InputDesc::Kind::Library - : offloading::InputDesc::Kind::File; - Desc.WholeArchive = WholeArchive; - InputDescs.push_back(Desc); +static Expected<SmallVector<std::string>> getInput(const ArgList &Args) { + // Collect all input bitcode files to be passed to the linking stage. + SmallVector<std::string> BitcodeFiles; + auto Inputs = Args.filtered(OPT_INPUT); + if (Inputs.empty()) + return createStringError("No input files provided"); + for (const opt::Arg *Arg : Inputs) { + StringRef Filename = Arg->getValue(); + if (!sys::fs::exists(Filename) || sys::fs::is_directory(Filename)) + return createStringError("Input file '" + Filename + "' does not exist"); + file_magic Magic; + if (auto EC = identify_magic(Filename, Magic)) + return createStringError("Failed to open file " + Filename); + // TODO: Current use case involves LLVM IR bitcode files as input. + // This will be extended to support SPIR-V IR files. + if (Magic != file_magic::bitcode) + return createStringError("Unsupported file type for '" + Filename + "'"); + BitcodeFiles.push_back(std::string(Filename)); } + return BitcodeFiles; +} - if (InputDescs.empty()) - return createStringError("No input files provided"); +/// Handle cases where input file is a LLVM IR bitcode file. +/// When clang-sycl-linker is called via clang-linker-wrapper tool, input files +/// are LLVM IR bitcode files. +// TODO: Support SPIR-V IR files. +static Expected<std::unique_ptr<Module>> getBitcodeModule(StringRef File, + LLVMContext &C) { + SMDiagnostic Err; + + auto M = getLazyIRFileModule(File, Err, C); + if (M) + return std::move(M); + return createStringError(Err.getMessage()); +} - // Gather search paths and forced undefined symbols +static std::optional<std::string> findFile(StringRef Dir, const Twine &Name) { + SmallString<128> Path(Dir); + llvm::sys::path::append(Path, Name); + if (sys::fs::exists(Path) && !sys::fs::is_directory(Path)) + return std::string(Path); + return std::nullopt; +} + +static std::optional<std::string> +searchLibrary(StringRef Name, ArrayRef<StringRef> SearchPaths) { + // An absolute path is taken as-is; -L paths are only consulted for relative + // names. + if (sys::path::is_absolute(Name)) { + if (sys::fs::exists(Name) && !sys::fs::is_directory(Name)) + return std::string(Name); + return std::nullopt; + } + for (StringRef Dir : SearchPaths) + if (std::optional<std::string> File = findFile(Dir, Name)) + return File; + return std::nullopt; +} + +/// Gather all library files. The list of files and its location are passed from +/// driver. +static Expected<SmallVector<std::string>> +getBCLibraryNames(const ArgList &Args) { SmallVector<StringRef> LibraryPaths; for (const opt::Arg *Arg : Args.filtered(OPT_library_path)) LibraryPaths.push_back(Arg->getValue()); - std::vector<std::string> ForcedUndefStorage = Args.getAllArgValues(OPT_u); - SmallVector<StringRef> ForcedUndefs(ForcedUndefStorage.begin(), - ForcedUndefStorage.end()); - - Expected<offloading::ResolvedInputs> ResolvedOrErr = - offloading::resolveArchiveMembers(InputDescs, LibraryPaths, ForcedUndefs); - if (!ResolvedOrErr) - return ResolvedOrErr.takeError(); - - if (ResolvedOrErr->Buffers.empty()) - return createStringError("No input files could be resolved"); + SmallVector<std::string> LibraryFiles; + for (const opt::Arg *Arg : Args.filtered(OPT_bc_library)) { + std::optional<std::string> LibName = + searchLibrary(Arg->getValue(), LibraryPaths); + if (!LibName) + return createStringError("'" + Twine(Arg->getValue()) + + "' library file not found"); + LibraryFiles.push_back(std::move(*LibName)); + } - return std::move(ResolvedOrErr->Buffers); + return LibraryFiles; } namespace { @@ -247,12 +280,19 @@ struct LinkResult { /// first input that supplies a triple as canonical. Issue an error if any /// triple inputs disagree. /// 2. Link all input bitcode images into one image using the linkInModule API. -static Expected<LinkResult> -linkInputs(ArrayRef<std::unique_ptr<MemoryBuffer>> InputBuffers, - const ArgList &Args, LLVMContext &C) { +/// 3. Gather all library bitcode images. +/// 4. Link all the images gathered in Step 3 with the output of Step 2 using +/// linkInModule API. LinkOnlyNeeded flag is used. +static Expected<LinkResult> linkInputs(ArrayRef<std::string> InputFiles, + const ArgList &Args, LLVMContext &C) { llvm::TimeTraceScope TimeScope("Link code"); - assert(InputBuffers.size() && "No inputs to link"); + assert(InputFiles.size() && "No inputs to link"); + + // Get all library files. + Expected<SmallVector<std::string>> BCLibFiles = getBCLibraryNames(Args); + if (!BCLibFiles) + return BCLibFiles.takeError(); // Create a new file to write the linked file to. auto BitcodeOutput = @@ -261,12 +301,11 @@ linkInputs(ArrayRef<std::unique_ptr<MemoryBuffer>> InputBuffers, return BitcodeOutput.takeError(); if (Verbose) { - std::string Inputs = llvm::join( - llvm::map_range(InputBuffers, - [](const auto &B) { return B->getBufferIdentifier(); }), - ", "); - errs() << formatv("link: inputs: {0} output: {1}\n", Inputs, - *BitcodeOutput); + std::string Inputs = llvm::join(InputFiles.begin(), InputFiles.end(), ", "); + std::string LibInputs = + llvm::join((*BCLibFiles).begin(), (*BCLibFiles).end(), ", "); + errs() << formatv("link: inputs: {0} libfiles: {1} output: {2}\n", Inputs, + LibInputs, *BitcodeOutput); } // Link input files. Resolve the target triple. @@ -275,14 +314,8 @@ linkInputs(ArrayRef<std::unique_ptr<MemoryBuffer>> InputBuffers, auto LinkerOutput = std::make_unique<Module>("linker-output", C); Linker L(*LinkerOutput); - for (const auto &Buffer : InputBuffers) { - // Check file type before attempting to parse as bitcode - file_magic Magic = identify_magic(Buffer->getBuffer()); - if (Magic != file_magic::bitcode) - return createStringError("Unsupported file type: '" + - Buffer->getBufferIdentifier() + "'"); - - auto ModOrErr = parseBitcodeFile(Buffer->getMemBufferRef(), C); + for (auto &File : InputFiles) { + auto ModOrErr = getBitcodeModule(File, C); if (!ModOrErr) return ModOrErr.takeError(); @@ -290,12 +323,11 @@ linkInputs(ArrayRef<std::unique_ptr<MemoryBuffer>> InputBuffers, if (!T.empty() && T != TargetTriple) { if (TargetTriple.empty()) { TargetTriple = T; - TripleSource = Buffer->getBufferIdentifier(); + TripleSource = File; } else { return createStringError( "conflicting target triples: '" + TargetTriple.str() + "' (from " + - TripleSource + ") vs '" + T.str() + "' (from " + - Buffer->getBufferIdentifier() + ")"); + TripleSource + ") vs '" + T.str() + "' (from " + File + ")"); } } @@ -307,6 +339,18 @@ linkInputs(ArrayRef<std::unique_ptr<MemoryBuffer>> InputBuffers, return createStringError( "Target triple must be specified or inferable from inputs"); + // Link in library files. + for (auto &File : *BCLibFiles) { + auto LibMod = getBitcodeModule(File, C); + if (!LibMod) + return LibMod.takeError(); + if ((*LibMod)->getTargetTriple() == TargetTriple) { + unsigned Flags = Linker::Flags::LinkOnlyNeeded; + if (L.linkInModule(std::move(*LibMod), Flags)) + return createStringError("Could not link IR"); + } + } + // Dump linked output for testing. if (Args.hasArg(OPT_print_linked_module)) outs() << *LinkerOutput; @@ -649,14 +693,13 @@ static bool canSkipModuleSplit(IRSplitMode Mode, const Module &M, /// 4. Optionally run AOT compilation when targeting an Intel HW arch. /// 5. Pack the resulting images into a single OffloadBinary written to the /// output file. -static Error runSYCLLink(ArrayRef<std::unique_ptr<MemoryBuffer>> Buffers, - const ArgList &Args) { +static Error runSYCLLink(ArrayRef<std::string> Files, const ArgList &Args) { llvm::TimeTraceScope TimeScope("SYCL linking"); LLVMContext C; // Link all input bitcode files and library files. - Expected<LinkResult> LinkedOrErr = linkInputs(Buffers, Args, C); + Expected<LinkResult> LinkedOrErr = linkInputs(Files, Args, C); if (!LinkedOrErr) return LinkedOrErr.takeError(); LinkResult &Result = *LinkedOrErr; @@ -806,10 +849,10 @@ int main(int argc, char **argv) { reportError(createStringError("Output file must be specified")); OutputFile = Args.getLastArgValue(OPT_o); - // Get the input buffers to pass to the linking stage. - auto BuffersOrErr = getInput(Args); - if (!BuffersOrErr) - reportError(BuffersOrErr.takeError()); + // Get the input files to pass to the linking stage. + auto FilesOrErr = getInput(Args); + if (!FilesOrErr) + reportError(FilesOrErr.takeError()); if (auto *A = Args.getLastArg(OPT_spirv_dump_device_code_EQ)) { StringRef V = A->getValue(); @@ -828,7 +871,7 @@ int main(int argc, char **argv) { } // Run SYCL linking process on the generated inputs. - if (Error Err = runSYCLLink(*BuffersOrErr, Args)) + if (Error Err = runSYCLLink(*FilesOrErr, Args)) reportError(std::move(Err)); // Remove the temporary files created. diff --git a/clang/tools/clang-sycl-linker/SYCLLinkOpts.td b/clang/tools/clang-sycl-linker/SYCLLinkOpts.td index 40f758cc7d837..e00e63aa1767d 100644 --- a/clang/tools/clang-sycl-linker/SYCLLinkOpts.td +++ b/clang/tools/clang-sycl-linker/SYCLLinkOpts.td @@ -24,21 +24,12 @@ def library_path_S : Separate<["--", "-"], "library-path">, Flags<[HelpHidden]>, def library_path_EQ : Joined<["--", "-"], "library-path=">, Flags<[HelpHidden]>, Alias<library_path>; -def library : JoinedOrSeparate<["-"], "l">, MetaVarName<"<libname>">, - HelpText<"Search for library <libname>">; -def library_S : Separate<["--", "-"], "library">, Flags<[HelpHidden]>, - Alias<library>; -def library_EQ : Joined<["--", "-"], "library=">, Flags<[HelpHidden]>, - Alias<library>; - -def whole_archive : Flag<["--", "-"], "whole-archive">, - HelpText<"Include all archive members in the link">; -def no_whole_archive : Flag<["--", "-"], "no-whole-archive">, - HelpText<"Only include archive members that resolve undefined symbols (default)">; - -def u : JoinedOrSeparate<["-"], "u">, MetaVarName<"<symbol>">, - HelpText<"Force undefined symbol during linking">; -def undefined : JoinedOrSeparate<["--"], "undefined">, Alias<u>; +def bc_library : Separate<["--", "-"], "bc-library">, MetaVarName<"<name>">, + HelpText<"Add LLVM bitcode library <name> (with extension) to the link. A " + "relative <name> is resolved against -L paths; an absolute path is " + "taken as-is.">; +def bc_library_EQ : Joined<["--", "-"], "bc-library=">, Flags<[HelpHidden]>, + Alias<bc_library>; def arch_EQ : Joined<["--", "-"], "arch=">, Flags<[LinkerOnlyOption]>, diff --git a/llvm/include/llvm/Frontend/Offloading/ArchiveLinker.h b/llvm/include/llvm/Frontend/Offloading/ArchiveLinker.h deleted file mode 100644 index 8da555e427f3b..0000000000000 --- a/llvm/include/llvm/Frontend/Offloading/ArchiveLinker.h +++ /dev/null @@ -1,116 +0,0 @@ -//===----------------------------------------------------------------------===// -// -// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. -// See https://llvm.org/LICENSE.txt for license information. -// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception -// -//===----------------------------------------------------------------------===// -// -// This file declares shared functionality for linking static libraries -// (archives) in offloading tools. It provides a symbol-driven fixed-point -// archive member selection algorithm used by both clang-nvlink-wrapper and -// clang-sycl-linker. -// -//===----------------------------------------------------------------------===// - -#ifndef LLVM_FRONTEND_OFFLOADING_ARCHIVELINKER_H -#define LLVM_FRONTEND_OFFLOADING_ARCHIVELINKER_H - -#include "llvm/ADT/ArrayRef.h" -#include "llvm/ADT/SmallVector.h" -#include "llvm/ADT/StringMap.h" -#include "llvm/ADT/StringRef.h" -#include "llvm/Object/IRSymtab.h" -#include "llvm/Object/SymbolicFile.h" -#include "llvm/Support/Error.h" -#include "llvm/Support/MemoryBufferRef.h" -#include "llvm/TargetParser/Triple.h" -#include <cstdint> -#include <memory> - -namespace llvm { -class MemoryBuffer; - -namespace object { -class SymbolRef; -} // namespace object - -namespace offloading { - -/// A minimum symbol interface that provides the necessary information to -/// extract archive members and resolve LTO symbols. -struct Symbol { - enum Flags { - None = 0, - Undefined = 1 << 0, - Weak = 1 << 1, - }; - - Symbol() : File(), SymFlags(None), UsedInRegularObj(false) {} - Symbol(Symbol::Flags F) : File(), SymFlags(F), UsedInRegularObj(true) {} - - Symbol(MemoryBufferRef File, const irsymtab::Reader::SymbolRef Sym) - : File(File), SymFlags(0), UsedInRegularObj(false) { - if (Sym.isUndefined()) - SymFlags |= Undefined; - if (Sym.isWeak()) - SymFlags |= Weak; - } - - /// Create a Symbol from an object file symbol reference. - /// Returns an error if symbol flags cannot be retrieved. - static Expected<Symbol> createFromObject(MemoryBufferRef File, - const object::SymbolRef &Sym); - - bool isWeak() const { return SymFlags & Weak; } - bool isUndefined() const { return SymFlags & Undefined; } - - MemoryBufferRef File; - uint32_t SymFlags; - bool UsedInRegularObj; -}; - -/// Description of a single input (file or library). -struct InputDesc { - enum class Kind { File, Library }; - - StringRef Value; // File path, or library name for -l (the value after -l). - Kind InputKind = Kind::File; - bool WholeArchive = false; // --whole-archive state in effect at this input. -}; - -/// Result of archive member resolution. -struct ResolvedInputs { - SmallVector<std::unique_ptr<MemoryBuffer>> - Buffers; // Members to link, in order. - StringMap<Symbol> SymTab; // Symbol table (for LTO resolution). -}; - -/// Resolve archive members from the given inputs using a symbol-driven -/// fixed-point algorithm. For each input: -/// - If it's a Library, search for lib<name>.a or :<name> in SearchPaths -/// - If it's a File, use the path directly -/// - Archives are expanded and members are lazily extracted based on symbol -/// references unless WholeArchive is true -/// - Non-archive inputs (bitcode, ELF objects) are always included -/// -/// Returns the buffers to link and the symbol table for LTO resolution. -/// -/// \param Order Positional inputs + -l libraries in order. -/// \param SearchPaths -L paths for library search. -/// \param ForcedUndefs -u symbols (may be empty). -/// \param Root Sysroot for "=" prefixed paths ("" if none). -/// \param DeviceArchs Architectures of the device code being linked. When -/// non-empty, any ELF input whose architecture is not in this list (or -/// which cannot be parsed as an object) is treated as a "fat binary" -/// and passed through without symbol scanning (e.g., nvlink's cubin -/// detection). When empty, all inputs are scanned normally. -Expected<ResolvedInputs> resolveArchiveMembers( - ArrayRef<InputDesc> Order, ArrayRef<StringRef> SearchPaths, - ArrayRef<StringRef> ForcedUndefs = {}, StringRef Root = "", - ArrayRef<Triple::ArchType> DeviceArchs = {}); - -} // namespace offloading -} // namespace llvm - -#endif // LLVM_FRONTEND_OFFLOADING_ARCHIVELINKER_H diff --git a/llvm/lib/Frontend/Offloading/ArchiveLinker.cpp b/llvm/lib/Frontend/Offloading/ArchiveLinker.cpp deleted file mode 100644 index 49b1640fada5b..0000000000000 --- a/llvm/lib/Frontend/Offloading/ArchiveLinker.cpp +++ /dev/null @@ -1,301 +0,0 @@ -//===----------------------------------------------------------------------===// -// -// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. -// See https://llvm.org/LICENSE.txt for license information. -// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception -// -//===----------------------------------------------------------------------===// -// -// This file implements shared functionality for linking static libraries -// (archives) in offloading tools. -// -//===----------------------------------------------------------------------===// - -#include "llvm/Frontend/Offloading/ArchiveLinker.h" -#include "llvm/ADT/STLExtras.h" -#include "llvm/BinaryFormat/Magic.h" -#include "llvm/Object/Archive.h" -#include "llvm/Object/IRObjectFile.h" -#include "llvm/Object/ObjectFile.h" -#include "llvm/Support/Error.h" -#include "llvm/Support/FileSystem.h" -#include "llvm/Support/MemoryBuffer.h" -#include "llvm/Support/Path.h" -#include <optional> -#include <string> - -using namespace llvm; -using namespace llvm::object; - -namespace llvm { -namespace offloading { - -Expected<Symbol> Symbol::createFromObject(MemoryBufferRef File, - const SymbolRef &Sym) { - Symbol Result; - Result.File = File; - - auto FlagsOrErr = Sym.getFlags(); - if (!FlagsOrErr) - return FlagsOrErr.takeError(); - - if (*FlagsOrErr & SymbolRef::SF_Undefined) - Result.SymFlags |= Undefined; - if (*FlagsOrErr & SymbolRef::SF_Weak) - Result.SymFlags |= Weak; - - return Result; -} - -static std::optional<std::string> findFile(StringRef Dir, StringRef Root, - const Twine &Name) { - SmallString<128> Path; - if (Dir.starts_with("=")) - sys::path::append(Path, Root, Dir.substr(1), Name); - else - sys::path::append(Path, Dir, Name); - - if (sys::fs::exists(Path)) - return static_cast<std::string>(Path); - return std::nullopt; -} - -static std::optional<std::string> -findFromSearchPaths(StringRef Name, StringRef Root, - ArrayRef<StringRef> SearchPaths) { - for (StringRef Dir : SearchPaths) - if (std::optional<std::string> File = findFile(Dir, Root, Name)) - return File; - return std::nullopt; -} - -/// Search for static libraries in the linker's library path given input like -/// `-lfoo` or `-l:libfoo.a`. -static std::optional<std::string> -searchLibrary(StringRef Input, StringRef Root, - ArrayRef<StringRef> SearchPaths) { - if (Input.starts_with(":")) - return findFromSearchPaths(Input.drop_front(), Root, SearchPaths); - SmallString<128> LibName; - ("lib" + Input + ".a").toVector(LibName); - return findFromSearchPaths(LibName, Root, SearchPaths); -} - -static Expected<bool> getSymbolsFromBitcode(MemoryBufferRef Buffer, - StringMap<Symbol> &SymTab, - bool IsLazy) { - Expected<IRSymtabFile> IRSymtabOrErr = readIRSymtab(Buffer); - if (!IRSymtabOrErr) - return IRSymtabOrErr.takeError(); - bool Extracted = !IsLazy; - StringMap<Symbol> PendingSymbols; - for (unsigned I = 0; I != IRSymtabOrErr->Mods.size(); ++I) { - for (const auto &IRSym : IRSymtabOrErr->TheReader.module_symbols(I)) { - if (IRSym.isFormatSpecific() || !IRSym.isGlobal()) - continue; - - StringMap<Symbol> &Target = - (IsLazy && !SymTab.count(IRSym.getName())) ? PendingSymbols : SymTab; - Symbol &OldSym = Target[IRSym.getName()]; - Symbol Sym = Symbol(Buffer, IRSym); - if (OldSym.SymFlags == Symbol::None) - OldSym = Sym; - - bool ResolvesReference = - !Sym.isUndefined() && - (OldSym.isUndefined() || (OldSym.isWeak() && !Sym.isWeak())) && - !(OldSym.isWeak() && OldSym.isUndefined() && IsLazy); - Extracted |= ResolvesReference; - - Sym.UsedInRegularObj = OldSym.UsedInRegularObj; - if (ResolvesReference) - OldSym = Sym; - } - } - if (Extracted) - for (const auto &[Name, Symbol] : PendingSymbols) - SymTab[Name] = Symbol; - return Extracted; -} - -static Expected<bool> getSymbolsFromObject(ObjectFile &ObjFile, - StringMap<Symbol> &SymTab, - bool IsLazy) { - bool Extracted = !IsLazy; - StringMap<Symbol> PendingSymbols; - for (SymbolRef ObjSym : ObjFile.symbols()) { - auto NameOrErr = ObjSym.getName(); - if (!NameOrErr) - return NameOrErr.takeError(); - - StringMap<Symbol> &Target = - (IsLazy && !SymTab.count(*NameOrErr)) ? PendingSymbols : SymTab; - Symbol &OldSym = Target[*NameOrErr]; - - auto SymOrErr = - Symbol::createFromObject(ObjFile.getMemoryBufferRef(), ObjSym); - if (!SymOrErr) - return SymOrErr.takeError(); - Symbol Sym = *SymOrErr; - - if (OldSym.SymFlags == Symbol::None) - OldSym = Sym; - - bool ResolvesReference = OldSym.isUndefined() && !Sym.isUndefined() && - (!OldSym.isWeak() || !IsLazy); - Extracted |= ResolvesReference; - - if (ResolvesReference) - OldSym = Sym; - OldSym.UsedInRegularObj = true; - } - if (Extracted) - for (const auto &[Name, Symbol] : PendingSymbols) - SymTab[Name] = Symbol; - return Extracted; -} - -/// Identify "fat binary" inputs that should be passed through to the linker -/// without symbol-driven extraction. An input is a fat binary if \p DeviceArchs -/// is non-empty and the input is an ELF object whose architecture is not one of -/// the device architectures (or which fails to parse as an object file). -static bool isFatBinary(MemoryBufferRef Buffer, - ArrayRef<Triple::ArchType> DeviceArchs) { - if (DeviceArchs.empty()) - return false; - if (identify_magic(Buffer.getBuffer()) != file_magic::elf_relocatable) - return false; - Expected<std::unique_ptr<ObjectFile>> ObjFile = - ObjectFile::createObjectFile(Buffer); - if (!ObjFile) { - // Assume fat binary if the object creation fails. - consumeError(ObjFile.takeError()); - return true; - } - return !llvm::is_contained(DeviceArchs, (*ObjFile)->getArch()); -} - -static Expected<bool> getSymbols(MemoryBufferRef Buffer, - StringMap<Symbol> &SymTab, bool IsLazy) { - switch (identify_magic(Buffer.getBuffer())) { - case file_magic::bitcode: { - return getSymbolsFromBitcode(Buffer, SymTab, IsLazy); - } - case file_magic::elf_relocatable: { - Expected<std::unique_ptr<ObjectFile>> ObjFile = - ObjectFile::createObjectFile(Buffer); - if (!ObjFile) - return ObjFile.takeError(); - return getSymbolsFromObject(**ObjFile, SymTab, IsLazy); - } - default: - return createStringError("Unsupported file type: '" + - Buffer.getBufferIdentifier() + "'"); - } -} - -Expected<ResolvedInputs> -resolveArchiveMembers(ArrayRef<InputDesc> Order, - ArrayRef<StringRef> SearchPaths, - ArrayRef<StringRef> ForcedUndefs, StringRef Root, - ArrayRef<Triple::ArchType> DeviceArchs) { - ResolvedInputs Result; - SmallVector<std::pair<std::unique_ptr<MemoryBuffer>, bool>> InputFiles; - - // Process each input descriptor. - for (const InputDesc &Desc : Order) { - std::optional<std::string> Filename; - - if (Desc.InputKind == InputDesc::Kind::Library) { - Filename = searchLibrary(Desc.Value, Root, SearchPaths); - if (!Filename) - return createStringError("unable to find library -l%s", - Desc.Value.str().c_str()); - if (sys::fs::is_directory(*Filename)) - return createStringError("'%s': Is a directory", Filename->c_str()); - } else { - if (!sys::fs::exists(Desc.Value)) - return createStringError("input file not found: '" + Desc.Value + "'"); - if (sys::fs::is_directory(Desc.Value)) - return createStringError("'" + Desc.Value + "': Is a directory"); - Filename = Desc.Value.str(); - } - - if (!Filename) - continue; - - auto BufferOrErr = - errorOrToExpected(MemoryBuffer::getFileOrSTDIN(*Filename)); - if (!BufferOrErr) - return createFileError(*Filename, BufferOrErr.takeError()); - - MemoryBufferRef Buffer = (*BufferOrErr)->getMemBufferRef(); - switch (identify_magic(Buffer.getBuffer())) { - case file_magic::bitcode: - case file_magic::elf_relocatable: - InputFiles.emplace_back(std::move(*BufferOrErr), /*IsLazy=*/false); - break; - case file_magic::archive: { - Expected<std::unique_ptr<object::Archive>> LibFile = - object::Archive::create(Buffer); - if (!LibFile) - return LibFile.takeError(); - Error Err = Error::success(); - for (auto Child : (*LibFile)->children(Err)) { - auto ChildBufferOrErr = Child.getMemoryBufferRef(); - if (!ChildBufferOrErr) - return ChildBufferOrErr.takeError(); - // Include archive name in buffer identifier for better diagnostics. - std::string BufferIdentifier = - (*Filename + "(" + ChildBufferOrErr->getBufferIdentifier() + ")") - .str(); - std::unique_ptr<MemoryBuffer> ChildBuffer = - MemoryBuffer::getMemBufferCopy(ChildBufferOrErr->getBuffer(), - BufferIdentifier); - InputFiles.emplace_back(std::move(ChildBuffer), !Desc.WholeArchive); - } - if (Err) - return Err; - break; - } - default: - return createStringError("Unsupported file type: '" + *Filename + "'"); - } - } - - // Seed symbol table with forced undefined symbols. - for (StringRef Sym : ForcedUndefs) - Result.SymTab[Sym] = Symbol(Symbol::Undefined); - - // Fixed-point loop to extract archive members. - bool Extracted = true; - while (Extracted) { - Extracted = false; - for (auto &[Input, IsLazy] : InputFiles) { - if (!Input) - continue; - - // Check if this is a fat binary that should be passed through. - if (isFatBinary(*Input, DeviceArchs)) { - Result.Buffers.emplace_back(std::move(Input)); - continue; - } - - // Archive members only extract if they define needed symbols. - Expected<bool> ExtractOrErr = getSymbols(*Input, Result.SymTab, IsLazy); - if (!ExtractOrErr) - return ExtractOrErr.takeError(); - - Extracted |= *ExtractOrErr; - if (!*ExtractOrErr) - continue; - - Result.Buffers.emplace_back(std::move(Input)); - } - } - - return std::move(Result); -} - -} // namespace offloading -} // namespace llvm diff --git a/llvm/lib/Frontend/Offloading/CMakeLists.txt b/llvm/lib/Frontend/Offloading/CMakeLists.txt index 82c49018b9bf3..9747dbde043da 100644 --- a/llvm/lib/Frontend/Offloading/CMakeLists.txt +++ b/llvm/lib/Frontend/Offloading/CMakeLists.txt @@ -1,5 +1,4 @@ add_llvm_component_library(LLVMFrontendOffloading - ArchiveLinker.cpp Utility.cpp OffloadWrapper.cpp PropertySet.cpp _______________________________________________ cfe-commits mailing list [email protected] https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
