Oops, I meant __declspec(dllexport) void exportedMethod() {} with braces, not brackets
On Thu, Apr 18, 2024 at 8:27 PM Julian Waters <tanksherma...@gmail.com> wrote: > > I have good (and bad news) for you! > > The good news is that from memory, ld and gcc (but I assume you are > only concerned about ld) can work entirely on cl compiled object files > with no hitch whatsoever, and partial linking is fully functional with > ld on Windows! Symbol hiding also works, but with a gotcha. > > Now, the bad news. --localize-hidden, to my knowledge, only works on > ELF visibilities. This means that for the following file: > > exports.c > __declspec(dllexport) void exportedMethod() () > void unexportedMethod() {} > > When objcopy is run over exports.o with --localize-hidden, it will > work on the cl compiled object file just fine, but it will not > recognise unexportedMethod() as a hidden method, because dllexport and > hidden visibility are entirely separate concepts to gcc. The resulting > exports-objcopy.o file will still have unexportedMethod as public. > However, not all is lost just yet. --localize-symbol=<symbol> still > works perfectly fine, and can be used as a workaround if we can find a > way to check for non-dllexport methods in cl object files and then > feed that through the command line. I should note that clang also > suffers from the same issue, as --localize-hidden on clang's objcopy > also leaves unexportedMethod as public. > > Happy to help! > > best regards, > Julian > > On Thu, Apr 18, 2024 at 6:28 PM Magnus Ihse Bursie > <magnus.ihse.bur...@oracle.com> wrote: > > > > On 2024-04-17 14:06, Julian Waters wrote: > > > > > Hi Magnus, > > > > > > Yes, I'm talking about the MSYS2 objcopy, but to a lesser extent also > > > standalone Windows objcopy builds too. objcopy should be able to > > > handle .o files from cl.exe (I assume that's what everyone here is > > > after considering all the talk about .o files?), but to answer that > > > question properly, I'd need a bit more detail. What kind of usage of > > > objcopy do you have in mind? A general-ish command line example could > > > be helpful > > > > To make a static library behave somewhat like a dynamic library, and not > > expose all its internal symbols to the rest of the world, there are > > basically two operations that needs to be done: > > > > 1) Combine all .o files into a single .o file, to make it look like it > > was compiled by a single source code. That way, symbols that were > > created in one source file and referenced in another will now behave as > > if they are internal to the "compilation unit", i.e. like they were > > declared static. > > > > 2) Modify the symbol status so that symbols that are not exported are > > changed so they look like they were actually declared "static" in the > > source code. > > > > These operations are based on concepts that exists in the gcc and clang > > toolchain, about symbol visibility. I am not sure how well they > > translate to a Microsoft setting. But, if the dll_export marker > > corresponds to visible symbols, then I guess we should be able to > > achieve something similar. > > > > What needs to be done then is: > > > > 1) Combine multiple .obj (COFF object files) into one. > > > > 2) Change the visibility of symbols that are not marked as dll_export:ed > > to they appear like they were declared static. > > > > In the clang/gcc world, the first step is done by "partial linking" by > > ld. That is our first blocker -- link.exe cannot do that. So the first > > question is really, is there a Windows build of ld that can work on COFF > > files to achieve this? > > > > The second step is done by objcopy using the "--localize-hidden" > > argument. The second question is, could this work on a COFF object file? > > > > /Magnus > > > > > > > > > > best regards, > > > Julian > > > > > > On Wed, Apr 17, 2024 at 5:55 PM Magnus Ihse Bursie > > > <magnus.ihse.bur...@oracle.com> wrote: > > >> On 2024-04-16 07:23, Julian Waters wrote: > > >> > > >>>> And finally, on top of all of this, is the question of widening the > > >>>> platform support. To support linux/gcc with objcopy is trivial, but > > >>>> the question about Windows still remain. > > >>> objcopy is also available on Windows, if the question about > > >>> alternative tooling is still unanswered :) > > >> At this point, I think support for static build on Windows is to either > > >> require additional tooling on top of the Microsoft Visual Studio > > >> toolchain, or to drop it completely, so I am definitely interested in > > >> researching alternatives. > > >> > > >> Can objcopy (I assume this is from msys?) deal with COFF files generated > > >> by cl? > > >> > > >> Switching the entire toolchain is not relevant at this point (but if a > > >> non-Microsoft toolchain build for Windows is ever integrated, it might > > >> get static builds with no extra work as a bonus), but I could certainly > > >> accept the idea of having one or a few additional tools required to get > > >> the normal Microsoft toolchain to produce static builds. > > >> > > >> /Magnus > > >> > > >>> best regards, > > >>> Julian > > >>> > > >>> On Fri, Apr 12, 2024 at 7:52 PM Magnus Ihse Bursie > > >>> <magnus.ihse.bur...@oracle.com> wrote: > > >>>> On 2024-04-02 21:16, Jiangli Zhou wrote: > > >>>> > > >>>> Hi Magnus, > > >>>> > > >>>> In today's zoom meeting with Alan, Ron, Liam and Chuck, we (briefly) > > >>>> discussed how to move forward contributing the static Java related > > >>>> changes (additional runtime fixes/enhancements on top of the existing > > >>>> static support in JDK) from > > >>>> https://github.com/openjdk/leyden/tree/hermetic-java-runtime to JDK > > >>>> mainline. > > >>>> > > >>>> Just a bit more details/context below, which may be useful for others > > >>>> reading this thread. > > >>>> > > >>>> The https://github.com/openjdk/leyden/tree/hermetic-java-runtime > > >>>> branch currently contains following for supporting hermetic Java > > >>>> (without the launcher work for runtime support): > > >>>> > > >>>> 1. Build change for linking the Java launcher (as bin/javastatic) with > > >>>> JDK/hotspot static libraries (.a), mainly in > > >>>> https://github.com/openjdk/leyden/blob/hermetic-java-runtime/make/StaticLink.gmk. > > >>>> The part for creating the complete sets of static libraries > > >>>> (including libjvm.a) has already been included in the mainline since > > >>>> last year. > > >>>> https://github.com/openjdk/leyden/blob/hermetic-java-runtime/make/StaticLink.gmk > > >>>> is in a very raw state and is intended to demonstrate the capability > > >>>> of building a static Java launcher. > > >>>> > > >>>> Indeed. It is nowhere near being able to be integrated. > > >>>> > > >>>> > > >>>> 2. Additional runtime fixes/enhancements on top of the existing static > > >>>> support in JDK, e.g. support further lookup dynamic native library if > > >>>> the built-in native library cannot be found. > > >>>> > > >>>> 3. Some initial (prototype) work on supporting hermetic JDK resource > > >>>> files in the jimage (JDK modules image). > > >>>> > > >>>> To move forward, one of the earliest items needed is to add the > > >>>> capability of building the fully statically linked Java launcher in > > >>>> JDK mainline. The other static Java runtime changes can be followed up > > >>>> after the launcher linking part, so they can be built and tested as > > >>>> individual PRs created for the JDK mainline. Magnus, you have > > >>>> expressed interest in helping get the launcher linking part (refactor > > >>>> from > > >>>> https://github.com/openjdk/leyden/blob/hermetic-java-runtime/make/StaticLink.gmk) > > >>>> into JDK mainline. What's your thought on prioritizing the launcher > > >>>> static linking part before other makefile clean ups for static > > >>>> libraries? > > >>>> > > >>>> Trust me, my absolute top priority now is working on getting the > > >>>> proper build support needed for Hermetic Java. I can't prioritize it > > >>>> any higher. > > >>>> > > >>>> I am not sure what you are asking for. We can't just merge > > >>>> StaticLink.gmk from your prototype. And even if we did, what good will > > >>>> it do you? > > >>>> > > >>>> The problem you are running into is that the build system has not been > > >>>> designed to properly support static linking. There are already 3-4 > > >>>> hacks in place to get something sort-of useful out, but they are prone > > >>>> to breaking. I assume that we agree that for Hermetic Java to become a > > >>>> success, we need to have a stable foundation for static builds. > > >>>> > > >>>> The core problem of all static linking hacks is that they are not > > >>>> integrated in the right place. They need to be a core part of what > > >>>> NativeCompilation delivers, not something done in a separate file. To > > >>>> put it in other words, StaticLink.gmk from your branch do not need > > >>>> cleanup -- it needs to go away, and the functionality moved to the > > >>>> proper place. > > >>>> > > >>>> My approach is that NativeCompilation should support doing either only > > >>>> dynamic linking (as today), or static linking (as today with > > >>>> STATIC_LIBS or STATIC_BUILD), or both. The assumption is that the > > >>>> latter will be default, or at least should be tested by default in > > >>>> GHA. For this to work, we need to compile the source code to .o files > > >>>> only once, and then link these .o files either into a dynamic or a > > >>>> static library (or both). > > >>>> > > >>>> This, in turn, require several changes: > > >>>> > > >>>> 1) The linking code needs to be cleaned up, and all technical debt > > >>>> needs to be resolved. This is what I have been doing since I started > > >>>> working on static builds for Hermetic Java. JDK-8329704 (which was > > >>>> integrated yesterday) was the first major milestone of this cleanup. > > >>>> Now, the path were to find a library created by the JDK (static or > > >>>> dynamic) is encapsulated in ResolveLibPath. This is currently a > > >>>> monster, but at least all knowledge is collected in a single location, > > >>>> instead of spread over the code base. Getting this simplified is the > > >>>> next step. > > >>>> > > >>>> 2) We need to stop passing the STATIC_BUILD define when compiling. > > >>>> This is partially addressed in your PR, where you have replaced #ifdef > > >>>> STATIC_BUILD with a dynamic lookup. But there is also the problem of > > >>>> JNI/JVMTI entry points. I have been pondering how we can compile the > > >>>> code in a way so we support both dynamic and static name resolution, > > >>>> and I think I have a solution. > > >>>> > > >>>> This is unfortunately quite complex, and I have started a discussion > > >>>> with Alan if it is possible to update the JNI spec so that both static > > >>>> and dynamic entry points can have the form > > >>>> "JNI_OnLoad_<library-name>". Ideally, I'd like to see us push for this > > >>>> with as much effort as possible. If we got this in place, static > > >>>> builds would be much easier, and the changes required for Hermetic > > >>>> Java even smaller. > > >>>> > > >>>> And finally, on top of all of this, is the question of widening the > > >>>> platform support. To support linux/gcc with objcopy is trivial, but > > >>>> the question about Windows still remain. I have two possible ways > > >>>> forward, one is to check if there is alternative tooling to use (the > > >>>> prime candidate is the clang-ldd), and the other is to try to "fake" a > > >>>> partial linking by concatenating all source code before compiling. > > >>>> This is not ideal, though, for many reasons, and I am not keen on > > >>>> implementing it, not even for testing. And at this point, I have not > > >>>> had time to investigate any of these options much further, since I > > >>>> have been focusing on 1) above. > > >>>> > > >>>> A third option is of course to just say that due to toolchain > > >>>> limitations, static linking is not available on Windows. > > >>>> > > >>>> My recommendation is that you keep on working to resolve the (much > > >>>> more thorny) issues of resource access in Hermetic Java in your > > >>>> branch, where you have a prototype static build that works for you. In > > >>>> the meantime, I will make sure that there will be a functioning, > > >>>> stable and robust way of creating static builds in the mainline, that > > >>>> can be regularly tested and not bit-rot, like the static build hacks > > >>>> that has gone in before. > > >>>> > > >>>> /Magnus > > >>>> > > >>>> > > >>>> > > >>>> Thanks! > > >>>> Jiangli > > >>>> > > >>>> On Thu, Feb 15, 2024 at 12:01 PM Jiangli Zhou <jiangliz...@google.com> > > >>>> wrote: > > >>>>> On Wed, Feb 14, 2024 at 5:07 PM Jiangli Zhou <jiangliz...@google.com> > > >>>>> wrote: > > >>>>>> Hi Magnus, > > >>>>>> > > >>>>>> Thanks for looking into this from the build perspective. > > >>>>>> > > >>>>>> On Wed, Feb 14, 2024 at 1:00 AM Magnus Ihse Bursie > > >>>>>> <magnus.ihse.bur...@oracle.com> wrote: > > >>>>>>> First some background for build-dev: I have spent some time looking > > >>>>>>> at > > >>>>>>> the build implications of the Hermetic Java effort, which is part of > > >>>>>>> Project Leyden. A high-level overview is available here: > > >>>>>>> https://cr.openjdk.org/~jiangli/hermetic_java.pdf and the current > > >>>>>>> source > > >>>>>>> code is here: > > >>>>>>> https://github.com/openjdk/leyden/tree/hermetic-java-runtime. > > >>>>>> Some additional hermetic Java related references that are also > > >>>>>> useful: > > >>>>>> > > >>>>>> - https://bugs.openjdk.org/browse/JDK-8303796 is an umbrella bug that > > >>>>>> links to the issues for resolving static linking issues so far > > >>>>>> - https://github.com/openjdk/jdk21/pull/26 is the enhancement for > > >>>>>> building the complete set of static libraries in JDK/VM, particularly > > >>>>>> including libjvm.a > > >>>>>> > > >>>>>>> Hermetic Java faces several challenges, but the part that is > > >>>>>>> relevant > > >>>>>>> for the build system is the ability to create static libraries. > > >>>>>>> We've > > >>>>>>> had this functionality (in three different ways...) for some time, > > >>>>>>> but > > >>>>>>> it is rather badly implemented. > > >>>>>>> > > >>>>>>> As a result of my investigations, I have a bunch of questions. :-) I > > >>>>>>> have gotten some answers in private discussion, but for the sake of > > >>>>>>> transparency I will repeat them here, to foster an open dialogue. > > >>>>>>> > > >>>>>>> 1. Am I correct in understanding that the ultimate goal of this > > >>>>>>> exercise > > >>>>>>> is to be able to have jmods which include static libraries (*.a) of > > >>>>>>> the > > >>>>>>> native code which the module uses, and that the user can then run a > > >>>>>>> special jlink command to have this linked into a single executable > > >>>>>>> binary (which also bundles the *.class files and any additional > > >>>>>>> resources needed)? > > >>>>>>> > > >>>>>>> 2. If so, is the idea to create special kinds of static jmods, like > > >>>>>>> java.base-static.jmod, that contains *.a files instead of lib*.so > > >>>>>>> files? > > >>>>>>> Or is the idea that the normal jmod should contain both? > > >>>>>>> > > >>>>>>> 3. Linking .o and .a files into an executable is a formidable task. > > >>>>>>> Is > > >>>>>>> the intention to have jlink call a system-provided ld, or to bundle > > >>>>>>> ld > > >>>>>>> with jlink, or to reimplement this functionality in Java? > > >>>>>> I have a similar view as Alan responded in your other email thread. > > >>>>>> Things are still in the early stage for the general solution. > > >>>>>> > > >>>>>> In the https://github.com/openjdk/leyden/tree/hermetic-java-runtime > > >>>>>> branch, when configuring JDK with --with-static-java=yes, the JDK > > >>>>>> binary contains the following extra artifacts: > > >>>>>> > > >>>>>> - static-libs/*.a: The complete set of JDK/VM static libraries > > >>>>>> - jdk/bin/javastatic: A demo Java launcher fully statically linked > > >>>>>> with the selected JDK .a libraries (e.g. it currently statically link > > >>>>>> with the headless) and libjvm.a. It's the standard Java launcher > > >>>>>> without additional work for hermetic Java. > > >>>>>> > > >>>>>> In our prototype for hermetic Java, we build the hermetic executable > > >>>>>> image (a single image) from the following input (see description on > > >>>>>> singlejar packaging tool in > > >>>>>> https://cr.openjdk.org/~jiangli/hermetic_java.pdf): > > >>>>>> > > >>>>>> - A customized launcher (with additional work for hermetic) > > >>>>>> executable > > >>>>>> fully statically linked with JDK/VM static libraries (.a files), > > >>>>>> application natives and dependencies (e.g. in .a static libraries) > > >>>>>> - JDK lib/modules, JDK resource files > > >>>>>> - Application classes and resource files > > >>>>>> > > >>>>>> Including a JDK library .a into the corresponding .jmod would require > > >>>>>> extracting the .a for linking with the executable. In some systems > > >>>>>> that may cause memory overhead due to the extracted copy of the .a > > >>>>>> files. I think we should consider the memory overhead issue. > > >>>>>> > > >>>>>> One possibility (as Alan described in his response) is for jlink to > > >>>>>> invoke the ld on the build system. jlink could pass the needed JDK > > >>>>>> static libraries and libjvm.a (provided as part of the JDK binary) to > > >>>>>> ld based on the modules required for the application. > > >>>>>> > > >>>>> I gave a bit more thoughts on this one. For jlink to trigger ld, it > > >>>>> would need to know the complete linker options and inputs. Those > > >>>>> include options and inputs related to the application part as well. In > > >>>>> some usages, it might be easier to handle native linking separately > > >>>>> and pass the linker output, the executable to jlink directly. Maybe we > > >>>>> could consider supporting different modes for various usages > > >>>>> requirements, from static libraries and native linking point of view: > > >>>>> > > >>>>> Mode #1 > > >>>>> Support .jmod packaged natives static libraries, for both JDK/VM .a > > >>>>> and application natives and dependencies. If the inputs to jlink > > >>>>> include .jmods, jlink can extract the .a libraries and pass the > > >>>>> information to ld to link the executable. > > >>>>> > > >>>>> Mode #2 > > >>>>> Support separate .a as jlink input. Jlink could pass the path > > >>>>> information to the .a libraries and other linker options to ld to > > >>>>> create the executable. > > >>>>> > > >>>>> For both mode #1 and #2, jlink would then use the linker output > > >>>>> executable to create the final hermetic image. > > >>>>> > > >>>>> Mode #3 > > >>>>> Support a fully linked executable as a jlink input. When a linked > > >>>>> executable is given to jlink, it can process it directly with other > > >>>>> JDK data/files to create the final image, without native linking step. > > >>>>> > > >>>>> Any other thoughts and considerations? > > >>>>> > > >>>>> Best, > > >>>>> Jiangli > > >>>>> > > >>>>>>> 4. Is the intention is to allow users to create their own jmods with > > >>>>>>> static libraries, and have these linked in as well? This seems to > > >>>>>>> be the > > >>>>>>> case. > > >>>>>> An alternative with less memory overhead could be using application > > >>>>>> modular JAR and separate .a as the input for jlink. > > >>>>>> > > >>>>>>> If that is so, then there will always be the risk for name > > >>>>>>> collisions, and we can only minimize the risk by making sure any > > >>>>>>> global > > >>>>>>> names are as unique as possible. > > >>>>>> Part of the current effort includes resolving the discovered symbol > > >>>>>> collision issues with static linking. Will respond to your other > > >>>>>> email > > >>>>>> on the symbol issue separately later. > > >>>>>> > > >>>>>>> 5. The original implementation of static builds in the JDK, created > > >>>>>>> for > > >>>>>>> the Mobile project, used a configure flag, --enable-static-builds, > > >>>>>>> to > > >>>>>>> change the entire behavior of the build system to only produce *.a > > >>>>>>> files > > >>>>>>> instead of lib*.so. In contrast, the current system is using a > > >>>>>>> special > > >>>>>>> target instead. > > >>>>>> I think we would need both configure flag and special target for the > > >>>>>> static builds. > > >>>>>> > > >>>>>>> In my eyes, this is a much worse solution. Apart from > > >>>>>>> the conceptual principle (if the build should generate static or > > >>>>>>> dynamic > > >>>>>>> libraries is definitely a property of what a "configuration" means), > > >>>>>>> this makes it much harder to implement efficiently, since we cannot > > >>>>>>> make > > >>>>>>> changes in NativeCompilation.gmk, where they are needed. > > >>>>>> For the potential objcopy work to resolve symbol issues, we can add > > >>>>>> that conditionally in NativeCompilation.gmk if STATIC_LIBS is true. > > >>>>>> We > > >>>>>> have an internal prototype (not included in > > >>>>>> https://github.com/openjdk/leyden/tree/hermetic-java-runtime yet) > > >>>>>> done > > >>>>>> by one of colleagues for localizing symbols in libfreetype using > > >>>>>> objcopy. > > >>>>>> > > >>>>>>> That was not as much a question as a statement. 🙂 But here is the > > >>>>>>> question: Do you think it would be reasonable to restore the old > > >>>>>>> behavior but with the new methods, so that we don't use special > > >>>>>>> targets, > > >>>>>>> but instead tells configure to generate static libraries? I'm > > >>>>>>> thinking > > >>>>>>> we should have a flag like "--with-library-type=" that can have > > >>>>>>> values > > >>>>>>> "dynamic" (which is default), "static" or "both". > > >>>>>> If we want to also build a fully statically linked launcher, maybe > > >>>>>> --with-static-java? Being able to configure either dynamic, static or > > >>>>>> both as you suggested also seems to be a good idea. > > >>>>>> > > >>>>>>> I am not sure if "both" are needed, but if we want to bundle both > > >>>>>>> lib*.so and *.a files > > >>>>>>> into a single jmod file (see question 2 above), then it definitely > > >>>>>>> is. > > >>>>>>> In general, the cost of producing two kinds of libraries are quite > > >>>>>>> small, compared to the cost of compiling the source code to object > > >>>>>>> files. > > >>>>>> Completely agree. It would be good to avoid recompiling the .o file > > >>>>>> for static and dynamic builds. As proposed in > > >>>>>> https://bugs.openjdk.org/browse/JDK-8303796: > > >>>>>> > > >>>>>> It's beneficial to be able to build both .so and .a from the same set > > >>>>>> of .o files. That would involve some changes to handle the dynamic > > >>>>>> JDK > > >>>>>> and static JDK difference at runtime, instead of relying on the > > >>>>>> STATIC_BUILD macro. > > >>>>>> > > >>>>>>> Finally, I have looked at how to manipulate symbol visibility. There > > >>>>>>> seems many ways forward, so I feel confident that we can find a good > > >>>>>>> solution. > > >>>>>>> > > >>>>>>> One way forward is to use objcopy to manipulate symbol status > > >>>>>>> (global/local). There is an option --localize-symbol in objcopy, > > >>>>>>> that > > >>>>>>> has been available in objcopy since at least 2.15, which was > > >>>>>>> released > > >>>>>>> 2004, so it should be safe to use. But ideally we should avoid using > > >>>>>>> objcopy and do this as part of the linking process. This should be > > >>>>>>> possible to do, given that we make changes in NativeCompilation.gmk > > >>>>>>> -- > > >>>>>>> see question 5 above. > > >>>>>>> > > >>>>>>> As a fallback, it is also possible to rename symbols, either > > >>>>>>> piecewise > > >>>>>>> or wholesale, using objcopy. There are many ways to do this, using > > >>>>>>> --prefix-symbols, --redefine-sym or --redefine-syms (note the -s, > > >>>>>>> this > > >>>>>>> takes a file with a list of symbols). Thus we can always introduce a > > >>>>>>> "post factum namespace" by renaming symbols. > > >>>>>> Renaming or redefining the symbol at build time could cause > > >>>>>> confusions > > >>>>>> with debugging. That's a concern raised in > > >>>>>> https://github.com/openjdk/jdk/pull/17456 discussions. > > >>>>>> > > >>>>>> Additionally, redefining symbols using tools like objcopy may not > > >>>>>> handle member names referenced in string literals. For example, in > > >>>>>> https://github.com/openjdk/jdk/pull/17456 additional changes are > > >>>>>> needed in assembling and SA to reflect the symbol change. > > >>>>>> > > >>>>>>> So in the end, I think it will be fully possible to produce .a files > > >>>>>>> that only has global symbols for the functions that are part of the > > >>>>>>> API > > >>>>>>> exposed by that library, and have all other symbols local, and make > > >>>>>>> this > > >>>>>>> is in a way that is consistent with the rest of the build system. > > >>>>>>> > > >>>>>>> Finally, a note on Hotspot. Due to debugging reasons, we export > > >>>>>>> basically all symbols in hotspot as global. This is not reasonable > > >>>>>>> to do > > >>>>>>> for a static build. The effect of not exporting those symbols will > > >>>>>>> be > > >>>>>>> that SA will not function to 100%. On the other hand, I have no > > >>>>>>> idea if > > >>>>>>> SA works at all with a static build. Have you tested this? Is this > > >>>>>>> part > > >>>>>>> of the plan to support, or will it be officially dropped for > > >>>>>>> Hermetic Java? > > >>>>>> We have done some testing with jtreg SA related tests for the fully > > >>>>>> statically linked `javastatic`. > > >>>>>> > > >>>>>> If we use objcopy to localize symbols in hotspot, it's not yet clear > > >>>>>> what's the impact on SA. We could do some tests. The other question > > >>>>>> that I raised is the supported gcc versions (for partial linking) > > >>>>>> related to the solution. > > >>>>>> > > >>>>>> Best, > > >>>>>> Jiangli > > >>>>>> > > >>>>>>> /Magnus > > >>>>>>>