We had a discussion on static Java this morning. Outcome of today's
discussion:
For the first step in earlier outlined steps, there's a preference to have
a minimum buildable and runnable (be able to run HelloWorld) `javastatic`
as the initial integration point for the mainline. Following are the
updated plan/steps:
Step 1 - Get the makefile changes for linking `javastatic` with minimum
needed runtime changes into mainline;
`javastatic` is buildable and runnable - can run HelloWorld
Can enable build and testing for `javastatic` in github
workflow
Step 2 - Incrementally get the runtime changes reviewed and integrated into
mainline;
Revert any of the #ifdef changes if they were introduced in
the first step
Enable testing tier 1 for `javastatic` in workflow when we
can run jtreg tests with the static launcher - could be done in a later
step;
Step 3 - Remove all STATIC_BUILD macros in JDK runtime code; Also cleanup
the macros in tests (can be done later)
CSR and JNI specification work to support
JNI_OnLoad_<lib_name> and friends for JNI dynamic library and builtin
library
Step 4 - Build (makefile) changes to support linking .a and .so libraries
using the same set of .o objects, to avoid compiling the .c/.c++ source
twice
According to Magnus, his #ifdef changes only affect about half a dozen
files. Those #ifdef are inserted in places where the
JLI_IsStaticJDK|JVM_IsStaticJDK checks are used in the
hermetic-java-runtime branch. Magnus will send out his changes as PR draft
for initial review for deciding on how to move forward with the
non-makefile changes. If the #ifdef changes are not too disruptive, we
could include those in the initial integration work. Then the followup
runtime changes would revert the #ifdef changes.
Best,
Jiangli
On Mon, May 20, 2024 at 10:17 PM Jiangli Zhou <jiangliz...@google.com>
wrote:
> On Tue, May 7, 2024 at 5:26 AM Magnus Ihse Bursie <
> magnus.ihse.bur...@oracle.com> wrote:
>
>> On 2024-05-07 06:04, Jiangli Zhou wrote:
>>
>> On Tue, Apr 30, 2024 at 5:42 AM Magnus Ihse
>> Bursie<magnus.ihse.bur...@oracle.com> <magnus.ihse.bur...@oracle.com> wrote:
>>
>> I am not sure why clang insisted on picking up ld and not lld. I remeber
>> trying with -fuse-ld=lld, and that it did not work either.
>> Unfortunately, I don't remember exactly what the problems were.
>>
>> I started reinstalling my Linux workstation yesterday, but something
>> went wrong, and it failed so hard that it got semi-bricked by the new
>> installation, so I need to redo everything from scratch. :-( After that
>> is done, I'll re-test. Hopefully this was just my old installation that
>> was too broken.
>>
>> I decided to spend the time to reinstall my machine. Now linking with
>> clang works. Kind of. For some reason, it still picks up binutils ld and
>> not lld, and then -l:libc++.a does not work, but when I replaced it with
>> -l:libstdc++.a it worked just fine. I guess we need to either forcefully
>> add -fuse-ld=lld to our clang compilation lines, or figure out if clang is
>> going to call the binutils or llvm ld, and select the right option.
>>
>
> https://lld.llvm.org/#using-lld has some information on using lld instead
> of the default linker.
>
>
>> I still find the logic for how clang and gcc locates the default linker
>> to be mostly magic. I guess I need to make a deep dive in understanding
>> this to be able to resolve this properly.
>>
>> The JDK and VM code has pre-existing assumptions about the JDK
>> directories and dynamic linking (e.g. the .so).
>> JLI_IsStaticJDK|JLI_SetStaticJDK|JVM_IsStaticJDK|JVM_SetStaticJDK is
>> needed for static JDK support to handle those cases correctly.
>> CreateExecutionEnvironment that I mentioned earlier is one of the
>> examples.
>>
>> I'm quite certain the issue that you are running into is due to the
>> incorrect static check/handling in CreateExecutionEnvironment.
>>
>> I'll have a look at that, thanks for the pointer.
>>
>> In my branch, I am only using compile-time #ifdef checks for static vs
>> dynamic. In the long run, the runtime checks that you have done are a
>> good thing, but at the moment they are just adding intrusive changes
>> without providing any benefit -- if we can't reuse .o files between
>> dynamic and static compilation, there is no point in introducing a
>> runtime check when we already have a working compile-time check.
>>
>> I haven't seen your branch/code. I'd suggest not going with the #ifdef
>> checks as that's the opposite direction of what we want to achieve. It
>> doesn't seem to be worth your effort to add more #ifdef checks in
>> order to do static linking build work, even those are for temporary
>> testing reasons.
>>
>> Okaaaaay... My understanding was that you wanted to push for the quickest
>> possible integration of building a static java launcher into mainline.
>>
> That's correct. Please see more details below.
>
>> To do that as fast as possible, we need to use the existing framework for
>> separating statically and dynamically linked libraries, which means doing
>> compile time checks using #ifdefs.
>>
> Using #ifdefs is not the most efficient path for us to get static Java
> launcher support in mainline. That's because most of the runtime changes
> for static Java support in hermetic-java-runtime branch are already done
> using `JLI_IsStaticJDK|JVM_IsStaticJDK` checks. We should not convert those
> to use #ifdefs then later convert the #ifdef back to runtime checks again
> during the integration work.
>
> As suggested and discussed earlier we can aim to get the static Java
> related changes into mainline incrementally. Following is a path that I
> think would work effectively and "fast" by limiting potentially wasted
> efforts:
>
> Step 1 - Get the makefile changes for linking `javastatic` without any of
> the runtime changes; Don't enable any build and testing for `javastatic` in
> this step yet
> Step 2 - Incrementally get the runtime changes reviewed and integrated
> into mainline;
> Enable building for `javastatic` as a test in github
> workflow when we can run HelloWorld using static launcher in mainline;
> Enable testing tier 1 for `javastatic` in workflow when we
> can run jtreg tests with the static launcher - could be done in a later
> step;
> Step 3 - Remove all STATIC_BUILD macros in JDK runtime code; Also cleanup
> the macros in tests (can be done later)
> CSR and JNI specification work to support
> JNI_OnLoad_<lib_name> and friends for JNI dynamic library and builtin
> library
> Step 4 - Build (makefile) changes to support linking .a and .so libraries
> using the same set of .o objects, to avoid compiling the .c/.c++ source
> twice
>
> Those lay the foundation for the hermetic Java work in mainline.
>
>> Are you saying now that the priorities has changed, and that you want to
>> start by introducing your framework for the runtime lookup if we are static
>> or dynamic?
>>
> By "runtime lookup", I think you were referring to the JNI native library
> lookup. We can handle them as part of the step 2 above.
>
> I think for any of the runtime changes, we need to be able to build in the
> mainline (although initially not included in the github workflow).
>
>>
>> To be honest, I think your prototype is rather hacky in how you implement
>> this, and I reckon that it will require quite a lot of work to be accepted
>> into mainline. I also think you need a CSR for changing the Hotspot/JDK
>> behavior wrt this, which further adds to the process.
>>
>
> For CSR work, we can do that as part of step #3. Actually, for the
> builtin/dynamic library lookup support, I think the enhancements in
> hermetic-java-runtime are already close to the proper shape (not hacky).
>
>
>> If you want to go that route instead, then I'll put my work on hold until
>> you have gotten a working solution for the runtime lookup in mainline. I
>> gather this means that there is no real stress for me anymore.
>>
> Ron and Alan mentioned Tuesday morning PT may not work the best for you.
> Would you be open for a separate time to discuss the details on moving
> forward?
>
> Best,
> Jiangli
>
>
>>
>> /Magnus
>>
>
