Why not build 2 binary packages: Java 8 and Java 11? Use the java 8 JDK to build the java 8 version and Java 11 JDK to build java 11 version.
Then no need to copy rt.jar On Thu, May 25, 2023 at 2:35 PM Arnout Engelen <[email protected]> wrote: > On Thu, May 25, 2023 at 2:50 PM Claude Warren, Jr > <[email protected]> wrote: > > Assume Java 11 is installed and no Java 8 is installed on the system > where > > Pekko is being deployed. > > Assume Java 11 compilation with rt.jar present for the build of the Pekko > > package being deployed. (My understanding that even with Java 11 the > rt.jar > > is required for compilation) > > > > When the code is executed on the deployed system and the class from the > > rt.jar is required, what happens? > > I would expect that a NoClassDefFoundError to be thrown. If the error > is > > not thrown I can only see two possibilities: > > > > 1. the class(es) from the rt.jar are defined somewhere else in which > > case why can that not be used during the build? > > 2. the class(es) are not used and the dependency on the rt.jar can be > > removed. > > > > So what happens and why? > > No NoClassDefFoundError will be thrown, because the runtime for Java > 11 is backwards compatible with the runtime for Java 8, so there > should be no classes/methods that exist in the jdk8 rt.jar that are > not also available in the jdk11 runtime. > > From your two possibilities, '1' is true: they are defined both in the > Java 8 and in the Java 11 runtimes. Indeed if we removed the rt.jar > everything would still work on Java 11. Just no longer on Java 8. This > is because the runtimes are not *forwards* compatible: there are > things in the Java 11 runtime that are not in the Java 8 one. > > To illustrate and make this more tangible, one specific example is > ByteBuffer.clear(): on Java 8, this method returned a Buffer. On Java > 11, there is an implementation that returns a ByteBuffer. If you > compile a class that calls this method against the Java 8 runtime and > then run it against the Java 11 one, this is fine, because Java 11 has > *both* a clear() method that returns a ByteBuffer and (for backwards > compatibility) one that returns a Buffer. If you compile the same code > against the Java 11 runtime and then run it against Java 8, it will > pick the one that returns a ByteBuffer at compile time, and you will > get a NoSuchMethod exception at runtime because this new method does > not exist in the Java 8 runtime. > > Of course we could change the code to make sure this *particular* > problem does not occur in Pekko, but that is risky as we don't know > how many similar situations there are. > > Is there a better solution to this than compiling with two JVM's > installed? Yes! Invoking the JDK11 compiler with '--release 8' will > also generate the correct reference. However, '--release 8' does not > allow access to sun.misc.Unsafe API's, which unfortunately Pekko still > relies on - so we can't do that just yet. > > > Kind regards, > > Arnout > > > On Wed, May 24, 2023 at 9:23 PM Matthew Benedict de Detrich > > <[email protected]> wrote: > > > > > > If you build Pekko with both Java 11 and Java 8 installed, the > classes > > > we will produce in our jars will be in Java 8 class format. > > > > > > More specifically, normally if you compile Java/Scala source with JDK > 11 > > > (which is what Pekko does, JDK 8 is only necessary for its rt.jar) it > will > > > produce > > > classfiles compatible with JDK 11 and above. You can however provide a > flag > > > to > > > javac/scalac so that it generates class files with a specific bytecode > > > format, i.e. > > > > > > > https://github.com/apache/incubator-pekko/blob/eaea9b116f4c47aed1686d62adb674a37600f6b2/project/JdkOptions.scala#L60 > > > and > > > > > > > https://github.com/apache/incubator-pekko/blob/eaea9b116f4c47aed1686d62adb674a37600f6b2/project/JdkOptions.scala#L73 > > > does. > > > > > > In short, Pekko is built using JDK 11 with Xtarget/release flags to > force > > > JDK 8 > > > compatible bytecode generation in addition to providing the JDK8 > rt.jar via > > > the > > > -bootclasspath javac flag. > > > > > > On Wed, May 24, 2023 at 9:44 PM PJ Fanning <[email protected]> > wrote: > > > > > > > > Q: does the build package the rt.jar in the final binary package? > > > > > > > > No. We only produce our own jars that we push to Maven - and these do > > > > not embed any rt.jar classes (shaded or otherwise). We have no other > > > > type of 'binary' package. > > > > > > > > > Q: if it does not, what happens if you try to run using Java 11 on > a > > > > system > > > > > that does not have Java 8 nor the rt.jar installed? > > > > > > > > Java compiler produces class files. These have a Java version > > > > associated with them and you can't use a class that is in Java 11 > > > > format in a Java 8 runtime. > > > > If you build Pekko with both Java 11 and Java 8 installed, the > classes > > > > we will produce in our jars will be in Java 8 class format. > > > > > > > > On Wed, 24 May 2023 at 16:45, Matthew Benedict de Detrich > > > > <[email protected]> wrote: > > > > > > > > > > afaik Pekko will just fail to build at all if you use JDK 11 > without > > > > > pointing to JDK 8 rt.jar. > > > > > > > > > > This is because Pekko uses sun.misc.unsafe which is only shipping > with > > > > JDK > > > > > 8 and the > > > > > reason why we have to build on JDK 11 is because there are features > > > > > shipping in JDK 11 > > > > > which pekko uses. > > > > > > > > > > On Wed, May 24, 2023 at 5:40 PM Claude Warren, Jr > > > > > <[email protected]> wrote: > > > > > > > > > > > I have a couple of questions: > > > > > > > > > > > > Q: does the build package the rt.jar in the final binary package? > > > > > > > > > > > > Q: if it does not, what happens if you try to run using Java 11 > on a > > > > system > > > > > > that does not have Java 8 nor the rt.jar installed? > > > > > > > > > > > > > > > > > > > > > > > > On Wed, May 24, 2023 at 4:22 PM Matthew Benedict de Detrich > > > > > > <[email protected]> wrote: > > > > > > > > > > > > > > We could provide a Docker image with the right sbt and Java > setup > > > > for > > > > > > > convenience but I think it is fair not to require users to use > > > > Docker. > > > > > > > > > > > > > > Considering how much implicit behaviour there is in making a > full > > > > build > > > > > > > (as Arnout mentioned earlier, you can compile Pekko fine with > just > > > > JDK 8 > > > > > > > just fine however it will be missing post java 8 optional > > > features) I > > > > > > would > > > > > > > prefer > > > > > > > to strongly encourage people to use Docker to do a proper > build. > > > As a > > > > > > bare > > > > > > > minimum we can also print a warning in sbt telling users if > they > > > are > > > > not > > > > > > > going > > > > > > > to be making a complete Pekko build when building on JDK 8 (as > > > > stated it > > > > > > > already errors out if you build on JDK 11 without pointing to > a JDK > > > > > > > rt.jar). > > > > > > > > > > > > > > On Wed, May 24, 2023 at 11:22 AM PJ Fanning < > [email protected]> > > > > wrote: > > > > > > > > > > > > > > > Thanks Arnout. > > > > > > > > > > > > > > > > I think if we clearly document what you need to set up to do > a > > > > build, > > > > > > > > that should be enough. I think it is fair to assume that > users > > > can > > > > > > > > install Java and sbt - and that they should be able to > install 2 > > > > > > > > versions of Java if we are explicit about how to do this and > why > > > > this > > > > > > > > is the best setup for building Pekko. > > > > > > > > > > > > > > > > We will need to expand what we have in > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://github.com/apache/incubator-pekko/blob/main/README.md#building-from-source > > > > > > > > > > > > > > > > This README file will appear in the source releases. > > > > > > > > > > > > > > > > We could provide a Docker image with the right sbt and Java > setup > > > > for > > > > > > > > convenience but I think it is fair not to require users to > use > > > > Docker. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Wed, 24 May 2023 at 10:09, Arnout Engelen < > [email protected] > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > On Wed, May 24, 2023 at 10:57 AM PJ Fanning < > > > [email protected] > > > > > > > > > > > > wrote: > > > > > > > > > > Ok. It seems the nightly job to publish the snapshot jars > > > uses > > > > Java > > > > > > > > > > 11. It does not explicitly install Java 8 but maybe, it's > > > > possible > > > > > > > > > > that Java 8 is installed silently, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://github.com/actions/runner-images/blob/main/images/linux/Ubuntu2004-Readme.md > > > > > > > > > suggests so, yes. > > > > > > > > > > > > > > > > > > > The nightly build does produce jars that can be used by > Java > > > 8. > > > > > > > > > > > > > > > > > > > > So if this nightly build relies on the silent > installation of > > > > Java > > > > > > 8, > > > > > > > > > > should we add a check in our sbt script that will fail if > > > Java > > > > 8 is > > > > > > > > > > not found? This could be useful for users who are > building on > > > > their > > > > > > > > > > own machines. > > > > > > > > > > > > > > > > > > This check already exists: when I build with jdk11 without > > > having > > > > > > jdk8 > > > > > > > > > available the build fails with: > > > > > > > > > > > > > > > > > > [error] A JDK 8 installation was not found, but is > required to > > > > build > > > > > > > > > Apache Pekko. To manually specify a JDK 8 installation, > use the > > > > "set > > > > > > > > > every jdk8home := \"/path/to/jdk\" sbt command. If you > have no > > > > JDK 8 > > > > > > > > > installation, target your system JDK with the "set every > > > > > > > > > targetSystemJdk := true" sbt command, but beware resulting > > > > artifacts > > > > > > > > > will not work on JDK 8 > > > > > > > > > > > > > > > > > > It would be good to also document the JAVA_8_HOME > environment > > > > > > variable > > > > > > > > > in this error message. Created > > > > > > > > > https://github.com/apache/incubator-pekko/pull/333 for > that. > > > > > > > > > > > > > > > > > > > > > > > > > > > Kind regards, > > > > > > > > > > > > > > > > > > Arnout > > > > > > > > > > > > > > > > > > > On Wed, 24 May 2023 at 09:38, Arnout Engelen < > > > > [email protected]> > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > On Wed, May 24, 2023 at 10:12 AM PJ Fanning < > > > > [email protected] > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > Only building the website docs needs Java 11 (and > up). > > > > > > Everything > > > > > > > > else > > > > > > > > > > > > builds with Java 8 (and up). > > > > > > > > > > > > > > > > > > > > > > Yes, though builds created with only Java 8 will miss > some > > > > > > optional > > > > > > > > > > > post-Java-8 features/implementations for the > "actor-typed", > > > > > > > "stream", > > > > > > > > > > > "remote" and "cluster-sharding" modules - so for a > faithful > > > > test > > > > > > > > you'd > > > > > > > > > > > ideally build with JDK 9-or-later while having the > Java 8 > > > > runtime > > > > > > > > > > > available. > > > > > > > > > > > > > > > > > > > > > > (TBH I don't think this is _too_ different from other > > > > projects > > > > > > > where > > > > > > > > > > > you have to have some compile-time dependencies > installed > > > to > > > > be > > > > > > > able > > > > > > > > > > > to build the project) > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Kind regards, > > > > > > > > > > > > > > > > > > > > > > Arnout > > > > > > > > > > > > > > > > > > > > > > > On Wed, 24 May 2023 at 08:05, Claude Warren, Jr > > > > > > > > > > > > <[email protected]> wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > Justin, > > > > > > > > > > > > > > > > > > > > > > > > > > There is an issue in the code base that is > reasonably > > > > common > > > > > > > > among systems > > > > > > > > > > > > > that operate at low levels and are migrating from > Java > > > 8 > > > > > > > > implementation > > > > > > > > > > > > > wherein they need the rt.jar from java 8 for > > > compilation. > > > > > > > > There are > > > > > > > > > > > > > several ways to solve this problem, Pekko (and I > > > suppose > > > > > > Akka) > > > > > > > > require that > > > > > > > > > > > > > Java 8 be installed so that the build process can > find > > > > the > > > > > > Java > > > > > > > > 8 rt.jar. > > > > > > > > > > > > > I suspect that all that is actually required to > get the > > > > > > current > > > > > > > > code to > > > > > > > > > > > > > compile is setting JAVA_8_HOME to point to the > > > directory > > > > with > > > > > > > > the rt.jar in > > > > > > > > > > > > > it. However, the Java 8 rt.jar is still > required. I > > > > believe > > > > > > > > there is a > > > > > > > > > > > > > ticket to resolve this issue but I may be mistaken > > > about > > > > > > this. > > > > > > > > > > > > > > > > > > > > > > > > > > Claude > > > > > > > > > > > > > > > > > > > > > > > > > > On Wed, May 24, 2023 at 7:38 AM Justin Mclean < > > > > > > > > [email protected]> > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > Hi, > > > > > > > > > > > > > > > > > > > > > > > > > > > > Part of the requirement of a release is that > someone > > > > can > > > > > > > build > > > > > > > > it on their > > > > > > > > > > > > > > system, and that ideally should be easy to do. I > > > would > > > > > > assume > > > > > > > > by their > > > > > > > > > > > > > > system than means a single version of the JDK > which > > > > they > > > > > > use > > > > > > > > and is > > > > > > > > > > > > > > supported. I’m not sure there is be a need for > > > someone > > > > > > > testing > > > > > > > > the release > > > > > > > > > > > > > > to compile against multiple JDKs? > > > > > > > > > > > > > > > > > > > > > > > > > > > > Kind Regards, > > > > > > > > > > > > > > Justin > > > > > > > > > > > > > > > > > > > > > > > > > > --------------------------------------------------------------------- > > > > > > > > > > > > > > To unsubscribe, e-mail: > > > > [email protected] > > > > > > > > > > > > > > For additional commands, e-mail: > > > > [email protected] > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > --------------------------------------------------------------------- > > > > > > > > > > > > To unsubscribe, e-mail: > [email protected] > > > > > > > > > > > > For additional commands, e-mail: > > > [email protected] > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > > Arnout Engelen > > > > > > > > > > > ASF Security Response > > > > > > > > > > > Committer on Apache Pekko > > > > > > > > > > > Committer on NixOS > > > > > > > > > > > Independent Open Source consultant > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > --------------------------------------------------------------------- > > > > > > > > > > > To unsubscribe, e-mail: > [email protected] > > > > > > > > > > > For additional commands, e-mail: > [email protected] > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > --------------------------------------------------------------------- > > > > > > > > > > To unsubscribe, e-mail: [email protected] > > > > > > > > > > For additional commands, e-mail: > [email protected] > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > Arnout Engelen > > > > > > > > > ASF Security Response > > > > > > > > > Committer on Apache Pekko > > > > > > > > > Committer on NixOS > > > > > > > > > Independent Open Source consultant > > > > > > > > > > > > > > > > > > > > > > --------------------------------------------------------------------- > > > > > > > > > To unsubscribe, e-mail: [email protected] > > > > > > > > > For additional commands, e-mail: [email protected] > > > > > > > > > > > > > > > > > > > > > > > > > > > > > --------------------------------------------------------------------- > > > > > > > > To unsubscribe, e-mail: [email protected] > > > > > > > > For additional commands, e-mail: [email protected] > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > > > > > Matthew de Detrich > > > > > > > > > > > > > > *Aiven Deutschland GmbH* > > > > > > > > > > > > > > Immanuelkirchstraße 26, 10405 Berlin > > > > > > > > > > > > > > Amtsgericht Charlottenburg, HRB 209739 B > > > > > > > > > > > > > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen > > > > > > > > > > > > > > *m:* +491603708037 > > > > > > > > > > > > > > *w:* aiven.io *e:* [email protected] > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > Matthew de Detrich > > > > > > > > > > *Aiven Deutschland GmbH* > > > > > > > > > > Immanuelkirchstraße 26, 10405 Berlin > > > > > > > > > > Amtsgericht Charlottenburg, HRB 209739 B > > > > > > > > > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen > > > > > > > > > > *m:* +491603708037 > > > > > > > > > > *w:* aiven.io *e:* [email protected] > > > > > > > > --------------------------------------------------------------------- > > > > To unsubscribe, e-mail: [email protected] > > > > For additional commands, e-mail: [email protected] > > > > > > > > > > > > > > -- > > > > > > Matthew de Detrich > > > > > > *Aiven Deutschland GmbH* > > > > > > Immanuelkirchstraße 26, 10405 Berlin > > > > > > Amtsgericht Charlottenburg, HRB 209739 B > > > > > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen > > > > > > *m:* +491603708037 > > > > > > *w:* aiven.io *e:* [email protected] > > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > >
