On Thu, May 25, 2023 at 2:50 PM Claude Warren, Jr <[email protected]> wrote: > Assume Java 11 is installed and no Java 8 is installed on the system where > Pekko is being deployed. > Assume Java 11 compilation with rt.jar present for the build of the Pekko > package being deployed. (My understanding that even with Java 11 the rt.jar > is required for compilation) > > When the code is executed on the deployed system and the class from the > rt.jar is required, what happens? > I would expect that a NoClassDefFoundError to be thrown. If the error is > not thrown I can only see two possibilities: > > 1. the class(es) from the rt.jar are defined somewhere else in which > case why can that not be used during the build? > 2. the class(es) are not used and the dependency on the rt.jar can be > removed. > > So what happens and why?
No NoClassDefFoundError will be thrown, because the runtime for Java 11 is backwards compatible with the runtime for Java 8, so there should be no classes/methods that exist in the jdk8 rt.jar that are not also available in the jdk11 runtime. >From your two possibilities, '1' is true: they are defined both in the Java 8 and in the Java 11 runtimes. Indeed if we removed the rt.jar everything would still work on Java 11. Just no longer on Java 8. This is because the runtimes are not *forwards* compatible: there are things in the Java 11 runtime that are not in the Java 8 one. To illustrate and make this more tangible, one specific example is ByteBuffer.clear(): on Java 8, this method returned a Buffer. On Java 11, there is an implementation that returns a ByteBuffer. If you compile a class that calls this method against the Java 8 runtime and then run it against the Java 11 one, this is fine, because Java 11 has *both* a clear() method that returns a ByteBuffer and (for backwards compatibility) one that returns a Buffer. If you compile the same code against the Java 11 runtime and then run it against Java 8, it will pick the one that returns a ByteBuffer at compile time, and you will get a NoSuchMethod exception at runtime because this new method does not exist in the Java 8 runtime. Of course we could change the code to make sure this *particular* problem does not occur in Pekko, but that is risky as we don't know how many similar situations there are. Is there a better solution to this than compiling with two JVM's installed? Yes! Invoking the JDK11 compiler with '--release 8' will also generate the correct reference. However, '--release 8' does not allow access to sun.misc.Unsafe API's, which unfortunately Pekko still relies on - so we can't do that just yet. Kind regards, Arnout > On Wed, May 24, 2023 at 9:23 PM Matthew Benedict de Detrich > <[email protected]> wrote: > > > > If you build Pekko with both Java 11 and Java 8 installed, the classes > > we will produce in our jars will be in Java 8 class format. > > > > More specifically, normally if you compile Java/Scala source with JDK 11 > > (which is what Pekko does, JDK 8 is only necessary for its rt.jar) it will > > produce > > classfiles compatible with JDK 11 and above. You can however provide a flag > > to > > javac/scalac so that it generates class files with a specific bytecode > > format, i.e. > > > > https://github.com/apache/incubator-pekko/blob/eaea9b116f4c47aed1686d62adb674a37600f6b2/project/JdkOptions.scala#L60 > > and > > > > https://github.com/apache/incubator-pekko/blob/eaea9b116f4c47aed1686d62adb674a37600f6b2/project/JdkOptions.scala#L73 > > does. > > > > In short, Pekko is built using JDK 11 with Xtarget/release flags to force > > JDK 8 > > compatible bytecode generation in addition to providing the JDK8 rt.jar via > > the > > -bootclasspath javac flag. > > > > On Wed, May 24, 2023 at 9:44 PM PJ Fanning <[email protected]> wrote: > > > > > > Q: does the build package the rt.jar in the final binary package? > > > > > > No. We only produce our own jars that we push to Maven - and these do > > > not embed any rt.jar classes (shaded or otherwise). We have no other > > > type of 'binary' package. > > > > > > > Q: if it does not, what happens if you try to run using Java 11 on a > > > system > > > > that does not have Java 8 nor the rt.jar installed? > > > > > > Java compiler produces class files. These have a Java version > > > associated with them and you can't use a class that is in Java 11 > > > format in a Java 8 runtime. > > > If you build Pekko with both Java 11 and Java 8 installed, the classes > > > we will produce in our jars will be in Java 8 class format. > > > > > > On Wed, 24 May 2023 at 16:45, Matthew Benedict de Detrich > > > <[email protected]> wrote: > > > > > > > > afaik Pekko will just fail to build at all if you use JDK 11 without > > > > pointing to JDK 8 rt.jar. > > > > > > > > This is because Pekko uses sun.misc.unsafe which is only shipping with > > > JDK > > > > 8 and the > > > > reason why we have to build on JDK 11 is because there are features > > > > shipping in JDK 11 > > > > which pekko uses. > > > > > > > > On Wed, May 24, 2023 at 5:40 PM Claude Warren, Jr > > > > <[email protected]> wrote: > > > > > > > > > I have a couple of questions: > > > > > > > > > > Q: does the build package the rt.jar in the final binary package? > > > > > > > > > > Q: if it does not, what happens if you try to run using Java 11 on a > > > system > > > > > that does not have Java 8 nor the rt.jar installed? > > > > > > > > > > > > > > > > > > > > On Wed, May 24, 2023 at 4:22 PM Matthew Benedict de Detrich > > > > > <[email protected]> wrote: > > > > > > > > > > > > We could provide a Docker image with the right sbt and Java setup > > > for > > > > > > convenience but I think it is fair not to require users to use > > > Docker. > > > > > > > > > > > > Considering how much implicit behaviour there is in making a full > > > build > > > > > > (as Arnout mentioned earlier, you can compile Pekko fine with just > > > JDK 8 > > > > > > just fine however it will be missing post java 8 optional > > features) I > > > > > would > > > > > > prefer > > > > > > to strongly encourage people to use Docker to do a proper build. > > As a > > > > > bare > > > > > > minimum we can also print a warning in sbt telling users if they > > are > > > not > > > > > > going > > > > > > to be making a complete Pekko build when building on JDK 8 (as > > > stated it > > > > > > already errors out if you build on JDK 11 without pointing to a JDK > > > > > > rt.jar). > > > > > > > > > > > > On Wed, May 24, 2023 at 11:22 AM PJ Fanning <[email protected]> > > > wrote: > > > > > > > > > > > > > Thanks Arnout. > > > > > > > > > > > > > > I think if we clearly document what you need to set up to do a > > > build, > > > > > > > that should be enough. I think it is fair to assume that users > > can > > > > > > > install Java and sbt - and that they should be able to install 2 > > > > > > > versions of Java if we are explicit about how to do this and why > > > this > > > > > > > is the best setup for building Pekko. > > > > > > > > > > > > > > We will need to expand what we have in > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://github.com/apache/incubator-pekko/blob/main/README.md#building-from-source > > > > > > > > > > > > > > This README file will appear in the source releases. > > > > > > > > > > > > > > We could provide a Docker image with the right sbt and Java setup > > > for > > > > > > > convenience but I think it is fair not to require users to use > > > Docker. > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Wed, 24 May 2023 at 10:09, Arnout Engelen <[email protected] > > > > > > > > wrote: > > > > > > > > > > > > > > > > On Wed, May 24, 2023 at 10:57 AM PJ Fanning < > > [email protected] > > > > > > > > > > wrote: > > > > > > > > > Ok. It seems the nightly job to publish the snapshot jars > > uses > > > Java > > > > > > > > > 11. It does not explicitly install Java 8 but maybe, it's > > > possible > > > > > > > > > that Java 8 is installed silently, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://github.com/actions/runner-images/blob/main/images/linux/Ubuntu2004-Readme.md > > > > > > > > suggests so, yes. > > > > > > > > > > > > > > > > > The nightly build does produce jars that can be used by Java > > 8. > > > > > > > > > > > > > > > > > > So if this nightly build relies on the silent installation of > > > Java > > > > > 8, > > > > > > > > > should we add a check in our sbt script that will fail if > > Java > > > 8 is > > > > > > > > > not found? This could be useful for users who are building on > > > their > > > > > > > > > own machines. > > > > > > > > > > > > > > > > This check already exists: when I build with jdk11 without > > having > > > > > jdk8 > > > > > > > > available the build fails with: > > > > > > > > > > > > > > > > [error] A JDK 8 installation was not found, but is required to > > > build > > > > > > > > Apache Pekko. To manually specify a JDK 8 installation, use the > > > "set > > > > > > > > every jdk8home := \"/path/to/jdk\" sbt command. If you have no > > > JDK 8 > > > > > > > > installation, target your system JDK with the "set every > > > > > > > > targetSystemJdk := true" sbt command, but beware resulting > > > artifacts > > > > > > > > will not work on JDK 8 > > > > > > > > > > > > > > > > It would be good to also document the JAVA_8_HOME environment > > > > > variable > > > > > > > > in this error message. Created > > > > > > > > https://github.com/apache/incubator-pekko/pull/333 for that. > > > > > > > > > > > > > > > > > > > > > > > > Kind regards, > > > > > > > > > > > > > > > > Arnout > > > > > > > > > > > > > > > > > On Wed, 24 May 2023 at 09:38, Arnout Engelen < > > > [email protected]> > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > On Wed, May 24, 2023 at 10:12 AM PJ Fanning < > > > [email protected] > > > > > > > > > > > > > wrote: > > > > > > > > > > > Only building the website docs needs Java 11 (and up). > > > > > Everything > > > > > > > else > > > > > > > > > > > builds with Java 8 (and up). > > > > > > > > > > > > > > > > > > > > Yes, though builds created with only Java 8 will miss some > > > > > optional > > > > > > > > > > post-Java-8 features/implementations for the "actor-typed", > > > > > > "stream", > > > > > > > > > > "remote" and "cluster-sharding" modules - so for a faithful > > > test > > > > > > > you'd > > > > > > > > > > ideally build with JDK 9-or-later while having the Java 8 > > > runtime > > > > > > > > > > available. > > > > > > > > > > > > > > > > > > > > (TBH I don't think this is _too_ different from other > > > projects > > > > > > where > > > > > > > > > > you have to have some compile-time dependencies installed > > to > > > be > > > > > > able > > > > > > > > > > to build the project) > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Kind regards, > > > > > > > > > > > > > > > > > > > > Arnout > > > > > > > > > > > > > > > > > > > > > On Wed, 24 May 2023 at 08:05, Claude Warren, Jr > > > > > > > > > > > <[email protected]> wrote: > > > > > > > > > > > > > > > > > > > > > > > > Justin, > > > > > > > > > > > > > > > > > > > > > > > > There is an issue in the code base that is reasonably > > > common > > > > > > > among systems > > > > > > > > > > > > that operate at low levels and are migrating from Java > > 8 > > > > > > > implementation > > > > > > > > > > > > wherein they need the rt.jar from java 8 for > > compilation. > > > > > > > There are > > > > > > > > > > > > several ways to solve this problem, Pekko (and I > > suppose > > > > > Akka) > > > > > > > require that > > > > > > > > > > > > Java 8 be installed so that the build process can find > > > the > > > > > Java > > > > > > > 8 rt.jar. > > > > > > > > > > > > I suspect that all that is actually required to get the > > > > > current > > > > > > > code to > > > > > > > > > > > > compile is setting JAVA_8_HOME to point to the > > directory > > > with > > > > > > > the rt.jar in > > > > > > > > > > > > it. However, the Java 8 rt.jar is still required. I > > > believe > > > > > > > there is a > > > > > > > > > > > > ticket to resolve this issue but I may be mistaken > > about > > > > > this. > > > > > > > > > > > > > > > > > > > > > > > > Claude > > > > > > > > > > > > > > > > > > > > > > > > On Wed, May 24, 2023 at 7:38 AM Justin Mclean < > > > > > > > [email protected]> > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > Hi, > > > > > > > > > > > > > > > > > > > > > > > > > > Part of the requirement of a release is that someone > > > can > > > > > > build > > > > > > > it on their > > > > > > > > > > > > > system, and that ideally should be easy to do. I > > would > > > > > assume > > > > > > > by their > > > > > > > > > > > > > system than means a single version of the JDK which > > > they > > > > > use > > > > > > > and is > > > > > > > > > > > > > supported. I’m not sure there is be a need for > > someone > > > > > > testing > > > > > > > the release > > > > > > > > > > > > > to compile against multiple JDKs? > > > > > > > > > > > > > > > > > > > > > > > > > > Kind Regards, > > > > > > > > > > > > > Justin > > > > > > > > > > > > > > > > > > > > > > > --------------------------------------------------------------------- > > > > > > > > > > > > > To unsubscribe, e-mail: > > > [email protected] > > > > > > > > > > > > > For additional commands, e-mail: > > > [email protected] > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > --------------------------------------------------------------------- > > > > > > > > > > > To unsubscribe, e-mail: [email protected] > > > > > > > > > > > For additional commands, e-mail: > > [email protected] > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > Arnout Engelen > > > > > > > > > > ASF Security Response > > > > > > > > > > Committer on Apache Pekko > > > > > > > > > > Committer on NixOS > > > > > > > > > > Independent Open Source consultant > > > > > > > > > > > > > > > > > > > > > > > > > > > > --------------------------------------------------------------------- > > > > > > > > > > To unsubscribe, e-mail: [email protected] > > > > > > > > > > For additional commands, e-mail: [email protected] > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > --------------------------------------------------------------------- > > > > > > > > > To unsubscribe, e-mail: [email protected] > > > > > > > > > For additional commands, e-mail: [email protected] > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > Arnout Engelen > > > > > > > > ASF Security Response > > > > > > > > Committer on Apache Pekko > > > > > > > > Committer on NixOS > > > > > > > > Independent Open Source consultant > > > > > > > > > > > > > > > > > > > --------------------------------------------------------------------- > > > > > > > > To unsubscribe, e-mail: [email protected] > > > > > > > > For additional commands, e-mail: [email protected] > > > > > > > > > > > > > > > > > > > > > > > > > --------------------------------------------------------------------- > > > > > > > To unsubscribe, e-mail: [email protected] > > > > > > > For additional commands, e-mail: [email protected] > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > > > Matthew de Detrich > > > > > > > > > > > > *Aiven Deutschland GmbH* > > > > > > > > > > > > Immanuelkirchstraße 26, 10405 Berlin > > > > > > > > > > > > Amtsgericht Charlottenburg, HRB 209739 B > > > > > > > > > > > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen > > > > > > > > > > > > *m:* +491603708037 > > > > > > > > > > > > *w:* aiven.io *e:* [email protected] > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > Matthew de Detrich > > > > > > > > *Aiven Deutschland GmbH* > > > > > > > > Immanuelkirchstraße 26, 10405 Berlin > > > > > > > > Amtsgericht Charlottenburg, HRB 209739 B > > > > > > > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen > > > > > > > > *m:* +491603708037 > > > > > > > > *w:* aiven.io *e:* [email protected] > > > > > > --------------------------------------------------------------------- > > > To unsubscribe, e-mail: [email protected] > > > For additional commands, e-mail: [email protected] > > > > > > > > > > -- > > > > Matthew de Detrich > > > > *Aiven Deutschland GmbH* > > > > Immanuelkirchstraße 26, 10405 Berlin > > > > Amtsgericht Charlottenburg, HRB 209739 B > > > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen > > > > *m:* +491603708037 > > > > *w:* aiven.io *e:* [email protected] > > --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
