Can we just document this? In the end of the day, we produce jars that can be used in Java 8 and up.
We intend to keep this for the 1.0.0 release. We might consider dropping Java 8 support at some stage. I work on a number of Open Source projects and many of them have similar requirements to use multiple JDKs to build the jars. The vast majority of users will use the jars that we publish to Maven Central. A small minority will build from source. Building from source is not as hard as some people are assuming it is. We can document it. We can answer questions for people who are having issues. On Thu, 25 May 2023 at 15:18, Claude Warren, Jr <[email protected]> wrote: > > Why not build 2 binary packages: Java 8 and Java 11? Use the java 8 JDK > to build the java 8 version and Java 11 JDK to build java 11 version. > > Then no need to copy rt.jar > > On Thu, May 25, 2023 at 2:35 PM Arnout Engelen <[email protected]> wrote: > > > On Thu, May 25, 2023 at 2:50 PM Claude Warren, Jr > > <[email protected]> wrote: > > > Assume Java 11 is installed and no Java 8 is installed on the system > > where > > > Pekko is being deployed. > > > Assume Java 11 compilation with rt.jar present for the build of the Pekko > > > package being deployed. (My understanding that even with Java 11 the > > rt.jar > > > is required for compilation) > > > > > > When the code is executed on the deployed system and the class from the > > > rt.jar is required, what happens? > > > I would expect that a NoClassDefFoundError to be thrown. If the error > > is > > > not thrown I can only see two possibilities: > > > > > > 1. the class(es) from the rt.jar are defined somewhere else in which > > > case why can that not be used during the build? > > > 2. the class(es) are not used and the dependency on the rt.jar can be > > > removed. > > > > > > So what happens and why? > > > > No NoClassDefFoundError will be thrown, because the runtime for Java > > 11 is backwards compatible with the runtime for Java 8, so there > > should be no classes/methods that exist in the jdk8 rt.jar that are > > not also available in the jdk11 runtime. > > > > From your two possibilities, '1' is true: they are defined both in the > > Java 8 and in the Java 11 runtimes. Indeed if we removed the rt.jar > > everything would still work on Java 11. Just no longer on Java 8. This > > is because the runtimes are not *forwards* compatible: there are > > things in the Java 11 runtime that are not in the Java 8 one. > > > > To illustrate and make this more tangible, one specific example is > > ByteBuffer.clear(): on Java 8, this method returned a Buffer. On Java > > 11, there is an implementation that returns a ByteBuffer. If you > > compile a class that calls this method against the Java 8 runtime and > > then run it against the Java 11 one, this is fine, because Java 11 has > > *both* a clear() method that returns a ByteBuffer and (for backwards > > compatibility) one that returns a Buffer. If you compile the same code > > against the Java 11 runtime and then run it against Java 8, it will > > pick the one that returns a ByteBuffer at compile time, and you will > > get a NoSuchMethod exception at runtime because this new method does > > not exist in the Java 8 runtime. > > > > Of course we could change the code to make sure this *particular* > > problem does not occur in Pekko, but that is risky as we don't know > > how many similar situations there are. > > > > Is there a better solution to this than compiling with two JVM's > > installed? Yes! Invoking the JDK11 compiler with '--release 8' will > > also generate the correct reference. However, '--release 8' does not > > allow access to sun.misc.Unsafe API's, which unfortunately Pekko still > > relies on - so we can't do that just yet. > > > > > > Kind regards, > > > > Arnout > > > > > On Wed, May 24, 2023 at 9:23 PM Matthew Benedict de Detrich > > > <[email protected]> wrote: > > > > > > > > If you build Pekko with both Java 11 and Java 8 installed, the > > classes > > > > we will produce in our jars will be in Java 8 class format. > > > > > > > > More specifically, normally if you compile Java/Scala source with JDK > > 11 > > > > (which is what Pekko does, JDK 8 is only necessary for its rt.jar) it > > will > > > > produce > > > > classfiles compatible with JDK 11 and above. You can however provide a > > flag > > > > to > > > > javac/scalac so that it generates class files with a specific bytecode > > > > format, i.e. > > > > > > > > > > https://github.com/apache/incubator-pekko/blob/eaea9b116f4c47aed1686d62adb674a37600f6b2/project/JdkOptions.scala#L60 > > > > and > > > > > > > > > > https://github.com/apache/incubator-pekko/blob/eaea9b116f4c47aed1686d62adb674a37600f6b2/project/JdkOptions.scala#L73 > > > > does. > > > > > > > > In short, Pekko is built using JDK 11 with Xtarget/release flags to > > force > > > > JDK 8 > > > > compatible bytecode generation in addition to providing the JDK8 > > rt.jar via > > > > the > > > > -bootclasspath javac flag. > > > > > > > > On Wed, May 24, 2023 at 9:44 PM PJ Fanning <[email protected]> > > wrote: > > > > > > > > > > Q: does the build package the rt.jar in the final binary package? > > > > > > > > > > No. We only produce our own jars that we push to Maven - and these do > > > > > not embed any rt.jar classes (shaded or otherwise). We have no other > > > > > type of 'binary' package. > > > > > > > > > > > Q: if it does not, what happens if you try to run using Java 11 on > > a > > > > > system > > > > > > that does not have Java 8 nor the rt.jar installed? > > > > > > > > > > Java compiler produces class files. These have a Java version > > > > > associated with them and you can't use a class that is in Java 11 > > > > > format in a Java 8 runtime. > > > > > If you build Pekko with both Java 11 and Java 8 installed, the > > classes > > > > > we will produce in our jars will be in Java 8 class format. > > > > > > > > > > On Wed, 24 May 2023 at 16:45, Matthew Benedict de Detrich > > > > > <[email protected]> wrote: > > > > > > > > > > > > afaik Pekko will just fail to build at all if you use JDK 11 > > without > > > > > > pointing to JDK 8 rt.jar. > > > > > > > > > > > > This is because Pekko uses sun.misc.unsafe which is only shipping > > with > > > > > JDK > > > > > > 8 and the > > > > > > reason why we have to build on JDK 11 is because there are features > > > > > > shipping in JDK 11 > > > > > > which pekko uses. > > > > > > > > > > > > On Wed, May 24, 2023 at 5:40 PM Claude Warren, Jr > > > > > > <[email protected]> wrote: > > > > > > > > > > > > > I have a couple of questions: > > > > > > > > > > > > > > Q: does the build package the rt.jar in the final binary package? > > > > > > > > > > > > > > Q: if it does not, what happens if you try to run using Java 11 > > on a > > > > > system > > > > > > > that does not have Java 8 nor the rt.jar installed? > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Wed, May 24, 2023 at 4:22 PM Matthew Benedict de Detrich > > > > > > > <[email protected]> wrote: > > > > > > > > > > > > > > > > We could provide a Docker image with the right sbt and Java > > setup > > > > > for > > > > > > > > convenience but I think it is fair not to require users to use > > > > > Docker. > > > > > > > > > > > > > > > > Considering how much implicit behaviour there is in making a > > full > > > > > build > > > > > > > > (as Arnout mentioned earlier, you can compile Pekko fine with > > just > > > > > JDK 8 > > > > > > > > just fine however it will be missing post java 8 optional > > > > features) I > > > > > > > would > > > > > > > > prefer > > > > > > > > to strongly encourage people to use Docker to do a proper > > build. > > > > As a > > > > > > > bare > > > > > > > > minimum we can also print a warning in sbt telling users if > > they > > > > are > > > > > not > > > > > > > > going > > > > > > > > to be making a complete Pekko build when building on JDK 8 (as > > > > > stated it > > > > > > > > already errors out if you build on JDK 11 without pointing to > > a JDK > > > > > > > > rt.jar). > > > > > > > > > > > > > > > > On Wed, May 24, 2023 at 11:22 AM PJ Fanning < > > [email protected]> > > > > > wrote: > > > > > > > > > > > > > > > > > Thanks Arnout. > > > > > > > > > > > > > > > > > > I think if we clearly document what you need to set up to do > > a > > > > > build, > > > > > > > > > that should be enough. I think it is fair to assume that > > users > > > > can > > > > > > > > > install Java and sbt - and that they should be able to > > install 2 > > > > > > > > > versions of Java if we are explicit about how to do this and > > why > > > > > this > > > > > > > > > is the best setup for building Pekko. > > > > > > > > > > > > > > > > > > We will need to expand what we have in > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://github.com/apache/incubator-pekko/blob/main/README.md#building-from-source > > > > > > > > > > > > > > > > > > This README file will appear in the source releases. > > > > > > > > > > > > > > > > > > We could provide a Docker image with the right sbt and Java > > setup > > > > > for > > > > > > > > > convenience but I think it is fair not to require users to > > use > > > > > Docker. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Wed, 24 May 2023 at 10:09, Arnout Engelen < > > [email protected] > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > On Wed, May 24, 2023 at 10:57 AM PJ Fanning < > > > > [email protected] > > > > > > > > > > > > > > wrote: > > > > > > > > > > > Ok. It seems the nightly job to publish the snapshot jars > > > > uses > > > > > Java > > > > > > > > > > > 11. It does not explicitly install Java 8 but maybe, it's > > > > > possible > > > > > > > > > > > that Java 8 is installed silently, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://github.com/actions/runner-images/blob/main/images/linux/Ubuntu2004-Readme.md > > > > > > > > > > suggests so, yes. > > > > > > > > > > > > > > > > > > > > > The nightly build does produce jars that can be used by > > Java > > > > 8. > > > > > > > > > > > > > > > > > > > > > > So if this nightly build relies on the silent > > installation of > > > > > Java > > > > > > > 8, > > > > > > > > > > > should we add a check in our sbt script that will fail if > > > > Java > > > > > 8 is > > > > > > > > > > > not found? This could be useful for users who are > > building on > > > > > their > > > > > > > > > > > own machines. > > > > > > > > > > > > > > > > > > > > This check already exists: when I build with jdk11 without > > > > having > > > > > > > jdk8 > > > > > > > > > > available the build fails with: > > > > > > > > > > > > > > > > > > > > [error] A JDK 8 installation was not found, but is > > required to > > > > > build > > > > > > > > > > Apache Pekko. To manually specify a JDK 8 installation, > > use the > > > > > "set > > > > > > > > > > every jdk8home := \"/path/to/jdk\" sbt command. If you > > have no > > > > > JDK 8 > > > > > > > > > > installation, target your system JDK with the "set every > > > > > > > > > > targetSystemJdk := true" sbt command, but beware resulting > > > > > artifacts > > > > > > > > > > will not work on JDK 8 > > > > > > > > > > > > > > > > > > > > It would be good to also document the JAVA_8_HOME > > environment > > > > > > > variable > > > > > > > > > > in this error message. Created > > > > > > > > > > https://github.com/apache/incubator-pekko/pull/333 for > > that. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Kind regards, > > > > > > > > > > > > > > > > > > > > Arnout > > > > > > > > > > > > > > > > > > > > > On Wed, 24 May 2023 at 09:38, Arnout Engelen < > > > > > [email protected]> > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > On Wed, May 24, 2023 at 10:12 AM PJ Fanning < > > > > > [email protected] > > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > Only building the website docs needs Java 11 (and > > up). > > > > > > > Everything > > > > > > > > > else > > > > > > > > > > > > > builds with Java 8 (and up). > > > > > > > > > > > > > > > > > > > > > > > > Yes, though builds created with only Java 8 will miss > > some > > > > > > > optional > > > > > > > > > > > > post-Java-8 features/implementations for the > > "actor-typed", > > > > > > > > "stream", > > > > > > > > > > > > "remote" and "cluster-sharding" modules - so for a > > faithful > > > > > test > > > > > > > > > you'd > > > > > > > > > > > > ideally build with JDK 9-or-later while having the > > Java 8 > > > > > runtime > > > > > > > > > > > > available. > > > > > > > > > > > > > > > > > > > > > > > > (TBH I don't think this is _too_ different from other > > > > > projects > > > > > > > > where > > > > > > > > > > > > you have to have some compile-time dependencies > > installed > > > > to > > > > > be > > > > > > > > able > > > > > > > > > > > > to build the project) > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Kind regards, > > > > > > > > > > > > > > > > > > > > > > > > Arnout > > > > > > > > > > > > > > > > > > > > > > > > > On Wed, 24 May 2023 at 08:05, Claude Warren, Jr > > > > > > > > > > > > > <[email protected]> wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > Justin, > > > > > > > > > > > > > > > > > > > > > > > > > > > > There is an issue in the code base that is > > reasonably > > > > > common > > > > > > > > > among systems > > > > > > > > > > > > > > that operate at low levels and are migrating from > > Java > > > > 8 > > > > > > > > > implementation > > > > > > > > > > > > > > wherein they need the rt.jar from java 8 for > > > > compilation. > > > > > > > > > There are > > > > > > > > > > > > > > several ways to solve this problem, Pekko (and I > > > > suppose > > > > > > > Akka) > > > > > > > > > require that > > > > > > > > > > > > > > Java 8 be installed so that the build process can > > find > > > > > the > > > > > > > Java > > > > > > > > > 8 rt.jar. > > > > > > > > > > > > > > I suspect that all that is actually required to > > get the > > > > > > > current > > > > > > > > > code to > > > > > > > > > > > > > > compile is setting JAVA_8_HOME to point to the > > > > directory > > > > > with > > > > > > > > > the rt.jar in > > > > > > > > > > > > > > it. However, the Java 8 rt.jar is still > > required. I > > > > > believe > > > > > > > > > there is a > > > > > > > > > > > > > > ticket to resolve this issue but I may be mistaken > > > > about > > > > > > > this. > > > > > > > > > > > > > > > > > > > > > > > > > > > > Claude > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Wed, May 24, 2023 at 7:38 AM Justin Mclean < > > > > > > > > > [email protected]> > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Hi, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Part of the requirement of a release is that > > someone > > > > > can > > > > > > > > build > > > > > > > > > it on their > > > > > > > > > > > > > > > system, and that ideally should be easy to do. I > > > > would > > > > > > > assume > > > > > > > > > by their > > > > > > > > > > > > > > > system than means a single version of the JDK > > which > > > > > they > > > > > > > use > > > > > > > > > and is > > > > > > > > > > > > > > > supported. I’m not sure there is be a need for > > > > someone > > > > > > > > testing > > > > > > > > > the release > > > > > > > > > > > > > > > to compile against multiple JDKs? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Kind Regards, > > > > > > > > > > > > > > > Justin > > > > > > > > > > > > > > > > > > > > > > > > > > > > > --------------------------------------------------------------------- > > > > > > > > > > > > > > > To unsubscribe, e-mail: > > > > > [email protected] > > > > > > > > > > > > > > > For additional commands, e-mail: > > > > > [email protected] > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > --------------------------------------------------------------------- > > > > > > > > > > > > > To unsubscribe, e-mail: > > [email protected] > > > > > > > > > > > > > For additional commands, e-mail: > > > > [email protected] > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > > > Arnout Engelen > > > > > > > > > > > > ASF Security Response > > > > > > > > > > > > Committer on Apache Pekko > > > > > > > > > > > > Committer on NixOS > > > > > > > > > > > > Independent Open Source consultant > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > --------------------------------------------------------------------- > > > > > > > > > > > > To unsubscribe, e-mail: > > [email protected] > > > > > > > > > > > > For additional commands, e-mail: > > [email protected] > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > --------------------------------------------------------------------- > > > > > > > > > > > To unsubscribe, e-mail: [email protected] > > > > > > > > > > > For additional commands, e-mail: > > [email protected] > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > Arnout Engelen > > > > > > > > > > ASF Security Response > > > > > > > > > > Committer on Apache Pekko > > > > > > > > > > Committer on NixOS > > > > > > > > > > Independent Open Source consultant > > > > > > > > > > > > > > > > > > > > > > > > > --------------------------------------------------------------------- > > > > > > > > > > To unsubscribe, e-mail: [email protected] > > > > > > > > > > For additional commands, e-mail: [email protected] > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > --------------------------------------------------------------------- > > > > > > > > > To unsubscribe, e-mail: [email protected] > > > > > > > > > For additional commands, e-mail: [email protected] > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > > > > > > > Matthew de Detrich > > > > > > > > > > > > > > > > *Aiven Deutschland GmbH* > > > > > > > > > > > > > > > > Immanuelkirchstraße 26, 10405 Berlin > > > > > > > > > > > > > > > > Amtsgericht Charlottenburg, HRB 209739 B > > > > > > > > > > > > > > > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen > > > > > > > > > > > > > > > > *m:* +491603708037 > > > > > > > > > > > > > > > > *w:* aiven.io *e:* [email protected] > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > > > Matthew de Detrich > > > > > > > > > > > > *Aiven Deutschland GmbH* > > > > > > > > > > > > Immanuelkirchstraße 26, 10405 Berlin > > > > > > > > > > > > Amtsgericht Charlottenburg, HRB 209739 B > > > > > > > > > > > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen > > > > > > > > > > > > *m:* +491603708037 > > > > > > > > > > > > *w:* aiven.io *e:* [email protected] > > > > > > > > > > --------------------------------------------------------------------- > > > > > To unsubscribe, e-mail: [email protected] > > > > > For additional commands, e-mail: [email protected] > > > > > > > > > > > > > > > > > > -- > > > > > > > > Matthew de Detrich > > > > > > > > *Aiven Deutschland GmbH* > > > > > > > > Immanuelkirchstraße 26, 10405 Berlin > > > > > > > > Amtsgericht Charlottenburg, HRB 209739 B > > > > > > > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen > > > > > > > > *m:* +491603708037 > > > > > > > > *w:* aiven.io *e:* [email protected] > > > > > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: [email protected] > > For additional commands, e-mail: [email protected] > > > > --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
