> Assume Java 11 is installed and no Java 8 is installed on the system where Pekko is being deployed.
If by deployed you mean compile (i.e. generate .class files) this will just fail to compile with the following error [error] A JDK 8 installation was not found, but is required to build Apache Pekko. To manually specify a JDK 8 installation, set the JAVA_8_HOME environment variable to its path or use the "set every jdk8home := \"/path/to/jdk\" sbt command. If you have no JDK 8 installation, target your system JDK with the "set every targetSystemJdk := true" sbt command, but beware resulting artifacts will not work on JDK 8 If by deployed you mean running, Pekko will run fine on JDK 8. We have github actions CI setup that triggers on every pull request to verify this. > Assume Java 11 compilation with rt.jar present for the build of the Pekko package being deployed. (My understanding that even with Java 11 the rt.jar is required for compilation) Under the hood when Pekko is built correctly it uses JDK 11 with specific javac/scalc flags to generate JDK 8 bytecode will passing in JDK 8's rt.jar using -bootclasspath> > When the code is executed on the deployed system and the class from the rt.jar is required, what happens? Afaik The reason why rt.jar from JDK 8 is required is otherwise javac will fail in linking during compile face because it cannot find references to sun.misc.unsafe within JDK 8's rt.jar that Pekko uses (and this is because we are building with JDK 11) > I would expect that a NoClassDefFoundError to be thrown. If the error is not thrown I can only see two possibilities: It either wouldn't compile or you would get a NoMethodFound error or something along those lines at runtime (as you state). In Pekko you have source code that references certain methods in sun.misc.unsafe which was removed in JDK 11 (or more specifically JDK 11's rt.jar doesn't contain does methods). On a related note, sun.misc.unsafe is a critical part of Pekko. It's used in the core actor module for understandable reasons (actor's need to be as fast as possible, it's not uncommon to have millions of instances of actors just like in Erlang and it's the core concurrency primitive in Akka). In order to not use sun.misc.unsafe we have to migrate to the more modern replacements. With this being said, while it's definitely enlightening to have this conversation to figure out how and why Pekko compiles/builds, I would be cautious in deliberating on this point too much. Without dropping JDK 8 we can't really solve this issue, and as pointed out by Arnout the sbt build has already been very comprehensively setup so that you cannot shoot yourself in the foot (my very minor complaint is that if Pekko sees that you are building on Java 8 it should warn you that certain artifacts will not have full functionality). The best way forward is to simply document and to make things easier (i.e. providing a Docker build). Even if we could improve this somehow, there is a very big risk we can break something silently without knowing, last time I tried to improve this part of the build it involved having to bring in maintainers of Scalac (see https://github.com/apache/incubator-pekko/pull/258) along with manually inspecting JVM bytecode to make sure everything went as planned so I would say thats its not a controversial statement that the community (especially myself) doesn't really have a tolerance for touching this part of the code right now. A much more ideal solution would be to just make a release via github actions CI as we do now with snapshots, but for various reasons (some related to ASF process and some technical) we are not at the stage to explore it and even if we could it would very likely be a lot more work than a Docker build hence the timing not the best. On Thu, May 25, 2023 at 2:50 PM Claude Warren, Jr <[email protected]> wrote: > Assume Java 11 is installed and no Java 8 is installed on the system where > Pekko is being deployed. > Assume Java 11 compilation with rt.jar present for the build of the Pekko > package being deployed. (My understanding that even with Java 11 the rt.jar > is required for compilation) > > When the code is executed on the deployed system and the class from the > rt.jar is required, what happens? > I would expect that a NoClassDefFoundError to be thrown. If the error is > not thrown I can only see two possibilities: > > 1. the class(es) from the rt.jar are defined somewhere else in which > case why can that not be used during the build? > 2. the class(es) are not used and the dependency on the rt.jar can be > removed. > > So what happens and why? > > > On Wed, May 24, 2023 at 9:23 PM Matthew Benedict de Detrich > <[email protected]> wrote: > > > > If you build Pekko with both Java 11 and Java 8 installed, the classes > > we will produce in our jars will be in Java 8 class format. > > > > More specifically, normally if you compile Java/Scala source with JDK 11 > > (which is what Pekko does, JDK 8 is only necessary for its rt.jar) it > will > > produce > > classfiles compatible with JDK 11 and above. You can however provide a > flag > > to > > javac/scalac so that it generates class files with a specific bytecode > > format, i.e. > > > > > https://github.com/apache/incubator-pekko/blob/eaea9b116f4c47aed1686d62adb674a37600f6b2/project/JdkOptions.scala#L60 > > and > > > > > https://github.com/apache/incubator-pekko/blob/eaea9b116f4c47aed1686d62adb674a37600f6b2/project/JdkOptions.scala#L73 > > does. > > > > In short, Pekko is built using JDK 11 with Xtarget/release flags to force > > JDK 8 > > compatible bytecode generation in addition to providing the JDK8 rt.jar > via > > the > > -bootclasspath javac flag. > > > > On Wed, May 24, 2023 at 9:44 PM PJ Fanning <[email protected]> wrote: > > > > > > Q: does the build package the rt.jar in the final binary package? > > > > > > No. We only produce our own jars that we push to Maven - and these do > > > not embed any rt.jar classes (shaded or otherwise). We have no other > > > type of 'binary' package. > > > > > > > Q: if it does not, what happens if you try to run using Java 11 on a > > > system > > > > that does not have Java 8 nor the rt.jar installed? > > > > > > Java compiler produces class files. These have a Java version > > > associated with them and you can't use a class that is in Java 11 > > > format in a Java 8 runtime. > > > If you build Pekko with both Java 11 and Java 8 installed, the classes > > > we will produce in our jars will be in Java 8 class format. > > > > > > On Wed, 24 May 2023 at 16:45, Matthew Benedict de Detrich > > > <[email protected]> wrote: > > > > > > > > afaik Pekko will just fail to build at all if you use JDK 11 without > > > > pointing to JDK 8 rt.jar. > > > > > > > > This is because Pekko uses sun.misc.unsafe which is only shipping > with > > > JDK > > > > 8 and the > > > > reason why we have to build on JDK 11 is because there are features > > > > shipping in JDK 11 > > > > which pekko uses. > > > > > > > > On Wed, May 24, 2023 at 5:40 PM Claude Warren, Jr > > > > <[email protected]> wrote: > > > > > > > > > I have a couple of questions: > > > > > > > > > > Q: does the build package the rt.jar in the final binary package? > > > > > > > > > > Q: if it does not, what happens if you try to run using Java 11 on > a > > > system > > > > > that does not have Java 8 nor the rt.jar installed? > > > > > > > > > > > > > > > > > > > > On Wed, May 24, 2023 at 4:22 PM Matthew Benedict de Detrich > > > > > <[email protected]> wrote: > > > > > > > > > > > > We could provide a Docker image with the right sbt and Java > setup > > > for > > > > > > convenience but I think it is fair not to require users to use > > > Docker. > > > > > > > > > > > > Considering how much implicit behaviour there is in making a full > > > build > > > > > > (as Arnout mentioned earlier, you can compile Pekko fine with > just > > > JDK 8 > > > > > > just fine however it will be missing post java 8 optional > > features) I > > > > > would > > > > > > prefer > > > > > > to strongly encourage people to use Docker to do a proper build. > > As a > > > > > bare > > > > > > minimum we can also print a warning in sbt telling users if they > > are > > > not > > > > > > going > > > > > > to be making a complete Pekko build when building on JDK 8 (as > > > stated it > > > > > > already errors out if you build on JDK 11 without pointing to a > JDK > > > > > > rt.jar). > > > > > > > > > > > > On Wed, May 24, 2023 at 11:22 AM PJ Fanning <[email protected] > > > > > wrote: > > > > > > > > > > > > > Thanks Arnout. > > > > > > > > > > > > > > I think if we clearly document what you need to set up to do a > > > build, > > > > > > > that should be enough. I think it is fair to assume that users > > can > > > > > > > install Java and sbt - and that they should be able to install > 2 > > > > > > > versions of Java if we are explicit about how to do this and > why > > > this > > > > > > > is the best setup for building Pekko. > > > > > > > > > > > > > > We will need to expand what we have in > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://github.com/apache/incubator-pekko/blob/main/README.md#building-from-source > > > > > > > > > > > > > > This README file will appear in the source releases. > > > > > > > > > > > > > > We could provide a Docker image with the right sbt and Java > setup > > > for > > > > > > > convenience but I think it is fair not to require users to use > > > Docker. > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Wed, 24 May 2023 at 10:09, Arnout Engelen < > [email protected] > > > > > > > > wrote: > > > > > > > > > > > > > > > > On Wed, May 24, 2023 at 10:57 AM PJ Fanning < > > [email protected] > > > > > > > > > > wrote: > > > > > > > > > Ok. It seems the nightly job to publish the snapshot jars > > uses > > > Java > > > > > > > > > 11. It does not explicitly install Java 8 but maybe, it's > > > possible > > > > > > > > > that Java 8 is installed silently, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://github.com/actions/runner-images/blob/main/images/linux/Ubuntu2004-Readme.md > > > > > > > > suggests so, yes. > > > > > > > > > > > > > > > > > The nightly build does produce jars that can be used by > Java > > 8. > > > > > > > > > > > > > > > > > > So if this nightly build relies on the silent installation > of > > > Java > > > > > 8, > > > > > > > > > should we add a check in our sbt script that will fail if > > Java > > > 8 is > > > > > > > > > not found? This could be useful for users who are building > on > > > their > > > > > > > > > own machines. > > > > > > > > > > > > > > > > This check already exists: when I build with jdk11 without > > having > > > > > jdk8 > > > > > > > > available the build fails with: > > > > > > > > > > > > > > > > [error] A JDK 8 installation was not found, but is required > to > > > build > > > > > > > > Apache Pekko. To manually specify a JDK 8 installation, use > the > > > "set > > > > > > > > every jdk8home := \"/path/to/jdk\" sbt command. If you have > no > > > JDK 8 > > > > > > > > installation, target your system JDK with the "set every > > > > > > > > targetSystemJdk := true" sbt command, but beware resulting > > > artifacts > > > > > > > > will not work on JDK 8 > > > > > > > > > > > > > > > > It would be good to also document the JAVA_8_HOME environment > > > > > variable > > > > > > > > in this error message. Created > > > > > > > > https://github.com/apache/incubator-pekko/pull/333 for that. > > > > > > > > > > > > > > > > > > > > > > > > Kind regards, > > > > > > > > > > > > > > > > Arnout > > > > > > > > > > > > > > > > > On Wed, 24 May 2023 at 09:38, Arnout Engelen < > > > [email protected]> > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > On Wed, May 24, 2023 at 10:12 AM PJ Fanning < > > > [email protected] > > > > > > > > > > > > > wrote: > > > > > > > > > > > Only building the website docs needs Java 11 (and up). > > > > > Everything > > > > > > > else > > > > > > > > > > > builds with Java 8 (and up). > > > > > > > > > > > > > > > > > > > > Yes, though builds created with only Java 8 will miss > some > > > > > optional > > > > > > > > > > post-Java-8 features/implementations for the > "actor-typed", > > > > > > "stream", > > > > > > > > > > "remote" and "cluster-sharding" modules - so for a > faithful > > > test > > > > > > > you'd > > > > > > > > > > ideally build with JDK 9-or-later while having the Java 8 > > > runtime > > > > > > > > > > available. > > > > > > > > > > > > > > > > > > > > (TBH I don't think this is _too_ different from other > > > projects > > > > > > where > > > > > > > > > > you have to have some compile-time dependencies installed > > to > > > be > > > > > > able > > > > > > > > > > to build the project) > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Kind regards, > > > > > > > > > > > > > > > > > > > > Arnout > > > > > > > > > > > > > > > > > > > > > On Wed, 24 May 2023 at 08:05, Claude Warren, Jr > > > > > > > > > > > <[email protected]> wrote: > > > > > > > > > > > > > > > > > > > > > > > > Justin, > > > > > > > > > > > > > > > > > > > > > > > > There is an issue in the code base that is reasonably > > > common > > > > > > > among systems > > > > > > > > > > > > that operate at low levels and are migrating from > Java > > 8 > > > > > > > implementation > > > > > > > > > > > > wherein they need the rt.jar from java 8 for > > compilation. > > > > > > > There are > > > > > > > > > > > > several ways to solve this problem, Pekko (and I > > suppose > > > > > Akka) > > > > > > > require that > > > > > > > > > > > > Java 8 be installed so that the build process can > find > > > the > > > > > Java > > > > > > > 8 rt.jar. > > > > > > > > > > > > I suspect that all that is actually required to get > the > > > > > current > > > > > > > code to > > > > > > > > > > > > compile is setting JAVA_8_HOME to point to the > > directory > > > with > > > > > > > the rt.jar in > > > > > > > > > > > > it. However, the Java 8 rt.jar is still required. I > > > believe > > > > > > > there is a > > > > > > > > > > > > ticket to resolve this issue but I may be mistaken > > about > > > > > this. > > > > > > > > > > > > > > > > > > > > > > > > Claude > > > > > > > > > > > > > > > > > > > > > > > > On Wed, May 24, 2023 at 7:38 AM Justin Mclean < > > > > > > > [email protected]> > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > Hi, > > > > > > > > > > > > > > > > > > > > > > > > > > Part of the requirement of a release is that > someone > > > can > > > > > > build > > > > > > > it on their > > > > > > > > > > > > > system, and that ideally should be easy to do. I > > would > > > > > assume > > > > > > > by their > > > > > > > > > > > > > system than means a single version of the JDK which > > > they > > > > > use > > > > > > > and is > > > > > > > > > > > > > supported. I’m not sure there is be a need for > > someone > > > > > > testing > > > > > > > the release > > > > > > > > > > > > > to compile against multiple JDKs? > > > > > > > > > > > > > > > > > > > > > > > > > > Kind Regards, > > > > > > > > > > > > > Justin > > > > > > > > > > > > > > > > > > > > > > > --------------------------------------------------------------------- > > > > > > > > > > > > > To unsubscribe, e-mail: > > > [email protected] > > > > > > > > > > > > > For additional commands, e-mail: > > > [email protected] > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > --------------------------------------------------------------------- > > > > > > > > > > > To unsubscribe, e-mail: > [email protected] > > > > > > > > > > > For additional commands, e-mail: > > [email protected] > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > Arnout Engelen > > > > > > > > > > ASF Security Response > > > > > > > > > > Committer on Apache Pekko > > > > > > > > > > Committer on NixOS > > > > > > > > > > Independent Open Source consultant > > > > > > > > > > > > > > > > > > > > > > > > > > > > --------------------------------------------------------------------- > > > > > > > > > > To unsubscribe, e-mail: [email protected] > > > > > > > > > > For additional commands, e-mail: > [email protected] > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > --------------------------------------------------------------------- > > > > > > > > > To unsubscribe, e-mail: [email protected] > > > > > > > > > For additional commands, e-mail: [email protected] > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > Arnout Engelen > > > > > > > > ASF Security Response > > > > > > > > Committer on Apache Pekko > > > > > > > > Committer on NixOS > > > > > > > > Independent Open Source consultant > > > > > > > > > > > > > > > > > > > --------------------------------------------------------------------- > > > > > > > > To unsubscribe, e-mail: [email protected] > > > > > > > > For additional commands, e-mail: [email protected] > > > > > > > > > > > > > > > > > > > > > > > > > --------------------------------------------------------------------- > > > > > > > To unsubscribe, e-mail: [email protected] > > > > > > > For additional commands, e-mail: [email protected] > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > > > Matthew de Detrich > > > > > > > > > > > > *Aiven Deutschland GmbH* > > > > > > > > > > > > Immanuelkirchstraße 26, 10405 Berlin > > > > > > > > > > > > Amtsgericht Charlottenburg, HRB 209739 B > > > > > > > > > > > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen > > > > > > > > > > > > *m:* +491603708037 > > > > > > > > > > > > *w:* aiven.io *e:* [email protected] > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > Matthew de Detrich > > > > > > > > *Aiven Deutschland GmbH* > > > > > > > > Immanuelkirchstraße 26, 10405 Berlin > > > > > > > > Amtsgericht Charlottenburg, HRB 209739 B > > > > > > > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen > > > > > > > > *m:* +491603708037 > > > > > > > > *w:* aiven.io *e:* [email protected] > > > > > > --------------------------------------------------------------------- > > > To unsubscribe, e-mail: [email protected] > > > For additional commands, e-mail: [email protected] > > > > > > > > > > -- > > > > Matthew de Detrich > > > > *Aiven Deutschland GmbH* > > > > Immanuelkirchstraße 26, 10405 Berlin > > > > Amtsgericht Charlottenburg, HRB 209739 B > > > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen > > > > *m:* +491603708037 > > > > *w:* aiven.io *e:* [email protected] > > > -- Matthew de Detrich *Aiven Deutschland GmbH* Immanuelkirchstraße 26, 10405 Berlin Amtsgericht Charlottenburg, HRB 209739 B Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen *m:* +491603708037 *w:* aiven.io *e:* [email protected]
