In my opinion we can stay on 8, we should change this kind of stuff at the beginning of a new iteration and not before the release. IIRC We are not in a hurry, having a stable build process is a value.
Enrico Il Mer 6 Gen 2021, 17:51 Damien Diederen <ddiede...@apache.org> ha scritto: > > Dear ZooKeeper team, > > Andor just reminded me of this JDK 11 vs. 8 discussion, for which we did > not reach a resolution. Do we want to make a move for the 3.7.0 release? > > The original proposal, by Christopher, can be found here: > > > https://mail-archives.apache.org/mod_mbox/zookeeper-dev/202010.mbox/%3ccal5zq9z2tnfawjz7etxpf91qmrovuwv+khnxgpsd_msekfp...@mail.gmail.com%3e > > I am explicitly replying to Christopher's nice and useful summary, which > you will find directly below. An archived copy can be found here: > > > https://mail-archives.apache.org/mod_mbox/zookeeper-dev/202010.mbox/%3cCAL5zq9ao9-4S9bmzxaYJM8n8=aolxidz-0zohunvnayqei0...@mail.gmail.com%3e > > It contains three proposals, to which we can add the trivial one: > > 1. Compile/test with JDK 11, support JDK 8 "passively"; > > 2. Compile/test with JDK 11, test client only with JDK 8 (requires test > suite adaptations); > > 3. Move to JDK 11 fully; > > 4. Postpone any change to 3.8 (or 4.0). > > Flavio wrote: > > > [Christopher] mentioned a PMC vote, and I don't think this should be a > > closed vote, independent of how the conversation goes. > > Do we have a better idea of what we want? Or should we organize a vote? > > Cheers, -D > > > --8<--------------- original message ------------->8--- > > Christopher <ctubb...@apache.org> writes: > > I'm happy that this discussion has been so lively! I just want to > > emphasize a few things: > > > > I really do understand the desire to continue to support Java 8... I > > get it. But all the conversations around this seem based on what > > people are doing *today*. But, ZK 3.7 is *tomorrow's* version... a > > *future* release... so it should be based more on reasonable > > expectations for users in the future, and less based on what is > > happening today. I suspect *most* people today are still using 3.4 > > anyway (it was just so stable for so long...), but that shouldn't mean > > the developers should hold back development on 3.5 and 3.6, any more > > than today's users of 3.5/3.6 should hold back 3.7. > > > > Some of the opinions expressed in this discussion seem to propose a > > scenario where users are going to be updating to "bleeding edge" > > versions of ZooKeeper, but are going to insist on using Java 8. > > Personally, I find this to be implausible. In my experience, people > > either upgrade everything as soon as they are able to, or they upgrade > > each thing individually, only when they are forced to. The first group > > will be happy to move to Java 11 and ZK 3.7. The second group will > > probably avoid 3.7 anyway, and are fine sticking with 3.6, but if they > > had to update to 3.7, they'd also be fine updating to Java 11 if they > > had to in order to use 3.7. I can't imagine the scenario where people > > are eagerly choosing to upgrade to ZK 3.7, but miserly insisting on > > using Java 8. Perhaps that scenario exists, but it's hard for me to > > imagine. Even so, my proposal would still support even that group of > > people. > > > > I think there are now effectively three proposals being discussed in > > this thread: > > > > 1. (Christopher's original proposal) passively support Java 8 at > > runtime by making JDK 11 the minimum requirement to build and test. > > This scenario involves continuing to fix bugs, as they are discovered > > and reported, that affect JDK 8, but passively, rather than > > proactively. This proposal does *not* drop Java 8 support, but merely > > de-emphasizes it in development of what will be 3.7 in the future, and > > drops the requirement to do dedicated testing with Java 8. I think > > this is low risk, because it is very unlikely that the ZK devs would > > introduce a bug that would affect only Java 8 and the compiler > > wouldn't catch it... because the cross-compilation features of newer > > JDKs are really good. > > > > 2. (Enrico's alternate proposal) this variation of my proposal would > > involve continuing to proactively support Java 8 by creating a > > dedicated testing suite to test client code on Java 8. I think this is > > a good option, but since it involves a significantly higher amount of > > work than option 1, I think the cost-benefit analysis would show this > > to be not worth the effort. Also, if it were implemented, it would > > need to be done carefully to avoid requiring developers to have > > concurrently installed both Java 8 and Java 11 in order to perform a > > build, because requiring Java 8 at build time while developing would > > be worse than we have today. > > > > 3. (Andor's preference) move to JDK 11 fully. This would provide no > > support, passive or active, for Java 8 in ZK 3.7. To be honest, this > > is my personal favorite, and is the simplest to implement and > > communicate clearly to end users in release notes. The only reason I > > proposed a passive support of Java 8 instead of this is because I was > > trying to seek a compromise from the start. But, I think by far, this > > is the best option for the next *future* release of ZK. If you wanted > > to make the change even more visible to users, the version could even > > be bumped to 4.0. > > > > If this were to come to a VOTE by the PMC, in order to make a final > > decision, I would recommend they vote on option 3, and then if that > > fails, vote on option 1, and if that fails, keep things the way they > > are (because option 2 is more work). > > > > Christopher > > > > P.S. as for Hadoop on Java 11... I've been running Hadoop 3 on JDK 11 > > and it works just fine there (as long as you add the missing runtime > > jar for javax.activation:javax.activation-api:1.2.0 to its class path, > > but that was fixed in Hadoop 3.3). > > > > On Wed, Oct 21, 2020 at 5:42 PM Tamas Penzes > > <tam...@cloudera.com.invalid> wrote: > >> > >> Hi All, > >> > >> I've just talked with a Hadoop/HDFS developer who told me what I > guessed. > >> With Hadoop3 they have just dropped JDK7 support, dropping JDK8 would > mean > >> a release of Hadoop4. > >> Since HADOOP-15338 is finished, they test with JDK8 and JDK11 both. As > of > >> today most of the Hadoop users are still on Hadoop2, he doesn't expect > >> Hadoop4 soon. > >> As many Apache components depend on Hadoop and ZooKeeper they won't > hurry > >> to JDK11 until they have to (they will probably go one-by-one very > slowly), > >> which means if Hadoop stays on JDK8, they would use the last ZK version > >> which works on JDK8. > >> Do we want a ZK 3.4 again? > >> > >> Regards, Tamaas > >> > >> > >> On Wed, Oct 21, 2020 at 11:23 PM Tamas Penzes <tam...@cloudera.com> > wrote: > >> > >> > Hi All, > >> > > >> > Just to add my two cents. > >> > > >> > Upgrading to JDK11 looks inevitable sooner or later and I would > definitely > >> > not wait until 2030 or 2026 when the extended support of JDK8 ends. > >> > But on the other side I have to agree with Enrico and Patrick that > far too > >> > many users are tied to JDK8 yet (not because they want to use JDK8, > but > >> > because they have to), some of them are components of the Hadoop > ecosystem, > >> > which would be a loss to tie them to 3.6.x for years. > >> > Do we know the state of Hadoop? It builds with JDK8 at the moment, > but do > >> > we know what are their plans to go to JDK11? > >> > When they move, we should move too, but I don't think it would be > wise to > >> > do it earlier. > >> > > >> > Christopher's option looks like the golden path, but it needs some > >> > investment on the testing side as Enrico pointed it out. > >> > Could we agree on it as a compromise? > >> > > >> > Regards, Tamaas > >> > > >> > On Wed, Oct 21, 2020 at 7:11 PM Brent <brentwritesc...@gmail.com> > wrote: > >> > > >> >> I think I was reacting to Enrico's earlier comment of: > >> >> > >> >> " ZooKeeper client is used by tons of users and unfortunately many > >> >> projects > >> >> are still on JDK8, if we move ZooKeeper to JDK11 the risk is to block > >> >> users > >> >> from the adoption, that is that we will see the world to stay on > 3.6.x and > >> >> we will have again a long lived release line, like 3.4." > >> >> > >> >> It's a matter of whether or not a long-lived release line is > >> >> desirable/undesirable. If everyone is OK keeping 3.6.x up-to-date > >> >> security/patch-wise (if not feature-wise) for the next N years, then > >> >> that's > >> >> a potentially valid approach. I interpreted that comment as "a > long-lived > >> >> release line is undesirable", but no one explicitly said that, I > just read > >> >> it that way. > >> >> > >> >> ~Brent > >> >> > >> >> On Wed, Oct 21, 2020 at 9:49 AM Patrick Hunt <ph...@apache.org> > wrote: > >> >> > >> >> > On Wed, Oct 21, 2020 at 9:03 AM Andor Molnar <an...@apache.org> > wrote: > >> >> > > >> >> > > As far as I know Hbase, Solr and Kafka are already Java 11 ready. > >> >> > > > >> >> > > IMHO contributors of those projects should also put efforts into > >> >> moving > >> >> > > forward. > >> >> > > > >> >> > > We’re not saying that you _have_ to move to Java 11. > >> >> > > Staying on Java 8? No problem, 3.6 is for you. > >> >> > > Want the fancy new features of 3.7? Work on it on your side too. > >> >> > > > >> >> > > > >> >> > ppl want things like security fixes. I believe the highlighted > downside > >> >> is > >> >> > that we would need to continue to maintain 3.6.x rather than > allowing > >> >> > users, and ourselves, to focus on trunk for production -"64 > percent" > >> >> would > >> >> > be blocked. > >> >> > > >> >> > Patrick > >> >> > > >> >> > > >> >> > > Andor > >> >> > > > >> >> > > > >> >> > > > >> >> > > > On 2020. Oct 21., at 17:52, Enrico Olivelli < > eolive...@gmail.com> > >> >> > wrote: > >> >> > > > > >> >> > > > Il giorno mer 21 ott 2020 alle ore 17:49 Andor Molnar < > >> >> > an...@apache.org> > >> >> > > ha > >> >> > > > scritto: > >> >> > > > > >> >> > > >> Correct me if I'm wrong, but Oracle gets paid for extended > support. > >> >> > > >> Java 8 support until 2030 is not free of charge. > >> >> > > >> > >> >> > > >> "ZK may end up with a lot of users potentially locking > themselves > >> >> to > >> >> > > >> 3.6.x for a while as Enrico mentioned." > >> >> > > >> > >> >> > > >> That's true. What's the downside of that which we should > invest in > >> >> to > >> >> > > >> avoid? > >> >> > > >> > >> >> > > > > >> >> > > > I see ZooKeeper used in many many projects, all of the > >> >> > > > HBase/Pulsar/Kafka/Solr ecosystem... > >> >> > > > they will have to move to JDK11 in order to move to the new ZK > >> >> version > >> >> > > > so probably they will stay on ZK 3.6 > >> >> > > > > >> >> > > > Probably with Java 17 LTS released the cards will change on the > >> >> table > >> >> > > > > >> >> > > > Enrico > >> >> > > > > >> >> > > > > >> >> > > > > >> >> > > >> > >> >> > > >> Andor > >> >> > > >> > >> >> > > >> > >> >> > > >> > >> >> > > >> On Wed, 2020-10-21 at 08:03 -0700, Brent wrote: > >> >> > > >>> As a slightly different consideration, if you look at the > >> >> Long-Term > >> >> > > >>> Support > >> >> > > >>> (LTS) roadmaps for Java, currently Java 8 is set to have full > >> >> support > >> >> > > >>> until > >> >> > > >>> 2030 from Oracle and at least 2026 from OpenJDK & Corretto: > >> >> > > >>> > >> >> > > >>> > >> >> > > https://www.oracle.com/java/technologies/java-se-support-roadmap.html > >> >> > > >>> https://en.wikipedia.org/wiki/Java_version_history > >> >> > > >>> > >> >> > > >>> My guess is that a number of companies are still heavily > invested > >> >> at > >> >> > > >>> the > >> >> > > >>> Java 8 level (I know a few) and with that kind of time > horizon, > >> >> they > >> >> > > >>> have > >> >> > > >>> no real motivation to upgrade for quite a while. If the > recent > >> >> > > >>> Python 2 > >> >> > > >>> deprecation is anything to go by, they won't do it until > they have > >> >> > > >>> to. > >> >> > > >>> > >> >> > > >>> Not saying Java 8 isn't *very* old (2014 release it seems > like?) > >> >> and > >> >> > > >>> I'm > >> >> > > >>> not invested heavily either way, but this might suggest that > ZK > >> >> may > >> >> > > >>> end up > >> >> > > >>> with a lot of users potentially locking themselves to 3.6.x > for a > >> >> > > >>> while as > >> >> > > >>> Enrico mentioned. > >> >> > > >>> > >> >> > > >>> (Not a major contributor, but wanted to chime in since I > just had > >> >> > > >>> this > >> >> > > >>> conversation with a bunch of people professionally recently) > >> >> > > >>> > >> >> > > >>> Brent > >> >> > > >>> > >> >> > > >>> On Wed, Oct 21, 2020 at 2:07 AM Andor Molnar < > an...@apache.org> > >> >> > > >>> wrote: > >> >> > > >>> > >> >> > > >>>> Thanks for the summary. > >> >> > > >>>> > >> >> > > >>>> I still vote for option 1). Move 3.7.0 to JDK 11 fully. > Other > >> >> > > >>>> projects > >> >> > > >>>> will upgrade once they’re JDK11 compliant, otherwise they > will > >> >> stay > >> >> > > >>>> on 3.5 > >> >> > > >>>> or 3.6. Both version are quite recent in ZooKeeper-terms, we > >> >> > > >>>> already > >> >> > > >>>> planned big changes for 3.7.0 and JDK 11 could be one of > them. > >> >> > > >>>> > >> >> > > >>>> Don’t put extra burden on the ZK community to help others > staying > >> >> > > >>>> on > >> >> > > >>>> ancient Java versions. > >> >> > > >>>> > >> >> > > >>>> Andor > >> >> > > >>>> > >> >> > > >>>> > >> >> > > >>>> > >> >> > > >>>>> On 2020. Oct 21., at 10:57, Enrico Olivelli < > >> >> eolive...@gmail.com> > >> >> > > >>>>> wrote: > >> >> > > >>>>> > >> >> > > >>>>> Let me recap > >> >> > > >>>>> - Christopher is proposing to move to JDK11 > >> >> > > >>>>> - ZooKeeper client and server are bundled and coded and > tested > >> >> > > >>>>> together > >> >> > > >>>> in > >> >> > > >>>>> zookeeper-server > >> >> > > >>>>> - Enrico is concerned about the need of testing ZooKeeper > client > >> >> > > >>>>> on JDK8 > >> >> > > >>>>> (not a problem to move the server to JDK11) > >> >> > > >>>>> > >> >> > > >>>>> ZooKeeper client is used by tons of users and > unfortunately many > >> >> > > >>>>> projects > >> >> > > >>>>> are still on JDK8, if we move ZooKeeper to JDK11 the risk > is to > >> >> > > >>>>> block > >> >> > > >>>> users > >> >> > > >>>>> from the adoption, > >> >> > > >>>>> that is that we will see the world to stay on 3.6.x and we > will > >> >> > > >>>>> have > >> >> > > >>>> again > >> >> > > >>>>> a long lived release line, like 3.4. > >> >> > > >>>>> > >> >> > > >>>>> Testing the client on JDK8 would be possible if we create > some > >> >> > > >>>>> kind of > >> >> > > >>>>> additional module with system tests, then we can start the > >> >> server > >> >> > > >>>>> on > >> >> > > >>>> docker > >> >> > > >>>>> on JDK11+ and start a client on JDK8 > >> >> > > >>>>> with Maven toolchain it should possible to run surefire > tests > >> >> > > >>>>> using a > >> >> > > >>>>> separate JVM. > >> >> > > >>>>> > >> >> > > >>>>> So in my vision 2 options: > >> >> > > >>>>> 1) fully JDK11 - drop JDK8 at all > >> >> > > >>>>> 2) build with JDK11 - server only on JDK11 - add system > tests > >> >> > > >>>>> with docker > >> >> > > >>>>> and toolchains that ensure the ZooKeeper client (and all > >> >> > > >>>>> dependencies) > >> >> > > >>>>> still work on JDK8 > >> >> > > >>>>> > >> >> > > >>>>> From my point of view about the ZooKeeper ecosystem option > 2) > >> >> > > >>>>> will be far > >> >> > > >>>>> better, but we need resources to work on a new test suite. > >> >> > > >>>>> > >> >> > > >>>>> Enrico > >> >> > > >>>>> > >> >> > > >>>>> > >> >> > > >>>>> Il giorno mer 21 ott 2020 alle ore 10:43 Andor Molnar < > >> >> > > >>>>> an...@apache.org> > >> >> > > >>>> ha > >> >> > > >>>>> scritto: > >> >> > > >>>>> > >> >> > > >>>>>> Tamas, Enrico, > >> >> > > >>>>>> > >> >> > > >>>>>> Sorry I don’t follow. Why do we have to test the client > with > >> >> > > >>>>>> JDK 8 in > >> >> > > >>>>>> version 3.7.0? > >> >> > > >>>>>> > >> >> > > >>>>>> Andor > >> >> > > >>>>>> > >> >> > > >>>>>> > >> >> > > >>>>>> > >> >> > > >>>>>> > >> >> > > >>>>>>> On 2020. Oct 20., at 22:29, Tamas Penzes < > >> >> > > >>>>>>> tam...@cloudera.com.INVALID> > >> >> > > >>>>>> wrote: > >> >> > > >>>>>>> Hi Enrico, > >> >> > > >>>>>>> > >> >> > > >>>>>>> Separating ZooKeeper client and server is a huge work, > but we > >> >> > > >>>>>>> might not > >> >> > > >>>>>>> need it. > >> >> > > >>>>>>> As you mentioned we have to test ZK client with Java 8, > what > >> >> > > >>>>>>> about > >> >> > > >>>>>>> separating only the test cases which we need to run with > >> >> > > >>>>>>> Java8 too? > >> >> > > >>>>>>> In Curator we have the ZK compatibility tests where we > run a > >> >> > > >>>>>>> limited > >> >> > > >>>>>> amount > >> >> > > >>>>>>> of Curator's jUnit tests with a different ZK version. > >> >> > > >>>>>>> We might be able to do the same here, tag tests which are > >> >> > > >>>>>>> testing ZK > >> >> > > >>>>>> client > >> >> > > >>>>>>> and run them separately with Java 8. The only limitation > is > >> >> > > >>>>>>> that these > >> >> > > >>>>>>> tests must stay JDK8 compatible. > >> >> > > >>>>>>> But from the tags we will see which ones are those. > >> >> > > >>>>>>> > >> >> > > >>>>>>> What do you think? > >> >> > > >>>>>>> > >> >> > > >>>>>>> Regards, Tamaas > >> >> > > >>>>>>> > >> >> > > >>>>>>> On Sat, Oct 17, 2020 at 7:45 AM Enrico Olivelli < > >> >> > > >>>>>>> eolive...@gmail.com> > >> >> > > >>>>>> wrote: > >> >> > > >>>>>>>> Christopher > >> >> > > >>>>>>>> I appreciate your idea and I also moved lots of my > projects > >> >> > > >>>>>>>> to work > >> >> > > >>>> the > >> >> > > >>>>>> way > >> >> > > >>>>>>>> you are suggesting. > >> >> > > >>>>>>>> > >> >> > > >>>>>>>> > >> >> > > >>>>>>>> We must run tests using real jdk8 to test the Zookeeper > >> >> > > >>>>>>>> client. We > >> >> > > >>>> must > >> >> > > >>>>>>>> ensure that Zookeeper works well, especially while > dealing > >> >> > > >>>>>>>> with > >> >> > > >>>> security > >> >> > > >>>>>>>> stuff. > >> >> > > >>>>>>>> Currently the client is in the same module of the server > >> >> > > >>>>>>>> and it will > >> >> > > >>>>>> take a > >> >> > > >>>>>>>> good (huge) amount of work to separate them > >> >> > > >>>>>>>> > >> >> > > >>>>>>>> > >> >> > > >>>>>>>> Enrico > >> >> > > >>>>>>>> > >> >> > > >>>>>>>> Il Ven 16 Ott 2020, 23:25 Christopher < > ctubb...@apache.org> > >> >> > > >>>>>>>> ha > >> >> > > >>>> scritto: > >> >> > > >>>>>>>>> Hi ZK Devs, > >> >> > > >>>>>>>>> > >> >> > > >>>>>>>>> With recent advancements in Java (since Java 9), it is > >> >> > > >>>>>>>>> now generally > >> >> > > >>>>>>>>> no longer necessary to require that software be > developed > >> >> > > >>>>>>>>> on an older > >> >> > > >>>>>>>>> JDK in order to have confidence that it will run on the > >> >> > > >>>>>>>>> older version > >> >> > > >>>>>>>>> of Java. This is because, as of Java 9, all JDK > releases > >> >> > > >>>>>>>>> have better > >> >> > > >>>>>>>>> support for cross-compilation to older Java versions. > >> >> > > >>>>>>>>> > >> >> > > >>>>>>>>> What this means is that developers can confidently make > >> >> > > >>>>>>>>> the build > >> >> > > >>>>>>>>> requirements for a project higher than the Java version > >> >> > > >>>>>>>>> that will > >> >> > > >>>>>>>>> actually be supported at runtime. > >> >> > > >>>>>>>>> > >> >> > > >>>>>>>>> In fact, ZooKeeper already supports the necessary flags > >> >> > > >>>>>>>>> in its Maven > >> >> > > >>>>>>>>> build configuration to ensure that it uses JDK 8 > >> >> > > >>>>>>>>> compliance when > >> >> > > >>>>>>>>> building on a newer JDK (I added this way back in > >> >> > > >>>>>>>>> ZOOKEEPER-3739 / > >> >> > > >>>>>>>>> https://github.com/apache/zookeeper/pull/1269) > >> >> > > >>>>>>>>> > >> >> > > >>>>>>>>> So, I propose that we make JDK 11 the new minimum > version > >> >> > > >>>>>>>>> to *build* > >> >> > > >>>>>>>>> ZooKeeper with. This would not change the runtime > >> >> > > >>>>>>>>> requirement, which > >> >> > > >>>>>>>>> would remain at JDK 8. > >> >> > > >>>>>>>>> > >> >> > > >>>>>>>>> The only necessary change to make this happen would be > to > >> >> > > >>>>>>>>> add the > >> >> > > >>>>>>>>> minimum Java version to the maven-enforcer-plugin (like > >> >> > > >>>>>>>>> > >> >> > > >>>>>>>>> > >> >> > > >>>> > >> >> > > >> > >> >> > > > >> >> > > >> >> > > > https://github.com/apache/accumulo/blob/438f0efd34ef9d200bc8c7ecdd11d5dedb146519/pom.xml#L1162-L1164 > >> >> > > >>>>>>>>> ) > >> >> > > >>>>>>>>> > >> >> > > >>>>>>>>> This would allow ZooKeeper to to streamline its > >> >> > > >>>>>>>>> development process a > >> >> > > >>>>>>>>> little bit by reducing the amount of CI testing that is > >> >> > > >>>>>>>>> done as part > >> >> > > >>>>>>>>> of the build. In other words, we can drop the CI builds > >> >> > > >>>>>>>>> for JDK 8, > >> >> > > >>>>>>>>> which saves on build resources and time. The return on > >> >> > > >>>>>>>>> investment is > >> >> > > >>>>>>>>> so low for the JDK 8 builds anyway, because of the > >> >> > > >>>>>>>>> improved > >> >> > > >>>>>>>>> cross-compilation in newer JDKs. So, there's not much > >> >> > > >>>>>>>>> value in > >> >> > > >>>>>>>>> building on JDK 8 anyway. > >> >> > > >>>>>>>>> > >> >> > > >>>>>>>>> Of course, I am only recommending this for *new* > release > >> >> > > >>>>>>>>> lines, > >> >> > > >>>>>>>>> starting with ZooKeeper 3.7.0/master branch, because I > >> >> > > >>>>>>>>> would not want > >> >> > > >>>>>>>>> to change expectations for users who will build their > own > >> >> > > >>>>>>>>> 3.5 and 3.6 > >> >> > > >>>>>>>>> versions as they continue to have patch versions > >> >> > > >>>>>>>>> released. > >> >> > > >>>>>>>>> > >> >> > > >>>>>>>>> What do you think? > >> >> > > >>>>>>>>> > >> >> > > >>>>>>>>> Kind Regards, > >> >> > > >>>>>>>>> Christopher > >> >> > > >>>>>>>>> > >> >> > > >> > >> >> > > >> > >> >> > > > >> >> > > > >> >> > > >> >> > >> > >
