I'd expect the automated backporting process to only work for fairly trivial patches which do not use protobuf, etc. More involved patches would need manual work anyway.
If we want to make sure that everything compiles with JDK8, it's easier to just compile the master branch with JDK8 (along with 11/17), and fail the CI check if it doesn't. We need to find a balance between using the new Java features and keeping the workload manageable. We could keep compiling master with JDK8 for a year or two, and when activity on the 2.x branches tapers off, we could remove that restriction. On Tue, May 7, 2024 at 3:56 PM Andrew Purtell <andrew.purt...@gmail.com> wrote: > I also like the suggestion to have CI help us here too. > > > On May 7, 2024, at 9:42 AM, Bryan Beaudreault <bbeaudrea...@apache.org> > wrote: > > > > I'm nervous about creating more big long-term divergences between the > > branches. Already I sometimes get caught up on HBaseTestingUtil vs > > HBaseTestingUtility. And we all know the burden of maintaining the old > > HTable impl. > > > > I'm not sure if this is a useful suggestion since it would require > someone > > to do a good deal of work, but I wonder if we could automate backport > > testing a bit. Our yetus checks already check the patch, maybe it could > > apply the patch to branch-2. This would increase the cost of master > branch > > PRs but maybe speed us up overall. > > > >> On Tue, May 7, 2024 at 9:21 AM 张铎(Duo Zhang) <palomino...@gmail.com> > wrote: > >> > >> The problem is that, if we only compile and run tests on JDK11+, the > >> contributors may implicitly use some JDK11+ only features and > >> introduce difference when backporting to branch-2.x. > >> > >> Maybe a possible policy is that, once a patch should go into > >> branch-2.x too, before mering the master PR, we should make sure the > >> contributor open a PR for branch-2.x too, so we can catch the > >> differences between the 2 PRs, and whether to align them. > >> > >> WDYT? > >> > >> Thanks. > >> > >> Andrew Purtell <apurt...@apache.org> 于2024年5月7日周二 20:20写道： > >>> > >>> I don’t expect 2.x to wind down for up to several more years. We will > be > >>> still using it in production at my employer for a long time and I would > >>> continue my role as RM for 2.x as needed. HBase 3 is great but not GA > yet > >>> and then some users will want to wait one to a couple years before > >> adopting > >>> the new major version, especially if migration is not seamless. (We > even > >>> faced breaking changes in a minor upgrade from 2.4 to 2.5 that brought > >> down > >>> a cluster during a rolling upgrade, so there should be no expectation > of > >> a > >>> seamless upgrade.) My plan is to continue releasing 2.x until, like > with > >>> 1.x, the commits to branch-2 essentially stop, or until the PMC stops > >>> allowing release of the candidates. > >>> > >>> Perhaps we do not need to do a total ban on use of 11 features. We > should > >>> allow a case by case discussion. We can minimize their scope and even > >>> potentially offer multiversion support like we do with Unsafe access > >>> utility classes in hbase-thirdparty. There are no planned uses of new > 11+ > >>> APIs and features now anyhow. > >>> > >>> > >>> On Tue, May 7, 2024 at 7:40 AM 张铎(Duo Zhang) <palomino...@gmail.com> > >> wrote: > >>> > >>>> For me I think Istvan's plan is also acceptable. > >>>> > >>>> So in conclusion, we should > >>>> > >>>> 1. Jump to JDK11/JDK17(we could start a new thread to discuss this, > >>>> maybe also on the user mailing list) > >>>> 2. Claim and also make sure 3.x does not work with JDK8 > >>>> 3. Introduce a policy to only allow JDK8 features on master and > >>>> branch-3.x for a while(maybe still keep the release version as 8?) > >>>> > >>>> Any other suggestions? > >>>> > >>>> Thanks. > >>>> > >>>> Istvan Toth <st...@cloudera.com.invalid> 于2024年4月30日周二 12:45写道： > >>>>> > >>>>> Spring is a good argument for JDK17. > >>>>> > >>>>> Duo's suggestion is a great step forward, firmly stating that JDK8 > >> is not > >>>>> officially supported solves most of our expected future CVE problems. > >>>>> > >>>>> However, I think that ripping off the bandaid, and making sure that > >>>> HBase 3 > >>>>> does not work with Java 8 would be better. > >>>>> It's easier to accept such a change in a major version than in a > >> minor > >>>>> version. > >>>>> > >>>>> IMO users that are so conservative that they are still using Java 8 > >> are > >>>>> unlikely to be first movers to a new major release anyway. > >>>>> > >>>>> I think that the following upgrade path would optimal: > >>>>> > >>>>> - User stays on (supported) Hbase 2.x until ready to upgrade Java > >>>>> - User upgrades to Java 11/17 with the same HBase > >>>>> - User upgrades to Hbase 3.x > >>>>> > >>>>> As noted, we will need to support 2.x for some time anyway (just > >> like 1.x > >>>>> was supported for a long time). > >>>>> > >>>>> As for the backporting issues: > >>>>> We could make it a policy to avoid using Java 11+ features in Hbase > >> code > >>>>> until 2.x supports winds down. > >>>>> This has worked quite well for Phoenix with Java 7 / Java 8. > >>>>> > >>>>> > >>>>> > >>>>> > >>>>> > >>>>> > >>>>> > >>>>> > >>>>> > >>>>> On Tue, Apr 30, 2024 at 3:59 AM 张铎(Duo Zhang) <palomino...@gmail.com > >>> > >>>> wrote: > >>>>> > >>>>>> AFAIK spring 6 and spring-boot 3 have jumped to java17 directly, > >> so if > >>>> we > >>>>>> want to upgrade, I also suggest that we jump to java 17 directly. > >>>>>> > >>>>>> While upgrading to java 17 can reduce our compatibility work on > >>>> branch-3+, > >>>>>> but consider the widely usage for java 8, I think we still need to > >>>> support > >>>>>> branch-2 for several years, then this will increase the > >> compatibility > >>>> work > >>>>>> as the code between branch-3+ and branch-2.x will be more and more > >>>>>> different. > >>>>>> > >>>>>> So for me, a workable solution is > >>>>>> > >>>>>> 1. We first claim that branch-3+ will move minimum java support to > >> 11 > >>>> or > >>>>>> 17. > >>>>>> 2. Start to move the compilation to java 11 or 17, but still keep > >>>> release > >>>>>> version 8, and still keep the pre commit pipeline to run java 8, > >> 11, > >>>> 17, to > >>>>>> minimum our compatibility work before we have the first 3.0.0 > >> release. > >>>>>> 3. Cut branch-3.0 and release 3.0.0, so we have a 3.0.0 release, > >>>> actually > >>>>>> which can still run on java 8, so it will be easier for our users > >> to > >>>>>> upgrade to 3.x and reduce our pressure on maintaining branch-2, > >>>> especially > >>>>>> do not need to back port new features there. > >>>>>> 4. Start to move the release version to 11 or 17 on branch-3+, and > >>>> prepare > >>>>>> for 3.1.0 release, which will be the real 11 or 17 only release. > >>>>>> > >>>>>> Thanks. > >>>>>> > >>>>>> Bryan Beaudreault <bbeaudrea...@apache.org>于2024年4月30日 周二02:54写道： > >>>>>> > >>>>>>> I am a huge +1 for dropping java8. > >>>>>>> > >>>>>>> One reason I would suggest going to 17 is that it seems so hard > >> to > >>>> change > >>>>>>> these things given our long development cycle on major releases. > >>>> There > >>>>>> are > >>>>>>> some nice language features in 17, but more importantly is that > >> the > >>>>>> initial > >>>>>>> release of java11 was released 6 years ago and java17 released 3 > >>>> years. > >>>>>>> Java21 is already released as well. So I could see java17 being > >>>> widely > >>>>>>> available enough that we could jump "in the middle" rather than > >> to > >>>> the > >>>>>>> oldest LTS. > >>>>>>> > >>>>>>> I will say that we're already running java 21 on all of our > >>>> hbase/hadoop > >>>>>> in > >>>>>>> prod (70 clusters, 7k regionservers). I know not every > >> organization > >>>> can > >>>>>> be > >>>>>>> that aggressive, and I wouldn't suggest jumping to 21 in the > >>>> codebase. > >>>>>> Just > >>>>>>> pointing it out in terms of basic support already existing and > >> being > >>>>>>> stable. > >>>>>>> > >>>>>>> On Mon, Apr 29, 2024 at 2:33 PM Andrew Purtell < > >>>> andrew.purt...@gmail.com > >>>>>>> > >>>>>>> wrote: > >>>>>>> > >>>>>>>> I also agree that mitigation of security problems in > >> dependencies > >>>> will > >>>>>> be > >>>>>>>> increasingly difficult, as we cannot expect our dependencies to > >>>>>> continue > >>>>>>> to > >>>>>>>> support Java 8. They might, but as time goes on it is less > >> likely. > >>>>>>>> > >>>>>>>> A minimum of Java 11 makes a lot of sense. This is where the > >>>> center of > >>>>>>>> gravity of the Java ecosystem is, probably. > >>>>>>>> > >>>>>>>> A minimum of 17 is aggressive and I don’t see the point unless > >>>> there > >>>>>> is a > >>>>>>>> feature in 17 that we would like to base an improvement on. > >>>>>>>> > >>>>>>>>> On Apr 29, 2024, at 1:23 PM, chrajeshbab...@gmail.com wrote: > >>>>>>>>> > >>>>>>>>> Hi! > >>>>>>>>> > >>>>>>>>> With 3.0 on the horizon, we could look into bumping the > >> minimum > >>>>>>> required > >>>>>>>>> Java version for HBase. > >>>>>>>>> > >>>>>>>>> The last discussion I could find was four years ago, when > >>>> dropping > >>>>>> 8.0 > >>>>>>>>> support was rejected. > >>>>>>>>> > >>>>>>>>> > >> https://lists.apache.org/thread/ph8xry0x37cvjj89fp2jk1k48yb7gs46 > >>>>>>>>> > >>>>>>>>> Now it's four years later, and the end of OpenJDK support for > >>>> Java 8 > >>>>>>> and > >>>>>>>> 11 > >>>>>>>>> are much closer. > >>>>>>>>> (Oracle public support is so short that I consider that > >>>> irrelevant) > >>>>>>>>> > >>>>>>>>> Some critical dependencies (like Jetty) have ended even > >> regular > >>>>>>> security > >>>>>>>>> support for Java 8. > >>>>>>>>> > >>>>>>>>> By supporting Java 8 we are alse limiting ourselves to using > >> an > >>>>>> already > >>>>>>>> 10 > >>>>>>>>> year old Java release, ignoring any developments in the > >> language. > >>>>>>>>> > >>>>>>>>> My take is that with the current dogmatic emphasis on CVE > >>>> mitigation > >>>>>>> the > >>>>>>>>> benefits of bumping the required JDK version outweigh the > >>>> benefits > >>>>>> even > >>>>>>>> for > >>>>>>>>> the legacy install base, especially it's getting harder and > >>>> harder to > >>>>>>> be > >>>>>>>>> CVE free with Java 8. > >>>>>>>>> > >>>>>>>>> Furthermore, with RedHat dropping JDK11 support this year, I > >>>> think we > >>>>>>>> could > >>>>>>>>> also consider bumping the minimum requirement straight to > >> JDK 17. > >>>>>>>>> > >>>>>>>>> Hadoop is still on Java 8, but previously it has dropped > >> Java 7 > >>>>>> support > >>>>>>>> in > >>>>>>>>> a patch release, and I wouldn't be surprised if it dropped > >> Java > >>>> 8 in > >>>>>> a > >>>>>>>>> similar manner, so I would not put too much stock in that. > >>>>>>>>> > >>>>>>>>> What do you think ? > >>>>>>>>> > >>>>>>>>> Thanks, > >>>>>>>>> Rajeshbabu. > >>>>>>>> > >>>>>>> > >>>>>> > >>>>> > >>>>> > >>>>> -- > >>>>> *István Tóth* | Sr. Staff Software Engineer > >>>>> *Email*: st...@cloudera.com > >>>>> cloudera.com <https://www.cloudera.com> > >>>>> [image: Cloudera] <https://www.cloudera.com/> > >>>>> [image: Cloudera on Twitter] <https://twitter.com/cloudera> [image: > >>>>> Cloudera on Facebook] <https://www.facebook.com/cloudera> [image: > >>>> Cloudera > >>>>> on LinkedIn] <https://www.linkedin.com/company/cloudera> > >>>>> ------------------------------ > >>>>> ------------------------------ > >>>> > >> > -- *István Tóth* | Sr. Staff Software Engineer *Email*: st...@cloudera.com cloudera.com <https://www.cloudera.com> [image: Cloudera] <https://www.cloudera.com/> [image: Cloudera on Twitter] <https://twitter.com/cloudera> [image: Cloudera on Facebook] <https://www.facebook.com/cloudera> [image: Cloudera on LinkedIn] <https://www.linkedin.com/company/cloudera> ------------------------------ ------------------------------
