Hi 

I have created an umbrella ticket for Migrate from Jetty 9 to Jetty 12 at 
HBASE-29224

Also I have submitted PRs for first 2 tasks of phase one and are ready for 
review:
* HBASE-29225 Add module for Jetty 12 with EE8 to hbase-thirdparty
* HBASE-29226 Migrate to jetty 12 with EE8 and bump java servlet to 4.0.1

Have fixed most of the UTs locally. May need more testing: try SSL, and other 
stuff; Will do as part of sub-task HBASE-29227.

Also could someone please help/guide me with pushing the snapshot thirdparty 
changes to artifactory to test out the main code and trigger UT without merging?

> For step 2 could we duplicate AuthenticationFilter ?

We could do that but that would lead us to copy a lot of files. Alternately we 
could temporarily create a shaded hadoop jar with relocated import packages? 
Will need to see what is best once we get there.

> Are the Jetty EE modules exclusive for the same namespace ?

Jetty 12 with EE8 is the only module which supports javax.
Jetty 12 with EE9/EE10 strictly required jakarta. Please refer 
https://stackoverflow.com/a/77010707/3778833 for exact matrix.

Regards,
Nihal

On 2025/03/12 06:57:40 Istvan Toth wrote:
> Sounds good.
> 
> The 2.x backport still depends on bumping the minimum Java requirement.
> For step 2 could we duplicate AuthenticationFilter ?
> Are the Jetty EE modules exclusive for the same namespace ?
> 
> Istvan
> 
> On Tue, Mar 11, 2025 at 9:36 PM Nihal Jain <nihalj...@apache.org> wrote:
> 
> > Hi all,
> >
> > Sorry for delay in responding. I have been spending some time trying to
> > get a feel of what all we might need to do by changing code (roughly),
> > hence thought to reply once I have something substantial ready.
> >
> > > If we could target EE 9 instead of EE 8 that would align with some
> > internal
> > > requirements at $dayjob. For what it's worth. Mostly the point of EE 9
> > is a
> > > move of the EE stuff into a new namespace, allowing for further
> > development
> > > under the Eclipse Foundation without trademark conflicts with Oracle's
> > > "Java" branding.
> >
> > Sure we can aim for EE9 + jakarta.
> >
> > > I'd split that into two tickets.
> > >
> > > First move to Jetty 12 while keeping the old javax package names, and
> > once
> > > that's done and tested, we can look into moving to jakarta.
> >
> > But I think we should do this in 2 steps as mentioned by Istvan.
> >
> > As per me we should break the migration into below phases:
> >
> > Phase 1:
> > * Add module for Jetty 12 with EE8 to hbase-thirdparty
> > * Next consume this version of hbase-thirdparty, move to jetty 12 with EE8
> > and bump java servlet to 4.0.1
> > * Test and verify everything is working as expected.
> > * NOTE: We could also consider this as the fix we backport to branch-2 as
> > it seemingly does not change code/compat much.
> >
> > Phase 2:
> > * Add Jetty 12 with EE9 to hbase-thirdparty and jersey 3. And may be some
> > other artifacts (not sure at this point)
> > * Next consume this version of hbase-thirdparty, move to jetty 12 with
> > EE9, bump jakarta servlet to 5.x / 6.x, tomcat to 10.x / 11.x and migrate
> > all the dependencies and code to jakarta namespace
> > * Blockers?? Hadoop AuthenticationFilter dependent and related code need
> > to be either shaded to move from javax to jakarta, or we would need to wait
> > for hadoop for move to jakarta. (In my rough analysis, I have identified
> > this till now, when we attempt it might be more stuff)
> > * Test and verify everything is working as expected.
> >
> > Update on phase 1: I have posted a draft working set in github. Have done
> > basic sanity of hbase, rest and thrift, looks good at high level. May need
> > more testing: try SSL, and other stuff. Have not tried UTs and other things
> > yet.
> >
> > See draft PRs at:
> > * https://github.com/apache/hbase-thirdparty/pull/131
> > * https://github.com/apache/hbase/pull/6783
> >
> > Regards,
> > Nihal
> >
> >
> > References:
> > *
> > https://stackoverflow.com/questions/77007560/missing-jetty-servlet-12-0-0-dependency
> > * https://jetty.org/download.html#version-history
> > * https://tomcat.apache.org/whichversion.html
> > *
> > https://tomcat.apache.org/migration-10.html#Migrating_from_9.0.x_to_10.0.x
> > *
> > https://github.com/apache/hadoop/blob/branch-3.4.1/hadoop-common-project/hadoop-auth/src/main/java/org/apache/hadoop/security/authentication/server/AuthenticationFilter.java
> >
> > On 2025/03/07 17:13:41 Andrew Purtell wrote:
> > > I was not aware that Jetty 12 requires Java 17 but should have checked.
> > At some point we will need to concede to raising our minimum Java version
> > when a combination of security issues and dependency maintenance status
> > require the bump to solve for the constraints. We can try to get ahead of
> > it too. I think it makes sense.
> > >
> > > Let me look at PHOENIX-7164!
> > >
> > > > On Mar 7, 2025, at 7:35 AM, Istvan Toth <st...@cloudera.com.invalid>
> > wrote:
> > > >
> > > > The only way to get Jetty 17 into 2.x would be by bumping the
> > required Java
> > > > version for 2.7 or 2.8 to 17.
> > > > This is not necessarily out of the question, but we haven't discussed
> > that
> > > > yet.
> > > >
> > > > The main reason I pushed for bumping the minimum Java to 17 for 3.0
> > was so
> > > > that we can move off all the old dependencies we are stuck on.
> > > >
> > > > BTW I heard noises (well, read it on ticket) that Hadoop 3.5 is also
> > > > planned to require JDK17.
> > > >
> > > > Aside:
> > > > Andrew, there are Phoenix PRs waiting for reviews which add the bulk of
> > > > changes required for HBase 3.0 support. (Mostly just getting rid of
> > HBase
> > > > 1.0 APIs)
> > > > I'd appreciate it if someone from your team could review them. See
> > > > PHOENIX-7164.  I'd like to add support for HBase 3.0
> > > > to Phoenix promptly after 3.0 is released.
> > > >
> > > > Istvan
> > > >
> > > >
> > > >> On Fri, Mar 7, 2025 at 2:56 AM Andrew Purtell <apurt...@apache.org>
> > wrote:
> > > >>
> > > >> I'm not sure what other user appetites for HBase 3 in production are,
> > but
> > > >> at the $dayjob it's got to be 18 months to 2 years out. First, we are
> > a
> > > >> Phoenix shop, so Phoenix must adapt their code for HBase 3 and prepare
> > > >> releases based on HBase 3 that will be production ready, and as far
> > as I
> > > >> know this isn't on the radar for them yet. (It's quite possible our
> > shop
> > > >> would do the work, but not until next year at the earliest.) Then, we
> > must
> > > >> update all of our internal code for API differences, and rigorously
> > test
> > > >> incremental upgrade and rollback, until we can move up and back
> > seamlessly,
> > > >> and that will take time. I think we might consider allocating
> > resources for
> > > >> it beginning next year. I don't think we are unusual in that regard.
> > > >>
> > > >> All this to say, maintenance changes like a Jetty uplift should not be
> > > >> restricted to 3.x only, because 2.x users are going to remain on 2.x
> > for a
> > > >> while, and 2.x users have just as much exposure to security blockers
> > > >> related to end of support versions of Jetty. If the project doesn't
> > land
> > > >> things like Jetty uplift back to 2.x, that's just more work for us to
> > do
> > > >> it, and less bandwidth to pursue more forward looking activities.
> > > >>
> > > >> For your consideration.
> > > >>
> > > >> On Thu, Mar 6, 2025 at 5:45 PM 张铎(Duo Zhang) <palomino...@gmail.com>
> > > >> wrote:
> > > >>
> > > >>> If we can speed up the 3.x release, since it has already moved up the
> > > >>> JDK17+, it will be easier to land these changes in 3.x only.
> > > >>>
> > > >>> Istvan Toth <st...@cloudera.com.invalid> 于2025年3月7日周五 03:42写道:
> > > >>>>
> > > >>>> I'd split that into two tickets.
> > > >>>>
> > > >>>> First move to Jetty 12 while keeping the old javax package names,
> > and
> > > >>> once
> > > >>>> that's done and tested, we can look into moving to jakarta.
> > > >>>>
> > > >>>> Istvan
> > > >>>>
> > > >>>> On Thu, Mar 6, 2025 at 8:33 PM Andrew Purtell <apurt...@apache.org>
> > > >>> wrote:
> > > >>>>
> > > >>>>>> Therefore, I plan to use the EE8 environment on Jetty 12.
> > > >>>>>
> > > >>>>> Ah, thanks for mentioning EE.
> > > >>>>>
> > > >>>>> Do you have a requirement to stay with EE 8 ?
> > > >>>>>
> > > >>>>> If we could target EE 9 instead of EE 8 that would align with some
> > > >>> internal
> > > >>>>> requirements at $dayjob. For what it's worth. Mostly the point of
> > EE
> > > >> 9
> > > >>> is a
> > > >>>>> move of the EE stuff into a new namespace, allowing for further
> > > >>> development
> > > >>>>> under the Eclipse Foundation without trademark conflicts with
> > > >> Oracle's
> > > >>>>> "Java" branding. The trademark issue and follow on Java licensing
> > > >>> concerns
> > > >>>>> are going to be relevant for anyone who might be visited by an
> > Oracle
> > > >>>>> lawyer-shark.
> > > >>>>>
> > > >>>>> On Tue, Mar 4, 2025 at 6:42 AM Nihal Jain <nihalj...@apache.org>
> > > >>> wrote:
> > > >>>>>
> > > >>>>>>> I think that the best course of action would be adding a separate
> > > >>>>> Jetty11
> > > >>>>>> module to hbase-thirdparty, this way 9.4.x and 11.0.x can be both
> > > >>>>>> used/updated by the 2.x / 3.x branches.
> > > >>>>>>
> > > >>>>>> +1, I would like to volunteer for this work.
> > > >>>>>>
> > > >>>>>> Maintaining Jetty 9.4.x could be quite challenging from a CVE
> > > >>> perspective
> > > >>>>>> since it has been EOCS (End of Community Support) since June 2022,
> > > >>> and
> > > >>>>> new
> > > >>>>>> releases may not be forthcoming for a long time. I faced similar
> > > >>> issues
> > > >>>>>> during our last hbase-thirdparty release. See this issue comment:
> > > >>>>>>
> > > >>>>>
> > > >>>
> > > >>
> > https://github.com/jetty/jetty.project/issues/12630#issuecomment-2604701092
> > > >>>>>>
> > > >>>>>> Regarding the version and CVEs, I agree with @Andrew and suggest
> > > >>> that we
> > > >>>>>> jump directly to Jetty 12, bypassing Jetty 11, to support
> > > >>> javax.servlet
> > > >>>>> for
> > > >>>>>> JSP 2.3.1. Therefore, I plan to use the EE8 environment on Jetty
> > > >> 12.
> > > >>> See
> > > >>>>>> Jetty version history:
> > > >>> https://jetty.org/download.html#version-history
> > > >>>>>>
> > > >>>>>> @Istvan, is there any particular reason you recommend moving to
> > > >>> Jetty 11,
> > > >>>>>> which is also EOCS?
> > > >>>>>>
> > > >>>>>> If others are fine with me taking this up, I can create the
> > > >> necessary
> > > >>>>>> JIRAs for the Jetty migration project and start the work soon.
> > > >>>>>>
> > > >>>>>> Regards,
> > > >>>>>> Nihal
> > > >>>>>>
> > > >>>>>> On 2025/03/03 18:08:53 Wei-Chiu Chuang wrote:
> > > >>>>>>> FYI I think Hadoop will need to make similar moves too.
> > > >>>>>>> Sounds like Hadoop should start the work to drop JDK8 as well. I
> > > >>> know
> > > >>>>>> Steve
> > > >>>>>>> has a patch that requires JDK17+ due to Iceberg
> > > >>>>>>> https://github.com/apache/hadoop/pull/7316
> > > >>>>>>>
> > > >>>>>>> ---------- Forwarded message ---------
> > > >>>>>>> From: Istvan Toth <st...@apache.org>
> > > >>>>>>> Date: Sun, Mar 2, 2025 at 9:41 PM
> > > >>>>>>> Subject: [DISCUSS] Plans for JDK22 / SecurityManager removal
> > > >>>>>>> To: HBase Dev List <dev@hbase.apache.org>
> > > >>>>>>>
> > > >>>>>>>
> > > >>>>>>> Hi!
> > > >>>>>>>
> > > >>>>>>> The last big incompatible JDK change is the JEP411/JEP486
> > > >>>>> SecurityManager
> > > >>>>>>> removal process, which has gotten serious with JDK22, which
> > > >>> disables
> > > >>>>>>> Subject.doAs*() Subject.getSubject() by default (while providing
> > > >>> the
> > > >>>>> kind
> > > >>>>>>> of equivalent callAs() and current() methods), and kills
> > > >> automatic
> > > >>>>>> Subject
> > > >>>>>>> propagation to new threads.
> > > >>>>>>>
> > > >>>>>>> To add insult to injury the new methods are only available from
> > > >>> JDK18,
> > > >>>>> so
> > > >>>>>>> it is not possible to just move to the new API without breaking
> > > >>> JDK17.
> > > >>>>>>>
> > > >>>>>>> Some of this can only be solved in Hadoop (hopefully 3.5) , but
> > > >>> HBase
> > > >>>>> can
> > > >>>>>>> also take steps to move towards JDK22 compatibility in the
> > > >>> meantime.
> > > >>>>>>>
> > > >>>>>>> Here's the plan I have in mind:
> > > >>>>>>>
> > > >>>>>>> - Review and the dependencies WRT SecurityManage usage (just
> > > >>> search for
> > > >>>>>> API
> > > >>>>>>> calls)
> > > >>>>>>> - Update the dependencies as needed
> > > >>>>>>>
> > > >>>>>>> - Add a reflection-based compatibility Utility class based on
> > > >>> Jetty's
> > > >>>>>>> SecurityUtils, (or one of its derivatives)
> > > >>>>>>> - Replace the deprecated method calls with the compatibility
> > > >>> versions
> > > >>>>>>> - Wrap Runnables/Callables with Subject current()/callAs()
> > > >> methods.
> > > >>>>>>>
> > > >>>>>>> One large dependency I have identified is Jetty.
> > > >>>>>>> While technically we MAY be able to keep using 9.4 by overriding
> > > >>> its
> > > >>>>>>> default ThreadFactory, I think that considering that even
> > > >>> semi-formal
> > > >>>>>> Jetty
> > > >>>>>>> 9.4 support is on its very last legs (the latest update was never
> > > >>> even
> > > >>>>>>> announced and is not listed in the downloads page),  the
> > > >>> advantages of
> > > >>>>>>> updating to Jetty 11 on branch-3+ now far outweighs the added
> > > >>>>> maintenance
> > > >>>>>>> burden of the branch divergence.
> > > >>>>>>>
> > > >>>>>>> I think that the best course of action would be adding a separate
> > > >>>>> Jetty11
> > > >>>>>>> module to hbase-thirdparty, this way 9.4.x and 11.0.x can be both
> > > >>>>>>> used/updated by the 2.x / 3.x branches.
> > > >>>>>>>
> > > >>>>>>> What do you think ?
> > > >>>>>>>
> > > >>>>>>> - Do you agree that JEP411/486 should be treated as a priority ?
> > > >>>>>>> - Would you volunteer for the dependency audit ?
> > > >>>>>>> - Do you agree with the Jetty update, and the plan outlined
> > > >> above ?
> > > >>>>>>>
> > > >>>>>>> Istvan
> > > >>>>>>>
> > > >>>>>>> ps: Sorry, no links because GMail seems to react very bady to
> > > >> any.
> > > >>>>> Please
> > > >>>>>>> use search.
> > > >>>>>>>
> > > >>>>>>
> > > >>>>>>
> > > >> ---------------------------------------------------------------------
> > > >>>>>> To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
> > > >>>>>> For additional commands, e-mail:
> > common-dev-h...@hadoop.apache.org
> > > >>>>>>
> > > >>>>>>
> > > >>>>>
> > > >>>>> --
> > > >>>>> Best regards,
> > > >>>>> Andrew
> > > >>>>>
> > > >>>>> Unrest, ignorance distilled, nihilistic imbeciles -
> > > >>>>>    It's what we’ve earned
> > > >>>>> Welcome, apocalypse, what’s taken you so long?
> > > >>>>> Bring us the fitting end that we’ve been counting on
> > > >>>>>   - A23, Welcome, Apocalypse
> > > >>>>>
> > > >>>>
> > > >>>>
> > > >>>> --
> > > >>>> *István Tóth* | Sr. Staff Software Engineer
> > > >>>> *Email*: st...@cloudera.com
> > > >>>> cloudera.com <https://www.cloudera.com>
> > > >>>> [image: Cloudera] <https://www.cloudera.com/>
> > > >>>> [image: Cloudera on Twitter] <https://twitter.com/cloudera> [image:
> > > >>>> Cloudera on Facebook] <https://www.facebook.com/cloudera> [image:
> > > >>> Cloudera
> > > >>>> on LinkedIn] <https://www.linkedin.com/company/cloudera>
> > > >>>> ------------------------------
> > > >>>> ------------------------------
> > > >>>
> > > >>
> > > >>
> > > >> --
> > > >> Best regards,
> > > >> Andrew
> > > >>
> > > >> Unrest, ignorance distilled, nihilistic imbeciles -
> > > >>    It's what we’ve earned
> > > >> Welcome, apocalypse, what’s taken you so long?
> > > >> Bring us the fitting end that we’ve been counting on
> > > >>   - A23, Welcome, Apocalypse
> > > >>
> > > >
> > > >
> > > > --
> > > > *István Tóth* | Sr. Staff Software Engineer
> > > > *Email*: st...@cloudera.com
> > > > cloudera.com <https://www.cloudera.com>
> > > > [image: Cloudera] <https://www.cloudera.com/>
> > > > [image: Cloudera on Twitter] <https://twitter.com/cloudera> [image:
> > > > Cloudera on Facebook] <https://www.facebook.com/cloudera> [image:
> > Cloudera
> > > > on LinkedIn] <https://www.linkedin.com/company/cloudera>
> > > > ------------------------------
> > > > ------------------------------
> > >
> >
> 
> 
> -- 
> *István Tóth* | Sr. Staff Software Engineer
> *Email*: st...@cloudera.com
> cloudera.com <https://www.cloudera.com>
> [image: Cloudera] <https://www.cloudera.com/>
> [image: Cloudera on Twitter] <https://twitter.com/cloudera> [image:
> Cloudera on Facebook] <https://www.facebook.com/cloudera> [image: Cloudera
> on LinkedIn] <https://www.linkedin.com/company/cloudera>
> ------------------------------
> ------------------------------
> 

Reply via email to