[
https://issues.apache.org/jira/browse/HADOOP-19395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jose Luis López updated HADOOP-19395:
-------------------------------------
Description:
Move Hadoop from the Java EE {{javax.*}} APIs to their Jakarta EE {{jakarta.*}}
successors: Jetty 12 ee10, Jersey 3, JAXB 4. This is the umbrella; the work is
done in its sub-tasks.
h3. Background
Since Jakarta EE 9 the servlet, JAX-RS, JAXB and related APIs live in the
{{jakarta}} namespace, and their {{javax}} versions get no new releases.
Downstream projects that build on Hadoop's servlet and authentication classes
are blocked until Hadoop moves: HBase (HBASE-29542) and Knox (KNOX-3309).
This issue starts from HADOOP-19972, which moves Hadoop to Jetty 12 on the ee8
environment, still serving {{javax.servlet}}. With the container already
maintained, what is left here is the namespace.
h3. Size
An inventory of the tree on the HADOOP-19972 branch finds:
* 3,368 Java EE references in 44 modules: JAX-RS 1,191, servlet 1,156, JAXB
892, {{javax.inject}} 88, others about 40;
* 140 references to Jetty's ee8 types, which become ee10;
* 86 Public or LimitedPrivate classes that use these types, among them
hadoop-auth's {{AuthenticationHandler}} family, the {{org.apache.hadoop.http}}
servlets and filters, and the yarn-api timeline records;
* 31 managed third-party dependencies bound to the javax APIs: Jersey 2, Jetty
ee8, Guice, the Jackson JAX-RS and JAXB modules, jettison, swagger, the JAXB
runtime and the API jars.
h3. Scope
* Every Java EE API that moved to {{jakarta.*}}, in main code, tests, resources
and POM shade configurations.
* Jetty 12 ee8 → ee10 (or ee11).
* The third-party dependencies bound to those APIs, as mapped in HADOOP-19625.
* The Public/LimitedPrivate classes that expose javax types, under a
compatibility policy agreed on the dev list.
* Shaded clients, LICENSE/NOTICE-binary, the release note, and a migration note
for downstream projects.
h3. Out of scope
* The container upgrade itself (HADOOP-19972).
* JDK packages that keep the {{javax}} name: {{javax.net}}, {{javax.crypto}},
{{javax.security.auth}}, {{javax.management}} and the like.
* jsr305 annotations ({{javax.annotation.Nullable}}, {{Nonnull}}) and JCache
({{javax.cache}}), which are not Jakarta EE.
* Backports to 3.x. This is an incompatible change for 4.0.
h3. Approach
Every daemon serves its web UI and REST APIs through {{HttpServer2}}, and
Jersey 3 cannot run on a javax container. The namespace therefore has to change
in all web modules at once; it cannot land module by module on trunk. The work
is split in two phases.
*Phase A, on trunk.* Preparation that changes no namespace:
* HADOOP-19971: remove servlet and Jetty types from classes that don't need
them.
* HADOOP-19625: map each javax-bound dependency to its jakarta replacement.
* Upgrade Guice 5.1.0 → 6.x, which accepts both {{javax.inject}} and
{{jakarta.inject}}.
* Remove the Jersey 1 leftover {{javax.ws.rs:jsr311-api}}.
* Agree the compatibility policy for the Public/LimitedPrivate classes.
* Add the javax inventory and an OpenRewrite recipe to dev-support.
*Phase B, on a feature branch.* The switch itself. The mechanical part is
generated with the OpenRewrite {{JavaxMigrationToJakarta}} recipe; each
sub-task holds the manual fixes for one group of modules, in dependency order:
* B1: common and auth: {{HttpServer2}}, the filters, and the dependency
versions.
* B2: HDFS and KMS.
* B3: YARN API and web framework.
* B4: YARN client and daemons.
* B5: MapReduce.
* B6: applications, tools and cloud connectors.
* B7: shaded clients, packaging and docs.
The branch is merged to trunk by vote once complete.
h3. Done when
* The javax inventory reports no Java EE references outside the out-of-scope
list, and a CI check keeps it that way.
* Every daemon serves on Jetty 12 ee10 with Jersey 3, and the full test suite
passes.
* The shaded clients build and pass their checks.
* The release note lists every incompatible change, and the migration note for
downstream projects is published.
h3. Open decisions (dev list)
* The compatibility policy for the Public/LimitedPrivate classes: change them
in 4.0, or deprecate first.
* Target ee10 (Jakarta EE 10, Jersey 3.1) or ee11 (Jakarta EE 11, Jersey 4).
* The app catalog, which runs on Solr 8 and Jetty 9.4: upgrade Solr, or retire
the module.
* Feature branch or a single large change.
was:
The project uses old libraries in the {{javax.*}} package namespace, mainly
{{javax.servlet}} & {{{}javax.annotation{}}}.
For example, the old servlet-api is included via the following dependency
management:
{code:xml}
<dependency>
<groupId>jakarta.servlet</groupId>
<artifactId>jakarta.servlet-api</artifactId>
<version>4.0.4</version>
</dependency>
{code}
Note that while the artifact is in maven's {{jakarta.servlet}} namespace, it
includes the {{javax.servlet}} classes - versions 5 and above include the
{{jakarta.servlet}} classes.
The scope of its use is extensive, spanning most modules, and in many cases in
public classes - however I'm not sure if any are considered a public API.
One caveat is that the current Jetty version (9.4.x) doesn't work with the
{{jakarta.servlet}} namespace, so it will still need to use the previous
{{javax.servlet}} namespace, but the latter can be used by getting
{{javax.servlet:javax.servlet-api:4.0.1}} specifically where it is used - and
having the old namespace for the maven coordinates as well means it wouldn't
cause a conflict.
I believe that the first logical step is to upgrade to version 5.0.0, which is
the first version that uses the {{jakarta.*}} package namespace, but is also
the last that is Java 8 compatible, and upgrade again to 6.1.0 once hadoop
migrates to Java 17 as a minimum A new Java 17 baseline will also support
upgrading Jetty, which at the latest version 12 can work with all servlet-api
versions.
> Upgrade javax to jakarta
> ------------------------
>
> Key: HADOOP-19395
> URL: https://issues.apache.org/jira/browse/HADOOP-19395
> Project: Hadoop Common
> Issue Type: Improvement
> Components: common
> Affects Versions: 3.5.0
> Reporter: Yaniv Kunda
> Priority: Major
>
> Move Hadoop from the Java EE {{javax.*}} APIs to their Jakarta EE
> {{jakarta.*}} successors: Jetty 12 ee10, Jersey 3, JAXB 4. This is the
> umbrella; the work is done in its sub-tasks.
> h3. Background
> Since Jakarta EE 9 the servlet, JAX-RS, JAXB and related APIs live in the
> {{jakarta}} namespace, and their {{javax}} versions get no new releases.
> Downstream projects that build on Hadoop's servlet and authentication classes
> are blocked until Hadoop moves: HBase (HBASE-29542) and Knox (KNOX-3309).
> This issue starts from HADOOP-19972, which moves Hadoop to Jetty 12 on the
> ee8 environment, still serving {{javax.servlet}}. With the container already
> maintained, what is left here is the namespace.
> h3. Size
> An inventory of the tree on the HADOOP-19972 branch finds:
> * 3,368 Java EE references in 44 modules: JAX-RS 1,191, servlet 1,156, JAXB
> 892, {{javax.inject}} 88, others about 40;
> * 140 references to Jetty's ee8 types, which become ee10;
> * 86 Public or LimitedPrivate classes that use these types, among them
> hadoop-auth's {{AuthenticationHandler}} family, the
> {{org.apache.hadoop.http}} servlets and filters, and the yarn-api timeline
> records;
> * 31 managed third-party dependencies bound to the javax APIs: Jersey 2,
> Jetty ee8, Guice, the Jackson JAX-RS and JAXB modules, jettison, swagger, the
> JAXB runtime and the API jars.
> h3. Scope
> * Every Java EE API that moved to {{jakarta.*}}, in main code, tests,
> resources and POM shade configurations.
> * Jetty 12 ee8 → ee10 (or ee11).
> * The third-party dependencies bound to those APIs, as mapped in HADOOP-19625.
> * The Public/LimitedPrivate classes that expose javax types, under a
> compatibility policy agreed on the dev list.
> * Shaded clients, LICENSE/NOTICE-binary, the release note, and a migration
> note for downstream projects.
> h3. Out of scope
> * The container upgrade itself (HADOOP-19972).
> * JDK packages that keep the {{javax}} name: {{javax.net}}, {{javax.crypto}},
> {{javax.security.auth}}, {{javax.management}} and the like.
> * jsr305 annotations ({{javax.annotation.Nullable}}, {{Nonnull}}) and JCache
> ({{javax.cache}}), which are not Jakarta EE.
> * Backports to 3.x. This is an incompatible change for 4.0.
> h3. Approach
> Every daemon serves its web UI and REST APIs through {{HttpServer2}}, and
> Jersey 3 cannot run on a javax container. The namespace therefore has to
> change in all web modules at once; it cannot land module by module on trunk.
> The work is split in two phases.
> *Phase A, on trunk.* Preparation that changes no namespace:
> * HADOOP-19971: remove servlet and Jetty types from classes that don't need
> them.
> * HADOOP-19625: map each javax-bound dependency to its jakarta replacement.
> * Upgrade Guice 5.1.0 → 6.x, which accepts both {{javax.inject}} and
> {{jakarta.inject}}.
> * Remove the Jersey 1 leftover {{javax.ws.rs:jsr311-api}}.
> * Agree the compatibility policy for the Public/LimitedPrivate classes.
> * Add the javax inventory and an OpenRewrite recipe to dev-support.
> *Phase B, on a feature branch.* The switch itself. The mechanical part is
> generated with the OpenRewrite {{JavaxMigrationToJakarta}} recipe; each
> sub-task holds the manual fixes for one group of modules, in dependency order:
> * B1: common and auth: {{HttpServer2}}, the filters, and the dependency
> versions.
> * B2: HDFS and KMS.
> * B3: YARN API and web framework.
> * B4: YARN client and daemons.
> * B5: MapReduce.
> * B6: applications, tools and cloud connectors.
> * B7: shaded clients, packaging and docs.
> The branch is merged to trunk by vote once complete.
> h3. Done when
> * The javax inventory reports no Java EE references outside the out-of-scope
> list, and a CI check keeps it that way.
> * Every daemon serves on Jetty 12 ee10 with Jersey 3, and the full test suite
> passes.
> * The shaded clients build and pass their checks.
> * The release note lists every incompatible change, and the migration note
> for downstream projects is published.
> h3. Open decisions (dev list)
> * The compatibility policy for the Public/LimitedPrivate classes: change them
> in 4.0, or deprecate first.
> * Target ee10 (Jakarta EE 10, Jersey 3.1) or ee11 (Jakarta EE 11, Jersey 4).
> * The app catalog, which runs on Solr 8 and Jetty 9.4: upgrade Solr, or
> retire the module.
> * Feature branch or a single large change.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
