[
https://issues.apache.org/jira/browse/HADOOP-19395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jose Luis López updated HADOOP-19395:
-------------------------------------
Description:
Move Hadoop from the Java EE javax. APIs to their Jakarta EE {{jakarta.}}{*}
successors: Jetty 12 ee10, Jersey 3, JAXB 4. This is the umbrella; the work is
done in its sub-tasks. This work needs first the migration to Jetty12-ee8
described in HADOOP-19912.
h3. Background
Since Jakarta EE 9 the servlet, JAX-RS, JAXB and related APIs live in the
{{jakarta}} namespace, and their {{javax}} versions get no new releases.
Downstream projects that build on Hadoop's servlet and authentication classes
are blocked until Hadoop moves: HBase (HBASE-29542) and Knox (KNOX-3309).
This issue starts from HADOOP-19912, which moves Hadoop to Jetty 12 on the ee8
environment, still serving {{{}javax.servlet{}}}: HADOOP-19972 upgrades the web
servers, and HADOOP-19998 moves the YARN application catalog, the last module
on Jetty 9.4, off it. With the container already maintained, what is left here
is the namespace.
h3. Size
An inventory of the tree on the HADOOP-19972 branch finds:
* 3,368 Java EE references in 44 modules: JAX-RS 1,191, servlet 1,156, JAXB
892, {{javax.inject}} 88, others about 40;
* 140 references to Jetty's ee8 types, which become ee10;
* 86 Public or LimitedPrivate classes that use these types, among them
hadoop-auth's {{AuthenticationHandler}} family, the {{org.apache.hadoop.http}}
servlets and filters, and the yarn-api timeline records;
* 31 managed third-party dependencies bound to the javax APIs: Jersey 2, Jetty
ee8, Guice, the Jackson JAX-RS and JAXB modules, jettison, swagger, the JAXB
runtime and the API jars.
h3. Scope
* Every Java EE API that moved to {{{}jakarta.*{}}}, in main code, tests,
resources and POM shade configurations.
* Jetty 12 ee8 → ee10 (or ee11).
* The third-party dependencies bound to those APIs, as mapped in HADOOP-19625.
* The Public/LimitedPrivate classes that expose javax types, under a
compatibility policy agreed on the dev list.
* Shaded clients, LICENSE/NOTICE-binary, the release note, and a migration
note for downstream projects.
h3. Out of scope
* The container upgrade itself, Jetty 12 on ee8, and the YARN application
catalog's move off Jetty 9.4: HADOOP-19912.
* JDK packages that keep the {{javax}} name: {{{}javax.net{}}},
{{{}javax.crypto{}}}, {{{}javax.security.auth{}}}, {{javax.management}} and the
like.
* jsr305 annotations ({{{}javax.annotation.Nullable{}}}, {{{}Nonnull{}}}) and
JCache ({{{}javax.cache{}}}), which are not Jakarta EE.
* Backports to 3.x. This is an incompatible change for 4.0.
h3. Approach
Every daemon serves its web UI and REST APIs through {{{}HttpServer2{}}}, and
Jersey 3 cannot run on a javax container. The namespace therefore has to change
in all web modules at once; it cannot land module by module on trunk. The work
is split in two phases.
*Phase A, on trunk.* Preparation that changes no namespace:
* HADOOP-19971: remove servlet and Jetty types from classes that don't need
them.
* HADOOP-19625: map each javax-bound dependency to its jakarta replacement.
* Agree the compatibility policy for the Public/LimitedPrivate classes.
* Add the javax inventory and an OpenRewrite recipe to dev-support.
*Phase B, on a feature branch.* The switch itself. The mechanical part is
generated with the OpenRewrite {{JavaxMigrationToJakarta}} recipe; each
sub-task holds the manual fixes for one group of modules, in dependency order:
* B1: common and auth: {{{}HttpServer2{}}}, the filters, and the dependency
versions.
* B2: HDFS and KMS.
* B3: YARN API and web framework.
* B4: YARN client and daemons.
* B5: MapReduce.
* B6: applications, tools and cloud connectors.
* B7: shaded clients, packaging and docs.
The branch is merged to trunk by vote once complete.
h3. Done when
* The javax inventory reports no Java EE references outside the out-of-scope
list, and a CI check keeps it that way.
* Every daemon serves on Jetty 12 ee10 with Jersey 3, and the full test suite
passes.
* The shaded clients build and pass their checks.
* The release note lists every incompatible change, and the migration note for
downstream projects is published.
h3. Open decisions
* The compatibility policy for the Public/LimitedPrivate classes: change them
in 4.0, or deprecate first.
* Target ee10 (Jakarta EE 10, Jersey 3.1) or ee11 (Jakarta EE 11, Jersey 4).
* Feature branch or a single large change.
was:
Move Hadoop from the Java EE javax. APIs to their Jakarta EE {{jakarta.}}{*}
successors: Jetty 12 ee10, Jersey 3, JAXB 4. This is the umbrella; the work is
done in its sub-tasks. This work needs first the migration to Jetty12-ee8
described in HADOOP-19912.
h3. Background
Since Jakarta EE 9 the servlet, JAX-RS, JAXB and related APIs live in the
{{jakarta}} namespace, and their {{javax}} versions get no new releases.
Downstream projects that build on Hadoop's servlet and authentication classes
are blocked until Hadoop moves: HBase (HBASE-29542) and Knox (KNOX-3309).
This issue starts from HADOOP-19972, which moves Hadoop to Jetty 12 on the ee8
environment, still serving {{{}javax.servlet{}}}. With the container already
maintained, what is left here is the namespace.
h3. Size
An inventory of the tree on the HADOOP-19972 branch finds:
* 3,368 Java EE references in 44 modules: JAX-RS 1,191, servlet 1,156, JAXB
892, {{javax.inject}} 88, others about 40;
* 140 references to Jetty's ee8 types, which become ee10;
* 86 Public or LimitedPrivate classes that use these types, among them
hadoop-auth's {{AuthenticationHandler}} family, the {{org.apache.hadoop.http}}
servlets and filters, and the yarn-api timeline records;
* 31 managed third-party dependencies bound to the javax APIs: Jersey 2, Jetty
ee8, Guice, the Jackson JAX-RS and JAXB modules, jettison, swagger, the JAXB
runtime and the API jars.
h3. Scope
* Every Java EE API that moved to {{{}jakarta.*{}}}, in main code, tests,
resources and POM shade configurations.
* Jetty 12 ee8 → ee10 (or ee11).
* The third-party dependencies bound to those APIs, as mapped in HADOOP-19625.
* The Public/LimitedPrivate classes that expose javax types, under a
compatibility policy agreed on the dev list.
* Shaded clients, LICENSE/NOTICE-binary, the release note, and a migration
note for downstream projects.
h3. Out of scope
* The container upgrade itself (HADOOP-19972).
* JDK packages that keep the {{javax}} name: {{{}javax.net{}}},
{{{}javax.crypto{}}}, {{{}javax.security.auth{}}}, {{javax.management}} and the
like.
* jsr305 annotations ({{{}javax.annotation.Nullable{}}}, {{{}Nonnull{}}}) and
JCache ({{{}javax.cache{}}}), which are not Jakarta EE.
* Backports to 3.x. This is an incompatible change for 4.0.
h3. Approach
Every daemon serves its web UI and REST APIs through {{{}HttpServer2{}}}, and
Jersey 3 cannot run on a javax container. The namespace therefore has to change
in all web modules at once; it cannot land module by module on trunk. The work
is split in two phases.
*Phase A, on trunk.* Preparation that changes no namespace:
* HADOOP-19971: remove servlet and Jetty types from classes that don't need
them.
* HADOOP-19625: map each javax-bound dependency to its jakarta replacement.
* Agree the compatibility policy for the Public/LimitedPrivate classes.
* Add the javax inventory and an OpenRewrite recipe to dev-support.
*Phase B, on a feature branch.* The switch itself. The mechanical part is
generated with the OpenRewrite {{JavaxMigrationToJakarta}} recipe; each
sub-task holds the manual fixes for one group of modules, in dependency order:
* B1: common and auth: {{{}HttpServer2{}}}, the filters, and the dependency
versions.
* B2: HDFS and KMS.
* B3: YARN API and web framework.
* B4: YARN client and daemons.
* B5: MapReduce.
* B6: applications, tools and cloud connectors.
* B7: shaded clients, packaging and docs.
The branch is merged to trunk by vote once complete.
h3. Done when
* The javax inventory reports no Java EE references outside the out-of-scope
list, and a CI check keeps it that way.
* Every daemon serves on Jetty 12 ee10 with Jersey 3, and the full test suite
passes.
* The shaded clients build and pass their checks.
* The release note lists every incompatible change, and the migration note for
downstream projects is published.
h3. Open decisions
* The compatibility policy for the Public/LimitedPrivate classes: change them
in 4.0, or deprecate first.
* Target ee10 (Jakarta EE 10, Jersey 3.1) or ee11 (Jakarta EE 11, Jersey 4).
* Feature branch or a single large change.
> Migrate Java EE APIs from javax to jakarta (servlet, JAX-RS, JAXB)
> ------------------------------------------------------------------
>
> Key: HADOOP-19395
> URL: https://issues.apache.org/jira/browse/HADOOP-19395
> Project: Hadoop Common
> Issue Type: Improvement
> Components: common
> Affects Versions: 3.6.0
> Reporter: Yaniv Kunda
> Assignee: Jose Luis López
> Priority: Major
>
> Move Hadoop from the Java EE javax. APIs to their Jakarta EE {{jakarta.}}{*}
> successors: Jetty 12 ee10, Jersey 3, JAXB 4. This is the umbrella; the work
> is done in its sub-tasks. This work needs first the migration to Jetty12-ee8
> described in HADOOP-19912.
> h3. Background
> Since Jakarta EE 9 the servlet, JAX-RS, JAXB and related APIs live in the
> {{jakarta}} namespace, and their {{javax}} versions get no new releases.
> Downstream projects that build on Hadoop's servlet and authentication classes
> are blocked until Hadoop moves: HBase (HBASE-29542) and Knox (KNOX-3309).
> This issue starts from HADOOP-19912, which moves Hadoop to Jetty 12 on the
> ee8 environment, still serving {{{}javax.servlet{}}}: HADOOP-19972 upgrades
> the web servers, and HADOOP-19998 moves the YARN application catalog, the
> last module on Jetty 9.4, off it. With the container already maintained, what
> is left here is the namespace.
> h3. Size
> An inventory of the tree on the HADOOP-19972 branch finds:
> * 3,368 Java EE references in 44 modules: JAX-RS 1,191, servlet 1,156, JAXB
> 892, {{javax.inject}} 88, others about 40;
> * 140 references to Jetty's ee8 types, which become ee10;
> * 86 Public or LimitedPrivate classes that use these types, among them
> hadoop-auth's {{AuthenticationHandler}} family, the
> {{org.apache.hadoop.http}} servlets and filters, and the yarn-api timeline
> records;
> * 31 managed third-party dependencies bound to the javax APIs: Jersey 2,
> Jetty ee8, Guice, the Jackson JAX-RS and JAXB modules, jettison, swagger, the
> JAXB runtime and the API jars.
> h3. Scope
> * Every Java EE API that moved to {{{}jakarta.*{}}}, in main code, tests,
> resources and POM shade configurations.
> * Jetty 12 ee8 → ee10 (or ee11).
> * The third-party dependencies bound to those APIs, as mapped in
> HADOOP-19625.
> * The Public/LimitedPrivate classes that expose javax types, under a
> compatibility policy agreed on the dev list.
> * Shaded clients, LICENSE/NOTICE-binary, the release note, and a migration
> note for downstream projects.
> h3. Out of scope
> * The container upgrade itself, Jetty 12 on ee8, and the YARN application
> catalog's move off Jetty 9.4: HADOOP-19912.
> * JDK packages that keep the {{javax}} name: {{{}javax.net{}}},
> {{{}javax.crypto{}}}, {{{}javax.security.auth{}}}, {{javax.management}} and
> the like.
> * jsr305 annotations ({{{}javax.annotation.Nullable{}}}, {{{}Nonnull{}}})
> and JCache ({{{}javax.cache{}}}), which are not Jakarta EE.
> * Backports to 3.x. This is an incompatible change for 4.0.
> h3. Approach
> Every daemon serves its web UI and REST APIs through {{{}HttpServer2{}}}, and
> Jersey 3 cannot run on a javax container. The namespace therefore has to
> change in all web modules at once; it cannot land module by module on trunk.
> The work is split in two phases.
> *Phase A, on trunk.* Preparation that changes no namespace:
> * HADOOP-19971: remove servlet and Jetty types from classes that don't need
> them.
> * HADOOP-19625: map each javax-bound dependency to its jakarta replacement.
> * Agree the compatibility policy for the Public/LimitedPrivate classes.
> * Add the javax inventory and an OpenRewrite recipe to dev-support.
> *Phase B, on a feature branch.* The switch itself. The mechanical part is
> generated with the OpenRewrite {{JavaxMigrationToJakarta}} recipe; each
> sub-task holds the manual fixes for one group of modules, in dependency order:
> * B1: common and auth: {{{}HttpServer2{}}}, the filters, and the dependency
> versions.
> * B2: HDFS and KMS.
> * B3: YARN API and web framework.
> * B4: YARN client and daemons.
> * B5: MapReduce.
> * B6: applications, tools and cloud connectors.
> * B7: shaded clients, packaging and docs.
> The branch is merged to trunk by vote once complete.
> h3. Done when
> * The javax inventory reports no Java EE references outside the out-of-scope
> list, and a CI check keeps it that way.
> * Every daemon serves on Jetty 12 ee10 with Jersey 3, and the full test
> suite passes.
> * The shaded clients build and pass their checks.
> * The release note lists every incompatible change, and the migration note
> for downstream projects is published.
> h3. Open decisions
> * The compatibility policy for the Public/LimitedPrivate classes: change
> them in 4.0, or deprecate first.
> * Target ee10 (Jakarta EE 10, Jersey 3.1) or ee11 (Jakarta EE 11, Jersey 4).
> * Feature branch or a single large change.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
