[
https://issues.apache.org/jira/browse/HADOOP-19395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18123162#comment-18123162
]
Jose Luis López commented on HADOOP-19395:
------------------------------------------
Hi, I have been working on the phase 1 migration to jetty12-ee8, and there is a
need to organize the work and divide it. Here is a proposal. If you agree,
let's redefine the scope of this Jira to be an UMBRELLA for all subtasks
related with the namespace change.
h2. Proposal: splitting HADOOP-19395 (javax → jakarta)
h3. What has to change
These numbers come from an inventory of every git-tracked file, taken on the
Jetty 12 ee8 branch of HADOOP-19972.
* *3,368 javax EE references* across 44 modules:
** JAX-RS/Jersey: 1,191
** servlet: 1,156
** JAXB: 892
** {{{}javax.inject{}}}: 88
** others: about 40
* Most are imports (2,607). The rest are fully-qualified names in code (588),
POM shade configs (80), javadoc (16), resources (10) and string literals (6).
* *140 Jetty ee8 references* ({{{}org.eclipse.jetty.ee8.*{}}}) that become
ee10.
* {*}86 Public or LimitedPrivate classes use EE types{*}, counting class-level
or package-level {{{}@InterfaceAudience{}}}. The main groups:
** hadoop-auth: the {{AuthenticationHandler}} family, which HBase and Knox
build on;
** {{{}org.apache.hadoop.http{}}}: the servlets, filters and
{{{}FilterContainer{}}};
** {{{}CrossOriginFilter{}}}, {{{}RestCsrfPreventionFilter{}}},
{{{}XFrameOptionsFilter{}}};
** the yarn-api timeline records with their JAXB readers and writers (36
files);
** the yarn-common {{webapp}} framework, the yarn-services records,
{{LogsCLI}} and {{{}SchedConfCLI{}}}.
* Flagging is by import, so a few classes may use the type only internally.
Changing the rest is an incompatible API change, which is why this targets 4.0.
* *31 third-party artifacts managed in {{hadoop-project}} are bound to javax:*
** Jersey 2: 11
** Jetty: 6 {{jetty-ee8}} artifacts plus {{jetty-servlet-api}}
** Guice: {{{}guice{}}}, {{guice-servlet}}
** Jackson: {{{}jackson-jaxrs-json-provider{}}},
{{jackson-module-jaxb-annotations}}
** others: {{{}jettison{}}}, {{{}swagger-annotations{}}},
{{{}jaxb-runtime{}}}, {{{}javax.json{}}}, plus the API jars
* *Not in scope:*
** 1,150 JDK references ({{{}javax.net{}}}, {{{}javax.crypto{}}},
{{{}javax.security.auth{}}}…), which stay;
** 274 jsr305 annotations ({{{}javax.annotation.Nullable{}}}/{{{}Nonnull{}}}),
which are not Jakarta EE;
** JCache ({{{}javax.cache{}}}), which stays.
h3. Why it can't land module by module on trunk
Every daemon serves its web UI and REST APIs through {{{}HttpServer2{}}}, and
Jersey 3 can't run on a javax container. When {{HttpServer2}} moves to ee10,
every webapp module has to move with it. Tests make it tighter: yarn-client's
tests run the YARN daemons, so YARN can't be split cleanly either.
Jetty 12 could host ee8 and ee10 contexts in the same server. Using that would
mean shipping {{AuthenticationFilter}} and friends in both namespaces, so I'd
reject it.
That gives two phases: preparation that lands on trunk now, then the switch on
a feature branch.
h3. Phase A: preparation on trunk
None of these change the namespace, and each lands independently.
||JIRA||Scope||
|HADOOP-19971 (exists)|Remove servlet and Jetty types from classes that don't
need them. This shrinks the public surface before the switch.|
|HADOOP-19625 (exists, refocused)|Identify the jakarta replacements of the 31
javax-bound dependencies: Jersey 2 → 3.1/4.0, Jetty ee8 → ee10, JAXB 4, Guice 6
→ 7, {{jackson-jaxrs-*}} → {{{}jackson-jakarta-rs-*{}}},
{{jackson-module-jaxb-annotations}} →
{{{}jackson-module-jakarta-xmlbind-annotations{}}}. Flag the ones with no
jakarta release (jettison, {{{}swagger-annotations{}}}). No code change; to be
made a sub-task of HADOOP-19395.|
|*new:* Upgrade Guice 5.1.0 → 6.x|Guice 6 accepts both {{javax.inject}} and
{{{}jakarta.inject{}}}, so the switch becomes a Guice 7 version bump. About 104
main files in the YARN and MapReduce webapps use Guice or {{{}javax.inject{}}}.|
|*new:* Remove {{javax.ws.rs:jsr311-api}} 1.1.1 from
timelineservice-hbase-tests|Jersey 1 leftover.|
|*new:* Compatibility policy for the 86 Public/LimitedPrivate classes|List them
and decide on the dev list whether 4.0 changes them outright or deprecates
first. Needs a release note and a migration note for downstream (HBase, Knox,
Hive…).|
|*new:* Add the javax inventory and an OpenRewrite recipe to dev-support|Makes
the switch reproducible and reviewable. After the switch, CI fails on new javax
EE imports.|
|HADOOP-19861 (exists)|Rescope it: hadoop-cos only uses jsr305
{{{}Nullable{}}}, which isn't Jakarta EE, so it doesn't belong under
HADOOP-19395.|
h3. Phase B: the switch, on a HADOOP-19395 feature branch
The branch is merged to trunk by vote once complete. Most of the change is
mechanical: the OpenRewrite {{JavaxMigrationToJakarta}} recipe, reviewed as a
recipe rather than as 3,000 changed imports. Each sub-JIRA covers the manual
fixes for one group, in dependency order:
||Sub-JIRA (new)||Modules||EE refs||Jetty ee8 refs||Main / test files||Public
API files||
|B1. Common and auth: Jetty 12 ee10, Jersey 3, JAXB 4|{{hadoop-project}}
(dependency versions, from HADOOP-19625), hadoop-auth, hadoop-common
({{{}HttpServer2{}}}, filters), hadoop-auth-examples|422|33|49 / 34|19|
|B2. HDFS and KMS|hadoop-hdfs-client, hadoop-hdfs, hadoop-hdfs-httpfs,
hadoop-hdfs-rbf, hadoop-kms|357|10|51 / 19|1|
|B3. YARN API and web framework|hadoop-yarn-api (JAXB records),
hadoop-yarn-common ({{{}webapp{}}}), hadoop-yarn-server-common,
hadoop-yarn-server-web-proxy|512|7|86 / 19|51|
|B4. YARN client and daemons|hadoop-yarn-client, ResourceManager, NodeManager,
ApplicationHistoryService, timelineservice (with its HBase server and tests),
SharedCacheManager, Router, GlobalPolicyGenerator|1,475|45|161 / 100|2|
|B5. MapReduce|mapreduce-client-core, mapreduce-client-app,
mapreduce-client-hs, mapreduce-client-jobclient|363|2|39 / 23|1|
|B6. Applications, tools and cloud connectors|yarn-services-core,
yarn-services-api, distributedshell, app catalog, resourceestimator, sls,
extras, aws, huaweicloud, bos, gcp, cloud-storage|188|29|33 / 8|12|
|B7. Shaded clients, packaging and docs|hadoop-client, hadoop-client-api,
hadoop-client-runtime, hadoop-client-minicluster (relocations and exclusions),
LICENSE/NOTICE-binary, release note, downstream migration note|51|14|—|—|
|*Total*| |*3,368*|*140*| |*86*|
Notes:
* *B1 is what downstream is waiting for.* HBase's phase 2 (HBASE-29542) is
blocked on it, and Knox has the same dependency (KNOX-3309). Done first, it
gives them something to test against early.
* *B1 needs HADOOP-19625 first.* The dependency versions B1 sets in
{{hadoop-project}} come from its mapping.
* *B3 holds 51 of the 86 public API files.* That's where the compatibility
policy from Phase A matters most.
* *B4 includes yarn-client.* Its tests run the YARN daemons, so it can't be
finished before them.
* *B6 needs one decision first:* the app catalog runs on Solr 8, which is
pinned to Jetty 9.4 and javax. It either needs a Solr upgrade or should be
retired.
* *Ten configuration entries need a manual look:* bare {{javax.annotation}}
and {{javax.xml}} prefixes in
{{org.apache.hadoop.application-classloader.properties}} and in the shaded-jar
content check. They decide which classes user code shares with Hadoop, so they
probably need {{jakarta.}} added.
> Upgrade javax to jakarta
> ------------------------
>
> Key: HADOOP-19395
> URL: https://issues.apache.org/jira/browse/HADOOP-19395
> Project: Hadoop Common
> Issue Type: Improvement
> Components: common
> Affects Versions: 3.5.0
> Reporter: Yaniv Kunda
> Priority: Major
>
> The project uses old libraries in the {{javax.*}} package namespace, mainly
> {{javax.servlet}} & {{{}javax.annotation{}}}.
> For example, the old servlet-api is included via the following dependency
> management:
> {code:xml}
> <dependency>
> <groupId>jakarta.servlet</groupId>
> <artifactId>jakarta.servlet-api</artifactId>
> <version>4.0.4</version>
> </dependency>
> {code}
> Note that while the artifact is in maven's {{jakarta.servlet}} namespace, it
> includes the {{javax.servlet}} classes - versions 5 and above include the
> {{jakarta.servlet}} classes.
> The scope of its use is extensive, spanning most modules, and in many cases
> in public classes - however I'm not sure if any are considered a public API.
> One caveat is that the current Jetty version (9.4.x) doesn't work with the
> {{jakarta.servlet}} namespace, so it will still need to use the previous
> {{javax.servlet}} namespace, but the latter can be used by getting
> {{javax.servlet:javax.servlet-api:4.0.1}} specifically where it is used - and
> having the old namespace for the maven coordinates as well means it wouldn't
> cause a conflict.
> I believe that the first logical step is to upgrade to version 5.0.0, which
> is the first version that uses the {{jakarta.*}} package namespace, but is
> also the last that is Java 8 compatible, and upgrade again to 6.1.0 once
> hadoop migrates to Java 17 as a minimum A new Java 17 baseline will also
> support upgrading Jetty, which at the latest version 12 can work with all
> servlet-api versions.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
