Jialiang Cai created AMBARI-26642:
-------------------------------------
Summary: Modernize Java dependencies and build tooling baseline
Key: AMBARI-26642
URL: https://issues.apache.org/jira/browse/AMBARI-26642
Project: Ambari
Issue Type: Improvement
Reporter: Jialiang Cai
h2. Problem
Ambari trunk targets JDK 17 and has already migrated its primary web stack to
Jakarta APIs, Jersey 3, Guice 7, Spring 6, Jetty 11, and EclipseLink 4.
However, the Java dependency and build baseline still contains several
obsolete, unsupported, conflicting, or disconnected components. These issues
increase build latency, make framework upgrades unsafe, keep the API
documentation stale, and make tests difficult to maintain on JDK 17.
This issue tracks the Java dependency modernization as one coherent
deliverable. The implementation should be split into independently reviewable
commits while retaining this JIRA key.
h2. Evidence from trunk
The audit was performed against trunk commit {{eb23cba9c1}}.
h3. Build and dependency governance
* The root POM still declares legacy java.net, GlassFish, Atlassian, Spring
milestone, Sonatype staging, and ASF staging repositories. Maven attempts these
repositories for missing artifacts and metadata, which makes ordinary
dependency resolution slow and unreliable.
* {{ambari-web}} declares {{maven-resources-plugin}} three times and
{{ambari-admin}} declares it four times. Maven warns that the effective models
are malformed and may be rejected by future Maven versions.
* The build uses old core plugins, including Maven Compiler 3.5, Resources 2.6,
Antrun 1.7, Build Helper 1.8, Exec 1.2.1, Checkstyle Plugin 2.17 with
Checkstyle 8.9, Assembly 2.2-beta-5, Clean 2.5, Build Number 1.2, and Clover
3.1.11.
* The root POM declares {{jdk.version=17}}, but Maven Enforcer still accepts
Java 8 and the compiler uses {{source}} and {{target}} rather than {{release}}.
* There is no imported dependency BOM, dependency-convergence rule, upper-bound
rule, banned-dependency policy, automated dependency update configuration, or
generated SBOM.
* Dependency management still contains apparently unused entries for Spring
Shell 1.1.0.RC3, Spring Boot 1.0.2.RELEASE, Groovy 2.4.21, HTTP Builder 0.7.1,
and JLine 2.11.
h3. Conflicting runtime frameworks
* Ambari Server contains both {{slf4j-reload4j:2.0.0}} and
{{logback-classic:1.2.13}}. Runtime classpath inspection selects
{{org.slf4j.reload4j.Reload4jLoggerFactory}}, while
{{DatabaseConsistencyChecker}} casts an SLF4J logger to
{{ch.qos.logback.classic.Logger}}. That path can fail with
{{ClassCastException}}.
* Ambari Server compiles against {{jakarta.servlet-api:6.0.0}}, but Jetty
11.0.24 supplies {{jetty-jakarta-servlet-api:5.0.2}} at runtime. Compilation
can therefore use APIs that are absent from the deployed runtime.
* EclipseLink is managed as 4.0.2 but explicitly overridden to 4.0.1 in
{{ambari-server/pom.xml}}.
* Spring Framework 6.0.23 and Spring Security 6.0.8 are combined with Spring
LDAP 2.3.2.RELEASE instead of a supported, aligned Spring 6 dependency set.
* The server still includes the legacy Jackson JAX-RS provider while also using
Jersey 3's Jackson integration.
h3. Swagger and API documentation
* Swagger 1.x dependencies are fixed at {{io.swagger:*:1.6.8}}.
* 139 Ambari Server production Java files import Swagger 1 annotations.
{{ambari-utility}} also contains a custom {{AmbariSwaggerReader}} tightly
coupled to Swagger 1 model and Kongchen plugin APIs.
* The Kongchen Swagger Maven plugin and Swagger Codegen 2.2.2 executions in
{{ambari-server/pom.xml}} are commented out, so the specification is not
generated by the normal build.
* The committed Swagger 2.0 document was last updated in 2018 and currently
contains 91 paths and 164 operations. It cannot be treated as a reliable
representation of the current REST API.
* The bundled classic Swagger UI identifies itself as {{v2.1.1-M2}} and
includes old vendored JavaScript dependencies.
h3. Test stack
* The repository contains 802 Java test files using JUnit 4 and no JUnit
Jupiter imports.
* Ambari Server resolves JUnit 4.10, Mockito 3.5.10, and PowerMock 2.0.9 on its
test classpath.
* Ambari Server has 41 test files importing PowerMock, including 39 runner/rule
users and 40 {{PrepareForTest}} users. PowerMock relies on classloader and
deep-reflection behavior that obstructs JDK upgrades.
* Ambari Server also has 261 EasyMock test files and 35 Mockito test files.
EasyMock 5.2 is not itself an urgent upgrade blocker and should not be
mechanically rewritten as part of the first phase.
h3. Legacy and unsupported libraries
The Ambari Server compile classpath directly includes the following old
dependencies:
* H2 {{1.4.193}}: upgrade to a supported 2.x release and adapt SQL/test
configuration.
* Nimbus JOSE JWT {{5.9}}: upgrade to a supported 10.x release and retest
signature and claim validation.
* JSch {{0.1.54}}: remove if no runtime consumer remains; otherwise use the
maintained {{com.github.mwiede:jsch}} fork.
* SNMP4J {{1.10.1}}: upgrade to 3.x and test all notification versions and
authentication modes.
* c3p0 {{0.9.5.4}}: upgrade first, then evaluate a compatibility-preserving
migration to HikariCP separately.
* Dropwizard Metrics {{3.1.0}}: upgrade to Metrics 4 as an interim step or
replace the single JVM implementation with Micrometer.
* JNA {{4.2.2}} and libpam4j {{1.10}}: upgrade together and run native PAM
authentication tests.
* JavaMail {{1.5.2}}: migrate to Jakarta Mail API with the Angus Mail
implementation.
* Commons Net {{1.4.1}}: upgrade to 3.x.
* Commons IO {{2.8.0}}: upgrade to a supported 2.x release.
* Commons CSV {{1.1}}: upgrade to a supported 1.x release.
* Commons Codec {{1.8}}: upgrade to a supported release.
* Jettison {{1.1}}: replace the two source consumers with Jackson.
* PostgreSQL JDBC {{42.3.9}}: upgrade to the supported 42.7.x line.
* Testcontainers {{1.17.6}}: upgrade to the latest compatible 1.x line before
considering 2.x.
* ZooKeeper {{3.5.5}}: upgrade only with Hadoop and Curator compatibility
validation.
Additional large migration surfaces are:
* Commons Lang 2.6 is imported by 317 Java files, while Commons Lang 3 is used
by only 27 files. Migration must be performed module by module because some
APIs moved or changed semantics.
* Apache HttpClient 4 is imported by 57 Java files. HttpClient 5 uses different
packages and should be handled as a dedicated migration.
* 135 Java files contain 420 {{javax.xml.bind}} imports, while EclipseLink
already brings Jakarta JAXB 4. This leaves both JAXB generations on the server
classpath.
* {{ambari-metrics-common:2.7.0.0.0}} brings Jackson 1.9 and older Hadoop-era
dependencies. This should be upgraded at its source or isolated rather than
overridden blindly in Ambari Server.
* Apache Directory API {{2.0.0.AM1}} and ApacheDS Kerberos {{2.0.0-M24}} should
be upgraded with focused LDAP and Kerberos integration coverage.
h2. Proposed changes
h3. 1. Establish a supported build baseline
* Require Maven 3.9.x and JDK 17 through Maven Enforcer and CI images.
* Upgrade core Maven plugins and use Maven Compiler {{release=17}}.
* Remove unused repositories and duplicate plugin declarations.
* Centralize version properties or import appropriate BOMs for aligned
framework families.
* Add dependency convergence/upper-bound checks with documented exclusions for
Hadoop-controlled dependencies.
* Add banned dependencies for Swagger 1, PowerMock, obsolete {{javax}} APIs
after their migrations, and known legacy logging providers.
* Replace Clover with JaCoCo and generate a CycloneDX SBOM in CI.
* Remove unused dependency-management entries.
h3. 2. Converge the runtime stack
* Select one SLF4J provider for Ambari Server. The existing Logback
configuration and direct Logback usage make a supported Logback line the least
disruptive target.
* Remove {{slf4j-reload4j}} from the server runtime and migrate or bridge
remaining Log4j API callers.
* As an immediate safety fix, align the compile-time Servlet API with Jetty 11.
Then migrate to Jetty 12 EE10 and Servlet 6 in a separately reviewable commit.
* Align EclipseLink, Spring Framework, Spring Security, Spring LDAP, Jersey,
Jackson, and Jetty to mutually compatible stable release lines. Do not accept
milestone or release-candidate upgrades suggested by automated version tools.
* Remove the legacy Jackson JAX-RS provider after confirming Jersey 3
serialization parity.
h3. 3. Replace Swagger 1 with an OpenAPI 3 Jakarta toolchain
* Use the Jakarta artifacts from swagger-core v3 for JAX-RS scanning. Ambari is
a Jersey/JAX-RS application, so a Spring MVC-specific springdoc integration is
not appropriate.
* Reimplement nested subresource handling currently provided by
{{AmbariSwaggerReader}} and the custom {{SwaggerPreferredParent}} and
{{SwaggerOverwriteNestedAPI}} annotations.
* Move specification generation to an explicit, reproducible Maven/CI task
instead of coupling it to every Java compilation.
* Use OpenAPI Generator 7.x if generated HTML or clients are required.
* Replace the vendored Swagger UI with a maintained OpenAPI 3-compatible
distribution.
* Store and compare a canonical OpenAPI specification in CI. Treat
operation/path loss and incompatible schema changes as failures unless
explicitly approved.
h3. 4. Modernize tests incrementally
* Upgrade JUnit 4 to 4.13.2 as an interim compatibility step.
* Upgrade to Mockito 5 and remove the separate {{mockito-inline}} dependency
where Mockito 5's default mock maker is sufficient.
* Remove PowerMock by introducing injectable collaborators and using Mockito
static/construction mocking only where redesign is not yet practical.
* Introduce JUnit Jupiter with the Vintage engine, migrate module by module,
and prohibit new JUnit 4/PowerMock tests.
* Retain EasyMock temporarily; new tests should use the selected
Jupiter/Mockito baseline.
h3. 5. Upgrade or replace legacy runtime libraries
* Handle low-risk dependency upgrades in focused commits with their existing
unit/integration tests.
* Migrate JAXB and Mail from {{javax}} to Jakarta packages and remove duplicate
legacy implementations.
* Migrate Commons Lang 2 and HttpClient 4 in module-sized batches.
* Preserve existing c3p0 configuration keys and behavior if HikariCP is
introduced; provide a deprecation and compatibility path rather than silently
changing pool semantics.
* Coordinate Hadoop, ZooKeeper, Curator, Ambari Metrics, database driver, LDAP,
and Kerberos changes with their protocol and deployment compatibility matrices.
h2. Compatibility constraints
* Preserve the public Ambari REST API and generated OpenAPI contract unless a
change is explicitly documented and approved.
* Preserve existing database support and validate schema creation, upgrades,
and runtime behavior for PostgreSQL, MySQL/MariaDB, Oracle, and SQL Server.
* Preserve PAM, LDAP, Kerberos, SNMP, email, proxy, and HTTPS authentication
behavior.
* Remain on JDK 17 unless a separate project decision changes the Java baseline.
* Do not combine Spring 7, Jersey milestone, Jackson 3 release candidate, or
other unrelated major framework migrations into mechanical dependency updates.
h2. Suggested implementation order
# Maven/JDK baseline, repository cleanup, duplicate POM cleanup, dependency
policy, and SBOM.
# Logging provider correction, Servlet/Jetty alignment, EclipseLink
convergence, and stable Spring family alignment.
# Low-risk security and maintenance upgrades such as PostgreSQL, Commons
libraries, Nimbus, H2, JNA/libpam4j, Mail, SNMP4J, and Testcontainers.
# OpenAPI 3 generation, nested-resource support, specification diff, and
maintained UI.
# PowerMock removal and JUnit Jupiter migration.
# Commons Lang 2, HttpClient 4, JAXB, Metrics, c3p0, and upstream-managed
dependency migrations.
h2. Acceptance criteria
* A clean checkout builds with the documented Maven 3.9.x and JDK 17 toolchain
without malformed effective-POM warnings.
* CI rejects Java versions below 17, unresolved dependency convergence,
prohibited legacy dependencies, and unexpected OpenAPI contract changes.
* Ambari Server packages exactly one SLF4J provider and the database
consistency checker works without backend-specific cast failures.
* Compile-time and runtime Servlet API levels match.
* Swagger 1 dependencies, the disabled Swagger 1 generation configuration, and
the obsolete bundled Swagger UI are removed.
* OpenAPI 3 generation is reproducible and covers at least the existing 91
paths and 164 operations unless reviewed API changes explain the difference.
* PowerMock is removed, JUnit Jupiter is enabled, and no new JUnit 4 tests are
introduced.
* Legacy {{javax.xml.bind}} and JavaMail implementations are removed from
Ambari Server after Jakarta migration.
* Each dependency upgrade includes focused failure and recovery tests
appropriate to its subsystem.
* The final dependency tree, packaged runtime classpath, SBOM, and
supported-version documentation are reviewed and committed.
h2. Audit commands
The following read-only analysis commands completed successfully in the local
Ambari build container:
{code:shell}
mvn -pl ambari-server -DskipTests dependency:tree -Dscope=compile
mvn -pl ambari-server -DskipTests dependency:tree -Dscope=test
mvn -pl ambari-server -DskipTests
org.codehaus.mojo:versions-maven-plugin:2.18.0:display-dependency-updates
mvn -N -DskipTests
org.codehaus.mojo:versions-maven-plugin:2.18.0:display-plugin-updates
mvn -pl ambari-server -DskipTests dependency:build-classpath
-Dmdep.includeScope=runtime
{code}
No implementation compilation or test suite is claimed by this audit. Each
implementation commit must record its exact container build and test commands.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
