[
https://issues.apache.org/jira/browse/AMBARI-26642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jialiang Cai reassigned AMBARI-26642:
-------------------------------------
Assignee: Jialiang Cai
> Modernize Java dependencies and build tooling baseline
> ------------------------------------------------------
>
> Key: AMBARI-26642
> URL: https://issues.apache.org/jira/browse/AMBARI-26642
> Project: Ambari
> Issue Type: Improvement
> Reporter: Jialiang Cai
> Assignee: Jialiang Cai
> Priority: Major
> Time Spent: 20m
> Remaining Estimate: 0h
>
> h2. Problem
> Ambari trunk targets JDK 17 and has already migrated its primary web stack to
> Jakarta APIs, Jersey 3, Guice 7, Spring 6, Jetty 11, and EclipseLink 4.
> However, the Java dependency and build baseline still contains several
> obsolete, unsupported, conflicting, or disconnected components. These issues
> increase build latency, make framework upgrades unsafe, keep the API
> documentation stale, and make tests difficult to maintain on JDK 17.
> This issue tracks the Java dependency modernization as one coherent
> deliverable. The implementation should be split into independently reviewable
> commits while retaining this JIRA key.
> h2. Evidence from trunk
> The audit was performed against trunk commit {{eb23cba9c1}}.
> h3. Build and dependency governance
> * The root POM still declares legacy java.net, GlassFish, Atlassian, Spring
> milestone, Sonatype staging, and ASF staging repositories. Maven attempts
> these repositories for missing artifacts and metadata, which makes ordinary
> dependency resolution slow and unreliable.
> * {{ambari-web}} declares {{maven-resources-plugin}} three times and
> {{ambari-admin}} declares it four times. Maven warns that the effective
> models are malformed and may be rejected by future Maven versions.
> * The build uses old core plugins, including Maven Compiler 3.5, Resources
> 2.6, Antrun 1.7, Build Helper 1.8, Exec 1.2.1, Checkstyle Plugin 2.17 with
> Checkstyle 8.9, Assembly 2.2-beta-5, Clean 2.5, Build Number 1.2, and Clover
> 3.1.11.
> * The root POM declares {{jdk.version=17}}, but Maven Enforcer still accepts
> Java 8 and the compiler uses {{source}} and {{target}} rather than
> {{release}}.
> * There is no imported dependency BOM, dependency-convergence rule,
> upper-bound rule, banned-dependency policy, automated dependency update
> configuration, or generated SBOM.
> * Dependency management still contains apparently unused entries for Spring
> Shell 1.1.0.RC3, Spring Boot 1.0.2.RELEASE, Groovy 2.4.21, HTTP Builder
> 0.7.1, and JLine 2.11.
> h3. Conflicting runtime frameworks
> * Ambari Server contains both {{slf4j-reload4j:2.0.0}} and
> {{logback-classic:1.2.13}}. Runtime classpath inspection selects
> {{org.slf4j.reload4j.Reload4jLoggerFactory}}, while
> {{DatabaseConsistencyChecker}} casts an SLF4J logger to
> {{ch.qos.logback.classic.Logger}}. That path can fail with
> {{ClassCastException}}.
> * Ambari Server compiles against {{jakarta.servlet-api:6.0.0}}, but Jetty
> 11.0.24 supplies {{jetty-jakarta-servlet-api:5.0.2}} at runtime. Compilation
> can therefore use APIs that are absent from the deployed runtime.
> * EclipseLink is managed as 4.0.2 but explicitly overridden to 4.0.1 in
> {{ambari-server/pom.xml}}.
> * Spring Framework 6.0.23 and Spring Security 6.0.8 are combined with Spring
> LDAP 2.3.2.RELEASE instead of a supported, aligned Spring 6 dependency set.
> * The server still includes the legacy Jackson JAX-RS provider while also
> using Jersey 3's Jackson integration.
> h3. Swagger and API documentation
> * Swagger 1.x dependencies are fixed at {{io.swagger:*:1.6.8}}.
> * 139 Ambari Server production Java files import Swagger 1 annotations.
> {{ambari-utility}} also contains a custom {{AmbariSwaggerReader}} tightly
> coupled to Swagger 1 model and Kongchen plugin APIs.
> * The Kongchen Swagger Maven plugin and Swagger Codegen 2.2.2 executions in
> {{ambari-server/pom.xml}} are commented out, so the specification is not
> generated by the normal build.
> * The committed Swagger 2.0 document was last updated in 2018 and currently
> contains 91 paths and 164 operations. It cannot be treated as a reliable
> representation of the current REST API.
> * The bundled classic Swagger UI identifies itself as {{v2.1.1-M2}} and
> includes old vendored JavaScript dependencies.
> h3. Test stack
> * The repository contains 802 Java test files using JUnit 4 and no JUnit
> Jupiter imports.
> * Ambari Server resolves JUnit 4.10, Mockito 3.5.10, and PowerMock 2.0.9 on
> its test classpath.
> * Ambari Server has 41 test files importing PowerMock, including 39
> runner/rule users and 40 {{PrepareForTest}} users. PowerMock relies on
> classloader and deep-reflection behavior that obstructs JDK upgrades.
> * Ambari Server also has 261 EasyMock test files and 35 Mockito test files.
> EasyMock 5.2 is not itself an urgent upgrade blocker and should not be
> mechanically rewritten as part of the first phase.
> h3. Legacy and unsupported libraries
> The Ambari Server compile classpath directly includes the following old
> dependencies:
> * H2 {{1.4.193}}: upgrade to a supported 2.x release and adapt SQL/test
> configuration.
> * Nimbus JOSE JWT {{5.9}}: upgrade to a supported 10.x release and retest
> signature and claim validation.
> * JSch {{0.1.54}}: remove if no runtime consumer remains; otherwise use the
> maintained {{com.github.mwiede:jsch}} fork.
> * SNMP4J {{1.10.1}}: upgrade to 3.x and test all notification versions and
> authentication modes.
> * c3p0 {{0.9.5.4}}: upgrade first, then evaluate a compatibility-preserving
> migration to HikariCP separately.
> * Dropwizard Metrics {{3.1.0}}: upgrade to Metrics 4 as an interim step or
> replace the single JVM implementation with Micrometer.
> * JNA {{4.2.2}} and libpam4j {{1.10}}: upgrade together and run native PAM
> authentication tests.
> * JavaMail {{1.5.2}}: migrate to Jakarta Mail API with the Angus Mail
> implementation.
> * Commons Net {{1.4.1}}: upgrade to 3.x.
> * Commons IO {{2.8.0}}: upgrade to a supported 2.x release.
> * Commons CSV {{1.1}}: upgrade to a supported 1.x release.
> * Commons Codec {{1.8}}: upgrade to a supported release.
> * Jettison {{1.1}}: replace the two source consumers with Jackson.
> * PostgreSQL JDBC {{42.3.9}}: upgrade to the supported 42.7.x line.
> * Testcontainers {{1.17.6}}: upgrade to the latest compatible 1.x line before
> considering 2.x.
> * ZooKeeper {{3.5.5}}: upgrade only with Hadoop and Curator compatibility
> validation.
> Additional large migration surfaces are:
> * Commons Lang 2.6 is imported by 317 Java files, while Commons Lang 3 is
> used by only 27 files. Migration must be performed module by module because
> some APIs moved or changed semantics.
> * Apache HttpClient 4 is imported by 57 Java files. HttpClient 5 uses
> different packages and should be handled as a dedicated migration.
> * 135 Java files contain 420 {{javax.xml.bind}} imports, while EclipseLink
> already brings Jakarta JAXB 4. This leaves both JAXB generations on the
> server classpath.
> * {{ambari-metrics-common:2.7.0.0.0}} brings Jackson 1.9 and older Hadoop-era
> dependencies. This should be upgraded at its source or isolated rather than
> overridden blindly in Ambari Server.
> * Apache Directory API {{2.0.0.AM1}} and ApacheDS Kerberos {{2.0.0-M24}}
> should be upgraded with focused LDAP and Kerberos integration coverage.
> h2. Proposed changes
> h3. 1. Establish a supported build baseline
> * Require Maven 3.9.x and JDK 17 through Maven Enforcer and CI images.
> * Upgrade core Maven plugins and use Maven Compiler {{release=17}}.
> * Remove unused repositories and duplicate plugin declarations.
> * Centralize version properties or import appropriate BOMs for aligned
> framework families.
> * Add dependency convergence/upper-bound checks with documented exclusions
> for Hadoop-controlled dependencies.
> * Add banned dependencies for Swagger 1, PowerMock, obsolete {{javax}} APIs
> after their migrations, and known legacy logging providers.
> * Replace Clover with JaCoCo and generate a CycloneDX SBOM in CI.
> * Remove unused dependency-management entries.
> h3. 2. Converge the runtime stack
> * Select one SLF4J provider for Ambari Server. The existing Logback
> configuration and direct Logback usage make a supported Logback line the
> least disruptive target.
> * Remove {{slf4j-reload4j}} from the server runtime and migrate or bridge
> remaining Log4j API callers.
> * As an immediate safety fix, align the compile-time Servlet API with Jetty
> 11. Then migrate to Jetty 12 EE10 and Servlet 6 in a separately reviewable
> commit.
> * Align EclipseLink, Spring Framework, Spring Security, Spring LDAP, Jersey,
> Jackson, and Jetty to mutually compatible stable release lines. Do not accept
> milestone or release-candidate upgrades suggested by automated version tools.
> * Remove the legacy Jackson JAX-RS provider after confirming Jersey 3
> serialization parity.
> h3. 3. Replace Swagger 1 with an OpenAPI 3 Jakarta toolchain
> * Use the Jakarta artifacts from swagger-core v3 for JAX-RS scanning. Ambari
> is a Jersey/JAX-RS application, so a Spring MVC-specific springdoc
> integration is not appropriate.
> * Reimplement nested subresource handling currently provided by
> {{AmbariSwaggerReader}} and the custom {{SwaggerPreferredParent}} and
> {{SwaggerOverwriteNestedAPI}} annotations.
> * Move specification generation to an explicit, reproducible Maven/CI task
> instead of coupling it to every Java compilation.
> * Use OpenAPI Generator 7.x if generated HTML or clients are required.
> * Replace the vendored Swagger UI with a maintained OpenAPI 3-compatible
> distribution.
> * Store and compare a canonical OpenAPI specification in CI. Treat
> operation/path loss and incompatible schema changes as failures unless
> explicitly approved.
> h3. 4. Modernize tests incrementally
> * Upgrade JUnit 4 to 4.13.2 as an interim compatibility step.
> * Upgrade to Mockito 5 and remove the separate {{mockito-inline}} dependency
> where Mockito 5's default mock maker is sufficient.
> * Remove PowerMock by introducing injectable collaborators and using Mockito
> static/construction mocking only where redesign is not yet practical.
> * Introduce JUnit Jupiter with the Vintage engine, migrate module by module,
> and prohibit new JUnit 4/PowerMock tests.
> * Retain EasyMock temporarily; new tests should use the selected
> Jupiter/Mockito baseline.
> h3. 5. Upgrade or replace legacy runtime libraries
> * Handle low-risk dependency upgrades in focused commits with their existing
> unit/integration tests.
> * Migrate JAXB and Mail from {{javax}} to Jakarta packages and remove
> duplicate legacy implementations.
> * Migrate Commons Lang 2 and HttpClient 4 in module-sized batches.
> * Preserve existing c3p0 configuration keys and behavior if HikariCP is
> introduced; provide a deprecation and compatibility path rather than silently
> changing pool semantics.
> * Coordinate Hadoop, ZooKeeper, Curator, Ambari Metrics, database driver,
> LDAP, and Kerberos changes with their protocol and deployment compatibility
> matrices.
> h2. Compatibility constraints
> * Preserve the public Ambari REST API and generated OpenAPI contract unless a
> change is explicitly documented and approved.
> * Preserve existing database support and validate schema creation, upgrades,
> and runtime behavior for PostgreSQL, MySQL/MariaDB, Oracle, and SQL Server.
> * Preserve PAM, LDAP, Kerberos, SNMP, email, proxy, and HTTPS authentication
> behavior.
> * Remain on JDK 17 unless a separate project decision changes the Java
> baseline.
> * Do not combine Spring 7, Jersey milestone, Jackson 3 release candidate, or
> other unrelated major framework migrations into mechanical dependency updates.
> h2. Suggested implementation order
> # Maven/JDK baseline, repository cleanup, duplicate POM cleanup, dependency
> policy, and SBOM.
> # Logging provider correction, Servlet/Jetty alignment, EclipseLink
> convergence, and stable Spring family alignment.
> # Low-risk security and maintenance upgrades such as PostgreSQL, Commons
> libraries, Nimbus, H2, JNA/libpam4j, Mail, SNMP4J, and Testcontainers.
> # OpenAPI 3 generation, nested-resource support, specification diff, and
> maintained UI.
> # PowerMock removal and JUnit Jupiter migration.
> # Commons Lang 2, HttpClient 4, JAXB, Metrics, c3p0, and upstream-managed
> dependency migrations.
> h2. Acceptance criteria
> * A clean checkout builds with the documented Maven 3.9.x and JDK 17
> toolchain without malformed effective-POM warnings.
> * CI rejects Java versions below 17, unresolved dependency convergence,
> prohibited legacy dependencies, and unexpected OpenAPI contract changes.
> * Ambari Server packages exactly one SLF4J provider and the database
> consistency checker works without backend-specific cast failures.
> * Compile-time and runtime Servlet API levels match.
> * Swagger 1 dependencies, the disabled Swagger 1 generation configuration,
> and the obsolete bundled Swagger UI are removed.
> * OpenAPI 3 generation is reproducible and covers at least the existing 91
> paths and 164 operations unless reviewed API changes explain the difference.
> * PowerMock is removed, JUnit Jupiter is enabled, and no new JUnit 4 tests
> are introduced.
> * Legacy {{javax.xml.bind}} and JavaMail implementations are removed from
> Ambari Server after Jakarta migration.
> * Each dependency upgrade includes focused failure and recovery tests
> appropriate to its subsystem.
> * The final dependency tree, packaged runtime classpath, SBOM, and
> supported-version documentation are reviewed and committed.
> h2. Audit commands
> The following read-only analysis commands completed successfully in the local
> Ambari build container:
> {code:shell}
> mvn -pl ambari-server -DskipTests dependency:tree -Dscope=compile
> mvn -pl ambari-server -DskipTests dependency:tree -Dscope=test
> mvn -pl ambari-server -DskipTests
> org.codehaus.mojo:versions-maven-plugin:2.18.0:display-dependency-updates
> mvn -N -DskipTests
> org.codehaus.mojo:versions-maven-plugin:2.18.0:display-plugin-updates
> mvn -pl ambari-server -DskipTests dependency:build-classpath
> -Dmdep.includeScope=runtime
> {code}
> No implementation compilation or test suite is claimed by this audit. Each
> implementation commit must record its exact container build and test commands.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
