elek opened a new pull request #1627: URL: https://github.com/apache/ozone/pull/1627
## What changes were proposed in this pull request? Ozone uses Netty either as direct dependency (ozone-csi) or from the ratis shaded dependency (for ratis gprc server). Both use Netty 4.x. But netty 3 is also included in share/lib/ozone which is not required. The declared netty 3 version has security issues, we need to remove it to make it clear it's not used. (And make classpath safer) It turned out that netty (and other dependencies) came with the test-jar dependencies used from Hadoop. Based on the reference of Maven, compile time dependencies of a test dependency should be used as test dependency (https://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html) but in this case it doesn't work: ``` cd hadoop-hdds/container-service mvn dependency:tree ... [INFO] +- org.apache.hadoop:hadoop-hdfs:test-jar:tests:3.2.1:test [INFO] | +- org.eclipse.jetty:jetty-server:jar:9.4.34.v20201102:test [INFO] | | +- org.eclipse.jetty:jetty-http:jar:9.4.34.v20201102:test [INFO] | | \- org.eclipse.jetty:jetty-io:jar:9.4.34.v20201102:test [INFO] | +- org.eclipse.jetty:jetty-util-ajax:jar:9.4.34.v20201102:test [INFO] | +- com.sun.jersey:jersey-core:jar:1.19:test [INFO] | | \- javax.ws.rs:jsr311-api:jar:1.1.1:test [INFO] | +- com.sun.jersey:jersey-server:jar:1.19:test [INFO] | +- commons-cli:commons-cli:jar:1.2:compile [INFO] | +- commons-codec:commons-codec:jar:1.11:compile [INFO] | +- commons-daemon:commons-daemon:jar:1.0.13:test [INFO] | +- javax.servlet:javax.servlet-api:jar:3.1.0:test [INFO] | +- io.netty:netty:jar:3.10.5.Final:compile [INFO] | +- org.apache.htrace:htrace-core4:jar:4.1.0-incubating:compile [INFO] | \- com.fasterxml.jackson.core:jackson-databind:jar:2.10.3:compile [INFO] \- junit:junit:jar:4.11:test [INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESS [INFO] ------------------------------------------------------------------------ ... ``` Here all the dependencies of the hadop-hdfs:test-jar suppposed to have test scope. I didn't find the exact MVN issue, but found that there are multiple open issues related to transitive dependency resolution (can be the https://issues.apache.org/jira/browse/MNG-1378, but there are other open issues, too). As a result, we should remain on the same side. I ssugest: 1. Exclude ALL the TRANSITIVE test dependencies for hadoop test-jars. Hadoop test-jars can still be used, but if we need any other class, they should be requested with an explicit dependency 2. hadoop-ozone-dependency-test should be used everywhere instead of using hadoop-hdfs or hadoop-common test jars (because it includes all the required excludes ) ## What is the link to the Apache JIRA https://issues.apache.org/jira/browse/HDDS-4512 ## How was this patch tested? With full CI on the fork. If all the components are started and testable, it supposed to be OK (and it was green) ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
