Marton Elek created HDDS-4512:
---------------------------------
Summary: Remove unused netty3 transitive dependency
Key: HDDS-4512
URL: https://issues.apache.org/jira/browse/HDDS-4512
Project: Hadoop Distributed Data Store
Issue Type: Bug
Reporter: Marton Elek
Ozone uses Netty either as direct dependency (ozone-csi) or from the ratis
shaded dependency (for ratis gprc server). Both use Netty 4.x.
But netty 3 is also included in share/lib/ozone which is not required. The
declared netty 3 version has security issues, we need to remove it to make it
clear it's not used. (And make classpath safer)
It turned out that netty (and other dependencies) came with the test-jar
dependencies used from Hadoop.
Based on the reference of Maven, compile time dependencies of a test dependency
should be used as test dependency
(https://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html)
but in this case it doesn't work:
{code}
cd hadoop-hdds/container-service
mvn dependency:tree
...
[INFO] +- org.apache.hadoop:hadoop-hdfs:test-jar:tests:3.2.1:test
[INFO] | +- org.eclipse.jetty:jetty-server:jar:9.4.34.v20201102:test
[INFO] | | +- org.eclipse.jetty:jetty-http:jar:9.4.34.v20201102:test
[INFO] | | \- org.eclipse.jetty:jetty-io:jar:9.4.34.v20201102:test
[INFO] | +- org.eclipse.jetty:jetty-util-ajax:jar:9.4.34.v20201102:test
[INFO] | +- com.sun.jersey:jersey-core:jar:1.19:test
[INFO] | | \- javax.ws.rs:jsr311-api:jar:1.1.1:test
[INFO] | +- com.sun.jersey:jersey-server:jar:1.19:test
[INFO] | +- commons-cli:commons-cli:jar:1.2:compile
[INFO] | +- commons-codec:commons-codec:jar:1.11:compile
[INFO] | +- commons-daemon:commons-daemon:jar:1.0.13:test
[INFO] | +- javax.servlet:javax.servlet-api:jar:3.1.0:test
[INFO] | +- io.netty:netty:jar:3.10.5.Final:compile
[INFO] | +- org.apache.htrace:htrace-core4:jar:4.1.0-incubating:compile
[INFO] | \- com.fasterxml.jackson.core:jackson-databind:jar:2.10.3:compile
[INFO] \- junit:junit:jar:4.11:test
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
...
{code}
Here all the dependencies of the hadop-hdfs:test-jar suppposed to have test
scope.
I didn't find the exact MVN issue, but found that there are multiple open
issues related to transitive dependency resolution (can be the
https://issues.apache.org/jira/browse/MNG-1378, but there are other open
issues, too).
As a result, we should remain on the same side. I ssugest:
1. Exclude ALL the TRANSITIVE test dependencies for hadoop test-jars. Hadoop
test-jars can still be used, but if we need any other class, they should be
requested with an explicit dependency
2. hadoop-ozone-dependency-test should be used everywhere instead of using
hadoop-hdfs or hadoop-common test jars (because it includes all the required
excludes ;-) )
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
