abstractdog commented on PR #419:
URL: https://github.com/apache/tez/pull/419#issuecomment-3026984646
> Thanx @abstractdog for the details. I think maybe the problem is we are
forcing the scope to test in `dependencyManagement` of the parent pom. I
believe if you just remove the scope from the parent pom, that should do. The
other modules can have it in test scope.
>
> Moreover the version of bouncycastle from hadoop seems to be 1.78.1 where
in Tez it is 1.78. Maybe we should keep them in sync.
>
> I believe something like this might just do
>
> ```
> diff --git a/pom.xml b/pom.xml
> index 8dfdec9ec..d1031c6c4 100644
> --- a/pom.xml
> +++ b/pom.xml
> @@ -58,7 +58,7 @@
>
> <!--dependency versions in alphabetical order-->
> <asynchttpclient.version>2.12.4</asynchttpclient.version>
> - <bouncycastle.version>1.78</bouncycastle.version>
> + <bouncycastle.version>1.78.1</bouncycastle.version>
>
<build-helper-maven-plugin.version>1.8</build-helper-maven-plugin.version>
>
<buildnumber-maven-plugin.version>1.1</buildnumber-maven-plugin.version>
> <checkstyle.version>8.35</checkstyle.version>
> @@ -791,13 +791,11 @@
> <groupId>org.bouncycastle</groupId>
> <artifactId>bcprov-jdk18on</artifactId>
> <version>${bouncycastle.version}</version>
> - <scope>test</scope>
> </dependency>
> <dependency>
> <groupId>org.bouncycastle</groupId>
> <artifactId>bcpkix-jdk18on</artifactId>
> <version>${bouncycastle.version}</version>
> - <scope>test</scope>
> </dependency>
> <dependency>
> <groupId>org.fusesource.leveldbjni</groupId>
> ```
>
> What changed, and why was it working earlier?
>
> I have a theory related to
[HADOOP-19024](https://issues.apache.org/jira/browse/HADOOP-19024), which was
introduced in Hadoop 3.4.1. Previously, Hadoop depended on `bcprov-jdk15on`,
whereas Tez declared `bcprov-jdk18on` in its dependencyManagement. Since Tez
did not explicitly declare `bcprov-jdk15on`, it was being pulled transitively
from Hadoop with its default (compile) scope, so it ended up being packaged
correctly.
>
> However, after
[HADOOP-19024](https://issues.apache.org/jira/browse/HADOOP-19024), Hadoop
itself now declares `bcprov-jdk18on`, which conflicts with Tez’s declaration
that forces its scope to test. As a result, the dependency is omitted from the
final package.
thanks @ayushtkn , absolutely makes sense
what's weird is that upstream tez, the fix works also without the version
harmonization, so only changing the root pom.xml
however, I need to consider 2 things:
1. version harmonization makes sense, I'll most probably add it
2. this fix doesn't work downstream, only if I change the tez-api pom.xml
too...I need to understand the difference before proceeding here
I'll keep you posted
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
