[
https://issues.apache.org/jira/browse/HADOOP-14284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15996429#comment-15996429
]
Tsuyoshi Ozawa commented on HADOOP-14284:
-----------------------------------------
[~vinodkv] and [~djp], thanks a lot for your feedback.
> A couple of questions: can we just shade some client jars instead of
> everywhere?
> As we keep doing this for other libraries, I'm concerned if our code becomes
> more brittle (changing imports everywhere) and if the build times explode.
We're now doing to shade Guava and Curator in hadoop-shaded-thirdparty, and
trying to import it from hadoop-* projects. The build time of Hadoop doesn't
get increased so much with this approach because of just referring to
hadoop-shaded-thirdparty project from hadoop-* projects. However, I found one
problem in this approach: shaded artifacts(shaded Guava and Curator) in
hadoop-shaded-thirdparty is NOT in classpath, if I understand correctly. To go
with this approach, we need to unzip source code and compile it like HBase does
in hbase-protocol-shaded. This can make Hadoop build fragile and the build time
of Hadoop can increase as Junping Vinod mentioned.
https://github.com/apache/hbase/blob/7700a7fac1262934fe538a96b040793c6ff171ce/hbase-protocol-shaded/pom.xml#L321
Gradle seems to have a feature to do this.
http://stackoverflow.com/questions/26244936/how-to-include-only-project-and-relocated-classes-when-using-gradle-shadow-plugi
> Isn't it better to just shade our final artifacts instead of shading
> individual libraries' jars?
Do you mean that we prepare new project "hadoop-server-modules" and shading
Guava and Curator inside them like hadoop-client-modules? It sounds better
approach to me. By adding skipShade option here, we can overcome build time
problem. [~andrew.wang] [~busbey] [~ajisakaa] What do you think?
> Shade Guava everywhere
> ----------------------
>
> Key: HADOOP-14284
> URL: https://issues.apache.org/jira/browse/HADOOP-14284
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
> Affects Versions: 3.0.0-alpha3
> Reporter: Andrew Wang
> Assignee: Tsuyoshi Ozawa
> Priority: Blocker
> Attachments: HADOOP-14238.pre001.patch, HADOOP-14284.002.patch,
> HADOOP-14284.004.patch, HADOOP-14284.007.patch, HADOOP-14284.010.patch
>
>
> HADOOP-10101 upgraded the guava version for 3.x to 21.
> Guava is broadly used by Java projects that consume our artifacts.
> Unfortunately, these projects also consume our private artifacts like
> {{hadoop-hdfs}}. They also are unlikely on the new shaded client introduced
> by HADOOP-11804, currently only available in 3.0.0-alpha2.
> We should shade Guava everywhere to proactively avoid breaking downstreams.
> This isn't a requirement for all dependency upgrades, but it's necessary for
> known-bad dependencies like Guava.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
