[
https://issues.apache.org/jira/browse/HBASE-20332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated HBASE-20332:
--------------------------------
Resolution: Fixed
Release Note:
<!-- markdown -->
The `hbase-shaded-mapreduce` artifact no longer include its own copy of Hadoop
classes. Users who make use of the artifact via YARN should be able to get
these classes from YARN's classpath without having to make any changes.
Status: Resolved (was: Patch Available)
pushed to master and branch-2. I missed signed-off-by lines on master. :/
> shaded mapreduce module shouldn't include hadoop
> ------------------------------------------------
>
> Key: HBASE-20332
> URL: https://issues.apache.org/jira/browse/HBASE-20332
> Project: HBase
> Issue Type: Sub-task
> Components: mapreduce, shading
> Affects Versions: 2.0.0
> Reporter: Sean Busbey
> Assignee: Sean Busbey
> Priority: Critical
> Fix For: 3.0.0, 2.1.0
>
> Attachments: HBASE-20332.0.patch, HBASE-20332.1.WIP.patch,
> HBASE-20332.2.WIP.patch, HBASE-20332.3.patch, HBASE-20332.4.patch,
> HBASE-20332.5.patch, HBASE-20332.6.patch, HBASE-20332.7.patch
>
>
> AFAICT, we should just entirely skip including hadoop in our shaded mapreduce
> module
> 1) Folks expect to run yarn / mr apps via {{hadoop jar}} / {{yarn jar}}
> 2) those commands include all the needed Hadoop jars in your classpath by
> default (both client side and in the containers)
> 3) If you try to use "user classpath first" for your job as a workaround
> (e.g. for some library your application needs that hadoop provides) then our
> inclusion of *some but not all* hadoop classes then causes everything to fall
> over because of mixing rewritten and non-rewritten hadoop classes
> 4) if you don't use "user classpath first" then all of our
> non-relocated-but-still-shaded hadoop classes are ignored anyways so we're
> just wasting space
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
