[
https://issues.apache.org/jira/browse/MAPREDUCE-4647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13452105#comment-13452105
]
Robert Joseph Evans commented on MAPREDUCE-4647:
------------------------------------------------
I think that it is a regression. It looks like Yarn/MRV2 changed the handling
of job.jar to be through the distributed cache. Originally we only added the
jar to the classpath as a cache file. MAPREDUCE-4068 then changed it to be a
cache archive so that the classes and lib directories could be added to the
classpath. So both of those together effectively undid MAPREDUCE-967. I am
not sure how simple it will be to try and fully implement MAPREUDCE-967 again,
because the job.jar is going through the distributed cache and this really is
behavior that seems to be MAPREDUCE specific, unless we want to some how try
and make it generic for everyone on YARN.
That is why I wanted to have the client look at the jar and decided if it
should be a cache file, or a cache archive. From what I have seen very few
jars actually rely on this behavior. If you would prefer to make it generic
for YARN I can look into that instead. It should not be too hard, but it might
require a small change to the container launch context.
> We should only unjar jobjar if there is a lib directory in it.
> --------------------------------------------------------------
>
> Key: MAPREDUCE-4647
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4647
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.3
> Reporter: Robert Joseph Evans
> Assignee: Robert Joseph Evans
>
> For backwards compatibility we recently added made is so we would unjar the
> job.jar and add anything to the classpath in the lib directory of that jar.
> But this also slows job startup down a lot if the jar is large. We should
> only unjar it if actually doing so would add something new to the classpath.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
