[
https://issues.apache.org/jira/browse/MAPREDUCE-6128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14211295#comment-14211295
]
Jason Lowe commented on MAPREDUCE-6128:
---------------------------------------
Thanks for updating the patch, Gera.
The application classloader feature with '$' should be a separate JIRA and
deserving of its own unit test. The filter test classes feature also seems
separate (although I could see the two classloader features done in
conjunction). But do we really need the test filter feature? Seems like a
hack to work around a very specific case with the minicluster. Can we
configure a minicluster to run without the jobclient tests in the classpath
instead?
Speaking of the '$' behavior, would it be better if users simply listed the
class and internally isSystemClass checked not only for equality but also
starting with the classname + '$'? Trying to think of when a user would ever
want a particular class to be a system class but _not_ have the inner classes
of that class loaded from the same place. In other words, if I'm listing a
specific class then when wouldn't I want to tack on the '$'?
JobSubmitter is only checking files specified by -libjars for conflicts, but it
needs to check all other distributed cache files. $PWD/* is added to the
classpath, so one doesn't need to use -libjars to add a jar, and a distributed
cache conflict is bad even if the file isn't in the classpath. Also need to
watch out for renames during localization (i.e.: the fragment of the URI).
USE_MANIFEST_CLASSES should be JOB_USE_MANIFEST_CLASSES and it should be
mapreduce.job.include.manifest.classpath since this is a job-level concept not
a task-level concept. Also USE_MANIFEST_CLASSES should be
JOB_USE_MANIFEST_CLASSES or JOB_INCLUDE_MANIFEST_CLASSES.
Nit: USE_MANIFEST_CLASSES_DEFAULT should be grouped next to the property it
defaults, whitespace-wise, rather than something else.
"names ending with a '$' are treated as inner classes" is a bit confusing.
What does it mean to treat it as an inner class? Probably clearer to state
that any inner classes of the specified class are also included.
> Automatic addition of bundled jars to distributed cache
> --------------------------------------------------------
>
> Key: MAPREDUCE-6128
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6128
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: client
> Affects Versions: 2.5.1
> Reporter: Gera Shegalov
> Assignee: Gera Shegalov
> Attachments: MAPREDUCE-6128.v01.patch, MAPREDUCE-6128.v02.patch,
> MAPREDUCE-6128.v03.patch, MAPREDUCE-6128.v04.patch, MAPREDUCE-6128.v05.patch
>
>
> On the client side, JDK adds Class-Path elements from the job jar manifest
> on the classpath. In theory there could be many bundled jars in many
> directories such that adding them manually via libjars or similar means to
> task classpaths is cumbersome. If this property is enabled, the same jars are
> added
> to the task classpaths automatically.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
