[
https://issues.apache.org/jira/browse/MAPREDUCE-1938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12972409#action_12972409
]
Alejandro Abdelnur commented on MAPREDUCE-1938:
-----------------------------------------------
Mahadev,
I'm not working on a patch for this at the moment. I wouldn't mind (I'll have
to allocate some time for it), but before doing so, I need a clear cut on the
APIs that are interface, and they should be in their own JAR file, separate
from the JAR file with the implementation classes.
Regarding your comment on 'task code does not load that many libraries', I'm
not sure is a valid statement if you are doing Pig or Hive, just check how many
more JARs they bring into the classpath, and then check of those JARs how many
of them overlap with Hadoop JARs, and then check their versions.
Owen,
Users get in trouble without making conflicts in a deliberate way, they get
into those conflicts because of the lack of isolation of the current model
(refer to second paragraph of answer to Mahadev).
Having the use in control as the default is, IMO, the wrong way to go, all the
testing you've done on a release becomes meaningless as the users classpath
will override any library used by the cluster by default.
My take is that we should fix this via classloader isolation from the get go.
If folks consider this to be a TOO big task (IMO, it will aways be) and the
user&system classpath swap is a stop gap alternative, my take is that we should
for a default of system-user classpath.
I personally see this as a big issue (using Hadoop from Oozie means that I have
to deal with libraries from Hadoop/Pig/Hive/Sqoop/Hbase in the classpath).
> Ability for having user's classes take precedence over the system classes for
> tasks' classpath
> ----------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-1938
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1938
> Project: Hadoop Map/Reduce
> Issue Type: New Feature
> Components: job submission, task, tasktracker
> Reporter: Devaraj Das
> Assignee: Krishna Ramachandran
> Priority: Blocker
> Fix For: 0.22.0
>
> Attachments: mapred-1938-1, mapred-1938-2.patch, mapred-1938-3.patch,
> mr-1938-bp20.1.patch, mr-1938-bp20.patch
>
>
> It would be nice to have the ability in MapReduce to allow users to specify
> for their jobs alternate implementations of classes that are already defined
> in the MapReduce libraries. For example, an alternate implementation for
> CombineFileInputFormat.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
