[
https://issues.apache.org/jira/browse/MAPREDUCE-1938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12888482#action_12888482
]
Doug Cutting commented on MAPREDUCE-1938:
-----------------------------------------
Owen, I agree with your analysis. I'm just trying to put this patch in context
of these other related discussions.
This patch addresses some issues relevant to separation of kernel & library.
In common cases one can merely provide an alternate version of the library
class in one's job. Fully separating kernel & library with a well-defined,
minimal kernel API is clearly aesthetically better. Are there use cases that
will that enable that this patch will not? I think mostly it will just make it
clear which classes are safe to replace with updated versions and which are
not. Does that sound right?
The issue of user versions of libraries that the kernel uses (like Avro, log4j,
HttpClient, etc.) is not entirely addressed by this patch. If the user's
version is backwards compatible with the kernel's version then this patch is
sufficient. But if the user's version of a library makes incompatible changes
then we'd need a classloader/OSGI solution. Even then, I think it only works
if user and kernel code do not interchange instances of classes defined by
these libraries. A minimal kernel API will help reduce that risk. Does this
analysis sound right?
I'm trying to understand how far this patch gets us towards those goals: what
it solves and what it doesn't.
> Ability for having user's classes take precedence over the system classes for
> tasks' classpath
> ----------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-1938
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1938
> Project: Hadoop Map/Reduce
> Issue Type: New Feature
> Components: job submission, task, tasktracker
> Reporter: Devaraj Das
> Fix For: 0.22.0
>
> Attachments: mr-1938-bp20.patch
>
>
> It would be nice to have the ability in MapReduce to allow users to specify
> for their jobs alternate implementations of classes that are already defined
> in the MapReduce libraries. For example, an alternate implementation for
> CombineFileInputFormat.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
