[
https://issues.apache.org/jira/browse/MAPREDUCE-2584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13049195#comment-13049195
]
Harsh J commented on MAPREDUCE-2584:
------------------------------------
{{From Lists, think its more appropriate here}}
On Tue, Jun 14, 2011 at 6:19 PM, Sudharsan Sampath
<sudharsan.sampath-at-ericsson-dot-com> wrote:
> Just my thoughts...
>
> To me throwing a specific exception would be better. The checkSerializerSpecs
> attempts to see if we can initialise a new instance of the
> serializer/deserializer from the jvm where the job is submitted. How does it
> guarantee that the libs/jars from which these classes are loaded are
> available for the jvm that executes the job or vice versa? Even if this
> methods succeeds isn't there a chance then the original problem might occur
> due to the libs missing from the actual jvm that executes the job?
What you describe about the JVM can surely occur and I've pushed some handling
at the MapOutputBuffer level as well but in any case its better to catch it as
early as possible.
Don't think guaranteeing that the client is indeed packing things along with
the job for the DC is practically possible, but at least it is an improvement
to where the client follows a bit of general guidelines I think?
> Check for serializers early, and give out more information regarding missing
> serializers
> ----------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-2584
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2584
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: task
> Affects Versions: 0.20.2
> Reporter: Harsh J
> Assignee: Harsh J
> Labels: serializers, tasks
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-2584.r3.diff
>
>
> As discussed on HADOOP-7328, MapReduce can handle serializers in a much
> better way in case of bad configuration, improper imports (Some odd Text
> class instead of the Writable Text set as key), etc..
> This issue covers the MapReduce parts of the improvements (made to IFile,
> MapOutputBuffer, etc. and possible early-check of serializer availability
> pre-submit) that provide more information than just an NPE as is the current
> case.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
