[
https://issues.apache.org/jira/browse/MAPREDUCE-1905?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12895995#action_12895995
]
Vinod K V commented on MAPREDUCE-1905:
--------------------------------------
Functionality wise, the patch fixes the current bug and looks fine.
But I sense something wrong. The new TaskAttemptContextImpl doesn't know about
reporter even though the old org.apache.hadoop.mapred.TaskAttemptContextImpl
knows about it.
- progress() calls are made throughout, but they may be getting reported
because either the calls are made on the the old context object (in Task) or
directly on the old Reporter class (in Shuffle etc). So, no bug here as of now,
but once we try switching to new API context objects in the implementation of
Task etc, we may encounter new bugs.
- setStatus() on the new context object are made in several RecordReaders -
from the code, it seems these status updates will not get reported because of
the lack of reporter.
- New Mapper/Reducer classes also make setStatus() calls but on
TaskInputOutputContextImpl which, though knows about the reporter, doesn't
override the method from TaskAttemptContextImpl. This will be fixed by the
current patch.
May be, then, the correct and complete solution is to let
TaskAttemptContextImpl know about reporter (via constructor), do setStatus()
and progress() in TaskAttemptContextImpl itself and not override them in
TaskInputOutputContextImpl? This will be slightly more changes, so we can
confirm from someone who knows this area well.
> Context.setStatus() and progress() api are ignored
> --------------------------------------------------
>
> Key: MAPREDUCE-1905
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1905
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: task
> Affects Versions: 0.21.0
> Reporter: Amareshwari Sriramadasu
> Assignee: Amareshwari Sriramadasu
> Priority: Blocker
> Attachments: patch-1905.txt
>
>
> TaskAttemptContext.setStatus() and progress() were overriden in
> TaskInputOutputContext, inbranch 0.20, to call the underlying reporter apis.
> But the methods are no more over-riden in TaskInputOutputContextImpl after
> MAPREDUCE-954.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
