[
https://issues.apache.org/jira/browse/HADOOP-1542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Owen O'Malley updated HADOOP-1542:
----------------------------------
Fix Version/s: 0.14.0
Assignee: Owen O'Malley (was: Devaraj Das)
Description:
The change in HADOOP-1440 broke map/reduce by breaking the assumption that
Task.getPartition() corresponded to the JobInProgress.map[] order.
Currently JobInProgress.findNewTask uses Task.getPartition as the index of the
map to run. This can be a completely different tip, which will cause incorrect
tasks to be run, including duplicates of tasks that are already running.
was:
Speculative execution is now on by default. When running TestDFSIO, I set
speculative execution off in my mapred-default.xml since this test has maps
that create files in DFS (side-effects).
However, it seems that speculative tasks get started even though I have set
speculation off. I'll attached the NN and JT logs.
Affects Version/s: 0.14.0
Summary: Incorrect task/tip being scheduled (looks like
speculative execution) (was: Speculative execution used when property set to
false)
> Incorrect task/tip being scheduled (looks like speculative execution)
> ---------------------------------------------------------------------
>
> Key: HADOOP-1542
> URL: https://issues.apache.org/jira/browse/HADOOP-1542
> Project: Hadoop
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.14.0
> Reporter: Nigel Daley
> Assignee: Owen O'Malley
> Priority: Blocker
> Fix For: 0.14.0
>
> Attachments: jobtracker.log, namenode.log
>
>
> The change in HADOOP-1440 broke map/reduce by breaking the assumption that
> Task.getPartition() corresponded to the JobInProgress.map[] order.
> Currently JobInProgress.findNewTask uses Task.getPartition as the index of
> the map to run. This can be a completely different tip, which will cause
> incorrect tasks to be run, including duplicates of tasks that are already
> running.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
