[
https://issues.apache.org/jira/browse/MAPREDUCE-964?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12758171#action_12758171
]
Sreekanth Ramakrishnan commented on MAPREDUCE-964:
--------------------------------------------------
The reason for the very high map slot seconds is because of following senario,
We hand out of a task to tracker, tracker gets lost, then we kill the task, as
TT has not reported back during lost period our task start time is not updated
according to JT the task has not been launched and the time and then we fail
the task and the task has a finish time but not start time.
Following is log from JT:
{noformat}
2009-09-18 13:28:20,990 INFO org.apache.hadoop.mapred.JobTracker: Adding task
'attempt_200909180756_0009_m_000003_2' to tip task_200909180756_0009_m_000003,
for tracker 'tracker_xxx:localhost/127.0.0.1:45507'
2009-09-18 13:39:51,482 INFO org.apache.hadoop.mapred.TaskInProgress: Error
from attempt_200909180756_0009_m_000003_2: Lost task tracker:
tracker_xxx:localhost/127.0.0.1:45507
2009-09-18 13:39:56,997 INFO org.apache.hadoop.mapred.JobTracker: Adding task
(cleanup)'attempt_200909180756_0009_m_000003_2' to tip
task_200909180756_0009_m_000003, for tracker
'tracker_xxx:localhost/127.0.0.1:60187'
2009-09-18 13:40:00,000 INFO org.apache.hadoop.mapred.JobInProgress: TaskDebug
attemptId : attempt_200909180756_0009_m_000003_2 slots : SLOTS_MILLIS_MAPS
tip.numslots is: 1 difference to add : 1253281197003 status start : 0 status
end time : 1253281197003
2009-09-18 13:40:00,000 INFO org.apache.hadoop.mapred.JobTracker: Removed
completed task 'attempt_200909180756_0009_m_000003_2' from
'tracker_xxx:localhost/127.0.0.1:60187'
{noformat}
> Inaccurate values in jobSummary logs
> ------------------------------------
>
> Key: MAPREDUCE-964
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-964
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Affects Versions: 0.20.1
> Reporter: Rajiv Chittajallu
> Assignee: Sreekanth Ramakrishnan
> Priority: Critical
>
> For some jobs the mapSlotSeconds is incorrect.
> negative value
> 09/09/01 18:31:44 INFOmapred.JobInProgress$JobSummary:
> jobId=job_200908270718_4568,submitTime=1251823543976,launchTime=1251823554310,finishTime=1251829904565,
>
> numMaps=7965,numSlotsPerMap=1,numReduces=40,numSlotsPerReduce=1,user=wile,queue=runner,status=SUCCEEDED,
>
> mapSlotSeconds=-2503133523,reduceSlotsSeconds=186536,clusterMapCapacity=11262,clusterReduceCapacity=3754
> or too high
> 09/09/02 23:59:57 INFO mapred.JobInProgress$JobSummary:
> jobId=job_200908270718_5861,submitTime=1251935672924,launchTime=1251935687698,finishTime=1251935997949,
>
> numMaps=1026,numSlotsPerMap=1,numReduces=10,numSlotsPerReduce=1,user=dfsload,queue=gridops,status=SUCCEEDED,
>
> mapSlotSeconds=1251949742,reduceSlotsSeconds=537,clusterMapCapacity=11262,clusterReduceCapacity=3754
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
