> On June 3, 2013, 5:36 p.m., Ben Mahler wrote:
> > What differences were you seeing between job.pendingMaps() and this new
> > technique?
> >
> > Looking at JobInProgress.pendingMaps():
> >
> > public synchronized int pendingMaps() {
> > return numMapTasks - runningMapTasks - failedMapTIPs -
> > finishedMapTasks + speculativeMapTasks;
> > }
> >
> > vs. jobdetails_jsp.printTaskSummary() {
> > int totalTasks = tasks.length;
> > int runningTasks = 0;
> > int finishedTasks = 0;
> > int killedTasks = 0;
> > int failedTaskAttempts = 0;
> > int killedTaskAttempts = 0;
> > for(int i=0; i < totalTasks; ++i) {
> > TaskInProgress task = tasks[i];
> > if (task.isComplete()) {
> > finishedTasks += 1;
> > } else if (task.isRunning()) {
> > runningTasks += 1;
> > } else if (task.wasKilled()) {
> > killedTasks += 1;
> > }
> > failedTaskAttempts += task.numTaskFailures();
> > killedTaskAttempts += task.numKilledTasks();
> > }
> > int pendingTasks = totalTasks - runningTasks - killedTasks -
> > finishedTasks;
> > ...
> > }
> >
> > It seems like the difference here might be between failed_ vs. killed_
> > and/or the fact that the latter case uses speculativeMapTasks?
Yes, possibly. To be honest I don't remember, it's been months since I fixed
this. All I remember specifically is that it was wrong.
- Brenden
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/11116/#review21326
-----------------------------------------------------------
On May 22, 2013, 7:49 p.m., Brenden Matthews wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/11116/
> -----------------------------------------------------------
>
> (Updated May 22, 2013, 7:49 p.m.)
>
>
> Review request for mesos.
>
>
> Description
> -------
>
> From dc2c7f4116dec16bf0e22ab4c9fccac8f3a45020 Mon Sep 17 00:00:00 2001
> From: Brenden Matthews <[email protected]>
> Date: Mon, 13 May 2013 15:27:45 -0700
> Subject: [PATCH 09/24] Fix TaskTracker pending tasks calculation.
>
> Review: https://reviews.apache.org/r/11116
> ---
> .../org/apache/hadoop/mapred/MesosScheduler.java | 33
> ++++++++++++++++++--
> 1 file changed, 31 insertions(+), 2 deletions(-)
>
>
> Diffs
> -----
>
> hadoop/mesos/src/java/org/apache/hadoop/mapred/MesosScheduler.java
> afe401f5265e3d9494af7eace42eec45943184a3
>
> Diff: https://reviews.apache.org/r/11116/diff/
>
>
> Testing
> -------
>
> Used in production at airbnb.
>
>
> Thanks,
>
> Brenden Matthews
>
>
