Hi Joe,
What do you mean tez REST API ? Do you mean you get these info through the ATS
(application timeline service ) ?
<1> what does " numTasks" contains ? what are relationships among numTasks,
numberCompletedTasks, numKilledTasks , numFailedTasks, numSucceededTasks ?
numTasks means the number of tasks of this vertex should run. If you
have some knowledge of map reduce, you can think of this as the number of tasks
in mapper/reducer.
numberCompletedTasks means the number of task that has finished.
There're 3 possible states for task finishing: Succeeded/Failed/Killed. So
that means numberCompletedTasks should be equals to the sum of numKilledTasks ,
numFailedTasks, numSucceededTasks
<2> why does not contains running tasks? How can get the number of running task
and not started tasks?
If you get the data from ATS, then it is not possible to get the running
tasks number. Because these data are pushed to ATS when the vertex is completed
which means there's no running tasks at that time.
But you can use the Tez API to get the status of running vertex where
you can get the number of running tasks.
<3> whether any task in this vertex failed will called the whole vertex failed??
Yes, vertex only succeeded when no task is failed/killed. That means
numberCompletedTasks should be equal to numSucceededTasks if the vertex is
succeeded.
Best Regard,
Jeff Zhang
From: "Joe Zhang (SDE)" <[email protected]<mailto:[email protected]>>
Reply-To: "[email protected]<mailto:[email protected]>"
<[email protected]<mailto:[email protected]>>
Date: Friday, April 3, 2015 at 3:30 PM
To: "[email protected]<mailto:[email protected]>"
<[email protected]<mailto:[email protected]>>
Cc: Xiaoyong Zhu <[email protected]<mailto:[email protected]>>,
Yifung Lin <[email protected]<mailto:[email protected]>>, HDInsight VS
Tooling V-team <[email protected]<mailto:[email protected]>>
Subject: Tez Vertex Info analysis
Hi Tez experts:
I am using tez REST API to analysis vertex running information, below is
what I get . but I am wandering some concepts
<1> what does " numTasks" contains ? what are relationships among numTasks,
numberCompletedTasks, numKilledTasks , numFailedTasks, numSucceededTasks ?
<2> why does not contains running tasks? How can get the number of running task
and not started tasks?
<3> whether any task in this vertex failed will called the whole vertex failed??
[cid:[email protected]]
Best wishes
Joe zhang
