Yes, we mean the ATS APIs. What Tez API did you refer to? Are there additional REST APIs to get the Tez info directly?
Xiaoyong From: Jianfeng (Jeff) Zhang [mailto:[email protected]] Sent: Friday, April 3, 2015 5:29 PM To: [email protected] Cc: Xiaoyong Zhu; Yifung Lin; HDInsight VS Tooling V-team Subject: Re: Tez Vertex Info analysis Hi Joe, What do you mean tez REST API ? Do you mean you get these info through the ATS (application timeline service ) ? <1> what does " numTasks" contains ? what are relationships among numTasks, numberCompletedTasks, numKilledTasks , numFailedTasks, numSucceededTasks ? numTasks means the number of tasks of this vertex should run. If you have some knowledge of map reduce, you can think of this as the number of tasks in mapper/reducer. numberCompletedTasks means the number of task that has finished. There're 3 possible states for task finishing: Succeeded/Failed/Killed. So that means numberCompletedTasks should be equals to the sum of numKilledTasks , numFailedTasks, numSucceededTasks <2> why does not contains running tasks? How can get the number of running task and not started tasks? If you get the data from ATS, then it is not possible to get the running tasks number. Because these data are pushed to ATS when the vertex is completed which means there's no running tasks at that time. But you can use the Tez API to get the status of running vertex where you can get the number of running tasks. <3> whether any task in this vertex failed will called the whole vertex failed?? Yes, vertex only succeeded when no task is failed/killed. That means numberCompletedTasks should be equal to numSucceededTasks if the vertex is succeeded. Best Regard, Jeff Zhang From: "Joe Zhang (SDE)" <[email protected]<mailto:[email protected]>> Reply-To: "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>> Date: Friday, April 3, 2015 at 3:30 PM To: "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>> Cc: Xiaoyong Zhu <[email protected]<mailto:[email protected]>>, Yifung Lin <[email protected]<mailto:[email protected]>>, HDInsight VS Tooling V-team <[email protected]<mailto:[email protected]>> Subject: Tez Vertex Info analysis Hi Tez experts: I am using tez REST API to analysis vertex running information, below is what I get . but I am wandering some concepts <1> what does " numTasks" contains ? what are relationships among numTasks, numberCompletedTasks, numKilledTasks , numFailedTasks, numSucceededTasks ? <2> why does not contains running tasks? How can get the number of running task and not started tasks? <3> whether any task in this vertex failed will called the whole vertex failed?? [cid:[email protected]] Best wishes Joe zhang
