Hello Everyone,
Can someone help me explain what are the numbers next to Map 1 / Map 2 and
Reducer 3 .
~~~
Status: Running (application id: application_1404180111945_438880)
Map 1: -/- Map 2: -/- Reducer 3: 0/1
Map 1: 0/2 Map 2: -/- Reducer 3: 0/1
Map 1: 0/2 Map 2: 0/8 Reducer 3: 0/1
Hi
This looks like a 3-vertex DAG. It could be possibly be a linear DAG such as
Map1 -> Map2 -> Reduce3 or a Join DAG where
Map1 -> Reduce3 and Map2 -> Reduce3.
If you can get the application logs from YARN ( using bin/yarn logs
-applicationId application_1404180111945_438880 ), you will be a
Thanks Hitesh. That explains the DAG.
When you said completed vs total tasks for a given vertex, does it mean,
there was a total of 0/2 + 0/8 = 0/10 ( 10 tasks ) for this tez job.
Which means, when i ran the same query in hive MR, it launched 16 tasks and
now it is launching only 10 tasks. Also,
There are multiple reasons for Tez having different no. of tasks:
- Hive itself will behave differently. With MR, it may be have been
processing data from 2 tables in the same map stage which affects no. of tasks.
For Tez, it may end up processing each table in a separate vertex.
- Tez do
