Hi all: Maybe in last mail the attached picture is not shown. I re-described my question here. I saw following statistics about the runtime when running query.
The Run DAG is 318s. But it is not the sum of DURATION of all VERTICES((59549+4069+3055+3055+1004+1006+132736+34248+11077+1003+439+140896+35260+8070)/1000=435s Not the sum of CPU_TIME. There are several indicator "RUN DAG","DURATION","CPU_TIME", which indicator I should use when measure the performance? Sometimes I found there is significant improvement in sum of (CPU_TIME) while there is no significant improvement in "RUN DAG". Is this normal? Appreciate to get some feedback from you! 2017-10-12T16:29:39,262 INFO [main] SessionState: ---------------------------------------------------------------------------------------------- 2017-10-12T16:29:39,262 INFO [main] SessionState: OPERATION DURATION 2017-10-12T16:29:39,262 INFO [main] SessionState: ---------------------------------------------------------------------------------------------- 2017-10-12T16:29:39,263 INFO [main] SessionState: Compile Query 3.72s 2017-10-12T16:29:39,263 INFO [main] SessionState: Prepare Plan 0.60s 2017-10-12T16:29:39,263 INFO [main] SessionState: Submit Plan 0.61s 2017-10-12T16:29:39,263 INFO [main] SessionState: Start DAG 0.52s 2017-10-12T16:29:39,263 INFO [main] SessionState: Run DAG 318.54s 2017-10-12T16:29:39,263 INFO [main] SessionState: ---------------------------------------------------------------------------------------------- 2017-10-12T16:29:39,263 INFO [main] SessionState: 2017-10-12T16:29:39,289 INFO [cea2258c-aa47-46a1-af5b-39860a6edbb3 main] counters.Limits: Counter limits initialized with parameters: GROUP_NAME_MAX=256, MAX_GROUPS=500, COUNTER_NAME_MAX=64, MAX_COUNTERS=2000 2017-10-12T16:29:39,294 INFO [main] SessionState: Task Execution Summary 2017-10-12T16:29:39,294 INFO [main] SessionState: ---------------------------------------------------------------------------------------------- 2017-10-12T16:29:39,294 INFO [main] SessionState: VERTICES DURATION(ms) CPU_TIME(ms) GC_TIME(ms) INPUT_RECORDS OUTPUT_RECORDS 2017-10-12T16:29:39,294 INFO [main] SessionState: ---------------------------------------------------------------------------------------------- 2017-10-12T16:29:39,298 INFO [main] SessionState: Map 1 59549.00 1,355,520 28,565 550,076,554 1,602,119,842 2017-10-12T16:29:39,300 INFO [main] SessionState: Map 12 4069.00 15,670 522 73,049 732 2017-10-12T16:29:39,300 INFO [main] SessionState: Map 13 3055.00 14,030 567 212 212 2017-10-12T16:29:39,301 INFO [main] SessionState: Map 14 3055.00 13,820 606 212 212 2017-10-12T16:29:39,303 INFO [main] SessionState: Reducer 10 1004.00 13,450 265 4 4 2017-10-12T16:29:39,305 INFO [main] SessionState: Reducer 11 1006.00 4,290 71 216 212 2017-10-12T16:29:39,307 INFO [main] SessionState: Reducer 2 132736.00 2,362,160 83,029 537,120,745 107,740,258 2017-10-12T16:29:39,308 INFO [main] SessionState: Reducer 3 34248.00 643,350 20,661 107,740,470 203 2017-10-12T16:29:39,310 INFO [main] SessionState: Reducer 4 11077.00 77,020 1,496 203 31 2017-10-12T16:29:39,311 INFO [main] SessionState: Reducer 5 1003.00 40,030 824 10 10 2017-10-12T16:29:39,312 INFO [main] SessionState: Reducer 6 439.00 590 0 10 0 2017-10-12T16:29:39,314 INFO [main] SessionState: Reducer 7 140896.00 1,925,760 52,784 537,120,745 107,740,258 2017-10-12T16:29:39,316 INFO [main] SessionState: Reducer 8 35260.00 590,200 22,331 107,740,470 76 2017-10-12T16:29:39,318 INFO [main] SessionState: Reducer 9 8070.00 24,630 249 76 4 2017-10-12T16:29:39,318 INFO [main] SessionState: ------------------------------------------------------------------- From: Zhang, Liyun [mailto:liyun.zh...@intel.com] Sent: Thursday, October 12, 2017 4:40 PM To: dev@hive.apache.org Subject: How to measure the execution time of query on Hive on Tez Hi all: Anyone knows how to view the detail execution time of every map/reduce task in hive on tez? I screenshot the result: Run DAG is 324.s . But this is not the sum of DURATION time of every tasks( 665=163+22+1+1+2+3+143+31+4+0+254+29+8+2+1+1). So which parameter DURATION(ms) or CPU_TIME(ms) should be used? [cid:image001.png@01D34378.B0FDB5B0] Appreciate to get some feedback from you! Best Regards Kelly Zhang/Zhang,Liyun