Question about running Hive on Tez

2013-12-13 Thread Zhenxiao Luo 
Hi,

Excuse me. May I ask a question about running Hive on Tez?

I've installed Hive on Tez, and running a simple query from hiveCli,

hive set hive.optimize.tez=true;
hive select * from table order by title_id limit 5;

While, each time, I could see from the TezJobMonitor that, all the map
jobs are done, but the reducer never get started, and the job is
running forever there.

I tried a number of times, and each time the same failure(job running
hangs) happens again and again.
Does anyone successfully running queries using Hive on Tez? Are there
any tips or suggestions?

Here is my job log:

13/12/13 20:57:31 INFO client.TezSession: Submitting dag to
TezSession, sessionName=HIVE-365b35bc-2461-4e2f-83f9-8da1fa356a86,
applicationId=application_1386896881353_0027
13/12/13 20:57:33 INFO client.TezSession: Submitted dag to TezSession,
sessionName=HIVE-365b35bc-2461-4e2f-83f9-8da1fa356a86,
applicationId=application_1386896881353_0027,
dagId=dag_1386896881353_0027_1
13/12/13 20:57:33 INFO client.RMProxy: Connecting to ResourceManager
at /10.183.195.180:9022
13/12/13 20:57:33 INFO log.PerfLogger: /PERFLOG method=TezSubmitDag
start=1386968251250 end=1386968253338 duration=2088
from=org.apache.hadoop.hive.ql.exec.tez.TezTask


13/12/13 20:57:33 INFO tez.TezJobMonitor:

13/12/13 20:57:33 INFO log.PerfLogger: PERFLOG method=TezRunDag
from=org.apache.hadoop.hive.ql.exec.tez.TezJobMonitor
13/12/13 20:57:33 INFO log.PerfLogger: PERFLOG
method=TezSubmitToRunningDag
from=org.apache.hadoop.hive.ql.exec.tez.TezJobMonitor
13/12/13 20:57:33 INFO log.PerfLogger: /PERFLOG
method=TezSubmitToRunningDag start=1386968253341 end=1386968253402
duration=61 from=org.apache.hadoop.hive.ql.exec.tez.TezJobMonitor
Status: Running (application id: application_1386896881353_0027)

13/12/13 20:57:33 INFO tez.TezJobMonitor: Status: Running (application
id: application_1386896881353_0027)

13/12/13 20:57:33 INFO log.PerfLogger: PERFLOG
method=TezRunVertex.Reducer 2
from=org.apache.hadoop.hive.ql.exec.tez.TezJobMonitor
13/12/13 20:57:33 INFO log.PerfLogger: PERFLOG
method=TezRunVertex.Map 1
from=org.apache.hadoop.hive.ql.exec.tez.TezJobMonitor
Map 1: -/- Reducer 2: -/-
13/12/13 20:57:33 INFO tez.TezJobMonitor: Map 1: -/- Reducer 2: -/-
Map 1: -/- Reducer 2: 0/1
13/12/13 20:57:33 INFO tez.TezJobMonitor: Map 1: -/- Reducer 2: 0/1
Map 1: 0/16 Reducer 2: 0/1
13/12/13 20:57:34 INFO tez.TezJobMonitor: Map 1: 0/16 Reducer 2: 0/1
Map 1: 0/16 Reducer 2: 0/1
13/12/13 20:57:37 INFO tez.TezJobMonitor: Map 1: 0/16 Reducer 2: 0/1
Map 1: 0/16 Reducer 2: 0/1
13/12/13 20:57:40 INFO tez.TezJobMonitor: Map 1: 0/16 Reducer 2: 0/1
Map 1: 0/16 Reducer 2: 0/1
13/12/13 20:57:43 INFO tez.TezJobMonitor: Map 1: 0/16 Reducer 2: 0/1
Map 1: 0/16 Reducer 2: 0/1
13/12/13 20:57:46 INFO tez.TezJobMonitor: Map 1: 0/16 Reducer 2: 0/1
Map 1: 0/16 Reducer 2: 0/1
13/12/13 20:57:49 INFO tez.TezJobMonitor: Map 1: 0/16 Reducer 2: 0/1
Map 1: 0/16 Reducer 2: 0/1
13/12/13 20:57:52 INFO tez.TezJobMonitor: Map 1: 0/16 Reducer 2: 0/1
Map 1: 0/16 Reducer 2: 0/1
13/12/13 20:57:55 INFO tez.TezJobMonitor: Map 1: 0/16 Reducer 2: 0/1
Map 1: 1/16 Reducer 2: 0/1
13/12/13 20:57:56 INFO tez.TezJobMonitor: Map 1: 1/16 Reducer 2: 0/1
Map 1: 2/16 Reducer 2: 0/1
13/12/13 20:57:58 INFO tez.TezJobMonitor: Map 1: 2/16 Reducer 2: 0/1
Map 1: 3/16 Reducer 2: 0/1
13/12/13 20:57:58 INFO tez.TezJobMonitor: Map 1: 3/16 Reducer 2: 0/1
Map 1: 5/16 Reducer 2: 0/1
13/12/13 20:57:59 INFO tez.TezJobMonitor: Map 1: 5/16 Reducer 2: 0/1
Map 1: 8/16 Reducer 2: 0/1
13/12/13 20:57:59 INFO tez.TezJobMonitor: Map 1: 8/16 Reducer 2: 0/1
Map 1: 12/16 Reducer 2: 0/1
13/12/13 20:57:59 INFO tez.TezJobMonitor: Map 1: 12/16 Reducer 2: 0/1
Map 1: 15/16 Reducer 2: 0/1
13/12/13 20:58:00 INFO tez.TezJobMonitor: Map 1: 15/16 Reducer 2: 0/1
13/12/13 20:58:00 INFO log.PerfLogger: /PERFLOG
method=TezRunVertex.Map 1 start=1386968253402 end=1386968280223
duration=26821 from=org.apache.hadoop.hive.ql.exec.tez.TezJobMonitor
Map 1: 16/16 Reducer 2: 0/1
13/12/13 20:58:00 INFO tez.TezJobMonitor: Map 1: 16/16 Reducer 2: 0/1
Map 1: 16/16 Reducer 2: 0/1
13/12/13 20:58:03 INFO tez.TezJobMonitor: Map 1: 16/16 Reducer 2: 0/1
Map 1: 16/16 Reducer 2: 0/1
13/12/13 20:58:06 INFO tez.TezJobMonitor: Map 1: 16/16 Reducer 2: 0/1
Map 1: 16/16 Reducer 2: 0/1
13/12/13 20:58:09 INFO tez.TezJobMonitor: Map 1: 16/16 Reducer 2: 0/1
Map 1: 16/16 Reducer 2: 0/1
13/12/13 20:58:12 INFO tez.TezJobMonitor: Map 1: 16/16 Reducer 2: 0/1
Map 1: 16/16 Reducer 2: 0/1
13/12/13 20:58:15 INFO tez.TezJobMonitor: Map 1: 16/16 Reducer 2: 0/1
Map 1: 16/16 Reducer 2: 0/1
13/12/13 20:58:18 INFO tez.TezJobMonitor: Map 1: 16/16 Reducer 2: 0/1
Map 1: 16/16 Reducer 2: 0/1
13/12/13 20:58:21 INFO tez.TezJobMonitor: Map 1: 16/16 Reducer 2: 0/1
Map 1: 16/16 Reducer 2: 0/1
13/12/13 20:58:24 INFO tez.TezJobMonitor: Map 1: 16/16 Reducer 2: 0/1
Map 1: 16/16 Reducer 2: 0/1
13/12/13 20:58:27 INFO tez.TezJobMonitor: Map 1: 16/16 Reducer 2: 0/1
Map 1: 16/16 Reducer 2: 0/1
13/12/13 20:58:30 INFO

Question about Hive on Tez

2013-12-11 Thread Zhenxiao Luo 
Excuse me. May I ask a question about Hive on Tez?

We just started evaluating Hive on Tez.  Would like to know, is
Hive on Tez development done yet? Is there any documentation we could
reference to run benchmarks? Also, is there any known limitations in
Hive on Tez? Is it in Hive0.12?

Thanks,
Zhenxiao