With "hive -hiveconf hive.root.logger=DEBUG,DRFA -e ... "
/tmp/<username>/hive.log seems to have pretty detailed log messages
including debug msgs. I don't see the "initialization failed" message
and the stack trace mentioned in HADOOP-5759 - is there any other place
I need to check. On the UI I only see map task in pending state and no
further information (this is with hadoop-0.20.1). With a more recent
hadoop I see no tasks launched at all. This used to work a month before
- am wondering if any changes in hive caused this.
Thanks,
Pradeep
Amareshwari Sri Ramadasu wrote:
Pradeep, you might be hitting HADOOP-5759 and the job is not getting
initialized at all. Look in JobTracker logs for the jobid to confirm
the same.
On 9/28/10 6:28 AM, "Pradeep Kamath" <prade...@yahoo-inc.com> wrote:
Here is some relevant stuff from /tmp/pradeepk/hive.logs - can't
make much out of it:
2010-09-27 17:40:01,081 INFO exec.MapRedTask
(SessionState.java:printInfo(268)) - Starting Job =
job_201009251752_1341, Tracking URL =
http://<hostname>:50030/jobdetails.jsp?jobid=job_201009251752_1341
<http://%3Chostname%3E:50030/jobdetails.jsp?jobid=job_201009251752_1341>
2010-09-27 17:40:01,081 INFO exec.MapRedTask
(SessionState.java:printInfo(268)) - Kill Command =
/homes/pradeepk/hadoopcluster/hadoop/bin/../bin/hadoop job
-Dmapred.job.tracker=<hostname>:50020 -kill job_201009251752_1341
2010-09-27 17:40:01,081 DEBUG ipc.Client
(Client.java:sendParam(469)) - IPC Client (47) connection to
<hostname>/216.252.118.203:50020 from pradeepk sending #129
2010-09-27 17:40:01,083 DEBUG ipc.Client
(Client.java:receiveResponse(504)) - IPC Client (47) connection to
<hostname>/216.252.118.203:50020 from pradeepk got value #129
2010-09-27 17:40:01,083 DEBUG ipc.RPC (RPC.java:invoke(225)) -
Call: getJobStatus 2
2010-09-27 17:40:02,086 DEBUG ipc.Client
(Client.java:sendParam(469)) - IPC Client (47) connection to
<hostname>/216.252.118.203:50020 from pradeepk sending #130
2010-09-27 17:40:02,090 DEBUG ipc.Client
(Client.java:receiveResponse(504)) - IPC Client (47) connection to
<hostname>/216.252.118.203:50020 from pradeepk got value #130
2010-09-27 17:40:02,091 DEBUG ipc.RPC (RPC.java:invoke(225)) -
Call: getJobStatus 5
2010-09-27 17:40:02,092 DEBUG ipc.Client
(Client.java:sendParam(469)) - IPC Client (47) connection to
<hostname>/216.252.118.203:50020 from pradeepk sending #131
2010-09-27 17:40:02,093 DEBUG ipc.Client
(Client.java:receiveResponse(504)) - IPC Client (47) connection to
<hostname>/216.252.118.203:50020 from pradeepk got value #131
2010-09-27 17:40:02,094 DEBUG ipc.RPC (RPC.java:invoke(225)) -
Call: getJobStatus 2
2010-09-27 17:40:02,094 DEBUG ipc.Client
(Client.java:sendParam(469)) - IPC Client (47) connection to
<hostname>/216.252.118.203:50020 from pradeepk sending #132
2010-09-27 17:40:02,096 DEBUG ipc.Client
(Client.java:receiveResponse(504)) - IPC Client (47) connection to
<hostname>/216.252.118.203:50020 from pradeepk got value #132
2010-09-27 17:40:02,096 DEBUG ipc.RPC (RPC.java:invoke(225)) -
Call: getJobProfile 2
2010-09-27 17:40:02,096 DEBUG ipc.Client
(Client.java:sendParam(469)) - IPC Client (47) connection to
<hostname>/216.252.118.203:50020 from pradeepk sending #133
2010-09-27 17:40:02,100 DEBUG ipc.Client
(Client.java:receiveResponse(504)) - IPC Client (47) connection to
<hostname>/216.252.118.203:50020 from pradeepk got value #133
2010-09-27 17:40:02,100 DEBUG ipc.RPC (RPC.java:invoke(225)) -
Call: getJobCounters 4
2010-09-27 17:40:02,101 DEBUG mapred.Counters
(Counters.java:<init>(151)) - Creating group
org.apache.hadoop.hive.ql.exec.Operator$ProgressCounter with nothing
2010-09-27 17:40:02,101 DEBUG mapred.Counters
(Counters.java:getCounterForName(277)) - Adding CREATED_FILES
2010-09-27 17:40:02,103 INFO exec.MapRedTask
(SessionState.java:printInfo(268)) - 2010-09-27 17:40:02,103
Stage-2 map = 100%, reduce = 100%
2010-09-27 17:40:02,104 DEBUG ipc.Client
(Client.java:sendParam(469)) - IPC Client (47) connection to
<hostname>/216.252.118.203:50020 from pradeepk sending #134
2010-09-27 17:40:02,105 DEBUG ipc.Client
(Client.java:receiveResponse(504)) - IPC Client (47) connection to
<hostname>/216.252.118.203:50020 from pradeepk got value #134
2010-09-27 17:40:02,106 DEBUG ipc.RPC (RPC.java:invoke(225)) -
Call: getJobStatus 2
2010-09-27 17:40:02,106 DEBUG ipc.Client
(Client.java:sendParam(469)) - IPC Client (47) connection to
<hostname>/216.252.118.203:50020 from pradeepk sending #135
2010-09-27 17:40:02,108 DEBUG ipc.Client
(Client.java:receiveResponse(504)) - IPC Client (47) connection to
<hostname>/216.252.118.203:50020 from pradeepk got value #135
2010-09-27 17:40:02,108 DEBUG ipc.RPC (RPC.java:invoke(225)) -
Call: getJobCounters 2
2010-09-27 17:40:02,109 DEBUG mapred.Counters
(Counters.java:<init>(151)) - Creating group
org.apache.hadoop.hive.ql.exec.Operator$ProgressCounter with nothing
2010-09-27 17:40:02,109 DEBUG mapred.Counters
(Counters.java:getCounterForName(277)) - Adding CREATED_FILES
2010-09-27 17:40:02,109 DEBUG ipc.Client
(Client.java:sendParam(469)) - IPC Client (47) connection to
<hostname>/216.252.118.203:50020 from pradeepk sending #136
2010-09-27 17:40:02,111 DEBUG ipc.Client
(Client.java:receiveResponse(504)) - IPC Client (47) connection to
<hostname>/216.252.118.203:50020 from pradeepk got value #136
2010-09-27 17:40:02,111 DEBUG ipc.RPC (RPC.java:invoke(225)) -
Call: getJobStatus 2
2010-09-27 17:40:02,112 ERROR exec.MapRedTask
(SessionState.java:printError(277)) - Ended Job =
job_201009251752_1341 with errors
Ning Zhang wrote:
From the error info, it seems the 2nd job has been launched
and failed. So I'm assuming there are map tasks started? If
not, you can find the error message in the client log file
/tmp/<userid>/hive.log at the machine running hive after
setting the hive.root.logger property Steven mentioned.
On Sep 27, 2010, at 1:11 PM, Steven Wong wrote:
>>>>> 2010-09-24 13:29:03,504 Stage-2 map = 100%, reduce =
100%
>>>>>
>>>>> Ended Job = job_201009241059_0282 with errors
>>>>>
>>>>> FAILED: Execution Error, return code 2 from
>>>>> org.apache.hadoop.hive.ql.exec.MapRedTask
>>>>>
>>>>>
