muxin created HIVE-15821:
----------------------------
Summary: hiveserver2 cannot print diagnostic message in JobDebugger
Key: HIVE-15821
URL: https://issues.apache.org/jira/browse/HIVE-15821
Project: Hive
Issue Type: Bug
Components: HiveServer2
Affects Versions: 2.1.1, 1.2.1
Reporter: muxin
we meet this issue when upgrading to hiveserver2. when error happened running
mapreduce jobs, diagnostic message are printed in hive log file but cannot be
seen in beeline.
here is steps to reproduce this issue:
1. write a simple udf, assuming com.example.udf.DateFormat1(String date, String
srcFormat, String DestFormat), notice do not catch exceptions in it.
2. add the udf jar, create temporary function dateformat1 as
'com.example.udf.DateFormat'.
3. run ==> create temporary table test_table as select dateformat1('20170101',
'dd/MMM/yyyy', 'yyyy-MM-dd'). of course it will end up failed, in hive.log we
can see the full diagnostic message and error stacktrace, but in beeline we
only see one line :
return code 1 from org.apache.hadoop.hive.ql.exec.mr.MapredTask
after analysis we found the cause:
jobDebuger is started in a subthread, and hiveserver2 didnot pass current
operationlog into it, so beeline cannot fetch log printed in jobDebugger.
so our solution is :
1. add a private parameter operationlog to JobDebugger,
before start jobdebugger thread in HadoopJobExecHelper.progress(), set
jobDebugger.operationlog to OperationLog.getCurrentOperationLog()
2. in JobDebugger.run(), replace
showJobFailDebugInfo();
with:
OperationLog.setCurrentOperationLog(operationLog);
showJobFailDebugInfo();
OperationLog.removeCurrentOperationLog();
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
