Aditya Shah created HIVE-21280:
----------------------------------
Summary: Null pointer exception on runnning compaction against a
MM table.
Key: HIVE-21280
URL: https://issues.apache.org/jira/browse/HIVE-21280
Project: Hive
Issue Type: Bug
Affects Versions: 3.1.1, 3.0.0
Reporter: Aditya Shah
On running compaction on MM table, got a null pointer exception while getting
HDFS session path. The error seemed to me that the session state was not
started for these queries. Even after making it start it further fails in
running a Teztask for insert overwrite on temp table with the contents of the
original table. The cause for this is Tezsession state is not able to
initialize due to Illegal Argument exception being thrown at the time of
setting up caller context in Tez task due to caller id which uses queryid being
an empty string.
I do think session state needs to be started and each of the queries running
for compaction (I'm also doubtful for stats updater thread's queries) should
have a query id. Some details are as follows:
Steps to reproduce:
1) Using beeline with HS2 and HMS
2) create an MM table
3) Insert a few values in the table
4) alter table mm_table compact 'major';
Stack trace on HMS:
{code:java}
compactor.Worker: Caught exception while trying to compact
id:8,dbname:default,tableName:acid_mm_orc,partName:null,state:^@,type:MAJOR,properties:null,runAs:null,tooManyAborts:false,highestWriteId:0.
Marking failed to avoid repeated failures, java.io.IOException:
org.apache.hadoop.hive.ql.metadata.HiveException: Failed to run create
temporary table default.tmp_compactor_acid_mm_orc_1550222367257(`a` int, `b`
string) ROW FORMAT SERDE 'org.apache.hadoop.hive.ql.io.orc.OrcSerde'WITH
SERDEPROPERTIES (
'serialization.format'='1')STORED AS INPUTFORMAT
'org.apache.hadoop.hive.ql.io.orc.OrcInputFormat' OUTPUTFORMAT
'org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat' LOCATION
'hdfs://localhost:9000/user/hive/warehouse/acid_mm_orc/_tmp_2d8a096c-2db5-4ed8-921c-b3f6d31e079e/_base'
TBLPROPERTIES ('transactional'='false')
at
org.apache.hadoop.hive.ql.txn.compactor.CompactorMR.runMmCompaction(CompactorMR.java:373)
at org.apache.hadoop.hive.ql.txn.compactor.CompactorMR.run(CompactorMR.java:241)
at org.apache.hadoop.hive.ql.txn.compactor.Worker.run(Worker.java:174)
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Failed to run
create temporary table default.tmp_compactor_acid_mm_orc_1550222367257(`a` int,
`b` string) ROW FORMAT SERDE 'org.apache.hadoop.hive.ql.io.orc.OrcSerde'WITH
SERDEPROPERTIES (
'serialization.format'='1')STORED AS INPUTFORMAT
'org.apache.hadoop.hive.ql.io.orc.OrcInputFormat' OUTPUTFORMAT
'org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat' LOCATION
'hdfs://localhost:9000/user/hive/warehouse/acid_mm_orc/_tmp_2d8a096c-2db5-4ed8-921c-b3f6d31e079e/_base'
TBLPROPERTIES ('transactional'='false')
at
org.apache.hadoop.hive.ql.txn.compactor.CompactorMR.runOnDriver(CompactorMR.java:525)
at
org.apache.hadoop.hive.ql.txn.compactor.CompactorMR.runMmCompaction(CompactorMR.java:365)
... 2 more
Caused by: java.lang.NullPointerException: Non-local session path expected to
be non-null
at com.google.common.base.Preconditions.checkNotNull(Preconditions.java:228)
at
org.apache.hadoop.hive.ql.session.SessionState.getHDFSSessionPath(SessionState.java:815)
at org.apache.hadoop.hive.ql.Context.<init>(Context.java:309)
at org.apache.hadoop.hive.ql.Context.<init>(Context.java:295)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:591)
at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1684)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1807)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1567)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1556)
at
org.apache.hadoop.hive.ql.txn.compactor.CompactorMR.runOnDriver(CompactorMR.java:522)
... 3 more
{code}
cc: [~ekoifman] [~vgumashta] [~sershe]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
