Chiran Ravani created HIVE-26374:
------------------------------------

             Summary: Query based compaction fails for tables with CDT and 
columns with Reserved Keywords
                 Key: HIVE-26374
                 URL: https://issues.apache.org/jira/browse/HIVE-26374
             Project: Hive
          Issue Type: Bug
          Components: Hive, Transactions
    Affects Versions: 4.0.0-alpha-1, 3.1.3
            Reporter: Chiran Ravani


Query based compaction fails on Tables having complex data types with reserved 
keywords for columns. The compaction fails while creating a temporary table as 
it does not quote the columns correctly.

 

Below are the steps to reproduce the issue.
{code:java}
create table complex_dt_compact2(col1 array<struct<arr_col1:int, 
`timestamp`:string>>);
insert into complex_dt_compact2 SELECT 
ARRAY(NAMED_STRUCT('arr_col1',1,'timestamp','2022-07-05 21:51:20.371'));
insert into complex_dt_compact2 SELECT 
ARRAY(NAMED_STRUCT('arr_col1',2,'timestamp','2022-07-05 21:51:20.371'));
alter table complex_dt_compact2 compact 'major' and wait; {code}

Error:
{code:java}
2022-07-05T22:15:47.710Z 
hiveserver2-0.hiveserver2-service.compute-1657056457-xkcx.svc.cluster.local 
hiveserver2 1 dbb4011d-c788-4b99-a31d-06bb6dd7182e [mdc@18060 
class="compactor.Worker" level="ERROR" 
thread="hiveserver2-0.hiveserver2-service.compute-1657056457-xkcx.svc.cluster.local-64_executor"]
 Caught exception while trying to compact 
id:3,dbname:default,tableName:complex_dt_compact2,partName:null,state:,type:MAJOR,enqueueTime:0,start:0,properties:null,runAs:hive,tooManyAborts:false,hasOldAbort:false,highestWriteId:3,errorMessage:null,workerId:
 null,initiatorId: null,retryRetention0. Marking failed to avoid repeated 
failures
    java.io.IOException: org.apache.hadoop.hive.ql.metadata.HiveException: 
Failed to run CREATE temporary external table 
default_tmp_compactor_complex_dt_compact2_1657059347578(`operation` int, 
`originalTransaction` bigint, `bucket` int, `rowId` bigint, 
`currentTransaction` bigint, `row` struct<`col1` 
:array<struct<arr_col1:int,timestamp:string>>>)  stored as orc LOCATION 
's3a://obfuscated/clusters/obfuscated/obfuscated/warehouse/tablespace/managed/hive/complex_dt_compact2/base_0000003_v0000038'
 TBLPROPERTIES ('compactiontable'='true', 'transactional'='false')
        at 
org.apache.hadoop.hive.ql.txn.compactor.QueryCompactor.runCompactionQueries(QueryCompactor.java:120)
        at 
org.apache.hadoop.hive.ql.txn.compactor.MajorQueryCompactor.runCompaction(MajorQueryCompactor.java:63)
        at 
org.apache.hadoop.hive.ql.txn.compactor.Worker.findNextCompactionAndExecute(Worker.java:517)
        at 
org.apache.hadoop.hive.ql.txn.compactor.Worker.lambda$run$0(Worker.java:120)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:750)
    Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Failed to run 
CREATE temporary external table 
default_tmp_compactor_complex_dt_compact2_1657059347578(`operation` int, 
`originalTransaction` bigint, `bucket` int, `rowId` bigint, 
`currentTransaction` bigint, `row` struct<`col1` 
:array<struct<arr_col1:int,timestamp:string>>>)  stored as orc LOCATION 
's3a://obfuscated/clusters/obfuscated/obfuscated/warehouse/tablespace/managed/hive/complex_dt_compact2/base_0000003_v0000038'
 TBLPROPERTIES ('compactiontable'='true', 'transactional'='false')
        at 
org.apache.hadoop.hive.ql.DriverUtils.runOnDriver(DriverUtils.java:73)
        at 
org.apache.hadoop.hive.ql.DriverUtils.runOnDriver(DriverUtils.java:50)
        at 
org.apache.hadoop.hive.ql.txn.compactor.QueryCompactor.runCompactionQueries(QueryCompactor.java:113)
        ... 7 more
    Caused by: (responseCode = 40000, errorMessage = FAILED: ParseException 
line 1:241 cannot recognize input near 'timestamp' ':' 'string' in column 
specification, SQLState = 42000, exception = line 1:241 cannot recognize input 
near 'timestamp' ':' 'string' in column specification)
        at 
org.apache.hadoop.hive.ql.DriverUtils.createProcessorException(DriverUtils.java:143)
        at org.apache.hadoop.hive.ql.Compiler.handleException(Compiler.java:466)
        at org.apache.hadoop.hive.ql.Compiler.compile(Compiler.java:122)
        at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:197)
        at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:636)
        at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:694)
        at org.apache.hadoop.hive.ql.Driver.run(Driver.java:526)
        at org.apache.hadoop.hive.ql.Driver.run(Driver.java:515)
        at 
org.apache.hadoop.hive.ql.DriverUtils.runOnDriver(DriverUtils.java:70)
        ... 9 more{code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Reply via email to