[
https://issues.apache.org/jira/browse/HIVE-26374?focusedWorklogId=788418&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-788418
]
ASF GitHub Bot logged work on HIVE-26374:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 06/Jul/22 21:52
Start Date: 06/Jul/22 21:52
Worklog Time Spent: 10m
Work Description: cravani opened a new pull request, #3419:
URL: https://github.com/apache/hive/pull/3419
…mns with Reserved Keywords
<!--
Thanks for sending a pull request! Here are some tips for you:
1. If this is your first time, please read our contributor guidelines:
https://cwiki.apache.org/confluence/display/Hive/HowToContribute
2. Ensure that you have created an issue on the Hive project JIRA:
https://issues.apache.org/jira/projects/HIVE/summary
3. Ensure you have added or run the appropriate tests for your PR:
4. If the PR is unfinished, add '[WIP]' in your PR title, e.g.,
'[WIP]HIVE-XXXXX: Your PR title ...'.
5. Be sure to keep the PR description updated to reflect all changes.
6. Please write your PR title to summarize what this PR proposes.
7. If possible, provide a concise example to reproduce the issue for a
faster review.
-->
### What changes were proposed in this pull request?
Query based compaction fails on Tables having complex data types with
reserved keywords for columns. The compaction fails while creating a temporary
table as it does not quote the columns correctly.
The changes intends to align the code used in
[DDLPlanUtils.java](https://github.com/apache/hive/blob/53009126f6fe7ccf24cf052fd6c156542f38b19d/ql/src/java/org/apache/hadoop/hive/ql/exec/DDLPlanUtils.java#L824)
### Why are the changes needed?
This fixes the current bug with Query based compaction
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
Currently tested manually, I plan to add Unit test based on CI run.
Issue Time Tracking
-------------------
Worklog Id: (was: 788418)
Remaining Estimate: 0h
Time Spent: 10m
> Query based compaction fails for tables with CDT and columns with Reserved
> Keywords
> -----------------------------------------------------------------------------------
>
> Key: HIVE-26374
> URL: https://issues.apache.org/jira/browse/HIVE-26374
> Project: Hive
> Issue Type: Bug
> Components: Hive, Transactions
> Affects Versions: 3.1.3, 4.0.0-alpha-1
> Reporter: Chiran Ravani
> Assignee: Chiran Ravani
> Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Query based compaction fails on Tables having complex data types with
> reserved keywords for columns. The compaction fails while creating a
> temporary table as it does not quote the columns correctly.
>
> Below are the steps to reproduce the issue.
> {code:java}
> create table complex_dt_compact2(col1 array<struct<arr_col1:int,
> `timestamp`:string>>);
> insert into complex_dt_compact2 SELECT
> ARRAY(NAMED_STRUCT('arr_col1',1,'timestamp','2022-07-05 21:51:20.371'));
> insert into complex_dt_compact2 SELECT
> ARRAY(NAMED_STRUCT('arr_col1',2,'timestamp','2022-07-05 21:51:20.371'));
> alter table complex_dt_compact2 compact 'major' and wait; {code}
> Error:
> {code:java}
> 2022-07-05T22:15:47.710Z
> hiveserver2-0.hiveserver2-service.compute-1657056457-xkcx.svc.cluster.local
> hiveserver2 1 dbb4011d-c788-4b99-a31d-06bb6dd7182e [mdc@18060
> class="compactor.Worker" level="ERROR"
> thread="hiveserver2-0.hiveserver2-service.compute-1657056457-xkcx.svc.cluster.local-64_executor"]
> Caught exception while trying to compact
> id:3,dbname:default,tableName:complex_dt_compact2,partName:null,state:,type:MAJOR,enqueueTime:0,start:0,properties:null,runAs:hive,tooManyAborts:false,hasOldAbort:false,highestWriteId:3,errorMessage:null,workerId:
> null,initiatorId: null,retryRetention0. Marking failed to avoid repeated
> failures
> java.io.IOException: org.apache.hadoop.hive.ql.metadata.HiveException:
> Failed to run CREATE temporary external table
> default_tmp_compactor_complex_dt_compact2_1657059347578(`operation` int,
> `originalTransaction` bigint, `bucket` int, `rowId` bigint,
> `currentTransaction` bigint, `row` struct<`col1`
> :array<struct<arr_col1:int,timestamp:string>>>) stored as orc LOCATION
> 's3a://obfuscated/clusters/obfuscated/obfuscated/warehouse/tablespace/managed/hive/complex_dt_compact2/base_0000003_v0000038'
> TBLPROPERTIES ('compactiontable'='true', 'transactional'='false')
> at
> org.apache.hadoop.hive.ql.txn.compactor.QueryCompactor.runCompactionQueries(QueryCompactor.java:120)
> at
> org.apache.hadoop.hive.ql.txn.compactor.MajorQueryCompactor.runCompaction(MajorQueryCompactor.java:63)
> at
> org.apache.hadoop.hive.ql.txn.compactor.Worker.findNextCompactionAndExecute(Worker.java:517)
> at
> org.apache.hadoop.hive.ql.txn.compactor.Worker.lambda$run$0(Worker.java:120)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:750)
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Failed to
> run CREATE temporary external table
> default_tmp_compactor_complex_dt_compact2_1657059347578(`operation` int,
> `originalTransaction` bigint, `bucket` int, `rowId` bigint,
> `currentTransaction` bigint, `row` struct<`col1`
> :array<struct<arr_col1:int,timestamp:string>>>) stored as orc LOCATION
> 's3a://obfuscated/clusters/obfuscated/obfuscated/warehouse/tablespace/managed/hive/complex_dt_compact2/base_0000003_v0000038'
> TBLPROPERTIES ('compactiontable'='true', 'transactional'='false')
> at
> org.apache.hadoop.hive.ql.DriverUtils.runOnDriver(DriverUtils.java:73)
> at
> org.apache.hadoop.hive.ql.DriverUtils.runOnDriver(DriverUtils.java:50)
> at
> org.apache.hadoop.hive.ql.txn.compactor.QueryCompactor.runCompactionQueries(QueryCompactor.java:113)
> ... 7 more
> Caused by: (responseCode = 40000, errorMessage = FAILED: ParseException
> line 1:241 cannot recognize input near 'timestamp' ':' 'string' in column
> specification, SQLState = 42000, exception = line 1:241 cannot recognize
> input near 'timestamp' ':' 'string' in column specification)
> at
> org.apache.hadoop.hive.ql.DriverUtils.createProcessorException(DriverUtils.java:143)
> at
> org.apache.hadoop.hive.ql.Compiler.handleException(Compiler.java:466)
> at org.apache.hadoop.hive.ql.Compiler.compile(Compiler.java:122)
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:197)
> at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:636)
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:694)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:526)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:515)
> at
> org.apache.hadoop.hive.ql.DriverUtils.runOnDriver(DriverUtils.java:70)
> ... 9 more{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
