[
https://issues.apache.org/jira/browse/HIVE-26217?focusedWorklogId=776220&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-776220
]
ASF GitHub Bot logged work on HIVE-26217:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 31/May/22 10:39
Start Date: 31/May/22 10:39
Worklog Time Spent: 10m
Work Description: deniskuzZ commented on code in PR #3281:
URL: https://github.com/apache/hive/pull/3281#discussion_r885472281
##########
ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java:
##########
@@ -7598,6 +7602,26 @@ protected Operator genFileSinkPlan(String dest, QB qb,
Operator input)
destTableIsTransactional = tblProps != null &&
AcidUtils.isTablePropertyTransactional(tblProps);
if (destTableIsTransactional) {
+ isNonNativeTable = MetaStoreUtils.isNonNativeTable(tblProps);
+ boolean isCtas = tblDesc != null && tblDesc.isCTAS();
+ isMmTable = isMmCreate = AcidUtils.isInsertOnlyTable(tblProps);
+ if (!isNonNativeTable && !destTableIsTemporary && isCtas) {
+ destTableIsFullAcid = AcidUtils.isFullAcidTable(tblProps);
+ acidOperation = getAcidType(dest);
+ isDirectInsert = isDirectInsert(destTableIsFullAcid, acidOperation);
+ boolean enableSuffixing = HiveConf.getBoolVar(conf,
ConfVars.HIVE_ACID_CREATE_TABLE_USE_SUFFIX)
+ || HiveConf.getBoolVar(conf,
ConfVars.HIVE_ACID_LOCKLESS_READS_ENABLED);
+ if (isDirectInsert || isMmTable) {
+ destinationPath = getCTASDestinationTableLocation(tblDesc,
enableSuffixing);
+ // Setting the location so that metadata transformers
+ // does not change the location later while creating the table.
+ tblDesc.setLocation(destinationPath.toString());
+ // Property SOFT_DELETE_TABLE needs to be added to indicate that
suffixing is used.
+ if (enableSuffixing && tblDesc.getLocation().matches("(.*)" +
SOFT_DELETE_TABLE_PATTERN)) {
Review Comment:
can't we handle suffix here:
````
if (createTableUseSuffix) {
long txnId = ctx.getHiveTxnManager().getCurrentTxnId();
suffix = SOFT_DELETE_PATH_SUFFIX + String.format(DELTA_DIGITS, txnId)
tblDesc.getTblProps().put(SOFT_DELETE_TABLE, Boolean.TRUE.toString());
destinationPath = new Path(destinationPath + suffix);
}
tblDesc.setLocation(destinationPath.toString());
````
Issue Time Tracking
-------------------
Worklog Id: (was: 776220)
Time Spent: 9h 20m (was: 9h 10m)
> Make CTAS use Direct Insert Semantics
> -------------------------------------
>
> Key: HIVE-26217
> URL: https://issues.apache.org/jira/browse/HIVE-26217
> Project: Hive
> Issue Type: Improvement
> Reporter: Sourabh Badhya
> Assignee: Sourabh Badhya
> Priority: Major
> Labels: pull-request-available
> Time Spent: 9h 20m
> Remaining Estimate: 0h
>
> CTAS on transactional tables currently does a copy from staging location to
> table location. This can be avoided by using Direct Insert semantics. Added
> support for suffixed table locations as well.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
