[
https://issues.apache.org/jira/browse/HIVE-25325?focusedWorklogId=624413&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-624413
]
ASF GitHub Bot logged work on HIVE-25325:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 19/Jul/21 14:42
Start Date: 19/Jul/21 14:42
Worklog Time Spent: 10m
Work Description: kuczoram commented on a change in pull request #2471:
URL: https://github.com/apache/hive/pull/2471#discussion_r672358898
##########
File path:
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HMSHandler.java
##########
@@ -3360,42 +3360,50 @@ public CmRecycleResponse cm_recycle(final
CmRecycleRequest request) throws MetaE
public void truncate_table(final String dbName, final String tableName,
List<String> partNames)
throws NoSuchObjectException, MetaException {
// Deprecated path, won't work for txn tables.
- truncateTableInternal(dbName, tableName, partNames, null, -1);
+ truncateTableInternal(dbName, tableName, partNames, null, -1, null);
}
@Override
public TruncateTableResponse truncate_table_req(TruncateTableRequest req)
throws MetaException, TException {
truncateTableInternal(req.getDbName(), req.getTableName(),
req.getPartNames(),
- req.getValidWriteIdList(), req.getWriteId());
+ req.getValidWriteIdList(), req.getWriteId(),
req.getEnvironmentContext());
return new TruncateTableResponse();
}
private void truncateTableInternal(String dbName, String tableName,
List<String> partNames,
- String validWriteIds, long writeId)
throws MetaException, NoSuchObjectException {
+ String validWriteIds, long writeId,
EnvironmentContext context) throws MetaException, NoSuchObjectException {
boolean isSkipTrash = false, needCmRecycle = false;
try {
String[] parsedDbName = parseDbName(dbName, conf);
Table tbl = get_table_core(parsedDbName[CAT_NAME],
parsedDbName[DB_NAME], tableName);
- boolean truncateFiles = !TxnUtils.isTransactionalTable(tbl) ||
- !MetastoreConf.getBoolVar(getConf(),
MetastoreConf.ConfVars.TRUNCATE_ACID_USE_BASE);
-
- if (truncateFiles) {
- isSkipTrash = MetaStoreUtils.isSkipTrash(tbl.getParameters());
- Database db = get_database_core(parsedDbName[CAT_NAME],
parsedDbName[DB_NAME]);
- needCmRecycle = ReplChangeManager.shouldEnableCm(db, tbl);
+ boolean skipDataDeletion = false;
+ if (context != null && context.getProperties() != null
+ && context.getProperties().get("truncateSkipDataDeletion") != null) {
+ skipDataDeletion =
Boolean.parseBoolean(context.getProperties().get("truncateSkipDataDeletion"));
}
- // This is not transactional
- for (Path location : getLocationsForTruncate(getMS(),
parsedDbName[CAT_NAME],
- parsedDbName[DB_NAME], tableName, tbl, partNames)) {
- FileSystem fs = location.getFileSystem(getConf());
+
+ if (!skipDataDeletion) {
+ boolean truncateFiles = !TxnUtils.isTransactionalTable(tbl)
+ || !MetastoreConf.getBoolVar(getConf(),
MetastoreConf.ConfVars.TRUNCATE_ACID_USE_BASE);
+
if (truncateFiles) {
- truncateDataFiles(location, fs, isSkipTrash, needCmRecycle);
- } else {
- // For Acid tables we don't need to delete the old files, only write
an empty baseDir.
- // Compaction and cleaner will take care of the rest
- addTruncateBaseFile(location, writeId, fs);
+ isSkipTrash = MetaStoreUtils.isSkipTrash(tbl.getParameters());
+ Database db = get_database_core(parsedDbName[CAT_NAME],
parsedDbName[DB_NAME]);
+ needCmRecycle = ReplChangeManager.shouldEnableCm(db, tbl);
+ }
+ // This is not transactional
+ for (Path location : getLocationsForTruncate(getMS(),
parsedDbName[CAT_NAME], parsedDbName[DB_NAME], tableName,
+ tbl, partNames)) {
+ FileSystem fs = location.getFileSystem(getConf());
+ if (truncateFiles) {
Review comment:
I am not sure what you mean by same code as before. The truncateFiles is
used for different purpose. It is used to treat the acid tables differently if
the proper flags are set: instead of deleting the files, create a new base. But
we don't want any of these logic for Iceberg tables, so that's why it is put
inside the if (!skipDataDeletion).
I guess the check in line 3391 is because those steps should be executed
once and not for all path. But it is part of the original implementation which
I only put inside an if and didn't change otherwise, so I am just guessing
about the original intentions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
-------------------
Worklog Id: (was: 624413)
Time Spent: 2.5h (was: 2h 20m)
> Add TRUNCATE TABLE support for Hive Iceberg tables
> --------------------------------------------------
>
> Key: HIVE-25325
> URL: https://issues.apache.org/jira/browse/HIVE-25325
> Project: Hive
> Issue Type: Improvement
> Reporter: Marta Kuczora
> Assignee: Marta Kuczora
> Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Time Spent: 2.5h
> Remaining Estimate: 0h
>
> Implement the TRUNCATE operation for Hive Iceberg tables. Since these tables
> are unpartitioned in Hive, only the truncate unpartitioned table use case has
> to be supported.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
