[
https://issues.apache.org/jira/browse/HIVE-24625?focusedWorklogId=549507&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-549507
]
ASF GitHub Bot logged work on HIVE-24625:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 08/Feb/21 10:55
Start Date: 08/Feb/21 10:55
Worklog Time Spent: 10m
Work Description: zeroflag commented on a change in pull request #1856:
URL: https://github.com/apache/hive/pull/1856#discussion_r571952378
##########
File path:
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/MetastoreDefaultTransformer.java
##########
@@ -583,29 +584,34 @@ public Table transformCreateTable(Table table,
List<String> processorCapabilitie
throw new MetaException("Database " + dbName + " for table " +
table.getTableName() + " could not be found");
}
- if (TableType.MANAGED_TABLE.name().equals(tableType)) {
+ if (TableType.MANAGED_TABLE.name().equals(tableType)) {
LOG.debug("Table is a MANAGED_TABLE");
txnal = params.get(TABLE_IS_TRANSACTIONAL);
txn_properties = params.get(TABLE_TRANSACTIONAL_PROPERTIES);
+ boolean ctas = Boolean.valueOf(params.getOrDefault(TABLE_IS_CTAS,
"false"));
isInsertAcid = (txn_properties != null &&
txn_properties.equalsIgnoreCase("insert_only"));
if ((txnal == null || txnal.equalsIgnoreCase("FALSE")) && !isInsertAcid)
{ // non-ACID MANAGED TABLE
- LOG.info("Converting " + newTable.getTableName() + " to EXTERNAL
tableType for " + processorId);
- newTable.setTableType(TableType.EXTERNAL_TABLE.toString());
- params.remove(TABLE_IS_TRANSACTIONAL);
- params.remove(TABLE_TRANSACTIONAL_PROPERTIES);
- params.put("EXTERNAL", "TRUE");
- params.put(EXTERNAL_TABLE_PURGE, "TRUE");
- params.put("TRANSLATED_TO_EXTERNAL", "TRUE");
- newTable.setParameters(params);
- LOG.info("Modified table params are:" + params.toString());
-
- if (!table.isSetSd() || table.getSd().getLocation() == null) {
- try {
- Path newPath = hmsHandler.getWh().getDefaultTablePath(db,
table.getTableName(), true);
- newTable.getSd().setLocation(newPath.toString());
- LOG.info("Modified location from null to " + newPath);
- } catch (Exception e) {
- LOG.warn("Exception determining external table location:" +
e.getMessage());
+ if (ctas) {
Review comment:
We remove it from the outside at HiveMetaStore>>create_table_core. Since
transformCreateTable is not always called (only if the transformer is not null).
```
if (tbl.getParameters() != null) {
tbl.getParameters().remove(TABLE_IS_CTAS);
}
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
-------------------
Worklog Id: (was: 549507)
Time Spent: 40m (was: 0.5h)
> CTAS with TBLPROPERTIES ('transactional'='false') loads data into incorrect
> directory
> -------------------------------------------------------------------------------------
>
> Key: HIVE-24625
> URL: https://issues.apache.org/jira/browse/HIVE-24625
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2, Metastore
> Reporter: Attila Magyar
> Assignee: Attila Magyar
> Priority: Major
> Labels: pull-request-available
> Time Spent: 40m
> Remaining Estimate: 0h
>
> MetastoreDefaultTransformer in HMS converts a managed non transactional table
> to external table. MoveTask still uses the managed path when loading the
> data, resulting an always empty table.
> {code:java}
> create table tbl1 TBLPROPERTIES ('transactional'='false') as select * from
> other;{code}
> After the conversion the table location points to an external directory:
> Location: |
> hdfs://c670-node2.coelab.cloudera.com:8020/warehouse/tablespace/external/hive/tbl1
> Move task uses the managed location"
> {code:java}
> INFO : Moving data to directory
> hdfs://...:8020/warehouse/tablespace/managed/hive/tbl1 from
> hdfs://...:8020/warehouse/tablespace/managed/hive/.hive-staging_hive_2021-01-05_16-10-39_973_41005081081760609-4/-ext-1000
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
