pvary commented on a change in pull request #2025:
URL: https://github.com/apache/iceberg/pull/2025#discussion_r551849275
##########
File path: mr/src/main/java/org/apache/iceberg/mr/hive/HiveIcebergMetaHook.java
##########
@@ -114,6 +114,10 @@ public void
preCreateTable(org.apache.hadoop.hive.metastore.api.Table hmsTable)
hmsTable.getParameters().put(InputFormatConfig.EXTERNAL_TABLE_PURGE,
"TRUE");
}
+ // For a table created by Hive DDL to be readable by Impala, we need the
Input and OutputFormat set explicitly
+
hmsTable.getSd().setInputFormat(HiveIcebergInputFormat.class.getCanonicalName());
+
hmsTable.getSd().setOutputFormat(HiveIcebergOutputFormat.class.getCanonicalName());
+
Review comment:
Discussed this with @marton-bod.
After @boroknagyz's PR (#1751) `hive.engine.enabled` drives the value for
these properties for HiveCatalog tables.
For Hive tables above Iceberg tables stored in other Iceberg Catalogs these
values are still not set. So it might worth to move this change for the
non-HiveCatalog codepath here:
https://github.com/apache/iceberg/blob/05752965f5eb453d895db44fb2d072d270646644/mr/src/main/java/org/apache/iceberg/mr/hive/HiveIcebergMetaHook.java#L78-L81
This would allow Impala to read tables created through Hive but stored in
non-HiveCatalogs.
For tables created from Hive and stored in HiveCatalog we set
`hive.engine.enabled` to `true` here:
https://github.com/apache/iceberg/blob/05752965f5eb453d895db44fb2d072d270646644/mr/src/main/java/org/apache/iceberg/mr/hive/HiveIcebergMetaHook.java#L133-L137
Setting `hive.engine.enabled` for other Iceberg Catalogs does not have any
affect, so I think we are good there.
Thanks,
Peter
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
