deniskuzZ commented on code in PR #6789:
URL: https://github.com/apache/hive/pull/6789#discussion_r4247799573
##########
iceberg/iceberg-handler/src/main/java/org/apache/iceberg/mr/hive/HiveTableUtil.java:
##########
@@ -259,6 +261,71 @@ public static Table deserializeTable(Configuration config,
String name) {
return table;
}
+ /**
+ * Resolves the Iceberg {@link Table} for split generation ({@code
IcebergInputFormat#getSplits}).
+ *
+ * <p>Serialized tables ({@link SerializableTable}) only carry a metadata
snapshot and produce
+ * {@link org.apache.iceberg.DataTableScan} (client-side manifest planning).
When REST catalog
+ * server-side scan planning is enabled, reload the live table from the
catalog so
+ * {@code table.newScan()} returns {@link
org.apache.iceberg.rest.RESTTableScan} and issues
+ * {@code POST /plan} on the REST server.
+ *
+ * <p>Intra-transaction read-after-write ({@link
InputFormatConfig#TABLE_METADATA_LOCATION}) still
+ * uses the deserialized snapshot so uncommitted metadata is visible.
+ */
+ public static Table resolveTableForScanPlanning(Configuration conf, String
tableIdentifier) {
+ if (shouldReloadForServerSideScanPlanning(conf)) {
+ Table table = Catalogs.loadTable(conf);
+ checkAndSetIoConfig(conf, table);
+ IcebergVendedCredentialUtil.applyFromJobConf(table,
resolveCatalogName(conf), conf);
+ return table;
+ }
+
+ Table table = deserializeTable(conf, tableIdentifier);
+ if (table == null) {
+ table = Catalogs.loadTable(conf);
Review Comment:
The two `loadTable` paths treat credentials differently: the server-planning
branch calls `IcebergVendedCredentialUtil.applyFromJobConf(...)`, the
`deserializeTable == null` fallback doesn't, even though both load from the
same catalog via `Catalogs.loadTable(conf)`.
Could this be a single load path? E.g. skip deserialization when server-side
planning applies and fall through to one `loadTable` + `checkAndSetIoConfig` +
`applyFromJobConf` block:
````
Table table = shouldReloadForServerSideScanPlanning(conf) ? null :
deserializeTable(conf, tableIdentifier);
if (table == null) {
table = Catalogs.loadTable(conf);
checkAndSetIoConfig(conf, table);
IcebergVendedCredentialUtil.applyFromJobConf(table,
resolveCatalogName(conf), conf);
}
return table;
````
Two things to confirm with that shape:
- `applyFromJobConf` is a no-op when the job conf has no vended credentials
(the fallback also runs for non-REST catalogs);
- which credentials should win when a fresh REST load already returns vended
credentials in `LoadTableResponse` — the fresh ones, or the ones HS2 vended at
compile time?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
