difin commented on code in PR #6474:
URL: https://github.com/apache/hive/pull/6474#discussion_r3455665616
##########
iceberg/iceberg-handler/src/main/java/org/apache/iceberg/mr/hive/HiveTableUtil.java:
##########
@@ -244,6 +244,7 @@ public static Table deserializeTable(Configuration config,
String name) {
table = readTableObjectFromFile(location, config);
}
checkAndSetIoConfig(config, table);
+ IcebergVendedCredentialUtil.applyFromJobConf(table, config);
Review Comment:
You're right that most Iceberg clients don't need to ser/de credentials
themselves. Hive does, because we serialize the Iceberg Table
(SerializableTable) into JobConf for Tez/LLAP, and vended credentials on FileIO
typically don't survive that round-trip. Executors rebuild the table from job
conf and don't re-run REST `loadTable`, so we propagate credentials separately
(`VENDED_STORAGE_CREDENTIALS` + `S3A` bucket keys) and restore them in
`deserializeTable` via `applyFromJobConf`.
There is one place where the vended credentials can be updated:
`withConfigurationOverrides` method. REST catalogs can vend connectivity
settings from their network view (e.g. `http://minio:9000` when REST catalog
runs in Docker), while Hive session config sets a host-reachable endpoint
(`iceberg.catalog.ice01.s3.endpoint`=`http://host:9000`). This method overrides
only non-secret fields (`s3.endpoint`, `s3.path-style-access`) so Iceberg
`FileIO` and S3A agree on connectivity; vended keys are preserved. It runs at
both store time (`propagateToJob`, so the blob on executors is self-contained)
and restore time (`applyFromJobConf`, e.g. when commit still has the
catalog-internal endpoint on FileIO from `loadTable`).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
