difin commented on code in PR #6474:
URL: https://github.com/apache/hive/pull/6474#discussion_r3455665616
##########
iceberg/iceberg-handler/src/main/java/org/apache/iceberg/mr/hive/HiveTableUtil.java:
##########
@@ -244,6 +244,7 @@ public static Table deserializeTable(Configuration config,
String name) {
table = readTableObjectFromFile(location, config);
}
checkAndSetIoConfig(config, table);
+ IcebergVendedCredentialUtil.applyFromJobConf(table, config);
Review Comment:
You're right that most Iceberg clients don't need to ser/de credentials
themselves. Hive does, because we serialize the Iceberg Table
(`SerializableTable`) into `JobConf` for Tez/LLAP, and vended credentials on
`FileIO` typically don't survive that round-trip. Executors rebuild the table
from job conf and don't re-run REST loadTable, so we propagate credentials
separately (`VENDED_STORAGE_CREDENTIALS` + S3A bucket keys) and restore them in
`deserializeTable` via `applyFromJobConf`.
The main place we mutate vended credential content is
`withConfigurationOverrides()` method. REST catalogs can vend connectivity
settings from their network view (e.g. `http://minio:9000` when the catalog
runs in Docker), while Hive session config sets a host-reachable endpoint
(`iceberg.catalog.ice01.s3.endpoint=http://host:9000`). That method overrides
only non-secret fields (`s3.endpoint`, `s3.path-style-access`) so Iceberg
`FileIO` and S3A agree on connectivity; vended keys are preserved. It runs at
both store time (`propagateToJob`, so the blob on executors is self-contained)
and restore time (`applyFromJobConf`, e.g. when commit still has the
catalog-internal endpoint on `FileIO` from loadTable).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
