singhpk234 commented on PR #13225: URL: https://github.com/apache/iceberg/pull/13225#issuecomment-4372418010
Thanks for the response @smaheshwar-pltr @ggershinsky ! Unlike other catalogs, REST catalog works on minting creds at table which it expects the client to use to do subsequent operations, we have a dedicated `/credentials` endpoint which works on refreshing this creds so this machinery is in place already. Now the concern for me the following : 1. How do we handle things like VAULT, given rest catalog wants to return per table level creds to talk to vault store (this came up recently), so far this PR is advocating storage creds contains the creds of KMS too, but if we end up supporting vault, this can't be mixed with storage creds right (side note : catalogs have been mixing KMS creds with this obj for things like SSE which entirely an object store level thing), i believe we should conclude this dicussion, let me start a thread for this 2. If catalog returns you cred, it overrides the client side creds, so this means the catalog which is not aware of encryption and is just treating metadata as is will have no clue it needs to vend creds (naively thinking) also client should put additional assertion that the creds returned are valid since there is encryption enabled on the table otherwise fail directly rather than during the execution. 3. Backward compatibility : lets say we release this client and we expect storage will always contain this creds for kms too but later in protocol we introduce a new dedidcate field in loadTable response, we will not need to think of backward compatibilty to support both the ways of returning creds. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
