talatuyarer commented on PR #17980: URL: https://github.com/apache/iceberg/pull/17980#issuecomment-5572344801
Thanks @pvary — that history matches what I found in #8555. On delegation tokens: I looked into that. I don't think the DT framework can carry the vended storage credentials themselves, providers are configured from cluster config at startup and push to all TMs, while vending is per-table and per-job, the catalog URI/auth lives in the job graph, and with the dynamic sink the table set isn't even known upfront). Tokens for job A's tables would land on TMs running only job B, which loses the per-table scoping that's the point of vending. But I think you're right that DT is the natural transport for one piece: the **catalog session token**. Today each task-side provider must re-authenticate to the catalog from serialized properties, which means either shipping a client secret/token to every TM. That single credential is exactly DT-shaped, one token, obtained and renewed on the JM, broadcast to all TMs. So the combination would be: DT distributes the catalog token, the credentials endpoint vends the per-table storage credentials. Wdyt ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
