Hi all, One thing to add: the “different refresh URI” option is usable today, but mostly for Iceberg Java clients.
Iceberg/Java FileIO implementations for S3, GCS and ADLS already have provider-specific refresh endpoint config properties. So Polaris could return a URI specific to each staged-create in the LoadTableResult and that endpoint could vend fresh credentials for the staged location/context. But that URI cannot be the proof by itself. It either needs to contain a signed, short-lived staged operation token, or reference server-side staged/lease state that Polaris validates. This requires either cryptographic infrastructure or persisted state. It's probably worth pointing out: this is not a portable Iceberg REST solution. The Iceberg spec work was apache/iceberg#15280 [1], moving toward a StorageCredential refresh-token model. That PR was closed stale, not rejected, so there is no standard cross-client contract yet. So maybe the practical path might be: Use the Java hook for now, but model the Polaris side as a staged operation/lease with cryptographic or server-side proof, so it can map cleanly to the Iceberg refresh-token design if/when that comes back. Robert [1] https://github.com/apache/iceberg/pull/15280 On Fri, May 29, 2026 at 1:20 AM Dmitri Bourlatchkov <[email protected]> wrote: > Hi All, > > As Sung highlighted in today's Comminity Sync call, there is a general > problem with refreshing credentials vended in the context of a Staged > Create operation (usually CTAS). > > The same problem is beling discussed in the Iceberg community [1] > > Recap: > > * It is generally not possible for a client to refresh credentials received > from the initial Staged Create REST API call because the related table is > not created until commit, thus leading to 404 response from the regular > /credentials endpoint > > * Conceptually, AuthZ checks required on refreshing these credentials are > different between the initial request and the refresh. > > The initial request implies that no data files exist yet and as such > resembles an ordinary CREATE TABLE request. > > When the refresh credential call comes in, some data files are likely to > exist already. Therefore, the credential refresh request is more similar to > an UPDATE from the AuthZ perspective. > > Optoins: > > * Wait for the Iceberg REST API spec change. > > This will allow associating a particular Stage Create "session" with a > particular client. > > Yet, this does not address the AuthZ aspect at all. > > * Pre-register a Table Entity at Staged Create time. > > This will reserve the entity name for the anticipated commit and allow the > Polaris Authorizer to distinguish subsequent requests for refreshing > credentials from the initial "create" request. > > * Use a different credential refresh URI for Staged Create environments. > > This URI is part of the Staged Create REST response, IIRC, and can be > expected to override other client-side configs. The URI can use custom path > segments or query parameters to identity the specific Staged env. context. > > * Something else? Please share ideas. > > [1] https://lists.apache.org/thread/35lj7vhtqtwl5nv9rzpln4mw4fbh7gdp > > Thanks, > Dmitri. >
