One thing worth noting is that the Iceberg discussion in PR #15280( https://github.com/apache/iceberg/pull/15280) and the dev@ thread discussion[1] seems to have evolved away from the earlier stagingSession idea and toward a stateless storage refresh token model. Server-side sessions/state were discarded due to the complexity and the desire to support stateless catalog implementations. To me, the storage refresh token also feels better since it ties to the storage credential itself rather than a specific staged create workflow. It avoids maintaining server-side session and state and easier to extend to other refreshing use cases in the future.
If we want to align with the Iceberg direction, the storage refresh token should be closer to that long-term vision. We may table a staged-create specific session or lease model now. 1. https://lists.apache.org/thread/35lj7vhtqtwl5nv9rzpln4mw4fbh7gdp Yufei On Fri, May 29, 2026 at 9:01 AM Robert Stupp <[email protected]> wrote: > Hi all, > > One thing to add: the “different refresh URI” option is usable today, > but mostly for Iceberg Java clients. > > Iceberg/Java FileIO implementations for S3, GCS and ADLS already have > provider-specific refresh endpoint config properties. > So Polaris could return a URI specific to each staged-create in the > LoadTableResult and that endpoint could vend fresh credentials for the > staged location/context. > > But that URI cannot be the proof by itself. > It either needs to contain a signed, short-lived staged operation token, or > reference server-side staged/lease state that Polaris validates. > This requires either cryptographic infrastructure or persisted state. > > It's probably worth pointing out: this is not a portable Iceberg REST > solution. > The Iceberg spec work was apache/iceberg#15280 [1], moving toward a > StorageCredential refresh-token model. > That PR was closed stale, not rejected, so there is no standard > cross-client contract yet. > > So maybe the practical path might be: > Use the Java hook for now, but model the Polaris side as a staged > operation/lease with cryptographic or server-side proof, so it can map > cleanly to the Iceberg refresh-token design if/when that comes back. > > Robert > > [1] https://github.com/apache/iceberg/pull/15280 > > > On Fri, May 29, 2026 at 1:20 AM Dmitri Bourlatchkov <[email protected]> > wrote: > > > Hi All, > > > > As Sung highlighted in today's Comminity Sync call, there is a general > > problem with refreshing credentials vended in the context of a Staged > > Create operation (usually CTAS). > > > > The same problem is beling discussed in the Iceberg community [1] > > > > Recap: > > > > * It is generally not possible for a client to refresh credentials > received > > from the initial Staged Create REST API call because the related table is > > not created until commit, thus leading to 404 response from the regular > > /credentials endpoint > > > > * Conceptually, AuthZ checks required on refreshing these credentials are > > different between the initial request and the refresh. > > > > The initial request implies that no data files exist yet and as such > > resembles an ordinary CREATE TABLE request. > > > > When the refresh credential call comes in, some data files are likely to > > exist already. Therefore, the credential refresh request is more similar > to > > an UPDATE from the AuthZ perspective. > > > > Optoins: > > > > * Wait for the Iceberg REST API spec change. > > > > This will allow associating a particular Stage Create "session" with a > > particular client. > > > > Yet, this does not address the AuthZ aspect at all. > > > > * Pre-register a Table Entity at Staged Create time. > > > > This will reserve the entity name for the anticipated commit and allow > the > > Polaris Authorizer to distinguish subsequent requests for refreshing > > credentials from the initial "create" request. > > > > * Use a different credential refresh URI for Staged Create environments. > > > > This URI is part of the Staged Create REST response, IIRC, and can be > > expected to override other client-side configs. The URI can use custom > path > > segments or query parameters to identity the specific Staged env. > context. > > > > * Something else? Please share ideas. > > > > [1] https://lists.apache.org/thread/35lj7vhtqtwl5nv9rzpln4mw4fbh7gdp > > > > Thanks, > > Dmitri. > > >
