Hi Yufei, > Could you share more details about the workload identity use case?
Clients (Engines) may have ambient credentials in the execution env., which allows them to access storage (e.g. GCS). In this case, Polaris does not have to vend credentials. However, with the current GCS storage integration, Polaris cannot avoid vending credentials. If the admin user elects to use workload identity for clients (ambient credentials), Polaris credential vending work is useless overhead. Moreover it may present a security gap by exposing different credentials to clients, which are supposed to authenticate via another (workload identity) channel. I believe it is important for the admin user to be able to minimize credential exposure via Polaris in this case. > I think we need a concrete use case to decide whether we need this [...] >From my POV the use case I described above is self-sufficient. A similar practical use case relates to your reasonable and timely proposal to limit the use of SKIP_CREDENTIAL_SUBSCOPING_INDIRECTION [2]. If we are to take the next step and adapt our Integration Test to act as examples of how to avoid relying on this flag, this results in certain coding difficulties, which are evident [1] in [5884]. > [...] where the setting should go. The natural place for a new setting that controls access to derived credentials seems to be in the type-specific storage config [5884], as I proposed in my original email in this thread. Do you have an alternative suggestion for how to address this use case? Thanks, Dmitri. [1] https://github.com/apache/polaris/pull/5756#discussion_r4362688654 [2] https://lists.apache.org/thread/1trcnbm04zzqztpxkzrzs1pvrlyd49bg [5756] https://github.com/apache/polaris/pull/5756 [5884] https://github.com/apache/polaris/pull/5884 On Mon, Nov 2, 2026 at 3:31 PM Yufei Gu <[email protected]> wrote: > Hi Dmitri, > > Thanks for clarifying. My earlier comment was about clients choosing > not to request credential vending through X-Iceberg-Access-Delegation. > I understand that this is a server-side admin decision, which differs > from clients opting out. > > Could you share more details about the workload identity use case? I’m > still not clear why admins would need to disable credential vending in > that scenario. > > stsUnavailable is for S3-compatible storage without STS support. If > there’s a similar limitation for Azure/GCS, these settings make sense > to me. But if this is an admin policy, I’m not sure storage > configuration is the right place, given that a storage config may > apply to multiple tables or namespaces in the future (Rishi's > proposal). > > I think we need a concrete use case to decide whether we need this and > where the setting should go. WDYT? > > > Yufei > > On Fri, Oct 30, 2026 at 3:17 PM Dmitri Bourlatchkov via dev > <[email protected]> wrote: > > > > Hi Yufei, > > > > > [...] we could use the header to achieve the same result. > > > > The proposed change allows admin users to configure Polaris to avoid > > vending credentials in Azure and GCS environments. It is a deliberate > > server-side admin decision. > > > > This was possible for S3 before. PR [5884] aligns Polaris capabilities in > > Azure and GCS with existing Polaris capabilities in S3 environments. > > > > What alternative do you suggest? I'm not sure I can see that in JB's or > > Ayush's messages. > > > > Thanks, > > Dmitri. > > > > [5884] https://github.com/apache/polaris/pull/5884 > > > > On Thu, Oct 29, 2026 at 7:30 PM Yufei Gu <[email protected]> wrote: > > > > > Apologies for the late reply. > > > > > > Thanks Dmitri for raising this, and Vignesh for working onit. As Ayush > and > > > JB mentioned, we could use the header to achieve the same result. Do we > > > want to introduce another spec change, or am I missing some use cases? > > > > > > Yufei > > > > > > > > > On Thu, Oct 29, 2026 at 2:05 PM Dmitri Bourlatchkov <[email protected]> > > > wrote: > > > > > > > Hi All, > > > > > > > > PR [5884] seems to be in a good shape. As far as I can tell, all > existing > > > > points have been addressed. > > > > > > > > If there are no objections, I propose merging it on Oct 30. > > > > > > > > Thanks, > > > > Dmitri. > > > > > > > > [5884] https://github.com/apache/polaris/pull/5884 > > > > > > > > On Tue, Oct 27, 2026 at 5:46 PM Dmitri Bourlatchkov < > > > > [email protected]> wrote: > > > > > > > > > Hi JB, Ayush, > > > > > > > > > > Good points re: DefaultAccessDelegationModeResolver! > > > > > > > > > > Vignesh contributed PR [5884] to implement the new properties. > This PR > > > > > adjusts the logic in DefaultAccessDelegationModeResolver too. > Kudos to > > > > > Vugnesh! > > > > > > > > > > As far as I can tell, after [5884] if a client requests > > > > > "vended-credentials" but the server is configured > > > > > with skipCredentialDownscoping = true (for example), then the > client > > > will > > > > > receive a clear error message indicating that credential vending > is not > > > > > available. > > > > > > > > > > This is related to the Iceberg API clarification vote [1]. If that > spec > > > > > change passes in Iceberg, Polaris will be able to support "none" > as a > > > > > vending mode for a better client config experience, IMHO. > > > > > > > > > > In any case, please review [5884]. I think it still makes sense in > > > > Polaris > > > > > even without the Iceberg spec change. > > > > > > > > > > In Polaris code the main use case may be in tests, but some users > may > > > > want > > > > > to consider the new options for workload identity-based deployments > > > too. > > > > > > > > > > [1] > https://lists.apache.org/thread/hc2764vozz7ygx5fjj64104hotrof0lk > > > > > > > > > > [5884] https://github.com/apache/polaris/pull/5884 > > > > > > > > > > Cheers, > > > > > Dmitri. > > > > > > > > > > On Sun, Oct 25, 2026 at 5:53 AM Jean-Baptiste Onofré < > [email protected]> > > > > > wrote: > > > > > > > > > >> Hi all, > > > > >> > > > > >> +1 on the catalog-level > AzureStorageConfigInfo.skipSasTokenGeneration > > > > and > > > > >> GcpStorageConfigInfo.skipCredentialDownscoping properties. A > typed, > > > well > > > > >> defined setting is a better home for "don't vend" than the > > > > >> SKIP_CREDENTIAL_SUBSCOPING_INDIRECTION flag, which only applies > at the > > > > >> realm level. It also matches what stsUnavailable already does for > S3. > > > > >> > > > > >> I agree with Ayush on the access delegation mode. > > > > >> > > > > >> I think it would be great to clarify two points before working on > the > > > > >> implementation PR: > > > > >> - If a catalog opts out of vending and the client asks only for > > > > >> vend-credentials, we should reject the request up front with a > clear > > > > >> error. > > > > >> Failing later inside the vending path is harder to diagnose imho. > This > > > > >> should apply to all three providers, so the S3 stsUnavailable case > > > > behaves > > > > >> the same way. If changing the S3 behavior is out of the scope for > the > > > > PR, > > > > >> we can track as a follow-up, but we need consistency here imho. > > > > >> - Both properties should default to false, so existing catalogs > keep > > > > >> vending credentials. They should be optional in the Management > API, so > > > > >> this > > > > >> is a backward compatible addition. We should also document them > next > > > to > > > > >> stsUnavailable (in site/content/in-dev/unreleased/), and say there > > > that > > > > >> they are meant for non-cloud or emulator deployments. > > > > >> > > > > >> Happy to review the PRs ;) > > > > >> > > > > >> Regards > > > > >> JB > > > > >> > > > > >> On Fri, Oct 23, 2026 at 8:15 PM Dmitri Bourlatchkov < > [email protected] > > > > > > > > >> wrote: > > > > >> > > > > >> > Hi All, > > > > >> > > > > > >> > As discovered during the review of PR 5756 [1], now > > > > >> > that SKIP_CREDENTIAL_SUBSCOPING_INDIRECTION is not recommended > for > > > > >> > mainstream use [3], deployments (e.g. tests) using non-cloud > options > > > > for > > > > >> > ADLS and GS (GCS) storage need a mechanism for opting out of > minting > > > > >> > session tokens and resolving to local (server and/or client) > > > > credentials > > > > >> > for storage access. > > > > >> > > > > > >> > I propose adding the following properties at the Management API > > > level > > > > >> [1]. > > > > >> > > > > > >> > * AzureStorageConfigInfo.skipSasTokenGeneration > > > > >> > > > > > >> > * GcpStorageConfigInfo.skipCredentialDownscoping > > > > >> > > > > > >> > Both names relate to the corresponding native APIs in each > > > technology. > > > > >> > > > > > >> > Setting these properties will effectively disable credential > > > vending. > > > > >> This > > > > >> > is similar to stsUnavailable in AwsStorageConfigInfo. > > > > >> > > > > > >> > The main purpose of the new properties is establishing a > > > catalog-level > > > > >> opt > > > > >> > in/out mechanism and providing well-defined user-level settings. > > > > >> > > > > > >> > PolarisStorageIntegration implementations still retain value > without > > > > >> > credential vending because they potentially define > non-credential > > > > >> > properties in StorageAccessConfig. The latter case currently > applies > > > > >> only > > > > >> > to S3, but downstream customizations may utilize it too. > > > > >> > > > > > >> > [1] > > > > https://github.com/apache/polaris/pull/5756#discussion_r4362688654 > > > > >> > > > > > >> > [2] > > > > >> > > > > > >> > > > > > >> > > > > > > > > https://github.com/apache/polaris/blob/5fafc67a832a8900fefb399abc00549d864470d6/spec/polaris-management-service.yml#L21 > > > > >> > > > > > >> > [3] > > > https://lists.apache.org/thread/1trcnbm04zzqztpxkzrzs1pvrlyd49bg > > > > >> > > > > > >> > Thoughts? > > > > >> > > > > > >> > Thanks, > > > > >> > Dmitri. > > > > >> > > > > > >> > > > > > > > > > > > > > > > -- > > > > > Dmitri Bourlatchkov > > > > > Senior Staff Software Engineer, Dremio > > > > > Dremio.com > > > > > < > > > > > > > > https://www.dremio.com/?utm_medium=email&utm_source=signature&utm_term=na&utm_content=email-signature&utm_campaign=email-signature > > > > > > > > / > > > > > Follow Us on LinkedIn <https://www.linkedin.com/company/dremio> / > Get > > > > > Started <https://www.dremio.com/get-started/> > > > > > > > > > > > > > > > The Agentic Lakehouse > > > > > The only lakehouse built for agents, managed by agents > > > > > > > > > > > > > > > > > > > > > > >
