Hi Rajesh, Thanks for the use case description. I think it is quite reasonable.
I suppose the next step would be to develop a POC implementation for a more concrete review and to assess the impact on other use cases. Ideally, I think, the new feature should probably be under a config flag, enabled by the Polaris admin only when explicitly intended. Looking forward to your contribution :) Cheers, Dmitri. On Wed, Jul 1, 2026 at 3:36 PM Rajesh Bulleddula < [email protected]> wrote: > Hi Dmitri, > > Ideally, yes, we would want the child BU catalog to take the end user's > identity into account when determining the scope of the vended credentials. > Also, identity propagation would address the concern Yufei raised earlier > about unintentionally exposing the privileges associated with the > federation service account. But as you said, that would require the > central catalog to propagate the authenticated user's identity to the child > BU catalog on federated API calls which doesn't happen today. > > That said, I don't think identity propagation is a prerequisite for > credential pass-through, atleast for our usecases. > > Even with the current model, where the central catalog communicates with > the child catalog using a service account: > > > - The parent catalog remains responsible for authenticating the user and > enforcing RBAC. > - Once authorized, the child catalog can mint vended credentials for its > own storage scoped to user operation and return them to the parent > and back to > client. > > This still preserves the desired ownership boundary, where each child BU > catalog remains the authority for credential vending over the storage it > owns, without requiring the central catalog to have AssumeRole permissions > into every BU-managed storage account. > > About contributing to the feature, I will let you know in a couple days. > > On Tue, Jun 30, 2026 at 3:30 PM Dmitri Bourlatchkov <[email protected]> > wrote: > > > Hi Rajesh, > > > > Your federation use case makes sense to me. > > > > Do BU catalogs need to take the end user's identity into account when > > making decisions about the access scope of vended credentials? > > > > If yes, this will probably require the central catalog to propagate the > > user's identity to the BU catalog on all API calls. This current does not > > happen. BU catalogs are accessed by the central catalog under a "service > > account". > > > > All of this looks doable to me. Do you have the capacity to contribute > this > > feature? > > > > Thanks, > > Dmitri. > > > > On Tue, Jun 30, 2026 at 2:07 PM Rajesh Bulleddula < > > [email protected]> wrote: > > > > > Regarding vended credential pass-through ... > > > > > > In our environment, each Business Unit (BU) owns and manages its own > > object > > > storage. That includes: > > > > > > - > > > > > > Its own S3 buckets/prefixes > > > - > > > > > > The IAM roles used for credential vending > > > - > > > > > > Trust policies > > > - > > > > > > Storage access policies and governance > > > > > > Accordingly, each BU Polaris catalog has permission to assume only its > > own > > > storage access role and can mint appropriately scoped vended > credentials > > > for its own data. > > > > > > From our perspective, the central enterprise catalog is responsible for > > > providing a unified discovery and access endpoint. Clients interact > only > > > with the central catalog, which performs authentication and RBAC before > > > forwarding requests to the appropriate child BU catalog. > > > > > > In this model, the central catalog doesn't need direct access to every > > BU's > > > object storage. Instead, the child BU catalog which already owns the > > > storage integration, returns both the Iceberg metadata and vended > > > credentials scoped to the operation (read/write). > > > > > > If the central catalog is required to mint credentials itself, it would > > > need permission to assume roles for every BU's storage. In large > > > organizations, this significantly broadens the trust boundary. Many > > > organizations have security controls and organizational policies that > > > discourage or prohibit granting a central service principal > cross-account > > > assume-role access into every BU-managed storage account. > > > > > > From our perspective, credential pass-through preserves clear ownership > > > boundaries: > > > > > > - > > > > > > The central catalog remains responsible for authentication, > > > authorization, and request routing. > > > - > > > > > > Each BU child catalog remains responsible for storage authorization > > and > > > credential vending for the storage it owns. > > > > > > The motivation is primarily around security boundaries and operational > > > ownership. This aligns well with our intended federation model, where > > each > > > child BU catalog is the authority for its own storage while the central > > > catalog provides a single entry point for consumers. > > > > > > > > > On Tue, Jun 30, 2026 at 12:42 PM Dmitri Bourlatchkov <[email protected] > > > > > wrote: > > > > > > > Hi Rajesh, > > > > > > > > The current situation regarding vended credentials in federated > > catalogs > > > > has been explained in the linked GG discussion, I believe :) > > > > > > > > Re: future direction, you're part of the community too :) > > > > > > > > I wonder whether vended credential pass-through is something that is > > > > beneficial to your use cases... just trying to understand the > situation > > > > better. > > > > > > > > Why would you want the "BU" catalog to control credential vending? > > > > > > > > Thanks, > > > > Dmitri, > > > > > > > > On Tue, Jun 30, 2026 at 11:37 AM Rajesh Bulleddula < > > > > [email protected]> wrote: > > > > > > > > > Following the discussion on GitHub, starting a dev thread on > > > credential > > > > > vending behavior in Apache Polaris catalog federation. > > > > > > > > > > https://github.com/apache/polaris/discussions/4929 > > > > > > > > > > I am evaluating catalog federation behavior in a data lakehouse > setup > > > > using > > > > > Apache Iceberg with Apache Polaris. > > > > > > > > > > Architecture: > > > > > > > > > > - One central enterprise catalog > > > > > - Multiple BU (Business Unit) level child catalogs federated > into > > > the > > > > > central catalog > > > > > > > > > > Expected flow (based on my understanding): > > > > > > > > > > - Client queries an Iceberg table through the central catalog > > > > > - Central catalog forwards the request to the appropriate BU > > catalog > > > > > after RBAC validation > > > > > - BU catalog returns table metadata and vended credentials > > > > > - Central catalog returns the BU catalog’s metadata and the same > > > > vended > > > > > credentials back to the client > > > > > > > > > > Actual behavior observed: > > > > > > > > > > - BU catalog does return metadata + vended credentials > > > > > - Central catalog drops/ignores the BU-provided vended > credentials > > > > > - Central catalog generates new vended credentials and sends > that > > to > > > > the > > > > > client instead > > > > > > > > > > Question: > > > > > > > > > > What is the intended credential vending model for catalog > federation? > > > > > > > > > > - Should the central catalog propagate the vended credentials > > returned > > > by > > > > > the child catalog to the client (credential pass-through)? > > > > > > > > > > Or > > > > > > > > > > - Should the central catalog always generate and return its own > > vended > > > > > credentials, regardless of whether the child catalog has already > > vended > > > > > credentials? > > > > > > > > > > I'm interested in understanding the intended design philosophy for > > > > > federation and whether supporting vended credential pass-through is > > > > > something the community believes should be part of Polaris. > > > > > > > > > > -- > > > > > Thanks & Regards, > > > > > Rajesh Bulleddula > > > > > > > > > > > > > > > > > > -- > > > Thanks & Regards, > > > Rajesh Bulleddula > > > > > > > > -- > Thanks & Regards, > Rajesh Bulleddula >
