+1, thanks Grace for the proposal! I agree this will be a very useful opt-in feature.
We should be able to describe the model in terms of (possibly extending) the existing privilege hierarchy. For reference, here's a handy summary of the existing privilege design: https://docs.google.com/document/d/1fq7buKnsYKsf0MwyoeGLsG14zhGTpsKiryhgSbi-ChY/edit?tab=t.0 Given the way the privilege-hierarchy interacts with the container-hierarchy for privilege inheritance, technically the caller is already proven to have the TABLE_LIST privilege on the target table, by virtue of having it on the parent namespace. This means the idea of checking for just "any" privilege isn't quite well-defined for this use case. Instead, there are two options that fit in well with the existing model: 1. Make this option check the type-specific *_READ_PROPERTIES privilege for filtering purposes Pros: -Don't need to introduce any new privilege enum -Matches the actual behavior of Admin-API list operations (listCatalogs, listPrincipals, listPrincipalRoles, listCatalogRoles) because these diverge from the Iceberg-layer listNamespaces/listTables by already returning the full metadata Cons: -Per the current privilege model, in theory someone could have TABLE_DROP privilege on individual tables, which is intentionally *not* a parent privilege of TABLE_READ_PROPERTIES; in this case having TABLE_DELETE anywhere in the parent hierarchy would *not* allow actually seeing the table in the list. It's debatable whether we consider this working as intended or a shortcoming 2. Add a new privilege like *_REFERENCE for each type, make it have parent privileges *_READ_PROPERTIES and *_DROP Pros: -Can list tables that you either have READ_PROPERTIES *or* DROP on Cons: -Need to introduce new privilege enum The benefit of either of these approaches vs any custom privilege check is that the hierarchy generally works in our favor. For example, instead of needing to special-case looking for "management privileges" to short-circuit the filtering, we actually just check the specified *_READ_PROPERTIES or *_REFERENCE privilege *on* the catalog-level. This way, if someone was indeed granted TABLE_READ_PROPERTIES at the catalog-level without "management privileges", we should still be able to short-circuit any filtering to let them see any tables they list. To make this efficient, batch operations might indeed be needed in some form. The most basic way of doing this semi-efficiently is to at least use the Resolver interface to queue up the listed entities. Then if they're pulled into the EntityCache then the only call to the persistence layer is batched up for getting all the entityVersions. A second option would be using the relatively recently-added loadResolvedEntities method: https://github.com/apache/polaris/blob/6c1a12bfd25a77b34e9e803295cc92492dda47b6/polaris-core/src/main/java/org/apache/polaris/core/persistence/PolarisMetaStoreManager.java#L433 A third option would be pushing the request for fully-resolved entities (e.g. fetching the grants) into the original list call, basically adding a variation of listFullEntities: https://github.com/apache/polaris/blob/6c1a12bfd25a77b34e9e803295cc92492dda47b6/polaris-core/src/main/java/org/apache/polaris/core/persistence/PolarisMetaStoreManager.java#L149 It'd be something like listFullResolvedEntities - similar to loadResolvedEntities, except the method argument is the parent container instead of a list of entityIds, and then the persistence layer would be expected to load the grants alongside the entities in conjunction with its paginated list. A default persistence implementation can still just implement it as a second pass with loadResolvedEntities, but this way there's room for the persistence layer to be smarter about getting the entities with their grants in a single pass during list-iteration. I agree the batching option will need more thought/discussion in any case. On Tue, Jan 13, 2026 at 6:38 PM Yufei Gu <[email protected]> wrote: > Thanks for the proposal. Enabling privilege aware listing is the right > direction. > > Per entity authorization does introduce performance concerns, but I do not > see this as a blocker, and I don't consider batching authZ solves all > problems given that the number of entities is not bounded. > > Instead, it makes more sense to tie list filtering with pagination, which > provides a natural bound on the amount of authorization work performed per > request. With filtering enabled, the overhead scales with page size and > remains predictable and tunable by deployment. For this reason, it makes > more sense to require filtering to be used only with paginated list > operations, so the authorization cost stays bounded. This is consistent > with existing catalog services such as AWS Glue and Databricks Unity > Catalog, which rely on mandatory pagination and the ways of permission > check vary based on use cases. > > Page size effectively becomes the tuning mechanism for balancing latency > and throughput. Deployments with high latency authorizers may reasonably > choose to reduce page size, disable filtering, or rely on authorizer side > batching or caching to mitigate the overhead. > > Yufei > > > On Tue, Jan 13, 2026 at 5:08 PM Dmitri Bourlatchkov <[email protected]> > wrote: > > > Hi Grace, > > > > It definitely makes sense to enhance Polaris to provide authorization > > protection for listing individual entities. So thanks a lot for starting > > this discussion! > > > > As Robert noted, practical aspects of implementing this right now may be > > challenging and require substantial refactorings. > > > > Please do not consider this as discouragement, though :) If you have some > > specific code changes in mind, please feel free to share them as a PR and > > we can see how to make incremental progress. > > > > Are you thinking of making an end-to-end solution using the Polaris > native > > RBAC or something like OPA? > > > > I'd also propose to discuss this in the next Community Sync meeting [1] > > > > [1] https://groups.google.com/g/polaris-community-sync > > > > Cheers, > > Dmitri. > > > > On Mon, Jan 12, 2026 at 5:15 PM Zhiyang Chen <[email protected]> > wrote: > > > > > Hi everyone, > > > > > > I'd like to propose a change on how Polairs handles authorization for > > list > > > operations (LIST_CATALOGS, LIST_NAMESPACES, LIST_TABLES). > > > > > > Today, Polaris checks if a user has LIST_* privilege on the parent > > entity, > > > it doesn’t filter which child entities the user has access to. For > > example, > > > if a user has LIST_TABLES privilege on the namespace, all the tables > will > > > return, regardless of the tables that user has access to. > > > > > > This leads to two issues: > > > 1. Discoverability: a user may have access to one or more entities > (such > > > as tables) within a namespace, but not have the namespace-level LIST_* > > > privilege. In this case, the user cannot list or discover any of the > > > entities they are allowed to access. > > > > > > 2. Visibility of Unauthorized Entities: a user may have LIST_* > privileges > > > on a parent entity but not have access to all the child entities within > > it. > > > In this case, list operations return entities the user is not > authorized > > to > > > access, exposing unauthorized entity names and metadata. > > > > > > The proposal adds optional entity-level filtering at the > > PolarisAuthorizer > > > layer so that list results better reflect the access permissions. > > Existing > > > behavior would remain available and filtering would be opt-in. > > > > > > Attaching the short design doc with details: > > > > > > https://docs.google.com/document/d/1PyIISgK2FVK0m8t4104KZpKAWb4d1Wy0UhZQzut4pIc/edit?usp=sharing > > , > > > would like to hear your thoughts on this. > > > > > > Thanks, > > > Grace > > > > > >
