Hi Sung, Good point about exceptions. I agree that it is preferable to avoid them (as denial indicators) on the filtering call paths.
Re: feature flags, I suppose we cloud model this as a two-layered check: 1) per-batch permissions (same as now) and 2) per-element include/exclude decisions. The first check is always "on", the second can be controlled by a feature flag the the AuthZ layer. Callers (REST services) always get the filtering flags, but if the feature is disabled, the flag are always "include". This way the code in REST services can be uniform across all use cases, which will simplify testing. I'm not sure you meant the same approach in your message, but I hope it makes sense :) WDYT? Cheers, Dmitri. On Mon, Jun 15, 2026 at 10:08 PM Sung Yun <[email protected]> wrote: > Hi Grace and Venkateshwaran, > > Thanks for the discussion. I agree with the general direction of > introducing authorizer-layer filtering and returning callers a list scoped > to what they can access. > > One thing I wanted to float: I think it would help to make the > authorization input for list operations deterministic, rather than relying > on a fallback path. By fallback I mean the handler running the normal > LIST_* check and, on failure, dropping into a filtered path. That works, > but it leans on exception-driven branching that I'd argue is harder to > reason about and test. > > The alternative would be to treat this as a mode, enabled by a feature > flag for filtering on list operations. When it's off, behavior is exactly > what we have today. When it's on, every caller goes through the same > listing/filtering path, and whether you see everything underneath just > falls out of that check (admins included, via the container short-circuit) > instead of being a separate branch keyed on who the caller is. > > That being said I think we are generally in agreement, and we could iron > out these mechanisms on PR reviews as well. > > Cheers > Sung > > On 2026/06/05 23:23:30 Yufei Gu wrote: > > The proposed behavior makes sense. Returning a list of catalogs only a > > caller can access feels more aligned with least privilege and with how > most > > users would expect catalog discovery to work. > > > > Implementation does not seem particularly difficult. The existing admin > > behavior could remain unchanged, while non-admin callers receive a > filtered > > list based on catalog level authorization checks similar to those already > > performed by getCatalog(). There are minor perf concerns, but I think it > > should be acceptable in general. To me, the bigger question is agreeing > on > > the API semantics rather than the implementation itself. > > > > Overall, I'm supportive of the change. > > > > Yufei > > > > > > On Thu, Jun 4, 2026 at 5:05 PM Dmitri Bourlatchkov <[email protected]> > wrote: > > > > > It looks like this reply got disconnected from the original thread. > > > > > > Posting a link to it for reference: > > > https://lists.apache.org/thread/w58zrjt6jpq33mv9lvqzndqc6no5l7xb > > > > > > Cheers, > > > Dmitri. > > > > > > On Sun, May 31, 2026 at 12:15 PM venkateshwaran cholan < > > > [email protected]> wrote: > > > > > > > Hi all, > > > > > > > > Thanks Grace for putting together the proposal, and thanks everyone > for > > > the > > > > detailed feedback and discussion so far. > > > > > > > > Dimas pointed me to this discussion from #4573, and I wanted to > share an > > > > observation after tracing the current implementation on main. > > > > > > > > The discoverability gap there (catalog-role access works for > getCatalog > > > and > > > > Iceberg operations, but listCatalogs requires root CATALOG_LIST) > looks > > > like > > > > one concrete case of the broader visibility model discussed here, > > > > particularly if CATALOG_READ_PROPERTIES ends up being the selected > > > > visibility privilege for catalogs. > > > > > > > > listCatalogs() still gates on > > > > authorizeBasicRootOperationOrThrow(LIST_CATALOGS) against the root > > > > container, while getCatalog() authorizes per catalog. That matches > the > > > > asymmetry described in the issue. > > > > > > > > I agree that authorizer-layer filtering is preferable to a > > > > listCatalogs-only special case so default RBAC, OPA, and other > > > authorizers > > > > stay consistent. > > > > > > > > One thing I am still unclear on for LIST_CATALOGS: the proposal's > scope > > > > short-circuit does not apply because there is no scope container > (unlike > > > > LIST_NAMESPACES / LIST_TABLES). > > > > > > > > For #4573-style discoverability, it seems we would want callers > without > > > > root CATALOG_LIST to get a filtered list instead of 403. > > > > > > > > Separately, when entity filtering is enabled, should callers with > root > > > > CATALOG_LIST still get a filtered list, or is root CATALOG_LIST > intended > > > to > > > > mean "see all catalogs" with filtering only handling the > discoverability > > > > case? > > > > > > > > Happy to help with regression coverage either way. For example, an > authz > > > > test where a principal has catalog-scoped CATALOG_READ_PROPERTIES > via a > > > > catalog role but no root CATALOG_LIST: getCatalog succeeds and > > > listCatalogs > > > > fails today, documenting the gap until filtering lands. > > > > > > > > Thanks, > > > > Venkateshwaran > > > > > > > > > >
