Hello, Following up on GitHub issue #5311 <https://github.com/apache/polaris/issues/5311> (policy / generic-table list pagination): listGenericTables (#5533 <https://github.com/apache/polaris/issues/5533>) and listPolicies (#5557 <https://github.com/apache/polaris/issues/5557>) are merged. The only remaining gap is GET /polaris/v1/{prefix}/applicable-policies, which declares page-token, page-size, and next-page-token in the spec but still returns the full assembled result set.
I plan to implement the remaining work and would like consensus on how pagination should behave for this endpoint before opening a PR. Background ---------- Unlike listPolicies, getApplicablePolicies does not map to a single backing-store listing. PolicyCatalog#getEffectivePolicies walks the entity path (catalog / namespace / table), loads policy mappings per level, merges inheritable vs non-inheritable policies, and returns a List<ApplicablePolicy> that PolicyCatalogHandler currently wraps in a HashSet (no stable order, no continuation token). Proposal (for discussion) ------------------------- 1. Page boundary: After computing the full effective set for the request (same semantics as today), apply pagination in the service layer: sort into a deterministic order, return page-size items, and expose next-page-token when more items remain. The page token would encode an opaque cursor (e.g. offset into the sorted snapshot), not a metastore PageToken. This honors the REST contract and LIST_PAGINATION_MAX_PAGE_SIZE behavior without changing inheritance semantics. It does not reduce server work for large hierarchies. 2. Stable ordering: Replace HashSet assembly with a sorted list before slicing. Candidate sort key (lexicographic): policy identifier (namespace levels, then policy name), then policy type, then inheritable flag / source if needed for uniqueness. The response schema keeps uniqueItems. 3. Adapter/handler: Thread pageToken and pageSize through PolicyCatalogAdapter#getApplicablePolicies into PolicyCatalogHandler#getApplicablePolicies, mirroring listPolicies (PageToken.build, rejectIncompleteListing, next-page-token on the response). Questions for the list ---------------------- 1. Is application-level pagination (full compute + sort + slice) an acceptable v1 for this endpoint, given there is no single store listing to paginate? 2. What total order should clients rely on across pages? Is sorting by PolicyIdentifier (namespace + name) then policy type sufficient, or should we document a different key? 3. Any objection to encoding the continuation as an opaque in-memory cursor rather than reusing metastore PageToken encoding? 4. Scope: Should this thread cover only applicable-policies, or also any CLI follow-up (#5385-style) once the server returns next-page-token? Please let me know if this direction sounds reasonable. Thanks, Arnab
