Hi JB,

> I would state in the spec that an empty pageToken means the
> same as no pageToken.

Good point! +1

> LIST_PAGINATION_MAX_PAGE_SIZE is unlimited by default [...]

Also a very good point!

In practice I think LIST_PAGINATION_MAX_PAGE_SIZE applies to the IRC API
and acts as a safety valve to be configured by Polaris users.

I think new APIs that use Rishi's (and compatible) spec wording probably
need a _different_ config for limiting page sizes, and it has to be ON by
default.

I think this can be resolved in implementation PRs. The spec seems to be
unaffected by this.

I believe Anand already took this approach in [4115].

> Regarding getApplicablePolicies [...]

Let's have a separate discussion for that. I believe we cannot break this
API contract now that we have released it without any "beta" or
"experimental" annotations... that is per our evolutions guidelines [1]

Cheers,
Dmitri.

[1] https://polaris.apache.org/releases/1.9.0/evolution/

[4115] https://github.com/apache/polaris/pull/4115

On Sat, Nov 14, 2026 at 2:33 AM Jean-Baptiste Onofré <[email protected]>
wrote:

> Hi Rishi,
>
> I'm +1 on the approach. I don't think it conflicts with what we agreed in
> September. Fail-closed protects clients written against the Iceberg promise
> that a request with no pageToken gets the complete result. A new Polaris
> native operation makes no such promise, so "always follow the token" is the
> cleaner contract, and it bounds response size from day one. One shared
> definition is also better than a copy per operation.
>
> We had a similar discussion about pagination in the Directory Spec PR.
>
> I have a few comments (nothing blocking PR #6028):
> - As Prithvi noted in the September thread, Java Iceberg clients send an
> empty pageToken to start paginating. These operations always allow
> pagination, so I would state in the spec that an empty pageToken means the
> same as no pageToken. Then a client with Iceberg habits doesn't get a 400.
> - The storage configuration listings have filters (targetExists,
> referenced). It would be good to say that a client must send the same
> parameters on every page, apart from pageToken and pageSize. The spec could
> also say the server may reject a token that comes with different
> parameters.
> - LIST_PAGINATION_MAX_PAGE_SIZE is unlimited by default. If the new
> operations paginate only when an operator sets a maximum, a client that
> ignores the token works in every default deployment. Then it silently loses
> results the day someone sets a limit :) That's the September problem again,
> just moved :) I would suggest that these operations use a reasonable
> default page size even when no maximum is set, so clients use the token
> from the start. This belongs to the impl PRs after #5943, not in #6028.
> - #5533 showed that the generic tables CLI followed next-page-token only
> when --page-size was set. When the storage configuration operations land,
> the Python CLI should always follow the token.
>
> Regarding getApplicablePolicies, I would keep the Iceberg rules and fix
> #5311 under the current contract, the same as listPolicies. Putting one
> policy operation under different rules would be confusing, and it would
> change the contract for existing clients. If we ever want to move all the
> Polaris catalog listings (policies, generic tables, semantic models, ...)
> to these definitions, that should be its own discussion covering all of
> them.
>
> The Shares and Directory APIs already behave this way, so switching it to
> reference the new file could be a follow-up with no behavior change.
>
> Regards
> JB
>
> On Sat, Nov 14, 2026 at 2:18 AM Srinivas Rishindra <[email protected]
> >
> wrote:
>
> > Hi all,
> >
> > PR #6028 [1] adds spec/polaris-pagination.yaml, a small set of pagination
> > definitions for Polaris-specific list operations: a pageToken and a
> > pageSize
> >  query parameter, and a page token schema. Please give explicit sign-off
> > here before any operation starts using it, because the rules differ from
> > the Iceberg REST Catalog API. I'd like to settle that.
> >
> > The file states four rules:
> >
> >    1. Any response of an operation that uses these definitions may be
> >    paginated. A client must always check for a page token in the response
> > and,
> >    when one is present, send it unchanged as the pageToken query
> parameter
> >    to get the next page. The server omits the page token from the last
> > page. A
> >    client must treat a missing or null page token as the end of the
> > results.
> >    2. A client does not need to send pageToken or pageSize to start
> >    paginating. The server may paginate a response to a request that has
> >    neither.
> >    3. The server decides the page size. A page may hold fewer results
> than
> >    pageSize.
> >    4. A server that does not paginate returns all results in one
> response,
> >    with no page token.
> >
> > Scope: the rules apply only to operations that reference this file.
> Nothing
> > on main references it today. Operations that use the Iceberg REST Catalog
> > pagination definitions keep the Iceberg rules, including the Polaris
> > catalog listings that reuse them (policies, generic tables, semantic
> > models)
> > .
> >
> > How this relates to the September thread [2] and #5553 [3]: that thread
> > found that a configured LIST_PAGINATION_MAX_PAGE_SIZE truncated requests
> > with no pageToken, while the Iceberg spec promises such a request the
> > complete result. We agreed to reject those requests rather than truncate
> > them, and #5553 did that. The concern there was clients that rely on
> > the Iceberg
> > promise and never look for a token. These rules don't touch those
> > operations. For a new operation that uses this file, no such clients
> > exist: the
> > contract tells every client from day one to follow the token, so a first
> > page is never mistaken for the whole result. It also gives the server the
> > bound on response size that motivated the max page size in the first
> place.
> >
> > There is precedent: the Shares API spec on main (
> > spec/polaris-shares-api.yaml) already says that when pageSize is absent
> the
> > service's maximum is used, and that nextPageToken is present when more
> > results exist. This file writes that behavior down once for reuse.
> >
> > First intended users: the two list operations in the storage
> configuration
> > spec, PR #5943 [4] (listStorageConfigurations and listStorageMappings, in
> > the management API).
> >
> > One related question: getApplicablePolicies declares the Iceberg
> pagination
> > parameters but doesn't paginate yet (#5311 [5]). Should it keep the
> Iceberg
> > rules when that is fixed, like listPolicies next to it, or move to these
> > definitions? Moving it would change the contract of an existing
> operation,
> > so
> > I'd treat it as a separate decision either way.
> >
> > Please reply with +1 or any objections. If there are none in about 72
> > hours,
> >  I'll take that as agreement and ask for #6028 to be merged.
> >
> > [1] https://github.com/apache/polaris/pull/6028 [2]
> > https://lists.apache.org/thread/k81ptyktdbdf8gynncgk3o04mqt85zyk [3]
> > https://github.com/apache/polaris/pull/5553 [4]
> > https://github.com/apache/polaris/pull/5943 [5]
> > https://github.com/apache/polaris/issues/5311
> >
> > Thanks,
> > Rishi
> >
>

Reply via email to