In principle I agree with reconciling into the upstream Iceberg functionality since from what I remember the lack of the "overwrite" functionality was indeed one of the main reasons for going down the path of adding the sendNotification API instead. A much earlier incarnation of the similar proposal in Iceberg cited in the Polaris decision was https://github.com/apache/iceberg/issues/7261
However, some of the ways the data model has evolved may have indeed led to intentional divergence that would support keeping and even expanding this "notifications" surface. The main things we should consider, off the top of my head: 1. Basic parity or migrateability of the low-level behavior with respect to handling of ordering, consistency, etc 2. The sendNotifications functionality bakes in what amounts to a "mkdir -p" to auto-create parent namespaces so that we don't have separate API calls just for maintenance of namespaces - this could be argued to be better or worse than requiring explicit createNamespace by the sender 3. Most importantly, how we maintain the *segregation* of supported Iceberg REST operations based on the catalog type (and whether there's any reason to blur the current segregation) - right now EXTERNAL catalogs can *only* serve sendNotifications, so there's no mixture of internally owned tables vs externally owned ones, and vice versa for INTERNAL catalogs. I suppose registerTable(overwrite=true) basically forces INTERNAL catalogs to be *capable* of letting individual tables behave as if they're externally owned, but the semantic of internal vs external isn't reflected within the table itself. For (3), we're basically splitting registerTable(overwrite=true) into two distinct use cases: 1. Recovery/repairs on the source-of-truth catalog 2. Hook for mirroring from a remote source-of-truth catalog In the repair case, you can accept a mix of register(overwrite) and updateTable on the same table, because the overwrite was just a forced table-state that cooperates with subsequent updates. In the mirroring case, you very importantly do *not* want to allow updates since that creates a split-brain problem and unintentional table forking. I was also worried about whethe the authz operations had segregated SEND_NOTIFICATIONS, but it looks like we never did segregate them and SEND_NOTIFICATIONS just requires [TABLE_CREATE, TABLE_WRITE_PROPERTIES, TABLE_DROP, NAMESPACE_CREATE, NAMESPACE_DROP] so at least that's a data point in favor of reconciling into core Iceberg APIs. https://github.com/apache/polaris/blob/ec65ed725870564d3e460e58a942ca1fc8beccb2/polaris-core/src/main/java/org/apache/polaris/core/auth/RbacOperationSemantics.java#L330 Also, the innate "CREATE OR UPDATE" semantics of registerTable are another data point in favor of reconciling. So overall if we think through the low-level parity, decide on the "mkdir -p" semantics, and have the finer-grained segregation of allowed operations per catalog type, I think it's feasible to deprecate sendNotifications. The namespace-management in (2) is a tedious migration cost for senders though, and the lower-level distinction of allowed operations in (3) adds complexity to the Polaris codebase. On Wed, Aug 5, 2026 at 4:22 AM Robert Stupp <[email protected]> wrote: > Hi everyone, > > PR #5222 [1] prompted me to look more broadly at the status of the Polaris > notification API. > > This is separate from the technical scope of that PR. > Fixing the concurrency behavior of an existing endpoint seems reasonable, > and I do not think the PR needs to wait for this discussion. > > Despite its name, the endpoint is effectively an inbound > catalog-synchronization API. > A remote catalog or sync agent can use it to create, update, validate, or > drop externally managed table state in Polaris. > > The endpoint arrived with the initial Polaris code import and has since > acquired ordering and validation semantics. > > The January 2025 discussion [2] classified it as a Polaris-specific API, > resulting in PR #906 [3] moving it into a separate specification file. > The same discussion said that it had originally been intended for > contribution to Iceberg and was not expected to remain in the Polaris > specification long-term. > > Since then, Iceberg has added an `overwrite` option to table registration > [4]. > The upstream discussion explicitly considered complete metadata replacement > and push-based mirroring [5]. > Polaris now implements that operation [6], although currently only for > internal catalogs. > This appears to overlap substantially with the notification endpoint's > catalog-synchronization purpose. > > Snowflake is a documented consumer of the notification endpoint [7]. > I could not find public evidence of another deployed producer, although > code search is of course not usage telemetry. > > The endpoint is not marked experimental or beta. > Under the current Polaris evolution policy [8], it therefore appears to be > a stable, versioned REST API with ongoing compatibility expectations. > > Before we continue evolving it incrementally, should this remain a stable > Polaris-specific API, or should it be deprecated in favor of standard > Iceberg REST operations? > > If existing users rely on behavior that the standard endpoints cannot > provide, it would be useful to document those requirements and the case for > retaining the API. > > Does that match others' understanding? > > Robert > > > [1] https://github.com/apache/polaris/pull/5222 > [2] https://lists.apache.org/thread/1fqocs00pno0xfr4ss2p69d6dv5h8qzf > [3] https://github.com/apache/polaris/pull/906 > [4] https://github.com/apache/iceberg/pull/12239 > [5] https://lists.apache.org/thread/kkj1f98v035p2k25jpcb5mboc9bpsk58 > [6] https://github.com/apache/polaris/pull/4506 > [7] > > https://docs.snowflake.com/en/sql-reference/functions/system_send_notifications_to_catalog > [8] > > https://github.com/apache/polaris/blob/main/site/content/in-dev/unreleased/evolution.md >
