Hi folks, Notes from today's metrics architecture sync are in the usual doc: https://docs.google.com/document/d/100h7c4damrUzVuquYbBHM0EvA4LSWuW2IT2dN_7nYVA/edit?pli=1&tab=t.18iv1m7p6i6b
Highlights: Module placement. core/ now holds the entity data model only. A new spi/ takes every other shared contract: feature SPIs, feature non-SPI contracts, substrate SPIs, substrate non-SPI contracts, and contract-required types. That settles the question PR #5204 reopened, by placing into a new SPI module. Note that it changes slightly towards the referenced WIP RFC [1] recommendations: what proposed to place in core/, now should go to spi/ except for the entity data model. Metrics PRs. #5068 merged. #5204 needs reshaping onto the split. #4115 and #4756 stay open with action on the author to follow up on review comments. [1] https://docs.google.com/document/d/1mj3vGpix9zTDTFFJKtWT5ZxI3PTGI1YNb5xFqyWrOkc/edit?tab=t.0 On Wed, Jul 15, 2026 at 11:01 AM Anand Kumar Sankaran via dev < [email protected]> wrote: > Thanks EJ. My plan is to introduce PR0 in a new branch / PR and repurpose > #4115 and #4756 for PR1 and PR2 to maintain all the valuable discussions > that happened there. > > From: EJ Wang <[email protected]> > Date: Wednesday, July 15, 2026 at 10:41 AM > To: [email protected] <[email protected]> > Subject: Re: Proposal for REST endpoints for table metrics and events > > Hi folks, Thanks again for the discussion today. I updated the sync doc > with notes from today's metrics sync: https: //urldefense. com/v3/__https: > //docs. google. > com/document/d/100h7c4damrUzVuquYbBHM0EvA4LSWuW2IT2dN_7nYVA/edit?pli=1&tab=t. > 2ahd7wze5f__;!!Iz9xO38YGHZK!4rnlVIqBQQQ6xpGe1PMV58edOS_cGrKoqdfhw4qSp_JQCavvEjaxXk0OwGPnxyEcO60AzKQK_XDtp7H3DdkFXcrXJwAz1g0$ > > > Hi folks, > > Thanks again for the discussion today. I updated the sync doc with notes > from today's metrics sync: > > https://urldefense.com/v3/__https://docs.google.com/document/d/100h7c4damrUzVuquYbBHM0EvA4LSWuW2IT2dN_7nYVA/edit?pli=1&tab=t.2ahd7wze5f__;!!Iz9xO38YGHZK!4rnlVIqBQQQ6xpGe1PMV58edOS_cGrKoqdfhw4qSp_JQCavvEjaxXk0OwGPnxyEcO60AzKQK_XDtp7H3DdkFXcrXJwAz1g0$ > > See highlights below: > > In previous meetings, EJ proposed a two prs framing to push forward the > metrics query work in polaris. Dmitri pointed out a subtle point that pr2 > is currently also trying to do the persistence related code placement > refactor. For the sake of straightforward review process, EJ, Dmitri and > Anand (the author) agreed to separate the persistence refactoring into pr0, > and reframing the pr scopes as follow: > > 1. > > Pr0: refactor irc metrics spi > 1. > > SPI should stay in core/ with metric type enveloped > 2. > > default spi impl, no-op, in extensions/ - default selection at > runtime/ > 3. > > move metrics persistence and correspondings to extensions/ as an > alternative SPI impl > 2. > > Pr1: rest api for query > > https://urldefense.com/v3/__https://github.com/apache/polaris/pull/4115/changes__;!!Iz9xO38YGHZK!4rnlVIqBQQQ6xpGe1PMV58edOS_cGrKoqdfhw4qSp_JQCavvEjaxXk0OwGPnxyEcO60AzKQK_XDtp7H3DdkFXcrXFCbEbRk$ > 1. > > rest api spec in extensions/ as optional spec > 2. > > SPI in extensions/ as optional spec spi > 3. > > default spi impl in extensions/ no-op > 4. > > handler related in runtime/ - thin adaptor from http shape to spi > shape > 3. > > Pr2: jdbc impl for query > https://urldefense.com/v3/__https://github.com/apache/polaris/pull/4756__;!!Iz9xO38YGHZK!4rnlVIqBQQQ6xpGe1PMV58edOS_cGrKoqdfhw4qSp_JQCavvEjaxXk0OwGPnxyEcO60AzKQK_XDtp7H3DdkFXcrXRVMBzqE$ > 1. > > alternative spi impl in extensions/ -> query with jdbc > > > Thanks, > -ej > > On Wed, Jul 1, 2026 at 8:01 AM Alexandre Dutra <[email protected]> wrote: > > > Hi all, > > > > > While the current implementation stores metrics in the metastore, other > > deployments > > may choose a KV store, a dedicated metrics warehouse, or another backend > > entirely. In those cases, exposing a Polaris REST API over a specific > > schema feels like coupling the API to an implementation detail. > > > > I wouldn't say to an implementation detail, but rather to a particular > > setup / configuration. I think that's fine. If you don't store > > metrics, you don't read metrics. Similar situations will certainly > > happen in the future, e.g. if we ever introduce an API to read events, > > that would only work if events are being persisted in the first place. > > Again I think that's a fair trade-off. > > > > Thanks, > > Alex > > > > On Wed, Jul 1, 2026 at 7:23 AM Dmitri Bourlatchkov <[email protected]> > > wrote: > > > > > > Hi Yufei, > > > > > > From my POV, it is fine for Polaris to have multiple REST API surfaces > > with > > > different features and different degrees of "readiness". Each API can > > > evolve fairly independently. With proper modularization (which Anand > > did), > > > changes to the main Polaris code and these extra modules should > normally > > be > > > non-intrusive to each other. > > > > > > I assume Anand is going to be the first user of this API. However, if > we > > do > > > not merge it we'll never know whether other users are interested. I do > > not > > > think many end users read the dev ML, but they certainly read the docs, > > > code and release notes. > > > > > > There is no risk for Polaris as a project in exposing this experimental > > > API. If usage dwindles over time, we can remove it. If it gains a > > > considerable user / contributor base, we will promote it to a > > "first-class" > > > (non-beta) API. > > > > > > Cheers, > > > Dmitri. > > > > > > On Tue, Jun 30, 2026 at 7:53 PM Yufei Gu <[email protected]> wrote: > > > > > > > Hi all, > > > > > > > > Thanks for driving this work forward. Building this as a POC makes > > perfect > > > > sense if the goal is to validate the idea and gather feedback. Should > > we > > > > merge a POC into the main branch before agreeing on the overall > > direction? > > > > To me, that feels like a separate discussion from whether the > > > > implementation itself looks reasonable. > > > > > > > > One concern that I don't think has been fully addressed is whether we > > need > > > > to expose metrics consumption through a Polaris REST endpoint at all. > > > > > > > > My concern is less about the API itself and more about the boundary > > Polaris > > > > should own. The proposed REST API is tied to a particular persistence > > > > schema, but that schema is just one possible implementation. While > the > > > > current implementation stores metrics in the metastore, other > > deployments > > > > may choose a KV store, a dedicated metrics warehouse, or another > > backend > > > > entirely. In those cases, exposing a Polaris REST API over a specific > > > > schema feels like coupling the API to an implementation detail. > > > > > > > > Polaris isn't intended to become a full metrics system. I'd expect it > > to > > > > focus on producing metrics and let deployments choose the most > > appropriate > > > > storage and consumption model for their environment. > > > > > > > > I'm not opposed to exploring this direction, but we should first > align > > on > > > > whether exposing a REST API through Polaris is the right long term > > approach > > > > before merging it into the main branch. > > > > > > > > Thanks, > > > > Yufei > > > > > > > > Yufei > > > > > > > > > > > > On Mon, Jun 29, 2026 at 9:27 AM Dmitri Bourlatchkov < > [email protected]> > > > > wrote: > > > > > > > > > Hi All, > > > > > > > > > > It looks like Anand addressed all action items in [4115]. > > > > > > > > > > Please review the latest state of the code. The PR LGTM and I > > approved in > > > > > GH from my side. > > > > > > > > > > [4115] > https://urldefense.com/v3/__https://github.com/apache/polaris/pull/4115__;!!Iz9xO38YGHZK!4rnlVIqBQQQ6xpGe1PMV58edOS_cGrKoqdfhw4qSp_JQCavvEjaxXk0OwGPnxyEcO60AzKQK_XDtp7H3DdkFXcrXtFRscKs$ > > > > > > > > > > Thanks, > > > > > Dmitri. > > > > > > > > > > On Thu, Jun 25, 2026 at 1:56 PM Dmitri Bourlatchkov < > > [email protected]> > > > > > wrote: > > > > > > > > > > > Hi All, > > > > > > > > > > > > Recap of the discussion in today's Community Sync: > > > > > > > > > > > > * The modular approach for Metrics SPI(s) and implementation > seems > > to > > > > > have > > > > > > community consensus (please comment if you still have concerns) > > > > > > > > > > > > * What modules / feature flags are enabled in Apache Polaris > > images by > > > > > > default will be discussed later (when the feature is close to > > > > > completion). > > > > > > > > > > > > In any case, users will be able to turn the full feature on/off > at > > > > their > > > > > > discretion, regardless of the default flag values. > > > > > > > > > > > > * It is generally preferable to have smaller PRs and introduce > new > > > > > > features gradually to simplify reviews. > > > > > > > > > > > > What this means for the Metrics API, IMHO is: > > > > > > > > > > > > a) The scope of PR 4115 is probably ok as it stands right now. If > > > > someone > > > > > > wants to propose splitting it further, please comment. > > > > > > > > > > > > b) After 4115 is merged, Metrics Persistence PRs will follow, > with > > > > > smaller > > > > > > scope per PR as appropriate for the nature of the code changes. > > > > > > > > > > > > * The major concern was about exposing the new Metrics REST API > to > > end > > > > > > users. The nature of this concern was about avoiding the (false) > > > > > impression > > > > > > that the new API is a firm and adopted Polaris specification. > > > > > > > > > > > > I think this concern is reasonable. > > > > > > > > > > > > I propose that Anand update the documentation sections of PR 4115 > > to > > > > > > emphasise that the API is currently an experimental / POC API at > > this > > > > > time > > > > > > and is subject to change (including breaking changes). I believe > it > > > > > already > > > > > > has the "beta" label. So this request is mostly for doc changes > to > > > > > clarify > > > > > > user expectations, since the "beta" label alone may not be > > interpreted > > > > > the > > > > > > same way by all users. > > > > > > > > > > > > I hope this update will resolve all remaining concerns with 4115 > > and > > > > > allow > > > > > > for it to be merged so that the community will be able to proceed > > to > > > > > > follow-up PRs (as noted above). > > > > > > > > > > > > Please add/correct me if I missed anything relevant. > > > > > > > > > > > > Thanks, > > > > > > Dmitri. > > > > > > > > > > > > On Wed, Jun 24, 2026 at 8:11 PM Yufei Gu <[email protected]> > > wrote: > > > > > > > > > > > >> I do not think we have agreement on the query REST APIs yet. > > > > > >> > > > > > >> In particular, the current query API shape seems highly coupled > > to the > > > > > >> *example[1]* metrics in the Iceberg REST spec. I do not think > > that is > > > > a > > > > > >> good way to design a REST API contract. For example, fields like > > > > > >> resultDataFiles, resultDeleteFiles, totalFileSizeBytes, > > > > > >> totalDataManifests, > > > > > >> totalDeleteManifests, and scannedDataManifests are very specific > > to > > > > one > > > > > >> style of scan metric. That makes the API look more like a direct > > > > > >> projection > > > > > >> of the current example payload than a general query model. > > > > > >> > > > > > >> My preference is that Polaris should not define or strongly > opine > > on > > > > the > > > > > >> Iceberg scan or commit metrics query REST API at this point. > > Polaris > > > > can > > > > > >> resolve and authorize the table, accept the metrics report, and > > > > delegate > > > > > >> it > > > > > >> to the configured reporting path. Durable storage, filtering, > > > > > dashboards, > > > > > >> and query APIs should remain downstream impl. choices unless and > > until > > > > > the > > > > > >> Iceberg community standardizes a query contract, which I doubt > > will > > > > > >> happen. > > > > > >> > > > > > >> So I do not think the intake path has to be blocked, but I would > > > > prefer > > > > > >> not > > > > > >> to merge a Polaris specific query API that may be hard to evolve > > > > later. > > > > > >> > > > > > >> 1. > > > > > >> > > > > > >> > > > > > > > > > > > > https://urldefense.com/v3/__https://github.com/apache/iceberg/blob/2a6c556c0c883c04c6d3fbd68e6aeda36b91e0aa/open-api/rest-catalog-open-api.yaml*L4152__;Iw!!Iz9xO38YGHZK!4rnlVIqBQQQ6xpGe1PMV58edOS_cGrKoqdfhw4qSp_JQCavvEjaxXk0OwGPnxyEcO60AzKQK_XDtp7H3DdkFXcrXCDsqYGo$ > > > > > >> > > > > > >> Yufei > > > > > >> > > > > > >> > > > > > >> On Tue, Jun 23, 2026 at 9:30 AM Dmitri Bourlatchkov < > > [email protected] > > > > > > > > > > >> wrote: > > > > > >> > > > > > >> > Hi All, > > > > > >> > > > > > > >> > So, what are the blockers for merging [4115], if any? > > > > > >> > > > > > > >> > [4115] > https://urldefense.com/v3/__https://github.com/apache/polaris/pull/4115__;!!Iz9xO38YGHZK!4rnlVIqBQQQ6xpGe1PMV58edOS_cGrKoqdfhw4qSp_JQCavvEjaxXk0OwGPnxyEcO60AzKQK_XDtp7H3DdkFXcrXtFRscKs$ > > > > > >> > > > > > > >> > Thanks, > > > > > >> > Dmitri. > > > > > >> > > > > > > >> > On Wed, Jun 17, 2026 at 5:28 PM EJ Wang < > > > > > [email protected] > > > > > >> > > > > > > >> > wrote: > > > > > >> > > > > > > >> > > Hi folks, > > > > > >> > > > > > > > >> > > Thanks again for the discussion today. I updated the sync > doc > > with > > > > > >> notes > > > > > >> > > from today's metrics sync: > > > > > >> > > > > > > > >> > > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > > > > > > > > https://urldefense.com/v3/__https://docs.google.com/document/d/100h7c4damrUzVuquYbBHM0EvA4LSWuW2IT2dN_7nYVA/edit?tab=t.ezk0rgdx0c6m__;!!Iz9xO38YGHZK!4rnlVIqBQQQ6xpGe1PMV58edOS_cGrKoqdfhw4qSp_JQCavvEjaxXk0OwGPnxyEcO60AzKQK_XDtp7H3DdkFXcrX83IZkm0$ > > > > > >> > > > > > > > >> > > A few highlights: > > > > > >> > > > > > > > >> > > - No objection to keeping the default metrics behavior > > small: > > > > > >> > > no-op/log-only is enough as the built-in default. > > > > > >> > > - Durable storage, event forwarding, external queues, > > > > dashboards, > > > > > >> and > > > > > >> > > custom filtering should be implementation choices behind > > the > > > > > >> metrics > > > > > >> > > reporting path. > > > > > >> > > - For PR #4115, the REST layer should > resolve/authz/accept > > the > > > > > >> report, > > > > > >> > > then delegate to the selected metrics implementation. It > > should > > > > > not > > > > > >> > own > > > > > >> > > durable storage semantics. > > > > > >> > > - The rough boundary we discussed is: spec owns the wire > > > > > contract, > > > > > >> > > runtime owns framework wiring, API/contract modules own > > > > > >> > provider-facing > > > > > >> > > contracts, and extensions own replaceable > implementations. > > > > > >> > > - OpenLineage has a similar path-naming question. We > should > > > > avoid > > > > > >> > > occupying a generic lineage path too early if we want > room > > for > > > > > >> other > > > > > >> > > lineage systems later. > > > > > >> > > > > > > > >> > > I believe the boundary rules previewed in sync meeting are > > useful > > > > > for > > > > > >> > > reviewing PR #4115, but it is not > > > > > >> > > a final module-layout proposal. > > > > > >> > > > > > > > >> > > For metrics specifically, I think the target remains: > > > > > >> > > runtime > > > > > >> > > -> metrics reporting/emitting contract > > > > > >> > > -> default implementation > > > > > >> > > -> lower implementation details, if needed > > > > > >> > > > > > > > >> > > That keeps the default simple, while allowing durable JDBC, > > > > > >> event-backed, > > > > > >> > > or external-queue implementations to be added separately. > > > > > >> > > > > > > > >> > > We also touched on the split between Iceberg metrics > emitting > > and > > > > > >> > > Polaris-owned metrics querying. The querying side can keep > > > > evolving, > > > > > >> > > especially for dashboard or generic-table use cases, but I > do > > not > > > > > >> think > > > > > >> > > that needs to block the intake path in this PR. > > > > > >> > > > > > > > >> > > Thanks, > > > > > >> > > -ej > > > > > >> > > > > > > > >> > > On Tue, Jun 16, 2026 at 2:17 PM Yufei Gu < > > [email protected]> > > > > > >> wrote: > > > > > >> > > > > > > > >> > > > Thanks for chiming in, EJ. Agreed that the default battery > > > > should > > > > > >> stay > > > > > >> > > > small. I'm leaning toward not including metrics > persistence > > in > > > > the > > > > > >> > > default > > > > > >> > > > battery. > > > > > >> > > > > > > > > >> > > > Yufei > > > > > >> > > > > > > > > >> > > > > > > > > >> > > > On Thu, Jun 11, 2026 at 8:30 PM EJ Wang < > > > > > >> > [email protected]> > > > > > >> > > > wrote: > > > > > >> > > > > > > > > >> > > > > Hi Yufei, > > > > > >> > > > > > > > > > >> > > > > Thanks for connecting this back to the REST endpoint > > > > proposal. I > > > > > >> > > replied > > > > > >> > > > > with the fuller version on the event-forwarding thread, > > but > > > > > >> wanted to > > > > > >> > > add > > > > > >> > > > > the shorter version here too. > > > > > >> > > > > > > > > > >> > > > > I agree that metrics reporting/emitting is the right > > > > conceptual > > > > > >> > > boundary, > > > > > >> > > > > and I also think the event/listener path is a good > > > > > implementation > > > > > >> > > > direction > > > > > >> > > > > to explore. The distinction I want to keep clear is > > default > > > > > >> battery > > > > > >> > vs > > > > > >> > > > > extension implementation. > > > > > >> > > > > > > > > > >> > > > > For the REST/API side, I would keep the semantics > narrow: > > > > > Polaris > > > > > >> > > > resolves > > > > > >> > > > > and authorizes the table, accepts the Iceberg > scan/commit > > > > > metrics > > > > > >> > > report > > > > > >> > > > > into the configured reporting/emitting path, and returns > > 204. > > > > > That > > > > > >> > 204 > > > > > >> > > > > should mean Polaris accepted the report into the > ingestion > > > > path; > > > > > >> it > > > > > >> > > > should > > > > > >> > > > > not imply durable storage. > > > > > >> > > > > > > > > > >> > > > > Durable storage, event forwarding, external telemetry > > routing, > > > > > >> > > filtering, > > > > > >> > > > > and retention should sit behind that reporting/emitting > > > > boundary > > > > > >> as > > > > > >> > > > > implementation-layer behavior. A durable JDBC path can > be > > one > > > > > >> named > > > > > >> > > > > extension implementation. An event/listener forwarding > > path > > > > can > > > > > be > > > > > >> > > > another > > > > > >> > > > > named extension implementation. > > > > > >> > > > > > > > > > >> > > > > So I am +1 on the event-forwarding idea as a non-default > > > > > extension > > > > > >> > > > > implementation. I just would not make it the default > > battery > > > > or > > > > > >> the > > > > > >> > > core > > > > > >> > > > > API shape. The default battery can stay small and safe, > > while > > > > > >> > > deployments > > > > > >> > > > > that need forwarding, dashboards, or durable storage can > > opt > > > > > into > > > > > >> the > > > > > >> > > > > implementation that matches their operational model. > > > > > >> > > > > > > > > > >> > > > > Thanks, > > > > > >> > > > > -ej > > > > > >> > > > > > > > > > >> > > > > On Wed, Jun 10, 2026 at 11:43 AM Yufei Gu < > > > > [email protected] > > > > > > > > > > > >> > > wrote: > > > > > >> > > > > > > > > > >> > > > > > Thanks EJ. I agree that the reporting/emitting > boundary > > > > feels > > > > > >> like > > > > > >> > > the > > > > > >> > > > > > right SPI boundary, but I wonder if we can simplify > this > > > > even > > > > > >> > > further. > > > > > >> > > > > > > > > > > >> > > > > > Iceberg scan and commit metrics look very similar to > > events > > > > to > > > > > >> me. > > > > > >> > > They > > > > > >> > > > > are > > > > > >> > > > > > append only, asynchronously consumed, often forwarded > to > > > > > >> external > > > > > >> > > > > systems, > > > > > >> > > > > > and have similar retention and cleanup requirements. > > More > > > > > >> details > > > > > >> > can > > > > > >> > > > be > > > > > >> > > > > > found in this thread [1]. In that case, Polaris may > only > > > > need > > > > > to > > > > > >> > > accept > > > > > >> > > > > the > > > > > >> > > > > > report and emit an event through the existing event > > > > framework. > > > > > >> > > > Filtering, > > > > > >> > > > > > async delivery, custom sinks, and retention mechanisms > > could > > > > > >> then > > > > > >> > be > > > > > >> > > > > shared > > > > > >> > > > > > instead of introducing a separate metrics specific > > extension > > > > > >> layer > > > > > >> > > and > > > > > >> > > > > SPI. > > > > > >> > > > > > > > > > > >> > > > > > The main requirement I still see is querying metrics > > through > > > > > >> > Polaris. > > > > > >> > > > If > > > > > >> > > > > we > > > > > >> > > > > > want that battery included experience, we could work > on > > a > > > > > >> > persistence > > > > > >> > > > > > listener implementation that selectively saves metrics > > and > > > > > >> expose > > > > > >> > it > > > > > >> > > > via > > > > > >> > > > > > the IRC event endpoint which seems pretty close in > > Iceberg > > > > > >> > > > > > community. Another option I like more is to allow > users > > to > > > > > >> develop > > > > > >> > > > their > > > > > >> > > > > > own listener, so they can persist scan/commit metrics > > to any > > > > > >> > systems > > > > > >> > > > they > > > > > >> > > > > > prefer. In general, that feels like an implementation > > choice > > > > > on > > > > > >> top > > > > > >> > > of > > > > > >> > > > > the > > > > > >> > > > > > event framework rather than something that needs to > > shape > > > > the > > > > > >> core > > > > > >> > > > > > architecture. > > > > > >> > > > > > > > > > > >> > > > > > 1. > > > > > >> > > > > https://urldefense.com/v3/__https://lists.apache.org/thread/x9j8nscvy8hq61tyn01mj8yp6n9of0kp__;!!Iz9xO38YGHZK!4rnlVIqBQQQ6xpGe1PMV58edOS_cGrKoqdfhw4qSp_JQCavvEjaxXk0OwGPnxyEcO60AzKQK_XDtp7H3DdkFXcrXGcGDgks$ > > > > > >> > > > > > > > > > > >> > > > > > Thanks, > > > > > >> > > > > > Yufei > > > > > >> > > > > > > > > > > >> > > > > > > > > > > >> > > > > > On Thu, Jun 4, 2026 at 5:14 PM EJ Wang < > > > > > >> > > [email protected] > > > > > >> > > > > > > > > > >> > > > > > wrote: > > > > > >> > > > > > > > > > > >> > > > > > > Good point. I agree the existing > > PolarisMetricsReporter is > > > > > >> > already > > > > > >> > > > very > > > > > >> > > > > > > close to the right conceptual boundary. I am not > > > > proposing a > > > > > >> > second > > > > > >> > > > > > > parallel reporter concept. The distinction I am > > trying to > > > > > >> make is > > > > > >> > > > > > narrower: > > > > > >> > > > > > > > > > > > >> > > > > > > Current state: > > > > > >> > > > > > > > > > > > >> > > > > > > - PolarisMetricsReporter is the existing metrics > > > > > reporting > > > > > >> > hook. > > > > > >> > > > > > > - But it currently lives under runtime/service. > > > > > >> > > > > > > - Its contract is documented around Quarkus/CDI > > > > discovery > > > > > >> and > > > > > >> > > > > > > @Identifier > > > > > >> > > > > > > selection. > > > > > >> > > > > > > - The default implementation is log-only. > > > > > >> > > > > > > - The durable implementation writes through > > > > > >> > > PolarisMetricsManager > > > > > >> > > > > and > > > > > >> > > > > > > MetricsPersistence. > > > > > >> > > > > > > - MetricsPersistence is currently inherited by > > > > > >> > BasePersistence. > > > > > >> > > > > > > > > > > > >> > > > > > > My proposal: > > > > > >> > > > > > > > > > > > >> > > > > > > - Keep the reporting/emitting boundary as the > SPI. > > > > > >> > > > > > > - Revise that contract in a framework-agnostic > > optional > > > > > >> > Iceberg > > > > > >> > > > > > *metrics > > > > > >> > > > > > > extension API layer*. > > > > > >> > > > > > > - Keep runtime/service responsible for REST > > ingestion, > > > > > >> > > table/authz > > > > > >> > > > > > > validation, and runtime wiring. > > > > > >> > > > > > > - Keep the battery implementation log-only in the > > > > metrics > > > > > >> > > > extension > > > > > >> > > > > > API > > > > > >> > > > > > > layer, *not under runtime/service*. > > > > > >> > > > > > > - Treat durable JDBC metrics as one > implementation > > of > > > > the > > > > > >> > > > reporting > > > > > >> > > > > > SPI. > > > > > >> > > > > > > - Decompose the MetricsPersistence and > > BasePersistence > > > > > >> > coupling > > > > > >> > > as > > > > > >> > > > > > > durable implementation detail. > > > > > >> > > > > > > > > > > > >> > > > > > > So the difference is not “new reporter concept vs > old > > > > > reporter > > > > > >> > > > > concept.” > > > > > >> > > > > > > but: > > > > > >> > > > > > > > > > > > >> > > > > > > > > > > > >> > > > > > > - PolarisMetricsReporter today = runtime/service > > > > > CDI-shaped > > > > > >> > > hook. > > > > > >> > > > > > > - Target reporting SPI = same conceptual > boundary, > > but > > > > > >> placed > > > > > >> > in > > > > > >> > > > the > > > > > >> > > > > > > right extension API layer and kept > > framework-agnostic. > > > > > >> > > > > > > > > > > > >> > > > > > > For the current metrics PR, I think the minimal > > > > > architectural > > > > > >> > > cleanup > > > > > >> > > > > > > before merge is to *put the reporting SPI contract > > and the > > > > > >> > battery > > > > > >> > > > > > default > > > > > >> > > > > > > in the right place*. The durable JDBC > implementation, > > > > > >> > event-backed > > > > > >> > > > > async > > > > > >> > > > > > > implementation, external queue support, and deeper > > > > > persistence > > > > > >> > > > cleanup > > > > > >> > > > > > can > > > > > >> > > > > > > continue as follow-ups, as long as we do not lock > the > > SPI > > > > > >> > boundary > > > > > >> > > to > > > > > >> > > > > > > BasePersistence or to runtime/service wiring. > > > > > >> > > > > > > > > > > > >> > > > > > > -ej > > > > > >> > > > > > > > > > > > >> > > > > > > On Wed, Jun 3, 2026 at 3:00 PM Dmitri Bourlatchkov < > > > > > >> > > [email protected] > > > > > >> > > > > > > > > > >> > > > > > > wrote: > > > > > >> > > > > > > > > > > > >> > > > > > > > Hi EJ, > > > > > >> > > > > > > > > > > > > >> > > > > > > > Thanks for the recap / summary! > > > > > >> > > > > > > > > > > > > >> > > > > > > > > Do folks agree that the stable SPI boundary > > should be > > > > > >> > > > > > > > metrics reporting/emitting, not metrics > persistence? > > > > > >> > > > > > > > > > > > > >> > > > > > > > SGTM. > > > > > >> > > > > > > > > > > > > >> > > > > > > > > Does an optional Iceberg metrics extension API > > layer > > > > > sound > > > > > >> > like > > > > > >> > > > the > > > > > >> > > > > > > right > > > > > >> > > > > > > > home for this SPI? > > > > > >> > > > > > > > > > > > > >> > > > > > > > How is that different from current > > > > PolarisMetricsReporter? > > > > > >> > > > > > > > > > > > > >> > > > > > > > > Should the current durable metrics work be > > reframed > > > > as a > > > > > >> > > durable > > > > > >> > > > > > > > JDBC reference implementation of that SPI? > > > > > >> > > > > > > > > > > > > >> > > > > > > > SGTM. > > > > > >> > > > > > > > > > > > > >> > > > > > > > > What is the smallest PR sequence to get there > > without > > > > > >> > blocking > > > > > >> > > > > > > > the current metrics work unnecessarily? > > > > > >> > > > > > > > > > > > > >> > > > > > > > Let's get > > > https://urldefense.com/v3/__https://github.com/apache/polaris/pull/4397__;!!Iz9xO38YGHZK!4rnlVIqBQQQ6xpGe1PMV58edOS_cGrKoqdfhw4qSp_JQCavvEjaxXk0OwGPnxyEcO60AzKQK_XDtp7H3DdkFXcrXuu5jefU$ > > > > > >> merged > > > > > >> > > > first. > > > > > >> > > > > > > > > > > > > >> > > > > > > > Then, I'd propose isolating the Metrics schema > from > > the > > > > > >> > MetaStore > > > > > >> > > > > > schema. > > > > > >> > > > > > > > This will probably have some ripple effect into > the > > > > > >> bootstrap > > > > > >> > > > > > workflows, > > > > > >> > > > > > > so > > > > > >> > > > > > > > it's not a trivial change. > > > > > >> > > > > > > > > > > > > >> > > > > > > > Then, let's reassess. > > > > > >> > > > > > > > > > > > > >> > > > > > > > Cheers, > > > > > >> > > > > > > > Dmitri. > > > > > >> > > > > > > > > > > > > >> > > > > > > > On Wed, Jun 3, 2026 at 5:40 PM EJ Wang < > > > > > >> > > > > [email protected] > > > > > >> > > > > > > > > > > > >> > > > > > > > wrote: > > > > > >> > > > > > > > > > > > > >> > > > > > > > > Hi folks, > > > > > >> > > > > > > > > > > > > > >> > > > > > > > > Thanks again for the discussion today. I updated > > the > > > > > sync > > > > > >> doc > > > > > >> > > > with > > > > > >> > > > > > > notes > > > > > >> > > > > > > > > from the third metrics architecture sync: > > > > > >> > > > > > > > > > > > > > >> > > > > > > > > > > > > > >> > > > > > > > > > > > > > >> > > > > > > > > > > > > >> > > > > > > > > > > > >> > > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > > > > > > > > https://urldefense.com/v3/__https://docs.google.com/document/d/100h7c4damrUzVuquYbBHM0EvA4LSWuW2IT2dN_7nYVA/edit?tab=t.k96s2xyqr5u1__;!!Iz9xO38YGHZK!4rnlVIqBQQQ6xpGe1PMV58edOS_cGrKoqdfhw4qSp_JQCavvEjaxXk0OwGPnxyEcO60AzKQK_XDtp7H3DdkFXcrXrqCL-QE$ > > > > > >> > > > > > > > > > > > > > >> > > > > > > > > A few highlights from today’s discussion: > > > > > >> > > > > > > > > > > > > > >> > > > > > > > > - We clarified that Iceberg metrics reporting > > can > > > > be > > > > > >> > > > interpreted > > > > > >> > > > > > > > either > > > > > >> > > > > > > > > as sync or async handling from Polaris’s > > > > perspective, > > > > > >> so > > > > > >> > > > Polaris > > > > > >> > > > > > > > should > > > > > >> > > > > > > > > stay flexible as a platform instead of baking > > one > > > > > >> handling > > > > > >> > > > model > > > > > >> > > > > > > into > > > > > >> > > > > > > > > the > > > > > >> > > > > > > > > REST API behavior. > > > > > >> > > > > > > > > - We aligned that the built-in (battery) > > behavior > > > > for > > > > > >> > > metrics > > > > > >> > > > > > > emitting > > > > > >> > > > > > > > > can stay simple: no-op/log-only is enough as > > the > > > > > >> default. > > > > > >> > > > > > > > > - Durable metrics persistence should be > > treated as > > > > an > > > > > >> > > > > > implementation > > > > > >> > > > > > > > of > > > > > >> > > > > > > > > the metrics reporting path, not as the core > SPI > > > > > >> boundary. > > > > > >> > > > > > > > > - The existing durable metrics work can be > > reviewed > > > > > as > > > > > >> a > > > > > >> > > > > reference > > > > > >> > > > > > > > > implementation of the reporting SPI, with > > > > > >> > > persistence-related > > > > > >> > > > > > logic > > > > > >> > > > > > > > kept > > > > > >> > > > > > > > > self-contained under a metrics durable > > > > implementation > > > > > >> > module > > > > > >> > > > > > rather > > > > > >> > > > > > > > than > > > > > >> > > > > > > > > scattered through core entity persistence. > > > > > >> > > > > > > > > - Dashboard/insights remains a real use case, > > but > > > > we > > > > > >> > agreed > > > > > >> > > to > > > > > >> > > > > > keep > > > > > >> > > > > > > it > > > > > >> > > > > > > > > separate from the core metrics intake > > discussion > > > > for > > > > > >> now. > > > > > >> > > > > > > > > > > > > > >> > > > > > > > > I also did a quick source check after the > meeting > > to > > > > > make > > > > > >> > sure > > > > > >> > > we > > > > > >> > > > > are > > > > > >> > > > > > > > > describing the current state accurately. > > > > > >> > > > > > > > > > > > > > >> > > > > > > > > Current state: > > > > > >> > > > > > > > > > > > > > >> > > > > > > > > - Polaris already has a metrics reporting > hook: > > > > > >> > > > > > > > PolarisMetricsReporter. > > > > > >> > > > > > > > > - The default implementation is > > > > > DefaultMetricsReporter, > > > > > >> > > > selected > > > > > >> > > > > > by > > > > > >> > > > > > > > > > > polaris.iceberg-metrics.reporting.type=default. It > > > > is > > > > > >> > > > log-only, > > > > > >> > > > > > > > > effectively quiet unless metrics logging is > > > > enabled. > > > > > >> > > > > > > > > - There is also a PersistingMetricsReporter, > > > > selected > > > > > >> by > > > > > >> > > > > > > > > > > polaris.iceberg-metrics.reporting.type=persisting, > > > > > >> which > > > > > >> > > > > converts > > > > > >> > > > > > > > > Iceberg scan/commit reports into Polaris > > metrics > > > > > >> records > > > > > >> > and > > > > > >> > > > > > writes > > > > > >> > > > > > > > > through > > > > > >> > > > > > > > > PolarisMetricsManager -> MetricsPersistence. > > > > > >> > > > > > > > > - MetricsPersistence currently exists in the > > > > > >> persistence > > > > > >> > > layer > > > > > >> > > > > and > > > > > >> > > > > > > is > > > > > >> > > > > > > > > inherited by BasePersistence. > > > > > >> > > > > > > > > > > > > > >> > > > > > > > > My read from the discussion is that the target > > > > boundary > > > > > >> > should > > > > > >> > > > not > > > > > >> > > > > be > > > > > >> > > > > > > > > MetricsPersistence as inherited by > > BasePersistence. > > > > That > > > > > >> path > > > > > >> > > > > should > > > > > >> > > > > > be > > > > > >> > > > > > > > > decomposed as durable implementation detail > (taken > > > > care > > > > > >> of by > > > > > >> > > > > > > > > > https://urldefense.com/v3/__https://github.com/apache/polaris/pull/4397__;!!Iz9xO38YGHZK!4rnlVIqBQQQ6xpGe1PMV58edOS_cGrKoqdfhw4qSp_JQCavvEjaxXk0OwGPnxyEcO60AzKQK_XDtp7H3DdkFXcrXuu5jefU$). > The > > > > > stable > > > > > >> > > > extension > > > > > >> > > > > > > point > > > > > >> > > > > > > > > should instead be the metrics reporting/emitting > > > > > boundary. > > > > > >> > > > > > > > > > > > > > >> > > > > > > > > Concretely, I think the next proposal should be > > shaped > > > > > >> like > > > > > >> > > this: > > > > > >> > > > > > > > > > > > > > >> > > > > > > > > 1. > > > > > >> > > > > > > > > > > > > > >> > > > > > > > > Define the proper metrics reporting SPI at an > > > > > optional > > > > > >> > > Iceberg > > > > > >> > > > > > > metrics > > > > > >> > > > > > > > > extension API layer. > > > > > >> > > > > > > > > > > > > > >> > > > > > > > > Example direction: > > > > > >> > > > > > > > > > > > > > >> > > > > > > > > public interface IcebergMetricsReporter { > > > > > >> > > > > > > > > void > reportMetric(IcebergMetricsReportContext > > > > > >> context, > > > > > >> > > > > > > > > MetricsReport report); > > > > > >> > > > > > > > > } > > > > > >> > > > > > > > > > > > > > >> > > > > > > > > The context would carry the small > > Polaris-resolved > > > > > >> > envelope: > > > > > >> > > > > > > > > catalog/table identity, report type, received > > > > > >> timestamp, > > > > > >> > and > > > > > >> > > > > > > > > request/trace > > > > > >> > > > > > > > > context if available. The raw Iceberg > > MetricsReport > > > > > >> > remains > > > > > >> > > > the > > > > > >> > > > > > > > payload. > > > > > >> > > > > > > > > 2. > > > > > >> > > > > > > > > > > > > > >> > > > > > > > > Keep runtime/service as the REST ingestion > and > > > > wiring > > > > > >> > layer. > > > > > >> > > > > > > > > > > > > > >> > > > > > > > > The REST handler still resolves the table and > > > > > performs > > > > > >> > authz > > > > > >> > > > > > before > > > > > >> > > > > > > > > accepting the report. After that, it calls > the > > > > > selected > > > > > >> > > > > reporting > > > > > >> > > > > > > > > implementation. > > > > > >> > > > > > > > > 3. > > > > > >> > > > > > > > > > > > > > >> > > > > > > > > Keep the battery default no-op/log-only. > > > > > >> > > > > > > > > > > > > > >> > > > > > > > > This preserves an out-of-box safe default and > > > > avoids > > > > > >> > > > requiring a > > > > > >> > > > > > > > durable > > > > > >> > > > > > > > > metrics store for every Polaris deployment. > > > > > >> > > > > > > > > 4. > > > > > >> > > > > > > > > > > > > > >> > > > > > > > > Move durable JDBC metrics into a > self-contained > > > > > >> > > implementation > > > > > >> > > > > of > > > > > >> > > > > > > the > > > > > >> > > > > > > > > reporting SPI. > > > > > >> > > > > > > > > > > > > > >> > > > > > > > > That implementation can own its schema, > > bootstrap, > > > > > >> > > retention, > > > > > >> > > > > and > > > > > >> > > > > > > read > > > > > >> > > > > > > > > API support. It should not define the core > > > > reporting > > > > > >> SPI > > > > > >> > > > > boundary, > > > > > >> > > > > > > and > > > > > >> > > > > > > > > it > > > > > >> > > > > > > > > should not require metrics persistence to > > remain > > > > > >> inherited > > > > > >> > > > from > > > > > >> > > > > > > > > BasePersistence. > > > > > >> > > > > > > > > 5. > > > > > >> > > > > > > > > > > > > > >> > > > > > > > > Treat async/event-backed handling as another > > > > > >> > implementation > > > > > >> > > of > > > > > >> > > > > the > > > > > >> > > > > > > > same > > > > > >> > > > > > > > > reporting SPI. > > > > > >> > > > > > > > > > > > > > >> > > > > > > > > For example, an event-backed reporter could > > > > enqueue a > > > > > >> > > metrics > > > > > >> > > > > > event > > > > > >> > > > > > > > and > > > > > >> > > > > > > > > let listeners handle durable storage or other > > > > sinks. > > > > > >> If we > > > > > >> > > > later > > > > > >> > > > > > > need > > > > > >> > > > > > > > a > > > > > >> > > > > > > > > replaceable queue engine, that seems like a > > shared > > > > > >> > > > event/metrics > > > > > >> > > > > > > > > substrate > > > > > >> > > > > > > > > topic rather than a metrics-only requirement. > > > > > >> > > > > > > > > > > > > > >> > > > > > > > > This framing lets us keep the REST metrics > > endpoint > > > > > >> simple, > > > > > >> > > > > preserve > > > > > >> > > > > > > the > > > > > >> > > > > > > > > current default behavior, support durable > metrics > > > > users, > > > > > >> and > > > > > >> > > > still > > > > > >> > > > > > > leave > > > > > >> > > > > > > > > room for async/event-backed or > > external-queue-based > > > > > >> > > > > implementations. > > > > > >> > > > > > > > > > > > > > >> > > > > > > > > I think the main follow-up questions are: > > > > > >> > > > > > > > > > > > > > >> > > > > > > > > - Do folks agree that the stable SPI boundary > > > > should > > > > > be > > > > > >> > > > metrics > > > > > >> > > > > > > > > reporting/emitting, not metrics persistence? > > > > > >> > > > > > > > > - Does an optional Iceberg metrics extension > > API > > > > > layer > > > > > >> > sound > > > > > >> > > > > like > > > > > >> > > > > > > the > > > > > >> > > > > > > > > right home for this SPI? > > > > > >> > > > > > > > > - Should the current durable metrics work be > > > > reframed > > > > > >> as a > > > > > >> > > > > durable > > > > > >> > > > > > > > JDBC > > > > > >> > > > > > > > > reference implementation of that SPI? > > > > > >> > > > > > > > > - What is the smallest PR sequence to get > there > > > > > without > > > > > >> > > > blocking > > > > > >> > > > > > the > > > > > >> > > > > > > > > current metrics work unnecessarily? > > > > > >> > > > > > > > > > > > > > >> > > > > > > > > Thanks, > > > > > >> > > > > > > > > -ej > > > > > >> > > > > > > > > > > > > > >> > > > > > > > > On Fri, May 15, 2026 at 5:16 PM Dmitri > > Bourlatchkov < > > > > > >> > > > > > [email protected]> > > > > > >> > > > > > > > > wrote: > > > > > >> > > > > > > > > > > > > > >> > > > > > > > > > Hi JB, > > > > > >> > > > > > > > > > > > > > > >> > > > > > > > > > Could you set up another meeting, please? Same > > time > > > > on > > > > > >> > > > Wednesday > > > > > >> > > > > as > > > > > >> > > > > > > > last > > > > > >> > > > > > > > > > time... I hope it works for everyone. > > > > > >> > > > > > > > > > > > > > > >> > > > > > > > > > Cheers, > > > > > >> > > > > > > > > > Dmitri. > > > > > >> > > > > > > > > > > > > > > >> > > > > > > > > > On Fri, May 15, 2026 at 8:06 PM Yufei Gu < > > > > > >> > > [email protected] > > > > > >> > > > > > > > > > >> > > > > > > wrote: > > > > > >> > > > > > > > > > > > > > > >> > > > > > > > > > > +1 on another sync call next week. > > > > > >> > > > > > > > > > > > > > > > >> > > > > > > > > > > Yufei > > > > > >> > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > >> > > > > > > > > > > On Fri, May 15, 2026 at 4:52 PM Dmitri > > > > Bourlatchkov > > > > > < > > > > > >> > > > > > > > [email protected]> > > > > > >> > > > > > > > > > > wrote: > > > > > >> > > > > > > > > > > > > > > > >> > > > > > > > > > > > Hi All, > > > > > >> > > > > > > > > > > > > > > > > >> > > > > > > > > > > > WDYT about another sync call next week? > > > > > >> > > > > > > > > > > > > > > > > >> > > > > > > > > > > > Thanks, > > > > > >> > > > > > > > > > > > Dmitri. > > > > > >> > > > > > > > > > > > > > > > > >> > > > > > > > > > > > On Wed, May 6, 2026 at 5:29 PM Dmitri > > > > > Bourlatchkov < > > > > > >> > > > > > > > [email protected] > > > > > >> > > > > > > > > > > > > > > >> > > > > > > > > > > > wrote: > > > > > >> > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > Hi EJ, > > > > > >> > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > Thanks for the summary! It covers what > we > > > > > >> discussed > > > > > >> > in > > > > > >> > > > the > > > > > >> > > > > > > > meeting > > > > > >> > > > > > > > > > very > > > > > >> > > > > > > > > > > > > well, IMHO. > > > > > >> > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > Looking forward to concrete PRs :) > > > > > >> > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > Cheers, > > > > > >> > > > > > > > > > > > > Dmitri. > > > > > >> > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > On Wed, May 6, 2026 at 5:08 PM EJ Wang < > > > > > >> > > > > > > > > > [email protected] > > > > > >> > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > wrote: > > > > > >> > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > >> Hi folks, > > > > > >> > > > > > > > > > > > >> > > > > > >> > > > > > > > > > > > >> We had a community sync earlier, thanks > > JB > > > > for > > > > > >> > > > scheduling > > > > > >> > > > > > it. > > > > > >> > > > > > > > > Notes > > > > > >> > > > > > > > > > > from > > > > > >> > > > > > > > > > > > >> the first metrics architecture sync > (May > > 6, > > > > > >> 10-11am > > > > > >> > > PT). > > > > > >> > > > > > > > > Discussion > > > > > >> > > > > > > > > > > doc > > > > > >> > > > > > > > > > > > >> with per-section status: > > > > > >> > > > > > > > > > > > >> > > > > > >> > > > > > > > > > > > >> > > > > > >> > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > >> > > > > > > > > > > > > > >> > > > > > > > > > > > > >> > > > > > > > > > > > >> > > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > > > > > > > > https://urldefense.com/v3/__https://docs.google.com/document/d/100h7c4damrUzVuquYbBHM0EvA4LSWuW2IT2dN_7nYVA/edit?tab=t.0__;!!Iz9xO38YGHZK!4rnlVIqBQQQ6xpGe1PMV58edOS_cGrKoqdfhw4qSp_JQCavvEjaxXk0OwGPnxyEcO60AzKQK_XDtp7H3DdkFXcrXgIddeUs$ > > > > > >> > > > > > > > > > > > >> > > > > > >> > > > > > > > > > > > >> *The meeting covered both topics from > the > > > > doc. > > > > > >> > > > > > Direction-level > > > > > >> > > > > > > > > > > alignment > > > > > >> > > > > > > > > > > > >> was reached on the headline pieces; > > details > > > > > >> remain > > > > > >> > for > > > > > >> > > > PR > > > > > >> > > > > > > review > > > > > >> > > > > > > > > or > > > > > >> > > > > > > > > > > > >> follow-up sessions.* > > > > > >> > > > > > > > > > > > >> > > > > > >> > > > > > > > > > > > >> *Topic 1 — Persistence schema redesign* > > > > > >> > > > > > > > > > > > >> Idea-level alignment on consolidating > > > > per-type > > > > > >> > tables > > > > > >> > > > > > > > > > > > >> (scan_metrics_report, > > > > > >> > > > > > > > > > > > >> commit_metrics_report) into a single > > > > > >> metrics_report > > > > > >> > > > table. > > > > > >> > > > > > The > > > > > >> > > > > > > > > > > > motivating > > > > > >> > > > > > > > > > > > >> cost is the surface area added by every > > new > > > > > >> metric > > > > > >> > > type > > > > > >> > > > > > today: > > > > > >> > > > > > > > new > > > > > >> > > > > > > > > > > > table, > > > > > >> > > > > > > > > > > > >> SPI method, record class, model, > > converter, > > > > > >> schema > > > > > >> > > > > > migration. > > > > > >> > > > > > > > > > > > >> > > > > > >> > > > > > > > > > > > >> Most schema details are deferred to the > > > > schema > > > > > >> PR. A > > > > > >> > > few > > > > > >> > > > > > > > specific > > > > > >> > > > > > > > > > > points > > > > > >> > > > > > > > > > > > >> came up: > > > > > >> > > > > > > > > > > > >> • metric_schema_version: Yufei > prefers > > > > > dropping > > > > > >> > it, > > > > > >> > > > > since > > > > > >> > > > > > > > there > > > > > >> > > > > > > > > is > > > > > >> > > > > > > > > > > no > > > > > >> > > > > > > > > > > > >> spec-level concept of metrics > versioning > > > > today > > > > > >> and > > > > > >> > it > > > > > >> > > is > > > > > >> > > > > > hard > > > > > >> > > > > > > to > > > > > >> > > > > > > > > > > define > > > > > >> > > > > > > > > > > > >> unilaterally. Robert prefers keeping > it, > > > > given > > > > > >> IRC > > > > > >> > v2 > > > > > >> > > is > > > > > >> > > > > > > coming > > > > > >> > > > > > > > > and > > > > > >> > > > > > > > > > > the > > > > > >> > > > > > > > > > > > >> schema should be considered against its > > > > likely > > > > > >> > shape; > > > > > >> > > > > Robert > > > > > >> > > > > > > > also > > > > > >> > > > > > > > > > > raised > > > > > >> > > > > > > > > > > > >> how to differentiate various payload > > formats > > > > if > > > > > >> any. > > > > > >> > > > EJ's > > > > > >> > > > > > read > > > > > >> > > > > > > > is > > > > > >> > > > > > > > > > that > > > > > >> > > > > > > > > > > > >> this > > > > > >> > > > > > > > > > > > >> is a two-way-door decision. We can > start > > > > > without > > > > > >> the > > > > > >> > > > > field, > > > > > >> > > > > > > and > > > > > >> > > > > > > > if > > > > > >> > > > > > > > > > IRC > > > > > >> > > > > > > > > > > > v2 > > > > > >> > > > > > > > > > > > >> changes the shape we would likely roll > a > > > > > >> > corresponding > > > > > >> > > > new > > > > > >> > > > > > > > schema > > > > > >> > > > > > > > > > > > anyway, > > > > > >> > > > > > > > > > > > >> which is not particularly costly. > > > > > >> > > > > > > > > > > > >> • Payload format: Robert pointed out > > that > > > > > >> future > > > > > >> > > > formats > > > > > >> > > > > > > > beyond > > > > > >> > > > > > > > > > JSON > > > > > >> > > > > > > > > > > > may > > > > > >> > > > > > > > > > > > >> be worth supporting. The exact shape is > > > > > deferred > > > > > >> to > > > > > >> > > the > > > > > >> > > > > > schema > > > > > >> > > > > > > > > > > > discussion. > > > > > >> > > > > > > > > > > > >> • Partition strategy: Anand suggested > > > > monthly > > > > > >> > > > > partitioning > > > > > >> > > > > > > > based > > > > > >> > > > > > > > > > on > > > > > >> > > > > > > > > > > > his > > > > > >> > > > > > > > > > > > >> experience as potentially helpful at > > scale. > > > > > >> > > > > > > > > > > > >> > > > > > >> > > > > > > > > > > > >> *Topic 2 — Where metrics ingestion and > > > > storage > > > > > >> > belong* > > > > > >> > > > > > > > > > > > >> Idea-level alignment that metrics > should > > be a > > > > > >> > > separated > > > > > >> > > > > SPI > > > > > >> > > > > > > from > > > > > >> > > > > > > > > the > > > > > >> > > > > > > > > > > > >> entity > > > > > >> > > > > > > > > > > > >> persistence stack. Two reasons > surfaced: > > (a) > > > > > >> > workloads > > > > > >> > > > and > > > > > >> > > > > > > > > > capability > > > > > >> > > > > > > > > > > > >> requirements diverge enough that > coupling > > > > them > > > > > >> > creates > > > > > >> > > > > > > > artificial > > > > > >> > > > > > > > > > > > >> constraints, and (b) admin experience > > > > improves > > > > > >> when > > > > > >> > > > > metrics > > > > > >> > > > > > > has > > > > > >> > > > > > > > > its > > > > > >> > > > > > > > > > > own > > > > > >> > > > > > > > > > > > >> bootstrap, retention, and lifecycle. > > Dmitri > > > > > noted > > > > > >> > > > Polaris > > > > > >> > > > > > > being > > > > > >> > > > > > > > a > > > > > >> > > > > > > > > > > > platform > > > > > >> > > > > > > > > > > > >> should have the flexibility to support > > > > > different > > > > > >> > > > > persistence > > > > > >> > > > > > > > > > backends > > > > > >> > > > > > > > > > > > per > > > > > >> > > > > > > > > > > > >> concern, and pointed to a concrete next > > step > > > > of > > > > > >> > > > separating > > > > > >> > > > > > the > > > > > >> > > > > > > > > JDBC > > > > > >> > > > > > > > > > > > >> bootstrap for metrics from the > metastore > > > > > >> bootstrap. > > > > > >> > > > Robert > > > > > >> > > > > > > > > proposed > > > > > >> > > > > > > > > > an > > > > > >> > > > > > > > > > > > >> additional UX extension: detect an > > > > > unbootstrapped > > > > > >> > > > metrics > > > > > >> > > > > > > store > > > > > >> > > > > > > > on > > > > > >> > > > > > > > > > > first > > > > > >> > > > > > > > > > > > >> use and auto-bootstrap rather than > > requiring > > > > an > > > > > >> > > explicit > > > > > >> > > > > > > manual > > > > > >> > > > > > > > > > > > bootstrap > > > > > >> > > > > > > > > > > > >> step. > > > > > >> > > > > > > > > > > > >> The meeting also confirmed that Polaris > > > > metrics > > > > > >> can > > > > > >> > > > start > > > > > >> > > > > > > small > > > > > >> > > > > > > > > and > > > > > >> > > > > > > > > > > stay > > > > > >> > > > > > > > > > > > >> Iceberg-focused. Naming and persistence > > > > schema > > > > > >> can > > > > > >> > > lean > > > > > >> > > > > > > > > > > > Iceberg-specific. > > > > > >> > > > > > > > > > > > >> If a future expansion to generic-table > > > > metrics > > > > > or > > > > > >> > > > > > operational > > > > > >> > > > > > > > > > metrics > > > > > >> > > > > > > > > > > > >> arrives, an abstraction layer can be > > built on > > > > > >> top of > > > > > >> > > the > > > > > >> > > > > > > Iceberg > > > > > >> > > > > > > > > > > metrics > > > > > >> > > > > > > > > > > > >> reporter at that point. Robert remains > > on the > > > > > >> fence > > > > > >> > > and > > > > > >> > > > > > would > > > > > >> > > > > > > > > prefer > > > > > >> > > > > > > > > > > > >> something more generic but did not > block > > the > > > > > >> > > direction; > > > > > >> > > > > > > Dmitri's > > > > > >> > > > > > > > > > read > > > > > >> > > > > > > > > > > > was > > > > > >> > > > > > > > > > > > >> that the proposed framework already has > > > > enough > > > > > >> > > > flexibility > > > > > >> > > > > > to > > > > > >> > > > > > > > > absorb > > > > > >> > > > > > > > > > > > >> future > > > > > >> > > > > > > > > > > > >> expansion. > > > > > >> > > > > > > > > > > > >> > > > > > >> > > > > > > > > > > > >> The Trade-offs and Proposed structure > > > > sections > > > > > in > > > > > >> > the > > > > > >> > > > doc > > > > > >> > > > > > were > > > > > >> > > > > > > > not > > > > > >> > > > > > > > > > > > >> reviewed > > > > > >> > > > > > > > > > > > >> in detail. They remain open for either > > the > > > > next > > > > > >> sync > > > > > >> > > or > > > > > >> > > > PR > > > > > >> > > > > > > > review. > > > > > >> > > > > > > > > > > > >> > > > > > >> > > > > > > > > > > > >> *Cross-cutting alignment — > > battery-included > > > > > plus > > > > > >> > > > > pluggable* > > > > > >> > > > > > > > > > > > >> A common philosophy emerged from the > > > > > discussion. > > > > > >> EJ > > > > > >> > > > > > summarized > > > > > >> > > > > > > > it > > > > > >> > > > > > > > > > as: > > > > > >> > > > > > > > > > > > >> Polaris should provide a > > battery-included UX > > > > > for > > > > > >> > > > beginners > > > > > >> > > > > > and > > > > > >> > > > > > > > the > > > > > >> > > > > > > > > > > > >> flexibility for advanced users to swap > > the > > > > > >> included > > > > > >> > > > > battery > > > > > >> > > > > > > for > > > > > >> > > > > > > > > > > > something > > > > > >> > > > > > > > > > > > >> more powerful or tailored to their use > > case. > > > > > The > > > > > >> SPI > > > > > >> > > > > design > > > > > >> > > > > > > > needs > > > > > >> > > > > > > > > to > > > > > >> > > > > > > > > > > > >> enable > > > > > >> > > > > > > > > > > > >> both. > > > > > >> > > > > > > > > > > > >> > > > > > >> > > > > > > > > > > > >> The inputs that shaped this framing: > > > > > >> > > > > > > > > > > > >> • Anand described how his team uses > the > > > > > current > > > > > >> > > > metrics > > > > > >> > > > > > > > > > persistence > > > > > >> > > > > > > > > > > > >> (three metrics consumers in v1.4). > > > > > >> > > > > > > > > > > > >> • Yufei raised Grafana and dashboard > > > > > >> integrations > > > > > >> > > as a > > > > > >> > > > > > > > > destination > > > > > >> > > > > > > > > > > use > > > > > >> > > > > > > > > > > > >> case beyond the default. > > > > > >> > > > > > > > > > > > >> • Robert called out that the current > > design > > > > > is > > > > > >> > more > > > > > >> > > > > > > > > JDBC-focused. > > > > > >> > > > > > > > > > > > >> > > > > > >> > > > > > > > > > > > >> Two concrete instances: > > > > > >> > > > > > > > > > > > >> • Async metrics intake: Yufei's > initial > > > > > >> position > > > > > >> > was > > > > > >> > > > > that > > > > > >> > > > > > > > async > > > > > >> > > > > > > > > > > should > > > > > >> > > > > > > > > > > > >> largely live on the producer side and > > there > > > > is > > > > > >> not > > > > > >> > > much > > > > > >> > > > > > > Polaris > > > > > >> > > > > > > > > can > > > > > >> > > > > > > > > > > do. > > > > > >> > > > > > > > > > > > >> Robert suggested a Polaris-side default > > is > > > > > doable > > > > > >> > via > > > > > >> > > > > > Vert.x. > > > > > >> > > > > > > > > Dmitri > > > > > >> > > > > > > > > > > > >> agreed > > > > > >> > > > > > > > > > > > >> the direction is worth exploring. The > > meeting > > > > > >> > > converged > > > > > >> > > > > on a > > > > > >> > > > > > > > > > > > >> battery-included default (likely > > > > Vert.x-backed) > > > > > >> with > > > > > >> > > an > > > > > >> > > > > SPI > > > > > >> > > > > > > > shape > > > > > >> > > > > > > > > > that > > > > > >> > > > > > > > > > > > >> lets > > > > > >> > > > > > > > > > > > >> power users route to a more scalable > > backend > > > > > >> > > (k8s-hosted > > > > > >> > > > > > > queue, > > > > > >> > > > > > > > > AWS > > > > > >> > > > > > > > > > > SQS, > > > > > >> > > > > > > > > > > > >> etc.). > > > > > >> > > > > > > > > > > > >> • Pluggable destinations: combining > > Yufei's > > > > > >> > > dashboard > > > > > >> > > > > use > > > > > >> > > > > > > case > > > > > >> > > > > > > > > > with > > > > > >> > > > > > > > > > > > >> Robert's JDBC-focused call-out, the > > meeting > > > > > >> agreed > > > > > >> > the > > > > > >> > > > SPI > > > > > >> > > > > > > > should > > > > > >> > > > > > > > > be > > > > > >> > > > > > > > > > > > >> structured for multiple sinks so > > integrations > > > > > >> become > > > > > >> > > > impl > > > > > >> > > > > > > > choices > > > > > >> > > > > > > > > > > rather > > > > > >> > > > > > > > > > > > >> than architectural changes. > > > > > >> > > > > > > > > > > > >> > > > > > >> > > > > > > > > > > > >> The battery-included default is most > > likely > > > > to > > > > > >> use > > > > > >> > the > > > > > >> > > > > > > existing > > > > > >> > > > > > > > > > > > >> JDBC-backed > > > > > >> > > > > > > > > > > > >> approach. > > > > > >> > > > > > > > > > > > >> > > > > > >> > > > > > > > > > > > >> *Direction (idea-level alignment)* > > > > > >> > > > > > > > > > > > >> • Single metrics_report table > > consolidating > > > > > >> > per-type > > > > > >> > > > > > > metrics, > > > > > >> > > > > > > > > > > > replacing > > > > > >> > > > > > > > > > > > >> scan_metrics_report and > > commit_metrics_report > > > > > >> > > > > > > > > > > > >> • Iceberg-focused naming and schema > for > > > > now, > > > > > >> > revisit > > > > > >> > > > if > > > > > >> > > > > > > > > > > generic-table > > > > > >> > > > > > > > > > > > or > > > > > >> > > > > > > > > > > > >> operational metrics arrive > > > > > >> > > > > > > > > > > > >> • Metrics persistence as a separated > > SPI, > > > > not > > > > > >> on > > > > > >> > > > > > > > BasePersistence > > > > > >> > > > > > > > > > > > >> • Bootstrap path separated for > metrics, > > > > > >> > independent > > > > > >> > > of > > > > > >> > > > > > > > metastore > > > > > >> > > > > > > > > > > > >> bootstrap > > > > > >> > > > > > > > > > > > >> • "Battery-included plus pluggable" > as > > the > > > > > SPI > > > > > >> > > design > > > > > >> > > > > > > > philosophy > > > > > >> > > > > > > > > > > > >> > > > > > >> > > > > > > > > > > > >> *Open items* > > > > > >> > > > > > > > > > > > >> • Schema details: > > metric_schema_version, > > > > > >> payload > > > > > >> > > > format, > > > > > >> > > > > > IRC > > > > > >> > > > > > > > v2 > > > > > >> > > > > > > > > > > > >> forward-compat shape > > > > > >> > > > > > > > > > > > >> • SPI design details — full review > > either > > > > in > > > > > >> the > > > > > >> > > next > > > > > >> > > > > sync > > > > > >> > > > > > > or > > > > > >> > > > > > > > in > > > > > >> > > > > > > > > > the > > > > > >> > > > > > > > > > > > >> corresponding PR > > > > > >> > > > > > > > > > > > >> • Schema refactor PR ownership > > > > > >> > > > > > > > > > > > >> > > > > > >> > > > > > > > > > > > >> *Action items* > > > > > >> > > > > > > > > > > > >> • EJ to take a first stab at the SPI > > design > > > > > and > > > > > >> > > > > > potentially > > > > > >> > > > > > > > > > partner > > > > > >> > > > > > > > > > > > with > > > > > >> > > > > > > > > > > > >> Anand to incorporate the lessons > learned > > from > > > > > the > > > > > >> > > > existing > > > > > >> > > > > > > > > reporter > > > > > >> > > > > > > > > > > and > > > > > >> > > > > > > > > > > > >> persistence work. > > > > > >> > > > > > > > > > > > >> • Schema refactor PR ownership is not > > yet > > > > > >> decided. > > > > > >> > > If > > > > > >> > > > > > anyone > > > > > >> > > > > > > > is > > > > > >> > > > > > > > > > > > >> interested in driving it, reply on this > > > > thread. > > > > > >> > > > > > > > > > > > >> • JB to schedule the next sync, > > tentatively > > > > > in > > > > > >> two > > > > > >> > > > > weeks. > > > > > >> > > > > > > > > > > > >> > > > > > >> > > > > > > > > > > > >> -ej > > > > > >> > > > > > > > > > > > >> > > > > > >> > > > > > > > > > > > >> On Mon, Apr 27, 2026 at 3:07 PM EJ > Wang < > > > > > >> > > > > > > > > > > [email protected] > > > > > >> > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > >> wrote: > > > > > >> > > > > > > > > > > > >> > > > > > >> > > > > > > > > > > > >> > Thanks Yufei for the +1. > > > > > >> > > > > > > > > > > > >> > > > > > > >> > > > > > > > > > > > >> > JB, could you help add a biweekly > > metrics > > > > > >> > > architecture > > > > > >> > > > > > sync > > > > > >> > > > > > > to > > > > > >> > > > > > > > > the > > > > > >> > > > > > > > > > > > >> Polaris > > > > > >> > > > > > > > > > > > >> > community calendar? I'm thinking > > Thursdays > > > > at > > > > > >> > 9-10am > > > > > >> > > > PT, > > > > > >> > > > > > on > > > > > >> > > > > > > > the > > > > > >> > > > > > > > > > > > >> off-weeks > > > > > >> > > > > > > > > > > > >> > from the community meeting (starting > > May > > > > 7), > > > > > 60 > > > > > >> > > > minutes. > > > > > >> > > > > > > > > > > > >> > > > > > > >> > > > > > > > > > > > >> > Here's a rough agenda to work through > > over > > > > > the > > > > > >> > first > > > > > >> > > > few > > > > > >> > > > > > > > > sessions, > > > > > >> > > > > > > > > > > > >> grouped > > > > > >> > > > > > > > > > > > >> > by priority: > > > > > >> > > > > > > > > > > > >> > > > > > > >> > > > > > > > > > > > >> > *First: foundational direction* > > > > > >> > > > > > > > > > > > >> > > > > > > >> > > > > > > > > > > > >> > 1. MetricsPersistence: public SPI or > > > > > internal > > > > > >> > > > > > > implementation > > > > > >> > > > > > > > > > > detail? > > > > > >> > > > > > > > > > > > >> > • Marked @Beta, javadoc calls > it a > > > > > >> "Service > > > > > >> > > > > Provider > > > > > >> > > > > > > > > > > Interface", > > > > > >> > > > > > > > > > > > >> but > > > > > >> > > > > > > > > > > > >> > only one consumer > > > > (JdbcBasePersistenceImpl), > > > > > >> lives > > > > > >> > > on > > > > > >> > > > > > > > > > > BasePersistence. > > > > > >> > > > > > > > > > > > >> If > > > > > >> > > > > > > > > > > > >> > demoted to a private helper inside a > > > > > persisting > > > > > >> > > > reporter > > > > > >> > > > > > > impl, > > > > > >> > > > > > > > > > most > > > > > >> > > > > > > > > > > > >> > downstream design decisions become > > > > > >> implementation > > > > > >> > > > > details > > > > > >> > > > > > > > rather > > > > > >> > > > > > > > > > > than > > > > > >> > > > > > > > > > > > >> > contract questions. > > > > > >> > > > > > > > > > > > >> > > > > > > >> > > > > > > > > > > > >> > 2. Persistence schema redesign > > > > > >> > > > > > > > > > > > >> > • Current two-table layout > > > > > >> > > (scan_metrics_report, > > > > > >> > > > > > > > > > > > >> > commit_metrics_report) with ~25 > > flattened > > > > > >> columns > > > > > >> > > > each. > > > > > >> > > > > > > Every > > > > > >> > > > > > > > > new > > > > > >> > > > > > > > > > > > metric > > > > > >> > > > > > > > > > > > >> > type requires a new table, SPI > method, > > > > record > > > > > >> > class, > > > > > >> > > > > > model, > > > > > >> > > > > > > > > > > converter, > > > > > >> > > > > > > > > > > > >> and > > > > > >> > > > > > > > > > > > >> > schema migration. Direction to > explore: > > > > > single > > > > > >> > table > > > > > >> > > > > with > > > > > >> > > > > > > > > > > metric_type > > > > > >> > > > > > > > > > > > >> enum, > > > > > >> > > > > > > > > > > > >> > schema_version, and JSON payload > > column. > > > > > >> > > > > > > > > > > > >> > > > > > > >> > > > > > > > > > > > >> > *Second: design details once > direction > > is > > > > > set* > > > > > >> > > > > > > > > > > > >> > > > > > > >> > > > > > > > > > > > >> > 3. Partition key strategy > > > > > >> > > > > > > > > > > > >> > • Single-table design means scan > > > > metrics > > > > > >> at > > > > > >> > > scale > > > > > >> > > > > > will > > > > > >> > > > > > > > have > > > > > >> > > > > > > > > > > high > > > > > >> > > > > > > > > > > > >> > write concurrency per table. Schema > > needs > > > > to > > > > > >> > expose > > > > > >> > > > > enough > > > > > >> > > > > > > > > > structure > > > > > >> > > > > > > > > > > > for > > > > > >> > > > > > > > > > > > >> > backends to shard by entity or time > > range. > > > > > >> > > > > > > > > > > > >> > > > > > > >> > > > > > > > > > > > >> > 4. Read/write path consistency > > > > > >> > > > > > > > > > > > >> > • Writes go through > > > > > PolarisMetricsManager > > > > > >> on > > > > > >> > > > > > > > > > MetaStoreManager. > > > > > >> > > > > > > > > > > > >> Reads > > > > > >> > > > > > > > > > > > >> > bypass MetaStoreManager and go > > straight to > > > > > >> > > > > > BasePersistence, > > > > > >> > > > > > > > > > > excluding > > > > > >> > > > > > > > > > > > >> > non-JDBC backends from the read API. > > > > > >> > > > > > > > > > > > >> > > > > > > >> > > > > > > > > > > > >> > *Third: cleanup and alignment* > > > > > >> > > > > > > > > > > > >> > > > > > > >> > > > > > > > > > > > >> > 5. PolarisMetricsReporter naming > > > > > >> > > > > > > > > > > > >> > • Only handles IRC > > > > > >> (ScanReport/CommitReport), > > > > > >> > > > > doesn't > > > > > >> > > > > > > > cover > > > > > >> > > > > > > > > > > > generic > > > > > >> > > > > > > > > > > > >> > tables or operational metrics. Name > is > > > > > broader > > > > > >> > than > > > > > >> > > > > scope. > > > > > >> > > > > > > > > > > > >> > > > > > > >> > > > > > > > > > > > >> > 6. PolarisMetricsManager facade > > > > passthrough > > > > > >> > > > > > > > > > > > >> > • Entire default method is > > > > > >> > > > > > > > > > > > >> > callCtx.getMetaStore().writeScanReport(). > > > > > >> > > > > > > > > > > > >> > Zero logic, passes Level 1 straight > > through > > > > > to > > > > > >> > Level > > > > > >> > > > 3. > > > > > >> > > > > > Same > > > > > >> > > > > > > > > > > > >> anti-pattern > > > > > >> > > > > > > > > > > > >> > as PolarisEventManager. > > > > > >> > > > > > > > > > > > >> > > > > > > >> > > > > > > > > > > > >> > 7. Iceberg community alignment > > > > > >> > > > > > > > > > > > >> > • Payload-type extension needs > > > > > discussion > > > > > >> on > > > > > >> > > > > > > dev@iceberg. > > > > > >> > > > > > > > > > > > >> obelix74's > > > > > >> > > > > > > > > > > > >> > Feb thread got zero replies. Needs a > > > > > committer > > > > > >> > > voice. > > > > > >> > > > > > > > > > > > >> > > > > > > >> > > > > > > > > > > > >> > Lets confirm prioritization in the > > first > > > > > >> session. > > > > > >> > > > > > > > > > > > >> > > > > > > >> > > > > > > > > > > > >> > -ej > > > > > >> > > > > > > > > > > > >> > > > > > > >> > > > > > > > > > > > >> > On Tue, Apr 21, 2026 at 3:18 PM Yufei > > Gu < > > > > > >> > > > > > > > [email protected]> > > > > > >> > > > > > > > > > > > wrote: > > > > > >> > > > > > > > > > > > >> > > > > > > >> > > > > > > > > > > > >> >> Thanks everyone for continuing to > > drive > > > > this > > > > > >> > > > forward. I > > > > > >> > > > > > > agree > > > > > >> > > > > > > > > > that > > > > > >> > > > > > > > > > > > the > > > > > >> > > > > > > > > > > > >> >> problem is getting complex enough > > that a > > > > > more > > > > > >> > > > > structured > > > > > >> > > > > > > > > > discussion > > > > > >> > > > > > > > > > > > >> would > > > > > >> > > > > > > > > > > > >> >> help. > > > > > >> > > > > > > > > > > > >> >> > > > > > >> > > > > > > > > > > > >> >> +1 on setting up a biweekly sync for > > the > > > > > >> metrics > > > > > >> > > > > > > > architecture. > > > > > >> > > > > > > > > > I’m > > > > > >> > > > > > > > > > > > >> happy > > > > > >> > > > > > > > > > > > >> >> to > > > > > >> > > > > > > > > > > > >> >> join. > > > > > >> > > > > > > > > > > > >> >> > > > > > >> > > > > > > > > > > > >> >> Yufei > > > > > >> > > > > > > > > > > > >> >> > > > > > >> > > > > > > > > > > > >> >> > > > > > >> > > > > > > > > > > > >> >> On Tue, Apr 21, 2026 at 2:34 PM EJ > > Wang < > > > > > >> > > > > > > > > > > > >> [email protected]> > > > > > >> > > > > > > > > > > > >> >> wrote: > > > > > >> > > > > > > > > > > > >> >> > > > > > >> > > > > > > > > > > > >> >> > Also, I've been looking more > > closely at > > > > > the > > > > > >> > > > > > *persistence > > > > > >> > > > > > > > > schema > > > > > >> > > > > > > > > > > in > > > > > >> > > > > > > > > > > > >> the > > > > > >> > > > > > > > > > > > >> >> > current metrics work*, and I think > > > > > there's a > > > > > >> > > > > structural > > > > > >> > > > > > > > > > rigidity > > > > > >> > > > > > > > > > > > >> problem > > > > > >> > > > > > > > > > > > >> >> > worth raising before the shape > gets > > > > locked > > > > > >> in. > > > > > >> > > > > > > > > > > > >> >> > > > > > > >> > > > > > > > > > > > >> >> > Right now we have two separate > > tables > > > > > >> > > > > > > (scan_metrics_report > > > > > >> > > > > > > > > and > > > > > >> > > > > > > > > > > > >> >> > commit_metrics_report), each with > > ~25 > > > > > >> flattened > > > > > >> > > > > columns > > > > > >> > > > > > > > that > > > > > >> > > > > > > > > > > > directly > > > > > >> > > > > > > > > > > > >> >> > mirror the Iceberg report fields. > > The > > > > SPI > > > > > >> > follows > > > > > >> > > > the > > > > > >> > > > > > > same > > > > > >> > > > > > > > > > split: > > > > > >> > > > > > > > > > > > >> >> > writeScanReport and > > writeCommitReport as > > > > > >> > separate > > > > > >> > > > > > > methods, > > > > > >> > > > > > > > > with > > > > > >> > > > > > > > > > > > >> per-type > > > > > >> > > > > > > > > > > > >> >> > record classes, converters, and > > model > > > > > >> objects. > > > > > >> > > *The > > > > > >> > > > > > > > practical > > > > > >> > > > > > > > > > > cost: > > > > > >> > > > > > > > > > > > >> >> > adding a new metric type > > (operational > > > > > >> metrics, > > > > > >> > > for > > > > > >> > > > > > > example) > > > > > >> > > > > > > > > > > > requires > > > > > >> > > > > > > > > > > > >> a > > > > > >> > > > > > > > > > > > >> >> new > > > > > >> > > > > > > > > > > > >> >> > table, a new SPI method, a new > > record > > > > > >> class, a > > > > > >> > > new > > > > > >> > > > > > model > > > > > >> > > > > > > > > > class, a > > > > > >> > > > > > > > > > > > new > > > > > >> > > > > > > > > > > > >> >> > converter branch, and a schema > > > > migration*. > > > > > >> > > That's a > > > > > >> > > > > lot > > > > > >> > > > > > > of > > > > > >> > > > > > > > > > > surface > > > > > >> > > > > > > > > > > > >> area > > > > > >> > > > > > > > > > > > >> >> > for what should be "one more kind > of > > > > > >> metric." > > > > > >> > > > > > > > > > > > >> >> > > > > > > >> > > > > > > > > > > > >> >> > *My bias* would be toward a single > > > > metrics > > > > > >> > table > > > > > >> > > > with > > > > > >> > > > > > *a > > > > > >> > > > > > > > > typed > > > > > >> > > > > > > > > > > JSON > > > > > >> > > > > > > > > > > > >> >> > payload*. Something like: > > metric_type > > > > > >> (enum), > > > > > >> > > > > > entity_id, > > > > > >> > > > > > > > > > > > >> >> > table_identifier, snapshot_id > > > > (nullable), > > > > > >> > > > > received_ts, > > > > > >> > > > > > > > > > > > >> schema_version, > > > > > >> > > > > > > > > > > > >> >> and > > > > > >> > > > > > > > > > > > >> >> > a payload column for the > > metric-specific > > > > > >> data. > > > > > >> > > The > > > > > >> > > > > > > > > metric_type > > > > > >> > > > > > > > > > + > > > > > >> > > > > > > > > > > > >> >> > schema_version pair gives us a > > > > > >> > forward-compatible > > > > > >> > > > > > > contract > > > > > >> > > > > > > > > for > > > > > >> > > > > > > > > > > the > > > > > >> > > > > > > > > > > > >> >> payload > > > > > >> > > > > > > > > > > > >> >> > shape. Adding a new metric type > > becomes > > > > an > > > > > >> enum > > > > > >> > > > value > > > > > >> > > > > > > and a > > > > > >> > > > > > > > > > > payload > > > > > >> > > > > > > > > > > > >> >> schema, > > > > > >> > > > > > > > > > > > >> >> > not a schema migration. One thing > I > > > > think > > > > > we > > > > > >> > need > > > > > >> > > > to > > > > > >> > > > > be > > > > > >> > > > > > > > > > > deliberate > > > > > >> > > > > > > > > > > > >> >> about is > > > > > >> > > > > > > > > > > > >> >> > the partition key design. If all > > metric > > > > > >> types > > > > > >> > > land > > > > > >> > > > in > > > > > >> > > > > > one > > > > > >> > > > > > > > > > table, > > > > > >> > > > > > > > > > > > scan > > > > > >> > > > > > > > > > > > >> >> > metrics at scale (high > concurrency, > > high > > > > > >> > > frequency > > > > > >> > > > > > across > > > > > >> > > > > > > > > many > > > > > >> > > > > > > > > > > > >> tables) > > > > > >> > > > > > > > > > > > >> >> > could easily create hot > partitions. > > We'd > > > > > >> want > > > > > >> > the > > > > > >> > > > > > > > persistence > > > > > >> > > > > > > > > > > layer > > > > > >> > > > > > > > > > > > >> to > > > > > >> > > > > > > > > > > > >> >> be > > > > > >> > > > > > > > > > > > >> >> > able to shard by entity or time > > range, > > > > and > > > > > >> that > > > > > >> > > > means > > > > > >> > > > > > the > > > > > >> > > > > > > > > > logical > > > > > >> > > > > > > > > > > > >> schema > > > > > >> > > > > > > > > > > > >> >> > needs to expose enough structure > for > > > > > >> backends > > > > > >> > to > > > > > >> > > > > > > partition > > > > > >> > > > > > > > > on. > > > > > >> > > > > > > > > > I > > > > > >> > > > > > > > > > > > >> don't > > > > > >> > > > > > > > > > > > >> >> > think the current flattened layout > > gives > > > > > us > > > > > >> > that. > > > > > >> > > > > > > > > > > > >> >> > > > > > > >> > > > > > > > > > > > >> >> > This is getting complex enough > that > > I > > > > > don't > > > > > >> > think > > > > > >> > > > > > ad-hoc > > > > > >> > > > > > > > > PR/ML > > > > > >> > > > > > > > > > > > >> threads > > > > > >> > > > > > > > > > > > >> >> > will converge well. *Would people > be > > > > open > > > > > >> to a > > > > > >> > > > > biweekly > > > > > >> > > > > > > > sync > > > > > >> > > > > > > > > > for > > > > > >> > > > > > > > > > > > >> metrics > > > > > >> > > > > > > > > > > > >> >> > architecture?* I think 30 minutes > > every > > > > > two > > > > > >> > weeks > > > > > >> > > > > with > > > > > >> > > > > > > > > > interested > > > > > >> > > > > > > > > > > > >> >> parties > > > > > >> > > > > > > > > > > > >> >> > would be enough to work through > the > > > > > schema, > > > > > >> SPI > > > > > >> > > > > shape, > > > > > >> > > > > > > and > > > > > >> > > > > > > > > read > > > > > >> > > > > > > > > > > API > > > > > >> > > > > > > > > > > > >> >> design > > > > > >> > > > > > > > > > > > >> >> > together. Happy to help set that > up. > > > > > >> > > > > > > > > > > > >> >> > > > > > > >> > > > > > > > > > > > >> >> > -ej > > > > > >> > > > > > > > > > > > >> >> > > > > > > >> > > > > > > > > > > > >> >> > On Mon, Apr 20, 2026 at 2:19 PM EJ > > Wang > > > > < > > > > > >> > > > > > > > > > > > >> [email protected] > > > > > >> > > > > > > > > > > > >> >> > > > > > > >> > > > > > > > > > > > >> >> > wrote: > > > > > >> > > > > > > > > > > > >> >> > > > > > > >> > > > > > > > > > > > >> >> >> Reviewed #4115, left a comment on > > the > > > > > code > > > > > >> > > > > > organization > > > > > >> > > > > > > > > side. > > > > > >> > > > > > > > > > > > >> >> >> > > > > > >> > > > > > > > > > > > >> >> >> One thing stood out: the metrics > > write > > > > > path > > > > > >> > > enters > > > > > >> > > > > > > through > > > > > >> > > > > > > > > > > > >> >> >> PolarisMetricsManager on > > > > > MetaStoreManager, > > > > > >> but > > > > > >> > > the > > > > > >> > > > > new > > > > > >> > > > > > > > read > > > > > >> > > > > > > > > > path > > > > > >> > > > > > > > > > > > >> >> bypasses > > > > > >> > > > > > > > > > > > >> >> >> MetaStoreManager entirely and > goes > > > > > >> straight to > > > > > >> > > > > > > > > BasePersistence > > > > > >> > > > > > > > > > > via > > > > > >> > > > > > > > > > > > >> >> >> callContext.getMetaStore(). That > > means > > > > > the > > > > > >> > read > > > > > >> > > > API > > > > > >> > > > > > only > > > > > >> > > > > > > > > works > > > > > >> > > > > > > > > > > for > > > > > >> > > > > > > > > > > > >> >> backends > > > > > >> > > > > > > > > > > > >> >> >> that implement BasePersistence. > > NoSQL > > > > and > > > > > >> > remote > > > > > >> > > > > > > backends > > > > > >> > > > > > > > > > can't > > > > > >> > > > > > > > > > > > >> >> participate. > > > > > >> > > > > > > > > > > > >> >> >> > > > > > >> > > > > > > > > > > > >> >> >> Stepping back, I think the > metrics > > > > > >> subsystem > > > > > >> > is > > > > > >> > > > > > growing > > > > > >> > > > > > > > into > > > > > >> > > > > > > > > > > > >> something > > > > > >> > > > > > > > > > > > >> >> >> real (write + read + REST API + > > AuthZ + > > > > > >> > > > pagination) > > > > > >> > > > > > *but > > > > > >> > > > > > > > the > > > > > >> > > > > > > > > > > > >> >> persistence > > > > > >> > > > > > > > > > > > >> >> >> side is split across two layers > in > > a > > > > way > > > > > >> > that's > > > > > >> > > > hard > > > > > >> > > > > > to > > > > > >> > > > > > > > > > > extend*. I > > > > > >> > > > > > > > > > > > >> put > > > > > >> > > > > > > > > > > > >> >> >> together two diagrams to show > what > > I > > > > mean > > > > > >> (my > > > > > >> > > best > > > > > >> > > > > > > > effort). > > > > > >> > > > > > > > > > > > >> >> >> > > > > > >> > > > > > > > > > > > >> >> >> *Current state* (Diagram 1): > three > > > > > >> interfaces > > > > > >> > at > > > > > >> > > > > three > > > > > >> > > > > > > > > > different > > > > > >> > > > > > > > > > > > >> >> levels. > > > > > >> > > > > > > > > > > > >> >> >> The engine-facing SPI > > > > > >> (PolarisMetricsReporter) > > > > > >> > > is > > > > > >> > > > > > clean. > > > > > >> > > > > > > > But > > > > > >> > > > > > > > > > > > >> >> >> PolarisMetricsManager on > > > > MetaStoreManager > > > > > >> is a > > > > > >> > > > > > > passthrough > > > > > >> > > > > > > > > to > > > > > >> > > > > > > > > > > > >> >> >> MetricsPersistence on > > BasePersistence. > > > > > The > > > > > >> > @Beta > > > > > >> > > > > > > > annotation > > > > > >> > > > > > > > > > and > > > > > >> > > > > > > > > > > > SPI > > > > > >> > > > > > > > > > > > >> >> javadoc > > > > > >> > > > > > > > > > > > >> >> >> are on the BasePersistence layer, > > while > > > > > the > > > > > >> > > actual > > > > > >> > > > > > > > extension > > > > > >> > > > > > > > > > > > points > > > > > >> > > > > > > > > > > > >> >> >> (PolarisMetricsReporter, > > > > > >> > PolarisMetricsManager) > > > > > >> > > > > carry > > > > > >> > > > > > no > > > > > >> > > > > > > > > > > stability > > > > > >> > > > > > > > > > > > >> >> >> annotation. The write path goes > > through > > > > > the > > > > > >> > > > > > > > MetaStoreManager > > > > > >> > > > > > > > > > > > layer, > > > > > >> > > > > > > > > > > > >> the > > > > > >> > > > > > > > > > > > >> >> >> read path doesn't. > > > > > >> > > > > > > > > > > > >> >> >> > > > > > >> > > > > > > > > > > > >> >> >> *What I envision* (Diagram 2): > two > > SPIs > > > > > at > > > > > >> two > > > > > >> > > > > levels. > > > > > >> > > > > > > > > > > > >> >> >> PolarisMetricsReporter stays as > the > > > > > >> > > engine-facing > > > > > >> > > > > SPI. > > > > > >> > > > > > > > > > > > >> >> >> PolarisMetricsManager becomes the > > > > > >> > backend-facing > > > > > >> > > > SPI > > > > > >> > > > > > > with > > > > > >> > > > > > > > > both > > > > > >> > > > > > > > > > > > write > > > > > >> > > > > > > > > > > > >> >> and > > > > > >> > > > > > > > > > > > >> >> >> read methods at the > > MetaStoreManager > > > > > level, > > > > > >> > > where > > > > > >> > > > > any > > > > > >> > > > > > > > > backend > > > > > >> > > > > > > > > > > > (JDBC, > > > > > >> > > > > > > > > > > > >> >> NoSQL, > > > > > >> > > > > > > > > > > > >> >> >> remote) can implement them. > > > > > >> MetricsPersistence > > > > > >> > > on > > > > > >> > > > > > > > > > > BasePersistence > > > > > >> > > > > > > > > > > > >> goes > > > > > >> > > > > > > > > > > > >> >> >> away. Where metrics actually land > > is an > > > > > >> > > > > implementation > > > > > >> > > > > > > > > detail, > > > > > >> > > > > > > > > > > > not a > > > > > >> > > > > > > > > > > > >> >> core > > > > > >> > > > > > > > > > > > >> >> >> interface. > > > > > >> > > > > > > > > > > > >> >> >> > > > > > >> > > > > > > > > > > > >> >> >> *Minor naming thing*: > > > > > >> PolarisMetricsReporter > > > > > >> > is > > > > > >> > > > > > broader > > > > > >> > > > > > > > than > > > > > >> > > > > > > > > > > what > > > > > >> > > > > > > > > > > > it > > > > > >> > > > > > > > > > > > >> >> >> actually handles. It only accepts > > > > Iceberg > > > > > >> REST > > > > > >> > > > > Catalog > > > > > >> > > > > > > > > metrics > > > > > >> > > > > > > > > > > > >> >> (ScanReport, > > > > > >> > > > > > > > > > > > >> >> >> CommitReport via MetricsReport). > > > > Generic > > > > > >> table > > > > > >> > > > > metrics > > > > > >> > > > > > > or > > > > > >> > > > > > > > > > > > >> operational > > > > > >> > > > > > > > > > > > >> >> >> metrics aren't in scope. Not > > blocking, > > > > > but > > > > > >> > worth > > > > > >> > > > > > noting > > > > > >> > > > > > > if > > > > > >> > > > > > > > > the > > > > > >> > > > > > > > > > > > >> metrics > > > > > >> > > > > > > > > > > > >> >> >> surface expands. > > > > > >> > > > > > > > > > > > >> >> >> > > > > > >> > > > > > > > > > > > >> >> >> *Rough sketch of how to get > there*: > > > > > >> > > > > > > > > > > > >> >> >> 1. Add read methods to > > > > > >> PolarisMetricsManager > > > > > >> > > > > > > > > > (listScanReports, > > > > > >> > > > > > > > > > > > >> >> >> listCommitReports) with default > > no-op, > > > > > >> same as > > > > > >> > > the > > > > > >> > > > > > > > existing > > > > > >> > > > > > > > > > > write > > > > > >> > > > > > > > > > > > >> >> methods. > > > > > >> > > > > > > > > > > > >> >> >> (Probably make > > PolarisMetricsManager > > > > more > > > > > >> > > explicit > > > > > >> > > > > on > > > > > >> > > > > > > > being > > > > > >> > > > > > > > > > > > Iceberg > > > > > >> > > > > > > > > > > > >> >> >> specific like package name or > class > > > > name > > > > > >> etc.) > > > > > >> > > > > > > > > > > > >> >> >> 2. Wire MetricsReportsService > > through > > > > > >> > > > > > MetaStoreManager > > > > > >> > > > > > > > > > instead > > > > > >> > > > > > > > > > > > of > > > > > >> > > > > > > > > > > > >> >> >> callContext.getMetaStore(). > > > > > >> > > > > > > > > > > > >> >> >> 3. Extract metrics persistence > > from > > > > > >> > > > > > > > > JdbcBasePersistenceImpl > > > > > >> > > > > > > > > > > into > > > > > >> > > > > > > > > > > > >> its > > > > > >> > > > > > > > > > > > >> >> >> own class. That file carries ~7 > > > > > >> > > responsibilities, > > > > > >> > > > > > > metrics > > > > > >> > > > > > > > > > being > > > > > >> > > > > > > > > > > > one > > > > > >> > > > > > > > > > > > >> of > > > > > >> > > > > > > > > > > > >> >> them. > > > > > >> > > > > > > > > > > > >> >> >> 4. Remove MetricsPersistence > from > > > > > >> > > > BasePersistence. > > > > > >> > > > > > > > > > > > >> >> >> > > > > > >> > > > > > > > > > > > >> >> >> *None of this needs to happen in > > #4115. > > > > > >> But if > > > > > >> > > the > > > > > >> > > > > > > > direction > > > > > >> > > > > > > > > > > makes > > > > > >> > > > > > > > > > > > >> >> sense, > > > > > >> > > > > > > > > > > > >> >> >> it would be good to align before > > the > > > > > >> metrics > > > > > >> > > > surface > > > > > >> > > > > > > grows > > > > > >> > > > > > > > > > > > further. > > > > > >> > > > > > > > > > > > >> >> Curious > > > > > >> > > > > > > > > > > > >> >> >> what others think.* > > > > > >> > > > > > > > > > > > >> >> >> > > > > > >> > > > > > > > > > > > >> >> >> *My mental model note*: Level 1 > > > > > >> > > MetaStoreManager; > > > > > >> > > > > > level > > > > > >> > > > > > > 2 > > > > > >> > > > > > > > > > > > >> transactional > > > > > >> > > > > > > > > > > > >> >> >> persistence; level 3 base > > persistence > > > > > >> > > > > > > > > > > > >> >> >> > > > > > >> > > > > > > > > > > > >> >> >> Diagram 1 > > > > > >> > > > > > > > > > > > >> >> >> < > > > > > >> > > > > > > > > > > > >> >> > > > > > >> > > > > > > > > > > > >> > > > > > >> > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > >> > > > > > > > > > > > > > >> > > > > > > > > > > > > >> > > > > > > > > > > > >> > > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > > > > > > > > https://urldefense.com/v3/__https://www.plantuml.com/plantuml/uml/bLHDR-Cs4BthLmpIYupw0zbkKQ1r3M-S7Bp8xhhM7WCOb3IM65EaGD9EX2RzxHrHb4CxRelwa4YSDu_lpOVcnZ9jzvM8BBS2uGjQpJC3dtHMSekPtMk44IpsMgEqa5XcCOhCZikQQLP1pR8TAp2n3ILhmZDP20m0fcIvUkAoW2qJXd9z1bpToO9BX3WXu0ucy5rpgGPNm0nW5_epUWtm2Ue3pn3kMOFQmKntGZW0BYtgBSi8k5A2QMwybJNMIbFiGSR9QZc4nUqIvikStF0jHprua5C-amge42aNt3R0f5JaaoivdV2Pkqbx4hee4ymOkBh5BTiB-_uIeGeo8zL8rPsPl4DktdEiK1jkB1NdZCRbrSTecDe_mlHbF0wvBmCkaOH5_S8a_TTTKI6-nmCAkEw4LpxsZ-LbYLKQFKMNOgf_wuM7_bV9gOer5SYMMksBSWXFcbi49KNZXNLicwfe3TETC7gPdPqI7uBcHMb1RSzYq34c6PDUM9mn8HRsUTZEiDBve3NjVZumBj0U7SS37mGO7vcwtiK-_pU7U7L_f-digo9YbhSwIfMRwIITKGXbxdIUTCGF1SeCJxloKsU-3k9ddRbX1eDq1q_fx1JbBGT0glVyXimDuP4TQ5qpCAmnGEj2s_6n5mtn1z-97-63itFQZLPO1Ev2tu_WF7Ju-VPc0Skg5bYXxBhkY1xpD7EM_7fyflSpIsqMgVth5xhVr4eQxWQ8enaSAJQSG16yFSDuJ798rrcXr_3n-lfdk7icQjEBmFujL7AodiP_Y4Z7-YxvtZNs4zMgpNTl6tF8sglyPsmqchrjvQ-m-aP94r-TwCA2Ka8upPJZwtvSpoYCXkYMZU2NXvRMBfq9P3i3Le4VAZUAlUZ_oPKsxPgY0Q_BSKLkyr9bhQhQrJjo_x3TPlIB0DPjnMfcIoYP0QaYw1a0fTKDr8fB6ntNuvmoL1ZGkXa69Njh43zf9GiGxHQrA_jDYWRSzF5--WmTVrN97_Sm8LbLUy_lGBmLanJjFkDlGkRqjA_4tm00__;!!Iz9xO38YGHZK!4rnlVIqBQQQ6xpGe1PMV58edOS_cGrKoqdfhw4qSp_JQCavvEjaxXk0OwGPnxyEcO60AzKQK_XDtp7H3DdkFXcrXuTBISHg$ > > > > > >> > > > > > > > > > > > >> >> > > > > > > >> > > > > > > > > > > > >> >> >> : > > > > > >> > > > > > > > > > > > >> >> >> > > > > > >> > > > > > > > > > > > >> >> >> [image: image.png] > > > > > >> > > > > > > > > > > > >> >> >> > > > > > >> > > > > > > > > > > > >> >> >> Diagram 2 > > > > > >> > > > > > > > > > > > >> >> >> < > > > > > >> > > > > > > > > > > > >> >> > > > > > >> > > > > > > > > > > > >> > > > > > >> > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > >> > > > > > > > > > > > > > >> > > > > > > > > > > > > >> > > > > > > > > > > > >> > > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > > > > > > > > https://urldefense.com/v3/__https://www.plantuml.com/plantuml/uml/VLLDR-8m4BtdLupO2sWBLVU8AaGB7AXAbssGzb896SS9RXqxjKqBwkv_tt7iV43fdaZYDpFlpRmnOsE9jhjSH9PRmM31hERKm8scMsuPjJlDe0yheZDc8RR4iYWoBrmMH9CS2a9VICPYUy1OZN0YCy5Q0BCbYNhdCeEK28En8G8wCvbnoQ0R8_05Bc6bkLIz3X03p1zzH7zR-9ZfDquPt9C3qoNCX2yV4G2NbkcKu5jdgGJHt0GbZwnG6i-UP3TUpk5gM6Ldqke350eZUqzoCft3U9xWHvxoa5-7K4nF1J46EbEMafsmdrCBbQ44gVggy18IZrn_ph5asd1ZiIKdQSgueZvjXrQFSFrdC3YN-nXmBacxbGiYyLVxLaBtdhqn0LSzdBDhqQtQoOJeGyad3z0lUqnYgpGB6Ns8oVyta00Dy_WnX0tIOZ8v6SYxHll1TrH6aejAik-mh-AphVFCwSUQqFypElag5QRGFDjQKEd96K1P8QP41c9TzA_IIQyvdAWyv_RSiS3skb0_EzDDkK2v5xWF6MiGFlvhpFLcD2Dq2pml14gaF67eQkmd8gulDoC4kSOu6KVpkvlUJg1RTbWISU40RdBUUS_9XfRZ2dwxm_SW8LYFISgm_MnlDQ6M9P1gbKEc4X-2pH_FvJCkCqm9pbVjD6LrwdLeOrDWfOaqc8Wh9BE85oNKxkNQ6o4yGRy_Eae0G_G8tZv81d3bHDB23WOdisohVr3nh_j6lbSjbNaLRTc8UgtPbAU1J_tygOfZX9DWEJeHDvYx-qmSi5FgNLPZwHrHcUsncGQ5-skhUclpE5fo4ounpFauYrUbkU6ccfnxMvitwag4IyerhTxj8In_Oj1bDO4pQru674loYrGlULHLEGCjwJJ8gDoVZR8MxO4BT3IzRvIcAQKezC6xpziGnTyImrfEGyJI_OcKfgtxIvnTqFEMS17L9Z-jsARN5FmTheP7HtSdtOMT0B4GY2FYHXxgQmMtj2bRqiLFGapiVe1_QVKDrkqXcm83aFEXnMYCZ-xlyHy__;!!Iz9xO38YGHZK!4rnlVIqBQQQ6xpGe1PMV58edOS_cGrKoqdfhw4qSp_JQCavvEjaxXk0OwGPnxyEcO60AzKQK_XDtp7H3DdkFXcrXSpmqcNk$ > > > > > >> > > > > > > > > > > > >> >> > > > > > > >> > > > > > > > > > > > >> >> >> : > > > > > >> > > > > > > > > > > > >> >> >> [image: image.png] > > > > > >> > > > > > > > > > > > >> >> >> > > > > > >> > > > > > > > > > > > >> >> >> -ej > > > > > >> > > > > > > > > > > > >> >> >> > > > > > >> > > > > > > > > > > > >> >> >> On Wed, Apr 15, 2026 at 8:22 AM > > Dmitri > > > > > >> > > > Bourlatchkov > > > > > >> > > > > < > > > > > >> > > > > > > > > > > > >> [email protected]> > > > > > >> > > > > > > > > > > > >> >> >> wrote: > > > > > >> > > > > > > > > > > > >> >> >> > > > > > >> > > > > > > > > > > > >> >> >>> Hi All, > > > > > >> > > > > > > > > > > > >> >> >>> > > > > > >> > > > > > > > > > > > >> >> >>> Heads up: The current state of > PR > > > > [4115] > > > > > >> > looks > > > > > >> > > > > pretty > > > > > >> > > > > > > > solid > > > > > >> > > > > > > > > > to > > > > > >> > > > > > > > > > > > me. > > > > > >> > > > > > > > > > > > >> I > > > > > >> > > > > > > > > > > > >> >> >>> believe this PR is approaching a > > > > > mergeable > > > > > >> > > > > condition. > > > > > >> > > > > > > > > > > > >> >> >>> > > > > > >> > > > > > > > > > > > >> >> >>> Please post your reviews if you > > have > > > > any > > > > > >> > > > comments. > > > > > >> > > > > > > > > > > > >> >> >>> > > > > > >> > > > > > > > > > > > >> >> >>> [4115] > > > > > >> > > > > https://urldefense.com/v3/__https://github.com/apache/polaris/pull/4115__;!!Iz9xO38YGHZK!4rnlVIqBQQQ6xpGe1PMV58edOS_cGrKoqdfhw4qSp_JQCavvEjaxXk0OwGPnxyEcO60AzKQK_XDtp7H3DdkFXcrXtFRscKs$ > > > > > >> > > > > > > > > > > > >> >> >>> > > > > > >> > > > > > > > > > > > >> >> >>> Thanks, > > > > > >> > > > > > > > > > > > >> >> >>> Dmitri. > > > > > >> > > > > > > > > > > > >> >> >>> > > > > > >> > > > > > > > > > > > >> >> >>> On Tue, Mar 3, 2026 at 3:29 PM > > Anand > > > > > Kumar > > > > > >> > > > Sankaran > > > > > >> > > > > > via > > > > > >> > > > > > > > > dev < > > > > > >> > > > > > > > > > > > >> >> >>> [email protected]> wrote: > > > > > >> > > > > > > > > > > > >> >> >>> > > > > > >> > > > > > > > > > > > >> >> >>> > Hi Yufei and Dmitri, > > > > > >> > > > > > > > > > > > >> >> >>> > > > > > > >> > > > > > > > > > > > >> >> >>> > Here is a proposal for the > REST > > > > > >> endpoints > > > > > >> > for > > > > > >> > > > > > metrics > > > > > >> > > > > > > > and > > > > > >> > > > > > > > > > > > events. > > > > > >> > > > > > > > > > > > >> >> >>> > > > > > > >> > > > > > > > > > > > >> >> >>> > > > > > > >> > > > > > > https://urldefense.com/v3/__https://github.com/apache/polaris/pull/3924/changes__;!!Iz9xO38YGHZK!4rnlVIqBQQQ6xpGe1PMV58edOS_cGrKoqdfhw4qSp_JQCavvEjaxXk0OwGPnxyEcO60AzKQK_XDtp7H3DdkFXcrXcjoPhSs$ > > > > > >> > > > > > > > > > > > >> >> >>> > > > > > > >> > > > > > > > > > > > >> >> >>> > I did not see any precursors > for > > > > > >> raising a > > > > > >> > PR > > > > > >> > > > for > > > > > >> > > > > > > > > > proposals, > > > > > >> > > > > > > > > > > so > > > > > >> > > > > > > > > > > > >> >> trying > > > > > >> > > > > > > > > > > > >> >> >>> > this. Please let me know what > > you > > > > > >> think. > > > > > >> > > > > > > > > > > > >> >> >>> > > > > > > >> > > > > > > > > > > > >> >> >>> > - > > > > > >> > > > > > > > > > > > >> >> >>> > Anand > > > > > >> > > > > > > > > > > > >> >> >>> > > > > > > >> > > > > > > > > > > > >> >> >>> > From: Anand Kumar Sankaran < > > > > > >> > > > > > > [email protected] > > > > > >> > > > > > > > > > > > > > >> > > > > > > > > > > > >> >> >>> > Date: Monday, March 2, 2026 at > > > > > 10:25 AM > > > > > >> > > > > > > > > > > > >> >> >>> > To: [email protected] < > > > > > >> > > > > [email protected] > > > > > >> > > > > > > > > > > > >> > > > > > > > > > > > >> >> >>> > Subject: Re: Polaris Telemetry > > and > > > > > Audit > > > > > >> > > Trail > > > > > >> > > > > > > > > > > > >> >> >>> > > > > > > >> > > > > > > > > > > > >> >> >>> > About the REST API, based on > my > > use > > > > > >> cases: > > > > > >> > > > > > > > > > > > >> >> >>> > > > > > > >> > > > > > > > > > > > >> >> >>> > > > > > > >> > > > > > > > > > > > >> >> >>> > 1. > > > > > >> > > > > > > > > > > > >> >> >>> > I want to be able to query > > commit > > > > > >> metrics > > > > > >> > to > > > > > >> > > > > track > > > > > >> > > > > > > > files > > > > > >> > > > > > > > > > > added > > > > > >> > > > > > > > > > > > / > > > > > >> > > > > > > > > > > > >> >> >>> removed > > > > > >> > > > > > > > > > > > >> >> >>> > per commit, along with record > > > > counts. > > > > > >> The > > > > > >> > > > > ingestion > > > > > >> > > > > > > > > > pipeline > > > > > >> > > > > > > > > > > > that > > > > > >> > > > > > > > > > > > >> >> >>> writes > > > > > >> > > > > > > > > > > > >> >> >>> > this data is owned by us and > we > > are > > > > > >> > > guaranteed > > > > > >> > > > to > > > > > >> > > > > > > write > > > > > >> > > > > > > > > > this > > > > > >> > > > > > > > > > > > >> >> >>> information > > > > > >> > > > > > > > > > > > >> >> >>> > for each write. > > > > > >> > > > > > > > > > > > >> >> >>> > 2. > > > > > >> > > > > > > > > > > > >> >> >>> > I want to be able to query > scan > > > > > metrics > > > > > >> for > > > > > >> > > > > read. I > > > > > >> > > > > > > > > > > understand > > > > > >> > > > > > > > > > > > >> >> clients > > > > > >> > > > > > > > > > > > >> >> >>> do > > > > > >> > > > > > > > > > > > >> >> >>> > not fulfill this requirement. > > > > > >> > > > > > > > > > > > >> >> >>> > 3. > > > > > >> > > > > > > > > > > > >> >> >>> > I want to be able to query the > > > > events > > > > > >> table > > > > > >> > > > > (events > > > > > >> > > > > > > are > > > > > >> > > > > > > > > > > > >> persisted) - > > > > > >> > > > > > > > > > > > >> >> >>> this > > > > > >> > > > > > > > > > > > >> >> >>> > may supersede #2, I am not > sure > > yet. > > > > > >> > > > > > > > > > > > >> >> >>> > > > > > > >> > > > > > > > > > > > >> >> >>> > All this information is in the > > JDBC > > > > > >> based > > > > > >> > > > > > persistence > > > > > >> > > > > > > > > model > > > > > >> > > > > > > > > > > and > > > > > >> > > > > > > > > > > > >> is > > > > > >> > > > > > > > > > > > >> >> >>> > persisted in the metastore. I > > > > > currently > > > > > >> > don’t > > > > > >> > > > > have > > > > > >> > > > > > a > > > > > >> > > > > > > > need > > > > > >> > > > > > > > > > to > > > > > >> > > > > > > > > > > > >> query > > > > > >> > > > > > > > > > > > >> >> >>> > prometheus or open telemetry. > I > > do > > > > > >> publish > > > > > >> > > some > > > > > >> > > > > > > events > > > > > >> > > > > > > > to > > > > > >> > > > > > > > > > > > >> Prometheus > > > > > >> > > > > > > > > > > > >> >> >>> and > > > > > >> > > > > > > > > > > > >> >> >>> > they are forwarded to our > > dashboards > > > > > >> > > elsewhere. > > > > > >> > > > > > > > > > > > >> >> >>> > > > > > > >> > > > > > > > > > > > >> >> >>> > About the CLI utilities, I > > meant the > > > > > >> admin > > > > > >> > > user > > > > > >> > > > > > > > > utilities. > > > > > >> > > > > > > > > > In > > > > > >> > > > > > > > > > > > >> one of > > > > > >> > > > > > > > > > > > >> >> >>> the > > > > > >> > > > > > > > > > > > >> >> >>> > earliest drafts of my > proposal, > > > > > Prashant > > > > > >> > > > > mentioned > > > > > >> > > > > > > that > > > > > >> > > > > > > > > the > > > > > >> > > > > > > > > > > > >> metrics > > > > > >> > > > > > > > > > > > >> >> >>> tables > > > > > >> > > > > > > > > > > > >> >> >>> > can grow indefinitely and > that a > > > > > similar > > > > > >> > > > problem > > > > > >> > > > > > > exists > > > > > >> > > > > > > > > > with > > > > > >> > > > > > > > > > > > the > > > > > >> > > > > > > > > > > > >> >> events > > > > > >> > > > > > > > > > > > >> >> >>> > table as well. We discussed > that > > > > > >> cleaning > > > > > >> > up > > > > > >> > > of > > > > > >> > > > > old > > > > > >> > > > > > > > > records > > > > > >> > > > > > > > > > > > from > > > > > >> > > > > > > > > > > > >> >> both > > > > > >> > > > > > > > > > > > >> >> >>> > metrics tables and events > > tables can > > > > > be > > > > > >> > done > > > > > >> > > > via > > > > > >> > > > > a > > > > > >> > > > > > > CLI > > > > > >> > > > > > > > > > > utility. > > > > > >> > > > > > > > > > > > >> >> >>> > > > > > > >> > > > > > > > > > > > >> >> >>> > I see that Yufei has covered > the > > > > > >> discussion > > > > > >> > > > about > > > > > >> > > > > > > > > > > datasources. > > > > > >> > > > > > > > > > > > >> >> >>> > > > > > > >> > > > > > > > > > > > >> >> >>> > - > > > > > >> > > > > > > > > > > > >> >> >>> > Anand > > > > > >> > > > > > > > > > > > >> >> >>> > > > > > > >> > > > > > > > > > > > >> >> >>> > > > > > > >> > > > > > > > > > > > >> >> >>> > > > > > > >> > > > > > > > > > > > >> >> >>> > From: Yufei Gu < > > > > [email protected]> > > > > > >> > > > > > > > > > > > >> >> >>> > Date: Friday, February 27, > 2026 > > at > > > > > >> 9:54 PM > > > > > >> > > > > > > > > > > > >> >> >>> > To: [email protected] < > > > > > >> > > > > [email protected] > > > > > >> > > > > > > > > > > > >> > > > > > > > > > > > >> >> >>> > Subject: Re: Polaris Telemetry > > and > > > > > Audit > > > > > >> > > Trail > > > > > >> > > > > > > > > > > > >> >> >>> > > > > > > >> > > > > > > > > > > > >> >> >>> > This Message Is From an > External > > > > > Sender > > > > > >> > > > > > > > > > > > >> >> >>> > This message came from outside > > your > > > > > >> > > > organization. > > > > > >> > > > > > > > > > > > >> >> >>> > Report Suspicious< > > > > > >> > > > > > > > > > > > >> >> >>> > > > > > > >> > > > > > > > > > > > >> >> >>> > > > > > >> > > > > > > > > > > > >> >> > > > > > >> > > > > > > > > > > > >> > > > > > >> > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > >> > > > > > > > > > > > > > >> > > > > > > > > > > > > >> > > > > > > > > > > > >> > > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > > > > > > > > https://us-phishalarm-ewt.proofpoint.com/EWT/v1/Iz9xO38YGHZK!YhNDZABkHi1B699ote2uMwpOZw8i0QMCGO2Szc-HshuABGhGvwPJcymE6G2oUUxtS8xDkSrtGTPm_I3QnVDHoLMk50m9v8z_nZKTkd-bnVUbreF1u0WnfV_X5eYevZl_$ > > > > > >> > > > > > > > > > > > >> >> >>> > > > > > > > >> > > > > > > > > > > > >> >> >>> > > > > > > >> > > > > > > > > > > > >> >> >>> > > > > > > >> > > > > > > > > > > > >> >> >>> > As I mentioned in > > > > > >> > > > > > > > > > > > >> >> >>> > > > > > > >> > > > > > > > > > > > >> >> >>> > > > > > >> > > > > > > > > > > > >> >> > > > > > >> > > > > > > > > > > > >> > > > > > >> > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > >> > > > > > > > > > > > > > >> > > > > > > > > > > > > >> > > > > > > > > > > > >> > > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > > > > > > > > https://urldefense.com/v3/__https://github.com/apache/polaris/issues/3890__;!!Iz9xO38YGHZK!5EuyFFkk3vhRWVIRvQAWBSQfpJkTMA9HxugzDwXmN0LPPqhEFxYkFRGVhtb8AqUwXtDh2OplcMnbMDHKOxrvDU0$ > >> > > >> > > > > > > > > > > > >> >> >>> , > > > > > >> > > > > > > > > > > > >> >> >>> > supporting > > > > > >> > > > > > > > > > > > >> >> >>> > multiple data sources is not a > > > > trivial > > > > > >> > > change. > > > > > >> > > > I > > > > > >> > > > > > > would > > > > > >> > > > > > > > > > > strongly > > > > > >> > > > > > > > > > > > >> >> >>> recommend > > > > > >> > > > > > > > > > > > >> >> >>> > starting with a design > document > > to > > > > > >> > carefully > > > > > >> > > > > > evaluate > > > > > >> > > > > > > > the > > > > > >> > > > > > > > > > > > >> >> architectural > > > > > >> > > > > > > > > > > > >> >> >>> > implications and long term > > impact. > > > > > >> > > > > > > > > > > > >> >> >>> > > > > > > >> > > > > > > > > > > > >> >> >>> > A REST endpoint to query > metrics > > > > seems > > > > > >> > > > reasonable > > > > > >> > > > > > > given > > > > > >> > > > > > > > > the > > > > > >> > > > > > > > > > > > >> current > > > > > >> > > > > > > > > > > > >> >> >>> JDBC > > > > > >> > > > > > > > > > > > >> >> >>> > based persistence model. That > > said, > > > > we > > > > > >> may > > > > > >> > > also > > > > > >> > > > > > > > consider > > > > > >> > > > > > > > > > > > >> alternative > > > > > >> > > > > > > > > > > > >> >> >>> > storage models. For example, > if > > we > > > > > later > > > > > >> > > adopt > > > > > >> > > > a > > > > > >> > > > > > time > > > > > >> > > > > > > > > > series > > > > > >> > > > > > > > > > > > >> system > > > > > >> > > > > > > > > > > > >> >> >>> such as > > > > > >> > > > > > > > > > > > >> >> >>> > Prometheus to store metrics, > the > > > > query > > > > > >> > model > > > > > >> > > > and > > > > > >> > > > > > > access > > > > > >> > > > > > > > > > > > patterns > > > > > >> > > > > > > > > > > > >> >> would > > > > > >> > > > > > > > > > > > >> >> >>> be > > > > > >> > > > > > > > > > > > >> >> >>> > fundamentally different. > > Designing > > > > the > > > > > >> REST > > > > > >> > > API > > > > > >> > > > > > > without > > > > > >> > > > > > > > > > > > >> considering > > > > > >> > > > > > > > > > > > >> >> >>> these > > > > > >> > > > > > > > > > > > >> >> >>> > potential evolutions may limit > > > > > >> flexibility. > > > > > >> > > I'd > > > > > >> > > > > > > suggest > > > > > >> > > > > > > > > to > > > > > >> > > > > > > > > > > > start > > > > > >> > > > > > > > > > > > >> >> with > > > > > >> > > > > > > > > > > > >> >> >>> the > > > > > >> > > > > > > > > > > > >> >> >>> > use case. > > > > > >> > > > > > > > > > > > >> >> >>> > > > > > > >> > > > > > > > > > > > >> >> >>> > Yufei > > > > > >> > > > > > > > > > > > >> >> >>> > > > > > > >> > > > > > > > > > > > >> >> >>> > > > > > > >> > > > > > > > > > > > >> >> >>> > On Fri, Feb 27, 2026 at > 3:42 PM > > > > Dmitri > > > > > >> > > > > > Bourlatchkov < > > > > > >> > > > > > > > > > > > >> >> [email protected]> > > > > > >> > > > > > > > > > > > >> >> >>> > wrote: > > > > > >> > > > > > > > > > > > >> >> >>> > > > > > > >> > > > > > > > > > > > >> >> >>> > > Hi Anand, > > > > > >> > > > > > > > > > > > >> >> >>> > > > > > > > >> > > > > > > > > > > > >> >> >>> > > Sharing my view... subject > to > > > > > >> discussion: > > > > > >> > > > > > > > > > > > >> >> >>> > > > > > > > >> > > > > > > > > > > > >> >> >>> > > 1. Adding non-IRC REST API > to > > > > > Polaris > > > > > >> is > > > > > >> > > > > > perfectly > > > > > >> > > > > > > > > fine. > > > > > >> > > > > > > > > > > > >> >> >>> > > > > > > > >> > > > > > > > > > > > >> >> >>> > > Figuring out specific > endpoint > > > > URIs > > > > > >> and > > > > > >> > > > > payloads > > > > > >> > > > > > > > might > > > > > >> > > > > > > > > > > > require > > > > > >> > > > > > > > > > > > >> a > > > > > >> > > > > > > > > > > > >> >> few > > > > > >> > > > > > > > > > > > >> >> >>> > > roundtrips, so opening a > > separate > > > > > >> thread > > > > > >> > > for > > > > > >> > > > > that > > > > > >> > > > > > > > might > > > > > >> > > > > > > > > > be > > > > > >> > > > > > > > > > > > >> best. > > > > > >> > > > > > > > > > > > >> >> >>> > > Contributors commonly create > > > > Google > > > > > >> Docs > > > > > >> > > for > > > > > >> > > > > new > > > > > >> > > > > > > API > > > > > >> > > > > > > > > > > > proposals > > > > > >> > > > > > > > > > > > >> too > > > > > >> > > > > > > > > > > > >> >> >>> (they > > > > > >> > > > > > > > > > > > >> >> >>> > > fairly easy to update as the > > email > > > > > >> > > discussion > > > > > >> > > > > > > > > > progresses). > > > > > >> > > > > > > > > > > > >> >> >>> > > > > > > > >> > > > > > > > > > > > >> >> >>> > > There was a suggestion to > try > > > > > Markdown > > > > > >> > > (with > > > > > >> > > > > PRs) > > > > > >> > > > > > > for > > > > > >> > > > > > > > > > > > proposals > > > > > >> > > > > > > > > > > > >> >> [1] > > > > > >> > > > > > > > > > > > >> >> >>> ... > > > > > >> > > > > > > > > > > > >> >> >>> > > feel free to give it a try > if > > you > > > > > are > > > > > >> > > > > comfortable > > > > > >> > > > > > > > with > > > > > >> > > > > > > > > > > that. > > > > > >> > > > > > > > > > > > >> >> >>> > > > > > > > >> > > > > > > > > > > > >> >> >>> > > 2. Could you clarify whether > > you > > > > > mean > > > > > >> end > > > > > >> > > > user > > > > > >> > > > > > > > > utilities > > > > > >> > > > > > > > > > or > > > > > >> > > > > > > > > > > > >> admin > > > > > >> > > > > > > > > > > > >> >> >>> user > > > > > >> > > > > > > > > > > > >> >> >>> > > utilities? In the latter > case > > > > those > > > > > >> might > > > > > >> > > be > > > > > >> > > > > more > > > > > >> > > > > > > > > > suitable > > > > > >> > > > > > > > > > > > for > > > > > >> > > > > > > > > > > > >> the > > > > > >> > > > > > > > > > > > >> >> >>> Admin > > > > > >> > > > > > > > > > > > >> >> >>> > > CLI (java) not the Python > CLI, > > > > IMHO. > > > > > >> > > > > > > > > > > > >> >> >>> > > > > > > > >> > > > > > > > > > > > >> >> >>> > > Why would these utilities be > > > > common > > > > > >> with > > > > > >> > > > > events? > > > > > >> > > > > > > > IMHO, > > > > > >> > > > > > > > > > > event > > > > > >> > > > > > > > > > > > >> use > > > > > >> > > > > > > > > > > > >> >> >>> cases > > > > > >> > > > > > > > > > > > >> >> >>> > are > > > > > >> > > > > > > > > > > > >> >> >>> > > distinct from scan/commit > > metrics. > > > > > >> > > > > > > > > > > > >> >> >>> > > > > > > > >> > > > > > > > > > > > >> >> >>> > > 3. I'd prefer separating > > metrics > > > > > >> > > persistence > > > > > >> > > > > from > > > > > >> > > > > > > > > > MetaStore > > > > > >> > > > > > > > > > > > >> >> >>> persistence > > > > > >> > > > > > > > > > > > >> >> >>> > at > > > > > >> > > > > > > > > > > > >> >> >>> > > the code level, so that they > > could > > > > > be > > > > > >> > mixed > > > > > >> > > > and > > > > > >> > > > > > > > matched > > > > > >> > > > > > > > > > > > >> >> >>> independently. > > > > > >> > > > > > > > > > > > >> >> >>> > The > > > > > >> > > > > > > > > > > > >> >> >>> > > separate datasource question > > will > > > > > >> become > > > > > >> > a > > > > > >> > > > > > > non-issue > > > > > >> > > > > > > > > with > > > > > >> > > > > > > > > > > > that > > > > > >> > > > > > > > > > > > >> >> >>> approach, > > > > > >> > > > > > > > > > > > >> >> >>> > I > > > > > >> > > > > > > > > > > > >> >> >>> > > guess. > > > > > >> > > > > > > > > > > > >> >> >>> > > > > > > > >> > > > > > > > > > > > >> >> >>> > > The rationale for separating > > scan > > > > > >> metrics > > > > > >> > > and > > > > > >> > > > > > > > metastore > > > > > >> > > > > > > > > > > > >> >> persistence > > > > > >> > > > > > > > > > > > >> >> >>> is > > > > > >> > > > > > > > > > > > >> >> >>> > that > > > > > >> > > > > > > > > > > > >> >> >>> > > "cascading deletes" between > > them > > > > are > > > > > >> > hardly > > > > > >> > > > > ever > > > > > >> > > > > > > > > > required. > > > > > >> > > > > > > > > > > > >> >> >>> Furthermore, > > > > > >> > > > > > > > > > > > >> >> >>> > the > > > > > >> > > > > > > > > > > > >> >> >>> > > data and query patterns are > > very > > > > > >> > different > > > > > >> > > so > > > > > >> > > > > > > > different > > > > > >> > > > > > > > > > > > >> >> technologies > > > > > >> > > > > > > > > > > > >> >> >>> > might > > > > > >> > > > > > > > > > > > >> >> >>> > > be beneficial in each case. > > > > > >> > > > > > > > > > > > >> >> >>> > > > > > > > >> > > > > > > > > > > > >> >> >>> > > [1] > > > > > >> > > > > > > > > > > > >> >> >>> > > > > > > >> > > > > > > > > > > > >> >> >>> > > > > > >> > > > > > > > > > > > >> >> > > > > > >> > > > > > > > > > > > >> > > > > > >> > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > >> > > > > > > > > > > > > > >> > > > > > > > > > > > > >> > > > > > > > > > > > >> > > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > > > > > > > > https://urldefense.com/v3/__https://lists.apache.org/thread/yto2wp982t43h1mqjwnslswhws5z47cy__;!!Iz9xO38YGHZK!5EuyFFkk3vhRWVIRvQAWBSQfpJkTMA9HxugzDwXmN0LPPqhEFxYkFRGVhtb8AqUwXtDh2OplcMnbMDHKxYDakNU$ > >> > > >> > > > > > > > > > > > >> >> >>> > > > > > > > >> > > > > > > > > > > > >> >> >>> > > Cheers, > > > > > >> > > > > > > > > > > > >> >> >>> > > Dmitri. > > > > > >> > > > > > > > > > > > >> >> >>> > > > > > > > >> > > > > > > > > > > > >> >> >>> > > On Fri, Feb 27, 2026 at > > 6:19 PM > > > > > Anand > > > > > >> > Kumar > > > > > >> > > > > > > Sankaran > > > > > >> > > > > > > > > via > > > > > >> > > > > > > > > > > dev > > > > > >> > > > > > > > > > > > < > > > > > >> > > > > > > > > > > > >> >> >>> > > [email protected]> > > wrote: > > > > > >> > > > > > > > > > > > >> >> >>> > > > > > > > >> > > > > > > > > > > > >> >> >>> > > > Thanks all. This PR is > > merged > > > > now. > > > > > >> > > > > > > > > > > > >> >> >>> > > > > > > > > >> > > > > > > > > > > > >> >> >>> > > > Here are the follow-up > > features > > > > / > > > > > >> work > > > > > >> > > > > needed. > > > > > >> > > > > > > > These > > > > > >> > > > > > > > > > > were > > > > > >> > > > > > > > > > > > >> all > > > > > >> > > > > > > > > > > > >> >> >>> part of > > > > > >> > > > > > > > > > > > >> >> >>> > > the > > > > > >> > > > > > > > > > > > >> >> >>> > > > merged PR at some point in > > time > > > > > and > > > > > >> > were > > > > > >> > > > > > removed > > > > > >> > > > > > > to > > > > > >> > > > > > > > > > > reduce > > > > > >> > > > > > > > > > > > >> >> scope. > > > > > >> > > > > > > > > > > > >> >> >>> > > > > > > > > >> > > > > > > > > > > > >> >> >>> > > > Please let me know what > you > > > > think. > > > > > >> > > > > > > > > > > > >> >> >>> > > > > > > > > >> > > > > > > > > > > > >> >> >>> > > > > > > > > >> > > > > > > > > > > > >> >> >>> > > > 1. A REST API to > paginate > > > > > through > > > > > >> > > table > > > > > >> > > > > > > metrics. > > > > > >> > > > > > > > > > This > > > > > >> > > > > > > > > > > > >> will be > > > > > >> > > > > > > > > > > > >> >> >>> > non-IRC > > > > > >> > > > > > > > > > > > >> >> >>> > > > standard addition. > > > > > >> > > > > > > > > > > > >> >> >>> > > > 2. Utilities for > > managing old > > > > > >> > records, > > > > > >> > > > > > should > > > > > >> > > > > > > be > > > > > >> > > > > > > > > > > common > > > > > >> > > > > > > > > > > > >> with > > > > > >> > > > > > > > > > > > >> >> >>> events. > > > > > >> > > > > > > > > > > > >> >> >>> > > > There was some discussion > > that > > > > it > > > > > >> > belongs > > > > > >> > > > to > > > > > >> > > > > > the > > > > > >> > > > > > > > CLI. > > > > > >> > > > > > > > > > > > >> >> >>> > > > 3. Separate datasource > > > > > (metrics, > > > > > >> > > events, > > > > > >> > > > > > even > > > > > >> > > > > > > > > other > > > > > >> > > > > > > > > > > > >> tables?). > > > > > >> > > > > > > > > > > > >> >> >>> > > > > > > > > >> > > > > > > > > > > > >> >> >>> > > > > > > > > >> > > > > > > > > > > > >> >> >>> > > > Anything else? > > > > > >> > > > > > > > > > > > >> >> >>> > > > > > > > > >> > > > > > > > > > > > >> >> >>> > > > - > > > > > >> > > > > > > > > > > > >> >> >>> > > > Anand > > > > > >> > > > > > > > > > > > >> >> >>> > > > > > > > > >> > > > > > > > > > > > >> >> >>> > > > > > > > > >> > > > > > > > > > > > >> >> >>> > > > > > > > >> > > > > > > > > > > > >> >> >>> > > > > > > >> > > > > > > > > > > > >> >> >>> > > > > > > >> > > > > > > > > > > > >> >> >>> > > > > > >> > > > > > > > > > > > >> >> >> > > > > > >> > > > > > > > > > > > >> >> > > > > > >> > > > > > > > > > > > >> > > > > > > >> > > > > > > > > > > > >> > > > > > >> > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > >> > > > > > > > > > > > > > >> > > > > > > > > > > > > >> > > > > > > > > > > > >> > > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > > > > > > > > > > > > > > >
