Hi JB, Yufei, A dedicated drop counter (e.g. polaris.metrics.reports.dropped, tagged by reporter) makes sense to me. Operators can alert on it directly, and Polaris doesn't have to return a 500 on POST .../metrics.
I'll add it on #5395 [1]. Thanks, Vignesh [1] https://github.com/apache/polaris/pull/5395 On Sun, 25 Oct 2026 at 15:02, Jean-Baptiste Onofré <[email protected]> wrote: > Hi all, > > Nice discussion! > > I lean towards the proposal in #5395. The reasons are the ones Dmitri gave. > The metrics POST is unidirectional and telemetry-like, so clients gain > nothing from seeing a persistence failure. The only status the IRC spec > allows for it is a 5xx, which is not really actionable for them :) > > I also agree with Yufei that a silent drop should not leave operators > "blind". Today the 500 is the only server-side signal we got, and a WARN > log is probably easy to miss (or ignore intentionally :)). > Maybe we could add a dedicated counter for dropped reports, for instance > polaris.metrics.reports.dropped, tagged by reporter type? Operators could > then alert on it directly. That would cover the visibility use case without > surfacing as catalog 5xx. > > Yufei's point about excluding the metrics endpoint from the core > availability SLO is also valid. Imho, that depends on how each deployment > aggregates its metrics, so I would consider a complement more than a > replacement for the server-side change. > > If the counter is acceptable, I'm +1 landing #5395 with it. > > Thanks! > > Regards > JB > > On Thu, Oct 22, 2026 at 8:12 PM vignesh a <[email protected]> wrote: > > > Hi all, > > > > PR #5395 [1] is about to land, and I wanted to flag it here because it > > changes REST behaviour compared to today. > > > > With this change, POST .../metrics no longer returns a 500 when the > > persisting reporter's DB write fails. The error is logged and the > > report is dropped. Errors from other reporters still propagate through > > the handler as before, and authorization is unchanged. > > > > Iceberg Java already ignores errors from the metrics endpoint. This > > change keeps a broken metrics store from showing up as catalog 5xx > > errors. > > > > Thoughts welcome, especially if anyone relies on the current 500 > > behaviour. > > > > Thanks, > > Vignesh > > > > [1] https://github.com/apache/polaris/pull/5395 > > >
