Hi Prithvi, Thanks for outlining the gaps. I'm not sure we need stronger consistency guarantees between request processing and audit persistence, or fail-closed behavior when the sink is unavailable, at this stage. It would help to understand the concrete requirements and availability tradeoffs before committing to those guarantees. Documenting the current delivery semantics sounds like a useful first step.
Expanding authN/authZ event coverage is something we can move forward with. Authentication failures and authorization allow/deny decisions would be useful first-class events, including identity and request context where available. We can improve that coverage independently of the delivery guarantees discussion. Yufei On Sun, Sep 20, 2026 at 12:23 PM Prithvi S <[email protected]> wrote: > Hi all, > > I opened https://github.com/apache/polaris/issues/5561 to discuss a > supported audit trail for Polaris operators. > > The event pipeline is already the right foundation: listeners since 1.0, > JDBC events + CloudWatch in 1.2.0 (preview), multi-listener in 1.5.0, the > async executor in 1.6.0, and Kafka plus OpenTelemetry listeners in 1.7.0. > Events can carry principal, event type, timestamp, request id, and > catalog/namespace/table identifiers, and DefaultEventSanitizer keeps > credentials out of the default payload. > > That is not yet an operator-facing audit story. HTTP access logs and the > default server log format do not reliably include the principal. Listener > delivery is best-effort (bounded backlog drops; the JDBC buffer is > in-memory, async, and outside the request transaction). Authentication > failures never reach the event delegator. Authorization denials are INFO > log lines, not PolarisEventTypes. polaris_schema.events has no query API, > secondary indexes, purge, or Polaris ACL, and NoSQL does not implement > writeEvents. > > This is not a request for Polaris to become a SIEM, add a Management API, > or build an in-process query engine. > > Questions for this thread: > > 1. Confirm that external listeners are the supported production audit path, > and that polaris_schema.events is not. > 2. Whether authentication failures and authorization allow/deny should > become first-class events. Success-only coverage is not enough for an audit > trail. > 3. Whether Polaris should document a production profile that does not > silently drop events, or fail closed when the sink is down. > 4. Whether phase 1 is docs only (recommended listener set, delivery > semantics, how to investigate a 403 in the sink), with coverage and > delivery changes as follow-ups. > > Cheers, > Prithvi S >
