Hi Ayush, I agree that a required migration should not be a surprise in the release note (it something we discussed while ago when we "moved" from EclipseLink to JDBC afair).
I'm +1 with the approach. I have a few comments though: - We have two PRs touching the schema. #6013 bumps the schema to v7, to add an index on events(realm_id, timestamp_ms), while #4756 changes the definition of idx_scan_report_lookup and idx_commit_report_lookup. As CREATE INDEX IF NOT EXISTS keeps the existing index, existing deployment won't get the new definition without a migration (same situation as idx_locations in v6). As 1.9.0 is already on vote, it means these changes will land in 1.10.0. I would prefer a single schema v7 for 1.10.0, with one migration section in the docs, and one note in the CHANGELOG, rather than v7 and v8 in the same release. Whichever PR is merged first, introducing v7, the others add their DDL to the same v7 bump. - Both PRs add an admin tool command deleting events in batches (purge events by age, purge/drain for realms). It would be good to align the command naming and share the batched delete logic before both are merged. - The schema version check is a strict equality: a 1.9.0 node fails against v7 schema, and a 1.10.0 node fails against v6. So, we must no mix versions during the upgrade. As realm_purge is pure addition (1.9.0 never touches it), the docs could split the migration in two steps: create the table while still running 1.9.0 (no impact), then bump the version. That keeps the downtime window to the version update and restart. - The time series data of realms purged before the upgrade is not in realm_purge, so purge/drain will never clean it. Maybe we can either state it explicitly in the docs, or provide an optional one-off SQL in the v7 migration section enqueueing the realm ids present in events and the metrics report tables that no longer exist in entities. Regards JB On Thu, Nov 12, 2026 at 7:19 AM Ayush Saxena <[email protected]> wrote: > Hi all, > > PR #5089 adds a table to the relational JDBC schema, so it needs a > schema version bump and a migration that operators must apply before > starting the release that carries it. Raising it here for awareness, > since a required migration should not be something people meet for the > first time in a release note. > > The table records realms whose time-series data is still pending > deletion. Purging a realm today leaves its events and metrics reports > behind, keyed by a realm id that no longer exists, so nothing reclaims > them afterwards. A new admin tool command drains the queue in bounded > batches, rather than deleting that data inline in the purge > transaction, which is the statement most likely to make a purge time > out on a realm with a long history. > > The migration SQL and the rest of the detail are here: > https://github.com/apache/polaris/pull/5089 > > Let me know in case anyone has any objections, feedback or better ideas. > > -Ayush >
