mikebridge commented on code in PR #43490:
URL: https://github.com/apache/superset/pull/43490#discussion_r4007924182
##########
UPDATING.md:
##########
@@ -945,7 +980,7 @@ With the flag on, delete confirmations across the
chart/dashboard/dataset list p
This also resolves the limitation noted under *Soft delete and restore for
datasets*: a database blocked by soft-deleted datasets can now be freed by
purging those datasets (per-entity endpoint, retention task, or `force-purge`
CLI) instead of hard-deleting `tables` rows out-of-band.
-The `purge_audit_log` table is **never pruned by design** — the audit must
survive the entities it names; operators who need to age it out should prune
manually.
+Automatic pruning of the `purge_audit_log` table is available but **off by
default**: set `PURGE_AUDIT_PRUNING_ENABLED = True` to enable the
`deletion_retention.prune_purge_audit` Celery beat task (daily, 03:30) so the
table no longer grows unbounded and does not need manual pruning. Left at its
default (`PURGE_AUDIT_PRUNING_ENABLED = False`) the table is never pruned and
grows indefinitely — enabling it is an explicit operator choice. The policy is
written to preserve the audit's meaning rather than trade it away: within an
entity's current blockage streak the earliest — "blocked since" — record always
survives (only redundant duplicate `blocked` records are collapsed), and
completed-destruction evidence (`confirmed`, `target_absent`) is **never**
removed unless the separate `PURGE_AUDIT_EVIDENCE_RETENTION_DAYS` opt-in is
explicitly set. What ages out is operational noise — `blocked` records from
already-resolved streaks and `failed` records — once older than `PURGE_AUDIT
_OPERATIONAL_RETENTION_DAYS` (default 90). See the release-note entry above
for the beat-schedule and `CELERY_CONFIG` details.
Review Comment:
Addressed in 07ade68f7e — the bounded-growth promise is gone; both the
release-note summary and the detailed section now state that force-triggered
`blocked` records are retained permanently (exempt from duplicate collapse and
operational age-out, including in resolved streaks), so repeated force-purge
attempts against a persistently blocked entity still add a record each.
##########
UPDATING.md:
##########
@@ -243,6 +243,41 @@ unknown impact as zero. Chart and dashboard purge
endpoints are unchanged.
- The dashboard datasource-based visibility fallback now fails closed: a
dashboard whose member charts’ datasources cannot be resolved (deleted
datasource rows, missing `datasource_id`, or unsupported datasource types) is
no longer accessible to users without explicit editor/viewer rights, and a
dashboard composed of semantic-view charts now requires `datasource_access` on
(at least one of) its semantic views or their parent semantic layer —
previously any authenticated user could open such a dashboard’s shell. Because
the fallback now considers every member chart rather than only table-backed
ones, a user holding `datasource_access` on any single member datasource —
including a semantic view or its parent layer — can open a mixed dashboard that
previously denied them. Dashboards with no charts remain accessible, and
dashboards with explicit viewers are unaffected. Conversely, holders of
`all_datasource_access` now see every published no-viewer dashboard in the
dashboard l
ist — including chart-less ones previously hidden by the inner joins —
matching what the object-level gate already allowed them to open.
- Version restore (`POST
/api/v1/{chart,dashboard,dataset}/<uuid>/versions/<version_uuid>/restore`) now
refuses an **externally managed** entity (`is_managed_externally = True`) with
HTTP 403, enforcing server-side what the docs already promised. Previously the
refusal existed only in the browser, so an otherwise-authorized editor could
restore such an entity by calling the endpoint directly and have the restore
overwritten on the next external sync. Soft-delete recovery is deliberately
unaffected — it changes visibility, not content.
+- The purge audit log can now be pruned automatically. The new
+ `deletion_retention.prune_purge_audit` Celery beat task (daily, 03:30, in the
+ default `CeleryConfig.beat_schedule`) removes duplicate `blocked` records
+ within an entity's current blockage streak (the earliest — "blocked since" —
+ record and the first record after each change of block `reason` always
+ survive, mirroring the audit writer's own suppression rule) and ages out
+ operational records (`blocked` from
+ resolved streaks, `failed`) older than
+ `PURGE_AUDIT_OPERATIONAL_RETENTION_DAYS` (default 90). A streak is ended
+ only by proof the object is gone (`confirmed`/`target_absent`); a `failed`
+ attempt does not reset the "blocked since" record. Completed-destruction
+ evidence (`confirmed`, `target_absent`) is **never touched** unless the
+ separate `PURGE_AUDIT_EVIDENCE_RETENTION_DAYS` opt-in is explicitly set,
+ which is the operator's assertion that an approved compliance policy
+ permits expiring destruction evidence. Automatic deletion is disabled by
+ default; set `PURGE_AUDIT_PRUNING_ENABLED = True` after reviewing these
Review Comment:
Addressed in 07ade68f7e — documented as an explicit two-phase rollout:
migrations + the coordinated writer code on every audit-writing process
(web/API, CLI, Celery) with old in-flight writes drained BEFORE the flag is
enabled, and pruning disabled with running batches finished before rolling any
writer back.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
