mikebridge opened a new pull request, #45029: URL: https://github.com/apache/superset/pull/45029
### SUMMARY The scheduled soft-delete purge and version-history prune remove everything eligible in a single run. After cleanup has been paused, or the first time retention is enabled on an existing deployment, that can be a large backlog deleted at once. This adds a per-run cap to both scheduled tasks: - `SOFT_DELETE_PURGE_MAX_PER_RUN` (default 1000): soft-deleted root objects purged per run. A chart, dashboard or dataset together with its cascade counts as one; a cascade is never split. - `VERSION_HISTORY_PRUNE_MAX_TRANSACTIONS_PER_RUN` (default 1000): whole version-history transactions pruned per run; a transaction is never split. - `0` or `None` means no cap. Invalid values fail closed before any destructive work. A capped run reports `cap_reached` and how much remains, and the rest drains on later runs. Counting happens only after commit. The cap only ever reduces what a run deletes: eligibility, blocked-by-reference, restore and live-anchor protections are unchanged. Dry-run (including the new `VERSION_HISTORY_PRUNE_DRY_RUN`) counts the whole eligible backlog and reports `estimated_capped_runs`, so operators can size the first live run. This is a behaviour change: the cap is on by default. UPDATING.md and the version-history and recently-archived docs describe it. ### TESTING INSTRUCTIONS - `pytest tests/unit_tests/tasks/test_deletion_retention.py tests/unit_tests/tasks/test_version_history_retention.py` (142 pass) - Integration: `test_cap_preserves_whole_transactions_across_runs` - Covered: backlogs spanning batches and models, exact cap and remainder across repeated runs, 0/None and invalid values, dry-run with no writes, retry accounting and the cap-reached signal. - Not run locally: PostgreSQL and MySQL behaviour. ### ADDITIONAL INFORMATION - [ ] Has associated issue: - [ ] Required feature flags: - [ ] Changes UI - [ ] Includes DB Migration - [x] Introduces new feature or API - [ ] Removes existing feature or API Draft while two review items are addressed: the dry-run count's transaction isolation, and documenting that the cap bounds deletions rather than evaluations. 🤖 Generated with [Claude Code](https://claude.com/claude-code) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
