yuqi1129 opened a new issue, #11233: URL: https://github.com/apache/gravitino/issues/11233
### What would you like to be improved? `entity_change_log` is currently append-only in runtime. Although there is low-level mapper support for pruning old rows, Gravitino does not have an automatic cleanup mechanism for expired change log records stored in the database. As the table keeps growing, full change consumption or catch-up scans become slower, and the database keeps consuming more storage for stale change log records. ### How should we improve? Add automatic retention cleanup for `entity_change_log`. Suggested behavior: - Keep one day of change log records by default. - Automatically delete records older than the retention period. - Reuse the existing pruning capability where possible, such as `EntityChangeLogMapper.pruneOldEntityChanges`. - Make the retention period and cleanup interval configurable if needed. - Ensure cleanup does not break HA cache invalidation semantics or poller startup/catch-up behavior. - Add tests for the cleanup logic and supported JDBC SQL providers. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
