Hernan Gelaf-Romer created HBASE-29628:
------------------------------------------
Summary: Modern backups can cause oldWALs buildup
Key: HBASE-29628
URL: https://issues.apache.org/jira/browse/HBASE-29628
Project: HBase
Issue Type: Bug
Components: backup&restore
Reporter: Hernan Gelaf-Romer
Assignee: Hernan Gelaf-Romer
Modern backups store host:lastWalRollTime in the system table. The
BackupLogCleaner uses this information to decide which oldWALs are safe to
cleanup, and which ones still need to be backed up.
Each time we take a full, or incremental backup, we roll the WAL files for all
hosts, and update the system table with the new WAL roll times. However, we
don't make any effort to clean up hosts that no longer exist on the cluster.
This means that oldWALs for hosts that have been removed from the cluster will
never be cleaned up. This can cause a non-trivial amount of data to pile up.
We should look to clean up entries in the system table that pertain to hosts
that no longer exist after we've taken a backup.
There is more discussion in
[this|https://github.com/apache/hbase/pull/5408#discussion_r2360362258] comment
chain
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
