[
https://issues.apache.org/jira/browse/HBASE-25891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17357933#comment-17357933
]
Anoop Sam John commented on HBASE-25891:
----------------------------------------
bq.Since log roll during incremental or full backup is stored with prefix
`trslm:`. Can be used to identify which log files can be cleaned up
Means the WAL files will get renamed with this prefix? When those files become
eligible for deletion then?
Can u pls give a detailed design consideration then? Now that we dont have
this systen table at all, what happens when taking a full/incremental snapshot?
How WAL files been retained when backup refers to it? When that become eligible
for deletion? (Backup deleted/ another full backup came?) And how we make sure
we allow WAL deletion then?
> Remove dependence storing WAL filenames for backup
> --------------------------------------------------
>
> Key: HBASE-25891
> URL: https://issues.apache.org/jira/browse/HBASE-25891
> Project: HBase
> Issue Type: Improvement
> Components: backup&restore
> Affects Versions: 3.0.0-alpha-1
> Reporter: Mallikarjun
> Assignee: Mallikarjun
> Priority: Major
> Fix For: 3.0.0-alpha-1
>
>
> Currently WAL logs are stored in `backup:system` meta table
> {code:java}
> // code placeholder
> wals:preprod-dn-1%2C16020%2C1614844389000.1621996160175 column=meta:backupId,
> timestamp=1622003479895, value=backup_1622003358258
> wals:preprod-dn-1%2C16020%2C1614844389000.1621996160175 column=meta:file,
> timestamp=1622003479895,
> value=hdfs://store/hbase/oldWALs/preprod-dn-1%2C16020%2C1614844389000.1621996160175
> wals:preprod-dn-1%2C16020%2C1614844389000.1621996160175 column=meta:root,
> timestamp=1622003479895, value=s3a://2021-05-25--21-45-00--full/set1
> wals:preprod-dn-1%2C16020%2C1614844389000.1621999760280 column=meta:backupId,
> timestamp=1622003479895, value=backup_1622003358258
> wals:preprod-dn-1%2C16020%2C1614844389000.1621999760280 column=meta:file,
> timestamp=1622003479895,
> value=hdfs://store/hbase/oldWALs/preprod-dn-1%2C16020%2C1614844389000.1621999760280
> wals:preprod-dn-1%2C16020%2C1614844389000.1621999760280 column=meta:root,
> timestamp=1622003479895, value=s3a://2021-05-25--21-45-00--full/set1
> {code}
> Primarily used for following
> * WAL's are stored in meta table to check if a particular log has been
> backed up or not.
> * Check during incremental backup if a particular WAL is being backed up was
> covered during previous incremental backup or not.
> Changes for above 2 use cases.
> * Since log roll during incremental or full backup is stored with prefix
> `trslm:`. Can be used to identify which log files can be cleaned up
> * Check during incremental backup if a particular WAL is being backed up or
> not is redundant. No such a check is required
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
