[ 
https://issues.apache.org/jira/browse/HBASE-25891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mallikarjun updated HBASE-25891:
--------------------------------
    Description: 
Currently WAL logs are stored in `backup:system` meta table 
{code:java}
// code placeholder
wals:preprod-dn-1%2C16020%2C1614844389000.1621996160175 column=meta:backupId, 
timestamp=1622003479895, value=backup_1622003358258 
wals:preprod-dn-1%2C16020%2C1614844389000.1621996160175 column=meta:file, 
timestamp=1622003479895, 
value=hdfs://store/hbase/oldWALs/preprod-dn-1%2C16020%2C1614844389000.1621996160175
 wals:preprod-dn-1%2C16020%2C1614844389000.1621996160175 column=meta:root, 
timestamp=1622003479895, value=s3a://2021-05-25--21-45-00--full/set1 
wals:preprod-dn-1%2C16020%2C1614844389000.1621999760280 column=meta:backupId, 
timestamp=1622003479895, value=backup_1622003358258 
wals:preprod-dn-1%2C16020%2C1614844389000.1621999760280 column=meta:file, 
timestamp=1622003479895, 
value=hdfs://store/hbase/oldWALs/preprod-dn-1%2C16020%2C1614844389000.1621999760280
 wals:preprod-dn-1%2C16020%2C1614844389000.1621999760280 column=meta:root, 
timestamp=1622003479895, value=s3a://2021-05-25--21-45-00--full/set1 
{code}
 

This has several problems
 # Ever growing rows of wal's sourced for incremental backup is maintained and 
never cleaned up. 
 # Unnecessary to have wal log listed for performing incremental backup or for 
performing logcleaner.
 # No support for rsgroup. Hence tables belonging to rsgroups which doesn't 
have backup enabled also have to retain wals' and forever.

 

Proposed Solution:

 

 

  was:
Currently WAL logs are stored in `backup:system` meta table 
{code:java}
// code placeholder
wals:preprod-dn-1%2C16020%2C1614844389000.1621996160175 column=meta:backupId, 
timestamp=1622003479895, value=backup_1622003358258 
wals:preprod-dn-1%2C16020%2C1614844389000.1621996160175 column=meta:file, 
timestamp=1622003479895, 
value=hdfs://store/hbase/oldWALs/preprod-dn-1%2C16020%2C1614844389000.1621996160175
 wals:preprod-dn-1%2C16020%2C1614844389000.1621996160175 column=meta:root, 
timestamp=1622003479895, value=s3a://2021-05-25--21-45-00--full/set1 
wals:preprod-dn-1%2C16020%2C1614844389000.1621999760280 column=meta:backupId, 
timestamp=1622003479895, value=backup_1622003358258 
wals:preprod-dn-1%2C16020%2C1614844389000.1621999760280 column=meta:file, 
timestamp=1622003479895, 
value=hdfs://store/hbase/oldWALs/preprod-dn-1%2C16020%2C1614844389000.1621999760280
 wals:preprod-dn-1%2C16020%2C1614844389000.1621999760280 column=meta:root, 
timestamp=1622003479895, value=s3a://2021-05-25--21-45-00--full/set1 
{code}
 

This has several problems
 # Ever growing rows of wal's sourced for incremental backup is maintained and 
never cleaned up. 
 # Unnecessary to have wal log listed for performing incremental backup or for 
performing logcleaner.
 # No support for rsgroup. Hence all rsgroups which doesn't have backup enabled 
tables, WAL's are retained forever.

 


> Remove dependence storing WAL filenames for backup
> --------------------------------------------------
>
>                 Key: HBASE-25891
>                 URL: https://issues.apache.org/jira/browse/HBASE-25891
>             Project: HBase
>          Issue Type: Improvement
>          Components: backup&restore
>    Affects Versions: 3.0.0-alpha-1
>            Reporter: Mallikarjun
>            Assignee: Mallikarjun
>            Priority: Major
>             Fix For: 3.0.0-alpha-1
>
>
> Currently WAL logs are stored in `backup:system` meta table 
> {code:java}
> // code placeholder
> wals:preprod-dn-1%2C16020%2C1614844389000.1621996160175 column=meta:backupId, 
> timestamp=1622003479895, value=backup_1622003358258 
> wals:preprod-dn-1%2C16020%2C1614844389000.1621996160175 column=meta:file, 
> timestamp=1622003479895, 
> value=hdfs://store/hbase/oldWALs/preprod-dn-1%2C16020%2C1614844389000.1621996160175
>  wals:preprod-dn-1%2C16020%2C1614844389000.1621996160175 column=meta:root, 
> timestamp=1622003479895, value=s3a://2021-05-25--21-45-00--full/set1 
> wals:preprod-dn-1%2C16020%2C1614844389000.1621999760280 column=meta:backupId, 
> timestamp=1622003479895, value=backup_1622003358258 
> wals:preprod-dn-1%2C16020%2C1614844389000.1621999760280 column=meta:file, 
> timestamp=1622003479895, 
> value=hdfs://store/hbase/oldWALs/preprod-dn-1%2C16020%2C1614844389000.1621999760280
>  wals:preprod-dn-1%2C16020%2C1614844389000.1621999760280 column=meta:root, 
> timestamp=1622003479895, value=s3a://2021-05-25--21-45-00--full/set1 
> {code}
>  
> This has several problems
>  # Ever growing rows of wal's sourced for incremental backup is maintained 
> and never cleaned up. 
>  # Unnecessary to have wal log listed for performing incremental backup or 
> for performing logcleaner.
>  # No support for rsgroup. Hence tables belonging to rsgroups which doesn't 
> have backup enabled also have to retain wals' and forever.
>  
> Proposed Solution:
>  
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Reply via email to