[ https://issues.apache.org/jira/browse/HBASE-5547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13265613#comment-13265613 ]
jirapos...@reviews.apache.org commented on HBASE-5547: ------------------------------------------------------ ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4633/ ----------------------------------------------------------- (Updated 2012-05-01 02:42:34.215505) Review request for hbase, Michael Stack and Lars Hofhansl. Changes ------- - TableTracker is now an abstract class that the catalogjanitor and rs are both using to register for tracking a table. - Adding monitoring in admin to check for cj to join archiving (before was missed) and including test - Slight cleanup from Ted's comments. Summary ------- Essentially, whenever an hfile would be deleted, it is instead moved to the archive directory. In this impl, the archive directory is on a per table basis, but defaults to '.archive'. Removing hfiles occurs in three places - compaction, merge and catalog janitor. The former and two latter are distinctly different code paths, but but did pull out some similarities. The latter two end up calling the same method, so there should be a reasonable amount of overlap. Implementation wise: Updated the HMasterInterface to pass the calls onto the zookeeper. Added a zk listener to handle updates from the master to the RS to backup. Added a utility for removing files and finding archive directories Added tests for the regionserver and catalogjanitor approaches. Added creation of manager in regionserver. This addresses bug HBASE-5547. https://issues.apache.org/jira/browse/HBASE-5547 Diffs (updated) ----- src/main/java/org/apache/hadoop/hbase/HConstants.java a9d80a0 src/main/java/org/apache/hadoop/hbase/backup/HFileArchiveMonitor.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/backup/HFileArchiveTableTracker.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/backup/HFileArchiveTracker.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/backup/ServerHFileTableArchiveTracker.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java ee16e72 src/main/java/org/apache/hadoop/hbase/client/HFileArchiveManager.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/master/CatalogJanitor.java 79d5fdd src/main/java/org/apache/hadoop/hbase/master/HMaster.java d47b83a src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java 7858846 src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 61a5988 src/main/java/org/apache/hadoop/hbase/regionserver/RegionServerServices.java 6884d53 src/main/java/org/apache/hadoop/hbase/regionserver/SplitTransaction.java ea12da4 src/main/java/org/apache/hadoop/hbase/regionserver/Store.java bf1618e src/main/java/org/apache/hadoop/hbase/util/HFileArchiveCleanup.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/util/HFileArchiveUtil.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperWatcher.java 4fc105f src/main/resources/hbase-default.xml f54b345 src/test/java/org/apache/hadoop/hbase/master/MockRegionServer.java a59e152 src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java cedf31e src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionHFileArchiving.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/util/HFileArchiveTestingUtil.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/util/MockRegionServerServices.java 7d02759 src/test/java/org/apache/hadoop/hbase/util/TestHFileArchivingCleanup.java PRE-CREATION Diff: https://reviews.apache.org/r/4633/diff Testing ------- Added two tests for the separate cases - archiving via the regionserver and for the catalog tracker. Former runs in a mini cluster and also touches the changes to HMasterInterface and zookeeper. Thanks, Jesse > Don't delete HFiles when in "backup mode" > ----------------------------------------- > > Key: HBASE-5547 > URL: https://issues.apache.org/jira/browse/HBASE-5547 > Project: HBase > Issue Type: New Feature > Reporter: Lars Hofhansl > Assignee: Jesse Yates > > This came up in a discussion I had with Stack. > It would be nice if HBase could be notified that a backup is in progress (via > a znode for example) and in that case either: > 1. rename HFiles to be delete to <file>.bck > 2. rename the HFiles into a special directory > 3. rename them to a general trash directory (which would not need to be tied > to backup mode). > That way it should be able to get a consistent backup based on HFiles (HDFS > snapshots or hard links would be better options here, but we do not have > those). > #1 makes cleanup a bit harder. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira