[ https://issues.apache.org/jira/browse/HBASE-7987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13968568#comment-13968568 ]
Jonathan Hsieh commented on HBASE-7987: --------------------------------------- post on rb please. > Snapshot Manifest file instead of multiple empty files > ------------------------------------------------------ > > Key: HBASE-7987 > URL: https://issues.apache.org/jira/browse/HBASE-7987 > Project: HBase > Issue Type: Improvement > Components: snapshots > Reporter: Matteo Bertozzi > Assignee: Matteo Bertozzi > Fix For: 1.0.0 > > Attachments: HBASE-7987-v0.patch, HBASE-7987-v1.patch, > HBASE-7987-v2.patch, HBASE-7987-v2.sketch, HBASE-7987.sketch > > > Currently taking a snapshot means creating one empty file for each file in > the source table directory, plus copying the .regioninfo file for each > region, the table descriptor file and a snapshotInfo file. > during the restore or snapshot verification we traverse the filesystem > (fs.listStatus()) to find the snapshot files, and we open the .regioninfo > files to get the information. > to avoid hammering the NameNode and having lots of empty files, we can use a > manifest file that contains the list of files and information that we need. > To keep the RS parallelism that we have, each RS can write its own manifest. > {code} > message SnapshotDescriptor { > required string name; > optional string table; > optional int64 creationTime; > optional Type type; > optional int32 version; > } > message SnapshotRegionManifest { > optional int32 version; > required RegionInfo regionInfo; > repeated FamilyFiles familyFiles; > message StoreFile { > required string name; > optional Reference reference; > } > message FamilyFiles { > required bytes familyName; > repeated StoreFile storeFiles; > } > } > {code} > {code} > /hbase/.snapshot/<snapshotName> > /hbase/.snapshot/<snapshotName>/snapshotInfo > /hbase/.snapshot/<snapshotName>/<tableName> > /hbase/.snapshot/<snapshotName>/<tableName>/tableInfo > /hbase/.snapshot/<snapshotName>/<tableName>/regionManifest(.n) > {code} -- This message was sent by Atlassian JIRA (v6.2#6252)