[ https://issues.apache.org/jira/browse/HBASE-7987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13996424#comment-13996424 ]
Nick Dimiduk commented on HBASE-7987: ------------------------------------- I guess the same concern applies to 0.96 and 0.94 as I'd like to support mapred.TableSnapshotInputFormat everywhere there's a mapreduce version. [~apurtell], [~stack], [~lhofhansl], [~enis]? > Snapshot Manifest file instead of multiple empty files > ------------------------------------------------------ > > Key: HBASE-7987 > URL: https://issues.apache.org/jira/browse/HBASE-7987 > Project: HBase > Issue Type: Improvement > Components: snapshots > Reporter: Matteo Bertozzi > Assignee: Matteo Bertozzi > Fix For: 0.99.0 > > Attachments: HBASE-7987-v0.patch, HBASE-7987-v1.patch, > HBASE-7987-v2.patch, HBASE-7987-v2.sketch, HBASE-7987-v3.patch, > HBASE-7987-v4.patch, HBASE-7987-v5.patch, HBASE-7987-v6.patch, > HBASE-7987.sketch > > > Currently taking a snapshot means creating one empty file for each file in > the source table directory, plus copying the .regioninfo file for each > region, the table descriptor file and a snapshotInfo file. > during the restore or snapshot verification we traverse the filesystem > (fs.listStatus()) to find the snapshot files, and we open the .regioninfo > files to get the information. > to avoid hammering the NameNode and having lots of empty files, we can use a > manifest file that contains the list of files and information that we need. > To keep the RS parallelism that we have, each RS can write its own manifest. > {code} > message SnapshotDescriptor { > required string name; > optional string table; > optional int64 creationTime; > optional Type type; > optional int32 version; > } > message SnapshotRegionManifest { > optional int32 version; > required RegionInfo regionInfo; > repeated FamilyFiles familyFiles; > message StoreFile { > required string name; > optional Reference reference; > } > message FamilyFiles { > required bytes familyName; > repeated StoreFile storeFiles; > } > } > {code} > {code} > /hbase/.snapshot/<snapshotName> > /hbase/.snapshot/<snapshotName>/snapshotInfo > /hbase/.snapshot/<snapshotName>/<tableName> > /hbase/.snapshot/<snapshotName>/<tableName>/tableInfo > /hbase/.snapshot/<snapshotName>/<tableName>/regionManifest(.n) > {code} -- This message was sent by Atlassian JIRA (v6.2#6252)