I'm very excited to see that this project is nearing completion. I've been following the development pretty closely and am very much looking forward to getting this merged to trunk.
One thing that I do think we should address before the merge is moving the programmatic APIs for working with snapshots. I've brought this up before, and was told that it would be done in a separate JIRA, but I don't think that JIRA was ever filed. As it stands right now, the API for using snapshots is the following: 1. The API to create/delete/rename snapshots are in FileSystem. 2. The API to mark directories as snapshottable or not only exists in DistributedFileSystem and DFSAdmin, neither of which are intended to be public APIs. In my opinion (and I think this was shared by others at the last snapshots design meetup?) we should move #1 out of the FileSystem class since these are primarily administrative APIs, and it is unlikely that any other FileSystem implementation besides HDFS will ever implement these commands. Also, #2 should really be in some public (not necessarily stable, but public) class for use by tools which are used to administer HDFS. In my opinion the most natural place for both of these APIs is in the HdfsAdmin class, which is a public/evolving interface explicitly for these sorts of operations. What are others thoughts on this subject? Best, Aaron -- Aaron T. Myers Software Engineer, Cloudera On Sat, Apr 13, 2013 at 10:05 AM, Suresh Srinivas <sur...@hortonworks.com>wrote: > Support for snapshots feature is being worked on in the jira > https://issues.apache.org/jira/browse/HDFS-2802. This is an important and > a > large feature in HDFS. Please see a brief presentation that describes the > feature at a highlevel from the Snapshot discussion meetup we had a while > back - > https://issues.apache.org/jira/secure/attachment/12552861/Snapshots.pdf. > > I am exicted to announce that the feature development will soon be > completed. Please see the jira for the design and the details of the > subtasks. This is a heads up about the merge vote mail that will soon be > sent. > > Details of development and testing: > Development has been done in a separate branch - > https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-2802. The > design is posted at - > > https://issues.apache.org/jira/secure/attachment/12551474/Snapshots20121030.pdf > . > The feature development has involved close to 100 subtasks and close to 20K > lines of code. > > A lot of unit tests have been added as a part of the feature. We also have > been testing this in a cluster of 5 nodes with a long running test that > mimics a real cluster usage with emphasis on use cases related to > snapshots. Please see the test plan > > https://issues.apache.org/jira/secure/attachment/12575442/snapshot-testplan.pdffor > the details. > > Next steps, before calling for merge vote, we need to get the following > done: > - Add user documentation that describes the feature, and how to use it > - Complete some of the pending tasks > - Continue testing the feature and fix any bugs that might come up > - Update the design document > > Thanks to everyone who has participated in design and development of this > feature. Please review the work and help in testing the feature. > > Regards, > Suresh >
