[jira] [Updated] (HDFS-2802) Support for RW/RO snapshots in HDFS
[ https://issues.apache.org/jira/browse/HDFS-2802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun C Murthy updated HDFS-2802: Fix Version/s: (was: 2.3.0) 2.4.0 > Support for RW/RO snapshots in HDFS > --- > > Key: HDFS-2802 > URL: https://issues.apache.org/jira/browse/HDFS-2802 > Project: Hadoop HDFS > Issue Type: New Feature > Components: namenode >Reporter: Hari Mankude >Assignee: Tsz Wo (Nicholas), SZE > Fix For: 2.4.0 > > Attachments: 2802.diff, 2802.patch, 2802.patch, > HDFS-2802-meeting-minutes-121101.txt, HDFS-2802.20121101.patch, > HDFS-2802.branch-2.0510.patch, HDFSSnapshotsDesign.pdf, Snapshots.pdf, > Snapshots20120429.pdf, Snapshots20121018.pdf, Snapshots20121030.pdf, > h2802_20130417.patch, h2802_20130422.patch, h2802_20130423.patch, > h2802_20130425.patch, h2802_20130426.patch, h2802_20130430.patch, > h2802_20130430b.patch, h2802_20130508.patch, h2802_20130509.patch, > snap.patch, snapshot-design.pdf, snapshot-design.tex, snapshot-one-pager.pdf, > snapshot-testplan.pdf > > > Snapshots are point in time images of parts of the filesystem or the entire > filesystem. Snapshots can be a read-only or a read-write point in time copy > of the filesystem. There are several use cases for snapshots in HDFS. I will > post a detailed write-up soon with with more information. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (HDFS-2802) Support for RW/RO snapshots in HDFS
[ https://issues.apache.org/jira/browse/HDFS-2802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun C Murthy updated HDFS-2802: Fix Version/s: (was: 2.1.0-beta) 2.3.0 > Support for RW/RO snapshots in HDFS > --- > > Key: HDFS-2802 > URL: https://issues.apache.org/jira/browse/HDFS-2802 > Project: Hadoop HDFS > Issue Type: New Feature > Components: namenode >Reporter: Hari Mankude >Assignee: Tsz Wo (Nicholas), SZE > Fix For: 2.3.0 > > Attachments: 2802.diff, 2802.patch, 2802.patch, h2802_20130417.patch, > h2802_20130422.patch, h2802_20130423.patch, h2802_20130425.patch, > h2802_20130426.patch, h2802_20130430b.patch, h2802_20130430.patch, > h2802_20130508.patch, h2802_20130509.patch, HDFS-2802.20121101.patch, > HDFS-2802.branch-2.0510.patch, HDFS-2802-meeting-minutes-121101.txt, > HDFSSnapshotsDesign.pdf, snap.patch, snapshot-design.pdf, > snapshot-design.tex, snapshot-one-pager.pdf, Snapshots20120429.pdf, > Snapshots20121018.pdf, Snapshots20121030.pdf, Snapshots.pdf, > snapshot-testplan.pdf > > > Snapshots are point in time images of parts of the filesystem or the entire > filesystem. Snapshots can be a read-only or a read-write point in time copy > of the filesystem. There are several use cases for snapshots in HDFS. I will > post a detailed write-up soon with with more information. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2802) Support for RW/RO snapshots in HDFS
[ https://issues.apache.org/jira/browse/HDFS-2802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo (Nicholas), SZE updated HDFS-2802: - Component/s: (was: datanode) Fix Version/s: 2.0.5-beta Merged this to branch-2. > Support for RW/RO snapshots in HDFS > --- > > Key: HDFS-2802 > URL: https://issues.apache.org/jira/browse/HDFS-2802 > Project: Hadoop HDFS > Issue Type: New Feature > Components: namenode >Reporter: Hari Mankude >Assignee: Tsz Wo (Nicholas), SZE > Fix For: 2.0.5-beta > > Attachments: 2802.diff, 2802.patch, 2802.patch, h2802_20130417.patch, > h2802_20130422.patch, h2802_20130423.patch, h2802_20130425.patch, > h2802_20130426.patch, h2802_20130430b.patch, h2802_20130430.patch, > h2802_20130508.patch, h2802_20130509.patch, HDFS-2802.20121101.patch, > HDFS-2802.branch-2.0510.patch, HDFS-2802-meeting-minutes-121101.txt, > HDFSSnapshotsDesign.pdf, snap.patch, snapshot-design.pdf, > snapshot-design.tex, snapshot-one-pager.pdf, Snapshots20120429.pdf, > Snapshots20121018.pdf, Snapshots20121030.pdf, Snapshots.pdf, > snapshot-testplan.pdf > > > Snapshots are point in time images of parts of the filesystem or the entire > filesystem. Snapshots can be a read-only or a read-write point in time copy > of the filesystem. There are several use cases for snapshots in HDFS. I will > post a detailed write-up soon with with more information. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2802) Support for RW/RO snapshots in HDFS
[ https://issues.apache.org/jira/browse/HDFS-2802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo (Nicholas), SZE updated HDFS-2802: - Status: Open (was: Patch Available) > Support for RW/RO snapshots in HDFS > --- > > Key: HDFS-2802 > URL: https://issues.apache.org/jira/browse/HDFS-2802 > Project: Hadoop HDFS > Issue Type: New Feature > Components: datanode, namenode >Reporter: Hari Mankude >Assignee: Tsz Wo (Nicholas), SZE > Attachments: 2802.diff, 2802.patch, 2802.patch, h2802_20130417.patch, > h2802_20130422.patch, h2802_20130423.patch, h2802_20130425.patch, > h2802_20130426.patch, h2802_20130430b.patch, h2802_20130430.patch, > h2802_20130508.patch, h2802_20130509.patch, HDFS-2802.20121101.patch, > HDFS-2802.branch-2.0510.patch, HDFS-2802-meeting-minutes-121101.txt, > HDFSSnapshotsDesign.pdf, snap.patch, snapshot-design.pdf, > snapshot-design.tex, snapshot-one-pager.pdf, Snapshots20120429.pdf, > Snapshots20121018.pdf, Snapshots20121030.pdf, Snapshots.pdf, > snapshot-testplan.pdf > > > Snapshots are point in time images of parts of the filesystem or the entire > filesystem. Snapshots can be a read-only or a read-write point in time copy > of the filesystem. There are several use cases for snapshots in HDFS. I will > post a detailed write-up soon with with more information. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2802) Support for RW/RO snapshots in HDFS
[ https://issues.apache.org/jira/browse/HDFS-2802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-2802: Attachment: HDFS-2802.branch-2.0510.patch Initial patch for backporting snapshot to branch-2. Will run all the unit/system tests and post the results later. > Support for RW/RO snapshots in HDFS > --- > > Key: HDFS-2802 > URL: https://issues.apache.org/jira/browse/HDFS-2802 > Project: Hadoop HDFS > Issue Type: New Feature > Components: datanode, namenode >Reporter: Hari Mankude >Assignee: Tsz Wo (Nicholas), SZE > Attachments: 2802.diff, 2802.patch, 2802.patch, h2802_20130417.patch, > h2802_20130422.patch, h2802_20130423.patch, h2802_20130425.patch, > h2802_20130426.patch, h2802_20130430b.patch, h2802_20130430.patch, > h2802_20130508.patch, h2802_20130509.patch, HDFS-2802.20121101.patch, > HDFS-2802.branch-2.0510.patch, HDFS-2802-meeting-minutes-121101.txt, > HDFSSnapshotsDesign.pdf, snap.patch, snapshot-design.pdf, > snapshot-design.tex, snapshot-one-pager.pdf, Snapshots20120429.pdf, > Snapshots20121018.pdf, Snapshots20121030.pdf, Snapshots.pdf, > snapshot-testplan.pdf > > > Snapshots are point in time images of parts of the filesystem or the entire > filesystem. Snapshots can be a read-only or a read-write point in time copy > of the filesystem. There are several use cases for snapshots in HDFS. I will > post a detailed write-up soon with with more information. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2802) Support for RW/RO snapshots in HDFS
[ https://issues.apache.org/jira/browse/HDFS-2802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo (Nicholas), SZE updated HDFS-2802: - Attachment: h2802_20130509.patch h2802_20130509.patch > Support for RW/RO snapshots in HDFS > --- > > Key: HDFS-2802 > URL: https://issues.apache.org/jira/browse/HDFS-2802 > Project: Hadoop HDFS > Issue Type: New Feature > Components: datanode, namenode >Reporter: Hari Mankude >Assignee: Tsz Wo (Nicholas), SZE > Attachments: 2802.diff, 2802.patch, 2802.patch, h2802_20130417.patch, > h2802_20130422.patch, h2802_20130423.patch, h2802_20130425.patch, > h2802_20130426.patch, h2802_20130430b.patch, h2802_20130430.patch, > h2802_20130508.patch, h2802_20130509.patch, HDFS-2802.20121101.patch, > HDFS-2802-meeting-minutes-121101.txt, HDFSSnapshotsDesign.pdf, snap.patch, > snapshot-design.pdf, snapshot-design.tex, snapshot-one-pager.pdf, > Snapshots20120429.pdf, Snapshots20121018.pdf, Snapshots20121030.pdf, > Snapshots.pdf, snapshot-testplan.pdf > > > Snapshots are point in time images of parts of the filesystem or the entire > filesystem. Snapshots can be a read-only or a read-write point in time copy > of the filesystem. There are several use cases for snapshots in HDFS. I will > post a detailed write-up soon with with more information. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2802) Support for RW/RO snapshots in HDFS
[ https://issues.apache.org/jira/browse/HDFS-2802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo (Nicholas), SZE updated HDFS-2802: - Attachment: h2802_20130508.patch h2802_20130508.patch > Support for RW/RO snapshots in HDFS > --- > > Key: HDFS-2802 > URL: https://issues.apache.org/jira/browse/HDFS-2802 > Project: Hadoop HDFS > Issue Type: New Feature > Components: datanode, namenode >Reporter: Hari Mankude >Assignee: Tsz Wo (Nicholas), SZE > Attachments: 2802.diff, 2802.patch, 2802.patch, h2802_20130417.patch, > h2802_20130422.patch, h2802_20130423.patch, h2802_20130425.patch, > h2802_20130426.patch, h2802_20130430b.patch, h2802_20130430.patch, > h2802_20130508.patch, HDFS-2802.20121101.patch, > HDFS-2802-meeting-minutes-121101.txt, HDFSSnapshotsDesign.pdf, snap.patch, > snapshot-design.pdf, snapshot-design.tex, snapshot-one-pager.pdf, > Snapshots20120429.pdf, Snapshots20121018.pdf, Snapshots20121030.pdf, > Snapshots.pdf, snapshot-testplan.pdf > > > Snapshots are point in time images of parts of the filesystem or the entire > filesystem. Snapshots can be a read-only or a read-write point in time copy > of the filesystem. There are several use cases for snapshots in HDFS. I will > post a detailed write-up soon with with more information. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2802) Support for RW/RO snapshots in HDFS
[ https://issues.apache.org/jira/browse/HDFS-2802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HDFS-2802: -- Attachment: Snapshots20120429.pdf Updated design document > Support for RW/RO snapshots in HDFS > --- > > Key: HDFS-2802 > URL: https://issues.apache.org/jira/browse/HDFS-2802 > Project: Hadoop HDFS > Issue Type: New Feature > Components: datanode, namenode >Reporter: Hari Mankude >Assignee: Tsz Wo (Nicholas), SZE > Attachments: 2802.diff, 2802.patch, 2802.patch, h2802_20130417.patch, > h2802_20130422.patch, h2802_20130423.patch, h2802_20130425.patch, > h2802_20130426.patch, h2802_20130430b.patch, h2802_20130430.patch, > HDFS-2802.20121101.patch, HDFS-2802-meeting-minutes-121101.txt, > HDFSSnapshotsDesign.pdf, snap.patch, snapshot-design.pdf, > snapshot-design.tex, snapshot-one-pager.pdf, Snapshots20120429.pdf, > Snapshots20121018.pdf, Snapshots20121030.pdf, Snapshots.pdf, > snapshot-testplan.pdf > > > Snapshots are point in time images of parts of the filesystem or the entire > filesystem. Snapshots can be a read-only or a read-write point in time copy > of the filesystem. There are several use cases for snapshots in HDFS. I will > post a detailed write-up soon with with more information. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2802) Support for RW/RO snapshots in HDFS
[ https://issues.apache.org/jira/browse/HDFS-2802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HDFS-2802: -- Attachment: (was: Snapshots20120429.pdf) > Support for RW/RO snapshots in HDFS > --- > > Key: HDFS-2802 > URL: https://issues.apache.org/jira/browse/HDFS-2802 > Project: Hadoop HDFS > Issue Type: New Feature > Components: datanode, namenode >Reporter: Hari Mankude >Assignee: Tsz Wo (Nicholas), SZE > Attachments: 2802.diff, 2802.patch, 2802.patch, h2802_20130417.patch, > h2802_20130422.patch, h2802_20130423.patch, h2802_20130425.patch, > h2802_20130426.patch, h2802_20130430b.patch, h2802_20130430.patch, > HDFS-2802.20121101.patch, HDFS-2802-meeting-minutes-121101.txt, > HDFSSnapshotsDesign.pdf, snap.patch, snapshot-design.pdf, > snapshot-design.tex, snapshot-one-pager.pdf, Snapshots20120429.pdf, > Snapshots20121018.pdf, Snapshots20121030.pdf, Snapshots.pdf, > snapshot-testplan.pdf > > > Snapshots are point in time images of parts of the filesystem or the entire > filesystem. Snapshots can be a read-only or a read-write point in time copy > of the filesystem. There are several use cases for snapshots in HDFS. I will > post a detailed write-up soon with with more information. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2802) Support for RW/RO snapshots in HDFS
[ https://issues.apache.org/jira/browse/HDFS-2802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo (Nicholas), SZE updated HDFS-2802: - Attachment: h2802_20130430b.patch h2802_20130430b.patch: with the HDFS-4781 patch (h4781_20130430.patch) > Support for RW/RO snapshots in HDFS > --- > > Key: HDFS-2802 > URL: https://issues.apache.org/jira/browse/HDFS-2802 > Project: Hadoop HDFS > Issue Type: New Feature > Components: datanode, namenode >Reporter: Hari Mankude >Assignee: Tsz Wo (Nicholas), SZE > Attachments: 2802.diff, 2802.patch, 2802.patch, h2802_20130417.patch, > h2802_20130422.patch, h2802_20130423.patch, h2802_20130425.patch, > h2802_20130426.patch, h2802_20130430b.patch, h2802_20130430.patch, > HDFS-2802.20121101.patch, HDFS-2802-meeting-minutes-121101.txt, > HDFSSnapshotsDesign.pdf, snap.patch, snapshot-design.pdf, > snapshot-design.tex, snapshot-one-pager.pdf, Snapshots20120429.pdf, > Snapshots20121018.pdf, Snapshots20121030.pdf, Snapshots.pdf, > snapshot-testplan.pdf > > > Snapshots are point in time images of parts of the filesystem or the entire > filesystem. Snapshots can be a read-only or a read-write point in time copy > of the filesystem. There are several use cases for snapshots in HDFS. I will > post a detailed write-up soon with with more information. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2802) Support for RW/RO snapshots in HDFS
[ https://issues.apache.org/jira/browse/HDFS-2802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo (Nicholas), SZE updated HDFS-2802: - Attachment: h2802_20130430.patch h2802_20130430.patch > Support for RW/RO snapshots in HDFS > --- > > Key: HDFS-2802 > URL: https://issues.apache.org/jira/browse/HDFS-2802 > Project: Hadoop HDFS > Issue Type: New Feature > Components: datanode, namenode >Reporter: Hari Mankude >Assignee: Tsz Wo (Nicholas), SZE > Attachments: 2802.diff, 2802.patch, 2802.patch, h2802_20130417.patch, > h2802_20130422.patch, h2802_20130423.patch, h2802_20130425.patch, > h2802_20130426.patch, h2802_20130430.patch, HDFS-2802.20121101.patch, > HDFS-2802-meeting-minutes-121101.txt, HDFSSnapshotsDesign.pdf, snap.patch, > snapshot-design.pdf, snapshot-design.tex, snapshot-one-pager.pdf, > Snapshots20120429.pdf, Snapshots20121018.pdf, Snapshots20121030.pdf, > Snapshots.pdf, snapshot-testplan.pdf > > > Snapshots are point in time images of parts of the filesystem or the entire > filesystem. Snapshots can be a read-only or a read-write point in time copy > of the filesystem. There are several use cases for snapshots in HDFS. I will > post a detailed write-up soon with with more information. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2802) Support for RW/RO snapshots in HDFS
[ https://issues.apache.org/jira/browse/HDFS-2802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HDFS-2802: -- Attachment: Snapshots20120429.pdf Updated design document with the following changes: # Added a section “Phase 1 feature set” to describe the feature set implemented in phase 1 # Removed configuration “dfs.namenode.home.snapshots.allowed” from design. # Updated the design to indicate rename is allowed within and across snapshottable directory boundaries # Added how quota applies to snapshots # Added that permission and other attributes for a file/directory in snapshot cannot be changed. # Updated CLI list # Added information about rename design and implementation > Support for RW/RO snapshots in HDFS > --- > > Key: HDFS-2802 > URL: https://issues.apache.org/jira/browse/HDFS-2802 > Project: Hadoop HDFS > Issue Type: New Feature > Components: datanode, namenode >Reporter: Hari Mankude >Assignee: Tsz Wo (Nicholas), SZE > Attachments: 2802.diff, 2802.patch, 2802.patch, h2802_20130417.patch, > h2802_20130422.patch, h2802_20130423.patch, h2802_20130425.patch, > h2802_20130426.patch, HDFS-2802.20121101.patch, > HDFS-2802-meeting-minutes-121101.txt, HDFSSnapshotsDesign.pdf, snap.patch, > snapshot-design.pdf, snapshot-design.tex, snapshot-one-pager.pdf, > Snapshots20120429.pdf, Snapshots20121018.pdf, Snapshots20121030.pdf, > Snapshots.pdf, snapshot-testplan.pdf > > > Snapshots are point in time images of parts of the filesystem or the entire > filesystem. Snapshots can be a read-only or a read-write point in time copy > of the filesystem. There are several use cases for snapshots in HDFS. I will > post a detailed write-up soon with with more information. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2802) Support for RW/RO snapshots in HDFS
[ https://issues.apache.org/jira/browse/HDFS-2802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo (Nicholas), SZE updated HDFS-2802: - Attachment: h2802_20130426.patch h2802_20130426.patch > Support for RW/RO snapshots in HDFS > --- > > Key: HDFS-2802 > URL: https://issues.apache.org/jira/browse/HDFS-2802 > Project: Hadoop HDFS > Issue Type: New Feature > Components: datanode, namenode >Reporter: Hari Mankude >Assignee: Tsz Wo (Nicholas), SZE > Attachments: 2802.diff, 2802.patch, 2802.patch, h2802_20130417.patch, > h2802_20130422.patch, h2802_20130423.patch, h2802_20130425.patch, > h2802_20130426.patch, HDFS-2802.20121101.patch, > HDFS-2802-meeting-minutes-121101.txt, HDFSSnapshotsDesign.pdf, snap.patch, > snapshot-design.pdf, snapshot-design.tex, snapshot-one-pager.pdf, > Snapshots20121018.pdf, Snapshots20121030.pdf, Snapshots.pdf, > snapshot-testplan.pdf > > > Snapshots are point in time images of parts of the filesystem or the entire > filesystem. Snapshots can be a read-only or a read-write point in time copy > of the filesystem. There are several use cases for snapshots in HDFS. I will > post a detailed write-up soon with with more information. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2802) Support for RW/RO snapshots in HDFS
[ https://issues.apache.org/jira/browse/HDFS-2802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo (Nicholas), SZE updated HDFS-2802: - Attachment: h2802_20130425.patch h2802_20130425.patch > Support for RW/RO snapshots in HDFS > --- > > Key: HDFS-2802 > URL: https://issues.apache.org/jira/browse/HDFS-2802 > Project: Hadoop HDFS > Issue Type: New Feature > Components: datanode, namenode >Reporter: Hari Mankude >Assignee: Tsz Wo (Nicholas), SZE > Attachments: 2802.diff, 2802.patch, 2802.patch, h2802_20130417.patch, > h2802_20130422.patch, h2802_20130423.patch, h2802_20130425.patch, > HDFS-2802.20121101.patch, HDFS-2802-meeting-minutes-121101.txt, > HDFSSnapshotsDesign.pdf, snap.patch, snapshot-design.pdf, > snapshot-design.tex, snapshot-one-pager.pdf, Snapshots20121018.pdf, > Snapshots20121030.pdf, Snapshots.pdf, snapshot-testplan.pdf > > > Snapshots are point in time images of parts of the filesystem or the entire > filesystem. Snapshots can be a read-only or a read-write point in time copy > of the filesystem. There are several use cases for snapshots in HDFS. I will > post a detailed write-up soon with with more information. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2802) Support for RW/RO snapshots in HDFS
[ https://issues.apache.org/jira/browse/HDFS-2802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo (Nicholas), SZE updated HDFS-2802: - Attachment: h2802_20130423.patch h2802_20130423.patch: fixes javadoc and some other warnings. > Support for RW/RO snapshots in HDFS > --- > > Key: HDFS-2802 > URL: https://issues.apache.org/jira/browse/HDFS-2802 > Project: Hadoop HDFS > Issue Type: New Feature > Components: datanode, namenode >Reporter: Hari Mankude >Assignee: Tsz Wo (Nicholas), SZE > Attachments: 2802.diff, 2802.patch, 2802.patch, h2802_20130417.patch, > h2802_20130422.patch, h2802_20130423.patch, HDFS-2802.20121101.patch, > HDFS-2802-meeting-minutes-121101.txt, HDFSSnapshotsDesign.pdf, snap.patch, > snapshot-design.pdf, snapshot-design.tex, snapshot-one-pager.pdf, > Snapshots20121018.pdf, Snapshots20121030.pdf, Snapshots.pdf, > snapshot-testplan.pdf > > > Snapshots are point in time images of parts of the filesystem or the entire > filesystem. Snapshots can be a read-only or a read-write point in time copy > of the filesystem. There are several use cases for snapshots in HDFS. I will > post a detailed write-up soon with with more information. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2802) Support for RW/RO snapshots in HDFS
[ https://issues.apache.org/jira/browse/HDFS-2802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo (Nicholas), SZE updated HDFS-2802: - Attachment: h2802_20130422.patch h2802_20130422.patch: merged with inode id map. > Support for RW/RO snapshots in HDFS > --- > > Key: HDFS-2802 > URL: https://issues.apache.org/jira/browse/HDFS-2802 > Project: Hadoop HDFS > Issue Type: New Feature > Components: datanode, namenode >Reporter: Hari Mankude >Assignee: Tsz Wo (Nicholas), SZE > Attachments: 2802.diff, 2802.patch, 2802.patch, h2802_20130417.patch, > h2802_20130422.patch, HDFS-2802.20121101.patch, > HDFS-2802-meeting-minutes-121101.txt, HDFSSnapshotsDesign.pdf, snap.patch, > snapshot-design.pdf, snapshot-design.tex, snapshot-one-pager.pdf, > Snapshots20121018.pdf, Snapshots20121030.pdf, Snapshots.pdf, > snapshot-testplan.pdf > > > Snapshots are point in time images of parts of the filesystem or the entire > filesystem. Snapshots can be a read-only or a read-write point in time copy > of the filesystem. There are several use cases for snapshots in HDFS. I will > post a detailed write-up soon with with more information. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2802) Support for RW/RO snapshots in HDFS
[ https://issues.apache.org/jira/browse/HDFS-2802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo (Nicholas), SZE updated HDFS-2802: - Attachment: h2802_20130417.patch h2802_20130417.patch: from snapshot branch > Support for RW/RO snapshots in HDFS > --- > > Key: HDFS-2802 > URL: https://issues.apache.org/jira/browse/HDFS-2802 > Project: Hadoop HDFS > Issue Type: New Feature > Components: datanode, namenode >Reporter: Hari Mankude >Assignee: Tsz Wo (Nicholas), SZE > Attachments: 2802.diff, 2802.patch, 2802.patch, h2802_20130417.patch, > HDFS-2802.20121101.patch, HDFS-2802-meeting-minutes-121101.txt, > HDFSSnapshotsDesign.pdf, snap.patch, snapshot-design.pdf, > snapshot-design.tex, snapshot-one-pager.pdf, Snapshots20121018.pdf, > Snapshots20121030.pdf, Snapshots.pdf, snapshot-testplan.pdf > > > Snapshots are point in time images of parts of the filesystem or the entire > filesystem. Snapshots can be a read-only or a read-write point in time copy > of the filesystem. There are several use cases for snapshots in HDFS. I will > post a detailed write-up soon with with more information. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2802) Support for RW/RO snapshots in HDFS
[ https://issues.apache.org/jira/browse/HDFS-2802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-2802: Attachment: snapshot-testplan.pdf Upload the current test plan. We followed this test plan and tested the existing snapshot functionalities (except rename and concat) in a small cluster with 5 nodes for a week. The tests we have done include 1) the basic operation test, 2) the long-running test, and 3) the stress test described in the test plan. Currently the tests run well and the snapshot functionality seems stable. Several bugs have been caught and we have filed HDFS-4616, HDFS-4627, HDFS-4636, and HDFS-4637 for fix. We will continue running the tests and fixing bugs. > Support for RW/RO snapshots in HDFS > --- > > Key: HDFS-2802 > URL: https://issues.apache.org/jira/browse/HDFS-2802 > Project: Hadoop HDFS > Issue Type: New Feature > Components: datanode, namenode >Reporter: Hari Mankude >Assignee: Tsz Wo (Nicholas), SZE > Attachments: 2802.diff, 2802.patch, 2802.patch, > HDFS-2802.20121101.patch, HDFS-2802-meeting-minutes-121101.txt, > HDFSSnapshotsDesign.pdf, snap.patch, snapshot-design.pdf, > snapshot-design.tex, snapshot-one-pager.pdf, Snapshots20121018.pdf, > Snapshots20121030.pdf, Snapshots.pdf, snapshot-testplan.pdf > > > Snapshots are point in time images of parts of the filesystem or the entire > filesystem. Snapshots can be a read-only or a read-write point in time copy > of the filesystem. There are several use cases for snapshots in HDFS. I will > post a detailed write-up soon with with more information. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2802) Support for RW/RO snapshots in HDFS
[ https://issues.apache.org/jira/browse/HDFS-2802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eli Collins updated HDFS-2802: -- Target Version/s: 2.0.5-beta (was: 3.0.0) > Support for RW/RO snapshots in HDFS > --- > > Key: HDFS-2802 > URL: https://issues.apache.org/jira/browse/HDFS-2802 > Project: Hadoop HDFS > Issue Type: New Feature > Components: datanode, namenode >Reporter: Hari Mankude >Assignee: Tsz Wo (Nicholas), SZE > Attachments: 2802.diff, 2802.patch, 2802.patch, > HDFS-2802.20121101.patch, HDFS-2802-meeting-minutes-121101.txt, > HDFSSnapshotsDesign.pdf, snap.patch, snapshot-design.pdf, > snapshot-design.tex, snapshot-one-pager.pdf, Snapshots20121018.pdf, > Snapshots20121030.pdf, Snapshots.pdf > > > Snapshots are point in time images of parts of the filesystem or the entire > filesystem. Snapshots can be a read-only or a read-write point in time copy > of the filesystem. There are several use cases for snapshots in HDFS. I will > post a detailed write-up soon with with more information. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2802) Support for RW/RO snapshots in HDFS
[ https://issues.apache.org/jira/browse/HDFS-2802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HDFS-2802: -- Attachment: 2802.diff Latest diff patch attached. > Support for RW/RO snapshots in HDFS > --- > > Key: HDFS-2802 > URL: https://issues.apache.org/jira/browse/HDFS-2802 > Project: Hadoop HDFS > Issue Type: New Feature > Components: datanode, namenode >Reporter: Hari Mankude >Assignee: Tsz Wo (Nicholas), SZE > Attachments: 2802.diff, 2802.patch, 2802.patch, > HDFS-2802.20121101.patch, HDFS-2802-meeting-minutes-121101.txt, > HDFSSnapshotsDesign.pdf, snap.patch, snapshot-design.pdf, > snapshot-design.tex, snapshot-one-pager.pdf, Snapshots20121018.pdf, > Snapshots20121030.pdf, Snapshots.pdf > > > Snapshots are point in time images of parts of the filesystem or the entire > filesystem. Snapshots can be a read-only or a read-write point in time copy > of the filesystem. There are several use cases for snapshots in HDFS. I will > post a detailed write-up soon with with more information. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2802) Support for RW/RO snapshots in HDFS
[ https://issues.apache.org/jira/browse/HDFS-2802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo (Nicholas), SZE updated HDFS-2802: - Attachment: 2802.patch > Support for RW/RO snapshots in HDFS > --- > > Key: HDFS-2802 > URL: https://issues.apache.org/jira/browse/HDFS-2802 > Project: Hadoop HDFS > Issue Type: New Feature > Components: datanode, namenode >Reporter: Hari Mankude >Assignee: Tsz Wo (Nicholas), SZE > Attachments: 2802.patch, 2802.patch, HDFS-2802.20121101.patch, > HDFS-2802-meeting-minutes-121101.txt, HDFSSnapshotsDesign.pdf, snap.patch, > snapshot-design.pdf, snapshot-design.tex, snapshot-one-pager.pdf, > Snapshots20121018.pdf, Snapshots20121030.pdf, Snapshots.pdf > > > Snapshots are point in time images of parts of the filesystem or the entire > filesystem. Snapshots can be a read-only or a read-write point in time copy > of the filesystem. There are several use cases for snapshots in HDFS. I will > post a detailed write-up soon with with more information. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2802) Support for RW/RO snapshots in HDFS
[ https://issues.apache.org/jira/browse/HDFS-2802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HDFS-2802: -- Status: Patch Available (was: Open) > Support for RW/RO snapshots in HDFS > --- > > Key: HDFS-2802 > URL: https://issues.apache.org/jira/browse/HDFS-2802 > Project: Hadoop HDFS > Issue Type: New Feature > Components: datanode, namenode >Reporter: Hari Mankude >Assignee: Tsz Wo (Nicholas), SZE > Attachments: 2802.patch, HDFS-2802.20121101.patch, > HDFS-2802-meeting-minutes-121101.txt, HDFSSnapshotsDesign.pdf, snap.patch, > snapshot-design.pdf, snapshot-design.tex, snapshot-one-pager.pdf, > Snapshots20121018.pdf, Snapshots20121030.pdf, Snapshots.pdf > > > Snapshots are point in time images of parts of the filesystem or the entire > filesystem. Snapshots can be a read-only or a read-write point in time copy > of the filesystem. There are several use cases for snapshots in HDFS. I will > post a detailed write-up soon with with more information. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2802) Support for RW/RO snapshots in HDFS
[ https://issues.apache.org/jira/browse/HDFS-2802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HDFS-2802: -- Attachment: 2802.patch Early version of the merge patch. > Support for RW/RO snapshots in HDFS > --- > > Key: HDFS-2802 > URL: https://issues.apache.org/jira/browse/HDFS-2802 > Project: Hadoop HDFS > Issue Type: New Feature > Components: datanode, namenode >Reporter: Hari Mankude >Assignee: Tsz Wo (Nicholas), SZE > Attachments: 2802.patch, HDFS-2802.20121101.patch, > HDFS-2802-meeting-minutes-121101.txt, HDFSSnapshotsDesign.pdf, snap.patch, > snapshot-design.pdf, snapshot-design.tex, snapshot-one-pager.pdf, > Snapshots20121018.pdf, Snapshots20121030.pdf, Snapshots.pdf > > > Snapshots are point in time images of parts of the filesystem or the entire > filesystem. Snapshots can be a read-only or a read-write point in time copy > of the filesystem. There are several use cases for snapshots in HDFS. I will > post a detailed write-up soon with with more information. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2802) Support for RW/RO snapshots in HDFS
[ https://issues.apache.org/jira/browse/HDFS-2802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HDFS-2802: -- Assignee: Tsz Wo (Nicholas), SZE (was: Hari Mankude) > Support for RW/RO snapshots in HDFS > --- > > Key: HDFS-2802 > URL: https://issues.apache.org/jira/browse/HDFS-2802 > Project: Hadoop HDFS > Issue Type: New Feature > Components: datanode, namenode >Reporter: Hari Mankude >Assignee: Tsz Wo (Nicholas), SZE > Attachments: HDFS-2802.20121101.patch, > HDFS-2802-meeting-minutes-121101.txt, HDFSSnapshotsDesign.pdf, snap.patch, > snapshot-design.pdf, snapshot-design.tex, snapshot-one-pager.pdf, > Snapshots20121018.pdf, Snapshots20121030.pdf, Snapshots.pdf > > > Snapshots are point in time images of parts of the filesystem or the entire > filesystem. Snapshots can be a read-only or a read-write point in time copy > of the filesystem. There are several use cases for snapshots in HDFS. I will > post a detailed write-up soon with with more information. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2802) Support for RW/RO snapshots in HDFS
[ https://issues.apache.org/jira/browse/HDFS-2802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HDFS-2802: -- Attachment: Snapshots.pdf Slides from yesterday's meetup. > Support for RW/RO snapshots in HDFS > --- > > Key: HDFS-2802 > URL: https://issues.apache.org/jira/browse/HDFS-2802 > Project: Hadoop HDFS > Issue Type: New Feature > Components: data-node, name-node >Reporter: Hari Mankude >Assignee: Hari Mankude > Attachments: HDFS-2802.20121101.patch, > HDFS-2802-meeting-minutes-121101.txt, HDFSSnapshotsDesign.pdf, snap.patch, > snapshot-design.pdf, snapshot-design.tex, snapshot-one-pager.pdf, > Snapshots20121018.pdf, Snapshots20121030.pdf, Snapshots.pdf > > > Snapshots are point in time images of parts of the filesystem or the entire > filesystem. Snapshots can be a read-only or a read-write point in time copy > of the filesystem. There are several use cases for snapshots in HDFS. I will > post a detailed write-up soon with with more information. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2802) Support for RW/RO snapshots in HDFS
[ https://issues.apache.org/jira/browse/HDFS-2802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David S. Wang updated HDFS-2802: Attachment: HDFS-2802-meeting-minutes-121101.txt Minutes from meeting on November 1, 2012. > Support for RW/RO snapshots in HDFS > --- > > Key: HDFS-2802 > URL: https://issues.apache.org/jira/browse/HDFS-2802 > Project: Hadoop HDFS > Issue Type: New Feature > Components: data-node, name-node >Reporter: Hari Mankude >Assignee: Hari Mankude > Attachments: HDFS-2802.20121101.patch, > HDFS-2802-meeting-minutes-121101.txt, HDFSSnapshotsDesign.pdf, snap.patch, > snapshot-design.pdf, snapshot-design.tex, snapshot-one-pager.pdf, > Snapshots20121018.pdf, Snapshots20121030.pdf > > > Snapshots are point in time images of parts of the filesystem or the entire > filesystem. Snapshots can be a read-only or a read-write point in time copy > of the filesystem. There are several use cases for snapshots in HDFS. I will > post a detailed write-up soon with with more information. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2802) Support for RW/RO snapshots in HDFS
[ https://issues.apache.org/jira/browse/HDFS-2802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HDFS-2802: -- Attachment: HDFS-2802.20121101.patch Consolidated patch with early snapshot implementation and tests. > Support for RW/RO snapshots in HDFS > --- > > Key: HDFS-2802 > URL: https://issues.apache.org/jira/browse/HDFS-2802 > Project: Hadoop HDFS > Issue Type: New Feature > Components: data-node, name-node >Reporter: Hari Mankude >Assignee: Hari Mankude > Attachments: HDFS-2802.20121101.patch, HDFSSnapshotsDesign.pdf, > snap.patch, snapshot-design.pdf, snapshot-design.tex, snapshot-one-pager.pdf, > Snapshots20121018.pdf, Snapshots20121030.pdf > > > Snapshots are point in time images of parts of the filesystem or the entire > filesystem. Snapshots can be a read-only or a read-write point in time copy > of the filesystem. There are several use cases for snapshots in HDFS. I will > post a detailed write-up soon with with more information. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2802) Support for RW/RO snapshots in HDFS
[ https://issues.apache.org/jira/browse/HDFS-2802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron T. Myers updated HDFS-2802: - Attachment: snapshot-design.tex snapshot-design.pdf Hello all, attached please find an updated design proposal which attempts to merge the two designs as I previously described in [this comment|https://issues.apache.org/jira/browse/HDFS-2802?focusedCommentId=13485761&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13485761]. This hybrid design merges the concepts of a snapshottable directory and user-initiated snapshots from the designs posted by Nicholas and Suresh, with the NN snapshot representation and client materialization of snapshots from the earlier design I posted. This document also expands upon the CLI commands/API that would be supported, as well as provide more details on how users would be able to access the snapshots available to them. Please have a look and review. I'd appreciate any feedback you have. I'm also uploading the .tex file used to produce this document and have removed the author listing. I hope this facilitates collaborating on a single design that we can eventually all agree to. > Support for RW/RO snapshots in HDFS > --- > > Key: HDFS-2802 > URL: https://issues.apache.org/jira/browse/HDFS-2802 > Project: Hadoop HDFS > Issue Type: New Feature > Components: data-node, name-node >Reporter: Hari Mankude >Assignee: Hari Mankude > Attachments: HDFSSnapshotsDesign.pdf, snap.patch, > snapshot-design.pdf, snapshot-design.tex, snapshot-one-pager.pdf, > Snapshots20121018.pdf, Snapshots20121030.pdf > > > Snapshots are point in time images of parts of the filesystem or the entire > filesystem. Snapshots can be a read-only or a read-write point in time copy > of the filesystem. There are several use cases for snapshots in HDFS. I will > post a detailed write-up soon with with more information. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2802) Support for RW/RO snapshots in HDFS
[ https://issues.apache.org/jira/browse/HDFS-2802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HDFS-2802: -- Attachment: Snapshots20121030.pdf Attaching the updated design document. Hopefully it addresses the issues that have been rasied. This should give sufficient details about the implementation we are currently working on. To summarize: # *Snapshot allowed only at the root vs snapshot at the subdirectories* - Ability to snapshot a sub-directory is very important requirement for many Hadoop users. Please see the requirements in the document posted for more details. The alternate proposal to allow snapshots only at the root is a non-starter in this regard. # *Efficiency of snapshot creation and management* - The current design addresses the concerns raised. To summarize the creation of snapshot is O(1). The design uses copy-on-modify approach so that the cost of snapshot is zero when there is no modification and is proportional the modifications when they are made. Please provide feedback. # *Snapshot of being written files and consistency* - Our document describes several design choices - some easy, some complicated. Please see the proposed choice in the document. We could continue this discussion in HDFS-3960. I took a look at the alternate proposal. It is too high level without sufficient details to evaluate. From my limited understanding of the alternate proposal, the design document we have posted here has several significant advantages over it: # It supports sub-directory snapshots, an important use case for many Hadoop users. # It supports on-demand and user managed snapshots. # When snapshots are not created, there is no cost incurred in terms of memory. The alternate proposal has O(N) memory cost for storing tags. # Our design can also be extended to do RW snapshots, if we feel a need for it. Hopefully this clarifies the design better. We would like continue focus on implementing it. Any feedback provided will be incorporated into the design and the implementation. > Support for RW/RO snapshots in HDFS > --- > > Key: HDFS-2802 > URL: https://issues.apache.org/jira/browse/HDFS-2802 > Project: Hadoop HDFS > Issue Type: New Feature > Components: data-node, name-node >Reporter: Hari Mankude >Assignee: Hari Mankude > Attachments: HDFSSnapshotsDesign.pdf, snap.patch, > snapshot-one-pager.pdf, Snapshots20121018.pdf, Snapshots20121030.pdf > > > Snapshots are point in time images of parts of the filesystem or the entire > filesystem. Snapshots can be a read-only or a read-write point in time copy > of the filesystem. There are several use cases for snapshots in HDFS. I will > post a detailed write-up soon with with more information. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2802) Support for RW/RO snapshots in HDFS
[ https://issues.apache.org/jira/browse/HDFS-2802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eli Collins updated HDFS-2802: -- Target Version/s: 3.0.0 Setting the target version to 3.0 since this will require a file system upgrade due to metadata changes (eg HDFS-4126, HDFS-4086, and HDFS-4082) and we're heading to GA soon. > Support for RW/RO snapshots in HDFS > --- > > Key: HDFS-2802 > URL: https://issues.apache.org/jira/browse/HDFS-2802 > Project: Hadoop HDFS > Issue Type: New Feature > Components: data-node, name-node >Reporter: Hari Mankude >Assignee: Hari Mankude > Attachments: HDFSSnapshotsDesign.pdf, snap.patch, > snapshot-one-pager.pdf, Snapshots20121018.pdf > > > Snapshots are point in time images of parts of the filesystem or the entire > filesystem. Snapshots can be a read-only or a read-write point in time copy > of the filesystem. There are several use cases for snapshots in HDFS. I will > post a detailed write-up soon with with more information. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2802) Support for RW/RO snapshots in HDFS
[
https://issues.apache.org/jira/browse/HDFS-2802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aaron T. Myers updated HDFS-2802:
-
Attachment: HDFSSnapshotsDesign.pdf
Hi all, attached please find a somewhat different design for implementing
snapshot support in HDFS that myself and a few others have discussed. Please
have a look at it.
Though this design differs somewhat from the previous design posted by
Nicholas, I don't think the two designs insurmountably far apart. Though I
certainly don't expect to switch to this design wholesale, I would like to see
if we can come up with a hybrid design which incorporates some aspects of both.
Let me try to outline where I see the designs differing, and suggest ways we
can move forward with a hybrid design.
# *Efficiency of snapshot creation.* In the design posted by Nicholas, creation
of a snapshot is O{n} in terms of the number of files/directories captured by
the snapshot, both in terms of time and space efficiency. The design proposed
in this document would be O{1} at snapshot creation time, and then
copy-on-write thereafter for files/directories which are modified after the
snapshot is created. This is accomplished by assigning unique, increasing
integer IDs to snapshots and giving each INode a start_snap and end_snap ID to
denote which snapshots the INode should be a part of. I'm not wedded to the
precise design described in this document, but it seems like a reasonable
design to me, so I'd like to consider this for the design to implement
HDFS-4103 (Support O{1} snapshot creation).
# *Support for subdirectory snapshots.* The design posted by Nicholas allows
for individual subdirectories of an HDFS namespace to be snapshotted by
introducing "snapshottable directories." The design proposed in this document
would only support snapshots at the root level of the file system. I think an
easy way to produce a hybrid between these two designs would be to stick with
the "snapshottable directory" system described in the document posted by
Nicholas, and store the snapshot ID info at that INodeDirectory, instead of
globally for the whole file system as is described in the document I've just
posted. Such a scheme will allow both for efficient snapshot creation and
creation of snapshots of subdirectories of the file system.
# *Support for non-super users to create snapshots.* The design posted by
Nicholas allows for non-super users to create snapshots. The scheme described
in the document I've just posted would only allow super users to create
snapshots, in instances where administrators want tight control over the
snapshots in their system. I propose we stick with the design described in the
document posted by Nicholas, but allow for user-initiated snapshot creation to
be optionally disabled by the administrator, either globally or
per-snapshottable directory. This should allow for both use cases
simultaneously.
# *Materialization of snapshots.* The scheme described in the document posted
by Nicholas allows for the state of the FS in a snapshot to only be accessed
from the snapshot root, i.e. the snapshottable directory, and allows for
snapshots to be created with arbitrary names. The scheme described by the
document I've just posted would have the return value of
ClientProtocol#getListing modified on the fly by the NameNode so that a
".snapshots" directory will appear to be present in every directory which has a
snapshot available for it, with the available snapshots listed under this
"directory" by their snapshot ID. This is similar to the user experience that
users of WAFL file systems are familiar to, and so should be familiar to many
users of FS snapshots. I'd like us to consider going with this scheme.
Please consider this proposal. I'd love to discuss this further at the design
meeting later this week as previously mentioned by Suresh. By the way, can we
nail down the precise date/time for that meeting? Sanjay mentioned to me
offline that it would probably be on Wednesday, but I haven't heard anything
beyond that. I'd be happy to offer up space in the Cloudera office, if that
would be helpful. Let me know.
Thanks everyone.
> Support for RW/RO snapshots in HDFS
> ---
>
> Key: HDFS-2802
> URL: https://issues.apache.org/jira/browse/HDFS-2802
> Project: Hadoop HDFS
> Issue Type: New Feature
> Components: data-node, name-node
>Reporter: Hari Mankude
>Assignee: Hari Mankude
> Attachments: HDFSSnapshotsDesign.pdf, snap.patch,
> snapshot-one-pager.pdf, Snapshots20121018.pdf
>
>
> Snapshots are point in time images of parts of the filesystem or the entire
> filesystem. Snapshots can be a read-only or a read-write point in time copy
> of the filesystem. There are several use cases for snapshots in HDFS. I will
[jira] [Updated] (HDFS-2802) Support for RW/RO snapshots in HDFS
[ https://issues.apache.org/jira/browse/HDFS-2802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron T. Myers updated HDFS-2802: - Tags: (was: 1) Release Note: (was: 1) > Support for RW/RO snapshots in HDFS > --- > > Key: HDFS-2802 > URL: https://issues.apache.org/jira/browse/HDFS-2802 > Project: Hadoop HDFS > Issue Type: New Feature > Components: data-node, name-node >Reporter: Hari Mankude >Assignee: Hari Mankude > Attachments: snap.patch, snapshot-one-pager.pdf, Snapshots20121018.pdf > > > Snapshots are point in time images of parts of the filesystem or the entire > filesystem. Snapshots can be a read-only or a read-write point in time copy > of the filesystem. There are several use cases for snapshots in HDFS. I will > post a detailed write-up soon with with more information. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2802) Support for RW/RO snapshots in HDFS
[ https://issues.apache.org/jira/browse/HDFS-2802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo (Nicholas), SZE updated HDFS-2802: - Attachment: Snapshots20121018.pdf >From the last time the patch is posted for the prototype, I have been >continuing work on the code. The attachment (Snapshots20121018.pdf) is the >updated design document based on currently where the implementation is. It is >still fairly highlevel, but should has enough details to give an idea on how >the snapshot feature is being implemented. I will continue to update the >design as the implementation progresses. Here are some of the next steps: # Split the large patch and create subtasks in this jira, to make it easy to understand the implementation. # Will schedule a design review meeting to go over the requirements and design in detail with the folks who are interested. Especially to get feedback from other Hadoop projects. This meeting will be in the week of Oct 29, post Hadoop World. > Support for RW/RO snapshots in HDFS > --- > > Key: HDFS-2802 > URL: https://issues.apache.org/jira/browse/HDFS-2802 > Project: Hadoop HDFS > Issue Type: New Feature > Components: data-node, name-node >Reporter: Hari Mankude >Assignee: Hari Mankude > Attachments: snap.patch, snapshot-one-pager.pdf, Snapshots20121018.pdf > > > Snapshots are point in time images of parts of the filesystem or the entire > filesystem. Snapshots can be a read-only or a read-write point in time copy > of the filesystem. There are several use cases for snapshots in HDFS. I will > post a detailed write-up soon with with more information. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2802) Support for RW/RO snapshots in HDFS
[ https://issues.apache.org/jira/browse/HDFS-2802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HDFS-2802: -- Status: Open (was: Patch Available) > Support for RW/RO snapshots in HDFS > --- > > Key: HDFS-2802 > URL: https://issues.apache.org/jira/browse/HDFS-2802 > Project: Hadoop HDFS > Issue Type: New Feature > Components: data-node, name-node >Reporter: Hari Mankude >Assignee: Hari Mankude > Attachments: snap.patch, snapshot-one-pager.pdf > > > Snapshots are point in time images of parts of the filesystem or the entire > filesystem. Snapshots can be a read-only or a read-write point in time copy > of the filesystem. There are several use cases for snapshots in HDFS. I will > post a detailed write-up soon with with more information. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2802) Support for RW/RO snapshots in HDFS
[ https://issues.apache.org/jira/browse/HDFS-2802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anonymous updated HDFS-2802: Tags: 1 Target Version/s: (was: 0.24.0) Affects Version/s: (was: 0.24.0) Release Note: 1 Hadoop Flags: Incompatible change Status: Patch Available (was: Reopened) > Support for RW/RO snapshots in HDFS > --- > > Key: HDFS-2802 > URL: https://issues.apache.org/jira/browse/HDFS-2802 > Project: Hadoop HDFS > Issue Type: New Feature > Components: data-node, name-node >Reporter: Hari Mankude >Assignee: Hari Mankude > Labels: > Attachments: snap.patch, snapshot-one-pager.pdf > > > Snapshots are point in time images of parts of the filesystem or the entire > filesystem. Snapshots can be a read-only or a read-write point in time copy > of the filesystem. There are several use cases for snapshots in HDFS. I will > post a detailed write-up soon with with more information. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2802) Support for RW/RO snapshots in HDFS
[ https://issues.apache.org/jira/browse/HDFS-2802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hari Mankude updated HDFS-2802: --- Attachment: snap.patch > Support for RW/RO snapshots in HDFS > --- > > Key: HDFS-2802 > URL: https://issues.apache.org/jira/browse/HDFS-2802 > Project: Hadoop HDFS > Issue Type: New Feature > Components: data-node, name-node >Affects Versions: 0.24.0 >Reporter: Hari Mankude >Assignee: Hari Mankude > Attachments: snap.patch, snapshot-one-pager.pdf > > > Snapshots are point in time images of parts of the filesystem or the entire > filesystem. Snapshots can be a read-only or a read-write point in time copy > of the filesystem. There are several use cases for snapshots in HDFS. I will > post a detailed write-up soon with with more information. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2802) Support for RW/RO snapshots in HDFS
[ https://issues.apache.org/jira/browse/HDFS-2802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hari Mankude updated HDFS-2802: --- Attachment: snapshot-one-pager.pdf > Support for RW/RO snapshots in HDFS > --- > > Key: HDFS-2802 > URL: https://issues.apache.org/jira/browse/HDFS-2802 > Project: Hadoop HDFS > Issue Type: New Feature > Components: data-node, name-node >Affects Versions: 0.24.0 >Reporter: Hari Mankude >Assignee: Hari Mankude > Attachments: snapshot-one-pager.pdf > > > Snapshots are point in time images of parts of the filesystem or the entire > filesystem. Snapshots can be a read-only or a read-write point in time copy > of the filesystem. There are several use cases for snapshots in HDFS. I will > post a detailed write-up soon with with more information. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2802) Support for RW/RO snapshots in HDFS
[ https://issues.apache.org/jira/browse/HDFS-2802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron T. Myers updated HDFS-2802: - Component/s: data-node Target Version/s: 0.24.0 Affects Version/s: 0.24.0 Fix Version/s: (was: 0.24.0) > Support for RW/RO snapshots in HDFS > --- > > Key: HDFS-2802 > URL: https://issues.apache.org/jira/browse/HDFS-2802 > Project: Hadoop HDFS > Issue Type: New Feature > Components: data-node, name-node >Affects Versions: 0.24.0 >Reporter: Hari Mankude >Assignee: Hari Mankude > > Snapshots are point in time images of parts of the filesystem or the entire > filesystem. Snapshots can be a read-only or a read-write point in time copy > of the filesystem. There are several use cases for snapshots in HDFS. I will > post a detailed write-up soon with with more information. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
