[jira] [Commented] (HDFS-4167) Add support for restoring/rolling back to a snapshot
[
https://issues.apache.org/jira/browse/HDFS-4167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14948430#comment-14948430
]
Hadoop QA commented on HDFS-4167:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | pre-patch | 21m 55s | Pre-patch trunk has 748 extant
Findbugs (version 3.0.0) warnings. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | tests included | 0m 0s | The patch appears to
include 2 new or modified test files. |
| {color:green}+1{color} | javac | 8m 0s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 10m 26s | There were no new javadoc
warning messages. |
| {color:red}-1{color} | release audit | 0m 20s | The applied patch generated
1 release audit warnings. |
| {color:red}-1{color} | checkstyle | 2m 37s | The applied patch generated 2
new checkstyle issues (total was 157, now 158). |
| {color:red}-1{color} | checkstyle | 3m 39s | The applied patch generated 2
new checkstyle issues (total was 98, now 98). |
| {color:red}-1{color} | whitespace | 0m 36s | The patch has 2 line(s) that
end in whitespace. Use git apply --whitespace=fix. |
| {color:green}+1{color} | install | 1m 39s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 34s | The patch built with
eclipse:eclipse. |
| {color:red}-1{color} | findbugs | 6m 51s | The patch appears to introduce 4
new Findbugs (version 3.0.0) warnings. |
| {color:red}-1{color} | common tests | 7m 50s | Tests failed in
hadoop-common. |
| {color:red}-1{color} | hdfs tests | 189m 7s | Tests failed in hadoop-hdfs. |
| {color:green}+1{color} | hdfs tests | 0m 30s | Tests passed in
hadoop-hdfs-client. |
| | | 251m 32s | |
\\
\\
|| Reason || Tests ||
| FindBugs | module:hadoop-hdfs-client |
| Failed unit tests | hadoop.metrics2.impl.TestGangliaMetrics |
| | hadoop.hdfs.server.datanode.TestDataNodeMetrics |
| | hadoop.hdfs.tools.offlineEditsViewer.TestOfflineEditsViewer |
| | hadoop.hdfs.TestDFSInotifyEventInputStream |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12765523/HDFS-4167.08.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / 35affec |
| Pre-patch Findbugs warnings |
https://builds.apache.org/job/PreCommit-HDFS-Build/12854/artifact/patchprocess/trunkFindbugsWarningshadoop-hdfs-client.html
|
| Release Audit |
https://builds.apache.org/job/PreCommit-HDFS-Build/12854/artifact/patchprocess/patchReleaseAuditProblems.txt
|
| checkstyle |
https://builds.apache.org/job/PreCommit-HDFS-Build/12854/artifact/patchprocess/diffcheckstylehadoop-common.txt
https://builds.apache.org/job/PreCommit-HDFS-Build/12854/artifact/patchprocess/diffcheckstylehadoop-hdfs-client.txt
|
| whitespace |
https://builds.apache.org/job/PreCommit-HDFS-Build/12854/artifact/patchprocess/whitespace.txt
|
| Findbugs warnings |
https://builds.apache.org/job/PreCommit-HDFS-Build/12854/artifact/patchprocess/newPatchFindbugsWarningshadoop-hdfs-client.html
|
| hadoop-common test log |
https://builds.apache.org/job/PreCommit-HDFS-Build/12854/artifact/patchprocess/testrun_hadoop-common.txt
|
| hadoop-hdfs test log |
https://builds.apache.org/job/PreCommit-HDFS-Build/12854/artifact/patchprocess/testrun_hadoop-hdfs.txt
|
| hadoop-hdfs-client test log |
https://builds.apache.org/job/PreCommit-HDFS-Build/12854/artifact/patchprocess/testrun_hadoop-hdfs-client.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HDFS-Build/12854/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf903.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-HDFS-Build/12854/console |
This message was automatically generated.
> Add support for restoring/rolling back to a snapshot
>
>
> Key: HDFS-4167
> URL: https://issues.apache.org/jira/browse/HDFS-4167
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: namenode
>Reporter: Suresh Srinivas
>Assignee: Ajith S
> Attachments: HDFS-4167.000.patch, HDFS-4167.001.patch,
> HDFS-4167.002.patch, HDFS-4167.003.patch, HDFS-4167.004.patch,
> HDFS-4167.05.patch, HDFS-4167.06.patch, HDFS-4167.07.patch, HDFS-4167.08.patch
>
>
> This jira tracks work related to restoring a directory/file to a snapshot.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-4167) Add support for restoring/rolling back to a snapshot
[
https://issues.apache.org/jira/browse/HDFS-4167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14946774#comment-14946774
]
Hadoop QA commented on HDFS-4167:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | pre-patch | 22m 1s | Pre-patch trunk has 748 extant
Findbugs (version 3.0.0) warnings. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | tests included | 0m 0s | The patch appears to
include 1 new or modified test files. |
| {color:green}+1{color} | javac | 8m 21s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 11m 28s | There were no new javadoc
warning messages. |
| {color:red}-1{color} | release audit | 0m 21s | The applied patch generated
1 release audit warnings. |
| {color:red}-1{color} | checkstyle | 2m 39s | The applied patch generated 1
new checkstyle issues (total was 157, now 157). |
| {color:red}-1{color} | checkstyle | 3m 50s | The applied patch generated 3
new checkstyle issues (total was 98, now 99). |
| {color:red}-1{color} | whitespace | 0m 37s | The patch has 6 line(s) that
end in whitespace. Use git apply --whitespace=fix. |
| {color:green}+1{color} | install | 1m 48s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 37s | The patch built with
eclipse:eclipse. |
| {color:red}-1{color} | findbugs | 7m 24s | The patch appears to introduce 4
new Findbugs (version 3.0.0) warnings. |
| {color:red}-1{color} | common tests | 7m 8s | Tests failed in
hadoop-common. |
| {color:red}-1{color} | hdfs tests | 196m 25s | Tests failed in hadoop-hdfs. |
| {color:green}+1{color} | hdfs tests | 0m 33s | Tests passed in
hadoop-hdfs-client. |
| | | 260m 39s | |
\\
\\
|| Reason || Tests ||
| FindBugs | module:hadoop-hdfs-client |
| Failed unit tests | hadoop.fs.TestHarFileSystem |
| | hadoop.net.TestDNS |
| | hadoop.hdfs.tools.offlineEditsViewer.TestOfflineEditsViewer |
| | hadoop.hdfs.web.TestWebHDFSOAuth2 |
| | hadoop.hdfs.TestDFSInotifyEventInputStream |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12765337/HDFS-4167.07.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / 3112f26 |
| Pre-patch Findbugs warnings |
https://builds.apache.org/job/PreCommit-HDFS-Build/12829/artifact/patchprocess/trunkFindbugsWarningshadoop-hdfs-client.html
|
| Release Audit |
https://builds.apache.org/job/PreCommit-HDFS-Build/12829/artifact/patchprocess/patchReleaseAuditProblems.txt
|
| checkstyle |
https://builds.apache.org/job/PreCommit-HDFS-Build/12829/artifact/patchprocess/diffcheckstylehadoop-common.txt
https://builds.apache.org/job/PreCommit-HDFS-Build/12829/artifact/patchprocess/diffcheckstylehadoop-hdfs-client.txt
|
| whitespace |
https://builds.apache.org/job/PreCommit-HDFS-Build/12829/artifact/patchprocess/whitespace.txt
|
| Findbugs warnings |
https://builds.apache.org/job/PreCommit-HDFS-Build/12829/artifact/patchprocess/newPatchFindbugsWarningshadoop-hdfs-client.html
|
| hadoop-common test log |
https://builds.apache.org/job/PreCommit-HDFS-Build/12829/artifact/patchprocess/testrun_hadoop-common.txt
|
| hadoop-hdfs test log |
https://builds.apache.org/job/PreCommit-HDFS-Build/12829/artifact/patchprocess/testrun_hadoop-hdfs.txt
|
| hadoop-hdfs-client test log |
https://builds.apache.org/job/PreCommit-HDFS-Build/12829/artifact/patchprocess/testrun_hadoop-hdfs-client.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HDFS-Build/12829/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf900.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-HDFS-Build/12829/console |
This message was automatically generated.
> Add support for restoring/rolling back to a snapshot
>
>
> Key: HDFS-4167
> URL: https://issues.apache.org/jira/browse/HDFS-4167
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: namenode
>Reporter: Suresh Srinivas
>Assignee: Ajith S
> Attachments: HDFS-4167.000.patch, HDFS-4167.001.patch,
> HDFS-4167.002.patch, HDFS-4167.003.patch, HDFS-4167.004.patch,
> HDFS-4167.05.patch, HDFS-4167.06.patch, HDFS-4167.07.patch
>
>
> This jira tracks work related to restoring a directory/file to a snapshot.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-4167) Add support for restoring/rolling back to a snapshot
[
https://issues.apache.org/jira/browse/HDFS-4167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14946310#comment-14946310
]
Hadoop QA commented on HDFS-4167:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | patch | 0m 0s | The patch command could not apply
the patch during dryrun. |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12765135/HDFS-4167.06.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / 8b73393 |
| Console output |
https://builds.apache.org/job/PreCommit-HDFS-Build/12828/console |
This message was automatically generated.
> Add support for restoring/rolling back to a snapshot
>
>
> Key: HDFS-4167
> URL: https://issues.apache.org/jira/browse/HDFS-4167
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: namenode
>Reporter: Suresh Srinivas
>Assignee: Ajith S
> Attachments: HDFS-4167.000.patch, HDFS-4167.001.patch,
> HDFS-4167.002.patch, HDFS-4167.003.patch, HDFS-4167.004.patch,
> HDFS-4167.05.patch, HDFS-4167.06.patch
>
>
> This jira tracks work related to restoring a directory/file to a snapshot.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-4167) Add support for restoring/rolling back to a snapshot
[
https://issues.apache.org/jira/browse/HDFS-4167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14908449#comment-14908449
]
Hadoop QA commented on HDFS-4167:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch | 21m 48s | Pre-patch trunk compilation is
healthy. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | tests included | 0m 0s | The patch appears to
include 1 new or modified test files. |
| {color:green}+1{color} | javac | 8m 15s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 10m 15s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 23s | The applied patch does
not increase the total number of release audit warnings. |
| {color:red}-1{color} | checkstyle | 2m 32s | The applied patch generated 1
new checkstyle issues (total was 140, now 140). |
| {color:red}-1{color} | checkstyle | 3m 34s | The applied patch generated 1
new checkstyle issues (total was 50, now 51). |
| {color:red}-1{color} | whitespace | 0m 33s | The patch has 5 line(s) that
end in whitespace. Use git apply --whitespace=fix. |
| {color:green}+1{color} | install | 1m 40s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 34s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | findbugs | 6m 31s | The patch does not introduce
any new Findbugs (version 3.0.0) warnings. |
| {color:red}-1{color} | common tests | 23m 25s | Tests failed in
hadoop-common. |
| {color:red}-1{color} | hdfs tests | 71m 33s | Tests failed in hadoop-hdfs. |
| {color:green}+1{color} | hdfs tests | 0m 29s | Tests passed in
hadoop-hdfs-client. |
| | | 149m 6s | |
\\
\\
|| Reason || Tests ||
| Failed unit tests | hadoop.fs.TestHarFileSystem |
| | hadoop.fs.TestFilterFileSystem |
| | hadoop.hdfs.TestParallelShortCircuitRead |
| | hadoop.fs.contract.hdfs.TestHDFSContractMkdir |
| | hadoop.hdfs.server.datanode.TestDataNodeUUID |
| | hadoop.hdfs.qjournal.client.TestQuorumJournalManagerUnit |
| | hadoop.hdfs.server.namenode.TestAllowFormat |
| | hadoop.hdfs.server.namenode.TestCheckPointForSecurityTokens |
| | hadoop.hdfs.TestBlockStoragePolicy |
| | hadoop.hdfs.server.namenode.ha.TestBootstrapStandbyWithQJM |
| | hadoop.hdfs.protocol.TestBlockListAsLongs |
| | hadoop.hdfs.server.namenode.snapshot.TestSnapshotMetrics |
| | hadoop.hdfs.TestDFSInotifyEventInputStream |
| | hadoop.cli.TestDeleteCLI |
| | hadoop.hdfs.tools.TestDFSZKFailoverController |
| | hadoop.hdfs.TestFileLengthOnClusterRestart |
| | hadoop.hdfs.TestAppendSnapshotTruncate |
| | hadoop.hdfs.server.namenode.snapshot.TestSnapshottableDirListing |
| | hadoop.fs.contract.hdfs.TestHDFSContractRootDirectory |
| | hadoop.hdfs.server.namenode.snapshot.TestUpdatePipelineWithSnapshots |
| | hadoop.hdfs.server.namenode.TestDiskspaceQuotaUpdate |
| | hadoop.cli.TestHDFSCLI |
| | hadoop.hdfs.server.namenode.TestCheckpoint |
| | hadoop.hdfs.TestDFSUpgradeFromImage |
| | hadoop.hdfs.TestReplaceDatanodeOnFailure |
| | hadoop.hdfs.tools.TestGetGroups |
| | hadoop.hdfs.TestRemoteBlockReader2 |
| | hadoop.hdfs.server.namenode.TestStartup |
| | hadoop.hdfs.protocol.datatransfer.TestPacketReceiver |
| | hadoop.hdfs.server.namenode.TestNameNodeRetryCacheMetrics |
| | hadoop.hdfs.TestDFSStorageStateRecovery |
| | hadoop.hdfs.server.namenode.TestFSImageWithXAttr |
| | hadoop.hdfs.TestRemoteBlockReader |
| | hadoop.hdfs.TestMultiThreadedHflush |
| | hadoop.fs.contract.hdfs.TestHDFSContractRename |
| | hadoop.hdfs.TestBlockReaderLocal |
| | hadoop.cli.TestCacheAdminCLI |
| | hadoop.hdfs.server.mover.TestMover |
| | hadoop.hdfs.server.blockmanagement.TestOverReplicatedBlocks |
| | hadoop.hdfs.server.namenode.ha.TestInitializeSharedEdits |
| | hadoop.hdfs.server.namenode.web.resources.TestWebHdfsDataLocality |
| | hadoop.hdfs.server.namenode.TestNameNodeRecovery |
| | hadoop.hdfs.server.namenode.ha.TestFailureOfSharedDir |
| | hadoop.hdfs.TestKeyProviderCache |
| | hadoop.fs.loadGenerator.TestLoadGenerator |
| | hadoop.hdfs.server.namenode.TestFSImageWithAcl |
| | hadoop.hdfs.server.namenode.TestLargeDirectoryDelete |
| | hadoop.fs.TestFcHdfsSetUMask |
| | hadoop.hdfs.TestPread |
| | hadoop.hdfs.server.namenode.TestFSEditLogLoader |
| | hadoop.hdfs.server.datanode.TestFsDatasetCacheRevocation |
| | hadoop.hdfs.server.namenode.ha.TestQuotasWithHA |
| | hadoop.hdfs.crypto.TestHdfsCryptoStreams |
| | hadoop.fs.viewfs.TestViewFsFileStatusHdfs |
| | hadoop.hdfs.server.namenode.TestCommitBlockSynchronization |
| | hadoop.hdfs.tools.offlineImageViewer.TestOfflineImageViewerForAcl |
| | hadoop.hdfs.server.namenode.ha.TestHAConfiguration |
| |
[jira] [Commented] (HDFS-4167) Add support for restoring/rolling back to a snapshot
[ https://issues.apache.org/jira/browse/HDFS-4167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14707815#comment-14707815 ] Ajith S commented on HDFS-4167: --- Hi [~jingzhao] Can i please work on this patch in case you are not.?? Add support for restoring/rolling back to a snapshot Key: HDFS-4167 URL: https://issues.apache.org/jira/browse/HDFS-4167 Project: Hadoop HDFS Issue Type: Sub-task Components: namenode Reporter: Suresh Srinivas Assignee: Jing Zhao Attachments: HDFS-4167.000.patch, HDFS-4167.001.patch, HDFS-4167.002.patch, HDFS-4167.003.patch, HDFS-4167.004.patch This jira tracks work related to restoring a directory/file to a snapshot. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-4167) Add support for restoring/rolling back to a snapshot
[ https://issues.apache.org/jira/browse/HDFS-4167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14707824#comment-14707824 ] Jing Zhao commented on HDFS-4167: - Sure, please feel free to assign the jira to yourself. Add support for restoring/rolling back to a snapshot Key: HDFS-4167 URL: https://issues.apache.org/jira/browse/HDFS-4167 Project: Hadoop HDFS Issue Type: Sub-task Components: namenode Reporter: Suresh Srinivas Assignee: Jing Zhao Attachments: HDFS-4167.000.patch, HDFS-4167.001.patch, HDFS-4167.002.patch, HDFS-4167.003.patch, HDFS-4167.004.patch This jira tracks work related to restoring a directory/file to a snapshot. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-4167) Add support for restoring/rolling back to a snapshot
[
https://issues.apache.org/jira/browse/HDFS-4167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14307790#comment-14307790
]
Hadoop QA commented on HDFS-4167:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12645278/HDFS-4167.004.patch
against trunk revision c4980a2.
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/9444//console
This message is automatically generated.
Add support for restoring/rolling back to a snapshot
Key: HDFS-4167
URL: https://issues.apache.org/jira/browse/HDFS-4167
Project: Hadoop HDFS
Issue Type: Sub-task
Components: namenode
Reporter: Suresh Srinivas
Assignee: Jing Zhao
Attachments: HDFS-4167.000.patch, HDFS-4167.001.patch,
HDFS-4167.002.patch, HDFS-4167.003.patch, HDFS-4167.004.patch
This jira tracks work related to restoring a directory/file to a snapshot.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-4167) Add support for restoring/rolling back to a snapshot
[ https://issues.apache.org/jira/browse/HDFS-4167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14010822#comment-14010822 ] Guo Ruijing commented on HDFS-4167: --- I agree that it's better to keep it as a standalone HDFS metadata operation. It is easy to restore any snapshot if block is copied out for append as HDFS-6087 proposal. what's change in append? 1. file f1 includes (Block1, Block2, Block3) 2. append to f1 a) client request block3 information from namenode b) client request to datanode to copy block3 as block4 c) append to block4 d) commit block4 to namenode what happend in snapshot? snap1: f1 include (block1, block2, block3) snap2: f2 include (block1, block2, block4) how to restore snapshot? just restore snap1 as current file since no partial blocks are shared by different snap. Add support for restoring/rolling back to a snapshot Key: HDFS-4167 URL: https://issues.apache.org/jira/browse/HDFS-4167 Project: Hadoop HDFS Issue Type: Sub-task Components: namenode Reporter: Suresh Srinivas Assignee: Jing Zhao Attachments: HDFS-4167.000.patch, HDFS-4167.001.patch, HDFS-4167.002.patch, HDFS-4167.003.patch, HDFS-4167.004.patch This jira tracks work related to restoring a directory/file to a snapshot. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HDFS-4167) Add support for restoring/rolling back to a snapshot
[ https://issues.apache.org/jira/browse/HDFS-4167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14001360#comment-14001360 ] Tsz Wo Nicholas Sze commented on HDFS-4167: --- ... If the directory /foo/bar was deleted after snapshot s1, and then a user wants to restore /foo/.snapshot/s1/bar/file, it is hard to determine where we should put the file. ... The file should be restored to the original location, i.e. its location before taking snapshot s1. ... Also the filesystem API must be defined as you suggested in the jira. However, the relative path may seem confusing since users usually use relative path against the working directory of the file system. Then we may use the snapshot path as the parameter e.g. fs.restoreSnapshot(/foo/.snapshot/s0/bar). Add support for restoring/rolling back to a snapshot Key: HDFS-4167 URL: https://issues.apache.org/jira/browse/HDFS-4167 Project: Hadoop HDFS Issue Type: Sub-task Components: namenode Reporter: Suresh Srinivas Assignee: Jing Zhao Attachments: HDFS-4167.000.patch, HDFS-4167.001.patch, HDFS-4167.002.patch, HDFS-4167.003.patch, HDFS-4167.004.patch This jira tracks work related to restoring a directory/file to a snapshot. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HDFS-4167) Add support for restoring/rolling back to a snapshot
[ https://issues.apache.org/jira/browse/HDFS-4167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13998656#comment-13998656 ] Tsz Wo Nicholas Sze commented on HDFS-4167: --- Sure, let's make it as a metadata only operation and handle truncate later. Add support for restoring/rolling back to a snapshot Key: HDFS-4167 URL: https://issues.apache.org/jira/browse/HDFS-4167 Project: Hadoop HDFS Issue Type: Sub-task Components: namenode Reporter: Suresh Srinivas Assignee: Jing Zhao Attachments: HDFS-4167.000.patch, HDFS-4167.001.patch, HDFS-4167.002.patch, HDFS-4167.003.patch This jira tracks work related to restoring a directory/file to a snapshot. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HDFS-4167) Add support for restoring/rolling back to a snapshot
[
https://issues.apache.org/jira/browse/HDFS-4167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14000635#comment-14000635
]
Hadoop QA commented on HDFS-4167:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12645278/HDFS-4167.004.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs:
org.apache.hadoop.fs.TestHarFileSystem
org.apache.hadoop.fs.TestFilterFileSystem
org.apache.hadoop.hdfs.tools.offlineEditsViewer.TestOfflineEditsViewer
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HDFS-Build/6921//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/6921//console
This message is automatically generated.
Add support for restoring/rolling back to a snapshot
Key: HDFS-4167
URL: https://issues.apache.org/jira/browse/HDFS-4167
Project: Hadoop HDFS
Issue Type: Sub-task
Components: namenode
Reporter: Suresh Srinivas
Assignee: Jing Zhao
Attachments: HDFS-4167.000.patch, HDFS-4167.001.patch,
HDFS-4167.002.patch, HDFS-4167.003.patch, HDFS-4167.004.patch
This jira tracks work related to restoring a directory/file to a snapshot.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HDFS-4167) Add support for restoring/rolling back to a snapshot
[ https://issues.apache.org/jira/browse/HDFS-4167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13998131#comment-13998131 ] Jing Zhao commented on HDFS-4167: - Yeah, I think it's better to keep it as a standalone HDFS metadata operation. Maybe as a first step, we can fail the restore operation first when we find that we need to handle outside rename and/or file truncation? Then users can force the command if she/he agrees with our restore semantic. Add support for restoring/rolling back to a snapshot Key: HDFS-4167 URL: https://issues.apache.org/jira/browse/HDFS-4167 Project: Hadoop HDFS Issue Type: Sub-task Components: namenode Reporter: Suresh Srinivas Assignee: Jing Zhao Attachments: HDFS-4167.000.patch, HDFS-4167.001.patch, HDFS-4167.002.patch, HDFS-4167.003.patch This jira tracks work related to restoring a directory/file to a snapshot. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HDFS-4167) Add support for restoring/rolling back to a snapshot
[ https://issues.apache.org/jira/browse/HDFS-4167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13989241#comment-13989241 ] Tsz Wo Nicholas Sze commented on HDFS-4167: --- Suppose a (closed) file foo has size 100MB and a snapshot s0 is taken. Then foo is appended to 110MB. .. For restoring foo, we could recreate the file by reusing the first n-1 blocks and copy the last block with the original length. For restoring a large directory, we may need a MapReduce job to do the copying like HAR or DistCp. Add support for restoring/rolling back to a snapshot Key: HDFS-4167 URL: https://issues.apache.org/jira/browse/HDFS-4167 Project: Hadoop HDFS Issue Type: Sub-task Components: namenode Reporter: Suresh Srinivas Assignee: Jing Zhao Attachments: HDFS-4167.000.patch, HDFS-4167.001.patch, HDFS-4167.002.patch, HDFS-4167.003.patch This jira tracks work related to restoring a directory/file to a snapshot. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HDFS-4167) Add support for restoring/rolling back to a snapshot
[ https://issues.apache.org/jira/browse/HDFS-4167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13988703#comment-13988703 ] Tsz Wo Nicholas Sze commented on HDFS-4167: --- Suppose a (closed) file foo has size 100MB and a snapshot s0 is taken. Then foo is appended to 110MB. Now, restore s0 but foo is still 110MB. I think it may not be a good behavior. Add support for restoring/rolling back to a snapshot Key: HDFS-4167 URL: https://issues.apache.org/jira/browse/HDFS-4167 Project: Hadoop HDFS Issue Type: Sub-task Components: namenode Reporter: Suresh Srinivas Assignee: Jing Zhao Attachments: HDFS-4167.000.patch, HDFS-4167.001.patch, HDFS-4167.002.patch, HDFS-4167.003.patch This jira tracks work related to restoring a directory/file to a snapshot. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HDFS-4167) Add support for restoring/rolling back to a snapshot
[ https://issues.apache.org/jira/browse/HDFS-4167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13988708#comment-13988708 ] Tsz Wo Nicholas Sze commented on HDFS-4167: --- Some other comments: - It seems that the implementation does not support restoring a deleted file. For example, if /foo/.snapshot/s0/bar is a file in snapshot s0 and /foo/bar is deleted, then fs.restoreSnapshot(/foo/bar, /foo, s0) will fail since the INode of /foo/bar cannot be found. - For the new FileSystem API restoreSnapshot(Path restorePath, Path snapshotRoot, String snapshotName), I think restorePath should be a relative path to the snapshot and we should reorder the parameter to snapshotRoot, snapshotName, restorePath (continue with the previous example, fs.restoreSnapshot(/foo, s0, bar)) or simple combine them to a snapshot path, i.e. fs.restoreSnapshot(/foo/.snapshot/s0/bar). - Let's throw SnapshotException instead of IOException for the snapshot related exceptions. - In INodeDirectory.restoreChildrenList, use Collections.INodeemptyList(). Add support for restoring/rolling back to a snapshot Key: HDFS-4167 URL: https://issues.apache.org/jira/browse/HDFS-4167 Project: Hadoop HDFS Issue Type: Sub-task Components: namenode Reporter: Suresh Srinivas Assignee: Jing Zhao Attachments: HDFS-4167.000.patch, HDFS-4167.001.patch, HDFS-4167.002.patch, HDFS-4167.003.patch This jira tracks work related to restoring a directory/file to a snapshot. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HDFS-4167) Add support for restoring/rolling back to a snapshot
[
https://issues.apache.org/jira/browse/HDFS-4167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13980880#comment-13980880
]
Vinayakumar B commented on HDFS-4167:
-
Hi,
Its nice to see this feature on board instead of manual restore from the
snapshot.
I see that design document attached is in different approach than the patch
attached. It can be confusing .. ;)
Some quick comments on the patch
1. {code}
+ * @param restoreRoot
+ * The file/dir to restore * @param collectedBlocks blocks collected
+ * from the descents for further block deletion/update will be added
+ * to the given map.
{code}
should be properly formatted. collectedBlocks got mixed with restoreRoot's
description
2. {code}
+DirectoryWithSnapshotFeature sf = getDirectoryWithSnapshotFeature();
+Quota.Counts delta = Quota.Counts.newInstance();
+if (sf != null) {
+ sf.restoreSnapshot(this, restoreRoot, snapshot, collectedBlocks,
+ removedINodes);
+}
+return delta;
{code}
not using the return quota values from {{sf.restoreSnapshot(..)}}
3. FSDirectory.java changes could be ignored from the patch as its only
whitespace change.
Add support for restoring/rolling back to a snapshot
Key: HDFS-4167
URL: https://issues.apache.org/jira/browse/HDFS-4167
Project: Hadoop HDFS
Issue Type: Sub-task
Components: namenode
Affects Versions: Snapshot (HDFS-2802)
Reporter: Suresh Srinivas
Assignee: Jing Zhao
Attachments: HDFS Design Proposal.pdf, HDFS-4167.000.patch
This jira tracks work related to restoring a directory/file to a snapshot.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HDFS-4167) Add support for restoring/rolling back to a snapshot
[ https://issues.apache.org/jira/browse/HDFS-4167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981298#comment-13981298 ] Jing Zhao commented on HDFS-4167: - Thanks for the comments, [~vinayrpet]! I will update the patch to address your comments. Will also add new unit tests and fix remaining bugs. bq. I see that design document attached is in different approach than the patch attached. It can be confusing .. [~rguo], since the design doc is not tightly related to this jira, and has already been posted in HDFS-6087, I remove it from this jira first. Please feel free to create separate jiras for new design of snapshots. Add support for restoring/rolling back to a snapshot Key: HDFS-4167 URL: https://issues.apache.org/jira/browse/HDFS-4167 Project: Hadoop HDFS Issue Type: Sub-task Components: namenode Affects Versions: Snapshot (HDFS-2802) Reporter: Suresh Srinivas Assignee: Jing Zhao Attachments: HDFS-4167.000.patch This jira tracks work related to restoring a directory/file to a snapshot. -- This message was sent by Atlassian JIRA (v6.2#6252)
