[jira] [Updated] (HDFS-16187) SnapshotDiff behaviour with Xattrs and Acls is not consistent across NN restarts with checkpointing
[
https://issues.apache.org/jira/browse/HDFS-16187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Masatake Iwasaki updated HDFS-16187:
Fix Version/s: 3.2.4
(was: 3.2.3)
> SnapshotDiff behaviour with Xattrs and Acls is not consistent across NN
> restarts with checkpointing
> ---
>
> Key: HDFS-16187
> URL: https://issues.apache.org/jira/browse/HDFS-16187
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: snapshots
>Reporter: Srinivasu Majeti
>Assignee: Shashikant Banerjee
>Priority: Major
> Labels: pull-request-available
> Fix For: 3.4.0, 3.3.2, 3.2.4
>
> Time Spent: 5h
> Remaining Estimate: 0h
>
> The below test shows the snapshot diff between across snapshots is not
> consistent with Xattr(EZ here settinh the Xattr) across NN restarts with
> checkpointed FsImage.
> {code:java}
> @Test
> public void testEncryptionZonesWithSnapshots() throws Exception {
> final Path snapshottable = new Path("/zones");
> fsWrapper.mkdir(snapshottable, FsPermission.getDirDefault(),
> true);
> dfsAdmin.allowSnapshot(snapshottable);
> dfsAdmin.createEncryptionZone(snapshottable, TEST_KEY, NO_TRASH);
> fs.createSnapshot(snapshottable, "snap1");
> SnapshotDiffReport report =
> fs.getSnapshotDiffReport(snapshottable, "snap1", "");
> Assert.assertEquals(0, report.getDiffList().size());
> report =
> fs.getSnapshotDiffReport(snapshottable, "snap1", "");
> System.out.println(report);
> Assert.assertEquals(0, report.getDiffList().size());
> fs.setSafeMode(SafeModeAction.SAFEMODE_ENTER);
> fs.saveNamespace();
> fs.setSafeMode(SafeModeAction.SAFEMODE_LEAVE);
> cluster.restartNameNode(true);
> report =
> fs.getSnapshotDiffReport(snapshottable, "snap1", "");
> Assert.assertEquals(0, report.getDiffList().size());
> }{code}
> {code:java}
> Pre Restart:
> Difference between snapshot snap1 and current directory under directory
> /zones:
> Post Restart:
> Difference between snapshot snap1 and current directory under directory
> /zones:
> M .{code}
> The side effect of this behavior is : distcp with snapshot diff would fail
> with below error complaining that target cluster has some data changed .
> {code:java}
> WARN tools.DistCp: The target has been modified since snapshot x
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-16187) SnapshotDiff behaviour with Xattrs and Acls is not consistent across NN restarts with checkpointing
[
https://issues.apache.org/jira/browse/HDFS-16187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wei-Chiu Chuang updated HDFS-16187:
---
Fix Version/s: 3.3.2
> SnapshotDiff behaviour with Xattrs and Acls is not consistent across NN
> restarts with checkpointing
> ---
>
> Key: HDFS-16187
> URL: https://issues.apache.org/jira/browse/HDFS-16187
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: snapshots
>Reporter: Srinivasu Majeti
>Assignee: Shashikant Banerjee
>Priority: Major
> Labels: pull-request-available
> Fix For: 3.4.0, 3.3.2
>
> Time Spent: 5h
> Remaining Estimate: 0h
>
> The below test shows the snapshot diff between across snapshots is not
> consistent with Xattr(EZ here settinh the Xattr) across NN restarts with
> checkpointed FsImage.
> {code:java}
> @Test
> public void testEncryptionZonesWithSnapshots() throws Exception {
> final Path snapshottable = new Path("/zones");
> fsWrapper.mkdir(snapshottable, FsPermission.getDirDefault(),
> true);
> dfsAdmin.allowSnapshot(snapshottable);
> dfsAdmin.createEncryptionZone(snapshottable, TEST_KEY, NO_TRASH);
> fs.createSnapshot(snapshottable, "snap1");
> SnapshotDiffReport report =
> fs.getSnapshotDiffReport(snapshottable, "snap1", "");
> Assert.assertEquals(0, report.getDiffList().size());
> report =
> fs.getSnapshotDiffReport(snapshottable, "snap1", "");
> System.out.println(report);
> Assert.assertEquals(0, report.getDiffList().size());
> fs.setSafeMode(SafeModeAction.SAFEMODE_ENTER);
> fs.saveNamespace();
> fs.setSafeMode(SafeModeAction.SAFEMODE_LEAVE);
> cluster.restartNameNode(true);
> report =
> fs.getSnapshotDiffReport(snapshottable, "snap1", "");
> Assert.assertEquals(0, report.getDiffList().size());
> }{code}
> {code:java}
> Pre Restart:
> Difference between snapshot snap1 and current directory under directory
> /zones:
> Post Restart:
> Difference between snapshot snap1 and current directory under directory
> /zones:
> M .{code}
> The side effect of this behavior is : distcp with snapshot diff would fail
> with below error complaining that target cluster has some data changed .
> {code:java}
> WARN tools.DistCp: The target has been modified since snapshot x
> {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-16187) SnapshotDiff behaviour with Xattrs and Acls is not consistent across NN restarts with checkpointing
[
https://issues.apache.org/jira/browse/HDFS-16187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wei-Chiu Chuang updated HDFS-16187:
---
Fix Version/s: 3.2.3
> SnapshotDiff behaviour with Xattrs and Acls is not consistent across NN
> restarts with checkpointing
> ---
>
> Key: HDFS-16187
> URL: https://issues.apache.org/jira/browse/HDFS-16187
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: snapshots
>Reporter: Srinivasu Majeti
>Assignee: Shashikant Banerjee
>Priority: Major
> Labels: pull-request-available
> Fix For: 3.4.0, 3.2.3, 3.3.2
>
> Time Spent: 5h
> Remaining Estimate: 0h
>
> The below test shows the snapshot diff between across snapshots is not
> consistent with Xattr(EZ here settinh the Xattr) across NN restarts with
> checkpointed FsImage.
> {code:java}
> @Test
> public void testEncryptionZonesWithSnapshots() throws Exception {
> final Path snapshottable = new Path("/zones");
> fsWrapper.mkdir(snapshottable, FsPermission.getDirDefault(),
> true);
> dfsAdmin.allowSnapshot(snapshottable);
> dfsAdmin.createEncryptionZone(snapshottable, TEST_KEY, NO_TRASH);
> fs.createSnapshot(snapshottable, "snap1");
> SnapshotDiffReport report =
> fs.getSnapshotDiffReport(snapshottable, "snap1", "");
> Assert.assertEquals(0, report.getDiffList().size());
> report =
> fs.getSnapshotDiffReport(snapshottable, "snap1", "");
> System.out.println(report);
> Assert.assertEquals(0, report.getDiffList().size());
> fs.setSafeMode(SafeModeAction.SAFEMODE_ENTER);
> fs.saveNamespace();
> fs.setSafeMode(SafeModeAction.SAFEMODE_LEAVE);
> cluster.restartNameNode(true);
> report =
> fs.getSnapshotDiffReport(snapshottable, "snap1", "");
> Assert.assertEquals(0, report.getDiffList().size());
> }{code}
> {code:java}
> Pre Restart:
> Difference between snapshot snap1 and current directory under directory
> /zones:
> Post Restart:
> Difference between snapshot snap1 and current directory under directory
> /zones:
> M .{code}
> The side effect of this behavior is : distcp with snapshot diff would fail
> with below error complaining that target cluster has some data changed .
> {code:java}
> WARN tools.DistCp: The target has been modified since snapshot x
> {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-16187) SnapshotDiff behaviour with Xattrs and Acls is not consistent across NN restarts with checkpointing
[
https://issues.apache.org/jira/browse/HDFS-16187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wei-Chiu Chuang updated HDFS-16187:
---
Fix Version/s: 3.4.0
(was: 1.3.0)
> SnapshotDiff behaviour with Xattrs and Acls is not consistent across NN
> restarts with checkpointing
> ---
>
> Key: HDFS-16187
> URL: https://issues.apache.org/jira/browse/HDFS-16187
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: snapshots
>Reporter: Srinivasu Majeti
>Assignee: Shashikant Banerjee
>Priority: Major
> Labels: pull-request-available
> Fix For: 3.4.0
>
> Time Spent: 4h 10m
> Remaining Estimate: 0h
>
> The below test shows the snapshot diff between across snapshots is not
> consistent with Xattr(EZ here settinh the Xattr) across NN restarts with
> checkpointed FsImage.
> {code:java}
> @Test
> public void testEncryptionZonesWithSnapshots() throws Exception {
> final Path snapshottable = new Path("/zones");
> fsWrapper.mkdir(snapshottable, FsPermission.getDirDefault(),
> true);
> dfsAdmin.allowSnapshot(snapshottable);
> dfsAdmin.createEncryptionZone(snapshottable, TEST_KEY, NO_TRASH);
> fs.createSnapshot(snapshottable, "snap1");
> SnapshotDiffReport report =
> fs.getSnapshotDiffReport(snapshottable, "snap1", "");
> Assert.assertEquals(0, report.getDiffList().size());
> report =
> fs.getSnapshotDiffReport(snapshottable, "snap1", "");
> System.out.println(report);
> Assert.assertEquals(0, report.getDiffList().size());
> fs.setSafeMode(SafeModeAction.SAFEMODE_ENTER);
> fs.saveNamespace();
> fs.setSafeMode(SafeModeAction.SAFEMODE_LEAVE);
> cluster.restartNameNode(true);
> report =
> fs.getSnapshotDiffReport(snapshottable, "snap1", "");
> Assert.assertEquals(0, report.getDiffList().size());
> }{code}
> {code:java}
> Pre Restart:
> Difference between snapshot snap1 and current directory under directory
> /zones:
> Post Restart:
> Difference between snapshot snap1 and current directory under directory
> /zones:
> M .{code}
> The side effect of this behavior is : distcp with snapshot diff would fail
> with below error complaining that target cluster has some data changed .
> {code:java}
> WARN tools.DistCp: The target has been modified since snapshot x
> {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-16187) SnapshotDiff behaviour with Xattrs and Acls is not consistent across NN restarts with checkpointing
[
https://issues.apache.org/jira/browse/HDFS-16187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Srinivasu Majeti updated HDFS-16187:
Description:
The below test shows the snapshot diff between across snapshots is not
consistent with Xattr(EZ here settinh the Xattr) across NN restarts with
checkpointed FsImage.
{code:java}
@Test
public void testEncryptionZonesWithSnapshots() throws Exception {
final Path snapshottable = new Path("/zones");
fsWrapper.mkdir(snapshottable, FsPermission.getDirDefault(),
true);
dfsAdmin.allowSnapshot(snapshottable);
dfsAdmin.createEncryptionZone(snapshottable, TEST_KEY, NO_TRASH);
fs.createSnapshot(snapshottable, "snap1");
SnapshotDiffReport report =
fs.getSnapshotDiffReport(snapshottable, "snap1", "");
Assert.assertEquals(0, report.getDiffList().size());
report =
fs.getSnapshotDiffReport(snapshottable, "snap1", "");
System.out.println(report);
Assert.assertEquals(0, report.getDiffList().size());
fs.setSafeMode(SafeModeAction.SAFEMODE_ENTER);
fs.saveNamespace();
fs.setSafeMode(SafeModeAction.SAFEMODE_LEAVE);
cluster.restartNameNode(true);
report =
fs.getSnapshotDiffReport(snapshottable, "snap1", "");
Assert.assertEquals(0, report.getDiffList().size());
}{code}
{code:java}
Pre Restart:
Difference between snapshot snap1 and current directory under directory /zones:
Post Restart:
Difference between snapshot snap1 and current directory under directory /zones:
M .{code}
The side effect of this behavior is : distcp with snapshot diff would fail with
below error complaining that target cluster has some data changed .
{code:java}
WARN tools.DistCp: The target has been modified since snapshot x
{code}
was:
The below test shows the snapshot diff between across snapshots is not
consistent with Xattr(EZ here settinh the Xattr) across NN restarts with
checkpointed FsImage.
{code:java}
@Test
public void testEncryptionZonesWithSnapshots() throws Exception {
final Path snapshottable = new Path("/zones");
fsWrapper.mkdir(snapshottable, FsPermission.getDirDefault(),
true);
dfsAdmin.allowSnapshot(snapshottable);
dfsAdmin.createEncryptionZone(snapshottable, TEST_KEY, NO_TRASH);
fs.createSnapshot(snapshottable, "snap1");
SnapshotDiffReport report =
fs.getSnapshotDiffReport(snapshottable, "snap1", "");
Assert.assertEquals(0, report.getDiffList().size());
report =
fs.getSnapshotDiffReport(snapshottable, "snap1", "");
System.out.println(report);
Assert.assertEquals(0, report.getDiffList().size());
fs.setSafeMode(SafeModeAction.SAFEMODE_ENTER);
fs.saveNamespace();
fs.setSafeMode(SafeModeAction.SAFEMODE_LEAVE);
cluster.restartNameNode(true);
report =
fs.getSnapshotDiffReport(snapshottable, "snap1", "");
Assert.assertEquals(0, report.getDiffList().size());
}{code}
{code:java}
Pre Restart:
Difference between snapshot snap1 and current directory under directory /zones:
Post Restart:
Difference between snapshot snap1 and current directory under directory /zones:
M .{code}
> SnapshotDiff behaviour with Xattrs and Acls is not consistent across NN
> restarts with checkpointing
> ---
>
> Key: HDFS-16187
> URL: https://issues.apache.org/jira/browse/HDFS-16187
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: snapshots
>Reporter: Srinivasu Majeti
>Assignee: Shashikant Banerjee
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h
> Remaining Estimate: 0h
>
> The below test shows the snapshot diff between across snapshots is not
> consistent with Xattr(EZ here settinh the Xattr) across NN restarts with
> checkpointed FsImage.
> {code:java}
> @Test
> public void testEncryptionZonesWithSnapshots() throws Exception {
> final Path snapshottable = new Path("/zones");
> fsWrapper.mkdir(snapshottable, FsPermission.getDirDefault(),
> true);
> dfsAdmin.allowSnapshot(snapshottable);
> dfsAdmin.createEncryptionZone(snapshottable, TEST_KEY, NO_TRASH);
> fs.createSnapshot(snapshottable, "snap1");
> SnapshotDiffReport report =
> fs.getSnapshotDiffReport(snapshottable, "snap1", "");
> Assert.assertEquals(0, report.getDiffList().size());
> report =
> fs.getSnapshotDiffReport(snapshottable, "snap1", "");
> System.out.println(report);
> Assert.assertEquals(0, report.getDiffList().size());
> fs.setSafeMode(SafeModeAction.SAFEMODE_ENTER);
> fs.saveNamespace();
> fs.setSafeMode(SafeModeAction.SAFEMODE_LEAVE);
> cluster.restartNameNode(true);
> report =
> fs.getSnapshotDiffReport(snapshottable, "snap1", "");
> Assert.assertEquals(0, report.getDiffList().size());
> }{code}
> {code:java}
> Pre Restart:
> Difference between snapshot snap1 and
[jira] [Updated] (HDFS-16187) SnapshotDiff behaviour with Xattrs and Acls is not consistent across NN restarts with checkpointing
[
https://issues.apache.org/jira/browse/HDFS-16187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HDFS-16187:
--
Labels: pull-request-available (was: )
> SnapshotDiff behaviour with Xattrs and Acls is not consistent across NN
> restarts with checkpointing
> ---
>
> Key: HDFS-16187
> URL: https://issues.apache.org/jira/browse/HDFS-16187
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: snapshots
>Reporter: Srinivasu Majeti
>Assignee: Shashikant Banerjee
>Priority: Major
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> The below test shows the snapshot diff between across snapshots is not
> consistent with Xattr(EZ here settinh the Xattr) across NN restarts with
> checkpointed FsImage.
> {code:java}
> @Test
> public void testEncryptionZonesWithSnapshots() throws Exception {
> final Path snapshottable = new Path("/zones");
> fsWrapper.mkdir(snapshottable, FsPermission.getDirDefault(),
> true);
> dfsAdmin.allowSnapshot(snapshottable);
> dfsAdmin.createEncryptionZone(snapshottable, TEST_KEY, NO_TRASH);
> fs.createSnapshot(snapshottable, "snap1");
> SnapshotDiffReport report =
> fs.getSnapshotDiffReport(snapshottable, "snap1", "");
> Assert.assertEquals(0, report.getDiffList().size());
> report =
> fs.getSnapshotDiffReport(snapshottable, "snap1", "");
> System.out.println(report);
> Assert.assertEquals(0, report.getDiffList().size());
> fs.setSafeMode(SafeModeAction.SAFEMODE_ENTER);
> fs.saveNamespace();
> fs.setSafeMode(SafeModeAction.SAFEMODE_LEAVE);
> cluster.restartNameNode(true);
> report =
> fs.getSnapshotDiffReport(snapshottable, "snap1", "");
> Assert.assertEquals(0, report.getDiffList().size());
> }{code}
> {code:java}
> Pre Restart:
> Difference between snapshot snap1 and current directory under directory
> /zones:
> Post Restart:
> Difference between snapshot snap1 and current directory under directory
> /zones:
> M .{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
