[
https://issues.apache.org/jira/browse/HDFS-16187?focusedWorklogId=648396&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-648396
]
ASF GitHub Bot logged work on HDFS-16187:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 09/Sep/21 05:43
Start Date: 09/Sep/21 05:43
Worklog Time Spent: 10m
Work Description: bshashikant edited a comment on pull request #3340:
URL: https://github.com/apache/hadoop/pull/3340#issuecomment-915780582
> @bshashikant , thank you for the patch.
>
> I see that the patch works by converting logic from reference-equals to
value-equals. In `AclStorage`, we maintain a reference-counted mapping of every
immutable instance of every `AclFeature` currently in use by the namesystem.
Inodes with identical ACL entries all point to the same `AclFeature`, so
reference equals should work fine. This was an intentional choice of the ACL
design to lower memory footprint and provide an inexpensive equality operation.
>
> Xattrs do not use this same flyweight pattern though, so I can see why
deep equals would be necessary there. I wonder if this bug really only happens
for xattrs and not ACLs? If the bug happens for ACLs, then that might mean
there is really something wrong with reference counting in `AclStorage`,
causing duplicate instances for the same ACL entries.
>
> Also, if value equals is necessary, then there is similar logic in
`INodeFile` and `INodeFileAttributes`.
Thanks @cnauroth for the explanation. I tested the acl feature and it seems
the issue only seem to exist with Xattr feature not acl itself as you
suggested. I have modified the patch accordingly. Thank you very much for the
details.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
-------------------
Worklog Id: (was: 648396)
Time Spent: 1h 40m (was: 1.5h)
> SnapshotDiff behaviour with Xattrs and Acls is not consistent across NN
> restarts with checkpointing
> ---------------------------------------------------------------------------------------------------
>
> Key: HDFS-16187
> URL: https://issues.apache.org/jira/browse/HDFS-16187
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: snapshots
> Reporter: Srinivasu Majeti
> Assignee: Shashikant Banerjee
> Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
> The below test shows the snapshot diff between across snapshots is not
> consistent with Xattr(EZ here settinh the Xattr) across NN restarts with
> checkpointed FsImage.
> {code:java}
> @Test
> public void testEncryptionZonesWithSnapshots() throws Exception {
> final Path snapshottable = new Path("/zones");
> fsWrapper.mkdir(snapshottable, FsPermission.getDirDefault(),
> true);
> dfsAdmin.allowSnapshot(snapshottable);
> dfsAdmin.createEncryptionZone(snapshottable, TEST_KEY, NO_TRASH);
> fs.createSnapshot(snapshottable, "snap1");
> SnapshotDiffReport report =
> fs.getSnapshotDiffReport(snapshottable, "snap1", "");
> Assert.assertEquals(0, report.getDiffList().size());
> report =
> fs.getSnapshotDiffReport(snapshottable, "snap1", "");
> System.out.println(report);
> Assert.assertEquals(0, report.getDiffList().size());
> fs.setSafeMode(SafeModeAction.SAFEMODE_ENTER);
> fs.saveNamespace();
> fs.setSafeMode(SafeModeAction.SAFEMODE_LEAVE);
> cluster.restartNameNode(true);
> report =
> fs.getSnapshotDiffReport(snapshottable, "snap1", "");
> Assert.assertEquals(0, report.getDiffList().size());
> }{code}
> {code:java}
> Pre Restart:
> Difference between snapshot snap1 and current directory under directory
> /zones:
> Post Restart:
> Difference between snapshot snap1 and current directory under directory
> /zones:
> M .{code}
> The side effect of this behavior is : distcp with snapshot diff would fail
> with below error complaining that target cluster has some data changed .
> {code:java}
> WARN tools.DistCp: The target has been modified since snapshot xxxxx
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
