[
https://issues.apache.org/jira/browse/HDFS-15446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17149470#comment-17149470
]
Ayush Saxena commented on HDFS-15446:
-------------------------------------
Hey [~sodonnell]
I think for the edit log purpose only this much shall work :
{code:java}
public INodesInPath unprotectedResolvePath(String src)
throws FileNotFoundException {
byte[][] components = INode.getPathComponents(src);
boolean isRaw = isReservedRawName(components);
components = resolveComponents(components, this);
return INodesInPath.resolve(rootDir, components, isRaw);
}
{code}
The other client side logic checks {{isCreate}} and stuffs aren't required, we
don't pass it here as well so no need to check. With this you might not be able
to refactor and reuse this method above. But I am ok having a separate method.
Though the impacts are minor, but edits are replayed in huge numbers so minor
stuffs magnifies, Regarding the Snapshot operation. The snapshot operations are
there in abundance in many cases, One of which I have is: if the cluster is
part of Federation setup and there are tools running for load balancing between
the clusters. Those tools relies too much on snapshots.
So, I feel we can try this out, If you don't have any issues. :)
> CreateSnapshotOp fails during edit log loading for /.reserved/raw/path with
> error java.io.FileNotFoundException: Directory does not exist:
> /.reserved/raw/path
> ---------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HDFS-15446
> URL: https://issues.apache.org/jira/browse/HDFS-15446
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: hdfs
> Affects Versions: 3.2.0, 3.3.0
> Reporter: Srinivasu Majeti
> Assignee: Stephen O'Donnell
> Priority: Major
> Labels: reserved-word, snapshot
> Attachments: HDFS-15446.001.patch, HDFS-15446.002.patch
>
>
> After allowing snapshot creation for a path say /app-logs , when we try to
> create snapshot on
> /.reserved/raw/app-logs , its successful with snapshot creation but later
> when Standby Namenode is restarted and tries to load the edit record
> OP_CREATE_SNAPSHOT , we see it failing and Standby Namenode shuts down with
> an exception "ava.io.FileNotFoundException: Directory does not exist:
> /.reserved/raw/app-logs" .
> Here are the steps to reproduce :
> {code:java}
> # hdfs dfs -ls /.reserved/raw/
> Found 15 items
> drwxrwxrwt - yarn hadoop 0 2020-06-29 10:27
> /.reserved/raw/app-logs
> drwxr-xr-x - hive hadoop 0 2020-06-29 10:29 /.reserved/raw/prod
> ++++++++++++++
> [root@c3230-node2 ~]# hdfs dfsadmin -allowSnapshot /app-logs
> Allowing snapshot on /app-logs succeeded
> [root@c3230-node2 ~]# hdfs dfsadmin -allowSnapshot /prod
> Allowing snapshot on /prod succeeded
> ++++++++++++++
> # hdfs lsSnapshottableDir
> drwxrwxrwt 0 yarn hadoop 0 2020-06-29 10:27 1 65536 /app-logs
> drwxr-xr-x 0 hive hadoop 0 2020-06-29 10:29 1 65536 /prod
> ++++++++++++++
> [root@c3230-node2 ~]# hdfs dfs -createSnapshot /.reserved/raw/app-logs testSS
> Created snapshot /.reserved/raw/app-logs/.snapshot/testSS
> {code}
> Exception we see in Standby namenode while loading the snapshot creation edit
> record.
> {code:java}
> 2020-06-29 10:33:25,488 ERROR namenode.NameNode (NameNode.java:main(1715)) -
> Failed to start namenode.
> java.io.FileNotFoundException: Directory does not exist:
> /.reserved/raw/app-logs
> at
> org.apache.hadoop.hdfs.server.namenode.INodeDirectory.valueOf(INodeDirectory.java:60)
> at
> org.apache.hadoop.hdfs.server.namenode.snapshot.SnapshotManager.getSnapshottableRoot(SnapshotManager.java:259)
> at
> org.apache.hadoop.hdfs.server.namenode.snapshot.SnapshotManager.createSnapshot(SnapshotManager.java:307)
> at
> org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.applyEditLogOp(FSEditLogLoader.java:772)
> at
> org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadEditRecords(FSEditLogLoader.java:257)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
