[jira] [Commented] (HDFS-3063) NameNode should validate all coming file path
[
https://issues.apache.org/jira/browse/HDFS-3063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13226748#comment-13226748
]
Denny Ye commented on HDFS-3063:
Thank you, Daryn. We have same concerns about the maintainability of NameNode.
It's better to encapsulate all the validation for each interface method at
NameNode using common method. Another problem of this case is there should be
similar validation for all coming methods.
NameNode should validate all coming file path
-
Key: HDFS-3063
URL: https://issues.apache.org/jira/browse/HDFS-3063
Project: Hadoop HDFS
Issue Type: Improvement
Components: name-node
Affects Versions: 0.20.205.0
Reporter: Denny Ye
Priority: Minor
Labels: namenode
Attachments: HDFS-3063.patch
NameNode provides RPC service for not only DFS client but also user defined
program. A common case we always met is that user transfers file path
prefixed with HDFS protocol(hdfs://{namenode:{port}}/{folder}/{file}).
NameNode cannot map node meta-data with this path and always throw NPE. In
user client, we only see the NullPointerException, no other tips for which
step it occurs.
Also, NameNode should validate all coming file path with regular format.
One exception I met:
Exception in thread main org.apache.hadoop.ipc.RemoteException:
java.io.IOException: java.lang.NullPointerException
at
org.apache.hadoop.hdfs.server.namenode.INode.getPathComponents(INode.java:334)
at
org.apache.hadoop.hdfs.server.namenode.INode.getPathComponents(INode.java:329)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3063) NameNode should validate all coming file path
[
https://issues.apache.org/jira/browse/HDFS-3063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13225850#comment-13225850
]
Denny Ye commented on HDFS-3063:
hi Daryn, in case we use common place to path validation, does there need hook
on each method that client can be invoke? The change looks that with validation
of SafeMode. In my opinion, a possible place is RPC Server before reflection
invocation, but it's terrible for RPC independence
NameNode should validate all coming file path
-
Key: HDFS-3063
URL: https://issues.apache.org/jira/browse/HDFS-3063
Project: Hadoop HDFS
Issue Type: Improvement
Components: name-node
Affects Versions: 0.20.205.0
Reporter: Denny Ye
Priority: Minor
Labels: namenode
Attachments: HDFS-3063.patch
NameNode provides RPC service for not only DFS client but also user defined
program. A common case we always met is that user transfers file path
prefixed with HDFS protocol(hdfs://{namenode:{port}}/{folder}/{file}).
NameNode cannot map node meta-data with this path and always throw NPE. In
user client, we only see the NullPointerException, no other tips for which
step it occurs.
Also, NameNode should validate all coming file path with regular format.
One exception I met:
Exception in thread main org.apache.hadoop.ipc.RemoteException:
java.io.IOException: java.lang.NullPointerException
at
org.apache.hadoop.hdfs.server.namenode.INode.getPathComponents(INode.java:334)
at
org.apache.hadoop.hdfs.server.namenode.INode.getPathComponents(INode.java:329)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2760) HDFS notification
[ https://issues.apache.org/jira/browse/HDFS-2760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13181328#comment-13181328 ] Denny Ye commented on HDFS-2760: hi Harsh. I want the HDFS notifing file/folder/block event(create path, block allocation, ...) to client for more purposes. NameNode RPC cannot react with the method invocation on NameNode object HDFS notification -- Key: HDFS-2760 URL: https://issues.apache.org/jira/browse/HDFS-2760 Project: Hadoop HDFS Issue Type: New Feature Affects Versions: 0.20.2 Reporter: Denny Ye Priority: Minor Labels: HDFS Client would like to receive the interested namespace operation at HDFS, likes 'rename', 'block allocation' and so on. NameNode should support the changing notification to client with appropriate way. In my opinion, I suggest the notification should be applied between NameNode RPC server and actual NameNode object (it's proxy for NameNode object). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2760) HDFS notification
[ https://issues.apache.org/jira/browse/HDFS-2760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13181344#comment-13181344 ] Denny Ye commented on HDFS-2760: I took this demo in local. NameNode proxy object take the role for actual NameNode at NN RPC server. It only record and transfer the method invocation request to actual NameNode object, and does not need additional RPC call. You said right, but it's not a good solution for all clients that they need namespace notification and handle log parser respectively. Likewise, same function appear at Linux kernel. HDFS notification -- Key: HDFS-2760 URL: https://issues.apache.org/jira/browse/HDFS-2760 Project: Hadoop HDFS Issue Type: New Feature Affects Versions: 0.20.2 Reporter: Denny Ye Priority: Minor Labels: HDFS Client would like to receive the interested namespace operation at HDFS, likes 'rename', 'block allocation' and so on. NameNode should support the changing notification to client with appropriate way. In my opinion, I suggest the notification should be applied between NameNode RPC server and actual NameNode object (it's proxy for NameNode object). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-1335) HDFS side of HADOOP-6904: first step towards inter-version communications between dfs client and NameNode
[ https://issues.apache.org/jira/browse/HDFS-1335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13155761#comment-13155761 ] Denny Ye commented on HDFS-1335: Does Hadoop-0.23 contains this patch? HDFS side of HADOOP-6904: first step towards inter-version communications between dfs client and NameNode - Key: HDFS-1335 URL: https://issues.apache.org/jira/browse/HDFS-1335 Project: Hadoop HDFS Issue Type: Improvement Components: hdfs client, name-node Affects Versions: 0.22.0 Reporter: Hairong Kuang Assignee: Hairong Kuang Fix For: 0.23.0 Attachments: hdfsRPC.patch, hdfsRpcVersion.patch The idea is that for getProtocolVersion, NameNode checks if the client and server versions are compatible if the server version is greater than the client version. If no, throws a VersionIncompatible exception; otherwise, returns the server version. On the dfs client side, when creating a NameNode proxy, catches the VersionMismatch exception and then checks if the client version and the server version are compatible if the client version is greater than the server version. If not compatible, throws exception VersionIncomptible; otherwise, records the server version and continues. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
