[jira] [Commented] (HDFS-3063) NameNode should validate all coming file path
[
https://issues.apache.org/jira/browse/HDFS-3063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13226748#comment-13226748
]
Denny Ye commented on HDFS-3063:
Thank you, Daryn. We have same concerns about the maintainability of NameNode.
It's better to encapsulate all the validation for each interface method at
NameNode using common method. Another problem of this case is there should be
similar validation for all coming methods.
> NameNode should validate all coming file path
> -
>
> Key: HDFS-3063
> URL: https://issues.apache.org/jira/browse/HDFS-3063
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: name-node
>Affects Versions: 0.20.205.0
>Reporter: Denny Ye
>Priority: Minor
> Labels: namenode
> Attachments: HDFS-3063.patch
>
>
> NameNode provides RPC service for not only DFS client but also user defined
> program. A common case we always met is that user transfers file path
> prefixed with HDFS protocol("hdfs://{namenode:{port}}/{folder}/{file}").
> NameNode cannot map node meta-data with this path and always throw NPE. In
> user client, we only see the NullPointerException, no other tips for which
> step it occurs.
> Also, NameNode should validate all coming file path with regular format.
> One exception I met:
> Exception in thread "main" org.apache.hadoop.ipc.RemoteException:
> java.io.IOException: java.lang.NullPointerException
> at
> org.apache.hadoop.hdfs.server.namenode.INode.getPathComponents(INode.java:334)
> at
> org.apache.hadoop.hdfs.server.namenode.INode.getPathComponents(INode.java:329)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3063) NameNode should validate all coming file path
[
https://issues.apache.org/jira/browse/HDFS-3063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13225850#comment-13225850
]
Denny Ye commented on HDFS-3063:
hi Daryn, in case we use common place to path validation, does there need hook
on each method that client can be invoke? The change looks that with validation
of SafeMode. In my opinion, a possible place is RPC Server before reflection
invocation, but it's terrible for RPC independence
> NameNode should validate all coming file path
> -
>
> Key: HDFS-3063
> URL: https://issues.apache.org/jira/browse/HDFS-3063
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: name-node
>Affects Versions: 0.20.205.0
>Reporter: Denny Ye
>Priority: Minor
> Labels: namenode
> Attachments: HDFS-3063.patch
>
>
> NameNode provides RPC service for not only DFS client but also user defined
> program. A common case we always met is that user transfers file path
> prefixed with HDFS protocol("hdfs://{namenode:{port}}/{folder}/{file}").
> NameNode cannot map node meta-data with this path and always throw NPE. In
> user client, we only see the NullPointerException, no other tips for which
> step it occurs.
> Also, NameNode should validate all coming file path with regular format.
> One exception I met:
> Exception in thread "main" org.apache.hadoop.ipc.RemoteException:
> java.io.IOException: java.lang.NullPointerException
> at
> org.apache.hadoop.hdfs.server.namenode.INode.getPathComponents(INode.java:334)
> at
> org.apache.hadoop.hdfs.server.namenode.INode.getPathComponents(INode.java:329)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2760) HDFS notification
[ https://issues.apache.org/jira/browse/HDFS-2760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13181344#comment-13181344 ] Denny Ye commented on HDFS-2760: I took this demo in local. NameNode proxy object take the role for actual NameNode at NN RPC server. It only record and transfer the method invocation request to actual NameNode object, and does not need additional RPC call. You said right, but it's not a good solution for all clients that they need namespace notification and handle log parser respectively. Likewise, same function appear at Linux kernel. > HDFS notification > -- > > Key: HDFS-2760 > URL: https://issues.apache.org/jira/browse/HDFS-2760 > Project: Hadoop HDFS > Issue Type: New Feature >Affects Versions: 0.20.2 >Reporter: Denny Ye >Priority: Minor > Labels: HDFS > > Client would like to receive the interested namespace operation at HDFS, > likes 'rename', 'block allocation' and so on. NameNode should support the > changing notification to client with appropriate way. > In my opinion, I suggest the notification should be applied between NameNode > RPC server and actual NameNode object (it's proxy for NameNode object). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2760) HDFS notification
[ https://issues.apache.org/jira/browse/HDFS-2760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13181328#comment-13181328 ] Denny Ye commented on HDFS-2760: hi Harsh. I want the HDFS notifing file/folder/block event(create path, block allocation, ...) to client for more purposes. NameNode RPC cannot react with the method invocation on NameNode object > HDFS notification > -- > > Key: HDFS-2760 > URL: https://issues.apache.org/jira/browse/HDFS-2760 > Project: Hadoop HDFS > Issue Type: New Feature >Affects Versions: 0.20.2 >Reporter: Denny Ye >Priority: Minor > Labels: HDFS > > Client would like to receive the interested namespace operation at HDFS, > likes 'rename', 'block allocation' and so on. NameNode should support the > changing notification to client with appropriate way. > In my opinion, I suggest the notification should be applied between NameNode > RPC server and actual NameNode object (it's proxy for NameNode object). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-1335) HDFS side of HADOOP-6904: first step towards inter-version communications between dfs client and NameNode
[ https://issues.apache.org/jira/browse/HDFS-1335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13155761#comment-13155761 ] Denny Ye commented on HDFS-1335: Does Hadoop-0.23 contains this patch? > HDFS side of HADOOP-6904: first step towards inter-version communications > between dfs client and NameNode > - > > Key: HDFS-1335 > URL: https://issues.apache.org/jira/browse/HDFS-1335 > Project: Hadoop HDFS > Issue Type: Improvement > Components: hdfs client, name-node >Affects Versions: 0.22.0 >Reporter: Hairong Kuang >Assignee: Hairong Kuang > Fix For: 0.23.0 > > Attachments: hdfsRPC.patch, hdfsRpcVersion.patch > > > The idea is that for getProtocolVersion, NameNode checks if the client and > server versions are compatible if the server version is greater than the > client version. If no, throws a VersionIncompatible exception; otherwise, > returns the server version. > On the dfs client side, when creating a NameNode proxy, catches the > VersionMismatch exception and then checks if the client version and the > server version are compatible if the client version is greater than the > server version. If not compatible, throws exception VersionIncomptible; > otherwise, records the server version and continues. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
