[jira] [Commented] (HDFS-6371) In HA setup, the standby NN should redirect WebHDFS write requests to the active NN
[ https://issues.apache.org/jira/browse/HDFS-6371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13996394#comment-13996394 ] Daryn Sharp commented on HDFS-6371: --- Agreed on client side retries. With IP-based failover, a standby NN that attempts to redirect to the other NN will create a redirect loop to itself until the IP failover to the active occurs. > In HA setup, the standby NN should redirect WebHDFS write requests to the > active NN > --- > > Key: HDFS-6371 > URL: https://issues.apache.org/jira/browse/HDFS-6371 > Project: Hadoop HDFS > Issue Type: Improvement > Components: namenode, webhdfs >Reporter: Tsz Wo Nicholas Sze >Assignee: Tsz Wo Nicholas Sze > > The current WebHDFS implementation in namenode does not check its HA state -- > it does the same thing no matter it is active or standby. > Suppose a http client talk to the standby NN via WebHDFS. For the read > operations, there is no problem. For the write operations, if the operation > requires http redirect (e.g. creating a file), it will work since the standby > NN will also redirect the client to a DN. When the client connect to the DN, > the DN will fulfill the request with the active NN. However, for the write > operations not requiring http redirect (e.g. mkdir), the operation will fail > with StandbyException since it will be executed with the standby NN. > There are two solutions: > # The http client could catch StandbyException and then retries with the > other NN in this case. > # The standby NN redirects the request to the active NN. > The second solution seems better since the client does not need to know both > active NN and standby NN. > Note that WebHdfsFileSystem is already able to handle HA failover. The JIRA > is for other http clients. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HDFS-6371) In HA setup, the standby NN should redirect WebHDFS write requests to the active NN
[ https://issues.apache.org/jira/browse/HDFS-6371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13995293#comment-13995293 ] Haohui Mai commented on HDFS-6371: -- In HA design the client always takes responsibility of retries instead of the server. In the use case mentioned in HDFS-6334, the client and the sever might have different views on HA, thus it looks to me that retrying on the client-side might be the only viable solution. > In HA setup, the standby NN should redirect WebHDFS write requests to the > active NN > --- > > Key: HDFS-6371 > URL: https://issues.apache.org/jira/browse/HDFS-6371 > Project: Hadoop HDFS > Issue Type: Bug > Components: webhdfs >Reporter: Tsz Wo Nicholas Sze >Assignee: Tsz Wo Nicholas Sze > > The current WebHDFS implementation in namenode does not check its HA state -- > it does the same thing no matter it is active or standby. > Suppose a http client talk to the standby NN via WebHDFS. For the read > operations, there is no problem. For the write operations, if the operation > requires http redirect (e.g. creating a file), it will work since the standby > NN will also redirect the client to a DN. When the client connect to the DN, > the DN will fulfill the request with the active NN. However, for the write > operations not requiring http redirect (e.g. mkdir), the operation will fail > with StandbyException since it will be executed with the standby NN. > There are two solutions: > # The http client could catch StandbyException and then retries with the > other NN in this case. > # The standby NN redirects the request to the active NN. > The second solution seems better since the client does not need to know both > active NN and standby NN. > Note that WebHdfsFileSystem is already able to handle HA failover. The JIRA > is for other http clients. -- This message was sent by Atlassian JIRA (v6.2#6252)