[ https://issues.apache.org/jira/browse/HDFS-15079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17003148#comment-17003148 ]
Ayush Saxena commented on HDFS-15079: ------------------------------------- Thanx [~ferhui] for the report, The issue is at Router side, and the proposed solution gets changes both at NN and client side, I don't think the proposed solution would get much of agreement, So, I didn't dig in much about this solution. I need to check how actually this is happening, in general case, if it was a namenode-client scenario, I guess the client tries 10 times({{ipc.client.connect.max.retries}}) and then failover to the next. May be this isn't happening? I think I am missing some context. It would be good you can put up a UT to repro this. May be then it could be easy for us to find a solution. > RBF: Client maybe get an unexpected result with network anomaly > ---------------------------------------------------------------- > > Key: HDFS-15079 > URL: https://issues.apache.org/jira/browse/HDFS-15079 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: rbf > Affects Versions: 3.3.0 > Reporter: Fei Hui > Priority: Critical > > I find there is a critical problem on RBF, HDFS-15078 can resolve it on some > Scenarios, but i have no idea about the overall resolution. > The problem is that > Client with RBF(r0, r1) create a file HDFS file via r0, it gets Exception and > failovers to r1 > r0 has been send create rpc to namenode(1st create) > Client create a HDFS file via r1(2nd create) > Client writes the HDFS file and close it finally(3rd close) > Maybe namenode receiving the rpc in order as follow > 2nd create > 3rd close > 1st create > And overwrite is true by default, this would make the file had been written > an empty file. This is an critical problem > We had encountered this problem. There are many hive and spark jobs running > on our cluster, sometimes it occurs -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org