[
https://issues.apache.org/jira/browse/HDFS-15079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17008298#comment-17008298
]
Xiaoqiao He commented on HDFS-15079:
------------------------------------
Thanks [~ferhui] for your works and ping. I believe this issue is clear and
obvious based on above discussion.
About the solution, it seems they are same issue with data locality and we will
turn back to discussion again. Due to we have to involve other modules to
change and both solutions have its own pros and cons, we do not reach the final
conclusion up to now. I would like to list both solutions here.
a. limited open interface get/set ClientId and CallId to some super user.
b. set ClientId/CallId to CallerContext and parse them at NameNode side. The
detailed Pros and Cons have stated at HDFS-13248, FYI.
In my own opinion, I prefer to solution a + strict access control. pending some
other more suggestions?
> RBF: Client maybe get an unexpected result with network anomaly
> ----------------------------------------------------------------
>
> Key: HDFS-15079
> URL: https://issues.apache.org/jira/browse/HDFS-15079
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: rbf
> Affects Versions: 3.3.0
> Reporter: Fei Hui
> Priority: Critical
> Attachments: HDFS-15079.001.patch, UnexpectedOverWriteUT.patch
>
>
> I find there is a critical problem on RBF, HDFS-15078 can resolve it on some
> Scenarios, but i have no idea about the overall resolution.
> The problem is that
> Client with RBF(r0, r1) create a file HDFS file via r0, it gets Exception and
> failovers to r1
> r0 has been send create rpc to namenode(1st create)
> Client create a HDFS file via r1(2nd create)
> Client writes the HDFS file and close it finally(3rd close)
> Maybe namenode receiving the rpc in order as follow
> 2nd create
> 3rd close
> 1st create
> And overwrite is true by default, this would make the file had been written
> an empty file. This is an critical problem
> We had encountered this problem. There are many hive and spark jobs running
> on our cluster, sometimes it occurs
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
