[
https://issues.apache.org/jira/browse/HDFS-15079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17561642#comment-17561642
]
Ayush Saxena commented on HDFS-15079:
-------------------------------------
I think RetryCache of Namenode solves this problem, and that uses both callId
and clientId to fetch an entry, clientId we are still passing correct, if we
pass the callId also from the actual client, this should get sorted.
HDFS-13248 added clientIp details, I feel we can do similarly for callId as
well and things should be sorted post that.
> RBF: Client maybe get an unexpected result with network anomaly
> ----------------------------------------------------------------
>
> Key: HDFS-15079
> URL: https://issues.apache.org/jira/browse/HDFS-15079
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: rbf
> Affects Versions: 3.3.0
> Reporter: Hui Fei
> Priority: Critical
> Attachments: HDFS-15079.001.patch, HDFS-15079.002.patch,
> UnexpectedOverWriteUT.patch
>
>
> I find there is a critical problem on RBF, HDFS-15078 can resolve it on some
> Scenarios, but i have no idea about the overall resolution.
> The problem is that
> Client with RBF(r0, r1) create a file HDFS file via r0, it gets Exception and
> failovers to r1
> r0 has been send create rpc to namenode(1st create)
> Client create a HDFS file via r1(2nd create)
> Client writes the HDFS file and close it finally(3rd close)
> Maybe namenode receiving the rpc in order as follow
> 2nd create
> 3rd close
> 1st create
> And overwrite is true by default, this would make the file had been written
> an empty file. This is an critical problem
> We had encountered this problem. There are many hive and spark jobs running
> on our cluster, sometimes it occurs
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
