[
https://issues.apache.org/jira/browse/HDFS-15078?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17002621#comment-17002621
]
Xiaoqiao He commented on HDFS-15078:
------------------------------------
{quote}I find there is a critical problem on RBF, this issue can resolve it on
some Scenarios, but i have no idea about the overall resolution. Plan to file a
new jira to track it.{quote}
Thanks [~ferhui], this is pretty interesting observation, it seems to a
critical issue, we should file another JIRA to trace in my opinion. Would you
like to share some more information, is it batch processing or streaming
scenario? In our case, all requests from offline batch processing + Adhoc query
(Spark/MapReduce/Presto), and we do not meet this issue.
> RBF: Should check connection channel before sending rpc to namenode
> -------------------------------------------------------------------
>
> Key: HDFS-15078
> URL: https://issues.apache.org/jira/browse/HDFS-15078
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: rbf
> Affects Versions: 3.3.0
> Reporter: Fei Hui
> Assignee: Fei Hui
> Priority: Major
> Attachments: HDFS-15078.001.patch, HDFS-15078.002.patch
>
>
> dfsrouter logs show that
> {quote}
> 2019-12-20 04:11:26,724 WARN org.apache.hadoop.ipc.Server: IPC Server handler
> 6400 on 8888, call org.apache.hadoop.hdfs.protocol.ClientProtocol.create from
> 10.83.164.11:56908 Call#2 Retry#0: output error
> 2019-12-20 04:11:26,724 INFO org.apache.hadoop.ipc.Server: IPC Server handler
> 125 on 8888 caught an exception
> java.nio.channels.ClosedChannelException
> at
> sun.nio.ch.SocketChannelImpl.ensureWriteOpen(SocketChannelImpl.java:270)
> at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:461)
> at org.apache.hadoop.ipc.Server.channelWrite(Server.java:2731)
> at org.apache.hadoop.ipc.Server.access$2100(Server.java:134)
> at
> org.apache.hadoop.ipc.Server$Responder.processResponse(Server.java:1089)
> at org.apache.hadoop.ipc.Server$Responder.doRespond(Server.java:1161)
> at
> org.apache.hadoop.ipc.Server$Connection.sendResponse(Server.java:2109)
> at
> org.apache.hadoop.ipc.Server$Connection.access$400(Server.java:1229)
> at org.apache.hadoop.ipc.Server$Call.sendResponse(Server.java:631)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2245)
> {quote}
> Maybe checking connection between client and router is better before
> sendingrpc to namenode
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
