[
https://issues.apache.org/jira/browse/HDFS-941?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12866680#action_12866680
]
sam rash commented on HDFS-941:
-------------------------------
two other comments:
1. the number of sockets per address is limited, but not the number of
addresses. This may in practice not be a problem, but the cache can in theory
grow very large
2. the usedmap seems like a good place for a memory/object leak: I can take a
socket and never return it (why again, I vote for getting rid of this data
structure period--as far as a cache is concerned, an entry that someone else
owns shouldn't even be there). otherwise, you've got to periodically clean this
map up as well. Seems like it's only used for stats which I think you can do
w/o actually keeping a hash of used sockets.
> Datanode xceiver protocol should allow reuse of a connection
> ------------------------------------------------------------
>
> Key: HDFS-941
> URL: https://issues.apache.org/jira/browse/HDFS-941
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: data-node, hdfs client
> Affects Versions: 0.22.0
> Reporter: Todd Lipcon
> Assignee: bc Wong
> Attachments: HDFS-941-1.patch, HDFS-941-2.patch, HDFS-941-3.patch,
> HDFS-941-3.patch, HDFS-941-4.patch
>
>
> Right now each connection into the datanode xceiver only processes one
> operation.
> In the case that an operation leaves the stream in a well-defined state (eg a
> client reads to the end of a block successfully) the same connection could be
> reused for a second operation. This should improve random read performance
> significantly.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
