[
https://issues.apache.org/jira/browse/HDFS-941?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12847244#action_12847244
]
bc Wong commented on HDFS-941:
------------------------------
Thanks for the review, Todd. I'll add more tests, and look into making a cache
of size > 1.
bq. I think there is a concurrency issue here. Namely, the positional read API
calls through into fetchBlockByteRange, which will use the existing cached
socket, regardless of other concurrent operations. So we may end up with
multiple block readers on the same socket and everything will fall apart.
That should be fine. Each {{SocketCacheEntry}} has a unique {{Socket}}, owned
by its {{BlockReader}}. One of the reuse condition is that the {{BlockReader}}
has finished reading on that {{Socket}} ({{hasConsumedAll()}}). Note that we do
not reuse {{BlockReader}}. So at this point, it should be safe to take the
{{Socket}} away from its previous owner and give it to a new {{BlockReader}}.
I'll add tests for this though.
> Datanode xceiver protocol should allow reuse of a connection
> ------------------------------------------------------------
>
> Key: HDFS-941
> URL: https://issues.apache.org/jira/browse/HDFS-941
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: data-node, hdfs client
> Affects Versions: 0.22.0
> Reporter: Todd Lipcon
> Assignee: bc Wong
> Attachments: HDFS-941-1.patch
>
>
> Right now each connection into the datanode xceiver only processes one
> operation.
> In the case that an operation leaves the stream in a well-defined state (eg a
> client reads to the end of a block successfully) the same connection could be
> reused for a second operation. This should improve random read performance
> significantly.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
