[
https://issues.apache.org/jira/browse/HDFS-941?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12866664#action_12866664
]
sam rash commented on HDFS-941:
-------------------------------
todd: wow, those benchmarks do look impressive! do we have any idea if
standard sequential access gets any benefit?
bc: my point about the cache is you don't have to hard-code it as a static
member of ReaderSocketCache. I don't think it needs to be more generic--it can
be a socket cache. I do think it can be decoupled from BlockReader by getting
rid
of having owner.
why does a 'cache' create sockets? you can avoid the whole owner problem if
you simply let the client ask for a socket, and if there is none, create its
own, use it, and put it in the cache when its done with it (ie, it's usable).
This
should greatly reduce complexity (no need for free + used separately, owner,
etc). It seems like this is mixing up responsibilities of being a socket
factory and a socket cache (possibly why it seems complex to me)
<code>
boolean reusable() {
return ((owner == null || owner.hasConsumedAll()) &&
sock.isConnected() &&
!sock.isInputShutdown() &&
!sock.isOutputShutdown());
}
</code>
will only check socket if you can make this change
> Datanode xceiver protocol should allow reuse of a connection
> ------------------------------------------------------------
>
> Key: HDFS-941
> URL: https://issues.apache.org/jira/browse/HDFS-941
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: data-node, hdfs client
> Affects Versions: 0.22.0
> Reporter: Todd Lipcon
> Assignee: bc Wong
> Attachments: HDFS-941-1.patch, HDFS-941-2.patch, HDFS-941-3.patch,
> HDFS-941-3.patch, HDFS-941-4.patch
>
>
> Right now each connection into the datanode xceiver only processes one
> operation.
> In the case that an operation leaves the stream in a well-defined state (eg a
> client reads to the end of a block successfully) the same connection could be
> reused for a second operation. This should improve random read performance
> significantly.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
