[
https://issues.apache.org/jira/browse/HADOOP-6311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13477467#comment-13477467
]
Todd Lipcon commented on HADOOP-6311:
-------------------------------------
bq. With respect to security: there is always a possibility for a client to
open a socket with the same name as the server would have used. This is similar
to the problem with TCP/IP sockets of a malicious program grabbing the port
before the DataNode could get it (or after the DataNode has died.)
That's why secure clusters use low (privileged) ports for the data transfer
protocol.
bq. I don't think that on-disk format changes are that big of a deal for the
short-circuit pathway. We tell old clients they can't use short-circuit reads
on those files, and fix new clients to understand the new format.
Agreed, just need to make sure the "deny" pathway works and ideally some kind
of version number exposed.
bq. TCP optimizations are pretty cool, but not when you run on RHEL6, as many
folks do Maybe we should open a separate JIRA to investigate things like TCP
fast open, changing TCP kernel options, etc. might be used with Hadoop in the
future. There are also certain performance improvements we could do in the read
and write paths on the DataNode, but again, that's out of scope for this JIRA,
I think.
Agreed, but my question is more this: let's assume that unix sockets for the
data path are 3x as fast as local TCP sockets. If that's the case, then do we
still get a big benefit from short-circuit? I think the answer is probably yes
for random read, but no for sequential. The point about trying the "tcp
friends" in future versions is just one potential way of evaluating this
without having to write all the code for a unix socket data path. If "tcp
friends" is comparable to short circuit, then unix sockets would probably also
be comparable.
> Add support for unix domain sockets to JNI libs
> -----------------------------------------------
>
> Key: HADOOP-6311
> URL: https://issues.apache.org/jira/browse/HADOOP-6311
> Project: Hadoop Common
> Issue Type: New Feature
> Components: native
> Affects Versions: 0.20.0
> Reporter: Todd Lipcon
> Assignee: Colin Patrick McCabe
> Attachments: 6311-trunk-inprogress.txt, design.txt,
> HADOOP-6311.014.patch, HADOOP-6311.016.patch, HADOOP-6311.018.patch,
> HADOOP-6311.020b.patch, HADOOP-6311.020.patch, HADOOP-6311.021.patch,
> HADOOP-6311.022.patch, HADOOP-6311-0.patch, HADOOP-6311-1.patch,
> hadoop-6311.txt
>
>
> For HDFS-347 we need to use unix domain sockets. This JIRA is to include a
> library in common which adds a o.a.h.net.unix package based on the code from
> Android (apache 2 license)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
