[
https://issues.apache.org/jira/browse/HADOOP-9194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13549923#comment-13549923
]
Luke Lu commented on HADOOP-9194:
---------------------------------
bq. Is there a reason we need our own fields, if the same information is
present in DiffServ?
The field is needed in all layers. IP DS is for IP switches (layer-2/3), but
our RPC can use none IP network like IB/RDMA, loopback, shared memory and unix
domain socket (cf. your own work on HDFS-347). One specific example would be
HBase region server talking to DN on the same node via unix domain socket. You
want to be able differentiate OLTP traffic from compaction traffic and set io
priority on the fds accordingly (assuming underlying io scheduler supports it,
e.g. cfq).
The RPC field is also useful for layer-7 switches (application proxies, load
balancers) to implement QoS.
> RPC Support for QoS
> -------------------
>
> Key: HADOOP-9194
> URL: https://issues.apache.org/jira/browse/HADOOP-9194
> Project: Hadoop Common
> Issue Type: New Feature
> Components: ipc
> Affects Versions: 2.0.2-alpha
> Reporter: Luke Lu
>
> One of the next frontiers of Hadoop performance is QoS (Quality of Service).
> We need QoS support to fight the inevitable "buffer bloat" (including various
> queues, which are probably necessary for throughput) in our software stack.
> This is important for mixed workload with different latency and throughput
> requirements (e.g. OLTP vs OLAP, batch and even compaction I/O) against the
> same DFS.
> Any potential bottleneck will need to be managed by QoS mechanisms, starting
> with RPC.
> How about adding a one byte DS (differentiated services) field (a la the
> 6-bit DS field in IP header) in the RPC header to facilitate the QoS
> mechanisms (in separate JIRAs)? The byte at a fixed offset (how about 0?) of
> the header is helpful for implementing high performance QoS mechanisms in
> switches (software or hardware) and servers with minimum decoding effort.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
