[
https://issues.apache.org/jira/browse/HDFS-7966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Duo Zhang updated HDFS-7966:
----------------------------
Attachment: TestHttp2LargeReadPerformance.svg
Write a testcase which reads a 128MB block. The result is
{quote}
******* time based on http2 2776ms
******* time based on tcp 482ms
{quote}
This time, the client side is basically same, the overhead is at server side.
readWindowUpdateFrame is 14.27%, and the actual ChunkedBlockInput.readChunk is
8.54%, so the HTTP/2 overhead is still about 6%(of course the denominator maybe
different). And in ChunkedBlockInput.readChunk, only 2.91% of time is cost on
file read write and protobuf message build, most of time is cost on ByteBuf
management.
Will run test that reads data from different machine to see if ByteBuf
management and HTTP/2 are still costly.
Thanks.
> New Data Transfer Protocol via HTTP/2
> -------------------------------------
>
> Key: HDFS-7966
> URL: https://issues.apache.org/jira/browse/HDFS-7966
> Project: Hadoop HDFS
> Issue Type: New Feature
> Reporter: Haohui Mai
> Assignee: Qianqian Shi
> Labels: gsoc, gsoc2015, mentor
> Attachments: GSoC2015_Proposal.pdf,
> TestHttp2LargeReadPerformance.svg, TestHttp2Performance.svg
>
>
> The current Data Transfer Protocol (DTP) implements a rich set of features
> that span across multiple layers, including:
> * Connection pooling and authentication (session layer)
> * Encryption (presentation layer)
> * Data writing pipeline (application layer)
> All these features are HDFS-specific and defined by implementation. As a
> result it requires non-trivial amount of work to implement HDFS clients and
> servers.
> This jira explores to delegate the responsibilities of the session and
> presentation layers to the HTTP/2 protocol. Particularly, HTTP/2 handles
> connection multiplexing, QoS, authentication and encryption, reducing the
> scope of DTP to the application layer only. By leveraging the existing HTTP/2
> library, it should simplify the implementation of both HDFS clients and
> servers.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
