[
https://issues.apache.org/jira/browse/RATIS-1312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17294392#comment-17294392
]
runzhiwang edited comment on RATIS-1312 at 3/3/21, 9:03 AM:
------------------------------------------------------------
[~szetszwo] I find decodeDataStreamRequestByteBuf and allocating
DirectByteBuffer both caused by the same reason.
1. decodeDataStreamRequestByteBuf costs 6.42% caused by, netty splits 4MB to
many small packets, each about 60KB, even though we define buffer size 4MB, so
each 60KB small packet need to decodeDataStreamRequestByteBuf.
2. allocating DirectByteBuffer costs 8.79%, also caused by too many small
packets, because only when we receive all the packets up to 4MB then we can
continue to process, so DirectByteBuffer are not enough to hold all the 4MB
ByteBuf, then we need to allocate new DirectByteBuffer.
If we define buffer size to 60KB, it's slower than 4MB. Allocating
DirectByteBuffer did not cost CPU, but more than 20% cpu on: a lot of lambda
expression cost 16% in DataStreamManagement#read such as composeAsync,
encode/decode request/reply
was (Author: yjxxtd):
[~szetszwo] I find decodeDataStreamRequestByteBuf and allocating
DirectByteBuffer both caused by the same reason.
1. decodeDataStreamRequestByteBuf costs 6.42% caused by, netty splits 4MB to
many small packets, each about 60KB, even though we define buffer size 4MB, so
each 60KB small packet need to decodeDataStreamRequestByteBuf.
2. allocating DirectByteBuffer costs 8.79%, also caused by too many small
packets, because only when we receive all the packets up to 4MB then we can
continue to process, so DirectByteBuffer are not enough to hold all the 4MB
ByteBuf, then we need to allocate new DirectByteBuffer.
If we define buffer size to 60KB, it's slower than 4MB. Allocating
DirectByteBuffer did not cost CPU, but we more than 20% cpu on: a lot of lambda
expression cost 16% in DataStreamManagement#read such as composeAsync,
encode/decode request/reply
> Compare the performance between HDFS and DataStreamApi
> ------------------------------------------------------
>
> Key: RATIS-1312
> URL: https://issues.apache.org/jira/browse/RATIS-1312
> Project: Ratis
> Issue Type: Sub-task
> Reporter: runzhiwang
> Priority: Major
> Attachments: hdfs.svg, streaming.svg
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
