[
https://issues.apache.org/jira/browse/RATIS-1312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17295899#comment-17295899
]
runzhiwang edited comment on RATIS-1312 at 3/5/21, 9:30 AM:
------------------------------------------------------------
> Could you try also writing to disk?
[~szetszwo] Hi, I also test writting to disk, without sync.
1. Buffer size is 2MB:
1.a. file number is 1024, file size is 128MB
hdfs costs 133 seconds, streaming cost 210 seconds.
1.b. file number is 100, file size is 128MB
hdfs costs 13 seconds, streaming cost 20 seconds
2. Buffer size is 1MB.
2.a. file number is 1024, file size is 128MB
hdfs costs 130 seconds, streaming cost 221 seconds.
2.b. file number is 100, file size is 128MB
hdfs costs 11 seconds, streaming cost 23 seconds
was (Author: yjxxtd):
> Could you try also writing to disk?
[~szetszwo] Hi, I also test writting to disk, without sync.
1. Buffer size is 2MB:
1.a. file number is 1024, file size is 128MB
hdfs costs 133 seconds, streaming cost 210 seconds.
1.b. file number is 100, file size is 128MB
hdfs costs 13 seconds, streaming cost 20 seconds
2. Buffer size is 1MB.
2.a. file number is 1024, file size is 128MB
hdfs costs 130 seconds, streaming cost 221 seconds.
2.b. file number is 100, file size is 128MB
hdfs costs 11 seconds, streaming cost 23 seconds
> Compare the performance between HDFS and DataStreamApi
> ------------------------------------------------------
>
> Key: RATIS-1312
> URL: https://issues.apache.org/jira/browse/RATIS-1312
> Project: Ratis
> Issue Type: Sub-task
> Reporter: runzhiwang
> Priority: Major
> Attachments: hdfs.svg, screenshot-1.png, streaming.svg
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
