[
https://issues.apache.org/jira/browse/HADOOP-6683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12854316#action_12854316
]
Xiao Kang commented on HADOOP-6683:
-----------------------------------
A comparision test was performed on a 1.8GB web log file. The result is as
follows:
|| read file buffer size || write to compress stream buffer size || old
time(secs) || new time(secs) || decrease % ||
|65536| 100 |67| 49| 26.8%|
|65536| 200| 56.5| 46.5| 17.7%|
|65536| 400| 51.5| 45| 12.6%|
|65536| 800| 48.5| 44.5| 8.2%|
|65536| 1024| 46.8| 44.2| 9.8%|
|65536| 4096| 45| 43.5| 3.3%|
|65536| 65536| 44.6| 43.2| 3.1%|
Is there any standard benchmark for compression suitable for this case?
> the first optimization: ZlibCompressor does not fully utilize the buffer
> ------------------------------------------------------------------------
>
> Key: HADOOP-6683
> URL: https://issues.apache.org/jira/browse/HADOOP-6683
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: io
> Affects Versions: 0.20.2
> Reporter: Xiao Kang
> Attachments: ZlibCompressor.java.patch
>
>
> Thanks for Hong Tang's advice.
> Sub task created for the first optimization. HADOOP-6662 closed.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
