[
https://issues.apache.org/jira/browse/KAFKA-1253?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13939579#comment-13939579
]
Guozhang Wang edited comment on KAFKA-1253 at 3/18/14 11:03 PM:
----------------------------------------------------------------
The second experiments are done with 10K messages half of which are random bits
and the other half all 1's, batch size = 16K, linger time 5 seconds, recorded
#.reallocations.
GZIP:
|| message size || 1K || 10K || 100K ||
|heuristic0|312|0|0|
|heuristic1|0|0|0|
|heuristic2|0|0|0|
|heuristic3|0|0|0|
|heuristic4|312|2500|0|
SNAPPY:
|| message size || 1K || 10K || 100K ||
|heuristic0|1|0|0|
|heuristic1|1|0|0|
|heuristic2|1|0|0|
|heuristic3|0|0|0|
|heuristic4|312|0|0|
Memory usage:
GZIP:
|| message size || 1K || 10K || 100K ||
|heuristic0|0.5187830764937806|0.6361027327647318|0.5027070816956634|
|heuristic1|0.9398563032097622|0.6360584137828137|0.5027141148860828|
|heuristic2|0.9397809434478939|0.6361234861388692|0.5026772336472551|
|heuristic3|0.48929535651283906|0.32104673640806436|0.5027114615581034|
|heuristic4|0.5188796489295215|0.6345498820519015|0.5027165618824355|
SNAPPY:
|| message size || 1K || 10K || 100K ||
|heuristic0|0.9534642084279842|0.6618060085311569|0.5244664440669362|
|heuristic1|0.9532791947655151|0.6622619777918128|0.5244664440669362|
|heuristic2|0.9533003355592538|0.6618328095700078|0.5244664440669362|
|heuristic3|0.511382939911297|0.33245849609718725|0.5244664440669362|
|heuristic4|0.5445814384269451|0.6618052651191988|0.5244664440669362|
was (Author: guozhang):
The second experiments are done with 10K messages half of which are random bits
and the other half all 1's, batch size = 16K, recorded #.reallocations.
GZIP:
|| message size || 1K || 10K || 100K ||
|heuristic0|1|0|0|
|heuristic1|0|0|0|
|heuristic2|0|0|0|
|heuristic3|0|0|0|
|heuristic4|1|2400|0|
SNAPPY:
|| message size || 1K || 10K || 100K ||
|heuristic0|0|0|0|
|heuristic1|0|0|0|
|heuristic2|0|0|0|
|heuristic3|0|0|0|
|heuristic4|0|0|0|
> Implement compression in new producer
> -------------------------------------
>
> Key: KAFKA-1253
> URL: https://issues.apache.org/jira/browse/KAFKA-1253
> Project: Kafka
> Issue Type: Sub-task
> Components: producer
> Reporter: Jay Kreps
> Assignee: Guozhang Wang
> Attachments: KAFKA-1253.patch, KAFKA-1253_2014-02-21_16:15:21.patch,
> KAFKA-1253_2014-02-21_17:55:52.patch, KAFKA-1253_2014-02-24_13:31:50.patch,
> KAFKA-1253_2014-02-26_17:31:30.patch, KAFKA-1253_2014-03-06_17:48:11.patch,
> KAFKA-1253_2014-03-07_16:34:33.patch, KAFKA-1253_2014-03-10_14:35:56.patch,
> KAFKA-1253_2014-03-10_14:39:58.patch, KAFKA-1253_2014-03-10_15:27:47.patch,
> KAFKA-1253_2014-03-14_13:46:40.patch, KAFKA-1253_2014-03-14_17:39:53.patch,
> KAFKA-1253_2014-03-17_15:56:04.patch, compression-fix.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.2#6252)
