[
https://issues.apache.org/jira/browse/KAFKA-1374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14316691#comment-14316691
]
Manikumar Reddy commented on KAFKA-1374:
----------------------------------------
yes, we are using TestLogCleaning tool to test the changes.
TestLogCleaning stress test output for compressed messages
{code}
Producing 100000 messages...
Logging produce requests to
/tmp/kafka-log-cleaner-produced-6014466306002699464.txt
Sleeping for 120 seconds...
Consuming messages...
Logging consumed messages to
/tmp/kafka-log-cleaner-consumed-177538909590644701.txt
100000 rows of data produced, 13165 rows of data consumed (86.8% reduction).
De-duplicating and validating output files...
Validated 9005 values, 0 mismatches.
Producing 1000000 messages...
Logging produce requests to
/tmp/kafka-log-cleaner-produced-3298578695475992991.txt
Sleeping for 120 seconds...
Consuming messages...
Logging consumed messages to
/tmp/kafka-log-cleaner-consumed-7192293977610206930.txt
1000000 rows of data produced, 119926 rows of data consumed (88.0% reduction).
De-duplicating and validating output files...
Validated 89947 values, 0 mismatches.
Producing 10000000 messages...
Logging produce requests to
/tmp/kafka-log-cleaner-produced-3336255463347572934.txt
Sleeping for 120 seconds...
Consuming messages...
Logging consumed messages to
/tmp/kafka-log-cleaner-consumed-9149188270705707725.txt
10000000 rows of data produced, 1645281 rows of data consumed (83.5% reduction).
De-duplicating and validating output files...
Validated 899853 values, 0 mismatches.
{code}
> LogCleaner (compaction) does not support compressed topics
> ----------------------------------------------------------
>
> Key: KAFKA-1374
> URL: https://issues.apache.org/jira/browse/KAFKA-1374
> Project: Kafka
> Issue Type: Bug
> Reporter: Joel Koshy
> Assignee: Manikumar Reddy
> Labels: newbie++
> Fix For: 0.8.3
>
> Attachments: KAFKA-1374.patch, KAFKA-1374_2014-08-09_16:18:55.patch,
> KAFKA-1374_2014-08-12_22:23:06.patch, KAFKA-1374_2014-09-23_21:47:12.patch,
> KAFKA-1374_2014-10-03_18:49:16.patch, KAFKA-1374_2014-10-03_19:17:17.patch,
> KAFKA-1374_2015-01-18_00:19:21.patch
>
>
> This is a known issue, but opening a ticket to track.
> If you try to compact a topic that has compressed messages you will run into
> various exceptions - typically because during iteration we advance the
> position based on the decompressed size of the message. I have a bunch of
> stack traces, but it should be straightforward to reproduce.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
