[
https://issues.apache.org/jira/browse/KAFKA-406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13423216#comment-13423216
]
Lorenzo Alberton commented on KAFKA-406:
----------------------------------------
Hi Jay, thanks for your comments. When I wrote that the compression flag could
"slip" to 1 of course I meant there's nothing in the code to prevent nested
levels of compressed messages, as Jun noted. I'd add some checks to prevent
this at write time.
I agree batch compression with a message-at-a-time system can't play nice with
each other, so I don't have a better design to propose at this time.
The only alternative I could see would be to disallow mixed compressed and
non-compressed messages in the same topic, and having two different message set
iterators.
Anyway, for now I completely revamped the PHP client library to work with the
current design (tested against code in trunk), @see KAFKA-419, so at least we
can proceed :-).
> Gzipped payload is a fully wrapped Message (with headers), not just payload
> ---------------------------------------------------------------------------
>
> Key: KAFKA-406
> URL: https://issues.apache.org/jira/browse/KAFKA-406
> Project: Kafka
> Issue Type: Bug
> Components: core
> Affects Versions: 0.7.1
> Environment: N/A
> Reporter: Lorenzo Alberton
>
> When creating a gzipped MessageSet, the collection of Messages is passed to
> CompressionUtils.compress(), where each message is serialised [1] into a
> buffer (not just the payload, the full Message with headers, uncompressed),
> then gripped, and finally wrapped into another Message [2].
> In other words, the consumer has to unwrap the Message flagged as gzipped,
> unzip the payload, and unwrap the unzipped payload again as a non-compressed
> Message.
> Is this double-wrapping the intended behaviour?
> [1] messages.foreach(m => m.serializeTo(messageByteBuffer))
> [2] new Message(outputStream.toByteArray, compressionCodec)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
