[
https://issues.apache.org/jira/browse/KAFKA-1755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14234780#comment-14234780
]
Guozhang Wang commented on KAFKA-1755:
--------------------------------------
Here are my two cents:
1. At the end of the day, Kafka will have two types of topics, one type only
accepts keyed messages and log compaction is used; the other one accepts any
message and log cleaning is used. Those two types of topics never exchange,
i.e. once a topic is created with one of the two types, it will never change
its type until deletion.
2. Compressed message will be supported with log compaction, which will
de-serialize the message set and re-serialize.
3. With these two points in mind, I would suggest for now:
a. Broker reject non-keyed messages for compacted topics.
b. Broker reject compressed messages for compacted topics (this will be
lifted after KAFKA-1374 is checked in).
c. With this, it should never happen that compactor thread encountering a
non-keyed / compressed (this will be lifted after KAFKA-1374); if it happens,
this would be a FATAL error and we should throw an exception and halt the
server. It indicates some operations are needed and there are some code fixes
before it can be restarted.
> Improve error handling in log cleaner
> -------------------------------------
>
> Key: KAFKA-1755
> URL: https://issues.apache.org/jira/browse/KAFKA-1755
> Project: Kafka
> Issue Type: Bug
> Reporter: Joel Koshy
> Assignee: Joel Koshy
> Labels: newbie++
> Fix For: 0.8.3
>
>
> The log cleaner is a critical process when using compacted topics.
> However, if there is any error in any topic (notably if a key is missing)
> then the cleaner exits and all other compacted topics will also be adversely
> affected - i.e., compaction stops across the board.
> This can be improved by just aborting compaction for a topic on any error and
> keep the thread from exiting.
> Another improvement would be to reject messages without keys that are sent to
> compacted topics although this is not enough by itself.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
