Corp replica does not shutdown on IO error
------------------------------------------
Key: KAFKA-75
URL: https://issues.apache.org/jira/browse/KAFKA-75
Project: Kafka
Issue Type: Improvement
Reporter: Joel Koshy
The embedded consumer in the corp replica uses the low-level Log api to create
the replica. The append operation may fail and result in a corrupt log file,
due to an IO error which is currently caught and ignored.
The proposed fix is to switch to using the high-level producer API to create
the replica. Not only would this avoid the above issue, but it would also fit
better with the current design of the replication enhancement for kafka
(http://linkedin.jira.com/browse/KAFKA-23), since the low-level Log api is not
replication-aware. Another advantage is that compression is exposed at the
producer API-level. One caveat in this approach would be the following: the
async producer drops events when its queue is full. This behavior is unsuitable
for the embedded consumer, so we can expose a configuration option in the
producer to allow for (queue-level) blocking semantics.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
