[ https://issues.apache.org/jira/browse/KAFKA-4666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ray Chiang updated KAFKA-4666: ------------------------------ Component/s: documentation > Failure test for Kafka configured for consistency vs availability > ----------------------------------------------------------------- > > Key: KAFKA-4666 > URL: https://issues.apache.org/jira/browse/KAFKA-4666 > Project: Kafka > Issue Type: Improvement > Components: documentation > Reporter: Emanuele Cesena > Priority: Major > Attachments: consistency_test.py > > > We recently had an issue with our Kafka setup because of a misconfiguration. > In short, we thought we have configured Kafka for durability, but we didn't > set the producers to acks=all. During a full outage, we had situations where > some partitions were "partitioned", meaning that the followers started > without properly waiting for the right leader, and thus we lost data. Again, > this is not an issue with Kafka, but a misconfiguration on our side. > I think we reproduced the issue, and we built a docker test that proves that, > if the producer isn't set with acks=all, then data can be lost during an > almost full outage. The test is attached. > I was thinking to send a PR, but wanted to run this through you first, as > it's not necessarily proving that a feature works as expected. > In addition, I think the documentation could be slightly improved, for > instance in the section: > http://kafka.apache.org/documentation/#design_ha > by clearly stating that there are 3 steps one should do for configuring kafka > for consistency, the third being that producers should be set with acks=all > (which is now part of the 2nd point). > Please let me know what do you think, and I can send a PR if you agree. -- This message was sent by Atlassian JIRA (v7.6.3#76005)