Hi All, I would like to request inputs - pros and cons of setting up a 2 broker Kafka cluster in production.
Following is the proposed configuration: - 3 zookeeper nodes, 2 kafka brokers in AWS. - 1 topic with replication factor 1 and 4 partitions. - Producer with "acks = all" and min.insync.replicas = 1 1) According to Kafka documentation at https://kafka.apache.org/ documentation/ <https://kafka.apache.org/documentation/> - For a topic with replication factor N, we will tolerate up to N-1 server failures without losing any records committed to the log. Question: Based on the above guarantee, Kafka can tolerate 1-1 = 0 server failures. Does this mean that even if one of the brokers is lost, data loss can happen even though producer acks=all and min.insync.replicas=1. 2) What are the error scenarios that can happen if a 2 broker kafka cluster is deployed? Any pros or cons of using such a cluster? I would appreciate inputs. Thank you in advance.
