Producer acks=1, clean broker shutdown and data loss

2017-02-18 Thread Nick Travers
Hi - I'm trying to understand the expected behavior of the scenario in which I have a producer with `acks=1` (i.e. partition leader acks only) and I cleanly shut down a broker (via `KafkaServer#shutdown`). I am running my test scenario with three brokers (0.10.1.1), with a default replication coun

Re: Reassigning partitions to a non-existent broker

2017-01-20 Thread Nick Travers
(with the reassign-partitions script) - decommission the broker Opened KAFKA-4681 to track this issue. On Thu, Jan 19, 2017 at 4:50 PM, Nick Travers wrote: > We recently tried to rebalance partitions for a topic (via > kafka.admin.ReassignPartitionsCommand). > > In the .json f

Reassigning partitions to a non-existent broker

2017-01-19 Thread Nick Travers
We recently tried to rebalance partitions for a topic (via kafka.admin.ReassignPartitionsCommand). In the .json file with the desired end-state, an id for a non-existent broker was entered for a partition. Upon --execute, all partitions were moved without issue, but the partition with the bogus br

Brokers stuck trying to shrink ISR down to themselves

2017-01-13 Thread Nick Travers
Hey folks - we had a blip on one of our smaller clusters last night (3 nodes), around the same time as network maintenance in the DC that it was operating in. This caused a partition in the network between the brokers and Zk, as well as between the brokers themselves. The partition was for approx.