So, for 0.8 without "controlled.shutdown.enable", why would ShutdownBroker and restarting cause under-replication and producer exceptions? How can I upgrade gracefully?
What's up with 0.8.1 getting stuck in preferred leader election? On Fri, Mar 21, 2014 at 12:18 AM, Neha Narkhede <neha.narkh...@gmail.com>wrote: > Which brings up the question - Do we need ShutdownBroker anymore? It seems > like the config should handle controlled shutdown correctly anyway. > > Thanks, > Neha > > > On Thu, Mar 20, 2014 at 9:16 PM, Jun Rao <jun...@gmail.com> wrote: > > > We haven't been testing the ShutdownBroker command in 0.8.1 rigorously > > since in 0.8.1, one can do the controlled shutdown through the new config > > "controlled.shutdown.enable". Instead of running the ShutdownBroker > command > > during the upgrade, you can also wait until under replicated partition > > count drops to 0 after each restart before moving to the next one. > > > > Thanks, > > > > Jun > > > > > > On Thu, Mar 20, 2014 at 3:14 PM, Ryan Berdeen <rberd...@hubspot.com> > > wrote: > > > > > While upgrading from 0.8.0 to 0.8.1 in place, I observed some > surprising > > > behavior using kafka.admin.ShutdownBroker. At the start, there were no > > > underreplicated partitions. After running > > > > > > bin/kafka-run-class.sh kafka.admin.ShutdownBroker --broker 10 ... > > > > > > Partitions that had replicas on broker 10 were under-replicated: > > > > > > bin/kafka-topics.sh --describe --under-replicated-partitions ... > > > Topic: analytics-activity Partition: 2 Leader: 12 Replicas: 12,10 > > Isr: > > > 12 > > > Topic: analytics-activity Partition: 6 Leader: 11 Replicas: 11,10 > > Isr: > > > 11 > > > Topic: analytics-activity Partition: 14 Leader: 14 Replicas: 14,10 > > Isr: > > > 14 > > > ... > > > > > > While restarting the broker process, many produce requests failed with > > > kafka.common.UnknownTopicOrPartitionException. > > > > > > After each broker restart, I used the preferred leader election tool > for > > > all topics. Now, after finishing all of the broker restarts, the > cluster > > > seems to be stuck in leader election. Running the tool fails with > > > "kafka.admin.AdminOperationException: Preferred replica leader election > > > currently in progress..." > > > > > > Are any of these known issues? Is there a safer way to shutdown and > > restart > > > brokers that does not cause producer failures and under-replicated > > > partitions? > > > > > >
