Jason, Are you using ack = 0 in the producer? This mode doesn't work well with controlled shutdown (this is explained in FAQ i*n https://cwiki.apache.org/confluence/display/KAFKA/Replication+tools#)* * * Thanks,
Jun On Sun, Jun 23, 2013 at 1:45 AM, Jason Rosenberg <j...@squareup.com> wrote: > I'm working on trying on having seamless rolling restarts for my kafka > servers, running 0.8. I have it so that each server will be restarted > sequentially. Each server takes itself out of the load balancer (e.g. sets > a status that the lb will recognize, and then waits more than long enough > for the lb to stop sending meta-data requests to that server). Then I > initiate the shutdown (with controlled.shutdown.enable=true). This seems > to work well, however, I occasionally see warnings like this in the log > from the server, after restart: > > 2013-06-23 08:28:46,770 WARN [kafka-request-handler-2] server.KafkaApis - > [KafkaApi-508818741] Produce request with correlation id 7136261 from > client on partition [mytopic,0] failed due to Leader not local for > partition [mytopic,0] on broker 508818741 > > This WARN seems to persistently repeat, until the producer client initiates > a new meta-data request (e.g. every 10 minutes, by default). However, the > producer doesn't log any errors/exceptions when the server is logging this > WARN. > > What's happening here? Is the message silently being forwarded on to the > correct leader for the partition? Is the message dropped? Are these WARNS > particularly useful? > > Thanks, > > Jason >
