No producers were active. The only producer in our pipeline is a storm topology and had made it a point that the whole cluster was down and no topology running. What I did was nothing short of doing a fresh kafka cluster setup (barring the fact that I didn¹t wiped the zookeepers as its the same setup which our storm uses. But given the fact that I had dropped /brokers, it should not be concern).
So the question is where exactly is kafka brokers getting the info about the topic to recreate when they come up when there is no producers and all previous info has been wiped out. Regards, Virendra On 6/26/14, 9:29 AM, "Neha Narkhede" <neha.narkh...@gmail.com> wrote: >Firstly, I'm not sure this process of deleting topics completely works, >especially in corner cases. Having said that, for #3, you should just be >deleting /brokers/topics/<topics-to-be-deleted>. If producers are sending >data to the Kafka cluster, it will recreate the topics once the brokers >come up. > >Thanks, >Neha > > >On Wed, Jun 25, 2014 at 11:53 PM, Virendra Pratap Singh < >vpsi...@yahoo-inc.com.invalid> wrote: > >> I am aware of lack of programmatic way of deleting topics in kafka >>0.8.0. >> So using the sledge hammer approach. >> This is what I am doing: >> >> 1. Bring whole of my kafka cluster down. >> 2. Delete all the content on all the kafka clusters pointed via logs.dir >> setting. >> 3. Delete the topic metadata from zookeeper : rmr /brokers (note I am >>not >> wiping off the whole zookeeper but the znode /brokers where the kafka >> broker ids and topic metadata is stored) >> 4. Restart the kafka cluster again. >> >> One would expect that the kafka cluster will come up with no memory of >>any >> topic from previous. >> >> But guess what, and this is the place where I need help and need to >> understand, when the kafka cluster comes back, it somehow is able to >>obtain >> the info of the previous topics. It promptly goes ahead creating and >> assigning partitions/replicas to the brokers for the previous topics. >>Now I >> am completely at loss to understand where exactly is kafka able to get >>the >> info of previous topics when I have wiped it off the zookeeper and also >> dropped the logs.dir locations across the kafka cluster. >> >> An insight is much needed here. Where else is the topic meta data store >> which the kafka server is getting hold of after coming back alive? >> >> Regards, >> Virendra >> >>
