Re: Producer will pick one of the two brokers, but never the two at same time [0.8]

2013-06-13 Thread Alexandre Rodrigues
Hi Jun, I was using the 0.8 branch with 2 commits behind but now I am using the latest with the same issue. 3 topics A,B,C, created automatically with replication factor of 2 and partitions 2. 2 brokers (0 and 1). List of topics in zookeeper is the following: topic: A partition: 0leader: 1

Re: Producer will pick one of the two brokers, but never the two at same time [0.8]

2013-06-13 Thread Alexandre Rodrigues
I've tried the console producer, so I will assume that's not related with the producer. I keep seeing the same entries in the producer from time to time: [2013-06-13 11:04:00,670] WARN Error while fetching metadata [{TopicMetadata for topic C - No partition metadata for topic C due to

Re: Producer will pick one of the two brokers, but never the two at same time [0.8]

2013-06-13 Thread Alexandre Rodrigues
I think I know what's happening: I tried to run both brokers and ZK on the same machine and it worked. I also attempted to do the same but with a ZK node on other machine and it also worked. My guess is something related with ports. All the machines are on EC2 and there might be something

Re: Producer will pick one of the two brokers, but never the two at same time [0.8]

2013-06-13 Thread Jun Rao
Have you looked at #3 in http://kafka.apache.org/faq.html? Thanks, Jun On Thu, Jun 13, 2013 at 6:41 AM, Alexandre Rodrigues alexan...@blismedia.com wrote: I think I know what's happening: I tried to run both brokers and ZK on the same machine and it worked. I also attempted to do the

Re: Producer will pick one of the two brokers, but never the two at same time [0.8]

2013-06-13 Thread Alexandre Rodrigues
I have but this is a different thing. It's related with ports and security groups and not with the bind addresses. It's solved now. Thanks On 13 June 2013 15:42, Jun Rao jun...@gmail.com wrote: Have you looked at #3 in http://kafka.apache.org/faq.html? Thanks, Jun On Thu, Jun 13, 2013

Re: Producer will pick one of the two brokers, but never the two at same time [0.8]

2013-06-12 Thread Jun Rao
If the leaders exist in both brokers, the producer should be able to connect to both of them, assuming you don't provide any key when sending the data. Could you try restarting the producer? If there has been broker failures, it may take topic.metadata.refresh.interval.ms for the producer to pick

Re: Producer will pick one of the two brokers, but never the two at same time [0.8]

2013-06-12 Thread Alexandre Rodrigues
Hi Jun, Thanks for your prompt answer. The producer yields those errors in the beginning, so I think the topic metadata refresh has nothing to do with it. The problem is one of the brokers isn't leader on any partition assigned to it and because topics were created with a replication factor of

Re: Producer will pick one of the two brokers, but never the two at same time [0.8]

2013-06-12 Thread Jun Rao
Any error in state-change.log? Also, are you using the latest code in the 0.8 branch? Thanks, Jun On Wed, Jun 12, 2013 at 9:27 AM, Alexandre Rodrigues alexan...@blismedia.com wrote: Hi Jun, Thanks for your prompt answer. The producer yields those errors in the beginning, so I think the