Re: kafka.common.LeaderNotAvailableException: No leader for any partition

2013-05-30 Thread Jun Rao
Are you using the latest code from the 0.8 branch? Any exception/error in the broker log (including state-change.log)? Thanks, Jun On Wed, May 29, 2013 at 10:49 PM, Vadim Keylis wrote: > I am working on deploying Kafka 0.8 to our test cluster that consist of 2 > kafka servers and 2 zookeper no

Issue during commitOffsets using SimpleConsumer

2013-05-30 Thread arathi maddula
Hi, I get the following error on running SimpleConsumer.commitOffsets(). Could you tell me what is the issue? *java.io.EOFException*: Received -1 when reading from channel, socket has likely been closed. at kafka.utils.Utils$.read(Utils.scala:375) at kafka.network.BoundedByteBufferReceive.read

Re: kafka.common.LeaderNotAvailableException: No leader for any partition

2013-05-30 Thread Vadim Keylis
Jun. I rebuilt kafka this morning by following this steps > git clone https://git-wip-us.apache.org/repos/asf/kafka.git> cd kafka > git checkout -b 0.8 remotes/origin/0.8 > ./sbt update > ./sbt +package > ./sbt +assembly-package-dependency deployed to jar files kafka_2.9.2-0.8.0-SNAPSHOT.jar kaf

Re: kafka.common.LeaderNotAvailableException: No leader for any partition

2013-05-30 Thread Vadim Keylis
Your advice is appreciated On Thu, May 30, 2013 at 1:32 PM, Vadim Keylis wrote: > Jun. > I rebuilt kafka this morning by following this steps > > > git clone https://git-wip-us.apache.org/repos/asf/kafka.git> cd kafka > > git checkout -b 0.8 remotes/origin/0.8 > > ./sbt update > > ./sbt +packag

Hadoop example running DataGenerator causes kafka.message.Message cannot be cast to [B exception

2013-05-30 Thread Andrew Milkowski
Hi, Working of git master codebase and following instructions at https://github.com/apache/kafka/blob/trunk/contrib/hadoop-consumer/README https://github.com/apache/kafka when running ./run-class.sh kafka.etl.impl.DataGenerator test/test.properties an exception is thrown Exception in thread

Re: kafka.common.LeaderNotAvailableException: No leader for any partition

2013-05-30 Thread Neha Narkhede
Can you check if the value for zookeeper.connect is set correctly ? We changed the name of some configs in 08, you can find the new config names here - http://kafka.apache.org/08/configuration.html Thanks, Neha On Thu, May 30, 2013 at 1:33 PM, Vadim Keylis wrote: > Your advice is appreciated >

v0.8 in production?

2013-05-30 Thread d0t
Hi, We are interested in switching from 0.7.2 to v0.8 before an upcoming production launch deadline, mainly for bug fixes and replication capability.  I've noticed numerous people on this mailing list have already started using v0.8 with good success.  Two questions on this: -Can you share some

Re: kafka.common.LeaderNotAvailableException: No leader for any partition

2013-05-30 Thread Vadim Keylis
thanks so much to both of you guys. Looks like its working using your test clients. On Thu, May 30, 2013 at 1:57 PM, Neha Narkhede wrote: > Can you check if the value for zookeeper.connect is set correctly ? We > changed the name of some configs in 08, you can find the new config names > here -

Re: v0.8 in production?

2013-05-30 Thread Jun Rao
For 0.8 branch should be able to cross compile to Scala 2.9. Thanks, Jun On Thu, May 30, 2013 at 2:26 PM, d0t wrote: > Hi, > We are interested in switching from 0.7.2 to v0.8 before an upcoming > production launch deadline, mainly for bug fixes and replication > capability. I've noticed nume

Re: Issue during commitOffsets using SimpleConsumer

2013-05-30 Thread Jun Rao
This indicates an error on the broker. Any error/exception on the broker? Also, you are using trunk. The stable unreleased branch is 0.8. Thanks, Jun On Thu, May 30, 2013 at 11:22 AM, arathi maddula wrote: > Hi, > I get the following error on running SimpleConsumer.commitOffsets(). Could > yo

Re: Hadoop example running DataGenerator causes kafka.message.Message cannot be cast to [B exception

2013-05-30 Thread Jun Rao
This seems to be a bug. We should send the message string, instead the Message object in DataGenerator. Could you file a jira? Thanks, Jun On Thu, May 30, 2013 at 1:47 PM, Andrew Milkowski wrote: > Hi, > > Working of git master codebase > > and following instructions at > > https://github.com/

async producer and new ack levels

2013-05-30 Thread Jason Rosenberg
With 0.8, we now have ack levels when sending messages. I'm wondering how this applies when sending messages in async mode. Are there any guarantees at least that each async batch will wait for the requested ack level before sending the next batch? I assume there is still a disconnect between se

Re: async producer and new ack levels

2013-05-30 Thread Jun Rao
If async mode, messages are sent using the configured ack level. There is no callback in the async mode right now. We plan to enhance that in the future. Thanks, Jun On Thu, May 30, 2013 at 9:50 PM, Jason Rosenberg wrote: > With 0.8, we now have ack levels when sending messages. I'm wonderin

question about usage of SimpleConsumer

2013-05-30 Thread shangan chen
In Kafka, the consumers are responsible for maintaining state information (offset) on what has been consumed (refer from kafka design page).high-level consumer api will store its consumption state in zookeeper, while simple consumer shoud deal with these things itself. My doubt is what happened wh