Re: getting error in fetching topic metadata,when giving message to console producer in Kafka cluster

2013-10-17 Thread Monika Garg
Thanks Jun...You are right,bymistakenly I have given zookeeper port. Now its working fine. On Thu, Oct 17, 2013 at 8:50 PM, Jun Rao wrote: > I think you provided the wrong port in broker-list. 2181 is the ZK port. > Kafka port defaults to 9092. > > Thanks, > > Jun > > > On Thu, Oct 17, 2013 at

Re: Handling consumer rebalance when implementing synchronous auto-offset commit

2013-10-17 Thread Joel Koshy
We should be able to get this in after 0.8.1 and probably before the client rewrite. Thanks, Joel On Wednesday, October 16, 2013, Jason Rosenberg wrote: > This looks great. What is the time frame for this effort? > > Jason > > > On Wed, Oct 16, 2013 at 2:19 PM, Joel Koshy wrote: > > > Btw, a

Re: Broker bind address versus published hostname in ZooKeeper

2013-10-17 Thread Jun Rao
Tim, This seems like a reasonable requirement. Would you be interested in providing a patch to the jira? Thanks, Jun On Thu, Oct 17, 2013 at 3:20 PM, Timothy Chen wrote: > Hi Roger, > > That's exactly what I need in my end, and actually internally created a new > property called zkHost.name

Re: Broker bind address versus published hostname in ZooKeeper

2013-10-17 Thread Timothy Chen
Hi Roger, That's exactly what I need in my end, and actually internally created a new property called zkHost.name to publish a different host to zk. This is also needed for deploying Kafka into Azure. I also created zkHost.port since the internal and external ports that's exposed might be differe

Broker bind address versus published hostname in ZooKeeper

2013-10-17 Thread Roger Hoover
Hi all, I'm getting started experimenting with Kafka and ran into a configuration issue. Currently, in server.properties, you can configure host.name which gets used for two purposes: 1) to bind the socket 2) to publish the broker details to ZK for clients to use. There are times when these two

Re: Flush configuration per topic

2013-10-17 Thread Steve Morin
Is there only time delay or can it be set to flush for every message with the obvious performance hit? On Wed, Oct 16, 2013 at 9:49 AM, Jay Kreps wrote: > Yes, the change in trunk is that all log configurations are automatically > available at both the log level and the global default level and

Fwd: Special Bay Area HUG: Tajo and Samza

2013-10-17 Thread Jay Kreps
FYI. -- Forwarded message -- From: Jakob Homan Date: Thu, Oct 17, 2013 at 11:08 AM Subject: Special Bay Area HUG: Tajo and Samza To: d...@samza.incubator.apache.org Hey everybody- Join us at LinkedIn Nov. 5 for a special HUG dedicated to two new awesome Incubator projects, Ta

producer (or consumer?) statistics that was using metrics

2013-10-17 Thread S Ahmed
I remember a while back Jay was looking for someone to work on producer (or was it consumer) statisitics which was going to use metrics. Was this ever implemented?

Re: getting error in fetching topic metadata,when giving message to console producer in Kafka cluster

2013-10-17 Thread Jun Rao
I think you provided the wrong port in broker-list. 2181 is the ZK port. Kafka port defaults to 9092. Thanks, Jun On Thu, Oct 17, 2013 at 1:51 AM, Monika Garg wrote: > Hi, > > I had 3 nodes Kafka cluster with default.replication.factor=3 in all the > three nodes. > I create a topic "topic1" w

Re: Consumer doesn't start picking existing messages from topic until new data is added

2013-10-17 Thread Bruno D. Rodrigues
try this, not sure if it would help though props.put("auto.offset.reset", "smallest"); A 17/10/2013, às 14:13, Tarang Dawer escreveu: > Hi All > > I am facing issues with the kafka 0.8 consumer with the consumer not > picking existing messages in the topic for consumption. > > > > F

Consumer doesn't start picking existing messages from topic until new data is added

2013-10-17 Thread Tarang Dawer
Hi All I am facing issues with the kafka 0.8 consumer with the consumer not picking existing messages in the topic for consumption. Following is a snippet from my code : - public void run() { ConsumerIterator it = stream.iterator(); System.out.println("***

getting error in fetching topic metadata,when giving message to console producer in Kafka cluster

2013-10-17 Thread Monika Garg
Hi, I had 3 nodes Kafka cluster with default.replication.factor=3 in all the three nodes. I create a topic "topic1" with 7 partitions and with rep-factor = 3,my list topic command gave below o/p: topic: topic1partition: 0leader: 1replicas: 3,2,1isr: 1,2,3 topic: topic1partitio