generate message consumers as kafka server threads

Hi I have a question about kafka servers. Can I generate several consumer thread when I start kafka servers? In other words, right now in our program kafka servers and consumers are two different processes. We use SimpleConsumer instance and do inter-process communication in order to fetch message

Re: Copy kafka data between servers?

I tried doing the following, but couldn't reproduce your issue - 1. Start zookeeper and kafka 2. Send some messages 3. Shutdown kafka and zookeeper 4. Start zookeeper and Kafka 5. Consume all messages Do you mind sending around the entire Kafka and Consumer logs ? Thanks, Neha On Thu, Feb 28,

Re: How to view Topic registration info and Partition state info from zookeeper in 0.8?

I have found Kazoo to be very convenient for scripting purposes. It is a python zookeeper client that is very easy to use - http://kazoo.readthedocs.org/en/latest/# Thanks, Neha On Fri, Mar 1, 2013 at 10:16 AM, Premal Shah wrote: > You can also install netflix exhibitor. It gives you a GUI to v

Re: How to view Topic registration info and Partition state info from zookeeper in 0.8?

You can also install netflix exhibitor. It gives you a GUI to view zookeeper data. On Fri, Mar 1, 2013 at 10:12 AM, Swapnil Ghike wrote: > Hi Jason > > You can download zookeeper, and use a client as zookeeper/bin/zkCli.sh > -server zkHost:zkPort. Then you can use the ls and get functions to li

Re: How to view Topic registration info and Partition state info from zookeeper in 0.8?

Hi Jason You can download zookeeper, and use a client as zookeeper/bin/zkCli.sh -server zkHost:zkPort. Then you can use the ls and get functions to list the structure hierarchy or to see the contents at each of the nodes in the hierarchy respectively. The data structures described in the wiki can

Re: Remote Producer

Could you verify if you can telnet to ZK and Kafka port from the remote node? Thanks, Jun On Fri, Mar 1, 2013 at 9:14 AM, Kevin Dorfer wrote: > Hi, > I'm having a lot of issues trying to connect to zookeeper from a producer > I implemented. > the producer is based on my locale machine and the

Re: kafka 0.7.2 jmx issue

Hmm, not sure what the problem is. At LinkedIn, our producers typically run in a Tomcat or Jetty container and they seem to work. Thanks, Jun On Fri, Mar 1, 2013 at 6:58 AM, Said Elaissaoui wrote: > Hi all, > > I'm trying to integrate kafka producer to weblogic 11 server using jdk1.6, > and i

Remote Producer

Hi, I'm having a lot of issues trying to connect to zookeeper from a producer I implemented. the producer is based on my locale machine and the broker+zookeeper are located on a remote server. When I use the console directly on the remote server, everything is fine, producer and consumer work

kafka 0.7.2 jmx issue

Hi all, I'm trying to integrate kafka producer to weblogic 11 server using jdk1.6, and i saw in some errors in the logs: javax.management.NotCompliantMBeanException: MBean class kafka.producer.async.AsyncProducerStats does not implement DynamicMBean, neither follows the Standard MBean conventi

Re: How to view Topic registration info and Partition state info from zookeeper in 0.8?

Thanks - I will definitely check that out. Any other options to suggest? thanks, Jason On Fri, Mar 1, 2013 at 5:29 AM, Matan Safriel wrote: > You can use the eclipse zookeeper plugin, if you only want to > sporadically view state.. > > On Mar 1, 2013, at 11:49 AM, Jason Huang wrote: > >> Hell

Re: How to view Topic registration info and Partition state info from zookeeper in 0.8?

You can use the eclipse zookeeper plugin, if you only want to sporadically view state.. On Mar 1, 2013, at 11:49 AM, Jason Huang wrote: > Hello, > > I am reading the "Kafka data structures in Zookeeper" > (https://cwiki.apache.org/KAFKA/kafka-data-structures-in-zookeeper.html) > document and try

How to view Topic registration info and Partition state info from zookeeper in 0.8?

Hello, I am reading the "Kafka data structures in Zookeeper" (https://cwiki.apache.org/KAFKA/kafka-data-structures-in-zookeeper.html) document and trying to figure out how to view topic registration information from zookeeper. Are these information stored in the zookeeper snapshot file in the har