Some queries about java api for kafka producer

Hi, I want to send data to apache kafka using the java api of the kafka producer. The data will be in high volume, of the order of more than 5 thousand messages per second. Please help me with the following queries: 1. Should I create only one producer object for the entire app and use the same ob

Re: Some queries about java api for kafka producer

Hi, For (1) from the java docs "The producer is *thread safe* and should generally be shared among all threads for best performance" (2) (1) implies no pool is necessary. regards On Sun, Apr 12, 2015 at 12:38 AM, dhiraj prajapati wrote: > Hi, > I want to send data to apache kafka using the ja

Re: Some queries about java api for kafka producer

Clarification. My answer applies to the new producer API in 0.8.2. regards On Sun, Apr 12, 2015 at 4:00 PM, Manoj Khangaonkar wrote: > Hi, > > For (1) from the java docs "The producer is *thread safe* and should > generally be shared among all threads for best performance" > > (2) (1) implies n

java consumer client sometimes work,but sometimes not.

hi there. i got a really issue,consumer(java client) sometimes works,but sometimes not. i had read the sources code, bug got nothing,can u help me ? log [root@slave3 kafka_2.10-0.8.2.0]# ./bin/kafka-console-producer.sh

Producer does not recognize new brokers

I added several new brokers to the cluster, there should'v been a rebalance but it seemed that the producer was not aware of the new brokers. Data kept being sent to the old brokers and there were no partitions on the new brokers. I configured the old brokers to the producer and did not restart th

Re: Producer does not recognize new brokers

Hi, Simply adding the brokers to the cluster will not reassign or redistribute topic partitions to newly added brokers. As it is also mentioned in documentation. "*However these new servers will not automatically be assigned any data partitions, so unless partitions are moved to them they won't b

Re: Producer does not recognize new brokers

I think you need to re balance the cluster. something like bin/kafka-reassign-partitions.sh --zookeeper localhost:2181 --topics-to-move-json-file topics-to-move.json --broker-list "5,6" --generate On Mon, Apr 13, 2015 at 11:22 AM, shadyxu wrote: > I added several new brokers to the cluster, th

Topic to broker assignment

Hi Guys How do topics get assigned to brokers? I mean if I were to create a topic X and publish to it how does Kafka assign the topic and the message to a particular broker? If I have create a topic with say 10 partitions how does kafka assign each partition to a different broker? -- Cheers Bill