Re: Kafka ingestion data not equally distribute among brokers

2018-11-08 Thread Shiuh Rong Yong
is site?> > > https://kafka.apache.org/0100/javadoc/org/apache/kafka/streams/state/KeyValueStore.html> > > > >> > > Where can we set the key and value in Kafka?> > >> > > Thanks!> > >> > >> > > ----- Original Message ---

Re: Kafka ingestion data not equally distribute among brokers

2018-11-07 Thread Manoj Khangaonkar
nkar" > To: "users" > Sent: Tuesday, November 6, 2018 4:36:40 AM > Subject: Re: Kafka ingestion data not equally distribute among brokers > > Hi > > In Kafka topic data is split into partitions. Partitions are assigned to > brokers. > > I

Re: Kafka ingestion data not equally distribute among brokers

2018-11-06 Thread Shiuh Rong Yong
quot; Sent: Tuesday, November 6, 2018 4:36:40 AM Subject: Re: Kafka ingestion data not equally distribute among brokers Hi In Kafka topic data is split into partitions. Partitions are assigned to brokers. I assume what you are trying to says is that the distribution of messages across partit

Re: Kafka ingestion data not equally distribute among brokers

2018-11-05 Thread Manoj Khangaonkar
Hi In Kafka topic data is split into partitions. Partitions are assigned to brokers. I assume what you are trying to says is that the distribution of messages across partitions is not balanced. Messages are written to topics as key,value. The messages are distributed across partitions based on

Kafka ingestion data not equally distribute among brokers

2018-11-05 Thread yong
Hi Kafka experts, We have a 3 nodes kafka setup with data streams from mysql binlog, we noticed that from grafana monitoring in and out, some broker nodes are having lower data ingested in and out, is there anyway to balance it? Or it is nature of Kafka? Thanks!