Kafka broker slow down when consumer try to fetch large messages from topic

I have 4 node kafka broker with following configuration : Default Number of Partitions : num.partitions : 1 Default Replication Factor : default.replication.factor : 1 Maximum Message Size : message.max.bytes : 10 MB Replica Maximum Fetch Size : replica.fetch.max.bytes : 10 MB Right now I have

How can i secure Kafka topics

Hi Can i secure Kafka topics like Apache hbase and hdfs have kerberos security mechanism ? i am not able to find out any document related to Kafka security . please suggest how can i secure Kafka . Regards Prateek

Facing Issue to create asyn producer in kafka 0.8.2

Hi I want to create async producer so i can buffer messages in queue and send after every 5 sec . my kafka version is 0.8.2.0. and i am using kafka-clients 0.8.2.0 to create kafka producer in java. below is my sample code : package com.intel.labs.ive.cloud.testKafkaProducerJ; import

facing issue in async producer application

Hi I want to create producer in async mode so i can send message in 5 sec interval. below is my code : Map props = new HashMap(); props.put(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, metadataBroker); //

Re: Facing Issue to create asyn producer in kafka 0.8.2

are using default batch.size which is ~16K and it forces > the send of messages immediately as your single message is larger than > that. Try using larger batch.size. > > Thanks > Zakee > > > > > On Oct 14, 2015, at 10:29 AM, prateek arora <prateek.arora...@gmail.c

Re: Facing Issue to create asyn producer in kafka 0.8.2

producerconfigs. You will > likely want to set the "batch.size" and "linger.ms" to achieve your goal. > > Thanks, > Grant > > On Wed, Oct 14, 2015 at 1:29 PM, prateek arora <prateek.arora...@gmail.com > > > wrote: > > > Hi > > > >