Re: 答复: kafka performance question

2014-05-26 Thread svante karlsson
: robairrob...@gmail.com [mailto:robairrob...@gmail.com] 代表 Robert Turner 发送时间: 2014年5月16日 13:45 收件人: users@kafka.apache.org 主题: Re: kafka performance question A couple of thoughts spring to mind, are you sending the whole file as 1 message? and is your producer code using sync or async mode

答复: kafka performance question

2014-05-25 Thread Zhujie (zhujie, Smartcare)
: Re: kafka performance question A couple of thoughts spring to mind, are you sending the whole file as 1 message? and is your producer code using sync or async mode? Cheers Rob. On 14 May 2014 15:49, Jun Rao jun...@gmail.com wrote: How many brokers and partitions do you have? You may try

Re: kafka performance question

2014-05-16 Thread Robert Turner
A couple of thoughts spring to mind, are you sending the whole file as 1 message? and is your producer code using sync or async mode? Cheers Rob. On 14 May 2014 15:49, Jun Rao jun...@gmail.com wrote: How many brokers and partitions do you have? You may try increasing batch.num.messages.

Re: kafka performance question

2014-05-15 Thread Jun Rao
How many brokers and partitions do you have? You may try increasing batch.num.messages. Thanks, Jun On Tue, May 13, 2014 at 5:56 PM, Zhujie (zhujie, Smartcare) first.zhu...@huawei.com wrote: Dear all, We want to use kafka to collect and dispatch data file, but the performance is maybe

kafka performance question

2014-05-13 Thread Zhujie (zhujie, Smartcare)
Dear all, We want to use kafka to collect and dispatch data file, but the performance is maybe lower than we want. In our cluster,there is a provider and a broker. We use a one thread read file from local disk of provider and send it to broker. The average throughput is only 3 MB/S~4MB/S. But