Hi Vineet,
Try enabling compression. That improves throughput 3-4x usually for me.
Also, you can use async mode if you're willing to trade some chance of
dropping messages for more throughput.
kafka {
codec => 'json'
broker_list => "localhost:9092"
topic_id => "blah"
compression_codec => "snappy"
request_required_acks => 1
producer_type => "async"
}
On Wed, Mar 4, 2015 at 11:03 PM, Vineet Mishra <[email protected]>
wrote:
> Hi,
>
> I am having a Logstash Forwarder which is publishing events to Kafka, but
> as I can see the rate at which the events is published to Kafka is really
> very slow.
>
> With the reference to some links I could get the Kafka Publish throughput
> reaching in 50-60Mbs per second but in my case I am hardly getting few Kbs'
> of throughput.
>
> Looking for Expert intervention to the same.
>
> Thanks,
>
