Re: Kafka compression - results

ent 500 million > messages? (I assumed that mln = million) > > On Wed, 12 Sep. 2018, 9:54 pm darekAsz, wrote: > > > sorry, I wrote bad results :/ > > here are correctly > > > > Directory size after sending uncompressed data: 1.5 GB > > > > Directory s

Re: Kafka compression - results

So gzip and snappy compress data, but lz4 increases data size śr., 12 wrz 2018 o 11:33 Chanchal Chatterji napisał(a): > It's not just lz4 , except in case if gzip everything else increases the > directory size. > > -Original Message----- > From: darekAsz > Sent: Wedne

Kafka compression - results

Hi I made some tests of compression in kafka. First I want to check speed of producer with compression. There are my results: with no compression: 112.58 MB/s with gzip compression: 63.24 MB/s with snappy compression: 132.43 MB/s with lz4 compression: 136.66 MB/s Then I want to check how looks

Re: Batch vs buffer differences

ched records is sent. Buffer size controls producer memory. > > > On Tue, 11 Sep. 2018, 7:16 pm darekAsz, wrote: > > > Hi > > I still learn kafka, but in some cases documentation is not so clear to > > understand all things. Can someone more experienced explain me

Batch vs buffer differences

Hi I still learn kafka, but in some cases documentation is not so clear to understand all things. Can someone more experienced explain me what is buffer and what is batch on producer side. I readed that producer stores messages in buffer before sending and also that batch mechanism is used for

Re: Kafka broker monitoring

I didn't see this page before, thanks, I will check this czw., 6 wrz 2018 o 13:55 M. Manna napisał(a): > Have you checked this wiki site - > https://cwiki.apache.org/confluence/display/KAFKA/Ecosystem ? > > On Thu, 6 Sep 2018 at 12:51, darekAsz wrote: > > > Hi, >

Kafka broker monitoring

Hi, is any tool or script to monitor a kafka broker? I want to know metrics from broker side, speed, delays, etc.

Kafka flow

Hi, I need informations about workflow in kafka, I want to know what is going on when producer send message, as much details as possible about requests-responses. Can anyone tell me when I can find this? Best regards

Re: Kafka performance tests

Hi I am working on this too and I still try to search nice sources how to optimize results. I can recommend you this article Benchmarking and this presentation

Benchmark tests of producer

Hi, I make a benchmark test of producent to check, how fast producer send data to kafka broker. There are my results: RESULTS Can someone explain me why results look like they look? I send 100 mln messages 64B size. Why early logs are much faster? I noticed that