Hello, The last thread available regarding 10GBe is about 2 years old, with no obvious recommendations on tuning.
Is there a more complex tuning guide than the example production config available on Kafka's main site? Anything other than the list of possible configs? I currently have access to a rather substantial academic cluster to test on, including multiple machines with the following hardware: 10GBe NICs 250GB RAM each SSDs on each (also, optional access to single NVMe) Using Golang's Kafka interface, I can only seem to get about 80MB/s on the producer pushing to logs on the localhost, using no replication and reading from/logging to SSD. If it helps, I can post my configs. I've tried fiddling with a bunch of broker configs as well as producer configs, raising the memory limits, max message size, io&network threads etc. Since the last post from 2014 indicates that there is no public benchmarking for 10GBe, I'd be happy to run benchmarks /publish results on this hardware if we can get it tuned up properly. What kind of broker/producer/consumer settings would you recommend? Thanks! - chris
