Re: Kafka Streams RocksDB CPU usage

2020-10-27 Thread Sophie Blee-Goldman
You might want to start with a lower commit interval, if you can handle some additional latency. I would bet that the frequent flushing is a major part of your problem: not just the act of flushing itself, but the consequences for the structure of the data in each rocksdb. If you end up flushing un

Kafka Streams RocksDB CPU usage

2020-10-27 Thread Giselle van Dongen
Hi all, We have a Kafka Streams job which has high CPU utilization. When profiling the job, we saw that this was for a large part due to RocksDB methods: flush, seek, put, get, iteratorCF. We use the default settings for our RocksDB state store. Which configuration parameters are most importan