Eno Thereska created KAFKA-5174:
-----------------------------------
Summary: RocksDb might stall in environments with 1 core only
Key: KAFKA-5174
URL: https://issues.apache.org/jira/browse/KAFKA-5174
Project: Kafka
Issue Type: Bug
Components: streams
Affects Versions: 0.10.2.1
Reporter: Eno Thereska
- there is a regression in 0.10.2.1 that impacts RocksDb's behaviour (details
below for first impact) from this PR:
https://github.com/apache/kafka/commit/f9660e16dafb9e1109704bdecb978b7046f0cc6b#diff-046ca566243518c88e007b7499ec9f51
- worst-case impact: streams might not run out-of-box in deployments with 1
core only
- workaround: no new jars needed for the workaround, only a config change that
changes a RocksDb parameter, e.g. something like in this PR:
https://github.com/confluentinc/examples/pull/117/files
- full fix: add 1 to the current config (i.e., max cpus + 1) in the default
settings
The RocksDb code might have a bug itself, since when running with one core
it both sets the number of compaction threads to 1
(https://github.com/facebook/rocksdb/blob/5.0.fb/util/options.cc#L762) AND the
maximum number of compactions variable to 0
(https://github.com/facebook/rocksdb/blob/5.0.fb/util/options.cc#L760) which
doesn't make much sense. I've sent a message in RocksDb channel.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
