Il Dom 9 Ott 2022, 17:03 Hang Chen <chenh...@apache.org> ha scritto: > I found the PR https://github.com/apache/bookkeeper/pull/3056 has > changed the rocksDB default cache size from 10% of direct memory to > 206150041(196MB), which will lead to entry read performance decrease > when there are huge number of entries stored in the ledger directory. > It will have a huge impact on the read performance when the BookKeeper > cluster upgrades from 4.14.x to 4.15.x. >
I think that that patch shoud had passed from the BP process. It is a huge breaking change. Personally I missed it. BTW now it is there, we should have talked about it in the release notes, and we now must add some guidance to users upgrading to this version of BK. It is a pity that we cannot keep the automatic value. BTW in all the Pulsar cluster I know I think that they are overriding that value because the default value is not big enough. I think that at this point we only have to document on the website the breaking change and add a page about BK and RocksDB Enrico > The motivation of PR 3056 is to change RocksDB configuration to a > single configuration file, and it introduces the following drawbacks. > 1. Most users, especially those unfamiliar with RocksDB, will be > confused about tuning RocksDB performance due to lack of guidance on > some important key parameters. > 2. The RocksDB blockCacheSize configuration can only be set to a fixed > value, not a percentage of direct memory. > > In order to simplify the RocksDB configuration and make it easy to > tune the performance of the RocksDB, I prefer to separate the RocksDB > configuration into two parts. > 1. The most important configurations, which is usually changed to tune > RocksDB performance, will be located in conf/bk_server.conf > 2. Other advanced configuration will be in a separate RocksDB > configuration file > > For the default RocksDB blockCacheSize change, Do you need to trigger > a new release to change it back? Current Pulsar master branch uses > BookKeeper 4.14.x, and we can change the default value back to 10% of > direct memory to avoid the entry read performance degradation caused > by upgrading Pulsar's BookKeeper dependency to 4.15.x. > > Do you guys have any suggestions? > > Thanks, > Hang >
