I agree with Yong, we should back-support the existing configuration, otherwise, the pulsar helm chart also need to be changed when upgrade BookKeeper 4.14.x to 4.15.x
Thanks, Hang Enrico Olivelli <eolive...@gmail.com> 于2022年10月10日周一 16:07写道： > > Il Lun 10 Ott 2022, 07:44 Yong Zhang <zhangyong1025...@gmail.com> ha > scritto: > > > We shouldn't remove any existing configuration or feature in a so short > > time. I think at least we need to back-support the existing configuration. > > If we must do the break, we should make the existing things in a > > deprecation > > state for a while. > > > > Breaking existing configurations would make a huge impact. > > Maybe we can add back the previous configurations in the next minor > > release for 4.15. So user can upgrade their cluster to 4.15.3 without > > changing any configuration. > > > > WDYT? > > > > Great idea > > Enrico > > > > Yong > > > > > > > > > > On Sun, 9 Oct 2022 at 23:20, Enrico Olivelli <eolive...@gmail.com> wrote: > > > > > Il Dom 9 Ott 2022, 17:03 Hang Chen <chenh...@apache.org> ha scritto: > > > > > > > I found the PR https://github.com/apache/bookkeeper/pull/3056 has > > > > changed the rocksDB default cache size from 10% of direct memory to > > > > 206150041(196MB), which will lead to entry read performance decrease > > > > when there are huge number of entries stored in the ledger directory. > > > > It will have a huge impact on the read performance when the BookKeeper > > > > cluster upgrades from 4.14.x to 4.15.x. > > > > > > > > > > > > > I think that that patch shoud had passed from the BP process. > > > It is a huge breaking change. > > > Personally I missed it. > > > > > > BTW now it is there, we should have talked about it in the release notes, > > > and we now must add some guidance to users upgrading to this version of > > BK. > > > > > > It is a pity that we cannot keep the automatic value. > > > BTW in all the Pulsar cluster I know I think that they are overriding > > that > > > value because the default value is not big enough. > > > > > > I think that at this point we only have to document on the website the > > > breaking change and add a page about BK and RocksDB > > > > > > Enrico > > > > > > > > > > The motivation of PR 3056 is to change RocksDB configuration to a > > > > single configuration file, and it introduces the following drawbacks. > > > > 1. Most users, especially those unfamiliar with RocksDB, will be > > > > confused about tuning RocksDB performance due to lack of guidance on > > > > some important key parameters. > > > > 2. The RocksDB blockCacheSize configuration can only be set to a fixed > > > > value, not a percentage of direct memory. > > > > > > > > In order to simplify the RocksDB configuration and make it easy to > > > > tune the performance of the RocksDB, I prefer to separate the RocksDB > > > > configuration into two parts. > > > > 1. The most important configurations, which is usually changed to tune > > > > RocksDB performance, will be located in conf/bk_server.conf > > > > 2. Other advanced configuration will be in a separate RocksDB > > > > configuration file > > > > > > > > For the default RocksDB blockCacheSize change, Do you need to trigger > > > > a new release to change it back? Current Pulsar master branch uses > > > > BookKeeper 4.14.x, and we can change the default value back to 10% of > > > > direct memory to avoid the entry read performance degradation caused > > > > by upgrading Pulsar's BookKeeper dependency to 4.15.x. > > > > > > > > Do you guys have any suggestions? > > > > > > > > Thanks, > > > > Hang > > > > > > > > >
