Re: [VOTE] Release 4.14.7, release candidate #0
I opened one PR[1] to upgrade BookKeeper dependency to 4.14.7 for Pulsar branch-2.10 and all tests passed. Please help verify this candidate. Thanks a lot. Thanks, Hang [1] https://github.com/hangc0276/pulsar/pull/9 Hang Chen 于2023年2月9日周四 10:45写道： > > Hi everyone, > Please review and vote on the release candidate #0 for version 4.14.7, > as follows: > [ ] +1, Approve the release > [ ] -1, Do not approve the release (please provide specific comments) > > The complete staging area is available for your review, which includes: > * Release notes [1] > * The official Apache source and binary distributions to be deployed to > dist.apache.org [2] > * All artifacts to be deployed to the Maven Central Repository [3] > * Source code tag "v4.14.7-rc0" [4] with git sha [5] > > BookKeeper's KEYS file contains the PGP keys we used to sign this release: > https://dist.apache.org/repos/dist/release/bookkeeper/KEYS > > Please download these packages and review this release candidate: > > - Review release notes > - Download the source package (verify shasum, and asc) and follow the > instructions to build and run the bookkeeper service. > - Download the binary package (verify shasum, and asc) and follow the > instructions to run the bookkeeper service. > - Review the maven repo, release tag, licenses, and any other things you think > it is important to release. > > The vote will be open for at least 72 hours. It is adopted by the majority > approval, with at least 3 PMC affirmative votes. > > Thanks, > Release Manager > > [1] https://github.com/apache/bookkeeper/pull/3770/files > [2] https://dist.apache.org/repos/dist/dev/bookkeeper/bookkeeper-4.14.7-rc0/ > [3] > https://repository.apache.org/content/repositories/orgapachebookkeeper-1078 > [4] https://github.com/apache/bookkeeper/tree/v4.14.7-rc0 > [5] 3c85422a9497d168cadc5d264ebccbe0fadf3660
[VOTE] Release 4.14.7, release candidate #0
Hi everyone, Please review and vote on the release candidate #0 for version 4.14.7, as follows: [ ] +1, Approve the release [ ] -1, Do not approve the release (please provide specific comments) The complete staging area is available for your review, which includes: * Release notes [1] * The official Apache source and binary distributions to be deployed to dist.apache.org [2] * All artifacts to be deployed to the Maven Central Repository [3] * Source code tag "v4.14.7-rc0" [4] with git sha [5] BookKeeper's KEYS file contains the PGP keys we used to sign this release: https://dist.apache.org/repos/dist/release/bookkeeper/KEYS Please download these packages and review this release candidate: - Review release notes - Download the source package (verify shasum, and asc) and follow the instructions to build and run the bookkeeper service. - Download the binary package (verify shasum, and asc) and follow the instructions to run the bookkeeper service. - Review the maven repo, release tag, licenses, and any other things you think it is important to release. The vote will be open for at least 72 hours. It is adopted by the majority approval, with at least 3 PMC affirmative votes. Thanks, Release Manager [1] https://github.com/apache/bookkeeper/pull/3770/files [2] https://dist.apache.org/repos/dist/dev/bookkeeper/bookkeeper-4.14.7-rc0/ [3] https://repository.apache.org/content/repositories/orgapachebookkeeper-1078 [4] https://github.com/apache/bookkeeper/tree/v4.14.7-rc0 [5] 3c85422a9497d168cadc5d264ebccbe0fadf3660
Re: [Discuss] BookKeeper upgrade the RocksDB version to 7.9.x
+1 Enrico Il giorno mer 8 feb 2023 alle ore 17:44 Andrey Yegorov ha scritto: > > +1 to the plan. > We can configure RocksDb to not use kXXH3 in all branches and provide a > config option for the user to enable it when needed. > > On Sun, Jan 29, 2023 at 2:54 AM Hang Chen wrote: > > > Hi guys, > > When we introduced `deleteRange` for ledger index deletion, we > > found a segment fault in ledger index deletion [1]. I suggest > > upgrading the RocksDB version to the latest 7.9.x to see if the > > RocksDB deleteRange segment fault bug has been fixed. I have tested > > upgrading the RocksDB version from 6.10.2 to 7.9.2 and rollback to > > 6.10.2. This is the testing result. > > > > - The upgrade process works fine in the following cases > > - Pulsar producer keeps producing messages to Pulsar topic > > - Pulsar consumer consumes messages from the Pulsar topic, and the > > Pulsar broker fetches messages from the BookKeeper cluster > > - Trigger compaction to cleanup expired ledgers > > > > - Rollback RocksDB version from 7.9.2 to 6.10.2, and bookie failed to > > startup with the following exception. > > > > ```java > > 2023-01-29T17:09:27,794+0800 [main] ERROR > > org.apache.bookkeeper.server.Main - Failed to build bookie server > > java.io.IOException: Error open RocksDB database > > at > > org.apache.bookkeeper.bookie.storage.ldb.KeyValueStorageRocksDB.(KeyValueStorageRocksDB.java:200) > > ~[org.apache.bookkeeper-bookkeeper-server-4.14.6.jar:4.14.6] > > at > > org.apache.bookkeeper.bookie.storage.ldb.KeyValueStorageRocksDB.(KeyValueStorageRocksDB.java:89) > > ~[org.apache.bookkeeper-bookkeeper-server-4.14.6.jar:4.14.6] > > at > > org.apache.bookkeeper.bookie.storage.ldb.KeyValueStorageRocksDB.lambda$static$0(KeyValueStorageRocksDB.java:63) > > ~[org.apache.bookkeeper-bookkeeper-server-4.14.6.jar:4.14.6] > > at > > org.apache.bookkeeper.bookie.storage.ldb.LedgerMetadataIndex.(LedgerMetadataIndex.java:68) > > ~[org.apache.bookkeeper-bookkeeper-server-4.14.6.jar:4.14.6] > > at > > org.apache.bookkeeper.bookie.storage.ldb.SingleDirectoryDbLedgerStorage.(SingleDirectoryDbLedgerStorage.java:170) > > ~[org.apache.bookkeeper-bookkeeper-server-4.14.6.jar:4.14.6] > > at > > org.apache.bookkeeper.bookie.storage.ldb.DbLedgerStorage.newSingleDirectoryDbLedgerStorage(DbLedgerStorage.java:150) > > ~[org.apache.bookkeeper-bookkeeper-server-4.14.6.jar:4.14.6] > > at > > org.apache.bookkeeper.bookie.storage.ldb.DbLedgerStorage.initialize(DbLedgerStorage.java:129) > > ~[org.apache.bookkeeper-bookkeeper-server-4.14.6.jar:4.14.6] > > at org.apache.bookkeeper.bookie.Bookie.(Bookie.java:819) > > ~[org.apache.bookkeeper-bookkeeper-server-4.14.6.jar:4.14.6] > > at > > org.apache.bookkeeper.proto.BookieServer.newBookie(BookieServer.java:152) > > ~[org.apache.bookkeeper-bookkeeper-server-4.14.6.jar:4.14.6] > > at > > org.apache.bookkeeper.proto.BookieServer.(BookieServer.java:120) > > ~[org.apache.bookkeeper-bookkeeper-server-4.14.6.jar:4.14.6] > > at > > org.apache.bookkeeper.server.service.BookieService.(BookieService.java:52) > > ~[org.apache.bookkeeper-bookkeeper-server-4.14.6.jar:4.14.6] > > at > > org.apache.bookkeeper.server.Main.buildBookieServer(Main.java:304) > > ~[org.apache.bookkeeper-bookkeeper-server-4.14.6.jar:4.14.6] > > at org.apache.bookkeeper.server.Main.doMain(Main.java:226) > > ~[org.apache.bookkeeper-bookkeeper-server-4.14.6.jar:4.14.6] > > at org.apache.bookkeeper.server.Main.main(Main.java:208) > > ~[org.apache.bookkeeper-bookkeeper-server-4.14.6.jar:4.14.6] > > Caused by: org.rocksdb.RocksDBException: unknown checksum type 4 in > > data/bookkeeper/ledgers/current/ledgers/25.sst offset 1078 size 33 > > at org.rocksdb.RocksDB.open(Native Method) > > ~[org.rocksdb-rocksdbjni-6.10.2.jar:?] > > at org.rocksdb.RocksDB.open(RocksDB.java:239) > > ~[org.rocksdb-rocksdbjni-6.10.2.jar:?] > > at > > org.apache.bookkeeper.bookie.storage.ldb.KeyValueStorageRocksDB.(KeyValueStorageRocksDB.java:197) > > ~[org.apache.bookkeeper-bookkeeper-server-4.14.6.jar:4.14.6] > > ... 13 more > > ``` > > The root cause of this exception is that RocksDB 7.9.2 uses `kXXH3` > > by default, and `kXXH3` is only supported since RocksDB 6.27 > > > > https://github.com/facebook/rocksdb/blob/79e57a39a33dbe17c8f51167e40e66d6c91f8eb4/include/rocksdb/table.h#L56 > > > > For the BookKeeper master branch, we have upgraded the RocksDB to > > `6.29.4.1`, which can support RocksDB upgrade to 7.9.2 and rollback to > > 6.29.4.1. > > For the RocksDB < 6.27, we can push a fix to ensure RocksDB 7.9.2 does > > not use the latest checksum type `kXXH3` > > > > I suggest doing this upgradation. Do you have any concerns? > > > > [1] https://github.com/apache/bookkeeper/issues/3734 > > > > Thanks, > > Hang > > > > > -- > Andrey Yegorov
Re: [Discuss] BookKeeper upgrade the RocksDB version to 7.9.x
+1 to the plan. We can configure RocksDb to not use kXXH3 in all branches and provide a config option for the user to enable it when needed. On Sun, Jan 29, 2023 at 2:54 AM Hang Chen wrote: > Hi guys, > When we introduced `deleteRange` for ledger index deletion, we > found a segment fault in ledger index deletion [1]. I suggest > upgrading the RocksDB version to the latest 7.9.x to see if the > RocksDB deleteRange segment fault bug has been fixed. I have tested > upgrading the RocksDB version from 6.10.2 to 7.9.2 and rollback to > 6.10.2. This is the testing result. > > - The upgrade process works fine in the following cases > - Pulsar producer keeps producing messages to Pulsar topic > - Pulsar consumer consumes messages from the Pulsar topic, and the > Pulsar broker fetches messages from the BookKeeper cluster > - Trigger compaction to cleanup expired ledgers > > - Rollback RocksDB version from 7.9.2 to 6.10.2, and bookie failed to > startup with the following exception. > > ```java > 2023-01-29T17:09:27,794+0800 [main] ERROR > org.apache.bookkeeper.server.Main - Failed to build bookie server > java.io.IOException: Error open RocksDB database > at > org.apache.bookkeeper.bookie.storage.ldb.KeyValueStorageRocksDB.(KeyValueStorageRocksDB.java:200) > ~[org.apache.bookkeeper-bookkeeper-server-4.14.6.jar:4.14.6] > at > org.apache.bookkeeper.bookie.storage.ldb.KeyValueStorageRocksDB.(KeyValueStorageRocksDB.java:89) > ~[org.apache.bookkeeper-bookkeeper-server-4.14.6.jar:4.14.6] > at > org.apache.bookkeeper.bookie.storage.ldb.KeyValueStorageRocksDB.lambda$static$0(KeyValueStorageRocksDB.java:63) > ~[org.apache.bookkeeper-bookkeeper-server-4.14.6.jar:4.14.6] > at > org.apache.bookkeeper.bookie.storage.ldb.LedgerMetadataIndex.(LedgerMetadataIndex.java:68) > ~[org.apache.bookkeeper-bookkeeper-server-4.14.6.jar:4.14.6] > at > org.apache.bookkeeper.bookie.storage.ldb.SingleDirectoryDbLedgerStorage.(SingleDirectoryDbLedgerStorage.java:170) > ~[org.apache.bookkeeper-bookkeeper-server-4.14.6.jar:4.14.6] > at > org.apache.bookkeeper.bookie.storage.ldb.DbLedgerStorage.newSingleDirectoryDbLedgerStorage(DbLedgerStorage.java:150) > ~[org.apache.bookkeeper-bookkeeper-server-4.14.6.jar:4.14.6] > at > org.apache.bookkeeper.bookie.storage.ldb.DbLedgerStorage.initialize(DbLedgerStorage.java:129) > ~[org.apache.bookkeeper-bookkeeper-server-4.14.6.jar:4.14.6] > at org.apache.bookkeeper.bookie.Bookie.(Bookie.java:819) > ~[org.apache.bookkeeper-bookkeeper-server-4.14.6.jar:4.14.6] > at > org.apache.bookkeeper.proto.BookieServer.newBookie(BookieServer.java:152) > ~[org.apache.bookkeeper-bookkeeper-server-4.14.6.jar:4.14.6] > at > org.apache.bookkeeper.proto.BookieServer.(BookieServer.java:120) > ~[org.apache.bookkeeper-bookkeeper-server-4.14.6.jar:4.14.6] > at > org.apache.bookkeeper.server.service.BookieService.(BookieService.java:52) > ~[org.apache.bookkeeper-bookkeeper-server-4.14.6.jar:4.14.6] > at > org.apache.bookkeeper.server.Main.buildBookieServer(Main.java:304) > ~[org.apache.bookkeeper-bookkeeper-server-4.14.6.jar:4.14.6] > at org.apache.bookkeeper.server.Main.doMain(Main.java:226) > ~[org.apache.bookkeeper-bookkeeper-server-4.14.6.jar:4.14.6] > at org.apache.bookkeeper.server.Main.main(Main.java:208) > ~[org.apache.bookkeeper-bookkeeper-server-4.14.6.jar:4.14.6] > Caused by: org.rocksdb.RocksDBException: unknown checksum type 4 in > data/bookkeeper/ledgers/current/ledgers/25.sst offset 1078 size 33 > at org.rocksdb.RocksDB.open(Native Method) > ~[org.rocksdb-rocksdbjni-6.10.2.jar:?] > at org.rocksdb.RocksDB.open(RocksDB.java:239) > ~[org.rocksdb-rocksdbjni-6.10.2.jar:?] > at > org.apache.bookkeeper.bookie.storage.ldb.KeyValueStorageRocksDB.(KeyValueStorageRocksDB.java:197) > ~[org.apache.bookkeeper-bookkeeper-server-4.14.6.jar:4.14.6] > ... 13 more > ``` > The root cause of this exception is that RocksDB 7.9.2 uses `kXXH3` > by default, and `kXXH3` is only supported since RocksDB 6.27 > > https://github.com/facebook/rocksdb/blob/79e57a39a33dbe17c8f51167e40e66d6c91f8eb4/include/rocksdb/table.h#L56 > > For the BookKeeper master branch, we have upgraded the RocksDB to > `6.29.4.1`, which can support RocksDB upgrade to 7.9.2 and rollback to > 6.29.4.1. > For the RocksDB < 6.27, we can push a fix to ensure RocksDB 7.9.2 does > not use the latest checksum type `kXXH3` > > I suggest doing this upgradation. Do you have any concerns? > > [1] https://github.com/apache/bookkeeper/issues/3734 > > Thanks, > Hang > -- Andrey Yegorov
Re: [DISCUSS] BookKeeper 4.14.7 release
+1 Thanks ZhangJian He On Wed, 8 Feb 2023 at 16:45, steven lu wrote: > +1 > > Hang Chen 于2023年2月8日周三 15:16写道： > > > Hi everyone, > > > > One core dump issue[1] related to RocksDB was introduced by [2] and > > released in BookKeeper 4.14.6. To fix the RocksDB core dump issue, I > > pushed one PR [3] to revert the [2] to make branch-4.14 stable. > > > > Due to BookKeeper 4.14.6 having the risk of stability, I would like to > > discuss starting the 4.14.7 release for those critical bug fixes. > > > > Here are the PRs for 4.14.7: > > > > > https://github.com/apache/bookkeeper/pulls?q=is%3Apr+label%3Arelease%2F4.14.7+is%3Aclosed > > > > If you have PRs that want to be included in this release, please > > comment on that PR or tag the label `release/4.14.7`. > > > > After those PRs merged, I would like to volunteer to release this patch. > > > > Thanks, > > Hang > > > > [1] https://github.com/apache/bookkeeper/issues/3734 > > [2] https://github.com/apache/bookkeeper/pull/3653 > > [3] https://github.com/apache/bookkeeper/pull/3768 > > >
Re: [DISCUSS] BookKeeper 4.14.7 release
+1 Hang Chen 于2023年2月8日周三 15:16写道： > Hi everyone, > > One core dump issue[1] related to RocksDB was introduced by [2] and > released in BookKeeper 4.14.6. To fix the RocksDB core dump issue, I > pushed one PR [3] to revert the [2] to make branch-4.14 stable. > > Due to BookKeeper 4.14.6 having the risk of stability, I would like to > discuss starting the 4.14.7 release for those critical bug fixes. > > Here are the PRs for 4.14.7: > > https://github.com/apache/bookkeeper/pulls?q=is%3Apr+label%3Arelease%2F4.14.7+is%3Aclosed > > If you have PRs that want to be included in this release, please > comment on that PR or tag the label `release/4.14.7`. > > After those PRs merged, I would like to volunteer to release this patch. > > Thanks, > Hang > > [1] https://github.com/apache/bookkeeper/issues/3734 > [2] https://github.com/apache/bookkeeper/pull/3653 > [3] https://github.com/apache/bookkeeper/pull/3768 >
Re: [DISCUSS] BookKeeper 4.14.7 release
+1 Yong On Wed, 8 Feb 2023 at 16:17, Enrico Olivelli wrote: > +1 > > thanks > Enrico > > Il giorno mer 8 feb 2023 alle ore 08:16 Hang Chen > ha scritto: > > > > Hi everyone, > > > > One core dump issue[1] related to RocksDB was introduced by [2] and > > released in BookKeeper 4.14.6. To fix the RocksDB core dump issue, I > > pushed one PR [3] to revert the [2] to make branch-4.14 stable. > > > > Due to BookKeeper 4.14.6 having the risk of stability, I would like to > > discuss starting the 4.14.7 release for those critical bug fixes. > > > > Here are the PRs for 4.14.7: > > > https://github.com/apache/bookkeeper/pulls?q=is%3Apr+label%3Arelease%2F4.14.7+is%3Aclosed > > > > If you have PRs that want to be included in this release, please > > comment on that PR or tag the label `release/4.14.7`. > > > > After those PRs merged, I would like to volunteer to release this patch. > > > > Thanks, > > Hang > > > > [1] https://github.com/apache/bookkeeper/issues/3734 > > [2] https://github.com/apache/bookkeeper/pull/3653 > > [3] https://github.com/apache/bookkeeper/pull/3768 >
Re: [DISCUSS] BookKeeper 4.14.7 release
+1 thanks Enrico Il giorno mer 8 feb 2023 alle ore 08:16 Hang Chen ha scritto: > > Hi everyone, > > One core dump issue[1] related to RocksDB was introduced by [2] and > released in BookKeeper 4.14.6. To fix the RocksDB core dump issue, I > pushed one PR [3] to revert the [2] to make branch-4.14 stable. > > Due to BookKeeper 4.14.6 having the risk of stability, I would like to > discuss starting the 4.14.7 release for those critical bug fixes. > > Here are the PRs for 4.14.7: > https://github.com/apache/bookkeeper/pulls?q=is%3Apr+label%3Arelease%2F4.14.7+is%3Aclosed > > If you have PRs that want to be included in this release, please > comment on that PR or tag the label `release/4.14.7`. > > After those PRs merged, I would like to volunteer to release this patch. > > Thanks, > Hang > > [1] https://github.com/apache/bookkeeper/issues/3734 > [2] https://github.com/apache/bookkeeper/pull/3653 > [3] https://github.com/apache/bookkeeper/pull/3768
