Re: [DISCUSS] Proposal to Backport `dbStorage_readAheadCacheBatchBytesSize` to BookKeeper 4.16.x

[Hang Chen]

Thanks for your voting. I created the PR[1] to backport this feature.

[1] https://github.com/apache/bookkeeper/pull/4463

Best,
Hang

ZhangJian He  于2024年6月28日周五 08:27写道：
>
> +1, I revisit this pr, it's quite simple and default disabled.
>
> Thanks
> ZhangJian He
> Twitter: shoothzj
> Wechat: shoothzj
>
> On Fri, Jun 28, 2024 at 12:01 AM Matteo Merli  wrote:
> >
> > Sounds good to me. This won't add any new risk.
> > --
> > Matteo Merli
> > 
> >
> >
> > On Thu, Jun 27, 2024 at 4:30 PM Hang Chen  wrote:
> >
> > > Hi team,
> > >
> > > I would like to discuss the potential backporting of a recently
> > > introduced feature from the BookKeeper 4.17.0 release to the 4.16.x
> > > branch, which is used in Pulsar 3.0 LTS. The feature in question is
> > > the `dbStorage_readAheadCacheBatchBytesSize` configuration option,
> > > added through PR [1]. This configuration limits the prefetch size of
> > > the server read cache from the ledger disk.
> > >
> > > Upon testing, I found that this new configuration significantly
> > > improves catch-up read latency. Given that Pulsar 3.0 is a Long-Term
> > > Support (LTS) version and relies on BookKeeper 4.16.x, incorporating
> > > this feature could provide substantial benefits to users of this
> > > version.
> > >
> > > Importantly, this configuration is disabled by default, ensuring that
> > > its introduction to the 4.16.x branch would not lead to breaking
> > > changes. This makes it a safe candidate for cherry-picking to
> > > branch-4.16.
> > >
> > > I believe this enhancement will provide a tangible improvement to the
> > > performance of Pulsar 3.0 LTS. I would appreciate any feedback or
> > > concerns regarding this proposal.
> > >
> > > Thank you for considering this improvement.
> > >
> > > Best regards,
> > > Hang
> > >
> > >
> > >
> > > [1] https://github.com/apache/bookkeeper/pull/3895
> > >

[DISCUSS] Proposal to Backport `dbStorage_readAheadCacheBatchBytesSize` to BookKeeper 4.16.x

[Hang Chen]

Hi team,

I would like to discuss the potential backporting of a recently
introduced feature from the BookKeeper 4.17.0 release to the 4.16.x
branch, which is used in Pulsar 3.0 LTS. The feature in question is
the `dbStorage_readAheadCacheBatchBytesSize` configuration option,
added through PR [1]. This configuration limits the prefetch size of
the server read cache from the ledger disk.

Upon testing, I found that this new configuration significantly
improves catch-up read latency. Given that Pulsar 3.0 is a Long-Term
Support (LTS) version and relies on BookKeeper 4.16.x, incorporating
this feature could provide substantial benefits to users of this
version.

Importantly, this configuration is disabled by default, ensuring that
its introduction to the 4.16.x branch would not lead to breaking
changes. This makes it a safe candidate for cherry-picking to
branch-4.16.

I believe this enhancement will provide a tangible improvement to the
performance of Pulsar 3.0 LTS. I would appreciate any feedback or
concerns regarding this proposal.

Thank you for considering this improvement.

Best regards,
Hang



[1] https://github.com/apache/bookkeeper/pull/3895

Re: [DISCUSS] BookKeeper 4.17.1 release

[Hang Chen]

+1

Thanks,
Hang

ZhangJian He  于2024年6月11日周二 07:52写道：
>
> I have done cherry-pick, no pr in
> https://github.com/apache/bookkeeper/pulls?q=is%3Apr+label%3Arelease%2F4.17.1+is%3Amerged+-label%3Acherry-picked%2Fbranch-4.17
> .
>
> I'll start preparing for the RC as soon as I can.
>
> Thanks
> ZhangJian He
> Twitter: shoothzj
> Wechat: shoothzj
>
>
> On Thu, Jun 6, 2024 at 7:35 AM ZhangJian He  wrote:
>
> > TLDR: I would like to cherry-pick
> > https://github.com/apache/bookkeeper/pull/4413 and
> > https://github.com/apache/bookkeeper/pull/4419
> >
> > > https://github.com/apache/bookkeeper/pull/4407
> > I think currently windows is limited to some development jobs, there are
> > still many test failures to be solved. No need to cherry-pick.
> > > https://github.com/apache/bookkeeper/pull/4413
> > RocksDB release note:
> > https://github.com/facebook/rocksdb/releases/tag/v7.10.2 It contains many
> > bug fix, no public API changes.I think we can cherry-pick.
> > > https://github.com/apache/bookkeeper/pull/4419
> > BookKeeper CI branch-4.17
> > 
> >  Although
> > branch-4.17 CI works well, I still think it's worth cherry-picking. It
> > actually solves a problem.
> >
> > Thanks
> > ZhangJian He
> > Twitter: shoothzj
> > Wechat: shoothzj
> >
> >
> > On Thu, Jun 6, 2024 at 12:41 AM Andrey Yegorov 
> > wrote:
> >
> >> +1
> >>
> >> WDYT about adding these recent PRs:
> >> https://github.com/apache/bookkeeper/pull/4407
> >> https://github.com/apache/bookkeeper/pull/4413
> >> https://github.com/apache/bookkeeper/pull/4419
> >>
> >>
> >> On Fri, May 24, 2024 at 7:48 PM ZhangJian He  wrote:
> >>
> >> > Hi all,
> >> >
> >> > BookKeeper 4.17.0 was released one months ago, and I would
> >> > like to discuss the starting 4.17.1 release to include some critical
> >> > security and bug fixes.
> >> >
> >> > Here are the PRs for 4.17.1:
> >> > PR-4.17.1
> >> > <
> >> >
> >> https://github.com/apache/bookkeeper/pulls?q=is%3Apr+label%3Arelease%2F4.17.1+is%3Amerged
> >> > >
> >> >
> >> > I have cherry-picked it. I removed open PRs that were targeting 4.17.1
> >> > label.
> >> >
> >> > If you have other PRs that you want to be included in this release,
> >> please
> >> > tag the PR with the "release/4.17.1" and let me know in this thread.
> >> >
> >> > If there's any concern, I'd like to volunteer to guide this release.
> >> >
> >> > Thanks
> >> > ZhangJian He
> >> > Twitter: shoothzj
> >> > Wechat: shoothzj
> >> >
> >>
> >

Re: [DISCUSS] Introducing Typo CI Check to Apache BookKeeper

[Hang Chen]

+1

We need to keep an eye on the public methods and configurations.

Best,
Hang

Andrey Yegorov  于2024年5月22日周三 02:15写道：
>
>
> Awesome idea, thank you for tackling this.
>
> We need to keep an eye on changes in config parameter names in yaml/configs 
> to avoid unexpected breaks.
> So far I haven't noticed any, but it is an easy miss in a big PR.
>
> On 2024/05/19 07:20:30 ZhangJian He wrote:
> > Hi, I want to introduce new lint tools: typos[1]. This tool is designed to
> > automatically detect and correct typographical errors in our codebase,
> > documentation, and comments. It’s a valuable addition to ensure
> > high-quality contributions and maintain the professional standard of our
> > project.
> >
> > It has also been used in the Apache OpenDAL and Kvrocks projects. I created
> > a draft PR[2].
> >
> > I have been careful to avoid changes to public interfaces, but I may have
> > made mistakes, and the changes are relatively huge. I think I would prefer
> > to proceed as follows:
> >
> > First, commit typos in markdown, YAML, etc.
> > Second, commit typos in test files.
> > Lastly, fix other typos and commit the lint.
> >
> >
> > 1. https://github.com/crate-ci/typos
> > 2. https://github.com/apache/bookkeeper/pull/4375/files
> >
> > Thanks
> > ZhangJian He
> > Twitter: shoothzj
> > Wechat: shoothzj
> >

Re: [VOTE] Release 4.17.0, release candidate #0

[Hang Chen]

+1 (binding)

- Verified signatures and checksums
- Build from the source code with JDK 17, Maven 3.8.6 on M3 macbook
- All the StreamNative tests are passed
- Build Pulsar branch-3.2 with the staging packages
- Run Pulsar cluster with one broker, one bookie and one zookeeper, verified:
  - Produce messages to Pulsar topics
  - Offload topic data to Lakehouse with Lakehouse tiered storage
  - Pulsar consumer consume messages from Pulsar topics


Thanks for driving the release

Hang

Andrey Yegorov  于2024年4月10日周三 01:19写道：
>
> +1 (binding)
>
> - built from tag
> - built from archived sources
> - ran small subset of unit tests
> - ran standalone andlocalbookie from packaged bookkeeper-server
> - spot-checked staged maven repo
> - reviewed release notes
>
> Release notes need double-check, I found one PR listed as a "fix" while 
> actually the PR was closed without the merge. The rest lgtm.
>
> Thank you.
> --
> Andrey
>
> On 2024/04/02 08:33:56 Enrico Olivelli wrote:
> > Hi everyone,
> > Please review and vote on the release candidate #0 for the version
> > 4.17.0, as follows:
> > [ ] +1, Approve the release
> > [ ] -1, Do not approve the release (please provide specific comments)
> >
> > The complete staging area is available for your review, which includes:
> > * Release notes [1]
> > * The official Apache source and binary distributions to be deployed
> > to dist.apache.org [2]
> > * All artifacts to be deployed to the Maven Central Repository [3]
> > * Source code tag "v4.17.0-rc0"  [4] with git sha
> > cce4b6461691466c663f2cb4d00dd4d73dd9071e
> >
> > BookKeeper's KEYS file contains PGP keys we used to sign this release:
> > https://dist.apache.org/repos/dist/release/bookkeeper/KEYS
> >
> > Please download these packages and review this release candidate:
> >
> > - Review release notes
> > - Download the source package (verify shasum, and asc) and follow the
> > instructions to build and run the bookkeeper service.
> > - Download the binary package (verify shasum, and asc) and follow the
> > instructions to run the bookkeeper service.
> > - Review maven repo, release tag, licenses, and any other things you think
> > it is important to release.
> >
> > The vote will be open for at least 72 hours. It is adopted by majority
> > approval, with at least 3 PMC affirmative votes.
> >
> > Thanks,
> > Enrico Olivelli
> >
> > [1] 
> > https://github.com/apache/bookkeeper/pull/4260/files#diff-d361215d5d5abaa96ac3362e70a1d2f80f92d369a59c866e1976ff41ec435f53
> > [2] https://dist.apache.org/repos/dist/dev/bookkeeper/bookkeeper-4.17.0-rc0/
> > [3] 
> > https://repository.apache.org/content/repositories/orgapachebookkeeper-1095/
> > [4] https://github.com/apache/bookkeeper/tree/v4.17.0-rc0
> >

Re: [VOTE] Release 4.16.5, release candidate #1

[Hang Chen]

+1 (binding)

- Verified signatures and checksums
- Build from the source code with JDK 17, Maven 3.8.6 on M3 macbook
- All the StreamNative tests are passed
- Build Pulsar branch-3.0 with the staging packages
- Run Pulsar cluster with one broker, one bookie and one zookeeper, verified:
  - Produce messages to Pulsar topics
  - Offload topic data to Lakehouse with Lakehouse tiered storage
  - Pulsar consumer consume messages from Pulsar topics


Thanks for driving the release

Hang

Enrico Olivelli  于2024年4月9日周二 14:53写道：
>
> +1 (binding)
>
> - Veriified signatures and checksums
> - HerdDB tests are passing https://github.com/diennea/herddb/pull/828
> - built locally from sources and tests local binaries
>
> Thanks for driving the release
>
> Enrico
>
> Il giorno ven 5 apr 2024 alle ore 15:03 Lari Hotari
>  ha scritto:
> >
> > +1 (non-binding)
> >
> > - signatures ok
> > - checksums ok
> > - Pulsar master branch runs fine with staged maven artifacts
> > - https://github.com/lhotari/pulsar/pull/178
> > - ran simpletest with bookkeeper-all-4.16.5 binaries
> > ("bookkeeper localbookie 3" and "bookkeeper shell simpletest")
> > - source package compilation succeeds
> >
> > -Lari
> >
> > On 2024/04/03 12:43:45 Nicolò Boschi wrote:
> > > Hi everyone,
> > > Please review and vote on the release candidate #1 for the version 4.16.5,
> > > as follows:
> > > [ ] +1, Approve the release
> > > [ ] -1, Do not approve the release (please provide specific comments)
> > >
> > > The complete staging area is available for your review, which includes:
> > > * Release notes [1]
> > > * The official Apache source and binary distributions to be deployed to
> > > dist.apache.org [2]
> > > * All artifacts to be deployed to the Maven Central Repository [3]
> > > * Source code tag "v4.16.5-rc1" [4] with git sha 7018e3a
> > >
> > > BookKeeper's KEYS file contains PGP keys we used to sign this release:
> > > https://dist.apache.org/repos/dist/release/bookkeeper/KEYS
> > >
> > > Please download these packages and review this release candidate:
> > >
> > > - Review release notes
> > > - Download the source package (verify shasum, and asc) and follow the
> > > instructions to build and run the bookkeeper service.
> > > - Download the binary package (verify shasum, and asc) and follow the
> > > instructions to run the bookkeeper service.
> > > - Review maven repo, release tag, licenses, and any other things you think
> > > it is important for a release.
> > >
> > > The vote will be open for at least 72 hours. It is adopted by majority
> > > approval, with at least 3 PMC affirmative votes.
> > >
> > > Thanks,
> > > Release Manager
> > >
> > > [1] https://github.com/apache/bookkeeper/pull/4249
> > > [2] 
> > > https://dist.apache.org/repos/dist/dev/bookkeeper/bookkeeper-4.16.5-rc1/
> > > [3]
> > > https://repository.apache.org/content/repositories/orgapachebookkeeper-1096/
> > > [4] https://github.com/apache/bookkeeper/tree/v4.16.5-rc1
> > >

Re: [DISCUSS] Bookkeeper 4.17.0 release

[Hang Chen]

Hi Enrico,
Thank you for volunteering to take on the role of release manager
for this cycle. Your willingness to contribute in this capacity is
greatly appreciated.

Best,
Hang

Enrico Olivelli  于2024年2月27日周二 20:23写道：
>
> Hello Hang,
> Thanks for bringing this up
>
> I would be happy to be the release manager for this cycle
>
> Enrico
>
> Il Mer 21 Feb 2024, 07:37 Hang Chen  ha scritto:
>
> > Hi Community,
> >Bookkeeper 4.16.0 has been released for more than 9 months, and we
> > have introduced batch read support [1] for entry reading and many
> > important bug fixes[2] and improvements[3][4][5]. I would like to
> > start to discuss releasing 4.17.0. All the PRs in 4.17.0 can be found
> > at
> > https://github.com/apache/bookkeeper/pulls?q=is%3Aopen+is%3Apr+milestone%3A4.17.0
> >
> >Do you have any concerns? Do you have any other issue/ticket to add
> > for 4.17.0? You can add the target PRs into Milestone 4.17.0 if you
> > want to include them into 4.17.0.
> >
> >
> > [1] https://github.com/apache/bookkeeper/pull/4051
> > [2] https://github.com/apache/bookkeeper/pull/4140
> > [3] https://github.com/apache/bookkeeper/pull/4211
> > [4] https://github.com/apache/bookkeeper/pull/4139
> > [5] https://github.com/apache/bookkeeper/pull/4038
> >
> > Regards,
> > Hang
> >

[DISCUSS] Bookkeeper 4.17.0 release

[Hang Chen]

Hi Community,
   Bookkeeper 4.16.0 has been released for more than 9 months, and we
have introduced batch read support [1] for entry reading and many
important bug fixes[2] and improvements[3][4][5]. I would like to
start to discuss releasing 4.17.0. All the PRs in 4.17.0 can be found
at 
https://github.com/apache/bookkeeper/pulls?q=is%3Aopen+is%3Apr+milestone%3A4.17.0

   Do you have any concerns? Do you have any other issue/ticket to add
for 4.17.0? You can add the target PRs into Milestone 4.17.0 if you
want to include them into 4.17.0.


[1] https://github.com/apache/bookkeeper/pull/4051
[2] https://github.com/apache/bookkeeper/pull/4140
[3] https://github.com/apache/bookkeeper/pull/4211
[4] https://github.com/apache/bookkeeper/pull/4139
[5] https://github.com/apache/bookkeeper/pull/4038

Regards,
Hang

Re: [VOTE] Release 4.16.4, release candidate #0

[Hang Chen]

The vote is now closed, and the release was approved with 4 +1s(4
bindings). I will continue with the following steps.

Binding votes:
* Enrico Olivelli
* Andrey Yegorov
* Jia Zhai
* Hang Chen

Thank you all for verifying and voting!

Thanks,
Hang

Andrey Yegorov  于2024年1月27日周六 03:43写道：
>
> +1 (binding)
>
> * reviewed release notes
> * spot checked repository
> * ran bk from packaged binaries
> * built from packaged sources
>   ** builds ok with jdk 17
>   ** build error with jdk 21. This happens on master too, but I think this
> is NOT the release stopper. I added
> https://github.com/apache/bookkeeper/issues/4189 to address later
>
>
> On Thu, Jan 18, 2024 at 6:08 PM Hang Chen  wrote:
>
> > Hi everyone,
> > Please review and vote on the release candidate #0 for version 4.16.4,
> > as follows:
> > [ ] +1, Approve the release
> > [ ] -1, Do not approve the release (please provide specific comments)
> >
> > The complete staging area is available for your review, which includes:
> > * Release notes [1]
> > * The official Apache source and binary distributions to be deployed
> > to dist.apache.org [2]
> > * All artifacts to be deployed to the Maven Central Repository [3]
> > * Source code tag "v4.16.4-rc0" [4] with git sha [5]
> >
> > BookKeeper's KEYS file contains the PGP keys we used to sign this release:
> > https://dist.apache.org/repos/dist/release/bookkeeper/KEYS
> >
> > Please download these packages and review this release candidate:
> >
> > - Review release notes
> > - Download the source package (verify shasum, and asc) and follow the
> > instructions to build and run the bookkeeper service.
> > - Download the binary package (verify shasum, and asc) and follow the
> > instructions to run the bookkeeper service.
> > - Review the maven repo, release tag, licenses, and any other things
> > you think it is important to release.
> >
> > The vote will be open for at least 72 hours. It is adopted by the
> > majority approval, with at least 3 PMC affirmative votes.
> >
> > Thanks,
> > Release Manager
> >
> > [1] https://github.com/apache/bookkeeper/pull/4182/files
> > [2]
> > https://dist.apache.org/repos/dist/dev/bookkeeper/bookkeeper-4.16.4-rc0/
> > [3]
> > https://repository.apache.org/content/repositories/orgapachebookkeeper-1091
> > [4] https://github.com/apache/bookkeeper/tree/v4.16.4-rc0
> > [5] 16de712d8c8f1f0296697331bb68c1ad0eb5787d
> >
>
>
> --
> Andrey Yegorov

Re: [VOTE] Release 4.16.4, release candidate #0

[Hang Chen]

+1 (binding)

- Verified signatures and checksums
- Build from source code (MacOS M3 JDK 17.0.9, Maven 3.8.6)
- Build BookKeeper with Pulsar master (MacOS M3 JDK 17.0.9, Maven 3.8.6)
- Build the Pulsar master branch that integrates with BookKeeper
4.16.4 rc0 and start the Pulsar cluster with one Zookeeper, one
Bookie, and one broker. Run Pulsar perf produce and consume with
DirectIO disabled -> enabled -> disabled. Everything works as
expected.
- All unit tests passed when the BookKeeper dependency of the Pulsar
master branch was upgraded to 4.16.4. [1]
- All the internal tests, which integrate with Pulsar, are passed


[1] https://github.com/hangc0276/pulsar/pull/24

Thanks,
Hang

Enrico Olivelli  于2024年1月26日周五 17:50写道：
>
> +1 (binding)
>
> - Verified checksums and signatures
> - Passed all the HerdDB tests (https://github.com/diennea/herddb/pull/819)
> - Built locally on Mac, JDK11
>
> Thanks
> Enrico
>
> Il giorno ven 19 gen 2024 alle ore 03:08 Hang Chen
>  ha scritto:
> >
> > Hi everyone,
> > Please review and vote on the release candidate #0 for version 4.16.4,
> > as follows:
> > [ ] +1, Approve the release
> > [ ] -1, Do not approve the release (please provide specific comments)
> >
> > The complete staging area is available for your review, which includes:
> > * Release notes [1]
> > * The official Apache source and binary distributions to be deployed
> > to dist.apache.org [2]
> > * All artifacts to be deployed to the Maven Central Repository [3]
> > * Source code tag "v4.16.4-rc0" [4] with git sha [5]
> >
> > BookKeeper's KEYS file contains the PGP keys we used to sign this release:
> > https://dist.apache.org/repos/dist/release/bookkeeper/KEYS
> >
> > Please download these packages and review this release candidate:
> >
> > - Review release notes
> > - Download the source package (verify shasum, and asc) and follow the
> > instructions to build and run the bookkeeper service.
> > - Download the binary package (verify shasum, and asc) and follow the
> > instructions to run the bookkeeper service.
> > - Review the maven repo, release tag, licenses, and any other things
> > you think it is important to release.
> >
> > The vote will be open for at least 72 hours. It is adopted by the
> > majority approval, with at least 3 PMC affirmative votes.
> >
> > Thanks,
> > Release Manager
> >
> > [1] https://github.com/apache/bookkeeper/pull/4182/files
> > [2] https://dist.apache.org/repos/dist/dev/bookkeeper/bookkeeper-4.16.4-rc0/
> > [3] 
> > https://repository.apache.org/content/repositories/orgapachebookkeeper-1091
> > [4] https://github.com/apache/bookkeeper/tree/v4.16.4-rc0
> > [5] 16de712d8c8f1f0296697331bb68c1ad0eb5787d

[VOTE] Release 4.16.4, release candidate #0

[Hang Chen]

Hi everyone,
Please review and vote on the release candidate #0 for version 4.16.4,
as follows:
[ ] +1, Approve the release
[ ] -1, Do not approve the release (please provide specific comments)

The complete staging area is available for your review, which includes:
* Release notes [1]
* The official Apache source and binary distributions to be deployed
to dist.apache.org [2]
* All artifacts to be deployed to the Maven Central Repository [3]
* Source code tag "v4.16.4-rc0" [4] with git sha [5]

BookKeeper's KEYS file contains the PGP keys we used to sign this release:
https://dist.apache.org/repos/dist/release/bookkeeper/KEYS

Please download these packages and review this release candidate:

- Review release notes
- Download the source package (verify shasum, and asc) and follow the
instructions to build and run the bookkeeper service.
- Download the binary package (verify shasum, and asc) and follow the
instructions to run the bookkeeper service.
- Review the maven repo, release tag, licenses, and any other things
you think it is important to release.

The vote will be open for at least 72 hours. It is adopted by the
majority approval, with at least 3 PMC affirmative votes.

Thanks,
Release Manager

[1] https://github.com/apache/bookkeeper/pull/4182/files
[2] https://dist.apache.org/repos/dist/dev/bookkeeper/bookkeeper-4.16.4-rc0/
[3] https://repository.apache.org/content/repositories/orgapachebookkeeper-1091
[4] https://github.com/apache/bookkeeper/tree/v4.16.4-rc0
[5] 16de712d8c8f1f0296697331bb68c1ad0eb5787d

Re: [DISCUSS] BookKeeper 4.16.4 release

[Hang Chen]

I'm building the rc.

Thanks,
Hang

Enrico Olivelli  于2024年1月18日周四 15:14写道：
>
> +1
>
> If you want I can drive the release, it has been quite some time I haven't
> prepared an RC
>
> Enrico
>
>
> Il Gio 18 Gen 2024, 06:22 Lari Hotari  ha scritto:
>
> > +1
> >
> > It would be great to have 4.16.4 released to get the fix for the
> > Java9IntHash checksum bug https://github.com/apache/bookkeeper/pull/4140
> > . I guess this bug applies to all platforms that aren't Linux x86-64 since
> > that's when Java9IntHash is usually active.
> > I've come across the bug quite a few times when experimenting with Pulsar
> > on Mac M3 hardware.
> >
> > Are we ready to start the 4.16.4 release?
> >
> > -Lari
> >
> > On 2023/12/25 03:43:16 Hang Chen wrote:
> > > Hi everyone,
> > >
> > > BookKeeper 4.16.3 was released more than three months ago, and I would
> > > like to discuss the starting 4.16.4 release to include some critical
> > > bug fixes and improvements, such as [1] and [2]
> > >
> > > Here are the PRs for 4.16.4:
> > >
> > https://github.com/apache/bookkeeper/pulls?q=is%3Aopen+is%3Apr+label%3Arelease%2F4.16.4
> > >
> > > If you have PRs that want to be included in this release, please
> > > comment on that PR or tag the label `release/4.16.4`.
> > >
> > > After merging those PRs, I want to volunteer to release this patch.
> > >
> > > Thanks,
> > > Hang
> > >
> > > [1] https://github.com/apache/bookkeeper/pull/4163
> > > [2] https://github.com/apache/bookkeeper/pull/4128
> > >
> >

[DISCUSS] BookKeeper 4.16.4 release

[Hang Chen]

Hi everyone,

BookKeeper 4.16.3 was released more than three months ago, and I would
like to discuss the starting 4.16.4 release to include some critical
bug fixes and improvements, such as [1] and [2]

Here are the PRs for 4.16.4:
https://github.com/apache/bookkeeper/pulls?q=is%3Aopen+is%3Apr+label%3Arelease%2F4.16.4

If you have PRs that want to be included in this release, please
comment on that PR or tag the label `release/4.16.4`.

After merging those PRs, I want to volunteer to release this patch.

Thanks,
Hang

[1] https://github.com/apache/bookkeeper/pull/4163
[2] https://github.com/apache/bookkeeper/pull/4128

Re: [VOTE] Release 4.15.5, release candidate #0

[Hang Chen]

+1 (binding)

Thank Yong for driving this release. Overall it looks good.

- Verified signatures and checksums
- Build from source code (MacOS jdk 17, maven 3.8.6)
- Checked all the CI in BookKeeper branch-4.15 [1]
- Build Pulsar master code with this staging dependency and start
Pulsar standalone cluster and 1 bookie, 1 broker and 1 zookeeper
Pulsar cluster and run Pulsar perf produce and consume
- Verified the Prometheus metrics output

Thanks,
Hang


[1] https://github.com/apache/bookkeeper/commits/branch-4.15/

Yong Zhang  于2023年12月14日周四 09:31写道：
>
> I have done it on my branch: https://github.com/zymap/pulsar/pull/53
>
> On Wed, 13 Dec 2023 at 22:50, Hang Chen  wrote:
>
> > Hi Yong,
> >  Thank you for driving this release effort. I have submitted a
> > pull request [1] to upgrade the Pulsar branch-2.11 BookKeeper
> > dependency to version 4.15.5 for testing. Currently, I am awaiting the
> > completion of CI checks.
> >
> > Thanks,
> > Hang
> >
> >
> > [1] https://github.com/apache/pulsar/pull/21720
> >
> > Yong Zhang  于2023年12月12日周二 18:38写道：
> > >
> > > Hi everyone,
> > > Please review and vote on the release candidate #0 for the version
> > 4.15.5,
> > > as follows:
> > > [ ] +1, Approve the release
> > > [ ] -1, Do not approve the release (please provide specific comments)
> > >
> > > The complete staging area is available for your review, which includes:
> > > * Release notes [1]
> > > * The official Apache source and binary distributions to be deployed to
> > > dist.apache.org [2]
> > > * All artifacts to be deployed to the Maven Central Repository [3]
> > > * Source code tag "release-4.15.5" [4] with git sha
> > > 27de09989a6920b939e29dcf625720f437269b38
> > >
> > > BookKeeper's KEYS file contains PGP keys we used to sign this release:
> > > https://dist.apache.org/repos/dist/release/bookkeeper/KEYS
> > >
> > > Please download these packages and review this release candidate:
> > >
> > > - Review release notes
> > > - Download the source package (verify shasum, and asc) and follow the
> > > instructions to build and run the bookkeeper service.
> > > - Download the binary package (verify shasum, and asc) and follow the
> > > instructions to run the bookkeeper service.
> > > - Review maven repo, release tag, licenses, and any other things you
> > think
> > > it is important to a release.
> > >
> > > The vote will be open for at least 72 hours. It is adopted by majority
> > > approval, with at least 3 PMC affirmative votes.
> > >
> > > Thanks,
> > > Release Manager
> > >
> > > [1] https://github.com/apache/bookkeeper/pull/4151
> > > [2]
> > https://dist.apache.org/repos/dist/dev/bookkeeper/bookkeeper-4.15.5-rc0/
> > > [3]
> > >
> > https://repository.apache.org/content/repositories/orgapachebookkeeper-1090
> > > [4] https://github.com/apache/bookkeeper/releases/tag/v4.15.5-rc0
> > > [5] 27de09989a6920b939e29dcf625720f437269b38
> >

Re: [VOTE] Release 4.15.5, release candidate #0

[Hang Chen]

Hi Yong,
 Thank you for driving this release effort. I have submitted a
pull request [1] to upgrade the Pulsar branch-2.11 BookKeeper
dependency to version 4.15.5 for testing. Currently, I am awaiting the
completion of CI checks.

Thanks,
Hang


[1] https://github.com/apache/pulsar/pull/21720

Yong Zhang  于2023年12月12日周二 18:38写道：
>
> Hi everyone,
> Please review and vote on the release candidate #0 for the version 4.15.5,
> as follows:
> [ ] +1, Approve the release
> [ ] -1, Do not approve the release (please provide specific comments)
>
> The complete staging area is available for your review, which includes:
> * Release notes [1]
> * The official Apache source and binary distributions to be deployed to
> dist.apache.org [2]
> * All artifacts to be deployed to the Maven Central Repository [3]
> * Source code tag "release-4.15.5" [4] with git sha
> 27de09989a6920b939e29dcf625720f437269b38
>
> BookKeeper's KEYS file contains PGP keys we used to sign this release:
> https://dist.apache.org/repos/dist/release/bookkeeper/KEYS
>
> Please download these packages and review this release candidate:
>
> - Review release notes
> - Download the source package (verify shasum, and asc) and follow the
> instructions to build and run the bookkeeper service.
> - Download the binary package (verify shasum, and asc) and follow the
> instructions to run the bookkeeper service.
> - Review maven repo, release tag, licenses, and any other things you think
> it is important to a release.
>
> The vote will be open for at least 72 hours. It is adopted by majority
> approval, with at least 3 PMC affirmative votes.
>
> Thanks,
> Release Manager
>
> [1] https://github.com/apache/bookkeeper/pull/4151
> [2] https://dist.apache.org/repos/dist/dev/bookkeeper/bookkeeper-4.15.5-rc0/
> [3]
> https://repository.apache.org/content/repositories/orgapachebookkeeper-1090
> [4] https://github.com/apache/bookkeeper/releases/tag/v4.15.5-rc0
> [5] 27de09989a6920b939e29dcf625720f437269b38

[DISCUSS] New label type: category/*

[Hang Chen]

Hi all,

I would like to start a discussion about adding a new label type category/*

There are three labels are supposed to be added

- category/functionality
- category/reliability
- category/performance

It could be a good improvement to the labeling system that BookKeeper has today.
Now, BookKeeper mainly have four label categories

- type/* - type/bug, type/feature ...
- area/* - area/bookie, area/client ...
- release/* - release/4.14.7, release/4.16.0 ...
- cherry-picked/* - cherry-picked/branch-4.14, cherry-picked/branch-4.16 ...

type/*: is mainly to distinguish the issues and PRs are for bug
reporting, bug fix, feature requests, feature support.

area/*: you can know the issues and PRs that happen on which component
with the component labels, such as an issue with type/bug and
area/client means it's a bug report for the client component.

release/*: labels are indicating which version the issue/PR has been
fixed or will be fixed depending on if the version is released or not.

cherry-picked/*: labels are more mainly for BookKeeper committers to
ensure the fixes are cherry-picked to the release branches. The label
only can be added after the cherry-picking is done for a corresponding
branch. So that the release manager can have a list of PRs that should
be cherry-picked.

In addition to being able to identify which component that the issue,
PR is fixed or enhanced, the category labels will provide more
information about the fix or enhancement for functionality,
reliability, or performance. For most cases, the category labels only
work with type/bug and type/enhancement.

- category/functionality, some functions are not working, such as
getting errors.
- category/reliability, the function is working for most cases. It
does not work properly in certain specific environments or failures.
e.g. data lost, consumption stuck ...
- category/performance, performance issues fixed or improvements.

Thanks,
Hang

Re: [VOTE] Release 4.14.8, release candidate #0

[Hang Chen]

The vote is now closed, and the release was approved with
5+1s(4 bindings and 1 non-binding), and I will continue with the
following steps.

Binding votes:
* Enrico Olivelli
* Andrey Yegorov
* Jia Zhai
* Hang Chen

Non-Binding votes:
* Yong Zhang

Thank you all for verifying and voting!

Thanks,
Hang

Andrey Yegorov  于2023年9月7日周四 05:28写道：
>
> +1 (binding)
>
> * reviewed release notes
> * spot-checked maven repo
> * downloaded binary packages and tried bookie standalone/localbookie
>
> On Fri, Aug 18, 2023 at 2:28 AM Hang Chen  wrote:
>
> > Hi everyone,
> > Please review and vote on the release candidate #0 for version 4.14.8,
> > as follows:
> > [ ] +1, Approve the release
> > [ ] -1, Do not approve the release (please provide specific comments)
> >
> > The complete staging area is available for your review, which includes:
> > * Release notes [1]
> > * The official Apache source and binary distributions to be deployed
> > to dist.apache.org [2]
> > * All artifacts to be deployed to the Maven Central Repository [3]
> > * Source code tag "v4.14.8-rc0" [4] with git sha [5]
> >
> > BookKeeper's KEYS file contains the PGP keys we used to sign this release:
> > https://dist.apache.org/repos/dist/release/bookkeeper/KEYS
> >
> > Please download these packages and review this release candidate:
> >
> > - Review release notes
> > - Download the source package (verify shasum, and asc) and follow the
> > instructions to build and run the bookkeeper service.
> > - Download the binary package (verify shasum, and asc) and follow the
> > instructions to run the bookkeeper service.
> > - Review the maven repo, release tag, licenses, and any other things
> > you think it is important to release.
> >
> > The vote will be open for at least 72 hours. It is adopted by the
> > majority approval, with at least 3 PMC affirmative votes.
> >
> > Thanks,
> > Release Manager
> >
> > [1] https://github.com/apache/bookkeeper/pull/4053/files
> > [2]
> > https://dist.apache.org/repos/dist/dev/bookkeeper/bookkeeper-4.14.8-rc0/
> > [3]
> > https://repository.apache.org/content/repositories/orgapachebookkeeper-1088
> > [4] https://github.com/apache/bookkeeper/tree/v4.14.8-rc0
> > [5] 965c3627328787e2750f41c2a4eda6c1d709d7c6
> >
>
>
> --
> Andrey Yegorov

Re: [VOTE] Release 4.14.8, release candidate #0

[Hang Chen]

+1 (binding)

- Verified signatures and checksums
- Build from source code (MacOS jdk 11, maven 3.8.6)
- Build Pulsar branch-2.10 with the staging BookKeeper dependency
(MacOS JDK 17, Maven 3.8.6)
- Start Pulsar cluster with one Zookeeper, one Bookie and one broker.
Run Pulsar perf produce and consume.
- Pulsar branch-2.10 CI passed with staging BookKeeper 4.14.8 [1]

[1] https://github.com/hangc0276/pulsar/pull/21

Regards,
Hang

Enrico Olivelli  于2023年8月28日周一 23:19写道：
>
> +1 (binding)
>
> - checked signatures
> - run all the tests on JDK8, Mac M1
>
> Thanks for running the release
>
> Enrico
>
> Il giorno lun 28 ago 2023 alle ore 04:21 Yong Zhang
>  ha scritto:
> >
> > +1 (non-binding)
> >
> > - Signature and checksum look good
> > - Built from archived sources (m1, jdk17, maven 3.8.6)
> > - Run standalone and simpletest with the bookie-all package
> >
> > Thanks,
> > Yong
> >
> > On Fri, 18 Aug 2023 at 17:28, Hang Chen  wrote:
> >
> > > Hi everyone,
> > > Please review and vote on the release candidate #0 for version 4.14.8,
> > > as follows:
> > > [ ] +1, Approve the release
> > > [ ] -1, Do not approve the release (please provide specific comments)
> > >
> > > The complete staging area is available for your review, which includes:
> > > * Release notes [1]
> > > * The official Apache source and binary distributions to be deployed
> > > to dist.apache.org [2]
> > > * All artifacts to be deployed to the Maven Central Repository [3]
> > > * Source code tag "v4.14.8-rc0" [4] with git sha [5]
> > >
> > > BookKeeper's KEYS file contains the PGP keys we used to sign this release:
> > > https://dist.apache.org/repos/dist/release/bookkeeper/KEYS
> > >
> > > Please download these packages and review this release candidate:
> > >
> > > - Review release notes
> > > - Download the source package (verify shasum, and asc) and follow the
> > > instructions to build and run the bookkeeper service.
> > > - Download the binary package (verify shasum, and asc) and follow the
> > > instructions to run the bookkeeper service.
> > > - Review the maven repo, release tag, licenses, and any other things
> > > you think it is important to release.
> > >
> > > The vote will be open for at least 72 hours. It is adopted by the
> > > majority approval, with at least 3 PMC affirmative votes.
> > >
> > > Thanks,
> > > Release Manager
> > >
> > > [1] https://github.com/apache/bookkeeper/pull/4053/files
> > > [2]
> > > https://dist.apache.org/repos/dist/dev/bookkeeper/bookkeeper-4.14.8-rc0/
> > > [3]
> > > https://repository.apache.org/content/repositories/orgapachebookkeeper-1088
> > > [4] https://github.com/apache/bookkeeper/tree/v4.14.8-rc0
> > > [5] 965c3627328787e2750f41c2a4eda6c1d709d7c6
> > >

Re: [VOTE] Release 4.16.3, release candidate #0

[Hang Chen]

All unit tests passed when the BookKeeper dependency of the Pulsar
master branch was upgraded to 4.16.3. [1]


[1] https://github.com/zymap/pulsar/pull/52

Regards,
Hang

Hang Chen  于2023年9月5日周二 09:34写道：
>
> +1 (binding)
>
> - Verified signatures and checksums
> - Build from source code (MacOS jdk 11, maven 3.8.6)
> - Build BookKeeper with Pulsar master (MacOS JDK 17, Maven 3.8.6)
> - Build Pulsar master code with this staging dependency and start
> Pulsar cluster with one Zookeeper, one Bookie and one Broker. Run
> Pulsar perf produce and consume with DirectIO disabled -> enabled ->
> disabled on Ubuntu 20.04
> - Build Pulsar master code with MacOS build BookKeeper dependency and
> start Pulsar cluster with one Zookeeper, one Bookie and one broker.
> Run Pulsar perf produce and consume with DirectIO disabled -> enabled
> -> disabled on MacOS
>
> Regards,
> Hang
>
> Yong Zhang  于2023年8月31日周四 19:14写道：
> >
> > Hi everyone,
> > Please review and vote on the release candidate #0 for the version 4.16.3,
> > as follows:
> > [ ] +1, Approve the release
> > [ ] -1, Do not approve the release (please provide specific comments)
> >
> > The complete staging area is available for your review, which includes:
> > * Release notes [1]
> > * The official Apache source and binary distributions to be deployed to
> > dist.apache.org [2]
> > * All artifacts to be deployed to the Maven Central Repository [3]
> > * Source code tag "release-4.5.0" [4] with git sha
> > 5ca147b4a85a1044967002b3f389d17e468a264d
> >
> > BookKeeper's KEYS file contains PGP keys we used to sign this release:
> > https://dist.apache.org/repos/dist/release/bookkeeper/KEYS
> >
> > Please download these packages and review this release candidate:
> >
> > - Review release notes
> > - Download the source package (verify shasum, and asc) and follow the
> > instructions to build and run the bookkeeper service.
> > - Download the binary package (verify shasum, and asc) and follow the
> > instructions to run the bookkeeper service.
> > - Review maven repo, release tag, licenses, and any other things you think
> > it is important to a release.
> >
> > The vote will be open for at least 72 hours. It is adopted by majority
> > approval, with at least 3 PMC affirmative votes.
> >
> > Thanks,
> > Release Manager
> >
> > [1]
> > https://github.com/apache/bookkeeper/pull/4065/files#diff-d361215d5d5abaa96ac3362e70a1d2f80f92d369a59c866e1976ff41ec435f53R4
> > [2] https://dist.apache.org/repos/dist/dev/bookkeeper/bookkeeper-4.16.3-rc0/
> > [3]
> > https://repository.apache.org/content/repositories/orgapachebookkeeper-1089
> > [4] https://github.com/apache/bookkeeper/releases/tag/v4.16.3-rc0
> > [5] 5ca147b4a85a1044967002b3f389d17e468a264d

Re: [VOTE] Release 4.16.3, release candidate #0

[Hang Chen]

+1 (binding)

- Verified signatures and checksums
- Build from source code (MacOS jdk 11, maven 3.8.6)
- Build BookKeeper with Pulsar master (MacOS JDK 17, Maven 3.8.6)
- Build Pulsar master code with this staging dependency and start
Pulsar cluster with one Zookeeper, one Bookie and one Broker. Run
Pulsar perf produce and consume with DirectIO disabled -> enabled ->
disabled on Ubuntu 20.04
- Build Pulsar master code with MacOS build BookKeeper dependency and
start Pulsar cluster with one Zookeeper, one Bookie and one broker.
Run Pulsar perf produce and consume with DirectIO disabled -> enabled
-> disabled on MacOS

Regards,
Hang

Yong Zhang  于2023年8月31日周四 19:14写道：
>
> Hi everyone,
> Please review and vote on the release candidate #0 for the version 4.16.3,
> as follows:
> [ ] +1, Approve the release
> [ ] -1, Do not approve the release (please provide specific comments)
>
> The complete staging area is available for your review, which includes:
> * Release notes [1]
> * The official Apache source and binary distributions to be deployed to
> dist.apache.org [2]
> * All artifacts to be deployed to the Maven Central Repository [3]
> * Source code tag "release-4.5.0" [4] with git sha
> 5ca147b4a85a1044967002b3f389d17e468a264d
>
> BookKeeper's KEYS file contains PGP keys we used to sign this release:
> https://dist.apache.org/repos/dist/release/bookkeeper/KEYS
>
> Please download these packages and review this release candidate:
>
> - Review release notes
> - Download the source package (verify shasum, and asc) and follow the
> instructions to build and run the bookkeeper service.
> - Download the binary package (verify shasum, and asc) and follow the
> instructions to run the bookkeeper service.
> - Review maven repo, release tag, licenses, and any other things you think
> it is important to a release.
>
> The vote will be open for at least 72 hours. It is adopted by majority
> approval, with at least 3 PMC affirmative votes.
>
> Thanks,
> Release Manager
>
> [1]
> https://github.com/apache/bookkeeper/pull/4065/files#diff-d361215d5d5abaa96ac3362e70a1d2f80f92d369a59c866e1976ff41ec435f53R4
> [2] https://dist.apache.org/repos/dist/dev/bookkeeper/bookkeeper-4.16.3-rc0/
> [3]
> https://repository.apache.org/content/repositories/orgapachebookkeeper-1089
> [4] https://github.com/apache/bookkeeper/releases/tag/v4.16.3-rc0
> [5] 5ca147b4a85a1044967002b3f389d17e468a264d

Re: [DISCUSS] BookKeeper 4.16.3 release

[Hang Chen]

+1 for the new release.

Please include the following two PRs to 4.16.3, thanks.
https://github.com/apache/bookkeeper/pull/4041
https://github.com/apache/bookkeeper/pull/4058


Thanks,
Hang

Yong Zhang  于2023年8月29日周二 15:09写道：
>
> Hi everyone,
>
> Because the last release was published with arm, it causes the native lib
> only can run on the arm platform which breaks the amd64 supports.
> We would like to publish 4.16.3 to support the amd64 platform.
>
> See more context about the platform issue:
> https://lists.apache.org/thread/wkn453lrx3n205n21zk5lhkq3q1bqsdb
>
> Here are the PRs for 4.16.3:
> https://github.com/apache/bookkeeper/issues?q=label%3Arelease%2F4.16.3+is%3Aclosed+
>
> If you have PRs that want to be included in this release, please
> comment on that PR or tag the label `release/4.16.3`.
>
> After those PRs merged, I would like to volunteer to release this patch.
>
> Thanks,
> Yong

Re: Force to use linux/amd64 to build release

[Hang Chen]

+1

The BookKeeper 4.16.2 is released based on ARM platform and the direct
IO can't be enabled in 4.16.2.

Best,
Hang

Enrico Olivelli  于2023年8月28日周一 16:46写道：
>
> +1
>
> Thanks
> We should cut a new release as soon as possible
>
> Enrico
>
> Il Lun 28 Ago 2023, 10:34 Yong Zhang  ha
> scritto:
>
> > Hi everyone,
> >
> > When releasing the 4.16.2, I did the release with an M1 macbook pro.
> > That makes the native io lib compile and publish with the arm platform.
> > So some users get the wrong lib when running the bookie with native io.
> > https://github.com/apache/bookkeeper/issues/4043
> >
> > And the `NATIVE_IO.posix_fadvise(fd, offset, len, POSIX_FADV_DONTNEED);`
> > also affected because the lib was built for arm.
> >
> > https://github.com/apache/bookkeeper/blob/ba3127447630315c97594822b1510ecd509f7671/bookkeeper-server/src/main/java/org/apache/bookkeeper/util/PageCacheUtil.java#L97
> >
> > I made a change on the release Dockerfile to force the image to use the
> > linux/amd64 platform, we can ensure the lib is compiled on the amd64.
> > https://github.com/apache/bookkeeper/pull/4060
> >
> > Please help to review that PR. Once it merged, I would like to be a
> > volunteer to
> > release the 4.16.3 to fix the lib.
> >
> > Thanks,
> > Yong
> >

Re: BP62-New API for batched reads

[Hang Chen]

+1

Looking forward to this feature.

Thanks,
Hang

Enrico Olivelli  于2023年8月20日周日 21:24写道：
>
> Hello,
> The proposal is well written, I have no questions.
> We have been waiting for this feature for long time.
>
> I am supporting it
>
> Thanks
> Enrico
>
> Il Mar 15 Ago 2023, 04:28 horizonzy  ha scritto:
>
> > Hi, everyone:
> > There is a proposal about batched reading(
> > https://github.com/apache/bookkeeper/pull/4051), to introduce it to
> > improve
> > read performance.
> >
> > The objective of this proposal is to enhance the performance of entry
> > reading by introducing a batch entry reading protocol that takes into
> > account the expected count and size of entries.
> > 1. Optimize entry reading performance: By reading multiple entries in a
> > single RPC request, the network communication and RPC call
> > overhead can be reduced, thereby optimizing the reading performance.
> > 2. Minimize CPU resource consumption: The aggregation of multiple entries
> > into a single RPC request can help in reducing the number of requests and
> > responses, which in turn can lower the CPU resource consumption.
> > 3. Streamline client code: The ability to read entries based on the
> > anticipated count or size, such as Apache Pulsar's approach of calculating
> > the start and end entry IDs for each read request based on the average size
> > of past entries, can add unnecessary complexity to the implementation and
> > can't guarantee reliable behavioral outcomes.
> >
> > Here is the output of the BookKeeper perf tool with ensemble=1, write=1,
> > and ack=1.
> > Batch(100): Read 1000100 entries in 8904ms
> > Batch(500): Read 1000500 entries in 12182ms
> > Non-Batch: Read 1000130 entries in 199928ms
> >
> > If you have any questions, feel free to talk about it. Thanks!
> >

Re: Remove UnderReplicatedLedgersChangedCb in Auditor

[Hang Chen]

+1

The interface introduces a heavy load on Zookeeper, and we need to remove it.
For branch-4.15 and branch-4.16, we can keep the interface and remove
the implementation and remove the interface on the master branch.

Thanks,
Hang

Enrico Olivelli  于2023年8月20日周日 21:24写道：
>
> +1
>
> Enrico
>
> Il Gio 17 Ago 2023, 10:06 horizonzy  ha scritto:
>
> > In https://github.com/apache/bookkeeper/pull/2805, it introduces
> > UnderReplicatedLedgersChangedCb to watch the zookeeper `underreplication`
> > path. After a `underreplication` ledger mark is replicated, the
> > UnderReplicatedLedgersChangedCb will call back.
> >
> > In the callback, it will list all the `unnderreplication` ledgers to record
> > the metrics `underReplicatedLedgersGuageValue`. It's a heavy operation for
> > zookeeper.
> >
> > And in the pulsar, it introduces a deadlock after implementing the new API
> > LedgerUnderreplicationManager#notifyUnderReplicationLedgerChanged
> > in PulsarLedgerUnderreplicationManager. see
> > https://github.com/apache/pulsar/pull/21010.
> >
> > I would suggest you remove underReplicatedLedgersChangedCb in Auditor. If
> > the user wants to record the metrics, a scheduled task will be better than
> > a zk watch event to trigger it.
> >

[VOTE] Release 4.14.8, release candidate #0

[Hang Chen]

Hi everyone,
Please review and vote on the release candidate #0 for version 4.14.8,
as follows:
[ ] +1, Approve the release
[ ] -1, Do not approve the release (please provide specific comments)

The complete staging area is available for your review, which includes:
* Release notes [1]
* The official Apache source and binary distributions to be deployed
to dist.apache.org [2]
* All artifacts to be deployed to the Maven Central Repository [3]
* Source code tag "v4.14.8-rc0" [4] with git sha [5]

BookKeeper's KEYS file contains the PGP keys we used to sign this release:
https://dist.apache.org/repos/dist/release/bookkeeper/KEYS

Please download these packages and review this release candidate:

- Review release notes
- Download the source package (verify shasum, and asc) and follow the
instructions to build and run the bookkeeper service.
- Download the binary package (verify shasum, and asc) and follow the
instructions to run the bookkeeper service.
- Review the maven repo, release tag, licenses, and any other things
you think it is important to release.

The vote will be open for at least 72 hours. It is adopted by the
majority approval, with at least 3 PMC affirmative votes.

Thanks,
Release Manager

[1] https://github.com/apache/bookkeeper/pull/4053/files
[2] https://dist.apache.org/repos/dist/dev/bookkeeper/bookkeeper-4.14.8-rc0/
[3] https://repository.apache.org/content/repositories/orgapachebookkeeper-1088
[4] https://github.com/apache/bookkeeper/tree/v4.14.8-rc0
[5] 965c3627328787e2750f41c2a4eda6c1d709d7c6

Re: Data loss problem when DefaultEntryLogger switch to DirectEntryLogger, then switch back to DefaultEntryLogger.

[Hang Chen]

>You should start a new bookie with the new configuration.

If the DirectEntryLogger needs to set up a separate bookie service, it
would be hard for users to switch from the DefaultEntryLogger to
DriectEntryLogger.

>It is not generally possible to guarantee compatibility between different
implementations.

The DirectEntryLogger is designed to be compatible with the
DefaultEntryLogger, and can switch from the DefaultEntryLogger to
DirectEntryLogger smoothly. If we switch from DefaultEntryLogger to
DirectEntryLogger, but can't switch back, it will be hard to apply the
DirectEntryLogger in the existing BookKeeper cluster.

The main reason we can't switch from DirectEntryLogger back to
DefaultEntryLogger is that the EntryLogId generator is different and
the new EntryLogId generator in DirectEntryLogger can cover more cases
than the old one. The drawback of the old EntryLogId generator is as
follows:
- The last entrylogId is stored in the `lastId` file, and if this file
is broken or be tampered, the new generated entrylogId will runs into
unexpected value
- The new entrylogId is generated with +1 logic, and if the entrylogId
reaches Integer.MAX_VALUE, it will be reset to 0 [1]. If the 0.log
still exists (may be impossible in common case), the old 0.log file
will be overridden and cause data loss.

We just need to change the last entryId loader logic in the
DefaultEntryLogger to be compatible with the DirectEntryLogger, which
is valuable for this work.

[1] 
https://github.com/apache/bookkeeper/blob/3bf08fef12979bf3cff49ee953899f26c6281ad9/bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/EntryLoggerAllocator.java#L147-L151

Thanks,
Hang

Yan Zhao  于2023年7月31日周一 17:57写道：
>
> > You should never do this in production.
>
> But the user may not know the limitation, they may believe that the switching 
> is compatible, not introduce some problems.
>
> > BTW do you have a possible solution?
>
> see https://github.com/apache/bookkeeper/pull/4041

Re: [DISCUSS] BookKeeper 4.14.8 release

[Hang Chen]

Hi guys,
All the PRs with `4.14.8` label have been cherry-picked to
branch-4.18 and I will start to release 4.14.8.
https://github.com/apache/bookkeeper/commits/branch-4.14
   If you have any PRs that need to be included in 4.14.8, please let me know.

Thanks,
Hang

houxiaoyu  于2023年7月5日周三 18:22写道：
>
> +1
>
> Thanks,
> Xiaoyu Hou
>
> Hang Chen  于2023年5月30日周二 17:09写道：
>
> > Hi everyone,
> >
> > BookKeeper 4.14.7 was released more than 3 months, and I would like to
> > discuss the starting 4.14.8 release to include some critical bug fixes
> > and improvements, such as [1] and [2]
> >
> > Here are the PRs for 4.14.8:
> >
> > https://github.com/apache/bookkeeper/pulls?q=is%3Aopen+is%3Apr+label%3Arelease%2F4.14.8
> >
> > If you have PRs that want to be included in this release, please
> > comment on that PR or tag the label `release/4.14.8`.
> >
> > After those PRs merged, I would like to volunteer to release this patch.
> >
> > Thanks,
> > Hang
> >
> > [1] https://github.com/apache/bookkeeper/pull/3959
> > [2] https://github.com/apache/bookkeeper/pull/3917
> >

Re: [VOTE] Release 4.16.2, release candidate #1

[Hang Chen]

+1 (binding)
- Verified signatures and checksums
- Build from source code (MacOS jdk 11, maven 3.8.6)
- Tested local bookie
- Run bkperf journal test
- Build BookKeeper with Pulsar (MacOS JDK 17, Maven 3.8.6)
- Build Pulsar master code with this staging dependency and start
Pulsar cluster with one Zookeeper, one Bookie and one Broker.
- Run Pulsar perf produce and consume with DirectIO enabled

Thanks,
Hang

Wenbing Shen  于2023年6月28日周三 11:27写道：
>
> +1 (non-binding)
>
> - only verified checksum
> - ran standalone bookie and execute some bk shell commands
> - built from sources (mac 11.3.1, intel), ran some tests
>
> Thanks,
> Wenbing
>
> Yong Zhang  于2023年6月28日周三 10:40写道：
>
> > +1 (non-binding)
> >
> > - verified signature and checksum
> > - build successfully from the src package
> > - bookkeeper CI is passed
> >
> > https://github.com/apache/bookkeeper/actions/runs/5373253013/jobs/9747449604
> > - verified through pulsar CI testing using the new bookie version
> > https://github.com/zymap/pulsar/actions/runs/5374064690/jobs/9749054413
> >
> > Thanks,
> > Yong
> >
> > On Mon, 26 Jun 2023 at 15:17, Yong Zhang 
> > wrote:
> >
> > > Hi everyone,
> > > Please review and vote on the release candidate #1 for the version
> > > 4.16.2,, as follows:
> > > [ ] +1, Approve the release
> > > [ ] -1, Do not approve the release (please provide specific comments)
> > >
> > > The complete staging area is available for your review, which includes:
> > > * Release notes [1]
> > > * The official Apache source and binary distributions to be deployed to
> > > dist.apache.org [2]
> > > * All artifacts to be deployed to the Maven Central Repository [3]
> > > * Source code tag "release-4.5.0" [4] with git sha
> > > 0465f7e6ebc2289334cc97b06fbfb0a8f07e2e8b
> > >
> > > BookKeeper's KEYS file contains PGP keys we used to sign this release:
> > > https://dist.apache.org/repos/dist/release/bookkeeper/KEYS
> > >
> > > Please download these packages and review this release candidate:
> > >
> > > - Review release notes
> > > - Download the source package (verify shasum, and asc) and follow the
> > > instructions to build and run the bookkeeper service.
> > > - Download the binary package (verify shasum, and asc) and follow the
> > > instructions to run the bookkeeper service.
> > > - Review maven repo, release tag, licenses, and any other things you
> > think
> > > it is important to a release.
> > >
> > > The vote will be open for at least 72 hours. It is adopted by majority
> > > approval, with at least 3 PMC affirmative votes.
> > >
> > > Thanks,
> > > Release Manager
> > >
> > > [1]
> > >
> > https://github.com/apache/bookkeeper/pull/4002/files#diff-d361215d5d5abaa96ac3362e70a1d2f80f92d369a59c866e1976ff41ec435f53R4
> > > [2]
> > > https://dist.apache.org/repos/dist/dev/bookkeeper/bookkeeper-4.16.2-rc1/
> > > [3]
> > >
> > https://repository.apache.org/content/repositories/orgapachebookkeeper-1086/
> > > [4] https://github.com/apache/bookkeeper/releases/tag/v4.16.2-rc1
> > >
> >

[DISCUSS] Upgrade RocksDB version to 6.29.4.1 for branch-4.14

[Hang Chen]

Hi guys,

More than one year ago, we had a discussion[1] about reverting the
RocksDB version for branch-4.14, and the main concern is that RocksDB
6.17.3 breaks runtime compatibility with older versions and it
requires the code to be recompiled. Lari did a great investigation and
you can find more details in this Apache Pulsar pull
https://github.com/apache/pulsar/pull/14962. In the discussion, we
have two solutions, and we choose solution 1 in the end.
1) Do not upgrade RocksDB in BookKeeper 4.14.5 and give the user which
uses RocksDB an easy path to upgrade from BK 4.14.4 to 4.14.5
2) Upgrade RocksDB and requires to all the BK users to pay attention
to the RocksDB version they have in the classpath. The risk is to have
runtime errors which will require them to rebuild the application and
redeploy their services.

However, we found two issues about the current RocksDB version
(6.16.4) on branch-4.14
- In our production environment, we found both RocksDB 6.10.2 and
6.16.4 has native memory leak issue[2] and after upgraded the RocksDB
version to 6.29.4.1, the memory leak issue was fixed.
- We have upgraded the RocksDB version to 7.9.2 since 4.16.0, and the
RocksDB can only rollback to 6.27+. It means BookKeeper 4.16.0+ can
rollback to 4.15.x (RocksDB 6.29.4.1) but can't rollback to 4.14.x
(RocksDB 6.16.4) if we don't upgrade the RocksDB version to 6.29.4.1
for branch-4.14

The main reason for rolling back RocksDB version for branch-4.14 is
that RocksDB 6.17.3 breaks runtime compatibility with older versions
and it requires the code to be recompiled. In my opinion, if users
upgrade BookKeeper 4.14.7 to 4.14.8, which upgraded the RocksDB
version from 6.16.4 to 6.29.4.1, they need to use the new BookKeeper
package which has been compiled with new RocksDB version instead of
replace the BookKeeper-server library directly. So it is safe to
upgrade the RocksDB version. I have pushed one PR[4] to upgrade the
RocksDB version to 6.29.4.1 for branch-4.14.

Any concerns?

Thanks,
Hang

[1] https://lists.apache.org/thread/dsdltmoy5ggqx4oj7hrt13hqok80d4jf
[2] https://github.com/apache/bookkeeper/issues/3507
[3] https://github.com/apache/bookkeeper/issues/3734#issuecomment-1407626941
[4] https://github.com/apache/bookkeeper/pull/3947

Re: [DISCUSS] BookKeeper 4.16.2 release

[Hang Chen]

Sure, I added the label `release/4.16.2` and will include this PR, thanks Reid.

Thanks,
Hang

Reid Chan  于2023年6月15日周四 10:23写道：
>
> H teami,
>
> Is it ok to include this PR https://github.com/apache/bookkeeper/pull/3813?
> There are 3 approvals
>
>
> ---
> Best Regards,
> R.C
>
>
> On Wed, Jun 14, 2023 at 4:28 PM Enrico Olivelli  wrote:
>
> > +1
> >
> > Enrico
> >
> > Il Mer 14 Giu 2023, 09:35 Wenbing Shen  ha
> > scritto:
> >
> > > +10
> > >
> > > I noticed https://github.com/apache/bookkeeper/pull/3979 that issue was
> > > introduced in bookkeeper-4.16.0 by
> > > https://github.com/apache/bookkeeper/pull/3847,
> > > and in pulsar-3.0. 0 version introduced bk-4.16.1.
> > >
> > > Our online business runs a pulsar-3.0.0 cluster, we and other
> > pulsar-3.0.0
> > > users urgently need bk-4.16.2.
> > >
> > > Thanks,
> > > wenbingshen
> > >
> > > Hang Chen  于2023年6月13日周二 09:30写道：
> > >
> > > > Hi everyone,
> > > >
> > > > We have found one critical bug[1] in BookKeeper 4.16.1, and I would
> > > > like to discuss the starting 4.16.2 release to include some critical
> > > > bug fixes and improvements, such as [1] and [2]
> > > >
> > > > Here are the PRs for 4.16.2:
> > > >
> > > >
> > >
> > https://github.com/apache/bookkeeper/pulls?q=is%3Aopen+is%3Apr+label%3Arelease%2F4.16.2
> > > >
> > > > If you have PRs that want to be included in this release, please
> > > > comment on that PR or tag the label `release/4.16.2`.
> > > >
> > > > After those PRs merged, I would like to volunteer to release this
> > patch.
> > > >
> > > > Thanks,
> > > > Hang
> > > >
> > > > [1] https://github.com/apache/bookkeeper/pull/3979
> > > > [2] https://github.com/apache/bookkeeper/pull/3983
> > > >
> > >
> >

[DISCUSS] BookKeeper 4.16.2 release

[Hang Chen]

Hi everyone,

We have found one critical bug[1] in BookKeeper 4.16.1, and I would
like to discuss the starting 4.16.2 release to include some critical
bug fixes and improvements, such as [1] and [2]

Here are the PRs for 4.16.2:
https://github.com/apache/bookkeeper/pulls?q=is%3Aopen+is%3Apr+label%3Arelease%2F4.16.2

If you have PRs that want to be included in this release, please
comment on that PR or tag the label `release/4.16.2`.

After those PRs merged, I would like to volunteer to release this patch.

Thanks,
Hang

[1] https://github.com/apache/bookkeeper/pull/3979
[2] https://github.com/apache/bookkeeper/pull/3983

Re: Bookkeeper decommission may be blocked by ledgers that cannot be replicated

[Hang Chen]

Hi Andrey,
Sorry for the late reply. I double-checked the code and found the
`recover` command can solve the problem, but it has a performance
issue.

When we use `bin/bookkeeper shell recover` command to decommission one
bookie, the ledger replication is executed on the node where we run
the recover command and not run on the auto-recovery pods. If one
bookie holds 4TB of ledger data to be replicated, the ledger
replication can't be parallelized by adding more auto-recovery
instances.

IMO, we need another way to decommission a bookie instead of the
`recover` command.

Thanks,
Hang

Andrey Yegorov  于2023年3月30日周四 05:54写道：
>
> Hi,
>
> You can use "recover" command instead.
>
> Switch bookie to read-only (via REST API)
> bin/bookkeeper shell recover ..
> recover command also has a flag to delete the cookie in ZK.
> As an additional benefit, this way you can decomm bookie with ledgers
> created with write quorum = 1.
>
> HTH.
>
> On Sun, Mar 26, 2023 at 9:27 PM Hang Chen  wrote:
>
> > Hi guys, I found the BookKeeper decommission may be blocked by ledgers
> > that cannot be replicated.
> >
> > Current bookie decommissions process.
> >   - Step 1: Use the command `bin/bookkeeper shell listunderreplicated`
> > to check whether there are some ledgers in the under-replicated state
> >   - Step 2: After all the ledgers are replicated complete, stop the
> > bookie and use the command `bin/bookkeeper shell decommissionbookie
> > -bookieid ` to trigger decommission
> >   - Step 3: Wait for all the ledgers to be replicated and the bookie
> > decommission process will complete
> >
> > However, there is a bug in the decommissioning process.
> >
> > In Step 1, those under-replicated state ledgers are marked by the
> > following steps:
> >   - Auditor check lost bookie: it will be triggered by two cases: a)
> > One bookie lost after `lostBookieRecoveryDelay`, b) Check every
> > `auditorPeriodicBookieCheckInterval`.  The default is 24 hours.
> >   - Auditor checks all ledgers: triggered every
> > `auditorPeriodicCheckInterval`. The default is 7 days. It will check
> > every ledger's fragments with the following steps:
> > - For every fragment, calculate pending read entries according to
> > `auditorLedgerVerificationPercentage`, default is `0`, which means
> > only checking the first and last entries of this fragment.
> > - Read those entries from all the bookies in the ensemble list for
> > the pending read entries. If any entries read failed, mark the ledger
> > into an under-replicated state.
> >
> >
> > When we use the `bin/bookkeeper shell listunderreplicated` command to
> > check whether some are under-replicated, it only represents those
> > ledgers missing replicas before the last check. The lost bookie check
> > was 24 hours ago, and the all ledgers check was seven days ago. The
> > time range from the last check to the current timestamp won't mark any
> > missing replicas ledgers. Suppose we set EnsembleSize=3,
> > WriteQuorumSize=2, and AckQuorumSize=1, and decommission one bookie
> > with the current decommission process. In that case, it may result in
> > some ledgers that can't be replicated due to the only available
> > replica on the decommissioned bookie.
> >
> > Moreover, the Auditor checks all ledgers and only checks the first and
> > last entries of each fragment of those ledgers. If the bookie disabled
> > writing journals and some entries are lost in one fragment, but the
> > first and last entries still exist, the checker won't find it.
> >
> > ### Options
> > There are two options to tune the decommissioning process.
> >
> > 1. Trigger-check all ledgers before Step 1. It has the following
> > disadvantages.
> >- It will cost a lot of resources
> >- It only checks the first and last entries of each fragment of
> > those ledgers by default. It can't cover all the entries that check
> >
> >  2. Turn the bookie into read-only mode instead of shutting it down
> > before using the `bin/bookkeeper shell decommissionbookie -bookieid
> > ` command to trigger commission. When replicating
> > ledgers located on the decommission bookie, the ledgers can be
> > replicated successfully if one replica is available.
> >
> > I suggest choosing the second option to tune the current bookie
> > decommission process. Do you have any suggestions?
> >
> > Thanks,
> > Hang
> >
>
>
> --
> Andrey Yegorov

[DISCUSS] BookKeeper 4.14.8 release

[Hang Chen]

Hi everyone,

BookKeeper 4.14.7 was released more than 3 months, and I would like to
discuss the starting 4.14.8 release to include some critical bug fixes
and improvements, such as [1] and [2]

Here are the PRs for 4.14.8:
https://github.com/apache/bookkeeper/pulls?q=is%3Aopen+is%3Apr+label%3Arelease%2F4.14.8

If you have PRs that want to be included in this release, please
comment on that PR or tag the label `release/4.14.8`.

After those PRs merged, I would like to volunteer to release this patch.

Thanks,
Hang

[1] https://github.com/apache/bookkeeper/pull/3959
[2] https://github.com/apache/bookkeeper/pull/3917

[DISCUSS] Support skip invalid journal records in replying journal stage

[Hang Chen]

The journal file may be broken when we use the `kill -9` command to
stop one bookie pod. If we try to start up the bookie pod again, the
pod will start to fail with the following exception.
```
10:15:55.026 [main] ERROR org.apache.bookkeeper.bookie.Bookie -
Exception while replaying journals, shutting down
java.io.IOException: Invalid record found with negative length -448299468
at org.apache.bookkeeper.bookie.Journal.scanJournal(Journal.java:821)
~[org.apache.bookkeeper-bookkeeper-server-4.12.0.jar:4.12.0]
at org.apache.bookkeeper.bookie.Bookie.replay(Bookie.java:945)
~[org.apache.bookkeeper-bookkeeper-server-4.12.0.jar:4.12.0]
at org.apache.bookkeeper.bookie.Bookie.readJournal(Bookie.java:911)
~[org.apache.bookkeeper-bookkeeper-server-4.12.0.jar:4.12.0]
at org.apache.bookkeeper.bookie.Bookie.start(Bookie.java:965)
~[org.apache.bookkeeper-bookkeeper-server-4.12.0.jar:4.12.0]
at org.apache.bookkeeper.proto.BookieServer.start(BookieServer.java:156)
~[org.apache.bookkeeper-bookkeeper-server-4.12.0.jar:4.12.0]
at 
org.apache.bookkeeper.server.service.BookieService.doStart(BookieService.java:68)
~[org.apache.bookkeeper-bookkeeper-server-4.12.0.jar:4.12.0]
at 
org.apache.bookkeeper.common.component.AbstractLifecycleComponent.start(AbstractLifecycleComponent.java:83)
~[org.apache.bookkeeper-bookkeeper-common-4.12.0.jar:4.12.0]
at 
org.apache.bookkeeper.common.component.LifecycleComponentStack.lambda$start$4(LifecycleComponentStack.java:144)
~[org.apache.bookkeeper-bookkeeper-common-4.12.0.jar:4.12.0]
at 
com.google.common.collect.ImmutableList.forEach(ImmutableList.java:406)
[com.google.guava-guava-30.1-jre.jar:?]
at 
org.apache.bookkeeper.common.component.LifecycleComponentStack.start(LifecycleComponentStack.java:144)
[org.apache.bookkeeper-bookkeeper-common-4.12.0.jar:4.12.0]
at 
org.apache.bookkeeper.common.component.ComponentStarter.startComponent(ComponentStarter.java:85)
[org.apache.bookkeeper-bookkeeper-common-4.12.0.jar:4.12.0]
at org.apache.bookkeeper.server.Main.doMain(Main.java:234)
[org.apache.bookkeeper-bookkeeper-server-4.12.0.jar:4.12.0]
at org.apache.bookkeeper.server.Main.main(Main.java:208)
[org.apache.bookkeeper-bookkeeper-server-4.12.0.jar:4.12.0]
10:15:55.134 [main] INFO  org.apache.zookeeper.ZooKeeper - Session:
0x200064a14681be2 closed
```

We should have a way to make the bookie pod startup instead of
decommissioning it.

If we skip the broken journal files in the replying stage, it may lead
to some data loss, even though we enabled the fsync flag for the
journal. However, if we don't skip the broken journal files, we need
to decommission the bookie pod to make those ledgers located at the
bookie replicated to other bookie pods.

I think skipping those broken journal files is acceptable compared to
decommissioning the bookie pod. For those lost data, the auto-recovery
can replicate them from other replicas in other bookie pods.

The related PR is: https://github.com/apache/bookkeeper/pull/3956

Thanks,
Hang

Re: 回复：[Discuss] Bookie may lose data even though we turn on fsync for the journal

[Hang Chen]

> When flushing, only the lastMark value will be persisted to the file, but the 
> lastMark value will not be updated.
> The lastMark value is updated only when the ForceWriteRequest completes. So 
> when the flush is triggered here, the position of lastMark is not 100MB's 
> offset
Yes, you are right. But the persisted lastMark value will be near to
the latest lastMark value. Even though it is not 100MB's offset, it
may be 100MB - 1 offset.


The root cause of this bug is that EntryLogger1 triggers a checkpoint
when its write cache is full, updating both EntryLogger1  and
EntryLogger2's `lastMark` position. However, EntryLogger2's data may
still be in WriteCache, which may lead to data loss when the bookie
shutdown will `kill -9`

There are two solutions for this bug.
 Update `lastMark` position individually.
- When EntryLogger1 triggers the checkpoint, we only update
EntryLogger1's `lastMark` position instead of updating EntryLogger2's
`lastMark` position at the same time.
- When SyncThread triggers the checkpoint, we update all the
EntryLoggers' `lastMark` positions.
- When determining whether a journal file can be deleted, we should
get the smallest `lastMark` position among all the writeable
EntryLoggers, and delete the journal files which less than the
smallest `lastMark` position.
- When replying to the journal on bookie startups, we need to get the
smallest `lastMark` position, and reply to the journal files with this
position, otherwise, we will lose data.

However, one case is hard to handle in replying to the journal stage.
When one ledger disk transfers from ReadOnly to Writable mode, the
`lastMark` position is an old value. Using the old position to reply
to the journal files will lead to a target journal file not found
exception.

 Only update `lastMark` position in SyncThread
Two places can trigger a checkpoint.
- The scheduled tasks in SyncThread.doCheckpoint
- The ledgerDir write-cache full and then flush

The second way is the root cause of data loss if the ledger is
configured with multiple directories.
We can turn off the second way's update `lastMark` position operation
and only make SyncThread update the `lastMark` position in a
checkpoint when the ledger is configured with multiple directories.

This is the simplest way to fix this bug, but it has two drawbacks.
- The `lastMark` position updates depend on SyncThread doing
checkpoint intervals. In Pulsar, the default interval is 60s. It means
the journal file expires with at least 60s
- The bookie startup replying to the journal depends on the `lastMark`
position. It means the journal will reply to at least 60s journal data
before the start-up is complete. It may lead to the bookie start-up
speed slowing down.

IMO, the above two drawbacks can be acceptable compared to data loss.

Thanks,
Hang

Gavin Gao  于2022年6月10日周五 13:31写道：
>
> The problem is:
> flush when writeCache is full is per ledger disks independent, but they share 
> the same journal disk lastMark value.
>
> On 2022/06/07 04:16:58 lordcheng10 wrote:
> > > In flushing
> > the write cache, it will trigger a checkpoint to mark the journal’s
> > lastMark position (100MB’s offset)
> >
> >
> > When flushing, only the lastMark value will be persisted to the file, but 
> > the lastMark value will not be updated.
> > The lastMark value is updated only when the ForceWriteRequest completes. So 
> > when the flush is triggered here, the position of lastMark is not 100MB's 
> > offset
> >
> >
> > I’m not sure whether I missed some logic.
> >
> > -- 原始邮件 --
> > 发件人: "Hang Chen" > 发送时间: 2022年5月30日(星期一) 上午9:21
> > 收件人: "dev" > 主题: [Discuss] Bookie may lose data even though we turn on fsync for the 
> > journal
> >
> >
> >
> > We found one place where the bookie may lose data even though we turn
> > on fsync for the journal.
> > Condition:
> > - One journal disk, and turn on fsync for the journal
> > - Configure two ledger disks, ledger1, and ledger2
> >
> > Assume we write 100MB data into one bookie, 70MB data written into
> > ledger1's write cache, and 30 MB data written into ledger2's write
> > cache. Ledger1's write cache is full and triggers flush. In flushing
> > the write cache, it will trigger a checkpoint to mark the journal’s
> > lastMark position (100MB’s offset) and write the lastMark position
> > into both ledger1 and ledger2's lastMark file.
> >
> > At this time, this bookie shutdown without flush write cache, such as
> > shutdown by `kill -9` command, and ledger2's write cache (30MB)
> > doesn’t flush into ledger disk. But ledger2's lastMark position which
> > persisted into

[ANNOUNCE] Apache BookKeeper 4.16.1 released

[Hang Chen]

The Apache BookKeeper team is proud to announce Apache BookKeeper
version 4.16.1.

Apache BookKeeper is a scalable, fault-tolerant, and low-latency
storage service optimized for real-time workloads. It has been used
for a fundamental service to build reliable services. It is also the
log segment store for Apache DistributedLog and the message store for
Apache Pulsar.

This is the 47rd release of the Apache BookKeeper.

Release 4.16.1 fixed one critical regression introduced by 4.16.0

Apache BookKeeper users are encouraged to upgrade to 4.16.1. The
technical details of this release are summarized below.

For BookKeeper release details and downloads, visit:
https://bookkeeper.apache.org/releases

BookKeeper 4.16.1 Release Notes are at:
https://bookkeeper.apache.org/release-notes/#4161

We would like to thank the contributors that made the release possible.

Regards,

The BookKeeper Team

[ANNOUNCE] Apache BookKeeper 4.16.0 released

[Hang Chen]

The Apache BookKeeper team is proud to announce Apache BookKeeper
version 4.16.0.

Apache BookKeeper is a scalable, fault-tolerant, and low-latency
storage service optimized for real-time workloads. It has been used
for a fundamental service to build reliable services. It is also the
log segment store for Apache DistributedLog and the message store for
Apache Pulsar.

This is the 46rd release of the Apache BookKeeper.

Release 4.16.0 includes multiple important features, improvements, bug
fixes and some dependencies CVE fixes.

Due to this version having one critical regression in the BookKeeper
client, Apache BookKeeper users are encouraged to skip this version
and upgrade to 4.16.1.

The technical details of this release are summarized below.

For BookKeeper release details and downloads, visit:
https://bookkeeper.apache.org/releases

BookKeeper 4.16.0 Release Notes are at:
https://bookkeeper.apache.org/release-notes/#4160

We would like to thank the contributors that made the release possible.

Regards,

The BookKeeper Team

Re: [VOTE] Release 4.16.1, release candidate #0

[Hang Chen]

The vote is now closed, and the release was approved with 3 +1s(3
bindings), and I will continue with the following steps.

Binding votes:
* Enrico Olivelli
* Andrey Yegorov
* Jia Zhai

Thank you all for verifying and voting!

Thanks,
Hang

Jia Zhai  于2023年4月18日周二 08:37写道：
>
> +1 (binding)
>
> - verified packages checksum and signatures.
>
> - the source package is built and runs successfully.
>
> - binary package 'bin/bookkeeper standalone' runs well.
>
> On Tue, Apr 18, 2023 at 1:33 AM Andrey Yegorov  wrote:
>
> > +1 (binding)
> >
> > * reviewed release notes
> >
> > * built from tag, Mac/Java 11
> >
> > * checked signatures (ok) and checksums (ok)
> >
> > * bin/bookkeeper standalone starts (from packaged binaries)
> >
> > * bin/bookkeeper localbookie 3 starts (from packaged binaries)
> >
> >
> >
> > On Fri, Apr 14, 2023 at 9:18 PM Hang Chen  wrote:
> >
> > > Hi everyone,
> > > Please review and vote on the release candidate #0 for the version
> > 4.16.1,
> > > as follows:
> > > [ ] +1, Approve the release
> > > [ ] -1, Do not approve the release (please provide specific comments)
> > >
> > > The complete staging area is available for your review, which includes:
> > > * Release notes [1]
> > > * The official Apache source and binary distributions to be deployed to
> > > dist.apache.org [2]
> > > * All artifacts to be deployed to the Maven Central Repository [3]
> > > * Source code tag "v4.16.1-rc0" [4] with git sha [5]
> > >
> > > BookKeeper's KEYS file contains PGP keys we used to sign this release:
> > > https://dist.apache.org/repos/dist/release/bookkeeper/KEYS
> > >
> > > Please download these packages and review this release candidate:
> > >
> > > - Review release notes
> > > - Download the source package (verify shasum, and asc) and follow the
> > > instructions to build and run the bookkeeper service.
> > > - Download the binary package (verify shasum, and asc) and follow the
> > > instructions to run the bookkeeper service.
> > > - Review maven repo, release tag, licenses, and any other things you
> > think
> > > it is important to release.
> > >
> > > The vote will be open for at least 72 hours. It is adopted by majority
> > > approval, with at least 3 PMC affirmative votes.
> > >
> > > Thanks,
> > > Release Manager
> > >
> > > [1] https://github.com/apache/bookkeeper/pull/3922/files
> > > [2]
> > > https://dist.apache.org/repos/dist/dev/bookkeeper/bookkeeper-4.16.1-rc0/
> > > [3]
> > >
> > https://repository.apache.org/content/repositories/orgapachebookkeeper-1084/
> > > [4] https://github.com/apache/bookkeeper/tree/v4.16.1-rc0
> > > [5] 8122a87c78a0159e422fc0baac43201eda44ae94
> > >
> >

[VOTE] Release 4.16.1, release candidate #0

[Hang Chen]

Hi everyone,
Please review and vote on the release candidate #0 for the version 4.16.1,
as follows:
[ ] +1, Approve the release
[ ] -1, Do not approve the release (please provide specific comments)

The complete staging area is available for your review, which includes:
* Release notes [1]
* The official Apache source and binary distributions to be deployed to
dist.apache.org [2]
* All artifacts to be deployed to the Maven Central Repository [3]
* Source code tag "v4.16.1-rc0" [4] with git sha [5]

BookKeeper's KEYS file contains PGP keys we used to sign this release:
https://dist.apache.org/repos/dist/release/bookkeeper/KEYS

Please download these packages and review this release candidate:

- Review release notes
- Download the source package (verify shasum, and asc) and follow the
instructions to build and run the bookkeeper service.
- Download the binary package (verify shasum, and asc) and follow the
instructions to run the bookkeeper service.
- Review maven repo, release tag, licenses, and any other things you think
it is important to release.

The vote will be open for at least 72 hours. It is adopted by majority
approval, with at least 3 PMC affirmative votes.

Thanks,
Release Manager

[1] https://github.com/apache/bookkeeper/pull/3922/files
[2] https://dist.apache.org/repos/dist/dev/bookkeeper/bookkeeper-4.16.1-rc0/
[3] https://repository.apache.org/content/repositories/orgapachebookkeeper-1084/
[4] https://github.com/apache/bookkeeper/tree/v4.16.1-rc0
[5] 8122a87c78a0159e422fc0baac43201eda44ae94

[DISCUSS] BookKeeper 4.16.1 release

[Hang Chen]

Hi guys,
 We found one critical regression [1] on the BookKeeper 4.16.0
release, and we has one PR [2] to fix it.  Due to the BookKeeper
4.16.0 has the critical regression, I would like to discuss starting
4.16.1 release for this critical bug. In 4.16.1, we will only include.
this PR [2].

After this PR merged, I would like to volunteer to release this patch.

Thanks,
Hang


[1] https://lists.apache.org/thread/n4dpmbo5t9bvq5t1fp8fn4m6c6d9d9so
[2] https://github.com/apache/bookkeeper/pull/3919

Re: [VOTE] Release 4.16.0, release candidate #2

[Hang Chen]

The vote is now closed, and the release was approved with 3 +1s(3
bindings), and I will continue with the following steps.

Binding votes:
* Andrey Yegorov
* Matteo Merli
* Enrico Olivelli

Thank you all for verifying and voting!

Thanks,
Hang

Enrico Olivelli  于2023年4月7日周五 14:42写道：
>
> +1 (binding)
>
> - Built and run tests locally on jdk11 on M1
> - Run smoke tests
> - Verified checksum and signatures
>
>
> Enrico
>
> Il Sab 1 Apr 2023, 08:35 Andrey Yegorov  ha
> scritto:
>
> > > Have you ever run `mvn clean insteall -DskipTests` before
> > running the tests
> >
> > of course.
> >
> > as listed above:
> > $ git checkout v4.16.0-rc2
> > $ mvn clean install -DskipTests
> > $ mvn test
> >
> > On Fri, Mar 31, 2023 at 8:19 PM Hang Chen  wrote:
> >
> > > > I added the logs to the issue.
> > >
> > > It looks like the loopback address problem, and the Bookie server does
> > > not allow loopback interfaces as the primary interface. You can set
> > > allowLoopback to true and rerun the test.
> > >
> > >
> > > > FWIW I ran the tests from the command line, no from IntelliJ.
> > >
> > > >$ mvn -v
> > > >Apache Maven 3.8.6 (84538c9988a25aec085021c365c560670ad80f63)
> > > >Maven home: /usr/local/Cellar/maven/3.8.6/libexec
> > > >Java version: 11.0.11, vendor: AdoptOpenJDK, runtime:
> > >
> > >
> > >/Users/andreyyegorov/Library/Java/JavaVirtualMachines/adopt-openjdk-11.0.11/Contents/Home
> > > >Default locale: en_US, platform encoding: UTF-8
> > > >OS name: "mac os x", version: "11.5", arch: "x86_64", family: "mac"
> > >
> > > I ran the test from the command line without any changes, and the tests
> > > passed.
> > > $ mvn test -Dtest=DbLedgerStorageWithDirectEntryLoggerTest
> > > [INFO] Running
> > >
> > org.apache.bookkeeper.bookie.storage.ldb.DbLedgerStorageWithDirectEntryLoggerTest
> > > [INFO] Tests run: 14, Failures: 0, Errors: 0, Skipped: 0, Time
> > > elapsed: 16.021 s - in
> > >
> > >
> > org.apache.bookkeeper.bookie.storage.ldb.DbLedgerStorageWithDirectEntryLoggerTest
> > >
> > > $ mvn -v
> > > Apache Maven 3.8.6 (84538c9988a25aec085021c365c560670ad80f63)
> > > Maven home: /Users/hangc/Cloud/apache-maven-3.8.6
> > > Java version: 11.0.12, vendor: Oracle Corporation, runtime:
> > > /Library/Java/JavaVirtualMachines/jdk-11.0.12.jdk/Contents/Home
> > > Default locale: en_CN, platform encoding: UTF-8
> > > OS name: "mac os x", version: "11.5", arch: "x86_64", family: "mac"
> > >
> > > @Andrey Have you ever run `mvn clean insteall -DskipTests` before
> > > running the tests?
> > >
> > > Thanks,
> > > Hang
> > >
> > > Matteo Merli  于2023年4月1日周六 01:32写道：
> > > >
> > > > +1 binding
> > > >
> > > > * Checked signatures
> > > > * Check localbookie and local perf tool
> > > >
> > > >
> > > >
> > > > --
> > > > Matteo Merli
> > > > 
> > > >
> > > >
> > > > On Wed, Mar 29, 2023 at 7:06 AM Hang Chen  wrote:
> > > >
> > > > > Hi everyone,
> > > > > Please review and vote on the release candidate #2 for version
> > 4.16.0,
> > > > > as follows:
> > > > > [ ] +1, Approve the release
> > > > > [ ] -1, Do not approve the release (please provide specific comments)
> > > > >
> > > > > The complete staging area is available for your review, which
> > includes:
> > > > > * Release notes [1]
> > > > > * The official Apache source and binary distributions to be deployed
> > to
> > > > > dist.apache.org [2]
> > > > > * All artifacts to be deployed to the Maven Central Repository [3]
> > > > > * Source code tag "v4.16.0-rc2" [4] with git sha [5]
> > > > >
> > > > > BookKeeper's KEYS file contains the PGP keys we used to sign this
> > > release:
> > > > > https://dist.apache.org/repos/dist/release/bookkeeper/KEYS
> > > > >
> > > > > Please download these packages and review this release candidate:
> > > > >
> > > > > - Review release notes
> > > > > - Download the source package (verify shasum, and asc) and follow the
> > > > > instructions to build and run the bookkeeper service.
> > > > > - Download the binary package (verify shasum, and asc) and follow the
> > > > > instructions to run the bookkeeper service.
> > > > > - Review the maven repo, release tag, licenses, and any other things
> > > you
> > > > > think
> > > > > it is important to release.
> > > > >
> > > > > The vote will be open for at least 72 hours. It is adopted by the
> > > majority
> > > > > approval, with at least 3 PMC affirmative votes.
> > > > >
> > > > > Thanks,
> > > > > Release Manager
> > > > >
> > > > > [1] https://github.com/apache/bookkeeper/pull/3882/files
> > > > > [2]
> > > > >
> > > https://dist.apache.org/repos/dist/dev/bookkeeper/bookkeeper-4.16.0-rc2/
> > > > > [3]
> > > > >
> > >
> > https://repository.apache.org/content/repositories/orgapachebookkeeper-1083/
> > > > > [4] https://github.com/apache/bookkeeper/tree/v4.16.0-rc2
> > > > > [5] 8890a61418e3961e108b3c2e8ea176691e5f
> > > > >
> > >
> >
> >
> > --
> > Andrey Yegorov
> >

Re: [VOTE] Release 4.16.0, release candidate #2

[Hang Chen]

> I added the logs to the issue.

It looks like the loopback address problem, and the Bookie server does
not allow loopback interfaces as the primary interface. You can set
allowLoopback to true and rerun the test.


> FWIW I ran the tests from the command line, no from IntelliJ.

>$ mvn -v
>Apache Maven 3.8.6 (84538c9988a25aec085021c365c560670ad80f63)
>Maven home: /usr/local/Cellar/maven/3.8.6/libexec
>Java version: 11.0.11, vendor: AdoptOpenJDK, runtime:
>/Users/andreyyegorov/Library/Java/JavaVirtualMachines/adopt-openjdk-11.0.11/Contents/Home
>Default locale: en_US, platform encoding: UTF-8
>OS name: "mac os x", version: "11.5", arch: "x86_64", family: "mac"

I ran the test from the command line without any changes, and the tests passed.
$ mvn test -Dtest=DbLedgerStorageWithDirectEntryLoggerTest
[INFO] Running 
org.apache.bookkeeper.bookie.storage.ldb.DbLedgerStorageWithDirectEntryLoggerTest
[INFO] Tests run: 14, Failures: 0, Errors: 0, Skipped: 0, Time
elapsed: 16.021 s - in
org.apache.bookkeeper.bookie.storage.ldb.DbLedgerStorageWithDirectEntryLoggerTest

$ mvn -v
Apache Maven 3.8.6 (84538c9988a25aec085021c365c560670ad80f63)
Maven home: /Users/hangc/Cloud/apache-maven-3.8.6
Java version: 11.0.12, vendor: Oracle Corporation, runtime:
/Library/Java/JavaVirtualMachines/jdk-11.0.12.jdk/Contents/Home
Default locale: en_CN, platform encoding: UTF-8
OS name: "mac os x", version: "11.5", arch: "x86_64", family: "mac"

@Andrey Have you ever run `mvn clean insteall -DskipTests` before
running the tests?

Thanks,
Hang

Matteo Merli  于2023年4月1日周六 01:32写道：
>
> +1 binding
>
> * Checked signatures
> * Check localbookie and local perf tool
>
>
>
> --
> Matteo Merli
> 
>
>
> On Wed, Mar 29, 2023 at 7:06 AM Hang Chen  wrote:
>
> > Hi everyone,
> > Please review and vote on the release candidate #2 for version 4.16.0,
> > as follows:
> > [ ] +1, Approve the release
> > [ ] -1, Do not approve the release (please provide specific comments)
> >
> > The complete staging area is available for your review, which includes:
> > * Release notes [1]
> > * The official Apache source and binary distributions to be deployed to
> > dist.apache.org [2]
> > * All artifacts to be deployed to the Maven Central Repository [3]
> > * Source code tag "v4.16.0-rc2" [4] with git sha [5]
> >
> > BookKeeper's KEYS file contains the PGP keys we used to sign this release:
> > https://dist.apache.org/repos/dist/release/bookkeeper/KEYS
> >
> > Please download these packages and review this release candidate:
> >
> > - Review release notes
> > - Download the source package (verify shasum, and asc) and follow the
> > instructions to build and run the bookkeeper service.
> > - Download the binary package (verify shasum, and asc) and follow the
> > instructions to run the bookkeeper service.
> > - Review the maven repo, release tag, licenses, and any other things you
> > think
> > it is important to release.
> >
> > The vote will be open for at least 72 hours. It is adopted by the majority
> > approval, with at least 3 PMC affirmative votes.
> >
> > Thanks,
> > Release Manager
> >
> > [1] https://github.com/apache/bookkeeper/pull/3882/files
> > [2]
> > https://dist.apache.org/repos/dist/dev/bookkeeper/bookkeeper-4.16.0-rc2/
> > [3]
> > https://repository.apache.org/content/repositories/orgapachebookkeeper-1083/
> > [4] https://github.com/apache/bookkeeper/tree/v4.16.0-rc2
> > [5] 8890a61418e3961e108b3c2e8ea176691e5f
> >

Re: [VOTE] Release 4.16.0, release candidate #2

[Hang Chen]

Hi Andrey,
Thanks for your great job!

For the failed tests,
- org.apache.bookkeeper.test.BookieZKExpireTest is a test issue, and I
will fix it later. It is not a blocker for this release.
- org.apache.bookkeeper.client.TestSpeculativeRead, I have run the
test on my Mac multiple times, and they are all passed. Would you
please share the detailed logs with me? I think it also may not be a
blocker for this release.
- Direct entry logger tests fail on mac (probably by design, but tests
should handle this / don't run on mac). I replied under this issue
[1]. To make the tests run in the Intellij idea, we need to copy the
`libnative-io.jnilib` file to a target lib directory. After copying
the file, all the direct entry logger tests are passed on my Macbook.

For the gpg warning, my gpg key has been signed by two guys [2]. I
have no idea why it still reports not certified with a trusted
signature.

Thanks,
Hang

[1] https://github.com/apache/bookkeeper/issues/3897#issuecomment-1492138987
[2] https://pgp.mit.edu/pks/lookup?op=vindex&search=0xA641378C97944D34

Andrey Yegorov  于2023年3月31日周五 04:58写道：
>
> +1 (binding)
>
> There are test failures on mac but I believe they fail only on mac (please
> double check).
> I briefly looked at these but had to stop/timed out.
>
> What I did:
>
> * reviewed release notes (left some comments)
>
> * built/test from tag, Mac/Java 11
>
> $ git checkout v4.16.0-rc2
> $ mvn clean install -DskipTests
> $ mvn test
>
> Failures:
> 1. org.apache.bookkeeper.test.BookieZKExpireTest - added
> https://github.com/apache/bookkeeper/issues/3898
> 2. org.apache.bookkeeper.client.TestSpeculativeRead - added
> https://github.com/apache/bookkeeper/issues/3899
> 3. Direct entry logger tests fail on mac (probably by design but tests
> should handle this / don't run on mac) - added
> https://github.com/apache/bookkeeper/issues/3897
>
> * checked signatures (ok) and checksums (ok)
>
> $ cd ~/Downloads
> $ wget --no-parent -r https://dist.apache.org/repos/dist/dev/bookkeeper/
> bookkeeper-4.16.0-rc2/
> $ cd dist.apache.org/repos/dist/dev/bookkeeper/bookkeeper-4.16.0-rc2
> $ gpg --keyserver hkps://pgp.mit.edu/ --search-keys " email here>"
> $ for f in *.asc; do gpg --verify $f ${f%.*}; done
>
> All is good, there is a warning
>gpg: WARNING: This key is not certified with a trusted signature!
>gpg:  There is no indication that the signature belongs to the
> owner.
> I think you may need to "verify" your key, don't remember how.
>
> $ for f in *.tar.gz; do shasum -a 512 $f | diff $f.sha512 -; done
>
> * bin/bookkeeper standalone starts (from packaged binaries)
>
> * bin/bookkeeper localbookie 3 starts (from built binaries)
>
> * compared gzipped source with sources at tag (ok)
>
> * built from gzipped sources (didn't run the tests)
>
>
> On Wed, Mar 29, 2023 at 7:07 AM Hang Chen  wrote:
>
> > Hi everyone,
> > Please review and vote on the release candidate #2 for version 4.16.0,
> > as follows:
> > [ ] +1, Approve the release
> > [ ] -1, Do not approve the release (please provide specific comments)
> >
> > The complete staging area is available for your review, which includes:
> > * Release notes [1]
> > * The official Apache source and binary distributions to be deployed to
> > dist.apache.org [2]
> > * All artifacts to be deployed to the Maven Central Repository [3]
> > * Source code tag "v4.16.0-rc2" [4] with git sha [5]
> >
> > BookKeeper's KEYS file contains the PGP keys we used to sign this release:
> > https://dist.apache.org/repos/dist/release/bookkeeper/KEYS
> >
> > Please download these packages and review this release candidate:
> >
> > - Review release notes
> > - Download the source package (verify shasum, and asc) and follow the
> > instructions to build and run the bookkeeper service.
> > - Download the binary package (verify shasum, and asc) and follow the
> > instructions to run the bookkeeper service.
> > - Review the maven repo, release tag, licenses, and any other things you
> > think
> > it is important to release.
> >
> > The vote will be open for at least 72 hours. It is adopted by the majority
> > approval, with at least 3 PMC affirmative votes.
> >
> > Thanks,
> > Release Manager
> >
> > [1] https://github.com/apache/bookkeeper/pull/3882/files
> > [2]
> > https://dist.apache.org/repos/dist/dev/bookkeeper/bookkeeper-4.16.0-rc2/
> > [3]
> > https://repository.apache.org/content/repositories/orgapachebookkeeper-1083/
> > [4] https://github.com/apache/bookkeeper/tree/v4.16.0-rc2
> > [5] 8890a61418e3961e108b3c2e8ea176691e5f
> >
>
>
> --
> Andrey Yegorov

Re: [VOTE] Release 4.16.0, release candidate #2

[Hang Chen]

I have created one PR[1] to upgrade the BookKeeper dependency to
4.16.0 for Apache Pulsar, and all the tests have been passed.

Thanks,
Hang


[1] https://github.com/hangc0276/pulsar/pull/15

Hang Chen  于2023年3月29日周三 22:06写道：
>
> Hi everyone,
> Please review and vote on the release candidate #2 for version 4.16.0,
> as follows:
> [ ] +1, Approve the release
> [ ] -1, Do not approve the release (please provide specific comments)
>
> The complete staging area is available for your review, which includes:
> * Release notes [1]
> * The official Apache source and binary distributions to be deployed to
> dist.apache.org [2]
> * All artifacts to be deployed to the Maven Central Repository [3]
> * Source code tag "v4.16.0-rc2" [4] with git sha [5]
>
> BookKeeper's KEYS file contains the PGP keys we used to sign this release:
> https://dist.apache.org/repos/dist/release/bookkeeper/KEYS
>
> Please download these packages and review this release candidate:
>
> - Review release notes
> - Download the source package (verify shasum, and asc) and follow the
> instructions to build and run the bookkeeper service.
> - Download the binary package (verify shasum, and asc) and follow the
> instructions to run the bookkeeper service.
> - Review the maven repo, release tag, licenses, and any other things you think
> it is important to release.
>
> The vote will be open for at least 72 hours. It is adopted by the majority
> approval, with at least 3 PMC affirmative votes.
>
> Thanks,
> Release Manager
>
> [1] https://github.com/apache/bookkeeper/pull/3882/files
> [2] https://dist.apache.org/repos/dist/dev/bookkeeper/bookkeeper-4.16.0-rc2/
> [3] 
> https://repository.apache.org/content/repositories/orgapachebookkeeper-1083/
> [4] https://github.com/apache/bookkeeper/tree/v4.16.0-rc2
> [5] 8890a61418e3961e108b3c2e8ea176691e5f

Re: [VOTE] Release 4.16.0, release candidate #1

[Hang Chen]

Hi all,
We found one log4j dependency issue [1], one heap memory OOM issue
[2] and one CVE issue [3]. Due to this release is still in voting
stage, I prefer to include those fixes in 4.16.0. So I close the vote
and send out a new candidate soon.

Thanks,
Hang

[1] https://github.com/apache/bookkeeper/pull/3892
[2] https://github.com/apache/bookkeeper/pull/3888
[3] https://github.com/apache/bookkeeper/pull/3896

Enrico Olivelli  于2023年3月28日周二 00:11写道：
>
> +1 (binding)
>
> - Built from sources and run all the tests (it is the first time I see
> all tests passing the first time I launch them !!)
> - Verified signatures and checksums
> - Run some smoke tests on JDK11/Mac M1
>
>
> Enrico
>
> Il giorno lun 27 mar 2023 alle ore 03:41 Hang Chen
>  ha scritto:
> >
> > I have created one PR[1] to upgrade the BookKeeper dependency to
> > 4.16.0 for Apache Pulsar, and all the tests have been passed.
> >
> > Thanks,
> > Hang
> >
> >
> > [1] https://github.com/hangc0276/pulsar/pull/14
> >
> > Hang Chen  于2023年3月26日周日 13:26写道：
> > >
> > > Hi everyone,
> > > Please review and vote on the release candidate #1 for version 4.16.0,
> > > as follows:
> > > [ ] +1, Approve the release
> > > [ ] -1, Do not approve the release (please provide specific comments)
> > >
> > > The complete staging area is available for your review, which includes:
> > > * Release notes [1]
> > > * The official Apache source and binary distributions to be deployed to
> > > dist.apache.org [2]
> > > * All artifacts to be deployed to the Maven Central Repository [3]
> > > * Source code tag "v4.16.0-rc1" [4] with git sha [5]
> > >
> > > BookKeeper's KEYS file contains the PGP keys we used to sign this release:
> > > https://dist.apache.org/repos/dist/release/bookkeeper/KEYS
> > >
> > > Please download these packages and review this release candidate:
> > >
> > > - Review release notes
> > > - Download the source package (verify shasum, and asc) and follow the
> > > instructions to build and run the bookkeeper service.
> > > - Download the binary package (verify shasum, and asc) and follow the
> > > instructions to run the bookkeeper service.
> > > - Review the maven repo, release tag, licenses, and any other things you 
> > > think
> > > it is important to release.
> > >
> > > The vote will be open for at least 72 hours. It is adopted by the
> > > majority approval, with at least 3 PMC affirmative votes.
> > >
> > > Thanks,
> > > Release Manager
> > >
> > > [1] https://github.com/apache/bookkeeper/pull/3882/files
> > > [2] 
> > > https://dist.apache.org/repos/dist/dev/bookkeeper/bookkeeper-4.16.0-rc1/
> > > [3] 
> > > https://repository.apache.org/content/repositories/orgapachebookkeeper-1082/
> > > [4] https://github.com/apache/bookkeeper/tree/v4.16.0-rc1
> > > [5] 5474a84549de8e02470014b2c7462eb97e71034e

[VOTE] Release 4.16.0, release candidate #2

[Hang Chen]

Hi everyone,
Please review and vote on the release candidate #2 for version 4.16.0,
as follows:
[ ] +1, Approve the release
[ ] -1, Do not approve the release (please provide specific comments)

The complete staging area is available for your review, which includes:
* Release notes [1]
* The official Apache source and binary distributions to be deployed to
dist.apache.org [2]
* All artifacts to be deployed to the Maven Central Repository [3]
* Source code tag "v4.16.0-rc2" [4] with git sha [5]

BookKeeper's KEYS file contains the PGP keys we used to sign this release:
https://dist.apache.org/repos/dist/release/bookkeeper/KEYS

Please download these packages and review this release candidate:

- Review release notes
- Download the source package (verify shasum, and asc) and follow the
instructions to build and run the bookkeeper service.
- Download the binary package (verify shasum, and asc) and follow the
instructions to run the bookkeeper service.
- Review the maven repo, release tag, licenses, and any other things you think
it is important to release.

The vote will be open for at least 72 hours. It is adopted by the majority
approval, with at least 3 PMC affirmative votes.

Thanks,
Release Manager

[1] https://github.com/apache/bookkeeper/pull/3882/files
[2] https://dist.apache.org/repos/dist/dev/bookkeeper/bookkeeper-4.16.0-rc2/
[3] https://repository.apache.org/content/repositories/orgapachebookkeeper-1083/
[4] https://github.com/apache/bookkeeper/tree/v4.16.0-rc2
[5] 8890a61418e3961e108b3c2e8ea176691e5f

Bookkeeper decommission may be blocked by ledgers that cannot be replicated

[Hang Chen]

Hi guys, I found the BookKeeper decommission may be blocked by ledgers
that cannot be replicated.

Current bookie decommissions process.
  - Step 1: Use the command `bin/bookkeeper shell listunderreplicated`
to check whether there are some ledgers in the under-replicated state
  - Step 2: After all the ledgers are replicated complete, stop the
bookie and use the command `bin/bookkeeper shell decommissionbookie
-bookieid ` to trigger decommission
  - Step 3: Wait for all the ledgers to be replicated and the bookie
decommission process will complete

However, there is a bug in the decommissioning process.

In Step 1, those under-replicated state ledgers are marked by the
following steps:
  - Auditor check lost bookie: it will be triggered by two cases: a)
One bookie lost after `lostBookieRecoveryDelay`, b) Check every
`auditorPeriodicBookieCheckInterval`.  The default is 24 hours.
  - Auditor checks all ledgers: triggered every
`auditorPeriodicCheckInterval`. The default is 7 days. It will check
every ledger's fragments with the following steps:
- For every fragment, calculate pending read entries according to
`auditorLedgerVerificationPercentage`, default is `0`, which means
only checking the first and last entries of this fragment.
- Read those entries from all the bookies in the ensemble list for
the pending read entries. If any entries read failed, mark the ledger
into an under-replicated state.


When we use the `bin/bookkeeper shell listunderreplicated` command to
check whether some are under-replicated, it only represents those
ledgers missing replicas before the last check. The lost bookie check
was 24 hours ago, and the all ledgers check was seven days ago. The
time range from the last check to the current timestamp won't mark any
missing replicas ledgers. Suppose we set EnsembleSize=3,
WriteQuorumSize=2, and AckQuorumSize=1, and decommission one bookie
with the current decommission process. In that case, it may result in
some ledgers that can't be replicated due to the only available
replica on the decommissioned bookie.

Moreover, the Auditor checks all ledgers and only checks the first and
last entries of each fragment of those ledgers. If the bookie disabled
writing journals and some entries are lost in one fragment, but the
first and last entries still exist, the checker won't find it.

### Options
There are two options to tune the decommissioning process.

1. Trigger-check all ledgers before Step 1. It has the following disadvantages.
   - It will cost a lot of resources
   - It only checks the first and last entries of each fragment of
those ledgers by default. It can't cover all the entries that check

 2. Turn the bookie into read-only mode instead of shutting it down
before using the `bin/bookkeeper shell decommissionbookie -bookieid
` command to trigger commission. When replicating
ledgers located on the decommission bookie, the ledgers can be
replicated successfully if one replica is available.

I suggest choosing the second option to tune the current bookie
decommission process. Do you have any suggestions?

Thanks,
Hang

Re: [VOTE] Release 4.16.0, release candidate #1

[Hang Chen]

I have created one PR[1] to upgrade the BookKeeper dependency to
4.16.0 for Apache Pulsar, and all the tests have been passed.

Thanks,
Hang


[1] https://github.com/hangc0276/pulsar/pull/14

Hang Chen  于2023年3月26日周日 13:26写道：
>
> Hi everyone,
> Please review and vote on the release candidate #1 for version 4.16.0,
> as follows:
> [ ] +1, Approve the release
> [ ] -1, Do not approve the release (please provide specific comments)
>
> The complete staging area is available for your review, which includes:
> * Release notes [1]
> * The official Apache source and binary distributions to be deployed to
> dist.apache.org [2]
> * All artifacts to be deployed to the Maven Central Repository [3]
> * Source code tag "v4.16.0-rc1" [4] with git sha [5]
>
> BookKeeper's KEYS file contains the PGP keys we used to sign this release:
> https://dist.apache.org/repos/dist/release/bookkeeper/KEYS
>
> Please download these packages and review this release candidate:
>
> - Review release notes
> - Download the source package (verify shasum, and asc) and follow the
> instructions to build and run the bookkeeper service.
> - Download the binary package (verify shasum, and asc) and follow the
> instructions to run the bookkeeper service.
> - Review the maven repo, release tag, licenses, and any other things you think
> it is important to release.
>
> The vote will be open for at least 72 hours. It is adopted by the
> majority approval, with at least 3 PMC affirmative votes.
>
> Thanks,
> Release Manager
>
> [1] https://github.com/apache/bookkeeper/pull/3882/files
> [2] https://dist.apache.org/repos/dist/dev/bookkeeper/bookkeeper-4.16.0-rc1/
> [3] 
> https://repository.apache.org/content/repositories/orgapachebookkeeper-1082/
> [4] https://github.com/apache/bookkeeper/tree/v4.16.0-rc1
> [5] 5474a84549de8e02470014b2c7462eb97e71034e

[VOTE] Release 4.16.0, release candidate #1

[Hang Chen]

Hi everyone,
Please review and vote on the release candidate #1 for version 4.16.0,
as follows:
[ ] +1, Approve the release
[ ] -1, Do not approve the release (please provide specific comments)

The complete staging area is available for your review, which includes:
* Release notes [1]
* The official Apache source and binary distributions to be deployed to
dist.apache.org [2]
* All artifacts to be deployed to the Maven Central Repository [3]
* Source code tag "v4.16.0-rc1" [4] with git sha [5]

BookKeeper's KEYS file contains the PGP keys we used to sign this release:
https://dist.apache.org/repos/dist/release/bookkeeper/KEYS

Please download these packages and review this release candidate:

- Review release notes
- Download the source package (verify shasum, and asc) and follow the
instructions to build and run the bookkeeper service.
- Download the binary package (verify shasum, and asc) and follow the
instructions to run the bookkeeper service.
- Review the maven repo, release tag, licenses, and any other things you think
it is important to release.

The vote will be open for at least 72 hours. It is adopted by the
majority approval, with at least 3 PMC affirmative votes.

Thanks,
Release Manager

[1] https://github.com/apache/bookkeeper/pull/3882/files
[2] https://dist.apache.org/repos/dist/dev/bookkeeper/bookkeeper-4.16.0-rc1/
[3] https://repository.apache.org/content/repositories/orgapachebookkeeper-1082/
[4] https://github.com/apache/bookkeeper/tree/v4.16.0-rc1
[5] 5474a84549de8e02470014b2c7462eb97e71034e

Re: [VOTE] Release 4.16.0, release candidate #0

[Hang Chen]

I found one critical bug [1], which will cause the writing entry to
fail when skipping the journal.

I close this vote and send out a new candidate soon.

Thanks,
Hang


[1] https://github.com/apache/bookkeeper/pull/3884

Hang Chen  于2023年3月23日周四 16:05写道：
>
> Hi everyone,
> Please review and vote on the release candidate #0 for version 4.16.0,
> as follows:
> [ ] +1, Approve the release
> [ ] -1, Do not approve the release (please provide specific comments)
>
> The complete staging area is available for your review, which includes:
> * Release notes [1]
> * The official Apache source and binary distributions to be deployed
> to dist.apache.org [2]
> * All artifacts to be deployed to the Maven Central Repository [3]
> * Source code tag "v4.16.0-rc0" [4] with git sha [5]
>
> BookKeeper's KEYS file contains the PGP keys we used to sign this release:
> https://dist.apache.org/repos/dist/release/bookkeeper/KEYS
>
> Please download these packages and review this release candidate:
>
> - Review release notes
> - Download the source package (verify shasum, and asc) and follow the
> instructions to build and run the bookkeeper service.
> - Download the binary package (verify shasum, and asc) and follow the
> instructions to run the bookkeeper service.
> - Review the maven repo, release tag, licenses, and any other things
> you think it is important to release.
>
> The vote will be open for at least 72 hours. It is adopted by the
> majority approval, with at least 3 PMC affirmative votes.
>
> Thanks,
> Release Manager
>
> [1] https://github.com/apache/bookkeeper/pull/3882/files
> [2] https://dist.apache.org/repos/dist/dev/bookkeeper/bookkeeper-4.16.0-rc0/
> [3] 
> https://repository.apache.org/content/repositories/orgapachebookkeeper-1081/
> [4] https://github.com/apache/bookkeeper/tree/v4.16.0-rc0
> [5] dccae86a9614a651d937bd7983accae341b71723

[VOTE] Release 4.16.0, release candidate #0

[Hang Chen]

Hi everyone,
Please review and vote on the release candidate #0 for version 4.16.0,
as follows:
[ ] +1, Approve the release
[ ] -1, Do not approve the release (please provide specific comments)

The complete staging area is available for your review, which includes:
* Release notes [1]
* The official Apache source and binary distributions to be deployed
to dist.apache.org [2]
* All artifacts to be deployed to the Maven Central Repository [3]
* Source code tag "v4.16.0-rc0" [4] with git sha [5]

BookKeeper's KEYS file contains the PGP keys we used to sign this release:
https://dist.apache.org/repos/dist/release/bookkeeper/KEYS

Please download these packages and review this release candidate:

- Review release notes
- Download the source package (verify shasum, and asc) and follow the
instructions to build and run the bookkeeper service.
- Download the binary package (verify shasum, and asc) and follow the
instructions to run the bookkeeper service.
- Review the maven repo, release tag, licenses, and any other things
you think it is important to release.

The vote will be open for at least 72 hours. It is adopted by the
majority approval, with at least 3 PMC affirmative votes.

Thanks,
Release Manager

[1] https://github.com/apache/bookkeeper/pull/3882/files
[2] https://dist.apache.org/repos/dist/dev/bookkeeper/bookkeeper-4.16.0-rc0/
[3] https://repository.apache.org/content/repositories/orgapachebookkeeper-1081/
[4] https://github.com/apache/bookkeeper/tree/v4.16.0-rc0
[5] dccae86a9614a651d937bd7983accae341b71723

Re: [DISCUSS] Bookkeeper 4.16.0 release

[Hang Chen]

Hi all,
Due to the current branch-4.16 was cut few months ago, and it is
too old. I will delete the current branch-4.16 and use the master to
create a new branch-4.16.

This is the list of PRs will be released in 4.16.0:
https://github.com/apache/bookkeeper/pulls?q=is%3Aopen+is%3Apr+milestone%3A4.16.0

If no more PRs need to be included in 4.16.0, I will start the release.

Thanks,
Hang

Hang Chen  于2023年1月11日周三 09:59写道：
>
> I will start to release 4.16.0 this week. Do you have any other
> issues/tickets to add
> for 4.16.0?
>
> Thanks,
> Hang
>
> steven lu  于2022年7月28日周四 14:20写道：
> >
> > ok, I see , thanks
> >
> > Hang Chen  于2022年7月28日周四 10:29写道：
> >
> > > Hi Steven,
> > > Those Prs are in the 4.16.0 milestone, and they are already
> > > merged. They will be included in the 4.16.0 release.  Thanks for your
> > > great work!
> > >
> > > Thanks,
> > > Hang
> > >
> > > steven lu  于2022年7月28日周四 09:56写道：
> > > >
> > > > I think this index's optimization can release for 4.16.0
> > > >  for details, have a look
> > > https://github.com/apache/bookkeeper/issues/3419
> > > >
> > > > Hang Chen  于2022年7月20日周三 10:24写道：
> > > >
> > > > > Andrey, thanks for your noticing.
> > > > >
> > > > > I'm happy to be a volunteer for the version 4.16.0 release.
> > > > > I have created an issue to track the Prs to be included in 4.16.0
> > > > > https://github.com/apache/bookkeeper/issues/3412
> > > > >
> > > > > Feel free to add more PRs if you want them to be included in 4.16.0.
> > > > > If you have time, welcome to help review them.
> > > > >
> > > > > Thanks,
> > > > > Hang
> > > > >
> > > > > Nicolò Boschi  于2022年7月20日周三 01:18写道：
> > > > > >
> > > > > > +1 for 4.16.0
> > > > > >
> > > > > >
> > > > > > > Btw, why https://bookkeeper.apache.org/docs/4.14.5/overview/ says
> > > > > "Apache
> > > > > > > BookKeeper 4.14.5, which is no longer actively maintained"?
> > > > > > > I thought 4.14 was still maintained.
> > > > > >
> > > > > >
> > > > > > Andrey, thanks for noticing. I created a pull for fixing it:
> > > > > > https://github.com/apache/bookkeeper/pull/3411
> > > > > >
> > > > > >
> > > > > > Nicolò Boschi
> > > > > >
> > > > > >
> > > > > > Il giorno mar 19 lug 2022 alle ore 18:43 Andrey Yegorov <
> > > > > > andrey.yego...@datastax.com> ha scritto:
> > > > > >
> > > > > > > +1 for the 4.16 release.
> > > > > > > Hang, did you just volunteer to do the release? :)
> > > > > > >
> > > > > > > We need to do a pass on open PRs to review/merge what we can, 
> > > > > > > there
> > > > > are 100
> > > > > > > open PRs right now.
> > > > > > > I'd love to see at least these in the release:
> > > > > > > https://github.com/apache/bookkeeper/pull/3361
> > > > > > > https://github.com/apache/bookkeeper/pull/3383
> > > > > > > https://github.com/apache/bookkeeper/pull/3384
> > > > > > > https://github.com/apache/bookkeeper/pull/3390
> > > > > > > https://github.com/apache/bookkeeper/pull/3325
> > > > > > > https://github.com/apache/bookkeeper/pull/3329
> > > > > > > https://github.com/apache/bookkeeper/pull/3330
> > > > > > >
> > > > > > > As for the minor releases, 4.15.1 is ok.
> > > > > > > I think we also need 4.14.6; 4.15 was a major change and requires 
> > > > > > > a
> > > > > bit of
> > > > > > > work to upgrade to (https://github.com/apache/pulsar/pull/15142),
> > > e.g.
> > > > > > > older Pulsar versions (2.10 and below) could benefit from 4.14
> > > release.
> > > > > > >
> > > > > > > Btw, why https://bookkeeper.apache.org/docs/4.14.5/overview/ says
> > > > > "Apache
> > > > > > > BookKeeper 4.14.5, which is no longer actively maintained"?
&

[VOTE] BP-62: Streamline batch add requests

[Hang Chen]

This is the vote for BP-62.

### Motivation
The bookie server process add-entry requests pipeline:
- Get one request from the Netty socket channel
- Choose one thread to process the written request
- Write the entry into the target ledger entry logger's write
cache(memory table)
- Put the entry into the journal's pending queue
- Journal thread takes the entry from the pending queue and writes it
into PageCache/Journal Disk
- Write the callback response to the Netty buffer and flush to the
bookie client side.

For every add entry request, the bookie server needs to go through the
above steps one by one. It will introduce a lot of thread context
switches.

We can batch the add requests according to the Netty socket channel,
and write a batch of entries into the ledger entry logger and journal
disk.

### Modifications
The PR will change the add requests processing pipeline into the
following steps.
- Get a batch of add-entry requests from the socket channel until the
socket channel is empty or reached the max capacity (default is 1_000)
- Choose one thread to process the batch of add-entry requests.
- Write the entries into the target ledger entry logger's write cache one by one
- Put the batch of entries into the journal's pending queue
- Journal thread drains a batch of entries from the pending queue and
writes them into PageCache/Journal disk
- Write the callback response to the Netty buffer and flush to the
bookie client side.

With this change, we can save a lot of thread context switches.


### Performance
I start one bookie on my laptop and use the Bookkeeper benchmark to
test the performance
```shell
bin/benchmark writes -ensemble 1 -quorum 1 -ackQuorum 1 -ledgers 50
-throttle 2
```

**Before this change**

| times | ops/sec | p95 latency | p99 latency |
| --- | --- | --- | --- |
| 1 | 147507 | 114.93 | 122.42 |
| 2 | 154571 |  111.46 | 115.86 |
| 3 | 141459 | 117.23 | 124.18 |
| 4 | 142037 | 121.75 | 128.54 |
| 5 | 143682 | 121.05 | 127.97 |


**After this change**

| times | ops/sec | p95 latency | p99 latency |
| --- | --- | --- | --- |
| 1 | 157328 | 118.30 | 121.79 |
| 2 | 165774 |  112.86 | 115.69 |
| 3 | 144790 | 128.94 | 133.24 |
| 4 | 151984 | 121.88 | 125.32 |
| 5 | 154574 | 121.57 | 124.57 |

The new change has a 2.2% improvement.

Please leave +1/-1 in this thread to join the vote. And feel free to
leave any concerns.

Thanks,
Hang

Re: [DISCUSS] Streamline batch add requests

[Hang Chen]

If there are no objections, I will send out the vote.

Thanks,
Hang

Hang Chen  于2023年3月16日周四 10:15写道：
>
> ### Motivation
> The bookie server process add-entry requests pipeline:
> - Get one request from the Netty socket channel
> - Choose one thread to process the written request
> - Write the entry into the target ledger entry logger's write
> cache(memory table)
> - Put the entry into the journal's pending queue
> - Journal thread takes the entry from the pending queue and writes it
> into PageCache/Journal Disk
> - Write the callback response to the Netty buffer and flush to the
> bookie client side.
>
> For every add entry request, the bookie server needs to go through the
> above steps one by one. It will introduce a lot of thread context
> switches.
>
> We can batch the add requests according to the Netty socket channel,
> and write a batch of entries into the ledger entry logger and journal
> disk.
>
> ### Modifications
> The PR will change the add requests processing pipeline into the
> following steps.
> - Get a batch of add-entry requests from the socket channel until the
> socket channel is empty or reached the max capacity (default is 1_000)
> - Choose one thread to process the batch of add-entry requests.
> - Write the entries into the target ledger entry logger's write cache one by 
> one
> - Put the batch of entries into the journal's pending queue
> - Journal thread drains a batch of entries from the pending queue and
> writes them into PageCache/Journal disk
> - Write the callback response to the Netty buffer and flush to the
> bookie client side.
>
> With this change, we can save a lot of thread context switches.
>
>
> ### Performance
> I start one bookie on my laptop and use the Bookkeeper benchmark to
> test the performance
> ```shell
> bin/benchmark writes -ensemble 1 -quorum 1 -ackQuorum 1 -ledgers 50
> -throttle 2
> ```
>
> **Before this change**
>
> | times | ops/sec | p95 latency | p99 latency |
> | --- | --- | --- | --- |
> | 1 | 147507 | 114.93 | 122.42 |
> | 2 | 154571 |  111.46 | 115.86 |
> | 3 | 141459 | 117.23 | 124.18 |
> | 4 | 142037 | 121.75 | 128.54 |
> | 5 | 143682 | 121.05 | 127.97 |
>
>
> **After this change**
>
> | times | ops/sec | p95 latency | p99 latency |
> | --- | --- | --- | --- |
> | 1 | 157328 | 118.30 | 121.79 |
> | 2 | 165774 |  112.86 | 115.69 |
> | 3 | 144790 | 128.94 | 133.24 |
> | 4 | 151984 | 121.88 | 125.32 |
> | 5 | 154574 | 121.57 | 124.57 |
>
> The new change has a 2.2% improvement.
>
> Do you guys have any ideas?
>
> Thanks,
> Hang

[DISCUSS] Streamline batch add requests

[Hang Chen]

### Motivation
The bookie server process add-entry requests pipeline:
- Get one request from the Netty socket channel
- Choose one thread to process the written request
- Write the entry into the target ledger entry logger's write
cache(memory table)
- Put the entry into the journal's pending queue
- Journal thread takes the entry from the pending queue and writes it
into PageCache/Journal Disk
- Write the callback response to the Netty buffer and flush to the
bookie client side.

For every add entry request, the bookie server needs to go through the
above steps one by one. It will introduce a lot of thread context
switches.

We can batch the add requests according to the Netty socket channel,
and write a batch of entries into the ledger entry logger and journal
disk.

### Modifications
The PR will change the add requests processing pipeline into the
following steps.
- Get a batch of add-entry requests from the socket channel until the
socket channel is empty or reached the max capacity (default is 1_000)
- Choose one thread to process the batch of add-entry requests.
- Write the entries into the target ledger entry logger's write cache one by one
- Put the batch of entries into the journal's pending queue
- Journal thread drains a batch of entries from the pending queue and
writes them into PageCache/Journal disk
- Write the callback response to the Netty buffer and flush to the
bookie client side.

With this change, we can save a lot of thread context switches.


### Performance
I start one bookie on my laptop and use the Bookkeeper benchmark to
test the performance
```shell
bin/benchmark writes -ensemble 1 -quorum 1 -ackQuorum 1 -ledgers 50
-throttle 2
```

**Before this change**

| times | ops/sec | p95 latency | p99 latency |
| --- | --- | --- | --- |
| 1 | 147507 | 114.93 | 122.42 |
| 2 | 154571 |  111.46 | 115.86 |
| 3 | 141459 | 117.23 | 124.18 |
| 4 | 142037 | 121.75 | 128.54 |
| 5 | 143682 | 121.05 | 127.97 |


**After this change**

| times | ops/sec | p95 latency | p99 latency |
| --- | --- | --- | --- |
| 1 | 157328 | 118.30 | 121.79 |
| 2 | 165774 |  112.86 | 115.69 |
| 3 | 144790 | 128.94 | 133.24 |
| 4 | 151984 | 121.88 | 125.32 |
| 5 | 154574 | 121.57 | 124.57 |

The new change has a 2.2% improvement.

Do you guys have any ideas?

Thanks,
Hang

Re: [VOTE] Release 4.15.4, release candidate #0

[Hang Chen]

+1 (non-binding)

Thank Yong for driving this release. Overall it looks good.

- Verified signatures and checksums
- Build from source code (MacOS jdk 11, maven 3.8.6)
- Tested local bookie
- Tested standalone bookie
- Build Pulsar master code with this staging dependency and start
Pulsar standalone cluster and run Pulsar perf produce and consume
- Run Pulsar CI with the new BookKeeper release [1]

Thanks,
Hang

[1] https://github.com/hangc0276/pulsar/pull/11

Yong Zhang  于2023年3月2日周四 15:30写道：
>
> Hi everyone,
> Please review and vote on the release candidate #1 for version 4.15.4,
> as follows:
> [ ] +1, Approve the release
> [ ] -1, Do not approve the release (please provide specific comments)
>
> The complete staging area is available for your review, which includes:
> * Release notes [1]
> * The official Apache source and binary distributions to be deployed to
> dist.apache.org [2]
> * All artifacts to be deployed to the Maven Central Repository [3]
> * Source code tag "v4.15.4-rc0" [4] with git sha [5]
>
> BookKeeper's KEYS file contains the PGP keys we used to sign this release:
> https://dist.apache.org/repos/dist/release/bookkeeper/KEYS
>
> Please download these packages and review this release candidate:
>
> - Review release notes
> - Download the source package (verify shasum, and asc) and follow the
> instructions to build and run the bookkeeper service.
> - Download the binary package (verify shasum, and asc) and follow the
> instructions to run the bookkeeper service.
> - Review the maven repo, release tag, licenses, and any other things you
> think
> it is important to release.
>
> The vote will be open for at least 72 hours. It is adopted by majority
> approval, with at least 3 PMC affirmative votes.
>
> Thanks,
> Release Manager
>
> [1] https://github.com/apache/bookkeeper/pull/3831/files
> [2] https://dist.apache.org/repos/dist/dev/bookkeeper/bookkeeper-4.15.4-rc0/
> [3]
> https://repository.apache.org/content/repositories/orgapachebookkeeper-1080
> [4] https://github.com/apache/bookkeeper/releases/tag/v4.15.4-rc0
> [5] 44f87efb8121a9f5428e7ddc70656aee85f563bf

Re: [DISCUSS] BookKeeper 4.15.4 release

[Hang Chen]

Hi Yong,
 No more PRs from my side. Thanks for your great job!

Thanks,
Hang

Yong Zhang  于2023年2月22日周三 09:12写道：
>
> Hi all,
>
> All PRs have been cherry-picked to the branch-4.15.
> If no more PRs needed to join this release, I will start the release.
>
> Here is the list:
> https://github.com/apache/bookkeeper/pulls?q=is%3Apr+label%3Arelease%2F4.15.4+is%3Aclosed
>
> Thanks,
> Yong
>
> On Thu, 16 Feb 2023 at 18:17, Enrico Olivelli  wrote:
>
> > Il giorno gio 16 feb 2023 alle ore 10:13 Yong Zhang
> >  ha scritto:
> > >
> > > There have two PRs #3673 <https://github.com/apache/bookkeeper/pull/3673
> > >
> > > and #3674 <https://github.com/apache/bookkeeper/pull/3674> about
> > > the BP-59:Refactor ByteBuf
> > > release method has been cherry-picked into the branch-4.15.
> > > https://github.com/apache/bookkeeper/commits/branch-4.15
> > > Because that PIP is still in discussion, I will revert them from
> > branch-4.15
> >
> > Good idea !
> >
> > Enrico
> >
> > > first.
> > >
> > > Yong
> > >
> > > On Tue, 14 Feb 2023 at 15:21, Enrico Olivelli 
> > wrote:
> > >
> > > > Good idea
> > > >
> > > > +1
> > > >
> > > > Enrico
> > > >
> > > > Il Mar 14 Feb 2023, 02:23 Hang Chen  ha scritto:
> > > >
> > > > > +1
> > > > >
> > > > > -  As we discussed in [2], For the PRs list in [1], I think we can
> > > > > skip cherry-pick first.
> > > > > - For PR [3], I removed label `release/4.15.4` due to [4]
> > > > >
> > > > > Thanks,
> > > > > Hang
> > > > >
> > > > > [1] https://github.com/apache/bookkeeper/issues/3686
> > > > > [2] https://lists.apache.org/thread/9h7v7hs1pfp74mmopqtl7w4z0tq5rqhc
> > > > > [3] https://github.com/apache/bookkeeper/pull/3653
> > > > > [4] https://github.com/apache/bookkeeper/issues/3734
> > > > >
> > > > > Yong Zhang  于2023年2月14日周二 08:40写道：
> > > > > >
> > > > > > Hi everyone,
> > > > > >
> > > > > > It has been a long time since we did the last release
> > > > > > from branch-4.15. We made a lot of improvements and
> > > > > > bug fixes. I'd propose doing a minor release for the
> > > > > > branch-4.15.
> > > > > >
> > > > > > Here is the PRs list for 4.15.4.
> > > > > > <
> > > > >
> > > >
> > https://github.com/apache/bookkeeper/pulls?q=+is%3Apr+label%3Arelease%2F4.15.4+
> > > > > >
> > > > > >
> > > > > >
> > > > > > If you have PR want to include in this release, please
> > > > > > comment on that PR or tag the label `release/4.15.4`.
> > > > > >
> > > > > > Thanks,
> > > > > > Yong
> > > > >
> > > >
> >

[VOTE] BP-61: Revert BP-59 to release ByteBuf using ReferenceCountUtil.release() instead of ReferenceCountUtil.safeRelease()

[Hang Chen]

Hi, All

After a fascinating discussion, I would start the vote for BP-61.

We will replace the `ReferenceCountUtil.safeRelease()` with
`ReferenceCountUtil.release()`.

Please leave +1/-1 in this thread to join the vote. And feel free to
leave any concerns.

Thanks to you guys.

Best,
Hang

References:
- BP: https://github.com/apache/bookkeeper/issues/3792
- Discussion https://lists.apache.org/thread/9h7v7hs1pfp74mmopqtl7w4z0tq5rqhc

Re: [DISCUSS] BP-61: Revert BP-59 to release ByteBuf using ByteBuf.release() instead of ReferenceCountUtil.safeRelease()

[Hang Chen]

Hi Andrey,
Thanks for your reply. The `ReferenceCountUtil.release` will bring
convenient in the following case
```java
ByteBuf buf;
try {
  buf = allocate();
} finally {
  ReferenceCountUtil.release(buf);
}
```
If there is no objections, I will start a vote to replace
`ReferenceCountUtil. safeRelease()` with
`ReferenceCountUtil.release()`.

Thanks,
Hang

Andrey Yegorov  于2023年2月3日周五 08:24写道：
>
> Netty recommends to NOT use .safeRelease() unless one absolutely needs to
> swallow an exception:
> https://github.com/netty/netty/blob/48adb2609c7210ff55bb2ae42772a579052d06e7/common/src/main/java/io/netty/util/ReferenceCountUtil.java#L107-L112
>
> .release() is a convenient way to bypass null check if the object may
> happen to be null e.g. in the finally block when it is possible that
> allocation didn't happen.
> though BK's code still does the null check and then uses .release().
>
> I am not really sure why safeRelease() spread so much, maybe it was a quick
> way to deal with errors in unit tests caused by improper setup/tearDown.
>
> On an unrelated note, while looking through all that code I noticed that
> memetable uses NIO ByteBuffers.
> so naturally for the netty's ByteBuf entry has to be copied into nio buffer:
> "memTable.addEntry(ledgerId, entryId, entry.nioBuffer(), this)"
> And after getEntry() it is wrapped into netty's ByteBuf.
> I wonder if switching it to ByteBuf would improve performance / reduce java
> GC load.
>
>
> On Wed, Feb 1, 2023 at 1:28 AM Hang Chen  wrote:
>
> > > +1 , use new specification (ReferenceCountUtil.release) replace BP-59
> >
> > > I think it makes sense to replace ReferenceCountUtil.safeRelease
> > with ReferenceCountUtil.release.
> > FWIW ReferenceCountUtil.release's return value just says if the byte buf's
> > ref count reached zero or not, it is not an error handling mechanism.
> >
> > The `ReferenceCountUtil.release` checks whether the ByteBuf is
> > `ReferenceCounted`. If not, it will do nothing and return false.
> > ```java
> > public static boolean release(Object msg) {
> > return msg instanceof ReferenceCounted ?
> > ((ReferenceCounted)msg).release() : false;
> > }
> > ```
> > All the use cases in BookKeeper about `ReferenceCountUtil.release` are
> > not check the returned value.
> > Even Though all the ByteBuf in BookKeeper code are ReferenceCounted,
> > it still has risk of causing direct memory leak. One example is [1]
> >
> > My question is why not use `ByteBuf.release()` directly?
> >
> >
> > [1] https://github.com/apache/bookkeeper/issues/3527
> >
> > Thanks,
> > Hang
> >
> > Enrico Olivelli  于2023年2月1日周三 15:35写道：
> > >
> > > Horizon
> > >
> > > Il giorno lun 30 gen 2023 alle ore 11:14 Horizon
> > > <1060026...@qq.com.invalid> ha scritto:
> > > >
> > > > Hi, Enrico. Just confirm, only revert
> > https://github.com/apache/bookkeeper/pull/3528, it work well?
> > >
> > > yes
> > > This is the only thing we reverted in our fork
> > >
> > > see the history
> > > https://github.com/datastax/bookkeeper/commits/ds-4.14
> > >
> > > but I don't have all of the other commits in the history,
> > > so maybe other changes may be bad
> > >
> > > Unfortunately currently we (both BookKeeper community and in my
> > > company) don't have
> > > dedicated tests for BookKeeper but only for Pulsar,
> > > that makes things harder to track
> > >
> > > probably we should build some automated performance test:
> > > a set of scripts that create a BK cluster  on k8s, run some
> > > benchmarks/heavy load, verify that everything worked well
> > > The workbench should be re-usable by anyone, I am not saying that we
> > > should spend the ASF resources to run
> > > this kind of tests, but that we should provide a reference load test.
> > >
> > >
> > > Enrico
> > >
> > > >
> > > > > 2023年1月30日 16:04，Enrico Olivelli  写道：
> > > > >
> > > > > I think that we must do something, current master branch is not
> > stable.
> > > > >
> > > > > My colleague Massimiliano opened this issue
> > > > > https://github.com/apache/bookkeeper/issues/3751
> > > > >
> > > > > Basically in the current master there is some problem that leads to
> > > > > Netty BytBuf corruption.
> > > > >
> > > > > The problem is solved by reve

Re: The CI tests didn't cover bookkeeper V2 protocol.

[Hang Chen]

+1 for Yong's suggestion.

Thanks,
Hang

Yan Zhao  于2023年2月20日周一 16:55写道：
>
> > According to the original PR's motivation
> > , we wrapped a Netty
> > allocator
> > and want to configure something through bookkeeper.
> > So the user will use our customized allocator and need to obey the rules
> > we introduced. Then the Netty's configuration `io.netty.leakDetection.level`
> > seems useless for us. I think they would never have a chance to take it
> > differently.
> >
> > We defined a new configuration property to configure the
> > LeakDetectionPolicy.
> >
> >
> >
> >
> > *public LeakDetectionPolicy getAllocatorLeakDetectionPolicy() {
> > return LeakDetectionPolicy
> > .valueOf(this.getString(ALLOCATOR_LEAK_DETECTION_POLICY,
> > LeakDetectionPolicy.Paranoid.toString()));}*
> >
> > Some users may don't know the `*ALLOCATOR_LEAK_DETECTION_POLICY`*
> > and only configure* `*io.netty.leakDetection.level` to open the leak
> > detection.
> > I would suggest getting both `*ALLOCATOR_LEAK_DETECTION_POLICY`*
> > and `io.netty.leakDetection.level` from the system property, use the highest
> > policy of it. And then, no matter how you configured it, the detection
> > policy always worked.
>
> Agree.

Re: CI shows successfully but the tests actually has errors

[Hang Chen]

 Hi Yong,
 Thanks for finding this critical issue. I think it's dangerous to
merge new PRs before the CI is fixed, because we can't ensure whether
the new PRs introduces bugs.

Thanks,
Hang

Yong Zhang  于2023年2月21日周二 09:15写道：
>
> Hi folks,
>
> I found our CI may have some problems.
>
> In the master code, the tests failed, but the CI was successful.
> The following is the latest commit CI action:
> https://github.com/apache/bookkeeper/actions/runs/4219246330/jobs/7324432945
> Error:  Tests run: 633, Failures: 0, Errors: 29, Skipped: 0, Flakes: 1
>
> You can see there have errors but the CI passed.
>
> Also, the PR's check has the same issue.
> https://github.com/apache/bookkeeper/actions/runs/4217320319/jobs/7321051845
>
> We may need to fail the CI if there have errors.
>
> Should we stop merging PRs until we fix the CI? But looks like most
> failures are
> about bookie shell commands. I am not sure whether we need to stop merging.
>
> Best,
> Yong

[ANNOUNCE] Apache BookKeeper 4.14.7 released

[Hang Chen]

The Apache BookKeeper team is proud to announce Apache BookKeeper
version 4.14.7.

Apache BookKeeper is a scalable, fault-tolerant, and low-latency
storage service optimized for real-time workloads. It has been used
for a fundamental service to build reliable services. It is also the
log segment store for Apache DistributedLog and the message store for
Apache Pulsar.

This is the 44rd release of the Apache BookKeeper.

Release 4.14.7 contains performance and stability improvements and
various bug fixes.

Apache BookKeeper users are encouraged to upgrade to 4.14.7. The
technical details of this release are summarized below.

For BookKeeper release details and downloads, visit:
https://bookkeeper.apache.org/releases

BookKeeper 4.14.7 Release Notes are at:
https://bookkeeper.apache.org/release-notes/#4147

We would like to thank the contributors that made the release possible.

Regards,

The BookKeeper Team

Re: [VOTE] Release 4.14.7, release candidate #1

[Hang Chen]

The vote is now closed, and the release was approved with
5 +1s(3 bindings and 2 non-binding), and I will continue with the
following steps.

Binding votes:
* Enrico Olivelli
* Andrey Yegorov
* Jia Zhai

Non-Binding votes:
* Massimiliano Mirelli
* Yong Zhang

Thank you all for verifying and voting!

Thanks,
Hang

Jia Zhai  于2023年2月20日周一 08:42写道：
>
> +1 (binding)
>
> - verified packages checksum and signatures. They look good.
>
> - the source package build and test all run successfully.
>
> - 'bin/bookkeeper standalone' run well.
>
> On Sat, Feb 18, 2023 at 4:52 AM Massimiliano Mirelli <
> massimilianomirelli...@gmail.com> wrote:
>
> > +1 (non-binding)
> >
> > * verified that the regression (
> > https://github.com/apache/bookkeeper/issues/3751) has been resolved.
> >
> > Thank you for driving the release!
> >
> > On Fri, 17 Feb 2023 at 03:52, Andrey Yegorov 
> > wrote:
> >
> > > +1 (binding)
> > >
> > > * tested localbookie
> > > * tested standalone bookie
> > > * built from sources (Mac/Intel)
> > >  * ran some tests
> > >
> > > Thank you for driving the release!
> > >
> > > On Wed, Feb 15, 2023 at 12:08 AM Yong Zhang 
> > > wrote:
> > >
> > > > +1 (non-binding)
> > > >
> > > > - verified signatures and checksums
> > > > - run simple test with localbookie
> > > > - built from sources
> > > >
> > > > Yong
> > > >
> > > > On Tue, 14 Feb 2023 at 23:32, Enrico Olivelli 
> > > wrote:
> > > >
> > > > > +1 (binding)
> > > > > - run tests locally
> > > > > - built from sources
> > > > > - verified signatures and checksums
> > > > > - run Apache RAT for headers
> > > > > - verified LICENSE using the script that we have in dev/
> > > > >
> > > > > Thanks for driving the release
> > > > >
> > > > > Enrico
> > > > >
> > > > > Il giorno mar 14 feb 2023 alle ore 05:39 Hang Chen
> > > > >  ha scritto:
> > > > > >
> > > > > > I opened one PR[1] to upgrade BookKeeper dependency to 4.14.7 for
> > > > > > Pulsar branch-2.10 and all tests passed. Please help verify this
> > > > > > candidate. Thanks a lot.
> > > > > >
> > > > > > Thanks,
> > > > > > Hang
> > > > > >
> > > > > > [1] https://github.com/hangc0276/pulsar/pull/9
> > > > > >
> > > > > > Hang Chen  于2023年2月13日周一 21:17写道：
> > > > > > >
> > > > > > > Hi everyone,
> > > > > > > Please review and vote on the release candidate #1 for version
> > > > 4.14.7,
> > > > > > > as follows:
> > > > > > > [ ] +1, Approve the release
> > > > > > > [ ] -1, Do not approve the release (please provide specific
> > > comments)
> > > > > > >
> > > > > > > The complete staging area is available for your review, which
> > > > includes:
> > > > > > > * Release notes [1]
> > > > > > > * The official Apache source and binary distributions to be
> > > deployed
> > > > to
> > > > > > > dist.apache.org [2]
> > > > > > > * All artifacts to be deployed to the Maven Central Repository
> > [3]
> > > > > > > * Source code tag "v4.14.7-rc1" [4] with git sha [5]
> > > > > > >
> > > > > > > BookKeeper's KEYS file contains the PGP keys we used to sign this
> > > > > release:
> > > > > > > https://dist.apache.org/repos/dist/release/bookkeeper/KEYS
> > > > > > >
> > > > > > > Please download these packages and review this release candidate:
> > > > > > >
> > > > > > > - Review release notes
> > > > > > > - Download the source package (verify shasum, and asc) and follow
> > > the
> > > > > > > instructions to build and run the bookkeeper service.
> > > > > > > - Download the binary package (verify shasum, and asc) and follow
> > > the
> > > > > > > instructions to run the bookkeeper service.
> > > > > > > - Review the maven repo, release tag, licenses, and any other
> > > things
> > > > > you think
> > > > > > > it is important to release.
> > > > > > >
> > > > > > > The vote will be open for at least 72 hours. It is adopted by the
> > > > > majority
> > > > > > > approval, with at least 3 PMC affirmative votes.
> > > > > > >
> > > > > > > Thanks,
> > > > > > > Release Manager
> > > > > > >
> > > > > > > [1] https://github.com/apache/bookkeeper/pull/3770/files
> > > > > > > [2]
> > > > >
> > > https://dist.apache.org/repos/dist/dev/bookkeeper/bookkeeper-4.14.7-rc1/
> > > > > > > [3]
> > > > >
> > > >
> > >
> > https://repository.apache.org/content/repositories/orgapachebookkeeper-1079
> > > > > > > [4] https://github.com/apache/bookkeeper/tree/v4.14.7-rc1
> > > > > > > [5] c22136b03489db1643521f586e9cae2c4a511e10
> > > > >
> > > >
> > >
> > >
> > > --
> > > Andrey Yegorov
> > >
> >

Re: The CI tests didn't cover bookkeeper V2 protocol.

[Hang Chen]

Hi horizonzy,
Thanks for raising this discussion.
> the netty jvm param `-Dio.netty.leakDetection.level` didn't
work, it will be overridden by our custom config.
> In our tests, we config
`-Dio.netty.leakDetection.level=paranoid`, but it will be overridden
by `disabled`.

IMO, this is not expected behavior. In our test, both the client and
server's memory leak detection policy should be gotten from
`-Dio.netty.leakDetection.level` configuration instead of hard code as
`LeakDetectionPolicy.Paranoid`

Thanks,
Hang

horizonzy  于2023年2月17日周五 22:43写道：
>
> In https://github.com/apache/bookkeeper/issues/3751, it report a bug: The
> bookkeeper client read operation timeout.
> After our research, we found that the bookkeeper client ignore the response
> from bookie server. (See https://github.com/apache/bookkeeper/issues/3751
> to get the detailed).
>
> The problem is introduced by https://github.com/apache/bookkeeper/pull/3528.
> It change the ReadResponse, which is used by V2 Protocol. When config the
> netty memory detect policy upper than
> ResourceLeakDetector.setLevel(Level.DISABLED), the bookkeeper client will
> ignore the response from bookie server, so the read operation timeout.
>
> The https://github.com/apache/bookkeeper/pull/3528 all CI tests succeed and
> merged. The Test `Client Tests` shouldn't pass in theory, but it pass.
> So we research the root case.
> Finnaly, we found probelm which lead the ci pass.
> In bookkeeper, we use custom
> org.apache.bookkeeper.common.allocator.impl.ByteBufAllocatorImpl to
> allocate ByteBuf, the memory detect policy config by
> `org.apache.bookkeeper.conf.AbstractConfiguration#setAllocatorLeakDetectionPolicy`,
> the netty jvm param `-Dio.netty.leakDetection.level` didn't work, it will
> be overridden by our custom config.
> In our tests, we config `-Dio.netty.leakDetection.level=paranoid`, but it
> will be overridden by `disabled`.
>
> So I suggest in our tests, both the clientConf and the serverConfig should
> config
> `AbstractConfiguration#setAllocatorLeakDetectionPolicy(LeakDetectionPolicy.Paranoid)`.
> It can cover more cases.

Re: [VOTE] Release 4.14.7, release candidate #1

[Hang Chen]

I opened one PR[1] to upgrade BookKeeper dependency to 4.14.7 for
Pulsar branch-2.10 and all tests passed. Please help verify this
candidate. Thanks a lot.

Thanks,
Hang

[1] https://github.com/hangc0276/pulsar/pull/9

Hang Chen  于2023年2月13日周一 21:17写道：
>
> Hi everyone,
> Please review and vote on the release candidate #1 for version 4.14.7,
> as follows:
> [ ] +1, Approve the release
> [ ] -1, Do not approve the release (please provide specific comments)
>
> The complete staging area is available for your review, which includes:
> * Release notes [1]
> * The official Apache source and binary distributions to be deployed to
> dist.apache.org [2]
> * All artifacts to be deployed to the Maven Central Repository [3]
> * Source code tag "v4.14.7-rc1" [4] with git sha [5]
>
> BookKeeper's KEYS file contains the PGP keys we used to sign this release:
> https://dist.apache.org/repos/dist/release/bookkeeper/KEYS
>
> Please download these packages and review this release candidate:
>
> - Review release notes
> - Download the source package (verify shasum, and asc) and follow the
> instructions to build and run the bookkeeper service.
> - Download the binary package (verify shasum, and asc) and follow the
> instructions to run the bookkeeper service.
> - Review the maven repo, release tag, licenses, and any other things you think
> it is important to release.
>
> The vote will be open for at least 72 hours. It is adopted by the majority
> approval, with at least 3 PMC affirmative votes.
>
> Thanks,
> Release Manager
>
> [1] https://github.com/apache/bookkeeper/pull/3770/files
> [2] https://dist.apache.org/repos/dist/dev/bookkeeper/bookkeeper-4.14.7-rc1/
> [3] 
> https://repository.apache.org/content/repositories/orgapachebookkeeper-1079
> [4] https://github.com/apache/bookkeeper/tree/v4.14.7-rc1
> [5] c22136b03489db1643521f586e9cae2c4a511e10

Re: [DISCUSS] BookKeeper 4.15.4 release

[Hang Chen]

+1

-  As we discussed in [2], For the PRs list in [1], I think we can
skip cherry-pick first.
- For PR [3], I removed label `release/4.15.4` due to [4]

Thanks,
Hang

[1] https://github.com/apache/bookkeeper/issues/3686
[2] https://lists.apache.org/thread/9h7v7hs1pfp74mmopqtl7w4z0tq5rqhc
[3] https://github.com/apache/bookkeeper/pull/3653
[4] https://github.com/apache/bookkeeper/issues/3734

Yong Zhang  于2023年2月14日周二 08:40写道：
>
> Hi everyone,
>
> It has been a long time since we did the last release
> from branch-4.15. We made a lot of improvements and
> bug fixes. I'd propose doing a minor release for the
> branch-4.15.
>
> Here is the PRs list for 4.15.4.
> 
>
>
> If you have PR want to include in this release, please
> comment on that PR or tag the label `release/4.15.4`.
>
> Thanks,
> Yong

[VOTE] Release 4.14.7, release candidate #1

[Hang Chen]

Hi everyone,
Please review and vote on the release candidate #1 for version 4.14.7,
as follows:
[ ] +1, Approve the release
[ ] -1, Do not approve the release (please provide specific comments)

The complete staging area is available for your review, which includes:
* Release notes [1]
* The official Apache source and binary distributions to be deployed to
dist.apache.org [2]
* All artifacts to be deployed to the Maven Central Repository [3]
* Source code tag "v4.14.7-rc1" [4] with git sha [5]

BookKeeper's KEYS file contains the PGP keys we used to sign this release:
https://dist.apache.org/repos/dist/release/bookkeeper/KEYS

Please download these packages and review this release candidate:

- Review release notes
- Download the source package (verify shasum, and asc) and follow the
instructions to build and run the bookkeeper service.
- Download the binary package (verify shasum, and asc) and follow the
instructions to run the bookkeeper service.
- Review the maven repo, release tag, licenses, and any other things you think
it is important to release.

The vote will be open for at least 72 hours. It is adopted by the majority
approval, with at least 3 PMC affirmative votes.

Thanks,
Release Manager

[1] https://github.com/apache/bookkeeper/pull/3770/files
[2] https://dist.apache.org/repos/dist/dev/bookkeeper/bookkeeper-4.14.7-rc1/
[3] https://repository.apache.org/content/repositories/orgapachebookkeeper-1079
[4] https://github.com/apache/bookkeeper/tree/v4.14.7-rc1
[5] c22136b03489db1643521f586e9cae2c4a511e10

Re: [VOTE] Release 4.14.7, release candidate #0

[Hang Chen]

Hi guys,
 Sorry, we need to close this vote. We found one issue [1] is
caused by [2], which is included in 4.14.7 RC0. The issue has been
fixed by [3], but [3] is not included in 4.14.7. I will cherry-pick
[3] into 4.14.7 and trigger a new RC.

Thanks,
Hang

[1] https://github.com/apache/bookkeeper/issues/3751
[2] https://github.com/apache/bookkeeper/pull/3528
[3] https://github.com/apache/bookkeeper/pull/3560

Wenbing Shen  于2023年2月9日周四 16:53写道：
>
> +1
>
> * built from sources (mac 11.3.1, intel), ran some tests
> * ran bookkeeper standalone
>
> Thanks.
> Wenbing
>
> Hang Chen  于2023年2月9日周四 13:56写道：
>
> > I opened one PR[1] to upgrade BookKeeper dependency to 4.14.7 for
> > Pulsar branch-2.10 and all tests passed. Please help verify this
> > candidate. Thanks a lot.
> >
> > Thanks,
> > Hang
> >
> > [1] https://github.com/hangc0276/pulsar/pull/9
> >
> > Hang Chen  于2023年2月9日周四 10:45写道：
> > >
> > > Hi everyone,
> > > Please review and vote on the release candidate #0 for version 4.14.7,
> > > as follows:
> > > [ ] +1, Approve the release
> > > [ ] -1, Do not approve the release (please provide specific comments)
> > >
> > > The complete staging area is available for your review, which includes:
> > > * Release notes [1]
> > > * The official Apache source and binary distributions to be deployed to
> > > dist.apache.org [2]
> > > * All artifacts to be deployed to the Maven Central Repository [3]
> > > * Source code tag "v4.14.7-rc0" [4] with git sha [5]
> > >
> > > BookKeeper's KEYS file contains the PGP keys we used to sign this
> > release:
> > > https://dist.apache.org/repos/dist/release/bookkeeper/KEYS
> > >
> > > Please download these packages and review this release candidate:
> > >
> > > - Review release notes
> > > - Download the source package (verify shasum, and asc) and follow the
> > > instructions to build and run the bookkeeper service.
> > > - Download the binary package (verify shasum, and asc) and follow the
> > > instructions to run the bookkeeper service.
> > > - Review the maven repo, release tag, licenses, and any other things you
> > think
> > > it is important to release.
> > >
> > > The vote will be open for at least 72 hours. It is adopted by the
> > majority
> > > approval, with at least 3 PMC affirmative votes.
> > >
> > > Thanks,
> > > Release Manager
> > >
> > > [1] https://github.com/apache/bookkeeper/pull/3770/files
> > > [2]
> > https://dist.apache.org/repos/dist/dev/bookkeeper/bookkeeper-4.14.7-rc0/
> > > [3]
> > https://repository.apache.org/content/repositories/orgapachebookkeeper-1078
> > > [4] https://github.com/apache/bookkeeper/tree/v4.14.7-rc0
> > > [5] 3c85422a9497d168cadc5d264ebccbe0fadf3660
> >

Re: [VOTE] Release 4.14.7, release candidate #0

[Hang Chen]

I opened one PR[1] to upgrade BookKeeper dependency to 4.14.7 for
Pulsar branch-2.10 and all tests passed. Please help verify this
candidate. Thanks a lot.

Thanks,
Hang

[1] https://github.com/hangc0276/pulsar/pull/9

Hang Chen  于2023年2月9日周四 10:45写道：
>
> Hi everyone,
> Please review and vote on the release candidate #0 for version 4.14.7,
> as follows:
> [ ] +1, Approve the release
> [ ] -1, Do not approve the release (please provide specific comments)
>
> The complete staging area is available for your review, which includes:
> * Release notes [1]
> * The official Apache source and binary distributions to be deployed to
> dist.apache.org [2]
> * All artifacts to be deployed to the Maven Central Repository [3]
> * Source code tag "v4.14.7-rc0" [4] with git sha [5]
>
> BookKeeper's KEYS file contains the PGP keys we used to sign this release:
> https://dist.apache.org/repos/dist/release/bookkeeper/KEYS
>
> Please download these packages and review this release candidate:
>
> - Review release notes
> - Download the source package (verify shasum, and asc) and follow the
> instructions to build and run the bookkeeper service.
> - Download the binary package (verify shasum, and asc) and follow the
> instructions to run the bookkeeper service.
> - Review the maven repo, release tag, licenses, and any other things you think
> it is important to release.
>
> The vote will be open for at least 72 hours. It is adopted by the majority
> approval, with at least 3 PMC affirmative votes.
>
> Thanks,
> Release Manager
>
> [1] https://github.com/apache/bookkeeper/pull/3770/files
> [2] https://dist.apache.org/repos/dist/dev/bookkeeper/bookkeeper-4.14.7-rc0/
> [3] 
> https://repository.apache.org/content/repositories/orgapachebookkeeper-1078
> [4] https://github.com/apache/bookkeeper/tree/v4.14.7-rc0
> [5] 3c85422a9497d168cadc5d264ebccbe0fadf3660

[VOTE] Release 4.14.7, release candidate #0

[Hang Chen]

Hi everyone,
Please review and vote on the release candidate #0 for version 4.14.7,
as follows:
[ ] +1, Approve the release
[ ] -1, Do not approve the release (please provide specific comments)

The complete staging area is available for your review, which includes:
* Release notes [1]
* The official Apache source and binary distributions to be deployed to
dist.apache.org [2]
* All artifacts to be deployed to the Maven Central Repository [3]
* Source code tag "v4.14.7-rc0" [4] with git sha [5]

BookKeeper's KEYS file contains the PGP keys we used to sign this release:
https://dist.apache.org/repos/dist/release/bookkeeper/KEYS

Please download these packages and review this release candidate:

- Review release notes
- Download the source package (verify shasum, and asc) and follow the
instructions to build and run the bookkeeper service.
- Download the binary package (verify shasum, and asc) and follow the
instructions to run the bookkeeper service.
- Review the maven repo, release tag, licenses, and any other things you think
it is important to release.

The vote will be open for at least 72 hours. It is adopted by the majority
approval, with at least 3 PMC affirmative votes.

Thanks,
Release Manager

[1] https://github.com/apache/bookkeeper/pull/3770/files
[2] https://dist.apache.org/repos/dist/dev/bookkeeper/bookkeeper-4.14.7-rc0/
[3] https://repository.apache.org/content/repositories/orgapachebookkeeper-1078
[4] https://github.com/apache/bookkeeper/tree/v4.14.7-rc0
[5] 3c85422a9497d168cadc5d264ebccbe0fadf3660

[DISCUSS] BookKeeper 4.14.7 release

[Hang Chen]

Hi everyone,

One core dump issue[1] related to RocksDB was introduced by [2] and
released in BookKeeper 4.14.6. To fix the RocksDB core dump issue, I
pushed one PR [3] to revert the [2] to make branch-4.14 stable.

Due to BookKeeper 4.14.6 having the risk of stability, I would like to
discuss starting the 4.14.7 release for those critical bug fixes.

Here are the PRs for 4.14.7:
https://github.com/apache/bookkeeper/pulls?q=is%3Apr+label%3Arelease%2F4.14.7+is%3Aclosed

If you have PRs that want to be included in this release, please
comment on that PR or tag the label `release/4.14.7`.

After those PRs merged, I would like to volunteer to release this patch.

Thanks,
Hang

[1] https://github.com/apache/bookkeeper/issues/3734
[2] https://github.com/apache/bookkeeper/pull/3653
[3] https://github.com/apache/bookkeeper/pull/3768

Re: [DISCUSS] BP-61: Revert BP-59 to release ByteBuf using ByteBuf.release() instead of ReferenceCountUtil.safeRelease()

[Hang Chen]

> +1 , use new specification (ReferenceCountUtil.release) replace BP-59

> I think it makes sense to replace ReferenceCountUtil.safeRelease
with ReferenceCountUtil.release.
FWIW ReferenceCountUtil.release's return value just says if the byte buf's
ref count reached zero or not, it is not an error handling mechanism.

The `ReferenceCountUtil.release` checks whether the ByteBuf is
`ReferenceCounted`. If not, it will do nothing and return false.
```java
public static boolean release(Object msg) {
return msg instanceof ReferenceCounted ?
((ReferenceCounted)msg).release() : false;
}
```
All the use cases in BookKeeper about `ReferenceCountUtil.release` are
not check the returned value.
Even Though all the ByteBuf in BookKeeper code are ReferenceCounted,
it still has risk of causing direct memory leak. One example is [1]

My question is why not use `ByteBuf.release()` directly?


[1] https://github.com/apache/bookkeeper/issues/3527

Thanks,
Hang

Enrico Olivelli  于2023年2月1日周三 15:35写道：
>
> Horizon
>
> Il giorno lun 30 gen 2023 alle ore 11:14 Horizon
> <1060026...@qq.com.invalid> ha scritto:
> >
> > Hi, Enrico. Just confirm, only revert 
> > https://github.com/apache/bookkeeper/pull/3528, it work well?
>
> yes
> This is the only thing we reverted in our fork
>
> see the history
> https://github.com/datastax/bookkeeper/commits/ds-4.14
>
> but I don't have all of the other commits in the history,
> so maybe other changes may be bad
>
> Unfortunately currently we (both BookKeeper community and in my
> company) don't have
> dedicated tests for BookKeeper but only for Pulsar,
> that makes things harder to track
>
> probably we should build some automated performance test:
> a set of scripts that create a BK cluster  on k8s, run some
> benchmarks/heavy load, verify that everything worked well
> The workbench should be re-usable by anyone, I am not saying that we
> should spend the ASF resources to run
> this kind of tests, but that we should provide a reference load test.
>
>
> Enrico
>
> >
> > > 2023年1月30日 16:04，Enrico Olivelli  写道：
> > >
> > > I think that we must do something, current master branch is not stable.
> > >
> > > My colleague Massimiliano opened this issue
> > > https://github.com/apache/bookkeeper/issues/3751
> > >
> > > Basically in the current master there is some problem that leads to
> > > Netty BytBuf corruption.
> > >
> > > The problem is solved by reverting this PR
> > > https://github.com/apache/bookkeeper/pull/3528 Fix memory leak when
> > > reading entry but the connection disconnected
> > >
> > > Enrico
> > >
> > > Il giorno lun 30 gen 2023 alle ore 08:04 steven lu
> > >  ha scritto:
> > >>
> > >> There are three ways to write release in current pulsar and bookkeeper
> > >> projects:
> > >> 1.ByteBuf.release: Call release directly without handling any exceptions;
> > >> 2.ReferenceCountUtil.release: return processing status;
> > >> 3. ReferenceCountUtil.safeRelease: without return value, print exception
> > >> information
> > >>
> > >> I don't think it is necessary to revert these PRs. We can discuss what is
> > >> the correct way to release, and then modify these three ways into the
> > >> correct way.
> > >>
> > >> Hang Chen  于2023年1月30日周一 12:28写道：
> > >>
> > >>> Hi guys,
> > >>>   In BP-59, which was not discussed in the dev mail list and without
> > >>> a vote, refactored the ByteBuf release method by
> > >>> ReferenceCountUtil.safeRelease() instead of ByteBuf.release().
> > >>>
> > >>> In the `ReferenceCountUtil.safeRelease()`, it catches the release
> > >>> exception. This change can make the ByteBuf release without any
> > >>> exceptions and makes the code run well, but it will make bugs hide
> > >>> deeper and more challenging to find out.
> > >>>   ```java
> > >>>   public static void safeRelease(Object msg) {
> > >>>try {
> > >>>release(msg);
> > >>>} catch (Throwable t) {
> > >>>logger.warn("Failed to release a message: {}", msg, t);
> > >>>}
> > >>>}
> > >>>   ```
> > >>> For example, if there is a bug to release the ByteBuf twice, whose
> > >>> refcnt is 1, we can find out the bug quickly if we use
> > >>> ByteBuf.release(), but the

[DISCUSS] BP-61: Revert BP-59 to release ByteBuf using ByteBuf.release() instead of ReferenceCountUtil.safeRelease()

[Hang Chen]

Hi guys,
   In BP-59, which was not discussed in the dev mail list and without
a vote, refactored the ByteBuf release method by
ReferenceCountUtil.safeRelease() instead of ByteBuf.release().

In the `ReferenceCountUtil.safeRelease()`, it catches the release
exception. This change can make the ByteBuf release without any
exceptions and makes the code run well, but it will make bugs hide
deeper and more challenging to find out.
   ```java
   public static void safeRelease(Object msg) {
try {
release(msg);
} catch (Throwable t) {
logger.warn("Failed to release a message: {}", msg, t);
}
}
   ```
For example, if there is a bug to release the ByteBuf twice, whose
refcnt is 1, we can find out the bug quickly if we use
ByteBuf.release(), but the bug will be hard to find out if we use
`ReferenceCountUtil.safeRelease()`

There are 12 PRs to refactor the ByteBuf release method, and I suggest
reverting those PRs. Do you have any ideas?

https://github.com/apache/bookkeeper/pull/3673
https://github.com/apache/bookkeeper/pull/3674
https://github.com/apache/bookkeeper/pull/3687
https://github.com/apache/bookkeeper/pull/3688
https://github.com/apache/bookkeeper/pull/3689
https://github.com/apache/bookkeeper/pull/3691
https://github.com/apache/bookkeeper/pull/3693
https://github.com/apache/bookkeeper/pull/3694
https://github.com/apache/bookkeeper/pull/3695
https://github.com/apache/bookkeeper/pull/3698
https://github.com/apache/bookkeeper/pull/3700
https://github.com/apache/bookkeeper/pull/3703


Thanks,
Hang

[Discuss] BookKeeper upgrade the RocksDB version to 7.9.x

[Hang Chen]

Hi guys,
When we introduced `deleteRange` for ledger index deletion, we
found a segment fault in ledger index deletion [1]. I suggest
upgrading the RocksDB version to the latest 7.9.x to see if the
RocksDB deleteRange segment fault bug has been fixed. I have tested
upgrading the RocksDB version from 6.10.2 to 7.9.2 and rollback to
6.10.2. This is the testing result.

- The upgrade process works fine in the following cases
  - Pulsar producer keeps producing messages to Pulsar topic
  - Pulsar consumer consumes messages from the Pulsar topic, and the
Pulsar broker fetches messages from the BookKeeper cluster
  - Trigger compaction to cleanup expired ledgers

- Rollback RocksDB version from 7.9.2 to 6.10.2, and bookie failed to
startup with the following exception.

```java
2023-01-29T17:09:27,794+0800 [main] ERROR
org.apache.bookkeeper.server.Main - Failed to build bookie server
java.io.IOException: Error open RocksDB database
at 
org.apache.bookkeeper.bookie.storage.ldb.KeyValueStorageRocksDB.(KeyValueStorageRocksDB.java:200)
~[org.apache.bookkeeper-bookkeeper-server-4.14.6.jar:4.14.6]
at 
org.apache.bookkeeper.bookie.storage.ldb.KeyValueStorageRocksDB.(KeyValueStorageRocksDB.java:89)
~[org.apache.bookkeeper-bookkeeper-server-4.14.6.jar:4.14.6]
at 
org.apache.bookkeeper.bookie.storage.ldb.KeyValueStorageRocksDB.lambda$static$0(KeyValueStorageRocksDB.java:63)
~[org.apache.bookkeeper-bookkeeper-server-4.14.6.jar:4.14.6]
at 
org.apache.bookkeeper.bookie.storage.ldb.LedgerMetadataIndex.(LedgerMetadataIndex.java:68)
~[org.apache.bookkeeper-bookkeeper-server-4.14.6.jar:4.14.6]
at 
org.apache.bookkeeper.bookie.storage.ldb.SingleDirectoryDbLedgerStorage.(SingleDirectoryDbLedgerStorage.java:170)
~[org.apache.bookkeeper-bookkeeper-server-4.14.6.jar:4.14.6]
at 
org.apache.bookkeeper.bookie.storage.ldb.DbLedgerStorage.newSingleDirectoryDbLedgerStorage(DbLedgerStorage.java:150)
~[org.apache.bookkeeper-bookkeeper-server-4.14.6.jar:4.14.6]
at 
org.apache.bookkeeper.bookie.storage.ldb.DbLedgerStorage.initialize(DbLedgerStorage.java:129)
~[org.apache.bookkeeper-bookkeeper-server-4.14.6.jar:4.14.6]
at org.apache.bookkeeper.bookie.Bookie.(Bookie.java:819)
~[org.apache.bookkeeper-bookkeeper-server-4.14.6.jar:4.14.6]
at 
org.apache.bookkeeper.proto.BookieServer.newBookie(BookieServer.java:152)
~[org.apache.bookkeeper-bookkeeper-server-4.14.6.jar:4.14.6]
at 
org.apache.bookkeeper.proto.BookieServer.(BookieServer.java:120)
~[org.apache.bookkeeper-bookkeeper-server-4.14.6.jar:4.14.6]
at 
org.apache.bookkeeper.server.service.BookieService.(BookieService.java:52)
~[org.apache.bookkeeper-bookkeeper-server-4.14.6.jar:4.14.6]
at org.apache.bookkeeper.server.Main.buildBookieServer(Main.java:304)
~[org.apache.bookkeeper-bookkeeper-server-4.14.6.jar:4.14.6]
at org.apache.bookkeeper.server.Main.doMain(Main.java:226)
~[org.apache.bookkeeper-bookkeeper-server-4.14.6.jar:4.14.6]
at org.apache.bookkeeper.server.Main.main(Main.java:208)
~[org.apache.bookkeeper-bookkeeper-server-4.14.6.jar:4.14.6]
Caused by: org.rocksdb.RocksDBException: unknown checksum type 4 in
data/bookkeeper/ledgers/current/ledgers/25.sst offset 1078 size 33
at org.rocksdb.RocksDB.open(Native Method)
~[org.rocksdb-rocksdbjni-6.10.2.jar:?]
at org.rocksdb.RocksDB.open(RocksDB.java:239)
~[org.rocksdb-rocksdbjni-6.10.2.jar:?]
at 
org.apache.bookkeeper.bookie.storage.ldb.KeyValueStorageRocksDB.(KeyValueStorageRocksDB.java:197)
~[org.apache.bookkeeper-bookkeeper-server-4.14.6.jar:4.14.6]
... 13 more
```
The root cause of this exception is that RocksDB 7.9.2 uses `kXXH3`
by default, and `kXXH3` is only supported since RocksDB 6.27
https://github.com/facebook/rocksdb/blob/79e57a39a33dbe17c8f51167e40e66d6c91f8eb4/include/rocksdb/table.h#L56

For the BookKeeper master branch, we have upgraded the RocksDB to
`6.29.4.1`, which can support RocksDB upgrade to 7.9.2 and rollback to
6.29.4.1.
For the RocksDB < 6.27, we can push a fix to ensure RocksDB 7.9.2 does
not use the latest checksum type `kXXH3`

I suggest doing this upgradation. Do you have any concerns?

[1] https://github.com/apache/bookkeeper/issues/3734

Thanks,
Hang

Re: BP-60:Change PCBC limitStatsLogging default value to true

[Hang Chen]

+1

Thanks,
Hang

Andrey Yegorov  于2023年1月10日周二 06:15写道：
>
> +1
>
> On Mon, Jan 9, 2023 at 4:50 AM Enrico Olivelli  wrote:
>
> > I agree with the change.
> > I have never used those metrics
> >
> > Enrico
> >
> >
> > Il giorno lun 26 dic 2022 alle ore 10:29 Wenbing Shen
> >  ha scritto:
> > >
> > > Hi BookKeepers, I've changed the limitStatsLogging default value to true
> > > from false:
> > > BP-60 
> > >
> > > Motivation
> > >
> > > We have an efficient online bookie cluster with hundreds of bookie nodes
> > > deployed on SSD disks.
> > >
> > > We separate the AutoRecovery cluster and the Bookie cluster for
> > independent
> > > deployment.
> > >
> > > I observed that our AutoRecovery cluster GC is very frequent. After
> > > investigation, I found that the limitStatsLogging of the bookkeeper
> > client
> > > PCBC is disabled by default, and a large number of channel monitoring
> > > indicators are generated. Due to the large number of bookie cluster
> > nodes,
> > > this metric data occupies a large amount of heap memory.
> > >
> > > A single StringWriter object occupies 16MB of memory, of which nearly 70
> > > StringWriter objects are waiting for the next GC to be destroyed,
> > occupying
> > > 1GB+ heap memory.
> > > Proposal
> > >
> > > In my use, I haven't found any usefulness of these PCBC monitoring
> > metrics
> > > data, at least so far, I haven't used it effectively.
> > >
> > > If our AutoRecovery and Bookie cluster are mixed in one process, these
> > > large objects will affect the performance and stability of Bookie
> > cluster.
> > >
> > > Since I can't find the meaning of these metrics by default, I suggest to
> > > adjust the default value of limitStatsLogging to true.
> > >
> > > Everyone can choose to turn it on or off, but by default, it is difficult
> > > for users to find out what effect this parameter will have, so that when
> > > their cluster grows to hundreds or thousands, when they realize the
> > problem
> > > sometimes, it is necessary to restart hundreds to thousands of bookies
> > in a
> > > rolling manner.
> > >
> > > At the same time, I observed that in pulsar, various monitoring of the
> > > bookkeeper client is turned off by default, because they really affect
> > the
> > > performance of the pulsar service, which is enough to show that we should
> > > try to change it, especially some very redundant metrics created based on
> > > channels.
> > > Compatibility, Deprecation, and Migration PlanClients that rely on PCBC
> > > metrics monitoring need to pay attention to this upgrade, but this will
> > not
> > > affect the actual functions of the client, only the metrics data, and
> > users
> > > can choose to open it again.
> > >
> > >
> > > What do you think about it?
> > >
> > > Best.
> > > Wenbing
> >
>
>
> --
> Andrey Yegorov

Re: [DISCUSS] Bookkeeper 4.16.0 release

[Hang Chen]

I will start to release 4.16.0 this week. Do you have any other
issues/tickets to add
for 4.16.0?

Thanks,
Hang

steven lu  于2022年7月28日周四 14:20写道：
>
> ok, I see , thanks
>
> Hang Chen  于2022年7月28日周四 10:29写道：
>
> > Hi Steven,
> > Those Prs are in the 4.16.0 milestone, and they are already
> > merged. They will be included in the 4.16.0 release.  Thanks for your
> > great work!
> >
> > Thanks,
> > Hang
> >
> > steven lu  于2022年7月28日周四 09:56写道：
> > >
> > > I think this index's optimization can release for 4.16.0
> > >  for details, have a look
> > https://github.com/apache/bookkeeper/issues/3419
> > >
> > > Hang Chen  于2022年7月20日周三 10:24写道：
> > >
> > > > Andrey, thanks for your noticing.
> > > >
> > > > I'm happy to be a volunteer for the version 4.16.0 release.
> > > > I have created an issue to track the Prs to be included in 4.16.0
> > > > https://github.com/apache/bookkeeper/issues/3412
> > > >
> > > > Feel free to add more PRs if you want them to be included in 4.16.0.
> > > > If you have time, welcome to help review them.
> > > >
> > > > Thanks,
> > > > Hang
> > > >
> > > > Nicolò Boschi  于2022年7月20日周三 01:18写道：
> > > > >
> > > > > +1 for 4.16.0
> > > > >
> > > > >
> > > > > > Btw, why https://bookkeeper.apache.org/docs/4.14.5/overview/ says
> > > > "Apache
> > > > > > BookKeeper 4.14.5, which is no longer actively maintained"?
> > > > > > I thought 4.14 was still maintained.
> > > > >
> > > > >
> > > > > Andrey, thanks for noticing. I created a pull for fixing it:
> > > > > https://github.com/apache/bookkeeper/pull/3411
> > > > >
> > > > >
> > > > > Nicolò Boschi
> > > > >
> > > > >
> > > > > Il giorno mar 19 lug 2022 alle ore 18:43 Andrey Yegorov <
> > > > > andrey.yego...@datastax.com> ha scritto:
> > > > >
> > > > > > +1 for the 4.16 release.
> > > > > > Hang, did you just volunteer to do the release? :)
> > > > > >
> > > > > > We need to do a pass on open PRs to review/merge what we can, there
> > > > are 100
> > > > > > open PRs right now.
> > > > > > I'd love to see at least these in the release:
> > > > > > https://github.com/apache/bookkeeper/pull/3361
> > > > > > https://github.com/apache/bookkeeper/pull/3383
> > > > > > https://github.com/apache/bookkeeper/pull/3384
> > > > > > https://github.com/apache/bookkeeper/pull/3390
> > > > > > https://github.com/apache/bookkeeper/pull/3325
> > > > > > https://github.com/apache/bookkeeper/pull/3329
> > > > > > https://github.com/apache/bookkeeper/pull/3330
> > > > > >
> > > > > > As for the minor releases, 4.15.1 is ok.
> > > > > > I think we also need 4.14.6; 4.15 was a major change and requires a
> > > > bit of
> > > > > > work to upgrade to (https://github.com/apache/pulsar/pull/15142),
> > e.g.
> > > > > > older Pulsar versions (2.10 and below) could benefit from 4.14
> > release.
> > > > > >
> > > > > > Btw, why https://bookkeeper.apache.org/docs/4.14.5/overview/ says
> > > > "Apache
> > > > > > BookKeeper 4.14.5, which is no longer actively maintained"?
> > > > > > I thought 4.14 was still maintained.
> > > > > >
> > > > > > On Fri, Jul 15, 2022 at 12:05 AM Enrico Olivelli <
> > eolive...@gmail.com>
> > > > > > wrote:
> > > > > >
> > > > > > > Hang,
> > > > > > >
> > > > > > > Il giorno ven 15 lug 2022 alle ore 08:25 ZhangJian He
> > > > > > >  ha scritto:
> > > > > > > >
> > > > > > > > Hi, Hang Chen. I hope to get this change into version 4.16.0.
> > > > > > > > https://github.com/apache/bookkeeper/pull/3406
> > > > > > > >
> > > > > > > > Thanks
> > > > > > > > ZhangJian He
> > > > > > > >

Re: [VOTE] Release 4.14.6, release candidate #1

[Hang Chen]

The vote is now closed, and the release was approved with
5 +1s(3 bindings and 2 non-binding), and I will continue with the
following steps.

Binding votes:
* Andrey Yegorov
* Enrico Olivelli
* Jia Zhai

Non-Binding votes:
* Nicolò Boschi
* steven lu

Thank you all for verifying and voting!

Jia Zhai  于2023年1月3日周二 10:50写道：
>
> +1 (binding)
>
> - verified packages checksum and signatures, they look good.
>
> - the source package build and test all run successfully.
>
> - 'bin/bookkeeper standalone' and 'bin/bookkeeper shell bookiesanity' run
> well.
>
>
> On Fri, Dec 30, 2022 at 2:55 PM steven lu  wrote:
>
> > +1 non binding
> >
> > Hang Chen  于2022年12月19日周一 13:15写道：
> >
> > > Hi everyone,
> > > Please review and vote on the release candidate #1 for version 4.14.6,
> > > as follows:
> > > [ ] +1, Approve the release
> > > [ ] -1, Do not approve the release (please provide specific comments)
> > >
> > > The complete staging area is available for your review, which includes:
> > > * Release notes [1]
> > > * The official Apache source and binary distributions to be deployed to
> > > dist.apache.org [2]
> > > * All artifacts to be deployed to the Maven Central Repository [3]
> > > * Source code tag "v4.14.6-rc1" [4] with git sha [5]
> > >
> > > BookKeeper's KEYS file contains the PGP keys we used to sign this
> > release:
> > > https://dist.apache.org/repos/dist/release/bookkeeper/KEYS
> > >
> > > Please download these packages and review this release candidate:
> > >
> > > - Review release notes
> > > - Download the source package (verify shasum and asc) and follow the
> > > instructions to build and run the bookkeeper service.
> > > - Download the binary package (verify shasum and asc) and follow the
> > > instructions to run the bookkeeper service.
> > > - Review the maven repo, release tag, licenses, and any other things you
> > > think
> > > it is essential to release.
> > >
> > > The vote will be open for at least 72 hours. The majority adopts it
> > > approval, with at least 3 PMC affirmative votes.
> > >
> > > Thanks,
> > > Release Manager
> > >
> > > [1] https://github.com/apache/bookkeeper/pull/3659/files
> > > [2]
> > > https://dist.apache.org/repos/dist/dev/bookkeeper/bookkeeper-4.14.6-rc1/
> > > [3]
> > >
> > https://repository.apache.org/content/repositories/orgapachebookkeeper-1077/
> > > [4] https://github.com/apache/bookkeeper/tree/v4.14.6-rc1
> > > [5] a96884ec279f5fbcd628c7ae3791d99a8e19ca69
> > >
> >

Re: [VOTE] Release 4.14.6, release candidate #1

[Hang Chen]

I opened one PR[1] to upgrade BookKeeper dependency to 4.14.6 for
Pulsar branch-2.10, and all tests passed. Please help verify this
candidate.

Thanks,
Hang

[1] https://github.com/hangc0276/pulsar/pull/8

Hang Chen  于2022年12月19日周一 13:14写道：
>
> Hi everyone,
> Please review and vote on the release candidate #1 for version 4.14.6,
> as follows:
> [ ] +1, Approve the release
> [ ] -1, Do not approve the release (please provide specific comments)
>
> The complete staging area is available for your review, which includes:
> * Release notes [1]
> * The official Apache source and binary distributions to be deployed to
> dist.apache.org [2]
> * All artifacts to be deployed to the Maven Central Repository [3]
> * Source code tag "v4.14.6-rc1" [4] with git sha [5]
>
> BookKeeper's KEYS file contains the PGP keys we used to sign this release:
> https://dist.apache.org/repos/dist/release/bookkeeper/KEYS
>
> Please download these packages and review this release candidate:
>
> - Review release notes
> - Download the source package (verify shasum and asc) and follow the
> instructions to build and run the bookkeeper service.
> - Download the binary package (verify shasum and asc) and follow the
> instructions to run the bookkeeper service.
> - Review the maven repo, release tag, licenses, and any other things you think
> it is essential to release.
>
> The vote will be open for at least 72 hours. The majority adopts it
> approval, with at least 3 PMC affirmative votes.
>
> Thanks,
> Release Manager
>
> [1] https://github.com/apache/bookkeeper/pull/3659/files
> [2] https://dist.apache.org/repos/dist/dev/bookkeeper/bookkeeper-4.14.6-rc1/
> [3] 
> https://repository.apache.org/content/repositories/orgapachebookkeeper-1077/
> [4] https://github.com/apache/bookkeeper/tree/v4.14.6-rc1
> [5] a96884ec279f5fbcd628c7ae3791d99a8e19ca69

[VOTE] Release 4.14.6, release candidate #1

[Hang Chen]

Hi everyone,
Please review and vote on the release candidate #1 for version 4.14.6,
as follows:
[ ] +1, Approve the release
[ ] -1, Do not approve the release (please provide specific comments)

The complete staging area is available for your review, which includes:
* Release notes [1]
* The official Apache source and binary distributions to be deployed to
dist.apache.org [2]
* All artifacts to be deployed to the Maven Central Repository [3]
* Source code tag "v4.14.6-rc1" [4] with git sha [5]

BookKeeper's KEYS file contains the PGP keys we used to sign this release:
https://dist.apache.org/repos/dist/release/bookkeeper/KEYS

Please download these packages and review this release candidate:

- Review release notes
- Download the source package (verify shasum and asc) and follow the
instructions to build and run the bookkeeper service.
- Download the binary package (verify shasum and asc) and follow the
instructions to run the bookkeeper service.
- Review the maven repo, release tag, licenses, and any other things you think
it is essential to release.

The vote will be open for at least 72 hours. The majority adopts it
approval, with at least 3 PMC affirmative votes.

Thanks,
Release Manager

[1] https://github.com/apache/bookkeeper/pull/3659/files
[2] https://dist.apache.org/repos/dist/dev/bookkeeper/bookkeeper-4.14.6-rc1/
[3] https://repository.apache.org/content/repositories/orgapachebookkeeper-1077/
[4] https://github.com/apache/bookkeeper/tree/v4.14.6-rc1
[5] a96884ec279f5fbcd628c7ae3791d99a8e19ca69

Re: [VOTE] Release 4.14.6, release candidate #0

[Hang Chen]

Hi guys,
 One issue is the direct memory metric, incompatible with JDK8
introduced by [1] and fixed by [2]. I will close this vote and create
a new candidate soon.

Thanks,
Hang

[1] https://github.com/apache/bookkeeper/pull/3252
[2] https://github.com/apache/bookkeeper/pull/3677

Enrico Olivelli  于2022年12月19日周一 05:20写道：
>
> +1 (binding)
>
> Enrico
>
> Il giorno gio 1 dic 2022 alle ore 07:30 Yong Zhang
>  ha scritto:
> >
> > okay. Make sense.
> >
> > --
> > Yong
> >
> > On Thu, 1 Dec 2022 at 07:52, Andrey Yegorov 
> > wrote:
> >
> > > +1 (binding)
> > >
> > > * ran bookkeeper standalone
> > > * ran bookkeeper localbookie
> > > * built from sources (mac, intel), ran some tests
> > > * spot-checked maven repo
> > >
> > > @Yong - as I understand, build on M1 is not a regression for 4.14.x so it
> > > should not be a blocker.
> > >
> > > On Mon, Nov 21, 2022 at 4:03 AM Hang Chen  wrote:
> > >
> > > > Hi everyone,
> > > > Please review and vote on the release candidate #0 for version 4.14.6,
> > > > as follows:
> > > > [ ] +1, Approve the release
> > > > [ ] -1, Do not approve the release (please provide specific comments)
> > > >
> > > > The complete staging area is available for your review, which includes:
> > > > * Release notes [1]
> > > > * The official Apache source and binary distributions to be deployed to
> > > > dist.apache.org [2]
> > > > * All artifacts to be deployed to the Maven Central Repository [3]
> > > > * Source code tag "v4.14.6-rc0" [4] with git sha [5]
> > > >
> > > > BookKeeper's KEYS file contains the PGP keys we used to sign this
> > > release:
> > > > https://dist.apache.org/repos/dist/release/bookkeeper/KEYS
> > > >
> > > > Please download these packages and review this release candidate:
> > > >
> > > > - Review release notes
> > > > - Download the source package (verify shasum, and asc) and follow the
> > > > instructions to build and run the bookkeeper service.
> > > > - Download the binary package (verify shasum, and asc) and follow the
> > > > instructions to run the bookkeeper service.
> > > > - Review the maven repo, release tag, licenses, and any other things you
> > > > think
> > > > it is important to release.
> > > >
> > > > The vote will be open for at least 72 hours. It is adopted by the
> > > majority
> > > > approval, with at least 3 PMC affirmative votes.
> > > >
> > > > Thanks,
> > > > Release Manager
> > > >
> > > > [1] https://github.com/apache/bookkeeper/pull/3659/files
> > > > [2]
> > > > https://dist.apache.org/repos/dist/dev/bookkeeper/bookkeeper-4.14.6-rc0/
> > > > [3]
> > > >
> > > https://repository.apache.org/content/repositories/orgapachebookkeeper-1076/
> > > > [4] https://github.com/apache/bookkeeper/tree/v4.14.6-rc0
> > > > [5] 9f453834c6039fac46dc19130288dbf0b2996f2e
> > > >
> > >
> > >
> > > --
> > > Andrey Yegorov
> > >

Re: [ANNOUNCE] Apache BookKeeper 4.15.3 released

[Hang Chen]

Thank you a lot for driving this release!

Best,
Hang

Andrey Yegorov  于2022年11月24日周四 02:21写道：
>
> Thank you for driving this release!
>
> On Wed, Nov 16, 2022 at 7:01 PM Yong Zhang  wrote:
>
> > The Apache BookKeeper team is proud to announce Apache BookKeeper
> > version 4.15.3.
> >
> > Apache BookKeeper is a scalable, fault-tolerant, and low-latency storage
> > service optimized for real-time workloads. It has been used for a
> > fundamental
> > service to build reliable services. It is also the log segment store for
> > Apache
> > DistributedLog and the message store for Apache Pulsar.
> >
> > This is the 35rd release of the Apache BookKeeper.
> >
> > Release 4.15.3 contains performance and stability improvements, new
> > metrics,
> > and various bug fixes.
> >
> > Apache BookKeeper users are encouraged to upgrade to 4.15.3. The technical
> > details of this release are summarized below.
> > For BookKeeper release details and downloads, visit:
> >
> > https://bookkeeper.apache.org/releases
> >
> > BookKeeper 4.15.3 Release Notes are at:
> >
> > https://bookkeeper.apache.org/release-notes/#4153
> >
> > We would like to thank the contributors that made the release possible.
> >
> > Regards,
> >
> > The BookKeeper Team
> >
>
>
> --
> Andrey Yegorov

[VOTE] Release 4.14.6, release candidate #0

[Hang Chen]

Hi everyone,
Please review and vote on the release candidate #0 for version 4.14.6,
as follows:
[ ] +1, Approve the release
[ ] -1, Do not approve the release (please provide specific comments)

The complete staging area is available for your review, which includes:
* Release notes [1]
* The official Apache source and binary distributions to be deployed to
dist.apache.org [2]
* All artifacts to be deployed to the Maven Central Repository [3]
* Source code tag "v4.14.6-rc0" [4] with git sha [5]

BookKeeper's KEYS file contains the PGP keys we used to sign this release:
https://dist.apache.org/repos/dist/release/bookkeeper/KEYS

Please download these packages and review this release candidate:

- Review release notes
- Download the source package (verify shasum, and asc) and follow the
instructions to build and run the bookkeeper service.
- Download the binary package (verify shasum, and asc) and follow the
instructions to run the bookkeeper service.
- Review the maven repo, release tag, licenses, and any other things you think
it is important to release.

The vote will be open for at least 72 hours. It is adopted by the majority
approval, with at least 3 PMC affirmative votes.

Thanks,
Release Manager

[1] https://github.com/apache/bookkeeper/pull/3659/files
[2] https://dist.apache.org/repos/dist/dev/bookkeeper/bookkeeper-4.14.6-rc0/
[3] https://repository.apache.org/content/repositories/orgapachebookkeeper-1076/
[4] https://github.com/apache/bookkeeper/tree/v4.14.6-rc0
[5] 9f453834c6039fac46dc19130288dbf0b2996f2e

Re: [VOTE] Release 4.15.3, release candidate #1

[Hang Chen]

+1 (non-binding)

- Build from source code (JDK8, maven 3.8.6 on MacOs)
- Verified checksums and signatures
- Verified the licenses
- Run bkperf to test journal write
- Integrate with Pulsar master code and Pulsar unit tests passed
（https://github.com/hangc0276/pulsar/pull/7）


Thanks Yong for driving the release

Thanks,
Hang

Andrey Yegorov  于2022年11月12日周六 00:08写道：
>
> +1 (binding)
>
> On Thu, Nov 10, 2022 at 8:06 PM Yong Zhang 
> wrote:
>
> > Hi Andrey,
> >
> > I run the test multiple times, I found that depends on how the DNS
> > resolve the address.
> > When the DNS resolves to 127.0.0.1/localhost, the test will be failed.
> > But if it resolves another INet interface, like lo0, the hostname will be
> > 192.168.50.240(in my computer) and the test passed.
> >
> > Thanks,
> > Yong
> >
> > On Fri, 11 Nov 2022 at 09:46, Yong Zhang 
> > wrote:
> >
> > > I downloaded the src package and run the tests successfully.
> > > And build from branch-4.15 passed as well. But failed when
> > > I run it with IDE.
> > >
> > > It looks like sometimes it uses the loopback address and we
> > > are not allowed that.
> > > But it shouldn't be a problem for this release.
> > >
> > > Thanks,
> > > Yong
> > >
> > > On Fri, 11 Nov 2022 at 09:01, Andrey Yegorov <
> > andrey.yego...@datastax.com>
> > > wrote:
> > >
> > >> I tested the following:
> > >>
> > >> * bookkeeper-all
> > >>   ** unpacks
> > >>   ** shell works, standalone starts, localbookie starts
> > >> * src
> > >>   ** unpacks, builds
> > >>   ** tests - there are some know flakes
> > >> and org.apache.bookkeeper.client.TestSpeculativeRead fails for me
> > >>
> > >> I don't recall any issues with TestSpeculativeRead in the previous
> > >> releases; does it repro for you (I built from unpacked sources for the
> > >> RC)?
> > >>
> > >> $ mvn test -f bookkeeper-server/pom.xml
> > >> -Dtest=org.apache.bookkeeper.client.TestSpeculativeRead
> > >>
> > >> [ERROR] Tests run: 6, Failures: 0, Errors: 1, Skipped: 0, Time elapsed:
> > >> 73.673 s <<< FAILURE! - in
> > >> org.apache.bookkeeper.client.TestSpeculativeRead
> > >> [ERROR]
> > >>
> > >>
> > org.apache.bookkeeper.client.TestSpeculativeRead.testSequenceReadLocalEnsemble
> > >>  Time elapsed: 6.341 s  <<< ERROR!
> > >> java.lang.RuntimeException:
> > >> java.net.UnknownHostException: Trying to listen on loopback address,
> > >> 127.0.0.1:3181 but this is forbidden by default (see
> > >> ServerConfiguration#getAllowLoopback()).
> > >> If this happen, you can consider specifying the network interface to
> > >> listen
> > >> on (e.g. listeningInterface=eth0) or specifying the advertised address
> > >> (e.g. advertisedAddress=172.x.y.z)
> > >> at
> > >>
> > >>
> > org.apache.bookkeeper.bookie.LocalBookieEnsemblePlacementPolicy.initialize(LocalBookieEnsemblePlacementPolicy.java:68)
> > >> at
> > >>
> > >>
> > org.apache.bookkeeper.client.BookKeeper.initializeEnsemblePlacementPolicy(BookKeeper.java:583)
> > >> at org.apache.bookkeeper.client.BookKeeper.(BookKeeper.java:507)
> > >> at
> > >>
> > >>
> > org.apache.bookkeeper.client.BookKeeperTestClient.(BookKeeperTestClient.java:51)
> > >> at
> > >>
> > >>
> > org.apache.bookkeeper.client.TestSpeculativeRead.testSequenceReadLocalEnsemble(TestSpeculativeRead.java:395)
> > >> at
> > java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> > >> Method)
> > >> at
> > >>
> > >>
> > java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> > >> at
> > >>
> > >>
> > java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> > >> at java.base/java.lang.reflect.Method.invoke(Method.java:566)
> > >> at
> > >>
> > >>
> > org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> > >> at
> > >>
> > >>
> > org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> > >> at
> > >>
> > >>
> > org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> > >> at
> > >>
> > >>
> > org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> > >> at
> > >>
> > >>
> > org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> > >> at
> > >>
> > >>
> > org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> > >> at
> > >>
> > >>
> > org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:298)
> > >> at
> > >>
> > >>
> > org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:292)
> > >> at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
> > >> at java.base/java.lang.Thread.run(Thread.java:829)
> > >> Caused by: java.net.UnknownHostException: Trying to listen on loopback
> > >> address, 127.0.0.1:3181 but this is forbidden by default (see
> > >> ServerConfiguration#getAllowLoopback()).
> > >> If this happen, you can consider specifying the network interface to
> 

Introduce entry log file size checking during garbage collection

[Hang Chen]

Hi guys,
   When we use TransactionalEntryLogCompactor to compact the entry log
files in garbage collection, it will generate a lot of small entry log
files, and for those files, the file usage is usually greater than
90%, which can not be compacted unless the file usage decreased. As
time goes on, the number of small entry log files will increase and
bring heavy pressure on heap memory to store entryLogMetaMap and also
bring heavy pressure on RocksDB to store the index.

   After we change the TransactionalEntryLogCompactor to
EntryLogCompactor, it will not generate new small entry log files, but
those old small entry log files can't be compacted.

   The root cause of those small entry log files that can't be
compacted is that we only check the file usage before compaction. If
the file usage is below the threshold, this entry log file will be
compacted, otherwise, it will be skipped in garbage collection.

   I want to introduce the entry log file size checking. If the entry
log file total size is below the specific percentage of the max entry
log file size, we will compact this entry log file no matter whether
the file usage is greater than the threshold or not. The percentage
can be configured in conf/bk_server.conf, and it will be set to 0.0,
which means disabling the file size checking.

   The PR: https://github.com/apache/bookkeeper/pull/3631

   Do you have any ideas?

   Thanks,
   Hang

Re: Release 4.15.3, release candidate #0

[Hang Chen]

-1 (non-binding)

- Build from source code (jdk8 on MacOs)
- Verified checksums and signatures
- Checked the licenses
- Run journal perf produce
- Integrate with Pulsar master code and Pulsar unit tests passed

>From the license checking, I found the following license mismatch.
```
ch.qos.logback-logback-classic-1.2.10.jar unaccounted for in LICENSE
ch.qos.logback-logback-core-1.2.10.jar unaccounted for in LICENSE
io.netty-netty-tcnative-2.0.48.Final.jar unaccounted for in LICENSE
io.netty-netty-transport-native-epoll-4.1.75.Final.jar unaccounted for
in LICENSE

It looks like there are issues with the LICENSE/NOTICE in
../bookkeeper-server-4.15.3-bin.tar.gz.
```

Thanks for Yong driving the release.


Thanks,
Hang

Enrico Olivelli  于2022年11月8日周二 17:34写道：
>
> +1 (binding)
>
> - Built from sources (JDK11, MacOs)
> - Run some smoke tests with localbookie
> - Built and run smoke tests with Pulsar 2.11-SNAPSHOT
> - Signatures/checksums look good
>
> Thanks for driving the release
>
> Enrico
>
> Il giorno ven 4 nov 2022 alle ore 03:43 Yong Zhang
>  ha scritto:
> >
> > Hi everyone,
> > Please review and vote on the release candidate #0 for the version 4.15.3,
> > as follows:
> > [ ] +1, Approve the release
> > [ ] -1, Do not approve the release (please provide specific comments)
> >
> > The complete staging area is available for your review, which includes:
> > * Release notes [1]
> > * The official Apache source and binary distributions to be deployed to
> > dist.apache.org [2]
> > * All artifacts to be deployed to the Maven Central Repository [3]
> > * Source code tag "v4.15.3-rc0" [4] with git sha [5]
> >
> > BookKeeper's KEYS file contains PGP keys we used to sign this release:
> > https://dist.apache.org/repos/dist/release/bookkeeper/KEYS
> >
> > Please download these packages and review this release candidate:
> >
> > - Review release notes
> > - Download the source package (verify shasum, and asc) and follow the
> > instructions to build and run the bookkeeper service.
> > - Download the binary package (verify shasum, and asc) and follow the
> > instructions to run the bookkeeper service.
> > - Review maven repo, release tag, licenses, and any other things you think
> > it is important to a release.
> >
> > The vote will be open for at least 72 hours. It is adopted by majority
> > approval, with at least 3 PMC affirmative votes.
> >
> > Thanks,
> > Release Manager
> >
> > [1]
> > https://github.com/apache/bookkeeper/pull/3602/files#diff-d361215d5d5abaa96ac3362e70a1d2f80f92d369a59c866e1976ff41ec435f53R4
> > 
> > [2] https://dist.apache.org/repos/dist/dev/bookkeeper/bookkeeper-4.15.3-rc0/
> > [3]
> > https://repository.apache.org/content/repositories/orgapachebookkeeper-1074
> > [4] https://github.com/apache/bookkeeper/releases/tag/v4.15.3-rc0
> > [5] 029622128e82c5120d3c70f9fc0761e3687e05af

[DISCUSS] BookKeeper 4.14.6 release

[Hang Chen]

Hi everyone,

BookKeeper 4.14.5 has been released for 6 months and I would like to
discuss the starting 4.14.6 release to include some bug fixes and
improvements.

Here are the PRs for 4.14.6:
https://github.com/apache/bookkeeper/pulls?q=is%3Aopen+is%3Apr+label%3Arelease%2F4.14.6

If you have PRs that want to be included in this release, please
comment on that PR or tag the label `release/4.14.6`.

After those PRs merged, I would like to be the volunteer to release this patch.

Thanks,
Hang

Re: [DISCUSS] BookKeeper 4.15.3 release

[Hang Chen]

+1

Thanks for your great work!

Thanks,
Hang

Yong Zhang  于2022年11月3日周四 16:28写道：
>
> If no one objected, I will start to release 4.15.3 soon.
>
> On Wed, 26 Oct 2022 at 09:23, Yong Zhang  wrote:
>
> > Hi everyone,
> >
> > I want to discuss starting 4.15.3 release to including some
> > fixes/improvements
> > and compatibility fix(The RocksDB configuration values).
> >
> >
> > Here are the PRs for 4.15.3:
> >
> > https://github.com/apache/bookkeeper/pulls?q=+is%3Apr+label%3Arelease%2F4.15.3+
> >
> > If you have PR want to include in this release, please comment on that PR
> > or tag
> > the label `release/4.15.1`.
> >
> > After those PR merged, I would like to be the volunteer to release this
> > patch.
> >
> > Thanks,
> > Yong
> >

Re: [DISCUSS] BP-57 Add a public API for creating embeddable servers

[Hang Chen]

+1

Great feature! I support this feature.

Thanks,
Hang

steven lu  于2022年9月19日周一 15:47写道：
>
> +1
> this idea seems really useful
>
> Diego Salvi  于2022年9月18日周日 21:02写道：
>
> > Hi BookKeepers,
> > I've added a new BP to add a public API for creating embeddable servers:
> > BP-57 
> >
> > *Motivation*
> > BK is used and run in many OS project as an embedded resource.
> > Every time a project needs to embed BK (think about Pulsar, Pravega or
> > HerdDB as some examples) boilerplate code need to be written (and then
> > mantained as per 4.15 changes).
> > Depending from the contest new feature cannot be made directly available
> > and more code need to be added.
> > To run an embedded bookie you need to instantiate many components that are
> > really in "public" api and can and will be subject to changes.
> >
> > *Proposal*
> > We already have the whole code to start a BK server instance, it just needs
> > to be refactored and exposed as a public API.
> > My idea is to refactor all the code currently at
> > org.apache.bookkeeper.server.Main#buildBookieServer and expose it with a
> > builder pattern.
> > The builder will accept an initial BookieConfiguration (and with no more
> > interaction will produce the same server currently produced from
> > org.apache.bookkeeper.server.Main#buildBookieServer) and will create from
> > the configuration any component not directly provided to the builder.
> > For example you can provide your instances for StatsProvider or
> > MetadataBookieDriver instance or your custom ByteBufAllocator without the
> > need to rewrite the same plain old code.
> > Another big improvement: new features that whould need additional code to
> > be added (just think when integrity checking has been added in BP-46) are
> > already and directly usable.
> >
> > What do you think about it?
> >
> > Diego Salvi
> >

Re: [DISCUSS] BookKeeper RocksDB configuration changed default blockCacheSize since 4.15.0

[Hang Chen]

For the pr https://github.com/apache/bookkeeper/pull/3523

In BookKeeper and Pulsar's conf folder, it already has default RocksDB
configuration files. In this Pr, the loading order is RocksDB
Configuration file > conf/bk_server.conf, which means it will still
load the RocksDB configuration file by default and will cause a
compatibility issue in the BookKeeper version upgrade.

Thanks,
Hang

Nicolò Boschi  于2022年10月17日周一 16:08写道：
>
> The pull has been merged and cherry-picked to branch-4.15
>
> Thanks,
> Nicolò Boschi
>
>
> Il giorno gio 13 ott 2022 alle ore 12:58 Shiji Lu  ha
> scritto:
>
> > I think we can start a new discussion about the modification. The
> > modification method can be done better.
> > we automatically calculate the blockCacheSize through the configuration
> > started by bin/bookkeeper, and then update
> > conf/entry_location_rocksdb.conf, or I also mention a pr to make a scheme
> > comparison ? Or make this function a switch to choose the implementation
> >
> >
> > On 2022/10/09 15:03:01 Hang Chen wrote:
> > > I found the PR https://github.com/apache/bookkeeper/pull/3056 has
> > > changed the rocksDB default cache size from 10% of direct memory to
> > > 206150041(196MB), which will lead to entry read performance decrease
> > > when there are huge number of entries stored in the ledger directory.
> > > It will have a huge impact on the read performance when the BookKeeper
> > > cluster upgrades from 4.14.x to 4.15.x.
> > >
> > > The motivation of PR 3056 is to change RocksDB configuration to a
> > > single configuration file, and it introduces the following drawbacks.
> > > 1. Most users, especially those unfamiliar with RocksDB, will be
> > > confused about tuning RocksDB performance due to lack of guidance on
> > > some important key parameters.
> > > 2. The RocksDB blockCacheSize configuration can only be set to a fixed
> > > value, not a percentage of direct memory.
> > >
> > > In order to simplify the RocksDB configuration and make it easy to
> > > tune the performance of the RocksDB, I prefer to separate the RocksDB
> > > configuration into two parts.
> > > 1. The most important configurations, which is usually changed to tune
> > > RocksDB performance, will be located in conf/bk_server.conf
> > > 2. Other advanced configuration will be in a separate RocksDB
> > configuration file
> > >
> > > For the default RocksDB blockCacheSize change, Do you need to trigger
> > > a new release to change it back? Current Pulsar master branch uses
> > > BookKeeper 4.14.x, and we can change the default value back to 10% of
> > > direct memory to avoid the entry read performance degradation caused
> > > by upgrading Pulsar's BookKeeper dependency to 4.15.x.
> > >
> > > Do you guys have any suggestions?
> > >
> > > Thanks,
> > > Hang
> > >
> >

[DISCUSS] BookKeeper versions on the website.

[Hang Chen]

Hi guys,

On the BookKeeper website, we only show the latest patch release
version document, for example, we have BookKeeper versions [4.14.0,
4.14.1, 4.14.2, 4.14.3, 4.14.4, 4.14.5] and we only show 4.14.5's
document in the website. It is reasonable because the path release
version only contains bug fixes and the document won't have break
changes. However, it will be confusing for new users. If he uses
BookKeeper version 4.14.3, but cannot find related documents on the
website.

One solution is to change the BookKeeper version on the website from
4.14.5 to 4.14.x to represent all the patch release version documents
in 4.14. The same way works in Pulsar[1] and Kafka[2] website
documents.

Any thoughts?

[1] https://pulsar.apache.org/docs/
[2] https://kafka.apache.org/documentation/

Thanks,
Hang

Re: [DISCUSS] BookKeeper RocksDB configuration changed default blockCacheSize since 4.15.0

[Hang Chen]

I agree with Yong, we should back-support the existing configuration,
otherwise, the pulsar helm chart also need to be changed when upgrade
BookKeeper 4.14.x to 4.15.x

Thanks,
Hang

Enrico Olivelli  于2022年10月10日周一 16:07写道：
>
> Il Lun 10 Ott 2022, 07:44 Yong Zhang  ha
> scritto:
>
> > We shouldn't remove any existing configuration or feature in a so short
> > time. I think at least we need to back-support the existing configuration.
> > If we must do the break, we should make the existing things in a
> > deprecation
> > state for a while.
> >
> > Breaking existing configurations would make a huge impact.
> > Maybe we can add back the previous configurations in the next minor
> > release for 4.15. So user can upgrade their cluster to 4.15.3 without
> > changing any configuration.
> >
> > WDYT?
> >
>
> Great idea
>
> Enrico
>
>
> > Yong
> >
> >
> >
> >
> > On Sun, 9 Oct 2022 at 23:20, Enrico Olivelli  wrote:
> >
> > > Il Dom 9 Ott 2022, 17:03 Hang Chen  ha scritto:
> > >
> > > > I found the PR https://github.com/apache/bookkeeper/pull/3056 has
> > > > changed the rocksDB default cache size from 10% of direct memory to
> > > > 206150041(196MB), which will lead to entry read performance decrease
> > > > when there are huge number of entries stored in the ledger directory.
> > > > It will have a huge impact on the read performance when the BookKeeper
> > > > cluster upgrades from 4.14.x to 4.15.x.
> > > >
> > >
> > >
> > > I think that that patch shoud had passed from the BP process.
> > > It is a huge breaking change.
> > > Personally I missed it.
> > >
> > > BTW now it is there, we should have talked about it in the release notes,
> > > and we now must add some guidance to users upgrading to this version of
> > BK.
> > >
> > > It is a pity that we cannot keep the automatic value.
> > >  BTW in all the Pulsar cluster I know I think that they are overriding
> > that
> > > value because the default value is not big enough.
> > >
> > > I think that at this point we only have to document on the website the
> > > breaking change and add a page about BK and RocksDB
> > >
> > > Enrico
> > >
> > >
> > > > The motivation of PR 3056 is to change RocksDB configuration to a
> > > > single configuration file, and it introduces the following drawbacks.
> > > > 1. Most users, especially those unfamiliar with RocksDB, will be
> > > > confused about tuning RocksDB performance due to lack of guidance on
> > > > some important key parameters.
> > > > 2. The RocksDB blockCacheSize configuration can only be set to a fixed
> > > > value, not a percentage of direct memory.
> > > >
> > > > In order to simplify the RocksDB configuration and make it easy to
> > > > tune the performance of the RocksDB, I prefer to separate the RocksDB
> > > > configuration into two parts.
> > > > 1. The most important configurations, which is usually changed to tune
> > > > RocksDB performance, will be located in conf/bk_server.conf
> > > > 2. Other advanced configuration will be in a separate RocksDB
> > > > configuration file
> > > >
> > > > For the default RocksDB blockCacheSize change, Do you need to trigger
> > > > a new release to change it back? Current Pulsar master branch uses
> > > > BookKeeper 4.14.x, and we can change the default value back to 10% of
> > > > direct memory to avoid the entry read performance degradation caused
> > > > by upgrading Pulsar's BookKeeper dependency to 4.15.x.
> > > >
> > > > Do you guys have any suggestions?
> > > >
> > > > Thanks,
> > > > Hang
> > > >
> > >
> >

[DISCUSS] BookKeeper RocksDB configuration changed default blockCacheSize since 4.15.0

[Hang Chen]

I found the PR https://github.com/apache/bookkeeper/pull/3056 has
changed the rocksDB default cache size from 10% of direct memory to
206150041(196MB), which will lead to entry read performance decrease
when there are huge number of entries stored in the ledger directory.
It will have a huge impact on the read performance when the BookKeeper
cluster upgrades from 4.14.x to 4.15.x.

The motivation of PR 3056 is to change RocksDB configuration to a
single configuration file, and it introduces the following drawbacks.
1. Most users, especially those unfamiliar with RocksDB, will be
confused about tuning RocksDB performance due to lack of guidance on
some important key parameters.
2. The RocksDB blockCacheSize configuration can only be set to a fixed
value, not a percentage of direct memory.

In order to simplify the RocksDB configuration and make it easy to
tune the performance of the RocksDB, I prefer to separate the RocksDB
configuration into two parts.
1. The most important configurations, which is usually changed to tune
RocksDB performance, will be located in conf/bk_server.conf
2. Other advanced configuration will be in a separate RocksDB configuration file

For the default RocksDB blockCacheSize change, Do you need to trigger
a new release to change it back? Current Pulsar master branch uses
BookKeeper 4.14.x, and we can change the default value back to 10% of
direct memory to avoid the entry read performance degradation caused
by upgrading Pulsar's BookKeeper dependency to 4.15.x.

Do you guys have any suggestions?

Thanks,
Hang

Re: [VOTE] Release 4.15.2, release candidate #0

[Hang Chen]

Hi Yong,
Sorry for -1(non-binding)
I found the PR https://github.com/apache/bookkeeper/pull/3056 has
changed the rocksDB default cache size from 10% of direct memory to
206150041(196MB), which will lead to entry read performance decrease
when there are huge number of entries stored in the ledger directory.
It will has huge impact to the read performance when BookKeeper
cluster upgrade from 4.14.x to 4.15.x.

Thanks,
Hang

Yong Zhang  于2022年10月8日周六 15:08写道：
>
> The vote is now closed and the release was approved with 5 +1s(3 bindings
> and 2 non-binding);
>
> Binding votes:
> * Enrico Olivelli
> * Andrey Yegorov
> * Jia Zhai
>
> Non-Binding votes:
> * Nicolò Boschi
> * steven lu
>
> Thank you all guys for voting!
>
> On Sat, 8 Oct 2022 at 13:48, Jia Zhai  wrote:
>
> > +1 (binding)
> >
> > - verified packages checksum and signatures, they look good.
> >
> > - the source package build and test runs well.
> >
> > - in both binary packages, 'bin/bookkeeper standalone' and
> > 'bin/bookkeeper shell bookiesanity' runs well.
> >
> > On Thu, Oct 6, 2022 at 8:22 PM steven lu  wrote:
> >
> > > +1 - non binding
> > > - deployed and verified in our own test environment, no problems were
> > found
> > >
> > >
> > > Enrico Olivelli  于2022年10月6日周四 18:03写道：
> > >
> > > > Il Mer 5 Ott 2022, 03:25 steven lu  ha scritto:
> > > >
> > > > > +1 (binding)
> > > > >
> > > >
> > > > Sorry, but binding votes are only from PMC members
> > > >
> > > > Enrico
> > > >
> > > > deployed and verified in our own test environment, no problems were
> > found
> > > > >
> > > > > Yong Zhang  于2022年9月23日周五 15:32写道：
> > > > >
> > > > > > Hi everyone,
> > > > > > Please review and vote on the release candidate #0 for the version
> > > > > 4.15.2,
> > > > > > as follows:
> > > > > > [ ] +1, Approve the release
> > > > > > [ ] -1, Do not approve the release (please provide specific
> > comments)
> > > > > >
> > > > > > The complete staging area is available for your review, which
> > > includes:
> > > > > > * Release notes [1]
> > > > > > * The official Apache source and binary distributions to be
> > deployed
> > > to
> > > > > > dist.apache.org [2]
> > > > > > * All artifacts to be deployed to the Maven Central Repository [3]
> > > > > > * Source code tag "v4.15.2-rc0" [4] with git sha [5]
> > > > > >
> > > > > > BookKeeper's KEYS file contains PGP keys we used to sign this
> > > release:
> > > > > > https://dist.apache.org/repos/dist/release/bookkeeper/KEYS
> > > > > >
> > > > > > Please download these packages and review this release candidate:
> > > > > >
> > > > > > - Review release notes
> > > > > > - Download the source package (verify shasum, and asc) and follow
> > the
> > > > > > instructions to build and run the bookkeeper service.
> > > > > > - Download the binary package (verify shasum, and asc) and follow
> > the
> > > > > > instructions to run the bookkeeper service.
> > > > > > - Review maven repo, release tag, licenses, and any other things
> > you
> > > > > think
> > > > > > it is important to a release.
> > > > > >
> > > > > > The vote will be open for at least 72 hours. It is adopted by
> > > majority
> > > > > > approval, with at least 3 PMC affirmative votes.
> > > > > >
> > > > > > Thanks,
> > > > > > Release Manager
> > > > > >
> > > > > > [1]
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> > https://github.com/apache/bookkeeper/pull/3500/files#diff-d361215d5d5abaa96ac3362e70a1d2f80f92d369a59c866e1976ff41ec435f53R4
> > > > > > [2]
> > > > > >
> > > >
> > https://dist.apache.org/repos/dist/dev/bookkeeper/bookkeeper-4.15.2-rc0/
> > > > > > [3]
> > > > > >
> > > > >
> > > >
> > >
> > https://repository.apache.org/content/repositories/orgapachebookkeeper-1073
> > > > > > [4] https://github.com/apache/bookkeeper/releases/tag/v4.15.2-rc0
> > > > > > [5] 3c9e4f66ab77a408fe208a3d757929527ff7bc0f
> > > > > >
> > > > >
> > > >
> > >
> >

Re: [DISCUSS] End-Of-Life for some release lines

[Hang Chen]

I Agree.

> I propose to declare end-of-life on all the release lines prior to 4.14.

I think we should have a clear end-of-life policy for BookKeeper.

> For me having two release lines is acceptable.

We'd better not define the end-of-life policy based on the release lines number.

Thanks,
Hang

Yong Zhang  于2022年9月28日周三 09:05写道：
>
> Agree.
>
> >In my opinion we should formalize this process in order to have a clear
> vision when 4.16.0 will be out.
> This will also help users to understand the entire lifecycle of the
> project.
> For me having two release lines is acceptable.
>
> I think that depends on how long we will stay on a major version.
>
> Most of the time, we release the major version to include some new features.
> If we have some new features in a short time, we may have multiple major
> releases. In this case, we need to have more release lines.
>
> how about a fixed timeline?
>
> > Makes sense to me. Is the upgrade path straightforward? For example,
> Apache Pulsar 2.7.5 currently uses release line 4.12, so it would need
> to be upgraded to at least 4.14 if we wanted to stay on a supported
> version.
>
> If we have good compatibility, that should be easy :)
>
> Thanks,
> Yong
>
> On Wed, 28 Sept 2022 at 01:50, Michael Marshall 
> wrote:
>
> > Makes sense to me. Is the upgrade path straightforward? For example,
> > Apache Pulsar 2.7.5 currently uses release line 4.12, so it would need
> > to be upgraded to at least 4.14 if we wanted to stay on a supported
> > version.
> >
> > Thanks,
> > Michael
> >
> >
> > On Tue, Sep 27, 2022 at 3:08 AM steven lu  wrote:
> > >
> > > +1 I agree.
> > >
> > > Nicolò Boschi  于2022年9月27日周二 16:00写道：
> > >
> > > > I agree. And it actually reflects what we're doing in the community
> > (only
> > > > cherry-pick to branches 4.14 and 4.15).
> > > >
> > > > In my opinion we should formalize this process in order to have a clear
> > > > vision when 4.16.0 will be out.
> > > > This will also help users to understand the entire lifecycle of the
> > > > project.
> > > >
> > > > For me having two release lines is acceptable.
> > > >
> > > > Nicolò Boschi
> > > >
> > > >
> > > > Il giorno mar 27 set 2022 alle ore 08:43 Enrico Olivelli <
> > > > eolive...@gmail.com> ha scritto:
> > > >
> > > > > Hello,
> > > > > I propose to declare end-of-life on all the release lines prior to
> > 4.14.
> > > > >
> > > > > We wanted to follow a time-based release plan, but actually we are
> > not
> > > > > following it.
> > > > >
> > > > > This is the doc on our website
> > > > > https://bookkeeper.apache.org/community/releases
> > > > >
> > > > > If nobody objects, I will start a separate thread with a VOTE for
> > this
> > > > > resolution.
> > > > >
> > > > > End-of-life generally means "no more releases" so:
> > > > > - no more bug fixes
> > > > > - no more security fixes
> > > > > - no patches accepted to be cherry-picked to those branches
> > > > >
> > > > > Please chime in with your thoughts
> > > > >
> > > > > Enrico
> > > > >
> > > >
> >

Re: [VOTE] Release 4.15.1, release candidate #3

[Hang Chen]

Hi Yong, thanks for your great work!

+1 (non-binding)

Verified the following list.
- Verified the license and checksum
- Build from the source code with JDK17
- Run bin/bkperf to check journal writing
- Integrate with Pulsar branch-2.11
- build from the source code with BookKeeper 4.15.1 dependency
- start a standalone cluster and run pulsar perf produce and consume.

We'd better keep the tag name sync with previous one, using
`release-4.15.1` instead of `v4.15.1`

Thanks,
Hang

steven lu  于2022年8月31日周三 11:33写道：
>
> +1 (binding)
> deployed and verified in our own test environment, no problems were found
>
> thank you !
>
> Yong Zhang  于2022年8月24日周三 10:41写道：
>
> > Hi everyone,
> > Please review and vote on the release candidate #1 for the version 4.15.1,
> > as follows:
> > [ ] +1, Approve the release
> > [ ] -1, Do not approve the release (please provide specific comments)
> >
> > The complete staging area is available for your review, which includes:
> > * Release notes [1]
> > * The official Apache source and binary distributions to be deployed to
> > dist.apache.org [2]
> > * All artifacts to be deployed to the Maven Central Repository [3]
> > * Source code tag "v4.15.1-rc3" [4] with git sha [5]
> >
> > BookKeeper's KEYS file contains PGP keys we used to sign this release:
> > https://dist.apache.org/repos/dist/release/bookkeeper/KEYS
> >
> > Please download these packages and review this release candidate:
> >
> > - Review release notes
> > - Download the source package (verify shasum, and asc) and follow the
> > instructions to build and run the bookkeeper service.
> > - Download the binary package (verify shasum, and asc) and follow the
> > instructions to run the bookkeeper service.
> > - Review maven repo, release tag, licenses, and any other things you think
> > it is important to a release.
> >
> > The vote will be open for at least 72 hours. It is adopted by majority
> > approval, with at least 3 PMC affirmative votes.
> >
> > Thanks,
> > Release Manager
> >
> > [1]
> >
> > https://github.com/apache/bookkeeper/pull/3440/files#diff-d361215d5d5abaa96ac3362e70a1d2f80f92d369a59c866e1976ff41ec435f53
> > [2]
> > https://dist.apache.org/repos/dist/dev/bookkeeper/bookkeeper-4.15.1-rc3/
> > [3]
> > https://repository.apache.org/content/repositories/orgapachebookkeeper-1072
> > [4] https://github.com/apache/bookkeeper/releases/tag/v4.15.1-rc3
> > [5] 1f930273aa329c9b04a19f27a11417624bc02954
> >

Re: [DISCUSS] Bookkeeper 4.16.0 release

[Hang Chen]

Hi Steven,
Those Prs are in the 4.16.0 milestone, and they are already
merged. They will be included in the 4.16.0 release.  Thanks for your
great work!

Thanks,
Hang

steven lu  于2022年7月28日周四 09:56写道：
>
> I think this index's optimization can release for 4.16.0
>  for details, have a look https://github.com/apache/bookkeeper/issues/3419
>
> Hang Chen  于2022年7月20日周三 10:24写道：
>
> > Andrey, thanks for your noticing.
> >
> > I'm happy to be a volunteer for the version 4.16.0 release.
> > I have created an issue to track the Prs to be included in 4.16.0
> > https://github.com/apache/bookkeeper/issues/3412
> >
> > Feel free to add more PRs if you want them to be included in 4.16.0.
> > If you have time, welcome to help review them.
> >
> > Thanks,
> > Hang
> >
> > Nicolò Boschi  于2022年7月20日周三 01:18写道：
> > >
> > > +1 for 4.16.0
> > >
> > >
> > > > Btw, why https://bookkeeper.apache.org/docs/4.14.5/overview/ says
> > "Apache
> > > > BookKeeper 4.14.5, which is no longer actively maintained"?
> > > > I thought 4.14 was still maintained.
> > >
> > >
> > > Andrey, thanks for noticing. I created a pull for fixing it:
> > > https://github.com/apache/bookkeeper/pull/3411
> > >
> > >
> > > Nicolò Boschi
> > >
> > >
> > > Il giorno mar 19 lug 2022 alle ore 18:43 Andrey Yegorov <
> > > andrey.yego...@datastax.com> ha scritto:
> > >
> > > > +1 for the 4.16 release.
> > > > Hang, did you just volunteer to do the release? :)
> > > >
> > > > We need to do a pass on open PRs to review/merge what we can, there
> > are 100
> > > > open PRs right now.
> > > > I'd love to see at least these in the release:
> > > > https://github.com/apache/bookkeeper/pull/3361
> > > > https://github.com/apache/bookkeeper/pull/3383
> > > > https://github.com/apache/bookkeeper/pull/3384
> > > > https://github.com/apache/bookkeeper/pull/3390
> > > > https://github.com/apache/bookkeeper/pull/3325
> > > > https://github.com/apache/bookkeeper/pull/3329
> > > > https://github.com/apache/bookkeeper/pull/3330
> > > >
> > > > As for the minor releases, 4.15.1 is ok.
> > > > I think we also need 4.14.6; 4.15 was a major change and requires a
> > bit of
> > > > work to upgrade to (https://github.com/apache/pulsar/pull/15142), e.g.
> > > > older Pulsar versions (2.10 and below) could benefit from 4.14 release.
> > > >
> > > > Btw, why https://bookkeeper.apache.org/docs/4.14.5/overview/ says
> > "Apache
> > > > BookKeeper 4.14.5, which is no longer actively maintained"?
> > > > I thought 4.14 was still maintained.
> > > >
> > > > On Fri, Jul 15, 2022 at 12:05 AM Enrico Olivelli 
> > > > wrote:
> > > >
> > > > > Hang,
> > > > >
> > > > > Il giorno ven 15 lug 2022 alle ore 08:25 ZhangJian He
> > > > >  ha scritto:
> > > > > >
> > > > > > Hi, Hang Chen. I hope to get this change into version 4.16.0.
> > > > > > https://github.com/apache/bookkeeper/pull/3406
> > > > > >
> > > > > > Thanks
> > > > > > ZhangJian He
> > > > > >
> > > > > > Hang Chen  于2022年7月15日周五 11:32写道：
> > > > > >
> > > > > > > Hi Community,
> > > > > > >Bookkeeper 4.15.0 has been released for 2 months, and we have
> > > > > > > introduced Direct IO support[1] for ledger and many important bug
> > > > > > > fixes, such as fixing memory leaks [2][3], applying back
> > pressure for
> > > > > > > v2 protocol[4], and so on.
> > > > >
> > > > > for bug fixes we should cut a 4.15.1 release as well
> > > > >
> > > > > I am +1 to cut a release for 4.16.0
> > > > >
> > > > > I would also add this patch (that only needs a second reviewer)
> > > > > https://github.com/apache/bookkeeper/pull/3390
> > > > >
> > > > > Enrico
> > > > >
> > > > >
> > > > > I would like to start to discuss releasing
> > > > > > > 4.16.0.
> > > > > > >
> > > > > > >Do you have any concerns? Do you have any other issue/ticket
> > to
> > > > add
> > > > > > > for 4.16.0?
> > > > > > >
> > > > > > > [1]
> > > > > > >
> > > > >
> > > >
> > https://github.com/apache/bookkeeper/pulls?q=is%3Apr+bp-47+is%3Aclosed+author%3Ahangc0276
> > > > > > > [2] https://github.com/apache/bookkeeper/pull/3321
> > > > > > > [3] https://github.com/apache/bookkeeper/pull/3323
> > > > > > > [4] https://github.com/apache/bookkeeper/pull/3324
> > > > > > >
> > > > >
> > > >
> > > >
> > > > --
> > > > Andrey Yegorov
> > > >
> >

Re: [DISCUSS] Bookkeeper 4.16.0 release

[Hang Chen]

Andrey, thanks for your noticing.

I'm happy to be a volunteer for the version 4.16.0 release.
I have created an issue to track the Prs to be included in 4.16.0
https://github.com/apache/bookkeeper/issues/3412

Feel free to add more PRs if you want them to be included in 4.16.0.
If you have time, welcome to help review them.

Thanks,
Hang

Nicolò Boschi  于2022年7月20日周三 01:18写道：
>
> +1 for 4.16.0
>
>
> > Btw, why https://bookkeeper.apache.org/docs/4.14.5/overview/ says "Apache
> > BookKeeper 4.14.5, which is no longer actively maintained"?
> > I thought 4.14 was still maintained.
>
>
> Andrey, thanks for noticing. I created a pull for fixing it:
> https://github.com/apache/bookkeeper/pull/3411
>
>
> Nicolò Boschi
>
>
> Il giorno mar 19 lug 2022 alle ore 18:43 Andrey Yegorov <
> andrey.yego...@datastax.com> ha scritto:
>
> > +1 for the 4.16 release.
> > Hang, did you just volunteer to do the release? :)
> >
> > We need to do a pass on open PRs to review/merge what we can, there are 100
> > open PRs right now.
> > I'd love to see at least these in the release:
> > https://github.com/apache/bookkeeper/pull/3361
> > https://github.com/apache/bookkeeper/pull/3383
> > https://github.com/apache/bookkeeper/pull/3384
> > https://github.com/apache/bookkeeper/pull/3390
> > https://github.com/apache/bookkeeper/pull/3325
> > https://github.com/apache/bookkeeper/pull/3329
> > https://github.com/apache/bookkeeper/pull/3330
> >
> > As for the minor releases, 4.15.1 is ok.
> > I think we also need 4.14.6; 4.15 was a major change and requires a bit of
> > work to upgrade to (https://github.com/apache/pulsar/pull/15142), e.g.
> > older Pulsar versions (2.10 and below) could benefit from 4.14 release.
> >
> > Btw, why https://bookkeeper.apache.org/docs/4.14.5/overview/ says "Apache
> > BookKeeper 4.14.5, which is no longer actively maintained"?
> > I thought 4.14 was still maintained.
> >
> > On Fri, Jul 15, 2022 at 12:05 AM Enrico Olivelli 
> > wrote:
> >
> > > Hang,
> > >
> > > Il giorno ven 15 lug 2022 alle ore 08:25 ZhangJian He
> > >  ha scritto:
> > > >
> > > > Hi, Hang Chen. I hope to get this change into version 4.16.0.
> > > > https://github.com/apache/bookkeeper/pull/3406
> > > >
> > > > Thanks
> > > > ZhangJian He
> > > >
> > > > Hang Chen  于2022年7月15日周五 11:32写道：
> > > >
> > > > > Hi Community,
> > > > >Bookkeeper 4.15.0 has been released for 2 months, and we have
> > > > > introduced Direct IO support[1] for ledger and many important bug
> > > > > fixes, such as fixing memory leaks [2][3], applying back pressure for
> > > > > v2 protocol[4], and so on.
> > >
> > > for bug fixes we should cut a 4.15.1 release as well
> > >
> > > I am +1 to cut a release for 4.16.0
> > >
> > > I would also add this patch (that only needs a second reviewer)
> > > https://github.com/apache/bookkeeper/pull/3390
> > >
> > > Enrico
> > >
> > >
> > > I would like to start to discuss releasing
> > > > > 4.16.0.
> > > > >
> > > > >Do you have any concerns? Do you have any other issue/ticket to
> > add
> > > > > for 4.16.0?
> > > > >
> > > > > [1]
> > > > >
> > >
> > https://github.com/apache/bookkeeper/pulls?q=is%3Apr+bp-47+is%3Aclosed+author%3Ahangc0276
> > > > > [2] https://github.com/apache/bookkeeper/pull/3321
> > > > > [3] https://github.com/apache/bookkeeper/pull/3323
> > > > > [4] https://github.com/apache/bookkeeper/pull/3324
> > > > >
> > >
> >
> >
> > --
> > Andrey Yegorov
> >

[DISCUSS] Bookkeeper 4.16.0 release

[Hang Chen]

Hi Community,
   Bookkeeper 4.15.0 has been released for 2 months, and we have
introduced Direct IO support[1] for ledger and many important bug
fixes, such as fixing memory leaks [2][3], applying back pressure for
v2 protocol[4], and so on. I would like to start to discuss releasing
4.16.0.

   Do you have any concerns? Do you have any other issue/ticket to add
for 4.16.0?

[1] 
https://github.com/apache/bookkeeper/pulls?q=is%3Apr+bp-47+is%3Aclosed+author%3Ahangc0276
[2] https://github.com/apache/bookkeeper/pull/3321
[3] https://github.com/apache/bookkeeper/pull/3323
[4] https://github.com/apache/bookkeeper/pull/3324

[Discuss] Bookie may lose data even though we turn on fsync for the journal

[Hang Chen]

We found one place where the bookie may lose data even though we turn
on fsync for the journal.
Condition:
- One journal disk, and turn on fsync for the journal
- Configure two ledger disks, ledger1, and ledger2

Assume we write 100MB data into one bookie, 70MB data written into
ledger1's write cache, and 30 MB data written into ledger2's write
cache. Ledger1's write cache is full and triggers flush. In flushing
the write cache, it will trigger a checkpoint to mark the journal’s
lastMark position (100MB’s offset) and write the lastMark position
into both ledger1 and ledger2's lastMark file.

At this time, this bookie shutdown without flush write cache, such as
shutdown by `kill -9` command, and ledger2's write cache (30MB)
doesn’t flush into ledger disk. But ledger2's lastMark position which
persisted into lastMark file has been updated to 100MB’s offset.

When the bookie starts up, the journal reply position will be
`min(ledger1's lastMark, ledger2's lastMark)`, and it will be 100MB’s
offset. The ledger2's 30MB data won’t reply and that data will be
lost.

Please help take a look.  I’m not sure whether I missed some logic.

Thanks,
Hang

Re: [ANNOUNCE] New committer ZhangJian He (shoothzj)

[Hang Chen]

Congrats Zhangjian!

Thanks,
Hang

Andrey Yegorov  于2022年5月4日周三 23:52写道：
>
> Congrats!
>
> On Wed, May 4, 2022 at 5:36 AM Enrico Olivelli  wrote:
>
> > The Project Management Committee (PMC) for Apache BookKeeper
> > has invited  ZhangJian He to become a committer and we are pleased
> > to announce that he has accepted.
> >
> > Being a committer enables easier contribution to the
> > project since there is no need to go via the patch
> > submission process. This should enable better productivity.
> > A PMC member helps manage and guide the direction of the project.
> >
> > In order to  know more about how the ASF works you can check this link:
> > https://www.apache.org/foundation/how-it-works.html
> >
> >
> > Congratulations ZhangJian !
> > Enrico
> >
>
>
> --
> Andrey Yegorov