Re: [docs] Pulsar 3.3 javadoc is in Chinese

[Cong Zhao]

I push pr (https://github.com/apache/pulsar-site/pull/912) to regenerate 
english javadoc for 3.3.0.

On 2024/06/10 16:48:12 Andrey Yegorov wrote:
> Hello,
> 
> Something went wrong with the release Pulsar 3.3 - the JavaDocs are Chinese
> instead of English:
> 
> e.g.
> https://pulsar.apache.org/api/client/3.3.x/org/apache/pulsar/client/api/MessageRouter
> as linked at
> https://pulsar.apache.org/docs/3.3.x/concepts-messaging/#routing-modes
> 
> 3.2's javadocs are ok:
> https://pulsar.apache.org/api/client/3.2.x/org/apache/pulsar/client/api/MessageRouter
> 
> --
> Andrey
> 

Re: [docs] Pulsar 3.3 javadoc is in Chinese

[Cong Zhao]

Thanks for report this issue

I will regenerate and republish javadoc.

On 2024/06/10 19:01:56 Lari Hotari wrote:
> Thanks for reporting this issue.
> 
> I created this commit to the Python script that is used to create the 
> javadocs:
> https://github.com/apache/pulsar-site/commit/cfc7e9d1
> It should prevent this happening in the future by setting the locale property 
> for the javadoc task [1]. This is the step used in the release process to 
> update the javadocs: 
> https://pulsar.apache.org/contribute/release-process/#javadoc
> 
> Cong Zhao, would you be able to regenerate and republish the javadocs for the 
> 3.3.0 release?
> Thanks!
> 
> -Lari
> 
> 1 - 
> https://maven.apache.org/plugins/maven-javadoc-plugin/resource-bundle-mojo.html#locale
> 
> On 2024/06/10 16:48:12 Andrey Yegorov wrote:
> > Hello,
> > 
> > Something went wrong with the release Pulsar 3.3 - the JavaDocs are Chinese
> > instead of English:
> > 
> > e.g.
> > https://pulsar.apache.org/api/client/3.3.x/org/apache/pulsar/client/api/MessageRouter
> > as linked at
> > https://pulsar.apache.org/docs/3.3.x/concepts-messaging/#routing-modes
> > 
> > 3.2's javadocs are ok:
> > https://pulsar.apache.org/api/client/3.2.x/org/apache/pulsar/client/api/MessageRouter
> > 
> > --
> > Andrey
> > 
> 

[ANNOUNCE] Apache Pulsar 3.3.0 released

[Cong Zhao]

The Apache Pulsar team is proud to announce Apache Pulsar version 3.3.0

Pulsar is a highly scalable, low latency messaging platform running on
commodity hardware. It provides simple pub-sub semantics over topics,
guaranteed at-least-once delivery of messages, automatic cursor management
for
subscribers, and cross-datacenter replication.

For Pulsar release details and downloads, visit:
https://pulsar.apache.org/download

Release Notes are at:
https://pulsar.apache.org/release-notes/versioned/pulsar-3.3.0/

We would like to thank the contributors that made the release possible.

Regards,

The Pulsar Team

Re: [VOTE] Release Apache Pulsar 3.3.0 based on 3.3.0-candidate-4

[Cong Zhao]

Hello all,

The vote to release Apache Pulsar version 3.3.0 based on 3.3.0-candidate-4 is 
now closed.

The vote PASSED with 4 binding "+1", 1 non-binding "+1" and 0 "-1" votes:

"+1" Binding votes:

- Lari Hotari
- Guo jiwei
- Yunze xu
- Zike Yang

"+1" non-binding votes:

- Andrey Yegorov

I'll continue with the release process and the release announcement will follow 
shortly.

Thanks,
Cong

On 2024/06/03 10:51:46 Cong Zhao wrote:
> Hello Apache Pulsar Community,
> 
> This is a call for the vote to release the Apache Pulsar version 3.3.0 based 
> on 3.3.0-candidate-4.
> 
> Included changes since the previous release:
> https://github.com/apache/pulsar/milestone/38?closed=1
> 
> *** Please download, test and vote on this release. This vote will stay open
> for at least 72 hours ***
> 
> Only votes from PMC members are binding, but members of the community are
> encouraged to test the release and vote with "(non-binding)".
> 
> Note that we are voting upon the source (tag), binaries are provided for
> convenience.
> 
> The release candidate is available at:
> https://dist.apache.org/repos/dist/dev/pulsar/pulsar-3.3.0-candidate-4/
> 
> SHA-512 checksums:
> a19f80caa7dcc92d6df235af9f71e90d9ab52739e2d7d1ddc59944ed11be8f5f6a083db4792cafd87802d68ef9edce88e1ed91ee43cac2a98c3782b8925f07f6
> 03692f2b6d574ce523418850d47391b9aa9f71a2bde0b593d6e1eaaf34eb6ebe080e598231d6faf2902a9f071192c0a6982403095a9374d4357b7bebb18e5e93
> 
> Maven staging repo:
> https://repository.apache.org/content/repositories/orgapachepulsar-1300
> 
> The tag to be voted upon:
> v3.3.0-candidate-4 (commit c0aab493aafa5386dbf93c0b58a66a666aeba17a)
> https://github.com/apache/pulsar/releases/tag/v3.3.0-candidate-4
> 
> Pulsar's KEYS file containing PGP keys you use to sign the release:
> https://downloads.apache.org/pulsar/KEYS
> 
> Docker images:
> docker pull czcoder/pulsar3.3.0-c0aab49
> https://hub.docker.com/layers/czcoder/pulsar/3.3.0-c0aab49/images/sha256-996c6babec1f887cf2f43cac0692e06f879364fe87372fcb6c8b9d3ac779c7d6?context=explore
> docker pull czcoder/pulsar-all:3.3.0-c0aab49
> https://hub.docker.com/layers/czcoder/pulsar-all/3.3.0-c0aab49/images/sha256-dd781dfcc72dfb3db6c0053c656dfdb750c8d36e4c57f4bbad87aee4a9cdce28?context=explore
> 
> Please download the source package, and follow the README to build
> and run the Pulsar standalone service.
> 
> More advanced release validation instructions can be found at
> https://pulsar.apache.org/contribute/validate-release-candidate/
> 
> Thanks,
> Cong Zhao
> 

[VOTE] Release Apache Pulsar 3.3.0 based on 3.3.0-candidate-4

[Cong Zhao]

Hello Apache Pulsar Community,

This is a call for the vote to release the Apache Pulsar version 3.3.0 based on 
3.3.0-candidate-4.

Included changes since the previous release:
https://github.com/apache/pulsar/milestone/38?closed=1

*** Please download, test and vote on this release. This vote will stay open
for at least 72 hours ***

Only votes from PMC members are binding, but members of the community are
encouraged to test the release and vote with "(non-binding)".

Note that we are voting upon the source (tag), binaries are provided for
convenience.

The release candidate is available at:
https://dist.apache.org/repos/dist/dev/pulsar/pulsar-3.3.0-candidate-4/

SHA-512 checksums:
a19f80caa7dcc92d6df235af9f71e90d9ab52739e2d7d1ddc59944ed11be8f5f6a083db4792cafd87802d68ef9edce88e1ed91ee43cac2a98c3782b8925f07f6
03692f2b6d574ce523418850d47391b9aa9f71a2bde0b593d6e1eaaf34eb6ebe080e598231d6faf2902a9f071192c0a6982403095a9374d4357b7bebb18e5e93

Maven staging repo:
https://repository.apache.org/content/repositories/orgapachepulsar-1300

The tag to be voted upon:
v3.3.0-candidate-4 (commit c0aab493aafa5386dbf93c0b58a66a666aeba17a)
https://github.com/apache/pulsar/releases/tag/v3.3.0-candidate-4

Pulsar's KEYS file containing PGP keys you use to sign the release:
https://downloads.apache.org/pulsar/KEYS

Docker images:
docker pull czcoder/pulsar3.3.0-c0aab49
https://hub.docker.com/layers/czcoder/pulsar/3.3.0-c0aab49/images/sha256-996c6babec1f887cf2f43cac0692e06f879364fe87372fcb6c8b9d3ac779c7d6?context=explore
docker pull czcoder/pulsar-all:3.3.0-c0aab49
https://hub.docker.com/layers/czcoder/pulsar-all/3.3.0-c0aab49/images/sha256-dd781dfcc72dfb3db6c0053c656dfdb750c8d36e4c57f4bbad87aee4a9cdce28?context=explore

Please download the source package, and follow the README to build
and run the Pulsar standalone service.

More advanced release validation instructions can be found at
https://pulsar.apache.org/contribute/validate-release-candidate/

Thanks,
Cong Zhao

Re: [VOTE] Release Apache Pulsar 3.3.0 based on 3.3.0-candidate-3

[Cong Zhao]

Hello everyone,

Due to https://github.com/apache/pulsar/pull/22804 fix the snappy load issue, 
so I will build 3.3.0-candidate-4 as soon as possible.

On 2024/05/29 02:16:59 Cong Zhao wrote:
> Hello all,
> 
> The vote to release Apache Pulsar version 3.3.0 based on 3.3.0-candidate-3 is 
> now closed.
> 
> The vote PASSED with 3 binding "+1", 0 non-binding "+1" and 0 "-1" votes:
> 
> "+1" Binding votes:
> 
> - Jiwei Guo
> - PengHui Li
> - Baodi Shi
> 
> I'll continue with the release process and the release announcement will 
> follow shortly.
> 
> Thanks,
> Cong
> 
> On 2024/05/24 02:28:22 Cong Zhao wrote:
> > Hello Apache Pulsar Community,
> > 
> > This is a call for the vote to release the Apache Pulsar version 3.3.0 
> > based on 3.3.0-candidate-3.
> > 
> > Included changes since the previous release:
> > https://github.com/apache/pulsar/milestone/38?closed=1
> > 
> > *** Please download, test and vote on this release. This vote will stay open
> > for at least 72 hours ***
> > 
> > Only votes from PMC members are binding, but members of the community are
> > encouraged to test the release and vote with "(non-binding)".
> > 
> > Note that we are voting upon the source (tag), binaries are provided for
> > convenience.
> > 
> > The release candidate is available at:
> > https://dist.apache.org/repos/dist/dev/pulsar/pulsar-3.3.0-candidate-3/
> > 
> > SHA-512 checksums:
> > 59fcf6e77ef47ea88696bc1a0f67dc09f0d3f2d791d23e09ff721678beb5512f58bbfb328bbb2980981243aff1aeda686c332b4d8c57dc0c4bffc0ec0a4dbb4d
> > b35fb4d9d20111f55b7d9c7016871daab0fe8edb2f09220e013fc51b2d828c482a05a7b19b3b1cd39bd2072162f77c3279722c9fdf3e876f22e90e2aea10c9e3
> > 
> > Maven staging repo:
> > https://repository.apache.org/content/repositories/orgapachepulsar-1297
> > 
> > The tag to be voted upon:
> > v3.3.0-candidate-3 (commit 0771f818ba74f94b95cd0987997079d0f3e73f94)
> > https://github.com/apache/pulsar/releases/tag/v3.3.0-candidate-3
> > 
> > Pulsar's KEYS file containing PGP keys you use to sign the release:
> > https://downloads.apache.org/pulsar/KEYS
> > 
> > Docker images:
> > docker pull czcoder/pulsar:3.3.0-0771f81
> > https://hub.docker.com/layers/czcoder/pulsar/3.3.0-0771f81/images/sha256-e34f1e83bb2a1e8233a3429847bb8fc2cdac0f558a0e5c13658c28773287b07c?context=explore
> > docker pull czcoder/pulsar-all:3.3.0-0771f81
> > https://hub.docker.com/layers/czcoder/pulsar-all/3.3.0-0771f81/images/sha256-49cf20acab7e71c8916cf594690c86eaaaf8426c8a13013a8c25944338d7be7c?context=explore
> > 
> > Please download the source package, and follow the README to build
> > and run the Pulsar standalone service.
> > 
> > More advanced release validation instructions can be found at
> > https://pulsar.apache.org/contribute/validate-release-candidate/
> > 
> > Thanks,
> > Cong Zhao
> > 
> 

Re: [VOTE] Release Apache Pulsar 3.3.0 based on 3.3.0-candidate-3

[Cong Zhao]

Hello all,

The vote to release Apache Pulsar version 3.3.0 based on 3.3.0-candidate-3 is 
now closed.

The vote PASSED with 3 binding "+1", 0 non-binding "+1" and 0 "-1" votes:

"+1" Binding votes:

- Jiwei Guo
- PengHui Li
- Baodi Shi

I'll continue with the release process and the release announcement will follow 
shortly.

Thanks,
Cong

On 2024/05/24 02:28:22 Cong Zhao wrote:
> Hello Apache Pulsar Community,
> 
> This is a call for the vote to release the Apache Pulsar version 3.3.0 based 
> on 3.3.0-candidate-3.
> 
> Included changes since the previous release:
> https://github.com/apache/pulsar/milestone/38?closed=1
> 
> *** Please download, test and vote on this release. This vote will stay open
> for at least 72 hours ***
> 
> Only votes from PMC members are binding, but members of the community are
> encouraged to test the release and vote with "(non-binding)".
> 
> Note that we are voting upon the source (tag), binaries are provided for
> convenience.
> 
> The release candidate is available at:
> https://dist.apache.org/repos/dist/dev/pulsar/pulsar-3.3.0-candidate-3/
> 
> SHA-512 checksums:
> 59fcf6e77ef47ea88696bc1a0f67dc09f0d3f2d791d23e09ff721678beb5512f58bbfb328bbb2980981243aff1aeda686c332b4d8c57dc0c4bffc0ec0a4dbb4d
> b35fb4d9d20111f55b7d9c7016871daab0fe8edb2f09220e013fc51b2d828c482a05a7b19b3b1cd39bd2072162f77c3279722c9fdf3e876f22e90e2aea10c9e3
> 
> Maven staging repo:
> https://repository.apache.org/content/repositories/orgapachepulsar-1297
> 
> The tag to be voted upon:
> v3.3.0-candidate-3 (commit 0771f818ba74f94b95cd0987997079d0f3e73f94)
> https://github.com/apache/pulsar/releases/tag/v3.3.0-candidate-3
> 
> Pulsar's KEYS file containing PGP keys you use to sign the release:
> https://downloads.apache.org/pulsar/KEYS
> 
> Docker images:
> docker pull czcoder/pulsar:3.3.0-0771f81
> https://hub.docker.com/layers/czcoder/pulsar/3.3.0-0771f81/images/sha256-e34f1e83bb2a1e8233a3429847bb8fc2cdac0f558a0e5c13658c28773287b07c?context=explore
> docker pull czcoder/pulsar-all:3.3.0-0771f81
> https://hub.docker.com/layers/czcoder/pulsar-all/3.3.0-0771f81/images/sha256-49cf20acab7e71c8916cf594690c86eaaaf8426c8a13013a8c25944338d7be7c?context=explore
> 
> Please download the source package, and follow the README to build
> and run the Pulsar standalone service.
> 
> More advanced release validation instructions can be found at
> https://pulsar.apache.org/contribute/validate-release-candidate/
> 
> Thanks,
> Cong Zhao
> 

Re: [VOTE] Release Apache Pulsar 3.3.0 based on 3.3.0-candidate-3

[Cong Zhao]

Thanks Yong,

Yes, we can continue to release 3.3 and fix this issue in later versions.

On 2024/05/27 03:10:18 Yong Zhang wrote:
> If the snappy in zookeeper is not used very commonly, this might not be a
> blocker for this release.
> 
> So I would be -0.
> 
> Thanks,
> Yong
> On Fri, 24 May 2024 at 15:17, Yong Zhang  wrote:
> 
> > -1
> >
> > There is an issue with the snappy usage in the amd64-based image.
> >
> > It will get the error
> >
> > java.lang.UnsatisfiedLinkError: /tmp/
> > snappy-1.1.10-b0757287-8557-44b9-9499-afca52f102ec-libsnappyjava.so:
> > Error relocating /lib/ld-linux-x86-64.so.2: unsupported relocation type 37.
> >
> > Reproduce steps:
> > 1. docker run -it -u root --platform=linux/amd64
> > czcoder/pulsar:3.3.0-0771f81 bash
> > 2. export
> > PULSAR_EXTRA_OPTS="-Dzookeeper.snapshot.compression.method=snappy"
> > 3. export PULSAR_STANDALONE_USE_ZOOKEEPER=true
> > 4. bin/pulsar standalone -nss -nfw
> >
> > Then you will get the error. More information:
> > https://github.com/sgerrand/alpine-pkg-glibc/issues/194
> >
> > we need to resolve it before publishing the new release.
> >
> > Yong
> >
> > On Fri, 24 May 2024 at 13:00, guo jiwei  wrote:
> >
> >> +1 (binding)
> >>
> >> - Built from source
> >> - Checked the signatures
> >> - Run standalone
> >> - Checked producer and consumer
> >> - Verified the Cassandra connector
> >> - Verified the Stateful function
> >>
> >>
> >> Regards
> >> Jiwei Guo (Tboy)
> >>
> >>
> >> On Fri, May 24, 2024 at 10:28 AM Cong Zhao  wrote:
> >>
> >> > Hello Apache Pulsar Community,
> >> >
> >> > This is a call for the vote to release the Apache Pulsar version 3.3.0
> >> > based on 3.3.0-candidate-3.
> >> >
> >> > Included changes since the previous release:
> >> > https://github.com/apache/pulsar/milestone/38?closed=1
> >> >
> >> > *** Please download, test and vote on this release. This vote will stay
> >> > open
> >> > for at least 72 hours ***
> >> >
> >> > Only votes from PMC members are binding, but members of the community
> >> are
> >> > encouraged to test the release and vote with "(non-binding)".
> >> >
> >> > Note that we are voting upon the source (tag), binaries are provided for
> >> > convenience.
> >> >
> >> > The release candidate is available at:
> >> > https://dist.apache.org/repos/dist/dev/pulsar/pulsar-3.3.0-candidate-3/
> >> >
> >> > SHA-512 checksums:
> >> >
> >> >
> >> 59fcf6e77ef47ea88696bc1a0f67dc09f0d3f2d791d23e09ff721678beb5512f58bbfb328bbb2980981243aff1aeda686c332b4d8c57dc0c4bffc0ec0a4dbb4d
> >> >
> >> >
> >> b35fb4d9d20111f55b7d9c7016871daab0fe8edb2f09220e013fc51b2d828c482a05a7b19b3b1cd39bd2072162f77c3279722c9fdf3e876f22e90e2aea10c9e3
> >> >
> >> > Maven staging repo:
> >> > https://repository.apache.org/content/repositories/orgapachepulsar-1297
> >> >
> >> > The tag to be voted upon:
> >> > v3.3.0-candidate-3 (commit 0771f818ba74f94b95cd0987997079d0f3e73f94)
> >> > https://github.com/apache/pulsar/releases/tag/v3.3.0-candidate-3
> >> >
> >> > Pulsar's KEYS file containing PGP keys you use to sign the release:
> >> > https://downloads.apache.org/pulsar/KEYS
> >> >
> >> > Docker images:
> >> > docker pull czcoder/pulsar:3.3.0-0771f81
> >> >
> >> >
> >> https://hub.docker.com/layers/czcoder/pulsar/3.3.0-0771f81/images/sha256-e34f1e83bb2a1e8233a3429847bb8fc2cdac0f558a0e5c13658c28773287b07c?context=explore
> >> > docker pull czcoder/pulsar-all:3.3.0-0771f81
> >> >
> >> >
> >> https://hub.docker.com/layers/czcoder/pulsar-all/3.3.0-0771f81/images/sha256-49cf20acab7e71c8916cf594690c86eaaaf8426c8a13013a8c25944338d7be7c?context=explore
> >> >
> >> > Please download the source package, and follow the README to build
> >> > and run the Pulsar standalone service.
> >> >
> >> > More advanced release validation instructions can be found at
> >> > https://pulsar.apache.org/contribute/validate-release-candidate/
> >> >
> >> > Thanks,
> >> > Cong Zhao
> >> >
> >>
> >
> 

[VOTE] Release Apache Pulsar 3.3.0 based on 3.3.0-candidate-3

[Cong Zhao]

Hello Apache Pulsar Community,

This is a call for the vote to release the Apache Pulsar version 3.3.0 based on 
3.3.0-candidate-3.

Included changes since the previous release:
https://github.com/apache/pulsar/milestone/38?closed=1

*** Please download, test and vote on this release. This vote will stay open
for at least 72 hours ***

Only votes from PMC members are binding, but members of the community are
encouraged to test the release and vote with "(non-binding)".

Note that we are voting upon the source (tag), binaries are provided for
convenience.

The release candidate is available at:
https://dist.apache.org/repos/dist/dev/pulsar/pulsar-3.3.0-candidate-3/

SHA-512 checksums:
59fcf6e77ef47ea88696bc1a0f67dc09f0d3f2d791d23e09ff721678beb5512f58bbfb328bbb2980981243aff1aeda686c332b4d8c57dc0c4bffc0ec0a4dbb4d
b35fb4d9d20111f55b7d9c7016871daab0fe8edb2f09220e013fc51b2d828c482a05a7b19b3b1cd39bd2072162f77c3279722c9fdf3e876f22e90e2aea10c9e3

Maven staging repo:
https://repository.apache.org/content/repositories/orgapachepulsar-1297

The tag to be voted upon:
v3.3.0-candidate-3 (commit 0771f818ba74f94b95cd0987997079d0f3e73f94)
https://github.com/apache/pulsar/releases/tag/v3.3.0-candidate-3

Pulsar's KEYS file containing PGP keys you use to sign the release:
https://downloads.apache.org/pulsar/KEYS

Docker images:
docker pull czcoder/pulsar:3.3.0-0771f81
https://hub.docker.com/layers/czcoder/pulsar/3.3.0-0771f81/images/sha256-e34f1e83bb2a1e8233a3429847bb8fc2cdac0f558a0e5c13658c28773287b07c?context=explore
docker pull czcoder/pulsar-all:3.3.0-0771f81
https://hub.docker.com/layers/czcoder/pulsar-all/3.3.0-0771f81/images/sha256-49cf20acab7e71c8916cf594690c86eaaaf8426c8a13013a8c25944338d7be7c?context=explore

Please download the source package, and follow the README to build
and run the Pulsar standalone service.

More advanced release validation instructions can be found at
https://pulsar.apache.org/contribute/validate-release-candidate/

Thanks,
Cong Zhao

Re: [VOTE] Release Apache Pulsar 3.3.0 based on 3.3.0-candidate-2

[Cong Zhao]

Thanks Jiwei,

https://github.com/apache/pulsar/pull/22764 will fix this issue, I will build 
candidate 3 after pr is merged.

On 2024/05/23 06:53:25 Guo Jiwei wrote:
> -1 (binding)
> 
> When verifying the `Functions` parts, it can't get the functions even if 
> created them.
> The broker logs show that creating fn using the `default` namespace, but 
> geting fn using the `test-namespace`. Then find the related fix in 
> https://github.com/apache/pulsar/pull/22209, it adds the default tenant and 
> namespace.
> 
> 
> 
> 
> On 2024/05/20 04:25:10 Cong Zhao wrote:
> > Hello Apache Pulsar Community,
> > 
> > This is a call for the vote to release the Apache Pulsar version 3.3.0 
> > based on 3.3.0-candidate-2.
> > 
> > Included changes since the previous release:
> > https://github.com/apache/pulsar/milestone/38?closed=1
> > 
> > *** Please download, test and vote on this release. This vote will stay open
> > for at least 72 hours ***
> > 
> > Only votes from PMC members are binding, but members of the community are
> > encouraged to test the release and vote with "(non-binding)".
> > 
> > Note that we are voting upon the source (tag), binaries are provided for
> > convenience.
> > 
> > The release candidate is available at:
> > https://dist.apache.org/repos/dist/dev/pulsar/pulsar-3.3.0-candidate-2/
> > 
> > SHA-512 checksums:
> > 88a0e66f164308613f0d6d0f5fe84e6b98a901dc3a8a5bf932b385f66548e5de6d39b2fe65bca5c661ca141e024450f0d85f9d7e2f8c2e21508fb95e7c09129a
> > a6b473d63b673ef9818ac62a6a98fa1ac9c11c084a4c713d7659a489dedb8a1126f3c9bd18b7250ea8943e51392d116c4bb398bf60cd52f7c871c67a00992d31
> > 
> > Maven staging repo:
> > https://repository.apache.org/content/repositories/orgapachepulsar-1296
> > 
> > The tag to be voted upon:
> > v3.3.0-candidate-2 (commit 55ed82346b804f24a0aa2185618bafa73f5a394b)
> > https://github.com/apache/pulsar/releases/tag/v3.3.0-candidate-2
> > 
> > Pulsar's KEYS file containing PGP keys you use to sign the release:
> > https://downloads.apache.org/pulsar/KEYS
> > 
> > Docker images:
> > docker pull czcoder/pulsar:3.3.0-55ed823
> > https://hub.docker.com/layers/czcoder/pulsar/3.3.0-55ed823/images/sha256-12928b392d55c19eca547b27fddbe8f1f90fc89cca3f3a5bf99d5bfa27239964?context=explore
> > docker pull czcoder/pulsar-all:3.3.0-55ed823
> > https://hub.docker.com/layers/czcoder/pulsar-all/3.3.0-55ed823/images/sha256-2bb78e60e88d5fabc5956b5aa07b6f62724c548be7cdfdc35eccad7ac0cc2f9b?context=explore
> > 
> > Please download the source package, and follow the README to build
> > and run the Pulsar standalone service.
> > 
> > More advanced release validation instructions can be found at
> > https://pulsar.apache.org/contribute/validate-release-candidate/
> > 
> > Thanks,
> > 
> > Cong Zhao
> > 
> 

Re: Release Apache Pulsar 3.3.0 based on 3.3.0-candidate-2

[Cong Zhao]

The email title is missing VOTE, please ignore this email.

On 2024/05/20 04:19:54 Cong Zhao wrote:
> Hello Apache Pulsar Community,
> 
> This is a call for the vote to release the Apache Pulsar version 3.3.0 based 
> on 3.3.0-candidate-2.
> 
> Included changes since the previous release:
> https://github.com/apache/pulsar/milestone/38?closed=1
> 
> *** Please download, test and vote on this release. This vote will stay open
> for at least 72 hours ***
> 
> Only votes from PMC members are binding, but members of the community are
> encouraged to test the release and vote with "(non-binding)".
> 
> Note that we are voting upon the source (tag), binaries are provided for
> convenience.
> 
> The release candidate is available at:
> https://dist.apache.org/repos/dist/dev/pulsar/pulsar-3.3.0-candidate-2/
> 
> SHA-512 checksums:
> 88a0e66f164308613f0d6d0f5fe84e6b98a901dc3a8a5bf932b385f66548e5de6d39b2fe65bca5c661ca141e024450f0d85f9d7e2f8c2e21508fb95e7c09129a
> a6b473d63b673ef9818ac62a6a98fa1ac9c11c084a4c713d7659a489dedb8a1126f3c9bd18b7250ea8943e51392d116c4bb398bf60cd52f7c871c67a00992d31
> 
> Maven staging repo:
> https://repository.apache.org/content/repositories/orgapachepulsar-1296
> 
> The tag to be voted upon:
> v3.3.0-candidate-2 (commit 55ed82346b804f24a0aa2185618bafa73f5a394b)
> https://github.com/apache/pulsar/releases/tag/v3.3.0-candidate-2
> 
> Pulsar's KEYS file containing PGP keys you use to sign the release:
> https://downloads.apache.org/pulsar/KEYS
> 
> Docker images:
> docker pull czcoder/pulsar:3.3.0-55ed823
> https://hub.docker.com/layers/czcoder/pulsar/3.3.0-55ed823/images/sha256-12928b392d55c19eca547b27fddbe8f1f90fc89cca3f3a5bf99d5bfa27239964?context=explore
> docker pull czcoder/pulsar-all:3.3.0-55ed823
> https://hub.docker.com/layers/czcoder/pulsar-all/3.3.0-55ed823/images/sha256-2bb78e60e88d5fabc5956b5aa07b6f62724c548be7cdfdc35eccad7ac0cc2f9b?context=explore
> 
> Please download the source package, and follow the README to build
> and run the Pulsar standalone service.
> 
> More advanced release validation instructions can be found at
> https://pulsar.apache.org/contribute/validate-release-candidate/
> 
> Thanks,
> 
> Cong Zhao
> 

[VOTE] Release Apache Pulsar 3.3.0 based on 3.3.0-candidate-2

[Cong Zhao]

Hello Apache Pulsar Community,

This is a call for the vote to release the Apache Pulsar version 3.3.0 based on 
3.3.0-candidate-2.

Included changes since the previous release:
https://github.com/apache/pulsar/milestone/38?closed=1

*** Please download, test and vote on this release. This vote will stay open
for at least 72 hours ***

Only votes from PMC members are binding, but members of the community are
encouraged to test the release and vote with "(non-binding)".

Note that we are voting upon the source (tag), binaries are provided for
convenience.

The release candidate is available at:
https://dist.apache.org/repos/dist/dev/pulsar/pulsar-3.3.0-candidate-2/

SHA-512 checksums:
88a0e66f164308613f0d6d0f5fe84e6b98a901dc3a8a5bf932b385f66548e5de6d39b2fe65bca5c661ca141e024450f0d85f9d7e2f8c2e21508fb95e7c09129a
a6b473d63b673ef9818ac62a6a98fa1ac9c11c084a4c713d7659a489dedb8a1126f3c9bd18b7250ea8943e51392d116c4bb398bf60cd52f7c871c67a00992d31

Maven staging repo:
https://repository.apache.org/content/repositories/orgapachepulsar-1296

The tag to be voted upon:
v3.3.0-candidate-2 (commit 55ed82346b804f24a0aa2185618bafa73f5a394b)
https://github.com/apache/pulsar/releases/tag/v3.3.0-candidate-2

Pulsar's KEYS file containing PGP keys you use to sign the release:
https://downloads.apache.org/pulsar/KEYS

Docker images:
docker pull czcoder/pulsar:3.3.0-55ed823
https://hub.docker.com/layers/czcoder/pulsar/3.3.0-55ed823/images/sha256-12928b392d55c19eca547b27fddbe8f1f90fc89cca3f3a5bf99d5bfa27239964?context=explore
docker pull czcoder/pulsar-all:3.3.0-55ed823
https://hub.docker.com/layers/czcoder/pulsar-all/3.3.0-55ed823/images/sha256-2bb78e60e88d5fabc5956b5aa07b6f62724c548be7cdfdc35eccad7ac0cc2f9b?context=explore

Please download the source package, and follow the README to build
and run the Pulsar standalone service.

More advanced release validation instructions can be found at
https://pulsar.apache.org/contribute/validate-release-candidate/

Thanks,

Cong Zhao

Release Apache Pulsar 3.3.0 based on 3.3.0-candidate-2

[Cong Zhao]

Hello Apache Pulsar Community,

This is a call for the vote to release the Apache Pulsar version 3.3.0 based on 
3.3.0-candidate-2.

Included changes since the previous release:
https://github.com/apache/pulsar/milestone/38?closed=1

*** Please download, test and vote on this release. This vote will stay open
for at least 72 hours ***

Only votes from PMC members are binding, but members of the community are
encouraged to test the release and vote with "(non-binding)".

Note that we are voting upon the source (tag), binaries are provided for
convenience.

The release candidate is available at:
https://dist.apache.org/repos/dist/dev/pulsar/pulsar-3.3.0-candidate-2/

SHA-512 checksums:
88a0e66f164308613f0d6d0f5fe84e6b98a901dc3a8a5bf932b385f66548e5de6d39b2fe65bca5c661ca141e024450f0d85f9d7e2f8c2e21508fb95e7c09129a
a6b473d63b673ef9818ac62a6a98fa1ac9c11c084a4c713d7659a489dedb8a1126f3c9bd18b7250ea8943e51392d116c4bb398bf60cd52f7c871c67a00992d31

Maven staging repo:
https://repository.apache.org/content/repositories/orgapachepulsar-1296

The tag to be voted upon:
v3.3.0-candidate-2 (commit 55ed82346b804f24a0aa2185618bafa73f5a394b)
https://github.com/apache/pulsar/releases/tag/v3.3.0-candidate-2

Pulsar's KEYS file containing PGP keys you use to sign the release:
https://downloads.apache.org/pulsar/KEYS

Docker images:
docker pull czcoder/pulsar:3.3.0-55ed823
https://hub.docker.com/layers/czcoder/pulsar/3.3.0-55ed823/images/sha256-12928b392d55c19eca547b27fddbe8f1f90fc89cca3f3a5bf99d5bfa27239964?context=explore
docker pull czcoder/pulsar-all:3.3.0-55ed823
https://hub.docker.com/layers/czcoder/pulsar-all/3.3.0-55ed823/images/sha256-2bb78e60e88d5fabc5956b5aa07b6f62724c548be7cdfdc35eccad7ac0cc2f9b?context=explore

Please download the source package, and follow the README to build
and run the Pulsar standalone service.

More advanced release validation instructions can be found at
https://pulsar.apache.org/contribute/validate-release-candidate/

Thanks,

Cong Zhao

Re: [VERIFY] Pulsar Release 3.3.0 Candidate 1

[Cong Zhao]

I found the reason, the curl command is missing on the new image, I will add 
the curl command to the image because many components need to rely on curl when 
testing.

In addition, there are some other issues with the image that need to be fixed.
https://github.com/apache/pulsar/pull/22730、https://github.com/apache/pulsar/pull/22731

I will build 3.3.0 Candidate 2 after the fix PRs are merged.

Regards
Cong Zhao

On 2024/05/17 02:53:28 Yunze Xu wrote:
> -1 (binding)
> 
> There seems something wrong with the docker image, see
> https://github.com/BewareMyPower/pulsar-client-cpp/actions/runs/9121814458/job/25082141665?pr=31
> 
> After upgrading the dependency, the Pulsar standalone in CI of
> pulsar-client-cpp cannot be started (always stuck in waiting for the
> metrics ready).
> 
> Thanks,
> Yunze
> 
> On Fri, May 17, 2024 at 12:06 AM Cong Zhao  wrote:
> >
> > This is the first release candidate for Apache Pulsar version 3.3.0.
> >
> > It fixes the following issues:
> > https://github.com/apache/pulsar/milestone/38?closed=1
> >
> > *** Please download, test and verify on this release. This release
> > candidate verification will stay open until May 22 ***
> >
> > Note that we are verifying upon the source (tag), binaries are provided for
> > convenience.
> >
> > Source and binary files:
> > https://dist.apache.org/repos/dist/dev/pulsar/pulsar-3.3.0-candidate-1/
> >
> > SHA-512 checksums:
> >
> > 52a29b59b53e15f295acb88afdf8c1ad9a18fc3befef61310565a527df1991a96b54d570a13cba2b9b390c7119b4b14d85cfd519c179c4c138c6e35276902c5f
> >
> > apache-pulsar-3.3.0-bin.tar.gz
> >
> > 5e9752a03a85672374759d19c75f37d44e52ef77912f0e4169f7995abce9fa204e0ce79875bb8055f6e4588eca451da8344ee83e31335488b2fcc464b7b34391
> >
> > apache-pulsar-3.3.0-src.tar.gz
> >
> > Maven staging repo:
> > https://repository.apache.org/content/repositories/orgapachepulsar-1293/
> >
> > The tag to verify:
> > v3.3.0-candidate-1 (3c3ce71c13a9cc0982089e7d87d97bc89a4a26c2)
> > https://github.com/apache/pulsar/commits/v3.3.0-candidate-1/
> >
> > Pulsar's KEYS file containing PGP keys you use to sign the release:
> > https://dist.apache.org/repos/dist/dev/pulsar/KEYS
> >
> > Docker images:
> >
> > pulsar images:
> > docker pull czcoder/pulsar:3.3.0-3c3ce71
> > https://hub.docker.com/layers/czcoder/pulsar/3.3.0-3c3ce71/images/sha256-ce6ab616eca8b0679a9d0cec50578dc744d7091d305cf4d2af8f533fe4e38641?context=explore
> >
> > pulsar-all images:
> > docekr pull czcoder/pulsar-all:3.3.0-3c3ce71
> > https://hub.docker.com/layers/czcoder/pulsar-all/3.3.0-3c3ce71/images/sha256-bd8b6bb3fbbacdf4d21b2a70cd3b4be7f0de839d1649652eda005f3ed46b78fd?context=explore
> >
> >
> > Please download the source package, and follow the README to build
> > and run the Pulsar standalone service.
> >
> > Note that this RC doesn't require a formal vote, but we would also
> > appreciate your feedback with +1/-1. And please provide specific
> > comments if your feedback is not +1.
> >
> >
> > Regards
> > Cong Zhao
> 

[VERIFY] Pulsar Release 3.3.0 Candidate 1

[Cong Zhao]

This is the first release candidate for Apache Pulsar version 3.3.0.

It fixes the following issues:
https://github.com/apache/pulsar/milestone/38?closed=1

*** Please download, test and verify on this release. This release
candidate verification will stay open until May 22 ***

Note that we are verifying upon the source (tag), binaries are provided for
convenience.

Source and binary files:
https://dist.apache.org/repos/dist/dev/pulsar/pulsar-3.3.0-candidate-1/

SHA-512 checksums:

52a29b59b53e15f295acb88afdf8c1ad9a18fc3befef61310565a527df1991a96b54d570a13cba2b9b390c7119b4b14d85cfd519c179c4c138c6e35276902c5f

apache-pulsar-3.3.0-bin.tar.gz

5e9752a03a85672374759d19c75f37d44e52ef77912f0e4169f7995abce9fa204e0ce79875bb8055f6e4588eca451da8344ee83e31335488b2fcc464b7b34391

apache-pulsar-3.3.0-src.tar.gz

Maven staging repo:
https://repository.apache.org/content/repositories/orgapachepulsar-1293/

The tag to verify:
v3.3.0-candidate-1 (3c3ce71c13a9cc0982089e7d87d97bc89a4a26c2)
https://github.com/apache/pulsar/commits/v3.3.0-candidate-1/

Pulsar's KEYS file containing PGP keys you use to sign the release:
https://dist.apache.org/repos/dist/dev/pulsar/KEYS

Docker images:

pulsar images:
docker pull czcoder/pulsar:3.3.0-3c3ce71
https://hub.docker.com/layers/czcoder/pulsar/3.3.0-3c3ce71/images/sha256-ce6ab616eca8b0679a9d0cec50578dc744d7091d305cf4d2af8f533fe4e38641?context=explore

pulsar-all images:
docekr pull czcoder/pulsar-all:3.3.0-3c3ce71
https://hub.docker.com/layers/czcoder/pulsar-all/3.3.0-3c3ce71/images/sha256-bd8b6bb3fbbacdf4d21b2a70cd3b4be7f0de839d1649652eda005f3ed46b78fd?context=explore


Please download the source package, and follow the README to build
and run the Pulsar standalone service.

Note that this RC doesn't require a formal vote, but we would also
appreciate your feedback with +1/-1. And please provide specific
comments if your feedback is not +1.


Regards
Cong Zhao

Re: [DISCUSS] Apache Pulsar 3.3.0 release

[Cong Zhao]

Hi community
I have cut branch 3.3 and will freeze the code for the next two weeks.

Regards
Cong Zhao

On 2024/04/28 12:17:41 Cong Zhao wrote:
> Hi community,
> It has been more than three months since the release of 3.2.0. we now
> have more than 280 commits
> <https://github.com/apache/pulsar/pulls?q=is%3Apr+milestone%3A3.3.0+is%3Amerged+>
> and
> 11 PIPs
> <https://github.com/apache/pulsar/pulls?q=is%3Apr+milestone%3A3.3.0+is%3Amerged+label%3Atype%2FPIP+>
> merged. We'd better prepare for the 3.3.0 release. I would like to cut
> branch-3.3 in the next week and freeze the code for two weeks.
> Please leave any ideas or concerns.
> 
> 
> Regards
> Cong Zhao
> 

Re: [DISCUSS] Apache Pulsar 3.3.0 release

[Cong Zhao]

Hello Enrico,

Can you list which PRs are important? I see that only has 6 PRs closed on 
4.17.1.

On 2024/04/29 07:02:34 Enrico Olivelli wrote:
> +1
> 
> Thanks for driving the release.
> 
> FYI On the BookKeeper side there are some fixes about metrics, maybe we
> could ask for a new 4.17.1 release
> 
> Enrico
> 
> Il giorno dom 28 apr 2024 alle ore 21:04 PengHui Li  ha
> scritto:
> 
> > +1
> >
> > Thanks for driving the release.
> >
> > Penghui
> >
> > On Sun, Apr 28, 2024 at 8:17 PM Cong Zhao  wrote:
> >
> > > Hi community,
> > > It has been more than three months since the release of 3.2.0. we now
> > > have more than 280 commits
> > > <
> > >
> > https://github.com/apache/pulsar/pulls?q=is%3Apr+milestone%3A3.3.0+is%3Amerged+
> > > >
> > > and
> > > 11 PIPs
> > > <
> > >
> > https://github.com/apache/pulsar/pulls?q=is%3Apr+milestone%3A3.3.0+is%3Amerged+label%3Atype%2FPIP+
> > > >
> > > merged. We'd better prepare for the 3.3.0 release. I would like to cut
> > > branch-3.3 in the next week and freeze the code for two weeks.
> > > Please leave any ideas or concerns.
> > >
> > >
> > > Regards
> > > Cong Zhao
> > >
> >
> 

Re: [DISCUSS] Apache Pulsar 3.3.0 release

[Cong Zhao]

hello Enrico,

About when BookKeeper 4.17.1 will be released, I want to cut off the 3.3 branch 
this week, do we need to wait for BookKeeper 4.17.1 release?

On 2024/04/29 07:02:34 Enrico Olivelli wrote:
> +1
> 
> Thanks for driving the release.
> 
> FYI On the BookKeeper side there are some fixes about metrics, maybe we
> could ask for a new 4.17.1 release
> 
> Enrico
> 
> Il giorno dom 28 apr 2024 alle ore 21:04 PengHui Li  ha
> scritto:
> 
> > +1
> >
> > Thanks for driving the release.
> >
> > Penghui
> >
> > On Sun, Apr 28, 2024 at 8:17 PM Cong Zhao  wrote:
> >
> > > Hi community,
> > > It has been more than three months since the release of 3.2.0. we now
> > > have more than 280 commits
> > > <
> > >
> > https://github.com/apache/pulsar/pulls?q=is%3Apr+milestone%3A3.3.0+is%3Amerged+
> > > >
> > > and
> > > 11 PIPs
> > > <
> > >
> > https://github.com/apache/pulsar/pulls?q=is%3Apr+milestone%3A3.3.0+is%3Amerged+label%3Atype%2FPIP+
> > > >
> > > merged. We'd better prepare for the 3.3.0 release. I would like to cut
> > > branch-3.3 in the next week and freeze the code for two weeks.
> > > Please leave any ideas or concerns.
> > >
> > >
> > > Regards
> > > Cong Zhao
> > >
> >
> 

[DISCUSS] Apache Pulsar 3.3.0 release

[Cong Zhao]

Hi community,
It has been more than three months since the release of 3.2.0. we now
have more than 280 commits
<https://github.com/apache/pulsar/pulls?q=is%3Apr+milestone%3A3.3.0+is%3Amerged+>
and
11 PIPs
<https://github.com/apache/pulsar/pulls?q=is%3Apr+milestone%3A3.3.0+is%3Amerged+label%3Atype%2FPIP+>
merged. We'd better prepare for the 3.3.0 release. I would like to cut
branch-3.3 in the next week and freeze the code for two weeks.
Please leave any ideas or concerns.


Regards
Cong Zhao

Re: [ANNOUNCE] New Committer: Asaf Mesika

[Cong Zhao]

Congratulations!

Thanks,
Cong

On 2024/02/20 16:50:29 Lari Hotari wrote:
> The Apache Pulsar Project Management Committee (PMC) has invited
> Asaf Mesika https://github.com/asafm to become a committer and we
> are pleased to announce that he has accepted.
> 
> Welcome and Congratulations, Asaf Mesika!
> 
> Please join us in congratulating and welcoming Asaf onboard!
> 
> Best Regards,
> 
> Lari Hotari
> on behalf of the Pulsar PMC
> 

Re: [VOTE] PIP-315: Configurable max delay limit for delayed delivery

[Cong Zhao]

+1 (non-binding)

Thanks,
Cong Zhao

On 2023/11/15 03:59:46 Kevin Lu wrote:
> Hi All,
> 
> This thread is to start a vote for PIP-315.
> 
> PIP: https://github.com/apache/pulsar/pull/21490
> Discussion thread:
> https://lists.apache.org/thread/285nm08842or324rxc2zy83wxgqxtcjp
> 
> Regards,
> Kevin
> 

Re: [VOTE] PIP-318: Don't retain null-key messages during topic compaction

[Cong Zhao]

Close the vote with 4+ (binding):
Penghui
Mattison
Yunze
Yubiao

1+(non-binding):
Zixuan

Regards
Cong Zhao

On 2023/11/10 04:12:14 Cong Zhao wrote:
> Hi Community,
> 
> This thread is to start a vote for PIP-318.
> 
> PIP: https://github.com/apache/pulsar/pull/21541
> Discussion thread: 
> https://lists.apache.org/thread/68k6vrghfp3np601lrfx5mbfmghbbrjh
> 
> Thanks,
> Cong Zhao
> 

Re: [ANNOUNCE] Yubiao Feng as new PMC member in Apache Pulsar

[Cong Zhao]

Congrats! Yubiao

Thanks,
Cong

On 2023/11/13 07:36:27 mattison chao wrote:
> Dear Community, 
> 
> We are thrilled to announce that Yubiao Feng https://github.com/poorbarcode 
> has been invited and has accepted the role of a member of the Apache Pulsar 
> Project Management Committee (PMC). 
> 
> Yubiao Feng has proven to be an invaluable asset to our community, 
> consistently showcasing dedication and active engagement through substantial 
> contributions. Beyond his noteworthy technical input, Yubiao plays a crucial 
> role in meticulously reviewing pull requests, thereby ensuring the overall 
> excellence of our project. We eagerly anticipate and appreciate his ongoing 
> contributions. On behalf of the Pulsar PMC, we extend a heartfelt welcome and 
> congratulations to Yubiao Feng.
> 
> Sincerely,
> Mattison

Re: [DISCUSS] PIP-315: Configurable max delay limit for delayed delivery

[Cong Zhao]

I think this is a good point.

Thanks,
Cong Zhao

On 2023/10/31 22:21:56 Kevin Lu wrote:
> Hi All,
> 
> I just submitted a PIP to add an optional config to limit the max delay
> allowed for delayed delivery. This would be helpful for Pulsar
> administrators if they wanted to provide some limits on delayed delivery.
> 
> Link - https://github.com/apache/pulsar/pull/21490
> 
> Thanks,
> Kevin
> 

[VOTE] PIP-318: Don't retain null-key messages during topic compaction

[Cong Zhao]

Hi Community,

This thread is to start a vote for PIP-318.

PIP: https://github.com/apache/pulsar/pull/21541
Discussion thread: 
https://lists.apache.org/thread/68k6vrghfp3np601lrfx5mbfmghbbrjh

Thanks,
Cong Zhao

Re: [DISSCUSS] Don't retain null-key messages during topic compaction

[Cong Zhao]

Hi Enrico,

I don't think they conflict. We can do them separately.

We can add the policy to the per topic and per namespace again when we need it 
in the future, and the current PIP can just add a configuration to broker.conf 
first so that we can not retain the null-key messages and cherry-pick this fix 
into other branches with `compactionRemainNullKey=true` default to keep 
compatibility.

Thanks,
Cong Zhao

On 2023/11/08 07:26:30 Enrico Olivelli wrote:
> This is a good point.
> 
> Is it worth to make it configurable per topic and per namespace?
> 
> Even if this behavior is kind if a side effect or bug, maybe some
> applications rely on it.
> Flags that change the behavior per cluster are hard to adopt in nig cluster
> with many tenants.
> 
> We should always take multi tenancy in mind when we design features or
> changes.
> 
> 
> Thanks
> Enrico
> 
> Il Mer 8 Nov 2023, 07:55 Cong Zhao  ha scritto:
> 
> > Hello everyone,
> >
> > I open a PIP-318: https://github.com/apache/pulsar/pull/21541 for this
> > discussion.
> >
> > Any feedback and suggestions are welcome.
> >
> > Thanks,
> > Cong Zhao
> >
> > On 2023/11/07 02:55:52 Cong Zhao wrote:
> > > Hi, Pulsar community
> > >
> > > Currently, we retain all null-key messages during topic compaction,
> > which I
> > > don't think is necessary because when you use topic compaction,
> > > it means that you want to retain the value according to the key, so
> > > retaining null-key messages is meaningless.
> > >
> > > Additionally, retaining all null-key messages will double the storage
> > cost,
> > > and we'll never be able to clean them up since the compacted topic has
> > not
> > > supported the retention policy yet.
> > >
> > > In summary, I don't think we should retain null-key messages during topic
> > > compaction.
> > > Looking forward to your feedback!
> > >
> > > Thanks,
> > > Cong Zhao
> > >
> >
> 

Re: [DISSCUSS] Don't retain null-key messages during topic compaction

[Cong Zhao]

Hello everyone,

I open a PIP-318: https://github.com/apache/pulsar/pull/21541 for this 
discussion.

Any feedback and suggestions are welcome.

Thanks,
Cong Zhao

On 2023/11/07 02:55:52 Cong Zhao wrote:
> Hi, Pulsar community
> 
> Currently, we retain all null-key messages during topic compaction, which I
> don't think is necessary because when you use topic compaction,
> it means that you want to retain the value according to the key, so
> retaining null-key messages is meaningless.
> 
> Additionally, retaining all null-key messages will double the storage cost,
> and we'll never be able to clean them up since the compacted topic has not
> supported the retention policy yet.
> 
> In summary, I don't think we should retain null-key messages during topic
> compaction.
> Looking forward to your feedback!
> 
> Thanks,
> Cong Zhao
> 

Re: [DISSCUSS] Don't retain null-key messages during topic compaction

[Cong Zhao]

Hi mattison,

Thanks for your suggestion, I agree with you, we can add a configuration to 
smooth the migrate this change.

Let's see if anyone else has any other ideas, and if everyone agrees with this 
approach, I'll implement it.

Thanks,
Cong Zhao

On 2023/11/07 03:03:58 mattison chao wrote:
> Hi, Cong
> 
> IMO, Please do not break the previous directly. We can migrate it smoothly.  
> We can add a configuration and give the
> Timeline of making this configuration default and removing it in the next 
> release version.
> 
> For example:
> 
> - Add configuration: `compactionRemainNullKey=true` by default (current 
> behaviour)
> - Make `compactionRemainNullKey=false` default  in the 3.2.0
> - Delete the configuration `compactionRemainNullKey` in 3.3.0.
> 
> This approach will avoid breaking changes and give our users enough time to 
> migrate their usage.
> 
> Plus, I think it’s fair to cherry-pick it to all the previous active 
> branches. 
> 
> 
> Thanks!
> Mattison
> 
> 
> 
> > On Nov 7, 2023, at 10:55, Cong Zhao  wrote:
> > 
> > Hi, Pulsar community
> > 
> > Currently, we retain all null-key messages during topic compaction, which I
> > don't think is necessary because when you use topic compaction,
> > it means that you want to retain the value according to the key, so
> > retaining null-key messages is meaningless.
> > 
> > Additionally, retaining all null-key messages will double the storage cost,
> > and we'll never be able to clean them up since the compacted topic has not
> > supported the retention policy yet.
> > 
> > In summary, I don't think we should retain null-key messages during topic
> > compaction.
> > Looking forward to your feedback!
> > 
> > Thanks,
> > Cong Zhao
> 
> 

[DISSCUSS] Don't retain null-key messages during topic compaction

[Cong Zhao]

Hi, Pulsar community

Currently, we retain all null-key messages during topic compaction, which I
don't think is necessary because when you use topic compaction,
it means that you want to retain the value according to the key, so
retaining null-key messages is meaningless.

Additionally, retaining all null-key messages will double the storage cost,
and we'll never be able to clean them up since the compacted topic has not
supported the retention policy yet.

In summary, I don't think we should retain null-key messages during topic
compaction.
Looking forward to your feedback!

Thanks,
Cong Zhao

Re: [VOTE] PIP-307: Support subscribing multi-topics for WebSocket

[Cong Zhao]

+1(no-binding)

Thanks,
Cong Zhao

On 2023/10/19 12:47:45 guo jiwei wrote:
> Hi dev,
>Currently WebSocket only supports the consumption of a single topic,
> which cannot satisfy users' consumption scenarios of multiple topics.  So
> in order to support consumption of multiple topics or pattern topics, I
> would like to start a vote for PIP-307
> <https://github.com/apache/pulsar/pull/21390>.
> 
> 
> Ref:
> • Discuss Mail:
> https://lists.apache.org/thread/co8396ywny161x91dffzvxlt993mo1ht
> • PIP-307: https://github.com/apache/pulsar/pull/21390
> 
> 
> Regards
> Jiwei Guo (Tboy)
> 

Re: [VOTE] PIP-286: Make the TopicCompactionService to support find entry based on publishTime or index

[Cong Zhao]

Thanks.
Close the vote with 3+ (binding):
Penghui
Mattison
Jiwei Guo

1+(non-binding):
Zike

Regards
Cong Zhao

On 2023/09/14 03:23:29 Cong Zhao wrote:
> Hi, all
> 
> Since there are no other concerns in the discussion, I'm delighted to
> start the voting process for the PIP-286.
> 
> Here is the link to the PIP: https://github.com/apache/pulsar/pull/20867
> 
> Thanks,
> Cong Zhao
> 

[VOTE] PIP-286: Make the TopicCompactionService to support find entry based on publishTime or index

[Cong Zhao]

Hi, all

Since there are no other concerns in the discussion, I'm delighted to
start the voting process for the PIP-286.

Here is the link to the PIP: https://github.com/apache/pulsar/pull/20867

Thanks,
Cong Zhao

Re: [DISCUSS] PIP-286: Add findNewestPosition method in the TopicCompactionService API

[Cong Zhao]

Hi, guys

I noticed that the only globally ordered fields in the metadata are 
'publishTime' and 'entryIndex' (offset), so I split the previous method into 
`findEntryByPublishTime` and `findEntryByEntryIndex`,
this will make method definitions more precise.

The method returns the entry directly instead of RawEntryMetadata to avoid 
introducing a new interface, it's important to note that the caller needs to 
release it manually.

I already updated the proposal, please help review it.

Thanks,
Cong Zhao

On 2023/07/25 04:12:09 Cong Zhao wrote:
> Hi Pulsar Community, I am writing to start the discussion on PIP 286  PR
> with PIP contents: https://github.com/apache/pulsar/pull/20867  Thanks,
> Cong Zhao
> 

Re: [DISCUSS] PIP-286: Add findNewestPosition method in the TopicCompactionService API

[Cong Zhao]

Hi Penghui,

Thanks for the suggestion.

First, I changed the method to findFirstEntryMetadata because I think it's more 
common to look for the first matching entry compared to finding the entry of 
the last match, e.g. to look for the first position greater than or equal to a 
specific publish time.

As to return of 'RawEntryMetadata' is to reduce the complexity of the 
implementation of considerations, because when we use binary search managing 
the payload reference counter is complex. I think this method just returns 
entry metadata, If we need to get payload then we can be called again to call 
the readCompactedEntries method.

Thanks,
Cong Zhao

On 2023/08/28 01:31:14 PengHui Li wrote:
> Hi Cong,
> 
> Thanks for driving the proposal.
> Is it better to use `readNewestCompactedEntry(@Nonnull
> Predicate condition)`
> 
> So that you don't need to introduce a new interface, RawEntryMetadata
> And it's a more general solution for the TopicCompactionService.
> 
> WDYT?
> 
> Regards,
> Penghui
> 
> On Tue, Jul 25, 2023 at 12:12 PM Cong Zhao  wrote:
> 
> > Hi Pulsar Community, I am writing to start the discussion on PIP 286  PR
> > with PIP contents: https://github.com/apache/pulsar/pull/20867  Thanks,
> > Cong Zhao
> >
> 

[DISCUSS] Prevent trimming ledgers smaller than or equal to slowestNonDurationLedgerId

[Cong Zhao]

Hi Pulsar community,

Currently, the ledger is trimmed without taking NonDurableCursor into account, 
which may cause the reader to lose some of its messages.

I think we should prevent ledgers smaller than or equal to 
'slowestNonDurationLedgerId' from being trimmed during trimming ledgers.

Looking forward to your feedback!

Best regards,
Cong Zhao

Re: [VOTE] Accept pulsar-admin-go as part of the Apache Pulsar project

[Cong Zhao]

Great!

+1 non-binding

Thanks,
Cong Zhao

On 2023/08/05 04:14:24 tison wrote:
> Hi,
> 
> I'd like to start a vote thread for accepting pulsar-admin-go[1] as part of
> the Apache Pulsar project. pulsar-admin-go is a Golang client library
> interacting with Pulsar Admin API. You can see the full proposal, including
> the motivation and features at [2].
> 
> Please vote with your opinions. The VOTE will remain open for at least 72
> hours.
> 
> [ ] +1 approve
> [ ] +0 no opinion
> [ ] -1 disapprove with the reason
> 
> Vote from PMC members is binding.
> 
> Best,
> tison.
> 
> [1] https://github.com/streamnative/pulsar-admin-go
> [2] https://lists.apache.org/thread/tb6c10qlkg9fj56qc0ldkwc79j7qm0vc
> 

[DISCUSS] PIP-286: Add findNewestPosition method in the TopicCompactionService API

[Cong Zhao]

Hi Pulsar Community, I am writing to start the discussion on PIP 286  PR
with PIP contents: https://github.com/apache/pulsar/pull/20867  Thanks,
Cong Zhao

Re: [VOTE] PIP-278: Support pluggable topic compaction service

[Cong Zhao]

Close this vote by

3 binding +1s
* Qiang Zhao
* Yunze Xu
* PengHui Li

Thanks,
Cong Zhao

On 2023/06/26 02:01:06 Cong Zhao wrote:
> Hello, community:
> 
> This thread is to start a vote for PIP-278: Support pluggable topic
> compaction service.
> 
> Discussion thread:
> https://lists.apache.org/thread/ox2bot3p9j9fydqkw3v5gt5twc8jslvd
> 
> PIP PR: https://github.com/apache/pulsar/pull/20624
> 
> Sincerely
> Cong Zhao
> 

[VOTE] PIP-278: Support pluggable topic compaction service

[Cong Zhao]

Hello, community:

This thread is to start a vote for PIP-278: Support pluggable topic
compaction service.

Discussion thread:
https://lists.apache.org/thread/ox2bot3p9j9fydqkw3v5gt5twc8jslvd

PIP PR: https://github.com/apache/pulsar/pull/20624

Sincerely
Cong Zhao

[DISCUSS] PIP-278: Support pluggable topic compaction service

[Cong Zhao]

Hi Pulsar Community,

I am writing to start the discussion on PIP 278 to support pluggable compaction 
service

PR with PIP contents: https://github.com/apache/pulsar/pull/20624

Thanks,
Cong Zhao

[DISCUSS] PIP-274: Support pluggable topic compactor

[Cong Zhao]

Hi Pulsar Community,

I am writing to start the discussion on PIP 274 to support pluggable topic 
compactor

PR with PIP contents: https://github.com/apache/pulsar/pull/20493

Thanks,
Cong Zhao

[ANNOUNCE] Apache Pulsar 2.9.5 released

[Cong Zhao]

The Apache Pulsar team is proud to announce Apache Pulsar version 2.9.5.

Pulsar is a highly scalable, low latency messaging platform running on
commodity hardware. It provides simple pub-sub semantics over topics,
guaranteed at-least-once delivery of messages, automatic cursor management
for
subscribers, and cross-datacenter replication.

For Pulsar release details and downloads, visit:

https://pulsar.apache.org/download

Release Notes are at:
https://pulsar.apache.org/release-notes

We would like to thank the contributors that made the release possible.

Regards,

The Pulsar Team

Re: Pulsar doesn't support cgroup v2 which became the default in Kubernetes v1.25+ (AKS v1.25, GKE v1.26, EKS v1.26)

[Cong Zhao]

Hi Lari Hotar,

I would like to pick up this work, I will update 
https://github.com/apache/pulsar/pull/16832 as soon.

Thanks,
Cong Zhao

On 2023/04/26 15:17:37 Lari Hotari wrote:
> Hi all,
> 
> Pulsar doesn't support cgroup v2 which becomes default in Kubernetes v1.25+.
> Kubernetes announcement:
> https://kubernetes.io/blog/2022/08/31/cgroupv2-ga-1-25/ .
> Pulsar issue: https://github.com/apache/pulsar/issues/16601
> 
> The impact of this is that the Pulsar load balancer won't have correct
> CPU and memory information for making load balancing decisions.
> 
> The cloud provider managed Kubernetes services have already switched
> to cgroup v2 as the default. This happened in AKS v1.25, GKE v1.26 and
> in EKS v1.26.
> For GKE, it's possible to keep using cgroup v1 also in GKE v1.26
> (https://cloud.google.com/kubernetes-engine/docs/how-to/node-system-config#cgroup-mode-options).
> For AKS and EKS, it's unknown whether such a configuration option
> exists.
> 
> There's a previous attempt in this PR to add cgroup v2 support to
> Pulsar: https://github.com/apache/pulsar/pull/16832 . Would it be
> possible to continue the work for supporting cgroup v2 in Pulsar
> either with the existing PR or a new one?
> 
> Who would like to pick up this work?
> This is urgent since cgroup v2 is enabled by default for all latest
> managed Kubernetes services (AKS v1.25, GKE v1.26 and EKS v1.26).
> 
> Regards,
> 
> -Lari
> 

Re: [NOTICE] Please don't cherry-pick commits to branch-3.0 without consensus

[Cong Zhao]

I agree with this, other PRs can move to 3.0.1.

Thanks,
Cong Zhao

On 2023/04/21 09:43:20 Christophe Bornet wrote:
> So to respect the code freeze, I propose to revert all those commits except:
> https://github.com/apache/pulsar/pull/19849
> https://github.com/apache/pulsar/pull/20086
> 
> I understand that the optimization PRs are nice but if we start to
> accept them, why not accepting all the others and the code freeze
> becomes pointless.
> We can do a 3.0.1 shortly after to include all those enhancements.
> 
> Do we have an agreement on this ?
> 
> Christophe
> 
> Le ven. 21 avr. 2023 à 11:36, Cong Zhao  a écrit :
> >
> > I strongly agree that these cherry-pick should be notified to the release 
> > managers to avoid unintended consequences.
> >
> > PR https://github.com/apache/pulsar/pull/20086 is to solve a serious bug it 
> > will lead to the pulsar-io block, so it is very necessary to cherry-pick 
> > into 3.0. The rest of the PRs are optimized for large amounts of data, we 
> > also can discuss whether they are necessary to cherry-pick into 3.0.
> >
> > Thanks,
> > Cong Zhao
> >
> > On 2023/04/21 07:26:43 Nicolò Boschi wrote:
> > > I believe that currently there's no clear process in place in order to
> > > decide whether a commit should be cherry-picked into the frozen branch.
> > > I know that we have a slack channel for the release coordination
> > > #pulsar-release-3_0, which is open to everyone.
> > >
> > > One solution would be to let the release managers cherry-pick the commits 
> > > -
> > > they are the contributors more focused on the stability of the release
> > > branch and they might be able to discuss if it's worthwhile.
> > > Their opinion is not more important than others, but they need to know it
> > > anyway and they could spot incompatibilities and avoid unintended
> > > consequences.
> > > The mailing list is too "slow" for this kind of stuff. For more complex
> > > discussions, there's always the possibility to start a thread here.
> > >
> > > When a committer would like to cherry-pick a commit, instead of directly
> > > going for it without any discussion, they can ask in the release slack
> > > channel.
> > > The release managers will eventually cherry-pick it.
> > > If there's no consensus then the discussion is moved to the mailing list. 
> > > I
> > > believe this wouldn't happen often, considering that currently we rely on
> > > the common sense of committers.
> > >
> > > What do you think?
> > > Nicolò Boschi
> > >
> > >
> > > Il giorno ven 21 apr 2023 alle ore 07:58 Cong Zhao  
> > > ha
> > > scritto:
> > >
> > > > Hi Zike,
> > > >
> > > > I'm sorry for cherry-picking the new delayed message PRs to branch-3.0.
> > > >
> > > > Would be very grateful if we could get to 3.0 since it is the new 
> > > > feature
> > > > of the delayed message and has no impact on other components.
> > > >
> > > > These PR is important to PIP-195, they will fix some problem with the 
> > > > new
> > > > delayed message and make this new feature work better.
> > > >
> > > > Need to cherry-pick PR:
> > > > https://github.com/apache/pulsar/pull/20086
> > > > https://github.com/apache/pulsar/pull/20111
> > > > https://github.com/apache/pulsar/pull/20126
> > > > https://github.com/apache/pulsar/pull/20136
> > > > https://github.com/apache/pulsar/pull/20117
> > > > https://github.com/apache/pulsar/pull/20155
> > > > https://github.com/apache/pulsar/pull/20156
> > > > https://github.com/apache/pulsar/pull/20158
> > > >
> > > > Thanks,
> > > > Cong Zhao
> > > >
> > > > On 2023/04/21 05:36:26 Zike Yang wrote:
> > > > > Hi, all
> > > > >
> > > > > We found that there were a lot of commits that were cherry-picked to
> > > > > branch-3.0 without any discussion or notification to reach a
> > > > > consensus. Some PRs marked for the 3.1.0 milestone were also
> > > > > cherry-picked into branch-3.0. Here are the corresponding
> > > > > cherry-picked commits:
> > > > > *
> > > > https://github.com/apache/pulsar/commit/3da39b2e1553cecbc6d6b85e8bc7844f611d5637
> > > > > *
> > > > https://github.com/apache/pulsar/commit/e248e1447314276

Re: [NOTICE] Please don't cherry-pick commits to branch-3.0 without consensus

[Cong Zhao]

I strongly agree that these cherry-pick should be notified to the release 
managers to avoid unintended consequences.

PR https://github.com/apache/pulsar/pull/20086 is to solve a serious bug it 
will lead to the pulsar-io block, so it is very necessary to cherry-pick into 
3.0. The rest of the PRs are optimized for large amounts of data, we also can 
discuss whether they are necessary to cherry-pick into 3.0.

Thanks,
Cong Zhao

On 2023/04/21 07:26:43 Nicolò Boschi wrote:
> I believe that currently there's no clear process in place in order to
> decide whether a commit should be cherry-picked into the frozen branch.
> I know that we have a slack channel for the release coordination
> #pulsar-release-3_0, which is open to everyone.
> 
> One solution would be to let the release managers cherry-pick the commits -
> they are the contributors more focused on the stability of the release
> branch and they might be able to discuss if it's worthwhile.
> Their opinion is not more important than others, but they need to know it
> anyway and they could spot incompatibilities and avoid unintended
> consequences.
> The mailing list is too "slow" for this kind of stuff. For more complex
> discussions, there's always the possibility to start a thread here.
> 
> When a committer would like to cherry-pick a commit, instead of directly
> going for it without any discussion, they can ask in the release slack
> channel.
> The release managers will eventually cherry-pick it.
> If there's no consensus then the discussion is moved to the mailing list. I
> believe this wouldn't happen often, considering that currently we rely on
> the common sense of committers.
> 
> What do you think?
> Nicolò Boschi
> 
> 
> Il giorno ven 21 apr 2023 alle ore 07:58 Cong Zhao  ha
> scritto:
> 
> > Hi Zike,
> >
> > I'm sorry for cherry-picking the new delayed message PRs to branch-3.0.
> >
> > Would be very grateful if we could get to 3.0 since it is the new feature
> > of the delayed message and has no impact on other components.
> >
> > These PR is important to PIP-195, they will fix some problem with the new
> > delayed message and make this new feature work better.
> >
> > Need to cherry-pick PR:
> > https://github.com/apache/pulsar/pull/20086
> > https://github.com/apache/pulsar/pull/20111
> > https://github.com/apache/pulsar/pull/20126
> > https://github.com/apache/pulsar/pull/20136
> > https://github.com/apache/pulsar/pull/20117
> > https://github.com/apache/pulsar/pull/20155
> > https://github.com/apache/pulsar/pull/20156
> > https://github.com/apache/pulsar/pull/20158
> >
> > Thanks,
> > Cong Zhao
> >
> > On 2023/04/21 05:36:26 Zike Yang wrote:
> > > Hi, all
> > >
> > > We found that there were a lot of commits that were cherry-picked to
> > > branch-3.0 without any discussion or notification to reach a
> > > consensus. Some PRs marked for the 3.1.0 milestone were also
> > > cherry-picked into branch-3.0. Here are the corresponding
> > > cherry-picked commits:
> > > *
> > https://github.com/apache/pulsar/commit/3da39b2e1553cecbc6d6b85e8bc7844f611d5637
> > > *
> > https://github.com/apache/pulsar/commit/e248e14473142765db1df324c20a081a2422980e
> > > *
> > https://github.com/apache/pulsar/commit/e27abe9e128fb71b65ffe06417574c9a7f3facbd
> > > *
> > https://github.com/apache/pulsar/commit/e1d63990644700bf61b3d7af1ef6d4d62145c2bb
> > > *
> > https://github.com/apache/pulsar/commit/ff59240165c73a9c3a3dcca20702ab44b0b18d33
> > > *
> > https://github.com/apache/pulsar/commit/49480ea558e647169e8df01bfd2e871a5386e19e
> > > *
> > https://github.com/apache/pulsar/commit/1d1a3ef864a65c995ceda4b7875ed934c2574298
> > > *
> > https://github.com/apache/pulsar/commit/0df741259505b9189147d72c6f013b2c18f0436c
> > > *
> > https://github.com/apache/pulsar/commit/d05871213adc351d4c718c2a6fb0909b01d279ff
> > >
> > > According to our release policy[0] and the code freeze notification
> > > [1], all commits that need to be cherry-picked into branch-3.0 will
> > > require reaching the consensus before cherry-picking.
> > > Before cherry-picking the commit, we need to provide the context for
> > > why it needs to be cherry-picked into branch-3.0. Then mark it with
> > > the 3.0.0 milestone and raise the discussion to reach a consensus.
> > >
> > > I would like to start a discussion regarding the context for the above
> > > cherry-picked commits. Should we include them in Pulsar 3.0?
> > > If there is still no consensus for the above commits, we may need to
> > > revert them before the RC3.
> > >
> > > If you have any questions or concerns, please do not hesitate to let
> > > us know. Thanks for your cooperation.
> > >
> > > [0] https://pulsar.apache.org/contribute/release-policy/#release-cycles
> > > [1] https://lists.apache.org/thread/43d28rtzsx7x3o4zd523jr5dmnczrn4h
> > >
> > > Thanks,
> > > Zike Yang
> > >
> >
> 

Re: [NOTICE] Please don't cherry-pick commits to branch-3.0 without consensus

[Cong Zhao]

Hi Zike, 

I'm sorry for cherry-picking the new delayed message PRs to branch-3.0.

Would be very grateful if we could get to 3.0 since it is the new feature of 
the delayed message and has no impact on other components. 

These PR is important to PIP-195, they will fix some problem with the new 
delayed message and make this new feature work better.

Need to cherry-pick PR:
https://github.com/apache/pulsar/pull/20086
https://github.com/apache/pulsar/pull/20111
https://github.com/apache/pulsar/pull/20126
https://github.com/apache/pulsar/pull/20136
https://github.com/apache/pulsar/pull/20117
https://github.com/apache/pulsar/pull/20155
https://github.com/apache/pulsar/pull/20156
https://github.com/apache/pulsar/pull/20158

Thanks,
Cong Zhao

On 2023/04/21 05:36:26 Zike Yang wrote:
> Hi, all
> 
> We found that there were a lot of commits that were cherry-picked to
> branch-3.0 without any discussion or notification to reach a
> consensus. Some PRs marked for the 3.1.0 milestone were also
> cherry-picked into branch-3.0. Here are the corresponding
> cherry-picked commits:
> * 
> https://github.com/apache/pulsar/commit/3da39b2e1553cecbc6d6b85e8bc7844f611d5637
> * 
> https://github.com/apache/pulsar/commit/e248e14473142765db1df324c20a081a2422980e
> * 
> https://github.com/apache/pulsar/commit/e27abe9e128fb71b65ffe06417574c9a7f3facbd
> * 
> https://github.com/apache/pulsar/commit/e1d63990644700bf61b3d7af1ef6d4d62145c2bb
> * 
> https://github.com/apache/pulsar/commit/ff59240165c73a9c3a3dcca20702ab44b0b18d33
> * 
> https://github.com/apache/pulsar/commit/49480ea558e647169e8df01bfd2e871a5386e19e
> * 
> https://github.com/apache/pulsar/commit/1d1a3ef864a65c995ceda4b7875ed934c2574298
> * 
> https://github.com/apache/pulsar/commit/0df741259505b9189147d72c6f013b2c18f0436c
> * 
> https://github.com/apache/pulsar/commit/d05871213adc351d4c718c2a6fb0909b01d279ff
> 
> According to our release policy[0] and the code freeze notification
> [1], all commits that need to be cherry-picked into branch-3.0 will
> require reaching the consensus before cherry-picking.
> Before cherry-picking the commit, we need to provide the context for
> why it needs to be cherry-picked into branch-3.0. Then mark it with
> the 3.0.0 milestone and raise the discussion to reach a consensus.
> 
> I would like to start a discussion regarding the context for the above
> cherry-picked commits. Should we include them in Pulsar 3.0?
> If there is still no consensus for the above commits, we may need to
> revert them before the RC3.
> 
> If you have any questions or concerns, please do not hesitate to let
> us know. Thanks for your cooperation.
> 
> [0] https://pulsar.apache.org/contribute/release-policy/#release-cycles
> [1] https://lists.apache.org/thread/43d28rtzsx7x3o4zd523jr5dmnczrn4h
> 
> Thanks,
> Zike Yang
> 

Re: [VOTE] Pulsar Release 2.9.5 Candidate 3

[Cong Zhao]

Close this vote with 4 +1 (binding)
- Mattison
- Penghui
- Jiwei
- Yunze

On 2023/04/10 07:38:09 Cong Zhao wrote:
> This is the third release candidate for Apache Pulsar, version 2.9.5.
> 
> This release contains 105 commits by 32 contributors.
> https://github.com/apache/pulsar/compare/v2.9.4...v2.9.5-candidate-3
> 
> *** Please download, test, and vote on this release. This vote will stay
> open
> for at least 72 hours ***
> 
> Note that we are voting upon the source (tag), binaries are provided for
> convenience.
> 
> Source and binary files:
> https://dist.apache.org/repos/dist/dev/pulsar/pulsar-2.9.5-candidate-3/
> 
> SHA-512 checksums:
> c5f1b06f2f7249616a07b732daa036546591c1aa9e9f37e88ddce6d3eaf3d78b2c6da548ef42946d20d70f9397cd5b55ea4e01fe5cb4aa96fb4608bd62635f67
> apache-pulsar-2.9.5-bin.tar.gz
> 
> 72dee0fb642a269c5d0aedfa3e56aee503f56af319b67e4628f39da8cdd44f0bbdcd019e121eafca32b7b7665959770a3d91f0618d157837808da9511e9011a40f
> apache-pulsar-2.9.5-src.tar.gz
> 
> 738478bbcf323080487f5645b4c1edb9f45d126a8a2cd7e8c5678c8569ec7b21042ecb6fdd4796b65c6a24bd1e5ad082f5c0d15eb1a64408be4f4c53a06d3ef660
> apache-pulsar-offloaders-2.9.5-bin.tar.gz
> 
> Maven staging repo:
> https://repository.apache.org/content/repositories/orgapachepulsar-1223/
> 
> The tag to be voted upon:
> v2.9.5-candidate-3 (7337470f33586aa639854266efe95c2fa32f48db)
> https://github.com/apache/pulsar/releases/tag/v2.9.5-candidate-3
> 
> Pulsar's KEYS file containing PGP keys you use to sign the release:
> https://downloads.apache.org/pulsar/KEYS
> 
> Docker images:
> 
> 
> https://hub.docker.com/layers/czcoder/pulsar/2.9.5/images/sha256-9bcbad78a65a09b3fe5d3acd158574ad0eda4531cc8c49db16760a2ed1364070?context=explore
> 
> 
> https://hub.docker.com/layers/czcoder/pulsar-all/2.9.5/images/sha256-94e50aa1d5b91be9698c959c787eaff0677bb3c65dd1a578b1a641fa5a1d1715?context=explore
> 
> 
> https://hub.docker.com/layers/czcoder/pulsar-grafana/2.9.5/images/sha256-d857951e85b41fe7172769380ffbcac126c75dc476fdcb4755623600b3eb03e0?context=explore
> 
> Please download the source package, and follow the README to build
> and run the Pulsar standalone service.
> 
> Thanks
> Cong Zhao
> 

Re: [DISCUSS] Sorting out pulsar's internal thread pools

[Cong Zhao]

I think this is a good idea for the reasonable use of pulsar threads.

Thanks,
Cong Zhao

On 2023/04/18 02:07:55 mattisonc...@gmail.com wrote:
> 
> Hello, folks.
> 
> I would like to start discussing the pulsar internal thread pool sorting out.
> 
> How did I get this idea?
> 
> Recently, we met some problems with the BK operation timeout. After 
> investigating, we found an issue that is we share the IO executor(workgroup) 
> with the Bookkeeper client and internal client and do some other async task 
> in the dispatcher or somewhere to avoid deadlock.
> 
> But the problem over here. If we use this executor to do some kind of 
> `blocking`(or spend much time computing. e.g. reply to many delayed messages) 
> operation, it will block BK clients from sending requests if they are using 
> the same thread.
> 
> And then, I checked all the usage of the thread pool. We need the rule to 
> constrain what thread pool we should use.
> 
> What am I expecting?
> 
> I want to collect all the thread pools and define a clear usage guide to 
> avoid wrong use and improve the fault tolerance(the component problem 
> shouldn't affect the whole broker)
> 
> 
> 
> I need to hear your guy's opinions. Please feel free to leave any questions. 
> Thanks!
> 
> 
> Best,
> Mattison
> 
> 
> 

[VOTE] Pulsar Release 2.9.5 Candidate 3

[Cong Zhao]

This is the third release candidate for Apache Pulsar, version 2.9.5.

This release contains 105 commits by 32 contributors.
https://github.com/apache/pulsar/compare/v2.9.4...v2.9.5-candidate-3

*** Please download, test, and vote on this release. This vote will stay
open
for at least 72 hours ***

Note that we are voting upon the source (tag), binaries are provided for
convenience.

Source and binary files:
https://dist.apache.org/repos/dist/dev/pulsar/pulsar-2.9.5-candidate-3/

SHA-512 checksums:
c5f1b06f2f7249616a07b732daa036546591c1aa9e9f37e88ddce6d3eaf3d78b2c6da548ef42946d20d70f9397cd5b55ea4e01fe5cb4aa96fb4608bd62635f67
apache-pulsar-2.9.5-bin.tar.gz

72dee0fb642a269c5d0aedfa3e56aee503f56af319b67e4628f39da8cdd44f0bbdcd019e121eafca32b7b7665959770a3d91f0618d157837808da9511e9011a40f
apache-pulsar-2.9.5-src.tar.gz

738478bbcf323080487f5645b4c1edb9f45d126a8a2cd7e8c5678c8569ec7b21042ecb6fdd4796b65c6a24bd1e5ad082f5c0d15eb1a64408be4f4c53a06d3ef660
apache-pulsar-offloaders-2.9.5-bin.tar.gz

Maven staging repo:
https://repository.apache.org/content/repositories/orgapachepulsar-1223/

The tag to be voted upon:
v2.9.5-candidate-3 (7337470f33586aa639854266efe95c2fa32f48db)
https://github.com/apache/pulsar/releases/tag/v2.9.5-candidate-3

Pulsar's KEYS file containing PGP keys you use to sign the release:
https://downloads.apache.org/pulsar/KEYS

Docker images:


https://hub.docker.com/layers/czcoder/pulsar/2.9.5/images/sha256-9bcbad78a65a09b3fe5d3acd158574ad0eda4531cc8c49db16760a2ed1364070?context=explore


https://hub.docker.com/layers/czcoder/pulsar-all/2.9.5/images/sha256-94e50aa1d5b91be9698c959c787eaff0677bb3c65dd1a578b1a641fa5a1d1715?context=explore


https://hub.docker.com/layers/czcoder/pulsar-grafana/2.9.5/images/sha256-d857951e85b41fe7172769380ffbcac126c75dc476fdcb4755623600b3eb03e0?context=explore

Please download the source package, and follow the README to build
and run the Pulsar standalone service.

Thanks
Cong Zhao

Re: [DISCUSS] Dropping the StreamingDispatcher

[Cong Zhao]

+1, I support removing it if the code isn't being used or maintained.

Thanks,
Cong Zhao

On 2023/04/04 06:47:24 Enrico Olivelli wrote:
> Hello,
> It has been a long time that we have in the Pulsar code a new
> experimental Dispatcher implementation named StreamingDispatcher.
> 
> https://github.com/apache/pulsar/pull/9056
> 
> There are many flaky tests about that feature and I believe that it
> has never been used in Production by anyone, because it happened a few
> times that we did some changes in the regular Dispatcher and
> introduced bugs on the StreamingDispacther (usually manifested as
> flaky tests)
> 
> 
> I propose to drop the StreamingDispatcher code for Pulsar 3.0.
> I don't think we need a PIP for this, it is an experimental code that
> was never delivered as a production ready feature.
> 
> If anyone is aware of users please chime in.
> 
> If anyone wants to sponsor that feature and objects in removing this
> dead code (that we still have to maintain) please help us in
> completing the feature.
> 
> On paper it is a very appealing feature, and I am disappointed in dropping it.
> On the other hand, this is dead code that we have to maintain with zero 
> benefit
> 
> Thoughts ?
> 
> Enrico
> 

Re: [VOTE] Pulsar Release 2.9.5 Candidate 2

[Cong Zhao]

Yes, Thanks Tison and Yubiao. Close this vote.

On 2023/04/03 04:34:50 Yubiao Feng wrote:
> -1 (non-binding)
> 
> - The PR 19989[1] fixes an issue that non-super role users cannot access
> the tenant's API event if disabled authorization. We should wait for this
> PR merge,
> 
> Thanks
> Yubiao Feng
> 
> [1] https://github.com/apache/pulsar/pull/19989
> 
> On Mon, Mar 27, 2023 at 11:09 PM Cong Zhao  wrote:
> 
> > This is the third release candidate for Apache Pulsar, version 2.9.5.
> >
> > This release contains 103 commits by 30 contributors.
> > https://github.com/apache/pulsar/compare/v2.9.4...v2.9.5-candidate-2
> >
> > *** Please download, test, and vote on this release. This vote will stay
> > open
> > for at least 72 hours ***
> >
> > Note that we are voting upon the source (tag), binaries are provided for
> > convenience.
> >
> > Source and binary files:
> > https://dist.apache.org/repos/dist/dev/pulsar/pulsar-2.9.5-candidate-2/
> >
> > SHA-512 checksums:
> >
> > 5e1d0c1b38441cdcb36a2f4e59ab9755b39a5c4a0136e078e91ab9bc2169016f195268692cafd6f13a45248dba2e97959b41f3cfbc8659e3cbd0bade0c954998
> > apache-pulsar-2.9.5-bin.tar.gz
> >
> >
> > 72c9f47005636c6e629dd5117b15fdc13bfd9c7efe107be77a9d55b7dfcdda2f941003eb120ea8beeffe44c41bc41c385a2e5a9cb6540d2fe83a6d04ea53a7389d
> > apache-pulsar-2.9.5-src.tar.gz
> >
> >
> > 73d286af64e189cf91c0511d360d98371b7ade1eec67bc6acf6ff766784e9e40388d3da8ae99a206369feaf398b254fff36e2206077041c37b8055ee7edde86eea
> > apache-pulsar-offloaders-2.9.5-bin.tar.gz
> >
> > Maven staging repo:
> > https://repository.apache.org/content/repositories/orgapachepulsar-1222/
> >
> > The tag to be voted upon:
> > v2.9.5-candidate-2 (c75c811ee48f51cf74f399f5b364bc1527186b34)
> > https://github.com/apache/pulsar/releases/tag/v2.9.5-candidate-2
> >
> > Pulsar's KEYS file containing PGP keys you use to sign the release:
> > https://downloads.apache.org/pulsar/KEYS
> >
> > Docker images:
> >
> > 
> >
> > https://hub.docker.com/layers/czcoder/pulsar/2.9.5/images/sha256-c6d3435d5699cb3697ee2ddc4f8a45e0ac5e35d8aefd557e280b7cf91366b981?context=explore
> >
> > 
> >
> > https://hub.docker.com/layers/czcoder/pulsar-all/2.9.5/images/sha256-a09a8e177ca7856c29dc8b9828cd293c6d44b473add20d1877bb3137b94a20c5?context=explore
> >
> > 
> >
> > https://hub.docker.com/layers/czcoder/pulsar-grafana/2.9.5/images/sha256-c43a489c65cf6c407d6c3be6fc7a001227805b1aaa9413115cf55ba11a1e329f?context=explore
> >
> > Please download the source package, and follow the README to build
> > and run the Pulsar standalone service.
> >
> > Thanks
> > Cong Zhao
> >
> 

Re: [VOTE] Pulsar Release 2.9.5 Candidate 2

[Cong Zhao]

We can remove 2.9.5 tag, it's just a tag, I don't create a github release.

On 2023/04/03 04:48:46 Michael Marshall wrote:
> I see that the 2.9.5 tag was pushed [0]. How do we handle that situation?
> 
> Thanks,
> Michael
> 
> [0] https://lists.apache.org/thread/rqojf378x9j8p58h0q8zvxg31fo07gn4
> 
> On Sun, Apr 2, 2023 at 11:35 PM Yubiao Feng
>  wrote:
> >
> > -1 (non-binding)
> >
> > - The PR 19989[1] fixes an issue that non-super role users cannot access
> > the tenant's API event if disabled authorization. We should wait for this
> > PR merge,
> >
> > Thanks
> > Yubiao Feng
> >
> > [1] https://github.com/apache/pulsar/pull/19989
> >
> > On Mon, Mar 27, 2023 at 11:09 PM Cong Zhao  wrote:
> >
> > > This is the third release candidate for Apache Pulsar, version 2.9.5.
> > >
> > > This release contains 103 commits by 30 contributors.
> > > https://github.com/apache/pulsar/compare/v2.9.4...v2.9.5-candidate-2
> > >
> > > *** Please download, test, and vote on this release. This vote will stay
> > > open
> > > for at least 72 hours ***
> > >
> > > Note that we are voting upon the source (tag), binaries are provided for
> > > convenience.
> > >
> > > Source and binary files:
> > > https://dist.apache.org/repos/dist/dev/pulsar/pulsar-2.9.5-candidate-2/
> > >
> > > SHA-512 checksums:
> > >
> > > 5e1d0c1b38441cdcb36a2f4e59ab9755b39a5c4a0136e078e91ab9bc2169016f195268692cafd6f13a45248dba2e97959b41f3cfbc8659e3cbd0bade0c954998
> > > apache-pulsar-2.9.5-bin.tar.gz
> > >
> > >
> > > 72c9f47005636c6e629dd5117b15fdc13bfd9c7efe107be77a9d55b7dfcdda2f941003eb120ea8beeffe44c41bc41c385a2e5a9cb6540d2fe83a6d04ea53a7389d
> > > apache-pulsar-2.9.5-src.tar.gz
> > >
> > >
> > > 73d286af64e189cf91c0511d360d98371b7ade1eec67bc6acf6ff766784e9e40388d3da8ae99a206369feaf398b254fff36e2206077041c37b8055ee7edde86eea
> > > apache-pulsar-offloaders-2.9.5-bin.tar.gz
> > >
> > > Maven staging repo:
> > > https://repository.apache.org/content/repositories/orgapachepulsar-1222/
> > >
> > > The tag to be voted upon:
> > > v2.9.5-candidate-2 (c75c811ee48f51cf74f399f5b364bc1527186b34)
> > > https://github.com/apache/pulsar/releases/tag/v2.9.5-candidate-2
> > >
> > > Pulsar's KEYS file containing PGP keys you use to sign the release:
> > > https://downloads.apache.org/pulsar/KEYS
> > >
> > > Docker images:
> > >
> > > 
> > >
> > > https://hub.docker.com/layers/czcoder/pulsar/2.9.5/images/sha256-c6d3435d5699cb3697ee2ddc4f8a45e0ac5e35d8aefd557e280b7cf91366b981?context=explore
> > >
> > > 
> > >
> > > https://hub.docker.com/layers/czcoder/pulsar-all/2.9.5/images/sha256-a09a8e177ca7856c29dc8b9828cd293c6d44b473add20d1877bb3137b94a20c5?context=explore
> > >
> > > 
> > >
> > > https://hub.docker.com/layers/czcoder/pulsar-grafana/2.9.5/images/sha256-c43a489c65cf6c407d6c3be6fc7a001227805b1aaa9413115cf55ba11a1e329f?context=explore
> > >
> > > Please download the source package, and follow the README to build
> > > and run the Pulsar standalone service.
> > >
> > > Thanks
> > > Cong Zhao
> > >
> 

Re: [ANNOUNCE] Qiang Zhao as new PMC member in Apache Pulsar

[Cong Zhao]

Congrats! Qiang.


Thanks,
Cong Zhao

On 2023/03/29 03:22:43 guo jiwei wrote:
> Dear Community,
> 
> We are thrilled to announce that Qiang Zhao
> (https://github.com/mattisonchao) has been invited and has accepted the
> role of member of the Apache Pulsar Project Management Committee (PMC).
> 
> Qiang has been a vital asset to our community, consistently
> demonstrating his dedication and active participation through
> significant contributions. In addition to his technical contributions,
> Qiang also plays an important role in reviewing pull requests and
> ensuring the overall quality of our project. We look forward to his
> continued contributions.
> 
> On behalf of the Pulsar PMC, we extend a warm welcome and
> congratulations to Qiang Zhao.
> 
> Best regards
> Jiwei
> 

Re: [VOTE] Pulsar Release 2.9.5 Candidate 1

[Cong Zhao]

Hi, community,

Sorry to tell everyone that we may need to abort the release
2.9.5-candidate-1 due to license problem.

Please forward to 2.9.5-candidate-2 [0]

Thanks
Cong Zhao

[0]: https://lists.apache.org/thread/nkylf740pwwvb4g0b4dtplqn4lyrp4lv

On 2023/03/20 04:00:56 Cong Zhao wrote:
> This is the third release candidate for Apache Pulsar, version 2.9.5.
> 
> This release contains 82 commits by 30 contributors.
> https://github.com/apache/pulsar/compare/v2.9.4...v2.9.5-candidate-1
> 
> *** Please download, test, and vote on this release. This vote will stay
> open
> for at least 72 hours ***
> 
> Note that we are voting upon the source (tag), binaries are provided for
> convenience.
> 
> Source and binary files:
> https://dist.apache.org/repos/dist/dev/pulsar/pulsar-2.9.5-candidate-1/
> 
> SHA-512 checksums:
> cd4e1d57db29e4e06f2bf9d036741785b10905d924a3d6a33e2de472bcf2032b3451451efa9d2fed5acffd9534edffb3189e2f90cdc8f58f523bd8288d067381
> apache-pulsar-2.9.5-bin.tar.gz
> 293b2a094a62f08223b6c7367f55fb75f79910c23cd24c96fccf71cc6deeb10707825a1b45cbece34788f0a7f3bf65996c9b6c0a4fd1fdd575cedd4828fa4e85d44
> apache-pulsar-2.9.5-src.tar.gz
> 294654ad51084c04bcfa35e07555ec628a187824538f328651d00294f4ec459898ca2506f720ecc6d29994abfd85424bd734f4b4c6dc57276223e9d51068be9
> apache-pulsar-offloaders-2.9.5-bin.tar.gz
> 
> Maven staging repo:
> https://repository.apache.org/content/repositories/orgapachepulsar-1219/
> 
> The tag to be voted upon:
> v2.9.5-candidate-1 (4322b70ccb6a17ca8d43cc60aaa02d9c86a23cd5)
> https://github.com/apache/pulsar/releases/tag/v2.9.5-candidate-1
> 
> Pulsar's KEYS file containing PGP keys you use to sign the release:
> https://downloads.apache.org/pulsar/KEYS
> 
> Docker images:
> 
> 
> https://hub.docker.com/layers/czcoder/pulsar/2.9.5/images/sha256-ff679eafaf46ec94ee211bb185e81e9767df04f4ab8ca420fc1271eab7aca8cd?context=explore
> 
> 
> https://hub.docker.com/layers/czcoder/pulsar-all/2.9.5/images/sha256-e29c4b9076e56ce11067564c12de8da993ceb5986084c868488ab7c71483286e?context=explore
> 
> 
> https://hub.docker.com/layers/czcoder/pulsar-grafana/2.9.5/images/sha256-8d34754e5192937570bfd839d7395d80a1492b5eda8dcd0a35dffddfef30b772?context=explore
> 
> Please download the source package, and follow the README to build
> and run the Pulsar standalone service.
> 

[VOTE] Pulsar Release 2.9.5 Candidate 2

[Cong Zhao]

This is the third release candidate for Apache Pulsar, version 2.9.5.

This release contains 103 commits by 30 contributors.
https://github.com/apache/pulsar/compare/v2.9.4...v2.9.5-candidate-2

*** Please download, test, and vote on this release. This vote will stay
open
for at least 72 hours ***

Note that we are voting upon the source (tag), binaries are provided for
convenience.

Source and binary files:
https://dist.apache.org/repos/dist/dev/pulsar/pulsar-2.9.5-candidate-2/

SHA-512 checksums:
5e1d0c1b38441cdcb36a2f4e59ab9755b39a5c4a0136e078e91ab9bc2169016f195268692cafd6f13a45248dba2e97959b41f3cfbc8659e3cbd0bade0c954998
apache-pulsar-2.9.5-bin.tar.gz

72c9f47005636c6e629dd5117b15fdc13bfd9c7efe107be77a9d55b7dfcdda2f941003eb120ea8beeffe44c41bc41c385a2e5a9cb6540d2fe83a6d04ea53a7389d
apache-pulsar-2.9.5-src.tar.gz

73d286af64e189cf91c0511d360d98371b7ade1eec67bc6acf6ff766784e9e40388d3da8ae99a206369feaf398b254fff36e2206077041c37b8055ee7edde86eea
apache-pulsar-offloaders-2.9.5-bin.tar.gz

Maven staging repo:
https://repository.apache.org/content/repositories/orgapachepulsar-1222/

The tag to be voted upon:
v2.9.5-candidate-2 (c75c811ee48f51cf74f399f5b364bc1527186b34)
https://github.com/apache/pulsar/releases/tag/v2.9.5-candidate-2

Pulsar's KEYS file containing PGP keys you use to sign the release:
https://downloads.apache.org/pulsar/KEYS

Docker images:


https://hub.docker.com/layers/czcoder/pulsar/2.9.5/images/sha256-c6d3435d5699cb3697ee2ddc4f8a45e0ac5e35d8aefd557e280b7cf91366b981?context=explore


https://hub.docker.com/layers/czcoder/pulsar-all/2.9.5/images/sha256-a09a8e177ca7856c29dc8b9828cd293c6d44b473add20d1877bb3137b94a20c5?context=explore


https://hub.docker.com/layers/czcoder/pulsar-grafana/2.9.5/images/sha256-c43a489c65cf6c407d6c3be6fc7a001227805b1aaa9413115cf55ba11a1e329f?context=explore

Please download the source package, and follow the README to build
and run the Pulsar standalone service.

Thanks
Cong Zhao

Re: [VOTE] PIP-254: Support configuring client version with a description suffix

[Cong Zhao]

+1 (non-binding)

Thanks,
Cong Zhao

On 2023/03/15 07:54:20 Yunze Xu wrote:
> Hi all,
> 
> This thread is to start the vote for PIP-254.
> 
> Discussion thread:
> https://lists.apache.org/thread/65cf7w76tt23sbsjnr8rpfxqf1nt9s9l
> 
> PIP link: https://github.com/apache/pulsar/issues/19705
> 
> Thanks,
> Yunze
> 

[VOTE] Pulsar Release 2.9.5 Candidate 1

[Cong Zhao]

This is the third release candidate for Apache Pulsar, version 2.9.5.

This release contains 82 commits by 30 contributors.
https://github.com/apache/pulsar/compare/v2.9.4...v2.9.5-candidate-1

*** Please download, test, and vote on this release. This vote will stay
open
for at least 72 hours ***

Note that we are voting upon the source (tag), binaries are provided for
convenience.

Source and binary files:
https://dist.apache.org/repos/dist/dev/pulsar/pulsar-2.9.5-candidate-1/

SHA-512 checksums:
cd4e1d57db29e4e06f2bf9d036741785b10905d924a3d6a33e2de472bcf2032b3451451efa9d2fed5acffd9534edffb3189e2f90cdc8f58f523bd8288d067381
apache-pulsar-2.9.5-bin.tar.gz
293b2a094a62f08223b6c7367f55fb75f79910c23cd24c96fccf71cc6deeb10707825a1b45cbece34788f0a7f3bf65996c9b6c0a4fd1fdd575cedd4828fa4e85d44
apache-pulsar-2.9.5-src.tar.gz
294654ad51084c04bcfa35e07555ec628a187824538f328651d00294f4ec459898ca2506f720ecc6d29994abfd85424bd734f4b4c6dc57276223e9d51068be9
apache-pulsar-offloaders-2.9.5-bin.tar.gz

Maven staging repo:
https://repository.apache.org/content/repositories/orgapachepulsar-1219/

The tag to be voted upon:
v2.9.5-candidate-1 (4322b70ccb6a17ca8d43cc60aaa02d9c86a23cd5)
https://github.com/apache/pulsar/releases/tag/v2.9.5-candidate-1

Pulsar's KEYS file containing PGP keys you use to sign the release:
https://downloads.apache.org/pulsar/KEYS

Docker images:


https://hub.docker.com/layers/czcoder/pulsar/2.9.5/images/sha256-ff679eafaf46ec94ee211bb185e81e9767df04f4ab8ca420fc1271eab7aca8cd?context=explore


https://hub.docker.com/layers/czcoder/pulsar-all/2.9.5/images/sha256-e29c4b9076e56ce11067564c12de8da993ceb5986084c868488ab7c71483286e?context=explore


https://hub.docker.com/layers/czcoder/pulsar-grafana/2.9.5/images/sha256-8d34754e5192937570bfd839d7395d80a1492b5eda8dcd0a35dffddfef30b772?context=explore

Please download the source package, and follow the README to build
and run the Pulsar standalone service.

Re: [DISCUSS] Release 2.9.5

[Cong Zhao]

Hello, Pulsar community:

The cherry-pick of 2.9.5 is primarily completed.
Contains 80 PRs [0].
If you have some PRs that must be included in release-2.9.5, you can reply
to me in the email.
I will wait for these PRs to be completed before releasing 2.9.5.

Thanks,
Cong

[0] 
https://github.com/apache/pulsar/pulls?q=is%3Amerged+is%3Apr+label%3Arelease%2F2.9.5+label%3Acherry-picked%2Fbranch-2.9+

On 2023/02/20 04:57:15 mattisonc...@gmail.com wrote:
> Hello, Pulsar community:
> 
> I'd like to propose releasing Apache Pulsar 2.9.5. It's been about
> two months since 2.9.4 was released.
> 
> There are 54 PRs [0] needed to cherry-pick in branch-2.9. I will
> cherry-pick these PRs for branch-2.9. Exclude some PRs that merge
> directly into branch-2.9.
> 
> There are 21 PRs [1] opened. I'll follow up on each of those PRs to
> see if they will be completed soon or will need to be pushed to 2.9.6
> 
> If you have any important fixes or any questions, please reply to this
> email, and we will evaluate whether to include them in 2.9.5
> 
> 
> Best,
> Mattison
> 
> [0] 
> https://github.com/apache/pulsar/pulls?q=is%3Apr+label%3Arelease%2F2.9.5+-label%3Acherry-picked%2Fbranch-2.9+is%3Aclosed+
> [1] 
> https://github.com/apache/pulsar/pulls?q=is%3Aopen+is%3Apr+label%3Arelease%2F2.9.5
> 

Re: [PROPOSAL] Roadmap for 3.0 release

[Cong Zhao]

+1 Looks great!

Thanks,
Cong

> 2023年2月18日 06:44，Matteo Merli  写道：
> 
> Since the LTS release model has been formally approved, I'm proposing
> the following schedule for the release:
> 
> * Tue - 2023-05-11
>  - RC-1
>  - Code Freeze -- Only critical fixes will be merged in the 3.0
> release branch. Contributors should plan to have all the changes merged in
> before this date. Exceptions should be extremely rare and strongly
> motivated.
> 
> * Tue - 2023-05-18 - RC-2
> * Tue - 2023-05-25 - RC-3
> * Tue - 2023-05-02 - Announce 3.0 Release
> 
> These dates will be published on the website to present users with a
> "roadmap" and we should commit to and respect these dates.
> 
> I also wanted to propose trying out a model where we have 3 release
> managers for all major releases.
> 
> The reasoning behind this is for this small group of people to collaborate
> and divide the tasks for the release: merging patches from the "master"
> branch, preparing RC, and testing.
> 
> Since everyone also has other work duties and unexpected tasks that can pop
> up at any time, it will help to have redundancy in the release-management
> "team", so that we can release on the exact dates.
> 
> Thanks,
> Matteo
> 
> --
> Matteo Merli
> 

Re: [PROPOSAL] Roadmap for 3.0 release

[Cong Zhao]

+1 Looks great!

Thanks,
Cong

On 2023/02/17 22:44:34 Matteo Merli wrote:
> Since the LTS release model has been formally approved, I'm proposing
> the following schedule for the release:
> 
>  * Tue - 2023-05-11
>- RC-1
>- Code Freeze -- Only critical fixes will be merged in the 3.0
> release branch. Contributors should plan to have all the changes merged in
> before this date. Exceptions should be extremely rare and strongly
> motivated.
> 
>  * Tue - 2023-05-18 - RC-2
>  * Tue - 2023-05-25 - RC-3
>  * Tue - 2023-05-02 - Announce 3.0 Release
> 
> These dates will be published on the website to present users with a
> "roadmap" and we should commit to and respect these dates.
> 
> I also wanted to propose trying out a model where we have 3 release
> managers for all major releases.
> 
> The reasoning behind this is for this small group of people to collaborate
> and divide the tasks for the release: merging patches from the "master"
> branch, preparing RC, and testing.
> 
> Since everyone also has other work duties and unexpected tasks that can pop
> up at any time, it will help to have redundancy in the release-management
> "team", so that we can release on the exact dates.
> 
> Thanks,
> Matteo
> 
> --
> Matteo Merli
> 
> 

Re: [VOTE][PIP-242] Topic name restriction

[Cong Zhao]

+1 (non-binding)

Thanks,
Cong

On 2023/02/18 08:58:26 mattisonc...@gmail.com wrote:
> Hi, All
> 
> After a fascinating discussion, I would start the vote of PIP-242.
> 
> We have chosen to drop out the `system topic` related improvement to another 
> PIP. Therefore, the current version is simple enough and it has a clear 
> boundary.
> 
> Please leave +1/-1 in this thread to join the vote. and feel free to leave 
> any concerns.
> 
> Thanks to you guys.
> 
> Best,
> Mattison
> 
> References:
> 
> • PIP https://github.com/apache/pulsar/issues/19239
> • Discussion https://lists.apache.org/thread/oz79m0f2nw059jctq4cmms74yq5n2l1m
> 
> 
> 

Re: [ANNOUNCE] Nicolò Boschi as new PMC member in Apache Pulsar

[Cong Zhao]

Congratulations!

Thanks,
Cong

On 2023/01/20 12:36:54 Lari Hotari wrote:
> Dear Community,
> 
> We are thrilled to announce that Nicolò Boschi
> (https://github.com/nicoloboschi) has been invited and has accepted the
> role of member of the Apache Pulsar Project Management Committee (PMC).
> 
> Nicolò Boschi has been a vital asset to our community, consistently
> demonstrating his dedication and active participation through
> significant contributions such as the development of the Pulsar Shell
> and numerous bug fixes, security enhancements, and improvements to
> Pulsar and its CI pipeline. In addition to his technical contributions,
> Nicolò also plays an important role in reviewing pull requests and
> ensuring the overall quality of our project. We look forward to his
> continued contributions.
> 
> On behalf of the Pulsar PMC, we extend a warm welcome and
> congratulations to Nicolò Boschi.
> 
> Best regards, Lari, on behalf of the Pulsar PMC
> 

Re: [ANNOUNCE] Bo Cong as new PMC member in Apache Pulsar

[Cong Zhao]

Congratulations!

Thanks,
Cong

On 2023/01/18 13:50:12 PengHui Li wrote:
> Hi all,
> 
> The Apache Pulsar Project Management Committee (PMC) has invited Bo Cong
> (https://github.com/congbobo184) as a member of the PMC and we are
> pleased to announce that he has accepted.
> 
> He is very active in the community in the past few years and made a lot of
> great contributions
> such as transactions and schemas.
> 
> Welcome Bo Cong to the Apache Pulsar PMC.
> 
> Best Regards,
> Penghui on behalf of the Pulsar PMC
> 

Re: [ANNOUNCE] New Committer: Baodi Shi

[Cong Zhao]

Congratulations!

Thanks,
Cong

On 2023/01/18 13:35:41 Yunze Xu wrote:
> The Project Management Committee (PMC) for Apache Pulsar has invited
> Baodi Shi (https://github.com/shibd) to become a committer and we are
> pleased to announce that he has accepted.
> 
> Being a committer enables easier contribution to the project since
> there is no need to go via the patch submission process. This should
> enable better productivity.
> 
> Welcome and congratulations, Baodi Shi!
> 
> Please join us in congratulating and welcoming Baodi Shi onboard!
> 
> Thanks,
> Yunze on behalf of the Pulsar PMC
> 

Re: [DISCUSS] Introduce oshi library to sensory OS resources

[Cong Zhao]

+1

Thanks,
Cong

On 2022/12/19 10:19:07 mattisonc...@gmail.com wrote:
> 
> Hi, All
> 
> I would like to introduce a new library oshi[1] to help Apache Pulsar sensory 
> OS resources. It can help us to get away from the complex file manipulation 
> and cross-platform compatibility issues in some operating systems.
> 
> code example:   https://github.com/apache/pulsar/pull/18984
> 
> Please feel free to left comments.
> 
> 
> Best,
> Mattison
> 
> 
> [1] https://github.com/oshi/oshi
> 

Re: [ANNOUNCE] Yunze Xu as a new PMC member in Apache Pulsar

[Cong Zhao]

Congratulations! Yunze

Best,
Cong

On 2022/12/29 12:42:36 Haiting Jiang wrote:
> Hi all,
> 
> The Apache Pulsar Project Management Committee (PMC) has invited Yunze Xu
> (https://github.com/BewareMyPower) as a member of the PMC and we are
> pleased to announce that he has accepted.
> 
> He is very active in the community in the past few years and made a lot of 
> great contributions.
> 
> Welcome Yunze to the Apache Pulsar PMC.
> 
> Best Regards,
> Haiting Jiang on behalf of the Pulsar PMC
> 

Re: [VOTE] PIP-219: Support full scan and trim ledger

[Cong Zhao]

+1(non-binding)

Thanks,
Cong

On 2022/11/01 14:28:14 linlin wrote:
> Hi folks,
> 
> I'd like to start a vote for the PIP-219: Support full scan and trim ledger
> (
> https://github.com/apache/pulsar/issues/18128)
> 
> The discussion can be found here:
> https://lists.apache.org/thread/wy8sqs2fdmw3kcdfos7t1ztpccfdmv72
> 
> Best regards.
> 
> linlin
> 

Re: [VOTE] PIP-195: New bucket based delayed message tracker

[Cong Zhao]

Hi,

The vote for PIP-195: New bucket based delayed message tracker passed with 3
+1 (binding) and 2 +1 (non-binding).

+1 (binding)
- Penghui
- Jiwei Guo
- Enrico Olivelli

+1 (non-binding)
- Ran Gao
- lordcheng

Thanks,
Cong Zhao

On 2022/08/08 06:51:31 Cong Zhao wrote:
>  Hi Pulsar Community,
> 
> I would like to start a VOTE on "New bucket based delayed message tracker"
> (PIP-195).
> 
> The proposal can be read at https://github.com/apache/pulsar/issues/16763
> and the discussion thread is available at
> https://lists.apache.org/thread/1krdhrvs803kb6rqzdh17q0f199nroz4
> 
> Voting will stay open for at least 48h.
> 
> Thanks,
> Cong Zhao
> 

Re: [VOTE] PIP-195: New bucket based delayed message tracker

[Cong Zhao]

Hi Pulsar Community,

In order to facilitate community review, I copied this proposal to Google Docs, 
please review it and complete the vote.

PIP: 
https://docs.google.com/document/d/17HE88w4WuEsLoz0DExPpRN555UC_Tc3MEIKckVWksDc/edit

On 2022/08/08 06:51:31 Cong Zhao wrote:
>  Hi Pulsar Community,
> 
> I would like to start a VOTE on "New bucket based delayed message tracker"
> (PIP-195).
> 
> The proposal can be read at https://github.com/apache/pulsar/issues/16763
> and the discussion thread is available at
> https://lists.apache.org/thread/1krdhrvs803kb6rqzdh17q0f199nroz4
> 
> Voting will stay open for at least 48h.
> 
> Thanks,
> Cong Zhao
> 

Re: [ANNOUNCE] Jiwei Guo as a new PMC member in Pulsar

[Cong Zhao]

Congratulations!

Thanks,
Cong Zhao

On 2022/08/18 11:24:01 PengHui Li wrote:
> Hi, all
> 
> I'm glad to announce that the Apache Pulsar PMC invited Jiwei Guo to join
> the
> PMC and he accepted.
> 
> Please join in celebrating!
> 
> Best,
> Penghui
> 

[VOTE] PIP-195: New bucket based delayed message tracker

[Cong Zhao]

 Hi Pulsar Community,

I would like to start a VOTE on "New bucket based delayed message tracker"
(PIP-195).

The proposal can be read at https://github.com/apache/pulsar/issues/16763
and the discussion thread is available at
https://lists.apache.org/thread/1krdhrvs803kb6rqzdh17q0f199nroz4

Voting will stay open for at least 48h.

Thanks,
Cong Zhao

Re: [DISCUSS] PIP-195: New bucket based delayed message tracker

[Cong Zhao]

Hi Penghui,
Thanks for your feed.

I have added them, please see the proposal again.

Thanks,
Cong Zhao

On 2022/07/29 05:08:10 PengHui Li wrote:
> We should also add a part for observability. Add some metrics to the delayed 
> index buckets and snapshots will help users to tune the
> configurations.
> 
> And please add a section to describe the upgrade and downgrade.
> 
> +1 for the proposal
> 
> Penghui
> On Jul 27, 2022, 23:50 +0800, Cong Zhao , wrote:
> > Hi Enrico,
> > Thanks for your feedback.
> >
> > > How do you enable this feature? A flag in broker.conf?
> > Yes, we add a flag in broker.conf to enable this feature, I will update it
> >
> > > Is it possible to enable the feature by doing a rolling upgrade of the
> > > brokers? What happens if you have brokers that are still running the old
> > > tracker?
> > I think it is possible to enable the feature by doing a rolling upgrade of 
> > the brokers, because the delayed message index in the old tracker only 
> > exists in the memory.
> >
> > > Is it possible to rollback seamlessly?
> > It is possible to rollback seamlessly, the old memory tracker will rebuild 
> > the delayed message index.
> >
> > On 2022/07/27 06:19:28 Enrico Olivelli wrote:
> > > Good proposal.
> > >
> > > How do you enable this feature? A flag in broker.conf?
> > >
> > > Is it possible to enable the feature by doing a rolling upgrade of the
> > > brokers? What happens if you have brokers that are still running the old
> > > tracker?
> > >
> > > Is it possible to rollback seamlessly?
> > >
> > > Enrico
> > >
> > > Il Mer 27 Lug 2022, 05:39 Dezhi Liu  ha scritto:
> > >
> > > > +1, good job.
> > > >
> > > > Thanks,
> > > > Dezhi
> > > >
> > > > On 2022/07/25 02:22:00 Cong Zhao wrote:
> > > > > Hello Pulsar Community,
> > > > >
> > > > >
> > > > > I would like to open a discussion here about PIP-195 : New bucket 
> > > > > based
> > > > > delayed message tracker. I look forward to your feedback.
> > > > >
> > > > >
> > > > > PIP: https://github.com/apache/pulsar/issues/16763
> > > > >
> > > > >
> > > > > Thanks,
> > > > >
> > > > > Cong Zhao
> > > > >
> > > >
> > >
> >
> > Thanks,
> > Cong Zhao
> 

Re: [DISCUSS] PIP-195: New bucket based delayed message tracker

[Cong Zhao]

Hi Enrico,
Thanks for your feedback.

> How do you enable this feature? A flag in broker.conf?
Yes, we add a flag in broker.conf to enable this feature, I will update it

> Is it possible to enable the feature by doing a rolling upgrade of the
> brokers? What happens if you have brokers that are still running the old
> tracker?
I think it is possible to enable the feature by doing a rolling upgrade of the 
brokers, because the delayed message index in the old tracker only exists in 
the memory.

> Is it possible to rollback seamlessly?
It is possible to rollback seamlessly, the old memory tracker will rebuild the 
delayed message index.

On 2022/07/27 06:19:28 Enrico Olivelli wrote:
> Good proposal.
> 
> How do you enable this feature? A flag in broker.conf?
> 
> Is it possible to enable the feature by doing a rolling upgrade of the
> brokers? What happens if you have brokers that are still running the old
> tracker?
> 
> Is it possible to rollback seamlessly?
> 
> Enrico
> 
> Il Mer 27 Lug 2022, 05:39 Dezhi Liu  ha scritto:
> 
> > +1, good job.
> >
> > Thanks,
> > Dezhi
> >
> > On 2022/07/25 02:22:00 Cong Zhao wrote:
> > > Hello Pulsar Community,
> > >
> > >
> > > I would like to open a discussion here about PIP-195 : New bucket based
> > > delayed message tracker. I look forward to your feedback.
> > >
> > >
> > > PIP: https://github.com/apache/pulsar/issues/16763
> > >
> > >
> > > Thanks,
> > >
> > > Cong Zhao
> > >
> >
> 

Thanks,
Cong Zhao

[DISCUSS] PIP-195: New bucket based delayed message tracker

[Cong Zhao]

Hello Pulsar Community,


I would like to open a discussion here about PIP-195 : New bucket based
delayed message tracker. I look forward to your feedback.


PIP: https://github.com/apache/pulsar/issues/16763


Thanks,

Cong Zhao

Re: [DISCUSS] PIP-172: Introduce the HEALTH_CHECK command in the binary protocol

[Cong Zhao]

Hi Michael,

Thanks for your feedback.

> What are the use cases where a client needs to know the cluster health
> other than for auto-failover? 
For the time being, I only think of the case of auto-failover.

> The question of cluster health is
> complicated, given that Pulsar is a distributed system, and I do not
> view the broker health check as a reliable proxy for cluster health. I
> view it as an indicator of a single broker's health at a single point
> in time.
This is indeed a problem, but I think using the `HEALTH_CHECK` command to check 
the health of the cluster for auto-failover cases should be better than just 
probing the port before.

Thanks,
Cong Zhao

On 2022/07/01 05:47:46 Michael Marshall wrote:
> Thanks for your response Cong Zhao.
> 
> > I think don't need to handle it, because the client API should be 
> > consistent with the `HEALTH_CHECK` command result, and users can retry it 
> > if they need.
> 
> I think I may be coming at this from a different perspective. A client
> application does not need to know about the health of an individual
> broker, so I don't think we should expose it explicitly through the
> client API. A client's primary concern is whether it can send/receive
> its messages, and a client's notion of cluster health should only be
> based upon whether it can send/receive its messages.
> 
> > Also, for the main problem that this proposal wants to solve (how to check 
> > a new cluster is healthy). Do you have another better idea?
> 
> What are the use cases where a client needs to know the cluster health
> other than for auto-failover? The question of cluster health is
> complicated, given that Pulsar is a distributed system, and I do not
> view the broker health check as a reliable proxy for cluster health. I
> view it as an indicator of a single broker's health at a single point
> in time.
> 
> Thanks,
> Michael
> 
> 
> On Fri, Jun 24, 2022 at 4:43 AM Cong Zhao  wrote:
> >
> > Hi Michael,
> >
> > > I think the current PIP might need some clarification on how errors
> > > are handled. For example, if a single broker fails to respond because
> > > it was being restarted, how would the client handle that kind of
> > > failure with this feature?
> >
> > I think don't need to handle it, because the client API should be 
> > consistent with the `HEALTH_CHECK` command result, and users can retry it 
> > if they need.
> >
> > > I wasn't suggesting that the client would need to ask the broker for
> > > each of the producers/consumers, but rather that the client would
> > > monitor producers/consumers locally and make decisions about cluster
> > > health. For example, if a producer cannot connect to its target topic
> > > after some amount of time or some number of retries, or if a producer
> > > can connect but cannot publish a message successfully within some
> > > amount of time, then the client could consider the cluster to be
> > > unhealthy.
> > >
> > > > This proposal mainly provides a means to check whether there is 
> > > > available topic in the cluster, and I think this is meaningful in most 
> > > > cases.
> > >
> > > The client will discover if one of its targeted topics is unavailable,
> > > so instead of monitoring the broker's health check topic, I think the
> > > client should monitor/failover when a targeted topic is "unavailable"
> > > for some configured length of time.
> > >
> > > I support making the auto-failover logic more robust, but I don't
> > > think the broker health check is the right signal to use for overall
> > > cluster health. In my view, the broker's health check is meant to
> > > signal to orchestrators (like Kubernetes) when a broker ought to be
> > > restarted.
> >
> > For the currently connected cluster, we really can't think the current 
> > topic is unavailable just because the `HEALTH_CHECK` command result is 
> > unhealthy, the current means for auto-failover are relatively rude. I think 
> > we improve it by adding extra measures such as mentioned above to you, but 
> > this doesn't fall within the scope of the proposal.
> >
> > Also, for the main problem that this proposal wants to solve (how to check 
> > a new cluster is healthy). Do you have another better idea?
> >
> > Thanks,
> > Cong Zhao
> >
> > On 2022/06/24 05:25:46 Michael Marshall wrote:
> > > Thanks for your replies Cong Zhao.
> > >
> > > I think the current PIP might need some clarification on how errors
> > > are handled. 

Re: [VOTE] PIP-168: Support zero-copy of NIC to NIC on Proxy

[Cong Zhao]

I forgot to close this vote.

I'm closing the vote.
The proposal has been accepted with 4 binding +1, non-binding +1 votes.

Thanks,
Cong Zhao

On 2022/05/26 13:40:30 zhaocong wrote:
> Hi Pulsar Community,
> 
> 
> I would like to start a VOTE on "Support zero-copy of NIC to NIC on Proxy"
> (PIP-168).
> 
> 
> The proposal can be read at https://github.com/apache/pulsar/issues/15631
> 
> and the discussion thead is available at
> 
> https://lists.apache.org/thread/gjys9tvbd5hy28mbkbcq7wkqfldycn7v
> 
> 
> Voting will stay open for at least 48h.
> 
> 
> Thanks,
> 
> Cong Zhao
> 

Re: [VOTE] PIP-177: Add the classLoader field for SchemaDefinition

[Cong Zhao]

Thanks for your participation.
I'm closing the vote.
The proposal has been accepted with 5 binding +1, non-binding +2 votes.

On 2022/06/17 01:21:14 Cong Zhao wrote:
> Hi Pulsar Community,
> 
> 
> I would like to start a VOTE on "Add the classLoader field for
> SchemaDefinition" (PIP-177).
> 
> 
> The proposal can be read at https://github.com/apache/pulsar/issues/16058
> 
> and the discussion thead is available at
> 
> https://lists.apache.org/thread/3wjvmpzo3pq1ff62f4cops7ckyrcgfhf
> 
> 
> Voting will stay open for at least 48h.
> 
> 
> Thanks,
> 
> Cong Zhao
> 

Re: [DISCUSS] PIP-172: Introduce the HEALTH_CHECK command in the binary protocol

[Cong Zhao]

Hi Michael,

> I think the current PIP might need some clarification on how errors
> are handled. For example, if a single broker fails to respond because
> it was being restarted, how would the client handle that kind of
> failure with this feature?

I think don't need to handle it, because the client API should be consistent 
with the `HEALTH_CHECK` command result, and users can retry it if they need.

> I wasn't suggesting that the client would need to ask the broker for
> each of the producers/consumers, but rather that the client would
> monitor producers/consumers locally and make decisions about cluster
> health. For example, if a producer cannot connect to its target topic
> after some amount of time or some number of retries, or if a producer
> can connect but cannot publish a message successfully within some
> amount of time, then the client could consider the cluster to be
> unhealthy.
> 
> > This proposal mainly provides a means to check whether there is available 
> > topic in the cluster, and I think this is meaningful in most cases.
> 
> The client will discover if one of its targeted topics is unavailable,
> so instead of monitoring the broker's health check topic, I think the
> client should monitor/failover when a targeted topic is "unavailable"
> for some configured length of time.
> 
> I support making the auto-failover logic more robust, but I don't
> think the broker health check is the right signal to use for overall
> cluster health. In my view, the broker's health check is meant to
> signal to orchestrators (like Kubernetes) when a broker ought to be
> restarted.

For the currently connected cluster, we really can't think the current topic is 
unavailable just because the `HEALTH_CHECK` command result is unhealthy, the 
current means for auto-failover are relatively rude. I think we improve it by 
adding extra measures such as mentioned above to you, but this doesn't fall 
within the scope of the proposal.  

Also, for the main problem that this proposal wants to solve (how to check a 
new cluster is healthy). Do you have another better idea?

Thanks,
Cong Zhao

On 2022/06/24 05:25:46 Michael Marshall wrote:
> Thanks for your replies Cong Zhao.
> 
> I think the current PIP might need some clarification on how errors
> are handled. For example, if a single broker fails to respond because
> it was being restarted, how would the client handle that kind of
> failure with this feature?
> 
> > This is a good definition of cluster health, but we can't check all topics 
> > that would add a lot of load on cleint and broker.
> 
> I wasn't suggesting that the client would need to ask the broker for
> each of the producers/consumers, but rather that the client would
> monitor producers/consumers locally and make decisions about cluster
> health. For example, if a producer cannot connect to its target topic
> after some amount of time or some number of retries, or if a producer
> can connect but cannot publish a message successfully within some
> amount of time, then the client could consider the cluster to be
> unhealthy.
> 
> > This proposal mainly provides a means to check whether there is available 
> > topic in the cluster, and I think this is meaningful in most cases.
> 
> The client will discover if one of its targeted topics is unavailable,
> so instead of monitoring the broker's health check topic, I think the
> client should monitor/failover when a targeted topic is "unavailable"
> for some configured length of time.
> 
> I support making the auto-failover logic more robust, but I don't
> think the broker health check is the right signal to use for overall
> cluster health. In my view, the broker's health check is meant to
> signal to orchestrators (like Kubernetes) when a broker ought to be
> restarted.
> 
> Thanks,
> Michael
> 
> 
> On Thu, Jun 23, 2022 at 12:35 AM Cong Zhao  wrote:
> >
> > Hi Michael,
> >
> > Thanks for your feedback.
> >
> > > I define a client's primary cluster as "healthy" when it is "healthy"
> > for all of its producers and consumers. I define a healthy producer as
> > one that can connect to a topic and publish messages within certain
> > latency and throughput thresholds (configured by the user), and I
> > define a healthy consumer as one that can connect to a topic and
> > consume messages when there are messages to be consumed (possibly
> > within a certain latency?).
> >
> > This is a good definition of cluster health, but we can't check all topics 
> > that would add a lot of load on cleint and broker.
> >
> > > By the above definitions, I don't think the broker's health check will
> > give

Re: [DISCUSS] PIP-172: Introduce the HEALTH_CHECK command in the binary protocol

[Cong Zhao]

Hi Michael,

Thanks for your feedback.

> I define a client's primary cluster as "healthy" when it is "healthy"
for all of its producers and consumers. I define a healthy producer as
one that can connect to a topic and publish messages within certain
latency and throughput thresholds (configured by the user), and I
define a healthy consumer as one that can connect to a topic and
consume messages when there are messages to be consumed (possibly
within a certain latency?).

This is a good definition of cluster health, but we can't check all topics that 
would add a lot of load on cleint and broker.

> By the above definitions, I don't think the broker's health check will
give us the right notion of "healthy" because that health check
monitors producing/consuming to/from the health check topic, not the
client's target topics. One primary difference is that a health check
topic could have a different persistence policy, which means the
client could incorrectly classify the broker as healthy when there
aren't enough available bookies for a producer's target topic.

This proposal mainly provides a means to check whether there is available topic 
in the cluster, and I think this is meaningful in most cases.

I think if the client implementation doesn't meet the user's needs, they can 
also override the `healthCheck` method based on the `HEALTH_CHECK` command.

Thanks,
Cong Zhao

On 2022/06/22 19:06:25 Michael Marshall wrote:
> I'd like to clarify the motivation for this PIP. My understanding is
> that the primary motivation is to give clients a robust way to
> classify a cluster as "healthy". The initial beneficiary of this
> feature is the auto failover use case. I think the feature makes
> sense, but before using the broker's concept of "healthy" as defined
> in the broker health check, I think we should define what constitutes
> a "healthy cluster" from the client's perspective.
> 
> I define a client's primary cluster as "healthy" when it is "healthy"
> for all of its producers and consumers. I define a healthy producer as
> one that can connect to a topic and publish messages within certain
> latency and throughput thresholds (configured by the user), and I
> define a healthy consumer as one that can connect to a topic and
> consume messages when there are messages to be consumed (possibly
> within a certain latency?).
> 
> By the above definitions, I don't think the broker's health check will
> give us the right notion of "healthy" because that health check
> monitors producing/consuming to/from the health check topic, not the
> client's target topics. One primary difference is that a health check
> topic could have a different persistence policy, which means the
> client could incorrectly classify the broker as healthy when there
> aren't enough available bookies for a producer's target topic.
> 
> The broker health check also includes checks that we probably don't
> want to use to classify whole clusters as "unhealthy". For example, if
> the broker is deadlocked, it will be considered unhealthy. In
> Kubernetes, that broker will be restarted "soon", and the topics will
> be scheduled to another broker. I probably wouldn't consider a
> whole cluster as "unhealthy" because a single broker was deadlocked.
> Instead, I'd consider a cluster unhealthy when latency/throughput are
> not meeting expectations, which could happen because a broker is
> deadlocked. Further, there is a chance that the deadlock in the broker
> didn't affect the client's producers and consumers, which is yet
> another reason not to failover to another cluster based on a failed
> broker health check.
> 
> I look forward to hearing your definitions of client health.
> 
> Thanks,
> Michael
> 
> 
> 
> On Wed, Jun 22, 2022 at 8:30 AM Cong Zhao  wrote:
> >
> > Yes, there may have multiple clients request the HC at the same time in the 
> > AutoFailover case, so we should add some cache to reduce broker load.
> >
> > On 2022/06/22 12:55:49 Enrico Olivelli wrote:
> > > Il giorno mer 22 giu 2022 alle ore 14:45 Cong Zhao
> > >  ha scritto:
> > > >
> > > > Hi Enrico,
> > > >
> > > > > Also, I would like to understand in which usecase you can use the
> > > > > binary endpoint and not the HTTP endpoint.
> > > >
> > > > We can't use the HTTP endpoint when the client did not have the admin 
> > > > auth to do a health check. but we need it in some cases such as auto 
> > > > failover on the client-side 
> > > > (https://github.com/apache/pulsar/pull/13316#discussion_r773313991)
> > > AutoFailover is a valid use case

Re: [DISCUSS] PIP-172: Introduce the HEALTH_CHECK command in the binary protocol

[Cong Zhao]

Yes, there may have multiple clients request the HC at the same time in the 
AutoFailover case, so we should add some cache to reduce broker load.

On 2022/06/22 12:55:49 Enrico Olivelli wrote:
> Il giorno mer 22 giu 2022 alle ore 14:45 Cong Zhao
>  ha scritto:
> >
> > Hi Enrico,
> >
> > > Also, I would like to understand in which usecase you can use the
> > > binary endpoint and not the HTTP endpoint.
> >
> > We can't use the HTTP endpoint when the client did not have the admin auth 
> > to do a health check. but we need it in some cases such as auto failover on 
> > the client-side 
> > (https://github.com/apache/pulsar/pull/13316#discussion_r773313991)
> AutoFailover is a valid use case for me.
> Thanks
> 
> >
> > > Health Check is good for scripts and for probes, I don't expect a
> > > "client application" to run the HC
> >
> > Adding a health check API to the client-side just to make it easier to use 
> > this feature, this check still works on broker.
> 
> makes sense, but usually the HC, like in k8s or in other environments
> is run every X seconds and usually not concurrently
> 
> if you have multiple (tens? hundreds?) of Pulsar clients that require
> the HC, this will be a big problem,
> is this the reason why you want to add some cache to the response of the HC ?
> 
> Enrico
> 
> 
> >
> >
> > On 2022/06/22 10:19:52 Enrico Olivelli wrote:
> > > I believe that this proposal is too broad.
> > >
> > > the PIP reads about:
> > > - adding HEALTHCHECK to the binary protocol
> > > - add a HEALTHCHECK cache on the broker
> > >
> > > Also, I would like to understand in which usecase you can use the
> > > binary endpoint and not the HTTP endpoint.
> > >
> > > Health Check is good for scripts and for probes, I don't expect a
> > > "client application" to run the HC
> > >
> > > Can you please illustrate some practical use cases?
> > >
> > > Enric
> > >
> > > Il giorno mer 8 giu 2022 alle ore 05:22 zhaocong 
> > > ha scritto:
> > > >
> > > > Hello Pulsar Community,
> > > >
> > > >
> > > > Here is a PIP to introduce the HEALTH_CHECK command in the binary 
> > > > protocol.
> > > > I look forward to your feedback.
> > > >
> > > >
> > > > PIP: https://github.com/apache/pulsar/issues/15859
> > > >
> > > >
> > > > Thanks,
> > > >
> > > > Cong Zhao
> > >
> 

Re: [DISCUSS] PIP-172: Introduce the HEALTH_CHECK command in the binary protocol

[Cong Zhao]

Hi Enrico,

> Also, I would like to understand in which usecase you can use the
> binary endpoint and not the HTTP endpoint.

We can't use the HTTP endpoint when the client did not have the admin auth to 
do a health check. but we need it in some cases such as auto failover on the 
client-side (https://github.com/apache/pulsar/pull/13316#discussion_r773313991)

> Health Check is good for scripts and for probes, I don't expect a
> "client application" to run the HC

Adding a health check API to the client-side just to make it easier to use this 
feature, this check still works on broker.


On 2022/06/22 10:19:52 Enrico Olivelli wrote:
> I believe that this proposal is too broad.
> 
> the PIP reads about:
> - adding HEALTHCHECK to the binary protocol
> - add a HEALTHCHECK cache on the broker
> 
> Also, I would like to understand in which usecase you can use the
> binary endpoint and not the HTTP endpoint.
> 
> Health Check is good for scripts and for probes, I don't expect a
> "client application" to run the HC
> 
> Can you please illustrate some practical use cases?
> 
> Enric
> 
> Il giorno mer 8 giu 2022 alle ore 05:22 zhaocong 
> ha scritto:
> >
> > Hello Pulsar Community,
> >
> >
> > Here is a PIP to introduce the HEALTH_CHECK command in the binary protocol.
> > I look forward to your feedback.
> >
> >
> > PIP: https://github.com/apache/pulsar/issues/15859
> >
> >
> > Thanks,
> >
> > Cong Zhao
> 

[VOTE] PIP-172: Introduce the HEALTH_CHECK command in the binary protocol

[Cong Zhao]

Hi Pulsar Community,

I would like to start a VOTE on "Introduce the HEALTH_CHECK command in the
binary protocol" (PIP-172).

The proposal can be read at https://github.com/apache/pulsar/issues/15859
and the discussion thead is available at
https://lists.apache.org/thread/6td3wyfybsys4vf1tgf7fxy36w10nb5k

Voting will stay open for at least 48h.

Thanks,
Cong Zhao

[VOTE] PIP-177: Add the classLoader field for SchemaDefinition

[Cong Zhao]

Hi Pulsar Community,


I would like to start a VOTE on "Add the classLoader field for
SchemaDefinition" (PIP-177).


The proposal can be read at https://github.com/apache/pulsar/issues/16058

and the discussion thead is available at

https://lists.apache.org/thread/3wjvmpzo3pq1ff62f4cops7ckyrcgfhf


Voting will stay open for at least 48h.


Thanks,

Cong Zhao

Re: [DISCUSS] PIP-177: Add the classLoader field for SchemaDefinition

[Cong Zhao]

The priority of the classLoader field will be higher than by the 
pojoClass.getClassLoader() if with non JSON Schemas.

On 2022/06/14 18:54:15 Enrico Olivelli wrote:
> Hello,
> how will it work with non JSON Schemas ?
> 
> Enrico
> 
> Il giorno mar 14 giu 2022 alle ore 17:57 Cong Zhao
>  ha scritto:
> >
> > Hello Pulsar Community,
> >
> >
> > Here is a PIP to add the classLoader field for SchemaDefinition. I look
> > forward to your feedback.
> >
> >
> > PIP: https://github.com/apache/pulsar/issues/16058
> >
> >
> > Thanks,
> >
> > Cong Zhao
> 

[DISCUSS] PIP-177: Add the classLoader field for SchemaDefinition

[Cong Zhao]

Hello Pulsar Community,


Here is a PIP to add the classLoader field for SchemaDefinition. I look
forward to your feedback.


PIP: https://github.com/apache/pulsar/issues/16058


Thanks,

Cong Zhao

Re: [VOTE] PIP-168: Support zero-copy of NIC to NIC on Proxy

[Cong Zhao]

ok, I have added the new configuration to the proposal.

On 2022/05/31 05:14:13 PengHui Li wrote:
> +1
> 
> Could you please also add the new configuration to the proposal?
> I see you added in the PR https://github.com/apache/pulsar/pull/15678/files
> but not present in the proposal
> 
> Thanks,
> Penghui
> 
> On Thu, May 26, 2022 at 9:40 PM zhaocong  wrote:
> 
> > Hi Pulsar Community,
> >
> >
> > I would like to start a VOTE on "Support zero-copy of NIC to NIC on Proxy"
> > (PIP-168).
> >
> >
> > The proposal can be read at https://github.com/apache/pulsar/issues/15631
> >
> > and the discussion thead is available at
> >
> > https://lists.apache.org/thread/gjys9tvbd5hy28mbkbcq7wkqfldycn7v
> >
> >
> > Voting will stay open for at least 48h.
> >
> >
> > Thanks,
> >
> > Cong Zhao
> >
> 

[DISCUSS] PIP-168: Support zero-copy of NIC to NIC on Proxy

[Cong Zhao]

Hello Pulsar Community,


Here is a PIP to support zero-copy of NIC to NIC on the proxy server. I
look forward to your feedback.


PIP: https://github.com/apache/pulsar/issues/15631


Thanks,

Cong Zhao

Re: [DISCUSS] PIP-168: Support zero-copy of NIC to NIC on Proxy

[Cong Zhao]

sorry, I made a mistake. I'll send another one

On Fri, May 20, 2022 at 10:09 AM Cong Zhao  wrote:

> Hello Pulsar Community,
>
> Here is a PIP to support zero-copy of NIC to NIC on the proxy server. I
> look forward to your feedback.
>
> PIP: https://github.com/apache/pulsar/issues/15597 Thanks, Cong Zhao
>

[DISCUSS] PIP-168: Support zero-copy of NIC to NIC on Proxy

[Cong Zhao]

Hello Pulsar Community,

Here is a PIP to support zero-copy of NIC to NIC on the proxy server. I
look forward to your feedback.

PIP: https://github.com/apache/pulsar/issues/15597 Thanks, Cong Zhao