Re: [DISCUSS] Separate the Python Functions from the installation of the Python client

[Yunze Xu]

Hi Pengcheng,

> I think there is no need to setup a "functions" lib?

Yes. Even before 3.2.0 (not released yet), users can still set up the
functions by installing `pulsar-client` without the `functions` extra
component and installing the other dependencies manually.

Thanks,
Yunze

On Thu, Apr 20, 2023 at 10:49 AM Pengcheng Jiang
 wrote:
>
> Hi Yunze,
>
> +1 for separating
>
> And for the migration progress:
> 1. since the `pulsar-functions` lib doesn't have any code but just a list
> of dependencies, I think there is no need to setup a "functions" lib?
> 2. we should update docs about what dependencies are required for Python
> functions
> 3. the Dockerfile of Pulsar image should be updated to install the required
> python dependencies so it can start python function in process runtime mode
> 4. the python runner image's Dockerfile should be updated to install the
> required python dependencies so it can run in k8s runtime mode
>
> Regards,
> Pengcheng Jiang
>
>
> Yunze Xu  于2023年4月20日周四 10:46写道：
>
> > Hi Neng,
> >
> > I think currently a simple solution is to document which dependencies
> > should users install to use the Python Functions.
> >
> > Before:
> >
> > ```
> > # For version < 3.2.0
> > pip install pulsar-client[functions]
> > ```
> >
> > Now:
> >
> > ```
> > # For version >= 3.2.0
> > pip install pulsar-client
> > pip install grpc-io
> > # Other dependencies...
> > ```
> >
> > In future, we can provide a separate PyPI package like `pulsar-functions`.
> >
> > BTW, currently the functions extra dependencies cannot be installed
> > for Python 3.10 and 3.11. I have tested the following images with the
> > `pip install 'pulsar-client[functions]==3.1.0'` command.
> > - python:3.11.2-bullseye
> > - python:3.10.11-bullseye
> >
> > The reason is the version incompatibility of grpcio.
> >
> > > ERROR: Failed building wheel for grpcio
> >
> > Though I only tested 3.1.0, since the dependencies never changed, they
> > should also fail for older Python client releases like 2.10.0.
> >
> > Thanks,
> > Yunze
> >
> > On Thu, Apr 20, 2023 at 12:26 AM Neng Lu  wrote:
> > >
> > > Hi Yunze,
> > >
> > > +1 for separating Python client and Python Pulsar Functions pip
> > installation.
> > > On the Java side, the client lib and functions lib are also published
> > separately.
> > >
> > > My concern is how the migration progress should look like,
> > > 1. we need to set up functions lib so that users can install it using
> > `pip install pulsar-functions`
> > > 2. the current `pip install pulsar-client[functions]` should prompt user
> > to use the new way
> > > 3. all docs need to be updated
> > > 4. for historical versions, what can we do?
> > >
> > >
> > >
> > > On 2023/04/19 15:23:49 Yunze Xu wrote:
> > > > Hi all,
> > > >
> > > > The Python client has been separated since PIP-209 [1] and now the
> > > > Python client is maintained in a separated repository [2]. However,
> > > > the Python Function is still maintained in the main repo [3].
> > > >
> > > > Currently, we can install the Python client with the following ways:
> > > > 1. pip install pulsar-client
> > > > 2. pip install pulsar-client[avro]
> > > > 3. pip install pulsar-client[functions]
> > > > 4. pip install pulsar-client[all]
> > > >
> > > > See [4] for the difference. However, for the 3rd and 4th ways, it
> > > > installs all the dependencies required by the Python Functions.
> > > > However, they are broken for the recent releases because of the
> > > > outdated dependencies [5]. However, these dependencies are from the
> > > > Python Functions [3], not the Python client library itself. Also,
> > > > there are no tests in the Python client repo [2] for these functions
> > > > so these dependencies cannot be verified.
> > > >
> > > > IMO, these dependencies should be maintained in the directory of the
> > > > Python Functions. We should not rely on the Python client to install
> > > > the dependencies for the Python Functions.
> > > >
> > > > Therefore, my suggestion is to drop the 3rd and 4th installation
> > > > methods in future releases of the Python client. After that, we should
> > > > update the scripts in the main repo to install the Python Functions in
> > > > [3].
> > > >
> > > > I'm not familiar with the Pulsar Functions, so feel free to show your
> > > > suggestions if any of you have any concerns.
> > > >
> > > > [1] https://github.com/apache/pulsar/pull/17881
> > > > [2] http://github.com/apache/pulsar-client-python
> > > > [3]
> > https://github.com/apache/pulsar/tree/master/pulsar-functions/instance/src/main/python
> > > > [4] https://pulsar.apache.org/docs/2.11.x/client-libraries-python/
> > > > [5]
> > https://github.com/apache/pulsar-client-python/blob/a6476d9c45508f55a7af4b25001038a8e3a27489/setup.py#L80-L88
> > > >
> > > > Thanks,
> > > > Yunze
> > > >
> >
>
>
> --
>
> 
>
> Pengcheng Jiang
>
> Software Engineer
>
> e: pengcheng.ji...@streamnative.io
>
> p: 13540631948
>
> streamnative.io
>
> 

Re: [DISCUSS] Separate the Python Functions from the installation of the Python client

[Pengcheng Jiang]

Hi Yunze,

+1 for separating

And for the migration progress:
1. since the `pulsar-functions` lib doesn't have any code but just a list
of dependencies, I think there is no need to setup a "functions" lib?
2. we should update docs about what dependencies are required for Python
functions
3. the Dockerfile of Pulsar image should be updated to install the required
python dependencies so it can start python function in process runtime mode
4. the python runner image's Dockerfile should be updated to install the
required python dependencies so it can run in k8s runtime mode

Regards,
Pengcheng Jiang


Yunze Xu  于2023年4月20日周四 10:46写道：

> Hi Neng,
>
> I think currently a simple solution is to document which dependencies
> should users install to use the Python Functions.
>
> Before:
>
> ```
> # For version < 3.2.0
> pip install pulsar-client[functions]
> ```
>
> Now:
>
> ```
> # For version >= 3.2.0
> pip install pulsar-client
> pip install grpc-io
> # Other dependencies...
> ```
>
> In future, we can provide a separate PyPI package like `pulsar-functions`.
>
> BTW, currently the functions extra dependencies cannot be installed
> for Python 3.10 and 3.11. I have tested the following images with the
> `pip install 'pulsar-client[functions]==3.1.0'` command.
> - python:3.11.2-bullseye
> - python:3.10.11-bullseye
>
> The reason is the version incompatibility of grpcio.
>
> > ERROR: Failed building wheel for grpcio
>
> Though I only tested 3.1.0, since the dependencies never changed, they
> should also fail for older Python client releases like 2.10.0.
>
> Thanks,
> Yunze
>
> On Thu, Apr 20, 2023 at 12:26 AM Neng Lu  wrote:
> >
> > Hi Yunze,
> >
> > +1 for separating Python client and Python Pulsar Functions pip
> installation.
> > On the Java side, the client lib and functions lib are also published
> separately.
> >
> > My concern is how the migration progress should look like,
> > 1. we need to set up functions lib so that users can install it using
> `pip install pulsar-functions`
> > 2. the current `pip install pulsar-client[functions]` should prompt user
> to use the new way
> > 3. all docs need to be updated
> > 4. for historical versions, what can we do?
> >
> >
> >
> > On 2023/04/19 15:23:49 Yunze Xu wrote:
> > > Hi all,
> > >
> > > The Python client has been separated since PIP-209 [1] and now the
> > > Python client is maintained in a separated repository [2]. However,
> > > the Python Function is still maintained in the main repo [3].
> > >
> > > Currently, we can install the Python client with the following ways:
> > > 1. pip install pulsar-client
> > > 2. pip install pulsar-client[avro]
> > > 3. pip install pulsar-client[functions]
> > > 4. pip install pulsar-client[all]
> > >
> > > See [4] for the difference. However, for the 3rd and 4th ways, it
> > > installs all the dependencies required by the Python Functions.
> > > However, they are broken for the recent releases because of the
> > > outdated dependencies [5]. However, these dependencies are from the
> > > Python Functions [3], not the Python client library itself. Also,
> > > there are no tests in the Python client repo [2] for these functions
> > > so these dependencies cannot be verified.
> > >
> > > IMO, these dependencies should be maintained in the directory of the
> > > Python Functions. We should not rely on the Python client to install
> > > the dependencies for the Python Functions.
> > >
> > > Therefore, my suggestion is to drop the 3rd and 4th installation
> > > methods in future releases of the Python client. After that, we should
> > > update the scripts in the main repo to install the Python Functions in
> > > [3].
> > >
> > > I'm not familiar with the Pulsar Functions, so feel free to show your
> > > suggestions if any of you have any concerns.
> > >
> > > [1] https://github.com/apache/pulsar/pull/17881
> > > [2] http://github.com/apache/pulsar-client-python
> > > [3]
> https://github.com/apache/pulsar/tree/master/pulsar-functions/instance/src/main/python
> > > [4] https://pulsar.apache.org/docs/2.11.x/client-libraries-python/
> > > [5]
> https://github.com/apache/pulsar-client-python/blob/a6476d9c45508f55a7af4b25001038a8e3a27489/setup.py#L80-L88
> > >
> > > Thanks,
> > > Yunze
> > >
>


-- 



Pengcheng Jiang

Software Engineer

e: pengcheng.ji...@streamnative.io

p: 13540631948

streamnative.io

Re: [DISCUSS] Separate the Python Functions from the installation of the Python client

[Yunze Xu]

Hi Neng,

I think currently a simple solution is to document which dependencies
should users install to use the Python Functions.

Before:

```
# For version < 3.2.0
pip install pulsar-client[functions]
```

Now:

```
# For version >= 3.2.0
pip install pulsar-client
pip install grpc-io
# Other dependencies...
```

In future, we can provide a separate PyPI package like `pulsar-functions`.

BTW, currently the functions extra dependencies cannot be installed
for Python 3.10 and 3.11. I have tested the following images with the
`pip install 'pulsar-client[functions]==3.1.0'` command.
- python:3.11.2-bullseye
- python:3.10.11-bullseye

The reason is the version incompatibility of grpcio.

> ERROR: Failed building wheel for grpcio

Though I only tested 3.1.0, since the dependencies never changed, they
should also fail for older Python client releases like 2.10.0.

Thanks,
Yunze

On Thu, Apr 20, 2023 at 12:26 AM Neng Lu  wrote:
>
> Hi Yunze,
>
> +1 for separating Python client and Python Pulsar Functions pip installation.
> On the Java side, the client lib and functions lib are also published 
> separately.
>
> My concern is how the migration progress should look like,
> 1. we need to set up functions lib so that users can install it using `pip 
> install pulsar-functions`
> 2. the current `pip install pulsar-client[functions]` should prompt user to 
> use the new way
> 3. all docs need to be updated
> 4. for historical versions, what can we do?
>
>
>
> On 2023/04/19 15:23:49 Yunze Xu wrote:
> > Hi all,
> >
> > The Python client has been separated since PIP-209 [1] and now the
> > Python client is maintained in a separated repository [2]. However,
> > the Python Function is still maintained in the main repo [3].
> >
> > Currently, we can install the Python client with the following ways:
> > 1. pip install pulsar-client
> > 2. pip install pulsar-client[avro]
> > 3. pip install pulsar-client[functions]
> > 4. pip install pulsar-client[all]
> >
> > See [4] for the difference. However, for the 3rd and 4th ways, it
> > installs all the dependencies required by the Python Functions.
> > However, they are broken for the recent releases because of the
> > outdated dependencies [5]. However, these dependencies are from the
> > Python Functions [3], not the Python client library itself. Also,
> > there are no tests in the Python client repo [2] for these functions
> > so these dependencies cannot be verified.
> >
> > IMO, these dependencies should be maintained in the directory of the
> > Python Functions. We should not rely on the Python client to install
> > the dependencies for the Python Functions.
> >
> > Therefore, my suggestion is to drop the 3rd and 4th installation
> > methods in future releases of the Python client. After that, we should
> > update the scripts in the main repo to install the Python Functions in
> > [3].
> >
> > I'm not familiar with the Pulsar Functions, so feel free to show your
> > suggestions if any of you have any concerns.
> >
> > [1] https://github.com/apache/pulsar/pull/17881
> > [2] http://github.com/apache/pulsar-client-python
> > [3] 
> > https://github.com/apache/pulsar/tree/master/pulsar-functions/instance/src/main/python
> > [4] https://pulsar.apache.org/docs/2.11.x/client-libraries-python/
> > [5] 
> > https://github.com/apache/pulsar-client-python/blob/a6476d9c45508f55a7af4b25001038a8e3a27489/setup.py#L80-L88
> >
> > Thanks,
> > Yunze
> >

Re: [VOTE] Pulsar Release 2.9.5 Candidate 3

[Cong Zhao]

Close this vote with 4 +1 (binding)
- Mattison
- Penghui
- Jiwei
- Yunze

On 2023/04/10 07:38:09 Cong Zhao wrote:
> This is the third release candidate for Apache Pulsar, version 2.9.5.
> 
> This release contains 105 commits by 32 contributors.
> https://github.com/apache/pulsar/compare/v2.9.4...v2.9.5-candidate-3
> 
> *** Please download, test, and vote on this release. This vote will stay
> open
> for at least 72 hours ***
> 
> Note that we are voting upon the source (tag), binaries are provided for
> convenience.
> 
> Source and binary files:
> https://dist.apache.org/repos/dist/dev/pulsar/pulsar-2.9.5-candidate-3/
> 
> SHA-512 checksums:
> c5f1b06f2f7249616a07b732daa036546591c1aa9e9f37e88ddce6d3eaf3d78b2c6da548ef42946d20d70f9397cd5b55ea4e01fe5cb4aa96fb4608bd62635f67
> apache-pulsar-2.9.5-bin.tar.gz
> 
> 72dee0fb642a269c5d0aedfa3e56aee503f56af319b67e4628f39da8cdd44f0bbdcd019e121eafca32b7b7665959770a3d91f0618d157837808da9511e9011a40f
> apache-pulsar-2.9.5-src.tar.gz
> 
> 738478bbcf323080487f5645b4c1edb9f45d126a8a2cd7e8c5678c8569ec7b21042ecb6fdd4796b65c6a24bd1e5ad082f5c0d15eb1a64408be4f4c53a06d3ef660
> apache-pulsar-offloaders-2.9.5-bin.tar.gz
> 
> Maven staging repo:
> https://repository.apache.org/content/repositories/orgapachepulsar-1223/
> 
> The tag to be voted upon:
> v2.9.5-candidate-3 (7337470f33586aa639854266efe95c2fa32f48db)
> https://github.com/apache/pulsar/releases/tag/v2.9.5-candidate-3
> 
> Pulsar's KEYS file containing PGP keys you use to sign the release:
> https://downloads.apache.org/pulsar/KEYS
> 
> Docker images:
> 
> 
> https://hub.docker.com/layers/czcoder/pulsar/2.9.5/images/sha256-9bcbad78a65a09b3fe5d3acd158574ad0eda4531cc8c49db16760a2ed1364070?context=explore
> 
> 
> https://hub.docker.com/layers/czcoder/pulsar-all/2.9.5/images/sha256-94e50aa1d5b91be9698c959c787eaff0677bb3c65dd1a578b1a641fa5a1d1715?context=explore
> 
> 
> https://hub.docker.com/layers/czcoder/pulsar-grafana/2.9.5/images/sha256-d857951e85b41fe7172769380ffbcac126c75dc476fdcb4755623600b3eb03e0?context=explore
> 
> Please download the source package, and follow the README to build
> and run the Pulsar standalone service.
> 
> Thanks
> Cong Zhao
> 

[ANNOUNCE] Apache Pulsar 2.11.1 released

[guo jiwei]

The Apache Pulsar team is proud to announce Apache Pulsar version 2.11.1.

Pulsar is a highly scalable, low latency messaging platform running on
commodity hardware. It provides simple pub-sub semantics over topics,
guaranteed at-least-once delivery of messages, automatic cursor management for
subscribers, and cross-datacenter replication.

For Pulsar release details and downloads, visit:

https://pulsar.apache.org/download

Release Notes are at:
https://pulsar.apache.org/release-notes

We would like to thank the contributors that made the release possible.

Regards
Jiwei Guo (Tboy)

Re: [DISCUSS] Separate the Python Functions from the installation of the Python client

[Neng Lu]

Hi Yunze,

+1 for separating Python client and Python Pulsar Functions pip installation.
On the Java side, the client lib and functions lib are also published 
separately.

My concern is how the migration progress should look like,
1. we need to set up functions lib so that users can install it using `pip 
install pulsar-functions`
2. the current `pip install pulsar-client[functions]` should prompt user to use 
the new way
3. all docs need to be updated
4. for historical versions, what can we do?



On 2023/04/19 15:23:49 Yunze Xu wrote:
> Hi all,
> 
> The Python client has been separated since PIP-209 [1] and now the
> Python client is maintained in a separated repository [2]. However,
> the Python Function is still maintained in the main repo [3].
> 
> Currently, we can install the Python client with the following ways:
> 1. pip install pulsar-client
> 2. pip install pulsar-client[avro]
> 3. pip install pulsar-client[functions]
> 4. pip install pulsar-client[all]
> 
> See [4] for the difference. However, for the 3rd and 4th ways, it
> installs all the dependencies required by the Python Functions.
> However, they are broken for the recent releases because of the
> outdated dependencies [5]. However, these dependencies are from the
> Python Functions [3], not the Python client library itself. Also,
> there are no tests in the Python client repo [2] for these functions
> so these dependencies cannot be verified.
> 
> IMO, these dependencies should be maintained in the directory of the
> Python Functions. We should not rely on the Python client to install
> the dependencies for the Python Functions.
> 
> Therefore, my suggestion is to drop the 3rd and 4th installation
> methods in future releases of the Python client. After that, we should
> update the scripts in the main repo to install the Python Functions in
> [3].
> 
> I'm not familiar with the Pulsar Functions, so feel free to show your
> suggestions if any of you have any concerns.
> 
> [1] https://github.com/apache/pulsar/pull/17881
> [2] http://github.com/apache/pulsar-client-python
> [3] 
> https://github.com/apache/pulsar/tree/master/pulsar-functions/instance/src/main/python
> [4] https://pulsar.apache.org/docs/2.11.x/client-libraries-python/
> [5] 
> https://github.com/apache/pulsar-client-python/blob/a6476d9c45508f55a7af4b25001038a8e3a27489/setup.py#L80-L88
> 
> Thanks,
> Yunze
> 

[DISCUSS] Separate the Python Functions from the installation of the Python client

[Yunze Xu]

Hi all,

The Python client has been separated since PIP-209 [1] and now the
Python client is maintained in a separated repository [2]. However,
the Python Function is still maintained in the main repo [3].

Currently, we can install the Python client with the following ways:
1. pip install pulsar-client
2. pip install pulsar-client[avro]
3. pip install pulsar-client[functions]
4. pip install pulsar-client[all]

See [4] for the difference. However, for the 3rd and 4th ways, it
installs all the dependencies required by the Python Functions.
However, they are broken for the recent releases because of the
outdated dependencies [5]. However, these dependencies are from the
Python Functions [3], not the Python client library itself. Also,
there are no tests in the Python client repo [2] for these functions
so these dependencies cannot be verified.

IMO, these dependencies should be maintained in the directory of the
Python Functions. We should not rely on the Python client to install
the dependencies for the Python Functions.

Therefore, my suggestion is to drop the 3rd and 4th installation
methods in future releases of the Python client. After that, we should
update the scripts in the main repo to install the Python Functions in
[3].

I'm not familiar with the Pulsar Functions, so feel free to show your
suggestions if any of you have any concerns.

[1] https://github.com/apache/pulsar/pull/17881
[2] http://github.com/apache/pulsar-client-python
[3] 
https://github.com/apache/pulsar/tree/master/pulsar-functions/instance/src/main/python
[4] https://pulsar.apache.org/docs/2.11.x/client-libraries-python/
[5] 
https://github.com/apache/pulsar-client-python/blob/a6476d9c45508f55a7af4b25001038a8e3a27489/setup.py#L80-L88

Thanks,
Yunze

Re: [VERIFY] Pulsar Release 3.0.0 Candidate 2

[Yunze Xu]

I have opened a PR to move the new interface to the pulsar-common
module. https://github.com/apache/pulsar/pull/20139

Thanks,
Yunze

On Wed, Apr 19, 2023 at 8:44 PM PengHui Li  wrote:
>
> There is a discussion about the new API introduced in 3.0.0
> but it should not be a public API.
>
> Details to see: https://github.com/apache/pulsar/issues/18950
>
> Yunze is checking more details.
>
> Regards,
> Penghui
>
> On Wed, Apr 19, 2023 at 7:02 PM Zike Yang  wrote:
>
> > This is the second release candidate for Apache Pulsar, version 3.0.0.
> >
> > It fixes the following issues:
> > https://github.com/apache/pulsar/milestone/34?closed=1
> >
> > *** Please download, test and verify on this release. This release
> > candidate verification will stay open until April 25 ***
> >
> > Note that we are verifying upon the source (tag), binaries are provided for
> > convenience.
> >
> > Source and binary files:
> > https://dist.apache.org/repos/dist/dev/pulsar/pulsar-3.0.0-candidate-2/
> >
> > SHA-512 checksums:
> >
> > 09d4c6ece070e4af3bee85f360936433067583699e768fb2a3c743e2e07ba31e9cfe7fa1151bde19a41fcbb107c8d18114ad938bda0d983dc72e94c12de7cb06
> > apache-pulsar-3.0.0-bin.tar.gz
> >
> > ebb2d91aca806f3e19d9505d9b6dac4a61e8548ed1bb5b63c2be0116102b5ea6d51ad3c13ee9f79ac105b260934aec9e7a34294eca050ce8a26027d0d0fc996b
> > apache-pulsar-3.0.0-src.tar.gz
> >
> > Maven staging repo:
> > https://repository.apache.org/content/repositories/orgapachepulsar-1229/
> >
> > The tag to verify:
> > v3.0.0-candidate-2 (7d72df306461da76cca2c895064abefcebb456ad)
> > https://github.com/apache/pulsar/releases/tag/v3.0.0-candidate-2
> >
> > Pulsar's KEYS file containing PGP keys you use to sign the release:
> > https://dist.apache.org/repos/dist/dev/pulsar/KEYS
> >
> > Docker images:
> >
> > pulsar images:
> > https://hub.docker.com/layers/9947090/pulsar/3.0.0/images/sha256-41175161ae2372a96fa17d3e9e9b991af569c65d7a98e0bce8fcef97358806a5?context=explore
> >
> > pulsar-all images:
> >
> > https://hub.docker.com/layers/9947090/pulsar-all/3.0.0/images/sha256-c2b846859d567220943ba3d320e9ea54650972aa880769cf6f9891e992984fd0?context=explore
> >
> > Please download the source package, and follow the README to build
> > and run the Pulsar standalone service.
> >
> > Note that this RC doesn't require a formal vote, but we would also
> > appreciate your feedback with +1/-1. And please provide specific
> > comments if your feedback is not +1.
> >
> > Thanks,
> > Zike Yang
> >

Re: [VERIFY] Pulsar Release 3.0.0 Candidate 2

[PengHui Li]

There is a discussion about the new API introduced in 3.0.0
but it should not be a public API.

Details to see: https://github.com/apache/pulsar/issues/18950

Yunze is checking more details.

Regards,
Penghui

On Wed, Apr 19, 2023 at 7:02 PM Zike Yang  wrote:

> This is the second release candidate for Apache Pulsar, version 3.0.0.
>
> It fixes the following issues:
> https://github.com/apache/pulsar/milestone/34?closed=1
>
> *** Please download, test and verify on this release. This release
> candidate verification will stay open until April 25 ***
>
> Note that we are verifying upon the source (tag), binaries are provided for
> convenience.
>
> Source and binary files:
> https://dist.apache.org/repos/dist/dev/pulsar/pulsar-3.0.0-candidate-2/
>
> SHA-512 checksums:
>
> 09d4c6ece070e4af3bee85f360936433067583699e768fb2a3c743e2e07ba31e9cfe7fa1151bde19a41fcbb107c8d18114ad938bda0d983dc72e94c12de7cb06
> apache-pulsar-3.0.0-bin.tar.gz
>
> ebb2d91aca806f3e19d9505d9b6dac4a61e8548ed1bb5b63c2be0116102b5ea6d51ad3c13ee9f79ac105b260934aec9e7a34294eca050ce8a26027d0d0fc996b
> apache-pulsar-3.0.0-src.tar.gz
>
> Maven staging repo:
> https://repository.apache.org/content/repositories/orgapachepulsar-1229/
>
> The tag to verify:
> v3.0.0-candidate-2 (7d72df306461da76cca2c895064abefcebb456ad)
> https://github.com/apache/pulsar/releases/tag/v3.0.0-candidate-2
>
> Pulsar's KEYS file containing PGP keys you use to sign the release:
> https://dist.apache.org/repos/dist/dev/pulsar/KEYS
>
> Docker images:
>
> pulsar images:
> https://hub.docker.com/layers/9947090/pulsar/3.0.0/images/sha256-41175161ae2372a96fa17d3e9e9b991af569c65d7a98e0bce8fcef97358806a5?context=explore
>
> pulsar-all images:
>
> https://hub.docker.com/layers/9947090/pulsar-all/3.0.0/images/sha256-c2b846859d567220943ba3d320e9ea54650972aa880769cf6f9891e992984fd0?context=explore
>
> Please download the source package, and follow the README to build
> and run the Pulsar standalone service.
>
> Note that this RC doesn't require a formal vote, but we would also
> appreciate your feedback with +1/-1. And please provide specific
> comments if your feedback is not +1.
>
> Thanks,
> Zike Yang
>

[VERIFY] Pulsar Release 3.0.0 Candidate 2

[Zike Yang]

This is the second release candidate for Apache Pulsar, version 3.0.0.

It fixes the following issues:
https://github.com/apache/pulsar/milestone/34?closed=1

*** Please download, test and verify on this release. This release
candidate verification will stay open until April 25 ***

Note that we are verifying upon the source (tag), binaries are provided for
convenience.

Source and binary files:
https://dist.apache.org/repos/dist/dev/pulsar/pulsar-3.0.0-candidate-2/

SHA-512 checksums:
09d4c6ece070e4af3bee85f360936433067583699e768fb2a3c743e2e07ba31e9cfe7fa1151bde19a41fcbb107c8d18114ad938bda0d983dc72e94c12de7cb06
apache-pulsar-3.0.0-bin.tar.gz
ebb2d91aca806f3e19d9505d9b6dac4a61e8548ed1bb5b63c2be0116102b5ea6d51ad3c13ee9f79ac105b260934aec9e7a34294eca050ce8a26027d0d0fc996b
apache-pulsar-3.0.0-src.tar.gz

Maven staging repo:
https://repository.apache.org/content/repositories/orgapachepulsar-1229/

The tag to verify:
v3.0.0-candidate-2 (7d72df306461da76cca2c895064abefcebb456ad)
https://github.com/apache/pulsar/releases/tag/v3.0.0-candidate-2

Pulsar's KEYS file containing PGP keys you use to sign the release:
https://dist.apache.org/repos/dist/dev/pulsar/KEYS

Docker images:

pulsar images: 
https://hub.docker.com/layers/9947090/pulsar/3.0.0/images/sha256-41175161ae2372a96fa17d3e9e9b991af569c65d7a98e0bce8fcef97358806a5?context=explore

pulsar-all images:
https://hub.docker.com/layers/9947090/pulsar-all/3.0.0/images/sha256-c2b846859d567220943ba3d320e9ea54650972aa880769cf6f9891e992984fd0?context=explore

Please download the source package, and follow the README to build
and run the Pulsar standalone service.

Note that this RC doesn't require a formal vote, but we would also
appreciate your feedback with +1/-1. And please provide specific
comments if your feedback is not +1.

Thanks,
Zike Yang

Re: ANNOUNCE] Apache Pulsar 2.10.4 released

[Zike Yang]

Hi, Xiangying

Thanks for the announcement.
I think we also need to send this email to us...@pulsar.apache.org and
annou...@apache.org.

BR,
Zike Yang

On Wed, Apr 19, 2023 at 12:37 PM Xiangying Meng  wrote:
>
> The Apache Pulsar team is proud to announce Apache Pulsar version 2.10.4.
>
> Pulsar is a highly scalable, low latency messaging platform running on
> commodity hardware. It provides simple pub-sub semantics over topics,
> guaranteed at-least-once delivery of messages, automatic cursor management
> for
> subscribers, and cross-datacenter replication.
>
> For Pulsar release details and downloads, visit:
>
> https://pulsar.apache.org/download
>
> Release Notes are at:
> https://pulsar.apache.org/release-notes
>
> We would like to thank the contributors that made the release possible.
>
> Regards,
>
> The Pulsar Team

Re: [Discuss] Suggestion for a "clear" parameter in Pulsar-admin to simplify tenant and namespace cleanup

[Zike Yang]

Is it possible to use regex to delete namespaces/topics here?
For instance, we can use the following command to delete all topics in
the default namespace:
```
pulsar-admin topics delete public/default/*
```
We can also use it to delete some specific topics:
```
pulsar-admin topics delete public/default/*topic
```
This way, we don't need to introduce a new command
`delete-all-topics`. It also makes it easier for users to delete
specific topic lists.

> Part-1: Give a summary print of the namespaces, and topics to delete,
>  and ask the user to confirm if they want to delete the resources.
> Part-2: If users select “yes”, the deletion will be really executed.
> Part-3: Print a summary of the results

+1 for this. This will minimize the risk of accidentally deleting
important resources.

BR,
Zike Yang

On Wed, Apr 19, 2023 at 3:34 PM Yubiao Feng
 wrote:
>
> > Just wondering - since it is such a dangerous
> > command, how can we help the user not make
> > an accidental mass deletion?
>
> Just a suggestion: We can make this command executed on two-part
>
> Part-1: Give a summary print of the namespaces, and topics to delete,
>  and ask the user to confirm if they want to delete the resources.
>
> Part-2: If users select “yes”, the deletion will be really executed.
>
> Part-3: Print a summary of the results
>
>
> Thanks
> Yubiao Feng
>
> On Sun, Apr 16, 2023 at 9:45 PM Asaf Mesika  wrote:
>
> > How about "truncate" instead of "clear"?
> >
> > Just wondering - since it is such a dangerous command, how can we help the
> > user not make an accidental mass deletion?
> >
> > On Sat, Apr 15, 2023 at 1:12 PM Girish Sharma 
> > wrote:
> >
> > > > However, the current goal is to keep the tenant and namespace intact
> > > while
> > > > cleaning up their contents.
> > > Ah, I see now. Yes, in that case a clear command is better. Will this
> > > command also take into account the value of the broker config
> > > `forceDeleteNamespaceAllowed` in case someone is clearing the owner
> > tenant?
> > >
> > > Regards
> > >
> > > On Sat, Apr 15, 2023 at 3:39 PM Enrico Olivelli 
> > > wrote:
> > >
> > > > The proposal sounds really useful, especially for automated testing.
> > > > +1
> > > >
> > > > Enrico
> > > >
> > > > Il giorno sab 15 apr 2023 alle ore 12:07 Xiangying Meng
> > > >  ha scritto:
> > > > >
> > > > > Dear Girish,
> > > > >
> > > > > Thank you for your response and suggestion to extend the use of the
> > > > > `boolean force` flag for namespaces and tenants.
> > > > > I understand that the `force` flag is already implemented for
> > deleting
> > > > > topics, namespaces, and tenants,
> > > > > and it provides a consistent way to perform these actions.
> > > > >
> > > > > However, the current goal is to keep the tenant and namespace intact
> > > > while
> > > > > cleaning up their contents.
> > > > > In other words, I want to have a way to remove all topics within a
> > > > > namespace or all namespaces and topics
> > > > > within a tenant without actually deleting the namespace or tenant
> > > itself.
> > > > >
> > > > > To achieve this goal, I proposed adding a `clear` command for
> > > > `namespaces`
> > > > > and `tenants`.
> > > > >
> > > > > This approach would allow users to keep the tenant and namespace
> > > > structures
> > > > > in place
> > > > > while cleaning up their contents.
> > > > > I hope this clarifies my intention, and I would like to hear your
> > > > thoughts
> > > > > on this proposal.
> > > > >
> > > > > Best regards,
> > > > > Xiangying
> > > > >
> > > > > On Sat, Apr 15, 2023 at 5:49 PM Girish Sharma <
> > scrapmachi...@gmail.com
> > > >
> > > > > wrote:
> > > > >
> > > > > > Hello Xiangying,
> > > > > > This indeed is a cumbersome task to delete a filled namespace or
> > > > tenant. We
> > > > > > face this challenge in our organization where we use the
> > > multi-tenancy
> > > > > > feature of pulsar heavily.
> > > > > >
> > > > > > I would like to suggest a different command to do this though..
> > > > Similar to
> > > > > > how you cannot delete a topic without deleting its
> > > > > > subscribers/producers/consumers, unless we use the `boolean force`
> > > > flag.
> > > > > > Why not extend this to namespace and tenant as well and let the
> > force
> > > > param
> > > > > > do the cleanup (which your suggested `clear` command would do).
> > > > > >
> > > > > > As of today, using force to delete a namespace just returns 405
> > > saying
> > > > > > broker doesn't allow force delete of namespace containing topics.
> > > > > >
> > > > > > Any thoughts?
> > > > > >
> > > > > > On Sat, Apr 15, 2023 at 3:07 PM Xiangying Meng <
> > xiangy...@apache.org
> > > >
> > > > > > wrote:
> > > > > >
> > > > > > > Dear Apache Pulsar Community,
> > > > > > >
> > > > > > > I hope this email finds you well.I am writing to suggest a
> > > potential
> > > > > > > improvement to the Pulsar-admin tool,
> > > > > > >  which I believe could simplify the process of cleaning up
> > tenants
> > > > and
> > > > 

Re: [Discuss] Suggestion for a "clear" parameter in Pulsar-admin to simplify tenant and namespace cleanup

[Yubiao Feng]

> Just wondering - since it is such a dangerous
> command, how can we help the user not make
> an accidental mass deletion?

Just a suggestion: We can make this command executed on two-part

Part-1: Give a summary print of the namespaces, and topics to delete,
 and ask the user to confirm if they want to delete the resources.

Part-2: If users select “yes”, the deletion will be really executed.

Part-3: Print a summary of the results


Thanks
Yubiao Feng

On Sun, Apr 16, 2023 at 9:45 PM Asaf Mesika  wrote:

> How about "truncate" instead of "clear"?
>
> Just wondering - since it is such a dangerous command, how can we help the
> user not make an accidental mass deletion?
>
> On Sat, Apr 15, 2023 at 1:12 PM Girish Sharma 
> wrote:
>
> > > However, the current goal is to keep the tenant and namespace intact
> > while
> > > cleaning up their contents.
> > Ah, I see now. Yes, in that case a clear command is better. Will this
> > command also take into account the value of the broker config
> > `forceDeleteNamespaceAllowed` in case someone is clearing the owner
> tenant?
> >
> > Regards
> >
> > On Sat, Apr 15, 2023 at 3:39 PM Enrico Olivelli 
> > wrote:
> >
> > > The proposal sounds really useful, especially for automated testing.
> > > +1
> > >
> > > Enrico
> > >
> > > Il giorno sab 15 apr 2023 alle ore 12:07 Xiangying Meng
> > >  ha scritto:
> > > >
> > > > Dear Girish,
> > > >
> > > > Thank you for your response and suggestion to extend the use of the
> > > > `boolean force` flag for namespaces and tenants.
> > > > I understand that the `force` flag is already implemented for
> deleting
> > > > topics, namespaces, and tenants,
> > > > and it provides a consistent way to perform these actions.
> > > >
> > > > However, the current goal is to keep the tenant and namespace intact
> > > while
> > > > cleaning up their contents.
> > > > In other words, I want to have a way to remove all topics within a
> > > > namespace or all namespaces and topics
> > > > within a tenant without actually deleting the namespace or tenant
> > itself.
> > > >
> > > > To achieve this goal, I proposed adding a `clear` command for
> > > `namespaces`
> > > > and `tenants`.
> > > >
> > > > This approach would allow users to keep the tenant and namespace
> > > structures
> > > > in place
> > > > while cleaning up their contents.
> > > > I hope this clarifies my intention, and I would like to hear your
> > > thoughts
> > > > on this proposal.
> > > >
> > > > Best regards,
> > > > Xiangying
> > > >
> > > > On Sat, Apr 15, 2023 at 5:49 PM Girish Sharma <
> scrapmachi...@gmail.com
> > >
> > > > wrote:
> > > >
> > > > > Hello Xiangying,
> > > > > This indeed is a cumbersome task to delete a filled namespace or
> > > tenant. We
> > > > > face this challenge in our organization where we use the
> > multi-tenancy
> > > > > feature of pulsar heavily.
> > > > >
> > > > > I would like to suggest a different command to do this though..
> > > Similar to
> > > > > how you cannot delete a topic without deleting its
> > > > > subscribers/producers/consumers, unless we use the `boolean force`
> > > flag.
> > > > > Why not extend this to namespace and tenant as well and let the
> force
> > > param
> > > > > do the cleanup (which your suggested `clear` command would do).
> > > > >
> > > > > As of today, using force to delete a namespace just returns 405
> > saying
> > > > > broker doesn't allow force delete of namespace containing topics.
> > > > >
> > > > > Any thoughts?
> > > > >
> > > > > On Sat, Apr 15, 2023 at 3:07 PM Xiangying Meng <
> xiangy...@apache.org
> > >
> > > > > wrote:
> > > > >
> > > > > > Dear Apache Pulsar Community,
> > > > > >
> > > > > > I hope this email finds you well.I am writing to suggest a
> > potential
> > > > > > improvement to the Pulsar-admin tool,
> > > > > >  which I believe could simplify the process of cleaning up
> tenants
> > > and
> > > > > > namespaces in Apache Pulsar.
> > > > > >
> > > > > > Currently, cleaning up all the namespaces and topics within a
> > tenant
> > > or
> > > > > > cleaning up all the topics within a namespace requires several
> > manual
> > > > > > steps,
> > > > > > such as listing the namespaces, listing the topics, and then
> > deleting
> > > > > each
> > > > > > topic individually.
> > > > > > This process can be time-consuming and error-prone for users.
> > > > > >
> > > > > > To address this issue, I propose the addition of a "clear"
> > parameter
> > > to
> > > > > the
> > > > > > Pulsar-admin tool,
> > > > > > which would automate the cleanup process for tenants and
> > namespaces.
> > > > > Here's
> > > > > > a conceptual implementation:
> > > > > >
> > > > > > 1. To clean up all namespaces and topics within a tenant:
> > > > > > ``` bash
> > > > > > pulsar-admin tenants clear 
> > > > > > ```
> > > > > > 2. To clean up all topics within a namespace:
> > > > > > ```bash
> > > > > > pulsar-admin namespaces clear /
> > > > > > ```
> > > > > >
> > > > > > By implementing these new 

Re: [Discuss] Suggestion for a "clear" parameter in Pulsar-admin to simplify tenant and namespace cleanup

[Girish Sharma]

Hello Yubiao,

What additional advantage would one get by using that approach rather than
simply using a one liner script to just call delete topic for each of those
topics if the list of topics is known.

Regards

On Wed, Apr 19, 2023 at 12:54 PM Yubiao Feng
 wrote:

> In addition to these two, It is recommended to add a method to batch delete
> topics, such as this:
>
> ```
> pulsar-admin topics delete-all-topics , 
>
> or
>
> pulsar-admin topics delete-all-topic 
> ```
>
> Thanks
> Yubiao Feng
>
> On Sat, Apr 15, 2023 at 5:37 PM Xiangying Meng 
> wrote:
>
> > Dear Apache Pulsar Community,
> >
> > I hope this email finds you well.I am writing to suggest a potential
> > improvement to the Pulsar-admin tool,
> >  which I believe could simplify the process of cleaning up tenants and
> > namespaces in Apache Pulsar.
> >
> > Currently, cleaning up all the namespaces and topics within a tenant or
> > cleaning up all the topics within a namespace requires several manual
> > steps,
> > such as listing the namespaces, listing the topics, and then deleting
> each
> > topic individually.
> > This process can be time-consuming and error-prone for users.
> >
> > To address this issue, I propose the addition of a "clear" parameter to
> the
> > Pulsar-admin tool,
> > which would automate the cleanup process for tenants and namespaces.
> Here's
> > a conceptual implementation:
> >
> > 1. To clean up all namespaces and topics within a tenant:
> > ``` bash
> > pulsar-admin tenants clear 
> > ```
> > 2. To clean up all topics within a namespace:
> > ```bash
> > pulsar-admin namespaces clear /
> > ```
> >
> > By implementing these new parameters, users would be able to perform
> > cleanup operations more efficiently and with fewer manual steps.
> > I believe this improvement would greatly enhance the user experience when
> > working with Apache Pulsar.
> >
> > I'd like to discuss the feasibility of this suggestion and gather
> feedback
> > from the community.
> > If everyone agrees, I can work on implementing this feature and submit a
> > pull request for review.
> >
> > Looking forward to hearing your thoughts on this.
> >
> > Best regards,
> > Xiangying
> >
>


-- 
Girish Sharma

Re: [Discuss] Suggestion for a "clear" parameter in Pulsar-admin to simplify tenant and namespace cleanup

[Yubiao Feng]

In addition to these two, It is recommended to add a method to batch delete
topics, such as this:

```
pulsar-admin topics delete-all-topics , 

or

pulsar-admin topics delete-all-topic 
```

Thanks
Yubiao Feng

On Sat, Apr 15, 2023 at 5:37 PM Xiangying Meng  wrote:

> Dear Apache Pulsar Community,
>
> I hope this email finds you well.I am writing to suggest a potential
> improvement to the Pulsar-admin tool,
>  which I believe could simplify the process of cleaning up tenants and
> namespaces in Apache Pulsar.
>
> Currently, cleaning up all the namespaces and topics within a tenant or
> cleaning up all the topics within a namespace requires several manual
> steps,
> such as listing the namespaces, listing the topics, and then deleting each
> topic individually.
> This process can be time-consuming and error-prone for users.
>
> To address this issue, I propose the addition of a "clear" parameter to the
> Pulsar-admin tool,
> which would automate the cleanup process for tenants and namespaces. Here's
> a conceptual implementation:
>
> 1. To clean up all namespaces and topics within a tenant:
> ``` bash
> pulsar-admin tenants clear 
> ```
> 2. To clean up all topics within a namespace:
> ```bash
> pulsar-admin namespaces clear /
> ```
>
> By implementing these new parameters, users would be able to perform
> cleanup operations more efficiently and with fewer manual steps.
> I believe this improvement would greatly enhance the user experience when
> working with Apache Pulsar.
>
> I'd like to discuss the feasibility of this suggestion and gather feedback
> from the community.
> If everyone agrees, I can work on implementing this feature and submit a
> pull request for review.
>
> Looking forward to hearing your thoughts on this.
>
> Best regards,
> Xiangying
>