Pulsar Community Meeting Notes 2022/09/15, (8:30 AM PST)

Hi Pulsar Community, Here are the meeting notes from Thursday's community meeting. Thanks to all who participated! Disclaimer: if something is misattributed or misrepresented, please send a correction to this list. Source google doc:

Re: [DISCUSS] PIP-209: Separate C++/Python clients to own repositories

Great proposal, thanks Matteo. I think I agree with splitting out the client into two repos. One issue is that new C++ features will lag in the python client because the C++ client will first need to be released. Quick releases will likely help there. > Instead of just fetching the main repo

[GitHub] [pulsar] raunakagrawal47 edited a comment on the discussion: Need max redelivery count at message level.

GitHub user raunakagrawal47 edited a comment on the discussion: Need max redelivery count at message level. since I am manually sending the message to the DLQ after some retry based on message, how do I ack the message from the retry queue? (To remove it from retry queue and send it to DLQ)

[GitHub] [pulsar] raunakagrawal47 edited a comment on the discussion: Need max redelivery count at message level.

GitHub user raunakagrawal47 edited a comment on the discussion: Need max redelivery count at message level. since I am manually sending the message to the DLQ after some retry based on message, how do I ack the message from the retry queue? (To remove it from retry queue and send it to DLQ)

[GitHub] [pulsar] raunakagrawal47 edited a comment on the discussion: Need max redelivery count at message level.

GitHub user raunakagrawal47 edited a comment on the discussion: Need max redelivery count at message level. since I am manually sending the message to the DLQ after some retry based on message, how do I ack the message from the retry queue? (To remove it from retry queue and send it to DLQ)

Re: Announcing first milestone release of Spring for Apache Pulsar

Greate work Thanks ZhangJian He On Wed, 21 Sept 2022 at 04:05, Christophe Bornet wrote: > Awesome Soby ! > > Le mar. 20 sept. 2022 à 18:10, Soby Chacko a écrit > : > > > Hi Apache Pulsar Community, > > > > We are very happy to announce the first milestone release of Spring for > > Apache

[DISCUSSION] Options for improving Pulsar build time by adding resource.

Community, There has been discussed before that the build pipeline has been struggling to keep up with PRs. What are our options in term of compute resources for running build runners? I'm aware that the current build actions are tied to Apache's shared resource. Is there options the public

Re: Announcing first milestone release of Spring for Apache Pulsar

Awesome Soby ! Le mar. 20 sept. 2022 à 18:10, Soby Chacko a écrit : > Hi Apache Pulsar Community, > > We are very happy to announce the first milestone release of Spring for > Apache Pulsar. More details can be found here: > >

[GitHub] [pulsar-helm-chart] ThelonKarrde commented on pull request #284: Add nodeSelector to cluster initialize pod

ThelonKarrde commented on PR #284: URL: https://github.com/apache/pulsar-helm-chart/pull/284#issuecomment-1252718787 @michaeljmarshall indeed I pushed the wrong changes. Thank you for the fix. It works locally. Also pushed a fix for typo in values description -- This is an automated

Re: [DISCUSS] PIP-209: Separate C++/Python clients to own repositories

-- Matteo Merli On Mon, Sep 19, 2022 at 7:30 PM Baodi Shi wrote: > > The suggestion is to start the new releases for both C++ and Python from > > 3.0.0. > > In the future, Can we need to maintain a compatibility list? For the user, > how should he choose the appropriate client version to

Re: [DISCUSS] PIP-209: Separate C++/Python clients to own repositories

On Tue, Sep 20, 2022 at 9:32 AM Yunze Xu wrote: > > Hi Tison & Matteo, > > I noticed a comment here: > https://github.com/apache/pulsar/pull/17733#discussion_r975552686 > > Currently the Python Functions relies on the Python client. I see there are > still some Python Function examples in

Re: [DISCUSS] PIP-209: Separate C++/Python clients to own repositories

Hi Tison & Matteo, I noticed a comment here: https://github.com/apache/pulsar/pull/17733#discussion_r975552686 Currently the Python Functions relies on the Python client. I see there are still some Python Function examples in pulsar-functions/python-examples directory. How should we deal with

[GitHub] [pulsar] raunakagrawal47 added a comment to the discussion: Need max redelivery count at message level.

GitHub user raunakagrawal47 added a comment to the discussion: Need max redelivery count at message level. @Jason918 since I am manually sending the message to the DLQ after some retry based on message, how do I ack the message from the retry queue? (To remove it from retry queue and send it

Announcing first milestone release of Spring for Apache Pulsar

Hi Apache Pulsar Community, We are very happy to announce the first milestone release of Spring for Apache Pulsar. More details can be found here: https://spring.io/blog/2022/09/20/spring-for-apache-pulsar-0-1-0-m1-is-now-available Any feedback and suggestions are greatly appreciated!! Thank

[GitHub] [pulsar] Jason918 added a comment to the discussion: Need max redelivery count at message level.

GitHub user Jason918 added a comment to the discussion: Need max redelivery count at message level. > 1. If I just set` enableRetry(true)`, how much time does it wait to confirm > whether an acknowledgement is done or not and replay the message? Because we > are not using any timeout here.

Re: [DISCUSS] PIP-209: Separate C++/Python clients to own repositories

One more thing to mention here: Currently, Pulsar Docker Image bundles C++ client and Python client, and from my perspective, the image is mainly used as a server, perhaps we can remove these clients from bundling. Best, tison. Yunze Xu 于2022年9月20日周二 16:50写道： > Hi Tison, > > Sorry I just

[GitHub] [pulsar] raunakagrawal47 edited a comment on the discussion: Need max redelivery count at message level.

GitHub user raunakagrawal47 edited a comment on the discussion: Need max redelivery count at message level. @Jason918 @michaeljmarshall 1. If I just set` enableRetry(true)`, how much time does it wait to confirm whether an acknowledgement is done or not and replay the message? Because we

Re: [DISCUSS] PIP-208: HTTP Sink

Christophe, very good initiative! I support it Some comments inline below Enrico Il giorno lun 19 set 2022 alle ore 19:10 Christophe Bornet ha scritto: > > Hi all, > > I have drafted PIP-208: HTTP Sink > > PIP link: > https://github.com/apache/pulsar/issues/17719 > > Here's a copy of the

[GitHub] [pulsar] gvolpe added a comment to the discussion: [Pulsar 2.10.1] - Unable to get Debezium PostgreSQL working

GitHub user gvolpe added a comment to the discussion: [Pulsar 2.10.1] - Unable to get Debezium PostgreSQL working Ok, managed to get this working with the following configuration: ```yaml tenant: "public" namespace: "default" name: "debezium-postgres-source" topicName:

[GitHub] [pulsar] gvolpe edited a discussion: [Pulsar 2.10.1] - Unable to get Debezium PostgreSQL working

GitHub user gvolpe edited a discussion: [Pulsar 2.10.1] - Unable to get Debezium PostgreSQL working Hi, I've followed the [documentation](https://pulsar.apache.org/docs/io-cdc-debezium/) trying to use my own database and tables, but I'm unable to get this working. ### Docker Compose

[GitHub] [pulsar] gvolpe edited a discussion: [Pulsar 2.10.1] - Unable to get Debezium PostgreSQL working

GitHub user gvolpe edited a discussion: [Pulsar 2.10.1] - Unable to get Debezium PostgreSQL working Hi, I've followed the [documentation](https://pulsar.apache.org/docs/io-cdc-debezium/) trying to use my own database and tables, but I'm unable to get this working. ### Docker Compose

[GitHub] [pulsar] gvolpe edited a discussion: [Pulsar 2.10.0] - Unable to get Debezium PostgreSQL working

GitHub user gvolpe edited a discussion: [Pulsar 2.10.0] - Unable to get Debezium PostgreSQL working Hi, I've followed the [documentation](https://pulsar.apache.org/docs/io-cdc-debezium/) trying to use my own database and tables, but I'm unable to get this working. ### Docker Compose

Re: [DISCUSS] PIP-209: Separate C++/Python clients to own repositories

Hi Tison, Sorry I just missed that. Thanks for your reminder. Thanks, Yunze > On Sep 20, 2022, at 16:35, tison wrote: > > Hi Yunze, > >> Just wondering if there is a way to retain the git history in the > pulsar-client-cpp directory? > > Matteo's proposal already write: > >> git

Re: [DISCUSS] PIP-209: Separate C++/Python clients to own repositories

I agree that we need to work on a more comprehensive documentation-building process. Perhaps let's start with removing old code https://github.com/apache/pulsar-site/pull/215. I'd say it needs a code bash to sort current logic - it's somewhat brittle depending on relative path, coupled shell

Re: [DISCUSS] Consumer reconnection causes repeated consumption messages

Hi Haiting, In this case, the user wants to reset to the first message reconsume all the messages, but the user can't receive a message anymore, the `admin.topics().resetCursor(topic, subName, MessageId.earliest)` may use HTTP to call, so it is wrong behavior. Thanks, Bo Haiting Jiang

Re: [DISCUSS] PIP-209: Separate C++/Python clients to own repositories

Hi Yunze, > Just wondering if there is a way to retain the git history in the pulsar-client-cpp directory? Matteo's proposal already write: > git filter-repo --subdirectory-filter pulsar-client-cpp So you will retain the git history. Best, tison. Yunze Xu 于2022年9月20日周二 16:27写道： > LGTM. I

Re: [DISCUSS] PIP-209: Separate C++/Python clients to own repositories

LGTM. I also listed the related files outside the pulsar-client-cpp directory recently: - pulsar-common/src/main/proto/PulsarApi.proto: the Pulsar binary proto file - src/gen-pulsar-version-macro.py: generate the internal version info - pulsar-client/src/test/proto/*.proto: test the protobuf

[GitHub] [pulsar] raunakagrawal47 edited a comment on the discussion: Need max redelivery count at message level.

GitHub user raunakagrawal47 edited a comment on the discussion: Need max redelivery count at message level. 1. If I just set` enableRetry(true)`, how much time does it wait to confirm whether an acknowledgement is done or not and replay the message? Because we are not using any timeout here.

[GitHub] [pulsar] raunakagrawal47 added a comment to the discussion: Need max redelivery count at message level.

GitHub user raunakagrawal47 added a comment to the discussion: Need max redelivery count at message level. 1. If I just set` enableRetry(true)`, how much time does it wait to confirm whether an acknowledgement is done or not and replay the message? Because we are not using any timeout here.