date:20230220

Hi, all

Currently, the Pulsar broker uses the field `client_version` to get
the version of the client. The client will send the client_version to
the broker through `CommandConnect` [0] during the connect or
`CommandAuthResponse ` [1] during the auth challenge. We could get the
client_version from the admin using `pulsar-admin topics stats xxx`
[2].

But the `client_version ` only shows the version information. And
clients in different languages use their own separate version numbers.
This can lead to conflict. For example, the java client 2.10.2 uses
the same value `2.10.2` as the C++ client 2.10.2. And C++ client 3.0.0
may conflict with the future java client 3.0.0. The information of
`client_version` is incomplete. We could not determine what
language(or specific library) the client uses. This raises
inconvenience for debugging.

I would like to propose adding the client language information to the
`client_version`. Like:
* `java-2.11.1` for Java client 2.11
* `cpp-3.1.2` for C++ client 3.1.2
* `go-0.9.0` for go client 0.9.0
We can easily do that because the `client_version` is a string type.

In addition, the Nodejs client and Python client all use the
client_version of C++ client they depend on. So it will use `3.1.2` as
the client_verion in Nodejs client 1.8.0. This is incorrect, and I
think we need to fix them. We don't need to print the C++ client
version because we can get that information if we know the Nodejs or
Python client version.

What do you think? Please share your thoughts if you have any ideas.

[0]
https://github.com/apache/pulsar/blob/e0b50c9ec5f12d0fb8275f235d8ac00e87a9099e/pulsar-common/src/main/proto/PulsarApi.proto#L269
[1]
https://github.com/apache/pulsar/blob/e0b50c9ec5f12d0fb8275f235d8ac00e87a9099e/pulsar-common/src/main/proto/PulsarApi.proto#L309
[2] https://pulsar.apache.org/docs/next/admin-api-topics/#get-stats

Thanks,
Zike Yang

Re: Force redirect questions from Slack to GitHub Discussions or StackOverflow?

2023-02-20 Thread Dave Fisher

We do not have the right to move an individual’s message to a platform where 
they have not agreed to the terms of use. We must not “force redirect”. We 
don’t own these questions, the person asking the question does.

+1 to pinning a message to Slack channels.

Best,
Dave

Sent from my iPhone

> On Feb 20, 2023, at 2:43 AM, Kiryl Valkovich  
> wrote:
> 
> Do you mean, to do it for all messages in the #general channel (maybe only 
> for messages that contain the question mark)?
> 
> I think it makes sense.
> 
> From: Asaf Mesika 
> Date: Sunday, February 19, 2023 at 11:54 AM
> To: dev@pulsar.apache.org 
> Subject: Re: Force redirect questions from Slack to GitHub Discussions or 
> StackOverflow?
> I would have the bot open a Thread for the message, *suggesting* the user
> to click to convert this question into a GitHub Discussion question. This
> way you can have the actual GitHub user asking the question and not a bot
> one.
> 
>> On Fri, Feb 17, 2023 at 10:59 PM Kiryl Valkovich 
>> wrote:
>> 
>> What about such wording?
>> 
>> ---
>> Your question was moved here:
>> https://github.com/apache/pulsar/discussions/123
>> 
>> Please consider asking new questions here:
>> 
>>  *   At StackOverflow using apache-pulsar tag.
>>  *   In the Q category at GitHub Discussions.
>>  *   Apache Pulsar User Mailing List.
>> 
>> 
>> It will make it searchable by others. Also, this way we can collect a
>> knowledge base outside of Slack over time.
>> 
>> I can’t see how the words “please consider” force the user to do something.
>> 
>> Users who have an account on StackOverflow or GitHub can use these
>> platforms next time.
>> Others can send their question via the mailing list.
>> 
>> From: Dave Fisher 
>> Date: Friday, February 17, 2023 at 9:28 PM
>> To: dev@pulsar.apache.org 
>> Subject: Re: Force redirect questions from Slack to GitHub Discussions or
>> StackOverflow?
>> My concern is that users should have a choice on where to post their
>> questions. They might have concerns about GitHub’s terms and conditions. We
>> can pin a message to slack pointing to GitHub discussions and stackoverflow.
>> 
>> Best,
>> Dave
>> 
>> Sent from my iPhone
>> 
>>> On Feb 17, 2023, at 9:22 AM, Kiryl Valkovich 
>> wrote:
>>> 
>>> I’m the owner of this account.
>>> The goal is to test drive duplicating Slack questions to the GitHub
>> discussions.
>>> With the current level of activity in Slack it’s not so hard to do it
>> manually.
>>> 
>>> I’m in CET now. I can share the account credentials with people who can
>> post questions to GitHub Discussions on behalf of this account in other
>> time zones.
>>> Or I can do it once a day.
>>> 
>>> If someone doesn’t find it useful or has ideas on how to do it in a
>> better way, just say it directly.
>>> 
>>> From: Enrico Olivelli 
>>> Date: Friday, February 17, 2023 at 3:43 PM
>>> To: dev@pulsar.apache.org 
>>> Subject: Re: Force redirect questions from Slack to GitHub Discussions
>> or StackOverflow?
>>> Hello,
>>> I see that some "Pulsar Community Bot" appeared in Slack
>>> 
>>> it is connected to this email address "pulsar.community@gmail.com"
>>> 
>>> While I find this thing "amazing"I wonder if I missed something,
>>> who is the owner of this "bot" ?
>>> 
>>> 
>>> Enrico
>>> 
 Il giorno gio 16 feb 2023 alle ore 16:03 Kiryl Valkovich
  ha scritto:
 
 Played with Slack and StackOverflow APIs a bit.
 
 The Slack API works as expected. After clicking the message button, it
>> sends a message descriptor to my app where I can do anything with its
>> content.
 
 It’s possible to post messages via the StackOverflow API, but it’s
>> unlikely that any Slack message can be converted to a StackOverflow
>> question.
 
 I encountered several types of validation errors:
 
 -  This question body does not meet our quality standards.
>> Please make sure that it completely describes your problem - including what
>> you have already tried - and is written using proper grammar.
 
 *   Something similar to “This message looks like a duplicate of
>> another message”.
 
 I believe, GitHub Discussions don’t have such kind of “quality
>> standards” validation.
 
 From: Kiryl Valkovich 
 Date: Thursday, February 16, 2023 at 1:33 PM
 To: dev@pulsar.apache.org 
 Subject: Re: Force redirect questions from Slack to GitHub Discussions
>> or StackOverflow?
 If there are no hidden obstacles, we can implement it.
 StackOverflow supports question creation using REST API:
>> https://api.stackexchange.com/docs/create-question
 
 From: Zike Yang 
 Date: Thursday, February 16, 2023 at 11:41 AM
 To: dev@pulsar.apache.org 
 Subject: Re: Force redirect questions from Slack to GitHub Discussions
>> or StackOverflow?
 +1, That sounds great to me.
 
> 2. You click the three dots button on his message -> “Convert to a
>> GitHub discussion”.
 
 Is it a similar workflow for converting to

Re: Does anyone build UI for Pulsar?

Il giorno lun 20 feb 2023 alle ore 14:41 Kiryl Valkovich
 ha scritto:
>
> Enrico, it’s easy.
>
> When I tried it, the basic functionality just didn’t work.
>
> Just make a side-by-side comparison of Pulsar Manager with any of the 
> following options:
> - Conduktor (Commercial).
> - Kafka UI by Provectus (Apache License 2.0).
> - Redpanda Console (ex-Kowl) (Mixed license).
>
> I see a significant gap in user experience between any of them and the Pulsar 
> Manager.
> Also, I just don’t see much contribution activity.
>
> Redpanda Console and Kafka UI, each have about x10 times more commits in a 
> year, than Pulsar Manager has in the last 3 years. Therefore I can’t conclude 
> that the project is alive.

(Unfortunately) I 100% agree with you.

>
> Regarding the “Why from scratch?” question:
> - I found it easier for me personally because of the different tech stack I 
> use.
> - Starting from scratch is a good opportunity to learn Pulsar itself.
> - I’d want to make a product around that.
>
> I'm not claiming that my result will be 100% better, but why not try?

I understand your points.
I hope you will share your project with OSS license, this way it will
be easier for Pulsar users to try it out and contribute.
>
> If someone is already doing or plans to do similar work, let’s collaborate!

Thanks for sharing

Enrico

>
> From: Enrico Olivelli 
> Date: Monday, February 20, 2023 at 2:04 PM
> To: dev@pulsar.apache.org 
> Subject: Re: Does anyone build UI for Pulsar?
> Kiryl,
>
> Il giorno lun 20 feb 2023 alle ore 12:18 Kiryl Valkovich
>  ha scritto:
> >
> > Enrico, it seems you read only the mail message title.
>
> Sorry about that,
> I have re-read the message, and I have just realised that I skipped
> the very first line :-)
>
> I have one question.
> IIUC you started from scratch a new UI, could you please explain why
> Pulsar Manager doesn't work for you?
>
>
> Cheers
> Enrico
>
> >
> > From: Enrico Olivelli 
> > Date: Monday, February 20, 2023 at 11:59 AM
> > To: dev@pulsar.apache.org 
> > Subject: Re: Does anyone build UI for Pulsar?
> > Kiryl,
> >
> > You can use the official Apache Pulsar Manager the is maintained by
> > this community
> > https://github.com/apache/pulsar-manager
> >
> > At DataStax we also maintain this other UI that is also 100% opensource
> > https://github.com/datastax/pulsar-admin-console
> >
> > For the BookKeeper part there is BKVM (BookKeeper Visual Manager)
> > https://github.com/diennea/bookkeeper-visual-manager
> > This is also bundled with Apache Pulsar Manager
> >
> >
> > I hope that helps
> >
> > Enrico
> >
> >
> >
> > Il giorno lun 20 feb 2023 alle ore 10:51 Kiryl Valkovich
> >  ha scritto:
> > >
> > > Hi everyone!
> > >
> > > Does anyone personally or some company work on UI for Pulsar other than 
> > > pulsar-manager or pulsar-admin-console?
> > >
> > > I understand that StreamNative and DataStax have managed solutions and 
> > > obviously work on their UI.
> > >
> > > I rather looking for an open-source or commercial tool that can be used 
> > > in pair with any Pulsar deployment.
> > >
> > > I spent some time implementing UI for Apache Pulsar. It’s not ready to 
> > > release yet. As usual, the most difficult 20% of the work remained.
> > >
> > > I’m asking that to avoid situations when several people or teams are 
> > > doing the same thing.
> > > Recently I proposed the site redesign and some teams are already doing 
> > > that as it’s figured out.
> > >
> > > See Asaf’s comment:
> > > https://github.com/apache/pulsar-site/pull/426#issuecomment-1436589392
> > >
> > > Please at least DM me if it’s not public information yet.
> > >
> > > Also DM me if you think that we can try to cooperate.
> > >
> > > Thank you.

Re: [DISCUSS] PIP-246: Improved PROTOBUF_NATIVE schema compatibility checks without using avro-protobuf

[Edit] Sorry, it’s documented here: 
https://pulsar.apache.org/docs/2.11.x/schema-understand/#schema-compatibility-check

From: Kiryl Valkovich 
Date: Monday, February 20, 2023 at 3:48 PM
To: dev@pulsar.apache.org 
Subject: Re: [DISCUSS] PIP-246: Improved PROTOBUF_NATIVE schema compatibility 
checks without using avro-protobuf
Hm. I tend to think that for my Pulsar use case, it would be good to have the 
ability to implement a custom schema compatibility checker.

For example, buf.build (a popular Protobuf linter and build) offers the 
following list of breaking rules. Half of them prefixed with 
FIELD_SAME_XXLANG_PACKAGE_ aren’t relevant.

And I would want to use a subset of them if we are checking a single message 
descriptor. Most likely:
ENUM_VALUE_NO_DELETE
FIELD_NO_DELETE
FIELD_SAME_TYPE
ONEOF_NO_DELETE
FIELD_SAME_LABEL
RESERVED_ENUM_NO_DELETE
RESERVED_MESSAGE_NO_DELETE

I found out that the Pulsar broker has the following configuration option: 
schemaRegistryCompatibilityCheckers
[org.apache.pulsar.broker.service.schema.JsonSchemaCompatibilityCheck, 
org.apache.pulsar.broker.service.schema.AvroSchemaCompatibilityCheck, 
org.apache.pulsar.broker.service.schema.ProtobufNativeSchemaCompatibilityCheck]

But I can’t find documentation for it.

At first look, it seems that for my need it would be enough to have a such 
custom configurable Protobuf message descriptor checker class that implements 
SchemaCompatibilityCheck.

https://github.com/apache/pulsar/blob/82237d3684fe506bcb6426b3b23f413422e6e4fb/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/schema/ProtobufNativeSchemaCompatibilityCheck.java#L31

[cid:image002.png@01D94541.991807E0]

[Text  Description automatically generated]

From: SiNan Liu 
Date: Monday, February 20, 2023 at 1:41 PM
To: dev@pulsar.apache.org 
Subject: Re: [DISCUSS] PIP-246: Improved PROTOBUF_NATIVE schema compatibility 
checks without using avro-protobuf
It seems that we should only warn the user about changes to the field type,
not define a compatibility check, or throw an exception.
*Just like this: *
*log.warn("proto.read.ProtobufSchema.uint64Field field type for uint64, was
changed into a uint32");*

I will update this in the PIP issue Alternatives and discuss both designs
with other people.

Kiryl Valkovich  于2023年2月20日周一 18:14写道：

> 1. Sure, I didn’t mean that it's only about the required fields.
> 2. I found the page you are referring to.
> https://protobuf.dev/programming-guides/proto3/#updating
>
> QUOTE START
> If a number is parsed from the wire which doesn’t fit in the corresponding
> type, you will get the same effect as if you had cast the number to that
> type in C++ (for example, if a 64-bit number is read as an int32, it will
> be truncated to 32 bits).
> QUOTE END
>
> It’s discussable. Maybe I’m just missing something.
>
> I personally wouldn’t rely on such compatibility guarantees in a real
> application.
> If my check amount (> int32 lol) would be truncated because someone
> changed the field type in a schema, I would be quite upset.
>
> From: SiNan Liu 
> Date: Monday, February 20, 2023 at 2:30 AM
> To: dev@pulsar.apache.org 
> Subject: Re: [DISCUSS] PIP-246: Improved PROTOBUF_NATIVE schema
> compatibility checks without using avro-protobuf
> Thank you for your suggestions and questions.
> 1. Your first question. It's not just a matter of the required field. There
> should be many differences between proto3 and proto2. I will later test the
> current code for compatibility checks in proto3, and also look at
> compatibility between different protobuf versions.
> 2. On your second question. This is the compatibility rule for field type
> changes I found on the official website. You mean that this rule should not
> pass the compatibility check. Or should an exception be thrown whenever the
> field type changes?
>
>
> Kiryl Valkovich  于 2023年2月20日周一 上午12:31写道：
>
> > First, thank you for looking into it!
> >
> > There is a thing caught my eye:
> >
> > > The writtenSchema cannot add required fields, …
> >
> > As far as I know, the required fields were removed in Protocol Buffers
> v3.
> >
> > I see proto3 usage in existing schema registry tests:
> >
> >
> https://github.com/apache/pulsar/blob/6704f12104219611164aa2bb5bbdfc929613f1bf/pulsar-broker/src/test/proto/ProtobufSchemaTest.proto#L19
> >
> >
> >
> https://github.com/apache/pulsar/blob/1ea4ad814f5f30b8c371db2a86626cd568ace553/pulsar-sql/presto-pulsar/src/test/java/org/apache/pulsar/sql/presto/decoder/protobufnative/TestMsg.proto#L19
> >
> > I see proto2 usage in the proposed changes:
> >
> >
> >
> https://github.com/apache/pulsar/pull/19566/files#diff-f2f7463a3df6dc6366f3ee3a416707e655f0d5b8d2ae623a3f05a209c1d6ec88R19
> >
> >
> >
> https://github.com/apache/pulsar/pull/19566/files#diff-f2f7463a3df6dc6366f3ee3a416707e655f0d5b8d2ae623a3f05a209c1d6ec88R19
> >
> > Also, I’m not sure about this:
> >
> > > int32, uint32, int64, uint64, and bool are all compatible �C this means
> > you can change

Re: [DISCUSS] PIP-246: Improved PROTOBUF_NATIVE schema compatibility checks without using avro-protobuf

Hm. I tend to think that for my Pulsar use case, it would be good to have the 
ability to implement a custom schema compatibility checker.

For example, buf.build (a popular Protobuf linter and build) offers the 
following list of breaking rules. Half of them prefixed with 
FIELD_SAME_XXLANG_PACKAGE_ aren’t relevant.

And I would want to use a subset of them if we are checking a single message 
descriptor. Most likely:
ENUM_VALUE_NO_DELETE
FIELD_NO_DELETE
FIELD_SAME_TYPE
ONEOF_NO_DELETE
FIELD_SAME_LABEL
RESERVED_ENUM_NO_DELETE
RESERVED_MESSAGE_NO_DELETE

I found out that the Pulsar broker has the following configuration option: 
schemaRegistryCompatibilityCheckers
[org.apache.pulsar.broker.service.schema.JsonSchemaCompatibilityCheck, 
org.apache.pulsar.broker.service.schema.AvroSchemaCompatibilityCheck, 
org.apache.pulsar.broker.service.schema.ProtobufNativeSchemaCompatibilityCheck]

But I can’t find documentation for it.

At first look, it seems that for my need it would be enough to have a such 
custom configurable Protobuf message descriptor checker class that implements 
SchemaCompatibilityCheck.

https://github.com/apache/pulsar/blob/82237d3684fe506bcb6426b3b23f413422e6e4fb/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/schema/ProtobufNativeSchemaCompatibilityCheck.java#L31

[cid:image002.png@01D94541.991807E0]

[Text  Description automatically generated]

From: SiNan Liu 
Date: Monday, February 20, 2023 at 1:41 PM
To: dev@pulsar.apache.org 
Subject: Re: [DISCUSS] PIP-246: Improved PROTOBUF_NATIVE schema compatibility 
checks without using avro-protobuf
It seems that we should only warn the user about changes to the field type,
not define a compatibility check, or throw an exception.
*Just like this: *
*log.warn("proto.read.ProtobufSchema.uint64Field field type for uint64, was
changed into a uint32");*

I will update this in the PIP issue Alternatives and discuss both designs
with other people.


Kiryl Valkovich  于2023年2月20日周一 18:14写道：

> 1. Sure, I didn’t mean that it's only about the required fields.
> 2. I found the page you are referring to.
> https://protobuf.dev/programming-guides/proto3/#updating
>
> QUOTE START
> If a number is parsed from the wire which doesn’t fit in the corresponding
> type, you will get the same effect as if you had cast the number to that
> type in C++ (for example, if a 64-bit number is read as an int32, it will
> be truncated to 32 bits).
> QUOTE END
>
> It’s discussable. Maybe I’m just missing something.
>
> I personally wouldn’t rely on such compatibility guarantees in a real
> application.
> If my check amount (> int32 lol) would be truncated because someone
> changed the field type in a schema, I would be quite upset.
>
> From: SiNan Liu 
> Date: Monday, February 20, 2023 at 2:30 AM
> To: dev@pulsar.apache.org 
> Subject: Re: [DISCUSS] PIP-246: Improved PROTOBUF_NATIVE schema
> compatibility checks without using avro-protobuf
> Thank you for your suggestions and questions.
> 1. Your first question. It's not just a matter of the required field. There
> should be many differences between proto3 and proto2. I will later test the
> current code for compatibility checks in proto3, and also look at
> compatibility between different protobuf versions.
> 2. On your second question. This is the compatibility rule for field type
> changes I found on the official website. You mean that this rule should not
> pass the compatibility check. Or should an exception be thrown whenever the
> field type changes?
>
>
> Kiryl Valkovich  于 2023年2月20日周一 上午12:31写道：
>
> > First, thank you for looking into it!
> >
> > There is a thing caught my eye:
> >
> > > The writtenSchema cannot add required fields, …
> >
> > As far as I know, the required fields were removed in Protocol Buffers
> v3.
> >
> > I see proto3 usage in existing schema registry tests:
> >
> >
> https://github.com/apache/pulsar/blob/6704f12104219611164aa2bb5bbdfc929613f1bf/pulsar-broker/src/test/proto/ProtobufSchemaTest.proto#L19
> >
> >
> >
> https://github.com/apache/pulsar/blob/1ea4ad814f5f30b8c371db2a86626cd568ace553/pulsar-sql/presto-pulsar/src/test/java/org/apache/pulsar/sql/presto/decoder/protobufnative/TestMsg.proto#L19
> >
> > I see proto2 usage in the proposed changes:
> >
> >
> >
> https://github.com/apache/pulsar/pull/19566/files#diff-f2f7463a3df6dc6366f3ee3a416707e655f0d5b8d2ae623a3f05a209c1d6ec88R19
> >
> >
> >
> https://github.com/apache/pulsar/pull/19566/files#diff-f2f7463a3df6dc6366f3ee3a416707e655f0d5b8d2ae623a3f05a209c1d6ec88R19
> >
> > Also, I’m not sure about this:
> >
> > > int32, uint32, int64, uint64, and bool are all compatible �C this means
> > you can change a field from one of these types to another without
> breaking
> > forwards- or backwards-compatibility.
> >
> > It leads to JS/PHP-like implicit conversions if I understand it right.
> >
> > From: SiNan Liu 
> > Date: Sunday, February 19, 2023 at 1:59 PM
> > To: dev@pulsar.apache.org 
> > Subject: [DISCUSS] PIP-246: Improved

Re: [PROPOSAL] Roadmap for 3.0 release

2023-02-20 Thread Christophe Bornet

+1

Also, I'd like to volunteer as a release manager for this release.

Christophe




Le ven. 17 févr. 2023 à 23:44, Matteo Merli  a écrit :
>
> Since the LTS release model has been formally approved, I'm proposing
> the following schedule for the release:
>
>  * Tue - 2023-05-11
>- RC-1
>- Code Freeze -- Only critical fixes will be merged in the 3.0
> release branch. Contributors should plan to have all the changes merged in
> before this date. Exceptions should be extremely rare and strongly
> motivated.
>
>  * Tue - 2023-05-18 - RC-2
>  * Tue - 2023-05-25 - RC-3
>  * Tue - 2023-05-02 - Announce 3.0 Release
>
> These dates will be published on the website to present users with a
> "roadmap" and we should commit to and respect these dates.
>
> I also wanted to propose trying out a model where we have 3 release
> managers for all major releases.
>
> The reasoning behind this is for this small group of people to collaborate
> and divide the tasks for the release: merging patches from the "master"
> branch, preparing RC, and testing.
>
> Since everyone also has other work duties and unexpected tasks that can pop
> up at any time, it will help to have redundancy in the release-management
> "team", so that we can release on the exact dates.
>
> Thanks,
> Matteo
>
> --
> Matteo Merli
>

Re: Does anyone build UI for Pulsar?

Enrico, it’s easy.

When I tried it, the basic functionality just didn’t work.

Just make a side-by-side comparison of Pulsar Manager with any of the following 
options:
- Conduktor (Commercial).
- Kafka UI by Provectus (Apache License 2.0).
- Redpanda Console (ex-Kowl) (Mixed license).

I see a significant gap in user experience between any of them and the Pulsar 
Manager.
Also, I just don’t see much contribution activity.

Redpanda Console and Kafka UI, each have about x10 times more commits in a 
year, than Pulsar Manager has in the last 3 years. Therefore I can’t conclude 
that the project is alive.

Regarding the “Why from scratch?” question:
- I found it easier for me personally because of the different tech stack I use.
- Starting from scratch is a good opportunity to learn Pulsar itself.
- I’d want to make a product around that.

I'm not claiming that my result will be 100% better, but why not try?

If someone is already doing or plans to do similar work, let’s collaborate!

From: Enrico Olivelli 
Date: Monday, February 20, 2023 at 2:04 PM
To: dev@pulsar.apache.org 
Subject: Re: Does anyone build UI for Pulsar?
Kiryl,

Il giorno lun 20 feb 2023 alle ore 12:18 Kiryl Valkovich
 ha scritto:
>
> Enrico, it seems you read only the mail message title.

Sorry about that,
I have re-read the message, and I have just realised that I skipped
the very first line :-)

I have one question.
IIUC you started from scratch a new UI, could you please explain why
Pulsar Manager doesn't work for you?


Cheers
Enrico

>
> From: Enrico Olivelli 
> Date: Monday, February 20, 2023 at 11:59 AM
> To: dev@pulsar.apache.org 
> Subject: Re: Does anyone build UI for Pulsar?
> Kiryl,
>
> You can use the official Apache Pulsar Manager the is maintained by
> this community
> https://github.com/apache/pulsar-manager
>
> At DataStax we also maintain this other UI that is also 100% opensource
> https://github.com/datastax/pulsar-admin-console
>
> For the BookKeeper part there is BKVM (BookKeeper Visual Manager)
> https://github.com/diennea/bookkeeper-visual-manager
> This is also bundled with Apache Pulsar Manager
>
>
> I hope that helps
>
> Enrico
>
>
>
> Il giorno lun 20 feb 2023 alle ore 10:51 Kiryl Valkovich
>  ha scritto:
> >
> > Hi everyone!
> >
> > Does anyone personally or some company work on UI for Pulsar other than 
> > pulsar-manager or pulsar-admin-console?
> >
> > I understand that StreamNative and DataStax have managed solutions and 
> > obviously work on their UI.
> >
> > I rather looking for an open-source or commercial tool that can be used in 
> > pair with any Pulsar deployment.
> >
> > I spent some time implementing UI for Apache Pulsar. It’s not ready to 
> > release yet. As usual, the most difficult 20% of the work remained.
> >
> > I’m asking that to avoid situations when several people or teams are doing 
> > the same thing.
> > Recently I proposed the site redesign and some teams are already doing that 
> > as it’s figured out.
> >
> > See Asaf’s comment:
> > https://github.com/apache/pulsar-site/pull/426#issuecomment-1436589392
> >
> > Please at least DM me if it’s not public information yet.
> >
> > Also DM me if you think that we can try to cooperate.
> >
> > Thank you.

Re: [DISCUSS] Release 2.9.5

+1

Enrico

Il giorno lun 20 feb 2023 alle ore 10:41 Zike Yang  ha scritto:
>
> +1
>
> Thanks,
> Zike Yang
>
> On Mon, Feb 20, 2023 at 12:57 PM  wrote:
> >
> > Hello, Pulsar community:
> >
> > I'd like to propose releasing Apache Pulsar 2.9.5. It's been about
> > two months since 2.9.4 was released.
> >
> > There are 54 PRs [0] needed to cherry-pick in branch-2.9. I will
> > cherry-pick these PRs for branch-2.9. Exclude some PRs that merge
> > directly into branch-2.9.
> >
> > There are 21 PRs [1] opened. I'll follow up on each of those PRs to
> > see if they will be completed soon or will need to be pushed to 2.9.6
> >
> > If you have any important fixes or any questions, please reply to this
> > email, and we will evaluate whether to include them in 2.9.5
> >
> >
> > Best,
> > Mattison
> >
> > [0] 
> > https://github.com/apache/pulsar/pulls?q=is%3Apr+label%3Arelease%2F2.9.5+-label%3Acherry-picked%2Fbranch-2.9+is%3Aclosed+
> > [1] 
> > https://github.com/apache/pulsar/pulls?q=is%3Aopen+is%3Apr+label%3Arelease%2F2.9.5

Re: Does anyone build UI for Pulsar?

Kiryl,

Il giorno lun 20 feb 2023 alle ore 12:18 Kiryl Valkovich
 ha scritto:
>
> Enrico, it seems you read only the mail message title.

Sorry about that,
I have re-read the message, and I have just realised that I skipped
the very first line :-)

I have one question.
IIUC you started from scratch a new UI, could you please explain why
Pulsar Manager doesn't work for you?


Cheers
Enrico

>
> From: Enrico Olivelli 
> Date: Monday, February 20, 2023 at 11:59 AM
> To: dev@pulsar.apache.org 
> Subject: Re: Does anyone build UI for Pulsar?
> Kiryl,
>
> You can use the official Apache Pulsar Manager the is maintained by
> this community
> https://github.com/apache/pulsar-manager
>
> At DataStax we also maintain this other UI that is also 100% opensource
> https://github.com/datastax/pulsar-admin-console
>
> For the BookKeeper part there is BKVM (BookKeeper Visual Manager)
> https://github.com/diennea/bookkeeper-visual-manager
> This is also bundled with Apache Pulsar Manager
>
>
> I hope that helps
>
> Enrico
>
>
>
> Il giorno lun 20 feb 2023 alle ore 10:51 Kiryl Valkovich
>  ha scritto:
> >
> > Hi everyone!
> >
> > Does anyone personally or some company work on UI for Pulsar other than 
> > pulsar-manager or pulsar-admin-console?
> >
> > I understand that StreamNative and DataStax have managed solutions and 
> > obviously work on their UI.
> >
> > I rather looking for an open-source or commercial tool that can be used in 
> > pair with any Pulsar deployment.
> >
> > I spent some time implementing UI for Apache Pulsar. It’s not ready to 
> > release yet. As usual, the most difficult 20% of the work remained.
> >
> > I’m asking that to avoid situations when several people or teams are doing 
> > the same thing.
> > Recently I proposed the site redesign and some teams are already doing that 
> > as it’s figured out.
> >
> > See Asaf’s comment:
> > https://github.com/apache/pulsar-site/pull/426#issuecomment-1436589392
> >
> > Please at least DM me if it’s not public information yet.
> >
> > Also DM me if you think that we can try to cooperate.
> >
> > Thank you.

Re: [DISCUSS] PIP-246: Improved PROTOBUF_NATIVE schema compatibility checks without using avro-protobuf

2023-02-20 Thread SiNan Liu

It seems that we should only warn the user about changes to the field type,
not define a compatibility check, or throw an exception.
*Just like this: *
*log.warn("proto.read.ProtobufSchema.uint64Field field type for uint64, was
changed into a uint32");*

I will update this in the PIP issue Alternatives and discuss both designs
with other people.


Kiryl Valkovich  于2023年2月20日周一 18:14写道：

> 1. Sure, I didn’t mean that it's only about the required fields.
> 2. I found the page you are referring to.
> https://protobuf.dev/programming-guides/proto3/#updating
>
> QUOTE START
> If a number is parsed from the wire which doesn’t fit in the corresponding
> type, you will get the same effect as if you had cast the number to that
> type in C++ (for example, if a 64-bit number is read as an int32, it will
> be truncated to 32 bits).
> QUOTE END
>
> It’s discussable. Maybe I’m just missing something.
>
> I personally wouldn’t rely on such compatibility guarantees in a real
> application.
> If my check amount (> int32 lol) would be truncated because someone
> changed the field type in a schema, I would be quite upset.
>
> From: SiNan Liu 
> Date: Monday, February 20, 2023 at 2:30 AM
> To: dev@pulsar.apache.org 
> Subject: Re: [DISCUSS] PIP-246: Improved PROTOBUF_NATIVE schema
> compatibility checks without using avro-protobuf
> Thank you for your suggestions and questions.
> 1. Your first question. It's not just a matter of the required field. There
> should be many differences between proto3 and proto2. I will later test the
> current code for compatibility checks in proto3, and also look at
> compatibility between different protobuf versions.
> 2. On your second question. This is the compatibility rule for field type
> changes I found on the official website. You mean that this rule should not
> pass the compatibility check. Or should an exception be thrown whenever the
> field type changes?
>
>
> Kiryl Valkovich  于 2023年2月20日周一 上午12:31写道：
>
> > First, thank you for looking into it!
> >
> > There is a thing caught my eye:
> >
> > > The writtenSchema cannot add required fields, …
> >
> > As far as I know, the required fields were removed in Protocol Buffers
> v3.
> >
> > I see proto3 usage in existing schema registry tests:
> >
> >
> https://github.com/apache/pulsar/blob/6704f12104219611164aa2bb5bbdfc929613f1bf/pulsar-broker/src/test/proto/ProtobufSchemaTest.proto#L19
> >
> >
> >
> https://github.com/apache/pulsar/blob/1ea4ad814f5f30b8c371db2a86626cd568ace553/pulsar-sql/presto-pulsar/src/test/java/org/apache/pulsar/sql/presto/decoder/protobufnative/TestMsg.proto#L19
> >
> > I see proto2 usage in the proposed changes:
> >
> >
> >
> https://github.com/apache/pulsar/pull/19566/files#diff-f2f7463a3df6dc6366f3ee3a416707e655f0d5b8d2ae623a3f05a209c1d6ec88R19
> >
> >
> >
> https://github.com/apache/pulsar/pull/19566/files#diff-f2f7463a3df6dc6366f3ee3a416707e655f0d5b8d2ae623a3f05a209c1d6ec88R19
> >
> > Also, I’m not sure about this:
> >
> > > int32, uint32, int64, uint64, and bool are all compatible – this means
> > you can change a field from one of these types to another without
> breaking
> > forwards- or backwards-compatibility.
> >
> > It leads to JS/PHP-like implicit conversions if I understand it right.
> >
> > From: SiNan Liu 
> > Date: Sunday, February 19, 2023 at 1:59 PM
> > To: dev@pulsar.apache.org 
> > Subject: [DISCUSS] PIP-246: Improved PROTOBUF_NATIVE schema compatibility
> > checks without using avro-protobuf
> > Hi all,
> >
> > I made a PIP to discuss: https://github.com/apache/pulsar/issues/19565.
> >
> > We can talk about the current design here. Especially for the field type
> > change check rules, please give your valuable advice.
> >
> > Thanks,
> > Sinan
> >
>

Re: Does anyone build UI for Pulsar?

Enrico, it seems you read only the mail message title.

From: Enrico Olivelli 
Date: Monday, February 20, 2023 at 11:59 AM
To: dev@pulsar.apache.org 
Subject: Re: Does anyone build UI for Pulsar?
Kiryl,

You can use the official Apache Pulsar Manager the is maintained by
this community
https://github.com/apache/pulsar-manager

At DataStax we also maintain this other UI that is also 100% opensource
https://github.com/datastax/pulsar-admin-console

For the BookKeeper part there is BKVM (BookKeeper Visual Manager)
https://github.com/diennea/bookkeeper-visual-manager
This is also bundled with Apache Pulsar Manager


I hope that helps

Enrico



Il giorno lun 20 feb 2023 alle ore 10:51 Kiryl Valkovich
 ha scritto:
>
> Hi everyone!
>
> Does anyone personally or some company work on UI for Pulsar other than 
> pulsar-manager or pulsar-admin-console?
>
> I understand that StreamNative and DataStax have managed solutions and 
> obviously work on their UI.
>
> I rather looking for an open-source or commercial tool that can be used in 
> pair with any Pulsar deployment.
>
> I spent some time implementing UI for Apache Pulsar. It’s not ready to 
> release yet. As usual, the most difficult 20% of the work remained.
>
> I’m asking that to avoid situations when several people or teams are doing 
> the same thing.
> Recently I proposed the site redesign and some teams are already doing that 
> as it’s figured out.
>
> See Asaf’s comment:
> https://github.com/apache/pulsar-site/pull/426#issuecomment-1436589392
>
> Please at least DM me if it’s not public information yet.
>
> Also DM me if you think that we can try to cooperate.
>
> Thank you.

Re: Does anyone build UI for Pulsar?

Kiryl,

You can use the official Apache Pulsar Manager the is maintained by
this community
https://github.com/apache/pulsar-manager

At DataStax we also maintain this other UI that is also 100% opensource
https://github.com/datastax/pulsar-admin-console

For the BookKeeper part there is BKVM (BookKeeper Visual Manager)
https://github.com/diennea/bookkeeper-visual-manager
This is also bundled with Apache Pulsar Manager


I hope that helps

Enrico



Il giorno lun 20 feb 2023 alle ore 10:51 Kiryl Valkovich
 ha scritto:
>
> Hi everyone!
>
> Does anyone personally or some company work on UI for Pulsar other than 
> pulsar-manager or pulsar-admin-console?
>
> I understand that StreamNative and DataStax have managed solutions and 
> obviously work on their UI.
>
> I rather looking for an open-source or commercial tool that can be used in 
> pair with any Pulsar deployment.
>
> I spent some time implementing UI for Apache Pulsar. It’s not ready to 
> release yet. As usual, the most difficult 20% of the work remained.
>
> I’m asking that to avoid situations when several people or teams are doing 
> the same thing.
> Recently I proposed the site redesign and some teams are already doing that 
> as it’s figured out.
>
> See Asaf’s comment:
> https://github.com/apache/pulsar-site/pull/426#issuecomment-1436589392
>
> Please at least DM me if it’s not public information yet.
>
> Also DM me if you think that we can try to cooperate.
>
> Thank you.

Re: Force redirect questions from Slack to GitHub Discussions or StackOverflow?

Do you mean, to do it for all messages in the #general channel (maybe only for 
messages that contain the question mark)?

I think it makes sense.

From: Asaf Mesika 
Date: Sunday, February 19, 2023 at 11:54 AM
To: dev@pulsar.apache.org 
Subject: Re: Force redirect questions from Slack to GitHub Discussions or 
StackOverflow?
I would have the bot open a Thread for the message, *suggesting* the user
to click to convert this question into a GitHub Discussion question. This
way you can have the actual GitHub user asking the question and not a bot
one.

On Fri, Feb 17, 2023 at 10:59 PM Kiryl Valkovich 
wrote:

> What about such wording?
>
> ---
> Your question was moved here:
> https://github.com/apache/pulsar/discussions/123
>
> Please consider asking new questions here:
>
>   *   At StackOverflow using apache-pulsar tag.
>   *   In the Q category at GitHub Discussions.
>   *   Apache Pulsar User Mailing List.
>
>
> It will make it searchable by others. Also, this way we can collect a
> knowledge base outside of Slack over time.
>
> I can’t see how the words “please consider” force the user to do something.
>
> Users who have an account on StackOverflow or GitHub can use these
> platforms next time.
> Others can send their question via the mailing list.
>
> From: Dave Fisher 
> Date: Friday, February 17, 2023 at 9:28 PM
> To: dev@pulsar.apache.org 
> Subject: Re: Force redirect questions from Slack to GitHub Discussions or
> StackOverflow?
> My concern is that users should have a choice on where to post their
> questions. They might have concerns about GitHub’s terms and conditions. We
> can pin a message to slack pointing to GitHub discussions and stackoverflow.
>
> Best,
> Dave
>
> Sent from my iPhone
>
> > On Feb 17, 2023, at 9:22 AM, Kiryl Valkovich 
> wrote:
> >
> > I’m the owner of this account.
> > The goal is to test drive duplicating Slack questions to the GitHub
> discussions.
> > With the current level of activity in Slack it’s not so hard to do it
> manually.
> >
> > I’m in CET now. I can share the account credentials with people who can
> post questions to GitHub Discussions on behalf of this account in other
> time zones.
> > Or I can do it once a day.
> >
> > If someone doesn’t find it useful or has ideas on how to do it in a
> better way, just say it directly.
> >
> > From: Enrico Olivelli 
> > Date: Friday, February 17, 2023 at 3:43 PM
> > To: dev@pulsar.apache.org 
> > Subject: Re: Force redirect questions from Slack to GitHub Discussions
> or StackOverflow?
> > Hello,
> > I see that some "Pulsar Community Bot" appeared in Slack
> >
> > it is connected to this email address "pulsar.community@gmail.com"
> >
> > While I find this thing "amazing"I wonder if I missed something,
> > who is the owner of this "bot" ?
> >
> >
> > Enrico
> >
> >> Il giorno gio 16 feb 2023 alle ore 16:03 Kiryl Valkovich
> >>  ha scritto:
> >>
> >> Played with Slack and StackOverflow APIs a bit.
> >>
> >> The Slack API works as expected. After clicking the message button, it
> sends a message descriptor to my app where I can do anything with its
> content.
> >>
> >> It’s possible to post messages via the StackOverflow API, but it’s
> unlikely that any Slack message can be converted to a StackOverflow
> question.
> >>
> >> I encountered several types of validation errors:
> >>
> >> -  This question body does not meet our quality standards.
> Please make sure that it completely describes your problem - including what
> you have already tried - and is written using proper grammar.
> >>
> >>  *   Something similar to “This message looks like a duplicate of
> another message”.
> >>
> >> I believe, GitHub Discussions don’t have such kind of “quality
> standards” validation.
> >>
> >> From: Kiryl Valkovich 
> >> Date: Thursday, February 16, 2023 at 1:33 PM
> >> To: dev@pulsar.apache.org 
> >> Subject: Re: Force redirect questions from Slack to GitHub Discussions
> or StackOverflow?
> >> If there are no hidden obstacles, we can implement it.
> >> StackOverflow supports question creation using REST API:
> https://api.stackexchange.com/docs/create-question
> >>
> >> From: Zike Yang 
> >> Date: Thursday, February 16, 2023 at 11:41 AM
> >> To: dev@pulsar.apache.org 
> >> Subject: Re: Force redirect questions from Slack to GitHub Discussions
> or StackOverflow?
> >> +1, That sounds great to me.
> >>
> >>> 2. You click the three dots button on his message -> “Convert to a
> GitHub discussion”.
> >>
> >> Is it a similar workflow for converting to a StackOverflow question?
> >>
> >> BR,
> >> Zike Yang
> >>
> >>> On Thu, Feb 16, 2023 at 12:14 PM  wrote:
> >>>
> >>> +1
> >>>
> >>> Since web pages are more easily and public.
> >>>
> >>> Best
> >>> Mattison
> >>> On Feb 16, 2023, 07:58 +0800, Christophe Bornet <
> bornet.ch...@gmail.com>, wrote:
>  +100
>  Also note that for good or bad reasons, the number of questions on
>  StaOverflow is often used as a metric for the popularity of a project.
> 
>

Re: [PROPOSAL] Roadmap for 3.0 release

+1

Zike Yang


On Sat, Feb 18, 2023 at 4:43 PM  wrote:
>
> +1
> On Feb 18, 2023, 14:56 +0800, Michael Marshall , wrote:
> > +1 - this timeline sounds even better :)
> >
> > On Sat, Feb 18, 2023 at 12:41 AM Matteo Merli  
> > wrote:
> > >
> > > Ups,
> > >
> > > I started from the release date I was meaning April for the RCs:
> > >
> > > * Tue - 2023-04-11
> > > * Tue - 2023-04-18 - RC-2
> > > * Tue - 2023-04-25 - RC-3
> > > * Tue - 2023-05-02 - Announce 3.0 Release
> > >
> > > Sorry for the mixup!
> > >
> > > --
> > > Matteo Merli
> > > 
> > >
> > >
> > > On Fri, Feb 17, 2023 at 10:39 PM Dave Fisher  
> > > wrote:
> > >
> > > > > +1.
> > > > >
> > > > > I think that there is a typo. See below.
> > > > >
> > > > > Sent from my iPhone
> > > > >
> > > > > > > On Feb 17, 2023, at 2:44 PM, Matteo Merli  
> > > > > > > wrote:
> > > > > > >
> > > > > > > Since the LTS release model has been formally approved, I'm 
> > > > > > > proposing
> > > > > > > the following schedule for the release:
> > > > > > >
> > > > > > > * Tue - 2023-05-11
> > > > > > > - RC-1
> > > > > > > - Code Freeze -- Only critical fixes will be merged in the 3.0
> > > > > > > release branch. Contributors should plan to have all the changes 
> > > > > > > merged
> > > > > in
> > > > > > > before this date. Exceptions should be extremely rare and strongly
> > > > > > > motivated.
> > > > > > >
> > > > > > > * Tue - 2023-05-18 - RC-2
> > > > > > > * Tue - 2023-05-25 - RC-3
> > > > > > > * Tue - 2023-05-02 - Announce 3.0 Release
> > > > >
> > > > > You meant June 2, 2023?
> > > > >
> > > > > Best,
> > > > > Dave
> > > > > > >
> > > > > > > These dates will be published on the website to present users 
> > > > > > > with a
> > > > > > > "roadmap" and we should commit to and respect these dates.
> > > > > > >
> > > > > > > I also wanted to propose trying out a model where we have 3 
> > > > > > > release
> > > > > > > managers for all major releases.
> > > > > > >
> > > > > > > The reasoning behind this is for this small group of people to
> > > > > collaborate
> > > > > > > and divide the tasks for the release: merging patches from the 
> > > > > > > "master"
> > > > > > > branch, preparing RC, and testing.
> > > > > > >
> > > > > > > Since everyone also has other work duties and unexpected tasks 
> > > > > > > that can
> > > > > pop
> > > > > > > up at any time, it will help to have redundancy in the 
> > > > > > > release-management
> > > > > > > "team", so that we can release on the exact dates.
> > > > > > >
> > > > > > > Thanks,
> > > > > > > Matteo
> > > > > > >
> > > > > > > --
> > > > > > > Matteo Merli
> > > > > > > 
> > > > >
> > > > >

Re: [DISCUSS] PIP-246: Improved PROTOBUF_NATIVE schema compatibility checks without using avro-protobuf

1. Sure, I didn’t mean that it's only about the required fields.
2. I found the page you are referring to.
https://protobuf.dev/programming-guides/proto3/#updating

QUOTE START
If a number is parsed from the wire which doesn’t fit in the corresponding 
type, you will get the same effect as if you had cast the number to that type 
in C++ (for example, if a 64-bit number is read as an int32, it will be 
truncated to 32 bits).
QUOTE END

It’s discussable. Maybe I’m just missing something.

I personally wouldn’t rely on such compatibility guarantees in a real 
application.
If my check amount (> int32 lol) would be truncated because someone changed the 
field type in a schema, I would be quite upset.

From: SiNan Liu 
Date: Monday, February 20, 2023 at 2:30 AM
To: dev@pulsar.apache.org 
Subject: Re: [DISCUSS] PIP-246: Improved PROTOBUF_NATIVE schema compatibility 
checks without using avro-protobuf
Thank you for your suggestions and questions.
1. Your first question. It's not just a matter of the required field. There
should be many differences between proto3 and proto2. I will later test the
current code for compatibility checks in proto3, and also look at
compatibility between different protobuf versions.
2. On your second question. This is the compatibility rule for field type
changes I found on the official website. You mean that this rule should not
pass the compatibility check. Or should an exception be thrown whenever the
field type changes?


Kiryl Valkovich  于 2023年2月20日周一 上午12:31写道：

> First, thank you for looking into it!
>
> There is a thing caught my eye:
>
> > The writtenSchema cannot add required fields, …
>
> As far as I know, the required fields were removed in Protocol Buffers v3.
>
> I see proto3 usage in existing schema registry tests:
>
> https://github.com/apache/pulsar/blob/6704f12104219611164aa2bb5bbdfc929613f1bf/pulsar-broker/src/test/proto/ProtobufSchemaTest.proto#L19
>
>
> https://github.com/apache/pulsar/blob/1ea4ad814f5f30b8c371db2a86626cd568ace553/pulsar-sql/presto-pulsar/src/test/java/org/apache/pulsar/sql/presto/decoder/protobufnative/TestMsg.proto#L19
>
> I see proto2 usage in the proposed changes:
>
>
> https://github.com/apache/pulsar/pull/19566/files#diff-f2f7463a3df6dc6366f3ee3a416707e655f0d5b8d2ae623a3f05a209c1d6ec88R19
>
>
> https://github.com/apache/pulsar/pull/19566/files#diff-f2f7463a3df6dc6366f3ee3a416707e655f0d5b8d2ae623a3f05a209c1d6ec88R19
>
> Also, I’m not sure about this:
>
> > int32, uint32, int64, uint64, and bool are all compatible �C this means
> you can change a field from one of these types to another without breaking
> forwards- or backwards-compatibility.
>
> It leads to JS/PHP-like implicit conversions if I understand it right.
>
> From: SiNan Liu 
> Date: Sunday, February 19, 2023 at 1:59 PM
> To: dev@pulsar.apache.org 
> Subject: [DISCUSS] PIP-246: Improved PROTOBUF_NATIVE schema compatibility
> checks without using avro-protobuf
> Hi all,
>
> I made a PIP to discuss: https://github.com/apache/pulsar/issues/19565.
>
> We can talk about the current design here. Especially for the field type
> change check rules, please give your valuable advice.
>
> Thanks,
> Sinan
>

Re: [VOTE][PIP-242] Topic name restriction

+1 (non-binding)

Thanks,
Zike Yang


On Mon, Feb 20, 2023 at 1:53 PM PengHui Li  wrote:
>
> +1(binding)
>
> Thanks,
> Penghui
>
> On Mon, Feb 20, 2023 at 11:54 AM Cong Zhao  wrote:
>
> > +1 (non-binding)
> >
> > Thanks,
> > Cong
> >
> > On 2023/02/18 08:58:26 mattisonc...@gmail.com wrote:
> > > Hi, All
> > >
> > > After a fascinating discussion, I would start the vote of PIP-242.
> > >
> > > We have chosen to drop out the `system topic` related improvement to
> > another PIP. Therefore, the current version is simple enough and it has a
> > clear boundary.
> > >
> > > Please leave +1/-1 in this thread to join the vote. and feel free to
> > leave any concerns.
> > >
> > > Thanks to you guys.
> > >
> > > Best,
> > > Mattison
> > >
> > > References:
> > >
> > > • PIP https://github.com/apache/pulsar/issues/19239
> > > • Discussion
> > https://lists.apache.org/thread/oz79m0f2nw059jctq4cmms74yq5n2l1m
> > >
> > >
> > >
> >

Does anyone build UI for Pulsar?

Hi everyone!

Does anyone personally or some company work on UI for Pulsar other than 
pulsar-manager or pulsar-admin-console?

I understand that StreamNative and DataStax have managed solutions and 
obviously work on their UI.

I rather looking for an open-source or commercial tool that can be used in pair 
with any Pulsar deployment.

I spent some time implementing UI for Apache Pulsar. It’s not ready to release 
yet. As usual, the most difficult 20% of the work remained.

I’m asking that to avoid situations when several people or teams are doing the 
same thing.
Recently I proposed the site redesign and some teams are already doing that as 
it’s figured out.

See Asaf’s comment:
https://github.com/apache/pulsar-site/pull/426#issuecomment-1436589392

Please at least DM me if it’s not public information yet.

Also DM me if you think that we can try to cooperate.

Thank you.

Re: [DISCUSS] Release 2.9.5

+1

Thanks,
Zike Yang

On Mon, Feb 20, 2023 at 12:57 PM  wrote:
>
> Hello, Pulsar community:
>
> I'd like to propose releasing Apache Pulsar 2.9.5. It's been about
> two months since 2.9.4 was released.
>
> There are 54 PRs [0] needed to cherry-pick in branch-2.9. I will
> cherry-pick these PRs for branch-2.9. Exclude some PRs that merge
> directly into branch-2.9.
>
> There are 21 PRs [1] opened. I'll follow up on each of those PRs to
> see if they will be completed soon or will need to be pushed to 2.9.6
>
> If you have any important fixes or any questions, please reply to this
> email, and we will evaluate whether to include them in 2.9.5
>
>
> Best,
> Mattison
>
> [0] 
> https://github.com/apache/pulsar/pulls?q=is%3Apr+label%3Arelease%2F2.9.5+-label%3Acherry-picked%2Fbranch-2.9+is%3Aclosed+
> [1] 
> https://github.com/apache/pulsar/pulls?q=is%3Aopen+is%3Apr+label%3Arelease%2F2.9.5

Re: [DISCUSS] PIP-245: Subscriptions expiration for NonPersistentTopic only

> Regarding "+1 to throw this exception in future versions." - this means you
complete this issue once it throws an exception right?

More precisely, I mean we could print warning logs in the next feature
release 3.0.0, and throw exceptions in 3.1.0.
We better give users time to transition and modify the code.
And yes, the final effect of this PIP is to throw that exception.

BR,
Zike Yang

On Thu, Feb 16, 2023 at 9:27 PM Asaf Mesika  wrote:
>
> Regarding "+1 to throw this exception in future versions." - this means you
> complete this issue once it throws an exception right?
>
>
> On Wed, Feb 15, 2023 at 8:54 AM Zike Yang  wrote:
>
> > > if we change the default
> > subscription mode to be non-durable for non-persistent topics, then what
> > will happen to users who previously had durable subscriptions on that
> > topic? Are we ruining something if we create non-durable subscriptions
> > instead?
> >
> > I think it doesn't matter. Currently, the durable subscription on
> > non-persistent topics is actually non durable. It will be removed
> > after restarting the broker. Users should not depend on this logic. It
> > seems that it does not affect existing user behavior too much.
> >
> > > Can we add another phase to this PIP - say on version x we add WARN and
> > override mode to non-durable, and on version x+1 we fail the client?
> >
> > +1. We can add a section called the `Compatibility` section to this
> > PIP. +1 to throw this exception in future versions.
> >
> > Thanks,
> > Zike Yang
> >
> >
> > On Tue, Feb 14, 2023 at 5:11 PM Asaf Mesika  wrote:
> > >
> > > Can we add another phase to this PIP - say on version x we add WARN and
> > > override mode to non-durable, and on version x+1 we fail the client?
> > >
> > > On Tue, Feb 14, 2023 at 9:48 AM Jiuming Tao
> > 
> > > wrote:
> > >
> > > > Hi Asaf,
> > > >
> > > > If we fail the client when users use Durable subscription to subscribe
> > > > NonPersistentTopic, it will lead to break change.
> > > >
> > > > Users have to change their code, so we can change the
> > `subscriptionMode`
> > > > from `Durable` to `NonDurable` and print WARN log when users choose
> > > > `Durable`, then, users don’t need to change their code and we will not
> > > > introduce break change.
> > > >
> > > > The PIP will deprecate Durable subscriptions on NonPersistentTopic, all
> > > > the subscriptions on NonPersistentTopic will be NonDurable.
> > > >
> > > >
> > > > Thanks,
> > > > Tao Jiuming
> > > >
> > > >
> > > >
> > > >
> > > > > 2023年2月14日 03:24，Asaf Mesika  写道：
> > > > >
> > > > > Clarifying: as Pengu wrote in the pip comment - if we change the
> > default
> > > > > subscription mode to be non-durable for non-persistent topics, then
> > what
> > > > > will happen to users who previously had durable subscriptions on that
> > > > > topic? Are we ruining something if we create non-durable
> > subscriptions
> > > > > instead?
> > > > >
> > > > >
> > > > > On Mon, Feb 13, 2023 at 9:16 PM Asaf Mesika 
> > > > wrote:
> > > > >
> > > > >> Can we fail the client if they choose NonDurable subscription mode
> > on a
> > > > >> non-persistent topic, instead of writing WARN log messages?
> > > > >>
> > > > >>
> > > > >> On Mon, Feb 13, 2023 at 4:26 AM guo jiwei 
> > wrote:
> > > > >>
> > > > >>> Agree，+1
> > > > >>>
> > > > >>>
> > > > >>> Regards
> > > > >>> Jiwei Guo (Tboy)
> > > > >>>
> > > > >>> On Thu, Feb 9, 2023 at 7:07 PM Jiuming Tao
> > > > >>>  wrote:
> > > > 
> > > > 
> > > >  It makes sense
> > > > 
> > > >  Thanks,
> > > >  Tao Jiuming
> > > > 
> > > > 
> > > > > 2023年2月9日 15:44，Baodi Shi  写道：
> > > > >
> > > > > Agree, and we can add verification on the client side. When the
> > topic
> > > > >>> is
> > > > > non-persistent and uses durable to subscribe, print warns logs:
> > > > > “non-persistent not support durable subscribe will use
> > non-durable to
> > > > > subscribe.”
> > > > >
> > > > >
> > > > >
> > > > > Thanks,
> > > > > Baodi Shi
> > > > >
> > > > > 在 2023年2月9日 15:10:12 上，Jiuming Tao  > >
> > > > >>> 写道：
> > > > >
> > > > >> Agree. But we need to take care of the compatibility. How do we
> > > > >>> handle
> > > > >>
> > > > >> this compatibility?
> > > > >>
> > > > >>
> > > > >> How about set `isDurable = false` when create
> > > > >>> `NonPersistentSubscription`?
> > > > >>
> > > > >> In `NonPersistentSubscription`, we can change the constructor
> > from
> > > > >> ```
> > > > >> public NonPersistentSubscription(NonPersistentTopic topic,
> > String
> > > > >> subscriptionName, boolean isDurable,
> > > > >> Map properties) {
> > > > >> this.topic = topic;
> > > > >> this.topicName = topic.getName();
> > > > >> this.subName = subscriptionName;
> > > > >> this.fullName =
> > MoreObjects.toStringHelper(this).add("topic",
> > > > >> topicName).add("name", subName).toString();
> > > >

Re: [VOTE] Pulsar Node.js Client Release 1.8.1 Candidate 1

2023-02-20 Thread Nozomi Kurihara

+1 (binding)

* checked license headers
* verified checksum and signature
* install from npm and run producer/consumer

Thanks,
Nozomi

2023年2月17日(金) 19:12 Baodi Shi :

> Hi everyone,
>
> This is the first release candidate for Apache Pulsar Node.js client,
> version 1.8.1.
>
> It fixes the following
> issues:
> https://github.com/apache/pulsar-client-node/pulls?q=is%3Apr+label%3Arelease%2Fv1.8.1+is%3Aclosed
>
> Please download the source files and review this release candidate:
> - Download the source package, verify shasum and asc
> - Follow the README.md to build and run the Pulsar Node.js client.
>
> The release candidate package has been published to the npm
> registry:https://www.npmjs.com/package/pulsar-client/v/1.8.1-rc.1
> You can install it by `npm i pulsar-client@1.8.1-rc.1
> --pulsar_binary_host_mirror=
> https://dist.apache.org/repos/dist/dev/pulsar/pulsar-client-node/`
> 
> and verify the package.
>
> The vote will be open for at least 72 hours. It is adopted by majority
> approval, with at least 3 PMC affirmative votes.
>
> Source files:
> https://dist.apache.org/repos/dist/dev/pulsar/pulsar-client-node/pulsar-client-node-1.8.1-rc.1/
>
> Pulsar's KEYS file containing PGP keys we use to sign the
> release:https://dist.apache.org/repos/dist/dev/pulsar/KEYS
>
> SHA-512 checksum:
>
> ed89b4ad467d3cb75ed37096b35d91b872cd93d36cd953512fc7edcb75dbac5162592f6f51b5ab08f26b3dca1c57a3d3fe7a5e4f109551c66943a5b09392d51a
>  apache-pulsar-client-node-1.8.1.tar.gz
> The tag to be voted upon:
> v1.8.1-rc.1(3e843f0)
> https://github.com/apache/pulsar-client-node/releases/tag/v1.8.1-rc.1
>
> Please review and vote on the release candidate #1 for the version
> 1.8.1, as follows:
> [ ] +1, Approve the release
> [ ] -1, Do not approve the release (please provide specific comments)
>
>
>
>
> Thanks,
> Baodi Shi
>

Re: [Vote] PIP-245: Make subscriptions of non-persistent topic non-durable