Re: [Discuss] PIP to add system topic for topic creation/deletion events

2021-04-21 Thread Michael Marshall
Hi Joe, I agree there is a risk in adding more interdependencies between brokers. I will point out that we have already accepted this risk with the implementation of PIP 39, which propagates namespace policy changes to other brokers using messages sent to a system topic. However, that doesn't nece

Re: Transaction Manager Coordinator Deployment

2021-04-21 Thread Devin Bost
How computationally intensive is the responsibility of the Transaction Coordinator? -- Devin G. Bost On Wed, Apr 21, 2021, 3:01 PM Sijie Guo wrote: > Enrico - I was suggesting not even making it a standalone. TC is a > component that really tied to Pulsar protocol and logic. We shouldn't do it

Re: [Discuss] PIP to add system topic for topic creation/deletion events

2021-04-21 Thread Joe F
I would be very careful about implementing such a feature, because of introducing undesirable interdependencies. Broker processes only talk to the metadata store or data store. This keeps brokers isolated from each other - one broker is not dependent on the functioning of another broker. A broke

Re: [Discuss] PIP to add system topic for topic creation/deletion events

2021-04-21 Thread Michael Marshall
Thanks for your response, PengHui. I think this feature would be useful to end users for cluster management, which is why I want to contribute a first class feature instead of writing my own plugin that would add little value to the community. > With the broker interceptor you can intercept all t

Re: Transaction Manager Coordinator Deployment

2021-04-21 Thread Sijie Guo
Enrico - I was suggesting not even making it a standalone. TC is a component that really tied to Pulsar protocol and logic. We shouldn't do it until it is really needed. - Sijie On Wed, Apr 21, 2021 at 1:21 PM Enrico Olivelli wrote: > Il giorno mer 21 apr 2021 alle ore 21:57 Sijie Guo > ha scr

Re: Transaction Manager Coordinator Deployment

2021-04-21 Thread Enrico Olivelli
Il giorno mer 21 apr 2021 alle ore 21:57 Sijie Guo ha scritto: > > We build transaction components in modules. It provides the flexibility for > people to run those components independently. > > However, I don't think we should encourage this deployment at the > beginning. Because most of the user

Re: PIP-26 - Delayed Messages and Exclusive subcriptions

2021-04-21 Thread Sijie Guo
> My main problem is that when you use some subscription types the feature does not work and there is no error or any kind of feedback on the consumer side. If the problem is "there is no error or any kind of feedback on the consumer side", we should fix the error message. But I don't think we sh

Re: Transaction Manager Coordinator Deployment

2021-04-21 Thread Sijie Guo
We build transaction components in modules. It provides the flexibility for people to run those components independently. However, I don't think we should encourage this deployment at the beginning. Because most of the users don't have deep knowledge about how the transaction works and have no ide

Re: Transaction Manager Coordinator Deployment

2021-04-21 Thread Chris Herzog
Modularity like this is a huge plus for production deployments - it gives the ability to scale more effectively and maintain a closer match between resources and services where appropriate. I think it's a solid idea. We're almost certainly going to want to end up with this level of control so it

Re: Transaction Manager Coordinator Deployment

2021-04-21 Thread Enrico Olivelli
Il giorno mer 21 apr 2021 alle ore 16:18 PengHui Li ha scritto: > > Hi Enrico, > > The current implementation of the transaction coordinator is run with the > broker, but the transaction coordinator > Is a separate module in Pulsar. This is also mentioned in PIP-31 > https://github.com/apache/pu

Re: Transaction Manager Coordinator Deployment

2021-04-21 Thread PengHui Li
Hi Enrico, The current implementation of the transaction coordinator is run with the broker, but the transaction coordinator Is a separate module in Pulsar. This is also mentioned in PIP-31  https://github.com/apache/pulsar/wiki/PIP-31%3A-Transaction-Support Thanks, Penghui On Apr 21, 2021, 7:24

[GitHub] [pulsar-helm-chart] zymap opened a new pull request #115: Reduce the TLS common name length

2021-04-21 Thread GitBox
zymap opened a new pull request #115: URL: https://github.com/apache/pulsar-helm-chart/pull/115 --- *Motivation* Reduce the TLS command name to avoid getting a too long name that could not generate a certificate. -- This is an automated message from the Apache Git Ser

Transaction Manager Coordinator Deployment

2021-04-21 Thread Enrico Olivelli
Hello, Pulsar 2.8.0 is coming near to the release and we want transactions to be "production ready". If I understand correctly now we start the Transaction Coordinator service inside the Broker. Does it make sense to have the option to run the TC as a separate service, as we do for the Functions

Re: Cutting 2.7.2 release?

2021-04-21 Thread PengHui Li
I see, Thanks, Penghui Penghui On Apr 21, 2021, 6:09 PM +0800, Enrico Olivelli , wrote: > Il giorno mer 21 apr 2021 alle ore 11:57 PengHui Li > ha scritto: > > > > Hi enrico, > > > > I noticed there are some issues and PRs are marked as release/2.7.2 > > https://github.com/apache/pulsar/labels/

Re: Cutting 2.7.2 release?

2021-04-21 Thread Enrico Olivelli
Il giorno mer 21 apr 2021 alle ore 11:57 PengHui Li ha scritto: > > Hi enrico, > > I noticed there are some issues and PRs are marked as release/2.7.2 > https://github.com/apache/pulsar/labels/release%2F2.7.2 Thanks ! I will check and ensure that everything is done. I am still testing the relea

Re: Cutting 2.7.2 release?

2021-04-21 Thread PengHui Li
Hi enrico, I noticed there are some issues and PRs are marked as release/2.7.2  https://github.com/apache/pulsar/labels/release%2F2.7.2 We’d better to start a discussion through the email thread to check if some of them can be complete in the next couple days or some of them can Be moved to 2.7.3

Re: [Discuss] PIP to add system topic for topic creation/deletion events

2021-04-21 Thread PengHui Li
Hi Michael, Currently, Pulsar supports a pluginable Broker Interceptor, you can find it here  https://github.com/apache/pulsar/blob/6704f12104219611164aa2bb5bbdfc929613f1bf/pulsar-broker/src/main/java/org/apache/pulsar/broker/intercept/BrokerInterceptor.java With the broker interceptor you can i

Re: Cutting 2.7.2 release?

2021-04-21 Thread Enrico Olivelli
I have started the release procedure I have come to a problem and submitted a patch Can someone please take a look ? https://github.com/apache/pulsar/pull/10302 Enrico Il giorno lun 19 apr 2021 alle ore 11:50 Enrico Olivelli ha scritto: > > Devin, > > Il giorno ven 16 apr 2021 alle ore 10:28 De