RE: Proposal for Consumer Filtering in Pulsar brokers

2020-11-19 Thread Kramer, Andre
t: Re: Proposal for Consumer Filtering in Pulsar brokers Joe - Very comprehensive writeup! > So my vote is not to allow this (and any other server side logic implementations) into the base dispatcher, but permit these kinds of changes as configurable dispatchers. I hope I have explained the reasons

Re: Proposal for Consumer Filtering in Pulsar brokers

2020-11-16 Thread Rajan Dhabalia
your benchmarking/testing > > methodology > > > and the benchmark results to the community. > > > > > > 2. How are you going to handle topics with end-to-end encryption > enabled? > > > > > > 3. How do you handle acknowledgment for the messages that have

Re: Proposal for Consumer Filtering in Pulsar brokers

2020-11-16 Thread Sijie Guo
;t see it is discussed > > in the PIP. Especially, how is it related to different subscription > types? > > > > One suggestion - If this PIP is approved, my recommendation is to use the > > NAR classloader to load the class. You can check how Pulsar uses NAR >

Re: Proposal for Consumer Filtering in Pulsar brokers

2020-11-16 Thread Dave Fisher
ion types? >> >> One suggestion - If this PIP is approved, my recommendation is to use the >> NAR classloader to load the class. You can check how Pulsar uses NAR >> classloader for other interfaces. >> >> Thanks, >> Sijie >> >> On Mon, N

Re: Proposal for Consumer Filtering in Pulsar brokers

2020-11-16 Thread Joe F
doc to wiki pages. It's mainly text so > > can be converted easily. > > > > Cheers, > > Andre > > > > -----Original Message- > > From: Sijie Guo > > Sent: 13 November 2020 19:08 > > To: Dev > > Subject: Re: Proposal for Consumer Filt

Re: Proposal for Consumer Filtering in Pulsar brokers

2020-11-16 Thread Sijie Guo
t; > X%20-%20Consumer-filtering.pdf Hopefully that could be used to start > > the discussion? > > > > Regards, > > Andre > > > > -Original Message- > > From: Sijie Guo > > Sent: 12 November 2020 18:32 > > To: Dev > > Subject: Re:

RE: Proposal for Consumer Filtering in Pulsar brokers

2020-11-16 Thread Kramer, Andre
Sure, please feel free to copy the doc to wiki pages. It's mainly text so can be converted easily. Cheers, Andre -Original Message- From: Sijie Guo Sent: 13 November 2020 19:08 To: Dev Subject: Re: Proposal for Consumer Filtering in Pulsar brokers Andre, Is it possible to put

RE: Proposal for Consumer Filtering in Pulsar brokers

2020-11-16 Thread Kramer, Andre
roposal for Consumer Filtering in Pulsar brokers Hi Andre, Thanks for this proposal. Besides Sijie's comments, there is also a PIP 70: https://github.com/apache/pulsar/wiki/PIP-70%3A-Introduce-lightweight-raw-Message-metadata, do you think it could help on this proposal? We have discussed this con

Re: Proposal for Consumer Filtering in Pulsar brokers

2020-11-15 Thread Jia Zhai
st: > > > https://github.com/andrekramer1/pulsar/blob/consumer-filter2-7-0/PIP-XX%20-%20Consumer-filtering.pdf > > Hopefully that could be used to start the discussion? > > > > Regards, > > Andre > > > > -Original Message- > > From: Sijie Guo

Re: Proposal for Consumer Filtering in Pulsar brokers

2020-11-13 Thread Sijie Guo
m: Sijie Guo > Sent: 12 November 2020 18:32 > To: Dev > Subject: Re: Proposal for Consumer Filtering in Pulsar brokers > > Hi Andre, > > I didn't see the attached writeup. Can you write a PIP for this feature? > Given it is a big feature, it would be good to discuss it

RE: Proposal for Consumer Filtering in Pulsar brokers

2020-11-13 Thread Kramer, Andre
November 2020 18:32 To: Dev Subject: Re: Proposal for Consumer Filtering in Pulsar brokers Hi Andre, I didn't see the attached writeup. Can you write a PIP for this feature? Given it is a big feature, it would be good to discuss it through a PIP. - Sijie On Thu, Nov 12, 2020 at 6:17 AM Kramer,

Re: Proposal for Consumer Filtering in Pulsar brokers

2020-11-12 Thread Sijie Guo
Hi Andre, I didn't see the attached writeup. Can you write a PIP for this feature? Given it is a big feature, it would be good to discuss it through a PIP. - Sijie On Thu, Nov 12, 2020 at 6:17 AM Kramer, Andre wrote: > Hello everyone, > > > > We at Software AG have prototyped adding filtering

Proposal for Consumer Filtering in Pulsar brokers

2020-11-12 Thread Kramer, Andre
Hello everyone, We at Software AG have prototyped adding filtering on Consumer subscriptions in the Pulsar broker and are submitting our changes for consideration under Apache 2.0 license. Please see pull request [Consumer Filtering #8544 https://github.com/apache/pulsar/pull/8544] and attached