Is https://github.com/apache/pulsar/pull/19235 somehow related?
On Sat, Feb 18, 2023 at 10:38 AM <mattisonc...@gmail.com> wrote: > Hi, All > > After discussing with Enrico and Michael offline. > I will split the discussed topic into two PIP. > > 1. Topic name restrictions > a. `-partition-` keyword. > b. enable topic name character pattern. > 2. System topic > a. System topic name pattern. > b. System topic authorisation. > c. ... > > In this approach, we will get a clear boundary and avoid going off the > initial scope. > > Since we don't have any question about the first scope. I will start vote > next week. > > Thanks to all participant. > > Best, > Mattison > > > > On Feb 18, 2023, 14:24 +0800, Michael Marshall <mmarsh...@apache.org>, > wrote: > > I support breaking this into two PIPs. It was my fault the two PIPs > > were merged in the first place. I am sorry if I created any confusion. > > My intention was only to point out that names are a meaningful way to > > simplify logic, and we should reserve certain names for Pulsar's own > > usage with a well defined pattern so that we can simplify lifecycle > > operations. > > > > Thanks, > > Michael > > > > On Fri, Feb 17, 2023 at 1:55 AM Enrico Olivelli <eolive...@gmail.com> > wrote: > > > > > > Mattison, > > > > > > Il giorno gio 16 feb 2023 alle ore 00:27 <mattisonc...@gmail.com> ha > scritto: > > > > > > > > > > > > I am sorry but I am not sure that this is enough to > preventreads/writes from unallowed clients. > > > > > IMO, We can consider the authorisation part in another PIP because > We are just focusing on adding the topic name constraint of topic creation. > > > > > > > > > > Maybe we can use another PIP to clearify all of system topic's > behaviour, like authorisation something. > > > > > e.g. we just allow superusers to read/write the data to that > system topic. > > > > > > > We should elaborate more on this topic on the PIP > > > > > I will add the internal system topic creation logic in the PIP. > > > Why do you think that this is enough ? > > > > > > I think that we are going off the initial scope of the PIP. > > > The initial problem is about preventing clients from creating topics > > > that contain the "-partition-" keyword. > > > > > > I totally agree that there must be a clear way to distinguish topics > > > that are not meant to be accessed by "regular clients". > > > > > > The answer is in Micheal's words: only super users are allowed to > > > access topics that are not meant to be accessed by clients. > > > Broker to Broker communications are always running with a "super user" > > > role, so it is not a problem. > > > > > > BTW I wonder if it is better to narrow down the scope of the PIP and > > > go back to "-partition-" > > > > > > > > > Enrico > > > > > > > > > > > > > > > > Best, > > > > > Mattison > > > > > On Feb 16, 2023, 00:41 +0800, Enrico Olivelli <eolive...@gmail.com>, > wrote: > > > > > > > Il giorno mer 15 feb 2023 alle ore 17:07 < > mattisonc...@gmail.com> ha scritto: > > > > > > > > > > > > > > > > > > Hi Enrico > > > > > > > > > > > > > > > > > > I think it's a good question. We can introduce a new > method in the BrokerService to help brokers create the topic internally > first(maybe just metadata is enough), and then to use a pulsar client to > connect to it. > > > > > > > > > > > > > > I am sorry but I am not sure that this is enough to prevent > > > > > > > reads/writes from unallowed clients. > > > > > > > We should elaborate more on this topic on the PIP > > > > > > > > > > > > > > Enrico > > > > > > > > > > > > > > > > > > > > > > > > > WDYT? > > > > > > > > > > > > > > > > > > > > > > > > > > > Best, > > > > > > > > > Mattison > > > > > > > > > On Feb 16, 2023, 00:01 +0800, Enrico Olivelli < > eolive...@gmail.com>, wrote: > > > > > > > > > > > > > I have one question (apologies for the top > posting). > > > > > > > > > > > > > > > > > > > > > > > > > > The Broker (and the other Pulsar components) use > the regular Pulsar > > > > > > > > > > > > > client to connect to "system topics" > > > > > > > > > > > > > and in general they use the Pulsar wire protocol. > > > > > > > > > > > > > > > > > > > > > > > > > > The question is "how do you distinguish an > internal component from a > > > > > > > > > > > > > user component ?" > > > > > > > > > > > > > How can you say that the broker is able to connect > to a system topic > > > > > > > > > > > > > and any other client cannot do it ? > > > > > > > > > > > > > > > > > > > > > > > > > > Enrico > > > > > > > > > > > > > > > > > > > > > > > > > > Il giorno mer 15 feb 2023 alle ore 15:38 < > mattisonc...@gmail.com> ha scritto: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Hi Asaf > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > There is a link to introduce the dynamic > configuration. > > > > > > > > > > > > > > > > > > https://pulsar.apache.org/docs/2.10.x/admin-api-brokers/#dynamic-broker-configuration > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Best, > > > > > > > > > > > > > > > > > Mattison > > > > > > > > > > > > > > > > > On Feb 14, 2023, 17:06 +0800, Asaf Mesika < > asaf.mes...@gmail.com>, wrote: > > > > > > > > > > > > > > > > > > > > > > > > > On Tue, Feb 14, 2023 at > 3:46 AM <mattisonc...@gmail.com> wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Hi, Asaf > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Welcome to > join this discussion. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > You mean that allows the *system* to use it when it's > a partitioned > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > topic? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Sorry, I > didn't get your point. What do you mean by *system*? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > This sentence was a reply > to: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > 2. Make the `-partition-` > string the keyword. That allows the user to use > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > it when > it's a partitioned topic. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I wanted to say that this > sentence should be: > > > > > > > > > > > > > > > > > > > > > > > > > Make the `-partition-` > string the keyword, that allows the *system* to use > > > > > > > > > > > > > > > > > > > > > > > > > it when it's a partitioned > topic. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Why postfix of `__`?Why uppercase ?Maybe > `__system__<name>`? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Yes, That > is a key point that I want to discuss in this > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > thread. > `__system__<name>` is good for me. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Can you please elaborate what it means to make it > dynamic exactly? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Sorry, I > will refine it. it means we can update this configuration > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > dynamically. (using rest api or sth) > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I'm unfamiliar with how > Pulsar supports dynamic configuration. I would > > > > > > > > > > > > > > > > > > > > > > > > > love it if you can share a > link or explain it briefly, thus explaining what > > > > > > > > > > > > > > > > > > > > > > > > > exactly you are going to > change to support dynamic configuration. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > General > question: In the last thread you said something about > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > configurablerules, etc? You decided not to use this idea? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > IMO, That > idea is an advanced feature. we may need more time to discuss > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > the > details and for the topic name restriction, maybe we don't have strong > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > reason to > use that. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > We can > introduce this advanced feature when we have a need for it. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > WDYT? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I agree. Future PIP and > discussion. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Best, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Mattison > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Feb 13, > 2023, 22:21 +0800, Asaf Mesika <asaf.mes...@gmail.com>, wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > You mean that allows the *system* to use it when it's > a partitioned > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > topic? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
