I support breaking this into two PIPs. It was my fault the two PIPs were merged in the first place. I am sorry if I created any confusion. My intention was only to point out that names are a meaningful way to simplify logic, and we should reserve certain names for Pulsar's own usage with a well defined pattern so that we can simplify lifecycle operations.
Thanks, Michael On Fri, Feb 17, 2023 at 1:55 AM Enrico Olivelli <eolive...@gmail.com> wrote: > > Mattison, > > Il giorno gio 16 feb 2023 alle ore 00:27 <mattisonc...@gmail.com> ha scritto: > > > > > I am sorry but I am not sure that this is enough to preventreads/writes > > > from unallowed clients. > > IMO, We can consider the authorisation part in another PIP because We are > > just focusing on adding the topic name constraint of topic creation. > > > > Maybe we can use another PIP to clearify all of system topic's behaviour, > > like authorisation something. > > e.g. we just allow superusers to read/write the data to that system topic. > > > We should elaborate more on this topic on the PIP > > I will add the internal system topic creation logic in the PIP. > Why do you think that this is enough ? > > I think that we are going off the initial scope of the PIP. > The initial problem is about preventing clients from creating topics > that contain the "-partition-" keyword. > > I totally agree that there must be a clear way to distinguish topics > that are not meant to be accessed by "regular clients". > > The answer is in Micheal's words: only super users are allowed to > access topics that are not meant to be accessed by clients. > Broker to Broker communications are always running with a "super user" > role, so it is not a problem. > > BTW I wonder if it is better to narrow down the scope of the PIP and > go back to "-partition-" > > > Enrico > > > > > > Best, > > Mattison > > On Feb 16, 2023, 00:41 +0800, Enrico Olivelli <eolive...@gmail.com>, wrote: > > > Il giorno mer 15 feb 2023 alle ore 17:07 <mattisonc...@gmail.com> ha > > > scritto: > > > > > > > > Hi Enrico > > > > > > > > I think it's a good question. We can introduce a new method in the > > > > BrokerService to help brokers create the topic internally first(maybe > > > > just metadata is enough), and then to use a pulsar client to connect to > > > > it. > > > > > > I am sorry but I am not sure that this is enough to prevent > > > reads/writes from unallowed clients. > > > We should elaborate more on this topic on the PIP > > > > > > Enrico > > > > > > > > > > > WDYT? > > > > > > > > > > > > Best, > > > > Mattison > > > > On Feb 16, 2023, 00:01 +0800, Enrico Olivelli <eolive...@gmail.com>, > > > > wrote: > > > > > > I have one question (apologies for the top posting). > > > > > > > > > > > > The Broker (and the other Pulsar components) use the regular Pulsar > > > > > > client to connect to "system topics" > > > > > > and in general they use the Pulsar wire protocol. > > > > > > > > > > > > The question is "how do you distinguish an internal component from a > > > > > > user component ?" > > > > > > How can you say that the broker is able to connect to a system topic > > > > > > and any other client cannot do it ? > > > > > > > > > > > > Enrico > > > > > > > > > > > > Il giorno mer 15 feb 2023 alle ore 15:38 <mattisonc...@gmail.com> > > > > > > ha scritto: > > > > > > > > > > > > > > > > Hi Asaf > > > > > > > > > > > > > > > > There is a link to introduce the dynamic configuration. > > > > > > > > https://pulsar.apache.org/docs/2.10.x/admin-api-brokers/#dynamic-broker-configuration > > > > > > > > > > > > > > > > Best, > > > > > > > > Mattison > > > > > > > > On Feb 14, 2023, 17:06 +0800, Asaf Mesika > > > > > > > > <asaf.mes...@gmail.com>, wrote: > > > > > > > > > > > > On Tue, Feb 14, 2023 at 3:46 AM > > > > > > > > > > > > <mattisonc...@gmail.com> wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > Hi, Asaf > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Welcome to join this discussion. > > > > > > > > > > > > > > > > > > > > > > > > You mean that allows the > > > > > > > > > > > > > > > > > > > > > > > > *system* to use it when it's a > > > > > > > > > > > > > > > > > > > > > > > > partitioned > > > > > > > > > > > > > > > > topic? > > > > > > > > > > > > > > > > Sorry, I didn't get your point. What do you > > > > > > > > > > > > > > > > mean by *system*? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > This sentence was a reply to: > > > > > > > > > > > > > > > > > > > > > > > > 2. Make the `-partition-` string the keyword. That > > > > > > > > > > > > allows the user to use > > > > > > > > > > > > > > > > it when it's a partitioned topic. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I wanted to say that this sentence should be: > > > > > > > > > > > > Make the `-partition-` string the keyword, that allows > > > > > > > > > > > > the *system* to use > > > > > > > > > > > > it when it's a partitioned topic. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Why postfix of `__`?Why > > > > > > > > > > > > > > > > > > > > > > > > uppercase ?Maybe > > > > > > > > > > > > > > > > > > > > > > > > `__system__<name>`? > > > > > > > > > > > > > > > > Yes, That is a key point that I want to discuss > > > > > > > > > > > > > > > > in this > > > > > > > > > > > > > > > > thread. `__system__<name>` is good for me. > > > > > > > > > > > > > > > > > > > > > > > > Can you please elaborate what > > > > > > > > > > > > > > > > > > > > > > > > it means to make it dynamic > > > > > > > > > > > > > > > > > > > > > > > > exactly? > > > > > > > > > > > > > > > > Sorry, I will refine it. it means we can update > > > > > > > > > > > > > > > > this configuration > > > > > > > > > > > > > > > > dynamically. (using rest api or sth) > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I'm unfamiliar with how Pulsar supports dynamic > > > > > > > > > > > > configuration. I would > > > > > > > > > > > > love it if you can share a link or explain it briefly, > > > > > > > > > > > > thus explaining what > > > > > > > > > > > > exactly you are going to change to support dynamic > > > > > > > > > > > > configuration. > > > > > > > > > > > > > > > > > > > > > > > > > > > > General question: In the last thread you said > > > > > > > > > > > > > > > > something about > > > > > > > > > > > > > > > > configurablerules, etc? You decided not to use > > > > > > > > > > > > > > > > this idea? > > > > > > > > > > > > > > > > IMO, That idea is an advanced feature. we may > > > > > > > > > > > > > > > > need more time to discuss > > > > > > > > > > > > > > > > the details and for the topic name restriction, > > > > > > > > > > > > > > > > maybe we don't have strong > > > > > > > > > > > > > > > > reason to use that. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > We can introduce this advanced feature when we > > > > > > > > > > > > > > > > have a need for it. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > WDYT? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I agree. Future PIP and discussion. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Best, > > > > > > > > > > > > > > > > Mattison > > > > > > > > > > > > > > > > On Feb 13, 2023, 22:21 +0800, Asaf Mesika > > > > > > > > > > > > > > > > <asaf.mes...@gmail.com>, wrote: > > > > > > > > > > > > > > > > > > > > > > > > You mean that allows the > > > > > > > > > > > > > > > > > > > > > > > > *system* to use it when it's a > > > > > > > > > > > > > > > > > > > > > > > > partitioned > > > > > > > > > > > > > > > > topic? > > > > > > > > > > > > > > > >
