Mattison, Il giorno gio 16 feb 2023 alle ore 00:27 <mattisonc...@gmail.com> ha scritto: > > > I am sorry but I am not sure that this is enough to preventreads/writes > > from unallowed clients. > IMO, We can consider the authorisation part in another PIP because We are > just focusing on adding the topic name constraint of topic creation. > > Maybe we can use another PIP to clearify all of system topic's behaviour, > like authorisation something. > e.g. we just allow superusers to read/write the data to that system topic. > > We should elaborate more on this topic on the PIP > I will add the internal system topic creation logic in the PIP. Why do you think that this is enough ?
I think that we are going off the initial scope of the PIP. The initial problem is about preventing clients from creating topics that contain the "-partition-" keyword. I totally agree that there must be a clear way to distinguish topics that are not meant to be accessed by "regular clients". The answer is in Micheal's words: only super users are allowed to access topics that are not meant to be accessed by clients. Broker to Broker communications are always running with a "super user" role, so it is not a problem. BTW I wonder if it is better to narrow down the scope of the PIP and go back to "-partition-" Enrico > > Best, > Mattison > On Feb 16, 2023, 00:41 +0800, Enrico Olivelli <eolive...@gmail.com>, wrote: > > Il giorno mer 15 feb 2023 alle ore 17:07 <mattisonc...@gmail.com> ha > > scritto: > > > > > > Hi Enrico > > > > > > I think it's a good question. We can introduce a new method in the > > > BrokerService to help brokers create the topic internally first(maybe > > > just metadata is enough), and then to use a pulsar client to connect to > > > it. > > > > I am sorry but I am not sure that this is enough to prevent > > reads/writes from unallowed clients. > > We should elaborate more on this topic on the PIP > > > > Enrico > > > > > > > > WDYT? > > > > > > > > > Best, > > > Mattison > > > On Feb 16, 2023, 00:01 +0800, Enrico Olivelli <eolive...@gmail.com>, > > > wrote: > > > > > I have one question (apologies for the top posting). > > > > > > > > > > The Broker (and the other Pulsar components) use the regular Pulsar > > > > > client to connect to "system topics" > > > > > and in general they use the Pulsar wire protocol. > > > > > > > > > > The question is "how do you distinguish an internal component from a > > > > > user component ?" > > > > > How can you say that the broker is able to connect to a system topic > > > > > and any other client cannot do it ? > > > > > > > > > > Enrico > > > > > > > > > > Il giorno mer 15 feb 2023 alle ore 15:38 <mattisonc...@gmail.com> ha > > > > > scritto: > > > > > > > > > > > > > > Hi Asaf > > > > > > > > > > > > > > There is a link to introduce the dynamic configuration. > > > > > > > https://pulsar.apache.org/docs/2.10.x/admin-api-brokers/#dynamic-broker-configuration > > > > > > > > > > > > > > Best, > > > > > > > Mattison > > > > > > > On Feb 14, 2023, 17:06 +0800, Asaf Mesika > > > > > > > <asaf.mes...@gmail.com>, wrote: > > > > > > > > > > > On Tue, Feb 14, 2023 at 3:46 AM <mattisonc...@gmail.com> > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > Hi, Asaf > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Welcome to join this discussion. > > > > > > > > > > > > > > > > > > > > > > > You mean that allows the *system* > > > > > > > > > > > > > > > > > > > > > > > to use it when it's a partitioned > > > > > > > > > > > > > > > topic? > > > > > > > > > > > > > > > Sorry, I didn't get your point. What do you mean > > > > > > > > > > > > > > > by *system*? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > This sentence was a reply to: > > > > > > > > > > > > > > > > > > > > > > 2. Make the `-partition-` string the keyword. That allows > > > > > > > > > > > the user to use > > > > > > > > > > > > > > > it when it's a partitioned topic. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I wanted to say that this sentence should be: > > > > > > > > > > > Make the `-partition-` string the keyword, that allows > > > > > > > > > > > the *system* to use > > > > > > > > > > > it when it's a partitioned topic. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Why postfix of `__`?Why uppercase > > > > > > > > > > > > > > > > > > > > > > > ?Maybe `__system__<name>`? > > > > > > > > > > > > > > > Yes, That is a key point that I want to discuss > > > > > > > > > > > > > > > in this > > > > > > > > > > > > > > > thread. `__system__<name>` is good for me. > > > > > > > > > > > > > > > > > > > > > > > Can you please elaborate what it > > > > > > > > > > > > > > > > > > > > > > > means to make it dynamic exactly? > > > > > > > > > > > > > > > Sorry, I will refine it. it means we can update > > > > > > > > > > > > > > > this configuration > > > > > > > > > > > > > > > dynamically. (using rest api or sth) > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I'm unfamiliar with how Pulsar supports dynamic > > > > > > > > > > > configuration. I would > > > > > > > > > > > love it if you can share a link or explain it briefly, > > > > > > > > > > > thus explaining what > > > > > > > > > > > exactly you are going to change to support dynamic > > > > > > > > > > > configuration. > > > > > > > > > > > > > > > > > > > > > > > > > > General question: In the last thread you said > > > > > > > > > > > > > > > something about > > > > > > > > > > > > > > > configurablerules, etc? You decided not to use > > > > > > > > > > > > > > > this idea? > > > > > > > > > > > > > > > IMO, That idea is an advanced feature. we may > > > > > > > > > > > > > > > need more time to discuss > > > > > > > > > > > > > > > the details and for the topic name restriction, > > > > > > > > > > > > > > > maybe we don't have strong > > > > > > > > > > > > > > > reason to use that. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > We can introduce this advanced feature when we > > > > > > > > > > > > > > > have a need for it. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > WDYT? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I agree. Future PIP and discussion. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Best, > > > > > > > > > > > > > > > Mattison > > > > > > > > > > > > > > > On Feb 13, 2023, 22:21 +0800, Asaf Mesika > > > > > > > > > > > > > > > <asaf.mes...@gmail.com>, wrote: > > > > > > > > > > > > > > > > > > > > > > > You mean that allows the *system* > > > > > > > > > > > > > > > > > > > > > > > to use it when it's a partitioned > > > > > > > > > > > > > > > topic? > > > > > > > > > > > > > > >
