Hi, All
First of all, I want to list all of the system topics as follows. That Yunze
has mentioned before.
Namespace level:
• pulsar/system
• transaction_coordinator_assign
• __transaction_log_
• resource-usage
• pulsar/<host>
• healthcheck
Topic level:
• __change_events
• __transaction_buffer_snapshot_segment
• ...
We can check the details here. [0]
Secondly. I think we need focus on the system topic name prefix. we have some
options as follows:
• __SYSTEM__
• __system__
Both them make sense for me. As there are already two people prefer `__SYSTEM`.
if no other people have concern, I will chose it.
Best,
Mattison
-[0]
https://github.com/apache/pulsar/blob/master/pulsar-common/src/main/java/org/apache/pulsar/common/naming/SystemTopicNames.java
On Feb 15, 2023, 11:36 +0800, Michael Marshall <mmarsh...@apache.org>, wrote:
> I support this PIP, thank you for driving it forward Mattison.
>
> I support using the prefix __SYSTEM__ for system topics, and I agree
> that DLQ and RETRY topics do not qualify as system topics because they
> are produced to and consumed by regular clients.
>
> I think we might benefit from a generic definition for a system topic.
> I provided a loose definition for system topics in this email [0]. I
> copy it here:
>
> A system topic is a topic that is completely internal to Pulsar
> components. Internally, it is a normal topic. It requires elevated
> permission to produce/consume when authorization is enabled, even if
> topic level policies are not enabled. Generic calls like
> `clearNamespaceBacklog` should not affect system topics. Deleting a
> namespace or tenant should delete the system topics within it.
>
> Thanks,
> Michael
>
> [0] https://lists.apache.org/thread/sr01hvqmdrnk4lxwfwzcpg7y21psj6tt
>
>
> On Tue, Feb 14, 2023 at 9:55 AM <mattisonc...@gmail.com> wrote:
> >
> > > > Could you list all existing system topic names that are used so we
> > > > canunderstand the rule better?
> > Yes, sure. I will list it later.
> > > > And I saw that this proposal only forbids the creation of thesetopics.
> > > > What about writing messages to them? I think it's better notto allow
> > > > Pulsar clients to write messages to them. The Pulsar clientshould
> > > > configure something to get the permission to write messages tothem. To
> > > > keep the compatibility, maybe we can make use of the protocolversion.
> > Sure, I agree with your point, because the current proposal just want to
> > have topic name restriction. maybe we can revise the system topic
> > permission and etc in the next proposal. (relate to system topic)
> >
> > Best,
> > Mattison
> > On Feb 14, 2023, 21:22 +0800, Yunze Xu <y...@streamnative.io.invalid>,
> > wrote:
> > > > Could you list all existing system topic names that are used so we can
> > > > understand the rule better? Such as the "-RETRY" and "-DLQ" topics
> > > > mentioned above.
> > > >
> > > > And I saw that this proposal only forbids the creation of these
> > > > topics. What about writing messages to them? I think it's better not
> > > > to allow Pulsar clients to write messages to them. The Pulsar client
> > > > should configure something to get the permission to write messages to
> > > > them. To keep the compatibility, maybe we can make use of the protocol
> > > > version.
> > > >
> > > > Thanks
> > > > Yunze
> > > >
> > > > On Tue, Feb 14, 2023 at 5:38 PM Yubiao Feng
> > > > <yubiao.f...@streamnative.io.invalid> wrote:
> > > > > >
> > > > > > Hi Qiang
> > > > > >
> > > > > > ### System topic name
> > > > > > '__system__<name>' I think this format is clearer.
> > > > > > Now the system automatically creates topics of type retry consumer
> > > > > > letters
> > > > > > and dead letters.
> > > > > > These topics all end in uppercase letters, such as `-RETRY,` `-DLQ.`
> > > > > > Is it better to define the system topic name in uppercase(
> > > > > > '__SYSTEM__<name>' )?
> > > > > >
> > > > > > ### Another comment
> > > > > > If you now redesign the topic name restrictions, should we make
> > > > > > `-RETRY`
> > > > > > and `-DLQ` keywords?
> > > > > >
> > > > > > Thanks
> > > > > > Yubiao Feng
> > > > > >
> > > > > > On Tue, Feb 14, 2023 at 5:06 PM Asaf Mesika <asaf.mes...@gmail.com>
> > > > > > wrote:
> > > > > >
> > > > > > > > > > On Tue, Feb 14, 2023 at 3:46 AM <mattisonc...@gmail.com>
> > > > > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > > > > > Hi, Asaf
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Welcome to join this discussion.
> > > > > > > > > > > > > > > > > > You mean that allows the *system* to use it
> > > > > > > > > > > > > > > > > > when it's a partitioned
> > > > > > > > > > > > > > topic?
> > > > > > > > > > > > > > Sorry, I didn't get your point. What do you mean by
> > > > > > > > > > > > > > *system*?
> > > > > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > This sentence was a reply to:
> > > > > > > > > >
> > > > > > > > > > 2. Make the `-partition-` string the keyword. That allows
> > > > > > > > > > the user to use
> > > > > > > > > > > > > > it when it's a partitioned topic.
> > > > > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > I wanted to say that this sentence should be:
> > > > > > > > > > Make the `-partition-` string the keyword, that allows the
> > > > > > > > > > *system* to use
> > > > > > > > > > it when it's a partitioned topic.
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > > > > > > > > Why postfix of `__`?Why uppercase ?Maybe
> > > > > > > > > > > > > > > > > > `__system__<name>`?
> > > > > > > > > > > > > > Yes, That is a key point that I want to discuss in
> > > > > > > > > > > > > > this
> > > > > > > > > > > > > > thread. `__system__<name>` is good for me.
> > > > > > > > > > > > > > > > > > Can you please elaborate what it means to
> > > > > > > > > > > > > > > > > > make it dynamic exactly?
> > > > > > > > > > > > > > Sorry, I will refine it. it means we can update
> > > > > > > > > > > > > > this configuration
> > > > > > > > > > > > > > dynamically. (using rest api or sth)
> > > > > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > I'm unfamiliar with how Pulsar supports dynamic
> > > > > > > > > > configuration. I would
> > > > > > > > > > love it if you can share a link or explain it briefly, thus
> > > > > > > > > > explaining what
> > > > > > > > > > exactly you are going to change to support dynamic
> > > > > > > > > > configuration.
> > > > > > > > > >
> > > > > > > > > > > > > > General question: In the last thread you said
> > > > > > > > > > > > > > something about
> > > > > > > > > > > > > > configurablerules, etc? You decided not to use this
> > > > > > > > > > > > > > idea?
> > > > > > > > > > > > > > IMO, That idea is an advanced feature. we may need
> > > > > > > > > > > > > > more time to discuss
> > > > > > > > > > > > > > the details and for the topic name restriction,
> > > > > > > > > > > > > > maybe we don't have
> > > > > > > > > > strong
> > > > > > > > > > > > > > reason to use that.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > We can introduce this advanced feature when we have
> > > > > > > > > > > > > > a need for it.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > WDYT?
> > > > > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > I agree. Future PIP and discussion.
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Best,
> > > > > > > > > > > > > > Mattison
> > > > > > > > > > > > > > On Feb 13, 2023, 22:21 +0800, Asaf Mesika
> > > > > > > > > > > > > > <asaf.mes...@gmail.com>,
> > > > > > > > > > wrote:
> > > > > > > > > > > > > > > > > > You mean that allows the *system* to use it
> > > > > > > > > > > > > > > > > > when it's a partitioned
> > > > > > > > > > > > > > topic?
> > > > > > > > > > > > > >
> > > > > > > > > >
