Thanks for starting this thread, Mattison.
> The topic name character validation is already done by
> `NamedEntity#checkName`.
Based on my reading of the code, only the tenant and the namespace
names are validated using that method. There is a call [0] that looks
like it validates topic names, but that method is only called by
tests.
> But I have a concern that whether we should
> treat all topics that start with the long underscore ("__") as system
> topics?
This is a reasonable concern, and my primary motivation in proposing
this change is to make it easier for the broker to handle system
topics, which often get unique treatment.
I wrote on this topic in several replies on this thread from a year ago [1].
In the context of PIP 242, we're introducing a config to optionally
enforce strict topic names. As such, we could rely on the config to
either use the "cheap" check to see if the topic starts with __ or we
could use the more expensive check to determine if the topic name is
one of many possible system topic names. Because we want to maintain
backwards compatibility, we cannot completely get rid of the old
logic. I like self describing names because they are elegant and
efficient.
> If yes, how would you like to allow users to access the system topics?
I proposed some ideas at the end of that thread [1]. We should have a
clear definition of system topics and how they are or are not accessed by
users. Ultimately, we continue to create new system topics without
reserving a designated naming structure and without defining how these
topics ought to be interacted with, as Yunze points out. Note that any
system topic we introduce could conflict with existing user topics, so
proactively reserving a set of names makes it easier for forwards
compatibility.
Thanks,
Michael
[0]
https://github.com/apache/pulsar/blob/b880b1d240ade864181935aa360bfca03a5aa67f/pulsar-common/src/main/java/org/apache/pulsar/common/naming/NamespaceName.java#L159
[1] https://lists.apache.org/thread/pj4n4wzm3do8nkc52l7g7obh0sktzm17
On Wed, Feb 1, 2023 at 11:28 PM r...@apache.org <ranxiaolong...@gmail.com>
wrote:
>
> Hi Mattison:
>
> Agree with Yong's idea. We can expose `disallowed topic` as a configuration
> to the user side, and a more flexible way is to expose it as a
> namespace-level policy. This can ensure that there is no need to do special
> processing on customized keywords in the future, and the expected effect
> can be achieved by modifying the configuration.
>
> Think Yunze's concerns are justified for the system topic. Is it okay if we
> use hard code? Because the identification of any keyword is likely to be
> hit by the user. The hard code method is used to filter out system topics
> and not allow users to operate during delete and create operations.
>
> --
> Thanks
> Xiaolong Ran
>
> Dave Fisher <wave4d...@comcast.net> 于2023年2月2日周四 11:26写道：
>
> >
> >
> > Sent from my iPhone
> >
> > > On Feb 1, 2023, at 6:52 PM, Yong Zhang <zhangyong1025...@gmail.com>
> > wrote:
> > >
> > > Mattison,
> > >
> > > I agree with you about restricting the topic name.
> > >
> > > How about using a blacklist way to restrict it?
> >
> > If we do then please call it by another name like “disallowed”.
> >
> > >
> > > We can have a blacklist on the topic name restriction and make it
> > > configurable. Add the keywords you mentioned in the default
> > configuration.
> > > That would have a more general way to block a topic name creation.
> > > If we have more restrictions on the topic name in the future, this way
> > > can make it easy to fit them without changing any code.
> >
> > Is there anyone asking for this feature?
> >
> > Best,
> > Dave
> > >
> > > Thanks,
> > > Yong
> > >
> > >> On Thu, 2 Feb 2023 at 07:33, <mattisonc...@gmail.com> wrote:
> > >>
> > >> Hi, All
> > >>
> > >> In the current implementation, pulsar didn't support topic name
> > >> restriction. It's a good chance to discuss it.
> > >>
> > >> I think this discussion aims to identify what types of topic names we
> > all
> > >> need to restrict.
> > >>
> > >> I know three topic names that need to be restricted at the moment.
> > >>
> > >> 1. The `-partition-` keyword.
> > >> 2. Topic name characters validation.
> > >> 3. System topic prefix `__`.
> > >>
> > >>
> > >> Please feel free to leave your comments.
> > >> I will keep this discussion for a week. If there are no more new types
> > of
> > >> restrictions, I will refine the previous PIP-242[0] to explain more
> > details.
> > >>> If we have other restrictions behind this discussion. We can draft a
> > new
> > >> PIP to add it directly.
> > >> Thanks to Michael's opinion[1], we can expand the PIP-242 scopes to help
> > >> pulsar have a good topic name restriction.
> > >>
> > >> Best,
> > >> Mattison
> > >>
> > >> [0] https://github.com/apache/pulsar/issues/19239
> > >> [1] https://lists.apache.org/thread/dd1kxhodjvovtb8yyojkk209st4o0ft2
> > >>
> >
> >
