Hi Joe, You're correct. The initial design of the replication policy leaves room for improvement. To address this, we aim to refine the cluster settings at the namespace level in a way that won't impact the existing system. The replication clusters should solely be used to establish full mesh replication for that specific namespace, without having any other definitions or functionalities.
BR, Xiangying On Mon, Dec 4, 2023 at 1:52 PM Joe F <joefranc...@gmail.com> wrote: > >if users want to change the replication policy for > topic-n and do not change the replication policy of other topics, they need > to change all the topic policy under this namespace. > > This PIP unfortunately flows from attempting to solve bad application > design > > A namespace is supposed to represent an application, and the namespace > policy is an umbrella for a similar set of policies that applies to all > topics. The exceptions would be if a topic had a need for a deficit, The > case of one topic in the namespace sticking out of the namespace policy > umbrella is bad application design in my opinion > > -Joe. > > > > On Sun, Dec 3, 2023 at 6:00 PM Xiangying Meng <xiangy...@apache.org> > wrote: > > > Hi Rajan and Girish, > > Thanks for your reply. About the question you mentioned, there is some > > information I want to share with you. > > >If anyone wants to setup different replication clusters then either > > >those topics can be created under different namespaces or defined at > topic > > >level policy. > > > > >And users can anyway go and update the namespace's cluster list to add > the > > >missing cluster. > > Because the replication clusters also mean the clusters where the topic > can > > be created or loaded, the topic-level replication clusters can only be > the > > subset of namespace-level replication clusters. > > Just as Girish mentioned, the users can update the namespace's cluster > list > > to add the missing cluster. > > But there is a problem because the replication clusters as the namespace > > level will create a full mesh replication for that namespace across the > > clusters defined in > > replication-clusters if users want to change the replication policy for > > topic-n and do not change the replication policy of other topics, they > need > > to change all the topic policy under this namespace. > > > > > Pulsar is being used by many legacy systems and changing its > > >semantics for specific usecases without considering consequences are > > >creating a lot of pain and incompatibility problems for other existing > > >systems and let's avoid doing it as we are struggling with such changes > > and > > >breaking compatibility or changing semantics are just not acceptable. > > > > This proposal will not introduce an incompatibility problem, because the > > default value of the namespace policy of allowed-clusters and > > topic-policy-synchronized-clusters are the replication-clusters. > > > > >Allowed clusters defined at tenant level > > >will restrict tenants to create namespaces in regions/clusters where > they > > >are not allowed. > > >As Rajan also mentioned, allowed-clusters field has a different > > meaning/purpose. > > > > Allowed clusters defined at the tenant level will restrict tenants from > > creating namespaces in regions/clusters where they are not allowed. > > Similarly, the allowed clusters defined at the namespace level will > > restrict the namespace from creating topics in regions/clusters where > they > > are not allowed. > > What's wrong with this? > > > > Regards, > > Xiangying > > > > On Fri, Dec 1, 2023 at 2:35 PM Girish Sharma <scrapmachi...@gmail.com> > > wrote: > > > > > Hi Xiangying, > > > > > > Shouldn't the solution to the issue mentioned in #21564 [0] mostly be > > > around validating that topic level replication clusters are subset of > > > namespace level replication clusters? > > > It would be a completely compatible change as even today the case > where a > > > topic has a cluster not defined in namespaces's replication-clusters > > > doesn't really work. > > > And users can anyway go and update the namespace's cluster list to add > > the > > > missing cluster. > > > > > > As Rajan also mentioned, allowed-clusters field has a different > > > meaning/purpose. > > > Regards > > > > > > On Thu, Nov 30, 2023 at 10:56 AM Xiangying Meng <xiangy...@apache.org> > > > wrote: > > > > > > > Hi, Pulsar Community > > > > > > > > I drafted a proposal to make the configuration of clusters at the > > > namespace > > > > level clearer. This helps solve the problem of geo-replication not > > > working > > > > correctly at the topic level. > > > > > > > > https://github.com/apache/pulsar/pull/21648 > > > > > > > > I'm looking forward to hearing from you. > > > > > > > > BR > > > > Xiangying > > > > > > > > > > > > > -- > > > Girish Sharma > > > > > >
