nodece commented on code in PR #23770: URL: https://github.com/apache/pulsar/pull/23770#discussion_r1959583186
########## pip/pip-398.md: ########## @@ -0,0 +1,245 @@ +# PIP-398: Subscription replication on the broker, namespace and topic levels + +# Background knowledge + +https://github.com/apache/pulsar/pull/4299 introduces the subscription replication feature on the consumer level: + +```java +Consumer<byte[]> consumer = pulsarClient.newConsumer().topic("topic").replicateSubscriptionState(false/*true*/) + .subscriptionName("sub").subscribe(); +``` + +While this provides flexibility, it introduces overhead in managing replication for a large number of consumers. Users +need to manually enable the `replicateSubscriptionState` flag for each consumer, which can become cumbersome in +large-scale deployments. + +# Motivation + +The key motivation behind this PIP is to simplify subscription replication configuration, especially in failover +scenarios. When a main cluster goes down and a backup cluster is activated, ensuring that subscription states are +consistently replicated across clusters is critical for failover scenarios. By extending the replication configuration +to the broker, namespace and topic levels, the system reduces the need for explicit consumer-level configuration. + +# Goals + +## In Scope + +The PIP aims to provide management of subscription replication at the broker, namespace and topic levels using the +Pulsar Admin CLI and API. + +# High Level Design + +Introduces a configuration used for enabling subscription replication on the broker, namespace and topic levels, when +enabled, all consumers under the broker/namespace/topic will automatically replicate their subscription states to remote +clusters. + +The priority for the subscription replication configuration is as follows: + +- consumer level > topic level > namespace level > broker level. +- If `replicateSubscriptionState` is set at the consumer level, configurations at the topic, namespace, and broker levels are + ignored. +- If set at the topic level, the namespace-level configuration is ignored. +- If set at the namespace level, the broker-level configuration is ignored. Review Comment: > I think that there could be a need to override any value set on the consumer side. It's also possible. > I also added some comments that controlling the behavior at namespace and consumer level should perhaps be a policy concern instead of creating a separate concept for handling the setting. Yes. This config is present in the broker/namespace policies/topic policies/cursor property. > If the setting at namespace and topic level would be more than just a true/false. It would be possible to add a separate setting that would allow overriding the consumer level setting. I don't want to introduce more config, so when the consume level is false, the broker/namespace policies/topic policies can override the consume level setting. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
