Hi community I started a PIP about "Just auto-create no-partitioned DLQ And Prevent auto-create a DLQ for a DLQ".
PIP link: https://github.com/apache/pulsar/issues/20033 ### Motivation #### Just auto-create no-partitioned DLQ/Retry Topic If enabled the config `allowAutoTopicCreation,` Pulsar will auto-create a topic when the client loads it; After setting config `allowAutoTopicCreationType=partitioned, defaultNumPartitions=2`, Pulsar will auto-create a partitioned topic(which have two partitions) when the client loads it. After the above, if using the feature [Retry Topic]( https://pulsar.apache.org/docs/2.11.x/concepts-messaging/#retry-letter-topic) and [DLQ]( https://pulsar.apache.org/docs/2.11.x/concepts-messaging/#dead-letter-topic) enable topic auto-creation, we will get a partitioned DLQ and a partitioned Retry Topic like this: - `{primary_topic_name}-{sub_name}-DLQ` -`{primary_topic_name}-{sub_name}-DLQ-partition-0` -`{primary_topic_name}-{sub_name}-DLQ-partition-1` - `{primary_topic_name}-{sub_name}-RETRY` -`{primary_topic_name}-{sub_name}-RETRY-partition-0` -`{primary_topic_name}-{sub_name}-RETRY-partition-1` ---- I feel that almost all users will not use the multi-partitioned DLQ or multi-partitioned Retry topic because there is a bug that causes the above behavior to be incorrect, but we have yet to receive any issues about it. This bug causes the above behavior to look like this: When the partitioned DLQ is auto-created for the topic `tp1-partition-0`, Pulsar will create a partitioned topic meta which has two partitioned but only create a topic named `{primary_topic_name}-{sub_name}-DLQ,` there is no topic named `{primary_topic_name}-{sub_name}-DLQ-partition-x.` Please look at this [PR]( https://github.com/apache/pulsar/pull/19841) for a detailed bug description. So I want to change the behavior to Just auto-create no-partitioned DLQ/Retry Topic. ---- #### Prevent auto-create the DLQ for a DLQ Please look at this [Discussion]( https://lists.apache.org/thread/q1m23ckyy10wvtzy65v8bwqwnh7r0gc8) for the detail. ---- ### Goal - Just auto-create no-partitioned DLQ/Retry Topic(with the other words: prevent auto-create partitioned DLQ) - DLQ/Retry topic should not create for a DLQ/Retry Topic - roles: - DLQ will not auto-create for a DLQ - Retry Topic will not auto-create for a Retry Topic - DLQ will not auto-create for a Retry Topic - Retry Topic will not auto-create for a DLQ - client changes: Clients will not create a DLQ for a DLQ - broker changes: rejected the request which wants to auto-create a DLQ for a DLQ ---- ### API Changes #### CommandSubscribe.java ```java /** * This is an enumeration value with tree options: "standard", "dead letter", "retry letter". */ private String topicPurpose; ``` #### Properties of Topic ```properties "purposeOfAutoCreatedTopic": value with tree options: "standard", "dead letter", "retry letter" ``` Why not use two properties: `isAutoCreated` and `topicPurpose`? Because there is a scenario like this: auto-create a topic, use it as a DLQ after a few days, and not use it as a DLQ after a few days, this Topic will be allowed to have DLQ/Retry Topic. We only mark the topics created for DLQ/Retry purposes. Thanks Yubiao Feng
