Hi Yubiao If it's just for compatibility, wouldn't it be better to add a global property? There is no need to add it for every topic.
Thanks On Fri, Aug 23, 2024 at 5:23 PM Yubiao Feng <yubiao.f...@streamnative.io.invalid> wrote: > > Hi Xiangying > > > Must we set the system property to > > false for topics that do not use > > transactions? > > It is needed because the `false` can be used to determine whether the topic > was created before upgrading to the version that contains the current PIP > or not, which is useful for compatibility support. see detail at the > section Compatibility in the PIP. For example: > - Before upgrading: the topic's property is empty, and it contains TXN > messages. Brokers should start a TB recovery task for this topic because > the property is empty. > - After upgrading: the topics property is `false`, and it does not contain > TXN messages. > > Thanks > Yubiao Feng > > On Fri, Aug 23, 2024 at 4:47 PM xiangying meng <xiangy...@apache.org> wrote: > > > Hi yubiao > > Then return to your plan, must we set the system property to false for > > topics that do not use transactions? I understand that it is enough to > > only set the system properties of the topics that used the > > transaction. > > > > Thanks, > > Xiangying Meng > > > > On Fri, Aug 23, 2024 at 4:16 PM Yubiao Feng > > <yubiao.f...@streamnative.io.invalid> wrote: > > > > > > Hi Xiangying > > > > > > > Since we can add data to the topic attribute, > > > > why not directly put the > > > > snapshot or snapshot segment position into > > > > the topic position? > > > > > > Reason 1: > > > - The rate of loading the topic is once many days > > > - The rate of taking Transaction Buffer snapshot is per 5 seconds > > > > > > The changes you suggested are to use write ZK per 5 seconds instead of > > > reading data from BK once many days, it is not better. > > > > > > Reason 2: > > > Once you move snapshots from BK to ZK, we can difficultly guarantee the > > > compatibility of upgrading and downgrading. > > > > > > Thanks > > > Yubioa Feng > > > > > > On Thu, Aug 22, 2024 at 4:52 PM xiangying meng <xiangy...@apache.org> > > wrote: > > > > > > > Good idea. I still have some questions about this. > > > > > > > > Since we can add data to the topic attribute, why not directly put the > > > > snapshot or snapshot segment position into the topic position? > > > > This method has two advantages: > > > > 1. System properties of topics that do not use transactions are not > > > > added. It will not affect common topics. > > > > 2. Topics using transactions can be recovered faster. > > > > > > > > Of course, the disadvantage is that topics using transactions will > > > > need to update these properties frequently. > > > > > > > > Thanks, > > > > > > > > Xiangying > > > > > > > > On Thu, Aug 22, 2024 at 10:53 AM Ran Gao <r...@apache.org> wrote: > > > > > > > > > > LGTM > > > > > > > > > > Not all topics use the transaction feature, it can save a lot of > > > > resources to help reduce broker recovery time, and it's convenient to > > > > calculate topics count that use transactions. > > > > > > > > > > Thanks, > > > > > Ran Gao > > > > > > > > > > On 2024/08/21 14:27:25 Yubiao Feng wrote: > > > > > > Hi all > > > > > > > > > > > > I drafted a proposal to add a topic's system prop that indicates > > > > whether > > > > > > users have published TXN messages in before. > > > > > > > > > > > > https://github.com/apache/pulsar/pull/23210 > > > > > > > > > > > > I'm looking forward to hearing from you. > > > > > > > > > > > > Thanks > > > > > > Yubiao Feng > > > > > > > > > > > >
