Hi Xiangying > Must we set the system property to > false for topics that do not use > transactions?
It is needed because the `false` can be used to determine whether the topic was created before upgrading to the version that contains the current PIP or not, which is useful for compatibility support. see detail at the section Compatibility in the PIP. For example: - Before upgrading: the topic's property is empty, and it contains TXN messages. Brokers should start a TB recovery task for this topic because the property is empty. - After upgrading: the topics property is `false`, and it does not contain TXN messages. Thanks Yubiao Feng On Fri, Aug 23, 2024 at 4:47 PM xiangying meng <xiangy...@apache.org> wrote: > Hi yubiao > Then return to your plan, must we set the system property to false for > topics that do not use transactions? I understand that it is enough to > only set the system properties of the topics that used the > transaction. > > Thanks, > Xiangying Meng > > On Fri, Aug 23, 2024 at 4:16 PM Yubiao Feng > <yubiao.f...@streamnative.io.invalid> wrote: > > > > Hi Xiangying > > > > > Since we can add data to the topic attribute, > > > why not directly put the > > > snapshot or snapshot segment position into > > > the topic position? > > > > Reason 1: > > - The rate of loading the topic is once many days > > - The rate of taking Transaction Buffer snapshot is per 5 seconds > > > > The changes you suggested are to use write ZK per 5 seconds instead of > > reading data from BK once many days, it is not better. > > > > Reason 2: > > Once you move snapshots from BK to ZK, we can difficultly guarantee the > > compatibility of upgrading and downgrading. > > > > Thanks > > Yubioa Feng > > > > On Thu, Aug 22, 2024 at 4:52 PM xiangying meng <xiangy...@apache.org> > wrote: > > > > > Good idea. I still have some questions about this. > > > > > > Since we can add data to the topic attribute, why not directly put the > > > snapshot or snapshot segment position into the topic position? > > > This method has two advantages: > > > 1. System properties of topics that do not use transactions are not > > > added. It will not affect common topics. > > > 2. Topics using transactions can be recovered faster. > > > > > > Of course, the disadvantage is that topics using transactions will > > > need to update these properties frequently. > > > > > > Thanks, > > > > > > Xiangying > > > > > > On Thu, Aug 22, 2024 at 10:53 AM Ran Gao <r...@apache.org> wrote: > > > > > > > > LGTM > > > > > > > > Not all topics use the transaction feature, it can save a lot of > > > resources to help reduce broker recovery time, and it's convenient to > > > calculate topics count that use transactions. > > > > > > > > Thanks, > > > > Ran Gao > > > > > > > > On 2024/08/21 14:27:25 Yubiao Feng wrote: > > > > > Hi all > > > > > > > > > > I drafted a proposal to add a topic's system prop that indicates > > > whether > > > > > users have published TXN messages in before. > > > > > > > > > > https://github.com/apache/pulsar/pull/23210 > > > > > > > > > > I'm looking forward to hearing from you. > > > > > > > > > > Thanks > > > > > Yubiao Feng > > > > > > > > >
