On Fri, Jun 23, 2023 at 7:03 PM Melih Mutlu <m.melihmu...@gmail.com> wrote: > > You can find the updated patchset attached. > I worked to address the reviews and made some additional changes. > > Let me first explain the new patchset. > 0001: Refactors the logical replication code, mostly worker.c and > tablesync.c. Although this patch makes it easier to reuse workers, I > believe that it's useful even by itself without other patches. It does > not improve performance or anything but aims to increase readability > and such. > 0002: This is only to reuse worker processes, everything else stays > the same (replication slots/origins etc.). > 0003: Adds a new command for streaming replication protocol to create > a snapshot by an existing replication slot. > 0004: Reuses replication slots/origins together with workers. > > Even only 0001 and 0002 are enough to improve table sync performance > at the rates previously shared on this thread. This also means that > currently 0004 (reusing replication slots/origins) does not improve as > much as I would expect, even though it does not harm either. > I just wanted to share what I did so far, while I'm continuing to > investigate it more to see what I'm missing in patch 0004. >
I think the reason why you don't see the benefit of the 0004 patches is that it still pays the cost of disconnect/connect and we haven't saved much on network transfer costs because of the new snapshot you are creating in patch 0003. Is it possible to avoid disconnect/connect each time the patch needs to reuse the same tablesync worker? Once, we do that and save the cost of drop_slot and associated network round trip, you may see the benefit of 0003 and 0004 patches. -- With Regards, Amit Kapila.
