On Mon, Dec 7, 2020 at 8:35 AM Craig Ringer <craig.rin...@enterprisedb.com> wrote: > > Reply follows inline. I addressed your last point first, so it's out of order. > > On Fri, 4 Dec 2020 at 15:33, Andrey Borodin <x4...@yandex-team.ru> wrote > > We'd also need to separate the existing apply worker into a "receiver" and > "apply/writer" part, so the wire-protocol handling isn't tightly coupled with > the actual change apply code, in order to make it possible to actually > consume those archives and apply them to the database. In pglogical3 we did > that by splitting them into two processes, connected by a shm_mq. Originally > the process split was optional and you could run a combined receiver/writer > process without the shm_mq if you wanted, but we quickly found it difficult > to reliably handle locking issues etc that way so the writers all moved > out-of-process. > > That was done mainly to make it possible to support parallelism in logical > decoding apply. But we also have the intention of supporting an alternative > reader process that can ingest "logical archives" and send them to the writer > to apply them, as if they'd been received from the on-wire stream. That's not > implemented at this time though. It'd be useful for a number of things: > > * PITR-style logical replay and recovery > * Ability to pre-decode a txn once on the upstream then send the buffered > protocol-stream to multiple subscribers, saving on logical decoding and > reorder buffering overheads and write-multiplication costs >
I think doing parallel apply and ability to decode a txn once are really good improvements independent of all the work you listed. Thanks for sharing your knowledge. -- With Regards, Amit Kapila.
