On 8 May 2017 05:56, "Daniele Varrazzo" <[email protected]> wrote:
On Sun, May 7, 2017 at 8:04 PM, Andres Freund <[email protected]> wrote: > Hi, > > On 2017-05-07 19:27:08 +0100, Daniele Varrazzo wrote: >> I'm putting together a replication system based on logical >> replication. > > Interesting. If you very briefly could recap what it's about... ;) I need to replicate some tables from a central database into the database that should run a secondary system. For a similar use case we have used Londiste in the past, which has served us good, but its usage has not been problem-free. Logical decoding seems much less invasive on the source database than a trigger-based replication solution, and has less moving part to care about and maintain. For the moment I'm hacking into a fork of Euler project for wal decoding into json (https://github.com/dvarrazzo/wal2json), mostly adding configurability, so that we may be able to replicate only the tables we need, skip certain fields etc. I'm also taking a look at minimising the amount of information produced: sending over and over the column names and types for every record seems a waste, hence my question. Sounds like you're reimplementing pglogical ( http://2ndquadrant.com/pglogical) on top of a json protocol. Pglogical is open source and hackable to meet your needs. We're also happy to accept patches with appropriate design discussion and code review to make sure they will aid or not hinder other users and won't add undue maintenance burden. Rather than repeat the same work, maybe it's worth picking it up as a starting point. It's the extension-based progenitor of the in-core logical rep code, but portable from 9.4 to 9.6 (and soon 10) with a bunch of features that didn't make it into Pg's version yet. I have no reason to object to your doing it yourself, and you're welcome to use pglogical as a reference for how to do things (see the license). It just seems like a waste.
