Re: Local visibility with logical decoding

2020-07-20 Thread Andres Freund 
Hi,

On 2020-07-20 17:27:30 +0300, Marko Tiikkaja wrote:
> It appears that when logical decoding sends out the data from the output
> plugin, it is not guaranteed that the decoded transaction's effects are
> visible on the source server.  Is this the way it's supposed to work?

At the moment the visibility behaviour is basically the same as crash
recovery / standbys. And they just look at the WAL...


> If so, would doing something like this in the output plugin be reasonable?
> 
> TransactionId xid = transaction->xid;
> if (transaction->is_known_as_subxact)
> xid = transaction->toplevel_xid;
> 
> if (TransactionIdIsInProgress(xid))
> XactLockTableWait(xid, NULL, NULL, XLTW_None);

I'd not be surprised if this had a potential to cause deadlocks.

Greetings,

Andres Freund

Re: Local visibility with logical decoding

2020-07-20 Thread Marko Tiikkaja 
On Mon, Jul 20, 2020 at 7:36 PM Antonin Houska  wrote:

> Marko Tiikkaja  wrote:
>
> > It appears that when logical decoding sends out the data from the output
> > plugin, it is not guaranteed that the decoded transaction's effects are
> > visible on the source server.  Is this the way it's supposed to work?
>
> Can you please share the test that indicates this behavior? As far as I
> understand, the transaction must have been committed before the output
> plugin
> starts to receive the changes.
>

I don't have a reliable test program, but you can reproduce quite easily
with test_decoding if you put a breakpoint before the SyncRepWaitForLSN()
call in src/backend/access/transam/xact.c.  pg_logicalrecv will see the
changes while the session is sitting on the breakpoint, and not finishing
its commit.


-marko

Re: Local visibility with logical decoding

2020-07-20 Thread Antonin Houska 
Marko Tiikkaja  wrote:

> It appears that when logical decoding sends out the data from the output
> plugin, it is not guaranteed that the decoded transaction's effects are
> visible on the source server.  Is this the way it's supposed to work?

Can you please share the test that indicates this behavior? As far as I
understand, the transaction must have been committed before the output plugin
starts to receive the changes.

-- 
Antonin Houska
Web: https://www.cybertec-postgresql.com