Re: [HACKERS] Logical replication - TRAP: FailedAssertion in pgstat.c

[Masahiko Sawada]

On Tue, May 9, 2017 at 1:26 AM, Petr Jelinek
 wrote:
> On 08/05/17 17:52, Masahiko Sawada wrote:
>> On Fri, May 5, 2017 at 8:13 PM, Petr Jelinek
>>  wrote:
>>> On 03/05/17 13:23, Erik Rijkers wrote:
 On 2017-05-03 08:17, Petr Jelinek wrote:
> On 02/05/17 20:43, Robert Haas wrote:
>> On Thu, Apr 20, 2017 at 2:58 PM, Peter Eisentraut

>>> code path that calls CommitTransactionCommand() should have one, no?
>>
>> Is there anything left to be committed here?
>>
>
> Afaics the fix was not committed. Peter wanted more comprehensive fix
> which didn't happen. I think something like attached should do the job.

 I'm running my pgbench-over-logical-replication test in chunk of 15
 minutes, wth different pgbench -c (num clients) and -s (scale) values.

 With this patch (and nothing else)  on top of master (8f8b9be51fd7 to be
 precise):

> fix-statistics-reporting-in-logical-replication-work.patch

 logical replication is still often failing (as expected, I suppose; it
 seems because of "inital snapshot too large") but indeed I do not see
>>>
>>> Yes that's different thing that we've been discussing a bit in snapbuild
>>> woes thread.
>>>
 the 'TRAP: FailedAssertion in pgstat.c' anymore.

 (If there is any other configuration of patches worth testing please let
 me know)

>>>
>>> Thanks, so the patch works.
>>>
>>
>> I think that we should commit the local transaction that did initial
>> data copy, and then report stat as well. Currently table sync worker
>> doesn't commit the local transaction in LogicalRepSyncTableStart
>> (maybe until apply commit record?) if its status is changed to
>> SUBREL_STATE_CATCHUP. That's why the table sync worker issues
>> assertion failure.
>>
>
> That would fix the assert as well yes, but it would also mean that if
> the worker crashed between the initial copy and the end of catchup there
> would be no way to restart it without manual intervention from user
> since the synchronization position would be lost. Hence the fix I
> proposed which does it differently and has the whole sync in a single
> transaction.

I understood that the data synchronization even including apply
logical record after changed to SUBREL_STATE_CATCHUP should be done in
a single transaction. Thank you for explanation.

Regards,

--
Masahiko Sawada
NIPPON TELEGRAPH AND TELEPHONE CORPORATION
NTT Open Source Software Center


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Logical replication - TRAP: FailedAssertion in pgstat.c

[Petr Jelinek]

On 08/05/17 18:12, Peter Eisentraut wrote:
> On 5/5/17 07:13, Petr Jelinek wrote:
>> Yes that's different thing that we've been discussing a bit in snapbuild
>> woes thread.
>>
>>> the 'TRAP: FailedAssertion in pgstat.c' anymore.
>>>
>>> (If there is any other configuration of patches worth testing please let
>>> me know)
>>
>> Thanks, so the patch works.
> 
> Committed, with s/xact_started/started_tx/, to match nearby code.
> 

Thanks!

-- 
  Petr Jelinek  http://www.2ndQuadrant.com/
  PostgreSQL Development, 24x7 Support, Training & Services


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Logical replication - TRAP: FailedAssertion in pgstat.c

[Petr Jelinek]

On 08/05/17 17:52, Masahiko Sawada wrote:
> On Fri, May 5, 2017 at 8:13 PM, Petr Jelinek
>  wrote:
>> On 03/05/17 13:23, Erik Rijkers wrote:
>>> On 2017-05-03 08:17, Petr Jelinek wrote:
 On 02/05/17 20:43, Robert Haas wrote:
> On Thu, Apr 20, 2017 at 2:58 PM, Peter Eisentraut
>>>
>> code path that calls CommitTransactionCommand() should have one, no?
>
> Is there anything left to be committed here?
>

 Afaics the fix was not committed. Peter wanted more comprehensive fix
 which didn't happen. I think something like attached should do the job.
>>>
>>> I'm running my pgbench-over-logical-replication test in chunk of 15
>>> minutes, wth different pgbench -c (num clients) and -s (scale) values.
>>>
>>> With this patch (and nothing else)  on top of master (8f8b9be51fd7 to be
>>> precise):
>>>
 fix-statistics-reporting-in-logical-replication-work.patch
>>>
>>> logical replication is still often failing (as expected, I suppose; it
>>> seems because of "inital snapshot too large") but indeed I do not see
>>
>> Yes that's different thing that we've been discussing a bit in snapbuild
>> woes thread.
>>
>>> the 'TRAP: FailedAssertion in pgstat.c' anymore.
>>>
>>> (If there is any other configuration of patches worth testing please let
>>> me know)
>>>
>>
>> Thanks, so the patch works.
>>
> 
> I think that we should commit the local transaction that did initial
> data copy, and then report stat as well. Currently table sync worker
> doesn't commit the local transaction in LogicalRepSyncTableStart
> (maybe until apply commit record?) if its status is changed to
> SUBREL_STATE_CATCHUP. That's why the table sync worker issues
> assertion failure.
> 

That would fix the assert as well yes, but it would also mean that if
the worker crashed between the initial copy and the end of catchup there
would be no way to restart it without manual intervention from user
since the synchronization position would be lost. Hence the fix I
proposed which does it differently and has the whole sync in a single
transaction.

-- 
  Petr Jelinek  http://www.2ndQuadrant.com/
  PostgreSQL Development, 24x7 Support, Training & Services


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Logical replication - TRAP: FailedAssertion in pgstat.c

[Peter Eisentraut]

On 5/5/17 07:13, Petr Jelinek wrote:
> Yes that's different thing that we've been discussing a bit in snapbuild
> woes thread.
> 
>> the 'TRAP: FailedAssertion in pgstat.c' anymore.
>>
>> (If there is any other configuration of patches worth testing please let
>> me know)
> 
> Thanks, so the patch works.

Committed, with s/xact_started/started_tx/, to match nearby code.

-- 
Peter Eisentraut  http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Logical replication - TRAP: FailedAssertion in pgstat.c

[Masahiko Sawada]

On Fri, May 5, 2017 at 8:13 PM, Petr Jelinek
 wrote:
> On 03/05/17 13:23, Erik Rijkers wrote:
>> On 2017-05-03 08:17, Petr Jelinek wrote:
>>> On 02/05/17 20:43, Robert Haas wrote:
 On Thu, Apr 20, 2017 at 2:58 PM, Peter Eisentraut
>>
> code path that calls CommitTransactionCommand() should have one, no?

 Is there anything left to be committed here?

>>>
>>> Afaics the fix was not committed. Peter wanted more comprehensive fix
>>> which didn't happen. I think something like attached should do the job.
>>
>> I'm running my pgbench-over-logical-replication test in chunk of 15
>> minutes, wth different pgbench -c (num clients) and -s (scale) values.
>>
>> With this patch (and nothing else)  on top of master (8f8b9be51fd7 to be
>> precise):
>>
>>> fix-statistics-reporting-in-logical-replication-work.patch
>>
>> logical replication is still often failing (as expected, I suppose; it
>> seems because of "inital snapshot too large") but indeed I do not see
>
> Yes that's different thing that we've been discussing a bit in snapbuild
> woes thread.
>
>> the 'TRAP: FailedAssertion in pgstat.c' anymore.
>>
>> (If there is any other configuration of patches worth testing please let
>> me know)
>>
>
> Thanks, so the patch works.
>

I think that we should commit the local transaction that did initial
data copy, and then report stat as well. Currently table sync worker
doesn't commit the local transaction in LogicalRepSyncTableStart
(maybe until apply commit record?) if its status is changed to
SUBREL_STATE_CATCHUP. That's why the table sync worker issues
assertion failure.

Regards,

--
Masahiko Sawada
NIPPON TELEGRAPH AND TELEPHONE CORPORATION
NTT Open Source Software Center


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Logical replication - TRAP: FailedAssertion in pgstat.c

[Petr Jelinek]

On 03/05/17 13:23, Erik Rijkers wrote:
> On 2017-05-03 08:17, Petr Jelinek wrote:
>> On 02/05/17 20:43, Robert Haas wrote:
>>> On Thu, Apr 20, 2017 at 2:58 PM, Peter Eisentraut
> 
 code path that calls CommitTransactionCommand() should have one, no?
>>>
>>> Is there anything left to be committed here?
>>>
>>
>> Afaics the fix was not committed. Peter wanted more comprehensive fix
>> which didn't happen. I think something like attached should do the job.
> 
> I'm running my pgbench-over-logical-replication test in chunk of 15
> minutes, wth different pgbench -c (num clients) and -s (scale) values.
> 
> With this patch (and nothing else)  on top of master (8f8b9be51fd7 to be
> precise):
> 
>> fix-statistics-reporting-in-logical-replication-work.patch
> 
> logical replication is still often failing (as expected, I suppose; it
> seems because of "inital snapshot too large") but indeed I do not see

Yes that's different thing that we've been discussing a bit in snapbuild
woes thread.

> the 'TRAP: FailedAssertion in pgstat.c' anymore.
> 
> (If there is any other configuration of patches worth testing please let
> me know)
> 

Thanks, so the patch works.

-- 
  Petr Jelinek  http://www.2ndQuadrant.com/
  PostgreSQL Development, 24x7 Support, Training & Services


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Logical replication - TRAP: FailedAssertion in pgstat.c

[Noah Misch]

On Wed, May 03, 2017 at 01:23:19PM +0200, Erik Rijkers wrote:
> On 2017-05-03 08:17, Petr Jelinek wrote:
> >On 02/05/17 20:43, Robert Haas wrote:
> >>On Thu, Apr 20, 2017 at 2:58 PM, Peter Eisentraut
> 
> >>>code path that calls CommitTransactionCommand() should have one, no?
> >>
> >>Is there anything left to be committed here?
> >>
> >
> >Afaics the fix was not committed. Peter wanted more comprehensive fix
> >which didn't happen. I think something like attached should do the job.
> 
> I'm running my pgbench-over-logical-replication test in chunk of 15 minutes,
> wth different pgbench -c (num clients) and -s (scale) values.
> 
> With this patch (and nothing else)  on top of master (8f8b9be51fd7 to be
> precise):
> 
> >fix-statistics-reporting-in-logical-replication-work.patch
> 
> logical replication is still often failing (as expected, I suppose; it seems
> because of "inital snapshot too large") but indeed I do not see the 'TRAP:
> FailedAssertion in pgstat.c' anymore.
> 
> (If there is any other configuration of patches worth testing please let me
> know)

[Action required within three days.  This is a generic notification.]

The above-described topic is currently a PostgreSQL 10 open item.  Peter,
since you committed the patch believed to have created it, you own this open
item.  If some other commit is more relevant or if this does not belong as a
v10 open item, please let us know.  Otherwise, please observe the policy on
open item ownership[1] and send a status update within three calendar days of
this message.  Include a date for your subsequent status update.  Testers may
discover new open items at any time, and I want to plan to get them all fixed
well in advance of shipping v10.  Consequently, I will appreciate your efforts
toward speedy resolution.  Thanks.

[1] 
https://www.postgresql.org/message-id/20170404140717.GA2675809%40tornado.leadboat.com


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Logical replication - TRAP: FailedAssertion in pgstat.c

[Erik Rijkers]

On 2017-05-03 08:17, Petr Jelinek wrote:

On 02/05/17 20:43, Robert Haas wrote:

On Thu, Apr 20, 2017 at 2:58 PM, Peter Eisentraut



code path that calls CommitTransactionCommand() should have one, no?


Is there anything left to be committed here?



Afaics the fix was not committed. Peter wanted more comprehensive fix
which didn't happen. I think something like attached should do the job.


I'm running my pgbench-over-logical-replication test in chunk of 15 
minutes, wth different pgbench -c (num clients) and -s (scale) values.


With this patch (and nothing else)  on top of master (8f8b9be51fd7 to be 
precise):



fix-statistics-reporting-in-logical-replication-work.patch


logical replication is still often failing (as expected, I suppose; it 
seems because of "inital snapshot too large") but indeed I do not see 
the 'TRAP: FailedAssertion in pgstat.c' anymore.


(If there is any other configuration of patches worth testing please let 
me know)


thanks

Erik Rijkers



--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Logical replication - TRAP: FailedAssertion in pgstat.c

[Petr Jelinek]

On 02/05/17 20:43, Robert Haas wrote:
> On Thu, Apr 20, 2017 at 2:58 PM, Peter Eisentraut
>  wrote:
>> On 4/16/17 16:11, Petr Jelinek wrote:
>>> Yeah it is, it needs to be fenced to happen only after commit, which is
>>> not guaranteed at the point of code, we probably need to put the
>>> pgstat_report_stat() inside the if above after the
>>> CommitTransactionCommand() (that will make it report stats for changes
>>> apply did to pg_subscription_rel after next transaction though)
>>
>> I think to avoid the latter, we should add more pgstat_report_stat()
>> calls, such as in process_syncing_tables_for_apply().  Basically every
>> code path that calls CommitTransactionCommand() should have one, no?
> 
> Is there anything left to be committed here?
> 

Afaics the fix was not committed. Peter wanted more comprehensive fix
which didn't happen. I think something like attached should do the job.

-- 
  Petr Jelinek  http://www.2ndQuadrant.com/
  PostgreSQL Development, 24x7 Support, Training & Services


fix-statistics-reporting-in-logical-replication-work.patch
Description: binary/octet-stream

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Logical replication - TRAP: FailedAssertion in pgstat.c

[Robert Haas]

On Thu, Apr 20, 2017 at 2:58 PM, Peter Eisentraut
 wrote:
> On 4/16/17 16:11, Petr Jelinek wrote:
>> Yeah it is, it needs to be fenced to happen only after commit, which is
>> not guaranteed at the point of code, we probably need to put the
>> pgstat_report_stat() inside the if above after the
>> CommitTransactionCommand() (that will make it report stats for changes
>> apply did to pg_subscription_rel after next transaction though)
>
> I think to avoid the latter, we should add more pgstat_report_stat()
> calls, such as in process_syncing_tables_for_apply().  Basically every
> code path that calls CommitTransactionCommand() should have one, no?

Is there anything left to be committed here?

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Logical replication - TRAP: FailedAssertion in pgstat.c

[Peter Eisentraut]

On 4/16/17 16:11, Petr Jelinek wrote:
> Yeah it is, it needs to be fenced to happen only after commit, which is
> not guaranteed at the point of code, we probably need to put the
> pgstat_report_stat() inside the if above after the
> CommitTransactionCommand() (that will make it report stats for changes
> apply did to pg_subscription_rel after next transaction though)

I think to avoid the latter, we should add more pgstat_report_stat()
calls, such as in process_syncing_tables_for_apply().  Basically every
code path that calls CommitTransactionCommand() should have one, no?

-- 
Peter Eisentraut  http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Logical replication - TRAP: FailedAssertion in pgstat.c

[Erik Rijkers]

On 2017-04-17 15:59, Stas Kelvich wrote:

On 17 Apr 2017, at 10:30, Erik Rijkers  wrote:

On 2017-04-16 20:41, Andres Freund wrote:

On 2017-04-16 10:46:21 +0200, Erik Rijkers wrote:

On 2017-04-15 04:47, Erik Rijkers wrote:
>
> 0001-Reserve-global-xmin-for-create-slot-snasphot-export.patch +
> 0002-Don-t-use-on-disk-snapshots-for-snapshot-export-in-l.patch+
> 0003-Prevent-snapshot-builder-xmin-from-going-backwards.patch  +
> 0004-Fix-xl_running_xacts-usage-in-snapshot-builder.patch  +
> 0005-Skip-unnecessary-snapshot-builds.patch
I am now using these newer patches:
https://www.postgresql.org/message-id/30242bc6-eca4-b7bb-670e-8d0458753a8c%402ndquadrant.com
> It builds fine, but when I run the old pbench-over-logical-replication
> test I get:
>
> TRAP: FailedAssertion("!(entry->trans == ((void *)0))", File:
> "pgstat.c", Line: 828)
To get that error:

I presume this is the fault of
http://git.postgresql.org/gitweb/?p=postgresql.git;a=commit;h=139eb9673cb84c76f493af7e68301ae204199746
if you git revert that individual commit, do things work again?


Yes, compiled from 67c2def11d4 with the above 4 patches, it runs 
flawlessly again. (flawlessly= a few hours without any error)




I’ve reproduced failure, this happens under tablesync worker and 
putting

pgstat_report_stat() under the previous condition block should help.

However for me it took about an hour of running this script to catch
original assert.

Can you check with that patch applied?



Your patch on top of the 5 patches above seem to solve the matter too: 
no problems after running for 2 hours (previously it failed within half 
a minute).




Erik Rijkers




--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Logical replication - TRAP: FailedAssertion in pgstat.c

[Stas Kelvich]

> On 17 Apr 2017, at 10:30, Erik Rijkers  wrote:
> 
> On 2017-04-16 20:41, Andres Freund wrote:
>> On 2017-04-16 10:46:21 +0200, Erik Rijkers wrote:
>>> On 2017-04-15 04:47, Erik Rijkers wrote:
>>> >
>>> > 0001-Reserve-global-xmin-for-create-slot-snasphot-export.patch +
>>> > 0002-Don-t-use-on-disk-snapshots-for-snapshot-export-in-l.patch+
>>> > 0003-Prevent-snapshot-builder-xmin-from-going-backwards.patch  +
>>> > 0004-Fix-xl_running_xacts-usage-in-snapshot-builder.patch  +
>>> > 0005-Skip-unnecessary-snapshot-builds.patch
>>> I am now using these newer patches:
>>> https://www.postgresql.org/message-id/30242bc6-eca4-b7bb-670e-8d0458753a8c%402ndquadrant.com
>>> > It builds fine, but when I run the old pbench-over-logical-replication
>>> > test I get:
>>> >
>>> > TRAP: FailedAssertion("!(entry->trans == ((void *)0))", File:
>>> > "pgstat.c", Line: 828)
>>> To get that error:
>> I presume this is the fault of
>> http://git.postgresql.org/gitweb/?p=postgresql.git;a=commit;h=139eb9673cb84c76f493af7e68301ae204199746
>> if you git revert that individual commit, do things work again?
> 
> Yes, compiled from 67c2def11d4 with the above 4 patches, it runs flawlessly 
> again. (flawlessly= a few hours without any error)
> 

I’ve reproduced failure, this happens under tablesync worker and putting
pgstat_report_stat() under the previous condition block should help.

However for me it took about an hour of running this script to catch original 
assert.

Can you check with that patch applied?




logical_worker.patch
Description: Binary data

Stas Kelvich
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company



-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Logical replication - TRAP: FailedAssertion in pgstat.c

[Erik Rijkers]

On 2017-04-16 20:41, Andres Freund wrote:

On 2017-04-16 10:46:21 +0200, Erik Rijkers wrote:

On 2017-04-15 04:47, Erik Rijkers wrote:
>
> 0001-Reserve-global-xmin-for-create-slot-snasphot-export.patch +
> 0002-Don-t-use-on-disk-snapshots-for-snapshot-export-in-l.patch+
> 0003-Prevent-snapshot-builder-xmin-from-going-backwards.patch  +
> 0004-Fix-xl_running_xacts-usage-in-snapshot-builder.patch  +
> 0005-Skip-unnecessary-snapshot-builds.patch

I am now using these newer patches:
https://www.postgresql.org/message-id/30242bc6-eca4-b7bb-670e-8d0458753a8c%402ndquadrant.com

> It builds fine, but when I run the old pbench-over-logical-replication
> test I get:
>
> TRAP: FailedAssertion("!(entry->trans == ((void *)0))", File:
> "pgstat.c", Line: 828)


To get that error:


I presume this is the fault of
http://git.postgresql.org/gitweb/?p=postgresql.git;a=commit;h=139eb9673cb84c76f493af7e68301ae204199746
if you git revert that individual commit, do things work again?



Yes, compiled from 67c2def11d4 with the above 4 patches, it runs 
flawlessly again. (flawlessly= a few hours without any error)




--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Logical replication - TRAP: FailedAssertion in pgstat.c

[Petr Jelinek]

On 16/04/17 20:41, Andres Freund wrote:
> On 2017-04-16 10:46:21 +0200, Erik Rijkers wrote:
>> On 2017-04-15 04:47, Erik Rijkers wrote:
>>>
>>> 0001-Reserve-global-xmin-for-create-slot-snasphot-export.patch +
>>> 0002-Don-t-use-on-disk-snapshots-for-snapshot-export-in-l.patch+
>>> 0003-Prevent-snapshot-builder-xmin-from-going-backwards.patch  +
>>> 0004-Fix-xl_running_xacts-usage-in-snapshot-builder.patch  +
>>> 0005-Skip-unnecessary-snapshot-builds.patch
>>
>> I am now using these newer patches:
>> https://www.postgresql.org/message-id/30242bc6-eca4-b7bb-670e-8d0458753a8c%402ndquadrant.com
>>
>>> It builds fine, but when I run the old pbench-over-logical-replication
>>> test I get:
>>>
>>> TRAP: FailedAssertion("!(entry->trans == ((void *)0))", File:
>>> "pgstat.c", Line: 828)
>>
>>
>> To get that error:
> 
> I presume this is the fault of
> http://git.postgresql.org/gitweb/?p=postgresql.git;a=commit;h=139eb9673cb84c76f493af7e68301ae204199746
> if you git revert that individual commit, do things work again?
> 
> - Andres
> 

Yeah it is, it needs to be fenced to happen only after commit, which is
not guaranteed at the point of code, we probably need to put the
pgstat_report_stat() inside the if above after the
CommitTransactionCommand() (that will make it report stats for changes
apply did to pg_subscription_rel after next transaction though) or put
it into it's own if that checks if tx was indeed committed (which is bit
ugly).

-- 
  Petr Jelinek  http://www.2ndQuadrant.com/
  PostgreSQL Development, 24x7 Support, Training & Services


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Logical replication - TRAP: FailedAssertion in pgstat.c

[Andres Freund]

On 2017-04-16 10:46:21 +0200, Erik Rijkers wrote:
> On 2017-04-15 04:47, Erik Rijkers wrote:
> > 
> > 0001-Reserve-global-xmin-for-create-slot-snasphot-export.patch +
> > 0002-Don-t-use-on-disk-snapshots-for-snapshot-export-in-l.patch+
> > 0003-Prevent-snapshot-builder-xmin-from-going-backwards.patch  +
> > 0004-Fix-xl_running_xacts-usage-in-snapshot-builder.patch  +
> > 0005-Skip-unnecessary-snapshot-builds.patch
> 
> I am now using these newer patches:
> https://www.postgresql.org/message-id/30242bc6-eca4-b7bb-670e-8d0458753a8c%402ndquadrant.com
> 
> > It builds fine, but when I run the old pbench-over-logical-replication
> > test I get:
> > 
> > TRAP: FailedAssertion("!(entry->trans == ((void *)0))", File:
> > "pgstat.c", Line: 828)
> 
> 
> To get that error:

I presume this is the fault of
http://git.postgresql.org/gitweb/?p=postgresql.git;a=commit;h=139eb9673cb84c76f493af7e68301ae204199746
if you git revert that individual commit, do things work again?

- Andres


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Logical replication - TRAP: FailedAssertion in pgstat.c

[Erik Rijkers]

On 2017-04-15 04:47, Erik Rijkers wrote:


0001-Reserve-global-xmin-for-create-slot-snasphot-export.patch +
0002-Don-t-use-on-disk-snapshots-for-snapshot-export-in-l.patch+
0003-Prevent-snapshot-builder-xmin-from-going-backwards.patch  +
0004-Fix-xl_running_xacts-usage-in-snapshot-builder.patch  +
0005-Skip-unnecessary-snapshot-builds.patch


I am now using these newer patches:
https://www.postgresql.org/message-id/30242bc6-eca4-b7bb-670e-8d0458753a8c%402ndquadrant.com


It builds fine, but when I run the old pbench-over-logical-replication
test I get:

TRAP: FailedAssertion("!(entry->trans == ((void *)0))", File: 
"pgstat.c", Line: 828)



To get that error:

--
#!/bin/sh

port1=6972 port2=6973 scale=25 clients=16 duration=60

   echo "drop table if exists pgbench_accounts;
 drop table if exists pgbench_branches;
 drop table if exists pgbench_tellers;
 drop table if exists pgbench_history;" | psql -qXp $port1 \
&& echo "drop table if exists pgbench_accounts;
 drop table if exists pgbench_branches;
 drop table if exists pgbench_tellers;
 drop table if exists pgbench_history;" | psql -qXp $port2 \
&& pgbench -p $port1 -qis ${scale//_/} && echo "
alter table pgbench_history add column hid serial primary key;
" | psql -q1Xp $port1  \
  && pg_dump -F c -p $port1   \
   --exclude-table-data=pgbench_history  \
   --exclude-table-data=pgbench_accounts \
   --exclude-table-data=pgbench_branches \
   --exclude-table-data=pgbench_tellers  \
   -t pgbench_history  \
   -t pgbench_accounts \
   -t pgbench_branches \
   -t pgbench_tellers  \
  | pg_restore -1 -p $port2 -d testdb

appname=pgbench_derail
echo "create publication pub1 for all tables;" | psql -p $port1 -aqtAX
echo "create subscription sub1 connection 'port=${port1} 
application_name=${appname}' publication pub1 with (disabled);

alter subscription sub1 enable;
" | psql -p $port2 -aqtAX

echo "-- pgbench -p $port1 -c $clients -T $duration -n   -- scale $scale 
"

 pgbench -p $port1 -c $clients -T $duration -n

--


Erik Rijkers


--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

[HACKERS] Logical replication - TRAP: FailedAssertion in pgstat.c

[Erik Rijkers]

Testing logical replication, with the following patches on top of 
yesterday's master:


0001-Reserve-global-xmin-for-create-slot-snasphot-export.patch +
0002-Don-t-use-on-disk-snapshots-for-snapshot-export-in-l.patch+
0003-Prevent-snapshot-builder-xmin-from-going-backwards.patch  +
0004-Fix-xl_running_xacts-usage-in-snapshot-builder.patch  +
0005-Skip-unnecessary-snapshot-builds.patch

Is applying that patch set is still correct?

It builds fine, but when I run the old pbench-over-logical-replication 
test I get:


TRAP: FailedAssertion("!(entry->trans == ((void *)0))", File: 
"pgstat.c", Line: 828)


reliably (often within a minute).


The test itself does not fail, at least not that I saw (but I only ran a 
few).



thanks,


Erik Rijkers




--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers