Re: [HACKERS] [COMMITTERS] pgsql: Replication lag tracking for walsenders

Mark Dilger writes: >> On Apr 22, 2017, at 11:40 AM, Tom Lane wrote: >> In short then, I propose the attached patch to make these cases fail >> more reliably. We might extend this later to allow the old behaviors >> to be explicitly opted-into, but

Re: [HACKERS] [COMMITTERS] pgsql: Replication lag tracking for walsenders

> On Apr 22, 2017, at 11:40 AM, Tom Lane wrote: > > I wrote: >> Whoa. This just turned into a much larger can of worms than I expected. >> How can it be that processes are getting assertion crashes and yet the >> test framework reports success anyway? That's impossibly >>

[HACKERS] Statistics "dependency"

In looking at the new multi-column statistics "dependency" option in Postgres 10, I am quite confused by the term "dependency". Wouldn't "correlation" be clearer and less confusing as "column dependency" already means something else. -- Bruce Momjian

Re: [HACKERS] A note about debugging TAP failures

On Sun, Apr 23, 2017 at 7:48 AM, Daniel Gustafsson wrote: > Skipping the tempdir and instead using ${testname}_data_${name} without a > random suffix, we can achieve this with something along the lines of the > attached PoC. It works as now (retain of failure, remove on success

Re: [HACKERS] [COMMITTERS] pgsql: Replication lag tracking for walsenders

On Sun, Apr 23, 2017 at 3:41 AM, Tom Lane wrote: > Thomas Munro writes: >> The assertion fails reliably for me, because standby2's reported write >> LSN jumps backwards after the timeline changes: for example I see >> 302 then 3028470 then

Re: [HACKERS] logical replication and PANIC during shutdown checkpoint in publisher

On 21/04/17 06:11, Michael Paquier wrote: > On Fri, Apr 21, 2017 at 12:29 AM, Peter Eisentraut > wrote: >> On 4/20/17 07:52, Petr Jelinek wrote: >>> On 20/04/17 05:57, Michael Paquier wrote: 2nd thoughts here... Ah now I see your point. True that there is no

Re: [HACKERS] PostgresNode::append_conf considered dangerous

On 04/22/2017 04:19 PM, Tom Lane wrote: > > I think instead we should change append_conf to append a newline, and > simplify the callers that could thereby be simplified. I don't see > any callers that would be broken by that; and we do not have any config > files in which extra blank lines are

Re: [HACKERS] StandbyRecoverPreparedTransactions recovers subtrans links incorrectly

On 2017-04-22 19:55:18 -0400, Tom Lane wrote: > Now that we've got consistent failure reports about the 009_twophase.pl > recovery test, I set out to find out why it's failing. It looks to me > like the reason is that this (twophase.c:2145): > > SubTransSetParent(xid, subxid,

[HACKERS] StandbyRecoverPreparedTransactions recovers subtrans links incorrectly

Now that we've got consistent failure reports about the 009_twophase.pl recovery test, I set out to find out why it's failing. It looks to me like the reason is that this (twophase.c:2145): SubTransSetParent(xid, subxid, overwriteOK); ought to be this:

[HACKERS] Remove dead interfaces added by mistake in 7c4f52409

Hi, Seems like couple of declarations for couple of functions we never actually implemented and are not used got past review of logical replication support for initial copy path (commit 7c4f52409). Attached patch gets rid of them. -- Petr Jelinek http://www.2ndQuadrant.com/

[HACKERS] Time based lag tracking for logical replication

Hi, The time based lag tracking commit [1] added interface for logging progress of replication so that we can report lag as time interval instead of just bytes. But the patch didn't contain patch for the builtin logical replication. So I wrote something that implements this. I didn't like all

Re: [HACKERS] A note about debugging TAP failures

> On 23 Apr 2017, at 00:06, Andres Freund wrote: > > On 2017-04-22 23:59:11 +0200, Daniel Gustafsson wrote: >> Since we have the name of the running testscript, can’t we just add that to >> the >> tempdir to make the name more descriptive? With the attached patch I get >>

Re: [HACKERS] A note about debugging TAP failures

On 2017-04-22 23:59:11 +0200, Daniel Gustafsson wrote: > Since we have the name of the running testscript, can’t we just add that to > the > tempdir to make the name more descriptive? With the attached patch I get > tmp_check/001_pgbench_data_main_ItEm when running tests in src/bin/pgbench for >

Re: [HACKERS] A note about debugging TAP failures

> On 22 Apr 2017, at 23:25, Tom Lane wrote: > > Andres Freund writes: >> On 2017-04-22 16:22:59 -0400, Tom Lane wrote: >>> In the particular case I was interested in here, the test script thought >>> everything was successful :-(. I'm working on fixing

Re: [HACKERS] A note about debugging TAP failures

On 2017-04-22 17:25:49 -0400, Tom Lane wrote: > Andres Freund writes: > > On 2017-04-22 16:22:59 -0400, Tom Lane wrote: > >> In the particular case I was interested in here, the test script thought > >> everything was successful :-(. I'm working on fixing that little problem,

Re: [HACKERS] A note about debugging TAP failures

Andres Freund writes: > On 2017-04-22 16:22:59 -0400, Tom Lane wrote: >> In the particular case I was interested in here, the test script thought >> everything was successful :-(. I'm working on fixing that little problem, >> but I do not believe that the TAP scripts are so

Re: [HACKERS] A note about debugging TAP failures

On 2017-04-22 16:22:59 -0400, Tom Lane wrote: > Andres Freund writes: > > On 2017-04-22 13:51:30 -0400, Tom Lane wrote: > >> I think we need to fix TestLib and/or PostgresNode so that there's a way > >> to make TAP tests not auto-clean their data directories at end of run, >

[HACKERS] recovery tests vs windows

After we got over the Test::More version issue, the recovery tests proceeded to fail fairly spectacularly in a test run on jacana. Test 6 fails because there is a CR in the returned stdout from psql. I'm inclined to adjust that in PostgresNode::safe_psql so we don't have to do it all over the

Re: [HACKERS] A note about debugging TAP failures

Andres Freund writes: > On 2017-04-22 13:51:30 -0400, Tom Lane wrote: >> I think we need to fix TestLib and/or PostgresNode so that there's a way >> to make TAP tests not auto-clean their data directories at end of run, >> without having to resort to editing the script like

[HACKERS] PostgresNode::append_conf considered dangerous

Okay, so append_conf is perfectly straightforward about what it does: A newline is NOT automatically appended to the string. However, I count at least eight places that are being lazy, like this: # Change a setting and restart $node->append_conf('postgresql.conf', 'hot_standby =

Re: [HACKERS] pg_dump emits ALTER TABLE ONLY partitioned_table

On Thu, Apr 20, 2017 at 09:53:28PM -0400, Stephen Frost wrote: > * Noah Misch (n...@leadboat.com) wrote: > > On Mon, Apr 17, 2017 at 03:41:25PM -0400, Stephen Frost wrote: > > > I've put up a new patch for review on the thread and plan to commit > > > that tomorrow, assuming there isn't anything

Re: [HACKERS] tablesync patch broke the assumption that logical rep depends on?

On 21/04/17 16:31, Petr Jelinek wrote: > On 21/04/17 16:23, Peter Eisentraut wrote: >> On 4/21/17 10:11, Petr Jelinek wrote: >>> On 21/04/17 16:09, Peter Eisentraut wrote: On 4/20/17 14:29, Petr Jelinek wrote: > + /* Find unused worker slot. */ > + if (!w->in_use)

Re: [HACKERS] A note about debugging TAP failures

Hi, On 2017-04-22 13:51:30 -0400, Tom Lane wrote: > I think we need to fix TestLib and/or PostgresNode so that there's a way > to make TAP tests not auto-clean their data directories at end of run, > without having to resort to editing the script like this. I think leaving the directory around

Re: [HACKERS] A note about debugging TAP failures

I think we need to fix TestLib and/or PostgresNode so that there's a way to make TAP tests not auto-clean their data directories at end of run, without having to resort to editing the script like this. It'd also be helpful if the data directory pathname included the test script's name. Rely

Re: [HACKERS] valgrind error in subscription code

Hi, On 2017-04-22 21:08:18 +0200, Petr Jelinek wrote: > Thanks, here is patch to fix that - I also removed the individual > settings of everything to NULL/0/InvalidOid etc and just replaced it all > with memset. Cool. > diff --git a/src/backend/replication/logical/relation.c >

Re: [HACKERS] valgrind error in subscription code

On 22/04/17 20:31, Andres Freund wrote: > Hi, > > I enabled skink / the valgrind animal to run the tap tests too (hugely > increasing the test time :(), and that paid of: > https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=skink=2017-04-22%2004%3A52%3A13 > > ==606== VALGRINDERROR-BEGIN >

Re: [HACKERS] [COMMITTERS] pgsql: Replication lag tracking for walsenders

I wrote: > Whoa. This just turned into a much larger can of worms than I expected. > How can it be that processes are getting assertion crashes and yet the > test framework reports success anyway? That's impossibly > broken/unacceptable. I poked into this on my laptop, where I'm able to

[HACKERS] valgrind error in subscription code

Hi, I enabled skink / the valgrind animal to run the tap tests too (hugely increasing the test time :(), and that paid of: https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=skink=2017-04-22%2004%3A52%3A13 ==606== VALGRINDERROR-BEGIN ==606== Conditional jump or move depends on uninitialised

Re: [HACKERS] Patch - Tcl 8.6 version support for PostgreSQL

Tcl8.6 is already supported in PostgreSQL. It was just missing the required changes for Windows build system. Can someone please review and commit the attached patches? - Mkvcbuild_Tcl86_94-92.patch (This applies to 9.4, 9.3 and 9.2) - Mkvcbuild_Tcl86_95-master.patch (This applies to 9.5, 9.6 and

[HACKERS] A note about debugging TAP failures

I've whined before about how developer-unfriendly the TAP test infrastructure is. One concrete problem is that if there is a failure, there is absolutely no way to get any information beyond what is in the logs, because the test installation's data directory is unconditionally blown away at run

Re: [HACKERS] Small patch for pg_basebackup argument parsing

On Saturday, April 22, 2017 11:31:58 PM CEST Michael Paquier wrote: > On Sat, Apr 22, 2017 at 11:12 PM, Pierre Ducroquet wrote: > > Following your advice, I went through the source tree and cleaned up most > > instances of that pattern. > > I have attached the corresponding

Re: [HACKERS] [COMMITTERS] pgsql: Replication lag tracking for walsenders

I wrote: > So 6 of 15 critters are getting the walsender.c assertion, > and those six plus six more are seeing the subtrans.c one, > and three are seeing neither one. There's probably a pattern > to that, don't know what it is. Ah, got it: skink *is* seeing the subtrans.c assertion, but not the

Re: [HACKERS] [COMMITTERS] pgsql: Replication lag tracking for walsenders

I wrote: > Taking a quick census of other buildfarm machines that are known to be > running the recovery test, it appears that most (not all) are seeing > one or both traps. But the test is reporting success anyway, everywhere > except on Noah's 32-bit AIX critters. Or, to be a bit more

Re: [HACKERS] [COMMITTERS] pgsql: Replication lag tracking for walsenders

I wrote: > Whoa. This just turned into a much larger can of worms than I expected. > How can it be that processes are getting assertion crashes and yet the > test framework reports success anyway? That's impossibly > broken/unacceptable. Taking a quick census of other buildfarm machines that

Re: [HACKERS] walsender & parallelism

On 21/04/17 04:37, Petr Jelinek wrote: > On 21/04/17 04:32, Craig Ringer wrote: >> On 21 April 2017 at 10:20, Petr Jelinek wrote: >>> On 21/04/17 03:40, Andres Freund wrote: Since [1] walsender (not receiver as commit message says) can execute SQL

Re: [HACKERS] Interval for launching the table sync worker

On Fri, Apr 21, 2017 at 11:19 PM, Masahiko Sawada wrote: > On Fri, Apr 21, 2017 at 5:33 PM, Kyotaro HORIGUCHI > wrote: >> Hello, >> >> At Thu, 20 Apr 2017 13:21:14 +0900, Masahiko Sawada >> wrote in

Re: [HACKERS] [COMMITTERS] pgsql: Replication lag tracking for walsenders

Thomas Munro writes: > The assertion fails reliably for me, because standby2's reported write > LSN jumps backwards after the timeline changes: for example I see > 302 then 3028470 then 302 followed by a normal progression. > Surprisingly,

Re: [HACKERS] Small patch for pg_basebackup argument parsing

On Sat, Apr 22, 2017 at 11:12 PM, Pierre Ducroquet wrote: > Following your advice, I went through the source tree and cleaned up most > instances of that pattern. > I have attached the corresponding patch to this mail. > If you think this patch is indeed interesting, what

Re: [HACKERS] Old versions of Test::More

On 04/21/2017 09:22 PM, Craig Ringer wrote: > > > On 22 Apr. 2017 4:23 am, "Tom Lane" > wrote: > > Peter Eisentraut > writes: > > On 4/21/17 14:49, Andrew

Re: [HACKERS] multithreading in Batch/pipelining mode for libpq

On 21 April 2017 at 21:31, Ilya Roublev wrote: > What I need is to make a huge amount of inserts This may be a silly question but I assume you've already considered using server-side COPY? That's the most efficient way to load a lot of data currently. -- greg -- Sent

Re: [HACKERS] Small patch for pg_basebackup argument parsing

On Friday, April 14, 2017 8:59:03 AM CEST Robert Haas wrote: > On Fri, Apr 14, 2017 at 2:32 AM, Pierre Ducroquet wrote: > > On Friday, April 14, 2017 8:44:37 AM CEST Michael Paquier wrote: > >> On Fri, Apr 14, 2017 at 6:32 AM, Pierre Ducroquet > > > >

Re: [HACKERS] [COMMITTERS] pgsql: Replication lag tracking for walsenders

On Sat, Apr 22, 2017 at 9:13 PM, Simon Riggs wrote: > On 22 April 2017 at 06:45, Thomas Munro wrote: > >> Thanks. I'm away from my computer right now but will investigate this >> and send a fix later today. > > Thanks. I'll review later

Re: [HACKERS] WITH clause in CREATE STATISTICS

On 22 April 2017 at 21:30, Simon Riggs wrote: > CREATE STATISTICS s1 USING (dependencies, ndistinct) ON (a, b) FROM t1 > WHERE partial-stuff; +1 Seems much more CREATE INDEX like, and that's pretty good given that most of the complaints so far were about it bearing enough

Re: [HACKERS] WITH clause in CREATE STATISTICS

2017-04-22 11:30 GMT+02:00 Simon Riggs : > On 21 April 2017 at 01:21, Tomas Vondra > wrote: > > On 04/21/2017 12:13 AM, Tom Lane wrote: > >> > >> Alvaro Herrera writes: > >>> > >>> Simon just pointed out that having

Re: [HACKERS] WITH clause in CREATE STATISTICS

On 21 April 2017 at 01:21, Tomas Vondra wrote: > On 04/21/2017 12:13 AM, Tom Lane wrote: >> >> Alvaro Herrera writes: >>> >>> Simon just pointed out that having the WITH clause appear in the middle >>> of the CREATE STATISTICS command looks

Re: [HACKERS] [COMMITTERS] pgsql: Replication lag tracking for walsenders

On 22 April 2017 at 06:45, Thomas Munro wrote: > Thanks. I'm away from my computer right now but will investigate this > and send a fix later today. Thanks. I'll review later today. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL

Re: [HACKERS] WITH clause in CREATE STATISTICS

On 21 April 2017 at 01:21, Tomas Vondra wrote: > On 04/21/2017 12:13 AM, Tom Lane wrote: >> >> Alvaro Herrera writes: >>> >>> Simon just pointed out that having the WITH clause appear in the middle >>> of the CREATE STATISTICS command looks