RE: [BUG] "FailedAssertion" reported when streaming in logical replication

2021-04-28 Thread tanghy.f...@fujitsu.com
On Thursday, April 29, 2021 3:18 PM, Dilip Kumar wrote >I tried to think about how to write a test case for this scenario, but >I think it will not be possible to generate an automated test case for this. >Basically, we need 2 concurrent transactions and out of that, >we need one transaction w

Re: Replication slot stats misgivings

2021-04-28 Thread Amit Kapila
On Thu, Apr 29, 2021 at 11:14 AM Masahiko Sawada wrote: > > > > > How about doing both of the above suggestions? Alternatively, we can > > wait for both 'drop' and 'create' message to be delivered but that > > might be overkill. > > Agreed. Attached the patch doing both things. > Thanks, the patc

Re: [BUG] "FailedAssertion" reported when streaming in logical replication

2021-04-28 Thread Dilip Kumar
On Wed, Apr 28, 2021 at 1:02 PM Dilip Kumar wrote: > > On Wed, Apr 28, 2021 at 12:25 PM tanghy.f...@fujitsu.com > wrote: > > > > > I have modified the patch based on the above comments. > > > > Thanks for your patch. > > I tested again after applying your patch and the problem is fixed. > > Thank

RE: [BUG]"FailedAssertion" reported in lazy_scan_heap() when running logical replication

2021-04-28 Thread tanghy.f...@fujitsu.com
On Thursday, April 29, 2021 1:22 PM, Peter Geoghegan wrote >Is setting all_visible_according_to_vm false as below enough to avoid >the assertion failure? > >diff --git a/src/backend/access/heap/vacuumlazy.c >b/src/backend/access/heap/vacuumlazy.c >index c3fc12d76c..76c17e063e 100644 >--- a/src/ba

Re: Replication slot stats misgivings

2021-04-28 Thread Masahiko Sawada
On Thu, Apr 29, 2021 at 11:55 AM Amit Kapila wrote: > > On Thu, Apr 29, 2021 at 4:58 AM Masahiko Sawada wrote: > > > > On Thu, Apr 29, 2021 at 5:41 AM Tom Lane wrote: > > > > > > It seems that the test case added by f5fc2f5b2 is still a bit > > > unstable, even after c64dcc7fe: > > > > Hmm, I do

Re: Forget close an open relation in ReorderBufferProcessTXN()

2021-04-28 Thread Amit Kapila
On Wed, Apr 28, 2021 at 5:36 PM osumi.takami...@fujitsu.com wrote: > > On Monday, April 26, 2021 2:05 PM Amit Kapila wrote: > > On Fri, Apr 23, 2021 at 8:03 PM osumi.takami...@fujitsu.com > > wrote: > > I think we are allowed to decode the operations on user catalog tables > > because we are usi

Re: Unresolved repliaction hang and stop problem.

2021-04-28 Thread Amit Kapila
On Wed, Apr 28, 2021 at 7:36 PM Alvaro Herrera wrote: > > On 2021-Apr-28, Amit Kapila wrote: > > > On Wed, Apr 28, 2021 at 6:48 AM Alvaro Herrera > > wrote: > > > > Hmm ... On what does it depend (other than plain git conflicts, which > > > are aplenty)? On a quick look to the commit, it's clea

Re: pg_hba.conf.sample wording improvement

2021-04-28 Thread Peter Eisentraut
On 28.04.21 16:09, Alvaro Herrera wrote: Looking at it now, I wonder how well do the "hostno" options work. If I say "hostnogssenc", is an SSL-encrypted socket good? If I say "hostnossl", is a GSS-encrypted socket good? If so, how does that make sense? I think for example if you want to enfo

Re: Replication slot stats misgivings

2021-04-28 Thread Amit Kapila
On Wed, Apr 28, 2021 at 7:43 PM Masahiko Sawada wrote: > > On Wed, Apr 28, 2021 at 3:25 PM Amit Kapila wrote: > > > > > I am not sure if any of these alternatives are a good idea. What do > > you think? Do you have any other ideas for this? > > I've been considering some ideas but don't come up w

Re: pg_hba.conf.sample wording improvement

2021-04-28 Thread Peter Eisentraut
On 28.04.21 16:20, Tom Lane wrote: Another thought is to switch the phrase order: +# - "local" is a Unix-domain socket +# - "host" is a TCP/IP socket (encrypted or not) +# - "hostssl" is a TCP/IP socket that is SSL-encrypted +# - "hostnossl" is a TCP/IP socket that is not SSL-encrypted +# - "hos

Re: Patch to allow pg_filedump to support reading of pg_filenode.map

2021-04-28 Thread Justin Pryzby
This is separate from the postgresql server repo. https://git.postgresql.org/gitweb/?p=pg_filedump.git +#define RELMAPPER_FILEMAGIC 0x592717 +char magic_buffer[8]; ... + if ( (int) magic_buffer & RELMAPPER_FILEMAGIC ) { This is doing bitwise arithmetic on a pointer, which seems badly wrong.

Re: WIP: WAL prefetch (another approach)

2021-04-28 Thread Thomas Munro
On Thu, Apr 29, 2021 at 3:14 PM Andres Freund wrote: > To me it looks like a smaller version of the problem is present in < 14, > albeit only when the page header is at a record boundary. In that case > we don't validate the page header immediately, only once it's completely > read. But we do beli

Re: Replication slot stats misgivings

2021-04-28 Thread Amit Kapila
On Thu, Apr 29, 2021 at 8:50 AM Tom Lane wrote: > > Amit Kapila writes: > > This is the first test and inserts just one small record, so how it > > can lead to spill of data. Do you mean to say that may be some > > background process has written some transaction which leads to a spill > > of data

Re: [PATCH] force_parallel_mode and GUC categories

2021-04-28 Thread Justin Pryzby
On Fri, Apr 09, 2021 at 10:50:53AM +0900, Michael Paquier wrote: > - {"track_commit_timestamp", PGC_POSTMASTER, REPLICATION, > + {"track_commit_timestamp", PGC_POSTMASTER, REPLICATION_SENDING, > I can get behind this change for clarity where it gets actively used. I'm not sure what you

Re: [BUG]"FailedAssertion" reported in lazy_scan_heap() when running logical replication

2021-04-28 Thread Peter Geoghegan
On Wed, Apr 28, 2021 at 7:34 PM tanghy.f...@fujitsu.com wrote: > >TRAP: FailedAssertion("!all_visible_according_to_vm || > >prunestate.all_visible", File: "vacuumlazy.c", Line: 1347, PID: 1274675) > > BTW, in asynchronous mode, the same problem can also happen but in a low > frequency.(I tried m

Re: Use simplehash.h instead of dynahash in SMgr

2021-04-28 Thread David Rowley
On Thu, 29 Apr 2021 at 12:30, Yura Sokolov wrote: > > David Rowley писал 2021-04-29 02:51: > > Another line of thought for making it go faster would be to do > > something like get rid of the hash status field from SMgrEntry. That > > could be either coded into a single bit we'd borrow from the h

Re: Small issues with CREATE TABLE COMPRESSION

2021-04-28 Thread Justin Pryzby
On Tue, Apr 27, 2021 at 03:22:25PM +0900, Michael Paquier wrote: > Hi all, > > I have been looking at and testing the patch set for CREATE TABLE > COMPRESSION, and spotted a couple of things in parallel of some work > done by Jacob (added in CC). > > The behavior around CREATE TABLE AS and matvie

Re: Replication slot stats misgivings

2021-04-28 Thread Andres Freund
On 2021-04-28 23:20:00 -0400, Tom Lane wrote: > (At least not before the fabled stats collector rewrite, which may well > introduce some entirely new set of failure modes.) FWIW, I added a function that forces a flush there. That can be done synchronously and the underlying functionality needs to

Re: WIP: WAL prefetch (another approach)

2021-04-28 Thread Tom Lane
Andres Freund writes: > I was now able to reproduce the problem again, and I'm afraid that the > bug I hit is likely separate from Tom's. Yeah, I think so --- the symptoms seem quite distinct. My score so far today on the G4 is: 12 error-free regression test cycles on b3ee4c503 (plus one more

Re: Replication slot stats misgivings

2021-04-28 Thread Tom Lane
Amit Kapila writes: > This is the first test and inserts just one small record, so how it > can lead to spill of data. Do you mean to say that may be some > background process has written some transaction which leads to a spill > of data? autovacuum, say? > Yeah, something like this could happen

Re: WIP: WAL prefetch (another approach)

2021-04-28 Thread Andres Freund
Hi, On 2021-04-28 17:59:22 -0700, Andres Freund wrote: > I can however say that pg_waldump on the standby's pg_wal does also > fail. The failure as part of the backend is "invalid memory alloc > request size", whereas in pg_waldump I get the much more helpful: > pg_waldump: fatal: error in WAL rec

Re: Replication slot stats misgivings

2021-04-28 Thread Amit Kapila
On Thu, Apr 29, 2021 at 4:58 AM Masahiko Sawada wrote: > > On Thu, Apr 29, 2021 at 5:41 AM Tom Lane wrote: > > > > It seems that the test case added by f5fc2f5b2 is still a bit > > unstable, even after c64dcc7fe: > > Hmm, I don't see the exact cause yet but there are two possibilities: > some tra

Re: Error in libpq docs for target_session_attrs, prefer-standby mode

2021-04-28 Thread Michael Paquier
On Wed, Apr 28, 2021 at 01:32:05PM +0200, Laurenz Albe wrote: > You are right, and the patch is good. Thanks, fixed. -- Michael signature.asc Description: PGP signature

Re: [BUG]"FailedAssertion" reported in lazy_scan_heap() when running logical replication

2021-04-28 Thread Michael Paquier
On Thu, Apr 29, 2021 at 02:34:21AM +, tanghy.f...@fujitsu.com wrote: > I met an assertion failure at the publisher in lazy_scan_heap() when > synchronous running logical replication. Could someone please take a > look at it? This assertion is new as of 7136bf34. Peter? -- Michael signature.

Re: [PATCH] We install pg_regress and isolationtester but not pg_isolation_regress

2021-04-28 Thread Tom Lane
Michael Paquier writes: > On Wed, Apr 28, 2021 at 12:44:45PM +0300, Aleksander Alekseev wrote: >> I just wanted to let you know that TimescaleDB uses >> pg_isolation_regress and occasionally there are reports from some >> suffering/puzzled users/developers, e.g. [1]. Not 100% sure if it >> makes i

[BUG]"FailedAssertion" reported in lazy_scan_heap() when running logical replication

2021-04-28 Thread tanghy.f...@fujitsu.com
Hi I met an assertion failure at the publisher in lazy_scan_heap() when synchronous running logical replication. Could someone please take a look at it? Here's what I did to produce the problem. First, use './configure --enable-cassert' to build the PG. Then, I created multiple publications at

Re: [PATCH] We install pg_regress and isolationtester but not pg_isolation_regress

2021-04-28 Thread Michael Paquier
On Wed, Apr 28, 2021 at 12:44:45PM +0300, Aleksander Alekseev wrote: > I just wanted to let you know that TimescaleDB uses > pg_isolation_regress and occasionally there are reports from some > suffering/puzzled users/developers, e.g. [1]. Not 100% sure if it > makes investing the time into backpatc

Re: WIP: WAL prefetch (another approach)

2021-04-28 Thread Andres Freund
Hi, On 2021-04-28 17:59:22 -0700, Andres Freund wrote: > I can however say that pg_waldump on the standby's pg_wal does also > fail. The failure as part of the backend is "invalid memory alloc > request size", whereas in pg_waldump I get the much more helpful: > pg_waldump: fatal: error in WAL rec

Re: [HACKERS] logical decoding of two-phase transactions

2021-04-28 Thread Ajin Cherian
Modified pgbench's "tpcb-like" builtin query as below to do two-phase commits and then ran a 4 cascade replication setup. "BEGIN;\n" "UPDATE pgbench_accounts SET abalance = abalance + :delta WHERE aid = :aid;\n" "SELECT abalance FROM pgbench_accounts WHERE aid = :aid;\n"

RE: [bug?] Missed parallel safety checks, and wrong parallel safety

2021-04-28 Thread houzj.f...@fujitsu.com
> >>> I'm curious. The FmgrBuiltin struct includes the "strict" flag, so > >>> that would "lock down the value" of the strict flag, wouldn't it? > > >> It does, but that's much more directly a property of the function's C > >> code than parallel-safety is. > > > I'm not sure I agree with that, bu

Re: WIP: WAL prefetch (another approach)

2021-04-28 Thread Andres Freund
Hi, On 2021-04-28 20:24:43 -0400, Tom Lane wrote: > Andres Freund writes: > > Oh! I was about to ask how much shared buffers your primary / standby > > have. > Default configurations, so 128MB each. I thought that possibly initdb would detect less or something... I assume this is 32bit? I did

Patch to allow pg_filedump to support reading of pg_filenode.map

2021-04-28 Thread Richard Yen
Hello Hackers, I recently had to work on a case where some catalog files were corrupt and/or missing. One of the things we sought to inspect was pg_filenode.map, but there was no tooling available to do so. With the help of Álvaro H.. I've put together a patch to allow pg_filedump to do some rud

Re: Use simplehash.h instead of dynahash in SMgr

2021-04-28 Thread Yura Sokolov
David Rowley писал 2021-04-29 02:51: On Thu, 29 Apr 2021 at 00:28, Yura Sokolov wrote: The best result is with just: return (uint32)rnode->node.relNode; ie, relNode could be taken without mixing at all. relNode is unique inside single database, and almost unique among whole cluster sinc

Re: WIP: WAL prefetch (another approach)

2021-04-28 Thread Tom Lane
Andres Freund writes: > On 2021-04-28 19:24:53 -0400, Tom Lane wrote: >> IOW, we've spent over twice as many CPU cycles shipping data to the >> standby as we did in applying the WAL on the standby. > I don't really know how the time calculation works on mac. Is there a > chance it includes time s

Re: WIP: WAL prefetch (another approach)

2021-04-28 Thread Andres Freund
Hi, On 2021-04-28 19:24:53 -0400, Tom Lane wrote: > But I happened to notice the accumulated CPU time for the background > processes: > > USER PID %CPU %MEM VSZRSS TT STAT STARTED TIME COMMAND > tgl 19048 0.0 4.4 229952 92196 ?? Ss3:19PM 19:59.19 > post

Re: Use simplehash.h instead of dynahash in SMgr

2021-04-28 Thread David Rowley
On Thu, 29 Apr 2021 at 00:28, Yura Sokolov wrote: > The best result is with just: > > return (uint32)rnode->node.relNode; > > ie, relNode could be taken without mixing at all. > relNode is unique inside single database, and almost unique among whole > cluster > since it is Oid. I admit to hav

Re: Replication slot stats misgivings

2021-04-28 Thread Masahiko Sawada
On Thu, Apr 29, 2021 at 5:41 AM Tom Lane wrote: > > It seems that the test case added by f5fc2f5b2 is still a bit > unstable, even after c64dcc7fe: Hmm, I don't see the exact cause yet but there are two possibilities: some transactions were really spilled, and it showed the old stats due to losin

Re: WIP: WAL prefetch (another approach)

2021-04-28 Thread Tom Lane
Thomas Munro writes: > FWIW I also chewed through many megawatts trying to reproduce this on > a PowerPC system in 64 bit big endian mode, with an emulator. No > cigar. However, it's so slow that I didn't make it to 10 runs... Speaking of megawatts ... my G4 has now finished about ten cycles of

Re: WIP: WAL prefetch (another approach)

2021-04-28 Thread Thomas Munro
On Thu, Apr 29, 2021 at 4:45 AM Tom Lane wrote: > Andres Freund writes: > > Tom, any chance you could check if your machine repros the issue before > > these commits? > > Wilco, but it'll likely take a little while to get results ... FWIW I also chewed through many megawatts trying to reproduce

gcc 11.1.0 warnings in llvmjit_expr.c

2021-04-28 Thread Erik Rijkers
Hello, gcc 11.1.0 produces quite a litany of second thoughts when compiling llvmjit_expr.c; is there something in it? (all 'warning' or 'note') llvmjit_expr.c: In function ‘llvm_compile_expr’: llvmjit_expr.c:69:9: warning: ‘build_EvalXFuncInt’ accessing 8 bytes in a region of size 0 [-Wstr

Re: SQL/JSON: functions

2021-04-28 Thread Andrew Dunstan
On Fri, Mar 26, 2021 at 9:14 PM Nikita Glukhov wrote: > Attached 54th version of the patches rebased onto current master. > > > On 27.03.2021 01:30, Andrew Dunstan wrote: > > Specifically, patch 4 (SQL-JSON-query-functions) fails with this when > built with LLVM: > > > There is also a bug that re

Re: pg_upgrade fails to detect unsupported arrays and ranges

2021-04-28 Thread Tom Lane
Daniel Gustafsson writes: > On 28 Apr 2021, at 22:47, Tom Lane wrote: >> This wording is copied-and-pasted from the other similar tests. I agree >> that it's advocating a solution that might be overkill, but if we change >> it we should also change the existing messages. > Good point. >> I d

Re: pg_upgrade fails to detect unsupported arrays and ranges

2021-04-28 Thread Daniel Gustafsson
> On 28 Apr 2021, at 22:47, Tom Lane wrote: > > Daniel Gustafsson writes: >>> On 28 Apr 2021, at 17:09, Tom Lane wrote: >>> + pg_fatal("Your installation contains system-defined composite >>> type(s) in user tables.\n" >>> +"These type OIDs are not stable

Re: pg_upgrade fails to detect unsupported arrays and ranges

2021-04-28 Thread Tom Lane
Daniel Gustafsson writes: >> On 28 Apr 2021, at 17:09, Tom Lane wrote: >> +pg_fatal("Your installation contains system-defined composite >> type(s) in user tables.\n" >> + "These type OIDs are not stable across >> PostgreSQL versions,\n" >> +

Re: Replication slot stats misgivings

2021-04-28 Thread Tom Lane
It seems that the test case added by f5fc2f5b2 is still a bit unstable, even after c64dcc7fe: https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=peripatus&dt=2021-04-23%2006%3A20%3A12 https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=peripatus&dt=2021-04-24%2018%3A20%3A10 https://buil

Re: pg_upgrade fails to detect unsupported arrays and ranges

2021-04-28 Thread Daniel Gustafsson
> On 28 Apr 2021, at 17:09, Tom Lane wrote: > > [ blast-from-the-past department ] > > I wrote: >> Daniel Gustafsson writes: >>> I can see the appeal of >>> including it before the wrap, even though I personally would've held off. > >> Nah, I'm not gonna risk it at this stage. I concur with y

Re: Better sanity checking for message length words

2021-04-28 Thread Tom Lane
Aleksander Alekseev writes: > Sorry, my bad. It was about lines separating on different platforms. The > patch is fine and passes installcheck-world on MacOS. Pushed, thanks for looking at it! regards, tom lane

Re: ALTER TABLE .. DETACH PARTITION CONCURRENTLY

2021-04-28 Thread Álvaro Herrera
Pushed that now, with a one-line addition to the docs that only one partition can be marked detached. -- Álvaro Herrera39°49'30"S 73°17'W "That sort of implies that there are Emacs keystrokes which aren't obscure. I've been using it daily for 2 years now and have yet t

Re: strange case of "if ((a & b))"

2021-04-28 Thread Tom Lane
Justin Pryzby writes: > These look strange to me - the inner parens don't do anything. > I wouldn't write it with 2x parens for the same reason I wouldn't write it > with > 8x parens. Agreed, but shouldn't we just drop the excess parens rather than doubling down on useless notation?

strange case of "if ((a & b))"

2021-04-28 Thread Justin Pryzby
These look strange to me - the inner parens don't do anything. I wouldn't write it with 2x parens for the same reason I wouldn't write it with 8x parens. diff --git a/contrib/amcheck/verify_heapam.c b/contrib/amcheck/verify_heapam.c index 9f159eb3db..3bbc13c443 100644 --- a/contrib/amcheck/verify_

Re: WIP: WAL prefetch (another approach)

2021-04-28 Thread Tom Lane
Andres Freund writes: > Tom, any chance you could check if your machine repros the issue before > these commits? Wilco, but it'll likely take a little while to get results ... regards, tom lane

Re: WIP: WAL prefetch (another approach)

2021-04-28 Thread Andres Freund
Hi, On 2021-04-22 13:59:58 +1200, Thomas Munro wrote: > On Thu, Apr 22, 2021 at 1:21 PM Tom Lane wrote: > > I've also tried to reproduce on 32-bit and 64-bit Intel, without > > success. So if this is real, maybe it's related to being big-endian > > hardware? But it's also quite sensitive to $du

Re: ALTER TABLE .. DETACH PARTITION CONCURRENTLY

2021-04-28 Thread Álvaro Herrera
Thanks for re-reviewing! This one I hope is the last version. On Wed, Apr 28, 2021, at 10:21 AM, Amit Langote wrote: > I noticed that rd_partdesc_nodetached_xmin can sometimes end up with > value 0. While you seem to be already aware of that, because otherwise > you wouldn't have added Transaction

Re: [PATCH] expand the units that pg_size_pretty supports on output

2021-04-28 Thread David Christensen
A second patch to teach the same units to pg_size_bytes(). Best, David On Wed, Apr 14, 2021 at 11:13 AM David Christensen < david.christen...@crunchydata.com> wrote: > Hi folks, > > Enclosed is a patch that expands the unit output for > pg_size_pretty(numeric) going up to Yottabytes; I reworked

回复:Attach to shared memory after fork()

2021-04-28 Thread 邱宇航(烛远)
> Windows has CreateProcess, which isn't available elsewhere. Yes, we still need fork() on *nix. So the solution is to reduce the overhead of fork(). Attach to shared memory after fork() might be a "Better shared memory management". > This is one of the reasons for using a connection pooler like p

Re: pg_upgrade fails to detect unsupported arrays and ranges

2021-04-28 Thread Tom Lane
[ blast-from-the-past department ] I wrote: > Daniel Gustafsson writes: >> I can see the appeal of >> including it before the wrap, even though I personally would've held off. > Nah, I'm not gonna risk it at this stage. I concur with your point > that this is an ancient bug, and one that is unl

Re: pg_hba.conf.sample wording improvement

2021-04-28 Thread Tom Lane
Alvaro Herrera writes: > Looking at it now, I wonder how well do the "hostno" options work. If I > say "hostnogssenc", is an SSL-encrypted socket good? If I say > "hostnossl", is a GSS-encrypted socket good? If so, how does that make > sense? Kind of off-topic for this thread, but I wonder if

Re: Skip temporary table schema name from explain-verbose output.

2021-04-28 Thread Tom Lane
Greg Stark writes: >> On Tue, Apr 27, 2021 at 7:08 PM Bharath Rupireddy >> > Make sense, we would lose the ability to differentiate temporary >> tables from the auto_explain logs. > There's no useful differentiation that can be done with the temp > schema name. Agreed. > I would say it makes se

Re: ALTER TABLE .. DETACH PARTITION CONCURRENTLY

2021-04-28 Thread Amit Langote
On Wed, Apr 28, 2021 at 8:32 AM Alvaro Herrera wrote: > On 2021-Apr-27, Alvaro Herrera wrote: > > > This v3 handles things as you suggested and works correctly AFAICT. I'm > > going to add some more tests cases to verify the behavior in the > > scenarios you showed, and get them to run under cach

Re: pg_hba.conf.sample wording improvement

2021-04-28 Thread Tom Lane
Peter Eisentraut writes: > I propose the attached patch to shake up the wording in the connection > type section of pg_hba.conf.sample a bit. After the hostgssenc part was > added on, the whole thing became a bit wordy, and it's also a bit > inaccurate for example in that the current wording f

Re: Skip temporary table schema name from explain-verbose output.

2021-04-28 Thread Greg Stark
> On Tue, Apr 27, 2021 at 7:08 PM Bharath Rupireddy > Make sense, we would lose the ability to differentiate temporary > tables from the auto_explain logs. There's no useful differentiation that can be done with the temp schema name. They're assigned on connection start randomly from the pool of

Re: Replication slot stats misgivings

2021-04-28 Thread Masahiko Sawada
On Wed, Apr 28, 2021 at 3:25 PM Amit Kapila wrote: > > On Wed, Apr 28, 2021 at 9:37 AM Masahiko Sawada wrote: > > > > On Wed, Apr 28, 2021 at 12:29 PM Amit Kapila > > wrote: > > > > > > On Tue, Apr 27, 2021 at 11:02 AM vignesh C wrote: > > > > > > > > On Tue, Apr 27, 2021 at 9:48 AM vignesh C

Re: pg_hba.conf.sample wording improvement

2021-04-28 Thread Alvaro Herrera
On 2021-Apr-28, Peter Eisentraut wrote: > I propose the attached patch to shake up the wording in the connection type > section of pg_hba.conf.sample a bit. After the hostgssenc part was added > on, the whole thing became a bit wordy, and it's also a bit inaccurate for > example in that the curre

Re: Unresolved repliaction hang and stop problem.

2021-04-28 Thread Alvaro Herrera
On 2021-Apr-28, Amit Kapila wrote: > On Wed, Apr 28, 2021 at 6:48 AM Alvaro Herrera > wrote: > > Hmm ... On what does it depend (other than plain git conflicts, which > > are aplenty)? On a quick look to the commit, it's clear that we need to > > be careful in order not to cause an ABI break,

Re: Since '2001-09-09 01:46:40'::timestamp microseconds are lost when extracting epoch

2021-04-28 Thread Tom Lane
Thomas Munro writes: > On Wed, Apr 28, 2021 at 3:56 AM Tom Lane wrote: >> Of course Wikipedia has been known to contain errors, but now >> I'm inclined to think I blew this. Anyone want to check my work? > I tried a couple of examples not from Wikipedia. ... Thanks for checking! I'll go adju

Re: Some doubious error messages and comments

2021-04-28 Thread Justin Pryzby
On Wed, Apr 28, 2021 at 05:36:33PM +0900, Kyotaro Horiguchi wrote: > 0001: I found some typos in a error message and a comment. > > multirangetypes.c: 1420 > > errmsg("range_intersect_agg must be called with a multirange"))); > > This "range_intersect_agg" looks like a typo of "multirange_..". >

Re: Use simplehash.h instead of dynahash in SMgr

2021-04-28 Thread Yura Sokolov
David Rowley писал 2021-04-26 09:43: I tried that and it got a median result of 113.795 seconds over 14 runs with this recovery benchmark test. LOG: size: 4096, members: 2032, filled: 0.496094, total chain: 1014, max chain: 6, avg chain: 0.499016, total_collisions: 428, max_collisions: 3, avg_c

Re: Some oversights in query_id calculation

2021-04-28 Thread Aleksander Alekseev
Hi Julien, > You should see failures doing a check-world or simply a make -C > contrib/pg_stat_statements check Sorry, my bad. I was running make check-world, but did it with -j4 flag which was a mistake. The patch is OK. On Wed, Apr 28, 2021 at 1:27 PM Julien Rouhaud wrote: > Hi Aleksander,

RE: Forget close an open relation in ReorderBufferProcessTXN()

2021-04-28 Thread osumi.takami...@fujitsu.com
On Monday, April 26, 2021 2:05 PM Amit Kapila wrote: > On Fri, Apr 23, 2021 at 8:03 PM osumi.takami...@fujitsu.com > wrote: > > > > On Saturday, April 17, 2021 4:13 PM Amit Kapila > wrote: > > > > I also don't find a test for this. It is introduced in > > > > 5dfd1e5a6696, wrote by Simon Riggs,

Re: Error in libpq docs for target_session_attrs, prefer-standby mode

2021-04-28 Thread Laurenz Albe
On Wed, 2021-04-28 at 12:55 +1000, Greg Nancarrow wrote: > I spotted an error in the development version documentation for > libpq's connection parameter "target_session_attrs" (34.1.2 Parameter > Key Words). > In the description for the "prefer-standby" mode, it says "... but if > none of the list

Re: Replication slot stats misgivings

2021-04-28 Thread Amit Kapila
On Wed, Apr 28, 2021 at 4:51 PM Masahiko Sawada wrote: > > On Wed, Apr 28, 2021 at 6:39 PM Amit Kapila wrote: > > > > > I think we can fix it by keeping track of total_size in toptxn as we > > are doing for the streaming case in ReorderBufferChangeMemoryUpdate. > > We can probably do it for non-s

Re: Replication slot stats misgivings

2021-04-28 Thread Masahiko Sawada
On Wed, Apr 28, 2021 at 6:39 PM Amit Kapila wrote: > > On Wed, Apr 28, 2021 at 12:49 PM Masahiko Sawada > wrote: > > > > > > BTW regarding the commit f5fc2f5b23 that added total_txns and > > total_bytes, we add the reorder buffer size (i.g., rb->size) to > > rb->totalBytes but I think we should

Re: Result Cache node shows per-worker info even for workers not launched

2021-04-28 Thread Amit Khandekar
On Wed, 28 Apr 2021 at 13:54, David Rowley wrote: > So, given that I removed the parallel test in partition_prune.sql, and > don't have any EXPLAIN ANALYZE output for parallel tests in > resultcache.sql, it should be safe enough to put that cache_misses == > 0 test back into explain.c > > I've at

Re: Result Cache node shows per-worker info even for workers not launched

2021-04-28 Thread Amit Khandekar
On Wed, 28 Apr 2021 at 16:14, David Rowley wrote: > However, I did add 1 test that sets work_mem down to 64kB to ensure > the eviction code does get some exercise. You'll notice that I pass > "true" to explain_resultcache() to hide the hits and misses there. We > can't test the exact number of hi

Re: Result Cache node shows per-worker info even for workers not launched

2021-04-28 Thread David Rowley
On Wed, 28 Apr 2021 at 21:38, Bharath Rupireddy wrote: > > On Wed, Apr 28, 2021 at 1:54 PM David Rowley wrote: > > I plan to push this in the next 24 hours or so. > > I happen to see explain_resultcache in resultcache.sql, seems like two > of the tests still have numbers for cache hits and misses

Re: Result Cache node shows per-worker info even for workers not launched

2021-04-28 Thread Amit Khandekar
On Wed, 28 Apr 2021 at 15:08, Bharath Rupireddy wrote: > > On Wed, Apr 28, 2021 at 1:54 PM David Rowley wrote: > > I plan to push this in the next 24 hours or so. > > I happen to see explain_resultcache in resultcache.sql, seems like two > of the tests still have numbers for cache hits and misses

Re: Some oversights in query_id calculation

2021-04-28 Thread Julien Rouhaud
Hi Aleksander, On Wed, Apr 28, 2021 at 01:19:36PM +0300, Aleksander Alekseev wrote: > Hi Julien, > > > I'm attaching a patch that fixes those, with regression tests to reproduce > > each > > problem. > > I believe something could be not quite right with the patch. Here is what I > did: > > $

Re: Parallel INSERT SELECT take 2

2021-04-28 Thread Greg Nancarrow
On Wed, Apr 28, 2021 at 12:44 PM houzj.f...@fujitsu.com wrote: > > 0003: > 1) Temporarily, add the check of built-in function by adding a member for > proparallel in FmgrBuiltin. > To avoid enlarging FmgrBuiltin struct , change the existing bool members > strict and and retset into > one member

Re: Some oversights in query_id calculation

2021-04-28 Thread Aleksander Alekseev
Hi Julien, > I'm attaching a patch that fixes those, with regression tests to reproduce > each > problem. I believe something could be not quite right with the patch. Here is what I did: $ git apply ... # revert the changes in the code but keep the new tests $ git checkout src/backend/utils/mis

Re: Built-in connection pooler

2021-04-28 Thread Antonin Houska
Konstantin Knizhnik wrote: > People asked me to resubmit built-in connection pooler patch to commitfest. > Rebased version of connection pooler is attached. I've reviewd the patch but haven't read the entire thread thoroughly. I hope that I don't duplicate many comments posted earlier by others.

Re: [PATCH] We install pg_regress and isolationtester but not pg_isolation_regress

2021-04-28 Thread Aleksander Alekseev
Hi hackers, > Any opinions on backpatching? > Other than src/test/modules/brin, the ISOLATION users don't look > much like real extensions (rather than test scaffolding), either. > Out of core, the only thing I can see with isolation tests is rum, but > it uses a workaround to have an access to

Re: Replication slot stats misgivings

2021-04-28 Thread Amit Kapila
On Wed, Apr 28, 2021 at 12:49 PM Masahiko Sawada wrote: > > > BTW regarding the commit f5fc2f5b23 that added total_txns and > total_bytes, we add the reorder buffer size (i.g., rb->size) to > rb->totalBytes but I think we should use the transaction size (i.g., > txn->size) instead: > You are righ

Re: Result Cache node shows per-worker info even for workers not launched

2021-04-28 Thread Bharath Rupireddy
On Wed, Apr 28, 2021 at 1:54 PM David Rowley wrote: > I plan to push this in the next 24 hours or so. I happen to see explain_resultcache in resultcache.sql, seems like two of the tests still have numbers for cache hits and misses - Hits: 980 Misses: 20, won't these make tests unstable? Will thes

Re: "Multiple table synchronizations are processed serially" still happens

2021-04-28 Thread Guillaume Lelarge
Le mer. 28 avr. 2021 à 11:12, Guillaume Lelarge a écrit : > Hi, > > One of my customers has an issue with logical replication. As $SUBJECT > says, multiple table synchronization happens serially. To be honest, it > doesn't do this every time. It happens when the tables are big enough. > > This is

"Multiple table synchronizations are processed serially" still happens

2021-04-28 Thread Guillaume Lelarge
Hi, One of my customers has an issue with logical replication. As $SUBJECT says, multiple table synchronization happens serially. To be honest, it doesn't do this every time. It happens when the tables are big enough. This issue was already described on this thread (from 2017): https://www.postgr

Re: Better sanity checking for message length words

2021-04-28 Thread Aleksander Alekseev
Hi Tom, > scratches head ... ] The patch still applies perfectly cleanly > for me, using either "patch" or "git apply". Sorry, my bad. It was about lines separating on different platforms. The patch is fine and passes installcheck-world on MacOS. -- Best regards, Aleksander Alekseev

Some doubious error messages and comments

2021-04-28 Thread Kyotaro Horiguchi
Hello. 0001: I found some typos in a error message and a comment. multirangetypes.c: 1420 > errmsg("range_intersect_agg must be called with a multirange"))); This "range_intersect_agg" looks like a typo of "multirange_..". operatorcmds.c:303 > * Look up a join estimator function ny name, and ve

Re: Result Cache node shows per-worker info even for workers not launched

2021-04-28 Thread David Rowley
On Wed, 28 Apr 2021 at 00:39, Amit Khandekar wrote: > If planned parallel workers do not get launched, the Result Cache plan > node shows all-0 stats for each of those workers: Thanks for reporting this and for the patch. You're right that there is a problem here. I did in fact have code to skip

Re: [BUG] "FailedAssertion" reported when streaming in logical replication

2021-04-28 Thread Dilip Kumar
On Wed, Apr 28, 2021 at 12:25 PM tanghy.f...@fujitsu.com wrote: > > > I have modified the patch based on the above comments. > > Thanks for your patch. > I tested again after applying your patch and the problem is fixed. Thanks for confirming. -- Regards, Dilip Kumar EnterpriseDB: http://www.en

Re: Replication slot stats misgivings

2021-04-28 Thread Masahiko Sawada
On Fri, Apr 16, 2021 at 2:58 PM Amit Kapila wrote: > > On Thu, Apr 15, 2021 at 4:35 PM Masahiko Sawada wrote: > > > > Thank you for the update! The patch looks good to me. > > BTW regarding the commit f5fc2f5b23 that added total_txns and total_bytes, we add the reorder buffer size (i.g., rb->siz