Re: Multiple Query IDs for a rewritten parse tree

2022-01-09 Thread Andrey V. Lepikhov
On 1/10/22 9:51 AM, Julien Rouhaud wrote: On Mon, Jan 10, 2022 at 09:10:59AM +0500, Andrey V. Lepikhov wrote: I can add one more use case. Our extension for freezing query plan uses query tree comparison technique to prove, that the plan can be applied (and we don't need to execute planning

Re: Why is src/test/modules/committs/t/002_standby.pl flaky?

2022-01-09 Thread Alexander Lakhin
10.01.2022 05:00, Thomas Munro wrote: > On Mon, Jan 10, 2022 at 8:06 AM Thomas Munro wrote: >> On Mon, Jan 10, 2022 at 12:00 AM Alexander Lakhin >> wrote: >>> Going down through the call chain, I see that at the end of it >>> WaitForMultipleObjects() hangs while waiting for the primary

Re: pg_upgrade verbosity when redirecting output to log file

2022-01-09 Thread Andres Freund
Hi, On 2022-01-10 01:14:32 -0500, Tom Lane wrote: > Andres Freund writes: > Fun! That seems to me to be a strong argument for not letting > the behavior vary depending on isatty(). Yea. > I think I'd vote for just nuking that output altogether. > It seems of very dubious value. It seems

Re: row filtering for logical replication

2022-01-09 Thread Amit Kapila
On Mon, Jan 10, 2022 at 8:41 AM houzj.f...@fujitsu.com wrote: > > Attach the v61 patch set. > Few comments: == 1. pgoutput_row_filter() { .. + + oldctx = MemoryContextSwitchTo(CacheMemoryContext); + rfnode = n_filters > 1 ? makeBoolExpr(OR_EXPR, rfnodes[idx], -1) :

Re: pg_upgrade verbosity when redirecting output to log file

2022-01-09 Thread Tom Lane
Andres Freund writes: > On 2022-01-09 20:28:40 -0800, Andres Freund wrote: >> Turns out that it only happens when the output is not a tty. And I notice it >> whenever I redirect the log output to a file, pipe, or such. > Ah. More precisely, it happens when doing > make -s -Otarget -j32

Re: ICU for global collation

2022-01-09 Thread Julien Rouhaud
On Mon, Jan 10, 2022 at 11:25:08AM +0800, Julien Rouhaud wrote: > On Fri, Jan 07, 2022 at 03:25:28PM +0100, Peter Eisentraut wrote: > > > > I tested this a bit. I used the following setup: > > > > create table t1 (a text); > > insert into t1 select md5(generate_series(1, 1000)::text); > >

Re: Is it correct to update db state in control file as "shutting down" during end-of-recovery checkpoint?

2022-01-09 Thread Bharath Rupireddy
On Mon, Jan 10, 2022 at 10:58 AM Jaime Casanova wrote: > > On Thu, Dec 09, 2021 at 07:41:52AM +0530, Bharath Rupireddy wrote: > > On Wed, Dec 8, 2021 at 11:02 PM Bossart, Nathan wrote: > > > > > > On 12/8/21, 3:29 AM, "Bharath Rupireddy" > > > wrote: > > > > Thanks for your thoughts. I'm fine

Re: Is it correct to update db state in control file as "shutting down" during end-of-recovery checkpoint?

2022-01-09 Thread Jaime Casanova
On Thu, Dec 09, 2021 at 07:41:52AM +0530, Bharath Rupireddy wrote: > On Wed, Dec 8, 2021 at 11:02 PM Bossart, Nathan wrote: > > > > On 12/8/21, 3:29 AM, "Bharath Rupireddy" > > wrote: > > > Thanks for your thoughts. I'm fine either way, hence attaching two > > > patches here with and I will

Re: Fix a possible typo in rewriteheap.c code comments

2022-01-09 Thread Amit Kapila
On Mon, Jan 10, 2022 at 9:42 AM Bharath Rupireddy wrote: > > Hi, > > It looks like there's a typo, attaching a tiny patch to fix it. > > * > * When doing logical decoding - which relies on using cmin/cmax of catalog > * tuples, via xl_heap_new_cid records - heap rewrites have to log enough

Re: Multiple Query IDs for a rewritten parse tree

2022-01-09 Thread Julien Rouhaud
On Mon, Jan 10, 2022 at 09:10:59AM +0500, Andrey V. Lepikhov wrote: > On 1/9/22 5:13 PM, Julien Rouhaud wrote: > > For now the queryid mixes two different things: fingerprinting and query > > text > > normalization. Should each calculation method be allowed to do a different > > normalization

Re: pg_upgrade verbosity when redirecting output to log file

2022-01-09 Thread Andres Freund
Hi, On 2022-01-09 20:28:40 -0800, Andres Freund wrote: > Turns out that it only happens when the output is not a tty. And I notice it > whenever I redirect the log output to a file, pipe, or such. Ah. More precisely, it happens when doing make -s -Otarget -j32 check-world, but *not* when

pg_upgrade verbosity when redirecting output to log file

2022-01-09 Thread Andres Freund
Hi, I've been annoyed for quite a while that pg_upgrade tests lists all copied files - it's ~1500 lines or so in the make -C src/bin/pg_ugprade check case (obviously for a cluster with more objects it's basically unlimited). But whenever I was looking into something about the issue, I didn't see

Re: TAP test to cover "EndOfLogTLI != replayTLI" case

2022-01-09 Thread Amul Sul
On Mon, Jan 10, 2022 at 8:25 AM Andres Freund wrote: > > Hi, > > On 2021-11-23 11:43:21 +0530, Amul Sul wrote: > > Attached patch covers a case where TLI in the filename for a > > record being read is different from where it belongs to. In other > > words, it covers following case noted in

Fix a possible typo in rewriteheap.c code comments

2022-01-09 Thread Bharath Rupireddy
Hi, It looks like there's a typo, attaching a tiny patch to fix it. * * When doing logical decoding - which relies on using cmin/cmax of catalog * tuples, via xl_heap_new_cid records - heap rewrites have to log enough - * information to allow the decoding backend to updates its internal

Re: Multiple Query IDs for a rewritten parse tree

2022-01-09 Thread Andrey V. Lepikhov
On 1/9/22 5:13 PM, Julien Rouhaud wrote: For now the queryid mixes two different things: fingerprinting and query text normalization. Should each calculation method be allowed to do a different normalization too, and if yes where should be stored the state data needed for that? If not, we

RE: row filtering for logical replication

2022-01-09 Thread houzj.f...@fujitsu.com
On Wednesday, January 5, 2022 2:01 PM vignesh C wrote: > On Tue, Jan 4, 2022 at 9:58 AM Peter Smith wrote: > > > > Here is the v58* patch set: > > > > Main changes from v57* are > > 1. Couple of review comments fixed > > > > ~~ > > > > Review comments (details) > > = > >

Re: ICU for global collation

2022-01-09 Thread Julien Rouhaud
On Fri, Jan 07, 2022 at 03:25:28PM +0100, Peter Eisentraut wrote: > > I tested this a bit. I used the following setup: > > create table t1 (a text); > insert into t1 select md5(generate_series(1, 1000)::text); > select count(*) from t1 where a > ''; > > And then I changed in varstr_cmp():

RE: row filtering for logical replication

2022-01-09 Thread houzj.f...@fujitsu.com
On Friday, January 7, 2022 3:40 PM Peter Smith wrote: > Below are some review comments for the v60 patches. > > 1e. > "Subsequent UPDATE or DELETE statements have no effect." > > Why won't they have an effect? The first impression is the newly updated tuple > now matches the filter, I think

Re: TAP test to cover "EndOfLogTLI != replayTLI" case

2022-01-09 Thread Andres Freund
Hi, On 2021-11-23 11:43:21 +0530, Amul Sul wrote: > Attached patch covers a case where TLI in the filename for a > record being read is different from where it belongs to. In other > words, it covers following case noted in StartupXLOG(): > Thoughts? Suggestions? It seems the test isn't quite

Re: pg_dump/restore --no-tableam

2022-01-09 Thread Andres Freund
Hi, On 2022-01-03 15:44:24 -0600, Justin Pryzby wrote: > @cfbot: rebased > From 69ae2ed5d00a97d351e1f6c45a9e406f33032898 Mon Sep 17 00:00:00 2001 > From: Justin Pryzby > Date: Sun, 7 Mar 2021 19:35:37 -0600 > Subject: [PATCH] Add pg_dump/restore --no-table-am.. > > This was for some reason

Re: null iv parameter passed to combo_init()

2022-01-09 Thread Zhihong Yu
On Sun, Jan 9, 2022 at 1:27 PM Zhihong Yu wrote: > > > On Sun, Jan 9, 2022 at 12:38 PM Zhihong Yu wrote: > >> >> >> On Sun, Jan 9, 2022 at 8:48 AM Noah Misch wrote: >> >>> On Sun, Jan 09, 2022 at 04:37:32AM -0800, Zhihong Yu wrote: >>> > On Sat, Jan 8, 2022 at 11:32 PM Tom Lane wrote: >>> > >

Re: [PATCH] Prefer getenv("HOME") to find the UNIX home directory

2022-01-09 Thread Tom Lane
I wrote: > Meh. I guess there's not much point in arguing with facts on the > ground. Anders' proposed behavior seems like the way to go then, > at least so far as libpq is concerned. So I pushed that, but while working on it I grew quite annoyed at the messy API exhibited by src/port/thread.c,

Re: Why is src/test/modules/committs/t/002_standby.pl flaky?

2022-01-09 Thread Thomas Munro
On Mon, Jan 10, 2022 at 8:06 AM Thomas Munro wrote: > On Mon, Jan 10, 2022 at 12:00 AM Alexander Lakhin wrote: > > Going down through the call chain, I see that at the end of it > > WaitForMultipleObjects() hangs while waiting for the primary connection > > socket event. So it looks like the

Re: Is it worth adding ReplicationSlot active_pid to ReplicationSlotPersistentData?

2022-01-09 Thread Bharath Rupireddy
On Wed, Jan 5, 2022 at 12:13 PM Kyotaro Horiguchi wrote: > > Here's the v2 patch. Please provide your thoughts. > > Thanks! I have three comments on this version. Thanks for your review. > - I still think "acquire/release" logs on creation/dropping should be > silenced. Adding the third

Re: Adding CI to our tree

2022-01-09 Thread Andres Freund
Hi, On 2021-10-02 12:59:09 -0700, Andres Freund wrote: > On 2021-10-02 11:05:20 -0400, Tom Lane wrote: > > I don't know enough about Windows to evaluate 0001, but I'm a little > > worried about it because it looks like it's changing our *production* > > error handling on that platform. > > Yea.

Re: Windows crash / abort handling

2022-01-09 Thread Andres Freund
Hi, On 2021-10-06 14:11:51 +0800, Craig Ringer wrote: > On Wed, 6 Oct 2021, 03:30 Andres Freund, wrote: > > > Hi, > > > > > > My first attempt was to try to use the existing crashdump stuff in > > pgwin32_install_crashdump_handler(). That's not really quite what I want, > > because it only

Re: [PATCH] Prefer getenv("HOME") to find the UNIX home directory

2022-01-09 Thread Tom Lane
Christoph Moench-Tegeder writes: > ## Tom Lane (t...@sss.pgh.pa.us): >> Isn't that a flat out violation of POSIX 8.3 Other Environment Variables? > After poking around across some Linuxes, it looks like people silently > agreed that "services" are not logged-in users: among the daemons, > having

Re: [PATCH] Prefer getenv("HOME") to find the UNIX home directory

2022-01-09 Thread Christoph Moench-Tegeder
## Tom Lane (t...@sss.pgh.pa.us): > Isn't that a flat out violation of POSIX 8.3 Other Environment Variables? > > HOME > The system shall initialize this variable at the time of login to > be a pathname of the user's home directory. See . > > To claim it's not, you have to

Re: [PATCH] Prefer getenv("HOME") to find the UNIX home directory

2022-01-09 Thread Tom Lane
Christoph Moench-Tegeder writes: > ## Tom Lane (t...@sss.pgh.pa.us): >> Given the POSIX requirements, it's basically impossible to believe >> that there are interesting cases where $HOME isn't set. > When I look at a random Debian with the usual PGDG packages, the > postmaster process (and every

Re: [PATCH] Prefer getenv("HOME") to find the UNIX home directory

2022-01-09 Thread Christoph Moench-Tegeder
## Tom Lane (t...@sss.pgh.pa.us): > Given the POSIX requirements, it's basically impossible to believe > that there are interesting cases where $HOME isn't set. When I look at a random Debian with the usual PGDG packages, the postmaster process (and every backend) has a rather minimal

Re: [PATCH] Prefer getenv("HOME") to find the UNIX home directory

2022-01-09 Thread Justin Pryzby
On Sun, Jan 09, 2022 at 01:59:02PM -0500, Tom Lane wrote: > Given the POSIX requirements, it's basically impossible to believe > that there are interesting cases where $HOME isn't set. I've run into this before - children of init may not have HOME set. It's easy enough to add it if it's needed,

Re: null iv parameter passed to combo_init()

2022-01-09 Thread Zhihong Yu
On Sun, Jan 9, 2022 at 12:38 PM Zhihong Yu wrote: > > > On Sun, Jan 9, 2022 at 8:48 AM Noah Misch wrote: > >> On Sun, Jan 09, 2022 at 04:37:32AM -0800, Zhihong Yu wrote: >> > On Sat, Jan 8, 2022 at 11:32 PM Tom Lane wrote: >> > > Noah Misch writes: >> > > > On further thought, I would write

Re: [PATCH] Prefer getenv("HOME") to find the UNIX home directory

2022-01-09 Thread Anders Kaseorg
On 1/9/22 13:04, Tom Lane wrote: The only case that the v1 patch helps such a user for is if they unset HOME or set it precisely to ''. If they set it to anything else, it's still broken from their perspective. So I do not find that that argument holds water. Moreover, ISTM that the only

Re: [PATCH] Prefer getenv("HOME") to find the UNIX home directory

2022-01-09 Thread Tom Lane
Anders Kaseorg writes: > On 1/9/22 10:59, Tom Lane wrote: >> Given the POSIX requirements, it's basically impossible to believe >> that there are interesting cases where $HOME isn't set. Thus, it >> seems to me that keeping the getpwuid calls will just mean carrying >> untestable dead code, so

Re: [PATCH] Prefer getenv("HOME") to find the UNIX home directory

2022-01-09 Thread Anders Kaseorg
On 1/9/22 10:59, Tom Lane wrote: Given the POSIX requirements, it's basically impossible to believe that there are interesting cases where $HOME isn't set. Thus, it seems to me that keeping the getpwuid calls will just mean carrying untestable dead code, so we should simplify matters by ripping

Re: null iv parameter passed to combo_init()

2022-01-09 Thread Zhihong Yu
On Sun, Jan 9, 2022 at 8:48 AM Noah Misch wrote: > On Sun, Jan 09, 2022 at 04:37:32AM -0800, Zhihong Yu wrote: > > On Sat, Jan 8, 2022 at 11:32 PM Tom Lane wrote: > > > Noah Misch writes: > > > > On further thought, I would write it this way: > > > > > > > - else > > > > +

Re: warn if GUC set to an invalid shared library

2022-01-09 Thread Maciek Sakrejda
On Sat, Jan 8, 2022 at 2:07 PM Justin Pryzby wrote: > Unfortunately, the output for dlopen() is not portable, which (I think) means > most of what I wrote can't be made to work.. Since it doesn't work to call > dlopen() when using SET, I tried using just stat(). But that also fails on >

Re: Adding CI to our tree

2022-01-09 Thread Andres Freund
Hi, On 2022-01-09 13:16:50 -0600, Justin Pryzby wrote: > I noticed a patch failing in cfbot everywhere except windows: > > https://commitfest.postgresql.org/36/3476/ > | Invalid relcache when ADD PRIMARY KEY USING INDEX > > It's because vcregress skips tests which have NO_INSTALLCHECK=1. > Is

Re: Adding CI to our tree

2022-01-09 Thread Justin Pryzby
I noticed a patch failing in cfbot everywhere except windows: https://commitfest.postgresql.org/36/3476/ | Invalid relcache when ADD PRIMARY KEY USING INDEX It's because vcregress skips tests which have NO_INSTALLCHECK=1. Is it desirable to enable more module/contrib tests for windows CI ?

Re: Non-superuser subscription owners

2022-01-09 Thread Tom Lane
Jeff Davis writes: > On Sat, 2022-01-08 at 12:57 -0500, Tom Lane wrote: >> ... btw, I'd like to complain that this new test script consumes >> a completely excessive amount of time. > Should be fixed now; I brought the number of tests down from 100 to 14. > It's not running in 2 seconds on my

Re: Why is src/test/modules/committs/t/002_standby.pl flaky?

2022-01-09 Thread Thomas Munro
On Mon, Jan 10, 2022 at 12:00 AM Alexander Lakhin wrote: > Going down through the call chain, I see that at the end of it > WaitForMultipleObjects() hangs while waiting for the primary connection > socket event. So it looks like the socket, that is closed by the > primary, can get into a state

Re: [PATCH] Prefer getenv("HOME") to find the UNIX home directory

2022-01-09 Thread Tom Lane
Anders Kaseorg writes: > On 10/20/21 04:55, Daniel Gustafsson wrote: >> Is the proposed change portable across all linux/unix systems we support? >> Reading aobut indicates that it's likely to be, but neither NetBSD nor >> FreeBSD >> have the upthread referenced wording in their manpages. >

Re: Non-superuser subscription owners

2022-01-09 Thread Jeff Davis
On Sat, 2022-01-08 at 12:57 -0500, Tom Lane wrote: > ... btw, I'd like to complain that this new test script consumes > a completely excessive amount of time. Should be fixed now; I brought the number of tests down from 100 to 14. It's not running in 2 seconds on my machine, but it's in line with

Re: null iv parameter passed to combo_init()

2022-01-09 Thread Tom Lane
Noah Misch writes: > On Sun, Jan 09, 2022 at 04:37:32AM -0800, Zhihong Yu wrote: >> On Sat, Jan 8, 2022 at 11:32 PM Tom Lane wrote: >>> FWIW, I liked the "ivlen > 0" formulation better. They should be >>> equivalent, because ivlen is unsigned, but it just seems like "> 0" >>> is more natural.

Re: Why is src/test/modules/committs/t/002_standby.pl flaky?

2022-01-09 Thread Tom Lane
Alexander Lakhin writes: > 09.01.2022 04:17, Tom Lane wrote: >> ... wait a minute. After some more study of the buildfarm logs, >> it was brought home to me that these failures started happening >> just after 6051857fc went in: > I've managed to reproduce this failure too. > Removing

Re: null iv parameter passed to combo_init()

2022-01-09 Thread Noah Misch
On Sun, Jan 09, 2022 at 04:37:32AM -0800, Zhihong Yu wrote: > On Sat, Jan 8, 2022 at 11:32 PM Tom Lane wrote: > > Noah Misch writes: > > > On further thought, I would write it this way: > > > > > - else > > > + else if (ivlen != 0) > > >

Add lasterrno setting for dir_existsfile()

2022-01-09 Thread Wei Sun
Hi, Some time ago,the following patch clean up error handling in pg_basebackup's walmethods.c. https://github.com/postgres/postgres/commit/248c3a9 This patch keep the error state in the DirectoryMethodData struct, in most functions, the lasterrnois set correctly, but in function

Re: null iv parameter passed to combo_init()

2022-01-09 Thread Zhihong Yu
On Sat, Jan 8, 2022 at 11:32 PM Tom Lane wrote: > Noah Misch writes: > > On further thought, I would write it this way: > > > - else > > + else if (ivlen != 0) > > memcpy(ivbuf, iv, ivlen); > > FWIW, I liked the "ivlen > 0" formulation better. They

Re: Multiple Query IDs for a rewritten parse tree

2022-01-09 Thread Julien Rouhaud
On Sat, Jan 08, 2022 at 07:49:59PM -0500, Tom Lane wrote: > > The idea I'd been vaguely thinking about is to allow attaching a list > of query-hash nodes to a Query, where each node would contain a "tag" > identifying the specific hash calculation method, and also the value > of the query's hash

Re: Multiple Query IDs for a rewritten parse tree

2022-01-09 Thread Julien Rouhaud
On Sun, Jan 09, 2022 at 12:43:06PM +0100, Dmitry Dolgov wrote: > > An extension that wants a slightly modified version of hash calculation > implementation from the core would still need to copy everything. The > core probably has to provide more than one (hash, method) pair to cover > some basic

Re: Multiple Query IDs for a rewritten parse tree

2022-01-09 Thread Dmitry Dolgov
> On Sat, Jan 08, 2022 at 07:49:59PM -0500, Tom Lane wrote: > > The idea I'd been vaguely thinking about is to allow attaching a list > of query-hash nodes to a Query, where each node would contain a "tag" > identifying the specific hash calculation method, and also the value > of the query's hash

Re: [PATCH] reduce page overlap of GiST indexes built using sorted method

2022-01-09 Thread sergei sh.
Hi, here are some benchmark results for GiST patch: https://gist.github.com/mngr777/88ae200c9c30ba5656583d92e8d2cf9e Code used for benchmarking: https://github.com/mngr777/pg_index_bm2, see README for the list of test queries. Results show query performance close to indexes built with no

Re: Why is src/test/modules/committs/t/002_standby.pl flaky?

2022-01-09 Thread Alexander Lakhin
Hello Tom, 09.01.2022 04:17, Tom Lane wrote: >> So for some reason, on these machines detection of walsender-initiated >> connection close is unreliable ... or maybe, the walsender didn't close >> the connection, but is somehow still hanging around? Don't have much idea >> where to dig beyond