Re: improve --with-lz4 install documentation

2022-02-17 Thread Michael Paquier
On Wed, Feb 16, 2022 at 11:36:31PM +0530, Jeevan Ladhe wrote: > Now that lz4 also supports backup compression, decompression, and more > future enhancements that can be done here, I think it is better to make > the --with-lz4 description more generic than adding specific details in > there. > >

Re: Report checkpoint progress with pg_stat_progress_checkpoint (was: Report checkpoint progress in server logs)

2022-02-17 Thread Julien Rouhaud
Hi, On Fri, Feb 18, 2022 at 12:20:26PM +0530, Nitin Jadhav wrote: > > > > If there's a checkpoint timed triggered and then someone calls > > pg_start_backup() which then wait for the end of the current checkpoint > > (possibly after changing the flags), I think the view should reflect that in > >

Re: Time to drop plpython2?

2022-02-17 Thread Mikael Kjellström
On 2022-02-17 19:08, Andres Freund wrote: I've pinged the owners of the animals failing so far: Now also pinged: - curculio Should be fixed by now. I did install the python3-package but the binary was called: /usr/local/bin/python3.5 for some reason so configure didn't pick it up.

Re: Timeout control within tests

2022-02-17 Thread Noah Misch
On Thu, Feb 17, 2022 at 09:48:25PM -0800, Andres Freund wrote: > On 2022-02-17 21:28:42 -0800, Noah Misch wrote: > > I propose to have environment variable PG_TEST_TIMEOUT_DEFAULT control the > > timeout used in the places that currently hard-code 180s. > > Meson's test runner has the concept of

Re: automatically generating node support functions

2022-02-17 Thread Peter Eisentraut
On 14.02.22 18:09, Tom Lane wrote: * It's time for action on the business about extracting comments from the to-be-deleted code. done * The Perl script is kind of under-commented for my taste. It lacks a copyright notice, too. done * In the same vein, I should not have to

Re: Report checkpoint progress with pg_stat_progress_checkpoint (was: Report checkpoint progress in server logs)

2022-02-17 Thread Nitin Jadhav
> > > > Thank you for sharing the information. 'triggering backend PID' (int) > > > > - can be stored without any problem. > > > > > > There can be multiple processes triggering a checkpoint, or at least > > > wanting it > > > to happen or happen faster. > > > > Yes. There can be multiple

Re: row filtering for logical replication

2022-02-17 Thread Peter Smith
On Thu, Feb 17, 2022 at 5:37 PM Amit Kapila wrote: ... > As there is a new version, I would like to wait for a few more days > before committing. I am planning to commit this early next week (by > Tuesday) unless others or I see any more things that can be improved. I have no more review

Re: Fix CheckIndexCompatible comment

2022-02-17 Thread Yugo NAGATA
On Fri, 18 Feb 2022 12:22:32 +0900 Fujii Masao wrote: > > > On 2022/02/07 19:14, Yugo NAGATA wrote: > > Agreed. I updated the patch to add a comment about 'oldId'. > > Thanks for updating the patch! I slightly modified the patch and pushed it. Thanks! > > Regards, > > -- > Fujii Masao >

RE: Failed transaction statistics to measure the logical replication progress

2022-02-17 Thread tanghy.f...@fujitsu.com
On Wed, Jan 12, 2022 8:35 PM osumi.takami...@fujitsu.com wrote: > > The attached v21 has a couple of other minor updates > like a modification of error message text. > > Thanks for updating the patch. Here are some comments. 1) I saw the following description about

Re: Report checkpoint progress with pg_stat_progress_checkpoint (was: Report checkpoint progress in server logs)

2022-02-17 Thread Nitin Jadhav
> Interesting idea, and overall a nice addition to the > pg_stat_progress_* reporting infrastructure. > > Could you add your patch to the current commitfest at > https://commitfest.postgresql.org/37/? > > See below for some comments on the patch: Thanks you for reviewing. I have added it to the

Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

2022-02-17 Thread Dilip Kumar
On Fri, Feb 18, 2022 at 5:09 AM Andres Freund wrote: Thanks a lot Andres for taking time to read the thread and patch. > > The only other kind of hazard I can think of is: could it be unsafe to > > try to interpret the contents of a database to which no one else is > > connected at present due

Re: Condition pushdown: why (=) is pushed down into join, but BETWEEN or >= is not?

2022-02-17 Thread Andy Fan
On Fri, Feb 18, 2022 at 4:15 AM Robert Haas wrote: > On Tue, Feb 1, 2022 at 10:08 AM Andy Fan wrote: > > To address the row estimation issue, The most straightforward way to fix > this is to > > ignore the derived clauses when figuring out the RelOptInfo->rows on > base relation. > > To note

Re: Race conditions in 019_replslot_limit.pl

2022-02-17 Thread Andres Freund
Hi, On 2022-02-17 19:41:40 -0800, Andres Freund wrote: > At this point the psql "walsender" is *not* terminated, because the fatal > message is waiting behind all the WAL data sent. Which also means that the > slot isn't yet released. Isn't it pretty bonkers that we allow error processing to get

Re: Timeout control within tests

2022-02-17 Thread Andres Freund
Hi, On 2022-02-17 21:28:42 -0800, Noah Misch wrote: > I propose to have environment variable PG_TEST_TIMEOUT_DEFAULT control the > timeout used in the places that currently hard-code 180s. Meson's test runner has the concept of a "timeout multiplier" for ways of running tests. Meson's stuff is

Timeout control within tests

2022-02-17 Thread Noah Misch
On Fri, Feb 05, 2021 at 03:55:20PM -0500, Tom Lane wrote: > We have, almost invariably, regretted it when we tried to use short > timeouts in test cases. > More generally, sometimes people want to do things like run a test > under valgrind. So it's not just "underpowered machines" that may >

Re: Fix a comment in worker.c

2022-02-17 Thread Masahiko Sawada
On Fri, Feb 18, 2022 at 1:48 PM Amit Kapila wrote: > > On Wed, Feb 16, 2022 at 4:56 PM Amit Kapila wrote: > > > > On Wed, Feb 16, 2022 at 5:57 AM Masahiko Sawada > > wrote: > > > > > > While reading the code, I realized that the second sentence of the > > > following comment in worker.c is not

Re: Fix a comment in worker.c

2022-02-17 Thread Amit Kapila
On Wed, Feb 16, 2022 at 4:56 PM Amit Kapila wrote: > > On Wed, Feb 16, 2022 at 5:57 AM Masahiko Sawada wrote: > > > > While reading the code, I realized that the second sentence of the > > following comment in worker.c is not correct: > > > >/* > > * Exit if the subscription was

Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

2022-02-17 Thread Dilip Kumar
On Fri, Feb 18, 2022 at 4:30 AM Robert Haas wrote: > > > > This thread is long. Could you summarize what lead you to consider other > > approaches (e.g. looking in the filesystem for relfilenodes) as not > > feasible / > > too ugly / ...? > > I don't think it's infeasible to look at the

Re: Race conditions in 019_replslot_limit.pl

2022-02-17 Thread Tom Lane
Andres Freund writes: > There've not been any new reports in the last 18 hours, but that's probably > just due to lower commit activity triggering fewer runs. myna just showed the same symptom: https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=myna=2022-02-18%2003%3A00%3A17 # Failed

Re: Race conditions in 019_replslot_limit.pl

2022-02-17 Thread Andres Freund
Hi, On 2022-02-17 16:34:34 -0800, Andres Freund wrote: > I've done this for a couple hundred iterations, under load, subsequently with > additional assertions, without being able to reproduce. Playing around with this further I did get into one interesting state: I started psql with

Re: Fix CheckIndexCompatible comment

2022-02-17 Thread Fujii Masao
On 2022/02/07 19:14, Yugo NAGATA wrote: Agreed. I updated the patch to add a comment about 'oldId'. Thanks for updating the patch! I slightly modified the patch and pushed it. Regards, -- Fujii Masao Advanced Computing Technology Center Research and Development Headquarters NTT DATA

Re: Assert in pageinspect with NULL pages

2022-02-17 Thread Michael Paquier
On Thu, Feb 17, 2022 at 09:00:20PM -0600, Justin Pryzby wrote: > BRIN can also crash if passed a non-brin index. > > I've been sitting on this one for awhile. Feel free to include it in your > patchset. Ugh. Thanks! I am keeping a note about this one. -- Michael signature.asc Description:

Fix formatting of Interval output

2022-02-17 Thread Joseph Koshakow
When formatting the output of an Interval, we call abs() on the hours field of the Interval. Calling abs(INT_MIN) returns back INT_MIN causing the output to contain two '-' characters. The attached patch fixes that issue by special casing INT_MIN hours. Here is an example of the issue:

Re: Assert in pageinspect with NULL pages

2022-02-17 Thread Michael Paquier
On Thu, Feb 17, 2022 at 05:40:41PM +0800, Julien Rouhaud wrote: > About the patch, it's incorrectly using a hardcoded 8192 block-size rather > than > using the computed :block_size (or computing one when it's not already the > case). Well, the tests of pageinspect fail would already fail when

Re: Assert in pageinspect with NULL pages

2022-02-17 Thread Justin Pryzby
BRIN can also crash if passed a non-brin index. I've been sitting on this one for awhile. Feel free to include it in your patchset. commit 08010a6037fc4e24a9ba05e5386e766f4310d35e Author: Justin Pryzby Date: Tue Jan 19 00:25:15 2021 -0600 pageinspect: brin_page_items(): check that given

Re: Logical replication timeout problem

2022-02-17 Thread Ajin Cherian
On Tue, Feb 8, 2022 at 1:59 PM wangw.f...@fujitsu.com wrote: > > On Wed, Jan 26, 2022 at 11:37 AM I wrote: > > On Sat, Jan 22, 2022 at 7:12 PM Amit Kapila wrote: > > > Now, one idea to solve this problem could be that whenever we skip > > > sending any change we do try to update the plugin

Re: Fix overflow in DecodeInterval

2022-02-17 Thread Joseph Koshakow
Ok, so I've attached a patch with my final unprompted changes. It contains the following two changes: 1. I found some more overflows with the ISO8601 formats and have included some fixes. 2. I reverted the overflow checks for the seconds field. It's actually a bit more complicated than I thought.

Re: Support escape sequence for cluster_name in postgres_fdw.application_name

2022-02-17 Thread Fujii Masao
On 2022/02/15 8:52, r.takahash...@fujitsu.com wrote: Hi Fujii san, Thank you for updating the patch. I have no additional comments. Thanks for the review! Pushed. Regards, -- Fujii Masao Advanced Computing Technology Center Research and Development Headquarters NTT DATA CORPORATION

Re: adding 'zstd' as a compression algorithm

2022-02-17 Thread Andres Freund
Hi, On 2022-02-17 13:34:08 +0900, Michael Paquier wrote: > %define needs to include HAVE_LIBZSTD, HAVE_ZSTD_H and USE_ZSTD, so > this version fails the sanity check between pg_config.h.in and the > MSVC scripts checking that all flags exist. Do we really need all three defines? How about using

Re: Small and unaffected typo in pg_logical_slot_get_changes_guts()

2022-02-17 Thread Michael Paquier
On Thu, Feb 17, 2022 at 11:33:55AM +, Dagfinn Ilmari Mannsåker wrote: > Would it be possible to make that macro only defined when building > extensions, but not when building Postgres itself? For example, Perl > has a PERL_CORE macro that's only defined when building Perl itself, but > not

Re: adding 'zstd' as a compression algorithm

2022-02-17 Thread Michael Paquier
On Thu, Feb 17, 2022 at 09:40:13AM -0500, Robert Haas wrote: > Ah, OK, cool. It seems like we have consensus on this being a good > direction, so I plan to commit this later today unless objections or > additional review comments turn up. So, will there be a part of the system where we'll make

Re: Silencing upcoming warning about stack_base_ptr

2022-02-17 Thread Tom Lane
Andres Freund writes: > On 2022-02-17 18:44:27 -0500, Tom Lane wrote: >> (that's visible now on buildfarm member caiman). We probably >> should take some thought for silencing this before it starts >> to be in people's faces during routine development. > We could try using

Re: Race conditions in 019_replslot_limit.pl

2022-02-17 Thread Andres Freund
Hi, On 2022-02-16 20:03:00 -0800, Andres Freund wrote: > I'll run the test in a loop, perhaps I can reproduce... I've done this for a couple hundred iterations, under load, subsequently with additional assertions, without being able to reproduce. There've not been any new reports in the last 18

Re: Nonrandom scanned_pages distorts pg_class.reltuples set by VACUUM

2022-02-17 Thread Peter Geoghegan
On Thu, Feb 17, 2022 at 4:18 PM Andres Freund wrote: > > What substantial changes are you referring to? The one thing that did > > change was the commit message, which framed everything in terms of the > > later work. It really is true that the patch that I committed was > > essentially the same

Re: Nonrandom scanned_pages distorts pg_class.reltuples set by VACUUM

2022-02-17 Thread Andres Freund
Hi, On 2022-02-17 14:23:51 -0800, Peter Geoghegan wrote: > On Thu, Feb 17, 2022 at 10:33 AM Andres Freund wrote: > > On 2022-02-16 19:43:09 -0800, Peter Geoghegan wrote: > > > It makes the code in vacuumlazy.c much cleaner. In fact, that's how commit > > > 44fa8488 started off -- purely as

Re: Silencing upcoming warning about stack_base_ptr

2022-02-17 Thread Andres Freund
Hi, On 2022-02-17 18:44:27 -0500, Tom Lane wrote: > (that's visible now on buildfarm member caiman). We probably > should take some thought for silencing this before it starts > to be in people's faces during routine development. Agreed. One annoying thing I recently encountered, related to

Silencing upcoming warning about stack_base_ptr

2022-02-17 Thread Tom Lane
GCC 12, coming soon to a distro near you, complains like this: postgres.c: In function 'set_stack_base': postgres.c:3430:24: warning: storing the address of local variable 'stack_base' in 'stack_base_ptr' [-Wdangling-pointer=] 3430 | stack_base_ptr = _base; |

Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

2022-02-17 Thread Andres Freund
Hi, On 2022-02-17 18:00:19 -0500, Robert Haas wrote: > Now you pointed out earlier one problem that it doesn't fix: unlike > the current method, this method involves reading buffers from the > template database into shared_buffers, and those buffers, once read, > stick around even after the

Re: O(n) tasks cause lengthy startups and checkpoints

2022-02-17 Thread Andres Freund
Hi, On 2022-02-17 14:58:38 -0800, Nathan Bossart wrote: > On Thu, Feb 17, 2022 at 02:28:29PM -0800, Andres Freund wrote: > > As far as I understand, the primary concern are logical decoding serialized > > snapshots, because a lot of them can accumulate if there e.g. is an old > > unused > > /

Re: Use generation context to speed up tuplesorts

2022-02-17 Thread David Rowley
On Sun, 13 Feb 2022 at 09:56, Tomas Vondra wrote: > I'm not against pushing the generation context improvements, e.g. > something like the patches posted in [1], because those seem reasonable > in general. But I'm somewhat confused about the last patch (adjusting > allocChunkLimit) causing fairly

Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

2022-02-17 Thread Robert Haas
On Thu, Feb 17, 2022 at 4:13 PM Andres Freund wrote: > Could you or Dilip outline how it now works, and what exactly makes it safe > etc (e.g. around locking, invalidation processing, snapshots, xid horizons)? > > I just scrolled through the patchset without finding such an explanation, so > it's

Re: O(n) tasks cause lengthy startups and checkpoints

2022-02-17 Thread Nathan Bossart
On Thu, Feb 17, 2022 at 02:28:29PM -0800, Andres Freund wrote: > As far as I understand, the primary concern are logical decoding serialized > snapshots, because a lot of them can accumulate if there e.g. is an old unused > / far behind slot. It should be easy to reduce the number of those

Re: O(n) tasks cause lengthy startups and checkpoints

2022-02-17 Thread Andres Freund
Hi, On 2022-02-17 13:00:22 -0800, Nathan Bossart wrote: > Okay. So IIUC the problem might already exist today, but offloading these > tasks to a separate process could make it more likely. Vastly more, yes. Before checkpoints not happening would be a (but not a great) form of backpressure. You

Re: Nonrandom scanned_pages distorts pg_class.reltuples set by VACUUM

2022-02-17 Thread Peter Geoghegan
On Thu, Feb 17, 2022 at 10:33 AM Andres Freund wrote: > On 2022-02-16 19:43:09 -0800, Peter Geoghegan wrote: > > It makes the code in vacuumlazy.c much cleaner. In fact, that's how commit > > 44fa8488 started off -- purely as refactoring work. > > The problem is that it didn't end up as that. You

Re: Nonrandom scanned_pages distorts pg_class.reltuples set by VACUUM

2022-02-17 Thread Peter Geoghegan
On Thu, Feb 17, 2022 at 1:36 PM Robert Haas wrote: > > It's not that simple. As I said in the fix-up commit message, and in > > the opening email to this thread, it basically isn't a new behavior at > > all. It would be much more accurate to describe it as a behavior that > > originated with

Re: Condition pushdown: why (=) is pushed down into join, but BETWEEN or >= is not?

2022-02-17 Thread Robert Haas
On Thu, Feb 17, 2022 at 4:17 PM Tomas Vondra wrote: > IMHO the whole problem is we're unable to estimate the join clause as a > conditional probability, i.e. > >P(A.x = B.x | (A.x < 42) & (B.x < 42)) > > so maybe instead of trying to generate additional RelOptInfo items we > should think

Re: Nonrandom scanned_pages distorts pg_class.reltuples set by VACUUM

2022-02-17 Thread Robert Haas
On Thu, Feb 17, 2022 at 4:10 PM Peter Geoghegan wrote: > > I would say it differently: I think the commit message does a poor job > > describing what the commit actually does. For example, it says nothing > > about changing VACUUM to always scan the last page of every heap > > relation. This

Re: Condition pushdown: why (=) is pushed down into join, but BETWEEN or >= is not?

2022-02-17 Thread Tomas Vondra
On 2/17/22 21:15, Robert Haas wrote: > On Tue, Feb 1, 2022 at 10:08 AM Andy Fan wrote: >> To address the row estimation issue, The most straightforward way to fix >> this is to >> ignore the derived clauses when figuring out the RelOptInfo->rows on base >> relation. >> To note which clause is

Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

2022-02-17 Thread Andres Freund
Hi, On 2022-02-17 14:27:09 -0500, Robert Haas wrote: > The other one is trickier, because AFAICT it's basically an opinion > question: is accessing pg_class in the template database from some backend > that is connected to another database too ugly to be acceptable? Several > people have

Re: Nonrandom scanned_pages distorts pg_class.reltuples set by VACUUM

2022-02-17 Thread Peter Geoghegan
On Thu, Feb 17, 2022 at 6:17 AM Robert Haas wrote: > Let's back up a minute and talk about the commit of $SUBJECT. The > commit message contains a Discussion link to this thread. This thread, > at the time you put that link in there, had exactly one post: from > you. That's not much of a

Re: O(n) tasks cause lengthy startups and checkpoints

2022-02-17 Thread Nathan Bossart
On Thu, Feb 17, 2022 at 11:27:09AM -0800, Andres Freund wrote: > On 2022-02-17 10:23:37 -0800, Nathan Bossart wrote: >> On Wed, Feb 16, 2022 at 10:59:38PM -0800, Andres Freund wrote: >> > They're accessed by xid. The LSN is just for cleanup. Accessing files >> > left over from a previous

Re: buildfarm warnings

2022-02-17 Thread Robert Haas
On Thu, Feb 17, 2022 at 3:51 PM Andres Freund wrote: > On 2022-02-17 15:22:08 -0500, Robert Haas wrote: > > OK, sounds good, thanks. I couldn't (and still can't) think of a good > > way of testing the progress-reporting code either. I mean I guess if > > you could convince pg_basebackup not to

Re: buildfarm warnings

2022-02-17 Thread Andres Freund
On 2022-02-17 15:22:08 -0500, Robert Haas wrote: > OK, sounds good, thanks. I couldn't (and still can't) think of a good > way of testing the progress-reporting code either. I mean I guess if > you could convince pg_basebackup not to truncate the filenames, maybe > by convincing it that your

Re: killing perl2host

2022-02-17 Thread Andres Freund
On 2022-02-17 15:23:36 -0500, Andrew Dunstan wrote: > Very well. I think the easiest way will be to stash $host_os in the > environment and let the script pick it up similarly to what I suggested > with MSYSTEM. WFM.

Re: killing perl2host

2022-02-17 Thread Andrew Dunstan
On 2/17/22 15:09, Andres Freund wrote: > Hi, > > On 2022-02-17 14:40:01 -0500, Andrew Dunstan wrote: >> Sure, that could be done, but what's the issue? Msys2 normally defines >> MSYSTEM for you - see /etc/msystem which is sourced by /etc/profile. > It seems not a great idea to me to use

Re: buildfarm warnings

2022-02-17 Thread Robert Haas
On Thu, Feb 17, 2022 at 2:36 PM Tom Lane wrote: > Yeah, I came to the same conclusion while out doing some errands. > There's no very convincing reason to believe that what's passed to > progress_update_filename has got adequate lifespan either, or that > that would remain true even if it's true

Re: Condition pushdown: why (=) is pushed down into join, but BETWEEN or >= is not?

2022-02-17 Thread Robert Haas
On Tue, Feb 1, 2022 at 10:08 AM Andy Fan wrote: > To address the row estimation issue, The most straightforward way to fix this > is to > ignore the derived clauses when figuring out the RelOptInfo->rows on base > relation. > To note which clause is derived from this patch, I added a new field

Re: killing perl2host

2022-02-17 Thread Andres Freund
Hi, On 2022-02-17 14:40:01 -0500, Andrew Dunstan wrote: > Sure, that could be done, but what's the issue? Msys2 normally defines > MSYSTEM for you - see /etc/msystem which is sourced by /etc/profile. It seems not a great idea to me to use different sources of truth about build target. And I

Re: killing perl2host

2022-02-17 Thread Andrew Dunstan
On 2/17/22 13:10, Andres Freund wrote: > On 2022-02-17 13:08:01 -0500, Andrew Dunstan wrote: perhaps something like:     my $msystem = $ENV{MSYSTEM} || 'undef';     die "incompatible perl" if $Config{osname} eq 'msys' && $msystem ne 'MSYS'; >>> Why tests

Re: buildfarm warnings

2022-02-17 Thread Tom Lane
Robert Haas writes: > If not, I think that your quick-and-dirty fix is about right, except > that we probably need to do it every place where we set > progress_filename, and we should arrange to free the memory later. > With -Ft, you won't have enough archives to matter, but with -Fp, you > might

Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

2022-02-17 Thread Robert Haas
On Tue, Feb 15, 2022 at 6:49 AM Dilip Kumar wrote: > Here is the updated version of the patch, the changes are 1) Fixed > review comments given by Robert and one open comment from Ashutosh. > 2) Preserved the old create db method. 3) As agreed upthread for now > we are using the new strategy only

Re: O(n) tasks cause lengthy startups and checkpoints

2022-02-17 Thread Andres Freund
Hi, On 2022-02-17 10:23:37 -0800, Nathan Bossart wrote: > On Wed, Feb 16, 2022 at 10:59:38PM -0800, Andres Freund wrote: > > They're accessed by xid. The LSN is just for cleanup. Accessing files > > left over from a previous transaction with the same xid wouldn't be > > good - we'd read wrong

Proposal: Support custom authentication methods using hooks

2022-02-17 Thread samay sharma
Hi all, I wanted to submit a patch to expose 2 new hooks (one for the authentication check and another one for error reporting) in auth.c. These will allow users to implement their own authentication methods for Postgres or add custom logic around authentication. A use case where this is useful

Re: [PATCH] Support pg_ident mapping for LDAP

2022-02-17 Thread Jacob Champion
On Fri, 2021-10-29 at 17:38 +, Jacob Champion wrote: > v3 attached, which uses the above naming scheme and removes the stale > TODO. Changes in since-v2. v4 rebases over the recent TAP changes. --Jacob From e0f36725013610eade9bc83414c4d1f5adea17e2 Mon Sep 17 00:00:00 2001 From: Jacob

Re: support for CREATE MODULE

2022-02-17 Thread Bruce Momjian
On Tue, Feb 15, 2022 at 12:29:54PM -0800, Swaha Miller wrote: > On Mon, Feb 14, 2022 at 4:58 PM Bruce Momjian wrote: > > I was working on a talk about microservices today and decided to create > > two schemas --- a public one that has USAGE permission for other > services > > with

Isn't gist_page_items broken for opclasses with compress methods?

2022-02-17 Thread Tomas Vondra
Hi, I've been looking at a report of a crash an in-place upgrade [1], related to a GiST index, and I tried to use the new GiST support added to pageinspect in 756ab29124 (so it's in 14). My knowledge of GiST is somewhat limited, but after struggling with it for a while I wonder if it's actually

Re: Condition pushdown: why (=) is pushed down into join, but BETWEEN or >= is not?

2022-02-17 Thread Finnerty, Jim
So I think knowing what bad it is to have this feature is the key point to discussion now. > While I've only read your description of the patch not the patch itself, This comment applies to me also. Is the join selectivity properly calculated in all cases, e.g. in the n:m join case in

Re: Nonrandom scanned_pages distorts pg_class.reltuples set by VACUUM

2022-02-17 Thread Andres Freund
Hi, On 2022-02-16 19:43:09 -0800, Peter Geoghegan wrote: > It makes the code in vacuumlazy.c much cleaner. In fact, that's how commit > 44fa8488 started off -- purely as refactoring work. The problem is that it didn't end up as that. You combined refactoring with substantial changes. And

Re: O(n) tasks cause lengthy startups and checkpoints

2022-02-17 Thread Nathan Bossart
On Wed, Feb 16, 2022 at 10:59:38PM -0800, Andres Freund wrote: > On 2022-02-16 20:14:04 -0800, Nathan Bossart wrote: >> >> - while ((spc_de = ReadDirExtended(spc_dir, "pg_tblspc", LOG)) != NULL) >> >> + while (!ShutdownRequestPending && >> >> +(spc_de = ReadDirExtended(spc_dir,

Re: Nonrandom scanned_pages distorts pg_class.reltuples set by VACUUM

2022-02-17 Thread Andres Freund
Hi, On 2022-02-17 09:17:04 -0500, Robert Haas wrote: > Commit messages need to describe what the commit actually changes. > Theoretical ideas are fine, but if I, as a committer who have done > significant work in this area in the past, can't read the commit > message and understand what is

Re: killing perl2host

2022-02-17 Thread Andres Freund
On 2022-02-17 13:08:01 -0500, Andrew Dunstan wrote: > >> perhaps something like: > >> > >> > >>     my $msystem = $ENV{MSYSTEM} || 'undef'; > >> > >>     die "incompatible perl" if $Config{osname} eq 'msys' && $msystem ne > >> 'MSYS'; > > Why tests MSYSTEM instead of $host_os? > Is that available

Re: Time to drop plpython2?

2022-02-17 Thread Andres Freund
Hi, On 2022-02-16 23:14:46 -0800, Andres Freund wrote: > > > > Done. Curious how red the BF will turn out to be. Let's hope it's not > > too bad. > > I've pinged the owners of the animals failing so far: Now also pinged: - curculio - guaibasaurus - snapper - gadwall, takin > - snakefly,

Re: killing perl2host

2022-02-17 Thread Andrew Dunstan
On 2/17/22 12:12, Andres Freund wrote: > Hi, > > On 2022-02-17 09:20:56 -0500, Andrew Dunstan wrote: >> I don't think we have or have ever had a buildfarm animal targeting >> msys. In general I think of msys as a build environment to create native >> binaries. But if we want to support targeting

Re: buildfarm warnings

2022-02-17 Thread Robert Haas
On Thu, Feb 17, 2022 at 12:08 PM Tom Lane wrote: > I wrote: > > Justin Pryzby writes: > >> pg_basebackup.c:1261:35: warning: storing the address of local variable > >> archive_filename in progress_filename [-Wdangling-pointer=] > >> => new in 23a1c6578 - looks like a real error > > > I saw that

Re: Per-table storage parameters for TableAM/IndexAM extensions

2022-02-17 Thread Sadhuprasad Patro
> On Sat, Feb 12, 2022 at 2:35 AM Robert Haas wrote: >> >> >> Imagine that I am using the "foo" tableam with "compression=lots" and >> I want to switch to the "bar" AM which does not support that option. >> If I remove the "compression=lots" option using a separate command, >> the "foo" table AM

Re: CREATEROLE and role ownership hierarchies

2022-02-17 Thread Robert Haas
On Mon, Jan 31, 2022 at 1:57 PM Joshua Brindle wrote: > This is precisely the use case I am trying to accomplish with this > patchset, roughly: > > - An automated bot that creates users and adds them to the employees role > - Bot cannot access any employee (or other roles) table data > - Bot

Re: [PATCH] Accept IP addresses in server certificate SANs

2022-02-17 Thread Jacob Champion
On Tue, 2022-02-15 at 15:16 +0900, Kyotaro Horiguchi wrote: > (This needs rebasing) Done in v6, attached. > # I forgot to mention that, the test fails for me even without the > # change. I didn't checked what is wrong there, though. Ah. We should probably figure that out, then -- what failures

Re: Report checkpoint progress with pg_stat_progress_checkpoint (was: Report checkpoint progress in server logs)

2022-02-17 Thread Julien Rouhaud
Hi, On Thu, Feb 17, 2022 at 10:39:02PM +0530, Nitin Jadhav wrote: > > > Thank you for sharing the information. 'triggering backend PID' (int) > > > - can be stored without any problem. > > > > There can be multiple processes triggering a checkpoint, or at least > > wanting it > > to happen or

Re: killing perl2host

2022-02-17 Thread Andres Freund
Hi, On 2022-02-17 09:20:56 -0500, Andrew Dunstan wrote: > I don't think we have or have ever had a buildfarm animal targeting > msys. In general I think of msys as a build environment to create native > binaries. But if we want to support targeting msys we should have an > animal doing that.

Re: Report checkpoint progress with pg_stat_progress_checkpoint (was: Report checkpoint progress in server logs)

2022-02-17 Thread Nitin Jadhav
> > Thank you for sharing the information. 'triggering backend PID' (int) > > - can be stored without any problem. > > There can be multiple processes triggering a checkpoint, or at least wanting > it > to happen or happen faster. Yes. There can be multiple processes but there will be one

Re: buildfarm warnings

2022-02-17 Thread Tom Lane
I wrote: > Justin Pryzby writes: >> pg_basebackup.c:1261:35: warning: storing the address of local variable >> archive_filename in progress_filename [-Wdangling-pointer=] >> => new in 23a1c6578 - looks like a real error > I saw that one a few days ago but didn't get around to looking > more

Re: initdb / bootstrap design

2022-02-17 Thread Andrew Dunstan
On 2/17/22 10:36, Robert Haas wrote: > On Wed, Feb 16, 2022 at 2:50 PM Andres Freund wrote: >>> initdb is already plenty fast enough for any plausible production >>> usage; it's cases like check-world where we wish it were faster. >> It's not just our own usage though. I've seen it be a

Re: postgres_fdw: commit remote (sub)transactions in parallel during pre-commit

2022-02-17 Thread Fujii Masao
On 2022/02/11 21:59, Etsuro Fujita wrote: I tweaked comments/docs a little bit as well. Thanks for updating the patches! I reviewed 0001 patch. It looks good to me except the following minor things. If these are addressed, I think that the 001 patch can be marked as ready for committer.

Re: pgsql: pg_upgrade: Preserve relfilenodes and tablespace OIDs.

2022-02-17 Thread Robert Haas
On Sun, Feb 13, 2022 at 6:51 AM Christoph Berg wrote: > Re: Robert Haas > > pg_upgrade: Preserve relfilenodes and tablespace OIDs. > > > src/bin/pg_dump/pg_dumpall.c | 3 + > > --- a/src/bin/pg_dump/pg_dumpall.c > +++ b/src/bin/pg_dump/pg_dumpall.c > @@ -1066,6 +1066,9 @@

Re: [PATCH] Fix out-of-bouds access (src/common/wchar.c)

2022-02-17 Thread Daniel Gustafsson
> On 17 Feb 2022, at 16:05, Tom Lane wrote: > > Daniel Gustafsson writes: >> Question remains though, should src/interfaces/ecpg/test/sql/sqlda.pgc really >> be using WIN32 and not _WIN32, or doesn't it matter? (or does it only matter >> for consistency?) WIN32 and _WIN32 aren't very

Re: initdb / bootstrap design

2022-02-17 Thread Robert Haas
On Wed, Feb 16, 2022 at 2:50 PM Andres Freund wrote: > > initdb is already plenty fast enough for any plausible production > > usage; it's cases like check-world where we wish it were faster. > > It's not just our own usage though. I've seen it be a noticable time in test > suites of applications

Re: [PATCH] Fix out-of-bouds access (src/common/wchar.c)

2022-02-17 Thread Tom Lane
Daniel Gustafsson writes: > Question remains though, should src/interfaces/ecpg/test/sql/sqlda.pgc really > be using WIN32 and not _WIN32, or doesn't it matter? (or does it only matter > for consistency?) WIN32 and _WIN32 aren't very informative searchterms to use > for finding more information.

Re: do only critical work during single-user vacuum?

2022-02-17 Thread Robert Haas
On Wed, Feb 16, 2022 at 10:08 PM Peter Geoghegan wrote: > > 6. Sometimes the user decides to run VACUUM FULL instead of plain > > VACUUM because it sounds better. > > It's a pity that the name suggests otherwise. If only we'd named it > something that suggests "option of last resort". Oh well.

Re: PGEventProcs must not be allowed to break libpq

2022-02-17 Thread Tom Lane
Alvaro Herrera writes: > Not really related to this complaint and patch, but as far as I can see, > libpq events go completely untested in the core source. Maybe we should > come up with a test module or something? Yeah, I suppose. The libpq part of it is pretty simple, but still...

Re: PGEventProcs must not be allowed to break libpq

2022-02-17 Thread Alvaro Herrera
Not really related to this complaint and patch, but as far as I can see, libpq events go completely untested in the core source. Maybe we should come up with a test module or something? -- Álvaro Herrera 39°49'30"S 73°17'W — https://www.EnterpriseDB.com/

Re: adding 'zstd' as a compression algorithm

2022-02-17 Thread Robert Haas
On Wed, Feb 16, 2022 at 11:34 PM Michael Paquier wrote: > Thanks. This looks pretty much right, except for two things that I > have taken the freedom to fix as of the v3 attached. Ah, OK, cool. It seems like we have consensus on this being a good direction, so I plan to commit this later today

Re: killing perl2host

2022-02-17 Thread Andrew Dunstan
On 2/16/22 21:17, Andres Freund wrote: > Hi, > > On 2022-02-16 14:42:30 -0800, Andres Freund wrote: >> On February 16, 2022 1:10:35 PM PST, Andrew Dunstan >> wrote: >>> So something like this in Utils.pm: >>> >>> >>> die "Msys targeted perl is unsupported for running TAP tests" if >>>

Re: Nonrandom scanned_pages distorts pg_class.reltuples set by VACUUM

2022-02-17 Thread Robert Haas
On Wed, Feb 16, 2022 at 10:43 PM Peter Geoghegan wrote: > How can you be surprised that I committed 44fa8488? It's essentially > the same patch as the first version, posted November 22 -- almost 3 > months ago. And it's certainly not a big patch (though it is > complicated). Let's back up a

Re: Nonrandom scanned_pages distorts pg_class.reltuples set by VACUUM

2022-02-17 Thread Justin Pryzby
On Wed, Feb 16, 2022 at 07:43:09PM -0800, Peter Geoghegan wrote: > > I also have a hard time making heads or tails out of the commit message of > > 44fa84881ff. It's quite long without being particularly descriptive. The > > commit just changes a lot of things at once, making it hard to precisely

Re: postgres_fdw and skip locked

2022-02-17 Thread Ashutosh Bapat
On Wed, Feb 16, 2022 at 8:38 PM Alexander Pyhalov wrote: > > Ashutosh Bapat писал 2022-02-16 16:40: > > On Mon, Feb 14, 2022 at 4:23 PM Alexander Pyhalov > > wrote: > >> > >> Hi. > >> > >> Now select ... for update ... [skip locked|nowait] options are not > >> pushed down to remote servers. I

Re: [PATCH] Fix out-of-bouds access (src/common/wchar.c)

2022-02-17 Thread Ranier Vilela
Em qui., 17 de fev. de 2022 às 10:18, Daniel Gustafsson escreveu: > > On 17 Feb 2022, at 13:59, Ranier Vilela wrote: > > > > Em qui., 17 de fev. de 2022 às 09:52, Daniel Gustafsson > escreveu: > > > On 17 Feb 2022, at 13:19, Ranier Vilela ranier...@gmail.com>> wrote: >

Re: [PATCH] Fix out-of-bouds access (src/common/wchar.c)

2022-02-17 Thread Daniel Gustafsson
> On 17 Feb 2022, at 13:59, Ranier Vilela wrote: > > Em qui., 17 de fev. de 2022 às 09:52, Daniel Gustafsson > escreveu: > > On 17 Feb 2022, at 13:19, Ranier Vilela > > wrote: > > > 1. One #ifdef with a mistake, the correct is _WIN32 and not

Re: [PATCH] Fix out-of-bouds access (src/common/wchar.c)

2022-02-17 Thread Ranier Vilela
Em qui., 17 de fev. de 2022 às 09:52, Daniel Gustafsson escreveu: > > On 17 Feb 2022, at 13:19, Ranier Vilela wrote: > > > 1. One #ifdef with a mistake, the correct is _WIN32 and not WIN32. > > Can you elaborate on this, we are using WIN32 pretty extensively in the > code: > > $ git grep

Re: [PATCH] Fix out-of-bouds access (src/common/wchar.c)

2022-02-17 Thread Daniel Gustafsson
> On 17 Feb 2022, at 13:19, Ranier Vilela wrote: > 1. One #ifdef with a mistake, the correct is _WIN32 and not WIN32. Can you elaborate on this, we are using WIN32 pretty extensively in the code: $ git grep "if[n]\{0,1\}def WIN32$"|wc -l 511 $ git grep "if[n]\{0,1\}def _WIN32$"|wc -l

[PATCH] Fix possible minor memory leak (src/backend/catalog/heap.c)

2022-02-17 Thread Ranier Vilela
Hi, Per Coverity. Maybe this is a mistake, but, Is it necessary or not to free the memory allocated by nodeToString? If yes, the patch attached fixes this. regards, Ranier Vilela fix_minor_memory_leak_heap.patch Description: Binary data

  1   2   >