Re: make_ctags: use -I option to ignore pg_node_attr macro

2023-02-06 Thread Tatsuo Ishii
> On 2022/10/19 13:25, Tatsuo Ishii wrote: >> Thanks. the v6 patch pushed to master branch. > > Since this commit, make_etags has started failing to generate > tags files with the following error messages, on my MacOS. > > $ src/tools/make_etags >

RE: Deadlock between logrep apply worker and tablesync worker

2023-02-06 Thread houzj.f...@fujitsu.com
On Tuesday, February 7, 2023 12:12 PM Peter Smith wrote: > On Fri, Feb 3, 2023 at 6:58 PM houzj.f...@fujitsu.com > wrote: > > > ... > > > Right, I think that case could be addressed by Tom's patch to some > > > extent but I am thinking we should also try to analyze if we can > > > completely

Re: Perform streaming logical transactions by background workers and parallel apply

2023-02-06 Thread Amit Kapila
On Tue, Feb 7, 2023 at 12:41 PM Masahiko Sawada wrote: > > On Fri, Feb 3, 2023 at 6:44 PM Amit Kapila wrote: > > > We need to think of a predictable > > way to test this path which may not be difficult. But I guess it would > > be better to wait for some feedback from the field about this

RE: [PATCH] Reuse Workers and Replication Slots during Logical Replication

2023-02-06 Thread wangw.f...@fujitsu.com
On Wed, Feb 1, 2023 20:07 PM Melih Mutlu wrote: > Thanks for pointing out this review. I somehow skipped that, sorry. > > Please see attached patches. Thanks for updating the patch set. Here are some comments. 1. In the function ApplyWorkerMain. + /* This is main apply

Re: OpenSSL 3.0.0 vs old branches

2023-02-06 Thread Peter Eisentraut
On 06.02.23 16:56, Andrew Dunstan wrote: I recently moved crake to a new machine running Fedora 36, which has OpenSSL 3.0.0. This causes the SSL tests to fail on branches earlier than release 13, so I propose to backpatch commit f0d2c65f17 to the release 11 and 12 branches. This is not the

Re: Perform streaming logical transactions by background workers and parallel apply

2023-02-06 Thread Masahiko Sawada
On Fri, Feb 3, 2023 at 6:44 PM Amit Kapila wrote: > > On Fri, Feb 3, 2023 at 1:28 PM Masahiko Sawada wrote: > > > > On Fri, Feb 3, 2023 at 12:29 PM houzj.f...@fujitsu.com > > wrote: > > > > > > On Friday, February 3, 2023 11:04 AM Amit Kapila > > > wrote: > > > > > > > > On Thu, Feb 2, 2023

A bug in make_outerjoininfo

2023-02-06 Thread Richard Guo
In cases where we have any clauses between two outer joins, these clauses should be treated as degenerate clauses in the upper OJ, and they may prevent us from re-ordering the two outer joins. Previously we have the flag 'delay_upper_joins' to help avoid the re-ordering in such cases. In

Re: Remove some useless casts to (void *)

2023-02-06 Thread Peter Eisentraut
On 03.02.23 00:59, Corey Huinker wrote: On Thu, Feb 2, 2023 at 5:22 PM Peter Eisentraut > wrote: I have found that in some corners of the code some calls to standard C functions are decorated with casts to (void *) for no reason, and this

Re: Make mesage at end-of-recovery less scary.

2023-02-06 Thread Kyotaro Horiguchi
Thanks! At Fri, 3 Feb 2023 15:16:02 +0100, Alvaro Herrera wrote in > So this patch is now failing because it applies new tests to > 011_crash_recovery.pl, which was removed recently. Can you please move > them elsewhere? I don't find an appropriate file to move to. In the end I created a new

Re: A problem in deconstruct_distribute_oj_quals

2023-02-06 Thread Tom Lane
Richard Guo writes: > In deconstruct_distribute_oj_quals, when we've identified a commutable > left join which provides join clause with flexible semantics, we try to > generate multiple versions of the join clause. Here we have the logic > that puts back any ojrelids that were removed from its

Re: [PATCH] Compression dictionaries for JSONB

2023-02-06 Thread Nikita Malakhov
Hi, On updating dictionary - >You cannot "just" refresh a dictionary used once to compress an >object, because you need it to decompress the object too. and when you have many - updating an existing dictionary requires going through all objects compressed with it in the whole database. It's a

Re: Time delayed LR (WAS Re: logical replication restrictions)

2023-02-06 Thread Amit Kapila
On Tue, Feb 7, 2023 at 10:42 AM Peter Smith wrote: > > On Tue, Feb 7, 2023 at 4:02 PM Amit Kapila wrote: > > > > On Tue, Feb 7, 2023 at 10:07 AM Kyotaro Horiguchi > > wrote: > > > > > > At Tue, 7 Feb 2023 09:10:01 +0530, Amit Kapila > > > wrote in > > > > On Tue, Feb 7, 2023 at 6:03 AM Peter

Re: Time delayed LR (WAS Re: logical replication restrictions)

2023-02-06 Thread Amit Kapila
On Tue, Feb 7, 2023 at 10:13 AM Kyotaro Horiguchi wrote: > > At Mon, 6 Feb 2023 13:10:01 +, "Takamichi Osumi (Fujitsu)" > wrote in > > The attached patch v29 has included your changes. > > catalogs.sgml > > + > + The minimum delay (ms) for applying changes. > + > > I think

Re: Time delayed LR (WAS Re: logical replication restrictions)

2023-02-06 Thread Peter Smith
On Tue, Feb 7, 2023 at 4:02 PM Amit Kapila wrote: > > On Tue, Feb 7, 2023 at 10:07 AM Kyotaro Horiguchi > wrote: > > > > At Tue, 7 Feb 2023 09:10:01 +0530, Amit Kapila > > wrote in > > > On Tue, Feb 7, 2023 at 6:03 AM Peter Smith wrote: > > > > 5b. > > > > Since there are no translator

Re: Time delayed LR (WAS Re: logical replication restrictions)

2023-02-06 Thread Amit Kapila
On Tue, Feb 7, 2023 at 10:07 AM Kyotaro Horiguchi wrote: > > At Tue, 7 Feb 2023 09:10:01 +0530, Amit Kapila > wrote in > > On Tue, Feb 7, 2023 at 6:03 AM Peter Smith wrote: > > > 5b. > > > Since there are no translator considerations here why not write the > > > second error like: > > > > > >

Re: Where is the logig to create a table file?

2023-02-06 Thread Kyotaro Horiguchi
At Fri, 3 Feb 2023 13:44:46 +0400, Pavel Borisov wrote in > Hi, Jack > > On Fri, 3 Feb 2023 at 13:19, jack...@gmail.com wrote: > > > > When I use 'create table t(a int);'; suppose that this table t's oid is > > 1200, > > then postgres will create a file named 1200 in the $PGDATA/base, So

Re: Exit walsender before confirming remote flush in logical replication

2023-02-06 Thread Andres Freund
Hi, On 2023-02-07 09:00:13 +0530, Amit Kapila wrote: > On Tue, Feb 7, 2023 at 2:04 AM Andres Freund wrote: > > How about we make it an option in START_REPLICATION? Delayed logical rep can > > toggle that on by default. > Works for me. So, when this option is set in START_REPLICATION > message,

Re: In-placre persistance change of a relation

2023-02-06 Thread Kyotaro Horiguchi
Thank you for the comment! At Fri, 3 Feb 2023 08:42:52 +0100, Heikki Linnakangas wrote in > I want to call out this part of this patch: > > > Also this allows for the cleanup of files left behind in the crash of > > the transaction that created it. > > This is interesting to a lot wider

Re: tests against running server occasionally fail, postgres_fdw & tenk1

2023-02-06 Thread Andres Freund
Hi, On 2023-02-06 19:29:46 -0800, Andres Freund wrote: > There's something off. Isolationtester's control connection emits *loads* of > invalidation messages: > 2023-02-06 19:29:06.430 PST [2125297][client > backend][6/0:121864][isolation/receipt-report/control connection] LOG: > previously

Re: run pgindent on a regular basis / scripted manner

2023-02-06 Thread Noah Misch
On Mon, Feb 06, 2023 at 06:17:02PM +0100, Peter Eisentraut wrote: > Also, pgindent takes tens of seconds to run, so hooking that into the git > push process would slow this down quite a bit. The pre-receive hook would do a full pgindent when you change typedefs.list. Otherwise, it would reindent

Re: Time delayed LR (WAS Re: logical replication restrictions)

2023-02-06 Thread Kyotaro Horiguchi
Thanks! At Mon, 6 Feb 2023 13:10:01 +, "Takamichi Osumi (Fujitsu)" wrote in > The attached patch v29 has included your changes. catalogs.sgml + + The minimum delay (ms) for applying changes. + I think we don't use unit symbols that way. Namely I think we would write it

Re: heapgettup refactoring

2023-02-06 Thread David Rowley
On Fri, 3 Feb 2023 at 15:26, David Rowley wrote: > I've pushed all but the final 2 patches now. I just pushed the final patch in the series. I held back on moving the setting of rs_inited back into the heapgettup_initial_block() helper function as I wondered if we should even keep that field.

Re: Time delayed LR (WAS Re: logical replication restrictions)

2023-02-06 Thread Kyotaro Horiguchi
At Tue, 7 Feb 2023 09:10:01 +0530, Amit Kapila wrote in > On Tue, Feb 7, 2023 at 6:03 AM Peter Smith wrote: > > 5b. > > Since there are no translator considerations here why not write the > > second error like: > > > > errmsg("%d ms is outside the valid range for parameter > >

Re: Deadlock between logrep apply worker and tablesync worker

2023-02-06 Thread Peter Smith
On Fri, Feb 3, 2023 at 6:58 PM houzj.f...@fujitsu.com wrote: > ... > > Right, I think that case could be addressed by Tom's patch to some extent > > but > > I am thinking we should also try to analyze if we can completely avoid the > > need > > to remove origins from both processes. One idea

Re: Time delayed LR (WAS Re: logical replication restrictions)

2023-02-06 Thread Amit Kapila
On Tue, Feb 7, 2023 at 6:03 AM Peter Smith wrote: > > 5. > +defGetMinApplyDelay(DefElem *def) > +{ > + char*input_string; > + int result; > + const char *hintmsg; > + > + input_string = defGetString(def); > + > + /* > + * Parse given string as parameter which has millisecond unit > + */ > +

Re: Exit walsender before confirming remote flush in logical replication

2023-02-06 Thread Amit Kapila
On Tue, Feb 7, 2023 at 2:04 AM Andres Freund wrote: > > On 2023-02-06 12:23:54 +0530, Amit Kapila wrote: > > On Mon, Feb 6, 2023 at 10:33 AM Andres Freund wrote: > > > Smart shutdown is practically unusable. I don't think it makes sense to > > > tie behavior of walsender to it in any way. > > >

Re: tests against running server occasionally fail, postgres_fdw & tenk1

2023-02-06 Thread Andres Freund
Hi, On 2023-02-06 17:53:00 -0800, Andres Freund wrote: > WRT the fdw_retry_check: I wonder if we should increase the log level of > a) pgfdw_inval_callback deciding to disconnect > b) ReceiveSharedInvalidMessages() deciding to reset > > to DEBUG1, at least temporarily? > > Alternatively we could

Re: Perform streaming logical transactions by background workers and parallel apply

2023-02-06 Thread Amit Kapila
On Mon, Feb 6, 2023 at 3:43 PM houzj.f...@fujitsu.com wrote: > > while reading the code, I noticed that in pa_send_data() we set wait event > to WAIT_EVENT_LOGICAL_PARALLEL_APPLY_STATE_CHANGE while sending the > message to the queue. Because this state is used in multiple places, user > might >

A problem in deconstruct_distribute_oj_quals

2023-02-06 Thread Richard Guo
In deconstruct_distribute_oj_quals, when we've identified a commutable left join which provides join clause with flexible semantics, we try to generate multiple versions of the join clause. Here we have the logic that puts back any ojrelids that were removed from its min_righthand. /*

RE: Time delayed LR (WAS Re: logical replication restrictions)

2023-02-06 Thread Hayato Kuroda (Fujitsu)
Dear Peter, Thank you for reviewing! PSA new version. > == > Commit Message > > 1. > Discussion: > https://postgr.es/m/CAB-JLwYOYwL=XTyAXKiH5CtM_Vm8KjKh7aaitCKvmCh4r > zr...@mail.gmail.com > > tmp > > ~ > > What's that "tmp" doing there? A typo? Removed. It was a typo. I used `git

RE: [PATCH] Reuse Workers and Replication Slots during Logical Replication

2023-02-06 Thread shiy.f...@fujitsu.com
On Thu, Feb 2, 2023 11:48 AM shveta malik wrote: > > On Wed, Feb 1, 2023 at 5:42 PM Melih Mutlu > wrote: > > > > Hi Shveta, > > > > shveta malik , 1 Şub 2023 Çar, 15:01 tarihinde > şunu yazdı: > >> > >> On Wed, Feb 1, 2023 at 5:05 PM Melih Mutlu > wrote: > >> 2) I found a crash in the

RE: Perform streaming logical transactions by background workers and parallel apply

2023-02-06 Thread Hayato Kuroda (Fujitsu)
Dear Hou, > I think PARALLEL_APPLY_MAIN waits for two kinds of event: 1) wait for new > message from the queue 2) wait for the partial file state to be set. So, I > think introducing a new general event for them is better and it is also > consistent with the WAIT_EVENT_LOGICAL_APPLY_MAIN which is

Re: tests against running server occasionally fail, postgres_fdw & tenk1

2023-02-06 Thread Andres Freund
Hi, On 2022-12-08 16:36:07 -0800, Andres Freund wrote: > On 2022-12-08 16:15:11 -0800, Andres Freund wrote: > > Unfortunately cfbot shows that that doesn't work entirely reliably. > > > > The most frequent case is postgres_fdw, which somewhat regularly fails with > > a > > regression.diff like

Re: Time delayed LR (WAS Re: logical replication restrictions)

2023-02-06 Thread Peter Smith
Here are my review comments for v29-0001. == Commit Message 1. Discussion: https://postgr.es/m/CAB-JLwYOYwL=xtyaxkih5ctm_vm8kjkh7aaitckvmch4rzr...@mail.gmail.com tmp ~ What's that "tmp" doing there? A typo? == doc/src/sgml/catalogs.sgml 2. + + + subminapplydelay

Re: Generating code for query jumbling through gen_node_support.pl

2023-02-06 Thread Michael Paquier
On Sun, Feb 05, 2023 at 07:40:57PM +0900, Michael Paquier wrote: > Comments or objections are welcome, of course. Done this part. > (FWIW, I'd like to think that there is an argument to normalize the > A_Const nodes for a portion of the DDL queries, by ignoring their > values in the query

Re: lockup in parallel hash join on dikkop (freebsd 14.0-current)

2023-02-06 Thread Thomas Munro
On Tue, Feb 7, 2023 at 1:06 PM Tomas Vondra wrote: > On 2/7/23 00:48, Thomas Munro wrote: > > On Tue, Feb 7, 2023 at 12:46 PM Tomas Vondra > > wrote: > >> No, I left the workload as it was for the first lockup, so `make check` > >> runs everything as is up until the "join" test suite. > > > >

Re: lockup in parallel hash join on dikkop (freebsd 14.0-current)

2023-02-06 Thread Tomas Vondra
On 2/7/23 00:48, Thomas Munro wrote: > On Tue, Feb 7, 2023 at 12:46 PM Tomas Vondra > wrote: >> No, I left the workload as it was for the first lockup, so `make check` >> runs everything as is up until the "join" test suite. > > Wait, shouldn't that be join_hash? No, because join_hash does not

Re: lockup in parallel hash join on dikkop (freebsd 14.0-current)

2023-02-06 Thread Thomas Munro
On Tue, Feb 7, 2023 at 12:46 PM Tomas Vondra wrote: > No, I left the workload as it was for the first lockup, so `make check` > runs everything as is up until the "join" test suite. Wait, shouldn't that be join_hash?

Re: lockup in parallel hash join on dikkop (freebsd 14.0-current)

2023-02-06 Thread Tomas Vondra
On 2/6/23 20:20, Andres Freund wrote: > Hi, > > On 2023-02-06 19:51:19 +0100, Tomas Vondra wrote: >>> No. The only thing the machine is doing is >>> >>> while /usr/bin/true; do >>> make check >>> done >>> >>> I can't reduce the workload further, because the "join" test is in a >>>

Re: gokiburi versus the back branches

2023-02-06 Thread Tom Lane
Michael Paquier writes: > On Mon, Feb 06, 2023 at 06:27:50PM -0500, Tom Lane wrote: >> Is it time to back-patch that commit? > Yes, this is my intention as of this message from last week, once this > week's release is tagged: > https://www.postgresql.org/message-id/y9smhxo51hrxa...@paquier.xyz

Re: pg_upgrade test failure

2023-02-06 Thread Andres Freund
On 2023-02-06 14:14:22 -0800, Andres Freund wrote: > On 2023-02-07 11:03:18 +1300, Thomas Munro wrote: > > What I see is that there were 1254 FreeBSD tasks run in that window, of > > which 163 failed, and (more interestingly) 111 of those failures succeeded > > on every other platform. And

Re: Temporary tables versus wraparound... again

2023-02-06 Thread Andres Freund
Hi, On 2023-02-05 15:30:57 -0800, Andres Freund wrote: > Hm, I suspect the problem is that we didn't shut down the server due to > the error, so the log file was changing while cirrus was trying to > upload. Pushed a fix for that. Greetings, Andres Freund

Re: gokiburi versus the back branches

2023-02-06 Thread Michael Paquier
On Mon, Feb 06, 2023 at 06:27:50PM -0500, Tom Lane wrote: > Is it time to back-patch that commit? The alternative would be > to run the animal with an ASLR-disabling environment variable. > On the whole I think testing that f3e78069d works is more > useful than working around lack of it. Yes,

gokiburi versus the back branches

2023-02-06 Thread Tom Lane
I notice that Michael's new BF animal gokiburi is failing in all the pre-v15 branches, though it's fine in v15 and HEAD. It's evidently dying from ASLR effects because it's trying to build with EXEC_BACKEND on Linux: there's lots of 2023-02-06 06:07:02.131 GMT [1503972] FATAL: could not reattach

Re: pg_upgrade test failure

2023-02-06 Thread Andres Freund
Hi, On 2023-02-07 11:03:18 +1300, Thomas Munro wrote: > On Tue, Feb 7, 2023 at 10:57 AM Andres Freund wrote: > > On February 6, 2023 1:51:20 PM PST, Thomas Munro > > wrote: > > >Next up: the new "running" tests, spuriously failing around 8.8% of CI > > >builds on FreeBSD. I'll go and ping

Re: pg_upgrade test failure

2023-02-06 Thread Thomas Munro
On Tue, Feb 7, 2023 at 11:03 AM Thomas Munro wrote: > On Tue, Feb 7, 2023 at 10:57 AM Andres Freund wrote: > > On February 6, 2023 1:51:20 PM PST, Thomas Munro > > wrote: > > >Next up: the new "running" tests, spuriously failing around 8.8% of CI > > >builds on FreeBSD. I'll go and ping that

Re: pg_upgrade test failure

2023-02-06 Thread Thomas Munro
On Tue, Feb 7, 2023 at 10:57 AM Andres Freund wrote: > On February 6, 2023 1:51:20 PM PST, Thomas Munro > wrote: > >Next up: the new "running" tests, spuriously failing around 8.8% of CI > >builds on FreeBSD. I'll go and ping that thread... > > Is that rate unchanged? I thought I fixed the

Re: OpenSSL 3.0.0 vs old branches

2023-02-06 Thread Tom Lane
Andrew Dunstan writes: > On 2023-02-06 Mo 11:13, Tom Lane wrote: >> So presumably, changing this test would break it for OpenSSL 0.9.8, >> which is still nominally supported in those branches. On the other >> hand, this test isn't run by default, so users would likely never >> notice anyway. >

Re: pg_upgrade test failure

2023-02-06 Thread Andres Freund
Hi, On February 6, 2023 1:51:20 PM PST, Thomas Munro wrote: >Next up: the new "running" tests, spuriously failing around 8.8% of CI >builds on FreeBSD. I'll go and ping that thread... Is that rate unchanged? I thought I fixed the main issue last week? Greetings, Andres -- Sent from my

Re: pg_upgrade test failure

2023-02-06 Thread Thomas Munro
On Wed, Feb 1, 2023 at 2:44 PM Thomas Munro wrote: > OK, I pushed that. Third time lucky? I pulled down logs for a week of Windows CI, just over ~1k builds. The failure rate was a few per day before, but there are no failures like that after that went in. There are logs that contain the

Re: run pgindent on a regular basis / scripted manner

2023-02-06 Thread Tom Lane
Andres Freund writes: > On 2023-02-06 18:17:02 +0100, Peter Eisentraut wrote: >> First, as a matter of principle, it would introduce another level of >> gatekeeping power. Right now, the committers are as a group in charge of >> what gets into the tree. Adding commit hooks that are installed >>

Re: run pgindent on a regular basis / scripted manner

2023-02-06 Thread Andres Freund
Hi, On 2023-02-06 18:17:02 +0100, Peter Eisentraut wrote: > First, as a matter of principle, it would introduce another level of > gatekeeping power. Right now, the committers are as a group in charge of > what gets into the tree. Adding commit hooks that are installed > somewhere(?) by

Re: OpenSSL 3.0.0 vs old branches

2023-02-06 Thread Andrew Dunstan
On 2023-02-06 Mo 11:13, Tom Lane wrote: Andrew Dunstan writes: I recently moved crake to a new machine running Fedora 36, which has OpenSSL 3.0.0. This causes the SSL tests to fail on branches earlier than release 13, so I propose to backpatch commit f0d2c65f17 to the release 11 and 12

Re: Allow tailoring of ICU locales with custom rules

2023-02-06 Thread Peter Eisentraut
On 04.02.23 14:41, Daniel Verite wrote: However it still leaves "daticurules" empty in the destination db, because of an actual bug in the current patch. Looking at createdb() in commands.c, it creates this variable: @@ -711,6 +714,7 @@ createdb(ParseState *pstate, const CreatedbStmt *stmt)

Re: run pgindent on a regular basis / scripted manner

2023-02-06 Thread Andrew Dunstan
On 2023-02-06 Mo 12:17, Peter Eisentraut wrote: On 02.02.23 07:40, Tom Lane wrote: Noah Misch writes: Regarding the concern about a pre-receive hook blocking an emergency push, the hook could approve every push where a string like "pgindent: no" appears in a commit message within the push. 

Re: BUG: Postgres 14 + vacuum_defer_cleanup_age + FOR UPDATE + UPDATE

2023-02-06 Thread Andres Freund
Hi, On 2023-02-04 11:10:55 -0800, Peter Geoghegan wrote: > On Sat, Feb 4, 2023 at 2:57 AM Andres Freund wrote: > > Is there a good way to make breakage in the page recycling mechanism > > visible with gist? I guess to see corruption, I'd have to halt a scan > > before a page is visited with gdb,

Re: Exit walsender before confirming remote flush in logical replication

2023-02-06 Thread Andres Freund
Hi, On 2023-02-06 12:23:54 +0530, Amit Kapila wrote: > On Mon, Feb 6, 2023 at 10:33 AM Andres Freund wrote: > > Smart shutdown is practically unusable. I don't think it makes sense to tie > > behavior of walsender to it in any way. > > > > So, we have the following options: (a) do nothing for

Re: SLRUs in the main buffer pool - Page Header definitions

2023-02-06 Thread Andres Freund
Hi, > From 098f37c0a17fc32a94bff43817414e01fcfb234f Mon Sep 17 00:00:00 2001 > From: Rishu Bagga > Date: Thu, 15 Sep 2022 00:55:25 + > Subject: [PATCH] slru to buffercache + page headers + upgrade > > --- > contrib/amcheck/verify_nbtree.c |2 +- > [...] > 65 files changed,

Re: Pluggable toaster

2023-02-06 Thread Nikita Malakhov
Hi, It'll be great to see more opinions on the approach as a whole. >The problem that I see being raised here, is that there was little >discussion and no observed community consensus about the design of >this complex feature *before* this patch with high complexity was >provided. >The next

Re: pglz compression performance, take two

2023-02-06 Thread Andrey Borodin
On Mon, Feb 6, 2023 at 11:57 AM Tomas Vondra wrote: > > I wonder what that means for the patch. I haven't investigated this at > all, but it seems as if the optimization means we fail to find a match, > producing a tad larger output. That may be still be a good tradeoff, as > long as the output

Re: Pluggable toaster

2023-02-06 Thread Matthias van de Meent
On Mon, 6 Feb 2023 at 15:38, Aleksander Alekseev wrote: > I would like to point out however that there were several other pieces > of feedback that could have been missed: > > * No one wants to see this as an extension. This was my original > proposal (adding ZSON to /contrib/) and it was

Re: Pluggable toaster

2023-02-06 Thread Matthias van de Meent
On Mon, 6 Feb 2023 at 20:24, Andres Freund wrote: > > Hi, > > On 2023-02-06 16:38:01 +0300, Nikita Malakhov wrote: > > So we decided to reduce changes in the core to the minimum necessary > > to make it available through the hooks, because the hooks part is very > > lightweight and simple to keep

Re: pglz compression performance, take two

2023-02-06 Thread Tomas Vondra
On 2/6/23 03:00, Andrey Borodin wrote: > On Sun, Feb 5, 2023 at 5:51 PM Tomas Vondra > wrote: >> >> On 2/5/23 19:36, Andrey Borodin wrote: >>> On Fri, Jan 6, 2023 at 10:02 PM Andrey Borodin >>> wrote: Hello! Please find attached v8. >>> >>> I got some interesting feedback from some

Re: Non-superuser subscription owners

2023-02-06 Thread Robert Haas
On Mon, Feb 6, 2023 at 2:18 PM Andres Freund wrote: > It's decidedly not great, yes. I don't know if it's quite a CVE type issue, > after all, the same is true for any other type of query the superuser > executes. But at the very least the documentation needs to be better, with a > big red box

Re: [PATCH] Compression dictionaries for JSONB

2023-02-06 Thread Andres Freund
Hi, On 2023-02-06 16:16:41 +0100, Matthias van de Meent wrote: > On Mon, 6 Feb 2023 at 15:03, Aleksander Alekseev > wrote: > > > > Hi, > > > > I see your point regarding the fact that creating dictionaries on a > > training set is too beneficial to neglect it. Can't argue with this. > > > > What

Re: Pluggable toaster

2023-02-06 Thread Andres Freund
Hi, On 2023-02-06 16:38:01 +0300, Nikita Malakhov wrote: > So we decided to reduce changes in the core to the minimum necessary > to make it available through the hooks, because the hooks part is very > lightweight and simple to keep rebasing onto the vanilla core. At least I don't think we

Re: GUCs to control abbreviated sort keys

2023-02-06 Thread Thomas Munro
On Fri, Jan 27, 2023 at 12:29 PM Jeff Davis wrote: > I am using these GUCs for testing the various collation paths in my > collation refactoring branch. Speaking of testing, has anyone ever tried porting Tom's random test program[1] to ICU? [1]

Re: lockup in parallel hash join on dikkop (freebsd 14.0-current)

2023-02-06 Thread Andres Freund
Hi, On 2023-02-06 19:51:19 +0100, Tomas Vondra wrote: > > No. The only thing the machine is doing is > > > > while /usr/bin/true; do > > make check > > done > > > > I can't reduce the workload further, because the "join" test is in a > > separate parallel group (I cut down

Re: Non-superuser subscription owners

2023-02-06 Thread Andres Freund
Hi, On 2023-02-06 14:07:39 -0500, Robert Haas wrote: > On Fri, Feb 3, 2023 at 3:47 AM Andres Freund wrote: > > On 2023-02-02 09:28:03 -0500, Robert Haas wrote: > > > I don't know what you mean by this. DML doesn't confer privileges. If > > > code gets executed and runs with the replication

Re: Non-superuser subscription owners

2023-02-06 Thread Robert Haas
On Fri, Feb 3, 2023 at 3:47 AM Andres Freund wrote: > On 2023-02-02 09:28:03 -0500, Robert Haas wrote: > > I don't know what you mean by this. DML doesn't confer privileges. If > > code gets executed and runs with the replication user's credentials, > > that could lead to privilege escalation,

Re: undersized unions

2023-02-06 Thread Robert Haas
On Mon, Feb 6, 2023 at 1:28 PM Andres Freund wrote: > Perhaps something like Yeah, that'd work. You'd want a big ol' warning comment here: > typedef struct NumericData > { > int32 vl_len_;/* varlena header (do not > touch directly!) */ > NumericBase

Re: lockup in parallel hash join on dikkop (freebsd 14.0-current)

2023-02-06 Thread Tomas Vondra
On 1/29/23 19:08, Tomas Vondra wrote: > > > On 1/29/23 18:53, Andres Freund wrote: >> Hi, >> >> On 2023-01-29 18:39:05 +0100, Tomas Vondra wrote: >>> Will do, but I'll wait for another lockup to see how frequent it >>> actually is. I'm now at ~90 runs total, and it didn't happen again yet. >>>

Re: GUCs to control abbreviated sort keys

2023-02-06 Thread Peter Geoghegan
On Fri, Jan 27, 2023 at 12:34 PM Jeff Davis wrote: > It is non-deterministic, but I tried with two generated files, and got > similar results. Jeff and I coordinated off-list. It turned out that the nondeterministic nature of the program to generate test data was behind my initial inability to

Re: undersized unions

2023-02-06 Thread Andres Freund
Hi, On 2023-02-06 11:55:40 -0500, Tom Lane wrote: > I am, however, very dubious that Andres is correct that there's a > problem here. Given that two of the variants of union NumericChoice > are structs ending with a flexible array, any compiler that thinks > it knows the size of the union

Re: undersized unions

2023-02-06 Thread Andres Freund
Hi On 2023-02-06 11:42:57 -0500, Robert Haas wrote: > On Sun, Feb 5, 2023 at 6:28 AM Andres Freund wrote: > > On the other hand, it also just seems risky from a code writing > > perspective. It's not immediate obvious that it'd be unsafe to create an > > on-stack Numeric by assigning *ptr. But

2023-02-09 release announcement draft

2023-02-06 Thread Jonathan S. Katz
Hi, Attached is a draft of the announcement for the 2023-02-09 update release. Please review and provide corrections, notable omissions, and suggestions no later than 2023-02-09 0:00 AoE. Thanks! Jonathan The PostgreSQL Global Development Group has released an update to all supported

Re: [PATCH] Add pretty-printed XML output option

2023-02-06 Thread Jim Jones
On 06.02.23 17:23, Tom Lane wrote: Your patch needs to also update expected/xml_1.out to match the output the test produces when run without --with-libxml. Thanks a lot! It seems to do the trick. xml_1.out updated in v4. Best, Jim From ced9fccddc033de98709a6e93dc6530ce68149db Mon Sep 17

Re: run pgindent on a regular basis / scripted manner

2023-02-06 Thread Andrew Dunstan
On 2023-02-06 Mo 12:03, Andrew Dunstan wrote: On 2023-02-06 Mo 10:36, Robert Haas wrote: On Mon, Feb 6, 2023 at 10:21 AM Andrew Dunstan wrote: Good suggestions. 1 and 3 seem fairly straightforward. I'll start on those, and look into 2. Thanks! Here's a quick patch for 1 and 3. Would

Re: Logical Replication Custom Column Expression

2023-02-06 Thread Stavros Koureas
>> And yes, probably you need to change the way you reply to email on >> this list. Top-posting is generally avoided. See >> https://wiki.postgresql.org/wiki/Mailing_Lists. >Thanks for bringing this into the discussion :) Thinking these days more about this topic, subscriber name is not a bad

Re: Understanding years part of Interval

2023-02-06 Thread Marcos Pegoraro
Em seg., 6 de fev. de 2023 às 10:59, Erik Wienhold escreveu: > > On 06/02/2023 12:20 CET Marcos Pegoraro wrote: > > > > I was just playing with some random timestamps for a week, for a month, > > for a year ... > > > > select distinct current_date+((random()::numeric)||'month')::interval > from

Re: run pgindent on a regular basis / scripted manner

2023-02-06 Thread Peter Eisentraut
On 02.02.23 07:40, Tom Lane wrote: Noah Misch writes: Regarding the concern about a pre-receive hook blocking an emergency push, the hook could approve every push where a string like "pgindent: no" appears in a commit message within the push. You'd still want to make the tree clean sometime

Re: run pgindent on a regular basis / scripted manner

2023-02-06 Thread Andrew Dunstan
On 2023-02-06 Mo 10:36, Robert Haas wrote: On Mon, Feb 6, 2023 at 10:21 AM Andrew Dunstan wrote: Good suggestions. 1 and 3 seem fairly straightforward. I'll start on those, and look into 2. Thanks! Here's a quick patch for 1 and 3. Would also need to adjust the docco. cheers andrew

Re: undersized unions

2023-02-06 Thread Tom Lane
Robert Haas writes: > I do not really know exactly how to do that, though. Our usual pattern > is to just have a struct and end with a variable-length array, or > alternatively add a comment says "other stuff follows!" at the end of > the struct definition, without doing anything that C knows

Re: undersized unions

2023-02-06 Thread Robert Haas
On Sun, Feb 5, 2023 at 6:28 AM Andres Freund wrote: > On the other hand, it also just seems risky from a code writing perspective. > It's not immediate obvious that it'd be unsafe to create an on-stack Numeric > by assigning *ptr. But it is. Well, I think that is pretty obvious: we have lots

Re: Parallelize correlated subqueries that execute within each worker

2023-02-06 Thread Antonin Houska
James Coleman wrote: > On Sat, Jan 21, 2023 at 10:07 PM James Coleman wrote: > > ... > > While working through Tomas's comment about a conditional in the > > max_parallel_hazard_waker being guaranteed true I realized that in the > > current version of the patch the safe_param_ids tracking in >

Re: [PATCH] Add pretty-printed XML output option

2023-02-06 Thread Tom Lane
Jim Jones writes: > The cfbot on "Windows - Server 2019, VS 2019 - Meson & ninja" is failing > the regression tests with the error: > ERROR:  unsupported XML feature > DETAIL:  This functionality requires the server to be built with libxml > support. > Is there anything I can do to enable

Re: when the startup process doesn't (logging startup delays)

2023-02-06 Thread Robert Haas
On Mon, Feb 6, 2023 at 11:15 AM Tom Lane wrote: > Robert Haas writes: > > On Mon, Feb 6, 2023 at 11:07 AM Tom Lane wrote: > >> Umm ... is this really the sort of patch to be committing on a > >> release wrap day? > > > Oh, shoot, I wasn't thinking about that. Would you like me to revert > > it

Re: postgres_fdw, dblink, and CREATE SUBSCRIPTION security

2023-02-06 Thread Robert Haas
On Wed, Feb 1, 2023 at 3:37 PM Jacob Champion wrote: > > I'm not an expert on this stuff, but to me that feels like a weak and > > fuzzy concept. If the client is going to tell the server something, > > I'd much rather have it say something like "i'm proxying a request > > from my local user

Re: [PATCH] Add pretty-printed XML output option

2023-02-06 Thread Jim Jones
Hi, The cfbot on "Windows - Server 2019, VS 2019 - Meson & ninja" is failing the regression tests with the error: ERROR:  unsupported XML feature DETAIL:  This functionality requires the server to be built with libxml support. Is there anything I can do to enable libxml to run my

Re: make_ctags: use -I option to ignore pg_node_attr macro

2023-02-06 Thread Fujii Masao
On 2022/10/19 13:25, Tatsuo Ishii wrote: Thanks. the v6 patch pushed to master branch. Since this commit, make_etags has started failing to generate tags files with the following error messages, on my MacOS. $ src/tools/make_etags

Re: Where is the filter?

2023-02-06 Thread Robert Haas
On Sat, Feb 4, 2023 at 11:29 PM jack...@gmail.com wrote: > When I use 'select * from t where a = 1'; And I debug to find where the 'a = > 1' is used, > when I arrive ExecScan in src/backend/executor/execScan.c, line 158, where > this 'a = 1' is > stored in? It depends somewhat on what query

Re: when the startup process doesn't (logging startup delays)

2023-02-06 Thread Tom Lane
Robert Haas writes: > On Mon, Feb 6, 2023 at 11:07 AM Tom Lane wrote: >> Umm ... is this really the sort of patch to be committing on a >> release wrap day? > Oh, shoot, I wasn't thinking about that. Would you like me to revert > it in v15 for now? Yeah, seems like the safest course. I

Re: OpenSSL 3.0.0 vs old branches

2023-02-06 Thread Tom Lane
Andrew Dunstan writes: > I recently moved crake to a new machine running Fedora 36, which has > OpenSSL 3.0.0. This causes the SSL tests to fail on branches earlier > than release 13, so I propose to backpatch commit f0d2c65f17 to the > release 11 and 12 branches. Hmm ... according to that

Re: when the startup process doesn't (logging startup delays)

2023-02-06 Thread Robert Haas
On Mon, Feb 6, 2023 at 11:07 AM Tom Lane wrote: > Umm ... is this really the sort of patch to be committing on a > release wrap day? Oh, shoot, I wasn't thinking about that. Would you like me to revert it in v15 for now? -- Robert Haas EDB: http://www.enterprisedb.com

Re: when the startup process doesn't (logging startup delays)

2023-02-06 Thread Tom Lane
Robert Haas writes: > On Thu, Feb 2, 2023 at 11:22 PM Bharath Rupireddy > wrote: >> I took the v4 patch, added some comments and attached it as the v7 >> patch here. Please find it. > Committed and back-patched to v15. Umm ... is this really the sort of patch to be committing on a release wrap

Re: when the startup process doesn't (logging startup delays)

2023-02-06 Thread Robert Haas
On Thu, Feb 2, 2023 at 11:22 PM Bharath Rupireddy wrote: > I took the v4 patch, added some comments and attached it as the v7 > patch here. Please find it. Committed and back-patched to v15. -- Robert Haas EDB: http://www.enterprisedb.com

OpenSSL 3.0.0 vs old branches

2023-02-06 Thread Andrew Dunstan
I recently moved crake to a new machine running Fedora 36, which has OpenSSL 3.0.0. This causes the SSL tests to fail on branches earlier than release 13, so I propose to backpatch commit f0d2c65f17 to the release 11 and 12 branches. cheers andrew -- Andrew Dunstan

Re: run pgindent on a regular basis / scripted manner

2023-02-06 Thread Robert Haas
On Mon, Feb 6, 2023 at 10:21 AM Andrew Dunstan wrote: > Good suggestions. 1 and 3 seem fairly straightforward. I'll start on those, > and look into 2. Thanks! -- Robert Haas EDB: http://www.enterprisedb.com

Re: run pgindent on a regular basis / scripted manner

2023-02-06 Thread Robert Haas
On Mon, Feb 6, 2023 at 10:16 AM Tom Lane wrote: > I'll just note that adding features like those to a Perl script > is going to be a ton easier than doing it inside pg_bsd_indent. +1. -- Robert Haas EDB: http://www.enterprisedb.com

Re: pgbench: using prepared BEGIN statement in a pipeline could cause an error

2023-02-06 Thread Tom Lane
Alvaro Herrera writes: > I'm writing my own patch for this problem. While playing around with > it, I noticed this: > struct Command { > /* size: 2168, cachelines: 34, members: 11 */ > /* sum members: 2164, holes: 1, sum holes: 4 */ > /* last cacheline: 56 bytes */ > }; I think the

  1   2   >