pg_dump/pg_restore: Fix stdin/stdout handling of custom format on Win32

2022-12-18 Thread Daniel Watzinger
Well, this is embarassing. Sorry for the inconvenience. Some part of my company's network infrastruture must have mangled the attachment. Both mails were sent using a combination of git format-patch and git send-email. However, as this is my first foray into this email-based workflow, I won't ru

Standard REGEX functions

2022-12-18 Thread Vik Fearing
The standard uses XQuery regular expressions, which I believe are subtly different from ours. Because of that, I had been hesitant to add some standard functions to our library such as . While looking at commit 6424337073589476303b10f6d7cc74f501b8d9d7 from last year (which will come up soon f

Re: Standard REGEX functions

2022-12-18 Thread Tom Lane
Vik Fearing writes: > Are there any objections to me writing a patch to add SQL Standard > regular expression functions even though they call for XQuery and we > would use our own language? Yes. If we provide spec-defined syntax it should have spec-defined behavior. I really don't see the val

Re: Error-safe user functions

2022-12-18 Thread Andrew Dunstan
On 2022-12-14 We 17:37, Tom Lane wrote: > Andrew Dunstan writes: >> Thanks, I have been looking at jsonpath, but I'm not quite sure how to >> get the escontext argument to the yyerror calls in jsonath_scan.l. Maybe >> I need to specify a lex-param setting? > You want a parse-param option in jsonp

Re: Transaction timeout

2022-12-18 Thread Andrey Borodin
On Wed, Dec 7, 2022 at 1:30 PM Andrey Borodin wrote: > I hope to address other feedback on the weekend. Andres, here's my progress on working with your review notes. > > @@ -3277,6 +3282,7 @@ ProcessInterrupts(void) > >*/ > > lock_timeout_occurred = get_timeout_indi

Re: Standard REGEX functions

2022-12-18 Thread Vik Fearing
On 12/18/22 15:24, Tom Lane wrote: Vik Fearing writes: Are there any objections to me writing a patch to add SQL Standard regular expression functions even though they call for XQuery and we would use our own language? Yes. If we provide spec-defined syntax it should have spec-defined behavi

Re: allow granting CLUSTER, REFRESH MATERIALIZED VIEW, and REINDEX

2022-12-18 Thread Nathan Bossart
Here is a new version of the patch. Besides some cleanup, I added an index on reltoastrelid for the toast-to-main-relation lookup. Before I bother adjusting the tests and documentation, I'm curious to hear thoughts on whether this seems like a viable approach. On Sat, Dec 17, 2022 at 04:39:29AM

Re: Standard REGEX functions

2022-12-18 Thread Tom Lane
Vik Fearing writes: > I don't suppose project policy would allow us to use an external > library. I assume there is one out there that implements XQuery regular > expressions. Probably, but is it in C and does it have a compatible license? The bigger picture here is that our track record with

Re: wake up logical workers after ALTER SUBSCRIPTION

2022-12-18 Thread Nathan Bossart
On Thu, Dec 15, 2022 at 02:47:21PM -0800, Nathan Bossart wrote: > I tried setting wal_retrieve_retry_interval to 1ms for all TAP tests > (similar to what was done in 2710ccd), and I noticed that the recovery > tests consistently took much longer. Upon further inspection, it looks > like the same (

Re: allow granting CLUSTER, REFRESH MATERIALIZED VIEW, and REINDEX

2022-12-18 Thread Justin Pryzby
On Sat, Dec 17, 2022 at 04:39:29AM -0800, Ted Yu wrote: > + List *ancestors = get_partition_ancestors(relid); > + Oid root = InvalidOid; > > nit: it would be better if the variable `root` can be aligned with variable > `ancestors`. It is aligned, but only aft

Re: allow granting CLUSTER, REFRESH MATERIALIZED VIEW, and REINDEX

2022-12-18 Thread Ted Yu
On Sun, Dec 18, 2022 at 3:30 PM Nathan Bossart wrote: > Here is a new version of the patch. Besides some cleanup, I added an index > on reltoastrelid for the toast-to-main-relation lookup. Before I bother > adjusting the tests and documentation, I'm curious to hear thoughts on > whether this se

Re: allow granting CLUSTER, REFRESH MATERIALIZED VIEW, and REINDEX

2022-12-18 Thread Nathan Bossart
On Sun, Dec 18, 2022 at 04:25:15PM -0800, Ted Yu wrote: > + * Note: Because this function assumes that the realtion whose OID is > passed as > > Typo: realtion -> relation Thanks, fixed. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com >From 224614b6ebf2c8d919d8d8500c8f9d6bdaf9a0b

RE: pg_upgrade: Make testing different transfer modes easier

2022-12-18 Thread Shinoda, Noriyoshi (PN Japan FSIP)
Hi, With the addition of --copy option, pg_upgrade now has three possible transfer mode options. Currently, an error does not occur even if multiple transfer modes are specified. For example, we can also run "pg_upgrade --link --copy --clone" command. As discussed in Horiguchi-san's previous em

[BUG] pg_upgrade test fails from older versions.

2022-12-18 Thread Anton A. Melnikov
Hello! Found that pg_upgrade test has broken for upgrades from older versions. This happened for two reasons. 1) In 7b378237a the format of "aclitem" changed so upgrade from <=15 fails with error: "Your installation contains the "aclitem" data type in user tables. The internal format of "aclitem"

Re: [PATCH] Backport perl tests for pg_upgrade from 322becb60

2022-12-18 Thread Anton A. Melnikov
Hello! On 09.12.2022 08:19, Michael Paquier wrote: On Mon, Aug 01, 2022 at 01:02:21AM +0300, Anton A. Melnikov wrote: As far as i understand from this thread: https://www.postgresql.org/message-id/flat/Yox1ME99GhAemMq1%40paquier.xyz, the aim of the perl version for the pg_upgrade tests is to a

Re: [PATCH] Backport perl tests for pg_upgrade from 322becb60

2022-12-18 Thread Michael Paquier
On Mon, Dec 19, 2022 at 04:16:53AM +0300, Anton A. Melnikov wrote: > I have withdrawn the patch with the backport, but then the question is > whether we > will make fixes in older test.sh tests seems to be remains open. > Will we fix it? Justin is not sure if anyone needs this: > https://www.postg

Re: [BUG] pg_upgrade test fails from older versions.

2022-12-18 Thread Tom Lane
"Anton A. Melnikov" writes: > 2) In 60684dd83 and b5d63824 there are two changes in the set of specific > privileges. > The thing is that in the privileges.sql test there is REVOKE DELETE command > which becomes pair of REVOKE ALL and GRANT all specific privileges except > DELETE > in the result

Re: [BUG] pg_upgrade test fails from older versions.

2022-12-18 Thread Michael Paquier
On Mon, Dec 19, 2022 at 03:50:19AM +0300, Anton A. Melnikov wrote: > +-- The internal format of "aclitem" changed in PostgreSQL version 16 > +-- so replace it with text type > +\if :oldpgversion_le15 > +DO $$ > +DECLARE > +change_aclitem_type TEXT; > +BEGIN > +FOR change_aclitem_type IN > +

Re: [BUG] pg_upgrade test fails from older versions.

2022-12-18 Thread Michael Paquier
On Sun, Dec 18, 2022 at 08:56:48PM -0500, Tom Lane wrote: > "Anton A. Melnikov" writes: >> 2) In 60684dd83 and b5d63824 there are two changes in the set of specific >> privileges. >> The thing is that in the privileges.sql test there is REVOKE DELETE command >> which becomes pair of REVOKE ALL an

Re: [PATCH] random_normal function

2022-12-18 Thread Michael Paquier
On Sat, Dec 17, 2022 at 05:49:15PM +0100, Fabien COELHO wrote: > Overall, I think that there should be a clearer discussion and plan about > which random functionS postgres should provide to complement the standard > instead of going there… randomly:-) So, what does the specification tells about s

Re: Add LZ4 compression in pg_dump

2022-12-18 Thread Michael Paquier
On Sat, Dec 17, 2022 at 05:26:15PM -0600, Justin Pryzby wrote: > 001: still refers to "gzip", which is correct for -Fp and -Fd but not > for -Fc, for which it's more correct to say "zlib". Or should we begin by changing all these existing "not built with zlib support" error strings to the more ge

GROUP BY ALL

2022-12-18 Thread Andrey Borodin
Hi hackers! I saw a thread in a social network[0] about GROUP BY ALL. The idea seems useful. I always was writing something like select datname, usename, count(*) from pg_stat_activity group by 1,2; and then rewriting to select datname, usename, query, count(*) from pg_stat_activity group

Re: (non) translatable string splicing

2022-12-18 Thread Michael Paquier
On Fri, Dec 16, 2022 at 07:24:52AM -0600, Justin Pryzby wrote: > Due to incomplete translation, that allows some pretty fancy output, > like: > | You must identify the directory where the residen los binarios del clúster > antiguo. > > That commit also does this a couple times: > > +

Re: GROUP BY ALL

2022-12-18 Thread Tom Lane
Andrey Borodin writes: > I saw a thread in a social network[0] about GROUP BY ALL. The idea seems > useful. Isn't that just a nonstandard spelling of SELECT DISTINCT? What would happen if there are aggregate functions in the tlist? I'm not especially on board with "ALL" meaning "ALL (oh, but no

Re: GROUP BY ALL

2022-12-18 Thread Andrey Borodin
On Sun, Dec 18, 2022 at 8:30 PM Tom Lane wrote: > > I'm not especially on board with "ALL" meaning "ALL (oh, but not > aggregates)". Yes, that's the weak part of the proposal. I even thought about renaming it to "GROUP BY SOMEHOW" or even "GROUP BY SURPRISE ME". I mean I see some cases when it's

Re: GROUP BY ALL

2022-12-18 Thread David G. Johnston
On Sunday, December 18, 2022, Tom Lane wrote: > Andrey Borodin writes: > > I saw a thread in a social network[0] about GROUP BY ALL. The idea seems > useful. > > Isn't that just a nonstandard spelling of SELECT DISTINCT? > > What would happen if there are aggregate functions in the tlist? > I'm

Simplifications for error messages related to compression

2022-12-18 Thread Michael Paquier
Hi all, While looking at a different patch, I have noticed that the error messages produced by pg_basebackup and pg_dump are a bit inconsistent with the other others. Why not switching all of them as of the attached? This reduces the overall translation effort, using more: "this build does not s

isolationtester - allow a session specific connection string

2022-12-18 Thread Peter Smith
Hi hackers. I wanted some way to test overlapping transactions from different publisher sessions so I could better test the logical replication "parallel apply" feature being developed in another thread [1]. AFAIK currently there is no way to do this kind of test except manually (e.g. using separa

Re: isolationtester - allow a session specific connection string

2022-12-18 Thread Tom Lane
Peter Smith writes: > My patch/idea makes a small change to the isolationtester spec > grammar. Now each session can optionally specify its own connection > string. When specified, this will override any connection string for > that session that would otherwise have been used. This is the only > c

appendBinaryStringInfo stuff

2022-12-18 Thread Peter Eisentraut
I found a couple of adjacent weird things: There are a bunch of places in the json code that use appendBinaryStringInfo() where appendStringInfoString() could be used, e.g., appendBinaryStringInfo(buf, ".size()", 7); Is there a reason for this? Are we that stretched for performance? I

Re: [PATCH] Teach pg_waldump to extract FPIs from the WAL

2022-12-18 Thread Michael Paquier
On Thu, Dec 15, 2022 at 05:17:46PM -0600, David Christensen wrote: > On Thu, Dec 15, 2022 at 12:36 AM Michael Paquier wrote: > This v10 should incorporate your feedback as well as Bharath's. Thanks for the new version. I have minor comments. >> It seems to me that you could allow things to work

Re: isolationtester - allow a session specific connection string

2022-12-18 Thread Peter Smith
On Mon, Dec 19, 2022 at 5:04 PM Tom Lane wrote: > > Peter Smith writes: > > My patch/idea makes a small change to the isolationtester spec > > grammar. Now each session can optionally specify its own connection > > string. When specified, this will override any connection string for > > that sess

Re: pg_upgrade allows itself to be run twice

2022-12-18 Thread Michael Paquier
On Fri, Dec 16, 2022 at 07:38:09AM -0600, Justin Pryzby wrote: > However, setting FirstNormalOid in initdb itself (rather than in the > next invocation of postgres, if it isn't in single-user-mode) was the > mechanism by which to avoid the original problem that pg_upgrade can be > run twice, if the

Re: Add LSN along with offset to error messages reported for WAL file read/write/validate header failures

2022-12-18 Thread Michael Paquier
On Tue, Dec 06, 2022 at 04:27:50PM +0900, Kyotaro Horiguchi wrote: > At Mon, 5 Dec 2022 10:03:55 +0900, Michael Paquier > wrote in >> Hence I would tend to let XLogFromFileName do the job, while having a >> SQL function that is just a thin wrapper around it that returns the >> segment TLI and it

Re: isolationtester - allow a session specific connection string

2022-12-18 Thread Peter Smith
On Mon, Dec 19, 2022 at 5:35 PM Peter Smith wrote: > > On Mon, Dec 19, 2022 at 5:04 PM Tom Lane wrote: > > > > Peter Smith writes: > > > My patch/idea makes a small change to the isolationtester spec > > > grammar. Now each session can optionally specify its own connection > > > string. When spe