Re: split TOAST support out of postgres.h

2022-12-28 Thread Nikita Malakhov
Hi, I've thought about this while working on Pluggable TOAST and 64-bit TOAST value ID myself. Agree with #2 to seem the best of the above. There are not so many places where a new header should be included. On Wed, Dec 28, 2022 at 6:08 PM Tom Lane wrote: > Peter Eisentraut writes: > > ... The

Re: Avoiding unnecessary clog lookups while freezing

2022-12-28 Thread Peter Geoghegan
On Wed, Dec 28, 2022 at 4:43 PM Andres Freund wrote: > > > Hm. I dimply recall that we had repeated cases where the hint bits were > > > set > > > wrongly due to some of the multixact related bugs. I think I was trying > > > to be > > > paranoid about not freezing stuff in those situations, sinc

psql: stop at error immediately during \copy

2022-12-28 Thread Peifeng Qiu
Hi, hackers. psql \copy command can stream data from client host just like the normal copy command can do on server host. Let's assume we want to stream a local data file from psql: pgsql =# \copy tbl from '/tmp/datafile' (format 'text'); If there's error inside the data file, \copy will still str

Re: BUG #17717: Regression in vacuumdb (15 is slower than 10/11 and possible memory issue)

2022-12-28 Thread Andres Freund
Hi, On 2022-12-28 15:13:23 -0500, Tom Lane wrote: > [ redirecting to -hackers because patch attached ] > > Peter Geoghegan writes: > > On Fri, Dec 16, 2022 at 6:49 AM Tom Lane wrote: > >> That is a really good point. How about teaching VACUUM to track > >> the oldest original relfrozenxid and

Re: BUG #17717: Regression in vacuumdb (15 is slower than 10/11 and possible memory issue)

2022-12-28 Thread Tom Lane
Justin Pryzby writes: > On Wed, Dec 28, 2022 at 03:13:23PM -0500, Tom Lane wrote: >> I'm forced to the conclusion that we have to expose some VACUUM >> options if we want this to work well. Attached is a draft patch >> that invents SKIP_DATABASE_STATS and ONLY_DATABASE_STATS options >> (name bike

Re: windows/meson cfbot warnings

2022-12-28 Thread Andres Freund
Hi, On 2022-12-28 18:35:38 -0600, Justin Pryzby wrote: > Since a few days ago, the windows/meson task has been spewing messages for > each tap > test: > > | Unknown TAP version. The first line MUST be `TAP version `. Assuming > version 12. > > I guess because the image is updated to use meson

Re: BUG #17717: Regression in vacuumdb (15 is slower than 10/11 and possible memory issue)

2022-12-28 Thread Justin Pryzby
On Wed, Dec 28, 2022 at 03:13:23PM -0500, Tom Lane wrote: > Peter Geoghegan writes: > > On Fri, Dec 16, 2022 at 6:49 AM Tom Lane wrote: > >> That is a really good point. How about teaching VACUUM to track > >> the oldest original relfrozenxid and relminmxid among the table(s) > >> it processed,

Re: Avoiding unnecessary clog lookups while freezing

2022-12-28 Thread Andres Freund
On 2022-12-28 16:37:27 -0800, Peter Geoghegan wrote: > On Wed, Dec 28, 2022 at 4:20 PM Andres Freund wrote: > > > Theoretically this is an old issue that dates back to commit > > > 699bf7d05c, as opposed to an issue in the page-level freezing patch. > > > But that's not really true in a real pract

Re: Avoiding unnecessary clog lookups while freezing

2022-12-28 Thread Peter Geoghegan
On Wed, Dec 28, 2022 at 4:20 PM Andres Freund wrote: > > Theoretically this is an old issue that dates back to commit > > 699bf7d05c, as opposed to an issue in the page-level freezing patch. > > But that's not really true in a real practical sense. In practice the > > calls to TransactionIdDidComm

windows/meson cfbot warnings

2022-12-28 Thread Justin Pryzby
Since a few days ago, the windows/meson task has been spewing messages for each tap test: | Unknown TAP version. The first line MUST be `TAP version `. Assuming version 12. I guess because the image is updated to use meson v1.0.0. https://github.com/mesonbuild/meson/commit/b7a5c384a1f1ba80c0990

Re: Avoiding unnecessary clog lookups while freezing

2022-12-28 Thread Andres Freund
Hi, On 2022-12-28 15:24:28 -0800, Peter Geoghegan wrote: > I took another look at the code coverage situation around freezing > following pushing the page-level freezing patch earlier today. I > spotted an issue that I'd missed up until now: certain sanity checks > in heap_prepare_freeze_tuple() c

Re: perl 5.36, C99, -Wdeclaration-after-statement -Wshadow=compatible-local

2022-12-28 Thread Tom Lane
Andres Freund writes: > On 2022-12-28 13:43:27 -0500, Tom Lane wrote: >> Hmm ... I guess the buildfarm would tell us whether that detection works >> correctly on platforms where it matters. Let's keep it simple if we >> can. > Quick clarification question: Are you suggesting to use #ifdef __GNUC

Re: perl 5.36, C99, -Wdeclaration-after-statement -Wshadow=compatible-local

2022-12-28 Thread Andres Freund
On 2022-12-28 13:43:27 -0500, Tom Lane wrote: > > In the compiler test, I chose to not check whether -Werror=unknown-pragmas > > is > > supported - it appears to be an old gcc flag, and the worst outcome is that > > HAVE_PRAGMA_SYSTEM_HEADER isn't defined. > > We could alternatively define HAVE_PR

Re: pg_stat_bgwriter.buffers_backend is pretty meaningless (and more?)

2022-12-28 Thread Andres Freund
Hi, On 2022-10-06 13:42:09 -0400, Melanie Plageman wrote: > > Additionally, some minor notes: > > > > - Since the stats are counting blocks, it would make sense to prefix the > > view columns with "blks_", and word them in the past tense (to match > > current style), i.e. "blks_written", "blks_r

Avoiding unnecessary clog lookups while freezing

2022-12-28 Thread Peter Geoghegan
I took another look at the code coverage situation around freezing following pushing the page-level freezing patch earlier today. I spotted an issue that I'd missed up until now: certain sanity checks in heap_prepare_freeze_tuple() call TransactionIdDidCommit() more often than really seems necessar

Re: Support logical replication of DDLs

2022-12-28 Thread Zheng Li
On Wed, Dec 28, 2022 at 5:42 PM Zheng Li wrote: > > >- CreatePublication has a long list of command tags; is that good? > >Maybe it'd be better to annotate the list in cmdtaglist.h somehow. > > I've addressed this comment by introducing a new flag ddlreplok in the > PG_CMDTAG macro and modified Cr

Re: BUG #17717: Regression in vacuumdb (15 is slower than 10/11 and possible memory issue)

2022-12-28 Thread Tom Lane
Nathan Bossart writes: > On Wed, Dec 28, 2022 at 03:13:23PM -0500, Tom Lane wrote: >> +executeCommand(conn, "VACUUM (ONLY_DATABASE_STATS);", echo); > When I looked at this, I thought it would be better to send the command > through the parallel slot machinery so that failures would us

Re: add \dpS to psql

2022-12-28 Thread Nathan Bossart
On Wed, Dec 28, 2022 at 02:46:23PM +0300, Maxim Orlov wrote: > The patch applies with no problem, implements what it declared, CF bot is > happy. > Without patch \dpS shows 0 rows, after applying system objects are shown. > Consider this patch useful, hope it will be committed soon. Thanks for rev

Re: BUG #17717: Regression in vacuumdb (15 is slower than 10/11 and possible memory issue)

2022-12-28 Thread Nathan Bossart
On Wed, Dec 28, 2022 at 04:20:19PM -0500, Tom Lane wrote: > Nathan Bossart writes: >> I think the main difference between your patch and mine is that I've >> exposed vac_update_datfrozenxid() via a function instead of a VACUUM >> option. IMHO that feels a little more natural, but I can't say I fe

Re: BUG #17717: Regression in vacuumdb (15 is slower than 10/11 and possible memory issue)

2022-12-28 Thread Nathan Bossart
On Wed, Dec 28, 2022 at 03:13:23PM -0500, Tom Lane wrote: > + /* If we used SKIP_DATABASE_STATS, mop up with ONLY_DATABASE_STATS */ > + if (vacopts->skip_database_stats && stage == ANALYZE_NO_STAGE && > !failed) > + { > + executeCommand(conn, "VACUUM (ONLY_DATABASE_STATS);"

Re: BUG #17717: Regression in vacuumdb (15 is slower than 10/11 and possible memory issue)

2022-12-28 Thread Tom Lane
Nathan Bossart writes: > I think the main difference between your patch and mine is that I've > exposed vac_update_datfrozenxid() via a function instead of a VACUUM > option. IMHO that feels a little more natural, but I can't say I feel too > strongly about it. I thought about that but it seems

Re: BUG #17717: Regression in vacuumdb (15 is slower than 10/11 and possible memory issue)

2022-12-28 Thread Nathan Bossart
On Wed, Dec 28, 2022 at 03:13:23PM -0500, Tom Lane wrote: > I'm forced to the conclusion that we have to expose some VACUUM > options if we want this to work well. Attached is a draft patch > that invents SKIP_DATABASE_STATS and ONLY_DATABASE_STATS options > (name bikeshedding welcome) and teaches

Re: BUG #17717: Regression in vacuumdb (15 is slower than 10/11 and possible memory issue)

2022-12-28 Thread Tom Lane
[ redirecting to -hackers because patch attached ] Peter Geoghegan writes: > On Fri, Dec 16, 2022 at 6:49 AM Tom Lane wrote: >> That is a really good point. How about teaching VACUUM to track >> the oldest original relfrozenxid and relminmxid among the table(s) >> it processed, and skip vac_upd

Re: perl 5.36, C99, -Wdeclaration-after-statement -Wshadow=compatible-local

2022-12-28 Thread Tom Lane
Andres Freund writes: > The attached *prototype* patch is a slightly different spin on the idea of > using -isystem: It adds a > #pragma GCC system_header > to plperl.h if supported by the compiler. That also avoids warnings from > within plperl and subsidiary headers. > I don't really have an

Re: perl 5.36, C99, -Wdeclaration-after-statement -Wshadow=compatible-local

2022-12-28 Thread Andres Freund
Hi, On 2022-11-02 17:03:34 -0700, Andres Freund wrote: > On 2022-11-02 19:57:45 -0400, Tom Lane wrote: > > Andres Freund writes: > > > On 2022-11-01 17:00:27 -0400, Peter Eisentraut wrote: > > >> Python has the same issues. There are a few other Python-embedding > > >> projects > > >> that use

Re: Allow placeholders in ALTER ROLE w/o superuser

2022-12-28 Thread Justin Pryzby
On Tue, Dec 27, 2022 at 11:29:40PM -0500, Tom Lane wrote: > Justin Pryzby writes: > > This fails when run more than once: > > time meson test --setup running --print > > test_pg_db_role_setting-running/regress > > Ah. > > > It didn't fail for you because it says: > > ./src/test/modules/test_pg_

Re: Making Vars outer-join aware

2022-12-28 Thread Tom Lane
Richard Guo writes: > I think I see where the problem is. And I can see currently in > get_eclass_for_sort_expr we always use the top JoinDomain. So although > the equality clause 'a.x = b.y' belongs to JoinDomain {B}, we set up ECs > for 'a.x' and 'b.y' that belong to the top JoinDomain {A, B,

Re: split TOAST support out of postgres.h

2022-12-28 Thread Tom Lane
Peter Eisentraut writes: > ... Then we could either > 1) Include varatt.h in postgres.h, similar to elog.h and palloc.h. That > way we clean up the files a bit but don't change any external interfaces. > 2) Just let everyone who needs it include the new file. > 3) Compromise: You can avoid mo

Re: Error-safe user functions

2022-12-28 Thread Andrew Dunstan
On 2022-12-28 We 01:00, Amul Sul wrote: > On Tue, Dec 27, 2022 at 11:17 PM Tom Lane wrote: >> Andrew Dunstan writes: >>> Here's a patch that covers the ltree and intarray contrib modules. >> I would probably have done this a little differently --- I think >> the added "res" parameters aren't re

Re: Underscores in numeric literals

2022-12-28 Thread Andrew Dunstan
On 2022-12-27 Tu 09:55, Tom Lane wrote: > We already accept that numeric input is different from numeric > literals: you can't write Infinity or NaN in SQL without quotes. > So I don't see an argument that we have to allow this in numeric > input for consistency. > That's almost the same, but no

Re: split TOAST support out of postgres.h

2022-12-28 Thread Isaac Morland
On Wed, 28 Dec 2022 at 08:07, Peter Eisentraut < peter.eisentr...@enterprisedb.com> wrote: > Most backend code doesn't actually need the variable-length data types > support (TOAST support) in postgres.h. So I figured we could try to put > it into a separate header file. That makes postgres.h mo

split TOAST support out of postgres.h

2022-12-28 Thread Peter Eisentraut
Most backend code doesn't actually need the variable-length data types support (TOAST support) in postgres.h. So I figured we could try to put it into a separate header file. That makes postgres.h more manageable, and it avoids including a bunch of complicated unused stuff everywhere. I picke

Re: add \dpS to psql

2022-12-28 Thread Maxim Orlov
> Here it is: https://commitfest.postgresql.org/41/4043/ > Hi! The patch applies with no problem, implements what it declared, CF bot is happy. Without patch \dpS shows 0 rows, after applying system objects are shown. Consider this patch useful, hope it will be committed soon. -- Best regards,

Re: Data loss on logical replication, 12.12 to 14.5, ALTER SUBSCRIPTION

2022-12-28 Thread Michail Nikolaev
Hello. > None of these entries are from the point mentioned by you [1] > yesterday where you didn't find the corresponding data in the > subscriber. How did you identify that the entries corresponding to > that timing were missing? Some of the before the interval, some after... But the source dat

Re: Add SHELL_EXIT_CODE to psql

2022-12-28 Thread Maxim Orlov
Hi! The patch is implementing what is declared to do. Shell return code is now accessible is psql var. Overall code is in a good condition. Applies with no errors on master. Unfortunately, regression tests are failing on the macOS due to the different shell output. @@ -1308,13 +1308,13 @@ deallo

Add BufFileRead variants with short read and EOF detection

2022-12-28 Thread Peter Eisentraut
Most callers of BufFileRead() want to check whether they read the full specified length. Checking this at every call site is very tedious. This patch provides additional variants BufFileReadExact() and BufFileReadMaybeEOF() that include the length checks. I considered changing BufFileRead() i

Re: Add 64-bit XIDs into PostgreSQL 15

2022-12-28 Thread Maxim Orlov
Hi! I want to make a quick summary here. 1. An overall consensus has been reached: we shall focus on committing SLRU changes first. 2. I've created an appropriate patch set here [0]. 3. How [0] is waiting for a review. As always, all opinions will be welcome. 4. While discussing error/warning mes

RE: Exit walsender before confirming remote flush in logical replication

2022-12-28 Thread Hayato Kuroda (Fujitsu)
Dear Amit, > Thanks for the verification. BTW, do you think we should document this > either with time-delayed replication or otherwise unless this is > already documented? I think this should be documented at "Shutting Down the Server" section in runtime.sgml or logical-replicaiton.sgml, but I

Re: [PATCH] Support using "all" for the db user in pg_ident.conf

2022-12-28 Thread Jelte Fennema
@Michael > Anyway, it is a bit confusing to see a patch touching parts of the > ident code related to the system-username while it claims to provide a > mean to shortcut a check on the database-username.  That's totally fair, I attached a new iteration of this patchset where this refactoring and

Re: Making Vars outer-join aware

2022-12-28 Thread Richard Guo
On Tue, Dec 27, 2022 at 11:31 PM Tom Lane wrote: > The thing that I couldn't get around before is that if you have, > say, a mergejoinable equality clause in an outer join: > > select ... from a left join b on a.x = b.y; > > that equality clause can only be associated with the join domain > f