Re: Direct I/O

2023-04-08 Thread Thomas Munro
Indeed, I can't reproduce this with (our) checksums on. I also can't reproduce it with O_DIRECT off. I also can't reproduce it if I use "mkdir pgdata && chattr +C pgdata && initdb -D pgdata" to have a pgdata directory with copy-on-write and (their) checksums disabled. But it reproduces quite

Re: Direct I/O

2023-04-08 Thread Noah Misch
On Sat, Apr 08, 2023 at 11:08:16AM -0700, Andres Freund wrote: > On 2023-04-07 23:04:08 -0700, Andres Freund wrote: > > There were some failures in CI (e.g. [1] (and perhaps also bf, didn't yet > > check), about "no unpinned buffers available". I was worried for a moment > > that this could

Re: Proposal: %T Prompt parameter for psql for current time (like Oracle has)

2023-04-08 Thread Pavel Stehule
ne 9. 4. 2023 v 3:54 odesílatel Tom Lane napsal: > Greg Stark writes: > > I'm not sure if the *prompt* is a sensible place for it though. The > > place it seems like it would be most useful is reading the output of > > script executions where there would be no prompts. Perhaps it's the > >

Re: Use fadvise in wal replay

2023-04-08 Thread Gregory Stark (as CFM)
On Thu, 19 Jan 2023 at 18:19, Andres Freund wrote: > > On 2023-01-19 22:19:10 +0100, Tomas Vondra wrote: > > > So I'm a bit unsure about this patch. I doesn't seem like it can perform > > better than read-ahead (although perhaps it does, on a different storage > > system). > > I really don't see

Re: Direct I/O

2023-04-08 Thread Thomas Munro
On Sun, Apr 9, 2023 at 2:18 PM Andres Freund wrote: > On 2023-04-09 13:55:33 +1200, Thomas Munro wrote: > > I think that particular thing might relate to modifications of the > > user buffer while a write is in progress (breaking btrfs's internal > > checksums). I don't think we should ever do

Re: Direct I/O

2023-04-08 Thread Andres Freund
Hi, On 2023-04-09 13:55:33 +1200, Thomas Munro wrote: > I think that particular thing might relate to modifications of the > user buffer while a write is in progress (breaking btrfs's internal > checksums). I don't think we should ever do that ourselves (not least > because it'd break our own

Re: Direct I/O

2023-04-08 Thread Tom Lane
Thomas Munro writes: > It's odd, though, if it is their bug and not ours: I'd expect our > friends in other databases to have hit all that sort of thing years > ago, since many comparable systems have a direct I/O knob*. Yeah, it seems moderately likely that it's our own bug ... but this code's

RE: PGDOCS - function pg_get_publication_tables is not documented?

2023-04-08 Thread Yu Shi (Fujitsu)
On Fri, Mar 24, 2023 6:26 AM Tom Lane wrote: > > I do see a docs change that I think would be worth making: get > rid of the explicit mention of it in create_subscription.sgml > in favor of using that view. > I agree and I tried to modify the query to use the view. Please see the attached

Re: Direct I/O

2023-04-08 Thread Thomas Munro
On Sun, Apr 9, 2023 at 11:05 AM Tom Lane wrote: > Googling finds a lot of suggestions that O_DIRECT doesn't play nice > with btrfs, for example > > https://www.mail-archive.com/linux-btrfs@vger.kernel.org/msg92824.html > > It's not clear to me how much of that lore is still current, > but it's

Re: Proposal: %T Prompt parameter for psql for current time (like Oracle has)

2023-04-08 Thread Tom Lane
Greg Stark writes: > I'm not sure if the *prompt* is a sensible place for it though. The > place it seems like it would be most useful is reading the output of > script executions where there would be no prompts. Perhaps it's the > command tags and \echo statements that should be timestamped.

Re: Commitfest 2023-03 starting tomorrow!

2023-04-08 Thread Tom Lane
"Gregory Stark (as CFM)" writes: > Some of these, especially the last category, are challenging for me to > determine. If I move forward a patch of yours that you think makes > sense to treat as an open issue that should be resolved in this > release then feel free to say. I think that's largely

Re: Commitfest 2023-03 starting tomorrow!

2023-04-08 Thread Gregory Stark (as CFM)
On Sat, 8 Apr 2023 at 11:37, Greg Stark wrote: > > c) Pick out the Bug Fixes, cleanup patches, and other non-feature > patches that might be open issues for v16. So on further examination it seems there are multiple category of patches that are worth holding onto rather than shifting to the next

Re: Proposal: %T Prompt parameter for psql for current time (like Oracle has)

2023-04-08 Thread Greg Stark
On Wed, 22 Feb 2023 at 13:38, Kirk Wolak wrote: > > I already requested ONLY the HH24 format. 8 characters of output. no > options. It's a waste of time. > After all these years, sqlplus still has only one setting (show it, or not). > I am asking the same here. > And I will gladly defend

Re: Direct I/O

2023-04-08 Thread Tom Lane
Thomas Munro writes: > On Sun, Apr 9, 2023 at 10:08 AM Andrew Dunstan wrote: >> btrfs > Aha! Googling finds a lot of suggestions that O_DIRECT doesn't play nice with btrfs, for example https://www.mail-archive.com/linux-btrfs@vger.kernel.org/msg92824.html It's not clear to me how much of

Re: Direct I/O

2023-04-08 Thread Thomas Munro
On Sun, Apr 9, 2023 at 10:17 AM Andrew Dunstan wrote: > I can run the test in isolation, and it's get an error reliably. Random idea: it looks like you have compression enabled. What if you turn it off in the directory where the test runs? Something like btrfs property set compression ...

Re: longfin missing gssapi_ext.h

2023-04-08 Thread Thomas Munro
On Sun, Apr 9, 2023 at 6:40 AM Tom Lane wrote: > The exact same thing applies to FreeBSD, except that their in-core > Heimdal is ancient (1.5.2). Also, they do have MIT Kerberos > available as a package [1]. I'd been misled by the lack of a hit > on "kerberos", but "krb5" finds it. Our code

Re: Direct I/O

2023-04-08 Thread Andrew Dunstan
On 2023-04-08 Sa 17:23, Andres Freund wrote: Hi, On 2023-04-08 17:10:19 -0400, Tom Lane wrote: Thomas Munro writes: Now crake is doing this: 2023-04-08 16:50:03.177 EDT [2023-04-08 16:50:03 EDT 3257645:3] 004_io_direct.pl LOG: statement: select count(*) from t1 2023-04-08 16:50:03.316 EDT

Re: Direct I/O

2023-04-08 Thread Thomas Munro
On Sun, Apr 9, 2023 at 10:08 AM Andrew Dunstan wrote: > btrfs Aha!

Re: Direct I/O

2023-04-08 Thread Andrew Dunstan
On 2023-04-08 Sa 17:42, Andres Freund wrote: Hi, On 2023-04-08 17:31:02 -0400, Tom Lane wrote: Andres Freund writes: On 2023-04-08 17:10:19 -0400, Tom Lane wrote: It's also odd that it's just crake having the issue. It's just a linux host, afaics. Indeed. I'm guessing from the compiler

Re: Direct I/O

2023-04-08 Thread Andres Freund
Hi, On 2023-04-08 17:31:02 -0400, Tom Lane wrote: > Andres Freund writes: > > On 2023-04-08 17:10:19 -0400, Tom Lane wrote: > > It's also odd that it's just crake having the issue. It's just a linux host, > > afaics. > > Indeed. I'm guessing from the compiler version that it's Fedora 37 now

Re: Direct I/O

2023-04-08 Thread Tom Lane
Andres Freund writes: > On 2023-04-08 17:10:19 -0400, Tom Lane wrote: >> (BTW, why are the log lines doubly timestamped?) > It's odd. Oh, I guess that's intentional, because crake has 'log_line_prefix = \'%m [%s %p:%l] %q%a \'', > It's also odd that it's just crake having the

Re: Direct I/O

2023-04-08 Thread Andres Freund
Hi, On 2023-04-08 17:10:19 -0400, Tom Lane wrote: > Thomas Munro writes: > Now crake is doing this: > > 2023-04-08 16:50:03.177 EDT [2023-04-08 16:50:03 EDT 3257645:3] > 004_io_direct.pl LOG: statement: select count(*) from t1 > 2023-04-08 16:50:03.316 EDT [2023-04-08 16:50:03 EDT 3257646:1]

Re: Direct I/O

2023-04-08 Thread Thomas Munro
On Sun, Apr 9, 2023 at 9:10 AM Tom Lane wrote: > 2023-04-08 16:50:03.177 EDT [2023-04-08 16:50:03 EDT 3257645:3] > 004_io_direct.pl LOG: statement: select count(*) from t1 > 2023-04-08 16:50:03.316 EDT [2023-04-08 16:50:03 EDT 3257646:1] ERROR: > invalid page in block 56 of relation

Re: Direct I/O

2023-04-08 Thread Tom Lane
Thomas Munro writes: > On Sun, Apr 9, 2023 at 6:55 AM Andres Freund wrote: >> Given the frequency of failures on this in the buildfarm, I propose using the >> temporary workaround of using wal_level=replica. That avoids the use of the >> over-eager log_newpage_range(). > Will do. Now crake is

Re: Direct I/O

2023-04-08 Thread Thomas Munro
On Sun, Apr 9, 2023 at 6:55 AM Andres Freund wrote: > Given the frequency of failures on this in the buildfarm, I propose using the > temporary workaround of using wal_level=replica. That avoids the use of the > over-eager log_newpage_range(). Will do.

Re: When to drop src/tools/msvc support

2023-04-08 Thread Tom Lane
Andres Freund writes: > Do we want to drop src/tools/msvc support in 16 (i.e. now), or do it early in > 17? On the one hand, it feels like something we shouldn't do after feature freeze. On the other hand, continuing to maintain three build systems is a real drag (although you could argue that

Re: When to drop src/tools/msvc support

2023-04-08 Thread Jonathan S. Katz
On 4/8/23 3:10 PM, Andres Freund wrote: Hi, I'd planned to write this soon anyway, but it was just brought up in [1]. Originally we had planned to drop src/tools/msvc support shortly after meson went in. Unfortunately, it took a bit longer than originally hoped for to merge meson support and

When to drop src/tools/msvc support

2023-04-08 Thread Andres Freund
Hi, I'd planned to write this soon anyway, but it was just brought up in [1]. Originally we had planned to drop src/tools/msvc support shortly after meson went in. Unfortunately, it took a bit longer than originally hoped for to merge meson support and then longer than hoped to add buildfarm

Re: Direct I/O

2023-04-08 Thread Andres Freund
Hi, Given the frequency of failures on this in the buildfarm, I propose using the temporary workaround of using wal_level=replica. That avoids the use of the over-eager log_newpage_range(). Greetings, Andres Freund

Re: longfin missing gssapi_ext.h

2023-04-08 Thread Tom Lane
I wrote: > * NetBSD's package collection[1] includes both Heimdal and MIT Kerberos > (mit-krb5). Apparently what's installed on at least some of the buildfarm > animals is the former. Oh! New data: the core NetBSD OS includes a copy of Heimdal (looks to be 7.7.0 in the 10.0_BETA sources). The

Re: Parallel Full Hash Join

2023-04-08 Thread Melanie Plageman
On Sat, Apr 8, 2023 at 1:30 PM Melanie Plageman wrote: > > On Sat, Apr 8, 2023 at 12:33 PM Tom Lane wrote: > > > > Thomas Munro writes: > > > I committed the main patch. > > > > BTW, it was easy to miss in all the buildfarm noise from > > last-possible-minute patches, but chimaera just showed

Re: Direct I/O

2023-04-08 Thread Andres Freund
Hi, On 2023-04-07 23:04:08 -0700, Andres Freund wrote: > There were some failures in CI (e.g. [1] (and perhaps also bf, didn't yet > check), about "no unpinned buffers available". I was worried for a moment > that this could actually be relation to the bulk extension patch. > > But it looks

Re: longfin missing gssapi_ext.h

2023-04-08 Thread Stephen Frost
Greetings, * Tom Lane (t...@sss.pgh.pa.us) wrote: > Stephen Frost writes: > > I suspected there would be an issue with OSX but hadn't expected an > > issue with NetBSD. I had tested this across a few Linux platforms and > > cfbot showed it wasn't causing issues on Windows or the platforms that

Re: Parallel Full Hash Join

2023-04-08 Thread Tom Lane
Melanie Plageman writes: > Having not done much debugging on buildfarm animals before, I don't > suppose there is any way to get access to the core itself? I'd like to > see how many participants the batch barrier had at the time of the > assertion failure. I assume it was 2, but I just wanted to

Re: longfin missing gssapi_ext.h

2023-04-08 Thread Tom Lane
Stephen Frost writes: > I suspected there would be an issue with OSX but hadn't expected an > issue with NetBSD. I had tested this across a few Linux platforms and > cfbot showed it wasn't causing issues on Windows or the platforms that > are run there. Would be really great to have a way to

Re: Parallel Full Hash Join

2023-04-08 Thread Melanie Plageman
On Sat, Apr 8, 2023 at 12:33 PM Tom Lane wrote: > > Thomas Munro writes: > > I committed the main patch. > > BTW, it was easy to miss in all the buildfarm noise from > last-possible-minute patches, but chimaera just showed something > that looks like a bug in this code [1]: > > 2023-04-08

Re: Minimal logical decoding on standbys

2023-04-08 Thread Jonathan S. Katz
On 4/8/23 5:27 AM, Andres Freund wrote: Hi, On 2023-04-07 14:27:09 -0700, Andres Freund wrote: I think I'll push these in a few hours. While this needed more changes than I'd like shortly before the freeze, I think they're largely not in very interesting bits and pieces - and this feature has

Re: Parallel Full Hash Join

2023-04-08 Thread Tom Lane
Thomas Munro writes: > I committed the main patch. BTW, it was easy to miss in all the buildfarm noise from last-possible-minute patches, but chimaera just showed something that looks like a bug in this code [1]: 2023-04-08 12:25:28.709 UTC [18027:321] pg_regress/join_hash LOG: statement:

Re: Making background psql nicer to use in tap tests

2023-04-08 Thread Mikael Kjellström
On 2023-04-08 09:53, Daniel Gustafsson wrote: I recommend just setting up this one test to SKIP if IPC::Run is too old. Yes, will do that when I have a little more time at hand for monitoring the BF later today. So what do you want me to do about grison and morepork? I guess I could try

Re: Partial aggregates pushdown

2023-04-08 Thread Bruce Momjian
On Sat, Apr 8, 2023 at 10:15:40AM -0400, Bruce Momjian wrote: > On Fri, Apr 7, 2023 at 09:16:14PM -0700, Andres Freund wrote: > > extensions (string) > > > > This option is a comma-separated list of names of PostgreSQL extensions > > that are installed, in compatible versions, on both the

Re: Commitfest 2023-03 starting tomorrow!

2023-04-08 Thread Greg Stark
So here we are at the end of the CF: Status summary: March 15March 22March 28April 4 April 8 Needs review: 152 128 116 96 74 Waiting on Author: 42 36 30 21 14 Ready for Committer: 39 32

Re: broken master branch

2023-04-08 Thread Tom Lane
Thomas Munro writes: > On Sat, Apr 8, 2023 at 8:04 PM Pavel Stehule wrote: >> on fresh Fedora 38, I cannot to run regress tests > Looks like the new LLVM 16. I'll try to look at this again next week. > In the meantime you could try using 15. I've become entirely desensitized to seawasp

Re: is_superuser is not documented

2023-04-08 Thread Joseph Koshakow
On Mon, Apr 3, 2023 at 10:47 AM Fujii Masao wrote: >Yes, the patch has not been committed yet because of lack of review comments. >Do you have any review comments on this patch? >Or you think it's ready for committer? I'm not very familiar with this code, so I'm not sure how much my

Re: Non-replayable WAL records through overflows and >MaxAllocSize lengths

2023-04-08 Thread Matthias van de Meent
On Fri, 7 Apr 2023 at 08:05, Michael Paquier wrote: > > On Fri, Apr 07, 2023 at 08:59:22AM +0900, Michael Paquier wrote: > > Okay, cool! > > Done this one with 8fcb32d. Thanks a lot! I'll post the separation of record construction and write-out to xlog in a future thread for 17. One remaining

Re: Partial aggregates pushdown

2023-04-08 Thread Bruce Momjian
On Fri, Apr 7, 2023 at 09:16:14PM -0700, Andres Freund wrote: > On 2023-04-07 22:53:53 -0400, Bruce Momjian wrote: > > > postgres_fdw has no business pushing down calls to non-builtin functions > > > unless the user has explicitly authorized that via the existing > > > whitelisting mechanism. I

Re: Making background psql nicer to use in tap tests

2023-04-08 Thread Tom Lane
=?UTF-8?Q?Mikael_Kjellstr=c3=b6m?= writes: > So what do you want me to do about grison and morepork? I guess I could > try to install a newer version of IPC::Run from CPAN or should I just > leave it be? I think "leave it be" is fine. regards, tom lane

Re: PostgreSQL 16 Release Management Team & Feature Freeze

2023-04-08 Thread Jonathan S. Katz
On 4/6/23 5:37 PM, Jonathan S. Katz wrote: On 3/21/23 11:35 AM, Jonathan S. Katz wrote: Additionally, the RMT has set the feature freeze to be **April 8, 2023 at 0:00 AoE**[1]. This is the last time to commit features for PostgreSQL 16. In  other words, no new PostgreSQL 16 feature can be

Re: daitch_mokotoff module

2023-04-08 Thread Andrew Dunstan
On 2023-04-07 Fr 23:25, Tom Lane wrote: I wrote: Anyway, I assume this is just syntactic sugar for something we can do another way? If it's at all fundamental, I'll have to back the patch out. On closer inspection, this script is completely devoid of any need to deal in non-ASCII data at

Re: longfin missing gssapi_ext.h

2023-04-08 Thread Stephen Frost
Greetings, * Andres Freund (and...@anarazel.de) wrote: > On 2023-04-07 22:50:18 -0400, Tom Lane wrote: > > Or should credential delegation be viewed as an incremental feature that we > > can support or not? > > That seems like the best way forward here. Yeah, that's certainly doable too, though

Re: longfin missing gssapi_ext.h

2023-04-08 Thread Stephen Frost
Greetings, * Tom Lane (t...@sss.pgh.pa.us) wrote: > Stephen Frost writes: > > I'm open to considering support for older versions, however ... > > NetBSD 9.3, which is their *latest production release*, doesn't have > gssapi_ext.h [1]. For that matter, it doesn't look like their >

Re: longfin missing gssapi_ext.h

2023-04-08 Thread Andres Freund
Hi, On 2023-04-07 22:50:18 -0400, Tom Lane wrote: > Or should credential delegation be viewed as an incremental feature that we > can support or not? That seems like the best way forward here. Greetings, Andres Freund

Re: Minimal logical decoding on standbys

2023-04-08 Thread Andres Freund
Hi, On 2023-04-07 14:27:09 -0700, Andres Freund wrote: > I think I'll push these in a few hours. While this needed more changes than > I'd like shortly before the freeze, I think they're largely not in very > interesting bits and pieces - and this feature has been in the works for about > three

Re: Direct I/O

2023-04-08 Thread Thomas Munro
On Sat, Apr 8, 2023 at 4:59 PM Thomas Munro wrote: > On Sat, Apr 8, 2023 at 4:47 PM Thomas Munro wrote: > > After a bit more copy-editing on docs and comments and a round of > > automated indenting, I have now pushed this. I will now watch the > > build farm. I tested on quite a few OSes that

Re: broken master branch

2023-04-08 Thread Pavel Stehule
so 8. 4. 2023 v 10:38 odesílatel Thomas Munro napsal: > On Sat, Apr 8, 2023 at 8:04 PM Pavel Stehule > wrote: > > on fresh Fedora 38, I cannot to run regress tests > > Looks like the new LLVM 16. I'll try to look at this again next week. > In the meantime you could try using 15. > ok Thank

Re: broken master branch

2023-04-08 Thread Thomas Munro
On Sat, Apr 8, 2023 at 8:04 PM Pavel Stehule wrote: > on fresh Fedora 38, I cannot to run regress tests Looks like the new LLVM 16. I'll try to look at this again next week. In the meantime you could try using 15.

broken master branch

2023-04-08 Thread Pavel Stehule
Hi on fresh Fedora 38, I cannot to run regress tests +ERROR: could not load library "/home/pavel/src/postgresql.master/tmp_install/usr/local/pgsql/master/lib/llvmjit.so": /home/pavel/src/p ostgresql.master/tmp_install/usr/local/pgsql/master/lib/llvmjit.so: undefined symbol: LLVMBuildGEP SELECT

Re: Making background psql nicer to use in tap tests

2023-04-08 Thread Daniel Gustafsson
> On 8 Apr 2023, at 02:49, Tom Lane wrote: > > I wrote: >> I've been doing some checking with perlbrew locally. It appears to not >> be about IO::Pty so much as IPC::Run: it works with IPC::Run 0.99 but >> not 0.79. Still bisecting to identify exactly what's the minimum >> okay version. > >

Re: Minimal logical decoding on standbys

2023-04-08 Thread Bertrand Drouvot
Hi, New wording works for me, thanks! Bertrand Le sam. 8 avr. 2023, 08:26, Andres Freund a écrit : > Hi, > > On 2023-04-07 11:12:26 -0700, Andres Freund wrote: > > + > > + > > + confl_active_logicalslot > bigint > > + > > + > > + Number of active logical

Re: Minimal logical decoding on standbys

2023-04-08 Thread Andres Freund
Hi, On 2023-04-07 11:12:26 -0700, Andres Freund wrote: > + > + > + confl_active_logicalslot > bigint > + > + > + Number of active logical slots in this database that have been > + invalidated because they conflict with recovery (note that inactive > ones

Re: Direct I/O

2023-04-08 Thread Andres Freund
Hi, On 2023-04-08 16:59:20 +1200, Thomas Munro wrote: > On Sat, Apr 8, 2023 at 4:47 PM Thomas Munro wrote: > > After a bit more copy-editing on docs and comments and a round of > > automated indenting, I have now pushed this. I will now watch the > > build farm. I tested on quite a few OSes