Re: Direct I/O

2023-04-10 Thread Andrea Gelmini
Il giorno lun 10 apr 2023 alle ore 04:58 Andres Freund ha scritto: > We should definitely let the brtfs folks know of this issue... It's possible > that this bug was recently introduced even. What kernel version did you repro > this on Thomas? In these days on BTRFS ml they are discussing about D

Re: Direct I/O

2023-04-10 Thread Thomas Munro
On Mon, Apr 10, 2023 at 2:57 PM Andres Freund wrote: > Have you tried to write a reproducer for this that doesn't involve postgres? I tried a bit. I'll try harder soon. > ... What kernel version did you repro > this on Thomas? Debian's 6.0.10-2 kernel (Debian 12 on a random laptop). Here's ho

Re: Direct I/O

2023-04-10 Thread Thomas Munro
On Mon, Apr 10, 2023 at 7:27 PM Thomas Munro wrote: > Debian's 6.0.10-2 kernel (Debian 12 on a random laptop). Realising I hadn't updated for a bit, I did so and it still reproduces on: $ uname -a Linux x1 6.1.0-7-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.1.20-1 (2023-03-19) x86_64 GNU/Linux

Re: pg_stat_bgwriter.buffers_backend is pretty meaningless (and more?)

2023-04-10 Thread Pavel Luzanov
On 05.04.2023 03:41, Melanie Plageman wrote: On Tue, Apr 4, 2023 at 4:35 PM Pavel Luzanov wrote: After a little thought... I'm not sure about the term 'bootstrap process'. I can't find this term in the documentation. There are various mentions of "bootstrap" peppered throughout the docs but n

eclg -C ORACLE breaks data

2023-04-10 Thread Kyotaro Horiguchi
Hello, we encountered unexpected behavior from an ECPG program complied with the -C ORACLE option. The program executes the following query: SELECT 123::numeric(3,0), 't'::char(2)"; Compilation and execution steps: $ ecpg -C ORACLE ecpgtest.pgc (attached) $ gcc -g -o ecpgtest ecpgtest.c -L `pg

Fix the miss consideration of tuple_fraction during add_paths_to_append_rel

2023-04-10 Thread Andy Fan
When I am working on "Pushing limit into subqueries of a union" [1], I found we already have a great infrastructure to support this. For a query like subquery-1 UNION ALL subquery-2 LIMIT 3; We have considered the root->tuple_fraction when planning the subqueries without an extra Limit node as an

RE: [PoC] pg_upgrade: allow to upgrade publisher node

2023-04-10 Thread Hayato Kuroda (Fujitsu)
Dear Julien, > Well, even if physical replication slots were eventually preserved during > pg_upgrade, maybe users would like to only keep one kind of the others so > having both options could make sense. You meant to say that we can rename options like "logical-*" and later add a new option for

RE: [PoC] pg_upgrade: allow to upgrade publisher node

2023-04-10 Thread Hayato Kuroda (Fujitsu)
Dear Julien, Thank you for giving idea! I have analyzed about it. > > > If > > > yes, how does this work if some subscriber node isn't connected when the > > > publisher node is stopped? I guess you could add a check in pg_upgrade to > make > > > sure that all logical slot are indeed caught up a

Re: Using each rel as both outer and inner for JOIN_ANTI

2023-04-10 Thread Richard Guo
On Fri, Apr 7, 2023 at 3:28 PM Richard Guo wrote: > On Tue, Aug 2, 2022 at 3:13 PM Richard Guo wrote: > >> On Sun, Jul 31, 2022 at 12:07 AM Tom Lane wrote: >> >>> [ wanders away wondering if JOIN_RIGHT_SEMI should become a thing ... ] >> >> Maybe this is something we can do. Currently for the q

RE: Support logical replication of DDLs

2023-04-10 Thread Zhijie Hou (Fujitsu)
On Friday, April 7, 2023 11:23 amhouzj.f...@fujitsu.com wrote: > > On Friday, April 7, 2023 11:13 AM houzj.f...@fujitsu.com > > > > > On Tuesday, April 4, 2023 7:35 PM shveta malik > > > > wrote: > > > > > > On Tue, Apr 4, 2023 at 8:43 AM houzj.f...@fujitsu.com > > > wrote: > > > > > > > Atta

Re: Support logical replication of DDLs

2023-04-10 Thread Amit Kapila
On Fri, Apr 7, 2023 at 8:52 AM houzj.f...@fujitsu.com wrote: > > Sorry, there was a miss when rebasing the patch which could cause the > CFbot to fail and here is the correct patch set. > I see the following note in the patch: "Note: For ATTACH/DETACH PARTITION, we haven't added extra logic on th

Re: Unnecessary confirm work on logical replication

2023-04-10 Thread Ashutosh Bapat
On Fri, Apr 7, 2023 at 11:06 PM Emre Hasegeli wrote: > > I was reading the logical replication code and found a little > unnecessary work we are doing. > > The confirmed_flushed_lsn cannot reasonably be ahead of the > current_lsn, so there is no point of calling > LogicalConfirmReceivedLocation()

Re: Fix the miss consideration of tuple_fraction during add_paths_to_append_rel

2023-04-10 Thread Zhang Mingli
HI, On Apr 10, 2023, 16:35 +0800, Andy Fan , wrote: > When I am working on "Pushing limit into subqueries of a union" [1], I > found we already have a great infrastructure to support this. For a query > like > > subquery-1 UNION ALL subquery-2 LIMIT 3; > > We have considered the root->tuple_fract

Re: longfin missing gssapi_ext.h

2023-04-10 Thread Stephen Frost
Greetings, * Thomas Munro (thomas.mu...@gmail.com) wrote: > On Sun, Apr 9, 2023 at 6:40 AM Tom Lane wrote: > > The exact same thing applies to FreeBSD, except that their in-core > > Heimdal is ancient (1.5.2). Also, they do have MIT Kerberos > > available as a package [1]. I'd been misled by th

Re: longfin missing gssapi_ext.h

2023-04-10 Thread Tom Lane
Stephen Frost writes: > Yeah, I wouldn't be the least bit surprised if many folks running > FreeBSD with any interest in Kerberos have MIT Kerberos installed given > that Heimdal doesn't seem to be under any kind of ongoing active > development and is just in this maintenance mode. Yeah, that's a

Re: longfin missing gssapi_ext.h

2023-04-10 Thread Stephen Frost
Greetings, * Tom Lane (t...@sss.pgh.pa.us) wrote: > Stephen Frost writes: > > Yeah, I wouldn't be the least bit surprised if many folks running > > FreeBSD with any interest in Kerberos have MIT Kerberos installed given > > that Heimdal doesn't seem to be under any kind of ongoing active > > deve

Re: longfin missing gssapi_ext.h

2023-04-10 Thread Tom Lane
Stephen Frost writes: > * Tom Lane (t...@sss.pgh.pa.us) wrote: >> IOW, maybe it'd be okay to de-revert 3d4fa227b and add documentation >> saying that --with-gssapi requires MIT Kerberos not Heimdal. > I'd be happy with that and can add the appropriate documentation noting > that we require MIT Ke

Re: Non-superuser subscription owners

2023-04-10 Thread Robert Haas
On Sat, Apr 8, 2023 at 1:35 AM Amit Kapila wrote: > Do we need to have a check for this new option "password_required" in > maybe_reread_subscription() where we "Exit if any parameter that > affects the remote connection was changed."? This new option is > related to the remote connection so I tho

Re: min/max aggregation for jsonb

2023-04-10 Thread Daneel Yaitskov
Nonetheless PostgreSQL min/max functions don't work with JSON - array_agg distinct does! I was working on an experimental napkin audit feature. It rewrites a chain of SQL queries to thread through meta data about all computations contributed to every column. Every data column gets a meta column wi

Re: PGDOCS - function pg_get_publication_tables is not documented?

2023-04-10 Thread Tom Lane
"Yu Shi (Fujitsu)" writes: > On Fri, Mar 24, 2023 6:26 AM Tom Lane wrote: >> I do see a docs change that I think would be worth making: get >> rid of the explicit mention of it in create_subscription.sgml >> in favor of using that view. > I agree and I tried to modify the query to use the view.

Re: CREATE SUBSCRIPTION -- add missing tab-completes

2023-04-10 Thread Robert Haas
On Fri, Apr 7, 2023 at 9:29 AM Masahiko Sawada wrote: > I think that parameters used by the backend process when performing > CREATE SUBSCRIPTION belong to the first category. And other parameters > used by apply workers and tablesync workers belong to the second > category. Since slot_name is use

Re: When to drop src/tools/msvc support

2023-04-10 Thread Robert Haas
On Sat, Apr 8, 2023 at 3:30 PM Tom Lane wrote: > I guess I'd vote for pulling the trigger in v16 if we can get that > done by the end of April. Once we're close to beta I think it > must wait for v17 to open. I think that sounds reasonable. It would be to the project's advantage not to have to m

Re: is_superuser is not documented

2023-04-10 Thread Robert Haas
On Sat, Apr 8, 2023 at 10:54 AM Joseph Koshakow wrote: > is_superuser feels a little out of place in this file. All of > the options here apply to the entire PostgreSQL server, while > is_superuser only applies to the current session. The description of > this file says : > > > These options repor

Re: When to drop src/tools/msvc support

2023-04-10 Thread Tom Lane
Robert Haas writes: > However, if this is the direction we're going, we probably need to > give pgsql-packagers a heads up ASAP, because anybody who is still > relying on the MSVC system to build Windows binaries is presumably > going to need some time to adjust. If we rip out the build system > s

Re: When to drop src/tools/msvc support

2023-04-10 Thread Robert Haas
On Mon, Apr 10, 2023 at 12:56 PM Tom Lane wrote: > Robert Haas writes: > > However, if this is the direction we're going, we probably need to > > give pgsql-packagers a heads up ASAP, because anybody who is still > > relying on the MSVC system to build Windows binaries is presumably > > going to

Re: [PATCH] Allow Postgres to pick an unused port to listen

2023-04-10 Thread Robert Haas
On Fri, Apr 7, 2023 at 5:34 PM Yurii Rashkovskii wrote: > I'm trying to understand what's wrong with reading port from the pid file (if > Postgres writes information there, it's surely so that somebody can read it, > otherwise, why write it in the first placd)? The proposed solution uses > oper

Re: When to drop src/tools/msvc support

2023-04-10 Thread Dave Page
On Mon, 10 Apr 2023 at 18:34, Robert Haas wrote: > On Mon, Apr 10, 2023 at 12:56 PM Tom Lane wrote: > > Robert Haas writes: > > > However, if this is the direction we're going, we probably need to > > > give pgsql-packagers a heads up ASAP, because anybody who is still > > > relying on the MSVC

Re: Show various offset arrays for heap WAL records

2023-04-10 Thread Peter Geoghegan
On Sun, Apr 9, 2023 at 5:12 PM Peter Geoghegan wrote: > I noticed that the nbtree VACUUM and DELETE record types have their > update/xl_btree_update arrays output incorrectly. We cannot use the > generic array_desc() approach with xl_btree_update elements, because > they're variable-width elements

Re: Add index scan progress to pg_stat_progress_vacuum

2023-04-10 Thread Imseih (AWS), Sami
> + case 'P': /* Parallel progress reporting */ I kept this comment as-is but inside case code block I added more comments. This is to avoid cluttering up the one-liner comment. > + * Increase and report the number of index scans. Also, we reset the progress > + * counters. > The counters rese

Re: When to drop src/tools/msvc support

2023-04-10 Thread Magnus Hagander
On Mon, Apr 10, 2023 at 6:56 PM Tom Lane wrote: > > Robert Haas writes: > > However, if this is the direction we're going, we probably need to > > give pgsql-packagers a heads up ASAP, because anybody who is still > > relying on the MSVC system to build Windows binaries is presumably > > going to

Re: When to drop src/tools/msvc support

2023-04-10 Thread Tom Lane
Magnus Hagander writes: > Thus, +1 on actually keeping it up and dropping it immediately as v17 > opens, giving them a year of advantage. And probably updating the docs > (if anybody were to read them.. but at least then we tried) stating > that it's deprecated and will be removed in v17. Yeah, I

Re: Show various offset arrays for heap WAL records

2023-04-10 Thread Melanie Plageman
On Sun, Apr 9, 2023 at 8:12 PM Peter Geoghegan wrote: > > On Fri, Apr 7, 2023 at 4:46 PM Peter Geoghegan wrote: > > Pushed that one too. > > I noticed that the nbtree VACUUM and DELETE record types have their > update/xl_btree_update arrays output incorrectly. We cannot use the > generic array_de

Re: When to drop src/tools/msvc support

2023-04-10 Thread Andres Freund
Hi, On 2023-04-10 19:55:35 +0100, Dave Page wrote: > Projects other than the EDB installers use the MSVC build system - e.g. > pgAdmin uses it’s own builds of libpq and other tools (psql, pg_dump etc) > that are pretty heavily baked into a fully automated build system (even the > build servers and

Re: When to drop src/tools/msvc support

2023-04-10 Thread Andres Freund
Hi, On 2023-04-10 16:50:20 -0400, Tom Lane wrote: > Yeah, I think that's the only feasible answer at this point. > Maybe a month or two back we could have done differently, > but there's not a lot of runway now. > > Once we do drop src/tools/msvc from HEAD, we should make a point > of reminding -

Re: Add index scan progress to pg_stat_progress_vacuum

2023-04-10 Thread Andres Freund
Hi, On 2023-04-10 08:14:18 +0900, Michael Paquier wrote: > On Fri, Apr 07, 2023 at 12:01:17PM -0700, Andres Freund wrote: > > Why would it mean that? Parallel workers are updated together with the > > leader, > > so there's no compatibility issue? > > My point is that the callback system would s

Re: Show various offset arrays for heap WAL records

2023-04-10 Thread Peter Geoghegan
On Mon, Apr 10, 2023 at 3:04 PM Melanie Plageman wrote: > I took a look at the first patch even though you've pushed the bugfix > part. Any reason you didn't use array_desc() for the inner array (of > "ptids")? I find that following the pattern of using array_desc (when it > is correct, of course!

Re: When to drop src/tools/msvc support

2023-04-10 Thread Michael Paquier
On Mon, Apr 10, 2023 at 03:32:19PM -0700, Andres Freund wrote: > On IM Thomas made some point about CI - I wonder if we should add building 16 > with src/tools/msvc as an optional CI task? We can't enable it by default > (yet), because we'd not have enough resources to also run that for cfbot. Once

Re: When to drop src/tools/msvc support

2023-04-10 Thread Jonathan S. Katz
On 4/10/23 4:50 PM, Tom Lane wrote: Magnus Hagander writes: Thus, +1 on actually keeping it up and dropping it immediately as v17 opens, giving them a year of advantage. And probably updating the docs (if anybody were to read them.. but at least then we tried) stating that it's deprecated and w

Re: Should vacuum process config file reload more often

2023-04-10 Thread Melanie Plageman
On Fri, Apr 7, 2023 at 9:07 AM Melanie Plageman wrote: > > On Fri, Apr 7, 2023 at 2:53 AM Masahiko Sawada wrote: > > > > On Fri, Apr 7, 2023 at 8:08 AM Daniel Gustafsson wrote: > > > > > > > On 7 Apr 2023, at 00:12, Melanie Plageman > > > > wrote: > > > > > > > > On Thu, Apr 6, 2023 at 5:45 PM

Re: Show various offset arrays for heap WAL records

2023-04-10 Thread Melanie Plageman
On Mon, Apr 10, 2023 at 04:31:44PM -0700, Peter Geoghegan wrote: > On Mon, Apr 10, 2023 at 3:04 PM Melanie Plageman > wrote: > > > > I will say that the prefix of p in "ptid" makes it sound like pointer to > > a tid, which I don't believe is what you meant. > > I was thinking of the symbol name

Re: Show various offset arrays for heap WAL records

2023-04-10 Thread Peter Geoghegan
On Mon, Apr 10, 2023 at 5:23 PM Melanie Plageman wrote: > If you keep the name, I'd explain it briefly in a comment above the code > then -- for those of us who spend less time with btrees. It is a tool > that will be often used by developers, so it is not unreasonable to > assume they may read th

Re: Direct I/O

2023-04-10 Thread Andres Freund
Hi, On 2023-04-10 19:27:27 +1200, Thomas Munro wrote: > On Mon, Apr 10, 2023 at 2:57 PM Andres Freund wrote: > > Have you tried to write a reproducer for this that doesn't involve postgres? > > I tried a bit. I'll try harder soon. > > > ... What kernel version did you repro > > this on Thomas?

RE: running logical replication as the subscription owner

2023-04-10 Thread Shinoda, Noriyoshi (PN Japan FSIP)
Hi hackers, Thank you for developing a great feature. The following commit added a column to the pg_subscription catalog. https://git.postgresql.org/gitweb/?p=postgresql.git;a=commit;h=482675987bcdffb390ae735cfd5f34b485ae97c6 https://git.postgresql.org/gitweb/?p=postgresql.git;a=commit;h=c3afe

Re: Direct I/O

2023-04-10 Thread Andres Freund
Hi, On 2023-04-10 18:55:26 -0700, Andres Freund wrote: > On 2023-04-10 19:27:27 +1200, Thomas Munro wrote: > > On Mon, Apr 10, 2023 at 2:57 PM Andres Freund wrote: > > > Have you tried to write a reproducer for this that doesn't involve > > > postgres? > > > > I tried a bit. I'll try harder so

Re: Direct I/O

2023-04-10 Thread Thomas Munro
On Tue, Apr 11, 2023 at 2:15 PM Andres Freund wrote: > And the fix has been merged into > https://git.kernel.org/pub/scm/linux/kernel/git/kdave/linux.git/log/?h=for-next > > I think that means it'll have to wait for 6.4 development to open (in a few > weeks), and then will be merged into the stabl

Re: longfin missing gssapi_ext.h

2023-04-10 Thread Jonathan S. Katz
On 4/10/23 11:37 AM, Tom Lane wrote: Stephen Frost writes: * Tom Lane (t...@sss.pgh.pa.us) wrote: IOW, maybe it'd be okay to de-revert 3d4fa227b and add documentation saying that --with-gssapi requires MIT Kerberos not Heimdal. I'd be happy with that and can add the appropriate documentatio

Re: longfin missing gssapi_ext.h

2023-04-10 Thread Thomas Munro
On Tue, Apr 11, 2023 at 2:31 AM Stephen Frost wrote: > Have you tried running the tests in src/test/kerberos with elver? Or is > it configured to run them? Would be awesome if it could be, or if > there's issues with running the tests on FBSD w/ MIT Kerberos, I'd be > happy to try and help work

Re: Direct I/O

2023-04-10 Thread Thomas Munro
On Tue, Apr 11, 2023 at 2:31 PM Thomas Munro wrote: > I tried to find out what POSIX says about this (But of course whatever it might say is of especially limited value when O_DIRECT is in the picture, being completely unstandardised. Really I guess all they meant was "if you *copy* something tha

Can we rely on the ordering of paths in pathlist?

2023-04-10 Thread Richard Guo
As the comment above add_path() says, 'The pathlist is kept sorted by total_cost, with cheaper paths at the front.' And it seems that get_cheapest_parallel_safe_total_inner() relies on this ordering (without being mentioned in the comments, which I think we should do). I'm wondering if this is the

RE: [PATCH] Support % wildcard in extension upgrade filenames

2023-04-10 Thread Regina Obe
> On Mon, Apr 03, 2023 at 09:26:25AM +0700, Yurii Rashkovskii wrote: > > I want to chime in on the issue of lower-number releases that are > > released after higher-number releases. The way I see this particular > > problem is that we always put upgrade SQL files in release "packages," > > and they

Re: Can we rely on the ordering of paths in pathlist?

2023-04-10 Thread Andy Fan
On Tue, Apr 11, 2023 at 11:03 AM Richard Guo wrote: > As the comment above add_path() says, 'The pathlist is kept sorted by > total_cost, with cheaper paths at the front.' And it seems that > get_cheapest_parallel_safe_total_inner() relies on this ordering > (without being mentioned in the comme

RE: Support logical replication of DDLs

2023-04-10 Thread Yu Shi (Fujitsu)
On Fri, Apr 7, 2023 11:23 AM houzj.f...@fujitsu.com wrote: > > On Friday, April 7, 2023 11:13 AM houzj.f...@fujitsu.com > > > > > On Tuesday, April 4, 2023 7:35 PM shveta malik > > wrote: > > > > > > On Tue, Apr 4, 2023 at 8:43 AM houzj.f...@fujitsu.com > > > wrote: > > > > > > > Attach the n

RE: Support logical replication of DDLs

2023-04-10 Thread Wei Wang (Fujitsu)
On Fri, Apr 7, 2023 11:23 AM Hou, Zhijie/侯 志杰 wrote: > Thanks for updating the patch set. Here are some comments: 1. The function deparse_drop_command in 0001 patch and the function publication_deparse_ddl_command_end in 0002 patch. ``` +/* + * Handle deparsing of DROP commands. + * + * Verbose

Re: longfin missing gssapi_ext.h

2023-04-10 Thread Thomas Munro
On Tue, Apr 11, 2023 at 2:53 PM Thomas Munro wrote: > On Tue, Apr 11, 2023 at 2:31 AM Stephen Frost wrote: > > Have you tried running the tests in src/test/kerberos with elver? Or is > > it configured to run them? Would be awesome if it could be, or if > > there's issues with running the tests

Re: Minimal logical decoding on standbys

2023-04-10 Thread Noah Misch
On Fri, Apr 07, 2023 at 11:12:26AM -0700, Andres Freund wrote: > --- /dev/null > +++ b/src/test/recovery/t/035_standby_logical_decoding.pl > @@ -0,0 +1,720 @@ > +# logical decoding on standby : test logical decoding, > +# recovery conflict and standby promotion. ... > +$node_primary->append_conf('p

Re: "an SQL" vs. "a SQL"

2023-04-10 Thread David Rowley
On Fri, 11 Jun 2021 at 13:44, David Rowley wrote: > Anyway, I'll set an alarm for this time next year so I can check on > how many inconsistencies have crept back in over the development > cycle. That alarm went off today. There seem to be only 3 "a SQL"s in the docs to change to "an SQL". This

Re: Fix fseek() detection of unseekable files on WIN32

2023-04-10 Thread Michael Paquier
On Mon, Mar 20, 2023 at 07:06:22AM +0900, Michael Paquier wrote: > Not sure about this one. I have considered it and dirmod.c includes > also bits for cygwin, while being aimed for higher-level routines like > rename(), unlink() or symlink(). This patch is only for WIN32, and > aimed for common p

Add information about command path and version of flex in meson output

2023-04-10 Thread Michael Paquier
Hi all, While doing a few things on Windows with meson, I have noticed that, while we output some information related to bison after a setup step, there is nothing about flex. I think that adding something about flex in the "Programs" section would be pretty useful, particularly for Windows as th

Re: "an SQL" vs. "a SQL"

2023-04-10 Thread Michael Paquier
On Tue, Apr 11, 2023 at 05:43:04PM +1200, David Rowley wrote: > That alarm went off today. > > There seem to be only 3 "a SQL"s in the docs to change to "an SQL". > > This is a pretty old thread, so here's a link [1] to the discussion. Good catches! -- Michael signature.asc Description: PGP si

Re: pg_recvlogical prints bogus error when interrupted

2023-04-10 Thread Michael Paquier
On Mon, Oct 24, 2022 at 08:15:11AM +0530, Bharath Rupireddy wrote: > The attached patch (pg_recvlogical_graceful_interrupt.text) has a > couple of problems, I believe. We're losing prepareToTerminate() with > keepalive true and we're not skipping pg_log_error("unexpected > termination of replicatio

cfbot is listing committed patches?

2023-04-10 Thread Peter Smith
cfbot [1] is listing some already committed patches under the "Needs Review" category. For example here are some of mine [1][2]. And because they are already committed, the 'apply' fails, so they get flagged by cfbot as needed rebase. Something seems broken. -- [1] http://cfbot.cputube.org/n

Re: Improve logging when using Huge Pages

2023-04-10 Thread Michael Paquier
On Thu, Mar 23, 2023 at 05:25:46PM +0900, Kyotaro Horiguchi wrote: > Wouldn't storing the value in the shared memory itself work? Though, > that space does become almost dead for the server's lifetime... Sure, it would work. However, we'd still need an interface for the extra function. At this p

Re: cfbot is listing committed patches?

2023-04-10 Thread Thomas Munro
On Tue, Apr 11, 2023 at 6:16 PM Peter Smith wrote: > cfbot [1] is listing some already committed patches under the "Needs > Review" category. For example here are some of mine [1][2]. And > because they are already committed, the 'apply' fails, so they get > flagged by cfbot as needed rebase. > >