Re: [PoC] Let libpq reject unexpected authentication requests

2023-03-13 Thread Michael Paquier
On Mon, Mar 13, 2023 at 12:38:10PM -0700, Jacob Champion wrote: > Here's a v16: > - updated 0001 patch message > - all test names should have commas rather than colons now > - new test for an empty require_auth > - new SSPI suite (note that it doesn't run by default on Cirrus, due > to the use of

Re: Combine pg_walinspect till_end_of_wal functions with others

2023-03-13 Thread Bharath Rupireddy
On Tue, Mar 14, 2023 at 5:02 AM Michael Paquier wrote: > > So let's be clean and drop these slots to keep the tests > self-contained. pg_walinspect in REL_15_STABLE gets that right, IMV, > and that's no different from the role cleanup, as one example. Hm, added replication slot drop back. > >

Re: Improve logging when using Huge Pages

2023-03-13 Thread Kyotaro Horiguchi
At Mon, 13 Mar 2023 21:33:31 +0100, Stephen Frost wrote in > > On Thu, Mar 09, 2023 at 03:02:29PM -0500, Stephen Frost wrote: > > > * Justin Pryzby (pry...@telsasoft.com) wrote: > > > Is there an agreement to use a function, instead ? > > > > Alvaro was -1 on using a function > > > I don’t

Re: psql \watch 2nd argument: iteration count

2023-03-13 Thread Kyotaro Horiguchi
At Tue, 14 Mar 2023 11:36:17 +0900, Michael Paquier wrote in > Ok, thanks for looking. Let's wait a bit and see if others have an > opinion to offer. At least, the CI is green. + if (*opt_end) + pg_log_error("\\watch:

Re: CI and test improvements

2023-03-13 Thread Justin Pryzby
On Mon, Mar 13, 2023 at 07:39:52AM +0100, Peter Eisentraut wrote: > On 03.02.23 15:26, Justin Pryzby wrote: > > rebased, and re-including a patch to show code coverage of changed > > files. > > This constant flow of patches under one subject doesn't lend itself well to > the commit fest model of

Re: Improve WALRead() to suck data directly from WAL buffers when possible

2023-03-13 Thread Bharath Rupireddy
On Sun, Mar 12, 2023 at 11:00 PM Bharath Rupireddy wrote: > > I have some review comments to fix on > v8-0001, so, I'll be sending out v9 patch-set soon. Please find the attached v9 patch set for further review. I moved the check for just-initialized WAL buffer pages before reading the page. Up

Re: Allow logical replication to copy tables in binary format

2023-03-13 Thread Amit Kapila
On Tue, Mar 14, 2023 at 6:18 AM Peter Smith wrote: > > On Tue, Mar 14, 2023 at 11:06 AM Peter Smith wrote: > > > > Here are some review comments for patch v12-0001 > > > > == > > General > > > > 1. > > There is no new test code. Are we sure that there are already > > sufficient TAP tests

Re: Sub-millisecond [autovacuum_]vacuum_cost_delay broken

2023-03-13 Thread Thomas Munro
On Tue, Mar 14, 2023 at 12:10 PM Nathan Bossart wrote: > > * NOTE: although the delay is specified in microseconds, the effective > > - * resolution is only 1/HZ, or 10 milliseconds, on most Unixen. Expect > > - * the requested delay to be rounded up to the next resolution boundary. > > + *

Re: psql \watch 2nd argument: iteration count

2023-03-13 Thread Michael Paquier
On Mon, Mar 13, 2023 at 06:14:18PM -0700, Andrey Borodin wrote: > Looks good to me. Ok, thanks for looking. Let's wait a bit and see if others have an opinion to offer. At least, the CI is green. -- Michael signature.asc Description: PGP signature

Re: Allow logical replication to copy tables in binary format

2023-03-13 Thread Peter Smith
Here are some review comments for patch v12-0001 (test code only) == src/test/subscription/t/014_binary.pl # Check the synced data on subscribers ~ There are a couple of comments like the above that say: "on subscribers" instead of "on subscriber". ~~~ I wondered if it might be useful to

Re: Show various offset arrays for heap WAL records

2023-03-13 Thread Peter Geoghegan
On Mon, Mar 13, 2023 at 4:01 PM Melanie Plageman wrote: > On Fri, Jan 27, 2023 at 3:02 PM Robert Haas wrote: > > I'm not sure what's best in terms of formatting details but I > > definitely like the idea of making pg_waldump show more details. > If I'm not mistaken, this would be quite

Re: psql \watch 2nd argument: iteration count

2023-03-13 Thread Andrey Borodin
On Mon, Mar 13, 2023 at 5:26 PM Michael Paquier wrote: > > I have tweaked things as bit as of the attached, and ran pgindent. > What do you think? > Looks good to me. Thanks! Best regards, Andrey Borodin.

Re: ICU 54 and earlier are too dangerous

2023-03-13 Thread Andres Freund
Hi, On 2023-03-13 16:39:04 -0700, Jeff Davis wrote: > In ICU 54 and earlier, if ucol_open() is unable to find a matching > locale, it will fall back to the *environment*. > > Using ICU 54: > > initdb -D data -N --locale="en_US.UTF-8" > pg_ctl -D data -l logfile start > psql postgres -c

Re: Allow logical replication to copy tables in binary format

2023-03-13 Thread Peter Smith
On Tue, Mar 14, 2023 at 11:06 AM Peter Smith wrote: > > Here are some review comments for patch v12-0001 > > == > General > > 1. > There is no new test code. Are we sure that there are already > sufficient TAP tests doing binary testing with/without copy_data and > covering all the necessary

Re: windows CI failing PMSignalState->PMChildFlags[slot] == PM_CHILD_ASSIGNED

2023-03-13 Thread Andres Freund
On 2023-03-14 09:29:56 +0900, Michael Paquier wrote: > The emulation of waitpid() for WIN32 is now in postmaster.c. Could it > make sense for some of the frontend code to be able to rely on that, > as well? Please not as part of this bugfix. It's intricately tied to postmaster.c specific code,

Re: windows CI failing PMSignalState->PMChildFlags[slot] == PM_CHILD_ASSIGNED

2023-03-13 Thread Michael Paquier
On Tue, Mar 14, 2023 at 01:01:28PM +1300, Thomas Munro wrote: > Ahhh. Right, of course. The handle thing makes total sense now that > you point it out, and although I couldn't find it in the fine manual, > a higher authority has it in black and white[1]. Even without knowing > which of those

Re: ICU 54 and earlier are too dangerous

2023-03-13 Thread Tom Lane
Jeff Davis writes: > In ICU 54 and earlier, if ucol_open() is unable to find a matching > locale, it will fall back to the *environment*. That's not great, but ... > Unless someone has a better idea, I think we need to bump the minimum > required ICU version to 55. That would solve the issue in

Re: psql \watch 2nd argument: iteration count

2023-03-13 Thread Michael Paquier
On Sun, Mar 12, 2023 at 08:59:44PM -0700, Andrey Borodin wrote: > I've tried this approach, but could not come up with sufficiently > different error messages... > >> Wouldn't it be better to have a couple of regression >> tests, as well? > Added two tests. It should have three tests with one

Re: Allow logical replication to copy tables in binary format

2023-03-13 Thread Peter Smith
Here are some review comments for patch v12-0001 == General 1. There is no new test code. Are we sure that there are already sufficient TAP tests doing binary testing with/without copy_data and covering all the necessary combinations? == Commit Message 2. Without this patch, table are

Re: windows CI failing PMSignalState->PMChildFlags[slot] == PM_CHILD_ASSIGNED

2023-03-13 Thread Thomas Munro
On Tue, Mar 14, 2023 at 11:20 AM Andres Freund wrote: > On windows it looks like pids can't be reused as long as there are handles for > the process. Unfortunately, we close the handle for the process in > pgwin32_deadchild_callback(), which runs in a separate thread, so the pid can > be reused

Re: pg_dump/pg_restore: Fix stdin/stdout handling of custom format on Win32

2023-03-13 Thread Michael Paquier
On Mon, Mar 13, 2023 at 05:49:41PM +0100, Juan José Santamaría Flecha wrote: > WFM, making fseek() behaviour more resilient seems like a good improvement > overall. I have not looked in details, but my guess would be to add a win32seek.c similar to win32stat.c with a port of fseek() that's more

Re: pg_dump versus hash partitioning

2023-03-13 Thread Tom Lane
Julien Rouhaud writes: > On Sun, Mar 12, 2023 at 03:46:52PM -0400, Tom Lane wrote: >> The trick is to detect in pg_restore whether pg_dump chose to do >> load-via-partition-root. > Given that this approach wouldn't help with existing dump files (at least if > using COPY, in any case the one

ICU 54 and earlier are too dangerous

2023-03-13 Thread Jeff Davis
In ICU 54 and earlier, if ucol_open() is unable to find a matching locale, it will fall back to the *environment*. Using ICU 54: initdb -D data -N --locale="en_US.UTF-8" pg_ctl -D data -l logfile start psql postgres -c "create collation asdf(provider=icu, locale='asdf')" # returns true

Re: Combine pg_walinspect till_end_of_wal functions with others

2023-03-13 Thread Michael Paquier
On Mon, Mar 13, 2023 at 03:53:37PM +0530, Bharath Rupireddy wrote: > On Mon, Mar 13, 2023 at 12:26 PM Michael Paquier wrote: >> +-- Make sure checkpoints don't interfere with the test. >> +SELECT 'init' FROM >> pg_create_physical_replication_slot('regress_pg_walinspect_slot', true, >> false);

Re: Testing autovacuum wraparound (including failsafe)

2023-03-13 Thread Peter Geoghegan
On Mon, Mar 13, 2023 at 3:25 PM Jacob Champion wrote: > Does https://commitfest.postgresql.org/42/4128/ address that > independently enough? I wasn't aware of that patch. It looks like it does exactly what I was arguing in favor of. So yes. -- Peter Geoghegan

Re: lz4 --rm on Ubuntu 18.04 (Add LZ4 compression to pg_dump)

2023-03-13 Thread Tomas Vondra
On 3/9/23 19:00, Tomas Vondra wrote: > > > On 3/9/23 01:30, Michael Paquier wrote: >> On Thu, Mar 09, 2023 at 12:39:08AM +0100, Tomas Vondra wrote: >>> IMO we should fix that. We have a bunch of buildfarm members running on >>> Ubuntu 18.04 (or older) - it's true none of them seems to be

Re: Sub-millisecond [autovacuum_]vacuum_cost_delay broken

2023-03-13 Thread Nathan Bossart
> * NOTE: although the delay is specified in microseconds, the effective > - * resolution is only 1/HZ, or 10 milliseconds, on most Unixen. Expect > - * the requested delay to be rounded up to the next resolution boundary. > + * resolution is only 1/HZ on systems that use periodic kernel ticks

Re: Show various offset arrays for heap WAL records

2023-03-13 Thread Melanie Plageman
Thanks for the various perspectives and feedback. Attached v2 has additional info for xl_btree_vacuum and xl_btree_delete. I've quoted various emails by various senders below and replied. On Fri, Jan 27, 2023 at 3:02 PM Robert Haas wrote: > > On Fri, Jan 27, 2023 at 12:24 PM Melanie Plageman >

Re: meson: Non-feature feature options

2023-03-13 Thread Nathan Bossart
On Mon, Mar 13, 2023 at 02:45:29PM -0700, Andres Freund wrote: > Pushed the patch. Thanks for the prompt fix. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: Testing autovacuum wraparound (including failsafe)

2023-03-13 Thread Jacob Champion
On Sat, Mar 11, 2023 at 8:47 PM Peter Geoghegan wrote: > I was joking. But I did have a real point: once we have tests for the > xidStopLimit mechanism, why not take the opportunity to correct the > long standing issue with the documentation advising the use of single > user mode? Does

Re: windows CI failing PMSignalState->PMChildFlags[slot] == PM_CHILD_ASSIGNED

2023-03-13 Thread Andres Freund
Hi, On 2023-03-13 17:00:00 +0300, Alexander Lakhin wrote: > 12.03.2023 10:18, Thomas Munro wrote: > > And again: > > > > TRAP: failed Assert("PMSignalState->PMChildFlags[slot] == > > PM_CHILD_ASSIGNED"), File: "../src/backend/storage/ipc/pmsigna... > > > >

Re: Microsecond-based timeouts

2023-03-13 Thread Andres Freund
Hi, On 2023-03-13 18:23:02 +1300, Thomas Munro wrote: > One question is whether it'd be better to use nanoseconds instead, > since the relevant high-resolution primitives use those under the > covers (struct timespec). On the other hand, microseconds are a good > match for our TimestampTz which

RE: Ability to reference other extensions by schema in extension scripts

2023-03-13 Thread Regina Obe
> On Sat, Mar 11, 2023 at 03:18:18AM -0500, Regina Obe wrote: > > Attached is a revised patch with these changes in place. > > I've given a try to this patch. It builds and regresses fine. > > My own tests also worked fine. As long as ext1 was found in the ext2's > no_relocate list it could not

Re: Add LZ4 compression in pg_dump

2023-03-13 Thread Tomas Vondra
Hi Justin, Thanks for the patch. On 3/8/23 02:45, Justin Pryzby wrote: > On Wed, Mar 01, 2023 at 04:52:49PM +0100, Tomas Vondra wrote: >> Thanks. That seems correct to me, but I find it somewhat confusing, >> because we now have >> >> DeflateCompressorInit vs. InitCompressorGzip >> >>

Re: meson: Non-feature feature options

2023-03-13 Thread Andres Freund
Hi, On 2023-03-13 23:46:41 +0300, Nazir Bilal Yavuz wrote: > Thanks for the patch, I understand the problem now and your patch fixes this. Pushed the patch. Greetings, Andres Freund

Re: optimize several list functions with SIMD intrinsics

2023-03-13 Thread Nathan Bossart
Thanks for taking a look. On Sat, Mar 11, 2023 at 09:41:18AM +, Ankit Kumar Pandey wrote: > 1. In list_member_ptr, will it be okay to bring `const ListCell *cell` from > #ifdef USE_NO_SIMD > const ListCell *cell; > #endif > to #else like as mentioned below? This will make visual

Re: Operation log for major operations

2023-03-13 Thread Dmitry Koval
Kirk, I'm sorry about the long pause in my reply. >We need some kind of semaphore flag that tells us something awkward >happened. When it happened, and a little bit of extra information. I agree that we do not have this kind of information. Additionally, legal events like start of pg_rewind,

Re: [BUG] pg_stat_statements and extended query protocol

2023-03-13 Thread David Zhang
It appears you must "make clean; make install" to correctly compile after applying the patch. In a git repository, I've learnt to rely on this simple formula, even if it means extra cycles when running ./configure: git clean -d -x -f Thank you all for pointing out that it needs make clean

Re: pg_usleep for multisecond delays

2023-03-13 Thread Nathan Bossart
On Fri, Mar 10, 2023 at 12:28:28PM -0500, Tom Lane wrote: > A quick grep for pg_usleep with large intervals finds rather more > than you touched: > > [...] > > Did you have reasons for excluding the rest of these? I'm still looking into each of these, but my main concerns were 1) ensuring latch

Re: Transparent column encryption

2023-03-13 Thread Andres Freund
Hi, On 2023-03-13 13:41:19 -0700, Andres Freund wrote: > On 2023-03-13 21:22:29 +0100, Peter Eisentraut wrote: > > On 12.03.23 01:11, Andres Freund wrote: > > > Have you done benchmarks of some simple workloads to verify this doesn't > > > cause > > > slowdowns (when not using encryption,

Re: meson: Non-feature feature options

2023-03-13 Thread Nazir Bilal Yavuz
Hi, On Mon, 13 Mar 2023 at 23:14, Andres Freund wrote: > > Hi, > > On 2023-03-13 21:57:22 +0300, Nazir Bilal Yavuz wrote: > > On Mon, 13 Mar 2023 at 21:04, Nathan Bossart > > wrote: > > > > > > On Mon, Mar 13, 2023 at 07:27:18AM +0100, Peter Eisentraut wrote: > > > > I have committed it like

Re: Transparent column encryption

2023-03-13 Thread Andres Freund
Hi, On 2023-03-13 21:22:29 +0100, Peter Eisentraut wrote: > On 12.03.23 01:11, Andres Freund wrote: > > Have you done benchmarks of some simple workloads to verify this doesn't > > cause > > slowdowns (when not using encryption, obviously)? printtup.c is a > > performance > > sensitive portion

Re: Improve logging when using Huge Pages

2023-03-13 Thread Stephen Frost
Greetings, On Mon, Mar 13, 2023 at 21:03 Justin Pryzby wrote: > On Thu, Mar 09, 2023 at 03:02:29PM -0500, Stephen Frost wrote: > > * Justin Pryzby (pry...@telsasoft.com) wrote: > > > On Thu, Mar 09, 2023 at 09:34:10AM -0500, Stephen Frost wrote: > > > > * Nathan Bossart

Re: Request for comment on setting binary format output per session

2023-03-13 Thread Dave Cramer
Dave Cramer On Sat, 4 Mar 2023 at 19:39, Dave Cramer wrote: > > > On Sat, 4 Mar 2023 at 19:06, Tom Lane wrote: > >> Jeff Davis writes: >> > On Sat, 2023-03-04 at 18:04 -0500, Dave Cramer wrote: >> >> Most of the clients know how to decode the builtin types. I'm not >> >> sure there is a use

Re: meson: Non-feature feature options

2023-03-13 Thread Nathan Bossart
On Mon, Mar 13, 2023 at 01:13:31PM -0700, Andres Freund wrote: > On 2023-03-13 11:04:32 -0700, Nathan Bossart wrote: >> I noticed that after 6a30027, if you don't have the OpenSSL headers >> installed, 'meson setup' will fail: >> >> meson.build:1195:4: ERROR: C header 'openssl/ssl.h' not

Re: Transparent column encryption

2023-03-13 Thread Peter Eisentraut
On 12.03.23 01:11, Andres Freund wrote: Have you done benchmarks of some simple workloads to verify this doesn't cause slowdowns (when not using encryption, obviously)? printtup.c is a performance sensitive portion for simple queries, particularly when they return multiple columns. The

Re: Add LZ4 compression in pg_dump

2023-03-13 Thread Tomas Vondra
On 3/11/23 11:50, gkokola...@pm.me wrote: > --- Original Message --- > On Saturday, March 11th, 2023 at 7:00 AM, Alexander Lakhin > wrote: > >> Hello, >> 23.02.2023 23:24, Tomas Vondra wrote: >> >>> On 2/23/23 16:26, Tomas Vondra wrote: >>> Thanks for v30 with the updated commit

Re: meson: Non-feature feature options

2023-03-13 Thread Andres Freund
Hi, On 2023-03-13 21:57:22 +0300, Nazir Bilal Yavuz wrote: > On Mon, 13 Mar 2023 at 21:04, Nathan Bossart wrote: > > > > On Mon, Mar 13, 2023 at 07:27:18AM +0100, Peter Eisentraut wrote: > > > I have committed it like this. > > > > I noticed that after 6a30027, if you don't have the OpenSSL

Re: meson: Non-feature feature options

2023-03-13 Thread Andres Freund
Hi, On 2023-03-13 11:04:32 -0700, Nathan Bossart wrote: > On Mon, Mar 13, 2023 at 07:27:18AM +0100, Peter Eisentraut wrote: > > I have committed it like this. > > I noticed that after 6a30027, if you don't have the OpenSSL headers > installed, 'meson setup' will fail: > >

Re: Improve logging when using Huge Pages

2023-03-13 Thread Justin Pryzby
On Thu, Mar 09, 2023 at 03:02:29PM -0500, Stephen Frost wrote: > * Justin Pryzby (pry...@telsasoft.com) wrote: > > On Thu, Mar 09, 2023 at 09:34:10AM -0500, Stephen Frost wrote: > > > * Nathan Bossart (nathandboss...@gmail.com) wrote: > > > > On Wed, Feb 15, 2023 at 10:13:17AM -0800, Nathan

Re: [PoC] Let libpq reject unexpected authentication requests

2023-03-13 Thread Jacob Champion
On Fri, Mar 10, 2023 at 3:16 PM Jacob Champion wrote: > > Could you send a new patch with all these adjustments? That would > > help a lot. > > Will do! Here's a v16: - updated 0001 patch message - all test names should have commas rather than colons now - new test for an empty require_auth -

Re: meson: Non-feature feature options

2023-03-13 Thread Nathan Bossart
On Mon, Mar 13, 2023 at 09:57:22PM +0300, Nazir Bilal Yavuz wrote: > On Mon, 13 Mar 2023 at 21:04, Nathan Bossart wrote: >> Shouldn't "auto" cause Postgres to be built without OpenSSL if the required >> headers are not present? > > Yes, I tested again and it is working as expected on my end. It

Re: meson: Non-feature feature options

2023-03-13 Thread Nazir Bilal Yavuz
Hi, On Mon, 13 Mar 2023 at 21:04, Nathan Bossart wrote: > > On Mon, Mar 13, 2023 at 07:27:18AM +0100, Peter Eisentraut wrote: > > I have committed it like this. > > I noticed that after 6a30027, if you don't have the OpenSSL headers > installed, 'meson setup' will fail: > >

RE: [PATCH] Support % wildcard in extension upgrade filenames

2023-03-13 Thread Regina Obe
> > I wonder if a solution to this problem might be to provide some kind > > of a version map file. Let's suppose that the map file is a bunch of > > lines of the form X Y Z, where X, Y, and Z are version numbers. The > > semantics could be: we (the extension authors) promise that if you > > want

Re: Reducing connection overhead in pg_upgrade compat check phase

2023-03-13 Thread Nathan Bossart
On Mon, Mar 13, 2023 at 03:10:58PM +0100, Daniel Gustafsson wrote: > The attached v3 is a rebase to handle conflicts and with the above comments > adressed. Thanks for the new version of the patch. I noticed that git-am complained when I applied the patch: Applying: pg_upgrade: run all data

Re: Lock mode in ExecMergeMatched()

2023-03-13 Thread Alvaro Herrera
On 2023-Mar-11, Alexander Korotkov wrote: > I wonder why does ExecMergeMatched() determine the lock mode using > ExecUpdateLockMode(). Why don't we use lock mode set by > table_tuple_update() like ExecUpdate() does? I skim through the > MERGE-related threads, but didn't find an answer. > > I

Re: meson: Non-feature feature options

2023-03-13 Thread Nathan Bossart
On Mon, Mar 13, 2023 at 07:27:18AM +0100, Peter Eisentraut wrote: > I have committed it like this. I noticed that after 6a30027, if you don't have the OpenSSL headers installed, 'meson setup' will fail: meson.build:1195:4: ERROR: C header 'openssl/ssl.h' not found Shouldn't "auto" cause

Re: Bug in jsonb_in function (14 & 15 version are affected)

2023-03-13 Thread Tom Lane
Nikolay Shaplov writes: > I found a bug in jsonb_in function (it converts json from sting representation > into jsonb internal representation). Yeah. Looks like json_lex_string is failing to honor the invariant that it needs to set token_terminator ... although the documentation of the

Re: [PATCH] Extend the length of BackgroundWorker.bgw_library_name

2023-03-13 Thread Yurii Rashkovskii
Nathan, Thank you for your review. Indeed, my motivation for doing the change the way I did it was that only bgw_library_name is expected to be longer, whereas it is much less of a concern for other fields. If we have increased BGW_MAXLEN, it would have increased the size of BackgroundWorker for

Re: Lock mode in ExecMergeMatched()

2023-03-13 Thread Alvaro Herrera
On 2023-Mar-13, Dean Rasheed wrote: > I'm wondering now if it really matters what lock mode we use here. If > the point of calling table_tuple_lock() after a concurrent update is > detected is to prevent more concurrent updates, so that the retry is > guaranteed to succeed, then wouldn't even

Re: [PATCH] Extend the length of BackgroundWorker.bgw_library_name

2023-03-13 Thread Nathan Bossart
On Mon, Mar 13, 2023 at 07:57:47AM -0700, Yurii Rashkovskii wrote: > However, there are use cases where [potentially] longer names are > expected/desired; for example, test benches (where library files may not > [or can not] be copied to Postgres installation) or alternative library > installation

Re: pg_dump/pg_restore: Fix stdin/stdout handling of custom format on Win32

2023-03-13 Thread Juan José Santamaría Flecha
On Fri, Mar 10, 2023 at 2:37 AM Michael Paquier wrote: > On Fri, Mar 10, 2023 at 12:12:37AM +0100, Juan José Santamaría Flecha > wrote: > > I've broken the patch in two: > > 1. fixes the detection of unseekable files in checkSeek(), using logic > that > > hopefully is backpatchable, > > 2. the

Re: ICU locale validation / canonicalization

2023-03-13 Thread Jeff Davis
On Mon, 2023-03-13 at 08:25 +0100, Peter Eisentraut wrote: > For clarification, I wasn't complaining about the notice, but about > the > automatic conversion from old-style ICU locale ID to language tag. Canonicalization means that we pick one format, and automatically convert to it, right? >

Re: [Proposal] Allow pg_dump to include all child tables with the root table

2023-03-13 Thread Gilles Darold
Le 12/03/2023 à 19:05, Stéphane Tachoires a écrit : Hi Gilles, On Ubuntu 22.04.2 all deb's updated, I have an error on a test I'll try to find where and why, but I think you should know first. 1/1 postgresql:pg_dump / pg_dump/002_pg_dump        ERROR            24.40s   exit status 1

[PATCH] Extend the length of BackgroundWorker.bgw_library_name

2023-03-13 Thread Yurii Rashkovskii
Hi, I want to suggest a patch against master (it may also be worth backporting it) that makes it possible to use longer filenames (such as those with absolute paths) in `BackgroundWorker.bgw_library_name`. `BackgroundWorker.bgw_library_name` currently allows names up to BGW_MAXLEN-1, which is

Re: [PoC] Improve dead tuple storage for lazy vacuum

2023-03-13 Thread Masahiko Sawada
On Mon, Mar 13, 2023 at 10:28 PM John Naylor wrote: > > On Mon, Mar 13, 2023 at 8:41 AM Masahiko Sawada wrote: > > > > On Sun, Mar 12, 2023 at 12:54 AM John Naylor > > wrote: > > > > > > On Fri, Mar 10, 2023 at 9:30 PM Masahiko Sawada > > > wrote: > > > > > * Additional size classes. It's

Re: Progress report of CREATE INDEX for nested partitioned tables

2023-03-13 Thread Tom Lane
Justin Pryzby writes: > On Sun, Mar 12, 2023 at 06:25:13PM -0400, Tom Lane wrote: >> I agree that adding such a field to IndexStmt would be a very bad idea. >> However, adding another parameter to DefineIndex doesn't seem like a >> problem. > It's a problem since this is a bug and it's desirable

RE: Ability to reference other extensions by schema in extension scripts

2023-03-13 Thread Regina Obe
> I've given a try to this patch. It builds and regresses fine. > > My own tests also worked fine. As long as ext1 was found in the ext2's > no_relocate list it could not be relocated, and proper error message is given > to user trying it. > > Nitpicking, there are a few things that are weird to

Re: buildfarm + meson

2023-03-13 Thread Andrew Dunstan
On 2023-03-10 Fr 18:05, Andres Freund wrote: Hi, On 2023-03-09 11:55:57 -0800, Andres Freund wrote: On 2023-03-09 14:47:36 -0500, Andrew Dunstan wrote: On 2023-03-09 Th 08:28, Andrew Dunstan wrote: At this stage I think I'm prepared to turn this loose on a couple of my buildfarm animals,

Bug in jsonb_in function (14 & 15 version are affected)

2023-03-13 Thread Nikolay Shaplov
Hi! I found a bug in jsonb_in function (it converts json from sting representation into jsonb internal representation). To reproduce this bug (the way I found it) you should get 8bit instance of postgres db: 1. add en_US locale (dpkg-reconfigure locales in debian) 2. initdb with latin1

Re: [PATCH] Use indexes on the subscriber when REPLICA IDENTITY is full on the publisher

2023-03-13 Thread Önder Kalacı
Hi Amit, all > > > > For that test, my goal was to ensure/show that the invalidation callback > > is triggered after `DROP / CREATE INDEX` commands. > > > > Fair point. I suggest in that case just keep one of the tests for Drop > Index such that after that it will pick up a sequence scan.

Re: Add support for DEFAULT specification in COPY FROM

2023-03-13 Thread Andrew Dunstan
On 2022-12-02 Fr 09:11, Israel Barth Rubio wrote: Hello all, I'm submitting a new version of the patch. Instead of changing signature of several functions in order to use the defaults parameter, it is now storing that in the cstate structure, which is already passed to all functions that

Re: Reducing connection overhead in pg_upgrade compat check phase

2023-03-13 Thread Daniel Gustafsson
> On 23 Feb 2023, at 15:12, Daniel Gustafsson wrote: > >> On 22 Feb 2023, at 20:20, Nathan Bossart wrote: > >> One thing I noticed is that the >> "failed check" log is only printed once, even if multiple data type checks >> failed. I believe this is because this message uses PG_STATUS. If I

Re: windows CI failing PMSignalState->PMChildFlags[slot] == PM_CHILD_ASSIGNED

2023-03-13 Thread Alexander Lakhin
12.03.2023 10:18, Thomas Munro wrote: And again: TRAP: failed Assert("PMSignalState->PMChildFlags[slot] == PM_CHILD_ASSIGNED"), File: "../src/backend/storage/ipc/pmsigna... https://cirrus-ci.com/task/6558324615806976

Re: Progress report of CREATE INDEX for nested partitioned tables

2023-03-13 Thread Justin Pryzby
On Sun, Mar 12, 2023 at 06:25:13PM -0400, Tom Lane wrote: > Justin Pryzby writes: > > On Sun, Mar 12, 2023 at 04:14:06PM -0400, Tom Lane wrote: > >> Hm. Could we get rid of count_leaf_partitions by doing the work in > >> ProcessUtilitySlow? Or at least passing that OID list forward instead > >>

Re: MERGE ... RETURNING

2023-03-13 Thread Dean Rasheed
On Sun, 26 Feb 2023 at 09:50, Dean Rasheed wrote: > > Another rebase. > And another rebase. Regards, Dean diff --git a/doc/src/sgml/dml.sgml b/doc/src/sgml/dml.sgml new file mode 100644 index cbbc5e2..ff2a827 --- a/doc/src/sgml/dml.sgml +++ b/doc/src/sgml/dml.sgml @@ -283,10 +283,15 @@ DELETE

Re: [PoC] Improve dead tuple storage for lazy vacuum

2023-03-13 Thread John Naylor
On Mon, Mar 13, 2023 at 8:41 AM Masahiko Sawada wrote: > > On Sun, Mar 12, 2023 at 12:54 AM John Naylor > wrote: > > > > On Fri, Mar 10, 2023 at 9:30 PM Masahiko Sawada wrote: > > > * Additional size classes. It's important for an alternative of path > > > compression as well as supporting our

Re: Refactoring SysCacheGetAttr to know when attr cannot be NULL

2023-03-13 Thread Daniel Gustafsson
> On 2 Mar 2023, at 15:44, Tom Lane wrote: > > Peter Eisentraut writes: >> I think an error message like >> "unexpected null value in system cache %d column %d" >> is sufficient. Since these are "can't happen" errors, we don't need to >> spend too much extra effort to make it prettier. >

Re: [PATCH] Use indexes on the subscriber when REPLICA IDENTITY is full on the publisher

2023-03-13 Thread Amit Kapila
On Mon, Mar 13, 2023 at 6:14 PM Önder Kalacı wrote: > >> >> >> 3. Removed the cases for dropping the index. This ensures that after >> dropping the index on the table we switch to either an index scan (if >> a new index is created) or to a sequence scan. It doesn't seem like a >> very interesting

Re: Dropped and generated columns might cause wrong data on subs when REPLICA IDENTITY FULL

2023-03-13 Thread Önder Kalacı
Hi Shi Yu, > 1. > @@ -243,6 +243,17 @@ tuples_equal(TupleTableSlot *slot1, TupleTableSlot > *slot2, > Form_pg_attribute att; > TypeCacheEntry *typentry; > > + > + Form_pg_attribute attr = > TupleDescAttr(slot1->tts_tupleDescriptor, attrnum); > + > >

Re: [PATCH] Use indexes on the subscriber when REPLICA IDENTITY is full on the publisher

2023-03-13 Thread Önder Kalacı
Hi Amit, Peter, all > > If the reason for the stats polling was only to know if some index is > > chosen or not, I was wondering if you can just convey the same > > information to the TAP test via some conveniently placed (DEBUG?) > > logging. > > > > I had thought about it but didn't convince

Re: Option to not use ringbuffer in VACUUM, using it in failsafe mode

2023-03-13 Thread Ants Aasma
On Sat, 11 Mar 2023 at 16:55, Melanie Plageman wrote: > > > On Tue, Feb 28, 2023 at 3:16 AM Bharath Rupireddy > > wrote: > > > > > On Thu, Jan 12, 2023 at 6:06 AM Andres Freund wrote: > > > > > > > > On 2023-01-11 17:26:19 -0700, David G. Johnston wrote: > > > > > Should we just add

Re: [PATCH] Support % wildcard in extension upgrade filenames

2023-03-13 Thread 'Sandro Santilli'
On Wed, Mar 08, 2023 at 03:18:06PM -0500, Regina Obe wrote: > > Then question arises if you have such a map file and you have files as well > (the old way). One idea I had in the past about the .control file was to advertise an executable that would take current version and next version and

Re: [PATCH] Support % wildcard in extension upgrade filenames

2023-03-13 Thread Sandro Santilli
On Wed, Mar 08, 2023 at 08:27:29AM -0500, Robert Haas wrote: > I wonder if a solution to this problem might be to provide some kind > of a version map file. Let's suppose that the map file is a bunch of > lines of the form X Y Z, where X, Y, and Z are version numbers. The > semantics could be: we

Re: Lock mode in ExecMergeMatched()

2023-03-13 Thread Dean Rasheed
> On Fri, 10 Mar 2023 at 21:42, Alexander Korotkov wrote: > > > > I wonder why does ExecMergeMatched() determine the lock mode using > > ExecUpdateLockMode(). Why don't we use lock mode set by > > table_tuple_update() like ExecUpdate() does? I skim through the > > MERGE-related threads, but

Re: [PATCH] Add pretty-printed XML output option

2023-03-13 Thread Jim Jones
On 10.03.23 15:32, Tom Lane wrote: Jim Jones writes: On 09.03.23 21:21, Tom Lane wrote: I've looked through this now, and have some minor complaints and a major one. The major one is that it doesn't work for XML that doesn't satisfy IS DOCUMENT. For example, How do you suggest the output

Re: Ability to reference other extensions by schema in extension scripts

2023-03-13 Thread 'Sandro Santilli'
On Sat, Mar 11, 2023 at 03:18:18AM -0500, Regina Obe wrote: > Attached is a revised patch with these changes in place. I've given a try to this patch. It builds and regresses fine. My own tests also worked fine. As long as ext1 was found in the ext2's no_relocate list it could not be relocated,

Re: [PATCH] Use indexes on the subscriber when REPLICA IDENTITY is full on the publisher

2023-03-13 Thread Amit Kapila
On Mon, Mar 13, 2023 at 2:44 AM Peter Smith wrote: > > On Sat, Mar 11, 2023 at 6:05 PM Amit Kapila wrote: > > > > On Fri, Mar 10, 2023 at 7:58 PM Önder Kalacı wrote: > > > > > > > > > I think one option could be to drop some cases altogether, but not sure > > > we'd want that. > > > > > > As a

RE: Dropped and generated columns might cause wrong data on subs when REPLICA IDENTITY FULL

2023-03-13 Thread shiy.f...@fujitsu.com
On Sun, Mar 12, 2023 4:00 AM Önder Kalacı wrote: > > Attaching a patch that could possibly solve the problem. > Thanks for your patch. I tried it and it worked well. Here are some minor comments. 1. @@ -243,6 +243,17 @@ tuples_equal(TupleTableSlot *slot1, TupleTableSlot *slot2,

Re: [PATCH] Use indexes on the subscriber when REPLICA IDENTITY is full on the publisher

2023-03-13 Thread Önder Kalacı
Hi Hou zj, Shi-san, all > In this function, it used the local column number(keycol) to match the > remote > column number(attkeys), I think it will cause problem if the column order > between pub/sub doesn't match. Like: > > --- > - pub > CREATE TABLE test_replica_id_full (x int, y int); >

Re: Allow logical replication to copy tables in binary format

2023-03-13 Thread Melih Mutlu
Hi, Attached v12 with a unified option. Setting binary = true now allows the initial sync to happen in binary format. Thanks, -- Melih Mutlu Microsoft v12-0001-Allow-logical-replication-to-copy-table-in-binary.patch Description: Binary data

Re: Combine pg_walinspect till_end_of_wal functions with others

2023-03-13 Thread Bharath Rupireddy
On Mon, Mar 13, 2023 at 12:26 PM Michael Paquier wrote: > > On Fri, Mar 10, 2023 at 04:45:06PM +0530, Bharath Rupireddy wrote: > > After that comes the rest of the patch, and I have found a couple of > mistakes. > > - pg_get_wal_records_info(start_lsn pg_lsn, end_lsn pg_lsn) > +

Re: Allow tests to pass in OpenSSL FIPS mode

2023-03-13 Thread Daniel Gustafsson
> On 13 Mar 2023, at 11:06, Peter Eisentraut > wrote: > On 06.03.23 17:06, Daniel Gustafsson wrote: >> fipshash() with an explanatory comments sounds like a good idea. > > committed like that +1. Looks like there is a just a slight diff in the compression.sql test suite. -- Daniel Gustafsson

Re: Allow tests to pass in OpenSSL FIPS mode

2023-03-13 Thread Peter Eisentraut
On 06.03.23 17:06, Daniel Gustafsson wrote: On 6 Mar 2023, at 15:55, Tom Lane wrote: Daniel Gustafsson writes: For readers without all context, wouldn't it be better to encode in the function name why we're not just calling a hash like md5? Something like fips_allowed_hash() or similar?

Re: [BUG] pg_stat_statements and extended query protocol

2023-03-13 Thread Drouvot, Bertrand
Hi, On 3/2/23 8:27 AM, Michael Paquier wrote: On Wed, Jan 25, 2023 at 11:22:04PM +, Imseih (AWS), Sami wrote: Doing some work with extended query protocol, I encountered the same issue that was discussed in [1]. It appears when a client is using extended query protocol and sends an Execute

Re: Assert failure of the cross-check for nullingrels

2023-03-13 Thread Richard Guo
On Mon, Mar 13, 2023 at 5:03 PM Richard Guo wrote: > Back to the original issue, if a join has more than one quals, actually > we treat them as a whole when we check if identity 3 applies as well as > when we adjust them to be suitable for commutation according to identity > 3. So when we check

RE: [PATCH] Use indexes on the subscriber when REPLICA IDENTITY is full on the publisher

2023-03-13 Thread houzj.f...@fujitsu.com
On Monday, March 13, 2023 2:23 PM Önder Kalacı wrote: Hi, > > > > > > > > > Reading [1], I think I can follow what you suggest. So, basically, > > > if the leftmost column is not filtered, we have the following: > > > > > >> but the entire index would have to be scanned, so in most cases the

Re: Assert failure of the cross-check for nullingrels

2023-03-13 Thread Richard Guo
On Fri, Mar 10, 2023 at 4:13 PM Richard Guo wrote: > I wonder if we should consider syn_xxxhand rather than min_xxxhand in > clause_is_computable_at when we check if clause mentions any nullable > Vars. But I'm not sure about that. > No, considering syn_xxxhand is not right. After some join

Re: pg_dump versus hash partitioning

2023-03-13 Thread Julien Rouhaud
On Sun, Mar 12, 2023 at 03:46:52PM -0400, Tom Lane wrote: > Julien Rouhaud writes: > > The BEGIN + TRUNCATE is only there to avoid generating WAL records just in > > case > > the wal_level is minimal. I don't remember if that optimization still > > exists, > > but if yes we could avoid doing

Re: [PATCH] Align GSS and TLS error handling in PQconnectPoll()

2023-03-13 Thread Michael Paquier
On Fri, Mar 10, 2023 at 10:42:08AM +0900, Michael Paquier wrote: > Perhaps you are right and there is no actual reason to worry here. I have been thinking about that for the last few days, and yes a backpatch should be OK, so done now down to 12. -- Michael signature.asc Description: PGP

  1   2   >