Re: [PoC] Improve dead tuple storage for lazy vacuum

2023-01-11 Thread John Naylor
I wrote: > I see. IIUC from a brief re-reading of the code, saving that chunk would only save us from re-loading "parent->shift" from L1 cache and shifting the key. The cycles spent doing that seem small compared to the rest of the work involved in growing a node. Expressions like "if (idx < 0)

Re: Time delayed LR (WAS Re: logical replication restrictions)

2023-01-11 Thread shveta malik
On Tue, Jan 10, 2023 at 7:42 PM Takamichi Osumi (Fujitsu) wrote: > > On Tuesday, January 3, 2023 4:01 PM vignesh C wrote: > Hi, thanks for your review ! > > > > 1) This global variable can be removed as it is used only in send_feedback > > which > > is called from maybe_delay_apply so we could

Re: Exposing the lock manager's WaitForLockers() to SQL

2023-01-11 Thread Will Mortensen
Hi Marco, thanks for the reply! Glad to know you'd find it useful too. :-) On Tue, Jan 10, 2023 at 1:01 AM Marco Slot wrote: > I'm wondering whether it could be an option of the LOCK command. > (LOCK WAIT ONLY?) I assume that's doable, but just from looking at the docs, it might be a little

Re: [EXTERNAL] Re: [PATCH] Support using "all" for the db user in pg_ident.conf

2023-01-11 Thread Jelte Fennema
> The confusion that 0001 is addressing is fair (cough, fc579e1, cough), > still I am wondering whether we could do a bit better to be more Yeah, even after 0001 it's definitely suboptimal. I tried to keep the changes minimal to not distract from the main purpose of this patch. But I'll update

Re: [PATCH] Support % wildcard in extension upgrade filenames

2023-01-11 Thread Sandro Santilli
On Tue, Jan 10, 2023 at 06:50:31PM -0500, Tom Lane wrote: > With the proposed % feature, if foo--%--3.0.sql exists then the > system will invoke it and expect the end result to be a valid > 3.0 installation, whether or not the script actually has any > ability to do a downgrade. It is sane,

Re: Time delayed LR (WAS Re: logical replication restrictions)

2023-01-11 Thread vignesh C
On Tue, 10 Jan 2023 at 19:41, Takamichi Osumi (Fujitsu) wrote: > > On Tuesday, January 3, 2023 4:01 PM vignesh C wrote: > Hi, thanks for your review ! > > Please have a look at the updated patch. Thanks for the updated patch, few comments: 1) Comment inconsistency across create and alter

Re: Add proper planner support for ORDER BY / DISTINCT aggregates

2023-01-11 Thread Richard Guo
On Wed, Jan 11, 2023 at 12:13 PM David Rowley wrote: > postgres=# set enable_presorted_aggregate=0; > SET > postgres=# select string_agg(random()::text, ',' order by random()) > from generate_series(1,3); > string_agg >

RE: Allow logical replication to copy tables in binary format

2023-01-11 Thread shiy.f...@fujitsu.com
On Mon, Nov 14, 2022 8:08 PM Melih Mutlu wrote: > > Attached patch with updated version of this patch. Thanks for your patch. I tried to do a performance test for this patch, the result looks good to me. (The steps are similar to what Melih shared [1].) The time to synchronize about 1GB data

Re: Flush SLRU counters in checkpointer process

2023-01-11 Thread Aleksander Alekseev
Hi Anthonin, > This patch adds a flush of SLRU stats to the end of checkpoints. The patch looks good to me and passes the tests but let's see if anyone else has any feedback. Also I added a CF entry: https://commitfest.postgresql.org/42/4120/ -- Best regards, Aleksander Alekseev

Re: Add a new pg_walinspect function to extract FPIs from WAL records

2023-01-11 Thread Bharath Rupireddy
On Wed, Jan 11, 2023 at 10:07 AM Michael Paquier wrote: > > +postgres=# SELECT lsn, tablespace_oid, database_oid, relfile_number, > block_number, fork_name, length(fpi) > 0 as fpi_ok FROM > pg_get_wal_fpi_info('0/7418E60', '0/7518218'); > > This query in the docs is too long IMO. Could you split

Re: [PATCH] Improve ability to display optimizer analysis using OPTIMIZER_DEBUG

2023-01-11 Thread John Naylor
On Wed, Jan 11, 2023 at 1:38 PM Ankit Kumar Pandey wrote: > > > > On 11/01/23 09:57, Tom Lane wrote: > > IME it's typically a lot more productive to approach things via > > "scratch your own itch". If a problem is biting you directly, then > > at least you have some clear idea of what it is that

Re: Schema variables - new implementation for Postgres 15 (typo)

2023-01-11 Thread Julien Rouhaud
On Tue, Jan 10, 2023 at 08:35:16PM +0100, Pavel Stehule wrote: > út 10. 1. 2023 v 3:20 odesílatel Julien Rouhaud napsal: > > > > Another new behavior I see is the new rowtype_only parameter for > > LookupVariable. Has this been discussed? > > > > I think so it was discussed about table shadowing

Re: Strengthen pg_waldump's --save-fullpage tests

2023-01-11 Thread Drouvot, Bertrand
Hi, On 1/11/23 5:17 AM, Bharath Rupireddy wrote: On Wed, Jan 11, 2023 at 6:32 AM Michael Paquier wrote: On Tue, Jan 10, 2023 at 05:25:44PM +0100, Drouvot, Bertrand wrote: I like the idea of comparing the full page (and not just the LSN) but I'm not sure that adding the pageinspect

RE: [Proposal] Add foreign-server health checks infrastructure

2023-01-11 Thread Hayato Kuroda (Fujitsu)
Dear hackers, I was not sure, but the cfbot could not be accepted the previous version. I made the patch again from HEAD(5f6401) without any changes, so I did not count up the version number. Best Regards, Hayato Kuroda FUJITSU LIMITED

Re: meson oddities

2023-01-11 Thread Peter Eisentraut
On 04.01.23 23:53, Andres Freund wrote: dir_data = get_option('datadir') -if not (dir_data.contains('pgsql') or dir_data.contains('postgres')) +if not ((dir_prefix/dir_data).contains('pgsql') or (dir_prefix/dir_data).contains('postgres')) dir_data = dir_data / pkg endif Hm. Perhaps we

Re: [PATCH] support tab-completion for single quote input with equal sign

2023-01-11 Thread torikoshia
On 2023-01-11 11:28, Tom Lane wrote: I wrote: I've spent some effort previously on getting tab-completion to deal sanely with single-quoted strings, but everything I've tried has crashed and burned :-(, mainly because it's not clear when to take the whole literal as one "word" and when not.

Re: Allow logical replication to copy tables in binary format

2023-01-11 Thread Melih Mutlu
Hi, Thanks for your review. shiy.f...@fujitsu.com , 11 Oca 2023 Çar, 11:56 tarihinde şunu yazdı: > On Mon, Nov 14, 2022 8:08 PM Melih Mutlu wrote: > 1. > +# Binary enabled subscription should fail > +$node_subscriber_binary->wait_for_log("ERROR: insufficient data left in > message"); > >

RE: Time delayed LR (WAS Re: logical replication restrictions)

2023-01-11 Thread Hayato Kuroda (Fujitsu)
Dear Shveta, Thanks for reviewing! PSA new version. > 1. > + errmsg("min_apply_delay must not be set when streaming = parallel"))); > we give the same error msg for both the cases: > a. when subscription is created with streaming=parallel but we are > trying to alter subscription to set

RE: Time delayed LR (WAS Re: logical replication restrictions)

2023-01-11 Thread Hayato Kuroda (Fujitsu)
> 2. > I think users can set ' wal_receiver_status_interval ' to 0 or more > than 'wal_sender_timeout'. But is this a frequent use-case scenario or > do we see DBAs setting these in such a way by mistake? If so, then I > think, it is better to give Warning message in such a case when a user >

Re: fix and document CLUSTER privileges

2023-01-11 Thread Gilles Darold
Le 06/01/2023 à 01:26, Nathan Bossart a écrit : Apparently I forgot to run all the tests before posting v4. Here is a new version of the patch that should pass all tests. Review status: The patch applies and compiles without issues, make check and checkinstall tests are running without

Re: [PATCH] Reuse Workers and Replication Slots during Logical Replication

2023-01-11 Thread Melih Mutlu
Hi hackers, Rebased the patch to resolve conflicts. Best, -- Melih Mutlu Microsoft v3-0001-Add-replication-protocol-cmd-to-create-a-snapshot.patch Description: Binary data v7-0002-Reuse-Logical-Replication-Background-worker.patch Description: Binary data

Adjust the description of OutputPluginCallbacks in pg-doc

2023-01-11 Thread wangw.f...@fujitsu.com
Hi, When I was reading the "Logical Decoding Output Plugins" chapter in pg-doc [1], I think in the summary section, only the callback message_cb is not described whether it is required or optional, and the description of callback stream_prepare_cb seems inaccurate. And after the summary section,

Re: XID formatting and SLRU refactorings (was: Add 64-bit XIDs into PostgreSQL 15)

2023-01-11 Thread Aleksander Alekseev
Hi Maxim, > Here is a new patch set. > I've added comments and make use GetClogDirName call in copy_subdir_files. Jacob Champion pointed out (offlist, cc:'ed) that we may be wrong on this one: > 0001 patch changes the SLRU internals without affecting the callers. > 0001 - should make SLRU

Re: Time delayed LR (WAS Re: logical replication restrictions)

2023-01-11 Thread shveta malik
On Wed, Jan 11, 2023 at 3:27 PM shveta malik wrote: > > On Tue, Jan 10, 2023 at 7:42 PM Takamichi Osumi (Fujitsu) > wrote: > > > > On Tuesday, January 3, 2023 4:01 PM vignesh C wrote: > > Hi, thanks for your review ! > > > > > > > 1) This global variable can be removed as it is used only in >

RE: Logical replication timeout problem

2023-01-11 Thread wangw.f...@fujitsu.com
On Mon, Jan 9, 2023 at 13:04 PM Amit Kapila wrote: > Thanks for your comments. > One more thing, I think it would be better to expose a new callback > API via reorder buffer as suggested previously [2] similar to other > reorder buffer APIs instead of directly using reorderbuffer API to >

Re: [EXTERNAL] Re: [PATCH] Support using "all" for the db user in pg_ident.conf

2023-01-11 Thread Michael Paquier
On Wed, Jan 11, 2023 at 09:04:56AM +, Jelte Fennema wrote: > It's very different. I think easiest is to explain by example: > > If there exist three users on the postgres server: admin, jelte and michael > > Then this rule (your suggested rule): > mapname /^(.*)$ \1 > > Is equivalent to: >

Re: daitch_mokotoff module

2023-01-11 Thread Paul Ramsey
On Mon, Jan 2, 2023 at 2:03 PM Dag Lem wrote: > I also improved on the documentation example (using Full Text Search). > AFAIK you can't make general queries like that using arrays, however in > any case I must admit that text arrays seem like more natural building > blocks than space delimited

Re: low wal_retrieve_retry_interval causes missed signals on Windows

2023-01-11 Thread Andres Freund
Hi, On 2023-01-10 22:11:16 -0800, Nathan Bossart wrote: > The attached patch fixes this by always calling WaitLatch(), even if > wal_retrieve_retry_interval milliseconds have already elapsed and the > timeout is 0. It doesn't seem right to call WaitLatch() just for that purpose - nor necessarily

Re: pgsql: Add new GUC createrole_self_grant.

2023-01-11 Thread Tom Lane
Robert Haas writes: > If you want to make safe a SECURITY DEFINER function written using sql > or plpgsql, you either have to schema-qualify every single reference > or, more realistically, attach a SET clause to the function to set the > search_path to a sane value during the time that the

Re: ATTACH PARTITION seems to ignore column generation status

2023-01-11 Thread Tom Lane
Amit Langote writes: > I've updated your disallow-generated-child-columns-2.patch to do this, > and have also merged the delta post that I had attached with my last > email, whose contents you sound to agree with. Pushed with some further work to improve the handling of multiple- inheritance

Re: doc: mentioned CREATE+ATTACH PARTITION as an alternative to CREATE TABLE..PARTITION OF

2023-01-11 Thread Robert Haas
On Wed, Jan 11, 2023 at 10:48 AM Robert Treat wrote: > > @Robert: I wonder why shouldn't CREATE..PARTITION OF *also* be patched > > to first create a table, and then attach the partition, transparently > > doing what everyone would want, without having to re-read the updated > > docs or know to

Re: Option to not use ringbuffer in VACUUM, using it in failsafe mode

2023-01-11 Thread Tom Lane
Peter Geoghegan writes: > On Wed, Jan 11, 2023 at 11:18 AM Andres Freund wrote: >> I don't like that - it's also quite useful to disable use of ringbuffers when >> you actually need to clean up indexes. Especially when we have a lot of dead >> tuples we'll rescan indexes over and over... >

Re: logical decoding and replication of sequences, take 2

2023-01-11 Thread Tomas Vondra
On 1/11/23 21:58, Andres Freund wrote: > Hi, > > On 2023-01-11 15:41:45 -0500, Robert Haas wrote: >> I wonder, then, what happens if somebody wants to do parallel apply. That >> would seem to require some relaxation of this rule, but then doesn't that >> break what this patch wants to do? >

Re: pg_stat_bgwriter.buffers_backend is pretty meaningless (and more?)

2023-01-11 Thread Justin Pryzby
> Subject: [PATCH v45 4/5] Add system view tracking IO ops per backend type The patch can/will fail with: CREATE TABLESPACE test_io_shared_stats_tblspc LOCATION ''; +WARNING: tablespaces created by regression test cases should have names starting with "regress_" CREATE TABLESPACE test_stats

Re: [PATCH] Add `verify-system` sslmode to use system CA pool for server cert

2023-01-11 Thread Magnus Hagander
On Wed, Jan 11, 2023 at 8:06 PM Jacob Champion wrote: > On Wed, Jan 11, 2023 at 10:23 AM Magnus Hagander > wrote: > > Sorry to jump in (very) late in this game. So first, I like this general > approach :) > > Thanks! > > > It feels icky to have to add configure tests just to make a test work. >

Re: allowing for control over SET ROLE

2023-01-11 Thread Robert Haas
On Wed, Jan 11, 2023 at 10:16 AM Noah Misch wrote: > A "git grep 'direct or indirect mem'" found a few more: > > doc/src/sgml/ref/alter_collation.sgml:42: To alter the owner, you must also > be a direct or indirect member of the new > doc/src/sgml/ref/create_database.sgml:92:role, you

Re: Minimal logical decoding on standbys

2023-01-11 Thread Andres Freund
Hi, On 2023-01-06 10:52:06 +0100, Drouvot, Bertrand wrote: > On 1/6/23 4:40 AM, Andres Freund wrote: > > ISTM that the ordering of patches isn't quite right later on. ISTM that it > > doesn't make sense to introduce working logic decoding without first fixing > > WalSndWaitForWal() (i.e. patch

Re: logical decoding and replication of sequences, take 2

2023-01-11 Thread Andres Freund
Hi, On 2023-01-11 15:41:45 -0500, Robert Haas wrote: > I wonder, then, what happens if somebody wants to do parallel apply. That > would seem to require some relaxation of this rule, but then doesn't that > break what this patch wants to do? I don't think it'd pose a direct problem - presumably

Re: Can we let extensions change their dumped catalog schemas?

2023-01-11 Thread Tom Lane
Jacob Champion writes: > On Tue, Jan 10, 2023 at 7:53 PM Tom Lane wrote: >> I also fear that it will break >> a bunch of use-cases that work fine today, which are exactly the >> ones for which we originally defined pg_dump as *not* committing >> to a particular extension version. > Right, I

Re: Remove source code display from \df+?

2023-01-11 Thread Tom Lane
Pavel Stehule writes: > st 11. 1. 2023 v 19:31 odesílatel Magnus Hagander > napsal: >> This is only about Internal and C, isn't it? Isn't the oid of these >> static, and identified by INTERNALlanguageId and ClanguageId respectively? >> So we could just have the query show the prosrc column if

Re: Option to not use ringbuffer in VACUUM, using it in failsafe mode

2023-01-11 Thread Andres Freund
Hi, On 2023-01-11 16:18:34 -0500, Tom Lane wrote: > Peter Geoghegan writes: > > On Wed, Jan 11, 2023 at 11:18 AM Andres Freund wrote: > >> I don't like that - it's also quite useful to disable use of ringbuffers > >> when > >> you actually need to clean up indexes. Especially when we have a

Re: pgsql: Delay commit status checks until freezing executes.

2023-01-11 Thread Peter Geoghegan
On Sat, Jan 7, 2023 at 7:25 PM Andres Freund wrote: > Probably a good idea, although it doesn't neatly fit right now. I'll leave it for now. Attached is v2, which changes things based on your feedback. Would like to get this out of the way soon. > > Also, does amcheck's get_xid_status() need a

Re: [PATCH] psql: Add tab-complete for optional view parameters

2023-01-11 Thread Thomas Munro
On Thu, Jan 12, 2023 at 5:50 AM vignesh C wrote: > For some reason CFBot is not able to apply the patch, please have a > look and post an updated version if required, also check and handle > Dean Rasheed and Jim Jones comment if applicable: > === Applying patches on top of PostgreSQL commit ID >

Re: logical decoding and replication of sequences, take 2

2023-01-11 Thread Robert Haas
On Wed, Jan 11, 2023 at 1:29 PM Tomas Vondra wrote: > > I agree that it's fine for the sequence to be slightly ahead, but I > > think that it can't be too far ahead without causing problems. Suppose > > for example that transaction #1 creates a sequence. Transaction #2 > > does nextval on the

Re: logical decoding and replication of sequences, take 2

2023-01-11 Thread Andres Freund
Hi, On 2023-01-11 15:23:18 -0500, Robert Haas wrote: > Yeah, I meant if #1 had committed and then #2 started to do its thing. > I was worried that decoding might reach the nextval operations in > transaction #2 before it replayed #1. > > This worry may be entirely based on me not understanding

Re: Option to not use ringbuffer in VACUUM, using it in failsafe mode

2023-01-11 Thread Andres Freund
Hi, On 2023-01-11 14:38:34 -0600, Justin Pryzby wrote: > On Wed, Jan 11, 2023 at 10:58:54AM -0800, Andres Freund wrote: > > Some ideas: > > > > USE_RING_BUFFERS on|off > > REUSE_BUFFERS on|off > > +1 for either of these. Then I'd go for REUSE_BUFFERS. What made you prefer it over

No Callbacks on FATAL

2023-01-11 Thread Ed Behn
I'm developing a module that implements Haskell as a procedural language ( https://www.postgresql.org/about/news/plhaskell-v10-released-2519/) I'm using a callback function that is called when a memory context is deleted to remove a temporary file. This works fine when the transaction ends

Re: logical decoding and replication of sequences, take 2

2023-01-11 Thread Andres Freund
Hi, Heikki, CCed you due to the point about 2c03216d8311 below. On 2023-01-10 19:32:12 +0100, Tomas Vondra wrote: > 0001 is a fix for the pre-existing issue in logicalmsg_decode, > attempting to build a snapshot before getting into a consistent state. > AFAICS this only affects assert-enabled

Re: Exposing the lock manager's WaitForLockers() to SQL

2023-01-11 Thread Andres Freund
Hi, On 2023-01-10 10:01:25 +0100, Marco Slot wrote: > On Fri, Dec 23, 2022 at 11:43 AM Will Mortensen wrote: > > We'd like to be able to call the lock manager's WaitForLockers() and > > WaitForLockersMultiple() from SQL. Below I describe our use case, but > > basically I'm wondering if this: > >

Re: Option to not use ringbuffer in VACUUM, using it in failsafe mode

2023-01-11 Thread Justin Pryzby
On Wed, Jan 11, 2023 at 10:58:54AM -0800, Andres Freund wrote: > Hi, > > On 2023-01-11 10:35:19 -0800, Peter Geoghegan wrote: > > On Wed, Jan 11, 2023 at 10:27 AM Andres Freund wrote: > > > Therefore I'd like to add an option to the VACUUM command to use to > > > disable > > > the use of the

Re: logical decoding and replication of sequences, take 2

2023-01-11 Thread Robert Haas
On Wed, Jan 11, 2023 at 3:28 PM Andres Freund wrote: > On 2023-01-11 15:23:18 -0500, Robert Haas wrote: > > Yeah, I meant if #1 had committed and then #2 started to do its thing. > > I was worried that decoding might reach the nextval operations in > > transaction #2 before it replayed #1. > > >

Re: pgsql: Add new GUC createrole_self_grant.

2023-01-11 Thread Robert Haas
On Wed, Jan 11, 2023 at 4:00 PM Tom Lane wrote: > Robert Haas writes: > > If you want to make safe a SECURITY DEFINER function written using sql > > or plpgsql, you either have to schema-qualify every single reference > > or, more realistically, attach a SET clause to the function to set the > >

Re: logical decoding and replication of sequences, take 2

2023-01-11 Thread Tomas Vondra
On 1/11/23 21:12, Andres Freund wrote: > Hi, > > > Heikki, CCed you due to the point about 2c03216d8311 below. > > > On 2023-01-10 19:32:12 +0100, Tomas Vondra wrote: >> 0001 is a fix for the pre-existing issue in logicalmsg_decode, >> attempting to build a snapshot before getting into a

Re: logical decoding and replication of sequences, take 2

2023-01-11 Thread Andres Freund
Hi, On 2023-01-11 22:30:42 +0100, Tomas Vondra wrote: > On 1/11/23 21:58, Andres Freund wrote: > > If you're thinking of decoding changes in parallel (rather than streaming > > out > > large changes before commit when possible), you'd only be able to do that in > > cases when transaction haven't

Re: Add SHELL_EXIT_CODE to psql

2023-01-11 Thread Corey Huinker
> > > > The patch does not apply on top of HEAD as in [1], please post a rebased > patch: > > Conflict was due to the doc patch applying id tags to psql variable names. I've rebased and added my own id tags to the two new variables. From 9e2827a6f955e7cebf87ca538fab113a359951b4 Mon Sep 17 00:00:00

Re: [PATCH] pgbench: add multiconnect option

2023-01-11 Thread vignesh C
On Tue, 8 Nov 2022 at 02:16, Fabien COELHO wrote: > > > Hello Ian, > > > cfbot reports the patch no longer applies. As CommitFest 2022-11 is > > currently underway, this would be an excellent time to update the patch. > > Attached a v5 which is just a rebase. The patch does not apply on top of

[Proposal] Allow pg_dump to include all child tables with the root table

2023-01-11 Thread Gilles Darold
Hi all, I would like to propose a new pg_dump option called --with-child to include or exclude from a dump all child and partition tables when a parent table is specified using option -t/--table or -T/--exclude-table. The whole tree is dumped with the root table. To include all partitions

Re: mprove tab completion for ALTER EXTENSION ADD/DROP

2023-01-11 Thread vignesh C
On Wed, 11 Jan 2023 at 12:19, Michael Paquier wrote: > > On Wed, Jan 11, 2023 at 12:10:33PM +0900, Kyotaro Horiguchi wrote: > > It suggests the *kinds* of objects that are part of the extension, but > > lists the objects of that kind regardless of dependency. I read > > Michael suggested (and I

Re: Remove source code display from \df+?

2023-01-11 Thread Pavel Stehule
st 11. 1. 2023 v 17:50 odesílatel Isaac Morland napsal: > I find \df+ much less useful than it should be because it tends to be > cluttered up with source code. Now that we have \sf, would it be reasonable > to remove the source code from the \df+ display? This would make it easier > to see

Re: Remove source code display from \df+?

2023-01-11 Thread Magnus Hagander
On Wed, Jan 11, 2023 at 6:19 PM Pavel Stehule wrote: > > > st 11. 1. 2023 v 17:50 odesílatel Isaac Morland > napsal: > >> I find \df+ much less useful than it should be because it tends to be >> cluttered up with source code. Now that we have \sf, would it be reasonable >> to remove the source

Re: fix and document CLUSTER privileges

2023-01-11 Thread Nathan Bossart
On Wed, Jan 11, 2023 at 02:22:26PM +0100, Gilles Darold wrote: > I'm moving this commitfest entry to Ready for Committers. Thank you for reviewing. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: Remove source code display from \df+?

2023-01-11 Thread Isaac Morland
Right, for internal or C functions it just gives a symbol name or something similar. I've never been annoyed seeing that, just having pages of PL/PGSQL (I use a lot of that, possibly biased towards the “too much” direction) take up all the space. A bit hacky, but what about only showing the first

Re: Option to not use ringbuffer in VACUUM, using it in failsafe mode

2023-01-11 Thread Peter Geoghegan
On Wed, Jan 11, 2023 at 10:58 AM Andres Freund wrote: > Any idea about the name? The obvious thing is to reference ring buffers in the > option name, but that's more of an implementation detail... What are the chances that anybody using this feature via a manual VACUUM command will also use

Re: [PATCH] Add `verify-system` sslmode to use system CA pool for server cert

2023-01-11 Thread Jacob Champion
On Wed, Jan 11, 2023 at 10:23 AM Magnus Hagander wrote: > Sorry to jump in (very) late in this game. So first, I like this general > approach :) Thanks! > It feels icky to have to add configure tests just to make a test work. But I > guess there isn't really a way around that if we want to

Re: Option to not use ringbuffer in VACUUM, using it in failsafe mode

2023-01-11 Thread Peter Geoghegan
On Wed, Jan 11, 2023 at 11:18 AM Andres Freund wrote: > I don't like that - it's also quite useful to disable use of ringbuffers when > you actually need to clean up indexes. Especially when we have a lot of dead > tuples we'll rescan indexes over and over... That's a fair point. My vote goes

Re: recovery modules

2023-01-11 Thread Nathan Bossart
Thanks for taking a look. On Wed, Jan 11, 2023 at 04:53:39PM +0900, Michael Paquier wrote: > Hmm. Is passing down the file name used as a cutoff point the best > interface for the modules? Perhaps passing down the redo LSN and its > TLI would be a cleaner approach in terms of flexibility? I

Re: Transparent column encryption

2023-01-11 Thread vignesh C
On Sat, 31 Dec 2022 at 19:47, Peter Eisentraut wrote: > > On 21.12.22 06:46, Peter Eisentraut wrote: > > And another update. The main changes are that I added an 'unspecified' > > CMK algorithm, which indicates that the external KMS knows what it is > > but the database system doesn't. This was

Re: MultiXact\SLRU buffers configuration

2023-01-11 Thread Andrey Borodin
Hi Dilip! Thank you for the review! On Tue, Jan 10, 2023 at 9:58 PM Dilip Kumar wrote: > > On Mon, Jan 9, 2023 at 9:49 AM Andrey Borodin wrote: > > > > On Tue, Jan 3, 2023 at 5:02 AM vignesh C wrote: > > > does not apply on top of HEAD as in [1], please post a rebased patch: > > > > > Thanks!

Re: Remove source code display from \df+?

2023-01-11 Thread Justin Pryzby
Or, only show the source in \df++. But it'd be a bit unfortunate if the C language function wasn't shown in \df+

Re: pgsql: Add new GUC createrole_self_grant.

2023-01-11 Thread Robert Haas
On Tue, Jan 10, 2023 at 10:25 PM Tom Lane wrote: > > Of course, if it's possible for a non-CREATEROLE user to set the value > > that a CREATEROLE user experiences, that'd be more of a problem -- > > That's exactly the case I'm worried about, and it's completely reachable > if a CREATEROLE user

Re: Remove source code display from \df+?

2023-01-11 Thread Isaac Morland
On Wed, 11 Jan 2023 at 13:11, Pavel Stehule wrote: please, don't send top post replies - > https://en.wikipedia.org/wiki/Posting_style > Sorry about that; I do know to do it properly and usually get it right. GMail doesn’t seem to have an option (that I can find) to leave no space at the top

Option to not use ringbuffer in VACUUM, using it in failsafe mode

2023-01-11 Thread Andres Freund
Hi, The use of the ringbuffer in VACUUM often causes very substantial slowdowns. The primary reason for that is that often most/all the buffers in the ringbuffer have been dirtied when they were processed, with an associated WAL record. When we then reuse the buffer via the (quite small)

Re: Can we let extensions change their dumped catalog schemas?

2023-01-11 Thread Jacob Champion
On Tue, Jan 10, 2023 at 7:53 PM Tom Lane wrote: > Jacob Champion writes: > > Unless I'm missing something obvious (please, let it be that) there's no > > way to do this safely. Once you've marked an internal table as dumpable, > > its schema is effectively frozen if you want your dumps to work

Re: Option to not use ringbuffer in VACUUM, using it in failsafe mode

2023-01-11 Thread Peter Geoghegan
On Wed, Jan 11, 2023 at 10:27 AM Andres Freund wrote: > Therefore I'd like to add an option to the VACUUM command to use to disable > the use of the ringbuffer. Not sure about the name yet. Sounds like a good idea. > I think we should auto-enable that mode once we're using the failsafe mode, >

Re: [PATCH] psql: Add tab-complete for optional view parameters

2023-01-11 Thread vignesh C
On Fri, 9 Dec 2022 at 16:01, Christoph Heiss wrote: > > Thanks for the review! > > On 12/8/22 12:19, Melih Mutlu wrote: > > Hi Christoph, > > > > I just took a quick look at your patch. > > Some suggestions: > > > > + else if (Matches("ALTER", "VIEW", MatchAny, "SET", "(")) > > +

Remove source code display from \df+?

2023-01-11 Thread Isaac Morland
I find \df+ much less useful than it should be because it tends to be cluttered up with source code. Now that we have \sf, would it be reasonable to remove the source code from the \df+ display? This would make it easier to see function permissions and comments. If somebody wants to see the full

Re: XID formatting and SLRU refactorings (was: Add 64-bit XIDs into PostgreSQL 15)

2023-01-11 Thread Jacob Champion
On Wed, Jan 11, 2023 at 1:48 AM Aleksander Alekseev wrote: > After reading [1] carefully it looks like we shouldn't worry about > this. The upgrade procedure explicitly requires to run `pg_ctl stop` > during step 8 of the upgrade procedure, i.e. not in the immediate mode > [2]. Yeah, pg_upgrade

Re: [PATCH] Add `verify-system` sslmode to use system CA pool for server cert

2023-01-11 Thread Jacob Champion
On Wed, Jan 11, 2023 at 6:37 AM Jelte Fennema wrote: > > LGTM. As far as I can tell this is ready for a committer. Thanks for the reviews! --Jacob

Re: Remove source code display from \df+?

2023-01-11 Thread Pavel Stehule
st 11. 1. 2023 v 18:25 odesílatel Magnus Hagander napsal: > > > On Wed, Jan 11, 2023 at 6:19 PM Pavel Stehule > wrote: > >> >> >> st 11. 1. 2023 v 17:50 odesílatel Isaac Morland >> napsal: >> >>> I find \df+ much less useful than it should be because it tends to be >>> cluttered up with source

Re: Remove source code display from \df+?

2023-01-11 Thread Pavel Stehule
Hi st 11. 1. 2023 v 18:57 odesílatel Isaac Morland napsal: > Right, for internal or C functions it just gives a symbol name or > something similar. I've never been annoyed seeing that, just having pages > of PL/PGSQL (I use a lot of that, possibly biased towards the “too much” > direction) take

Re: [PATCH] Add `verify-system` sslmode to use system CA pool for server cert

2023-01-11 Thread Magnus Hagander
On Wed, Jan 11, 2023 at 6:27 PM Jacob Champion wrote: > On Wed, Jan 11, 2023 at 6:37 AM Jelte Fennema wrote: > > > > LGTM. As far as I can tell this is ready for a committer. > > Thanks for the reviews! > Sorry to jump in (very) late in this game. So first, I like this general approach :) It

Re: Remove source code display from \df+?

2023-01-11 Thread Magnus Hagander
On Wed, Jan 11, 2023 at 7:24 PM Isaac Morland wrote: > On Wed, 11 Jan 2023 at 13:11, Pavel Stehule > wrote: > > please, don't send top post replies - >> https://en.wikipedia.org/wiki/Posting_style >> > > Sorry about that; I do know to do it properly and usually get it right. > GMail doesn’t

Re: Option to not use ringbuffer in VACUUM, using it in failsafe mode

2023-01-11 Thread Andres Freund
Hi, On 2023-01-11 11:06:26 -0800, Peter Geoghegan wrote: > On Wed, Jan 11, 2023 at 10:58 AM Andres Freund wrote: > > Any idea about the name? The obvious thing is to reference ring buffers in > > the > > option name, but that's more of an implementation detail... > > What are the chances that

Re: Allow +group in pg_ident.conf

2023-01-11 Thread Jelte Fennema
I'm working on a new patchset for my commitfest entry. I'll make sure to include a third patch for the +group support, and I'll include you (Andrew) in the thread when I send it. On Wed, 11 Jan 2023 at 02:14, Michael Paquier wrote: > > On Tue, Jan 10, 2023 at 09:42:19AM -0500, Andrew Dunstan

Re: XID formatting and SLRU refactorings (was: Add 64-bit XIDs into PostgreSQL 15)

2023-01-11 Thread Aleksander Alekseev
Hi Maxim, > Secondly, shouldn't we introduce a new WAL record type in order to > make the code backward compatible with previous PG versions? I'm not > 100% sure how the upgrade procedure works in all the details. If it > requires the DBMS to be gracefully shut down before the upgrade then > we

Re: Common function for percent placeholder replacement

2023-01-11 Thread Peter Eisentraut
On 09.01.23 18:53, Nathan Bossart wrote: + nativePath = pstrdup(path); + make_native_path(nativePath); + nativePath = pstrdup(xlogpath); + make_native_path(nativePath); Should these be freed? committed with that fixed

Re: [PATCH] Support % wildcard in extension upgrade filenames

2023-01-11 Thread 'Sandro Santilli'
On Tue, Jan 10, 2023 at 11:09:23PM -0500, Regina Obe wrote: > The only way we can fix that in the current setup, is to move to a minor > version mode which means we can > never do micro updates. Or just not with standard PostgreSQL syntax, because we can of course do upgrades using the `SELECT

RE: Time delayed LR (WAS Re: logical replication restrictions)

2023-01-11 Thread Hayato Kuroda (Fujitsu)
Dear Vignesh, Thanks for reviewing! > 1) Comment inconsistency across create and alter subscription, better > to keep it same: A comment for CREATE SUBSCRIPTION became same as ALTER's one. > 2) ereport inconsistency, braces around errcode is present in few > places and not present in few

Re: Minimal logical decoding on standbys

2023-01-11 Thread Andres Freund
Hi, On 2023-01-11 13:02:13 +0530, Bharath Rupireddy wrote: > 3. Is this feature still a 'minimal logical decoding on standby'? > Firstly, why is it 'minimal'? It's minimal in comparison to other proposals at the time that did explicit / active coordination between primary and standby to allow

Re: logical decoding and replication of sequences, take 2

2023-01-11 Thread Tomas Vondra
On 1/10/23 20:52, Robert Haas wrote: > On Tue, Jan 10, 2023 at 1:32 PM Tomas Vondra > wrote: >> 0001 is a fix for the pre-existing issue in logicalmsg_decode, >> attempting to build a snapshot before getting into a consistent state. >> AFAICS this only affects assert-enabled builds and is

Re: Remove source code display from \df+?

2023-01-11 Thread Pavel Stehule
st 11. 1. 2023 v 19:31 odesílatel Magnus Hagander napsal: > On Wed, Jan 11, 2023 at 7:24 PM Isaac Morland > wrote: > >> On Wed, 11 Jan 2023 at 13:11, Pavel Stehule >> wrote: >> >> please, don't send top post replies - >>> https://en.wikipedia.org/wiki/Posting_style >>> >> >> Sorry about that;

Re: Common function for percent placeholder replacement

2023-01-11 Thread Nathan Bossart
On Wed, Jan 11, 2023 at 11:09:01AM +0100, Peter Eisentraut wrote: > committed with that fixed While rebasing my recovery modules patch set, I noticed a couple of small things that might be worth cleaning up. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com diff --git

Re: Option to not use ringbuffer in VACUUM, using it in failsafe mode

2023-01-11 Thread Andres Freund
Hi, On 2023-01-11 10:27:20 -0800, Andres Freund wrote: > On cloud hardware with higher fsync latency I've seen > 15x time differences > between using the ringbuffers and avoiding them by using pg_prewarm. A slightly edited version of what I've in the past to defeat the ringbuffers using

Re: Option to not use ringbuffer in VACUUM, using it in failsafe mode

2023-01-11 Thread Andres Freund
Hi, On 2023-01-11 10:35:19 -0800, Peter Geoghegan wrote: > On Wed, Jan 11, 2023 at 10:27 AM Andres Freund wrote: > > Therefore I'd like to add an option to the VACUUM command to use to disable > > the use of the ringbuffer. Not sure about the name yet. > > Sounds like a good idea. Any idea

Re: Use windows VMs instead of windows containers on the CI

2023-01-11 Thread Justin Pryzby
On Tue, Jan 10, 2023 at 03:20:18PM +0300, Nazir Bilal Yavuz wrote: > Hi, > > I propose using windows VMs instead of containers, the patch is attached. > Currently, windows containers are used on the CI, but these container images > are needs to get pulled on every CI run, also they are slow to

Re: pgsql: Delay commit status checks until freezing executes.

2023-01-11 Thread Peter Geoghegan
On Wed, Jan 11, 2023 at 3:06 PM Andres Freund wrote: > > + * We can't use TransactionIdDidAbort here because it won't treat > > transactions > > + * that were in progress during a crash as aborted by now. We determine > > that > > + * transactions aborted/crashed through process of elimination

Re: Option to not use ringbuffer in VACUUM, using it in failsafe mode

2023-01-11 Thread Andres Freund
Hi, On 2023-01-11 17:26:19 -0700, David G. Johnston wrote: > Should we just add "ring_buffers" to the existing "shared_buffers" and > "temp_buffers" settings? The different types of ring buffers have different sizes, for good reasons. So I don't see that working well. I also think it'd be more

Re: Reducing the WAL overhead of freezing in VACUUM by deduplicating per-tuple freeze plans

2023-01-11 Thread Andres Freund
Hi, On 2023-01-11 16:06:31 -0800, Peter Geoghegan wrote: > On Mon, Jan 9, 2023 at 2:18 PM Peter Geoghegan wrote: > > I'll try to get back to it this week. > > Attached patch fixes up these issues. It's almost totally mechanical. Looks better, thanks! > (Ended up using "git diff

Re: Switching XLog source from archive to streaming when primary available

2023-01-11 Thread Nathan Bossart
On Tue, Oct 18, 2022 at 07:31:33AM +0530, Bharath Rupireddy wrote: > In summary, the standby state machine in WaitForWALToBecomeAvailable() > exhausts all the WAL in pg_wal before switching to streaming after > failing to fetch from archive. The v8 patch proposed upthread deviates > from this

Re: No Callbacks on FATAL

2023-01-11 Thread Andres Freund
Hi, On 2023-01-11 18:10:33 -0500, Tom Lane wrote: > Ed Behn writes: > > I'm using a callback function that is called when a memory context is > > deleted to remove a temporary file. This works fine when the transaction > > ends normally or raises an ERROR. However, when a FATAL event happens,

  1   2   >