Re: Remove unnecessary 'always:' from CompilerWarnings task

2023-11-08 Thread Nazir Bilal Yavuz
Hi, Thanks for the review. On Wed, 8 Nov 2023 at 10:31, Peter Eisentraut wrote: > > On 05.09.23 12:25, Nazir Bilal Yavuz wrote: > > There are multiple 'always:' keywords under the CompilerWarnings task. > > Instead of that, we can use one 'always:' and move the instructions > > under this. So,

Re: A recent message added to pg_upgade

2023-11-08 Thread Michael Paquier
On Thu, Nov 09, 2023 at 01:12:54PM +0530, Amit Kapila wrote: > I think it is in an email[1]. Noted. > I can take care of this unless we see some opposition to this idea. Thanks! -- Michael signature.asc Description: PGP signature

Re: A recent message added to pg_upgade

2023-11-08 Thread Amit Kapila
On Thu, Nov 9, 2023 at 12:38 PM Michael Paquier wrote: > > On Thu, Nov 09, 2023 at 05:04:28PM +1100, Peter Smith wrote: > > Having a GUC hook for the "max_slot_wal_keep_size" seemed OK to me. If > > the user overrides a GUC value (admittedly, maybe there is no reason > > why they would want to)

Re: Fix output of zero privileges in psql

2023-11-08 Thread Laurenz Albe
On Thu, 2023-11-09 at 03:40 +0100, Erik Wienhold wrote: > On 2023-11-08 13:23 +0100, Laurenz Albe wrote: > > I wonder how to proceed with this patch. The main disagreement is > > whether default privileges should be displayed as NULL (less invasive, > > but more confusing for beginners) or

Re: make pg_ctl more friendly

2023-11-08 Thread Crisp Lee
Hi, I thought the PITR shutdown was DB_SHUTDOWN. I made a mistake. The v2 attach looks good. On Thu, Nov 9, 2023 at 3:19 PM Junwang Zhao wrote: > On Thu, Nov 9, 2023 at 3:08 PM Junwang Zhao wrote: > > > > On Thu, Nov 9, 2023 at 9:57 AM Crisp Lee > wrote: > > > > > > Hi, > > > > > > I know

Re: Add new option 'all' to pg_stat_reset_shared()

2023-11-08 Thread Michael Paquier
On Thu, Nov 09, 2023 at 01:50:34PM +0900, torikoshia wrote: > PGSTAT_KIND_SLRU cannot be reset by pg_stat_reset_shared(), so I feel > uncomfortable to delete it all together. > It might be better after pg_stat_reset_shared() has been modified to take > 'slru' as an argument, though. Not sure how

Re: make pg_ctl more friendly

2023-11-08 Thread Junwang Zhao
On Thu, Nov 9, 2023 at 3:08 PM Junwang Zhao wrote: > > On Thu, Nov 9, 2023 at 9:57 AM Crisp Lee wrote: > > > > Hi, > > > > I know it. But my question is not that. I did a PITR operation with > > recovery_target_name and recovery_target_action('shutdown'). The PITR > > process was very short

Re: RFC: Logging plan of the running query

2023-11-08 Thread Ashutosh Bapat
On Thu, Nov 9, 2023 at 12:03 PM torikoshia wrote: > >> > >> 1. When a backend is running nested queries, we will see the plan of > >> the innermost query. That query may not be the actual culprit if the > >> user query is running slowly. E.g a query being run as part of inner > >> side nested

Re: A recent message added to pg_upgade

2023-11-08 Thread Michael Paquier
On Thu, Nov 09, 2023 at 05:04:28PM +1100, Peter Smith wrote: > Having a GUC hook for the "max_slot_wal_keep_size" seemed OK to me. If > the user overrides a GUC value (admittedly, maybe there is no reason > why they would want to) then at least the hook will give an error, > rather than us

Re: make pg_ctl more friendly

2023-11-08 Thread Junwang Zhao
On Thu, Nov 9, 2023 at 9:57 AM Crisp Lee wrote: > > Hi, > > I know it. But my question is not that. I did a PITR operation with > recovery_target_name and recovery_target_action('shutdown'). The PITR process > was very short and the PITR was done before pg_ctl check. The postmaster > shutdown

Re: pg_upgrade and logical replication

2023-11-08 Thread Michael Paquier
On Thu, Nov 09, 2023 at 01:14:05PM +1100, Peter Smith wrote: > Looks like v12 accidentally forgot to update this to the modified > function name 'binary_upgrade_add_sub_rel_state' This v12 is overall cleaner than its predecessors. Nice to see. +my $result = $publisher->safe_psql('postgres',

Re: A recent message added to pg_upgade

2023-11-08 Thread Kyotaro Horiguchi
At Thu, 9 Nov 2023 12:00:59 +0530, Amit Kapila wrote in > I have also proposed that as one of the alternatives but didn't get > many votes. And, I think if the user is passing a special value of > max_slot_wal_keep_size during the upgrade, it has to be a special > case, and rejecting it upfront

Re: RFC: Logging plan of the running query

2023-11-08 Thread torikoshia
On 2023-11-06 15:32, Ashutosh Bapat wrote: Thanks for the suggestion and example. On Fri, Nov 3, 2023 at 7:31 PM Ashutosh Bapat wrote: I have following questions related to the functionality. (Please point me to the relevant discussion if this has been already discussed.) 1. When a backend

Re: A recent message added to pg_upgade

2023-11-08 Thread Amit Kapila
On Thu, Nov 9, 2023 at 11:40 AM Kyotaro Horiguchi wrote: > > At Thu, 9 Nov 2023 09:53:07 +0530, Amit Kapila > wrote in > > Michael, Horiguchi-San, and others, do you have any thoughts on what > > is the best way to proceed? > > As I previously mentioned, I believe that if rejection is to be the

Re: pg_rewind WAL segments deletion pitfall

2023-11-08 Thread torikoshia
On 2023-11-06 23:58, Alexander Kukushkin wrote: Hi Torikoshia, On Thu, 2 Nov 2023 at 04:24, torikoshia wrote: +extern void preserve_file(char *filepath); Is this necessary? This function was defined in older version patch, but no longer seems to exist. +# We use "perl -e 'exit(1)'" as a

Re: pg_upgrade and logical replication

2023-11-08 Thread Amit Kapila
On Wed, Nov 8, 2023 at 10:52 PM vignesh C wrote: > > Upgrading 2 node circular logical replication cluster: > 1) Let's say we have a circular logical replication setup Node1->Node2 > & Node2->Node1. Here Node2 is subscribing to Node1 and Node1 is > subscribing to Node2. > 2) Stop the server in

Re: A recent message added to pg_upgade

2023-11-08 Thread Kyotaro Horiguchi
At Thu, 9 Nov 2023 09:53:07 +0530, Amit Kapila wrote in > Michael, Horiguchi-San, and others, do you have any thoughts on what > is the best way to proceed? As I previously mentioned, I believe that if rejection is to be the course of action, it would be best to proceed with it sooner rather

Re: Relids instead of Bitmapset * in plannode.h

2023-11-08 Thread Tom Lane
Ashutosh Bapat writes: > On Tue, Nov 7, 2023 at 8:54 PM Tom Lane wrote: >> Yes, I'm pretty sure that's exactly the reason, and I'm strongly >> against the initially-proposed patch. The include footprint of >> pathnodes.h would be greatly expanded, for no real benefit. > As I mentioned in [1]

Re: A recent message added to pg_upgade

2023-11-08 Thread Peter Smith
On Thu, Nov 9, 2023 at 3:55 PM Michael Paquier wrote: > > On Thu, Nov 09, 2023 at 09:53:07AM +0530, Amit Kapila wrote: > > On Tue, Nov 7, 2023 at 4:16 PM Amit Kapila wrote: > >> But then we don't need the hardcoded value of > >> max_logical_replication_workers as zero by pg_upgrade. I think

Re: Relids instead of Bitmapset * in plannode.h

2023-11-08 Thread Ashutosh Bapat
On Tue, Nov 7, 2023 at 8:54 PM Tom Lane wrote: > > Alvaro Herrera writes: > > On 2023-Oct-31, Ashutosh Bapat wrote: > >> For some reason plannode.h has declared variable to hold RTIs as > >> Bitmapset * instead of Relids like other places. Here's patch to fix > >> it. This is superficial change

Re: GUC names in messages

2023-11-08 Thread Kyotaro Horiguchi
At Thu, 9 Nov 2023 12:55:44 +1100, Peter Smith wrote in > The most common pattern there is "You might need to increase %s.". .. > Here is a patch to modify those other similar variations so they share > that common wording. > > PSA. I'm uncertain whether the phrases "Consider doing something"

Re: A recent message added to pg_upgade

2023-11-08 Thread Michael Paquier
On Thu, Nov 09, 2023 at 09:53:07AM +0530, Amit Kapila wrote: > On Tue, Nov 7, 2023 at 4:16 PM Amit Kapila wrote: >> But then we don't need the hardcoded value of >> max_logical_replication_workers as zero by pg_upgrade. I think doing >> IsBinaryUpgrade for slots won't be neat, so we anyway need

Re: Add new option 'all' to pg_stat_reset_shared()

2023-11-08 Thread torikoshia
On Thu, Nov 09, 2023 at 10:10:39AM +0900, torikoshia wrote: I'll attach the patch. Attached. On Mon, Nov 6, 2023 at 5:30 PM Bharath Rupireddy 3. I think the new reset all stats function must also consider resetting all SLRU stats, no? /* stats for fixed-numbered objects */

Re: POC PATCH: copy from ... exceptions to: (was Re: VLDB Features)

2023-11-08 Thread zhihuifan1213
Tom Lane writes: > Daniel Gustafsson writes: >>> On 8 Nov 2023, at 19:18, Tom Lane wrote: >>> I think an actually usable feature of this sort would involve >>> copying all the failed lines to some alternate output medium, >>> perhaps a second table with a TEXT column to receive the original

Re: A recent message added to pg_upgade

2023-11-08 Thread Amit Kapila
On Tue, Nov 7, 2023 at 4:16 PM Amit Kapila wrote: > > On Tue, Nov 7, 2023 at 8:12 AM Michael Paquier wrote: > > > > On Tue, Nov 07, 2023 at 07:59:46AM +0530, Amit Kapila wrote: > > > Do you mean to say that if 'IsBinaryUpgrade' is true then let's not > > > allow to launch launcher or apply

Re: Parallel query behaving different with custom GUCs

2023-11-08 Thread Michael Paquier
On Mon, Oct 30, 2023 at 09:21:56AM -0400, Robert Haas wrote: > I'm also alert to my own possible bias. Perhaps since I designed this > mechanism, I'm more prone to viewing its deficiencies as minor than a > neutral observer would be. So if anyone is sitting there reading this > and thinking "wow,

Re: Doubled test for SET statements in pg_stat_statements tests

2023-11-08 Thread Michael Paquier
On Wed, Nov 08, 2023 at 10:33:23AM +0300, Sergei Kornilov wrote: > is checked twice in contrib/pg_stat_statements/sql/utility.sql on > lines 278-286 and 333-341. Is this on any purpose? I think the > second set of tests is not needed and can be removed, as in the > attached patch. Thanks,

Re: Synchronizing slots from primary to standby

2023-11-08 Thread Amit Kapila
On Thu, Nov 9, 2023 at 8:11 AM Amit Kapila wrote: > > On Wed, Nov 8, 2023 at 8:09 PM Drouvot, Bertrand > wrote: > > > > > Unrelated to above, if there is a user slot on standby with the same > > > name which the slot-sync worker is trying to create, then shall it > > > emit a warning and skip

Re: Requiring recovery.signal or standby.signal when recovering with a backup_label

2023-11-08 Thread Michael Paquier
On Thu, Nov 09, 2023 at 12:04:19PM +0900, Michael Paquier wrote: > Sure, sorry for the confusion. By "we'd do nothing", I mean precirely > "to take no specific action related to archive recovery and recovery > parameters at the end of recovery", meaning that a combination of > backup_label with

Re: Requiring recovery.signal or standby.signal when recovering with a backup_label

2023-11-08 Thread Michael Paquier
On Wed, Nov 08, 2023 at 01:16:58PM -0500, Robert Haas wrote: > On Tue, Oct 31, 2023 at 7:39 PM Michael Paquier wrote: As you're telling me, and I've considered that as an option as well, perhaps we should just consider the presence of a backup_label file with no .signal files as a

Re: [PATCHES] Post-special page storage TDE support

2023-11-08 Thread Peter Geoghegan
On Wed, Nov 8, 2023 at 6:04 AM Stephen Frost wrote: > In conversations with folks (my memory specifically is a discussion with > Peter G, added to CC, and my apologies to Peter if I'm misremembering) > there was a pretty strong push that a page should be able to 'stand > alone' and not depend on

Re: Synchronizing slots from primary to standby

2023-11-08 Thread Amit Kapila
On Wed, Nov 8, 2023 at 8:09 PM Drouvot, Bertrand wrote: > > > Unrelated to above, if there is a user slot on standby with the same > > name which the slot-sync worker is trying to create, then shall it > > emit a warning and skip the sync of that slot or shall it throw an > > error? > > > > I'd

Re: Fix output of zero privileges in psql

2023-11-08 Thread Erik Wienhold
On 2023-11-08 13:23 +0100, Laurenz Albe wrote: > I wonder how to proceed with this patch. The main disagreement is > whether default privileges should be displayed as NULL (less invasive, > but more confusing for beginners) or "(default)" (more invasive, > but nicer for beginners). Are there any

Re: Eager page freeze criteria clarification

2023-11-08 Thread Melanie Plageman
On Wed, Oct 11, 2023 at 8:43 PM Andres Freund wrote: > > A rough sketch of a freezing heuristic: > > - We concluded that to intelligently control opportunistic freezing we need > statistics about the number of freezes and unfreezes > > - We should track page freezes / unfreezes in shared

Re: pg_upgrade and logical replication

2023-11-08 Thread Peter Smith
Thanks for addressing my previous review comments. I re-checked the latest patch v12-0001 and found the following: == Commit message 1. The new SQL binary_upgrade_create_sub_rel_state function has the following syntax: SELECT binary_upgrade_create_sub_rel_state(subname text, relid oid,

Re: [PATCHES] Post-special page storage TDE support

2023-11-08 Thread Stephen Frost
Greetings, On Wed, Nov 8, 2023 at 20:55 David Christensen < david.christen...@crunchydata.com> wrote: > On Wed, Nov 8, 2023 at 8:04 AM Stephen Frost wrote: > >> * Andres Freund (and...@anarazel.de) wrote: >> > On 2023-05-09 17:08:26 -0500, David Christensen wrote: >> > > From

Re: make pg_ctl more friendly

2023-11-08 Thread Crisp Lee
Hi, I know it. But my question is not that. I did a PITR operation with recovery_target_name and recovery_target_action('shutdown'). The PITR process was very short and the PITR was done before pg_ctl check. The postmaster shutdown due to recovery_target_action, and there was no crash. But pg_ctl

Re: GUC names in messages

2023-11-08 Thread Peter Smith
On Thu, Nov 2, 2023 at 1:25 AM Tom Lane wrote: > ... > Our error message style guidelines say not to assemble messages out > of separate parts, because it makes translation difficult. Originally > we applied that rule to GUC names mentioned in messages as well. > Awhile ago the translation team

Re: [PATCHES] Post-special page storage TDE support

2023-11-08 Thread David Christensen
On Wed, Nov 8, 2023 at 8:04 AM Stephen Frost wrote: > Greetings, > > * Andres Freund (and...@anarazel.de) wrote: > > On 2023-05-09 17:08:26 -0500, David Christensen wrote: > > > From 965309ea3517fa734c4bc89c144e2031cdf6c0c3 Mon Sep 17 00:00:00 2001 > > > From: David Christensen > > > Date: Tue,

Re: make pg_ctl more friendly

2023-11-08 Thread Andres Freund
Hi, On 2023-11-09 09:29:32 +0800, Crisp Lee wrote: > How to judge from 'DB_SHUTDOWNED' that PITR ends normally? 'DB_SHUTDOWNED' > is just a state, it could not give more meaning, so I reuse the > recovery.done. DB_SHUTDOWNED cannot be encountered while recovery is ongoing. If there was a hard

Re: Add new option 'all' to pg_stat_reset_shared()

2023-11-08 Thread Andres Freund
Hi, On 2023-11-09 10:25:18 +0900, Michael Paquier wrote: > On Thu, Nov 09, 2023 at 10:10:39AM +0900, torikoshia wrote: > > I am a little concerned about that the reset time is not the same and that > > GetCurrentTimestamp() is called multiple times, but I think it would be > > acceptable because

Re: make pg_ctl more friendly

2023-11-08 Thread Crisp Lee
How to judge from 'DB_SHUTDOWNED' that PITR ends normally? 'DB_SHUTDOWNED' is just a state, it could not give more meaning, so I reuse the recovery.done. On Sat, Nov 4, 2023 at 9:56 AM Andres Freund wrote: > Hi, > > On 2023-11-02 14:50:14 +0800, Crisp Lee wrote: > > I got a basebackup using

Re: Add new option 'all' to pg_stat_reset_shared()

2023-11-08 Thread Michael Paquier
On Thu, Nov 09, 2023 at 10:10:39AM +0900, torikoshia wrote: > I am a little concerned about that the reset time is not the same and that > GetCurrentTimestamp() is called multiple times, but I think it would be > acceptable because the function is probably not used that often and the > reset time

Re: ensure, not insure

2023-11-08 Thread Michael Paquier
On Wed, Nov 08, 2023 at 08:31:28PM +1300, David Rowley wrote: > On Wed, 8 Nov 2023 at 19:56, Peter Smith wrote: >> In passing I found/fixed a bunch of similar misuses in comments. > > Those all look fine to me too. +1. -- Michael signature.asc Description: PGP signature

Re: Add new option 'all' to pg_stat_reset_shared()

2023-11-08 Thread torikoshia
On 2023-11-09 08:58, Michael Paquier wrote: On Wed, Nov 08, 2023 at 02:15:24PM +0530, Bharath Rupireddy wrote: On Wed, Nov 8, 2023 at 9:43 AM Andres Freund wrote: It's not like oids are a precious resource. It's a more confusing API to have to have to specify a NULL as an argument than not

Re: meson documentation build open issues

2023-11-08 Thread Andres Freund
Hi, On 2023-11-08 16:19:51 +0100, Peter Eisentraut wrote: > On 08.11.23 13:55, Christoph Berg wrote: > > Re: Peter Eisentraut > > > > If the problem is broken doc patches, then maybe a solution is to > > > > include the `xmllint --noout --valid` target in whatever the check-world > > > >

Re: meson documentation build open issues

2023-11-08 Thread Andres Freund
Hi, On 2023-11-08 13:55:02 +0100, Christoph Berg wrote: > Re: Peter Eisentraut > > > If the problem is broken doc patches, then maybe a solution is to > > > include the `xmllint --noout --valid` target in whatever the check-world > > > equivalent is for meson. Looking at

Re: Force the old transactions logs cleanup even if checkpoint is skipped

2023-11-08 Thread Michael Paquier
On Wed, Nov 08, 2023 at 12:44:09PM +, Zakhlystov, Daniil (Nebius) wrote: >> I am not sure to understand your last sentence here. Once the >> archiver is back up, you mean that the WAL segments that were not >> previously archived still are still not archived? Or do you mean that >> because

Re: POC PATCH: copy from ... exceptions to: (was Re: VLDB Features)

2023-11-08 Thread Damir Belyalov
Hello everyone! Thanks for turning back to this patch. I had already thought about storing errors in the table / separate file / logfile and it seems to me that the best way is to output errors in logfile. As for user it is more convenient to look for errors in the place where they are usually

Re: POC PATCH: copy from ... exceptions to: (was Re: VLDB Features)

2023-11-08 Thread Andres Freund
Hi, On 2023-11-08 19:00:01 -0500, Tom Lane wrote: > Andres Freund writes: > > On 2023-11-08 13:18:39 -0500, Tom Lane wrote: > >> I think an actually usable feature of this sort would involve > >> copying all the failed lines to some alternate output medium, > >> perhaps a second table with a

Re: POC PATCH: copy from ... exceptions to: (was Re: VLDB Features)

2023-11-08 Thread Tom Lane
Andres Freund writes: > On 2023-11-08 13:18:39 -0500, Tom Lane wrote: >> I think an actually usable feature of this sort would involve >> copying all the failed lines to some alternate output medium, >> perhaps a second table with a TEXT column to receive the original >> data line. > If we go in

Re: Add new option 'all' to pg_stat_reset_shared()

2023-11-08 Thread Michael Paquier
On Wed, Nov 08, 2023 at 02:15:24PM +0530, Bharath Rupireddy wrote: > On Wed, Nov 8, 2023 at 9:43 AM Andres Freund wrote: >> It's not like oids are a precious resource. It's a more confusing API to have >> to have to specify a NULL as an argument than not having to do so. If we >> really want to

Re: POC PATCH: copy from ... exceptions to: (was Re: VLDB Features)

2023-11-08 Thread Andres Freund
Hi, On 2023-11-08 13:18:39 -0500, Tom Lane wrote: > Damir writes: > > [ v7-0002-Add-new-COPY-option-IGNORE_DATATYPE_ERRORS.patch ] > > Sorry for being so late to the party, but ... I don't think this > is a well-designed feature as it stands. Simply dropping failed rows > seems like an

Some deleted GUCs are still referred to

2023-11-08 Thread Peter Smith
Hi, I happened to notice that some GUC names "max_fsm_pages" and "max_fsm_relations" are still mentioned in these translation files (from the REL_16_1 source zip) src\backend\po\fr.po src\backend\po\tr.po ~~ Should those be removed? There was a commit [1] that said these all traces of those

Re: ResourceOwner refactoring

2023-11-08 Thread Heikki Linnakangas
On 08/11/2023 20:00, Alexander Lakhin wrote: Please look at a new assertion failure, I've managed to trigger with this script: CREATE TABLE t( i01 int, i02 int, i03 int, i04 int, i05 int, i06 int, i07 int, i08 int, i09 int, i10 int, i11 int, i12 int, i13 int, i14 int, i15 int, i16 int, i17

Re: meson documentation build open issues

2023-11-08 Thread Andres Freund
Hi, I really like the idea of an 'help' target that prints the targets. It seemed annoying to document such targets in both the sgml docs and the input for a the help target. Particularly due to the redundancies between id attributes, the target name etc. First I generated the list of targets

Re: speed up a logical replica setup

2023-11-08 Thread Michael Paquier
On Wed, Nov 08, 2023 at 09:50:47AM -0300, Euler Taveira wrote: > On Tue, Nov 7, 2023, at 8:12 PM, Michael Paquier wrote: > I used the CreateReplicationSlot() from streamutil.h but decided to use the > CREATE_REPLICATION_SLOT command directly because it needs the LSN as output. > As > you noticed

Re: Remove MSVC scripts from the tree

2023-11-08 Thread Michael Paquier
On Wed, Nov 08, 2023 at 09:41:19AM +0100, Peter Eisentraut wrote: > I don't think we should rely on sed being there on Windows. Maybe it's true > now on the handful of buildfarm/CI machines and early adopters, but do we > have any indication that that is systematic or just an accident? Or both?

Re: Moving forward with TDE [PATCH v3]

2023-11-08 Thread David Christensen
On Tue, Nov 7, 2023 at 5:49 PM Andres Freund wrote: > Hi, > > On 2023-11-06 09:56:37 -0500, Stephen Frost wrote: > > * Andres Freund (and...@anarazel.de) wrote: > > > I still am quite quite unconvinced that using the LSN as a nonce is a > good > > > design decision. > > > > This is a really

Re: pg_upgrade and logical replication

2023-11-08 Thread Michael Paquier
On Wed, Nov 08, 2023 at 10:52:29PM +0530, vignesh C wrote: > Upgrading logical replication nodes requires multiple steps to be > performed. Because not all operations are transactional, the user is > advised to take backups. > Backups can be taken as described in >

Re: GUC names in messages

2023-11-08 Thread Peter Smith
On Wed, Nov 8, 2023 at 7:40 AM Peter Smith wrote: > > FWIW, I am halfway through doing regex checking of the PG16 source for > all GUC names in messages to see what current styles are in use today. > > Not sure if those numbers will influence the decision. > > I hope I can post my findings today

Re: Moving forward with TDE [PATCH v3]

2023-11-08 Thread David Christensen
On Tue, Nov 7, 2023 at 6:47 PM Andres Freund wrote: > Hi, > > On 2023-11-06 11:26:44 +0100, Matthias van de Meent wrote: > > On Sat, 4 Nov 2023 at 03:38, Andres Freund wrote: > > > On 2023-11-02 22:09:40 +0100, Matthias van de Meent wrote: > > > > I'm quite surprised at the significant number

Re: Wrong rows estimations with joins of CTEs slows queries by more than factor 500

2023-11-08 Thread Tom Lane
Jian Guo writes: > I found a new approach to fix this issue, which seems better, so I would like > to post another version of the patch here. The origin patch made the > assumption of the values of Vars from CTE must be unique, which could be very > wrong. This patch examines variables for

Re: Wrong sentence in the README?

2023-11-08 Thread Bruce Momjian
On Sun, Sep 22, 2019 at 12:28:02PM -0400, Tom Lane wrote: > "Daniel Westermann (DWE)" writes: > > in the README, top level, there is this: > > > PostgreSQL has many language interfaces, many of which are listed here: > > https://www.postgresql.org/download > > > I don't think the download page

Re: XX000: tuple concurrently deleted during DROP STATISTICS

2023-11-08 Thread Tomas Vondra
On 11/8/23 20:58, Tom Lane wrote: > Tomas Vondra writes: >> On 11/8/23 16:52, Tom Lane wrote: >>> Shouldn't DROP STATISTICS be taking a lock on the associated table >>> that is strong enough to lock out ANALYZE? > >> Yes, I think that's the correct thing to do. I recall having a >> discussion

Re: Syncrep and improving latency due to WAL throttling

2023-11-08 Thread Andres Freund
Hi, On 2023-11-08 19:29:38 +0100, Tomas Vondra wrote: > >>> I haven't checked, but I'd assume that 100bytes back and forth should > >>> easily > >>> fit a new message to update LSNs and the existing feedback response. Even > >>> just > >>> the difference between sending 100 bytes and sending

Re: max_parallel_workers question

2023-11-08 Thread Bruce Momjian
On Sat, Sep 28, 2019 at 12:10:53AM -0400, Robert Haas wrote: > On Fri, Sep 27, 2019 at 8:07 PM Jeff Davis wrote: > > The current docs for max_parallel_workers start out: > > > > "Sets the maximum number of workers that the system can support for > > parallel operations..." > > > > In my

Re: XID-wraparound hazards in LISTEN/NOTIFY

2023-11-08 Thread Bruce Momjian
On Wed, Nov 8, 2023 at 02:52:16PM -0500, Tom Lane wrote: > Bruce Momjian writes: > > On Sat, Nov 23, 2019 at 12:10:56PM -0500, Tom Lane wrote: > >>> It suddenly strikes me to worry that we have an XID wraparound hazard > >>> for entries in the notify queue. > > > Is this still an open issue?

Re: POC PATCH: copy from ... exceptions to: (was Re: VLDB Features)

2023-11-08 Thread Tom Lane
Daniel Gustafsson writes: >> On 8 Nov 2023, at 19:18, Tom Lane wrote: >> I think an actually usable feature of this sort would involve >> copying all the failed lines to some alternate output medium, >> perhaps a second table with a TEXT column to receive the original >> data line. (Or maybe an

Re: XX000: tuple concurrently deleted during DROP STATISTICS

2023-11-08 Thread Tom Lane
Tomas Vondra writes: > On 11/8/23 16:52, Tom Lane wrote: >> Shouldn't DROP STATISTICS be taking a lock on the associated table >> that is strong enough to lock out ANALYZE? > Yes, I think that's the correct thing to do. I recall having a > discussion about this with someone while working on the

Re: XID-wraparound hazards in LISTEN/NOTIFY

2023-11-08 Thread Tom Lane
Bruce Momjian writes: > On Sat, Nov 23, 2019 at 12:10:56PM -0500, Tom Lane wrote: >>> It suddenly strikes me to worry that we have an XID wraparound hazard >>> for entries in the notify queue. > Is this still an open issue? Should it be a TODO item? I don't think anyone's done anything about

Re: POC PATCH: copy from ... exceptions to: (was Re: VLDB Features)

2023-11-08 Thread Daniel Gustafsson
> On 8 Nov 2023, at 19:18, Tom Lane wrote: > I think an actually usable feature of this sort would involve > copying all the failed lines to some alternate output medium, > perhaps a second table with a TEXT column to receive the original > data line. (Or maybe an array of text that could

Re: XX000: tuple concurrently deleted during DROP STATISTICS

2023-11-08 Thread Tomas Vondra
On 11/8/23 16:52, Tom Lane wrote: > Tomas Vondra writes: >> On 11/8/23 16:10, Justin Pryzby wrote: >>> I found this in our logs, and reproduced it under v11-v16. >>> >>> CREATE TABLE t(a int, b int); >>> INSERT INTO t SELECT generate_series(1,999); >>> CREATE STATISTICS t_stats ON a,b FROM t; >>>

Re: Fix some memory leaks in ecpg.addons

2023-11-08 Thread Alexander Lakhin
Hello Tristan, 08.11.2023 20:37, Tristan Partin wrote: Are people using some suppression file or setting ASAN_OPTIONS to something? I use the following: ASAN_OPTIONS=detect_leaks=0:abort_on_error=1:print_stacktrace=1:\ disable_coredump=0:strict_string_checks=1:check_initialization_order=1:\

Re: XID-wraparound hazards in LISTEN/NOTIFY

2023-11-08 Thread Bruce Momjian
On Sat, Nov 23, 2019 at 12:10:56PM -0500, Tom Lane wrote: > Mark Dilger writes: > > On 11/23/19 8:34 AM, Tom Lane wrote: > >> It suddenly strikes me to worry that we have an XID wraparound hazard > >> for entries in the notify queue. > > > Is it worth checking for this condition in autovacuum? >

Re: Syncrep and improving latency due to WAL throttling

2023-11-08 Thread Tomas Vondra
On 11/8/23 18:11, Andres Freund wrote: > Hi, > > On 2023-11-08 13:59:55 +0100, Tomas Vondra wrote: >>> I used netperf's tcp_rr between my workstation and my laptop on a local >>> 10Gbit >>> network (albeit with a crappy external card for my laptop), to put some >>> numbers to this. I used -r

Re: POC PATCH: copy from ... exceptions to: (was Re: VLDB Features)

2023-11-08 Thread Tom Lane
Damir writes: > [ v7-0002-Add-new-COPY-option-IGNORE_DATATYPE_ERRORS.patch ] Sorry for being so late to the party, but ... I don't think this is a well-designed feature as it stands. Simply dropping failed rows seems like an unusable definition for any application that has pretensions of

Re: Requiring recovery.signal or standby.signal when recovering with a backup_label

2023-11-08 Thread Robert Haas
On Tue, Oct 31, 2023 at 7:39 PM Michael Paquier wrote: > Point 7. of what you quote says to use one? True that this needs a > refresh, and perhaps a bit fat warning about the fact that these are > required if you want to fetch WAL from other sources than the local > pg_wal/. Perhaps there may

Re: ResourceOwner refactoring

2023-11-08 Thread Alexander Lakhin
Hello Heikki, 08.11.2023 14:37, Heikki Linnakangas wrote: Fixed these, and pushed. Thanks everyone for reviewing! Please look at a new assertion failure, I've managed to trigger with this script: CREATE TABLE t( i01 int, i02 int, i03 int, i04 int, i05 int, i06 int, i07 int, i08 int, i09

Re: cataloguing NOT NULL constraints

2023-11-08 Thread Alvaro Herrera
On 2023-Oct-12, Alexander Lakhin wrote: Hello, > I've discovered that that commit added several recursive functions, and > some of them are not protected from stack overflow. True. I reproduced the first two, but didn't attempt to reproduce the third one -- patching all these to check for

Re: Fix some memory leaks in ecpg.addons

2023-11-08 Thread Tom Lane
"Tristan Partin" writes: > On Wed Nov 8, 2023 at 11:18 AM CST, Michael Meskes wrote: >> Agreed, it's not exactly uncommon for tools like ecpg to not worry >> about memory. After all it gets freed when the program ends. > In the default configuration of AddressSanitizer, I can't even complete >

Re: Version 14/15 documentation Section "Alter Default Privileges"

2023-11-08 Thread Bruce Momjian
On Wed, Nov 8, 2023 at 01:12:24PM +0100, Laurenz Albe wrote: > On Tue, 2023-11-07 at 17:30 -0500, Bruce Momjian wrote: > > You didn't seem to like my SET ROLE suggestion so I removed it. > > I thought that the information that you can use SET ROLE to assume > the identity of another role is

Re: Fix some memory leaks in ecpg.addons

2023-11-08 Thread Tristan Partin
On Wed Nov 8, 2023 at 11:18 AM CST, Michael Meskes wrote: Am Mittwoch, dem 08.11.2023 um 12:07 -0500 schrieb Tom Lane: > "Tristan Partin" writes: > > clang and gcc both now support -fsanitize=address,undefined. These > > are > > really useful to me personally when trying to debug issues. > >

Re: [PoC] pg_upgrade: allow to upgrade publisher node

2023-11-08 Thread vignesh C
On Wed, 8 Nov 2023 at 08:43, vignesh C wrote: > > On Tue, 7 Nov 2023 at 13:25, Amit Kapila wrote: > > > > On Tue, Nov 7, 2023 at 10:01 AM Zhijie Hou (Fujitsu) > > wrote: > > > > > > On Tuesday, November 7, 2023 12:14 PM Kuroda, Hayato/黒田 隼人 > > > wrote: > > > > > > > > Dear hackers, > > > > >

Re: meson documentation build open issues

2023-11-08 Thread Andres Freund
Hi, On 2023-11-08 12:04:30 +0100, Peter Eisentraut wrote: > Ok, I didn't know about ninja install-world. That works for me. Maybe a > "world" target would also be good. Yea, I thought so as well. I'll send out a patch shortly. Kinda wondering if its worth backpatching to 16. Uniformity seems

Re: pg_upgrade and logical replication

2023-11-08 Thread vignesh C
On Thu, 2 Nov 2023 at 17:01, Amit Kapila wrote: > > On Thu, Nov 2, 2023 at 3:41 PM vignesh C wrote: > > > > I have slightly modified it now and also made it consistent with the > > replication slot upgrade, but I was not sure if we need to add > > anything more. Let me know if anything else

Re: Add the ability to limit the amount of memory that can be allocated to backends.

2023-11-08 Thread Andres Freund
Hi, On 2023-11-07 15:55:48 -0500, Stephen Frost wrote: > * Andres Freund (and...@anarazel.de) wrote: > > On 2023-11-06 13:02:50 -0500, Stephen Frost wrote: > > > > > The max_total_memory limit is checked whenever the global counters are > > > > > updated. There is no special error handling if a

Re: Fix some memory leaks in ecpg.addons

2023-11-08 Thread Michael Meskes
Am Mittwoch, dem 08.11.2023 um 12:07 -0500 schrieb Tom Lane: > "Tristan Partin" writes: > > clang and gcc both now support -fsanitize=address,undefined. These > > are > > really useful to me personally when trying to debug issues. > > Unfortunately ecpg code has a ton of memory leaks, which

Re: Syncrep and improving latency due to WAL throttling

2023-11-08 Thread Andres Freund
Hi, On 2023-11-08 13:59:55 +0100, Tomas Vondra wrote: > > I used netperf's tcp_rr between my workstation and my laptop on a local > > 10Gbit > > network (albeit with a crappy external card for my laptop), to put some > > numbers to this. I used -r $s,100 to test sending a variable sized data to

Re: Fix some memory leaks in ecpg.addons

2023-11-08 Thread Tom Lane
"Tristan Partin" writes: > clang and gcc both now support -fsanitize=address,undefined. These are > really useful to me personally when trying to debug issues. > Unfortunately ecpg code has a ton of memory leaks, which makes builds > really painful. It would be great to fix all of them, but I

Fix some memory leaks in ecpg.addons

2023-11-08 Thread Tristan Partin
clang and gcc both now support -fsanitize=address,undefined. These are really useful to me personally when trying to debug issues. Unfortunately ecpg code has a ton of memory leaks, which makes builds really painful. It would be great to fix all of them, but I don't have the patience to try to

Re: Add PQsendSyncMessage() to libpq

2023-11-08 Thread Alvaro Herrera
On 2023-Nov-07, Jelte Fennema-Nio wrote: > I think this function should be named something with the "PQsend" > prefix since that's the way we name all our public async message > sending functions in libpq. The "Put" word we only use in internal > libpq functions, so I feel it has no place in the

Re: Call pqPipelineFlush from PQsendFlushRequest

2023-11-08 Thread Alvaro Herrera
On 2023-Nov-07, Michael Paquier wrote: > On Tue, Nov 07, 2023 at 10:38:04AM +0100, Jelte Fennema-Nio wrote: > > In pipeline mode after queuing a message to be sent we would flush the > > buffer if the size of the buffer passed some threshold. The only message > > type that we didn't do that for

Re: 64-bit integer subtraction bug on some platforms

2023-11-08 Thread Tom Lane
Laurenz Albe writes: > On Wed, 2023-11-08 at 11:58 +, Dean Rasheed wrote: >> This should overflow, since the correct result (+9223372036854775808) >> is out of range. However, on platforms without integer overflow >> builtins or 128-bit integers, pg_sub_s64_overflow() does the >> following:

Re: Fix use of openssl.path() if openssl isn't found

2023-11-08 Thread Tristan Partin
On Wed Nov 8, 2023 at 2:31 AM CST, Michael Paquier wrote: On Wed, Nov 08, 2023 at 12:07:49AM -0600, Tristan Partin wrote: >'with_ssl': ssl_library, > - 'OPENSSL': openssl.path(), > + 'OPENSSL': openssl.found() ? openssl.path : '', Except that this was incorrect. I've fixed

Re: Cleaning up array_in()

2023-11-08 Thread Tom Lane
Alexander Lakhin writes: > Thank you for the update! I haven't looked into the code, just did manual > testing and rechecked commands given in the arrays documentation ([1]). > Everything works correctly, except for one minor difference: > INSERT INTO sal_emp >     VALUES ('Bill', >    

Re: meson documentation build open issues

2023-11-08 Thread Tristan Partin
Looks good to me. Thanks for finding this. -- Tristan Partin Neon (https://neon.tech)

Re: meson documentation build open issues

2023-11-08 Thread Alvaro Herrera
On 2023-Nov-08, Peter Eisentraut wrote: > I think we could build doc/src/sgml/postgres-full.xml by default. That > takes less than 0.5 seconds here and it's an intermediate target for html > and man. If that detects problems like the id attributes you mentioned, apart from the other checks in

Re: XX000: tuple concurrently deleted during DROP STATISTICS

2023-11-08 Thread Tom Lane
Tomas Vondra writes: > On 11/8/23 16:10, Justin Pryzby wrote: >> I found this in our logs, and reproduced it under v11-v16. >> >> CREATE TABLE t(a int, b int); >> INSERT INTO t SELECT generate_series(1,999); >> CREATE STATISTICS t_stats ON a,b FROM t; >> >> while :; do psql postgres -qtxc

Re: XX000: tuple concurrently deleted during DROP STATISTICS

2023-11-08 Thread Tomas Vondra
On 11/8/23 16:10, Justin Pryzby wrote: > I found this in our logs, and reproduced it under v11-v16. > > CREATE TABLE t(a int, b int); > INSERT INTO t SELECT generate_series(1,999); > CREATE STATISTICS t_stats ON a,b FROM t; > > while :; do psql postgres -qtxc "ANALYZE t"; done & > while :; do

  1   2   >