Re: [BUG] Fix DETACH with FK pointing to a partitioned table fails

2024-09-26 Thread Jehan-Guillaume de Rorthais
On Wed, 25 Sep 2024 16:14:07 +0200 Jehan-Guillaume de Rorthais wrote: > On Wed, 25 Sep 2024 14:42:40 +0200 > Jehan-Guillaume de Rorthais wrote: > > > On Thu, 5 Sep 2024 00:57:28 +0200 > > Jehan-Guillaume de Rorthais wrote: > […] > > > > > > Pleas

Re: [BUG] Fix DETACH with FK pointing to a partitioned table fails

2024-09-25 Thread Jehan-Guillaume de Rorthais
On Wed, 25 Sep 2024 14:42:40 +0200 Jehan-Guillaume de Rorthais wrote: > On Thu, 5 Sep 2024 00:57:28 +0200 > Jehan-Guillaume de Rorthais wrote: […] > > > > Please, find in attachment a patch implementing this idea. > > Please, find in attachment a set of patch b

Re: [BUG] Fix DETACH with FK pointing to a partitioned table fails

2024-09-25 Thread Jehan-Guillaume de Rorthais
On Thu, 5 Sep 2024 00:57:28 +0200 Jehan-Guillaume de Rorthais wrote: > On Mon, 2 Sep 2024 23:01:47 +0200 > Jehan-Guillaume de Rorthais wrote: > > […] > > > My proposal was to clean everything related to the old FK and use some > > existing code path to create a

Re: [BUG] Fix DETACH with FK pointing to a partitioned table fails

2024-09-04 Thread Jehan-Guillaume de Rorthais
On Mon, 2 Sep 2024 23:01:47 +0200 Jehan-Guillaume de Rorthais wrote: […] > My proposal was to clean everything related to the old FK and use some > existing code path to create a fresh and cleaner one. This requires some > refactoring in existing code, but we would win a commo

Re: [BUG] Fix DETACH with FK pointing to a partitioned table fails

2024-09-03 Thread Jehan-Guillaume de Rorthais
Hi Tender, On Tue, 3 Sep 2024 10:16:44 +0800 Tender Wang wrote: > Jehan-Guillaume de Rorthais 于2024年9月3日周二 05:02写道: […] > > * Constraint & trigger catalog cleanup [1] (this thread) > > * FK broken after DETACH [2] > > * Maintenance consideration about self referenc

Re: [BUG] Fix DETACH with FK pointing to a partitioned table fails

2024-09-02 Thread Jehan-Guillaume de Rorthais
Hi, On Tue, 20 Aug 2024 23:09:27 -0400 Alvaro Herrera wrote: > On 2024-Aug-20, Jehan-Guillaume de Rorthais wrote: > > > I'm back on this issue as well. I start poking at this patch to review it, > > test it, challenge it and then report here. > > > > I&#

Re: [BUG] Fix DETACH with FK pointing to a partitioned table fails

2024-08-20 Thread Jehan-Guillaume de Rorthais
On Wed, 7 Aug 2024 18:50:10 -0400 Alvaro Herrera wrote: > On 2024-Jul-26, Tender Wang wrote: > > > Junwang Zhao 于2024年7月26日周五 14:57写道: > > > > > There is a bug report[0] Tender comments might be the same issue as > > > this one, but I tried Alvaro's and mine patch, neither could solve > > > th

Re: Sorting regression of text function result since commit 586b98fdf1aae

2023-12-12 Thread Jehan-Guillaume de Rorthais
On Mon, 11 Dec 2023 15:43:12 -0500 Tom Lane wrote: > Jehan-Guillaume de Rorthais writes: > > It looks like since 586b98fdf1aae, the result type collation of > > "convert_from" is forced to "C", like the patch does for type "name", > >

Sorting regression of text function result since commit 586b98fdf1aae

2023-12-11 Thread Jehan-Guillaume de Rorthais
Hi, A customer found what looks like a sort regression while testing his code from v11 on a higher version. We hunt this regression down to commit 586b98fdf1aae, introduced in v12. Consider the following test case: createdb -l fr_FR.utf8 -T template0 reg psql reg <<<" BEGIN; CREATE TABLE

Re: Query execution in Perl TAP tests needs work

2023-11-06 Thread Jehan-Guillaume de Rorthais
On Wed, 18 Oct 2023 18:25:01 +0200 Alvaro Herrera wrote: > On 2023-Oct-18, Robert Haas wrote: > > > Without FFI::Platypus, we have to write Perl code that can speak the > > wire protocol directly. Basically, we're writing our own PostgreSQL > > driver for Perl, though we might need only a subset

Re: Detoasting optionally to make Explain-Analyze less misleading

2023-09-13 Thread Jehan-Guillaume de Rorthais
Hi Stepan & all, On Tue, 12 Sep 2023 17:16:00 +0200 stepan rutz wrote: ... > Attached a new patch. Hoping for feedback, Nice addition to EXPLAIN! On the feature front, what about adding the actual detoasting/serializing time in the explain output? That could be: => explain (analyze,serial

Re: EBCDIC sorting as a use case for ICU rules

2023-08-24 Thread Jehan-Guillaume de Rorthais
Hi, Sorry to chime in so lately, I was waiting for some customer feedback. On Wed, 21 Jun 2023 15:28:38 +0200 "Daniel Verite" wrote: > At a conference this week I was asked if ICU could be able to > sort like EBCDIC [2]. > It turns out it has been already asked on > -general a few years ag

Re: [BUG] Fix DETACH with FK pointing to a partitioned table fails

2023-08-10 Thread Jehan-Guillaume de Rorthais
On Thu, 3 Aug 2023 11:02:43 +0200 Alvaro Herrera wrote: > On 2023-Aug-03, tender wang wrote: > > > I think old "sub-FK" should not be dropped, that will be violates foreign > > key constraint. > > Yeah, I've been playing more with the patch and it is definitely not > doing the right things.

Re: Issue attaching a table to a partitioned table with an auto-referenced foreign key

2023-07-07 Thread Jehan-Guillaume de Rorthais
So I gave a look at this one... And it's a tricky one. The current policy about DETACHing a partition is to keep/adjust all FK referencing it or referenced by it. However, in this exact self-referencing usecase, we can have rows referencing rows from the same partition OR another one. It seems li

[BUG] Fix DETACH with FK pointing to a partitioned table fails

2023-07-05 Thread Jehan-Guillaume de Rorthais
about it. At the very least, it help understanding the problem and sketch a possible solution. Regards, >From 5fc7997b9f9a17ee5a31f059c18e6c01fd716c04 Mon Sep 17 00:00:00 2001 From: Jehan-Guillaume de Rorthais Date: Wed, 5 Jul 2023 19:19:40 +0200 Subject: [PATCH v1] Remove useless parted-FK

Re: Memory leak from ExecutorState context?

2023-05-22 Thread Jehan-Guillaume de Rorthais
On Fri, 19 May 2023 17:23:56 +0200 Tomas Vondra wrote: > On 5/19/23 00:27, Melanie Plageman wrote: > > v10 LGTM. > > Thanks! > > I've pushed 0002 and 0003, after some general bikeshedding and minor > rewording (a bit audacious, admittedly). Thank you Melanie et Tomas for your help, review et

Re: Memory leak from ExecutorState context?

2023-05-17 Thread Jehan-Guillaume de Rorthais
On Wed, 17 May 2023 13:46:35 -0400 Melanie Plageman wrote: > On Wed, May 17, 2023 at 07:10:08PM +0200, Jehan-Guillaume de Rorthais wrote: > > On Tue, 16 May 2023 16:00:52 -0400 > > Melanie Plageman wrote: > > > ... > > > There are some existing indentation i

Re: Memory leak from ExecutorState context?

2023-05-17 Thread Jehan-Guillaume de Rorthais
On Tue, 16 May 2023 16:00:52 -0400 Melanie Plageman wrote: > On Tue, May 16, 2023 at 04:00:51PM +0200, Jehan-Guillaume de Rorthais wrote: > > > From e5ecd466172b7bae2f1be294c1a5e70ce2b43ed8 Mon Sep 17 00:00:00 2001 > > From: Melanie Plageman > > Date: Thu, 30

Re: Memory leak from ExecutorState context?

2023-05-16 Thread Jehan-Guillaume de Rorthais
Hi, On Tue, 16 May 2023 12:01:51 +0200 Tomas Vondra wrote: > On 5/16/23 00:15, Jehan-Guillaume de Rorthais wrote: > > On Sat, 13 May 2023 23:47:53 +0200 > > Tomas Vondra wrote: > ... > >> I'm not really sure about calling this "hybrid hash-join". Wha

Re: Memory leak from ExecutorState context?

2023-05-15 Thread Jehan-Guillaume de Rorthais
ll forward? + * * PARALLELISM * * Hash joins can participate in parallel query execution in several ways. A -- 2.40.1 >From 52e989d5eb6405e86e0d460dffbfaca292bc4274 Mon Sep 17 00:00:00 2001 From: Jehan-Guillaume de Rorthais Date: Mon, 27 Mar 2023 15:54:39 +0200 Subject: [PATCH 2/3] Allocate hash batches rel

Re: Memory leak from ExecutorState context?

2023-05-15 Thread Jehan-Guillaume de Rorthais
On Sun, 14 May 2023 00:10:00 +0200 Tomas Vondra wrote: > On 5/12/23 23:36, Melanie Plageman wrote: > > Thanks for continuing to work on this. > > > > Are you planning to modify what is displayed for memory usage in > > EXPLAIN ANALYZE? Yes, I already start to work on this. Tracking spilling mem

Re: Unlinking Parallel Hash Join inner batch files sooner

2023-05-10 Thread Jehan-Guillaume de Rorthais
Hi, Thanks for working on this! On Wed, 10 May 2023 15:11:20 +1200 Thomas Munro wrote: > One complaint about PHJ is that it can, in rare cases, use a > surprising amount of temporary disk space where non-parallel HJ would > not. When it decides that it needs to double the number of batches to

Re: Memory leak from ExecutorState context?

2023-05-10 Thread Jehan-Guillaume de Rorthais
llCxt"--indicating it > is for the memory required for spilling to permanent storage while > executing hash joins. "Spilling" seems fair and a large enough net to grab everything around temp files and accessing them. > I discuss this more in my code review below. > > &

Re: Memory leak from ExecutorState context?

2023-05-04 Thread Jehan-Guillaume de Rorthais
Hi, On Fri, 21 Apr 2023 16:44:48 -0400 Melanie Plageman wrote: > On Fri, Apr 7, 2023 at 8:01 PM Jehan-Guillaume de Rorthais > wrote: > > > > On Fri, 31 Mar 2023 14:06:11 +0200 > > Jehan-Guillaume de Rorthais wrote: > > > > > > [...] > > >

Re: Commitfest 2023-03 starting tomorrow!

2023-04-21 Thread Jehan-Guillaume de Rorthais
Hi, After catching up with this thread, where pending bugs are listed and discussed, I wonder if the current patches trying to lower the HashJoin memory explosion[1] could be added to the "Older bugs affecting stable branches" list of https://wiki.postgresql.org/wiki/PostgreSQL_16_Open_Items as I

[BUG] FK broken after DETACHing referencing part

2023-04-20 Thread Jehan-Guillaume de Rorthais
Hi, Considering two partitionned tables with a FK between them: DROP TABLE IF EXISTS p, c, c_1 CASCADE; -- -- Parent table + partition + data CREATE TABLE p ( id bigint PRIMARY KEY ) PARTITION BY list (id); CREATE TABLE p_1 PARTITION OF p FOR VA

Re: OOM in hash join

2023-04-14 Thread Jehan-Guillaume de Rorthais
On Fri, 14 Apr 2023 13:21:05 +0200 Matthias van de Meent wrote: > On Fri, 14 Apr 2023 at 12:59, Konstantin Knizhnik wrote: > > > > Hi hackers, > > > > Too small value of work_mem cause memory overflow in parallel hash join > > because of too much number batches. > > There is the plan: > > [..

Re: Memory leak from ExecutorState context?

2023-04-11 Thread Jehan-Guillaume de Rorthais
On Sat, 8 Apr 2023 02:01:19 +0200 Jehan-Guillaume de Rorthais wrote: > On Fri, 31 Mar 2023 14:06:11 +0200 > Jehan-Guillaume de Rorthais wrote: > > [...] > > After rebasing Tomas' memory balancing patch, I did some memory measures > to answer some of my questions.

Re: Memory leak from ExecutorState context?

2023-03-31 Thread Jehan-Guillaume de Rorthais
80 total in 3256 blocks; 970128 free (38180 chunks); 27635552 used HashBatchContext: 960544 total in 23 blocks; 7928 free (0 chunks); 952616 used Regards, >From 6814994fa0576a8ba6458412ac5f944135fc3813 Mon Sep 17 00:00:00 2001 From: Jehan-Guillaume de Rorthais D

Re: Memory leak from ExecutorState context?

2023-03-28 Thread Jehan-Guillaume de Rorthais
Hi, Sorry for the late answer, I was reviewing the first patch and it took me some time to study and dig around. On Thu, 23 Mar 2023 08:07:04 -0400 Melanie Plageman wrote: > On Fri, Mar 10, 2023 at 1:51 PM Jehan-Guillaume de Rorthais > wrote: > > > So I guess the best thing

Re: Memory leak from ExecutorState context?

2023-03-28 Thread Jehan-Guillaume de Rorthais
On Tue, 28 Mar 2023 00:43:34 +0200 Tomas Vondra wrote: > On 3/27/23 23:13, Jehan-Guillaume de Rorthais wrote: > > Please, find in attachment a patch to allocate bufFiles in a dedicated > > context. I picked up your patch, backpatch'd it, went through it and did > > s

Re: Memory leak from ExecutorState context?

2023-03-27 Thread Jehan-Guillaume de Rorthais
Hi, On Mon, 20 Mar 2023 15:12:34 +0100 Jehan-Guillaume de Rorthais wrote: > On Mon, 20 Mar 2023 09:32:17 +0100 > Tomas Vondra wrote: > > > >> * Patch 1 could be rebased/applied/backpatched > > > > > > Would it help if I rebase Patch 1 ("mo

Re: Memory leak from ExecutorState context?

2023-03-20 Thread Jehan-Guillaume de Rorthais
On Mon, 20 Mar 2023 09:32:17 +0100 Tomas Vondra wrote: > >> * Patch 1 could be rebased/applied/backpatched > > > > Would it help if I rebase Patch 1 ("move BufFile stuff into separate > > context")? > > Yeah, I think this is something we'd want to do. It doesn't change the > behavior, but it

Re: Memory leak from ExecutorState context?

2023-03-17 Thread Jehan-Guillaume de Rorthais
Hi there, On Fri, 10 Mar 2023 19:51:14 +0100 Jehan-Guillaume de Rorthais wrote: > > So I guess the best thing would be to go through these threads, see what > > the status is, restart the discussion and propose what to do. If you do > > that, I'm happy to rebase the patc

Re: Memory leak from ExecutorState context?

2023-03-10 Thread Jehan-Guillaume de Rorthais
Hi, > So I guess the best thing would be to go through these threads, see what > the status is, restart the discussion and propose what to do. If you do > that, I'm happy to rebase the patches, and maybe see if I could improve > them in some way. OK! It took me some time, but I did it. I'll try t

Re: Memory leak from ExecutorState context?

2023-03-02 Thread Jehan-Guillaume de Rorthais
On Thu, 2 Mar 2023 19:53:14 +0100 Tomas Vondra wrote: > On 3/2/23 19:15, Jehan-Guillaume de Rorthais wrote: ... > > There was some thoughts about how to make a better usage of the memory. As > > memory is exploding way beyond work_mem, at least, avoid to waste it with > >

Re: Memory leak from ExecutorState context?

2023-03-02 Thread Jehan-Guillaume de Rorthais
On Thu, 2 Mar 2023 19:15:30 +0100 Jehan-Guillaume de Rorthais wrote: [...] > For what it worth, these two patches seems really interesting to me. Do you > need any help to revive it? To avoid confusion, the two patches I meant were: * 0001-move-BufFile-stuff-into-separate-context.patch

Re: Memory leak from ExecutorState context?

2023-03-02 Thread Jehan-Guillaume de Rorthais
Hi! On Thu, 2 Mar 2023 13:44:52 +0100 Tomas Vondra wrote: > Well, yeah and no. > > In principle we could/should have allocated the BufFiles in a different > context (possibly hashCxt). But in practice it probably won't make any > difference, because the query will probably run all the hashjoins

Re: Memory leak from ExecutorState context?

2023-03-02 Thread Jehan-Guillaume de Rorthais
On Thu, 2 Mar 2023 01:30:27 +0100 Tomas Vondra wrote: > On 3/2/23 00:18, Jehan-Guillaume de Rorthais wrote: > >>> ExecHashIncreaseNumBatches was really chatty, having hundreds of thousands > >>> of calls, always short-cut'ed to 1048576, I guess because of the &g

Re: Memory leak from ExecutorState context?

2023-03-01 Thread Jehan-Guillaume de Rorthais
Hi, On Wed, 1 Mar 2023 20:29:11 +0100 Tomas Vondra wrote: > On 3/1/23 18:48, Jehan-Guillaume de Rorthais wrote: > > On Tue, 28 Feb 2023 20:51:02 +0100 > > Tomas Vondra wrote: > >> On 2/28/23 19:06, Jehan-Guillaume de Rorthais wrote: > >>> * HashBatchConte

Re: Memory leak from ExecutorState context?

2023-03-01 Thread Jehan-Guillaume de Rorthais
On Wed, 1 Mar 2023 20:34:08 +0100 Tomas Vondra wrote: > On 3/1/23 19:09, Jehan-Guillaume de Rorthais wrote: > > On Wed, 1 Mar 2023 18:48:40 +0100 > > Jehan-Guillaume de Rorthais wrote: > > ... > >> You'll find some intermediate stats I already collected in

Re: Memory leak from ExecutorState context?

2023-03-01 Thread Jehan-Guillaume de Rorthais
On Wed, 1 Mar 2023 18:48:40 +0100 Jehan-Guillaume de Rorthais wrote: ... > You'll find some intermediate stats I already collected in attachment: > > * break 1, 2 and 3 are from AllocSetAlloc, break 4 is from AllocSetFree. > * most of the non-free'd chunk are allocated si

Re: Memory leak from ExecutorState context?

2023-03-01 Thread Jehan-Guillaume de Rorthais
Hi Tomas, On Tue, 28 Feb 2023 20:51:02 +0100 Tomas Vondra wrote: > On 2/28/23 19:06, Jehan-Guillaume de Rorthais wrote: > > * HashBatchContext goes up to 1441MB after 240s then stay flat until the end > > (400s as the last record) > > That's interesting. We

Re: Memory leak from ExecutorState context?

2023-03-01 Thread Jehan-Guillaume de Rorthais
Hi Justin, On Tue, 28 Feb 2023 12:25:08 -0600 Justin Pryzby wrote: > On Tue, Feb 28, 2023 at 07:06:43PM +0100, Jehan-Guillaume de Rorthais wrote: > > Hello all, > > > > A customer is facing out of memory query which looks similar to this > > situa

Re: Transparent column encryption

2022-11-24 Thread Jehan-Guillaume de Rorthais
On Wed, 23 Nov 2022 19:45:06 +0100 Peter Eisentraut wrote: > On 28.10.22 12:16, Jehan-Guillaume de Rorthais wrote: [...] > >* I wonder if encryption related fields in ParameterDescription and > > RowDescription could be optional somehow? The former might be quite >

Re: Fix proposal for comparaison bugs in PostgreSQL::Version

2022-11-04 Thread Jehan-Guillaume de Rorthais
On Thu, 3 Nov 2022 13:11:18 -0500 Justin Pryzby wrote: > On Tue, Jun 28, 2022 at 06:17:40PM -0400, Andrew Dunstan wrote: > > Nice catch, but this looks like massive overkill. I think we can very > > simply fix the test in just a few lines of code, instead of a 190 line > > fix and a 130 line TAP

Re: [BUG] parenting a PK constraint to a self-FK one (Was: Self FK oddity when attaching a partition)

2022-11-04 Thread Jehan-Guillaume de Rorthais
On Thu, 3 Nov 2022 20:44:16 +0100 Alvaro Herrera wrote: > On 2022-Oct-05, Alvaro Herrera wrote: > > > I've been giving the patches a look and it caused me to notice two > > additional bugs in the same area: > > > > - FKs in partitions are sometimes marked NOT VALID. This is because of > > mi

Re: psql: Add command to use extended query protocol

2022-11-02 Thread Jehan-Guillaume de Rorthais
On Wed, 02 Nov 2022 16:04:02 +0100 "Daniel Verite" wrote: > Jehan-Guillaume de Rorthais wrote: > > > As I wrote in my TCE review, would it be possible to use psql vars to set > > some named parameters for the prepared query? This would looks like: > >

Re: psql: Add command to use extended query protocol

2022-11-02 Thread Jehan-Guillaume de Rorthais
Hi, On Fri, 28 Oct 2022 08:52:51 +0200 Peter Eisentraut wrote: > This adds a new psql command \gp that works like \g (or semicolon) but > uses the extended query protocol. Parameters can also be passed, like > > SELECT $1, $2 \gp 'foo' 'bar' As I wrote in my TCE review, would it be possi

Re: Commitfest documentation

2022-10-31 Thread Jehan-Guillaume de Rorthais
Hi Aleksander, Thank you for your help! On Mon, 31 Oct 2022 16:51:23 +0300 Aleksander Alekseev wrote: [...] > > In the commitfest application, I was wondering today what was the exact > > meaning and difference between open/closed status (is it only for the > > current commitfest?) > > Close

Commitfest documentation

2022-10-31 Thread Jehan-Guillaume de Rorthais
Hi, In the commitfest application, I was wondering today what was the exact meaning and difference between open/closed status (is it only for the current commitfest?) and between «waiting for author» and «Returned with feedback». I couldn't find a clear definition searching the wiki, the mailing

Re: Transparent column encryption

2022-10-28 Thread Jehan-Guillaume de Rorthais
Hi, I did a review of the documentation and usability. # Applying patch The patch applied on top of f13b2088fa2 without trouble. Notice a small warning during compilation: colenccmds.c:134:27: warning: ‘encval’ may be used uninitialized A simple fix could be: +++ b/src/bac

Re: [BUG] parenting a PK constraint to a self-FK one (Was: Self FK oddity when attaching a partition)

2022-10-03 Thread Jehan-Guillaume de Rorthais
On Fri, 30 Sep 2022 16:11:09 -0700 Zhihong Yu wrote: > On Fri, Sep 30, 2022 at 3:30 PM Jehan-Guillaume de Rorthais > wrote: ... > > +* Self-Foreign keys are ignored as the index was preliminary > created > > preliminary created -> primarily created Than

Re: [BUG] parenting a PK constraint to a self-FK one (Was: Self FK oddity when attaching a partition)

2022-09-30 Thread Jehan-Guillaume de Rorthais
handle missing index without paying attention to related constraints? Regards, On Wed, 24 Aug 2022 12:49:13 +0200 Alvaro Herrera wrote: > On 2022-Aug-24, Jehan-Guillaume de Rorthais wrote: > > > I was naively wondering about such a patch, but was worrying about potential > >

Re: [BUG] wrong FK constraint name when colliding name on ATTACH

2022-09-08 Thread Jehan-Guillaume de Rorthais
On Thu, 8 Sep 2022 13:25:15 +0200 Alvaro Herrera wrote: > On 2022-Sep-08, Jehan-Guillaume de Rorthais wrote: > > > Hi there, > > > > I believe this very small bug and its fix are really trivial and could be > > push out of the way quite quickly. It's just ab

Re: [BUG] wrong FK constraint name when colliding name on ATTACH

2022-09-08 Thread Jehan-Guillaume de Rorthais
not wrong :) Regards, On Thu, 1 Sep 2022 18:41:56 +0200 Jehan-Guillaume de Rorthais wrote: > While studying and hacking on the parenting constraint issue, I found an > incoherent piece of code leading to badly chosen fk name. If a constraint > name collision is detected, while choosing

[BUG] wrong FK constraint name when colliding name on ATTACH

2022-09-01 Thread Jehan-Guillaume de Rorthais
the original code wanted to add. The expected name is "child1_id_abc_no_part_fkey". Find in attachment a simple fix, moving the name assignation after the FK attributes are populated. Regards, >From f1eeacb1eb3face38d76325666aff57019ef84c9 Mon Sep 17 00:00:00 2001 From: Jehan-Gu

Re: [BUG] parenting a PK constraint to a self-FK one (Was: Self FK oddity when attaching a partition)

2022-08-24 Thread Jehan-Guillaume de Rorthais
On Tue, 23 Aug 2022 18:30:06 +0200 Alvaro Herrera wrote: > On 2022-Aug-23, Jehan-Guillaume de Rorthais wrote: > > Hi, > > [...] > > > However, it seems get_relation_idx_constraint_oid(), introduced in > > eb7ed3f3063, assume there could be only ONE constraint dep

[BUG] parenting a PK constraint to a self-FK one (Was: Self FK oddity when attaching a partition)

2022-08-23 Thread Jehan-Guillaume de Rorthais
Hi all, I've been able to work on this issue and isolate where in the code the oddity is laying. During ATExecAttachPartition(), AttachPartitionEnsureIndexes() look for existing required index on the partition to attach. It creates missing index, or sets the parent's index when a matching one exi

Re: Fix proposal for comparaison bugs in PostgreSQL::Version

2022-07-06 Thread Jehan-Guillaume de Rorthais
On Tue, 5 Jul 2022 09:59:42 -0400 Andrew Dunstan wrote: > On 2022-07-03 Su 16:12, Jehan-Guillaume de Rorthais wrote: > > On Sun, 3 Jul 2022 10:40:21 -0400 > > Andrew Dunstan wrote: > > > >> On 2022-06-29 We 05:09, Jehan-Guillaume de Rorthais wrote: > >&

Re: Fix proposal for comparaison bugs in PostgreSQL::Version

2022-07-03 Thread Jehan-Guillaume de Rorthais
On Sun, 3 Jul 2022 10:40:21 -0400 Andrew Dunstan wrote: > On 2022-06-29 We 05:09, Jehan-Guillaume de Rorthais wrote: > > On Tue, 28 Jun 2022 18:17:40 -0400 > > Andrew Dunstan wrote: > > > >> On 2022-06-28 Tu 16:53, Jehan-Guillaume de Rorthais wrote: > >&

Re: Fix proposal for comparaison bugs in PostgreSQL::Version

2022-06-29 Thread Jehan-Guillaume de Rorthais
On Tue, 28 Jun 2022 18:17:40 -0400 Andrew Dunstan wrote: > On 2022-06-28 Tu 16:53, Jehan-Guillaume de Rorthais wrote: > > ... > > A better fix would be to store the version internally as version_num that > > are trivial to compute and compare. Please, find in attachment an

Fix proposal for comparaison bugs in PostgreSQL::Version

2022-06-28 Thread Jehan-Guillaume de Rorthais
fix would be to store the version internally as version_num that are trivial to compute and compare. Please, find in attachment an implementation of this. The patch is a bit bigger because it improved the devel version to support rc/beta/alpha comparison like 14rc2 > 14rc1. Moreover, it adds a

Self FK oddity when attaching a partition

2022-06-03 Thread Jehan-Guillaume de Rorthais
Hi all, While studying the issue discussed in thread "Detaching a partition with a FK on itself is not possible"[1], I stumbled across an oddity while attaching a partition having the same multiple self-FK than the parent table. Only one of the self-FK is found as a duplicate. Find in attachment

Re: Detaching a partition with a FK on itself is not possible

2022-03-21 Thread Jehan-Guillaume de Rorthais
Hi, On Thu, 17 Mar 2022 17:58:04 + Arne Roland wrote: > I don't think this a bug, but a feature request. I therefore think hackers > would be more appropriate. +1 I changed the list destination > I don't see how an additional syntax to modify the constraint should help. Me neiher. > If

Re: [Proposal] Add accumulated statistics for wait event

2021-06-16 Thread Jehan-Guillaume de Rorthais
Hi Andres, On Mon, 14 Jun 2021 15:01:14 -0700 Andres Freund wrote: > On 2021-06-14 23:20:47 +0200, Jehan-Guillaume de Rorthais wrote: > > > On 2021-06-14 16:10:32 +0200, Jehan-Guillaume de Rorthais wrote: > > > > In the patch in attachment, I tried to fix

Re: [Proposal] Add accumulated statistics for wait event

2021-06-14 Thread Jehan-Guillaume de Rorthais
Hi, On Mon, 14 Jun 2021 11:27:21 -0700 Andres Freund wrote: > On 2021-06-14 16:10:32 +0200, Jehan-Guillaume de Rorthais wrote: > > In the patch in attachment, I tried to fix this by using kind of an internal > > hook for pgstat_report_wait_start and pgstat_report_wait_end.

Re: [Proposal] Add accumulated statistics for wait event

2021-06-14 Thread Jehan-Guillaume de Rorthais
work/review the original approach is required. See bellow for comments and some more explanations about what I think might be improvements over the previous patch. On Fri, 11 Jun 2021 12:18:07 -0700 Andres Freund wrote: > On 2021-06-05 00:53:44 +0200, Jehan-Guillaume de Rorthais wrote: >

Re: [Proposal] Add accumulated statistics for wait event

2021-06-04 Thread Jehan-Guillaume de Rorthais
were probably saturated on the frontend side, waiting for the app and slowing down the whole transfert. Devels fixed the problem by moving away two fields transformations (unaccent) from their loop fetching the rows. >From 88c2779679c5c9625ca5348eec0543daab5ccab4 Mon Sep 17 00:00:00 2

Re: when the startup process doesn't

2021-04-21 Thread Jehan-Guillaume de Rorthais
On Wed, 21 Apr 2021 12:36:05 -0700 Andres Freund wrote: > [...] > > I don't think that concern equally applies for what I am proposing > here. For one, we already have minRecoveryPoint in ControlData, and we > already use it for the purpose of determining where we need to recover > to, albeit

Re: when the startup process doesn't

2021-04-20 Thread Jehan-Guillaume de Rorthais
On Tue, 20 Apr 2021 15:04:28 +0200 Magnus Hagander wrote: [...] > Yeah, I think we should definitely limit this to local access, one way > or another. Realistically using pg_hba is going to require catalog > access, isn't it? And we can't just go ignore those rows in pg_hba > that for example refe

Re: multi-install PostgresNode fails with older postgres versions

2021-04-19 Thread Jehan-Guillaume de Rorthais
On Mon, 19 Apr 2021 10:35:39 -0700 Mark Dilger wrote: > > On Apr 19, 2021, at 10:25 AM, Jehan-Guillaume de Rorthais > > wrote: > > > > On Mon, 19 Apr 2021 12:37:08 -0400 > > Andrew Dunstan wrote: > > > >> > >> On 4/19/21 10:43 AM, Mark

Re: multi-install PostgresNode fails with older postgres versions

2021-04-19 Thread Jehan-Guillaume de Rorthais
On Mon, 19 Apr 2021 12:37:08 -0400 Andrew Dunstan wrote: > > On 4/19/21 10:43 AM, Mark Dilger wrote: > > > >> On Apr 19, 2021, at 5:11 AM, Andrew Dunstan wrote: > >> > >> I think therefore I'm inclined for now to do nothing for old version > >> compatibility. > > I agree with waiting until the

Re: multi-install PostgresNode fails with older postgres versions

2021-04-19 Thread Jehan-Guillaume de Rorthais
On Mon, 19 Apr 2021 07:43:58 -0700 Mark Dilger wrote: > > On Apr 19, 2021, at 5:11 AM, Andrew Dunstan wrote: > > > > I think therefore I'm inclined for now to do nothing for old version > > compatibility. > > I agree with waiting until the v15 development cycle. Agree.

Re: Retry in pgbench

2021-04-16 Thread Jehan-Guillaume de Rorthais
On Fri, 16 Apr 2021 10:28:48 +0900 (JST) Tatsuo Ishii wrote: > > By the way, I've been playing with the idea of failing gracefully and retry > > indefinitely (or until given -T) on SQL error AND connection issue. > > > > It would be useful to test replicating clusters with a (switch|fail)over >

Re: Retry in pgbench

2021-04-13 Thread Jehan-Guillaume de Rorthais
Hi, On Tue, 13 Apr 2021 16:12:59 +0900 (JST) Tatsuo Ishii wrote: > [...] > [...] > [...] > > Thanks for the pointer. It seems we need to resume the discussion. By the way, I've been playing with the idea of failing gracefully and retry indefinitely (or until given -T) on SQL error AND

Re: multi-install PostgresNode fails with older postgres versions

2021-04-12 Thread Jehan-Guillaume de Rorthais
On Mon, 12 Apr 2021 09:52:24 -0400 Andrew Dunstan wrote: > On 4/12/21 8:59 AM, Jehan-Guillaume de Rorthais wrote: > > Hi, > > > > On Wed, 7 Apr 2021 20:07:41 +0200 > > Jehan-Guillaume de Rorthais wrote: > > [...] > >>>> Let me k

Re: multi-install PostgresNode fails with older postgres versions

2021-04-12 Thread Jehan-Guillaume de Rorthais
Hi, On Wed, 7 Apr 2021 20:07:41 +0200 Jehan-Guillaume de Rorthais wrote: [...] > > > Let me know if it worth that I work on an official patch. > > > > Let's give it a try ... > > OK So, as promised, here is my take to port my previous work on PostgreS

Re: multi-install PostgresNode fails with older postgres versions

2021-04-07 Thread Jehan-Guillaume de Rorthais
On Wed, 7 Apr 2021 13:38:39 -0400 Andrew Dunstan wrote: > On 4/7/21 1:19 PM, Jehan-Guillaume de Rorthais wrote: > > On Wed, 7 Apr 2021 12:51:55 -0400 > > Alvaro Herrera wrote: > > > >> On 2021-Apr-07, Jehan-Guillaume de Rorthais wrote: > >> > >&

Re: multi-install PostgresNode fails with older postgres versions

2021-04-07 Thread Jehan-Guillaume de Rorthais
On Wed, 7 Apr 2021 10:50:26 -0700 Mark Dilger wrote: > > On Apr 7, 2021, at 10:36 AM, Alvaro Herrera wrote: > > > >> Yes, it would be much saner to make PostgresNode the factory class. Plus, > >> some more logic could be injected there to either auto-detect the version > >> (current behavior)

Re: multi-install PostgresNode fails with older postgres versions

2021-04-07 Thread Jehan-Guillaume de Rorthais
On Wed, 7 Apr 2021 13:36:31 -0400 Alvaro Herrera wrote: > On 2021-Apr-07, Jehan-Guillaume de Rorthais wrote: > > > Yes, it would be much saner to make PostgresNode the factory class. Plus, > > some more logic could be injected there to either auto-detect the version > >

Re: multi-install PostgresNode fails with older postgres versions

2021-04-07 Thread Jehan-Guillaume de Rorthais
On Wed, 7 Apr 2021 12:51:55 -0400 Alvaro Herrera wrote: > On 2021-Apr-07, Jehan-Guillaume de Rorthais wrote: > > > When I'm creating a new node, I'm using the "pgaTester" factory class. It > > relies on PATH to check the major version using pg_config

Re: why pg_walfile_name() cannot be executed during recovery?

2021-04-07 Thread Jehan-Guillaume de Rorthais
On Fri, 2 Apr 2021 08:22:09 -0400 Robert Haas wrote: > On Fri, Apr 2, 2021 at 4:23 AM SATYANARAYANA NARLAPURAM > wrote: > > Why pg_walfile_name() can't be executed under recovery? > > I believe the issue is that the backend executing the function might > not have an accurate idea about which

Re: multi-install PostgresNode fails with older postgres versions

2021-04-07 Thread Jehan-Guillaume de Rorthais
On Wed, 7 Apr 2021 09:08:31 -0700 Mark Dilger wrote: > > On Apr 7, 2021, at 7:37 AM, Jehan-Guillaume de Rorthais > > And here is a demo test file: > > https://github.com/ioguix/check_pgactivity/blob/tests/t/01-streaming_delta.t > > > > My limited set of tests are

Re: multi-install PostgresNode fails with older postgres versions

2021-04-07 Thread Jehan-Guillaume de Rorthais
On Wed, 7 Apr 2021 11:54:36 -0400 Andrew Dunstan wrote: > On 4/7/21 10:37 AM, Jehan-Guillaume de Rorthais wrote: > > Hi all, > > > > First, sorry to step in this discussion this late. I didn't noticed it > > before :( > > > > I did some work about th

Re: multi-install PostgresNode fails with older postgres versions

2021-04-07 Thread Jehan-Guillaume de Rorthais
Hi all, First, sorry to step in this discussion this late. I didn't noticed it before :( I did some work about these compatibility issues in late 2020 to use PostgresNode in the check_pgactivity TAP tests. See https://github.com/ioguix/check_pgactivity/tree/tests/t/lib PostgresNode.pm, TestLib.

Re: [PATCH] Disable bgworkers during servers start in pg_upgrade

2021-01-27 Thread Jehan-Guillaume de Rorthais
Oh, I forgot another point before sending my previous email. Maybe it might worth adding some final safety checks in pg_upgrade itself? Eg. checking controldata and mxid files coherency between old and new cluster would have catch the inconsistency here.

Re: [PATCH] Disable bgworkers during servers start in pg_upgrade

2021-01-27 Thread Jehan-Guillaume de Rorthais
Hi, On Wed, 27 Jan 2021 11:25:11 +0100 Denis Laxalde wrote: > Andres Freund a écrit : > > On 2021-01-21 16:23:58 +0100, Denis Laxalde wrote: > > > We found an issue in pg_upgrade on a cluster with a third-party > > > background worker. The upgrade goes fine, but the new cluster is then in > >

Re: vacuum -vs reltuples on insert only index

2020-11-09 Thread Jehan-Guillaume de Rorthais
On Wed, 4 Nov 2020 18:44:03 -0800 Peter Geoghegan wrote: > On Mon, Nov 2, 2020 at 10:03 AM Peter Geoghegan wrote: > > Actually, it seems better to always count num_index_tuples the old way > > during cleanup-only index VACUUMs, despite the inaccuracy that that > > creates with posting list tuple

Re: The ultimate extension hook.

2020-10-23 Thread Jehan-Guillaume de Rorthais
On Thu, 24 Sep 2020 17:08:44 +1200 David Rowley wrote: [...] > I wondered if there was much in the way of use-cases like a traffic > filter, or statement replication. I wasn't sure if it was a solution > looking for a problem or not, but it seems like it could be productive > to talk about possibi

vacuum -vs reltuples on insert only index

2020-10-23 Thread Jehan-Guillaume de Rorthais
Hello, I've found a behavior change with pg_class.reltuples on btree index. With only insert activity on a table, when an index is processed, its related reltuples is set to 0. Here is a demo script: -- force index cleanup set vacuum_cleanup_index_scale_factor to 0; drop table if exists t;

Re: [patch] demote

2020-09-01 Thread Jehan-Guillaume de Rorthais
On Tue, 18 Aug 2020 17:41:31 +0200 Jehan-Guillaume de Rorthais wrote: > Hi, > > Please find in attachment v5 of the patch set rebased on master after various > conflicts. > > Regards, > > On Wed, 5 Aug 2020 00:04:53 +0200 > Jehan-Guillaume de Rorthais wrote: >

Re: [patch] demote

2020-08-18 Thread Jehan-Guillaume de Rorthais
Hi, Please find in attachment v5 of the patch set rebased on master after various conflicts. Regards, On Wed, 5 Aug 2020 00:04:53 +0200 Jehan-Guillaume de Rorthais wrote: > Demote now keeps backends with no active xid alive. Smart mode keeps all > backends: it waits for them to finish

Re: [patch] demote

2020-08-04 Thread Jehan-Guillaume de Rorthais
to conclude this patch set should keep growing to see were it goes, I wonder if/when I should add it to the commitfest. Advice? Opinion? Regards, >From da3c4575f8ea40c089483b9cfa209db4993148ff Mon Sep 17 00:00:00 2001 From: Jehan-Guillaume de Rorthais Date: Fri, 31 Jul 2020 10:58:40 +0200 Sub

BUG #15285: Query used index over field with ICU collation in some cases wrongly return 0 rows

2020-07-15 Thread Jehan-Guillaume de Rorthais
fix it on our side using eg a workaround (see patch), I suppose this small bug should be documented somewhere so people are not lost alone in the wild. Opinions? Regards, Begin forwarded message: Date: Sat, 13 Jun 2020 00:43:22 +0200 From: Jehan-Guillaume de Rorthais To: Thomas Munro , Peter Geo

Re: [patch] demote

2020-07-14 Thread Jehan-Guillaume de Rorthais
On Tue, 14 Jul 2020 12:49:51 -0700 Andres Freund wrote: > Hi, > > On 2020-07-14 17:26:37 +0530, Amul Sul wrote: > > On Mon, Jul 13, 2020 at 8:35 PM Jehan-Guillaume de Rorthais > > wrote: > > > > > > Hi, > > > > > > Another summary

Re: [patch] demote

2020-07-13 Thread Jehan-Guillaume de Rorthais
* Robert's concern about snapshot during hot standby * some more coding style cleanup/refactoring * anything else reported to me :) Thanks, On Fri, 3 Jul 2020 00:12:10 +0200 Jehan-Guillaume de Rorthais wrote: > Hi, > > Here is a small activity summary since last report. > &g

Re: [patch] demote

2020-07-02 Thread Jehan-Guillaume de Rorthais
Hi, Here is a small activity summary since last report. On Thu, 25 Jun 2020 19:27:54 +0200 Jehan-Guillaume de Rorthais wrote: [...] > I hadn't time to investigate Robert's concern about shared memory for snapshot > during recovery. I hadn't time to dig very far, but I

Re: Remove Deprecated Exclusive Backup Mode

2020-07-02 Thread Jehan-Guillaume de Rorthais
On Thu, 2 Jul 2020 12:32:14 +0200 Magnus Hagander wrote: [...] > > non-exclusive backup...this is not that easy anymore. And > > pg_is_in_backup() is quite misleading if the admin found it without reading > > doc. Maybe an admin > > Yeah, as it is now it should really be called pg_is_in_exclusiv

Re: Remove Deprecated Exclusive Backup Mode

2020-07-01 Thread Jehan-Guillaume de Rorthais
On Wed, 1 Jul 2020 15:58:57 -0400 Robert Haas wrote: > On Wed, Jul 1, 2020 at 3:50 PM Magnus Hagander wrote: > > As far as I've seen, the one thing that people have problems with in the > > exclusive mode backups are precisely the fact that they have to keep a > > persistent conneciton open, and

  1   2   >