Re: Sorting regression of text function result since commit 586b98fdf1aae

2023-12-12 Thread Jehan-Guillaume de Rorthais
On Mon, 11 Dec 2023 15:43:12 -0500 Tom Lane wrote: > Jehan-Guillaume de Rorthais writes: > > It looks like since 586b98fdf1aae, the result type collation of > > "convert_from" is forced to "C", like the patch does for type "name", > >

Sorting regression of text function result since commit 586b98fdf1aae

2023-12-11 Thread Jehan-Guillaume de Rorthais
Hi, A customer found what looks like a sort regression while testing his code from v11 on a higher version. We hunt this regression down to commit 586b98fdf1aae, introduced in v12. Consider the following test case: createdb -l fr_FR.utf8 -T template0 reg psql reg <<<" BEGIN; CREATE

Re: Query execution in Perl TAP tests needs work

2023-11-06 Thread Jehan-Guillaume de Rorthais
On Wed, 18 Oct 2023 18:25:01 +0200 Alvaro Herrera wrote: > On 2023-Oct-18, Robert Haas wrote: > > > Without FFI::Platypus, we have to write Perl code that can speak the > > wire protocol directly. Basically, we're writing our own PostgreSQL > > driver for Perl, though we might need only a

Re: Detoasting optionally to make Explain-Analyze less misleading

2023-09-13 Thread Jehan-Guillaume de Rorthais
Hi Stepan & all, On Tue, 12 Sep 2023 17:16:00 +0200 stepan rutz wrote: ... > Attached a new patch. Hoping for feedback, Nice addition to EXPLAIN! On the feature front, what about adding the actual detoasting/serializing time in the explain output? That could be: => explain

Re: EBCDIC sorting as a use case for ICU rules

2023-08-24 Thread Jehan-Guillaume de Rorthais
Hi, Sorry to chime in so lately, I was waiting for some customer feedback. On Wed, 21 Jun 2023 15:28:38 +0200 "Daniel Verite" wrote: > At a conference this week I was asked if ICU could be able to > sort like EBCDIC [2]. > It turns out it has been already asked on > -general a few years

Re: [BUG] Fix DETACH with FK pointing to a partitioned table fails

2023-08-10 Thread Jehan-Guillaume de Rorthais
On Thu, 3 Aug 2023 11:02:43 +0200 Alvaro Herrera wrote: > On 2023-Aug-03, tender wang wrote: > > > I think old "sub-FK" should not be dropped, that will be violates foreign > > key constraint. > > Yeah, I've been playing more with the patch and it is definitely not > doing the right things.

Re: Issue attaching a table to a partitioned table with an auto-referenced foreign key

2023-07-07 Thread Jehan-Guillaume de Rorthais
So I gave a look at this one... And it's a tricky one. The current policy about DETACHing a partition is to keep/adjust all FK referencing it or referenced by it. However, in this exact self-referencing usecase, we can have rows referencing rows from the same partition OR another one. It seems

[BUG] Fix DETACH with FK pointing to a partitioned table fails

2023-07-05 Thread Jehan-Guillaume de Rorthais
t. At the very least, it help understanding the problem and sketch a possible solution. Regards, >From 5fc7997b9f9a17ee5a31f059c18e6c01fd716c04 Mon Sep 17 00:00:00 2001 From: Jehan-Guillaume de Rorthais Date: Wed, 5 Jul 2023 19:19:40 +0200 Subject: [PATCH v1] Remove useless parted-FK constraint

Re: Memory leak from ExecutorState context?

2023-05-22 Thread Jehan-Guillaume de Rorthais
On Fri, 19 May 2023 17:23:56 +0200 Tomas Vondra wrote: > On 5/19/23 00:27, Melanie Plageman wrote: > > v10 LGTM. > > Thanks! > > I've pushed 0002 and 0003, after some general bikeshedding and minor > rewording (a bit audacious, admittedly). Thank you Melanie et Tomas for your help, review et

Re: Memory leak from ExecutorState context?

2023-05-17 Thread Jehan-Guillaume de Rorthais
On Wed, 17 May 2023 13:46:35 -0400 Melanie Plageman wrote: > On Wed, May 17, 2023 at 07:10:08PM +0200, Jehan-Guillaume de Rorthais wrote: > > On Tue, 16 May 2023 16:00:52 -0400 > > Melanie Plageman wrote: > > > ... > > > There are some existing indentation i

Re: Memory leak from ExecutorState context?

2023-05-17 Thread Jehan-Guillaume de Rorthais
On Tue, 16 May 2023 16:00:52 -0400 Melanie Plageman wrote: > On Tue, May 16, 2023 at 04:00:51PM +0200, Jehan-Guillaume de Rorthais wrote: > > > From e5ecd466172b7bae2f1be294c1a5e70ce2b43ed8 Mon Sep 17 00:00:00 2001 > > From: Melanie Plageman > > Date: Thu, 30

Re: Memory leak from ExecutorState context?

2023-05-16 Thread Jehan-Guillaume de Rorthais
Hi, On Tue, 16 May 2023 12:01:51 +0200 Tomas Vondra wrote: > On 5/16/23 00:15, Jehan-Guillaume de Rorthais wrote: > > On Sat, 13 May 2023 23:47:53 +0200 > > Tomas Vondra wrote: > ... > >> I'm not really sure about calling this "hybrid hash-join". What doe

Re: Memory leak from ExecutorState context?

2023-05-15 Thread Jehan-Guillaume de Rorthais
participate in parallel query execution in several ways. A -- 2.40.1 >From 52e989d5eb6405e86e0d460dffbfaca292bc4274 Mon Sep 17 00:00:00 2001 From: Jehan-Guillaume de Rorthais Date: Mon, 27 Mar 2023 15:54:39 +0200 Subject: [PATCH 2/3] Allocate hash batches related BufFile in a dedicated context --- src/backend/

Re: Memory leak from ExecutorState context?

2023-05-15 Thread Jehan-Guillaume de Rorthais
On Sun, 14 May 2023 00:10:00 +0200 Tomas Vondra wrote: > On 5/12/23 23:36, Melanie Plageman wrote: > > Thanks for continuing to work on this. > > > > Are you planning to modify what is displayed for memory usage in > > EXPLAIN ANALYZE? Yes, I already start to work on this. Tracking spilling

Re: Unlinking Parallel Hash Join inner batch files sooner

2023-05-10 Thread Jehan-Guillaume de Rorthais
Hi, Thanks for working on this! On Wed, 10 May 2023 15:11:20 +1200 Thomas Munro wrote: > One complaint about PHJ is that it can, in rare cases, use a > surprising amount of temporary disk space where non-parallel HJ would > not. When it decides that it needs to double the number of batches to

Re: Memory leak from ExecutorState context?

2023-05-10 Thread Jehan-Guillaume de Rorthais
;--indicating it > is for the memory required for spilling to permanent storage while > executing hash joins. "Spilling" seems fair and a large enough net to grab everything around temp files and accessing them. > I discuss this more in my code review below. > > > From c

Re: Memory leak from ExecutorState context?

2023-05-04 Thread Jehan-Guillaume de Rorthais
Hi, On Fri, 21 Apr 2023 16:44:48 -0400 Melanie Plageman wrote: > On Fri, Apr 7, 2023 at 8:01 PM Jehan-Guillaume de Rorthais > wrote: > > > > On Fri, 31 Mar 2023 14:06:11 +0200 > > Jehan-Guillaume de Rorthais wrote: > > > > > > [...] > > >

Re: Commitfest 2023-03 starting tomorrow!

2023-04-21 Thread Jehan-Guillaume de Rorthais
Hi, After catching up with this thread, where pending bugs are listed and discussed, I wonder if the current patches trying to lower the HashJoin memory explosion[1] could be added to the "Older bugs affecting stable branches" list of https://wiki.postgresql.org/wiki/PostgreSQL_16_Open_Items as

[BUG] FK broken after DETACHing referencing part

2023-04-20 Thread Jehan-Guillaume de Rorthais
Hi, Considering two partitionned tables with a FK between them: DROP TABLE IF EXISTS p, c, c_1 CASCADE; -- -- Parent table + partition + data CREATE TABLE p ( id bigint PRIMARY KEY ) PARTITION BY list (id); CREATE TABLE p_1 PARTITION OF p FOR

Re: OOM in hash join

2023-04-14 Thread Jehan-Guillaume de Rorthais
On Fri, 14 Apr 2023 13:21:05 +0200 Matthias van de Meent wrote: > On Fri, 14 Apr 2023 at 12:59, Konstantin Knizhnik wrote: > > > > Hi hackers, > > > > Too small value of work_mem cause memory overflow in parallel hash join > > because of too much number batches. > > There is the plan: > >

Re: Memory leak from ExecutorState context?

2023-04-11 Thread Jehan-Guillaume de Rorthais
On Sat, 8 Apr 2023 02:01:19 +0200 Jehan-Guillaume de Rorthais wrote: > On Fri, 31 Mar 2023 14:06:11 +0200 > Jehan-Guillaume de Rorthais wrote: > > [...] > > After rebasing Tomas' memory balancing patch, I did some memory measures > to answer some of my questions. Pleas

Re: Memory leak from ExecutorState context?

2023-03-31 Thread Jehan-Guillaume de Rorthais
n 3256 blocks; 970128 free (38180 chunks); 27635552 used HashBatchContext: 960544 total in 23 blocks; 7928 free (0 chunks); 952616 used Regards, >From 6814994fa0576a8ba6458412ac5f944135fc3813 Mon Sep 17 00:00:00 2001 From: Jehan-Guillaume de Rorthais Date: Mon,

Re: Memory leak from ExecutorState context?

2023-03-28 Thread Jehan-Guillaume de Rorthais
Hi, Sorry for the late answer, I was reviewing the first patch and it took me some time to study and dig around. On Thu, 23 Mar 2023 08:07:04 -0400 Melanie Plageman wrote: > On Fri, Mar 10, 2023 at 1:51 PM Jehan-Guillaume de Rorthais > wrote: > > > So I guess the best thing

Re: Memory leak from ExecutorState context?

2023-03-28 Thread Jehan-Guillaume de Rorthais
On Tue, 28 Mar 2023 00:43:34 +0200 Tomas Vondra wrote: > On 3/27/23 23:13, Jehan-Guillaume de Rorthais wrote: > > Please, find in attachment a patch to allocate bufFiles in a dedicated > > context. I picked up your patch, backpatch'd it, went through it and did > > some m

Re: Memory leak from ExecutorState context?

2023-03-27 Thread Jehan-Guillaume de Rorthais
Hi, On Mon, 20 Mar 2023 15:12:34 +0100 Jehan-Guillaume de Rorthais wrote: > On Mon, 20 Mar 2023 09:32:17 +0100 > Tomas Vondra wrote: > > > >> * Patch 1 could be rebased/applied/backpatched > > > > > > Would it help if I rebase Patch 1 ("mo

Re: Memory leak from ExecutorState context?

2023-03-20 Thread Jehan-Guillaume de Rorthais
On Mon, 20 Mar 2023 09:32:17 +0100 Tomas Vondra wrote: > >> * Patch 1 could be rebased/applied/backpatched > > > > Would it help if I rebase Patch 1 ("move BufFile stuff into separate > > context")? > > Yeah, I think this is something we'd want to do. It doesn't change the > behavior, but

Re: Memory leak from ExecutorState context?

2023-03-17 Thread Jehan-Guillaume de Rorthais
Hi there, On Fri, 10 Mar 2023 19:51:14 +0100 Jehan-Guillaume de Rorthais wrote: > > So I guess the best thing would be to go through these threads, see what > > the status is, restart the discussion and propose what to do. If you do > > that, I'm happy to rebase the patches,

Re: Memory leak from ExecutorState context?

2023-03-10 Thread Jehan-Guillaume de Rorthais
Hi, > So I guess the best thing would be to go through these threads, see what > the status is, restart the discussion and propose what to do. If you do > that, I'm happy to rebase the patches, and maybe see if I could improve > them in some way. OK! It took me some time, but I did it. I'll try

Re: Memory leak from ExecutorState context?

2023-03-02 Thread Jehan-Guillaume de Rorthais
On Thu, 2 Mar 2023 19:53:14 +0100 Tomas Vondra wrote: > On 3/2/23 19:15, Jehan-Guillaume de Rorthais wrote: ... > > There was some thoughts about how to make a better usage of the memory. As > > memory is exploding way beyond work_mem, at least, avoid to waste it with >

Re: Memory leak from ExecutorState context?

2023-03-02 Thread Jehan-Guillaume de Rorthais
On Thu, 2 Mar 2023 19:15:30 +0100 Jehan-Guillaume de Rorthais wrote: [...] > For what it worth, these two patches seems really interesting to me. Do you > need any help to revive it? To avoid confusion, the two patches I meant were: * 0001-move-BufFile-stuff-into-separate-context.patch

Re: Memory leak from ExecutorState context?

2023-03-02 Thread Jehan-Guillaume de Rorthais
Hi! On Thu, 2 Mar 2023 13:44:52 +0100 Tomas Vondra wrote: > Well, yeah and no. > > In principle we could/should have allocated the BufFiles in a different > context (possibly hashCxt). But in practice it probably won't make any > difference, because the query will probably run all the hashjoins

Re: Memory leak from ExecutorState context?

2023-03-02 Thread Jehan-Guillaume de Rorthais
On Thu, 2 Mar 2023 01:30:27 +0100 Tomas Vondra wrote: > On 3/2/23 00:18, Jehan-Guillaume de Rorthais wrote: > >>> ExecHashIncreaseNumBatches was really chatty, having hundreds of thousands > >>> of calls, always short-cut'ed to 1048576, I guess because of the > &g

Re: Memory leak from ExecutorState context?

2023-03-01 Thread Jehan-Guillaume de Rorthais
Hi, On Wed, 1 Mar 2023 20:29:11 +0100 Tomas Vondra wrote: > On 3/1/23 18:48, Jehan-Guillaume de Rorthais wrote: > > On Tue, 28 Feb 2023 20:51:02 +0100 > > Tomas Vondra wrote: > >> On 2/28/23 19:06, Jehan-Guillaume de Rorthais wrote: > >>> * HashBatch

Re: Memory leak from ExecutorState context?

2023-03-01 Thread Jehan-Guillaume de Rorthais
On Wed, 1 Mar 2023 20:34:08 +0100 Tomas Vondra wrote: > On 3/1/23 19:09, Jehan-Guillaume de Rorthais wrote: > > On Wed, 1 Mar 2023 18:48:40 +0100 > > Jehan-Guillaume de Rorthais wrote: > > ... > >> You'll find some intermediate stats I already collected in atta

Re: Memory leak from ExecutorState context?

2023-03-01 Thread Jehan-Guillaume de Rorthais
On Wed, 1 Mar 2023 18:48:40 +0100 Jehan-Guillaume de Rorthais wrote: ... > You'll find some intermediate stats I already collected in attachment: > > * break 1, 2 and 3 are from AllocSetAlloc, break 4 is from AllocSetFree. > * most of the non-free'd chunk are allocated since the ve

Re: Memory leak from ExecutorState context?

2023-03-01 Thread Jehan-Guillaume de Rorthais
Hi Tomas, On Tue, 28 Feb 2023 20:51:02 +0100 Tomas Vondra wrote: > On 2/28/23 19:06, Jehan-Guillaume de Rorthais wrote: > > * HashBatchContext goes up to 1441MB after 240s then stay flat until the end > > (400s as the last record) > > That's interesting. We're u

Re: Memory leak from ExecutorState context?

2023-03-01 Thread Jehan-Guillaume de Rorthais
Hi Justin, On Tue, 28 Feb 2023 12:25:08 -0600 Justin Pryzby wrote: > On Tue, Feb 28, 2023 at 07:06:43PM +0100, Jehan-Guillaume de Rorthais wrote: > > Hello all, > > > > A customer is facing out of memory query which looks similar to this > > situa

Re: Transparent column encryption

2022-11-24 Thread Jehan-Guillaume de Rorthais
On Wed, 23 Nov 2022 19:45:06 +0100 Peter Eisentraut wrote: > On 28.10.22 12:16, Jehan-Guillaume de Rorthais wrote: [...] > >* I wonder if encryption related fields in ParameterDescription and > > RowDescription could be optional somehow? The former might be quite >

Re: Fix proposal for comparaison bugs in PostgreSQL::Version

2022-11-04 Thread Jehan-Guillaume de Rorthais
On Thu, 3 Nov 2022 13:11:18 -0500 Justin Pryzby wrote: > On Tue, Jun 28, 2022 at 06:17:40PM -0400, Andrew Dunstan wrote: > > Nice catch, but this looks like massive overkill. I think we can very > > simply fix the test in just a few lines of code, instead of a 190 line > > fix and a 130 line TAP

Re: [BUG] parenting a PK constraint to a self-FK one (Was: Self FK oddity when attaching a partition)

2022-11-04 Thread Jehan-Guillaume de Rorthais
On Thu, 3 Nov 2022 20:44:16 +0100 Alvaro Herrera wrote: > On 2022-Oct-05, Alvaro Herrera wrote: > > > I've been giving the patches a look and it caused me to notice two > > additional bugs in the same area: > > > > - FKs in partitions are sometimes marked NOT VALID. This is because of > >

Re: psql: Add command to use extended query protocol

2022-11-02 Thread Jehan-Guillaume de Rorthais
On Wed, 02 Nov 2022 16:04:02 +0100 "Daniel Verite" wrote: > Jehan-Guillaume de Rorthais wrote: > > > As I wrote in my TCE review, would it be possible to use psql vars to set > > some named parameters for the prepared query? This would looks like: > >

Re: psql: Add command to use extended query protocol

2022-11-02 Thread Jehan-Guillaume de Rorthais
Hi, On Fri, 28 Oct 2022 08:52:51 +0200 Peter Eisentraut wrote: > This adds a new psql command \gp that works like \g (or semicolon) but > uses the extended query protocol. Parameters can also be passed, like > > SELECT $1, $2 \gp 'foo' 'bar' As I wrote in my TCE review, would it be

Re: Commitfest documentation

2022-10-31 Thread Jehan-Guillaume de Rorthais
Hi Aleksander, Thank you for your help! On Mon, 31 Oct 2022 16:51:23 +0300 Aleksander Alekseev wrote: [...] > > In the commitfest application, I was wondering today what was the exact > > meaning and difference between open/closed status (is it only for the > > current commitfest?) > >

Commitfest documentation

2022-10-31 Thread Jehan-Guillaume de Rorthais
Hi, In the commitfest application, I was wondering today what was the exact meaning and difference between open/closed status (is it only for the current commitfest?) and between «waiting for author» and «Returned with feedback». I couldn't find a clear definition searching the wiki, the mailing

Re: Transparent column encryption

2022-10-28 Thread Jehan-Guillaume de Rorthais
Hi, I did a review of the documentation and usability. # Applying patch The patch applied on top of f13b2088fa2 without trouble. Notice a small warning during compilation: colenccmds.c:134:27: warning: ‘encval’ may be used uninitialized A simple fix could be: +++

Re: [BUG] parenting a PK constraint to a self-FK one (Was: Self FK oddity when attaching a partition)

2022-10-03 Thread Jehan-Guillaume de Rorthais
On Fri, 30 Sep 2022 16:11:09 -0700 Zhihong Yu wrote: > On Fri, Sep 30, 2022 at 3:30 PM Jehan-Guillaume de Rorthais > wrote: ... > > +* Self-Foreign keys are ignored as the index was preliminary > created > > preliminary created -> primarily created

Re: [BUG] parenting a PK constraint to a self-FK one (Was: Self FK oddity when attaching a partition)

2022-09-30 Thread Jehan-Guillaume de Rorthais
elated constraints? Regards, On Wed, 24 Aug 2022 12:49:13 +0200 Alvaro Herrera wrote: > On 2022-Aug-24, Jehan-Guillaume de Rorthais wrote: > > > I was naively wondering about such a patch, but was worrying about potential > > side effects on ATExecAttachPartitionIdx(),

Re: [BUG] wrong FK constraint name when colliding name on ATTACH

2022-09-08 Thread Jehan-Guillaume de Rorthais
On Thu, 8 Sep 2022 13:25:15 +0200 Alvaro Herrera wrote: > On 2022-Sep-08, Jehan-Guillaume de Rorthais wrote: > > > Hi there, > > > > I believe this very small bug and its fix are really trivial and could be > > push out of the way quite quickly. It's just about a

Re: [BUG] wrong FK constraint name when colliding name on ATTACH

2022-09-08 Thread Jehan-Guillaume de Rorthais
:) Regards, On Thu, 1 Sep 2022 18:41:56 +0200 Jehan-Guillaume de Rorthais wrote: > While studying and hacking on the parenting constraint issue, I found an > incoherent piece of code leading to badly chosen fk name. If a constraint > name collision is detected, while choosing a

[BUG] wrong FK constraint name when colliding name on ATTACH

2022-09-01 Thread Jehan-Guillaume de Rorthais
d. The expected name is "child1_id_abc_no_part_fkey". Find in attachment a simple fix, moving the name assignation after the FK attributes are populated. Regards, >From f1eeacb1eb3face38d76325666aff57019ef84c9 Mon Sep 17 00:00:00 2001 From: Jehan-Guillaume de Rorthais Date: Thu

Re: [BUG] parenting a PK constraint to a self-FK one (Was: Self FK oddity when attaching a partition)

2022-08-24 Thread Jehan-Guillaume de Rorthais
On Tue, 23 Aug 2022 18:30:06 +0200 Alvaro Herrera wrote: > On 2022-Aug-23, Jehan-Guillaume de Rorthais wrote: > > Hi, > > [...] > > > However, it seems get_relation_idx_constraint_oid(), introduced in > > eb7ed3f3063, assume there could be only ONE constr

[BUG] parenting a PK constraint to a self-FK one (Was: Self FK oddity when attaching a partition)

2022-08-23 Thread Jehan-Guillaume de Rorthais
Hi all, I've been able to work on this issue and isolate where in the code the oddity is laying. During ATExecAttachPartition(), AttachPartitionEnsureIndexes() look for existing required index on the partition to attach. It creates missing index, or sets the parent's index when a matching one

Re: Fix proposal for comparaison bugs in PostgreSQL::Version

2022-07-06 Thread Jehan-Guillaume de Rorthais
On Tue, 5 Jul 2022 09:59:42 -0400 Andrew Dunstan wrote: > On 2022-07-03 Su 16:12, Jehan-Guillaume de Rorthais wrote: > > On Sun, 3 Jul 2022 10:40:21 -0400 > > Andrew Dunstan wrote: > > > >> On 2022-06-29 We 05:09, Jehan-Guillaume de Rorthais wrote: > >&

Re: Fix proposal for comparaison bugs in PostgreSQL::Version

2022-07-03 Thread Jehan-Guillaume de Rorthais
On Sun, 3 Jul 2022 10:40:21 -0400 Andrew Dunstan wrote: > On 2022-06-29 We 05:09, Jehan-Guillaume de Rorthais wrote: > > On Tue, 28 Jun 2022 18:17:40 -0400 > > Andrew Dunstan wrote: > > > >> On 2022-06-28 Tu 16:53, Jehan-Guillaume de Rorthais wrote: >

Re: Fix proposal for comparaison bugs in PostgreSQL::Version

2022-06-29 Thread Jehan-Guillaume de Rorthais
On Tue, 28 Jun 2022 18:17:40 -0400 Andrew Dunstan wrote: > On 2022-06-28 Tu 16:53, Jehan-Guillaume de Rorthais wrote: > > ... > > A better fix would be to store the version internally as version_num that > > are trivial to compute and compare. Please, find in attachment

Fix proposal for comparaison bugs in PostgreSQL::Version

2022-06-28 Thread Jehan-Guillaume de Rorthais
to compute and compare. Please, find in attachment an implementation of this. The patch is a bit bigger because it improved the devel version to support rc/beta/alpha comparison like 14rc2 > 14rc1. Moreover, it adds a bunch of TAP tests to check various use cases. Regards, >From d6b28440ad8

Self FK oddity when attaching a partition

2022-06-03 Thread Jehan-Guillaume de Rorthais
Hi all, While studying the issue discussed in thread "Detaching a partition with a FK on itself is not possible"[1], I stumbled across an oddity while attaching a partition having the same multiple self-FK than the parent table. Only one of the self-FK is found as a duplicate. Find in attachment

Re: Detaching a partition with a FK on itself is not possible

2022-03-21 Thread Jehan-Guillaume de Rorthais
Hi, On Thu, 17 Mar 2022 17:58:04 + Arne Roland wrote: > I don't think this a bug, but a feature request. I therefore think hackers > would be more appropriate. +1 I changed the list destination > I don't see how an additional syntax to modify the constraint should help. Me neiher. > If

Re: [Proposal] Add accumulated statistics for wait event

2021-06-16 Thread Jehan-Guillaume de Rorthais
Hi Andres, On Mon, 14 Jun 2021 15:01:14 -0700 Andres Freund wrote: > On 2021-06-14 23:20:47 +0200, Jehan-Guillaume de Rorthais wrote: > > > On 2021-06-14 16:10:32 +0200, Jehan-Guillaume de Rorthais wrote: > > > > In the patch in attachment, I tried t

Re: [Proposal] Add accumulated statistics for wait event

2021-06-14 Thread Jehan-Guillaume de Rorthais
Hi, On Mon, 14 Jun 2021 11:27:21 -0700 Andres Freund wrote: > On 2021-06-14 16:10:32 +0200, Jehan-Guillaume de Rorthais wrote: > > In the patch in attachment, I tried to fix this by using kind of an internal > > hook for pgstat_report_wait_start and pgstat_report_wait_e

Re: [Proposal] Add accumulated statistics for wait event

2021-06-14 Thread Jehan-Guillaume de Rorthais
to work/review the original approach is required. See bellow for comments and some more explanations about what I think might be improvements over the previous patch. On Fri, 11 Jun 2021 12:18:07 -0700 Andres Freund wrote: > On 2021-06-05 00:53:44 +0200, Jehan-Guillaume de Rorthais wrote: >

Re: [Proposal] Add accumulated statistics for wait event

2021-06-04 Thread Jehan-Guillaume de Rorthais
waiting for the app and slowing down the whole transfert. Devels fixed the problem by moving away two fields transformations (unaccent) from their loop fetching the rows. >From 88c2779679c5c9625ca5348eec0543daab5ccab4 Mon Sep 17 00:00:00 2001 From: Jehan-Guillaume de Rorthais Date: Tue,

Re: when the startup process doesn't

2021-04-21 Thread Jehan-Guillaume de Rorthais
On Wed, 21 Apr 2021 12:36:05 -0700 Andres Freund wrote: > [...] > > I don't think that concern equally applies for what I am proposing > here. For one, we already have minRecoveryPoint in ControlData, and we > already use it for the purpose of determining where we need to recover > to,

Re: when the startup process doesn't

2021-04-20 Thread Jehan-Guillaume de Rorthais
On Tue, 20 Apr 2021 15:04:28 +0200 Magnus Hagander wrote: [...] > Yeah, I think we should definitely limit this to local access, one way > or another. Realistically using pg_hba is going to require catalog > access, isn't it? And we can't just go ignore those rows in pg_hba > that for example

Re: multi-install PostgresNode fails with older postgres versions

2021-04-19 Thread Jehan-Guillaume de Rorthais
On Mon, 19 Apr 2021 10:35:39 -0700 Mark Dilger wrote: > > On Apr 19, 2021, at 10:25 AM, Jehan-Guillaume de Rorthais > > wrote: > > > > On Mon, 19 Apr 2021 12:37:08 -0400 > > Andrew Dunstan wrote: > > > >> > >> On 4/19/21 10:43 AM, Mark

Re: multi-install PostgresNode fails with older postgres versions

2021-04-19 Thread Jehan-Guillaume de Rorthais
On Mon, 19 Apr 2021 12:37:08 -0400 Andrew Dunstan wrote: > > On 4/19/21 10:43 AM, Mark Dilger wrote: > > > >> On Apr 19, 2021, at 5:11 AM, Andrew Dunstan wrote: > >> > >> I think therefore I'm inclined for now to do nothing for old version > >> compatibility. > > I agree with waiting until the

Re: multi-install PostgresNode fails with older postgres versions

2021-04-19 Thread Jehan-Guillaume de Rorthais
On Mon, 19 Apr 2021 07:43:58 -0700 Mark Dilger wrote: > > On Apr 19, 2021, at 5:11 AM, Andrew Dunstan wrote: > > > > I think therefore I'm inclined for now to do nothing for old version > > compatibility. > > I agree with waiting until the v15 development cycle. Agree.

Re: Retry in pgbench

2021-04-16 Thread Jehan-Guillaume de Rorthais
On Fri, 16 Apr 2021 10:28:48 +0900 (JST) Tatsuo Ishii wrote: > > By the way, I've been playing with the idea of failing gracefully and retry > > indefinitely (or until given -T) on SQL error AND connection issue. > > > > It would be useful to test replicating clusters with a (switch|fail)over >

Re: Retry in pgbench

2021-04-13 Thread Jehan-Guillaume de Rorthais
Hi, On Tue, 13 Apr 2021 16:12:59 +0900 (JST) Tatsuo Ishii wrote: > [...] > [...] > [...] > > Thanks for the pointer. It seems we need to resume the discussion. By the way, I've been playing with the idea of failing gracefully and retry indefinitely (or until given -T) on SQL error

Re: multi-install PostgresNode fails with older postgres versions

2021-04-12 Thread Jehan-Guillaume de Rorthais
On Mon, 12 Apr 2021 09:52:24 -0400 Andrew Dunstan wrote: > On 4/12/21 8:59 AM, Jehan-Guillaume de Rorthais wrote: > > Hi, > > > > On Wed, 7 Apr 2021 20:07:41 +0200 > > Jehan-Guillaume de Rorthais wrote: > > [...] > >>>> Let me k

Re: multi-install PostgresNode fails with older postgres versions

2021-04-12 Thread Jehan-Guillaume de Rorthais
Hi, On Wed, 7 Apr 2021 20:07:41 +0200 Jehan-Guillaume de Rorthais wrote: [...] > > > Let me know if it worth that I work on an official patch. > > > > Let's give it a try ... > > OK So, as promised, here is my take to port my previous work on PostgreSQL so

Re: multi-install PostgresNode fails with older postgres versions

2021-04-07 Thread Jehan-Guillaume de Rorthais
On Wed, 7 Apr 2021 13:38:39 -0400 Andrew Dunstan wrote: > On 4/7/21 1:19 PM, Jehan-Guillaume de Rorthais wrote: > > On Wed, 7 Apr 2021 12:51:55 -0400 > > Alvaro Herrera wrote: > > > >> On 2021-Apr-07, Jehan-Guillaume de Rorthais wrote: > >> > &

Re: multi-install PostgresNode fails with older postgres versions

2021-04-07 Thread Jehan-Guillaume de Rorthais
On Wed, 7 Apr 2021 10:50:26 -0700 Mark Dilger wrote: > > On Apr 7, 2021, at 10:36 AM, Alvaro Herrera wrote: > > > >> Yes, it would be much saner to make PostgresNode the factory class. Plus, > >> some more logic could be injected there to either auto-detect the version > >> (current

Re: multi-install PostgresNode fails with older postgres versions

2021-04-07 Thread Jehan-Guillaume de Rorthais
On Wed, 7 Apr 2021 13:36:31 -0400 Alvaro Herrera wrote: > On 2021-Apr-07, Jehan-Guillaume de Rorthais wrote: > > > Yes, it would be much saner to make PostgresNode the factory class. Plus, > > some more logic could be injected there to either auto-detect the version > >

Re: multi-install PostgresNode fails with older postgres versions

2021-04-07 Thread Jehan-Guillaume de Rorthais
On Wed, 7 Apr 2021 12:51:55 -0400 Alvaro Herrera wrote: > On 2021-Apr-07, Jehan-Guillaume de Rorthais wrote: > > > When I'm creating a new node, I'm using the "pgaTester" factory class. It > > relies on PATH to check the major version using pg_config, then loa

Re: why pg_walfile_name() cannot be executed during recovery?

2021-04-07 Thread Jehan-Guillaume de Rorthais
On Fri, 2 Apr 2021 08:22:09 -0400 Robert Haas wrote: > On Fri, Apr 2, 2021 at 4:23 AM SATYANARAYANA NARLAPURAM > wrote: > > Why pg_walfile_name() can't be executed under recovery? > > I believe the issue is that the backend executing the function might > not have an accurate idea about which

Re: multi-install PostgresNode fails with older postgres versions

2021-04-07 Thread Jehan-Guillaume de Rorthais
On Wed, 7 Apr 2021 09:08:31 -0700 Mark Dilger wrote: > > On Apr 7, 2021, at 7:37 AM, Jehan-Guillaume de Rorthais > > And here is a demo test file: > > https://github.com/ioguix/check_pgactivity/blob/tests/t/01-streaming_delta.t > > > > My limited set of tests

Re: multi-install PostgresNode fails with older postgres versions

2021-04-07 Thread Jehan-Guillaume de Rorthais
On Wed, 7 Apr 2021 11:54:36 -0400 Andrew Dunstan wrote: > On 4/7/21 10:37 AM, Jehan-Guillaume de Rorthais wrote: > > Hi all, > > > > First, sorry to step in this discussion this late. I didn't noticed it > > before :( > > > > I did some work about these c

Re: multi-install PostgresNode fails with older postgres versions

2021-04-07 Thread Jehan-Guillaume de Rorthais
Hi all, First, sorry to step in this discussion this late. I didn't noticed it before :( I did some work about these compatibility issues in late 2020 to use PostgresNode in the check_pgactivity TAP tests. See https://github.com/ioguix/check_pgactivity/tree/tests/t/lib PostgresNode.pm,

Re: [PATCH] Disable bgworkers during servers start in pg_upgrade

2021-01-27 Thread Jehan-Guillaume de Rorthais
Oh, I forgot another point before sending my previous email. Maybe it might worth adding some final safety checks in pg_upgrade itself? Eg. checking controldata and mxid files coherency between old and new cluster would have catch the inconsistency here.

Re: [PATCH] Disable bgworkers during servers start in pg_upgrade

2021-01-27 Thread Jehan-Guillaume de Rorthais
Hi, On Wed, 27 Jan 2021 11:25:11 +0100 Denis Laxalde wrote: > Andres Freund a écrit : > > On 2021-01-21 16:23:58 +0100, Denis Laxalde wrote: > > > We found an issue in pg_upgrade on a cluster with a third-party > > > background worker. The upgrade goes fine, but the new cluster is then in >

Re: vacuum -vs reltuples on insert only index

2020-11-09 Thread Jehan-Guillaume de Rorthais
On Wed, 4 Nov 2020 18:44:03 -0800 Peter Geoghegan wrote: > On Mon, Nov 2, 2020 at 10:03 AM Peter Geoghegan wrote: > > Actually, it seems better to always count num_index_tuples the old way > > during cleanup-only index VACUUMs, despite the inaccuracy that that > > creates with posting list

Re: The ultimate extension hook.

2020-10-23 Thread Jehan-Guillaume de Rorthais
On Thu, 24 Sep 2020 17:08:44 +1200 David Rowley wrote: [...] > I wondered if there was much in the way of use-cases like a traffic > filter, or statement replication. I wasn't sure if it was a solution > looking for a problem or not, but it seems like it could be productive > to talk about

vacuum -vs reltuples on insert only index

2020-10-23 Thread Jehan-Guillaume de Rorthais
Hello, I've found a behavior change with pg_class.reltuples on btree index. With only insert activity on a table, when an index is processed, its related reltuples is set to 0. Here is a demo script: -- force index cleanup set vacuum_cleanup_index_scale_factor to 0; drop table if exists

Re: [patch] demote

2020-09-01 Thread Jehan-Guillaume de Rorthais
On Tue, 18 Aug 2020 17:41:31 +0200 Jehan-Guillaume de Rorthais wrote: > Hi, > > Please find in attachment v5 of the patch set rebased on master after various > conflicts. > > Regards, > > On Wed, 5 Aug 2020 00:04:53 +0200 > Jehan-Guillaume de Rorthais wrote: >

Re: [patch] demote

2020-08-18 Thread Jehan-Guillaume de Rorthais
Hi, Please find in attachment v5 of the patch set rebased on master after various conflicts. Regards, On Wed, 5 Aug 2020 00:04:53 +0200 Jehan-Guillaume de Rorthais wrote: > Demote now keeps backends with no active xid alive. Smart mode keeps all > backends: it waits for them to

Re: [patch] demote

2020-08-04 Thread Jehan-Guillaume de Rorthais
onclude this patch set should keep growing to see were it goes, I wonder if/when I should add it to the commitfest. Advice? Opinion? Regards, >From da3c4575f8ea40c089483b9cfa209db4993148ff Mon Sep 17 00:00:00 2001 From: Jehan-Guillaume de Rorthais Date: Fri, 31 Jul 2020 10:58:40 +0200 Subject:

BUG #15285: Query used index over field with ICU collation in some cases wrongly return 0 rows

2020-07-15 Thread Jehan-Guillaume de Rorthais
it on our side using eg a workaround (see patch), I suppose this small bug should be documented somewhere so people are not lost alone in the wild. Opinions? Regards, Begin forwarded message: Date: Sat, 13 Jun 2020 00:43:22 +0200 From: Jehan-Guillaume de Rorthais To: Thomas Munro , Peter Geoghegan Cc

Re: [patch] demote

2020-07-14 Thread Jehan-Guillaume de Rorthais
On Tue, 14 Jul 2020 12:49:51 -0700 Andres Freund wrote: > Hi, > > On 2020-07-14 17:26:37 +0530, Amul Sul wrote: > > On Mon, Jul 13, 2020 at 8:35 PM Jehan-Guillaume de Rorthais > > wrote: > > > > > > Hi, > > > > > > Another sum

Re: [patch] demote

2020-07-13 Thread Jehan-Guillaume de Rorthais
bert's concern about snapshot during hot standby * some more coding style cleanup/refactoring * anything else reported to me :) Thanks, On Fri, 3 Jul 2020 00:12:10 +0200 Jehan-Guillaume de Rorthais wrote: > Hi, > > Here is a small activity summary since last report. > > On Thu,

Re: [patch] demote

2020-07-02 Thread Jehan-Guillaume de Rorthais
Hi, Here is a small activity summary since last report. On Thu, 25 Jun 2020 19:27:54 +0200 Jehan-Guillaume de Rorthais wrote: [...] > I hadn't time to investigate Robert's concern about shared memory for snapshot > during recovery. I hadn't time to dig very far, but I suppose this

Re: Remove Deprecated Exclusive Backup Mode

2020-07-02 Thread Jehan-Guillaume de Rorthais
On Thu, 2 Jul 2020 12:32:14 +0200 Magnus Hagander wrote: [...] > > non-exclusive backup...this is not that easy anymore. And > > pg_is_in_backup() is quite misleading if the admin found it without reading > > doc. Maybe an admin > > Yeah, as it is now it should really be called

Re: Remove Deprecated Exclusive Backup Mode

2020-07-01 Thread Jehan-Guillaume de Rorthais
On Wed, 1 Jul 2020 15:58:57 -0400 Robert Haas wrote: > On Wed, Jul 1, 2020 at 3:50 PM Magnus Hagander wrote: > > As far as I've seen, the one thing that people have problems with in the > > exclusive mode backups are precisely the fact that they have to keep a > > persistent conneciton open,

Re: [patch] demote

2020-07-01 Thread Jehan-Guillaume de Rorthais
On Fri, 26 Jun 2020 16:14:38 +0900 (JST) Kyotaro Horiguchi wrote: > Hello. > > At Thu, 25 Jun 2020 19:27:54 +0200, Jehan-Guillaume de Rorthais > wrote in > > Here is a summary of my work during the last few days on this demote > > approach. > > > > Pleas

Re: [patch] demote

2020-06-25 Thread Jehan-Guillaume de Rorthais
: 1. add demote checkpoint 2. support demote: mostly postmaster, startup/xlog and checkpointer related code 3. cli using pg_ctl demote ...But I'm not sure it worth it. Regards, >From 03c41dd706648cd20df90a128db64eee6b6dad97 Mon Sep 17 00:00:00 2001 From: Jehan-Guillaume de Rorthais Date: Fri

Re: [patch] demote

2020-06-18 Thread Jehan-Guillaume de Rorthais
On Thu, 18 Jun 2020 11:22:47 -0400 Robert Haas wrote: > On Thu, Jun 18, 2020 at 8:41 AM Fujii Masao > wrote: > > ISTM that a clean switchover is possible without "ALTER SYSTEM READ ONLY". > > What about the following procedure? > > > > 1. Demote the primary to a standby. Then this demoted

Re: [patch] demote

2020-06-18 Thread Jehan-Guillaume de Rorthais
On Thu, 18 Jun 2020 11:15:02 -0400 Robert Haas wrote: > On Thu, Jun 18, 2020 at 6:02 AM Jehan-Guillaume de Rorthais > wrote: > > At some expense, Admin can already set the system as readonly from the > > application point of view, using: > > > > alter system set

Re: [Patch] ALTER SYSTEM READ ONLY

2020-06-18 Thread Jehan-Guillaume de Rorthais
On Thu, 18 Jun 2020 10:52:49 -0400 Robert Haas wrote: [...] > But what you want is that if it does happen to go down for some reason before > all the WAL is streamed, you can bring it back up and finish streaming the > WAL without generating any new WAL. Thanks to cascading replication, it

Re: [Patch] ALTER SYSTEM READ ONLY

2020-06-18 Thread Jehan-Guillaume de Rorthais
On Wed, 17 Jun 2020 12:07:22 -0400 Robert Haas wrote: [...] > > Commands that involve a whole > > bunch of subtle interlocking --- and, therefore, aren't going to work if > > anything has gone wrong already anywhere in the server --- seem like a > > particularly poor thing to be hanging your HA

Re: [patch] demote

2020-06-18 Thread Jehan-Guillaume de Rorthais
On Wed, 17 Jun 2020 11:14:47 -0700 Andres Freund wrote: > Hi, > > On 2020-06-17 17:44:51 +0200, Jehan-Guillaume de Rorthais wrote: > > As Amul sent a patch about "ALTER SYSTEM READ ONLY"[1], with similar futur > > objectives than mine, I decided to shar

  1   2   >