Re: Add contrib/pg_logicalsnapinspect

2024-09-25 Thread Bertrand Drouvot
not open file \"%s\": %m", > path))); > > Couldn't you just use the second branch for all errno's? Yeah, I think it comes from copying/pasting from SnapBuildRestore() too "quickly". v10 attached uses the second branch only. Regards, -- Bertrand

Re: Add contrib/pg_logicalsnapinspect

2024-09-25 Thread Bertrand Drouvot
So, the Int64GetDatum() output would still stay in the "xid" range. Keeping xid in the .sql makes it clear that we are dealing with transaction ID. Regards, -- Bertrand Drouvot PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com

Re: Add contrib/pg_logicalsnapinspect

2024-09-24 Thread Bertrand Drouvot
Hi, On Tue, Sep 24, 2024 at 09:15:31AM +0530, shveta malik wrote: > On Fri, Sep 20, 2024 at 12:22 PM Bertrand Drouvot > wrote: > > > > > > Please find attached v8, that: > > > > Thank You for the patch. In one of my tests, I noticed that I got > negative

Re: Add contrib/pg_logicalsnapinspect

2024-09-24 Thread Bertrand Drouvot
s to be done" -- the word "needs" is > incorrect here. > > How about: > "...if a change needs to be made to SnapBuildState." Thanks, used this one in v9. [1]: https://www.postgresql.org/message-id/CAJpy0uCppUNdod4F3NaPpMCtrySdw1S0T1d8CA-2c4CX%3DShMOQ%40mail.gm

Re: Add contrib/pg_logicalsnapinspect

2024-09-19 Thread Bertrand Drouvot
it that way instead of implementing the enum -> text mapping in snapbuild.h (as the mapping is only used by the module). [1]: https://www.postgresql.org/message-id/CAJpy0uDJ65QHUZfww7n6TBZAGp-SP74P5U3fUorV%2B%3DbaaRu6Dw%40mail.gmail.com [2]: https://www.postgresql.org/message-id/CAA4eK1JgW1o9wOTwgR

Re: Track the amount of time waiting due to cost_delay

2024-09-19 Thread Bertrand Drouvot
Hi, On Wed, Sep 18, 2024 at 04:04:53PM -0500, Nathan Bossart wrote: > On Thu, Sep 05, 2024 at 04:59:54AM +0000, Bertrand Drouvot wrote: > > Please find attached v6, a mandatory rebase due to catversion bump conflict. > > I'm removing the catversion bump from the patch as it ge

Re: define pg_structiszero(addr, s, r)

2024-09-18 Thread Bertrand Drouvot
Hi, On Wed, Sep 18, 2024 at 10:03:21AM +0200, Peter Eisentraut wrote: > On 18.09.24 06:16, Bertrand Drouvot wrote: > > +#define pg_structiszero(addr, s, r) > >

Re: Add contrib/pg_logicalsnapinspect

2024-09-18 Thread Bertrand Drouvot
d.xcnt; (*not xcnt_space*) > + * TransactionId * catchange.xcnt; > + * > + */ > +typedef struct SnapBuildOnDisk > > was directly beneath the comment: > -/* --- > - * Snapshot serialization support > - * --- >

Re: Add contrib/pg_logicalsnapinspect

2024-09-18 Thread Bertrand Drouvot
l/snapshots directory > (i.e. use server keyword, similar to how pg_ls_logicalsnapdir , > pg_ls_logicalmapdir explains it) Agree, done that way in v6. Regards, -- Bertrand Drouvot PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com >From a

Re: define pg_structiszero(addr, s, r)

2024-09-18 Thread Bertrand Drouvot
Hi, On Wed, Sep 18, 2024 at 03:07:15PM +0900, Michael Paquier wrote: > On Wed, Sep 18, 2024 at 04:16:12AM +0000, Bertrand Drouvot wrote: > > The macro is created in pgstat_internal.h as it looks like that "only" the > > statistics related code would benefit of it cur

define pg_structiszero(addr, s, r)

2024-09-17 Thread Bertrand Drouvot
o is created in pgstat_internal.h as it looks like that "only" the statistics related code would benefit of it currently (could be moved to other header file later on if needed). [1]: https://www.postgresql.org/message-id/20230105002733.ealhzubjaiqis6ua%40awork3.anarazel.de Looki

Re: per backend I/O statistics

2024-09-17 Thread Bertrand Drouvot
Hi, On Tue, Sep 17, 2024 at 04:47:51PM +0300, Nazir Bilal Yavuz wrote: > Hi, > > On Tue, 17 Sept 2024 at 16:07, Bertrand Drouvot > wrote: > > So I think it's better to keep both functions as they behave differently. > > > > Thoughts? > > Yes, that is

Re: per backend I/O statistics

2024-09-17 Thread Bertrand Drouvot
Hi, On Tue, Sep 17, 2024 at 02:52:01PM +0300, Nazir Bilal Yavuz wrote: > Hi, > > On Fri, 13 Sept 2024 at 19:09, Bertrand Drouvot > wrote: > > On Fri, Sep 13, 2024 at 04:45:08PM +0300, Nazir Bilal Yavuz wrote: > > > - The pgstat_reset_io_counter_

Re: Add contrib/pg_logicalsnapinspect

2024-09-17 Thread Bertrand Drouvot
that was my first remark in [1]) , as the module is mainly for debugging purpose, I was thinking that the one using it could refer to "snapbuild.h". Let's see what others think. [1]: https://www.postgresql.org/message-id/ZscuZ92uGh3wm4tW%40ip-10-97-1-34.eu-west-3.compu

Re: Add contrib/pg_logicalsnapinspect

2024-09-17 Thread Bertrand Drouvot
the volatility but it is not: postgres=# select * from pg_stat_get_xact_blocks_fetched(NULL); pg_stat_get_xact_blocks_fetched - (1 row) postgres=# select * from pg_get_multixact_members(NULL); xid | mode -+-- (0 rows) while both are volatile. I think both

Re: Add contrib/pg_logicalsnapinspect

2024-09-16 Thread Bertrand Drouvot
Hi, On Mon, Sep 16, 2024 at 04:02:51PM +0530, shveta malik wrote: > On Wed, Sep 11, 2024 at 4:21 PM Bertrand Drouvot > wrote: > > > > > > Yeah, good idea. Done that way in v3 attached. > > > > Thanks for the patch. +1 on the patch's idea. I have star

Re: per backend I/O statistics

2024-09-13 Thread Bertrand Drouvot
Hi, On Fri, Sep 13, 2024 at 04:45:08PM +0300, Nazir Bilal Yavuz wrote: > Hi, > > Thanks for working on this! > > Your patch applies and builds cleanly. Thanks for looking at it! > On Fri, 6 Sept 2024 at 18:03, Bertrand Drouvot > wrote: > > >

Re: per backend I/O statistics

2024-09-12 Thread Bertrand Drouvot
Hi, On Fri, Sep 06, 2024 at 03:03:17PM +, Bertrand Drouvot wrote: > The struct size is 1040 Bytes and that's much more reasonable than the size > needed for per backend I/O stats in v1 (about 16KB). One could think that's a high increase of shared memory usage with

Re: Switch PgStat_HashKey.objoid from Oid to uint64

2024-09-12 Thread Bertrand Drouvot
Hi, On Fri, Sep 13, 2024 at 07:34:21AM +0900, Michael Paquier wrote: > On Thu, Sep 12, 2024 at 01:37:52PM +0000, Bertrand Drouvot wrote: > > There is also some manipulation around the 2 new uint32 fields (objid_hi and > > objid_lo) in the xactdesc.c and pgstat_xact.c files that

Re: Switch PgStat_HashKey.objoid from Oid to uint64

2024-09-12 Thread Bertrand Drouvot
MyDatabaseId, proid); } " as pgstat_create_transactional is now: -pgstat_create_transactional(PgStat_Kind kind, Oid dboid, Oid objoid) +pgstat_create_transactional(PgStat_Kind kind, Oid dboid, uint64 objid) That's not an issue as both are unsigned

Re: Allow CI to only run the compiler warnings task

2024-09-11 Thread Bertrand Drouvot
Hi, On Wed, Sep 11, 2024 at 04:39:57PM +0300, Nazir Bilal Yavuz wrote: > Hi, > > On Wed, 11 Sept 2024 at 15:36, Bertrand Drouvot > wrote: > > > > Hi hackers, > > > > While working on a new pg_logicalinspect module ([1]), I reached a point > > where

Allow CI to only run the compiler warnings task

2024-09-11 Thread Bertrand Drouvot
r feedback, Regards, -- Bertrand Drouvot PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com >From b7289ee0f909e905b0e083cd3be41910c9c6a827 Mon Sep 17 00:00:00 2001 From: Bertrand Drouvot Date: Wed, 11 Sep 2024 11:01:41 + Subject: [PATCH v1] All

Re: Add contrib/pg_logicalsnapinspect

2024-09-11 Thread Bertrand Drouvot
Hi, On Wed, Sep 11, 2024 at 10:30:37AM +0530, Amit Kapila wrote: > On Tue, Sep 10, 2024 at 8:56 PM Bertrand Drouvot > wrote: > > > > On Mon, Sep 09, 2024 at 04:24:09PM +0530, Amit Kapila wrote: > > > On Fri, Aug 30, 2024 at 5:18 PM Bertrand Drouvot > > >

Re: Add contrib/pg_logicalsnapinspect

2024-09-10 Thread Bertrand Drouvot
Hi, On Mon, Sep 09, 2024 at 04:24:09PM +0530, Amit Kapila wrote: > On Fri, Aug 30, 2024 at 5:18 PM Bertrand Drouvot > wrote: > > as we decided not to expose the SnapBuildOnDisk and SnapBuild structs to > > public > > and to create/expose 2 new functions in snapbuild

Re: relfilenode statistics

2024-09-09 Thread Bertrand Drouvot
Hi, On Thu, Sep 05, 2024 at 04:48:36AM +, Bertrand Drouvot wrote: > Please find attached a mandatory rebase. > > In passing, checking if based on the previous discussion (and given that we > don't have the relation OID when writing buffers out) you see another approach &g

Re: per backend I/O statistics

2024-09-08 Thread Bertrand Drouvot
Hi, On Fri, Sep 06, 2024 at 03:03:17PM +, Bertrand Drouvot wrote: > - If we agree on the current proposal then I'll do some refactoring in > pg_stat_get_backend_io(), pg_stat_get_my_io() and pg_stat_get_io() to avoid > duplicated code (it's not done yet to ease the re

Re: Remove one TimestampTzGetDatum call in pg_stat_get_io()

2024-09-06 Thread Bertrand Drouvot
that that would display incorrect results on 32-bit (and not crashing). And since the tests don't check the values then we did not notice. Regards, -- Bertrand Drouvot PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com

Re: per backend I/O statistics

2024-09-06 Thread Bertrand Drouvot
Hi, On Thu, Sep 05, 2024 at 02:14:47PM +, Bertrand Drouvot wrote: > Hi, > > On Thu, Sep 05, 2024 at 03:03:32PM +0200, Alvaro Herrera wrote: > > So it seems to > > me that storing one of these struct for "my backend stats" is wrong: I > > think you should

Remove one TimestampTzGetDatum call in pg_stat_get_io()

2024-09-06 Thread Bertrand Drouvot
://www.postgresql.org/message-id/flat/ZtXR%2BCtkEVVE/LHF%40ip-10-97-1-34.eu-west-3.compute.internal Regards, -- Bertrand Drouvot PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com >From 09f7815a1dfe066ee3652b4f962b3f575aeee88e Mon Sep 17 00:00:00 2001 F

Re: per backend I/O statistics

2024-09-05 Thread Bertrand Drouvot
Hi, On Thu, Sep 05, 2024 at 03:03:32PM +0200, Alvaro Herrera wrote: > On 2024-Sep-03, Bertrand Drouvot wrote: > > > Cons is probably allocating shared memory space that might not be used ( > > sizeof(PgStat_IO) is 16392 so that could be a concern for a high number of > >

Re: Add parallel columns for seq scan and index scan on pg_stat_all_tables and _indexes

2024-09-04 Thread Bertrand Drouvot
Hi, On Wed, Sep 04, 2024 at 04:37:19PM +0200, Guillaume Lelarge wrote: > Hi, > > Le mer. 4 sept. 2024 à 16:18, Bertrand Drouvot > a écrit : > > What about adding a comment instead of this extra check? > > > > > Done too in v3. Thanks! 1 ===

Re: Track the amount of time waiting due to cost_delay

2024-09-04 Thread Bertrand Drouvot
Hi, On Mon, Sep 02, 2024 at 05:11:36AM +, Bertrand Drouvot wrote: > Hi, > > On Tue, Aug 20, 2024 at 12:48:29PM +, Bertrand Drouvot wrote: > > As it looks like we have a consensus not to wait on [0] (as reducing the > > number > > of interrupts makes sense on

Re: relfilenode statistics

2024-09-04 Thread Bertrand Drouvot
Hi, On Mon, Aug 05, 2024 at 05:28:22AM +, Bertrand Drouvot wrote: > Hi, > > On Thu, Jul 11, 2024 at 06:10:23AM +, Bertrand Drouvot wrote: > > Hi, > > > > On Thu, Jul 11, 2024 at 01:58:19PM +0900, Michael Paquier wrote: > > > On Wed, Jul 10, 2024 a

Re: Add parallel columns for seq scan and index scan on pg_stat_all_tables and _indexes

2024-09-04 Thread Bertrand Drouvot
Hi, On Wed, Sep 04, 2024 at 02:51:51PM +0200, Guillaume Lelarge wrote: > Hi, > > Le mer. 4 sept. 2024 à 10:47, Bertrand Drouvot > a écrit : > > > What about to get rid of the pgstat_count_parallel_heap_scan and add an > > extra > > bolean parameter to pgst

Re: Add parallel columns for seq scan and index scan on pg_stat_all_tables and _indexes

2024-09-04 Thread Bertrand Drouvot
Hi, On Wed, Sep 04, 2024 at 02:51:51PM +0200, Guillaume Lelarge wrote: > Le mer. 4 sept. 2024 à 10:47, Bertrand Drouvot > a écrit : > > > I don't see a CF entry for this patch. Would you mind creating one so that > > we don't lost track of it? > > > &g

Re: Add parallel columns for seq scan and index scan on pg_stat_all_tables and _indexes

2024-09-04 Thread Bertrand Drouvot
w columns to a statistics catalog. I'm actually > not sure that we were right to change pg_proc.dat manually. I think that's the right way to do. I don't see a CF entry for this patch. Would you mind creating one so that we don't lost track of it? Regards, -- Bertrand Drouvot PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com

Re: Parallel workers stats in pg_stat_database

2024-09-03 Thread Bertrand Drouvot
eed" to get parallel workers involved? For example, a "parallel_maint_workers" only test could be done in vacuum_parallel.sql. Regards, -- Bertrand Drouvot PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com

Re: Add callbacks for fixed-numbered stats flush in pgstats

2024-09-03 Thread Bertrand Drouvot
Hi, On Wed, Sep 04, 2024 at 03:12:37PM +0900, Michael Paquier wrote: > On Wed, Sep 04, 2024 at 05:28:56AM +0000, Bertrand Drouvot wrote: > > On Wed, Sep 04, 2024 at 02:05:46PM +0900, Kyotaro Horiguchi wrote: > >> The generalization sounds good to me, and hiding the private fla

Re: Add callbacks for fixed-numbered stats flush in pgstats

2024-09-03 Thread Bertrand Drouvot
ion could be to add only one callback (the flush_fixed_cb one) and get rid of the have_fixed_pending_cb one. Then add an extra parameter to the flush_fixed_cb that would only check if there is pending stats (without flushing them). I think those 2 callbacks are highly related that's why I thin

Re: per backend I/O statistics

2024-09-03 Thread Bertrand Drouvot
eed, I see this feature more like an "Administrator" one, where the administrator wants to be able to find out which session is reponsible of what (from an I/O point of view): like being able to anwser "which session is generating this massive amount of reads"? If we allow each

Re: DOCS - pg_replication_slot . Fix the 'inactive_since' description

2024-09-03 Thread Bertrand Drouvot
Hi, On Tue, Sep 03, 2024 at 05:52:53PM +1000, Peter Smith wrote: > On Tue, Sep 3, 2024 at 4:35 PM Bertrand Drouvot > wrote: > > > > Hi, > > > > On Tue, Sep 03, 2024 at 10:43:14AM +0530, Amit Kapila wrote: > > > On Mon, Sep 2, 2024 at 9:14 AM shveta mali

Re: per backend I/O statistics

2024-09-03 Thread Bertrand Drouvot
Hi, On Tue, Sep 03, 2024 at 03:37:49PM +0900, Kyotaro Horiguchi wrote: > At Mon, 2 Sep 2024 14:55:52 +0000, Bertrand Drouvot > wrote in > > Hi hackers, > > > > Please find attached a patch to implement $SUBJECT. > > > > While pg_stat_io provides cluster-wi

Re: DOCS - pg_replication_slot . Fix the 'inactive_since' description

2024-09-02 Thread Bertrand Drouvot
onvinced the current wording is confusing because: - the field type is described as a "timestamptz". - there is no duration unit in the wording (if we were to describe a duration, we would probably add an unit to it, like "The time (in s)..."). That said, if we want to highlight t

Re: Add callback in pgstats for backend initialization

2024-09-02 Thread Bertrand Drouvot
ttached. Should we add a few words about this new callback (and the others in PgStat_KindInfo while at it) in the "Custom Cumulative Statistics" section of xfunc.sgml? Regards, -- Bertrand Drouvot PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com

Re: define PG_REPLSLOT_DIR

2024-09-02 Thread Bertrand Drouvot
Hi, On Tue, Sep 03, 2024 at 09:15:50AM +0900, Michael Paquier wrote: > On Fri, Aug 30, 2024 at 12:21:29PM +0000, Bertrand Drouvot wrote: > > That said, I don't have a strong opinion on this one, I think that also > > makes > > sense to leave it as it is. Please

Re: Track IO times in pg_stat_io

2024-09-02 Thread Bertrand Drouvot
Hi, On Fri, Aug 23, 2024 at 07:32:16AM +, Bertrand Drouvot wrote: > Hi, > > On Wed, Mar 08, 2023 at 04:34:38PM -0800, Andres Freund wrote: > > On 2023-03-08 12:55:34 +0100, Drouvot, Bertrand wrote: > > > - pg_stat_io is "global" across all sessions. So, e

per backend I/O statistics

2024-09-02 Thread Bertrand Drouvot
tch_consistency. [1]: https://www.postgresql.org/message-id/20230309003438.rectf7xo7pw5t5cj%40awork3.anarazel.de Looking forward to your feedback, Regards, -- Bertrand Drouvot PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com >From 04c0a9e

Re: Track the amount of time waiting due to cost_delay

2024-09-01 Thread Bertrand Drouvot
Hi, On Tue, Aug 20, 2024 at 12:48:29PM +, Bertrand Drouvot wrote: > As it looks like we have a consensus not to wait on [0] (as reducing the > number > of interrupts makes sense on its own), then please find attached v4, a rebase > version (that also makes clear in the doc t

Re: define PG_REPLSLOT_DIR

2024-08-30 Thread Bertrand Drouvot
Hi, On Fri, Aug 30, 2024 at 03:34:56PM +0900, Michael Paquier wrote: > On Tue, Aug 20, 2024 at 04:23:06PM +0000, Bertrand Drouvot wrote: > > Please find attached v3 that: > > > > - takes care of your comments (and also removed the use of PG_TBLSPC_DIR in >

Re: Add contrib/pg_logicalsnapinspect

2024-08-30 Thread Bertrand Drouvot
Hi, On Fri, Aug 30, 2024 at 03:43:12PM +0530, Amit Kapila wrote: > On Thu, Aug 29, 2024 at 6:33 PM Bharath Rupireddy > wrote: > > > > On Thu, Aug 29, 2024 at 3:44 PM Bertrand Drouvot > > wrote: > > > > > > Yeah that's fair. And now I'm

Re: Add contrib/pg_logicalsnapinspect

2024-08-30 Thread Bertrand Drouvot
Hi, On Thu, Aug 29, 2024 at 02:15:47PM +, Bertrand Drouvot wrote: > I don't see any use case where it could be useful when the server is down. So, > I think I'll move forward with in core functions (unless someone has a > different > opinion). > Please find v2 at

Re: Add contrib/pg_logicalsnapinspect

2024-08-29 Thread Bertrand Drouvot
Hi, On Thu, Aug 29, 2024 at 06:33:19PM +0530, Bharath Rupireddy wrote: > On Thu, Aug 29, 2024 at 3:44 PM Bertrand Drouvot > wrote: > > > > That's right. I think this one would be simply enough to expose one or two > > functions in core too (and probably would not n

Re: Add contrib/pg_logicalsnapinspect

2024-08-29 Thread Bertrand Drouvot
Hi, On Thu, Aug 29, 2024 at 02:51:36PM +0530, Amit Kapila wrote: > On Wed, Aug 28, 2024 at 1:25 AM Bertrand Drouvot > wrote: > > > > On Mon, Aug 26, 2024 at 07:05:27PM +0530, Amit Kapila wrote: > > > On Thu, Aug 22, 2024 at 5:56 PM Bertrand Drouvot > >

Re: Flush pgstats file during checkpoints

2024-08-27 Thread Bertrand Drouvot
7;number of stats after crash'); +is($numcalls, '3', 'number of stats after crash'); $fixedstats = $node->safe_psql('postgres', "SELECT * FROM injection_points_stats_fixed();"); -is($fixedstats, '0|0|0|0|0', 'fixed s

Re: Add contrib/pg_logicalsnapinspect

2024-08-27 Thread Bertrand Drouvot
Hi, On Mon, Aug 26, 2024 at 07:05:27PM +0530, Amit Kapila wrote: > On Thu, Aug 22, 2024 at 5:56 PM Bertrand Drouvot > wrote: > > > > Please find attached a patch to $SUBJECT. > > > > This module provides SQL functions to inspect the contents of serialized > >

Re: Track IO times in pg_stat_io

2024-08-23 Thread Bertrand Drouvot
ld be > particularly hard. FWIW, I think that would be great and plan to have a look at this (unless someone beats me to it). Regards, -- Bertrand Drouvot PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com

Add contrib/pg_logicalsnapinspect

2024-08-22 Thread Bertrand Drouvot
g). 6. Related documentation has been added. 7. A test has been added. 8. I don't like the module name that much but it follows the same as for pg_walinspect. Looking forward to your feedback, Regards, -- Bertrand Drouvot PostgreSQL Contributors Team RDS O

Re: Restart pg_usleep when interrupted

2024-08-21 Thread Bertrand Drouvot
if CF 5118 gets committed and we move to WaitLatchUs() and 2.2/ it won't break anything if CF gets committed and we don't move to WaitLatchUs(). For 1.1 it would still prevent the leader to be waked up too frequently by the parallel workers. 2. still discuss the "need" and get a

Re: define PG_REPLSLOT_DIR

2024-08-20 Thread Bertrand Drouvot
Hi, On Wed, Aug 21, 2024 at 10:14:20AM +0530, Ashutosh Bapat wrote: > If this isn't in commitfest already, please add it there. > Done in [0]. [0]: https://commitfest.postgresql.org/49/5193/ Thanks! Regards, -- Bertrand Drouvot PostgreSQL Contributors Team RDS Open Source Datab

Re: define PG_REPLSLOT_DIR

2024-08-20 Thread Bertrand Drouvot
Hi, On Tue, Aug 20, 2024 at 12:06:52PM +, Bertrand Drouvot wrote: > Hi, > > On Tue, Aug 20, 2024 at 05:41:48PM +0900, Michael Paquier wrote: > > On Tue, Aug 20, 2024 at 11:10:46AM +0530, Ashutosh Bapat wrote: > > > Since these are all related changes, doing th

Re: define PG_REPLSLOT_DIR

2024-08-20 Thread Bertrand Drouvot
he macros from the comments). Thanks! Regards, -- Bertrand Drouvot PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com

Re: define PG_REPLSLOT_DIR

2024-08-20 Thread Bertrand Drouvot
Hi, On Tue, Aug 20, 2024 at 10:15:44AM -0400, Alvaro Herrera wrote: > On 2024-Aug-19, Bertrand Drouvot wrote: > > > diff --git a/src/include/common/relpath.h b/src/include/common/relpath.h > > index 6f006d5a93..a6cb091635 100644 > > --- a/src/include/common/relpath.h >

Re: Restart pg_usleep when interrupted

2024-08-20 Thread Bertrand Drouvot
kes sense, I just provided a rebase version of the 1 Hz version (see [0], that also makes clear in the doc that the new field might show slightly old values). [0]: https://www.postgresql.org/message-id/ZsSQnS9OW9EWSOk4%40ip-10-97-1-34.eu-west-3.compute.internal Regards, -- Bertrand

Re: Track the amount of time waiting due to cost_delay

2024-08-20 Thread Bertrand Drouvot
Hi, On Mon, Jul 01, 2024 at 04:59:25AM +, Bertrand Drouvot wrote: > Hi, > > On Fri, Jun 28, 2024 at 08:07:39PM +, Imseih (AWS), Sami wrote: > > > 46ebdfe164 will interrupt the leaders sleep every time a parallel workers > > > reports > > > pro

Re: define PG_REPLSLOT_DIR

2024-08-20 Thread Bertrand Drouvot
Hi, On Tue, Aug 20, 2024 at 05:47:57PM +0900, Michael Paquier wrote: > On Mon, Aug 19, 2024 at 02:11:55PM +0000, Bertrand Drouvot wrote: > > I made the changes for pg_tblspc in pg_combinebackup.c as the number of > > occurences > > are greater that the "pg_wal&q

Re: define PG_REPLSLOT_DIR

2024-08-20 Thread Bertrand Drouvot
> Doing multiple commits with individual definitions for each path would > be the way to go for me. All that is mechanical, still that feels > slightly cleaner. Right, that's what v2 has done. If there is a need for v3 then I'll add one dedicated patch for Ashutosh's proposal

Re: define PG_REPLSLOT_DIR

2024-08-19 Thread Bertrand Drouvot
Hi, On Tue, Aug 20, 2024 at 09:26:59AM +0530, Ashutosh Bapat wrote: > On Mon, Aug 19, 2024 at 7:47 PM Bertrand Drouvot > wrote: > > > > Hi, > > > > On Mon, Aug 19, 2024 at 04:11:31PM +0530, Ashutosh Bapat wrote: > > > On Wed, Aug 14, 2024 a

Re: Avoid orphaned objects dependencies, take 3

2024-08-19 Thread Bertrand Drouvot
Hi, On Wed, Jul 10, 2024 at 07:31:06AM +, Bertrand Drouvot wrote: > So, to sum up: > > A. Locking is now done exclusively with LockNotPinnedObject(Oid classid, Oid > objid) > so that it's now always clear what object we want to acquire a lock for. It > means &g

Re: Restart pg_usleep when interrupted

2024-08-19 Thread Bertrand Drouvot
Hi, On Thu, Aug 15, 2024 at 04:13:29PM -0500, Nathan Bossart wrote: > On Wed, Aug 14, 2024 at 06:00:06AM +0000, Bertrand Drouvot wrote: > > I gave it more thoughts and I don't think we have to choose between the two. > > The 1 Hz approach reduces the number of interru

Re: define PG_REPLSLOT_DIR

2024-08-19 Thread Bertrand Drouvot
Hi, On Mon, Aug 19, 2024 at 04:11:31PM +0530, Ashutosh Bapat wrote: > On Wed, Aug 14, 2024 at 5:02 PM Bertrand Drouvot > wrote: > > > > Hi hackers, > > > > while working on a replication slot tool (idea is to put it in contrib, not > > shared yet), I reali

Re: define PG_REPLSLOT_DIR

2024-08-19 Thread Bertrand Drouvot
Hi, On Fri, Aug 16, 2024 at 01:31:11PM +0900, Yugo Nagata wrote: > On Wed, 14 Aug 2024 11:32:14 + > Bertrand Drouvot wrote: > > Looking forward to your feedback, > > /* restore all slots by iterating over all on-disk entries */ > - replication_dir = All

Re: define PG_REPLSLOT_DIR

2024-08-19 Thread Bertrand Drouvot
Hi, On Thu, Aug 15, 2024 at 08:40:42PM -0400, Alvaro Herrera wrote: > On 2024-Aug-14, Bertrand Drouvot wrote: > > > Out of curiosity, checking the sizes of affected files (O2, no debug): > > > > with patch: > > > >textdata bss dec hex

define PG_REPLSLOT_DIR

2024-08-14 Thread Bertrand Drouvot
ind/filemap.o Also, I think we could do the same for: pg_notify pg_serial pg_subtrans pg_wal pg_multixact pg_tblspc pg_logical And I volunteer to do so if you think that makes sense. Looking forward to your feedback, Regards, -- Bertrand Drouvot PostgreSQL Contributors Team RDS Open Sou

Re: Restart pg_usleep when interrupted

2024-08-13 Thread Bertrand Drouvot
of interrupts and Sami's patch provides a way to get "accurate" delay in case of interrupts. I think both have their own benefit. Regards, -- Bertrand Drouvot PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com

Re: Historic snapshot doesn't track txns committed in BUILDING_SNAPSHOT state

2024-08-13 Thread Bertrand Drouvot
fficult to follow. > > I'm not sure what happened but I attach the new thread to the CF: Unfortunately your last reply did start a new email thread again. Regards, -- Bertrand Drouvot PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com

Re: Historic snapshot doesn't track txns committed in BUILDING_SNAPSHOT state

2024-08-12 Thread Bertrand Drouvot
Not sure what happened but it looks like your reply in [0] is not part of the initial thread [1], but created a new thread instead, making the whole conversation difficult to follow. [0]: https://www.postgresql.org/message-id/tencent_8DEC9842690A9B6AFD52D4659EF0700E9409%40qq.com [1]: https://www.p

Re: Restart pg_usleep when interrupted

2024-08-12 Thread Bertrand Drouvot
s compare to [1]) though. [0]: https://commitfest.postgresql.org/49/5027/ [1]: https://git.postgresql.org/gitweb/?p=postgresql.git;a=commit;h=f1889729dd Regards, -- Bertrand Drouvot PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com

Re: Restart pg_usleep when interrupted

2024-08-12 Thread Bertrand Drouvot
Hi, On Mon, Aug 12, 2024 at 10:19:56AM -0500, Sami Imseih wrote: > v10 attached in the previous message addresses > Bertrands last comment and replaces “This” with “this" > Thanks, v10 LGTM. Regards, -- Bertrand Drouvot PostgreSQL Contributors Team RDS Open Source Databa

Re: Historic snapshot doesn't track txns committed in BUILDING_SNAPSHOT state

2024-08-12 Thread Bertrand Drouvot
r my proposal) so that we can jump directly in the appropriate case. I think that way the code is easier to maintain instead of having to deal with the same info values in distinct places. Regards, -- Bertrand Drouvot PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com

Re: Historic snapshot doesn't track txns committed in BUILDING_SNAPSHOT state

2024-08-11 Thread Bertrand Drouvot
tate(builder) == SNAPBUILD_BUILDING_SNAPSHOT && info != XLOG_HEAP_INPLACE) || ctx->fast_forward) return; That way we'd still rely on what's being done in the XLOG_HEAP_INPLACE case ( should it change in the future). 6 === v3-0002 LGTM. Regards, -- Ber

Re: Restart pg_usleep when interrupted

2024-08-11 Thread Bertrand Drouvot
Hi, On Sat, Aug 10, 2024 at 08:27:56AM -0500, Sami Imseih wrote: > > > v9 has some has some minor corrections to the comments. > Thanks! 1 === + * Unlike pg_usleep, This function continues s/This/this/? Apart from the above, LGTM. Regards, -- Bertrand Drouvot PostgreSQL

Re: Restart pg_usleep when interrupted

2024-08-09 Thread Bertrand Drouvot
sewhere. Perhaps we should move it > to pgsleep.c and rename it to something to the effect of > pg_usleep_non_interruptable(). Yeah, I had the same thought in [1], so +1. [1]: https://www.postgresql.org/message-id/ZpDhS4nFX66ItAze%40ip-10-97-1-34.eu-west-3.compute.internal Regards, -- Bertrand

Re: Historic snapshot doesn't track txns committed in BUILDING_SNAPSHOT state

2024-08-09 Thread Bertrand Drouvot
build correctly + what about? Test tracking of committed transactions during BUILDING_SNAPSHOT Regards, -- Bertrand Drouvot PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com

Re: Restart pg_usleep when interrupted

2024-08-07 Thread Bertrand Drouvot
seconds since the Epoch. It means that we’re currently about 237 years away from the limit. So even, if we were to say add 2 "recents" of them we are still about 183 years away from the limit. Regards, -- Bertrand Drouvot PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com

Re: Restart pg_usleep when interrupted

2024-08-07 Thread Bertrand Drouvot
Hi, On Wed, Aug 07, 2024 at 09:11:19AM -0500, Sami Imseih wrote: > > > > On Aug 7, 2024, at 1:00 AM, Bertrand Drouvot > > wrote: > > > > add t (in microseconds) to x” > > > I was attempting to be more verbose in the comment, > but what you ha

Re: Historic snapshot doesn't track txns committed in BUILDING_SNAPSHOT state

2024-08-07 Thread Bertrand Drouvot
produce with a single instance and multiple sessions : replace in your repro the "(sub)create subscription" by a "logical slot creation" and finish the test by "pg_logical_slot_get_changes" on the slot created above. Regards, -- Bertrand Drouvot PostgreSQL Contributo

Re: Restart pg_usleep when interrupted

2024-08-06 Thread Bertrand Drouvot
me" -> "add t (in microseconds) to x" Does that make sense to you? That said, it looks good to me. Regards, -- Bertrand Drouvot PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com >From 619d4fea1eada6fb65fec673bc9600f8502f9b00 Mon Sep 1

Re: Fix comments in instr_time.h and remove an unneeded cast to int64

2024-08-06 Thread Bertrand Drouvot
Hi, On Tue, Aug 06, 2024 at 05:49:32PM +0300, Heikki Linnakangas wrote: > On 06/08/2024 17:20, Tom Lane wrote: > > Heikki Linnakangas writes: > > > On 06/08/2024 11:54, Bertrand Drouvot wrote: > > > > Please find attached a tiny patch to correct those and, in

Re: Restart pg_usleep when interrupted

2024-08-06 Thread Bertrand Drouvot
f it. 2 === What about providing a quick comment about the 2 new macros in header of instr_time.h? (like it is done for the others macros) Regards, -- Bertrand Drouvot PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com

Fix comments in instr_time.h and remove an unneeded cast to int64

2024-08-06 Thread Bertrand Drouvot
comment for INSTR_TIME_GET_NANOSEC(). Please find attached a tiny patch to correct those and, in passing, remove what I think is an unneeded cast to int64. [1]: https://www.postgresql.org/message-id/19E276C9-2C2B-435A-B275-8FA2AEB8%40gmail.com Regards, -- Bertrand Drouvot PostgreSQL

Re: relfilenode statistics

2024-08-04 Thread Bertrand Drouvot
Hi, On Thu, Jul 11, 2024 at 06:10:23AM +, Bertrand Drouvot wrote: > Hi, > > On Thu, Jul 11, 2024 at 01:58:19PM +0900, Michael Paquier wrote: > > On Wed, Jul 10, 2024 at 01:38:06PM +, Bertrand Drouvot wrote: > > > So, I think it makes sense to link the hashkey t

Re: remove volatile qualifiers from pg_stat_statements

2024-07-31 Thread Bertrand Drouvot
ssing something? Or can we remove these qualifiers now? I share the same understanding and I think those can be removed. The patch LGTM. Regards, -- Bertrand Drouvot PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com

Re: Flush pgstats file during checkpoints

2024-07-30 Thread Bertrand Drouvot
Hi, On Tue, Jul 30, 2024 at 03:25:31PM +0900, Michael Paquier wrote: > On Mon, Jul 29, 2024 at 04:46:17AM +0000, Bertrand Drouvot wrote: > > Thanks! 0001 attached is > > v4-0001-Revert-Test-that-vacuum-removes-tuples-older-than.patch > > so I guess you did not attached the

Re: Restart pg_usleep when interrupted

2024-07-29 Thread Bertrand Drouvot
Hi, On Mon, Jul 29, 2024 at 06:15:49PM -0500, Sami Imseih wrote: > > On Jul 26, 2024, at 3:27 AM, Bertrand Drouvot > > wrote: > > 3 === > > > > I gave more thoughts and I think it can be simplified a bit to reduce the > > number of operations in the while lo

Re: Allow logical failover slots to wait on synchronous replication

2024-07-28 Thread Bertrand Drouvot
ke > the lock, but adds noise? > What do you think? > > "slot.c:2756:46: warning: passing argument 2 of ‘memcpy’ discards > ‘volatile’ qualifier from pointer target type > [-Wdiscarded-qualifiers]" Right, we may want to cast it then but given that the for loop is &q

Re: Flush pgstats file during checkpoints

2024-07-28 Thread Bertrand Drouvot
Hi, On Tue, Jul 23, 2024 at 12:52:11PM +0900, Michael Paquier wrote: > On Mon, Jul 22, 2024 at 07:01:41AM +0000, Bertrand Drouvot wrote: > > 3 === > > > > + /* > > +* Read the redo LSN stored in the file. > > +*/ > > +

Re: Restart pg_usleep when interrupted

2024-07-26 Thread Bertrand Drouvot
ent_time); if (current_time.ticks > absolute.ticks) { break; Regards, -- Bertrand Drouvot PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com

Re: Flush pgstats file during checkpoints

2024-07-22 Thread Bertrand Drouvot
Hi, On Wed, Jul 17, 2024 at 12:52:12PM +0900, Michael Paquier wrote: > On Tue, Jul 16, 2024 at 10:37:39AM +0900, Michael Paquier wrote: > > On Fri, Jul 12, 2024 at 01:01:19PM +, Bertrand Drouvot wrote: > >> Instead of removing the stat file, should we keep it around until th

Re: Restart pg_usleep when interrupted

2024-07-15 Thread Bertrand Drouvot
h the last one shared in [2]. [1]: https://www.postgresql.org/message-id/Zo0UdeE3i9d0Wt5E%40ip-10-97-1-34.eu-west-3.compute.internal [2]: https://www.postgresql.org/message-id/C18017A1-EDFD-4F2F-BCA1-0572D4CCC92B%40gmail.com Regards, -- Bertrand Drouvot PostgreSQL Contributors Team RDS Open S

Re: Flush pgstats file during checkpoints

2024-07-12 Thread Bertrand Drouvot
Hi, On Fri, Jul 12, 2024 at 12:10:26PM +, Bertrand Drouvot wrote: > Need to spend more time and thoughts on 0002+. I think there is a corner case, say: 1. shutdown checkpoint at LSN1 2. startup->reads the stat file (contains LSN1)->all good->read stat file and remove it

  1   2   3   4   5   >