Re: Inconsistency in reporting checkpointer stats

2024-01-29 Thread Michael Paquier
On Sat, Jan 20, 2024 at 08:10:03AM +0530, vignesh C wrote: > The patch does not apply anymore, please post a rebased version of the patch : There is more to it. Some of the columns of pg_stat_bgwriter have been moved to a different view, aka pg_stat_checkpointer. I have marked the patch as

Re: Inconsistency in reporting checkpointer stats

2024-01-19 Thread vignesh C
On Sun, 19 Feb 2023 at 15:28, Nitin Jadhav wrote: > > > This doesn't pass the tests, because the regression tests weren't adjusted: > > https://api.cirrus-ci.com/v1/artifact/task/5937624817336320/testrun/build/testrun/regress/regress/regression.diffs > > Thanks for sharing this. I have fixed this

Re: Inconsistency in reporting checkpointer stats

2023-03-20 Thread Gregory Stark (as CFM)
On Sun, 19 Feb 2023 at 04:58, Nitin Jadhav wrote: > > > This doesn't pass the tests, because the regression tests weren't adjusted: > > https://api.cirrus-ci.com/v1/artifact/task/5937624817336320/testrun/build/testrun/regress/regress/regression.diffs > > Thanks for sharing this. I have fixed this

Re: Inconsistency in reporting checkpointer stats

2023-02-19 Thread Nitin Jadhav
> This doesn't pass the tests, because the regression tests weren't adjusted: > https://api.cirrus-ci.com/v1/artifact/task/5937624817336320/testrun/build/testrun/regress/regress/regression.diffs Thanks for sharing this. I have fixed this in the patch attached. >> IMO, there's no need for 2

Re: Inconsistency in reporting checkpointer stats

2023-02-13 Thread Andres Freund
On 2022-12-21 17:14:12 +0530, Nitin Jadhav wrote: > Kindly review and share your comments. This doesn't pass the tests, because the regression tests weren't adjusted: https://api.cirrus-ci.com/v1/artifact/task/5937624817336320/testrun/build/testrun/regress/regress/regression.diffs

Re: Inconsistency in reporting checkpointer stats

2023-01-27 Thread Nitin Jadhav
> IMO, there's no need for 2 separate patches for these changes. I will make it a single patch while sharing the next patch. > +(errmsg("restartpoint complete: wrote %d buffers (%.1f%%), " > +"wrote %d slru buffers (%.1f%%); %d WAL > file(s) added, " > +

Re: Inconsistency in reporting checkpointer stats

2023-01-26 Thread Bharath Rupireddy
On Wed, Dec 21, 2022 at 5:15 PM Nitin Jadhav wrote: > > I have modified the code accordingly and attached the new version of > patches. patch 0001 fixes the inconsistency in checkpointer stats and > patch 0002 separates main buffer and SLRU buffer count from checkpoint > complete log message.

Re: Inconsistency in reporting checkpointer stats

2022-12-21 Thread Kyotaro Horiguchi
At Wed, 21 Dec 2022 17:14:12 +0530, Nitin Jadhav wrote in > [1]: > 2022-12-21 10:52:25.931 UTC [63530] LOG: checkpoint complete: wrote > 4670 buffers (28.5%), wrote 3 slru buffers (0.0%); 0 WAL file(s) > added, 0 removed, 4 recycled; write=0.045 s, sync=0.161 s, total=0.244 > s; sync files=25,

Re: Inconsistency in reporting checkpointer stats

2022-12-21 Thread Kyotaro Horiguchi
At Mon, 19 Dec 2022 18:05:38 +0530, Bharath Rupireddy wrote in > On Fri, Dec 16, 2022 at 2:14 PM Kyotaro Horiguchi > wrote: > > > > In the first place I don't like that we count the same things twice. > > Couldn't we count the number only by any one of them? > > > > If we remove

Re: Inconsistency in reporting checkpointer stats

2022-12-21 Thread Nitin Jadhav
Thanks Robert and Andres for sharing your thoughts. I have modified the code accordingly and attached the new version of patches. patch 0001 fixes the inconsistency in checkpointer stats and patch 0002 separates main buffer and SLRU buffer count from checkpoint complete log message. In 0001, I

Re: Inconsistency in reporting checkpointer stats

2022-12-21 Thread Bharath Rupireddy
On Tue, Dec 20, 2022 at 11:08 PM Andres Freund wrote: > > On 2022-12-20 08:18:36 -0500, Robert Haas wrote: > > I think that the SLRU information is potentially useful, but mixing it > > with the information about regular buffers just seems confusing. > > +1 > > At least for now, it'd be different

Re: Inconsistency in reporting checkpointer stats

2022-12-20 Thread Andres Freund
On 2022-12-20 08:18:36 -0500, Robert Haas wrote: > I think that the SLRU information is potentially useful, but mixing it > with the information about regular buffers just seems confusing. +1 At least for now, it'd be different if/when we manage to move SLRUs to the main buffer pool. - Andres

Re: Inconsistency in reporting checkpointer stats

2022-12-20 Thread Robert Haas
On Tue, Dec 20, 2022 at 8:03 AM Nitin Jadhav wrote: > Thanks Robert for sharing your thoughts. > My first thought was to just remove counting SLRU buffers, then after > some more analysis, I found that the checkpointer is responsible for > including both regular data buffers and SLRU buffers. I

Re: Inconsistency in reporting checkpointer stats

2022-12-20 Thread Nitin Jadhav
> Presumably we could make this consistent either by counting SLRU > writes in both places, or by counting them in neither place. This > proposal would count them in both places. But why is that the right > thing to do? > > I'm somewhat inclined to think that we should use "buffers" to mean >

Re: Inconsistency in reporting checkpointer stats

2022-12-19 Thread Robert Haas
On Wed, Dec 14, 2022 at 2:32 AM Nitin Jadhav wrote: > In order to fix this, the > PendingCheckpointerStats.buf_written_checkpoints should be incremented > in SlruInternalWritePage() similar to > CheckpointStats.ckpt_bufs_written. I have attached the patch for the > same. Please share your

Re: Inconsistency in reporting checkpointer stats

2022-12-19 Thread Bharath Rupireddy
On Fri, Dec 16, 2022 at 2:14 PM Kyotaro Horiguchi wrote: > > In the first place I don't like that we count the same things twice. > Couldn't we count the number only by any one of them? > > If we remove CheckPointerStats.ckpt_bufs_written, CreateCheckPoint can > get the final number as the

Re: Inconsistency in reporting checkpointer stats

2022-12-16 Thread Kyotaro Horiguchi
At Wed, 14 Dec 2022 16:54:53 +0530, Bharath Rupireddy wrote in > Indeed PendingCheckpointerStats.buf_written_checkpoints needs to count > buffer writes in SlruInternalWritePage(). However, does it need to be > done immediately there? The stats will not be visible to the users > until the next

Re: Inconsistency in reporting checkpointer stats

2022-12-15 Thread Nitin Jadhav
> /* If part of a checkpoint, count this as a buffer written. */ >if (fdata) >CheckpointStats.ckpt_bufs_written++; > + PendingCheckpointerStats.buf_written_checkpoints++; > Also, the proposed patch would touch PendingCheckpointerStats even > when there is no fdata, aka outside

Re: Inconsistency in reporting checkpointer stats

2022-12-15 Thread Nitin Jadhav
is before the end of the checkpoint. Please share if you have any other ideas. On Wed, Dec 14, 2022 at 4:55 PM Bharath Rupireddy wrote: > > On Wed, Dec 14, 2022 at 1:02 PM Nitin Jadhav > wrote: > > > > Hi, > > > > While working on checkpoint related stuff, I

Re: Inconsistency in reporting checkpointer stats

2022-12-14 Thread Michael Paquier
On Wed, Dec 14, 2022 at 04:54:53PM +0530, Bharath Rupireddy wrote: > Indeed PendingCheckpointerStats.buf_written_checkpoints needs to count > buffer writes in SlruInternalWritePage(). However, does it need to be > done immediately there? The stats will not be visible to the users > until the next

Re: Inconsistency in reporting checkpointer stats

2022-12-14 Thread Bharath Rupireddy
On Wed, Dec 14, 2022 at 1:02 PM Nitin Jadhav wrote: > > Hi, > > While working on checkpoint related stuff, I have encountered that > there is some inconsistency while reporting checkpointer stats. When a > checkpoint gets completed, a checkpoint complete message gets logged.

Inconsistency in reporting checkpointer stats

2022-12-13 Thread Nitin Jadhav
Hi, While working on checkpoint related stuff, I have encountered that there is some inconsistency while reporting checkpointer stats. When a checkpoint gets completed, a checkpoint complete message gets logged. This message has a lot of information including the buffers written