Re: Extension Enhancement: Buffer Invalidation in pg_buffercache

2023-07-05 Thread Nitin Jadhav
uffer before > invalidating. > +If the buffer is still dirty it returns false. > +*/ > +bool The star(*) and space are missing here. Please refer to the style of function comments and change accordingly. Thanks & Regards, Nitin Jadhav On Fri, Jun 30, 2023 at 4:17 PM Palak Chaturvedi wrote:

Re: Improve WALRead() to suck data directly from WAL buffers when possible

2023-03-11 Thread Nitin Jadhav
ter_delay=1ms ./pgbench --initialize --scale=300 postgres WAL buffers hit=37214 WAL buffers miss=844 Please share your thoughts. Thanks & Regards, Nitin Jadhav On Tue, Mar 7, 2023 at 12:39 PM Bharath Rupireddy wrote: > > On Tue, Mar 7, 2023 at 3:30 AM Nathan Bossart >

Re: Inconsistency in reporting checkpointer stats

2023-02-19 Thread Nitin Jadhav
o need for 2 separate patches for these changes. > I will make it a single patch while sharing the next patch. Clubbed both patches into one. Thanks & Regards, Nitin Jadhav On Tue, Feb 14, 2023 at 6:08 AM Andres Freund wrote: > > > On 2022-12-21 17:14:12 +0530, Nitin Jadhav wrote:

Re: Fix GUC_NO_SHOW_ALL test scenario in 003_check_guc.pl

2023-02-08 Thread Nitin Jadhav
ne in 0002, only for > HEAD, as that would be a new check. > > Remains > 0002, that I am letting sleep to see if there's interest for it, or > perhaps more ideas around it. Makes sense and the patch looks good to me. Thanks & Regards, Nitin Jadhav On Wed, Feb 8, 2023 at 1:29 PM M

Re: Add a test case related to the error "cannot fetch toast data without an active snapshot"

2023-02-07 Thread Nitin Jadhav
); DO $$ DECLARE v_r record; DECLARE vref_cursor REFCURSOR; BEGIN OPEN vref_cursor FOR SELECT data FROM toasted; LOOP fetch vref_cursor into v_r; INSERT INTO toasted(data) VALUES(v_r.data); COMMIT; END LOOP; END;$$; Thanks & Regards, Nitin Jadhav On Fri, Jan 27, 2023 at 6:26 PM Nitin Jadhav wro

Re: Fix GUC_NO_SHOW_ALL test scenario in 003_check_guc.pl

2023-02-04 Thread Nitin Jadhav
s & Regards, Nitin Jadhav On Sat, Feb 4, 2023 at 1:07 AM Tom Lane wrote: > > Nitin Jadhav writes: > > My concern is if we do this, then we will end up having some policies > > (which can be read from pg_show_all_settings()) in guc.sql and some in > > guc.c. I feel al

Re: Fix GUC_NO_SHOW_ALL test scenario in 003_check_guc.pl

2023-02-03 Thread Nitin Jadhav
his (as true) while creating the table tab_settings_flags in guc.sq and just remove (NO_SHOW_ALL && !NO_RESET_ALL) check from guc.sql. I don't think doing this is a problem as we can retain the support of existing signatures of the pg_show_all_settings function as suggested by Justin upthr

Re: Fix GUC_NO_SHOW_ALL test scenario in 003_check_guc.pl

2023-01-30 Thread Nitin Jadhav
we have a GUC marked as GUC_NO_SHOW_ALL but not GUC_NOT_IN_CONFIG. For me it makes sense if a GUC is marked as NO_SHOW_ALL and it can be present in a sample file. It's up to the author/developer to choose which all flags are applicable to the newly introduced GUCs. Please share your thoughts. Thanks & Regard

Re: Fix GUC_NO_SHOW_ALL test scenario in 003_check_guc.pl

2023-01-29 Thread Nitin Jadhav
ZMOGEtjk%2Beh0Zeiz%3D46ETVkr0koYAjWt8SoJUJJUe9g%40mail.gmail.com#04705e421e0dc63b1f0c862ae4929e6f Thanks & Regards, Nitin Jadhav On Wed, Jan 18, 2023 at 12:31 PM Nitin Jadhav wrote: > > > We would miss the names of the parameters that are marked as NO_SHOW, > > missing from pg_settings, making

Re: Inconsistency in reporting checkpointer stats

2023-01-27 Thread Nitin Jadhav
t you are suggesting. But the question is not about tracking slru buffers, it is about separating this information from main buffers count during checkpoint. I think there is enough discussion done upthread to exclude slru buffers from CheckpointStats.ckpt_bufs_written. Please share if you still strongl

Re: Improve GetConfigOptionValues function

2023-01-27 Thread Nitin Jadhav
; Regards, Nitin Jadhav On Wed, Jan 25, 2023 at 9:23 PM Tom Lane wrote: > > Nitin Jadhav writes: > > I agree that the developer can use both GUC_NO_SHOW_ALL and > > GUC_EXPLAIN knowingly or unknowingly for a single GUC. If used by > > mistake then accor

Add a test case related to the error "cannot fetch toast data without an active snapshot"

2023-01-27 Thread Nitin Jadhav
* from detecting the problem. But it's better than nothing, and for sure * we shouldn't expend code on masking the problem more.) */ Thanks & Regards, Nitin Jadhav

Re: Improve GetConfigOptionValues function

2023-01-25 Thread Nitin Jadhav
owingly or unknowingly for a single GUC. If used by mistake then according to the existing code (without patch), GUC_NO_SHOW_ALL takes higher precedence whether it is marked first or last in the code. I am more convinced with this behaviour as I feel it is safer than exposing the information which th

Re: Improve GetConfigOptionValues function

2023-01-25 Thread Nitin Jadhav
ces the code size. Thanks & Regards, Nitin Jadhav On Mon, Jan 23, 2023 at 9:51 PM Tom Lane wrote: > > Bharath Rupireddy writes: > > LGTM. I've marked it RfC. > > After looking at this, it seemed to me that the factorization > wasn't quite right after all: specific

Re: Improve GetConfigOptionValues function

2023-01-23 Thread Nitin Jadhav
one, to not lose track of it. Added https://commitfest.postgresql.org/42/4140/ Thanks & Regards, Nitin Jadhav On Mon, Jan 23, 2023 at 11:30 AM Bharath Rupireddy wrote: > > On Thu, Jan 19, 2023 at 3:27 PM Nitin Jadhav > wrote: > > > > > Possibly a better answer is to refac

Re: Improve GetConfigOptionValues function

2023-01-19 Thread Nitin Jadhav
hed a patch for the same. Please share the comments if any. Thanks & Regards, Nitin Jadhav On Wed, Jan 18, 2023 at 9:44 PM Tom Lane wrote: > > Nitin Jadhav writes: > > GetConfigOptionValues function extracts the config parameters for the > > given variabl

Re: Improve GetConfigOptionValues function

2023-01-19 Thread Nitin Jadhav
of such parameters (GUC_NO_SHOW_ALL and GUC_SUPERUSER_ONLY) are less, we may not see improvements in performance. We can treat it as a kind of refactoring. Thanks & Regards, Nitin Jadhav On Wed, Jan 18, 2023 at 1:47 PM Bharath Rupireddy wrote: > > On Wed, Jan 18, 2023 at 1:24 PM Niti

Re: Improve GetConfigOptionValues function

2023-01-17 Thread Nitin Jadhav
Attaching the patch. On Wed, Jan 18, 2023 at 1:21 PM Nitin Jadhav wrote: > > Hi, > > GetConfigOptionValues function extracts the config parameters for the > given variable irrespective of whether it results in noshow or not. > But the parent function show_all_settings

Improve GetConfigOptionValues function

2023-01-17 Thread Nitin Jadhav
a return statement in GetConfigOptionValues() when noshow is set to true is needed. Attached the patch for the same. Please share your thoughts. Thanks & Regards, Nitin Jadhav

Re: Fix GUC_NO_SHOW_ALL test scenario in 003_check_guc.pl

2023-01-17 Thread Nitin Jadhav
((conf->flags & GUC_SUPERUSER_ONLY) && !has_privs_of_role(GetUserId(), ROLE_PG_READ_ALL_SETTINGS))) *noshow = true; else *noshow = false; } - - - } On Mon, Jan 16, 2023 at 7:58 AM Michael Paquier wrote: > > On Sat, Jan 14, 2023 at

Re: Fix GUC_NO_SHOW_ALL test scenario in 003_check_guc.pl

2023-01-14 Thread Nitin Jadhav
ay' in 003_check_guc.pl whenever we add such GUCs. I am not able to choose any of the above options as each has some disadvantages but if no other options exist, then I would like to go with option-3 as it validates more than the one currently doing. Please share if any other better ideas. Thanks

Fix GUC_NO_SHOW_ALL test scenario in 003_check_guc.pl

2023-01-13 Thread Nitin Jadhav
to work on the patch if a fix is required. [1]: SELECT name FROM pg_settings WHERE NOT 'NOT_IN_SAMPLE' = ANY (pg_settings_get_flags(name)) AND name <> 'config_file' ORDER BY 1; Thanks & Regards, Nitin Jadhav

Re: Split index and table statistics into different types of stats

2023-01-09 Thread Nitin Jadhav
to table/index stats? +1 to keep common functions/code between table and index stats. Only the data structure should be different as the goal is to shrink the current memory usage. Thanks & Regards, Nitin Jadhav On Thu, Jan 5, 2023 at 3:35 PM Drouvot, Bertrand wrote: > > Hi, > &g

Re: Inconsistency in reporting checkpointer stats

2022-12-21 Thread Nitin Jadhav
; write=0.045 s, sync=0.161 s, total=0.244 s; sync files=25, longest=0.146 s, average=0.007 s; distance=66130 kB, estimate=66130 kB; lsn=0/5557C78, redo lsn=0/5557C40 Thanks & Regards, Nitin Jadhav On Tue, Dec 20, 2022 at 11:08 PM Andres Freund wrote: > > On 2022-12-20 08:18:36 -0500, R

Re: Inconsistency in reporting checkpointer stats

2022-12-20 Thread Nitin Jadhav
rocessSyncRequests() up into CheckPointGuts() to make it clearer that it applies to all the SLRU mini-buffer-pools as well as the main buffer pool. Rearrange things so that data collected in CheckpointStats includes SLRU activity. Thanks & Regards, Nitin Jadhav On Tue, Dec 20, 2022 at 2:38 AM Ro

Re: Inconsistency in reporting checkpointer stats

2022-12-15 Thread Nitin Jadhav
fdata, aka outside the context of a checkpoint or > shutdown sequence. Sorry. I missed adding braces. Fixed in the v2 patch attached. Thanks & Regards, Nitin Jadhav On Thu, Dec 15, 2022 at 3:16 AM Michael Paquier wrote: > > On Wed, Dec 14, 2022 at 04:54:53PM +0530, Bhar

Re: Inconsistency in reporting checkpointer stats

2022-12-15 Thread Nitin Jadhav
is before the end of the checkpoint. Please share if you have any other ideas. On Wed, Dec 14, 2022 at 4:55 PM Bharath Rupireddy wrote: > > On Wed, Dec 14, 2022 at 1:02 PM Nitin Jadhav > wrote: > > > > Hi, > > > > While working on checkpoint related stuff, I

Inconsistency in reporting checkpointer stats

2022-12-13 Thread Nitin Jadhav
for the same. Please share your thoughts. Thanks & Regards, Nitin Jadhav v1-0001-Fix-inconsistency-in-checkpointer-stats.patch Description: Binary data

Re: Introduce a new view for checkpointer related stats

2022-12-13 Thread Nitin Jadhav
The patch looks good to me. Thanks & Regards, Nitin Jadhav On Fri, Dec 2, 2022 at 11:20 AM Bharath Rupireddy wrote: > > On Wed, Nov 30, 2022 at 5:15 PM Bharath Rupireddy > wrote: > > > > I don't have a strong opinion about changing column names. However, if > >

Re: Report checkpoint progress with pg_stat_progress_checkpoint (was: Report checkpoint progress in server logs)

2022-11-15 Thread Nitin Jadhav
lete message, we should increment all three here, buffers_total, buffers_processed and buffers_written. So the total number of buffers calculated earlier may not always be the same. If this looks good, I will update this in the next patch. Thanks & Regards, Nitin Jadhav On Fri, Nov 4, 2022 at

Re: Skipping schema changes in publication

2022-08-18 Thread Nitin Jadhav
I spent some time on understanding the proposal and the patch. Here are a few comments wrt the test cases. > +ALTER PUBLICATION testpub_reset ADD TABLE pub_sch1.tbl1; > + > +-- Verify that tables associated with the publication are dropped after RESET > +\dRp+ testpub_reset > +ALTER PUBLICATION

Re: Generalize ereport_startup_progress infrastructure

2022-08-10 Thread Nitin Jadhav
ons = startup, postmaster" will enable logging for startup and postmaster operations and disables logging of other long running operations. With this the number of GUCs will be limited to 2 and it is simple and easy for the user. Thanks & Regards, Nitin Jadhav On

Re: Report checkpoint progress with pg_stat_progress_checkpoint (was: Report checkpoint progress in server logs)

2022-07-28 Thread Nitin Jadhav
kpoints. Here are the details. With patch: 2.457 s Without patch: 2.334 s Please share your thoughts. Thanks & Regards, Nitin Jadhav On Thu, Jul 7, 2022 at 5:34 AM Andres Freund wrote: > > Hi, > > On 2022-06-13 19:08:35 +0530, Nitin Jadhav wrote: > > > Have you

Re: Report checkpoint progress with pg_stat_progress_checkpoint (was: Report checkpoint progress in server logs)

2022-06-13 Thread Nitin Jadhav
inting replication slots' > + WHEN 4 THEN 'checkpointing logical replication > snapshot files' > + WHEN 5 THEN 'checkpointing logical rewrite mapping > files' > + WHEN 6 THEN 'checkpointing replication origin' > +

Re: Report checkpoint progress with pg_stat_progress_checkpoint (was: Report checkpoint progress in server logs)

2022-06-13 Thread Nitin Jadhav
to a measurable part of > the size of an empty database: Thank you so much for sharing the detailed analysis. We can remove a few fields which are not so important to make it simple. Thanks & Regards, Nitin Jadhav On Sat, Mar 19, 2022 at 5:45 AM Andres Freund wrote: > > Hi, >

Re: Report checkpoint progress with pg_stat_progress_checkpoint (was: Report checkpoint progress in server logs)

2022-06-06 Thread Nitin Jadhav
Hi, Here is the update patch which fixes the previous comments discussed in this thread. I am sorry for the long gap in the discussion. Kindly let me know if I have missed any of the comments or anything new. Thanks & Regards, Nitin Jadhav On Fri, Mar 18, 2022 at 4:52 PM Nitin Jadhav w

Re: pg_walinspect - a new extension to get raw WAL data and WAL stats

2022-03-18 Thread Nitin Jadhav
_walinspect.c:415:16: error: ‘XLogReaderState’ {aka ‘struct XLogReaderState’} has no member named ‘blocks’ 415 | record->blocks[block_id].hole_length); |^~ make: *** [../../src/Makefile.global:941: pg_walinspect.o] Error 1 Thanks & Regards, Nitin Jadhav On Th

Re: Report checkpoint progress with pg_stat_progress_checkpoint (was: Report checkpoint progress in server logs)

2022-03-18 Thread Nitin Jadhav
t;how the checkpoint is really behaving > right > now". The whole point of having a progress view is to have something dynamic > that reflects the current activity. As of now I will not consider adding this information to the view. If required and nobody opposes having th

Re: Query about time zone patterns in to_char

2022-03-15 Thread Nitin Jadhav
th that, so it appears correct > to me. > > Therefore, I have committed it. Thank you so much. Thanks & Regards, Nitin Jadhav On Tue, Mar 15, 2022 at 2:22 AM Robert Haas wrote: > > On Fri, Jul 9, 2021 at 10:44 AM Tomas Vondra > wrote: > > Yeah, does not seem to be wor

Re: Report checkpoint progress with pg_stat_progress_checkpoint (was: Report checkpoint progress in server logs)

2022-03-14 Thread Nitin Jadhav
operations quickly to take up next requests. If we update this information in the 'flags' field of the view, it says that the current checkpoint is started with CHECKPOINT_IMMEDIATE which is not true. Hence I had thought of adding a new field ('next flags' or 'upcoming flags') which contain all the flag

Re: Report checkpoint progress with pg_stat_progress_checkpoint (was: Report checkpoint progress in server logs)

2022-03-11 Thread Nitin Jadhav
again confuses the user. 'new request' - The value will be set to Yes/No if any new checkpoint requests. This just indicates whether new requests have been made or not. It can not be used to infer other information. Thought? Thanks & Regards, Nitin Jadhav On Fri, Mar 11, 2022 at 3:34 PM Julien

Fix typo in progress reporting doc

2022-03-11 Thread Nitin Jadhav
Hi, I have observed that the table naming conventions used in 'progress-reporting.html' are not consistent across different sections. For some cases "Phases" (Table 28.37. CREATE INDEX Phases) is used and for some cases "phases" (Table 28.35. ANALYZE phases) is used. I have attached a patch to

Re: Report checkpoint progress with pg_stat_progress_checkpoint (was: Report checkpoint progress in server logs)

2022-03-11 Thread Nitin Jadhav
, the user may not understand that it affects current checkpoint behaviour unless the user knows the internals of the checkpoint. How about naming the field to 'throttled' (Yes/ No) since our objective is to show that the current checkpoint is throttled or not. Thanks & Regards, Nitin Jadhav O

Re: Report checkpoint progress with pg_stat_progress_checkpoint (was: Report checkpoint progress in server logs)

2022-03-11 Thread Nitin Jadhav
|| '}' Basically, a separate CASE statement is used to decide whether a comma has to be printed or not, which is done by checking whether the previous flag bit is enabled (so that the appropriate flag has to be displayed) and if any next bits are enabled (So there are some more flags to be displayed). Kindl

Re: Report checkpoint progress with pg_stat_progress_checkpoint (was: Report checkpoint progress in server logs)

2022-03-08 Thread Nitin Jadhav
y only CHECKPOINT_IMMEDIATE flag and all other checkpoint requests done in case of createdb(), dropdb(), etc gets called with CHECKPOINT_IMMEDIATE flag. I have updated this in the v5 patch. Please share your thoughts. Thanks & Regards, Nitin Jadhav On Thu, Mar 3, 2022 at 11:58 PM Julien Rouhaud wrote: &

Re: Report checkpoint progress with pg_stat_progress_checkpoint (was: Report checkpoint progress in server logs)

2022-03-08 Thread Nitin Jadhav
comments. > > I will update in the next patch. The current format matches with the server log message for the checkpoint start in LogCheckpointStart(). Just to be consistent, I have not changed the code. I have taken care of the rest of the comments in v5 patch for which there was clarit

Re: Report checkpoint progress with pg_stat_progress_checkpoint (was: Report checkpoint progress in server logs)

2022-03-08 Thread Nitin Jadhav
because some operation (for which the checkpoint is necessary) is forced the checkpoint". Fixed other comments as per the discussion above. Please find the v5 patch attached and share your thoughts. Thanks & Regards, Nitin Jadhav On Mon, Mar 7, 2022 at 7

Re: Report checkpoint progress with pg_stat_progress_checkpoint (was: Report checkpoint progress in server logs)

2022-03-07 Thread Nitin Jadhav
hink we can add a couple of more information to this view - > start_time for buffer write operation and start_time for buffer sync > operation. These are two very time consuming tasks in a checkpoint and > people would find it useful to know how much time is being taken by > the checkpoi

Re: Report checkpoint progress with pg_stat_progress_checkpoint (was: Report checkpoint progress in server logs)

2022-03-07 Thread Nitin Jadhav
until it's completion > + Wait for completion before returning. > > 13) "removing unneeded or flushing needed logical rewrite mapping files" > + The checkpointer process is currently removing/flushing the logical > > 14) "old WAL files" > +

Re: Report checkpoint progress with pg_stat_progress_checkpoint (was: Report checkpoint progress in server logs)

2022-03-04 Thread Nitin Jadhav
eckpoint may be doing if nothing else happens. > It just feels wrong. You could even use that ckpt_flags info to know > that at least one backend has requested a new checkpoint, if you don't > want to have a number of backends. I think using ckpt_flags to display whether any new requests

Re: pg_walinspect - a new extension to get raw WAL data and WAL stats

2022-03-03 Thread Nitin Jadhav
rst_and_last_lsn :s/pg_get_wal_record_info/pg_wal_record_info :s/pg_get_wal_stats/pg_wal_stats 5) Even 'pg_get_wal_stats' and 'pg_get_wal_stats_till_end_of_wal' can be clubbed as one function. The above comments are trying to simplify the extension APIs and to make it easy for the user to u

Add CHECKPOINT_REQUESTED flag to the log message in LogCheckpointStart()

2022-03-02 Thread Nitin Jadhav
feel we should support this. Thoughts? Please find the patch attached. Thanks & Regards, Nitin Jadhav v1-0001-add-checkpoint_requested-flag-to-the-log-message.patch Description: Binary data

Re: Report checkpoint progress with pg_stat_progress_checkpoint (was: Report checkpoint progress in server logs)

2022-03-02 Thread Nitin Jadhav
If you requested a new checkpoint, it doesn't matter if it's only your backend > that triggered it, another backend or a few other dozen, the result will be > the > same and you have the information that the request has been seen. We could > store just a bool for that but having a num

Refactor statistics collector, backend status reporting and command progress reporting

2022-03-01 Thread Nitin Jadhav
would like to work on the patch. Thanks & Regards, Nitin Jadhav

Re: Report checkpoint progress with pg_stat_progress_checkpoint (was: Report checkpoint progress in server logs)

2022-03-01 Thread Nitin Jadhav
fine. The value '946684800' is equal to ((POSTGRES_EPOCH_JDATE - UNIX_EPOCH_JDATE) * SECS_PER_DAY). I am not sure whether it is good practice to use this way. Kindly share your thoughts. Thanks & Regards, Nitin Jadhav On Mon, Feb 28, 2022 at 6:40 PM Matthias van de Meent wrote: > > On S

Re: Report checkpoint progress with pg_stat_progress_checkpoint (was: Report checkpoint progress in server logs)

2022-02-25 Thread Nitin Jadhav
pointRequested() && >IsCheckpointOnSchedule(progress)) I understand that the checkpointer considers flags as well as the shmem flags and if CHECKPOINT_IMMEDIATE flag is set, it affects the current checkpoint operation (No further delay) but does not change the current f

Re: Report checkpoint progress with pg_stat_progress_checkpoint (was: Report checkpoint progress in server logs)

2022-02-25 Thread Nitin Jadhav
, val); > + } > > Any specific reason for recording the timelineID in checkpoint stats > table? Will this ever change in our case? The timelineID is used to decide whether the current operation is checkpoint or restartpoint. There is a field in the view to display this in

Re: Report checkpoint progress with pg_stat_progress_checkpoint (was: Report checkpoint progress in server logs)

2022-02-25 Thread Nitin Jadhav
end-of-recovery makes very little sense (see upthread) and > should be removed, regardless of whether seperate functions stay. Removed since log based reporting is not part of the current patch. > > diff --git a/src/include/commands/progress.h > > b/src/include/commands/progress.h >

Re: Report checkpoint progress with pg_stat_progress_checkpoint (was: Report checkpoint progress in server logs)

2022-02-24 Thread Nitin Jadhav
ntaining a separate field to show upcoming checkpoint flags is it makes the view complex. Please share your thoughts. Thanks & Regards, On Thu, Feb 24, 2022 at 10:45 PM Matthias van de Meent wrote: > > On Wed, 23 Feb 2022 at 14:28, Nitin Jadhav > wrote: > > > > Sharing t

Re: Report checkpoint progress with pg_stat_progress_checkpoint (was: Report checkpoint progress in server logs)

2022-02-23 Thread Nitin Jadhav
than calling it separately in all the scenarios. I am planning to include log based reporting in the next patch. Even after that if using the same function is not recommended, I am happy to change. Thanks & Regards, Nitin Jadhav On Wed, Feb 23, 2022 at 12:13 AM Matthias van de Meent wrot

Re: Report checkpoint progress with pg_stat_progress_checkpoint (was: Report checkpoint progress in server logs)

2022-02-23 Thread Nitin Jadhav
gt; + else > + checkpoint_progress_update_param(flags, > PROGRESS_CHECKPOINT_KIND, > + > PROGRESS_CHECKPOINT_KIND_UNKNOWN); > + } > +} > > -- > With Regards, > Ashutosh Sharma. > > On Thu, Feb 10, 2022 at 12:23 PM Nitin Jadhav

Re: Report checkpoint progress with pg_stat_progress_checkpoint (was: Report checkpoint progress in server logs)

2022-02-23 Thread Nitin Jadhav
ning to take care in the next patch. --- > If pg_is_in_recovery() is true, then it's a restartpoint, otherwise it's a > restartpoint if the checkpoint's timeline is different from the current > timeline? Fixed. Sharing the v2 patch. Kindly have a look and share your comments. Than

Re: Report checkpoint progress with pg_stat_progress_checkpoint (was: Report checkpoint progress in server logs)

2022-02-21 Thread Nitin Jadhav
stat(my_int64); Thanks for sharing. It works. I will include this in the next patch. On Sat, Feb 19, 2022 at 11:02 AM Julien Rouhaud wrote: > > Hi, > > On Fri, Feb 18, 2022 at 08:07:05PM +0530, Nitin Jadhav wrote: > > > > The backend_pid conta

Re: Report checkpoint progress with pg_stat_progress_checkpoint (was: Report checkpoint progress in server logs)

2022-02-18 Thread Nitin Jadhav
the information while giving a bit more > information for the same memory usage. Can you please describe more about how checkpoint/restartpoint can be confirmed using the timeline id. Thanks & Regards, Nitin Jadhav On Fri, Feb 18, 2022 at 1:13 PM Julien Rouhaud wrote: > >

Re: Report checkpoint progress with pg_stat_progress_checkpoint (was: Report checkpoint progress in server logs)

2022-02-17 Thread Nitin Jadhav
eturns false. But there are some cleanup operations happen as part of the checkpoint. During this scenario, we may get false value for pg_is_in_recovery(). Please refer following piece of code which is present in CreateRestartpoint(). if (!RecoveryInProgress()) replayTLI = XLogCtl->Ins

Re: Report checkpoint progress with pg_stat_progress_checkpoint (was: Report checkpoint progress in server logs)

2022-02-17 Thread Nitin Jadhav
ration of a > >checkpoint_timeout< interval "The checkpoint is started because checkpoint_timeout expired". > > + The checkpoint operation is forced even if no XLOG activity has > > occurred > > + since the last one. > > + Some operation forced a checkpoint.

Re: Report checkpoint progress with pg_stat_progress_checkpoint (was: Report checkpoint progress in server logs)

2022-02-17 Thread Nitin Jadhav
AFAIK we cannot use pg_is_in_recovery() to predict whether it is a checkpoint or restartpoint because if the system exits from recovery mode during restartpoint then any query to pg_stat_progress_checkpoint view will return it as a checkpoint which is ideally not correct. Please correct me if I am

Re: Report checkpoint progress with pg_stat_progress_checkpoint (was: Report checkpoint progress in server logs)

2022-02-16 Thread Nitin Jadhav
nt64 type here rather than Timestamptz for this purpose? 'checkpoint start location' (lsn = uint64) - I feel we cannot use progress parameters for this case. As assigning uint64 to int64 type would be an issue for larger values and can lead to hidden bugs. Thoughts? Thanks & Regards, Nitin Jadhav

Re: Refactor CheckpointWriteDelay()

2022-02-15 Thread Nitin Jadhav
I feel the information related to the pending shutdown request should also be added. Thanks & Regards, Nitin Jadhav On Thu, Feb 10, 2022 at 5:32 PM Bharath Rupireddy wrote: > > Hi, > > Given that CheckpointWriteDelay() is really a hot code path i.e. gets > called for every dir

Re: Report checkpoint progress with pg_stat_progress_checkpoint (was: Report checkpoint progress in server logs)

2022-02-15 Thread Nitin Jadhav
a types, I would like to discuss this in the new thread. On Thu, Feb 10, 2022 at 12:22 PM Nitin Jadhav wrote: > > > > We need at least a trace of the number of buffers to sync (num_to_scan) > > > before the checkpoint start, instead of just emitting the stats at the > > >

Re: Report checkpoint progress with pg_stat_progress_checkpoint (was: Report checkpoint progress in server logs)

2022-02-09 Thread Nitin Jadhav
uring shutdown and end-of-recovery cases as we cannot access pg_stat_progress_checkpoint in those cases. I am working on this to use log_startup_progress_interval mechanism to log the progress in the server logs. Kindly review the patch and share your thoughts. On Fri, Jan 28, 2022 at 12:24 PM Bharath Rupired

Re: Report checkpoint progress in server logs

2022-01-20 Thread Nitin Jadhav
ync > progress locally, and we may need some refactoring here. I agree to provide above mentioned information as part of showing the progress of current checkpoint operation. I am currently looking into the code to know if any other information can be added. Thanks & Regards, Nitin Jadhav

Re: Multi-Column List Partitioning

2021-11-02 Thread Nitin Jadhav
> I noticed that there's no commitfest entry for this. Will you please > add this to the next one? I have added it to Nov commitfest. Thanks & Regards, Nitin Jadhav On Fri, Oct 29, 2021 at 1:40 PM Amit Langote wrote: > > Hi Nitin, > > On Fri, Oct 22, 2021 at 6:48 PM

Re: when the startup process doesn't (logging startup delays)

2021-10-29 Thread Nitin Jadhav
t; how to avoid the race condition otherwise. Right. There is a possibility of race conditions. In that case the above changes look good to me. Thanks & Regards, Nitin Jadhav On Fri, Oct 29, 2021 at 6:10 PM Robert Haas wrote: > > On Fri, Oct 29, 2021 at 7:37

Re: when the startup process doesn't (logging startup delays)

2021-10-29 Thread Nitin Jadhav
e) but the flag is already set to true by the previous timer. Hence the next call to ereport_startup_progress() logs a message. So I feel just setting 'startup_progress_timer_expired' to false in begin_startup_progress_phase() would work. Please correct me if I am wrong. Thanks & Regards, Nitin Jadhav On Thu, Oct 28,

Re: when the startup process doesn't (logging startup delays)

2021-10-19 Thread Nitin Jadhav
or standalone backend. As per earlier discussion we need support for this in the case of startup process and standalone backend. Hence guarded this with '!IsBootstrapProcessingMode()'. Also verified the behaviour in both of the cases. Please correct me if I am wrong. Thanks & Regards, Nitin Jadhav On Mo

Re: when the startup process doesn't (logging startup delays)

2021-09-29 Thread Nitin Jadhav
unction name. So I changed the function name accordingly. > Please make sure to add ereport_startup_progress() as a translation > trigger in src/backend/nls.mk. I have added ereport_startup_progress() under the section GETTEXT_TRIGGERS and GETTEXT_FLAGS in src/backend/nls.mk. Also verified the

Re: when the startup process doesn't (logging startup delays)

2021-09-28 Thread Nitin Jadhav
l take care of this in the next patch. The answer for the question of "I don't understand why you posted the previous version of the patch without testing that it works" is, for the value of -1, the above description was my understanding and for the value of 0, the older versions of the p

Re: when the startup process doesn't (logging startup delays)

2021-09-27 Thread Nitin Jadhav
ed -1. Please find the patch attached. On Thu, Sep 23, 2021 at 9:44 PM Robert Haas wrote: > > On Wed, Sep 22, 2021 at 10:28 AM Nitin Jadhav > wrote: > > Thanks Justin for the detailed explanation. Done the necessary changes. > > Not really. The documentation here does no

Re: when the startup process doesn't (logging startup delays)

2021-09-22 Thread Nitin Jadhav
ink the convention is to for GUC global vars to be initialized with the > same default as in guc.c, so both should be 1, like: > > +int log_startup_progress_interval = 10 * 1000 /* 10sec */ Thanks Justin for the detailed explanation. Done the necessary changes. Please find the upda

Re: when the startup process doesn't (logging startup delays)

2021-09-13 Thread Nitin Jadhav
> > > purposes and didn't initially understand why it wasn't working. I > > > don't think there's any reason it can't work. > > > > As suggested, I have changed it to GUC_UNIT_MS and kept the default > > value to 10s. I would like to know the reason why it can't b

Re: when the startup process doesn't (logging startup delays)

2021-09-07 Thread Nitin Jadhav
GUC_UNIT_MS, > The unit is incorrect here, as that would default to 10ms, contrary to > what the documentation says about 10s. Kindly refer the previous few discussions wrt this point. On Tue, Sep 7, 2021 at 10:58 AM Michael Paquier wrote: > > On Fri, Sep 03, 2021 at 09:23:27PM -0500

Re: when the startup process doesn't (logging startup delays)

2021-09-07 Thread Nitin Jadhav
ess() for this purpose is right. On Tue, Sep 7, 2021 at 10:58 AM Michael Paquier wrote: > > On Fri, Sep 03, 2021 at 09:23:27PM -0500, Justin Pryzby wrote: > > On Fri, Sep 03, 2021 at 01:23:56PM +0530, Nitin Jadhav wrote: > > > Please find the updated patch attached. > > &g

Re: when the startup process doesn't (logging startup delays)

2021-09-03 Thread Nitin Jadhav
atch attached. On Wed, Aug 18, 2021 at 12:23 PM Nitin Jadhav wrote: > > > Anything that's not used in other files should be declared static in > > the file itself, and not present in the header. Once you fix this for > > reset_startup_progress_timeout, the header won't need to

Re: Partition Check not updated when insert into a partition

2021-08-27 Thread Nitin Jadhav
); + } Can we avoid using the extra variable 'partOid' and directly pass 'lfirst_oid(l)' to CacheInvalidateRelcacheByRelid(). Thanks & Regards, Nitin Jadhav On Fri, Aug 27, 2021 at 2:50 PM Amit Langote wrote: > > On Thu, Aug 5, 2021 at 11:32 AM Amit Langote wrote: > > On Wed, Jul 14,

Re: Multi-Column List Partitioning

2021-08-27 Thread Nitin Jadhav
in the next patch? Yes. The code changes related to partition-wise join are in progress. I will handle these in the next patch. Thanks & Regards, Nitin Jadhav On Thu, Aug 26, 2021 at 2:40 AM Zhihong Yu wrote: > > > > On Wed, Aug 25, 2021 at 5:41 AM Nitin Jadhav > wrote: >

Re: when the startup process doesn't (logging startup delays)

2021-08-18 Thread Nitin Jadhav
> Anything that's not used in other files should be declared static in > the file itself, and not present in the header. Once you fix this for > reset_startup_progress_timeout, the header won't need to include > datatype/timestamp.h any more, which is good, because we don't want > header files to

Re: when the startup process doesn't (logging startup delays)

2021-08-12 Thread Nitin Jadhav
is is not the one you wanted me to correct. > You mentioned previously that you would add documentation, but I do > not see it here. Sorry. I missed this. I have added the documentation in the attached patch. On Tue, Aug 10, 2021 at 8:56 PM Robert Haas wrote: > > On Tue, Aug 10, 2021 at 9:28

Re: when the startup process doesn't (logging startup delays)

2021-08-10 Thread Nitin Jadhav
uot;now + interval" in such a case instead of trying to adjust the drift. Please find the updated patch attached. On Tue, Aug 10, 2021 at 1:06 AM Robert Haas wrote: > > On Mon, Aug 9, 2021 at 11:20 AM Nitin Jadhav > wrote: > > Modified the reset_startup_progress_t

Re: when the startup process doesn't (logging startup delays)

2021-08-09 Thread Nitin Jadhav
meout + interval). So the condition (next_timeout < now) will fail and the next_timeout will get reset to 3.5 sec (2.5 + 1) and it continues. Is this behaviour ok or should we set the next_timeout to 3 sec. Please share your thoughts on this. Thanks & Regards, Nitin Jadhav On Thu, Aug 5, 2021 at 7:

Re: when the startup process doesn't (logging startup delays)

2021-08-05 Thread Nitin Jadhav
reinit.c would > include startup.h instead of xlog.h, which seems better, because I > don't think we want any actual xlog operations to happen from within > that file, so better not to be including xlog.h. > > The patch currently lacks documentation. It needs to update config.sgml.

Re: when the startup process doesn't (logging startup delays)

2021-08-03 Thread Nitin Jadhav
& Regards, Nitin Jadhav On Fri, Jul 30, 2021 at 10:40 AM Nitin Jadhav wrote: > > > Thanks. Can you please have a look at what I suggested down toward the > > bottom of > > http://postgr.es/m/ca+tgmoap2wefsktmcgwt9lxuz7y99hnduyshpk7qnfuqb98...@mail.gmail.com > >

Re: when the startup process doesn't (logging startup delays)

2021-07-29 Thread Nitin Jadhav
ces (I feel such scenario will not occur in future also. Even if there is similar operation, it can be distinguished like resetting unlogged relations is distinguished by init and clean. Kindly mention if you can oversee any such scenario), hence the approach you are suggesting will be a best suit. Th

Re: when the startup process doesn't (logging startup delays)

2021-07-29 Thread Nitin Jadhav
> The InitPostgres() case occurs when the server is started in bootstrap > mode (during initdb) or in single-user mode (postgres --single). I do > not see any reason why we shouldn't produce progress messages in at > least the latter case. I suspect that someone who is in the rather > desperate

Re: when the startup process doesn't (logging startup delays)

2021-07-28 Thread Nitin Jadhav
> > > I saw that you fixed it by calling InitStartupProgress() after the > > > walkdir() > > > calls which do pre_sync_fname. So then walkdir is calling > > > LogStartupProgress(STARTUP_PROCESS_OP_FSYNC) even when it's not doing > > > fsync, > > > and then LogStartupProgress() is returning

Re: when the startup process doesn't (logging startup delays)

2021-07-23 Thread Nitin Jadhav
op); > before calling InitStartupProgress(). > > This shows StartupXLOG() calling ResetUnloggedRelations() twice. > Should they both be shown ? If so, maybe they should be distinguished as > here: > > elog(DEBUG1, "resetting unlogged relations: cleanup %d init %d",

Re: when the startup process doesn't (logging startup delays)

2021-07-21 Thread Nitin Jadhav
Kindly let me know if there is any specific reason which would help changing it. > 13) Can we combine LogStartupProgress and CloseStartupProgress? Let's > have a single function LogStartupProgress(StartupProcessOp operation, > const char *path, bool start);, and differentiate t

Re: when the startup process doesn't (logging startup delays)

2021-06-21 Thread Nitin Jadhav
ND > ... That was by mistake and I have corrected it in the attached patch. Thanks & Regards, Nitin Jadhav On Thu, Jun 17, 2021 at 6:22 PM Justin Pryzby wrote: > > + * Codes of the operations performed during startup process > + */ > +typedef enum StartupPro

Re: when the startup process doesn't

2021-06-17 Thread Nitin Jadhav
bility. 5. Updated the comments and some cosmetic changes. Kindly share your comments. Thanks & Regards, Nitin Jadhav On Thu, Jun 10, 2021 at 3:31 PM Justin Pryzby wrote: > > On Thu, Jun 10, 2021 at 03:19:20PM +0530, Nitin Jadhav wrote: > > > > > +

Re: when the startup process doesn't

2021-06-10 Thread Nitin Jadhav
as setting in a user session is not at all useful. But I am confused between PGC_POSTMASTER and PGC_SIGHUP. We should use PGC_SIGHUP if we would like to allow the change during restart after a crash. Otherwise PGC_POSTMASTER would be sufficient. Kindly share your thoughts. Thanks & Reg

  1   2   >