Hi, I would like to resume work on checkpoint progress reporting, previously discussed in [1]. I am starting a new thread to present a revised design addressing the architectural and performance concerns raised there. This is a design proposal. I do not yet have an updated patch.
The earlier patch exposed pg_stat_progress_checkpoint using the generic per-backend progress-reporting machinery. Robert questioned whether that machinery was a good fit for the singleton checkpointer process and suggested dedicated shared-memory state instead. Other concerns included the complexity of the SQL view definition, the overhead of frequent progress updates, and how to report changes in checkpoint pacing. Since that discussion, pg_stat_checkpointer has been introduced, and shared-buffer and SLRU write accounting have been separated. There is also a newer proposal to expose the last checkpoint's timestamp and duration [2]. These improvements are useful, but serve a different purpose from reporting the phase and work completed versus total for an ongoing checkpoint or restartpoint. I propose the following approach: - Maintain dedicated, typed live-progress state owned by the checkpointer, embedded in or associated with CheckpointerShmemStruct. Keep this separate from cumulative statistics in PgStat_CheckpointerStats. - Expose the state through a C accessor backing pg_stat_progress_checkpoint. Return appropriately typed values and perform phase/flag decoding in C, keeping the SQL view definition small. - Initially report the checkpointer PID, checkpoint/restartpoint type, start time, current phase, shared-buffer progress, and file-sync progress. - Distinguish the initial checkpoint causes from its current pacing behavior. In particular, a pending fast-checkpoint request can cause the running checkpoint to stop delaying writes. The view should make that behavior understandable without requiring users to interpret another checkpoint's flags. - Publish counters periodically during checkpoint processing, with additional updates at phase boundaries and completion, avoiding synchronized updates for every buffer. The synchronization mechanism and publication interval would be worked out in the implementation and evaluated with benchmarks. The implementation would define how readers obtain a consistent snapshot across phase transitions and consecutive checkpoints, and how active state is cleared on completion or error. I propose returning one row while a checkpoint or restartpoint is active, and no rows while idle. I would leave progress logging during startup/shutdown outside the initial patch. Does dedicated shared-memory state, separate from cumulative statistics, seem a reasonable direction for this work? In particular, are there objections to this approach or concerns about the proposed reporting semantics that should be addressed before preparing a revised patch? [1] Previous checkpoint progress discussion: https://www.postgresql.org/message-id/flat/CAMm1aWZDxnNpPcGnom9sWLEKX7nuRzym4%3DhPYpnp1DWYSCcSPw%40mail.gmail.com [2] Expose checkpoint timestamp and duration in pg_stat_checkpointer: https://commitfest.postgresql.org/patch/6307/ Best Regards, Nitin Jadhav Microsoft
