On Thu, Jul 6, 2023 at 2:28 PM Masahiko Sawada wrote:
>
> On Thu, Jul 6, 2023 at 11:15 AM Michael Paquier wrote:
> >
> > On Thu, Jul 06, 2023 at 11:07:14AM +0900, Masahiko Sawada wrote:
> > > Thank you for updating the patch. It looks good to me too. I've
> > > updated the commit message.
> >
> >
On Thu, Jul 6, 2023 at 11:15 AM Michael Paquier wrote:
>
> On Thu, Jul 06, 2023 at 11:07:14AM +0900, Masahiko Sawada wrote:
> > Thank you for updating the patch. It looks good to me too. I've
> > updated the commit message.
>
> Thanks. I was planning to review this patch today and/or tomorrow now
On Thu, Jul 06, 2023 at 11:07:14AM +0900, Masahiko Sawada wrote:
> Thank you for updating the patch. It looks good to me too. I've
> updated the commit message.
Thanks. I was planning to review this patch today and/or tomorrow now
that my stack of things to do is getting slightly lower (registere
On Wed, Apr 12, 2023 at 9:22 PM Imseih (AWS), Sami wrote:
>
> > This should be OK (also checked the code paths where the reports are
> > added). Note that the patch needed a few adjustments for its
> > indentation.
>
> Thanks for the formatting corrections! This looks good to me.
Thank you for up
> This should be OK (also checked the code paths where the reports are
> added). Note that the patch needed a few adjustments for its
> indentation.
Thanks for the formatting corrections! This looks good to me.
--
Sami
On Mon, Apr 10, 2023 at 07:20:42PM +, Imseih (AWS), Sami wrote:
> See v28 addressing the comments.
This should be OK (also checked the code paths where the reports are
added). Note that the patch needed a few adjustments for its
indentation.
--
Michael
From 9267342ba2a1b8b120efaa98871b736a5bb
Hi,
On 2023-04-10 08:14:18 +0900, Michael Paquier wrote:
> On Fri, Apr 07, 2023 at 12:01:17PM -0700, Andres Freund wrote:
> > Why would it mean that? Parallel workers are updated together with the
> > leader,
> > so there's no compatibility issue?
>
> My point is that the callback system would s
> + case 'P': /* Parallel progress reporting */
I kept this comment as-is but inside case code block I added
more comments. This is to avoid cluttering up the one-liner comment.
> + * Increase and report the number of index scans. Also, we reset the progress
> + * counters.
> The counters rese
On Fri, Apr 07, 2023 at 12:01:17PM -0700, Andres Freund wrote:
> Why would it mean that? Parallel workers are updated together with the leader,
> so there's no compatibility issue?
My point is that the callback system would still need to be maintained
in a stable branch, and, while useful, it coul
On Fri, Apr 07, 2023 at 07:27:17PM +, Imseih (AWS), Sami wrote:
> If called by a worker, it will send a 'P' message to the front end
> passing both the progress index, i.e. PROGRESS_VACUUM_INDEXES_PROCESSED
> And the value to increment by, i.e. 1 for index vacuum progress.
>
> With that, the a
> The arguments of pgstat_progress_update_param() would be given by the
> worker directly as components of the 'P' message. It seems to me that
> this approach would have the simplicity to not require the setup of a
> shmem area for the extra counters, and there would be no need for a
> callback. H
Hi,
On 2023-04-06 12:28:04 +0900, Michael Paquier wrote:
> As some say, the introduction of a new message type in pqmq.c would be
> basically a one-way door, because we'd have to maintain it in a stable
> branch.
Why would it mean that? Parallel workers are updated together with the leader,
so th
On Thu, Apr 06, 2023 at 03:14:20PM +, Imseih (AWS), Sami wrote:
>> Could it be worth thinking about a different design where
>> the value incremented and the parameters of
>> pgstat_progress_update_param() are passed through the 'P' message
>> instead?
>
> I am not sure how this is different t
> As one thing,
> for example, it introduces a dependency to parallel.h to do progress
> reporting without touching at backend_progress.h.
Containing the logic in backend_progress.h is a reasonable point
from a maintenance standpoint.
We can create a new function in backend_progress.h called
p
On Wed, Apr 05, 2023 at 02:31:54PM +, Imseih (AWS), Sami wrote:
>> That seems a valid argument. I was thinking that such an asynchronous
>> state update mechanism would be a good infrastructure for progress
>> reporting of parallel operations. It might be worth considering to use
>> it in more
> > The key point of the patch is here. From what I understand based on
> > the information of the thread, this is used as a way to make the
> progress reporting done by the leader more responsive so as we'd
> > update the index counters each time the leader is poked at with a 'P'
> > message by on
On Wed, Apr 5, 2023 at 4:47 PM Michael Paquier wrote:
>
> On Fri, Feb 24, 2023 at 03:16:10PM +0900, Masahiko Sawada wrote:
> > Thanks! It looks good to me so I've marked it as Ready for Committer.
>
> + case 'P': /* Parallel progress reporting */
> + {
> +
On Fri, Feb 24, 2023 at 03:16:10PM +0900, Masahiko Sawada wrote:
> Thanks! It looks good to me so I've marked it as Ready for Committer.
+ case 'P': /* Parallel progress reporting */
+ {
+ /* Call the progress reporting callback */
+ Asser
On Tue, Feb 21, 2023 at 1:48 AM Imseih (AWS), Sami wrote:
>
> Thanks!
>
> > I think PROGRESS_VACUUM_INDEXES_TOTAL and
> > PROGRESS_VACUUM_INDEXES_PROCESSED are better for consistency. The rest
> > looks good to me.
>
> Took care of that in v25.
Thanks! It looks good to me so I've marked it a
Thanks!
> I think PROGRESS_VACUUM_INDEXES_TOTAL and
> PROGRESS_VACUUM_INDEXES_PROCESSED are better for consistency. The rest
> looks good to me.
Took care of that in v25.
Regards
--
Sami Imseih
Amazon Web Services
v25-0001-Add-2-new-columns-to-pg_stat_progress_vacuum.-Th.patch
Descript
On Sat, Feb 18, 2023 at 11:46 AM Imseih (AWS), Sami wrote:
>
> Thanks for the review!
>
> >+
> >+ ParallelVacuumFinish
> >+ Waiting for parallel vacuum workers to finish index
> >vacuum.
> >+
>
> >This change is out-of-date.
>
> That was an oversight. Th
Thanks for the review!
>+
>+ ParallelVacuumFinish
>+ Waiting for parallel vacuum workers to finish index
>vacuum.
>+
>This change is out-of-date.
That was an oversight. Thanks for catching.
>Total number of indexes that will be vacuumed or cleaned
On Wed, Feb 8, 2023 at 11:03 AM Imseih (AWS), Sami wrote:
>
> Hi,
>
> Thanks for your reply!
>
> I addressed the latest comments in v23.
>
> 1/ cleaned up the asserts as discussed.
> 2/ used pq_putmessage to send the message on index scan completion.
Thank you for updating the patch! Here are so
Hi,
Thanks for your reply!
I addressed the latest comments in v23.
1/ cleaned up the asserts as discussed.
2/ used pq_putmessage to send the message on index scan completion.
Thanks
--
Sami Imseih
Amazon Web Services (AWS)
v23-0001-Add-2-new-columns-to-pg_stat_progress_vacuum.-Th.patch
Desc
On Fri, Jan 20, 2023 at 11:38 PM Imseih (AWS), Sami wrote:
>
> >Number of indexes that will be vacuumed or cleaned up. This counter only
> >advances when the phase is vacuuming indexes or cleaning up indexes.
>
> I agree, this reads better.
>
> ---
> -/* Report that we are
>Number of indexes that will be vacuumed or cleaned up. This counter only
>advances when the phase is vacuuming indexes or cleaning up indexes.
I agree, this reads better.
---
-/* Report that we are now vacuuming indexes */
-pgstat_progress_update_param(PROGRES
On Thu, Jan 12, 2023 at 11:02 PM Imseih (AWS), Sami wrote:
>
> Thanks for the feedback and I apologize for the delay in response.
>
> >I think the problem here is that you're basically trying to work around
> > the
> >lack of an asynchronous state update mechanism between leader and
> >
Thanks for the feedback and I apologize for the delay in response.
>I think the problem here is that you're basically trying to work around the
>lack of an asynchronous state update mechanism between leader and workers.
> The
>workaround is to add a lot of different places that poll w
Hi,
On 2023-01-07 01:59:40 +, Imseih (AWS), Sami wrote:
> --- a/src/backend/access/nbtree/nbtree.c
> +++ b/src/backend/access/nbtree/nbtree.c
> @@ -998,6 +998,8 @@ btvacuumscan(IndexVacuumInfo *info, IndexBulkDeleteResult
> *stats,
> if (info->report_progress)
>
> My point is whether we should show
> indexes_total throughout the vacuum execution (even also in not
> relevant phases such as heap scanning/vacuum/truncation).
That is a good point. We should only show indexes_total
and indexes_completed only during the relevant phases.
V21 addresses this alo
On Fri, Jan 6, 2023 at 12:07 PM Imseih (AWS), Sami wrote:
>
> >Similar to above three cases, vacuum can bypass index vacuuming if
> >there are almost zero TIDs. Should we set indexes_total to 0 in this
> >case too? If so, I think we can set both indexes_total and
> >indexes_complet
>Similar to above three cases, vacuum can bypass index vacuuming if
>there are almost zero TIDs. Should we set indexes_total to 0 in this
>case too? If so, I think we can set both indexes_total and
>indexes_completed at the beginning of the index vacuuming/cleanup and
>reset the
On Thu, Jan 5, 2023 at 4:24 AM Imseih (AWS), Sami wrote:
>
> Thanks for the review!
>
> Addressed the comments.
>
> > "Increment the indexes completed." (dot at the end) instead?
>
> Used the commenting format being used in other places in this
> file with an inclusion of a double-dash. i.,e.
> /*
Thanks for the review!
Addressed the comments.
> "Increment the indexes completed." (dot at the end) instead?
Used the commenting format being used in other places in this
file with an inclusion of a double-dash. i.,e.
/* Wraparound emergency -- end current index scan */
> It seems to me that "
Hi,
On 1/3/23 5:46 PM, Imseih (AWS), Sami wrote:
cirrus-ci.com/task/4557389261701120
I earlier compiled without building with --enable-cassert,
which is why the compilation errors did not produce on my
buid.
Fixed in v19.
Thanks
Thanks for the updated patch!
Some comments about it:
+
+
> cirrus-ci.com/task/4557389261701120
I earlier compiled without building with --enable-cassert,
which is why the compilation errors did not produce on my
buid.
Fixed in v19.
Thanks
--
Sami Imseih
Amazon Web Services: https://aws.amazon.com
v19-0001-Add-2-new-columns-to-pg_stat_progres
On Mon, 2 Jan 2023 at 10:04, Imseih (AWS), Sami wrote:
>
> >cfbot is complaining that this patch no longer applies. Sami, would you
> >mind rebasing it?
>
> Rebased patch attached.
CFBot shows some compilation errors as in [1], please post an updated
version for the same:
[07:01:58.889]
>cfbot is complaining that this patch no longer applies. Sami, would you
>mind rebasing it?
Rebased patch attached.
--
Sami Imseih
Amazon Web Services: https://aws.amazon.com
v18-0001-Add-2-new-columns-to-pg_stat_progress_vacuum.-Th.patch
Description: v18-0001-Add-2-new-columns-to
On Wed, Dec 14, 2022 at 05:09:46AM +, Imseih (AWS), Sami wrote:
> Attached version addresses the above and the previous comments.
cfbot is complaining that this patch no longer applies. Sami, would you
mind rebasing it?
--
Nathan Bossart
Amazon Web Services: https://aws.amazon.com
>First of all, I don't think we need to declare ParallelVacuumProgress
>in vacuum.c since it's set and used only in vacuumparallel.c. But I
>don't even think it's a good idea to declare it in vacuumparallel.c as
>a static variable. The primary reason is that it adds things we need
>
On Tue, Dec 13, 2022 at 1:40 PM Imseih (AWS), Sami wrote:
>
> Thanks for the feedback. I agree with the feedback, except
> for
>
> >need to have ParallelVacuumProgress. I see
> >parallel_vacuum_update_progress() uses this value but I think it's
> >better to pass ParallelVacuumState to
Thanks for the feedback. I agree with the feedback, except
for
>need to have ParallelVacuumProgress. I see
>parallel_vacuum_update_progress() uses this value but I think it's
>better to pass ParallelVacuumState to via IndexVacuumInfo.
I was trying to avoid passing a pointer to
Parall
Hi,
Thank you for updating the patch!
On Tue, Nov 29, 2022 at 8:57 AM Imseih (AWS), Sami wrote:
>
> > I think that indexes_total should be 0 also when INDEX_CLEANUP is off.
>
> Patch updated for handling of INDEX_CLEANUP = off, with an update to
> the documentation as well.
>
> >I think we
> I think that indexes_total should be 0 also when INDEX_CLEANUP is off.
Patch updated for handling of INDEX_CLEANUP = off, with an update to
the documentation as well.
>I think we don't need to reset it at the end of index vacuuming. There
>is a small window before switching to the nex
On Sat, Nov 12, 2022 at 4:10 AM Imseih (AWS), Sami wrote:
>
> >I don't think any of these progress callbacks should be done while
> > pinning a
> >buffer and ...
>
> Good point.
>
> >I also don't understand why info->parallel_progress_callback exists?
> > It's only
> >set to para
>I don't think any of these progress callbacks should be done while pinning
> a
>buffer and ...
Good point.
>I also don't understand why info->parallel_progress_callback exists? It's
> only
>set to parallel_vacuum_progress_report(). Why make this stuff more
> expensive
>tha
Hi,
On 2022-11-04 13:27:34 +, Imseih (AWS), Sami wrote:
> diff --git a/src/backend/access/gin/ginvacuum.c
> b/src/backend/access/gin/ginvacuum.c
> index b4fa5f6bf8..3d5e4600dc 100644
> --- a/src/backend/access/gin/ginvacuum.c
> +++ b/src/backend/access/gin/ginvacuum.c
> @@ -633,6 +633,9 @@ gi
On Thu, Nov 3, 2022 at 1:52 AM Imseih (AWS), Sami wrote:
>
> Attached is v13-0001--Show-progress-for-index-vacuums.patch which addresses
> the latest comments.
Thank you for updating the patch!
> 4/ Went back to calling parallel_vacuum_progress_report inside
> WaitForParallelWorkersToFinish to c
Resubmitting patches with proper format.
Thanks
Sami Imseih
Amazon Web Services (AWS)
v14-0002-Function-to-return-currently-vacuumed-or-cleaned.patch
Description: v14-0002-Function-to-return-currently-vacuumed-or-cleaned.patch
v14-0001-Add-2-new-columns-to-pg_stat_progress_vacuum.-Th.pat
2022年11月3日(木) 1:52 Imseih (AWS), Sami :
>
> Attached is v13-0001--Show-progress-for-index-vacuums.patch which addresses
> the latest comments. The main changes are:
>
> 1/ Call the parallel_vacuum_progress_report inside the AMs rather than
> vacuum_delay_point.
>
> 2/ A Boolean when set to True in
Attached is v13-0001--Show-progress-for-index-vacuums.patch which addresses
the latest comments. The main changes are:
1/ Call the parallel_vacuum_progress_report inside the AMs rather than
vacuum_delay_point.
2/ A Boolean when set to True in vacuumparallel.c will be used to determine if
callin
Thank you for the feedback!
>While it seems to be a good idea to have the atomic counter for the
>number of indexes completed, I think we should not use the global
>variable referencing the counter as follow:
>+static pg_atomic_uint32 *index_vacuum_completed = NULL;
>:
>+v
On Tue, Oct 11, 2022 at 10:50 PM Imseih (AWS), Sami wrote:
>
> >One idea would be to add a flag, say report_parallel_vacuum_progress,
> >to IndexVacuumInfo struct and expect index AM to check and update the
> >parallel index vacuum progress, say every 1GB blocks processed. The
> >f
>One idea would be to add a flag, say report_parallel_vacuum_progress,
>to IndexVacuumInfo struct and expect index AM to check and update the
>parallel index vacuum progress, say every 1GB blocks processed. The
>flag is true only when the leader process is vacuuming an index.
>
> One idea would be to add a flag, say report_parallel_vacuum_progress,
> to IndexVacuumInfo struct and expect index AM to check and update the
> parallel index vacuum progress, say every 1GB blocks processed. The
> flag is true only when the leader process is vacuuming an index.
Sorry for the lon
This entry has been waiting on author input for a while (our current
threshold is roughly two weeks), so I've marked it Returned with
Feedback.
Once you think the patchset is ready for review again, you (or any
interested party) can resurrect the patch entry by visiting
https://commitfest.pos
On Mon, Jun 6, 2022 at 11:42 PM Robert Haas wrote:
>
> On Thu, May 26, 2022 at 11:43 AM Masahiko Sawada
> wrote:
> > Another idea I came up with is that we can wait for all index vacuums
> > to finish while checking and updating the progress information, and
> > then calls WaitForParallelWorkers
On Thu, May 26, 2022 at 11:43 AM Masahiko Sawada wrote:
> Another idea I came up with is that we can wait for all index vacuums
> to finish while checking and updating the progress information, and
> then calls WaitForParallelWorkersToFinish after confirming all index
> status became COMPLETED. Th
On Fri, May 27, 2022 at 10:52 AM Imseih (AWS), Sami wrote:
>
> >Another idea I came up with is that we can wait for all index vacuums
> >to finish while checking and updating the progress information, and
> >then calls WaitForParallelWorkersToFinish after confirming all index
> >st
>Another idea I came up with is that we can wait for all index vacuums
>to finish while checking and updating the progress information, and
>then calls WaitForParallelWorkersToFinish after confirming all index
>status became COMPLETED. That way, we don’t need to change the
>para
On Fri, May 6, 2022 at 4:26 AM Imseih (AWS), Sami wrote:
>
> Thank you for the feedback!
>
> >I think we can pass the progress update function to
> > WaitForParallelWorkersToFinish(), which seems simpler. And we can call
>
> Directly passing the callback to WaitForParallelWorkersToFinish
> w
Thank you for the feedback!
>I think we can pass the progress update function to
> WaitForParallelWorkersToFinish(), which seems simpler. And we can call
Directly passing the callback to WaitForParallelWorkersToFinish
will require us to modify the function signature.
To me, it seemed simpl
On Thu, Apr 14, 2022 at 10:32 AM Imseih (AWS), Sami wrote:
>
> Taking the above feedback, I modified the patches
> and I believe this approach should be acceptable.
>
> For now, I also reduced the scope to only exposing
> Indexes_total and indexes_completed in
> pg_stat_progress_vacuum. I will cre
It looks like this patch got feedback from Andres and Robert with some
significant design change recommendations. I'm marking the patch
Returned with Feedback. Feel free to add it back to a future
commitfest when a new version is ready.
On Thu, Apr 7, 2022 at 10:20 PM Robert Haas wrote:
>
> On Wed, Apr 6, 2022 at 5:22 PM Imseih (AWS), Sami wrote:
> > >At the beginning of a parallel operation, we allocate a chunk of>
> > >dynamic shared memory which persists even after some or all workers
> > >have exited. It's only t
On Wed, Apr 6, 2022 at 5:22 PM Imseih (AWS), Sami wrote:
> >At the beginning of a parallel operation, we allocate a chunk of>
> >dynamic shared memory which persists even after some or all workers
> >have exited. It's only torn down at the end of the parallel operation.
> >That see
>At the beginning of a parallel operation, we allocate a chunk of>
>dynamic shared memory which persists even after some or all workers
>have exited. It's only torn down at the end of the parallel operation.
>That seems like the appropriate place to be storing any kind of data
>
>Can't the progress data trivially be inferred by the fact that the worker
>completed?
Yes, at some point, this idea was experimented with in
0004-Expose-progress-for-the-vacuuming-indexes-cleanup-ph.patch.
This patch did the calculation in system_views.sql
However, the view is complex an
On 2022-04-05 16:42:28 +, Imseih (AWS), Sami wrote:
> >Why isn't the obvious thing to do here to provide a way to associate
> > workers
> >with their leaders in shared memory, but to use the existing progress
> > fields
> >to report progress? Then, when querying progress, the lead
On Tue, Apr 5, 2022 at 12:42 PM Imseih (AWS), Sami wrote:
> >Why isn't the obvious thing to do here to provide a way to associate
> > workers
> >with their leaders in shared memory, but to use the existing progress
> > fields
> >to report progress? Then, when querying progress, the l
>I nevertheless think that's not acceptable. The whole premise of the
> progress
>reporting infrastructure is to be low overhead. It's OK to require locking
> to
>initialize parallel progress reporting, it's definitely not ok to require
>locking to report progress.
Fair point.
>
Hi,
On 2022-03-29 12:25:52 +, Imseih (AWS), Sami wrote:
> > I think that's an absolute no-go. Adding locking to progress reporting,
> > particularly a single central lwlock, is going to *vastly* increase the
> > overhead incurred by progress reporting.
>
> Sorry for the late reply.
>
> The u
> I think that's an absolute no-go. Adding locking to progress reporting,
> particularly a single central lwlock, is going to *vastly* increase the
> overhead incurred by progress reporting.
Sorry for the late reply.
The usage of the shared memory will be limited
to PARALLEL maintenance operation
Sorry for the late reply.
> additional complexity and a possible lag of progress updates. So if we
> go with the current approach, I think we need to make sure enough (and
> not too many) hash table entries.
The hash table can be set 4 times the size of
max_worker_processes which should give mor
On Wed, Mar 23, 2022 at 6:57 AM Imseih (AWS), Sami wrote:
>
> >Can the leader pass a callback that checks PVIndStats to ambulkdelete
> >an amvacuumcleanup callbacks? I think that in the passed callback, the
> >leader checks if the number of processed indexes and updates its
> >prog
>Can the leader pass a callback that checks PVIndStats to ambulkdelete
>an amvacuumcleanup callbacks? I think that in the passed callback, the
>leader checks if the number of processed indexes and updates its
>progress information if the current progress needs to be updated.
Thanks
On Tue, Mar 22, 2022 at 4:27 PM Imseih (AWS), Sami wrote:
>
> >BTW have we discussed another idea I mentioned before that we have the
> >leader process periodically check the number of completed indexes and
> >advertise it in its progress information? I'm not sure which one is
> >b
>BTW have we discussed another idea I mentioned before that we have the
>leader process periodically check the number of completed indexes and
>advertise it in its progress information? I'm not sure which one is
>better but this idea would require only changes of vacuum code and
>
Hi,
On 2022-03-16 21:47:49 +, Imseih (AWS), Sami wrote:
> From 85c47dfb3bb72f764b9052e74a7282c19ebd9898 Mon Sep 17 00:00:00 2001
> From: "Sami Imseih (AWS)"
> Date: Wed, 16 Mar 2022 20:39:52 +
> Subject: [PATCH 1/1] Add infrastructure for parallel progress reporting
>
> Infrastructure to
Sorry for the late reply.
On Tue, Mar 15, 2022 at 1:20 AM Imseih (AWS), Sami wrote:
>
> >I'm still unsure the current design of 0001 patch is better than other
> >approaches we’ve discussed. Even users who don't use parallel vacuum
> >are forced to allocate shared memory for index vac
On Wed, Mar 16, 2022 at 09:47:49PM +, Imseih (AWS), Sami wrote:
> Spoke to Nathan offline and fixed some more comments/nitpicks in the patch.
I don't have any substantial comments for v9, so I think this can be marked
as ready-for-committer. However, we probably should first see whether
Sawad
>I'm still unsure the current design of 0001 patch is better than other
>approaches we’ve discussed. Even users who don't use parallel vacuum
>are forced to allocate shared memory for index vacuum progress, with
>GetMaxBackends() entries from the beginning. Also, it’s likely to
>
On Sat, Mar 12, 2022 at 4:00 PM Imseih (AWS), Sami wrote:
>
> > nitpick: Can we remove the extra spaces in the parentheses?
>
> fixed
>
> > What does it mean if there isn't an entry in the map? Is this actually
> > expected, or should we ERROR instead?
>
> I cleaned up the code here and added com
On Sat, Mar 12, 2022 at 07:00:06AM +, Imseih (AWS), Sami wrote:
>> nitpick: Can we remove the extra spaces in the parentheses?
>
> fixed
>
>> What does it mean if there isn't an entry in the map? Is this actually
>> expected, or should we ERROR instead?
>
> I cleaned up the code here and ad
On Thu, Mar 10, 2022 at 09:30:57PM +, Imseih (AWS), Sami wrote:
> Attached v4 which includes accounting for the hash size on startup, removal
> of the no longer needed comment in pgstatfuncs.c and a change in both
> code/docs to only reset the indexes_total to 0 when failsafe is triggered.
T
> > BTreeShmemInit();
> > SyncScanShmemInit();
> > AsyncShmemInit();
> > + vacuum_worker_init();
> > Don't we also need to add the size of the hash table to
> > CalculateShmemSize()?
> No, ShmemInitHash takes the min and max size o
>I took a look at the latest patch set.
>+
>+ indexes_total bigint
>+
>+
>+ The number of indexes to be processed in the
>+ vacuuming indexes
>+ or cleaning up indexes phase. It is set to
>+ 0 when vacuum is not in
I took a look at the latest patch set.
+
+ indexes_total bigint
+
+
+ The number of indexes to be processed in the
+ vacuuming indexes
+ or cleaning up indexes phase. It is set to
+ 0 when vacuum is not in any of these phases.
+
Could we avo
>I think if it's a better approach we can do that including adding a
>new infrastructure for it.
+1 This is a beneficial idea, especially to other progress reporting, but I see
this as a separate thread targeting the next major version.
On Wed, Mar 9, 2022 at 11:35 AM Imseih (AWS), Sami wrote:
>
> >Indeed.
>
> >It might have already been discussed but other than using a new shmem
> >hash for parallel vacuum, I wonder if we can allow workers to change
> >the leader’s progress information. It would break the assumpt
>Indeed.
>It might have already been discussed but other than using a new shmem
>hash for parallel vacuum, I wonder if we can allow workers to change
>the leader’s progress information. It would break the assumption that
>the backend status entry is modified by its own backend,
On Wed, Mar 9, 2022 at 12:41 AM Imseih (AWS), Sami wrote:
>
> ++/*
> ++ * vacuum_worker_init --- initialize this module's shared memory hash
> ++ * to track the progress of a vacuum worker
> ++ */
> ++void
> ++vacuum_worker_init(void)
> ++{
> ++ HASHCTL in
++/*
++ * vacuum_worker_init --- initialize this module's shared memory hash
++ * to track the progress of a vacuum worker
++ */
++void
++vacuum_worker_init(void)
++{
++ HASHCTL info;
++ longmax_table_size = GetMaxBackends();
++
++
On Thu, Mar 3, 2022 at 2:08 PM Imseih (AWS), Sami wrote:
>
> >>> If the failsafe kicks in midway through a vacuum, the number
> > indexes_total will not be reset to 0. If INDEX_CLEANUP is turned off, then
> > the value will be 0 at the start of the vacuum.
> >>
> >> The way that this
> On Wed, Feb 23, 2022 at 10:02 AM Imseih (AWS), Sami
wrote:
>> If the failsafe kicks in midway through a vacuum, the number
indexes_total will not be reset to 0. If INDEX_CLEANUP is turned off, then the
value will be 0 at the start of the vacuum.
>
> The way that this works wit
On Wed, Feb 23, 2022 at 10:41:36AM -0800, Peter Geoghegan wrote:
> On Wed, Feb 23, 2022 at 10:02 AM Imseih (AWS), Sami
> wrote:
>> If the failsafe kicks in midway through a vacuum, the number indexes_total
>> will not be reset to 0. If INDEX_CLEANUP is turned off, then the value will
>> be 0 at
On Wed, Feb 23, 2022 at 10:02 AM Imseih (AWS), Sami wrote:
> If the failsafe kicks in midway through a vacuum, the number indexes_total
> will not be reset to 0. If INDEX_CLEANUP is turned off, then the value will
> be 0 at the start of the vacuum.
The way that this works with num_index_scans i
+
+
+ indexes_total bigint
+
+
+ The number of indexes to be processed in the
+ vacuuming indexes or cleaning up
indexes phase
+ of the vacuum.
+
+
+
+
+
+ indexes_pro
On Mon, Feb 21, 2022 at 07:03:39PM +, Imseih (AWS), Sami wrote:
> Sending again with patch files renamed to ensure correct apply order.
I haven't had a chance to test this too much, but I did look through the
patch set and have a couple of small comments.
+
+
+ indexes_total
On 1/12/22, 1:28 PM, "Bossart, Nathan" wrote:
On 1/11/22, 11:46 PM, "Masahiko Sawada" wrote:
> Regarding the new pg_stat_progress_vacuum_index view, why do we need
> to have a separate view? Users will have to check two views. If this
> view is expected to be used together with a
1 - 100 of 120 matches
Mail list logo
