On Tue, Aug 15, 2023 at 09:14:15AM +0900, Masahiro Ikeda wrote:
> OK. I'll make a new patch and start a new thread.
Cool, thanks!
--
Michael
signature.asc
Description: PGP signature
On 2023-08-14 18:26, Michael Paquier wrote:
On Mon, Aug 14, 2023 at 05:55:42PM +0900, Masahiro Ikeda wrote:
I'm thinking a name like "contrib name + description summary" would
be nice. The "contrib name" is namespace-like and the "description
summary"
is the same as the name of the waiting
On Mon, Aug 14, 2023 at 05:55:42PM +0900, Masahiro Ikeda wrote:
> I'm thinking a name like "contrib name + description summary" would
> be nice. The "contrib name" is namespace-like and the "description summary"
> is the same as the name of the waiting event name in core. For example,
>
On 2023-08-14 15:28, Michael Paquier wrote:
On Mon, Aug 14, 2023 at 12:31:05PM +0900, Masahiro Ikeda wrote:
Thanks! I confirmed the changes, and all tests passed.
Okay, cool. I got some extra time today and applied that, with a few
more tweaks.
Thanks for applying master branch!
This
On Mon, Aug 14, 2023 at 12:31:05PM +0900, Masahiro Ikeda wrote:
> Thanks! I confirmed the changes, and all tests passed.
Okay, cool. I got some extra time today and applied that, with a few
more tweaks.
--
Michael
signature.asc
Description: PGP signature
On 2023-08-14 08:06, Michael Paquier wrote:
On Thu, Aug 10, 2023 at 05:37:55PM +0900, Michael Paquier wrote:
This looks correct, but perhaps we need to think harder about the
custom event names and define a convention when more of this stuff is
added to the core modules.
Okay, I have put my
On Thu, Aug 10, 2023 at 05:37:55PM +0900, Michael Paquier wrote:
> This looks correct, but perhaps we need to think harder about the
> custom event names and define a convention when more of this stuff is
> added to the core modules.
Okay, I have put my hands on that, fixing a couple of typos,
On Thu, Aug 10, 2023 at 01:08:39PM +0900, Masahiro Ikeda wrote:
> In addition, I change the followings:
> * update about custom wait events in sgml. we don't need to use
> shmem_startup_hook.
> * change the hash names for readability.
> (ex. WaitEventExtensionNameHash ->
Hi,
Thanks for your comments about the v2 patches. I updated to v3 patches.
The main changes are:
* remove the AddinShmemInitLock assertion
* add the new lock (WaitEventExtensionLock) to wait_event_names.txt
* change "static const int wee_hash_XXX_size" to "#define XXX"
* simplify worker_spi. I
On Wed, Aug 09, 2023 at 08:10:42PM +0900, Masahiro Ikeda wrote:
> * create second hash table to find a event id from a name to
> identify uniquness. It enable extensions which don't use share
> memory for their use to define custom wait events because
> WaitEventExtensionNew() will not
Hi,
On 2023-08-09 20:10:42 +0900, Masahiro Ikeda wrote:
> * Is there any way to not force extensions that don't use shared
> memory for their use like dblink to acquire AddinShmemInitLock?;
I think the caller shouldn't need to do deal with AddinShmemInitLock at all.
Greetings,
Andres Freund
Hi,
Thanks for your comments to v1 patch.
I made v2 patch. Main changes are
* change to NAMEDATALEN
* change to use dynahash from dshash
* remove worker_spi_init()
* create second hash table to find a event id from a name to
identify uniquness. It enable extensions which don't use share
Hi,
On 2023-08-09 08:03:29 +0900, Michael Paquier wrote:
> On Tue, Aug 08, 2023 at 03:59:54PM -0700, Andres Freund wrote:
> > On 2023-08-08 08:54:10 +0900, Michael Paquier wrote:
> >> - WaitEventExtensionShmemInit() should gain a dshash_create(), to make
> >> sure that the shared table is around,
On Tue, Aug 08, 2023 at 03:59:54PM -0700, Andres Freund wrote:
> On 2023-08-08 08:54:10 +0900, Michael Paquier wrote:
>> - WaitEventExtensionShmemInit() should gain a dshash_create(), to make
>> sure that the shared table is around, and we are going to have a
>> reference to it in
Hi,
On 2023-08-08 08:54:10 +0900, Michael Paquier wrote:
> - WaitEventExtensionShmemInit() should gain a dshash_create(), to make
> sure that the shared table is around, and we are going to have a
> reference to it in WaitEventExtensionCounterData, saved from
> dshash_get_hash_table_handle().
On Tue, Aug 08, 2023 at 08:40:53PM +0900, Masahiro Ikeda wrote:
> I accidentally attached a patch in one previous email.
> But, you don't need to check it, sorry.
> (v1-0001-Change-to-manage-custom-wait-events-in-shared-hash.patch)
Sure, no worries. With that in place, the init function in
I accidentally attached a patch in one previous email.
But, you don't need to check it, sorry.
(v1-0001-Change-to-manage-custom-wait-events-in-shared-hash.patch)
Regards,
--
Masahiro Ikeda
NTT DATA CORPORATION
On 2023-08-08 10:05, Michael Paquier wrote:
On Tue, Aug 08, 2023 at 09:39:02AM +0900, Masahiro Ikeda wrote:
I am thinking a bit that we also need another hash where the key
is a custom string. For extensions that have no dependencies
with shared_preload_libraries, I think we need to avoid that
On Tue, Aug 08, 2023 at 09:39:02AM +0900, Masahiro Ikeda wrote:
> I am thinking a bit that we also need another hash where the key
> is a custom string. For extensions that have no dependencies
> with shared_preload_libraries, I think we need to avoid that
> WaitEventExtensionNew() is called
On 2023-08-08 08:54, Michael Paquier wrote:
On Wed, Aug 02, 2023 at 06:34:15PM +0900, Masahiro Ikeda wrote:
On 2023-08-01 12:23, Andres Freund wrote:
This is why the scheme as implemented doesn't really make sense to
me.
It'd be
much easier to use if we had a shared hashtable with the
On Wed, Aug 02, 2023 at 06:34:15PM +0900, Masahiro Ikeda wrote:
> On 2023-08-01 12:23, Andres Freund wrote:
>> This is why the scheme as implemented doesn't really make sense to me.
>> It'd be
>> much easier to use if we had a shared hashtable with the identifiers
>> than
>> what's been merged
On 2023-08-01 12:23, Andres Freund wrote:
Hi,
On 2023-08-01 12:14:49 +0900, Michael Paquier wrote:
On Tue, Aug 01, 2023 at 11:51:35AM +0900, Masahiro Ikeda wrote:
> Thanks for committing the main patch.
>
> In my understanding, the rest works are
> * to support WaitEventExtensionMultiple()
> *
Em ter., 1 de ago. de 2023 às 21:34, Masahiro Ikeda <
ikeda...@oss.nttdata.com> escreveu:
> On 2023-08-02 08:38, Ranier Vilela wrote:
> > Hi,
> >
> > On Tue, Aug 01, 2023 at 11:51:35AM +0900, Masahiro Ikeda wrote:
> >> Thanks for committing the main patch.
> >
> > Latest head
> > Ubuntu 64 bits
>
On 2023-08-02 08:38, Ranier Vilela wrote:
Hi,
On Tue, Aug 01, 2023 at 11:51:35AM +0900, Masahiro Ikeda wrote:
Thanks for committing the main patch.
Latest head
Ubuntu 64 bits
gcc 13 64 bits
./configure --without-icu
make clean
make
In file included from ../../src/include/pgstat.h:20,
Hi,
On Tue, Aug 01, 2023 at 11:51:35AM +0900, Masahiro Ikeda wrote:
> Thanks for committing the main patch.
Latest head
Ubuntu 64 bits
gcc 13 64 bits
./configure --without-icu
make clean
make
In file included from ../../src/include/pgstat.h:20,
from controldata_utils.c:38:
Hi,
On 2023-08-01 12:14:49 +0900, Michael Paquier wrote:
> On Tue, Aug 01, 2023 at 11:51:35AM +0900, Masahiro Ikeda wrote:
> > Thanks for committing the main patch.
> >
> > In my understanding, the rest works are
> > * to support WaitEventExtensionMultiple()
> > * to replace WAIT_EVENT_EXTENSION
On Tue, Aug 01, 2023 at 11:51:35AM +0900, Masahiro Ikeda wrote:
> Thanks for committing the main patch.
>
> In my understanding, the rest works are
> * to support WaitEventExtensionMultiple()
> * to replace WAIT_EVENT_EXTENSION to custom wait events
>
> Do someone already works for them? If not,
On 2023-07-31 19:22, Michael Paquier wrote:
I am not sure that any of that is necessary. Anyway, I have applied
v11 to get the basics done.
Thanks for committing the main patch.
In my understanding, the rest works are
* to support WaitEventExtensionMultiple()
* to replace
On Mon, Jul 31, 2023 at 3:54 PM Michael Paquier wrote:
>
> On Mon, Jul 31, 2023 at 05:10:21PM +0900, Kyotaro Horiguchi wrote:
> > +/*
> > + * Return the name of an wait event ID for extension.
> > + */
> > +static const char *
> > +GetWaitEventExtensionIdentifier(uint16 eventId)
> >
> > This
On Mon, Jul 31, 2023 at 05:10:21PM +0900, Kyotaro Horiguchi wrote:
> +/*
> + * Return the name of an wait event ID for extension.
> + */
> +static const char *
> +GetWaitEventExtensionIdentifier(uint16 eventId)
>
> This looks inconsistent. Shouldn't it be GetWaitEventExtentionName()?
This is an
On Mon, Jul 31, 2023 at 01:37:49PM +0530, Bharath Rupireddy wrote:
> Do you think it's worth adding a note here in the docs about an
> external module defining more than one custom wait event? A pseudo
> code if possible or just a note? Also, how about a XXX comment atop
> WaitEventExtensionNew
At Mon, 31 Jul 2023 16:28:16 +0900, Michael Paquier wrote
in
> Attaching a v11 based on Bharath's feedback and yours, for now. I
> have also applied the addition of the two masking variables in
> wait_event.c separately with 7395a90.
+/*
+ * Return the name of an wait event ID for extension.
On Mon, Jul 31, 2023 at 12:58 PM Michael Paquier wrote:
>
>
> Attaching a v11 based on Bharath's feedback and yours, for now. I
> have also applied the addition of the two masking variables in
> wait_event.c separately with 7395a90.
+uint32 WaitEventExtensionNew(void)
+
+ Next, each process
On 2023-07-31 16:28, Michael Paquier wrote:
On Mon, Jul 31, 2023 at 03:53:27PM +0900, Masahiro Ikeda wrote:
/* This should only be called for user-defined wait event. */
if (eventId < NUM_BUILTIN_WAIT_EVENT_EXTENSION)
ereport(ERROR,
On Mon, Jul 31, 2023 at 03:53:27PM +0900, Masahiro Ikeda wrote:
> I think the order in which they are mentioned should be matched. I mean that
> - so an LWLock or Extension wait
> + so an Extension or LWLock wait
Makes sense.
> /* This should only be called for user-defined wait
On Mon, Jul 31, 2023 at 12:07:40PM +0530, Bharath Rupireddy wrote:
> We're not giving up and returning an unknown state in the v10 patch
> unlike v9, no? This comment needs to change.
Right. Better to be consistent with lwlock.c here.
>> No, it cannot because we need the custom wait event
On 2023-07-31 10:10, Michael Paquier wrote:
Attached is a new version.
Thanks for all the improvements.
I have some comments for v10.
(1)
- Extensions can add LWLock types to the list
shown in
- . In some cases, the
name
+ Extensions can add Extension and
+
On Mon, Jul 31, 2023 at 6:40 AM Michael Paquier wrote:
>
> You are right that I am making things inconsistent here. Having a
> behavior close to the existing LWLock and use "extension" when the
> event cannot be found makes the most sense. I have been a bit
> confused with the wording though of
On Fri, Jul 28, 2023 at 12:43:36PM +0530, Bharath Rupireddy wrote:
> 1.
> - so an LWLock wait event might be reported as
> - just extension rather than the
> - extension-assigned name.
> + if the extension's library is not loaded; so a custom wait event might
> + be reported as
On Fri, Jul 28, 2023 at 6:36 AM Michael Paquier wrote:
>
> I have spent more time polishing the docs and the comments. This v9
> looks in a rather committable shape now with docs, tests and core
> routines in place.
Thanks. Here are some comments on v9 patch:
1.
- so an LWLock wait event
On Thu, Jul 27, 2023 at 06:29:22PM +0900, Masahiro Ikeda wrote:
> I suspect that I forgot to specify "volatile" to the variable
> for the spinlock.
+ if (!IsUnderPostmaster)
+ {
+ /* Allocate space in shared memory. */
+ waitEventExtensionCounter = (WaitEventExtensionCounter *)
+
Hi, all.
Sorry for late reply.
I am still mid-way through the review of the core APIs, but attached
is my current version in progress, labelled v8. I'll continue
tomorrow. I'm aware of some typos in the commit message of this
patch, and the dynamic bgworker launch is failing in the CI for
On Wed, Jul 19, 2023 at 10:57:39AM -0500, Tristan Partin wrote:
> > +
> > +Custom Wait Events for Add-ins
>
> This would be the second use of "Add-ins" ever, according to my search.
> Should this be "Extensions" instead?
Yes, I would think that just "Custom Wait Events" is enough here.
On Wed, Jul 19, 2023 at 11:16:34AM -0500, Tristan Partin wrote:
> On Tue Jul 11, 2023 at 6:29 PM CDT, Michael Paquier wrote:
>> This style comes from LWLockRegisterTranche() in lwlock.c. Do you
>> think that it would be more adapted to change that to
>> pg_nextpower2_size_t() with a Size? We
On Tue Jul 11, 2023 at 6:29 PM CDT, Michael Paquier wrote:
On Tue, Jul 11, 2023 at 12:39:52PM -0500, Tristan Partin wrote:
> Given the context of our last conversation, I assume this code was
> copied from somewhere else. Since this is new code, I think it would
> make more sense if newalloc was
Thanks for continuing to work on this patchset. I only have
prose-related comments.
To support custom wait events, it add 2 APIs to define new wait events
for extensions dynamically.
Remove the "it" here.
The APIs are
* WaitEventExtensionNew()
* WaitEventExtensionRegisterName()
These are
On Wed, Jul 19, 2023 at 11:49 AM Masahiro Ikeda
wrote:
>
> I updated the patch since the cfbot found a bug.
> * v7-0001-Support-custom-wait-events-for-extensions.patch
Thanks for working on this feature. +1. I've wanted this capability
for a while because extensions have many different wait
On Wed, Jul 19, 2023 at 12:52:10PM +0900, Masahiro Ikeda wrote:
> I would like to change the wait event name of contrib modules for example
> postgres_fdw. But, I think it's better to do so after the APIs are
> committed.
Agreed to do things one step at a time here. Let's focus on the core
APIs
On 2023-07-19 12:52, Masahiro Ikeda wrote:
Hi,
I updated the patches.
* v6-0001-Support-custom-wait-events-for-extensions.patch
I updated the patch since the cfbot found a bug.
* v7-0001-Support-custom-wait-events-for-extensions.patch
Regards,
--
Masahiro Ikeda
NTT DATA CORPORATIONFrom
Hi,
I updated the patches.
* v6-0001-Support-custom-wait-events-for-extensions.patch
The main diffs are
* rebase it atop current HEAD
* update docs to show users how to use the APIs
* rename of functions and variables
* fix typos
* define a new spinlock in shared memory for this purpose
*
On Thu, Jul 13, 2023 at 10:26:35AM +0900, Masahiro Ikeda wrote:
> Thanks for your quick response. I'll rebase for the commit.
Okay, thanks. I'll wait for the rebased version before moving on with
the next review, then.
--
Michael
signature.asc
Description: PGP signature
On 2023-07-13 09:12, Michael Paquier wrote:
On Wed, Jul 12, 2023 at 05:46:31PM +0900, Michael Paquier wrote:
On Wed, Jul 12, 2023 at 04:52:38PM +0900, Masahiro Ikeda wrote:
If the behavior is unexpected, we need to change the current code.
I have created a patch for the areas that I felt
On Wed, Jul 12, 2023 at 05:46:31PM +0900, Michael Paquier wrote:
> On Wed, Jul 12, 2023 at 04:52:38PM +0900, Masahiro Ikeda wrote:
>> If the behavior is unexpected, we need to change the current code.
>> I have created a patch for the areas that I felt needed to be changed.
>> -
On Wed, Jul 12, 2023 at 04:52:38PM +0900, Masahiro Ikeda wrote:
> In my understanding, the first column of the row for WaitEventExtension in
> wait_event_names.txt can be any value and the above code should not die.
> But if I use the following input, it falls on the last line.
>
> #
On 2023-07-12 09:36, Andres Freund wrote:
Hi,
On 2023-07-11 16:45:26 +0900, Michael Paquier wrote:
+/* --
+ * Wait Events - Extension
+ *
+ * Use this category when the server process is waiting for some
condition
+ * defined by an extension module.
+ *
+ * Extensions can define
On 2023-07-12 02:39, Tristan Partin wrote:
From bf06b8100cb747031959fe81a2d19baabc4838cf Mon Sep 17 00:00:00 2001
From: Masahiro Ikeda
Date: Fri, 16 Jun 2023 11:53:29 +0900
Subject: [PATCH 1/2] Support custom wait events for extensions.
+ * This is indexed by event ID minus
On 2023-07-11 16:45, Michael Paquier wrote:
On Fri, Jun 23, 2023 at 05:56:26PM +0900, Masahiro Ikeda wrote:
I updated the patches to handle the warning mentioned
by PostgreSQL Patch Tester, and removed unnecessary spaces.
I have begun hacking on that, and the API layer inspired from the
On Tue, Jul 11, 2023 at 05:36:47PM -0700, Andres Freund wrote:
> On 2023-07-11 16:45:26 +0900, Michael Paquier wrote:
>> +$node->init;
>> +$node->append_conf(
>> +'postgresql.conf',
>> +"shared_preload_libraries = 'test_custom_wait_events'"
>> +);
>> +$node->start;
>
> I think this should
Hi,
On 2023-07-11 16:45:26 +0900, Michael Paquier wrote:
> +/* --
> + * Wait Events - Extension
> + *
> + * Use this category when the server process is waiting for some condition
> + * defined by an extension module.
> + *
> + * Extensions can define custom wait events. First, call
> +
On Tue, Jul 11, 2023 at 12:39:52PM -0500, Tristan Partin wrote:
> Given the context of our last conversation, I assume this code was
> copied from somewhere else. Since this is new code, I think it would
> make more sense if newalloc was a uint16 or size_t.
This style comes from
> From bf06b8100cb747031959fe81a2d19baabc4838cf Mon Sep 17 00:00:00 2001
> From: Masahiro Ikeda
> Date: Fri, 16 Jun 2023 11:53:29 +0900
> Subject: [PATCH 1/2] Support custom wait events for extensions.
> + * This is indexed by event ID minus NUM_BUILTIN_WAIT_EVENT_EXTENSION, and
> + * stores the
On Fri, Jun 23, 2023 at 05:56:26PM +0900, Masahiro Ikeda wrote:
> I updated the patches to handle the warning mentioned
> by PostgreSQL Patch Tester, and removed unnecessary spaces.
I have begun hacking on that, and the API layer inspired from the
LWLocks is sound. I have been playing with it in
Hi,
I updated the patches to handle the warning mentioned
by PostgreSQL Patch Tester, and removed unnecessary spaces.
Regards,
--
Masahiro Ikeda
NTT DATA CORPORATIONFrom 1bb78fa2cbe6b030cea7a570bec88bd4d68f314a Mon Sep 17 00:00:00 2001
From: Masahiro Ikeda
Date: Fri, 23 Jun 2023 17:38:38 +0900
On 2023-06-20 18:26, Masahiro Ikeda wrote:
The followings are TODO items.
* to check that meson.build works since I tested with old command
`make` now
I test with meson and I updated the patches to work with it.
My test procedure is the following.
```
export builddir=/mnt/tmp/build
export
Hi,
I updated the patches. The main changes are
* to support only dynamic wait event allocation
* to add a regression test
I appreciate any feedback.
The followings are TODO items.
* to check that meson.build works since I tested with old command `make`
now
* to make documents
* to add custom
On 2023/06/17 1:16, Tristan Partin wrote:
> I will take a look at your V2 when it is ready! I will also pass along
> that this is wanted by Neon customers :).
Thanks!
Regards,
--
Masahiro Ikeda
NTT DATA CORPORATION
I will take a look at your V2 when it is ready! I will also pass along
that this is wanted by Neon customers :).
--
Tristan Partin
Neon (https://neon.tech)
On 2023-06-16 16:46, Michael Paquier wrote:
On Fri, Jun 16, 2023 at 11:14:05AM +0900, Masahiro Ikeda wrote:
I tried to query on pg_stat_activity to check the background worker
invoked by pg_prewarm. But, I found that pg_stat_activity doesn't show
it although I may be missing something...
So, I
On Fri, Jun 16, 2023 at 11:14:05AM +0900, Masahiro Ikeda wrote:
> I tried to query on pg_stat_activity to check the background worker
> invoked by pg_prewarm. But, I found that pg_stat_activity doesn't show
> it although I may be missing something...
>
> So, I tried to implement TAP tests. But I
Subject: [PATCH 2/3] Support to define custom wait events for
extensions.
Currently, only one PG_WAIT_EXTENSION event can be used as a
wait event for extensions. Therefore, in environments with multiple
extensions are installed, it could take time to identify bottlenecks.
"exten
On 2023-06-15 22:21, Drouvot, Bertrand wrote:
Hi,
On 6/15/23 10:00 AM, Michael Paquier wrote:
On Thu, Jun 15, 2023 at 03:06:01PM +0900, Masahiro Ikeda wrote:
Currently, only one PG_WAIT_EXTENSION event can be used as a
wait event for extensions. Therefore, in environments with multiple
Thanks for replying and your kind advice!
On 2023-06-15 17:00, Michael Paquier wrote:
On Thu, Jun 15, 2023 at 03:06:01PM +0900, Masahiro Ikeda wrote:
In the core, the requested wait events are dynamically registered in
shared
memory. The extension then obtains the wait event information with
On Thu, Jun 15, 2023 at 11:13:57AM -0500, Tristan Partin wrote:
> What's the Postgres policy on the following?
>
> for (int i = 0; ...)
> for (i = 0; ...)
>
> You are using 2 different patterns in WaitEventShmemInit() and
> InitializeExtensionWaitEventTranches().
C99 style is OK since v12, so
:57:29 +0900
Subject: [PATCH 2/3] Support to define custom wait events for extensions.
> Currently, only one PG_WAIT_EXTENSION event can be used as a
> wait event for extensions. Therefore, in environments with multiple
> extensions are installed, it could take time to identify bottlenecks.
Hi,
On 6/15/23 10:00 AM, Michael Paquier wrote:
On Thu, Jun 15, 2023 at 03:06:01PM +0900, Masahiro Ikeda wrote:
Currently, only one PG_WAIT_EXTENSION event can be used as a
wait event for extensions. Therefore, in environments with multiple
extensions are installed, it could take time to
On Thu, Jun 15, 2023 at 03:06:01PM +0900, Masahiro Ikeda wrote:
> Currently, only one PG_WAIT_EXTENSION event can be used as a
> wait event for extensions. Therefore, in environments with multiple
> extensions are installed, it could take time to identify which
> extension is the bottleneck.
iro Ikeda
Date: Thu, 15 Jun 2023 12:57:29 +0900
Subject: [PATCH 2/3] Support to define custom wait events for extensions.
Currently, only one PG_WAIT_EXTENSION event can be used as a
wait event for extensions. Therefore, in environments with multiple
extensions are installed, it could take time to
77 matches
Mail list logo
