Re: Exposing the lock manager's WaitForLockers() to SQL

2024-03-26 Thread Will Mortensen
Rebased, fixed a couple typos, and reordered the isolation tests to put the most elaborate pair last. v11-0001-Refactor-GetLockConflicts-into-more-general-GetL.patch Description: Binary data v11-0002-Allow-specifying-single-lockmode-in-WaitForLocke.patch Description: Binary data

Re: Exposing the lock manager's WaitForLockers() to SQL

2024-03-08 Thread Will Mortensen
Rebased and fixed conflicts. FWIW re: Andrey's comment in his excellent CF summary email[0]: we're currently using vanilla Postgres (via Gentoo) on single nodes, and not anything fancy like Citus. The Citus relationship is just that we were inspired by Marco's blog post there. We have a variety

Re: Exposing the lock manager's WaitForLockers() to SQL

2024-01-28 Thread Will Mortensen
Minor style fix; sorry for the spam. v9-0001-Refactor-GetLockConflicts-into-more-general-GetLo.patch Description: Binary data v9-0002-Allow-specifying-single-lockmode-in-WaitForLocker.patch Description: Binary data v9-0003-Add-pg_wait_for_lockers-function.patch Description: Binary data

Re: Exposing the lock manager's WaitForLockers() to SQL

2024-01-28 Thread Will Mortensen
I guess the output of the deadlock test was unstable, so I simply removed it in v8 here, but I can try to fix it instead if it seems important to test that. v8-0001-Refactor-GetLockConflicts-into-more-general-GetLo.patch Description: Binary data v8-0003-Add-pg_wait_for_lockers-function.patch

Re: Exposing the lock manager's WaitForLockers() to SQL

2024-01-28 Thread Will Mortensen
On Fri, Jan 26, 2024 at 4:54 AM vignesh C wrote: > > CFBot shows that there is one warning as in [1]: > patching file doc/src/sgml/libpq.sgml > ... > [09:30:40.000] [943/2212] Compiling C object > src/backend/postgres_lib.a.p/storage_lmgr_lock.c.obj > [09:30:40.000]

Re: Exposing the lock manager's WaitForLockers() to SQL

2024-01-26 Thread vignesh C
On Thu, 11 Jan 2024 at 15:22, Will Mortensen wrote: > > Here is a new series adding a single pg_wait_for_lockers() function > that takes a boolean argument to control the interpretation of the > lock mode. It omits LOCK's handling of descendant tables so it > requires permissions directly on

Re: Exposing the lock manager's WaitForLockers() to SQL

2024-01-11 Thread Will Mortensen
Here is a new series adding a single pg_wait_for_lockers() function that takes a boolean argument to control the interpretation of the lock mode. It omits LOCK's handling of descendant tables so it requires permissions directly on descendants in order to wait for locks on them. Not sure if that

Re: Exposing the lock manager's WaitForLockers() to SQL

2024-01-09 Thread Will Mortensen
Hi Laurenz, thanks for taking a look! On Sat, Jan 6, 2024 at 4:00 AM Laurenz Albe wrote: > While your original use case is valid, I cannot think of > any other use case. So it is a special-purpose statement that is only > useful for certain processing of append-only tables. It is definitely

Re: Exposing the lock manager's WaitForLockers() to SQL

2024-01-06 Thread Laurenz Albe
On Sat, 2024-01-06 at 02:57 -0800, Will Mortensen wrote: > Simplified the code and docs, and rewrote the example with more prose > instead of PL/pgSQL, which unfortunately made it longer, although it > could be truncated. Not really sure what's best... I thought about this idea, and I have some

Re: Exposing the lock manager's WaitForLockers() to SQL

2024-01-06 Thread Will Mortensen
Simplified the code and docs, and rewrote the example with more prose instead of PL/pgSQL, which unfortunately made it longer, although it could be truncated. Not really sure what's best... v5-0001-Refactor-GetLockConflicts-into-more-general-GetLo.patch Description: Binary data

Re: Exposing the lock manager's WaitForLockers() to SQL

2023-12-23 Thread Will Mortensen
I meant to add that the example in the doc is adapted from Marco Slot's blog post linked earlier: https://www.citusdata.com/blog/2018/06/14/scalable-incremental-data-aggregation/

Re: Exposing the lock manager's WaitForLockers() to SQL

2023-12-23 Thread Will Mortensen
On Sun, Sep 3, 2023 at 11:16 PM Will Mortensen wrote: > I realized that for our use case, we'd ideally wait for holders of > RowExclusiveLock only, and not e.g. VACUUM holding > ShareUpdateExclusiveLock. Waiting for lockers in a specific mode seems > possible by generalizing/duplicating

Re: Exposing the lock manager's WaitForLockers() to SQL

2023-09-04 Thread Will Mortensen
I realized that for our use case, we'd ideally wait for holders of RowExclusiveLock only, and not e.g. VACUUM holding ShareUpdateExclusiveLock. Waiting for lockers in a specific mode seems possible by generalizing/duplicating WaitForLockersMultiple() and GetLockConflicts(), but I'd love to have a

Re: Exposing the lock manager's WaitForLockers() to SQL

2023-08-03 Thread Will Mortensen
Updated docs a bit. I'll see about adding this to the next CF in hopes of attracting a reviewer. :-) v3-0001-Add-WAIT-ONLY-option-to-LOCK-command.patch Description: Binary data

Re: Exposing the lock manager's WaitForLockers() to SQL

2023-07-04 Thread Will Mortensen
Updated patch with more tests and a first attempt at doc updates. As the commit message and doc now point out, using WaitForLockersMultiple() makes for a behavior difference with actually locking multiple tables, in that the combined set of conflicting locks is obtained only once for all tables,

Re: Exposing the lock manager's WaitForLockers() to SQL

2023-02-01 Thread Will Mortensen
Here is a first attempt at a WIP patch. Sorry about the MIME type. It doesn't take any locks on the tables, but I'm not super confident that that's safe, so any input would be appreciated. I omitted view support for simplicity, but if that seems like a requirement I'll see about adding it. I

Re: Exposing the lock manager's WaitForLockers() to SQL

2023-01-12 Thread Will Mortensen
Hi Andres, On Thu, Jan 12, 2023 at 7:49 PM Andres Freund wrote: > Consider a scenario like this: > > tx 1: acquires RowExclusiveLock on tbl1 to insert rows > tx 2: acquires AccessShareLock on tbl1 > tx 2: WaitForLockers(ShareRowExclusiveLock, tbl1) ends up waiting for tx1 > tx 1: truncate tbl1

Re: Exposing the lock manager's WaitForLockers() to SQL

2023-01-12 Thread Andres Freund
Hi, On 2023-01-12 19:21:00 -0800, Will Mortensen wrote: > FWIW re: deadlocks in general, I probably didn't highlight it well in my > original email, but the existing solution for this use case (as Marco > described in his blog post) is to actually lock the table momentarily. > Marco's blog post

Re: Exposing the lock manager's WaitForLockers() to SQL

2023-01-12 Thread Will Mortensen
Hi Andres, On Thu, Jan 12, 2023 at 11:31 AM Andres Freund wrote: > I know that WaitForLockers() is an existing function :). I'm not sure it's > entirely suitable for your use case. So I mainly wanted to point out that if > you end up writing a separate version of it, you still need to integrate

Re: Exposing the lock manager's WaitForLockers() to SQL

2023-01-12 Thread Andres Freund
Hi, On 2023-01-11 23:03:30 -0800, Will Mortensen wrote: > On Wed, Jan 11, 2023 at 12:33 PM Andres Freund wrote: > > I think such a function would still have to integrate enough with the lock > > manager infrastructure to participate in the deadlock detector. Otherwise I > > think you'd trivially

Re: Exposing the lock manager's WaitForLockers() to SQL

2023-01-12 Thread Will Mortensen
I suppose if it's correct that we need to lock the table first (at least in ACCESS SHARE mode), an option to LOCK perhaps makes more sense. Maybe you could specify two modes like: LOCK TABLE IN _lockmode_ MODE AND THEN WAIT FOR CONFLICTS WITH _waitmode_ MODE; But that might be excessive. :-D And

Re: Exposing the lock manager's WaitForLockers() to SQL

2023-01-11 Thread Will Mortensen
Hi Andres, On Wed, Jan 11, 2023 at 12:33 PM Andres Freund wrote: > I think such a function would still have to integrate enough with the lock > manager infrastructure to participate in the deadlock detector. Otherwise I > think you'd trivially end up with loads of deadlocks. Could you elaborate

Re: Exposing the lock manager's WaitForLockers() to SQL

2023-01-11 Thread Andres Freund
Hi, On 2023-01-10 10:01:25 +0100, Marco Slot wrote: > On Fri, Dec 23, 2022 at 11:43 AM Will Mortensen wrote: > > We'd like to be able to call the lock manager's WaitForLockers() and > > WaitForLockersMultiple() from SQL. Below I describe our use case, but > > basically I'm wondering if this: > >

Re: Exposing the lock manager's WaitForLockers() to SQL

2023-01-11 Thread Will Mortensen
Hi Marco, thanks for the reply! Glad to know you'd find it useful too. :-) On Tue, Jan 10, 2023 at 1:01 AM Marco Slot wrote: > I'm wondering whether it could be an option of the LOCK command. > (LOCK WAIT ONLY?) I assume that's doable, but just from looking at the docs, it might be a little

Re: Exposing the lock manager's WaitForLockers() to SQL

2023-01-10 Thread Marco Slot
On Fri, Dec 23, 2022 at 11:43 AM Will Mortensen wrote: > We'd like to be able to call the lock manager's WaitForLockers() and > WaitForLockersMultiple() from SQL. Below I describe our use case, but > basically I'm wondering if this: > > 1. Seems like a reasonable thing to do > > 2. Would

Exposing the lock manager's WaitForLockers() to SQL

2022-12-23 Thread Will Mortensen
Hi there, We'd like to be able to call the lock manager's WaitForLockers() and WaitForLockersMultiple() from SQL. Below I describe our use case, but basically I'm wondering if this: 1. Seems like a reasonable thing to do 2. Would be of interest upstream 3. Should be done with a new