Re: Connections hang indefinitely while taking a gin index's LWLock buffer_content lock(PG10.7)

2019-11-19 Thread Alexander Korotkov
On Sun, Nov 17, 2019 at 9:18 PM Alexander Korotkov wrote: > So, I've put this explanation into README patch. I just change > designation to better match with Lehman & Yao paper and did some minor > enchantments. > > I'm going to push this patchset if no objections. So, pushed with minor changes

Re: Connections hang indefinitely while taking a gin index's LWLock buffer_content lock(PG10.7)

2019-11-17 Thread Alexander Korotkov
On Mon, Nov 11, 2019 at 2:42 AM Alexander Korotkov wrote: > I'm sorry for late reply. I was busy with various things. Also > digging into these details took some time. Please find my explanation > below. > > On Wed, Oct 30, 2019 at 2:34 AM Peter Geoghegan wrote: > > In general, it seems very

Re: Connections hang indefinitely while taking a gin index's LWLock buffer_content lock(PG10.7)

2019-11-10 Thread Alexander Korotkov
Hi! I'm sorry for late reply. I was busy with various things. Also digging into these details took some time. Please find my explanation below. On Wed, Oct 30, 2019 at 2:34 AM Peter Geoghegan wrote: > In general, it seems very important to be clear about exactly how the > key space works.

Re: Connections hang indefinitely while taking a gin index's LWLock buffer_content lock(PG10.7)

2019-10-29 Thread Peter Geoghegan
On Sun, Oct 27, 2019 at 12:04 PM Alexander Korotkov wrote: > (0001-Fix-deadlock-between-ginDeletePage-and-ginStepRigh-3.patch) Some thoughts on the first patch: * How do comparisons of items in each type of B-Tree work? I would like to see a statement about invariants similar to nbtree's

Re: Connections hang indefinitely while taking a gin index's LWLock buffer_content lock(PG10.7)

2019-10-28 Thread Alexander Korotkov
On Mon, Oct 28, 2019 at 2:00 PM Peter Geoghegan wrote: > On Sun, Oct 27, 2019 at 7:04 PM Alexander Korotkov > wrote: > > It tool me a lot of efforts to revise a concurrency section in README. > > I can't judge the result, but I probably did my best. I'd like to > > commit this patchset

Re: Connections hang indefinitely while taking a gin index's LWLock buffer_content lock(PG10.7)

2019-10-28 Thread Peter Geoghegan
Hi Alexander, On Sun, Oct 27, 2019 at 7:04 PM Alexander Korotkov wrote: > It tool me a lot of efforts to revise a concurrency section in README. > I can't judge the result, but I probably did my best. I'd like to > commit this patchset including both bug fixes and README update. But > I'd like

Re: Connections hang indefinitely while taking a gin index's LWLock buffer_content lock(PG10.7)

2019-10-27 Thread Alexander Korotkov
Hi, Peter! On Fri, Oct 4, 2019 at 12:05 AM Alexander Korotkov wrote: > I'm planning to continue work on README, comments and commit messages. It tool me a lot of efforts to revise a concurrency section in README. I can't judge the result, but I probably did my best. I'd like to commit this

Re: Connections hang indefinitely while taking a gin index's LWLock buffer_content lock(PG10.7)

2019-10-03 Thread Alexander Korotkov
On Tue, Oct 1, 2019 at 5:55 AM Alexander Korotkov wrote: > On Mon, Sep 30, 2019 at 10:54 PM Peter Geoghegan wrote: > > > > On Sun, Sep 29, 2019 at 8:12 AM Alexander Korotkov > > wrote: > > > I just managed to reproduce this using two sessions on master branch. > > > > > > session 1 > > >

Re: Connections hang indefinitely while taking a gin index's LWLock buffer_content lock(PG10.7)

2019-09-30 Thread Alexander Korotkov
On Mon, Sep 30, 2019 at 10:54 PM Peter Geoghegan wrote: > > On Sun, Sep 29, 2019 at 8:12 AM Alexander Korotkov > wrote: > > I just managed to reproduce this using two sessions on master branch. > > > > session 1 > > session 2 > > Was the involvement of the pending list stuff in Chen's

Re: Connections hang indefinitely while taking a gin index's LWLock buffer_content lock(PG10.7)

2019-09-30 Thread Peter Geoghegan
On Sun, Sep 29, 2019 at 8:12 AM Alexander Korotkov wrote: > I just managed to reproduce this using two sessions on master branch. > > session 1 > session 2 Was the involvement of the pending list stuff in Chen's example just a coincidence? Can you recreate the problem while eliminating that

Re: Connections hang indefinitely while taking a gin index's LWLock buffer_content lock(PG10.7)

2019-09-30 Thread Peter Geoghegan
On Mon, Sep 30, 2019 at 11:00 AM Alexander Korotkov wrote: > Thank you for pointing. I remember this thread, but don't remember > details. I'll reread it. I think that doing this now would be a good idea: """ Defensively checking the page type (pending, posting, etc) within

Re: Connections hang indefinitely while taking a gin index's LWLock buffer_content lock(PG10.7)

2019-09-30 Thread Peter Geoghegan
On Sun, Sep 29, 2019 at 10:38 PM Andrey Borodin wrote: > As far as I understand deleted page is stamped with > GinPageSetDeleteXid(page, ReadNewTransactionId()); > It will not be recycled until that Xid is far behind. That only gets used within posting tree pages, though. ginInsertCleanup() is

Re: Connections hang indefinitely while taking a gin index's LWLock buffer_content lock(PG10.7)

2019-09-30 Thread Alexander Korotkov
On Sun, Sep 29, 2019 at 10:53 PM Peter Geoghegan wrote: > On Sun, Sep 29, 2019 at 7:38 AM Alexander Korotkov > wrote: > > Starting from root seems OK for me, because vacuum blocks all > > concurrent inserts before doing this. But this needs to be properly > > documented in readme. > > I never

Re: Connections hang indefinitely while taking a gin index's LWLock buffer_content lock(PG10.7)

2019-09-30 Thread Alexander Korotkov
On Mon, Sep 30, 2019 at 8:38 AM Andrey Borodin wrote: > > > 29 сент. 2019 г., в 21:27, Alexander Korotkov > > написал(а): > > > > Patch with fix is attached. Idea is simple: ginScanToDelete() now > > keeps exclusive lock on left page eliminating the need to relock it. > > So, we preserve

Re: Connections hang indefinitely while taking a gin index's LWLock buffer_content lock(PG10.7)

2019-09-29 Thread Andrey Borodin
> 29 сент. 2019 г., в 21:27, Alexander Korotkov > написал(а): > > Patch with fix is attached. Idea is simple: ginScanToDelete() now > keeps exclusive lock on left page eliminating the need to relock it. > So, we preserve left-to-right locking order and can't deadlock with > ginStepRight().

Re: Connections hang indefinitely while taking a gin index's LWLock buffer_content lock(PG10.7)

2019-09-29 Thread Peter Geoghegan
On Sun, Sep 29, 2019 at 7:38 AM Alexander Korotkov wrote: > Starting from root seems OK for me, because vacuum blocks all > concurrent inserts before doing this. But this needs to be properly > documented in readme. I never got an adequate answer to this closely related question almost two

Re: Connections hang indefinitely while taking a gin index's LWLock buffer_content lock(PG10.7)

2019-09-29 Thread Alexander Korotkov
On Sun, Sep 29, 2019 at 6:12 PM Alexander Korotkov wrote: > On Sun, Sep 29, 2019 at 5:38 PM Alexander Korotkov > wrote: > > On Sun, Sep 29, 2019 at 11:17 AM chenhj wrote: > > > Does the locking order of autovacuum process(root->right->left) correct? > > > While insert process lock gin buffer

Re: Connections hang indefinitely while taking a gin index's LWLock buffer_content lock(PG10.7)

2019-09-29 Thread Alexander Korotkov
On Sun, Sep 29, 2019 at 5:38 PM Alexander Korotkov wrote: > On Sun, Sep 29, 2019 at 11:17 AM chenhj wrote: > > Does the locking order of autovacuum process(root->right->left) correct? > > While insert process lock gin buffer by order of bottom->top and > > left->right. > > > > 1.

Re: Connections hang indefinitely while taking a gin index's LWLock buffer_content lock(PG10.7)

2019-09-29 Thread Alexander Korotkov
Hi! Thank you for reporting. On Sun, Sep 29, 2019 at 11:17 AM chenhj wrote: > Does the locking order of autovacuum process(root->right->left) correct? > While insert process lock gin buffer by order of bottom->top and left->right. > > 1. vacuum(root->right->left): Starting from root seems OK

Connections hang indefinitely while taking a gin index's LWLock buffer_content lock(PG10.7)

2019-09-29 Thread chenhj
Hi,all In our PostgreSQL 10.7(rhel 6.3) database, autovacuum process and many insert processes blocked in gin index's LWLock:buffer_content for long time. In other words, the following gin index lwlock deadlock phenomenon has occurred again. Since the following bug in 10.7 has been fixed.