Re: [HACKERS] [PATCHES] GIN improvements

[Teodor Sigaev]

Reworked version of fast insertion patch for GIN.

* shiftList does LockBufferForCleanup, which means that it can be blocked

can result in deadlocks --- which will be deadlocks on LWLocks and thus
not even detected by the system.

for an indefinitely long time by a concurrent scan, and since it's holding exclusive lock on metapage no new scans or insertions can start meanwhile. This is not only horrid from a performance standpoint but it very probably

Ops, I see possible scenario: UPDATE tbl SET gin_indexed_field = ... where gin_indexed_field .... with concurrent shiftList. Will fix. Thank you.

Fixed, see below

* GIN index scans release lock and pin on one pending-list page before
acquiring pin and lock on the next, which means there's a race condition:
shiftList could visit and delete the next page before we get to it,
because there's a window where we're holding no buffer lock at all.

Agree, will fix.

Fixed

* There is a bigger race condition, which is that after a scan has
returned a tuple from a pending page, vacuum could move the index entry
into the main index structure, and then that same scan could return that

fix.

same index entry a second time. This is a no-no, and I don't see any easy

Fixed. TIDBitmap is used for that and for preventing deadlock mentioned above.


Patch extends TIDBitmap interface by 2 functions:
bool tbm_check_tuple(TIDBitmap *tbm, const ItemPointer tid);
        returns true if tid already exists in bitmap
bool   tbm_has_lossy(TIDBitmap *tbm);
        returns true if bitmap becomes lossy



--
Teodor Sigaev                                   E-mail: [EMAIL PROTECTED]
                                                   WWW: http://www.sigaev.ru/

TIDBitmap is used for collectiong matched tuples from pending pages and after that it used as filter for results from regular GIN's scan. Also, sequential scan on pending page is replaced to binary search for performance. It's not a big win but it might improve performance.

fast_insert_gin-0.15.gz
Description: Unix tar archive

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers