Re: pgsql: Add deduplication to nbtree.

2020-04-30 Thread Peter Geoghegan
On Thu, Apr 30, 2020 at 11:37 AM Peter Geoghegan wrote: > > The following change fixes it: > > Your fix looks good to me. Please go ahead and commit it. Actually, never mind. I just pushed your fix myself a moment ago. Thanks again -- Peter Geoghegan

Re: pgsql: Add deduplication to nbtree.

2020-04-30 Thread Peter Geoghegan
On Thu, Apr 30, 2020 at 11:33 AM Peter Eisentraut wrote: > AddressSanitizer has a use-after-scope complaint related to this patch. > > The following change fixes it: Your fix looks good to me. Please go ahead and commit it. Thanks! -- Peter Geoghegan

Re: pgsql: Add deduplication to nbtree.

2020-04-30 Thread Peter Eisentraut
On 2020-02-26 22:06, Peter Geoghegan wrote: > Add deduplication to nbtree. AddressSanitizer has a use-after-scope complaint related to this patch. The following change fixes it: diff --git a/src/backend/access/nbtree/nbtinsert.c b/src/backend/access/nbtree/nbtinsert.c index a70b64d964..8246ab4f

Re: pgsql: Add deduplication to nbtree.

2020-03-29 Thread Andres Freund
Hi, On 2020-03-29 15:19:50 -0700, Peter Geoghegan wrote: > On Sun, Mar 29, 2020 at 3:15 PM Andres Freund wrote: > > Is it perhaps possible to silence the warnign with somethign along the > > lines of > > Assert(nhtids + vacposting->ndeletedtids == > > BTreeTupleGetNPosting(origtuple)) > > I don'

Re: pgsql: Add deduplication to nbtree.

2020-03-29 Thread Peter Geoghegan
On Sun, Mar 29, 2020 at 3:15 PM Andres Freund wrote: > Is it perhaps possible to silence the warnign with somethign along the > lines of > Assert(nhtids + vacposting->ndeletedtids == BTreeTupleGetNPosting(origtuple)) > I don't know this code, but it looks like that'd have to be true? > Perhaps tha

Re: pgsql: Add deduplication to nbtree.

2020-03-29 Thread Andres Freund
Hi, On 2020-03-01 16:09:37 -0800, Peter Geoghegan wrote: > On Sun, Mar 1, 2020 at 3:01 PM Peter Geoghegan wrote: > > I am happy to add parallel-to-_bt_form_posting() assertions about > > alignment to _bt_form_posting(), to nail it down completely. Plus I'll > > add the assertion I suggested alrea

Re: pgsql: Add deduplication to nbtree.

2020-03-02 Thread Peter Geoghegan
On Mon, Mar 2, 2020 at 9:47 AM Tom Lane wrote: > I suppose this can be silenced with an appropriate cast, and doing so > would seem like a good idea. I pushed a commit that silenced the cpluspluscheck warning just now. Thanks -- Peter Geoghegan

Re: pgsql: Add deduplication to nbtree.

2020-03-02 Thread Tom Lane
Another issue I just noticed is that src/tools/pginclude/cpluspluscheck complains thusly: ./src/include/access/nbtree.h: In function 'void BTreeTupleSetPosting(IndexTupleData*, int, int)': ./src/include/access/nbtree.h:384: warning: comparison between signed and unsigned integer expressions I s

Re: pgsql: Add deduplication to nbtree.

2020-03-02 Thread Peter Geoghegan
On Sun, Mar 1, 2020 at 8:06 PM Tom Lane wrote: > > I was thinking of the approach taken in the attached patch. It more or > > less copies over the assertions from _bt_form_posting() that did not > > appear in _bt_update_posting(). > > OK by me. Pushed. Thanks. -- Peter Geoghegan

Re: pgsql: Add deduplication to nbtree.

2020-03-01 Thread Tom Lane
Peter Geoghegan writes: > On Sun, Mar 1, 2020 at 3:01 PM Peter Geoghegan wrote: >> I am happy to add parallel-to-_bt_form_posting() assertions about >> alignment to _bt_form_posting(), to nail it down completely. Plus I'll >> add the assertion I suggested already. Once I commit a patch with >> th

Re: pgsql: Add deduplication to nbtree.

2020-03-01 Thread Peter Geoghegan
On Sun, Mar 1, 2020 at 3:01 PM Peter Geoghegan wrote: > I am happy to add parallel-to-_bt_form_posting() assertions about > alignment to _bt_form_posting(), to nail it down completely. Plus I'll > add the assertion I suggested already. Once I commit a patch with > these two new assertions, I think

Re: pgsql: Add deduplication to nbtree.

2020-03-01 Thread Peter Geoghegan
On Sun, Mar 1, 2020 at 2:14 PM Tom Lane wrote: > > Attached patch shows how this could work. I prefer my original > > approach, but I can see the argument for doing it this way. > > This does seem a bit duplicative ... and shouldn't both code paths > include a final "Assert(d == vacposting->ndelet

Re: pgsql: Add deduplication to nbtree.

2020-03-01 Thread Tom Lane
Peter Geoghegan writes: > Attached patch shows how this could work. I prefer my original > approach, but I can see the argument for doing it this way. This does seem a bit duplicative ... and shouldn't both code paths include a final "Assert(d == vacposting->ndeletedtids)"? So maybe we're better

Re: pgsql: Add deduplication to nbtree.

2020-03-01 Thread Peter Geoghegan
On Sun, Mar 1, 2020 at 11:42 AM Peter Geoghegan wrote: > > Do you want to try coding it that way and see what it > > comes out like? > > Sure. Attached patch shows how this could work. I prefer my original approach, but I can see the argument for doing it this way. If we keep my original approac

Re: pgsql: Add deduplication to nbtree.

2020-03-01 Thread Peter Geoghegan
On Sun, Mar 1, 2020 at 11:29 AM Tom Lane wrote: > Hm. That would probably be enough to shut up Coverity, but I'm unsure > whether it'd really be an improvement from the legibility and safety > viewpoints. I noticed that _bt_update_posting() behaves as if the origtuple might not be a posting list

Re: pgsql: Add deduplication to nbtree.

2020-03-01 Thread Tom Lane
Peter Geoghegan writes: > On Sun, Mar 1, 2020 at 10:24 AM Tom Lane wrote: >> I can see its point: asserting after the fact that you didn't clobber >> memory isn't a terribly safe coding method, especially in a production >> build where you won't even have the asserts. Not sure if there's a >> be

Re: pgsql: Add deduplication to nbtree.

2020-03-01 Thread Peter Geoghegan
On Sun, Mar 1, 2020 at 10:24 AM Tom Lane wrote: > I can see its point: asserting after the fact that you didn't clobber > memory isn't a terribly safe coding method, especially in a production > build where you won't even have the asserts. Not sure if there's a > better way though. I found it sl

Re: pgsql: Add deduplication to nbtree.

2020-03-01 Thread Tom Lane
Peter Geoghegan writes: > Add deduplication to nbtree. Coverity isn't very happy with the coding in _bt_update_posting(): *** CID 1460433: Memory - corruptions (ARRAY_VS_SINGLETON) /srv/coverity/git/pgsql-git/postgresql/src/backend/access/nbtree/nbtdedup.c: 723 in _bt_update_posting() 717

Re: pgsql: Add deduplication to nbtree.

2020-02-28 Thread Peter Geoghegan
On Thu, Feb 27, 2020 at 9:26 AM Peter Geoghegan wrote: > > Are no changes to the "pageinspect" contrib required? > > There are. Those will be pushed either today or tomorrow. Attached is a draft patch for this, with updated documentation. I'd like to push this tomorrow (Saturday), but if you coul

Re: pgsql: Add deduplication to nbtree.

2020-02-27 Thread Peter Geoghegan
On Thu, Feb 27, 2020 at 9:25 AM Laurenz Albe wrote: > This is great! Thanks! Thanks! > Are no changes to the "pageinspect" contrib required? There are. Those will be pushed either today or tomorrow. -- Peter Geoghegan

Re: pgsql: Add deduplication to nbtree.

2020-02-27 Thread Laurenz Albe
On Wed, 2020-02-26 at 21:06 +, Peter Geoghegan wrote: > Add deduplication to nbtree. This is great! Thanks! Are no changes to the "pageinspect" contrib required? Yours, Laurenz Albe

pgsql: Add deduplication to nbtree.

2020-02-26 Thread Peter Geoghegan
Add deduplication to nbtree. Deduplication reduces the storage overhead of duplicates in indexes that use the standard nbtree index access method. The deduplication process is applied lazily, after the point where opportunistic deletion of LP_DEAD-marked index tuples occurs. Deduplication is onl