Re: [GENERAL] new index type with clustering in mind.

2014-12-11 Thread Jack Douglas
Currently, one issue you're going to face is that brin doesn't rescan a range to find the tighest possible summary tuple. That's going to be an issue I think, thanks for mentioning it. We'd need some sort of mechanism for achieving this without a complete REINDEX, even if it only reset the

Re: [GENERAL] new index type with clustering in mind.

2014-12-10 Thread Jack Douglas
-general@postgresql.org Subject: Re: [GENERAL] new index type with clustering in mind. Martijn van Oosterhout klep...@svana.org writes: On Sat, May 24, 2014 at 05:58:37PM +0100, Jack Douglas wrote: Would the following be practical to implement: A btree-like index type that points to *pages* rather

Re: [GENERAL] new index type with clustering in mind.

2014-12-10 Thread Alvaro Herrera
Jack Douglas wrote: in 9.4, GIN indexes are pretty close to this already Do I understand correctly that BRIN indexes will be even closer to this? Yeah, in a way. You could say they are closer from the opposite end. There is one index tuple in a BRIN index for each page range (contiguous

Re: [GENERAL] new index type with clustering in mind.

2014-12-10 Thread Jack Douglas
If the values are perfectly clustered, the index is optimal because you scan the minimal set of pages. That's the bit I'm particularly interested in, as my plan would be to keep the pages well clustered: http://dba.stackexchange.com/a/66293/1396 Do you see any blocker preventing BRIN being used

Re: [GENERAL] new index type with clustering in mind.

2014-12-10 Thread Alvaro Herrera
Jack Douglas wrote: If the values are perfectly clustered, the index is optimal because you scan the minimal set of pages. That's the bit I'm particularly interested in, as my plan would be to keep the pages well clustered: http://dba.stackexchange.com/a/66293/1396 Do you see any blocker

Re: [GENERAL] new index type with clustering in mind.

2014-06-03 Thread Jack Douglas
To reduce complexity (eg MVCC/snapshot related issues), index entries would be added when a row is inserted, but they would not be removed when the row is updated/deleted (or when an insert is rolled back): It's an interesting idea, but, how can you *ever* delete index entries?

Re: [GENERAL] new index type with clustering in mind.

2014-05-26 Thread Jack Douglas
The discussions at PGCon pointed out that with the posting-list compression logic added in 9.4, GIN indexes are pretty close to this already. Multiple items on the same heap page will typically only take one byte of index space per item; but there is an identifiable entry, so you don't get into

Re: [GENERAL] new index type with clustering in mind.

2014-05-24 Thread Martijn van Oosterhout
On Sat, May 24, 2014 at 05:58:37PM +0100, Jack Douglas wrote: Would the following be practical to implement: A btree-like index type that points to *pages* rather than individual rows. Ie if there are many rows in a page with the same data (in the indexed columns), only one index entry will

Re: [GENERAL] new index type with clustering in mind.

2014-05-24 Thread Tom Lane
Martijn van Oosterhout klep...@svana.org writes: On Sat, May 24, 2014 at 05:58:37PM +0100, Jack Douglas wrote: Would the following be practical to implement: A btree-like index type that points to *pages* rather than individual rows. It's an interesting idea, but, how can you *ever* delete