Re: [HACKERS] bitmap indexes - performance

2010-07-22 Thread Simon Riggs
On Thu, 2010-07-01 at 16:30 +, Leonardo F wrote: > To sum up: IMHO nor improvements in memory usage nor > in startup time would be good reasons to switch to bitmap > indexes... but bulk index creation time (10 minutes to index > what it takes 60 minutes with btree... and maybe more if tables

Re: [HACKERS] bitmap indexes - performance

2010-07-07 Thread Josh Berkus
Are these improvements (index creation time, index size) worth enough to keep on working on this? I mean: given that bitmaps don't give any benefits in query times, but only benefits related to disk size and bulk index creation times, and will have horrible performance for insertions/deletions:

Re: [HACKERS] bitmap indexes - performance

2010-07-02 Thread Mark Kirkwood
On 02/07/10 20:30, Mark Kirkwood wrote: I recall that for (some/most? of) those low cardinality cases, (on disk) bitmap indexes would perform better too. I think the size saving alone is a huge win for serious data warehousing situations. On the other hand problems I recall are possibly reduc

Re: [HACKERS] bitmap indexes - performance

2010-07-02 Thread Mark Kirkwood
On 02/07/10 13:31, Bruce Momjian wrote: Leonardo F wrote: I'm trying to find more docs that explain the "improvements" of bitmap indexes in other products... but most of what I've found talks about bitmapAND/OR which is something that is very cool, but that postgres already does even wit

Re: [HACKERS] bitmap indexes - performance

2010-07-01 Thread Bruce Momjian
Leonardo F wrote: > I'm trying to find more docs that explain the "improvements" of > bitmap indexes in other products... but most of what I've found > talks about bitmapAND/OR which is something that is very > cool, but that postgres already does even with btree indexes... > or index creation

Re: [HACKERS] bitmap indexes - performance

2010-07-01 Thread Leonardo F
> In > principle a bitmap index scan should be significantly faster if the > index can > return the bitmap more or less "natively" rather than having > to construct > it. The problem I'm seeing is that even on a 20M rows table, doing a select * from t where c1=10 and c2=1 where c1 and c2 ar

Re: [HACKERS] bitmap indexes - performance

2010-07-01 Thread Robert Haas
On Thu, Jul 1, 2010 at 11:21 AM, Tom Lane wrote: > In particular, I recall some discussions about developing a "streaming > API" whereby an index AM could return a bitmap page-by-page or so, > rather than having to construct the whole thing in-memory before > anything could happen.  This would be

Re: [HACKERS] bitmap indexes - performance

2010-07-01 Thread Tom Lane
Robert Haas writes: > Hmm... no performance improvement? That's not encouraging. > The index being smaller ought to by itself provide some performance > improvement if, say, the smaller index can fit in cache and the larger > one can't. With a 6-15x size difference, that's presumably not an > i

Re: [HACKERS] bitmap indexes - performance

2010-07-01 Thread Robert Haas
On Thu, Jul 1, 2010 at 9:23 AM, Leonardo F wrote: > Using as a starting point the old bitmap patch in: > > http://archives.postgresql.org/message-id/20081101000154.go27...@fune > > > I re-applied and re-worked the patch to see what kind of improvements over > btrees bitmaps actually provided. > >

Re: [HACKERS] bitmap indexes - performance

2010-07-01 Thread Bruce Momjian
Leonardo F wrote: > Using as a starting point the old bitmap patch in: > > http://archives.postgresql.org/message-id/20081101000154.go27...@fune > > > I re-applied and re-worked the patch to see what kind of improvements over > btrees bitmaps actually provided. > > Using a 20M rows table of 10/

[HACKERS] bitmap indexes - performance

2010-07-01 Thread Leonardo F
Using as a starting point the old bitmap patch in: http://archives.postgresql.org/message-id/20081101000154.go27...@fune I re-applied and re-worked the patch to see what kind of improvements over btrees bitmaps actually provided. Using a 20M rows table of 10/100/1000 random values, I've found t