Re: [HACKERS] Bitmap index scans use of filters on available columns

2015-11-08 Thread Jeff Janes
On Sun, Nov 8, 2015 at 12:34 PM, Tom Lane wrote: > Jeff Janes writes: >> On Fri, Nov 6, 2015 at 7:15 PM, Tomas Vondra >> wrote: >>> I've however also noticed that all the 'like' procedures are marked as not >>> leak proof, which is a bit unfortunate because that's the example from >>> Jeff's e-m

Re: [HACKERS] Bitmap index scans use of filters on available columns

2015-11-08 Thread Tom Lane
Jeff Janes writes: > On Fri, Nov 6, 2015 at 7:15 PM, Tomas Vondra > wrote: >> I've however also noticed that all the 'like' procedures are marked as not >> leak proof, which is a bit unfortunate because that's the example from >> Jeff's e-mail that started this thread. > Is there a reason they a

Re: [HACKERS] Bitmap index scans use of filters on available columns

2015-11-08 Thread Jeff Janes
On Fri, Nov 6, 2015 at 7:15 PM, Tomas Vondra wrote: > Hi, > > On 11/07/2015 02:18 AM, Robert Haas wrote: >> >> On Fri, Nov 6, 2015 at 7:11 PM, Tomas Vondra >> wrote: I think LEAKPROOF is probably fine for this. How would the new thing be different? >>> >>> >>> I don't think so - A

Re: [HACKERS] Bitmap index scans use of filters on available columns

2015-11-06 Thread Amit Kapila
On Fri, Nov 6, 2015 at 10:28 AM, Kyotaro HORIGUCHI < horiguchi.kyot...@lab.ntt.co.jp> wrote: > > Hello, > > At Fri, 6 Nov 2015 09:49:30 +0530, Amit Kapila wrote in > > Apart from other problems discussed, I think it could also lead to > > a performance penality for the cases when the qual conditi

Re: [HACKERS] Bitmap index scans use of filters on available columns

2015-11-06 Thread Tomas Vondra
Hi, On 11/07/2015 02:18 AM, Robert Haas wrote: On Fri, Nov 6, 2015 at 7:11 PM, Tomas Vondra wrote: I think LEAKPROOF is probably fine for this. How would the new thing be different? I don't think so - AFAIK "leakproof" is about not revealing information about arguments, nothing more and not

Re: [HACKERS] Bitmap index scans use of filters on available columns

2015-11-06 Thread Robert Haas
On Fri, Nov 6, 2015 at 7:11 PM, Tomas Vondra wrote: >> I think LEAKPROOF is probably fine for this. How would the new thing >> be different? > > I don't think so - AFAIK "leakproof" is about not revealing information > about arguments, nothing more and nothing less. It does not say whether it's >

Re: [HACKERS] Bitmap index scans use of filters on available columns

2015-11-06 Thread Tomas Vondra
Hi, On 11/05/2015 07:36 PM, Robert Haas wrote: On Thu, Nov 5, 2015 at 1:29 PM, Tomas Vondra wrote: But then again, can we come up with a way to distinguish operators that are safe to evaluate on indexes - either automatic or manual? We already do that with the indexable operators with explici

Re: [HACKERS] Bitmap index scans use of filters on available columns

2015-11-05 Thread Kyotaro HORIGUCHI
Hello, At Fri, 6 Nov 2015 09:49:30 +0530, Amit Kapila wrote in > Apart from other problems discussed, I think it could also lead to > a performance penality for the cases when the qual condition is > costly as evaluating such a qual against lot of dead rows would be a > time consuming operation

Re: [HACKERS] Bitmap index scans use of filters on available columns

2015-11-05 Thread Amit Kapila
On Thu, Nov 5, 2015 at 7:45 AM, Tomas Vondra wrote: > Hi, > > On 11/04/2015 11:32 PM, Tom Lane wrote: > >> Jeff Janes writes: >> >>> On Wed, Nov 4, 2015 at 7:14 AM, Tom Lane wrote: >>> You're missing my point: that is possible in an indexscan, but *not* in a bitmap indexscan, because

Re: [HACKERS] Bitmap index scans use of filters on available columns

2015-11-05 Thread Robert Haas
On Thu, Nov 5, 2015 at 1:29 PM, Tomas Vondra wrote: >> Well, one thing is that you might leak information about >> already-deleted rows, which could be a security vulnerability, or more >> mundanely cause a function to error out when there are no actually >> visible rows that could trigger such an

Re: [HACKERS] Bitmap index scans use of filters on available columns

2015-11-05 Thread Tomas Vondra
Hi, On 11/05/2015 06:51 PM, Robert Haas wrote: On Wed, Nov 4, 2015 at 9:15 PM, Tomas Vondra wrote: I certainly understand there are cases that require care - like the leakproof thing pointed out by Robert for example. I don't immediately see why evaluation against dead rows would be a problem.

Re: [HACKERS] Bitmap index scans use of filters on available columns

2015-11-05 Thread Robert Haas
On Wed, Nov 4, 2015 at 9:15 PM, Tomas Vondra wrote: > I certainly understand there are cases that require care - like the > leakproof thing pointed out by Robert for example. I don't immediately see > why evaluation against dead rows would be a problem. Well, one thing is that you might leak info

Re: [HACKERS] Bitmap index scans use of filters on available columns

2015-11-04 Thread Tomas Vondra
Hi, On 11/04/2015 11:32 PM, Tom Lane wrote: Jeff Janes writes: On Wed, Nov 4, 2015 at 7:14 AM, Tom Lane wrote: You're missing my point: that is possible in an indexscan, but *not* in a bitmap indexscan, because the index AM APIs are totally different in the two cases. In a bitmap scan, nothi

Re: [HACKERS] Bitmap index scans use of filters on available columns

2015-11-04 Thread Tom Lane
Jeff Janes writes: > On Wed, Nov 4, 2015 at 7:14 AM, Tom Lane wrote: >> You're missing my point: that is possible in an indexscan, but *not* in a >> bitmap indexscan, because the index AM APIs are totally different in the >> two cases. In a bitmap scan, nothing more than a TID bitmap is ever >>

Re: [HACKERS] Bitmap index scans use of filters on available columns

2015-11-04 Thread Jeff Janes
On Wed, Nov 4, 2015 at 7:14 AM, Tom Lane wrote: > Simon Riggs writes: >> On 4 November 2015 at 15:54, Tom Lane wrote: >> We generate this plan >> Index Scan using f_x_y_idx on f (cost=0.42..26075.71 rows=209 width=37) >>Index Cond: (x = 5) >>Filter: (y ~~ '%abc%'::text) > >> So it shou

Re: [HACKERS] Bitmap index scans use of filters on available columns

2015-11-04 Thread Robert Haas
On Wed, Nov 4, 2015 at 10:59 AM, Tom Lane wrote: > Simon Riggs writes: >> On 4 November 2015 at 16:14, Tom Lane wrote: >>> You're missing my point: that is possible in an indexscan, but *not* in a >>> bitmap indexscan, because the index AM APIs are totally different in the >>> two cases. In a b

Re: [HACKERS] Bitmap index scans use of filters on available columns

2015-11-04 Thread Simon Riggs
On 4 November 2015 at 16:59, Tom Lane wrote: > Simon Riggs writes: > > On 4 November 2015 at 16:14, Tom Lane wrote: > >> You're missing my point: that is possible in an indexscan, but *not* in > a > >> bitmap indexscan, because the index AM APIs are totally different in the > >> two cases. In

Re: [HACKERS] Bitmap index scans use of filters on available columns

2015-11-04 Thread Tom Lane
Simon Riggs writes: > On 4 November 2015 at 16:14, Tom Lane wrote: >> You're missing my point: that is possible in an indexscan, but *not* in a >> bitmap indexscan, because the index AM APIs are totally different in the >> two cases. In a bitmap scan, nothing more than a TID bitmap is ever >> re

Re: [HACKERS] Bitmap index scans use of filters on available columns

2015-11-04 Thread Simon Riggs
On 4 November 2015 at 16:14, Tom Lane wrote: > Simon Riggs writes: > > On 4 November 2015 at 15:54, Tom Lane wrote: > > We generate this plan > > Index Scan using f_x_y_idx on f (cost=0.42..26075.71 rows=209 width=37) > >Index Cond: (x = 5) > >Filter: (y ~~ '%abc%'::text) > > > So it

Re: [HACKERS] Bitmap index scans use of filters on available columns

2015-11-04 Thread Tom Lane
Simon Riggs writes: > On 4 November 2015 at 15:54, Tom Lane wrote: > We generate this plan > Index Scan using f_x_y_idx on f (cost=0.42..26075.71 rows=209 width=37) >Index Cond: (x = 5) >Filter: (y ~~ '%abc%'::text) > So it should be possible to do the Filter condition on the BitmapInd

Re: [HACKERS] Bitmap index scans use of filters on available columns

2015-11-04 Thread Simon Riggs
On 4 November 2015 at 15:54, Tom Lane wrote: > Nicolas Barbier writes: > > 2015-11-04 Antonin Houska : > >> (see expand_indexqual_opclause()), I'm not sure any kind of expansion is > >> possible for '%abc%' which would result in a b-tree searchable > condition. > > > I think the question is not

Re: [HACKERS] Bitmap index scans use of filters on available columns

2015-11-04 Thread Tom Lane
Nicolas Barbier writes: > 2015-11-04 Antonin Houska : >> (see expand_indexqual_opclause()), I'm not sure any kind of expansion is >> possible for '%abc%' which would result in a b-tree searchable condition. > I think the question is not about using the b-tree for checking the > condition, but abo

Re: [HACKERS] Bitmap index scans use of filters on available columns

2015-11-04 Thread Nicolas Barbier
2015-11-04 Antonin Houska : > While prefix expression > > y like 'abc%' > > can be converted to > > y >= 'abc' > > (see expand_indexqual_opclause()), I'm not sure any kind of expansion is > possible for '%abc%' which would result in a b-tree searchable condition. I think the question is not about

Re: [HACKERS] Bitmap index scans use of filters on available columns

2015-11-04 Thread Antonin Houska
While prefix expression y like 'abc%' can be converted to y >= 'abc' (see expand_indexqual_opclause()), I'm not sure any kind of expansion is possible for '%abc%' which would result in a b-tree searchable condition. Jeff Janes wrote: > Is there a fundamental reason the filter on y is not bei

[HACKERS] Bitmap index scans use of filters on available columns

2015-11-03 Thread Jeff Janes
create table f as select (random()*100)::int as x, md5(random()::text) as y from generate_series(1,100); create index on f (x, y); analyze verbose f; --dont vacuum explain select * from f where x=5 and y like '%abc%'; QUERY PLAN -

Re: [HACKERS] bitmap-index-scan faster than seq-scan on full-table-scan (gin index)

2010-05-31 Thread Jesper Krogh
On 2010-05-31 22:09, Tom Lane wrote: Jesper Krogh writes: Conceptually searching for the "full dataset" would always be fastest solved by a seq-scan. The query planner enforces this so much, so not even "enable_seqscan=off" can convince it to to something else. ... Would it be possible to i

Re: [HACKERS] bitmap-index-scan faster than seq-scan on full-table-scan (gin index)

2010-05-31 Thread Tom Lane
Jesper Krogh writes: > Conceptually searching for the "full dataset" would always be fastest > solved by a seq-scan. The query planner enforces this so much, so not > even "enable_seqscan=off" can convince it to to something else. > ... > Would it be possible to implement the "Filtering" using the

[HACKERS] bitmap-index-scan faster than seq-scan on full-table-scan (gin index)

2010-05-31 Thread Jesper Krogh
Hi. The test data a set of generated terms using this perl-script http://shrek.krogh.cc/~jesper/build-test.pl and http://shrek.krogh.cc/~jesper/words.txt I have generated a test dataset with an average tsvector length of around 250 and 200.000 tuples in the dataset. Conceptually searching for t

Re: [HACKERS] Bitmap index - first look

2008-11-07 Thread Gianni Ciolli
On Fri, Nov 07, 2008 at 03:56:25PM +0300, Teodor Sigaev wrote: > After looking at additional heap and b-tree index placed in > pg_bitmapindex namespace... > > Additional heap contains unique values and page's number with offset > number in bitmap index, b-tree index contains tuples with the same

Re: [HACKERS] Bitmap index - first look

2008-11-07 Thread Teodor Sigaev
Additional heap contains unique values and page's number with offset number in bitmap index, b-tree index contains tuples with the same values and ItemPointer to heap's row. So, heap is an unnecessary step - b-tree index should store ItemPointer to the bitmap index directly. B-tree points to LO

Re: [HACKERS] Bitmap index - first look

2008-11-07 Thread Vladimir Sitnikov
Could you please put more comments on the index build procedure? I guess it strongly relies on the fact the block number increases as tuples are returned from "heap scan", doesn't it? However, thanks to synchronized scans the order of tuples could be a little bit different. I found no evidence of

Re: [HACKERS] Bitmap index - first look

2008-11-07 Thread Teodor Sigaev
After looking at additional heap and b-tree index placed in pg_bitmapindex namespace... Additional heap contains unique values and page's number with offset number in bitmap index, b-tree index contains tuples with the same values and ItemPointer to heap's row. So, heap is an unnecessary step

Re: [HACKERS] Bitmap index - first look

2008-11-06 Thread Vladimir Sitnikov
One more point on pg_am: amsearchnull is equal to "f" however the index stores and could find nulls perfectly. Regards, Vladimir Sitnikov

[HACKERS] Bitmap index - first look

2008-11-06 Thread Teodor Sigaev
http://archives.postgresql.org/message-id/[EMAIL PROTECTED] 1) Sometimes index doesn't find all matching rows: postgres=# SELECT * FROM qq WHERE t ='asd'; i | t ---+- 2 | asd 1 | asd 2 | asd (3 rows) postgres=# SET enable_seqscan=off; SET postgres=# SELECT * FROM qq WHERE t ='asd'; i |

Re: [HACKERS] Bitmap index thoughts (another segfault)

2007-04-08 Thread Gavin Sherry
On Sat, 7 Apr 2007, Mark Kirkwood wrote: > Mark Kirkwood wrote: > bitmap=# SELECT count(*) FROM bitmaptest > WHERE val1 in (1,7) > AND val0 IN (4,3) > ; > > ERROR: XX000: unknown stream type 2 > LOCATION: stream_add_node, tidbitmap.c:1033 Thanks. Turned out t

Re: [HACKERS] Bitmap index thoughts (another segfault)

2007-04-08 Thread Gavin Sherry
> I'm seeing a segfault on a size TPC-H size 10 database. The patch and > code are: > - bitmap patch from 12 Mar > - 8.3 dev from 27 Mar Thanks Mark. I tracked this down. I'll post a new patch soon. Gavin ---(end of broadcast)--- TIP 6: explain an

Re: [HACKERS] Bitmap index thoughts (another segfault)

2007-04-06 Thread Mark Kirkwood
Mark Kirkwood wrote: I'm seeing a segfault on a size TPC-H size 10 database. The patch and code are: - bitmap patch from 12 Mar - 8.3 dev from 27 Mar SELECT count(distinct(o_orderkey)) FROM orders orders_alias WHERE o_orderpriority IN ('1-URGENT', '3-MEDIUM') AND o_orderstatus='P'; (gdb) bt #

Re: [HACKERS] Bitmap index thoughts (another segfault)

2007-04-05 Thread Mark Kirkwood
I'm seeing a segfault on a size TPC-H size 10 database. The patch and code are: - bitmap patch from 12 Mar - 8.3 dev from 27 Mar SELECT count(distinct(o_orderkey)) FROM orders orders_alias WHERE o_orderpriority IN ('1-URGENT', '3-MEDIUM') AND o_orderstatus='P'; (gdb) bt #0 0x in ?? ()

Re: [HACKERS] Bitmap index stuff

2007-02-26 Thread Gavin Sherry
On Mon, 26 Feb 2007, Heikki Linnakangas wrote: > Hi, > > How are you doing with the bitmap indexes? I need to send of a patch fixing the last bug you pointed out. The code needs a merge of HEAD. > > I've been trying to get my head around the patch a couple of times to > add the vacuum support, b

[HACKERS] Bitmap index stuff

2007-02-26 Thread Heikki Linnakangas
Hi, How are you doing with the bitmap indexes? I've been trying to get my head around the patch a couple of times to add the vacuum support, but no matter how simple I try to keep it, I just always seem to get stuck. It looks like vacuum support would need: - something similar to read_words

Re: [HACKERS] Bitmap index thoughts

2007-02-08 Thread Gavin Sherry
On Thu, 8 Feb 2007, Heikki Linnakangas wrote: > Gavin Sherry wrote: > > I will update the code tomorrow. The focus will be cleaning up the > > executor modifications. Please look else where for now. > > I'm getting a segfault with this test script: > > > CREATE TABLE bmtest (i int); > > I

Re: [HACKERS] Bitmap index thoughts

2007-02-08 Thread Heikki Linnakangas
Gavin Sherry wrote: I will update the code tomorrow. The focus will be cleaning up the executor modifications. Please look else where for now. I'm getting a segfault with this test script: CREATE TABLE bmtest (i int); INSERT INTO bmtest SELECT 1 FROM generate_series(1,10) a; INSE

Re: [HACKERS] Bitmap index thoughts

2007-02-06 Thread Heikki Linnakangas
Gavin Sherry wrote: On Thu, 1 Feb 2007, Bruce Momjian wrote: Where are we on this patch? Does it have performance tests to show where it is beneificial? Is it ready to be reviewed? Here's an updated patch: http://www.alcove.com.au/~swm/bitmap-2007-02-02.patch In this patch, I rewrote the

Re: [HACKERS] Bitmap index thoughts

2007-02-02 Thread Gavin Sherry
On Thu, 1 Feb 2007, Bruce Momjian wrote: > > Where are we on this patch? Does it have performance tests to show > where it is beneificial? Is it ready to be reviewed? Here's an updated patch: http://www.alcove.com.au/~swm/bitmap-2007-02-02.patch In this patch, I rewrote the index build system

Re: [HACKERS] Bitmap index thoughts

2007-02-01 Thread Gavin Sherry
On Thu, 1 Feb 2007, Bruce Momjian wrote: > > Where are we on this patch? Does it have performance tests to show > where it is beneificial? Is it ready to be reviewed? I've got an updated patch which adds significant performance improvements for worse case data distributions. It also contains a

Re: [HACKERS] Bitmap index thoughts

2007-02-01 Thread Bruce Momjian
Where are we on this patch? Does it have performance tests to show where it is beneificial? Is it ready to be reviewed? --- Heikki Linnakangas wrote: > I've been skimming through the bitmap index patch... > > A scan needs

Re: [HACKERS] Bitmap index thoughts

2006-12-28 Thread Jie Zhang
On 12/27/06 3:16 AM, "Gavin Sherry" <[EMAIL PROTECTED]> wrote: > On Wed, 27 Dec 2006, Heikki Linnakangas wrote: > >> Jie Zhang wrote: >>> The "bitmap data segment" sounds good in terms of space. The problem is that >>> one bitmap is likely to occupy more pages than before, which may hurt the >>>

Re: [HACKERS] Bitmap index thoughts

2006-12-28 Thread Jie Zhang
On 12/27/06 3:10 AM, "Gavin Sherry" <[EMAIL PROTECTED]> wrote: > On Wed, 27 Dec 2006, Heikki Linnakangas wrote: > >> Gavin Sherry wrote: >>> On Tue, 26 Dec 2006, Heikki Linnakangas wrote: for typical bitmap index use cases and most of the needed pages should stay in memory, but could w

Re: [HACKERS] Bitmap index thoughts

2006-12-27 Thread Simon Riggs
On Wed, 2006-12-27 at 22:16 +1100, Gavin Sherry wrote: > > But actually I'm not convinced we need to worry about efficient storage > > of small bitmaps at all. The typical use case for bitmap indexes is > > large tables with small number of distinct values, and the problem > > doesn't really arise

Re: [HACKERS] Bitmap index thoughts

2006-12-27 Thread mark
On Wed, Dec 27, 2006 at 03:42:36PM +, Heikki Linnakangas wrote: > >I wonder what would happen if somebody implemented automatic index > >exclusion conditions after use of an INDEX proved to be in the realm > >of the worst case scenario? :-) > I'm sorry, I don't understand that sentence... I wa

Re: [HACKERS] Bitmap index thoughts

2006-12-27 Thread Heikki Linnakangas
[EMAIL PROTECTED] wrote: On Wed, Dec 27, 2006 at 10:16:54PM +1100, Gavin Sherry wrote: On Wed, 27 Dec 2006, Heikki Linnakangas wrote: But actually I'm not convinced we need to worry about efficient storage of small bitmaps at all. The typical use case for bitmap indexes is large tables with sma

Re: [HACKERS] Bitmap index thoughts

2006-12-27 Thread mark
On Wed, Dec 27, 2006 at 10:16:54PM +1100, Gavin Sherry wrote: > On Wed, 27 Dec 2006, Heikki Linnakangas wrote: > > But actually I'm not convinced we need to worry about efficient storage > > of small bitmaps at all. The typical use case for bitmap indexes is > > large tables with small number of di

Re: [HACKERS] Bitmap index thoughts

2006-12-27 Thread Gavin Sherry
On Wed, 27 Dec 2006, Heikki Linnakangas wrote: > Jie Zhang wrote: > > The "bitmap data segment" sounds good in terms of space. The problem is that > > one bitmap is likely to occupy more pages than before, which may hurt the > > query performance. > > We could have segments of say 1/5 of page. Whe

Re: [HACKERS] Bitmap index thoughts

2006-12-27 Thread Gavin Sherry
On Wed, 27 Dec 2006, Heikki Linnakangas wrote: > Gavin Sherry wrote: > > On Tue, 26 Dec 2006, Heikki Linnakangas wrote: > >> for typical bitmap index use cases and most of the needed pages should > >> stay in memory, but could we simplify this? Why do we need the auxiliary > >> heap, couldn't we j

Re: [HACKERS] Bitmap index thoughts

2006-12-27 Thread Heikki Linnakangas
Jie Zhang wrote: The "bitmap data segment" sounds good in terms of space. The problem is that one bitmap is likely to occupy more pages than before, which may hurt the query performance. We could have segments of say 1/5 of page. When a bitmap grows larger than that, the bitmap would be moved

Re: [HACKERS] Bitmap index thoughts

2006-12-27 Thread Heikki Linnakangas
Gavin Sherry wrote: On Tue, 26 Dec 2006, Heikki Linnakangas wrote: for typical bitmap index use cases and most of the needed pages should stay in memory, but could we simplify this? Why do we need the auxiliary heap, couldn't we just store the blk+offset of the LOV item directly in the b-tree in

Re: [HACKERS] Bitmap index thoughts

2006-12-27 Thread Jie Zhang
>> And instead of having separate LOV pages that store a number of LOV >> items, how about storing each LOV item on a page of it's own, and using >> the rest of the page to store the last chunk of the bitmap. That would >> eliminate one page access, but more importantly, maybe we could then get >>

Re: [HACKERS] Bitmap index thoughts

2006-12-26 Thread Gavin Sherry
Hey Heikki, On Tue, 26 Dec 2006, Heikki Linnakangas wrote: > I've been skimming through the bitmap index patch... > > A scan needs to access at least five pages: > > 1. B-tree index (root+others, depending on depth) > 2. The auxiliary heap page > 3. bitmap index meta page > 4. LOV page > 5. bitma

[HACKERS] Bitmap index thoughts

2006-12-26 Thread Heikki Linnakangas
I've been skimming through the bitmap index patch... A scan needs to access at least five pages: 1. B-tree index (root+others, depending on depth) 2. The auxiliary heap page 3. bitmap index meta page 4. LOV page 5. bitmap page That seems like a lot of indirection. A high startup cost is probabl

Re: [HACKERS] Bitmap index status

2006-10-21 Thread Bruce Momjian
Gavin Sherry wrote: > On Wed, 18 Oct 2006, Heikki Linnakangas wrote: > > > Hi, > > > > I don't want to harass you :), but what's the status with the bitmap > > index code? Is there something I can do to help? > > > > Hi Heikki, > > The streaming is implemented, as are range queries. I need to br

Re: [HACKERS] Bitmap index status

2006-10-18 Thread Gavin Sherry
On Wed, 18 Oct 2006, Heikki Linnakangas wrote: > Hi, > > I don't want to harass you :), but what's the status with the bitmap > index code? Is there something I can do to help? > Hi Heikki, The streaming is implemented, as are range queries. I need to bring it up to HEAD and back-patch to bizgre

Re: [HACKERS] Bitmap index status

2006-10-18 Thread Jie Zhang
On 10/18/06 2:41 AM, "Heikki Linnakangas" <[EMAIL PROTECTED]> wrote: > Hi, > > I don't want to harass you :), but what's the status with the bitmap > index code? Is there something I can do to help? Not at all. We will send you the new patch soon. Your comments are most appreciated. Thanks, Ji

[HACKERS] Bitmap index status

2006-10-18 Thread Heikki Linnakangas
Hi, I don't want to harass you :), but what's the status with the bitmap index code? Is there something I can do to help? -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com ---(end of broadcast)--- TIP 9: In versions below 8.0,

Re: [HACKERS] Bitmap index status

2006-09-28 Thread Mark Wong
Luke Lonergan wrote: Mark, On 9/25/06 11:32 AM, "Mark Wong" <[EMAIL PROTECTED]> wrote: Yeah, basically gather as many stats as I can to accurately profile the overall system performance. I thought it would be appropriate to use a TPC-H based workload as one measuring stick to use for bitmap i

Re: [HACKERS] Bitmap index status

2006-09-26 Thread Gavin Sherry
On Tue, 26 Sep 2006, Heikki Linnakangas wrote: > Looks a bit better now, though I think you need to think more about the > encapsulation of the structs. More detailed comments below. > > Jie Zhang wrote: > > Essentially, we want to have a stream bitmap object that has an iterator, > > which will b

Re: [HACKERS] Bitmap index status

2006-09-26 Thread Heikki Linnakangas
Looks a bit better now, though I think you need to think more about the encapsulation of the structs. More detailed comments below. Jie Zhang wrote: Essentially, we want to have a stream bitmap object that has an iterator, which will be able to iterate through the bitmaps. The BitmapIndexscan,

Re: [HACKERS] Bitmap index status

2006-09-25 Thread Luke Lonergan
Mark, On 9/25/06 11:32 AM, "Mark Wong" <[EMAIL PROTECTED]> wrote: > Yeah, basically gather as many stats as I can to accurately profile the > overall system performance. I thought it would be appropriate to use a > TPC-H based workload as one measuring stick to use for bitmap indexes. Note that

Re: [HACKERS] Bitmap index status

2006-09-25 Thread Mark Wong
Hi Jie, Yeah, basically gather as many stats as I can to accurately profile the overall system performance. I thought it would be appropriate to use a TPC-H based workload as one measuring stick to use for bitmap indexes. Mark Jie Zhang wrote: Hi Mark, Thanks for doing the test. I checked

Re: [HACKERS] Bitmap index status

2006-09-25 Thread Jie Zhang
Hi Mark, Thanks for doing the test. I checked out the link you provided below. I am a little confused about the goal of these tests. Do you plan to test the overall performance of postgreSQL on handling TPC-H queries? Thanks, Jie On 9/22/06 3:45 PM, "Mark Wong" <[EMAIL PROTECTED]> wrote: > Jie

Re: [HACKERS] Bitmap index status

2006-09-23 Thread Jie Zhang
Gavin & Heikki, >> >> The handling of stream and hash bitmaps looks pretty complicated to me. >> All the bitmap-related nodes have logic to handle both types slightly >> differently. It all seems to come down to that if a subnode (or >> amgetbitmap in a bitmap index scan node) returns a StreamBit

Re: [HACKERS] Bitmap index status

2006-09-22 Thread Mark Wong
Jie Zhang wrote: Hi Heikki and all, I just sent the latest bitmap index patch to the list. I am not sure if there is any size limit for this mailing list. If you have received my previous email, please let me know. Hi Jie, I know I said I was going to get testing on this months ago but I've

Re: [HACKERS] Bitmap index status

2006-09-20 Thread Jie Zhang
On 9/19/06 5:15 AM, "Gavin Sherry" <[EMAIL PROTECTED]> wrote: > On Tue, 19 Sep 2006, Heikki Linnakangas wrote: > >> Jie Zhang wrote: >>> Hi Heikki and all, >>> >>> Please find the latest bitmap index patch in the attachment. This patch is >>> generated against the postgresql cvs head. >>> >>

Re: [HACKERS] Bitmap index status

2006-09-19 Thread Gavin Sherry
On Tue, 19 Sep 2006, Heikki Linnakangas wrote: > Jie Zhang wrote: > > Hi Heikki and all, > > > > Please find the latest bitmap index patch in the attachment. This patch is > > generated against the postgresql cvs head. > > > > Thanks. > > The handling of stream and hash bitmaps looks pretty compli

Re: [HACKERS] Bitmap index status

2006-09-19 Thread Heikki Linnakangas
Jie Zhang wrote: Hi Heikki and all, Please find the latest bitmap index patch in the attachment. This patch is generated against the postgresql cvs head. Thanks. The handling of stream and hash bitmaps looks pretty complicated to me. All the bitmap-related nodes have logic to handle both

Re: [HACKERS] Bitmap index status

2006-09-17 Thread Jie Zhang
Hi Heikki and all, I just sent the latest bitmap index patch to the list. I am not sure if there is any size limit for this mailing list. If you have received my previous email, please let me know. Thanks, Jie On 9/12/06 2:43 AM, "Heikki Linnakangas" <[EMAIL PROTECTED]> wrote: > Hi, > > What's

Re: [HACKERS] Bitmap index status

2006-09-12 Thread Jie Zhang
Hi Heikki, Gavin and I are trying to merge our changes together this week. We will post a new patch by the end of this week. This patch will include some style fixes, bug fixes, and the stream bitmap implementation. I will look into the problems you have mentioned in this email. Yes, vacuum curre

Re: [HACKERS] Bitmap index status

2006-09-12 Thread Tom Lane
Heikki Linnakangas <[EMAIL PROTECTED]> writes: > What's the status of the bitmap index patch? Have you worked on it since > the last posted patch > (http://archives.postgresql.org/pgsql-patches/2006-08/msg3.php)? Gavin and Jie have made major changes since that version (or at least they'd be

[HACKERS] Bitmap index status

2006-09-12 Thread Heikki Linnakangas
Hi, What's the status of the bitmap index patch? Have you worked on it since the last posted patch (http://archives.postgresql.org/pgsql-patches/2006-08/msg3.php)? I've started to review it, to get it into CVS early in the 8.3 cycle. I just want to make sure that I'm working on the lates

Re: [HACKERS] Bitmap index

2004-11-27 Thread Patrick B Kelly
On Nov 22, 2004, at 2:57 AM, Pawe Niewiadomski wrote: Hello, I saw discussion about bitmap indexes few weeks ago. I wonder if any of you is working on it (in secret)? I will be chosing subject of my master thesis and thougth about implementing bitmap indexes. -- **PaweŠNiewiadomski**, new()foo-ba

Re: [HACKERS] Bitmap index

2004-11-26 Thread Yann Michel
Hi, On Fri, Nov 26, 2004 at 10:25:41AM -, Pawel Niewiadomski wrote: > > My promoter accepted the subject. I'm waiting for an official acceptance > of the subject. Until then I would want to get familiar with PostgreSQL > internals and bitmap index implementations. I will appreciate > any lin

Re: [HACKERS] Bitmap index

2004-11-26 Thread Pawel Niewiadomski
On Wed, 24 Nov 2004 19:26:41 +1100, Neil Conway <[EMAIL PROTECTED]> wrote : > On Mon, 2004-11-22 at 07:57 +, PaweX Niewiadomski wrote: > > I saw discussion about bitmap indexes few weeks ago. I wonder if > > any of you is working on it (in secret)? > > For what it's worth, I don't know of an

Re: [HACKERS] Bitmap index

2004-11-24 Thread Neil Conway
On Mon, 2004-11-22 at 07:57 +, PaweX Niewiadomski wrote: > I saw discussion about bitmap indexes few weeks ago. I wonder if > any of you is working on it (in secret)? For what it's worth, I don't know of anyone working on them. > I will be chosing subject > of my master thesis and thougth abo

[HACKERS] Bitmap index

2004-11-23 Thread Pawe³ Niewiadomski
Hello, I saw discussion about bitmap indexes few weeks ago. I wonder if any of you is working on it (in secret)? I will be chosing subject of my master thesis and thougth about implementing bitmap indexes. -- **Paweł Niewiadomski**, new()foo-baz.com, http://new.foo-baz.com/ Podręcznik Administrat

[HACKERS] Bitmap index

2004-02-01 Thread Hans-Jürgen Schönig
Hello ... I remember that somebody was working on some sort of bitmap index some time ago (maybe 1 year or two). Does anybody know if there is some sort of half-ready code or so around? Does anybody know if there is somebody working on that? Regards, Hans -- Cybertec Geschwinde u Schoenig S

Re: [HACKERS] BITMAP Index support (and other DSS info.)

2003-01-02 Thread Sailesh Krishnamurthy
> "Shahbaz" == Shahbaz Chaudhary <[EMAIL PROTECTED]> writes: Shahbaz> There are bound to be people in the academia (grad Shahbaz> students, professors of CS, etc.) on this mailing list, Shahbaz> yet I see few RDBMS courses using postgresql as an Shahbaz> example. If people sti

[HACKERS] BITMAP Index support (and other DSS info.)

2002-12-29 Thread Shahbaz Chaudhary
Hi all, I've recently started using Postgresql and am impressed with how complete an RDBMS system it really is. It seems that while some corporations may balk at using it for day to day operational use, bringing in PGSQL for a more niche use will allow many professionals to see its potential. One

Re: [HACKERS] Bitmap index

2000-12-04 Thread Nathan Myers
On Mon, Dec 04, 2000 at 04:28:47PM +0100, [EMAIL PROTECTED] wrote: > > on other RDBMS (Oracle,etc...),there is an index called bitmap index > that greatly improve performance compared to btree index for boolean > value (such as for a sex value,it's either M or F),i would like to > know if such in

[HACKERS] Bitmap index

2000-12-04 Thread pejac
Hi, on other RDBMS (Oracle,etc...),there is an index called bitmap index that greatly improve performance compared to btree index for boolean value (such as for a sex value,it's either M or F),i would like to know if such index will be implemented inside PostgreSQL. Best regards, PEJAC Pascal

[HACKERS] Bitmap index

2000-12-04 Thread pejac
Hi, on other RDBMS (Oracle,etc...),there is an index called bitmap index that greatly improve performance compared to btree index for boolean value (such as for a sex value,it's either M or F),i would like to know if such index will be implemented inside PostgreSQL.

[HACKERS] Bitmap index

2000-12-01 Thread pejac
Hello, Please , excuse me for my bad english. One question on bitmaps index. In them Commercial data bases (oracle DB2), Let bitmap type index is supported.This index is used for fields of type sex or Boolean generally, would be it(he) supported in postgres??? If not is foreseen it??? Best re