Re: [HACKERS] Bitmap index thoughts

2006-12-28 Thread Jie Zhang
On 12/27/06 3:16 AM, "Gavin Sherry" <[EMAIL PROTECTED]> wrote: > On Wed, 27 Dec 2006, Heikki Linnakangas wrote: > >> Jie Zhang wrote: >>> The "bitmap data segment" sounds good in terms of space. The problem is that >>> one bitmap is lik

Re: [HACKERS] Bitmap index thoughts

2006-12-28 Thread Jie Zhang
On 12/27/06 3:10 AM, "Gavin Sherry" <[EMAIL PROTECTED]> wrote: > On Wed, 27 Dec 2006, Heikki Linnakangas wrote: > >> Gavin Sherry wrote: >>> On Tue, 26 Dec 2006, Heikki Linnakangas wrote: for typical bitmap index use cases and most of the needed pages should stay in memory, but could w

Re: [HACKERS] Bitmap index thoughts

2006-12-27 Thread Jie Zhang
>> And instead of having separate LOV pages that store a number of LOV >> items, how about storing each LOV item on a page of it's own, and using >> the rest of the page to store the last chunk of the bitmap. That would >> eliminate one page access, but more importantly, maybe we could then get >>

Re: [HACKERS] Bitmap index status

2006-10-18 Thread Jie Zhang
On 10/18/06 2:41 AM, "Heikki Linnakangas" <[EMAIL PROTECTED]> wrote: > Hi, > > I don't want to harass you :), but what's the status with the bitmap > index code? Is there something I can do to help? Not at all. We will send you the new patch soon. Your comments are most appreciated. Thanks, Ji

Re: [HACKERS] Bitmap index status

2006-09-25 Thread Jie Zhang
Hi Mark, Thanks for doing the test. I checked out the link you provided below. I am a little confused about the goal of these tests. Do you plan to test the overall performance of postgreSQL on handling TPC-H queries? Thanks, Jie On 9/22/06 3:45 PM, "Mark Wong" <[EMAIL PROTECTED]&g

Re: [HACKERS] Bitmap index status

2006-09-23 Thread Jie Zhang
Gavin & Heikki, >> >> The handling of stream and hash bitmaps looks pretty complicated to me. >> All the bitmap-related nodes have logic to handle both types slightly >> differently. It all seems to come down to that if a subnode (or >> amgetbitmap in a bitmap index scan node) returns a StreamBit

Re: [HACKERS] Bitmap index status

2006-09-20 Thread Jie Zhang
On 9/19/06 5:15 AM, "Gavin Sherry" <[EMAIL PROTECTED]> wrote: > On Tue, 19 Sep 2006, Heikki Linnakangas wrote: > >> Jie Zhang wrote: >>> Hi Heikki and all, >>> >>> Please find the latest bitmap index patch in the attachment. This

Re: [HACKERS] Bitmap index status

2006-09-17 Thread Jie Zhang
Hi Heikki and all, I just sent the latest bitmap index patch to the list. I am not sure if there is any size limit for this mailing list. If you have received my previous email, please let me know. Thanks, Jie On 9/12/06 2:43 AM, "Heikki Linnakangas" <[EMAIL PROTECTED]> wrote: > Hi, > > What's

Re: [HACKERS] Bitmap index status

2006-09-12 Thread Jie Zhang
Hi Heikki, Gavin and I are trying to merge our changes together this week. We will post a new patch by the end of this week. This patch will include some style fixes, bug fixes, and the stream bitmap implementation. I will look into the problems you have mentioned in this email. Yes, vacuum curre

Re: [HACKERS] [PATCHES] WIP: bitmap indexes

2006-08-17 Thread Jie Zhang
On 8/17/06 12:29 PM, "Tom Lane" <[EMAIL PROTECTED]> wrote: > "Jie Zhang" <[EMAIL PROTECTED]> writes: >> This sounds good. Another problem is about ScalarArrayOpExpr support in >> current nodeBitmapIndexscan. This will not work for stream bitmaps.

Re: [HACKERS] [PATCHES] WIP: bitmap indexes

2006-08-17 Thread Jie Zhang
On 8/17/06 5:54 AM, "Tom Lane" <[EMAIL PROTECTED]> wrote: > "Jie Zhang" <[EMAIL PROTECTED]> writes: >> This sounds great. One thing I am concern about is that this will add the >> dependency of node types into the access methods. If we still keep

Re: [HACKERS] [PATCHES] WIP: bitmap indexes

2006-08-17 Thread Jie Zhang
> It occurs to me that what tbm_begin_iterate really is is a constructor > for a stream bitmap object that reads out the contents of a tbm bitmap > (we need a decent name for the non-stream data structure ... maybe > hash bitmap?). If we think of it like that then we can unify the > ideas some mor

Re: [HACKERS] [PATCHES] WIP: bitmap indexes

2006-08-16 Thread Jie Zhang
On 8/15/06 6:18 AM, "Tom Lane" <[EMAIL PROTECTED]> wrote: > Gavin Sherry <[EMAIL PROTECTED]> writes: >> On Mon, 14 Aug 2006, Tom Lane wrote: >>> Correct me if I'm wrong, but isn't the patch's present hacking on the >>> executor intended to make it happen like this? > >> Not really. It reads ahe

Re: [HACKERS] On-disk bitmap index patch

2006-07-27 Thread Jie Zhang
On 7/26/06 11:50 PM, "Tom Lane" <[EMAIL PROTECTED]> wrote: > "Jie Zhang" <[EMAIL PROTECTED]> writes: >> On 7/26/06 10:14 PM, "Tom Lane" <[EMAIL PROTECTED]> wrote: >>> ... A nonuniform distribution would probably mean that some

Re: [HACKERS] On-disk bitmap index patch

2006-07-26 Thread Jie Zhang
On 7/26/06 10:14 PM, "Tom Lane" <[EMAIL PROTECTED]> wrote: > Mark Kirkwood <[EMAIL PROTECTED]> writes: >> An obvious deduction is that the TPCH dataset is much more amenable to >> run compression than my synthetic Zipfian data was. The interesting >> question is how well "real" datasets are run

Re: [HACKERS] On-disk bitmap index patch

2006-07-26 Thread Jie Zhang
On 7/26/06 8:55 PM, "Luke Lonergan" <[EMAIL PROTECTED]> wrote: > Tom, > > On 7/26/06 7:26 AM, "Tom Lane" <[EMAIL PROTECTED]> wrote: > >> I wonder >> whether they oughtn't use 16-bit instead of 8-bit HRL_WORDs > > We tried variations from 8-bit to 32-bit and came to the conclusion that > 8-bit

Re: [HACKERS] On-disk bitmap index patch

2006-07-24 Thread Jie Zhang
On 7/24/06 6:04 PM, "Bruce Momjian" <[EMAIL PROTECTED]> wrote: > Jie Zhang wrote: >>> IIRC they quoted the cardinality of 1 as something that is still >>> faster than btree for several usecases. >>> >>> And also for AND-s of several

Re: [HACKERS] On-disk bitmap index patch

2006-07-24 Thread Jie Zhang
On 7/24/06 6:59 AM, "Hannu Krosing" <[EMAIL PROTECTED]> wrote: > Ühel kenal päeval, P, 2006-07-23 kell 20:25, kirjutas Tom Lane: >> Gavin Sherry <[EMAIL PROTECTED]> writes: >>> On Sun, 23 Jul 2006, Tom Lane wrote: However, the main problem I've got with this is that a new index AM is a

Re: [HACKERS] On-disk bitmap index patch

2006-07-23 Thread Jie Zhang
Thanks Tom and Gavin for your comments! Yes, this patch is generated against 8.2 in a short time. As long as the code is working, I post the patch to get some comments and help. >> >> * The xlog routines need help; they seem to not be updated for recent >> changes in the API for xlog recovery c

[HACKERS] On-disk bitmap index patch

2006-07-18 Thread Jie Zhang
Hi, I have posted a patch to the CVS head for on-disk bitmap index to pgsql-patches. If this can get in 8.2, that would be great. Any comments and suggestions are welcome. I still need to add several items: (1) README file in src/backend/access/bitmap. (2) Bitmap index documentation. (3) Hiding

[HACKERS] An In-memory Bitmap Index Bug

2005-07-23 Thread Jie Zhang
tached in this email. Sincerely, Jie Zhang Greenplum pgsql-bitmapscan-bugfix.patch Description: pgsql-bitmapscan-bugfix.patch ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq