Re: [HACKERS] prefix btree implementation

2005-10-07 Thread Simon Riggs
On Thu, 2005-10-06 at 22:43 -0400, Bruce Momjian wrote: Jim C. Nasby wrote: On Wed, Oct 05, 2005 at 03:40:43PM -0700, Qingqing Zhou wrote: We do the prefix sharing when we build up index only, never on the fly. So are you saying that inserts of new data wouldn't make any use of this?

Re: [HACKERS] prefix btree implementation

2005-10-07 Thread Bruce Momjian
OK, TODO updated: * Consider compressing indexes by storing key values duplicated in several rows as a single index entry This is difficult because it requires datatype-specific knowledge. --- Simon Riggs wrote: On

Re: [HACKERS] prefix btree implementation

2005-10-06 Thread Simon Riggs
On Wed, 2005-10-05 at 00:50 -0400, Tom Lane wrote: Qingqing Zhou [EMAIL PROTECTED] writes: 1/ What types of prefix compression shall we support? Given the requirement of datatype independence, this idea seems a complete nonstarter to me... It would be possible to compress on similar

Re: [HACKERS] prefix btree implementation

2005-10-06 Thread Tom Lane
Simon Riggs [EMAIL PROTECTED] writes: It might be worth teaching the optimiser that if it has an index on an immutable function that if we have WHERE x = k and a functional index on f(x) then we can access the functional index with f(x) = f(k), as long as we also reapply the original WHERE

Re: [HACKERS] prefix btree implementation

2005-10-06 Thread Martijn van Oosterhout
On Thu, Oct 06, 2005 at 08:53:25AM +0100, Simon Riggs wrote: It would be possible to compress on similar values, since we know the output of the comparison in the final stage of the sort of the index build. That wouldn't need to rely upon anything to do with the datatype, since they are equal

Re: [HACKERS] prefix btree implementation

2005-10-06 Thread Simon Riggs
On Thu, 2005-10-06 at 09:38 -0400, Tom Lane wrote: Simon Riggs [EMAIL PROTECTED] writes: It might be worth teaching the optimiser that if it has an index on an immutable function that if we have WHERE x = k and a functional index on f(x) then we can access the functional index with f(x)

Re: [HACKERS] prefix btree implementation

2005-10-06 Thread Jim C. Nasby
On Wed, Oct 05, 2005 at 03:40:43PM -0700, Qingqing Zhou wrote: We do the prefix sharing when we build up index only, never on the fly. So are you saying that inserts of new data wouldn't make any use of this? ISTM that greatly reduces the usefulness, though I'm not objecting because compression

Re: [HACKERS] prefix btree implementation

2005-10-06 Thread Bruce Momjian
Jim C. Nasby wrote: On Wed, Oct 05, 2005 at 03:40:43PM -0700, Qingqing Zhou wrote: We do the prefix sharing when we build up index only, never on the fly. So are you saying that inserts of new data wouldn't make any use of this? ISTM that greatly reduces the usefulness, though I'm not

Re: [HACKERS] prefix btree implementation

2005-10-06 Thread Qingqing Zhou
On Thu, 6 Oct 2005, Jim C. Nasby wrote: On Wed, Oct 05, 2005 at 03:40:43PM -0700, Qingqing Zhou wrote: We do the prefix sharing when we build up index only, never on the fly. So are you saying that inserts of new data wouldn't make any use of this? ISTM that greatly reduces the

Re: [HACKERS] prefix btree implementation

2005-10-05 Thread Tom Lane
Qingqing Zhou [EMAIL PROTECTED] writes: 1/ What types of prefix compression shall we support? Given the requirement of datatype independence, this idea seems a complete nonstarter to me... regards, tom lane ---(end of

Re: [HACKERS] prefix btree implementation

2005-10-05 Thread Alvaro Herrera
On Wed, Oct 05, 2005 at 12:50:41AM -0400, Tom Lane wrote: Qingqing Zhou [EMAIL PROTECTED] writes: 1/ What types of prefix compression shall we support? Given the requirement of datatype independence, this idea seems a complete nonstarter to me... How about having each type optionally

Re: [HACKERS] prefix btree implementation

2005-10-05 Thread Bricklen Anderson
Qingqing Zhou wrote: I am not sure if this idea was mentioned before. The basic prefix btree idea is quite straightforward, i.e., try to compress the key items within a data page by sharing the common prefix. Thus the fanout of the page is increased and the benefits is obvious

Re: [HACKERS] prefix btree implementation

2005-10-05 Thread Qingqing Zhou
Bricklen Anderson [EMAIL PROTECTED] wrote Oracle implements something similar called index compression, but I believe it is only for common column values. I haven't checked in versions9r1 so maybe there are other options implemented by now. Jonathan Lewis describes some pros and cons

Re: [HACKERS] prefix btree implementation

2005-10-05 Thread Qingqing Zhou
Alvaro Herrera [EMAIL PROTECTED] wrote On Wed, Oct 05, 2005 at 12:50:41AM -0400, Tom Lane wrote: Qingqing Zhou [EMAIL PROTECTED] writes: 1/ What types of prefix compression shall we support? Given the requirement of datatype independence, this idea seems a complete nonstarter to me...

Resultset duplicates (was Re: [HACKERS] prefix btree implementation)

2005-10-05 Thread Richard Huxton
Qingqing Zhou wrote: Oracle 9 uses the grammar like this: CREATE INDEX ... [ COMPRESS number_of_first_columns ] So it gives the flexibility of choosing optimal number of coulumns to the user. The script mentioned in the article guesses the optimal number by estimating the size of each

Re: [HACKERS] prefix btree implementation

2005-10-05 Thread Junji TERAMOTO
Hello all, I also was examining a similar compression method just. Qingqing Zhou wrote: We can find a way to handle the above case, but it is better to find a general way to handle any data types(include UDT). Each type optionally provide the required routines could be a way, more details?

[HACKERS] prefix btree implementation

2005-10-04 Thread Qingqing Zhou
I am not sure if this idea was mentioned before. The basic prefix btree idea is quite straightforward, i.e., try to compress the key items within a data page by sharing the common prefix. Thus the fanout of the page is increased and the benefits is obvious theorectically. Consider the following