Re: [HACKERS] GIN - Generalized Inverted iNdex. Try 3.

2006-04-28 Thread Teodor Sigaev
"Takes clustering into account" means nothing. We don't need that. Any such consideration would be handled by the AM-specific amcostestimate function. Ok, I see. Sorry for misunderstanding. I thought that planner use that. -- Teodor Sigaev E-mail: [EMAIL PROT

Re: [HACKERS] GIN - Generalized Inverted iNdex. Try 3.

2006-04-28 Thread Tom Lane
Teodor Sigaev <[EMAIL PROTECTED]> writes: >> Huh? Why two? Either you are allowed to cluster on indexes of this >> type, or you're not. I don't see the point of any other distinction. > amclusterable - as you suggest: Does cluster command something or not? This is what we need. > amclustered

Re: [HACKERS] GIN - Generalized Inverted iNdex. Try 3.

2006-04-28 Thread Teodor Sigaev
ok. amskipcheck? Oh, I was thinking of having VACUUM put the heap tuple count into the struct and then amvacuumcleanup could copy it over to the index tuple count. A "skipcheck" flag only solves the cosmetic problem of not getting the warning, it doesn't fix things so that the correct count e

Re: [HACKERS] GIN - Generalized Inverted iNdex. Try 3.

2006-04-28 Thread Tom Lane
Teodor Sigaev <[EMAIL PROTECTED]> writes: >> We could probably fix that by adding it to the stats structs that are >> passed around during VACUUM. I'd rather not hardwire a GIN special case >> in vacuum.c as per your "quick hack". > ok. amskipcheck? Oh, I was thinking of having VACUUM put the

Re: [HACKERS] GIN - Generalized Inverted iNdex. Try 3.

2006-04-28 Thread Teodor Sigaev
We could probably fix that by adding it to the stats structs that are passed around during VACUUM. I'd rather not hardwire a GIN special case in vacuum.c as per your "quick hack". ok. amskipcheck? Here I think it would be best to add an indclusterable column to pg_am. GIN is _always_ clust

Re: [HACKERS] GIN - Generalized Inverted iNdex. Try 3.

2006-04-28 Thread Martijn van Oosterhout
On Fri, Apr 28, 2006 at 10:14:09AM -0400, Tom Lane wrote: > Here I think it would be best to add an indclusterable column to pg_am. > Actually, does clustering on *any* current index type except btree make > sense? None of them have semantically interesting index ordering > AFAIR, so maybe we shou

Re: [HACKERS] GIN - Generalized Inverted iNdex. Try 3.

2006-04-28 Thread Tom Lane
Teodor Sigaev <[EMAIL PROTECTED]> writes: > One problem: ambulkdelete hasn't any access to heap or heap's statistics > (num_tuples in scan_index() and vacuum_index() in vacuum.c). So, ambulkdelete > can't set stats->num_index_tuples equal to num_tuples. We could probably fix that by adding it to

Re: [HACKERS] GIN - Generalized Inverted iNdex. Try 3.

2006-04-28 Thread Teodor Sigaev
There's a definitional issue here, which is what does it mean to be counting index tuples. I think GIN could bypass the VACUUM error check by always returning the heap tuple count as its index tuple count. This One problem: ambulkdelete hasn't any access to heap or heap's statistics (num_tupl

Re: [HACKERS] GIN - Generalized Inverted iNdex. Try 3.

2006-04-27 Thread Teodor Sigaev
The ideal thing would be for GIN to return a count of the number of distinct heap tuples referenced by the entries in the index, but I suppose that would be impractical for VACUUM to compute. Of course, in this case we should collect heap pointers in memory.. -- Teodor Sigaev

Re: [HACKERS] GIN - Generalized Inverted iNdex. Try 3.

2006-04-27 Thread Teodor Sigaev
How about adding a column to pg_am indicating "these indexes must always keep same tuple count as heap". This would be true for all current AMs, false for GIN. Yes, it's simplest solution, but it doesn't check of index consistency. Possible, we can count number of itempointers to heap tuple du

Re: [HACKERS] GIN - Generalized Inverted iNdex. Try 3.

2006-04-27 Thread Tom Lane
Alvaro Herrera <[EMAIL PROTECTED]> writes: > Teodor Sigaev wrote: >> We appreciate any comments, help and suggestions. For third item we haven't >> idea how fix it except to exclude GIN index from check. > How about adding a column to pg_am indicating "these indexes must always > keep same tuple

Re: [HACKERS] GIN - Generalized Inverted iNdex. Try 3.

2006-04-27 Thread Alvaro Herrera
Teodor Sigaev wrote: > * GIN stores several ItemPointer to heap tuple, so VACUUM FULL produces > this warning message: > WARNING: index "idx" contains 88395 row versions, but table contains > 51812 row versions > HINT: Rebuild the index with REINDEX. > > We a

Re: [HACKERS] GIN - Generalized Inverted iNdex. Try 3.

2006-04-27 Thread Teodor Sigaev
I took the liberty of revising your README.txt as a native speaker :) I cleaned up the grammar a lot, etc. Thank you very much. I added your README to patch. New version of GIN is available: http://www.sigaev.ru/gin/gin.gz http://www.sigaev.ru/gin/README.txt Changes from Try 2: * add reg

Re: [HACKERS] GIN - Generalized Inverted iNdex. Try 2.

2006-04-26 Thread Christopher Kings-Lynne
Oh I can't read - ignore me :) Teodor Sigaev wrote: Changes from previous patch: * add support for tsearch2 * add 'fuzzy' limit * fixes ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgre

Re: [HACKERS] GIN - Generalized Inverted iNdex. Try 2.

2006-04-26 Thread Christopher Kings-Lynne
What changed between Try 1 and Try 2? Teodor Sigaev wrote: We (me and Oleg) are glad to present GIN to PostgreSQL. If community will agree, we will commit it to HEAD branch. ---(end of broadcast)--- TIP 6: explain analyze is your friend

Re: [HACKERS] GIN - Generalized Inverted iNdex. Try 2.

2006-04-26 Thread Teodor Sigaev
I just built this, and noticed that the regression test for "opr_sanity" fails with your patch. I attached the regression.diffs. Sorry, this part isn't done yet, because we are waiting of community decision.. We don't add regression test yet. If community don't like to include GIN in core, w

Re: [HACKERS] GIN - Generalized Inverted iNdex. Try 2.

2006-04-26 Thread Jeremy Drake
On Wed, 26 Apr 2006, Teodor Sigaev wrote: > > We (me and Oleg) are glad to present GIN to PostgreSQL. If community will > agree, we will commit it to HEAD branch. > > http://www.sigaev.ru/gin/gin.gz > http://www.sigaev.ru/gin/README.txt > > Install: > % cd pgsql > % zcat gin.gz | patch -p0 > make

[HACKERS] GIN - Generalized Inverted iNdex. Try 2.

2006-04-26 Thread Teodor Sigaev
We (me and Oleg) are glad to present GIN to PostgreSQL. If community will agree, we will commit it to HEAD branch. http://www.sigaev.ru/gin/gin.gz http://www.sigaev.ru/gin/README.txt Install: % cd pgsql % zcat gin.gz | patch -p0 make and initdb, install tsearch2 Changes from previous patch

Re: [HACKERS] GIN - Generalized Inverted iNdex.

2006-04-07 Thread Oleg Bartunov
On Fri, 7 Apr 2006, Martijn van Oosterhout wrote: On Fri, Apr 07, 2006 at 05:05:12PM +0400, Teodor Sigaev wrote: We (me and Oleg) are glad to present GIN to PostgreSQL. If community will agree, we will commit it to HEAD branch. http://www.sigaev.ru/gin/gin.gz http://www.sigaev.ru/gin/README

Re: [HACKERS] GIN - Generalized Inverted iNdex.

2006-04-07 Thread Oleg Bartunov
One addition - some results of Gin testing is available: http://www.sai.msu.su/~megera/oddmuse/index.cgi/GinTest Oleg On Fri, 7 Apr 2006, Teodor Sigaev wrote: We (me and Oleg) are glad to present GIN to PostgreSQL. If community will agree, we will commit it to HEAD branch. http://www.sigaev.

Re: [HACKERS] GIN - Generalized Inverted iNdex.

2006-04-07 Thread Martijn van Oosterhout
On Fri, Apr 07, 2006 at 05:05:12PM +0400, Teodor Sigaev wrote: > We (me and Oleg) are glad to present GIN to PostgreSQL. If community will > agree, we will commit it to HEAD branch. > > http://www.sigaev.ru/gin/gin.gz > http://www.sigaev.ru/gin/README Fantastic! I was thinking about this a while

[HACKERS] GIN - Generalized Inverted iNdex.

2006-04-07 Thread Teodor Sigaev
We (me and Oleg) are glad to present GIN to PostgreSQL. If community will agree, we will commit it to HEAD branch. http://www.sigaev.ru/gin/gin.gz http://www.sigaev.ru/gin/README Install: % cd pgsql % zcat gin.gz | patch -p0 make and initdb README: Gin for PostgreSQL Gin stands for Generali