Re: [HACKERS] Include Lists for Text Search

2008-03-10 Thread Oleg Bartunov
On Sun, 9 Mar 2008, Tom Lane wrote: Simon Riggs [EMAIL PROTECTED] writes: I've coded a small patch to allow CaseSensitive synonyms. Applied with corrections (it'd be good if you at least pretended to test stuff before submitting it). Would a similar parameter be useful for any of the other

Re: [HACKERS] Include Lists for Text Search

2008-03-10 Thread Simon Riggs
On Sun, 2008-03-09 at 23:03 -0400, Tom Lane wrote: Simon Riggs [EMAIL PROTECTED] writes: I've coded a small patch to allow CaseSensitive synonyms. Applied with corrections (it'd be good if you at least pretended to test stuff before submitting it). It is a frequent accusation of yours that

Re: [HACKERS] Include Lists for Text Search

2008-03-10 Thread Andrew Dunstan
Simon Riggs wrote: As Greg mentions on another thread, not all patches are *intended* to be production quality by their authors. Many patches are shared for the purpose of eliciting general feedback. You yourself encourage a group development approach and specifically punish those people

Re: [HACKERS] Include Lists for Text Search

2008-03-10 Thread Simon Riggs
On Mon, 2008-03-10 at 08:24 -0400, Andrew Dunstan wrote: Simon Riggs wrote: As Greg mentions on another thread, not all patches are *intended* to be production quality by their authors. Many patches are shared for the purpose of eliciting general feedback. You yourself encourage a group

Re: [HACKERS] Include Lists for Text Search

2008-03-10 Thread Tom Lane
Andrew Dunstan [EMAIL PROTECTED] writes: I think if you post something marked Work In Progress, there is an implied commitment on your part to post something complete at a later stage. It *wasn't* marked Work In Progress, and Simon went out of his way to cross-post it to -patches, where the

Re: [HACKERS] Include Lists for Text Search

2008-03-10 Thread Simon Riggs
On Mon, 2008-03-10 at 09:42 -0400, Andrew Dunstan wrote: I think if you post something marked Work In Progress, there is an implied commitment on your part to post something complete at a later stage. So if it's forgotten you would be the one doing the forgetting. ;-) But if they aren't on

Re: [HACKERS] Include Lists for Text Search

2008-03-10 Thread Andrew Dunstan
Simon Riggs wrote: On Mon, 2008-03-10 at 08:24 -0400, Andrew Dunstan wrote: Simon Riggs wrote: As Greg mentions on another thread, not all patches are *intended* to be production quality by their authors. Many patches are shared for the purpose of eliciting general feedback. You

Re: [HACKERS] Include Lists for Text Search

2008-03-10 Thread Tom Lane
Oleg Bartunov [EMAIL PROTECTED] writes: On Sun, 9 Mar 2008, Tom Lane wrote: Would a similar parameter be useful for any of the other dictionary types? There are many things desirable to do with dictionaries, for example, say dictionary to return an original word plus it's normal form.

Re: [HACKERS] Include Lists for Text Search

2008-03-10 Thread Simon Riggs
On Mon, 2008-03-10 at 10:01 -0400, Tom Lane wrote: In future perhaps I should take it as a given that Simon doesn't expect his patches to be applied? I think you should take it as a given that Simon would like to try to work together, sharing ideas and code, without insults and public derision

Re: [PATCHES] [HACKERS] Include Lists for Text Search

2008-03-10 Thread Bruce Momjian
Andrew Dunstan wrote: Simon Riggs wrote: As Greg mentions on another thread, not all patches are *intended* to be production quality by their authors. Many patches are shared for the purpose of eliciting general feedback. You yourself encourage a group development approach and

Re: [HACKERS] Include Lists for Text Search

2008-03-10 Thread Teodor Sigaev
Hmm, I can see how some middleware would help with folding or not folding the input token, but what about the words coming from the dictionary file (particularly the *output* lexeme)? It's not apparent to me that it's sensible to try to control that from outside the dictionary. Right now I see

Re: [PATCHES] [HACKERS] Include Lists for Text Search

2008-03-10 Thread Teodor Sigaev
Well, if you think this can/should be done somewhere outside the dictionary, should I revert the applied patch? No, that patch is about case sensitivity of synonym dictionary. I suppose, Simon wants to replace 'bill' to 'account', but doesn't want to get 'account Clinton' For another

Re: [PATCHES] [HACKERS] Include Lists for Text Search

2008-03-10 Thread Tom Lane
Teodor Sigaev [EMAIL PROTECTED] writes: Hmm, I can see how some middleware would help with folding or not folding the input token, but what about the words coming from the dictionary file (particularly the *output* lexeme)? It's not apparent to me that it's sensible to try to control that

Re: [HACKERS] Include Lists for Text Search

2008-03-10 Thread Teodor Sigaev
Right now I see an significant advantage of such layer: two possible extension of dictionary (filtering and storing original form) are One more extension: drop too long words. For example, decrease limit of max length of word to prevent long to be indexed - word with 100 characters is

Re: [HACKERS] Include Lists for Text Search

2008-03-09 Thread Tom Lane
Simon Riggs [EMAIL PROTECTED] writes: I've coded a small patch to allow CaseSensitive synonyms. Applied with corrections (it'd be good if you at least pretended to test stuff before submitting it). Would a similar parameter be useful for any of the other dictionary types?

Re: [HACKERS] Include Lists for Text Search

2007-09-26 Thread Bruce Momjian
This has been saved for the 8.4 release: http://momjian.postgresql.org/cgi-bin/pgpatches_hold --- Simon Riggs wrote: On Mon, 2007-09-10 at 10:21 -0400, Tom Lane wrote: Oleg Bartunov [EMAIL PROTECTED] writes:

[HACKERS] Include Lists for Text Search

2007-09-10 Thread Simon Riggs
It seems possible to write your own functions to support various possibilities with text search. One of the more common thoughts is to have a list of words that you would like to include, i.e. the opposite of a stop word list. There are clear indications that indexing too many words is a

Re: [HACKERS] Include Lists for Text Search

2007-09-10 Thread Heikki Linnakangas
Simon Riggs wrote: It seems possible to write your own functions to support various possibilities with text search. One of the more common thoughts is to have a list of words that you would like to include, i.e. the opposite of a stop word list. There are clear indications that indexing

Re: [HACKERS] Include Lists for Text Search

2007-09-10 Thread Oleg Bartunov
On Mon, 10 Sep 2007, Simon Riggs wrote: It seems possible to write your own functions to support various possibilities with text search. One of the more common thoughts is to have a list of words that you would like to include, i.e. the opposite of a stop word list. There are clear

Re: [HACKERS] Include Lists for Text Search

2007-09-10 Thread Simon Riggs
On Mon, 2007-09-10 at 12:58 +0100, Heikki Linnakangas wrote: Simon Riggs wrote: It seems possible to write your own functions to support various possibilities with text search. One of the more common thoughts is to have a list of words that you would like to include, i.e. the opposite

Re: [HACKERS] Include Lists for Text Search

2007-09-10 Thread Oleg Bartunov
On Mon, 10 Sep 2007, Simon Riggs wrote: On Mon, 2007-09-10 at 12:58 +0100, Heikki Linnakangas wrote: Simon Riggs wrote: It seems possible to write your own functions to support various possibilities with text search. One of the more common thoughts is to have a list of words that you would

Re: [HACKERS] Include Lists for Text Search

2007-09-10 Thread Oleg Bartunov
On Mon, 10 Sep 2007, Simon Riggs wrote: On Mon, 2007-09-10 at 16:10 +0400, Oleg Bartunov wrote: On Mon, 10 Sep 2007, Simon Riggs wrote: It seems possible to write your own functions to support various possibilities with text search. One of the more common thoughts is to have a list of words

Re: [HACKERS] Include Lists for Text Search

2007-09-10 Thread Teodor Sigaev
How does that allow me to limit the number of words to a known list? If all dictionaries returns NULL for token the this token will not be indexed at all. -- Teodor Sigaev E-mail: [EMAIL PROTECTED] WWW:

Re: [HACKERS] Include Lists for Text Search

2007-09-10 Thread Simon Riggs
On Mon, 2007-09-10 at 16:10 +0400, Oleg Bartunov wrote: On Mon, 10 Sep 2007, Simon Riggs wrote: It seems possible to write your own functions to support various possibilities with text search. One of the more common thoughts is to have a list of words that you would like to include,

Re: [HACKERS] Include Lists for Text Search

2007-09-10 Thread Teodor Sigaev
There are clear indications that indexing too many words is a problem for both GIN and GIST. If people already know what they'll be looking GIN doesn't depend strongly on number of words. It has log(N) behaviour for numbers of words because of using B-Tree over words. -- Teodor Sigaev

Re: [HACKERS] Include Lists for Text Search

2007-09-10 Thread Simon Riggs
On Mon, 2007-09-10 at 16:35 +0400, Oleg Bartunov wrote: On Mon, 10 Sep 2007, Simon Riggs wrote: On Mon, 2007-09-10 at 16:10 +0400, Oleg Bartunov wrote: On Mon, 10 Sep 2007, Simon Riggs wrote: It seems possible to write your own functions to support various possibilities with text

Re: [HACKERS] Include Lists for Text Search

2007-09-10 Thread Simon Riggs
On Mon, 2007-09-10 at 16:48 +0400, Teodor Sigaev wrote: There are clear indications that indexing too many words is a problem for both GIN and GIST. If people already know what they'll be looking GIN is great, sorry if that sounded negative. GIN doesn't depend strongly on number of words.

Re: [HACKERS] Include Lists for Text Search

2007-09-10 Thread Oleg Bartunov
On Mon, 10 Sep 2007, Simon Riggs wrote: On Mon, 2007-09-10 at 16:48 +0400, Teodor Sigaev wrote: There are clear indications that indexing too many words is a problem for both GIN and GIST. If people already know what they'll be looking GIN is great, sorry if that sounded negative. GIN

Re: [HACKERS] Include Lists for Text Search

2007-09-10 Thread Oleg Bartunov
On Mon, 10 Sep 2007, Simon Riggs wrote: On Mon, 2007-09-10 at 16:35 +0400, Oleg Bartunov wrote: On Mon, 10 Sep 2007, Simon Riggs wrote: On Mon, 2007-09-10 at 16:10 +0400, Oleg Bartunov wrote: On Mon, 10 Sep 2007, Simon Riggs wrote: It seems possible to write your own functions to support

Re: [HACKERS] Include Lists for Text Search

2007-09-10 Thread Tom Lane
Oleg Bartunov [EMAIL PROTECTED] writes: On Mon, 10 Sep 2007, Simon Riggs wrote: Can we include that functionality now? This could be realized very easyly using dict_strict, which returns only known words, and mapping contains only this dictionary. So, feel free to write it and submit. ...

Re: [HACKERS] Include Lists for Text Search

2007-09-10 Thread Oleg Bartunov
On Mon, 10 Sep 2007, Tom Lane wrote: Oleg Bartunov [EMAIL PROTECTED] writes: On Mon, 10 Sep 2007, Simon Riggs wrote: Can we include that functionality now? This could be realized very easyly using dict_strict, which returns only known words, and mapping contains only this dictionary. So,

Re: [HACKERS] Include Lists for Text Search

2007-09-10 Thread Simon Riggs
On Mon, 2007-09-10 at 10:21 -0400, Tom Lane wrote: Oleg Bartunov [EMAIL PROTECTED] writes: On Mon, 10 Sep 2007, Simon Riggs wrote: Can we include that functionality now? This could be realized very easyly using dict_strict, which returns only known words, and mapping contains only this