Re: Synonym filter with support for phrases?

2009-04-28 Thread Dawid Weiss
Apologies for the delay, guys. I tried to solve certain issues that didn't pop up in my application (as Kirill said, the problem is indeed quite complex). I didn't find all the answers I had been looking for, but nonetheless -- the patch that works for my needs is in JIRA. I would be really

Re: Synonym filter with support for phrases?

2009-04-23 Thread Michael McCandless
On Wed, Apr 22, 2009 at 5:12 AM, Earwin Burrfoot ear...@gmail.com wrote: Your synonyms will break if you try searching for phrases. Building on your example, food place in new york will find nothing, because 'place' and 'in' share the same position. It'd be great to get multi-word synonyms

Re: Synonym filter with support for phrases?

2009-04-23 Thread Earwin Burrfoot
On Wed, Apr 22, 2009 at 5:12 AM, Earwin Burrfoot ear...@gmail.com wrote: Your synonyms will break if you try searching for phrases. Building on your example, food place in new york will find nothing, because 'place' and 'in' share the same position. It'd be great to get multi-word synonyms

Re: Synonym filter with support for phrases?

2009-04-23 Thread Dawid Weiss
It'd be great to get multi-word synonyms fully working... I agree -- this is something that seems to be useful for a wider bunch of people. How would you change how Lucene indexes token positions to do this correctly? Kirill has some interesting points to this. I have a busy day today,

Re: Synonym filter with support for phrases?

2009-04-23 Thread Earwin Burrfoot
engine. So guys looking for MSU CMC really want to get Московский Государственный Университет, факультет ВМиК and his friends. And? How often do they extend this particular phrase with further terms? They don't need to. Variations of this phrase alone killed my first several approaches to

Re: Synonym filter with support for phrases?

2009-04-22 Thread Earwin Burrfoot
Hello everyone, I'm looking for feedback and thoughts on the following problem (it's more of development than user-centered problem, hope the dev list is appropriate): - a token stream is given, - a set of synonyms is given, where synonyms are token sequences to be matched and token

Re: Synonym filter with support for phrases?

2009-04-22 Thread Dawid Weiss
Your synonyms will break if you try searching for phrases. Good point, I did write that filter, but I never actually got to searching for exact phrases in it (there was a very specific scenario and we used prefix queries which worked quite well). Building on your example, food place in

Re: Synonym filter with support for phrases?

2009-04-22 Thread Earwin Burrfoot
Building on your example, food place in new york will find nothing, because 'place' and 'in' share the same position. You're right, but is it such a big problem in real life? Well, everyone has his own requirements for the search quality. For us it was a problem. User enters a query, then

Re: Synonym filter with support for phrases?

2009-04-22 Thread Dawid Weiss
Well, everyone has his own requirements for the search quality. For us it was a problem. The topic is subjective... I don't see this as a deterioration in search quality. Let me explain. Your example concerns phrase queries, so somebody would have to keep adding terms to a phrase. My

Re: Synonym filter with support for phrases?

2009-04-22 Thread Earwin Burrfoot
Your example concerns phrase queries, so somebody would have to keep adding terms to a phrase. My experience with open search queries (I had access to a larger slice of queries from Microsoft Live) is that phrases are a minority of all searches. In the most common case, people will look for a

Re: Synonym filter with support for phrases?

2009-04-22 Thread Dawid Weiss
engine. So guys looking for MSU CMC really want to get Московский Государственный Университет, факультет ВМиК and his friends. And? How often do they extend this particular phrase with further terms? It must be fun to have an index running concurrently on multi language synonyms, mixing the