Here is what it means by injecting at query time:
This is the text field definition i have in my schema
<fieldType name="text" class="solr.TextField" positionIncrementGap="100">
<analyzer type="index">
<tokenizer class="solr.WhitespaceTokenizerFactory"/>
<filter class="solr.TrimFilterFactory" />
<!--<filter class="solr.SynonymFilterFactory"
synonyms="synonyms.txt" ignoreCase="true" expand="true"/> -->
<filter class="solr.StopFilterFactory" ignoreCase="true"
words="stopwords.txt"/>
<filter class="solr.WordDelimiterFilterFactory"
generateWordParts="1" generateNumberParts="1" catenateWords="1"
catenateNumbers="1" catenateAll="0"/>
<filter class="solr.LowerCaseFilterFactory"/>
<filter class="solr.EnglishPorterFilterFactory"
protected="protwords.txt"/>
<filter class="solr.RemoveDuplicatesTokenFilterFactory"/>
</analyzer>
<analyzer type="query">
<tokenizer class="solr.WhitespaceTokenizerFactory"/>
<filter class="solr.TrimFilterFactory" />
<filter class="solr.SynonymFilterFactory" synonyms="synonyms.txt"
ignoreCase="true" expand="true"/>
<filter class="solr.StopFilterFactory" ignoreCase="true"
words="stopwords.txt"/>
<filter class="solr.WordDelimiterFilterFactory"
generateWordParts="1" generateNumberParts="1" catenateWords="1"
catenateNumbers="1" catenateAll="1"/>
<filter class="solr.LowerCaseFilterFactory"/>
<filter class="solr.EnglishPorterFilterFactory"
protected="protwords.txt"/>
<filter class="solr.RemoveDuplicatesTokenFilterFactory"/>
</analyzer>
</fieldType>
and a catchall field of type 'text'
<field name="text" type="text" indexed="true" stored="false"
multiValued="true"/>
and i do a copyfield to copy my important fields that needs to be searched
during phrase matching to 'text' field at index time.
and the default search field in my case is this catchall field
<defaultSearchField>text</defaultSearchField>
You can see that i've commented out the syn filter at index time but
included it at query time.
Just add your synonyms to the synonyms.txt file and they'll be taken in to
account during query time. You can check that in the parsedquery_tostring
using ur admin tool (localhost:8983/solr/admin/form.jsp)
Like its been discussed here, injecting at index time is helpful in finding
more matches except that everytime u add synonyms, u'll have to re-index
data. It wasn't ideal in my case as my index is huge so i had to do it only
at query time, but that poses some problems sometimes..
i have a couple of unanswered questions , if you may know the answer please
help me.
http://www.nabble.com/Index-time-synonyms-td15073889.html
http://www.nabble.com/solr-synonyms-behaviour-td15051211.html
Ravish Bhagdev wrote:
>
> I see, thanks a lot for this, makes things clear now.
>
> So just to make sure I understand this bit, by "injecting synonyms at
> query time" you mean basically adding terms implicitly to keywords
> behind the scenes before passing it to solr? Or is there are more
> conventional method or interface that is being suggested?
>
> Thanks for all the help!
>
> Ravish
>
> On Jan 25, 2008 3:59 PM, Erick Erickson <[EMAIL PROTECTED]> wrote:
>> To me, it's really a question of where the work should be done given your
>> problem space. Injecting synonyms at index time allows the queries to be
>> simpler/faster. Injecting the synonyms at query time gets complex but is
>> more flexible.
>>
>> As always, it's a time/space tradeoff. If you're willing to pay the space
>> penalty for increased query speed, inject at index time. Otherwise
>> you can inject at query time.
>>
>> And the query-time injection performance hit may not be trivial.
>> Consider,
>> for instance, span queries. Do you want to pay the price at query time
>> for,
>> say a BooleanQuery that is composed of 5 SpanQueries where each
>> term in each SpanQuery consists of several OR clauses because of
>> synonym injection? Perhaps you do and perhaps you don't. It all depends
>> upon what your data looks like and what your performance criteria are.
>>
>> And you can do other tricks. Consider rather than indexing all the terms,
>> only index the canonical term. That is, consider "hit" and the synonyms
>> "strike", "popular", "punch". you could index "hit" for any of the 4
>> terms,
>> then do the same substitution for your query. Which would make your
>> index smaller *and* your queries faster.
>>
>> But you're right. Injecting synonyms at index time really requires a
>> fixed
>> synonym list that doesn't vary by user. So if you want synonym
>> lists on a per-user basis, you're probably going to have to inject
>> synonyms
>> at query time.
>>
>> Best
>> Erick
>>
>>
>> On Jan 25, 2008 9:46 AM, Ravish Bhagdev <[EMAIL PROTECTED]> wrote:
>>
>> > Yes, I'm fairly new as well.
>> >
>> > So do you mean adding words to the query effectively doing an "or"
>> > between synonymous terms? That sounds simple way of doing it, if this
>> > works, what makes indexing with synonyms useful?
>> >
>> > Ravish
>> >
>> > On Jan 25, 2008 2:42 PM, Jon Lehto <[EMAIL PROTECTED]> wrote:
>> > > Hi Ravish,
>> > >
>> > > You may want to think about the synonym dictionary as being a tool on
>> > the side, rather than each indexed document having a copy of the
>> synonyms.
>> > At indexing time, one might normalize synonyms to a single value, and
>> at
>> > query time do the same to get the match.
>> > >
>> > > Alternately, use the synonym dictionary at run-time to expand a
>> > > user's query terms, like a thesaurus.
>> > >
>> > > That said, I'm new to the tool, and not clear on how synonyms are
>> > implemented.
>> > >
>> > > Jon
>> > > =====================
>> > > From: Ravish Bhagdev <[EMAIL PROTECTED]>
>> > > Date: 2008/01/25 Fri AM 08:24:33 CST
>> > > To: solr-user@lucene.apache.org
>> > > Subject: Is it possible to add synonyms run time?
>> > >
>> > >
>> > > As I understood from available documentation, synonyms need to be
>> > > defined before starting the indexing process. Is it possible to add
>> > > synonyms at run time such that all index fields of all documents get
>> > > updated? Does it work for newly added documents atleast?
>> > >
>> > > Also, how to make each user of application define his own set of
>> > > synonyms that others should be oblivious to (others get normal
>> results
>> > > without synon considered)
>> > >
>> > > Thanks,
>> > > Ravish
>> > >
>> > >
>> >
>>
>
>
--
View this message in context:
http://www.nabble.com/Re%3A-Is-it-possible-to-add-synonyms-run-time--tp15089111p15092894.html
Sent from the Solr - User mailing list archive at Nabble.com.
