Re: facet+shingle in autosuggest

2010-11-11 Thread Lukas Kahwe Smith 

On 11.11.2010, at 17:42, Erick Erickson wrote:

> I don't know all the implications here, but can't you just
> insert the StopwordFilterFactory before the ShingleFilterFactory
> and turn it loose?


havent tried this, but i would suspect that i would then get in trouble with 
stuff like "united states of america". it would then generate a shingle with 
"united states america" which in turn wouldnt generate a proper phrase search 
string.

one option of course would be to restrict the shingles to 2 words and then 
using the stop word filter would work as expected.

regards,
Lukas Kahwe Smith
m...@pooteeweet.org

Re: facet+shingle in autosuggest

2010-11-11 Thread Erick Erickson 
I don't know all the implications here, but can't you just
insert the StopwordFilterFactory before the ShingleFilterFactory
and turn it loose?

Best
Erick

On Thu, Nov 11, 2010 at 4:02 PM, Lukas Kahwe Smith wrote:

> Hi,
>
> I am using a facet.prefix search with shingle's in my autosuggest:
> positionIncrementGap="100" stored="false" multiValued="true">
>  
>
>
>
>  maxShingleSize="3" outputUnigrams="true"
> outputUnigramIfNoNgram="false" />
>  
>
>
> Now I would like to prevent stop words to appear in the suggestions:
>
> 
> 52
> 6
> 6
> 5
> 25
> 7
> 
>
> Here I would like to filter out the last 4 suggestions really. Is there a
> way I can sensibly bring in a stop word filter here? Actually in theory the
> stop words could appear as the first or second word as well.
>
> So I guess when producing shingle's I want to skip any stop word from being
> part of any shingle.
>
> regards,
> Lukas Kahwe Smith
> m...@pooteeweet.org
>
>
>
>

facet+shingle in autosuggest

2010-11-11 Thread Lukas Kahwe Smith 
Hi,

I am using a facet.prefix search with shingle's in my autosuggest:

  




  


Now I would like to prevent stop words to appear in the suggestions:


52
6
6
5
25
7


Here I would like to filter out the last 4 suggestions really. Is there a way I 
can sensibly bring in a stop word filter here? Actually in theory the stop 
words could appear as the first or second word as well.

So I guess when producing shingle's I want to skip any stop word from being 
part of any shingle.

regards,
Lukas Kahwe Smith
m...@pooteeweet.org