Re: PhraseQuery with term positions

2010-01-19 Thread Avi Rosenschein
Index is pretty large (50GB, divided into 8 shards). I'm afraid I would start running into memory issues by adding the stop words (though it is definitely something I would like to test at some point). My question was more to try to understand if this was known behavior in lucene, since I can't re

Re: PhraseQuery with term positions

2010-01-19 Thread Erick Erickson
How big is your index? Because the simplest thing would be to just not remove stopwords at index or query time. Perhaps in a duplicate field depending upon your needs. Erick On Tue, Jan 19, 2010 at 6:50 AM, Avi Rosenschein wrote: > Hi, > > I am using PhraseQuery with explicitly set term position

PhraseQuery with term positions

2010-01-19 Thread Avi Rosenschein
Hi, I am using PhraseQuery with explicitly set term positions and slop=0, in order to skip stop words. The field in my index is indexed with TermVector positions. When I do a query with stop words skipped, for example "internet for research" (translated into PhraseQuery: "internet ? research"), I