Re: Index keeps growing, then shrinks on restart

2014-11-11 Thread Rob Nikander
On Tue, Nov 11, 2014 at 4:26 AM, Ian Lea ian@gmail.com wrote: Telling us the version of lucene and the OS you're running on is always a good idea. Oops, yes. Lucene 4.10.0, Linux. A guess here is that you aren't closing index readers, so the JVM will be holding on to deleted files

Index keeps growing, then shrinks on restart

2014-11-10 Thread Rob Nikander
Hi, I have an index that's about 700 MB, and it grows over days to until it causes problems with disk size, at about 5GB. If the JVM process ends, the index shrinks back to about 700MB, I'm calling IndexWriter.commit() all the time. What else do you call to get it to compact it's use of space?

phrase query, stop words, and highlighting?

2014-09-22 Thread Rob Nikander
Hi, I just noticed that a search like rooms to go is failing to highlight. (I use FastVectorHighlighter). I know it's caused the stop word (to). Is there a recommended way to fix this? I may just re-index without stop words, and see if that causes any problems. thanks, Rob

Re: indexing all suffixes to support leading wildcard?

2014-08-29 Thread Rob Nikander
, Erick On Thu, Aug 28, 2014 at 10:38 AM, Rob Nikander rob.nikan...@gmail.com wrote: Hi, I've got some short fields (phone num, email) that I'd like to search using good old string matching. (The full query is a boolean or that also uses real text fields.) I see the warnings about wildcard

Re: indexing all suffixes to support leading wildcard?

2014-08-29 Thread Rob Nikander
Doh. Nevermind, I see it. I was searching with same analyzer that I used to index. Usually that's right, but in this case, no. Rob On Fri, Aug 29, 2014 at 10:59 AM, Rob Nikander rob.nikan...@gmail.com wrote: Thanks. That got the search working. Do you know if there's a trick for using

How to not span fields with phrase query?

2014-08-28 Thread Rob Nikander
Hi, If I have document with multiple fields title title: A B C title: X Y Z A phrase search for title:B C X matches this document. Can I prevent that? thanks, Rob

Re: How to not span fields with phrase query?

2014-08-28 Thread Rob Nikander
instead of 3, 4, 5, which is probably what you have now -Mike On 08/28/2014 09:53 AM, Rob Nikander wrote: Hi, If I have document with multiple fields title title: A B C title: X Y Z A phrase search for title:B C X matches this document. Can I prevent that? thanks, Rob

indexing all suffixes to support leading wildcard?

2014-08-28 Thread Rob Nikander
Hi, I've got some short fields (phone num, email) that I'd like to search using good old string matching. (The full query is a boolean or that also uses real text fields.) I see the warnings about wildcard queries that start with *, and I'm wondering... do you think it would be a good idea to

Can I update one field in doc?

2014-08-28 Thread Rob Nikander
I tried something like this, to loop through all docs in my index and patch a field. But it appears to wipe out some parts of the stored values in the document. For example, highlighting stopped working. [ scala code ] val q = new MatchAllDocsQuery() val topDocs = searcher.search(q, 100) val

Re: Can I update one field in doc?

2014-08-28 Thread Rob Nikander
I used the Luke tool to look at my documents. It shows that the positions and offsets in the term vectors get wiped out, in all fields. I'm using Lucene 4.8. I guess I'll just rebuild the entire doc. Rob On Thu, Aug 28, 2014 at 5:33 PM, Rob Nikander rob.nikan...@gmail.com wrote: I tried

Consistent colors from highlighter, multiple fields?

2014-08-25 Thread Rob Nikander
Hi, I'm using FastVectorHighlighter, and I wanted to get highlights from multiple fields that matched, so I called `highlighter.getBestFragment` for each field. It returns null if it had nothing for that field. The problem is the colors don't match, so it looks confusing. For example, I search

Re: stemming irregular plurals?

2014-07-29 Thread Rob Nikander
Mmm. I don’t see a way to construct one, except passing an FST, which isn’t exactly a map. I look at the FST javadoc; it’s a rabbit hole. Rob On Jul 29, 2014, at 10:14 AM, Robert Muir rcm...@gmail.com wrote: You can put this thing before your stemmer, with a custom map of exceptions:

Re: stemming irregular plurals?

2014-07-29 Thread Rob Nikander
Ah, yes, that does it. Thank you both. Rob On Jul 29, 2014, at 10:30 AM, Alexandre Patry alexandre.pa...@keatext.com wrote: On 29/07/2014 10:28, Rob Nikander wrote: Mmm. I don’t see a way to construct one, except passing an FST, which isn’t exactly a map. I look at the FST javadoc