RE: Getting results for a specific date

2009-06-17 Thread Scott Smith
Clarification: Obviously, I should have said "June 11" when I talked of a newer date. From: Scott Smith [mailto:ssm...@mainstreamdata.com] Sent: Tue 6/16/2009 5:41 PM To: java-user@lucene.apache.org Subject: Getting results for a specific date Mostly, our user

RE: Queries and Filters

2009-06-17 Thread Scott Smith
I thought of that. However, I verified that there are queries. The query code also does what you suggest with MatchAllDocsQuery() if there were no queries, but the test case doesn't need this since there are queries. I'd really like to understand why the code doesn't work before I try the Co

Re: Determining lucene version programmatically

2009-06-17 Thread Michael McCandless
As of 2.9, org.apache.lucene.util.Constants.LUCENE_VERSION is a good way to get this. That constant is set by first looking at the implementation version, and then making sure the version Lucene thinks it is (2.9 on trunk now) is contained in the resulting string. It also takes care of a null imp

Lucene performance: is search time linear to the index size?

2009-06-17 Thread Teruhiko Kurosaka
I am seeing my Lucene application's search time grows pretty much linearly to the number of Documents. Is this how Lucene is supposed to work, or does it depend on the nature of query? I am not using FuzzyQuery that was the subject of the recent discussion by the way. -Kuro -

Re: Lucene performance: is search time linear to the index size?

2009-06-17 Thread Ian Lea
It depends on lots of things, but the time to execute a search would not typically grow linearly with the number of documents. But the time to retrieve data from all the hits might, if the number of hits is growing in line with the number of documents. Are you doing that by any chance, as opposed

Re: Lucene performance: is search time linear to the index size?

2009-06-17 Thread Erick Erickson
Are you measuring search time *only* or are you measuring total response time including assembling whatever you assemble? If you're measuring total response time, everything from network latency to what you're doing with each hit may affect response time. This is especially true if you're iteratin

RE: Lucene performance: is search time linear to the index size?

2009-06-17 Thread Teruhiko Kurosaka
Thank you, Ian and Erick, This is a stand-alone search application that runs a query and exits, as opposed to the typical web-based search program. The query is consists of combination of SpanQuery, TermQuery, BooleanClause, SpanNearQuery, and SpanTermQuery. I get the top 50 hits at most and no

how to deal with too many clause error in boolean query.

2009-06-17 Thread vanshi
Hello all, I have a situation where a field is indexed like this (FAC_NAME(Field.Store.NO, Field.Index.NO_NORMS)) and keyword analyzer is used on this field. Although, I'm aware that NO_NORMS doesn't use any analyzer. Now, the query +(FAC_NAME:fa*) is failing with 'Too many clause exception' in

Re: how to deal with too many clause error in boolean query.

2009-06-17 Thread Tim Williams
On Wed, Jun 17, 2009 at 3:16 PM, vanshi wrote: > > Hello all, > > I have a situation where a field is indexed like this > (FAC_NAME(Field.Store.NO, Field.Index.NO_NORMS)) and keyword analyzer is > used on this field. Although, I'm aware that NO_NORMS doesn't use any > analyzer. > > Now, the query +

Re: Lucene performance: is search time linear to the index size?

2009-06-17 Thread Peter Keegan
There is a similar discussion on this topic here: http://www.gossamer-threads.com/lists/lucene/java-user/42824?search_string=Lucene%20search%20performance%3A%20linear%3F;#42824 or: *http://tinyurl.com/lpp3hf* On Wed, Jun 17, 2009 at 1:18 PM, Teruhiko Kurosaka wrote: > Thank you, Ian and Erick,

RE: Lucene performance: is search time linear to the index size?

2009-06-17 Thread Teruhiko Kurosaka
I've written a test program that uses the simplest form of search, TermQuery and measure the time it takes to search a term in a field on indices of various sizes. The result is a very linear growth of search time vs the index size in terms of # of Documents, not # of unique terms in that field.

Re: Best way to do date sort

2009-06-17 Thread Ganesh
I am having a million of documents per day to index. DateTime field is with minute resolution. How much memory i save by splitting this to multiple fields (one field containintg MMDD, one field with HH and one with MM). Could anyone provide me some calculation of calculating memory for so