Re: Query about Query.ToString()

2010-02-18 Thread Mark Harwood
> I have 3 questions: > > 1) XML query parser just focuses on the "difficult" queries, left the > "simple" ones to the standard query parser. > I think it'll be good to support all of the queries. Makes it easier to > implement, rather than combine two different parser together. "Lucene in Act

Re: Query about Query.ToString()

2010-02-18 Thread Chris Lu
It's great that this is supported. I have 3 questions: 1) XML query parser just focuses on the "difficult" queries, left the "simple" ones to the standard query parser. I think it'll be good to support all of the queries. Makes it easier to implement, rather than combine two different parser to

RE: BM25 Scoring Patch

2010-02-18 Thread Yuval Feinstein
-Original Message- From: Robert Muir [mailto:rcm...@gmail.com] Sent: Thursday, February 18, 2010 3:09 PM To: java-user@lucene.apache.org Subject: Re: BM25 Scoring Patch Yuval, don't we still need this 'document-level IDF' for BM25f? - Yes, we do need 'document-level IDF' for BM25f. - J

Re: questions on upgrading to 3.0: Version.LUCENE_* and Field.setOmitNorms()

2010-02-18 Thread Ian Lea
For the classes you show I don't think that it matters. It certainly can matter if you are using StandardAnalyzer or some other lucene supplied classes - see the javadocs. In my opinion you should review the lucene analyzers that you use and see if you rely on any of the behaviour that is version

Re: questions on upgrading to 3.0: Version.LUCENE_* and Field.setOmitNorms()

2010-02-18 Thread jm
Thanks for the replies Ian and Robert. In my case, I am in a bit of a uneasy position, cannot reindex, original docs are gone... What would you recommend? I have to choose one value, and some customers started using our system with lucene 2.3, others with lucene 2.4 and others with 2.9. My usage o

Re: questions on upgrading to 3.0: Version.LUCENE_* and Field.setOmitNorms()

2010-02-18 Thread Robert Muir
yes, if you use LUCENE_CURRENT, you may have to reindex (if any analyzers/tokenizers you are using have changed). if you use an actual version (for example LUCENE_30), you can upgrade your jar file to say a future 3.1 jar without reindexing, then later at your leisure (after testing/qa whatever yo

Re: questions on upgrading to 3.0: Version.LUCENE_* and Field.setOmitNorms()

2010-02-18 Thread Ian Lea
But typically you wouldn't need to reindex, would you? From the 3.0 javadocs for LUCENE_CURRENT: WARNING: if you use this setting, and then upgrade to a newer release of Lucene, sizable changes may happen. If precise back compatibility is important then you should instead explicitly specify an ac

Re: questions on upgrading to 3.0: Version.LUCENE_* and Field.setOmitNorms()

2010-02-18 Thread Robert Muir
Only use LUCENE_CURRENT if you do not care about backwards compatibility at all: e.g. you are perfectly happy re-indexing all data when you upgrade the lucene jar file in future. its not about relying on quirks in previous versions of lucene, its about being compatible with changes in future versi

Re: BM25 Scoring Patch

2010-02-18 Thread Robert Muir
Yuval, don't we still need this 'document-level IDF' for BM25f? On Thu, Feb 18, 2010 at 3:45 AM, Yuval Feinstein wrote: > We could solve this by saying we only incorporate BM25F into Lucene. > This is a field-based scoring method, so it saves us the need to deal with > documents. > Building on J

Re: questions on upgrading to 3.0: Version.LUCENE_* and Field.setOmitNorms()

2010-02-18 Thread Ian Lea
> previously I was using 2.9 (upgraded from 2.4 but did not fix warnings > etc). Now I have upgraded to 3.0, so I had to fix all deprecated > methods etc. My question is with Version type parameter in some > Token* classes. > > Some of our customers have our product with lucene 2.4 (some upgraded >

Re: questions on upgrading to 3.0: Version.LUCENE_* and Field.setOmitNorms()

2010-02-18 Thread jm
someone? On Tue, Feb 16, 2010 at 11:47 AM, jm wrote: > Hi, > > previously I was using 2.9 (upgraded from 2.4 but did not fix warnings > etc). Now I have upgraded to 3.0, so I had to fix all deprecated > methods etc. My question is with Version type parameter in some > Token* classes. > > Some of

RE: BM25 Scoring Patch

2010-02-18 Thread Yuval Feinstein
We could solve this by saying we only incorporate BM25F into Lucene. This is a field-based scoring method, so it saves us the need to deal with documents. Building on Joaquin's work, the extra parts needed IMO are: a. Support for storing average length per field during indexing. I think I saw som