[jira] Commented: (LUCENE-915) PorterStemmer is incorrectly truncating words ending in e

2007-06-06 Thread Paul Curren (JIRA)

[ 
https://issues.apache.org/jira/browse/LUCENE-915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12502233
 ] 

Paul Curren commented on LUCENE-915:


I understand, and thanks for you help.

Incidentally, i'm comparing against a commercial search engine which doesn't 
exhibit this stemming behaviour.
They must be using a different or modified algorithm - I don't know which i'm 
afraid.

This is the reason why I raise this as a bug - at the application level (forget 
about low level components and stemmers for now) there is an unexpected 
behaviour and it relates to an implementation detail within Lucene (the choice 
of algorithm).

That's all, thanks again.



> PorterStemmer is incorrectly truncating words ending in e
> -
>
> Key: LUCENE-915
> URL: https://issues.apache.org/jira/browse/LUCENE-915
> Project: Lucene - Java
>  Issue Type: Bug
>  Components: Index, QueryParser, Search
>Affects Versions: 1.9
> Environment: Java 1.5 on Mac OS X 10.4.
>Reporter: Paul Curren
>
> Searching for the word 'orange' will result incorrectly in matches for 
> 'orang'.
> Likewise, searching for 'apple' will incorrectly match 'appl'
> The problem is in step6() of the PorterStemmer class.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]



Re: Please help testing the release files

2007-06-06 Thread Michael Busch

Michael Busch wrote on 04/06/2007 18:59:49:



> So please help testing the release files on
> different platforms with different JVM versions.
  


For testing purposes I uploaded a current build from the 2.2 branch to

 http://people.apache.org/~buschmi/staging_area/lucene/


- Michael

-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]



PLEASE READ: 2.2 branch created, feature freeze in effect

2007-06-06 Thread Michael Busch

Hello Team,

well, first of all, let's take a deep breath! Behind us are a couple of
busy weeks. I would like to take this chance to thank everyone very much
for the great work! We're on track for our 2.2 release on the 19th of June.


As announced I created a Lucene 2.2 branch today from trunk revision
r545056 here:
 https://svn.apache.org/repos/asf/lucene/java/branches/lucene_2_2/

As of now a feature freeze of this branch is in effect. In detail this
means to the committers:

- No new features may be committed to the branch.
- Documentation patches, build patches and serious bug fixes may be
 committed to the branch. However, you should submit *all* patches you
 want to commit to Jira first to give others the chance to review and
 possibly vote against the patch. Keep in mind that it is our main
 intention to keep the branch as stable as possible.
- All patches that are intended for the branch should first be committed to
 trunk and then merged into the branch.
- Normal trunk development may continue as usual. However, if you plan to
 commit a big change to the trunk while the branch feature freeze is in
 effect, think twice: can't the addition wait a couple more days? Merges
 of bug fixes into the branch may become more difficult.
- *Only* Jira issues with Fix version "2.2" and priority "Blocker" will
 delay a release candidate build. If on June 17th none of those issue are
 in Jira I will build a release candidate and call a release vote on
 java-dev.

We should make use of this feature freeze and spend the next 10 days for
extensive testing and javadoc improvements. Also help with the maven patch
(LUCENE-622) is welcome.

Let's keep up the great work! I'm very confident that version 2.2 will be a
good one!

- Michael

-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]



[jira] Resolved: (LUCENE-915) PorterStemmer is incorrectly truncating words ending in e

2007-06-06 Thread Hoss Man (JIRA)

 [ 
https://issues.apache.org/jira/browse/LUCENE-915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Hoss Man resolved LUCENE-915.
-

Resolution: Invalid

> I'd imagine you aren't going to fix it since it would require explicit 
> 'exception word' 
> checking being added to the algorithm. 

...well, my point actually is that there is no bug to fix -- the algorithm is 
what it is, and the code implements the algorithm.

changing the code wouldn't be fixing a bug, it would be breaking the 
PorterStemmer class so that it no longer does what it says "implementing the 
Porter Stemming Algorithm"

i'm sure there are *lots* of other use cases unrelated the the ones you 
outlined where people could argue that the Porter algorithm does something they 
don't want -- but that's just the nature of algorithm stemmers.  as outlined 
onthe Porter Stemmer homepage...

"The most frequently asked question is why word X should be stemmed to x1, when 
one would have expected it to be stemmed to x2. It is important to remember 
that the stemming algorithm cannot achieve perfection. On balance it will (or 
may) improve IR performance, but in individual cases it may sometimes make what 
are, or what seem to be, errors."

> PorterStemmer is incorrectly truncating words ending in e
> -
>
> Key: LUCENE-915
> URL: https://issues.apache.org/jira/browse/LUCENE-915
> Project: Lucene - Java
>  Issue Type: Bug
>  Components: Index, QueryParser, Search
>Affects Versions: 1.9
> Environment: Java 1.5 on Mac OS X 10.4.
>Reporter: Paul Curren
>
> Searching for the word 'orange' will result incorrectly in matches for 
> 'orang'.
> Likewise, searching for 'apple' will incorrectly match 'appl'
> The problem is in step6() of the PorterStemmer class.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]



[jira] Commented: (LUCENE-915) PorterStemmer is incorrectly truncating words ending in e

2007-06-06 Thread Paul Curren (JIRA)

[ 
https://issues.apache.org/jira/browse/LUCENE-915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12502223
 ] 

Paul Curren commented on LUCENE-915:


I believe like you say that the algorithm is correctly implemented, however the 
algorithm has the above bug.

The end result is false positives in the search results.
It's not a big issue, however the search engine has a behaviour that leads to 
false hits being returned and therefore this is definitely a defect in 
application functionality.

I'd imagine you aren't going to fix it since it would require explicit 
'exception word' checking being added to the algorithm. So feel free to close 
the issue. The main reason I raise it is so others will hopefully spend less 
time that I did being confused by this.

> PorterStemmer is incorrectly truncating words ending in e
> -
>
> Key: LUCENE-915
> URL: https://issues.apache.org/jira/browse/LUCENE-915
> Project: Lucene - Java
>  Issue Type: Bug
>  Components: Index, QueryParser, Search
>Affects Versions: 1.9
> Environment: Java 1.5 on Mac OS X 10.4.
>Reporter: Paul Curren
>
> Searching for the word 'orange' will result incorrectly in matches for 
> 'orang'.
> Likewise, searching for 'apple' will incorrectly match 'appl'
> The problem is in step6() of the PorterStemmer class.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]



[jira] Commented: (LUCENE-915) PorterStemmer is incorrectly truncating words ending in e

2007-06-06 Thread Hoss Man (JIRA)

[ 
https://issues.apache.org/jira/browse/LUCENE-915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12502221
 ] 

Hoss Man commented on LUCENE-915:
-

can you elaborate on why you think this is a bug?  

This is a fairly basic function of the Porter Stemming Algorithm, and exists in 
the official java version of the algorith published by Martin Porter...

http://www.tartarus.org/~martin/PorterStemmer/java.txt

(you may not arge with Porter's decision to strip trailing Es, but it's in the 
algorithm, and the class implements the algorithm)

> PorterStemmer is incorrectly truncating words ending in e
> -
>
> Key: LUCENE-915
> URL: https://issues.apache.org/jira/browse/LUCENE-915
> Project: Lucene - Java
>  Issue Type: Bug
>  Components: Index, QueryParser, Search
>Affects Versions: 1.9
> Environment: Java 1.5 on Mac OS X 10.4.
>Reporter: Paul Curren
>
> Searching for the word 'orange' will result incorrectly in matches for 
> 'orang'.
> Likewise, searching for 'apple' will incorrectly match 'appl'
> The problem is in step6() of the PorterStemmer class.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]



[jira] Created: (LUCENE-915) PorterStemmer is incorrectly truncating words ending in e

2007-06-06 Thread Paul Curren (JIRA)
PorterStemmer is incorrectly truncating words ending in e
-

 Key: LUCENE-915
 URL: https://issues.apache.org/jira/browse/LUCENE-915
 Project: Lucene - Java
  Issue Type: Bug
  Components: Index, QueryParser, Search
Affects Versions: 1.9
 Environment: Java 1.5 on Mac OS X 10.4.
Reporter: Paul Curren


Searching for the word 'orange' will result incorrectly in matches for 'orang'.

Likewise, searching for 'apple' will incorrectly match 'appl'

The problem is in step6() of the PorterStemmer class.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]



[jira] Commented: (LUCENE-914) Scorer.skipTo(current) remains on current for some scorers

2007-06-06 Thread Doug Cutting (JIRA)

[ 
https://issues.apache.org/jira/browse/LUCENE-914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12502219
 ] 

Doug Cutting commented on LUCENE-914:
-

The text was mostly trying to describe what the specified implementation did, 
to provide as clear and unambiguous of a contract as possible.  Note that the 
specified implementation must call next() before doc(), since doc() isn't 
always defined unless next() has been called.

> Scorer.skipTo(current) remains on current for some scorers
> --
>
> Key: LUCENE-914
> URL: https://issues.apache.org/jira/browse/LUCENE-914
> Project: Lucene - Java
>  Issue Type: Bug
>  Components: Search
>Reporter: Doron Cohen
>Priority: Minor
> Attachments: lucene-914.patch
>
>
> Background in http://www.nabble.com/scorer.skipTo%28%29-contr-tf3880986.html
> It appears that several scorers do not strictly follow the spec of 
> Scorer.skipTo(n), and skip to current location remain in current location 
> whereas the spec says: "beyond current". 
> We should (probably) either relax the spec or fix the implementations.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]



Re: Documentation Brainstorming

2007-06-06 Thread Chris Hostetter

In hte spirit of improving documentation, i've started a document
summarizing the various comments/ideas about improving out
documentation...

http://wiki.apache.org/lucene-java/Documentation_Improvements

: 3.  There is a whole lot of knowledge stored in the email archives,
: how can we leverage it?

One email thread down, a bigillion to go!


-Hoss


-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]



[jira] Commented: (LUCENE-902) Check on PositionIncrement with StopFilter.

2007-06-06 Thread Hoss Man (JIRA)

[ 
https://issues.apache.org/jira/browse/LUCENE-902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12502213
 ] 

Hoss Man commented on LUCENE-902:
-

A few comments in no particular order...

1) in future patches, could you please use 2 spaces instead of tabs?

2) am i understanding correctly that the primary use case you are trying to 
address is stop word removal when the stop word has synonyms with a position 
increment of 0 (the expectation being that the synonyms also be removed) ? ... 
if so, wouldn't the most efficient thing be to do stop word removal before 
doing synonym expansion?  (it means having a bigger stop word list - with all 
the synonyms - but that still seems better to me) ... are there other use cases 
i'm not understanding? ... i freely admit i don't understand the "Japanese 
morphological analysis" comment.

3) i only glanced over the specifics of removeStopwordCollocatesNext() .. but 
would promoting BufferedTokenStream from Solr simplify the code (it seems to 
all be about buffering tokens) ...
http://svn.apache.org/viewvc/lucene/solr/trunk/src/java/org/apache/solr/analysis/BufferedTokenStream.java?view=markup

4) it would be useful if the test case could clarify not only the expected 
tokens text concatenated together, but also what the expected positions of 
position increments are for the tokens... i was certainly confused by the title 
of this issue.


> Check on PositionIncrement  with StopFilter.
> 
>
> Key: LUCENE-902
> URL: https://issues.apache.org/jira/browse/LUCENE-902
> Project: Lucene - Java
>  Issue Type: Bug
>  Components: Analysis
>Affects Versions: 2.2
>Reporter: Toru Matsuzawa
> Attachments: stopfilter.patch, stopfilter20070604.patch, 
> stopfilter20070605.patch
>
>
> PositionIncrement set with Tokenizer is not considered with StopFilter. 
> When PositionIncrement of Token is 1, it is deleted by StopFilter. However, 
> when PositionIncrement of Token following afterwards is 0, it is not deleted. 
> I think that it is necessary to be deleted. Because it is thought same Token 
> when PositionIncrement is 0.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]



[jira] Commented: (LUCENE-914) Scorer.skipTo(current) remains on current for some scorers

2007-06-06 Thread Yonik Seeley (JIRA)

[ 
https://issues.apache.org/jira/browse/LUCENE-914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12502186
 ] 

Yonik Seeley commented on LUCENE-914:
-

ConstantScorer is implemented like:

public boolean skipTo(int target) throws IOException {
  doc = bits.nextSetBit(target);  // requires JDK 1.4
  return doc >= 0;
}

The fix would be:

public boolean skipTo(int target) throws IOException {
  doc = bits.nextSetBit( Math.max(target, doc+1) );
  return doc >= 0;
}

But I'm really not sure I prefer that over just clarifying/restricting skipTo.


> Scorer.skipTo(current) remains on current for some scorers
> --
>
> Key: LUCENE-914
> URL: https://issues.apache.org/jira/browse/LUCENE-914
> Project: Lucene - Java
>  Issue Type: Bug
>  Components: Search
>Reporter: Doron Cohen
>Priority: Minor
> Attachments: lucene-914.patch
>
>
> Background in http://www.nabble.com/scorer.skipTo%28%29-contr-tf3880986.html
> It appears that several scorers do not strictly follow the spec of 
> Scorer.skipTo(n), and skip to current location remain in current location 
> whereas the spec says: "beyond current". 
> We should (probably) either relax the spec or fix the implementations.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]



Re: Lucene 2.2 - Suggested schedule

2007-06-06 Thread Michael Busch

Doron Cohen wrote:



Since the patches are in place we might want to commit LUCENE-912 and
LUCENE-913 before?



If there are no objections I plan to commit it later today.

  


912 and 913 are committed. Great job, Doron! Thank you!

Alright, everything seems to be in place. Good timing! I will make the 
2.2 branch soon.



- Michael

-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]



[jira] Updated: (LUCENE-914) Scorer.skipTo(current) remains on current for some scorers

2007-06-06 Thread Doron Cohen (JIRA)

 [ 
https://issues.apache.org/jira/browse/LUCENE-914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Doron Cohen updated LUCENE-914:
---

Attachment: lucene-914.patch

patch adds a test to QueryUtils that exposes this issue.
50 test cases in 9 test files fail with this.
(I know that I did not worry to fulfill this spec in search.function, for 
instance.)

> Scorer.skipTo(current) remains on current for some scorers
> --
>
> Key: LUCENE-914
> URL: https://issues.apache.org/jira/browse/LUCENE-914
> Project: Lucene - Java
>  Issue Type: Bug
>  Components: Search
>Reporter: Doron Cohen
>Priority: Minor
> Attachments: lucene-914.patch
>
>
> Background in http://www.nabble.com/scorer.skipTo%28%29-contr-tf3880986.html
> It appears that several scorers do not strictly follow the spec of 
> Scorer.skipTo(n), and skip to current location remain in current location 
> whereas the spec says: "beyond current". 
> We should (probably) either relax the spec or fix the implementations.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]



[jira] Resolved: (LUCENE-913) Two consecutive score() calls return different scores for Boolean Queries.

2007-06-06 Thread Doron Cohen (JIRA)

 [ 
https://issues.apache.org/jira/browse/LUCENE-913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Doron Cohen resolved LUCENE-913.


   Resolution: Fixed
Lucene Fields: [Patch Available]  (was: [New])

Committed, thanks Michael!

> Two consecutive score() calls return different scores for Boolean Queries.
> --
>
> Key: LUCENE-913
> URL: https://issues.apache.org/jira/browse/LUCENE-913
> Project: Lucene - Java
>  Issue Type: Bug
>  Components: Search
>Reporter: Doron Cohen
>Assignee: Doron Cohen
>Priority: Minor
> Fix For: 2.2
>
> Attachments: lucene-913.patch
>
>
> Two consecutive calls to score() return different scores (no next() or 
> skipTo() calls in between). 
> Background in LUCENE-912 .

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]



[jira] Updated: (LUCENE-913) Two consecutive score() calls return different scores for Boolean Queries.

2007-06-06 Thread Doron Cohen (JIRA)

 [ 
https://issues.apache.org/jira/browse/LUCENE-913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Doron Cohen updated LUCENE-913:
---

Attachment: lucene-913.patch

patch attached for future reference, contains:
- Michael's fix 
- new test in QueryUtils that exposes the bug
- some debugging improvements in QueryUtils.

> Two consecutive score() calls return different scores for Boolean Queries.
> --
>
> Key: LUCENE-913
> URL: https://issues.apache.org/jira/browse/LUCENE-913
> Project: Lucene - Java
>  Issue Type: Bug
>  Components: Search
>Reporter: Doron Cohen
>Assignee: Doron Cohen
>Priority: Minor
> Fix For: 2.2
>
> Attachments: lucene-913.patch
>
>
> Two consecutive calls to score() return different scores (no next() or 
> skipTo() calls in between). 
> Background in LUCENE-912 .

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]



[jira] Resolved: (LUCENE-912) DisjunctionMaxScorer.skipTo has bug that keeps it from skipping

2007-06-06 Thread Doron Cohen (JIRA)

 [ 
https://issues.apache.org/jira/browse/LUCENE-912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Doron Cohen resolved LUCENE-912.


Resolution: Fixed

Committed the original lucene-912.patch by Hoss. 
Thanks Hoss and Sudaakeran B. !

> DisjunctionMaxScorer.skipTo has bug that keeps it from skipping
> ---
>
> Key: LUCENE-912
> URL: https://issues.apache.org/jira/browse/LUCENE-912
> Project: Lucene - Java
>  Issue Type: Bug
>Affects Versions: 2.0.0, 2.1
>Reporter: Hoss Man
>Assignee: Doron Cohen
> Fix For: 2.2
>
> Attachments: checkTwoCallsToScore.patch, checkTwoCallsToScore.patch, 
> dismax_skipto.patch, lucene-912.patch
>
>
> as reported on the mailing list, DisjunctionMaxScorer.skipTo is broken if 
> called before next in some situations...
> http://www.nabble.com/Potential-issue-with-DisjunctionMaxScorer-tf3846366.html#a10894987

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]



[jira] Updated: (LUCENE-913) Two consecutive score() calls return different scores for Boolean Queries.

2007-06-06 Thread Sean Timm (JIRA)

 [ 
https://issues.apache.org/jira/browse/LUCENE-913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sean Timm updated LUCENE-913:
-

Summary: Two consecutive score() calls return different scores for Boolean 
Queries.  (was: Two consecutive score() calls return different scores for 
Bollean Queries.)

> Two consecutive score() calls return different scores for Boolean Queries.
> --
>
> Key: LUCENE-913
> URL: https://issues.apache.org/jira/browse/LUCENE-913
> Project: Lucene - Java
>  Issue Type: Bug
>  Components: Search
>Reporter: Doron Cohen
>Assignee: Doron Cohen
>Priority: Minor
> Fix For: 2.2
>
>
> Two consecutive calls to score() return different scores (no next() or 
> skipTo() calls in between). 
> Background in LUCENE-912 .

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]



[jira] Commented: (LUCENE-914) Scorer.skipTo(current) remains on current for some scorers

2007-06-06 Thread Yonik Seeley (JIRA)

[ 
https://issues.apache.org/jira/browse/LUCENE-914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12502162
 ] 

Yonik Seeley commented on LUCENE-914:
-

The spec text is ambiguous.
The biggest issue is to look if any scorers actually call skipTo(n) with 
n<=doc().
We could further restrict the spec to say that n *must* be > doc() for defined 
results.
That change would not hurt customers, since the current behavior is already 
somewhat undefined.

> Scorer.skipTo(current) remains on current for some scorers
> --
>
> Key: LUCENE-914
> URL: https://issues.apache.org/jira/browse/LUCENE-914
> Project: Lucene - Java
>  Issue Type: Bug
>  Components: Search
>Reporter: Doron Cohen
>Priority: Minor
>
> Background in http://www.nabble.com/scorer.skipTo%28%29-contr-tf3880986.html
> It appears that several scorers do not strictly follow the spec of 
> Scorer.skipTo(n), and skip to current location remain in current location 
> whereas the spec says: "beyond current". 
> We should (probably) either relax the spec or fix the implementations.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]



[jira] Updated: (LUCENE-908) MANIFEST.MF cleanup (main jar and luci customizations)

2007-06-06 Thread Michael Busch (JIRA)

 [ 
https://issues.apache.org/jira/browse/LUCENE-908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Busch updated LUCENE-908:
-

Fix Version/s: (was: 2.2)

Patch committed. Leaving this issue open for the simplifications suggested by 
Hoss.

> MANIFEST.MF cleanup (main jar and luci customizations)
> --
>
> Key: LUCENE-908
> URL: https://issues.apache.org/jira/browse/LUCENE-908
> Project: Lucene - Java
>  Issue Type: Bug
>  Components: Build
>Reporter: Michael Busch
>Assignee: Michael Busch
>Priority: Trivial
> Attachments: lucene-908.patch, LUCENE-908.patch
>
>
> there are several problems with the MANIFEST.MF file used in the core jar, 
> and some inconsistencies in th luci jar:
> Lucli's build.xml has an own "jar" target and does not use the jar target 
> from common-build.xml. The result
> is that the MANIFEST.MF file is not consistent and the META-INF dir does not 
> contain LICENSE.TXT and NOTICE.TXT.
> Is there a reason why lucli behaves different in this regard? If not I think 
> we should fix this.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]



Re: scorer.skipTo() contr

2007-06-06 Thread Yonik Seeley

On 6/6/07, Michael Busch <[EMAIL PROTECTED]> wrote:

I was just wondering if this behavior is what the average user would
expect from a skipTo() method
without reading the javadocs carefully enough.


skipTo() use isn't for the average user ;-)

This is very inner-loop stuff and should be primarily based on what
can perform the best.
So it may be OK for skipTo(doc()) to be undefined.  Do any scorers
currently call skipTo(n) when n <= doc()?

-Yonik

-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]



[jira] Commented: (LUCENE-908) MANIFEST.MF cleanup (main jar and luci customizations)

2007-06-06 Thread Michael Busch (JIRA)

[ 
https://issues.apache.org/jira/browse/LUCENE-908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12502156
 ] 

Michael Busch commented on LUCENE-908:
--

As always these are very good recommendations Hoss! I think I will commit my 
patch for 2.2, because it works fine. But I will leave this issue open (just 
clear the Fix version) to keep in mind that we want to make these improvements.


> MANIFEST.MF cleanup (main jar and luci customizations)
> --
>
> Key: LUCENE-908
> URL: https://issues.apache.org/jira/browse/LUCENE-908
> Project: Lucene - Java
>  Issue Type: Bug
>  Components: Build
>Reporter: Michael Busch
>Assignee: Michael Busch
>Priority: Trivial
> Fix For: 2.2
>
> Attachments: lucene-908.patch, LUCENE-908.patch
>
>
> there are several problems with the MANIFEST.MF file used in the core jar, 
> and some inconsistencies in th luci jar:
> Lucli's build.xml has an own "jar" target and does not use the jar target 
> from common-build.xml. The result
> is that the MANIFEST.MF file is not consistent and the META-INF dir does not 
> contain LICENSE.TXT and NOTICE.TXT.
> Is there a reason why lucli behaves different in this regard? If not I think 
> we should fix this.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]



Re: scorer.skipTo() contr

2007-06-06 Thread Michael Busch

Chris Hostetter wrote:

: But I think it should be the desired behavior for skipTo() to not skip
: at all
: if curDoc==target already? Which means we should clearify the javadocs.

i'm not certain about that ... in theory (given the way the
javadocs are currently written) shouldn't s.skipTo(0) always be
functionally equivilent to s.next() ?

  

True, that is what the javadocs currently describe.

I was just wondering if this behavior is what the average user would 
expect from a skipTo() method
without reading the javadocs carefully enough. (I guess when I started 
using Lucene I didn't expect

skipTo() to move to the next doc if target<=curDoc)

But on the other hand this would be a public API change and therefore we 
should probably not change

the spec. (as you just commented on 914)



-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]



[jira] Commented: (LUCENE-914) Scorer.skipTo(current) remains on current for some scorers

2007-06-06 Thread Hoss Man (JIRA)

[ 
https://issues.apache.org/jira/browse/LUCENE-914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12502153
 ] 

Hoss Man commented on LUCENE-914:
-

my gut says we should fix the Impls and leave the spec the way it is ... less 
risk of irate clients 2 years down the road wondering why their custom Query 
classes no longer work in boolean queries because we: a) relaxed the spec; b) 
made optimizations in BooleanQuery to take advantage of the relaxation.

> Scorer.skipTo(current) remains on current for some scorers
> --
>
> Key: LUCENE-914
> URL: https://issues.apache.org/jira/browse/LUCENE-914
> Project: Lucene - Java
>  Issue Type: Bug
>  Components: Search
>Reporter: Doron Cohen
>Priority: Minor
>
> Background in http://www.nabble.com/scorer.skipTo%28%29-contr-tf3880986.html
> It appears that several scorers do not strictly follow the spec of 
> Scorer.skipTo(n), and skip to current location remain in current location 
> whereas the spec says: "beyond current". 
> We should (probably) either relax the spec or fix the implementations.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]



[jira] Created: (LUCENE-914) Scorer.skipTo(current) remains on current for some scorers

2007-06-06 Thread Doron Cohen (JIRA)
Scorer.skipTo(current) remains on current for some scorers
--

 Key: LUCENE-914
 URL: https://issues.apache.org/jira/browse/LUCENE-914
 Project: Lucene - Java
  Issue Type: Bug
  Components: Search
Reporter: Doron Cohen
Priority: Minor


Background in http://www.nabble.com/scorer.skipTo%28%29-contr-tf3880986.html

It appears that several scorers do not strictly follow the spec of 
Scorer.skipTo(n), and skip to current location remain in current location 
whereas the spec says: "beyond current". 

We should (probably) either relax the spec or fix the implementations.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]



Re: scorer.skipTo() contr

2007-06-06 Thread Chris Hostetter

: But I think it should be the desired behavior for skipTo() to not skip
: at all
: if curDoc==target already? Which means we should clearify the javadocs.

i'm not certain about that ... in theory (given the way the
javadocs are currently written) shouldn't s.skipTo(0) always be
functionally equivilent to s.next() ?

I'm pbasingthis on the assumption that there must be some reason why the
"behaves as" clause was written as...

   *   boolean skipTo(int target) {
   * do {
   *   if (!next())
   * return false;
   * } while (target > doc());
   * return true;
   *   }

and not...

  boolean skipTo(int target) {
while (target > doc()) {
  if (!next()) return false;
}
return true;
  }

...which is what you are describing.



-Hoss


-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]



Re: scorer.skipTo() contr

2007-06-06 Thread Chris Hostetter

: The method states 'greater than OR EQUAL TO' so your d1 != d2 test is
: invalid.
:
: It should be assert (d2>=d1)

I think you are missreading Doron's spuedo code (either that or i am) ...
the 'greater than OR EQUAL TO' statement is regarding the current match
and the target (input to skipTo) ... i think Doron's point is that in
any calls to skipTo, no matter what the input (target) is, the
second call should advance beyond the "current" match so assert(d2 != d1)
is valid ... assert(d1 < d2) should also be valid.  it just so happens
that in this case the input to skipTo happens to bethe same as a previous
call.

I believe an equally valid test would be...

   Scorer s = getScorerFromSomewhere()
   if (s.next()) {
 int d1 = s.doc();
 if (s.skipTo(d1-1)) {
assert(d1 < s.doc());
 }
   }


-Hoss


-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]



Re: scorer.skipTo() contr

2007-06-06 Thread Michael Busch

robert engels wrote:
The method states 'greater than OR EQUAL TO' so your d1 != d2 test is 
invalid.


It should be assert (d2>=d1)


Well, but the javadoc says BEYOND the current.

But I think it should be the desired behavior for skipTo() to not skip 
at all
if curDoc==target already? Which means we should clearify the javadocs. 
But it

also means that we have to check if all scorers perform correctly in this
regard then.


-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]



Re: scorer.skipTo() contr

2007-06-06 Thread Doron Cohen
> The method states 'greater than OR EQUAL TO' so
> your d1 != d2 test is  invalid.
>
> It should be assert (d2>=d1)

Yes, the '>=' is regarding to the skip target, I am okay with this.

But "beyond the current" regards to where the scorer is currently
located, ie what doc() would have returned just before the second
call to skipTo()...


-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]



Re: scorer.skipTo() contr

2007-06-06 Thread Doron Cohen
Chris Hostetter wrote:

> skipTo semantics have always made my head hurt, but it may be that the
> contract is expressed too simplisticly ... i believe the expectation is
> that while skipTo should always "skip to the first match beyond the
> current whose document number is greater than or equal to a
> given target."
> there is a unwritten expectation that the "target" will always be greater
> then the current s.doc().
>
> I may be wrong about that ... i am afterall speculating about
> an unwritten
> assumption.
>
> The "behaves as if written" clause of the skipTo docs however make it
> clear that anytime skipTo is called, the document should be advanced at
> least one match no matter what the input is .. and your test seems to
> indicate this is also not the case.

Ok, I'll take it to JIRA then, so we can discuss if/how to fix this.


-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]



Re: scorer.skipTo() contr

2007-06-06 Thread Chris Hostetter
: Hi, following LUCENE-912, looking closer at skipTo() in
: DisjunctionMaxScorer, it seems not to follow Scorer.skipTo() "contract":
:
:  "Skips to the first match beyond the current whose document number is
: greater than or equal to a given target."
:
: It is the "__beyond__ the current" requirement that is not satisfied.

skipTo semantics have always made my head hurt, but it may be that the
contract is expressed too simplisticly ... i believe the expectation is
that while skipTo should always "skip to the first match beyond the
current whose document number is greater than or equal to a given target."
there is a unwritten expectation that the "target" will always be greater
then the current s.doc().

I may be wrong about that ... i am afterall speculating about an unwritten
assumption.

The "behaves as if written" clause of the skipTo docs however make it
clear that anytime skipTo is called, the document should be advanced at
least one match no matter what the input is .. and your test seems to
indicate this is also not the case.




-Hoss


-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]



Re: scorer.skipTo() contr

2007-06-06 Thread robert engels
The method states 'greater than OR EQUAL TO' so your d1 != d2 test is  
invalid.


It should be assert (d2>=d1)

On Jun 6, 2007, at 6:30 PM, Doron Cohen wrote:



Hi, following LUCENE-912, looking closer at skipTo() in
DisjunctionMaxScorer, it seems not to follow Scorer.skipTo()  
"contract":


 "Skips to the first match beyond the current whose document  
number is

greater than or equal to a given target."

It is the "__beyond__ the current" requirement that is not satisfied.

Enhancing QueryUtils.checkSkipTo again found many failures. Roughly  
the new

test is:

s.skipTo(n);
d1 = s.doc();
s.skipTo(n);// n==current
d2 = s.doc();
assert (d1 != d2)

If failed for so many test cases that I have to suspect maybe I am  
missing

something trivial?


-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]




-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]



Re: Lucene Compression

2007-06-06 Thread Chris Hostetter

First off: please don't send questions about *using* the Lucene Java
library to the java-dev list ... it is for discussion about *developing*
the internals of hte Lucene Java API. the [EMAIL PROTECTED] list is for
discussion about *using* the API in your own applications.

If you hav any followup questions, please start a new thread on that list.

: *   document.add(new Field("contents",contents,
: Field.Store.YES,Field.Index.TOKENIZED));

I'm going to guess that you don't really need to "store" that field since
it seems to be a meaningless concatenation of a bunch of data that you've
already stored in other fields.  Field.Store.NO will probably cut down
your index size by a significant ammount.


-Hoss


-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]



scorer.skipTo() contr

2007-06-06 Thread Doron Cohen

Hi, following LUCENE-912, looking closer at skipTo() in
DisjunctionMaxScorer, it seems not to follow Scorer.skipTo() "contract":

 "Skips to the first match beyond the current whose document number is
greater than or equal to a given target."

It is the "__beyond__ the current" requirement that is not satisfied.

Enhancing QueryUtils.checkSkipTo again found many failures. Roughly the new
test is:

s.skipTo(n);
d1 = s.doc();
s.skipTo(n);// n==current
d2 = s.doc();
assert (d1 != d2)

If failed for so many test cases that I have to suspect maybe I am missing
something trivial?


-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]



Re: Lucene 2.2 - Suggested schedule

2007-06-06 Thread Michael Busch

Doron Cohen wrote:


If there are no objections I plan to commit it later today.

  


+1. Thanks for taking care of these, Doron!

I'm working on LUCENE-908 in parallel. After all three are committed 
(908, 912, 913)

I will make the branch.

-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]



[jira] Updated: (LUCENE-913) Two consecutive score() calls return different scores for Bollean Queries.

2007-06-06 Thread Michael Busch (JIRA)

 [ 
https://issues.apache.org/jira/browse/LUCENE-913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Busch updated LUCENE-913:
-

Fix Version/s: 2.2

> Two consecutive score() calls return different scores for Bollean Queries.
> --
>
> Key: LUCENE-913
> URL: https://issues.apache.org/jira/browse/LUCENE-913
> Project: Lucene - Java
>  Issue Type: Bug
>  Components: Search
>Reporter: Doron Cohen
>Assignee: Doron Cohen
>Priority: Minor
> Fix For: 2.2
>
>
> Two consecutive calls to score() return different scores (no next() or 
> skipTo() calls in between). 
> Background in LUCENE-912 .

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]



Re: Lucene 2.2 - Suggested schedule

2007-06-06 Thread Doron Cohen
Michael Busch wrote:

> I haven't branched the trunk yet (I will send a different mail to this
> list afterwards).
> Since the patches are in place we might want to commit LUCENE-912 and
> LUCENE-913 before?

If there are no objections I plan to commit it later today.


-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]



Re: Lucene 2.2 - Suggested schedule

2007-06-06 Thread Yonik Seeley

On 6/6/07, Michael Busch <[EMAIL PROTECTED]> wrote:

I haven't branched the trunk yet (I will send a different mail to this
list afterwards).
Since the patches are in place we might want to commit LUCENE-912 and
LUCENE-913 before?


+1, these are medium-serious bugs with fixes that look unlikely to
have unwanted side-effects.

In any event, a branch can be made at any point in the past.

-Yonik

-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]



Re: Please help testing the release files

2007-06-06 Thread Chris Hostetter
: Hmmm, it is all nicely packaged in source, which satisfies the
: discussion you listed below.  As for the binary distributions, they
: are pretty much worthless unless you have some way of knowing what
: the dependencies are, right?  Or am I missing something?

I would assume so, but i don't use any contribs with dependencies, so i
don't really have a strong sense of the issue.  (it's not clear to me how
anyone ever used these contribs in the past, so i can't say whether it's
"useles" moving forward.)



-Hoss


-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]



Re: Please help testing the release files

2007-06-06 Thread Grant Ingersoll
Hmmm, it is all nicely packaged in source, which satisfies the  
discussion you listed below.  As for the binary distributions, they  
are pretty much worthless unless you have some way of knowing what  
the dependencies are, right?  Or am I missing something?



On Jun 6, 2007, at 5:45 PM, Chris Hostetter wrote:



: I feel like I am forgetting something with all the emails flying
: around about packaging, but don't the contrib jars also need their
: libs packaged with them in the binary distribution?  Or at least a
: way to specify what they are and where to get them?  They are there
: in the source distribution.

This was discussed back during the 2.1 release - but there the  
focus was
on making the contribs in the src-dist compilable ... there wasn't  
much

discussion about dependencies for the binary dist.

http://www.nabble.com/-VOTE--release-Lucene-2.1- 
tf3228536.html#a8981287
http://www.nabble.com/-VOTE--release-Lucene-2.1- 
tf3228536.html#a8980616



-Hoss


-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]





-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]



Re: Lucene 2.2 - Suggested schedule

2007-06-06 Thread Michael Busch

Doron Cohen wrote:


Michael, is there a need to hold commits to trunk while
the new branch is created?
  

No, normal trunk development may continue as usual.


(This is with LUCENE-912 and LUCENE-913 in mind - I think
they can wait for 2.3, there would always be new issues.)


  


I haven't branched the trunk yet (I will send a different mail to this 
list afterwards).

Since the patches are in place we might want to commit LUCENE-912 and
LUCENE-913 before?

-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]



Re: Lucene 2.2 - Suggested schedule

2007-06-06 Thread Doron Cohen
> --> Feature freeze from Wednesday (06/06)
>
> All features must be checked in by end of Tuesday. On Wednesday I will

Michael, is there a need to hold commits to trunk while
the new branch is created?

(This is with LUCENE-912 and LUCENE-913 in mind - I think
they can wait for 2.3, there would always be new issues.)


-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]



[jira] Created: (LUCENE-913) Two consecutive score() calls return different scores for Bollean Queries.

2007-06-06 Thread Doron Cohen (JIRA)
Two consecutive score() calls return different scores for Bollean Queries.
--

 Key: LUCENE-913
 URL: https://issues.apache.org/jira/browse/LUCENE-913
 Project: Lucene - Java
  Issue Type: Bug
  Components: Search
Reporter: Doron Cohen
Assignee: Doron Cohen
Priority: Minor


Two consecutive calls to score() return different scores (no next() or skipTo() 
calls in between). 
Background in LUCENE-912 .


-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]



Re: Please help testing the release files

2007-06-06 Thread Chris Hostetter

: I feel like I am forgetting something with all the emails flying
: around about packaging, but don't the contrib jars also need their
: libs packaged with them in the binary distribution?  Or at least a
: way to specify what they are and where to get them?  They are there
: in the source distribution.

This was discussed back during the 2.1 release - but there the focus was
on making the contribs in the src-dist compilable ... there wasn't much
discussion about dependencies for the binary dist.

http://www.nabble.com/-VOTE--release-Lucene-2.1-tf3228536.html#a8981287
http://www.nabble.com/-VOTE--release-Lucene-2.1-tf3228536.html#a8980616


-Hoss


-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]



[jira] Assigned: (LUCENE-912) DisjunctionMaxScorer.skipTo has bug that keeps it from skipping

2007-06-06 Thread Doron Cohen (JIRA)

 [ 
https://issues.apache.org/jira/browse/LUCENE-912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Doron Cohen reassigned LUCENE-912:
--

Assignee: Doron Cohen

> DisjunctionMaxScorer.skipTo has bug that keeps it from skipping
> ---
>
> Key: LUCENE-912
> URL: https://issues.apache.org/jira/browse/LUCENE-912
> Project: Lucene - Java
>  Issue Type: Bug
>Affects Versions: 2.0.0, 2.1
>Reporter: Hoss Man
>Assignee: Doron Cohen
> Fix For: 2.2
>
> Attachments: checkTwoCallsToScore.patch, checkTwoCallsToScore.patch, 
> dismax_skipto.patch, lucene-912.patch
>
>
> as reported on the mailing list, DisjunctionMaxScorer.skipTo is broken if 
> called before next in some situations...
> http://www.nabble.com/Potential-issue-with-DisjunctionMaxScorer-tf3846366.html#a10894987

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]



[jira] Commented: (LUCENE-912) DisjunctionMaxScorer.skipTo has bug that keeps it from skipping

2007-06-06 Thread Doron Cohen (JIRA)

[ 
https://issues.apache.org/jira/browse/LUCENE-912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12502121
 ] 

Doron Cohen commented on LUCENE-912:


Actually I changed my mind  (and saw your reply just before clicking "Add" 
on this comment): 
  Taking back my last comment - with Michael's fix there are no extra cycles in 
the expected use of score().
  If we try to avoid re-computation in case score() is called twice for same 
doc, there would be an additional if() for every call to score().

I think I will take it from here to a new issue and handle these two separately.


> DisjunctionMaxScorer.skipTo has bug that keeps it from skipping
> ---
>
> Key: LUCENE-912
> URL: https://issues.apache.org/jira/browse/LUCENE-912
> Project: Lucene - Java
>  Issue Type: Bug
>Affects Versions: 2.0.0, 2.1
>Reporter: Hoss Man
> Fix For: 2.2
>
> Attachments: checkTwoCallsToScore.patch, checkTwoCallsToScore.patch, 
> dismax_skipto.patch, lucene-912.patch
>
>
> as reported on the mailing list, DisjunctionMaxScorer.skipTo is broken if 
> called before next in some situations...
> http://www.nabble.com/Potential-issue-with-DisjunctionMaxScorer-tf3846366.html#a10894987

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]



[jira] Commented: (LUCENE-912) DisjunctionMaxScorer.skipTo has bug that keeps it from skipping

2007-06-06 Thread Michael Busch (JIRA)

[ 
https://issues.apache.org/jira/browse/LUCENE-912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12502120
 ] 

Michael Busch commented on LUCENE-912:
--

Yes I agree. Keeping the score in BooleanScorer until next() or skipTo() is 
called is a good optimization to avoid recomputation. I guess I was just 
curios about the reasons why the tests failed. 

> DisjunctionMaxScorer.skipTo has bug that keeps it from skipping
> ---
>
> Key: LUCENE-912
> URL: https://issues.apache.org/jira/browse/LUCENE-912
> Project: Lucene - Java
>  Issue Type: Bug
>Affects Versions: 2.0.0, 2.1
>Reporter: Hoss Man
> Fix For: 2.2
>
> Attachments: checkTwoCallsToScore.patch, checkTwoCallsToScore.patch, 
> dismax_skipto.patch, lucene-912.patch
>
>
> as reported on the mailing list, DisjunctionMaxScorer.skipTo is broken if 
> called before next in some situations...
> http://www.nabble.com/Potential-issue-with-DisjunctionMaxScorer-tf3846366.html#a10894987

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]



[jira] Commented: (LUCENE-912) DisjunctionMaxScorer.skipTo has bug that keeps it from skipping

2007-06-06 Thread Doron Cohen (JIRA)

[ 
https://issues.apache.org/jira/browse/LUCENE-912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12502111
 ] 

Doron Cohen commented on LUCENE-912:


This would recompute, but correctly. 
If this fix is just for sanity, ie main search methods really call score() just 
once, then I guess this is good enough.
I was thinking more on not recomputing score if it was already computed (for 
current doc).
(That would require to also maintain the last score returned.)


> DisjunctionMaxScorer.skipTo has bug that keeps it from skipping
> ---
>
> Key: LUCENE-912
> URL: https://issues.apache.org/jira/browse/LUCENE-912
> Project: Lucene - Java
>  Issue Type: Bug
>Affects Versions: 2.0.0, 2.1
>Reporter: Hoss Man
> Fix For: 2.2
>
> Attachments: checkTwoCallsToScore.patch, checkTwoCallsToScore.patch, 
> dismax_skipto.patch, lucene-912.patch
>
>
> as reported on the mailing list, DisjunctionMaxScorer.skipTo is broken if 
> called before next in some situations...
> http://www.nabble.com/Potential-issue-with-DisjunctionMaxScorer-tf3846366.html#a10894987

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]



[jira] Commented: (LUCENE-908) MANIFEST.MF cleanup (main jar and luci customizations)

2007-06-06 Thread Hoss Man (JIRA)

[ 
https://issues.apache.org/jira/browse/LUCENE-908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12502104
 ] 

Hoss Man commented on LUCENE-908:
-

A couple of random thoughts...

1) macro's can take multiple optional named  tags to embed in their 
bodies ... so instead of declaring a refid for what to include in the metinf, 
callers of the macro could put  call directly in the call to 

2) one way to reduce some redundancy in the build files (between jar-core, 
jar-demo, and war-demo) might be to use the  task instead of the 
 sub element of the  task ... there are a few subtle differences 
but the main key is that the  taks let's you build up a file which 
you can then refer to by name from the  task ... we could have a single 
 macro with all of the common attributes in it and then it could 
be called from the various jar/war targets just before building the actual jar 
using attributes and  tags to customize things that need to be 
different. 


...neither of these are crucial, they're just things you might want to consider 
to keep the build files smaller (and arguably simpler)

> MANIFEST.MF cleanup (main jar and luci customizations)
> --
>
> Key: LUCENE-908
> URL: https://issues.apache.org/jira/browse/LUCENE-908
> Project: Lucene - Java
>  Issue Type: Bug
>  Components: Build
>Reporter: Michael Busch
>Assignee: Michael Busch
>Priority: Trivial
> Fix For: 2.2
>
> Attachments: lucene-908.patch, LUCENE-908.patch
>
>
> there are several problems with the MANIFEST.MF file used in the core jar, 
> and some inconsistencies in th luci jar:
> Lucli's build.xml has an own "jar" target and does not use the jar target 
> from common-build.xml. The result
> is that the MANIFEST.MF file is not consistent and the META-INF dir does not 
> contain LICENSE.TXT and NOTICE.TXT.
> Is there a reason why lucli behaves different in this regard? If not I think 
> we should fix this.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]



[jira] Updated: (LUCENE-912) DisjunctionMaxScorer.skipTo has bug that keeps it from skipping

2007-06-06 Thread Michael Busch (JIRA)

 [ 
https://issues.apache.org/jira/browse/LUCENE-912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Busch updated LUCENE-912:
-

Attachment: lucene-912.patch

In BooleanScorer2.score() the coordinator is initialized: coordinator.initDoc();
This results in Coordinator.nrMatchers = 0. 

Now look at the statements that change the value of nrMatchers in BooleanScorer.
There are three of those. E. g. in countingDisjunctionSumScorer:
if (this.doc() > lastScoredDoc) {
  lastScoredDoc = this.doc();
  coordinator.nrMatchers += super.nrMatchers;
}

nrMatchers is only increased in case this.doc() is greater than lastScoredDoc.
But if score() is called twice, than those values are equal. So we have to
fix the if statement to 
if (this.doc() >= lastScoredDoc) {

With this patch and dismax_skipto.patch now all tests pass (including your
new ones, Doron).


> DisjunctionMaxScorer.skipTo has bug that keeps it from skipping
> ---
>
> Key: LUCENE-912
> URL: https://issues.apache.org/jira/browse/LUCENE-912
> Project: Lucene - Java
>  Issue Type: Bug
>Affects Versions: 2.0.0, 2.1
>Reporter: Hoss Man
> Fix For: 2.2
>
> Attachments: checkTwoCallsToScore.patch, checkTwoCallsToScore.patch, 
> dismax_skipto.patch, lucene-912.patch
>
>
> as reported on the mailing list, DisjunctionMaxScorer.skipTo is broken if 
> called before next in some situations...
> http://www.nabble.com/Potential-issue-with-DisjunctionMaxScorer-tf3846366.html#a10894987

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]



[jira] Updated: (LUCENE-912) DisjunctionMaxScorer.skipTo has bug that keeps it from skipping

2007-06-06 Thread Doron Cohen (JIRA)

 [ 
https://issues.apache.org/jira/browse/LUCENE-912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Doron Cohen updated LUCENE-912:
---

Attachment: checkTwoCallsToScore.patch

Updating  checkTwoCallsToScore.patch  (unintended comment out in previous file).

> DisjunctionMaxScorer.skipTo has bug that keeps it from skipping
> ---
>
> Key: LUCENE-912
> URL: https://issues.apache.org/jira/browse/LUCENE-912
> Project: Lucene - Java
>  Issue Type: Bug
>Affects Versions: 2.0.0, 2.1
>Reporter: Hoss Man
> Fix For: 2.2
>
> Attachments: checkTwoCallsToScore.patch, checkTwoCallsToScore.patch, 
> dismax_skipto.patch
>
>
> as reported on the mailing list, DisjunctionMaxScorer.skipTo is broken if 
> called before next in some situations...
> http://www.nabble.com/Potential-issue-with-DisjunctionMaxScorer-tf3846366.html#a10894987

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]



[jira] Updated: (LUCENE-912) DisjunctionMaxScorer.skipTo has bug that keeps it from skipping

2007-06-06 Thread Doron Cohen (JIRA)

 [ 
https://issues.apache.org/jira/browse/LUCENE-912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Doron Cohen updated LUCENE-912:
---

Attachment: checkTwoCallsToScore.patch

Attached adds to QueryUtils.check(Query q1, Searcher s):
- score() stability test (two calls to score() should return same score)
- first skipTo tests

The score() stability tests has failures for BooleanQueries and for 
DisjunctionMaxQuery.

I didn't look into the failures cause yet.

> DisjunctionMaxScorer.skipTo has bug that keeps it from skipping
> ---
>
> Key: LUCENE-912
> URL: https://issues.apache.org/jira/browse/LUCENE-912
> Project: Lucene - Java
>  Issue Type: Bug
>Affects Versions: 2.0.0, 2.1
>Reporter: Hoss Man
> Fix For: 2.2
>
> Attachments: checkTwoCallsToScore.patch, dismax_skipto.patch
>
>
> as reported on the mailing list, DisjunctionMaxScorer.skipTo is broken if 
> called before next in some situations...
> http://www.nabble.com/Potential-issue-with-DisjunctionMaxScorer-tf3846366.html#a10894987

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]



[jira] Updated: (LUCENE-908) MANIFEST.MF cleanup (main jar and luci customizations)

2007-06-06 Thread Michael Busch (JIRA)

 [ 
https://issues.apache.org/jira/browse/LUCENE-908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Busch updated LUCENE-908:
-

Attachment: lucene-908.patch

In addition to Hoss' great patch this one:

- changes the MANIFEST file in the demo jar and war in a consistent way
- adds a patternset via refid to the metainf type of jarify. This can
  be used by contrib modules to add additional files to the META-INF dir.
  I use it to add SNOWBALL-LICENSE.txt to META-INF of the snowball jar.
  
I'm planning to commit this soon.

> MANIFEST.MF cleanup (main jar and luci customizations)
> --
>
> Key: LUCENE-908
> URL: https://issues.apache.org/jira/browse/LUCENE-908
> Project: Lucene - Java
>  Issue Type: Bug
>  Components: Build
>Reporter: Michael Busch
>Assignee: Michael Busch
>Priority: Trivial
> Fix For: 2.2
>
> Attachments: lucene-908.patch, LUCENE-908.patch
>
>
> there are several problems with the MANIFEST.MF file used in the core jar, 
> and some inconsistencies in th luci jar:
> Lucli's build.xml has an own "jar" target and does not use the jar target 
> from common-build.xml. The result
> is that the MANIFEST.MF file is not consistent and the META-INF dir does not 
> contain LICENSE.TXT and NOTICE.TXT.
> Is there a reason why lucli behaves different in this regard? If not I think 
> we should fix this.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]



[jira] Commented: (LUCENE-912) DisjunctionMaxScorer.skipTo has bug that keeps it from skipping

2007-06-06 Thread Yonik Seeley (JIRA)

[ 
https://issues.apache.org/jira/browse/LUCENE-912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12502054
 ] 

Yonik Seeley commented on LUCENE-912:
-

Yes, two calls to score() should yield the same score.
What scorer was broken?


> DisjunctionMaxScorer.skipTo has bug that keeps it from skipping
> ---
>
> Key: LUCENE-912
> URL: https://issues.apache.org/jira/browse/LUCENE-912
> Project: Lucene - Java
>  Issue Type: Bug
>Affects Versions: 2.0.0, 2.1
>Reporter: Hoss Man
> Fix For: 2.2
>
> Attachments: dismax_skipto.patch
>
>
> as reported on the mailing list, DisjunctionMaxScorer.skipTo is broken if 
> called before next in some situations...
> http://www.nabble.com/Potential-issue-with-DisjunctionMaxScorer-tf3846366.html#a10894987

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]



[jira] Commented: (LUCENE-912) DisjunctionMaxScorer.skipTo has bug that keeps it from skipping

2007-06-06 Thread Doron Cohen (JIRA)

[ 
https://issues.apache.org/jira/browse/LUCENE-912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12502048
 ] 

Doron Cohen commented on LUCENE-912:


> dismax score calculation isn't incremental... it's all done during the call 
> to score()). 
> Is there an issue I'm not thinking of? 

I was wondering why QueryUtils.checkSkipTo() did not expose that. But this is 
because that check skips doc by doc, always to docs found by the tested query. 

Enhancing that test reveals that for some scorers, two consecutive calls to 
score() return different scores (no next() or skipTo() calls in between).
This seems like a bug.

> DisjunctionMaxScorer.skipTo has bug that keeps it from skipping
> ---
>
> Key: LUCENE-912
> URL: https://issues.apache.org/jira/browse/LUCENE-912
> Project: Lucene - Java
>  Issue Type: Bug
>Affects Versions: 2.0.0, 2.1
>Reporter: Hoss Man
> Fix For: 2.2
>
> Attachments: dismax_skipto.patch
>
>
> as reported on the mailing list, DisjunctionMaxScorer.skipTo is broken if 
> called before next in some situations...
> http://www.nabble.com/Potential-issue-with-DisjunctionMaxScorer-tf3846366.html#a10894987

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]



[jira] Commented: (LUCENE-908) MANIFEST.MF cleanup (main jar and luci customizations)

2007-06-06 Thread Michael Busch (JIRA)

[ 
https://issues.apache.org/jira/browse/LUCENE-908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12502040
 ] 

Michael Busch commented on LUCENE-908:
--

> * manifest file in any of gdata's jars/war (it doesn't use the 
> contrib-build.xml either) 
> * should luci's "Class-Path" refer to the full name of the lucene core jar? 

I would like to ask the contrib owners to take care of these issues.

> * spec version must match "digit+{.digit+}*" ... this is true for our 
> official releases, 
>   but broken in our nightlies. 

I will leave this for now as this patch doesn't change the spec version.

> * need to svn remove the existing luci MANIFEST file 
> * manifest file in demo war file 

Will take care...


> MANIFEST.MF cleanup (main jar and luci customizations)
> --
>
> Key: LUCENE-908
> URL: https://issues.apache.org/jira/browse/LUCENE-908
> Project: Lucene - Java
>  Issue Type: Bug
>  Components: Build
>Reporter: Michael Busch
>Assignee: Michael Busch
>Priority: Trivial
> Fix For: 2.2
>
> Attachments: LUCENE-908.patch
>
>
> there are several problems with the MANIFEST.MF file used in the core jar, 
> and some inconsistencies in th luci jar:
> Lucli's build.xml has an own "jar" target and does not use the jar target 
> from common-build.xml. The result
> is that the MANIFEST.MF file is not consistent and the META-INF dir does not 
> contain LICENSE.TXT and NOTICE.TXT.
> Is there a reason why lucli behaves different in this regard? If not I think 
> we should fix this.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]



[jira] Commented: (LUCENE-912) DisjunctionMaxScorer.skipTo has bug that keeps it from skipping

2007-06-06 Thread Yonik Seeley (JIRA)

[ 
https://issues.apache.org/jira/browse/LUCENE-912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12502011
 ] 

Yonik Seeley commented on LUCENE-912:
-

> I'm not sure, but the patch allows to increment all generators right after 
> the first time, and that might cause it to make a mistake in the score 
> computation for its first doc. 

If all the sub-scorers are on the correct document after skipTo is called 
(which looks to be the case), then everything should be OK when 
DisjunctionMaxScorer.score() is called after that (dismax score calculation 
isn't incremental... it's all done during the call to score()).  Is there an 
issue I'm not thinking of?



> DisjunctionMaxScorer.skipTo has bug that keeps it from skipping
> ---
>
> Key: LUCENE-912
> URL: https://issues.apache.org/jira/browse/LUCENE-912
> Project: Lucene - Java
>  Issue Type: Bug
>Affects Versions: 2.0.0, 2.1
>Reporter: Hoss Man
> Fix For: 2.2
>
> Attachments: dismax_skipto.patch
>
>
> as reported on the mailing list, DisjunctionMaxScorer.skipTo is broken if 
> called before next in some situations...
> http://www.nabble.com/Potential-issue-with-DisjunctionMaxScorer-tf3846366.html#a10894987

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]



[EMAIL PROTECTED]: Project lucene-java (in module lucene-java) failed

2007-06-06 Thread Jason van Zyl
To whom it may engage...

This is an automated request, but not an unsolicited one. For 
more information please visit http://gump.apache.org/nagged.html, 
and/or contact the folk at [EMAIL PROTECTED]

Project lucene-java has an issue affecting its community integration.
This issue affects 4 projects,
 and has been outstanding for 13 runs.
The current state of this project is 'Failed', with reason 'Build Failed'.
For reference only, the following projects are affected by this:
- eyebrowse :  Web-based mail archive browsing
- jakarta-lucene :  Java Based Search Engine
- jakarta-slide :  Content Management System based on WebDAV technology
- lucene-java :  Java Based Search Engine


Full details are available at:
http://vmgump.apache.org/gump/public/lucene-java/lucene-java/index.html

That said, some information snippets are provided here.

The following annotations (debug/informational/warning/error messages) were 
provided:
 -DEBUG- Sole output [lucene-core-06062007.jar] identifier set to project name
 -DEBUG- Dependency on javacc exists, no need to add for property javacc.home.
 -INFO- Failed with reason build failed
 -INFO- Failed to extract fallback artifacts from Gump Repository



The following work was performed:
http://vmgump.apache.org/gump/public/lucene-java/lucene-java/gump_work/build_lucene-java_lucene-java.html
Work Name: build_lucene-java_lucene-java (Type: Build)
Work ended in a state of : Failed
Elapsed: 1 min 2 secs
Command Line: /opt/jdk1.5/bin/java -Djava.awt.headless=true 
-Xbootclasspath/p:/usr/local/gump/public/workspace/xml-commons/java/external/build/xml-apis.jar:/usr/local/gump/public/workspace/xml-xerces2/build/xercesImpl.jar
 org.apache.tools.ant.Main -Dgump.merge=/x1/gump/public/gump/work/merge.xml 
-Dbuild.sysclasspath=only -Dversion=06062007 
-Djavacc.home=/usr/local/gump/packages/javacc-3.1 package 
[Working Directory: /usr/local/gump/public/workspace/lucene-java]
CLASSPATH: 
/opt/jdk1.5/lib/tools.jar:/usr/local/gump/public/workspace/lucene-java/build/classes/java:/usr/local/gump/public/workspace/lucene-java/build/classes/demo:/usr/local/gump/public/workspace/lucene-java/build/classes/test:/usr/local/gump/public/workspace/lucene-java/contrib/db/bdb/lib/db-4.3.29.jar:/usr/local/gump/public/workspace/ant/dist/lib/ant-jmf.jar:/usr/local/gump/public/workspace/ant/dist/lib/ant-swing.jar:/usr/local/gump/public/workspace/ant/dist/lib/ant-apache-resolver.jar:/usr/local/gump/public/workspace/ant/dist/lib/ant-trax.jar:/usr/local/gump/public/workspace/ant/dist/lib/ant-junit.jar:/usr/local/gump/public/workspace/ant/dist/lib/ant-launcher.jar:/usr/local/gump/public/workspace/ant/dist/lib/ant-nodeps.jar:/usr/local/gump/public/workspace/ant/dist/lib/ant.jar:/usr/local/gump/packages/junit3.8.1/junit.jar:/usr/local/gump/public/workspace/xml-commons/java/build/resolver.jar:/usr/local/gump/packages/je-1.7.1/lib/je.jar:/usr/local/gump/public/workspace/jakarta-commons/digester/dist/commons-digester.jar:/usr/local/gump/public/workspace/jakarta-regexp/build/jakarta-regexp-06062007.jar:/usr/local/gump/packages/javacc-3.1/bin/lib/javacc.jar:/usr/local/gump/public/workspace/jline/target/jline-0.9.92-SNAPSHOT.jar:/usr/local/gump/packages/jtidy-04aug2000r7-dev/build/Tidy.jar:/usr/local/gump/public/workspace/junit/dist/junit-06062007.jar:/usr/local/gump/public/workspace/xml-commons/java/external/build/xml-apis-ext.jar
-
[javac]   ^
[javac] 
/x1/gump/public/workspace/lucene-java/contrib/gdata-server/src/core/src/java/org/apache/lucene/gdata/data/ServerBaseFeed.java:142:
 cannot find symbol
[javac] symbol  : class Link
[javac] location: class org.apache.lucene.gdata.data.ServerBaseFeed
[javac] public void addLinks(final Collection collection) {
[javac]   ^
[javac] 
/x1/gump/public/workspace/lucene-java/contrib/gdata-server/src/core/src/java/org/apache/lucene/gdata/data/ServerBaseFeed.java:185:
 cannot find symbol
[javac] symbol  : class Person
[javac] location: class org.apache.lucene.gdata.data.ServerBaseFeed
[javac] public void addAuthor(final Person person) {
[javac] ^
[javac] 
/x1/gump/public/workspace/lucene-java/contrib/gdata-server/src/core/src/java/org/apache/lucene/gdata/data/ServerBaseFeed.java:193:
 cannot find symbol
[javac] symbol  : class BaseEntry
[javac] location: class org.apache.lucene.gdata.data.ServerBaseFeed
[javac] public BaseEntry createEntry() {
[javac]^
[javac] 
/x1/gump/public/workspace/lucene-java/contrib/gdata-server/src/core/src/java/org/apache/lucene/gdata/data/ServerBaseFeed.java:202:
 cannot find symbol
[javac] symbol  : class XmlWriter
[javac] location: class org.apache.lucene.gdata.data.ServerBaseFeed
[javac] public void generateAtom(XmlWriter arg0, ExtensionProfile arg1) 
throws IOException {
[j

[EMAIL PROTECTED]: Project lucene-java (in module lucene-java) failed

2007-06-06 Thread Jason van Zyl
To whom it may engage...

This is an automated request, but not an unsolicited one. For 
more information please visit http://gump.apache.org/nagged.html, 
and/or contact the folk at [EMAIL PROTECTED]

Project lucene-java has an issue affecting its community integration.
This issue affects 4 projects,
 and has been outstanding for 13 runs.
The current state of this project is 'Failed', with reason 'Build Failed'.
For reference only, the following projects are affected by this:
- eyebrowse :  Web-based mail archive browsing
- jakarta-lucene :  Java Based Search Engine
- jakarta-slide :  Content Management System based on WebDAV technology
- lucene-java :  Java Based Search Engine


Full details are available at:
http://vmgump.apache.org/gump/public/lucene-java/lucene-java/index.html

That said, some information snippets are provided here.

The following annotations (debug/informational/warning/error messages) were 
provided:
 -DEBUG- Sole output [lucene-core-06062007.jar] identifier set to project name
 -DEBUG- Dependency on javacc exists, no need to add for property javacc.home.
 -INFO- Failed with reason build failed
 -INFO- Failed to extract fallback artifacts from Gump Repository



The following work was performed:
http://vmgump.apache.org/gump/public/lucene-java/lucene-java/gump_work/build_lucene-java_lucene-java.html
Work Name: build_lucene-java_lucene-java (Type: Build)
Work ended in a state of : Failed
Elapsed: 1 min 2 secs
Command Line: /opt/jdk1.5/bin/java -Djava.awt.headless=true 
-Xbootclasspath/p:/usr/local/gump/public/workspace/xml-commons/java/external/build/xml-apis.jar:/usr/local/gump/public/workspace/xml-xerces2/build/xercesImpl.jar
 org.apache.tools.ant.Main -Dgump.merge=/x1/gump/public/gump/work/merge.xml 
-Dbuild.sysclasspath=only -Dversion=06062007 
-Djavacc.home=/usr/local/gump/packages/javacc-3.1 package 
[Working Directory: /usr/local/gump/public/workspace/lucene-java]
CLASSPATH: 
/opt/jdk1.5/lib/tools.jar:/usr/local/gump/public/workspace/lucene-java/build/classes/java:/usr/local/gump/public/workspace/lucene-java/build/classes/demo:/usr/local/gump/public/workspace/lucene-java/build/classes/test:/usr/local/gump/public/workspace/lucene-java/contrib/db/bdb/lib/db-4.3.29.jar:/usr/local/gump/public/workspace/ant/dist/lib/ant-jmf.jar:/usr/local/gump/public/workspace/ant/dist/lib/ant-swing.jar:/usr/local/gump/public/workspace/ant/dist/lib/ant-apache-resolver.jar:/usr/local/gump/public/workspace/ant/dist/lib/ant-trax.jar:/usr/local/gump/public/workspace/ant/dist/lib/ant-junit.jar:/usr/local/gump/public/workspace/ant/dist/lib/ant-launcher.jar:/usr/local/gump/public/workspace/ant/dist/lib/ant-nodeps.jar:/usr/local/gump/public/workspace/ant/dist/lib/ant.jar:/usr/local/gump/packages/junit3.8.1/junit.jar:/usr/local/gump/public/workspace/xml-commons/java/build/resolver.jar:/usr/local/gump/packages/je-1.7.1/lib/je.jar:/usr/local/gump/public/workspace/jakarta-commons/digester/dist/commons-digester.jar:/usr/local/gump/public/workspace/jakarta-regexp/build/jakarta-regexp-06062007.jar:/usr/local/gump/packages/javacc-3.1/bin/lib/javacc.jar:/usr/local/gump/public/workspace/jline/target/jline-0.9.92-SNAPSHOT.jar:/usr/local/gump/packages/jtidy-04aug2000r7-dev/build/Tidy.jar:/usr/local/gump/public/workspace/junit/dist/junit-06062007.jar:/usr/local/gump/public/workspace/xml-commons/java/external/build/xml-apis-ext.jar
-
[javac]   ^
[javac] 
/x1/gump/public/workspace/lucene-java/contrib/gdata-server/src/core/src/java/org/apache/lucene/gdata/data/ServerBaseFeed.java:142:
 cannot find symbol
[javac] symbol  : class Link
[javac] location: class org.apache.lucene.gdata.data.ServerBaseFeed
[javac] public void addLinks(final Collection collection) {
[javac]   ^
[javac] 
/x1/gump/public/workspace/lucene-java/contrib/gdata-server/src/core/src/java/org/apache/lucene/gdata/data/ServerBaseFeed.java:185:
 cannot find symbol
[javac] symbol  : class Person
[javac] location: class org.apache.lucene.gdata.data.ServerBaseFeed
[javac] public void addAuthor(final Person person) {
[javac] ^
[javac] 
/x1/gump/public/workspace/lucene-java/contrib/gdata-server/src/core/src/java/org/apache/lucene/gdata/data/ServerBaseFeed.java:193:
 cannot find symbol
[javac] symbol  : class BaseEntry
[javac] location: class org.apache.lucene.gdata.data.ServerBaseFeed
[javac] public BaseEntry createEntry() {
[javac]^
[javac] 
/x1/gump/public/workspace/lucene-java/contrib/gdata-server/src/core/src/java/org/apache/lucene/gdata/data/ServerBaseFeed.java:202:
 cannot find symbol
[javac] symbol  : class XmlWriter
[javac] location: class org.apache.lucene.gdata.data.ServerBaseFeed
[javac] public void generateAtom(XmlWriter arg0, ExtensionProfile arg1) 
throws IOException {
[j

Lucene Compression

2007-06-06 Thread Sebastin Naveen

Hi All,
   I am a lucene developer. i saw your benchmark in lucene website
http://lucene.apache.org
I have records upto 45 GB.when i compress the record it goes to 80
GB.howcan i compress to 10 GB or lower than that.
please help me in this regards.

i have send you the source code that i use:



*public class MediationIndexer {

public static void main(String[] args) throws Exception{
   String slNo="";
String fileName="";
String callType="";
String callingPartyNumber="";
String calledPartyNumber="";
String dateSc="";
String timeSc="";
String chargDur="";
String outgoingRoute="";
String incomingRoute="";
String orgCalledNumber="";
String redirectingNumber="";
String imsiNumber="";



   File   indexDir = new File("C:/Sample/Mediatio/Index");
   Analyzer analyzer = new StandardAnalyzer();
   IndexWriter indexWriter = new IndexWriter(indexDir,analyzer,true);
  // indexWriter.setUseCompoundFile(true);

   File mediationFiles=new File("C:/mediation files");
   File   fileDir  = new File("C:/mediation files");
   long startTime = new Date().getTime();
   String mediFiles[]=mediationFiles.list();
   for(int j=0;jhttp://field.index.no/>));
  document.add(new
Field("callingPartyNumber",callingPartyNumber,Field.Store.YES,Field.Index.NO));

  document.add(new
Field("calledPartyNumber",calledPartyNumber,Field.Store.YES,Field.Index.NO
));
  document.add(new Field("dateSc",dateSc,
Field.Store.YES,Field.Index.TOKENIZED));
  document.add(new Field("timeSc",timeSc,
Field.Store.YES,Field.Index.NO ));
  document.add (new Field("chargDur",chargDur,
Field.Store.YES,Field.Index.NO ));
  document.add(new Field("outgoingRoute",outgoingRoute,
Field.Store.YES, Field.Index.NO ));
  document.add(new Field("incomingRoute",incomingRoute,
Field.Store.YES,Field.Index.NO ));
  }



  record = br.readLine();
  if(record.equalsIgnoreCase("")) {
  record=null;
  }


  }


 }catch (IOException e) {
  // catch possible io errors from readLine()
  e.printStackTrace();
   }
   }
   }
   long endTime = new Date().getTime();
   System.out.println("It took " + (endTime - startTime)
+ " milliseconds to create an index for the files in the
directory "
+ fileDir.getPath());
   }


}
*


--
Regards,
Sebastin Naveen


Re: Please help testing the release files

2007-06-06 Thread Grant Ingersoll
I feel like I am forgetting something with all the emails flying  
around about packaging, but don't the contrib jars also need their  
libs packaged with them in the binary distribution?  Or at least a  
way to specify what they are and where to get them?  They are there  
in the source distribution.


On Jun 5, 2007, at 4:23 PM, Michael Busch wrote:


Michael Busch wrote:


I checked on Ubuntu Linux 7.0.4 32 Bit.

With all Sun JDKs 1.4, 5.0 and 6.0:

lucene-2.2-dev.tar.gz:
 + md5: OK
 + LICENSE.TXT:  ? (see below)
 + NOTICE.TXT:  ? (see below)
 + ant clean war-demo: OK

lucene-2.2-dev-src.tar.gz:
 + md5: OK
 + ant clean test: OK


* the two files LICENSE.TXT and NOTICE.TXT are located in the  
META_INF dir
of lucene-core-2.2-dev.jar which is OK, but neither in lucene- 
demos-2.2-dev.jar, nor
in the contrib jars. I guess we have to add them to those jars as  
well?


I just fixed this with LUCENE-907 and uploaded new release files to  
the same location:

http://people.apache.org/~buschmi/staging_area/lucene/

lucene-2.2-dev.tar.gz:
+ LICENSE.TXT:  OK
+ NOTICE.TXT:  OK


-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]





-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]



[jira] Resolved: (LUCENE-905) left nav of docs/index.html in dist artifacts links to hudson for javadocs

2007-06-06 Thread Grant Ingersoll (JIRA)

 [ 
https://issues.apache.org/jira/browse/LUCENE-905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Grant Ingersoll resolved LUCENE-905.


   Resolution: Fixed
Lucene Fields:   (was: [New])

Fixed

> left nav of docs/index.html in dist artifacts links to hudson for javadocs
> --
>
> Key: LUCENE-905
> URL: https://issues.apache.org/jira/browse/LUCENE-905
> Project: Lucene - Java
>  Issue Type: Bug
>  Components: Build
>Reporter: Hoss Man
>Assignee: Grant Ingersoll
>Priority: Minor
> Fix For: 2.2
>
>
> When building the zip or tgz release artifacts, the docs/index.html file 
> contained in that release (the starter point for people to read 
> documentation) links "API Docs" to 
> http://lucene.zones.apache.org:8080/hudson/job/Lucene-Nightly/javadoc/ 
> instead of to ./api/index.html (the local copy of the javadocs)
> this relates to the initial migration to hudson for the nightly builds and a 
> plan to copy the javadocs back to lucene.apache.org that wasn't considered 
> urgent since it was just for transient nightly docs, but a side affect is 
> that the release documentation also links to hudson.
> even if we don't modify the nightly build process before the 2.2 release, we 
> should update the link in the left nav in the 2.2 release branch before 
> building the final release.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]



[jira] Commented: (LUCENE-912) DisjunctionMaxScorer.skipTo has bug that keeps it from skipping

2007-06-06 Thread Paul Elschot (JIRA)

[ 
https://issues.apache.org/jira/browse/LUCENE-912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12501827
 ] 

Paul Elschot commented on LUCENE-912:
-

I'm not sure, but the patch allows to increment all generators right after the 
first time, and that might cause it to make a mistake in the score computation 
for its first doc.

It might be worthwhile to make DisjunctionMaxScorer use the same specialized 
queue/heap as DisjunctionSumScorer.
Factoring this out could lead to a common superclass that provides e.g. an 
array of Scorers that match the disjunction.
At the same time the score computations of the subscorers could be delayed 
until an actual score value is needed.
That is a rather drastic approach, but the specialized queue/heap of 
DisjunctionSumScorer might also help performance of DisjunctionMaxScorer


> DisjunctionMaxScorer.skipTo has bug that keeps it from skipping
> ---
>
> Key: LUCENE-912
> URL: https://issues.apache.org/jira/browse/LUCENE-912
> Project: Lucene - Java
>  Issue Type: Bug
>Affects Versions: 2.0.0, 2.1
>Reporter: Hoss Man
> Fix For: 2.2
>
> Attachments: dismax_skipto.patch
>
>
> as reported on the mailing list, DisjunctionMaxScorer.skipTo is broken if 
> called before next in some situations...
> http://www.nabble.com/Potential-issue-with-DisjunctionMaxScorer-tf3846366.html#a10894987

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]