[jira] Updated: (LUCENE-1536) if a filter can support random access API, we should use it

[Jason Rutherglen (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-1536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jason Rutherglen updated LUCENE-1536:
-

Attachment: LUCENE-1536.patch

* The test case includes testConstantCoreQuery

* If LUCENE-1632 were updated to use the new DocIdSetIterator API,
  it could be useful for the iterators of AndRandomAccessDocIdSet
  and NotRandomAccessDocIdSet.

> if a filter can support random access API, we should use it
> ---
>
> Key: LUCENE-1536
> URL: https://issues.apache.org/jira/browse/LUCENE-1536
> Project: Lucene - Java
>  Issue Type: Improvement
>  Components: Search
>Affects Versions: 2.4
>Reporter: Michael McCandless
>Assignee: Michael McCandless
>Priority: Minor
> Fix For: 3.1
>
> Attachments: LUCENE-1536.patch, LUCENE-1536.patch, LUCENE-1536.patch, 
> LUCENE-1536.patch, LUCENE-1536.patch
>
>
> I ran some performance tests, comparing applying a filter via
> random-access API instead of current trunk's iterator API.
> This was inspired by LUCENE-1476, where we realized deletions should
> really be implemented just like a filter, but then in testing found
> that switching deletions to iterator was a very sizable performance
> hit.
> Some notes on the test:
>   * Index is first 2M docs of Wikipedia.  Test machine is Mac OS X
> 10.5.6, quad core Intel CPU, 6 GB RAM, java 1.6.0_07-b06-153.
>   * I test across multiple queries.  1-X means an OR query, eg 1-4
> means 1 OR 2 OR 3 OR 4, whereas +1-4 is an AND query, ie 1 AND 2
> AND 3 AND 4.  "u s" means "united states" (phrase search).
>   * I test with multiple filter densities (0, 1, 2, 5, 10, 25, 75, 90,
> 95, 98, 99, 99.9 (filter is non-null but all bits are set),
> 100 (filter=null, control)).
>   * Method high means I use random-access filter API in
> IndexSearcher's main loop.  Method low means I use random-access
> filter API down in SegmentTermDocs (just like deleted docs
> today).
>   * Baseline (QPS) is current trunk, where filter is applied as iterator up
> "high" (ie in IndexSearcher's search loop).

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


-
To unsubscribe, e-mail: java-dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: java-dev-h...@lucene.apache.org

Build failed in Hudson: Lucene-trunk #950

[Apache Hudson Server]

See http://hudson.zones.apache.org/hudson/job/Lucene-trunk/950/changes

Changes:

[doronc] LUCENE-1908: Scoring documentation imrovements in Similarity javadocs.

[uschindler] LUCENE-1872: Javadocs updates of Numeric*

--
[...truncated 14477 lines...]
jflex-notice:

common.init:

build-lucene:

build-lucene-tests:

init:

compile-test:
 [echo] Building swing...

javacc-uptodate-check:

javacc-notice:

jflex-uptodate-check:

jflex-notice:

common.init:

build-lucene:

build-lucene-tests:

init:

clover.setup:

clover.info:

clover:

compile-core:

common.compile-test:

common.test:
[mkdir] Created dir: 
http://hudson.zones.apache.org/hudson/job/Lucene-trunk/ws/trunk/build/contrib/swing/test
 
[junit] Testsuite: org.apache.lucene.swing.models.TestBasicList
[junit] Tests run: 2, Failures: 0, Errors: 0, Time elapsed: 0.621 sec
[junit] 
[junit] Testsuite: org.apache.lucene.swing.models.TestBasicTable
[junit] Tests run: 3, Failures: 0, Errors: 0, Time elapsed: 0.6 sec
[junit] 
[junit] Testsuite: org.apache.lucene.swing.models.TestSearchingList
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.648 sec
[junit] 
[junit] Testsuite: org.apache.lucene.swing.models.TestSearchingTable
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.98 sec
[junit] 
[junit] Testsuite: org.apache.lucene.swing.models.TestUpdatingList
[junit] Tests run: 4, Failures: 0, Errors: 0, Time elapsed: 1.199 sec
[junit] 
[junit] Testsuite: org.apache.lucene.swing.models.TestUpdatingTable
[junit] Tests run: 4, Failures: 0, Errors: 0, Time elapsed: 0.937 sec
[junit] 
   [delete] Deleting: 
http://hudson.zones.apache.org/hudson/job/Lucene-trunk/ws/trunk/build/contrib/swing/test/junitfailed.flag
 
 [echo] Building wikipedia...

javacc-uptodate-check:

javacc-notice:

jflex-uptodate-check:

jflex-notice:

common.init:

build-lucene:

build-lucene-tests:

init:

test:
 [echo] Building wikipedia...

javacc-uptodate-check:

javacc-notice:

jflex-uptodate-check:

jflex-notice:

common.init:

build-lucene:

build-lucene-tests:

init:

compile-test:
 [echo] Building wikipedia...

javacc-uptodate-check:

javacc-notice:

jflex-uptodate-check:

jflex-notice:

common.init:

build-lucene:

build-lucene-tests:

init:

clover.setup:

clover.info:

clover:

compile-core:

common.compile-test:

common.test:
[mkdir] Created dir: 
http://hudson.zones.apache.org/hudson/job/Lucene-trunk/ws/trunk/build/contrib/wikipedia/test
 
[junit] Testsuite: 
org.apache.lucene.wikipedia.analysis.WikipediaTokenizerTest
[junit] Tests run: 5, Failures: 0, Errors: 0, Time elapsed: 0.423 sec
[junit] 
   [delete] Deleting: 
http://hudson.zones.apache.org/hudson/job/Lucene-trunk/ws/trunk/build/contrib/wikipedia/test/junitfailed.flag
 
 [echo] Building wordnet...

javacc-uptodate-check:

javacc-notice:

jflex-uptodate-check:

jflex-notice:

common.init:

build-lucene:

build-lucene-tests:

init:

test:
 [echo] Building xml-query-parser...

javacc-uptodate-check:

javacc-notice:

jflex-uptodate-check:

jflex-notice:

common.init:

build-lucene:

build-lucene-tests:

init:

test:
 [echo] Building xml-query-parser...

javacc-uptodate-check:

javacc-notice:

jflex-uptodate-check:

jflex-notice:

common.init:

build-lucene:

build-lucene-tests:

init:

compile-test:
 [echo] Building xml-query-parser...

build-queries:

javacc-uptodate-check:

javacc-notice:

jflex-uptodate-check:

jflex-notice:

common.init:

build-lucene:

build-lucene-tests:

init:

clover.setup:

clover.info:

clover:

common.compile-core:

compile-core:

common.compile-test:

common.test:
[mkdir] Created dir: 
http://hudson.zones.apache.org/hudson/job/Lucene-trunk/ws/trunk/build/contrib/xml-query-parser/test
 
[junit] Testsuite: org.apache.lucene.xmlparser.TestParser
[junit] Tests run: 18, Failures: 0, Errors: 0, Time elapsed: 3.258 sec
[junit] 
[junit] Testsuite: org.apache.lucene.xmlparser.TestQueryTemplateManager
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 1.126 sec
[junit] 
   [delete] Deleting: 
http://hudson.zones.apache.org/hudson/job/Lucene-trunk/ws/trunk/build/contrib/xml-query-parser/test/junitfailed.flag
 

download-tag:
[mkdir] Created dir: 
http://hudson.zones.apache.org/hudson/job/Lucene-trunk/ws/trunk/tags/lucene_2_4_back_compat_tests_20090911
 
 [exec] svn: PROPFIND request failed on 
'/repos/asf/lucene/java/tags/lucene_2_4_back_compat_tests_20090911/src'
 [exec] svn: PROPFIND of 
'/repos/asf/lucene/java/tags/lucene_2_4_back_compat_tests_20090911/src': could 
not connect to server (http://svn.apache.org)
 [exec] Result: 1

jar-core:
  [jar] Building jar: 
http://hudson.zones.apache.org/hudson/job/Lucene-trunk/ws/trunk/build/lucene-core-2009-09-16_02-04-23.jar
 

test-tag:

BUILD FAILED
http://hudson.zones.apache.org/hudson/job/Lucene-trunk/ws/trunk/build.xml :119: 

[jira] Updated: (LUCENE-1536) if a filter can support random access API, we should use it

[Jason Rutherglen (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-1536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jason Rutherglen updated LUCENE-1536:
-

Attachment: LUCENE-1536.patch

* Added a IndexReader.termDocs(term, filter) method which
  culminates in passing a RandomAccessDocIdSet to the scorers.
  ConstantScoreQuery and FilterQuery need to AND the filters
  together. 

* Does explain need the filter?

* Spans aren't supported yet





> if a filter can support random access API, we should use it
> ---
>
> Key: LUCENE-1536
> URL: https://issues.apache.org/jira/browse/LUCENE-1536
> Project: Lucene - Java
>  Issue Type: Improvement
>  Components: Search
>Affects Versions: 2.4
>Reporter: Michael McCandless
>Assignee: Michael McCandless
>Priority: Minor
> Fix For: 3.1
>
> Attachments: LUCENE-1536.patch, LUCENE-1536.patch, LUCENE-1536.patch, 
> LUCENE-1536.patch
>
>
> I ran some performance tests, comparing applying a filter via
> random-access API instead of current trunk's iterator API.
> This was inspired by LUCENE-1476, where we realized deletions should
> really be implemented just like a filter, but then in testing found
> that switching deletions to iterator was a very sizable performance
> hit.
> Some notes on the test:
>   * Index is first 2M docs of Wikipedia.  Test machine is Mac OS X
> 10.5.6, quad core Intel CPU, 6 GB RAM, java 1.6.0_07-b06-153.
>   * I test across multiple queries.  1-X means an OR query, eg 1-4
> means 1 OR 2 OR 3 OR 4, whereas +1-4 is an AND query, ie 1 AND 2
> AND 3 AND 4.  "u s" means "united states" (phrase search).
>   * I test with multiple filter densities (0, 1, 2, 5, 10, 25, 75, 90,
> 95, 98, 99, 99.9 (filter is non-null but all bits are set),
> 100 (filter=null, control)).
>   * Method high means I use random-access filter API in
> IndexSearcher's main loop.  Method low means I use random-access
> filter API down in SegmentTermDocs (just like deleted docs
> today).
>   * Baseline (QPS) is current trunk, where filter is applied as iterator up
> "high" (ie in IndexSearcher's search loop).

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


-
To unsubscribe, e-mail: java-dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: java-dev-h...@lucene.apache.org

[jira] Updated: (LUCENE-753) Use NIO positional read to avoid synchronization in FSIndexInput

[Yonik Seeley (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yonik Seeley updated LUCENE-753:


Attachment: FileReadTest.java

Attaching new FileReadTest.java that fixes a concurrency bug in SeparateFile - 
each reader needed it's own file position.

> Use NIO positional read to avoid synchronization in FSIndexInput
> 
>
> Key: LUCENE-753
> URL: https://issues.apache.org/jira/browse/LUCENE-753
> Project: Lucene - Java
>  Issue Type: New Feature
>  Components: Store
>Reporter: Yonik Seeley
>Assignee: Michael McCandless
> Attachments: FileReadTest.java, FileReadTest.java, FileReadTest.java, 
> FileReadTest.java, FileReadTest.java, FileReadTest.java, FileReadTest.java, 
> FileReadTest.java, FSDirectoryPool.patch, FSIndexInput.patch, 
> FSIndexInput.patch, LUCENE-753.patch, LUCENE-753.patch, LUCENE-753.patch, 
> LUCENE-753.patch, LUCENE-753.patch, lucene-753.patch, lucene-753.patch
>
>
> As suggested by Doug, we could use NIO pread to avoid synchronization on the 
> underlying file.
> This could mitigate any MT performance drop caused by reducing the number of 
> files in the index format.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


-
To unsubscribe, e-mail: java-dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: java-dev-h...@lucene.apache.org

Re: Lucene 2.9 RC4 now available for testing

[Doron Cohen]

Committed!

Doron

On Tue, Sep 15, 2009 at 3:39 PM, Mark Miller  wrote:

> Please commit the sim scoring javadoc improvements and I'll make the
> voting artifacts (quite possibly today).
>

[jira] Resolved: (LUCENE-1908) Similarity javadocs for scoring function to relate more tightly to scoring models in effect

[Doron Cohen (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-1908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Doron Cohen resolved LUCENE-1908.
-

Resolution: Fixed

Committed, 
Thanks Mark, Shai, Ted, Jiri, and Marvin!

> Similarity javadocs for scoring function to relate more tightly to scoring 
> models in effect
> ---
>
> Key: LUCENE-1908
> URL: https://issues.apache.org/jira/browse/LUCENE-1908
> Project: Lucene - Java
>  Issue Type: Improvement
>  Components: Search
>Reporter: Doron Cohen
>Assignee: Doron Cohen
>Priority: Minor
> Fix For: 2.9
>
> Attachments: LUCENE-1908.patch, LUCENE-1908.patch, LUCENE-1908.patch, 
> LUCENE-1908.patch, LUCENE-1908.patch
>
>
> See discussion in the related issue.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


-
To unsubscribe, e-mail: java-dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: java-dev-h...@lucene.apache.org

[jira] Resolved: (LUCENE-1896) Modify confusing javadoc for queryNorm

[Mark Miller (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-1896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mark Miller resolved LUCENE-1896.
-

   Resolution: Fixed
 Assignee: Mark Miller
Lucene Fields: [New, Patch Available]  (was: [New])

> Modify confusing javadoc for queryNorm
> --
>
> Key: LUCENE-1896
> URL: https://issues.apache.org/jira/browse/LUCENE-1896
> Project: Lucene - Java
>  Issue Type: Improvement
>  Components: Javadocs
>Reporter: Jiri Kuhn
>Assignee: Mark Miller
>Priority: Minor
> Fix For: 2.9
>
> Attachments: LUCENE-1896.patch, queryNormAlternate.patch
>
>
> See http://markmail.org/message/arai6silfiktwcer
> The javadoc confuses me as well.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


-
To unsubscribe, e-mail: java-dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: java-dev-h...@lucene.apache.org

Re: Lucene 2.9 RC4 now available for testing

[Mark Miller]

Please commit the sim scoring javadoc improvements and I'll make the
voting artifacts (quite possibly today).

-- 
- Mark

http://www.lucidimagination.com



Doron Cohen wrote:
> hi guys,
> Returning from a long freeze in the middle of releasing I feel rusty
> about the process, so a quick question:
>
> Will checking in documentation fixes anyhow interfere with this?
> There are 3 such open issues marked for 2.9:
> - LECENE-1908 (Similarity scoring jdocs) that is ready and I would
> like to commit.
> - LUCENE-1872 (Numeric jdocs) that was reopened and I think is too
> ready to (re) commit?
> - LUCENE-1896 (queryNorm jdocs) that is committed and I think can be
> closed actually
>
> Thanks,
> Doron
>
> On Tue, Sep 15, 2009 at 4:14 AM, Grant Ingersoll  > wrote:
>
> +1 on calling a vote.
>
>
> On Sep 14, 2009, at 7:45 PM, Yonik Seeley wrote:
>
> We could cut a *real* release candidate that we could VOTE on
> soon,
> right?  It's bee 3.5 weeks since the first release candidate.
>
>



-
To unsubscribe, e-mail: java-dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: java-dev-h...@lucene.apache.org

RE: Lucene 2.9 RC4 now available for testing

[Uwe Schindler]

So lets commit the last javadocs updates and create the final artifacts and
vote for them!

Uwe

-
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: u...@thetaphi.de

> -Original Message-
> From: Mark Miller [mailto:markrmil...@gmail.com]
> Sent: Tuesday, September 15, 2009 1:19 PM
> To: java-dev@lucene.apache.org
> Cc: java-dev@lucene.apache.org
> Subject: Re: Lucene 2.9 RC4 now available for testing
> 
> The vote will be on the final artifacts - the final artifacts will
> just contain the javadoc changes.
> 
> - Mark
> 
> http://www.lucidimagination.com (mobile)
> 
> On Sep 15, 2009, at 6:57 AM, Grant Ingersoll 
> wrote:
> 
> >
> > On Sep 15, 2009, at 5:53 AM, Michael McCandless wrote:
> >
> >> On Tue, Sep 15, 2009 at 12:22 AM, Doron Cohen 
> >> wrote:
> >>
> >>> Will checking in documentation fixes anyhow interfere with this?
> >>
> >> The tentative consensus so far is that we are free to make javadocs
> >> only changes w/o rolling a new RC.
> >
> > AIUI, the vote needs to be on the final artifacts.
> >
> > -
> > To unsubscribe, e-mail: java-dev-unsubscr...@lucene.apache.org
> > For additional commands, e-mail: java-dev-h...@lucene.apache.org
> >
> 
> -
> To unsubscribe, e-mail: java-dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: java-dev-h...@lucene.apache.org



-
To unsubscribe, e-mail: java-dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: java-dev-h...@lucene.apache.org

Re: Lucene 2.9 RC4 now available for testing

[Mark Miller]

The vote will be on the final artifacts - the final artifacts will  
just contain the javadoc changes.


- Mark

http://www.lucidimagination.com (mobile)

On Sep 15, 2009, at 6:57 AM, Grant Ingersoll   
wrote:




On Sep 15, 2009, at 5:53 AM, Michael McCandless wrote:

On Tue, Sep 15, 2009 at 12:22 AM, Doron Cohen   
wrote:



Will checking in documentation fixes anyhow interfere with this?


The tentative consensus so far is that we are free to make javadocs
only changes w/o rolling a new RC.


AIUI, the vote needs to be on the final artifacts.

-
To unsubscribe, e-mail: java-dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: java-dev-h...@lucene.apache.org



-
To unsubscribe, e-mail: java-dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: java-dev-h...@lucene.apache.org

Re: Lucene 2.9 RC4 now available for testing

[Grant Ingersoll]

On Sep 15, 2009, at 5:53 AM, Michael McCandless wrote:

On Tue, Sep 15, 2009 at 12:22 AM, Doron Cohen   
wrote:



Will checking in documentation fixes anyhow interfere with this?


The tentative consensus so far is that we are free to make javadocs
only changes w/o rolling a new RC.


AIUI, the vote needs to be on the final artifacts.

-
To unsubscribe, e-mail: java-dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: java-dev-h...@lucene.apache.org

Re: Lucene 2.9 RC4 now available for testing

[Michael McCandless]

On Tue, Sep 15, 2009 at 12:22 AM, Doron Cohen  wrote:

> Will checking in documentation fixes anyhow interfere with this?

The tentative consensus so far is that we are free to make javadocs
only changes w/o rolling a new RC.

Mike

-
To unsubscribe, e-mail: java-dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: java-dev-h...@lucene.apache.org

[jira] Resolved: (LUCENE-1872) Improve javadocs for Numeric*

[Uwe Schindler (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-1872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Uwe Schindler resolved LUCENE-1872.
---

Resolution: Fixed
  Assignee: Michael McCandless  (was: Uwe Schindler)

Committed revision: 815195

> Improve javadocs for Numeric*
> -
>
> Key: LUCENE-1872
> URL: https://issues.apache.org/jira/browse/LUCENE-1872
> Project: Lucene - Java
>  Issue Type: Improvement
>Reporter: Michael McCandless
>Assignee: Michael McCandless
>Priority: Minor
> Fix For: 2.9
>
> Attachments: LUCENE-1872-cardinality.patch, LUCENE-1872-uwe.patch, 
> LUCENE-1872.patch, LUCENE-1872.patch, LUCENE-1872.patch
>
>
> I'm working on improving Numeric* javadocs.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


-
To unsubscribe, e-mail: java-dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: java-dev-h...@lucene.apache.org

RE: Lucene 2.9 RC4 now available for testing

[Uwe Schindler]

+1 for calling a vote. I will commit the jdocs updates in LUCENE-1872 soon.

 

-
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: u...@thetaphi.de

  _  

From: Doron Cohen [mailto:cdor...@gmail.com] 
Sent: Tuesday, September 15, 2009 6:22 AM
To: java-dev@lucene.apache.org
Subject: Re: Lucene 2.9 RC4 now available for testing

 

hi guys,
Returning from a long freeze in the middle of releasing I feel rusty about
the process, so a quick question:

Will checking in documentation fixes anyhow interfere with this? 
There are 3 such open issues marked for 2.9:
- LECENE-1908 (Similarity scoring jdocs) that is ready and I would like to
commit.
- LUCENE-1872 (Numeric jdocs) that was reopened and I think is too ready to
(re) commit?
- LUCENE-1896 (queryNorm jdocs) that is committed and I think can be closed
actually

Thanks,
Doron

On Tue, Sep 15, 2009 at 4:14 AM, Grant Ingersoll 
wrote:

+1 on calling a vote.



On Sep 14, 2009, at 7:45 PM, Yonik Seeley wrote:

We could cut a *real* release candidate that we could VOTE on soon,
right?  It's bee 3.5 weeks since the first release candidate.