Re: Back Compatibility

2008-01-24 Thread Doron Cohen
On Jan 24, 2008 12:31 AM, robert engels <[EMAIL PROTECTED]> wrote: > You must get the write lock before opening the reader if you want > transactional consistency and are performing updates. > > No other way to do it. > > Otherwise. > > A opens reader. > B opens reader. > A performs query decides

Re: 2.3.0 announcement draft

2008-01-24 Thread Michael McCandless
Looks great Michael! Maybe add: * TermVectorMapper to customize how term vectors are loaded * live backups (without pausing indexing) with SnapshotDeletionPolicy * CheckIndex tool to test & recover a corrupt index * pluggable MergePolicy & MergeScheduler * "partial" optimize(int maxNum

Re: 2.3.0 announcement draft

2008-01-24 Thread Michael Busch
Michael McCandless wrote: > Looks great Michael! > > Maybe add: > > * TermVectorMapper to customize how term vectors are loaded > * live backups (without pausing indexing) with SnapshotDeletionPolicy > * CheckIndex tool to test & recover a corrupt index > * pluggable MergePolicy & MergeSc

[jira] Commented: (LUCENE-1121) Use nio.transferTo when copying large blocks of bytes

2008-01-24 Thread Michael McCandless (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1121?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12561982#action_12561982 ] Michael McCandless commented on LUCENE-1121: OK I ran Sun JDK 1.6.0_04 on Linu

Re: Back Compatibility

2008-01-24 Thread Michael McCandless
Doron Cohen wrote: --=_Part_11325_2615585.1201162438596 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline On Jan 24, 2008 12:31 AM, robert engels <[EMAIL PROTECTED]> wrote: You must get the write lock before opening the reader if you

Re: Back Compatibility

2008-01-24 Thread Michael McCandless
Grant Ingersoll wrote: Yes, I agree these are what is about (despite the divergence into locking). As I see, it the question is about whether we should try to do major releases on the order of a year, rather than the current 2+ year schedule and also how to best handle bad behavior when

Re: Unique doc ids

2008-01-24 Thread Michael McCandless
Yonik Seeley wrote: On Jan 23, 2008 6:34 AM, Michael McCandless <[EMAIL PROTECTED]> wrote: writer.freezeDocIDs(); try { get docIDs from somewhere & call writer.deleteByDocID } finally { writer.unfreezeDocIDs(); } Interesting idea, but would require the IndexWriter to f

Hudson build is back to normal: Lucene-Nightly #347

2008-01-24 Thread hudson
See http://lucene.zones.apache.org:8080/hudson/job/Lucene-Nightly/347/changes - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]

Re: Unique doc ids

2008-01-24 Thread Yonik Seeley
On Jan 24, 2008 5:47 AM, Michael McCandless <[EMAIL PROTECTED]> wrote: > > Yonik Seeley wrote: > > > On Jan 23, 2008 6:34 AM, Michael McCandless > > <[EMAIL PROTECTED]> wrote: > >>writer.freezeDocIDs(); > >>try { > >> get docIDs from somewhere & call writer.deleteByDocID > >>} fina

Re: Back Compatibility

2008-01-24 Thread Grant Ingersoll
On Jan 24, 2008, at 4:27 AM, Michael McCandless wrote: Grant Ingersoll wrote: Yes, I agree these are what is about (despite the divergence into locking). As I see, it the question is about whether we should try to do major releases on the order of a year, rather than the current 2+ ye

JBoss Cache as a store

2008-01-24 Thread Manik Surtani
Hi guys I've just written a plugin for Lucene to use JBoss Cache as an index store. The benefits of something like this are: 1. Faster access to indexes as they will be in memory 2. Indexes replicated across a cluster of servers 3. Indexes "persisted" in clustered memory - faster that per

[jira] Resolved: (LUCENE-1129) ReadTask ignores traversalSize

2008-01-24 Thread Grant Ingersoll (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Grant Ingersoll resolved LUCENE-1129. - Resolution: Fixed Committed with LUCENE-1128 > ReadTask ignores traversalSize > ---

[jira] Resolved: (LUCENE-1128) Add Highlighting benchmark support to contrib/benchmark

2008-01-24 Thread Grant Ingersoll (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Grant Ingersoll resolved LUCENE-1128. - Resolution: Fixed Lucene Fields: (was: [New]) Committed on rev 614885 > Add Hi

[jira] Resolved: (LUCENE-1127) TokenSources.getTokenStream(Document...)

2008-01-24 Thread Grant Ingersoll (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Grant Ingersoll resolved LUCENE-1127. - Resolution: Fixed Lucene Fields: (was: [New]) > TokenSources.getTokenStream(Doc

Re: 2.3.0 announcement draft

2008-01-24 Thread Grant Ingersoll
+1. On Jan 24, 2008, at 3:55 AM, Michael Busch wrote: Michael McCandless wrote: Looks great Michael! Maybe add: * TermVectorMapper to customize how term vectors are loaded * live backups (without pausing indexing) with SnapshotDeletionPolicy * CheckIndex tool to test & recover a corrup

[jira] Resolved: (LUCENE-1137) Token type as BitSet: typeBits()

2008-01-24 Thread Grant Ingersoll (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Grant Ingersoll resolved LUCENE-1137. - Resolution: Fixed Lucene Fields: (was: [New]) Committed on 614891 > Token type

[jira] Resolved: (LUCENE-1133) WikipediaTokenizer needs a way of not tokenizing certain parts of the text

2008-01-24 Thread Grant Ingersoll (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Grant Ingersoll resolved LUCENE-1133. - Resolution: Fixed Lucene Fields: (was: [New]) Committed on 614895 > WikipediaT

Re: 2.3.0 announcement draft

2008-01-24 Thread Grant Ingersoll
I'll take a plug for contrib/wikipedia, too, if you don't mind: * New contrib module for working with Wikipedia content Thanks, -Grant On Jan 24, 2008, at 9:30 AM, Grant Ingersoll wrote: +1. On Jan 24, 2008, at 3:55 AM, Michael Busch wrote: Michael McCandless wrote: Looks great Michael!

[jira] Created: (LUCENE-1148) Create a new sub-class of SpanQuery to enable use of a RangeQuery within a SpanQuery

2008-01-24 Thread Michael Goddard (JIRA)
Create a new sub-class of SpanQuery to enable use of a RangeQuery within a SpanQuery Key: LUCENE-1148 URL: https://issues.apache.org/jira/browse/LUCENE-1148 Project:

[jira] Updated: (LUCENE-1148) Create a new sub-class of SpanQuery to enable use of a RangeQuery within a SpanQuery

2008-01-24 Thread Michael Goddard (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Goddard updated LUCENE-1148: Attachment: span_range_query_01.24.2008.patch This is the initial patch referenced in the

[jira] Commented: (LUCENE-794) Extend contrib Highlighter to properly support phrase queries and span queries

2008-01-24 Thread Michael Goddard (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12562103#action_12562103 ] Michael Goddard commented on LUCENE-794: Mark, Thanks for looking at that. I just

ConjuctionScorer bug?

2008-01-24 Thread eks dev
this code snippet in ConjuctionScorer does not do what the comment describes, or I have problems understanding it? // If first-time skip distance is any predictor of // scorer sparseness, then we should always try to skip first on // those scorers. // Keep last scorer in it's last

Re: Unique doc ids

2008-01-24 Thread Michael McCandless
Yonik Seeley wrote: On Jan 24, 2008 5:47 AM, Michael McCandless <[EMAIL PROTECTED]> wrote: Yonik Seeley wrote: On Jan 23, 2008 6:34 AM, Michael McCandless <[EMAIL PROTECTED]> wrote: writer.freezeDocIDs(); try { get docIDs from somewhere & call writer.deleteByDocID } finally

Re: Back Compatibility

2008-01-24 Thread robert engels
Sorry, I am using "gets lock" to mean 'opening the index'. I was simplifying the the procedure. I think your comment is not correct in this context. On Jan 24, 2008, at 3:16 AM, Michael McCandless wrote: Doron Cohen wrote: --=_Part_11325_2615585.1201162438596 Content-Type: text/plain;

Re: Back Compatibility

2008-01-24 Thread robert engels
Thanks, you are correct, but I am not sure it covers the complete case. Change it a bit to be: A opens reader. B opens reader. A performs query decides a new document is needed B performs query decides a new document is needed B gets writer, adds document, closes A gets writer, adds document, cl

Re: [ANNOUNCE] New Build Server

2008-01-24 Thread Nigel Daley
Quick temporary solution would be to put junit jar in your home account on the hudson build machine and then have the nightly script use -lib ~gsingers/path/to/junit.jar On Jan 23, 2008, at 5:53 PM, Grant Ingersoll wrote: We need to fix the JUnit thing first. I am just not sure how to bes

Re: Back Compatibility

2008-01-24 Thread Doron Cohen
On Jan 24, 2008 6:55 PM, robert engels <[EMAIL PROTECTED]> wrote: > Thanks, you are correct, but I am not sure it covers the complete case. > > Change it a bit to be: > > A opens reader. > B opens reader. > A performs query decides a new document is needed > B performs query decides a new document

[jira] Updated: (LUCENE-1136) add ability to not count sub-task doLogic increment to contri/benchmark

2008-01-24 Thread Doron Cohen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Doron Cohen updated LUCENE-1136: Attachment: lucene-1136.patch updated patch for reference (integrated with recent changes). > add

[jira] Resolved: (LUCENE-1136) add ability to not count sub-task doLogic increment to contri/benchmark

2008-01-24 Thread Doron Cohen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Doron Cohen resolved LUCENE-1136. - Resolution: Fixed Lucene Fields: [Patch Available] (was: [New]) Committed. > add abilit

Re: Back Compatibility

2008-01-24 Thread DM Smith
This is now a hijacked thread. It is very interesting, but it may be hard to find again. Wouldn't it be better to record this thread differently, perhaps opening a Jira issue to add XA to Lucene? -- DM Doron Cohen wrote: On Jan 24, 2008 6:55 PM, robert engels <[EMAIL PROTECTED]> wrote: T

Re: Back Compatibility

2008-01-24 Thread robert engels
I will do so. On Jan 24, 2008, at 12:44 PM, DM Smith wrote: This is now a hijacked thread. It is very interesting, but it may be hard to find again. Wouldn't it be better to record this thread differently, perhaps opening a Jira issue to add XA to Lucene? -- DM Doron Cohen wrote: On Jan

[jira] Created: (LUCENE-1149) add XA transaction support

2008-01-24 Thread robert engels (JIRA)
add XA transaction support -- Key: LUCENE-1149 URL: https://issues.apache.org/jira/browse/LUCENE-1149 Project: Lucene - Java Issue Type: New Feature Components: Index Reporter: robert engels Ne

Re: [parabuild] BUILD for lucene-daily (#170) on localhost was BROKEN: Script returned non-zero code "1"

2008-01-24 Thread Doron Cohen
Failure is this: [junit] Testsuite: org.apache.lucene.index.TestIndexWriter [junit] Tests run: 56, Failures: 0, Errors: 1, Time elapsed: 177.901 sec [junit] Test org.apache.lucene.index.TestIndexWriter FAILED [junit] Testcase: testIOExceptionDuringCloseDocStoreWithThreads(org.apac

Re: [parabuild] BUILD for lucene-daily (#170) on localhost was BROKEN: Script returned non-zero code "1"

2008-01-24 Thread Slava Imeshev
Maybe the build accumulates state. I have set it to do a clean checkout every time the build runs. Hope this helps. Slava --- Doron Cohen <[EMAIL PROTECTED]> wrote: > Failure is this: > > [junit] Testsuite: org.apache.lucene.index.TestIndexWriter > [junit] Tests run: 56, Failures: 0,

[jira] Updated: (LUCENE-794) Extend contrib Highlighter to properly support phrase queries and span queries

2008-01-24 Thread Michael Goddard (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Goddard updated LUCENE-794: --- Attachment: spanhighlighter_24_January_2008.patch Relocated the fir.close() to after the extr

Re: [parabuild] BUILD for lucene-daily (#170) on localhost was BROKEN: Script returned non-zero code "1"

2008-01-24 Thread Michael McCandless
I also saw this earlier from parabuild and have tried hard to repro but cannot (so far!). Does anyone know the details of the Parabuild build (OS, JVM, heap settings, etc)? I don't think past state would cause this. That test is using MockRAMDirectory which is entirely in RAM. This i

Re: [parabuild] BUILD for lucene-daily (#170) on localhost was BROKEN: Script returned non-zero code "1"

2008-01-24 Thread Slava Imeshev
Michael, Here are the details of the environment: Operating system name: Linux Operating system architecture: i386 Operating system version: 2.4.22-10mdkenterprise Java Runtime Environment version: 1.4.2_13 Java Runtime Environment vendor:Sun Microsystems Inc. Slava --- Mi

formatable changes log

2008-01-24 Thread Doron Cohen
As it is becoming hard to browse/navigate CHANGES.txt, how about maintaining it in a simple HTML file? Requirements are: - fancier formatting where adequate. - collapse/expand by release/subject - easy to maintain... Here is an example, containing the current (new) trunk and 2.3.0 - http://people

Re: formatable changes log

2008-01-24 Thread Michael Busch
Forrest has a plugin called projectInfo that can generate a list of changes and an RSS feed from a status.xml file: http://forrest.apache.org/pluginDocs/plugins_0_80/org.apache.forrest.plugin.input.projectInfo/index.html Maybe we should use that? -Michael Doron Cohen wrote: > As it is becoming

[ANNOUNCE] Lucene Java 2.3.0 release available

2008-01-24 Thread Michael Busch
Release 2.3.0 of Lucene Java is now available! Many new features, optimizations, and bug fixes have been added since 2.2, including: * significantly improved indexing performance * segment merging in background threads * refreshable IndexReaders * faster StandardAnalyzer and improved Toke

Re: 2.3.0 announcement draft

2008-01-24 Thread Michael Busch
Hi Team, I added your item to the list, Grant, and announced the release officially on java-user, java-dev, announce, and freshmeat. I also added a news item to the website. Thanks everyone for a smooth release! Happy Peanut Butter Day :) -Michael Grant Ingersoll wrote: > I'll take a plug for