[jira] Commented: (LUCENE-973) Token of "" returns in CJK

2008-07-30 Thread Steven Rowe (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12618540#action_12618540 ] Steven Rowe commented on LUCENE-973: Sorry Toru, I saw Koji's two most recent comments

[jira] Commented: (LUCENE-973) Token of "" returns in CJK

2008-07-30 Thread Koji Sekiguchi (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12618534#action_12618534 ] Koji Sekiguchi commented on LUCENE-973: --- Hi Steven, bq. The test class in your patch

Re: [jira] Commented: (LUCENE-1345) Allow Filter as clause to BooleanQuery

2008-07-30 Thread eks dev
than we conclude, comparison with 0 is faster :) Maybe something on my XP machine was doing something in background I have not noticed, stealing cycles, on Windows this can not be easily controlled. or when I tested it the other day, I used comparison with -1 while((doc=it.next()) >-1) could

[jira] Commented: (LUCENE-1345) Allow Filter as clause to BooleanQuery

2008-07-30 Thread Yonik Seeley (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12618513#action_12618513 ] Yonik Seeley commented on LUCENE-1345: -- Eks, I just tried your first TestIteratorPerf

Re: [jira] Updated: (LUCENE-1345) Allow Filter as clause to BooleanQuery

2008-07-30 Thread Yonik Seeley
On Wed, Jul 30, 2008 at 3:06 PM, Michael McCandless <[EMAIL PROTECTED]> wrote: > > Neat! > > Do you know how to get the corresponding asm that the hostpot compiler > produces? This way we can see if this difference "survives" through java... Thanks to the tool that Stephen pointed out, I can now

Re: [jira] Updated: (LUCENE-1345) Allow Filter as clause to BooleanQuery

2008-07-30 Thread Yonik Seeley
On Wed, Jul 30, 2008 at 3:17 PM, Stephen Green <[EMAIL PROTECTED]> wrote: > Might the description here: > > http://weblogs.java.net/blog/kohsuke/archive/2008/03/deep_dive_into.html > > help? Sweet! Thanks! -Yonik - To unsubscri

Re: [jira] Updated: (LUCENE-1345) Allow Filter as clause to BooleanQuery

2008-07-30 Thread Stephen Green
On Jul 30, 2008, at 3:12 PM, Yonik Seeley wrote: On Wed, Jul 30, 2008 at 3:06 PM, Michael McCandless <[EMAIL PROTECTED]> wrote: Neat! Do you know how to get the corresponding asm that the hostpot compiler produces? This way we can see if this difference "survives" through java... Unfor

Re: [jira] Updated: (LUCENE-1345) Allow Filter as clause to BooleanQuery

2008-07-30 Thread Yonik Seeley
On Wed, Jul 30, 2008 at 3:06 PM, Michael McCandless <[EMAIL PROTECTED]> wrote: > Neat! > > Do you know how to get the corresponding asm that the hostpot compiler > produces? This way we can see if this difference "survives" through java... Unfortunately, no. I've looked in the past and couldn't

Re: [jira] Updated: (LUCENE-1345) Allow Filter as clause to BooleanQuery

2008-07-30 Thread Michael McCandless
Neat! Do you know how to get the corresponding asm that the hostpot compiler produces? This way we can see if this difference "survives" through java... Mike Yonik Seeley wrote: disclaimer: this is just for fun differences should be in the noise in any complex system, and I'm not s

Re: Deadlock when multi-threading DocumentsWriter

2008-07-30 Thread Michael McCandless
Could you also post your mods to DocumentsWriter? Eg "doGetThreadState" and "finishDocWithThreadState"? Or, better yet, post a full patch with a unit test showing the hang? I think it should be OK to have one thread get a ThreadState and another thread finish indexing the doc with that th

Re: [jira] Updated: (LUCENE-1345) Allow Filter as clause to BooleanQuery

2008-07-30 Thread Yonik Seeley
disclaimer: this is just for fun differences should be in the noise in any complex system, and I'm not suggesting any code changes. Actually, with 32 bit registers, x<0 should be faster than x==-1 by one cycle. If it doesn't test faster, then it's because of some optimizations that could be p

[jira] Updated: (LUCENE-973) Token of "" returns in CJK

2008-07-30 Thread Steven Rowe (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steven Rowe updated LUCENE-973: --- Hi Koji, The test class in your patch is a nice addition. bq. There is no problem in CJKAnalyzer. The

[jira] Resolved: (LUCENE-1282) Sun hotspot compiler bug in 1.6.0_04/05 affects Lucene

2008-07-30 Thread Michael McCandless (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael McCandless resolved LUCENE-1282. Resolution: Fixed > Sun hotspot compiler bug in 1.6.0_04/05 affects Lucene > -

[jira] Commented: (LUCENE-1282) Sun hotspot compiler bug in 1.6.0_04/05 affects Lucene

2008-07-30 Thread Michael McCandless (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12618446#action_12618446 ] Michael McCandless commented on LUCENE-1282: Indeed, I can confirm that JDK 6

[jira] Updated: (LUCENE-973) Token of "" returns in CJK

2008-07-30 Thread Koji Sekiguchi (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Sekiguchi updated LUCENE-973: -- Attachment: with-patch.jpg without-patch.jpg I attached Solr analysis screen to

[jira] Updated: (LUCENE-1347) IndexWriter.rollback can hang if a previous call hit an exception

2008-07-30 Thread Michael McCandless (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1347?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael McCandless updated LUCENE-1347: --- Fix Version/s: (was: 2.3.2) 2.3.3 > IndexWriter.rollback can

RE: Deadlock when multi-threading DocumentsWriter

2008-07-30 Thread Jagadesh Nomula
Thanks again for the comments. I am trying to split the IndexWriter.addDocuent into 2 different methods getThreadState() and finishDoc(), the idea is to make them multithreaded for parallelWriter. I am running into this situation, wherein DocumentsWriter.getThreadState and DocumentsWriter.p

[jira] Resolved: (LUCENE-1347) IndexWriter.rollback can hang if a previous call hit an exception

2008-07-30 Thread Michael McCandless (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1347?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael McCandless resolved LUCENE-1347. Resolution: Fixed > IndexWriter.rollback can hang if a previous call hit an except

[jira] Commented: (LUCENE-1347) IndexWriter.rollback can hang if a previous call hit an exception

2008-07-30 Thread Michael McCandless (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1347?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12618392#action_12618392 ] Michael McCandless commented on LUCENE-1347: Super, I'll commit shortly -- tha

[jira] Commented: (LUCENE-1347) IndexWriter.rollback can hang if a previous call hit an exception

2008-07-30 Thread Yonik Seeley (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1347?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12618390#action_12618390 ] Yonik Seeley commented on LUCENE-1347: -- Yep... the success flag ensures that closing=

[jira] Updated: (LUCENE-1333) Token implementation needs improvements

2008-07-30 Thread Michael McCandless (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1333?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael McCandless updated LUCENE-1333: --- Attachment: LUCENE-1333.patch This patch looks good; thanks DM! I made a few small

Re: [VOTE] Break Back Compatibility "Contract" on Fieldable

2008-07-30 Thread Grant Ingersoll
On Jul 30, 2008, at 11:07 AM, DM Smith wrote: I'm not sure that the comment that "this gives anyone with a pulse enough time to react" is particularly accurate or helpful. It all depends upon effective communication (such as to Lucene user's mailing list and package maintainers). It was

Re: [VOTE] Break Back Compatibility "Contract" on Fieldable

2008-07-30 Thread DM Smith
As a user, I have no problem with this, as I have a pulse:) If I understand that it just breaks software backward compatibility and not compatibility with the index itself. Minor software changes are no big deal to me. I would still expect that a newer API would still read earlier indexes. Spe

[jira] Updated: (LUCENE-1347) IndexWriter.rollback can hang if a previous call hit an exception

2008-07-30 Thread Michael McCandless (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1347?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael McCandless updated LUCENE-1347: --- Attachment: LUCENE-1347.patch Woops, you're right -- thanks for the review! Two thr

[jira] Commented: (LUCENE-1347) IndexWriter.rollback can hang if a previous call hit an exception

2008-07-30 Thread Yonik Seeley (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1347?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12618356#action_12618356 ] Yonik Seeley commented on LUCENE-1347: -- Looks good Mike! Just one question it loo

Re: [VOTE] Break Back Compatibility "Contract" on Fieldable

2008-07-30 Thread Michael Busch
+1 to all three items. This is good stuff. -Michael Grant Ingersoll wrote: As they say, rules are meant to be broken... For a variety of reasons, some outlined below, I (and others) would like us to break our back compatibility requirements and allow for modifying the Fieldable interface in

Re: [VOTE] Break Back Compatibility "Contract" on Fieldable

2008-07-30 Thread Erik Hatcher
+1 to all three from me. Darn you, Java, for making object- orientation kludgey. Erik On Jul 30, 2008, at 8:44 AM, Grant Ingersoll wrote: As they say, rules are meant to be broken... For a variety of reasons, some outlined below, I (and others) would like us to break our back comp

Re: [VOTE] Break Back Compatibility "Contract" on Fieldable

2008-07-30 Thread Michael McCandless
+1 to all three. Mike Grant Ingersoll wrote: As they say, rules are meant to be broken... For a variety of reasons, some outlined below, I (and others) would like us to break our back compatibility requirements and allow for modifying the Fieldable interface in 2.x releases with the 3.x

[jira] Updated: (LUCENE-1347) IndexWriter.rollback can hang if a previous call hit an exception

2008-07-30 Thread Michael McCandless (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1347?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael McCandless updated LUCENE-1347: --- Attachment: LUCENE-1347.patch Attached patch. I plan to commit in a day or so, and

[jira] Created: (LUCENE-1347) IndexWriter.rollback can hang if a previous call hit an exception

2008-07-30 Thread Michael McCandless (JIRA)
IndexWriter.rollback can hang if a previous call hit an exception - Key: LUCENE-1347 URL: https://issues.apache.org/jira/browse/LUCENE-1347 Project: Lucene - Java Issue Type: Bu

[VOTE] Break Back Compatibility "Contract" on Fieldable

2008-07-30 Thread Grant Ingersoll
As they say, rules are meant to be broken... For a variety of reasons, some outlined below, I (and others) would like us to break our back compatibility requirements and allow for modifying the Fieldable interface in 2.x releases with the 3.x plan to be to separate out write side interfaces

[jira] Commented: (LUCENE-1340) Make it posible not to include TF information in index

2008-07-30 Thread Grant Ingersoll (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12618290#action_12618290 ] Grant Ingersoll commented on LUCENE-1340: - OK, I think we should call a vote on it

[jira] Commented: (LUCENE-1340) Make it posible not to include TF information in index

2008-07-30 Thread Michael McCandless (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12618272#action_12618272 ] Michael McCandless commented on LUCENE-1340: Sigh, I too missed that we broke

Re: Sort suggestion

2008-07-30 Thread Marcus Herou
Yep a disk sort is slow as hell compared to mem sort. What I was thinking was something like a db thinks. MySQL for example does exactly this. If the resultset do not fit properly in mem spool it on disk and sort it. The thing is that it would allow you to continue adding docs to the index even t

Re: Deadlock when multi-threading DocumentsWriter

2008-07-30 Thread Michael McCandless
The lock acquire order for all call stacks that lock on these two classes should be IndexWriter then DocumentsWriter, as is the case with IndexWriter.doFlush calling DocumentsWriter.pauseAllThreads. So you shouldn't hit a thread deadlock. Also, doFlush is called when it's time to write a