[jira] Commented: (LUCENE-2324) Per thread DocumentsWriters that write their own private segments

2010-04-15 Thread Jason Rutherglen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12857380#action_12857380 ] Jason Rutherglen commented on LUCENE-2324: -- bq. only one DW flushes at a time

[jira] Commented: (LUCENE-2324) Per thread DocumentsWriters that write their own private segments

2010-04-14 Thread Jason Rutherglen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12857112#action_12857112 ] Jason Rutherglen commented on LUCENE-2324: -- Michael, nice! I guess I should've

[jira] Commented: (LUCENE-2324) Per thread DocumentsWriters that write their own private segments

2010-04-05 Thread Jason Rutherglen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12853594#action_12853594 ] Jason Rutherglen commented on LUCENE-2324: -- Michael B, I was going to wait

Re: Query modifier

2010-03-30 Thread Jason Rutherglen
David, I totally agree with this idea. On Tue, Mar 30, 2010 at 9:58 AM, David Smiley (@MITRE.org) dsmi...@mitre.org wrote: I observed this problem when I started using Lucene (ages ago) and it's a shame this situation persists.  In summary, it would be tremendously useful if Query objects

[jira] Commented: (LUCENE-2324) Per thread DocumentsWriters that write their own private segments

2010-03-29 Thread Jason Rutherglen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12851017#action_12851017 ] Jason Rutherglen commented on LUCENE-2324: -- Michael B.: What you're talking about

[jira] Commented: (LUCENE-2324) Per thread DocumentsWriters that write their own private segments

2010-03-29 Thread Jason Rutherglen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12851099#action_12851099 ] Jason Rutherglen commented on LUCENE-2324: -- {quote}You only need one additional

[jira] Commented: (LUCENE-2324) Per thread DocumentsWriters that write their own private segments

2010-03-28 Thread Jason Rutherglen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12850754#action_12850754 ] Jason Rutherglen commented on LUCENE-2324: -- {quote}I don't think we should delete

[jira] Commented: (LUCENE-2324) Per thread DocumentsWriters that write their own private segments

2010-03-28 Thread Jason Rutherglen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12850780#action_12850780 ] Jason Rutherglen commented on LUCENE-2324: -- {quote}I mentioned the idea of having

[jira] Commented: (LUCENE-2324) Per thread DocumentsWriters that write their own private segments

2010-03-26 Thread Jason Rutherglen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12850211#action_12850211 ] Jason Rutherglen commented on LUCENE-2324: -- Mike, lets do option 1. I think

[jira] Commented: (LUCENE-2324) Per thread DocumentsWriters that write their own private segments

2010-03-26 Thread Jason Rutherglen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12850223#action_12850223 ] Jason Rutherglen commented on LUCENE-2324: -- Currently the doc writer manages

[jira] Commented: (LUCENE-2324) Per thread DocumentsWriters that write their own private segments

2010-03-26 Thread Jason Rutherglen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12850227#action_12850227 ] Jason Rutherglen commented on LUCENE-2324: -- Following up on the previous comment

[jira] Commented: (LUCENE-2324) Per thread DocumentsWriters that write their own private segments

2010-03-26 Thread Jason Rutherglen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12850253#action_12850253 ] Jason Rutherglen commented on LUCENE-2324: -- bq. Of course then you'd need two

[jira] Commented: (LUCENE-2324) Per thread DocumentsWriters that write their own private segments

2010-03-26 Thread Jason Rutherglen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12850260#action_12850260 ] Jason Rutherglen commented on LUCENE-2324: -- Also, I think we can remove the sync

[jira] Updated: (LUCENE-2324) Per thread DocumentsWriters that write their own private segments

2010-03-26 Thread Jason Rutherglen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2324?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Rutherglen updated LUCENE-2324: - Attachment: LUCENE-2324.patch The patch is not committable. The basics are here. We

[jira] Commented: (LUCENE-2324) Per thread DocumentsWriters that write their own private segments

2010-03-26 Thread Jason Rutherglen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12850440#action_12850440 ] Jason Rutherglen commented on LUCENE-2324: -- In regards to remap deletes. I looked

[jira] Commented: (LUCENE-2324) Per thread DocumentsWriters that write their own private segments

2010-03-26 Thread Jason Rutherglen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12850449#action_12850449 ] Jason Rutherglen commented on LUCENE-2324: -- Mike, can you describe the difference

[jira] Commented: (LUCENE-2324) Per thread DocumentsWriters that write their own private segments

2010-03-25 Thread Jason Rutherglen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12849806#action_12849806 ] Jason Rutherglen commented on LUCENE-2324: -- Michael, I'm guessing this patch

[jira] Commented: (LUCENE-2324) Per thread DocumentsWriters that write their own private segments

2010-03-25 Thread Jason Rutherglen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12849808#action_12849808 ] Jason Rutherglen commented on LUCENE-2324: -- Actually, I just browsed the patch

[jira] Commented: (LUCENE-2324) Per thread DocumentsWriters that write their own private segments

2010-03-25 Thread Jason Rutherglen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12849844#action_12849844 ] Jason Rutherglen commented on LUCENE-2324: -- Michael, I'm working on a patch

[jira] Commented: (LUCENE-2324) Per thread DocumentsWriters that write their own private segments

2010-03-25 Thread Jason Rutherglen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12849965#action_12849965 ] Jason Rutherglen commented on LUCENE-2324: -- I'm a little confused

[jira] Commented: (LUCENE-2312) Search on IndexWriter's RAM Buffer

2010-03-22 Thread Jason Rutherglen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12848188#action_12848188 ] Jason Rutherglen commented on LUCENE-2312: -- Brian, thanks for the explanation

[jira] Commented: (LUCENE-2312) Search on IndexWriter's RAM Buffer

2010-03-18 Thread Jason Rutherglen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12847123#action_12847123 ] Jason Rutherglen commented on LUCENE-2312: -- For the skip list, we could reuse

[jira] Commented: (LUCENE-2312) Search on IndexWriter's RAM Buffer

2010-03-18 Thread Jason Rutherglen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12847140#action_12847140 ] Jason Rutherglen commented on LUCENE-2312: -- I'm hitting a weird error where after

[jira] Commented: (LUCENE-2127) Improved large result handling

2010-03-18 Thread Jason Rutherglen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12847146#action_12847146 ] Jason Rutherglen commented on LUCENE-2127: -- What's the status of this one? I'm

[jira] Updated: (LUCENE-2312) Search on IndexWriter's RAM Buffer

2010-03-18 Thread Jason Rutherglen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Rutherglen updated LUCENE-2312: - Comment: was deleted (was: I'm hitting a weird error where after executing a ram buf

[jira] Commented: (LUCENE-2312) Search on IndexWriter's RAM Buffer

2010-03-17 Thread Jason Rutherglen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12846526#action_12846526 ] Jason Rutherglen commented on LUCENE-2312: -- Mike, can you clarify why intUptos

[jira] Commented: (LUCENE-2312) Search on IndexWriter's RAM Buffer

2010-03-17 Thread Jason Rutherglen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12846540#action_12846540 ] Jason Rutherglen commented on LUCENE-2312: -- I think the DW index reader needs

[jira] Commented: (LUCENE-2312) Search on IndexWriter's RAM Buffer

2010-03-17 Thread Jason Rutherglen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12846604#action_12846604 ] Jason Rutherglen commented on LUCENE-2312: -- Previously there was a discussion

[jira] Commented: (LUCENE-2312) Search on IndexWriter's RAM Buffer

2010-03-16 Thread Jason Rutherglen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12845729#action_12845729 ] Jason Rutherglen commented on LUCENE-2312: -- {quote}but my goal is it here

[jira] Commented: (LUCENE-2312) Search on IndexWriter's RAM Buffer

2010-03-16 Thread Jason Rutherglen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12845735#action_12845735 ] Jason Rutherglen commented on LUCENE-2312: -- {quote}This makes the reference

[jira] Commented: (LUCENE-2312) Search on IndexWriter's RAM Buffer

2010-03-16 Thread Jason Rutherglen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12845943#action_12845943 ] Jason Rutherglen commented on LUCENE-2312: -- I thought we're moving away from byte

[jira] Commented: (LUCENE-2312) Search on IndexWriter's RAM Buffer

2010-03-16 Thread Jason Rutherglen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12845950#action_12845950 ] Jason Rutherglen commented on LUCENE-2312: -- {quote}The tricky part is to make

[jira] Commented: (LUCENE-2312) Search on IndexWriter's RAM Buffer

2010-03-16 Thread Jason Rutherglen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12845971#action_12845971 ] Jason Rutherglen commented on LUCENE-2312: -- To clarify the above comment, DW's

[jira] Commented: (LUCENE-2324) Per thread DocumentsWriters that write their own private segments

2010-03-16 Thread Jason Rutherglen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12846028#action_12846028 ] Jason Rutherglen commented on LUCENE-2324: -- Carrying over from LUCENE-2312. I'm

[jira] Commented: (LUCENE-2324) Per thread DocumentsWriters that write their own private segments

2010-03-16 Thread Jason Rutherglen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12846037#action_12846037 ] Jason Rutherglen commented on LUCENE-2324: -- Are there going to be issues

[jira] Commented: (LUCENE-2324) Per thread DocumentsWriters that write their own private segments

2010-03-16 Thread Jason Rutherglen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12846102#action_12846102 ] Jason Rutherglen commented on LUCENE-2324: -- Michael, For LUCENE-2312, I think

[jira] Commented: (LUCENE-2324) Per thread DocumentsWriters that write their own private segments

2010-03-16 Thread Jason Rutherglen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12846110#action_12846110 ] Jason Rutherglen commented on LUCENE-2324: -- NormsWriterPerField has a growing

[jira] Commented: (LUCENE-2324) Per thread DocumentsWriters that write their own private segments

2010-03-16 Thread Jason Rutherglen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12846112#action_12846112 ] Jason Rutherglen commented on LUCENE-2324: -- Actually TermsHashField doesn't need

[jira] Commented: (LUCENE-2324) Per thread DocumentsWriters that write their own private segments

2010-03-16 Thread Jason Rutherglen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12846220#action_12846220 ] Jason Rutherglen commented on LUCENE-2324: -- Michael, Agreed, can you outline how

[jira] Commented: (LUCENE-2312) Search on IndexWriter's RAM Buffer

2010-03-16 Thread Jason Rutherglen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12846261#action_12846261 ] Jason Rutherglen commented on LUCENE-2312: -- I think the easiest way to test out

[jira] Commented: (LUCENE-2312) Search on IndexWriter's RAM Buffer

2010-03-15 Thread Jason Rutherglen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12845374#action_12845374 ] Jason Rutherglen commented on LUCENE-2312: -- {quote}Good question on skipping

[jira] Commented: (LUCENE-2312) Search on IndexWriter's RAM Buffer

2010-03-15 Thread Jason Rutherglen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12845404#action_12845404 ] Jason Rutherglen commented on LUCENE-2312: -- Pre-advanced apology for permanently

[jira] Commented: (LUCENE-2312) Search on IndexWriter's RAM Buffer

2010-03-15 Thread Jason Rutherglen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12845448#action_12845448 ] Jason Rutherglen commented on LUCENE-2312: -- The code is from

[jira] Commented: (LUCENE-2312) Search on IndexWriter's RAM Buffer

2010-03-15 Thread Jason Rutherglen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12845493#action_12845493 ] Jason Rutherglen commented on LUCENE-2312: -- {quote}Ahh, I think it's because

[jira] Commented: (LUCENE-2312) Search on IndexWriter's RAM Buffer

2010-03-15 Thread Jason Rutherglen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12845503#action_12845503 ] Jason Rutherglen commented on LUCENE-2312: -- Also wanted to add

[jira] Commented: (LUCENE-2312) Search on IndexWriter's RAM Buffer

2010-03-15 Thread Jason Rutherglen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12845663#action_12845663 ] Jason Rutherglen commented on LUCENE-2312: -- I have a test case showing the term

[jira] Commented: (LUCENE-2312) Search on IndexWriter's RAM Buffer

2010-03-15 Thread Jason Rutherglen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12845679#action_12845679 ] Jason Rutherglen commented on LUCENE-2312: -- Basic term positions working, need

[jira] Commented: (LUCENE-2312) Search on IndexWriter's RAM Buffer

2010-03-15 Thread Jason Rutherglen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12845680#action_12845680 ] Jason Rutherglen commented on LUCENE-2312: -- In thinking about the terms

[jira] Commented: (LUCENE-2312) Search on IndexWriter's RAM Buffer

2010-03-15 Thread Jason Rutherglen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12845696#action_12845696 ] Jason Rutherglen commented on LUCENE-2312: -- Payloads works (non-lazy loading

[jira] Commented: (LUCENE-2312) Search on IndexWriter's RAM Buffer

2010-03-15 Thread Jason Rutherglen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12845721#action_12845721 ] Jason Rutherglen commented on LUCENE-2312: -- Just to clarify, I think Mike's

[jira] Commented: (LUCENE-2312) Search on IndexWriter's RAM Buffer

2010-03-14 Thread Jason Rutherglen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12845149#action_12845149 ] Jason Rutherglen commented on LUCENE-2312: -- Mike, rollback is pausing all threads

[jira] Commented: (LUCENE-2312) Search on IndexWriter's RAM Buffer

2010-03-14 Thread Jason Rutherglen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12845179#action_12845179 ] Jason Rutherglen commented on LUCENE-2312: -- I got the basics of the term enum

[jira] Commented: (LUCENE-2312) Search on IndexWriter's RAM Buffer

2010-03-13 Thread Jason Rutherglen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12844891#action_12844891 ] Jason Rutherglen commented on LUCENE-2312: -- From LUCENE-2293: {quote}(b-tree

[jira] Commented: (LUCENE-2293) IndexWriter has hard limit on max concurrency

2010-03-13 Thread Jason Rutherglen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12844893#action_12844893 ] Jason Rutherglen commented on LUCENE-2293: -- {quote}Change the approach for how we

[jira] Commented: (LUCENE-2293) IndexWriter has hard limit on max concurrency

2010-03-13 Thread Jason Rutherglen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12844926#action_12844926 ] Jason Rutherglen commented on LUCENE-2293: -- bq. Probably one DW instance per

[jira] Commented: (LUCENE-2312) Search on IndexWriter's RAM Buffer

2010-03-13 Thread Jason Rutherglen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12845030#action_12845030 ] Jason Rutherglen commented on LUCENE-2312: -- Mike, Why does DocFieldConsumers have

[jira] Commented: (LUCENE-2312) Search on IndexWriter's RAM Buffer

2010-03-13 Thread Jason Rutherglen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12845036#action_12845036 ] Jason Rutherglen commented on LUCENE-2312: -- A few notes so far: * IW flush could

[jira] Commented: (LUCENE-2312) Search on IndexWriter's RAM Buffer

2010-03-13 Thread Jason Rutherglen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12845041#action_12845041 ] Jason Rutherglen commented on LUCENE-2312: -- * IW commitMerge calls docWriter's

[jira] Created: (LUCENE-2312) Search on IndexWriter's RAM Buffer

2010-03-12 Thread Jason Rutherglen (JIRA)
: 3.0.1 Reporter: Jason Rutherglen Fix For: 3.0.2 In order to offer user's near realtime search, without incurring an indexing performance penalty, we can implement search on IndexWriter's RAM buffer. This is the buffer that is filled in RAM as documents are indexed

[jira] Commented: (LUCENE-2312) Search on IndexWriter's RAM Buffer

2010-03-12 Thread Jason Rutherglen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12844749#action_12844749 ] Jason Rutherglen commented on LUCENE-2312: -- In regards to the terms dictionary

[jira] Commented: (LUCENE-2312) Search on IndexWriter's RAM Buffer

2010-03-12 Thread Jason Rutherglen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12844826#action_12844826 ] Jason Rutherglen commented on LUCENE-2312: -- I set out implementing a simple

[jira] Commented: (LUCENE-2293) IndexWriter has hard limit on max concurrency

2010-03-12 Thread Jason Rutherglen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12844828#action_12844828 ] Jason Rutherglen commented on LUCENE-2293: -- {quote}but does anyone out

[jira] Commented: (LUCENE-1709) Parallelize Tests

2010-03-09 Thread Jason Rutherglen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12843385#action_12843385 ] Jason Rutherglen commented on LUCENE-1709: -- Robert, very nice! Parallelize

[jira] Commented: (LUCENE-2293) IndexWriter has hard limit on max concurrency

2010-03-05 Thread Jason Rutherglen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12841960#action_12841960 ] Jason Rutherglen commented on LUCENE-2293: -- {quote}I think we have two options

[jira] Commented: (LUCENE-2293) IndexWriter has hard limit on max concurrency

2010-03-04 Thread Jason Rutherglen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12841342#action_12841342 ] Jason Rutherglen commented on LUCENE-2293: -- bq. But WaitQueue would go away

[jira] Commented: (LUCENE-2293) IndexWriter has hard limit on max concurrency

2010-03-03 Thread Jason Rutherglen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12840918#action_12840918 ] Jason Rutherglen commented on LUCENE-2293: -- Mike, good one! Would having a doc

Re: Uwe's question

2010-02-26 Thread Jason Rutherglen
Lets go to JUnit 4 if possible... Does it provide method level testing? (i.e. one doesn't need to execute every test method just to check the results of one method) On Thu, Feb 25, 2010 at 8:15 PM, Shai Erera ser...@gmail.com wrote: Ok this seems a discussion related to JUnit 4, so I'll port

Re: Uwe's question

2010-02-26 Thread Jason Rutherglen
probably a personal preference. I've never tried to learn a command-line invocation of a test case for a single test method, I've always just used the IDE to run individual methods Erick On Fri, Feb 26, 2010 at 11:31 AM, Jason Rutherglen jason.rutherg...@gmail.com wrote: Lets go

[jira] Commented: (LUCENE-1720) TimeLimitedIndexReader and associated utility class

2010-02-11 Thread Jason Rutherglen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12832638#action_12832638 ] Jason Rutherglen commented on LUCENE-1720: -- When's this ready to test with Solr

[jira] Commented: (LUCENE-2255) IndexWriter.getReader() allocates file handles

2010-02-08 Thread Jason Rutherglen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12831095#action_12831095 ] Jason Rutherglen commented on LUCENE-2255: -- You're not the first to report

[jira] Commented: (LUCENE-2255) IndexWriter.getReader() allocates file handles

2010-02-08 Thread Jason Rutherglen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12831116#action_12831116 ] Jason Rutherglen commented on LUCENE-2255: -- bq. What keeps you from attaching

Re: Nasty NIO behavior makes NIOFSDirectory silently close channel

2010-01-29 Thread Jason Rutherglen
Defaulting NIOFSDir could account for some of the recent speed improvements users have been reporting in Lucene 2.9. So removing it as a default could reverse those and people could then report Lucene 3.X has slowed... On Thu, Jan 28, 2010 at 5:24 AM, Michael McCandless luc...@mikemccandless.com

Re: NRT and IndexSearcher performance

2010-01-19 Thread Jason Rutherglen
J, The javadocs are illustrating there's no need to create new IndexSearchers for each query. Jason On Tue, Jan 19, 2010 at 5:04 PM, jchang jchangkihat...@gmail.com wrote: The javadocs for IndexSearcher in Lucene 3.0.0 read:  For performance reasons it is recommended to open only one

Re: Compound File Default

2010-01-12 Thread Jason Rutherglen
I always turn CFS off because it's extra work (no payoff), how's it possible to run into an out of fd limit with a merge factor of 10? On Tue, Jan 12, 2010 at 2:35 PM, Mark Miller markrmil...@gmail.com wrote: Otis Gospodnetic wrote: At the same time, seeing how some people benchmark systems

Re: Lucene 2.9.0 Near Real Time Indexing and Service Crashes/restarts

2010-01-12 Thread Jason Rutherglen
Greetin's John, 2.9 and 3.0 don't use a RAMDir... Deletes are held in RAM however so on power off, those would be lost. Jason On Tue, Jan 12, 2010 at 3:10 PM, jchang jchangkihat...@gmail.com wrote: Lucene 2.9.0 has near real time indexing, writing to a RAMDir which gets flushed to disk when

Re: Lucene 2.9.0 Near Real Time Indexing and lock timeouts

2010-01-12 Thread Jason Rutherglen
And generally, if anybody has any advice on high-throughput indexing with Lucene and what kind of numbers I can acheive, I'd welcome the feedback. I believe it's directly related to how often IW.getReader is called. The longer the duration between calls, larger the resultant new segment is

Re: Lucene 2.9.0 Near Real Time Indexing and Service Crashes/restarts

2010-01-12 Thread Jason Rutherglen
Zoie keeps track of an index version on disk alongside the Lucene index which it uses to decide where it must reindex from to catch up if it there have been incoming indexing events while the server was out of commission. This begs a little more clarity... Sounds like a transaction log. Oh

Re: Lucene 2.9.0 Near Real Time Indexing and Service Crashes/restarts

2010-01-12 Thread Jason Rutherglen
Actually, unless IW.commit is called, all changes after the last commit will be lost (because the segment infos file will not have been written). On Tue, Jan 12, 2010 at 3:37 PM, Jason Rutherglen jason.rutherg...@gmail.com wrote: Greetin's John, 2.9 and 3.0 don't use a RAMDir... Deletes

Re: Lucene 2.9.0 Near Real Time Indexing and Service Crashes/restarts

2010-01-12 Thread Jason Rutherglen
Mannix jake.man...@gmail.com wrote: On Tue, Jan 12, 2010 at 8:55 PM, Jason Rutherglen jason.rutherg...@gmail.com wrote: Zoie keeps track of an index version on disk alongside the Lucene index which it uses to decide where it must reindex from to catch up if it there have been incoming

Re: Compound File Default

2010-01-11 Thread Jason Rutherglen
Maybe the default can be conditional on the platform like NIOFSDirectory. On Mon, Jan 11, 2010 at 1:25 PM, Marvin Humphrey mar...@rectangular.com wrote: On Mon, Jan 11, 2010 at 03:20:17PM -0500, Grant Ingersoll wrote: Should we really still be defaulting to true for setUseCompoundFile?  Do

[jira] Commented: (LUCENE-2127) Improved large result handling

2010-01-07 Thread Jason Rutherglen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12797679#action_12797679 ] Jason Rutherglen commented on LUCENE-2127: -- bq. jira table output. Mike's Python

Re: IndexWriter.getReader() for existing index content

2010-01-07 Thread Jason Rutherglen
An, You're trying to add indexes and get a reader in a single call to IW? Jason On Thu, Jan 7, 2010 at 10:58 AM, An Hong an.h...@i365.com wrote: I’ve a requirement that when creating an IndexWriter on an existing index, I need its getReader() to see the existing content.  I’ve modified

[jira] Commented: (LUCENE-2127) Improved large result handling

2010-01-06 Thread Jason Rutherglen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12797490#action_12797490 ] Jason Rutherglen commented on LUCENE-2127: -- The benchmark code has jira table

[jira] Commented: (LUCENE-2127) Improved large result handling

2010-01-06 Thread Jason Rutherglen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12797499#action_12797499 ] Jason Rutherglen commented on LUCENE-2127: -- There's a bunch of formatting changes

[jira] Commented: (LUCENE-1313) Near Realtime Search (using a built in RAMDirectory)

2010-01-05 Thread Jason Rutherglen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12796705#action_12796705 ] Jason Rutherglen commented on LUCENE-1313: -- the patch version seems to be about

[jira] Commented: (LUCENE-1313) Near Realtime Search (using a built in RAMDirectory)

2010-01-05 Thread Jason Rutherglen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12796763#action_12796763 ] Jason Rutherglen commented on LUCENE-1313: -- {quote}Lucene 3.0 is able to process

[jira] Commented: (LUCENE-2127) Improved large result handling

2010-01-05 Thread Jason Rutherglen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12796823#action_12796823 ] Jason Rutherglen commented on LUCENE-2127: -- Grant, This issue could be useful

[jira] Commented: (LUCENE-2026) Refactoring of IndexWriter

2009-12-18 Thread Jason Rutherglen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12792629#action_12792629 ] Jason Rutherglen commented on LUCENE-2026: -- {quote}Anecdotally, at Eventful one

[jira] Commented: (LUCENE-2164) Make CMS smarter about thread priorities

2009-12-15 Thread Jason Rutherglen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12790827#action_12790827 ] Jason Rutherglen commented on LUCENE-2164: -- This issue illustrates why the ram

[jira] Commented: (LUCENE-2164) Make CMS smarter about thread priorities

2009-12-15 Thread Jason Rutherglen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12790986#action_12790986 ] Jason Rutherglen commented on LUCENE-2164: -- {quote}I've been wondering whether we

[jira] Commented: (LUCENE-2164) Make CMS smarter about thread priorities

2009-12-15 Thread Jason Rutherglen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12791125#action_12791125 ] Jason Rutherglen commented on LUCENE-2164: -- {quote}max(1, min(3, numberOfCores

[jira] Commented: (LUCENE-1785) Simple FieldCache merging

2009-12-14 Thread Jason Rutherglen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12790284#action_12790284 ] Jason Rutherglen commented on LUCENE-1785: -- bq. Can't we just merge-sort

[jira] Commented: (LUCENE-1785) Simple FieldCache merging

2009-12-14 Thread Jason Rutherglen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12790391#action_12790391 ] Jason Rutherglen commented on LUCENE-1785: -- bq. couldn't we pull a Term(s)Enum

[jira] Commented: (LUCENE-2026) Refactoring of IndexWriter

2009-12-13 Thread Jason Rutherglen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12789971#action_12789971 ] Jason Rutherglen commented on LUCENE-2026: -- I think large scale NRT installations

[jira] Commented: (LUCENE-1785) Simple FieldCache merging

2009-12-13 Thread Jason Rutherglen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12789985#action_12789985 ] Jason Rutherglen commented on LUCENE-1785: -- We probably need to figure out a way

[jira] Commented: (LUCENE-2138) Allow custom index readers when using IndexWriter.getReader

2009-12-09 Thread Jason Rutherglen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12788449#action_12788449 ] Jason Rutherglen commented on LUCENE-2138: -- I'm curious, will flex indexing

[jira] Created: (LUCENE-2138) Allow custom index readers when using IndexWriter.getReader

2009-12-08 Thread Jason Rutherglen (JIRA)
Components: Index Affects Versions: 3.0 Reporter: Jason Rutherglen Priority: Minor Fix For: 3.1 This is needed for backwards compatible support with Solr, and is a spin-off from SOLR-1606. -- This message is automatically generated by JIRA

[jira] Updated: (LUCENE-2138) Allow custom index readers when using IndexWriter.getReader

2009-12-08 Thread Jason Rutherglen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Rutherglen updated LUCENE-2138: - Attachment: LUCENE-2138.patch IndexWriter has an API to set and get

[jira] Updated: (LUCENE-1785) Simple FieldCache merging

2009-11-28 Thread Jason Rutherglen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Rutherglen updated LUCENE-1785: - Attachment: LUCENE-1785.patch The sanity check is fixed by skipping the entry

[jira] Commented: (LUCENE-2061) Create benchmark approach for testing Lucene's near real-time performance

2009-11-28 Thread Jason Rutherglen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12783322#action_12783322 ] Jason Rutherglen commented on LUCENE-2061: -- Mike, I tried running nrtBench.py

  1   2   3   4   5   6   7   8   9   >