[jira] Commented: (LUCENE-2481) Enhance SnapshotDeletionPolicy to allow taking multiple snapshots

2010-05-29 Thread Earwin Burrfoot (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12873324#action_12873324 ] Earwin Burrfoot commented on LUCENE-2481: - Still dislike that string key you have

Re: Thoughts on CMS and SMS

2010-05-28 Thread Earwin Burrfoot
We just need an Executor-based MS, then we can throw all other out of the window, as threading concerns are now resolved by a proper choice of Executor supplied to constructor. Also an application has much more control over threading in multiple-index situations, as single Executor can be reused

Re: Thoughts on CMS and SMS

2010-05-28 Thread Earwin Burrfoot
is something that just popped into my mind when writing this email, so perhaps it doesn't make a lot of sense and needs some more mulling. Maybe MS should be renamed to MergeManager or something? Though I'm fine w/ MS too. Shai On Friday, May 28, 2010, Earwin Burrfoot ear...@gmail.com wrote

Re: TestBackwardsCompatibility

2010-05-27 Thread Earwin Burrfoot
I wanted to do this for some time, so let's open an issue! On Thu, May 27, 2010 at 19:13, Shai Erera ser...@gmail.com wrote: Ok ... that was rather fast and short ! So regarding trunk, is SegmentInfos the only place to look in? Can you give me more pointers? I'd like to create an issue for

Re: [jira] Commented: (LUCENE-2458) queryparser shouldn't generate phrasequeries based on term count

2010-05-23 Thread Earwin Burrfoot
The QP should work like that: (1) It parses the query, creating fragments (2) It does some out-of-the-box handling of those fragments People should be able to override that handling of fragments. But people should not touch (1). In fact QP should work like that: (1) Tokenizer parses the

[jira] Commented: (LUCENE-2471) Supporting bulk copies in Directory

2010-05-20 Thread Earwin Burrfoot (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2471?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12869504#action_12869504 ] Earwin Burrfoot commented on LUCENE-2471: - Bad link? The issue is closed already

[jira] Commented: (LUCENE-2471) Supporting bulk copies in Directory

2010-05-20 Thread Earwin Burrfoot (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2471?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12869537#action_12869537 ] Earwin Burrfoot commented on LUCENE-2471: - Ah. Actually there was two methods, one

[jira] Commented: (LUCENE-2471) Supporting bulk copies in Directory

2010-05-20 Thread Earwin Burrfoot (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2471?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12869579#action_12869579 ] Earwin Burrfoot commented on LUCENE-2471: - The only reason for keeping

[jira] Commented: (LUCENE-2471) Supporting bulk copies in Directory

2010-05-20 Thread Earwin Burrfoot (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2471?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12869595#action_12869595 ] Earwin Burrfoot commented on LUCENE-2471: - I actually suggested separating so

[jira] Commented: (LUCENE-2468) reopen on NRT reader should share readers w/ unchanged segments

2010-05-18 Thread Earwin Burrfoot (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12868571#action_12868571 ] Earwin Burrfoot commented on LUCENE-2468: - Or, you do it so various caches

[jira] Commented: (LUCENE-2468) reopen on NRT reader should share readers w/ unchanged segments

2010-05-18 Thread Earwin Burrfoot (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12868604#action_12868604 ] Earwin Burrfoot commented on LUCENE-2468: - Reusing fieldCacheKey is probably

[jira] Commented: (LUCENE-2454) Nested Document query support

2010-05-11 Thread Earwin Burrfoot (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2454?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12866134#action_12866134 ] Earwin Burrfoot commented on LUCENE-2454: - Both things can be combined for sure

Re: Adding another dimension to Lucene searches

2010-05-07 Thread Earwin Burrfoot
I've used something very similar to fold matching documents by some field value, like author_id. The very same issue with keeping all the parts in same segment, solved with composite documents that go through all the pipeline and flushing segments manually. On Fri, May 7, 2010 at 20:25, mark

[jira] Commented: (LUCENE-2447) Add support for subsets of searchables inside a MultiSearcher/ParallelMultiSearcher instance's methods at runtime

2010-05-05 Thread Earwin Burrfoot (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12864448#action_12864448 ] Earwin Burrfoot commented on LUCENE-2447: - Is creating MultiSearcher/MultiReader

[jira] Commented: (LUCENE-2447) Add support for subsets of searchables inside a MultiSearcher/ParallelMultiSearcher instance's methods at runtime

2010-05-05 Thread Earwin Burrfoot (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12864460#action_12864460 ] Earwin Burrfoot commented on LUCENE-2447: - I think there was a recent issue about

[jira] Commented: (LUCENE-2447) Add support for subsets of searchables inside a MultiSearcher/ParallelMultiSearcher instance's methods at runtime

2010-05-05 Thread Earwin Burrfoot (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12864476#action_12864476 ] Earwin Burrfoot commented on LUCENE-2447: - I think it was LUCENE-2041? But yep

[jira] Commented: (LUCENE-2440) Add support for custom ExecutorServices in ParallelMultiSearcher

2010-05-05 Thread Earwin Burrfoot (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2440?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12864477#action_12864477 ] Earwin Burrfoot commented on LUCENE-2440: - +1 Add support for custom

Re: Complex Query Parsing and Tokenization: ANTLR, JavaCC, Solr

2010-04-27 Thread Earwin Burrfoot
We use ANTLR for query parsing. Works good for the lazy guys :) On Tue, Apr 27, 2010 at 06:17, Tavi Nathanson tavi.nathan...@gmail.com wrote: Hey everyone, My organization uses our own homebrew QueryParser class, unrelated to Lucene's JavaCC-based QueryParser, to parse our queries. We don't

Re: Enhance SnapshotDeletionPolicy to allow taking multiple snapshots

2010-04-27 Thread Earwin Burrfoot
I think we should enhance SDP. I also think we shouldn't do IDs. snapshot() returns IndexCommitPoint, release() should get a parameter accepting IndexCommitPoint, that's all. On Tue, Apr 27, 2010 at 18:54, Michael McCandless luc...@mikemccandless.com wrote: This would be great! I think we

Re: [VOTE] Take 2: Open up a separate line for unstable Solr/Lucene development

2010-04-26 Thread Earwin Burrfoot
I'd like to +1 on this with all my tiny non-committer might. On Mon, Apr 26, 2010 at 23:06, Michael McCandless luc...@mikemccandless.com wrote: This is exactly the intention behind the proposal we are voting on. Big changes, that'd be destabilizing if attempted on the stable branch, would be

Re: Proposal about Version API relaxation

2010-04-25 Thread Earwin Burrfoot
And, it's not the committer's job to port each little commit to stable over to the unstable branch.  Instead, we periodically re-sync stable -- unstable, like we did with the long-lived flex branch. So, then, little would change on how stable is developed, today.  And stable would still be

Re: Proposal about Version API relaxation

2010-04-22 Thread Earwin Burrfoot
Okay, let's live with parallel development, but make sure we 'always' port things from stable to trunk, and 'always' remove possible back-compat layers when doing such a port? On Thu, Apr 22, 2010 at 18:04, Mark Miller markrmil...@gmail.com wrote: I'd vote -1 on Shai's variation and +1 on Mike's

Re: official GIT repository / switch to GIT?

2010-04-17 Thread Earwin Burrfoot
Why can't people just use svn or mercurial as a client for subversion repository? What is the benefit of migrating repository itself? On Sat, Apr 17, 2010 at 11:20, Thomas Koch tho...@koch.ro wrote: Hi, at least since august 2009 nobody has dared to ask this question, so let's start a

Re: official GIT repository / switch to GIT?

2010-04-17 Thread Earwin Burrfoot
These are broken, by the way. We need to kick someone to merge entries for lucenesolr and point them to a new svn url. On Sun, Apr 18, 2010 at 04:10, John Wang john.w...@gmail.com wrote: Hi Thomas:      There is a git mirror already: http://github.com/apache/lucene      All of apache projects

Re: Proposal about Version API relaxation

2010-04-15 Thread Earwin Burrfoot
1.6? Well ... not even that because as I understand it, 3.2 can move to Java 1.6 ... no API back-compat right :). That's definitely a great step forward ! Shai On Thu, Apr 15, 2010 at 1:34 AM, Andi Vajda va...@osafoundation.org wrote: On Thu, 15 Apr 2010, Earwin Burrfoot wrote: Can't

Re: SnapshotDeletionPolicy throws NPE if no commit happened

2010-04-15 Thread Earwin Burrfoot
We should just let IW create a null commit on an empty directory, like it always did ;) Then a whole class of such problems disappears. On Thu, Apr 15, 2010 at 11:16, Shai Erera ser...@gmail.com wrote: SDP throws NPE if the index includes no commits, but snapshot() is called. This is an extreme

Re: Proposal about Version API relaxation

2010-04-15 Thread Earwin Burrfoot
I think an index upgrade tool is okay? While you still definetly have to code it, things like if idxVer==m doOneStuff elseif idxVer==n doOtherStuff else blowUp are kept away from lucene innards and we all profit? On Thu, Apr 15, 2010 at 16:21, Robert Muir rcm...@gmail.com wrote: its open source,

Re: Proposal about Version API relaxation

2010-04-15 Thread Earwin Burrfoot
I like the idea of index conversion tool over silent online upgrade because it is 1. controllable - with online upgrade you never know for sure when your index is completely upgraded, even optimize() won't help here, as it is a noop for already-optimized indexes 2. way easier to write - as flex

Re: Proposal about Version API relaxation

2010-04-15 Thread Earwin Burrfoot
On Thu, Apr 15, 2010 at 17:17, Yonik Seeley yo...@lucidimagination.com wrote: Seamless online upgrades have their place too... say you are upgrading one server at a time in a cluster. Nothing here that can't be solved with an upgrade tool. Down one server, upgrade index, upgrade sofware, up.

Re: Proposal about Version API relaxation

2010-04-15 Thread Earwin Burrfoot
On Thu, Apr 15, 2010 at 17:49, Robert Muir rcm...@gmail.com wrote: wrong, it doesnt fix the analyzers problem. you need to reindex. On Thu, Apr 15, 2010 at 9:39 AM, Earwin Burrfoot ear...@gmail.com wrote: On Thu, Apr 15, 2010 at 17:17, Yonik Seeley yo...@lucidimagination.com wrote

Re: Proposal about Version API relaxation

2010-04-15 Thread Earwin Burrfoot
reasonable, but changing APIs around when there's not a good reason behind it (other than someone liked the name a little better) should still be approached with caution. Changing names is a good enough reason :) They make a darn difference between having to read a book to be able to use some

Re: Proposal about Version API relaxation

2010-04-15 Thread Earwin Burrfoot
First, the index format. IMHO, it is a good thing for a major release to be able to read the prior major release's index. And the ability to convert it to the current format via optimize is also good. Whatever is decided on this thread should take this seriously. Optimize is a bad way to

Re: Proposal about Version API relaxation

2010-04-15 Thread Earwin Burrfoot
...@gmail.com wrote: On 04/15/2010 01:50 PM, Earwin Burrfoot wrote: First, the index format. IMHO, it is a good thing for a major release to be able to read the prior major release's index. And the ability to convert it to the current format via optimize is also good. Whatever is decided on this thread

Re: Proposal about Version API relaxation

2010-04-15 Thread Earwin Burrfoot
BTW Earwin, we can come up w/ a migrate() method on IW to accomplish manual migration on the segments that are still on old versions. That's not the point about whether optimize() is good or not. It is the difference between telling the customer to run a 5-day migration process, or a couple

Re: Proposal about Version API relaxation

2010-04-15 Thread Earwin Burrfoot
On Thu, Apr 15, 2010 at 23:07, DM Smith dmsmith...@gmail.com wrote: On 04/15/2010 03:04 PM, Earwin Burrfoot wrote: BTW Earwin, we can come up w/ a migrate() method on IW to accomplish manual migration on the segments that are still on old versions. That's not the point about whether optimize

Re: Proposal about Version API relaxation

2010-04-15 Thread Earwin Burrfoot
Not sure if plain users are allowed/encouraged to post in this list, but wanted to mention (just an opinion from a happy user), as other users have, that not all of us can reindex just like that. It would not be 10 min for one of our installations for sure... First, i would need to implement

Re: Proposal about Version API relaxation

2010-04-15 Thread Earwin Burrfoot
2010/4/15 Shai Erera ser...@gmail.com: The reason Earwin why online migration is faster is because when u finally need to *fully* migrate your index, most chances are that most of the segments are already on the newer format. Offline migration will just keep the application idle for some

Re: Proposal about Version API relaxation

2010-04-15 Thread Earwin Burrfoot
I think this should split off the mega-thread :) On Thu, Apr 15, 2010 at 23:28, Uwe Schindler u...@thetaphi.de wrote: Hi Earwin, I am strongly +1 on this. I would also make the Release Manager for 3.1, if nobody else wants to do this. I would like to take the preflex tag or some revisions

Re: Proposal about Version API relaxation

2010-04-14 Thread Earwin Burrfoot
The thread somehow got sidetracked. So, let's get this carriage back on its rails? Let me remind - we have an API on hands that is mandatory and tends to be cumbersome. Proposed solution does indeed have ultrascary word static in it. But if you brace yourself and look closer - the use of said

Re: Proposal about Version API relaxation

2010-04-14 Thread Earwin Burrfoot
Can't believe my eyes. +1 On Thu, Apr 15, 2010 at 01:22, Michael McCandless luc...@mikemccandless.com wrote: On Wed, Apr 14, 2010 at 12:06 AM, Marvin Humphrey mar...@rectangular.com wrote: Essentially, we're free to break back compat within Lucy at any time, but we're not able to break back

Re: Proposal about Version API relaxation

2010-04-13 Thread Earwin Burrfoot
I wholeheartedly support this anti-version riot :) On Tue, Apr 13, 2010 at 19:27, Shai Erera ser...@gmail.com wrote: Hi I'd like to propose a relaxation on the Version API. Uwe, please read the entire email before you reply :). I was thinking, following a question on the user list, that the

[jira] Commented: (LUCENE-2355) Refactor Directory/Multi/SegmentReader creation/reopening/cloning/closing

2010-04-13 Thread Earwin Burrfoot (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12856509#action_12856509 ] Earwin Burrfoot commented on LUCENE-2355: - - Norms do both backward and forward

Re: [jira] Account password

2010-04-13 Thread Earwin Burrfoot
Priceless On Wed, Apr 14, 2010 at 00:53, j...@apache.org wrote:  You (or someone else) has reset your password. - Your password has been changed to: MCwqNr You can change your password here:  

Re: [jira] Account password

2010-04-13 Thread Earwin Burrfoot
It wasn't On Wed, Apr 14, 2010 at 02:06, Erick Erickson erickerick...@gmail.com wrote: A, good. That means the very long e-mail that came to my regular account about someone hacking the JIRA server is bogus too I assume.. Erick On Tue, Apr 13, 2010 at 5:58 PM, Uwe Schindler

[jira] Commented: (LUCENE-2386) IndexWriter commits unnecessarily on fresh Directory

2010-04-12 Thread Earwin Burrfoot (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12855889#action_12855889 ] Earwin Burrfoot commented on LUCENE-2386: - Meh, that all is just a matter

[jira] Commented: (LUCENE-2386) IndexWriter commits unnecessarily on fresh Directory

2010-04-12 Thread Earwin Burrfoot (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12855890#action_12855890 ] Earwin Burrfoot commented on LUCENE-2386: - bq. I'm sure such apps are more

[jira] Commented: (LUCENE-2386) IndexWriter commits unnecessarily on fresh Directory

2010-04-12 Thread Earwin Burrfoot (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12855899#action_12855899 ] Earwin Burrfoot commented on LUCENE-2386: - bq. The point here was that we should

[jira] Commented: (LUCENE-2386) IndexWriter commits unnecessarily on fresh Directory

2010-04-12 Thread Earwin Burrfoot (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12855937#action_12855937 ] Earwin Burrfoot commented on LUCENE-2386: - bq. So if you pass CREATE on an already

[jira] Commented: (LUCENE-2386) IndexWriter commits unnecessarily on fresh Directory

2010-04-12 Thread Earwin Burrfoot (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12855961#action_12855961 ] Earwin Burrfoot commented on LUCENE-2386: - I'm surrendering the issue, any outcome

[jira] Commented: (LUCENE-2355) Refactor Directory/Multi/SegmentReader creation/reopening/cloning/closing

2010-04-12 Thread Earwin Burrfoot (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12855964#action_12855964 ] Earwin Burrfoot commented on LUCENE-2355: - - Norm holds a reference to papa

[jira] Commented: (LUCENE-2316) Define clear semantics for Directory.fileLength

2010-04-11 Thread Earwin Burrfoot (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12855787#action_12855787 ] Earwin Burrfoot commented on LUCENE-2316: - I like Marvin's suggestion - deprecate

[jira] Commented: (LUCENE-2386) IndexWriter commits unnecessarily on fresh Directory

2010-04-11 Thread Earwin Burrfoot (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12855791#action_12855791 ] Earwin Burrfoot commented on LUCENE-2386: - I don't see why this should be fixed

[jira] Commented: (LUCENE-2386) IndexWriter commits unnecessarily on fresh Directory

2010-04-11 Thread Earwin Burrfoot (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12855793#action_12855793 ] Earwin Burrfoot commented on LUCENE-2386: - The mode in ctor is perfectly fine

[jira] Commented: (LUCENE-2376) java.lang.OutOfMemoryError:Java heap space

2010-04-10 Thread Earwin Burrfoot (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12855615#action_12855615 ] Earwin Burrfoot commented on LUCENE-2376: - A field is basically an index in itself

[jira] Commented: (LUCENE-2386) IndexWriter commits unnecessarily on fresh Directory

2010-04-09 Thread Earwin Burrfoot (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12855360#action_12855360 ] Earwin Burrfoot commented on LUCENE-2386: - I'm at loss for words. No, seriously

Re: Getting fsync out of the loop

2010-04-08 Thread Earwin Burrfoot
But, IW doesn't let you hold on to checkpoints... only to commits. Ie SnapshotDP will only see actual commit/close calls, not intermediate checkpoints like a random segment merge completing, a flush happening, etc. Or... maybe you would in fact call commit frequently from the main threads

[jira] Commented: (LUCENE-2376) java.lang.OutOfMemoryError:Java heap space

2010-04-07 Thread Earwin Burrfoot (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12854393#action_12854393 ] Earwin Burrfoot commented on LUCENE-2376: - That's the duplicate of LUCENE-2361

Re: Getting fsync out of the loop

2010-04-07 Thread Earwin Burrfoot
time improves, but I'm curious to know by how much. Shai On Wed, Apr 7, 2010 at 2:26 AM, Earwin Burrfoot ear...@gmail.com wrote: Running out of disk space with fsync disabled won't lead to corruption. Even kill -9 the JRE process with fsync disabled won't corrupt. In these cases index just

Re: Getting fsync out of the loop

2010-04-07 Thread Earwin Burrfoot
No, this doesn't make sense.  The OS detects a disk full on accepting the write into the write cache, not [later] on flushing the write cache to disk.  If the OS accepts the write, then disk is not full (ie flushing the cache will succeed, unless some other not-disk-full problem happens).

[jira] Commented: (LUCENE-2369) Locale-based sort by field with low memory overhead

2010-04-06 Thread Earwin Burrfoot (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12853883#action_12853883 ] Earwin Burrfoot commented on LUCENE-2369: - I would like to note that new types

Getting fsync out of the loop

2010-04-06 Thread Earwin Burrfoot
So, I want to pump my IndexWriter hard and fast with documents. Removing fsync from FSDirectory helps. But for that I pay with possibility of index corruption, not only if my node suddenly loses power/kernelpanics, but also if it runs out of disk space (which happens more frequently). I invented

Re: Getting fsync out of the loop

2010-04-06 Thread Earwin Burrfoot
Running out of disk space with fsync disabled won't lead to corruption. Even kill -9 the JRE process with fsync disabled won't corrupt. In these cases index just falls back to last successful commit. It's only power loss / OS / machine crash where you need fsync to avoid possible corruption

[jira] Commented: (LUCENE-2373) Change StandardTermsDictWriter to work with streaming and append-only filesystems

2010-04-06 Thread Earwin Burrfoot (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12854254#action_12854254 ] Earwin Burrfoot commented on LUCENE-2373: - And then IndexOutput.seek() can

Term space continuity

2010-04-05 Thread Earwin Burrfoot
A random thought from some of the earlier discussions. Had anybody used the fact that Lucene Term space is continuous (single per-index/segment space instead of separate per-field spaces) at least once? I only see code around that copes with this somehow, like checking term.field() == field just

Re: Term space continuity

2010-04-05 Thread Earwin Burrfoot
Wow! Cool. On Tue, Apr 6, 2010 at 03:51, Michael McCandless luc...@mikemccandless.com wrote: The flex API isolates fields, ie you get a TermsEnum for a given field and it enums only the term's text (as a BytesRef). Mike On Mon, Apr 5, 2010 at 7:22 PM, Earwin Burrfoot ear...@gmail.com wrote

[jira] Commented: (LUCENE-2355) Refactor Directory/Multi/SegmentReader creation/reopening/cloning/closing

2010-04-04 Thread Earwin Burrfoot (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12853244#action_12853244 ] Earwin Burrfoot commented on LUCENE-2355: - - DR.clone() doesn't do ensureOpen

[jira] Commented: (LUCENE-2355) Refactor Directory/Multi/SegmentReader creation/reopening/cloning/closing

2010-04-04 Thread Earwin Burrfoot (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12853280#action_12853280 ] Earwin Burrfoot commented on LUCENE-2355: - - when SegmentReader / CoreReaders

[jira] Commented: (LUCENE-2361) OutOfMemoryException while Indexing

2010-04-02 Thread Earwin Burrfoot (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12852994#action_12852994 ] Earwin Burrfoot commented on LUCENE-2361: - Which JVM

[jira] Commented: (LUCENE-2361) OutOfMemoryException while Indexing

2010-04-02 Thread Earwin Burrfoot (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12852996#action_12852996 ] Earwin Burrfoot commented on LUCENE-2361: - Also, are you running under profiler

[jira] Commented: (LUCENE-2265) improve automaton performance by running on byte[]

2010-04-02 Thread Earwin Burrfoot (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12852998#action_12852998 ] Earwin Burrfoot commented on LUCENE-2265: - I probably missed something. Why can't

[jira] Commented: (LUCENE-2265) improve automaton performance by running on byte[]

2010-04-02 Thread Earwin Burrfoot (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12853005#action_12853005 ] Earwin Burrfoot commented on LUCENE-2265: - So? You aren't making some generic

[jira] Commented: (LUCENE-2265) improve automaton performance by running on byte[]

2010-04-02 Thread Earwin Burrfoot (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12853006#action_12853006 ] Earwin Burrfoot commented on LUCENE-2265: - I mean, high-level, users don't care

[jira] Commented: (LUCENE-2265) improve automaton performance by running on byte[]

2010-04-02 Thread Earwin Burrfoot (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12853010#action_12853010 ] Earwin Burrfoot commented on LUCENE-2265: - Hm. I'd say that your highlevel

[jira] Commented: (LUCENE-2361) OutOfMemoryException while Indexing

2010-04-02 Thread Earwin Burrfoot (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12853033#action_12853033 ] Earwin Burrfoot commented on LUCENE-2361: - 20 is absolutely ok. Does exception

Re: Build failed in Hudson: Lucene-trunk #1144

2010-04-01 Thread Earwin Burrfoot
No, no, no, Lucene still has no need for maven or ivy for dependency management. We can just hack around all issues with ant scripts. : ) On Thu, Apr 1, 2010 at 09:48, Chris Hostetter hossman_luc...@fucit.org wrote: : I was wondering yesterday why aren't the required libs checked in to SVN? We

Re: Welcome Uwe Schindler to the Lucene PMC

2010-04-01 Thread Earwin Burrfoot
Generics SpecOps made it to the top and are gonna rule us from the shadows :) Congrats! On Thu, Apr 1, 2010 at 16:37, Robert Muir rcm...@gmail.com wrote: Congrats Uwe! On Thu, Apr 1, 2010 at 7:05 AM, Grant Ingersoll gsing...@apache.org wrote: I'm pleased to announce that the Lucene PMC has

[jira] Commented: (LUCENE-2355) Refactor Directory/Multi/SegmentReader creation/reopening/cloning/closing

2010-04-01 Thread Earwin Burrfoot (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12852417#action_12852417 ] Earwin Burrfoot commented on LUCENE-2355: - Some more stuff I forgot: - No need

Re: Build failed in Hudson: Lucene-trunk #1144

2010-04-01 Thread Earwin Burrfoot
it doesn't really matter if it's ant scripts, or ivy declarations, or maven pom entries -- the point is the same. We can't distribute the jars, but we can distribute programatic means for users to fetch teh jars themselves. (even if we magicly switched to ivy or maven for dependency

[jira] Commented: (LUCENE-2310) Reduce Fieldable, AbstractField and Field complexity

2010-03-31 Thread Earwin Burrfoot (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12851819#action_12851819 ] Earwin Burrfoot commented on LUCENE-2310: - Yahoo! +1 for introducing barebones

[jira] Commented: (LUCENE-2310) Reduce Fieldable, AbstractField and Field complexity

2010-03-31 Thread Earwin Burrfoot (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12851839#action_12851839 ] Earwin Burrfoot commented on LUCENE-2310: - Can't we keep interfaces for indexing

Re: Incremental Field Updates

2010-03-29 Thread Earwin Burrfoot
Of course introducing the idea of updates also introduces the notion of a primary key and there's probably an entirely separate discussion to be had around user-supplied vs Lucene-generated keys. Not sure I see that need.  Can you explain your reasoning a bit more? If you want to update a

Re: Incremental Field Updates

2010-03-29 Thread Earwin Burrfoot
Of course introducing the idea of updates also introduces the notion of a primary key and there's probably an entirely separate discussion to be had around user-supplied vs Lucene-generated keys. Not sure I see that need.  Can you explain your reasoning a bit more? If you want to update a

Re: Incremental Field Updates

2010-03-29 Thread Earwin Burrfoot
If someone needs this, it can be built over lucene, without introducing it as a core feature and needlessly complicating things. I think with any partial-update feature the *absence* of primary key support would  needlessly complicate things: If Lucene is not capable of performing duplicate

Re: Incremental Field Updates

2010-03-29 Thread Earwin Burrfoot
Variant d) sounds most logical? And enables all sorts of fun stuff. So the duplicate-key docs can have different values for initial-insert fields but partial updates will cause sharing of  a common field value? And subsequent same-key doc inserts do or don't share these previous

Re: Incremental Field Updates

2010-03-29 Thread Earwin Burrfoot
Who ever said that some_condition should point to a unique document? My assumption was, for now, we were still talking about the simpler case of updating a single document. If we extend the discussion to support set-based updates it's worth considering the common requirements for updating

[jira] Commented: (LUCENE-2356) Enable setting the terms index divisor used by IndexWriter whenever it opens internal readers

2010-03-29 Thread Earwin Burrfoot (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12850928#action_12850928 ] Earwin Burrfoot commented on LUCENE-2356: - That's likely orthogonal. If you want

[jira] Created: (LUCENE-2355) Refactor Directory/Multi/SegmentReader creation/reopening/cloning/closing

2010-03-28 Thread Earwin Burrfoot (JIRA)
Issue Type: Improvement Reporter: Earwin Burrfoot *Reader lifecycle evolved over time to become some heavily tangled mess. It's hard to understand what's going on there, it's even harder to add some fields/logic while ensuring that all possible code paths preserve these fields

[jira] Commented: (LUCENE-2345) Make it possible to subclass SegmentReader

2010-03-28 Thread Earwin Burrfoot (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12850726#action_12850726 ] Earwin Burrfoot commented on LUCENE-2345: - Just created LUCENE-2355 to track

Re: svn commit: r928246 [1/6] - in /lucene/java/branches/flex_1458: ./ backwards/src/ backwards/src/java/org/apache/lucene/search/ backwards/src/test/org/apache/lucene/analysis/ backwards/src/test/org

2010-03-27 Thread Earwin Burrfoot
I think original Directory.copy() just copied everything in flex, without nocommits? Unlike before, now you can specify which files do you want to have copied, so people can query Codecs and whatnot themselves. Author: uschindler Date: Sat Mar 27 19:12:08 2010 New Revision: 928246 URL:

Re: svn commit: r928246 [1/6] - in /lucene/java/branches/flex_1458: ./ backwards/src/ backwards/src/java/org/apache/lucene/search/ backwards/src/test/org/apache/lucene/analysis/ backwards/src/test/org

2010-03-27 Thread Earwin Burrfoot
index files, but Directory.copyTo copies everything so you must provide your own list if this matters. Mike On Sat, Mar 27, 2010 at 4:24 PM, Earwin Burrfoot ear...@gmail.com wrote: I think original Directory.copy() just copied everything in flex, without nocommits? Unlike before, now

[jira] Commented: (LUCENE-2345) Make it possible to subclass SegmentReader

2010-03-26 Thread Earwin Burrfoot (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12850067#action_12850067 ] Earwin Burrfoot commented on LUCENE-2345: - bq. will this change really conflict w

[jira] Commented: (LUCENE-2345) Make it possible to subclass SegmentReader

2010-03-26 Thread Earwin Burrfoot (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12850084#action_12850084 ] Earwin Burrfoot commented on LUCENE-2345: - More often than not init() methods

[jira] Commented: (LUCENE-2345) Make it possible to subclass SegmentReader

2010-03-26 Thread Earwin Burrfoot (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12850088#action_12850088 ] Earwin Burrfoot commented on LUCENE-2345: - bq. Earwin, can you post a patch w

Re: Modules

2010-03-26 Thread Earwin Burrfoot
Sounds good to me. I guess one thing to think about is the analyzers in core (should they move to this module, too?). If so, perhaps we could make 'ant test' of lucene depend on this module, since core tests use analyzers. But you could use lucene without an analyzers module, it wouldnt be a

Re: Modules

2010-03-26 Thread Earwin Burrfoot
On Fri, Mar 26, 2010 at 18:24, Robert Muir rcm...@gmail.com wrote: I would really love to see them all in one place though, for the users. I think that the elegance of our tests should be second to the users ease. Perhaps we could just have a fast and dirty TestAnalyzer so the core tests

[jira] Commented: (LUCENE-2345) Make it possible to subclass SegmentReader

2010-03-26 Thread Earwin Burrfoot (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12850348#action_12850348 ] Earwin Burrfoot commented on LUCENE-2345: - I mentioned this issue earlier. My

[jira] Updated: (LUCENE-2339) Allow Directory.copy() to accept a collection of file names to be copied

2010-03-24 Thread Earwin Burrfoot (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2339?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Earwin Burrfoot updated LUCENE-2339: Attachment: LUCENE-2339.patch 'ere we go! Read javadocs for _closeSafely_, it mimics

[jira] Updated: (LUCENE-2339) Allow Directory.copy() to accept a collection of file names to be copied

2010-03-24 Thread Earwin Burrfoot (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2339?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Earwin Burrfoot updated LUCENE-2339: Attachment: LUCENE-2339.patch Moved default Dir.copyTo to new close/exception handling

[jira] Commented: (LUCENE-2339) Allow Directory.copy() to accept a collection of file names to be copied

2010-03-24 Thread Earwin Burrfoot (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12849468#action_12849468 ] Earwin Burrfoot commented on LUCENE-2339: - bq. closeSafely is not exactly what I

[jira] Commented: (LUCENE-2345) Make it possible to subclass SegmentReader

2010-03-24 Thread Earwin Burrfoot (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12849475#action_12849475 ] Earwin Burrfoot commented on LUCENE-2345: - Ahem, that directly clashes with my

<    1   2   3   4   5   6   7   >