[jira] Updated: (LUCENE-1470) Add TrieRangeQuery to contrib

2008-12-05 Thread Uwe Schindler (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uwe Schindler updated LUCENE-1470: -- Attachment: LUCENE-1470-readme.patch Here the readme changes. > Add TrieRangeQuery to contrib

[jira] Commented: (LUCENE-1470) Add TrieRangeQuery to contrib

2008-12-05 Thread Michael McCandless (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12653731#action_12653731 ] Michael McCandless commented on LUCENE-1470: Committed revision 723701. Thank

[jira] Commented: (LUCENE-1473) Implement standard Serialization across Lucene versions

2008-12-05 Thread Michael McCandless (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12653735#action_12653735 ] Michael McCandless commented on LUCENE-1473: SerializeUtils is missing from th

[jira] Assigned: (LUCENE-1478) Missing possibility to supply custom FieldParser when sorting search results

2008-12-05 Thread Michael McCandless (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael McCandless reassigned LUCENE-1478: -- Assignee: Michael McCandless > Missing possibility to supply custom FieldParse

[jira] Commented: (LUCENE-1478) Missing possibility to supply custom FieldParser when sorting search results

2008-12-05 Thread Michael McCandless (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12653741#action_12653741 ] Michael McCandless commented on LUCENE-1478: Patch looks good, thanks Uwe! Ba

[jira] Commented: (LUCENE-1476) BitVector implement DocIdSet

2008-12-05 Thread Michael McCandless (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12653752#action_12653752 ] Michael McCandless commented on LUCENE-1476: bq. But, SegmentReader needs rand

[jira] Commented: (LUCENE-831) Complete overhaul of FieldCache API/Implementation

2008-12-05 Thread Michael McCandless (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12653757#action_12653757 ] Michael McCandless commented on LUCENE-831: --- bq. change norm caching to use new c

Unnecessary messages creation by LogMergePolicy

2008-12-05 Thread Shai Erera
Hi As I looked at the code in LogMergePolicy (and its sub-classes), I came across such lines: message("findMergesToExpungeDeletes: " + numSegments + " segments"); Those lines print to the info stream (eventually) if it's not null. If one follows Java logging best practices, then any logging

Re: Unnecessary messages creation by LogMergePolicy

2008-12-05 Thread Michael McCandless
I agree, it is a best practice and we should follow it. Can you work out a patch & open an issue? I assume this means "if (infoStream != null)..." in this case. Mike Shai Erera wrote: Hi As I looked at the code in LogMergePolicy (and its sub-classes), I came across such lines:

Re: Unnecessary messages creation by LogMergePolicy

2008-12-05 Thread Shai Erera
I'll open an issue and work out a patch. Basically it means infoStream != null, although in LogMergePolicy I might add a specific method for that, because the messages are output if the IndexWriter member is not null and its infoStream is not null (this check is done by IndexWriter). Therefore I

[jira] Commented: (LUCENE-1476) BitVector implement DocIdSet

2008-12-05 Thread robert engels (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12653793#action_12653793 ] robert engels commented on LUCENE-1476: --- I don't think you can change this... In ma

[jira] Created: (LUCENE-1479) TrecDocMaker skips over documents when "Date" is missing from documents

2008-12-05 Thread Shai Erera (JIRA)
TrecDocMaker skips over documents when "Date" is missing from documents --- Key: LUCENE-1479 URL: https://issues.apache.org/jira/browse/LUCENE-1479 Project: Lucene - Java Is

[jira] Updated: (LUCENE-1479) TrecDocMaker skips over documents when "Date" is missing from documents

2008-12-05 Thread Shai Erera (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shai Erera updated LUCENE-1479: --- Attachment: LUCENE-1479.patch Patch to fix the bug > TrecDocMaker skips over documents when "Date"

[jira] Created: (LUCENE-1480) Wrap messages output with a check of InfoStream != null

2008-12-05 Thread Shai Erera (JIRA)
Wrap messages output with a check of InfoStream != null --- Key: LUCENE-1480 URL: https://issues.apache.org/jira/browse/LUCENE-1480 Project: Lucene - Java Issue Type: Improvement

[jira] Updated: (LUCENE-1480) Wrap messages output with a check of InfoStream != null

2008-12-05 Thread Shai Erera (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shai Erera updated LUCENE-1480: --- Attachment: LUCENE-1480.patch Patch that introduces messagesEnabled() in IndexWriter and fixes all c

Re: Unnecessary messages creation by LogMergePolicy

2008-12-05 Thread Michael McCandless
I like the method, but how about the name verbose(), ie: if (verbose()) ... Mike Shai Erera wrote: I'll open an issue and work out a patch. Basically it means infoStream != null, although in LogMergePolicy I might add a specific method for that, because the messages are output if

[jira] Assigned: (LUCENE-1480) Wrap messages output with a check of InfoStream != null

2008-12-05 Thread Michael McCandless (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael McCandless reassigned LUCENE-1480: -- Assignee: Michael McCandless > Wrap messages output with a check of InfoStream

[jira] Resolved: (LUCENE-1468) FSDirectory.list() is inconsistent

2008-12-05 Thread Michael McCandless (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael McCandless resolved LUCENE-1468. Resolution: Fixed Fix Version/s: 2.9 Committed revision 723789. Thanks Mar

Re: [jira] Commented: (LUCENE-1473) Implement Externalizable in main top level searcher classes

2008-12-05 Thread Doug Cutting
John Wang wrote: Thus we are enforcing users that care about Serialization to use the release jar. We already encourage folks to use a release jar if possible. So this is not a big change. Also, if folks choose to build their own jar, then they are expected to use that same jar everywhere,

[jira] Commented: (LUCENE-1473) Implement standard Serialization across Lucene versions

2008-12-05 Thread Doug Cutting (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12653869#action_12653869 ] Doug Cutting commented on LUCENE-1473: -- > How to write a unit test for multiple versi

[jira] Updated: (LUCENE-1473) Implement standard Serialization across Lucene versions

2008-12-05 Thread Jason Rutherglen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Rutherglen updated LUCENE-1473: - Attachment: LUCENE-1473.patch LUCENE-1473.patch Added Externalizable to Document, Field

[jira] Commented: (LUCENE-1480) Wrap messages output with a check of InfoStream != null

2008-12-05 Thread Shai Erera (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12653872#action_12653872 ] Shai Erera commented on LUCENE-1480: Hey Mike, I like the verbose() name. Would you li

[jira] Commented: (LUCENE-1480) Wrap messages output with a check of InfoStream != null

2008-12-05 Thread Michael McCandless (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12653877#action_12653877 ] Michael McCandless commented on LUCENE-1480: Sorry, could you make a new patch

Java logging in Lucene

2008-12-05 Thread Shai Erera
Hi I was wondering why doesn't the Lucene code uses Java logging, instead of the infoStream set in IndexWriter? Today, if I want to enable tracing of Lucene code, the only thing I can do is set an infoStream, but then I get many many messages. Moreoever, those messages seem to cover indexing code

[jira] Commented: (LUCENE-1476) BitVector implement DocIdSet

2008-12-05 Thread Marvin Humphrey (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12653883#action_12653883 ] Marvin Humphrey commented on LUCENE-1476: - > Marvin, in KS/Lucy are you using rand

Re: Java logging in Lucene

2008-12-05 Thread Grant Ingersoll
I think the main motivation has always been to have no dependencies in the core so as to keep it as fast and lightweight as possible. Then, of course, there is always the usual religious wars around which logging framework to use, not to mention the nightmare that is trying to manage multi

[jira] Commented: (LUCENE-1476) BitVector implement DocIdSet

2008-12-05 Thread Michael McCandless (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12653889#action_12653889 ] Michael McCandless commented on LUCENE-1476: bq. It seemed wrong to pay the m

[jira] Commented: (LUCENE-1476) BitVector implement DocIdSet

2008-12-05 Thread Michael McCandless (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12653891#action_12653891 ] Michael McCandless commented on LUCENE-1476: {quote} In many cases after you

Re: Java logging in Lucene

2008-12-05 Thread Doug Cutting
The infoStream stuff goes back to 1997, before there was log4j or any other Java logging framework. There's never been a big push to add logging to Lucene. It would add a dependency, and Lucene's jar has always been standalone, which is nice. Dependencies can conflict. If Lucene requires one

[jira] Assigned: (LUCENE-1448) add getFinalOffset() to TokenStream

2008-12-05 Thread Michael McCandless (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael McCandless reassigned LUCENE-1448: -- Assignee: Michael Busch (was: Michael McCandless) > add getFinalOffset() to T

[jira] Commented: (LUCENE-1448) add getFinalOffset() to TokenStream

2008-12-05 Thread Michael McCandless (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12653893#action_12653893 ] Michael McCandless commented on LUCENE-1448: {quote} What I'd like to work on

Re: Java logging in Lucene

2008-12-05 Thread Michael McCandless
I also feel that the primary usage of the internal messaging in Lucene today is debugging, and we don't need a logging framework for that. Mike Doug Cutting wrote: The infoStream stuff goes back to 1997, before there was log4j or any other Java logging framework. There's never been a big

[jira] Commented: (LUCENE-855) MemoryCachedRangeFilter to boost performance of Range queries

2008-12-05 Thread Otis Gospodnetic (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12653900#action_12653900 ] Otis Gospodnetic commented on LUCENE-855: - Hi Matt! :) Tim, want to benchmark the

[jira] Updated: (LUCENE-1480) Wrap messages output with a check of InfoStream != null

2008-12-05 Thread Shai Erera (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shai Erera updated LUCENE-1480: --- Attachment: LUCENE-1480-2.patch Renamed messagesEnabled to verbose. Thanks Mike ! > Wrap messages o

Re: [jira] Commented: (LUCENE-1473) Implement Externalizable in main top level searcher classes

2008-12-05 Thread John Wang
Doug: 1) "Incrementally upgrading distributed systems has, at least in the past, been outside the scope of Lucene" - That's good to know. Is it also out of the scope for distributed lucene effort (if it is still happening)? 2) I used the word broken to describe what happened for our deployment. I

Re: Java logging in Lucene

2008-12-05 Thread Shai Erera
Have you ever tried to debug your search application after it was shipped to a customer? When problems occur on the customer end, you cannot very easily reproduce problems because customers don't like to give you access to their systems, not always they are willing to share the index with you and l

[jira] Commented: (LUCENE-1476) BitVector implement DocIdSet

2008-12-05 Thread robert engels (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12653912#action_12653912 ] robert engels commented on LUCENE-1476: --- but IndexReader.document(n) throws an excep

[jira] Commented: (LUCENE-1476) BitVector implement DocIdSet

2008-12-05 Thread Michael McCandless (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12653915#action_12653915 ] Michael McCandless commented on LUCENE-1476: bq. but IndexReader.document thro

Re: [jira] Commented: (LUCENE-1473) Implement Externalizable in main top level searcher classes

2008-12-05 Thread Michael McCandless
John Wang wrote: My proposal is to add the suid to Serializable classes That's too brittle. If we do that, then what happens when we need to add a field to the class (eg, in 2.9 we've replaced "inclusive" in RangeQuery with "includeLower" and "includeUpper")? The standard answer is you bump

Re: Java logging in Lucene

2008-12-05 Thread Shai Erera
BTW, one thing I forgot to add: with infoStream, it's very difficult to extend the level of output if one wants, for example, to add logging messages to the search part (or other parts). The reason is one would need to permeate infoStream down too many classes. Instead, with Java logging, each clas

[jira] Commented: (LUCENE-1476) BitVector implement DocIdSet

2008-12-05 Thread Jason Rutherglen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12653923#action_12653923 ] Jason Rutherglen commented on LUCENE-1476: -- It would be great if instead of relyi

Re: Java logging in Lucene

2008-12-05 Thread Doug Cutting
Shai Erera wrote: Perhaps instead of introducing Java logging then (if you're too against it), we could introdue a static InfoStream class, with a static message() and isVerbose() methods. It's tempting to add our own logging API, as you suggest, but I fear that would re-invent what so many h

Re: [jira] Commented: (LUCENE-1473) Implement Externalizable in main top level searcher classes

2008-12-05 Thread John Wang
Mike: This has been gone back and forth on this thread already. Again, I agree it is not the perfect solution. I am comparing that to the current behavior, I don't think it is worse. (Only in my opinion). "live serialization" is not familiar to me. To understand it more, can you poin

Re: [jira] Commented: (LUCENE-1473) Implement Externalizable in main top level searcher classes

2008-12-05 Thread Doug Cutting
John Wang wrote: Also, if you find us addressing this issue being a hassle, e.g. addressing serialization in lucene is an incorrect thing to do, feel free to let us know and we can close the bug and terminate the thread. I don't know whether cross-version serialization belongs in Lucene. We

Re: Java logging in Lucene

2008-12-05 Thread John Wang
I thought the main point is to avoid a jar dependency. If we were to depend on a jar for logging, then why not log4j or commons-logging? -John On Fri, Dec 5, 2008 at 1:00 PM, Doug Cutting <[EMAIL PROTECTED]> wrote: > Shai Erera wrote: > >> Perhaps instead of introducing Java logging then (if yo

Re: Java logging in Lucene

2008-12-05 Thread Doug Cutting
John Wang wrote: If we were to depend on a jar for logging, then why not log4j or commons-logging? Lucene is used by many applications. Many of those applications already perform some kind of logging. We'd like whatever Lucene adds to fit in with their existing logging framework, not confli

Re: [jira] Commented: (LUCENE-1473) Implement Externalizable in main top level searcher classes

2008-12-05 Thread Doug Cutting
John Wang wrote: This has been gone back and forth on this thread already. Again, I agree it is not the perfect solution. I am comparing that to the current behavior, I don't think it is worse. (Only in my opinion). So, if it's good enough for you, a user of java serialization, then pe

Re: [jira] Commented: (LUCENE-1473) Implement Externalizable in main top level searcher classes

2008-12-05 Thread John Wang
Works for me. Thanks -John On Fri, Dec 5, 2008 at 1:23 PM, Doug Cutting <[EMAIL PROTECTED]> wrote: > John Wang wrote: > >> This has been gone back and forth on this thread already. Again, I >> agree it is not the perfect solution. I am comparing that to the current >> behavior, I don't th

Re: [jira] Commented: (LUCENE-1473) Implement Externalizable in main top level searcher classes

2008-12-05 Thread Michael McCandless
OK works for me too. John or Jason, can you update the patch on LUCENE-1743? We no longer need to implement Externalizable (just add fixed SUIDs), but we do need to update the javadocs for all classes implementing Serializable to state that cross-version compatibility is not guaranteed.

[jira] Commented: (LUCENE-1476) BitVector implement DocIdSet

2008-12-05 Thread Marvin Humphrey (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12653939#action_12653939 ] Marvin Humphrey commented on LUCENE-1476: - > Does it really need to throw an excep

Re: [jira] Commented: (LUCENE-1473) Implement Externalizable in main top level searcher classes

2008-12-05 Thread Jason Rutherglen
I think it's best to implement Externalizable as long as someone is willing to maintain it. I commit to maintaining the Externalizable code. The programming overhead is no more than implementing the equals method in the classes. New classes outside the Lucene code base simply need to implement S

Re: Java logging in Lucene

2008-12-05 Thread Grant Ingersoll
For some additional context, go over to the Solr mail archive and search for "Logging, SLF4J" or see http://lucene.markmail.org/message/gxifhjzmn6hgloy7?q=Solr+logging+SLF4J I personally don't like JUL and would be against using it. I could, maybe, just maybe, be talked into SLF4J. The oth

Re: Java logging in Lucene

2008-12-05 Thread Jason Rutherglen
As a developer who would like to eventually develop core code in Lucene (I started but then things changed drastically and so will wait for flexible indexing and other APIs?), a standard logging system would make development easier by making debugging easier. I rely heavily on log analysis in deve

Re: [jira] Commented: (LUCENE-1473) Implement Externalizable in main top level searcher classes

2008-12-05 Thread Doug Cutting
Jason Rutherglen wrote: I think it's best to implement Externalizable as long as someone is willing to maintain it. I commit to maintaining the Externalizable code. We need to agree to maintain things as a community, not as individuals. We can't rely on any particular individual being aroun

[jira] Commented: (LUCENE-1473) Implement standard Serialization across Lucene versions

2008-12-05 Thread Jason Rutherglen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12653955#action_12653955 ] Jason Rutherglen commented on LUCENE-1473: -- Doug wrote: We can save, in files, se

[jira] Commented: (LUCENE-1476) BitVector implement DocIdSet

2008-12-05 Thread robert engels (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12653954#action_12653954 ] robert engels commented on LUCENE-1476: --- That's my point, in complex multi-treaded s

[jira] Commented: (LUCENE-1476) BitVector implement DocIdSet

2008-12-05 Thread Marvin Humphrey (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12653959#action_12653959 ] Marvin Humphrey commented on LUCENE-1476: - > It would be great if instead of relyi

[jira] Commented: (LUCENE-1473) Implement standard Serialization across Lucene versions

2008-12-05 Thread Doug Cutting (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12653972#action_12653972 ] Doug Cutting commented on LUCENE-1473: -- > I guarantee 2.9 and above classes will be b

[jira] Updated: (LUCENE-1473) Implement standard Serialization across Lucene versions

2008-12-05 Thread Jason Rutherglen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Rutherglen updated LUCENE-1473: - Attachment: LUCENE-1473.patch LUCENE-1473.patch Added some more Externalizables. o.a

Re: [jira] Commented: (LUCENE-1473) Implement Externalizable in main top level searcher classes

2008-12-05 Thread Jason Rutherglen
The tests will be for backwards compatibility with previous versions of Lucene using the described process of including previous versioned encoded serialized objects into the test code base. Similar to how CFS index files are included in the test code tree. There is a an elegance to the RemoteSea

Re: Java logging in Lucene

2008-12-05 Thread Shai Erera
Doug, bq. It's tempting to add our own logging API, as you suggest, but I fear that would re-invent what so many have re-invented before. Haven't we already added our own logging API by introducing infoStream in IndexWriter? All I'm proposing (as an alternative to Java logging) is to make it a se