[jira] [Commented] (LUCENE-1344) Make the Lucene jar an OSGi bundle

[Luca Stancapiano (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-1344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13041023#comment-13041023
 ] 

Luca Stancapiano commented on LUCENE-1344:
--

I'm interested to this feauture too. Is it committed?

> Make the Lucene jar an OSGi bundle
> --
>
> Key: LUCENE-1344
> URL: https://issues.apache.org/jira/browse/LUCENE-1344
> Project: Lucene - Java
>  Issue Type: Improvement
>  Components: general/build
>Reporter: Nicolas Lalevée
>Priority: Minor
> Attachments: LUCENE-1344-3.0-branch.patch, LUCENE-1344-r679133.patch, 
> LUCENE-1344-r690675.patch, LUCENE-1344-r690691.patch, 
> LUCENE-1344-r696747.patch, LUCENE-1344.patch, LUCENE-1344.patch, 
> LUCENE-1344.patch, LUCENE-1344.patch, LUCENE-1344.patch, LUCENE-1344.patch, 
> MANIFEST.MF.diff
>
>
> In order to use Lucene in an OSGi environment, some additional headers are 
> needed in the manifest of the jar. As Lucene has no dependency, it is pretty 
> straight forward and it ill be easy to maintain I think.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-tests-only-docvalues-branch - Build # 1203 - Still Failing

[Apache Jenkins Server]

Build: 
https://builds.apache.org/hudson/job/Lucene-Solr-tests-only-docvalues-branch/1203/

All tests passed

Build Log (for compile errors):
[...truncated 12990 lines...]



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-1768) NumericRange support for new query parser

[Vinicius Barros (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-1768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vinicius Barros updated LUCENE-1768:


Attachment: week1.patch

This patch includes the work I did this first week. I started with one of the 
project's objective: restructure RangeQueryNode and its related classes to 
support number and text range queries.

I created some querynode interfaces, such as ValueQueryNode that abstract the 
value a leaf node may hold, since now, leaf nodes do not only hold text 
anymore, but also number values.

Let me know if you have any questions or any suggestions about the code.

I expect I created the patch correctly, as it's the first time I play with 
subversion

> NumericRange support for new query parser
> -
>
> Key: LUCENE-1768
> URL: https://issues.apache.org/jira/browse/LUCENE-1768
> Project: Lucene - Java
>  Issue Type: New Feature
>  Components: core/queryparser
>Affects Versions: 2.9
>Reporter: Uwe Schindler
>Assignee: Adriano Crestani
>  Labels: contrib, gsoc, gsoc2011, lucene-gsoc-11, mentor
> Fix For: 4.0
>
> Attachments: week1.patch
>
>
> It would be good to specify some type of "schema" for the query parser in 
> future, to automatically create NumericRangeQuery for different numeric 
> types? It would then be possible to index a numeric value 
> (double,float,long,int) using NumericField and then the query parser knows, 
> which type of field this is and so it correctly creates a NumericRangeQuery 
> for strings like "[1.567..*]" or "(1.787..19.5]".
> There is currently no way to extract if a field is numeric from the index, so 
> the user will have to configure the FieldConfig objects in the ConfigHandler. 
> But if this is done, it will not be that difficult to implement the rest.
> The only difference between the current handling of RangeQuery is then the 
> instantiation of the correct Query type and conversion of the entered numeric 
> values (simple Number.valueOf(...) cast of the user entered numbers). 
> Evenerything else is identical, NumericRangeQuery also supports the MTQ 
> rewrite modes (as it is a MTQ).
> Another thing is a change in Date semantics. There are some strange flags in 
> the current parser that tells it how to handle dates.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-2463) Using an evaluator outside the scope of an entity results in a null context

[Lance Norskog (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-2463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13040973#comment-13040973
 ] 

Lance Norskog commented on SOLR-2463:
-

How many Evaluators are there? I confess I have not seen this feature of DIH 
before. Are there unit tests for the feature, and its various permutations? If 
not, I would not rely on the feature.

> Using an evaluator outside the scope of an entity results in a null context
> ---
>
> Key: SOLR-2463
> URL: https://issues.apache.org/jira/browse/SOLR-2463
> Project: Solr
>  Issue Type: Bug
>  Components: contrib - DataImportHandler
>Affects Versions: 3.1, 3.1.1, 4.0
>Reporter: Robert Zotter
>Assignee: Shalin Shekhar Mangar
>Priority: Minor
> Fix For: 3.1.1
>
>
> When using an Evaluator outside an entity element the Context argument is 
> null.
> {code:title=foo.LowerCaseFunctionEvaluator.java|borderStyle=solid}
> public class LowerCaseFunctionEvaluator extends Evaluator {
>  public String evaluate(String expression, Context context) {
>List l = EvaluatorBag.parseParams(expression, 
> context.getVariableResolver());
>
>if (l.size() != 1) {
>  throw new RuntimeException("'toLowerCase' must have only one parameter 
> ");
>}
>return l.get(0).toString().toLowerCase();
>  }
> }
> {code}
> {code:title=data-config.xml|borderStyle=solid}
>  type="..."
> driver="..."
> url="..."
> user="${dataimporter.functions.toLowerCase('THIS_WILL_NOT_WORK')}"
> password="..."/>
> {code}
> {code:title=data-config.xml|borderStyle=solid}
>  dataSource="..."
> query="select * from 
> ${dataimporter.functions.toLowerCase('THIS_WILL_WORK')}"/>
> {code}
> This use case worked in 1.4

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: [JENKINS] Lucene-Solr-tests-only-docvalues-branch - Build # 1202 - Still Failing

[Simon Willnauer]

I committed a fix for those javadoc links


simon
On Sun, May 29, 2011 at 6:56 PM, Apache Jenkins Server
 wrote:
> Build: 
> https://builds.apache.org/hudson/job/Lucene-Solr-tests-only-docvalues-branch/1202/
>
> All tests passed
>
> Build Log (for compile errors):
> [...truncated 10565 lines...]
>
>
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org
>
>

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-tests-only-docvalues-branch - Build # 1202 - Still Failing

[Apache Jenkins Server]

Build: 
https://builds.apache.org/hudson/job/Lucene-Solr-tests-only-docvalues-branch/1202/

All tests passed

Build Log (for compile errors):
[...truncated 10565 lines...]



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-2424) extracted text from tika has no spaces

[Liam O'Boyle (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-2424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13040969#comment-13040969
 ] 

Liam O'Boyle commented on SOLR-2424:


Hi, sorry for the slow response, I don't seem to be receiving notifications of 
updates.  

You are correct; I used the Tika 0.9 command line tool, which worked correctly. 
 When I tried the 0.8 version the same problem occurs as is described in this 
ticket, so it appears that the bug is in Tika and that it is already resolved 
in the 0.9 release.

I'll try to update the version of Tika in use in my installation, although it's 
something that has caused more problems than it has solved when I've tried it 
in the past.

> extracted text from tika has no spaces
> --
>
> Key: SOLR-2424
> URL: https://issues.apache.org/jira/browse/SOLR-2424
> Project: Solr
>  Issue Type: Bug
>  Components: contrib - Solr Cell (Tika extraction)
>Affects Versions: 3.1
>Reporter: Yonik Seeley
> Attachments: ET2000 Service Manual.pdf
>
>
> Try this:
> curl 
> "http://localhost:8983/solr/update/extract?extractOnly=true&wt=json&indent=true";
>   -F "tutorial=@tutorial.pdf"
> And you get text output w/o spaces: 
> "ThisdocumentcoversthebasicsofrunningSolru"...

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Issue Comment Edited] (LUCENE-2793) Directory createOutput and openInput should take an IOContext

[Simon Willnauer (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-2793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13040906#comment-13040906
 ] 

Simon Willnauer edited comment on LUCENE-2793 at 5/30/11 12:19 AM:
---

bq. I already posted a patch to this issue a while back...

Jason I appreciate that you are still around on this issue. Varun is doing his 
GSoC project on this issue and others so there might be some duplication here 
and there or similarities with you patch but in this case this is ok though. He 
needs to get started as well as getting a feeling how things work here. So I 
hope you don't mind. I can only encourage you to help reviewing and commenting!


bq. If this is the right way to go ?

Thanks for the patch man! Good to get started eventually. here are some 
comments for you patch:

* When I look at IOContext I think it should be a little more sophisticated. 
What I have in mind is something similar to ScorerContext in 
org.apache.lucene.search.Weight.java. Some kind of a copy-on-write builder 
pattern that lets us provide some defaults for Merging, Searching and Indexing 
so by default we can reuse the same instance in many places. If we follow a 
this copy on write model we could also make this class non-final to let people 
customize the context if they needs to.
* I am not sure if we really need the enumeration in IO Context unless me make 
decisions based on what an input / output is used for. IMO it might make more 
sense to have default instances for Searching Indexing and Merging and set the 
flags like SEQUENTIAL and the buffers to good defaults and only tweak them when 
we are aware of the context ie. when we pull the input / output. The most of 
the usecases need good defaults and only some need tweaks but if we hold the 
use-case information on IOContext some directories might want to be very smart 
and this might duplicate code. 
* I wonder if it makes sense to bind the IO Context instance to the directory 
and add a factory to the directory like 
Directory#getIOContext(Merge|Search|Index) to enable the directory impl to set 
platform specific defaults. I think that would make things easier and 
customization would be straight forward.
* You should add a space after a comma in the arguments list like here: int 
bufferSize,IOContext context) 
* Enum elements should be CamelCase and start with a leading Uppercase character




  was (Author: simonw):
bq. I already posted a patch to this issue a while back...

Jason I appreciate that you are still around on this issue. Varun is doing his 
GSoC project on this issue and others so there might be some duplication here 
and there or similarities with you patch but in this case this is ok though. He 
needs to get started as well as getting a feeling how things work here. So I 
hope you don't mind. I can only encourage you to help reviewing and commenting!


bq. If this is the right way to go ?

Thanks for the patch man! Good to get started eventually. here are some 
comments for you patch:

* When I look at IOContext I think it should be a little more sophisticated. 
What I have in mind is something similar to ScorerContext in 
org.apache.lucene.search.Weight.java. Some kind of a copy-on-write builder 
pattern that lets us provide some defaults for Merging, Searching and Indexing 
so by default we can reuse the same instance in many places. If we follow a 
this copy on write model we could also make this class non-final to let people 
customize the context if they needs to.

* I am not sure if we really need the enumeration in IO Context unless me make 
decisions based on what an input / output is used for. IMO it might make more 
sense to have default instances for Searching Indexing and Merging and set the 
flags like SEQUENTIAL and the buffers to good defaults and only tweak them when 
we are aware of the context ie. when we pull the input / output. The most of 
the usecases need good defaults and only some need tweaks but if we hold the 
use-case information on IOContext some directories might want to be very smart 
and this might duplicate code. 
* I wonder if it makes sense to bind the IO Context instance to the directory 
and add a factory to the directory like 
Directory#getIOContext(Merge|Search|Index) to enable the directory impl to set 
platform specific defaults. I think that would make things easier and 
customization would be straight forward.
* You should add a space after a comma in the arguments list like here: int 
bufferSize,IOContext context) 
* Enum elements should be CamelCase and start with a leading Uppercase character



  
> Directory createOutput and openInput should take an IOContext
> -
>
> Key: LUCENE-2793
> URL: https://issues.apache.org/jira/browse/LUCENE-2793
> Project: Lucene - Java
>  Is

[jira] [Commented] (LUCENE-2793) Directory createOutput and openInput should take an IOContext

[Simon Willnauer (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-2793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13040906#comment-13040906
 ] 

Simon Willnauer commented on LUCENE-2793:
-

bq. I already posted a patch to this issue a while back...

Jason I appreciate that you are still around on this issue. Varun is doing his 
GSoC project on this issue and others so there might be some duplication here 
and there or similarities with you patch but in this case this is ok though. He 
needs to get started as well as getting a feeling how things work here. So I 
hope you don't mind. I can only encourage you to help reviewing and commenting!


bq. If this is the right way to go ?

Thanks for the patch man! Good to get started eventually. here are some 
comments for you patch:

* When I look at IOContext I think it should be a little more sophisticated. 
What I have in mind is something similar to ScorerContext in 
org.apache.lucene.search.Weight.java. Some kind of a copy-on-write builder 
pattern that lets us provide some defaults for Merging, Searching and Indexing 
so by default we can reuse the same instance in many places. If we follow a 
this copy on write model we could also make this class non-final to let people 
customize the context if they needs to.

* I am not sure if we really need the enumeration in IO Context unless me make 
decisions based on what an input / output is used for. IMO it might make more 
sense to have default instances for Searching Indexing and Merging and set the 
flags like SEQUENTIAL and the buffers to good defaults and only tweak them when 
we are aware of the context ie. when we pull the input / output. The most of 
the usecases need good defaults and only some need tweaks but if we hold the 
use-case information on IOContext some directories might want to be very smart 
and this might duplicate code. 
* I wonder if it makes sense to bind the IO Context instance to the directory 
and add a factory to the directory like 
Directory#getIOContext(Merge|Search|Index) to enable the directory impl to set 
platform specific defaults. I think that would make things easier and 
customization would be straight forward.
* You should add a space after a comma in the arguments list like here: int 
bufferSize,IOContext context) 
* Enum elements should be CamelCase and start with a leading Uppercase character




> Directory createOutput and openInput should take an IOContext
> -
>
> Key: LUCENE-2793
> URL: https://issues.apache.org/jira/browse/LUCENE-2793
> Project: Lucene - Java
>  Issue Type: Improvement
>  Components: core/store
>Reporter: Michael McCandless
>Assignee: Simon Willnauer
>  Labels: gsoc2011, lucene-gsoc-11, mentor
> Attachments: LUCENE-2793.patch, LUCENE-2793.patch
>
>
> Today for merging we pass down a larger readBufferSize than for searching 
> because we get better performance.
> I think we should generalize this to a class (IOContext), which would hold 
> the buffer size, but then could hold other flags like DIRECT (bypass OS's 
> buffer cache), SEQUENTIAL, etc.
> Then, we can make the DirectIOLinuxDirectory fully usable because we would 
> only use DIRECT/SEQUENTIAL during merging.
> This will require fixing how IW pools readers, so that a reader opened for 
> merging is not then used for searching, and vice/versa.  Really, it's only 
> all the open file handles that need to be different -- we could in theory 
> share del docs, norms, etc, if that were somehow possible.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-3099) Grouping module should allow subclasses to set the group key per document

[Martijn van Groningen (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-3099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13040893#comment-13040893
 ] 

Martijn van Groningen commented on LUCENE-3099:
---

I think the SecondPassGroupingCollector class needs the following two methods 
instead of getDocSlot:
{code:java}
  public void collect(int doc) throws IOException {
totalHitCount++;
if (gatherGroupedDocs(doc)) {
  retrieveGroup(doc).collector.collect(doc);
}
  }

  protected abstract boolean gatherGroupedDocs(int doc) throws IOException;

  protected abstract SearchGroupDocs retrieveGroup(int doc) 
throws IOException;
{code}

When working with fqs the slot is not practical, since there is no ords (like 
DocTermsIndex).
This setup works nicely in both impls. Downside is that for the Terms impl the 
ord key has to be looked up twice when gatherGroupedDocs(..) returns true.

> Grouping module should allow subclasses to set the group key per document
> -
>
> Key: LUCENE-3099
> URL: https://issues.apache.org/jira/browse/LUCENE-3099
> Project: Lucene - Java
>  Issue Type: Improvement
>Reporter: Michael McCandless
> Fix For: 3.2, 4.0
>
> Attachments: LUCENE-3099.patch, LUCENE-3099.patch, LUCENE-3099.patch, 
> LUCENE-3099.patch
>
>
> The new grouping module can only group by a single-valued indexed field.
> But, if we make the 'getGroupKey' a method that a subclass could override, 
> then I think we could refactor Solr over to the module, because it could do 
> function queries and normal queries via subclass (I think).
> This also makes the impl more extensible to apps that might have their own 
> interesting group values per document.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-3099) Grouping module should allow subclasses to set the group key per document

[Martijn van Groningen (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-3099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Martijn van Groningen updated LUCENE-3099:
--

Attachment: LUCENE-3099.patch

Attached an updated patch.

I'm currently busy with integrating the grouping module in trunk Solr. I 
noticed that FirstPassGroupingCollector and SecondPassGroupingCollector still 
has groupField as field and constructor argument. So I moved this to 
TermsFirstPassGroupingCollector and TermSecondPassGroupingCollector. Also made 
a small change in GroupDocs regarding generics.

> Grouping module should allow subclasses to set the group key per document
> -
>
> Key: LUCENE-3099
> URL: https://issues.apache.org/jira/browse/LUCENE-3099
> Project: Lucene - Java
>  Issue Type: Improvement
>Reporter: Michael McCandless
> Fix For: 3.2, 4.0
>
> Attachments: LUCENE-3099.patch, LUCENE-3099.patch, LUCENE-3099.patch, 
> LUCENE-3099.patch
>
>
> The new grouping module can only group by a single-valued indexed field.
> But, if we make the 'getGroupKey' a method that a subclass could override, 
> then I think we could refactor Solr over to the module, because it could do 
> function queries and normal queries via subclass (I think).
> This also makes the impl more extensible to apps that might have their own 
> interesting group values per document.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: [JENKINS] Lucene-Solr-tests-only-trunk - Build # 8497 - Failure

[Simon Willnauer]

it seems like there is a race in IW#closeInternal() that allows a
incoming thread to index a document into a DWPT that was empty (0
docs) when the "supposed to be" last flush kicked in. I will commit a
fix for it in a minute. I added an assert which I tripped immediately
while I wasn't able to reproduce the error. Let randomness take it
from here!

simon

On Sun, May 29, 2011 at 1:12 PM, Apache Jenkins Server
 wrote:
> Build: https://builds.apache.org/hudson/job/Lucene-Solr-tests-only-trunk/8497/
>
> 1 tests failed.
> REGRESSION:  
> org.apache.lucene.index.TestIndexWriterWithThreads.testCloseWithThreads
>
> Error Message:
> MockDirectoryWrapper: cannot close: there are still open files: {_s.fdt=1, 
> _s.tvx=1, _s.tvf=1, _s.tvd=1, _s.fdx=1}
>
> Stack Trace:
> java.lang.RuntimeException: MockDirectoryWrapper: cannot close: there are 
> still open files: {_s.fdt=1, _s.tvx=1, _s.tvf=1, _s.tvd=1, _s.fdx=1}
>        at 
> org.apache.lucene.store.MockDirectoryWrapper.close(MockDirectoryWrapper.java:461)
>        at 
> org.apache.lucene.index.TestIndexWriterWithThreads.testCloseWithThreads(TestIndexWriterWithThreads.java:214)
>        at 
> org.apache.lucene.util.LuceneTestCase$LuceneTestCaseRunner.runChild(LuceneTestCase.java:1334)
>        at 
> org.apache.lucene.util.LuceneTestCase$LuceneTestCaseRunner.runChild(LuceneTestCase.java:1252)
> Caused by: java.lang.RuntimeException: unclosed IndexOutput: _s.tvx
>        at 
> org.apache.lucene.store.MockDirectoryWrapper.addFileHandle(MockDirectoryWrapper.java:400)
>        at 
> org.apache.lucene.store.MockDirectoryWrapper.createOutput(MockDirectoryWrapper.java:376)
>        at 
> org.apache.lucene.index.TermVectorsTermsWriter.initTermVectorsWriter(TermVectorsTermsWriter.java:96)
>        at 
> org.apache.lucene.index.TermVectorsTermsWriter.finishDocument(TermVectorsTermsWriter.java:123)
>        at org.apache.lucene.index.TermsHash.finishDocument(TermsHash.java:138)
>        at 
> org.apache.lucene.index.DocInverter.finishDocument(DocInverter.java:95)
>        at 
> org.apache.lucene.index.DocFieldProcessor.finishDocument(DocFieldProcessor.java:285)
>        at 
> org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:246)
>        at 
> org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:372)
>        at 
> org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1469)
>        at 
> org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1441)
>        at 
> org.apache.lucene.index.TestIndexWriterWithThreads$IndexerThread.run(TestIndexWriterWithThreads.java:65)
>
>
>
>
> Build Log (for compile errors):
> [...truncated 3071 lines...]
>
>
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org
>
>

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-tests-only-docvalues-branch - Build # 1201 - Still Failing

[Apache Jenkins Server]

Build: 
https://builds.apache.org/hudson/job/Lucene-Solr-tests-only-docvalues-branch/1201/

All tests passed

Build Log (for compile errors):
[...truncated 10572 lines...]



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-3099) Grouping module should allow subclasses to set the group key per document

[Martijn van Groningen (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-3099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13040870#comment-13040870
 ] 

Martijn van Groningen commented on LUCENE-3099:
---

{quote}
Separately, we should merge the cool GroupValue, GroupValueSource,
GroupHolder, etc., from the first patch here, with Solr's equivalents,
factored out I think into a shared "common" module that the FQ module
(LUCENE-2883) can also use.
{quote}
+1! We need to find out what fq and grouping really needs under the hood. 
Performance can not be harmed by moving this into common module.
You think that we already should open a new issue for this?

> Grouping module should allow subclasses to set the group key per document
> -
>
> Key: LUCENE-3099
> URL: https://issues.apache.org/jira/browse/LUCENE-3099
> Project: Lucene - Java
>  Issue Type: Improvement
>Reporter: Michael McCandless
> Fix For: 3.2, 4.0
>
> Attachments: LUCENE-3099.patch, LUCENE-3099.patch, LUCENE-3099.patch
>
>
> The new grouping module can only group by a single-valued indexed field.
> But, if we make the 'getGroupKey' a method that a subclass could override, 
> then I think we could refactor Solr over to the module, because it could do 
> function queries and normal queries via subclass (I think).
> This also makes the impl more extensible to apps that might have their own 
> interesting group values per document.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-3099) Grouping module should allow subclasses to set the group key per document

[Martijn van Groningen (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-3099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Martijn van Groningen updated LUCENE-3099:
--

Attachment: LUCENE-3099.patch

Attached a new patch that is based on Mike's patch.
* All existing grouping tests pass
* AllGroupsCollector has also been included in this infrastructure.

Only the TermSecondPassGroupingCollector didn't work. The size groupDocs array 
was too small.

I think the following things need to be done:
* Update the documentation in package.html
* Backport to 3x

Should we rename all abstract collectors to Abstract*? To make it clear that 
these classes are abstract.

> Grouping module should allow subclasses to set the group key per document
> -
>
> Key: LUCENE-3099
> URL: https://issues.apache.org/jira/browse/LUCENE-3099
> Project: Lucene - Java
>  Issue Type: Improvement
>Reporter: Michael McCandless
> Fix For: 3.2, 4.0
>
> Attachments: LUCENE-3099.patch, LUCENE-3099.patch, LUCENE-3099.patch
>
>
> The new grouping module can only group by a single-valued indexed field.
> But, if we make the 'getGroupKey' a method that a subclass could override, 
> then I think we could refactor Solr over to the module, because it could do 
> function queries and normal queries via subclass (I think).
> This also makes the impl more extensible to apps that might have their own 
> interesting group values per document.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-tests-only-trunk - Build # 8497 - Failure

[Apache Jenkins Server]

Build: https://builds.apache.org/hudson/job/Lucene-Solr-tests-only-trunk/8497/

1 tests failed.
REGRESSION:  
org.apache.lucene.index.TestIndexWriterWithThreads.testCloseWithThreads

Error Message:
MockDirectoryWrapper: cannot close: there are still open files: {_s.fdt=1, 
_s.tvx=1, _s.tvf=1, _s.tvd=1, _s.fdx=1}

Stack Trace:
java.lang.RuntimeException: MockDirectoryWrapper: cannot close: there are still 
open files: {_s.fdt=1, _s.tvx=1, _s.tvf=1, _s.tvd=1, _s.fdx=1}
at 
org.apache.lucene.store.MockDirectoryWrapper.close(MockDirectoryWrapper.java:461)
at 
org.apache.lucene.index.TestIndexWriterWithThreads.testCloseWithThreads(TestIndexWriterWithThreads.java:214)
at 
org.apache.lucene.util.LuceneTestCase$LuceneTestCaseRunner.runChild(LuceneTestCase.java:1334)
at 
org.apache.lucene.util.LuceneTestCase$LuceneTestCaseRunner.runChild(LuceneTestCase.java:1252)
Caused by: java.lang.RuntimeException: unclosed IndexOutput: _s.tvx
at 
org.apache.lucene.store.MockDirectoryWrapper.addFileHandle(MockDirectoryWrapper.java:400)
at 
org.apache.lucene.store.MockDirectoryWrapper.createOutput(MockDirectoryWrapper.java:376)
at 
org.apache.lucene.index.TermVectorsTermsWriter.initTermVectorsWriter(TermVectorsTermsWriter.java:96)
at 
org.apache.lucene.index.TermVectorsTermsWriter.finishDocument(TermVectorsTermsWriter.java:123)
at org.apache.lucene.index.TermsHash.finishDocument(TermsHash.java:138)
at 
org.apache.lucene.index.DocInverter.finishDocument(DocInverter.java:95)
at 
org.apache.lucene.index.DocFieldProcessor.finishDocument(DocFieldProcessor.java:285)
at 
org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:246)
at 
org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:372)
at 
org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1469)
at 
org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1441)
at 
org.apache.lucene.index.TestIndexWriterWithThreads$IndexerThread.run(TestIndexWriterWithThreads.java:65)




Build Log (for compile errors):
[...truncated 3071 lines...]



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-2793) Directory createOutput and openInput should take an IOContext

[Varun Thacker (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-2793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13040850#comment-13040850
 ] 

Varun Thacker commented on LUCENE-2793:
---

I should have seen the patch! I have taken this up as a GSoC project. I'll try 
to use that patch too if it's ok.

> Directory createOutput and openInput should take an IOContext
> -
>
> Key: LUCENE-2793
> URL: https://issues.apache.org/jira/browse/LUCENE-2793
> Project: Lucene - Java
>  Issue Type: Improvement
>  Components: core/store
>Reporter: Michael McCandless
>Assignee: Simon Willnauer
>  Labels: gsoc2011, lucene-gsoc-11, mentor
> Attachments: LUCENE-2793.patch, LUCENE-2793.patch
>
>
> Today for merging we pass down a larger readBufferSize than for searching 
> because we get better performance.
> I think we should generalize this to a class (IOContext), which would hold 
> the buffer size, but then could hold other flags like DIRECT (bypass OS's 
> buffer cache), SEQUENTIAL, etc.
> Then, we can make the DirectIOLinuxDirectory fully usable because we would 
> only use DIRECT/SEQUENTIAL during merging.
> This will require fixing how IW pools readers, so that a reader opened for 
> merging is not then used for searching, and vice/versa.  Really, it's only 
> all the open file handles that need to be different -- we could in theory 
> share del docs, norms, etc, if that were somehow possible.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-3152) MockDirectoryWrapper should wrap the lockfactory

[Shai Erera (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-3152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13040848#comment-13040848
 ] 

Shai Erera commented on LUCENE-3152:


Robert, can we close this now that LUCENE-3147 is closed?

> MockDirectoryWrapper should wrap the lockfactory
> 
>
> Key: LUCENE-3152
> URL: https://issues.apache.org/jira/browse/LUCENE-3152
> Project: Lucene - Java
>  Issue Type: Bug
>  Components: general/test
>Reporter: Robert Muir
> Fix For: 3.3, 4.0
>
> Attachments: LUCENE-3152.patch
>
>
> After applying the patch from LUCENE-3147, I added a line to make the test 
> fail if it cannot remove its temporary directory.
> I ran 'ant test' on linux 50 times, and it passed all 50 times.
> But on windows, it failed often because of write.lock... this is because of 
> unclosed writers in the test.
> MockDirectoryWrapper is currently unaware of this write.lock, I think it 
> should wrap the lockfactory so that .close() will fail if there are any 
> outstanding locks.
> Then hopefully these tests would fail on linux too.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (LUCENE-3147) MockDirectoryWrapper should track open file handles of IndexOutput too

[Shai Erera (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-3147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shai Erera resolved LUCENE-3147.


Resolution: Fixed

Backported to 3x from trunk.

I decided to keep IW.closeMergeReaders as it is.

Thanks Mike, Robert and Simon. This was great teamwork !

> MockDirectoryWrapper should track open file handles of IndexOutput too
> --
>
> Key: LUCENE-3147
> URL: https://issues.apache.org/jira/browse/LUCENE-3147
> Project: Lucene - Java
>  Issue Type: Test
>  Components: general/test
>Reporter: Shai Erera
>Assignee: Shai Erera
>Priority: Minor
> Fix For: 3.2, 4.0
>
> Attachments: LUCENE-3147-3x.patch, LUCENE-3147.patch, 
> LUCENE-3147.patch, LUCENE-3147.patch, LUCENE-3147.patch, LUCENE-3147.patch, 
> LUCENE-3147.patch, LUCENE-3147.patch, LUCENE-3147.patch, LUCENE-3147.patch, 
> LUCENE-3147.patch, LUCENE-3147.patch, LUCENE-3147.patch, LUCENE-3147.patch
>
>
> MockDirectoryWrapper currently tracks open file handles of IndexInput only. 
> Therefore IO files that are not closed do not fail our tests, which can then 
> lead to test directories fail to delete on Windows. We should make sure all 
> open files are tracked and if they are left open, fail the test. I'll attach 
> a patch shortly.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-2793) Directory createOutput and openInput should take an IOContext

[Jason Rutherglen (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-2793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13040845#comment-13040845
 ] 

Jason Rutherglen commented on LUCENE-2793:
--

I already posted a patch to this issue a while back, 
https://issues.apache.org/jira/secure/attachment/12468030/LUCENE-2793.patch  It 
seems we're looping here.

> Directory createOutput and openInput should take an IOContext
> -
>
> Key: LUCENE-2793
> URL: https://issues.apache.org/jira/browse/LUCENE-2793
> Project: Lucene - Java
>  Issue Type: Improvement
>  Components: core/store
>Reporter: Michael McCandless
>Assignee: Simon Willnauer
>  Labels: gsoc2011, lucene-gsoc-11, mentor
> Attachments: LUCENE-2793.patch, LUCENE-2793.patch
>
>
> Today for merging we pass down a larger readBufferSize than for searching 
> because we get better performance.
> I think we should generalize this to a class (IOContext), which would hold 
> the buffer size, but then could hold other flags like DIRECT (bypass OS's 
> buffer cache), SEQUENTIAL, etc.
> Then, we can make the DirectIOLinuxDirectory fully usable because we would 
> only use DIRECT/SEQUENTIAL during merging.
> This will require fixing how IW pools readers, so that a reader opened for 
> merging is not then used for searching, and vice/versa.  Really, it's only 
> all the open file handles that need to be different -- we could in theory 
> share del docs, norms, etc, if that were somehow possible.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-2793) Directory createOutput and openInput should take an IOContext

[Varun Thacker (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-2793?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Varun Thacker updated LUCENE-2793:
--

Attachment: LUCENE-2793.patch

I have made a class IOContext and added a IOContext object to the Directory 
createOutput and openInput method parameters. 

If this is the right way to go ?

> Directory createOutput and openInput should take an IOContext
> -
>
> Key: LUCENE-2793
> URL: https://issues.apache.org/jira/browse/LUCENE-2793
> Project: Lucene - Java
>  Issue Type: Improvement
>  Components: core/store
>Reporter: Michael McCandless
>Assignee: Simon Willnauer
>  Labels: gsoc2011, lucene-gsoc-11, mentor
> Attachments: LUCENE-2793.patch, LUCENE-2793.patch
>
>
> Today for merging we pass down a larger readBufferSize than for searching 
> because we get better performance.
> I think we should generalize this to a class (IOContext), which would hold 
> the buffer size, but then could hold other flags like DIRECT (bypass OS's 
> buffer cache), SEQUENTIAL, etc.
> Then, we can make the DirectIOLinuxDirectory fully usable because we would 
> only use DIRECT/SEQUENTIAL during merging.
> This will require fixing how IW pools readers, so that a reader opened for 
> merging is not then used for searching, and vice/versa.  Really, it's only 
> all the open file handles that need to be different -- we could in theory 
> share del docs, norms, etc, if that were somehow possible.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: [JENKINS] Lucene-Solr-tests-only-trunk - Build # 8493 - Failure

[Shai Erera]

This is due to the latest commit on LUCENE-3147. I will commit a fix
shortly.

Shai

On Sun, May 29, 2011 at 8:44 PM, Apache Jenkins Server <
hud...@hudson.apache.org> wrote:

> Build:
> https://builds.apache.org/hudson/job/Lucene-Solr-tests-only-trunk/8493/
>
> 1 tests failed.
> REGRESSION:  org.apache.lucene.index.TestIndexWriter.testNoWaitClose
>
> Error Message:
> IndexFileDeleter doesn't know about file _d3_1.del
>
> Stack Trace:
> junit.framework.AssertionFailedError: IndexFileDeleter doesn't know about
> file _d3_1.del
>at
> org.apache.lucene.util.LuceneTestCase$LuceneTestCaseRunner.runChild(LuceneTestCase.java:1334)
>at
> org.apache.lucene.util.LuceneTestCase$LuceneTestCaseRunner.runChild(LuceneTestCase.java:1252)
>at
> org.apache.lucene.index.IndexWriter.filesExist(IndexWriter.java:3665)
>at
> org.apache.lucene.index.IndexWriter.startCommit(IndexWriter.java:3713)
>at
> org.apache.lucene.index.IndexWriter.prepareCommit(IndexWriter.java:2617)
>at
> org.apache.lucene.index.IndexWriter.commitInternal(IndexWriter.java:2688)
>at
> org.apache.lucene.index.IndexWriter.closeInternal(IndexWriter.java:1071)
>at org.apache.lucene.index.IndexWriter.close(IndexWriter.java:1014)
>at
> org.apache.lucene.index.TestIndexWriter.testNoWaitClose(TestIndexWriter.java:1302)
>
>
>
>
> Build Log (for compile errors):
> [...truncated 3054 lines...]
>
>
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org
>
>

[jira] [Commented] (SOLR-2553) Nested Field Collapsing

[Michael McCandless (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-2553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13040838#comment-13040838
 ] 

Michael McCandless commented on SOLR-2553:
--

The CachingCollector should reduce the search cost, since it can replay a 2nd 
time for the nested group collection.

We have to somehow open up the SecondPassGroupingCollector -- today it 
forcefully creates collectors specific to sort by score or field.

> Nested Field Collapsing
> ---
>
> Key: SOLR-2553
> URL: https://issues.apache.org/jira/browse/SOLR-2553
> Project: Solr
>  Issue Type: Improvement
>  Components: SearchComponents - other
>Reporter: Martijn Laarman
>
> Currently specifying grouping on multiple fields returns multiple datasets. 
> It would be nice if Solr supported cascading / nested grouping by applying 
> the first group over the entire result set, the next over each group and so 
> forth and so forth. 
> Even if limited to supporting nesting grouping 2 levels deep would cover alot 
> of use cases. 
> group.field=location&group.field=type
> -Location X
> ---Type 1
> -documents
> ---Type 2
> documents
> -Location Y
> ---Type 1
> documents
> ---Type 2
> documents
> instead of 
> -Location X
> -- documents
> -Location Y
> --documents
> -Type 1
> --documents
> -Type2
> --documents

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-2553) Nested Field Collapsing

[Michael McCandless (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-2553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13040837#comment-13040837
 ] 

Michael McCandless commented on SOLR-2553:
--

I think to represent this returned result we need to have GroupDocs extend 
TopDocs?  This way a GroupDocs, which currently uses TopDocs to hold each 
group's results, is free to recursively contain another GroupDocs.

> Nested Field Collapsing
> ---
>
> Key: SOLR-2553
> URL: https://issues.apache.org/jira/browse/SOLR-2553
> Project: Solr
>  Issue Type: Improvement
>  Components: SearchComponents - other
>Reporter: Martijn Laarman
>
> Currently specifying grouping on multiple fields returns multiple datasets. 
> It would be nice if Solr supported cascading / nested grouping by applying 
> the first group over the entire result set, the next over each group and so 
> forth and so forth. 
> Even if limited to supporting nesting grouping 2 levels deep would cover alot 
> of use cases. 
> group.field=location&group.field=type
> -Location X
> ---Type 1
> -documents
> ---Type 2
> documents
> -Location Y
> ---Type 1
> documents
> ---Type 2
> documents
> instead of 
> -Location X
> -- documents
> -Location Y
> --documents
> -Type 1
> --documents
> -Type2
> --documents

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-tests-only-trunk - Build # 8493 - Failure

[Apache Jenkins Server]

Build: https://builds.apache.org/hudson/job/Lucene-Solr-tests-only-trunk/8493/

1 tests failed.
REGRESSION:  org.apache.lucene.index.TestIndexWriter.testNoWaitClose

Error Message:
IndexFileDeleter doesn't know about file _d3_1.del

Stack Trace:
junit.framework.AssertionFailedError: IndexFileDeleter doesn't know about file 
_d3_1.del
at 
org.apache.lucene.util.LuceneTestCase$LuceneTestCaseRunner.runChild(LuceneTestCase.java:1334)
at 
org.apache.lucene.util.LuceneTestCase$LuceneTestCaseRunner.runChild(LuceneTestCase.java:1252)
at org.apache.lucene.index.IndexWriter.filesExist(IndexWriter.java:3665)
at 
org.apache.lucene.index.IndexWriter.startCommit(IndexWriter.java:3713)
at 
org.apache.lucene.index.IndexWriter.prepareCommit(IndexWriter.java:2617)
at 
org.apache.lucene.index.IndexWriter.commitInternal(IndexWriter.java:2688)
at 
org.apache.lucene.index.IndexWriter.closeInternal(IndexWriter.java:1071)
at org.apache.lucene.index.IndexWriter.close(IndexWriter.java:1014)
at 
org.apache.lucene.index.TestIndexWriter.testNoWaitClose(TestIndexWriter.java:1302)




Build Log (for compile errors):
[...truncated 3054 lines...]



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-3147) MockDirectoryWrapper should track open file handles of IndexOutput too

[Michael McCandless (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-3147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13040833#comment-13040833
 ] 

Michael McCandless commented on LUCENE-3147:


Well... the problem is, mergeMiddle isn't sync'd?  But, we could wrap the call 
to closeMergeReaders in sync block, and put the checkpoint() inside that?

> MockDirectoryWrapper should track open file handles of IndexOutput too
> --
>
> Key: LUCENE-3147
> URL: https://issues.apache.org/jira/browse/LUCENE-3147
> Project: Lucene - Java
>  Issue Type: Test
>  Components: general/test
>Reporter: Shai Erera
>Assignee: Shai Erera
>Priority: Minor
> Fix For: 3.2, 4.0
>
> Attachments: LUCENE-3147-3x.patch, LUCENE-3147.patch, 
> LUCENE-3147.patch, LUCENE-3147.patch, LUCENE-3147.patch, LUCENE-3147.patch, 
> LUCENE-3147.patch, LUCENE-3147.patch, LUCENE-3147.patch, LUCENE-3147.patch, 
> LUCENE-3147.patch, LUCENE-3147.patch, LUCENE-3147.patch, LUCENE-3147.patch
>
>
> MockDirectoryWrapper currently tracks open file handles of IndexInput only. 
> Therefore IO files that are not closed do not fail our tests, which can then 
> lead to test directories fail to delete on Windows. We should make sure all 
> open files are tracked and if they are left open, fail the test. I'll attach 
> a patch shortly.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-3147) MockDirectoryWrapper should track open file handles of IndexOutput too

[Shai Erera (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-3147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13040831#comment-13040831
 ] 

Shai Erera commented on LUCENE-3147:


I prefer that it's always called outside of closeMergeReaders. That's 
practically what happens already (checkpoint() is either called outside or 
inside in confusing code), so why not make it explicit and remove the confusing 
code?

> MockDirectoryWrapper should track open file handles of IndexOutput too
> --
>
> Key: LUCENE-3147
> URL: https://issues.apache.org/jira/browse/LUCENE-3147
> Project: Lucene - Java
>  Issue Type: Test
>  Components: general/test
>Reporter: Shai Erera
>Assignee: Shai Erera
>Priority: Minor
> Fix For: 3.2, 4.0
>
> Attachments: LUCENE-3147-3x.patch, LUCENE-3147.patch, 
> LUCENE-3147.patch, LUCENE-3147.patch, LUCENE-3147.patch, LUCENE-3147.patch, 
> LUCENE-3147.patch, LUCENE-3147.patch, LUCENE-3147.patch, LUCENE-3147.patch, 
> LUCENE-3147.patch, LUCENE-3147.patch, LUCENE-3147.patch, LUCENE-3147.patch
>
>
> MockDirectoryWrapper currently tracks open file handles of IndexInput only. 
> Therefore IO files that are not closed do not fail our tests, which can then 
> lead to test directories fail to delete on Windows. We should make sure all 
> open files are tracked and if they are left open, fail the test. I'll attach 
> a patch shortly.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-tests-only-docvalues-branch - Build # 1200 - Failure

[Apache Jenkins Server]

Build: 
https://builds.apache.org/hudson/job/Lucene-Solr-tests-only-docvalues-branch/1200/

All tests passed

Build Log (for compile errors):
[...truncated 10582 lines...]



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-3147) MockDirectoryWrapper should track open file handles of IndexOutput too

[Michael McCandless (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-3147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13040828#comment-13040828
 ] 

Michael McCandless commented on LUCENE-3147:


Ahh OK it is in fact necessary.  It's because closeMergeReaders is also called 
from the finally clause when there is an exception (hence the 
suppressExcs=true), and checkpoint is NOT then called from that location, so we 
do need to call it inside here.  Alternatively we could always call it inside 
closeMergeReaders, but then not above (inside commitMerge()).  Or we can leave 
it be for now :)

> MockDirectoryWrapper should track open file handles of IndexOutput too
> --
>
> Key: LUCENE-3147
> URL: https://issues.apache.org/jira/browse/LUCENE-3147
> Project: Lucene - Java
>  Issue Type: Test
>  Components: general/test
>Reporter: Shai Erera
>Assignee: Shai Erera
>Priority: Minor
> Fix For: 3.2, 4.0
>
> Attachments: LUCENE-3147-3x.patch, LUCENE-3147.patch, 
> LUCENE-3147.patch, LUCENE-3147.patch, LUCENE-3147.patch, LUCENE-3147.patch, 
> LUCENE-3147.patch, LUCENE-3147.patch, LUCENE-3147.patch, LUCENE-3147.patch, 
> LUCENE-3147.patch, LUCENE-3147.patch, LUCENE-3147.patch, LUCENE-3147.patch
>
>
> MockDirectoryWrapper currently tracks open file handles of IndexInput only. 
> Therefore IO files that are not closed do not fail our tests, which can then 
> lead to test directories fail to delete on Windows. We should make sure all 
> open files are tracked and if they are left open, fail the test. I'll attach 
> a patch shortly.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-3099) Grouping module should allow subclasses to set the group key per document

[Michael McCandless (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-3099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13040824#comment-13040824
 ] 

Michael McCandless commented on LUCENE-3099:



That patch is totally untested, and has at least 2 generics
warnings!  But hopefully the approach can work...

Basically the idea of the 2nd patch is to just make minimal extensions
points to the current grouping collectors, so that Solr could subclass
these and use its MutableValue/DocValues to manage the group keys.  I
think this would then mean Solr trunk and Solr 3.x could fully cutover
and not lose any functionality (and gain the benefits of the grouping
module).

Separately, we should merge the cool GroupValue, GroupValueSource,
GroupHolder, etc., from the first patch here, with Solr's equivalents,
factored out I think into a shared "common" module that the FQ module
(LUCENE-2883) can also use.


> Grouping module should allow subclasses to set the group key per document
> -
>
> Key: LUCENE-3099
> URL: https://issues.apache.org/jira/browse/LUCENE-3099
> Project: Lucene - Java
>  Issue Type: Improvement
>Reporter: Michael McCandless
> Fix For: 3.2, 4.0
>
> Attachments: LUCENE-3099.patch, LUCENE-3099.patch
>
>
> The new grouping module can only group by a single-valued indexed field.
> But, if we make the 'getGroupKey' a method that a subclass could override, 
> then I think we could refactor Solr over to the module, because it could do 
> function queries and normal queries via subclass (I think).
> This also makes the impl more extensible to apps that might have their own 
> interesting group values per document.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-2554) RandomSortField values are cached in the FieldCache

[Vadim Geshel (JIRA)]

RandomSortField values are cached in the FieldCache
---

 Key: SOLR-2554
 URL: https://issues.apache.org/jira/browse/SOLR-2554
 Project: Solr
  Issue Type: Bug
  Components: search
Affects Versions: 3.1
Reporter: Vadim Geshel


The values of RandomSortField get cached in the FieldCache. When using many 
RandomSortFields over time, this leads to running out of memory.

This may be one of the cases already covered in SOLR- but I'm not sure.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-3147) MockDirectoryWrapper should track open file handles of IndexOutput too

[Shai Erera (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-3147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13040823#comment-13040823
 ] 

Shai Erera commented on LUCENE-3147:


Hmm ... I think checkpoint() is needed after all. After I committed to trunk, I 
ran tests again on 3x before commit, and TestIndexWriter.testNoWaitClose failed:

{noformat}
[junit] Testsuite: org.apache.lucene.index.TestIndexWriter
[junit] Testcase: testNoWaitClose(org.apache.lucene.index.TestIndexWriter): 
FAILED
[junit] IndexFileDeleter doesn't know about file _a6_1.del
[junit] junit.framework.AssertionFailedError: IndexFileDeleter doesn't know 
about file _a6_1.del
[junit] at 
org.apache.lucene.util.LuceneTestCase$LuceneTestCaseRunner.runChild(LuceneTestCase.java:1207)
[junit] at 
org.apache.lucene.util.LuceneTestCase$LuceneTestCaseRunner.runChild(LuceneTestCase.java:1125)
[junit] at 
org.apache.lucene.index.IndexWriter.filesExist(IndexWriter.java:4421)
[junit] at 
org.apache.lucene.index.IndexWriter.startCommit(IndexWriter.java:4469)
[junit] at 
org.apache.lucene.index.IndexWriter.prepareCommit(IndexWriter.java:3354)
[junit] at 
org.apache.lucene.index.IndexWriter.commitInternal(IndexWriter.java:3425)
[junit] at 
org.apache.lucene.index.IndexWriter.closeInternal(IndexWriter.java:1870)
[junit] at 
org.apache.lucene.index.IndexWriter.close(IndexWriter.java:1813)
[junit] at 
org.apache.lucene.index.TestIndexWriter.testNoWaitClose(TestIndexWriter.java:1515)
[junit]
[junit]
[junit] Tests run: 68, Failures: 1, Errors: 0, Time elapsed: 24.884 sec
[junit]
[junit] - Standard Error -
[junit] NOTE: reproduce with: ant test -Dtestcase=TestIndexWriter 
-Dtestmethod=testNoWaitClose 
-Dtests.seed=8584244356995777356:6228535864161192995
[junit] NOTE: test params are: locale=da_DK, timezone=America/Menominee
[junit] NOTE: all tests run in this JVM:
[junit] [TestClassicAnalyzer, TestPerFieldAnalzyerWrapper, 
TestCharTermAttributeImpl, TestNumberTools, TestConcurrentMergeScheduler, 
TestFilterIndexReader, TestIndexWriter]
[junit] NOTE: Windows 7 6.1 build 7600 amd64/IBM Corporation 1.6.0 
(64-bit)/cpus=2,threads=3,free=1805128,total=34830336
[junit] -  ---
[junit] TEST org.apache.lucene.index.TestIndexWriter FAILED
{noformat}

Reproduced it on 3x, not on trunk. closeMergeReaders is called from 
commitMerge, which indeed calls checkpoint() afterwards unconditionally, but 
also from mergeMiddle, in its finally block if (!success). And the exception 
suggests that IFD does not know about a certain file, and checkpoint() calls 
deleter.refresh. So I'll revert my change to trunk and call checkpoint() if 
anyChanges + suppressExceptions. I don't want to change previous behavior.

> MockDirectoryWrapper should track open file handles of IndexOutput too
> --
>
> Key: LUCENE-3147
> URL: https://issues.apache.org/jira/browse/LUCENE-3147
> Project: Lucene - Java
>  Issue Type: Test
>  Components: general/test
>Reporter: Shai Erera
>Assignee: Shai Erera
>Priority: Minor
> Fix For: 3.2, 4.0
>
> Attachments: LUCENE-3147-3x.patch, LUCENE-3147.patch, 
> LUCENE-3147.patch, LUCENE-3147.patch, LUCENE-3147.patch, LUCENE-3147.patch, 
> LUCENE-3147.patch, LUCENE-3147.patch, LUCENE-3147.patch, LUCENE-3147.patch, 
> LUCENE-3147.patch, LUCENE-3147.patch, LUCENE-3147.patch, LUCENE-3147.patch
>
>
> MockDirectoryWrapper currently tracks open file handles of IndexInput only. 
> Therefore IO files that are not closed do not fail our tests, which can then 
> lead to test directories fail to delete on Windows. We should make sure all 
> open files are tracked and if they are left open, fail the test. I'll attach 
> a patch shortly.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-3099) Grouping module should allow subclasses to set the group key per document

[Michael McCandless (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-3099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael McCandless updated LUCENE-3099:
---

Attachment: LUCENE-3099.patch

Attached patch, with a possible more minimal approach to enabling Solr trunk to 
cutover to the grouping module.

> Grouping module should allow subclasses to set the group key per document
> -
>
> Key: LUCENE-3099
> URL: https://issues.apache.org/jira/browse/LUCENE-3099
> Project: Lucene - Java
>  Issue Type: Improvement
>Reporter: Michael McCandless
> Fix For: 3.2, 4.0
>
> Attachments: LUCENE-3099.patch, LUCENE-3099.patch
>
>
> The new grouping module can only group by a single-valued indexed field.
> But, if we make the 'getGroupKey' a method that a subclass could override, 
> then I think we could refactor Solr over to the module, because it could do 
> function queries and normal queries via subclass (I think).
> This also makes the impl more extensible to apps that might have their own 
> interesting group values per document.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-3147) MockDirectoryWrapper should track open file handles of IndexOutput too

[Shai Erera (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-3147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13040819#comment-13040819
 ] 

Shai Erera commented on LUCENE-3147:


ok I removed the call to checkpoint(), and do not track anyChanges anymore.

I tried "svn diff --force", but still nothing. And yes, it's an "svn merge" 
that I did. I guess we don't post patches after backport very often, so not a 
biggy.

I'll commit shortly.

> MockDirectoryWrapper should track open file handles of IndexOutput too
> --
>
> Key: LUCENE-3147
> URL: https://issues.apache.org/jira/browse/LUCENE-3147
> Project: Lucene - Java
>  Issue Type: Test
>  Components: general/test
>Reporter: Shai Erera
>Assignee: Shai Erera
>Priority: Minor
> Fix For: 3.2, 4.0
>
> Attachments: LUCENE-3147-3x.patch, LUCENE-3147.patch, 
> LUCENE-3147.patch, LUCENE-3147.patch, LUCENE-3147.patch, LUCENE-3147.patch, 
> LUCENE-3147.patch, LUCENE-3147.patch, LUCENE-3147.patch, LUCENE-3147.patch, 
> LUCENE-3147.patch, LUCENE-3147.patch, LUCENE-3147.patch, LUCENE-3147.patch
>
>
> MockDirectoryWrapper currently tracks open file handles of IndexInput only. 
> Therefore IO files that are not closed do not fail our tests, which can then 
> lead to test directories fail to delete on Windows. We should make sure all 
> open files are tracked and if they are left open, fail the test. I'll attach 
> a patch shortly.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: [JENKINS] Lucene-Solr-tests-only-3.x - Build # 8493 - Failure

[Michael McCandless]

My bad -- just committed another fix.

Mike

http://blog.mikemccandless.com

On Sun, May 29, 2011 at 12:12 PM, Apache Jenkins Server
 wrote:
> Build: https://builds.apache.org/hudson/job/Lucene-Solr-tests-only-3.x/8493/
>
> All tests passed
>
> Build Log (for compile errors):
> [...truncated 14096 lines...]
>
>
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org
>
>

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-tests-only-3.x - Build # 8493 - Failure

[Apache Jenkins Server]

Build: https://builds.apache.org/hudson/job/Lucene-Solr-tests-only-3.x/8493/

All tests passed

Build Log (for compile errors):
[...truncated 14096 lines...]



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: do the Jenkins while(true) builds now catch javadoc warnings...?

[Michael McCandless]

OK I committed that.

I also committed this, to hopefully fix the broken 3.x Maven build:

Index: hudson-lucene-solr-maven-3.x.sh
===
--- hudson-lucene-solr-maven-3.x.sh (revision 1128882)
+++ hudson-lucene-solr-maven-3.x.sh (working copy)
@@ -2,7 +2,7 @@
 . $WORKSPACE/nightly/hudson-settings.sh

 #Update the Version # when doing a release
-MAVEN_ARTIFACT_VERSION=3.2-SNAPSHOT
+MAVEN_ARTIFACT_VERSION=3.3-SNAPSHOT
 SOLR_JAVA_HOME=$JAVA_HOME_15

Mike

http://blog.mikemccandless.com

On Sun, May 29, 2011 at 10:55 AM, Michael McCandless
 wrote:
> OK I think this should do it?  I'll commit shortly if nobody objects...:
>
> Index: hudson-lusolr-tests-trunk.sh
> ===
> --- hudson-lusolr-tests-trunk.sh        (revision 1128882)
> +++ hudson-lusolr-tests-trunk.sh        (working copy)
> @@ -21,3 +21,29 @@
>   -Djunit.formatter=xml \
>   -Dargs="$TEST_JVM_ARGS" \
>   test
> +
> +# verify lucene core javadocs have no warnings w/ Java 1.5
> +cd $WORKSPACE/$CORE_DIR
> +JAVA_HOME=$JAVA_HOME_15 $ANT_HOME/bin/ant \
> +  javadocs-all javadocs-test-framework
> +
> +# verify lucene core javadocs have no warnings w/ Java 1.6
> +cd $WORKSPACE/$CORE_DIR
> +JAVA_HOME=$JAVA_HOME_16 $ANT_HOME/bin/ant \
> +  javadocs-all javadocs-test-framework
> +
> +# verify modules' javadocs have no warnings w/ Java 1.5
> +cd $WORKSPACE/$MODULES_DIR
> +JAVA_HOME=$JAVA_HOME_15 $ANT_HOME/bin/ant \
> +  javadocs
> +
> +# verify modules' javadocs have no warnings w/ Java 1.6
> +cd $WORKSPACE/$MODULES_DIR
> +JAVA_HOME=$JAVA_HOME_16 $ANT_HOME/bin/ant \
> +  javadocs
> +
> +# verify solr javadocs have no warnings
> +cd $WORKSPACE/$SOLR_DIR
> +JAVA_HOME=$JAVA_HOME_16 $ANT_HOME/bin/ant \
> +  javadocs
> +
> Index: hudson-lusolr-tests-3.x.sh
> ===
> --- hudson-lusolr-tests-3.x.sh  (revision 1128882)
> +++ hudson-lusolr-tests-3.x.sh  (working copy)
> @@ -25,3 +25,19 @@
>   -Djunit.formatter=xml \
>   -Dargs="$TEST_JVM_ARGS" \
>   test
> +
> +# verify lucene core javadocs have no warnings w/ Java 1.5
> +cd $WORKSPACE/$CORE_DIR
> +JAVA_HOME=$JAVA_HOME_15 $ANT_HOME/bin/ant \
> +  javadocs-all javadocs-test-framework
> +
> +# verify lucene core javadocs have no warnings w/ Java 1.6
> +cd $WORKSPACE/$CORE_DIR
> +JAVA_HOME=$JAVA_HOME_16 $ANT_HOME/bin/ant \
> +  javadocs-all javadocs-test-framework
> +
> +# verify solr javadocs have no warnings
> +cd $WORKSPACE/$SOLR_DIR
> +JAVA_HOME=$JAVA_HOME_16 $ANT_HOME/bin/ant \
> +  javadocs
> +
>
> Mike
>
> http://blog.mikemccandless.com
>
> On Sun, May 29, 2011 at 10:45 AM, Michael McCandless
>  wrote:
>> I think for Solr it's actually "ant javadocs" (confusingly!).  This
>> seems to subsume javadoc-all?
>>
>> Also, I don't think I'm getting javadoc warnings for Solr?  (I do get
>> lots of compilation warnings).  If you introduce a javadoc warning it
>> fails the build...
>>
>> I agree it'd be great if we could gen the non-Solr javadocs with both
>> 1.5 and 1.6.  In theory this should be easy?  "export JAVA_HOME=;
>> ant javadocs"?
>>
>> I'll try to make a patch for the nightly shell scripts...
>>
>> Mike
>>
>> http://blog.mikemccandless.com
>>
>> On Sun, May 29, 2011 at 10:20 AM, Steven A Rowe  wrote:
>>> I checked what it would take to test all of the javadocs on trunk, and 
>>> found that there is no one target to call.  To cover all javadocs:
>>>
>>>        cd lucene ; ant javadocs-all javadocs-test-framework
>>>        cd ../modules ; ant javadocs
>>>        cd ../solr ; ant javadoc-all javadoc-test-framework
>>>
>>> Ideally, the javadocs would be generated once under 1.5 and again under 
>>> 1.6, for all of branch_3x, and for lucene/ & modules/ on trunk.
>>>
>>> I got a bunch of javadoc warnings in Solr, so (assuming I don't have setup 
>>> issues) some work needs to be done to fix the warnings before bombarding 
>>> everybody with failure messages every 30 minutes.
>>>
>>> I assume the branch_3x issues are similar, but I haven't looked into it yet.
>>>
>>> Steve
>>>
 -Original Message-
 From: Michael McCandless [mailto:luc...@mikemccandless.com]
 Sent: Sunday, May 29, 2011 8:48 AM
 To: dev@lucene.apache.org Dev
 Subject: do the Jenkins while(true) builds now catch javadoc warnings...?

 I know we discussed that this was an easy fix to the shell script...
 but did anyone make the change?

 Ie, so that we catch JS warnings in the while(true) build, not just
 nightly/POM builds.

 Mike

 http://blog.mikemccandless.com

 -
 To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
 For additional commands, e-mail: dev-h...@lucene.apache.org
>>>
>>>
>>
>

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.ap

[jira] [Commented] (LUCENE-3147) MockDirectoryWrapper should track open file handles of IndexOutput too

[Michael McCandless (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-3147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13040809#comment-13040809
 ] 

Michael McCandless commented on LUCENE-3147:


bq. Also, why do we call checkpoint() only if suppressWarnings=true and not 
only if anyChanges=true? 

Good question!  My guess is the checkpoint() can be removed?  We call 
checkpoint up above after closeMergeReaders unconditionally...

bq. It's weird about MLFWrapper - when I 'svn stat' it appears w/ "A +" and not 
"?". Yet "svn diff MLFW" yields no diff ... If I "svn ci", I see that it's 
included in the list of files ... strange.

Ahh yes OK.  This is from "svn merge" I think?  Like "svn diff" won't show 
it... maybe there's some option to svn diff to show these "added due to merge" 
changes?

> MockDirectoryWrapper should track open file handles of IndexOutput too
> --
>
> Key: LUCENE-3147
> URL: https://issues.apache.org/jira/browse/LUCENE-3147
> Project: Lucene - Java
>  Issue Type: Test
>  Components: general/test
>Reporter: Shai Erera
>Assignee: Shai Erera
>Priority: Minor
> Fix For: 3.2, 4.0
>
> Attachments: LUCENE-3147-3x.patch, LUCENE-3147.patch, 
> LUCENE-3147.patch, LUCENE-3147.patch, LUCENE-3147.patch, LUCENE-3147.patch, 
> LUCENE-3147.patch, LUCENE-3147.patch, LUCENE-3147.patch, LUCENE-3147.patch, 
> LUCENE-3147.patch, LUCENE-3147.patch, LUCENE-3147.patch, LUCENE-3147.patch
>
>
> MockDirectoryWrapper currently tracks open file handles of IndexInput only. 
> Therefore IO files that are not closed do not fail our tests, which can then 
> lead to test directories fail to delete on Windows. We should make sure all 
> open files are tracked and if they are left open, fail the test. I'll attach 
> a patch shortly.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: do the Jenkins while(true) builds now catch javadoc warnings...?

[Michael McCandless]

OK I think this should do it?  I'll commit shortly if nobody objects...:

Index: hudson-lusolr-tests-trunk.sh
===
--- hudson-lusolr-tests-trunk.sh(revision 1128882)
+++ hudson-lusolr-tests-trunk.sh(working copy)
@@ -21,3 +21,29 @@
   -Djunit.formatter=xml \
   -Dargs="$TEST_JVM_ARGS" \
   test
+
+# verify lucene core javadocs have no warnings w/ Java 1.5
+cd $WORKSPACE/$CORE_DIR
+JAVA_HOME=$JAVA_HOME_15 $ANT_HOME/bin/ant \
+  javadocs-all javadocs-test-framework
+
+# verify lucene core javadocs have no warnings w/ Java 1.6
+cd $WORKSPACE/$CORE_DIR
+JAVA_HOME=$JAVA_HOME_16 $ANT_HOME/bin/ant \
+  javadocs-all javadocs-test-framework
+
+# verify modules' javadocs have no warnings w/ Java 1.5
+cd $WORKSPACE/$MODULES_DIR
+JAVA_HOME=$JAVA_HOME_15 $ANT_HOME/bin/ant \
+  javadocs
+
+# verify modules' javadocs have no warnings w/ Java 1.6
+cd $WORKSPACE/$MODULES_DIR
+JAVA_HOME=$JAVA_HOME_16 $ANT_HOME/bin/ant \
+  javadocs
+
+# verify solr javadocs have no warnings
+cd $WORKSPACE/$SOLR_DIR
+JAVA_HOME=$JAVA_HOME_16 $ANT_HOME/bin/ant \
+  javadocs
+
Index: hudson-lusolr-tests-3.x.sh
===
--- hudson-lusolr-tests-3.x.sh  (revision 1128882)
+++ hudson-lusolr-tests-3.x.sh  (working copy)
@@ -25,3 +25,19 @@
   -Djunit.formatter=xml \
   -Dargs="$TEST_JVM_ARGS" \
   test
+
+# verify lucene core javadocs have no warnings w/ Java 1.5
+cd $WORKSPACE/$CORE_DIR
+JAVA_HOME=$JAVA_HOME_15 $ANT_HOME/bin/ant \
+  javadocs-all javadocs-test-framework
+
+# verify lucene core javadocs have no warnings w/ Java 1.6
+cd $WORKSPACE/$CORE_DIR
+JAVA_HOME=$JAVA_HOME_16 $ANT_HOME/bin/ant \
+  javadocs-all javadocs-test-framework
+
+# verify solr javadocs have no warnings
+cd $WORKSPACE/$SOLR_DIR
+JAVA_HOME=$JAVA_HOME_16 $ANT_HOME/bin/ant \
+  javadocs
+

Mike

http://blog.mikemccandless.com

On Sun, May 29, 2011 at 10:45 AM, Michael McCandless
 wrote:
> I think for Solr it's actually "ant javadocs" (confusingly!).  This
> seems to subsume javadoc-all?
>
> Also, I don't think I'm getting javadoc warnings for Solr?  (I do get
> lots of compilation warnings).  If you introduce a javadoc warning it
> fails the build...
>
> I agree it'd be great if we could gen the non-Solr javadocs with both
> 1.5 and 1.6.  In theory this should be easy?  "export JAVA_HOME=;
> ant javadocs"?
>
> I'll try to make a patch for the nightly shell scripts...
>
> Mike
>
> http://blog.mikemccandless.com
>
> On Sun, May 29, 2011 at 10:20 AM, Steven A Rowe  wrote:
>> I checked what it would take to test all of the javadocs on trunk, and found 
>> that there is no one target to call.  To cover all javadocs:
>>
>>        cd lucene ; ant javadocs-all javadocs-test-framework
>>        cd ../modules ; ant javadocs
>>        cd ../solr ; ant javadoc-all javadoc-test-framework
>>
>> Ideally, the javadocs would be generated once under 1.5 and again under 1.6, 
>> for all of branch_3x, and for lucene/ & modules/ on trunk.
>>
>> I got a bunch of javadoc warnings in Solr, so (assuming I don't have setup 
>> issues) some work needs to be done to fix the warnings before bombarding 
>> everybody with failure messages every 30 minutes.
>>
>> I assume the branch_3x issues are similar, but I haven't looked into it yet.
>>
>> Steve
>>
>>> -Original Message-
>>> From: Michael McCandless [mailto:luc...@mikemccandless.com]
>>> Sent: Sunday, May 29, 2011 8:48 AM
>>> To: dev@lucene.apache.org Dev
>>> Subject: do the Jenkins while(true) builds now catch javadoc warnings...?
>>>
>>> I know we discussed that this was an easy fix to the shell script...
>>> but did anyone make the change?
>>>
>>> Ie, so that we catch JS warnings in the while(true) build, not just
>>> nightly/POM builds.
>>>
>>> Mike
>>>
>>> http://blog.mikemccandless.com
>>>
>>> -
>>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
>>> For additional commands, e-mail: dev-h...@lucene.apache.org
>>
>>
>

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: do the Jenkins while(true) builds now catch javadoc warnings...?

[Michael McCandless]

I think for Solr it's actually "ant javadocs" (confusingly!).  This
seems to subsume javadoc-all?

Also, I don't think I'm getting javadoc warnings for Solr?  (I do get
lots of compilation warnings).  If you introduce a javadoc warning it
fails the build...

I agree it'd be great if we could gen the non-Solr javadocs with both
1.5 and 1.6.  In theory this should be easy?  "export JAVA_HOME=;
ant javadocs"?

I'll try to make a patch for the nightly shell scripts...

Mike

http://blog.mikemccandless.com

On Sun, May 29, 2011 at 10:20 AM, Steven A Rowe  wrote:
> I checked what it would take to test all of the javadocs on trunk, and found 
> that there is no one target to call.  To cover all javadocs:
>
>        cd lucene ; ant javadocs-all javadocs-test-framework
>        cd ../modules ; ant javadocs
>        cd ../solr ; ant javadoc-all javadoc-test-framework
>
> Ideally, the javadocs would be generated once under 1.5 and again under 1.6, 
> for all of branch_3x, and for lucene/ & modules/ on trunk.
>
> I got a bunch of javadoc warnings in Solr, so (assuming I don't have setup 
> issues) some work needs to be done to fix the warnings before bombarding 
> everybody with failure messages every 30 minutes.
>
> I assume the branch_3x issues are similar, but I haven't looked into it yet.
>
> Steve
>
>> -Original Message-
>> From: Michael McCandless [mailto:luc...@mikemccandless.com]
>> Sent: Sunday, May 29, 2011 8:48 AM
>> To: dev@lucene.apache.org Dev
>> Subject: do the Jenkins while(true) builds now catch javadoc warnings...?
>>
>> I know we discussed that this was an easy fix to the shell script...
>> but did anyone make the change?
>>
>> Ie, so that we catch JS warnings in the while(true) build, not just
>> nightly/POM builds.
>>
>> Mike
>>
>> http://blog.mikemccandless.com
>>
>> -
>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
>> For additional commands, e-mail: dev-h...@lucene.apache.org
>
>

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: [VOTE] Release Lucene/Solr 3.2.0

[Mark Miller]

So I've looked over things fairly closely now. I saw a variety of things I'm 
not very concerned about that I dropped into IRC.

The only thing I am concerned about is that Solr does not use VERSION_32 in the 
example and so does not use the new Tiered
flush policy. I do think that's worth a respin (and wish I was more up to speed 
on signing and whats up with my keys and stuff so I could help out without as 
much investment as it would currently take).


- Mark


On May 27, 2011, at 11:50 AM, Michael McCandless wrote:

> Please vote to release the artifacts at:
> 
>  http://people.apache.org/~mikemccand/lucene_solr_320/rc1/
> 
> as Lucene 3.2.0 and Solr 3.2.0.
> 
> Mike
> 
> http://blog.mikemccandless.com
> 
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org
> 

- Mark Miller
lucidimagination.com





-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

RE: do the Jenkins while(true) builds now catch javadoc warnings...?

[Steven A Rowe]

I checked what it would take to test all of the javadocs on trunk, and found 
that there is no one target to call.  To cover all javadocs:

cd lucene ; ant javadocs-all javadocs-test-framework
cd ../modules ; ant javadocs
cd ../solr ; ant javadoc-all javadoc-test-framework

Ideally, the javadocs would be generated once under 1.5 and again under 1.6, 
for all of branch_3x, and for lucene/ & modules/ on trunk.

I got a bunch of javadoc warnings in Solr, so (assuming I don't have setup 
issues) some work needs to be done to fix the warnings before bombarding 
everybody with failure messages every 30 minutes.

I assume the branch_3x issues are similar, but I haven't looked into it yet.

Steve

> -Original Message-
> From: Michael McCandless [mailto:luc...@mikemccandless.com]
> Sent: Sunday, May 29, 2011 8:48 AM
> To: dev@lucene.apache.org Dev
> Subject: do the Jenkins while(true) builds now catch javadoc warnings...?
> 
> I know we discussed that this was an easy fix to the shell script...
> but did anyone make the change?
> 
> Ie, so that we catch JS warnings in the while(true) build, not just
> nightly/POM builds.
> 
> Mike
> 
> http://blog.mikemccandless.com
> 
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-3147) MockDirectoryWrapper should track open file handles of IndexOutput too

[Shai Erera (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-3147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13040805#comment-13040805
 ] 

Shai Erera commented on LUCENE-3147:


Also, why do we call checkpoint() only if suppressWarnings=true and not only if 
anyChanges=true? Why do we care about suppressWarnings or not? From what I 
read, the two are equivalent in that context, b/c drop=!suppressWarnings which 
means anyChanges=false whenever drop=true ...

It's weird about MLFWrapper - when I 'svn stat' it appears w/ "A  +" and not 
"?". Yet "svn diff MLFW" yields no diff ... If I "svn ci", I see that it's 
included in the list of files ... strange.

> MockDirectoryWrapper should track open file handles of IndexOutput too
> --
>
> Key: LUCENE-3147
> URL: https://issues.apache.org/jira/browse/LUCENE-3147
> Project: Lucene - Java
>  Issue Type: Test
>  Components: general/test
>Reporter: Shai Erera
>Assignee: Shai Erera
>Priority: Minor
> Fix For: 3.2, 4.0
>
> Attachments: LUCENE-3147-3x.patch, LUCENE-3147.patch, 
> LUCENE-3147.patch, LUCENE-3147.patch, LUCENE-3147.patch, LUCENE-3147.patch, 
> LUCENE-3147.patch, LUCENE-3147.patch, LUCENE-3147.patch, LUCENE-3147.patch, 
> LUCENE-3147.patch, LUCENE-3147.patch, LUCENE-3147.patch, LUCENE-3147.patch
>
>
> MockDirectoryWrapper currently tracks open file handles of IndexInput only. 
> Therefore IO files that are not closed do not fail our tests, which can then 
> lead to test directories fail to delete on Windows. We should make sure all 
> open files are tracked and if they are left open, fail the test. I'll attach 
> a patch shortly.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Keysigning @ Berlin Buzzwords

[Thomas Koch]

From my blog:


Whenever you download some software from the Apache Software Foundation, there 
is a small .asc file besides every software release. This tiny file is a 
cryptographic signature to protect you from accidentally downloading and 
running malware.
However this signature is only as good as the web-of-trust between you and the 
Release Manager of the Software project.
At BerlinBuzzWords many developers and users will gather at one place. This is 
a unique occasion to strengthen the web-of-trust. Therefor we want to 
encourage and remind you to use the occassion for keysigning! For lack of time 
there won't be an official keysigning party, but you have occassion during 
lunch, coffee break or the barbecue to do quick one-to-one keysignings.
You should bring print-outs of your key fingerprints and identification 
documents. If you put the fingerprints in your batch (see picture) everyone 
can easily see
them and ask you for a quick trust exchange. We may also mark a "keysigning 
corner" somewhere where people can go and meet others for key signing.

Hints:

* under Debian (and Ubuntu) you can get a PDF to print out your 
fingerprints many times via:
  sudo aptitude install signing-party ghostscript 
  gpg-key2ps KEYID | ps2pdf - fingerprints.pdf
* the signing-party package also contains the tool "caff" to batch process 
the signing of keys after the event
* please only use keys with at least 2024 bits!
* please make sure that you've uploaded your key to a public keyserver
* I'm happy to answer all remaining questions at the BarCamp on Sunday or 
during the event

Best regards,

Thomas Koch, http://www.koch.ro

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-2519) Improve the defaults for the "text" field type in default schema.xml

[Michael McCandless (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-2519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael McCandless updated SOLR-2519:
-

Fix Version/s: (was: 3.2)
   3.3

> Improve the defaults for the "text" field type in default schema.xml
> 
>
> Key: SOLR-2519
> URL: https://issues.apache.org/jira/browse/SOLR-2519
> Project: Solr
>  Issue Type: Bug
>Reporter: Michael McCandless
>Assignee: Michael McCandless
> Fix For: 3.3, 4.0
>
> Attachments: SOLR-2519.patch, SOLR-2519.patch, SOLR-2519.patch, 
> SOLR-2519.patch
>
>
> Spinoff from: http://lucene.markmail.org/thread/ww6mhfi3rfpngmc5
> The text fieldType in schema.xml is unusable for non-whitespace
> languages, because it has the dangerous auto-phrase feature (of
> Lucene's QP -- see LUCENE-2458) enabled.
> Lucene leaves this off by default, as does ElasticSearch
> (http://http://www.elasticsearch.org/).
> Furthermore, the "text" fieldType uses WhitespaceTokenizer when
> StandardTokenizer is a better cross-language default.
> Until we have language specific field types, I think we should fix
> the "text" fieldType to work well for all languages, by:
>   * Switching from WhitespaceTokenizer to StandardTokenizer
>   * Turning off auto-phrase

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-3140) Backport FSTs to 3.x

[Michael McCandless (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-3140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael McCandless updated LUCENE-3140:
---

Fix Version/s: (was: 3.2)
   3.3

> Backport FSTs to 3.x
> 
>
> Key: LUCENE-3140
> URL: https://issues.apache.org/jira/browse/LUCENE-3140
> Project: Lucene - Java
>  Issue Type: Improvement
>Reporter: Michael McCandless
>Assignee: Michael McCandless
> Fix For: 3.3, 4.0
>
> Attachments: LUCENE-3140.patch, LUCENE-3140.patch
>
>


--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (LUCENE-3140) Backport FSTs to 3.x

[Michael McCandless (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-3140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael McCandless resolved LUCENE-3140.


   Resolution: Fixed
Fix Version/s: 4.0

> Backport FSTs to 3.x
> 
>
> Key: LUCENE-3140
> URL: https://issues.apache.org/jira/browse/LUCENE-3140
> Project: Lucene - Java
>  Issue Type: Improvement
>Reporter: Michael McCandless
>Assignee: Michael McCandless
> Fix For: 3.2, 4.0
>
> Attachments: LUCENE-3140.patch, LUCENE-3140.patch
>
>


--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

do the Jenkins while(true) builds now catch javadoc warnings...?

[Michael McCandless]

I know we discussed that this was an easy fix to the shell script...
but did anyone make the change?

Ie, so that we catch JS warnings in the while(true) build, not just
nightly/POM builds.

Mike

http://blog.mikemccandless.com

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-152) [PATCH] KStem for Lucene

[Michael McCandless (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael McCandless updated LUCENE-152:
--

Fix Version/s: 4.0
   3.3

Setting fix version... seems like there's alot of interest for this stemmer and 
Lucid's improvements to it.

> [PATCH] KStem for Lucene
> 
>
> Key: LUCENE-152
> URL: https://issues.apache.org/jira/browse/LUCENE-152
> Project: Lucene - Java
>  Issue Type: Improvement
>  Components: modules/analysis
>Affects Versions: unspecified
> Environment: Operating System: other
> Platform: Other
>Reporter: Otis Gospodnetic
>Priority: Minor
> Fix For: 3.3, 4.0
>
>
> September 10th 2003 contributionn from "Sergio Guzman-Lara" 
> 
> Original email:
> Hi all,
>   I have ported the kstem stemmer to Java and incorporated it to 
> Lucene. You can get the source code (Kstem.jar) from the following website:
> http://ciir.cs.umass.edu/downloads/
>   Just click on "KStem Java Implementation" (you will need to register 
> your e-mail, for free of course, with the CIIR --Center for Intelligent 
> Information Retrieval, UMass -- and get an access code).
> Content of Kstem.jar:
> java/org/apache/lucene/analysis/KStemData1.java
> java/org/apache/lucene/analysis/KStemData2.java
> java/org/apache/lucene/analysis/KStemData3.java
> java/org/apache/lucene/analysis/KStemData4.java
> java/org/apache/lucene/analysis/KStemData5.java
> java/org/apache/lucene/analysis/KStemData6.java
> java/org/apache/lucene/analysis/KStemData7.java
> java/org/apache/lucene/analysis/KStemData8.java
> java/org/apache/lucene/analysis/KStemFilter.java
> java/org/apache/lucene/analysis/KStemmer.java
> KStemData1.java, ..., KStemData8.java   Contain several lists of words 
> used by Kstem
> KStemmer.java  Implements the Kstem algorithm 
> KStemFilter.java Extends TokenFilter applying Kstem
> To compile
> unjar the file Kstem.jar to Lucene's "src" directory, and compile it 
> there. 
> What is Kstem?
>   A stemmer designed by Bob Krovetz (for more information see 
> http://ciir.cs.umass.edu/pubfiles/ir-35.pdf). 
> Copyright issues
>   This is open source. The actual license agreement is included at the 
> top of every source file.
>  Any comments/questions/suggestions are welcome,
>   Sergio Guzman-Lara
>   Senior Research Fellow
>   CIIR UMass

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-2524) Adding grouping to Solr 3x

[Michael McCandless (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-2524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13040798#comment-13040798
 ] 

Michael McCandless commented on SOLR-2524:
--

bq. I was using this patch last week and didn't encounter problems.

Excellent -- thank you for testing :)

bq. Why is it actually blocked by LUCENE-3099, vs. merely stands to benefit 
from improvements there when it happens?

Well... Hoss is (rightly) nervous that we have divergent grouping
impls in trunk vs 3.x.

There are non-trivial differences... 3.x improves on trunk's impl:

  * Optionally uses CachingCollector, so certain queries will be
substantially faster (adds new request param
group.cache.maxSizeMB)

  * Optionally it's able to get total count of number of unique
groups (using AllGroupsCollector).

But, also, 3.x uses way more RAM than trunk's impl when grouping by a
non-term field, because the grouping module currently always pulls a
StringIndex (but LUCENE-3099 will fix this!).

Hoss also wanted to see all of Solr trunk's grouping tests backported,
and that is now done (thanks Martijn).

We could simply commit anyway, for 3.3, on the strong expectation that
we are going to cut trunk over to the grouping module before shipping
4.0...?  Any objections?  This way 3.3 will have grouping, and we let
the feature "bake" on Jenkins daily testing instead of aging as a patch...


> Adding grouping to Solr 3x
> --
>
> Key: SOLR-2524
> URL: https://issues.apache.org/jira/browse/SOLR-2524
> Project: Solr
>  Issue Type: New Feature
>Reporter: Martijn van Groningen
>Assignee: Michael McCandless
> Fix For: 3.3
>
> Attachments: SOLR-2524.patch, SOLR-2524.patch, SOLR-2524.patch, 
> SOLR-2524.patch, SOLR-2524.patch
>
>
> Grouping was recently added to Lucene 3x. See LUCENE-1421 for more 
> information.
> I think it would be nice if we expose this functionality also to the Solr 
> users that are bound to a 3.x version.
> The grouping feature added to Lucene is currently a subset of the 
> functionality that Solr 4.0-trunk offers. Mainly it doesn't support grouping 
> by function / query.
> The work involved getting the grouping contrib to work on Solr 3x is 
> acceptable. I have it more or less running here. It supports the response 
> format and request parameters (expect: group.query and group.func) described 
> in the FieldCollapse page on the Solr wiki.
> I think it would be great if this is included in the Solr 3.2 release. Many 
> people are using grouping as patch now and this would help them a lot. Any 
> thoughts?

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-3140) Backport FSTs to 3.x

[Michael McCandless (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-3140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13040797#comment-13040797
 ] 

Michael McCandless commented on LUCENE-3140:


Committed to 3.x.  I still need to move fst under oal.util for trunk...

> Backport FSTs to 3.x
> 
>
> Key: LUCENE-3140
> URL: https://issues.apache.org/jira/browse/LUCENE-3140
> Project: Lucene - Java
>  Issue Type: Improvement
>Reporter: Michael McCandless
>Assignee: Michael McCandless
> Fix For: 3.2
>
> Attachments: LUCENE-3140.patch, LUCENE-3140.patch
>
>


--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-2762) Don't leak deleted open file handles with pooled readers

[Michael McCandless (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-2762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13040795#comment-13040795
 ] 

Michael McCandless commented on LUCENE-2762:


bq. Could this be related to all the leaked handles we fixed at LUCENE-3147?

I think that's unlikely?  Ie, the leaks we are fixing in LUCENE-3147 are all 
cases where an exception was hit at some point?

Unless: Josef, are you regularly hitting exceptions somewhere?

> Don't leak deleted open file handles with pooled readers
> 
>
> Key: LUCENE-2762
> URL: https://issues.apache.org/jira/browse/LUCENE-2762
> Project: Lucene - Java
>  Issue Type: Bug
>Affects Versions: 2.9.4, 3.0.3, 3.1, 4.0
>Reporter: Michael McCandless
>Assignee: Michael McCandless
> Fix For: 2.9.4, 3.0.3, 3.1, 4.0
>
> Attachments: LUCENE-2762.patch
>
>
> If you have CFS enabled today, and pooling is enabled (either directly
> or because you've pulled an NRT reader), IndexWriter will hold open
> SegmentReaders against the non-CFS format of each merged segment.
> So even if you close all NRT readers you've pulled from the writer,
> you'll still see file handles open against files that have been
> deleted.
> This count will not grow unbounded, since it's limited by the number
> of segments in the index, but it's still a serious problem since the
> app had turned off CFS in the first place presumably to avoid risk of
> too-many-open-files.  It's also bad because it ties up disk space
> since these files would otherwise be deleted.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-3147) MockDirectoryWrapper should track open file handles of IndexOutput too

[Michael McCandless (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-3147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13040794#comment-13040794
 ] 

Michael McCandless commented on LUCENE-3147:


Nice catch in closeMergeReaders -- the changes there look great.  Much less 
code dup now :)

You need to "svn add" MockLockFWWrapper.java.

> MockDirectoryWrapper should track open file handles of IndexOutput too
> --
>
> Key: LUCENE-3147
> URL: https://issues.apache.org/jira/browse/LUCENE-3147
> Project: Lucene - Java
>  Issue Type: Test
>  Components: general/test
>Reporter: Shai Erera
>Assignee: Shai Erera
>Priority: Minor
> Fix For: 3.2, 4.0
>
> Attachments: LUCENE-3147-3x.patch, LUCENE-3147.patch, 
> LUCENE-3147.patch, LUCENE-3147.patch, LUCENE-3147.patch, LUCENE-3147.patch, 
> LUCENE-3147.patch, LUCENE-3147.patch, LUCENE-3147.patch, LUCENE-3147.patch, 
> LUCENE-3147.patch, LUCENE-3147.patch, LUCENE-3147.patch, LUCENE-3147.patch
>
>
> MockDirectoryWrapper currently tracks open file handles of IndexInput only. 
> Therefore IO files that are not closed do not fail our tests, which can then 
> lead to test directories fail to delete on Windows. We should make sure all 
> open files are tracked and if they are left open, fail the test. I'll attach 
> a patch shortly.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-3147) MockDirectoryWrapper should track open file handles of IndexOutput too

[Shai Erera (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-3147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shai Erera updated LUCENE-3147:
---

Attachment: LUCENE-3147-3x.patch

Changes backported from trunk.

I think I found another leak in IndexWriter.closeMergeReaders (not fixed on 
trunk yet) -- if suppressExceptions was 'false', it could fail on a release() / 
close() attempt w/o releasing/closing all of the pooled readers.

I modified the method to remove a lot of code duplication, so I'd appreciate a 
second eye on it.

All tests pass (at least so far). So if there are no objections, I'll commit to 
3x and port the IW fix to trunk as well.

> MockDirectoryWrapper should track open file handles of IndexOutput too
> --
>
> Key: LUCENE-3147
> URL: https://issues.apache.org/jira/browse/LUCENE-3147
> Project: Lucene - Java
>  Issue Type: Test
>  Components: general/test
>Reporter: Shai Erera
>Assignee: Shai Erera
>Priority: Minor
> Fix For: 3.2, 4.0
>
> Attachments: LUCENE-3147-3x.patch, LUCENE-3147.patch, 
> LUCENE-3147.patch, LUCENE-3147.patch, LUCENE-3147.patch, LUCENE-3147.patch, 
> LUCENE-3147.patch, LUCENE-3147.patch, LUCENE-3147.patch, LUCENE-3147.patch, 
> LUCENE-3147.patch, LUCENE-3147.patch, LUCENE-3147.patch, LUCENE-3147.patch
>
>
> MockDirectoryWrapper currently tracks open file handles of IndexInput only. 
> Therefore IO files that are not closed do not fail our tests, which can then 
> lead to test directories fail to delete on Windows. We should make sure all 
> open files are tracked and if they are left open, fail the test. I'll attach 
> a patch shortly.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-3150) Wherever we catch & suppress Throwable we should not suppress ThreadInterruptedException

[Shai Erera (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-3150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13040791#comment-13040791
 ] 

Shai Erera commented on LUCENE-3150:


We can fix IOUtils to do that -- now that we call it from many places in the 
code, it will catch a lot of instances. Also, perhaps we could have an 
Abortable interface and then call IOUtils from all places that impl abort() 
today.

> Wherever we catch & suppress Throwable we should not suppress 
> ThreadInterruptedException
> 
>
> Key: LUCENE-3150
> URL: https://issues.apache.org/jira/browse/LUCENE-3150
> Project: Lucene - Java
>  Issue Type: Bug
>Reporter: Michael McCandless
>Priority: Minor
> Fix For: 3.3, 4.0
>
>
> In various places we catch Throwable and suppress it, usually in exception 
> handlers where we want to just throw the first exc we had hit.
> But this is dangerous for a thread interrupt since it means we can swallow & 
> ignore the interrupt.
> We should at least catch the interrupt & restore the interrupt bit, if we 
> can't rethrow it.
> One example is in SegmentInfos where we write the segments.gen file... there 
> are many other examples in SegmentInfos too.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-2762) Don't leak deleted open file handles with pooled readers

[Shai Erera (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-2762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13040789#comment-13040789
 ] 

Shai Erera commented on LUCENE-2762:


Could this be related to all the leaked handles we fixed at LUCENE-3147?

> Don't leak deleted open file handles with pooled readers
> 
>
> Key: LUCENE-2762
> URL: https://issues.apache.org/jira/browse/LUCENE-2762
> Project: Lucene - Java
>  Issue Type: Bug
>Affects Versions: 2.9.4, 3.0.3, 3.1, 4.0
>Reporter: Michael McCandless
>Assignee: Michael McCandless
> Fix For: 2.9.4, 3.0.3, 3.1, 4.0
>
> Attachments: LUCENE-2762.patch
>
>
> If you have CFS enabled today, and pooling is enabled (either directly
> or because you've pulled an NRT reader), IndexWriter will hold open
> SegmentReaders against the non-CFS format of each merged segment.
> So even if you close all NRT readers you've pulled from the writer,
> you'll still see file handles open against files that have been
> deleted.
> This count will not grow unbounded, since it's limited by the number
> of segments in the index, but it's still a serious problem since the
> app had turned off CFS in the first place presumably to avoid risk of
> too-many-open-files.  It's also bad because it ties up disk space
> since these files would otherwise be deleted.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS-MAVEN] Lucene-Solr-Maven-3.x #137: POMs out of sync

[Apache Jenkins Server]

Build: https://builds.apache.org/hudson/job/Lucene-Solr-Maven-3.x/137/

1 tests failed.
FAILED:  org.apache.lucene.index.TestCheckIndex.testLuceneConstantVersion

Error Message:
Invalid version: 3.2-SNAPSHOT

Stack Trace:
java.lang.AssertionError: Invalid version: 3.2-SNAPSHOT
at org.junit.Assert.fail(Assert.java:91)
at org.junit.Assert.assertTrue(Assert.java:43)
at 
org.apache.lucene.index.TestCheckIndex.testLuceneConstantVersion(TestCheckIndex.java:98)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:616)
at 
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
at 
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
at 
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
at 
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
at org.junit.rules.TestWatchman$1.evaluate(TestWatchman.java:48)
at 
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
at 
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:31)
at 
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:76)
at 
org.apache.lucene.util.LuceneTestCase$LuceneTestCaseRunner.runChild(LuceneTestCase.java:1206)
at 
org.apache.lucene.util.LuceneTestCase$LuceneTestCaseRunner.runChild(LuceneTestCase.java:1124)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:193)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:52)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:191)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:42)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:184)
at 
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
at 
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:31)
at org.junit.runners.ParentRunner.run(ParentRunner.java:236)
at 
org.apache.maven.surefire.junit4.JUnit4TestSet.execute(JUnit4TestSet.java:35)
at 
org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:146)
at 
org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:97)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:616)
at 
org.apache.maven.surefire.booter.ProviderFactory$ClassLoaderProxy.invoke(ProviderFactory.java:103)
at $Proxy0.invoke(Unknown Source)
at 
org.apache.maven.surefire.booter.SurefireStarter.invokeProvider(SurefireStarter.java:145)
at 
org.apache.maven.surefire.booter.SurefireStarter.runSuitesInProcess(SurefireStarter.java:87)
at 
org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:69)




Build Log (for compile errors):
[...truncated 15085 lines...]



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (SOLR-2519) Improve the defaults for the "text" field type in default schema.xml

[Michael McCandless (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-2519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael McCandless resolved SOLR-2519.
--

Resolution: Fixed

Thanks everyone!

Robert, can you open followon issues for your other ideas?  I think they are 
important... we should put forth the best defaults Lucene/Solr has to offer 
here.

> Improve the defaults for the "text" field type in default schema.xml
> 
>
> Key: SOLR-2519
> URL: https://issues.apache.org/jira/browse/SOLR-2519
> Project: Solr
>  Issue Type: Bug
>Reporter: Michael McCandless
>Assignee: Michael McCandless
> Fix For: 3.2, 4.0
>
> Attachments: SOLR-2519.patch, SOLR-2519.patch, SOLR-2519.patch, 
> SOLR-2519.patch
>
>
> Spinoff from: http://lucene.markmail.org/thread/ww6mhfi3rfpngmc5
> The text fieldType in schema.xml is unusable for non-whitespace
> languages, because it has the dangerous auto-phrase feature (of
> Lucene's QP -- see LUCENE-2458) enabled.
> Lucene leaves this off by default, as does ElasticSearch
> (http://http://www.elasticsearch.org/).
> Furthermore, the "text" fieldType uses WhitespaceTokenizer when
> StandardTokenizer is a better cross-language default.
> Until we have language specific field types, I think we should fix
> the "text" fieldType to work well for all languages, by:
>   * Switching from WhitespaceTokenizer to StandardTokenizer
>   * Turning off auto-phrase

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Welcome Chris Male & Andi Vajda as full Solr / Lucene Committers

[Doron Cohen]

Welcome!

On Sat, May 28, 2011 at 10:39 PM, Erick Erickson wrote:

> Congrats!
>
> On Mon, May 23, 2011 at 7:03 PM, Chris Male  wrote:
> > Thanks everybody.
> >
> > On Tue, May 24, 2011 at 5:20 AM, Shai Erera  wrote:
> >>
> >> Welcome!
> >>
> >> Shai
> >>
> >> On Monday, May 23, 2011, Steven A Rowe  wrote:
> >> > Welcome!
> >> >
> >> >> -Original Message-
> >> >> From: Simon Willnauer [mailto:simon.willna...@googlemail.com]
> >> >> Sent: Monday, May 23, 2011 12:40 PM
> >> >> To: dev@lucene.apache.org; gene...@lucene.apache.org
> >> >> Subject: Welcome Chris Male & Andi Vajda as full Solr / Lucene
> >> >> Committers
> >> >>
> >> >> Hi folks,
> >> >>
> >> >> I am happy to announce that the Lucene PMC has accepted Chris Male
> and
> >> >> Andi Vajda as Lucene/Solr committers.
> >> >>
> >> >> Congratulations & Welcome on board, Chris & Andi!!
> >> >>
> >> >> Simon
> >> >>
> >> >> -
> >> >> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> >> >> For additional commands, e-mail: dev-h...@lucene.apache.org
> >> >
> >> >
> >>
> >> -
> >> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> >> For additional commands, e-mail: dev-h...@lucene.apache.org
> >>
> >
> >
> >
> > --
> > Chris Male | Software Developer | JTeam BV.| www.jteam.nl
> >
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org
>
>

[jira] [Commented] (LUCENE-3147) MockDirectoryWrapper should track open file handles of IndexOutput too

[Michael McCandless (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-3147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13040774#comment-13040774
 ] 

Michael McCandless commented on LUCENE-3147:


{quote}
bq. But: why do we ever "abort" a full flush...? It looks like it happens 
because one of the DWPTs hits an exc while flushing...?

I think this is the way to go here. if one DWPT can not flush why would be 
continue flushing the remaining ones. You will need to call rollback anyway or 
reindex all documents since the last commit. Its unnecessary to flush those 
DWPT since we are already lost the failed segment.
{quote}

In theory, if disk space freed up, the other DWPTs could flush?  Ie the failure 
of one DWPT to flush should only "have to" above that one DWPT?

But I agree, I think it's OK if we leave it this way for now... you're right 
that the app is going to have to do its own "recovery" anyway.  Just feels like 
we shouldn't be discarding healthy DWPTs... but I think it's minor.

> MockDirectoryWrapper should track open file handles of IndexOutput too
> --
>
> Key: LUCENE-3147
> URL: https://issues.apache.org/jira/browse/LUCENE-3147
> Project: Lucene - Java
>  Issue Type: Test
>  Components: general/test
>Reporter: Shai Erera
>Assignee: Shai Erera
>Priority: Minor
> Fix For: 3.2, 4.0
>
> Attachments: LUCENE-3147.patch, LUCENE-3147.patch, LUCENE-3147.patch, 
> LUCENE-3147.patch, LUCENE-3147.patch, LUCENE-3147.patch, LUCENE-3147.patch, 
> LUCENE-3147.patch, LUCENE-3147.patch, LUCENE-3147.patch, LUCENE-3147.patch, 
> LUCENE-3147.patch, LUCENE-3147.patch
>
>
> MockDirectoryWrapper currently tracks open file handles of IndexInput only. 
> Therefore IO files that are not closed do not fail our tests, which can then 
> lead to test directories fail to delete on Windows. We should make sure all 
> open files are tracked and if they are left open, fail the test. I'll attach 
> a patch shortly.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-2519) Improve the defaults for the "text" field type in default schema.xml

[Michael McCandless (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-2519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael McCandless updated SOLR-2519:
-

Attachment: SOLR-2519.patch

New patch attached.  Only change vs last patch is to remove _example from the 
field types, so now we have text_general, text_en, etc.

I think for the other fun improvements (less aggressive stemming, alternatives 
to stop word removal, better default query parser) we should open new issues?  
Progress not perfection...

I'll commit this one shortly.

> Improve the defaults for the "text" field type in default schema.xml
> 
>
> Key: SOLR-2519
> URL: https://issues.apache.org/jira/browse/SOLR-2519
> Project: Solr
>  Issue Type: Bug
>Reporter: Michael McCandless
>Assignee: Michael McCandless
> Fix For: 3.2, 4.0
>
> Attachments: SOLR-2519.patch, SOLR-2519.patch, SOLR-2519.patch, 
> SOLR-2519.patch
>
>
> Spinoff from: http://lucene.markmail.org/thread/ww6mhfi3rfpngmc5
> The text fieldType in schema.xml is unusable for non-whitespace
> languages, because it has the dangerous auto-phrase feature (of
> Lucene's QP -- see LUCENE-2458) enabled.
> Lucene leaves this off by default, as does ElasticSearch
> (http://http://www.elasticsearch.org/).
> Furthermore, the "text" fieldType uses WhitespaceTokenizer when
> StandardTokenizer is a better cross-language default.
> Until we have language specific field types, I think we should fix
> the "text" fieldType to work well for all languages, by:
>   * Switching from WhitespaceTokenizer to StandardTokenizer
>   * Turning off auto-phrase

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (LUCENE-2834) don't spawn thread statically in FSDirectory on Mac OS X

[Michael McCandless (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-2834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael McCandless resolved LUCENE-2834.


Resolution: Fixed

> don't spawn thread statically in FSDirectory on Mac OS X
> 
>
> Key: LUCENE-2834
> URL: https://issues.apache.org/jira/browse/LUCENE-2834
> Project: Lucene - Java
>  Issue Type: Bug
>Reporter: Robert Muir
>Assignee: Michael McCandless
> Fix For: 3.3, 4.0
>
> Attachments: LUCENE-2834.patch, LUCENE-2834.patch, LUCENE-2834.patch, 
> LUCENE-2834.patch
>
>
> on the Mac, creating the digester starts up a PKCS11 thread.
> I do not think threads should be created statically (I have this same issue 
> with TimeLimitedCollector and also FilterManager).
> Uwe discussed removing this md5 digester, I don't care if we remove it or 
> not, just as long as it doesn't create a thread,
> and just as long as it doesn't use the system default locale.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-2834) don't spawn thread statically in FSDirectory on Mac OS X

[Michael McCandless (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-2834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael McCandless updated LUCENE-2834:
---

Fix Version/s: 3.3
 Assignee: Michael McCandless

> don't spawn thread statically in FSDirectory on Mac OS X
> 
>
> Key: LUCENE-2834
> URL: https://issues.apache.org/jira/browse/LUCENE-2834
> Project: Lucene - Java
>  Issue Type: Bug
>Reporter: Robert Muir
>Assignee: Michael McCandless
> Fix For: 3.3, 4.0
>
> Attachments: LUCENE-2834.patch, LUCENE-2834.patch, LUCENE-2834.patch, 
> LUCENE-2834.patch
>
>
> on the Mac, creating the digester starts up a PKCS11 thread.
> I do not think threads should be created statically (I have this same issue 
> with TimeLimitedCollector and also FilterManager).
> Uwe discussed removing this md5 digester, I don't care if we remove it or 
> not, just as long as it doesn't create a thread,
> and just as long as it doesn't use the system default locale.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: [VOTE] Release Lucene/Solr 3.2.0

[Robert Muir]

On Sun, May 29, 2011 at 3:27 AM, Simon Willnauer
 wrote:
> and we
> the lucene PMC agreed on the fact that a release doesn't need to have
> large features. We simply don't want to let folks wait for years
> anymore.
>

I think its a lot more than this really... I think we should consider
all of the following things:

* the list of features we have in 3.2 is actually digestible, its
probably bad to wait until we have this huge intimidating list of
changes to release, from casual browsing I don't see huge long lists
from other open source projects.
* if a user contributes something, I think its really important to
turn that around into a release faster, its going to encourage them to
contribute again. Sure, to those of us in the choir we are ok with
contributing stuff knowing its moving the project along long-term, but
I think for more casual contributors turn-around is really
important... they are most likely using releases not SVN and are
interested in when their contribution will be in a release.
* we have a growing list of committers and activity on the project and
I think speeding up the release cycle reflects our productivity level,
I think the list of features in 3.2 is quite nice and will benefit a
pretty large percentage of the userbase.
* we are talking about issuing off our stable branch, which has the
benefit of getting backports of things that have baked for a while in
trunk (e.g. TieredMP baked before being backported). We take the time
to maintain this stable branch, we should exploit it for all its worth
and get safe features out there.
* we are way more aggressive about bugs lately, its not just
patch+test, but instead, identify the general problem, fix it across
the board, and share the same mechanism with users to test their own
code (e.g. https://issues.apache.org/jira/browse/LUCENE-3147 and
https://issues.apache.org/jira/browse/LUCENE-3113 are some recent
examples of this)
* we did all this infra work to make releasing faster... its so much
better now, I can list all of the things I've done to validate/fixup
releases before (test under different locales, fix javadocs warnings,
license problems), and all of this stuff is automatic now.

So these are some of the reasons why I think it makes perfect sense to
try to crank out releases from our stable branch that are incremental
and evolutionary versus revolutionary.

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: [VOTE] Release Lucene/Solr 3.2.0

[Simon Willnauer]

Hi David,

On Sat, May 28, 2011 at 11:14 PM, David Smiley (@MITRE.org)
 wrote:
> You're right; it shouldn't be shoved in at the last second -- I didn't mean
> to imply that. It should be committed and then we'll give it a comfortable
> amount of time.  When that time is up, and if v3.2 waits for it, then 3.2
> will probably be at about the 3-month mark since the last release -- in my
> view (and Grant's) the ideal suite-spot for a release window.
>
> Releasing too fast / too infrequent aside, I think a X.X release should have
> a notable feature (or a huge performance improvement), and 3.2 doesn't
> without result grouping, IMO. It's got plumbing and bug fixes.

I can understand you point of view but I can't recall any release
without at least one person within the same situation.
We promised to spinn more frequent releases and we are trying to make
our best effort to do so. In order to eventually succeed with what we
have planned we need to make a start and get it going. Once we got the
release process running frequently and in predictable intervals folks
don't need to be in the situation you are in right now anymore. If
feature X somebody needs to have can not be in a certain release
because its pretty new and needs some time to bake its not going to be
a problem anymore since it will be in the next one which is reasonably
close. Let me tell you I have tons of features I would love to have in
4.0 but some of them might not make it for various reasons.
Yet, to get there it needs to be kicked off and I am super happy that
mike is doing it. We are changing lots of things here in the process
and we are expecting to step on peoples foot once in a while but look
at the amount of users this project has, you will always step on
somebody's foot, right?
As Robert pointed out this release has a couple of improvements and we
the lucene PMC agreed on the fact that a release doesn't need to have
large features. We simply don't want to let folks wait for years
anymore.

I appreciate that you state your opinion on the grouping feature but
to be honest this has been developed within a couple of weeks with
great progress and lots of passion but I think its not ready to be
released and therefore it should not block it.

simon


>
> ~ David Smiley
>
> -
>  Author: https://www.packtpub.com/solr-1-4-enterprise-search-server/book
> --
> View this message in context: 
> http://lucene.472066.n3.nabble.com/VOTE-Release-Lucene-Solr-3-2-0-tp2993434p2997968.html
> Sent from the Lucene - Java Developer mailing list archive at Nabble.com.
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org
>
>

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-tests-only-trunk - Build # 8479 - Still Failing

[Apache Jenkins Server]

Build: https://builds.apache.org/hudson/job/Lucene-Solr-tests-only-trunk/8479/

No tests ran.

Build Log (for compile errors):
[...truncated 36 lines...]



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org