[jira] [Updated] (LUCENE-5207) lucene expressions module

[Robert Muir (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-5207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Robert Muir updated LUCENE-5207:


Attachment: LUCENE-5207.patch

Here's an applyable patch for review showing differences between branch and 
trunk.

I excluded generated code, so run 'ant regenerate' first.

> lucene expressions module
> -
>
> Key: LUCENE-5207
> URL: https://issues.apache.org/jira/browse/LUCENE-5207
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Ryan Ernst
> Attachments: LUCENE-5207.patch, LUCENE-5207.patch
>
>
> Expressions are geared at defining an alternative ranking function (e.g. 
> incorporating the text relevance score and other field values/ranking
> signals). So they are conceptually much more like ElasticSearch's scripting 
> support (http://www.elasticsearch.org/guide/reference/modules/scripting/) 
> than solr's function queries.
> Some additional notes:
> * In addition to referring to other fields, they can also refer to other 
> expressions, so they can be used as "computed fields".
> * You can rank documents easily by multiple expressions (its a SortField at 
> the end), e.g. Sort by year descending, then some function of score price and 
> time ascending.
> * The provided javascript expression syntax is much more efficient than using 
> a scripting engine, because it does not have dynamic typing (compiles to 
> .class files that work on doubles). Performance is similar to writing a 
> custom FieldComparator yourself, but much easier to do.
> * We have solr integration to contribute in the future, but this is just the 
> standalone lucene part as a start. Since lucene has no schema, it includes an 
> implementation of Bindings (SimpleBindings) that maps variable names to 
> SortField's or other expressions.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5207) lucene expressions module

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-5207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13767674#comment-13767674
 ] 

ASF subversion and git services commented on LUCENE-5207:
-

Commit 1523395 from [~rcmuir] in branch 'dev/branches/lucene5207'
[ https://svn.apache.org/r1523395 ]

LUCENE-5207: didnt mean to change unrelated test...

> lucene expressions module
> -
>
> Key: LUCENE-5207
> URL: https://issues.apache.org/jira/browse/LUCENE-5207
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Ryan Ernst
> Attachments: LUCENE-5207.patch
>
>
> Expressions are geared at defining an alternative ranking function (e.g. 
> incorporating the text relevance score and other field values/ranking
> signals). So they are conceptually much more like ElasticSearch's scripting 
> support (http://www.elasticsearch.org/guide/reference/modules/scripting/) 
> than solr's function queries.
> Some additional notes:
> * In addition to referring to other fields, they can also refer to other 
> expressions, so they can be used as "computed fields".
> * You can rank documents easily by multiple expressions (its a SortField at 
> the end), e.g. Sort by year descending, then some function of score price and 
> time ascending.
> * The provided javascript expression syntax is much more efficient than using 
> a scripting engine, because it does not have dynamic typing (compiles to 
> .class files that work on doubles). Performance is similar to writing a 
> custom FieldComparator yourself, but much easier to do.
> * We have solr integration to contribute in the future, but this is just the 
> standalone lucene part as a start. Since lucene has no schema, it includes an 
> implementation of Bindings (SimpleBindings) that maps variable names to 
> SortField's or other expressions.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5207) lucene expressions module

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-5207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13767672#comment-13767672
 ] 

ASF subversion and git services commented on LUCENE-5207:
-

Commit 1523394 from [~rcmuir] in branch 'dev/branches/lucene5207'
[ https://svn.apache.org/r1523394 ]

LUCENE-5207: simplify

> lucene expressions module
> -
>
> Key: LUCENE-5207
> URL: https://issues.apache.org/jira/browse/LUCENE-5207
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Ryan Ernst
> Attachments: LUCENE-5207.patch
>
>
> Expressions are geared at defining an alternative ranking function (e.g. 
> incorporating the text relevance score and other field values/ranking
> signals). So they are conceptually much more like ElasticSearch's scripting 
> support (http://www.elasticsearch.org/guide/reference/modules/scripting/) 
> than solr's function queries.
> Some additional notes:
> * In addition to referring to other fields, they can also refer to other 
> expressions, so they can be used as "computed fields".
> * You can rank documents easily by multiple expressions (its a SortField at 
> the end), e.g. Sort by year descending, then some function of score price and 
> time ascending.
> * The provided javascript expression syntax is much more efficient than using 
> a scripting engine, because it does not have dynamic typing (compiles to 
> .class files that work on doubles). Performance is similar to writing a 
> custom FieldComparator yourself, but much easier to do.
> * We have solr integration to contribute in the future, but this is just the 
> standalone lucene part as a start. Since lucene has no schema, it includes an 
> implementation of Bindings (SimpleBindings) that maps variable names to 
> SortField's or other expressions.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5207) lucene expressions module

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-5207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13767671#comment-13767671
 ] 

ASF subversion and git services commented on LUCENE-5207:
-

Commit 1523393 from [~rcmuir] in branch 'dev/branches/lucene5207'
[ https://svn.apache.org/r1523393 ]

LUCENE-5207: add test

> lucene expressions module
> -
>
> Key: LUCENE-5207
> URL: https://issues.apache.org/jira/browse/LUCENE-5207
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Ryan Ernst
> Attachments: LUCENE-5207.patch
>
>
> Expressions are geared at defining an alternative ranking function (e.g. 
> incorporating the text relevance score and other field values/ranking
> signals). So they are conceptually much more like ElasticSearch's scripting 
> support (http://www.elasticsearch.org/guide/reference/modules/scripting/) 
> than solr's function queries.
> Some additional notes:
> * In addition to referring to other fields, they can also refer to other 
> expressions, so they can be used as "computed fields".
> * You can rank documents easily by multiple expressions (its a SortField at 
> the end), e.g. Sort by year descending, then some function of score price and 
> time ascending.
> * The provided javascript expression syntax is much more efficient than using 
> a scripting engine, because it does not have dynamic typing (compiles to 
> .class files that work on doubles). Performance is similar to writing a 
> custom FieldComparator yourself, but much easier to do.
> * We have solr integration to contribute in the future, but this is just the 
> standalone lucene part as a start. Since lucene has no schema, it includes an 
> implementation of Bindings (SimpleBindings) that maps variable names to 
> SortField's or other expressions.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4984) etc/logging.properties in example should be removed

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-4984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13767668#comment-13767668
 ] 

ASF subversion and git services commented on SOLR-4984:
---

Commit 1523392 from [~markrmil...@gmail.com] in branch 'dev/branches/branch_4x'
[ https://svn.apache.org/r1523392 ]

SOLR-4984: etc/logging.properties in example should be removed

> etc/logging.properties in example should be removed
> ---
>
> Key: SOLR-4984
> URL: https://issues.apache.org/jira/browse/SOLR-4984
> Project: Solr
>  Issue Type: Bug
>Reporter: Ryan Ernst
>Assignee: Mark Miller
>Priority: Minor
> Fix For: 5.0, 4.6
>
> Attachments: SOLR-4984.patch
>
>
> I believe this used to be used when JCL logging was the default backend.  But 
> now with log4j being the backend, I don't think it is used. It should be 
> removed to not clutter the example with unnecessary files.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (SOLR-4984) etc/logging.properties in example should be removed

[Mark Miller (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-4984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mark Miller resolved SOLR-4984.
---

Resolution: Fixed

Thanks Ryan - I never did get the explicit configuration to work - can't get it 
to pick up the sys prop from jetty.xml. It finds it in resource because that 
ends up on the classpath. I've remove the ignored entry from jetty.xml as well.

> etc/logging.properties in example should be removed
> ---
>
> Key: SOLR-4984
> URL: https://issues.apache.org/jira/browse/SOLR-4984
> Project: Solr
>  Issue Type: Bug
>Reporter: Ryan Ernst
>Assignee: Mark Miller
>Priority: Minor
> Fix For: 5.0, 4.6
>
> Attachments: SOLR-4984.patch
>
>
> I believe this used to be used when JCL logging was the default backend.  But 
> now with log4j being the backend, I don't think it is used. It should be 
> removed to not clutter the example with unnecessary files.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4984) etc/logging.properties in example should be removed

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-4984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13767667#comment-13767667
 ] 

ASF subversion and git services commented on SOLR-4984:
---

Commit 1523391 from [~markrmil...@gmail.com] in branch 'dev/trunk'
[ https://svn.apache.org/r1523391 ]

SOLR-4984: etc/logging.properties in example should be removed

> etc/logging.properties in example should be removed
> ---
>
> Key: SOLR-4984
> URL: https://issues.apache.org/jira/browse/SOLR-4984
> Project: Solr
>  Issue Type: Bug
>Reporter: Ryan Ernst
>Assignee: Mark Miller
>Priority: Minor
> Fix For: 5.0, 4.6
>
> Attachments: SOLR-4984.patch
>
>
> I believe this used to be used when JCL logging was the default backend.  But 
> now with log4j being the backend, I don't think it is used. It should be 
> removed to not clutter the example with unnecessary files.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5207) lucene expressions module

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-5207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13767648#comment-13767648
 ] 

ASF subversion and git services commented on LUCENE-5207:
-

Commit 1523385 from [~rcmuir] in branch 'dev/branches/lucene5207'
[ https://svn.apache.org/r1523385 ]

LUCENE-5207: javadocs

> lucene expressions module
> -
>
> Key: LUCENE-5207
> URL: https://issues.apache.org/jira/browse/LUCENE-5207
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Ryan Ernst
> Attachments: LUCENE-5207.patch
>
>
> Expressions are geared at defining an alternative ranking function (e.g. 
> incorporating the text relevance score and other field values/ranking
> signals). So they are conceptually much more like ElasticSearch's scripting 
> support (http://www.elasticsearch.org/guide/reference/modules/scripting/) 
> than solr's function queries.
> Some additional notes:
> * In addition to referring to other fields, they can also refer to other 
> expressions, so they can be used as "computed fields".
> * You can rank documents easily by multiple expressions (its a SortField at 
> the end), e.g. Sort by year descending, then some function of score price and 
> time ascending.
> * The provided javascript expression syntax is much more efficient than using 
> a scripting engine, because it does not have dynamic typing (compiles to 
> .class files that work on doubles). Performance is similar to writing a 
> custom FieldComparator yourself, but much easier to do.
> * We have solr integration to contribute in the future, but this is just the 
> standalone lucene part as a start. Since lucene has no schema, it includes an 
> implementation of Bindings (SimpleBindings) that maps variable names to 
> SortField's or other expressions.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5207) lucene expressions module

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-5207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13767642#comment-13767642
 ] 

ASF subversion and git services commented on LUCENE-5207:
-

Commit 1523382 from [~rcmuir] in branch 'dev/branches/lucene5207'
[ https://svn.apache.org/r1523382 ]

LUCENE-5207: simplify some of operators

> lucene expressions module
> -
>
> Key: LUCENE-5207
> URL: https://issues.apache.org/jira/browse/LUCENE-5207
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Ryan Ernst
> Attachments: LUCENE-5207.patch
>
>
> Expressions are geared at defining an alternative ranking function (e.g. 
> incorporating the text relevance score and other field values/ranking
> signals). So they are conceptually much more like ElasticSearch's scripting 
> support (http://www.elasticsearch.org/guide/reference/modules/scripting/) 
> than solr's function queries.
> Some additional notes:
> * In addition to referring to other fields, they can also refer to other 
> expressions, so they can be used as "computed fields".
> * You can rank documents easily by multiple expressions (its a SortField at 
> the end), e.g. Sort by year descending, then some function of score price and 
> time ascending.
> * The provided javascript expression syntax is much more efficient than using 
> a scripting engine, because it does not have dynamic typing (compiles to 
> .class files that work on doubles). Performance is similar to writing a 
> custom FieldComparator yourself, but much easier to do.
> * We have solr integration to contribute in the future, but this is just the 
> standalone lucene part as a start. Since lucene has no schema, it includes an 
> implementation of Bindings (SimpleBindings) that maps variable names to 
> SortField's or other expressions.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-4984) etc/logging.properties in example should be removed

[Mark Miller (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-4984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mark Miller updated SOLR-4984:
--

 Priority: Minor  (was: Major)
Fix Version/s: 4.6
   5.0

> etc/logging.properties in example should be removed
> ---
>
> Key: SOLR-4984
> URL: https://issues.apache.org/jira/browse/SOLR-4984
> Project: Solr
>  Issue Type: Bug
>Reporter: Ryan Ernst
>Assignee: Mark Miller
>Priority: Minor
> Fix For: 5.0, 4.6
>
> Attachments: SOLR-4984.patch
>
>
> I believe this used to be used when JCL logging was the default backend.  But 
> now with log4j being the backend, I don't think it is used. It should be 
> removed to not clutter the example with unnecessary files.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Assigned] (SOLR-4984) etc/logging.properties in example should be removed

[Mark Miller (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-4984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mark Miller reassigned SOLR-4984:
-

Assignee: Mark Miller

> etc/logging.properties in example should be removed
> ---
>
> Key: SOLR-4984
> URL: https://issues.apache.org/jira/browse/SOLR-4984
> Project: Solr
>  Issue Type: Bug
>Reporter: Ryan Ernst
>Assignee: Mark Miller
> Attachments: SOLR-4984.patch
>
>
> I believe this used to be used when JCL logging was the default backend.  But 
> now with log4j being the backend, I don't think it is used. It should be 
> removed to not clutter the example with unnecessary files.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5207) lucene expressions module

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-5207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13767634#comment-13767634
 ] 

ASF subversion and git services commented on LUCENE-5207:
-

Commit 1523375 from [~thetaphi] in branch 'dev/branches/lucene5207'
[ https://svn.apache.org/r1523375 ]

LUCENE-5207: Simplify the comparisons

> lucene expressions module
> -
>
> Key: LUCENE-5207
> URL: https://issues.apache.org/jira/browse/LUCENE-5207
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Ryan Ernst
> Attachments: LUCENE-5207.patch
>
>
> Expressions are geared at defining an alternative ranking function (e.g. 
> incorporating the text relevance score and other field values/ranking
> signals). So they are conceptually much more like ElasticSearch's scripting 
> support (http://www.elasticsearch.org/guide/reference/modules/scripting/) 
> than solr's function queries.
> Some additional notes:
> * In addition to referring to other fields, they can also refer to other 
> expressions, so they can be used as "computed fields".
> * You can rank documents easily by multiple expressions (its a SortField at 
> the end), e.g. Sort by year descending, then some function of score price and 
> time ascending.
> * The provided javascript expression syntax is much more efficient than using 
> a scripting engine, because it does not have dynamic typing (compiles to 
> .class files that work on doubles). Performance is similar to writing a 
> custom FieldComparator yourself, but much easier to do.
> * We have solr integration to contribute in the future, but this is just the 
> standalone lucene part as a start. Since lucene has no schema, it includes an 
> implementation of Bindings (SimpleBindings) that maps variable names to 
> SortField's or other expressions.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (LUCENE-5209) Allow the license checker to optionally avoid check sum comparisons on SNAPSHOT dependencies.

[Mark Miller (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-5209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mark Miller resolved LUCENE-5209.
-

Resolution: Fixed

> Allow the license checker to optionally avoid check sum comparisons on 
> SNAPSHOT dependencies.
> -
>
> Key: LUCENE-5209
> URL: https://issues.apache.org/jira/browse/LUCENE-5209
> Project: Lucene - Core
>  Issue Type: Improvement
>  Components: general/build
>Reporter: Mark Miller
>Assignee: Mark Miller
>Priority: Minor
> Fix For: 5.0, 4.6
>
> Attachments: LUCENE-5209.patch
>
>
> SNAPSHOT's cannot actually be used and released by Lucene/Solr, but we use 
> them downstream in some cases during development - we have to harmonize jars 
> across multiple projects.
> It would be nice if we could avoid doing the check sum check on SNAPSHOT's, 
> but still do the license check (dev adds any dependency, dev must add license 
> immediately).
> This first patch adds a new system property called skipSnapshotsChecksum - if 
> you set it to true, SNAPSHOT dependency's will not be check sum compared.
> I think this change makes the license checker more consumable.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5207) lucene expressions module

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-5207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13767627#comment-13767627
 ] 

ASF subversion and git services commented on LUCENE-5207:
-

Commit 1523364 from [~thetaphi] in branch 'dev/branches/lucene5207'
[ https://svn.apache.org/r1523364 ]

LUCENE-5207: Rename field

> lucene expressions module
> -
>
> Key: LUCENE-5207
> URL: https://issues.apache.org/jira/browse/LUCENE-5207
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Ryan Ernst
> Attachments: LUCENE-5207.patch
>
>
> Expressions are geared at defining an alternative ranking function (e.g. 
> incorporating the text relevance score and other field values/ranking
> signals). So they are conceptually much more like ElasticSearch's scripting 
> support (http://www.elasticsearch.org/guide/reference/modules/scripting/) 
> than solr's function queries.
> Some additional notes:
> * In addition to referring to other fields, they can also refer to other 
> expressions, so they can be used as "computed fields".
> * You can rank documents easily by multiple expressions (its a SortField at 
> the end), e.g. Sort by year descending, then some function of score price and 
> time ascending.
> * The provided javascript expression syntax is much more efficient than using 
> a scripting engine, because it does not have dynamic typing (compiles to 
> .class files that work on doubles). Performance is similar to writing a 
> custom FieldComparator yourself, but much easier to do.
> * We have solr integration to contribute in the future, but this is just the 
> standalone lucene part as a start. Since lucene has no schema, it includes an 
> implementation of Bindings (SimpleBindings) that maps variable names to 
> SortField's or other expressions.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5207) lucene expressions module

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-5207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13767626#comment-13767626
 ] 

ASF subversion and git services commented on LUCENE-5207:
-

Commit 1523362 from [~thetaphi] in branch 'dev/branches/lucene5207'
[ https://svn.apache.org/r1523362 ]

LUCENE-5207: Rename method, use a switch statement again

> lucene expressions module
> -
>
> Key: LUCENE-5207
> URL: https://issues.apache.org/jira/browse/LUCENE-5207
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Ryan Ernst
> Attachments: LUCENE-5207.patch
>
>
> Expressions are geared at defining an alternative ranking function (e.g. 
> incorporating the text relevance score and other field values/ranking
> signals). So they are conceptually much more like ElasticSearch's scripting 
> support (http://www.elasticsearch.org/guide/reference/modules/scripting/) 
> than solr's function queries.
> Some additional notes:
> * In addition to referring to other fields, they can also refer to other 
> expressions, so they can be used as "computed fields".
> * You can rank documents easily by multiple expressions (its a SortField at 
> the end), e.g. Sort by year descending, then some function of score price and 
> time ascending.
> * The provided javascript expression syntax is much more efficient than using 
> a scripting engine, because it does not have dynamic typing (compiles to 
> .class files that work on doubles). Performance is similar to writing a 
> custom FieldComparator yourself, but much easier to do.
> * We have solr integration to contribute in the future, but this is just the 
> standalone lucene part as a start. Since lucene has no schema, it includes an 
> implementation of Bindings (SimpleBindings) that maps variable names to 
> SortField's or other expressions.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5207) lucene expressions module

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-5207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13767624#comment-13767624
 ] 

ASF subversion and git services commented on LUCENE-5207:
-

Commit 1523361 from [~thetaphi] in branch 'dev/branches/lucene5207'
[ https://svn.apache.org/r1523361 ]

LUCENE-5207: Simpler without cast

> lucene expressions module
> -
>
> Key: LUCENE-5207
> URL: https://issues.apache.org/jira/browse/LUCENE-5207
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Ryan Ernst
> Attachments: LUCENE-5207.patch
>
>
> Expressions are geared at defining an alternative ranking function (e.g. 
> incorporating the text relevance score and other field values/ranking
> signals). So they are conceptually much more like ElasticSearch's scripting 
> support (http://www.elasticsearch.org/guide/reference/modules/scripting/) 
> than solr's function queries.
> Some additional notes:
> * In addition to referring to other fields, they can also refer to other 
> expressions, so they can be used as "computed fields".
> * You can rank documents easily by multiple expressions (its a SortField at 
> the end), e.g. Sort by year descending, then some function of score price and 
> time ascending.
> * The provided javascript expression syntax is much more efficient than using 
> a scripting engine, because it does not have dynamic typing (compiles to 
> .class files that work on doubles). Performance is similar to writing a 
> custom FieldComparator yourself, but much easier to do.
> * We have solr integration to contribute in the future, but this is just the 
> standalone lucene part as a start. Since lucene has no schema, it includes an 
> implementation of Bindings (SimpleBindings) that maps variable names to 
> SortField's or other expressions.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5207) lucene expressions module

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-5207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13767621#comment-13767621
 ] 

ASF subversion and git services commented on LUCENE-5207:
-

Commit 1523360 from [~thetaphi] in branch 'dev/branches/lucene5207'
[ https://svn.apache.org/r1523360 ]

LUCENE-5207: Thanks Robert for help with this opcode

> lucene expressions module
> -
>
> Key: LUCENE-5207
> URL: https://issues.apache.org/jira/browse/LUCENE-5207
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Ryan Ernst
> Attachments: LUCENE-5207.patch
>
>
> Expressions are geared at defining an alternative ranking function (e.g. 
> incorporating the text relevance score and other field values/ranking
> signals). So they are conceptually much more like ElasticSearch's scripting 
> support (http://www.elasticsearch.org/guide/reference/modules/scripting/) 
> than solr's function queries.
> Some additional notes:
> * In addition to referring to other fields, they can also refer to other 
> expressions, so they can be used as "computed fields".
> * You can rank documents easily by multiple expressions (its a SortField at 
> the end), e.g. Sort by year descending, then some function of score price and 
> time ascending.
> * The provided javascript expression syntax is much more efficient than using 
> a scripting engine, because it does not have dynamic typing (compiles to 
> .class files that work on doubles). Performance is similar to writing a 
> custom FieldComparator yourself, but much easier to do.
> * We have solr integration to contribute in the future, but this is just the 
> standalone lucene part as a start. Since lucene has no schema, it includes an 
> implementation of Bindings (SimpleBindings) that maps variable names to 
> SortField's or other expressions.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5209) Allow the license checker to optionally avoid check sum comparisons on SNAPSHOT dependencies.

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-5209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13767620#comment-13767620
 ] 

ASF subversion and git services commented on LUCENE-5209:
-

Commit 1523359 from [~markrmil...@gmail.com] in branch 'dev/branches/branch_4x'
[ https://svn.apache.org/r1523359 ]

LUCENE-5209: Allow the license checker to optionally avoid check sum 
comparisons on SNAPSHOT dependencies.

> Allow the license checker to optionally avoid check sum comparisons on 
> SNAPSHOT dependencies.
> -
>
> Key: LUCENE-5209
> URL: https://issues.apache.org/jira/browse/LUCENE-5209
> Project: Lucene - Core
>  Issue Type: Improvement
>  Components: general/build
>Reporter: Mark Miller
>Assignee: Mark Miller
>Priority: Minor
> Fix For: 5.0, 4.6
>
> Attachments: LUCENE-5209.patch
>
>
> SNAPSHOT's cannot actually be used and released by Lucene/Solr, but we use 
> them downstream in some cases during development - we have to harmonize jars 
> across multiple projects.
> It would be nice if we could avoid doing the check sum check on SNAPSHOT's, 
> but still do the license check (dev adds any dependency, dev must add license 
> immediately).
> This first patch adds a new system property called skipSnapshotsChecksum - if 
> you set it to true, SNAPSHOT dependency's will not be check sum compared.
> I think this change makes the license checker more consumable.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5207) lucene expressions module

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-5207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13767619#comment-13767619
 ] 

ASF subversion and git services commented on LUCENE-5207:
-

Commit 1523358 from [~thetaphi] in branch 'dev/branches/lucene5207'
[ https://svn.apache.org/r1523358 ]

LUCENE-5207: More simplifications

> lucene expressions module
> -
>
> Key: LUCENE-5207
> URL: https://issues.apache.org/jira/browse/LUCENE-5207
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Ryan Ernst
> Attachments: LUCENE-5207.patch
>
>
> Expressions are geared at defining an alternative ranking function (e.g. 
> incorporating the text relevance score and other field values/ranking
> signals). So they are conceptually much more like ElasticSearch's scripting 
> support (http://www.elasticsearch.org/guide/reference/modules/scripting/) 
> than solr's function queries.
> Some additional notes:
> * In addition to referring to other fields, they can also refer to other 
> expressions, so they can be used as "computed fields".
> * You can rank documents easily by multiple expressions (its a SortField at 
> the end), e.g. Sort by year descending, then some function of score price and 
> time ascending.
> * The provided javascript expression syntax is much more efficient than using 
> a scripting engine, because it does not have dynamic typing (compiles to 
> .class files that work on doubles). Performance is similar to writing a 
> custom FieldComparator yourself, but much easier to do.
> * We have solr integration to contribute in the future, but this is just the 
> standalone lucene part as a start. Since lucene has no schema, it includes an 
> implementation of Bindings (SimpleBindings) that maps variable names to 
> SortField's or other expressions.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5207) lucene expressions module

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-5207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13767618#comment-13767618
 ] 

ASF subversion and git services commented on LUCENE-5207:
-

Commit 1523357 from [~rcmuir] in branch 'dev/branches/lucene5207'
[ https://svn.apache.org/r1523357 ]

LUCENE-5207: more tests

> lucene expressions module
> -
>
> Key: LUCENE-5207
> URL: https://issues.apache.org/jira/browse/LUCENE-5207
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Ryan Ernst
> Attachments: LUCENE-5207.patch
>
>
> Expressions are geared at defining an alternative ranking function (e.g. 
> incorporating the text relevance score and other field values/ranking
> signals). So they are conceptually much more like ElasticSearch's scripting 
> support (http://www.elasticsearch.org/guide/reference/modules/scripting/) 
> than solr's function queries.
> Some additional notes:
> * In addition to referring to other fields, they can also refer to other 
> expressions, so they can be used as "computed fields".
> * You can rank documents easily by multiple expressions (its a SortField at 
> the end), e.g. Sort by year descending, then some function of score price and 
> time ascending.
> * The provided javascript expression syntax is much more efficient than using 
> a scripting engine, because it does not have dynamic typing (compiles to 
> .class files that work on doubles). Performance is similar to writing a 
> custom FieldComparator yourself, but much easier to do.
> * We have solr integration to contribute in the future, but this is just the 
> standalone lucene part as a start. Since lucene has no schema, it includes an 
> implementation of Bindings (SimpleBindings) that maps variable names to 
> SortField's or other expressions.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5209) Allow the license checker to optionally avoid check sum comparisons on SNAPSHOT dependencies.

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-5209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13767617#comment-13767617
 ] 

ASF subversion and git services commented on LUCENE-5209:
-

Commit 1523356 from [~markrmil...@gmail.com] in branch 'dev/trunk'
[ https://svn.apache.org/r1523356 ]

LUCENE-5209: Allow the license checker to optionally avoid check sum 
comparisons on SNAPSHOT dependencies.

> Allow the license checker to optionally avoid check sum comparisons on 
> SNAPSHOT dependencies.
> -
>
> Key: LUCENE-5209
> URL: https://issues.apache.org/jira/browse/LUCENE-5209
> Project: Lucene - Core
>  Issue Type: Improvement
>  Components: general/build
>Reporter: Mark Miller
>Assignee: Mark Miller
>Priority: Minor
> Fix For: 5.0, 4.6
>
> Attachments: LUCENE-5209.patch
>
>
> SNAPSHOT's cannot actually be used and released by Lucene/Solr, but we use 
> them downstream in some cases during development - we have to harmonize jars 
> across multiple projects.
> It would be nice if we could avoid doing the check sum check on SNAPSHOT's, 
> but still do the license check (dev adds any dependency, dev must add license 
> immediately).
> This first patch adds a new system property called skipSnapshotsChecksum - if 
> you set it to true, SNAPSHOT dependency's will not be check sum compared.
> I think this change makes the license checker more consumable.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5207) lucene expressions module

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-5207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13767613#comment-13767613
 ] 

ASF subversion and git services commented on LUCENE-5207:
-

Commit 1523351 from [~rcmuir] in branch 'dev/branches/lucene5207'
[ https://svn.apache.org/r1523351 ]

LUCENE-5207: more parser tests

> lucene expressions module
> -
>
> Key: LUCENE-5207
> URL: https://issues.apache.org/jira/browse/LUCENE-5207
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Ryan Ernst
> Attachments: LUCENE-5207.patch
>
>
> Expressions are geared at defining an alternative ranking function (e.g. 
> incorporating the text relevance score and other field values/ranking
> signals). So they are conceptually much more like ElasticSearch's scripting 
> support (http://www.elasticsearch.org/guide/reference/modules/scripting/) 
> than solr's function queries.
> Some additional notes:
> * In addition to referring to other fields, they can also refer to other 
> expressions, so they can be used as "computed fields".
> * You can rank documents easily by multiple expressions (its a SortField at 
> the end), e.g. Sort by year descending, then some function of score price and 
> time ascending.
> * The provided javascript expression syntax is much more efficient than using 
> a scripting engine, because it does not have dynamic typing (compiles to 
> .class files that work on doubles). Performance is similar to writing a 
> custom FieldComparator yourself, but much easier to do.
> * We have solr integration to contribute in the future, but this is just the 
> standalone lucene part as a start. Since lucene has no schema, it includes an 
> implementation of Bindings (SimpleBindings) that maps variable names to 
> SortField's or other expressions.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5207) lucene expressions module

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-5207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13767609#comment-13767609
 ] 

ASF subversion and git services commented on LUCENE-5207:
-

Commit 1523350 from [~rcmuir] in branch 'dev/branches/lucene5207'
[ https://svn.apache.org/r1523350 ]

LUCENE-5207: add test

> lucene expressions module
> -
>
> Key: LUCENE-5207
> URL: https://issues.apache.org/jira/browse/LUCENE-5207
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Ryan Ernst
> Attachments: LUCENE-5207.patch
>
>
> Expressions are geared at defining an alternative ranking function (e.g. 
> incorporating the text relevance score and other field values/ranking
> signals). So they are conceptually much more like ElasticSearch's scripting 
> support (http://www.elasticsearch.org/guide/reference/modules/scripting/) 
> than solr's function queries.
> Some additional notes:
> * In addition to referring to other fields, they can also refer to other 
> expressions, so they can be used as "computed fields".
> * You can rank documents easily by multiple expressions (its a SortField at 
> the end), e.g. Sort by year descending, then some function of score price and 
> time ascending.
> * The provided javascript expression syntax is much more efficient than using 
> a scripting engine, because it does not have dynamic typing (compiles to 
> .class files that work on doubles). Performance is similar to writing a 
> custom FieldComparator yourself, but much easier to do.
> * We have solr integration to contribute in the future, but this is just the 
> standalone lucene part as a start. Since lucene has no schema, it includes an 
> implementation of Bindings (SimpleBindings) that maps variable names to 
> SortField's or other expressions.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5207) lucene expressions module

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-5207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13767604#comment-13767604
 ] 

ASF subversion and git services commented on LUCENE-5207:
-

Commit 1523345 from [~rcmuir] in branch 'dev/branches/lucene5207'
[ https://svn.apache.org/r1523345 ]

LUCENE-5207: update maven config

> lucene expressions module
> -
>
> Key: LUCENE-5207
> URL: https://issues.apache.org/jira/browse/LUCENE-5207
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Ryan Ernst
> Attachments: LUCENE-5207.patch
>
>
> Expressions are geared at defining an alternative ranking function (e.g. 
> incorporating the text relevance score and other field values/ranking
> signals). So they are conceptually much more like ElasticSearch's scripting 
> support (http://www.elasticsearch.org/guide/reference/modules/scripting/) 
> than solr's function queries.
> Some additional notes:
> * In addition to referring to other fields, they can also refer to other 
> expressions, so they can be used as "computed fields".
> * You can rank documents easily by multiple expressions (its a SortField at 
> the end), e.g. Sort by year descending, then some function of score price and 
> time ascending.
> * The provided javascript expression syntax is much more efficient than using 
> a scripting engine, because it does not have dynamic typing (compiles to 
> .class files that work on doubles). Performance is similar to writing a 
> custom FieldComparator yourself, but much easier to do.
> * We have solr integration to contribute in the future, but this is just the 
> standalone lucene part as a start. Since lucene has no schema, it includes an 
> implementation of Bindings (SimpleBindings) that maps variable names to 
> SortField's or other expressions.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5207) lucene expressions module

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-5207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13767602#comment-13767602
 ] 

ASF subversion and git services commented on LUCENE-5207:
-

Commit 1523341 from [~thetaphi] in branch 'dev/branches/lucene5207'
[ https://svn.apache.org/r1523341 ]

LUCENE-5207: Start to rewrite the bytecode generator using GeneratorAdapter

> lucene expressions module
> -
>
> Key: LUCENE-5207
> URL: https://issues.apache.org/jira/browse/LUCENE-5207
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Ryan Ernst
> Attachments: LUCENE-5207.patch
>
>
> Expressions are geared at defining an alternative ranking function (e.g. 
> incorporating the text relevance score and other field values/ranking
> signals). So they are conceptually much more like ElasticSearch's scripting 
> support (http://www.elasticsearch.org/guide/reference/modules/scripting/) 
> than solr's function queries.
> Some additional notes:
> * In addition to referring to other fields, they can also refer to other 
> expressions, so they can be used as "computed fields".
> * You can rank documents easily by multiple expressions (its a SortField at 
> the end), e.g. Sort by year descending, then some function of score price and 
> time ascending.
> * The provided javascript expression syntax is much more efficient than using 
> a scripting engine, because it does not have dynamic typing (compiles to 
> .class files that work on doubles). Performance is similar to writing a 
> custom FieldComparator yourself, but much easier to do.
> * We have solr integration to contribute in the future, but this is just the 
> standalone lucene part as a start. Since lucene has no schema, it includes an 
> implementation of Bindings (SimpleBindings) that maps variable names to 
> SortField's or other expressions.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS-MAVEN] Lucene-Solr-Maven-trunk #970: POMs out of sync

[Apache Jenkins Server]

Build: https://builds.apache.org/job/Lucene-Solr-Maven-trunk/970/

1 tests failed.
REGRESSION:  
org.apache.solr.cloud.FullSolrCloudDistribCmdsTest.testDistribSearch

Error Message:
null

Stack Trace:
java.lang.NullPointerException: null
at 
__randomizedtesting.SeedInfo.seed([2F458B444A16C4C1:AEA3055C3D49A4FD]:0)
at 
org.apache.solr.common.cloud.ZkCoreNodeProps.getBaseUrl(ZkCoreNodeProps.java:40)
at 
org.apache.solr.client.solrj.impl.CloudSolrServer.buildUrlMap(CloudSolrServer.java:406)
at 
org.apache.solr.client.solrj.impl.CloudSolrServer.directUpdate(CloudSolrServer.java:304)
at 
org.apache.solr.client.solrj.impl.CloudSolrServer.request(CloudSolrServer.java:498)
at 
org.apache.solr.client.solrj.request.AbstractUpdateRequest.process(AbstractUpdateRequest.java:117)
at 
org.apache.solr.cloud.AbstractFullDistribZkTestBase.index_specific(AbstractFullDistribZkTestBase.java:663)
at 
org.apache.solr.cloud.FullSolrCloudDistribCmdsTest.testThatCantForwardToLeaderFails(FullSolrCloudDistribCmdsTest.java:151)
at 
org.apache.solr.cloud.FullSolrCloudDistribCmdsTest.doTest(FullSolrCloudDistribCmdsTest.java:133)




Build Log:
[...truncated 24567 lines...]



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5207) lucene expressions module

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-5207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13767599#comment-13767599
 ] 

ASF subversion and git services commented on LUCENE-5207:
-

Commit 1523339 from [~rcmuir] in branch 'dev/branches/lucene5207'
[ https://svn.apache.org/r1523339 ]

LUCENE-5207: add test for huge amounts of variables

> lucene expressions module
> -
>
> Key: LUCENE-5207
> URL: https://issues.apache.org/jira/browse/LUCENE-5207
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Ryan Ernst
> Attachments: LUCENE-5207.patch
>
>
> Expressions are geared at defining an alternative ranking function (e.g. 
> incorporating the text relevance score and other field values/ranking
> signals). So they are conceptually much more like ElasticSearch's scripting 
> support (http://www.elasticsearch.org/guide/reference/modules/scripting/) 
> than solr's function queries.
> Some additional notes:
> * In addition to referring to other fields, they can also refer to other 
> expressions, so they can be used as "computed fields".
> * You can rank documents easily by multiple expressions (its a SortField at 
> the end), e.g. Sort by year descending, then some function of score price and 
> time ascending.
> * The provided javascript expression syntax is much more efficient than using 
> a scripting engine, because it does not have dynamic typing (compiles to 
> .class files that work on doubles). Performance is similar to writing a 
> custom FieldComparator yourself, but much easier to do.
> * We have solr integration to contribute in the future, but this is just the 
> standalone lucene part as a start. Since lucene has no schema, it includes an 
> implementation of Bindings (SimpleBindings) that maps variable names to 
> SortField's or other expressions.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: [jira] [Comment Edited] (SOLR-4787) Join Contrib

[Erick Erickson]

DId you apply to trunk (future 5.0) or the 4x branch? Work is often
done on the trunk and backported to 4x, don't know what's up with
this particular patch though.

Best,
Erick


On Sat, Sep 14, 2013 at 12:17 AM, Kranti Parisa (JIRA) wrote:

>
> [
> https://issues.apache.org/jira/browse/SOLR-4787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13767361#comment-13767361]
>
> Kranti Parisa edited comment on SOLR-4787 at 9/14/13 4:16 AM:
> --
>
> Something is missing in the Patch? I am seeing ByteArray compilation
> problem. Also does bjoin needs any specific types of field configs in
> schema.xml ?
>
>   was (Author: krantiparisa):
> Something missing in the Patch? I am seeing ByteArray compilation
> problem. Also does bjoin needs any specific types of field configs in
> schema.xml ?
>
> > Join Contrib
> > 
> >
> > Key: SOLR-4787
> > URL: https://issues.apache.org/jira/browse/SOLR-4787
> > Project: Solr
> >  Issue Type: New Feature
> >  Components: search
> >Affects Versions: 4.2.1
> >Reporter: Joel Bernstein
> >Priority: Minor
> > Fix For: 4.5, 5.0
> >
> > Attachments: SOLR-4787-deadlock-fix.patch, SOLR-4787.patch,
> SOLR-4787.patch, SOLR-4787.patch, SOLR-4787.patch, SOLR-4787.patch,
> SOLR-4787.patch, SOLR-4787.patch, SOLR-4787.patch, SOLR-4787.patch,
> SOLR-4787.patch, SOLR-4787.patch, SOLR-4787.patch,
> SOLR-4787-pjoin-long-keys.patch
> >
> >
> > This contrib provides a place where different join implementations can
> be contributed to Solr. This contrib currently includes 3 join
> implementations. The initial patch was generated from the Solr 4.3 tag.
> Because of changes in the FieldCache API this patch will only build with
> Solr 4.2 or above.
> > *HashSetJoinQParserPlugin aka hjoin*
> > The hjoin provides a join implementation that filters results in one
> core based on the results of a search in another core. This is similar in
> functionality to the JoinQParserPlugin but the implementation differs in a
> couple of important ways.
> > The first way is that the hjoin is designed to work with int and long
> join keys only. So, in order to use hjoin, int or long join keys must be
> included in both the to and from core.
> > The second difference is that the hjoin builds memory structures that
> are used to quickly connect the join keys. So, the hjoin will need more
> memory then the JoinQParserPlugin to perform the join.
> > The main advantage of the hjoin is that it can scale to join millions of
> keys between cores and provide sub-second response time. The hjoin should
> work well with up to two million results from the fromIndex and tens of
> millions of results from the main query.
> > The hjoin supports the following features:
> > 1) Both lucene query and PostFilter implementations. A *"cost"* > 99
> will turn on the PostFilter. The PostFilter will typically outperform the
> Lucene query when the main query results have been narrowed down.
> > 2) With the lucene query implementation there is an option to build the
> filter with threads. This can greatly improve the performance of the query
> if the main query index is very large. The "threads" parameter turns on
> threading. For example *threads=6* will use 6 threads to build the filter.
> This will setup a fixed threadpool with six threads to handle all hjoin
> requests. Once the threadpool is created the hjoin will always use it to
> build the filter. Threading does not come into play with the PostFilter.
> > 3) The *size* local parameter can be used to set the initial size of the
> hashset used to perform the join. If this is set above the number of
> results from the fromIndex then the you can avoid hashset resizing which
> improves performance.
> > 4) Nested filter queries. The local parameter "fq" can be used to nest a
> filter query within the join. The nested fq will filter the results of the
> join query. This can point to another join to support nested joins.
> > 5) Full caching support for the lucene query implementation. The
> filterCache and queryResultCache should work properly even with deep
> nesting of joins. Only the queryResultCache comes into play with the
> PostFilter implementation because PostFilters are not cacheable in the
> filterCache.
> > The syntax of the hjoin is similar to the JoinQParserPlugin except that
> the plugin is referenced by the string "hjoin" rather then "join".
> > fq=\{!hjoin fromIndex=collection2 from=id_i to=id_i threads=6
> fq=$qq\}user:customer1&qq=group:5
> > The example filter query above will search the fromIndex (collection2)
> for "user:customer1" applying the local fq parameter to filter the results.
> The lucene filter query will be built using 6 threads. This query will
> generate a list of values from the "from" field that will be used to filter
> the main query. 

[jira] [Commented] (SOLR-1301) Add a Solr contrib that allows for building Solr indexes via Hadoop's Map-Reduce.

[Mark Miller (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-1301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13767583#comment-13767583
 ] 

Mark Miller commented on SOLR-1301:
---

Latest patch with Solr Morphlines and the other items I mentioned above.

New issues though.

Something is still writing to {CWD}/target that I need to track down.

Precommit is not yet passing with Solr morphlines code - need to resolve use of 
forbidden apis:
[forbidden-apis] Forbidden class/interface use: 
com.sun.org.apache.xml.internal.serialize.OutputFormat [non-public internal 
runtime class]
[forbidden-apis]   in 
org.apache.solr.hadoop.morphline.solrcell.SolrCellBuilder$SolrCell 
(SolrCellBuilder.java:242)
[forbidden-apis] Forbidden class/interface use: 
com.sun.org.apache.xml.internal.serialize.XMLSerializer [non-public internal 
runtime class]
[forbidden-apis]   in 
org.apache.solr.hadoop.morphline.solrcell.SolrCellBuilder$SolrCell 
(SolrCellBuilder.java:242)

Also, I'm not sure where exactly the Solr morphlines should end up - as their 
own module or where I put them, but this is where they are for now.

> Add a Solr contrib that allows for building Solr indexes via Hadoop's 
> Map-Reduce.
> -
>
> Key: SOLR-1301
> URL: https://issues.apache.org/jira/browse/SOLR-1301
> Project: Solr
>  Issue Type: New Feature
>Reporter: Andrzej Bialecki 
>Assignee: Mark Miller
> Fix For: 4.5, 5.0
>
> Attachments: commons-logging-1.0.4.jar, 
> commons-logging-api-1.0.4.jar, hadoop-0.19.1-core.jar, 
> hadoop-0.20.1-core.jar, hadoop-core-0.20.2-cdh3u3.jar, hadoop.patch, 
> log4j-1.2.15.jar, README.txt, SOLR-1301-hadoop-0-20.patch, 
> SOLR-1301-hadoop-0-20.patch, SOLR-1301.patch, SOLR-1301.patch, 
> SOLR-1301.patch, SOLR-1301.patch, SOLR-1301.patch, SOLR-1301.patch, 
> SOLR-1301.patch, SOLR-1301.patch, SOLR-1301.patch, SOLR-1301.patch, 
> SOLR-1301.patch, SOLR-1301.patch, SOLR-1301.patch, SOLR-1301.patch, 
> SOLR-1301.patch, SolrRecordWriter.java
>
>
> This patch contains  a contrib module that provides distributed indexing 
> (using Hadoop) to Solr EmbeddedSolrServer. The idea behind this module is 
> twofold:
> * provide an API that is familiar to Hadoop developers, i.e. that of 
> OutputFormat
> * avoid unnecessary export and (de)serialization of data maintained on HDFS. 
> SolrOutputFormat consumes data produced by reduce tasks directly, without 
> storing it in intermediate files. Furthermore, by using an 
> EmbeddedSolrServer, the indexing task is split into as many parts as there 
> are reducers, and the data to be indexed is not sent over the network.
> Design
> --
> Key/value pairs produced by reduce tasks are passed to SolrOutputFormat, 
> which in turn uses SolrRecordWriter to write this data. SolrRecordWriter 
> instantiates an EmbeddedSolrServer, and it also instantiates an 
> implementation of SolrDocumentConverter, which is responsible for turning 
> Hadoop (key, value) into a SolrInputDocument. This data is then added to a 
> batch, which is periodically submitted to EmbeddedSolrServer. When reduce 
> task completes, and the OutputFormat is closed, SolrRecordWriter calls 
> commit() and optimize() on the EmbeddedSolrServer.
> The API provides facilities to specify an arbitrary existing solr.home 
> directory, from which the conf/ and lib/ files will be taken.
> This process results in the creation of as many partial Solr home directories 
> as there were reduce tasks. The output shards are placed in the output 
> directory on the default filesystem (e.g. HDFS). Such part-N directories 
> can be used to run N shard servers. Additionally, users can specify the 
> number of reduce tasks, in particular 1 reduce task, in which case the output 
> will consist of a single shard.
> An example application is provided that processes large CSV files and uses 
> this API. It uses a custom CSV processing to avoid (de)serialization overhead.
> This patch relies on hadoop-core-0.19.1.jar - I attached the jar to this 
> issue, you should put it in contrib/hadoop/lib.
> Note: the development of this patch was sponsored by an anonymous contributor 
> and approved for release under Apache License.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Comment Edited] (SOLR-1301) Add a Solr contrib that allows for building Solr indexes via Hadoop's Map-Reduce.

[Mark Miller (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-1301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13767583#comment-13767583
 ] 

Mark Miller edited comment on SOLR-1301 at 9/14/13 9:16 PM:


Latest patch with Solr Morphlines and the other items I mentioned above.

New issues though.

Something is still writing to \{CWD\}/target that I need to track down.

Precommit is not yet passing with Solr morphlines code - need to resolve use of 
forbidden apis:
[forbidden-apis] Forbidden class/interface use: 
com.sun.org.apache.xml.internal.serialize.OutputFormat [non-public internal 
runtime class]
[forbidden-apis]   in 
org.apache.solr.hadoop.morphline.solrcell.SolrCellBuilder$SolrCell 
(SolrCellBuilder.java:242)
[forbidden-apis] Forbidden class/interface use: 
com.sun.org.apache.xml.internal.serialize.XMLSerializer [non-public internal 
runtime class]
[forbidden-apis]   in 
org.apache.solr.hadoop.morphline.solrcell.SolrCellBuilder$SolrCell 
(SolrCellBuilder.java:242)

Also, I'm not sure where exactly the Solr morphlines should end up - as their 
own module or where I put them, but this is where they are for now.

  was (Author: markrmil...@gmail.com):
Latest patch with Solr Morphlines and the other items I mentioned above.

New issues though.

Something is still writing to {CWD}/target that I need to track down.

Precommit is not yet passing with Solr morphlines code - need to resolve use of 
forbidden apis:
[forbidden-apis] Forbidden class/interface use: 
com.sun.org.apache.xml.internal.serialize.OutputFormat [non-public internal 
runtime class]
[forbidden-apis]   in 
org.apache.solr.hadoop.morphline.solrcell.SolrCellBuilder$SolrCell 
(SolrCellBuilder.java:242)
[forbidden-apis] Forbidden class/interface use: 
com.sun.org.apache.xml.internal.serialize.XMLSerializer [non-public internal 
runtime class]
[forbidden-apis]   in 
org.apache.solr.hadoop.morphline.solrcell.SolrCellBuilder$SolrCell 
(SolrCellBuilder.java:242)

Also, I'm not sure where exactly the Solr morphlines should end up - as their 
own module or where I put them, but this is where they are for now.
  
> Add a Solr contrib that allows for building Solr indexes via Hadoop's 
> Map-Reduce.
> -
>
> Key: SOLR-1301
> URL: https://issues.apache.org/jira/browse/SOLR-1301
> Project: Solr
>  Issue Type: New Feature
>Reporter: Andrzej Bialecki 
>Assignee: Mark Miller
> Fix For: 4.5, 5.0
>
> Attachments: commons-logging-1.0.4.jar, 
> commons-logging-api-1.0.4.jar, hadoop-0.19.1-core.jar, 
> hadoop-0.20.1-core.jar, hadoop-core-0.20.2-cdh3u3.jar, hadoop.patch, 
> log4j-1.2.15.jar, README.txt, SOLR-1301-hadoop-0-20.patch, 
> SOLR-1301-hadoop-0-20.patch, SOLR-1301.patch, SOLR-1301.patch, 
> SOLR-1301.patch, SOLR-1301.patch, SOLR-1301.patch, SOLR-1301.patch, 
> SOLR-1301.patch, SOLR-1301.patch, SOLR-1301.patch, SOLR-1301.patch, 
> SOLR-1301.patch, SOLR-1301.patch, SOLR-1301.patch, SOLR-1301.patch, 
> SOLR-1301.patch, SolrRecordWriter.java
>
>
> This patch contains  a contrib module that provides distributed indexing 
> (using Hadoop) to Solr EmbeddedSolrServer. The idea behind this module is 
> twofold:
> * provide an API that is familiar to Hadoop developers, i.e. that of 
> OutputFormat
> * avoid unnecessary export and (de)serialization of data maintained on HDFS. 
> SolrOutputFormat consumes data produced by reduce tasks directly, without 
> storing it in intermediate files. Furthermore, by using an 
> EmbeddedSolrServer, the indexing task is split into as many parts as there 
> are reducers, and the data to be indexed is not sent over the network.
> Design
> --
> Key/value pairs produced by reduce tasks are passed to SolrOutputFormat, 
> which in turn uses SolrRecordWriter to write this data. SolrRecordWriter 
> instantiates an EmbeddedSolrServer, and it also instantiates an 
> implementation of SolrDocumentConverter, which is responsible for turning 
> Hadoop (key, value) into a SolrInputDocument. This data is then added to a 
> batch, which is periodically submitted to EmbeddedSolrServer. When reduce 
> task completes, and the OutputFormat is closed, SolrRecordWriter calls 
> commit() and optimize() on the EmbeddedSolrServer.
> The API provides facilities to specify an arbitrary existing solr.home 
> directory, from which the conf/ and lib/ files will be taken.
> This process results in the creation of as many partial Solr home directories 
> as there were reduce tasks. The output shards are placed in the output 
> directory on the default filesystem (e.g. HDFS). Such part-N directories 
> can be used to run N shard servers. Additionally, users can specify the 
> number of reduce tasks, in particular 1 reduce task, in which case the o

[jira] [Updated] (LUCENE-5189) Numeric DocValues Updates

[Shai Erera (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-5189?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shai Erera updated LUCENE-5189:
---

Attachment: LUCENE-5189.patch

Thanks Mike. Fixed the two ctors to call doClose {{if (!success)}}, as well as 
added a comment about gen=-1.

I also fixed the two tests which ensure we don't allow updating a field in a 
segment where it doesn't exist 
({{testUpdateSegmentWithPostingButNoDocValues()}} and 
{{testUpdateSegmentWithNoDocValues()}}) to set NoMergePolicy, otherwise the 
segments could merge and the update becomes legit. This exposes the weirdness 
of the exception -- you may not hit it if segments are merged. Once we gen FIS, 
these tests will change though, to ensure these cases ARE allowed :).

> Numeric DocValues Updates
> -
>
> Key: LUCENE-5189
> URL: https://issues.apache.org/jira/browse/LUCENE-5189
> Project: Lucene - Core
>  Issue Type: New Feature
>  Components: core/index
>Reporter: Shai Erera
>Assignee: Shai Erera
> Attachments: LUCENE-5189.patch, LUCENE-5189.patch, LUCENE-5189.patch, 
> LUCENE-5189.patch, LUCENE-5189.patch, LUCENE-5189.patch, LUCENE-5189.patch, 
> LUCENE-5189.patch, LUCENE-5189.patch, LUCENE-5189.patch
>
>
> In LUCENE-4258 we started to work on incremental field updates, however the 
> amount of changes are immense and hard to follow/consume. The reason is that 
> we targeted postings, stored fields, DV etc., all from the get go.
> I'd like to start afresh here, with numeric-dv-field updates only. There are 
> a couple of reasons to that:
> * NumericDV fields should be easier to update, if e.g. we write all the 
> values of all the documents in a segment for the updated field (similar to 
> how livedocs work, and previously norms).
> * It's a fairly contained issue, attempting to handle just one data type to 
> update, yet requires many changes to core code which will also be useful for 
> updating other data types.
> * It has value in and on itself, and we don't need to allow updating all the 
> data types in Lucene at once ... we can do that gradually.
> I have some working patch already which I'll upload next, explaining the 
> changes.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-1301) Add a Solr contrib that allows for building Solr indexes via Hadoop's Map-Reduce.

[Mark Miller (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-1301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mark Miller updated SOLR-1301:
--

Attachment: SOLR-1301.patch

> Add a Solr contrib that allows for building Solr indexes via Hadoop's 
> Map-Reduce.
> -
>
> Key: SOLR-1301
> URL: https://issues.apache.org/jira/browse/SOLR-1301
> Project: Solr
>  Issue Type: New Feature
>Reporter: Andrzej Bialecki 
>Assignee: Mark Miller
> Fix For: 4.5, 5.0
>
> Attachments: commons-logging-1.0.4.jar, 
> commons-logging-api-1.0.4.jar, hadoop-0.19.1-core.jar, 
> hadoop-0.20.1-core.jar, hadoop-core-0.20.2-cdh3u3.jar, hadoop.patch, 
> log4j-1.2.15.jar, README.txt, SOLR-1301-hadoop-0-20.patch, 
> SOLR-1301-hadoop-0-20.patch, SOLR-1301.patch, SOLR-1301.patch, 
> SOLR-1301.patch, SOLR-1301.patch, SOLR-1301.patch, SOLR-1301.patch, 
> SOLR-1301.patch, SOLR-1301.patch, SOLR-1301.patch, SOLR-1301.patch, 
> SOLR-1301.patch, SOLR-1301.patch, SOLR-1301.patch, SOLR-1301.patch, 
> SOLR-1301.patch, SolrRecordWriter.java
>
>
> This patch contains  a contrib module that provides distributed indexing 
> (using Hadoop) to Solr EmbeddedSolrServer. The idea behind this module is 
> twofold:
> * provide an API that is familiar to Hadoop developers, i.e. that of 
> OutputFormat
> * avoid unnecessary export and (de)serialization of data maintained on HDFS. 
> SolrOutputFormat consumes data produced by reduce tasks directly, without 
> storing it in intermediate files. Furthermore, by using an 
> EmbeddedSolrServer, the indexing task is split into as many parts as there 
> are reducers, and the data to be indexed is not sent over the network.
> Design
> --
> Key/value pairs produced by reduce tasks are passed to SolrOutputFormat, 
> which in turn uses SolrRecordWriter to write this data. SolrRecordWriter 
> instantiates an EmbeddedSolrServer, and it also instantiates an 
> implementation of SolrDocumentConverter, which is responsible for turning 
> Hadoop (key, value) into a SolrInputDocument. This data is then added to a 
> batch, which is periodically submitted to EmbeddedSolrServer. When reduce 
> task completes, and the OutputFormat is closed, SolrRecordWriter calls 
> commit() and optimize() on the EmbeddedSolrServer.
> The API provides facilities to specify an arbitrary existing solr.home 
> directory, from which the conf/ and lib/ files will be taken.
> This process results in the creation of as many partial Solr home directories 
> as there were reduce tasks. The output shards are placed in the output 
> directory on the default filesystem (e.g. HDFS). Such part-N directories 
> can be used to run N shard servers. Additionally, users can specify the 
> number of reduce tasks, in particular 1 reduce task, in which case the output 
> will consist of a single shard.
> An example application is provided that processes large CSV files and uses 
> this API. It uses a custom CSV processing to avoid (de)serialization overhead.
> This patch relies on hadoop-core-0.19.1.jar - I attached the jar to this 
> issue, you should put it in contrib/hadoop/lib.
> Note: the development of this patch was sponsored by an anonymous contributor 
> and approved for release under Apache License.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5207) lucene expressions module

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-5207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13767581#comment-13767581
 ] 

ASF subversion and git services commented on LUCENE-5207:
-

Commit 1523324 from [~thetaphi] in branch 'dev/branches/lucene5207'
[ https://svn.apache.org/r1523324 ]

LUCENE-5207: Remove useless check

> lucene expressions module
> -
>
> Key: LUCENE-5207
> URL: https://issues.apache.org/jira/browse/LUCENE-5207
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Ryan Ernst
> Attachments: LUCENE-5207.patch
>
>
> Expressions are geared at defining an alternative ranking function (e.g. 
> incorporating the text relevance score and other field values/ranking
> signals). So they are conceptually much more like ElasticSearch's scripting 
> support (http://www.elasticsearch.org/guide/reference/modules/scripting/) 
> than solr's function queries.
> Some additional notes:
> * In addition to referring to other fields, they can also refer to other 
> expressions, so they can be used as "computed fields".
> * You can rank documents easily by multiple expressions (its a SortField at 
> the end), e.g. Sort by year descending, then some function of score price and 
> time ascending.
> * The provided javascript expression syntax is much more efficient than using 
> a scripting engine, because it does not have dynamic typing (compiles to 
> .class files that work on doubles). Performance is similar to writing a 
> custom FieldComparator yourself, but much easier to do.
> * We have solr integration to contribute in the future, but this is just the 
> standalone lucene part as a start. Since lucene has no schema, it includes an 
> implementation of Bindings (SimpleBindings) that maps variable names to 
> SortField's or other expressions.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5189) Numeric DocValues Updates

[Michael McCandless (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-5189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13767578#comment-13767578
 ] 

Michael McCandless commented on LUCENE-5189:


New patch looks great!  The ref counting is very clean.  Maybe add a comment 
that gen is allowed to be -1, just means the field has no DV updates yet, in 
SegmentReader when building up the map?  And then call doClose() in a finally 
clause if !success so on exception we don't leak open files.

> Numeric DocValues Updates
> -
>
> Key: LUCENE-5189
> URL: https://issues.apache.org/jira/browse/LUCENE-5189
> Project: Lucene - Core
>  Issue Type: New Feature
>  Components: core/index
>Reporter: Shai Erera
>Assignee: Shai Erera
> Attachments: LUCENE-5189.patch, LUCENE-5189.patch, LUCENE-5189.patch, 
> LUCENE-5189.patch, LUCENE-5189.patch, LUCENE-5189.patch, LUCENE-5189.patch, 
> LUCENE-5189.patch, LUCENE-5189.patch
>
>
> In LUCENE-4258 we started to work on incremental field updates, however the 
> amount of changes are immense and hard to follow/consume. The reason is that 
> we targeted postings, stored fields, DV etc., all from the get go.
> I'd like to start afresh here, with numeric-dv-field updates only. There are 
> a couple of reasons to that:
> * NumericDV fields should be easier to update, if e.g. we write all the 
> values of all the documents in a segment for the updated field (similar to 
> how livedocs work, and previously norms).
> * It's a fairly contained issue, attempting to handle just one data type to 
> update, yet requires many changes to core code which will also be useful for 
> updating other data types.
> * It has value in and on itself, and we don't need to allow updating all the 
> data types in Lucene at once ... we can do that gradually.
> I have some working patch already which I'll upload next, explaining the 
> changes.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5207) lucene expressions module

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-5207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13767569#comment-13767569
 ] 

ASF subversion and git services commented on LUCENE-5207:
-

Commit 1523315 from [~thetaphi] in branch 'dev/branches/lucene5207'
[ https://svn.apache.org/r1523315 ]

LUCENE-5207: Add checks for classloader, so all methods in the Map are 
accessible

> lucene expressions module
> -
>
> Key: LUCENE-5207
> URL: https://issues.apache.org/jira/browse/LUCENE-5207
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Ryan Ernst
> Attachments: LUCENE-5207.patch
>
>
> Expressions are geared at defining an alternative ranking function (e.g. 
> incorporating the text relevance score and other field values/ranking
> signals). So they are conceptually much more like ElasticSearch's scripting 
> support (http://www.elasticsearch.org/guide/reference/modules/scripting/) 
> than solr's function queries.
> Some additional notes:
> * In addition to referring to other fields, they can also refer to other 
> expressions, so they can be used as "computed fields".
> * You can rank documents easily by multiple expressions (its a SortField at 
> the end), e.g. Sort by year descending, then some function of score price and 
> time ascending.
> * The provided javascript expression syntax is much more efficient than using 
> a scripting engine, because it does not have dynamic typing (compiles to 
> .class files that work on doubles). Performance is similar to writing a 
> custom FieldComparator yourself, but much easier to do.
> * We have solr integration to contribute in the future, but this is just the 
> standalone lucene part as a start. Since lucene has no schema, it includes an 
> implementation of Bindings (SimpleBindings) that maps variable names to 
> SortField's or other expressions.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-4771) Query-time join collectors could maybe be more efficient

[Robert Muir (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-4771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13767564#comment-13767564
 ] 

Robert Muir commented on LUCENE-4771:
-

By the way, I havent gone further here but I think this approach here can be 
done even faster.

The patch here would simple reduce some hashing and ram-usage, but still be 
slow for each query if there are many terms, because its 'aligning' terms 
per-query.

instead, someone could simply make an OrdinalMap (it takes TermsEnum[]) of the 
fields they are joining, which will "align" terms across all those 
segments/fields into a global space, and the joining can just work on simple 
ords.

this way you only align terms per-reopen instead of per-query. I think this 
would be much faster.

> Query-time join collectors could maybe be more efficient
> 
>
> Key: LUCENE-4771
> URL: https://issues.apache.org/jira/browse/LUCENE-4771
> Project: Lucene - Core
>  Issue Type: Improvement
>  Components: modules/join
>Reporter: Robert Muir
> Attachments: LUCENE-4771_prototype.patch, 
> LUCENE-4771-prototype.patch, LUCENE-4771_prototype_without_bug.patch
>
>
> I was looking @ these collectors on LUCENE-4765 and I noticed:
> * SingleValued collector (SV) pulls FieldCache.getTerms and adds the bytes to 
> a bytesrefhash per-collect.
> * MultiValued  collector (MV) pulls FieldCache.getDocTermsOrds, but doesnt 
> use the ords, just looks up each value and adds the bytes per-collect.
> I think instead its worth investigating if SV should use getTermsIndex, and 
> both collectors just collect-up their per-segment ords in something like a 
> BitSet[maxOrd]. 
> When asked for the terms at the end in getCollectorTerms(), they could merge 
> these into one BytesRefHash.
> Of course, if you are going to turn around and execute the query against the 
> same searcher anyway (is this the typical case?), this could even be more 
> efficient: No need to hash or instantiate all the terms in memory, we could 
> do postpone the lookups to SeekingTermSetTermsEnum.accept()/nextSeekTerm() i 
> think... somehow :)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-5189) Numeric DocValues Updates

[Shai Erera (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-5189?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shai Erera updated LUCENE-5189:
---

Attachment: LUCENE-5189.patch

Fixed NRT support -- DVProducers moved from SegmentCoreReaders to 
SegmentReader, where they are being wrapped by RefCount which keeps track of 
ref counts (new class I added). When an SR is shared, SR inc-refs the 
DVProducers that it uses (according to SIPC.getDVGen(field)) and dec-ref on 
doClose().

All Lucene tests pass, including the new numeric DV updates ones. A review 
would be appreciated though.

The remaining nocommits are for renames and FieldInfos.gen. I think I'll leave 
the renames as TODOs, to handle prior to merging to 4x (after this one bakes in 
trunk), that way avoiding potential messy merges. I can handle FieldInfos.gen 
either as part of this issue, or a separate one. Preferences?

> Numeric DocValues Updates
> -
>
> Key: LUCENE-5189
> URL: https://issues.apache.org/jira/browse/LUCENE-5189
> Project: Lucene - Core
>  Issue Type: New Feature
>  Components: core/index
>Reporter: Shai Erera
>Assignee: Shai Erera
> Attachments: LUCENE-5189.patch, LUCENE-5189.patch, LUCENE-5189.patch, 
> LUCENE-5189.patch, LUCENE-5189.patch, LUCENE-5189.patch, LUCENE-5189.patch, 
> LUCENE-5189.patch, LUCENE-5189.patch
>
>
> In LUCENE-4258 we started to work on incremental field updates, however the 
> amount of changes are immense and hard to follow/consume. The reason is that 
> we targeted postings, stored fields, DV etc., all from the get go.
> I'd like to start afresh here, with numeric-dv-field updates only. There are 
> a couple of reasons to that:
> * NumericDV fields should be easier to update, if e.g. we write all the 
> values of all the documents in a segment for the updated field (similar to 
> how livedocs work, and previously norms).
> * It's a fairly contained issue, attempting to handle just one data type to 
> update, yet requires many changes to core code which will also be useful for 
> updating other data types.
> * It has value in and on itself, and we don't need to allow updating all the 
> data types in Lucene at once ... we can do that gradually.
> I have some working patch already which I'll upload next, explaining the 
> changes.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-5180) ShingleFilter should make shingles from trailing holes

[Michael McCandless (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-5180?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael McCandless updated LUCENE-5180:
---

Fix Version/s: (was: 4.5)
   4.6

> ShingleFilter should make shingles from trailing holes
> --
>
> Key: LUCENE-5180
> URL: https://issues.apache.org/jira/browse/LUCENE-5180
> Project: Lucene - Core
>  Issue Type: Improvement
>  Components: modules/analysis
>Reporter: Michael McCandless
>Assignee: Michael McCandless
> Fix For: 5.0, 4.6
>
> Attachments: LUCENE-5180.patch
>
>
> When ShingleFilter hits a hole, it uses _ as the token, e.g. bigrams for "the 
> dog barked", if you have a StopFilter removing the, would be: "_ dog", "dog 
> barked".
> But if the input ends with a stopword, e.g. "wizard of", ShingleFilter fails 
> to produce "wizard _" due to LUCENE-3849 ... once we fix that I think we 
> should fix ShingleFilter to make shingles for trailing holes too ...

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-5180) ShingleFilter should make shingles from trailing holes

[Michael McCandless (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-5180?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael McCandless updated LUCENE-5180:
---

Attachment: LUCENE-5180.patch

Patch; it turned out to be easier than I expected: I just tapped into the 
existing logic that ShingleFilter has for handling holes between tokens.

> ShingleFilter should make shingles from trailing holes
> --
>
> Key: LUCENE-5180
> URL: https://issues.apache.org/jira/browse/LUCENE-5180
> Project: Lucene - Core
>  Issue Type: Improvement
>  Components: modules/analysis
>Reporter: Michael McCandless
>Assignee: Michael McCandless
> Fix For: 5.0, 4.6
>
> Attachments: LUCENE-5180.patch
>
>
> When ShingleFilter hits a hole, it uses _ as the token, e.g. bigrams for "the 
> dog barked", if you have a StopFilter removing the, would be: "_ dog", "dog 
> barked".
> But if the input ends with a stopword, e.g. "wizard of", ShingleFilter fails 
> to produce "wizard _" due to LUCENE-3849 ... once we fix that I think we 
> should fix ShingleFilter to make shingles for trailing holes too ...

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Assigned] (LUCENE-5180) ShingleFilter should make shingles from trailing holes

[Michael McCandless (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-5180?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael McCandless reassigned LUCENE-5180:
--

Assignee: Michael McCandless

> ShingleFilter should make shingles from trailing holes
> --
>
> Key: LUCENE-5180
> URL: https://issues.apache.org/jira/browse/LUCENE-5180
> Project: Lucene - Core
>  Issue Type: Improvement
>  Components: modules/analysis
>Reporter: Michael McCandless
>Assignee: Michael McCandless
> Fix For: 5.0, 4.5
>
> Attachments: LUCENE-5180.patch
>
>
> When ShingleFilter hits a hole, it uses _ as the token, e.g. bigrams for "the 
> dog barked", if you have a StopFilter removing the, would be: "_ dog", "dog 
> barked".
> But if the input ends with a stopword, e.g. "wizard of", ShingleFilter fails 
> to produce "wizard _" due to LUCENE-3849 ... once we fix that I think we 
> should fix ShingleFilter to make shingles for trailing holes too ...

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5212) java 7u40 causes sigsegv and corrupt term vectors

[Michael McCandless (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-5212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13767559#comment-13767559
 ] 

Michael McCandless commented on LUCENE-5212:


bq. in jenkins on os X, it didnt crash but had corrupt term vectors in several 
unrelated tests.

Very scary that this can also be a "silently introduces index corruption" JVM 
bug!  I'd much prefer the sudden death SEGV.

> java 7u40 causes sigsegv and corrupt term vectors
> -
>
> Key: LUCENE-5212
> URL: https://issues.apache.org/jira/browse/LUCENE-5212
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Robert Muir
> Attachments: hs_err_pid32714.log, jenkins.txt
>
>


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: [JENKINS] Lucene-Solr-trunk-Linux (64bit/jdk1.7.0_40) - Build # 7427 - Still Failing!

[Michael McCandless]

It looks like this is the new JVM bug:
https://issues.apache.org/jira/browse/LUCENE-5212

We don't know much about it yet ... except the JVM bug was not present
in 1.7.0_25 and is present in 1.7.0_40.

Mike McCandless

http://blog.mikemccandless.com


On Fri, Sep 13, 2013 at 12:08 PM, Robert Muir  wrote:
> Maybe it will reproduce with the master seed.
>
>
> On Fri, Sep 13, 2013 at 7:14 AM, Michael McCandless
>  wrote:
>> This doesn't repro on current trunk ...
>>
>> Mike McCandless
>>
>> http://blog.mikemccandless.com
>>
>>
>> On Thu, Sep 12, 2013 at 5:26 AM, Policeman Jenkins Server
>>  wrote:
>>> Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/7427/
>>> Java: 64bit/jdk1.7.0_40 -XX:-UseCompressedOops -XX:+UseG1GC
>>>
>>> 1 tests failed.
>>> REGRESSION:  
>>> org.apache.lucene.classification.KNearestNeighborClassifierTest.testPerformance
>>>
>>> Error Message:
>>> CheckIndex failed
>>>
>>> Stack Trace:
>>> java.lang.RuntimeException: CheckIndex failed
>>> at 
>>> __randomizedtesting.SeedInfo.seed([68F81CB00E47182B:AF19EE9265F32084]:0)
>>> at org.apache.lucene.util._TestUtil.checkIndex(_TestUtil.java:227)
>>> at 
>>> org.apache.lucene.store.MockDirectoryWrapper.close(MockDirectoryWrapper.java:659)
>>> at 
>>> org.apache.lucene.classification.ClassificationTestBase.tearDown(ClassificationTestBase.java:70)
>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>> at 
>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>>> at 
>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>> at java.lang.reflect.Method.invoke(Method.java:606)
>>> at 
>>> com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
>>> at 
>>> com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
>>> at 
>>> com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:795)
>>> at 
>>> org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
>>> at 
>>> org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
>>> at 
>>> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
>>> at 
>>> com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
>>> at 
>>> org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
>>> at 
>>> org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
>>> at 
>>> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
>>> at 
>>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>> at 
>>> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
>>> at 
>>> com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
>>> at 
>>> com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
>>> at 
>>> com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
>>> at 
>>> com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
>>> at 
>>> com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
>>> at 
>>> com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
>>> at 
>>> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
>>> at 
>>> org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
>>> at 
>>> com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
>>> at 
>>> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
>>> at 
>>> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
>>> at 
>>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>> at 
>>> org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
>>> at 
>>> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
>>> at 
>>> org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
>>> at 
>>> org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTes

[jira] [Commented] (LUCENE-5207) lucene expressions module

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-5207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13767554#comment-13767554
 ] 

ASF subversion and git services commented on LUCENE-5207:
-

Commit 1523301 from [~rcmuir] in branch 'dev/branches/lucene5207'
[ https://svn.apache.org/r1523301 ]

LUCENE-5207: check the class too

> lucene expressions module
> -
>
> Key: LUCENE-5207
> URL: https://issues.apache.org/jira/browse/LUCENE-5207
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Ryan Ernst
> Attachments: LUCENE-5207.patch
>
>
> Expressions are geared at defining an alternative ranking function (e.g. 
> incorporating the text relevance score and other field values/ranking
> signals). So they are conceptually much more like ElasticSearch's scripting 
> support (http://www.elasticsearch.org/guide/reference/modules/scripting/) 
> than solr's function queries.
> Some additional notes:
> * In addition to referring to other fields, they can also refer to other 
> expressions, so they can be used as "computed fields".
> * You can rank documents easily by multiple expressions (its a SortField at 
> the end), e.g. Sort by year descending, then some function of score price and 
> time ascending.
> * The provided javascript expression syntax is much more efficient than using 
> a scripting engine, because it does not have dynamic typing (compiles to 
> .class files that work on doubles). Performance is similar to writing a 
> custom FieldComparator yourself, but much easier to do.
> * We have solr integration to contribute in the future, but this is just the 
> standalone lucene part as a start. Since lucene has no schema, it includes an 
> implementation of Bindings (SimpleBindings) that maps variable names to 
> SortField's or other expressions.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5207) lucene expressions module

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-5207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13767555#comment-13767555
 ] 

ASF subversion and git services commented on LUCENE-5207:
-

Commit 1523302 from [~rcmuir] in branch 'dev/branches/lucene5207'
[ https://svn.apache.org/r1523302 ]

LUCENE-5207: fix oops in test

> lucene expressions module
> -
>
> Key: LUCENE-5207
> URL: https://issues.apache.org/jira/browse/LUCENE-5207
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Ryan Ernst
> Attachments: LUCENE-5207.patch
>
>
> Expressions are geared at defining an alternative ranking function (e.g. 
> incorporating the text relevance score and other field values/ranking
> signals). So they are conceptually much more like ElasticSearch's scripting 
> support (http://www.elasticsearch.org/guide/reference/modules/scripting/) 
> than solr's function queries.
> Some additional notes:
> * In addition to referring to other fields, they can also refer to other 
> expressions, so they can be used as "computed fields".
> * You can rank documents easily by multiple expressions (its a SortField at 
> the end), e.g. Sort by year descending, then some function of score price and 
> time ascending.
> * The provided javascript expression syntax is much more efficient than using 
> a scripting engine, because it does not have dynamic typing (compiles to 
> .class files that work on doubles). Performance is similar to writing a 
> custom FieldComparator yourself, but much easier to do.
> * We have solr integration to contribute in the future, but this is just the 
> standalone lucene part as a start. Since lucene has no schema, it includes an 
> implementation of Bindings (SimpleBindings) that maps variable names to 
> SortField's or other expressions.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5207) lucene expressions module

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-5207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13767550#comment-13767550
 ] 

ASF subversion and git services commented on LUCENE-5207:
-

Commit 1523300 from [~rcmuir] in branch 'dev/branches/lucene5207'
[ https://svn.apache.org/r1523300 ]

LUCENE-5207: add some checks and tests for illegal stuff

> lucene expressions module
> -
>
> Key: LUCENE-5207
> URL: https://issues.apache.org/jira/browse/LUCENE-5207
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Ryan Ernst
> Attachments: LUCENE-5207.patch
>
>
> Expressions are geared at defining an alternative ranking function (e.g. 
> incorporating the text relevance score and other field values/ranking
> signals). So they are conceptually much more like ElasticSearch's scripting 
> support (http://www.elasticsearch.org/guide/reference/modules/scripting/) 
> than solr's function queries.
> Some additional notes:
> * In addition to referring to other fields, they can also refer to other 
> expressions, so they can be used as "computed fields".
> * You can rank documents easily by multiple expressions (its a SortField at 
> the end), e.g. Sort by year descending, then some function of score price and 
> time ascending.
> * The provided javascript expression syntax is much more efficient than using 
> a scripting engine, because it does not have dynamic typing (compiles to 
> .class files that work on doubles). Performance is similar to writing a 
> custom FieldComparator yourself, but much easier to do.
> * We have solr integration to contribute in the future, but this is just the 
> standalone lucene part as a start. Since lucene has no schema, it includes an 
> implementation of Bindings (SimpleBindings) that maps variable names to 
> SortField's or other expressions.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5207) lucene expressions module

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-5207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13767545#comment-13767545
 ] 

ASF subversion and git services commented on LUCENE-5207:
-

Commit 1523297 from [~rcmuir] in branch 'dev/branches/lucene5207'
[ https://svn.apache.org/r1523297 ]

LUCENE-5207: allow specifying classloader when using custom functions

> lucene expressions module
> -
>
> Key: LUCENE-5207
> URL: https://issues.apache.org/jira/browse/LUCENE-5207
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Ryan Ernst
> Attachments: LUCENE-5207.patch
>
>
> Expressions are geared at defining an alternative ranking function (e.g. 
> incorporating the text relevance score and other field values/ranking
> signals). So they are conceptually much more like ElasticSearch's scripting 
> support (http://www.elasticsearch.org/guide/reference/modules/scripting/) 
> than solr's function queries.
> Some additional notes:
> * In addition to referring to other fields, they can also refer to other 
> expressions, so they can be used as "computed fields".
> * You can rank documents easily by multiple expressions (its a SortField at 
> the end), e.g. Sort by year descending, then some function of score price and 
> time ascending.
> * The provided javascript expression syntax is much more efficient than using 
> a scripting engine, because it does not have dynamic typing (compiles to 
> .class files that work on doubles). Performance is similar to writing a 
> custom FieldComparator yourself, but much easier to do.
> * We have solr integration to contribute in the future, but this is just the 
> standalone lucene part as a start. Since lucene has no schema, it includes an 
> implementation of Bindings (SimpleBindings) that maps variable names to 
> SortField's or other expressions.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5207) lucene expressions module

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-5207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13767544#comment-13767544
 ] 

ASF subversion and git services commented on LUCENE-5207:
-

Commit 1523296 from [~rcmuir] in branch 'dev/branches/lucene5207'
[ https://svn.apache.org/r1523296 ]

LUCENE-5207: add some simple tests for custom functions

> lucene expressions module
> -
>
> Key: LUCENE-5207
> URL: https://issues.apache.org/jira/browse/LUCENE-5207
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Ryan Ernst
> Attachments: LUCENE-5207.patch
>
>
> Expressions are geared at defining an alternative ranking function (e.g. 
> incorporating the text relevance score and other field values/ranking
> signals). So they are conceptually much more like ElasticSearch's scripting 
> support (http://www.elasticsearch.org/guide/reference/modules/scripting/) 
> than solr's function queries.
> Some additional notes:
> * In addition to referring to other fields, they can also refer to other 
> expressions, so they can be used as "computed fields".
> * You can rank documents easily by multiple expressions (its a SortField at 
> the end), e.g. Sort by year descending, then some function of score price and 
> time ascending.
> * The provided javascript expression syntax is much more efficient than using 
> a scripting engine, because it does not have dynamic typing (compiles to 
> .class files that work on doubles). Performance is similar to writing a 
> custom FieldComparator yourself, but much easier to do.
> * We have solr integration to contribute in the future, but this is just the 
> standalone lucene part as a start. Since lucene has no schema, it includes an 
> implementation of Bindings (SimpleBindings) that maps variable names to 
> SortField's or other expressions.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5207) lucene expressions module

[Robert Muir (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-5207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13767542#comment-13767542
 ] 

Robert Muir commented on LUCENE-5207:
-

{quote}
Finally its simple: Once we allow foreign, user-defined functions, we must add 
the Classloader argument to the compiler again, otherwise you cannot register 
methods from classes of foreign classloaders. An alternative would be to let 
the compiler figure out himself by passing a list of java.lang.reflect.Method 
and it chooses the correct classloader automatically.
{quote}

I think we can add this to the second method signature?

> lucene expressions module
> -
>
> Key: LUCENE-5207
> URL: https://issues.apache.org/jira/browse/LUCENE-5207
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Ryan Ernst
> Attachments: LUCENE-5207.patch
>
>
> Expressions are geared at defining an alternative ranking function (e.g. 
> incorporating the text relevance score and other field values/ranking
> signals). So they are conceptually much more like ElasticSearch's scripting 
> support (http://www.elasticsearch.org/guide/reference/modules/scripting/) 
> than solr's function queries.
> Some additional notes:
> * In addition to referring to other fields, they can also refer to other 
> expressions, so they can be used as "computed fields".
> * You can rank documents easily by multiple expressions (its a SortField at 
> the end), e.g. Sort by year descending, then some function of score price and 
> time ascending.
> * The provided javascript expression syntax is much more efficient than using 
> a scripting engine, because it does not have dynamic typing (compiles to 
> .class files that work on doubles). Performance is similar to writing a 
> custom FieldComparator yourself, but much easier to do.
> * We have solr integration to contribute in the future, but this is just the 
> standalone lucene part as a start. Since lucene has no schema, it includes an 
> implementation of Bindings (SimpleBindings) that maps variable names to 
> SortField's or other expressions.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5207) lucene expressions module

[Robert Muir (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-5207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13767540#comment-13767540
 ] 

Robert Muir commented on LUCENE-5207:
-

Hi Uwe: please see my commit (we can revert it, if we have to, but i think its 
an easy step).

The idea is: JavascriptFunction as a class provides nothing really, it is 
nothing more than a Method with some extra checks in its ctor.

So i dont think this class need exist at all, and the default list is nothing 
but a unmodifiable Map.

So in addition to:
{code}
public static Expression compile(String sourceText);
{code}

I added:
{code}
  public static Expression compile(String sourceText, Map 
functions) throws ParseException {
for (Method m : functions.values()) {
  checkFunction(m);
}
return new JavascriptCompiler(sourceText, functions).compileExpression();
  }
{code}

I will add some tests, but this is very simple and allows someone to expose 
whatever methods they want, not just add but also remove :)

> lucene expressions module
> -
>
> Key: LUCENE-5207
> URL: https://issues.apache.org/jira/browse/LUCENE-5207
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Ryan Ernst
> Attachments: LUCENE-5207.patch
>
>
> Expressions are geared at defining an alternative ranking function (e.g. 
> incorporating the text relevance score and other field values/ranking
> signals). So they are conceptually much more like ElasticSearch's scripting 
> support (http://www.elasticsearch.org/guide/reference/modules/scripting/) 
> than solr's function queries.
> Some additional notes:
> * In addition to referring to other fields, they can also refer to other 
> expressions, so they can be used as "computed fields".
> * You can rank documents easily by multiple expressions (its a SortField at 
> the end), e.g. Sort by year descending, then some function of score price and 
> time ascending.
> * The provided javascript expression syntax is much more efficient than using 
> a scripting engine, because it does not have dynamic typing (compiles to 
> .class files that work on doubles). Performance is similar to writing a 
> custom FieldComparator yourself, but much easier to do.
> * We have solr integration to contribute in the future, but this is just the 
> standalone lucene part as a start. Since lucene has no schema, it includes an 
> implementation of Bindings (SimpleBindings) that maps variable names to 
> SortField's or other expressions.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5207) lucene expressions module

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-5207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13767539#comment-13767539
 ] 

ASF subversion and git services commented on LUCENE-5207:
-

Commit 1523286 from [~rcmuir] in branch 'dev/branches/lucene5207'
[ https://svn.apache.org/r1523286 ]

LUCENE-5207: make functions pluggable

> lucene expressions module
> -
>
> Key: LUCENE-5207
> URL: https://issues.apache.org/jira/browse/LUCENE-5207
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Ryan Ernst
> Attachments: LUCENE-5207.patch
>
>
> Expressions are geared at defining an alternative ranking function (e.g. 
> incorporating the text relevance score and other field values/ranking
> signals). So they are conceptually much more like ElasticSearch's scripting 
> support (http://www.elasticsearch.org/guide/reference/modules/scripting/) 
> than solr's function queries.
> Some additional notes:
> * In addition to referring to other fields, they can also refer to other 
> expressions, so they can be used as "computed fields".
> * You can rank documents easily by multiple expressions (its a SortField at 
> the end), e.g. Sort by year descending, then some function of score price and 
> time ascending.
> * The provided javascript expression syntax is much more efficient than using 
> a scripting engine, because it does not have dynamic typing (compiles to 
> .class files that work on doubles). Performance is similar to writing a 
> custom FieldComparator yourself, but much easier to do.
> * We have solr integration to contribute in the future, but this is just the 
> standalone lucene part as a start. Since lucene has no schema, it includes an 
> implementation of Bindings (SimpleBindings) that maps variable names to 
> SortField's or other expressions.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5207) lucene expressions module

[Uwe Schindler (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-5207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13767538#comment-13767538
 ] 

Uwe Schindler commented on LUCENE-5207:
---

Hi Robert,
sorry for beeing a little bit confusing:
The two issues Classloader and Functions extensibility don't really have 
something to do, they are mostly separate. With the current code and private 
ctors and static functions, there is indeed no need for a custom classloader. 
This is why I argued to remove it.

The classloader also does not have anything to do with Bindings. The bindings 
was just an idea how to make it "easy" for the user to register own functions 
in a non-static way. I was under the impression, that the bindings are availabe 
while compiling. I don't want it "dynamic" - I wanted it still statically 
compiled. The idea was to add the reflected Method to the bindings, so when 
compiling the bytecode the method can be used (not per call). After reviewing 
the code again, I can see that the bindings are not available at the time of 
compiling - that was the misunderstanding, sorry (but it could still be fixed 
to support this). It just looked elegant to treat a function like a binding.

The current patch is fine, let's commit it and do all other stuff like 
extending custom functions later. For that we have to make the whole 
JavascriptFunction interface public and non-static (currently its a 
singleton-like factory). But this can be done in another issue.

The classloader problem is something that comes into the game when users are 
able to register own functions. The problem here is that the classloader used 
to load the ASM-generated class has the one of lucene-core as parent (because 
we use the compiler's own classloader: this.getClass().getClassloader()). This 
classloader does not necessarily have access to anything in a different 
classloader (e.g. a plugin in ElasticSearch or Solr).

Finally its simple: Once we allow foreign, user-defined functions, we must add 
the Classloader argument to the compiler again, otherwise you cannot register 
methods from classes of foreign classloaders. An alternative would be to let 
the compiler figure out himself by passing a list of java.lang.reflect.Method 
and it chooses the correct classloader automatically.

> lucene expressions module
> -
>
> Key: LUCENE-5207
> URL: https://issues.apache.org/jira/browse/LUCENE-5207
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Ryan Ernst
> Attachments: LUCENE-5207.patch
>
>
> Expressions are geared at defining an alternative ranking function (e.g. 
> incorporating the text relevance score and other field values/ranking
> signals). So they are conceptually much more like ElasticSearch's scripting 
> support (http://www.elasticsearch.org/guide/reference/modules/scripting/) 
> than solr's function queries.
> Some additional notes:
> * In addition to referring to other fields, they can also refer to other 
> expressions, so they can be used as "computed fields".
> * You can rank documents easily by multiple expressions (its a SortField at 
> the end), e.g. Sort by year descending, then some function of score price and 
> time ascending.
> * The provided javascript expression syntax is much more efficient than using 
> a scripting engine, because it does not have dynamic typing (compiles to 
> .class files that work on doubles). Performance is similar to writing a 
> custom FieldComparator yourself, but much easier to do.
> * We have solr integration to contribute in the future, but this is just the 
> standalone lucene part as a start. Since lucene has no schema, it includes an 
> implementation of Bindings (SimpleBindings) that maps variable names to 
> SortField's or other expressions.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS-MAVEN] Lucene-Solr-Maven-4.x #447: POMs out of sync

[Apache Jenkins Server]

Build: https://builds.apache.org/job/Lucene-Solr-Maven-4.x/447/

1 tests failed.
FAILED:  org.apache.solr.cloud.SyncSliceTest.testDistribSearch

Error Message:
expected:<5> but was:<4>

Stack Trace:
java.lang.AssertionError: expected:<5> but was:<4>
at 
__randomizedtesting.SeedInfo.seed([58B7245853B64FD5:D951AA4024E92FE9]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.junit.Assert.assertEquals(Assert.java:456)
at org.apache.solr.cloud.SyncSliceTest.doTest(SyncSliceTest.java:175)




Build Log:
[...truncated 24966 lines...]



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5207) lucene expressions module

[Robert Muir (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-5207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13767536#comment-13767536
 ] 

Robert Muir commented on LUCENE-5207:
-

{quote}
I thought a little bit about extensibility of Javascript functions: The current 
code is more or less "hardcoded" (although its configureable by a resource file 
in the JAR). But this is not extensible by users (all private, you have to 
rebuild JAR).
{quote}

Thats not exactly true. Someone can make plug in their own compiler that 
compiles String->Expression in some other way.

If we want to make this particular one more extensible, then we should give it 
ctors or protected methods to do that. 

This is easy to do and does not involve classloader hell. Doing functions 
through Bindings would be both slow and wrong.

> lucene expressions module
> -
>
> Key: LUCENE-5207
> URL: https://issues.apache.org/jira/browse/LUCENE-5207
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Ryan Ernst
> Attachments: LUCENE-5207.patch
>
>
> Expressions are geared at defining an alternative ranking function (e.g. 
> incorporating the text relevance score and other field values/ranking
> signals). So they are conceptually much more like ElasticSearch's scripting 
> support (http://www.elasticsearch.org/guide/reference/modules/scripting/) 
> than solr's function queries.
> Some additional notes:
> * In addition to referring to other fields, they can also refer to other 
> expressions, so they can be used as "computed fields".
> * You can rank documents easily by multiple expressions (its a SortField at 
> the end), e.g. Sort by year descending, then some function of score price and 
> time ascending.
> * The provided javascript expression syntax is much more efficient than using 
> a scripting engine, because it does not have dynamic typing (compiles to 
> .class files that work on doubles). Performance is similar to writing a 
> custom FieldComparator yourself, but much easier to do.
> * We have solr integration to contribute in the future, but this is just the 
> standalone lucene part as a start. Since lucene has no schema, it includes an 
> implementation of Bindings (SimpleBindings) that maps variable names to 
> SortField's or other expressions.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-5234) Allow SolrResourceLoader to load resources from URLs

[Mark Miller (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-5234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13767532#comment-13767532
 ] 

Mark Miller commented on SOLR-5234:
---

Have not looked at the patch yet, but the feature sounds okay to me.

> Allow SolrResourceLoader to load resources from URLs
> 
>
> Key: SOLR-5234
> URL: https://issues.apache.org/jira/browse/SOLR-5234
> Project: Solr
>  Issue Type: Improvement
>Reporter: Alan Woodward
>Assignee: Alan Woodward
>Priority: Minor
> Attachments: SOLR-5234.patch
>
>
> This would allow multiple solr instance to share large configuration files.  
> It would also help resolve problems caused by attempting to store >1Mb files 
> in zookeeper.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-5234) Allow SolrResourceLoader to load resources from URLs

[Mark Miller (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-5234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13767533#comment-13767533
 ] 

Mark Miller commented on SOLR-5234:
---

bq. I'm not sure how best to test the new functionality. 

Just an embedded jetty instance?

> Allow SolrResourceLoader to load resources from URLs
> 
>
> Key: SOLR-5234
> URL: https://issues.apache.org/jira/browse/SOLR-5234
> Project: Solr
>  Issue Type: Improvement
>Reporter: Alan Woodward
>Assignee: Alan Woodward
>Priority: Minor
> Attachments: SOLR-5234.patch
>
>
> This would allow multiple solr instance to share large configuration files.  
> It would also help resolve problems caused by attempting to store >1Mb files 
> in zookeeper.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5207) lucene expressions module

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-5207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13767530#comment-13767530
 ] 

ASF subversion and git services commented on LUCENE-5207:
-

Commit 1523279 from [~rcmuir] in branch 'dev/branches/lucene5207'
[ https://svn.apache.org/r1523279 ]

LUCENE-5207: add resources folder here

> lucene expressions module
> -
>
> Key: LUCENE-5207
> URL: https://issues.apache.org/jira/browse/LUCENE-5207
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Ryan Ernst
> Attachments: LUCENE-5207.patch
>
>
> Expressions are geared at defining an alternative ranking function (e.g. 
> incorporating the text relevance score and other field values/ranking
> signals). So they are conceptually much more like ElasticSearch's scripting 
> support (http://www.elasticsearch.org/guide/reference/modules/scripting/) 
> than solr's function queries.
> Some additional notes:
> * In addition to referring to other fields, they can also refer to other 
> expressions, so they can be used as "computed fields".
> * You can rank documents easily by multiple expressions (its a SortField at 
> the end), e.g. Sort by year descending, then some function of score price and 
> time ascending.
> * The provided javascript expression syntax is much more efficient than using 
> a scripting engine, because it does not have dynamic typing (compiles to 
> .class files that work on doubles). Performance is similar to writing a 
> custom FieldComparator yourself, but much easier to do.
> * We have solr integration to contribute in the future, but this is just the 
> standalone lucene part as a start. Since lucene has no schema, it includes an 
> implementation of Bindings (SimpleBindings) that maps variable names to 
> SortField's or other expressions.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-1301) Add a Solr contrib that allows for building Solr indexes via Hadoop's Map-Reduce.

[Mark Miller (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-1301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13767529#comment-13767529
 ] 

Mark Miller commented on SOLR-1301:
---

I've moved in the Solr Morphlines code from cdk-morphlines-solr-core and 
cdk-morphlines-solr-cell. I've made them compliant with the test framework and 
got the tests passing. There are still some ant precommit and license issues to 
handle, but otherwise this should be mostly done. I'm still unclear on what 
will be required for packaging, but I am tackling packaging last.

I've also updated the Tika parser dependencies to include a couple Solr did not 
have.

Once I wrap up the loose ends on this I'll attach my latest patch, and then, 
only two issues on the critical path:

* Get the tests to run without a hacked test.policy file.
* Dist packaging.

> Add a Solr contrib that allows for building Solr indexes via Hadoop's 
> Map-Reduce.
> -
>
> Key: SOLR-1301
> URL: https://issues.apache.org/jira/browse/SOLR-1301
> Project: Solr
>  Issue Type: New Feature
>Reporter: Andrzej Bialecki 
>Assignee: Mark Miller
> Fix For: 4.5, 5.0
>
> Attachments: commons-logging-1.0.4.jar, 
> commons-logging-api-1.0.4.jar, hadoop-0.19.1-core.jar, 
> hadoop-0.20.1-core.jar, hadoop-core-0.20.2-cdh3u3.jar, hadoop.patch, 
> log4j-1.2.15.jar, README.txt, SOLR-1301-hadoop-0-20.patch, 
> SOLR-1301-hadoop-0-20.patch, SOLR-1301.patch, SOLR-1301.patch, 
> SOLR-1301.patch, SOLR-1301.patch, SOLR-1301.patch, SOLR-1301.patch, 
> SOLR-1301.patch, SOLR-1301.patch, SOLR-1301.patch, SOLR-1301.patch, 
> SOLR-1301.patch, SOLR-1301.patch, SOLR-1301.patch, SOLR-1301.patch, 
> SolrRecordWriter.java
>
>
> This patch contains  a contrib module that provides distributed indexing 
> (using Hadoop) to Solr EmbeddedSolrServer. The idea behind this module is 
> twofold:
> * provide an API that is familiar to Hadoop developers, i.e. that of 
> OutputFormat
> * avoid unnecessary export and (de)serialization of data maintained on HDFS. 
> SolrOutputFormat consumes data produced by reduce tasks directly, without 
> storing it in intermediate files. Furthermore, by using an 
> EmbeddedSolrServer, the indexing task is split into as many parts as there 
> are reducers, and the data to be indexed is not sent over the network.
> Design
> --
> Key/value pairs produced by reduce tasks are passed to SolrOutputFormat, 
> which in turn uses SolrRecordWriter to write this data. SolrRecordWriter 
> instantiates an EmbeddedSolrServer, and it also instantiates an 
> implementation of SolrDocumentConverter, which is responsible for turning 
> Hadoop (key, value) into a SolrInputDocument. This data is then added to a 
> batch, which is periodically submitted to EmbeddedSolrServer. When reduce 
> task completes, and the OutputFormat is closed, SolrRecordWriter calls 
> commit() and optimize() on the EmbeddedSolrServer.
> The API provides facilities to specify an arbitrary existing solr.home 
> directory, from which the conf/ and lib/ files will be taken.
> This process results in the creation of as many partial Solr home directories 
> as there were reduce tasks. The output shards are placed in the output 
> directory on the default filesystem (e.g. HDFS). Such part-N directories 
> can be used to run N shard servers. Additionally, users can specify the 
> number of reduce tasks, in particular 1 reduce task, in which case the output 
> will consist of a single shard.
> An example application is provided that processes large CSV files and uses 
> this API. It uses a custom CSV processing to avoid (de)serialization overhead.
> This patch relies on hadoop-core-0.19.1.jar - I attached the jar to this 
> issue, you should put it in contrib/hadoop/lib.
> Note: the development of this patch was sponsored by an anonymous contributor 
> and approved for release under Apache License.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-4906) PostingsHighlighter's PassageFormatter should allow for rendering to arbitrary objects

[Robert Muir (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-4906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13767528#comment-13767528
 ] 

Robert Muir commented on LUCENE-4906:
-

Well I think its good for a few reasons:
1. even though protected still keeps it in javadocs, users who use an IDE and 
type "highlighter." and wait for autocomplete see less methods in their API. so 
its still less overwhelming here. This is the most important benefit.
2. encourages users to e.g. implement their public own method with a proper 
return value (e.g. JsonObject or whatever it is you are doing). This way there 
is just one cast from object and its contained inside their custom Highlighter, 
otherwise there rest of their app is type safe.

and I think the functionality of highlighting to something *other than string* 
is sufficiently custom that its not really an imposition. Are there other 
expert methods in PH that belong in the same category?

> PostingsHighlighter's PassageFormatter should allow for rendering to 
> arbitrary objects
> --
>
> Key: LUCENE-4906
> URL: https://issues.apache.org/jira/browse/LUCENE-4906
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Michael McCandless
>Assignee: Michael McCandless
> Fix For: 5.0, 4.6
>
> Attachments: LUCENE-4906.patch, LUCENE-4906.patch, LUCENE-4906.patch
>
>
> For example, in a server, I may want to render the highlight result to 
> JsonObject to send back to the front-end. Today since we render to string, I 
> have to render to JSON string and then re-parse to JsonObject, which is 
> inefficient...
> Or, if (Rob's idea:) we make a query that's like MoreLikeThis but it pulls 
> terms from snippets instead, so you get proximity-influenced salient/expanded 
> terms, then perhaps that renders to just an array of tokens or fragments or 
> something from each snippet.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5207) lucene expressions module

[Robert Muir (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-5207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13767523#comment-13767523
 ] 

Robert Muir commented on LUCENE-5207:
-

Uwe: can we defer that function stuff to another issue?

The whole discussion about classloader stuff doesnt make a lot of sense to me. 
Today Functions are implemented with just normal invokeVirtual. Bindings aren't 
used until you actually run the query... Today the way an expression interacts 
with the Bindings is *not* with invokeVirtual, its just another valuesource 
coming in to its array at runtime. 

So I dont think custom functions should go thru Bindings, its totally unrelated 
to how that class works at all. I also don't want to make the API very 
confusing just because solr does crazy things with classloaders quite yet: I 
think this is very close to usable for lucene users as-is.


> lucene expressions module
> -
>
> Key: LUCENE-5207
> URL: https://issues.apache.org/jira/browse/LUCENE-5207
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Ryan Ernst
> Attachments: LUCENE-5207.patch
>
>
> Expressions are geared at defining an alternative ranking function (e.g. 
> incorporating the text relevance score and other field values/ranking
> signals). So they are conceptually much more like ElasticSearch's scripting 
> support (http://www.elasticsearch.org/guide/reference/modules/scripting/) 
> than solr's function queries.
> Some additional notes:
> * In addition to referring to other fields, they can also refer to other 
> expressions, so they can be used as "computed fields".
> * You can rank documents easily by multiple expressions (its a SortField at 
> the end), e.g. Sort by year descending, then some function of score price and 
> time ascending.
> * The provided javascript expression syntax is much more efficient than using 
> a scripting engine, because it does not have dynamic typing (compiles to 
> .class files that work on doubles). Performance is similar to writing a 
> custom FieldComparator yourself, but much easier to do.
> * We have solr integration to contribute in the future, but this is just the 
> standalone lucene part as a start. Since lucene has no schema, it includes an 
> implementation of Bindings (SimpleBindings) that maps variable names to 
> SortField's or other expressions.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-5234) Allow SolrResourceLoader to load resources from URLs

[Alan Woodward (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-5234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13767515#comment-13767515
 ] 

Alan Woodward commented on SOLR-5234:
-

Anyone have any comments on this?  [~hossman_luc...@fucit.org] or 
[~markrmil...@gmail.com]?

> Allow SolrResourceLoader to load resources from URLs
> 
>
> Key: SOLR-5234
> URL: https://issues.apache.org/jira/browse/SOLR-5234
> Project: Solr
>  Issue Type: Improvement
>Reporter: Alan Woodward
>Assignee: Alan Woodward
>Priority: Minor
> Attachments: SOLR-5234.patch
>
>
> This would allow multiple solr instance to share large configuration files.  
> It would also help resolve problems caused by attempting to store >1Mb files 
> in zookeeper.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-5241) SimplePostToolTest is slow on some systmes - likely due to hostname resolution of "example.com"

[Dawid Weiss (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-5241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13767484#comment-13767484
 ] 

Dawid Weiss commented on SOLR-5241:
---

Thanks Uwe.

> SimplePostToolTest is slow on some systmes - likely due to hostname 
> resolution of "example.com"
> ---
>
> Key: SOLR-5241
> URL: https://issues.apache.org/jira/browse/SOLR-5241
> Project: Solr
>  Issue Type: Bug
>Reporter: Hoss Man
>Assignee: Hoss Man
> Attachments: SOLR-5241.patch, SOLR-5241.patch
>
>
> As noted by Shai on the dev @lucene list, SimplePostToolTest is ridiculously 
> slow when he ran from ant, but only takes 1 second in his IDE.
> problem seems to be relate to the URL class attempting to response 
> "example.com"

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5210) Unit tests for LicenseCheckTask.

[Mark Miller (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-5210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13767475#comment-13767475
 ] 

Mark Miller commented on LUCENE-5210:
-

bq. In addition: I disagree with adding the LicenseCheckTask (and all tools 
folders around Lucene) to Eclipse setup. With a good antunit test this is also 
not needed.

How is that? When it's not in the eclipse setup, editing the code is annoying 
and hard - when it is, it is easy. Does antunit make it nice to edit the code? 
I'm not sure how it could.

In any case, thanks for the tip.

bq. We should maybe discover removing the java code completely and create the 
license checker as a macro like the svn working copy checks.

Personally, I think doing anything in ant over java is a terrible idea. Ant is 
terrible when it's complicated and more and more of us have to be experts in it 
to do anything. Ant is so obtuse for anything that is not super straightfoward. 
Mixing in Groovy as well really makes my stomach turn :)

bq. To me the patch looks a little bit incorrect regarding resources

I'm not sure that matters, but it can just be a test resources folder as I 
intended instead.

Basically, I think being anal about what we end up with should be a clear step 
2. Having *any* testing at all for our license checker should be the clear step 
one. Once we have any testing, it will make me sad if you make it all ant and 
really hard for anyone else to figure out, but I don't really care too much as 
long as our very important LiceneCheckTask has tests - right now it's flying 
blind.

> Unit tests for LicenseCheckTask.
> 
>
> Key: LUCENE-5210
> URL: https://issues.apache.org/jira/browse/LUCENE-5210
> Project: Lucene - Core
>  Issue Type: Test
>  Components: general/build
>Reporter: Mark Miller
> Attachments: LUCENE-5210.patch, LUCENE-5210.patch
>
>
> While working on LUCENE-5209, I noticed the LicenseCheckTask is kind of a 
> second class citizen - excluded from UI src folder setup and with no units 
> tests. This was a little scary to me.
> I've started adding some units tests. So far I have mainly just done the 
> lifting of getting units tests to work as part of tools.
> I have added two super simple tests - really just the start - but something 
> to build on.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-4906) PostingsHighlighter's PassageFormatter should allow for rendering to arbitrary objects

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-4906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13767469#comment-13767469
 ] 

ASF subversion and git services commented on LUCENE-4906:
-

Commit 1523226 from [~mikemccand] in branch 'dev/branches/branch_4x'
[ https://svn.apache.org/r1523226 ]

LUCENE-4906: make expert Object method protected

> PostingsHighlighter's PassageFormatter should allow for rendering to 
> arbitrary objects
> --
>
> Key: LUCENE-4906
> URL: https://issues.apache.org/jira/browse/LUCENE-4906
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Michael McCandless
>Assignee: Michael McCandless
> Fix For: 5.0, 4.6
>
> Attachments: LUCENE-4906.patch, LUCENE-4906.patch, LUCENE-4906.patch
>
>
> For example, in a server, I may want to render the highlight result to 
> JsonObject to send back to the front-end. Today since we render to string, I 
> have to render to JSON string and then re-parse to JsonObject, which is 
> inefficient...
> Or, if (Rob's idea:) we make a query that's like MoreLikeThis but it pulls 
> terms from snippets instead, so you get proximity-influenced salient/expanded 
> terms, then perhaps that renders to just an array of tokens or fragments or 
> something from each snippet.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-4906) PostingsHighlighter's PassageFormatter should allow for rendering to arbitrary objects

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-4906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13767467#comment-13767467
 ] 

ASF subversion and git services commented on LUCENE-4906:
-

Commit 1523225 from [~mikemccand] in branch 'dev/trunk'
[ https://svn.apache.org/r1523225 ]

LUCENE-4906: make expert Object method protected

> PostingsHighlighter's PassageFormatter should allow for rendering to 
> arbitrary objects
> --
>
> Key: LUCENE-4906
> URL: https://issues.apache.org/jira/browse/LUCENE-4906
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Michael McCandless
>Assignee: Michael McCandless
> Fix For: 5.0, 4.6
>
> Attachments: LUCENE-4906.patch, LUCENE-4906.patch, LUCENE-4906.patch
>
>
> For example, in a server, I may want to render the highlight result to 
> JsonObject to send back to the front-end. Today since we render to string, I 
> have to render to JSON string and then re-parse to JsonObject, which is 
> inefficient...
> Or, if (Rob's idea:) we make a query that's like MoreLikeThis but it pulls 
> terms from snippets instead, so you get proximity-influenced salient/expanded 
> terms, then perhaps that renders to just an array of tokens or fragments or 
> something from each snippet.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5207) lucene expressions module

[Uwe Schindler (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-5207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13767464#comment-13767464
 ] 

Uwe Schindler commented on LUCENE-5207:
---

I think the branch is now in a quite good state.

I thought a little bit about extensibility of Javascript functions: The current 
code is more or less "hardcoded" (although its configureable by a resource file 
in the JAR). But this is not extensible by users (all private, you have to 
rebuild JAR).

My idea would be to extend Bindings to also allow to get "methods". So one 
could register a java.lang.reflect.Method binding. The code would check that it 
only accepts "double" as parameters and returns a "double". Solr could then use 
this to register stuff like "haversine". The compiler would be extended to 
first try to get a binding for the function name (if it is a Method instance) 
otherwise it falls back to the builtins.

Another thing to discuss (which is not yet a problem): If we allow function 
binding and e.g. a Solr contrib from SolrResourceLoader would register a 
function, the generated bytecode would fail to find the function: This is 
because we use the classloader of the JavaScript module not the one of the 
caller. We should either make this configureable or (I like this better): If 
somebody registers a custom function we should include this into the 
classloader, so the custom function impl class can be found by the JVM:
- the classloader of the custom function must be at least a child classloader 
of the function module, otherwise it won't work at all. If all registered 
custom functions don't have a common classloader hierarchy, throw exception.
- the child classloader check function is already part of Lucene, it is used 
when loading SPIs.

If we make the classloader configureable (like it was in the original patch), 
we must also check that our own classloader is a parent of the given one.

> lucene expressions module
> -
>
> Key: LUCENE-5207
> URL: https://issues.apache.org/jira/browse/LUCENE-5207
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Ryan Ernst
> Attachments: LUCENE-5207.patch
>
>
> Expressions are geared at defining an alternative ranking function (e.g. 
> incorporating the text relevance score and other field values/ranking
> signals). So they are conceptually much more like ElasticSearch's scripting 
> support (http://www.elasticsearch.org/guide/reference/modules/scripting/) 
> than solr's function queries.
> Some additional notes:
> * In addition to referring to other fields, they can also refer to other 
> expressions, so they can be used as "computed fields".
> * You can rank documents easily by multiple expressions (its a SortField at 
> the end), e.g. Sort by year descending, then some function of score price and 
> time ascending.
> * The provided javascript expression syntax is much more efficient than using 
> a scripting engine, because it does not have dynamic typing (compiles to 
> .class files that work on doubles). Performance is similar to writing a 
> custom FieldComparator yourself, but much easier to do.
> * We have solr integration to contribute in the future, but this is just the 
> standalone lucene part as a start. Since lucene has no schema, it includes an 
> implementation of Bindings (SimpleBindings) that maps variable names to 
> SortField's or other expressions.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5213) Use instead ant task

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-5213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13767457#comment-13767457
 ] 

ASF subversion and git services commented on LUCENE-5213:
-

Commit 1523215 from [~thetaphi] in branch 'dev/branches/branch_4x'
[ https://svn.apache.org/r1523215 ]

Merged revision(s) 1523214 from lucene/dev/trunk:
LUCENE-5213: Use  instead  ant task

> Use  instead  ant task
> 
>
> Key: LUCENE-5213
> URL: https://issues.apache.org/jira/browse/LUCENE-5213
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: general/build
>Reporter: Uwe Schindler
>Assignee: Uwe Schindler
> Fix For: 5.0, 4.6
>
> Attachments: LUCENE-5213.patch
>
>
> We currently use the  task to execute groovy script while building 
> (SVN working copy tests, determining javadocs URL,...), which works perfectly 
> fine.
> The backside is that the groovy classpath is passed as separate classpath to 
> ANT every time a  task is executed. This leads to the problem that 
> we get the usual ANT permgen problems, especially when we execute such tasks 
> on sub.modules (we currently don't do this).
> This is one reason why I did not yet port over the "svnversion" calls to 
> svnkit, because its executed on every module. The license checker may have 
> the same problem, once I port it over to be a simple groovy macro (see 
> LUCENE-5210).
> This patch uses the  taskdef. The taskdef is only loaded once and 
> can be inherited to subants.
> There is currently one task/macro that does not use the  task: 
> pegdown, because its implemented as a : It is used to filter 
> the markdown and convert to HTML when copying the main web page. There is no 
> replacement with groovy's taskdef unless I refactor it to not be a  
> inside a  task.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (LUCENE-5213) Use instead ant task

[Uwe Schindler (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-5213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Uwe Schindler resolved LUCENE-5213.
---

Resolution: Fixed

> Use  instead  ant task
> 
>
> Key: LUCENE-5213
> URL: https://issues.apache.org/jira/browse/LUCENE-5213
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: general/build
>Reporter: Uwe Schindler
>Assignee: Uwe Schindler
> Fix For: 5.0, 4.6
>
> Attachments: LUCENE-5213.patch
>
>
> We currently use the  task to execute groovy script while building 
> (SVN working copy tests, determining javadocs URL,...), which works perfectly 
> fine.
> The backside is that the groovy classpath is passed as separate classpath to 
> ANT every time a  task is executed. This leads to the problem that 
> we get the usual ANT permgen problems, especially when we execute such tasks 
> on sub.modules (we currently don't do this).
> This is one reason why I did not yet port over the "svnversion" calls to 
> svnkit, because its executed on every module. The license checker may have 
> the same problem, once I port it over to be a simple groovy macro (see 
> LUCENE-5210).
> This patch uses the  taskdef. The taskdef is only loaded once and 
> can be inherited to subants.
> There is currently one task/macro that does not use the  task: 
> pegdown, because its implemented as a : It is used to filter 
> the markdown and convert to HTML when copying the main web page. There is no 
> replacement with groovy's taskdef unless I refactor it to not be a  
> inside a  task.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5213) Use instead ant task

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-5213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13767456#comment-13767456
 ] 

ASF subversion and git services commented on LUCENE-5213:
-

Commit 1523214 from [~thetaphi] in branch 'dev/trunk'
[ https://svn.apache.org/r1523214 ]

LUCENE-5213: Use  instead  ant task

> Use  instead  ant task
> 
>
> Key: LUCENE-5213
> URL: https://issues.apache.org/jira/browse/LUCENE-5213
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: general/build
>Reporter: Uwe Schindler
>Assignee: Uwe Schindler
> Fix For: 5.0, 4.6
>
> Attachments: LUCENE-5213.patch
>
>
> We currently use the  task to execute groovy script while building 
> (SVN working copy tests, determining javadocs URL,...), which works perfectly 
> fine.
> The backside is that the groovy classpath is passed as separate classpath to 
> ANT every time a  task is executed. This leads to the problem that 
> we get the usual ANT permgen problems, especially when we execute such tasks 
> on sub.modules (we currently don't do this).
> This is one reason why I did not yet port over the "svnversion" calls to 
> svnkit, because its executed on every module. The license checker may have 
> the same problem, once I port it over to be a simple groovy macro (see 
> LUCENE-5210).
> This patch uses the  taskdef. The taskdef is only loaded once and 
> can be inherited to subants.
> There is currently one task/macro that does not use the  task: 
> pegdown, because its implemented as a : It is used to filter 
> the markdown and convert to HTML when copying the main web page. There is no 
> replacement with groovy's taskdef unless I refactor it to not be a  
> inside a  task.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5213) Use instead ant task

[Uwe Schindler (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-5213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13767455#comment-13767455
 ] 

Uwe Schindler commented on LUCENE-5213:
---

I will commit and resolve this soon.

> Use  instead  ant task
> 
>
> Key: LUCENE-5213
> URL: https://issues.apache.org/jira/browse/LUCENE-5213
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: general/build
>Reporter: Uwe Schindler
>Assignee: Uwe Schindler
> Fix For: 5.0, 4.6
>
> Attachments: LUCENE-5213.patch
>
>
> We currently use the  task to execute groovy script while building 
> (SVN working copy tests, determining javadocs URL,...), which works perfectly 
> fine.
> The backside is that the groovy classpath is passed as separate classpath to 
> ANT every time a  task is executed. This leads to the problem that 
> we get the usual ANT permgen problems, especially when we execute such tasks 
> on sub.modules (we currently don't do this).
> This is one reason why I did not yet port over the "svnversion" calls to 
> svnkit, because its executed on every module. The license checker may have 
> the same problem, once I port it over to be a simple groovy macro (see 
> LUCENE-5210).
> This patch uses the  taskdef. The taskdef is only loaded once and 
> can be inherited to subants.
> There is currently one task/macro that does not use the  task: 
> pegdown, because its implemented as a : It is used to filter 
> the markdown and convert to HTML when copying the main web page. There is no 
> replacement with groovy's taskdef unless I refactor it to not be a  
> inside a  task.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (LUCENE-5213) Use instead ant task

[Uwe Schindler (JIRA)]

Uwe Schindler created LUCENE-5213:
-

 Summary: Use  instead  ant task
 Key: LUCENE-5213
 URL: https://issues.apache.org/jira/browse/LUCENE-5213
 Project: Lucene - Core
  Issue Type: Bug
  Components: general/build
Reporter: Uwe Schindler
Assignee: Uwe Schindler
 Fix For: 5.0, 4.6
 Attachments: LUCENE-5213.patch

We currently use the  task to execute groovy script while building 
(SVN working copy tests, determining javadocs URL,...), which works perfectly 
fine.

The backside is that the groovy classpath is passed as separate classpath to 
ANT every time a  task is executed. This leads to the problem that we 
get the usual ANT permgen problems, especially when we execute such tasks on 
sub.modules (we currently don't do this).

This is one reason why I did not yet port over the "svnversion" calls to 
svnkit, because its executed on every module. The license checker may have the 
same problem, once I port it over to be a simple groovy macro (see LUCENE-5210).

This patch uses the  taskdef. The taskdef is only loaded once and can 
be inherited to subants.

There is currently one task/macro that does not use the  task: 
pegdown, because its implemented as a : It is used to filter the 
markdown and convert to HTML when copying the main web page. There is no 
replacement with groovy's taskdef unless I refactor it to not be a  
inside a  task.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-5213) Use instead ant task

[Uwe Schindler (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-5213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Uwe Schindler updated LUCENE-5213:
--

Attachment: LUCENE-5213.patch

Simple patch.

> Use  instead  ant task
> 
>
> Key: LUCENE-5213
> URL: https://issues.apache.org/jira/browse/LUCENE-5213
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: general/build
>Reporter: Uwe Schindler
>Assignee: Uwe Schindler
> Fix For: 5.0, 4.6
>
> Attachments: LUCENE-5213.patch
>
>
> We currently use the  task to execute groovy script while building 
> (SVN working copy tests, determining javadocs URL,...), which works perfectly 
> fine.
> The backside is that the groovy classpath is passed as separate classpath to 
> ANT every time a  task is executed. This leads to the problem that 
> we get the usual ANT permgen problems, especially when we execute such tasks 
> on sub.modules (we currently don't do this).
> This is one reason why I did not yet port over the "svnversion" calls to 
> svnkit, because its executed on every module. The license checker may have 
> the same problem, once I port it over to be a simple groovy macro (see 
> LUCENE-5210).
> This patch uses the  taskdef. The taskdef is only loaded once and 
> can be inherited to subants.
> There is currently one task/macro that does not use the  task: 
> pegdown, because its implemented as a : It is used to filter 
> the markdown and convert to HTML when copying the main web page. There is no 
> replacement with groovy's taskdef unless I refactor it to not be a  
> inside a  task.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-4906) PostingsHighlighter's PassageFormatter should allow for rendering to arbitrary objects

[Michael McCandless (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-4906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13767451#comment-13767451
 ] 

Michael McCandless commented on LUCENE-4906:


bq. Can we fix the visibility of this method to be protected?

So that apps that want to use it must subclass PostingsHighlighter?  OK.

> PostingsHighlighter's PassageFormatter should allow for rendering to 
> arbitrary objects
> --
>
> Key: LUCENE-4906
> URL: https://issues.apache.org/jira/browse/LUCENE-4906
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Michael McCandless
>Assignee: Michael McCandless
> Fix For: 5.0, 4.6
>
> Attachments: LUCENE-4906.patch, LUCENE-4906.patch, LUCENE-4906.patch
>
>
> For example, in a server, I may want to render the highlight result to 
> JsonObject to send back to the front-end. Today since we render to string, I 
> have to render to JSON string and then re-parse to JsonObject, which is 
> inefficient...
> Or, if (Rob's idea:) we make a query that's like MoreLikeThis but it pulls 
> terms from snippets instead, so you get proximity-influenced salient/expanded 
> terms, then perhaps that renders to just an array of tokens or fragments or 
> something from each snippet.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5210) Unit tests for LicenseCheckTask.

[Uwe Schindler (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-5210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13767441#comment-13767441
 ] 

Uwe Schindler commented on LUCENE-5210:
---

I while ago I was also discussing with Dawid and Robert about the 
LicenseChecker in general. At the time it was written, we had not yet 
experience with using scripting inside ANT.

We should maybe discover removing the java code completely and create the 
license checker as a macro like the svn working copy checks. We already have 
the framework in common-buil.xml to load the groovy scripting engine, porting 
over the java code to be a macro should be very simple (it is mostly copy/paste 
and some ANT magic). We can then also test the checker with Antunit, if we 
like. The simplicity of this checker does not rectify having the 
compile/classpath/ANT overhead. A simple "resolve-grovy" and a macro should do 
the work.

Can we keep this issue open for a while until I am back so I can check?

About your patch: To me the patch looks a little bit incorrect regarding 
resources: It creates a src/resources folder, which is per convention only for 
resources shipped with the JAR file. Test resources are currently next to the 
Test .java files. This is not an issue for the tool, because we dont package 
them as JAR file, but its still wrong, because the resources are available to 
the ANT task when its running "in production".

> Unit tests for LicenseCheckTask.
> 
>
> Key: LUCENE-5210
> URL: https://issues.apache.org/jira/browse/LUCENE-5210
> Project: Lucene - Core
>  Issue Type: Test
>  Components: general/build
>Reporter: Mark Miller
> Attachments: LUCENE-5210.patch, LUCENE-5210.patch
>
>
> While working on LUCENE-5209, I noticed the LicenseCheckTask is kind of a 
> second class citizen - excluded from UI src folder setup and with no units 
> tests. This was a little scary to me.
> I've started adding some units tests. So far I have mainly just done the 
> lifting of getting units tests to work as part of tools.
> I have added two super simple tests - really just the start - but something 
> to build on.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5207) lucene expressions module

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-5207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13767436#comment-13767436
 ] 

ASF subversion and git services commented on LUCENE-5207:
-

Commit 1523204 from [~thetaphi] in branch 'dev/branches/lucene5207'
[ https://svn.apache.org/r1523204 ]

LUCENE-5207: Throw correct exception in JavascriptFunction ctor

> lucene expressions module
> -
>
> Key: LUCENE-5207
> URL: https://issues.apache.org/jira/browse/LUCENE-5207
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Ryan Ernst
> Attachments: LUCENE-5207.patch
>
>
> Expressions are geared at defining an alternative ranking function (e.g. 
> incorporating the text relevance score and other field values/ranking
> signals). So they are conceptually much more like ElasticSearch's scripting 
> support (http://www.elasticsearch.org/guide/reference/modules/scripting/) 
> than solr's function queries.
> Some additional notes:
> * In addition to referring to other fields, they can also refer to other 
> expressions, so they can be used as "computed fields".
> * You can rank documents easily by multiple expressions (its a SortField at 
> the end), e.g. Sort by year descending, then some function of score price and 
> time ascending.
> * The provided javascript expression syntax is much more efficient than using 
> a scripting engine, because it does not have dynamic typing (compiles to 
> .class files that work on doubles). Performance is similar to writing a 
> custom FieldComparator yourself, but much easier to do.
> * We have solr integration to contribute in the future, but this is just the 
> standalone lucene part as a start. Since lucene has no schema, it includes an 
> implementation of Bindings (SimpleBindings) that maps variable names to 
> SortField's or other expressions.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5210) Unit tests for LicenseCheckTask.

[Uwe Schindler (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-5210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13767429#comment-13767429
 ] 

Uwe Schindler commented on LUCENE-5210:
---

In addition: I disagree with adding the LicenseCheckTask (and all tools folders 
around Lucene) to Eclipse setup. With a good antunit test this is also not 
needed.

> Unit tests for LicenseCheckTask.
> 
>
> Key: LUCENE-5210
> URL: https://issues.apache.org/jira/browse/LUCENE-5210
> Project: Lucene - Core
>  Issue Type: Test
>  Components: general/build
>Reporter: Mark Miller
> Attachments: LUCENE-5210.patch, LUCENE-5210.patch
>
>
> While working on LUCENE-5209, I noticed the LicenseCheckTask is kind of a 
> second class citizen - excluded from UI src folder setup and with no units 
> tests. This was a little scary to me.
> I've started adding some units tests. So far I have mainly just done the 
> lifting of getting units tests to work as part of tools.
> I have added two super simple tests - really just the start - but something 
> to build on.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-5241) SimplePostToolTest is slow on some systmes - likely due to hostname resolution of "example.com"

[Uwe Schindler (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-5241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13767421#comment-13767421
 ] 

Uwe Schindler commented on SOLR-5241:
-

>From analyzing the test:
The main probelm is just the DNS resolve. We can really use any IP (IPv6 or 
IPv4) here, because the stream handler never actually connects to anywhere. 
Theoretically we could also use 127.0.0.1, the blackhole is not related here, 
because it just looks up hostnames. The actual connection is prevented by the 
stream handler. So finally we can use any numeric IP, just no host name.

> SimplePostToolTest is slow on some systmes - likely due to hostname 
> resolution of "example.com"
> ---
>
> Key: SOLR-5241
> URL: https://issues.apache.org/jira/browse/SOLR-5241
> Project: Solr
>  Issue Type: Bug
>Reporter: Hoss Man
>Assignee: Hoss Man
> Attachments: SOLR-5241.patch, SOLR-5241.patch
>
>
> As noted by Shai on the dev @lucene list, SimplePostToolTest is ridiculously 
> slow when he ran from ant, but only takes 1 second in his IDE.
> problem seems to be relate to the URL class attempting to response 
> "example.com"

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-5241) SimplePostToolTest is slow on some systmes - likely due to hostname resolution of "example.com"

[Uwe Schindler (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-5241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13767411#comment-13767411
 ] 

Uwe Schindler commented on SOLR-5241:
-

Hi,
as I said in response to Shai's e-mail: The problem may indeed be related to 
the security policy. To check if a connection is allowed, the security manager 
has to resolve dns. As this gets a not found or whatever error, it will not 
throw a security exception. This also explains why there is a runtime env 
difference: With Eclipse the security manager is not used, with ANT it is.
The fix should be not to use hostnames for invalid URLs and use the same 
strategy like in Solr Cloud tests: Use a non-routeable IPv6 address. To me the 
problem is exactly what Robert and I thought is the reason.

See BaseDistributedTestCase:
{code:java}
this.deadServers = new String[] {"[ff01::114]:2" + context, 
 "[ff01::083]:2" + context, 
 "[ff01::213]:2" + context};
{code}

Please use these URLs and the problem should be gone. Real DNS names out of our 
control are bad to emulate failures. What happens if soebody links example.com 
to real IP adresses?

> SimplePostToolTest is slow on some systmes - likely due to hostname 
> resolution of "example.com"
> ---
>
> Key: SOLR-5241
> URL: https://issues.apache.org/jira/browse/SOLR-5241
> Project: Solr
>  Issue Type: Bug
>Reporter: Hoss Man
>Assignee: Hoss Man
> Attachments: SOLR-5241.patch, SOLR-5241.patch
>
>
> As noted by Shai on the dev @lucene list, SimplePostToolTest is ridiculously 
> slow when he ran from ant, but only takes 1 second in his IDE.
> problem seems to be relate to the URL class attempting to response 
> "example.com"

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5212) java 7u40 causes sigsegv and corrupt term vectors

[Robert Muir (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-5212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13767408#comment-13767408
 ] 

Robert Muir commented on LUCENE-5212:
-

I'm not complaining, I'm happy we are able to reproduce the 'whole run' in any 
shape or form. one extra command does not bother me, we shouldnt slow all of 
our jenkins jobs down to make jvm bug-debugging easier :)

> java 7u40 causes sigsegv and corrupt term vectors
> -
>
> Key: LUCENE-5212
> URL: https://issues.apache.org/jira/browse/LUCENE-5212
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Robert Muir
> Attachments: hs_err_pid32714.log, jenkins.txt
>
>


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5212) java 7u40 causes sigsegv and corrupt term vectors

[Dawid Weiss (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-5212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13767407#comment-13767407
 ] 

Dawid Weiss commented on LUCENE-5212:
-

> (sorry, dynamicAssignmentRatio=0 doesnt quite work),

The lack of determinism may be due to work stealing -- dynamicAssignmentRatio=0 
preallocates tasks to JVMs, while work stealing *may* actually assign 
differently on every run.

Perhaps we should run jenkins jobs with dynamicAssignmentRatio=0... but then 
all the benefits of work time balancing among cores would be gone and fixed 
task assignment only helps in very specific bug cases (inter-suite dependency 
or JVM bugs). Hard to tell.

> java 7u40 causes sigsegv and corrupt term vectors
> -
>
> Key: LUCENE-5212
> URL: https://issues.apache.org/jira/browse/LUCENE-5212
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Robert Muir
> Attachments: hs_err_pid32714.log, jenkins.txt
>
>


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-5241) SimplePostToolTest is slow on some systmes - likely due to hostname resolution of "example.com"

[Dawid Weiss (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-5241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13767404#comment-13767404
 ] 

Dawid Weiss commented on SOLR-5241:
---

While I appreciate Hoss's solution I think what Robert said -- we should get to 
the bottom of this problem, this difference in runtime is not easily explained 
and it may lead to a bigger can of worms.

> SimplePostToolTest is slow on some systmes - likely due to hostname 
> resolution of "example.com"
> ---
>
> Key: SOLR-5241
> URL: https://issues.apache.org/jira/browse/SOLR-5241
> Project: Solr
>  Issue Type: Bug
>Reporter: Hoss Man
>Assignee: Hoss Man
> Attachments: SOLR-5241.patch, SOLR-5241.patch
>
>
> As noted by Shai on the dev @lucene list, SimplePostToolTest is ridiculously 
> slow when he ran from ant, but only takes 1 second in his IDE.
> problem seems to be relate to the URL class attempting to response 
> "example.com"

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5210) Unit tests for LicenseCheckTask.

[Uwe Schindler (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-5210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13767400#comment-13767400
 ] 

Uwe Schindler commented on LUCENE-5210:
---

Hi,
to make tests easy there is antunit. I use this to test forbidden-apis. It is 
registered as a separate task in ANT and you write a build file that works as 
test. It can check inner tests and apply assertions on the output, file 
state,...

This would reduce the 2 simple tests to one build file that is executed as a 
subant task from the main build file. No Junit needed.

See as example: 
[https://code.google.com/p/forbidden-apis/source/browse/trunk#trunk%2Fsrc%2Ftest%2Fantunit]
It is called from here: 
[https://code.google.com/p/forbidden-apis/source/browse/trunk/build.xml?spec=svn207&r=207#346]

I can help with implementing this (unfortunately I am busy next week), it 
should be mostly copy/paste of the test setup in forbidden's build.xml and 
reuse some simple tests.

> Unit tests for LicenseCheckTask.
> 
>
> Key: LUCENE-5210
> URL: https://issues.apache.org/jira/browse/LUCENE-5210
> Project: Lucene - Core
>  Issue Type: Test
>  Components: general/build
>Reporter: Mark Miller
> Attachments: LUCENE-5210.patch, LUCENE-5210.patch
>
>
> While working on LUCENE-5209, I noticed the LicenseCheckTask is kind of a 
> second class citizen - excluded from UI src folder setup and with no units 
> tests. This was a little scary to me.
> I've started adding some units tests. So far I have mainly just done the 
> lifting of getting units tests to work as part of tools.
> I have added two super simple tests - really just the start - but something 
> to build on.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

RE: SimplePostToolTest very slow

[Uwe Schindler]

Hallo Shai,

 

the main problem with any security manager is: To check if a connection is 
allowed, it has to resolve DNS and look the IP up in the policy.

 

As Robert said before: The fix is easy: To emulate a broken URL use one of the 
non-routeable local IPv6 prefixes. We do this in Solr tests already to emulate 
broken cloud slaves, see the abstract solr cloud test class. We should use one 
of these URLs instead of example.com. I disagree with changing the security 
policy at all, because it helps to fix broken tests (like this one *g*) and 
prevents external hackers to overtake my own test instance, because Solr runs 
on 0.0.0.0, so while running tests you can easily connect to solr from 
internet. The security manager prevents this, too.

 

Uwe

 

-

Uwe Schindler

H.-H.-Meier-Allee 63, D-28213 Bremen

  http://www.thetaphi.de

eMail: u...@thetaphi.de

 

From: Shai Erera [mailto:ser...@gmail.com] 
Sent: Saturday, September 14, 2013 6:32 AM
To: dev@lucene.apache.org
Subject: Re: SimplePostToolTest very slow

 

OK, I think I've made some progress -- configuration issue, but not proxies, it 
seems to be our security manager/policy.

I printed system properties and env in setUp and ran only a single test 
(testIsOn) to compare the output. I didn't notice any proxy settings, but I did 
notice that from Ant we're using our own security manager and policy. So when I 
ran the test from eclipse using 
-Djava.security.manager=org.apache.lucene.util.TestSecurityManager 
-Djava.security.policy=D:\dev\lucene\lucene-trunk\lucene\tools\junit4\tests.policy,
 it ran for 23s too (I only ran a single test method).

I don't think it's related to TestSecurityManager, since it only checks that 
System.exit isn't called from tests.
Looking at tests.policy, there are a bunch of socket connection related lines 
.. I'm guess it's somewhere there, though I don't know this stuff.

I will try to look into it more later.

Shai

 

On Sat, Sep 14, 2013 at 6:59 AM, Shai Erera  wrote:

I still don't see why you'd get different timings between Eclipse and
Ant if you're running with the same VM -- it should be pretty
consistent (either it caches dns lookups or it doesn't).

 

I agree, it's suspicious. I searched for URL performance differences between 
eclipse and Ant and hit this page: http://ant.apache.org/manual/proxy.html. It 
suggests Ant uses different proxy settings by default for its own tasks as well 
as 3rd party tasks that use java.net.URL. I tried running with -autoproxy but 
from Ant each test method still takes ~23s vs Eclipse which is ~0.1s.

Will be interesting to identify the differences .. I think it's a configuration 
issue, as Eclipse needs to make URL connections for e.g. its marketplace, so 
maybe it comes pre-configured somehow. I'm now curious, I'll dig :).

Shai

 

On Sat, Sep 14, 2013 at 12:33 AM, Chris Hostetter  
wrote:


: If you want to experiment, a really trivial test is below -- on my system,

: there is a ~5 second gap between each pair of "INIT" and "H1" timestamps,
: but no other odd gaps in subsequent timestamps -- suggesting no caching of
: DNS per hostname, but that the URL class doesn't "re-lookup" on subsequent
: hashCode() calls.

Or maybe i could actually cut & paste the test this time...


import java.net.URL;
import org.apache.lucene.util.LuceneTestCase;
public class TestURLDNSAbsurdity extends LuceneTestCase {

  public void testHowSlowCanYouGo() throws Exception {
go("1");
go("2");
  }
  public void go(String s) throws Exception {
System.out.println(s + " PRE: " + System.currentTimeMillis());
URL url = new URL("http://example.com/";);
System.out.println(s + "INIT: " + System.currentTimeMillis());
int h1 = url.hashCode();
System.out.println(s + "  H1: " + System.currentTimeMillis());
int h2 = url.hashCode();
System.out.println(s + "  H2: " + System.currentTimeMillis());
boolean e1 = url.equals(this);
System.out.println(s + "  E1: " + System.currentTimeMillis());
boolean e2 = url.equals(this);
System.out.println(s + "  E2: " + System.currentTimeMillis());
assertEquals(h1,h2);
assertEquals(e1,e2);
  }
}

...

   [junit4] Started J0 PID(31843@frisbee).
   [junit4] Suite: org.apache.solr.util.TestURLDNSAbsurdity
   [junit4]   1> 1 PRE: 1379107971313
   [junit4]   1> 1INIT: 1379107971314
   [junit4]   1> 1  H1: 1379107976449
   [junit4]   1> 1  H2: 1379107976449
   [junit4]   1> 1  E1: 1379107976449
   [junit4]   1> 1  E2: 1379107976449
   [junit4]   1> 2 PRE: 1379107976450
   [junit4]   1> 2INIT: 1379107976450
   [junit4]   1> 2  H1: 1379107981510
   [junit4]   1> 2  H2: 1379107981510
   [junit4]   1> 2  E1: 1379107981510
   [junit4]   1> 2  E2: 1379107981510
   [junit4] OK  10.3s | TestURLDNSAbsurdity.testHowSlowCanYouGo





-Hoss

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e

[jira] [Updated] (LUCENE-5212) java 7u40 causes sigsegv and corrupt term vectors

[Robert Muir (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-5212?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Robert Muir updated LUCENE-5212:


Attachment: jenkins.txt

attached is original jenkins log (they get removed from the server pretty fast).

in jenkins on os X, it didnt crash but had corrupt term vectors in several 
unrelated tests.

> java 7u40 causes sigsegv and corrupt term vectors
> -
>
> Key: LUCENE-5212
> URL: https://issues.apache.org/jira/browse/LUCENE-5212
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Robert Muir
> Attachments: hs_err_pid32714.log, jenkins.txt
>
>


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5212) java 7u40 causes sigsegv and corrupt term vectors

[Robert Muir (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-5212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13767395#comment-13767395
 ] 

Robert Muir commented on LUCENE-5212:
-

The other seed i cant reproduce:

(from lucene/classification: ant test -Dtests.seed=68F81CB00E47182B 
-Dtests.jvms=2 -Dtests.dynamicAssignmentRatio=0 -Dargs="-XX:-UseCompressedOops 
-XX:+UseG1GC")

Note you have to rm -rf your .caches (sorry, dynamicAssignmentRatio=0 doesnt 
quite work), or the tests wont be executed in the same order as jenkins did. 

Still i cant reproduce that one, but the later seed above in core tests is 
easier anyway since it only launches 1 JVM and it SIGSEGVs.

> java 7u40 causes sigsegv and corrupt term vectors
> -
>
> Key: LUCENE-5212
> URL: https://issues.apache.org/jira/browse/LUCENE-5212
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Robert Muir
> Attachments: hs_err_pid32714.log
>
>


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-5167) Ability to use AnalyzingInfixSuggester in Solr

[Areek Zillur (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-5167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13767393#comment-13767393
 ] 

Areek Zillur commented on SOLR-5167:


To answer Varun's question, the storeDir parameter for the 
AnalyzingInfixSuggestions should be null, the reason being that this analyzer 
does not store the dictionary at the suggestor level, rather manages building 
and loading the index that gets created on its own. hence the store() and 
load() should return false (this is what gets used by the Suggester to load any 
persistent dictionary).

> Ability to use AnalyzingInfixSuggester in Solr
> --
>
> Key: SOLR-5167
> URL: https://issues.apache.org/jira/browse/SOLR-5167
> Project: Solr
>  Issue Type: New Feature
>  Components: SearchComponents - other
>Reporter: Varun Thacker
>Priority: Minor
> Fix For: 4.5, 5.0
>
> Attachments: SOLR-5167.patch, SOLR-5167.patch
>
>
> We should be able to use AnalyzingInfixSuggester in Solr by defining it in 
> solrconfig.xml

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5212) java 7u40 causes sigsegv and corrupt term vectors

[Robert Muir (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-5212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13767392#comment-13767392
 ] 

Robert Muir commented on LUCENE-5212:
-

i ran this 3 times... first 2 times SIGSEGV, third time it passed. so its not 
totally consistent but seems to reproduce.

I will now try to reproduce the other seed from this morning.

> java 7u40 causes sigsegv and corrupt term vectors
> -
>
> Key: LUCENE-5212
> URL: https://issues.apache.org/jira/browse/LUCENE-5212
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Robert Muir
> Attachments: hs_err_pid32714.log
>
>


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org