[jira] [Updated] (LUCENE-5692) Deprecate spatial DisjointSpatialFilter

[David Smiley (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-5692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

David Smiley updated LUCENE-5692:
-

Attachment: 
LUCENE-5692_Deprecate_DisjointSpatialFilter__RPT_stops_using_it.patch

> Deprecate spatial DisjointSpatialFilter
> ---
>
> Key: LUCENE-5692
> URL: https://issues.apache.org/jira/browse/LUCENE-5692
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: modules/spatial
>Affects Versions: 5.0
>Reporter: David Smiley
> Attachments: 
> LUCENE-5692_Deprecate_DisjointSpatialFilter__RPT_stops_using_it.patch
>
>
> The spatial predicate "IsDisjointTo" is almost the same as the inverse of 
> "Intersects", except that it shouldn't match documents without spatial data.  
> In another sense it's as if the query shape were inverted.
> DisjointSpatialFilter is a utility filter that works (or worked, rather) by 
> using the FieldCache to see which documents have spatial data 
> (getDocsWithField()). Calculating that was probably very slow but it was at 
> least cacheable. Since LUCENE-5666 (v5/trunk only), Rob replaced this to use 
> DocValues.  However for some SpatialStrategies (PrefixTree based) it wouldn't 
> make any sense to use DocValues *just* so that at search time you could call 
> getDocsWithField() when there's no other need for the un-inversion (e.g. no 
> need to lookup terms by document).
> Perhaps an immediate fix is simply to revert the change made to 
> DisjointSpatialFilter so that it uses the FieldCache again, if that works 
> (though it's not public?).  But stepping back a bit, this 
> DisjointSpatialFilter is really something unfortunate that doesn't work as 
> well as it could because it's not at the level of Solr or ES -- that is, 
> there's no access to a filter-cache.  So I propose I simply remove it, and if 
> a user wants to do this for real, they should index a boolean field marking 
> wether there's spatial data and then combine that with a NOT and Intersects, 
> in a straight-forward way.  
> Alternatively, some sort of inverting query shape could be developed, 
> although it wouldn't work with the SpatialPrefixTree technique because there 
> is no edge distinction -- the edge matches normally and notwithstanding 
> changes to RPT algorithms it would also match the edge of an inverted shape.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (SOLR-4186) QueryComponent NPE

[Shalin Shekhar Mangar (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-4186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shalin Shekhar Mangar resolved SOLR-4186.
-

   Resolution: Fixed
Fix Version/s: (was: 4.9)
   (was: 5.0)

> QueryComponent NPE
> --
>
> Key: SOLR-4186
> URL: https://issues.apache.org/jira/browse/SOLR-4186
> Project: Solr
>  Issue Type: Bug
>  Components: search, SolrCloud
>Affects Versions: 4.0, 5.0
> Environment: recent trunk exports
>Reporter: Markus Jelsma
>
> Similar to SOLR-4049, the QueryComponent throws the following NPE in case a 
> distributed search fails for some reason. This bug has been around for at 
> least one or two weeks.
> {code}
> 2012-12-14 10:00:41,611 ERROR [solr.servlet.SolrDispatchFilter] - 
> [http-8080-exec-5] - : null:java.lang.NullPointerException
> at 
> org.apache.solr.handler.component.QueryComponent.returnFields(QueryComponent.java:967)
> at 
> org.apache.solr.handler.component.QueryComponent.handleRegularResponses(QueryComponent.java:629)
> at 
> org.apache.solr.handler.component.QueryComponent.handleResponses(QueryComponent.java:604)
> at 
> org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:311)
> at 
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:146)
> at org.apache.solr.core.SolrCore.execute(SolrCore.java:1818)
> at 
> org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:479)
> at 
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:279)
> at 
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
> at 
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
> at 
> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
> at 
> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
> at 
> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
> at 
> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
> at 
> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
> at 
> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:293)
> at 
> org.apache.coyote.http11.Http11NioProcessor.process(Http11NioProcessor.java:889)
> at 
> org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.process(Http11NioProtocol.java:744)
> at 
> org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:2274)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> {code}



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4186) QueryComponent NPE

[Shalin Shekhar Mangar (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-4186?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14052795#comment-14052795
 ] 

Shalin Shekhar Mangar commented on SOLR-4186:
-

Yes, this is fixed by SOLR-6223. We have a test which demonstrates a failure in 
the GET_FIELDS phase. I'll mark this issue as a duplicate.

> QueryComponent NPE
> --
>
> Key: SOLR-4186
> URL: https://issues.apache.org/jira/browse/SOLR-4186
> Project: Solr
>  Issue Type: Bug
>  Components: search, SolrCloud
>Affects Versions: 4.0, 5.0
> Environment: recent trunk exports
>Reporter: Markus Jelsma
>
> Similar to SOLR-4049, the QueryComponent throws the following NPE in case a 
> distributed search fails for some reason. This bug has been around for at 
> least one or two weeks.
> {code}
> 2012-12-14 10:00:41,611 ERROR [solr.servlet.SolrDispatchFilter] - 
> [http-8080-exec-5] - : null:java.lang.NullPointerException
> at 
> org.apache.solr.handler.component.QueryComponent.returnFields(QueryComponent.java:967)
> at 
> org.apache.solr.handler.component.QueryComponent.handleRegularResponses(QueryComponent.java:629)
> at 
> org.apache.solr.handler.component.QueryComponent.handleResponses(QueryComponent.java:604)
> at 
> org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:311)
> at 
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:146)
> at org.apache.solr.core.SolrCore.execute(SolrCore.java:1818)
> at 
> org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:479)
> at 
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:279)
> at 
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
> at 
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
> at 
> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
> at 
> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
> at 
> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
> at 
> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
> at 
> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
> at 
> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:293)
> at 
> org.apache.coyote.http11.Http11NioProcessor.process(Http11NioProcessor.java:889)
> at 
> org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.process(Http11NioProtocol.java:744)
> at 
> org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:2274)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> {code}



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5692) Deprecate spatial DisjointSpatialFilter

[David Smiley (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-5692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14052792#comment-14052792
 ] 

David Smiley commented on LUCENE-5692:
--

Ok; I'm going to mark it deprecated, and add an Ignore annotation to RPT's 
Disjoint test.  And I'll have the trunk RecursivePrefixTreeStrategy remove 
support for it, leaving it in 4x.

Note to self: I do think I could make a simple variation of 
IntersectsPrefixTreeFilter (IPTF isn't much code any way) to implement 
disjoint.  Perhaps I'll do that sometime but not now.  But even if I did... it 
wouldn't be as fast as what should be recommended (what I said in the 
description):
bq. and if a user wants to do this for real, they should index a boolean field 
marking wether there's spatial data and then combine that with a NOT and 
Intersects, in a straight-forward way.

> Deprecate spatial DisjointSpatialFilter
> ---
>
> Key: LUCENE-5692
> URL: https://issues.apache.org/jira/browse/LUCENE-5692
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: modules/spatial
>Affects Versions: 5.0
>Reporter: David Smiley
>
> The spatial predicate "IsDisjointTo" is almost the same as the inverse of 
> "Intersects", except that it shouldn't match documents without spatial data.  
> In another sense it's as if the query shape were inverted.
> DisjointSpatialFilter is a utility filter that works (or worked, rather) by 
> using the FieldCache to see which documents have spatial data 
> (getDocsWithField()). Calculating that was probably very slow but it was at 
> least cacheable. Since LUCENE-5666 (v5/trunk only), Rob replaced this to use 
> DocValues.  However for some SpatialStrategies (PrefixTree based) it wouldn't 
> make any sense to use DocValues *just* so that at search time you could call 
> getDocsWithField() when there's no other need for the un-inversion (e.g. no 
> need to lookup terms by document).
> Perhaps an immediate fix is simply to revert the change made to 
> DisjointSpatialFilter so that it uses the FieldCache again, if that works 
> (though it's not public?).  But stepping back a bit, this 
> DisjointSpatialFilter is really something unfortunate that doesn't work as 
> well as it could because it's not at the level of Solr or ES -- that is, 
> there's no access to a filter-cache.  So I propose I simply remove it, and if 
> a user wants to do this for real, they should index a boolean field marking 
> wether there's spatial data and then combine that with a NOT and Intersects, 
> in a straight-forward way.  
> Alternatively, some sort of inverting query shape could be developed, 
> although it wouldn't work with the SpatialPrefixTree technique because there 
> is no edge distinction -- the edge matches normally and notwithstanding 
> changes to RPT algorithms it would also match the edge of an inverted shape.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-4396) BooleanScorer should sometimes be used for MUST clauses

[Da Huang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-4396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Da Huang updated LUCENE-4396:
-

Attachment: LUCENE-4396.patch

This is a patch based on the git mirror commit 
7f66461aea7bc2cb6f31a993cba77734e5e0f9d9.

In this patch, I take the bucketTable as an array but not a hash table.
It seems that its perf. is better than former patches' on most cases.

As you know, after putting required docs into bucketTable, I have to scan both 
the table and optional docs. Here, I have tried skipping to scan the 
bucketTable to improve the perf. The results is as follows.


{code}
No skip
TaskQPS baseline  StdDevQPS my_version  StdDev  
  Pct diff
   HighAndTonsLowNot6.56  (3.1%)2.59  (1.0%)  
-60.5% ( -62% -  -58%)
HighAndTonsLowOr6.43  (3.3%)2.58  (0.8%)  
-59.9% ( -61% -  -57%)
HighAndSomeLowOr8.49  (8.5%)4.05  (1.8%)  
-52.3% ( -57% -  -45%)
   HighAndSomeLowNot6.17  (8.6%)3.16  (2.1%)  
-48.8% ( -54% -  -41%)
 LowAndSomeLowOr  250.58  (2.0%)  194.86  (1.6%)  
-22.2% ( -25% -  -18%)
LowAndSomeLowNot  178.66  (1.6%)  147.67  (2.2%)  
-17.3% ( -20% -  -13%)
LowAndSomeHighOr   40.71  (2.8%)   41.50  (1.8%)
2.0% (  -2% -6%)
PKLookup   97.59  (3.0%)   99.52  (4.6%)
2.0% (  -5% -9%)
   LowAndSomeHighNot   20.76  (3.0%)   21.54  (2.3%)
3.7% (  -1% -9%)
  HighAndSomeHighNot2.22  (1.7%)2.67  (4.4%)   
20.3% (  13% -   26%)
   LowAndTonsHighNot3.81  (2.3%)4.60  (2.1%)   
20.8% (  15% -   25%)
LowAndTonsHighOr2.87  (2.3%)3.48  (2.6%)   
21.2% (  15% -   26%)
   HighAndSomeHighOr1.74  (2.1%)2.16  (3.5%)   
24.0% (  18% -   30%)
 LowAndTonsLowOr   18.66  (1.3%)   23.68  (1.9%)   
26.9% (  23% -   30%)
LowAndTonsLowNot   16.01  (1.4%)   22.16  (2.8%)   
38.4% (  33% -   43%)
   HighAndTonsHighOr0.04  (0.9%)0.11  (9.8%)  
158.2% ( 146% -  170%)
  HighAndTonsHighNot0.06  (1.1%)0.15 (13.5%)  
166.2% ( 149% -  182%)
  
---
Binary search skip
TaskQPS baseline  StdDevQPS my_version  StdDev  
  Pct diff
   HighAndTonsLowNot6.22  (3.8%)2.45  (0.9%)  
-60.6% ( -62% -  -58%)
HighAndSomeLowOr8.29 (11.2%)4.40  (3.0%)  
-46.9% ( -54% -  -36%)
   HighAndSomeLowNot   12.34  (7.1%)6.65  (2.6%)  
-46.1% ( -52% -  -39%)
 LowAndSomeLowOr  232.38  (2.9%)  165.05  (1.8%)  
-29.0% ( -32% -  -24%)
HighAndTonsLowOr5.17  (6.2%)3.75  (3.0%)  
-27.4% ( -34% -  -19%)
LowAndSomeLowNot  227.71  (2.6%)  171.13  (3.2%)  
-24.8% ( -29% -  -19%)
   HighAndSomeHighOr1.35  (3.9%)1.14  (3.5%)  
-16.1% ( -22% -   -9%)
LowAndSomeHighOr   50.17  (3.6%)   48.84  (3.7%)   
-2.7% (  -9% -4%)
   LowAndSomeHighNot   52.71  (3.0%)   51.55  (3.8%)   
-2.2% (  -8% -4%)
PKLookup   90.17  (3.5%)   91.38  (3.3%)
1.3% (  -5% -8%)
  HighAndSomeHighNot1.69  (2.9%)2.00  (6.3%)   
18.5% (   8% -   28%)
 LowAndTonsLowOr   15.61  (1.9%)   18.59  (2.8%)   
19.0% (  14% -   24%)
LowAndTonsHighOr1.82  (2.7%)2.20  (4.6%)   
20.7% (  13% -   28%)
LowAndTonsLowNot   15.51  (1.7%)   20.14  (3.8%)   
29.8% (  23% -   35%)
   LowAndTonsHighNot1.01  (2.9%)1.34  (6.5%)   
31.7% (  21% -   42%)
   HighAndTonsHighOr0.07  (0.9%)0.12  (6.9%)   
77.7% (  69% -   86%)
  HighAndTonsHighNot0.07  (1.4%)0.19 (11.9%)  
162.4% ( 146% -  178%)
  
---
8 steps skip
TaskQPS baseline  StdDevQPS my_version  StdDev  
  Pct diff
   HighAndTonsLowNot5.45  (3.3%)1.69  (1.3%)  
-69.0% ( -71% -  -66%)
HighAndSomeLowOr5.46 (11.0%)2.76  (4.4%)  
-49.5% ( -58% -  -38%)
   HighAndSomeLowNot   17.94  (5.7%)   10.40  (3.8%)  
-42.1% ( -48% -  -34%)
 LowAndSomeLowOr  306.62  (1.7%)  231.45  (1.5%)  
-24.5% ( -27% -  -21%)
LowAndSomeLowNot  286.30  (1.7%)  218.13  (2.0%)  
-23.8% ( -27% -  -20%)
HighAndTonsLowOr6.34 

[JENKINS] Lucene-Solr-Tests-4.x-Java7 - Build # 2022 - Still Failing

[Apache Jenkins Server]

Build: https://builds.apache.org/job/Lucene-Solr-Tests-4.x-Java7/2022/

All tests passed

Build Log:
[...truncated 29260 lines...]
check-licenses:
 [echo] License check under: 
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr
 [licenses] MISSING sha1 checksum file for: 
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/example/lib/ext/log4j-1.2.16.jar
 [licenses] EXPECTED sha1 checksum file : 
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/licenses/log4j-1.2.16.jar.sha1

[...truncated 1 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/build.xml:467:
 The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/build.xml:70:
 The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build.xml:254:
 The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/lucene/tools/custom-tasks.xml:62:
 License check failed. Check the logs.

Total time: 136 minutes 24 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Sending artifact delta relative to Lucene-Solr-Tests-4.x-Java7 #2008
Archived 1 artifacts
Archive block size is 32768
Received 0 blocks and 464 bytes
Compression is 0.0%
Took 11 ms
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4186) QueryComponent NPE

[Patanachai Tangchaisin (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-4186?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14052732#comment-14052732
 ] 

Patanachai Tangchaisin commented on SOLR-4186:
--

May be this is fixed by SOLR-6223?
Since the cause of the problem seems to be the same shards.tolerant=true.

> QueryComponent NPE
> --
>
> Key: SOLR-4186
> URL: https://issues.apache.org/jira/browse/SOLR-4186
> Project: Solr
>  Issue Type: Bug
>  Components: search, SolrCloud
>Affects Versions: 4.0, 5.0
> Environment: recent trunk exports
>Reporter: Markus Jelsma
> Fix For: 4.9, 5.0
>
>
> Similar to SOLR-4049, the QueryComponent throws the following NPE in case a 
> distributed search fails for some reason. This bug has been around for at 
> least one or two weeks.
> {code}
> 2012-12-14 10:00:41,611 ERROR [solr.servlet.SolrDispatchFilter] - 
> [http-8080-exec-5] - : null:java.lang.NullPointerException
> at 
> org.apache.solr.handler.component.QueryComponent.returnFields(QueryComponent.java:967)
> at 
> org.apache.solr.handler.component.QueryComponent.handleRegularResponses(QueryComponent.java:629)
> at 
> org.apache.solr.handler.component.QueryComponent.handleResponses(QueryComponent.java:604)
> at 
> org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:311)
> at 
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:146)
> at org.apache.solr.core.SolrCore.execute(SolrCore.java:1818)
> at 
> org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:479)
> at 
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:279)
> at 
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
> at 
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
> at 
> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
> at 
> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
> at 
> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
> at 
> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
> at 
> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
> at 
> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:293)
> at 
> org.apache.coyote.http11.Http11NioProcessor.process(Http11NioProcessor.java:889)
> at 
> org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.process(Http11NioProtocol.java:744)
> at 
> org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:2274)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> {code}



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-1782) stats.facet assumes FieldCache.StringIndex - fails horribly on multivalued fields

[Patanachai Tangchaisin (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-1782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Patanachai Tangchaisin updated SOLR-1782:
-

Attachment: SOLR-1782.solr-4.2.1.patch

Here is patch for Solr 4.2.1. I copied a test cases from latest previous patch 
(SOLR-1782.2013-04-10.patch).
Instead of modify UnInvertedField.java (it is too complex to modify), I used 
DocValues to get a value from a field to aggregate stats.  

We've used it in our production for some time now since we stuck on version 
4.2.1 due to some of our requirement. However, our fields that need stats.facet 
is DocValues field. So, I hope the previous patch test case cover a case where 
multi-value field is not a DocValue.

> stats.facet assumes FieldCache.StringIndex - fails horribly on multivalued 
> fields
> -
>
> Key: SOLR-1782
> URL: https://issues.apache.org/jira/browse/SOLR-1782
> Project: Solr
>  Issue Type: Bug
>  Components: search
>Affects Versions: 1.4
> Environment: reproduced on Win2k3 using 1.5.0-dev solr ($Id: 
> CHANGES.txt 906924 2010-02-05 12:43:11Z noble $)
>Reporter: Gerald DeConto
>Assignee: Hoss Man
> Attachments: SOLR-1782.2.patch, SOLR-1782.2013-01-07.patch, 
> SOLR-1782.2013-04-10.patch, SOLR-1782.patch, SOLR-1782.patch, 
> SOLR-1782.patch, SOLR-1782.solr-4.2.1.patch, SOLR-1782.test.patch, index.rar
>
>
> the StatsComponent assumes any field specified in the stats.facet param can 
> be faceted using FieldCache.DEFAULT.getStringIndex.  This can cause problems 
> with a variety of field types, but in the case of multivalued fields it can 
> either cause erroneous false stats when the number of distinct values is 
> small, or it can cause ArrayIndexOutOfBoundsException when the number of 
> distinct values is greater then the number of documents.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Solr and Maven

[Ahmet Arslan]

Hi Tom,

you might find https://issues.apache.org/jira/browse/LUCENE-5755 relevant.

Ahmet

On Saturday, July 5, 2014 1:26 AM, Tom Chen  wrote:



Hi,

The default tool to build Solr is ant ( plus ivy), while Maven support is 
provided. 

Regarding building with Maven,  some questions:

1) Is there any difference between the build created by ant and that created by 
Maven?
2) Any plan for Solr to use Maven as the default building tool?

Regards,
Tom

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Solr and Maven

[Tom Chen]

Hi,

The default tool to build Solr is ant ( plus ivy), while Maven support is
provided.

Regarding building with Maven,  some questions:

1) Is there any difference between the build created by ant and that
created by Maven?
2) Any plan for Solr to use Maven as the default building tool?

Regards,
Tom

[jira] [Updated] (SOLR-5480) Make MoreLikeThisHandler distributable

[Steve Molloy (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-5480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Steve Molloy updated SOLR-5480:
---

Attachment: SOLR-5480.patch

> Make MoreLikeThisHandler distributable
> --
>
> Key: SOLR-5480
> URL: https://issues.apache.org/jira/browse/SOLR-5480
> Project: Solr
>  Issue Type: Improvement
>Reporter: Steve Molloy
>Assignee: Noble Paul
> Attachments: SOLR-5480.patch, SOLR-5480.patch, SOLR-5480.patch, 
> SOLR-5480.patch, SOLR-5480.patch, SOLR-5480.patch, SOLR-5480.patch, 
> SOLR-5480.patch
>
>
> The MoreLikeThis component, when used in the standard search handler supports 
> distributed searches. But the MoreLikeThisHandler itself doesn't, which 
> prevents from say, passing in text to perform the query. I'll start looking 
> into adapting the SearchHandler logic to the MoreLikeThisHandler. If anyone 
> has some work done already and want to share, or want to contribute, any help 
> will be welcomed. 



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-5480) Make MoreLikeThisHandler distributable

[Steve Molloy (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-5480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14052636#comment-14052636
 ] 

Steve Molloy commented on SOLR-5480:


Adapted patch for 4.9. Also ensured all relevant unit tests are passing, added 
support for non-string ids and proper result sorting. For queries, first match 
will always be chosen according to score, not specified sort order which is 
used for final result sorting.

> Make MoreLikeThisHandler distributable
> --
>
> Key: SOLR-5480
> URL: https://issues.apache.org/jira/browse/SOLR-5480
> Project: Solr
>  Issue Type: Improvement
>Reporter: Steve Molloy
>Assignee: Noble Paul
> Attachments: SOLR-5480.patch, SOLR-5480.patch, SOLR-5480.patch, 
> SOLR-5480.patch, SOLR-5480.patch, SOLR-5480.patch, SOLR-5480.patch, 
> SOLR-5480.patch
>
>
> The MoreLikeThis component, when used in the standard search handler supports 
> distributed searches. But the MoreLikeThisHandler itself doesn't, which 
> prevents from say, passing in text to perform the query. I'll start looking 
> into adapting the SearchHandler logic to the MoreLikeThisHandler. If anyone 
> has some work done already and want to share, or want to contribute, any help 
> will be welcomed. 



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5806) Extend expression grammar to allow advanced "variables"

[Uwe Schindler (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-5806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14052601#comment-14052601
 ] 

Uwe Schindler commented on LUCENE-5806:
---

OK, this is what I expected. I am fine with that if the "binding" is normalized:

{noformat}
doc['foo'].value
doc ["foo"].value
doc[ 'foo' ].value
{noformat}

is all the same so, the binding should be somehow normalized? Like:

{noformat}
doc['foo'].value
{noformat}

Otherwise its hard to hardcode bindings like you mentioned in your last post.

> Extend expression grammar to allow advanced "variables"
> ---
>
> Key: LUCENE-5806
> URL: https://issues.apache.org/jira/browse/LUCENE-5806
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Ryan Ernst
> Attachments: LUCENE-5806.patch
>
>
> We currently allow dots in "variable" names in expressions, so that we can 
> fake out object access.  We should extend this to allow array access as well 
> (both integer and string keys).  This would allow faking out full object 
> nesting through bindings.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (SOLR-6223) SearchComponents may throw NPE when using shards.tolerant and there is a failure in the “GET_FIELDS/GET_HIGHLIGHTS/GET_DEBUG” phase

[Shalin Shekhar Mangar (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-6223?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shalin Shekhar Mangar resolved SOLR-6223.
-

   Resolution: Fixed
Fix Version/s: 4.10
   5.0

Thanks Tomás!

> SearchComponents may throw NPE when using shards.tolerant and there is a 
> failure in the “GET_FIELDS/GET_HIGHLIGHTS/GET_DEBUG” phase
> ---
>
> Key: SOLR-6223
> URL: https://issues.apache.org/jira/browse/SOLR-6223
> Project: Solr
>  Issue Type: Bug
>Affects Versions: 4.9, 5.0
>Reporter: Tomás Fernández Löbbe
>Assignee: Shalin Shekhar Mangar
> Fix For: 5.0, 4.10
>
> Attachments: SOLR-6223.patch
>
>
> I found that, when using shards.tolerant, if there is some kind of exception 
> in the second phase of the search, some component’s throw NPE. 
> I found it with the QueryComponent first, but then saw that other components 
> would suffer in the same way: DebugComponent, HighlightComponent and 
> MLTComponent. I only checked the components of the default chain.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-6223) SearchComponents may throw NPE when using shards.tolerant and there is a failure in the “GET_FIELDS/GET_HIGHLIGHTS/GET_DEBUG” phase

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-6223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14052597#comment-14052597
 ] 

ASF subversion and git services commented on SOLR-6223:
---

Commit 1607899 from sha...@apache.org in branch 'dev/branches/branch_4x'
[ https://svn.apache.org/r1607899 ]

SOLR-6223: SearchComponents may throw NPE when using shards.tolerant and there 
is a failure in the 'GET_FIELDS/GET_HIGHLIGHTS/GET_DEBUG' phase

> SearchComponents may throw NPE when using shards.tolerant and there is a 
> failure in the “GET_FIELDS/GET_HIGHLIGHTS/GET_DEBUG” phase
> ---
>
> Key: SOLR-6223
> URL: https://issues.apache.org/jira/browse/SOLR-6223
> Project: Solr
>  Issue Type: Bug
>Affects Versions: 4.9, 5.0
>Reporter: Tomás Fernández Löbbe
>Assignee: Shalin Shekhar Mangar
> Attachments: SOLR-6223.patch
>
>
> I found that, when using shards.tolerant, if there is some kind of exception 
> in the second phase of the search, some component’s throw NPE. 
> I found it with the QueryComponent first, but then saw that other components 
> would suffer in the same way: DebugComponent, HighlightComponent and 
> MLTComponent. I only checked the components of the default chain.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-6223) SearchComponents may throw NPE when using shards.tolerant and there is a failure in the “GET_FIELDS/GET_HIGHLIGHTS/GET_DEBUG” phase

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-6223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14052595#comment-14052595
 ] 

ASF subversion and git services commented on SOLR-6223:
---

Commit 1607897 from sha...@apache.org in branch 'dev/trunk'
[ https://svn.apache.org/r1607897 ]

SOLR-6223: SearchComponents may throw NPE when using shards.tolerant and there 
is a failure in the 'GET_FIELDS/GET_HIGHLIGHTS/GET_DEBUG' phase

> SearchComponents may throw NPE when using shards.tolerant and there is a 
> failure in the “GET_FIELDS/GET_HIGHLIGHTS/GET_DEBUG” phase
> ---
>
> Key: SOLR-6223
> URL: https://issues.apache.org/jira/browse/SOLR-6223
> Project: Solr
>  Issue Type: Bug
>Affects Versions: 4.9, 5.0
>Reporter: Tomás Fernández Löbbe
>Assignee: Shalin Shekhar Mangar
> Attachments: SOLR-6223.patch
>
>
> I found that, when using shards.tolerant, if there is some kind of exception 
> in the second phase of the search, some component’s throw NPE. 
> I found it with the QueryComponent first, but then saw that other components 
> would suffer in the same way: DebugComponent, HighlightComponent and 
> MLTComponent. I only checked the components of the default chain.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5806) Extend expression grammar to allow advanced "variables"

[Ryan Ernst (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-5806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14052586#comment-14052586
 ] 

Ryan Ernst commented on LUCENE-5806:


Note that parsing is only necessary in the bindings if you want to use these 
dynamically.  If you know that fields are accessed through "doc" and they are 
"foo" and "bar", then simply adding the following to bindings should work (with 
ES syntax as an example):

{noformat}
"doc['foo'].value"
"doc['bar'].value"
{noformat}

> Extend expression grammar to allow advanced "variables"
> ---
>
> Key: LUCENE-5806
> URL: https://issues.apache.org/jira/browse/LUCENE-5806
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Ryan Ernst
> Attachments: LUCENE-5806.patch
>
>
> We currently allow dots in "variable" names in expressions, so that we can 
> fake out object access.  We should extend this to allow array access as well 
> (both integer and string keys).  This would allow faking out full object 
> nesting through bindings.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5806) Extend expression grammar to allow advanced "variables"

[Ryan Ernst (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-5806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14052584#comment-14052584
 ] 

Ryan Ernst commented on LUCENE-5806:


I only added tests from the compiler side, but I can add one with bindings 
actually using extended variables.  It would continue to show up to bindings as 
a string name, which the binding would have to parse.

I thought about using the parser to break up the "variable" into components, 
and passing a variable context to the bindings instead of (or alternatively to) 
the simple string variable names.  It might be a little more efficient (since 
the parser has to find these pieces anyways when matching), but I opted for 
simplicity for now (let the bindings parse as they like).  In either case, the 
cost is still occurred once (at the beginning of an evaluate() call) and not 
per document.

> Extend expression grammar to allow advanced "variables"
> ---
>
> Key: LUCENE-5806
> URL: https://issues.apache.org/jira/browse/LUCENE-5806
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Ryan Ernst
> Attachments: LUCENE-5806.patch
>
>
> We currently allow dots in "variable" names in expressions, so that we can 
> fake out object access.  We should extend this to allow array access as well 
> (both integer and string keys).  This would allow faking out full object 
> nesting through bindings.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5806) Extend expression grammar to allow advanced "variables"

[Uwe Schindler (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-5806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14052574#comment-14052574
 ] 

Uwe Schindler commented on LUCENE-5806:
---

I did not find a test where such a "fake variable" is used in a binding. How 
would {{foo.bar.strindex['abc']}} look like in a binding?

> Extend expression grammar to allow advanced "variables"
> ---
>
> Key: LUCENE-5806
> URL: https://issues.apache.org/jira/browse/LUCENE-5806
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Ryan Ernst
> Attachments: LUCENE-5806.patch
>
>
> We currently allow dots in "variable" names in expressions, so that we can 
> fake out object access.  We should extend this to allow array access as well 
> (both integer and string keys).  This would allow faking out full object 
> nesting through bindings.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5806) Extend expression grammar to allow advanced "variables"

[Uwe Schindler (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-5806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14052571#comment-14052571
 ] 

Uwe Schindler commented on LUCENE-5806:
---

Cool! I was waiting so long!

> Extend expression grammar to allow advanced "variables"
> ---
>
> Key: LUCENE-5806
> URL: https://issues.apache.org/jira/browse/LUCENE-5806
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Ryan Ernst
> Attachments: LUCENE-5806.patch
>
>
> We currently allow dots in "variable" names in expressions, so that we can 
> fake out object access.  We should extend this to allow array access as well 
> (both integer and string keys).  This would allow faking out full object 
> nesting through bindings.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5806) Extend expression grammar to allow advanced "variables"

[Ryan Ernst (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-5806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14052570#comment-14052570
 ] 

Ryan Ernst commented on LUCENE-5806:


Yup!

> Extend expression grammar to allow advanced "variables"
> ---
>
> Key: LUCENE-5806
> URL: https://issues.apache.org/jira/browse/LUCENE-5806
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Ryan Ernst
> Attachments: LUCENE-5806.patch
>
>
> We currently allow dots in "variable" names in expressions, so that we can 
> fake out object access.  We should extend this to allow array access as well 
> (both integer and string keys).  This would allow faking out full object 
> nesting through bindings.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5806) Extend expression grammar to allow advanced "variables"

[Uwe Schindler (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-5806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14052566#comment-14052566
 ] 

Uwe Schindler commented on LUCENE-5806:
---

Elasticsearch scripts?

> Extend expression grammar to allow advanced "variables"
> ---
>
> Key: LUCENE-5806
> URL: https://issues.apache.org/jira/browse/LUCENE-5806
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Ryan Ernst
> Attachments: LUCENE-5806.patch
>
>
> We currently allow dots in "variable" names in expressions, so that we can 
> fake out object access.  We should extend this to allow array access as well 
> (both integer and string keys).  This would allow faking out full object 
> nesting through bindings.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-5806) Extend expression grammar to allow advanced "variables"

[Ryan Ernst (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-5806?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ryan Ernst updated LUCENE-5806:
---

Attachment: LUCENE-5806.patch

Patch with tests.

> Extend expression grammar to allow advanced "variables"
> ---
>
> Key: LUCENE-5806
> URL: https://issues.apache.org/jira/browse/LUCENE-5806
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Ryan Ernst
> Attachments: LUCENE-5806.patch
>
>
> We currently allow dots in "variable" names in expressions, so that we can 
> fake out object access.  We should extend this to allow array access as well 
> (both integer and string keys).  This would allow faking out full object 
> nesting through bindings.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (LUCENE-5806) Extend expression grammar to allow advanced "variables"

[Ryan Ernst (JIRA)]

Ryan Ernst created LUCENE-5806:
--

 Summary: Extend expression grammar to allow advanced "variables"
 Key: LUCENE-5806
 URL: https://issues.apache.org/jira/browse/LUCENE-5806
 Project: Lucene - Core
  Issue Type: Bug
Reporter: Ryan Ernst


We currently allow dots in "variable" names in expressions, so that we can fake 
out object access.  We should extend this to allow array access as well (both 
integer and string keys).  This would allow faking out full object nesting 
through bindings.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-5803) Add another AnalyzerWrapper class that does not have its own cache, so delegate-only wrappers don't create thread local resources several times

[Uwe Schindler (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-5803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Uwe Schindler updated LUCENE-5803:
--

Attachment: LUCENE-5803.patch

New patch:
- renamed a field and improved docs
- replaced a horrible Solr offender: PreAnalyzedField was creating the analyzer 
over and over instead of creating it one time after field type init().

I think this is ready.

> Add another AnalyzerWrapper class that does not have its own cache, so 
> delegate-only wrappers don't create thread local resources several times
> ---
>
> Key: LUCENE-5803
> URL: https://issues.apache.org/jira/browse/LUCENE-5803
> Project: Lucene - Core
>  Issue Type: Improvement
>  Components: modules/analysis
>Affects Versions: 4.9
>Reporter: Uwe Schindler
>Assignee: Uwe Schindler
> Fix For: 5.0, 4.10
>
> Attachments: LUCENE-5803.patch, LUCENE-5803.patch, LUCENE-5803.patch, 
> LUCENE-5803.patch, LUCENE-5803.patch, LUCENE-5803.patch
>
>
> This is a followup issue for the following Elasticsearch issue: 
> https://github.com/elasticsearch/elasticsearch/pull/6714
> Basically the problem is the following:
> - Elasticsearch has a pool of Analyzers that are used for analysis in several 
> indexes
> - Each index uses a different PerFieldAnalyzerWrapper
> PerFieldAnalyzerWrapper uses PER_FIELD_REUSE_STRATEGY. Because of this it 
> caches the tokenstreams for every field. If there are many fields, this are a 
> lot. In addition, the underlying analyzers may also cache tokenstreams and 
> other PerFieldAnalyzerWrappers do the same, although the delegate Analyzer 
> can always return the same components.
> We should add similar code to Elasticsearch's directly to Lucene: If the 
> delegating Analyzer just delegates per Field or just wraps CharFilters around 
> the Reader, there is no need to cache the TokenStreamComponents a second time 
> in the delegating Analyzers. This is only needed, if the delegating Analyzers 
> adds additional TokenFilters (like ShingleAnalyzerWrapper).
> We should name this new class DelegatingAnalyzerWrapper extends 
> AnalyzerWrapper. The wrapComponents method must be final, because we are not 
> allowed to add additional TokenFilters, but unlike ES, we don't need to 
> disallow wrapping with CharFilters.
> Internally this class uses a private ReuseStrategy that just delegates to the 
> underlying analyzer. It does not matter here if the strategy of the delegate 
> is global or per field, this is private to the delegate.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5803) Add another AnalyzerWrapper class that does not have its own cache, so delegate-only wrappers don't create thread local resources several times

[Uwe Schindler (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-5803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14052513#comment-14052513
 ] 

Uwe Schindler commented on LUCENE-5803:
---

Samuel García (‏https://twitter.com/samuelgmartinez) said on Twitter:
bq. @kimchy @thetaph1 we have a solr server cluster with 150 cores and about 20 
indexed fields. We lost 1.5gb due to these zz_buffer tlocal.

This patch will improve this situation, but not as good as in Elasticsearch. 
The difference in Solr is: Solr has a complete separation of cores (even with 
different classloader). Each core has its own schema with own field types. 
Every field type has its own analyzer. Those are combined in a 
PerFieldAnalyzer-like wrapper. If Solr would allow to define "field types" 
globally (across cores), this could be shared. But with crrent Solr, each core 
gets its own zz_buffer tlocal. The improvement in Solr due to this patch is:
In the past we had a separate threadlocal *per field name*, because the 
AnalyzerWraper had a per-field-reuse strategy. With this patch we now have a 
global reuse strategy *per* FieldType. So the imporvement is: If you define a 
field type one time and reuse it for 20 fields, you have only one cached 
TokenStream, not 20. This is because we now delegate to the underlying Analyzer 
(the one from the field type), which has GLOBAL_REUSE_STRATEGY.

> Add another AnalyzerWrapper class that does not have its own cache, so 
> delegate-only wrappers don't create thread local resources several times
> ---
>
> Key: LUCENE-5803
> URL: https://issues.apache.org/jira/browse/LUCENE-5803
> Project: Lucene - Core
>  Issue Type: Improvement
>  Components: modules/analysis
>Affects Versions: 4.9
>Reporter: Uwe Schindler
>Assignee: Uwe Schindler
> Fix For: 5.0, 4.10
>
> Attachments: LUCENE-5803.patch, LUCENE-5803.patch, LUCENE-5803.patch, 
> LUCENE-5803.patch, LUCENE-5803.patch
>
>
> This is a followup issue for the following Elasticsearch issue: 
> https://github.com/elasticsearch/elasticsearch/pull/6714
> Basically the problem is the following:
> - Elasticsearch has a pool of Analyzers that are used for analysis in several 
> indexes
> - Each index uses a different PerFieldAnalyzerWrapper
> PerFieldAnalyzerWrapper uses PER_FIELD_REUSE_STRATEGY. Because of this it 
> caches the tokenstreams for every field. If there are many fields, this are a 
> lot. In addition, the underlying analyzers may also cache tokenstreams and 
> other PerFieldAnalyzerWrappers do the same, although the delegate Analyzer 
> can always return the same components.
> We should add similar code to Elasticsearch's directly to Lucene: If the 
> delegating Analyzer just delegates per Field or just wraps CharFilters around 
> the Reader, there is no need to cache the TokenStreamComponents a second time 
> in the delegating Analyzers. This is only needed, if the delegating Analyzers 
> adds additional TokenFilters (like ShingleAnalyzerWrapper).
> We should name this new class DelegatingAnalyzerWrapper extends 
> AnalyzerWrapper. The wrapComponents method must be final, because we are not 
> allowed to add additional TokenFilters, but unlike ES, we don't need to 
> disallow wrapping with CharFilters.
> Internally this class uses a private ReuseStrategy that just delegates to the 
> underlying analyzer. It does not matter here if the strategy of the delegate 
> is global or per field, this is private to the delegate.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-5805) QueryNodeImpl.removeFromParent does a lot of work without any effect

[Christoph Kaser (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-5805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Christoph Kaser updated LUCENE-5805:


Description: 
The method _removeFromParent_ of _QueryNodeImpl_, calls _getChildren_ on the 
parent and removes any occurrence of "this" from the result.

However, since a few releases, _getChildren_ returns a *copy* of the children 
list, so the code has no effect (except creating a copy of the children list 
which will then be thrown away). 
Even worse, since _setChildren_ calls _removeFromParent_ on any previous child, 
_setChildren_ now has a complexity of O(n^2) and creates a lot of throw-away 
copies of the children list (for nodes with a lot of children)

{code}
public void removeFromParent() {
if (this.parent != null) {
  List parentChildren = this.parent.getChildren();
  Iterator it = parentChildren.iterator();
  
  while (it.hasNext()) {
if (it.next() == this) {
  it.remove();
}
  }
  
  this.parent = null;
}
  }
{code}

  was:
The method _removeFromParent_ of _QueryNodeImpl_, calls _getChildren_ on the 
parent and removes any occurrence of "this" from the result.

However, since a few releases, _getChildren_ returns a *copy* of the children 
list, so the code has no effect (except creating a copy of the children list 
which will then be thrown away). 
Even worse, since setChildren calls removeFromParent on any previous child, 
setChildren has a complexity of O(n^2) and creates a lot of throw-away copies 
of the children list (for nodes with a lot of children)

{code}
public void removeFromParent() {
if (this.parent != null) {
  List parentChildren = this.parent.getChildren();
  Iterator it = parentChildren.iterator();
  
  while (it.hasNext()) {
if (it.next() == this) {
  it.remove();
}
  }
  
  this.parent = null;
}
  }
{code}


> QueryNodeImpl.removeFromParent does a lot of work without any effect
> 
>
> Key: LUCENE-5805
> URL: https://issues.apache.org/jira/browse/LUCENE-5805
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: modules/queryparser
>Affects Versions: 4.7.2, 4.9
>Reporter: Christoph Kaser
>
> The method _removeFromParent_ of _QueryNodeImpl_, calls _getChildren_ on the 
> parent and removes any occurrence of "this" from the result.
> However, since a few releases, _getChildren_ returns a *copy* of the children 
> list, so the code has no effect (except creating a copy of the children list 
> which will then be thrown away). 
> Even worse, since _setChildren_ calls _removeFromParent_ on any previous 
> child, _setChildren_ now has a complexity of O(n^2) and creates a lot of 
> throw-away copies of the children list (for nodes with a lot of children)
> {code}
> public void removeFromParent() {
> if (this.parent != null) {
>   List parentChildren = this.parent.getChildren();
>   Iterator it = parentChildren.iterator();
>   
>   while (it.hasNext()) {
> if (it.next() == this) {
>   it.remove();
> }
>   }
>   
>   this.parent = null;
> }
>   }
> {code}



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-5805) QueryNodeImpl.removeFromParent does a lot of work without any effect

[Christoph Kaser (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-5805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Christoph Kaser updated LUCENE-5805:


Description: 
The method _removeFromParent_ of _QueryNodeImpl_, calls _getChildren_ on the 
parent and removes any occurrence of "this" from the result.

However, since a few releases, _getChildren_ returns a *copy* of the children 
list, so the code has no effect (except creating a copy of the children list 
which will then be thrown away). 
Even worse, since setChildren calls removeFromParent on any previous child, 
setChildren has a complexity of O(n^2) and creates a lot of throw-away copies 
of the children list (for nodes with a lot of children)

{code}
public void removeFromParent() {
if (this.parent != null) {
  List parentChildren = this.parent.getChildren();
  Iterator it = parentChildren.iterator();
  
  while (it.hasNext()) {
if (it.next() == this) {
  it.remove();
}
  }
  
  this.parent = null;
}
  }
{code}

  was:
The method _removeFromParent_ of _QueryNodeImpl_, calls _getChildren_ on the 
parent and removes any occurrence of "this" from the result.

However, since a few releases, _getChildren_ returns a *copy* of the children 
list, so the code has no effect (except creating a copy of the children list 
which will then be thrown away). 
Even worse, since setChildren calls removeFromParent on any previous child, 
setChildren has a complexity of O(n^2) and creates a lot of throw-away copies 
of the children list (for nodes with a lot of children)

{code]
public void removeFromParent() {
if (this.parent != null) {
  List parentChildren = this.parent.getChildren();
  Iterator it = parentChildren.iterator();
  
  while (it.hasNext()) {
if (it.next() == this) {
  it.remove();
}
  }
  
  this.parent = null;
}
  }
{code}


> QueryNodeImpl.removeFromParent does a lot of work without any effect
> 
>
> Key: LUCENE-5805
> URL: https://issues.apache.org/jira/browse/LUCENE-5805
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: modules/queryparser
>Affects Versions: 4.7.2, 4.9
>Reporter: Christoph Kaser
>
> The method _removeFromParent_ of _QueryNodeImpl_, calls _getChildren_ on the 
> parent and removes any occurrence of "this" from the result.
> However, since a few releases, _getChildren_ returns a *copy* of the children 
> list, so the code has no effect (except creating a copy of the children list 
> which will then be thrown away). 
> Even worse, since setChildren calls removeFromParent on any previous child, 
> setChildren has a complexity of O(n^2) and creates a lot of throw-away copies 
> of the children list (for nodes with a lot of children)
> {code}
> public void removeFromParent() {
> if (this.parent != null) {
>   List parentChildren = this.parent.getChildren();
>   Iterator it = parentChildren.iterator();
>   
>   while (it.hasNext()) {
> if (it.next() == this) {
>   it.remove();
> }
>   }
>   
>   this.parent = null;
> }
>   }
> {code}



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (LUCENE-5805) QueryNodeImpl.removeFromParent does a lot of work without any effect

[Christoph Kaser (JIRA)]

Christoph Kaser created LUCENE-5805:
---

 Summary: QueryNodeImpl.removeFromParent does a lot of work without 
any effect
 Key: LUCENE-5805
 URL: https://issues.apache.org/jira/browse/LUCENE-5805
 Project: Lucene - Core
  Issue Type: Bug
  Components: modules/queryparser
Affects Versions: 4.9, 4.7.2
Reporter: Christoph Kaser


The method _removeFromParent_ of _QueryNodeImpl_, calls _getChildren_ on the 
parent and removes any occurrence of "this" from the result.

However, since a few releases, _getChildren_ returns a *copy* of the children 
list, so the code has no effect (except creating a copy of the children list 
which will then be thrown away). 
Even worse, since setChildren calls removeFromParent on any previous child, 
setChildren has a complexity of O(n^2) and creates a lot of throw-away copies 
of the children list (for nodes with a lot of children)

{code]
public void removeFromParent() {
if (this.parent != null) {
  List parentChildren = this.parent.getChildren();
  Iterator it = parentChildren.iterator();
  
  while (it.hasNext()) {
if (it.next() == this) {
  it.remove();
}
  }
  
  this.parent = null;
}
  }
{code}



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS-MAVEN] Lucene-Solr-Maven-trunk #1161: POMs out of sync

[Apache Jenkins Server]

Build: https://builds.apache.org/job/Lucene-Solr-Maven-trunk/1161/

1 tests failed.
FAILED:  org.apache.solr.cloud.MultiThreadedOCPTest.testDistribSearch

Error Message:
Task 3002 did not complete, final state: running

Stack Trace:
java.lang.AssertionError: Task 3002 did not complete, final state: running
at 
__randomizedtesting.SeedInfo.seed([512EF185739F8455:D0C87F9D04C0E469]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at 
org.apache.solr.cloud.MultiThreadedOCPTest.testDeduplicationOfSubmittedTasks(MultiThreadedOCPTest.java:161)
at 
org.apache.solr.cloud.MultiThreadedOCPTest.doTest(MultiThreadedOCPTest.java:70)




Build Log:
[...truncated 54837 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Maven-trunk/build.xml:490: 
The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Maven-trunk/build.xml:182: 
The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Maven-trunk/extra-targets.xml:77:
 Java returned: 1

Total time: 163 minutes 5 seconds
Build step 'Invoke Ant' marked build as failure
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-5803) Add another AnalyzerWrapper class that does not have its own cache, so delegate-only wrappers don't create thread local resources several times

[Uwe Schindler (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-5803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Uwe Schindler updated LUCENE-5803:
--

Attachment: LUCENE-5803.patch

More improved test, does more checks by using a special non-delegating wrapper.

> Add another AnalyzerWrapper class that does not have its own cache, so 
> delegate-only wrappers don't create thread local resources several times
> ---
>
> Key: LUCENE-5803
> URL: https://issues.apache.org/jira/browse/LUCENE-5803
> Project: Lucene - Core
>  Issue Type: Improvement
>  Components: modules/analysis
>Affects Versions: 4.9
>Reporter: Uwe Schindler
>Assignee: Uwe Schindler
> Fix For: 5.0, 4.10
>
> Attachments: LUCENE-5803.patch, LUCENE-5803.patch, LUCENE-5803.patch, 
> LUCENE-5803.patch, LUCENE-5803.patch
>
>
> This is a followup issue for the following Elasticsearch issue: 
> https://github.com/elasticsearch/elasticsearch/pull/6714
> Basically the problem is the following:
> - Elasticsearch has a pool of Analyzers that are used for analysis in several 
> indexes
> - Each index uses a different PerFieldAnalyzerWrapper
> PerFieldAnalyzerWrapper uses PER_FIELD_REUSE_STRATEGY. Because of this it 
> caches the tokenstreams for every field. If there are many fields, this are a 
> lot. In addition, the underlying analyzers may also cache tokenstreams and 
> other PerFieldAnalyzerWrappers do the same, although the delegate Analyzer 
> can always return the same components.
> We should add similar code to Elasticsearch's directly to Lucene: If the 
> delegating Analyzer just delegates per Field or just wraps CharFilters around 
> the Reader, there is no need to cache the TokenStreamComponents a second time 
> in the delegating Analyzers. This is only needed, if the delegating Analyzers 
> adds additional TokenFilters (like ShingleAnalyzerWrapper).
> We should name this new class DelegatingAnalyzerWrapper extends 
> AnalyzerWrapper. The wrapComponents method must be final, because we are not 
> allowed to add additional TokenFilters, but unlike ES, we don't need to 
> disallow wrapping with CharFilters.
> Internally this class uses a private ReuseStrategy that just delegates to the 
> underlying analyzer. It does not matter here if the strategy of the delegate 
> is global or per field, this is private to the delegate.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-5803) Add another AnalyzerWrapper class that does not have its own cache, so delegate-only wrappers don't create thread local resources several times

[Uwe Schindler (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-5803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Uwe Schindler updated LUCENE-5803:
--

Attachment: LUCENE-5803.patch

I added a test to the analysis module that checks that wrapping (also mixed 
with default non-delegating AnalyzerWrapper) works as expected.

> Add another AnalyzerWrapper class that does not have its own cache, so 
> delegate-only wrappers don't create thread local resources several times
> ---
>
> Key: LUCENE-5803
> URL: https://issues.apache.org/jira/browse/LUCENE-5803
> Project: Lucene - Core
>  Issue Type: Improvement
>  Components: modules/analysis
>Affects Versions: 4.9
>Reporter: Uwe Schindler
>Assignee: Uwe Schindler
> Fix For: 5.0, 4.10
>
> Attachments: LUCENE-5803.patch, LUCENE-5803.patch, LUCENE-5803.patch, 
> LUCENE-5803.patch
>
>
> This is a followup issue for the following Elasticsearch issue: 
> https://github.com/elasticsearch/elasticsearch/pull/6714
> Basically the problem is the following:
> - Elasticsearch has a pool of Analyzers that are used for analysis in several 
> indexes
> - Each index uses a different PerFieldAnalyzerWrapper
> PerFieldAnalyzerWrapper uses PER_FIELD_REUSE_STRATEGY. Because of this it 
> caches the tokenstreams for every field. If there are many fields, this are a 
> lot. In addition, the underlying analyzers may also cache tokenstreams and 
> other PerFieldAnalyzerWrappers do the same, although the delegate Analyzer 
> can always return the same components.
> We should add similar code to Elasticsearch's directly to Lucene: If the 
> delegating Analyzer just delegates per Field or just wraps CharFilters around 
> the Reader, there is no need to cache the TokenStreamComponents a second time 
> in the delegating Analyzers. This is only needed, if the delegating Analyzers 
> adds additional TokenFilters (like ShingleAnalyzerWrapper).
> We should name this new class DelegatingAnalyzerWrapper extends 
> AnalyzerWrapper. The wrapComponents method must be final, because we are not 
> allowed to add additional TokenFilters, but unlike ES, we don't need to 
> disallow wrapping with CharFilters.
> Internally this class uses a private ReuseStrategy that just delegates to the 
> underlying analyzer. It does not matter here if the strategy of the delegate 
> is global or per field, this is private to the delegate.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Early Access builds for JDK 9 b21, JDK 8u20 b21 are available on java.net

[Rory O'Donnell Oracle, Dublin Ireland]

Hi Uwe,Dawid,

Early Access builds for JDK 9 b21   and 
JDK 8u20 b21   are available on 
java.net.


As we enter the later phases of development for JDK 8u20 , please log 
any show

stoppers as soon as possible.

Rgds, Rory

--
Rgds,Rory O'Donnell
Quality Engineering Manager
Oracle EMEA , Dublin, Ireland

[JENKINS] Lucene-Solr-NightlyTests-4.x - Build # 570 - Still Failing

[Apache Jenkins Server]

Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-4.x/570/

2 tests failed.
FAILED:  
junit.framework.TestSuite.org.apache.solr.handler.TestReplicationHandlerBackup

Error Message:
1 thread leaked from SUITE scope at 
org.apache.solr.handler.TestReplicationHandlerBackup: 1) Thread[id=18982, 
name=Thread-4347, state=RUNNABLE, group=TGRP-TestReplicationHandlerBackup]  
   at java.net.PlainSocketImpl.socketConnect(Native Method) at 
java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
 at 
java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
 at 
java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)  
   at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at 
java.net.Socket.connect(Socket.java:579) at 
java.net.Socket.connect(Socket.java:528) at 
sun.net.NetworkClient.doConnect(NetworkClient.java:180) at 
sun.net.www.http.HttpClient.openServer(HttpClient.java:432) at 
sun.net.www.http.HttpClient.openServer(HttpClient.java:527) at 
sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:652) at 
sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1323)
 at java.net.URL.openStream(URL.java:1037) at 
org.apache.solr.handler.TestReplicationHandlerBackup$BackupThread.run(TestReplicationHandlerBackup.java:314)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE 
scope at org.apache.solr.handler.TestReplicationHandlerBackup: 
   1) Thread[id=18982, name=Thread-4347, state=RUNNABLE, 
group=TGRP-TestReplicationHandlerBackup]
at java.net.PlainSocketImpl.socketConnect(Native Method)
at 
java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
at 
java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
at 
java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:579)
at java.net.Socket.connect(Socket.java:528)
at sun.net.NetworkClient.doConnect(NetworkClient.java:180)
at sun.net.www.http.HttpClient.openServer(HttpClient.java:432)
at sun.net.www.http.HttpClient.openServer(HttpClient.java:527)
at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:652)
at 
sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1323)
at java.net.URL.openStream(URL.java:1037)
at 
org.apache.solr.handler.TestReplicationHandlerBackup$BackupThread.run(TestReplicationHandlerBackup.java:314)
at __randomizedtesting.SeedInfo.seed([8BDBE10D84F0CD7E]:0)


FAILED:  
junit.framework.TestSuite.org.apache.solr.handler.TestReplicationHandlerBackup

Error Message:
There are still zombie threads that couldn't be terminated:1) 
Thread[id=18982, name=Thread-4347, state=RUNNABLE, 
group=TGRP-TestReplicationHandlerBackup] at 
java.net.PlainSocketImpl.socketConnect(Native Method) at 
java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
 at 
java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
 at 
java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)  
   at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at 
java.net.Socket.connect(Socket.java:579) at 
java.net.Socket.connect(Socket.java:528) at 
sun.net.NetworkClient.doConnect(NetworkClient.java:180) at 
sun.net.www.http.HttpClient.openServer(HttpClient.java:432) at 
sun.net.www.http.HttpClient.openServer(HttpClient.java:527) at 
sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:652) at 
sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1323)
 at java.net.URL.openStream(URL.java:1037) at 
org.apache.solr.handler.TestReplicationHandlerBackup$BackupThread.run(TestReplicationHandlerBackup.java:314)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie 
threads that couldn't be terminated:
   1) Thread[id=18982, name=Thread-4347, state=RUNNABLE, 
group=TGRP-TestReplicationHandlerBackup]
at java.net.PlainSocketImpl.socketConnect(Native Method)
at 
java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
at 
java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
at 
java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:579)
at java.net.Socket.connect(Socket.java:528)
at sun.net.NetworkClient.doConnect(NetworkClient.java:180)
at sun.

[jira] [Updated] (LUCENE-5803) Add another AnalyzerWrapper class that does not have its own cache, so delegate-only wrappers don't create thread local resources several times

[Uwe Schindler (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-5803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Uwe Schindler updated LUCENE-5803:
--

Attachment: LUCENE-5803.patch

This is my new patch, that does not break the common pattern: {{new 
ShingleAnalyzerWrapper(new PerFieldAnalyzerWrapper(...))}}. The previous patch 
would have thrown IllegalStateException.

In this patch, also the DelegatingAnalyzerWrapper gets a reuse strategy in its 
constructor, but this one is only used if you wrap a second time as fallback 
(this is why its called "defaultStrategy".

> Add another AnalyzerWrapper class that does not have its own cache, so 
> delegate-only wrappers don't create thread local resources several times
> ---
>
> Key: LUCENE-5803
> URL: https://issues.apache.org/jira/browse/LUCENE-5803
> Project: Lucene - Core
>  Issue Type: Improvement
>  Components: modules/analysis
>Affects Versions: 4.9
>Reporter: Uwe Schindler
>Assignee: Uwe Schindler
> Fix For: 5.0, 4.10
>
> Attachments: LUCENE-5803.patch, LUCENE-5803.patch, LUCENE-5803.patch
>
>
> This is a followup issue for the following Elasticsearch issue: 
> https://github.com/elasticsearch/elasticsearch/pull/6714
> Basically the problem is the following:
> - Elasticsearch has a pool of Analyzers that are used for analysis in several 
> indexes
> - Each index uses a different PerFieldAnalyzerWrapper
> PerFieldAnalyzerWrapper uses PER_FIELD_REUSE_STRATEGY. Because of this it 
> caches the tokenstreams for every field. If there are many fields, this are a 
> lot. In addition, the underlying analyzers may also cache tokenstreams and 
> other PerFieldAnalyzerWrappers do the same, although the delegate Analyzer 
> can always return the same components.
> We should add similar code to Elasticsearch's directly to Lucene: If the 
> delegating Analyzer just delegates per Field or just wraps CharFilters around 
> the Reader, there is no need to cache the TokenStreamComponents a second time 
> in the delegating Analyzers. This is only needed, if the delegating Analyzers 
> adds additional TokenFilters (like ShingleAnalyzerWrapper).
> We should name this new class DelegatingAnalyzerWrapper extends 
> AnalyzerWrapper. The wrapComponents method must be final, because we are not 
> allowed to add additional TokenFilters, but unlike ES, we don't need to 
> disallow wrapping with CharFilters.
> Internally this class uses a private ReuseStrategy that just delegates to the 
> underlying analyzer. It does not matter here if the strategy of the delegate 
> is global or per field, this is private to the delegate.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-5803) Add another AnalyzerWrapper class that does not have its own cache, so delegate-only wrappers don't create thread local resources several times

[Uwe Schindler (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-5803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Uwe Schindler updated LUCENE-5803:
--

Attachment: LUCENE-5803.patch

bq. And for that reason, you shouldnt be able to wrap it with a charfilter. Use 
the existing subclass for "tweaking" the analyzer. Let this one be for pure 
delegation...

I thought about that already and I agree. Although it is not really needed to 
explicitely forbid wrapping readers, we should still make the method final. 
Maybe in the future we will somehow also reuse the readers, so having a clear 
API contract here is a good idea.

Attached is a new patch, but I have a better idea about the 
IllegalStateException.

> Add another AnalyzerWrapper class that does not have its own cache, so 
> delegate-only wrappers don't create thread local resources several times
> ---
>
> Key: LUCENE-5803
> URL: https://issues.apache.org/jira/browse/LUCENE-5803
> Project: Lucene - Core
>  Issue Type: Improvement
>  Components: modules/analysis
>Affects Versions: 4.9
>Reporter: Uwe Schindler
>Assignee: Uwe Schindler
> Fix For: 5.0, 4.10
>
> Attachments: LUCENE-5803.patch, LUCENE-5803.patch
>
>
> This is a followup issue for the following Elasticsearch issue: 
> https://github.com/elasticsearch/elasticsearch/pull/6714
> Basically the problem is the following:
> - Elasticsearch has a pool of Analyzers that are used for analysis in several 
> indexes
> - Each index uses a different PerFieldAnalyzerWrapper
> PerFieldAnalyzerWrapper uses PER_FIELD_REUSE_STRATEGY. Because of this it 
> caches the tokenstreams for every field. If there are many fields, this are a 
> lot. In addition, the underlying analyzers may also cache tokenstreams and 
> other PerFieldAnalyzerWrappers do the same, although the delegate Analyzer 
> can always return the same components.
> We should add similar code to Elasticsearch's directly to Lucene: If the 
> delegating Analyzer just delegates per Field or just wraps CharFilters around 
> the Reader, there is no need to cache the TokenStreamComponents a second time 
> in the delegating Analyzers. This is only needed, if the delegating Analyzers 
> adds additional TokenFilters (like ShingleAnalyzerWrapper).
> We should name this new class DelegatingAnalyzerWrapper extends 
> AnalyzerWrapper. The wrapComponents method must be final, because we are not 
> allowed to add additional TokenFilters, but unlike ES, we don't need to 
> disallow wrapping with CharFilters.
> Internally this class uses a private ReuseStrategy that just delegates to the 
> underlying analyzer. It does not matter here if the strategy of the delegate 
> is global or per field, this is private to the delegate.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-trunk-Linux (32bit/jdk1.7.0_60) - Build # 10719 - Failure!

[Policeman Jenkins Server]

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/10719/
Java: 32bit/jdk1.7.0_60 -server -XX:+UseConcMarkSweepGC

1 tests failed.
FAILED:  
org.apache.lucene.spatial.prefix.RandomSpatialOpFuzzyPrefixTreeTest.testDisjoint
 {#3 seed=[ACE49C1CC6C8B2F3:10329BA021EF01FE]}

Error Message:


Stack Trace:
java.lang.AssertionError
at 
__randomizedtesting.SeedInfo.seed([ACE49C1CC6C8B2F3:10329BA021EF01FE]:0)
at 
org.apache.lucene.index.AssertingAtomicReader.getDocsWithField(AssertingAtomicReader.java:727)
at 
org.apache.lucene.index.DocValues.getDocsWithField(DocValues.java:316)
at 
org.apache.lucene.spatial.DisjointSpatialFilter.getDocIdSet(DisjointSpatialFilter.java:96)
at 
org.apache.lucene.search.ConstantScoreQuery$ConstantWeight.scorer(ConstantScoreQuery.java:157)
at org.apache.lucene.search.Weight.bulkScorer(Weight.java:131)
at 
org.apache.lucene.search.ConstantScoreQuery$ConstantWeight.bulkScorer(ConstantScoreQuery.java:141)
at 
org.apache.lucene.search.AssertingWeight.bulkScorer(AssertingWeight.java:74)
at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:611)
at 
org.apache.lucene.search.AssertingIndexSearcher.search(AssertingIndexSearcher.java:94)
at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:483)
at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:440)
at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:273)
at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:261)
at 
org.apache.lucene.spatial.SpatialTestCase.executeQuery(SpatialTestCase.java:140)
at 
org.apache.lucene.spatial.prefix.RandomSpatialOpFuzzyPrefixTreeTest.doTest(RandomSpatialOpFuzzyPrefixTreeTest.java:363)
at 
org.apache.lucene.spatial.prefix.RandomSpatialOpFuzzyPrefixTreeTest.testDisjoint(RandomSpatialOpFuzzyPrefixTreeTest.java:155)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.e

Having problem with indexing/ searching with _ or -

[Smitha Kuldeep (smtt)]

Hello team,
We are using lucen-core-2.9.1.jar for indexing and searching for text fields

We are having problem when our string has digits as prefix or suffix of _/-

Ex:
I have texts
ICMP6-a-b
ICMP6-echo

If I provide ICMP6 it doesn't search any of the above.
But if I have
ICMP-a-b
ICMP-echo
Providing ICMP will search above 2.

Similar issue is seen when I have _
Ex:
A_b_c_1
A_b_c_2

If I search A_B above 2 will be searched
But if I provide
A_b_c none will be searched.
Can you suggest the solution.

We are using StandardAnalyzer.
Tried to provide _ and - as stopString.
Doesn't solve the problem.
Thanks,
Smitha