[jira] [Updated] (SOLR-10229) See what it would take to shift many of our one-off schemas used for testing to managed schema and construct them as part of the tests

[Amrit Sarkar (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-10229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Amrit Sarkar updated SOLR-10229:

Attachment: SOLR-10229.patch

> See what it would take to shift many of our one-off schemas used for testing 
> to managed schema and construct them as part of the tests
> --
>
> Key: SOLR-10229
> URL: https://issues.apache.org/jira/browse/SOLR-10229
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Erick Erickson
>Assignee: Erick Erickson
>Priority: Minor
> Attachments: SOLR-10229.patch, SOLR-10229.patch
>
>
> The test schema files are intimidating. There are about a zillion of them, 
> and making a change in any of them risks breaking some _other_ test. That 
> leaves people three choices:
> 1> add what they need to some existing schema. Which makes schemas bigger and 
> bigger and bigger.
> 2> create a new schema file, adding to the proliferation thereof.
> 3> Look through all the existing tests to see if they have something that 
> works.
> The recent work on LUCENE-7705 is a case in point. We're adding a maxLen 
> parameter to some tokenizers. Putting those parameters into any of the 
> existing schemas, especially to test < 255 char tokens is virtually 
> guaranteed to break other tests, so the only safe thing to do is make another 
> schema file. Adding to the multiplication of files.
> As part of SOLR-5260 I tried creating the schema on the fly rather than 
> creating a new static schema file and it's not hard. WDYT about making this 
> into some better thought-out utility? 
> At present, this is pretty fuzzy, I wanted to get some reactions before 
> putting much effort into it. I expect that the utility methods would 
> eventually get a bunch of canned types. It's reasonably straightforward for 
> primitive types, if lengthy. But when you get into solr.TextField-based types 
> it gets less straight-forward.
> We could manage to just move the "intimidation" from the plethora of schema 
> files to a zillion fieldTypes in the utility to choose from...
> Also, forcing every test to define the fields up-front is arguably less 
> convenient than just having _some_ canned schemas we can use. And erroneous 
> schemas to test failure modes are probably not very good fits for any such 
> framework.
> [~steve_rowe] and [~hossman_luc...@fucit.org] in particular might have 
> something to say.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-10429) UpdateRequest#getRoutes() does not copy the response parser

[Noble Paul (JIRA)]

Noble Paul created SOLR-10429:
-

 Summary: UpdateRequest#getRoutes() does not copy the response 
parser
 Key: SOLR-10429
 URL: https://issues.apache.org/jira/browse/SOLR-10429
 Project: Solr
  Issue Type: Bug
  Security Level: Public (Default Security Level. Issues are Public)
Reporter: Noble Paul
Assignee: Noble Paul


So, update requests always use the default responseparser



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-Tests-6.x - Build # 825 - Unstable

[Apache Jenkins Server]

Build: https://builds.apache.org/job/Lucene-Solr-Tests-6.x/825/

1 tests failed.
FAILED:  org.apache.solr.core.snapshots.TestSolrCloudSnapshots.testSnapshots

Error Message:
Error from server at http://127.0.0.1:52439/solr: Could not fully remove 
collection: SolrCloudSnapshots

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error 
from server at http://127.0.0.1:52439/solr: Could not fully remove collection: 
SolrCloudSnapshots
at 
__randomizedtesting.SeedInfo.seed([D7D8ADCAF5492F4F:E194295580102D00]:0)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:610)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:279)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:268)
at 
org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:435)
at 
org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:387)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1364)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:1115)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:1054)
at 
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:160)
at 
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:177)
at 
org.apache.solr.core.snapshots.TestSolrCloudSnapshots.testSnapshots(TestSolrCloudSnapshots.java:266)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 

[JENKINS-EA] Lucene-Solr-master-Linux (64bit/jdk-9-ea+162) - Build # 19323 - Unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/19323/
Java: 64bit/jdk-9-ea+162 -XX:-UseCompressedOops -XX:+UseParallelGC

1 tests failed.
FAILED:  org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates.test_dv

Error Message:
java.lang.RuntimeException: Error from server at 
http://127.0.0.1:40490/solr/test_col: Async exception during distributed 
update: Error from server at 
http://127.0.0.1:32857/solr/test_col_shard1_replica2: Server Errorrequest: 
http://127.0.0.1:32857/solr/test_col_shard1_replica2/update?update.distrib=TOLEADER=http%3A%2F%2F127.0.0.1%3A40490%2Fsolr%2Ftest_col_shard2_replica1%2F=javabin=2
 Remote error message: Failed synchronous update on shard StdNode: 
http://127.0.0.1:35355/solr/test_col_shard1_replica1/ update: 
org.apache.solr.client.solrj.request.UpdateRequest@2f3723ed

Stack Trace:
java.util.concurrent.ExecutionException: java.lang.RuntimeException: Error from 
server at http://127.0.0.1:40490/solr/test_col: Async exception during 
distributed update: Error from server at 
http://127.0.0.1:32857/solr/test_col_shard1_replica2: Server Error



request: 
http://127.0.0.1:32857/solr/test_col_shard1_replica2/update?update.distrib=TOLEADER=http%3A%2F%2F127.0.0.1%3A40490%2Fsolr%2Ftest_col_shard2_replica1%2F=javabin=2
Remote error message: Failed synchronous update on shard StdNode: 
http://127.0.0.1:35355/solr/test_col_shard1_replica1/ update: 
org.apache.solr.client.solrj.request.UpdateRequest@2f3723ed
at 
__randomizedtesting.SeedInfo.seed([A864CA10B615A18E:9E70A8563C489B9F]:0)
at java.base/java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.base/java.util.concurrent.FutureTask.get(FutureTask.java:191)
at 
org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates.checkField(TestStressCloudBlindAtomicUpdates.java:281)
at 
org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates.test_dv(TestStressCloudBlindAtomicUpdates.java:193)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:547)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 

[JENKINS] Lucene-Solr-master-MacOSX (64bit/jdk1.8.0) - Build # 3946 - Unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-MacOSX/3946/
Java: 64bit/jdk1.8.0 -XX:-UseCompressedOops -XX:+UseG1GC

1 tests failed.
FAILED:  
org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testCollectionsAPI

Error Message:
Error from server at http://127.0.0.1:56230/solr/awhollynewcollection_0: 
Expected mime type application/octet-stream but got text/html.   
 
Error 510HTTP ERROR: 510 Problem 
accessing /solr/awhollynewcollection_0/select. Reason: 
{metadata={error-class=org.apache.solr.common.SolrException,root-error-class=org.apache.solr.common.SolrException},msg={awhollynewcollection_0:5},code=510}
 http://eclipse.org/jetty;>Powered by Jetty:// 
9.3.14.v20161028   

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error 
from server at http://127.0.0.1:56230/solr/awhollynewcollection_0: Expected 
mime type application/octet-stream but got text/html. 


Error 510 


HTTP ERROR: 510
Problem accessing /solr/awhollynewcollection_0/select. Reason:

{metadata={error-class=org.apache.solr.common.SolrException,root-error-class=org.apache.solr.common.SolrException},msg={awhollynewcollection_0:5},code=510}
http://eclipse.org/jetty;>Powered by Jetty:// 
9.3.14.v20161028



at 
__randomizedtesting.SeedInfo.seed([EF81F54EA752694E:A7F481FAA16146DB]:0)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:595)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:279)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:268)
at 
org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:439)
at 
org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:391)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1364)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:1115)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:1218)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:1218)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:1218)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:1218)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:1218)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:1054)
at 
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:160)
at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:942)
at 
org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testCollectionsAPI(CollectionsAPIDistributedZkTest.java:523)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at 

[jira] [Assigned] (SOLR-10418) metrics should return JVM system properties

[Noble Paul (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-10418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Noble Paul reassigned SOLR-10418:
-

Assignee: Andrzej Bialecki 

> metrics should return JVM system properties
> ---
>
> Key: SOLR-10418
> URL: https://issues.apache.org/jira/browse/SOLR-10418
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Noble Paul
>Assignee: Andrzej Bialecki 
>
> We need to stop using the custom solution used in rules and start using 
> metrics for everything



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-6.x-Linux (64bit/jdk1.8.0_121) - Build # 3209 - Still Unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-6.x-Linux/3209/
Java: 64bit/jdk1.8.0_121 -XX:-UseCompressedOops -XX:+UseConcMarkSweepGC

1 tests failed.
FAILED:  org.apache.solr.cloud.TestSolrCloudWithKerberosAlt.testBasics

Error Message:
Address already in use

Stack Trace:
java.net.BindException: Address already in use
at 
__randomizedtesting.SeedInfo.seed([E418AD886A6384D2:D9C003A4528DDAA2]:0)
at sun.nio.ch.Net.bind0(Native Method)
at sun.nio.ch.Net.bind(Net.java:433)
at sun.nio.ch.Net.bind(Net.java:425)
at 
sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:223)
at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)
at 
org.apache.mina.transport.socket.nio.NioSocketAcceptor.open(NioSocketAcceptor.java:252)
at 
org.apache.mina.transport.socket.nio.NioSocketAcceptor.open(NioSocketAcceptor.java:49)
at 
org.apache.mina.core.polling.AbstractPollingIoAcceptor.registerHandles(AbstractPollingIoAcceptor.java:525)
at 
org.apache.mina.core.polling.AbstractPollingIoAcceptor.access$200(AbstractPollingIoAcceptor.java:67)
at 
org.apache.mina.core.polling.AbstractPollingIoAcceptor$Acceptor.run(AbstractPollingIoAcceptor.java:409)
at 
org.apache.mina.util.NamePreservingRunnable.run(NamePreservingRunnable.java:65)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 12716 lines...]
   [junit4] Suite: org.apache.solr.cloud.TestSolrCloudWithKerberosAlt
   [junit4]   2> 1598393 WARN  
(TEST-TestSolrCloudWithKerberosAlt.testBasics-seed#[E418AD886A6384D2]) [] 
o.a.d.s.c.DefaultDirectoryService You didn't change the admin password of 
directory service instance 'DefaultKrbServer'.  Please update the admin 
password as soon as possible to prevent a possible security breach.
   [junit4]   2> NOTE: reproduce with: ant test  
-Dtestcase=TestSolrCloudWithKerberosAlt -Dtests.method=testBasics 
-Dtests.seed=E418AD886A6384D2 -Dtests.multiplier=3 -Dtests.slow=true 
-Dtests.locale=en-ZA -Dtests.timezone=Etc/GMT+7 -Dtests.asserts=true 
-Dtests.file.encoding=UTF-8
   [junit4] ERROR   19.0s J1 | TestSolrCloudWithKerberosAlt.testBasics <<<
   [junit4]> Throwable #1: java.net.BindException: Address already in use
   [junit4]>at 
__randomizedtesting.SeedInfo.seed([E418AD886A6384D2:D9C003A4528DDAA2]:0)
   [junit4]>at sun.nio.ch.Net.bind0(Native Method)
   [junit4]>at sun.nio.ch.Net.bind(Net.java:433)
   [junit4]>at sun.nio.ch.Net.bind(Net.java:425)
   [junit4]>at 
sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:223)
   [junit4]>at 
sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)
   [junit4]>at 
org.apache.mina.transport.socket.nio.NioSocketAcceptor.open(NioSocketAcceptor.java:252)
   [junit4]>at 
org.apache.mina.transport.socket.nio.NioSocketAcceptor.open(NioSocketAcceptor.java:49)
   [junit4]>at 
org.apache.mina.core.polling.AbstractPollingIoAcceptor.registerHandles(AbstractPollingIoAcceptor.java:525)
   [junit4]>at 
org.apache.mina.core.polling.AbstractPollingIoAcceptor.access$200(AbstractPollingIoAcceptor.java:67)
   [junit4]>at 
org.apache.mina.core.polling.AbstractPollingIoAcceptor$Acceptor.run(AbstractPollingIoAcceptor.java:409)
   [junit4]>at 
org.apache.mina.util.NamePreservingRunnable.run(NamePreservingRunnable.java:65)
   [junit4]>at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
   [junit4]>at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]>at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> NOTE: leaving temporary files on disk at: 
/home/jenkins/workspace/Lucene-Solr-6.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.TestSolrCloudWithKerberosAlt_E418AD886A6384D2-001
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene62): {}, 
docValues:{}, maxPointsInLeafNode=151, maxMBSortInHeap=6.337958194002499, 
sim=RandomSimilarity(queryNorm=true,coord=yes): {}, locale=en-ZA, 
timezone=Etc/GMT+7
   [junit4]   2> NOTE: Linux 4.4.0-66-generic amd64/Oracle Corporation 
1.8.0_121 (64-bit)/cpus=12,threads=1,free=141112896,total=518979584
   [junit4]   2> NOTE: All tests run in this JVM: [SearchHandlerTest, 
SpatialFilterTest, TestDelegationWithHadoopAuth, CoreAdminRequestStatusTest, 
SolrCoreMetricManagerTest, ZkSolrClientTest, TestSchemaSimilarityResource, 
ShardSplitTest, LargeFieldTest, TestStressVersions, TestSurroundQueryParser, 
DistributedQueryComponentOptimizationTest, SolrJmxReporterTest, SampleTest, 
TestSolrQueryParserDefaultOperatorResource, ChaosMonkeySafeLeaderTest, 

[jira] [Updated] (SOLR-10425) PointFields ignore indexed="false"

[JIRA]

 [ 
https://issues.apache.org/jira/browse/SOLR-10425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tomás Fernández Löbbe updated SOLR-10425:
-
Attachment: SOLR-10425.patch

Added {{testWhiteboxCreateFields}} to the patch. Testing all the point dynamic 
fields

> PointFields ignore indexed="false"
> --
>
> Key: SOLR-10425
> URL: https://issues.apache.org/jira/browse/SOLR-10425
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Hoss Man
> Attachments: SOLR-10425_hoss.patch, SOLR-10425.patch, 
> SOLR-10425.patch, SOLR-10425.patch, SOLR-10425.patch
>
>
> (NOTE: description below focuses on {{IntPointField}}, but problem seems to 
> affect all {{PointField}} subclasses)
> There seems to be a disconnect between {{PointField.createFields}} -> 
> {{IntPointField.createField}} -> {{PointField.isFieldUsed}} that results in 
> an {{org.apache.lucene.document.IntPoint}} being created for each field 
> value, even if field is {{indexed="false"}}
> Steps to reproduce...
> {noformat}
> bin/solr -e techproducts
> ...
> curl -X POST -H 'Content-type:application/json' --data-binary '{
>   "add-field":{
>  "name":"hoss_points_check",
>  "type":"pint",
>  "stored":true,
>  "docValues":false,
>  "indexed":false}
> }' http://localhost:8983/solr/techproducts/schema
> ...
> curl -X POST -H 'Content-type:application/json' --data-binary 
> '[{"id":"HOSS","hoss_points_check":42}]' 
> 'http://localhost:8983/solr/techproducts/update/json?commit=true'
> ...
> curl 'http://localhost:8983/solr/techproducts/query?q=id:HOSS'
> {
>   "responseHeader":{
> "status":0,
> "QTime":3,
> "params":{
>   "q":"id:HOSS"}},
>   "response":{"numFound":1,"start":0,"docs":[
>   {
> "id":"HOSS",
> "hoss_points_check":42,
> "_version_":1563795876337418240}]
>   }}
> curl 'http://localhost:8983/solr/techproducts/query?q=hoss_points_check:42'
> {
>   "responseHeader":{
> "status":0,
> "QTime":2,
> "params":{
>   "q":"hoss_points_check:42"}},
>   "response":{"numFound":1,"start":0,"docs":[
>   {
> "id":"HOSS",
> "hoss_points_check":42,
> "_version_":1563795876337418240}]
>   }}
> {noformat}
> Note that I can search on the doc using the  "hoss_points_check" field even 
> though it is {{docValues="false" indexed="false"}}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-NightlyTests-6.x - Build # 322 - Still Unstable

[Apache Jenkins Server]

Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-6.x/322/

2 tests failed.
FAILED:  org.apache.solr.cloud.ShardSplitTest.testSplitAfterFailedSplit

Error Message:
expected:<1> but was:<2>

Stack Trace:
java.lang.AssertionError: expected:<1> but was:<2>
at 
__randomizedtesting.SeedInfo.seed([5D68F7BAA2172BFD:A42564159E626677]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.junit.Assert.assertEquals(Assert.java:456)
at 
org.apache.solr.cloud.ShardSplitTest.testSplitAfterFailedSplit(ShardSplitTest.java:280)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 

[jira] [Commented] (SOLR-10426) Add shuffle Streaming Expression

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15957968#comment-15957968
 ] 

ASF subversion and git services commented on SOLR-10426:


Commit e606d901e461837ff858595470a0d64fb4bbde1e in lucene-solr's branch 
refs/heads/branch_6x from [~joel.bernstein]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=e606d90 ]

SOLR-10426: Add shuffle Streaming Expression


> Add shuffle Streaming Expression
> 
>
> Key: SOLR-10426
> URL: https://issues.apache.org/jira/browse/SOLR-10426
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Joel Bernstein
>Assignee: Joel Bernstein
> Fix For: 6.6
>
> Attachments: SOLR-10426.patch
>
>
> Currently the concept of MapReduce shuffling is lumped into the *search* 
> Streaming Expression. This causes quite a bit of confusion as users sometimes 
> perform a search using the /select handler by default, but actually wanted 
> shuffling behavior which requires the /export handler.
> We can solve this problem by creating a separate function called *shuffle* 
> that always uses the /export handler.
> This will also allow us to clean up some behaviors in the search expression 
> that are somewhat unexpected in future tickets.
> {code}
> parallel(workers, 
>  unique(shuffle(collection1, 
> q=*:*, 
> fl="id,a_s,a_i,a_f", 
> sort="a_f asc, a_i asc", 
> partitionKeys="a_f"), 
> over="a_f"), 
>  workers="2",
>  sort="a_f asc")
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10425) PointFields ignore indexed="false"

[Hoss Man (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10425?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15957956#comment-15957956
 ] 

Hoss Man commented on SOLR-10425:
-

tomas: i haven't had a change to look at your latest patch, and i have to go 
offline for the rest of the day, but FWIW here's the whitebox test i was 
working on...

{code}
  public void testWhiteboxCreateFields() throws Exception {
// TODO: add a "coverage" check that all points based dynamic fields in the 
schema are accounted for?

doWhiteboxCreateFields("whitebox_p_i_dv", IntPoint.class, 42, "42");
doWhiteboxCreateFields("whitebox_p_i_mv", IntPoint.class, 42, "42");

doWhiteboxCreateFields("whitebox_p_i_ni", IntPoint.class, 42, "42");

// nocommit: TEST ALL THE FIELDS


  }
  
  /** nocommit: jdocs, @see callAndCheckCreateFields, verifies same result for 
all values (string or type specific)
   */
  private void doWhiteboxCreateFields(final String fieldName, final Class 
pointType, final Object... values) throws Exception {
List firstResult = null;
for (Object value : values) {
  List result = callAndCheckCreateFields(fieldName, 
pointType, value);
  assertNotNull(value + " => null", result);
  if (null == firstResult) {
firstResult = result;
  } else {
// nocommit: why doesn't this check pass? are some IndexableFields not 
implementing equals?
// assertEquals(firstResult, result);
  }
}
  }

  /** nocommit: jdocs, returns the result of createFields after doing some 
sanity checks */
  private List callAndCheckCreateFields(final String fieldName, 
final Class pointType, final Object value) throws Exception {


final SchemaField sf = h.getCore().getLatestSchema().getField(fieldName);

final List results = sf.createFields(value);
final Set resultSet = new LinkedHashSet<>(results);
assertEquals("duplicates found in results? " + results.toString(),
 results.size(), resultSet.size());

final Set resultClasses = new HashSet<>();
for (IndexableField f : results) {
  resultClasses.add(f.getClass());
  
  if ( ! sf.hasDocValues() ) {
assertFalse(f.toString(),
(f instanceof NumericDocValuesField) ||
(f instanceof SortedNumericDocValuesField));
  }
}

assertEquals("stored", sf.stored(), 
resultClasses.contains(StoredField.class));
assertEquals("indexed", sf.indexed(), resultClasses.contains(pointType));
if (sf.multiValued()) {
  assertEquals("docvalues", sf.hasDocValues(), 
resultClasses.contains(SortedNumericDocValuesField.class));
} else {
  assertEquals("docvalues", sf.hasDocValues(), 
resultClasses.contains(NumericDocValuesField.class));
}

return results;
  }

{code}

> PointFields ignore indexed="false"
> --
>
> Key: SOLR-10425
> URL: https://issues.apache.org/jira/browse/SOLR-10425
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Hoss Man
> Attachments: SOLR-10425_hoss.patch, SOLR-10425.patch, 
> SOLR-10425.patch, SOLR-10425.patch
>
>
> (NOTE: description below focuses on {{IntPointField}}, but problem seems to 
> affect all {{PointField}} subclasses)
> There seems to be a disconnect between {{PointField.createFields}} -> 
> {{IntPointField.createField}} -> {{PointField.isFieldUsed}} that results in 
> an {{org.apache.lucene.document.IntPoint}} being created for each field 
> value, even if field is {{indexed="false"}}
> Steps to reproduce...
> {noformat}
> bin/solr -e techproducts
> ...
> curl -X POST -H 'Content-type:application/json' --data-binary '{
>   "add-field":{
>  "name":"hoss_points_check",
>  "type":"pint",
>  "stored":true,
>  "docValues":false,
>  "indexed":false}
> }' http://localhost:8983/solr/techproducts/schema
> ...
> curl -X POST -H 'Content-type:application/json' --data-binary 
> '[{"id":"HOSS","hoss_points_check":42}]' 
> 'http://localhost:8983/solr/techproducts/update/json?commit=true'
> ...
> curl 'http://localhost:8983/solr/techproducts/query?q=id:HOSS'
> {
>   "responseHeader":{
> "status":0,
> "QTime":3,
> "params":{
>   "q":"id:HOSS"}},
>   "response":{"numFound":1,"start":0,"docs":[
>   {
> "id":"HOSS",
> "hoss_points_check":42,
> "_version_":1563795876337418240}]
>   }}
> curl 'http://localhost:8983/solr/techproducts/query?q=hoss_points_check:42'
> {
>   "responseHeader":{
> "status":0,
> "QTime":2,
> "params":{
>   "q":"hoss_points_check:42"}},
>   "response":{"numFound":1,"start":0,"docs":[
>   {
> "id":"HOSS",
> "hoss_points_check":42,
> "_version_":1563795876337418240}]
>   }}
> {noformat}
> Note that I can search on the doc using the  

[jira] [Created] (SOLR-10428) CloudSolrClient: Qerying multiple collection aliases leads to SolrException: Collection not found

[Philip Pock (JIRA)]

Philip Pock created SOLR-10428:
--

 Summary: CloudSolrClient: Qerying multiple collection aliases 
leads to SolrException: Collection not found
 Key: SOLR-10428
 URL: https://issues.apache.org/jira/browse/SOLR-10428
 Project: Solr
  Issue Type: Bug
  Security Level: Public (Default Security Level. Issues are Public)
  Components: SolrJ
Affects Versions: 6.4.2, 6.4.0, 6.5, 6.4.1, master (7.0)
Reporter: Philip Pock
Priority: Minor


We have multiple collections and an alias is created for each of them. e.g.:
alias-a -> collection-a, alias-b -> collection-b

We search in multiple collections by passing the aliases of the collections in 
the collections parameter.

{code}solrClient.query("alias-a,alias-b", params, SolrRequest.METHOD.POST){code}

The client can't find the collection and throws an Exception. Relevant parts of 
the stacktrace using v6.5.0:
{noformat}
org.apache.solr.common.SolrException: Collection not found: collection-a
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.getCollectionNames(CloudSolrClient.java:1394)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:1087)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:1057)
at 
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:160)
at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:974)
{noformat}
Everything works fine with a single alias.
I think this issue was introduced with SOLR-9784. Please see my comment below.

{code:title=org.apache.solr.client.solrj.impl.CloudSolrClient }
Set getCollectionNames(String collection) {
List rawCollectionsList = StrUtils.splitSmart(collection, ",", 
true);
Set collectionNames = new HashSet<>();
for (String collectionName : rawCollectionsList) {
  if (stateProvider.getState(collectionName) == null) {
// I assume that collectionName should be passed to getAlias here
String alias = stateProvider.getAlias(collection);
if (alias != null) {
  List aliasList = StrUtils.splitSmart(alias, ",", true);
  collectionNames.addAll(aliasList);
  continue;
}

  throw new SolrException(ErrorCode.BAD_REQUEST, "Collection not found: 
" + collectionName);
}

  collectionNames.add(collectionName);
}
return collectionNames;
  }
{code}

The suggested change is similar to the previous revision: 
https://github.com/apache/lucene-solr/commit/5650939a8d41b7bad584947a2c9dcedf3774b8de#diff-c8d54eacd46180b332c86c7ae448abaeL1301



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10426) Add shuffle Streaming Expression

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15957948#comment-15957948
 ] 

ASF subversion and git services commented on SOLR-10426:


Commit 37b6c60548f3288ee057dbd8ce1e1594ab48d314 in lucene-solr's branch 
refs/heads/master from [~joel.bernstein]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=37b6c60 ]

SOLR-10426: Add shuffle Streaming Expression


> Add shuffle Streaming Expression
> 
>
> Key: SOLR-10426
> URL: https://issues.apache.org/jira/browse/SOLR-10426
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Joel Bernstein
>Assignee: Joel Bernstein
> Fix For: 6.6
>
> Attachments: SOLR-10426.patch
>
>
> Currently the concept of MapReduce shuffling is lumped into the *search* 
> Streaming Expression. This causes quite a bit of confusion as users sometimes 
> perform a search using the /select handler by default, but actually wanted 
> shuffling behavior which requires the /export handler.
> We can solve this problem by creating a separate function called *shuffle* 
> that always uses the /export handler.
> This will also allow us to clean up some behaviors in the search expression 
> that are somewhat unexpected in future tickets.
> {code}
> parallel(workers, 
>  unique(shuffle(collection1, 
> q=*:*, 
> fl="id,a_s,a_i,a_f", 
> sort="a_f asc, a_i asc", 
> partitionKeys="a_f"), 
> over="a_f"), 
>  workers="2",
>  sort="a_f asc")
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10295) Decide online location for Ref Guide HTML pages

[Cassandra Targett (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15957908#comment-15957908
 ] 

Cassandra Targett commented on SOLR-10295:
--

There are some really good ideas here. 

Let's start with what seems we have broad agreement on. Please let me know if I 
have misrepresented anything here:

# PDF version remains the "official" released Ref Guide.
** This is a release artifact, and is voted on before release, which will 
operate mostly the same as it does today. We vote, it is uploaded to mirrors, 
the release is formally announced, etc.
** Currently this still uses Subversion...we can take that up in a separate 
issue.
# HTML (online) version is considered a "convenience" version, and it will 
exist in multiple versions online
** Online versions correspond to a release, any pending releases, and master
** Doesn't require a vote to publish, can be updated with some degree of 
frequency

Then we have an idea that we need to think through a little bit more:

bq. I would fully automate the build and publishing of the various versions of 
the HTML ref-guide though a new bot

I like the general outlines of this idea. A few things to mention:

* Currently the HTML build has a few dependencies that must be installed 
locally on the build machine (Jekyll is one, which has dependencies on JRuby 
and some other stuff). I don't know how open INFRA would be to that being on 
their machines, we'd have to check.
* I'm not sure of the efficacy of building for every commit. I believe this is 
trying to address the issue of committers being able to see their changes? I 
like the idea of putting a comment in the related JIRA issue to the newly built 
version of the guide, but wonder how useful it will be in practice (and thus if 
it's worth building). I also wonder about publishing a new version for every 
change - I wonder if that will make the docs in a constant state of flux? We 
know from experience that users will more likely use the online version, so we 
want some degree of stability there, even if it's not really the "official" 
version. One thing that might mitigate that is publishing only pages which have 
changed - our build scripts don't handle this at all today, we'd have to 
rethink that some.
* We can add to the top navigation bar some text that changes depending on the 
version being reviewed. There are multiple ways we can do this - if/then 
statements in the Liquid template, Javascript, etc.

While I like this idea and would like to implement it (or some variant of it), 
I wonder if it might be more effective in order to make progress for us to 
split this specific idea into a separate issue to tackle it. We can have an 
easy publication process today - someone builds it on their local machine and 
puts it somewhere. It's the "where" part of that somewhere that we need to get 
resolved. If we split this off, we can work on it in conjunction with the other 
issues that remain unresolved, such as the actual work of the conversion, but 
still be able to cut over to the new approach and add automation shortly 
thereafter.

Anyone have an alternative to my idea of using the CMS to host the docs the 
same way the Javadocs are hosted? (Note, we need to use an apache.org server or 
the comments will not appear...We could check with INFRA about that, but it 
seems it should be apache.org hosted so it can really be considered something 
quasi-official.)


> Decide online location for Ref Guide HTML pages
> ---
>
> Key: SOLR-10295
> URL: https://issues.apache.org/jira/browse/SOLR-10295
> Project: Solr
>  Issue Type: Sub-task
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: documentation
>Reporter: Cassandra Targett
>
> One of the biggest decisions we need to make is where to put the new Solr Ref 
> Guide. Confluence at least had the whole web-hosting bits figured out; we 
> have to figure that out on our own.
> An obvious (maybe only to me) choice is to integrate the Ref Guide with the 
> Solr Website. However, due to the size of the Solr Ref Guide (nearly 200 
> pages), I believe trying to publish it solely with existing CMS tools will 
> create problems similar to those described in the Lucene ReleaseTodo when it 
> comes to publishing the Lucene/Solr javadocs (see 
> https://wiki.apache.org/lucene-java/ReleaseTodo#Website_.2B-.3D_javadocs).
> A solution exists already, and it's what is done for the javadocs. From the 
> above link:
> {quote}
> The solution: skip committing javadocs to the source tree, then staging, then 
> publishing, and instead commit javadocs directly to the production tree. 
> Ordinarily this would be problematic, because the CMS wants to keep the 
> production tree in sync with the staging tree, so anything it finds in the 
> 

[jira] [Commented] (SOLR-8975) SolrClient setters should be deprecated in favor of SolrClientBuilder methods

[Jason Gerlowski (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-8975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15957858#comment-15957858
 ] 

Jason Gerlowski commented on SOLR-8975:
---

So I've taken a few (failed) cracks at this.  I believe in the idea, but 
putting the patch together has been difficult.  {{SolrClient}} is used in so 
many places that the change takes some time to put together, and is quite 
large.  Both of which contribute to the patch encountering merge conflicts from 
other development before it's even uploaded.

In hopes of mitigating this, I would like to break to break this issue down by 
setter.  Should help avoid/lessen some of the pitfalls I've run into so far.

> SolrClient setters should be deprecated in favor of SolrClientBuilder methods
> -
>
> Key: SOLR-8975
> URL: https://issues.apache.org/jira/browse/SOLR-8975
> Project: Solr
>  Issue Type: Improvement
>  Components: SolrJ
>Reporter: Jason Gerlowski
>Priority: Minor
> Attachments: SOLR-8975.patch
>
>
> SOLR-8097 added a builder layer on top of each {{SolrClient}} implementation.
> Now that builders are in place for SolrClients, the setters used in each 
> SolrClient can be deprecated, and their functionality moved over to the 
> Builders.  This change brings a few benefits:
> - unifies SolrClient configuration under the new Builders.  It'll be nice to 
> have all the knobs, and levers used to tweak SolrClients available in a 
> single place (the Builders).
> - reduces SolrClient thread-safety concerns.  Currently, clients are mutable. 
>  Using some SolrClient setters can result in erratic and "trappy" behavior 
> when the clients are used across multiple threads.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-10425) PointFields ignore indexed="false"

[JIRA]

 [ 
https://issues.apache.org/jira/browse/SOLR-10425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tomás Fernández Löbbe updated SOLR-10425:
-
Attachment: SOLR-10425.patch

Merged patch

> PointFields ignore indexed="false"
> --
>
> Key: SOLR-10425
> URL: https://issues.apache.org/jira/browse/SOLR-10425
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Hoss Man
> Attachments: SOLR-10425_hoss.patch, SOLR-10425.patch, 
> SOLR-10425.patch, SOLR-10425.patch
>
>
> (NOTE: description below focuses on {{IntPointField}}, but problem seems to 
> affect all {{PointField}} subclasses)
> There seems to be a disconnect between {{PointField.createFields}} -> 
> {{IntPointField.createField}} -> {{PointField.isFieldUsed}} that results in 
> an {{org.apache.lucene.document.IntPoint}} being created for each field 
> value, even if field is {{indexed="false"}}
> Steps to reproduce...
> {noformat}
> bin/solr -e techproducts
> ...
> curl -X POST -H 'Content-type:application/json' --data-binary '{
>   "add-field":{
>  "name":"hoss_points_check",
>  "type":"pint",
>  "stored":true,
>  "docValues":false,
>  "indexed":false}
> }' http://localhost:8983/solr/techproducts/schema
> ...
> curl -X POST -H 'Content-type:application/json' --data-binary 
> '[{"id":"HOSS","hoss_points_check":42}]' 
> 'http://localhost:8983/solr/techproducts/update/json?commit=true'
> ...
> curl 'http://localhost:8983/solr/techproducts/query?q=id:HOSS'
> {
>   "responseHeader":{
> "status":0,
> "QTime":3,
> "params":{
>   "q":"id:HOSS"}},
>   "response":{"numFound":1,"start":0,"docs":[
>   {
> "id":"HOSS",
> "hoss_points_check":42,
> "_version_":1563795876337418240}]
>   }}
> curl 'http://localhost:8983/solr/techproducts/query?q=hoss_points_check:42'
> {
>   "responseHeader":{
> "status":0,
> "QTime":2,
> "params":{
>   "q":"hoss_points_check:42"}},
>   "response":{"numFound":1,"start":0,"docs":[
>   {
> "id":"HOSS",
> "hoss_points_check":42,
> "_version_":1563795876337418240}]
>   }}
> {noformat}
> Note that I can search on the doc using the  "hoss_points_check" field even 
> though it is {{docValues="false" indexed="false"}}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7767) SortedDocValues.ordValue() should throw in IOException

[Michael McCandless (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15957830#comment-15957830
 ] 

Michael McCandless commented on LUCENE-7767:


+1

> SortedDocValues.ordValue() should throw in IOException
> --
>
> Key: LUCENE-7767
> URL: https://issues.apache.org/jira/browse/LUCENE-7767
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Adrien Grand
> Fix For: master (7.0)
>
>
> It is currently inconsistent with NumericDocValues and BinaryDocValues that 
> both throw an IOException for the equivalent method.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS-EA] Lucene-Solr-6.x-Linux (64bit/jdk-9-ea+162) - Build # 3208 - Unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-6.x-Linux/3208/
Java: 64bit/jdk-9-ea+162 -XX:+UseCompressedOops -XX:+UseG1GC

2 tests failed.
FAILED:  org.apache.solr.TestDistributedSearch.test

Error Message:
Expected to find shardAddress in the up shard info

Stack Trace:
java.lang.AssertionError: Expected to find shardAddress in the up shard info
at 
__randomizedtesting.SeedInfo.seed([746A8B703BF8B2F6:FC3EB4AA9504DF0E]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at 
org.apache.solr.TestDistributedSearch.comparePartialResponses(TestDistributedSearch.java:1176)
at 
org.apache.solr.TestDistributedSearch.queryPartialResults(TestDistributedSearch.java:1117)
at 
org.apache.solr.TestDistributedSearch.test(TestDistributedSearch.java:977)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:547)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsRepeatStatement.callStatement(BaseDistributedSearchTestCase.java:1018)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 

[jira] [Commented] (SOLR-9217) {!join score=..}.. should delay join to createWeight

[gopikannan venugopalsamy (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15957751#comment-15957751
 ] 

gopikannan venugopalsamy commented on SOLR-9217:


[~mkhludnev], Thanks for the explanation, I was trying to assert behavior of 
join  with range query in filter() but it fails during parsing, The same join 
query works with out filter. Is this known?

http://localhost:8983/solr/techproducts/select?q=filter({!join%20from=id%20to=id}id:[1%20TO%205])

org.apache.solr.search.SyntaxError: Cannot parse 'id:[1': Encountered "" 
at line 1, column 5. Was expecting one of: "TO" ...  ... 
 ...

This works
http://localhost:8983/solr/techproducts/select?q={!join%20from=id%20to=id}id:[1%20TO%205]



> {!join score=..}.. should delay join to createWeight
> 
>
> Key: SOLR-9217
> URL: https://issues.apache.org/jira/browse/SOLR-9217
> Project: Solr
>  Issue Type: Improvement
>  Components: query parsers
>Affects Versions: 6.1
>Reporter: Mikhail Khludnev
>Priority: Minor
>  Labels: newbie, newdev
>
> {{ScoreJoinQParserPlugin.XxxCoreJoinQuery}} executes 
> {{JoinUtil.createJoinQuery}} on {{rewrite()}}, but it's inefficient in 
> {{filter(...)}} syntax -or fq- (!) I suppose it's {{filter()}} only problem, 
> not fq. It's better to do that in {{createWeigh()}} as it's done in classic 
> Solr {{JoinQuery}}, {{JoinQParserPlugin}}.
> All existing tests is enough, we just need to assert rewrite behavior - it 
> should rewrite on enclosing range query or so, and doesn't on plain term 
> query.  



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-1105) Use a different stored field for highlighting

[David Smiley (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-1105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15957712#comment-15957712
 ] 

David Smiley commented on SOLR-1105:


bq. Why did you add a boolean flag for this to FieldProperties with the related 
modification to SchemaField accordingly?

Answering my own question... I could imagine that it's useful metadata for a 
non-stored field to declare that some other field is the source of it's 
indexed/analyzed text.  But the schema already internally tracks copyField 
source/destination data.  Maybe what we could do is have highlighting 
automatically work on a non-stored field when we see that the field to be 
highlighted is a copyField target?  Then, in practice, most users wouldn't even 
need to specify hl.contentField (though as an explicit option, it's still nice).

> Use a different stored field for highlighting
> -
>
> Key: SOLR-1105
> URL: https://issues.apache.org/jira/browse/SOLR-1105
> Project: Solr
>  Issue Type: Improvement
>  Components: highlighter
>Reporter: Dmitry Lihachev
>Assignee: David Smiley
> Attachments: SOLR-1105-1_4_1.patch, SOLR-1105.patch, 
> SOLR-1105_shared_content_field_1.3.0.patch
>
>
> DefaultSolrHighlighter uses stored field content to highlight. It has some 
> disadvantages, because index grows up fast when using multilingual indexing 
> due to several fields has to be stored with same content. This patch allows 
> DefaultSolrHighlighter to use "contentField" attribute to loockup content in 
> external field.
> Excerpt from old schema:
> {code:xml}
> 
> 
> 
> 
> {code}
> The same after patching, highlighter will now get content stored in "title" 
> field
> {code:xml}
> 
>  contentField="title"/>
>  contentField="title"/>
>  contentField="title"/>
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: 6.5.1 release?

[Joel Bernstein]

Lots of bugs!

Joel Bernstein
http://joelsolr.blogspot.com/

On Wed, Apr 5, 2017 at 5:02 PM, Tomás Fernández Löbbe  wrote:

> I'd also like to include SOLR-10425. PointFields ignore indexed="false".
> We are working on a fix and some more tests with Hoss. Should be committed
> between today and tomorrow
>
> On Wed, Apr 5, 2017 at 11:48 AM, Steve Rowe  wrote:
>
>> I’d like to include SOLR-10423.  It’s a bug fix in that Solr
>> ShingleFilter queries currently don’t work when sow=false; the patch allows
>> for schema configuration that makes them function by disabling
>> QueryBuilder’s graph query construction.
>>
>> --
>> Steve
>> www.lucidworks.com
>>
>> > On Apr 4, 2017, at 2:29 PM, Christine Poerschke (BLOOMBERG/ LONDON) <
>> cpoersc...@bloomberg.net> wrote:
>> >
>> > I'd also like to include fix for https://issues.apache.org/jira
>> /browse/SOLR-10421 which should be just four lines of actual fix, plus
>> ideally a test for it which will be more than four lines ...
>> >
>> > - Original Message -
>> > From: dev@lucene.apache.org
>> > To: dev@lucene.apache.org
>> > At: 04/04/17 16:54:50
>> >
>> > I'd also like to include SOLR-10277 --- it is a serious problem for
>> > people running large number of collections. I'll review and commit the
>> > patch by tomorrow.
>> >
>> > On Tue, Apr 4, 2017 at 2:16 PM, Shalin Shekhar Mangar
>> >  wrote:
>> >> I would like to include https://issues.apache.org/jira
>> /browse/SOLR-10416
>> >>
>> >> It is a trivial fix.
>> >>
>> >> On Mon, Apr 3, 2017 at 11:54 PM, Joel Bernstein 
>> wrote:
>> >>> SOLR-10404 looks like a nice improvement!
>> >>>
>> >>> I'll shoot for Friday morning to create the release candidate. I've
>> never
>> >>> been a release manager before so I may need some guidance along the
>> way.
>> >>>
>> >>>
>> >>> Joel Bernstein
>> >>> http://joelsolr.blogspot.com/
>> >>>
>> >>> On Mon, Apr 3, 2017 at 12:21 PM, David Smiley <
>> david.w.smi...@gmail.com>
>> >>> wrote:
>> 
>>  Found & fixed a bug: https://issues.apache.org/jira
>> /browse/SOLR-10404  I'd
>>  like to get this into 6.5.1.  You might be interested in this one
>> Joel.
>> 
>>  On Mon, Apr 3, 2017 at 11:58 AM Steve Rowe  wrote:
>> >
>> >
>> >> On Apr 3, 2017, at 11:52 AM, Adrien Grand 
>> wrote:
>> >>
>> >> Building the first RC on April 6th sounds good to me! I'm wondering
>> >> whether the 6.5 Jenkins jobs are still running?
>> >
>> > I disabled the ASF Jenkins 6.5 jobs shortly after the release.  FYI
>> you
>> > can see which Lucene/Solr jobs are enabled here:
>> > .  I’ll re-enable the
>> 6.5 jobs
>> > now.
>> >
>> > --
>> > Steve
>> > www.lucidworks.com
>> >
>> >
>> > 
>> -
>> > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
>> > For additional commands, e-mail: dev-h...@lucene.apache.org
>> >
>>  --
>>  Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker
>>  LinkedIn: http://linkedin.com/in/davidwsmiley | Book:
>>  http://www.solrenterprisesearchserver.com
>> >>>
>> >>>
>> >>
>> >>
>> >>
>> >> --
>> >> Regards,
>> >> Shalin Shekhar Mangar.
>> >
>> >
>> >
>> > --
>> > Regards,
>> > Shalin Shekhar Mangar.
>> >
>> > -
>> > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
>> > For additional commands, e-mail: dev-h...@lucene.apache.org
>> >
>> >
>> >
>> > -
>> > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
>> > For additional commands, e-mail: dev-h...@lucene.apache.org
>> >
>>
>>
>> -
>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
>> For additional commands, e-mail: dev-h...@lucene.apache.org
>>
>>
>

[jira] [Commented] (SOLR-1105) Use a different stored field for highlighting

[David Smiley (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-1105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15957704#comment-15957704
 ] 

David Smiley commented on SOLR-1105:


I propose separating this issue into a Lucene portion and Solr portion.  I have 
some thoughts on the Lucene side but I'll save it for later when you post that.

I like the "hl.contentField" param name.  You declared it in 
{{HighlightParams}} in a spot that I think should be down in the "misc" 
category (pretty minor).

Why did you add a boolean flag for this to FieldProperties with the related 
modification to SchemaField accordingly?

> Use a different stored field for highlighting
> -
>
> Key: SOLR-1105
> URL: https://issues.apache.org/jira/browse/SOLR-1105
> Project: Solr
>  Issue Type: Improvement
>  Components: highlighter
>Reporter: Dmitry Lihachev
>Assignee: David Smiley
> Attachments: SOLR-1105-1_4_1.patch, SOLR-1105.patch, 
> SOLR-1105_shared_content_field_1.3.0.patch
>
>
> DefaultSolrHighlighter uses stored field content to highlight. It has some 
> disadvantages, because index grows up fast when using multilingual indexing 
> due to several fields has to be stored with same content. This patch allows 
> DefaultSolrHighlighter to use "contentField" attribute to loockup content in 
> external field.
> Excerpt from old schema:
> {code:xml}
> 
> 
> 
> 
> {code}
> The same after patching, highlighter will now get content stored in "title" 
> field
> {code:xml}
> 
>  contentField="title"/>
>  contentField="title"/>
>  contentField="title"/>
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10425) PointFields ignore indexed="false"

[JIRA]

[ 
https://issues.apache.org/jira/browse/SOLR-10425?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15957686#comment-15957686
 ] 

Tomás Fernández Löbbe commented on SOLR-10425:
--

thanks Hoss. 
bq. the test schema explicitly declares many fields as 
useDocValuesAsStored="true" but that's actually the default based on the schema 
version=1.6 – making me suspicious of tests that are depending on that, or 
tests assuming that other fields don't have that.
Good point. Maybe we can explicitly set useDocValuesAsStored="true" in other 
fields?
bq. when i tried to write a testNonReturnable it fails for fields that have 
docvalues – even using new fields that i explicitly put 
useDocValuesAsStored=false on.
I'll take a look. In the {{testInternals}} method I just added I included this 
code:
{code:java}
for (LeafReaderContext leave : ir.leaves()) {
LeafReader reader = leave.reader();
for (int i = 0; i < reader.numDocs(); i++) {
  Document doc = reader.document(i, Collections.singleton(field));
  if (sf.stored()) {
assertNotNull(doc.get(field));
  } else {
assertNull(doc.get(field));
  }
}
  }
{code}
That passes the tests...

> PointFields ignore indexed="false"
> --
>
> Key: SOLR-10425
> URL: https://issues.apache.org/jira/browse/SOLR-10425
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Hoss Man
> Attachments: SOLR-10425_hoss.patch, SOLR-10425.patch, SOLR-10425.patch
>
>
> (NOTE: description below focuses on {{IntPointField}}, but problem seems to 
> affect all {{PointField}} subclasses)
> There seems to be a disconnect between {{PointField.createFields}} -> 
> {{IntPointField.createField}} -> {{PointField.isFieldUsed}} that results in 
> an {{org.apache.lucene.document.IntPoint}} being created for each field 
> value, even if field is {{indexed="false"}}
> Steps to reproduce...
> {noformat}
> bin/solr -e techproducts
> ...
> curl -X POST -H 'Content-type:application/json' --data-binary '{
>   "add-field":{
>  "name":"hoss_points_check",
>  "type":"pint",
>  "stored":true,
>  "docValues":false,
>  "indexed":false}
> }' http://localhost:8983/solr/techproducts/schema
> ...
> curl -X POST -H 'Content-type:application/json' --data-binary 
> '[{"id":"HOSS","hoss_points_check":42}]' 
> 'http://localhost:8983/solr/techproducts/update/json?commit=true'
> ...
> curl 'http://localhost:8983/solr/techproducts/query?q=id:HOSS'
> {
>   "responseHeader":{
> "status":0,
> "QTime":3,
> "params":{
>   "q":"id:HOSS"}},
>   "response":{"numFound":1,"start":0,"docs":[
>   {
> "id":"HOSS",
> "hoss_points_check":42,
> "_version_":1563795876337418240}]
>   }}
> curl 'http://localhost:8983/solr/techproducts/query?q=hoss_points_check:42'
> {
>   "responseHeader":{
> "status":0,
> "QTime":2,
> "params":{
>   "q":"hoss_points_check:42"}},
>   "response":{"numFound":1,"start":0,"docs":[
>   {
> "id":"HOSS",
> "hoss_points_check":42,
> "_version_":1563795876337418240}]
>   }}
> {noformat}
> Note that I can search on the doc using the  "hoss_points_check" field even 
> though it is {{docValues="false" indexed="false"}}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: 6.5.1 release?

[Tomás Fernández Löbbe]

I'd also like to include SOLR-10425. PointFields ignore indexed="false". We
are working on a fix and some more tests with Hoss. Should be committed
between today and tomorrow

On Wed, Apr 5, 2017 at 11:48 AM, Steve Rowe  wrote:

> I’d like to include SOLR-10423.  It’s a bug fix in that Solr ShingleFilter
> queries currently don’t work when sow=false; the patch allows for schema
> configuration that makes them function by disabling QueryBuilder’s graph
> query construction.
>
> --
> Steve
> www.lucidworks.com
>
> > On Apr 4, 2017, at 2:29 PM, Christine Poerschke (BLOOMBERG/ LONDON) <
> cpoersc...@bloomberg.net> wrote:
> >
> > I'd also like to include fix for https://issues.apache.org/
> jira/browse/SOLR-10421 which should be just four lines of actual fix,
> plus ideally a test for it which will be more than four lines ...
> >
> > - Original Message -
> > From: dev@lucene.apache.org
> > To: dev@lucene.apache.org
> > At: 04/04/17 16:54:50
> >
> > I'd also like to include SOLR-10277 --- it is a serious problem for
> > people running large number of collections. I'll review and commit the
> > patch by tomorrow.
> >
> > On Tue, Apr 4, 2017 at 2:16 PM, Shalin Shekhar Mangar
> >  wrote:
> >> I would like to include https://issues.apache.org/
> jira/browse/SOLR-10416
> >>
> >> It is a trivial fix.
> >>
> >> On Mon, Apr 3, 2017 at 11:54 PM, Joel Bernstein 
> wrote:
> >>> SOLR-10404 looks like a nice improvement!
> >>>
> >>> I'll shoot for Friday morning to create the release candidate. I've
> never
> >>> been a release manager before so I may need some guidance along the
> way.
> >>>
> >>>
> >>> Joel Bernstein
> >>> http://joelsolr.blogspot.com/
> >>>
> >>> On Mon, Apr 3, 2017 at 12:21 PM, David Smiley <
> david.w.smi...@gmail.com>
> >>> wrote:
> 
>  Found & fixed a bug: https://issues.apache.org/jira/browse/SOLR-10404
> I'd
>  like to get this into 6.5.1.  You might be interested in this one
> Joel.
> 
>  On Mon, Apr 3, 2017 at 11:58 AM Steve Rowe  wrote:
> >
> >
> >> On Apr 3, 2017, at 11:52 AM, Adrien Grand 
> wrote:
> >>
> >> Building the first RC on April 6th sounds good to me! I'm wondering
> >> whether the 6.5 Jenkins jobs are still running?
> >
> > I disabled the ASF Jenkins 6.5 jobs shortly after the release.  FYI
> you
> > can see which Lucene/Solr jobs are enabled here:
> > .  I’ll re-enable the
> 6.5 jobs
> > now.
> >
> > --
> > Steve
> > www.lucidworks.com
> >
> >
> > 
> -
> > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> > For additional commands, e-mail: dev-h...@lucene.apache.org
> >
>  --
>  Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker
>  LinkedIn: http://linkedin.com/in/davidwsmiley | Book:
>  http://www.solrenterprisesearchserver.com
> >>>
> >>>
> >>
> >>
> >>
> >> --
> >> Regards,
> >> Shalin Shekhar Mangar.
> >
> >
> >
> > --
> > Regards,
> > Shalin Shekhar Mangar.
> >
> > -
> > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> > For additional commands, e-mail: dev-h...@lucene.apache.org
> >
> >
> >
> > -
> > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> > For additional commands, e-mail: dev-h...@lucene.apache.org
> >
>
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org
>
>

[jira] [Assigned] (SOLR-10323) SpellingQueryConverter does not remove ":" char when using fielded queries

[James Dyer (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-10323?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

James Dyer reassigned SOLR-10323:
-

Assignee: James Dyer

> SpellingQueryConverter does not remove ":" char when using fielded queries
> --
>
> Key: SOLR-10323
> URL: https://issues.apache.org/jira/browse/SOLR-10323
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: spellchecker
>Affects Versions: 6.2.1
>Reporter: Michael Pellegrini
>Assignee: James Dyer
> Attachments: SOLR-10323.patch, SOLR-10323.patch
>
>
> If you pass a fielded query to {{SpellingQueryConverter.convert}}, it returns 
> a token that is prefixed with a ":" char.
> Example: The query "foo:bar" is converted to ":bar"
> This bug seems to have been introduced by the fix for SOLR-2556. Before this 
> patch, {{SpellingQueryConverter.convert}} returned tokens without the leading 
> colon char.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-10423) ShingleFilter causes overly restrictive queries to be produced

[Steve Rowe (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-10423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Steve Rowe updated SOLR-10423:
--
Fix Version/s: 6.6
   master (7.0)

> ShingleFilter causes overly restrictive queries to be produced
> --
>
> Key: SOLR-10423
> URL: https://issues.apache.org/jira/browse/SOLR-10423
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: query parsers
>Affects Versions: 6.5
>Reporter: Steve Rowe
>Assignee: Steve Rowe
> Fix For: master (7.0), 6.6, 6.5.1
>
> Attachments: SOLR-10423.patch
>
>
> When {{sow=false}} and {{ShingleFilter}} is included in the query analyzer, 
> {{QueryBuilder}} produces queries that inappropriately require sequential 
> terms.  E.g. the query "A B C" produces {{(+A_B +B_C) A_B_C}} when the query 
> analyzer includes {{ maxShingleSize="3" outputUnigrams="false" tokenSeparator="_"/>}}.
> Aman Deep Singh reported this problem on the solr-user list. From 
> [http://mail-archives.apache.org/mod_mbox/lucene-solr-user/201703.mbox/%3ccanegtx9bwbpwqc-cxieac7qsas7x2tgzovomy5ztiagco1p...@mail.gmail.com%3e]:
> {quote}
> I was trying to use the shingle filter but it was not creating the query as
> desirable.
> my schema is
> {noformat}
>  positionIncrementGap="100">
>   
> 
>  maxShingleSize="4"/>
> 
>   
> 
> 
> {noformat}
> my solr query is
> {noformat}
> http://localhost:8983/solr/productCollection/select?
>  defType=edismax
> =true
> =one%20plus%20one%20four
> =nameShingle
> =false
> =xml
> {noformat}
> and it was creating the parsed query as
> {noformat}
> 
> (+(DisjunctionMaxQuery(((+nameShingle:one plus +nameShingle:plus one
> +nameShingle:one four))) DisjunctionMaxQuery(((+nameShingle:one plus
> +nameShingle:plus one four))) DisjunctionMaxQuery(((+nameShingle:one plus one 
> +nameShingle:one four))) DisjunctionMaxQuery((nameShingle:one plus one 
> four)))~1)/no_coord
> 
> 
> *++nameShingle:one plus +nameShingle:plus one +nameShingle:one four))
> ((+nameShingle:one plus +nameShingle:plus one four)) ((+nameShingle:one
> plus one +nameShingle:one four)) (nameShingle:one plus one four))~1)*
> 
> {noformat}
> So ideally token creations is perfect but in the query it is using boolean + 
> operator which is causing the problem as if i have a document with name as 
> "one plus one" ,according to the shingles it has to matched as its token will 
> be  ("one plus","one plus one","plus one") .
> I have tried using the q.op and played around the mm also but nothing is
> giving me the correct response.
> Any idea how i can fetch that document even if the document is missing any
> token.
> My expected response will be getting the document "one plus one" even the 
> user query has any additional term like "one plus one two" and so on.
> {quote}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (SOLR-10423) ShingleFilter causes overly restrictive queries to be produced

[Steve Rowe (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-10423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Steve Rowe resolved SOLR-10423.
---
   Resolution: Fixed
 Assignee: Steve Rowe
Fix Version/s: 6.5.1

> ShingleFilter causes overly restrictive queries to be produced
> --
>
> Key: SOLR-10423
> URL: https://issues.apache.org/jira/browse/SOLR-10423
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: query parsers
>Affects Versions: 6.5
>Reporter: Steve Rowe
>Assignee: Steve Rowe
> Fix For: 6.5.1
>
> Attachments: SOLR-10423.patch
>
>
> When {{sow=false}} and {{ShingleFilter}} is included in the query analyzer, 
> {{QueryBuilder}} produces queries that inappropriately require sequential 
> terms.  E.g. the query "A B C" produces {{(+A_B +B_C) A_B_C}} when the query 
> analyzer includes {{ maxShingleSize="3" outputUnigrams="false" tokenSeparator="_"/>}}.
> Aman Deep Singh reported this problem on the solr-user list. From 
> [http://mail-archives.apache.org/mod_mbox/lucene-solr-user/201703.mbox/%3ccanegtx9bwbpwqc-cxieac7qsas7x2tgzovomy5ztiagco1p...@mail.gmail.com%3e]:
> {quote}
> I was trying to use the shingle filter but it was not creating the query as
> desirable.
> my schema is
> {noformat}
>  positionIncrementGap="100">
>   
> 
>  maxShingleSize="4"/>
> 
>   
> 
> 
> {noformat}
> my solr query is
> {noformat}
> http://localhost:8983/solr/productCollection/select?
>  defType=edismax
> =true
> =one%20plus%20one%20four
> =nameShingle
> =false
> =xml
> {noformat}
> and it was creating the parsed query as
> {noformat}
> 
> (+(DisjunctionMaxQuery(((+nameShingle:one plus +nameShingle:plus one
> +nameShingle:one four))) DisjunctionMaxQuery(((+nameShingle:one plus
> +nameShingle:plus one four))) DisjunctionMaxQuery(((+nameShingle:one plus one 
> +nameShingle:one four))) DisjunctionMaxQuery((nameShingle:one plus one 
> four)))~1)/no_coord
> 
> 
> *++nameShingle:one plus +nameShingle:plus one +nameShingle:one four))
> ((+nameShingle:one plus +nameShingle:plus one four)) ((+nameShingle:one
> plus one +nameShingle:one four)) (nameShingle:one plus one four))~1)*
> 
> {noformat}
> So ideally token creations is perfect but in the query it is using boolean + 
> operator which is causing the problem as if i have a document with name as 
> "one plus one" ,according to the shingles it has to matched as its token will 
> be  ("one plus","one plus one","plus one") .
> I have tried using the q.op and played around the mm also but nothing is
> giving me the correct response.
> Any idea how i can fetch that document even if the document is missing any
> token.
> My expected response will be getting the document "one plus one" even the 
> user query has any additional term like "one plus one two" and so on.
> {quote}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10423) ShingleFilter causes overly restrictive queries to be produced

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10423?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15957656#comment-15957656
 ] 

ASF subversion and git services commented on SOLR-10423:


Commit dbd22a6ada774eb30aee4b9312eb7913dee6890e in lucene-solr's branch 
refs/heads/master from [~steve_rowe]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=dbd22a6 ]

SOLR-10423: Disable graph query production via schema configuration .  This fixes broken queries for 
ShingleFilter-containing query-time analyzers when request param sow=false.


> ShingleFilter causes overly restrictive queries to be produced
> --
>
> Key: SOLR-10423
> URL: https://issues.apache.org/jira/browse/SOLR-10423
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: query parsers
>Affects Versions: 6.5
>Reporter: Steve Rowe
> Attachments: SOLR-10423.patch
>
>
> When {{sow=false}} and {{ShingleFilter}} is included in the query analyzer, 
> {{QueryBuilder}} produces queries that inappropriately require sequential 
> terms.  E.g. the query "A B C" produces {{(+A_B +B_C) A_B_C}} when the query 
> analyzer includes {{ maxShingleSize="3" outputUnigrams="false" tokenSeparator="_"/>}}.
> Aman Deep Singh reported this problem on the solr-user list. From 
> [http://mail-archives.apache.org/mod_mbox/lucene-solr-user/201703.mbox/%3ccanegtx9bwbpwqc-cxieac7qsas7x2tgzovomy5ztiagco1p...@mail.gmail.com%3e]:
> {quote}
> I was trying to use the shingle filter but it was not creating the query as
> desirable.
> my schema is
> {noformat}
>  positionIncrementGap="100">
>   
> 
>  maxShingleSize="4"/>
> 
>   
> 
> 
> {noformat}
> my solr query is
> {noformat}
> http://localhost:8983/solr/productCollection/select?
>  defType=edismax
> =true
> =one%20plus%20one%20four
> =nameShingle
> =false
> =xml
> {noformat}
> and it was creating the parsed query as
> {noformat}
> 
> (+(DisjunctionMaxQuery(((+nameShingle:one plus +nameShingle:plus one
> +nameShingle:one four))) DisjunctionMaxQuery(((+nameShingle:one plus
> +nameShingle:plus one four))) DisjunctionMaxQuery(((+nameShingle:one plus one 
> +nameShingle:one four))) DisjunctionMaxQuery((nameShingle:one plus one 
> four)))~1)/no_coord
> 
> 
> *++nameShingle:one plus +nameShingle:plus one +nameShingle:one four))
> ((+nameShingle:one plus +nameShingle:plus one four)) ((+nameShingle:one
> plus one +nameShingle:one four)) (nameShingle:one plus one four))~1)*
> 
> {noformat}
> So ideally token creations is perfect but in the query it is using boolean + 
> operator which is causing the problem as if i have a document with name as 
> "one plus one" ,according to the shingles it has to matched as its token will 
> be  ("one plus","one plus one","plus one") .
> I have tried using the q.op and played around the mm also but nothing is
> giving me the correct response.
> Any idea how i can fetch that document even if the document is missing any
> token.
> My expected response will be getting the document "one plus one" even the 
> user query has any additional term like "one plus one two" and so on.
> {quote}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10423) ShingleFilter causes overly restrictive queries to be produced

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10423?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15957654#comment-15957654
 ] 

ASF subversion and git services commented on SOLR-10423:


Commit 3a6fbd741110b04d590ced10375b076321fb8bf7 in lucene-solr's branch 
refs/heads/branch_6x from [~steve_rowe]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=3a6fbd7 ]

SOLR-10423: Disable graph query production via schema configuration .  This fixes broken queries for 
ShingleFilter-containing query-time analyzers when request param sow=false.


> ShingleFilter causes overly restrictive queries to be produced
> --
>
> Key: SOLR-10423
> URL: https://issues.apache.org/jira/browse/SOLR-10423
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: query parsers
>Affects Versions: 6.5
>Reporter: Steve Rowe
> Attachments: SOLR-10423.patch
>
>
> When {{sow=false}} and {{ShingleFilter}} is included in the query analyzer, 
> {{QueryBuilder}} produces queries that inappropriately require sequential 
> terms.  E.g. the query "A B C" produces {{(+A_B +B_C) A_B_C}} when the query 
> analyzer includes {{ maxShingleSize="3" outputUnigrams="false" tokenSeparator="_"/>}}.
> Aman Deep Singh reported this problem on the solr-user list. From 
> [http://mail-archives.apache.org/mod_mbox/lucene-solr-user/201703.mbox/%3ccanegtx9bwbpwqc-cxieac7qsas7x2tgzovomy5ztiagco1p...@mail.gmail.com%3e]:
> {quote}
> I was trying to use the shingle filter but it was not creating the query as
> desirable.
> my schema is
> {noformat}
>  positionIncrementGap="100">
>   
> 
>  maxShingleSize="4"/>
> 
>   
> 
> 
> {noformat}
> my solr query is
> {noformat}
> http://localhost:8983/solr/productCollection/select?
>  defType=edismax
> =true
> =one%20plus%20one%20four
> =nameShingle
> =false
> =xml
> {noformat}
> and it was creating the parsed query as
> {noformat}
> 
> (+(DisjunctionMaxQuery(((+nameShingle:one plus +nameShingle:plus one
> +nameShingle:one four))) DisjunctionMaxQuery(((+nameShingle:one plus
> +nameShingle:plus one four))) DisjunctionMaxQuery(((+nameShingle:one plus one 
> +nameShingle:one four))) DisjunctionMaxQuery((nameShingle:one plus one 
> four)))~1)/no_coord
> 
> 
> *++nameShingle:one plus +nameShingle:plus one +nameShingle:one four))
> ((+nameShingle:one plus +nameShingle:plus one four)) ((+nameShingle:one
> plus one +nameShingle:one four)) (nameShingle:one plus one four))~1)*
> 
> {noformat}
> So ideally token creations is perfect but in the query it is using boolean + 
> operator which is causing the problem as if i have a document with name as 
> "one plus one" ,according to the shingles it has to matched as its token will 
> be  ("one plus","one plus one","plus one") .
> I have tried using the q.op and played around the mm also but nothing is
> giving me the correct response.
> Any idea how i can fetch that document even if the document is missing any
> token.
> My expected response will be getting the document "one plus one" even the 
> user query has any additional term like "one plus one two" and so on.
> {quote}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10423) ShingleFilter causes overly restrictive queries to be produced

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10423?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15957653#comment-15957653
 ] 

ASF subversion and git services commented on SOLR-10423:


Commit 213e50a982e7a6f4ecb0d47178e7509393b74a7a in lucene-solr's branch 
refs/heads/branch_6_5 from [~steve_rowe]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=213e50a ]

SOLR-10423: Disable graph query production via schema configuration .  This fixes broken queries for 
ShingleFilter-containing query-time analyzers when request param sow=false.

Conflicts:
solr/core/src/java/org/apache/solr/parser/QueryParser.java
solr/core/src/java/org/apache/solr/parser/QueryParser.jj
solr/core/src/java/org/apache/solr/search/ExtendedDismaxQParser.java
solr/core/src/test/org/apache/solr/search/TestSolrQueryParser.java


> ShingleFilter causes overly restrictive queries to be produced
> --
>
> Key: SOLR-10423
> URL: https://issues.apache.org/jira/browse/SOLR-10423
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: query parsers
>Affects Versions: 6.5
>Reporter: Steve Rowe
> Attachments: SOLR-10423.patch
>
>
> When {{sow=false}} and {{ShingleFilter}} is included in the query analyzer, 
> {{QueryBuilder}} produces queries that inappropriately require sequential 
> terms.  E.g. the query "A B C" produces {{(+A_B +B_C) A_B_C}} when the query 
> analyzer includes {{ maxShingleSize="3" outputUnigrams="false" tokenSeparator="_"/>}}.
> Aman Deep Singh reported this problem on the solr-user list. From 
> [http://mail-archives.apache.org/mod_mbox/lucene-solr-user/201703.mbox/%3ccanegtx9bwbpwqc-cxieac7qsas7x2tgzovomy5ztiagco1p...@mail.gmail.com%3e]:
> {quote}
> I was trying to use the shingle filter but it was not creating the query as
> desirable.
> my schema is
> {noformat}
>  positionIncrementGap="100">
>   
> 
>  maxShingleSize="4"/>
> 
>   
> 
> 
> {noformat}
> my solr query is
> {noformat}
> http://localhost:8983/solr/productCollection/select?
>  defType=edismax
> =true
> =one%20plus%20one%20four
> =nameShingle
> =false
> =xml
> {noformat}
> and it was creating the parsed query as
> {noformat}
> 
> (+(DisjunctionMaxQuery(((+nameShingle:one plus +nameShingle:plus one
> +nameShingle:one four))) DisjunctionMaxQuery(((+nameShingle:one plus
> +nameShingle:plus one four))) DisjunctionMaxQuery(((+nameShingle:one plus one 
> +nameShingle:one four))) DisjunctionMaxQuery((nameShingle:one plus one 
> four)))~1)/no_coord
> 
> 
> *++nameShingle:one plus +nameShingle:plus one +nameShingle:one four))
> ((+nameShingle:one plus +nameShingle:plus one four)) ((+nameShingle:one
> plus one +nameShingle:one four)) (nameShingle:one plus one four))~1)*
> 
> {noformat}
> So ideally token creations is perfect but in the query it is using boolean + 
> operator which is causing the problem as if i have a document with name as 
> "one plus one" ,according to the shingles it has to matched as its token will 
> be  ("one plus","one plus one","plus one") .
> I have tried using the q.op and played around the mm also but nothing is
> giving me the correct response.
> Any idea how i can fetch that document even if the document is missing any
> token.
> My expected response will be getting the document "one plus one" even the 
> user query has any additional term like "one plus one two" and so on.
> {quote}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (SOLR-10346) Clean up static page HTML top nav

[Cassandra Targett (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-10346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Cassandra Targett resolved SOLR-10346.
--
Resolution: Fixed

I'm going to call this resolved, and if other ideas for what the top nav should 
look like are proposed, we'll address them in a new issue.

> Clean up static page HTML top nav
> -
>
> Key: SOLR-10346
> URL: https://issues.apache.org/jira/browse/SOLR-10346
> Project: Solr
>  Issue Type: Sub-task
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: documentation
>Reporter: Cassandra Targett
> Attachments: SRG-top-nav-20170322.png, SRG-topnav-20170330.png
>
>
> For demo purposes, the top navigation bar for the HTML version of the Ref 
> Guide includes some stuff we probably don't want in production. This should 
> be cleaned up and finalized.
> I'll attach a screenshot of the current nav for reference. It currently has 
> these sections:
> * Home link. This should be made dynamic to update automatically for each 
> version the Guide applies to
> * News. Probably don't need this? Today it goes nowhere, but it could go to 
> the News section of the Solr website.
> * Jekyll Resources. Links to stuff about Jekyll. We don't want this.
> * Solr Resources. Links to Javadocs, Source code and Community page of Solr 
> website. Javadoc links should be dynamic.
> * Feedback. Javascript to open local Mail application to send an email. 
> Currently goes to my apache.org address, which I don't want.
> * Search box. This can stay, and we can modify it to do whatever we want it 
> to do when SOLR-10299 is resolved.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-10425) PointFields ignore indexed="false"

[Hoss Man (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-10425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Hoss Man updated SOLR-10425:

Attachment: SOLR-10425_hoss.patch

tomas: per our IRC conversation this morning, here's the (test only) patch i've 
been working on.

there's still some nocommits around beefing up the multivalued exact query 
testing that i'll working on after i grab some lunch, but the 2 other things i 
wanted to draw your attention to (which we probably want to split out into new 
jiras?)...

* the test schema explicitly declares many fields as 
{{useDocValuesAsStored="true"}} but that's actually the default based on the 
schema version=1.6 -- making me suspicious of tests that are depending on that, 
or tests assuming that other fields don't have that.
* when i tried to write a {{testNonReturnable}} it fails for fields that have 
docvalues -- even using new fields that i explicitly put 
{{useDocValuesAsStored=false}} on.

> PointFields ignore indexed="false"
> --
>
> Key: SOLR-10425
> URL: https://issues.apache.org/jira/browse/SOLR-10425
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Hoss Man
> Attachments: SOLR-10425_hoss.patch, SOLR-10425.patch, SOLR-10425.patch
>
>
> (NOTE: description below focuses on {{IntPointField}}, but problem seems to 
> affect all {{PointField}} subclasses)
> There seems to be a disconnect between {{PointField.createFields}} -> 
> {{IntPointField.createField}} -> {{PointField.isFieldUsed}} that results in 
> an {{org.apache.lucene.document.IntPoint}} being created for each field 
> value, even if field is {{indexed="false"}}
> Steps to reproduce...
> {noformat}
> bin/solr -e techproducts
> ...
> curl -X POST -H 'Content-type:application/json' --data-binary '{
>   "add-field":{
>  "name":"hoss_points_check",
>  "type":"pint",
>  "stored":true,
>  "docValues":false,
>  "indexed":false}
> }' http://localhost:8983/solr/techproducts/schema
> ...
> curl -X POST -H 'Content-type:application/json' --data-binary 
> '[{"id":"HOSS","hoss_points_check":42}]' 
> 'http://localhost:8983/solr/techproducts/update/json?commit=true'
> ...
> curl 'http://localhost:8983/solr/techproducts/query?q=id:HOSS'
> {
>   "responseHeader":{
> "status":0,
> "QTime":3,
> "params":{
>   "q":"id:HOSS"}},
>   "response":{"numFound":1,"start":0,"docs":[
>   {
> "id":"HOSS",
> "hoss_points_check":42,
> "_version_":1563795876337418240}]
>   }}
> curl 'http://localhost:8983/solr/techproducts/query?q=hoss_points_check:42'
> {
>   "responseHeader":{
> "status":0,
> "QTime":2,
> "params":{
>   "q":"hoss_points_check:42"}},
>   "response":{"numFound":1,"start":0,"docs":[
>   {
> "id":"HOSS",
> "hoss_points_check":42,
> "_version_":1563795876337418240}]
>   }}
> {noformat}
> Note that I can search on the doc using the  "hoss_points_check" field even 
> though it is {{docValues="false" indexed="false"}}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (SOLR-10422) Consolidate font directories

[Cassandra Targett (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-10422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Cassandra Targett resolved SOLR-10422.
--
Resolution: Fixed

I've committed to the {{jira/solr-10290}} branch these changes:

* consolidated HTML and PDF generation to use a single fonts directory, at 
{{solr-ref-guide/src/fonts}}
** changed {{solr-ref-guide/pdf/themes/refguide-theme.yml}} to point to the new 
directory
* changed the monospace text to use Inconsolata (Open Font licensed) in both 
HTML and PDF
** This also removes the external Google font dependency to pull in Droid Sans 
Mono for the HTML version
* changed HTML and PDF to use Noto Sans for all body text and headers
** previously headers in PDF used Raleway
* removed now unused font files for Raleway, Proxima Nova and Ubuntu Sans Mono
* moved fontawesome and glyphicon font files to new sub-directories under 
{{solr-ref-guide/src/fonts}}
** previously these were in that directory but are now in sub-dirs

> Consolidate font directories
> 
>
> Key: SOLR-10422
> URL: https://issues.apache.org/jira/browse/SOLR-10422
> Project: Solr
>  Issue Type: Sub-task
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: documentation
>Reporter: Cassandra Targett
>Assignee: Cassandra Targett
>Priority: Minor
>
> There are 2 font directories, one used for the HTML and another for the PDF. 
> The directory for the fonts is a parameter, so I think we could consolidate 
> and use only one directory for all fonts.
> (There were 3 directories, but I removed one with 
> https://git1-us-west.apache.org/repos/asf?p=lucene-solr.git;a=commit;h=6472b196372b387a43920781d3b2aad1d1d47544)
> It's not quite related, but maybe a little...the HTML uses Proxima Nova, 
> which may not be open licensed, while the PDF is using Noto Sans, which is 
> Apache licensed. We could further consolidate by changing the HTML to use the 
> same base fonts as the PDF.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-10375) Stored text retrieved via StoredFieldVisitor on doc in the document cache over-estimates needed byte[]

[David Smiley (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-10375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

David Smiley updated SOLR-10375:

Affects Version/s: (was: 6.2.1)
 Priority: Minor  (was: Major)
  Description: 
Using SolrIndexSearcher.doc(int n, StoredFieldVisitor visitor)   (as can happen 
with the UnifiedHighlighter in particular)

If the document cache has the document, will call visitFromCached, will get an 
out of memory error because of line 752 of SolrIndexSearcher - 
visitor.stringField(info, f.stringValue().getBytes(StandardCharsets.UTF_8));

{code}
 at java.lang.OutOfMemoryError.()V (OutOfMemoryError.java:48)
  at java.lang.StringCoding.encode(Ljava/nio/charset/Charset;[CII)[B 
(StringCoding.java:350)
  at java.lang.String.getBytes(Ljava/nio/charset/Charset;)[B (String.java:941)
  at 
org.apache.solr.search.SolrIndexSearcher.visitFromCached(Lorg/apache/lucene/document/Document;Lorg/apache/lucene/index/StoredFieldVisitor;)V
 (SolrIndexSearcher.java:685)
  at 
org.apache.solr.search.SolrIndexSearcher.doc(ILorg/apache/lucene/index/StoredFieldVisitor;)V
 (SolrIndexSearcher.java:652)
{code}

This is due to the current String.getBytes(Charset) implementation, which 
allocates the underlying byte array as a function of 
charArrayLength*maxBytesPerCharacter, which for UTF-8 is 3.  3 * 716MB is over 
Integer.MAX, and the JVM cannot allocate over this, so an out of memory 
exception is thrown because the allocation of this much memory for a single 
array is currently impossible.

The problem is not present when the document cache is disabled.

  was:
Using SolrIndexSearcher.doc(int n, StoredFieldVisitor visitor) - 

If the document cache has the document, will call visitFromCached, will get an 
out of memory error because of line 752 of SolrIndexSearcher - 
visitor.stringField(info, f.stringValue().getBytes(StandardCharsets.UTF_8));

{code}
 at java.lang.OutOfMemoryError.()V (OutOfMemoryError.java:48)
  at java.lang.StringCoding.encode(Ljava/nio/charset/Charset;[CII)[B 
(StringCoding.java:350)
  at java.lang.String.getBytes(Ljava/nio/charset/Charset;)[B (String.java:941)
  at 
org.apache.solr.search.SolrIndexSearcher.visitFromCached(Lorg/apache/lucene/document/Document;Lorg/apache/lucene/index/StoredFieldVisitor;)V
 (SolrIndexSearcher.java:685)
  at 
org.apache.solr.search.SolrIndexSearcher.doc(ILorg/apache/lucene/index/StoredFieldVisitor;)V
 (SolrIndexSearcher.java:652)
{code}

This is due to the current String.getBytes(Charset) implementation, which 
allocates the underlying byte array as a function of 
charArrayLength*maxBytesPerCharacter, which for UTF-8 is 3.  3 * 716MB is over 
Integer.MAX, and the JVM cannot allocate over this, so an out of memory 
exception is thrown because the allocation of this much memory for a single 
array is currently impossible.

The problem is not present when the document cache is disabled.

   Issue Type: Improvement  (was: Bug)
  Summary: Stored text retrieved via StoredFieldVisitor on doc in 
the document cache over-estimates needed byte[]  (was: Stored text > 716MB 
retrieval with StoredFieldVisitor causes out of memory error with document 
cache)

> Stored text retrieved via StoredFieldVisitor on doc in the document cache 
> over-estimates needed byte[]
> --
>
> Key: SOLR-10375
> URL: https://issues.apache.org/jira/browse/SOLR-10375
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
> Environment: Java 1.8.121, Linux x64
>Reporter: Michael Braun
>Priority: Minor
>
> Using SolrIndexSearcher.doc(int n, StoredFieldVisitor visitor)   (as can 
> happen with the UnifiedHighlighter in particular)
> If the document cache has the document, will call visitFromCached, will get 
> an out of memory error because of line 752 of SolrIndexSearcher - 
> visitor.stringField(info, f.stringValue().getBytes(StandardCharsets.UTF_8));
> {code}
>  at java.lang.OutOfMemoryError.()V (OutOfMemoryError.java:48)
>   at java.lang.StringCoding.encode(Ljava/nio/charset/Charset;[CII)[B 
> (StringCoding.java:350)
>   at java.lang.String.getBytes(Ljava/nio/charset/Charset;)[B (String.java:941)
>   at 
> org.apache.solr.search.SolrIndexSearcher.visitFromCached(Lorg/apache/lucene/document/Document;Lorg/apache/lucene/index/StoredFieldVisitor;)V
>  (SolrIndexSearcher.java:685)
>   at 
> org.apache.solr.search.SolrIndexSearcher.doc(ILorg/apache/lucene/index/StoredFieldVisitor;)V
>  (SolrIndexSearcher.java:652)
> {code}
> This is due to the current String.getBytes(Charset) implementation, which 
> allocates the underlying byte array as a function of 
> charArrayLength*maxBytesPerCharacter, which for UTF-8 is 3.  3 * 716MB is 
> over Integer.MAX, and the 

[jira] [Commented] (SOLR-10375) Stored text > 716MB retrieval with StoredFieldVisitor causes out of memory error with document cache

[David Smiley (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15957552#comment-15957552
 ] 

David Smiley commented on SOLR-10375:
-

bq.  at what size/length should Solr be expected to support for stored string 
values? I'd imagine making that call instead does come at some cost overall.

So we pick a threshold just like {{GrowableByteArrayDataOutput.writeString}} 
does.  Before the threshold is the simplest path, albeit one that might use 
larger arrays than necessary.  Over the threshold we scan the text to see how 
big we need to make the byte[].

Another route to take is to override 
{{org.apache.lucene.document.DocumentStoredFieldVisitor#stringField}} to 
conditionally use a Field/IndexableField subclass that holds the byte[] instead 
of immediately converting to a String, leaving the String conversion to occur 
on-demand.  The ultimate char length could be pre-computed and cached as well.  
This path is more work of course.

> Stored text > 716MB retrieval with StoredFieldVisitor causes out of memory 
> error with document cache
> 
>
> Key: SOLR-10375
> URL: https://issues.apache.org/jira/browse/SOLR-10375
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Affects Versions: 6.2.1
> Environment: Java 1.8.121, Linux x64
>Reporter: Michael Braun
>
> Using SolrIndexSearcher.doc(int n, StoredFieldVisitor visitor) - 
> If the document cache has the document, will call visitFromCached, will get 
> an out of memory error because of line 752 of SolrIndexSearcher - 
> visitor.stringField(info, f.stringValue().getBytes(StandardCharsets.UTF_8));
> {code}
>  at java.lang.OutOfMemoryError.()V (OutOfMemoryError.java:48)
>   at java.lang.StringCoding.encode(Ljava/nio/charset/Charset;[CII)[B 
> (StringCoding.java:350)
>   at java.lang.String.getBytes(Ljava/nio/charset/Charset;)[B (String.java:941)
>   at 
> org.apache.solr.search.SolrIndexSearcher.visitFromCached(Lorg/apache/lucene/document/Document;Lorg/apache/lucene/index/StoredFieldVisitor;)V
>  (SolrIndexSearcher.java:685)
>   at 
> org.apache.solr.search.SolrIndexSearcher.doc(ILorg/apache/lucene/index/StoredFieldVisitor;)V
>  (SolrIndexSearcher.java:652)
> {code}
> This is due to the current String.getBytes(Charset) implementation, which 
> allocates the underlying byte array as a function of 
> charArrayLength*maxBytesPerCharacter, which for UTF-8 is 3.  3 * 716MB is 
> over Integer.MAX, and the JVM cannot allocate over this, so an out of memory 
> exception is thrown because the allocation of this much memory for a single 
> array is currently impossible.
> The problem is not present when the document cache is disabled.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-10425) PointFields ignore indexed="false"

[JIRA]

 [ 
https://issues.apache.org/jira/browse/SOLR-10425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tomás Fernández Löbbe updated SOLR-10425:
-
Attachment: SOLR-10425.patch

This patch fixes the issue with the set query

> PointFields ignore indexed="false"
> --
>
> Key: SOLR-10425
> URL: https://issues.apache.org/jira/browse/SOLR-10425
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Hoss Man
> Attachments: SOLR-10425.patch, SOLR-10425.patch
>
>
> (NOTE: description below focuses on {{IntPointField}}, but problem seems to 
> affect all {{PointField}} subclasses)
> There seems to be a disconnect between {{PointField.createFields}} -> 
> {{IntPointField.createField}} -> {{PointField.isFieldUsed}} that results in 
> an {{org.apache.lucene.document.IntPoint}} being created for each field 
> value, even if field is {{indexed="false"}}
> Steps to reproduce...
> {noformat}
> bin/solr -e techproducts
> ...
> curl -X POST -H 'Content-type:application/json' --data-binary '{
>   "add-field":{
>  "name":"hoss_points_check",
>  "type":"pint",
>  "stored":true,
>  "docValues":false,
>  "indexed":false}
> }' http://localhost:8983/solr/techproducts/schema
> ...
> curl -X POST -H 'Content-type:application/json' --data-binary 
> '[{"id":"HOSS","hoss_points_check":42}]' 
> 'http://localhost:8983/solr/techproducts/update/json?commit=true'
> ...
> curl 'http://localhost:8983/solr/techproducts/query?q=id:HOSS'
> {
>   "responseHeader":{
> "status":0,
> "QTime":3,
> "params":{
>   "q":"id:HOSS"}},
>   "response":{"numFound":1,"start":0,"docs":[
>   {
> "id":"HOSS",
> "hoss_points_check":42,
> "_version_":1563795876337418240}]
>   }}
> curl 'http://localhost:8983/solr/techproducts/query?q=hoss_points_check:42'
> {
>   "responseHeader":{
> "status":0,
> "QTime":2,
> "params":{
>   "q":"hoss_points_check:42"}},
>   "response":{"numFound":1,"start":0,"docs":[
>   {
> "id":"HOSS",
> "hoss_points_check":42,
> "_version_":1563795876337418240}]
>   }}
> {noformat}
> Note that I can search on the doc using the  "hoss_points_check" field even 
> though it is {{docValues="false" indexed="false"}}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10303) Add date/time Stream Evaluators

[Dennis Gove (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15957541#comment-15957541
 ] 

Dennis Gove commented on SOLR-10303:


[~covolution] I'm sorry. I forgot to submit the review on Github.

> Add date/time Stream Evaluators
> ---
>
> Key: SOLR-10303
> URL: https://issues.apache.org/jira/browse/SOLR-10303
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Joel Bernstein
> Attachments: SOLR-10303.patch
>
>
> This ticket will add Stream Evaluators that extract date/time values from a 
> Solr date field. The following Evaluators will be supported:
> hour (date)
> minute (date)
> month (date) 
> monthname(date) 
> quarter(date) 
> second (date)
> year(date)
> Syntax:
> {code}
> select(id,
>year(recdate) as year,
>month(recdate) as month,
>day(recdate) as day,
>search(logs, q="blah", fl="id, recdate", sort="recdate asc", 
> qt="/export"))
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10303) Add date/time Stream Evaluators

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15957534#comment-15957534
 ] 

ASF GitHub Bot commented on SOLR-10303:
---

Github user dennisgove commented on a diff in the pull request:

https://github.com/apache/lucene-solr/pull/171#discussion_r107804517
  
--- Diff: 
solr/solrj/src/java/org/apache/solr/client/solrj/io/eval/DatePartEvaluator.java 
---
@@ -0,0 +1,169 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.solr.client.solrj.io.eval;
+
+import java.io.IOException;
+import java.time.Instant;
+import java.time.LocalDateTime;
+import java.time.ZoneOffset;
+import java.time.format.DateTimeParseException;
+import java.time.temporal.ChronoField;
+import java.time.temporal.IsoFields;
+import java.time.temporal.TemporalAccessor;
+import java.time.temporal.UnsupportedTemporalTypeException;
+import java.util.Arrays;
+import java.util.Date;
+import java.util.Locale;
+
+import org.apache.solr.client.solrj.io.Tuple;
+import org.apache.solr.client.solrj.io.stream.expr.Explanation;
+import org.apache.solr.client.solrj.io.stream.expr.StreamExpression;
+import 
org.apache.solr.client.solrj.io.stream.expr.StreamExpressionParameter;
+import org.apache.solr.client.solrj.io.stream.expr.StreamFactory;
+
+/**
+ * Provides numeric Date/Time stream evaluators
+ */
+public class DatePartEvaluator extends NumberEvaluator {
+
+  public enum FUNCTION {year, month, day, dayOfYear, dayOfQuarter, hour, 
minute, quarter, week, second, epoch}
+
+  private final FUNCTION function;
+
+  public DatePartEvaluator(StreamExpression expression, StreamFactory 
factory) throws IOException {
+super(expression, factory);
+
+String functionName = expression.getFunctionName();
+
+try {
+  this.function = FUNCTION.valueOf(functionName);
+} catch (IllegalArgumentException e) {
+  throw new IOException(String.format(Locale.ROOT, "Invalid date 
expression %s - expecting one of %s", functionName, 
Arrays.toString(FUNCTION.values(;
+}
+
+if (1 != subEvaluators.size()) {
+  throw new IOException(String.format(Locale.ROOT, "Invalid expression 
%s - expecting one value but found %d", expression, subEvaluators.size()));
+}
+  }
+
+  @Override
+  public Number evaluate(Tuple tuple) throws IOException {
+
+Instant instant = null;
+TemporalAccessor date = null;
+
+//First evaluate the parameter
+StreamEvaluator streamEvaluator = subEvaluators.get(0);
+Object tupleValue = streamEvaluator.evaluate(tuple);
+
+if (tupleValue == null) return null;
+
+if (tupleValue instanceof String) {
+  instant = getInstant((String) tupleValue);
+} else if (tupleValue instanceof Instant) {
+  instant = (Instant) tupleValue;
+} else if (tupleValue instanceof Date) {
+  instant = ((Date) tupleValue).toInstant();
+} else if (tupleValue instanceof TemporalAccessor) {
+  date = ((TemporalAccessor) tupleValue);
+}
+
+if (instant != null) {
+  if (function.equals(FUNCTION.epoch)) return instant.toEpochMilli();
--- End diff --

Even in a case like this, why not pass the instant down to the `private 
evaluate(Instant)` function. That way all the real decisions are made there.


> Add date/time Stream Evaluators
> ---
>
> Key: SOLR-10303
> URL: https://issues.apache.org/jira/browse/SOLR-10303
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Joel Bernstein
> Attachments: SOLR-10303.patch
>
>
> This ticket will add Stream Evaluators that extract date/time values from a 
> Solr date field. The following Evaluators will be supported:
> hour 

[jira] [Commented] (SOLR-10303) Add date/time Stream Evaluators

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15957533#comment-15957533
 ] 

ASF GitHub Bot commented on SOLR-10303:
---

Github user dennisgove commented on a diff in the pull request:

https://github.com/apache/lucene-solr/pull/171#discussion_r107802207
  
--- Diff: solr/core/src/java/org/apache/solr/handler/StreamHandler.java ---
@@ -199,10 +200,16 @@ public void inform(SolrCore core) {
   .withFunctionName("mult", MultiplyEvaluator.class)
   .withFunctionName("sub", SubtractEvaluator.class)
   .withFunctionName("log", NaturalLogEvaluator.class)
+
   // Conditional Stream Evaluators
   .withFunctionName("if", IfThenElseEvaluator.class)
   ;
 
+  // Date evaluators
--- End diff --

I'm not a huge fan of using the same class to handle multiple functions. 
There are places where we use the class to find the function name and if > 1 
functions are mapped to a class then these lookups no longer work.

See 
[this](https://github.com/dennisgove/lucene-solr/blob/master/solr/solrj/src/java/org/apache/solr/client/solrj/io/stream/expr/StreamFactory.java#L397)
 for where it wouldn't work.


> Add date/time Stream Evaluators
> ---
>
> Key: SOLR-10303
> URL: https://issues.apache.org/jira/browse/SOLR-10303
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Joel Bernstein
> Attachments: SOLR-10303.patch
>
>
> This ticket will add Stream Evaluators that extract date/time values from a 
> Solr date field. The following Evaluators will be supported:
> hour (date)
> minute (date)
> month (date) 
> monthname(date) 
> quarter(date) 
> second (date)
> year(date)
> Syntax:
> {code}
> select(id,
>year(recdate) as year,
>month(recdate) as month,
>day(recdate) as day,
>search(logs, q="blah", fl="id, recdate", sort="recdate asc", 
> qt="/export"))
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10303) Add date/time Stream Evaluators

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15957535#comment-15957535
 ] 

ASF GitHub Bot commented on SOLR-10303:
---

Github user dennisgove commented on a diff in the pull request:

https://github.com/apache/lucene-solr/pull/171#discussion_r107805373
  
--- Diff: 
solr/solrj/src/java/org/apache/solr/client/solrj/io/eval/DatePartEvaluator.java 
---
@@ -0,0 +1,169 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.solr.client.solrj.io.eval;
+
+import java.io.IOException;
+import java.time.Instant;
+import java.time.LocalDateTime;
+import java.time.ZoneOffset;
+import java.time.format.DateTimeParseException;
+import java.time.temporal.ChronoField;
+import java.time.temporal.IsoFields;
+import java.time.temporal.TemporalAccessor;
+import java.time.temporal.UnsupportedTemporalTypeException;
+import java.util.Arrays;
+import java.util.Date;
+import java.util.Locale;
+
+import org.apache.solr.client.solrj.io.Tuple;
+import org.apache.solr.client.solrj.io.stream.expr.Explanation;
+import org.apache.solr.client.solrj.io.stream.expr.StreamExpression;
+import 
org.apache.solr.client.solrj.io.stream.expr.StreamExpressionParameter;
+import org.apache.solr.client.solrj.io.stream.expr.StreamFactory;
+
+/**
+ * Provides numeric Date/Time stream evaluators
+ */
+public class DatePartEvaluator extends NumberEvaluator {
+
+  public enum FUNCTION {year, month, day, dayOfYear, dayOfQuarter, hour, 
minute, quarter, week, second, epoch}
+
+  private final FUNCTION function;
+
+  public DatePartEvaluator(StreamExpression expression, StreamFactory 
factory) throws IOException {
+super(expression, factory);
+
+String functionName = expression.getFunctionName();
+
+try {
+  this.function = FUNCTION.valueOf(functionName);
+} catch (IllegalArgumentException e) {
+  throw new IOException(String.format(Locale.ROOT, "Invalid date 
expression %s - expecting one of %s", functionName, 
Arrays.toString(FUNCTION.values(;
+}
+
+if (1 != subEvaluators.size()) {
+  throw new IOException(String.format(Locale.ROOT, "Invalid expression 
%s - expecting one value but found %d", expression, subEvaluators.size()));
+}
+  }
+
+  @Override
+  public Number evaluate(Tuple tuple) throws IOException {
+
+Instant instant = null;
+TemporalAccessor date = null;
+
+//First evaluate the parameter
+StreamEvaluator streamEvaluator = subEvaluators.get(0);
+Object tupleValue = streamEvaluator.evaluate(tuple);
+
+if (tupleValue == null) return null;
+
+if (tupleValue instanceof String) {
+  instant = getInstant((String) tupleValue);
+} else if (tupleValue instanceof Instant) {
+  instant = (Instant) tupleValue;
+} else if (tupleValue instanceof Date) {
+  instant = ((Date) tupleValue).toInstant();
+} else if (tupleValue instanceof TemporalAccessor) {
+  date = ((TemporalAccessor) tupleValue);
+}
+
+if (instant != null) {
+  if (function.equals(FUNCTION.epoch)) return instant.toEpochMilli();
+  date = LocalDateTime.ofInstant(instant, ZoneOffset.UTC);
+}
+
+if (date != null) {
+  return evaluate(date);
+}
+
+throw new IOException(String.format(Locale.ROOT, "Invalid parameter %s 
- The parameter must be a string formatted ISO_INSTANT or of type Instant,Date 
or LocalDateTime.", String.valueOf(tupleValue)));
+  }
+
+  private Instant getInstant(String dateStr) throws IOException {
+
+if (dateStr != null && !dateStr.isEmpty()) {
+  try {
+return Instant.parse(dateStr);
+  } catch (DateTimeParseException e) {
+throw new IOException(String.format(Locale.ROOT, "Invalid 
parameter %s - The String 

[jira] [Commented] (SOLR-10303) Add date/time Stream Evaluators

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15957536#comment-15957536
 ] 

ASF GitHub Bot commented on SOLR-10303:
---

Github user dennisgove commented on a diff in the pull request:

https://github.com/apache/lucene-solr/pull/171#discussion_r107804393
  
--- Diff: 
solr/solrj/src/java/org/apache/solr/client/solrj/io/eval/DatePartEvaluator.java 
---
@@ -0,0 +1,169 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.solr.client.solrj.io.eval;
+
+import java.io.IOException;
+import java.time.Instant;
+import java.time.LocalDateTime;
+import java.time.ZoneOffset;
+import java.time.format.DateTimeParseException;
+import java.time.temporal.ChronoField;
+import java.time.temporal.IsoFields;
+import java.time.temporal.TemporalAccessor;
+import java.time.temporal.UnsupportedTemporalTypeException;
+import java.util.Arrays;
+import java.util.Date;
+import java.util.Locale;
+
+import org.apache.solr.client.solrj.io.Tuple;
+import org.apache.solr.client.solrj.io.stream.expr.Explanation;
+import org.apache.solr.client.solrj.io.stream.expr.StreamExpression;
+import 
org.apache.solr.client.solrj.io.stream.expr.StreamExpressionParameter;
+import org.apache.solr.client.solrj.io.stream.expr.StreamFactory;
+
+/**
+ * Provides numeric Date/Time stream evaluators
+ */
+public class DatePartEvaluator extends NumberEvaluator {
+
+  public enum FUNCTION {year, month, day, dayOfYear, dayOfQuarter, hour, 
minute, quarter, week, second, epoch}
+
+  private final FUNCTION function;
+
+  public DatePartEvaluator(StreamExpression expression, StreamFactory 
factory) throws IOException {
+super(expression, factory);
+
+String functionName = expression.getFunctionName();
+
+try {
+  this.function = FUNCTION.valueOf(functionName);
+} catch (IllegalArgumentException e) {
+  throw new IOException(String.format(Locale.ROOT, "Invalid date 
expression %s - expecting one of %s", functionName, 
Arrays.toString(FUNCTION.values(;
+}
+
+if (1 != subEvaluators.size()) {
+  throw new IOException(String.format(Locale.ROOT, "Invalid expression 
%s - expecting one value but found %d", expression, subEvaluators.size()));
+}
+  }
+
+  @Override
+  public Number evaluate(Tuple tuple) throws IOException {
+
+Instant instant = null;
+TemporalAccessor date = null;
+
+//First evaluate the parameter
+StreamEvaluator streamEvaluator = subEvaluators.get(0);
+Object tupleValue = streamEvaluator.evaluate(tuple);
+
+if (tupleValue == null) return null;
+
+if (tupleValue instanceof String) {
+  instant = getInstant((String) tupleValue);
+} else if (tupleValue instanceof Instant) {
+  instant = (Instant) tupleValue;
+} else if (tupleValue instanceof Date) {
+  instant = ((Date) tupleValue).toInstant();
+} else if (tupleValue instanceof TemporalAccessor) {
+  date = ((TemporalAccessor) tupleValue);
+}
+
+if (instant != null) {
--- End diff --

I'd rather see this continue to act on an Instant instead of a Date. If the 
value is a TemporalAccessor (above if statement) you can convert that into an 
Instant.


> Add date/time Stream Evaluators
> ---
>
> Key: SOLR-10303
> URL: https://issues.apache.org/jira/browse/SOLR-10303
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Joel Bernstein
> Attachments: SOLR-10303.patch
>
>
> This ticket will add Stream Evaluators that extract date/time values from a 
> Solr date field. The following Evaluators will be supported:
> hour (date)
> minute (date)
> month (date) 
> monthname(date) 
> 

[GitHub] lucene-solr pull request #171: SOLR-10303

[dennisgove]

Github user dennisgove commented on a diff in the pull request:

https://github.com/apache/lucene-solr/pull/171#discussion_r107802207
  
--- Diff: solr/core/src/java/org/apache/solr/handler/StreamHandler.java ---
@@ -199,10 +200,16 @@ public void inform(SolrCore core) {
   .withFunctionName("mult", MultiplyEvaluator.class)
   .withFunctionName("sub", SubtractEvaluator.class)
   .withFunctionName("log", NaturalLogEvaluator.class)
+
   // Conditional Stream Evaluators
   .withFunctionName("if", IfThenElseEvaluator.class)
   ;
 
+  // Date evaluators
--- End diff --

I'm not a huge fan of using the same class to handle multiple functions. 
There are places where we use the class to find the function name and if > 1 
functions are mapped to a class then these lookups no longer work.

See 
[this](https://github.com/dennisgove/lucene-solr/blob/master/solr/solrj/src/java/org/apache/solr/client/solrj/io/stream/expr/StreamFactory.java#L397)
 for where it wouldn't work.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[GitHub] lucene-solr pull request #171: SOLR-10303

[dennisgove]

Github user dennisgove commented on a diff in the pull request:

https://github.com/apache/lucene-solr/pull/171#discussion_r107804393
  
--- Diff: 
solr/solrj/src/java/org/apache/solr/client/solrj/io/eval/DatePartEvaluator.java 
---
@@ -0,0 +1,169 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.solr.client.solrj.io.eval;
+
+import java.io.IOException;
+import java.time.Instant;
+import java.time.LocalDateTime;
+import java.time.ZoneOffset;
+import java.time.format.DateTimeParseException;
+import java.time.temporal.ChronoField;
+import java.time.temporal.IsoFields;
+import java.time.temporal.TemporalAccessor;
+import java.time.temporal.UnsupportedTemporalTypeException;
+import java.util.Arrays;
+import java.util.Date;
+import java.util.Locale;
+
+import org.apache.solr.client.solrj.io.Tuple;
+import org.apache.solr.client.solrj.io.stream.expr.Explanation;
+import org.apache.solr.client.solrj.io.stream.expr.StreamExpression;
+import 
org.apache.solr.client.solrj.io.stream.expr.StreamExpressionParameter;
+import org.apache.solr.client.solrj.io.stream.expr.StreamFactory;
+
+/**
+ * Provides numeric Date/Time stream evaluators
+ */
+public class DatePartEvaluator extends NumberEvaluator {
+
+  public enum FUNCTION {year, month, day, dayOfYear, dayOfQuarter, hour, 
minute, quarter, week, second, epoch}
+
+  private final FUNCTION function;
+
+  public DatePartEvaluator(StreamExpression expression, StreamFactory 
factory) throws IOException {
+super(expression, factory);
+
+String functionName = expression.getFunctionName();
+
+try {
+  this.function = FUNCTION.valueOf(functionName);
+} catch (IllegalArgumentException e) {
+  throw new IOException(String.format(Locale.ROOT, "Invalid date 
expression %s - expecting one of %s", functionName, 
Arrays.toString(FUNCTION.values(;
+}
+
+if (1 != subEvaluators.size()) {
+  throw new IOException(String.format(Locale.ROOT, "Invalid expression 
%s - expecting one value but found %d", expression, subEvaluators.size()));
+}
+  }
+
+  @Override
+  public Number evaluate(Tuple tuple) throws IOException {
+
+Instant instant = null;
+TemporalAccessor date = null;
+
+//First evaluate the parameter
+StreamEvaluator streamEvaluator = subEvaluators.get(0);
+Object tupleValue = streamEvaluator.evaluate(tuple);
+
+if (tupleValue == null) return null;
+
+if (tupleValue instanceof String) {
+  instant = getInstant((String) tupleValue);
+} else if (tupleValue instanceof Instant) {
+  instant = (Instant) tupleValue;
+} else if (tupleValue instanceof Date) {
+  instant = ((Date) tupleValue).toInstant();
+} else if (tupleValue instanceof TemporalAccessor) {
+  date = ((TemporalAccessor) tupleValue);
+}
+
+if (instant != null) {
--- End diff --

I'd rather see this continue to act on an Instant instead of a Date. If the 
value is a TemporalAccessor (above if statement) you can convert that into an 
Instant.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[GitHub] lucene-solr pull request #171: SOLR-10303

[dennisgove]

Github user dennisgove commented on a diff in the pull request:

https://github.com/apache/lucene-solr/pull/171#discussion_r107805373
  
--- Diff: 
solr/solrj/src/java/org/apache/solr/client/solrj/io/eval/DatePartEvaluator.java 
---
@@ -0,0 +1,169 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.solr.client.solrj.io.eval;
+
+import java.io.IOException;
+import java.time.Instant;
+import java.time.LocalDateTime;
+import java.time.ZoneOffset;
+import java.time.format.DateTimeParseException;
+import java.time.temporal.ChronoField;
+import java.time.temporal.IsoFields;
+import java.time.temporal.TemporalAccessor;
+import java.time.temporal.UnsupportedTemporalTypeException;
+import java.util.Arrays;
+import java.util.Date;
+import java.util.Locale;
+
+import org.apache.solr.client.solrj.io.Tuple;
+import org.apache.solr.client.solrj.io.stream.expr.Explanation;
+import org.apache.solr.client.solrj.io.stream.expr.StreamExpression;
+import 
org.apache.solr.client.solrj.io.stream.expr.StreamExpressionParameter;
+import org.apache.solr.client.solrj.io.stream.expr.StreamFactory;
+
+/**
+ * Provides numeric Date/Time stream evaluators
+ */
+public class DatePartEvaluator extends NumberEvaluator {
+
+  public enum FUNCTION {year, month, day, dayOfYear, dayOfQuarter, hour, 
minute, quarter, week, second, epoch}
+
+  private final FUNCTION function;
+
+  public DatePartEvaluator(StreamExpression expression, StreamFactory 
factory) throws IOException {
+super(expression, factory);
+
+String functionName = expression.getFunctionName();
+
+try {
+  this.function = FUNCTION.valueOf(functionName);
+} catch (IllegalArgumentException e) {
+  throw new IOException(String.format(Locale.ROOT, "Invalid date 
expression %s - expecting one of %s", functionName, 
Arrays.toString(FUNCTION.values(;
+}
+
+if (1 != subEvaluators.size()) {
+  throw new IOException(String.format(Locale.ROOT, "Invalid expression 
%s - expecting one value but found %d", expression, subEvaluators.size()));
+}
+  }
+
+  @Override
+  public Number evaluate(Tuple tuple) throws IOException {
+
+Instant instant = null;
+TemporalAccessor date = null;
+
+//First evaluate the parameter
+StreamEvaluator streamEvaluator = subEvaluators.get(0);
+Object tupleValue = streamEvaluator.evaluate(tuple);
+
+if (tupleValue == null) return null;
+
+if (tupleValue instanceof String) {
+  instant = getInstant((String) tupleValue);
+} else if (tupleValue instanceof Instant) {
+  instant = (Instant) tupleValue;
+} else if (tupleValue instanceof Date) {
+  instant = ((Date) tupleValue).toInstant();
+} else if (tupleValue instanceof TemporalAccessor) {
+  date = ((TemporalAccessor) tupleValue);
+}
+
+if (instant != null) {
+  if (function.equals(FUNCTION.epoch)) return instant.toEpochMilli();
+  date = LocalDateTime.ofInstant(instant, ZoneOffset.UTC);
+}
+
+if (date != null) {
+  return evaluate(date);
+}
+
+throw new IOException(String.format(Locale.ROOT, "Invalid parameter %s 
- The parameter must be a string formatted ISO_INSTANT or of type Instant,Date 
or LocalDateTime.", String.valueOf(tupleValue)));
+  }
+
+  private Instant getInstant(String dateStr) throws IOException {
+
+if (dateStr != null && !dateStr.isEmpty()) {
+  try {
+return Instant.parse(dateStr);
+  } catch (DateTimeParseException e) {
+throw new IOException(String.format(Locale.ROOT, "Invalid 
parameter %s - The String must be formatted in the ISO_INSTANT date format.", 
dateStr));
+  }
+}
+return null;
+  }
+
+  /**
+   * Evaluate the date based on the specified function
+   *
+   * @param date
+   * 

[GitHub] lucene-solr pull request #171: SOLR-10303

[dennisgove]

Github user dennisgove commented on a diff in the pull request:

https://github.com/apache/lucene-solr/pull/171#discussion_r107804517
  
--- Diff: 
solr/solrj/src/java/org/apache/solr/client/solrj/io/eval/DatePartEvaluator.java 
---
@@ -0,0 +1,169 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.solr.client.solrj.io.eval;
+
+import java.io.IOException;
+import java.time.Instant;
+import java.time.LocalDateTime;
+import java.time.ZoneOffset;
+import java.time.format.DateTimeParseException;
+import java.time.temporal.ChronoField;
+import java.time.temporal.IsoFields;
+import java.time.temporal.TemporalAccessor;
+import java.time.temporal.UnsupportedTemporalTypeException;
+import java.util.Arrays;
+import java.util.Date;
+import java.util.Locale;
+
+import org.apache.solr.client.solrj.io.Tuple;
+import org.apache.solr.client.solrj.io.stream.expr.Explanation;
+import org.apache.solr.client.solrj.io.stream.expr.StreamExpression;
+import 
org.apache.solr.client.solrj.io.stream.expr.StreamExpressionParameter;
+import org.apache.solr.client.solrj.io.stream.expr.StreamFactory;
+
+/**
+ * Provides numeric Date/Time stream evaluators
+ */
+public class DatePartEvaluator extends NumberEvaluator {
+
+  public enum FUNCTION {year, month, day, dayOfYear, dayOfQuarter, hour, 
minute, quarter, week, second, epoch}
+
+  private final FUNCTION function;
+
+  public DatePartEvaluator(StreamExpression expression, StreamFactory 
factory) throws IOException {
+super(expression, factory);
+
+String functionName = expression.getFunctionName();
+
+try {
+  this.function = FUNCTION.valueOf(functionName);
+} catch (IllegalArgumentException e) {
+  throw new IOException(String.format(Locale.ROOT, "Invalid date 
expression %s - expecting one of %s", functionName, 
Arrays.toString(FUNCTION.values(;
+}
+
+if (1 != subEvaluators.size()) {
+  throw new IOException(String.format(Locale.ROOT, "Invalid expression 
%s - expecting one value but found %d", expression, subEvaluators.size()));
+}
+  }
+
+  @Override
+  public Number evaluate(Tuple tuple) throws IOException {
+
+Instant instant = null;
+TemporalAccessor date = null;
+
+//First evaluate the parameter
+StreamEvaluator streamEvaluator = subEvaluators.get(0);
+Object tupleValue = streamEvaluator.evaluate(tuple);
+
+if (tupleValue == null) return null;
+
+if (tupleValue instanceof String) {
+  instant = getInstant((String) tupleValue);
+} else if (tupleValue instanceof Instant) {
+  instant = (Instant) tupleValue;
+} else if (tupleValue instanceof Date) {
+  instant = ((Date) tupleValue).toInstant();
+} else if (tupleValue instanceof TemporalAccessor) {
+  date = ((TemporalAccessor) tupleValue);
+}
+
+if (instant != null) {
+  if (function.equals(FUNCTION.epoch)) return instant.toEpochMilli();
--- End diff --

Even in a case like this, why not pass the instant down to the `private 
evaluate(Instant)` function. That way all the real decisions are made there.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10303) Add date/time Stream Evaluators

[Gethin James (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15957531#comment-15957531
 ] 

Gethin James commented on SOLR-10303:
-

Sorry, I can't find any comments on the pull request.

> Add date/time Stream Evaluators
> ---
>
> Key: SOLR-10303
> URL: https://issues.apache.org/jira/browse/SOLR-10303
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Joel Bernstein
> Attachments: SOLR-10303.patch
>
>
> This ticket will add Stream Evaluators that extract date/time values from a 
> Solr date field. The following Evaluators will be supported:
> hour (date)
> minute (date)
> month (date) 
> monthname(date) 
> quarter(date) 
> second (date)
> year(date)
> Syntax:
> {code}
> select(id,
>year(recdate) as year,
>month(recdate) as month,
>day(recdate) as day,
>search(logs, q="blah", fl="id, recdate", sort="recdate asc", 
> qt="/export"))
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10323) SpellingQueryConverter does not remove ":" char when using fielded queries

[Michael Pellegrini (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15957516#comment-15957516
 ] 

Michael Pellegrini commented on SOLR-10323:
---

[~jdyer] can you review this patch?

> SpellingQueryConverter does not remove ":" char when using fielded queries
> --
>
> Key: SOLR-10323
> URL: https://issues.apache.org/jira/browse/SOLR-10323
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: spellchecker
>Affects Versions: 6.2.1
>Reporter: Michael Pellegrini
> Attachments: SOLR-10323.patch, SOLR-10323.patch
>
>
> If you pass a fielded query to {{SpellingQueryConverter.convert}}, it returns 
> a token that is prefixed with a ":" char.
> Example: The query "foo:bar" is converted to ":bar"
> This bug seems to have been introduced by the fix for SOLR-2556. Before this 
> patch, {{SpellingQueryConverter.convert}} returned tokens without the leading 
> colon char.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-10425) PointFields ignore indexed="false"

[JIRA]

 [ 
https://issues.apache.org/jira/browse/SOLR-10425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tomás Fernández Löbbe updated SOLR-10425:
-
Attachment: SOLR-10425.patch

This is still WIP. Here is an attempt for fixing this issue and some tests. The 
new tests pass now after the changes in PointField, but other tests fail 
because we try to use PointInSet query for DV-only fields. 

> PointFields ignore indexed="false"
> --
>
> Key: SOLR-10425
> URL: https://issues.apache.org/jira/browse/SOLR-10425
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Hoss Man
> Attachments: SOLR-10425.patch
>
>
> (NOTE: description below focuses on {{IntPointField}}, but problem seems to 
> affect all {{PointField}} subclasses)
> There seems to be a disconnect between {{PointField.createFields}} -> 
> {{IntPointField.createField}} -> {{PointField.isFieldUsed}} that results in 
> an {{org.apache.lucene.document.IntPoint}} being created for each field 
> value, even if field is {{indexed="false"}}
> Steps to reproduce...
> {noformat}
> bin/solr -e techproducts
> ...
> curl -X POST -H 'Content-type:application/json' --data-binary '{
>   "add-field":{
>  "name":"hoss_points_check",
>  "type":"pint",
>  "stored":true,
>  "docValues":false,
>  "indexed":false}
> }' http://localhost:8983/solr/techproducts/schema
> ...
> curl -X POST -H 'Content-type:application/json' --data-binary 
> '[{"id":"HOSS","hoss_points_check":42}]' 
> 'http://localhost:8983/solr/techproducts/update/json?commit=true'
> ...
> curl 'http://localhost:8983/solr/techproducts/query?q=id:HOSS'
> {
>   "responseHeader":{
> "status":0,
> "QTime":3,
> "params":{
>   "q":"id:HOSS"}},
>   "response":{"numFound":1,"start":0,"docs":[
>   {
> "id":"HOSS",
> "hoss_points_check":42,
> "_version_":1563795876337418240}]
>   }}
> curl 'http://localhost:8983/solr/techproducts/query?q=hoss_points_check:42'
> {
>   "responseHeader":{
> "status":0,
> "QTime":2,
> "params":{
>   "q":"hoss_points_check:42"}},
>   "response":{"numFound":1,"start":0,"docs":[
>   {
> "id":"HOSS",
> "hoss_points_check":42,
> "_version_":1563795876337418240}]
>   }}
> {noformat}
> Note that I can search on the doc using the  "hoss_points_check" field even 
> though it is {{docValues="false" indexed="false"}}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10047) Mismatched Docvalue segments cause exception in Sorting/Facting; Uninvert per segment

[Keith Laban (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15957449#comment-15957449
 ] 

Keith Laban commented on SOLR-10047:


- updated javadoc
- renamed class to {{UninvertingDirectoryReaderPerSegmentMapping}}
- added IndexReader#getReaderCacheHelper (copied note from 
UninvertingDirectoryReader)
- removed old now unsued UninvertingDirectoryReader


> Mismatched Docvalue segments cause exception in Sorting/Facting; Uninvert per 
> segment
> -
>
> Key: SOLR-10047
> URL: https://issues.apache.org/jira/browse/SOLR-10047
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Keith Laban
>
> The configuration of UninvertingReader in SolrIndexSearch creates a global 
> mapping for the directory for fields to uninvert. If docvalues are enabled on 
> a field the creation of a new segment will cause the query to fail when 
> faceting/sorting on the recently docvalue enabled field. This happens because 
> the UninvertingReader is configured globally across the entire directory, and 
> a single segment containing DVs for a field will incorrectly indicate that 
> all segments contain DVs.
> This patch addresses the incorrect behavior by determining the fields to be 
> uninverted on a per-segment basis.
> With the fix, it is still recommended that a reindexing occur as data loss 
> will when a DV and non-DV segment are merged, SOLR-10046 addresses this 
> behavior. This fix is to be a stop gap for the time between enabling 
> docvalues and the duration of a reindex. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-10426) Add shuffle Streaming Expression

[Joel Bernstein (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-10426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joel Bernstein updated SOLR-10426:
--
Description: 
Currently the concept of MapReduce shuffling is lumped into the *search* 
Streaming Expression. This causes quite a bit of confusion as users sometimes 
perform a search using the /select handler by default, but actually wanted 
shuffling behavior which requires the /export handler.

We can solve this problem by creating a separate function called *shuffle* that 
always uses the /export handler.

This will also allow us to clean up some behaviors in the search expression 
that are somewhat unexpected in future tickets.
{code}
parallel(workers, 
 unique(shuffle(collection1, 
q=*:*, 
fl="id,a_s,a_i,a_f", 
sort="a_f asc, a_i asc", 
partitionKeys="a_f"), 
over="a_f"), 
 workers="2",
 sort="a_f asc")
{code}

  was:
Currently the concept of MapReduce shuffling is lumped into the *search* 
Streaming Expression. This causes quite a bit of confusion as users perform a 
search using the /select handler by default, but actually wanted shuffling 
behavior which requires the /export handler.

We can solve this problem by creating a separate function called *shuffle* that 
always uses the /export handler.

This will also allow us to clean up some behaviors in the search expression 
that are somewhat unexpected in future tickets.
{code}
parallel(workers, 
 unique(shuffle(collection1, 
q=*:*, 
fl="id,a_s,a_i,a_f", 
sort="a_f asc, a_i asc", 
partitionKeys="a_f"), 
over="a_f"), 
 workers="2",
 sort="a_f asc")
{code}


> Add shuffle Streaming Expression
> 
>
> Key: SOLR-10426
> URL: https://issues.apache.org/jira/browse/SOLR-10426
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Joel Bernstein
>Assignee: Joel Bernstein
> Fix For: 6.6
>
> Attachments: SOLR-10426.patch
>
>
> Currently the concept of MapReduce shuffling is lumped into the *search* 
> Streaming Expression. This causes quite a bit of confusion as users sometimes 
> perform a search using the /select handler by default, but actually wanted 
> shuffling behavior which requires the /export handler.
> We can solve this problem by creating a separate function called *shuffle* 
> that always uses the /export handler.
> This will also allow us to clean up some behaviors in the search expression 
> that are somewhat unexpected in future tickets.
> {code}
> parallel(workers, 
>  unique(shuffle(collection1, 
> q=*:*, 
> fl="id,a_s,a_i,a_f", 
> sort="a_f asc, a_i asc", 
> partitionKeys="a_f"), 
> over="a_f"), 
>  workers="2",
>  sort="a_f asc")
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Assigned] (SOLR-10426) Add shuffle Streaming Expression

[Joel Bernstein (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-10426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joel Bernstein reassigned SOLR-10426:
-

Assignee: Joel Bernstein

> Add shuffle Streaming Expression
> 
>
> Key: SOLR-10426
> URL: https://issues.apache.org/jira/browse/SOLR-10426
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Joel Bernstein
>Assignee: Joel Bernstein
> Fix For: 6.6
>
> Attachments: SOLR-10426.patch
>
>
> Currently the concept of MapReduce shuffling is lumped into the *search* 
> Streaming Expression. This causes quite a bit of confusion as users perform a 
> search using the /select handler by default, but actually wanted shuffling 
> behavior which requires the /export handler.
> We can solve this problem by creating a separate function called *shuffle* 
> that always uses the /export handler.
> This will also allow us to clean up some behaviors in the search expression 
> that are somewhat unexpected in future tickets.
> {code}
> parallel(workers, 
>  unique(shuffle(collection1, 
> q=*:*, 
> fl="id,a_s,a_i,a_f", 
> sort="a_f asc, a_i asc", 
> partitionKeys="a_f"), 
> over="a_f"), 
>  workers="2",
>  sort="a_f asc")
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-10426) Add shuffle Streaming Expression

[Joel Bernstein (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-10426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joel Bernstein updated SOLR-10426:
--
Fix Version/s: 6.6

> Add shuffle Streaming Expression
> 
>
> Key: SOLR-10426
> URL: https://issues.apache.org/jira/browse/SOLR-10426
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Joel Bernstein
>Assignee: Joel Bernstein
> Fix For: 6.6
>
> Attachments: SOLR-10426.patch
>
>
> Currently the concept of MapReduce shuffling is lumped into the *search* 
> Streaming Expression. This causes quite a bit of confusion as users perform a 
> search using the /select handler by default, but actually wanted shuffling 
> behavior which requires the /export handler.
> We can solve this problem by creating a separate function called *shuffle* 
> that always uses the /export handler.
> This will also allow us to clean up some behaviors in the search expression 
> that are somewhat unexpected in future tickets.
> {code}
> parallel(workers, 
>  unique(shuffle(collection1, 
> q=*:*, 
> fl="id,a_s,a_i,a_f", 
> sort="a_f asc, a_i asc", 
> partitionKeys="a_f"), 
> over="a_f"), 
>  workers="2",
>  sort="a_f asc")
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-10426) Add shuffle Streaming Expression

[Joel Bernstein (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-10426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joel Bernstein updated SOLR-10426:
--
Description: 
Currently the concept of MapReduce shuffling is lumped into the *search* 
Streaming Expression. This causes quite a bit of confusion as users perform a 
search using the /select handler by default, but actually wanted shuffling 
behavior which requires the /export handler.

We can solve this problem by creating a separate function called *shuffle* that 
always uses the /export handler.

This will also allow us to clean up some behaviors in the search expression 
that are somewhat unexpected in future tickets.
{code}
parallel(workers, 
 unique(shuffle(collection1, 
q=*:*, 
fl="id,a_s,a_i,a_f", 
sort="a_f asc, a_i asc", 
partitionKeys="a_f"), 
over="a_f"), 
 workers="2",
 sort="a_f asc")
{code}

  was:
Currently the concept of MapReduce shuffling is lumped into the *search* 
Streaming Expression. This causes quite a bit of confusion as users perform a 
search using the /select handler by default, but actually wanted shuffling 
behavior which requires the /export handler.

We can solve this problem by creating a separate function called *shuffle* that 
always uses the /export handler.

This will also allow us to clean up some behaviors in the search expression 
that are somewhat unexpected in future tickets.
{code}
parallel(workers, 
 unique(shuffle(collection1, 
q=*:*, 
fl="id,a_s,a_i,a_f", 
sort="a_f asc, a_i asc", 
partitionKeys="a_f"), 
over="a_f"), 
  workers="2",
  sort="a_f asc")
{code}


> Add shuffle Streaming Expression
> 
>
> Key: SOLR-10426
> URL: https://issues.apache.org/jira/browse/SOLR-10426
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Joel Bernstein
> Attachments: SOLR-10426.patch
>
>
> Currently the concept of MapReduce shuffling is lumped into the *search* 
> Streaming Expression. This causes quite a bit of confusion as users perform a 
> search using the /select handler by default, but actually wanted shuffling 
> behavior which requires the /export handler.
> We can solve this problem by creating a separate function called *shuffle* 
> that always uses the /export handler.
> This will also allow us to clean up some behaviors in the search expression 
> that are somewhat unexpected in future tickets.
> {code}
> parallel(workers, 
>  unique(shuffle(collection1, 
> q=*:*, 
> fl="id,a_s,a_i,a_f", 
> sort="a_f asc, a_i asc", 
> partitionKeys="a_f"), 
> over="a_f"), 
>  workers="2",
>  sort="a_f asc")
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-10426) Add shuffle Streaming Expression

[Joel Bernstein (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-10426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joel Bernstein updated SOLR-10426:
--
Description: 
Currently the concept of MapReduce shuffling is lumped into the *search* 
Streaming Expression. This causes quite a bit of confusion as users perform a 
search using the /select handler by default, but actually wanted shuffling 
behavior which requires the /export handler.

We can solve this problem by creating a separate function called *shuffle* that 
always uses the /export handler.

This will also allow us to clean up some behaviors in the search expression 
that are somewhat unexpected in future tickets.
{code}
parallel(workers, 
 unique(shuffle(collection1, 
q=*:*, 
fl="id,a_s,a_i,a_f", 
sort="a_f asc, a_i asc", 
partitionKeys="a_f"), 
   over="a_f"), 
   workers="2",sort="a_f asc")
{code}

  was:
Currently the concept of MapReduce shuffling is lumped into the *search* 
Streaming Expression. This causes quite a bit of confusion as users perform a 
search using the /select handler by default, but actually wanted shuffling 
behavior which requires the /export handler.

We can solve this problem by creating a separate function called *shuffle* that 
always uses the /export handler.

This will also allow us to clean up some behaviors in the search expression 
that are somewhat unexpected in future tickets.
{code}
parallel(workers, 
 unique(shuffle(collection1, 
q=*:*, 
fl="id,a_s,a_i,a_f", 
sort="a_f asc, a_i asc", 
partitionKeys="a_f"), 
 over="a_f"), 
   workers="2",sort="a_f asc")
{code}


> Add shuffle Streaming Expression
> 
>
> Key: SOLR-10426
> URL: https://issues.apache.org/jira/browse/SOLR-10426
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Joel Bernstein
> Attachments: SOLR-10426.patch
>
>
> Currently the concept of MapReduce shuffling is lumped into the *search* 
> Streaming Expression. This causes quite a bit of confusion as users perform a 
> search using the /select handler by default, but actually wanted shuffling 
> behavior which requires the /export handler.
> We can solve this problem by creating a separate function called *shuffle* 
> that always uses the /export handler.
> This will also allow us to clean up some behaviors in the search expression 
> that are somewhat unexpected in future tickets.
> {code}
> parallel(workers, 
>  unique(shuffle(collection1, 
> q=*:*, 
> fl="id,a_s,a_i,a_f", 
> sort="a_f asc, a_i asc", 
> partitionKeys="a_f"), 
>over="a_f"), 
>workers="2",sort="a_f asc")
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-10426) Add shuffle Streaming Expression

[Joel Bernstein (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-10426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joel Bernstein updated SOLR-10426:
--
Description: 
Currently the concept of MapReduce shuffling is lumped into the *search* 
Streaming Expression. This causes quite a bit of confusion as users perform a 
search using the /select handler by default, but actually wanted shuffling 
behavior which requires the /export handler.

We can solve this problem by creating a separate function called *shuffle* that 
always uses the /export handler.

This will also allow us to clean up some behaviors in the search expression 
that are somewhat unexpected in future tickets.
{code}
parallel(workers, 
 unique(shuffle(collection1, 
q=*:*, 
fl="id,a_s,a_i,a_f", 
sort="a_f asc, a_i asc", 
partitionKeys="a_f"), 
over="a_f"), 
  workers="2",
  sort="a_f asc")
{code}

  was:
Currently the concept of MapReduce shuffling is lumped into the *search* 
Streaming Expression. This causes quite a bit of confusion as users perform a 
search using the /select handler by default, but actually wanted shuffling 
behavior which requires the /export handler.

We can solve this problem by creating a separate function called *shuffle* that 
always uses the /export handler.

This will also allow us to clean up some behaviors in the search expression 
that are somewhat unexpected in future tickets.
{code}
parallel(workers, 
 unique(shuffle(collection1, 
q=*:*, 
fl="id,a_s,a_i,a_f", 
sort="a_f asc, a_i asc", 
partitionKeys="a_f"), 
   over="a_f"), 
  workers="2",
  sort="a_f asc")
{code}


> Add shuffle Streaming Expression
> 
>
> Key: SOLR-10426
> URL: https://issues.apache.org/jira/browse/SOLR-10426
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Joel Bernstein
> Attachments: SOLR-10426.patch
>
>
> Currently the concept of MapReduce shuffling is lumped into the *search* 
> Streaming Expression. This causes quite a bit of confusion as users perform a 
> search using the /select handler by default, but actually wanted shuffling 
> behavior which requires the /export handler.
> We can solve this problem by creating a separate function called *shuffle* 
> that always uses the /export handler.
> This will also allow us to clean up some behaviors in the search expression 
> that are somewhat unexpected in future tickets.
> {code}
> parallel(workers, 
>  unique(shuffle(collection1, 
> q=*:*, 
> fl="id,a_s,a_i,a_f", 
> sort="a_f asc, a_i asc", 
> partitionKeys="a_f"), 
> over="a_f"), 
>   workers="2",
>   sort="a_f asc")
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-10426) Add shuffle Streaming Expression

[Joel Bernstein (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-10426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joel Bernstein updated SOLR-10426:
--
Description: 
Currently the concept of MapReduce shuffling is lumped into the *search* 
Streaming Expression. This causes quite a bit of confusion as users perform a 
search using the /select handler by default, but actually wanted shuffling 
behavior which requires the /export handler.

We can solve this problem by creating a separate function called *shuffle* that 
always uses the /export handler.

This will also allow us to clean up some behaviors in the search expression 
that are somewhat unexpected in future tickets.
{code}
parallel(workers, 
 unique(shuffle(collection1, 
q=*:*, 
fl="id,a_s,a_i,a_f", 
sort="a_f asc, a_i asc", 
partitionKeys="a_f"), 
   over="a_f"), 
  workers="2",
  sort="a_f asc")
{code}

  was:
Currently the concept of MapReduce shuffling is lumped into the *search* 
Streaming Expression. This causes quite a bit of confusion as users perform a 
search using the /select handler by default, but actually wanted shuffling 
behavior which requires the /export handler.

We can solve this problem by creating a separate function called *shuffle* that 
always uses the /export handler.

This will also allow us to clean up some behaviors in the search expression 
that are somewhat unexpected in future tickets.
{code}
parallel(workers, 
 unique(shuffle(collection1, 
q=*:*, 
fl="id,a_s,a_i,a_f", 
sort="a_f asc, a_i asc", 
partitionKeys="a_f"), 
   over="a_f"), 
   workers="2",sort="a_f asc")
{code}


> Add shuffle Streaming Expression
> 
>
> Key: SOLR-10426
> URL: https://issues.apache.org/jira/browse/SOLR-10426
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Joel Bernstein
> Attachments: SOLR-10426.patch
>
>
> Currently the concept of MapReduce shuffling is lumped into the *search* 
> Streaming Expression. This causes quite a bit of confusion as users perform a 
> search using the /select handler by default, but actually wanted shuffling 
> behavior which requires the /export handler.
> We can solve this problem by creating a separate function called *shuffle* 
> that always uses the /export handler.
> This will also allow us to clean up some behaviors in the search expression 
> that are somewhat unexpected in future tickets.
> {code}
> parallel(workers, 
>  unique(shuffle(collection1, 
> q=*:*, 
> fl="id,a_s,a_i,a_f", 
> sort="a_f asc, a_i asc", 
> partitionKeys="a_f"), 
>over="a_f"), 
>   workers="2",
>   sort="a_f asc")
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-10426) Add shuffle Streaming Expression

[Joel Bernstein (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-10426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joel Bernstein updated SOLR-10426:
--
Description: 
Currently the concept of MapReduce shuffling is lumped into the *search* 
Streaming Expression. This causes quite a bit of confusion as users perform a 
search using the /select handler by default, but actually wanted shuffling 
behavior which requires the /export handler.

We can solve this problem by creating a separate function called *shuffle* that 
always uses the /export handler.

This will also allow us to clean up some behaviors in the search expression 
that are somewhat unexpected in future tickets.
{code}
parallel(workers, 
 unique(shuffle(collection1, 
q=*:*, 
fl="id,a_s,a_i,a_f", 
sort="a_f asc, a_i asc", 
partitionKeys="a_f"), 
  over="a_f"), 
workers="2",sort="a_f asc")
{code}

  was:
Currently the concept of MapReduce shuffling is lumped into the *search* 
Streaming Expression. This causes quite a bit of confusion as users perform a 
search using the /select handler by default, but actually wanted shuffling 
behavior which requires the /export handler.

We can solve this problem by creating a separate function called *shuffle* that 
always uses the /export handler.

This will also allow us to clean up some behaviors in the search expression 
that are somewhat unexpected in future tickets.


> Add shuffle Streaming Expression
> 
>
> Key: SOLR-10426
> URL: https://issues.apache.org/jira/browse/SOLR-10426
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Joel Bernstein
> Attachments: SOLR-10426.patch
>
>
> Currently the concept of MapReduce shuffling is lumped into the *search* 
> Streaming Expression. This causes quite a bit of confusion as users perform a 
> search using the /select handler by default, but actually wanted shuffling 
> behavior which requires the /export handler.
> We can solve this problem by creating a separate function called *shuffle* 
> that always uses the /export handler.
> This will also allow us to clean up some behaviors in the search expression 
> that are somewhat unexpected in future tickets.
> {code}
> parallel(workers, 
>  unique(shuffle(collection1, 
> q=*:*, 
> fl="id,a_s,a_i,a_f", 
> sort="a_f asc, a_i asc", 
> partitionKeys="a_f"), 
>   over="a_f"), 
> workers="2",sort="a_f asc")
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Comment Edited] (SOLR-10423) ShingleFilter causes overly restrictive queries to be produced

[Steve Rowe (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10423?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15957280#comment-15957280
 ] 

Steve Rowe edited comment on SOLR-10423 at 4/5/17 6:50 PM:
---

Patch with suggested fix and tests: specifying {{...}} allows functional queries over ShingleFilter'd 
fields.

Running tests and precommit now.  I'd like to include this in Solr 6.5.1.


was (Author: steve_rowe):
Patch with suggested fix and tests: specifying {{}} allows functional queries over ShingleFilter'd 
fields.

Running tests and precommit now.  I'd like to include this in Solr 6.5.1.

> ShingleFilter causes overly restrictive queries to be produced
> --
>
> Key: SOLR-10423
> URL: https://issues.apache.org/jira/browse/SOLR-10423
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: query parsers
>Affects Versions: 6.5
>Reporter: Steve Rowe
> Attachments: SOLR-10423.patch
>
>
> When {{sow=false}} and {{ShingleFilter}} is included in the query analyzer, 
> {{QueryBuilder}} produces queries that inappropriately require sequential 
> terms.  E.g. the query "A B C" produces {{(+A_B +B_C) A_B_C}} when the query 
> analyzer includes {{ maxShingleSize="3" outputUnigrams="false" tokenSeparator="_"/>}}.
> Aman Deep Singh reported this problem on the solr-user list. From 
> [http://mail-archives.apache.org/mod_mbox/lucene-solr-user/201703.mbox/%3ccanegtx9bwbpwqc-cxieac7qsas7x2tgzovomy5ztiagco1p...@mail.gmail.com%3e]:
> {quote}
> I was trying to use the shingle filter but it was not creating the query as
> desirable.
> my schema is
> {noformat}
>  positionIncrementGap="100">
>   
> 
>  maxShingleSize="4"/>
> 
>   
> 
> 
> {noformat}
> my solr query is
> {noformat}
> http://localhost:8983/solr/productCollection/select?
>  defType=edismax
> =true
> =one%20plus%20one%20four
> =nameShingle
> =false
> =xml
> {noformat}
> and it was creating the parsed query as
> {noformat}
> 
> (+(DisjunctionMaxQuery(((+nameShingle:one plus +nameShingle:plus one
> +nameShingle:one four))) DisjunctionMaxQuery(((+nameShingle:one plus
> +nameShingle:plus one four))) DisjunctionMaxQuery(((+nameShingle:one plus one 
> +nameShingle:one four))) DisjunctionMaxQuery((nameShingle:one plus one 
> four)))~1)/no_coord
> 
> 
> *++nameShingle:one plus +nameShingle:plus one +nameShingle:one four))
> ((+nameShingle:one plus +nameShingle:plus one four)) ((+nameShingle:one
> plus one +nameShingle:one four)) (nameShingle:one plus one four))~1)*
> 
> {noformat}
> So ideally token creations is perfect but in the query it is using boolean + 
> operator which is causing the problem as if i have a document with name as 
> "one plus one" ,according to the shingles it has to matched as its token will 
> be  ("one plus","one plus one","plus one") .
> I have tried using the q.op and played around the mm also but nothing is
> giving me the correct response.
> Any idea how i can fetch that document even if the document is missing any
> token.
> My expected response will be getting the document "one plus one" even the 
> user query has any additional term like "one plus one two" and so on.
> {quote}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-10426) Add shuffle Streaming Expression

[Joel Bernstein (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-10426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joel Bernstein updated SOLR-10426:
--
Description: 
Currently the concept of MapReduce shuffling is lumped into the *search* 
Streaming Expression. This causes quite a bit of confusion as users perform a 
search using the /select handler by default, but actually wanted shuffling 
behavior which requires the /export handler.

We can solve this problem by creating a separate function called *shuffle* that 
always uses the /export handler.

This will also allow us to clean up some behaviors in the search expression 
that are somewhat unexpected in future tickets.
{code}
parallel(workers, 
 unique(shuffle(collection1, 
q=*:*, 
fl="id,a_s,a_i,a_f", 
sort="a_f asc, a_i asc", 
partitionKeys="a_f"), 
 over="a_f"), 
   workers="2",sort="a_f asc")
{code}

  was:
Currently the concept of MapReduce shuffling is lumped into the *search* 
Streaming Expression. This causes quite a bit of confusion as users perform a 
search using the /select handler by default, but actually wanted shuffling 
behavior which requires the /export handler.

We can solve this problem by creating a separate function called *shuffle* that 
always uses the /export handler.

This will also allow us to clean up some behaviors in the search expression 
that are somewhat unexpected in future tickets.
{code}
parallel(workers, 
 unique(shuffle(collection1, 
q=*:*, 
fl="id,a_s,a_i,a_f", 
sort="a_f asc, a_i asc", 
partitionKeys="a_f"), 
  over="a_f"), 
workers="2",sort="a_f asc")
{code}


> Add shuffle Streaming Expression
> 
>
> Key: SOLR-10426
> URL: https://issues.apache.org/jira/browse/SOLR-10426
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Joel Bernstein
> Attachments: SOLR-10426.patch
>
>
> Currently the concept of MapReduce shuffling is lumped into the *search* 
> Streaming Expression. This causes quite a bit of confusion as users perform a 
> search using the /select handler by default, but actually wanted shuffling 
> behavior which requires the /export handler.
> We can solve this problem by creating a separate function called *shuffle* 
> that always uses the /export handler.
> This will also allow us to clean up some behaviors in the search expression 
> that are somewhat unexpected in future tickets.
> {code}
> parallel(workers, 
>  unique(shuffle(collection1, 
> q=*:*, 
> fl="id,a_s,a_i,a_f", 
> sort="a_f asc, a_i asc", 
> partitionKeys="a_f"), 
>  over="a_f"), 
>workers="2",sort="a_f asc")
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10423) ShingleFilter causes overly restrictive queries to be produced

[Steve Rowe (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10423?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15957426#comment-15957426
 ] 

Steve Rowe commented on SOLR-10423:
---

All Solr tests and precommit pass.

> ShingleFilter causes overly restrictive queries to be produced
> --
>
> Key: SOLR-10423
> URL: https://issues.apache.org/jira/browse/SOLR-10423
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: query parsers
>Affects Versions: 6.5
>Reporter: Steve Rowe
> Attachments: SOLR-10423.patch
>
>
> When {{sow=false}} and {{ShingleFilter}} is included in the query analyzer, 
> {{QueryBuilder}} produces queries that inappropriately require sequential 
> terms.  E.g. the query "A B C" produces {{(+A_B +B_C) A_B_C}} when the query 
> analyzer includes {{ maxShingleSize="3" outputUnigrams="false" tokenSeparator="_"/>}}.
> Aman Deep Singh reported this problem on the solr-user list. From 
> [http://mail-archives.apache.org/mod_mbox/lucene-solr-user/201703.mbox/%3ccanegtx9bwbpwqc-cxieac7qsas7x2tgzovomy5ztiagco1p...@mail.gmail.com%3e]:
> {quote}
> I was trying to use the shingle filter but it was not creating the query as
> desirable.
> my schema is
> {noformat}
>  positionIncrementGap="100">
>   
> 
>  maxShingleSize="4"/>
> 
>   
> 
> 
> {noformat}
> my solr query is
> {noformat}
> http://localhost:8983/solr/productCollection/select?
>  defType=edismax
> =true
> =one%20plus%20one%20four
> =nameShingle
> =false
> =xml
> {noformat}
> and it was creating the parsed query as
> {noformat}
> 
> (+(DisjunctionMaxQuery(((+nameShingle:one plus +nameShingle:plus one
> +nameShingle:one four))) DisjunctionMaxQuery(((+nameShingle:one plus
> +nameShingle:plus one four))) DisjunctionMaxQuery(((+nameShingle:one plus one 
> +nameShingle:one four))) DisjunctionMaxQuery((nameShingle:one plus one 
> four)))~1)/no_coord
> 
> 
> *++nameShingle:one plus +nameShingle:plus one +nameShingle:one four))
> ((+nameShingle:one plus +nameShingle:plus one four)) ((+nameShingle:one
> plus one +nameShingle:one four)) (nameShingle:one plus one four))~1)*
> 
> {noformat}
> So ideally token creations is perfect but in the query it is using boolean + 
> operator which is causing the problem as if i have a document with name as 
> "one plus one" ,according to the shingles it has to matched as its token will 
> be  ("one plus","one plus one","plus one") .
> I have tried using the q.op and played around the mm also but nothing is
> giving me the correct response.
> Any idea how i can fetch that document even if the document is missing any
> token.
> My expected response will be getting the document "one plus one" even the 
> user query has any additional term like "one plus one two" and so on.
> {quote}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: 6.5.1 release?

[Steve Rowe]

I’d like to include SOLR-10423.  It’s a bug fix in that Solr ShingleFilter 
queries currently don’t work when sow=false; the patch allows for schema 
configuration that makes them function by disabling QueryBuilder’s graph query 
construction.

--
Steve
www.lucidworks.com

> On Apr 4, 2017, at 2:29 PM, Christine Poerschke (BLOOMBERG/ LONDON) 
>  wrote:
> 
> I'd also like to include fix for 
> https://issues.apache.org/jira/browse/SOLR-10421 which should be just four 
> lines of actual fix, plus ideally a test for it which will be more than four 
> lines ...
> 
> - Original Message -
> From: dev@lucene.apache.org
> To: dev@lucene.apache.org
> At: 04/04/17 16:54:50
> 
> I'd also like to include SOLR-10277 --- it is a serious problem for
> people running large number of collections. I'll review and commit the
> patch by tomorrow.
> 
> On Tue, Apr 4, 2017 at 2:16 PM, Shalin Shekhar Mangar
>  wrote:
>> I would like to include https://issues.apache.org/jira/browse/SOLR-10416
>> 
>> It is a trivial fix.
>> 
>> On Mon, Apr 3, 2017 at 11:54 PM, Joel Bernstein  wrote:
>>> SOLR-10404 looks like a nice improvement!
>>> 
>>> I'll shoot for Friday morning to create the release candidate. I've never
>>> been a release manager before so I may need some guidance along the way.
>>> 
>>> 
>>> Joel Bernstein
>>> http://joelsolr.blogspot.com/
>>> 
>>> On Mon, Apr 3, 2017 at 12:21 PM, David Smiley 
>>> wrote:
 
 Found & fixed a bug: https://issues.apache.org/jira/browse/SOLR-10404  I'd
 like to get this into 6.5.1.  You might be interested in this one Joel.
 
 On Mon, Apr 3, 2017 at 11:58 AM Steve Rowe  wrote:
> 
> 
>> On Apr 3, 2017, at 11:52 AM, Adrien Grand  wrote:
>> 
>> Building the first RC on April 6th sounds good to me! I'm wondering
>> whether the 6.5 Jenkins jobs are still running?
> 
> I disabled the ASF Jenkins 6.5 jobs shortly after the release.  FYI you
> can see which Lucene/Solr jobs are enabled here:
> .  I’ll re-enable the 6.5 jobs
> now.
> 
> --
> Steve
> www.lucidworks.com
> 
> 
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org
> 
 --
 Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker
 LinkedIn: http://linkedin.com/in/davidwsmiley | Book:
 http://www.solrenterprisesearchserver.com
>>> 
>>> 
>> 
>> 
>> 
>> --
>> Regards,
>> Shalin Shekhar Mangar.
> 
> 
> 
> -- 
> Regards,
> Shalin Shekhar Mangar.
> 
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org
> 
> 
> 
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org
> 


-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-NightlyTests-6.5 - Build # 7 - Unstable

[Apache Jenkins Server]

Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-6.5/7/

2 tests failed.
FAILED:  org.apache.solr.cloud.FullSolrCloudDistribCmdsTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=20725, name=Thread-8692, 
state=RUNNABLE, group=TGRP-FullSolrCloudDistribCmdsTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught 
exception in thread: Thread[id=20725, name=Thread-8692, state=RUNNABLE, 
group=TGRP-FullSolrCloudDistribCmdsTest]
Caused by: java.lang.RuntimeException: 
org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting 
response from server at: http://127.0.0.1:52658/_dy/bb/collection1
at __randomizedtesting.SeedInfo.seed([4BBDDB607670A81A]:0)
at 
org.apache.solr.cloud.FullSolrCloudDistribCmdsTest$1IndexThread.run(FullSolrCloudDistribCmdsTest.java:644)
Caused by: org.apache.solr.client.solrj.SolrServerException: Timeout occured 
while waiting response from server at: http://127.0.0.1:52658/_dy/bb/collection1
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:621)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:279)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:268)
at 
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:160)
at 
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:177)
at 
org.apache.solr.cloud.FullSolrCloudDistribCmdsTest$1IndexThread.run(FullSolrCloudDistribCmdsTest.java:642)
Caused by: java.net.SocketTimeoutException: Read timed out
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
at java.net.SocketInputStream.read(SocketInputStream.java:171)
at java.net.SocketInputStream.read(SocketInputStream.java:141)
at 
org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:160)
at 
org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:84)
at 
org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:273)
at 
org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:140)
at 
org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
at 
org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261)
at 
org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:283)
at 
org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:251)
at 
org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:197)
at 
org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272)
at 
org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124)
at 
org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:685)
at 
org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:487)
at 
org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
at 
org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
at 
org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:515)
... 5 more


FAILED:  
org.apache.solr.store.blockcache.BlockDirectoryTest.ensureCacheConfigurable

Error Message:
Direct buffer memory

Stack Trace:
java.lang.OutOfMemoryError: Direct buffer memory
at java.nio.Bits.reserveMemory(Bits.java:693)
at java.nio.DirectByteBuffer.(DirectByteBuffer.java:123)
at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:311)
at 
org.apache.solr.store.blockcache.BlockCache.(BlockCache.java:75)
at 
org.apache.solr.store.blockcache.BlockDirectoryTest.setUp(BlockDirectoryTest.java:119)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:941)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at 

[jira] [Updated] (SOLR-10426) Add shuffle Streaming Expression

[Joel Bernstein (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-10426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joel Bernstein updated SOLR-10426:
--
Description: 
Currently the concept of MapReduce shuffling is lumped into the *search* 
Streaming Expression. This causes quite a bit of confusion as users perform a 
search using the /select handler by default, but actually wanted shuffling 
behavior which requires the /export handler.

We can solve this problem by creating a separate function called *shuffle* that 
always uses the /export handler.

This will also allow us to clean up some behaviors in the search expression 
that are somewhat unexpected in future tickets.

  was:
Currently the concept of MapReduce shuffling is lumped into the *search* 
Streaming Expression. This causes quite a bit of confusion as users perform a 
search using the /select handler by default, but actually wanted shuffling 
behavior which requires the /export handler.

We can solve this problem by creating a separate function called *shuffle* that 
always uses the /export handler.

This will also allow us to clean up some behaviors in the search expression 
that are somewhat unexpected.


> Add shuffle Streaming Expression
> 
>
> Key: SOLR-10426
> URL: https://issues.apache.org/jira/browse/SOLR-10426
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Joel Bernstein
> Attachments: SOLR-10426.patch
>
>
> Currently the concept of MapReduce shuffling is lumped into the *search* 
> Streaming Expression. This causes quite a bit of confusion as users perform a 
> search using the /select handler by default, but actually wanted shuffling 
> behavior which requires the /export handler.
> We can solve this problem by creating a separate function called *shuffle* 
> that always uses the /export handler.
> This will also allow us to clean up some behaviors in the search expression 
> that are somewhat unexpected in future tickets.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-10426) Add shuffle Streaming Expression

[Joel Bernstein (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-10426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joel Bernstein updated SOLR-10426:
--
Attachment: SOLR-10426.patch

Very simple patch for this.

> Add shuffle Streaming Expression
> 
>
> Key: SOLR-10426
> URL: https://issues.apache.org/jira/browse/SOLR-10426
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Joel Bernstein
> Attachments: SOLR-10426.patch
>
>
> Currently the concept of MapReduce shuffling is lumped into the *search* 
> Streaming Expression. This causes quite a bit of confusion as users perform a 
> search using the /select handler by default, but actually wanted shuffling 
> behavior which requires the /export handler.
> We can solve this problem by creating a separate function called *shuffle* 
> that always uses the /export handler.
> This will also allow us to clean up some behaviors in the search expression 
> that are somewhat unexpected.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7671) Enhance UpgradeIndexMergePolicy with additional options

[Keith Laban (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15957388#comment-15957388
 ] 

Keith Laban commented on LUCENE-7671:
-

The test now passes using the CodeReader interface. I also renamed the class to 
{{LiveUpgradeSegmentsMergePolicy}} because it upgrades on a segment by segment 
basis verse the {{UpgradeIndexMergePolicy}} which tries to ugprade all 
segments. 

It looks like someone has been touching  TestBackwardCompatibility class and 
this no longer cleanly lands on master. I'll take a look at catching it up later

> Enhance UpgradeIndexMergePolicy with additional options
> ---
>
> Key: LUCENE-7671
> URL: https://issues.apache.org/jira/browse/LUCENE-7671
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Keith Laban
>
> Enhance UpgradeIndexMergePolicy to be a MergePolicy that can be used outside 
> the scope the IndexUpgrader.
> The enhancement aims to allow the UpgradeIndexMergePolicy to:
> 1) Delegate normal force merges to the underlying merge policy
> 2) Enable a flag that will explicitly tell UpgradeIndexMergePolicy when it 
> should start looking for upgrades.
> 3) Allow new segments to be considered to be merged with old segments, 
> depending on underlying MergePolicy.
> 4) Be configurable for backwards compatibility such that only segments 
> needing an upgrade would be considered when merging, no explicit upgrades.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-7719) UnifiedHighlighter doesn't handle some AutomatonQuery's with multi-byte chars

[David Smiley (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-7719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

David Smiley updated LUCENE-7719:
-
Priority: Minor  (was: Major)

> UnifiedHighlighter doesn't handle some AutomatonQuery's with multi-byte chars
> -
>
> Key: LUCENE-7719
> URL: https://issues.apache.org/jira/browse/LUCENE-7719
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: modules/highlighter
>Reporter: David Smiley
>Assignee: David Smiley
>Priority: Minor
> Attachments: LUCENE_7719.patch
>
>
> In MultiTermHighlighting, a CharacterRunAutomaton is being created that takes 
> the result of AutomatonQuery.getAutomaton that in turn is byte oriented, not 
> character oriented.  For ASCII terms, this is safe but it's not for 
> multi-byte characters.  This is most likely going to rear it's head with a 
> WildcardQuery, but due to special casing in MultiTermHighlighting, 
> PrefixQuery isn't affected.  Nonetheless it'd be nice to get a general fix in 
> so that MultiTermHighlighting can remove special cases for PrefixQuery and 
> TermRangeQuery (both subclass AutomatonQuery).
> AFAICT, this bug was likely in the PostingsHighlighter since inception.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-7719) UnifiedHighlighter doesn't handle some AutomatonQuery's with multi-byte chars

[David Smiley (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-7719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

David Smiley updated LUCENE-7719:
-
Attachment: LUCENE_7719.patch

Here is a patch for consideration.

First of all, I realized that this bug is very minor because only _some_ 
{{AutomatonQuery.getAutomaton}} are binary (I thought they all were); others 
are char based.  The ones that are binary in Lucene are {{PrefixQuery}} and 
{{TermRangeQuery}}, both of which are special cased in 
{{MultiTermHighlighting}}.  So practically speaking I think the only users to 
see this bug would be anyone building a custom AutomatonQuery.  Nonetheless 
this should be cleaned up and actually tested.

The patch:
* Adds a fairly thorough test with lots of randomized unicode, testing 
PrefixQuery, TermRangeQuery, WildcardQuery, and FuzzyQuery
* removes the special casing of AutomatonQuery subclasses in 
MultiTermHighlighting.  AQ is handled generically now.
* added {{AutomatonQuery.isAutomatonBinary()}} with a new field to match.
* if the AQ.getAutomaton is _not_ binary, we follow a simple/obvious path
* if the automaton is binary, then I produce a CharRunAutomaton implementing 
run() to navigate the Automaton char by char.  It makes a BytesRunAutomaton on 
the automaton.  I inlined the one and two byte UTF char logic from 
{{UnicodeUtil.UTF16toUTF8}}.  If the char needs more bytes, then I call out to 
UTF16toUTF8 and work off the generated byte array for the remaining chars.  I 
think this is more maintainable, albeit slower, than reproducing the logic.

There is a TODO on making the BytesRunAutomaton on the binary Automaton.  This 
is redundant since the AutomatonQuery already has a compiledAutomaton which in 
turn already has a BytesRunAutomaton.  The construction cost seems less than 
trivial as it determinizes the automaton and does other work.  Is this a big 
deal?  It seems kinda sorta; it depends.  

> UnifiedHighlighter doesn't handle some AutomatonQuery's with multi-byte chars
> -
>
> Key: LUCENE-7719
> URL: https://issues.apache.org/jira/browse/LUCENE-7719
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: modules/highlighter
>Reporter: David Smiley
>Assignee: David Smiley
> Attachments: LUCENE_7719.patch
>
>
> In MultiTermHighlighting, a CharacterRunAutomaton is being created that takes 
> the result of AutomatonQuery.getAutomaton that in turn is byte oriented, not 
> character oriented.  For ASCII terms, this is safe but it's not for 
> multi-byte characters.  This is most likely going to rear it's head with a 
> WildcardQuery, but due to special casing in MultiTermHighlighting, 
> PrefixQuery isn't affected.  Nonetheless it'd be nice to get a general fix in 
> so that MultiTermHighlighting can remove special cases for PrefixQuery and 
> TermRangeQuery (both subclass AutomatonQuery).
> AFAICT, this bug was likely in the PostingsHighlighter since inception.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-6.x-MacOSX (64bit/jdk1.8.0) - Build # 808 - Unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-6.x-MacOSX/808/
Java: 64bit/jdk1.8.0 -XX:-UseCompressedOops -XX:+UseSerialGC

2 tests failed.
FAILED:  org.apache.solr.cloud.CustomCollectionTest.testCustomCollectionsAPI

Error Message:
Error from server at http://127.0.0.1:61738/solr: Failed to create shard

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error 
from server at http://127.0.0.1:61738/solr: Failed to create shard
at 
__randomizedtesting.SeedInfo.seed([48E471C079E67D5D:2205FFAB447CCB25]:0)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:610)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:279)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:268)
at 
org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:435)
at 
org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:387)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1364)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:1115)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:1054)
at 
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:160)
at 
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:177)
at 
org.apache.solr.cloud.CustomCollectionTest.testCustomCollectionsAPI(CustomCollectionTest.java:103)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 

[jira] [Commented] (LUCENE-7728) TestGrouping.testRandom() failures

[Adrien Grand (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15957297#comment-15957297
 ] 

Adrien Grand commented on LUCENE-7728:
--

Another reprodcing master failure: 
https://elasticsearch-ci.elastic.co/job/apache+lucene-solr+master/9262/console

{noformat}
   [junit4] Suite: org.apache.lucene.search.grouping.TestGrouping
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestGrouping 
-Dtests.method=testRandom -Dtests.seed=42883A34582024BB -Dtests.slow=true 
-Dtests.locale=sr-Latn-BA -Dtests.timezone=America/Miquelon 
-Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
   [junit4] FAILURE 6.98s J0 | TestGrouping.testRandom <<<
   [junit4]> Throwable #1: java.lang.AssertionError: expected:<477> but 
was:<468>
   [junit4]>at 
__randomizedtesting.SeedInfo.seed([42883A34582024BB:30C41F3BE94092C8]:0)
   [junit4]>at 
org.apache.lucene.search.grouping.TestGrouping.assertEquals(TestGrouping.java:1299)
   [junit4]>at 
org.apache.lucene.search.grouping.TestGrouping.testRandom(TestGrouping.java:1141)
   [junit4]>at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene70): 
{groupend=FST50, sort1=BlockTreeOrds(blocksize=128), sort2=FST50, 
content=PostingsFormat(name=Direct), group=BlockTreeOrds(blocksize=128)}, 
docValues:{author=DocValuesFormat(name=Lucene70), 
sort1=DocValuesFormat(name=Lucene70), id=DocValuesFormat(name=Lucene70), 
sort2=DocValuesFormat(name=Direct), group=DocValuesFormat(name=Lucene70)}, 
maxPointsInLeafNode=1782, maxMBSortInHeap=5.88541198629704, 
sim=RandomSimilarity(queryNorm=true): {content=DFI(Standardized)}, 
locale=sr-Latn-BA, timezone=America/Miquelon
   [junit4]   2> NOTE: Linux 4.4.35-33.55.amzn1.x86_64 amd64/Oracle Corporation 
1.8.0_121 (64-bit)/cpus=4,threads=1,free=320921624,total=331350016
   [junit4]   2> NOTE: All tests run in this JVM: [TestGrouping]
{noformat}

> TestGrouping.testRandom() failures
> --
>
> Key: LUCENE-7728
> URL: https://issues.apache.org/jira/browse/LUCENE-7728
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Steve Rowe
>
> My Jenkins found a reproducing branch_6x seed:
> {noformat}
>[junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestGrouping 
> -Dtests.method=testRandom -Dtests.seed=6F0B519BBDC786B3 -Dtests.slow=true 
> -Dtests.locale=en-CA -Dtests.timezone=Europe/Zagreb -Dtests.asserts=true 
> -Dtests.file.encoding=US-ASCII
>[junit4] FAILURE 2.50s J0 | TestGrouping.testRandom <<<
>[junit4]> Throwable #1: java.lang.AssertionError: expected:<10> but 
> was:<29>
>[junit4]>  at 
> __randomizedtesting.SeedInfo.seed([6F0B519BBDC786B3:1D4774940CA730C0]:0)
>[junit4]>  at 
> org.apache.lucene.search.grouping.TestGrouping.assertEquals(TestGrouping.java:1288)
>[junit4]>  at 
> org.apache.lucene.search.grouping.TestGrouping.testRandom(TestGrouping.java:1130)
>[junit4]>  at java.lang.Thread.run(Thread.java:745)
>[junit4]   2> NOTE: test params are: codec=Asserting(Lucene62): 
> {groupend=Lucene50(blocksize=128), sort1=PostingsFormat(name=Memory 
> doPackFST= true), sort2=Lucene50(blocksize=128), 
> content=PostingsFormat(name=Memory doPackFST= false), 
> group=PostingsFormat(name=Memory doPackFST= true)}, 
> docValues:{groupend=DocValuesFormat(name=Direct), 
> author=DocValuesFormat(name=Memory), sort1=DocValuesFormat(name=Memory), 
> id=DocValuesFormat(name=Memory), sort2=DocValuesFormat(name=Direct), 
> content=DocValuesFormat(name=Lucene54), group=DocValuesFormat(name=Memory)}, 
> maxPointsInLeafNode=454, maxMBSortInHeap=6.048552291438714, 
> sim=RandomSimilarity(queryNorm=false,coord=no): {content=DFI(Saturated)}, 
> locale=en-CA, timezone=Europe/Zagreb
>[junit4]   2> NOTE: Linux 4.1.0-custom2-amd64 amd64/Oracle Corporation 
> 1.8.0_77 (64-bit)/cpus=16,threads=1,free=485264792,total=514850816
> {noformat}
> {{git bisect}} says the first bad commit is this one: 
> [http://git-wip-us.apache.org/repos/asf/lucene-solr/commit/6d952076], but 
> that just removed {{BooleanSimilarity}} from the {{RandomSimilarity}}'s 
> candidates list, which in this case likely has the side effect of picking a 
> different similarity with this seed.  So the problem is almost certainly 
> older than this.
> Policeman Jenkins reported a master seed four weeks ago 
> [https://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/18890/] for a 
> failure that's still reproducing for me on Linux w/ Java8:
> {noformat}
>   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestGrouping 
> -Dtests.method=testRandom -Dtests.seed=381A6A2BB3B21736 -Dtests.multiplier=3 
> -Dtests.slow=true -Dtests.locale=nyn -Dtests.timezone=Antarctica/McMurdo 
> -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
>   

[jira] [Updated] (SOLR-10007) Clean up references to CoreContainer and CoreDescriptors

[Erick Erickson (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-10007?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Erick Erickson updated SOLR-10007:
--
Attachment: SOLR-10007.patch

First time I've ever had precompile fail but still be able to 'ant clean server 
dist' successfully due to a _renamed_ method.

Patch now passes precommit and test.

> Clean up references to CoreContainer and CoreDescriptors
> 
>
> Key: SOLR-10007
> URL: https://issues.apache.org/jira/browse/SOLR-10007
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Affects Versions: trunk, 6.4
>Reporter: Erick Erickson
>Assignee: Erick Erickson
> Attachments: SOLR-10007.patch, SOLR-10007.patch, SOLR-10007.patch
>
>
> It's a bit weird that CoreDescriptor contains a reference to CoreContainer. 
> It seems like the SolrCore should reference CoreContainer.
> Similarly, SolrCore keeps a copy of CoreDescriptor and another copy is kept 
> in the various CoreDescirptor lists making it difficult to track which is the 
> "real" CoreDescriptor.
> This is an umbrella issue as I think this (and perhaps other) issues can be 
> tackled separately.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10277) On 'downnode', lots of wasteful mutations are done to ZK

[Scott Blum (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10277?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15957285#comment-15957285
 ] 

Scott Blum commented on SOLR-10277:
---

Thank you Shalin!

> On 'downnode', lots of wasteful mutations are done to ZK
> 
>
> Key: SOLR-10277
> URL: https://issues.apache.org/jira/browse/SOLR-10277
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Affects Versions: 5.5.3, 5.5.4, 6.0.1, 6.2.1, 6.3, 6.4.2
>Reporter: Joshua Humphries
>Assignee: Scott Blum
>  Labels: leader, zookeeper
> Fix For: master (7.0), 6.5.1
>
> Attachments: SOLR-10277-5.5.3.patch, SOLR-10277.patch, 
> SOLR-10277.patch
>
>
> When a node restarts, it submits a single 'downnode' message to the 
> overseer's state update queue.
> When the overseer processes the message, it does way more writes to ZK than 
> necessary. In our cluster of 48 hosts, the majority of collections have only 
> 1 shard and 1 replica. So a single node restarting should only result in 
> ~1/40th of the collections being updated with new replica states (to indicate 
> the node that is no longer active).
> However, the current logic in NodeMutator#downNode always updates *every* 
> collection. So we end up having to do rolling restarts very slowly to avoid 
> having a severe outage due to the overseer having to do way too much work for 
> each host that is restarted. And subsequent shards becoming leader can't get 
> processed until the `downnode` message is fully processed. So a fast rolling 
> restart can result in the overseer queue growing incredibly large and nearly 
> all shards winding up in a leader-less state until that backlog is processed.
> The fix is a trivial logic change to only add a ZkWriteCommand for 
> collections that actually have an impacted replica.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Comment Edited] (SOLR-10423) ShingleFilter causes overly restrictive queries to be produced

[Steve Rowe (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10423?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15957280#comment-15957280
 ] 

Steve Rowe edited comment on SOLR-10423 at 4/5/17 5:25 PM:
---

Patch with suggested fix and tests: specifying {{}} allows functional queries over ShingleFilter'd 
fields.

Running tests and precommit now.  I'd like to include this in Solr 6.5.1.


was (Author: steve_rowe):
Patch with suggested fix and tests: on Solr {{}} allows functional queries over ShingleFilter'd 
fields.

Running tests and precommit now.  I'd like to include this in Solr 6.5.1.

> ShingleFilter causes overly restrictive queries to be produced
> --
>
> Key: SOLR-10423
> URL: https://issues.apache.org/jira/browse/SOLR-10423
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: query parsers
>Affects Versions: 6.5
>Reporter: Steve Rowe
> Attachments: SOLR-10423.patch
>
>
> When {{sow=false}} and {{ShingleFilter}} is included in the query analyzer, 
> {{QueryBuilder}} produces queries that inappropriately require sequential 
> terms.  E.g. the query "A B C" produces {{(+A_B +B_C) A_B_C}} when the query 
> analyzer includes {{ maxShingleSize="3" outputUnigrams="false" tokenSeparator="_"/>}}.
> Aman Deep Singh reported this problem on the solr-user list. From 
> [http://mail-archives.apache.org/mod_mbox/lucene-solr-user/201703.mbox/%3ccanegtx9bwbpwqc-cxieac7qsas7x2tgzovomy5ztiagco1p...@mail.gmail.com%3e]:
> {quote}
> I was trying to use the shingle filter but it was not creating the query as
> desirable.
> my schema is
> {noformat}
>  positionIncrementGap="100">
>   
> 
>  maxShingleSize="4"/>
> 
>   
> 
> 
> {noformat}
> my solr query is
> {noformat}
> http://localhost:8983/solr/productCollection/select?
>  defType=edismax
> =true
> =one%20plus%20one%20four
> =nameShingle
> =false
> =xml
> {noformat}
> and it was creating the parsed query as
> {noformat}
> 
> (+(DisjunctionMaxQuery(((+nameShingle:one plus +nameShingle:plus one
> +nameShingle:one four))) DisjunctionMaxQuery(((+nameShingle:one plus
> +nameShingle:plus one four))) DisjunctionMaxQuery(((+nameShingle:one plus one 
> +nameShingle:one four))) DisjunctionMaxQuery((nameShingle:one plus one 
> four)))~1)/no_coord
> 
> 
> *++nameShingle:one plus +nameShingle:plus one +nameShingle:one four))
> ((+nameShingle:one plus +nameShingle:plus one four)) ((+nameShingle:one
> plus one +nameShingle:one four)) (nameShingle:one plus one four))~1)*
> 
> {noformat}
> So ideally token creations is perfect but in the query it is using boolean + 
> operator which is causing the problem as if i have a document with name as 
> "one plus one" ,according to the shingles it has to matched as its token will 
> be  ("one plus","one plus one","plus one") .
> I have tried using the q.op and played around the mm also but nothing is
> giving me the correct response.
> Any idea how i can fetch that document even if the document is missing any
> token.
> My expected response will be getting the document "one plus one" even the 
> user query has any additional term like "one plus one two" and so on.
> {quote}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-10423) ShingleFilter causes overly restrictive queries to be produced

[Steve Rowe (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-10423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Steve Rowe updated SOLR-10423:
--
Attachment: SOLR-10423.patch

Patch with suggested fix and tests: on Solr {{}} allows functional queries over ShingleFilter'd 
fields.

Running tests and precommit now.  I'd like to include this in Solr 6.5.1.

> ShingleFilter causes overly restrictive queries to be produced
> --
>
> Key: SOLR-10423
> URL: https://issues.apache.org/jira/browse/SOLR-10423
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: query parsers
>Affects Versions: 6.5
>Reporter: Steve Rowe
> Attachments: SOLR-10423.patch
>
>
> When {{sow=false}} and {{ShingleFilter}} is included in the query analyzer, 
> {{QueryBuilder}} produces queries that inappropriately require sequential 
> terms.  E.g. the query "A B C" produces {{(+A_B +B_C) A_B_C}} when the query 
> analyzer includes {{ maxShingleSize="3" outputUnigrams="false" tokenSeparator="_"/>}}.
> Aman Deep Singh reported this problem on the solr-user list. From 
> [http://mail-archives.apache.org/mod_mbox/lucene-solr-user/201703.mbox/%3ccanegtx9bwbpwqc-cxieac7qsas7x2tgzovomy5ztiagco1p...@mail.gmail.com%3e]:
> {quote}
> I was trying to use the shingle filter but it was not creating the query as
> desirable.
> my schema is
> {noformat}
>  positionIncrementGap="100">
>   
> 
>  maxShingleSize="4"/>
> 
>   
> 
> 
> {noformat}
> my solr query is
> {noformat}
> http://localhost:8983/solr/productCollection/select?
>  defType=edismax
> =true
> =one%20plus%20one%20four
> =nameShingle
> =false
> =xml
> {noformat}
> and it was creating the parsed query as
> {noformat}
> 
> (+(DisjunctionMaxQuery(((+nameShingle:one plus +nameShingle:plus one
> +nameShingle:one four))) DisjunctionMaxQuery(((+nameShingle:one plus
> +nameShingle:plus one four))) DisjunctionMaxQuery(((+nameShingle:one plus one 
> +nameShingle:one four))) DisjunctionMaxQuery((nameShingle:one plus one 
> four)))~1)/no_coord
> 
> 
> *++nameShingle:one plus +nameShingle:plus one +nameShingle:one four))
> ((+nameShingle:one plus +nameShingle:plus one four)) ((+nameShingle:one
> plus one +nameShingle:one four)) (nameShingle:one plus one four))~1)*
> 
> {noformat}
> So ideally token creations is perfect but in the query it is using boolean + 
> operator which is causing the problem as if i have a document with name as 
> "one plus one" ,according to the shingles it has to matched as its token will 
> be  ("one plus","one plus one","plus one") .
> I have tried using the q.op and played around the mm also but nothing is
> giving me the correct response.
> Any idea how i can fetch that document even if the document is missing any
> token.
> My expected response will be getting the document "one plus one" even the 
> user query has any additional term like "one plus one two" and so on.
> {quote}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9959) SolrInfoMBean-s category and hierarchy cleanup

[Hoss Man (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15957234#comment-15957234
 ] 

Hoss Man commented on SOLR-9959:



bq. The config can specify multiple groups or registry names, or registry name 
prefixes that it applies to, and for each matching unique registry name a 
separate reporter instance will be created.

ok ... seems weird, but i'll take your word for it that it's expected ... my 
main concern was that we weren't excessively and unneccessarily 
"re-registering" equivilent MBeans with the MBean server every time a 
(seemingly duplicate) SolrJmxReporter was created ... if you're saying that's 
not happening then i'll take your word for it.

bq. JmxObjectNameFactory uses registry name as domain (if domain is not set), 
and it also inserts reporter's name (as configured in the plugin config) in the 
hierarchy, so for each reporter instance and for each registry the hierarchy is 
unique.

I'm not sure i follow -- largely because i'm still not really clear what you 
mean by "registry name" and "domain" -- to be clear about what I saw (as an end 
user) and what i'm suggesting:

* Sample current ObjectNames (with implicit default reporter) ...
** 
{{solr:dom1=core,dom2=techproducts,reporter=default,instance=6fd59eb3,category=CACHE,name=fieldCache}}
** 
{{solr:dom1=core,dom2=techproducts,reporter=default,instance=6fd59eb3,category=SEARCHER,scope=searcher,name=maxDoc}}
** 
{{solr:dom1=node,reporter=default,instance=4007f65e,category=CONTAINER,scope=threadPool,name=coreContainerWorkExecutor.completed}}
* Suggested replacement ObjectNames for the same MBeans (with implicit default 
JMX reporter) ...
** {{solr:dom1=core,dom2=techproducts,category=CACHE,name=fieldCache}}
** 
{{solr:dom1=core,dom2=techproducts,category=SEARCHER,scope=searcher,name=maxDoc}}
** 
{{solr:dom1=node,category=CONTAINER,scope=threadPool,name=coreContainerWorkExecutor.completed}}
* Suggested replacement ObjectNames for the same MBeans (with explicitly 
configured JMX reporter using reporter name "HOSSJMX") ...
** {{HOSSJMX:dom1=core,dom2=techproducts,category=CACHE,name=fieldCache}}
** 
{{HOSSJMX:dom1=core,dom2=techproducts,category=SEARCHER,scope=searcher,name=maxDoc}}
** 
{{HOSSJMX:dom1=node,category=CONTAINER,scope=threadPool,name=coreContainerWorkExecutor.completed}}


> SolrInfoMBean-s category and hierarchy cleanup
> --
>
> Key: SOLR-9959
> URL: https://issues.apache.org/jira/browse/SOLR-9959
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: metrics
>Affects Versions: master (7.0)
>Reporter: Andrzej Bialecki 
>Assignee: Andrzej Bialecki 
>Priority: Blocker
> Fix For: master (7.0)
>
> Attachments: SOLR-9959.patch, SOLR-9959.patch
>
>
> SOLR-9947 changed categories of some of {{SolrInfoMBean-s}}, and it also 
> added an alternative view in JMX, similar to the one produced by 
> {{SolrJmxReporter}}.
> Some changes were left out from that issue because they would break the 
> back-compatibility in 6.x, but they should be done before 7.0:
> * remove the old JMX view of {{SolrInfoMBean}}-s and improve the new one so 
> that it's more readable and useful.
> * in many cases {{SolrInfoMBean.getName()}} just returns a FQCN, but it could 
> be more informative - eg. for highlighter or query plugins this could be the 
> symbolic name of a plugin that users know and use in configuration.
> * top-level categories need more thought. On one hand it's best to minimize 
> their number, on the other hand they need to meaningfully represent the 
> functionality of components that use them. SOLR-9947 made some cosmetic 
> changes, but more discussion is necessary (eg. QUERY vs. SEARCHHANDLER)
> * we should consider removing some of the methods in {{SolrInfoMBean}} that 
> usually don't return any useful information, eg. {{getDocs}}, {{getSource()}} 
> and {{getVersion()}}.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: exapanding solr core over an additional ssd

[Dawid Weiss]

> locations.  There's no reason it wouldn't be *POSSIBLE* ... but doing it
> would require a different Lucene Directory implementation -- very low
> level custom code.

There's actually FileSwitchDirectory so you could split an index into
separate files stored on separate drives... not that it makes much
sense, like Shawn said.

> If you can add another drive that's twice as big, why don't you just
> move the Solr install to the other drive?

Or create an extension of your partition on another drive -- these are
operating system facilities, not Lucene's and they're way better
handled by the operating system than Lucene. Again: not that this
makes sense (and you'd be exercising os filesystem synchronization
idioms on such spanned volumes).

Dawid

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7738) Add new InetAddressRangeField

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15957156#comment-15957156
 ] 

ASF subversion and git services commented on LUCENE-7738:
-

Commit 2bdb3c7d1e738bbcfffa45bf47639446b0734f0f in lucene-solr's branch 
refs/heads/branch_6_5 from [~nknize]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=2bdb3c7 ]

LUCENE-7738: Fix min/max verification bug in InetAddressRange to correctly 
compare IPv4 and IPv6. Update tests.


> Add new InetAddressRangeField
> -
>
> Key: LUCENE-7738
> URL: https://issues.apache.org/jira/browse/LUCENE-7738
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Nicholas Knize
> Attachments: LUCENE-7738.patch, LUCENE-7738.patch
>
>
> This issue adda a new {{InetAddressRangeField}} for indexing and querying 
> InetAddress ranges.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7738) Add new InetAddressRangeField

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15957153#comment-15957153
 ] 

ASF subversion and git services commented on LUCENE-7738:
-

Commit 6eb00543bc969d375b43ed20bedcbdb60bc8f22a in lucene-solr's branch 
refs/heads/branch_6x from [~nknize]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=6eb0054 ]

LUCENE-7738: Fix min/max verification bug in InetAddressRange to correctly 
compare IPv4 and IPv6. Update tests.


> Add new InetAddressRangeField
> -
>
> Key: LUCENE-7738
> URL: https://issues.apache.org/jira/browse/LUCENE-7738
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Nicholas Knize
> Attachments: LUCENE-7738.patch, LUCENE-7738.patch
>
>
> This issue adda a new {{InetAddressRangeField}} for indexing and querying 
> InetAddress ranges.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7738) Add new InetAddressRangeField

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15957148#comment-15957148
 ] 

ASF subversion and git services commented on LUCENE-7738:
-

Commit ef8126e5eab7aec9c8775c2e08bd6c2bb1ef690f in lucene-solr's branch 
refs/heads/master from [~nknize]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=ef8126e ]

LUCENE-7738: Fix min/max verification bug in InetAddressRange to correctly 
compare IPv4 and IPv6. Update tests.


> Add new InetAddressRangeField
> -
>
> Key: LUCENE-7738
> URL: https://issues.apache.org/jira/browse/LUCENE-7738
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Nicholas Knize
> Attachments: LUCENE-7738.patch, LUCENE-7738.patch
>
>
> This issue adda a new {{InetAddressRangeField}} for indexing and querying 
> InetAddress ranges.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-10427) Add support for FloatPointField in Calcite

[Michael Suzuki (JIRA)]

Michael Suzuki created SOLR-10427:
-

 Summary: Add support for FloatPointField in Calcite
 Key: SOLR-10427
 URL: https://issues.apache.org/jira/browse/SOLR-10427
 Project: Solr
  Issue Type: Bug
  Security Level: Public (Default Security Level. Issues are Public)
Reporter: Michael Suzuki


Currently the SolrJDBC is unable to handle fields of type FloatPointField.
When we query the example techproducts 
{code}
 SELECT price FROM techproducts
{code} 
We get the following error: java.lang.Double cannot be cast to java.lang.String.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (SOLR-10009) Explore removing CoreDescriptor from SolrCore

[Erick Erickson (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-10009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Erick Erickson resolved SOLR-10009.
---
Resolution: Resolved

Incorporated in SOLR-10007

> Explore removing CoreDescriptor from SolrCore
> -
>
> Key: SOLR-10009
> URL: https://issues.apache.org/jira/browse/SOLR-10009
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Erick Erickson
>Assignee: Erick Erickson
>
> There should be one place where the CoreDescriptor resides, not a copy in 
> CoreContainer and one in SolrCore. Changing and persisting these from one 
> place or the other leads to inconsistencies.
> This JIRA is partly to get the discussion going of how to untangle these.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (SOLR-10008) Move the reference to CoreContainer from CoreDescriptor to SolrCore.

[Erick Erickson (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-10008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Erick Erickson resolved SOLR-10008.
---
Resolution: Resolved

Incorporated in SOLR-10007

> Move the reference to CoreContainer from CoreDescriptor to SolrCore.
> 
>
> Key: SOLR-10008
> URL: https://issues.apache.org/jira/browse/SOLR-10008
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Affects Versions: trunk, 6.4
>Reporter: Erick Erickson
>Assignee: Erick Erickson
> Attachments: SOLR-10008.patch
>
>
> CoreDescriptor referencing CoreContainer is just weird. If anyplace, the 
> SolrCore should have a reference to CoreContainer.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-10007) Clean up references to CoreContainer and CoreDescriptors

[Erick Erickson (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-10007?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Erick Erickson updated SOLR-10007:
--
Attachment: SOLR-10007.patch

I think this is very close to ready, running tests and precommit and such again.

It touches a _lot_ of files, but most of them are changing 
solrcore.getCoreDescriptor().getCoreContainer()
to
solrcore.getCoreContainer().

Along the way I renamed a few internal variables to more accurately reflect 
what they're supposed to be.

There may be a few gratuitous format changes, I upgraded IntelliJ and somehow 
had the "strip trailing spaces on save" set and "reformat whole file" rather 
than "only reformat VCS changed text". I went through and undid all the 
gratuitous junk I could find, but a few might have snuck through.

Structurally this is significantly better I think. There were a number of 
places where we were using the CoreDescriptors to infer the state of the 
SolrCore. The big thing this patch does is try to separate the use of 
CoreDescriptors and SolrCores.

The other change here is it moves the CoreContainer reference from the 
CoreDescriptor to SolrCore.

I'll commit this in a day or two unless there are objections. It' incorporates 
both SOLR-10008 and SOLR-10009 so I'll close those now.


> Clean up references to CoreContainer and CoreDescriptors
> 
>
> Key: SOLR-10007
> URL: https://issues.apache.org/jira/browse/SOLR-10007
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Affects Versions: trunk, 6.4
>Reporter: Erick Erickson
>Assignee: Erick Erickson
> Attachments: SOLR-10007.patch, SOLR-10007.patch
>
>
> It's a bit weird that CoreDescriptor contains a reference to CoreContainer. 
> It seems like the SolrCore should reference CoreContainer.
> Similarly, SolrCore keeps a copy of CoreDescriptor and another copy is kept 
> in the various CoreDescirptor lists making it difficult to track which is the 
> "real" CoreDescriptor.
> This is an umbrella issue as I think this (and perhaps other) issues can be 
> tackled separately.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-10426) Add shuffle Streaming Expression

[Joel Bernstein (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-10426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joel Bernstein updated SOLR-10426:
--
Description: 
Currently the concept of MapReduce shuffling is lumped into the *search* 
Streaming Expression. This causes quite a bit of confusion as users perform a 
search using the /select handler by default, but actually wanted shuffling 
behavior which requires the /export handler.

We can solve this problem by creating a separate function called *shuffle* that 
always uses the /export handler.

This will also allow us to clean up some behaviors in the search expression 
that are somewhat unexpected.

  was:
Currently the concept of MapReduce shuffling is lumped into the *search* 
Streaming Expression. This causes quite a bit of confusion as users perform a 
search using the /select handler by default, but actually wanted shuffling 
behavior which requires the /export handler.

We can solve this problem by creating a separate function called *shuffle* that 
always uses the export handler.

This will also allow us to clean up some behaviors in the search expression 
that are somewhat unexpected.


> Add shuffle Streaming Expression
> 
>
> Key: SOLR-10426
> URL: https://issues.apache.org/jira/browse/SOLR-10426
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Joel Bernstein
>
> Currently the concept of MapReduce shuffling is lumped into the *search* 
> Streaming Expression. This causes quite a bit of confusion as users perform a 
> search using the /select handler by default, but actually wanted shuffling 
> behavior which requires the /export handler.
> We can solve this problem by creating a separate function called *shuffle* 
> that always uses the /export handler.
> This will also allow us to clean up some behaviors in the search expression 
> that are somewhat unexpected.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: exapanding solr core over an additional ssd

[Shawn Heisey]

On 4/5/2017 2:06 AM, FOTACHE CHRISTIAN wrote:
> I'm having this problem: I'm running a solr instance on an 120Gb ssd,
> but the solr core is growing quickly and I badly need extra-space. I
> have another 240Gb ssd that I can attach to my laptop but I don't know
> how how to make the solr core expand naturally on the newly attach ssd.

I am not aware of a way to have one core use two different file
locations.  There's no reason it wouldn't be *POSSIBLE* ... but doing it
would require a different Lucene Directory implementation -- very low
level custom code.

If you can add another drive that's twice as big, why don't you just
move the Solr install to the other drive?  Because it would be the only
thing on that drive, you would actually have MORE than twice the
original space -- because that drive would not contain any operating
system files.

I hope you're not running Solr on a laptop in a production deployment. 
Laptops typically do not have any fault tolerance features other than
the battery.

Thanks,
Shawn


-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-10426) Add shuffle Streaming Expression

[Joel Bernstein (JIRA)]

Joel Bernstein created SOLR-10426:
-

 Summary: Add shuffle Streaming Expression
 Key: SOLR-10426
 URL: https://issues.apache.org/jira/browse/SOLR-10426
 Project: Solr
  Issue Type: Improvement
  Security Level: Public (Default Security Level. Issues are Public)
Reporter: Joel Bernstein


Currently the concept of MapReduce shuffling is lumped into the *search* 
Streaming Expression. This causes quite a bit of confusion as users perform a 
search using the /select handler by default, but actually wanted shuffling 
behavior which requires the export handler.

We can solve this problem by creating a separate function called *shuffle* that 
always uses the export handler.

This will also allow us to clean up some behaviors in the search expression 
that are somewhat unexpected.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-10426) Add shuffle Streaming Expression

[Joel Bernstein (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-10426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joel Bernstein updated SOLR-10426:
--
Description: 
Currently the concept of MapReduce shuffling is lumped into the *search* 
Streaming Expression. This causes quite a bit of confusion as users perform a 
search using the /select handler by default, but actually wanted shuffling 
behavior which requires the /export handler.

We can solve this problem by creating a separate function called *shuffle* that 
always uses the export handler.

This will also allow us to clean up some behaviors in the search expression 
that are somewhat unexpected.

  was:
Currently the concept of MapReduce shuffling is lumped into the *search* 
Streaming Expression. This causes quite a bit of confusion as users perform a 
search using the /select handler by default, but actually wanted shuffling 
behavior which requires the export handler.

We can solve this problem by creating a separate function called *shuffle* that 
always uses the export handler.

This will also allow us to clean up some behaviors in the search expression 
that are somewhat unexpected.


> Add shuffle Streaming Expression
> 
>
> Key: SOLR-10426
> URL: https://issues.apache.org/jira/browse/SOLR-10426
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Joel Bernstein
>
> Currently the concept of MapReduce shuffling is lumped into the *search* 
> Streaming Expression. This causes quite a bit of confusion as users perform a 
> search using the /select handler by default, but actually wanted shuffling 
> behavior which requires the /export handler.
> We can solve this problem by creating a separate function called *shuffle* 
> that always uses the export handler.
> This will also allow us to clean up some behaviors in the search expression 
> that are somewhat unexpected.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: VOTE: Apache Solr Ref Guide for 6.5 RC0

[Joel Bernstein]

Sorry for the delay but the SQL changes in the RefGuide are now complete.
People can read the cwiki directly to see the Apache Calcite changes to the
docs. The changes are actually pretty subtle, so the 6.5 PDF should be fine
for most situations.

Joel Bernstein
http://joelsolr.blogspot.com/

On Tue, Apr 4, 2017 at 3:35 PM, Cassandra Targett 
wrote:

> It looks like this vote has passed, thanks everyone. I'll start the
> release process now.
>
> Cassandra
>
> On Mon, Apr 3, 2017 at 6:36 AM, Shalin Shekhar Mangar
>  wrote:
> > +1
> >
> > On Sat, Apr 1, 2017 at 1:23 AM, Cassandra Targett 
> wrote:
> >> Please vote for the first release candidate of the Apache Solr
> >> Reference Guide for 6.5.
> >>
> >> Artifacts are available from:
> >> https://dist.apache.org/repos/dist/dev/lucene/solr/ref-
> guide/apache-solr-ref-guide-6.5-RC0/
> >>
> >> $ more apache-solr-ref-guide-6.5.pdf.sha1
> >> a80b3b776b840c59234b6a416b4908c8af7217d1  apache-solr-ref-guide-6.5.pdf
> >>
> >> Here's my +1.
> >>
> >> Thanks,
> >> Cassandra
> >>
> >> -
> >> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> >> For additional commands, e-mail: dev-h...@lucene.apache.org
> >>
> >
> >
> >
> > --
> > Regards,
> > Shalin Shekhar Mangar.
> >
> > -
> > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> > For additional commands, e-mail: dev-h...@lucene.apache.org
> >
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org
>
>

[jira] [Updated] (SOLR-10239) MOVEREPLICA API

[Cao Manh Dat (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-10239?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Cao Manh Dat updated SOLR-10239:

Attachment: SOLR-10239.patch

Updated patch, based on [~shalinmangar] about get dataDir directly from 
clusterstate.

> MOVEREPLICA API
> ---
>
> Key: SOLR-10239
> URL: https://issues.apache.org/jira/browse/SOLR-10239
> Project: Solr
>  Issue Type: Sub-task
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Ishan Chattopadhyaya
>Assignee: Cao Manh Dat
> Attachments: SOLR-10239.patch, SOLR-10239.patch, SOLR-10239.patch
>
>
> To move a replica from a node to another node, there should be an API 
> command. This should be better than having to do ADDREPLICA and DELETEREPLICA.
> The API will like this
> {code}
> /admin/collections?action=MOVEREPLICA=collection=shard=replica=nodeName=nodeName
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10303) Add date/time Stream Evaluators

[Dennis Gove (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15956946#comment-15956946
 ] 

Dennis Gove commented on SOLR-10303:


In case you missed them, I added comments to the PR at 
https://github.com/apache/lucene-solr/pull/171

> Add date/time Stream Evaluators
> ---
>
> Key: SOLR-10303
> URL: https://issues.apache.org/jira/browse/SOLR-10303
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Joel Bernstein
> Attachments: SOLR-10303.patch
>
>
> This ticket will add Stream Evaluators that extract date/time values from a 
> Solr date field. The following Evaluators will be supported:
> hour (date)
> minute (date)
> month (date) 
> monthname(date) 
> quarter(date) 
> second (date)
> year(date)
> Syntax:
> {code}
> select(id,
>year(recdate) as year,
>month(recdate) as month,
>day(recdate) as day,
>search(logs, q="blah", fl="id, recdate", sort="recdate asc", 
> qt="/export"))
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10303) Add date/time Stream Evaluators

[Gethin James (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15956929#comment-15956929
 ] 

Gethin James commented on SOLR-10303:
-

1) I like the idea of a TupleContext which the SelectStream can pass along.  
Are you thinking that should be done as part of this ticket?  Currently the 
implementation just acts on tuple evaluations so there is nothing specific for 
SelectStream.
2) The current implementation support a variety of types: String, Instant, 
Date, TemporalAccessor.  Its missing long support so I can add that.

> Add date/time Stream Evaluators
> ---
>
> Key: SOLR-10303
> URL: https://issues.apache.org/jira/browse/SOLR-10303
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Joel Bernstein
> Attachments: SOLR-10303.patch
>
>
> This ticket will add Stream Evaluators that extract date/time values from a 
> Solr date field. The following Evaluators will be supported:
> hour (date)
> minute (date)
> month (date) 
> monthname(date) 
> quarter(date) 
> second (date)
> year(date)
> Syntax:
> {code}
> select(id,
>year(recdate) as year,
>month(recdate) as month,
>day(recdate) as day,
>search(logs, q="blah", fl="id, recdate", sort="recdate asc", 
> qt="/export"))
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10424) /update/docs/json is swalling all fields

[Alexandre Rafalovitch (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15956930#comment-15956930
 ] 

Alexandre Rafalovitch commented on SOLR-10424:
--

It looks like this was introduced by SOLR-9557 as an example of configuration 
in the techproduct's params.json. Perhaps the example config should be somewhat 
less radical?

> /update/docs/json is swalling all fields
> 
>
> Key: SOLR-10424
> URL: https://issues.apache.org/jira/browse/SOLR-10424
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Affects Versions: 6.5, master (7.0)
>Reporter: Hoss Man
>
> I'm not sure when/how exactly this broke, but sending a list of documents to 
> {{/update/json/docs}} is currently useless -- regardless of what your 
> documents contain, all you get is 3 fields: {{id}}, {{\_version\_}}, and a 
> {{\_src\_}} field containing your original JSON, but none of the fields you 
> specified are added.
> Steps to reproduce...
> {noformat}
> git co releases/lucene-solr/6.5.0
> ...
> ant clean && cd solr && ant server
> ...
> bin/solr -e techproducts
> ...
> curl 'http://localhost:8983/solr/techproducts/update/json/docs?commit=true' 
> --data-binary @example/exampledocs/books.json -H 
> 'Content-type:application/json'
> ...
> curl 'http://localhost:8983/solr/techproducts/query?q=id:978-1933988177'
> {
>   "responseHeader":{
> "status":0,
> "QTime":5,
> "params":{
>   "q":"id:978-1933988177"}},
>   "response":{"numFound":1,"start":0,"docs":[
>   {
> "id":"978-1933988177",
> "_src_":"{\n\"id\" : \"978-1933988177\",\n\"cat\" : 
> [\"book\",\"paperback\"],\n\"name\" : \"Lucene in Action, Second 
> Edition\",\n\"author\" : \"Michael McCandless\",\n\"sequence_i\" : 
> 1,\n\"genre_s\" : \"IT\",\n\"inStock\" : true,\n\"price\" : 
> 30.50,\n\"pages_i\" : 475\n  }",
> "_version_":1563794703530328065}]
>   }}
> {noformat}
> Compare with using {{/update/json}} ...
> {noformat}
> curl 'http://localhost:8983/solr/techproducts/update/json?commit=true' 
> --data-binary @example/exampledocs/books.json -H 
> 'Content-type:application/json'
> ...
> curl 'http://localhost:8983/solr/techproducts/query?q=id:978-1933988177'
> {
>   "responseHeader":{
> "status":0,
> "QTime":0,
> "params":{
>   "q":"id:978-1933988177"}},
>   "response":{"numFound":1,"start":0,"docs":[
>   {
> "id":"978-1933988177",
> "cat":["book",
>   "paperback"],
> "name":"Lucene in Action, Second Edition",
> "author":"Michael McCandless",
> "author_s":"Michael McCandless",
> "sequence_i":1,
> "sequence_pi":1,
> "genre_s":"IT",
> "inStock":true,
> "price":30.5,
> "price_c":"30.5,USD",
> "pages_i":475,
> "pages_pi":475,
> "_version_":1563794766373584896}]
>   }}
> {noformat}
> According to the ref-guide, the only diff between these two endpoints should 
> be that {{/update/json/docs}} defaults {{json.command=false}} ... but since 
> the top level JSON structure in books.json is a list ({{"[ ... ]"}}) that 
> shouldn't matter because that's not the solr JSON command syntax.
> 
> If you try to send a singular JSON document tp {{/update/json/docs}}, you get 
> the same problem...
> {noformat}
> curl -X POST -H 'Content-type:application/json' --data-binary 
> '{"id":"HOSS","popularity":42}' 
> 'http://localhost:8983/solr/techproducts/update/json/docs?commit=true'
> ...
> curl 'http://localhost:8983/solr/techproducts/query?q=id:HOSS'{
>   "responseHeader":{
> "status":0,
> "QTime":0,
> "params":{
>   "q":"id:HOSS"}},
>   "response":{"numFound":1,"start":0,"docs":[
>   {
> "id":"HOSS",
> "_src_":"{\"id\":\"HOSS\",\"popularity\":42}",
> "_version_":1563795188162232320}]
>   }}
> {noformat}
> ...even though the same JSON works fine to 
> {{/update/json?json.command=false}} ...
> {noformat}
> curl -X POST -H 'Content-type:application/json' --data-binary 
> '{"id":"HOSS","popularity":42}' 
> 'http://localhost:8983/solr/techproducts/update/json?commit=true=false'
> ...
> curl 'http://localhost:8983/solr/techproducts/query?q=id:HOSS'{
>   "responseHeader":{
> "status":0,
> "QTime":1,
> "params":{
>   "q":"id:HOSS"}},
>   "response":{"numFound":1,"start":0,"docs":[
>   {
> "id":"HOSS",
> "popularity":42,
> "_version_":1563795262581768192}]
>   }}
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-8240) mapUniqueKeyOnly=true silently ignores field mapping params f=.. when indexing custom JSON

[Alexandre Rafalovitch (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-8240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexandre Rafalovitch updated SOLR-8240:

Component/s: examples

> mapUniqueKeyOnly=true silently ignores field mapping params f=.. when 
> indexing custom JSON
> --
>
> Key: SOLR-8240
> URL: https://issues.apache.org/jira/browse/SOLR-8240
> Project: Solr
>  Issue Type: Bug
>  Components: examples, update
>Affects Versions: 5.3.1
>Reporter: Mikhail Khludnev
>
> since SOLR-6633 most of published JSON examples doesn't work in 
> {{techproducts}} exmple, it provides unpleasant user experience.
> see [comment 
> 1|https://cwiki.apache.org/confluence/display/solr/Uploading+Data+with+Index+Handlers?focusedCommentId=61326627#comment-61326627]
>  and [comment 
> 2|https://issues.apache.org/jira/browse/SOLR-6304?focusedCommentId=14988279=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14988279]
>  
> It's proposed to explicitly throw an error if {{mapUniqueKeyOnly=true}} and 
> {{f=..}} are supplied, like it's done now with srcField=.. and split=.. : 
> {{"msg":"Raw data can be stored only if split=/","code":400,}}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Closed] (SOLR-9601) DIH: Radicially simplify Tika example to only show relevant configuration

[Alexandre Rafalovitch (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-9601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexandre Rafalovitch closed SOLR-9601.
---

> DIH: Radicially simplify Tika example to only show relevant configuration
> -
>
> Key: SOLR-9601
> URL: https://issues.apache.org/jira/browse/SOLR-9601
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: contrib - DataImportHandler, contrib - Solr Cell (Tika 
> extraction)
>Affects Versions: 6.x, master (7.0)
>Reporter: Alexandre Rafalovitch
>Assignee: Alexandre Rafalovitch
>  Labels: examples, usability
> Fix For: master (7.0), 6.6
>
> Attachments: tika2_20170308.tgz, tika2_20170316.tgz
>
>
> Solr DIH examples are legacy examples to show how DIH work. However, they 
> include full configurations that may obscure teaching points. This is no 
> longer needed as we have 3 full-blown examples in the configsets. 
> Specifically for Tika, the field types definitions were at some point 
> simplified to have less support files in the configuration directory. This, 
> however, means that we now have field definitions that have same names as 
> other examples, but different definitions. 
> Importantly, Tika does not use most (any?) of those modified definitions. 
> They are there just for completeness. Similarly, the solrconfig.xml includes 
> extract handler even though we are demonstrating a different path of using 
> Tika. Somebody grepping through config files may get confused about what 
> configuration aspects contributes to what experience.
> I am planning to significantly simplify configuration and schema of Tika 
> example to **only** show DIH Tika extraction path. It will end-up a very 
> short and focused example.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Closed] (SOLR-7383) DIH: rewrite XPathEntityProcessor/RSS example as the smallest good demo possible

[Alexandre Rafalovitch (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-7383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexandre Rafalovitch closed SOLR-7383.
---

> DIH: rewrite XPathEntityProcessor/RSS example as the smallest good demo 
> possible
> 
>
> Key: SOLR-7383
> URL: https://issues.apache.org/jira/browse/SOLR-7383
> Project: Solr
>  Issue Type: Bug
>  Components: contrib - DataImportHandler
>Affects Versions: 5.0, 6.0
>Reporter: Upayavira
>Assignee: Alexandre Rafalovitch
>Priority: Minor
> Fix For: master (7.0), 6.6
>
> Attachments: atom_20170315.tgz, rss-data-config.xml, SOLR-7383.patch
>
>
> The DIH example (solr/example/example-DIH/solr/rss/conf/rss-data-config.xml) 
> is broken again. See associated issues.
> Below is a config that should work.
> This is caused by Slashdot seemingly oscillating between RDF/RSS and pure 
> RSS. Perhaps we should depend upon something more static, rather than an 
> external service that is free to change as it desires.
> {code:xml}
> 
> 
> 
>  pk="link"
> url="http://rss.slashdot.org/Slashdot/slashdot;
> processor="XPathEntityProcessor"
> forEach="/RDF/item"
> transformer="DateFormatTransformer">
>   
>  commonField="true" />
>  commonField="true" />
>  commonField="true" />
>   
> 
> 
> 
> 
> 
>  dateTimeFormat="-MM-dd'T'HH:mm:ss" />
> 
> 
> 
> 
> 
> 
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10239) MOVEREPLICA API

[Shalin Shekhar Mangar (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15956872#comment-15956872
 ] 

Shalin Shekhar Mangar commented on SOLR-10239:
--

I have only skimmed the patch without running it. Overall, it looks fine. Just 
that there is no need to ask the replica for its data dir using core admin API 
-- in case of shared storage, the data and ulog directories are stored in 
cluster state itself -- see SOLR-8913.

> MOVEREPLICA API
> ---
>
> Key: SOLR-10239
> URL: https://issues.apache.org/jira/browse/SOLR-10239
> Project: Solr
>  Issue Type: Sub-task
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Ishan Chattopadhyaya
>Assignee: Cao Manh Dat
> Attachments: SOLR-10239.patch, SOLR-10239.patch
>
>
> To move a replica from a node to another node, there should be an API 
> command. This should be better than having to do ADDREPLICA and DELETEREPLICA.
> The API will like this
> {code}
> /admin/collections?action=MOVEREPLICA=collection=shard=replica=nodeName=nodeName
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: [nag][VOTE] Release PyLucene 6.5.0 (rc2) (now with Python 3 support)

[Jan Høydahl]

Tested on python3 on macOS 10.12.3

One error:
==
ERROR: testThroughLayerException (__main__.PythonExceptionTestCase)
--
Traceback (most recent call last):
  File "test3/test_PythonException.py", line 34, in testThroughLayerException
qp.parse("foo bar")
lucene.JavaError: 

--
Ran 1 test in 0.004s

FAILED (errors=1)
/usr/local/Cellar/python3/3.6.1/bin/python3 test3/test_PythonQueryParser.py
..
--
Ran 2 tests in 0.011s


PS: The http://lucene.apache.org/pylucene/install.html 
 guide could need a better 
recipe for python3, i.e. use “python3” instead of “python” or use variables.

--
Jan Høydahl, search solution architect
Cominvent AS - www.cominvent.com

> 4. apr. 2017 kl. 20.56 skrev Andi Vajda :
> 
> 
> One more PMC vote is needed to make this release !
> Thanks !
> 
> Andi..
> 
> -- Forwarded message --
> Date: Thu, 30 Mar 2017 12:27:53 -0700 (PDT)
> From: Andi Vajda 
> To: pylucene-dev@lucene.apache.org
> Cc: gene...@lucene.apache.rog
> Subject: [VOTE] Release PyLucene 6.5.0 (rc2) (now with Python 3 support)
> 
> 
> A few fixes were needed in JCC for better Windows support.
> The PyLucene 6.5.0 rc1 vote is thus cancelled.
> 
> I'm now calling for a vote on PyLucene 6.5.0 rc2.
> 
> The PyLucene 6.5.0 (rc2) release tracking the recent release of
> Apache Lucene 6.5.0 is ready.
> 
> A release candidate is available from:
>  https://dist.apache.org/repos/dist/dev/lucene/pylucene/6.5.0-rc2/
> 
> PyLucene 6.5.0 is built with JCC 3.0 included in these release artifacts.
> 
> JCC 3.0 now supports Python 3.3+ (in addition to Python 2.3+).
> PyLucene may be built with Python 2 or Python 3.
> 
> Please vote to release these artifacts as PyLucene 6.5.0.
> Anyone interested in this release can and should vote !
> 
> Thanks !
> 
> Andi..
> 
> ps: the KEYS file for PyLucene release signing is at:
> https://dist.apache.org/repos/dist/release/lucene/pylucene/KEYS
> https://dist.apache.org/repos/dist/dev/lucene/pylucene/KEYS
> 
> pps: here is my +1

[jira] [Assigned] (LUCENE-7719) UnifiedHighlighter doesn't handle some AutomatonQuery's with multi-byte chars

[David Smiley (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-7719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

David Smiley reassigned LUCENE-7719:


Assignee: David Smiley

> UnifiedHighlighter doesn't handle some AutomatonQuery's with multi-byte chars
> -
>
> Key: LUCENE-7719
> URL: https://issues.apache.org/jira/browse/LUCENE-7719
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: modules/highlighter
>Reporter: David Smiley
>Assignee: David Smiley
>
> In MultiTermHighlighting, a CharacterRunAutomaton is being created that takes 
> the result of AutomatonQuery.getAutomaton that in turn is byte oriented, not 
> character oriented.  For ASCII terms, this is safe but it's not for 
> multi-byte characters.  This is most likely going to rear it's head with a 
> WildcardQuery, but due to special casing in MultiTermHighlighting, 
> PrefixQuery isn't affected.  Nonetheless it'd be nice to get a general fix in 
> so that MultiTermHighlighting can remove special cases for PrefixQuery and 
> TermRangeQuery (both subclass AutomatonQuery).
> AFAICT, this bug was likely in the PostingsHighlighter since inception.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (SOLR-10404) The "fetch" streaming expression should escape join values in generated query

[David Smiley (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-10404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

David Smiley resolved SOLR-10404.
-
Resolution: Fixed

> The "fetch" streaming expression should escape join values in generated query
> -
>
> Key: SOLR-10404
> URL: https://issues.apache.org/jira/browse/SOLR-10404
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Affects Versions: 6.5
>Reporter: David Smiley
>Assignee: David Smiley
> Fix For: 6.5.1
>
> Attachments: SOLR_10404.patch
>
>
> The "fetch" streaming expression joins data from another collection.  Example:
> {{expr=fetch(collection,search(...), on="fieldA=fieldB"}}
>   Internally, it does this by building a Solr query that looks like  
> {{fieldB:(value1 value2 value3)}}.  *But those values are not escaped*; they 
> should be.  See FetchStream.java line 233.  The ramification is that, for 
> example, if a value contains a colon, then this isn't going to work.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10404) The "fetch" streaming expression should escape join values in generated query

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15956815#comment-15956815
 ] 

ASF subversion and git services commented on SOLR-10404:


Commit 55eb30cc9d77196227d8e9d86aae77a55545f869 in lucene-solr's branch 
refs/heads/branch_6_5 from [~dsmiley]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=55eb30c ]

SOLR-10404: fetch() streaming expression: escape values in generated query.

(cherry picked from commit 048705e)


> The "fetch" streaming expression should escape join values in generated query
> -
>
> Key: SOLR-10404
> URL: https://issues.apache.org/jira/browse/SOLR-10404
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Affects Versions: 6.5
>Reporter: David Smiley
>Assignee: David Smiley
> Fix For: 6.5.1
>
> Attachments: SOLR_10404.patch
>
>
> The "fetch" streaming expression joins data from another collection.  Example:
> {{expr=fetch(collection,search(...), on="fieldA=fieldB"}}
>   Internally, it does this by building a Solr query that looks like  
> {{fieldB:(value1 value2 value3)}}.  *But those values are not escaped*; they 
> should be.  See FetchStream.java line 233.  The ramification is that, for 
> example, if a value contains a colon, then this isn't going to work.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10404) The "fetch" streaming expression should escape join values in generated query

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15956804#comment-15956804
 ] 

ASF subversion and git services commented on SOLR-10404:


Commit 048705e0e47ac04b6914e73c7ec3cc8daa12f2e0 in lucene-solr's branch 
refs/heads/branch_6x from [~dsmiley]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=048705e ]

SOLR-10404: fetch() streaming expression: escape values in generated query.

(cherry picked from commit cb9f151)


> The "fetch" streaming expression should escape join values in generated query
> -
>
> Key: SOLR-10404
> URL: https://issues.apache.org/jira/browse/SOLR-10404
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Affects Versions: 6.5
>Reporter: David Smiley
>Assignee: David Smiley
> Fix For: 6.5.1
>
> Attachments: SOLR_10404.patch
>
>
> The "fetch" streaming expression joins data from another collection.  Example:
> {{expr=fetch(collection,search(...), on="fieldA=fieldB"}}
>   Internally, it does this by building a Solr query that looks like  
> {{fieldB:(value1 value2 value3)}}.  *But those values are not escaped*; they 
> should be.  See FetchStream.java line 233.  The ramification is that, for 
> example, if a value contains a colon, then this isn't going to work.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org