[jira] [Commented] (LUCENE-7970) Improve generation of circle plane

[Karl Wright (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16175907#comment-16175907
 ] 

Karl Wright commented on LUCENE-7970:
-

Also, I played around with the vincenti bearing formula, as described here, and 
determined that it works reasonably even at the poles.  Giving four direction 
bearings seemingly does yield reasonable point values.

http://www.movable-type.co.uk/scripts/latlong-vincenty.html

[~ivera], can you comment what the source is of your point-on-bearing code?  
I'd also want to be sure that we know in advance what the limits are -- is it 
limited to any particular maximum surface distance, for instance?


> Improve generation of circle plane 
> ---
>
> Key: LUCENE-7970
> URL: https://issues.apache.org/jira/browse/LUCENE-7970
> Project: Lucene - Core
>  Issue Type: Improvement
>  Components: modules/spatial3d
>Reporter: Ignacio Vera
>Assignee: Karl Wright
> Attachments: LUCENE_7970.patch, LUCENE-7970.patch, 
> LUCENE-7970-proposed.patch
>
>
> Hi [~Karl wright],
> How circles are currently build do not behave very well when the planet model 
> is not an sphere. when you are close to the border in WGS84 you might get 
> false positves or false negatives when checking if a point is WITHIN. I think 
> the reason is how the points to generate the circle plane are generated which 
> assumes a sphere.
> My proposal is the following:
> Add a new method to PlanetModel:  
> public GeoPoint pointOnBearing(GeoPoint from, double dist, double bearing);
> Which uses and algorithm that takes into account that the planet might not be 
> spherical. For example Vincenty's formulae 
> (https://en.wikipedia.org/wiki/Vincenty%27s_formulae).
> Use this method to generate the points for the circle plane. My experiments 
> shows that this approach removes false negatives in WGS84 meanwhile it works 
> nicely in the Sphere.
> Does it make sense?



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS-EA] Lucene-Solr-7.x-Linux (64bit/jdk-9-ea+181) - Build # 458 - Unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-7.x-Linux/458/
Java: 64bit/jdk-9-ea+181 -XX:-UseCompressedOops -XX:+UseSerialGC 
--illegal-access=deny

1 tests failed.
FAILED:  org.apache.solr.update.TestInPlaceUpdatesDistrib.test

Error Message:
inplace_updatable_float didn't match for replica at client: 
http://127.0.0.1:37077/fxgh/collection1 expected:<111.0> but was:<1.0>

Stack Trace:
java.lang.AssertionError: inplace_updatable_float didn't match for replica at 
client: http://127.0.0.1:37077/fxgh/collection1 expected:<111.0> but was:<1.0>
at 
__randomizedtesting.SeedInfo.seed([26FCEC5F308DF1CE:AEA8D3859E719C36]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at 
org.apache.solr.update.TestInPlaceUpdatesDistrib.assertReplicaValue(TestInPlaceUpdatesDistrib.java:980)
at 
org.apache.solr.update.TestInPlaceUpdatesDistrib.outOfOrderUpdatesIndividualReplicaTest(TestInPlaceUpdatesDistrib.java:643)
at 
org.apache.solr.update.TestInPlaceUpdatesDistrib.test(TestInPlaceUpdatesDistrib.java:144)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:564)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
 

[jira] [Commented] (SOLR-10990) QueryComponent.process breakup (for readability)

[David Smiley (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16175802#comment-16175802
 ] 

David Smiley commented on SOLR-10990:
-

Very nice change; thanks.

> QueryComponent.process breakup (for readability)
> 
>
> Key: SOLR-10990
> URL: https://issues.apache.org/jira/browse/SOLR-10990
> Project: Solr
>  Issue Type: Wish
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Christine Poerschke
>Assignee: Christine Poerschke
>Priority: Minor
> Fix For: master (8.0), 7.1
>
>
> The method is currently very long i.e. 
> https://github.com/apache/lucene-solr/blob/e2521b2a8baabdaf43b92192588f51e042d21e97/solr/core/src/java/org/apache/solr/handler/component/QueryComponent.java#L300-L565
>  and breaking it up along logical lines (ids, grouped distributed first 
> phase, grouped distributed second phase, undistributed grouped, ungrouped) 
> would make it more readable.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-11293) HttpPartitionTest fails often

[Cao Manh Dat (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-11293?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Cao Manh Dat updated SOLR-11293:

Priority: Blocker  (was: Major)

> HttpPartitionTest fails often
> -
>
> Key: SOLR-11293
> URL: https://issues.apache.org/jira/browse/SOLR-11293
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Noble Paul
>Assignee: Noble Paul
>Priority: Blocker
> Fix For: 7.1
>
> Attachments: SOLR-11293.patch, SOLR-11293.patch, SOLR-11293.patch
>
>
> https://jenkins.thetaphi.de/job/Lucene-Solr-master-MacOSX/4140/testReport/org.apache.solr.cloud/HttpPartitionTest/test/
> {code}
> Error Message
> Doc with id=1 not found in http://127.0.0.1:60897/b/xj/collMinRf_1x3 due to: 
> Path not found: /id; rsp={doc=null}
> Stacktrace
> java.lang.AssertionError: Doc with id=1 not found in 
> http://127.0.0.1:60897/b/xj/collMinRf_1x3 due to: Path not found: /id; 
> rsp={doc=null}
>   at 
> __randomizedtesting.SeedInfo.seed([ACF841744A332569:24AC7EAEE4CF4891]:0)
>   at org.junit.Assert.fail(Assert.java:93)
>   at org.junit.Assert.assertTrue(Assert.java:43)
>   at 
> org.apache.solr.cloud.HttpPartitionTest.assertDocExists(HttpPartitionTest.java:603)
>   at 
> org.apache.solr.cloud.HttpPartitionTest.assertDocsExistInAllReplicas(HttpPartitionTest.java:558)
>   at 
> org.apache.solr.cloud.HttpPartitionTest.testMinRf(HttpPartitionTest.java:249)
>   at 
> org.apache.solr.cloud.HttpPartitionTest.test(HttpPartitionTest.java:127)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-11377) Add expMovingAverage (exponential moving average) Stream Evaluator

[Mathew (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-11377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mathew updated SOLR-11377:
--
Attachment: SOLR-11377.patch

double stash...fixed.

> Add expMovingAverage (exponential moving average) Stream Evaluator
> --
>
> Key: SOLR-11377
> URL: https://issues.apache.org/jira/browse/SOLR-11377
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Joel Bernstein
>Assignee: Joel Bernstein
> Fix For: master (8.0), 7.1
>
> Attachments: SOLR-11377.patch, SOLR-11377.patch
>
>
> This ticket will add the exponential moving average Stream Evaluator.
> Syntax:
> {code}
> expMovingAvg(colA)
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS-EA] Lucene-Solr-7.x-Windows (32bit/jdk-9-ea+181) - Build # 204 - Still Unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-7.x-Windows/204/
Java: 32bit/jdk-9-ea+181 -client -XX:+UseSerialGC --illegal-access=deny

1 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.solr.analytics.NoFacetCloudTest

Error Message:
Could not find collection:collection1

Stack Trace:
java.lang.AssertionError: Could not find collection:collection1
at __randomizedtesting.SeedInfo.seed([B6EC1EC7F28FDFC2]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertNotNull(Assert.java:526)
at 
org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:155)
at 
org.apache.solr.analytics.AbstractAnalyticsStatsCloudTest.setupCluster(AbstractAnalyticsStatsCloudTest.java:76)
at 
org.apache.solr.analytics.NoFacetCloudTest.populate(NoFacetCloudTest.java:62)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:564)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:874)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.base/java.lang.Thread.run(Thread.java:844)




Build Log:
[...truncated 16742 lines...]
   [junit4] Suite: org.apache.solr.analytics.NoFacetCloudTest
   [junit4]   2> Creating dataDir: 
C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\solr\build\contrib\solr-analytics\test\J0\temp\solr.analytics.NoFacetCloudTest_B6EC1EC7F28FDFC2-001\init-core-data-001
   [junit4]   2> Sep 22, 2017 1:08:06 AM 
com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 1 leaked 
thread(s).
   [junit4]   2> NOTE: test params are: codec=Lucene70, 
sim=RandomSimilarity(queryNorm=false): {}, locale=en-SB, timezone=Africa/Lome
   [junit4]   2> NOTE: Windows 10 10.0 x86/Oracle Corporation 9 
(32-bit)/cpus=3,threads=1,free=13484760,total=49827840
   [junit4]   2> NOTE: All tests run in this JVM: [RangeFacetTest, 
FacetSortingTest, AbstractAnalyticsStatsCloudTest, NoFacetCloudTest]
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=NoFacetCloudTest 
-Dtests.seed=B6EC1EC7F28FDFC2 -Dtests.slow=true -Dtests.locale=en-SB 
-Dtests.timezone=Africa/Lome -Dtests.asserts=true 
-Dtests.file.encoding=ISO-8859-1
   [junit4] ERROR   0.00s J0 | NoFacetCloudTest (suite) <<<
   [junit4]> Throwable #1: java.lang.AssertionError: Could not find 
collection:collection1
   [junit4]>at 
__randomizedtesting.SeedInfo.seed([B6EC1EC7F28FDFC2]:0)
   [junit4]>at 
org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:155)
   [junit4]>at 

[jira] [Commented] (SOLR-11387) testPoissonDistribution fails too frequently

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16175767#comment-16175767
 ] 

ASF subversion and git services commented on SOLR-11387:


Commit 014fbf9e0c04692eba57514519bcbc236217fa3d in lucene-solr's branch 
refs/heads/branch_7x from [~joel.bernstein]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=014fbf9 ]

SOLR-11387: testPoissonDistribution fails too frequently


> testPoissonDistribution fails too frequently
> 
>
> Key: SOLR-11387
> URL: https://issues.apache.org/jira/browse/SOLR-11387
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Joel Bernstein
>
> Currently the test case for the Poisson distribution is failing to 
> frequently. Part of the test relies on the mean and standard deviation being 
> within a specific range. But the random sampling done in the test often falls 
> outside of this range. I'll first increase the range and see if that 
> stabilizes the test.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-11387) testPoissonDistribution fails too frequently

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16175766#comment-16175766
 ] 

ASF subversion and git services commented on SOLR-11387:


Commit fc8d7ee1b041fd714dbe10930c73d3eb36a87762 in lucene-solr's branch 
refs/heads/master from [~joel.bernstein]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=fc8d7ee ]

SOLR-11387: testPoissonDistribution fails too frequently


> testPoissonDistribution fails too frequently
> 
>
> Key: SOLR-11387
> URL: https://issues.apache.org/jira/browse/SOLR-11387
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Joel Bernstein
>
> Currently the test case for the Poisson distribution is failing to 
> frequently. Part of the test relies on the mean and standard deviation being 
> within a specific range. But the random sampling done in the test often falls 
> outside of this range. I'll first increase the range and see if that 
> stabilizes the test.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-11387) testPoissonDistribution fails too frequently

[Joel Bernstein (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-11387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joel Bernstein updated SOLR-11387:
--
Summary: testPoissonDistribution fails too frequently  (was: testPoisson 
distribution fails too frequently)

> testPoissonDistribution fails too frequently
> 
>
> Key: SOLR-11387
> URL: https://issues.apache.org/jira/browse/SOLR-11387
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Joel Bernstein
>
> Currently the test case for the Poisson distribution is failing to 
> frequently. Part of the test relies on the mean and standard deviation being 
> within a specific range. But the random sampling done in the test often falls 
> outside of this range. I'll first increase the range and see if that 
> stabilizes the test.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10181) CREATEALIAS and DELETEALIAS commands consistency problems under concurrency

[Erick Erickson (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16175763#comment-16175763
 ] 

Erick Erickson commented on SOLR-10181:
---

Samuel:

I've assigned this JIRA to myself as a method to track it so it doesn't fall 
off my radar. Please work up any patches you have time/desire to and attach 
them, if I start actively working on this I'll post a notice so we don't 
duplicate efforts. That said, I don't anticipate having much time in the near 
future so fire away!

> CREATEALIAS and DELETEALIAS commands consistency problems under concurrency
> ---
>
> Key: SOLR-10181
> URL: https://issues.apache.org/jira/browse/SOLR-10181
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Affects Versions: 5.3, 5.4, 5.5, 6.4.1
>Reporter: Samuel García Martínez
>Assignee: Erick Erickson
> Attachments: SOLR-10181_testcase.patch
>
>
> When several CREATEALIAS are run at the same time by the OCP it could happen 
> that, even tho the API response is OK, some of those CREATEALIAS request 
> changes are lost.
> h3. The problem
> The problem happens because the CREATEALIAS cmd implementation relies on 
> _zkStateReader.getAliases()_ to create the map that will be stored in ZK. If 
> several threads reach that line at the same time it will happen that only one 
> will be stored correctly and the others will be overridden.
> The code I'm referencing is [this 
> piece|https://github.com/apache/lucene-solr/blob/8c1e67e30e071ceed636083532d4598bf6a8791f/solr/core/src/java/org/apache/solr/cloud/CreateAliasCmd.java#L65].
>  As an example, let's say that the current aliases map has {a:colA, b:colB}. 
> If two CREATEALIAS (one adding c:colC and other creating d:colD) are 
> submitted to the _tpe_ and reach that line at the same time, the resulting 
> maps will look like {a:colA, b:colB, c:colC} and {a:colA, b:colB, d:colD} and 
> only one of them will be stored correctly in ZK, resulting in "data loss", 
> meaning that API is returning OK despite that it didn't work as expected.
> On top of this, another concurrency problem could happen when the command 
> checks if the alias has been set using _checkForAlias_ method. if these two 
> CREATEALIAS zk writes had ran at the same time, the alias check fir one of 
> the threads can timeout since only one of the writes has "survived" and has 
> been "committed" to the _zkStateReader.getAliases()_ map.
> h3. How to fix it
> I can post a patch to this if someone gives me directions on how it should be 
> fixed. As I see this, there are two places where the issue can be fixed: in 
> the processor (OverseerCollectionMessageHandler) in a generic way or inside 
> the command itself.
> h5. The processor fix
> The locking mechanism (_OverseerCollectionMessageHandler#lockTask_) should be 
> the place to fix this inside the processor. I thought that adding the 
> operation name instead of only "collection" or "name" to the locking key 
> would fix the issue, but I realized that the problem will happen anyway if 
> the concurrency happens between different operations modifying the same 
> resource (like CREATEALIAS and DELETEALIAS do). So, if this should be the 
> path to follow I don't know what should be used as a locking key.
> h5. The command fix
> Fixing it at the command level (_CreateAliasCmd_ and _DeleteAliasCmd_) would 
> be relatively easy. Using optimistic locking, i.e, using the aliases.json zk 
> version in the keeper.setData. To do that, Aliases class should offer the 
> aliases version so the commands can forward that version with the update and 
> retry when it fails.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-11387) testPoisson distribution fails too frequently

[Joel Bernstein (JIRA)]

Joel Bernstein created SOLR-11387:
-

 Summary: testPoisson distribution fails too frequently
 Key: SOLR-11387
 URL: https://issues.apache.org/jira/browse/SOLR-11387
 Project: Solr
  Issue Type: Bug
  Security Level: Public (Default Security Level. Issues are Public)
Reporter: Joel Bernstein


Currently the test case for the Poisson distribution is failing to frequently. 
Part of the test relies on the mean and standard deviation being within a 
specific range. But the random sampling done in the test often falls outside of 
this range. I'll first increase the range and see if that stabilizes the test.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10651) Streaming Expressions statistical functions library

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16175750#comment-16175750
 ] 

ASF subversion and git services commented on SOLR-10651:


Commit b6c7af10fe7aefb060713d0b6da8d6e74d90fb0b in lucene-solr's branch 
refs/heads/branch_7_0 from [~joel.bernstein]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=b6c7af1 ]

SOLR-10651: fix docs ...


> Streaming Expressions statistical functions library
> ---
>
> Key: SOLR-10651
> URL: https://issues.apache.org/jira/browse/SOLR-10651
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: streaming expressions
>Reporter: Joel Bernstein
>
> This is a ticket for organizing the new statistical programming features of 
> Streaming Expressions. It's also a place for the community to discuss what 
> functions are needed to support statistical programming. 
> Basic Syntax:
> {code}
> let(a = timeseries(...),
> b = timeseries(...),
> c = col(a, count(*)),
> d = col(b, count(*)),
> r = regress(c, d),
> tuple(p = predict(r, 50)))
> {code}
> The expression above is doing the following:
> 1) The let expression is setting variables (a, b, c, d, r).
> 2) Variables *a* and *b* are the output of timeseries() Streaming 
> Expressions. These will be stored in memory as lists of Tuples containing the 
> time series results.
> 3) Variables *c* and *d* are set using the *col* evaluator. The col evaluator 
> extracts a column of numbers from a list of tuples. In the example *col* is 
> extracting the count\(*\) field from the two time series result sets.
> 4) Variable *r* is the output from the *regress* evaluator. The regress 
> evaluator performs a simple regression analysis on two columns of numbers.
> 5) Once the variables are set, a single Streaming Expression is run by the 
> *let* expression. In the example the *tuple* expression is run. The tuple 
> expression outputs a single Tuple with name/value pairs. Any Streaming 
> Expression can be run by the *let* expression so this can be a complex 
> program. The streaming expression run by *let* has access to all the 
> variables defined earlier.
> 6) The tuple expression in the example has one name / value pair. The name 
> *p* is set to the output of the *predict* evaluator. The predict evaluator is 
> predicting the value of a dependent variable based on the independent 
> variable 50. The regression result stored in variable *r* is used to make the 
> prediction.
> 7) The output of this expression will be a single tuple with the value of the 
> predict function in the *p* field.
> The growing list of issues linked to this ticket are the array manipulation 
> and statistical functions that will form the basis of the stats library. The 
> vast majority of these functions are backed by algorithms in Apache Commons 
> Math. Other machine learning and math libraries will follow.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10651) Streaming Expressions statistical functions library

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16175746#comment-16175746
 ] 

ASF subversion and git services commented on SOLR-10651:


Commit fb4ff0b30ea249811abbd214725953fb52e4552f in lucene-solr's branch 
refs/heads/branch_7_0 from [~joel.bernstein]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=fb4ff0b ]

SOLR-10651: fix docs ...


> Streaming Expressions statistical functions library
> ---
>
> Key: SOLR-10651
> URL: https://issues.apache.org/jira/browse/SOLR-10651
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: streaming expressions
>Reporter: Joel Bernstein
>
> This is a ticket for organizing the new statistical programming features of 
> Streaming Expressions. It's also a place for the community to discuss what 
> functions are needed to support statistical programming. 
> Basic Syntax:
> {code}
> let(a = timeseries(...),
> b = timeseries(...),
> c = col(a, count(*)),
> d = col(b, count(*)),
> r = regress(c, d),
> tuple(p = predict(r, 50)))
> {code}
> The expression above is doing the following:
> 1) The let expression is setting variables (a, b, c, d, r).
> 2) Variables *a* and *b* are the output of timeseries() Streaming 
> Expressions. These will be stored in memory as lists of Tuples containing the 
> time series results.
> 3) Variables *c* and *d* are set using the *col* evaluator. The col evaluator 
> extracts a column of numbers from a list of tuples. In the example *col* is 
> extracting the count\(*\) field from the two time series result sets.
> 4) Variable *r* is the output from the *regress* evaluator. The regress 
> evaluator performs a simple regression analysis on two columns of numbers.
> 5) Once the variables are set, a single Streaming Expression is run by the 
> *let* expression. In the example the *tuple* expression is run. The tuple 
> expression outputs a single Tuple with name/value pairs. Any Streaming 
> Expression can be run by the *let* expression so this can be a complex 
> program. The streaming expression run by *let* has access to all the 
> variables defined earlier.
> 6) The tuple expression in the example has one name / value pair. The name 
> *p* is set to the output of the *predict* evaluator. The predict evaluator is 
> predicting the value of a dependent variable based on the independent 
> variable 50. The regression result stored in variable *r* is used to make the 
> prediction.
> 7) The output of this expression will be a single tuple with the value of the 
> predict function in the *p* field.
> The growing list of issues linked to this ticket are the array manipulation 
> and statistical functions that will form the basis of the stats library. The 
> vast majority of these functions are backed by algorithms in Apache Commons 
> Math. Other machine learning and math libraries will follow.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10651) Streaming Expressions statistical functions library

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16175749#comment-16175749
 ] 

ASF subversion and git services commented on SOLR-10651:


Commit 9521a29e71765be30dc6043ece4f6f3fc058ee31 in lucene-solr's branch 
refs/heads/branch_7_0 from [~joel.bernstein]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=9521a29 ]

SOLR-10651: fix docs ...


> Streaming Expressions statistical functions library
> ---
>
> Key: SOLR-10651
> URL: https://issues.apache.org/jira/browse/SOLR-10651
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: streaming expressions
>Reporter: Joel Bernstein
>
> This is a ticket for organizing the new statistical programming features of 
> Streaming Expressions. It's also a place for the community to discuss what 
> functions are needed to support statistical programming. 
> Basic Syntax:
> {code}
> let(a = timeseries(...),
> b = timeseries(...),
> c = col(a, count(*)),
> d = col(b, count(*)),
> r = regress(c, d),
> tuple(p = predict(r, 50)))
> {code}
> The expression above is doing the following:
> 1) The let expression is setting variables (a, b, c, d, r).
> 2) Variables *a* and *b* are the output of timeseries() Streaming 
> Expressions. These will be stored in memory as lists of Tuples containing the 
> time series results.
> 3) Variables *c* and *d* are set using the *col* evaluator. The col evaluator 
> extracts a column of numbers from a list of tuples. In the example *col* is 
> extracting the count\(*\) field from the two time series result sets.
> 4) Variable *r* is the output from the *regress* evaluator. The regress 
> evaluator performs a simple regression analysis on two columns of numbers.
> 5) Once the variables are set, a single Streaming Expression is run by the 
> *let* expression. In the example the *tuple* expression is run. The tuple 
> expression outputs a single Tuple with name/value pairs. Any Streaming 
> Expression can be run by the *let* expression so this can be a complex 
> program. The streaming expression run by *let* has access to all the 
> variables defined earlier.
> 6) The tuple expression in the example has one name / value pair. The name 
> *p* is set to the output of the *predict* evaluator. The predict evaluator is 
> predicting the value of a dependent variable based on the independent 
> variable 50. The regression result stored in variable *r* is used to make the 
> prediction.
> 7) The output of this expression will be a single tuple with the value of the 
> predict function in the *p* field.
> The growing list of issues linked to this ticket are the array manipulation 
> and statistical functions that will form the basis of the stats library. The 
> vast majority of these functions are backed by algorithms in Apache Commons 
> Math. Other machine learning and math libraries will follow.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7970) Improve generation of circle plane

[Karl Wright (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16175752#comment-16175752
 ] 

Karl Wright commented on LUCENE-7970:
-

My initial observations:

(1) The "inner" and "outer" planes are in fact parallel to one another -- so we 
would want to compute first the one that goes through the upper latitude point 
and the lower latitude point, and then compute an "adjusted" parallel plane.  
The problem is how to figure out the adjustment.  This is what I still need to 
consider.  One way to do it is to find a point-on-bearing to the left or right 
and pick the second plane's D coefficient based on that point.  I think this 
would work quite well actually.
(2) The choice of which plane is "inner" and which plane is "outer" depends on 
whether c > ab, and on whether the radius of the circle is greater than 180 
degrees.  Need to think this through too.


> Improve generation of circle plane 
> ---
>
> Key: LUCENE-7970
> URL: https://issues.apache.org/jira/browse/LUCENE-7970
> Project: Lucene - Core
>  Issue Type: Improvement
>  Components: modules/spatial3d
>Reporter: Ignacio Vera
>Assignee: Karl Wright
> Attachments: LUCENE_7970.patch, LUCENE-7970.patch, 
> LUCENE-7970-proposed.patch
>
>
> Hi [~Karl wright],
> How circles are currently build do not behave very well when the planet model 
> is not an sphere. when you are close to the border in WGS84 you might get 
> false positves or false negatives when checking if a point is WITHIN. I think 
> the reason is how the points to generate the circle plane are generated which 
> assumes a sphere.
> My proposal is the following:
> Add a new method to PlanetModel:  
> public GeoPoint pointOnBearing(GeoPoint from, double dist, double bearing);
> Which uses and algorithm that takes into account that the planet might not be 
> spherical. For example Vincenty's formulae 
> (https://en.wikipedia.org/wiki/Vincenty%27s_formulae).
> Use this method to generate the points for the circle plane. My experiments 
> shows that this approach removes false negatives in WGS84 meanwhile it works 
> nicely in the Sphere.
> Does it make sense?



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10651) Streaming Expressions statistical functions library

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16175742#comment-16175742
 ] 

ASF subversion and git services commented on SOLR-10651:


Commit 6b1e292c5ca412f2ed9f8f4da4ce57f8f2732b0b in lucene-solr's branch 
refs/heads/branch_7_0 from [~joel.bernstein]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=6b1e292 ]

SOLR-10651: Statistical function docs for 7.0 Part 2


> Streaming Expressions statistical functions library
> ---
>
> Key: SOLR-10651
> URL: https://issues.apache.org/jira/browse/SOLR-10651
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: streaming expressions
>Reporter: Joel Bernstein
>
> This is a ticket for organizing the new statistical programming features of 
> Streaming Expressions. It's also a place for the community to discuss what 
> functions are needed to support statistical programming. 
> Basic Syntax:
> {code}
> let(a = timeseries(...),
> b = timeseries(...),
> c = col(a, count(*)),
> d = col(b, count(*)),
> r = regress(c, d),
> tuple(p = predict(r, 50)))
> {code}
> The expression above is doing the following:
> 1) The let expression is setting variables (a, b, c, d, r).
> 2) Variables *a* and *b* are the output of timeseries() Streaming 
> Expressions. These will be stored in memory as lists of Tuples containing the 
> time series results.
> 3) Variables *c* and *d* are set using the *col* evaluator. The col evaluator 
> extracts a column of numbers from a list of tuples. In the example *col* is 
> extracting the count\(*\) field from the two time series result sets.
> 4) Variable *r* is the output from the *regress* evaluator. The regress 
> evaluator performs a simple regression analysis on two columns of numbers.
> 5) Once the variables are set, a single Streaming Expression is run by the 
> *let* expression. In the example the *tuple* expression is run. The tuple 
> expression outputs a single Tuple with name/value pairs. Any Streaming 
> Expression can be run by the *let* expression so this can be a complex 
> program. The streaming expression run by *let* has access to all the 
> variables defined earlier.
> 6) The tuple expression in the example has one name / value pair. The name 
> *p* is set to the output of the *predict* evaluator. The predict evaluator is 
> predicting the value of a dependent variable based on the independent 
> variable 50. The regression result stored in variable *r* is used to make the 
> prediction.
> 7) The output of this expression will be a single tuple with the value of the 
> predict function in the *p* field.
> The growing list of issues linked to this ticket are the array manipulation 
> and statistical functions that will form the basis of the stats library. The 
> vast majority of these functions are backed by algorithms in Apache Commons 
> Math. Other machine learning and math libraries will follow.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10651) Streaming Expressions statistical functions library

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16175747#comment-16175747
 ] 

ASF subversion and git services commented on SOLR-10651:


Commit 4c72691de4a4df6263fff9e04a899c6f93969e99 in lucene-solr's branch 
refs/heads/branch_7_0 from [~joel.bernstein]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=4c72691 ]

SOLR-10651: fix docs ...


> Streaming Expressions statistical functions library
> ---
>
> Key: SOLR-10651
> URL: https://issues.apache.org/jira/browse/SOLR-10651
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: streaming expressions
>Reporter: Joel Bernstein
>
> This is a ticket for organizing the new statistical programming features of 
> Streaming Expressions. It's also a place for the community to discuss what 
> functions are needed to support statistical programming. 
> Basic Syntax:
> {code}
> let(a = timeseries(...),
> b = timeseries(...),
> c = col(a, count(*)),
> d = col(b, count(*)),
> r = regress(c, d),
> tuple(p = predict(r, 50)))
> {code}
> The expression above is doing the following:
> 1) The let expression is setting variables (a, b, c, d, r).
> 2) Variables *a* and *b* are the output of timeseries() Streaming 
> Expressions. These will be stored in memory as lists of Tuples containing the 
> time series results.
> 3) Variables *c* and *d* are set using the *col* evaluator. The col evaluator 
> extracts a column of numbers from a list of tuples. In the example *col* is 
> extracting the count\(*\) field from the two time series result sets.
> 4) Variable *r* is the output from the *regress* evaluator. The regress 
> evaluator performs a simple regression analysis on two columns of numbers.
> 5) Once the variables are set, a single Streaming Expression is run by the 
> *let* expression. In the example the *tuple* expression is run. The tuple 
> expression outputs a single Tuple with name/value pairs. Any Streaming 
> Expression can be run by the *let* expression so this can be a complex 
> program. The streaming expression run by *let* has access to all the 
> variables defined earlier.
> 6) The tuple expression in the example has one name / value pair. The name 
> *p* is set to the output of the *predict* evaluator. The predict evaluator is 
> predicting the value of a dependent variable based on the independent 
> variable 50. The regression result stored in variable *r* is used to make the 
> prediction.
> 7) The output of this expression will be a single tuple with the value of the 
> predict function in the *p* field.
> The growing list of issues linked to this ticket are the array manipulation 
> and statistical functions that will form the basis of the stats library. The 
> vast majority of these functions are backed by algorithms in Apache Commons 
> Math. Other machine learning and math libraries will follow.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10651) Streaming Expressions statistical functions library

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16175743#comment-16175743
 ] 

ASF subversion and git services commented on SOLR-10651:


Commit e8dbf2f21307d36283cea6c3c3afe521a84de20a in lucene-solr's branch 
refs/heads/branch_7_0 from [~joel.bernstein]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=e8dbf2f ]

SOLR-10651: Statistical function docs for 7.0 Part 3


> Streaming Expressions statistical functions library
> ---
>
> Key: SOLR-10651
> URL: https://issues.apache.org/jira/browse/SOLR-10651
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: streaming expressions
>Reporter: Joel Bernstein
>
> This is a ticket for organizing the new statistical programming features of 
> Streaming Expressions. It's also a place for the community to discuss what 
> functions are needed to support statistical programming. 
> Basic Syntax:
> {code}
> let(a = timeseries(...),
> b = timeseries(...),
> c = col(a, count(*)),
> d = col(b, count(*)),
> r = regress(c, d),
> tuple(p = predict(r, 50)))
> {code}
> The expression above is doing the following:
> 1) The let expression is setting variables (a, b, c, d, r).
> 2) Variables *a* and *b* are the output of timeseries() Streaming 
> Expressions. These will be stored in memory as lists of Tuples containing the 
> time series results.
> 3) Variables *c* and *d* are set using the *col* evaluator. The col evaluator 
> extracts a column of numbers from a list of tuples. In the example *col* is 
> extracting the count\(*\) field from the two time series result sets.
> 4) Variable *r* is the output from the *regress* evaluator. The regress 
> evaluator performs a simple regression analysis on two columns of numbers.
> 5) Once the variables are set, a single Streaming Expression is run by the 
> *let* expression. In the example the *tuple* expression is run. The tuple 
> expression outputs a single Tuple with name/value pairs. Any Streaming 
> Expression can be run by the *let* expression so this can be a complex 
> program. The streaming expression run by *let* has access to all the 
> variables defined earlier.
> 6) The tuple expression in the example has one name / value pair. The name 
> *p* is set to the output of the *predict* evaluator. The predict evaluator is 
> predicting the value of a dependent variable based on the independent 
> variable 50. The regression result stored in variable *r* is used to make the 
> prediction.
> 7) The output of this expression will be a single tuple with the value of the 
> predict function in the *p* field.
> The growing list of issues linked to this ticket are the array manipulation 
> and statistical functions that will form the basis of the stats library. The 
> vast majority of these functions are backed by algorithms in Apache Commons 
> Math. Other machine learning and math libraries will follow.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10651) Streaming Expressions statistical functions library

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16175745#comment-16175745
 ] 

ASF subversion and git services commented on SOLR-10651:


Commit e81d047f6bebf3c223332ecc9beae41243b925a6 in lucene-solr's branch 
refs/heads/branch_7_0 from [~joel.bernstein]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=e81d047 ]

SOLR-10651: fix docs ...


> Streaming Expressions statistical functions library
> ---
>
> Key: SOLR-10651
> URL: https://issues.apache.org/jira/browse/SOLR-10651
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: streaming expressions
>Reporter: Joel Bernstein
>
> This is a ticket for organizing the new statistical programming features of 
> Streaming Expressions. It's also a place for the community to discuss what 
> functions are needed to support statistical programming. 
> Basic Syntax:
> {code}
> let(a = timeseries(...),
> b = timeseries(...),
> c = col(a, count(*)),
> d = col(b, count(*)),
> r = regress(c, d),
> tuple(p = predict(r, 50)))
> {code}
> The expression above is doing the following:
> 1) The let expression is setting variables (a, b, c, d, r).
> 2) Variables *a* and *b* are the output of timeseries() Streaming 
> Expressions. These will be stored in memory as lists of Tuples containing the 
> time series results.
> 3) Variables *c* and *d* are set using the *col* evaluator. The col evaluator 
> extracts a column of numbers from a list of tuples. In the example *col* is 
> extracting the count\(*\) field from the two time series result sets.
> 4) Variable *r* is the output from the *regress* evaluator. The regress 
> evaluator performs a simple regression analysis on two columns of numbers.
> 5) Once the variables are set, a single Streaming Expression is run by the 
> *let* expression. In the example the *tuple* expression is run. The tuple 
> expression outputs a single Tuple with name/value pairs. Any Streaming 
> Expression can be run by the *let* expression so this can be a complex 
> program. The streaming expression run by *let* has access to all the 
> variables defined earlier.
> 6) The tuple expression in the example has one name / value pair. The name 
> *p* is set to the output of the *predict* evaluator. The predict evaluator is 
> predicting the value of a dependent variable based on the independent 
> variable 50. The regression result stored in variable *r* is used to make the 
> prediction.
> 7) The output of this expression will be a single tuple with the value of the 
> predict function in the *p* field.
> The growing list of issues linked to this ticket are the array manipulation 
> and statistical functions that will form the basis of the stats library. The 
> vast majority of these functions are backed by algorithms in Apache Commons 
> Math. Other machine learning and math libraries will follow.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10651) Streaming Expressions statistical functions library

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16175751#comment-16175751
 ] 

ASF subversion and git services commented on SOLR-10651:


Commit 38105b4a649c33f339133d4711798056baf3e45b in lucene-solr's branch 
refs/heads/branch_7_0 from [~joel.bernstein]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=38105b4 ]

SOLR-10651: fix docs ...


> Streaming Expressions statistical functions library
> ---
>
> Key: SOLR-10651
> URL: https://issues.apache.org/jira/browse/SOLR-10651
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: streaming expressions
>Reporter: Joel Bernstein
>
> This is a ticket for organizing the new statistical programming features of 
> Streaming Expressions. It's also a place for the community to discuss what 
> functions are needed to support statistical programming. 
> Basic Syntax:
> {code}
> let(a = timeseries(...),
> b = timeseries(...),
> c = col(a, count(*)),
> d = col(b, count(*)),
> r = regress(c, d),
> tuple(p = predict(r, 50)))
> {code}
> The expression above is doing the following:
> 1) The let expression is setting variables (a, b, c, d, r).
> 2) Variables *a* and *b* are the output of timeseries() Streaming 
> Expressions. These will be stored in memory as lists of Tuples containing the 
> time series results.
> 3) Variables *c* and *d* are set using the *col* evaluator. The col evaluator 
> extracts a column of numbers from a list of tuples. In the example *col* is 
> extracting the count\(*\) field from the two time series result sets.
> 4) Variable *r* is the output from the *regress* evaluator. The regress 
> evaluator performs a simple regression analysis on two columns of numbers.
> 5) Once the variables are set, a single Streaming Expression is run by the 
> *let* expression. In the example the *tuple* expression is run. The tuple 
> expression outputs a single Tuple with name/value pairs. Any Streaming 
> Expression can be run by the *let* expression so this can be a complex 
> program. The streaming expression run by *let* has access to all the 
> variables defined earlier.
> 6) The tuple expression in the example has one name / value pair. The name 
> *p* is set to the output of the *predict* evaluator. The predict evaluator is 
> predicting the value of a dependent variable based on the independent 
> variable 50. The regression result stored in variable *r* is used to make the 
> prediction.
> 7) The output of this expression will be a single tuple with the value of the 
> predict function in the *p* field.
> The growing list of issues linked to this ticket are the array manipulation 
> and statistical functions that will form the basis of the stats library. The 
> vast majority of these functions are backed by algorithms in Apache Commons 
> Math. Other machine learning and math libraries will follow.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10651) Streaming Expressions statistical functions library

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16175744#comment-16175744
 ] 

ASF subversion and git services commented on SOLR-10651:


Commit 1f46c71c20fe1ece5909626528ac879907b4e1af in lucene-solr's branch 
refs/heads/branch_7_0 from [~joel.bernstein]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=1f46c71 ]

SOLR-10651: fix docs


> Streaming Expressions statistical functions library
> ---
>
> Key: SOLR-10651
> URL: https://issues.apache.org/jira/browse/SOLR-10651
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: streaming expressions
>Reporter: Joel Bernstein
>
> This is a ticket for organizing the new statistical programming features of 
> Streaming Expressions. It's also a place for the community to discuss what 
> functions are needed to support statistical programming. 
> Basic Syntax:
> {code}
> let(a = timeseries(...),
> b = timeseries(...),
> c = col(a, count(*)),
> d = col(b, count(*)),
> r = regress(c, d),
> tuple(p = predict(r, 50)))
> {code}
> The expression above is doing the following:
> 1) The let expression is setting variables (a, b, c, d, r).
> 2) Variables *a* and *b* are the output of timeseries() Streaming 
> Expressions. These will be stored in memory as lists of Tuples containing the 
> time series results.
> 3) Variables *c* and *d* are set using the *col* evaluator. The col evaluator 
> extracts a column of numbers from a list of tuples. In the example *col* is 
> extracting the count\(*\) field from the two time series result sets.
> 4) Variable *r* is the output from the *regress* evaluator. The regress 
> evaluator performs a simple regression analysis on two columns of numbers.
> 5) Once the variables are set, a single Streaming Expression is run by the 
> *let* expression. In the example the *tuple* expression is run. The tuple 
> expression outputs a single Tuple with name/value pairs. Any Streaming 
> Expression can be run by the *let* expression so this can be a complex 
> program. The streaming expression run by *let* has access to all the 
> variables defined earlier.
> 6) The tuple expression in the example has one name / value pair. The name 
> *p* is set to the output of the *predict* evaluator. The predict evaluator is 
> predicting the value of a dependent variable based on the independent 
> variable 50. The regression result stored in variable *r* is used to make the 
> prediction.
> 7) The output of this expression will be a single tuple with the value of the 
> predict function in the *p* field.
> The growing list of issues linked to this ticket are the array manipulation 
> and statistical functions that will form the basis of the stats library. The 
> vast majority of these functions are backed by algorithms in Apache Commons 
> Math. Other machine learning and math libraries will follow.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10651) Streaming Expressions statistical functions library

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16175731#comment-16175731
 ] 

ASF subversion and git services commented on SOLR-10651:


Commit a9aa94a2806eeec474389c1afb5fe52f367ca1ea in lucene-solr's branch 
refs/heads/branch_7x from [~joel.bernstein]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=a9aa94a ]

SOLR-10651: fix docs ...


> Streaming Expressions statistical functions library
> ---
>
> Key: SOLR-10651
> URL: https://issues.apache.org/jira/browse/SOLR-10651
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: streaming expressions
>Reporter: Joel Bernstein
>
> This is a ticket for organizing the new statistical programming features of 
> Streaming Expressions. It's also a place for the community to discuss what 
> functions are needed to support statistical programming. 
> Basic Syntax:
> {code}
> let(a = timeseries(...),
> b = timeseries(...),
> c = col(a, count(*)),
> d = col(b, count(*)),
> r = regress(c, d),
> tuple(p = predict(r, 50)))
> {code}
> The expression above is doing the following:
> 1) The let expression is setting variables (a, b, c, d, r).
> 2) Variables *a* and *b* are the output of timeseries() Streaming 
> Expressions. These will be stored in memory as lists of Tuples containing the 
> time series results.
> 3) Variables *c* and *d* are set using the *col* evaluator. The col evaluator 
> extracts a column of numbers from a list of tuples. In the example *col* is 
> extracting the count\(*\) field from the two time series result sets.
> 4) Variable *r* is the output from the *regress* evaluator. The regress 
> evaluator performs a simple regression analysis on two columns of numbers.
> 5) Once the variables are set, a single Streaming Expression is run by the 
> *let* expression. In the example the *tuple* expression is run. The tuple 
> expression outputs a single Tuple with name/value pairs. Any Streaming 
> Expression can be run by the *let* expression so this can be a complex 
> program. The streaming expression run by *let* has access to all the 
> variables defined earlier.
> 6) The tuple expression in the example has one name / value pair. The name 
> *p* is set to the output of the *predict* evaluator. The predict evaluator is 
> predicting the value of a dependent variable based on the independent 
> variable 50. The regression result stored in variable *r* is used to make the 
> prediction.
> 7) The output of this expression will be a single tuple with the value of the 
> predict function in the *p* field.
> The growing list of issues linked to this ticket are the array manipulation 
> and statistical functions that will form the basis of the stats library. The 
> vast majority of these functions are backed by algorithms in Apache Commons 
> Math. Other machine learning and math libraries will follow.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10651) Streaming Expressions statistical functions library

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16175732#comment-16175732
 ] 

ASF subversion and git services commented on SOLR-10651:


Commit d62998facc0f2136e4c8da0ffa9e6f8e1dfad1a9 in lucene-solr's branch 
refs/heads/branch_7x from [~joel.bernstein]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=d62998fa ]

SOLR-10651: fix docs ...


> Streaming Expressions statistical functions library
> ---
>
> Key: SOLR-10651
> URL: https://issues.apache.org/jira/browse/SOLR-10651
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: streaming expressions
>Reporter: Joel Bernstein
>
> This is a ticket for organizing the new statistical programming features of 
> Streaming Expressions. It's also a place for the community to discuss what 
> functions are needed to support statistical programming. 
> Basic Syntax:
> {code}
> let(a = timeseries(...),
> b = timeseries(...),
> c = col(a, count(*)),
> d = col(b, count(*)),
> r = regress(c, d),
> tuple(p = predict(r, 50)))
> {code}
> The expression above is doing the following:
> 1) The let expression is setting variables (a, b, c, d, r).
> 2) Variables *a* and *b* are the output of timeseries() Streaming 
> Expressions. These will be stored in memory as lists of Tuples containing the 
> time series results.
> 3) Variables *c* and *d* are set using the *col* evaluator. The col evaluator 
> extracts a column of numbers from a list of tuples. In the example *col* is 
> extracting the count\(*\) field from the two time series result sets.
> 4) Variable *r* is the output from the *regress* evaluator. The regress 
> evaluator performs a simple regression analysis on two columns of numbers.
> 5) Once the variables are set, a single Streaming Expression is run by the 
> *let* expression. In the example the *tuple* expression is run. The tuple 
> expression outputs a single Tuple with name/value pairs. Any Streaming 
> Expression can be run by the *let* expression so this can be a complex 
> program. The streaming expression run by *let* has access to all the 
> variables defined earlier.
> 6) The tuple expression in the example has one name / value pair. The name 
> *p* is set to the output of the *predict* evaluator. The predict evaluator is 
> predicting the value of a dependent variable based on the independent 
> variable 50. The regression result stored in variable *r* is used to make the 
> prediction.
> 7) The output of this expression will be a single tuple with the value of the 
> predict function in the *p* field.
> The growing list of issues linked to this ticket are the array manipulation 
> and statistical functions that will form the basis of the stats library. The 
> vast majority of these functions are backed by algorithms in Apache Commons 
> Math. Other machine learning and math libraries will follow.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10651) Streaming Expressions statistical functions library

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16175735#comment-16175735
 ] 

ASF subversion and git services commented on SOLR-10651:


Commit 52b26695e62d3c6a658b74b8458c16a994494599 in lucene-solr's branch 
refs/heads/branch_7x from [~joel.bernstein]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=52b2669 ]

SOLR-10651: fix docs ...


> Streaming Expressions statistical functions library
> ---
>
> Key: SOLR-10651
> URL: https://issues.apache.org/jira/browse/SOLR-10651
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: streaming expressions
>Reporter: Joel Bernstein
>
> This is a ticket for organizing the new statistical programming features of 
> Streaming Expressions. It's also a place for the community to discuss what 
> functions are needed to support statistical programming. 
> Basic Syntax:
> {code}
> let(a = timeseries(...),
> b = timeseries(...),
> c = col(a, count(*)),
> d = col(b, count(*)),
> r = regress(c, d),
> tuple(p = predict(r, 50)))
> {code}
> The expression above is doing the following:
> 1) The let expression is setting variables (a, b, c, d, r).
> 2) Variables *a* and *b* are the output of timeseries() Streaming 
> Expressions. These will be stored in memory as lists of Tuples containing the 
> time series results.
> 3) Variables *c* and *d* are set using the *col* evaluator. The col evaluator 
> extracts a column of numbers from a list of tuples. In the example *col* is 
> extracting the count\(*\) field from the two time series result sets.
> 4) Variable *r* is the output from the *regress* evaluator. The regress 
> evaluator performs a simple regression analysis on two columns of numbers.
> 5) Once the variables are set, a single Streaming Expression is run by the 
> *let* expression. In the example the *tuple* expression is run. The tuple 
> expression outputs a single Tuple with name/value pairs. Any Streaming 
> Expression can be run by the *let* expression so this can be a complex 
> program. The streaming expression run by *let* has access to all the 
> variables defined earlier.
> 6) The tuple expression in the example has one name / value pair. The name 
> *p* is set to the output of the *predict* evaluator. The predict evaluator is 
> predicting the value of a dependent variable based on the independent 
> variable 50. The regression result stored in variable *r* is used to make the 
> prediction.
> 7) The output of this expression will be a single tuple with the value of the 
> predict function in the *p* field.
> The growing list of issues linked to this ticket are the array manipulation 
> and statistical functions that will form the basis of the stats library. The 
> vast majority of these functions are backed by algorithms in Apache Commons 
> Math. Other machine learning and math libraries will follow.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10651) Streaming Expressions statistical functions library

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16175734#comment-16175734
 ] 

ASF subversion and git services commented on SOLR-10651:


Commit 88181f44ebf4112afe256be3a0d83b22d54d16b0 in lucene-solr's branch 
refs/heads/branch_7x from [~joel.bernstein]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=88181f4 ]

SOLR-10651: fix docs ...


> Streaming Expressions statistical functions library
> ---
>
> Key: SOLR-10651
> URL: https://issues.apache.org/jira/browse/SOLR-10651
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: streaming expressions
>Reporter: Joel Bernstein
>
> This is a ticket for organizing the new statistical programming features of 
> Streaming Expressions. It's also a place for the community to discuss what 
> functions are needed to support statistical programming. 
> Basic Syntax:
> {code}
> let(a = timeseries(...),
> b = timeseries(...),
> c = col(a, count(*)),
> d = col(b, count(*)),
> r = regress(c, d),
> tuple(p = predict(r, 50)))
> {code}
> The expression above is doing the following:
> 1) The let expression is setting variables (a, b, c, d, r).
> 2) Variables *a* and *b* are the output of timeseries() Streaming 
> Expressions. These will be stored in memory as lists of Tuples containing the 
> time series results.
> 3) Variables *c* and *d* are set using the *col* evaluator. The col evaluator 
> extracts a column of numbers from a list of tuples. In the example *col* is 
> extracting the count\(*\) field from the two time series result sets.
> 4) Variable *r* is the output from the *regress* evaluator. The regress 
> evaluator performs a simple regression analysis on two columns of numbers.
> 5) Once the variables are set, a single Streaming Expression is run by the 
> *let* expression. In the example the *tuple* expression is run. The tuple 
> expression outputs a single Tuple with name/value pairs. Any Streaming 
> Expression can be run by the *let* expression so this can be a complex 
> program. The streaming expression run by *let* has access to all the 
> variables defined earlier.
> 6) The tuple expression in the example has one name / value pair. The name 
> *p* is set to the output of the *predict* evaluator. The predict evaluator is 
> predicting the value of a dependent variable based on the independent 
> variable 50. The regression result stored in variable *r* is used to make the 
> prediction.
> 7) The output of this expression will be a single tuple with the value of the 
> predict function in the *p* field.
> The growing list of issues linked to this ticket are the array manipulation 
> and statistical functions that will form the basis of the stats library. The 
> vast majority of these functions are backed by algorithms in Apache Commons 
> Math. Other machine learning and math libraries will follow.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10651) Streaming Expressions statistical functions library

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16175729#comment-16175729
 ] 

ASF subversion and git services commented on SOLR-10651:


Commit 34ede83456180f96bab87bcff0bac6f5349b0e7d in lucene-solr's branch 
refs/heads/branch_7x from [~joel.bernstein]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=34ede83 ]

SOLR-10651: fix docs


> Streaming Expressions statistical functions library
> ---
>
> Key: SOLR-10651
> URL: https://issues.apache.org/jira/browse/SOLR-10651
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: streaming expressions
>Reporter: Joel Bernstein
>
> This is a ticket for organizing the new statistical programming features of 
> Streaming Expressions. It's also a place for the community to discuss what 
> functions are needed to support statistical programming. 
> Basic Syntax:
> {code}
> let(a = timeseries(...),
> b = timeseries(...),
> c = col(a, count(*)),
> d = col(b, count(*)),
> r = regress(c, d),
> tuple(p = predict(r, 50)))
> {code}
> The expression above is doing the following:
> 1) The let expression is setting variables (a, b, c, d, r).
> 2) Variables *a* and *b* are the output of timeseries() Streaming 
> Expressions. These will be stored in memory as lists of Tuples containing the 
> time series results.
> 3) Variables *c* and *d* are set using the *col* evaluator. The col evaluator 
> extracts a column of numbers from a list of tuples. In the example *col* is 
> extracting the count\(*\) field from the two time series result sets.
> 4) Variable *r* is the output from the *regress* evaluator. The regress 
> evaluator performs a simple regression analysis on two columns of numbers.
> 5) Once the variables are set, a single Streaming Expression is run by the 
> *let* expression. In the example the *tuple* expression is run. The tuple 
> expression outputs a single Tuple with name/value pairs. Any Streaming 
> Expression can be run by the *let* expression so this can be a complex 
> program. The streaming expression run by *let* has access to all the 
> variables defined earlier.
> 6) The tuple expression in the example has one name / value pair. The name 
> *p* is set to the output of the *predict* evaluator. The predict evaluator is 
> predicting the value of a dependent variable based on the independent 
> variable 50. The regression result stored in variable *r* is used to make the 
> prediction.
> 7) The output of this expression will be a single tuple with the value of the 
> predict function in the *p* field.
> The growing list of issues linked to this ticket are the array manipulation 
> and statistical functions that will form the basis of the stats library. The 
> vast majority of these functions are backed by algorithms in Apache Commons 
> Math. Other machine learning and math libraries will follow.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10651) Streaming Expressions statistical functions library

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16175728#comment-16175728
 ] 

ASF subversion and git services commented on SOLR-10651:


Commit eb0fce17f56c640e3e00117492627080fdce37b5 in lucene-solr's branch 
refs/heads/branch_7x from [~joel.bernstein]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=eb0fce1 ]

SOLR-10651: Statistical function docs for 7.0 Part 3


> Streaming Expressions statistical functions library
> ---
>
> Key: SOLR-10651
> URL: https://issues.apache.org/jira/browse/SOLR-10651
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: streaming expressions
>Reporter: Joel Bernstein
>
> This is a ticket for organizing the new statistical programming features of 
> Streaming Expressions. It's also a place for the community to discuss what 
> functions are needed to support statistical programming. 
> Basic Syntax:
> {code}
> let(a = timeseries(...),
> b = timeseries(...),
> c = col(a, count(*)),
> d = col(b, count(*)),
> r = regress(c, d),
> tuple(p = predict(r, 50)))
> {code}
> The expression above is doing the following:
> 1) The let expression is setting variables (a, b, c, d, r).
> 2) Variables *a* and *b* are the output of timeseries() Streaming 
> Expressions. These will be stored in memory as lists of Tuples containing the 
> time series results.
> 3) Variables *c* and *d* are set using the *col* evaluator. The col evaluator 
> extracts a column of numbers from a list of tuples. In the example *col* is 
> extracting the count\(*\) field from the two time series result sets.
> 4) Variable *r* is the output from the *regress* evaluator. The regress 
> evaluator performs a simple regression analysis on two columns of numbers.
> 5) Once the variables are set, a single Streaming Expression is run by the 
> *let* expression. In the example the *tuple* expression is run. The tuple 
> expression outputs a single Tuple with name/value pairs. Any Streaming 
> Expression can be run by the *let* expression so this can be a complex 
> program. The streaming expression run by *let* has access to all the 
> variables defined earlier.
> 6) The tuple expression in the example has one name / value pair. The name 
> *p* is set to the output of the *predict* evaluator. The predict evaluator is 
> predicting the value of a dependent variable based on the independent 
> variable 50. The regression result stored in variable *r* is used to make the 
> prediction.
> 7) The output of this expression will be a single tuple with the value of the 
> predict function in the *p* field.
> The growing list of issues linked to this ticket are the array manipulation 
> and statistical functions that will form the basis of the stats library. The 
> vast majority of these functions are backed by algorithms in Apache Commons 
> Math. Other machine learning and math libraries will follow.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10651) Streaming Expressions statistical functions library

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16175733#comment-16175733
 ] 

ASF subversion and git services commented on SOLR-10651:


Commit 1336ae566ead80785ef84016f99b1a08eb7c77a2 in lucene-solr's branch 
refs/heads/branch_7x from [~joel.bernstein]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=1336ae5 ]

SOLR-10651: fix docs ...


> Streaming Expressions statistical functions library
> ---
>
> Key: SOLR-10651
> URL: https://issues.apache.org/jira/browse/SOLR-10651
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: streaming expressions
>Reporter: Joel Bernstein
>
> This is a ticket for organizing the new statistical programming features of 
> Streaming Expressions. It's also a place for the community to discuss what 
> functions are needed to support statistical programming. 
> Basic Syntax:
> {code}
> let(a = timeseries(...),
> b = timeseries(...),
> c = col(a, count(*)),
> d = col(b, count(*)),
> r = regress(c, d),
> tuple(p = predict(r, 50)))
> {code}
> The expression above is doing the following:
> 1) The let expression is setting variables (a, b, c, d, r).
> 2) Variables *a* and *b* are the output of timeseries() Streaming 
> Expressions. These will be stored in memory as lists of Tuples containing the 
> time series results.
> 3) Variables *c* and *d* are set using the *col* evaluator. The col evaluator 
> extracts a column of numbers from a list of tuples. In the example *col* is 
> extracting the count\(*\) field from the two time series result sets.
> 4) Variable *r* is the output from the *regress* evaluator. The regress 
> evaluator performs a simple regression analysis on two columns of numbers.
> 5) Once the variables are set, a single Streaming Expression is run by the 
> *let* expression. In the example the *tuple* expression is run. The tuple 
> expression outputs a single Tuple with name/value pairs. Any Streaming 
> Expression can be run by the *let* expression so this can be a complex 
> program. The streaming expression run by *let* has access to all the 
> variables defined earlier.
> 6) The tuple expression in the example has one name / value pair. The name 
> *p* is set to the output of the *predict* evaluator. The predict evaluator is 
> predicting the value of a dependent variable based on the independent 
> variable 50. The regression result stored in variable *r* is used to make the 
> prediction.
> 7) The output of this expression will be a single tuple with the value of the 
> predict function in the *p* field.
> The growing list of issues linked to this ticket are the array manipulation 
> and statistical functions that will form the basis of the stats library. The 
> vast majority of these functions are backed by algorithms in Apache Commons 
> Math. Other machine learning and math libraries will follow.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10651) Streaming Expressions statistical functions library

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16175730#comment-16175730
 ] 

ASF subversion and git services commented on SOLR-10651:


Commit 18f04659d7d59ab498f8ec82ca0b5b53d624d9f9 in lucene-solr's branch 
refs/heads/branch_7x from [~joel.bernstein]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=18f0465 ]

SOLR-10651: fix docs ...


> Streaming Expressions statistical functions library
> ---
>
> Key: SOLR-10651
> URL: https://issues.apache.org/jira/browse/SOLR-10651
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: streaming expressions
>Reporter: Joel Bernstein
>
> This is a ticket for organizing the new statistical programming features of 
> Streaming Expressions. It's also a place for the community to discuss what 
> functions are needed to support statistical programming. 
> Basic Syntax:
> {code}
> let(a = timeseries(...),
> b = timeseries(...),
> c = col(a, count(*)),
> d = col(b, count(*)),
> r = regress(c, d),
> tuple(p = predict(r, 50)))
> {code}
> The expression above is doing the following:
> 1) The let expression is setting variables (a, b, c, d, r).
> 2) Variables *a* and *b* are the output of timeseries() Streaming 
> Expressions. These will be stored in memory as lists of Tuples containing the 
> time series results.
> 3) Variables *c* and *d* are set using the *col* evaluator. The col evaluator 
> extracts a column of numbers from a list of tuples. In the example *col* is 
> extracting the count\(*\) field from the two time series result sets.
> 4) Variable *r* is the output from the *regress* evaluator. The regress 
> evaluator performs a simple regression analysis on two columns of numbers.
> 5) Once the variables are set, a single Streaming Expression is run by the 
> *let* expression. In the example the *tuple* expression is run. The tuple 
> expression outputs a single Tuple with name/value pairs. Any Streaming 
> Expression can be run by the *let* expression so this can be a complex 
> program. The streaming expression run by *let* has access to all the 
> variables defined earlier.
> 6) The tuple expression in the example has one name / value pair. The name 
> *p* is set to the output of the *predict* evaluator. The predict evaluator is 
> predicting the value of a dependent variable based on the independent 
> variable 50. The regression result stored in variable *r* is used to make the 
> prediction.
> 7) The output of this expression will be a single tuple with the value of the 
> predict function in the *p* field.
> The growing list of issues linked to this ticket are the array manipulation 
> and statistical functions that will form the basis of the stats library. The 
> vast majority of these functions are backed by algorithms in Apache Commons 
> Math. Other machine learning and math libraries will follow.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10651) Streaming Expressions statistical functions library

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16175727#comment-16175727
 ] 

ASF subversion and git services commented on SOLR-10651:


Commit bf9ad75e64054edf3269732f23578b9f481123d5 in lucene-solr's branch 
refs/heads/branch_7x from [~joel.bernstein]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=bf9ad75 ]

SOLR-10651: Statistical function docs for 7.0 Part 2


> Streaming Expressions statistical functions library
> ---
>
> Key: SOLR-10651
> URL: https://issues.apache.org/jira/browse/SOLR-10651
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: streaming expressions
>Reporter: Joel Bernstein
>
> This is a ticket for organizing the new statistical programming features of 
> Streaming Expressions. It's also a place for the community to discuss what 
> functions are needed to support statistical programming. 
> Basic Syntax:
> {code}
> let(a = timeseries(...),
> b = timeseries(...),
> c = col(a, count(*)),
> d = col(b, count(*)),
> r = regress(c, d),
> tuple(p = predict(r, 50)))
> {code}
> The expression above is doing the following:
> 1) The let expression is setting variables (a, b, c, d, r).
> 2) Variables *a* and *b* are the output of timeseries() Streaming 
> Expressions. These will be stored in memory as lists of Tuples containing the 
> time series results.
> 3) Variables *c* and *d* are set using the *col* evaluator. The col evaluator 
> extracts a column of numbers from a list of tuples. In the example *col* is 
> extracting the count\(*\) field from the two time series result sets.
> 4) Variable *r* is the output from the *regress* evaluator. The regress 
> evaluator performs a simple regression analysis on two columns of numbers.
> 5) Once the variables are set, a single Streaming Expression is run by the 
> *let* expression. In the example the *tuple* expression is run. The tuple 
> expression outputs a single Tuple with name/value pairs. Any Streaming 
> Expression can be run by the *let* expression so this can be a complex 
> program. The streaming expression run by *let* has access to all the 
> variables defined earlier.
> 6) The tuple expression in the example has one name / value pair. The name 
> *p* is set to the output of the *predict* evaluator. The predict evaluator is 
> predicting the value of a dependent variable based on the independent 
> variable 50. The regression result stored in variable *r* is used to make the 
> prediction.
> 7) The output of this expression will be a single tuple with the value of the 
> predict function in the *p* field.
> The growing list of issues linked to this ticket are the array manipulation 
> and statistical functions that will form the basis of the stats library. The 
> vast majority of these functions are backed by algorithms in Apache Commons 
> Math. Other machine learning and math libraries will follow.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7970) Improve generation of circle plane

[Karl Wright (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16175726#comment-16175726
 ] 

Karl Wright commented on LUCENE-7970:
-

[~ivera], with luck I'll be able to do more on this tomorrow, since things are 
slowing down now at work somewhat.

My thought is that it should be possible to pick the outer and inner planes 
based on (a) whether planetModel.c > planetModel.ab, and (b) a precise 
computation of the top and bottom latitude points of the circle.  I'll think 
about this further tomorrow morning and let's see what you come up with too.


> Improve generation of circle plane 
> ---
>
> Key: LUCENE-7970
> URL: https://issues.apache.org/jira/browse/LUCENE-7970
> Project: Lucene - Core
>  Issue Type: Improvement
>  Components: modules/spatial3d
>Reporter: Ignacio Vera
>Assignee: Karl Wright
> Attachments: LUCENE_7970.patch, LUCENE-7970.patch, 
> LUCENE-7970-proposed.patch
>
>
> Hi [~Karl wright],
> How circles are currently build do not behave very well when the planet model 
> is not an sphere. when you are close to the border in WGS84 you might get 
> false positves or false negatives when checking if a point is WITHIN. I think 
> the reason is how the points to generate the circle plane are generated which 
> assumes a sphere.
> My proposal is the following:
> Add a new method to PlanetModel:  
> public GeoPoint pointOnBearing(GeoPoint from, double dist, double bearing);
> Which uses and algorithm that takes into account that the planet might not be 
> spherical. For example Vincenty's formulae 
> (https://en.wikipedia.org/wiki/Vincenty%27s_formulae).
> Use this method to generate the points for the circle plane. My experiments 
> shows that this approach removes false negatives in WGS84 meanwhile it works 
> nicely in the Sphere.
> Does it make sense?



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS-EA] Lucene-Solr-master-Windows (32bit/jdk-9-ea+181) - Build # 6913 - Still Unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Windows/6913/
Java: 32bit/jdk-9-ea+181 -server -XX:+UseParallelGC --illegal-access=deny

1 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.solr.core.TestNRTOpen

Error Message:
Could not remove the following files (in the order of attempts):
C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.core.TestNRTOpen_1DE54C0DC41ED01B-001\init-core-data-001:
 java.nio.file.AccessDeniedException: 
C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.core.TestNRTOpen_1DE54C0DC41ED01B-001\init-core-data-001

C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.core.TestNRTOpen_1DE54C0DC41ED01B-001:
 java.nio.file.DirectoryNotEmptyException: 
C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.core.TestNRTOpen_1DE54C0DC41ED01B-001
 

Stack Trace:
java.io.IOException: Could not remove the following files (in the order of 
attempts):
   
C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.core.TestNRTOpen_1DE54C0DC41ED01B-001\init-core-data-001:
 java.nio.file.AccessDeniedException: 
C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.core.TestNRTOpen_1DE54C0DC41ED01B-001\init-core-data-001
   
C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.core.TestNRTOpen_1DE54C0DC41ED01B-001:
 java.nio.file.DirectoryNotEmptyException: 
C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.core.TestNRTOpen_1DE54C0DC41ED01B-001

at __randomizedtesting.SeedInfo.seed([1DE54C0DC41ED01B]:0)
at org.apache.lucene.util.IOUtils.rm(IOUtils.java:329)
at 
org.apache.lucene.util.TestRuleTemporaryFilesCleanup.afterAlways(TestRuleTemporaryFilesCleanup.java:216)
at 
com.carrotsearch.randomizedtesting.rules.TestRuleAdapter$1.afterAlways(TestRuleAdapter.java:31)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:43)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.base/java.lang.Thread.run(Thread.java:844)




Build Log:
[...truncated 11 lines...]
ERROR: Error fetching remote repo 'origin'
hudson.plugins.git.GitException: Failed to fetch from 
git://git.apache.org/lucene-solr.git
at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:817)
at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1084)
at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1115)
at hudson.scm.SCM.checkout(SCM.java:495)
at hudson.model.AbstractProject.checkout(AbstractProject.java:1212)
at 
hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:566)
at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
at 
hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:491)
at hudson.model.Run.execute(Run.java:1724)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:419)
Caused by: hudson.plugins.git.GitException: 
org.eclipse.jgit.api.errors.TransportException: Short read of block.
at 
org.jenkinsci.plugins.gitclient.JGitAPIImpl$2.execute(JGitAPIImpl.java:624)
at 
org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$1.call(RemoteGitImpl.java:153)
at 
org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$1.call(RemoteGitImpl.java:146)
at hudson.remoting.UserRequest.perform(UserRequest.java:205)
at hudson.remoting.UserRequest.perform(UserRequest.java:52)
at hudson.remoting.Request$2.run(Request.java:356)
at 
hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
at java.util.concurrent.FutureTask.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at 

[JENKINS-EA] Lucene-Solr-7.x-Linux (64bit/jdk-9-ea+181) - Build # 456 - Unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-7.x-Linux/456/
Java: 64bit/jdk-9-ea+181 -XX:+UseCompressedOops -XX:+UseG1GC 
--illegal-access=deny

1 tests failed.
FAILED:  
org.apache.solr.client.solrj.io.stream.StreamExpressionTest.testPoissonDistribution

Error Message:
expected:<99.> but was:<103.22675723572304>

Stack Trace:
java.lang.AssertionError: expected:<99.> but was:<103.22675723572304>
at 
__randomizedtesting.SeedInfo.seed([88217FB1150D87C0:10318010B7B4FFBC]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:443)
at org.junit.Assert.assertEquals(Assert.java:512)
at 
org.apache.solr.client.solrj.io.stream.StreamExpressionTest.testPoissonDistribution(StreamExpressionTest.java:6318)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:564)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.base/java.lang.Thread.run(Thread.java:844)




Build Log:
[...truncated 14561 lines...]
   [junit4] Suite: 

[jira] [Comment Edited] (SOLR-11386) Extracting learning to rank features fails when word ordering of EFI argument changed.

[Michael A. Alcorn (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16175297#comment-16175297
 ] 

Michael A. Alcorn edited comment on SOLR-11386 at 9/21/17 9:51 PM:
---

Thanks for the reply, [~cpoerschke]. After further investigation, I think the 
issue is the multi-term EFI arguments. The query I thought was working is 
actually only returning the extracted feature values corresponding to _the 
first term_. So:

{code}
http://gss-test-fusion.usersys.redhat.com:8983/solr/access/query?q=couple of 
fiber channel added={!ltr model=redhat_efi_model reRankDocs=1 
efi.case_summary=the efi.case_description=of fiber channel added couple 
efi.case_issue=the efi.case_environment=the}=id,score,[features]=10
{code}

is the same as:

{code}
http://gss-test-fusion.usersys.redhat.com:8983/solr/access/query?q=couple of 
fiber channel added={!ltr model=redhat_efi_model reRankDocs=1 
efi.case_summary=the efi.case_description=of efi.case_issue=the 
efi.case_environment=the}=id,score,[features]=10
{code}

(i.e., all zeros because "of" is a stop word). And:

{code}
http://gss-test-fusion.usersys.redhat.com:8983/solr/access/query?q=couple of 
fiber channel added={!ltr model=redhat_efi_model reRankDocs=1 
efi.case_summary=the efi.case_description=fiber channel added couple of 
efi.case_issue=the efi.case_environment=the}=id,score,[features]=10
{code}

is the same as:

{code}
http://gss-test-fusion.usersys.redhat.com:8983/solr/access/query?q=couple of 
fiber channel added={!ltr model=redhat_efi_model reRankDocs=1 
efi.case_summary=the efi.case_description=fiber efi.case_issue=the 
efi.case_environment=the}=id,score,[features]=10
{code}

This is an example of how we're defining the features:

{code}
{
"store": "redhat_efi_feature_store",
"name": "case_description_issue_tfidf",
"class": "org.apache.solr.ltr.feature.SolrFeature",
"params": {
"q": "{!field f=issue_tfidf}${case_description}"
}
}
{code}


was (Author: malcorn_redhat):
Thanks for the reply, [~cpoerschke]. After further investigation, I think the 
issue is the multi-term EFI arguments. The query I thought was working is 
actually only returning the extracted feature values corresponding to _the 
first term_. So:

{code}
http://gss-test-fusion.usersys.redhat.com:8983/solr/access/query?q=couple of 
fiber channel added={!ltr model=redhat_efi_model reRankDocs=1 
efi.case_summary=the efi.case_description=of fiber channel added couple 
efi.case_issue=the efi.case_environment=the}=id,score,[features]=10
{code}

is the same as:

{code}
http://gss-test-fusion.usersys.redhat.com:8983/solr/access/query?q=couple of 
fiber channel added={!ltr model=redhat_efi_model reRankDocs=1 
efi.case_summary=the efi.case_description=of efi.case_issue=the 
efi.case_environment=the}=id,score,[features]=10
{code}

(i.e., all zeros because "of" is a stop word). And:

{code}
http://gss-test-fusion.usersys.redhat.com:8983/solr/access/query?q=couple of 
fiber channel added={!ltr model=redhat_efi_model reRankDocs=1 
efi.case_summary=the efi.case_description=fiber channel added couple of 
efi.case_issue=the efi.case_environment=the}=id,score,[features]=10
{code}

is the same as:

{code}
http://gss-test-fusion.usersys.redhat.com:8983/solr/access/query?q=couple of 
fiber channel added={!ltr model=redhat_efi_model reRankDocs=1 
efi.case_summary=the efi.case_description=fiber efi.case_issue=the 
efi.case_environment=the}=id,score,[features]=10
{code}

This is an example of how we're defining the feature values:

{code}
{
"store": "redhat_efi_feature_store",
"name": "case_description_issue_tfidf",
"class": "org.apache.solr.ltr.feature.SolrFeature",
"params": {
"q": "{!field f=issue_tfidf}${case_description}"
}
}
{code}

> Extracting learning to rank features fails when word ordering of EFI argument 
> changed.
> --
>
> Key: SOLR-11386
> URL: https://issues.apache.org/jira/browse/SOLR-11386
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: contrib - LTR
>Affects Versions: 6.5.1
>Reporter: Michael A. Alcorn
>
> I'm getting some extremely strange behavior when trying to extract features 
> for a learning to rank model. The following query incorrectly says all 
> features have zero values:
> {code}
> http://gss-test-fusion.usersys.redhat.com:8983/solr/access/query?q=added 
> couple of fiber channel={!ltr model=redhat_efi_model reRankDocs=1 
> efi.case_summary=the efi.case_description=added couple of fiber channel 
> efi.case_issue=the efi.case_environment=the}=id,score,[features]=10
> {code}
> But this query, which simply moves the word 

[jira] [Commented] (SOLR-11297) Message "Lock held by this virtual machine" during startup. Solr is trying to start some cores twice

[Erick Erickson (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16175513#comment-16175513
 ] 

Erick Erickson commented on SOLR-11297:
---

Well, the patch should apply pretty cleanly to the 6.6 branch, although I admit 
I created it against Master. Is there any way you can apply it and give it a 
try?



> Message "Lock held by this virtual machine" during startup.  Solr is trying 
> to start some cores twice
> -
>
> Key: SOLR-11297
> URL: https://issues.apache.org/jira/browse/SOLR-11297
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Affects Versions: 6.6
>Reporter: Shawn Heisey
>Assignee: Erick Erickson
> Attachments: SOLR-11297.patch, SOLR-11297.patch, SOLR-11297.sh, 
> solr6_6-startup.log
>
>
> Sometimes when Solr is restarted, I get some "lock held by this virtual 
> machine" messages in the log, and the admin UI has messages about a failure 
> to open a new searcher.  It doesn't happen on all cores, and the list of 
> cores that have the problem changes on subsequent restarts.  The cores that 
> exhibit the problems are working just fine -- the first core load is 
> successful, the failure to open a new searcher is on a second core load 
> attempt, which fails.
> None of the cores in the system are sharing an instanceDir or dataDir.  This 
> has been verified several times.
> The index is sharded manually, and the servers are not running in cloud mode.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-7.x-Solaris (64bit/jdk1.8.0) - Build # 199 - Unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-7.x-Solaris/199/
Java: 64bit/jdk1.8.0 -XX:+UseCompressedOops -XX:+UseSerialGC

1 tests failed.
FAILED:  
junit.framework.TestSuite.org.apache.solr.cloud.TestSolrCloudWithSecureImpersonation

Error Message:
1 thread leaked from SUITE scope at 
org.apache.solr.cloud.TestSolrCloudWithSecureImpersonation: 1) 
Thread[id=1260, name=jetty-launcher-192-thread-1-EventThread, 
state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation] 
at sun.misc.Unsafe.park(Native Method) at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) 
at 
java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037)
 at 
java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)
 at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277)  
   at 
org.apache.curator.CuratorZookeeperClient.internalBlockUntilConnectedOrTimedOut(CuratorZookeeperClient.java:323)
 at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:105)  
   at 
org.apache.curator.framework.imps.GetDataBuilderImpl.pathInForeground(GetDataBuilderImpl.java:288)
 at 
org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:279)
 at 
org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:41)
 at 
org.apache.curator.framework.recipes.shared.SharedValue.readValue(SharedValue.java:244)
 at 
org.apache.curator.framework.recipes.shared.SharedValue.access$100(SharedValue.java:44)
 at 
org.apache.curator.framework.recipes.shared.SharedValue$1.process(SharedValue.java:61)
 at 
org.apache.curator.framework.imps.NamespaceWatcher.process(NamespaceWatcher.java:67)
 at 
org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:530)   
  at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:505)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE 
scope at org.apache.solr.cloud.TestSolrCloudWithSecureImpersonation: 
   1) Thread[id=1260, name=jetty-launcher-192-thread-1-EventThread, 
state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation]
at sun.misc.Unsafe.park(Native Method)
at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at 
java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037)
at 
java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)
at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277)
at 
org.apache.curator.CuratorZookeeperClient.internalBlockUntilConnectedOrTimedOut(CuratorZookeeperClient.java:323)
at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:105)
at 
org.apache.curator.framework.imps.GetDataBuilderImpl.pathInForeground(GetDataBuilderImpl.java:288)
at 
org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:279)
at 
org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:41)
at 
org.apache.curator.framework.recipes.shared.SharedValue.readValue(SharedValue.java:244)
at 
org.apache.curator.framework.recipes.shared.SharedValue.access$100(SharedValue.java:44)
at 
org.apache.curator.framework.recipes.shared.SharedValue$1.process(SharedValue.java:61)
at 
org.apache.curator.framework.imps.NamespaceWatcher.process(NamespaceWatcher.java:67)
at 
org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:530)
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:505)
at __randomizedtesting.SeedInfo.seed([25FB88C4C6FA430E]:0)




Build Log:
[...truncated 11488 lines...]
   [junit4] Suite: org.apache.solr.cloud.TestSolrCloudWithSecureImpersonation
   [junit4]   2> Creating dataDir: 
/export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.TestSolrCloudWithSecureImpersonation_25FB88C4C6FA430E-001/init-core-data-001
   [junit4]   2> 154287 WARN  
(SUITE-TestSolrCloudWithSecureImpersonation-seed#[25FB88C4C6FA430E]-worker) [   
 ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=1 numCloses=1
   [junit4]   2> 154288 INFO  
(SUITE-TestSolrCloudWithSecureImpersonation-seed#[25FB88C4C6FA430E]-worker) [   
 ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) 
w/NUMERIC_DOCVALUES_SYSPROP=true
   [junit4]   2> 154289 INFO  
(SUITE-TestSolrCloudWithSecureImpersonation-seed#[25FB88C4C6FA430E]-worker) [   
 ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (true) via: 
@org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN, clientAuth=NaN)
   [junit4]   2> 154289 INFO  

Release Announcement: General Availability of JDK 9

[Rory O'Donnell]

Hi Uwe & Dawid,

Three items to share with you today

 * *JDK 9 General Availability
   *
 o GPL'd binaries from Oracle are available here:
 + http://jdk.java.net/9
 o See Mark Reinhold's email for more details on the Release [1]
 + delivery of Project Jigsaw [2]

 * Are you JDK 9 Ready ?
 o The Quality Outreach wiki has been updated to include a JDK 9
   Ready column.
 o If you would like us to identify your project as JDK 9 ready ,
   please let me know and I will add it to the wiki.

 * Quality Outreach Report for September 2017**is available
 o many thanks for your continued support and welcome to the new
   projects!

Rgds,Rory

[1] 
http://mail.openjdk.java.net/pipermail/announce/2017-September/000230.html

[2] https://mreinhold.org/blog/jigsaw-complete

--
Rgds,Rory O'Donnell
Quality Engineering Manager
Oracle EMEA , Dublin, Ireland

[JENKINS] Lucene-Solr-master-Solaris (64bit/jdk1.8.0) - Build # 1428 - Still Unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Solaris/1428/
Java: 64bit/jdk1.8.0 -XX:+UseCompressedOops -XX:+UseSerialGC

3 tests failed.
FAILED:  
junit.framework.TestSuite.org.apache.solr.cloud.TestSolrCloudWithSecureImpersonation

Error Message:
2 threads leaked from SUITE scope at 
org.apache.solr.cloud.TestSolrCloudWithSecureImpersonation: 1) 
Thread[id=13930, name=jetty-launcher-2922-thread-1-SendThread(127.0.0.1:39323), 
state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation] 
at java.lang.Thread.sleep(Native Method) at 
org.apache.zookeeper.ClientCnxnSocketNIO.cleanup(ClientCnxnSocketNIO.java:230)  
   at 
org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:1246)
 at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1170)2) 
Thread[id=13931, name=jetty-launcher-2922-thread-1-EventThread, state=WAITING, 
group=TGRP-TestSolrCloudWithSecureImpersonation] at 
sun.misc.Unsafe.park(Native Method) at 
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
 at 
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) 
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:501)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 2 threads leaked from SUITE 
scope at org.apache.solr.cloud.TestSolrCloudWithSecureImpersonation: 
   1) Thread[id=13930, 
name=jetty-launcher-2922-thread-1-SendThread(127.0.0.1:39323), 
state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation]
at java.lang.Thread.sleep(Native Method)
at 
org.apache.zookeeper.ClientCnxnSocketNIO.cleanup(ClientCnxnSocketNIO.java:230)
at 
org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:1246)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1170)
   2) Thread[id=13931, name=jetty-launcher-2922-thread-1-EventThread, 
state=WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at 
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:501)
at __randomizedtesting.SeedInfo.seed([B62E94913FD284E5]:0)


FAILED:  
junit.framework.TestSuite.org.apache.solr.cloud.TestSolrCloudWithSecureImpersonation

Error Message:
There are still zombie threads that couldn't be terminated:1) 
Thread[id=13930, name=jetty-launcher-2922-thread-1-SendThread(127.0.0.1:39323), 
state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation] 
at java.lang.Thread.sleep(Native Method) at 
org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101)
 at 
org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:997)
 at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1060)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie 
threads that couldn't be terminated:
   1) Thread[id=13930, 
name=jetty-launcher-2922-thread-1-SendThread(127.0.0.1:39323), 
state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation]
at java.lang.Thread.sleep(Native Method)
at 
org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101)
at 
org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:997)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1060)
at __randomizedtesting.SeedInfo.seed([B62E94913FD284E5]:0)


FAILED:  
org.apache.solr.client.solrj.io.stream.StreamExpressionTest.testExecutorStream

Error Message:
Error from server at http://127.0.0.1:57362/solr/workQueue_shard2_replica_n2: 
Expected mime type application/octet-stream but got text/html.   
 
Error 404HTTP ERROR: 404 Problem 
accessing /solr/workQueue_shard2_replica_n2/update. Reason: Can not 
find: /solr/workQueue_shard2_replica_n2/update http://eclipse.org/jetty;>Powered by Jetty:// 9.3.20.v20170531 
  

Stack Trace:
org.apache.solr.client.solrj.impl.CloudSolrClient$RouteException: Error from 
server at http://127.0.0.1:57362/solr/workQueue_shard2_replica_n2: Expected 
mime type application/octet-stream but got text/html. 


Error 404 


HTTP ERROR: 404
Problem accessing /solr/workQueue_shard2_replica_n2/update. Reason:
Can not find: /solr/workQueue_shard2_replica_n2/update
http://eclipse.org/jetty;>Powered by Jetty:// 
9.3.20.v20170531



at 
__randomizedtesting.SeedInfo.seed([BA7D7C893753F05D:98BDFD721439DA4D]:0)
at 

[JENKINS] Lucene-Solr-Tests-7.x - Build # 163 - Unstable

[Apache Jenkins Server]

Build: https://builds.apache.org/job/Lucene-Solr-Tests-7.x/163/

8 tests failed.
FAILED:  org.apache.solr.cloud.AssignBackwardCompatibilityTest.test

Error Message:
Could not load collection from ZK: collection1

Stack Trace:
org.apache.solr.common.SolrException: Could not load collection from ZK: 
collection1
at 
__randomizedtesting.SeedInfo.seed([995B4A163C40F912:110F75CC92BC94EA]:0)
at 
org.apache.solr.common.cloud.ZkStateReader.getCollectionLive(ZkStateReader.java:1115)
at 
org.apache.solr.common.cloud.ZkStateReader$LazyCollectionRef.get(ZkStateReader.java:648)
at 
org.apache.solr.common.cloud.ClusterState.getCollectionOrNull(ClusterState.java:128)
at 
org.apache.solr.common.cloud.ClusterState.getCollection(ClusterState.java:108)
at 
org.apache.solr.cloud.SolrCloudTestCase.getCollectionState(SolrCloudTestCase.java:247)
at 
org.apache.solr.cloud.AssignBackwardCompatibilityTest.test(AssignBackwardCompatibilityTest.java:90)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 

[jira] [Updated] (SOLR-11386) Extracting learning to rank features fails when word ordering of EFI argument changed.

[Michael A. Alcorn (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-11386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael A. Alcorn updated SOLR-11386:
-
Description: 
I'm getting some extremely strange behavior when trying to extract features for 
a learning to rank model. The following query incorrectly says all features 
have zero values:

{code}
http://gss-test-fusion.usersys.redhat.com:8983/solr/access/query?q=added couple 
of fiber channel={!ltr model=redhat_efi_model reRankDocs=1 
efi.case_summary=the efi.case_description=added couple of fiber channel 
efi.case_issue=the efi.case_environment=the}=id,score,[features]=10
{code}

But this query, which simply moves the word "added" from the front of the 
provided text to the back, properly fills in the feature values:

{code}
http://gss-test-fusion.usersys.redhat.com:8983/solr/access/query?q=couple of 
fiber channel added={!ltr model=redhat_efi_model reRankDocs=1 
efi.case_summary=the efi.case_description=couple of fiber channel added 
efi.case_issue=the efi.case_environment=the}=id,score,[features]=10
{code}

The explain output for the failing query can be found here:

https://gist.github.com/manisnesan/18a8f1804f29b1b62ebfae1211f38cc4

and the explain output for the properly functioning query can be found here:

https://gist.github.com/manisnesan/47685a561605e2229434b38aed11cc65

  was:
I'm getting some extremely strange behavior when trying to extract features for 
a learning to rank model. The following query incorrectly says all features 
have zero values:

http://gss-test-fusion.usersys.redhat.com:8983/solr/access/query?q=added couple 
of fiber channel={!ltr model=redhat_efi_model reRankDocs=1 
efi.case_summary=the efi.case_description=added couple of fiber channel 
efi.case_issue=the efi.case_environment=the}=id,score,[features]=10

But this query, which simply moves the word "added" from the front of the 
provided text to the back, properly fills in the feature values:

http://gss-test-fusion.usersys.redhat.com:8983/solr/access/query?q=couple of 
fiber channel added={!ltr model=redhat_efi_model reRankDocs=1 
efi.case_summary=the efi.case_description=couple of fiber channel added 
efi.case_issue=the efi.case_environment=the}=id,score,[features]=10

The explain output for the failing query can be found here:

https://gist.github.com/manisnesan/18a8f1804f29b1b62ebfae1211f38cc4

and the explain output for the properly functioning query can be found here:

https://gist.github.com/manisnesan/47685a561605e2229434b38aed11cc65


> Extracting learning to rank features fails when word ordering of EFI argument 
> changed.
> --
>
> Key: SOLR-11386
> URL: https://issues.apache.org/jira/browse/SOLR-11386
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: contrib - LTR
>Affects Versions: 6.5.1
>Reporter: Michael A. Alcorn
>
> I'm getting some extremely strange behavior when trying to extract features 
> for a learning to rank model. The following query incorrectly says all 
> features have zero values:
> {code}
> http://gss-test-fusion.usersys.redhat.com:8983/solr/access/query?q=added 
> couple of fiber channel={!ltr model=redhat_efi_model reRankDocs=1 
> efi.case_summary=the efi.case_description=added couple of fiber channel 
> efi.case_issue=the efi.case_environment=the}=id,score,[features]=10
> {code}
> But this query, which simply moves the word "added" from the front of the 
> provided text to the back, properly fills in the feature values:
> {code}
> http://gss-test-fusion.usersys.redhat.com:8983/solr/access/query?q=couple of 
> fiber channel added={!ltr model=redhat_efi_model reRankDocs=1 
> efi.case_summary=the efi.case_description=couple of fiber channel added 
> efi.case_issue=the efi.case_environment=the}=id,score,[features]=10
> {code}
> The explain output for the failing query can be found here:
> https://gist.github.com/manisnesan/18a8f1804f29b1b62ebfae1211f38cc4
> and the explain output for the properly functioning query can be found here:
> https://gist.github.com/manisnesan/47685a561605e2229434b38aed11cc65



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Comment Edited] (SOLR-11386) Extracting learning to rank features fails when word ordering of EFI argument changed.

[Michael A. Alcorn (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16175297#comment-16175297
 ] 

Michael A. Alcorn edited comment on SOLR-11386 at 9/21/17 7:10 PM:
---

Thanks for the reply, [~cpoerschke]. After further investigation, I think the 
issue is the multi-term EFI arguments. The query I thought was working is 
actually only returning the extracted feature values corresponding to _the 
first term_. So:

{code}
http://gss-test-fusion.usersys.redhat.com:8983/solr/access/query?q=couple of 
fiber channel added={!ltr model=redhat_efi_model reRankDocs=1 
efi.case_summary=the efi.case_description=of fiber channel added couple 
efi.case_issue=the efi.case_environment=the}=id,score,[features]=10
{code}

is the same as:

{code}
http://gss-test-fusion.usersys.redhat.com:8983/solr/access/query?q=couple of 
fiber channel added={!ltr model=redhat_efi_model reRankDocs=1 
efi.case_summary=the efi.case_description=of efi.case_issue=the 
efi.case_environment=the}=id,score,[features]=10
{code}

(i.e., all zeros because "of" is a stop word). And:

{code}
http://gss-test-fusion.usersys.redhat.com:8983/solr/access/query?q=couple of 
fiber channel added={!ltr model=redhat_efi_model reRankDocs=1 
efi.case_summary=the efi.case_description=fiber channel added couple of 
efi.case_issue=the efi.case_environment=the}=id,score,[features]=10
{code}

is the same as:

{code}
http://gss-test-fusion.usersys.redhat.com:8983/solr/access/query?q=couple of 
fiber channel added={!ltr model=redhat_efi_model reRankDocs=1 
efi.case_summary=the efi.case_description=fiber efi.case_issue=the 
efi.case_environment=the}=id,score,[features]=10
{code}

This is an example of how we're defining the feature values:

{code}
{
"store": "redhat_efi_feature_store",
"name": "case_description_issue_tfidf",
"class": "org.apache.solr.ltr.feature.SolrFeature",
"params": {
"q": "{!field f=issue_tfidf}${case_description}"
}
}
{code}


was (Author: malcorn_redhat):
Thanks for the reply, [~cpoerschke]. After further investigation, I think the 
issue is the multi-term arguments. The query I thought was working is actually 
only returning the extracted feature values corresponding to _the first term_. 
So:

{code}
http://gss-test-fusion.usersys.redhat.com:8983/solr/access/query?q=couple of 
fiber channel added={!ltr model=redhat_efi_model reRankDocs=1 
efi.case_summary=the efi.case_description=of fiber channel added couple 
efi.case_issue=the efi.case_environment=the}=id,score,[features]=10
{code}

is the same as:

{code}
http://gss-test-fusion.usersys.redhat.com:8983/solr/access/query?q=couple of 
fiber channel added={!ltr model=redhat_efi_model reRankDocs=1 
efi.case_summary=the efi.case_description=of efi.case_issue=the 
efi.case_environment=the}=id,score,[features]=10
{code}

(i.e., all zeros because "of" is a stop word). And:

{code}
http://gss-test-fusion.usersys.redhat.com:8983/solr/access/query?q=couple of 
fiber channel added={!ltr model=redhat_efi_model reRankDocs=1 
efi.case_summary=the efi.case_description=fiber channel added couple of 
efi.case_issue=the efi.case_environment=the}=id,score,[features]=10
{code}

is the same as:

{code}
http://gss-test-fusion.usersys.redhat.com:8983/solr/access/query?q=couple of 
fiber channel added={!ltr model=redhat_efi_model reRankDocs=1 
efi.case_summary=the efi.case_description=fiber efi.case_issue=the 
efi.case_environment=the}=id,score,[features]=10
{code}

This is an example of how we're defining the feature values:

{code}
{
"store": "redhat_efi_feature_store",
"name": "case_description_issue_tfidf",
"class": "org.apache.solr.ltr.feature.SolrFeature",
"params": {
"q": "{!field f=issue_tfidf}${case_description}"
}
}
{code}

> Extracting learning to rank features fails when word ordering of EFI argument 
> changed.
> --
>
> Key: SOLR-11386
> URL: https://issues.apache.org/jira/browse/SOLR-11386
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: contrib - LTR
>Affects Versions: 6.5.1
>Reporter: Michael A. Alcorn
>
> I'm getting some extremely strange behavior when trying to extract features 
> for a learning to rank model. The following query incorrectly says all 
> features have zero values:
> http://gss-test-fusion.usersys.redhat.com:8983/solr/access/query?q=added 
> couple of fiber channel={!ltr model=redhat_efi_model reRankDocs=1 
> efi.case_summary=the efi.case_description=added couple of fiber channel 
> efi.case_issue=the efi.case_environment=the}=id,score,[features]=10
> But this query, which simply moves the word "added" from the front 

[jira] [Commented] (SOLR-11386) Extracting learning to rank features fails when word ordering of EFI argument changed.

[Michael A. Alcorn (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16175297#comment-16175297
 ] 

Michael A. Alcorn commented on SOLR-11386:
--

Thanks for the reply, [~cpoerschke]. After further investigation, I think the 
issue is the multi-term arguments. The query I thought was working is actually 
only returning the extracted feature values corresponding to _the first term_. 
So:

{code}
http://gss-test-fusion.usersys.redhat.com:8983/solr/access/query?q=couple of 
fiber channel added={!ltr model=redhat_efi_model reRankDocs=1 
efi.case_summary=the efi.case_description=of fiber channel added couple 
efi.case_issue=the efi.case_environment=the}=id,score,[features]=10
{code}

is the same as:

{code}
http://gss-test-fusion.usersys.redhat.com:8983/solr/access/query?q=couple of 
fiber channel added={!ltr model=redhat_efi_model reRankDocs=1 
efi.case_summary=the efi.case_description=of efi.case_issue=the 
efi.case_environment=the}=id,score,[features]=10
{code}

(i.e., all zeros because "of" is a stop word). And:

{code}
http://gss-test-fusion.usersys.redhat.com:8983/solr/access/query?q=couple of 
fiber channel added={!ltr model=redhat_efi_model reRankDocs=1 
efi.case_summary=the efi.case_description=fiber channel added couple of 
efi.case_issue=the efi.case_environment=the}=id,score,[features]=10
{code}

is the same as:

{code}
http://gss-test-fusion.usersys.redhat.com:8983/solr/access/query?q=couple of 
fiber channel added={!ltr model=redhat_efi_model reRankDocs=1 
efi.case_summary=the efi.case_description=fiber efi.case_issue=the 
efi.case_environment=the}=id,score,[features]=10
{code}

This is an example of how we're defining the feature values:

{code}
{
"store": "redhat_efi_feature_store",
"name": "case_description_issue_tfidf",
"class": "org.apache.solr.ltr.feature.SolrFeature",
"params": {
"q": "{!field f=issue_tfidf}${case_description}"
}
}
{code}

> Extracting learning to rank features fails when word ordering of EFI argument 
> changed.
> --
>
> Key: SOLR-11386
> URL: https://issues.apache.org/jira/browse/SOLR-11386
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: contrib - LTR
>Affects Versions: 6.5.1
>Reporter: Michael A. Alcorn
>
> I'm getting some extremely strange behavior when trying to extract features 
> for a learning to rank model. The following query incorrectly says all 
> features have zero values:
> http://gss-test-fusion.usersys.redhat.com:8983/solr/access/query?q=added 
> couple of fiber channel={!ltr model=redhat_efi_model reRankDocs=1 
> efi.case_summary=the efi.case_description=added couple of fiber channel 
> efi.case_issue=the efi.case_environment=the}=id,score,[features]=10
> But this query, which simply moves the word "added" from the front of the 
> provided text to the back, properly fills in the feature values:
> http://gss-test-fusion.usersys.redhat.com:8983/solr/access/query?q=couple of 
> fiber channel added={!ltr model=redhat_efi_model reRankDocs=1 
> efi.case_summary=the efi.case_description=couple of fiber channel added 
> efi.case_issue=the efi.case_environment=the}=id,score,[features]=10
> The explain output for the failing query can be found here:
> https://gist.github.com/manisnesan/18a8f1804f29b1b62ebfae1211f38cc4
> and the explain output for the properly functioning query can be found here:
> https://gist.github.com/manisnesan/47685a561605e2229434b38aed11cc65



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-7.x-MacOSX (64bit/jdk1.8.0) - Build # 200 - Still Unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-7.x-MacOSX/200/
Java: 64bit/jdk1.8.0 -XX:-UseCompressedOops -XX:+UseSerialGC

3 tests failed.
FAILED:  org.apache.solr.security.BasicAuthIntegrationTest.testBasicAuth

Error Message:
Too many open files

Stack Trace:
java.net.SocketException: Too many open files
at 
__randomizedtesting.SeedInfo.seed([F41D92893DF2B0AB:4873E49B99A133D1]:0)
at java.net.Socket.createImpl(Socket.java:460)
at java.net.Socket.getImpl(Socket.java:520)
at java.net.Socket.setSoTimeout(Socket.java:1141)
at 
org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:120)
at 
org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:359)
at 
org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:381)
at 
org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:237)
at 
org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185)
at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)
at 
org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111)
at 
org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
at 
org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
at 
org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:108)
at 
org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)
at 
org.apache.solr.security.BasicAuthIntegrationTest.verifySecurityStatus(BasicAuthIntegrationTest.java:261)
at 
org.apache.solr.security.BasicAuthIntegrationTest.verifySecurityStatus(BasicAuthIntegrationTest.java:248)
at 
org.apache.solr.security.BasicAuthIntegrationTest.testBasicAuth(BasicAuthIntegrationTest.java:108)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 

[jira] [Commented] (SOLR-11277) Add auto hard commit setting based on tlog size

[Anshum Gupta (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11277?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16175195#comment-16175195
 ] 

Anshum Gupta commented on SOLR-11277:
-

I'll take a look at this.

> Add auto hard commit setting based on tlog size
> ---
>
> Key: SOLR-11277
> URL: https://issues.apache.org/jira/browse/SOLR-11277
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Rupa Shankar
> Attachments: max_size_auto_commit.patch
>
>
> When indexing documents of variable sizes and at variable schedules, it can 
> be hard to estimate the optimal auto hard commit maxDocs or maxTime settings. 
> We’ve had some occurrences of really huge tlogs, resulting in serious issues, 
> so in an attempt to avoid this, it would be great to have a “maxSize” setting 
> based on the tlog size on disk. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Assigned] (SOLR-11277) Add auto hard commit setting based on tlog size

[Anshum Gupta (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-11277?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Anshum Gupta reassigned SOLR-11277:
---

Assignee: Anshum Gupta

> Add auto hard commit setting based on tlog size
> ---
>
> Key: SOLR-11277
> URL: https://issues.apache.org/jira/browse/SOLR-11277
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Rupa Shankar
>Assignee: Anshum Gupta
> Attachments: max_size_auto_commit.patch
>
>
> When indexing documents of variable sizes and at variable schedules, it can 
> be hard to estimate the optimal auto hard commit maxDocs or maxTime settings. 
> We’ve had some occurrences of really huge tlogs, resulting in serious issues, 
> so in an attempt to avoid this, it would be great to have a “maxSize” setting 
> based on the tlog size on disk. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-11386) Extracting learning to rank features fails when word ordering of EFI argument changed.

[Christine Poerschke (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16175189#comment-16175189
 ] 

Christine Poerschke commented on SOLR-11386:



Hi Michael,

Thanks for reporting this strange behaviour! Saw your post on the mailing list 
yesterday but was unable to try out the 
http://gss-test-fusion.usersys.redhat.com:8983 links you shared (and perhaps am 
not meant to be able to try them anyhow).

Everyone is investigating things differently of course, but here's a few things 
I might try (and perhaps you've already tried them too), in more or less this 
order:

* The gist for the broken scenario shows
{code}
...
... efi.case_description=added couple of fiber channel efi.case_issue= ...
...
{code}
whereas the gist for the working scenario shows
{code}
...
... efi.case_description=couple of fiber channel added  efi.case_issue= ...
...
{code}
with the latter have an extra space before the {{efi.case_issue=}} - just 
curious if that might be relevant or not.

* It seems that perhaps the {{efi.case_description}} being multi-term has 
something to do with it.
** If it's just one term does it work then (I'd guess so) and is it the 
addition of the second or subsequent terms that results in the strange 
behaviour?
** Do the terms matter maybe? I'd guess not but worth just trying out.
** The {{efi.case_description}} is accompanied by {{efi.case_summary}} and 
{{efi.case_issue}} and {{efi.case_environment}} - and that should work and the 
order should not matter - but it might be worth exploring if changing of the 
order ( or removing of the accompanying {{efi}} s) brings any insights.
*** Specifically I'm wondering if the {{efi.case_description}} 'grabbed' the 
subsequent {{efi}} s (but did not grab them when there was the extra space) and 
then the subsequent {{efi}} s being missing causes the strange behavior. _(If 
that is so then obviously it would be helpful to receive a better error message 
or something from Solr.)_

* Could the example be transferred to and reproduced with Solr's techproducts 
example. 
https://lucene.apache.org/solr/guide/6_6/learning-to-rank.html#LearningToRank-QuickStartExample

* Addition of a test case e.g. in TestExternalFeatures.java to reproduce the 
strange behavior.

> Extracting learning to rank features fails when word ordering of EFI argument 
> changed.
> --
>
> Key: SOLR-11386
> URL: https://issues.apache.org/jira/browse/SOLR-11386
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: contrib - LTR
>Affects Versions: 6.5.1
>Reporter: Michael A. Alcorn
>
> I'm getting some extremely strange behavior when trying to extract features 
> for a learning to rank model. The following query incorrectly says all 
> features have zero values:
> http://gss-test-fusion.usersys.redhat.com:8983/solr/access/query?q=added 
> couple of fiber channel={!ltr model=redhat_efi_model reRankDocs=1 
> efi.case_summary=the efi.case_description=added couple of fiber channel 
> efi.case_issue=the efi.case_environment=the}=id,score,[features]=10
> But this query, which simply moves the word "added" from the front of the 
> provided text to the back, properly fills in the feature values:
> http://gss-test-fusion.usersys.redhat.com:8983/solr/access/query?q=couple of 
> fiber channel added={!ltr model=redhat_efi_model reRankDocs=1 
> efi.case_summary=the efi.case_description=couple of fiber channel added 
> efi.case_issue=the efi.case_environment=the}=id,score,[features]=10
> The explain output for the failing query can be found here:
> https://gist.github.com/manisnesan/18a8f1804f29b1b62ebfae1211f38cc4
> and the explain output for the properly functioning query can be found here:
> https://gist.github.com/manisnesan/47685a561605e2229434b38aed11cc65



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-master-Linux (32bit/jdk1.8.0_144) - Build # 20513 - Unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/20513/
Java: 32bit/jdk1.8.0_144 -server -XX:+UseParallelGC

2 tests failed.
FAILED:  org.apache.solr.cloud.LeaderFailoverAfterPartitionTest.test

Error Message:
Expected 2 of 3 replicas to be active but only found 1; 
[core_node5:{"core":"c8n_1x3_lf_shard1_replica_n2","base_url":"http://127.0.0.1:33273/d/sf","node_name":"127.0.0.1:33273_d%2Fsf","state":"active","type":"NRT","leader":"true"}];
 clusterState: 
DocCollection(c8n_1x3_lf//collections/c8n_1x3_lf/state.json/16)={   
"pullReplicas":"0",   "replicationFactor":"1",   "shards":{"shard1":{   
"range":"8000-7fff",   "state":"active",   "replicas":{ 
"core_node4":{   "state":"down",   
"base_url":"http://127.0.0.1:37057/d/sf;,   
"core":"c8n_1x3_lf_shard1_replica_n1",   
"node_name":"127.0.0.1:37057_d%2Fsf",   "type":"NRT"}, 
"core_node5":{   "core":"c8n_1x3_lf_shard1_replica_n2",   
"base_url":"http://127.0.0.1:33273/d/sf;,   
"node_name":"127.0.0.1:33273_d%2Fsf",   "state":"active",   
"type":"NRT",   "leader":"true"}, "core_node6":{   
"core":"c8n_1x3_lf_shard1_replica_n3",   
"base_url":"http://127.0.0.1:38449/d/sf;,   
"node_name":"127.0.0.1:38449_d%2Fsf",   "state":"down",   
"type":"NRT",   "router":{"name":"compositeId"},   "maxShardsPerNode":"1",  
 "autoAddReplicas":"false",   "nrtReplicas":"3",   "tlogReplicas":"0"}

Stack Trace:
java.lang.AssertionError: Expected 2 of 3 replicas to be active but only found 
1; 
[core_node5:{"core":"c8n_1x3_lf_shard1_replica_n2","base_url":"http://127.0.0.1:33273/d/sf","node_name":"127.0.0.1:33273_d%2Fsf","state":"active","type":"NRT","leader":"true"}];
 clusterState: DocCollection(c8n_1x3_lf//collections/c8n_1x3_lf/state.json/16)={
  "pullReplicas":"0",
  "replicationFactor":"1",
  "shards":{"shard1":{
  "range":"8000-7fff",
  "state":"active",
  "replicas":{
"core_node4":{
  "state":"down",
  "base_url":"http://127.0.0.1:37057/d/sf;,
  "core":"c8n_1x3_lf_shard1_replica_n1",
  "node_name":"127.0.0.1:37057_d%2Fsf",
  "type":"NRT"},
"core_node5":{
  "core":"c8n_1x3_lf_shard1_replica_n2",
  "base_url":"http://127.0.0.1:33273/d/sf;,
  "node_name":"127.0.0.1:33273_d%2Fsf",
  "state":"active",
  "type":"NRT",
  "leader":"true"},
"core_node6":{
  "core":"c8n_1x3_lf_shard1_replica_n3",
  "base_url":"http://127.0.0.1:38449/d/sf;,
  "node_name":"127.0.0.1:38449_d%2Fsf",
  "state":"down",
  "type":"NRT",
  "router":{"name":"compositeId"},
  "maxShardsPerNode":"1",
  "autoAddReplicas":"false",
  "nrtReplicas":"3",
  "tlogReplicas":"0"}
at 
__randomizedtesting.SeedInfo.seed([9AC715AA55C7ACF1:12932A70FB3BC109]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at 
org.apache.solr.cloud.LeaderFailoverAfterPartitionTest.testRf3WithLeaderFailover(LeaderFailoverAfterPartitionTest.java:169)
at 
org.apache.solr.cloud.LeaderFailoverAfterPartitionTest.test(LeaderFailoverAfterPartitionTest.java:56)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 

[JENKINS] Lucene-Solr-master-MacOSX (64bit/jdk1.8.0) - Build # 4179 - Still Unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-MacOSX/4179/
Java: 64bit/jdk1.8.0 -XX:+UseCompressedOops -XX:+UseG1GC

3 tests failed.
FAILED:  org.apache.solr.core.TestLazyCores.testNoCommit

Error Message:
Exception during query

Stack Trace:
java.lang.RuntimeException: Exception during query
at 
__randomizedtesting.SeedInfo.seed([2A68C7CF10EC9E9A:F508661EDBCBFD3F]:0)
at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:884)
at org.apache.solr.core.TestLazyCores.check10(TestLazyCores.java:847)
at 
org.apache.solr.core.TestLazyCores.testNoCommit(TestLazyCores.java:829)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.RuntimeException: REQUEST FAILED: 
xpath=//result[@numFound='10']
xml response was: 

00*:*


request was:q=*:*
at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:877)
... 41 more


FAILED:  

[jira] [Resolved] (SOLR-11385) Update asciidoctor-ant to 1.6.0-alpha.5

[Cassandra Targett (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-11385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Cassandra Targett resolved SOLR-11385.
--
Resolution: Fixed

> Update asciidoctor-ant to 1.6.0-alpha.5
> ---
>
> Key: SOLR-11385
> URL: https://issues.apache.org/jira/browse/SOLR-11385
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: documentation
>Reporter: Cassandra Targett
>Assignee: Cassandra Targett
>
> A new version of asciidoctor-ant is out, which updates embedded 
> asciidoctor-pdf to 1.5.0.alpha.16. It also includes asciidoctor-diagram, 
> which will make SOLR-11275 much easier to finish up.
> Links to release notes: 
> * [asciidoctor-ant 
> 1.6.0-alpha.5|https://github.com/asciidoctor/asciidoctor-ant/releases/tag/v1.6.0-alpha.5]
> * [asciidoctor-pdf 
> 1.5.0.alpha.16|https://github.com/asciidoctor/asciidoctor-pdf/releases/tag/v1.5.0.alpha.16]
> Since asciidoctor-ant is a .jar, it shouldn't require any other changes (to 
> builds, etc.) besides updating the version in {{ivy-versions.properties}}.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Lucene/Solr Documentation Changes

[Steve Rowe]

+1 to fix the website text, even though that will not match the text in the 
release.

--
Steve
www.lucidworks.com

> On Sep 21, 2017, at 1:16 PM, Anshum Gupta  wrote:
> 
> Hi,
> 
> I just noticed that the migration guide text in the docs here 
> (http://lucene.apache.org/core/7_0_0/index.html) says “What changed in Lucene 
> 6; how to migrate code from Lucene 5.x.”.
> 
> This is picked up from lucene/site/xsl/index.xsl and wasn’t changed during 
> the release.
> 
> I have now changed this on master (to say versions 8 and 7.x), 7x, and 7.0 
> branches to have the correct versions for future releases and I’ll update the 
> wiki for RELEASE TODO, so this isn’t missed in the future.
> 
> This brings me to the question: should we fix this for the already released 
> version on the website i.e. the link I’ve shared above. If we do that, we’d 
> have different text on the documentation website, as compared to what is 
> shipped with Lucene. To be clear, by fixing it, I just intend to fix the text 
> on index.html and commit. 
> 
> Thoughts?
> 
> -Anshum
> 
> 
> 


-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-11385) Update asciidoctor-ant to 1.6.0-alpha.5

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16175156#comment-16175156
 ] 

ASF subversion and git services commented on SOLR-11385:


Commit 239d60104096a89cebeb1267c15efb1a61eecd38 in lucene-solr's branch 
refs/heads/branch_7_0 from [~ctargett]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=239d601 ]

SOLR-11385: Update asciidoctor-ant version to 1.6.0-alpha.5


> Update asciidoctor-ant to 1.6.0-alpha.5
> ---
>
> Key: SOLR-11385
> URL: https://issues.apache.org/jira/browse/SOLR-11385
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: documentation
>Reporter: Cassandra Targett
>Assignee: Cassandra Targett
>
> A new version of asciidoctor-ant is out, which updates embedded 
> asciidoctor-pdf to 1.5.0.alpha.16. It also includes asciidoctor-diagram, 
> which will make SOLR-11275 much easier to finish up.
> Links to release notes: 
> * [asciidoctor-ant 
> 1.6.0-alpha.5|https://github.com/asciidoctor/asciidoctor-ant/releases/tag/v1.6.0-alpha.5]
> * [asciidoctor-pdf 
> 1.5.0.alpha.16|https://github.com/asciidoctor/asciidoctor-pdf/releases/tag/v1.5.0.alpha.16]
> Since asciidoctor-ant is a .jar, it shouldn't require any other changes (to 
> builds, etc.) besides updating the version in {{ivy-versions.properties}}.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-11385) Update asciidoctor-ant to 1.6.0-alpha.5

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16175155#comment-16175155
 ] 

ASF subversion and git services commented on SOLR-11385:


Commit dd59822ac94ee0b5cc2bedcac38f60442c1af5e6 in lucene-solr's branch 
refs/heads/branch_7x from [~ctargett]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=dd59822 ]

SOLR-11385: Update asciidoctor-ant version to 1.6.0-alpha.5


> Update asciidoctor-ant to 1.6.0-alpha.5
> ---
>
> Key: SOLR-11385
> URL: https://issues.apache.org/jira/browse/SOLR-11385
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: documentation
>Reporter: Cassandra Targett
>Assignee: Cassandra Targett
>
> A new version of asciidoctor-ant is out, which updates embedded 
> asciidoctor-pdf to 1.5.0.alpha.16. It also includes asciidoctor-diagram, 
> which will make SOLR-11275 much easier to finish up.
> Links to release notes: 
> * [asciidoctor-ant 
> 1.6.0-alpha.5|https://github.com/asciidoctor/asciidoctor-ant/releases/tag/v1.6.0-alpha.5]
> * [asciidoctor-pdf 
> 1.5.0.alpha.16|https://github.com/asciidoctor/asciidoctor-pdf/releases/tag/v1.5.0.alpha.16]
> Since asciidoctor-ant is a .jar, it shouldn't require any other changes (to 
> builds, etc.) besides updating the version in {{ivy-versions.properties}}.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-11385) Update asciidoctor-ant to 1.6.0-alpha.5

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16175153#comment-16175153
 ] 

ASF subversion and git services commented on SOLR-11385:


Commit 7c77c8778f0a11c4e0608982bf6e364df1135803 in lucene-solr's branch 
refs/heads/master from [~ctargett]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=7c77c87 ]

SOLR-11385: Update asciidoctor-ant version to 1.6.0-alpha.5


> Update asciidoctor-ant to 1.6.0-alpha.5
> ---
>
> Key: SOLR-11385
> URL: https://issues.apache.org/jira/browse/SOLR-11385
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: documentation
>Reporter: Cassandra Targett
>Assignee: Cassandra Targett
>
> A new version of asciidoctor-ant is out, which updates embedded 
> asciidoctor-pdf to 1.5.0.alpha.16. It also includes asciidoctor-diagram, 
> which will make SOLR-11275 much easier to finish up.
> Links to release notes: 
> * [asciidoctor-ant 
> 1.6.0-alpha.5|https://github.com/asciidoctor/asciidoctor-ant/releases/tag/v1.6.0-alpha.5]
> * [asciidoctor-pdf 
> 1.5.0.alpha.16|https://github.com/asciidoctor/asciidoctor-pdf/releases/tag/v1.5.0.alpha.16]
> Since asciidoctor-ant is a .jar, it shouldn't require any other changes (to 
> builds, etc.) besides updating the version in {{ivy-versions.properties}}.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-7863) Don't repeat postings (and perhaps positions) on ReverseWF, EdgeNGram, etc

[Mikhail Khludnev (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-7863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mikhail Khludnev updated LUCENE-7863:
-
Attachment: LUCENE-7863.patch

[^LUCENE-7863.patch] replaces TreeMap to BytesRefArray see 
{{ByteArrayDerivativeWriter.java}}. Here are results for 5M docs
|round|indexing, mins|search req/sec|ram total, GB |index size, GB| 
| EdgeNGramm |85|27.82|2.3|23|
|derived edges|51|7.22|5.5|9.1|
We have index size and even index time gain that costs some ram as it's 
expected. 
EdgeNGramm cache can be made a little bit more compact. The trick is to append 
something to edgegramm to make it unique. 
The interesting thing is the 3 times slower search time, I suppose that posting 
offsets obtained during term expansion could be sorted before reading postings. 
 

> Don't repeat postings (and perhaps positions) on ReverseWF, EdgeNGram, etc  
> 
>
> Key: LUCENE-7863
> URL: https://issues.apache.org/jira/browse/LUCENE-7863
> Project: Lucene - Core
>  Issue Type: Improvement
>  Components: core/index
>Reporter: Mikhail Khludnev
> Attachments: benchmark-1m.out, LUCENE-7863.hazard, LUCENE-7863.patch, 
> LUCENE-7863.patch, LUCENE-7863.patch, LUCENE-7863.patch, LUCENE-7863.patch, 
> LUCENE-7863.patch, LUCENE-7863.patch, LUCENE-7863.patch, LUCENE-7863.patch
>
>
> h2. Context
> \*suffix and \*infix\* searches on large indexes. 
> h2. Problem
> Obviously applying {{ReversedWildcardFilter}} doubles an index size, and I'm 
> shuddering to think about EdgeNGrams...
> h2. Proposal 
> _DRY_



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Lucene/Solr Documentation Changes

[Anshum Gupta]

Hi,

I just noticed that the migration guide text in the docs here 
(http://lucene.apache.org/core/7_0_0/index.html) says “What changed in Lucene 
6; how to migrate code from Lucene 5.x.”.

This is picked up from lucene/site/xsl/index.xsl and wasn’t changed during the 
release.

I have now changed this on master (to say versions 8 and 7.x), 7x, and 7.0 
branches to have the correct versions for future releases and I’ll update the 
wiki for RELEASE TODO, so this isn’t missed in the future.

This brings me to the question: should we fix this for the already released 
version on the website i.e. the link I’ve shared above. If we do that, we’d 
have different text on the documentation website, as compared to what is 
shipped with Lucene. To be clear, by fixing it, I just intend to fix the text 
on index.html and commit. 

Thoughts?

-Anshum

[jira] [Created] (SOLR-11386) Extracting learning to rank features fails when word ordering of EFI argument changed.

[Michael A. Alcorn (JIRA)]

Michael A. Alcorn created SOLR-11386:


 Summary: Extracting learning to rank features fails when word 
ordering of EFI argument changed.
 Key: SOLR-11386
 URL: https://issues.apache.org/jira/browse/SOLR-11386
 Project: Solr
  Issue Type: Bug
  Security Level: Public (Default Security Level. Issues are Public)
  Components: contrib - LTR
Affects Versions: 6.5.1
Reporter: Michael A. Alcorn


I'm getting some extremely strange behavior when trying to extract features for 
a learning to rank model. The following query incorrectly says all features 
have zero values:

http://gss-test-fusion.usersys.redhat.com:8983/solr/access/query?q=added couple 
of fiber channel={!ltr model=redhat_efi_model reRankDocs=1 
efi.case_summary=the efi.case_description=added couple of fiber channel 
efi.case_issue=the efi.case_environment=the}=id,score,[features]=10

But this query, which simply moves the word "added" from the front of the 
provided text to the back, properly fills in the feature values:

http://gss-test-fusion.usersys.redhat.com:8983/solr/access/query?q=couple of 
fiber channel added={!ltr model=redhat_efi_model reRankDocs=1 
efi.case_summary=the efi.case_description=couple of fiber channel added 
efi.case_issue=the efi.case_environment=the}=id,score,[features]=10

The explain output for the failing query can be found here:

https://gist.github.com/manisnesan/18a8f1804f29b1b62ebfae1211f38cc4

and the explain output for the properly functioning query can be found here:

https://gist.github.com/manisnesan/47685a561605e2229434b38aed11cc65



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10962) replicationHandler's reserveCommitDuration configurable in SolrCloud mode

[Christine Poerschke (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16175119#comment-16175119
 ] 

Christine Poerschke commented on SOLR-10962:


Ran precommit and solr/core tests. Committed to master branch today and 
planning to cherry-pick to branch_7x in a couple of days.

_(Any further post-commit comments and input welcome as usual. Thank you.)_

> replicationHandler's reserveCommitDuration configurable in SolrCloud mode
> -
>
> Key: SOLR-10962
> URL: https://issues.apache.org/jira/browse/SOLR-10962
> Project: Solr
>  Issue Type: New Feature
>  Components: replication (java)
>Reporter: Ramsey Haddad
>Assignee: Christine Poerschke
>Priority: Minor
> Attachments: SOLR-10962.patch, SOLR-10962.patch, SOLR-10962.patch, 
> SOLR-10962.patch, SOLR-10962.patch, SOLR-10962.patch, SOLR-10962.patch
>
>
> With SolrCloud mode, when doing replication via IndexFetcher, we occasionally 
> see the Fetch fail and then get restarted from scratch in cases where an 
> Index file is deleted after fetch manifest is computed and before the fetch 
> actually transfers the file. The risk of this happening can be reduced with a 
> higher value of reserveCommitDuration. However, the current configuration 
> only allows this value to be adjusted for "master" mode. This change allows 
> the value to also be changed when using "SolrCloud" mode.
> https://lucene.apache.org/solr/guide/6_6/index-replication.html



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-11297) Message "Lock held by this virtual machine" during startup. Solr is trying to start some cores twice

[Nawab Zada Asad iqbal (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16175113#comment-16175113
 ] 

Nawab Zada Asad iqbal commented on SOLR-11297:
--

FWIW, on my cluster with constant haproxy pings, i was hitting this issue 9/10 
times. And the core was not even usable. So my issue was slightly different 
from Shawn, but I hope that your fix will cover for all the issues. Looking 
forward for the fix. 

> Message "Lock held by this virtual machine" during startup.  Solr is trying 
> to start some cores twice
> -
>
> Key: SOLR-11297
> URL: https://issues.apache.org/jira/browse/SOLR-11297
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Affects Versions: 6.6
>Reporter: Shawn Heisey
>Assignee: Erick Erickson
> Attachments: SOLR-11297.patch, SOLR-11297.patch, SOLR-11297.sh, 
> solr6_6-startup.log
>
>
> Sometimes when Solr is restarted, I get some "lock held by this virtual 
> machine" messages in the log, and the admin UI has messages about a failure 
> to open a new searcher.  It doesn't happen on all cores, and the list of 
> cores that have the problem changes on subsequent restarts.  The cores that 
> exhibit the problems are working just fine -- the first core load is 
> successful, the failure to open a new searcher is on a second core load 
> attempt, which fails.
> None of the cores in the system are sharing an instanceDir or dataDir.  This 
> has been verified several times.
> The index is sharded manually, and the servers are not running in cloud mode.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Comment Edited] (LUCENE-7974) Add N-dimensional FloatPoint K-nearest-neighbor implementation

[Steve Rowe (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16175102#comment-16175102
 ] 

Steve Rowe edited comment on LUCENE-7974 at 9/21/17 5:04 PM:
-

Patch implementing the idea.  Reviews welcome.

Some implementation notes:

* Most of the implementation was directly stolen from {{NearestNeighbor}}, 
which should incidentally be renamed to indicate that it's specific to 
{{LatLonPoint}}; this patch doesn't do that. 
* The new test was also mostly directly stolen from {{TestNearest}}.  I removed 
testing of sorting multi-dimensional points by distance from a given point, 
though, since AFAICT {{FloatPoint}} doesn't have this functionality (see 
LUCENE-7099 for the LatLonPoint impl).
* Most comparisons are of distance squared rather than distance, to avoid 
{{Math.sqrt()}} calls.  There are a couple of (non-squared) distance-based 
optimizations though to disqualify cells and individual point values that are 
beyond the search radius in one dimension.
* I tried to switch the (bounded) hit queue from Java's {{PriorityQueue}} to 
Lucene's implementation (as mentioned in a TODO in the {{LatLonPoint}} impl), 
but it was much much slower (like an order of magnitude-ish).  I didn't look 
into why, but I reverted back to the Java impl.
* Since this is targetted at the sandbox module, I didn't want to modify 
{{FloatPoint}} to include a {{nearest()}} method (as was done on 
{{LatLonPoint}} in LUCENE-7069), so that method is instead included directly on 
{{FloatPointNearestNeighbor}}.  If this gets committed and then eventually 
moved to core, the public {{nearest()}} method should be moved to 
{{FloatPoint}}.


was (Author: steve_rowe):
Patch implementing the idea.  Reviews welcome.

Some implementation notes:

* Most of the implementation was directly stolen from {{NearestNeighbor}}, 
which should incidentally be renamed to indicate that it's specific to 
{{LatLonPoint}}; this patch doesn't do that. 
* The new test was also mostly directly stolen from {{TestNearest}}.  I removed 
testing of sorting multi-dimensional points by distance from a given point, 
though, since AFAICT {{FloatPoint}} doesn't have this functionality (see 
LUCENE-7099 for the LatLonPoint impl).
* Most comparisons are of distance squared rather than distance, to avoid 
{{Math.sqrt()}} calls.  There are a couple of (non-squared) distance-based 
optimizations though to disqualify cells and individual point values that are 
beyond the search radius in one dimension.
* I tried to switch the (bounded) hit queue from Java's {{PriorityQueue}} to 
Lucene's implementation (as mentioned in a TODO in the {{LatLonPoint}} impl), 
but it was much much slower (like an order of magnitude-ish).  I didn't look 
into why, but I reverted back to the Java impl.


> Add N-dimensional FloatPoint K-nearest-neighbor implementation
> --
>
> Key: LUCENE-7974
> URL: https://issues.apache.org/jira/browse/LUCENE-7974
> Project: Lucene - Core
>  Issue Type: New Feature
>  Components: modules/sandbox
>Reporter: Steve Rowe
>Assignee: Steve Rowe
>Priority: Minor
> Attachments: LUCENE-7974.patch
>
>
> From LUCENE-7069:
> {quote}
> KD trees (used by Lucene's new dimensional points) excel at finding "nearest 
> neighbors" to a given query point ... I think we should add this to Lucene's 
> sandbox
> [...]
> It could also be generalized to more than 2 dimensions, but for now I'm 
> making the class package private in sandbox for just the geo2d (lat/lon) use 
> case.
> {quote}
> This issue is to generalize {{LatLonPoint.nearest()}} to more than 2 
> dimensions.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10962) replicationHandler's reserveCommitDuration configurable in SolrCloud mode

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16175110#comment-16175110
 ] 

ASF subversion and git services commented on SOLR-10962:


Commit 56b8ad2f081e1e7d51e7c97fcbd1f0fbfdf07c98 in lucene-solr's branch 
refs/heads/master from [~cpoerschke]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=56b8ad2 ]

SOLR-10962: Make ReplicationHandler's commitReserveDuration configurable in 
SolrCloud mode.
(Ramsey Haddad, Christine Poerschke, hossman)


> replicationHandler's reserveCommitDuration configurable in SolrCloud mode
> -
>
> Key: SOLR-10962
> URL: https://issues.apache.org/jira/browse/SOLR-10962
> Project: Solr
>  Issue Type: New Feature
>  Components: replication (java)
>Reporter: Ramsey Haddad
>Assignee: Christine Poerschke
>Priority: Minor
> Attachments: SOLR-10962.patch, SOLR-10962.patch, SOLR-10962.patch, 
> SOLR-10962.patch, SOLR-10962.patch, SOLR-10962.patch, SOLR-10962.patch
>
>
> With SolrCloud mode, when doing replication via IndexFetcher, we occasionally 
> see the Fetch fail and then get restarted from scratch in cases where an 
> Index file is deleted after fetch manifest is computed and before the fetch 
> actually transfers the file. The risk of this happening can be reduced with a 
> higher value of reserveCommitDuration. However, the current configuration 
> only allows this value to be adjusted for "master" mode. This change allows 
> the value to also be changed when using "SolrCloud" mode.
> https://lucene.apache.org/solr/guide/6_6/index-replication.html



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-7974) Add N-dimensional FloatPoint K-nearest-neighbor implementation

[Steve Rowe (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-7974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Steve Rowe updated LUCENE-7974:
---
Attachment: LUCENE-7974.patch

Patch implementing the idea.  Reviews welcome.

Some implementation notes:

* Most of the implementation was directly stolen from {{NearestNeighbor}}, 
which should incidentally be renamed to indicate that it's specific to 
{{LatLonPoint}}; this patch doesn't do that. 
* The new test was also mostly directly stolen from {{TestNearest}}.  I removed 
testing of sorting multi-dimensional points by distance from a given point, 
though, since AFAICT {{FloatPoint}} doesn't have this functionality (see 
LUCENE-7099 for the LatLonPoint impl).
* Most comparisons are of distance squared rather than distance, to avoid 
{{Math.sqrt()}} calls.  There are a couple of (non-squared) distance-based 
optimizations though to disqualify cells and individual point values that are 
beyond the search radius in one dimension.
* I tried to switch the (bounded) hit queue from Java's {{PriorityQueue}} to 
Lucene's implementation (as mentioned in a TODO in the {{LatLonPoint}} impl), 
but it was much much slower (like an order of magnitude-ish).  I didn't look 
into why, but I reverted back to the Java impl.


> Add N-dimensional FloatPoint K-nearest-neighbor implementation
> --
>
> Key: LUCENE-7974
> URL: https://issues.apache.org/jira/browse/LUCENE-7974
> Project: Lucene - Core
>  Issue Type: New Feature
>  Components: modules/sandbox
>Reporter: Steve Rowe
>Assignee: Steve Rowe
>Priority: Minor
> Attachments: LUCENE-7974.patch
>
>
> From LUCENE-7069:
> {quote}
> KD trees (used by Lucene's new dimensional points) excel at finding "nearest 
> neighbors" to a given query point ... I think we should add this to Lucene's 
> sandbox
> [...]
> It could also be generalized to more than 2 dimensions, but for now I'm 
> making the class package private in sandbox for just the geo2d (lat/lon) use 
> case.
> {quote}
> This issue is to generalize {{LatLonPoint.nearest()}} to more than 2 
> dimensions.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-7.x-Windows (64bit/jdk1.8.0_144) - Build # 203 - Unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-7.x-Windows/203/
Java: 64bit/jdk1.8.0_144 -XX:-UseCompressedOops -XX:+UseSerialGC

1 tests failed.
FAILED:  org.apache.solr.cloud.DocValuesNotIndexedTest.testGroupingDVOnly

Error Message:
Unexpected number of elements in the group for intGSF: 8

Stack Trace:
java.lang.AssertionError: Unexpected number of elements in the group for 
intGSF: 8
at 
__randomizedtesting.SeedInfo.seed([DD81787414D7F7CD:463A162C598FC593]:0)
at org.junit.Assert.fail(Assert.java:93)
at 
org.apache.solr.cloud.DocValuesNotIndexedTest.testGroupingDVOnly(DocValuesNotIndexedTest.java:377)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)




Build Log:
[...truncated 11875 lines...]
   [junit4] Suite: org.apache.solr.cloud.DocValuesNotIndexedTest
   [junit4]   2> Creating dataDir: 

[jira] [Created] (LUCENE-7974) Add N-dimensional FloatPoint K-nearest-neighbor implementation

[Steve Rowe (JIRA)]

Steve Rowe created LUCENE-7974:
--

 Summary: Add N-dimensional FloatPoint K-nearest-neighbor 
implementation
 Key: LUCENE-7974
 URL: https://issues.apache.org/jira/browse/LUCENE-7974
 Project: Lucene - Core
  Issue Type: New Feature
  Components: modules/sandbox
Reporter: Steve Rowe
Assignee: Steve Rowe
Priority: Minor


>From LUCENE-7069:

{quote}
KD trees (used by Lucene's new dimensional points) excel at finding "nearest 
neighbors" to a given query point ... I think we should add this to Lucene's 
sandbox

[...]

It could also be generalized to more than 2 dimensions, but for now I'm making 
the class package private in sandbox for just the geo2d (lat/lon) use case.
{quote}

This issue is to generalize {{LatLonPoint.nearest()}} to more than 2 dimensions.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-11384) add support for distributed graph query

[Kevin Watters (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-11384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Watters updated SOLR-11384:
-
Priority: Minor  (was: Major)

> add support for distributed graph query
> ---
>
> Key: SOLR-11384
> URL: https://issues.apache.org/jira/browse/SOLR-11384
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Kevin Watters
>Priority: Minor
>
> Creating this ticket to track the work that I've done on the distributed 
> graph traversal support in solr.
> Current GraphQuery will only work on a single core, which introduces some 
> limits on where it can be used and also complexities if you want to scale it. 
>  I believe there's a strong desire to support a fully distributed method of 
> doing the Graph Query.  I'm working on a patch, it's not complete yet, but if 
> anyone would like to have a look at the approach and implementation,  I 
> welcome much feedback.  
> The flow for the distributed graph query is almost exactly the same as the 
> normal graph query.  The only difference is how it discovers the "frontier 
> query" at each level of the traversal.  
> When a distribute graph query request comes in, each shard begins by running 
> the root query, to know where to start on it's shard.  Each participating 
> shard then discovers it's edges for the next hop.  Those edges are then 
> broadcast to all other participating shards.  The shard then receives all the 
> parts of the frontier query , assembles it, and executes it.
> This process continues on each shard until there are no new edges left, or 
> the maxDepth of the traversal has finished.
> The approach is to introduce a FrontierBroker that resides as a singleton on 
> each one of the solr nodes in the cluster.  When a graph query is created, it 
> can do a getInstance() on it so it can listen on the frontier parts coming in.
> Initially, I was using an external Kafka broker to handle this, and it did 
> work pretty well.  The new approach is migrating the FrontierBroker to be a 
> request handler in Solr, and potentially to use the SolrJ client to publish 
> the edges to each node in the cluster.
> There are a few outstanding design questions, first being, how do we know 
> what the list of shards are that are participating in the current query 
> request?  Is that easy info to get at?
> Second,  currently, we are serializing a query object between the shards, 
> perhaps we should consider a slightly different abstraction, and serialize 
> lists of "edge" objects between the nodes.   The point of this would be to 
> batch the exploration/traversal of current frontier to help avoid large 
> bursts of memory being required.
> Thrid, what sort of caching strategy should be introduced for the frontier 
> queries, if any?  And if we do some caching there, how/when should the 
> entries be expired and auto-warmed.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-11385) Update asciidoctor-ant to 1.6.0-alpha.5

[Cassandra Targett (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16175030#comment-16175030
 ] 

Cassandra Targett commented on SOLR-11385:
--

Actually, I'm wrong about asciidoctor-diagram being included :(

> Update asciidoctor-ant to 1.6.0-alpha.5
> ---
>
> Key: SOLR-11385
> URL: https://issues.apache.org/jira/browse/SOLR-11385
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: documentation
>Reporter: Cassandra Targett
>Assignee: Cassandra Targett
>
> A new version of asciidoctor-ant is out, which updates embedded 
> asciidoctor-pdf to 1.5.0.alpha.16. It also includes asciidoctor-diagram, 
> which will make SOLR-11275 much easier to finish up.
> Links to release notes: 
> * [asciidoctor-ant 
> 1.6.0-alpha.5|https://github.com/asciidoctor/asciidoctor-ant/releases/tag/v1.6.0-alpha.5]
> * [asciidoctor-pdf 
> 1.5.0.alpha.16|https://github.com/asciidoctor/asciidoctor-pdf/releases/tag/v1.5.0.alpha.16]
> Since asciidoctor-ant is a .jar, it shouldn't require any other changes (to 
> builds, etc.) besides updating the version in {{ivy-versions.properties}}.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS-EA] Lucene-Solr-7.x-Linux (64bit/jdk-9-ea+181) - Build # 454 - Still Unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-7.x-Linux/454/
Java: 64bit/jdk-9-ea+181 -XX:-UseCompressedOops -XX:+UseG1GC 
--illegal-access=deny

2 tests failed.
FAILED:  
org.apache.solr.client.solrj.io.stream.StreamExpressionTest.testExecutorStream

Error Message:
Error from server at https://127.0.0.1:45531/solr/mainCorpus_shard2_replica_n2: 
Expected mime type application/octet-stream but got text/html.   
 
Error 404HTTP ERROR: 404 Problem 
accessing /solr/mainCorpus_shard2_replica_n2/update. Reason: Can not 
find: /solr/mainCorpus_shard2_replica_n2/update http://eclipse.org/jetty;>Powered by Jetty:// 9.3.20.v20170531 
  

Stack Trace:
org.apache.solr.client.solrj.impl.CloudSolrClient$RouteException: Error from 
server at https://127.0.0.1:45531/solr/mainCorpus_shard2_replica_n2: Expected 
mime type application/octet-stream but got text/html. 


Error 404 


HTTP ERROR: 404
Problem accessing /solr/mainCorpus_shard2_replica_n2/update. Reason:
Can not find: /solr/mainCorpus_shard2_replica_n2/update
http://eclipse.org/jetty;>Powered by Jetty:// 
9.3.20.v20170531



at 
__randomizedtesting.SeedInfo.seed([EFD66F0C85D5BEAC:CD16EEF7A6BF94BC]:0)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.directUpdate(CloudSolrClient.java:539)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:993)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:862)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:793)
at 
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:178)
at 
org.apache.solr.client.solrj.request.UpdateRequest.commit(UpdateRequest.java:233)
at 
org.apache.solr.client.solrj.io.stream.StreamExpressionTest.testExecutorStream(StreamExpressionTest.java:7192)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:564)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 

[jira] [Created] (SOLR-11385) Update asciidoctor-ant to 1.6.0-alpha.5

[Cassandra Targett (JIRA)]

Cassandra Targett created SOLR-11385:


 Summary: Update asciidoctor-ant to 1.6.0-alpha.5
 Key: SOLR-11385
 URL: https://issues.apache.org/jira/browse/SOLR-11385
 Project: Solr
  Issue Type: Improvement
  Security Level: Public (Default Security Level. Issues are Public)
  Components: documentation
Reporter: Cassandra Targett
Assignee: Cassandra Targett


A new version of asciidoctor-ant is out, which updates embedded asciidoctor-pdf 
to 1.5.0.alpha.16. It also includes asciidoctor-diagram, which will make 
SOLR-11275 much easier to finish up.

Links to release notes: 

* [asciidoctor-ant 
1.6.0-alpha.5|https://github.com/asciidoctor/asciidoctor-ant/releases/tag/v1.6.0-alpha.5]
* [asciidoctor-pdf 
1.5.0.alpha.16|https://github.com/asciidoctor/asciidoctor-pdf/releases/tag/v1.5.0.alpha.16]

Since asciidoctor-ant is a .jar, it shouldn't require any other changes (to 
builds, etc.) besides updating the version in {{ivy-versions.properties}}.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (SOLR-11383) Interaction between Solr and Zookeeper?

[Erick Erickson (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-11383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Erick Erickson resolved SOLR-11383.
---
Resolution: Not A Problem

Please raise informational questions on the user's list at 
solr-u...@lucene.apache.org, see: 
(http://lucene.apache.org/solr/community.html#mailing-lists-irc) there are a 
_lot_ more people watching that list who may be able to help. 

We try to reserve the JIRA system for code issues (bugs, enhancements and the 
like).


> Interaction between Solr and Zookeeper?
> ---
>
> Key: SOLR-11383
> URL: https://issues.apache.org/jira/browse/SOLR-11383
> Project: Solr
>  Issue Type: Wish
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Leviya Bl
>
> I've been working on a system where they use SolrCloud, which entails a 
> Zookeeper ensemble that helps "manage the overall structure so that both 
> indexing and search requests can be routed properly" (straight out of the 
> ([Solr 
> Documentation|https://lucene.apache.org/solr/guide/6_6/getting-started-with-solrcloud.html]).
> What exactly is this "management"? What information, what 
> data/configuration/information do the machines running Solr read/write from 
> the Zookeeper ensemble and why? Is the data in Zookeeper ever changed at 
> runtime by [SOLR|https://mindmajix.com/apache-solr-training]? Or do you 
> configure "the data" once and runtime is going to be reads all across 
> SolrCloud hosts?
> To put the question into perspective, this is my first contact with 
> Zookeeper, Solr, and in many ways with distributed systems.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-11384) add support for distributed graph query

[Kevin Watters (JIRA)]

Kevin Watters created SOLR-11384:


 Summary: add support for distributed graph query
 Key: SOLR-11384
 URL: https://issues.apache.org/jira/browse/SOLR-11384
 Project: Solr
  Issue Type: Bug
  Security Level: Public (Default Security Level. Issues are Public)
Reporter: Kevin Watters


Creating this ticket to track the work that I've done on the distributed graph 
traversal support in solr.

Current GraphQuery will only work on a single core, which introduces some 
limits on where it can be used and also complexities if you want to scale it.  
I believe there's a strong desire to support a fully distributed method of 
doing the Graph Query.  I'm working on a patch, it's not complete yet, but if 
anyone would like to have a look at the approach and implementation,  I welcome 
much feedback.  

The flow for the distributed graph query is almost exactly the same as the 
normal graph query.  The only difference is how it discovers the "frontier 
query" at each level of the traversal.  

When a distribute graph query request comes in, each shard begins by running 
the root query, to know where to start on it's shard.  Each participating shard 
then discovers it's edges for the next hop.  Those edges are then broadcast to 
all other participating shards.  The shard then receives all the parts of the 
frontier query , assembles it, and executes it.

This process continues on each shard until there are no new edges left, or the 
maxDepth of the traversal has finished.

The approach is to introduce a FrontierBroker that resides as a singleton on 
each one of the solr nodes in the cluster.  When a graph query is created, it 
can do a getInstance() on it so it can listen on the frontier parts coming in.

Initially, I was using an external Kafka broker to handle this, and it did work 
pretty well.  The new approach is migrating the FrontierBroker to be a request 
handler in Solr, and potentially to use the SolrJ client to publish the edges 
to each node in the cluster.

There are a few outstanding design questions, first being, how do we know what 
the list of shards are that are participating in the current query request?  Is 
that easy info to get at?

Second,  currently, we are serializing a query object between the shards, 
perhaps we should consider a slightly different abstraction, and serialize 
lists of "edge" objects between the nodes.   The point of this would be to 
batch the exploration/traversal of current frontier to help avoid large bursts 
of memory being required.

Thrid, what sort of caching strategy should be introduced for the frontier 
queries, if any?  And if we do some caching there, how/when should the entries 
be expired and auto-warmed.







--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-11297) Message "Lock held by this virtual machine" during startup. Solr is trying to start some cores twice

[Erick Erickson (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16174900#comment-16174900
 ] 

Erick Erickson commented on SOLR-11297:
---

P.S. I spent probably more time than I should have trying to make an automated 
test for this too, but failed miserably. It _seems_ like a pair of threads that 
queried/pinged/reloaded should have tripped this but somehow it didn't work for 
me.

> Message "Lock held by this virtual machine" during startup.  Solr is trying 
> to start some cores twice
> -
>
> Key: SOLR-11297
> URL: https://issues.apache.org/jira/browse/SOLR-11297
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Affects Versions: 6.6
>Reporter: Shawn Heisey
>Assignee: Erick Erickson
> Attachments: SOLR-11297.patch, SOLR-11297.patch, SOLR-11297.sh, 
> solr6_6-startup.log
>
>
> Sometimes when Solr is restarted, I get some "lock held by this virtual 
> machine" messages in the log, and the admin UI has messages about a failure 
> to open a new searcher.  It doesn't happen on all cores, and the list of 
> cores that have the problem changes on subsequent restarts.  The cores that 
> exhibit the problems are working just fine -- the first core load is 
> successful, the failure to open a new searcher is on a second core load 
> attempt, which fails.
> None of the cores in the system are sharing an instanceDir or dataDir.  This 
> has been verified several times.
> The index is sharded manually, and the servers are not running in cloud mode.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-11297) Message "Lock held by this virtual machine" during startup. Solr is trying to start some cores twice

[Erick Erickson (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-11297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Erick Erickson updated SOLR-11297:
--
Attachment: SOLR-11297.patch

This passes the shell script test (Thanks Luiz!) All tests pass.

This surrounds all the instances of createFromDescriptor with a call to 
waitAddPendingCoreOps/removeFromPendingOps. I think there were several more 
potential problems, create, load and reload all were unprotected.

Also added a fairly detailed comment on why moving this to createFromDescriptor 
won't work.

Shawn: 
Can you give this a spin? If you still see the errors let me know, plus a 
little more detail about your setup. Or is it just you have a bunch of cores 
and are pinging when starting Solr?


> Message "Lock held by this virtual machine" during startup.  Solr is trying 
> to start some cores twice
> -
>
> Key: SOLR-11297
> URL: https://issues.apache.org/jira/browse/SOLR-11297
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Affects Versions: 6.6
>Reporter: Shawn Heisey
>Assignee: Erick Erickson
> Attachments: SOLR-11297.patch, SOLR-11297.patch, SOLR-11297.sh, 
> solr6_6-startup.log
>
>
> Sometimes when Solr is restarted, I get some "lock held by this virtual 
> machine" messages in the log, and the admin UI has messages about a failure 
> to open a new searcher.  It doesn't happen on all cores, and the list of 
> cores that have the problem changes on subsequent restarts.  The cores that 
> exhibit the problems are working just fine -- the first core load is 
> successful, the failure to open a new searcher is on a second core load 
> attempt, which fails.
> None of the cores in the system are sharing an instanceDir or dataDir.  This 
> has been verified several times.
> The index is sharded manually, and the servers are not running in cloud mode.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-master-Windows (64bit/jdk1.8.0_144) - Build # 6912 - Unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Windows/6912/
Java: 64bit/jdk1.8.0_144 -XX:+UseCompressedOops -XX:+UseConcMarkSweepGC

2 tests failed.
FAILED:  
junit.framework.TestSuite.org.apache.solr.handler.component.SuggestComponentContextFilterQueryTest

Error Message:
Could not remove the following files (in the order of attempts):
C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J0\temp\solr.handler.component.SuggestComponentContextFilterQueryTest_3F7610F505D7B357-001\init-core-data-001:
 java.nio.file.AccessDeniedException: 
C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J0\temp\solr.handler.component.SuggestComponentContextFilterQueryTest_3F7610F505D7B357-001\init-core-data-001

C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J0\temp\solr.handler.component.SuggestComponentContextFilterQueryTest_3F7610F505D7B357-001:
 java.nio.file.DirectoryNotEmptyException: 
C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J0\temp\solr.handler.component.SuggestComponentContextFilterQueryTest_3F7610F505D7B357-001
 

Stack Trace:
java.io.IOException: Could not remove the following files (in the order of 
attempts):
   
C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J0\temp\solr.handler.component.SuggestComponentContextFilterQueryTest_3F7610F505D7B357-001\init-core-data-001:
 java.nio.file.AccessDeniedException: 
C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J0\temp\solr.handler.component.SuggestComponentContextFilterQueryTest_3F7610F505D7B357-001\init-core-data-001
   
C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J0\temp\solr.handler.component.SuggestComponentContextFilterQueryTest_3F7610F505D7B357-001:
 java.nio.file.DirectoryNotEmptyException: 
C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J0\temp\solr.handler.component.SuggestComponentContextFilterQueryTest_3F7610F505D7B357-001

at __randomizedtesting.SeedInfo.seed([3F7610F505D7B357]:0)
at org.apache.lucene.util.IOUtils.rm(IOUtils.java:329)
at 
org.apache.lucene.util.TestRuleTemporaryFilesCleanup.afterAlways(TestRuleTemporaryFilesCleanup.java:216)
at 
com.carrotsearch.randomizedtesting.rules.TestRuleAdapter$1.afterAlways(TestRuleAdapter.java:31)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:43)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)


FAILED:  
org.apache.solr.client.solrj.io.stream.StreamExpressionTest.testParallelExecutorStream

Error Message:
Error from server at https://127.0.0.1:50919/solr/workQueue_shard2_replica_n2: 
Expected mime type application/octet-stream but got text/html.   
 
Error 404HTTP ERROR: 404 Problem 
accessing /solr/workQueue_shard2_replica_n2/update. Reason: Can not 
find: /solr/workQueue_shard2_replica_n2/update http://eclipse.org/jetty;>Powered by Jetty:// 9.3.20.v20170531 
  

Stack Trace:
org.apache.solr.client.solrj.impl.CloudSolrClient$RouteException: Error from 
server at https://127.0.0.1:50919/solr/workQueue_shard2_replica_n2: Expected 
mime type application/octet-stream but got text/html. 


Error 404 


HTTP ERROR: 404
Problem accessing /solr/workQueue_shard2_replica_n2/update. Reason:
Can not find: /solr/workQueue_shard2_replica_n2/update
http://eclipse.org/jetty;>Powered by Jetty:// 
9.3.20.v20170531



at 
__randomizedtesting.SeedInfo.seed([46FDB432B3527891:FBEAC12B8A7E45CC]:0)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.directUpdate(CloudSolrClient.java:539)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:993)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:862)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:793)
at 
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:178)
at 
org.apache.solr.client.solrj.request.UpdateRequest.commit(UpdateRequest.java:233)
at 

[jira] [Commented] (LUCENE-7970) Improve generation of circle plane

[Ignacio Vera (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16174740#comment-16174740
 ] 

Ignacio Vera commented on LUCENE-7970:
--

(1) I was having problems with the maths so before going foward  in trying to 
solve/formalize it I wanted to see if the approach I was taken would be 
considered. First thing we need to do then, it is to formalize the way to 
compute the planes. I know there is something missing; there is a maximum 
cutoff angle which depends on the spheriod(ab,c) where the outerEllipse does 
not make sense and should be the whole planet. The same with the innerEllipse.  

(2) Ok, I think we can use only the outer plane as it is bigger than the shape. 
If null then bounds should be the whole world.Using the inner plane can give 
bounds smaller than the shape and we might want to avoid that.

(3) upps, my mistake ... shapes should not be public. 

I will try then to find the way to compute the planes that provides the desired 
inner and outer plane in a formal way. 

Thanks!



> Improve generation of circle plane 
> ---
>
> Key: LUCENE-7970
> URL: https://issues.apache.org/jira/browse/LUCENE-7970
> Project: Lucene - Core
>  Issue Type: Improvement
>  Components: modules/spatial3d
>Reporter: Ignacio Vera
>Assignee: Karl Wright
> Attachments: LUCENE_7970.patch, LUCENE-7970.patch, 
> LUCENE-7970-proposed.patch
>
>
> Hi [~Karl wright],
> How circles are currently build do not behave very well when the planet model 
> is not an sphere. when you are close to the border in WGS84 you might get 
> false positves or false negatives when checking if a point is WITHIN. I think 
> the reason is how the points to generate the circle plane are generated which 
> assumes a sphere.
> My proposal is the following:
> Add a new method to PlanetModel:  
> public GeoPoint pointOnBearing(GeoPoint from, double dist, double bearing);
> Which uses and algorithm that takes into account that the planet might not be 
> spherical. For example Vincenty's formulae 
> (https://en.wikipedia.org/wiki/Vincenty%27s_formulae).
> Use this method to generate the points for the circle plane. My experiments 
> shows that this approach removes false negatives in WGS84 meanwhile it works 
> nicely in the Sphere.
> Does it make sense?



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-10610) Add CanaryComponent, a search component to analyse requests

[Domenico Fabio Marino (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-10610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Domenico Fabio Marino updated SOLR-10610:
-
Attachment: SOLR-10610.patch

Updated patch to lessen the changes in QueryComponent.java

> Add CanaryComponent, a search component to analyse requests
> ---
>
> Key: SOLR-10610
> URL: https://issues.apache.org/jira/browse/SOLR-10610
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Domenico Fabio Marino
>Priority: Minor
> Attachments: SOLR-10610.patch, SOLR-10610.patch, SOLR-10610.patch
>
>
> This patch proposes a new component that analyses a request and reports 
> whether it is too complex to continue processing.
> This patch makes active use of SOLR-10880 , SOLR-10881 and SOLR-11343 .
> This component is meant to help with troublesome queries, as it allows to 
> perform a query on a subset of replicas (that could be isolated from user 
> traffic) to verify if the query execution is safe.
> Running this component is conditional and if enabled, will happen before 
> other components start processing.
> This is ensured by the fact that the whole execution of this component 
> happens in the {{STAGE_PARSE_QUERY}} (SOLR-10609 proposes to rename this 
> stage to a more meaningful name).
> CanaryComponent reports the status of the analysis as a Boolean via the 
> {{CanaryComponent.CANARY_SUCCESS}} response parameter and through a field in 
> {{ResponseBuilder}}.
> The same return convention applies to both:
> * {{null/non-existing}} when the CanaryComponent did not execute the query
> * {{true}} if the CanaryComponent processed the query and did not find any 
> problem
> * {{false}} if the query execution didn't terminate normally.
> The CanaryComponent needs to be properly set-up before being used.
> *Set-up:*
> 1) Tagging one or more replicas as "canary" replicas (Depends on SOLR-10880 
> and SOLR-10881):
>   Using replica properties, set a property (this can be independent from any 
> other shard filtering property, but it is not compulsory for it to be such), 
> to a value (canary type).
>   A collection can have many canaries of many types, for example:
>  *  {{shard1replica3, shard2replica1}} have the property 
> {{canaryColour=yellow}}
>  *  {{shard4replica2}} has the property {{canaryColour=red}}
>   And so on.
>   There can be multiple canary replicas per shard.
>   {{CanaryComponentTest.java}} shows an example of such tagging.
> 2) The CanaryComponent needs to be added to the {{/select}} RequestHandler 
> (Example included in the {{cloud-canary}} test config files).
>   Optional but encouraged: set the flag {{canary.timeout}} to a sensible Long 
> (time in milliseconds) value, this will ensure that all the requests have a 
> timeout specified.
>   Note: the timeout can be specified on a per-request basis.
> This concludes the initial set-up.
> *Usage*
> For each request that needs to be run through the CanaryComponent the 
> following parameters have to be added (depends on SOLR-10880):
> {{filterByReplicaProp=true}}
> and
> {{canary=CANARY_TYPE_PROPERTY:CANARY_TYPE}}
> A timeout needs to be specified, but for convenience it can be specified as 
> mentioned in point 2.
> Running a query on the canary without a timeout is not permitted, and an 
> exception will be thrown.
> For example: 
> {code:java}filterByReplicaProp=true=birdColour:yellow=5000{code}
> This means that the request needs the replica filtering framework enabled 
> (See SOLR-10880), that the canary requests will have to be routed to the 
> replicas having the property {{birdColour}} set to {{yellow}} and that this 
> request should timeout after 5 seconds.
> An example of the requests can be seen in {{CanaryComponentTest.java}}
> The request will only run on one canary replica, but if there are multiple 
> replicas matching {{CANARY_TYPE_PROPERTY:CANARY_TYPE}}, a random one will be 
> picked among them, should it be unreachable, another random one will be 
> chosen, and so on.
> Example:
> Given the following list of replicas matching 
> {{CANARY_TYPE_PROPERTY:CANARY_TYPE}} looking like {code:java}["", 
> "shard2replica2", "", "shard4replica3|shard4replica2"]{code}
> The component will rearrange them to look like this list 
> {code:java}["shard2replica2|shard4replica3|shard4replica2"]{code}
> The request will be executed exactly how the QueryComponent would execute it 
> (depends on SOLR-11343), this is done to ensure that the analysis is as 
> realistic as possible, however its execution will be performed in a separate 
> thread.
> This is done so that eventual exceptions thrown by the query can be caught, 
> and that its time of execution can be monitored at a finer level, the 
> execution of the query is 

[jira] [Comment Edited] (LUCENE-7970) Improve generation of circle plane

[Ignacio Vera (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16174442#comment-16174442
 ] 

Ignacio Vera edited comment on LUCENE-7970 at 9/21/17 1:02 PM:
---

Hi [~karl wright],

Not sure if we have something to do here as I think current implementation 
might be the best one to approximate the circle using one plane intersecting 
the spheroid. I don't think I can assure that any other implementation would be 
more precise.

Anyway, I was thinking what is needed to support circles in a spheroid and of 
course it needs a totally different approach and eventually in some cases you 
need to use brute force. I want to share what I think it would be a viable 
solution if Geo3d ever needs to add precision to circles in spheriods like 
WGS84:

The new shape should have two planes:

 innerEllipse: plane cutting the spheriod  producing the maximum ellipse 
within the circle. Anything within this plane is within the circle.
 outerEllipse: plane cutting the spheriod  producing the minimum ellipse 
contaning the circle. Anything not within this plane is not within the circle.

There is only one gray area between the two planes (within outerEllipse, not 
within innerEllipse) where brute force will be needed. 

I attach my implementation for such a shape. It seems to work for WGS84, except 
for verly large and very small angles. I was unable to find the mathematical 
model to generate those planes (is that possible to solve mathematically?). 

Thanks!


was (Author: ivera):
Hi [~karl wright],

Not sure if we have something to do here as I think current implementation 
might be the best one to approximate the circle using one plane intersecting 
the spheroid. I don't think I can assure that any other implementation would be 
more precise.

Anyway, I was thinking what is needed to support circles in a spheroid and of 
course it needs a totally different approach and eventually in some cases you 
need to use brute force. I want to share what I think it would be a viable 
solution if Geo3d ever needs to add precision to circles in spheriods like 
WGS84:

The new shape should have two planes:

 innerElipse: plane cutting the spheriod  producing the maximum ellipse 
within the circle. Anything within this plane is within the circle.
 outerElipse: plane cutting the spheriod  producing the maximum ellipse 
contaning the circle. Anything not within this plane is not within the circle.

There is only one gray area between the two planes (within outerEllipse, not 
within innerEllipse) where brute force will be needed. 

I attach my implementation for such a shape. It seems to work for WGS84, except 
for verly large and very small angles. I was unable to find the mathematical 
model to generate those planes (is that possible to solve mathematically?). 

Thanks!

> Improve generation of circle plane 
> ---
>
> Key: LUCENE-7970
> URL: https://issues.apache.org/jira/browse/LUCENE-7970
> Project: Lucene - Core
>  Issue Type: Improvement
>  Components: modules/spatial3d
>Reporter: Ignacio Vera
>Assignee: Karl Wright
> Attachments: LUCENE_7970.patch, LUCENE-7970.patch, 
> LUCENE-7970-proposed.patch
>
>
> Hi [~Karl wright],
> How circles are currently build do not behave very well when the planet model 
> is not an sphere. when you are close to the border in WGS84 you might get 
> false positves or false negatives when checking if a point is WITHIN. I think 
> the reason is how the points to generate the circle plane are generated which 
> assumes a sphere.
> My proposal is the following:
> Add a new method to PlanetModel:  
> public GeoPoint pointOnBearing(GeoPoint from, double dist, double bearing);
> Which uses and algorithm that takes into account that the planet might not be 
> spherical. For example Vincenty's formulae 
> (https://en.wikipedia.org/wiki/Vincenty%27s_formulae).
> Use this method to generate the points for the circle plane. My experiments 
> shows that this approach removes false negatives in WGS84 meanwhile it works 
> nicely in the Sphere.
> Does it make sense?



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-7.x-Linux (32bit/jdk1.8.0_144) - Build # 453 - Still Unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-7.x-Linux/453/
Java: 32bit/jdk1.8.0_144 -client -XX:+UseParallelGC

1 tests failed.
FAILED:  org.apache.solr.cloud.LeaderElectionContextKeyTest.test

Error Message:


Stack Trace:
java.lang.AssertionError
at 
__randomizedtesting.SeedInfo.seed([BCE70ECA02C76721:34B33110AC3B0AD9]:0)
at org.junit.Assert.fail(Assert.java:92)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertTrue(Assert.java:54)
at 
org.apache.solr.cloud.LeaderElectionContextKeyTest.test(LeaderElectionContextKeyTest.java:88)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)




Build Log:
[...truncated 12258 lines...]
   [junit4] Suite: org.apache.solr.cloud.LeaderElectionContextKeyTest
   [junit4]   2> 1129961 INFO  
(SUITE-LeaderElectionContextKeyTest-seed#[BCE70ECA02C76721]-worker) [] 
o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: 
test.solr.allowed.securerandom=null & 

[jira] [Commented] (LUCENE-7970) Improve generation of circle plane

[Karl Wright (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16174675#comment-16174675
 ] 

Karl Wright commented on LUCENE-7970:
-

[~ivera] I think this is a good addition to the stable of circle 
implementations, and I like how you solve the problem in a way that's as cheap 
as possible.

I have a few things that we should discuss:

(1) For the initial computation of the inner and outer planes, I would like 
more comments that explain exactly what the math is behind the construction of 
these planes.  It's currently unclear where that came from and I think we 
really need to know it.

(2) The getBounds() method shouldn't bother being exact; it should just use the 
outer plane (if present), or the inner plane (if not present) to set the 
bounds.  Bounds are defined as possibly being larger than the shape they 
contain, so that is OK.  Also, we need to be sure xyz bounds work OK as well; 
just doing points doesn't work for XYZ bounds -- you need the planes too.  
Maybe you can just use both planes; that would be safest.

(3) We shouldn't make the class public; it should be class scoped.  Instead, 
add a factory method to GeoCircleFactory.

Thanks!!


> Improve generation of circle plane 
> ---
>
> Key: LUCENE-7970
> URL: https://issues.apache.org/jira/browse/LUCENE-7970
> Project: Lucene - Core
>  Issue Type: Improvement
>  Components: modules/spatial3d
>Reporter: Ignacio Vera
>Assignee: Karl Wright
> Attachments: LUCENE_7970.patch, LUCENE-7970.patch, 
> LUCENE-7970-proposed.patch
>
>
> Hi [~Karl wright],
> How circles are currently build do not behave very well when the planet model 
> is not an sphere. when you are close to the border in WGS84 you might get 
> false positves or false negatives when checking if a point is WITHIN. I think 
> the reason is how the points to generate the circle plane are generated which 
> assumes a sphere.
> My proposal is the following:
> Add a new method to PlanetModel:  
> public GeoPoint pointOnBearing(GeoPoint from, double dist, double bearing);
> Which uses and algorithm that takes into account that the planet might not be 
> spherical. For example Vincenty's formulae 
> (https://en.wikipedia.org/wiki/Vincenty%27s_formulae).
> Use this method to generate the points for the circle plane. My experiments 
> shows that this approach removes false negatives in WGS84 meanwhile it works 
> nicely in the Sphere.
> Does it make sense?



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-NightlyTests-7.0 - Build # 46 - Failure

[Apache Jenkins Server]

Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-7.0/46/

14 tests failed.
FAILED:  org.apache.lucene.index.TestDuelingCodecsAtNight.testBigEquals

Error Message:
Test abandoned because suite timeout was reached.

Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
at __randomizedtesting.SeedInfo.seed([60889AF050FAA00D]:0)


FAILED:  
junit.framework.TestSuite.org.apache.lucene.index.TestDuelingCodecsAtNight

Error Message:
Suite timeout exceeded (>= 720 msec).

Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 720 msec).
at __randomizedtesting.SeedInfo.seed([60889AF050FAA00D]:0)


FAILED:  org.apache.lucene.index.TestIndexSorting.testRandom3

Error Message:
Test abandoned because suite timeout was reached.

Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
at __randomizedtesting.SeedInfo.seed([60889AF050FAA00D]:0)


FAILED:  junit.framework.TestSuite.org.apache.lucene.index.TestIndexSorting

Error Message:
Suite timeout exceeded (>= 720 msec).

Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 720 msec).
at __randomizedtesting.SeedInfo.seed([60889AF050FAA00D]:0)


FAILED:  
org.apache.lucene.index.TestIndexingSequenceNumbers.testStressConcurrentCommit

Error Message:
this IndexWriter is closed

Stack Trace:
org.apache.lucene.store.AlreadyClosedException: this IndexWriter is closed
at org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:896)
at org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:910)
at org.apache.lucene.index.IndexWriter.commit(IndexWriter.java:3377)
at 
org.apache.lucene.index.TestIndexingSequenceNumbers.testStressConcurrentCommit(TestIndexingSequenceNumbers.java:230)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 

[JENKINS] Lucene-Solr-master-Solaris (64bit/jdk1.8.0) - Build # 1427 - Still Unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Solaris/1427/
Java: 64bit/jdk1.8.0 -XX:+UseCompressedOops -XX:+UseParallelGC

1 tests failed.
FAILED:  org.apache.solr.update.HardAutoCommitTest.testCommitWithin

Error Message:
Exception during query

Stack Trace:
java.lang.RuntimeException: Exception during query
at 
__randomizedtesting.SeedInfo.seed([E2A317C326CCF72D:587178BBA5E21938]:0)
at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:884)
at 
org.apache.solr.update.HardAutoCommitTest.testCommitWithin(HardAutoCommitTest.java:100)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.RuntimeException: REQUEST FAILED: 
xpath=//result[@numFound=1]
xml response was: 

00


request was:q=id:529==0=20=2.2
at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:877)
... 40 more




Build Log:
[...truncated 11955 lines...]
   [junit4] Suite: 

[JENKINS] Lucene-Solr-7.x-Linux (64bit/jdk1.8.0_144) - Build # 452 - Unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-7.x-Linux/452/
Java: 64bit/jdk1.8.0_144 -XX:+UseCompressedOops -XX:+UseConcMarkSweepGC

2 tests failed.
FAILED:  
junit.framework.TestSuite.org.apache.solr.cloud.TestSolrCloudWithSecureImpersonation

Error Message:
3 threads leaked from SUITE scope at 
org.apache.solr.cloud.TestSolrCloudWithSecureImpersonation: 1) 
Thread[id=19968, name=jetty-launcher-2753-thread-1-SendThread(127.0.0.1:38031), 
state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation] 
at java.lang.Thread.sleep(Native Method) at 
org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101)
 at 
org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:997)
 at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1060)
2) Thread[id=19900, name=jetty-launcher-2753-thread-2-EventThread, 
state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation] 
at sun.misc.Unsafe.park(Native Method) at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) 
at 
java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037)
 at 
java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)
 at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277)  
   at 
org.apache.curator.CuratorZookeeperClient.internalBlockUntilConnectedOrTimedOut(CuratorZookeeperClient.java:323)
 at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:105)  
   at 
org.apache.curator.framework.imps.GetDataBuilderImpl.pathInForeground(GetDataBuilderImpl.java:288)
 at 
org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:279)
 at 
org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:41)
 at 
org.apache.curator.framework.recipes.shared.SharedValue.readValue(SharedValue.java:244)
 at 
org.apache.curator.framework.recipes.shared.SharedValue.access$100(SharedValue.java:44)
 at 
org.apache.curator.framework.recipes.shared.SharedValue$1.process(SharedValue.java:61)
 at 
org.apache.curator.framework.imps.NamespaceWatcher.process(NamespaceWatcher.java:67)
 at 
org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:530)   
  at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:505)   
 3) Thread[id=19969, name=jetty-launcher-2753-thread-1-EventThread, 
state=WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation] at 
sun.misc.Unsafe.park(Native Method) at 
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
 at 
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) 
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:501)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 3 threads leaked from SUITE 
scope at org.apache.solr.cloud.TestSolrCloudWithSecureImpersonation: 
   1) Thread[id=19968, 
name=jetty-launcher-2753-thread-1-SendThread(127.0.0.1:38031), 
state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation]
at java.lang.Thread.sleep(Native Method)
at 
org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101)
at 
org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:997)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1060)
   2) Thread[id=19900, name=jetty-launcher-2753-thread-2-EventThread, 
state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation]
at sun.misc.Unsafe.park(Native Method)
at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at 
java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037)
at 
java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)
at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277)
at 
org.apache.curator.CuratorZookeeperClient.internalBlockUntilConnectedOrTimedOut(CuratorZookeeperClient.java:323)
at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:105)
at 
org.apache.curator.framework.imps.GetDataBuilderImpl.pathInForeground(GetDataBuilderImpl.java:288)
at 
org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:279)
at 
org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:41)
at 
org.apache.curator.framework.recipes.shared.SharedValue.readValue(SharedValue.java:244)
at 

[jira] [Updated] (SOLR-10880) Support replica filtering by tag: shards.filter=replicaProp.region:EMEA

[Domenico Fabio Marino (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-10880?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Domenico Fabio Marino updated SOLR-10880:
-
Attachment: SOLR-10880.patch

Updated patch to match Christine's suggestions.
There's a small behaviour change, it is now safe to run ShardParams.FILTERNOT 
with a property/value that is not existing on any replica.

> Support replica filtering by tag: shards.filter=replicaProp.region:EMEA
> ---
>
> Key: SOLR-10880
> URL: https://issues.apache.org/jira/browse/SOLR-10880
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Domenico Fabio Marino
> Attachments: SOLR-10880.patch, SOLR-10880.patch, SOLR-10880.patch, 
> SOLR-10880.patch, SOLR-10880.patch, SOLR-10880.patch, SOLR-10880.patch
>
>
> Add a mechanism to allow queries to use only a subset of replicas(by 
> specifying the wanted replica tag).
> Replicas have to be marked with tags before running the query.
> *Setup needed from the replica side*
> Set the required properties to the required values in at least one replica.
> 
> *Setup needed from the query side*
> A query has to specify {{ShardParams.FILTER_BY_REPLICA_PROPERTY}} to specify 
> that it is indeed interested in replica property filtering.
> Then it should specify {{ShardParams.SHARDS_FILTER}} or 
> {{ShardParams.SHARDS_FILTERNOT}} set to {{ShardParams.REPLICA_PROP}} followed 
> by the property that has to be checked followed by ":" and then the value 
> wanted.
> Excample:
> Given that some replicas have a property named {{region}}:
> Adding the following params to the query:
> {{filterByReplicaProp=true=replicaProp.region:EMEA}}
> will ensure that the query uses replicas that have the property {{region}} 
> set to {{EMEA}}
> {{filterByReplicaProp=true=replicaProp.region:EMEA}}
> will ensure that the query *does not* use replica that have the property 
> {{region}} set to {{EMEA}} 
> 
> An example can be seen in the {{ReplicaTagTest}} included in this patch where 
> a dynamic cloud has some tags assigned to it both randomly and on a fixed 
> basis.
> A replica can have multiple tags attached to it, and these tags are separated 
> by default by "|"(pipe character), the delimiter can be changed by setting 
> {{ShardParams.REPLICA_TAG_DELIMITER}} in the query to anything else.
> The {{ShardParams.FILTER_BY_REPLICA_PROPERTY}} is needed because the 
> computation required to filter by property:value is quite complex and queries 
> that don't care about replica filtering should not incur into the performance 
> penalty.
> The {{ShardParams.REPLICA_PROP}} (currently set to {{replicaProp.}} is needed 
> to ensure that the system is extensible in the future.
> *Usage warnings*
> Using {{ShardParams.SHARDS_FILTER}} or {{ShardParams.SHARDS_FILTERNOT}} set 
> to {{ShardParams.REPLICA_PROP}} without 
> {{ShardParams.FILTER_BY_REPLICA_PROPERTY}} will cause the {{QueryComponent}} 
> to throw exceptions.
> Using {{ShardParams.FILTER_BY_REPLICA_PROPERTY}} without filters will not 
> cause any error, but will likely waste computation time.
> No validity check is performed on the tags, therefore one may get an array of 
> shard URLs that contains empty URLs, or that is null(when the property does 
> not exist), the user of this feature has to deal with it.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-7.x-MacOSX (64bit/jdk1.8.0) - Build # 199 - Unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-7.x-MacOSX/199/
Java: 64bit/jdk1.8.0 -XX:+UseCompressedOops -XX:+UseG1GC

1 tests failed.
FAILED:  org.apache.solr.update.TestInPlaceUpdatesDistrib.test

Error Message:
This doc was supposed to have been deleted, but was: SolrDocument{id=0, 
title_s=title0, id_i=0, inplace_updatable_float=1.0, 
_version_=1579141076483047424, inplace_updatable_int_with_default=666, 
inplace_updatable_float_with_default=42.0}

Stack Trace:
java.lang.AssertionError: This doc was supposed to have been deleted, but was: 
SolrDocument{id=0, title_s=title0, id_i=0, inplace_updatable_float=1.0, 
_version_=1579141076483047424, inplace_updatable_int_with_default=666, 
inplace_updatable_float_with_default=42.0}
at 
__randomizedtesting.SeedInfo.seed([A15276089B5A1BC7:290649D235A6763F]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertNull(Assert.java:551)
at 
org.apache.solr.update.TestInPlaceUpdatesDistrib.reorderedDBQsSimpleTest(TestInPlaceUpdatesDistrib.java:247)
at 
org.apache.solr.update.TestInPlaceUpdatesDistrib.test(TestInPlaceUpdatesDistrib.java:151)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
  

[jira] [Commented] (LUCENE-7966) build mr-jar and use some java 9 methods if available

[Uwe Schindler (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16174510#comment-16174510
 ] 

Uwe Schindler commented on LUCENE-7966:
---

No idea either!

> build mr-jar and use some java 9 methods if available
> -
>
> Key: LUCENE-7966
> URL: https://issues.apache.org/jira/browse/LUCENE-7966
> Project: Lucene - Core
>  Issue Type: Improvement
>  Components: general/build
>Reporter: Robert Muir
> Attachments: LUCENE-7966.patch, LUCENE-7966.patch, LUCENE-7966.patch, 
> LUCENE-7966.patch, LUCENE-7966.patch
>
>
> See background: http://openjdk.java.net/jeps/238
> It would be nice to use some of the newer array methods and range checking 
> methods in java 9 for example, without waiting for lucene 10 or something. If 
> we build an MR-jar, we can start migrating our code to use java 9 methods 
> right now, it will use optimized methods from java 9 when thats available, 
> otherwise fall back to java 8 code.  
> This patch adds:
> {code}
> Objects.checkIndex(int,int)
> Objects.checkFromToIndex(int,int,int)
> Objects.checkFromIndexSize(int,int,int)
> Arrays.mismatch(byte[],int,int,byte[],int,int)
> Arrays.compareUnsigned(byte[],int,int,byte[],int,int)
> Arrays.equal(byte[],int,int,byte[],int,int)
> // did not add char/int/long/short/etc but of course its possible if needed
> {code}
> It sets these up in {{org.apache.lucene.future}} as 1-1 mappings to java 
> methods. This way, we can simply directly replace call sites with java 9 
> methods when java 9 is a minimum. Simple 1-1 mappings mean also that we only 
> have to worry about testing that our java 8 fallback methods work.
> I found that many of the current byte array methods today are willy-nilly and 
> very lenient for example, passing invalid offsets at times and relying on 
> compare methods not throwing exceptions, etc. I fixed all the instances in 
> core/codecs but have not looked at the problems with AnalyzingSuggester. Also 
> SimpleText still uses a silly method in ArrayUtil in similar crazy way, have 
> not removed that one yet.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7966) build mr-jar and use some java 9 methods if available

[Michael McCandless (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16174500#comment-16174500
 ] 

Michael McCandless commented on LUCENE-7966:


I retested w/ Uwe's patch:

{noformat}
java 8 w/o uwe's patch
  done init top-level VE state; took 150.849s; 1062.73 MB RAM; 94244084 total 
unique families
  done init top-level VE state; took 151.488s; 1062.73 MB RAM; 94244084 total 
unique families

java 8 w/ uwe's patch
  done init top-level VE state; took 156.910s; 1062.73 MB RAM; 94244084 total 
unique families
  done init top-level VE state; took 154.836s; 1062.73 MB RAM; 94244084 total 
unique families

java 9 w/o uwe's patch
  done init top-level VE state; took 151.883s; 1062.73 MB RAM; 94244084 total 
unique families
  done init top-level VE state; took 151.123s; 1062.73 MB RAM; 94244084 total 
unique families

java 9 w/ uwe's patch
  done init top-level VE state; took 159.383s; 1062.73 MB RAM; 94244084 total 
unique families
  done init top-level VE state; took 156.662s; 1062.73 MB RAM; 94244084 total 
unique families
{noformat}

Still not sure what's going on ...

> build mr-jar and use some java 9 methods if available
> -
>
> Key: LUCENE-7966
> URL: https://issues.apache.org/jira/browse/LUCENE-7966
> Project: Lucene - Core
>  Issue Type: Improvement
>  Components: general/build
>Reporter: Robert Muir
> Attachments: LUCENE-7966.patch, LUCENE-7966.patch, LUCENE-7966.patch, 
> LUCENE-7966.patch, LUCENE-7966.patch
>
>
> See background: http://openjdk.java.net/jeps/238
> It would be nice to use some of the newer array methods and range checking 
> methods in java 9 for example, without waiting for lucene 10 or something. If 
> we build an MR-jar, we can start migrating our code to use java 9 methods 
> right now, it will use optimized methods from java 9 when thats available, 
> otherwise fall back to java 8 code.  
> This patch adds:
> {code}
> Objects.checkIndex(int,int)
> Objects.checkFromToIndex(int,int,int)
> Objects.checkFromIndexSize(int,int,int)
> Arrays.mismatch(byte[],int,int,byte[],int,int)
> Arrays.compareUnsigned(byte[],int,int,byte[],int,int)
> Arrays.equal(byte[],int,int,byte[],int,int)
> // did not add char/int/long/short/etc but of course its possible if needed
> {code}
> It sets these up in {{org.apache.lucene.future}} as 1-1 mappings to java 
> methods. This way, we can simply directly replace call sites with java 9 
> methods when java 9 is a minimum. Simple 1-1 mappings mean also that we only 
> have to worry about testing that our java 8 fallback methods work.
> I found that many of the current byte array methods today are willy-nilly and 
> very lenient for example, passing invalid offsets at times and relying on 
> compare methods not throwing exceptions, etc. I fixed all the instances in 
> core/codecs but have not looked at the problems with AnalyzingSuggester. Also 
> SimpleText still uses a silly method in ArrayUtil in similar crazy way, have 
> not removed that one yet.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-7970) Improve generation of circle plane

[Ignacio Vera (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-7970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ignacio Vera updated LUCENE-7970:
-
Attachment: LUCENE_7970.patch

Hi [~karl wright],

Not sure if we have something to do here as I think current implementation 
might be the best one to approximate the circle using one plane intersecting 
the spheroid. I don't think I can assure that any other implementation would be 
more precise.

Anyway, I was thinking what is needed to support circles in a spheroid and of 
course it needs a totally different approach and eventually in some cases you 
need to use brute force. I want to share what I think it would be a viable 
solution if Geo3d ever needs to add precision to circles in spheriods like 
WGS84:

The new shape should have two planes:

 innerElipse: plane cutting the spheriod  producing the maximum ellipse 
within the circle. Anything within this plane is within the circle.
 outerElipse: plane cutting the spheriod  producing the maximum ellipse 
contaning the circle. Anything not within this plane is not within the circle.

There is only one gray area between the two planes (within outerEllipse, not 
within innerEllipse) where brute force will be needed. 

I attach my implementation for such a shape. It seems to work for WGS84, except 
for verly large and very small angles. I was unable to find the mathematical 
model to generate those planes (is that possible to solve mathematically?). 

Thanks!

> Improve generation of circle plane 
> ---
>
> Key: LUCENE-7970
> URL: https://issues.apache.org/jira/browse/LUCENE-7970
> Project: Lucene - Core
>  Issue Type: Improvement
>  Components: modules/spatial3d
>Reporter: Ignacio Vera
>Assignee: Karl Wright
> Attachments: LUCENE_7970.patch, LUCENE-7970.patch, 
> LUCENE-7970-proposed.patch
>
>
> Hi [~Karl wright],
> How circles are currently build do not behave very well when the planet model 
> is not an sphere. when you are close to the border in WGS84 you might get 
> false positves or false negatives when checking if a point is WITHIN. I think 
> the reason is how the points to generate the circle plane are generated which 
> assumes a sphere.
> My proposal is the following:
> Add a new method to PlanetModel:  
> public GeoPoint pointOnBearing(GeoPoint from, double dist, double bearing);
> Which uses and algorithm that takes into account that the planet might not be 
> spherical. For example Vincenty's formulae 
> (https://en.wikipedia.org/wiki/Vincenty%27s_formulae).
> Use this method to generate the points for the circle plane. My experiments 
> shows that this approach removes false negatives in WGS84 meanwhile it works 
> nicely in the Sphere.
> Does it make sense?



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-master-MacOSX (64bit/jdk1.8.0) - Build # 4178 - Still Unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-MacOSX/4178/
Java: 64bit/jdk1.8.0 -XX:+UseCompressedOops -XX:+UseParallelGC

5 tests failed.
FAILED:  org.apache.solr.cloud.TestRebalanceLeaders.test

Error Message:
timeout waiting for collection in cluster state: collection=testcollection

Stack Trace:
java.lang.RuntimeException: timeout waiting for collection in cluster state: 
collection=testcollection
at 
__randomizedtesting.SeedInfo.seed([150F4026DD19027E:9D5B7FFC73E56F86]:0)
at 
org.apache.solr.cloud.AbstractFullDistribZkTestBase.waitForCollection(AbstractFullDistribZkTestBase.java:348)
at 
org.apache.solr.cloud.TestRebalanceLeaders.test(TestRebalanceLeaders.java:73)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 

[jira] [Created] (SOLR-11383) Interaction between Solr and Zookeeper?

[Leviya Bl (JIRA)]

Leviya Bl created SOLR-11383:


 Summary: Interaction between Solr and Zookeeper?
 Key: SOLR-11383
 URL: https://issues.apache.org/jira/browse/SOLR-11383
 Project: Solr
  Issue Type: Wish
  Security Level: Public (Default Security Level. Issues are Public)
Reporter: Leviya Bl


I've been working on a system where they use SolrCloud, which entails a 
Zookeeper ensemble that helps "manage the overall structure so that both 
indexing and search requests can be routed properly" (straight out of the 
([Solr 
Documentation|https://lucene.apache.org/solr/guide/6_6/getting-started-with-solrcloud.html]).

What exactly is this "management"? What information, what 
data/configuration/information do the machines running Solr read/write from the 
Zookeeper ensemble and why? Is the data in Zookeeper ever changed at runtime by 
[SOLR|https://mindmajix.com/apache-solr-training]? Or do you configure "the 
data" once and runtime is going to be reads all across SolrCloud hosts?

To put the question into perspective, this is my first contact with Zookeeper, 
Solr, and in many ways with distributed systems.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org