[JENKINS] Lucene-Solr-SmokeRelease-7.x - Build # 174 - Still Failing

[Apache Jenkins Server]

Build: https://builds.apache.org/job/Lucene-Solr-SmokeRelease-7.x/174/

No tests ran.

Build Log:
[...truncated 30126 lines...]
prepare-release-no-sign:
[mkdir] Created dir: 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-7.x/lucene/build/smokeTestRelease/dist
 [copy] Copying 491 files to 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-7.x/lucene/build/smokeTestRelease/dist/lucene
 [copy] Copying 230 files to 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-7.x/lucene/build/smokeTestRelease/dist/solr
   [smoker] Java 1.8 JAVA_HOME=/home/jenkins/tools/java/latest1.8
   [smoker] Java 9 JAVA_HOME=/home/jenkins/tools/java/latest1.9
   [smoker] NOTE: output encoding is UTF-8
   [smoker] 
   [smoker] Load release URL 
"file:/home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-7.x/lucene/build/smokeTestRelease/dist/"...
   [smoker] 
   [smoker] Test Lucene...
   [smoker]   test basics...
   [smoker]   get KEYS
   [smoker] 0.2 MB in 0.01 sec (37.2 MB/sec)
   [smoker]   check changes HTML...
   [smoker]   download lucene-7.4.0-src.tgz...
   [smoker] 31.9 MB in 0.03 sec (1078.8 MB/sec)
   [smoker] verify md5/sha1 digests
   [smoker]   download lucene-7.4.0.tgz...
   [smoker] 73.4 MB in 0.07 sec (978.5 MB/sec)
   [smoker] verify md5/sha1 digests
   [smoker]   download lucene-7.4.0.zip...
   [smoker] 83.9 MB in 0.09 sec (964.2 MB/sec)
   [smoker] verify md5/sha1 digests
   [smoker]   unpack lucene-7.4.0.tgz...
   [smoker] verify JAR metadata/identity/no javax.* or java.* classes...
   [smoker] test demo with 1.8...
   [smoker]   got 6300 hits for query "lucene"
   [smoker] checkindex with 1.8...
   [smoker] test demo with 9...
   [smoker]   got 6300 hits for query "lucene"
   [smoker] checkindex with 9...
   [smoker] check Lucene's javadoc JAR
   [smoker]   unpack lucene-7.4.0.zip...
   [smoker] verify JAR metadata/identity/no javax.* or java.* classes...
   [smoker] test demo with 1.8...
   [smoker]   got 6300 hits for query "lucene"
   [smoker] checkindex with 1.8...
   [smoker] test demo with 9...
   [smoker]   got 6300 hits for query "lucene"
   [smoker] checkindex with 9...
   [smoker] check Lucene's javadoc JAR
   [smoker]   unpack lucene-7.4.0-src.tgz...
   [smoker] make sure no JARs/WARs in src dist...
   [smoker] run "ant validate"
   [smoker] run tests w/ Java 8 and testArgs='-Dtests.badapples=false 
-Dtests.slow=false'...
   [smoker] test demo with 1.8...
   [smoker]   got 217 hits for query "lucene"
   [smoker] checkindex with 1.8...
   [smoker] generate javadocs w/ Java 8...
   [smoker] 
   [smoker] Crawl/parse...
   [smoker] 
   [smoker] Verify...
   [smoker] run tests w/ Java 9 and testArgs='-Dtests.badapples=false 
-Dtests.slow=false'...
   [smoker] 
   [smoker] command "export JAVA_HOME="/home/jenkins/tools/java/latest1.9" 
PATH="/home/jenkins/tools/java/latest1.9/bin:$PATH" 
JAVACMD="/home/jenkins/tools/java/latest1.9/bin/java"; ant clean test 
-Dtests.badapples=false -Dtests.slow=false" failed:
   [smoker] Buildfile: 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-7.x/lucene/build/smokeTestRelease/tmp/unpack/lucene-7.4.0/build.xml
   [smoker] 
   [smoker] clean:
   [smoker][delete] Deleting directory 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-7.x/lucene/build/smokeTestRelease/tmp/unpack/lucene-7.4.0/build
   [smoker] 
   [smoker] ivy-availability-check:
   [smoker] [loadresource] Do not set property disallowed.ivy.jars.list as its 
length is 0.
   [smoker] 
   [smoker] -ivy-fail-disallowed-ivy-version:
   [smoker] 
   [smoker] ivy-fail:
   [smoker] 
   [smoker] ivy-configure:
   [smoker] [ivy:configure] :: Apache Ivy 2.4.0 - 20141213170938 :: 
http://ant.apache.org/ivy/ ::
   [smoker] [ivy:configure] :: loading settings :: file = 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-7.x/lucene/build/smokeTestRelease/tmp/unpack/lucene-7.4.0/top-level-ivy-settings.xml
   [smoker] 
   [smoker] -clover.load:
   [smoker] 
   [smoker] resolve-groovy:
   [smoker] [ivy:cachepath] :: resolving dependencies :: 
org.codehaus.groovy#groovy-all-caller;working
   [smoker] [ivy:cachepath] confs: [default]
   [smoker] [ivy:cachepath] found org.codehaus.groovy#groovy-all;2.4.13 in 
public
   [smoker] [ivy:cachepath] :: resolution report :: resolve 769ms :: artifacts 
dl 14ms
   [smoker] 
-
   [smoker] |  |modules||   
artifacts   |
   [smoker] |   conf   | number| search|dwnlded|evicted|| 
number|dwnlded|
   [smoker] 
-
   [smoker] |  default |   1   |   0   |   0   |   0   ||   1   |   
0   |
   [smoker] 

Re: new candidate list for BadApple-ing on Saturday, 17-Mar

[Erick Erickson]

Gus:

Nice sleuthing!

The problem has been that there has been s much noise that
it's been almost impossible, for me at least, to get motivated to dig
into them. I certainly suspect systemic issues like this are behind
some of our intermittent failures, so I'm very glad you took the
initiative to dig.

If you see some test on the list that you want to keep running, please
just let me know. It's perfectly OK to have a test fail intermittently
if someone's actively looking at why, means it isn't lost in the
noise. And often I _can't_ get tests to fail locally no matter how
many times I run them so having them run on Jenkins is the only way to
pursue them.

My long-term goal here is to get the noise level down to tolerable
levels. Once that happens, an ongoing task will be to un-BaApple tests
that no longer fail as demonstrated by Hoss' roll-up and Mark's
beasting projects and see if we can get that test coverage back. This
whole BadApple thing is just a stop-gap to get to a stable point and
then start getting the coverage back

Best,
Erick



On Wed, Mar 14, 2018 at 8:36 PM, Mark Miller  wrote:
> Can you file a JIRA issue?
>
> In general, we want to be super forgiving by default and only harsh for a
> specific test that might demand it.
>
> - Mark
>
> On Wed, Mar 14, 2018 at 9:45 PM Gus Heck  wrote:
>>
>> Beginning to answer my own question...
>>
>> public abstract class AbstractZkTestCase extends SolrTestCaseJ4 {
>>   private static final String ZOOKEEPER_FORCE_SYNC =
>> "zookeeper.forceSync";
>>
>>   public static final int TIMEOUT = 45000;
>>
>> seems to be at least one place we are probably not getting what we
>> expect... the server's going to cut that back to the 20 second max...
>>
>> On Wed, Mar 14, 2018 at 10:36 PM, Gus Heck  wrote:
>>>
>>> Being slightly irritated by the fact one of my tests shows up in this, I
>>> did some digging and I found
>>>
>>>[junit4]   2> 485679 ERROR
>>> (OverseerThreadFactory-788-thread-1-processing-n:127.0.0.1:35771_solr)
>>> [n:127.0.0.1:35771_solr] o.a.s.c.a.c.OverseerCollectionMessageHandler
>>> Collection: testAliasCplx0 operation: createalias
>>> failed:org.apache.zookeeper.KeeperException$SessionExpiredException:
>>> KeeperErrorCode = Session expired for /configs/_default
>>>[junit4]   2>at
>>> org.apache.zookeeper.KeeperException.create(KeeperException.java:130)
>>>[junit4]   2>at
>>> org.apache.zookeeper.KeeperException.create(KeeperException.java:54)
>>>[junit4]   2>at
>>> org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1105)
>>>[junit4]   2>at
>>> org.apache.solr.common.cloud.SolrZkClient.lambda$exists$3(SolrZkClient.java:316)
>>>[junit4]   2>at
>>> org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
>>>[junit4]   2>at
>>> org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:316)
>>>[junit4]   2>at
>>> org.apache.solr.client.solrj.impl.ZkDistribStateManager.hasData(ZkDistribStateManager.java:58)
>>>[junit4]   2>at
>>> org.apache.solr.cloud.api.collections.OverseerCollectionMessageHandler.validateConfigOrThrowSolrException(OverseerCollectionMessageHandler.java:737)
>>>[junit4]   2>at
>>> org.apache.solr.cloud.api.collections.CreateCollectionCmd.call(CreateCollectionCmd.java:114)
>>>[junit4]   2>at
>>> org.apache.solr.cloud.api.collections.MaintainRoutedAliasCmd.createCollectionAndWait(MaintainRoutedAliasCmd.java:282)
>>>[junit4]   2>at
>>> org.apache.solr.cloud.api.collections.CreateAliasCmd.callCreateRoutedAlias(CreateAliasCmd.java:124)
>>>[junit4]   2>at
>>> org.apache.solr.cloud.api.collections.CreateAliasCmd.call(CreateAliasCmd.java:65)
>>>[junit4]   2>at
>>> org.apache.solr.cloud.api.collections.OverseerCollectionMessageHandler.processMessage(OverseerCollectionMessageHandler.java:252)
>>>[junit4]   2>at
>>> org.apache.solr.cloud.OverseerTaskProcessor$Runner.run(OverseerTaskProcessor.java:469)
>>>[junit4]   2>at
>>> org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
>>>[junit4]   2>at
>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>>>[junit4]   2>at
>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>>>[junit4]   2>at java.lang.Thread.run(Thread.java:748)
>>>[junit4]   2>
>>>
>>> in the full log for
>>> https://jenkins.thetaphi.de/view/Lucene-Solr/job/Lucene-Solr-master-Solaris/1719/console
>>>
>>> After some digging I found this code in ZkTestServer:
>>>
>>> public void run() throws InterruptedException {
>>>   log.info("STARTING ZK TEST SERVER");
>>>   // we don't call super.distribSetUp
>>>   zooThread = new Thread() {
>>>
>>> @Override
>>> public void run() {
>>>   ServerConfig config = new ServerConfig() {
>>>
>>> {
>>>   

[JENKINS] Lucene-Solr-NightlyTests-7.x - Build # 175 - Failure

[Apache Jenkins Server]

Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-7.x/175/

4 tests failed.
FAILED:  org.apache.solr.cloud.RollingRestartTest.test

Error Message:
Unable to restart (#6): CloudJettyRunner 
[url=http://127.0.0.1:46425/bcb/collection1_shard1_replica_n67]

Stack Trace:
java.lang.AssertionError: Unable to restart (#6): CloudJettyRunner 
[url=http://127.0.0.1:46425/bcb/collection1_shard1_replica_n67]
at 
__randomizedtesting.SeedInfo.seed([E7C8D9B9A63EB0:88B3F703175A5348]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at 
org.apache.solr.cloud.RollingRestartTest.restartWithRolesTest(RollingRestartTest.java:103)
at 
org.apache.solr.cloud.RollingRestartTest.test(RollingRestartTest.java:52)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)

Re: TestReplicationHandler is failing 100% (master and 7.x / 7.3)

[Erick Erickson]

OK. Note that only certain Jenkins jobs run with BadApple=true set.

Mark's beastit stuff (he's got that back up and running) runs with
badapple=true though, so maybe if it survives a few days running there
we can un-annotate it.

How about this. I'll check Marks stuff and, if by Saturday there are
no failures for that test, un-BadApple it on Saturday?

Erick

On Wed, Mar 14, 2018 at 9:02 PM, Shalin Shekhar Mangar
 wrote:
> Hi Erick,
>
> The test was disabled because it used to fail sometimes on slow machines. I
> haven't beasted it to see if that still happens but it probably does. I only
> fixed it so that it doesn't always fail. So let's see how Jenkins is doing
> and then decide.
>
> On Thu, Mar 15, 2018 at 3:31 AM, Erick Erickson 
> wrote:
>>
>> Shalin:
>>
>> Should we remove (actually, comment out with a date?) the BadApple
>> annotation for doTestIndexFetchOnMasterRestart? And do you think your
>> fixes have any influence on the other BadApple
>> (doTestIndexAndConfigReplication)?
>>
>> There's no problem with un-BadApple-ing test that have been or are
>> being worked on, and we'd get more test coverage that way.
>>
>> Or I can do that on Saturday if you'd prefer, assuming the Jenkins
>> BadApple tests don't show failures.
>>
>>
>>
>> On Wed, Mar 14, 2018 at 1:57 PM, Shalin Shekhar Mangar
>>  wrote:
>> > This is fixed. I committed the fix to master, branch_7x and branch_7_3
>> > branches.
>> >
>> > On Wed, Mar 14, 2018 at 9:44 PM, Alan Woodward 
>> > wrote:
>> >>
>> >> Thanks Shalin!
>> >>
>> >>
>> >> On 14 Mar 2018, at 15:50, Shalin Shekhar Mangar
>> >> 
>> >> wrote:
>> >>
>> >> I'll take a look at it tomorrow morning my time.
>> >>
>> >> On Wed, Mar 14, 2018 at 9:07 PM, Andrzej Białecki
>> >>  wrote:
>> >>>
>> >>> Well … I looked at it briefly but I have no idea what’s going on
>> >>> there. I
>> >>> could dig into it nonetheless, but if there’s someone who already
>> >>> knows the
>> >>> replication handler ins and outs it would probably get fixed sooner...
>> >>>
>> >>>
>> >>> On 14 Mar 2018, at 14:23, Alan Woodward  wrote:
>> >>>
>> >>> I’m happy either way, but if it’s a bug can we get it fixed quickly?
>> >>> Can
>> >>> you take ownership of this one Andrzej?
>> >>>
>> >>> On 14 Mar 2018, at 11:24, Andrzej Białecki  wrote:
>> >>>
>> >>> Hi,
>> >>>
>> >>> This test has always been fragile, but recently it’s been failing
>> >>> 100%,
>> >>> most often in ‘doTestIndexFetchOnMasterRestart’.
>> >>>
>> >>> I don’t know the replication handler enough to be able to find the
>> >>> real
>> >>> reason behind these failures, but there are two possibilities that I
>> >>> see:
>> >>>
>> >>> * the test has a bug and needs to be fixed - and if we can’t fix it
>> >>> soon
>> >>> then with 7.3 release imminent we could BadApple it until it’s
>> >>> properly
>> >>> fixed
>> >>>
>> >>> * or actually the replication handler has a bug, which needs to be
>> >>> fixed
>> >>> - in which case I propose to bump up SOLR-12078 to Blocker.
>> >>>
>> >>> I’m open to suggestions.
>> >>>
>> >>> —
>> >>>
>> >>> Andrzej Białecki
>> >>>
>> >>>
>> >>>
>> >>
>> >>
>> >>
>> >> --
>> >> Regards,
>> >> Shalin Shekhar Mangar.
>> >>
>> >>
>> >
>> >
>> >
>> > --
>> > Regards,
>> > Shalin Shekhar Mangar.
>>
>> -
>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
>> For additional commands, e-mail: dev-h...@lucene.apache.org
>>
>
>
>
> --
> Regards,
> Shalin Shekhar Mangar.

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-master-Linux (64bit/jdk-9.0.4) - Build # 21642 - Still Unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/21642/
Java: 64bit/jdk-9.0.4 -XX:+UseCompressedOops -XX:+UseSerialGC

4 tests failed.
FAILED:  org.apache.lucene.index.TestIndexWriter.testSoftUpdatesConcurrently

Error Message:
expected:<2> but was:<48>

Stack Trace:
java.lang.AssertionError: expected:<2> but was:<48>
at 
__randomizedtesting.SeedInfo.seed([D0D8AEC78CFC154D:805E0EC7306051EB]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.junit.Assert.assertEquals(Assert.java:456)
at 
org.apache.lucene.index.TestIndexWriter.testSoftUpdatesConcurrently(TestIndexWriter.java:3214)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:564)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.base/java.lang.Thread.run(Thread.java:844)


FAILED:  org.apache.lucene.index.TestIndexWriter.testSoftUpdatesConcurrently

Error Message:
expected:<2> but was:<48>

Stack Trace:
java.lang.AssertionError: expected:<2> but was:<48>
at 
__randomizedtesting.SeedInfo.seed([D0D8AEC78CFC154D:805E0EC7306051EB]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)

[jira] [Updated] (SOLR-10912) Adding automatic patch validation

[Steve Rowe (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-10912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Steve Rowe updated SOLR-10912:
--
Attachment: SOLR-10912.patch

> Adding automatic patch validation
> -
>
> Key: SOLR-10912
> URL: https://issues.apache.org/jira/browse/SOLR-10912
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Build
>Reporter: Mano Kovacs
>Assignee: Steve Rowe
>Priority: Major
> Attachments: SOLR-10912.ok-patch-in-core.patch, SOLR-10912.patch, 
> SOLR-10912.patch, SOLR-10912.sample-patch.patch, 
> SOLR-10912.solj-contrib-facet-error.patch
>
>
> Proposing introduction of automated patch validation, similar what Hadoop or 
> other Apache projects are using (see link). This would ensure that every 
> patch passes a certain set of criterions before getting approved. It would 
> save time for developer (faster feedback loop), save time for committers 
> (less step to do manually), and would increase quality.
> Hadoop is currently using Apache Yetus to run validations, which seems to be 
> a good direction to start. This jira could be the board of discussing the 
> preferred solution.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: TestReplicationHandler is failing 100% (master and 7.x / 7.3)

[Shalin Shekhar Mangar]

Hi Erick,

The test was disabled because it used to fail sometimes on slow machines. I
haven't beasted it to see if that still happens but it probably does. I
only fixed it so that it doesn't always fail. So let's see how Jenkins is
doing and then decide.

On Thu, Mar 15, 2018 at 3:31 AM, Erick Erickson 
wrote:

> Shalin:
>
> Should we remove (actually, comment out with a date?) the BadApple
> annotation for doTestIndexFetchOnMasterRestart? And do you think your
> fixes have any influence on the other BadApple
> (doTestIndexAndConfigReplication)?
>
> There's no problem with un-BadApple-ing test that have been or are
> being worked on, and we'd get more test coverage that way.
>
> Or I can do that on Saturday if you'd prefer, assuming the Jenkins
> BadApple tests don't show failures.
>
>
>
> On Wed, Mar 14, 2018 at 1:57 PM, Shalin Shekhar Mangar
>  wrote:
> > This is fixed. I committed the fix to master, branch_7x and branch_7_3
> > branches.
> >
> > On Wed, Mar 14, 2018 at 9:44 PM, Alan Woodward 
> wrote:
> >>
> >> Thanks Shalin!
> >>
> >>
> >> On 14 Mar 2018, at 15:50, Shalin Shekhar Mangar  >
> >> wrote:
> >>
> >> I'll take a look at it tomorrow morning my time.
> >>
> >> On Wed, Mar 14, 2018 at 9:07 PM, Andrzej Białecki
> >>  wrote:
> >>>
> >>> Well … I looked at it briefly but I have no idea what’s going on
> there. I
> >>> could dig into it nonetheless, but if there’s someone who already
> knows the
> >>> replication handler ins and outs it would probably get fixed sooner...
> >>>
> >>>
> >>> On 14 Mar 2018, at 14:23, Alan Woodward  wrote:
> >>>
> >>> I’m happy either way, but if it’s a bug can we get it fixed quickly?
> Can
> >>> you take ownership of this one Andrzej?
> >>>
> >>> On 14 Mar 2018, at 11:24, Andrzej Białecki  wrote:
> >>>
> >>> Hi,
> >>>
> >>> This test has always been fragile, but recently it’s been failing 100%,
> >>> most often in ‘doTestIndexFetchOnMasterRestart’.
> >>>
> >>> I don’t know the replication handler enough to be able to find the real
> >>> reason behind these failures, but there are two possibilities that I
> see:
> >>>
> >>> * the test has a bug and needs to be fixed - and if we can’t fix it
> soon
> >>> then with 7.3 release imminent we could BadApple it until it’s properly
> >>> fixed
> >>>
> >>> * or actually the replication handler has a bug, which needs to be
> fixed
> >>> - in which case I propose to bump up SOLR-12078 to Blocker.
> >>>
> >>> I’m open to suggestions.
> >>>
> >>> —
> >>>
> >>> Andrzej Białecki
> >>>
> >>>
> >>>
> >>
> >>
> >>
> >> --
> >> Regards,
> >> Shalin Shekhar Mangar.
> >>
> >>
> >
> >
> >
> > --
> > Regards,
> > Shalin Shekhar Mangar.
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org
>
>


-- 
Regards,
Shalin Shekhar Mangar.

[jira] [Comment Edited] (SOLR-10912) Adding automatic patch validation

[Steve Rowe (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16399885#comment-16399885
 ] 

Steve Rowe edited comment on SOLR-10912 at 3/15/18 4:01 AM:


{quote}The current change is here: 
[https://github.com/apache/lucene-solr/compare/master...manokovacs:feature/SOLR-10912]
{quote}
I've attached a patch with a modified version of Mano's branch, and I plan on 
committing it shortly in order to start testing the Jenkins job I've set up at 
[https://builds.apache.org/view/PreCommit+Builds/job/PreCommit-LUCENE-Build/] 
(the script in this job's config, a copy of which is in the patch, expects the 
Yetus personality to be committed to the Git repo).
h2. From Mano's "Steps remaining":
{quote}Multiple module test currently duplicates the failed test result, fixing 
shortly.
{quote}
Fixed.
{quote}Add test to call -validate-source-patterns
{quote}
Added. FYI, I had to rename the Ant task to remove the leading dash so that it 
can be invoked from the cmdline.
{quote}Finalize the runner script to setup Yetus, similarly to Hadoop.
{quote}
Done.
{quote}Verify the test-patch with Github PR
{quote}
Since [~aw] wrote above that "The job on Jenkins that feeds test-patch is NOT 
github aware", I don't plan on doing this verification. I'll include this on a 
TODO list below.
{quote}Add Patch Available status for SOLR project (and update the script to 
look for that).
{quote}
Done.
{quote}Add Precommit-SOLR and Precommit-LUCENE jenkins jobs
{quote}
I've added a PreCommit-LUCENE-Build job (linked above), and once I've committed 
the patch I've attached here (on master only initially), I'll manually invoke 
it for testing (via "Build with Parameters").
h2. From Mano's "Open questions":
{quote}1. As you can see, a patch with two log entries had 6 (flaky) test 
failures. Assuming flaky tests will not go away very soon, would it still be 
useful to have this test-patch?
{quote}
I think now is a perfect time to do this, given the current efforts focused on 
reducing test flakiness.
{quote}2. I propose starting with the smallest set of tests (ie. affected 
modules) and extend them later to dependent modules.
{quote}
+1.
h2. Stuff I changed from Mano's branch that is not already mentioned above:
 # Renamed the personality file (was {{solr-yetus-personality.sh}}, now 
{{lucene-solr-yetus-personality.sh}}).
 # Improved handling of Lucene modules.
 # Added basic ref guide validation, via {{ant bare-bones-html-validation}}; 
note that this is not the kind of missing-doc validation discussed above in 
this issue; that idea was spun off as SOLR-11419
 # Standardized Lucene/Solr-specific plugins to make them run only if they need 
to.
 # Added some user- and dev-level documentation to the local runner script and 
personality files.

h2. Short-term TODO:
 # Commit current patch to master
 # Manually test the Jenkins precommit job
 # Iterate above two steps until testing is successful
 # Backport the patch to branch_7x
 # Copy the Precommit-LUCENE-Build job to a new PreCommit-SOLR-Build job.
 # Request ASF Infrastructure to add LUCENE and SOLR to the list of projects 
that use the PreCommit-Admin Jenkins job to enqueue precommit runs for new 
patches on LUCENE/SOLR JIRAs with the "Patch Available" state. (I'll make a 
JIRA for this and link it to this issue.)
 # Attach a patch with the finalized Lucene/Solr personality on HADOOP-14538, 
for inclusion in future Yetus releases

*edit*: Added Shawn's improvement ideas to the TODO list below:

h2. Longer-term TODO (to be dealt with on further issues):
 # Solr missing-doc validation: SOLR-11419.
 # Add handling of module dependencies and build ordering.
 # Currently when any test or non-test file is changed in a module, all unit 
tests are run in that module. I think a faster version of this is possible when 
there are test-only changes: just the changed test suites could be run, rather 
than all of the module's tests.
 # If there's not an entry in CHANGES.txt that mentions the issue number 
(either the lucene or solr file as appropriate), that should be a -1 (or maybe 
-0?)
# How about a -1 if a SOLR patch makes changes to lucene, or vice versa? If 
there is an entry in the appropriate CHANGES.txt file for the issue, turn that 
into a -0. That way, we have better assurance that if a commit for one part of 
the project requires changes to the other part, there will be a release note.


was (Author: steve_rowe):
{quote}The current change is here: 
[https://github.com/apache/lucene-solr/compare/master...manokovacs:feature/SOLR-10912]
{quote}
I've attached a patch with a modified version of Mano's branch, and I plan on 
committing it shortly in order to start testing the Jenkins job I've set up at 
[https://builds.apache.org/view/PreCommit+Builds/job/PreCommit-LUCENE-Build/] 
(the script in this job's config, a copy of which is in the patch, expects the 
Yetus personality 

[jira] [Commented] (SOLR-10912) Adding automatic patch validation

[Steve Rowe (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16399885#comment-16399885
 ] 

Steve Rowe commented on SOLR-10912:
---

{quote}The current change is here: 
[https://github.com/apache/lucene-solr/compare/master...manokovacs:feature/SOLR-10912]
{quote}
I've attached a patch with a modified version of Mano's branch, and I plan on 
committing it shortly in order to start testing the Jenkins job I've set up at 
[https://builds.apache.org/view/PreCommit+Builds/job/PreCommit-LUCENE-Build/] 
(the script in this job's config, a copy of which is in the patch, expects the 
Yetus personality to be committed to the Git repo).
h2. From Mano's "Steps remaining":
{quote}Multiple module test currently duplicates the failed test result, fixing 
shortly.
{quote}
Fixed.
{quote}Add test to call -validate-source-patterns
{quote}
Added. FYI, I had to rename the Ant task to remove the leading dash so that it 
can be invoked from the cmdline.
{quote}Finalize the runner script to setup Yetus, similarly to Hadoop.
{quote}
Done.
{quote}Verify the test-patch with Github PR
{quote}
Since [~aw] wrote above that "The job on Jenkins that feeds test-patch is NOT 
github aware", I don't plan on doing this verification. I'll include this on a 
TODO list below.
{quote}Add Patch Available status for SOLR project (and update the script to 
look for that).
{quote}
Done.
{quote}Add Precommit-SOLR and Precommit-LUCENE jenkins jobs
{quote}
I've added a PreCommit-LUCENE-Build job (linked above), and once I've committed 
the patch I've attached here (on master only initially), I'll manually invoke 
it for testing (via "Build with Parameters").
h2. From Mano's "Open questions":
{quote}1. As you can see, a patch with two log entries had 6 (flaky) test 
failures. Assuming flaky tests will not go away very soon, would it still be 
useful to have this test-patch?
{quote}
I think now is a perfect time to do this, given the current efforts focused on 
reducing test flakiness.
{quote}2. I propose starting with the smallest set of tests (ie. affected 
modules) and extend them later to dependent modules.
{quote}
+1.
h2. Stuff I changed from Mano's branch that is not already mentioned above:
 # Renamed the personality file (was {{solr-yetus-personality.sh}}, now 
{{lucene-solr-yetus-personality.sh}}).
 # Improved handling of Lucene modules.
 # Added basic ref guide validation, via {{ant bare-bones-html-validation}}; 
note that this is not the kind of missing-doc validation discussed above in 
this issue; that idea was spun off as SOLR-11419
 # Standardized Lucene/Solr-specific plugins to make them run only if they need 
to.
 # Added some user- and dev-level documentation to the local runner script and 
personality files.

h2. Short-term TODO:
 # Commit current patch to master
 # Manually test the Jenkins precommit job
 # Iterate above two steps until testing is successful
 # Backport the patch to branch_7x
 # Copy the Precommit-LUCENE-Build job to a new PreCommit-SOLR-Build job.
 # Request ASF Infrastructure to add LUCENE and SOLR to the list of projects 
that use the PreCommit-Admin Jenkins job to enqueue precommit runs for new 
patches on LUCENE/SOLR JIRAs with the "Patch Available" state. (I'll make a 
JIRA for this and link it to this issue.)
 # Attach a patch with the finalized Lucene/Solr personality on HADOOP-14538, 
for inclusion in future Yetus releases

h2. Longer-term TODO (to be dealt with on further issues):
 # Solr missing-doc validation: SOLR-11419.
 # Add handling of module dependencies and build ordering.
 # Currently when any test or non-test file is changed in a module, all unit 
tests are run in that module. I think a faster version of this is possible when 
there are test-only changes: just the changed test suites could be run, rather 
than all of the module's tests.

> Adding automatic patch validation
> -
>
> Key: SOLR-10912
> URL: https://issues.apache.org/jira/browse/SOLR-10912
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Build
>Reporter: Mano Kovacs
>Assignee: Steve Rowe
>Priority: Major
> Attachments: SOLR-10912.ok-patch-in-core.patch, SOLR-10912.patch, 
> SOLR-10912.sample-patch.patch, SOLR-10912.solj-contrib-facet-error.patch
>
>
> Proposing introduction of automated patch validation, similar what Hadoop or 
> other Apache projects are using (see link). This would ensure that every 
> patch passes a certain set of criterions before getting approved. It would 
> save time for developer (faster feedback loop), save time for committers 
> (less step to do manually), and would increase quality.
> Hadoop is currently using Apache Yetus to run validations, which seems to be 
> a good direction to start. This 

Re: new candidate list for BadApple-ing on Saturday, 17-Mar

[Mark Miller]

Can you file a JIRA issue?

In general, we want to be super forgiving by default and only harsh for a
specific test that might demand it.

- Mark

On Wed, Mar 14, 2018 at 9:45 PM Gus Heck  wrote:

> Beginning to answer my own question...
>
> public abstract class AbstractZkTestCase extends SolrTestCaseJ4 {
>   private static final String ZOOKEEPER_FORCE_SYNC = "zookeeper.forceSync";
>
>   public static final int TIMEOUT = 45000;
>
> seems to be at least one place we are probably not getting what we
> expect... the server's going to cut that back to the 20 second max...
>
> On Wed, Mar 14, 2018 at 10:36 PM, Gus Heck  wrote:
>
>> Being slightly irritated by the fact one of my tests shows up in this, I
>> did some digging and I found
>>
>>[junit4]   2> 485679 ERROR 
>> (OverseerThreadFactory-788-thread-1-processing-n:127.0.0.1:35771_solr) 
>> [n:127.0.0.1:35771_solr] o.a.s.c.a.c.OverseerCollectionMessageHandler 
>> Collection: testAliasCplx0 operation: createalias 
>> failed:org.apache.zookeeper.KeeperException$SessionExpiredException: 
>> KeeperErrorCode = Session expired for /configs/_default
>>[junit4]   2> at 
>> org.apache.zookeeper.KeeperException.create(KeeperException.java:130)
>>[junit4]   2> at 
>> org.apache.zookeeper.KeeperException.create(KeeperException.java:54)
>>[junit4]   2> at 
>> org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1105)
>>[junit4]   2> at 
>> org.apache.solr.common.cloud.SolrZkClient.lambda$exists$3(SolrZkClient.java:316)
>>[junit4]   2> at 
>> org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
>>[junit4]   2> at 
>> org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:316)
>>[junit4]   2> at 
>> org.apache.solr.client.solrj.impl.ZkDistribStateManager.hasData(ZkDistribStateManager.java:58)
>>[junit4]   2> at 
>> org.apache.solr.cloud.api.collections.OverseerCollectionMessageHandler.validateConfigOrThrowSolrException(OverseerCollectionMessageHandler.java:737)
>>[junit4]   2> at 
>> org.apache.solr.cloud.api.collections.CreateCollectionCmd.call(CreateCollectionCmd.java:114)
>>[junit4]   2> at 
>> org.apache.solr.cloud.api.collections.MaintainRoutedAliasCmd.createCollectionAndWait(MaintainRoutedAliasCmd.java:282)
>>[junit4]   2> at 
>> org.apache.solr.cloud.api.collections.CreateAliasCmd.callCreateRoutedAlias(CreateAliasCmd.java:124)
>>[junit4]   2> at 
>> org.apache.solr.cloud.api.collections.CreateAliasCmd.call(CreateAliasCmd.java:65)
>>[junit4]   2> at 
>> org.apache.solr.cloud.api.collections.OverseerCollectionMessageHandler.processMessage(OverseerCollectionMessageHandler.java:252)
>>[junit4]   2> at 
>> org.apache.solr.cloud.OverseerTaskProcessor$Runner.run(OverseerTaskProcessor.java:469)
>>[junit4]   2> at 
>> org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
>>[junit4]   2> at 
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>>[junit4]   2> at 
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>>[junit4]   2> at java.lang.Thread.run(Thread.java:748)
>>[junit4]   2>
>>
>> in the full log for
>> https://jenkins.thetaphi.de/view/Lucene-Solr/job/Lucene-Solr-master-Solaris/1719/console
>>
>> After some digging I found this code in ZkTestServer:
>>
>> public void run() throws InterruptedException {
>>   log.info("STARTING ZK TEST SERVER");
>>   // we don't call super.distribSetUp
>>   zooThread = new Thread() {
>>
>> @Override
>> public void run() {
>>   ServerConfig config = new ServerConfig() {
>>
>> {
>>   setClientPort(ZkTestServer.this.clientPort);
>>   this.dataDir = zkDir;
>>   this.dataLogDir = zkDir;
>>   this.tickTime = theTickTime;
>> }
>>
>> public void setClientPort(int clientPort) {
>>   if (clientPortAddress != null) {
>> try {
>>   this.clientPortAddress = new InetSocketAddress(
>>   
>> InetAddress.getByName(clientPortAddress.getHostName()), clientPort);
>> } catch (UnknownHostException e) {
>>   throw new RuntimeException(e);
>> }
>>   } else {
>> this.clientPortAddress = new InetSocketAddress(clientPort);
>>   }
>>   log.info("client port:" + this.clientPortAddress);
>> }
>>   };
>>
>>   try {
>> zkServer.runFromConfig(config);
>>   } catch (Throwable e) {
>> throw new RuntimeException(e);
>>   }
>> }
>>   };
>>
>> And what I noticed is that min/max timeouts are unset and theTickTime is
>> onlly ever set to a big blue 1000 leading to default min/max timeout
>> values of 2/20 seconds (
>> 

[jira] [Assigned] (SOLR-10912) Adding automatic patch validation

[Steve Rowe (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-10912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Steve Rowe reassigned SOLR-10912:
-

Assignee: Steve Rowe

> Adding automatic patch validation
> -
>
> Key: SOLR-10912
> URL: https://issues.apache.org/jira/browse/SOLR-10912
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Build
>Reporter: Mano Kovacs
>Assignee: Steve Rowe
>Priority: Major
> Attachments: SOLR-10912.ok-patch-in-core.patch, SOLR-10912.patch, 
> SOLR-10912.sample-patch.patch, SOLR-10912.solj-contrib-facet-error.patch
>
>
> Proposing introduction of automated patch validation, similar what Hadoop or 
> other Apache projects are using (see link). This would ensure that every 
> patch passes a certain set of criterions before getting approved. It would 
> save time for developer (faster feedback loop), save time for committers 
> (less step to do manually), and would increase quality.
> Hadoop is currently using Apache Yetus to run validations, which seems to be 
> a good direction to start. This jira could be the board of discussing the 
> preferred solution.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-10912) Adding automatic patch validation

[Steve Rowe (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-10912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Steve Rowe updated SOLR-10912:
--
Attachment: SOLR-10912.patch

> Adding automatic patch validation
> -
>
> Key: SOLR-10912
> URL: https://issues.apache.org/jira/browse/SOLR-10912
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Build
>Reporter: Mano Kovacs
>Priority: Major
> Attachments: SOLR-10912.ok-patch-in-core.patch, SOLR-10912.patch, 
> SOLR-10912.sample-patch.patch, SOLR-10912.solj-contrib-facet-error.patch
>
>
> Proposing introduction of automated patch validation, similar what Hadoop or 
> other Apache projects are using (see link). This would ensure that every 
> patch passes a certain set of criterions before getting approved. It would 
> save time for developer (faster feedback loop), save time for committers 
> (less step to do manually), and would increase quality.
> Hadoop is currently using Apache Yetus to run validations, which seems to be 
> a good direction to start. This jira could be the board of discussing the 
> preferred solution.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-master-MacOSX (64bit/jdk1.8.0) - Build # 4498 - Still unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-MacOSX/4498/
Java: 64bit/jdk1.8.0 -XX:-UseCompressedOops -XX:+UseParallelGC

2 tests failed.
FAILED:  
junit.framework.TestSuite.org.apache.solr.prometheus.exporter.SolrExporterTest

Error Message:
Error starting up MiniSolrCloudCluster

Stack Trace:
java.lang.Exception: Error starting up MiniSolrCloudCluster
at __randomizedtesting.SeedInfo.seed([84B88B34B3A2D213]:0)
at 
org.apache.solr.cloud.MiniSolrCloudCluster.checkForExceptions(MiniSolrCloudCluster.java:513)
at 
org.apache.solr.cloud.MiniSolrCloudCluster.(MiniSolrCloudCluster.java:251)
at 
org.apache.solr.cloud.SolrCloudTestCase$Builder.configure(SolrCloudTestCase.java:190)
at 
org.apache.solr.prometheus.exporter.SolrExporterTestBase.setupCluster(SolrExporterTestBase.java:43)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:874)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Suppressed: java.lang.AssertionError
at 
sun.reflect.generics.reflectiveObjects.WildcardTypeImpl.getUpperBoundASTs(WildcardTypeImpl.java:86)
at 
sun.reflect.generics.reflectiveObjects.WildcardTypeImpl.getUpperBounds(WildcardTypeImpl.java:122)
at 
sun.reflect.generics.reflectiveObjects.WildcardTypeImpl.toString(WildcardTypeImpl.java:190)
at java.lang.reflect.Type.getTypeName(Type.java:46)
at 
sun.reflect.generics.reflectiveObjects.ParameterizedTypeImpl.toString(ParameterizedTypeImpl.java:234)
at java.lang.reflect.Type.getTypeName(Type.java:46)
at 
java.lang.reflect.Method.specificToGenericStringHeader(Method.java:421)
at 
java.lang.reflect.Executable.sharedToGenericString(Executable.java:163)
at java.lang.reflect.Method.toGenericString(Method.java:415)
at java.beans.MethodRef.set(MethodRef.java:46)
at 
java.beans.MethodDescriptor.setMethod(MethodDescriptor.java:117)
at java.beans.MethodDescriptor.(MethodDescriptor.java:72)
at java.beans.MethodDescriptor.(MethodDescriptor.java:56)
at 
java.beans.Introspector.getTargetMethodInfo(Introspector.java:1205)
at java.beans.Introspector.getBeanInfo(Introspector.java:426)
at java.beans.Introspector.getBeanInfo(Introspector.java:173)
at java.beans.Introspector.getBeanInfo(Introspector.java:260)
at java.beans.Introspector.(Introspector.java:407)
at java.beans.Introspector.getBeanInfo(Introspector.java:173)
at 

[JENKINS] Lucene-Solr-Tests-master - Build # 2425 - Still Unstable

[Apache Jenkins Server]

Build: https://builds.apache.org/job/Lucene-Solr-Tests-master/2425/

9 tests failed.
FAILED:  org.apache.solr.cloud.ReplaceNodeTest.test

Error Message:


Stack Trace:
java.lang.AssertionError
at 
__randomizedtesting.SeedInfo.seed([BA4CB527817F8441:32188AFD2F83E9B9]:0)
at org.junit.Assert.fail(Assert.java:92)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertFalse(Assert.java:68)
at org.junit.Assert.assertFalse(Assert.java:79)
at org.apache.solr.cloud.ReplaceNodeTest.test(ReplaceNodeTest.java:99)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.TestMiniSolrCloudClusterSSL.testNoSsl

Error Message:
Could not find collection:first_collection

Stack Trace:
java.lang.AssertionError: Could not find collection:first_collection
at 
__randomizedtesting.SeedInfo.seed([BA4CB527817F8441:89F491A9227FB887]:0)
at org.junit.Assert.fail(Assert.java:93)

Re: new candidate list for BadApple-ing on Saturday, 17-Mar

[Gus Heck]

Beginning to answer my own question...

public abstract class AbstractZkTestCase extends SolrTestCaseJ4 {
  private static final String ZOOKEEPER_FORCE_SYNC = "zookeeper.forceSync";

  public static final int TIMEOUT = 45000;

seems to be at least one place we are probably not getting what we
expect... the server's going to cut that back to the 20 second max...

On Wed, Mar 14, 2018 at 10:36 PM, Gus Heck  wrote:

> Being slightly irritated by the fact one of my tests shows up in this, I
> did some digging and I found
>
>[junit4]   2> 485679 ERROR 
> (OverseerThreadFactory-788-thread-1-processing-n:127.0.0.1:35771_solr) 
> [n:127.0.0.1:35771_solr] o.a.s.c.a.c.OverseerCollectionMessageHandler 
> Collection: testAliasCplx0 operation: createalias 
> failed:org.apache.zookeeper.KeeperException$SessionExpiredException: 
> KeeperErrorCode = Session expired for /configs/_default
>[junit4]   2>  at 
> org.apache.zookeeper.KeeperException.create(KeeperException.java:130)
>[junit4]   2>  at 
> org.apache.zookeeper.KeeperException.create(KeeperException.java:54)
>[junit4]   2>  at 
> org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1105)
>[junit4]   2>  at 
> org.apache.solr.common.cloud.SolrZkClient.lambda$exists$3(SolrZkClient.java:316)
>[junit4]   2>  at 
> org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
>[junit4]   2>  at 
> org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:316)
>[junit4]   2>  at 
> org.apache.solr.client.solrj.impl.ZkDistribStateManager.hasData(ZkDistribStateManager.java:58)
>[junit4]   2>  at 
> org.apache.solr.cloud.api.collections.OverseerCollectionMessageHandler.validateConfigOrThrowSolrException(OverseerCollectionMessageHandler.java:737)
>[junit4]   2>  at 
> org.apache.solr.cloud.api.collections.CreateCollectionCmd.call(CreateCollectionCmd.java:114)
>[junit4]   2>  at 
> org.apache.solr.cloud.api.collections.MaintainRoutedAliasCmd.createCollectionAndWait(MaintainRoutedAliasCmd.java:282)
>[junit4]   2>  at 
> org.apache.solr.cloud.api.collections.CreateAliasCmd.callCreateRoutedAlias(CreateAliasCmd.java:124)
>[junit4]   2>  at 
> org.apache.solr.cloud.api.collections.CreateAliasCmd.call(CreateAliasCmd.java:65)
>[junit4]   2>  at 
> org.apache.solr.cloud.api.collections.OverseerCollectionMessageHandler.processMessage(OverseerCollectionMessageHandler.java:252)
>[junit4]   2>  at 
> org.apache.solr.cloud.OverseerTaskProcessor$Runner.run(OverseerTaskProcessor.java:469)
>[junit4]   2>  at 
> org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
>[junit4]   2>  at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>[junit4]   2>  at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>[junit4]   2>  at java.lang.Thread.run(Thread.java:748)
>[junit4]   2>
>
> in the full log for https://jenkins.thetaphi.de/view/Lucene-Solr/job/
> Lucene-Solr-master-Solaris/1719/console
>
> After some digging I found this code in ZkTestServer:
>
> public void run() throws InterruptedException {
>   log.info("STARTING ZK TEST SERVER");
>   // we don't call super.distribSetUp
>   zooThread = new Thread() {
>
> @Override
> public void run() {
>   ServerConfig config = new ServerConfig() {
>
> {
>   setClientPort(ZkTestServer.this.clientPort);
>   this.dataDir = zkDir;
>   this.dataLogDir = zkDir;
>   this.tickTime = theTickTime;
> }
>
> public void setClientPort(int clientPort) {
>   if (clientPortAddress != null) {
> try {
>   this.clientPortAddress = new InetSocketAddress(
>   InetAddress.getByName(clientPortAddress.getHostName()), 
> clientPort);
> } catch (UnknownHostException e) {
>   throw new RuntimeException(e);
> }
>   } else {
> this.clientPortAddress = new InetSocketAddress(clientPort);
>   }
>   log.info("client port:" + this.clientPortAddress);
> }
>   };
>
>   try {
> zkServer.runFromConfig(config);
>   } catch (Throwable e) {
> throw new RuntimeException(e);
>   }
> }
>   };
>
> And what I noticed is that min/max timeouts are unset and theTickTime is
> onlly ever set to a big blue 1000 leading to default min/max timeout
> values of 2/20 seconds (https://discuss.pivotal.io/
> hc/en-us/articles/205187157-Pivotal-HD-About-how-to-
> correctly-config-zookeeper-session-timeout-parameter-
> minSessionTimeout-and-maxSessionTimeout --> jibes with the zk code I see
> in my editor). So my question is whether or not our big blue friend can be
> given more time, or is this by design and a config level we wish to
> explicitly support?
>
> Note 

[JENKINS] Lucene-Solr-7.x-Linux (32bit/jdk1.8.0_162) - Build # 1534 - Unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-7.x-Linux/1534/
Java: 32bit/jdk1.8.0_162 -server -XX:+UseG1GC

3 tests failed.
FAILED:  org.apache.lucene.index.TestIndexWriter.testSoftUpdatesConcurrently

Error Message:
expected:<2> but was:<56>

Stack Trace:
java.lang.AssertionError: expected:<2> but was:<56>
at 
__randomizedtesting.SeedInfo.seed([A4DEA0EBFCE94193:F45800EB40750535]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.junit.Assert.assertEquals(Assert.java:456)
at 
org.apache.lucene.index.TestIndexWriter.testSoftUpdatesConcurrently(TestIndexWriter.java:3237)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.lucene.index.TestIndexWriter.testSoftUpdatesConcurrently

Error Message:
expected:<2> but was:<56>

Stack Trace:
java.lang.AssertionError: expected:<2> but was:<56>
at 
__randomizedtesting.SeedInfo.seed([A4DEA0EBFCE94193:F45800EB40750535]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
  

Re: new candidate list for BadApple-ing on Saturday, 17-Mar

[Gus Heck]

Being slightly irritated by the fact one of my tests shows up in this, I
did some digging and I found

   [junit4]   2> 485679 ERROR
(OverseerThreadFactory-788-thread-1-processing-n:127.0.0.1:35771_solr)
[n:127.0.0.1:35771_solr]
o.a.s.c.a.c.OverseerCollectionMessageHandler Collection:
testAliasCplx0 operation: createalias
failed:org.apache.zookeeper.KeeperException$SessionExpiredException:
KeeperErrorCode = Session expired for /configs/_default
   [junit4]   2>at
org.apache.zookeeper.KeeperException.create(KeeperException.java:130)
   [junit4]   2>at
org.apache.zookeeper.KeeperException.create(KeeperException.java:54)
   [junit4]   2>at 
org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1105)
   [junit4]   2>at
org.apache.solr.common.cloud.SolrZkClient.lambda$exists$3(SolrZkClient.java:316)
   [junit4]   2>at
org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
   [junit4]   2>at
org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:316)
   [junit4]   2>at
org.apache.solr.client.solrj.impl.ZkDistribStateManager.hasData(ZkDistribStateManager.java:58)
   [junit4]   2>at
org.apache.solr.cloud.api.collections.OverseerCollectionMessageHandler.validateConfigOrThrowSolrException(OverseerCollectionMessageHandler.java:737)
   [junit4]   2>at
org.apache.solr.cloud.api.collections.CreateCollectionCmd.call(CreateCollectionCmd.java:114)
   [junit4]   2>at
org.apache.solr.cloud.api.collections.MaintainRoutedAliasCmd.createCollectionAndWait(MaintainRoutedAliasCmd.java:282)
   [junit4]   2>at
org.apache.solr.cloud.api.collections.CreateAliasCmd.callCreateRoutedAlias(CreateAliasCmd.java:124)
   [junit4]   2>at
org.apache.solr.cloud.api.collections.CreateAliasCmd.call(CreateAliasCmd.java:65)
   [junit4]   2>at
org.apache.solr.cloud.api.collections.OverseerCollectionMessageHandler.processMessage(OverseerCollectionMessageHandler.java:252)
   [junit4]   2>at
org.apache.solr.cloud.OverseerTaskProcessor$Runner.run(OverseerTaskProcessor.java:469)
   [junit4]   2>at
org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
   [junit4]   2>at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
   [junit4]   2>at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
   [junit4]   2>at java.lang.Thread.run(Thread.java:748)
   [junit4]   2>

in the full log for
https://jenkins.thetaphi.de/view/Lucene-Solr/job/Lucene-Solr-master-Solaris/1719/console

After some digging I found this code in ZkTestServer:

public void run() throws InterruptedException {
  log.info("STARTING ZK TEST SERVER");
  // we don't call super.distribSetUp
  zooThread = new Thread() {

@Override
public void run() {
  ServerConfig config = new ServerConfig() {

{
  setClientPort(ZkTestServer.this.clientPort);
  this.dataDir = zkDir;
  this.dataLogDir = zkDir;
  this.tickTime = theTickTime;
}

public void setClientPort(int clientPort) {
  if (clientPortAddress != null) {
try {
  this.clientPortAddress = new InetSocketAddress(

InetAddress.getByName(clientPortAddress.getHostName()), clientPort);
} catch (UnknownHostException e) {
  throw new RuntimeException(e);
}
  } else {
this.clientPortAddress = new InetSocketAddress(clientPort);
  }
  log.info("client port:" + this.clientPortAddress);
}
  };

  try {
zkServer.runFromConfig(config);
  } catch (Throwable e) {
throw new RuntimeException(e);
  }
}
  };

And what I noticed is that min/max timeouts are unset and theTickTime is
onlly ever set to a big blue 1000 leading to default min/max timeout values
of 2/20 seconds (
https://discuss.pivotal.io/hc/en-us/articles/205187157-Pivotal-HD-About-how-to-correctly-config-zookeeper-session-timeout-parameter-minSessionTimeout-and-maxSessionTimeout
--> jibes with the zk code I see in my editor). So my question is whether
or not our big blue friend can be given more time, or is this by design and
a config level we wish to explicitly support?

Note that this is not the default zk config, which would be a 3 sec tick
time. Maybe we're unintentionally being harsh on ourselves? I have noticed
that a lot of the tests that fail in my local builds are zk realated



On Wed, Mar 14, 2018 at 5:52 PM, Erick Erickson 
wrote:

> We had a drop off in the number of failing tests over the last couple
> of days, so I'm
> going to ignore the fails 11-12 Mar. Or it was a temporary increase
> for those days, take your
> pick ;)
>
>
> I'll check against Hoss' and Mark's lists and only BadApple tests
> that've failed Friday
> or later.
>
> In particular what do the 

[jira] [Commented] (SOLR-11891) BinaryResponseWriter fetches unnecessary fields

[Hoss Man (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16399802#comment-16399802
 ] 

Hoss Man commented on SOLR-11891:
-



bq. FYI I attached the diff we made to DocsStreamer.  

Thanks wei -- unfortunately your patch breaks compilation of some other classes 
(on master) but also suffers from an NPE in the case where globs are used (ex: 
{{fl=\*}}

I started down the road of a more "optimized" patch with what i suggested 
above...

bq. I think the ideal "fix" is that the SolrReturnFields.getLuceneFieldNames() 
should get passed down all the way into convertLuceneDocToSolrDoc (or something 
we refactor it into) such that we do an runtime check of which list is smaller: 
SolrReturnFields.getLuceneFieldNames() or Document.getFields() – and then loop 
over that (smallest) list.

...and i've currently got a patch which implements this along with a whitebox 
test to assert that the "optimization" is being used -- but while working on it 
i realized this isn't actually an optimization...

{code}
for (String fname : returnFieldNames) {
  for (IndexableField f : doc.getFields(fname)) {
// do stuff
  }
}
{code}
The problem is that {{Document}} isn't a Map -- it doesn't have efficient 
lookup of the values associated with a fieldname.  In order to do the 
{{fieldname=>value[]}} lookup of {{doc.getFields(fname)}}, it has to do an 
iterative scan all of the internal {{IndexableField}} (it can't even short 
circut out when it finds one because there could be multiples with the same 
name, and there's no garuntee they are in a predictible order)

So with this "optimization" we're actually introducing *more* loops over all 
the {{IndexableField}} instances.

The key reason wei was probably aple to see an improvement with hte change 
mentioned, is because at least when {{convertDocumentToSolrDocument}} is done, 
the final {{SolrDocumnet}} is as small as possible, so the *subsequent* scans 
in the ResponseWriter are faster.

We should be able to accomplish the same speed up "safely" by ensuring that 
when we do loop over the {{IndexableField}} instances, we check 
{{ReturnField.wantsField(fname)}}

I'll work on a revised (and much simpler) patch tomorow.




> BinaryResponseWriter fetches unnecessary fields
> ---
>
> Key: SOLR-11891
> URL: https://issues.apache.org/jira/browse/SOLR-11891
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Response Writers
>Affects Versions: 5.4, 6.4.2, 6.6.2
>Reporter: wei wang
>Priority: Major
> Attachments: DocsStreamer.java.diff, SOLR-11891.patch.BAD
>
>
> We observe that solr query time increases significantly with the number of 
> rows requested,  even all we retrieve for each document is just fl=id,score.  
> Debugged a bit and see that most of the increased time was spent in 
> BinaryResponseWriter,  converting lucene document into SolrDocument.  Inside 
> convertLuceneDocToSolrDoc():   
> [https://github.com/apache/lucene-solr/blob/df874432b9a17b547acb24a01d3491839e6a6b69/solr/core/src/java/org/apache/solr/response/DocsStreamer.java#L182]
>  
> I am a bit puzzled why we need to iterate through all the fields in the 
> document. Why can’t we just iterate through the requested field list?    
> [https://github.com/apache/lucene-solr/blob/df874432b9a17b547acb24a01d3491839e6a6b69/solr/core/src/java/org/apache/solr/response/DocsStreamer.java#L156]
>  
> e.g. when pass in the field list as 
> sdoc = convertLuceneDocToSolrDoc(doc, rctx.getSearcher().getSchema(), fnames)
> and just iterate through fnames,  there is a significant performance boost in 
> our case.  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-11891) BinaryResponseWriter fetches unnecessary fields

[Hoss Man (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-11891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Hoss Man updated SOLR-11891:

Attachment: SOLR-11891.patch.BAD

> BinaryResponseWriter fetches unnecessary fields
> ---
>
> Key: SOLR-11891
> URL: https://issues.apache.org/jira/browse/SOLR-11891
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Response Writers
>Affects Versions: 5.4, 6.4.2, 6.6.2
>Reporter: wei wang
>Priority: Major
> Attachments: DocsStreamer.java.diff, SOLR-11891.patch.BAD
>
>
> We observe that solr query time increases significantly with the number of 
> rows requested,  even all we retrieve for each document is just fl=id,score.  
> Debugged a bit and see that most of the increased time was spent in 
> BinaryResponseWriter,  converting lucene document into SolrDocument.  Inside 
> convertLuceneDocToSolrDoc():   
> [https://github.com/apache/lucene-solr/blob/df874432b9a17b547acb24a01d3491839e6a6b69/solr/core/src/java/org/apache/solr/response/DocsStreamer.java#L182]
>  
> I am a bit puzzled why we need to iterate through all the fields in the 
> document. Why can’t we just iterate through the requested field list?    
> [https://github.com/apache/lucene-solr/blob/df874432b9a17b547acb24a01d3491839e6a6b69/solr/core/src/java/org/apache/solr/response/DocsStreamer.java#L156]
>  
> e.g. when pass in the field list as 
> sdoc = convertLuceneDocToSolrDoc(doc, rctx.getSearcher().getSchema(), fnames)
> and just iterate through fnames,  there is a significant performance boost in 
> our case.  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-12059) Unable to rename solr.xml

[Edwin Yeo Zheng Lin (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-12059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16399726#comment-16399726
 ] 

Edwin Yeo Zheng Lin commented on SOLR-12059:


Probably the focus here is to just allow the rename of solr.xml, since other 
filenames in the first layer can be successfully renamed except solr.xml? The 
SolrXmlConfig is under the Jar file solr-core-7.2.1.jar, which we have renamed 
it to my-core-7.2.1.jar
For other things in the log, we can deal with them again another time if 
required. I do see some of the entries in the log are hard coded in the source 
code.
 

> Unable to rename solr.xml
> -
>
> Key: SOLR-12059
> URL: https://issues.apache.org/jira/browse/SOLR-12059
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Affects Versions: 6.5.1
> Environment: Renaming of solr,xml in the $SOLR_HOME directory
>Reporter: Edwin Yeo Zheng Lin
>Priority: Major
>
> I am able to rename the flie names like solrconfig.xml and solr.log to custom 
> names like myconfig.xml and my.log quite seamlessly. 
> However, I am not able to rename the same for solr.xml. Understand that the 
> solr.xml is hard-coded at the SolrXmlConfig.java. Meaning it requires a 
> re-compile of the Jar file in order to rename it.
> Since we can rename files like solrconfig.xml from the properties files, so 
> we should do the same for solr.xml?
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-7.x-Windows (32bit/jdk1.8.0_144) - Build # 505 - Still unstable!

[Policeman Jenkins Server]

Error processing tokens: Error while parsing action 
'Text/ZeroOrMore/FirstOf/Token/DelimitedToken/DelimitedToken_Action3' at input 
position (line 79, pos 4):
)"}
   ^

java.lang.OutOfMemoryError: Java heap space

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-Tests-7.x - Build # 503 - Still Unstable

[Apache Jenkins Server]

Build: https://builds.apache.org/job/Lucene-Solr-Tests-7.x/503/

1 tests failed.
FAILED:  org.apache.solr.cloud.MoveReplicaHDFSTest.testFailedMove

Error Message:
No live SolrServers available to handle this 
request:[https://127.0.0.1:38276/solr/MoveReplicaHDFSTest_failed_coll_true, 
https://127.0.0.1:51017/solr/MoveReplicaHDFSTest_failed_coll_true]

Stack Trace:
org.apache.solr.client.solrj.SolrServerException: No live SolrServers available 
to handle this 
request:[https://127.0.0.1:38276/solr/MoveReplicaHDFSTest_failed_coll_true, 
https://127.0.0.1:51017/solr/MoveReplicaHDFSTest_failed_coll_true]
at 
__randomizedtesting.SeedInfo.seed([EEBE4A76D80EAE:AA236DB8C10BDB7E]:0)
at 
org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:462)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1105)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:885)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:992)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:992)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:818)
at 
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:942)
at 
org.apache.solr.cloud.MoveReplicaTest.testFailedMove(MoveReplicaTest.java:307)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 

[JENKINS] Lucene-Solr-master-Linux (32bit/jdk1.8.0_162) - Build # 21641 - Unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/21641/
Java: 32bit/jdk1.8.0_162 -client -XX:+UseParallelGC

1 tests failed.
FAILED:  org.apache.solr.cloud.SSLMigrationTest.test

Error Message:
Replica didn't have the proper urlScheme in the ClusterState

Stack Trace:
java.lang.AssertionError: Replica didn't have the proper urlScheme in the 
ClusterState
at 
__randomizedtesting.SeedInfo.seed([7CFBE227E3AB8330:F4AFDDFD4D57EEC8]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at 
org.apache.solr.cloud.SSLMigrationTest.assertReplicaInformation(SSLMigrationTest.java:103)
at 
org.apache.solr.cloud.SSLMigrationTest.testMigrateSSL(SSLMigrationTest.java:96)
at org.apache.solr.cloud.SSLMigrationTest.test(SSLMigrationTest.java:60)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 

[jira] [Comment Edited] (SOLR-12088) Shards with dead replicas cause increased write latency

[Jerry Bao (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-12088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16399608#comment-16399608
 ] 

Jerry Bao edited comment on SOLR-12088 at 3/14/18 11:35 PM:


We've been running on Solr 7.2.1, so its all been state.json and not 
clusterstate.json.

In regards to re-issuing the DELETEREPLICA command, sometimes that fails and I 
filed a Jira for that here: SOLR-12087. That was what was causing this second 
issue here.

For example purposes, our indexing latency went from 2s to 1.7s after 
successfully deleting the dead replicas. One thing I did notice is that the 
dead replicas spam the logs with "unable to unload non-existent core" on the 
machine that hosts the dead replicas. Could be a side affect?


was (Author: jerry.bao):
We've been running on Solr 7.2.1, so its all been state.json and not 
clusterstate.json.

In regards to re-issuing the DELETEREPLICA command, sometimes that fails and I 
filed a Jira for that here: SOLR-12087. That was what was causing this second 
issue here.

For example purposes, our indexing latency went from 2s to 1.7s after deleting 
the dead replicas. One thing I did notice is that the dead replicas spam the 
logs with "unable to unload non-existent core" on the machine that hosts the 
dead replicas. Could be a side affect?

> Shards with dead replicas cause increased write latency
> ---
>
> Key: SOLR-12088
> URL: https://issues.apache.org/jira/browse/SOLR-12088
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Affects Versions: 7.2
>Reporter: Jerry Bao
>Priority: Major
>
> If a collection's shard contains dead replicas, write latency to the 
> collection is increased. For example, if a collection has 10 shards with a 
> replication factor of 3, and one of those shards contains 3 replicas and 3 
> downed replicas, write latency is increased in comparison to a shard that 
> contains only 3 replicas.
> My feeling here is that downed replicas should be completely ignored and not 
> cause issues to other alive replicas in terms of write latency.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-12088) Shards with dead replicas cause increased write latency

[Jerry Bao (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-12088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16399608#comment-16399608
 ] 

Jerry Bao commented on SOLR-12088:
--

We've been running on Solr 7.2.1, so its all been state.json and not 
clusterstate.json.

In regards to re-issuing the DELETEREPLICA command, sometimes that fails and I 
filed a Jira for that here: SOLR-12087. That was what was causing this second 
issue here.

For example purposes, our indexing latency went from 2s to 1.7s after deleting 
the dead replicas. One thing I did notice is that the dead replicas spam the 
logs with "unable to unload non-existent core" on the machine that hosts the 
dead replicas. Could be a side affect?

> Shards with dead replicas cause increased write latency
> ---
>
> Key: SOLR-12088
> URL: https://issues.apache.org/jira/browse/SOLR-12088
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Affects Versions: 7.2
>Reporter: Jerry Bao
>Priority: Major
>
> If a collection's shard contains dead replicas, write latency to the 
> collection is increased. For example, if a collection has 10 shards with a 
> replication factor of 3, and one of those shards contains 3 replicas and 3 
> downed replicas, write latency is increased in comparison to a shard that 
> contains only 3 replicas.
> My feeling here is that downed replicas should be completely ignored and not 
> cause issues to other alive replicas in terms of write latency.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-7.x-Solaris (64bit/jdk1.8.0) - Build # 493 - Unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-7.x-Solaris/493/
Java: 64bit/jdk1.8.0 -XX:+UseCompressedOops -XX:+UseParallelGC

2 tests failed.
FAILED:  org.apache.solr.cloud.MoveReplicaHDFSTest.testFailedMove

Error Message:
No live SolrServers available to handle this 
request:[http://127.0.0.1:50617/solr/MoveReplicaHDFSTest_failed_coll_true, 
http://127.0.0.1:41003/solr/MoveReplicaHDFSTest_failed_coll_true]

Stack Trace:
org.apache.solr.client.solrj.SolrServerException: No live SolrServers available 
to handle this 
request:[http://127.0.0.1:50617/solr/MoveReplicaHDFSTest_failed_coll_true, 
http://127.0.0.1:41003/solr/MoveReplicaHDFSTest_failed_coll_true]
at 
__randomizedtesting.SeedInfo.seed([1EF46114A7261B3A:B439B2E610F5CEEA]:0)
at 
org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:462)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1105)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:885)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:992)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:818)
at 
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:942)
at 
org.apache.solr.cloud.MoveReplicaTest.testFailedMove(MoveReplicaTest.java:307)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 

[JENKINS] Lucene-Solr-Tests-7.3 - Build # 4 - Unstable

[Apache Jenkins Server]

Build: https://builds.apache.org/job/Lucene-Solr-Tests-7.3/4/

1 tests failed.
FAILED:  
org.apache.solr.handler.component.TestDistributedStatsComponentCardinality.test

Error Message:
int_i: goodEst=13932, poorEst=13970, real=13964, 
p=q=id_i1:[329+TO+14292]=0=true={!cardinality%3D0.0014525142554553394+key%3Dlow_int_i}int_i={!cardinality%3D0.0014525142554553394+key%3Dlow_int_i_prehashed_l+hllPreHashed%3Dtrue}int_i_prehashed_l={!cardinality%3D0.5014525142554553+key%3Dhigh_int_i}int_i={!cardinality%3D0.5014525142554553+key%3Dhigh_int_i_prehashed_l+hllPreHashed%3Dtrue}int_i_prehashed_l={!cardinality%3D0.0014525142554553394+key%3Dlow_long_l}long_l={!cardinality%3D0.0014525142554553394+key%3Dlow_long_l_prehashed_l+hllPreHashed%3Dtrue}long_l_prehashed_l={!cardinality%3D0.5014525142554553+key%3Dhigh_long_l}long_l={!cardinality%3D0.5014525142554553+key%3Dhigh_long_l_prehashed_l+hllPreHashed%3Dtrue}long_l_prehashed_l={!cardinality%3D0.0014525142554553394+key%3Dlow_string_s}string_s={!cardinality%3D0.0014525142554553394+key%3Dlow_string_s_prehashed_l+hllPreHashed%3Dtrue}string_s_prehashed_l={!cardinality%3D0.5014525142554553+key%3Dhigh_string_s}string_s={!cardinality%3D0.5014525142554553+key%3Dhigh_string_s_prehashed_l+hllPreHashed%3Dtrue}string_s_prehashed_l

Stack Trace:
java.lang.AssertionError: int_i: goodEst=13932, poorEst=13970, real=13964, 
p=q=id_i1:[329+TO+14292]=0=true={!cardinality%3D0.0014525142554553394+key%3Dlow_int_i}int_i={!cardinality%3D0.0014525142554553394+key%3Dlow_int_i_prehashed_l+hllPreHashed%3Dtrue}int_i_prehashed_l={!cardinality%3D0.5014525142554553+key%3Dhigh_int_i}int_i={!cardinality%3D0.5014525142554553+key%3Dhigh_int_i_prehashed_l+hllPreHashed%3Dtrue}int_i_prehashed_l={!cardinality%3D0.0014525142554553394+key%3Dlow_long_l}long_l={!cardinality%3D0.0014525142554553394+key%3Dlow_long_l_prehashed_l+hllPreHashed%3Dtrue}long_l_prehashed_l={!cardinality%3D0.5014525142554553+key%3Dhigh_long_l}long_l={!cardinality%3D0.5014525142554553+key%3Dhigh_long_l_prehashed_l+hllPreHashed%3Dtrue}long_l_prehashed_l={!cardinality%3D0.0014525142554553394+key%3Dlow_string_s}string_s={!cardinality%3D0.0014525142554553394+key%3Dlow_string_s_prehashed_l+hllPreHashed%3Dtrue}string_s_prehashed_l={!cardinality%3D0.5014525142554553+key%3Dhigh_string_s}string_s={!cardinality%3D0.5014525142554553+key%3Dhigh_string_s_prehashed_l+hllPreHashed%3Dtrue}string_s_prehashed_l
at 
__randomizedtesting.SeedInfo.seed([B20EF2A0F2E717D1:3A5ACD7A5C1B7A29]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at 
org.apache.solr.handler.component.TestDistributedStatsComponentCardinality.test(TestDistributedStatsComponentCardinality.java:223)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at 

Re: TestReplicationHandler is failing 100% (master and 7.x / 7.3)

[Erick Erickson]

Shalin:

Should we remove (actually, comment out with a date?) the BadApple
annotation for doTestIndexFetchOnMasterRestart? And do you think your
fixes have any influence on the other BadApple
(doTestIndexAndConfigReplication)?

There's no problem with un-BadApple-ing test that have been or are
being worked on, and we'd get more test coverage that way.

Or I can do that on Saturday if you'd prefer, assuming the Jenkins
BadApple tests don't show failures.



On Wed, Mar 14, 2018 at 1:57 PM, Shalin Shekhar Mangar
 wrote:
> This is fixed. I committed the fix to master, branch_7x and branch_7_3
> branches.
>
> On Wed, Mar 14, 2018 at 9:44 PM, Alan Woodward  wrote:
>>
>> Thanks Shalin!
>>
>>
>> On 14 Mar 2018, at 15:50, Shalin Shekhar Mangar 
>> wrote:
>>
>> I'll take a look at it tomorrow morning my time.
>>
>> On Wed, Mar 14, 2018 at 9:07 PM, Andrzej Białecki
>>  wrote:
>>>
>>> Well … I looked at it briefly but I have no idea what’s going on there. I
>>> could dig into it nonetheless, but if there’s someone who already knows the
>>> replication handler ins and outs it would probably get fixed sooner...
>>>
>>>
>>> On 14 Mar 2018, at 14:23, Alan Woodward  wrote:
>>>
>>> I’m happy either way, but if it’s a bug can we get it fixed quickly?  Can
>>> you take ownership of this one Andrzej?
>>>
>>> On 14 Mar 2018, at 11:24, Andrzej Białecki  wrote:
>>>
>>> Hi,
>>>
>>> This test has always been fragile, but recently it’s been failing 100%,
>>> most often in ‘doTestIndexFetchOnMasterRestart’.
>>>
>>> I don’t know the replication handler enough to be able to find the real
>>> reason behind these failures, but there are two possibilities that I see:
>>>
>>> * the test has a bug and needs to be fixed - and if we can’t fix it soon
>>> then with 7.3 release imminent we could BadApple it until it’s properly
>>> fixed
>>>
>>> * or actually the replication handler has a bug, which needs to be fixed
>>> - in which case I propose to bump up SOLR-12078 to Blocker.
>>>
>>> I’m open to suggestions.
>>>
>>> —
>>>
>>> Andrzej Białecki
>>>
>>>
>>>
>>
>>
>>
>> --
>> Regards,
>> Shalin Shekhar Mangar.
>>
>>
>
>
>
> --
> Regards,
> Shalin Shekhar Mangar.

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-master-Solaris (64bit/jdk1.8.0) - Build # 1732 - Failure!

[Policeman Jenkins Server]

Error processing tokens: Error while parsing action 
'Text/ZeroOrMore/FirstOf/Token/DelimitedToken/DelimitedToken_Action3' at input 
position (line 79, pos 4):
)"}
   ^

java.lang.OutOfMemoryError: Java heap space

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

new candidate list for BadApple-ing on Saturday, 17-Mar

[Erick Erickson]

We had a drop off in the number of failing tests over the last couple
of days, so I'm
going to ignore the fails 11-12 Mar. Or it was a temporary increase
for those days, take your
pick ;)


I'll check against Hoss' and Mark's lists and only BadApple tests
that've failed Friday
or later.

In particular what do the Lucene folks think about the two Lucene tests?


junit.framework.TestSuite.org.apache.solr.cloud.TestLeaderElectionZkExpiry

org.apache.lucene.index.TestIndexSorting.testRandom3
org.apache.lucene.index.TestIndexWriterWithThreads.testCloseWithThreads

org.apache.solr.cloud.api.collections.CollectionsAPIAsyncDistributedZkTest.testAsyncIdBackCompat
org.apache.solr.cloud.autoscaling.ComputePlanActionTest.testNodeWithMultipleReplicasLost
org.apache.solr.cloud.autoscaling.ScheduledMaintenanceTriggerTest.testInactiveShardCleanup
org.apache.solr.cloud.ConcurrentCreateRoutedAliasTest.testConcurrentCreateRoutedAliasComplex
org.apache.solr.cloud.DocValuesNotIndexedTest.testGroupingDVOnly
org.apache.solr.cloud.LeaderVoteWaitTimeoutTest.basicTest
org.apache.solr.cloud.LeaderVoteWaitTimeoutTest.testMostInSyncReplicasCanWinElection
org.apache.solr.cloud.MoveReplicaHDFSTest.testFailedMove
org.apache.solr.cloud.SSLMigrationTest.test
org.apache.solr.cloud.TestTlogReplica.testCreateDelete
org.apache.solr.handler.admin.SegmentsInfoRequestHandlerTest.testSegmentInfosVersion
org.apache.solr.handler.TestSolrConfigHandlerCloud.test
org.apache.solr.logging.TestLogWatcher.testLog4jWatcher
org.apache.solr.spelling.SpellCheckCollatorTest.testEstimatedHitCounts


Fails by day are below for reference. If they're _not_ listed above, they'll be
left to run for another week:

11-Mar fails:

junit.framework.TestSuite.org.apache.solr.cloud.BasicDistributedZkTestream.StreamExpressionTest.testGammaDistribution
junit.framework.TestSuite.org.apache.solr.cloud.TestCloudPivotFacet
junit.framework.TestSuite.org.apache.solr.cloud.TriLevelCompositeIdRoutingTest
junit.framework.TestSuite.org.apache.solr.cloud.ZkControllerTest
junit.framework.TestSuite.org.apache.solr.search.join.BlockJoinFacetDistribTest
org.apache.solr.client.solrj.io.storg.apache.solr.cloud.BasicDistributedZkTest.test
org.apache.solr.cloud.autoscaling.ScheduledMaintenanceTriggerTest.testInactiveShardCleanup
org.apache.solr.cloud.BasicDistributedZkTest.test
org.apache.solr.cloud.FullSolrCloudDistribCmdsTest
org.apache.solr.cloud.FullSolrCloudDistribCmdsTest.test
org.apache.solr.cloud.hdfs.HdfsUnloadDistributedZkTest
org.apache.solr.cloud.hdfs.HdfsUnloadDistributedZkTest.test
org.apache.solr.cloud.hdfs.StressHdfsTest.test
org.apache.solr.cloud.MoveReplicaHDFSTest.testFailedMove
org.apache.solr.cloud.TestCloudPivotFacet.test
org.apache.solr.cloud.TriLevelCompositeIdRoutingTest.test
org.apache.solr.logging.TestLogWatcher.testLog4jWatcher

12-Mar fails:
junit.framework.TestSuite.org.apache.lucene.search.spans.TestSpanSearchEquivalence
junit.framework.TestSuite.org.apache.solr.cloud.autoscaling.TriggerIntegrationTest
junit.framework.TestSuite.org.apache.solr.cloud.BasicDistributedZkTest
junit.framework.TestSuite.org.apache.solr.cloud.hdfs.HdfsUnloadDistributedZkTest
junit.framework.TestSuite.org.apache.solr.cloud.TestLeaderElectionZkExpiry
org.apache.lucene.index.TestDuelingCodecsAtNight.testBigEquals
org.apache.lucene.search.spans.TestSpanSearchEquivalence.testSpanNearIncreasingSloppiness
org.apache.solr.cloud.autoscaling.AutoAddReplicasIntegrationTest
org.apache.solr.cloud.autoscaling.HdfsAutoAddReplicasIntegrationTest
org.apache.solr.cloud.autoscaling.ScheduledMaintenanceTriggerTest
org.apache.solr.cloud.autoscaling.ScheduledMaintenanceTriggerTest.testInactiveShardCleanup
org.apache.solr.cloud.autoscaling.TriggerIntegrationTest.testEventQueue
org.apache.solr.cloud.autoscaling.TriggerIntegrationTest.testMetricTrigger
org.apache.solr.cloud.BasicDistributedZkTest.test
org.apache.solr.cloud.CollectionsAPISolrJTest.testSplitShard
org.apache.solr.cloud.LeaderVoteWaitTimeoutTest.testMostInSyncReplicasCanWinElection
org.apache.solr.cloud.MoveReplicaHDFSTest.testFailedMove
org.apache.solr.cloud.SSLMigrationTest.test
org.apache.solr.cloud.TestRandomFlRTGCloud.testRandomizedUpdatesAndRTGs
org.apache.solr.core.TestJmxIntegration
org.apache.solr.handler.admin.AutoscalingHistoryHandlerTest
org.apache.solr.handler.TestReplicationHandler
org.apache.solr.handler.TestReplicationHandler.doTestReplicateAfterCoreReload
org.apache.solr.logging.TestLogWatcher.testLog4jWatcher
org.apache.solr.ltr.TestLTRReRankingPipeline
org.apache.solr.TestDistributedSearch.test

13-Mar fails:
junit.framework.TestSuite.org.apache.solr.cloud.TestLeaderElectionZkExpiry
org.apache.lucene.index.TestIndexSorting.testRandom3
org.apache.lucene.index.TestIndexWriterWithThreads.testCloseWithThreads
org.apache.solr.cloud.autoscaling.ScheduledMaintenanceTriggerTest.testInactiveShardCleanup
org.apache.solr.cloud.ConcurrentCreateRoutedAliasTest.testConcurrentCreateRoutedAliasComplex

[jira] [Commented] (SOLR-11882) SolrMetric registries retain references to SolrCores when closed

[Erick Erickson (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16399440#comment-16399440
 ] 

Erick Erickson commented on SOLR-11882:
---

[~ab] OK, I think the light finally dawned. We're talking about two different 
cases and they both have to be handled.

1> transient core case, the one I'm started with. In this case, the core is 
closed out and _may_, some time in the near or far future be opened again. In 
this case the patch from 28-Jan is probably almost fine although there's still 
a (probably small but unacceptable) chance that a new version of the core would 
be opened before the closer thread got 'round to closing the old one.

2> reopening a core which is the case you're talking about in your comment 
1-Feb.

In <2> there's no problem with cores accumulating due to the reference in the 
metrics code since they've been released by the new assignment already.

Does that make sense?

And is there a good way other than inspection to test any fixes I make?

Thanks!

> SolrMetric registries retain references to SolrCores when closed
> 
>
> Key: SOLR-11882
> URL: https://issues.apache.org/jira/browse/SOLR-11882
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: metrics, Server
>Affects Versions: 7.1
>Reporter: Eros Taborelli
>Assignee: Erick Erickson
>Priority: Major
> Attachments: SOLR-11882.patch, SOLR-11882.patch, SOLR-11882.patch, 
> SOLR-11882.patch, create-cores.zip, solr-dump-full_Leak_Suspects.zip, 
> solr.config.zip
>
>
> *Description:*
> Our setup involves using a lot of small cores (possibly hundred thousand), 
> but working only on a few of them at any given time.
> We already followed all recommendations in this guide: 
> [https://wiki.apache.org/solr/LotsOfCores]
> We noticed that after creating/loading around 1000-2000 empty cores, with no 
> documents inside, the heap consumption went through the roof despite having 
> set transientCacheSize to only 64 (heap size set to 12G).
> All cores are correctly set to loadOnStartup=false and transient=true, and we 
> have verified via logs that the cores in excess are actually being closed.
> However, a reference remains in the 
> org.apache.solr.metrics.SolrMetricManager#registries that is never removed 
> until a core if fully unloaded.
> Restarting the JVM loads all cores in the admin UI, but doesn't populate the 
> ConcurrentHashMap until a core is actually fully loaded.
> I reproduced the issue on a smaller scale (transientCacheSize = 5, heap size 
> = 512m) and made a report (attached) using eclipse MAT.
> *Desired outcome:*
> When a transient core is closed, the references in the SolrMetricManager 
> should be removed, in the same fashion the reporters for the core are also 
> closed and removed.
> In alternative, a unloadOnClose=true|false flag could be implemented to fully 
> unload a transient core when closed due to the cache size.
> *Note:*
> The documentation mentions everywhere that the unused cores will be unloaded, 
> but it's misleading as the cores are never fully unloaded.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-12063) Fix tlog entry indexes in UpdateLog for CDCR to work smoothly.

[Amrit Sarkar (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-12063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Amrit Sarkar updated SOLR-12063:

Attachment: SOLR-12063.patch

> Fix tlog entry indexes in UpdateLog for CDCR to work smoothly.
> --
>
> Key: SOLR-12063
> URL: https://issues.apache.org/jira/browse/SOLR-12063
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: CDCR
>Affects Versions: 7.2, 7.2.1
>Reporter: Amrit Sarkar
>Assignee: Varun Thacker
>Priority: Major
> Attachments: SOLR-12063.patch, SOLR-12063.patch, SOLR-12063.patch, 
> SOLR-12063.patch, SOLR-12063.patch, test-report-PeerSyncTest, 
> test-report-TestStressRecovery
>
>
> In *UpdateLog*, {{RecentUpdates}} reads the entry of tlogs, and throughout 
> the project the entry indexes for various operations are consistent, but odd 
> in this part. As we included new entry in TransactionLog for CDCR, read 
> operations in {{update()}} method of {{RecentUpdates}} throw error rightfully 
> as elements are read from wrong indexes of tlog entry. The entry indexes of 
> llog should be consistent throughout.
> {code}
>   [beaster]   2> 27394 WARN  (qtp97093533-72) [n:127.0.0.1:44658_solr 
> c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] 
> o.a.s.u.UpdateLog Unexpected log entry or corrupt log.  Entry=[2, 
> -1594312216007409664, [B@28e6859c, true]
>   [beaster]   2> java.lang.ClassCastException: java.lang.Boolean cannot be 
> cast to [B
>   [beaster]   2>  at 
> org.apache.solr.update.UpdateLog$RecentUpdates.update(UpdateLog.java:1443)
>   [beaster]   2>  at 
> org.apache.solr.update.UpdateLog$RecentUpdates.(UpdateLog.java:1340)
>   [beaster]   2>  at 
> org.apache.solr.update.UpdateLog.getRecentUpdates(UpdateLog.java:1513)
>   [beaster]   2>  at 
> org.apache.solr.handler.CdcrRequestHandler.handleShardCheckpointAction(CdcrRequestHandler.java:448)
>   [beaster]   2>  at 
> org.apache.solr.handler.CdcrRequestHandler.handleRequestBody(CdcrRequestHandler.java:198)
>   [beaster]   2>  at 
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:195)
>   [beaster]   2>  at 
> org.apache.solr.core.SolrCore.execute(SolrCore.java:2503)
>   [beaster]   2>  at 
> org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:711)
>   [beaster]   2>  at 
> org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:517)
>   [beaster]   2>  at 
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:384)
>   [beaster]   2>  at 
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:330)
>   [beaster]   2>  at 
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1637)
>   [beaster]   2>  at 
> org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139)
>   [beaster]   2>  at 
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1637)
>   [beaster]   2>  at 
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533)
>   [beaster]   2>  at 
> org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:188)
>   [beaster]   2>  at 
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595)
>   [beaster]   2>  at 
> org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:188)
>   [beaster]   2>  at 
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1253)
>   [beaster]   2>  at 
> org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:168)
>   [beaster]   2>  at 
> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473)
>   [beaster]   2>  at 
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564)
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-12063) Fix tlog entry indexes in UpdateLog for CDCR to work smoothly.

[Amrit Sarkar (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-12063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16399425#comment-16399425
 ] 

Amrit Sarkar commented on SOLR-12063:
-

Updated patch, included {{TestStressRecovery}}, precommit, beasts of 10 rounds 
pass.

> Fix tlog entry indexes in UpdateLog for CDCR to work smoothly.
> --
>
> Key: SOLR-12063
> URL: https://issues.apache.org/jira/browse/SOLR-12063
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: CDCR
>Affects Versions: 7.2, 7.2.1
>Reporter: Amrit Sarkar
>Assignee: Varun Thacker
>Priority: Major
> Attachments: SOLR-12063.patch, SOLR-12063.patch, SOLR-12063.patch, 
> SOLR-12063.patch, test-report-PeerSyncTest, test-report-TestStressRecovery
>
>
> In *UpdateLog*, {{RecentUpdates}} reads the entry of tlogs, and throughout 
> the project the entry indexes for various operations are consistent, but odd 
> in this part. As we included new entry in TransactionLog for CDCR, read 
> operations in {{update()}} method of {{RecentUpdates}} throw error rightfully 
> as elements are read from wrong indexes of tlog entry. The entry indexes of 
> llog should be consistent throughout.
> {code}
>   [beaster]   2> 27394 WARN  (qtp97093533-72) [n:127.0.0.1:44658_solr 
> c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] 
> o.a.s.u.UpdateLog Unexpected log entry or corrupt log.  Entry=[2, 
> -1594312216007409664, [B@28e6859c, true]
>   [beaster]   2> java.lang.ClassCastException: java.lang.Boolean cannot be 
> cast to [B
>   [beaster]   2>  at 
> org.apache.solr.update.UpdateLog$RecentUpdates.update(UpdateLog.java:1443)
>   [beaster]   2>  at 
> org.apache.solr.update.UpdateLog$RecentUpdates.(UpdateLog.java:1340)
>   [beaster]   2>  at 
> org.apache.solr.update.UpdateLog.getRecentUpdates(UpdateLog.java:1513)
>   [beaster]   2>  at 
> org.apache.solr.handler.CdcrRequestHandler.handleShardCheckpointAction(CdcrRequestHandler.java:448)
>   [beaster]   2>  at 
> org.apache.solr.handler.CdcrRequestHandler.handleRequestBody(CdcrRequestHandler.java:198)
>   [beaster]   2>  at 
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:195)
>   [beaster]   2>  at 
> org.apache.solr.core.SolrCore.execute(SolrCore.java:2503)
>   [beaster]   2>  at 
> org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:711)
>   [beaster]   2>  at 
> org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:517)
>   [beaster]   2>  at 
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:384)
>   [beaster]   2>  at 
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:330)
>   [beaster]   2>  at 
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1637)
>   [beaster]   2>  at 
> org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139)
>   [beaster]   2>  at 
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1637)
>   [beaster]   2>  at 
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533)
>   [beaster]   2>  at 
> org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:188)
>   [beaster]   2>  at 
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595)
>   [beaster]   2>  at 
> org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:188)
>   [beaster]   2>  at 
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1253)
>   [beaster]   2>  at 
> org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:168)
>   [beaster]   2>  at 
> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473)
>   [beaster]   2>  at 
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564)
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-12095) AutoScalingHandler should validate triggers before updating zookeeper

[Shalin Shekhar Mangar (JIRA)]

Shalin Shekhar Mangar created SOLR-12095:


 Summary: AutoScalingHandler should validate triggers before 
updating zookeeper
 Key: SOLR-12095
 URL: https://issues.apache.org/jira/browse/SOLR-12095
 Project: Solr
  Issue Type: Sub-task
  Security Level: Public (Default Security Level. Issues are Public)
  Components: AutoScaling, SolrCloud
Reporter: Shalin Shekhar Mangar
 Fix For: 7.4, master (8.0)


We validate policy and preferences before updating the configuration in 
Zookeeper but we don't do that today for triggers. So users can put wrong or 
unknown parameters and there won't be any complains from the API but the at 
runtime exceptions will be thrown/logged.

We should change the trigger API to have a validation step. The catch here is 
that it may require us to instantiate the trigger class.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (SOLR-12078) Reproducable Failure in TestReplicationHandler.doTestIndexFetchOnMasterRestart

[Shalin Shekhar Mangar (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-12078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shalin Shekhar Mangar resolved SOLR-12078.
--
   Resolution: Fixed
 Assignee: Shalin Shekhar Mangar
Fix Version/s: master (8.0)
   7.3

> Reproducable Failure in TestReplicationHandler.doTestIndexFetchOnMasterRestart
> --
>
> Key: SOLR-12078
> URL: https://issues.apache.org/jira/browse/SOLR-12078
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Affects Versions: 7.3, master (8.0)
> Environment: Building on Ubuntu 17.4
> openjdk version "1.8.0_151"
> OpenJDK Runtime Environment (build 1.8.0_151-8u151-b12-0ubuntu0.17.04.2-b12)
> OpenJDK 64-Bit Server VM (build 25.151-b12, mixed mode)
>Reporter: Gus Heck
>Assignee: Shalin Shekhar Mangar
>Priority: Major
> Fix For: 7.3, master (8.0)
>
> Attachments: SOLR-12078.patch
>
>
> With the recent focus on bad tests lately, I decided to inspect some failures 
> that occurred in tests unrelated to my present task when I ran the tests 
> preparing for a pull request and found this failure which reproduces:
> ant test  -Dtestcase=TestReplicationHandler 
> -Dtests.method=doTestIndexFetchOnMasterRestart -Dtests.seed=884DCF71D210D14A 
> -Dtests.slow=true -Dtests.badapples=true -Dtests.locale=et 
> -Dtests.timezone=Cuba -Dtests.asserts=true -Dtests.file.encoding=UTF-8
>  
> Key excerpt of the log:
> {code:java}
>    [junit4]   2> NOTE: reproduce with: ant test  
> -Dtestcase=TestReplicationHandler 
> -Dtests.method=doTestIndexFetchOnMasterRestart -Dtests.seed=884DCF71D210D14A 
> -Dtests.slow=true -Dtests.badapples=true -Dtests.locale=et 
> -Dtests.timezone=Cuba -Dtests.asserts=true -Dtests.file.encoding=UTF-8
>    [junit4] ERROR   2.35s | 
> TestReplicationHandler.doTestIndexFetchOnMasterRestart <<<
>    [junit4]    > Throwable #1: 
> org.apache.solr.client.solrj.SolrServerException: IOException occured when 
> talking to server at: http://127.0.0.1:37753/solr/collection1
>    [junit4]    >    at 
> __randomizedtesting.SeedInfo.seed([884DCF71D210D14A:50BA0B9579CB1316]:0)
>    [junit4]    >    at 
> org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:657)
>    [junit4]    >    at 
> org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
>    [junit4]    >    at 
> org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
>    [junit4]    >    at 
> org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
>    [junit4]    >    at 
> org.apache.solr.client.solrj.SolrClient.add(SolrClient.java:173)
>    [junit4]    >    at 
> org.apache.solr.client.solrj.SolrClient.add(SolrClient.java:138)
>    [junit4]    >    at 
> org.apache.solr.client.solrj.SolrClient.add(SolrClient.java:152)
>    [junit4]    >    at 
> org.apache.solr.handler.TestReplicationHandler.index(TestReplicationHandler.java:180)
>    [junit4]    >    at 
> org.apache.solr.handler.TestReplicationHandler.doTestIndexFetchOnMasterRestart(TestReplicationHandler.java:643)
>    [junit4]    >    at java.lang.Thread.run(Thread.java:748)
>    [junit4]    > Caused by: org.apache.http.NoHttpResponseException: 
> 127.0.0.1:37753 failed to respond
>    [junit4]    >    at 
> org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:141)
>    [junit4]    >    at 
> org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56)
>    [junit4]    >    at 
> org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:259)
>    [junit4]    >    at 
> org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:163)
>    [junit4]    >    at 
> org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165)
>    [junit4]    >    at 
> org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:273)
>    [junit4]    >    at 
> org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125)
>    [junit4]    >    at 
> org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272)
>    [junit4]    >    at 
> org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185)
>    [junit4]    >    at 
> org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)
>    [junit4]    >    at 
> org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111)
>    [junit4]    >    at 
> org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
>    

Re: TestReplicationHandler is failing 100% (master and 7.x / 7.3)

[Shalin Shekhar Mangar]

This is fixed. I committed the fix to master, branch_7x and branch_7_3
branches.

On Wed, Mar 14, 2018 at 9:44 PM, Alan Woodward  wrote:

> Thanks Shalin!
>
>
> On 14 Mar 2018, at 15:50, Shalin Shekhar Mangar 
> wrote:
>
> I'll take a look at it tomorrow morning my time.
>
> On Wed, Mar 14, 2018 at 9:07 PM, Andrzej Białecki <
> andrzej.biale...@lucidworks.com> wrote:
>
>> Well … I looked at it briefly but I have no idea what’s going on there. I
>> could dig into it nonetheless, but if there’s someone who already knows the
>> replication handler ins and outs it would probably get fixed sooner...
>>
>>
>> On 14 Mar 2018, at 14:23, Alan Woodward  wrote:
>>
>> I’m happy either way, but if it’s a bug can we get it fixed quickly?  Can
>> you take ownership of this one Andrzej?
>>
>> On 14 Mar 2018, at 11:24, Andrzej Białecki  wrote:
>>
>> Hi,
>>
>> This test has always been fragile, but recently it’s been failing 100%,
>> most often in ‘doTestIndexFetchOnMasterRestart’.
>>
>> I don’t know the replication handler enough to be able to find the real
>> reason behind these failures, but there are two possibilities that I see:
>>
>> * the test has a bug and needs to be fixed - and if we can’t fix it soon
>> then with 7.3 release imminent we could BadApple it until it’s properly
>> fixed
>>
>> * or actually the replication handler has a bug, which needs to be fixed
>> - in which case I propose to bump up SOLR-12078 to Blocker.
>>
>> I’m open to suggestions.
>>
>> —
>>
>> Andrzej Białecki
>>
>>
>>
>>
>
>
> --
> Regards,
> Shalin Shekhar Mangar.
>
>
>


-- 
Regards,
Shalin Shekhar Mangar.

[jira] [Commented] (SOLR-12078) Reproducable Failure in TestReplicationHandler.doTestIndexFetchOnMasterRestart

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-12078?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16399396#comment-16399396
 ] 

ASF subversion and git services commented on SOLR-12078:


Commit ace126bb0ed6fbfec12e133c2e15633a68e84aad in lucene-solr's branch 
refs/heads/branch_7_3 from [~shalinmangar]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=ace126b ]

SOLR-12078: Fixed reproducable Failure in 
TestReplicationHandler.doTestIndexFetchOnMasterRestart that happened due to 
using stale http connections

(cherry picked from commit cb453ce)

(cherry picked from commit 46f1b7f)


> Reproducable Failure in TestReplicationHandler.doTestIndexFetchOnMasterRestart
> --
>
> Key: SOLR-12078
> URL: https://issues.apache.org/jira/browse/SOLR-12078
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Affects Versions: 7.3, master (8.0)
> Environment: Building on Ubuntu 17.4
> openjdk version "1.8.0_151"
> OpenJDK Runtime Environment (build 1.8.0_151-8u151-b12-0ubuntu0.17.04.2-b12)
> OpenJDK 64-Bit Server VM (build 25.151-b12, mixed mode)
>Reporter: Gus Heck
>Priority: Major
> Attachments: SOLR-12078.patch
>
>
> With the recent focus on bad tests lately, I decided to inspect some failures 
> that occurred in tests unrelated to my present task when I ran the tests 
> preparing for a pull request and found this failure which reproduces:
> ant test  -Dtestcase=TestReplicationHandler 
> -Dtests.method=doTestIndexFetchOnMasterRestart -Dtests.seed=884DCF71D210D14A 
> -Dtests.slow=true -Dtests.badapples=true -Dtests.locale=et 
> -Dtests.timezone=Cuba -Dtests.asserts=true -Dtests.file.encoding=UTF-8
>  
> Key excerpt of the log:
> {code:java}
>    [junit4]   2> NOTE: reproduce with: ant test  
> -Dtestcase=TestReplicationHandler 
> -Dtests.method=doTestIndexFetchOnMasterRestart -Dtests.seed=884DCF71D210D14A 
> -Dtests.slow=true -Dtests.badapples=true -Dtests.locale=et 
> -Dtests.timezone=Cuba -Dtests.asserts=true -Dtests.file.encoding=UTF-8
>    [junit4] ERROR   2.35s | 
> TestReplicationHandler.doTestIndexFetchOnMasterRestart <<<
>    [junit4]    > Throwable #1: 
> org.apache.solr.client.solrj.SolrServerException: IOException occured when 
> talking to server at: http://127.0.0.1:37753/solr/collection1
>    [junit4]    >    at 
> __randomizedtesting.SeedInfo.seed([884DCF71D210D14A:50BA0B9579CB1316]:0)
>    [junit4]    >    at 
> org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:657)
>    [junit4]    >    at 
> org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
>    [junit4]    >    at 
> org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
>    [junit4]    >    at 
> org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
>    [junit4]    >    at 
> org.apache.solr.client.solrj.SolrClient.add(SolrClient.java:173)
>    [junit4]    >    at 
> org.apache.solr.client.solrj.SolrClient.add(SolrClient.java:138)
>    [junit4]    >    at 
> org.apache.solr.client.solrj.SolrClient.add(SolrClient.java:152)
>    [junit4]    >    at 
> org.apache.solr.handler.TestReplicationHandler.index(TestReplicationHandler.java:180)
>    [junit4]    >    at 
> org.apache.solr.handler.TestReplicationHandler.doTestIndexFetchOnMasterRestart(TestReplicationHandler.java:643)
>    [junit4]    >    at java.lang.Thread.run(Thread.java:748)
>    [junit4]    > Caused by: org.apache.http.NoHttpResponseException: 
> 127.0.0.1:37753 failed to respond
>    [junit4]    >    at 
> org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:141)
>    [junit4]    >    at 
> org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56)
>    [junit4]    >    at 
> org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:259)
>    [junit4]    >    at 
> org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:163)
>    [junit4]    >    at 
> org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165)
>    [junit4]    >    at 
> org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:273)
>    [junit4]    >    at 
> org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125)
>    [junit4]    >    at 
> org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272)
>    [junit4]    >    at 
> org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185)
>    [junit4]    >    at 
> 

[jira] [Commented] (SOLR-12078) Reproducable Failure in TestReplicationHandler.doTestIndexFetchOnMasterRestart

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-12078?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16399390#comment-16399390
 ] 

ASF subversion and git services commented on SOLR-12078:


Commit 46f1b7f654259e9191dc5282e1d58a12ea9a4025 in lucene-solr's branch 
refs/heads/branch_7x from [~shalinmangar]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=46f1b7f ]

SOLR-12078: Fixed reproducable Failure in 
TestReplicationHandler.doTestIndexFetchOnMasterRestart that happened due to 
using stale http connections

(cherry picked from commit cb453ce)


> Reproducable Failure in TestReplicationHandler.doTestIndexFetchOnMasterRestart
> --
>
> Key: SOLR-12078
> URL: https://issues.apache.org/jira/browse/SOLR-12078
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Affects Versions: 7.3, master (8.0)
> Environment: Building on Ubuntu 17.4
> openjdk version "1.8.0_151"
> OpenJDK Runtime Environment (build 1.8.0_151-8u151-b12-0ubuntu0.17.04.2-b12)
> OpenJDK 64-Bit Server VM (build 25.151-b12, mixed mode)
>Reporter: Gus Heck
>Priority: Major
> Attachments: SOLR-12078.patch
>
>
> With the recent focus on bad tests lately, I decided to inspect some failures 
> that occurred in tests unrelated to my present task when I ran the tests 
> preparing for a pull request and found this failure which reproduces:
> ant test  -Dtestcase=TestReplicationHandler 
> -Dtests.method=doTestIndexFetchOnMasterRestart -Dtests.seed=884DCF71D210D14A 
> -Dtests.slow=true -Dtests.badapples=true -Dtests.locale=et 
> -Dtests.timezone=Cuba -Dtests.asserts=true -Dtests.file.encoding=UTF-8
>  
> Key excerpt of the log:
> {code:java}
>    [junit4]   2> NOTE: reproduce with: ant test  
> -Dtestcase=TestReplicationHandler 
> -Dtests.method=doTestIndexFetchOnMasterRestart -Dtests.seed=884DCF71D210D14A 
> -Dtests.slow=true -Dtests.badapples=true -Dtests.locale=et 
> -Dtests.timezone=Cuba -Dtests.asserts=true -Dtests.file.encoding=UTF-8
>    [junit4] ERROR   2.35s | 
> TestReplicationHandler.doTestIndexFetchOnMasterRestart <<<
>    [junit4]    > Throwable #1: 
> org.apache.solr.client.solrj.SolrServerException: IOException occured when 
> talking to server at: http://127.0.0.1:37753/solr/collection1
>    [junit4]    >    at 
> __randomizedtesting.SeedInfo.seed([884DCF71D210D14A:50BA0B9579CB1316]:0)
>    [junit4]    >    at 
> org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:657)
>    [junit4]    >    at 
> org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
>    [junit4]    >    at 
> org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
>    [junit4]    >    at 
> org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
>    [junit4]    >    at 
> org.apache.solr.client.solrj.SolrClient.add(SolrClient.java:173)
>    [junit4]    >    at 
> org.apache.solr.client.solrj.SolrClient.add(SolrClient.java:138)
>    [junit4]    >    at 
> org.apache.solr.client.solrj.SolrClient.add(SolrClient.java:152)
>    [junit4]    >    at 
> org.apache.solr.handler.TestReplicationHandler.index(TestReplicationHandler.java:180)
>    [junit4]    >    at 
> org.apache.solr.handler.TestReplicationHandler.doTestIndexFetchOnMasterRestart(TestReplicationHandler.java:643)
>    [junit4]    >    at java.lang.Thread.run(Thread.java:748)
>    [junit4]    > Caused by: org.apache.http.NoHttpResponseException: 
> 127.0.0.1:37753 failed to respond
>    [junit4]    >    at 
> org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:141)
>    [junit4]    >    at 
> org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56)
>    [junit4]    >    at 
> org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:259)
>    [junit4]    >    at 
> org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:163)
>    [junit4]    >    at 
> org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165)
>    [junit4]    >    at 
> org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:273)
>    [junit4]    >    at 
> org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125)
>    [junit4]    >    at 
> org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272)
>    [junit4]    >    at 
> org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185)
>    [junit4]    >    at 
> org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)
>    

[jira] [Commented] (SOLR-12078) Reproducable Failure in TestReplicationHandler.doTestIndexFetchOnMasterRestart

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-12078?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16399389#comment-16399389
 ] 

ASF subversion and git services commented on SOLR-12078:


Commit cb453ce110b1a0b03373909c36d3d1bc25983b71 in lucene-solr's branch 
refs/heads/master from [~shalinmangar]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=cb453ce ]

SOLR-12078: Fixed reproducable Failure in 
TestReplicationHandler.doTestIndexFetchOnMasterRestart that happened due to 
using stale http connections


> Reproducable Failure in TestReplicationHandler.doTestIndexFetchOnMasterRestart
> --
>
> Key: SOLR-12078
> URL: https://issues.apache.org/jira/browse/SOLR-12078
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Affects Versions: 7.3, master (8.0)
> Environment: Building on Ubuntu 17.4
> openjdk version "1.8.0_151"
> OpenJDK Runtime Environment (build 1.8.0_151-8u151-b12-0ubuntu0.17.04.2-b12)
> OpenJDK 64-Bit Server VM (build 25.151-b12, mixed mode)
>Reporter: Gus Heck
>Priority: Major
> Attachments: SOLR-12078.patch
>
>
> With the recent focus on bad tests lately, I decided to inspect some failures 
> that occurred in tests unrelated to my present task when I ran the tests 
> preparing for a pull request and found this failure which reproduces:
> ant test  -Dtestcase=TestReplicationHandler 
> -Dtests.method=doTestIndexFetchOnMasterRestart -Dtests.seed=884DCF71D210D14A 
> -Dtests.slow=true -Dtests.badapples=true -Dtests.locale=et 
> -Dtests.timezone=Cuba -Dtests.asserts=true -Dtests.file.encoding=UTF-8
>  
> Key excerpt of the log:
> {code:java}
>    [junit4]   2> NOTE: reproduce with: ant test  
> -Dtestcase=TestReplicationHandler 
> -Dtests.method=doTestIndexFetchOnMasterRestart -Dtests.seed=884DCF71D210D14A 
> -Dtests.slow=true -Dtests.badapples=true -Dtests.locale=et 
> -Dtests.timezone=Cuba -Dtests.asserts=true -Dtests.file.encoding=UTF-8
>    [junit4] ERROR   2.35s | 
> TestReplicationHandler.doTestIndexFetchOnMasterRestart <<<
>    [junit4]    > Throwable #1: 
> org.apache.solr.client.solrj.SolrServerException: IOException occured when 
> talking to server at: http://127.0.0.1:37753/solr/collection1
>    [junit4]    >    at 
> __randomizedtesting.SeedInfo.seed([884DCF71D210D14A:50BA0B9579CB1316]:0)
>    [junit4]    >    at 
> org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:657)
>    [junit4]    >    at 
> org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
>    [junit4]    >    at 
> org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
>    [junit4]    >    at 
> org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
>    [junit4]    >    at 
> org.apache.solr.client.solrj.SolrClient.add(SolrClient.java:173)
>    [junit4]    >    at 
> org.apache.solr.client.solrj.SolrClient.add(SolrClient.java:138)
>    [junit4]    >    at 
> org.apache.solr.client.solrj.SolrClient.add(SolrClient.java:152)
>    [junit4]    >    at 
> org.apache.solr.handler.TestReplicationHandler.index(TestReplicationHandler.java:180)
>    [junit4]    >    at 
> org.apache.solr.handler.TestReplicationHandler.doTestIndexFetchOnMasterRestart(TestReplicationHandler.java:643)
>    [junit4]    >    at java.lang.Thread.run(Thread.java:748)
>    [junit4]    > Caused by: org.apache.http.NoHttpResponseException: 
> 127.0.0.1:37753 failed to respond
>    [junit4]    >    at 
> org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:141)
>    [junit4]    >    at 
> org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56)
>    [junit4]    >    at 
> org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:259)
>    [junit4]    >    at 
> org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:163)
>    [junit4]    >    at 
> org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165)
>    [junit4]    >    at 
> org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:273)
>    [junit4]    >    at 
> org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125)
>    [junit4]    >    at 
> org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272)
>    [junit4]    >    at 
> org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185)
>    [junit4]    >    at 
> org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)
>    [junit4]    >    at 
> 

[jira] [Commented] (SOLR-12078) Reproducable Failure in TestReplicationHandler.doTestIndexFetchOnMasterRestart

[Shalin Shekhar Mangar (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-12078?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16399373#comment-16399373
 ] 

Shalin Shekhar Mangar commented on SOLR-12078:
--

This was interesting. The masterClient has an internal connection pool. This 
pool has HTTP connections made to the master jetty during the 
clearIndexWithReplication() method. The test attempts to make a request after 
jetty is restarted. At this point the pool returns a stale connection which 
results in the NoHttpResponseException. The connection pool has a stale check 
set to 3 seconds by default. So the fix is either to sleep for 3 seconds or 
close and re-create the masterClient. I opted for the latter to fix the test.

Now this did not use to be the case when I wrote this test. In SOLR-4509, the 
httpclient behavior was changed from performing a stale check at lease time to 
a periodic stale check but since the test was marked as AwaitsFix, we never ran 
into this problem. The problem was exposed when Erick marked the test as 
BadApple instead of AwaitsFix.

> Reproducable Failure in TestReplicationHandler.doTestIndexFetchOnMasterRestart
> --
>
> Key: SOLR-12078
> URL: https://issues.apache.org/jira/browse/SOLR-12078
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Affects Versions: 7.3, master (8.0)
> Environment: Building on Ubuntu 17.4
> openjdk version "1.8.0_151"
> OpenJDK Runtime Environment (build 1.8.0_151-8u151-b12-0ubuntu0.17.04.2-b12)
> OpenJDK 64-Bit Server VM (build 25.151-b12, mixed mode)
>Reporter: Gus Heck
>Priority: Major
> Attachments: SOLR-12078.patch
>
>
> With the recent focus on bad tests lately, I decided to inspect some failures 
> that occurred in tests unrelated to my present task when I ran the tests 
> preparing for a pull request and found this failure which reproduces:
> ant test  -Dtestcase=TestReplicationHandler 
> -Dtests.method=doTestIndexFetchOnMasterRestart -Dtests.seed=884DCF71D210D14A 
> -Dtests.slow=true -Dtests.badapples=true -Dtests.locale=et 
> -Dtests.timezone=Cuba -Dtests.asserts=true -Dtests.file.encoding=UTF-8
>  
> Key excerpt of the log:
> {code:java}
>    [junit4]   2> NOTE: reproduce with: ant test  
> -Dtestcase=TestReplicationHandler 
> -Dtests.method=doTestIndexFetchOnMasterRestart -Dtests.seed=884DCF71D210D14A 
> -Dtests.slow=true -Dtests.badapples=true -Dtests.locale=et 
> -Dtests.timezone=Cuba -Dtests.asserts=true -Dtests.file.encoding=UTF-8
>    [junit4] ERROR   2.35s | 
> TestReplicationHandler.doTestIndexFetchOnMasterRestart <<<
>    [junit4]    > Throwable #1: 
> org.apache.solr.client.solrj.SolrServerException: IOException occured when 
> talking to server at: http://127.0.0.1:37753/solr/collection1
>    [junit4]    >    at 
> __randomizedtesting.SeedInfo.seed([884DCF71D210D14A:50BA0B9579CB1316]:0)
>    [junit4]    >    at 
> org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:657)
>    [junit4]    >    at 
> org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
>    [junit4]    >    at 
> org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
>    [junit4]    >    at 
> org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
>    [junit4]    >    at 
> org.apache.solr.client.solrj.SolrClient.add(SolrClient.java:173)
>    [junit4]    >    at 
> org.apache.solr.client.solrj.SolrClient.add(SolrClient.java:138)
>    [junit4]    >    at 
> org.apache.solr.client.solrj.SolrClient.add(SolrClient.java:152)
>    [junit4]    >    at 
> org.apache.solr.handler.TestReplicationHandler.index(TestReplicationHandler.java:180)
>    [junit4]    >    at 
> org.apache.solr.handler.TestReplicationHandler.doTestIndexFetchOnMasterRestart(TestReplicationHandler.java:643)
>    [junit4]    >    at java.lang.Thread.run(Thread.java:748)
>    [junit4]    > Caused by: org.apache.http.NoHttpResponseException: 
> 127.0.0.1:37753 failed to respond
>    [junit4]    >    at 
> org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:141)
>    [junit4]    >    at 
> org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56)
>    [junit4]    >    at 
> org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:259)
>    [junit4]    >    at 
> org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:163)
>    [junit4]    >    at 
> org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165)
>    [junit4]    >    at 
> 

[jira] [Updated] (SOLR-12078) Reproducable Failure in TestReplicationHandler.doTestIndexFetchOnMasterRestart

[Shalin Shekhar Mangar (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-12078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shalin Shekhar Mangar updated SOLR-12078:
-
Attachment: SOLR-12078.patch

> Reproducable Failure in TestReplicationHandler.doTestIndexFetchOnMasterRestart
> --
>
> Key: SOLR-12078
> URL: https://issues.apache.org/jira/browse/SOLR-12078
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Affects Versions: 7.3, master (8.0)
> Environment: Building on Ubuntu 17.4
> openjdk version "1.8.0_151"
> OpenJDK Runtime Environment (build 1.8.0_151-8u151-b12-0ubuntu0.17.04.2-b12)
> OpenJDK 64-Bit Server VM (build 25.151-b12, mixed mode)
>Reporter: Gus Heck
>Priority: Major
> Attachments: SOLR-12078.patch
>
>
> With the recent focus on bad tests lately, I decided to inspect some failures 
> that occurred in tests unrelated to my present task when I ran the tests 
> preparing for a pull request and found this failure which reproduces:
> ant test  -Dtestcase=TestReplicationHandler 
> -Dtests.method=doTestIndexFetchOnMasterRestart -Dtests.seed=884DCF71D210D14A 
> -Dtests.slow=true -Dtests.badapples=true -Dtests.locale=et 
> -Dtests.timezone=Cuba -Dtests.asserts=true -Dtests.file.encoding=UTF-8
>  
> Key excerpt of the log:
> {code:java}
>    [junit4]   2> NOTE: reproduce with: ant test  
> -Dtestcase=TestReplicationHandler 
> -Dtests.method=doTestIndexFetchOnMasterRestart -Dtests.seed=884DCF71D210D14A 
> -Dtests.slow=true -Dtests.badapples=true -Dtests.locale=et 
> -Dtests.timezone=Cuba -Dtests.asserts=true -Dtests.file.encoding=UTF-8
>    [junit4] ERROR   2.35s | 
> TestReplicationHandler.doTestIndexFetchOnMasterRestart <<<
>    [junit4]    > Throwable #1: 
> org.apache.solr.client.solrj.SolrServerException: IOException occured when 
> talking to server at: http://127.0.0.1:37753/solr/collection1
>    [junit4]    >    at 
> __randomizedtesting.SeedInfo.seed([884DCF71D210D14A:50BA0B9579CB1316]:0)
>    [junit4]    >    at 
> org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:657)
>    [junit4]    >    at 
> org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
>    [junit4]    >    at 
> org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
>    [junit4]    >    at 
> org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
>    [junit4]    >    at 
> org.apache.solr.client.solrj.SolrClient.add(SolrClient.java:173)
>    [junit4]    >    at 
> org.apache.solr.client.solrj.SolrClient.add(SolrClient.java:138)
>    [junit4]    >    at 
> org.apache.solr.client.solrj.SolrClient.add(SolrClient.java:152)
>    [junit4]    >    at 
> org.apache.solr.handler.TestReplicationHandler.index(TestReplicationHandler.java:180)
>    [junit4]    >    at 
> org.apache.solr.handler.TestReplicationHandler.doTestIndexFetchOnMasterRestart(TestReplicationHandler.java:643)
>    [junit4]    >    at java.lang.Thread.run(Thread.java:748)
>    [junit4]    > Caused by: org.apache.http.NoHttpResponseException: 
> 127.0.0.1:37753 failed to respond
>    [junit4]    >    at 
> org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:141)
>    [junit4]    >    at 
> org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56)
>    [junit4]    >    at 
> org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:259)
>    [junit4]    >    at 
> org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:163)
>    [junit4]    >    at 
> org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165)
>    [junit4]    >    at 
> org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:273)
>    [junit4]    >    at 
> org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125)
>    [junit4]    >    at 
> org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272)
>    [junit4]    >    at 
> org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185)
>    [junit4]    >    at 
> org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)
>    [junit4]    >    at 
> org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111)
>    [junit4]    >    at 
> org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
>    [junit4]    >    at 
> org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
>    [junit4]    >    at 
> 

[jira] [Commented] (SOLR-12088) Shards with dead replicas cause increased write latency

[Erick Erickson (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-12088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16399238#comment-16399238
 ] 

Erick Erickson commented on SOLR-12088:
---

Jerry:

It Depends (tm). Most are reasonably short, 15 seconds to a couple of minutes. 
So if you're seeing this last much longer than that it's a red herring.

Solr itself should be able to clean up dead replicas, what version are you 
using? By that I mean you can re-issue DELETEREPLICA and it "should" work.

There's a bit of a rough patch if you have legacyCloud set. Prior to 7x this 
was the default, and nodes could reconstruct themselves in ZK, the key is 
whether your ZooKeeper tree has partial collections representations in 
/clusterstate.json, likely corresponding to these deleted replicas. If that's 
the case, you can 


> stop the Solr instance

> manually remove the dead replicas

> start Solr back up.

once all that's done for the dead replicas, 

> replace /clusterstate.json with a single pair of empty brackets {} but ONLY 
> if your /collections/whatever/state.json has the complete, accurate picture 
> of the collection in question. This caveat is _very_ important because if you 
> _don't_ have a valid state.json (i.e. you're in "state format 2") then you'll 
> lose your collections, so be _very_ cautious.

Now, all that said, if performance is still slow after many minutes, it's a bug 
we should fix. The cluster maintenance stuff is steadily improving BTW.

Erick

> Shards with dead replicas cause increased write latency
> ---
>
> Key: SOLR-12088
> URL: https://issues.apache.org/jira/browse/SOLR-12088
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Affects Versions: 7.2
>Reporter: Jerry Bao
>Priority: Major
>
> If a collection's shard contains dead replicas, write latency to the 
> collection is increased. For example, if a collection has 10 shards with a 
> replication factor of 3, and one of those shards contains 3 replicas and 3 
> downed replicas, write latency is increased in comparison to a shard that 
> contains only 3 replicas.
> My feeling here is that downed replicas should be completely ignored and not 
> cause issues to other alive replicas in terms of write latency.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-12083) RealTimeGetComponent fails for INPLACE_UPDATE when Cdcr enabled

[Varun Thacker (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-12083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16399224#comment-16399224
 ] 

Varun Thacker commented on SOLR-12083:
--

Until INFRA-15850 is resolved the user tagged with the commit will not be me 

> RealTimeGetComponent fails for INPLACE_UPDATE when Cdcr enabled 
> 
>
> Key: SOLR-12083
> URL: https://issues.apache.org/jira/browse/SOLR-12083
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: CDCR
>Affects Versions: 7.2, 7.2.1, 7.3
>Reporter: Amrit Sarkar
>Assignee: Varun Thacker
>Priority: Major
> Attachments: SOLR-12083-A-within-test-framework.patch, 
> SOLR-12083-B-wo-test-framework.patch, SOLR-12083.patch, SOLR-12083.patch, 
> SOLR-12083.patch, SOLR-12083.patch, SOLR-12083.patch, 
> add_support_for_random_ulog_in_tests.patch
>
>
> When we were adding bi-directional sync support in CDCR ( SOLR-11003 ) we 
> changed the CDCR Update Log codec to write an extra bits. 
> When we use the RealTimeGet component on a cluster running CDCR and have 
> in-place updates in the update log we will falsely trip an assert thus 
> causing the request to fail
> Here's the proposed change
> {code:java}
> - assert entry.size() == 5;
> + if (ulog instanceof CdcrUpdateLog) {
> +   assert entry.size() == 6;
> + }
> + else {
> +   assert entry.size() == 5;
> + }{code}
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-12083) RealTimeGetComponent fails for INPLACE_UPDATE when Cdcr enabled

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-12083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16399218#comment-16399218
 ] 

ASF subversion and git services commented on SOLR-12083:


Commit f8bbfcdc75af2fe9cfbd6e507fba81d720406402 in lucene-solr's branch 
refs/heads/branch_7x from [~varun_saxena]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=f8bbfcd ]

SOLR-12083: Fix RealTime GET to work on a cluster running CDCR when using 
Solr's in-place updates

(cherry picked from commit 57524f1)


> RealTimeGetComponent fails for INPLACE_UPDATE when Cdcr enabled 
> 
>
> Key: SOLR-12083
> URL: https://issues.apache.org/jira/browse/SOLR-12083
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: CDCR
>Affects Versions: 7.2, 7.2.1, 7.3
>Reporter: Amrit Sarkar
>Assignee: Varun Thacker
>Priority: Major
> Attachments: SOLR-12083-A-within-test-framework.patch, 
> SOLR-12083-B-wo-test-framework.patch, SOLR-12083.patch, SOLR-12083.patch, 
> SOLR-12083.patch, SOLR-12083.patch, SOLR-12083.patch, 
> add_support_for_random_ulog_in_tests.patch
>
>
> When we were adding bi-directional sync support in CDCR ( SOLR-11003 ) we 
> changed the CDCR Update Log codec to write an extra bits. 
> When we use the RealTimeGet component on a cluster running CDCR and have 
> in-place updates in the update log we will falsely trip an assert thus 
> causing the request to fail
> Here's the proposed change
> {code:java}
> - assert entry.size() == 5;
> + if (ulog instanceof CdcrUpdateLog) {
> +   assert entry.size() == 6;
> + }
> + else {
> +   assert entry.size() == 5;
> + }{code}
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-12083) RealTimeGetComponent fails for INPLACE_UPDATE when Cdcr enabled

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-12083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16399216#comment-16399216
 ] 

ASF subversion and git services commented on SOLR-12083:


Commit 57524f1d4179f3ab57ffa53ba8f5e4dd1e198a11 in lucene-solr's branch 
refs/heads/master from [~varun_saxena]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=57524f1 ]

SOLR-12083: Fix RealTime GET to work on a cluster running CDCR when using 
Solr's in-place updates


> RealTimeGetComponent fails for INPLACE_UPDATE when Cdcr enabled 
> 
>
> Key: SOLR-12083
> URL: https://issues.apache.org/jira/browse/SOLR-12083
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: CDCR
>Affects Versions: 7.2, 7.2.1, 7.3
>Reporter: Amrit Sarkar
>Assignee: Varun Thacker
>Priority: Major
> Attachments: SOLR-12083-A-within-test-framework.patch, 
> SOLR-12083-B-wo-test-framework.patch, SOLR-12083.patch, SOLR-12083.patch, 
> SOLR-12083.patch, SOLR-12083.patch, SOLR-12083.patch, 
> add_support_for_random_ulog_in_tests.patch
>
>
> When we were adding bi-directional sync support in CDCR ( SOLR-11003 ) we 
> changed the CDCR Update Log codec to write an extra bits. 
> When we use the RealTimeGet component on a cluster running CDCR and have 
> in-place updates in the update log we will falsely trip an assert thus 
> causing the request to fail
> Here's the proposed change
> {code:java}
> - assert entry.size() == 5;
> + if (ulog instanceof CdcrUpdateLog) {
> +   assert entry.size() == 6;
> + }
> + else {
> +   assert entry.size() == 5;
> + }{code}
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-12083) RealTimeGetComponent fails for INPLACE_UPDATE when Cdcr enabled

[Varun Thacker (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-12083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16399205#comment-16399205
 ] 

Varun Thacker commented on SOLR-12083:
--

precommit, all tests, beasted 10 rounds of TestInPlaceUpdatesDistrib and 
TestRealTimeGet all passed

Committing shortly

> RealTimeGetComponent fails for INPLACE_UPDATE when Cdcr enabled 
> 
>
> Key: SOLR-12083
> URL: https://issues.apache.org/jira/browse/SOLR-12083
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: CDCR
>Affects Versions: 7.2, 7.2.1, 7.3
>Reporter: Amrit Sarkar
>Assignee: Varun Thacker
>Priority: Major
> Attachments: SOLR-12083-A-within-test-framework.patch, 
> SOLR-12083-B-wo-test-framework.patch, SOLR-12083.patch, SOLR-12083.patch, 
> SOLR-12083.patch, SOLR-12083.patch, SOLR-12083.patch, 
> add_support_for_random_ulog_in_tests.patch
>
>
> When we were adding bi-directional sync support in CDCR ( SOLR-11003 ) we 
> changed the CDCR Update Log codec to write an extra bits. 
> When we use the RealTimeGet component on a cluster running CDCR and have 
> in-place updates in the update log we will falsely trip an assert thus 
> causing the request to fail
> Here's the proposed change
> {code:java}
> - assert entry.size() == 5;
> + if (ulog instanceof CdcrUpdateLog) {
> +   assert entry.size() == 6;
> + }
> + else {
> +   assert entry.size() == 5;
> + }{code}
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-12088) Shards with dead replicas cause increased write latency

[Jerry Bao (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-12088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16399184#comment-16399184
 ] 

Jerry Bao commented on SOLR-12088:
--

[~erickerickson] I don't have an answer to your question; this issue occurred 
from movement of replicas where the movement did not completely clean up the 
state of the replicas, causing it to be a zombie replicas (data gone but state 
still exists after movement).

Your thinking definitely could explain why theres a higher latency of indexing 
times. That makes the most sense to me. How long is this timeout?

> Shards with dead replicas cause increased write latency
> ---
>
> Key: SOLR-12088
> URL: https://issues.apache.org/jira/browse/SOLR-12088
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Affects Versions: 7.2
>Reporter: Jerry Bao
>Priority: Major
>
> If a collection's shard contains dead replicas, write latency to the 
> collection is increased. For example, if a collection has 10 shards with a 
> replication factor of 3, and one of those shards contains 3 replicas and 3 
> downed replicas, write latency is increased in comparison to a shard that 
> contains only 3 replicas.
> My feeling here is that downed replicas should be completely ignored and not 
> cause issues to other alive replicas in terms of write latency.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-Tests-7.x - Build # 502 - Unstable

[Apache Jenkins Server]

Build: https://builds.apache.org/job/Lucene-Solr-Tests-7.x/502/

1 tests failed.
FAILED:  org.apache.solr.cloud.MoveReplicaHDFSTest.testFailedMove

Error Message:
No live SolrServers available to handle this 
request:[http://127.0.0.1:48662/solr/MoveReplicaHDFSTest_failed_coll_true, 
http://127.0.0.1:45021/solr/MoveReplicaHDFSTest_failed_coll_true]

Stack Trace:
org.apache.solr.client.solrj.SolrServerException: No live SolrServers available 
to handle this 
request:[http://127.0.0.1:48662/solr/MoveReplicaHDFSTest_failed_coll_true, 
http://127.0.0.1:45021/solr/MoveReplicaHDFSTest_failed_coll_true]
at 
__randomizedtesting.SeedInfo.seed([5D01BD12E1A4D902:F7CC6EE056770CD2]:0)
at 
org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:462)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1105)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:885)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:992)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:818)
at 
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:942)
at 
org.apache.solr.cloud.MoveReplicaTest.testFailedMove(MoveReplicaTest.java:307)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 

[jira] [Commented] (SOLR-12088) Shards with dead replicas cause increased write latency

[Jerry Bao (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-12088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16399167#comment-16399167
 ] 

Jerry Bao commented on SOLR-12088:
--

Your scenario is what I experienced, so yes :)

1. 30 nodes in the cluster

2. There are no nodes part of the cluster that aren't hosting any replicas.

3. Indexing via Lucidwork's Fusion (which I assume is using a SolrJ based 
client)

4. Latency is measured through our own service's instrumentation of roundtrip 
time to index.

> Shards with dead replicas cause increased write latency
> ---
>
> Key: SOLR-12088
> URL: https://issues.apache.org/jira/browse/SOLR-12088
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Affects Versions: 7.2
>Reporter: Jerry Bao
>Priority: Major
>
> If a collection's shard contains dead replicas, write latency to the 
> collection is increased. For example, if a collection has 10 shards with a 
> replication factor of 3, and one of those shards contains 3 replicas and 3 
> downed replicas, write latency is increased in comparison to a shard that 
> contains only 3 replicas.
> My feeling here is that downed replicas should be completely ignored and not 
> cause issues to other alive replicas in terms of write latency.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-12088) Shards with dead replicas cause increased write latency

[Erick Erickson (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-12088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16399138#comment-16399138
 ] 

Erick Erickson commented on SOLR-12088:
---

Another question:

Does the latency persist longer than any system timeouts? Put another way:
If you start all the Solr instances fresh and some nodes are down, is there 
still latency? 

What I'm thinking of here is that it may take up until some timeout for each 
Solr instance to "see" that the node is down.

For instance, if I kill a Solr node with a -9, it has no chance to tell ZK (and 
ZK in turn inform the rest of the collection) that it's going away. The rest of 
the collection finds out about this by one of several methods, all involving 
some timeout (ZK occasionally pings, leader sends request to update etc.).

So if this is transient it may be functioning as expected, but if it lasts well 
past all the possible timeouts it's another story.


> Shards with dead replicas cause increased write latency
> ---
>
> Key: SOLR-12088
> URL: https://issues.apache.org/jira/browse/SOLR-12088
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Affects Versions: 7.2
>Reporter: Jerry Bao
>Priority: Major
>
> If a collection's shard contains dead replicas, write latency to the 
> collection is increased. For example, if a collection has 10 shards with a 
> replication factor of 3, and one of those shards contains 3 replicas and 3 
> downed replicas, write latency is increased in comparison to a shard that 
> contains only 3 replicas.
> My feeling here is that downed replicas should be completely ignored and not 
> cause issues to other alive replicas in terms of write latency.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-11960) Add collection level properties

[Peter Rusko (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16399125#comment-16399125
 ] 

Peter Rusko commented on SOLR-11960:


I've updated the patch. Removed the watcher creation/deletion when registering 
and unregistering the core and I'm also re-registering watchers in 
createClusterStateWatchersAndUpdate now.

> Add collection level properties
> ---
>
> Key: SOLR-11960
> URL: https://issues.apache.org/jira/browse/SOLR-11960
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Peter Rusko
>Assignee: Tomás Fernández Löbbe
>Priority: Blocker
> Fix For: 7.3
>
> Attachments: SOLR-11960.patch, SOLR-11960.patch, SOLR-11960.patch, 
> SOLR-11960.patch, SOLR-11960.patch, SOLR-11960_2.patch
>
>
> Solr has cluster properties, but no easy and extendable way of defining 
> properties that affect a single collection. Collection properties could be 
> stored in a single zookeeper node per collection, making it possible to 
> trigger zookeeper watchers for only those Solr nodes that have cores of that 
> collection.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-11960) Add collection level properties

[Peter Rusko (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-11960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Peter Rusko updated SOLR-11960:
---
Attachment: SOLR-11960_2.patch

> Add collection level properties
> ---
>
> Key: SOLR-11960
> URL: https://issues.apache.org/jira/browse/SOLR-11960
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Peter Rusko
>Assignee: Tomás Fernández Löbbe
>Priority: Blocker
> Fix For: 7.3
>
> Attachments: SOLR-11960.patch, SOLR-11960.patch, SOLR-11960.patch, 
> SOLR-11960.patch, SOLR-11960.patch, SOLR-11960_2.patch
>
>
> Solr has cluster properties, but no easy and extendable way of defining 
> properties that affect a single collection. Collection properties could be 
> stored in a single zookeeper node per collection, making it possible to 
> trigger zookeeper watchers for only those Solr nodes that have cores of that 
> collection.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-12083) RealTimeGetComponent fails for INPLACE_UPDATE when Cdcr enabled

[Varun Thacker (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-12083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Varun Thacker updated SOLR-12083:
-
Attachment: SOLR-12083.patch

> RealTimeGetComponent fails for INPLACE_UPDATE when Cdcr enabled 
> 
>
> Key: SOLR-12083
> URL: https://issues.apache.org/jira/browse/SOLR-12083
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: CDCR
>Affects Versions: 7.2, 7.2.1, 7.3
>Reporter: Amrit Sarkar
>Assignee: Varun Thacker
>Priority: Major
> Attachments: SOLR-12083-A-within-test-framework.patch, 
> SOLR-12083-B-wo-test-framework.patch, SOLR-12083.patch, SOLR-12083.patch, 
> SOLR-12083.patch, SOLR-12083.patch, SOLR-12083.patch, 
> add_support_for_random_ulog_in_tests.patch
>
>
> When we were adding bi-directional sync support in CDCR ( SOLR-11003 ) we 
> changed the CDCR Update Log codec to write an extra bits. 
> When we use the RealTimeGet component on a cluster running CDCR and have 
> in-place updates in the update log we will falsely trip an assert thus 
> causing the request to fail
> Here's the proposed change
> {code:java}
> - assert entry.size() == 5;
> + if (ulog instanceof CdcrUpdateLog) {
> +   assert entry.size() == 6;
> + }
> + else {
> +   assert entry.size() == 5;
> + }{code}
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-master-MacOSX (64bit/jdk-9) - Build # 4497 - Failure!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-MacOSX/4497/
Java: 64bit/jdk-9 -XX:+UseCompressedOops -XX:+UseParallelGC

All tests passed

Build Log:
[...truncated 1853 lines...]
   [junit4] JVM J0: stdout was not empty, see: 
/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/build/core/test/temp/junit4-J0-20180314_171215_27316613869731467521539.sysout
   [junit4] >>> JVM J0 emitted unexpected output (verbatim) 
   [junit4] codec: FastDecompressionCompressingStoredFields, pf: 
LuceneFixedGap, dvf: Asserting
   [junit4] <<< JVM J0: EOF 

[...truncated 15980 lines...]
   [junit4] JVM J0: stdout was not empty, see: 
/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/contrib/solr-analytics/test/temp/junit4-J0-20180314_183257_2318537364422441044304.sysout
   [junit4] >>> JVM J0 emitted unexpected output (verbatim) 
   [junit4] #
   [junit4] # A fatal error has been detected by the Java Runtime Environment:
   [junit4] #
   [junit4] #  SIGFPE (0x8) at pc=0x7fff8fdca143, pid=8722, tid=17667
   [junit4] #
   [junit4] # JRE version: Java(TM) SE Runtime Environment (9.0+181) (build 
9+181)
   [junit4] # Java VM: Java HotSpot(TM) 64-Bit Server VM (9+181, mixed mode, 
tiered, compressed oops, parallel gc, bsd-amd64)
   [junit4] # Problematic frame:
   [junit4] # C  [libsystem_kernel.dylib+0x11143]  __commpage_gettimeofday+0x43
   [junit4] #
   [junit4] # No core dump will be written. Core dumps have been disabled. To 
enable core dumping, try "ulimit -c unlimited" before starting Java again
   [junit4] #
   [junit4] # An error report file with more information is saved as:
   [junit4] # 
/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/contrib/solr-analytics/test/J0/hs_err_pid8722.log
   [junit4] #
   [junit4] # If you would like to submit a bug report, please visit:
   [junit4] #   http://bugreport.java.com/bugreport/crash.jsp
   [junit4] #
   [junit4] <<< JVM J0: EOF 

[...truncated 125 lines...]
   [junit4] ERROR: JVM J0 ended with an exception, command line: 
/Library/Java/JavaVirtualMachines/jdk-9.jdk/Contents/Home/bin/java 
-XX:+UseCompressedOops -XX:+UseParallelGC -XX:+HeapDumpOnOutOfMemoryError 
-XX:HeapDumpPath=/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/heapdumps 
-ea -esa --illegal-access=deny -Dtests.prefix=tests 
-Dtests.seed=F09BE95205071586 -Xmx512M -Dtests.iters= -Dtests.verbose=false 
-Dtests.infostream=false -Dtests.codec=random -Dtests.postingsformat=random 
-Dtests.docvaluesformat=random -Dtests.locale=random -Dtests.timezone=random 
-Dtests.directory=random -Dtests.linedocsfile=europarl.lines.txt.gz 
-Dtests.luceneMatchVersion=8.0.0 -Dtests.cleanthreads=perClass 
-Djava.util.logging.config.file=/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/tools/junit4/logging.properties
 -Dtests.nightly=false -Dtests.weekly=false -Dtests.monster=false 
-Dtests.slow=true -Dtests.asserts=true -Dtests.multiplier=1 -DtempDir=./temp 
-Djava.io.tmpdir=./temp 
-Djunit4.tempDir=/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/contrib/solr-analytics/test/temp
 -Dcommon.dir=/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene 
-Dclover.db.dir=/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/build/clover/db
 
-Djava.security.policy=/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/tools/junit4/solr-tests.policy
 -Dtests.LUCENE_VERSION=8.0.0 -Djetty.testMode=1 -Djetty.insecurerandom=1 
-Dsolr.directoryFactory=org.apache.solr.core.MockDirectoryFactory 
-Djava.awt.headless=true -Djdk.map.althashing.threshold=0 
-Dtests.src.home=/Users/jenkins/workspace/Lucene-Solr-master-MacOSX 
-Djava.security.egd=file:/dev/./urandom 
-Djunit4.childvm.cwd=/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/contrib/solr-analytics/test/J0
 -Djunit4.childvm.id=0 -Djunit4.childvm.count=2 -Dfile.encoding=ISO-8859-1 
-Dtests.disableHdfs=true 
-Djava.security.manager=org.apache.lucene.util.TestSecurityManager 
-Dtests.filterstacks=true -Dtests.leaveTemporary=false -Dtests.badapples=false 
-classpath 

[JENKINS] Lucene-Solr-Tests-master - Build # 2424 - Still Unstable

[Apache Jenkins Server]

Build: https://builds.apache.org/job/Lucene-Solr-Tests-master/2424/

1 tests failed.
FAILED:  
org.apache.solr.cloud.autoscaling.ComputePlanActionTest.testNodeWithMultipleReplicasLost

Error Message:
The operations computed by ComputePlanAction should not be null 
SolrClientNodeStateProvider.DEBUG{AFTER_ACTION=[compute_plan, null], 
BEFORE_ACTION=[compute_plan, null]}

Stack Trace:
java.lang.AssertionError: The operations computed by ComputePlanAction should 
not be null SolrClientNodeStateProvider.DEBUG{AFTER_ACTION=[compute_plan, 
null], BEFORE_ACTION=[compute_plan, null]}
at 
__randomizedtesting.SeedInfo.seed([DB3957B6117D9155:EBF9B634990F7009]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertNotNull(Assert.java:526)
at 
org.apache.solr.cloud.autoscaling.ComputePlanActionTest.testNodeWithMultipleReplicasLost(ComputePlanActionTest.java:291)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)

[jira] [Commented] (SOLR-12083) RealTimeGetComponent fails for INPLACE_UPDATE when Cdcr enabled

[Varun Thacker (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-12083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16399065#comment-16399065
 ] 

Varun Thacker commented on SOLR-12083:
--

Final patch which I think is ready! Thanks Amrit

Running tests and precommit now. I plan on committing it to master and branch7x 
within the next few hours. The current CHANGES entry for the change is under 
7.3 

If Jenkins is happy today I'll check with the RM if it's okay to backport it to 
the release branch. If it's too late then i'll move the CHANGES to 7.4 on 
master and branch7x

> RealTimeGetComponent fails for INPLACE_UPDATE when Cdcr enabled 
> 
>
> Key: SOLR-12083
> URL: https://issues.apache.org/jira/browse/SOLR-12083
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: CDCR
>Affects Versions: 7.2, 7.2.1, 7.3
>Reporter: Amrit Sarkar
>Assignee: Varun Thacker
>Priority: Major
> Attachments: SOLR-12083-A-within-test-framework.patch, 
> SOLR-12083-B-wo-test-framework.patch, SOLR-12083.patch, SOLR-12083.patch, 
> SOLR-12083.patch, SOLR-12083.patch, add_support_for_random_ulog_in_tests.patch
>
>
> When we were adding bi-directional sync support in CDCR ( SOLR-11003 ) we 
> changed the CDCR Update Log codec to write an extra bits. 
> When we use the RealTimeGet component on a cluster running CDCR and have 
> in-place updates in the update log we will falsely trip an assert thus 
> causing the request to fail
> Here's the proposed change
> {code:java}
> - assert entry.size() == 5;
> + if (ulog instanceof CdcrUpdateLog) {
> +   assert entry.size() == 6;
> + }
> + else {
> +   assert entry.size() == 5;
> + }{code}
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-12083) RealTimeGetComponent fails for INPLACE_UPDATE when Cdcr enabled

[Varun Thacker (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-12083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Varun Thacker updated SOLR-12083:
-
Attachment: SOLR-12083.patch

> RealTimeGetComponent fails for INPLACE_UPDATE when Cdcr enabled 
> 
>
> Key: SOLR-12083
> URL: https://issues.apache.org/jira/browse/SOLR-12083
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: CDCR
>Affects Versions: 7.2, 7.2.1, 7.3
>Reporter: Amrit Sarkar
>Assignee: Varun Thacker
>Priority: Major
> Attachments: SOLR-12083-A-within-test-framework.patch, 
> SOLR-12083-B-wo-test-framework.patch, SOLR-12083.patch, SOLR-12083.patch, 
> SOLR-12083.patch, SOLR-12083.patch, add_support_for_random_ulog_in_tests.patch
>
>
> When we were adding bi-directional sync support in CDCR ( SOLR-11003 ) we 
> changed the CDCR Update Log codec to write an extra bits. 
> When we use the RealTimeGet component on a cluster running CDCR and have 
> in-place updates in the update log we will falsely trip an assert thus 
> causing the request to fail
> Here's the proposed change
> {code:java}
> - assert entry.size() == 5;
> + if (ulog instanceof CdcrUpdateLog) {
> +   assert entry.size() == 6;
> + }
> + else {
> +   assert entry.size() == 5;
> + }{code}
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-8205) Simplify AbortingException handling and tragic event logic

[Simon Willnauer (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-8205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16399046#comment-16399046
 ] 

Simon Willnauer commented on LUCENE-8205:
-

no magic involved I guess. just a lot of changes unfortunately. I mean we just 
cut the branch so I think we have enough time to get stuff like this in. 
Cleaning up IW is overdue I guess.

> Simplify AbortingException handling and tragic event logic
> --
>
> Key: LUCENE-8205
> URL: https://issues.apache.org/jira/browse/LUCENE-8205
> Project: Lucene - Core
>  Issue Type: Improvement
>Affects Versions: 7.4, master (8.0)
>Reporter: Simon Willnauer
>Assignee: Simon Willnauer
>Priority: Major
> Fix For: 7.4, master (8.0)
>
> Attachments: LUCENE-8205.patch
>
>
> Today we try to signal via exception handling if an exception is aborting
>  and/or a tragic event. This causes today that we ignore certain exception if 
> we
> are in the process of handling a such which is generally bad practice. This
> change simplify the signaling of aborting exceptions and separates acting on
> tragic events and closing the writer because of a tragic event. This in-turn
> simplifies lock ordering since we never acquire a lock anymore inside the
> tragic event code.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (LUCENE-8206) TestIndexWriterWithThreads has wall-clock time dependency

[Dawid Weiss (JIRA)]

Dawid Weiss created LUCENE-8206:
---

 Summary: TestIndexWriterWithThreads has wall-clock time dependency
 Key: LUCENE-8206
 URL: https://issues.apache.org/jira/browse/LUCENE-8206
 Project: Lucene - Core
  Issue Type: Task
Reporter: Dawid Weiss
Assignee: Dawid Weiss


{code}
final long stopTime = System.currentTimeMillis() + 200;
{code}

This can cause failures on slower machines.
{code}
Build: https://builds.apache.org/job/Lucene-Solr-Tests-master/2422/

3 tests failed.
FAILED:  org.apache.lucene.index.TestIndexWriterWithThreads.testCloseWithThreads

Error Message:
thread failed before indexing a single document

Stack Trace:
java.lang.AssertionError: thread failed before indexing a single document
{code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-8205) Simplify AbortingException handling and tragic event logic

[Dawid Weiss (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-8205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16398982#comment-16398982
 ] 

Dawid Weiss commented on LUCENE-8205:
-

Eyeballing it with awe, but can't tell anything about correctness. Anything 
that simplifies the IW is good for me.

> Simplify AbortingException handling and tragic event logic
> --
>
> Key: LUCENE-8205
> URL: https://issues.apache.org/jira/browse/LUCENE-8205
> Project: Lucene - Core
>  Issue Type: Improvement
>Affects Versions: 7.4, master (8.0)
>Reporter: Simon Willnauer
>Assignee: Simon Willnauer
>Priority: Major
> Fix For: 7.4, master (8.0)
>
> Attachments: LUCENE-8205.patch
>
>
> Today we try to signal via exception handling if an exception is aborting
>  and/or a tragic event. This causes today that we ignore certain exception if 
> we
> are in the process of handling a such which is generally bad practice. This
> change simplify the signaling of aborting exceptions and separates acting on
> tragic events and closing the writer because of a tragic event. This in-turn
> simplifies lock ordering since we never acquire a lock anymore inside the
> tragic event code.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: [JENKINS] Lucene-Solr-7.x-Solaris (64bit/jdk1.8.0) - Build # 492 - Failure!

[Steve Rowe]

Thanks Alan,

I’ve seen this before but assumed that it was an internal Python error related 
to a broken connection used to download the Jenkins log.

I’ll take a closer look.

--
Steve
www.lucidworks.com

> On Mar 14, 2018, at 1:06 PM, Alan Woodward  wrote:
> 
> Seen a few of these now, it looks as though it’s caused by an error in the 
> [repro] script?
> 
> From the Jenkins log at 
> https://jenkins.thetaphi.de/job/Lucene-Solr-7.x-Solaris/492/console :
> 
> [repro] Jenkins log URL: 
> https://jenkins.thetaphi.de/job/Lucene-Solr-7.x-Solaris/492/consoleText
> 
> 
> [repro] Revision: db1360fac40fe472f881a37d6a0f0187491e308c
> 
> [repro] Ant options: "-Dargs=-XX:-UseCompressedOops -XX:+UseSerialGC"
> Traceback (most recent call last):
>   File "/usr/lib/python3.4/http/client.py", line 614, in _readinto_chunked
> chunk_left = self._read_next_chunk_size()
>   File "/usr/lib/python3.4/http/client.py", line 559, in _read_next_chunk_size
> return int(line, 16)
> ValueError: invalid literal for int() with base 16: b''
> 
> During handling of the above exception, another exception occurred:
> 
> Traceback (most recent call last):
>   File "dev-tools/scripts/reproduceJenkinsFailures.py", line 276, in 
> main()
>   File "dev-tools/scripts/reproduceJenkinsFailures.py", line 226, in main
> tests = fetchAndParseJenkinsLog(config.url)
>   File "dev-tools/scripts/reproduceJenkinsFailures.py", line 108, in 
> fetchAndParseJenkinsLog
> for rawLine in consoleText:
>   File "/usr/lib/python3.4/http/client.py", line 500, in read
> return super(HTTPResponse, self).read(amt)
>   File "/usr/lib/python3.4/http/client.py", line 529, in readinto
> return self._readinto_chunked(b)
>   File "/usr/lib/python3.4/http/client.py", line 618, in _readinto_chunked
> raise IncompleteRead(bytes(b[0:total_bytes]))
> http.client.IncompleteRead: IncompleteRead(0 bytes read)
> Build step 'Execute shell' marked build as failure
> 
> 
> 
>> On 14 Mar 2018, at 16:35, Policeman Jenkins Server  
>> wrote:
>> 
>> Error processing tokens: Error while parsing action 
>> 'Text/ZeroOrMore/FirstOf/Token/DelimitedToken/DelimitedToken_Action3' at 
>> input position (line 79, pos 4):
>> )"}
>>   ^
>> 
>> java.lang.OutOfMemoryError: Java heap space
>> 
>> -
>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
>> For additional commands, e-mail: dev-h...@lucene.apache.org
> 


-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-12094) JsonRecordReader ignores root fields after split

[Dawid Weiss (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-12094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dawid Weiss updated SOLR-12094:
---
Description: 
JsonRecordReader, when configured with other than top-level split, ignores all 
top-level JSON nodes after split ends, for example:

{code}
{
  "first": "John",
  "last": "Doe",
  "grade": 8,
  "exams": [
{
"subject": "Maths",
"test": "term1",
"marks": 90
},
{
"subject": "Biology",
"test": "term1",
"marks": 86
}
  ],
  "after": "456"
}
{code}

Node "after" won't be visible in SolrInputDocument constructed from 
/update/json/docs.

I don't have fix, only (breaking) patch for relevant test

  was:
JsonRecordReader, when configured with other than top-level split, ignores all 
top-level JSON nodes after split ends, for example:

 

{

{{  "first": "John",}}

{{  "last": "Doe",}}

{{  "grade": 8,}}

{{  "exams": [}}

{{    {}}

{{        "subject": "Maths",}}

{{        "test": "term1",}}

{{        "marks": 90}}

{{    },}}

{{    {}}

{{        "subject": "Biology",}}

{{        "test": "term1",}}

{{        "marks": 86}}

{{  ],}}

{{  "after": "456"}}{{}}}

 

Node "after" won't be visible in SolrInputDocument constructed from 
/update/json/docs.

 

I don't have fix, only (breaking) patch for relevant test


> JsonRecordReader ignores root fields after split
> 
>
> Key: SOLR-12094
> URL: https://issues.apache.org/jira/browse/SOLR-12094
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrJ
>Affects Versions: master (8.0)
>Reporter: Przemysław Szeremiota
>Priority: Major
> Attachments: json-record-reader-bug.patch
>
>
> JsonRecordReader, when configured with other than top-level split, ignores 
> all top-level JSON nodes after split ends, for example:
> {code}
> {
>   "first": "John",
>   "last": "Doe",
>   "grade": 8,
>   "exams": [
> {
> "subject": "Maths",
> "test": "term1",
> "marks": 90
> },
> {
> "subject": "Biology",
> "test": "term1",
> "marks": 86
> }
>   ],
>   "after": "456"
> }
> {code}
> Node "after" won't be visible in SolrInputDocument constructed from 
> /update/json/docs.
> I don't have fix, only (breaking) patch for relevant test



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-7.3-Windows (32bit/jdk1.8.0_144) - Build # 3 - Failure!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-7.3-Windows/3/
Java: 32bit/jdk1.8.0_144 -server -XX:+UseG1GC

No tests ran.

Build Log:
[...truncated 13 lines...]
FATAL: Could not delete file 
C:\Users\jenkins\workspace\Lucene-Solr-7.3-Windows\.caches\test-stats
java.io.IOException: Could not delete file 
C:\Users\jenkins\workspace\Lucene-Solr-7.3-Windows\.caches\test-stats
at org.eclipse.jgit.util.FileUtils.delete(FileUtils.java:197)
at org.eclipse.jgit.util.FileUtils.delete(FileUtils.java:166)
at org.eclipse.jgit.api.CleanCommand.cleanPath(CleanCommand.java:176)
at org.eclipse.jgit.api.CleanCommand.call(CleanCommand.java:133)
Also:   hudson.remoting.Channel$CallSiteStackTrace: Remote call to Windows VBOX
at 
hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1737)
at hudson.remoting.UserResponse.retrieve(UserRequest.java:313)
at hudson.remoting.Channel.call(Channel.java:952)
at 
hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:278)
at com.sun.proxy.$Proxy65.clean(Unknown Source)
at 
org.jenkinsci.plugins.gitclient.RemoteGitImpl.clean(RemoteGitImpl.java:450)
at 
hudson.plugins.git.extensions.impl.CleanBeforeCheckout.decorateFetchCommand(CleanBeforeCheckout.java:30)
at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:858)
at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1129)
at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1160)
at hudson.scm.SCM.checkout(SCM.java:504)
at 
hudson.model.AbstractProject.checkout(AbstractProject.java:1208)
at 
hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:574)
at 
jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
at 
hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499)
at hudson.model.Run.execute(Run.java:1727)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at 
hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:429)
Caused: org.eclipse.jgit.api.errors.JGitInternalException: Could not delete 
file C:\Users\jenkins\workspace\Lucene-Solr-7.3-Windows\.caches\test-stats
at org.eclipse.jgit.api.CleanCommand.call(CleanCommand.java:136)
at 
org.jenkinsci.plugins.gitclient.JGitAPIImpl.clean(JGitAPIImpl.java:1290)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at 
hudson.remoting.RemoteInvocationHandler$RPCRequest.perform(RemoteInvocationHandler.java:927)
at 
hudson.remoting.RemoteInvocationHandler$RPCRequest.call(RemoteInvocationHandler.java:901)
at 
hudson.remoting.RemoteInvocationHandler$RPCRequest.call(RemoteInvocationHandler.java:850)
at hudson.remoting.UserRequest.perform(UserRequest.java:210)
at hudson.remoting.UserRequest.perform(UserRequest.java:53)
at hudson.remoting.Request$2.run(Request.java:364)
at 
hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
at java.util.concurrent.FutureTask.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
Archiving artifacts
[WARNINGS] Skipping publisher since build result is FAILURE
Recording test results
ERROR: Step ‘Publish JUnit test result report’ failed: Test reports were found 
but none of them are new. Did leafNodes run? 
For example, 
C:\Users\jenkins\workspace\Lucene-Solr-7.3-Windows\lucene\build\analysis\common\test\TEST-org.apache.lucene.analysis.ar.TestArabicAnalyzer.xml
 is 8 hr 11 min old

Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-master-Windows (64bit/jdk-9.0.1) - Build # 7223 - Failure!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Windows/7223/
Java: 64bit/jdk-9.0.1 -XX:+UseCompressedOops -XX:+UseParallelGC

No tests ran.

Build Log:
[...truncated 11 lines...]
FATAL: Could not delete file 
C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-test-framework\classes\test
java.io.IOException: Could not delete file 
C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-test-framework\classes\test
at org.eclipse.jgit.util.FileUtils.delete(FileUtils.java:197)
at org.eclipse.jgit.util.FileUtils.delete(FileUtils.java:166)
at org.eclipse.jgit.util.FileUtils.delete(FileUtils.java:166)
at org.eclipse.jgit.util.FileUtils.delete(FileUtils.java:166)
at org.eclipse.jgit.api.CleanCommand.cleanPath(CleanCommand.java:176)
at org.eclipse.jgit.api.CleanCommand.call(CleanCommand.java:133)
Also:   hudson.remoting.Channel$CallSiteStackTrace: Remote call to Windows VBOX
at 
hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1737)
at hudson.remoting.UserResponse.retrieve(UserRequest.java:313)
at hudson.remoting.Channel.call(Channel.java:952)
at 
hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:278)
at com.sun.proxy.$Proxy65.clean(Unknown Source)
at 
org.jenkinsci.plugins.gitclient.RemoteGitImpl.clean(RemoteGitImpl.java:450)
at 
hudson.plugins.git.extensions.impl.CleanBeforeCheckout.decorateFetchCommand(CleanBeforeCheckout.java:30)
at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:858)
at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1129)
at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1160)
at hudson.scm.SCM.checkout(SCM.java:504)
at 
hudson.model.AbstractProject.checkout(AbstractProject.java:1208)
at 
hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:574)
at 
jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
at 
hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499)
at hudson.model.Run.execute(Run.java:1727)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at 
hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:429)
Caused: org.eclipse.jgit.api.errors.JGitInternalException: Could not delete 
file 
C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-test-framework\classes\test
at org.eclipse.jgit.api.CleanCommand.call(CleanCommand.java:136)
at 
org.jenkinsci.plugins.gitclient.JGitAPIImpl.clean(JGitAPIImpl.java:1290)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at 
hudson.remoting.RemoteInvocationHandler$RPCRequest.perform(RemoteInvocationHandler.java:927)
at 
hudson.remoting.RemoteInvocationHandler$RPCRequest.call(RemoteInvocationHandler.java:901)
at 
hudson.remoting.RemoteInvocationHandler$RPCRequest.call(RemoteInvocationHandler.java:850)
at hudson.remoting.UserRequest.perform(UserRequest.java:210)
at hudson.remoting.UserRequest.perform(UserRequest.java:53)
at hudson.remoting.Request$2.run(Request.java:364)
at 
hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
at java.util.concurrent.FutureTask.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
Archiving artifacts
[WARNINGS] Skipping publisher since build result is FAILURE
Recording test results
ERROR: Step ‘Publish JUnit test result report’ failed: No test report files 
were found. Configuration error?
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: [JENKINS] Lucene-Solr-7.x-Solaris (64bit/jdk1.8.0) - Build # 492 - Failure!

[Alan Woodward]

Seen a few of these now, it looks as though it’s caused by an error in the 
[repro] script?

From the Jenkins log at 
https://jenkins.thetaphi.de/job/Lucene-Solr-7.x-Solaris/492/console 
 :

[repro] Jenkins log URL: 
https://jenkins.thetaphi.de/job/Lucene-Solr-7.x-Solaris/492/consoleText 


[repro] Revision: db1360fac40fe472f881a37d6a0f0187491e308c

[repro] Ant options: "-Dargs=-XX:-UseCompressedOops -XX:+UseSerialGC"
Traceback (most recent call last):
  File "/usr/lib/python3.4/http/client.py", line 614, in _readinto_chunked
chunk_left = self._read_next_chunk_size()
  File "/usr/lib/python3.4/http/client.py", line 559, in _read_next_chunk_size
return int(line, 16)
ValueError: invalid literal for int() with base 16: b''

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "dev-tools/scripts/reproduceJenkinsFailures.py", line 276, in 
main()
  File "dev-tools/scripts/reproduceJenkinsFailures.py", line 226, in main
tests = fetchAndParseJenkinsLog(config.url)
  File "dev-tools/scripts/reproduceJenkinsFailures.py", line 108, in 
fetchAndParseJenkinsLog
for rawLine in consoleText:
  File "/usr/lib/python3.4/http/client.py", line 500, in read
return super(HTTPResponse, self).read(amt)
  File "/usr/lib/python3.4/http/client.py", line 529, in readinto
return self._readinto_chunked(b)
  File "/usr/lib/python3.4/http/client.py", line 618, in _readinto_chunked
raise IncompleteRead(bytes(b[0:total_bytes]))
http.client.IncompleteRead: IncompleteRead(0 bytes read)
Build step 'Execute shell' marked build as failure


> On 14 Mar 2018, at 16:35, Policeman Jenkins Server  
> wrote:
> 
> Error processing tokens: Error while parsing action 
> 'Text/ZeroOrMore/FirstOf/Token/DelimitedToken/DelimitedToken_Action3' at 
> input position (line 79, pos 4):
> )"}
>   ^
> 
> java.lang.OutOfMemoryError: Java heap space
> 
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (SOLR-12064) NullPointerException in JSON facet

[Yonik Seeley (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-12064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yonik Seeley resolved SOLR-12064.
-
   Resolution: Fixed
Fix Version/s: 7.3

> NullPointerException in JSON facet
> --
>
> Key: SOLR-12064
> URL: https://issues.apache.org/jira/browse/SOLR-12064
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Facet Module
>Affects Versions: 7.2
>Reporter: Karthik Ramachandran
>Assignee: Yonik Seeley
>Priority: Major
> Fix For: 7.3
>
> Attachments: SOLR-12064.patch, patch.diff, patch.diff
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> NullPointerException in JSON facet when using terms with limit -1 and more 
> than one facet
> {f1:{terms:{field:'cat_ldS', limit:-1, sort:'n1 desc', 
> facet:{n1:'avg(num_ldS)', n2:'sum(num_ldS)'}
> {noformat}
> java.lang.NullPointerException
> at org.apache.solr.search.facet.AvgSlotAcc.collect(SlotAcc.java:390)
> at 
> org.apache.solr.search.facet.FacetFieldProcessor$MultiAcc.collect(FacetFieldProcessor.java:510)
> at org.apache.solr.search.facet.SlotAcc.collect(SlotAcc.java:97)
> at 
> org.apache.solr.search.facet.FacetFieldProcessor.collectFirstPhase(FacetFieldProcessor.java:220)
> at 
> org.apache.solr.search.facet.UnInvertedField.collectDocsGeneric(UnInvertedField.java:431)
> at 
> org.apache.solr.search.facet.UnInvertedField.collectDocs(UnInvertedField.java:410)
> at 
> org.apache.solr.search.facet.FacetFieldProcessorByArrayUIF.collectDocs(FacetFieldProcessorByArrayUIF.java:64)
> at 
> org.apache.solr.search.facet.FacetFieldProcessorByArray.calcFacets(FacetFieldProcessorByArray.java:108)
> at 
> org.apache.solr.search.facet.FacetFieldProcessorByArray.process(FacetFieldProcessorByArray.java:58)
> at 
> org.apache.solr.search.facet.FacetProcessor.processSubs(FacetProcessor.java:460)
> at 
> org.apache.solr.search.facet.FacetProcessor.fillBucket(FacetProcessor.java:407)
> at 
> org.apache.solr.search.facet.FacetQueryProcessor.process(FacetQuery.java:64)
> at org.apache.solr.search.facet.FacetModule.process(FacetModule.java:154)
> at 
> org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:295)
> at 
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:180)
> at org.apache.solr.core.SolrCore.execute(SolrCore.java:2503)
> at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:710)
> at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:516)
> at 
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:382)
> at 
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:326)
> {noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-8205) Simplify AbortingException handling and tragic event logic

[Simon Willnauer (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-8205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16398857#comment-16398857
 ] 

Simon Willnauer commented on LUCENE-8205:
-

for reference here is the change mike is referring to 
https://github.com/s1monw/lucene-solr/pull/3

> Simplify AbortingException handling and tragic event logic
> --
>
> Key: LUCENE-8205
> URL: https://issues.apache.org/jira/browse/LUCENE-8205
> Project: Lucene - Core
>  Issue Type: Improvement
>Affects Versions: 7.4, master (8.0)
>Reporter: Simon Willnauer
>Assignee: Simon Willnauer
>Priority: Major
> Fix For: 7.4, master (8.0)
>
> Attachments: LUCENE-8205.patch
>
>
> Today we try to signal via exception handling if an exception is aborting
>  and/or a tragic event. This causes today that we ignore certain exception if 
> we
> are in the process of handling a such which is generally bad practice. This
> change simplify the signaling of aborting exceptions and separates acting on
> tragic events and closing the writer because of a tragic event. This in-turn
> simplifies lock ordering since we never acquire a lock anymore inside the
> tragic event code.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-12094) JsonRecordReader ignores root fields after split

[JIRA]

 [ 
https://issues.apache.org/jira/browse/SOLR-12094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Przemysław Szeremiota updated SOLR-12094:
-
Description: 
JsonRecordReader, when configured with other than top-level split, ignores all 
top-level JSON nodes after split ends, for example:

 

{

{{  "first": "John",}}

{{  "last": "Doe",}}

{{  "grade": 8,}}

{{  "exams": [}}

{{    {}}

{{        "subject": "Maths",}}

{{        "test": "term1",}}

{{        "marks": 90}}

{{    },}}

{{    {}}

{{        "subject": "Biology",}}

{{        "test": "term1",}}

{{        "marks": 86}}

{{  ],}}

{{  "after": "456"}}{{}}}

 

Node "after" won't be visible in SolrInputDocument constructed from 
/update/json/docs.

 

I don't have fix, only (breaking) patch for relevant test

  was:
JsonRecordReader, when configured with other than top-level split, ignores all 
top-level JSON nodes after split ends, for example:

 

{\{{}}

{{  "first": "John",}}

{{  "last": "Doe",}}

{{  "grade": 8,}}

{{  "exams": [}}

{{    {}}

{{        "subject": "Maths",}}

{{        "test": "term1",}}

{{        "marks": 90}}

{{    },}}

{{    {}}

{{        "subject": "Biology",}}

{{        "test": "term1",}}

{{        "marks": 86}}

{{  ],}}

{{  "after": "456"}}{{}}}

 

Node "after" won't be visible in SolrInputDocument constructed from 
/update/json/docs.

 

I don't have fix, only (breaking) patch for relevant test


> JsonRecordReader ignores root fields after split
> 
>
> Key: SOLR-12094
> URL: https://issues.apache.org/jira/browse/SOLR-12094
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrJ
>Affects Versions: master (8.0)
>Reporter: Przemysław Szeremiota
>Priority: Major
> Attachments: json-record-reader-bug.patch
>
>
> JsonRecordReader, when configured with other than top-level split, ignores 
> all top-level JSON nodes after split ends, for example:
>  
> {
> {{  "first": "John",}}
> {{  "last": "Doe",}}
> {{  "grade": 8,}}
> {{  "exams": [}}
> {{    {}}
> {{        "subject": "Maths",}}
> {{        "test": "term1",}}
> {{        "marks": 90}}
> {{    },}}
> {{    {}}
> {{        "subject": "Biology",}}
> {{        "test": "term1",}}
> {{        "marks": 86}}
> {{  ],}}
> {{  "after": "456"}}{{}}}
>  
> Node "after" won't be visible in SolrInputDocument constructed from 
> /update/json/docs.
>  
> I don't have fix, only (breaking) patch for relevant test



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-12094) JsonRecordReader ignores root fields after split

[JIRA]

 [ 
https://issues.apache.org/jira/browse/SOLR-12094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Przemysław Szeremiota updated SOLR-12094:
-
Description: 
JsonRecordReader, when configured with other than top-level split, ignores all 
top-level JSON nodes after split ends, for example:

 

{\{{}}

{{  "first": "John",}}

{{  "last": "Doe",}}

{{  "grade": 8,}}

{{  "exams": [}}

{{    {}}

{{        "subject": "Maths",}}

{{        "test": "term1",}}

{{        "marks": 90}}

{{    },}}

{{    {}}

{{        "subject": "Biology",}}

{{        "test": "term1",}}

{{        "marks": 86}}

{{  ],}}

{{  "after": "456"}}{{}}}

 

Node "after" won't be visible in SolrInputDocument constructed from 
/update/json/docs.

 

I don't have fix, only (breaking) patch for relevant test

  was:
JsonRecordReader, when configured with other than top-level split, ignores all 
top-level JSON nodes after split ends, for example:

 

{{{}}

{{  "first": "John",}}

{{  "last": "Doe",}}

{{  "grade": 8,}}

{{  "exams": [}}

{{    {}}

{{        "subject": "Maths",}}

{{        "test": "term1",}}

{{        "marks": 90}}

{{    },}}

{{    {}}

{{        "subject": "Biology",}}

{{        "test": "term1",}}

{{        "marks": 86}}

{{  ],}}

{{  "after": "456"}}{{}}}

 

Node "after" won't be visible in SolrInputDocument constructed from 
/update/json/docs.

 

I don't have fix, only (breaking) patch for relevant test


> JsonRecordReader ignores root fields after split
> 
>
> Key: SOLR-12094
> URL: https://issues.apache.org/jira/browse/SOLR-12094
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrJ
>Affects Versions: master (8.0)
>Reporter: Przemysław Szeremiota
>Priority: Major
> Attachments: json-record-reader-bug.patch
>
>
> JsonRecordReader, when configured with other than top-level split, ignores 
> all top-level JSON nodes after split ends, for example:
>  
> {\{{}}
> {{  "first": "John",}}
> {{  "last": "Doe",}}
> {{  "grade": 8,}}
> {{  "exams": [}}
> {{    {}}
> {{        "subject": "Maths",}}
> {{        "test": "term1",}}
> {{        "marks": 90}}
> {{    },}}
> {{    {}}
> {{        "subject": "Biology",}}
> {{        "test": "term1",}}
> {{        "marks": 86}}
> {{  ],}}
> {{  "after": "456"}}{{}}}
>  
> Node "after" won't be visible in SolrInputDocument constructed from 
> /update/json/docs.
>  
> I don't have fix, only (breaking) patch for relevant test



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-12094) JsonRecordReader ignores root fields after split

[JIRA]

 [ 
https://issues.apache.org/jira/browse/SOLR-12094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Przemysław Szeremiota updated SOLR-12094:
-
Description: 
JsonRecordReader, when configured with other than top-level split, ignores all 
top-level JSON nodes after split ends, for example:

 

{{{}}

{{  "first": "John",}}

{{  "last": "Doe",}}

{{  "grade": 8,}}

{{  "exams": [}}

{{    {}}

{{        "subject": "Maths",}}

{{        "test": "term1",}}

{{        "marks": 90}}

{{    },}}

{{    {}}

{{        "subject": "Biology",}}

{{        "test": "term1",}}

{{        "marks": 86}}

{{  ],}}

{{  "after": "456"}}{{}}}

 

Node "after" won't be visible in SolrInputDocument constructed from 
/update/json/docs.

 

I don't have fix, only (breaking) patch for relevant test

  was:
JsonRecordReader, when configured with other than top-level split, ignores all 
top-level JSON nodes after split ends, for example:

 

{\{{}}

{{  "first": "John",}}

{{  "last": "Doe",}}

{{  "grade": 8,}}

{{  "exams": [}}

{{    {}}

{{        "subject": "Maths",}}

{{        "test": "term1",}}

{{        "marks": 90}}

{{    },}}

{{    {}}

{{        "subject": "Biology",}}

{{        "test": "term1",}}

{{        "marks": 86}}

{{  ],}}

{{  "after": "456"}}{{}}}

 

Node "after" won't be visible in SolrInputDocument constructed from 
/update/json/docs.

 

I don't have fix, only (breaking) patch for relevant test


> JsonRecordReader ignores root fields after split
> 
>
> Key: SOLR-12094
> URL: https://issues.apache.org/jira/browse/SOLR-12094
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrJ
>Affects Versions: master (8.0)
>Reporter: Przemysław Szeremiota
>Priority: Major
> Attachments: json-record-reader-bug.patch
>
>
> JsonRecordReader, when configured with other than top-level split, ignores 
> all top-level JSON nodes after split ends, for example:
>  
> {{{}}
> {{  "first": "John",}}
> {{  "last": "Doe",}}
> {{  "grade": 8,}}
> {{  "exams": [}}
> {{    {}}
> {{        "subject": "Maths",}}
> {{        "test": "term1",}}
> {{        "marks": 90}}
> {{    },}}
> {{    {}}
> {{        "subject": "Biology",}}
> {{        "test": "term1",}}
> {{        "marks": 86}}
> {{  ],}}
> {{  "after": "456"}}{{}}}
>  
> Node "after" won't be visible in SolrInputDocument constructed from 
> /update/json/docs.
>  
> I don't have fix, only (breaking) patch for relevant test



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-7.x-Solaris (64bit/jdk1.8.0) - Build # 492 - Failure!

[Policeman Jenkins Server]

Error processing tokens: Error while parsing action 
'Text/ZeroOrMore/FirstOf/Token/DelimitedToken/DelimitedToken_Action3' at input 
position (line 79, pos 4):
)"}
   ^

java.lang.OutOfMemoryError: Java heap space

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-8205) Simplify AbortingException handling and tragic event logic

[Michael McCandless (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-8205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16398845#comment-16398845
 ] 

Michael McCandless commented on LUCENE-8205:


+1, I left a few small comments on the github PR; this is an important step 
forward to simplifying IW.

> Simplify AbortingException handling and tragic event logic
> --
>
> Key: LUCENE-8205
> URL: https://issues.apache.org/jira/browse/LUCENE-8205
> Project: Lucene - Core
>  Issue Type: Improvement
>Affects Versions: 7.4, master (8.0)
>Reporter: Simon Willnauer
>Assignee: Simon Willnauer
>Priority: Major
> Fix For: 7.4, master (8.0)
>
> Attachments: LUCENE-8205.patch
>
>
> Today we try to signal via exception handling if an exception is aborting
>  and/or a tragic event. This causes today that we ignore certain exception if 
> we
> are in the process of handling a such which is generally bad practice. This
> change simplify the signaling of aborting exceptions and separates acting on
> tragic events and closing the writer because of a tragic event. This in-turn
> simplifies lock ordering since we never acquire a lock anymore inside the
> tragic event code.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-12094) JsonRecordReader ignores root fields after split

[JIRA]

 [ 
https://issues.apache.org/jira/browse/SOLR-12094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Przemysław Szeremiota updated SOLR-12094:
-
Description: 
JsonRecordReader, when configured with other than top-level split, ignores all 
top-level JSON nodes after split ends, for example:

 

{\{{}}

{{  "first": "John",}}

{{  "last": "Doe",}}

{{  "grade": 8,}}

{{  "exams": [}}

{{    {}}

{{        "subject": "Maths",}}

{{        "test": "term1",}}

{{        "marks": 90}}

{{    },}}

{{    {}}

{{        "subject": "Biology",}}

{{        "test": "term1",}}

{{        "marks": 86}}

{{  ],}}

{{  "after": "456"}}{{}}}

 

Node "after" won't be visible in SolrInputDocument constructed from 
/update/json/docs.

 

I don't have fix, only (breaking) patch for relevant test

  was:
JsonRecordReader, when configured with other than top-level split, ignores all 
top-level JSON nodes after split ends, for example:

 

{{{}}

{{  "first": "John",}}

{{  "last": "Doe",}}

{{  "grade": 8,}}

{{  "exams": [}}

{{    {}}

{{        "subject": "Maths",}}

{{        "test": "term1",}}

{{        "marks": 90}}

{{    },}}

{{    {}}

{{        "subject": "Biology",}}

{{        "test": "term1",}}

{{        "marks": 86}}

{{    }}}

{{  ],}}

{{  "after": "456"}}{{}}}

 

Node "after" won't be visible in SolrInputDocument constructed from 
/update/json/docs.

 

I don't have fix, only (breaking) patch for relevant test


> JsonRecordReader ignores root fields after split
> 
>
> Key: SOLR-12094
> URL: https://issues.apache.org/jira/browse/SOLR-12094
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrJ
>Affects Versions: master (8.0)
>Reporter: Przemysław Szeremiota
>Priority: Major
> Attachments: json-record-reader-bug.patch
>
>
> JsonRecordReader, when configured with other than top-level split, ignores 
> all top-level JSON nodes after split ends, for example:
>  
> {\{{}}
> {{  "first": "John",}}
> {{  "last": "Doe",}}
> {{  "grade": 8,}}
> {{  "exams": [}}
> {{    {}}
> {{        "subject": "Maths",}}
> {{        "test": "term1",}}
> {{        "marks": 90}}
> {{    },}}
> {{    {}}
> {{        "subject": "Biology",}}
> {{        "test": "term1",}}
> {{        "marks": 86}}
> {{  ],}}
> {{  "after": "456"}}{{}}}
>  
> Node "after" won't be visible in SolrInputDocument constructed from 
> /update/json/docs.
>  
> I don't have fix, only (breaking) patch for relevant test



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-12094) JsonRecordReader ignores root fields after split

[JIRA]

Przemysław Szeremiota created SOLR-12094:


 Summary: JsonRecordReader ignores root fields after split
 Key: SOLR-12094
 URL: https://issues.apache.org/jira/browse/SOLR-12094
 Project: Solr
  Issue Type: Bug
  Security Level: Public (Default Security Level. Issues are Public)
  Components: SolrJ
Affects Versions: master (8.0)
Reporter: Przemysław Szeremiota
 Attachments: json-record-reader-bug.patch

JsonRecordReader, when configured with other than top-level split, ignores all 
top-level JSON nodes after split ends, for example:

 

{{{}}

{{  "first": "John",}}

{{  "last": "Doe",}}

{{  "grade": 8,}}

{{  "exams": [}}

{{    {}}

{{        "subject": "Maths",}}

{{        "test": "term1",}}

{{        "marks": 90}}

{{    },}}

{{    {}}

{{        "subject": "Biology",}}

{{        "test": "term1",}}

{{        "marks": 86}}

{{    }}}

{{  ],}}

{{  "after": "456"}}{{}}}

 

Node "after" won't be visible in SolrInputDocument constructed from 
/update/json/docs.

 

I don't have fix, only (breaking) patch for relevant test



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: [JENKINS] Lucene-Solr-master-Linux (32bit/jdk1.8.0_162) - Build # 21639 - Still Unstable!

[Adrien Grand]

Simon just pushed a fix.

Le mer. 14 mars 2018 à 16:52, Policeman Jenkins Server 
a écrit :

> Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/21639/
> Java: 32bit/jdk1.8.0_162 -server -XX:+UseParallelGC
>
> 3 tests failed.
> FAILED:
> org.apache.lucene.index.TestIndexWriter.testSoftUpdatesConcurrently
>
> Error Message:
>
>
> Stack Trace:
> java.lang.NullPointerException
> at
> __randomizedtesting.SeedInfo.seed([7A8E25613EC94BB9:2A08856182550F1F]:0)
> at
> org.apache.lucene.index.TestIndexWriter.testSoftUpdatesConcurrently(TestIndexWriter.java:3211)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
> at
> org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
> at
> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
> at
> org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
> at
> org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
> at
> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
> at
> com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
> at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
> at
> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
> at
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> at
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
> at
> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
> at
> org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
> at
> org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
> at java.lang.Thread.run(Thread.java:748)
>
>
> FAILED:
> org.apache.lucene.index.TestIndexWriter.testSoftUpdatesConcurrently
>
> Error Message:
>
>
> Stack Trace:
> java.lang.NullPointerException
> at
> __randomizedtesting.SeedInfo.seed([7A8E25613EC94BB9:2A08856182550F1F]:0)
> at
> org.apache.lucene.index.TestIndexWriter.testSoftUpdatesConcurrently(TestIndexWriter.java:3211)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> 

Re: [JENKINS] Lucene-Solr-master-Linux (32bit/jdk1.8.0_162) - Build # 21639 - Still Unstable!

[Simon Willnauer]

pushed a fix for this, sorry for the noise

On Wed, Mar 14, 2018 at 4:51 PM, Policeman Jenkins Server
 wrote:
> Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/21639/
> Java: 32bit/jdk1.8.0_162 -server -XX:+UseParallelGC
>
> 3 tests failed.
> FAILED:  org.apache.lucene.index.TestIndexWriter.testSoftUpdatesConcurrently
>
> Error Message:
>
>
> Stack Trace:
> java.lang.NullPointerException
> at 
> __randomizedtesting.SeedInfo.seed([7A8E25613EC94BB9:2A08856182550F1F]:0)
> at 
> org.apache.lucene.index.TestIndexWriter.testSoftUpdatesConcurrently(TestIndexWriter.java:3211)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at 
> com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
> at 
> com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
> at 
> com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
> at 
> com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
> at 
> org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
> at 
> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
> at 
> org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
> at 
> org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
> at 
> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
> at 
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at 
> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
> at 
> com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
> at 
> com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
> at 
> com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
> at 
> com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
> at 
> com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
> at 
> com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
> at 
> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
> at 
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at 
> org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
> at 
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> at 
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> at 
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at 
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at 
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at 
> org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
> at 
> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
> at 
> org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
> at 
> org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
> at 
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at 
> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
> at java.lang.Thread.run(Thread.java:748)
>
>
> FAILED:  org.apache.lucene.index.TestIndexWriter.testSoftUpdatesConcurrently
>
> Error Message:
>
>
> Stack Trace:
> java.lang.NullPointerException
> at 
> __randomizedtesting.SeedInfo.seed([7A8E25613EC94BB9:2A08856182550F1F]:0)
> at 
> org.apache.lucene.index.TestIndexWriter.testSoftUpdatesConcurrently(TestIndexWriter.java:3211)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

[jira] [Commented] (LUCENE-8200) Allow doc-values to be updated atomically together with a document

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-8200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16398837#comment-16398837
 ] 

ASF subversion and git services commented on LUCENE-8200:
-

Commit 7ea251a717c185ae50287a56283724ae0a649182 in lucene-solr's branch 
refs/heads/branch_7x from [~simonw]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=7ea251a ]

LUCENE-8200: Fix NPE in TestIndexWriter


>  Allow doc-values to be updated atomically together with a document
> ---
>
> Key: LUCENE-8200
> URL: https://issues.apache.org/jira/browse/LUCENE-8200
> Project: Lucene - Core
>  Issue Type: Improvement
>Affects Versions: 7.4, master (8.0)
>Reporter: Simon Willnauer
>Assignee: Simon Willnauer
>Priority: Major
> Attachments: LUCENE-8200.patch, LUCENE-8200.patch, LUCENE-8200.patch
>
>
> Today we can only update a document by deleting all previously indexed 
> documents for the given term. In some cases like when deletes are not `final` 
> in the way that documents that are marked as deleted should not be merged 
> away a `soft-delete` is needed which is possible when doc-values updates can 
> be done atomically just like delete and add in updateDocument(s)
> 
> This change introduces such a soft update that reuses all code paths from 
> deletes to update all previously updated documents for a given term instead 
> of marking it as deleted. This is a spinnoff from LUCENE-8198



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-8200) Allow doc-values to be updated atomically together with a document

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-8200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16398835#comment-16398835
 ] 

ASF subversion and git services commented on LUCENE-8200:
-

Commit b7793372a937395d393af17e2461bab4cecd94e0 in lucene-solr's branch 
refs/heads/master from [~simonw]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=b779337 ]

LUCENE-8200: Fix NPE in TestIndexWriter


>  Allow doc-values to be updated atomically together with a document
> ---
>
> Key: LUCENE-8200
> URL: https://issues.apache.org/jira/browse/LUCENE-8200
> Project: Lucene - Core
>  Issue Type: Improvement
>Affects Versions: 7.4, master (8.0)
>Reporter: Simon Willnauer
>Assignee: Simon Willnauer
>Priority: Major
> Attachments: LUCENE-8200.patch, LUCENE-8200.patch, LUCENE-8200.patch
>
>
> Today we can only update a document by deleting all previously indexed 
> documents for the given term. In some cases like when deletes are not `final` 
> in the way that documents that are marked as deleted should not be merged 
> away a `soft-delete` is needed which is possible when doc-values updates can 
> be done atomically just like delete and add in updateDocument(s)
> 
> This change introduces such a soft update that reuses all code paths from 
> deletes to update all previously updated documents for a given term instead 
> of marking it as deleted. This is a spinnoff from LUCENE-8198



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: TestReplicationHandler is failing 100% (master and 7.x / 7.3)

[Alan Woodward]

Thanks Shalin!

> On 14 Mar 2018, at 15:50, Shalin Shekhar Mangar  > wrote:
> 
> I'll take a look at it tomorrow morning my time.
> 
> On Wed, Mar 14, 2018 at 9:07 PM, Andrzej Białecki 
> > 
> wrote:
> Well … I looked at it briefly but I have no idea what’s going on there. I 
> could dig into it nonetheless, but if there’s someone who already knows the 
> replication handler ins and outs it would probably get fixed sooner...
> 
> 
>> On 14 Mar 2018, at 14:23, Alan Woodward > > wrote:
>> 
>> I’m happy either way, but if it’s a bug can we get it fixed quickly?  Can 
>> you take ownership of this one Andrzej?
>> 
>>> On 14 Mar 2018, at 11:24, Andrzej Białecki >> > wrote:
>>> 
>>> Hi,
>>> 
>>> This test has always been fragile, but recently it’s been failing 100%, 
>>> most often in ‘doTestIndexFetchOnMasterRestart’.
>>> 
>>> I don’t know the replication handler enough to be able to find the real 
>>> reason behind these failures, but there are two possibilities that I see:
>>> 
>>> * the test has a bug and needs to be fixed - and if we can’t fix it soon 
>>> then with 7.3 release imminent we could BadApple it until it’s properly 
>>> fixed
>>> 
>>> * or actually the replication handler has a bug, which needs to be fixed - 
>>> in which case I propose to bump up SOLR-12078 to Blocker.
>>> 
>>> I’m open to suggestions.
>>> 
>>> —
>>> 
>>> Andrzej Białecki
>>> 
>> 
> 
> 
> 
> 
> -- 
> Regards,
> Shalin Shekhar Mangar.

[jira] [Updated] (SOLR-8220) Read field from docValues for non stored fields

[JIRA]

 [ 
https://issues.apache.org/jira/browse/SOLR-8220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jan Høydahl updated SOLR-8220:
--
Description: 
useDocValuesAsStored

Many times a value will be both stored="true" and docValues="true" which 
requires redundant data to be stored on disk. Since reading from docValues is 
both efficient and a common practice (facets, analytics, streaming, etc), 
reading values from docValues when a stored version of the field does not exist 
would be a valuable disk usage optimization.

The only caveat with this that I can see would be for multiValued fields as 
they would always be returned sorted in the docValues approach. I believe this 
is a fair compromise.

I've done a rough implementation for this as a field transform, but I think it 
should live closer to where stored fields are loaded in the SolrIndexSearcher.

Two open questions/observations:

1) There doesn't seem to be a standard way to read values for docValues, 
facets, analytics, streaming, etc, all seem to be doing their own ways, perhaps 
some of this logic should be centralized.

2) What will the API behavior be? (Below is my proposed implementation)
 Parameters for fl:
 - fl="docValueField"
 -- return field from docValue if the field is not stored and in docValues, if 
the field is stored return it from stored fields
 - fl="*"
 -- return only stored fields
 - fl="+"
 -- return stored fields and docValue fields

2a - would be easiest implementation and might be sufficient for a first pass. 
2b - is current behavior

  was:
Many times a value will be both stored="true" and docValues="true" which 
requires redundant data to be stored on disk. Since reading from docValues is 
both efficient and a common practice (facets, analytics, streaming, etc), 
reading values from docValues when a stored version of the field does not exist 
would be a valuable disk usage optimization.

The only caveat with this that I can see would be for multiValued fields as 
they would always be returned sorted in the docValues approach. I believe this 
is a fair compromise.

I've done a rough implementation for this as a field transform, but I think it 
should live closer to where stored fields are loaded in the SolrIndexSearcher.

Two open questions/observations:

1) There doesn't seem to be a standard way to read values for docValues, 
facets, analytics, streaming, etc, all seem to be doing their own ways, perhaps 
some of this logic should be centralized.

2) What will the API behavior be? (Below is my proposed implementation)
Parameters for fl:
- fl="docValueField"
  -- return field from docValue if the field is not stored and in docValues, if 
the field is stored return it from stored fields
- fl="*"
  -- return only stored fields
- fl="+"
   -- return stored fields and docValue fields

2a - would be easiest implementation and might be sufficient for a first pass. 
2b - is current behavior


> Read field from docValues for non stored fields
> ---
>
> Key: SOLR-8220
> URL: https://issues.apache.org/jira/browse/SOLR-8220
> Project: Solr
>  Issue Type: Improvement
>Reporter: Keith Laban
>Assignee: Shalin Shekhar Mangar
>Priority: Major
> Fix For: 5.5, 6.0
>
> Attachments: SOLR-8220-5x.patch, SOLR-8220-branch_5x.patch, 
> SOLR-8220-ishan.patch, SOLR-8220-ishan.patch, SOLR-8220-ishan.patch, 
> SOLR-8220-ishan.patch, SOLR-8220.patch, SOLR-8220.patch, SOLR-8220.patch, 
> SOLR-8220.patch, SOLR-8220.patch, SOLR-8220.patch, SOLR-8220.patch, 
> SOLR-8220.patch, SOLR-8220.patch, SOLR-8220.patch, SOLR-8220.patch, 
> SOLR-8220.patch, SOLR-8220.patch, SOLR-8220.patch, SOLR-8220.patch, 
> SOLR-8220.patch, SOLR-8220.patch, SOLR-8220.patch, SOLR-8220.patch, 
> SOLR-8220.patch, SOLR-8220.patch, SOLR-8220.patch, SOLR-8220.patch, 
> SOLR-8220.patch, SOLR-8220.patch, SOLR-8220.patch, SOLR-8220.patch, 
> SOLR-8220.patch
>
>
> useDocValuesAsStored
> Many times a value will be both stored="true" and docValues="true" which 
> requires redundant data to be stored on disk. Since reading from docValues is 
> both efficient and a common practice (facets, analytics, streaming, etc), 
> reading values from docValues when a stored version of the field does not 
> exist would be a valuable disk usage optimization.
> The only caveat with this that I can see would be for multiValued fields as 
> they would always be returned sorted in the docValues approach. I believe 
> this is a fair compromise.
> I've done a rough implementation for this as a field transform, but I think 
> it should live closer to where stored fields are loaded in the 
> SolrIndexSearcher.
> Two open questions/observations:
> 1) There doesn't seem to be a standard way to read values for docValues, 
> facets, analytics, streaming, etc, all seem to be doing their own ways, 
> perhaps 

[jira] [Updated] (LUCENE-8205) Simplify AbortingException handling and tragic event logic

[Simon Willnauer (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-8205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Simon Willnauer updated LUCENE-8205:

Attachment: LUCENE-8205.patch

> Simplify AbortingException handling and tragic event logic
> --
>
> Key: LUCENE-8205
> URL: https://issues.apache.org/jira/browse/LUCENE-8205
> Project: Lucene - Core
>  Issue Type: Improvement
>Affects Versions: 7.4, master (8.0)
>Reporter: Simon Willnauer
>Assignee: Simon Willnauer
>Priority: Major
> Fix For: 7.4, master (8.0)
>
> Attachments: LUCENE-8205.patch
>
>
> Today we try to signal via exception handling if an exception is aborting
>  and/or a tragic event. This causes today that we ignore certain exception if 
> we
> are in the process of handling a such which is generally bad practice. This
> change simplify the signaling of aborting exceptions and separates acting on
> tragic events and closing the writer because of a tragic event. This in-turn
> simplifies lock ordering since we never acquire a lock anymore inside the
> tragic event code.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-8205) Simplify AbortingException handling and tragic event logic

[Simon Willnauer (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-8205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16398803#comment-16398803
 ] 

Simon Willnauer commented on LUCENE-8205:
-

here is a patch.

> Simplify AbortingException handling and tragic event logic
> --
>
> Key: LUCENE-8205
> URL: https://issues.apache.org/jira/browse/LUCENE-8205
> Project: Lucene - Core
>  Issue Type: Improvement
>Affects Versions: 7.4, master (8.0)
>Reporter: Simon Willnauer
>Assignee: Simon Willnauer
>Priority: Major
> Fix For: 7.4, master (8.0)
>
> Attachments: LUCENE-8205.patch
>
>
> Today we try to signal via exception handling if an exception is aborting
>  and/or a tragic event. This causes today that we ignore certain exception if 
> we
> are in the process of handling a such which is generally bad practice. This
> change simplify the signaling of aborting exceptions and separates acting on
> tragic events and closing the writer because of a tragic event. This in-turn
> simplifies lock ordering since we never acquire a lock anymore inside the
> tragic event code.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (LUCENE-8205) Simplify AbortingException handling and tragic event logic

[Simon Willnauer (JIRA)]

Simon Willnauer created LUCENE-8205:
---

 Summary: Simplify AbortingException handling and tragic event logic
 Key: LUCENE-8205
 URL: https://issues.apache.org/jira/browse/LUCENE-8205
 Project: Lucene - Core
  Issue Type: Improvement
Affects Versions: 7.4, master (8.0)
Reporter: Simon Willnauer
Assignee: Simon Willnauer
 Fix For: 7.4, master (8.0)


Today we try to signal via exception handling if an exception is aborting
 and/or a tragic event. This causes today that we ignore certain exception if we
are in the process of handling a such which is generally bad practice. This
change simplify the signaling of aborting exceptions and separates acting on
tragic events and closing the writer because of a tragic event. This in-turn
simplifies lock ordering since we never acquire a lock anymore inside the
tragic event code.




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-master-Linux (32bit/jdk1.8.0_162) - Build # 21639 - Still Unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/21639/
Java: 32bit/jdk1.8.0_162 -server -XX:+UseParallelGC

3 tests failed.
FAILED:  org.apache.lucene.index.TestIndexWriter.testSoftUpdatesConcurrently

Error Message:


Stack Trace:
java.lang.NullPointerException
at 
__randomizedtesting.SeedInfo.seed([7A8E25613EC94BB9:2A08856182550F1F]:0)
at 
org.apache.lucene.index.TestIndexWriter.testSoftUpdatesConcurrently(TestIndexWriter.java:3211)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.lucene.index.TestIndexWriter.testSoftUpdatesConcurrently

Error Message:


Stack Trace:
java.lang.NullPointerException
at 
__randomizedtesting.SeedInfo.seed([7A8E25613EC94BB9:2A08856182550F1F]:0)
at 
org.apache.lucene.index.TestIndexWriter.testSoftUpdatesConcurrently(TestIndexWriter.java:3211)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at 

Re: TestReplicationHandler is failing 100% (master and 7.x / 7.3)

[Shalin Shekhar Mangar]

I'll take a look at it tomorrow morning my time.

On Wed, Mar 14, 2018 at 9:07 PM, Andrzej Białecki <
andrzej.biale...@lucidworks.com> wrote:

> Well … I looked at it briefly but I have no idea what’s going on there. I
> could dig into it nonetheless, but if there’s someone who already knows the
> replication handler ins and outs it would probably get fixed sooner...
>
>
> On 14 Mar 2018, at 14:23, Alan Woodward  wrote:
>
> I’m happy either way, but if it’s a bug can we get it fixed quickly?  Can
> you take ownership of this one Andrzej?
>
> On 14 Mar 2018, at 11:24, Andrzej Białecki  wrote:
>
> Hi,
>
> This test has always been fragile, but recently it’s been failing 100%,
> most often in ‘doTestIndexFetchOnMasterRestart’.
>
> I don’t know the replication handler enough to be able to find the real
> reason behind these failures, but there are two possibilities that I see:
>
> * the test has a bug and needs to be fixed - and if we can’t fix it soon
> then with 7.3 release imminent we could BadApple it until it’s properly
> fixed
>
> * or actually the replication handler has a bug, which needs to be fixed -
> in which case I propose to bump up SOLR-12078 to Blocker.
>
> I’m open to suggestions.
>
> —
>
> Andrzej Białecki
>
>
>
>


-- 
Regards,
Shalin Shekhar Mangar.

Re: TestReplicationHandler is failing 100% (master and 7.x / 7.3)

[Andrzej Białecki]

Well … I looked at it briefly but I have no idea what’s going on there. I could 
dig into it nonetheless, but if there’s someone who already knows the 
replication handler ins and outs it would probably get fixed sooner...

> On 14 Mar 2018, at 14:23, Alan Woodward  wrote:
> 
> I’m happy either way, but if it’s a bug can we get it fixed quickly?  Can you 
> take ownership of this one Andrzej?
> 
>> On 14 Mar 2018, at 11:24, Andrzej Białecki > > wrote:
>> 
>> Hi,
>> 
>> This test has always been fragile, but recently it’s been failing 100%, most 
>> often in ‘doTestIndexFetchOnMasterRestart’.
>> 
>> I don’t know the replication handler enough to be able to find the real 
>> reason behind these failures, but there are two possibilities that I see:
>> 
>> * the test has a bug and needs to be fixed - and if we can’t fix it soon 
>> then with 7.3 release imminent we could BadApple it until it’s properly fixed
>> 
>> * or actually the replication handler has a bug, which needs to be fixed - 
>> in which case I propose to bump up SOLR-12078 to Blocker.
>> 
>> I’m open to suggestions.
>> 
>> —
>> 
>> Andrzej Białecki
>> 
> 

[JENKINS] Lucene-Solr-Tests-master - Build # 2423 - Still Unstable

[Apache Jenkins Server]

Build: https://builds.apache.org/job/Lucene-Solr-Tests-master/2423/

1 tests failed.
FAILED:  
org.apache.solr.cloud.api.collections.CollectionsAPIAsyncDistributedZkTest.testAsyncIdBackCompat

Error Message:
KeeperErrorCode = Session expired for /overseer

Stack Trace:
org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = 
Session expired for /overseer
at 
__randomizedtesting.SeedInfo.seed([995CC4B4A4EAA767:476C571808064337]:0)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:130)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:54)
at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1105)
at 
org.apache.solr.common.cloud.SolrZkClient.lambda$exists$2(SolrZkClient.java:304)
at 
org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
at 
org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:304)
at 
org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:483)
at 
org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:386)
at 
org.apache.solr.cloud.api.collections.CollectionsAPIAsyncDistributedZkTest.testAsyncIdBackCompat(CollectionsAPIAsyncDistributedZkTest.java:252)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
 

[jira] [Commented] (SOLR-9935) When hl.method=unified add support for hl.fragsize param

[Mohsen (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16398647#comment-16398647
 ] 

Mohsen commented on SOLR-9935:
--

Is this really resolved as of Solr 6.4? I tested with both Solr 6.4.1 and Solr 
7.1 installations and none of them recognize hl.fragsize when unified method is 
used.

> When hl.method=unified add support for hl.fragsize param
> 
>
> Key: SOLR-9935
> URL: https://issues.apache.org/jira/browse/SOLR-9935
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: highlighter
>Reporter: David Smiley
>Assignee: David Smiley
>Priority: Major
> Fix For: 6.4
>
> Attachments: SOLR_9935_UH_fragsize.patch, SOLR_9935_UH_fragsize.patch
>
>
> In LUCENE-7620 the UnifiedHighlighter is getting a BreakIterator that allows 
> it to support the equivalent of Solr's {{hl.fragsize}}.  So lets support this 
> on the Solr side.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-Tests-7.3 - Build # 1 - Unstable

[Apache Jenkins Server]

Build: https://builds.apache.org/job/Lucene-Solr-Tests-7.3/1/

1 tests failed.
FAILED:  
org.apache.solr.handler.admin.SegmentsInfoRequestHandlerTest.testSegmentInfosVersion

Error Message:
Exception during query

Stack Trace:
java.lang.RuntimeException: Exception during query
at 
__randomizedtesting.SeedInfo.seed([48A65369438C676E:B078C6813BE0B63D]:0)
at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:908)
at 
org.apache.solr.handler.admin.SegmentsInfoRequestHandlerTest.testSegmentInfosVersion(SegmentsInfoRequestHandlerTest.java:68)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.RuntimeException: REQUEST FAILED: 
xpath=2=count(//lst[@name='segments']/lst/str[@name='version'][.='7.3.0'])
xml response was: 

00_01215242018-03-14T12:57:27.215Zflush7.3.0_10190412018-03-14T12:57:27.218Zflush7.3.0_20208542018-03-14T12:57:27.229Zflush7.3.0_30190412018-03-14T12:57:27.230Zflush7.3.0


request 

[jira] [Updated] (SOLR-12038) SPLITSHARD should check disk space before performing the operation

[Cassandra Targett (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-12038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Cassandra Targett updated SOLR-12038:
-
Component/s: SolrCloud

> SPLITSHARD should check disk space before performing the operation
> --
>
> Key: SOLR-12038
> URL: https://issues.apache.org/jira/browse/SOLR-12038
> Project: Solr
>  Issue Type: Sub-task
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Reporter: Noble Paul
>Priority: Major
>
> SPLITSHARD should first check if it has enough space to do the split before 
> it accepts the operation



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-11911) TestLargeCluster.testSearchRate() failure

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16398587#comment-16398587
 ] 

ASF subversion and git services commented on SOLR-11911:


Commit 1824297a1fa6199003053f4e0a76eeb3464796f7 in lucene-solr's branch 
refs/heads/branch_7x from [~ab]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=1824297 ]

SOLR-11911: Add more details to failure logs, modify the test to create a 
single event
that contains all affected nodes.


> TestLargeCluster.testSearchRate() failure
> -
>
> Key: SOLR-11911
> URL: https://issues.apache.org/jira/browse/SOLR-11911
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Steve Rowe
>Assignee: Andrzej Bialecki 
>Priority: Major
>
> My Jenkins found a branch_7x seed that reproduced 4/5 times for me:
> {noformat}
> Checking out Revision af9706cb89335a5aa04f9bcae0c2558a61803b50 
> (refs/remotes/origin/branch_7x)
> [...]
>[junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestLargeCluster 
> -Dtests.method=testSearchRate -Dtests.seed=2D7724685882A83D -Dtests.slow=true 
> -Dtests.locale=be-BY -Dtests.timezone=Africa/Ouagadougou -Dtests.asserts=true 
> -Dtests.file.encoding=UTF-8
>[junit4] FAILURE 1.24s J0  | TestLargeCluster.testSearchRate <<<
>[junit4]> Throwable #1: java.lang.AssertionError: The trigger did not 
> fire at all
>[junit4]>  at 
> __randomizedtesting.SeedInfo.seed([2D7724685882A83D:703F3AE197440E72]:0)
>[junit4]>  at 
> org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.testSearchRate(TestLargeCluster.java:547)
>[junit4]>  at java.lang.Thread.run(Thread.java:748)
> [...]
>[junit4]   2> NOTE: test params are: codec=CheapBastard, 
> sim=RandomSimilarity(queryNorm=true): {}, locale=be-BY, 
> timezone=Africa/Ouagadougou
>[junit4]   2> NOTE: Linux 4.1.0-custom2-amd64 amd64/Oracle Corporation 
> 1.8.0_151 (64-bit)/cpus=16,threads=1,free=388243840,total=502267904
> {noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-11911) TestLargeCluster.testSearchRate() failure

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16398588#comment-16398588
 ] 

ASF subversion and git services commented on SOLR-11911:


Commit 5a59b5d95acf9d86c740c65fa413a28889541fe2 in lucene-solr's branch 
refs/heads/branch_7x from [~ab]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=5a59b5d ]

SOLR-11911: Wait a while for left-behind threads from executors.


> TestLargeCluster.testSearchRate() failure
> -
>
> Key: SOLR-11911
> URL: https://issues.apache.org/jira/browse/SOLR-11911
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Steve Rowe
>Assignee: Andrzej Bialecki 
>Priority: Major
>
> My Jenkins found a branch_7x seed that reproduced 4/5 times for me:
> {noformat}
> Checking out Revision af9706cb89335a5aa04f9bcae0c2558a61803b50 
> (refs/remotes/origin/branch_7x)
> [...]
>[junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestLargeCluster 
> -Dtests.method=testSearchRate -Dtests.seed=2D7724685882A83D -Dtests.slow=true 
> -Dtests.locale=be-BY -Dtests.timezone=Africa/Ouagadougou -Dtests.asserts=true 
> -Dtests.file.encoding=UTF-8
>[junit4] FAILURE 1.24s J0  | TestLargeCluster.testSearchRate <<<
>[junit4]> Throwable #1: java.lang.AssertionError: The trigger did not 
> fire at all
>[junit4]>  at 
> __randomizedtesting.SeedInfo.seed([2D7724685882A83D:703F3AE197440E72]:0)
>[junit4]>  at 
> org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.testSearchRate(TestLargeCluster.java:547)
>[junit4]>  at java.lang.Thread.run(Thread.java:748)
> [...]
>[junit4]   2> NOTE: test params are: codec=CheapBastard, 
> sim=RandomSimilarity(queryNorm=true): {}, locale=be-BY, 
> timezone=Africa/Ouagadougou
>[junit4]   2> NOTE: Linux 4.1.0-custom2-amd64 amd64/Oracle Corporation 
> 1.8.0_151 (64-bit)/cpus=16,threads=1,free=388243840,total=502267904
> {noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-11911) TestLargeCluster.testSearchRate() failure

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16398585#comment-16398585
 ] 

ASF subversion and git services commented on SOLR-11911:


Commit 5e02c18769a50a1600ab715cb3348b3da4933bcf in lucene-solr's branch 
refs/heads/master from [~ab]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=5e02c18 ]

SOLR-11911: Wait a while for left-behind threads from executors.


> TestLargeCluster.testSearchRate() failure
> -
>
> Key: SOLR-11911
> URL: https://issues.apache.org/jira/browse/SOLR-11911
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Steve Rowe
>Assignee: Andrzej Bialecki 
>Priority: Major
>
> My Jenkins found a branch_7x seed that reproduced 4/5 times for me:
> {noformat}
> Checking out Revision af9706cb89335a5aa04f9bcae0c2558a61803b50 
> (refs/remotes/origin/branch_7x)
> [...]
>[junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestLargeCluster 
> -Dtests.method=testSearchRate -Dtests.seed=2D7724685882A83D -Dtests.slow=true 
> -Dtests.locale=be-BY -Dtests.timezone=Africa/Ouagadougou -Dtests.asserts=true 
> -Dtests.file.encoding=UTF-8
>[junit4] FAILURE 1.24s J0  | TestLargeCluster.testSearchRate <<<
>[junit4]> Throwable #1: java.lang.AssertionError: The trigger did not 
> fire at all
>[junit4]>  at 
> __randomizedtesting.SeedInfo.seed([2D7724685882A83D:703F3AE197440E72]:0)
>[junit4]>  at 
> org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.testSearchRate(TestLargeCluster.java:547)
>[junit4]>  at java.lang.Thread.run(Thread.java:748)
> [...]
>[junit4]   2> NOTE: test params are: codec=CheapBastard, 
> sim=RandomSimilarity(queryNorm=true): {}, locale=be-BY, 
> timezone=Africa/Ouagadougou
>[junit4]   2> NOTE: Linux 4.1.0-custom2-amd64 amd64/Oracle Corporation 
> 1.8.0_151 (64-bit)/cpus=16,threads=1,free=388243840,total=502267904
> {noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: TestReplicationHandler is failing 100% (master and 7.x / 7.3)

[Alan Woodward]

I’m happy either way, but if it’s a bug can we get it fixed quickly?  Can you 
take ownership of this one Andrzej?

> On 14 Mar 2018, at 11:24, Andrzej Białecki  > wrote:
> 
> Hi,
> 
> This test has always been fragile, but recently it’s been failing 100%, most 
> often in ‘doTestIndexFetchOnMasterRestart’.
> 
> I don’t know the replication handler enough to be able to find the real 
> reason behind these failures, but there are two possibilities that I see:
> 
> * the test has a bug and needs to be fixed - and if we can’t fix it soon then 
> with 7.3 release imminent we could BadApple it until it’s properly fixed
> 
> * or actually the replication handler has a bug, which needs to be fixed - in 
> which case I propose to bump up SOLR-12078 to Blocker.
> 
> I’m open to suggestions.
> 
> —
> 
> Andrzej Białecki
> 

[jira] [Updated] (SOLR-12089) FileBasedSpellChecker docs have some missing params

[Cassandra Targett (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-12089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Cassandra Targett updated SOLR-12089:
-
Component/s: documentation

> FileBasedSpellChecker docs have some missing params
> ---
>
> Key: SOLR-12089
> URL: https://issues.apache.org/jira/browse/SOLR-12089
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: documentation
>Reporter: Varun Thacker
>Priority: Minor
>
> FileBasedSpellChecker has params like "fieldType" that is not documented.
> I'm creating a Jira to audit the params on all the spellcheckers and make 
> sure they are documented in the ref guide 
> http://lucene.apache.org/solr/guide/spell-checking.html 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-12093) LeaderVoteWaitTimeoutTest failures

[Andrzej Bialecki (JIRA)]

Andrzej Bialecki  created SOLR-12093:


 Summary: LeaderVoteWaitTimeoutTest failures
 Key: SOLR-12093
 URL: https://issues.apache.org/jira/browse/SOLR-12093
 Project: Solr
  Issue Type: Bug
  Security Level: Public (Default Security Level. Issues are Public)
Affects Versions: 7.4, master (8.0)
Reporter: Andrzej Bialecki 
Assignee: Cao Manh Dat


Recent failures from jenkins 
(https://builds.apache.org/view/L/view/Lucene/job/Lucene-Solr-NightlyTests-7.x/174/):

{code}
   [junit4]   2> NOTE: reproduce with: ant test  
-Dtestcase=LeaderVoteWaitTimeoutTest -Dtests.method=basicTest 
-Dtests.seed=F36F789B06B8474E -Dtests.multiplier=2 -Dtests.nightly=true 
-Dtests.slow=true 
-Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/test-data/enwiki.random.lines.txt
 -Dtests.locale=lv -Dtests.timezone=Africa/Dakar -Dtests.asserts=true 
-Dtests.file.encoding=UTF-8
   [junit4] ERROR   1.44s J0 | LeaderVoteWaitTimeoutTest.basicTest <<<
   [junit4]> Throwable #1: java.lang.NullPointerException
   [junit4]>at 
__randomizedtesting.SeedInfo.seed([F36F789B06B8474E:19B6FF9421D4A7D]:0)
   [junit4]>at 
org.apache.solr.client.solrj.embedded.JettySolrRunner.getNodeName(JettySolrRunner.java:347)
   [junit4]>at 
org.apache.solr.cloud.LeaderVoteWaitTimeoutTest.basicTest(LeaderVoteWaitTimeoutTest.java:96)
   [junit4]>at java.lang.Thread.run(Thread.java:748)
{code}

{code}
   [junit4]   2> NOTE: reproduce with: ant test  
-Dtestcase=LeaderVoteWaitTimeoutTest 
-Dtests.method=testMostInSyncReplicasCanWinElection 
-Dtests.seed=F36F789B06B8474E -Dtests.multiplier=2 -Dtests.nightly=true 
-Dtests.slow=true 
-Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/test-data/enwiki.random.lines.txt
 -Dtests.locale=lv -Dtests.timezone=Africa/Dakar -Dtests.asserts=true 
-Dtests.file.encoding=UTF-8
   [junit4] FAILURE 12.6s J0 | 
LeaderVoteWaitTimeoutTest.testMostInSyncReplicasCanWinElection <<<
   [junit4]> Throwable #1: java.lang.AssertionError: Timeout waiting for 
new leader
   [junit4]> null
   [junit4]> Live Nodes: [127.0.0.1:35393_solr, 127.0.0.1:39510_solr, 
127.0.0.1:42166_solr]
   [junit4]> Last available state: 
DocCollection(collection1//collections/collection1/state.json/14)={
   [junit4]>   "pullReplicas":"0",
   [junit4]>   "replicationFactor":"3",
   [junit4]>   "shards":{"shard1":{
   [junit4]>   "range":"8000-7fff",
   [junit4]>   "state":"active",
   [junit4]>   "replicas":{
   [junit4]> "core_node62":{
   [junit4]>   "core":"collection1_shard1_replica_n61",
   [junit4]>   "base_url":"https://127.0.0.1:45622/solr;,
   [junit4]>   "node_name":"127.0.0.1:45622_solr",
   [junit4]>   "state":"down",
   [junit4]>   "type":"NRT"},
   [junit4]> "core_node64":{
   [junit4]>   "core":"collection1_shard1_replica_n63",
   [junit4]>   "base_url":"https://127.0.0.1:42166/solr;,
   [junit4]>   "node_name":"127.0.0.1:42166_solr",
   [junit4]>   "state":"active",
   [junit4]>   "type":"NRT"},
   [junit4]> "core_node66":{
   [junit4]>   "core":"collection1_shard1_replica_n65",
   [junit4]>   "base_url":"https://127.0.0.1:39510/solr;,
   [junit4]>   "node_name":"127.0.0.1:39510_solr",
   [junit4]>   "state":"active",
   [junit4]>   "type":"NRT",
   [junit4]>   "router":{"name":"compositeId"},
   [junit4]>   "maxShardsPerNode":"1",
   [junit4]>   "autoAddReplicas":"false",
   [junit4]>   "nrtReplicas":"3",
   [junit4]>   "tlogReplicas":"0"}
   [junit4]>at 
__randomizedtesting.SeedInfo.seed([F36F789B06B8474E:5B736421C4F87364]:0)
   [junit4]>at 
org.apache.solr.cloud.SolrCloudTestCase.waitForState(SolrCloudTestCase.java:269)
   [junit4]>at 
org.apache.solr.cloud.LeaderVoteWaitTimeoutTest.testMostInSyncReplicasCanWinElection(LeaderVoteWaitTimeoutTest.java:189)
   [junit4]>at java.lang.Thread.run(Thread.java:748)
{code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (LUCENE-8204) ReqOptSumScorer should leverage sub scorers' per-block max scores

[Adrien Grand (JIRA)]

Adrien Grand created LUCENE-8204:


 Summary: ReqOptSumScorer should leverage sub scorers' per-block 
max scores
 Key: LUCENE-8204
 URL: https://issues.apache.org/jira/browse/LUCENE-8204
 Project: Lucene - Core
  Issue Type: Improvement
Reporter: Adrien Grand


Currently it only looks at max scores on the entire segment. Given that 
per-block max scores usually give lower upper bounds of the score, this should 
help.

This is especially important for LUCENE-8197 to work well since the main query 
would typically be added as a MUST clauses of a boolean query while the query 
that scores on features would be a SHOULD clause.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-7.3-Linux (64bit/jdk-9.0.4) - Build # 5 - Unstable!

[Policeman Jenkins Server]

Error processing tokens: Error while parsing action 
'Text/ZeroOrMore/FirstOf/Token/DelimitedToken/DelimitedToken_Action3' at input 
position (line 79, pos 4):
)"}
   ^

java.lang.OutOfMemoryError: Java heap space

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-master-MacOSX (64bit/jdk-9) - Build # 4496 - Still Unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-MacOSX/4496/
Java: 64bit/jdk-9 -XX:-UseCompressedOops -XX:+UseSerialGC

1 tests failed.
FAILED:  org.apache.solr.cloud.DocValuesNotIndexedTest.testGroupingDVOnly

Error Message:
Unexpected number of elements in the group for intGSF: 6

Stack Trace:
java.lang.AssertionError: Unexpected number of elements in the group for 
intGSF: 6
at 
__randomizedtesting.SeedInfo.seed([EC5AC5312FBE0C6:957EC20B5FA3D298]:0)
at org.junit.Assert.fail(Assert.java:93)
at 
org.apache.solr.cloud.DocValuesNotIndexedTest.testGroupingDVOnly(DocValuesNotIndexedTest.java:378)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:564)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.base/java.lang.Thread.run(Thread.java:844)




Build Log:
[...truncated 1847 lines...]
   [junit4] JVM J1: stdout was not empty, see: 
/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/build/core/test/temp/junit4-J1-20180314_103436_00965245789921747661.sysout
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) 
   [junit4] codec: 

[jira] [Comment Edited] (SOLR-12063) Fix tlog entry indexes in UpdateLog for CDCR to work smoothly.

[Amrit Sarkar (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-12063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16398180#comment-16398180
 ] 

Amrit Sarkar edited comment on SOLR-12063 at 3/14/18 12:07 PM:
---

If we got with the current SOLR-12083 patch, uploaded 14/03/18 1PM IST, we 
don't need to make any changes in {{TestStressRecovery}}, underlying test 
framework will randomise ULog at core initialization.


was (Author: sarkaramr...@gmail.com):
If we got with the current SOLR-12083 patch, uploaded 14/03/18 1PM IST, we 
don't need to make any changes in {{TestStressRecovery}}, underlying test 
framework with randomise at core initialization.

> Fix tlog entry indexes in UpdateLog for CDCR to work smoothly.
> --
>
> Key: SOLR-12063
> URL: https://issues.apache.org/jira/browse/SOLR-12063
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: CDCR
>Affects Versions: 7.2, 7.2.1
>Reporter: Amrit Sarkar
>Assignee: Varun Thacker
>Priority: Major
> Attachments: SOLR-12063.patch, SOLR-12063.patch, SOLR-12063.patch, 
> SOLR-12063.patch, test-report-PeerSyncTest, test-report-TestStressRecovery
>
>
> In *UpdateLog*, {{RecentUpdates}} reads the entry of tlogs, and throughout 
> the project the entry indexes for various operations are consistent, but odd 
> in this part. As we included new entry in TransactionLog for CDCR, read 
> operations in {{update()}} method of {{RecentUpdates}} throw error rightfully 
> as elements are read from wrong indexes of tlog entry. The entry indexes of 
> llog should be consistent throughout.
> {code}
>   [beaster]   2> 27394 WARN  (qtp97093533-72) [n:127.0.0.1:44658_solr 
> c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] 
> o.a.s.u.UpdateLog Unexpected log entry or corrupt log.  Entry=[2, 
> -1594312216007409664, [B@28e6859c, true]
>   [beaster]   2> java.lang.ClassCastException: java.lang.Boolean cannot be 
> cast to [B
>   [beaster]   2>  at 
> org.apache.solr.update.UpdateLog$RecentUpdates.update(UpdateLog.java:1443)
>   [beaster]   2>  at 
> org.apache.solr.update.UpdateLog$RecentUpdates.(UpdateLog.java:1340)
>   [beaster]   2>  at 
> org.apache.solr.update.UpdateLog.getRecentUpdates(UpdateLog.java:1513)
>   [beaster]   2>  at 
> org.apache.solr.handler.CdcrRequestHandler.handleShardCheckpointAction(CdcrRequestHandler.java:448)
>   [beaster]   2>  at 
> org.apache.solr.handler.CdcrRequestHandler.handleRequestBody(CdcrRequestHandler.java:198)
>   [beaster]   2>  at 
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:195)
>   [beaster]   2>  at 
> org.apache.solr.core.SolrCore.execute(SolrCore.java:2503)
>   [beaster]   2>  at 
> org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:711)
>   [beaster]   2>  at 
> org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:517)
>   [beaster]   2>  at 
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:384)
>   [beaster]   2>  at 
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:330)
>   [beaster]   2>  at 
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1637)
>   [beaster]   2>  at 
> org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139)
>   [beaster]   2>  at 
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1637)
>   [beaster]   2>  at 
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533)
>   [beaster]   2>  at 
> org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:188)
>   [beaster]   2>  at 
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595)
>   [beaster]   2>  at 
> org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:188)
>   [beaster]   2>  at 
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1253)
>   [beaster]   2>  at 
> org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:168)
>   [beaster]   2>  at 
> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473)
>   [beaster]   2>  at 
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564)
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org