[JENKINS] Lucene-Solr-6.x-Linux (64bit/jdk1.8.0_102) - Build # 1453 - Still Unstable!

[Policeman Jenkins Server]

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-6.x-Linux/1453/
Java: 64bit/jdk1.8.0_102 -XX:+UseCompressedOops -XX:+UseParallelGC

1 tests failed.
FAILED:  org.apache.solr.core.TestDynamicLoading.testDynamicLoading

Error Message:
Could not get expected value  'X val' for path 'x' full output: {   
"responseHeader":{ "status":0, "QTime":0},   "params":{"wt":"json"},   
"context":{ "webapp":"", "path":"/test1", "httpMethod":"GET"},   
"class":"org.apache.solr.core.BlobStoreTestRequestHandler",   "x":null},  from 
server:  null

Stack Trace:
java.lang.AssertionError: Could not get expected value  'X val' for path 'x' 
full output: {
  "responseHeader":{
"status":0,
"QTime":0},
  "params":{"wt":"json"},
  "context":{
"webapp":"",
"path":"/test1",
"httpMethod":"GET"},
  "class":"org.apache.solr.core.BlobStoreTestRequestHandler",
  "x":null},  from server:  null
at 
__randomizedtesting.SeedInfo.seed([F179491E1DECABF7:29346449EA310E57]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at 
org.apache.solr.core.TestSolrConfigHandler.testForResponseElement(TestSolrConfigHandler.java:535)
at 
org.apache.solr.core.TestDynamicLoading.testDynamicLoading(TestDynamicLoading.java:232)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org

[jira] [Updated] (LUCENE-7344) Deletion by query of uncommitted docs not working with DV updates

[Hoss Man (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-7344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Hoss Man updated LUCENE-7344:
-
Attachment: LUCENE-7344.patch
LUCENE-7344_hoss_experiment.patch


Something yonik pointed out in SOLR-5944 is that Solr could possibly work 
around this issue if we always opened an NRT Searcher/Reader prior to doing any 
deleteByQuery calls -- but that ideally we could do that conditionally based on 
knowing if/when any updateDocVal calls had happened to the IndexWriter since 
the last time an NRT Searcher/Reader was opened.

I didn't really understand that comment at the time, but talking about it 
offline with shalin/ishan a bit I realized it's essentially just doing what 
Shai suggested was the "To properly solve this problem..." solution, but by 
putting the burden on the caller to "force" the IndexWriter to reopen it's 
LeafReaders to return an uptodate NRT Reader, and then IndexWriter will use 
those uptodate LeafReaders for applying subsequent deletes.

The LUCENE-7344.patch I'm attaching here includes a proof of concept of this 
workaround -- written largely to convince myself it would actually work

In this LUCENE-7344.patch, I have:

* reverted Shai's changes in previous patches to BufferedUpdatesStream (so that 
Deletes by Query are always applied before updateDocValues just like on master 
today)
* updated TestNumericDocValuesUpdates so that an (unused) NRT reader is always 
opened before any deleteDocuments(Query) call that depends on a previous 
updateDocVals call (see {{FORCE_NEW_READER_HACK_PRE = true}})
* Updated biasedMixOfRandomUpdates to assert the sequence numbers returned by 
all IndexWriter method calls have the expected values
** This is unrelated to the other changes, It's just something I realized was a 
good idea to include as a sanity check in this test.

...with these changes, all tests seem to pass reliably.

Assuming we document that forcing a reader reopen like this is required for 
users in this situation (ie: a caveat on the updateDocVals javadocs that doing 
something like this is required in order for subsequent deleteDocuments(Query) 
calls to succeed if the delete Query refers to the updated DocValues field) 
that could get us most of the way towards a low-impact (on 
performance/complexity of IndexWriter) solution unless/until Shai can a more 
optimized solution working -- at which point we could just remove the caveat 
from the javadocs.

But...

I'm leary of a solution that:
# relies on users to read javadocs carefully
# relies on callers of deleteDocuments(Query) to know/realize if/when that 
Query involves DocValues *and* know if any _other_ callers may have used 
update(Binary|Numeric)DocValues
#* particularly since the "Query" may have come from a upstream code
# results in silent unexpected data loss when the rules spelled out in the 
javadocs aren't followed.

So with that in mind, I started wondering what it would take to:
* Provide a method to indicate if there are updateDocVal calls "pending", so 
callers could conditionally decide to force reopen the NRT reader before doing 
a deleteDocuments(Query)
* Throw an exception if a caller attempts to violate the "rules" by calling a 
deleteDocuments with a Query that involves a DocValues field with "pending" 
updates w/o forcting the NRT reader to reopen.

The first seemed like it would be somewhat straightforward, and not strictly 
neccessary for corectness, So I tried tackling the second one.

This experiment can be seen in the attached LUCENE-7344_hoss_experiment.patch 
(apply *after* LUCENE-7344.patch)

The crux of the change is adding new code to 
BufferedUpdatesStream.applyQueryDeletes to check if there are any DVUpdates, 
and if there are it wraps the LeafReader in a FilterLeafReader that throws an 
Exception if LeafReader.get(Binary|Numeric)DocValues(...) is called on any of 
the field names used in the DVUpdates -- so any usage of the DocValues on those 
fields in the deletion Query will trigger an exception and the callers will 
know something is wrong.

For the most part this experiment seems viable -- except that the new Exception 
also gets thrown on writer.close() (or when opening an NRT reader) when there 
have been dvUpdates _after_ a deleteDocuments(Query) that uses the same 
DocValues field -- for example:
{code}
writer.addDocument(doc(0));
writer.deleteDocuments(DocValuesRangeQuery.newLongRange("val", 5L, 10L, 
true, true));
writer.updateNumericDocValue(new Term("id", "doc-0"), "val",  7L);

// ...this would cause my new error checking code to throw 
CanNotDeleteByQueryOnUpdatedDocValues
writer.close(); 
{code}

I'm not really sure why that's happening, since (afaict) the 
deleteDocuments(Query) should have already been applied and been taken into 
account by a (reopened) reader, so it seems like the queriesIter passed to 
applyQueryDeletes when the writer is c

[JENKINS] Lucene-Solr-master-Linux (32bit/jdk1.8.0_102) - Build # 17547 - Unstable!

[Policeman Jenkins Server]

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/17547/
Java: 32bit/jdk1.8.0_102 -server -XX:+UseG1GC

1 tests failed.
FAILED:  org.apache.solr.handler.TestReqParamsAPI.test

Error Message:
Could not get expected value  'null' for path 'response/params/y/p' full 
output: {   "responseHeader":{ "status":0, "QTime":0},   "response":{   
  "znodeVersion":4, "params":{   "x":{ "a":"A val", 
"b":"B val", "_appends_":{"add":"first"}, 
"_invariants_":{"fixed":"f"}, "":{"v":1}},   "y":{ "p":"P 
val", "q":"Q val", "":{"v":2},  from server:  
https://127.0.0.1:35232/oh/collection1

Stack Trace:
java.lang.AssertionError: Could not get expected value  'null' for path 
'response/params/y/p' full output: {
  "responseHeader":{
"status":0,
"QTime":0},
  "response":{
"znodeVersion":4,
"params":{
  "x":{
"a":"A val",
"b":"B val",
"_appends_":{"add":"first"},
"_invariants_":{"fixed":"f"},
"":{"v":1}},
  "y":{
"p":"P val",
"q":"Q val",
"":{"v":2},  from server:  https://127.0.0.1:35232/oh/collection1
at 
__randomizedtesting.SeedInfo.seed([41D1574CE8147547:C985689646E818BF]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at 
org.apache.solr.core.TestSolrConfigHandler.testForResponseElement(TestSolrConfigHandler.java:535)
at 
org.apache.solr.handler.TestReqParamsAPI.testReqParams(TestReqParamsAPI.java:258)
at 
org.apache.solr.handler.TestReqParamsAPI.test(TestReqParamsAPI.java:61)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:985)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:960)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverri

[JENKINS] Lucene-Solr-6.x-Windows (32bit/jdk1.8.0_102) - Build # 381 - Still Unstable!

[Policeman Jenkins Server]

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-6.x-Windows/381/
Java: 32bit/jdk1.8.0_102 -client -XX:+UseSerialGC

3 tests failed.
FAILED:  org.apache.solr.handler.TestReplicationHandlerBackup.doTestBackup

Error Message:
no segments* file found in 
SimpleFSDirectory@C:\Users\jenkins\workspace\Lucene-Solr-6.x-Windows\solr\build\solr-core\test\J0\temp\solr.handler.TestReplicationHandlerBackup_F4DB430EC78019D6-001\solr-instance-002\collection1\data\snapshot.amtepehgi
 lockFactory=org.apache.lucene.store.NativeFSLockFactory@fdc793: files: [_0.si, 
_0_Lucene50_0.doc, _0_Lucene50_0.tim]

Stack Trace:
org.apache.lucene.index.IndexNotFoundException: no segments* file found in 
SimpleFSDirectory@C:\Users\jenkins\workspace\Lucene-Solr-6.x-Windows\solr\build\solr-core\test\J0\temp\solr.handler.TestReplicationHandlerBackup_F4DB430EC78019D6-001\solr-instance-002\collection1\data\snapshot.amtepehgi
 lockFactory=org.apache.lucene.store.NativeFSLockFactory@fdc793: files: [_0.si, 
_0_Lucene50_0.doc, _0_Lucene50_0.tim]
at 
__randomizedtesting.SeedInfo.seed([F4DB430EC78019D6:B550636BE03EEA99]:0)
at 
org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:680)
at 
org.apache.lucene.index.StandardDirectoryReader.open(StandardDirectoryReader.java:77)
at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:63)
at 
org.apache.solr.handler.TestReplicationHandlerBackup.verify(TestReplicationHandlerBackup.java:150)
at 
org.apache.solr.handler.TestReplicationHandlerBackup.doTestBackup(TestReplicationHandlerBackup.java:214)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 

[JENKINS-EA] Lucene-Solr-6.x-Linux (32bit/jdk-9-ea+129) - Build # 1452 - Unstable!

[Policeman Jenkins Server]

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-6.x-Linux/1452/
Java: 32bit/jdk-9-ea+129 -client -XX:+UseG1GC

1 tests failed.
FAILED:  org.apache.solr.cloud.ForceLeaderTest.testReplicasInLIRNoLeader

Error Message:
org.apache.solr.client.solrj.SolrServerException: No live SolrServers available 
to handle this 
request:[http://127.0.0.1:33606/forceleader_test_collection_shard1_replica2]

Stack Trace:
org.apache.solr.client.solrj.SolrServerException: 
org.apache.solr.client.solrj.SolrServerException: No live SolrServers available 
to handle this 
request:[http://127.0.0.1:33606/forceleader_test_collection_shard1_replica2]
at 
__randomizedtesting.SeedInfo.seed([36930A475280CD67:D0043E876B023406]:0)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.directUpdate(CloudSolrClient.java:774)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1172)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:1061)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:997)
at 
org.apache.solr.cloud.AbstractFullDistribZkTestBase.sendDocsWithRetry(AbstractFullDistribZkTestBase.java:753)
at 
org.apache.solr.cloud.AbstractFullDistribZkTestBase.sendDocsWithRetry(AbstractFullDistribZkTestBase.java:741)
at 
org.apache.solr.cloud.ForceLeaderTest.sendDoc(ForceLeaderTest.java:424)
at 
org.apache.solr.cloud.ForceLeaderTest.assertSendDocFails(ForceLeaderTest.java:315)
at 
org.apache.solr.cloud.ForceLeaderTest.testReplicasInLIRNoLeader(ForceLeaderTest.java:110)
at 
jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(java.base@9-ea/Native 
Method)
at 
jdk.internal.reflect.NativeMethodAccessorImpl.invoke(java.base@9-ea/NativeMethodAccessorImpl.java:62)
at 
jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(java.base@9-ea/DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(java.base@9-ea/Method.java:533)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedte

[JENKINS] Lucene-Solr-master-Windows (32bit/jdk1.8.0_102) - Build # 6045 - Still Unstable!

[Policeman Jenkins Server]

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-master-Windows/6045/
Java: 32bit/jdk1.8.0_102 -server -XX:+UseConcMarkSweepGC

7 tests failed.
FAILED:  
org.apache.solr.cloud.TestSolrCloudWithDelegationTokens.testDelegationTokenCancelFail

Error Message:
expected:<200> but was:<404>

Stack Trace:
java.lang.AssertionError: expected:<200> but was:<404>
at 
__randomizedtesting.SeedInfo.seed([124BAE16669EB14F:7AF49B3CB604A3A3]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.junit.Assert.assertEquals(Assert.java:456)
at 
org.apache.solr.cloud.TestSolrCloudWithDelegationTokens.cancelDelegationToken(TestSolrCloudWithDelegationTokens.java:137)
at 
org.apache.solr.cloud.TestSolrCloudWithDelegationTokens.testDelegationTokenCancelFail(TestSolrCloudWithDelegationTokens.java:282)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(Thr

[jira] [Updated] (SOLR-9319) DELETEREPLICA should be able to accept just count and remove replicas intelligenty

[Nitin Sharma (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-9319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nitin Sharma updated SOLR-9319:
---
Attachment: DeleteReplicaPatch.jpg
Delete_Replica_count_1.jpg
Delete_Replica_invalid.jpg
Delete_Replica_with_only_1replica.jpg
Delte_replica_before.jpg
Delete_replica_count2.jpg
Delte_replica_after.jpg
Old_deletereplica_api_works.jpg

Screenshots with 
a) API with Count param and intelligent removal
b) API with count and only 1 replica remaining
c) API with count and more replicas are requested than present
d) Old API call without count param working as expected


> DELETEREPLICA should be able to accept  just count and remove replicas 
> intelligenty
> ---
>
> Key: SOLR-9319
> URL: https://issues.apache.org/jira/browse/SOLR-9319
> Project: Solr
>  Issue Type: Sub-task
>  Components: SolrCloud
>Reporter: Noble Paul
> Fix For: 6.1
>
> Attachments: DeleteReplicaPatch.jpg, Delete_Replica_count_1.jpg, 
> Delete_Replica_invalid.jpg, Delete_Replica_with_only_1replica.jpg, 
> Delete_replica_count2.jpg, Delte_replica_after.jpg, Delte_replica_before.jpg, 
> Old_deletereplica_api_works.jpg, SOLR-9319.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-9319) DELETEREPLICA should be able to accept just count and remove replicas intelligenty

[Nitin Sharma (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-9319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nitin Sharma updated SOLR-9319:
---
Attachment: SOLR-9319.patch

> DELETEREPLICA should be able to accept  just count and remove replicas 
> intelligenty
> ---
>
> Key: SOLR-9319
> URL: https://issues.apache.org/jira/browse/SOLR-9319
> Project: Solr
>  Issue Type: Sub-task
>  Components: SolrCloud
>Reporter: Noble Paul
> Fix For: 6.1
>
> Attachments: SOLR-9319.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-6.x-Solaris (64bit/jdk1.8.0) - Build # 324 - Still Unstable!

[Policeman Jenkins Server]

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-6.x-Solaris/324/
Java: 64bit/jdk1.8.0 -XX:+UseCompressedOops -XX:+UseSerialGC

1 tests failed.
FAILED:  
org.apache.solr.handler.component.SpellCheckComponentTest.testThresholdTokenFrequency

Error Message:
Path not found: /spellcheck/suggestions/[1]/suggestion

Stack Trace:
java.lang.RuntimeException: Path not found: 
/spellcheck/suggestions/[1]/suggestion
at 
__randomizedtesting.SeedInfo.seed([41FD6DC3DC42C546:CB5AE23253A9FC3D]:0)
at org.apache.solr.SolrTestCaseJ4.assertJQ(SolrTestCaseJ4.java:871)
at org.apache.solr.SolrTestCaseJ4.assertJQ(SolrTestCaseJ4.java:818)
at 
org.apache.solr.handler.component.SpellCheckComponentTest.testThresholdTokenFrequency(SpellCheckComponentTest.java:277)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 10732 lines...]
   [junit4] Suite: org.apache.solr.handler.component.SpellCheckComponentTest
   [junit4]   2> Creati

[JENKINS] Lucene-Solr-master-Linux (32bit/jdk1.8.0_102) - Build # 17545 - Unstable!

[Policeman Jenkins Server]

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/17545/
Java: 32bit/jdk1.8.0_102 -server -XX:+UseParallelGC

1 tests failed.
FAILED:  org.apache.solr.cloud.LeaderFailoverAfterPartitionTest.test

Error Message:
No live SolrServers available to handle this 
request:[http://127.0.0.1:42842/xy/s/c8n_1x3_lf_shard1_replica3]

Stack Trace:
org.apache.solr.client.solrj.impl.CloudSolrClient$RouteException: No live 
SolrServers available to handle this 
request:[http://127.0.0.1:42842/xy/s/c8n_1x3_lf_shard1_replica3]
at 
__randomizedtesting.SeedInfo.seed([E58E1BF8E034DAF3:6DDA24224EC8B70B]:0)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.directUpdate(CloudSolrClient.java:760)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1172)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:1061)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:997)
at 
org.apache.solr.cloud.AbstractFullDistribZkTestBase.sendDocsWithRetry(AbstractFullDistribZkTestBase.java:753)
at 
org.apache.solr.cloud.AbstractFullDistribZkTestBase.sendDocsWithRetry(AbstractFullDistribZkTestBase.java:741)
at 
org.apache.solr.cloud.LeaderFailoverAfterPartitionTest.testRf3WithLeaderFailover(LeaderFailoverAfterPartitionTest.java:178)
at 
org.apache.solr.cloud.LeaderFailoverAfterPartitionTest.test(LeaderFailoverAfterPartitionTest.java:57)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:985)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:960)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com

[jira] [Updated] (SOLR-9310) PeerSync fails on a node restart due to IndexFingerPrint mismatch

[Pushkar Raste (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-9310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Pushkar Raste updated SOLR-9310:

Attachment: SOLR-9310_3ReplicaTest.patch
PeerSync_3Node_Setup.jpg

Here is my patch with update test for scenario Noble is concerned about and a 
small diagram depicting what is going on in the test.

Test for my patch still passes.

If it looks good, I will take stab at fixing PeerSync failure (due index 
fingerprint) during active indexing, however, due to race conditions, it might 
not work well. 

Can we setup a call to discuss this ?

> PeerSync fails on a node restart due to IndexFingerPrint mismatch
> -
>
> Key: SOLR-9310
> URL: https://issues.apache.org/jira/browse/SOLR-9310
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Pushkar Raste
>Assignee: Noble Paul
> Attachments: PeerSync_3Node_Setup.jpg, PeerSync_Experiment.patch, 
> SOLR-9310.patch, SOLR-9310.patch, SOLR-9310.patch, SOLR-9310.patch, 
> SOLR-9310_3ReplicaTest.patch
>
>
> I found that Peer Sync fails if a node restarts and documents were indexed 
> while node was down. IndexFingerPrint check fails after recovering node 
> applies updates. 
> This happens only when node restarts and not if node just misses updates due 
> reason other than it being down.
> Please check attached patch for the test.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7409) Look into making mmapdirectory's unmap safer

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15419405#comment-15419405
 ] 

ASF subversion and git services commented on LUCENE-7409:
-

Commit 99785c0e0949f4bd47486ef8dd7ff6af7ff1e3c1 in lucene-solr's branch 
refs/heads/branch_6x from [~thetaphi]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=99785c0 ]

LUCENE-7409: Fix comments as suggested by Dawid


> Look into making mmapdirectory's unmap safer
> 
>
> Key: LUCENE-7409
> URL: https://issues.apache.org/jira/browse/LUCENE-7409
> Project: Lucene - Core
>  Issue Type: Improvement
>  Components: core/store
>Reporter: Robert Muir
>Assignee: Uwe Schindler
>  Labels: crush, performance
> Fix For: master (7.0), 6.2
>
> Attachments: LUCENE-7409.patch, StoreStore.java
>
>
> I have seen a few bugs around this recently: of course its a bug in 
> application code but a JVM crash is not good. 
> I think we should see if we can prevent the crashes better than the current 
> weak map, e.g. make it a safer option.
> I made an ugly prototype here: 
> https://github.com/apache/lucene-solr/compare/master...rmuir:ace?expand=1
> It has a test that crashes the JVM without the patch but passes with.
> Hacky patch only implements readBytes() but has no problems with the 
> luceneutil benchmark (1M):
> {noformat}
> Report after iter 19:
> TaskQPS base  StdDev   QPS patch  StdDev  
>   Pct diff
>   IntNRQ  105.23 (17.6%)  100.42 (10.1%)   
> -4.6% ( -27% -   28%)
>  Respell  128.35 (13.2%)  125.88  (7.4%)   
> -1.9% ( -19% -   21%)
>   Fuzzy1  110.14 (17.2%)  108.28 (13.2%)   
> -1.7% ( -27% -   34%)
>LowPhrase  337.02 (13.0%)  333.72  (9.3%)   
> -1.0% ( -20% -   24%)
>MedPhrase  146.44 (12.9%)  145.55  (8.0%)   
> -0.6% ( -19% -   23%)
>  MedSpanNear   96.85 (13.1%)   96.57  (7.8%)   
> -0.3% ( -18% -   23%)
> HighSpanNear   95.85 (13.9%)   96.33  (8.2%)
> 0.5% ( -18% -   26%)
>   HighPhrase  146.84 (13.6%)  148.40  (8.4%)
> 1.1% ( -18% -   26%)
> HighTerm  295.15 (15.8%)  298.77  (9.5%)
> 1.2% ( -20% -   31%)
>  LowSpanNear  268.80 (12.4%)  272.16  (7.9%)
> 1.2% ( -16% -   24%)
> Wildcard  284.09 (11.7%)  290.91  (8.9%)
> 2.4% ( -16% -   25%)
>  Prefix3  212.50 (15.4%)  217.76 (10.0%)
> 2.5% ( -19% -   32%)
>OrHighLow  358.65 (15.0%)  368.93 (10.7%)
> 2.9% ( -19% -   33%)
>   AndHighMed  799.65 (13.2%)  834.74  (7.8%)
> 4.4% ( -14% -   29%)
>  MedSloppyPhrase  229.36 (15.9%)  239.95  (9.8%)
> 4.6% ( -18% -   36%)
>   Fuzzy2   69.58 (14.6%)   72.82 (14.5%)
> 4.7% ( -21% -   39%)
>  AndHighHigh  426.98 (12.8%)  451.77  (7.3%)
> 5.8% ( -12% -   29%)
>  MedTerm 1361.11 (14.5%) 1450.90  (9.2%)
> 6.6% ( -14% -   35%)
> PKLookup  266.61 (13.4%)  284.28  (8.4%)
> 6.6% ( -13% -   32%)
> HighSloppyPhrase  251.22 (16.9%)  268.32 (10.7%)
> 6.8% ( -17% -   41%)
>OrHighMed  235.92 (17.2%)  253.12 (12.8%)
> 7.3% ( -19% -   45%)
>   OrHighHigh  186.79 (13.5%)  201.15  (9.7%)
> 7.7% ( -13% -   35%)
>  LowSloppyPhrase  395.23 (15.9%)  425.93  (9.3%)
> 7.8% ( -15% -   39%)
>   AndHighLow 1128.28 (14.9%) 1242.11  (8.2%)   
> 10.1% ( -11% -   38%)
>  LowTerm 3024.62 (12.9%) 3367.65  (9.7%)   
> 11.3% (  -9% -   39%)
> {noformat}
> We should do more testing. Maybe its totally the wrong tradeoff, maybe we 
> only need handles for getters and everything inlines correctly, rather than 
> needing a ton for every getXYZ() method...



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7409) Look into making mmapdirectory's unmap safer

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15419402#comment-15419402
 ] 

ASF subversion and git services commented on LUCENE-7409:
-

Commit 97f6bb7d7ff43f4501492eee07e6a9200b402b13 in lucene-solr's branch 
refs/heads/master from [~thetaphi]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=97f6bb7 ]

LUCENE-7409: Fix comments as suggested by Dawid


> Look into making mmapdirectory's unmap safer
> 
>
> Key: LUCENE-7409
> URL: https://issues.apache.org/jira/browse/LUCENE-7409
> Project: Lucene - Core
>  Issue Type: Improvement
>  Components: core/store
>Reporter: Robert Muir
>Assignee: Uwe Schindler
>  Labels: crush, performance
> Fix For: master (7.0), 6.2
>
> Attachments: LUCENE-7409.patch, StoreStore.java
>
>
> I have seen a few bugs around this recently: of course its a bug in 
> application code but a JVM crash is not good. 
> I think we should see if we can prevent the crashes better than the current 
> weak map, e.g. make it a safer option.
> I made an ugly prototype here: 
> https://github.com/apache/lucene-solr/compare/master...rmuir:ace?expand=1
> It has a test that crashes the JVM without the patch but passes with.
> Hacky patch only implements readBytes() but has no problems with the 
> luceneutil benchmark (1M):
> {noformat}
> Report after iter 19:
> TaskQPS base  StdDev   QPS patch  StdDev  
>   Pct diff
>   IntNRQ  105.23 (17.6%)  100.42 (10.1%)   
> -4.6% ( -27% -   28%)
>  Respell  128.35 (13.2%)  125.88  (7.4%)   
> -1.9% ( -19% -   21%)
>   Fuzzy1  110.14 (17.2%)  108.28 (13.2%)   
> -1.7% ( -27% -   34%)
>LowPhrase  337.02 (13.0%)  333.72  (9.3%)   
> -1.0% ( -20% -   24%)
>MedPhrase  146.44 (12.9%)  145.55  (8.0%)   
> -0.6% ( -19% -   23%)
>  MedSpanNear   96.85 (13.1%)   96.57  (7.8%)   
> -0.3% ( -18% -   23%)
> HighSpanNear   95.85 (13.9%)   96.33  (8.2%)
> 0.5% ( -18% -   26%)
>   HighPhrase  146.84 (13.6%)  148.40  (8.4%)
> 1.1% ( -18% -   26%)
> HighTerm  295.15 (15.8%)  298.77  (9.5%)
> 1.2% ( -20% -   31%)
>  LowSpanNear  268.80 (12.4%)  272.16  (7.9%)
> 1.2% ( -16% -   24%)
> Wildcard  284.09 (11.7%)  290.91  (8.9%)
> 2.4% ( -16% -   25%)
>  Prefix3  212.50 (15.4%)  217.76 (10.0%)
> 2.5% ( -19% -   32%)
>OrHighLow  358.65 (15.0%)  368.93 (10.7%)
> 2.9% ( -19% -   33%)
>   AndHighMed  799.65 (13.2%)  834.74  (7.8%)
> 4.4% ( -14% -   29%)
>  MedSloppyPhrase  229.36 (15.9%)  239.95  (9.8%)
> 4.6% ( -18% -   36%)
>   Fuzzy2   69.58 (14.6%)   72.82 (14.5%)
> 4.7% ( -21% -   39%)
>  AndHighHigh  426.98 (12.8%)  451.77  (7.3%)
> 5.8% ( -12% -   29%)
>  MedTerm 1361.11 (14.5%) 1450.90  (9.2%)
> 6.6% ( -14% -   35%)
> PKLookup  266.61 (13.4%)  284.28  (8.4%)
> 6.6% ( -13% -   32%)
> HighSloppyPhrase  251.22 (16.9%)  268.32 (10.7%)
> 6.8% ( -17% -   41%)
>OrHighMed  235.92 (17.2%)  253.12 (12.8%)
> 7.3% ( -19% -   45%)
>   OrHighHigh  186.79 (13.5%)  201.15  (9.7%)
> 7.7% ( -13% -   35%)
>  LowSloppyPhrase  395.23 (15.9%)  425.93  (9.3%)
> 7.8% ( -15% -   39%)
>   AndHighLow 1128.28 (14.9%) 1242.11  (8.2%)   
> 10.1% ( -11% -   38%)
>  LowTerm 3024.62 (12.9%) 3367.65  (9.7%)   
> 11.3% (  -9% -   39%)
> {noformat}
> We should do more testing. Maybe its totally the wrong tradeoff, maybe we 
> only need handles for getters and everything inlines correctly, rather than 
> needing a ton for every getXYZ() method...



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7409) Look into making mmapdirectory's unmap safer

[Uwe Schindler (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15419380#comment-15419380
 ] 

Uwe Schindler commented on LUCENE-7409:
---

Hi, I can change the comment! I was about to do the same.

MutableCallSite does more so your test class works fine, if I use a real 
MutableCallSite. The store-store-barrier there is used to really do ordering 
(but this is related to MutableCallSite's native implementation of setTarget).

> Look into making mmapdirectory's unmap safer
> 
>
> Key: LUCENE-7409
> URL: https://issues.apache.org/jira/browse/LUCENE-7409
> Project: Lucene - Core
>  Issue Type: Improvement
>  Components: core/store
>Reporter: Robert Muir
>Assignee: Uwe Schindler
>  Labels: crush, performance
> Fix For: master (7.0), 6.2
>
> Attachments: LUCENE-7409.patch, StoreStore.java
>
>
> I have seen a few bugs around this recently: of course its a bug in 
> application code but a JVM crash is not good. 
> I think we should see if we can prevent the crashes better than the current 
> weak map, e.g. make it a safer option.
> I made an ugly prototype here: 
> https://github.com/apache/lucene-solr/compare/master...rmuir:ace?expand=1
> It has a test that crashes the JVM without the patch but passes with.
> Hacky patch only implements readBytes() but has no problems with the 
> luceneutil benchmark (1M):
> {noformat}
> Report after iter 19:
> TaskQPS base  StdDev   QPS patch  StdDev  
>   Pct diff
>   IntNRQ  105.23 (17.6%)  100.42 (10.1%)   
> -4.6% ( -27% -   28%)
>  Respell  128.35 (13.2%)  125.88  (7.4%)   
> -1.9% ( -19% -   21%)
>   Fuzzy1  110.14 (17.2%)  108.28 (13.2%)   
> -1.7% ( -27% -   34%)
>LowPhrase  337.02 (13.0%)  333.72  (9.3%)   
> -1.0% ( -20% -   24%)
>MedPhrase  146.44 (12.9%)  145.55  (8.0%)   
> -0.6% ( -19% -   23%)
>  MedSpanNear   96.85 (13.1%)   96.57  (7.8%)   
> -0.3% ( -18% -   23%)
> HighSpanNear   95.85 (13.9%)   96.33  (8.2%)
> 0.5% ( -18% -   26%)
>   HighPhrase  146.84 (13.6%)  148.40  (8.4%)
> 1.1% ( -18% -   26%)
> HighTerm  295.15 (15.8%)  298.77  (9.5%)
> 1.2% ( -20% -   31%)
>  LowSpanNear  268.80 (12.4%)  272.16  (7.9%)
> 1.2% ( -16% -   24%)
> Wildcard  284.09 (11.7%)  290.91  (8.9%)
> 2.4% ( -16% -   25%)
>  Prefix3  212.50 (15.4%)  217.76 (10.0%)
> 2.5% ( -19% -   32%)
>OrHighLow  358.65 (15.0%)  368.93 (10.7%)
> 2.9% ( -19% -   33%)
>   AndHighMed  799.65 (13.2%)  834.74  (7.8%)
> 4.4% ( -14% -   29%)
>  MedSloppyPhrase  229.36 (15.9%)  239.95  (9.8%)
> 4.6% ( -18% -   36%)
>   Fuzzy2   69.58 (14.6%)   72.82 (14.5%)
> 4.7% ( -21% -   39%)
>  AndHighHigh  426.98 (12.8%)  451.77  (7.3%)
> 5.8% ( -12% -   29%)
>  MedTerm 1361.11 (14.5%) 1450.90  (9.2%)
> 6.6% ( -14% -   35%)
> PKLookup  266.61 (13.4%)  284.28  (8.4%)
> 6.6% ( -13% -   32%)
> HighSloppyPhrase  251.22 (16.9%)  268.32 (10.7%)
> 6.8% ( -17% -   41%)
>OrHighMed  235.92 (17.2%)  253.12 (12.8%)
> 7.3% ( -19% -   45%)
>   OrHighHigh  186.79 (13.5%)  201.15  (9.7%)
> 7.7% ( -13% -   35%)
>  LowSloppyPhrase  395.23 (15.9%)  425.93  (9.3%)
> 7.8% ( -15% -   39%)
>   AndHighLow 1128.28 (14.9%) 1242.11  (8.2%)   
> 10.1% ( -11% -   38%)
>  LowTerm 3024.62 (12.9%) 3367.65  (9.7%)   
> 11.3% (  -9% -   39%)
> {noformat}
> We should do more testing. Maybe its totally the wrong tradeoff, maybe we 
> only need handles for getters and everything inlines correctly, rather than 
> needing a ton for every getXYZ() method...



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-9408) Add solr commit data in TreeMergeRecordWriter

[Jessica Cheng Mallet (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-9408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jessica Cheng Mallet updated SOLR-9408:
---
Attachment: SOLR-9408.patch

> Add solr commit data in TreeMergeRecordWriter
> -
>
> Key: SOLR-9408
> URL: https://issues.apache.org/jira/browse/SOLR-9408
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: contrib - MapReduce
>Reporter: Jessica Cheng Mallet
>  Labels: mapreduce, solrcloud
> Attachments: SOLR-9408.patch, SOLR-9408.patch
>
>
> The lucene index produced by TreeMergeRecordWriter when the segments are 
> merged doesn't contain Solr's commit data, specifically, commitTimeMsec.
> This means that when this index is subsequently loaded into SolrCloud and if 
> the index stays unchanged so no newer commits occurs, ADDREPLICA will appear 
> to succeed but will not actually do any full sync due to SOLR-9369, resulting 
> in adding an empty index as a replica.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-9412) Update Macro Expander for replacement logic

[Jon Dorando (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-9412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jon Dorando updated SOLR-9412:
--
Attachment: test_macro_expander.patch

> Update Macro Expander for replacement logic
> ---
>
> Key: SOLR-9412
> URL: https://issues.apache.org/jira/browse/SOLR-9412
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Jon Dorando
>Priority: Minor
> Attachments: test_macro_expander.patch
>
>
> MacroExpander class was updated to allow to return null when replacement 
> parameters are missing. Right now it defaults to a blank space and it isn't 
> easily verifiable that a parameter was missing. Additionally, unit tests were 
> added for this case and the original use cases of Macro Expander.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-9412) Update Macro Expander for replacement logic

[Jon Dorando (JIRA)]

Jon Dorando created SOLR-9412:
-

 Summary: Update Macro Expander for replacement logic
 Key: SOLR-9412
 URL: https://issues.apache.org/jira/browse/SOLR-9412
 Project: Solr
  Issue Type: Improvement
  Security Level: Public (Default Security Level. Issues are Public)
Reporter: Jon Dorando
Priority: Minor


MacroExpander class was updated to allow to return null when replacement 
parameters are missing. Right now it defaults to a blank space and it isn't 
easily verifiable that a parameter was missing. Additionally, unit tests were 
added for this case and the original use cases of Macro Expander.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7409) Look into making mmapdirectory's unmap safer

[Dawid Weiss (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15419329#comment-15419329
 ] 

Dawid Weiss commented on LUCENE-7409:
-

Sorry to chip in late, Uwe.

{code}
+  /** not volatile, we use store-store barrier! */
+  private boolean invalidated = false;

+  // this should trigger a happens-before - so flushes all caches
+  barrier.lazySet(0);
{code}

I wouldn't mention barriers here (I'm also pretty confident lazyset doesn't 
create an edge in happens-before; it indicates memory ordering and as such 
isn't even clearly specified in the JMM), instead making the intent of this 
code clearer. Something like this:

{code}
+  /** not volatile, see comments on visibility in [...] . */
+  private boolean invalidated = false;

+  // This call should hopefully flush any CPU caches and as a result make
+  // the "invalidated" field update visible to other threads. We 
specifically
+  // don't make "invalidated" field volatile for performance reasons, 
hoping the JVM
+  // won't optimize away reads of that field and hardware should ensure
+  // caches are in sync after this call. This isn't entirely "fool-proof" 
+  // (see LUCENE-7409 discussion), but it has been shown to work in 
practice
+  // and we count on this behavior.
+  barrier.lazySet(0);
{code}




> Look into making mmapdirectory's unmap safer
> 
>
> Key: LUCENE-7409
> URL: https://issues.apache.org/jira/browse/LUCENE-7409
> Project: Lucene - Core
>  Issue Type: Improvement
>  Components: core/store
>Reporter: Robert Muir
>Assignee: Uwe Schindler
>  Labels: crush, performance
> Fix For: master (7.0), 6.2
>
> Attachments: LUCENE-7409.patch, StoreStore.java
>
>
> I have seen a few bugs around this recently: of course its a bug in 
> application code but a JVM crash is not good. 
> I think we should see if we can prevent the crashes better than the current 
> weak map, e.g. make it a safer option.
> I made an ugly prototype here: 
> https://github.com/apache/lucene-solr/compare/master...rmuir:ace?expand=1
> It has a test that crashes the JVM without the patch but passes with.
> Hacky patch only implements readBytes() but has no problems with the 
> luceneutil benchmark (1M):
> {noformat}
> Report after iter 19:
> TaskQPS base  StdDev   QPS patch  StdDev  
>   Pct diff
>   IntNRQ  105.23 (17.6%)  100.42 (10.1%)   
> -4.6% ( -27% -   28%)
>  Respell  128.35 (13.2%)  125.88  (7.4%)   
> -1.9% ( -19% -   21%)
>   Fuzzy1  110.14 (17.2%)  108.28 (13.2%)   
> -1.7% ( -27% -   34%)
>LowPhrase  337.02 (13.0%)  333.72  (9.3%)   
> -1.0% ( -20% -   24%)
>MedPhrase  146.44 (12.9%)  145.55  (8.0%)   
> -0.6% ( -19% -   23%)
>  MedSpanNear   96.85 (13.1%)   96.57  (7.8%)   
> -0.3% ( -18% -   23%)
> HighSpanNear   95.85 (13.9%)   96.33  (8.2%)
> 0.5% ( -18% -   26%)
>   HighPhrase  146.84 (13.6%)  148.40  (8.4%)
> 1.1% ( -18% -   26%)
> HighTerm  295.15 (15.8%)  298.77  (9.5%)
> 1.2% ( -20% -   31%)
>  LowSpanNear  268.80 (12.4%)  272.16  (7.9%)
> 1.2% ( -16% -   24%)
> Wildcard  284.09 (11.7%)  290.91  (8.9%)
> 2.4% ( -16% -   25%)
>  Prefix3  212.50 (15.4%)  217.76 (10.0%)
> 2.5% ( -19% -   32%)
>OrHighLow  358.65 (15.0%)  368.93 (10.7%)
> 2.9% ( -19% -   33%)
>   AndHighMed  799.65 (13.2%)  834.74  (7.8%)
> 4.4% ( -14% -   29%)
>  MedSloppyPhrase  229.36 (15.9%)  239.95  (9.8%)
> 4.6% ( -18% -   36%)
>   Fuzzy2   69.58 (14.6%)   72.82 (14.5%)
> 4.7% ( -21% -   39%)
>  AndHighHigh  426.98 (12.8%)  451.77  (7.3%)
> 5.8% ( -12% -   29%)
>  MedTerm 1361.11 (14.5%) 1450.90  (9.2%)
> 6.6% ( -14% -   35%)
> PKLookup  266.61 (13.4%)  284.28  (8.4%)
> 6.6% ( -13% -   32%)
> HighSloppyPhrase  251.22 (16.9%)  268.32 (10.7%)
> 6.8% ( -17% -   41%)
>OrHighMed  235.92 (17.2%)  253.12 (12.8%)
> 7.3% ( -19% -   45%)
>   OrHighHigh  186.79 (13.5%)  201.15  (9.7%)
> 7.7% ( -13% -   35%)
>  LowSloppyPhrase  395.23 (15.9%)  425.93  (9.3%)
> 7.8% ( -15% -   39%)
>   AndHighLo

[jira] [Commented] (SOLR-9092) Add safety checks to delete replica/shard/collection commands

[Shalin Shekhar Mangar (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9092?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15419305#comment-15419305
 ] 

Shalin Shekhar Mangar commented on SOLR-9092:
-

+1 to the patch. To summarise, we always delete the replica from the cluster 
state. But we attempt to send the core admin delete request only if that node 
is actually up. No new parameters are introduced.

> Add safety checks to delete replica/shard/collection commands
> -
>
> Key: SOLR-9092
> URL: https://issues.apache.org/jira/browse/SOLR-9092
> Project: Solr
>  Issue Type: Improvement
>Reporter: Varun Thacker
>Assignee: Varun Thacker
>Priority: Minor
> Attachments: SOLR-9092.patch
>
>
> We should verify the delete commands against live_nodes to make sure the API 
> can atleast be executed correctly
> If we have a two node cluster, a collection with 1 shard 2 replica. Call the 
> delete replica command against for the replica whose node is currently down.
> You get an exception:
> {code}
> 
>
>   0
>   5173
>
>
>name="192.168.1.101:7574_solr">org.apache.solr.client.solrj.SolrServerException:Server
>  refused connection at: http://192.168.1.101:7574/solr
>
> 
> {code}
> At this point the entry for the replica is gone from state.json . The client 
> application retries since an error was thrown but the delete command will 
> never succeed now and an error like this will be seen-
> {code}
> 
>
>   400
>   137
>
>org.apache.solr.common.SolrException:org.apache.solr.common.SolrException:
>  Invalid replica : core_node3 in shard/collection : shard1/gettingstarted 
> available replicas are core_node1
>
>   Invalid replica : core_node3 in shard/collection : 
> shard1/gettingstarted available replicas are core_node1
>   400
>
>
>   
>  org.apache.solr.common.SolrException
>   name="root-error-class">org.apache.solr.common.SolrException
>   
>   Invalid replica : core_node3 in shard/collection : 
> shard1/gettingstarted available replicas are core_node1
>   400
>
> 
> {code}
> For create collection/add-replica we check the "createNodeSet" and "node" 
> params respectively against live_nodes to make sure it has a chance of 
> succeeding.
> We should add a check against live_nodes for the delete commands as well.
> Another situation where I saw this can be a problem - A second solr cluster 
> cloned from the first but the script didn't correctly change the hostnames in 
> the state.json file. When a delete command was issued against the second 
> cluster Solr deleted the replica from the first cluster.
> In the above case the script was buggy obviously but if we verify against 
> live_nodes then Solr wouldn't have gone ahead and deleted replicas not 
> belonging to its cluster.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-master-MacOSX (64bit/jdk1.8.0) - Build # 3472 - Unstable!

[Policeman Jenkins Server]

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-master-MacOSX/3472/
Java: 64bit/jdk1.8.0 -XX:-UseCompressedOops -XX:+UseConcMarkSweepGC

2 tests failed.
FAILED:  org.apache.solr.cloud.CustomCollectionTest.test

Error Message:
Error from server at http://127.0.0.1:63808: collection already exists: 
implicitcollwithShardField

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error 
from server at http://127.0.0.1:63808: collection already exists: 
implicitcollwithShardField
at 
__randomizedtesting.SeedInfo.seed([E140F475D91705AF:6914CBAF77EB6857]:0)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:608)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:261)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:250)
at 
org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:413)
at 
org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:366)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1291)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:1061)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:997)
at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
at 
org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1599)
at 
org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1553)
at 
org.apache.solr.cloud.CustomCollectionTest.testCustomCollectionsAPI(CustomCollectionTest.java:285)
at 
org.apache.solr.cloud.CustomCollectionTest.test(CustomCollectionTest.java:99)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:985)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:960)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
   

[jira] [Updated] (LUCENE-7409) Look into making mmapdirectory's unmap safer

[Uwe Schindler (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-7409?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Uwe Schindler updated LUCENE-7409:
--
   Labels: crush performance  (was: )
Fix Version/s: 6.2
   master (7.0)
  Component/s: core/store

> Look into making mmapdirectory's unmap safer
> 
>
> Key: LUCENE-7409
> URL: https://issues.apache.org/jira/browse/LUCENE-7409
> Project: Lucene - Core
>  Issue Type: Improvement
>  Components: core/store
>Reporter: Robert Muir
>Assignee: Uwe Schindler
>  Labels: crush, performance
> Fix For: master (7.0), 6.2
>
> Attachments: LUCENE-7409.patch, StoreStore.java
>
>
> I have seen a few bugs around this recently: of course its a bug in 
> application code but a JVM crash is not good. 
> I think we should see if we can prevent the crashes better than the current 
> weak map, e.g. make it a safer option.
> I made an ugly prototype here: 
> https://github.com/apache/lucene-solr/compare/master...rmuir:ace?expand=1
> It has a test that crashes the JVM without the patch but passes with.
> Hacky patch only implements readBytes() but has no problems with the 
> luceneutil benchmark (1M):
> {noformat}
> Report after iter 19:
> TaskQPS base  StdDev   QPS patch  StdDev  
>   Pct diff
>   IntNRQ  105.23 (17.6%)  100.42 (10.1%)   
> -4.6% ( -27% -   28%)
>  Respell  128.35 (13.2%)  125.88  (7.4%)   
> -1.9% ( -19% -   21%)
>   Fuzzy1  110.14 (17.2%)  108.28 (13.2%)   
> -1.7% ( -27% -   34%)
>LowPhrase  337.02 (13.0%)  333.72  (9.3%)   
> -1.0% ( -20% -   24%)
>MedPhrase  146.44 (12.9%)  145.55  (8.0%)   
> -0.6% ( -19% -   23%)
>  MedSpanNear   96.85 (13.1%)   96.57  (7.8%)   
> -0.3% ( -18% -   23%)
> HighSpanNear   95.85 (13.9%)   96.33  (8.2%)
> 0.5% ( -18% -   26%)
>   HighPhrase  146.84 (13.6%)  148.40  (8.4%)
> 1.1% ( -18% -   26%)
> HighTerm  295.15 (15.8%)  298.77  (9.5%)
> 1.2% ( -20% -   31%)
>  LowSpanNear  268.80 (12.4%)  272.16  (7.9%)
> 1.2% ( -16% -   24%)
> Wildcard  284.09 (11.7%)  290.91  (8.9%)
> 2.4% ( -16% -   25%)
>  Prefix3  212.50 (15.4%)  217.76 (10.0%)
> 2.5% ( -19% -   32%)
>OrHighLow  358.65 (15.0%)  368.93 (10.7%)
> 2.9% ( -19% -   33%)
>   AndHighMed  799.65 (13.2%)  834.74  (7.8%)
> 4.4% ( -14% -   29%)
>  MedSloppyPhrase  229.36 (15.9%)  239.95  (9.8%)
> 4.6% ( -18% -   36%)
>   Fuzzy2   69.58 (14.6%)   72.82 (14.5%)
> 4.7% ( -21% -   39%)
>  AndHighHigh  426.98 (12.8%)  451.77  (7.3%)
> 5.8% ( -12% -   29%)
>  MedTerm 1361.11 (14.5%) 1450.90  (9.2%)
> 6.6% ( -14% -   35%)
> PKLookup  266.61 (13.4%)  284.28  (8.4%)
> 6.6% ( -13% -   32%)
> HighSloppyPhrase  251.22 (16.9%)  268.32 (10.7%)
> 6.8% ( -17% -   41%)
>OrHighMed  235.92 (17.2%)  253.12 (12.8%)
> 7.3% ( -19% -   45%)
>   OrHighHigh  186.79 (13.5%)  201.15  (9.7%)
> 7.7% ( -13% -   35%)
>  LowSloppyPhrase  395.23 (15.9%)  425.93  (9.3%)
> 7.8% ( -15% -   39%)
>   AndHighLow 1128.28 (14.9%) 1242.11  (8.2%)   
> 10.1% ( -11% -   38%)
>  LowTerm 3024.62 (12.9%) 3367.65  (9.7%)   
> 11.3% (  -9% -   39%)
> {noformat}
> We should do more testing. Maybe its totally the wrong tradeoff, maybe we 
> only need handles for getters and everything inlines correctly, rather than 
> needing a ton for every getXYZ() method...



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7409) Look into making mmapdirectory's unmap safer

[Uwe Schindler (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15419216#comment-15419216
 ] 

Uwe Schindler commented on LUCENE-7409:
---

I keep this issue open for a while with status "in progress" to not forget to 
disable the test after a while. I want to run it on master several times with 
various Jenkins machines.

> Look into making mmapdirectory's unmap safer
> 
>
> Key: LUCENE-7409
> URL: https://issues.apache.org/jira/browse/LUCENE-7409
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Robert Muir
>Assignee: Uwe Schindler
> Attachments: LUCENE-7409.patch, StoreStore.java
>
>
> I have seen a few bugs around this recently: of course its a bug in 
> application code but a JVM crash is not good. 
> I think we should see if we can prevent the crashes better than the current 
> weak map, e.g. make it a safer option.
> I made an ugly prototype here: 
> https://github.com/apache/lucene-solr/compare/master...rmuir:ace?expand=1
> It has a test that crashes the JVM without the patch but passes with.
> Hacky patch only implements readBytes() but has no problems with the 
> luceneutil benchmark (1M):
> {noformat}
> Report after iter 19:
> TaskQPS base  StdDev   QPS patch  StdDev  
>   Pct diff
>   IntNRQ  105.23 (17.6%)  100.42 (10.1%)   
> -4.6% ( -27% -   28%)
>  Respell  128.35 (13.2%)  125.88  (7.4%)   
> -1.9% ( -19% -   21%)
>   Fuzzy1  110.14 (17.2%)  108.28 (13.2%)   
> -1.7% ( -27% -   34%)
>LowPhrase  337.02 (13.0%)  333.72  (9.3%)   
> -1.0% ( -20% -   24%)
>MedPhrase  146.44 (12.9%)  145.55  (8.0%)   
> -0.6% ( -19% -   23%)
>  MedSpanNear   96.85 (13.1%)   96.57  (7.8%)   
> -0.3% ( -18% -   23%)
> HighSpanNear   95.85 (13.9%)   96.33  (8.2%)
> 0.5% ( -18% -   26%)
>   HighPhrase  146.84 (13.6%)  148.40  (8.4%)
> 1.1% ( -18% -   26%)
> HighTerm  295.15 (15.8%)  298.77  (9.5%)
> 1.2% ( -20% -   31%)
>  LowSpanNear  268.80 (12.4%)  272.16  (7.9%)
> 1.2% ( -16% -   24%)
> Wildcard  284.09 (11.7%)  290.91  (8.9%)
> 2.4% ( -16% -   25%)
>  Prefix3  212.50 (15.4%)  217.76 (10.0%)
> 2.5% ( -19% -   32%)
>OrHighLow  358.65 (15.0%)  368.93 (10.7%)
> 2.9% ( -19% -   33%)
>   AndHighMed  799.65 (13.2%)  834.74  (7.8%)
> 4.4% ( -14% -   29%)
>  MedSloppyPhrase  229.36 (15.9%)  239.95  (9.8%)
> 4.6% ( -18% -   36%)
>   Fuzzy2   69.58 (14.6%)   72.82 (14.5%)
> 4.7% ( -21% -   39%)
>  AndHighHigh  426.98 (12.8%)  451.77  (7.3%)
> 5.8% ( -12% -   29%)
>  MedTerm 1361.11 (14.5%) 1450.90  (9.2%)
> 6.6% ( -14% -   35%)
> PKLookup  266.61 (13.4%)  284.28  (8.4%)
> 6.6% ( -13% -   32%)
> HighSloppyPhrase  251.22 (16.9%)  268.32 (10.7%)
> 6.8% ( -17% -   41%)
>OrHighMed  235.92 (17.2%)  253.12 (12.8%)
> 7.3% ( -19% -   45%)
>   OrHighHigh  186.79 (13.5%)  201.15  (9.7%)
> 7.7% ( -13% -   35%)
>  LowSloppyPhrase  395.23 (15.9%)  425.93  (9.3%)
> 7.8% ( -15% -   39%)
>   AndHighLow 1128.28 (14.9%) 1242.11  (8.2%)   
> 10.1% ( -11% -   38%)
>  LowTerm 3024.62 (12.9%) 3367.65  (9.7%)   
> 11.3% (  -9% -   39%)
> {noformat}
> We should do more testing. Maybe its totally the wrong tradeoff, maybe we 
> only need handles for getters and everything inlines correctly, rather than 
> needing a ton for every getXYZ() method...



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7409) Look into making mmapdirectory's unmap safer

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15419212#comment-15419212
 ] 

ASF subversion and git services commented on LUCENE-7409:
-

Commit 4ae13ef31a0f18f119c119997687b4f6e6bfd8f0 in lucene-solr's branch 
refs/heads/branch_6x from [~thetaphi]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=4ae13ef ]

LUCENE-7409: Disable test on stable branch


> Look into making mmapdirectory's unmap safer
> 
>
> Key: LUCENE-7409
> URL: https://issues.apache.org/jira/browse/LUCENE-7409
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Robert Muir
>Assignee: Uwe Schindler
> Attachments: LUCENE-7409.patch, StoreStore.java
>
>
> I have seen a few bugs around this recently: of course its a bug in 
> application code but a JVM crash is not good. 
> I think we should see if we can prevent the crashes better than the current 
> weak map, e.g. make it a safer option.
> I made an ugly prototype here: 
> https://github.com/apache/lucene-solr/compare/master...rmuir:ace?expand=1
> It has a test that crashes the JVM without the patch but passes with.
> Hacky patch only implements readBytes() but has no problems with the 
> luceneutil benchmark (1M):
> {noformat}
> Report after iter 19:
> TaskQPS base  StdDev   QPS patch  StdDev  
>   Pct diff
>   IntNRQ  105.23 (17.6%)  100.42 (10.1%)   
> -4.6% ( -27% -   28%)
>  Respell  128.35 (13.2%)  125.88  (7.4%)   
> -1.9% ( -19% -   21%)
>   Fuzzy1  110.14 (17.2%)  108.28 (13.2%)   
> -1.7% ( -27% -   34%)
>LowPhrase  337.02 (13.0%)  333.72  (9.3%)   
> -1.0% ( -20% -   24%)
>MedPhrase  146.44 (12.9%)  145.55  (8.0%)   
> -0.6% ( -19% -   23%)
>  MedSpanNear   96.85 (13.1%)   96.57  (7.8%)   
> -0.3% ( -18% -   23%)
> HighSpanNear   95.85 (13.9%)   96.33  (8.2%)
> 0.5% ( -18% -   26%)
>   HighPhrase  146.84 (13.6%)  148.40  (8.4%)
> 1.1% ( -18% -   26%)
> HighTerm  295.15 (15.8%)  298.77  (9.5%)
> 1.2% ( -20% -   31%)
>  LowSpanNear  268.80 (12.4%)  272.16  (7.9%)
> 1.2% ( -16% -   24%)
> Wildcard  284.09 (11.7%)  290.91  (8.9%)
> 2.4% ( -16% -   25%)
>  Prefix3  212.50 (15.4%)  217.76 (10.0%)
> 2.5% ( -19% -   32%)
>OrHighLow  358.65 (15.0%)  368.93 (10.7%)
> 2.9% ( -19% -   33%)
>   AndHighMed  799.65 (13.2%)  834.74  (7.8%)
> 4.4% ( -14% -   29%)
>  MedSloppyPhrase  229.36 (15.9%)  239.95  (9.8%)
> 4.6% ( -18% -   36%)
>   Fuzzy2   69.58 (14.6%)   72.82 (14.5%)
> 4.7% ( -21% -   39%)
>  AndHighHigh  426.98 (12.8%)  451.77  (7.3%)
> 5.8% ( -12% -   29%)
>  MedTerm 1361.11 (14.5%) 1450.90  (9.2%)
> 6.6% ( -14% -   35%)
> PKLookup  266.61 (13.4%)  284.28  (8.4%)
> 6.6% ( -13% -   32%)
> HighSloppyPhrase  251.22 (16.9%)  268.32 (10.7%)
> 6.8% ( -17% -   41%)
>OrHighMed  235.92 (17.2%)  253.12 (12.8%)
> 7.3% ( -19% -   45%)
>   OrHighHigh  186.79 (13.5%)  201.15  (9.7%)
> 7.7% ( -13% -   35%)
>  LowSloppyPhrase  395.23 (15.9%)  425.93  (9.3%)
> 7.8% ( -15% -   39%)
>   AndHighLow 1128.28 (14.9%) 1242.11  (8.2%)   
> 10.1% ( -11% -   38%)
>  LowTerm 3024.62 (12.9%) 3367.65  (9.7%)   
> 11.3% (  -9% -   39%)
> {noformat}
> We should do more testing. Maybe its totally the wrong tradeoff, maybe we 
> only need handles for getters and everything inlines correctly, rather than 
> needing a ton for every getXYZ() method...



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7409) Look into making mmapdirectory's unmap safer

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15419211#comment-15419211
 ] 

ASF subversion and git services commented on LUCENE-7409:
-

Commit f31ec6dd64b68bd947d6c82d9d5984af5b5d9688 in lucene-solr's branch 
refs/heads/branch_6x from [~thetaphi]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=f31ec6d ]

LUCENE-7409: Changed MMapDirectory's unmapping to work safer, but still with no 
guarantees


> Look into making mmapdirectory's unmap safer
> 
>
> Key: LUCENE-7409
> URL: https://issues.apache.org/jira/browse/LUCENE-7409
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Robert Muir
>Assignee: Uwe Schindler
> Attachments: LUCENE-7409.patch, StoreStore.java
>
>
> I have seen a few bugs around this recently: of course its a bug in 
> application code but a JVM crash is not good. 
> I think we should see if we can prevent the crashes better than the current 
> weak map, e.g. make it a safer option.
> I made an ugly prototype here: 
> https://github.com/apache/lucene-solr/compare/master...rmuir:ace?expand=1
> It has a test that crashes the JVM without the patch but passes with.
> Hacky patch only implements readBytes() but has no problems with the 
> luceneutil benchmark (1M):
> {noformat}
> Report after iter 19:
> TaskQPS base  StdDev   QPS patch  StdDev  
>   Pct diff
>   IntNRQ  105.23 (17.6%)  100.42 (10.1%)   
> -4.6% ( -27% -   28%)
>  Respell  128.35 (13.2%)  125.88  (7.4%)   
> -1.9% ( -19% -   21%)
>   Fuzzy1  110.14 (17.2%)  108.28 (13.2%)   
> -1.7% ( -27% -   34%)
>LowPhrase  337.02 (13.0%)  333.72  (9.3%)   
> -1.0% ( -20% -   24%)
>MedPhrase  146.44 (12.9%)  145.55  (8.0%)   
> -0.6% ( -19% -   23%)
>  MedSpanNear   96.85 (13.1%)   96.57  (7.8%)   
> -0.3% ( -18% -   23%)
> HighSpanNear   95.85 (13.9%)   96.33  (8.2%)
> 0.5% ( -18% -   26%)
>   HighPhrase  146.84 (13.6%)  148.40  (8.4%)
> 1.1% ( -18% -   26%)
> HighTerm  295.15 (15.8%)  298.77  (9.5%)
> 1.2% ( -20% -   31%)
>  LowSpanNear  268.80 (12.4%)  272.16  (7.9%)
> 1.2% ( -16% -   24%)
> Wildcard  284.09 (11.7%)  290.91  (8.9%)
> 2.4% ( -16% -   25%)
>  Prefix3  212.50 (15.4%)  217.76 (10.0%)
> 2.5% ( -19% -   32%)
>OrHighLow  358.65 (15.0%)  368.93 (10.7%)
> 2.9% ( -19% -   33%)
>   AndHighMed  799.65 (13.2%)  834.74  (7.8%)
> 4.4% ( -14% -   29%)
>  MedSloppyPhrase  229.36 (15.9%)  239.95  (9.8%)
> 4.6% ( -18% -   36%)
>   Fuzzy2   69.58 (14.6%)   72.82 (14.5%)
> 4.7% ( -21% -   39%)
>  AndHighHigh  426.98 (12.8%)  451.77  (7.3%)
> 5.8% ( -12% -   29%)
>  MedTerm 1361.11 (14.5%) 1450.90  (9.2%)
> 6.6% ( -14% -   35%)
> PKLookup  266.61 (13.4%)  284.28  (8.4%)
> 6.6% ( -13% -   32%)
> HighSloppyPhrase  251.22 (16.9%)  268.32 (10.7%)
> 6.8% ( -17% -   41%)
>OrHighMed  235.92 (17.2%)  253.12 (12.8%)
> 7.3% ( -19% -   45%)
>   OrHighHigh  186.79 (13.5%)  201.15  (9.7%)
> 7.7% ( -13% -   35%)
>  LowSloppyPhrase  395.23 (15.9%)  425.93  (9.3%)
> 7.8% ( -15% -   39%)
>   AndHighLow 1128.28 (14.9%) 1242.11  (8.2%)   
> 10.1% ( -11% -   38%)
>  LowTerm 3024.62 (12.9%) 3367.65  (9.7%)   
> 11.3% (  -9% -   39%)
> {noformat}
> We should do more testing. Maybe its totally the wrong tradeoff, maybe we 
> only need handles for getters and everything inlines correctly, rather than 
> needing a ton for every getXYZ() method...



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7409) Look into making mmapdirectory's unmap safer

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15419206#comment-15419206
 ] 

ASF subversion and git services commented on LUCENE-7409:
-

Commit 48cc5999369a1f99af159aa5eb756f5c6f118594 in lucene-solr's branch 
refs/heads/master from [~thetaphi]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=48cc599 ]

LUCENE-7409: Changed MMapDirectory's unmapping to work safer, but still with no 
guarantees


> Look into making mmapdirectory's unmap safer
> 
>
> Key: LUCENE-7409
> URL: https://issues.apache.org/jira/browse/LUCENE-7409
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Robert Muir
>Assignee: Uwe Schindler
> Attachments: LUCENE-7409.patch, StoreStore.java
>
>
> I have seen a few bugs around this recently: of course its a bug in 
> application code but a JVM crash is not good. 
> I think we should see if we can prevent the crashes better than the current 
> weak map, e.g. make it a safer option.
> I made an ugly prototype here: 
> https://github.com/apache/lucene-solr/compare/master...rmuir:ace?expand=1
> It has a test that crashes the JVM without the patch but passes with.
> Hacky patch only implements readBytes() but has no problems with the 
> luceneutil benchmark (1M):
> {noformat}
> Report after iter 19:
> TaskQPS base  StdDev   QPS patch  StdDev  
>   Pct diff
>   IntNRQ  105.23 (17.6%)  100.42 (10.1%)   
> -4.6% ( -27% -   28%)
>  Respell  128.35 (13.2%)  125.88  (7.4%)   
> -1.9% ( -19% -   21%)
>   Fuzzy1  110.14 (17.2%)  108.28 (13.2%)   
> -1.7% ( -27% -   34%)
>LowPhrase  337.02 (13.0%)  333.72  (9.3%)   
> -1.0% ( -20% -   24%)
>MedPhrase  146.44 (12.9%)  145.55  (8.0%)   
> -0.6% ( -19% -   23%)
>  MedSpanNear   96.85 (13.1%)   96.57  (7.8%)   
> -0.3% ( -18% -   23%)
> HighSpanNear   95.85 (13.9%)   96.33  (8.2%)
> 0.5% ( -18% -   26%)
>   HighPhrase  146.84 (13.6%)  148.40  (8.4%)
> 1.1% ( -18% -   26%)
> HighTerm  295.15 (15.8%)  298.77  (9.5%)
> 1.2% ( -20% -   31%)
>  LowSpanNear  268.80 (12.4%)  272.16  (7.9%)
> 1.2% ( -16% -   24%)
> Wildcard  284.09 (11.7%)  290.91  (8.9%)
> 2.4% ( -16% -   25%)
>  Prefix3  212.50 (15.4%)  217.76 (10.0%)
> 2.5% ( -19% -   32%)
>OrHighLow  358.65 (15.0%)  368.93 (10.7%)
> 2.9% ( -19% -   33%)
>   AndHighMed  799.65 (13.2%)  834.74  (7.8%)
> 4.4% ( -14% -   29%)
>  MedSloppyPhrase  229.36 (15.9%)  239.95  (9.8%)
> 4.6% ( -18% -   36%)
>   Fuzzy2   69.58 (14.6%)   72.82 (14.5%)
> 4.7% ( -21% -   39%)
>  AndHighHigh  426.98 (12.8%)  451.77  (7.3%)
> 5.8% ( -12% -   29%)
>  MedTerm 1361.11 (14.5%) 1450.90  (9.2%)
> 6.6% ( -14% -   35%)
> PKLookup  266.61 (13.4%)  284.28  (8.4%)
> 6.6% ( -13% -   32%)
> HighSloppyPhrase  251.22 (16.9%)  268.32 (10.7%)
> 6.8% ( -17% -   41%)
>OrHighMed  235.92 (17.2%)  253.12 (12.8%)
> 7.3% ( -19% -   45%)
>   OrHighHigh  186.79 (13.5%)  201.15  (9.7%)
> 7.7% ( -13% -   35%)
>  LowSloppyPhrase  395.23 (15.9%)  425.93  (9.3%)
> 7.8% ( -15% -   39%)
>   AndHighLow 1128.28 (14.9%) 1242.11  (8.2%)   
> 10.1% ( -11% -   38%)
>  LowTerm 3024.62 (12.9%) 3367.65  (9.7%)   
> 11.3% (  -9% -   39%)
> {noformat}
> We should do more testing. Maybe its totally the wrong tradeoff, maybe we 
> only need handles for getters and everything inlines correctly, rather than 
> needing a ton for every getXYZ() method...



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-7409) Look into making mmapdirectory's unmap safer

[Uwe Schindler (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-7409?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Uwe Schindler updated LUCENE-7409:
--
Attachment: LUCENE-7409.patch

Final patch. Committing in a moment!

> Look into making mmapdirectory's unmap safer
> 
>
> Key: LUCENE-7409
> URL: https://issues.apache.org/jira/browse/LUCENE-7409
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Robert Muir
>Assignee: Uwe Schindler
> Attachments: LUCENE-7409.patch, StoreStore.java
>
>
> I have seen a few bugs around this recently: of course its a bug in 
> application code but a JVM crash is not good. 
> I think we should see if we can prevent the crashes better than the current 
> weak map, e.g. make it a safer option.
> I made an ugly prototype here: 
> https://github.com/apache/lucene-solr/compare/master...rmuir:ace?expand=1
> It has a test that crashes the JVM without the patch but passes with.
> Hacky patch only implements readBytes() but has no problems with the 
> luceneutil benchmark (1M):
> {noformat}
> Report after iter 19:
> TaskQPS base  StdDev   QPS patch  StdDev  
>   Pct diff
>   IntNRQ  105.23 (17.6%)  100.42 (10.1%)   
> -4.6% ( -27% -   28%)
>  Respell  128.35 (13.2%)  125.88  (7.4%)   
> -1.9% ( -19% -   21%)
>   Fuzzy1  110.14 (17.2%)  108.28 (13.2%)   
> -1.7% ( -27% -   34%)
>LowPhrase  337.02 (13.0%)  333.72  (9.3%)   
> -1.0% ( -20% -   24%)
>MedPhrase  146.44 (12.9%)  145.55  (8.0%)   
> -0.6% ( -19% -   23%)
>  MedSpanNear   96.85 (13.1%)   96.57  (7.8%)   
> -0.3% ( -18% -   23%)
> HighSpanNear   95.85 (13.9%)   96.33  (8.2%)
> 0.5% ( -18% -   26%)
>   HighPhrase  146.84 (13.6%)  148.40  (8.4%)
> 1.1% ( -18% -   26%)
> HighTerm  295.15 (15.8%)  298.77  (9.5%)
> 1.2% ( -20% -   31%)
>  LowSpanNear  268.80 (12.4%)  272.16  (7.9%)
> 1.2% ( -16% -   24%)
> Wildcard  284.09 (11.7%)  290.91  (8.9%)
> 2.4% ( -16% -   25%)
>  Prefix3  212.50 (15.4%)  217.76 (10.0%)
> 2.5% ( -19% -   32%)
>OrHighLow  358.65 (15.0%)  368.93 (10.7%)
> 2.9% ( -19% -   33%)
>   AndHighMed  799.65 (13.2%)  834.74  (7.8%)
> 4.4% ( -14% -   29%)
>  MedSloppyPhrase  229.36 (15.9%)  239.95  (9.8%)
> 4.6% ( -18% -   36%)
>   Fuzzy2   69.58 (14.6%)   72.82 (14.5%)
> 4.7% ( -21% -   39%)
>  AndHighHigh  426.98 (12.8%)  451.77  (7.3%)
> 5.8% ( -12% -   29%)
>  MedTerm 1361.11 (14.5%) 1450.90  (9.2%)
> 6.6% ( -14% -   35%)
> PKLookup  266.61 (13.4%)  284.28  (8.4%)
> 6.6% ( -13% -   32%)
> HighSloppyPhrase  251.22 (16.9%)  268.32 (10.7%)
> 6.8% ( -17% -   41%)
>OrHighMed  235.92 (17.2%)  253.12 (12.8%)
> 7.3% ( -19% -   45%)
>   OrHighHigh  186.79 (13.5%)  201.15  (9.7%)
> 7.7% ( -13% -   35%)
>  LowSloppyPhrase  395.23 (15.9%)  425.93  (9.3%)
> 7.8% ( -15% -   39%)
>   AndHighLow 1128.28 (14.9%) 1242.11  (8.2%)   
> 10.1% ( -11% -   38%)
>  LowTerm 3024.62 (12.9%) 3367.65  (9.7%)   
> 11.3% (  -9% -   39%)
> {noformat}
> We should do more testing. Maybe its totally the wrong tradeoff, maybe we 
> only need handles for getters and everything inlines correctly, rather than 
> needing a ton for every getXYZ() method...



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7409) Look into making mmapdirectory's unmap safer

[Uwe Schindler (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15419194#comment-15419194
 ] 

Uwe Schindler commented on LUCENE-7409:
---

I committed a {{@Ignore}} with explanation to the test, but I commented it out. 
I'd like to run it a while on Jenkins, especially Multi-CPU machines like the 
Apache or Policeman one. Multi-CPU machines are morke likely to crush.

> Look into making mmapdirectory's unmap safer
> 
>
> Key: LUCENE-7409
> URL: https://issues.apache.org/jira/browse/LUCENE-7409
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Robert Muir
>Assignee: Uwe Schindler
> Attachments: StoreStore.java
>
>
> I have seen a few bugs around this recently: of course its a bug in 
> application code but a JVM crash is not good. 
> I think we should see if we can prevent the crashes better than the current 
> weak map, e.g. make it a safer option.
> I made an ugly prototype here: 
> https://github.com/apache/lucene-solr/compare/master...rmuir:ace?expand=1
> It has a test that crashes the JVM without the patch but passes with.
> Hacky patch only implements readBytes() but has no problems with the 
> luceneutil benchmark (1M):
> {noformat}
> Report after iter 19:
> TaskQPS base  StdDev   QPS patch  StdDev  
>   Pct diff
>   IntNRQ  105.23 (17.6%)  100.42 (10.1%)   
> -4.6% ( -27% -   28%)
>  Respell  128.35 (13.2%)  125.88  (7.4%)   
> -1.9% ( -19% -   21%)
>   Fuzzy1  110.14 (17.2%)  108.28 (13.2%)   
> -1.7% ( -27% -   34%)
>LowPhrase  337.02 (13.0%)  333.72  (9.3%)   
> -1.0% ( -20% -   24%)
>MedPhrase  146.44 (12.9%)  145.55  (8.0%)   
> -0.6% ( -19% -   23%)
>  MedSpanNear   96.85 (13.1%)   96.57  (7.8%)   
> -0.3% ( -18% -   23%)
> HighSpanNear   95.85 (13.9%)   96.33  (8.2%)
> 0.5% ( -18% -   26%)
>   HighPhrase  146.84 (13.6%)  148.40  (8.4%)
> 1.1% ( -18% -   26%)
> HighTerm  295.15 (15.8%)  298.77  (9.5%)
> 1.2% ( -20% -   31%)
>  LowSpanNear  268.80 (12.4%)  272.16  (7.9%)
> 1.2% ( -16% -   24%)
> Wildcard  284.09 (11.7%)  290.91  (8.9%)
> 2.4% ( -16% -   25%)
>  Prefix3  212.50 (15.4%)  217.76 (10.0%)
> 2.5% ( -19% -   32%)
>OrHighLow  358.65 (15.0%)  368.93 (10.7%)
> 2.9% ( -19% -   33%)
>   AndHighMed  799.65 (13.2%)  834.74  (7.8%)
> 4.4% ( -14% -   29%)
>  MedSloppyPhrase  229.36 (15.9%)  239.95  (9.8%)
> 4.6% ( -18% -   36%)
>   Fuzzy2   69.58 (14.6%)   72.82 (14.5%)
> 4.7% ( -21% -   39%)
>  AndHighHigh  426.98 (12.8%)  451.77  (7.3%)
> 5.8% ( -12% -   29%)
>  MedTerm 1361.11 (14.5%) 1450.90  (9.2%)
> 6.6% ( -14% -   35%)
> PKLookup  266.61 (13.4%)  284.28  (8.4%)
> 6.6% ( -13% -   32%)
> HighSloppyPhrase  251.22 (16.9%)  268.32 (10.7%)
> 6.8% ( -17% -   41%)
>OrHighMed  235.92 (17.2%)  253.12 (12.8%)
> 7.3% ( -19% -   45%)
>   OrHighHigh  186.79 (13.5%)  201.15  (9.7%)
> 7.7% ( -13% -   35%)
>  LowSloppyPhrase  395.23 (15.9%)  425.93  (9.3%)
> 7.8% ( -15% -   39%)
>   AndHighLow 1128.28 (14.9%) 1242.11  (8.2%)   
> 10.1% ( -11% -   38%)
>  LowTerm 3024.62 (12.9%) 3367.65  (9.7%)   
> 11.3% (  -9% -   39%)
> {noformat}
> We should do more testing. Maybe its totally the wrong tradeoff, maybe we 
> only need handles for getters and everything inlines correctly, rather than 
> needing a ton for every getXYZ() method...



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS-EA] Lucene-Solr-master-Linux (32bit/jdk-9-ea+129) - Build # 17543 - Still Unstable!

[Policeman Jenkins Server]

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/17543/
Java: 32bit/jdk-9-ea+129 -server -XX:+UseSerialGC

1 tests failed.
FAILED:  org.apache.solr.security.BasicAuthIntegrationTest.testBasics

Error Message:
IOException occured when talking to server at: 
http://127.0.0.1:37664/solr/testSolrCloudCollection_shard1_replica1

Stack Trace:
org.apache.solr.client.solrj.impl.CloudSolrClient$RouteException: IOException 
occured when talking to server at: 
http://127.0.0.1:37664/solr/testSolrCloudCollection_shard1_replica1
at 
__randomizedtesting.SeedInfo.seed([7F42B58E7E908F49:429A1BA2467ED139]:0)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.directUpdate(CloudSolrClient.java:760)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1172)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:1061)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:997)
at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
at 
org.apache.solr.security.BasicAuthIntegrationTest.doExtraTests(BasicAuthIntegrationTest.java:193)
at 
org.apache.solr.cloud.TestMiniSolrCloudClusterBase.testCollectionCreateSearchDelete(TestMiniSolrCloudClusterBase.java:196)
at 
org.apache.solr.cloud.TestMiniSolrCloudClusterBase.testBasics(TestMiniSolrCloudClusterBase.java:79)
at 
jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(java.base@9-ea/Native 
Method)
at 
jdk.internal.reflect.NativeMethodAccessorImpl.invoke(java.base@9-ea/NativeMethodAccessorImpl.java:62)
at 
jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(java.base@9-ea/DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(java.base@9-ea/Method.java:533)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertions

[jira] [Commented] (SOLR-8586) Implement hash over all documents to check for shard synchronization

[Yago Riveiro (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-8586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15419184#comment-15419184
 ] 

Yago Riveiro commented on SOLR-8586:


Then I do not understand, how this is possible:

https://www.dropbox.com/s/a6e2wrmedop7xjv/Screenshot%202016-08-12%2018.19.22.png?dl=0

Only with 5.5.x and 6.x the heap grows to the infinite. Rolling back to 5.4 the 
amount of memory needed to become up is constant ...

With only one node running 5.5.x I have no problems, when I start a second node 
with 5.5.x they never pass the phase where they are checking replica 
synchronization.

> Implement hash over all documents to check for shard synchronization
> 
>
> Key: SOLR-8586
> URL: https://issues.apache.org/jira/browse/SOLR-8586
> Project: Solr
>  Issue Type: Improvement
>  Components: SolrCloud
>Reporter: Yonik Seeley
>Assignee: Yonik Seeley
> Fix For: 5.5, 6.0
>
> Attachments: SOLR-8586.patch, SOLR-8586.patch, SOLR-8586.patch, 
> SOLR-8586.patch
>
>
> An order-independent hash across all of the versions in the index should 
> suffice.  The hash itself is pretty easy, but we need to figure out 
> when/where to do this check (for example, I think PeerSync is currently used 
> in multiple contexts and this check would perhaps not be appropriate for all 
> PeerSync calls?)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-8586) Implement hash over all documents to check for shard synchronization

[Yonik Seeley (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-8586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15419145#comment-15419145
 ] 

Yonik Seeley commented on SOLR-8586:


You can set the environment variable solr.disableFingerprint to "false" to 
disable the fingerprint check.

If your indexes ever have updates to existing documents, then you're still 
risking OOMs anyway (the first time a replica detects that an update may be 
reordered will cause the FieldCache to be populated for _version_ for that 
segment).  The fingerprint makes that happen up-front (what I meant to say in 
my previous message was "the maximum required amount of memory shouldn't be 
changed").

> Implement hash over all documents to check for shard synchronization
> 
>
> Key: SOLR-8586
> URL: https://issues.apache.org/jira/browse/SOLR-8586
> Project: Solr
>  Issue Type: Improvement
>  Components: SolrCloud
>Reporter: Yonik Seeley
>Assignee: Yonik Seeley
> Fix For: 5.5, 6.0
>
> Attachments: SOLR-8586.patch, SOLR-8586.patch, SOLR-8586.patch, 
> SOLR-8586.patch
>
>
> An order-independent hash across all of the versions in the index should 
> suffice.  The hash itself is pretty easy, but we need to figure out 
> when/where to do this check (for example, I think PeerSync is currently used 
> in multiple contexts and this check would perhaps not be appropriate for all 
> PeerSync calls?)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

RE: JDK, Maven, Ant versions have been consolidated.

[Uwe Schindler]

Hi Gav,

 

I disabled all auto-running Jenkins Jobs of Lucene/Solr as this spammed the 
mailing list with like 50 mails today. Can you send me a msg when you installed 
the required puppet modules?

 

Uwe

 

-

Uwe Schindler

uschind...@apache.org 

ASF Member, Apache Lucene PMC / Committer

Bremen, Germany

http://lucene.apache.org/

 

From: Gav [mailto:gmcdon...@apache.org] 
Sent: Friday, August 12, 2016 11:18 AM
To: Uwe Schindler 
Cc: builds 
Subject: Re: JDK, Maven, Ant versions have been consolidated.

 

Hi Uwe,

Will get on with your vm tomorrow. 



Gav...

 

On Fri, Aug 12, 2016 at 6:39 PM, Uwe Schindler mailto:uschind...@apache.org> > wrote:

Hi,

on the lucene-jenkins machine, the packages are still missing. I think you have 
to install the jenkins puppet module. Here the message:
FATAL: Cannot find executable from the chosen Ant installation "Ant 1.8.4"

See my e-mail from last week about using the "official" jenkins puppet module 
(answer to your question). Unless our customizations in the home dir keep 
preserved, we are fine with that:

My mail:


Hi Gav,

sorry for responding late – I am currently on vacation!

About your questions: In the past we required some extra modules (like older 
Subversion to be compatible with the older Jenkins Checkouts). As we are on Git 
now and the build scripts use JGit to fetch revision numbers and check for 
clean checkouts, this is obsolete. So if you want to install the “Jenkins 
puppet module”, we are fine.

There is one special case for Lucene:
We have a hardware-specific “lucene.build.properties” file in Jenkin’s root and 
also some additional “test files” (large wikiepdia checkouts) that are used for 
nightly tests. If the puppet module does not delete those files in current 
~/jenkins, it is ok. We would still need access to the machine and sudo to 
jenkins.

If this is all fine, go ahead!

Uwe
===

Uwe

-
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: u...@thetaphi.de  


> -Original Message-
> From: Gav [mailto:gmcdon...@apache.org  ]
> Sent: Friday, August 12, 2016 6:43 AM
> To: builds mailto:bui...@apache.org> >
> Subject: JDK, Maven, Ant versions have been consolidated.
>
> HI All,
>
> As per previous emails and as documented [1] [2] [3] we have now
> consolidated the JDK, Maven and Ant install versions.
>
> All versions are now installed/managed via puppet and any 'auto-install'
> versions have been removed.
>
> At the time of consolidation and removal of versions, over 200 jobs were
> still configured to use deprecated versions despite the emails I've been
> sending. Those 200+ jobs I have migrated myself to supported versions.
>
> Any issues with your jobs please do create a new INFRA jira or email this
> list.
>
> Thanks,
>
> Gav...
>
> [1] -
> https://cwiki.apache.org/confluence/display/INFRA/JDK+Installation+Matrix
> [2] -
> https://cwiki.apache.org/confluence/display/INFRA/Ant+Installation+Matrix
> [3] -
> https://cwiki.apache.org/confluence/display/INFRA/Maven+Installation+Mat
> rix

 

[JENKINS] Lucene-Solr-Tests-6.x - Build # 413 - Still Failing

[Apache Jenkins Server]

Build: https://builds.apache.org/job/Lucene-Solr-Tests-6.x/413/

No tests ran.

Build Log:
[...truncated 21 lines...]
FATAL: Cannot find executable from the chosen Ant installation "Ant 1.8.4"
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
ERROR: Step ‘Publish JUnit test result report’ failed: No test report files 
were found. Configuration error?
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any




-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Comment Edited] (SOLR-8586) Implement hash over all documents to check for shard synchronization

[Yonik Seeley (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-8586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15418799#comment-15418799
 ] 

Yonik Seeley edited comment on SOLR-8586 at 8/12/16 4:38 PM:
-

If the _version_ field doesn't have docValues, then it will be un-inverted 
(i.e. FieldCache entries will be built to support _version_ lookups, and that 
does require memory).
Since _version_ lookups are needed in the course of indexing anyway (to detect 
update reorders on replicas), this should really just change when these 
FieldCache entries are created... hence the maximum required amount of memory 
shouldn't be changed.


was (Author: ysee...@gmail.com):
If the _version_ field doesn't have docValues, then it will be un-inverted 
(i.e. FieldCache entries will be built to support _version_ lookups, and that 
does require memory).
Since _version_ lookups are needed in the course of indexing anyway (to detect 
update reorders on replicas), this should really just change when these 
FieldCache entries are created... hence the maximum required amount of memory 
should be changed.

> Implement hash over all documents to check for shard synchronization
> 
>
> Key: SOLR-8586
> URL: https://issues.apache.org/jira/browse/SOLR-8586
> Project: Solr
>  Issue Type: Improvement
>  Components: SolrCloud
>Reporter: Yonik Seeley
>Assignee: Yonik Seeley
> Fix For: 5.5, 6.0
>
> Attachments: SOLR-8586.patch, SOLR-8586.patch, SOLR-8586.patch, 
> SOLR-8586.patch
>
>
> An order-independent hash across all of the versions in the index should 
> suffice.  The hash itself is pretty easy, but we need to figure out 
> when/where to do this check (for example, I think PeerSync is currently used 
> in multiple contexts and this check would perhaps not be appropriate for all 
> PeerSync calls?)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Comment Edited] (SOLR-8586) Implement hash over all documents to check for shard synchronization

[Yago Riveiro (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-8586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15419078#comment-15419078
 ] 

Yago Riveiro edited comment on SOLR-8586 at 8/12/16 4:23 PM:
-

My index has 12T of data indexed with 4.0, the _version_ field only supports 
docValues since 4.7.

To Upgrade to 5.x I ran the lucene-core-5.x over all my data,but with this new 
feature I need to re-index all my data because I don't have docValues for 
__version__ field and this feature use instead the un-inverted method that 
creates a memory struct that doesn't fit the memory of my servers ...

To be honest, this never should be done in a minor release ... this mandatory 
feature is based in a optional configuration :/

I will die in 5.4 or spend several months re-indexing data and figure out how 
to update production without downtime.  Not an easy task.




was (Author: yriveiro):
My index has 12T of data indexed with 4.0, the _version_ field only support 
docValues since 4.7.

To Upgrade to 5.x I ran the lucene-core-5.x over all my data,but with this new 
feature I need to re-index all my data because I don't have docValues for 
__version__ field and this feature use instead the un-inverted method that 
creates a memory struct that doesn't fit the memory of my servers ...

To be honest, this never should be done in a minor release ... this mandatory 
feature is based in a optional configuration :/

I will die in 5.4 or spend several months re-indexing data and figure out how 
to update production without downtime.  Not an easy task.



> Implement hash over all documents to check for shard synchronization
> 
>
> Key: SOLR-8586
> URL: https://issues.apache.org/jira/browse/SOLR-8586
> Project: Solr
>  Issue Type: Improvement
>  Components: SolrCloud
>Reporter: Yonik Seeley
>Assignee: Yonik Seeley
> Fix For: 5.5, 6.0
>
> Attachments: SOLR-8586.patch, SOLR-8586.patch, SOLR-8586.patch, 
> SOLR-8586.patch
>
>
> An order-independent hash across all of the versions in the index should 
> suffice.  The hash itself is pretty easy, but we need to figure out 
> when/where to do this check (for example, I think PeerSync is currently used 
> in multiple contexts and this check would perhaps not be appropriate for all 
> PeerSync calls?)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-8586) Implement hash over all documents to check for shard synchronization

[Yago Riveiro (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-8586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15419078#comment-15419078
 ] 

Yago Riveiro commented on SOLR-8586:


My index has 12T of data indexed with 4.0, the _version_ field only support 
docValues since 4.7.

To Upgrade to 5.x I ran the lucene-core-5.x over all my data,but with this new 
feature I need to re-index all my data because I don't have docValues for 
__version__ field and this feature use instead the un-inverted method that 
creates a memory struct that doesn't fit the memory of my servers ...

To be honest, this never should be done in a minor release ... this mandatory 
feature is based in a optional configuration :/

I will die in 5.4 or spend several months re-indexing data and figure out how 
to update production without downtime.  Not an easy task.



> Implement hash over all documents to check for shard synchronization
> 
>
> Key: SOLR-8586
> URL: https://issues.apache.org/jira/browse/SOLR-8586
> Project: Solr
>  Issue Type: Improvement
>  Components: SolrCloud
>Reporter: Yonik Seeley
>Assignee: Yonik Seeley
> Fix For: 5.5, 6.0
>
> Attachments: SOLR-8586.patch, SOLR-8586.patch, SOLR-8586.patch, 
> SOLR-8586.patch
>
>
> An order-independent hash across all of the versions in the index should 
> suffice.  The hash itself is pretty easy, but we need to figure out 
> when/where to do this check (for example, I think PeerSync is currently used 
> in multiple contexts and this check would perhaps not be appropriate for all 
> PeerSync calls?)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-Tests-master - Build # 1355 - Still Failing

[Apache Jenkins Server]

Build: https://builds.apache.org/job/Lucene-Solr-Tests-master/1355/

No tests ran.

Build Log:
[...truncated 21 lines...]
FATAL: Cannot find executable from the chosen Ant installation "Ant 1.8.4"
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
ERROR: Step ‘Publish JUnit test result report’ failed: No test report files 
were found. Configuration error?
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any




-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7410) Make cache keys and closed listeners less trappy

[Robert Muir (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15419068#comment-15419068
 ] 

Robert Muir commented on LUCENE-7410:
-

At a glance a lot of these ideas sound great, but i need some time to look at 
the patch in detail, i only glanced thru it.

things like switching the filterreader default to not caching at all, well 
thats arguably less trappy, but at the same time a risky change. Are all of our 
filter readers really correct/optimal now? Maybe its just better for these 
classes to redeclare the method as 'abstract', forcing implementations to deal 
with it? Personally I think thats the best solution for this case.

Not a huge fan of the name CacheHelper, at the same time I don't have a better 
suggestion.


> Make cache keys and closed listeners less trappy
> 
>
> Key: LUCENE-7410
> URL: https://issues.apache.org/jira/browse/LUCENE-7410
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Adrien Grand
> Attachments: LUCENE-7410.patch
>
>
> IndexReader currently exposes getCoreCacheKey(), 
> getCombinedCoreAndDeletesKey(), addCoreClosedListener() and 
> addReaderClosedListener(). They are typically used to manage resources whose 
> lifetime needs to mimic the lifetime of segments/indexes, typically caches.
> I think this is trappy for various reasons:
> h3. Memory leaks
> When maintaining a cache, entries are added to the cache based on the cache 
> key and then evicted using the cache key that is given back by the close 
> listener, so it is very important that both keys are the same.
> But if a filter reader happens to delegate get*Key() and not 
> add*ClosedListener() or vice-versa then there is potential for a memory leak 
> since the closed listener will be called on a different key and entries will 
> never be evicted from the cache.
> h3. Lifetime expectations
> The expectation of using the core cache key is that it will not change in 
> case of deletions, but this is only true on SegmentReader and LeafReader 
> impls that delegate to it. Other implementations such as composite readers or 
> parallel leaf readers use the same key for "core" and "combined core and 
> deletes".
> h3. Throw-away wrappers cause cache trashing
> An application might want to either expose more (with a ParrallelReader or 
> MultiReader) or less information (by filtering fields/docs that can be seen) 
> depending on the user who is logged in. In that case the application would 
> typically maintain a DirectoryReader and then wrap it per request depending 
> on the logged user and throw away the wrapper once the request is completed.
> The problem is that these wrappers have their own cache keys and the 
> application may build something costly and put it in a cache to throw it away 
> a couple milliseconds later. I would rather like for such readers to have a 
> way to opt out from caching on order to avoid this performance trap.
> h3. Type safety
> The keys that are exposed are plain java.lang.Object instances, which 
> requires caches to look like a {{Map}} which makes it very easy to 
> either try to get, put or remove on the wrong object since any object would 
> be accepted.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-6.x-Windows (64bit/jdk1.8.0_102) - Build # 380 - Unstable!

[Policeman Jenkins Server]

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-6.x-Windows/380/
Java: 64bit/jdk1.8.0_102 -XX:+UseCompressedOops -XX:+UseConcMarkSweepGC

1 tests failed.
FAILED:  
org.apache.solr.cloud.TestSolrCloudWithDelegationTokens.testDelegationTokenCancelFail

Error Message:
expected:<200> but was:<404>

Stack Trace:
java.lang.AssertionError: expected:<200> but was:<404>
at 
__randomizedtesting.SeedInfo.seed([519634A85C898A26:392901828C1398CA]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.junit.Assert.assertEquals(Assert.java:456)
at 
org.apache.solr.cloud.TestSolrCloudWithDelegationTokens.cancelDelegationToken(TestSolrCloudWithDelegationTokens.java:141)
at 
org.apache.solr.cloud.TestSolrCloudWithDelegationTokens.testDelegationTokenCancelFail(TestSolrCloudWithDelegationTokens.java:286)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRun

[jira] [Commented] (LUCENE-7410) Make cache keys and closed listeners less trappy

[Adrien Grand (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15419055#comment-15419055
 ] 

Adrien Grand commented on LUCENE-7410:
--

Any opinions?

> Make cache keys and closed listeners less trappy
> 
>
> Key: LUCENE-7410
> URL: https://issues.apache.org/jira/browse/LUCENE-7410
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Adrien Grand
> Attachments: LUCENE-7410.patch
>
>
> IndexReader currently exposes getCoreCacheKey(), 
> getCombinedCoreAndDeletesKey(), addCoreClosedListener() and 
> addReaderClosedListener(). They are typically used to manage resources whose 
> lifetime needs to mimic the lifetime of segments/indexes, typically caches.
> I think this is trappy for various reasons:
> h3. Memory leaks
> When maintaining a cache, entries are added to the cache based on the cache 
> key and then evicted using the cache key that is given back by the close 
> listener, so it is very important that both keys are the same.
> But if a filter reader happens to delegate get*Key() and not 
> add*ClosedListener() or vice-versa then there is potential for a memory leak 
> since the closed listener will be called on a different key and entries will 
> never be evicted from the cache.
> h3. Lifetime expectations
> The expectation of using the core cache key is that it will not change in 
> case of deletions, but this is only true on SegmentReader and LeafReader 
> impls that delegate to it. Other implementations such as composite readers or 
> parallel leaf readers use the same key for "core" and "combined core and 
> deletes".
> h3. Throw-away wrappers cause cache trashing
> An application might want to either expose more (with a ParrallelReader or 
> MultiReader) or less information (by filtering fields/docs that can be seen) 
> depending on the user who is logged in. In that case the application would 
> typically maintain a DirectoryReader and then wrap it per request depending 
> on the logged user and throw away the wrapper once the request is completed.
> The problem is that these wrappers have their own cache keys and the 
> application may build something costly and put it in a cache to throw it away 
> a couple milliseconds later. I would rather like for such readers to have a 
> way to opt out from caching on order to avoid this performance trap.
> h3. Type safety
> The keys that are exposed are plain java.lang.Object instances, which 
> requires caches to look like a {{Map}} which makes it very easy to 
> either try to get, put or remove on the wrong object since any object would 
> be accepted.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-Tests-6.x - Build # 412 - Still Failing

[Apache Jenkins Server]

Build: https://builds.apache.org/job/Lucene-Solr-Tests-6.x/412/

No tests ran.

Build Log:
[...truncated 21 lines...]
FATAL: Cannot find executable from the chosen Ant installation "Ant 1.8.4"
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
ERROR: Step ‘Publish JUnit test result report’ failed: No test report files 
were found. Configuration error?
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any




-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-master-Linux (64bit/jdk1.8.0_102) - Build # 17542 - Unstable!

[Policeman Jenkins Server]

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/17542/
Java: 64bit/jdk1.8.0_102 -XX:+UseCompressedOops -XX:+UseConcMarkSweepGC

1 tests failed.
FAILED:  
org.apache.solr.common.cloud.TestCollectionStateWatchers.testWatchesWorkForStateFormat1

Error Message:
CollectionStateWatcher not notified of stateformat=1 collection creation

Stack Trace:
java.lang.AssertionError: CollectionStateWatcher not notified of stateformat=1 
collection creation
at 
__randomizedtesting.SeedInfo.seed([6D6F840868A78668:AA898E69C1FA2C9]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at 
org.apache.solr.common.cloud.TestCollectionStateWatchers.testWatchesWorkForStateFormat1(TestCollectionStateWatchers.java:290)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 13241 lines...]
   [junit4] Suite: org.apache.solr.common.cloud.TestCollectionStateWatchers
   [junit4]   2> 

[JENKINS] Lucene-Solr-Tests-master - Build # 1354 - Still Failing

[Apache Jenkins Server]

Build: https://builds.apache.org/job/Lucene-Solr-Tests-master/1354/

No tests ran.

Build Log:
[...truncated 21 lines...]
FATAL: Cannot find executable from the chosen Ant installation "Ant 1.8.4"
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
ERROR: Step ‘Publish JUnit test result report’ failed: No test report files 
were found. Configuration error?
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any




-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9310) PeerSync fails on a node restart due to IndexFingerPrint mismatch

[Noble Paul (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15418979#comment-15418979
 ] 

Noble Paul commented on SOLR-9310:
--

Of something goes wrong peersync fails

> PeerSync fails on a node restart due to IndexFingerPrint mismatch
> -
>
> Key: SOLR-9310
> URL: https://issues.apache.org/jira/browse/SOLR-9310
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Pushkar Raste
>Assignee: Noble Paul
> Attachments: PeerSync_Experiment.patch, SOLR-9310.patch, 
> SOLR-9310.patch, SOLR-9310.patch, SOLR-9310.patch
>
>
> I found that Peer Sync fails if a node restarts and documents were indexed 
> while node was down. IndexFingerPrint check fails after recovering node 
> applies updates. 
> This happens only when node restarts and not if node just misses updates due 
> reason other than it being down.
> Please check attached patch for the test.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-Tests-6.x - Build # 411 - Still Failing

[Apache Jenkins Server]

Build: https://builds.apache.org/job/Lucene-Solr-Tests-6.x/411/

No tests ran.

Build Log:
[...truncated 21 lines...]
FATAL: Cannot find executable from the chosen Ant installation "Ant 1.8.4"
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
ERROR: Step ‘Publish JUnit test result report’ failed: No test report files 
were found. Configuration error?
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any




-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7161) TestMoreLikeThis.testMultiFieldShouldReturnPerFieldBooleanQuery assertion error

[Michael McCandless (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15418936#comment-15418936
 ] 

Michael McCandless commented on LUCENE-7161:


Do we have a sense of whether these ongoing failures are a test bug or a 
problem with LUCENE-6954?  Maybe we should revert LUCENE-6954 for now?

> TestMoreLikeThis.testMultiFieldShouldReturnPerFieldBooleanQuery assertion 
> error
> ---
>
> Key: LUCENE-7161
> URL: https://issues.apache.org/jira/browse/LUCENE-7161
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Michael McCandless
>Assignee: Tommaso Teofili
> Fix For: master (7.0), 6.2
>
>
> I just hit this unrelated but reproducible on master 
> #cc75be53f9b3b86ec59cb93896c4fd5a9a5926b2 while tweaking earth's radius:
> {noformat}
>[junit4] Suite: org.apache.lucene.queries.mlt.TestMoreLikeThis
>[junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestMoreLikeThis 
> -Dtests.method=testMultiFieldShouldReturnPerFieldBooleanQuery 
> -Dtests.seed=794526110651C8E6 -Dtests.locale=es-HN 
> -Dtests.timezone=Brazil/West -Dtests.asserts=true 
> -Dtests.file.encoding=US-ASCII
>[junit4] FAILURE 0.25s | 
> TestMoreLikeThis.testMultiFieldShouldReturnPerFieldBooleanQuery <<<
>[junit4]> Throwable #1: java.lang.AssertionError
>[junit4]>  at 
> __randomizedtesting.SeedInfo.seed([794526110651C8E6:1DF67ED7BBBF4E1D]:0)
>[junit4]>  at 
> org.apache.lucene.queries.mlt.TestMoreLikeThis.testMultiFieldShouldReturnPerFieldBooleanQuery(TestMoreLikeThis.java:320)
>[junit4]>  at java.lang.Thread.run(Thread.java:745)
>[junit4]   2> NOTE: test params are: codec=CheapBastard, 
> sim=ClassicSimilarity, locale=es-HN, timezone=Brazil/West
>[junit4]   2> NOTE: Linux 3.13.0-71-generic amd64/Oracle Corporation 
> 1.8.0_60 (64-bit)/cpus=8,threads=1,free=409748864,total=504889344
>[junit4]   2> NOTE: All tests run in this JVM: [TestMoreLikeThis]
>[junit4] Completed [1/1 (1!)] in 0.45s, 1 test, 1 failure <<< FAILURES!
>[junit4] 
>[junit4] 
>[junit4] Tests with failures [seed: 794526110651C8E6]:
>[junit4]   - 
> org.apache.lucene.queries.mlt.TestMoreLikeThis.testMultiFieldShouldReturnPerFieldBooleanQuery
> {noformat}
> Likely related to LUCENE-6954?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9310) PeerSync fails on a node restart due to IndexFingerPrint mismatch

[Pushkar Raste (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15418920#comment-15418920
 ] 

Pushkar Raste commented on SOLR-9310:
-

If something goes wrong while applying updates, how can you assure that replica 
is in sync?

> PeerSync fails on a node restart due to IndexFingerPrint mismatch
> -
>
> Key: SOLR-9310
> URL: https://issues.apache.org/jira/browse/SOLR-9310
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Pushkar Raste
>Assignee: Noble Paul
> Attachments: PeerSync_Experiment.patch, SOLR-9310.patch, 
> SOLR-9310.patch, SOLR-9310.patch, SOLR-9310.patch
>
>
> I found that Peer Sync fails if a node restarts and documents were indexed 
> while node was down. IndexFingerPrint check fails after recovering node 
> applies updates. 
> This happens only when node restarts and not if node just misses updates due 
> reason other than it being down.
> Please check attached patch for the test.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9310) PeerSync fails on a node restart due to IndexFingerPrint mismatch

[Noble Paul (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15418915#comment-15418915
 ] 

Noble Paul commented on SOLR-9310:
--

Do you see a potential problem if fingerprint is compared before applying 
updates

> PeerSync fails on a node restart due to IndexFingerPrint mismatch
> -
>
> Key: SOLR-9310
> URL: https://issues.apache.org/jira/browse/SOLR-9310
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Pushkar Raste
>Assignee: Noble Paul
> Attachments: PeerSync_Experiment.patch, SOLR-9310.patch, 
> SOLR-9310.patch, SOLR-9310.patch, SOLR-9310.patch
>
>
> I found that Peer Sync fails if a node restarts and documents were indexed 
> while node was down. IndexFingerPrint check fails after recovering node 
> applies updates. 
> This happens only when node restarts and not if node just misses updates due 
> reason other than it being down.
> Please check attached patch for the test.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-SmokeRelease-master - Build # 565 - Failure

[Apache Jenkins Server]

Build: https://builds.apache.org/job/Lucene-Solr-SmokeRelease-master/565/

No tests ran.

Build Log:
[...truncated 21 lines...]
FATAL: Cannot find executable from the chosen Ant installation "Ant 1.8.4"
Build step 'Invoke Ant' marked build as failure
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any




-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-9320) A REPLACENODE command to decommission an existing node with another new node

[Varun Thacker (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-9320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Varun Thacker updated SOLR-9320:

Attachment: SOLR-9320_followup.patch

- The OverseerCollectionMessageHandler#checkRequired is not required . We 
already validate it earlier in CollectionsHandler
- Fixed logging
- Changed OverseerCollectionMessageHandler#Cmd#call to return void as we don't 
need to return anything . All results are passed in the namedlist.

Noble - What do you think about this patch?

> A REPLACENODE command to decommission an existing node with another new node
> 
>
> Key: SOLR-9320
> URL: https://issues.apache.org/jira/browse/SOLR-9320
> Project: Solr
>  Issue Type: Sub-task
>  Components: SolrCloud
>Reporter: Noble Paul
>Assignee: Noble Paul
> Fix For: 6.2, master (7.0)
>
> Attachments: DELETENODE.jpeg, REPLACENODE_After.jpeg, 
> REPLACENODE_Before.jpeg, REPLACENODE_call_response.jpeg, SOLR-9320.patch, 
> SOLR-9320.patch, SOLR-9320.patch, SOLR-9320.patch, SOLR-9320.patch, 
> SOLR-9320.patch, SOLR-9320.patch, SOLR-9320_followup.patch
>
>
> The command should accept a source node and target node. recreate the 
> replicas in source node in the target and do a DLETENODE of source node



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-master-Windows (64bit/jdk1.8.0_102) - Build # 6044 - Still Unstable!

[Policeman Jenkins Server]

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-master-Windows/6044/
Java: 64bit/jdk1.8.0_102 -XX:-UseCompressedOops -XX:+UseParallelGC

3 tests failed.
FAILED:  org.apache.solr.security.BasicAuthIntegrationTest.testBasics

Error Message:
IOException occured when talking to server at: 
http://127.0.0.1:59245/solr/testSolrCloudCollection_shard1_replica2

Stack Trace:
org.apache.solr.client.solrj.impl.CloudSolrClient$RouteException: IOException 
occured when talking to server at: 
http://127.0.0.1:59245/solr/testSolrCloudCollection_shard1_replica2
at 
__randomizedtesting.SeedInfo.seed([30A0E9E05F41E5D1:D7847CC67AFBBA1]:0)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.directUpdate(CloudSolrClient.java:760)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1172)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:1061)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:997)
at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
at 
org.apache.solr.security.BasicAuthIntegrationTest.doExtraTests(BasicAuthIntegrationTest.java:193)
at 
org.apache.solr.cloud.TestMiniSolrCloudClusterBase.testCollectionCreateSearchDelete(TestMiniSolrCloudClusterBase.java:196)
at 
org.apache.solr.cloud.TestMiniSolrCloudClusterBase.testBasics(TestMiniSolrCloudClusterBase.java:79)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.

[jira] [Resolved] (SOLR-8715) New Admin UI's Schema screen fails for fields with stored=false and indexed content

[Alexandre Rafalovitch (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-8715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexandre Rafalovitch resolved SOLR-8715.
-
   Resolution: Fixed
Fix Version/s: master (7.0)
   6.2

> New Admin UI's Schema screen fails for fields with stored=false and indexed 
> content
> ---
>
> Key: SOLR-8715
> URL: https://issues.apache.org/jira/browse/SOLR-8715
> Project: Solr
>  Issue Type: Bug
>  Components: UI
>Affects Versions: 5.5, 6.0
> Environment: mac, firefox
>Reporter: Alexandre Rafalovitch
>Assignee: Alexandre Rafalovitch
>  Labels: admin-interface
> Fix For: 6.2, master (7.0)
>
> Attachments: Problem shown in the released 5.5 version.png
>
>
> In techproducts example, using new Admin UI and trying to load the Schema for 
> text field causes blank screen and the Javascript error in the developer 
> console:
> {noformat}
> Error: row.flags is undefined
> getFieldProperties@http://localhost:8983/solr/js/angular/controllers/schema.js:482:40
> $scope.refresh/http://localhost:8983/solr/js/angular/controllers/schema.js:76:38
> 
> {noformat}
> Tested with 5.5rc3



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-8715) New Admin UI's Schema screen fails for fields with stored=false and indexed content

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-8715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15418895#comment-15418895
 ] 

ASF subversion and git services commented on SOLR-8715:
---

Commit a6ce37022e744f5ba535b1cb52c3661aa8cab687 in lucene-solr's branch 
refs/heads/branch_6x from [~arafalov]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=a6ce370 ]

SOLR-8715: Admin UI - Fix schema special case


> New Admin UI's Schema screen fails for fields with stored=false and indexed 
> content
> ---
>
> Key: SOLR-8715
> URL: https://issues.apache.org/jira/browse/SOLR-8715
> Project: Solr
>  Issue Type: Bug
>  Components: UI
>Affects Versions: 5.5, 6.0
> Environment: mac, firefox
>Reporter: Alexandre Rafalovitch
>Assignee: Alexandre Rafalovitch
>  Labels: admin-interface
> Attachments: Problem shown in the released 5.5 version.png
>
>
> In techproducts example, using new Admin UI and trying to load the Schema for 
> text field causes blank screen and the Javascript error in the developer 
> console:
> {noformat}
> Error: row.flags is undefined
> getFieldProperties@http://localhost:8983/solr/js/angular/controllers/schema.js:482:40
> $scope.refresh/http://localhost:8983/solr/js/angular/controllers/schema.js:76:38
> 
> {noformat}
> Tested with 5.5rc3



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-Tests-master - Build # 1353 - Still Failing

[Apache Jenkins Server]

Build: https://builds.apache.org/job/Lucene-Solr-Tests-master/1353/

No tests ran.

Build Log:
[...truncated 21 lines...]
FATAL: Cannot find executable from the chosen Ant installation "Ant 1.8.4"
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
ERROR: Step ‘Publish JUnit test result report’ failed: No test report files 
were found. Configuration error?
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any




-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-8715) New Admin UI's Schema screen fails for fields with stored=false and indexed content

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-8715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15418889#comment-15418889
 ] 

ASF GitHub Bot commented on SOLR-8715:
--

Github user arafalov closed the pull request at:

https://github.com/apache/lucene-solr/pull/55


> New Admin UI's Schema screen fails for fields with stored=false and indexed 
> content
> ---
>
> Key: SOLR-8715
> URL: https://issues.apache.org/jira/browse/SOLR-8715
> Project: Solr
>  Issue Type: Bug
>  Components: UI
>Affects Versions: 5.5, 6.0
> Environment: mac, firefox
>Reporter: Alexandre Rafalovitch
>Assignee: Alexandre Rafalovitch
>  Labels: admin-interface
> Attachments: Problem shown in the released 5.5 version.png
>
>
> In techproducts example, using new Admin UI and trying to load the Schema for 
> text field causes blank screen and the Javascript error in the developer 
> console:
> {noformat}
> Error: row.flags is undefined
> getFieldProperties@http://localhost:8983/solr/js/angular/controllers/schema.js:482:40
> $scope.refresh/http://localhost:8983/solr/js/angular/controllers/schema.js:76:38
> 
> {noformat}
> Tested with 5.5rc3



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[GitHub] lucene-solr pull request #55: SOLR-8715: Added special condition to match se...

[arafalov]

Github user arafalov closed the pull request at:

https://github.com/apache/lucene-solr/pull/55


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-8715) New Admin UI's Schema screen fails for fields with stored=false and indexed content

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-8715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1541#comment-1541
 ] 

ASF GitHub Bot commented on SOLR-8715:
--

Github user arafalov closed the pull request at:

https://github.com/apache/lucene-solr/pull/33


> New Admin UI's Schema screen fails for fields with stored=false and indexed 
> content
> ---
>
> Key: SOLR-8715
> URL: https://issues.apache.org/jira/browse/SOLR-8715
> Project: Solr
>  Issue Type: Bug
>  Components: UI
>Affects Versions: 5.5, 6.0
> Environment: mac, firefox
>Reporter: Alexandre Rafalovitch
>Assignee: Alexandre Rafalovitch
>  Labels: admin-interface
> Attachments: Problem shown in the released 5.5 version.png
>
>
> In techproducts example, using new Admin UI and trying to load the Schema for 
> text field causes blank screen and the Javascript error in the developer 
> console:
> {noformat}
> Error: row.flags is undefined
> getFieldProperties@http://localhost:8983/solr/js/angular/controllers/schema.js:482:40
> $scope.refresh/http://localhost:8983/solr/js/angular/controllers/schema.js:76:38
> 
> {noformat}
> Tested with 5.5rc3



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[GitHub] lucene-solr pull request #33: Skip flag checks for when they are unavailable...

[arafalov]

Github user arafalov closed the pull request at:

https://github.com/apache/lucene-solr/pull/33


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-8715) New Admin UI's Schema screen fails for fields with stored=false and indexed content

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-8715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15418879#comment-15418879
 ] 

ASF subversion and git services commented on SOLR-8715:
---

Commit cea8a488f0d48041abb4be0dbe29ab81f04522bf in lucene-solr's branch 
refs/heads/master from [~arafalov]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=cea8a48 ]

SOLR-8715: Admin UI - Fix schema special case


> New Admin UI's Schema screen fails for fields with stored=false and indexed 
> content
> ---
>
> Key: SOLR-8715
> URL: https://issues.apache.org/jira/browse/SOLR-8715
> Project: Solr
>  Issue Type: Bug
>  Components: UI
>Affects Versions: 5.5, 6.0
> Environment: mac, firefox
>Reporter: Alexandre Rafalovitch
>Assignee: Alexandre Rafalovitch
>  Labels: admin-interface
> Attachments: Problem shown in the released 5.5 version.png
>
>
> In techproducts example, using new Admin UI and trying to load the Schema for 
> text field causes blank screen and the Javascript error in the developer 
> console:
> {noformat}
> Error: row.flags is undefined
> getFieldProperties@http://localhost:8983/solr/js/angular/controllers/schema.js:482:40
> $scope.refresh/http://localhost:8983/solr/js/angular/controllers/schema.js:76:38
> 
> {noformat}
> Tested with 5.5rc3



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-8715) New Admin UI's Schema screen fails for fields with stored=false and indexed content

[Alexandre Rafalovitch (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-8715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexandre Rafalovitch updated SOLR-8715:

Summary: New Admin UI's Schema screen fails for fields with stored=false 
and indexed content  (was: New Admin UI's Schema screen fails for some fields)

> New Admin UI's Schema screen fails for fields with stored=false and indexed 
> content
> ---
>
> Key: SOLR-8715
> URL: https://issues.apache.org/jira/browse/SOLR-8715
> Project: Solr
>  Issue Type: Bug
>  Components: UI
>Affects Versions: 5.5, 6.0
> Environment: mac, firefox
>Reporter: Alexandre Rafalovitch
>Assignee: Alexandre Rafalovitch
>  Labels: admin-interface
> Attachments: Problem shown in the released 5.5 version.png
>
>
> In techproducts example, using new Admin UI and trying to load the Schema for 
> text field causes blank screen and the Javascript error in the developer 
> console:
> {noformat}
> Error: row.flags is undefined
> getFieldProperties@http://localhost:8983/solr/js/angular/controllers/schema.js:482:40
> $scope.refresh/http://localhost:8983/solr/js/angular/controllers/schema.js:76:38
> 
> {noformat}
> Tested with 5.5rc3



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-Tests-6.x - Build # 410 - Still Failing

[Apache Jenkins Server]

Build: https://builds.apache.org/job/Lucene-Solr-Tests-6.x/410/

No tests ran.

Build Log:
[...truncated 21 lines...]
FATAL: Cannot find executable from the chosen Ant installation "Ant 1.8.4"
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
ERROR: Step ‘Publish JUnit test result report’ failed: No test report files 
were found. Configuration error?
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any




-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9310) PeerSync fails on a node restart due to IndexFingerPrint mismatch

[Noble Paul (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15418843#comment-15418843
 ] 

Noble Paul commented on SOLR-9310:
--

If there is a way to get the fingerprint after applying the updates, it will be 
good. 

> PeerSync fails on a node restart due to IndexFingerPrint mismatch
> -
>
> Key: SOLR-9310
> URL: https://issues.apache.org/jira/browse/SOLR-9310
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Pushkar Raste
>Assignee: Noble Paul
> Attachments: PeerSync_Experiment.patch, SOLR-9310.patch, 
> SOLR-9310.patch, SOLR-9310.patch, SOLR-9310.patch
>
>
> I found that Peer Sync fails if a node restarts and documents were indexed 
> while node was down. IndexFingerPrint check fails after recovering node 
> applies updates. 
> This happens only when node restarts and not if node just misses updates due 
> reason other than it being down.
> Please check attached patch for the test.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (SOLR-9232) Can't Swap Cores in New Admin UI

[Alexandre Rafalovitch (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-9232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexandre Rafalovitch resolved SOLR-9232.
-
   Resolution: Fixed
Fix Version/s: master (7.0)
   6.2

> Can't Swap Cores in New Admin UI
> 
>
> Key: SOLR-9232
> URL: https://issues.apache.org/jira/browse/SOLR-9232
> Project: Solr
>  Issue Type: Bug
>  Components: UI
>Affects Versions: 6.0, 6.1
>Reporter: Nikitas Tampakis
>Assignee: Alexandre Rafalovitch
>Priority: Minor
>  Labels: Regression
> Fix For: 6.2, master (7.0)
>
> Attachments: swap-cores-ui-screenshot.png
>
>
> Using the Core Admin page on the new Admin UI, there is no way to submit a 
> Swap Cores request. After choosing the Swap action for an existing core and 
> selecting a core to swap with, clicking the Swap Cores button does nothing.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9232) Can't Swap Cores in New Admin UI

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15418835#comment-15418835
 ] 

ASF GitHub Bot commented on SOLR-9232:
--

Github user arafalov closed the pull request at:

https://github.com/apache/lucene-solr/pull/63


> Can't Swap Cores in New Admin UI
> 
>
> Key: SOLR-9232
> URL: https://issues.apache.org/jira/browse/SOLR-9232
> Project: Solr
>  Issue Type: Bug
>  Components: UI
>Affects Versions: 6.0, 6.1
>Reporter: Nikitas Tampakis
>Assignee: Alexandre Rafalovitch
>Priority: Minor
>  Labels: Regression
> Attachments: swap-cores-ui-screenshot.png
>
>
> Using the Core Admin page on the new Admin UI, there is no way to submit a 
> Swap Cores request. After choosing the Swap action for an existing core and 
> selecting a core to swap with, clicking the Swap Cores button does nothing.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[GitHub] lucene-solr pull request #63: SOLR-9232: Fixed Core Swap implementation

[arafalov]

Github user arafalov closed the pull request at:

https://github.com/apache/lucene-solr/pull/63


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9232) Can't Swap Cores in New Admin UI

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15418834#comment-15418834
 ] 

ASF subversion and git services commented on SOLR-9232:
---

Commit 9ba422ac4482bb6e367d2c0a7a7dc9c3998ab1f9 in lucene-solr's branch 
refs/heads/branch_6x from [~arafalov]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=9ba422a ]

SOLR-9232: Fix Swap Cores in Admin UI


> Can't Swap Cores in New Admin UI
> 
>
> Key: SOLR-9232
> URL: https://issues.apache.org/jira/browse/SOLR-9232
> Project: Solr
>  Issue Type: Bug
>  Components: UI
>Affects Versions: 6.0, 6.1
>Reporter: Nikitas Tampakis
>Assignee: Alexandre Rafalovitch
>Priority: Minor
>  Labels: Regression
> Attachments: swap-cores-ui-screenshot.png
>
>
> Using the Core Admin page on the new Admin UI, there is no way to submit a 
> Swap Cores request. After choosing the Swap action for an existing core and 
> selecting a core to swap with, clicking the Swap Cores button does nothing.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-Tests-master - Build # 1352 - Still Failing

[Apache Jenkins Server]

Build: https://builds.apache.org/job/Lucene-Solr-Tests-master/1352/

No tests ran.

Build Log:
[...truncated 21 lines...]
FATAL: Cannot find executable from the chosen Ant installation "Ant 1.8.4"
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
ERROR: Step ‘Publish JUnit test result report’ failed: No test report files 
were found. Configuration error?
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any




-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-8586) Implement hash over all documents to check for shard synchronization

[Yonik Seeley (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-8586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15418799#comment-15418799
 ] 

Yonik Seeley commented on SOLR-8586:


If the _version_ field doesn't have docValues, then it will be un-inverted 
(i.e. FieldCache entries will be built to support _version_ lookups, and that 
does require memory).
Since _version_ lookups are needed in the course of indexing anyway (to detect 
update reorders on replicas), this should really just change when these 
FieldCache entries are created... hence the maximum required amount of memory 
should be changed.

> Implement hash over all documents to check for shard synchronization
> 
>
> Key: SOLR-8586
> URL: https://issues.apache.org/jira/browse/SOLR-8586
> Project: Solr
>  Issue Type: Improvement
>  Components: SolrCloud
>Reporter: Yonik Seeley
>Assignee: Yonik Seeley
> Fix For: 5.5, 6.0
>
> Attachments: SOLR-8586.patch, SOLR-8586.patch, SOLR-8586.patch, 
> SOLR-8586.patch
>
>
> An order-independent hash across all of the versions in the index should 
> suffice.  The hash itself is pretty easy, but we need to figure out 
> when/where to do this check (for example, I think PeerSync is currently used 
> in multiple contexts and this check would perhaps not be appropriate for all 
> PeerSync calls?)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9232) Can't Swap Cores in New Admin UI

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15418794#comment-15418794
 ] 

ASF subversion and git services commented on SOLR-9232:
---

Commit f20e2f3a941c8ffd3f4bac5607bb4b5f782cc29d in lucene-solr's branch 
refs/heads/master from [~arafalov]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=f20e2f3 ]

SOLR-9232: Fix Swap Cores in Admin UI


> Can't Swap Cores in New Admin UI
> 
>
> Key: SOLR-9232
> URL: https://issues.apache.org/jira/browse/SOLR-9232
> Project: Solr
>  Issue Type: Bug
>  Components: UI
>Affects Versions: 6.0, 6.1
>Reporter: Nikitas Tampakis
>Assignee: Alexandre Rafalovitch
>Priority: Minor
>  Labels: Regression
> Attachments: swap-cores-ui-screenshot.png
>
>
> Using the Core Admin page on the new Admin UI, there is no way to submit a 
> Swap Cores request. After choosing the Swap action for an existing core and 
> selecting a core to swap with, clicking the Swap Cores button does nothing.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9241) Rebalance API for SolrCloud

[Varun Thacker (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15418789#comment-15418789
 ] 

Varun Thacker commented on SOLR-9241:
-

FYI I am working on a patch to improve the logging. I'll post a patch soon

> Rebalance API for SolrCloud
> ---
>
> Key: SOLR-9241
> URL: https://issues.apache.org/jira/browse/SOLR-9241
> Project: Solr
>  Issue Type: New Feature
>  Components: SolrCloud
>Affects Versions: 6.1
> Environment: Ubuntu, Mac OsX
>Reporter: Nitin Sharma
>  Labels: Cluster, SolrCloud
> Fix For: 6.1
>
> Attachments: Redistribute_After.jpeg, Redistribute_Before.jpeg, 
> Redistribute_call.jpeg, Replace_After.jpeg, Replace_Before.jpeg, 
> Replace_Call.jpeg, SOLR-9241-4.6.patch, SOLR-9241-6.1.patch
>
>   Original Estimate: 2,016h
>  Remaining Estimate: 2,016h
>
> This is the v1 of the patch for Solrcloud Rebalance api (as described in 
> http://engineering.bloomreach.com/solrcloud-rebalance-api/) , built at 
> Bloomreach by Nitin Sharma and Suruchi Shah. The goal of the API  is to 
> provide a zero downtime mechanism to perform data manipulation and  efficient 
> core allocation in solrcloud. This API was envisioned to be the base layer 
> that enables Solrcloud to be an auto scaling platform. (and work in unison 
> with other complementing monitoring and scaling features).
> Patch Status:
> ===
> The patch is work in progress and incremental. We have done a few rounds of 
> code clean up. We wanted to get the patch going first to get initial feed 
> back.  We will continue to work on making it more open source friendly and 
> easily testable.
>  Deployment Status:
> 
> The platform is deployed in production at bloomreach and has been battle 
> tested for large scale load. (millions of documents and hundreds of 
> collections).
>  Internals:
> =
> The internals of the API and performance : 
> http://engineering.bloomreach.com/solrcloud-rebalance-api/
> It is built on top of the admin collections API as an action (with various 
> flavors). At a high level, the rebalance api provides 2 constructs:
> Scaling Strategy:  Decides how to move the data.  Every flavor has multiple 
> options which can be reviewed in the api spec.
> Re-distribute  - Move around data in the cluster based on capacity/allocation.
> Auto Shard  - Dynamically shard a collection to any size.
> Smart Merge - Distributed Mode - Helps merging data from a larger shard setup 
> into smaller one.  (the source should be divisible by destination)
> Scale up -  Add replicas on the fly
> Scale Down - Remove replicas on the fly
> Allocation Strategy:  Decides where to put the data.  (Nodes with least 
> cores, Nodes that do not have this collection etc). Custom implementations 
> can be built on top as well. One other example is Availability Zone aware. 
> Distribute data such that every replica is placed on different availability 
> zone to support HA.
>  Detailed API Spec:
> 
>   https://github.com/bloomreach/solrcloud-rebalance-api
>  Contributors:
> =
>   Nitin Sharma
>   Suruchi Shah
>  Questions/Comments:
> =
>   You can reach me at nitin.sha...@bloomreach.com



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9232) Can't Swap Cores in New Admin UI

[Alexandre Rafalovitch (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15418782#comment-15418782
 ] 

Alexandre Rafalovitch commented on SOLR-9232:
-

In the cloud mode, there is Cores screen and the Collections screen does not 
have Swap button. So, we only need to cover the standalone mode.

> Can't Swap Cores in New Admin UI
> 
>
> Key: SOLR-9232
> URL: https://issues.apache.org/jira/browse/SOLR-9232
> Project: Solr
>  Issue Type: Bug
>  Components: UI
>Affects Versions: 6.0, 6.1
>Reporter: Nikitas Tampakis
>Assignee: Alexandre Rafalovitch
>Priority: Minor
>  Labels: Regression
> Attachments: swap-cores-ui-screenshot.png
>
>
> Using the Core Admin page on the new Admin UI, there is no way to submit a 
> Swap Cores request. After choosing the Swap action for an existing core and 
> selecting a core to swap with, clicking the Swap Cores button does nothing.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-Tests-6.x - Build # 409 - Still Failing

[Apache Jenkins Server]

Build: https://builds.apache.org/job/Lucene-Solr-Tests-6.x/409/

No tests ran.

Build Log:
[...truncated 21 lines...]
FATAL: Cannot find executable from the chosen Ant installation "Ant 1.8.4"
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
ERROR: Step ‘Publish JUnit test result report’ failed: No test report files 
were found. Configuration error?
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any




-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-6.x-Solaris (64bit/jdk1.8.0) - Build # 323 - Unstable!

[Policeman Jenkins Server]

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-6.x-Solaris/323/
Java: 64bit/jdk1.8.0 -XX:+UseCompressedOops -XX:+UseSerialGC

1 tests failed.
FAILED:  
org.apache.solr.handler.TestReplicationHandler.doTestIndexAndConfigAliasReplication

Error Message:
[/export/home/jenkins/workspace/Lucene-Solr-6.x-Solaris/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_ABA50C838C256140-001/solr-instance-004/./collection1/data/index.20160812115313801,
 
/export/home/jenkins/workspace/Lucene-Solr-6.x-Solaris/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_ABA50C838C256140-001/solr-instance-004/./collection1/data/index.20160812115314123,
 
/export/home/jenkins/workspace/Lucene-Solr-6.x-Solaris/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_ABA50C838C256140-001/solr-instance-004/./collection1/data,
 
/export/home/jenkins/workspace/Lucene-Solr-6.x-Solaris/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_ABA50C838C256140-001/solr-instance-004/./collection1/data/snapshot_metadata]
 expected:<3> but was:<4>

Stack Trace:
java.lang.AssertionError: 
[/export/home/jenkins/workspace/Lucene-Solr-6.x-Solaris/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_ABA50C838C256140-001/solr-instance-004/./collection1/data/index.20160812115313801,
 
/export/home/jenkins/workspace/Lucene-Solr-6.x-Solaris/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_ABA50C838C256140-001/solr-instance-004/./collection1/data/index.20160812115314123,
 
/export/home/jenkins/workspace/Lucene-Solr-6.x-Solaris/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_ABA50C838C256140-001/solr-instance-004/./collection1/data,
 
/export/home/jenkins/workspace/Lucene-Solr-6.x-Solaris/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_ABA50C838C256140-001/solr-instance-004/./collection1/data/snapshot_metadata]
 expected:<3> but was:<4>
at 
__randomizedtesting.SeedInfo.seed([ABA50C838C256140:5CD6E2DB4ACDCEA6]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at 
org.apache.solr.handler.TestReplicationHandler.checkForSingleIndex(TestReplicationHandler.java:904)
at 
org.apache.solr.handler.TestReplicationHandler.doTestIndexAndConfigAliasReplication(TestReplicationHandler.java:1336)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
or

[jira] [Resolved] (SOLR-9405) ConcurrentModificationException in ZkStateReader.getStateWatchers

[Shalin Shekhar Mangar (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-9405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shalin Shekhar Mangar resolved SOLR-9405.
-
Resolution: Fixed
  Assignee: Shalin Shekhar Mangar  (was: Alan Woodward)

Thanks Alan and Edward. I folded Alan's changes to use compute in 
getStateWatchers into the patch/commit.

> ConcurrentModificationException in ZkStateReader.getStateWatchers
> -
>
> Key: SOLR-9405
> URL: https://issues.apache.org/jira/browse/SOLR-9405
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Affects Versions: 6.1
>Reporter: Shalin Shekhar Mangar
>Assignee: Shalin Shekhar Mangar
> Fix For: 6.2, master (7.0)
>
> Attachments: SOLR-9405.patch, SOLR-9405.patch
>
>
> Jenkins found this: http://jenkins.thetaphi.de/job/Lucene-Solr-6.x-Linux/1432/
> {code}
> Stack Trace:
> java.util.ConcurrentModificationException
> at 
> __randomizedtesting.SeedInfo.seed([FA459DF725097EFF:A77E52876204E1C1]:0)
> at 
> java.util.HashMap$HashIterator.nextNode(java.base@9-ea/HashMap.java:1489)
> at 
> java.util.HashMap$KeyIterator.next(java.base@9-ea/HashMap.java:1513)
> at 
> java.util.AbstractCollection.addAll(java.base@9-ea/AbstractCollection.java:351)
> at java.util.HashSet.(java.base@9-ea/HashSet.java:119)
> at 
> org.apache.solr.common.cloud.ZkStateReader.getStateWatchers(ZkStateReader.java:1279)
> at 
> org.apache.solr.common.cloud.TestCollectionStateWatchers.testSimpleCollectionWatch(TestCollectionStateWatchers.java:116)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-9411) Better validation for Schema API

[JIRA]

 [ 
https://issues.apache.org/jira/browse/SOLR-9411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jan Høydahl updated SOLR-9411:
--
Description: 
Schema REST API needs better validation before doing changes.

* It should not be allowed to delete uniqueKey (also handled in SOLR-9349)
* When adding a dynamic field the API should test that it begins or ends with 
{{*}}. Today the change succeeds, but you get errors later

These are two known cases. We should harden validation across the board for all 
known schema requirements.

  was:
Schema REST API needs better validation before doing changes.

* It should not be allowed to delete uniqueKey
* When adding a dynamic field the API should test that it begins or ends with 
{{*}}. Today the change succeeds, but you get errors later

These are two known cases. We should harden validation across the board for all 
known schema requirements.


> Better validation for Schema API
> 
>
> Key: SOLR-9411
> URL: https://issues.apache.org/jira/browse/SOLR-9411
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Schema and Analysis
>Reporter: Jan Høydahl
>
> Schema REST API needs better validation before doing changes.
> * It should not be allowed to delete uniqueKey (also handled in SOLR-9349)
> * When adding a dynamic field the API should test that it begins or ends with 
> {{*}}. Today the change succeeds, but you get errors later
> These are two known cases. We should harden validation across the board for 
> all known schema requirements.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-Tests-master - Build # 1351 - Still Failing

[Apache Jenkins Server]

Build: https://builds.apache.org/job/Lucene-Solr-Tests-master/1351/

No tests ran.

Build Log:
[...truncated 21 lines...]
FATAL: Cannot find executable from the chosen Ant installation "Ant 1.8.4"
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
ERROR: Step ‘Publish JUnit test result report’ failed: No test report files 
were found. Configuration error?
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any




-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9405) ConcurrentModificationException in ZkStateReader.getStateWatchers

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15418744#comment-15418744
 ] 

ASF subversion and git services commented on SOLR-9405:
---

Commit 42254c2d9b0e93a96e280b44b2fcbd4b9b6693af in lucene-solr's branch 
refs/heads/branch_6x from [~shalinmangar]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=42254c2 ]

SOLR-9405: ConcurrentModificationException in ZkStateReader.getStateWatchers
(cherry picked from commit f82c3b1)


> ConcurrentModificationException in ZkStateReader.getStateWatchers
> -
>
> Key: SOLR-9405
> URL: https://issues.apache.org/jira/browse/SOLR-9405
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Affects Versions: 6.1
>Reporter: Shalin Shekhar Mangar
>Assignee: Alan Woodward
> Fix For: 6.2, master (7.0)
>
> Attachments: SOLR-9405.patch, SOLR-9405.patch
>
>
> Jenkins found this: http://jenkins.thetaphi.de/job/Lucene-Solr-6.x-Linux/1432/
> {code}
> Stack Trace:
> java.util.ConcurrentModificationException
> at 
> __randomizedtesting.SeedInfo.seed([FA459DF725097EFF:A77E52876204E1C1]:0)
> at 
> java.util.HashMap$HashIterator.nextNode(java.base@9-ea/HashMap.java:1489)
> at 
> java.util.HashMap$KeyIterator.next(java.base@9-ea/HashMap.java:1513)
> at 
> java.util.AbstractCollection.addAll(java.base@9-ea/AbstractCollection.java:351)
> at java.util.HashSet.(java.base@9-ea/HashSet.java:119)
> at 
> org.apache.solr.common.cloud.ZkStateReader.getStateWatchers(ZkStateReader.java:1279)
> at 
> org.apache.solr.common.cloud.TestCollectionStateWatchers.testSimpleCollectionWatch(TestCollectionStateWatchers.java:116)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9405) ConcurrentModificationException in ZkStateReader.getStateWatchers

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15418739#comment-15418739
 ] 

ASF subversion and git services commented on SOLR-9405:
---

Commit f82c3b1206011776c55867fb2b5027b824f99812 in lucene-solr's branch 
refs/heads/master from [~shalinmangar]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=f82c3b1 ]

SOLR-9405: ConcurrentModificationException in ZkStateReader.getStateWatchers


> ConcurrentModificationException in ZkStateReader.getStateWatchers
> -
>
> Key: SOLR-9405
> URL: https://issues.apache.org/jira/browse/SOLR-9405
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Affects Versions: 6.1
>Reporter: Shalin Shekhar Mangar
>Assignee: Alan Woodward
> Fix For: 6.2, master (7.0)
>
> Attachments: SOLR-9405.patch, SOLR-9405.patch
>
>
> Jenkins found this: http://jenkins.thetaphi.de/job/Lucene-Solr-6.x-Linux/1432/
> {code}
> Stack Trace:
> java.util.ConcurrentModificationException
> at 
> __randomizedtesting.SeedInfo.seed([FA459DF725097EFF:A77E52876204E1C1]:0)
> at 
> java.util.HashMap$HashIterator.nextNode(java.base@9-ea/HashMap.java:1489)
> at 
> java.util.HashMap$KeyIterator.next(java.base@9-ea/HashMap.java:1513)
> at 
> java.util.AbstractCollection.addAll(java.base@9-ea/AbstractCollection.java:351)
> at java.util.HashSet.(java.base@9-ea/HashSet.java:119)
> at 
> org.apache.solr.common.cloud.ZkStateReader.getStateWatchers(ZkStateReader.java:1279)
> at 
> org.apache.solr.common.cloud.TestCollectionStateWatchers.testSimpleCollectionWatch(TestCollectionStateWatchers.java:116)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-9405) ConcurrentModificationException in ZkStateReader.getStateWatchers

[Shalin Shekhar Mangar (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-9405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shalin Shekhar Mangar updated SOLR-9405:

Attachment: SOLR-9405.patch

Patch which changes the stateWatcher to a set backed by ConcurrentHashMap. It 
also includes the race condition fix that Alan had supplied for the 
TestCollectionStateWatchers in his last patch. I'll commit shortly.

> ConcurrentModificationException in ZkStateReader.getStateWatchers
> -
>
> Key: SOLR-9405
> URL: https://issues.apache.org/jira/browse/SOLR-9405
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Affects Versions: 6.1
>Reporter: Shalin Shekhar Mangar
>Assignee: Alan Woodward
> Fix For: 6.2, master (7.0)
>
> Attachments: SOLR-9405.patch, SOLR-9405.patch
>
>
> Jenkins found this: http://jenkins.thetaphi.de/job/Lucene-Solr-6.x-Linux/1432/
> {code}
> Stack Trace:
> java.util.ConcurrentModificationException
> at 
> __randomizedtesting.SeedInfo.seed([FA459DF725097EFF:A77E52876204E1C1]:0)
> at 
> java.util.HashMap$HashIterator.nextNode(java.base@9-ea/HashMap.java:1489)
> at 
> java.util.HashMap$KeyIterator.next(java.base@9-ea/HashMap.java:1513)
> at 
> java.util.AbstractCollection.addAll(java.base@9-ea/AbstractCollection.java:351)
> at java.util.HashSet.(java.base@9-ea/HashSet.java:119)
> at 
> org.apache.solr.common.cloud.ZkStateReader.getStateWatchers(ZkStateReader.java:1279)
> at 
> org.apache.solr.common.cloud.TestCollectionStateWatchers.testSimpleCollectionWatch(TestCollectionStateWatchers.java:116)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (LUCENE-7411) Regex Query with Backreferences

[Martin Braun (JIRA)]

Martin Braun created LUCENE-7411:


 Summary: Regex Query with Backreferences
 Key: LUCENE-7411
 URL: https://issues.apache.org/jira/browse/LUCENE-7411
 Project: Lucene - Core
  Issue Type: New Feature
  Components: core/search
Reporter: Martin Braun
Priority: Minor


Hi there,

I am currently working on a Regex Engine that supports Backreferences while not 
losing determinism. It uses Memory Occurence Automata (MOAs) in the engine 
which are more powerful than normal DFA/NFAs. The engine does no backtracking 
and recognizes Regexes that cannot be evaluated deterministically as malformed. 
It has become more and more mature in the last few weeks and I also implemented 
a Lucene Query that uses these Patterns in the background. Now my question is: 
Is there any interest for this work to be merged (or adapted) into Lucene core?

https://github.com/s4ke/moar

Usage example for the Lucene Query:

https://github.com/s4ke/moar/blob/master/lucene/src/test/java/com/github/s4ke/moar/lucene/query/test/MoarQueryTest.java#L126

Cheers,

Martin



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-Tests-6.x - Build # 408 - Still Failing

[Apache Jenkins Server]

Build: https://builds.apache.org/job/Lucene-Solr-Tests-6.x/408/

No tests ran.

Build Log:
[...truncated 21 lines...]
FATAL: Cannot find executable from the chosen Ant installation "Ant 1.8.4"
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
ERROR: Step ‘Publish JUnit test result report’ failed: No test report files 
were found. Configuration error?
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any




-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-9411) Better validation for Schema API

[JIRA]

Jan Høydahl created SOLR-9411:
-

 Summary: Better validation for Schema API
 Key: SOLR-9411
 URL: https://issues.apache.org/jira/browse/SOLR-9411
 Project: Solr
  Issue Type: Improvement
  Security Level: Public (Default Security Level. Issues are Public)
  Components: Schema and Analysis
Reporter: Jan Høydahl


Schema REST API needs better validation before doing changes.

* It should not be allowed to delete uniqueKey
* When adding a dynamic field the API should test that it begins or ends with 
{{*}}. Today the change succeeds, but you get errors later

These are two known cases. We should harden validation across the board for all 
known schema requirements.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9310) PeerSync fails on a node restart due to IndexFingerPrint mismatch

[Pushkar Raste (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15418720#comment-15418720
 ] 

Pushkar Raste commented on SOLR-9310:
-

Hesitation about your patch is it checks index fingerprint before applying
updates. Which Yonik agrees is not right.



> PeerSync fails on a node restart due to IndexFingerPrint mismatch
> -
>
> Key: SOLR-9310
> URL: https://issues.apache.org/jira/browse/SOLR-9310
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Pushkar Raste
>Assignee: Noble Paul
> Attachments: PeerSync_Experiment.patch, SOLR-9310.patch, 
> SOLR-9310.patch, SOLR-9310.patch, SOLR-9310.patch
>
>
> I found that Peer Sync fails if a node restarts and documents were indexed 
> while node was down. IndexFingerPrint check fails after recovering node 
> applies updates. 
> This happens only when node restarts and not if node just misses updates due 
> reason other than it being down.
> Please check attached patch for the test.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS-EA] Lucene-Solr-6.x-Linux (32bit/jdk-9-ea+129) - Build # 1446 - Still Unstable!

[Policeman Jenkins Server]

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-6.x-Linux/1446/
Java: 32bit/jdk-9-ea+129 -client -XX:+UseSerialGC

1 tests failed.
FAILED:  org.apache.solr.handler.TestReqParamsAPI.test

Error Message:
Could not get expected value  'f' for path 'params/fixed' full output: {   
"responseHeader":{ "status":0, "QTime":0},   "params":{ 
"add":"second", "a":"A val", "fixed":"changeit", "b":"B val", 
"wt":"json"},   "context":{ "webapp":"/ja_xfs", "path":"/dump1", 
"httpMethod":"GET"}},  from server:  http://127.0.0.1:33109/ja_xfs/collection1

Stack Trace:
java.lang.AssertionError: Could not get expected value  'f' for path 
'params/fixed' full output: {
  "responseHeader":{
"status":0,
"QTime":0},
  "params":{
"add":"second",
"a":"A val",
"fixed":"changeit",
"b":"B val",
"wt":"json"},
  "context":{
"webapp":"/ja_xfs",
"path":"/dump1",
"httpMethod":"GET"}},  from server:  
http://127.0.0.1:33109/ja_xfs/collection1
at 
__randomizedtesting.SeedInfo.seed([8718913B0CB8BC80:F4CAEE1A244D178]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at 
org.apache.solr.core.TestSolrConfigHandler.testForResponseElement(TestSolrConfigHandler.java:535)
at 
org.apache.solr.handler.TestReqParamsAPI.testReqParams(TestReqParamsAPI.java:241)
at 
org.apache.solr.handler.TestReqParamsAPI.test(TestReqParamsAPI.java:61)
at 
jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(java.base@9-ea/Native 
Method)
at 
jdk.internal.reflect.NativeMethodAccessorImpl.invoke(java.base@9-ea/NativeMethodAccessorImpl.java:62)
at 
jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(java.base@9-ea/DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(java.base@9-ea/Method.java:533)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.eva

[JENKINS] Lucene-Solr-Tests-master - Build # 1350 - Still Failing

[Apache Jenkins Server]

Build: https://builds.apache.org/job/Lucene-Solr-Tests-master/1350/

No tests ran.

Build Log:
[...truncated 21 lines...]
FATAL: Cannot find executable from the chosen Ant installation "Ant 1.8.4"
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
ERROR: Step ‘Publish JUnit test result report’ failed: No test report files 
were found. Configuration error?
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any




-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7409) Look into making mmapdirectory's unmap safer

[Robert Muir (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15418695#comment-15418695
 ] 

Robert Muir commented on LUCENE-7409:
-

I worried about that too Dawid. So I ran the test in a variety of ways, with 
-Xcomp, with tests.iters to reuse the same jvm over and over, and so on. Not 
saying thats perfect testing of those situations, but it still worked :)



> Look into making mmapdirectory's unmap safer
> 
>
> Key: LUCENE-7409
> URL: https://issues.apache.org/jira/browse/LUCENE-7409
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Robert Muir
>Assignee: Uwe Schindler
> Attachments: StoreStore.java
>
>
> I have seen a few bugs around this recently: of course its a bug in 
> application code but a JVM crash is not good. 
> I think we should see if we can prevent the crashes better than the current 
> weak map, e.g. make it a safer option.
> I made an ugly prototype here: 
> https://github.com/apache/lucene-solr/compare/master...rmuir:ace?expand=1
> It has a test that crashes the JVM without the patch but passes with.
> Hacky patch only implements readBytes() but has no problems with the 
> luceneutil benchmark (1M):
> {noformat}
> Report after iter 19:
> TaskQPS base  StdDev   QPS patch  StdDev  
>   Pct diff
>   IntNRQ  105.23 (17.6%)  100.42 (10.1%)   
> -4.6% ( -27% -   28%)
>  Respell  128.35 (13.2%)  125.88  (7.4%)   
> -1.9% ( -19% -   21%)
>   Fuzzy1  110.14 (17.2%)  108.28 (13.2%)   
> -1.7% ( -27% -   34%)
>LowPhrase  337.02 (13.0%)  333.72  (9.3%)   
> -1.0% ( -20% -   24%)
>MedPhrase  146.44 (12.9%)  145.55  (8.0%)   
> -0.6% ( -19% -   23%)
>  MedSpanNear   96.85 (13.1%)   96.57  (7.8%)   
> -0.3% ( -18% -   23%)
> HighSpanNear   95.85 (13.9%)   96.33  (8.2%)
> 0.5% ( -18% -   26%)
>   HighPhrase  146.84 (13.6%)  148.40  (8.4%)
> 1.1% ( -18% -   26%)
> HighTerm  295.15 (15.8%)  298.77  (9.5%)
> 1.2% ( -20% -   31%)
>  LowSpanNear  268.80 (12.4%)  272.16  (7.9%)
> 1.2% ( -16% -   24%)
> Wildcard  284.09 (11.7%)  290.91  (8.9%)
> 2.4% ( -16% -   25%)
>  Prefix3  212.50 (15.4%)  217.76 (10.0%)
> 2.5% ( -19% -   32%)
>OrHighLow  358.65 (15.0%)  368.93 (10.7%)
> 2.9% ( -19% -   33%)
>   AndHighMed  799.65 (13.2%)  834.74  (7.8%)
> 4.4% ( -14% -   29%)
>  MedSloppyPhrase  229.36 (15.9%)  239.95  (9.8%)
> 4.6% ( -18% -   36%)
>   Fuzzy2   69.58 (14.6%)   72.82 (14.5%)
> 4.7% ( -21% -   39%)
>  AndHighHigh  426.98 (12.8%)  451.77  (7.3%)
> 5.8% ( -12% -   29%)
>  MedTerm 1361.11 (14.5%) 1450.90  (9.2%)
> 6.6% ( -14% -   35%)
> PKLookup  266.61 (13.4%)  284.28  (8.4%)
> 6.6% ( -13% -   32%)
> HighSloppyPhrase  251.22 (16.9%)  268.32 (10.7%)
> 6.8% ( -17% -   41%)
>OrHighMed  235.92 (17.2%)  253.12 (12.8%)
> 7.3% ( -19% -   45%)
>   OrHighHigh  186.79 (13.5%)  201.15  (9.7%)
> 7.7% ( -13% -   35%)
>  LowSloppyPhrase  395.23 (15.9%)  425.93  (9.3%)
> 7.8% ( -15% -   39%)
>   AndHighLow 1128.28 (14.9%) 1242.11  (8.2%)   
> 10.1% ( -11% -   38%)
>  LowTerm 3024.62 (12.9%) 3367.65  (9.7%)   
> 11.3% (  -9% -   39%)
> {noformat}
> We should do more testing. Maybe its totally the wrong tradeoff, maybe we 
> only need handles for getters and everything inlines correctly, rather than 
> needing a ton for every getXYZ() method...



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9241) Rebalance API for SolrCloud

[Noble Paul (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15418692#comment-15418692
 ] 

Noble Paul commented on SOLR-9241:
--

It will be released in the next version after it is committed

> Rebalance API for SolrCloud
> ---
>
> Key: SOLR-9241
> URL: https://issues.apache.org/jira/browse/SOLR-9241
> Project: Solr
>  Issue Type: New Feature
>  Components: SolrCloud
>Affects Versions: 6.1
> Environment: Ubuntu, Mac OsX
>Reporter: Nitin Sharma
>  Labels: Cluster, SolrCloud
> Fix For: 6.1
>
> Attachments: Redistribute_After.jpeg, Redistribute_Before.jpeg, 
> Redistribute_call.jpeg, Replace_After.jpeg, Replace_Before.jpeg, 
> Replace_Call.jpeg, SOLR-9241-4.6.patch, SOLR-9241-6.1.patch
>
>   Original Estimate: 2,016h
>  Remaining Estimate: 2,016h
>
> This is the v1 of the patch for Solrcloud Rebalance api (as described in 
> http://engineering.bloomreach.com/solrcloud-rebalance-api/) , built at 
> Bloomreach by Nitin Sharma and Suruchi Shah. The goal of the API  is to 
> provide a zero downtime mechanism to perform data manipulation and  efficient 
> core allocation in solrcloud. This API was envisioned to be the base layer 
> that enables Solrcloud to be an auto scaling platform. (and work in unison 
> with other complementing monitoring and scaling features).
> Patch Status:
> ===
> The patch is work in progress and incremental. We have done a few rounds of 
> code clean up. We wanted to get the patch going first to get initial feed 
> back.  We will continue to work on making it more open source friendly and 
> easily testable.
>  Deployment Status:
> 
> The platform is deployed in production at bloomreach and has been battle 
> tested for large scale load. (millions of documents and hundreds of 
> collections).
>  Internals:
> =
> The internals of the API and performance : 
> http://engineering.bloomreach.com/solrcloud-rebalance-api/
> It is built on top of the admin collections API as an action (with various 
> flavors). At a high level, the rebalance api provides 2 constructs:
> Scaling Strategy:  Decides how to move the data.  Every flavor has multiple 
> options which can be reviewed in the api spec.
> Re-distribute  - Move around data in the cluster based on capacity/allocation.
> Auto Shard  - Dynamically shard a collection to any size.
> Smart Merge - Distributed Mode - Helps merging data from a larger shard setup 
> into smaller one.  (the source should be divisible by destination)
> Scale up -  Add replicas on the fly
> Scale Down - Remove replicas on the fly
> Allocation Strategy:  Decides where to put the data.  (Nodes with least 
> cores, Nodes that do not have this collection etc). Custom implementations 
> can be built on top as well. One other example is Availability Zone aware. 
> Distribute data such that every replica is placed on different availability 
> zone to support HA.
>  Detailed API Spec:
> 
>   https://github.com/bloomreach/solrcloud-rebalance-api
>  Contributors:
> =
>   Nitin Sharma
>   Suruchi Shah
>  Questions/Comments:
> =
>   You can reach me at nitin.sha...@bloomreach.com



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-master-Solaris (64bit/jdk1.8.0) - Build # 775 - Still Unstable!

[Policeman Jenkins Server]

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-master-Solaris/775/
Java: 64bit/jdk1.8.0 -XX:-UseCompressedOops -XX:+UseConcMarkSweepGC

2 tests failed.
FAILED:  org.apache.solr.cloud.LeaderFailoverAfterPartitionTest.test

Error Message:
Expected 2 of 3 replicas to be active but only found 1; 
[core_node3:{"core":"c8n_1x3_lf_shard1_replica3","base_url":"http://127.0.0.1:53076","node_name":"127.0.0.1:53076_","state":"active","leader":"true"}];
 clusterState: DocCollection(c8n_1x3_lf//clusterstate.json/35)={   
"replicationFactor":"3",   "shards":{"shard1":{   
"range":"8000-7fff",   "state":"active",   "replicas":{ 
"core_node1":{   "core":"c8n_1x3_lf_shard1_replica1",   
"base_url":"http://127.0.0.1:63193";,   "node_name":"127.0.0.1:63193_",  
 "state":"down"}, "core_node2":{   "state":"down",  
 "base_url":"http://127.0.0.1:58115";,   
"core":"c8n_1x3_lf_shard1_replica2",   "node_name":"127.0.0.1:58115_"}, 
"core_node3":{   "core":"c8n_1x3_lf_shard1_replica3",   
"base_url":"http://127.0.0.1:53076";,   "node_name":"127.0.0.1:53076_",  
 "state":"active",   "leader":"true",   
"router":{"name":"compositeId"},   "maxShardsPerNode":"1",   
"autoAddReplicas":"false"}

Stack Trace:
java.lang.AssertionError: Expected 2 of 3 replicas to be active but only found 
1; 
[core_node3:{"core":"c8n_1x3_lf_shard1_replica3","base_url":"http://127.0.0.1:53076","node_name":"127.0.0.1:53076_","state":"active","leader":"true"}];
 clusterState: DocCollection(c8n_1x3_lf//clusterstate.json/35)={
  "replicationFactor":"3",
  "shards":{"shard1":{
  "range":"8000-7fff",
  "state":"active",
  "replicas":{
"core_node1":{
  "core":"c8n_1x3_lf_shard1_replica1",
  "base_url":"http://127.0.0.1:63193";,
  "node_name":"127.0.0.1:63193_",
  "state":"down"},
"core_node2":{
  "state":"down",
  "base_url":"http://127.0.0.1:58115";,
  "core":"c8n_1x3_lf_shard1_replica2",
  "node_name":"127.0.0.1:58115_"},
"core_node3":{
  "core":"c8n_1x3_lf_shard1_replica3",
  "base_url":"http://127.0.0.1:53076";,
  "node_name":"127.0.0.1:53076_",
  "state":"active",
  "leader":"true",
  "router":{"name":"compositeId"},
  "maxShardsPerNode":"1",
  "autoAddReplicas":"false"}
at 
__randomizedtesting.SeedInfo.seed([314201FB93ABC85D:B9163E213D57A5A5]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at 
org.apache.solr.cloud.LeaderFailoverAfterPartitionTest.testRf3WithLeaderFailover(LeaderFailoverAfterPartitionTest.java:170)
at 
org.apache.solr.cloud.LeaderFailoverAfterPartitionTest.test(LeaderFailoverAfterPartitionTest.java:57)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:985)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:960)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakCo

[jira] [Commented] (SOLR-9241) Rebalance API for SolrCloud

[Yago Riveiro (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15418672#comment-15418672
 ] 

Yago Riveiro commented on SOLR-9241:


This feature will be released in 6.x branch or will be a 7.x feature?

> Rebalance API for SolrCloud
> ---
>
> Key: SOLR-9241
> URL: https://issues.apache.org/jira/browse/SOLR-9241
> Project: Solr
>  Issue Type: New Feature
>  Components: SolrCloud
>Affects Versions: 6.1
> Environment: Ubuntu, Mac OsX
>Reporter: Nitin Sharma
>  Labels: Cluster, SolrCloud
> Fix For: 6.1
>
> Attachments: Redistribute_After.jpeg, Redistribute_Before.jpeg, 
> Redistribute_call.jpeg, Replace_After.jpeg, Replace_Before.jpeg, 
> Replace_Call.jpeg, SOLR-9241-4.6.patch, SOLR-9241-6.1.patch
>
>   Original Estimate: 2,016h
>  Remaining Estimate: 2,016h
>
> This is the v1 of the patch for Solrcloud Rebalance api (as described in 
> http://engineering.bloomreach.com/solrcloud-rebalance-api/) , built at 
> Bloomreach by Nitin Sharma and Suruchi Shah. The goal of the API  is to 
> provide a zero downtime mechanism to perform data manipulation and  efficient 
> core allocation in solrcloud. This API was envisioned to be the base layer 
> that enables Solrcloud to be an auto scaling platform. (and work in unison 
> with other complementing monitoring and scaling features).
> Patch Status:
> ===
> The patch is work in progress and incremental. We have done a few rounds of 
> code clean up. We wanted to get the patch going first to get initial feed 
> back.  We will continue to work on making it more open source friendly and 
> easily testable.
>  Deployment Status:
> 
> The platform is deployed in production at bloomreach and has been battle 
> tested for large scale load. (millions of documents and hundreds of 
> collections).
>  Internals:
> =
> The internals of the API and performance : 
> http://engineering.bloomreach.com/solrcloud-rebalance-api/
> It is built on top of the admin collections API as an action (with various 
> flavors). At a high level, the rebalance api provides 2 constructs:
> Scaling Strategy:  Decides how to move the data.  Every flavor has multiple 
> options which can be reviewed in the api spec.
> Re-distribute  - Move around data in the cluster based on capacity/allocation.
> Auto Shard  - Dynamically shard a collection to any size.
> Smart Merge - Distributed Mode - Helps merging data from a larger shard setup 
> into smaller one.  (the source should be divisible by destination)
> Scale up -  Add replicas on the fly
> Scale Down - Remove replicas on the fly
> Allocation Strategy:  Decides where to put the data.  (Nodes with least 
> cores, Nodes that do not have this collection etc). Custom implementations 
> can be built on top as well. One other example is Availability Zone aware. 
> Distribute data such that every replica is placed on different availability 
> zone to support HA.
>  Detailed API Spec:
> 
>   https://github.com/bloomreach/solrcloud-rebalance-api
>  Contributors:
> =
>   Nitin Sharma
>   Suruchi Shah
>  Questions/Comments:
> =
>   You can reach me at nitin.sha...@bloomreach.com



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-Tests-6.x - Build # 407 - Still Failing

[Apache Jenkins Server]

Build: https://builds.apache.org/job/Lucene-Solr-Tests-6.x/407/

No tests ran.

Build Log:
[...truncated 21 lines...]
FATAL: Cannot find executable from the chosen Ant installation "Ant 1.8.4"
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
ERROR: Step ‘Publish JUnit test result report’ failed: No test report files 
were found. Configuration error?
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any




-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7409) Look into making mmapdirectory's unmap safer

[Dawid Weiss (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15418648#comment-15418648
 ] 

Dawid Weiss commented on LUCENE-7409:
-

bq. Its just a better best-effort 

Absolutely. I don't know how to do it better (other than providing a 
coarse-grained buffered reads synchronized using heavier locks).

As for tests, I'd be curious when it fails (and if). :) Uwe also pointed out 
that we don't know (and can't be sure of) what the test is actually running -- 
with which optimizations, when, etc. It may be passing because it didn't 
collect enough stats yet to optimize... This is absurdly complicated when you 
look at such fine-details.

> Look into making mmapdirectory's unmap safer
> 
>
> Key: LUCENE-7409
> URL: https://issues.apache.org/jira/browse/LUCENE-7409
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Robert Muir
>Assignee: Uwe Schindler
> Attachments: StoreStore.java
>
>
> I have seen a few bugs around this recently: of course its a bug in 
> application code but a JVM crash is not good. 
> I think we should see if we can prevent the crashes better than the current 
> weak map, e.g. make it a safer option.
> I made an ugly prototype here: 
> https://github.com/apache/lucene-solr/compare/master...rmuir:ace?expand=1
> It has a test that crashes the JVM without the patch but passes with.
> Hacky patch only implements readBytes() but has no problems with the 
> luceneutil benchmark (1M):
> {noformat}
> Report after iter 19:
> TaskQPS base  StdDev   QPS patch  StdDev  
>   Pct diff
>   IntNRQ  105.23 (17.6%)  100.42 (10.1%)   
> -4.6% ( -27% -   28%)
>  Respell  128.35 (13.2%)  125.88  (7.4%)   
> -1.9% ( -19% -   21%)
>   Fuzzy1  110.14 (17.2%)  108.28 (13.2%)   
> -1.7% ( -27% -   34%)
>LowPhrase  337.02 (13.0%)  333.72  (9.3%)   
> -1.0% ( -20% -   24%)
>MedPhrase  146.44 (12.9%)  145.55  (8.0%)   
> -0.6% ( -19% -   23%)
>  MedSpanNear   96.85 (13.1%)   96.57  (7.8%)   
> -0.3% ( -18% -   23%)
> HighSpanNear   95.85 (13.9%)   96.33  (8.2%)
> 0.5% ( -18% -   26%)
>   HighPhrase  146.84 (13.6%)  148.40  (8.4%)
> 1.1% ( -18% -   26%)
> HighTerm  295.15 (15.8%)  298.77  (9.5%)
> 1.2% ( -20% -   31%)
>  LowSpanNear  268.80 (12.4%)  272.16  (7.9%)
> 1.2% ( -16% -   24%)
> Wildcard  284.09 (11.7%)  290.91  (8.9%)
> 2.4% ( -16% -   25%)
>  Prefix3  212.50 (15.4%)  217.76 (10.0%)
> 2.5% ( -19% -   32%)
>OrHighLow  358.65 (15.0%)  368.93 (10.7%)
> 2.9% ( -19% -   33%)
>   AndHighMed  799.65 (13.2%)  834.74  (7.8%)
> 4.4% ( -14% -   29%)
>  MedSloppyPhrase  229.36 (15.9%)  239.95  (9.8%)
> 4.6% ( -18% -   36%)
>   Fuzzy2   69.58 (14.6%)   72.82 (14.5%)
> 4.7% ( -21% -   39%)
>  AndHighHigh  426.98 (12.8%)  451.77  (7.3%)
> 5.8% ( -12% -   29%)
>  MedTerm 1361.11 (14.5%) 1450.90  (9.2%)
> 6.6% ( -14% -   35%)
> PKLookup  266.61 (13.4%)  284.28  (8.4%)
> 6.6% ( -13% -   32%)
> HighSloppyPhrase  251.22 (16.9%)  268.32 (10.7%)
> 6.8% ( -17% -   41%)
>OrHighMed  235.92 (17.2%)  253.12 (12.8%)
> 7.3% ( -19% -   45%)
>   OrHighHigh  186.79 (13.5%)  201.15  (9.7%)
> 7.7% ( -13% -   35%)
>  LowSloppyPhrase  395.23 (15.9%)  425.93  (9.3%)
> 7.8% ( -15% -   39%)
>   AndHighLow 1128.28 (14.9%) 1242.11  (8.2%)   
> 10.1% ( -11% -   38%)
>  LowTerm 3024.62 (12.9%) 3367.65  (9.7%)   
> 11.3% (  -9% -   39%)
> {noformat}
> We should do more testing. Maybe its totally the wrong tradeoff, maybe we 
> only need handles for getters and everything inlines correctly, rather than 
> needing a ton for every getXYZ() method...



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7409) Look into making mmapdirectory's unmap safer

[Robert Muir (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15418642#comment-15418642
 ] 

Robert Muir commented on LUCENE-7409:
-

Its just a better best-effort :) Maybe we should remove the test, since there 
are no guarantees here? First time it fails, what will we do? we will just 
disable it.

> Look into making mmapdirectory's unmap safer
> 
>
> Key: LUCENE-7409
> URL: https://issues.apache.org/jira/browse/LUCENE-7409
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Robert Muir
>Assignee: Uwe Schindler
> Attachments: StoreStore.java
>
>
> I have seen a few bugs around this recently: of course its a bug in 
> application code but a JVM crash is not good. 
> I think we should see if we can prevent the crashes better than the current 
> weak map, e.g. make it a safer option.
> I made an ugly prototype here: 
> https://github.com/apache/lucene-solr/compare/master...rmuir:ace?expand=1
> It has a test that crashes the JVM without the patch but passes with.
> Hacky patch only implements readBytes() but has no problems with the 
> luceneutil benchmark (1M):
> {noformat}
> Report after iter 19:
> TaskQPS base  StdDev   QPS patch  StdDev  
>   Pct diff
>   IntNRQ  105.23 (17.6%)  100.42 (10.1%)   
> -4.6% ( -27% -   28%)
>  Respell  128.35 (13.2%)  125.88  (7.4%)   
> -1.9% ( -19% -   21%)
>   Fuzzy1  110.14 (17.2%)  108.28 (13.2%)   
> -1.7% ( -27% -   34%)
>LowPhrase  337.02 (13.0%)  333.72  (9.3%)   
> -1.0% ( -20% -   24%)
>MedPhrase  146.44 (12.9%)  145.55  (8.0%)   
> -0.6% ( -19% -   23%)
>  MedSpanNear   96.85 (13.1%)   96.57  (7.8%)   
> -0.3% ( -18% -   23%)
> HighSpanNear   95.85 (13.9%)   96.33  (8.2%)
> 0.5% ( -18% -   26%)
>   HighPhrase  146.84 (13.6%)  148.40  (8.4%)
> 1.1% ( -18% -   26%)
> HighTerm  295.15 (15.8%)  298.77  (9.5%)
> 1.2% ( -20% -   31%)
>  LowSpanNear  268.80 (12.4%)  272.16  (7.9%)
> 1.2% ( -16% -   24%)
> Wildcard  284.09 (11.7%)  290.91  (8.9%)
> 2.4% ( -16% -   25%)
>  Prefix3  212.50 (15.4%)  217.76 (10.0%)
> 2.5% ( -19% -   32%)
>OrHighLow  358.65 (15.0%)  368.93 (10.7%)
> 2.9% ( -19% -   33%)
>   AndHighMed  799.65 (13.2%)  834.74  (7.8%)
> 4.4% ( -14% -   29%)
>  MedSloppyPhrase  229.36 (15.9%)  239.95  (9.8%)
> 4.6% ( -18% -   36%)
>   Fuzzy2   69.58 (14.6%)   72.82 (14.5%)
> 4.7% ( -21% -   39%)
>  AndHighHigh  426.98 (12.8%)  451.77  (7.3%)
> 5.8% ( -12% -   29%)
>  MedTerm 1361.11 (14.5%) 1450.90  (9.2%)
> 6.6% ( -14% -   35%)
> PKLookup  266.61 (13.4%)  284.28  (8.4%)
> 6.6% ( -13% -   32%)
> HighSloppyPhrase  251.22 (16.9%)  268.32 (10.7%)
> 6.8% ( -17% -   41%)
>OrHighMed  235.92 (17.2%)  253.12 (12.8%)
> 7.3% ( -19% -   45%)
>   OrHighHigh  186.79 (13.5%)  201.15  (9.7%)
> 7.7% ( -13% -   35%)
>  LowSloppyPhrase  395.23 (15.9%)  425.93  (9.3%)
> 7.8% ( -15% -   39%)
>   AndHighLow 1128.28 (14.9%) 1242.11  (8.2%)   
> 10.1% ( -11% -   38%)
>  LowTerm 3024.62 (12.9%) 3367.65  (9.7%)   
> 11.3% (  -9% -   39%)
> {noformat}
> We should do more testing. Maybe its totally the wrong tradeoff, maybe we 
> only need handles for getters and everything inlines correctly, rather than 
> needing a ton for every getXYZ() method...



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9405) ConcurrentModificationException in ZkStateReader.getStateWatchers

[Shalin Shekhar Mangar (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15418628#comment-15418628
 ] 

Shalin Shekhar Mangar commented on SOLR-9405:
-

Thanks Edward. Yes that's what I was going for. We need to make the 
stateWatchers thread-safe either by wrapping with synchronizedMap or by using a 
set backed by ConcurrentHashMap. I'll put up a patch.

Have fun, Alan!

> ConcurrentModificationException in ZkStateReader.getStateWatchers
> -
>
> Key: SOLR-9405
> URL: https://issues.apache.org/jira/browse/SOLR-9405
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Affects Versions: 6.1
>Reporter: Shalin Shekhar Mangar
>Assignee: Alan Woodward
> Fix For: 6.2, master (7.0)
>
> Attachments: SOLR-9405.patch
>
>
> Jenkins found this: http://jenkins.thetaphi.de/job/Lucene-Solr-6.x-Linux/1432/
> {code}
> Stack Trace:
> java.util.ConcurrentModificationException
> at 
> __randomizedtesting.SeedInfo.seed([FA459DF725097EFF:A77E52876204E1C1]:0)
> at 
> java.util.HashMap$HashIterator.nextNode(java.base@9-ea/HashMap.java:1489)
> at 
> java.util.HashMap$KeyIterator.next(java.base@9-ea/HashMap.java:1513)
> at 
> java.util.AbstractCollection.addAll(java.base@9-ea/AbstractCollection.java:351)
> at java.util.HashSet.(java.base@9-ea/HashSet.java:119)
> at 
> org.apache.solr.common.cloud.ZkStateReader.getStateWatchers(ZkStateReader.java:1279)
> at 
> org.apache.solr.common.cloud.TestCollectionStateWatchers.testSimpleCollectionWatch(TestCollectionStateWatchers.java:116)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-Tests-master - Build # 1349 - Still Failing

[Apache Jenkins Server]

Build: https://builds.apache.org/job/Lucene-Solr-Tests-master/1349/

No tests ran.

Build Log:
[...truncated 21 lines...]
FATAL: Cannot find executable from the chosen Ant installation "Ant 1.8.4"
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
ERROR: Step ‘Publish JUnit test result report’ failed: No test report files 
were found. Configuration error?
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any




-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7409) Look into making mmapdirectory's unmap safer

[Dawid Weiss (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15418617#comment-15418617
 ] 

Dawid Weiss commented on LUCENE-7409:
-

bq. Did you try your quick program with a non-lazy set (store-load barrier)?

I don't have to -- it'll hang as well. That was my point. If there is a heavy 
loop then any access.* calls won't invalidate anything since the 
{{ensureValid}} method can be inlined and the read of {{invalidated}} can be 
moved outside of the loop (since the loop body doesn't change its value such an 
optimization is valid). If you dump the assembly of the program I attached 
you'll see this is the case here.

In practice, if bytebuffer is used inside the loop the optimization will most 
likely not take place due to inline limits and I/O calls inside. But it doesn't 
make the solution correct -- only "practically correct". :)

As for store-store vs. store-load: I think you should use store-load (because 
you wish other "readers" to pick up changes from cache, not just serialize 
stores to memory). But my understanding of this may be wrong as I always found 
this a bit confusing.

> Look into making mmapdirectory's unmap safer
> 
>
> Key: LUCENE-7409
> URL: https://issues.apache.org/jira/browse/LUCENE-7409
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Robert Muir
>Assignee: Uwe Schindler
> Attachments: StoreStore.java
>
>
> I have seen a few bugs around this recently: of course its a bug in 
> application code but a JVM crash is not good. 
> I think we should see if we can prevent the crashes better than the current 
> weak map, e.g. make it a safer option.
> I made an ugly prototype here: 
> https://github.com/apache/lucene-solr/compare/master...rmuir:ace?expand=1
> It has a test that crashes the JVM without the patch but passes with.
> Hacky patch only implements readBytes() but has no problems with the 
> luceneutil benchmark (1M):
> {noformat}
> Report after iter 19:
> TaskQPS base  StdDev   QPS patch  StdDev  
>   Pct diff
>   IntNRQ  105.23 (17.6%)  100.42 (10.1%)   
> -4.6% ( -27% -   28%)
>  Respell  128.35 (13.2%)  125.88  (7.4%)   
> -1.9% ( -19% -   21%)
>   Fuzzy1  110.14 (17.2%)  108.28 (13.2%)   
> -1.7% ( -27% -   34%)
>LowPhrase  337.02 (13.0%)  333.72  (9.3%)   
> -1.0% ( -20% -   24%)
>MedPhrase  146.44 (12.9%)  145.55  (8.0%)   
> -0.6% ( -19% -   23%)
>  MedSpanNear   96.85 (13.1%)   96.57  (7.8%)   
> -0.3% ( -18% -   23%)
> HighSpanNear   95.85 (13.9%)   96.33  (8.2%)
> 0.5% ( -18% -   26%)
>   HighPhrase  146.84 (13.6%)  148.40  (8.4%)
> 1.1% ( -18% -   26%)
> HighTerm  295.15 (15.8%)  298.77  (9.5%)
> 1.2% ( -20% -   31%)
>  LowSpanNear  268.80 (12.4%)  272.16  (7.9%)
> 1.2% ( -16% -   24%)
> Wildcard  284.09 (11.7%)  290.91  (8.9%)
> 2.4% ( -16% -   25%)
>  Prefix3  212.50 (15.4%)  217.76 (10.0%)
> 2.5% ( -19% -   32%)
>OrHighLow  358.65 (15.0%)  368.93 (10.7%)
> 2.9% ( -19% -   33%)
>   AndHighMed  799.65 (13.2%)  834.74  (7.8%)
> 4.4% ( -14% -   29%)
>  MedSloppyPhrase  229.36 (15.9%)  239.95  (9.8%)
> 4.6% ( -18% -   36%)
>   Fuzzy2   69.58 (14.6%)   72.82 (14.5%)
> 4.7% ( -21% -   39%)
>  AndHighHigh  426.98 (12.8%)  451.77  (7.3%)
> 5.8% ( -12% -   29%)
>  MedTerm 1361.11 (14.5%) 1450.90  (9.2%)
> 6.6% ( -14% -   35%)
> PKLookup  266.61 (13.4%)  284.28  (8.4%)
> 6.6% ( -13% -   32%)
> HighSloppyPhrase  251.22 (16.9%)  268.32 (10.7%)
> 6.8% ( -17% -   41%)
>OrHighMed  235.92 (17.2%)  253.12 (12.8%)
> 7.3% ( -19% -   45%)
>   OrHighHigh  186.79 (13.5%)  201.15  (9.7%)
> 7.7% ( -13% -   35%)
>  LowSloppyPhrase  395.23 (15.9%)  425.93  (9.3%)
> 7.8% ( -15% -   39%)
>   AndHighLow 1128.28 (14.9%) 1242.11  (8.2%)   
> 10.1% ( -11% -   38%)
>  LowTerm 3024.62 (12.9%) 3367.65  (9.7%)   
> 11.3% (  -9% -   39%)
> {noformat}
> We should do more testing. Maybe its totally the wrong tradeoff, maybe we 
> only need handles for getters and everything inlines correctly, rather than 
> 

[jira] [Updated] (SOLR-9410) make ReRankQParserPlugin's private ReRankWeight public and ReRankQuery part abstract

[Christine Poerschke (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-9410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Christine Poerschke updated SOLR-9410:
--
Attachment: SOLR-9410-ReRankWeight.patch

first of two proposed patches

> make ReRankQParserPlugin's private ReRankWeight public and ReRankQuery part 
> abstract
> 
>
> Key: SOLR-9410
> URL: https://issues.apache.org/jira/browse/SOLR-9410
> Project: Solr
>  Issue Type: Task
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Christine Poerschke
>Assignee: Christine Poerschke
>Priority: Minor
> Attachments: SOLR-9410-ReRankWeight.patch
>
>
> https://github.com/apache/lucene-solr/blob/master/solr/core/src/java/org/apache/solr/search/ReRankQParserPlugin.java



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-9410) make ReRankQParserPlugin's private ReRankWeight public and ReRankQuery part abstract

[Christine Poerschke (JIRA)]

Christine Poerschke created SOLR-9410:
-

 Summary: make ReRankQParserPlugin's private ReRankWeight public 
and ReRankQuery part abstract
 Key: SOLR-9410
 URL: https://issues.apache.org/jira/browse/SOLR-9410
 Project: Solr
  Issue Type: Task
  Security Level: Public (Default Security Level. Issues are Public)
Reporter: Christine Poerschke
Assignee: Christine Poerschke
Priority: Minor


https://github.com/apache/lucene-solr/blob/master/solr/core/src/java/org/apache/solr/search/ReRankQParserPlugin.java



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7409) Look into making mmapdirectory's unmap safer

[Uwe Schindler (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15418610#comment-15418610
 ] 

Uwe Schindler commented on LUCENE-7409:
---

bq. Did you try your quick program with a non-lazy set (store-load barrier)?

No it doesn't  - just tested :-)

> Look into making mmapdirectory's unmap safer
> 
>
> Key: LUCENE-7409
> URL: https://issues.apache.org/jira/browse/LUCENE-7409
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Robert Muir
>Assignee: Uwe Schindler
> Attachments: StoreStore.java
>
>
> I have seen a few bugs around this recently: of course its a bug in 
> application code but a JVM crash is not good. 
> I think we should see if we can prevent the crashes better than the current 
> weak map, e.g. make it a safer option.
> I made an ugly prototype here: 
> https://github.com/apache/lucene-solr/compare/master...rmuir:ace?expand=1
> It has a test that crashes the JVM without the patch but passes with.
> Hacky patch only implements readBytes() but has no problems with the 
> luceneutil benchmark (1M):
> {noformat}
> Report after iter 19:
> TaskQPS base  StdDev   QPS patch  StdDev  
>   Pct diff
>   IntNRQ  105.23 (17.6%)  100.42 (10.1%)   
> -4.6% ( -27% -   28%)
>  Respell  128.35 (13.2%)  125.88  (7.4%)   
> -1.9% ( -19% -   21%)
>   Fuzzy1  110.14 (17.2%)  108.28 (13.2%)   
> -1.7% ( -27% -   34%)
>LowPhrase  337.02 (13.0%)  333.72  (9.3%)   
> -1.0% ( -20% -   24%)
>MedPhrase  146.44 (12.9%)  145.55  (8.0%)   
> -0.6% ( -19% -   23%)
>  MedSpanNear   96.85 (13.1%)   96.57  (7.8%)   
> -0.3% ( -18% -   23%)
> HighSpanNear   95.85 (13.9%)   96.33  (8.2%)
> 0.5% ( -18% -   26%)
>   HighPhrase  146.84 (13.6%)  148.40  (8.4%)
> 1.1% ( -18% -   26%)
> HighTerm  295.15 (15.8%)  298.77  (9.5%)
> 1.2% ( -20% -   31%)
>  LowSpanNear  268.80 (12.4%)  272.16  (7.9%)
> 1.2% ( -16% -   24%)
> Wildcard  284.09 (11.7%)  290.91  (8.9%)
> 2.4% ( -16% -   25%)
>  Prefix3  212.50 (15.4%)  217.76 (10.0%)
> 2.5% ( -19% -   32%)
>OrHighLow  358.65 (15.0%)  368.93 (10.7%)
> 2.9% ( -19% -   33%)
>   AndHighMed  799.65 (13.2%)  834.74  (7.8%)
> 4.4% ( -14% -   29%)
>  MedSloppyPhrase  229.36 (15.9%)  239.95  (9.8%)
> 4.6% ( -18% -   36%)
>   Fuzzy2   69.58 (14.6%)   72.82 (14.5%)
> 4.7% ( -21% -   39%)
>  AndHighHigh  426.98 (12.8%)  451.77  (7.3%)
> 5.8% ( -12% -   29%)
>  MedTerm 1361.11 (14.5%) 1450.90  (9.2%)
> 6.6% ( -14% -   35%)
> PKLookup  266.61 (13.4%)  284.28  (8.4%)
> 6.6% ( -13% -   32%)
> HighSloppyPhrase  251.22 (16.9%)  268.32 (10.7%)
> 6.8% ( -17% -   41%)
>OrHighMed  235.92 (17.2%)  253.12 (12.8%)
> 7.3% ( -19% -   45%)
>   OrHighHigh  186.79 (13.5%)  201.15  (9.7%)
> 7.7% ( -13% -   35%)
>  LowSloppyPhrase  395.23 (15.9%)  425.93  (9.3%)
> 7.8% ( -15% -   39%)
>   AndHighLow 1128.28 (14.9%) 1242.11  (8.2%)   
> 10.1% ( -11% -   38%)
>  LowTerm 3024.62 (12.9%) 3367.65  (9.7%)   
> 11.3% (  -9% -   39%)
> {noformat}
> We should do more testing. Maybe its totally the wrong tradeoff, maybe we 
> only need handles for getters and everything inlines correctly, rather than 
> needing a ton for every getXYZ() method...



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7409) Look into making mmapdirectory's unmap safer

[Uwe Schindler (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15418607#comment-15418607
 ] 

Uwe Schindler commented on LUCENE-7409:
---

[~dawid.weiss]: Exactly that what I think it does. We cannot ensure that the 
other thread sees the change ASAP. This is why it originally also did not work 
with J9. The other trick we do here is Thread.yield() before the actual 
unmapping. This allows any "in flight" access to the bytebuffer to finish until 
our own thread gets schedulaed again. Of course this is also no guarantee, but 
helps with J9 and works around the race condition: the time between the check 
for invalidated until Unsafe read could allow another thread to unmap. The 
yield takes us out of scheduling.

Did you try your quick program with a non-lazy set (store-load barrier)?

> Look into making mmapdirectory's unmap safer
> 
>
> Key: LUCENE-7409
> URL: https://issues.apache.org/jira/browse/LUCENE-7409
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Robert Muir
>Assignee: Uwe Schindler
> Attachments: StoreStore.java
>
>
> I have seen a few bugs around this recently: of course its a bug in 
> application code but a JVM crash is not good. 
> I think we should see if we can prevent the crashes better than the current 
> weak map, e.g. make it a safer option.
> I made an ugly prototype here: 
> https://github.com/apache/lucene-solr/compare/master...rmuir:ace?expand=1
> It has a test that crashes the JVM without the patch but passes with.
> Hacky patch only implements readBytes() but has no problems with the 
> luceneutil benchmark (1M):
> {noformat}
> Report after iter 19:
> TaskQPS base  StdDev   QPS patch  StdDev  
>   Pct diff
>   IntNRQ  105.23 (17.6%)  100.42 (10.1%)   
> -4.6% ( -27% -   28%)
>  Respell  128.35 (13.2%)  125.88  (7.4%)   
> -1.9% ( -19% -   21%)
>   Fuzzy1  110.14 (17.2%)  108.28 (13.2%)   
> -1.7% ( -27% -   34%)
>LowPhrase  337.02 (13.0%)  333.72  (9.3%)   
> -1.0% ( -20% -   24%)
>MedPhrase  146.44 (12.9%)  145.55  (8.0%)   
> -0.6% ( -19% -   23%)
>  MedSpanNear   96.85 (13.1%)   96.57  (7.8%)   
> -0.3% ( -18% -   23%)
> HighSpanNear   95.85 (13.9%)   96.33  (8.2%)
> 0.5% ( -18% -   26%)
>   HighPhrase  146.84 (13.6%)  148.40  (8.4%)
> 1.1% ( -18% -   26%)
> HighTerm  295.15 (15.8%)  298.77  (9.5%)
> 1.2% ( -20% -   31%)
>  LowSpanNear  268.80 (12.4%)  272.16  (7.9%)
> 1.2% ( -16% -   24%)
> Wildcard  284.09 (11.7%)  290.91  (8.9%)
> 2.4% ( -16% -   25%)
>  Prefix3  212.50 (15.4%)  217.76 (10.0%)
> 2.5% ( -19% -   32%)
>OrHighLow  358.65 (15.0%)  368.93 (10.7%)
> 2.9% ( -19% -   33%)
>   AndHighMed  799.65 (13.2%)  834.74  (7.8%)
> 4.4% ( -14% -   29%)
>  MedSloppyPhrase  229.36 (15.9%)  239.95  (9.8%)
> 4.6% ( -18% -   36%)
>   Fuzzy2   69.58 (14.6%)   72.82 (14.5%)
> 4.7% ( -21% -   39%)
>  AndHighHigh  426.98 (12.8%)  451.77  (7.3%)
> 5.8% ( -12% -   29%)
>  MedTerm 1361.11 (14.5%) 1450.90  (9.2%)
> 6.6% ( -14% -   35%)
> PKLookup  266.61 (13.4%)  284.28  (8.4%)
> 6.6% ( -13% -   32%)
> HighSloppyPhrase  251.22 (16.9%)  268.32 (10.7%)
> 6.8% ( -17% -   41%)
>OrHighMed  235.92 (17.2%)  253.12 (12.8%)
> 7.3% ( -19% -   45%)
>   OrHighHigh  186.79 (13.5%)  201.15  (9.7%)
> 7.7% ( -13% -   35%)
>  LowSloppyPhrase  395.23 (15.9%)  425.93  (9.3%)
> 7.8% ( -15% -   39%)
>   AndHighLow 1128.28 (14.9%) 1242.11  (8.2%)   
> 10.1% ( -11% -   38%)
>  LowTerm 3024.62 (12.9%) 3367.65  (9.7%)   
> 11.3% (  -9% -   39%)
> {noformat}
> We should do more testing. Maybe its totally the wrong tradeoff, maybe we 
> only need handles for getters and everything inlines correctly, rather than 
> needing a ton for every getXYZ() method...



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-Tests-6.x - Build # 406 - Still Failing

[Apache Jenkins Server]

Build: https://builds.apache.org/job/Lucene-Solr-Tests-6.x/406/

No tests ran.

Build Log:
[...truncated 21 lines...]
FATAL: Cannot find executable from the chosen Ant installation "Ant 1.8.4"
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
ERROR: Step ‘Publish JUnit test result report’ failed: No test report files 
were found. Configuration error?
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any




-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7409) Look into making mmapdirectory's unmap safer

[Uwe Schindler (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15418603#comment-15418603
 ] 

Uwe Schindler commented on LUCENE-7409:
---

I did some minor cleanups, its ready now:
- renamed ByteBufferAccess to ByteBufferGuard
- made the barrier an instance field (this is more correct)
- I removed the assume for Oracle, because this also works with J9
- added some javadocs

> Look into making mmapdirectory's unmap safer
> 
>
> Key: LUCENE-7409
> URL: https://issues.apache.org/jira/browse/LUCENE-7409
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Robert Muir
>Assignee: Uwe Schindler
> Attachments: StoreStore.java
>
>
> I have seen a few bugs around this recently: of course its a bug in 
> application code but a JVM crash is not good. 
> I think we should see if we can prevent the crashes better than the current 
> weak map, e.g. make it a safer option.
> I made an ugly prototype here: 
> https://github.com/apache/lucene-solr/compare/master...rmuir:ace?expand=1
> It has a test that crashes the JVM without the patch but passes with.
> Hacky patch only implements readBytes() but has no problems with the 
> luceneutil benchmark (1M):
> {noformat}
> Report after iter 19:
> TaskQPS base  StdDev   QPS patch  StdDev  
>   Pct diff
>   IntNRQ  105.23 (17.6%)  100.42 (10.1%)   
> -4.6% ( -27% -   28%)
>  Respell  128.35 (13.2%)  125.88  (7.4%)   
> -1.9% ( -19% -   21%)
>   Fuzzy1  110.14 (17.2%)  108.28 (13.2%)   
> -1.7% ( -27% -   34%)
>LowPhrase  337.02 (13.0%)  333.72  (9.3%)   
> -1.0% ( -20% -   24%)
>MedPhrase  146.44 (12.9%)  145.55  (8.0%)   
> -0.6% ( -19% -   23%)
>  MedSpanNear   96.85 (13.1%)   96.57  (7.8%)   
> -0.3% ( -18% -   23%)
> HighSpanNear   95.85 (13.9%)   96.33  (8.2%)
> 0.5% ( -18% -   26%)
>   HighPhrase  146.84 (13.6%)  148.40  (8.4%)
> 1.1% ( -18% -   26%)
> HighTerm  295.15 (15.8%)  298.77  (9.5%)
> 1.2% ( -20% -   31%)
>  LowSpanNear  268.80 (12.4%)  272.16  (7.9%)
> 1.2% ( -16% -   24%)
> Wildcard  284.09 (11.7%)  290.91  (8.9%)
> 2.4% ( -16% -   25%)
>  Prefix3  212.50 (15.4%)  217.76 (10.0%)
> 2.5% ( -19% -   32%)
>OrHighLow  358.65 (15.0%)  368.93 (10.7%)
> 2.9% ( -19% -   33%)
>   AndHighMed  799.65 (13.2%)  834.74  (7.8%)
> 4.4% ( -14% -   29%)
>  MedSloppyPhrase  229.36 (15.9%)  239.95  (9.8%)
> 4.6% ( -18% -   36%)
>   Fuzzy2   69.58 (14.6%)   72.82 (14.5%)
> 4.7% ( -21% -   39%)
>  AndHighHigh  426.98 (12.8%)  451.77  (7.3%)
> 5.8% ( -12% -   29%)
>  MedTerm 1361.11 (14.5%) 1450.90  (9.2%)
> 6.6% ( -14% -   35%)
> PKLookup  266.61 (13.4%)  284.28  (8.4%)
> 6.6% ( -13% -   32%)
> HighSloppyPhrase  251.22 (16.9%)  268.32 (10.7%)
> 6.8% ( -17% -   41%)
>OrHighMed  235.92 (17.2%)  253.12 (12.8%)
> 7.3% ( -19% -   45%)
>   OrHighHigh  186.79 (13.5%)  201.15  (9.7%)
> 7.7% ( -13% -   35%)
>  LowSloppyPhrase  395.23 (15.9%)  425.93  (9.3%)
> 7.8% ( -15% -   39%)
>   AndHighLow 1128.28 (14.9%) 1242.11  (8.2%)   
> 10.1% ( -11% -   38%)
>  LowTerm 3024.62 (12.9%) 3367.65  (9.7%)   
> 11.3% (  -9% -   39%)
> {noformat}
> We should do more testing. Maybe its totally the wrong tradeoff, maybe we 
> only need handles for getters and everything inlines correctly, rather than 
> needing a ton for every getXYZ() method...



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org