[jira] [Comment Edited] (LUCENE-7013) Move license header before package declaration in all *.java files

[Steve Rowe (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15372207#comment-15372207
 ] 

Steve Rowe edited comment on LUCENE-7013 at 7/12/16 5:49 AM:
-

Patch to fail {{precommit}} (via {{-validate-source-patterns}}) when 
{{org.apache.*}} package declaration precedes the license header in {{.java}} 
files.


was (Author: steve_rowe):
Patch to detect fail precommit when the package declaration precedes the 
license header.

> Move license header before package declaration in all *.java files
> --
>
> Key: LUCENE-7013
> URL: https://issues.apache.org/jira/browse/LUCENE-7013
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Shai Erera
>Assignee: Shai Erera
>Priority: Minor
> Fix For: 5.5, trunk
>
> Attachments: LUCENE-7013-precommit.patch, LUCENE-7013.patch, 
> mvcopyright.py, mvcopyright.py
>
>
> In LUCENE-7012 we committed a change to the IDE templates to place the 
> license header before the package declaration in new Java files.
> I wrote a simple Python script which moves the header before the package 
> declaration. To be on the safe side, if a .java file does not already start 
> with the license header or with {{package org.apache}}, it doesn't modify it 
> and asks for manual intervention.
> It runs quite fast, so I don't mind running and committing one module at a 
> time.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-7013) Move license header before package declaration in all *.java files

[Steve Rowe (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-7013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Steve Rowe updated LUCENE-7013:
---
Attachment: LUCENE-7013-precommit.patch

Patch to detect fail precommit when the package declaration precedes the 
license header.

> Move license header before package declaration in all *.java files
> --
>
> Key: LUCENE-7013
> URL: https://issues.apache.org/jira/browse/LUCENE-7013
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Shai Erera
>Assignee: Shai Erera
>Priority: Minor
> Fix For: 5.5, trunk
>
> Attachments: LUCENE-7013-precommit.patch, LUCENE-7013.patch, 
> mvcopyright.py, mvcopyright.py
>
>
> In LUCENE-7012 we committed a change to the IDE templates to place the 
> license header before the package declaration in new Java files.
> I wrote a simple Python script which moves the header before the package 
> declaration. To be on the safe side, if a .java file does not already start 
> with the license header or with {{package org.apache}}, it doesn't modify it 
> and asks for manual intervention.
> It runs quite fast, so I don't mind running and committing one module at a 
> time.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Reopened] (LUCENE-7013) Move license header before package declaration in all *.java files

[Steve Rowe (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-7013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Steve Rowe reopened LUCENE-7013:


I added misplaced package detection to the {{-validate-source-patterns}} groovy 
script (patch in a sec), and it found 94 violations.

> Move license header before package declaration in all *.java files
> --
>
> Key: LUCENE-7013
> URL: https://issues.apache.org/jira/browse/LUCENE-7013
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Shai Erera
>Assignee: Shai Erera
>Priority: Minor
> Fix For: 5.5, trunk
>
> Attachments: LUCENE-7013-precommit.patch, LUCENE-7013.patch, 
> mvcopyright.py, mvcopyright.py
>
>
> In LUCENE-7012 we committed a change to the IDE templates to place the 
> license header before the package declaration in new Java files.
> I wrote a simple Python script which moves the header before the package 
> declaration. To be on the safe side, if a .java file does not already start 
> with the license header or with {{package org.apache}}, it doesn't modify it 
> and asks for manual intervention.
> It runs quite fast, so I don't mind running and committing one module at a 
> time.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9285) ArrayIndexOutOfBoundsException when ValueSourceAugmenter used with RTG on uncommitted doc

[David Smiley (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15372147#comment-15372147
 ] 

David Smiley commented on SOLR-9285:


Your plan makes sense to me Hoss.  Thanks for your thoroughness.

> ArrayIndexOutOfBoundsException when ValueSourceAugmenter used with RTG on 
> uncommitted doc
> -
>
> Key: SOLR-9285
> URL: https://issues.apache.org/jira/browse/SOLR-9285
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Hoss Man
> Attachments: SOLR-9285.patch
>
>
> Found in SOLR-9180 testing.
> Even in single node solr envs, doing an RTG for an uncommitted doc that uses 
> ValueSourceAugmenter (ie: simple field aliasing, or functions in fl) causes 
> an ArrayIndexOutOfBoundsException



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-9252) Feature selection and logistic regression on text

[Cao Manh Dat (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-9252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Cao Manh Dat updated SOLR-9252:
---
Attachment: SOLR-9252.patch

Updated patch. This patch includes :
- Support store textLogit & featureSelection by using updateStream.
- TextLogit model now support exact idfs by using SOLR-9243.

> Feature selection and logistic regression on text
> -
>
> Key: SOLR-9252
> URL: https://issues.apache.org/jira/browse/SOLR-9252
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Cao Manh Dat
>Assignee: Joel Bernstein
> Attachments: SOLR-9252.patch, SOLR-9252.patch, SOLR-9252.patch, 
> enron1.zip
>
>
> SOLR-9186 come up with a challenges that for each iterative we have to 
> rebuild the tf-idf vector for each documents. It is costly computation if we 
> represent doc by a lot of terms. Features selection can help reducing the 
> computation.
> Due to its computational efficiency and simple interpretation, information 
> gain is one of the most popular feature selection methods. It is used to 
> measure the dependence between features and labels and calculates the 
> information gain between the i-th feature and the class labels 
> (http://www.jiliang.xyz/publication/feature_selection_for_classification.pdf).
> I confirmed that by running logistics regressions on enron mail dataset (in 
> which each email is represented by top 100 terms that have highest 
> information gain) and got the accuracy by 92% and precision by 82%.
> This ticket will create two new streaming expression. Both of them use the 
> same *parallel iterative framework* as SOLR-8492.
> {code}
> featuresSelection(collection1, q="*:*",  field="tv_text", outcome="out_i", 
> positiveLabel=1, numTerms=100)
> {code}
> featuresSelection will emit top terms that have highest information gain 
> scores. It can be combined with new tlogit stream.
> {code}
> tlogit(collection1, q="*:*",
>  featuresSelection(collection1, 
>   q="*:*",  
>   field="tv_text", 
>   outcome="out_i", 
>   positiveLabel=1, 
>   numTerms=100),
>  field="tv_text",
>  outcome="out_i",
>  maxIterations=100)
> {code}
> In the iteration n, the text logistics regression will emit nth model, and 
> compute the error of (n-1)th model. Because the error will be wrong if we 
> compute the error dynamically in each iteration. 
> In each iteration tlogit will change learning rate based on error of previous 
> iteration. It will increase the learning rate by 5% if error is going down 
> and It will decrease the learning rate by 50% if error is going up.
> This will support use cases such as building models for spam detection, 
> sentiment analysis and threat detection. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-NightlyTests-6.x - Build # 117 - Still Failing

[Apache Jenkins Server]

Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-6.x/117/

1 tests failed.
FAILED:  
junit.framework.TestSuite.org.apache.solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest

Error Message:
ObjectTracker found 2 object(s) that were not released!!! [HdfsTransactionLog, 
HdfsTransactionLog]

Stack Trace:
java.lang.AssertionError: ObjectTracker found 2 object(s) that were not 
released!!! [HdfsTransactionLog, HdfsTransactionLog]
at __randomizedtesting.SeedInfo.seed([88A15EA81E630D4B]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertNull(Assert.java:551)
at 
org.apache.solr.SolrTestCaseJ4.teardownTestCases(SolrTestCaseJ4.java:258)
at sun.reflect.GeneratedMethodAccessor59.invoke(Unknown Source)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:834)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 11666 lines...]
   [junit4] Suite: org.apache.solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest
   [junit4]   2> Creating dataDir: 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_88A15EA81E630D4B-001/init-core-data-001
   [junit4]   2> 1219445 INFO  
(SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[88A15EA81E630D4B]-worker) [] 
o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (false) via: 
@org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
   [junit4]   2> 1219445 INFO  
(SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[88A15EA81E630D4B]-worker) [] 
o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
   [junit4]   1> Formatting using clusterid: testClusterID
   [junit4]   2> 1219482 WARN  
(SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[88A15EA81E630D4B]-worker) [] 
o.a.h.m.i.MetricsConfig Cannot locate configuration: tried 
hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
   [junit4]   2> 1219492 WARN  
(SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[88A15EA81E630D4B]-worker) [] 
o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 1219494 INFO  
(SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[88A15EA81E630D4B]-worker) [] 
o.m.log jetty-6.1.26
   [junit4]   2> 1219505 INFO  
(SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[88A15EA81E630D4B]-worker) [] 
o.m.log Extract 
jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.2-tests.jar!/webapps/hdfs
 to ./temp/Jetty_localhost_33211_hdfsy9hezg/webapp
   [junit4]   2> 1219868 INFO  
(SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[88A15EA81E630D4B]-worker) [] 
o.m.log Started 
HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:33211
   [junit4]   2> 1219940 WARN  
(SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[88A15EA81E630D4B]-worker) [ 

[JENKINS] Lucene-Solr-6.x-Windows (64bit/jdk1.8.0_92) - Build # 315 - Still Failing!

[Policeman Jenkins Server]

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-6.x-Windows/315/
Java: 64bit/jdk1.8.0_92 -XX:+UseCompressedOops -XX:+UseParallelGC

1 tests failed.
FAILED:  org.apache.solr.cloud.TestLocalFSCloudBackupRestore.test

Error Message:
Error from server at http://127.0.0.1:50254/solr: The backup directory already 
exists: 
file:///C:/Users/jenkins/workspace/Lucene-Solr-6.x-Windows/solr/build/solr-core/test/J1/temp/solr.cloud.TestLocalFSCloudBackupRestore_143039C471F747B7-001/tempDir-002/mytestbackup/

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error 
from server at http://127.0.0.1:50254/solr: The backup directory already 
exists: 
file:///C:/Users/jenkins/workspace/Lucene-Solr-6.x-Windows/solr/build/solr-core/test/J1/temp/solr.cloud.TestLocalFSCloudBackupRestore_143039C471F747B7-001/tempDir-002/mytestbackup/
at 
__randomizedtesting.SeedInfo.seed([143039C471F747B7:9C64061EDF0B2A4F]:0)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:590)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:259)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:248)
at 
org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:403)
at 
org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:356)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1228)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:998)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:934)
at 
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:149)
at 
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:166)
at 
org.apache.solr.cloud.AbstractCloudBackupRestoreTestCase.testBackupAndRestore(AbstractCloudBackupRestoreTestCase.java:207)
at 
org.apache.solr.cloud.AbstractCloudBackupRestoreTestCase.test(AbstractCloudBackupRestoreTestCase.java:127)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 

[JENKINS-EA] Lucene-Solr-6.x-Linux (64bit/jdk-9-ea+126) - Build # 1127 - Failure!

[Policeman Jenkins Server]

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-6.x-Linux/1127/
Java: 64bit/jdk-9-ea+126 -XX:-UseCompressedOops -XX:+UseConcMarkSweepGC

1 tests failed.
FAILED:  org.apache.solr.handler.TestReqParamsAPI.test

Error Message:
Could not get expected value  'CY val' for path 'params/c' full output: {   
"responseHeader":{ "status":0, "QTime":0},   "params":{ "a":"A 
val", "b":"B val", "wt":"json", "useParams":""},   "context":{ 
"webapp":"/_/xp", "path":"/dump1", "httpMethod":"GET"}},  from server:  
https://127.0.0.1:40680/_/xp/collection1

Stack Trace:
java.lang.AssertionError: Could not get expected value  'CY val' for path 
'params/c' full output: {
  "responseHeader":{
"status":0,
"QTime":0},
  "params":{
"a":"A val",
"b":"B val",
"wt":"json",
"useParams":""},
  "context":{
"webapp":"/_/xp",
"path":"/dump1",
"httpMethod":"GET"}},  from server:  
https://127.0.0.1:40680/_/xp/collection1
at 
__randomizedtesting.SeedInfo.seed([2BC280B3EB886678:A396BF6945740B80]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at 
org.apache.solr.core.TestSolrConfigHandler.testForResponseElement(TestSolrConfigHandler.java:481)
at 
org.apache.solr.handler.TestReqParamsAPI.testReqParams(TestReqParamsAPI.java:171)
at 
org.apache.solr.handler.TestReqParamsAPI.test(TestReqParamsAPI.java:61)
at 
jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(java.base@9-ea/Native 
Method)
at 
jdk.internal.reflect.NativeMethodAccessorImpl.invoke(java.base@9-ea/NativeMethodAccessorImpl.java:62)
at 
jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(java.base@9-ea/DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(java.base@9-ea/Method.java:533)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 

[JENKINS] Lucene-Solr-master-Windows (32bit/jdk1.8.0_92) - Build # 5978 - Still Failing!

[Policeman Jenkins Server]

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-master-Windows/5978/
Java: 32bit/jdk1.8.0_92 -server -XX:+UseConcMarkSweepGC

3 tests failed.
FAILED:  org.apache.solr.cloud.LeaderFailoverAfterPartitionTest.test

Error Message:
No live SolrServers available to handle this 
request:[http://127.0.0.1:57489/fpr/s/c8n_1x3_lf_shard1_replica2]

Stack Trace:
org.apache.solr.client.solrj.impl.CloudSolrClient$RouteException: No live 
SolrServers available to handle this 
request:[http://127.0.0.1:57489/fpr/s/c8n_1x3_lf_shard1_replica2]
at 
__randomizedtesting.SeedInfo.seed([641939ABDED07620:EC4D0671702C1BD8]:0)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.directUpdate(CloudSolrClient.java:739)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1151)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:1040)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:976)
at 
org.apache.solr.cloud.AbstractFullDistribZkTestBase.sendDocsWithRetry(AbstractFullDistribZkTestBase.java:753)
at 
org.apache.solr.cloud.AbstractFullDistribZkTestBase.sendDocsWithRetry(AbstractFullDistribZkTestBase.java:741)
at 
org.apache.solr.cloud.LeaderFailoverAfterPartitionTest.testRf3WithLeaderFailover(LeaderFailoverAfterPartitionTest.java:178)
at 
org.apache.solr.cloud.LeaderFailoverAfterPartitionTest.test(LeaderFailoverAfterPartitionTest.java:57)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:985)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:960)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)

[jira] [Comment Edited] (SOLR-9240) Support running the topic() Streaming Expression in parallel mode.

[Joel Bernstein (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371958#comment-15371958
 ] 

Joel Bernstein edited comment on SOLR-9240 at 7/12/16 1:17 AM:
---

This ticket is looking fairly good. I did a round of manual testing with the 
expression below which worked as expected.

{code}
parallel(
 workerCollection, 
 workers="2", 
 sort="_version_ desc", 
 daemon(
  update(
updateCollection, 
batchSize=200, 
topic(
checkpointCollection,
topicCollection, 
q=*:*, 
 id="topic40",
 fl="id, to , from", 
 partitionKeys="id",
 initialCheckpoint="0")), 
   runInterval="1000", 
   id="test3"))
{code}

This expression sends a daemon expression to two worker nodes. The daemon is 
wrapping an update expression which is wrapping a topic() expression. The topic 
has the new  *initialCheckpoint* parameter so it starts pulling records from 
checkpoint 0, which includes every record that matches the topic query in the 
index. The topic also has the *partitionKeys* parameter so each worker pulls a 
partition of records that match the topic query.

The daemon function will run the update() function iteratively. Each run will 
update the topic checkpoints for each worker.

The effect of this is that each worker will iterate though it's partition of 
the topic query, reindexing all the records that match the topic in another 
collection.



was (Author: joel.bernstein):
This ticket is looking fairly good. I did a round of manual testing with the 
expression below which worked as expected.

{code}
parallel(
 workerCollection, 
 workers="2", 
 sort="_version_ desc", 
 daemon(
  update(
updateCollection, 
batchSize=200, 
topic(
checkpointCollection,
topicCollection, 
q=*:*, 
 id="topic40",
 fl="id, to , from", 
 partitionKeys="id",
 initialCheckpoint="0")), 
   runInterval="1000", 
   id="test3"))
{code}

What this expression does is send a daemon expression to two worker nodes. The 
daemon is wrapping an update expression which is wrapping a topic() expression. 
The topic has the new initialCheckpoint parameter so it starts pulling records 
from checkpoint 0, which includes every record that matches the topic query in 
the index. The topic also has the partitionKeys parameter set so each worker 
pulls a partition of records that match the topic query.

The daemon function will run the update() function iteratively. Each run will 
update the topic checkpoints for each worker.

The effect of this is that each worker will iterate though it's partition of 
the topic query, reindexing all the records that match the topic in another 
collection.


> Support running the topic() Streaming Expression in parallel mode.
> --
>
> Key: SOLR-9240
> URL: https://issues.apache.org/jira/browse/SOLR-9240
> Project: Solr
>  Issue Type: Improvement
>Reporter: Joel Bernstein
>Assignee: Joel Bernstein
> Attachments: SOLR-9240.patch, SOLR-9240.patch
>
>
> Currently the topic() function won't run in parallel mode because each worker 
> needs to maintain a separate set of checkpoints. The proposed solution for 
> this is to append the worker ID to the topic ID, which will cause each worker 
> to have it's own checkpoints.
> It would be useful to support parallelizing the topic function because it 
> will provide a general purpose approach for processing text in parallel 
> across worker nodes.
> For example this would allow a classify() function to be wrapped around a 
> topic() function to classify documents in parallel across worker nodes. 
> Sample syntax:
> {code}
> parallel(daemon(update(classify(topic(..., partitionKeys="id")
> {code}
> The example above would send a daemon to worker nodes that would classify all 
> documents returned by the topic() function. The update function would send 
> the output of classify() to a SolrCloud collection for indexing.
> The partitionKeys parameter would ensure that each worker would receive a 
> partition of the results returned by the topic() function. This allows the 
> classify() function to be run in parallel.



--
This message was sent by Atlassian JIRA

[jira] [Updated] (SOLR-9285) ArrayIndexOutOfBoundsException when ValueSourceAugmenter used with RTG on uncommitted doc

[Hoss Man (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-9285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Hoss Man updated SOLR-9285:
---
Attachment: SOLR-9285.patch


The root cause of the AIOOBE is that when a doc is found in the {{UpdateLog}}, 
the RTG component explicitly passes -1 to {{DocTransformer.transform()}}, and 
currently {{ValueSourceAugmenter.transform()}} doesn't account for this 
possibility when using it's IndexReader to lookup the FunctionValues for the 
document.

Obviously, even if it did do some sanity checking for negative docids, that 
wouldn't really address the root of the problem -- getting usable 
FunctionValues.

The RTG Component already has a code path where it sometimes uses 
{{ulog.openRealtimeSearcher()}} when doc are found in the update log, instead 
of returning the document directly from the ulog command -- but that code path 
is currently confined to situations where {{fq}} params are included in the RTG 
request (so it can evaluate them against the Realtime Searcher to ensure the 
doc should really be returned)



I'm attaching a straw man patch that:

* refactors the {{ulog.openRealtimeSearcher()}} logic in RTG Component to also 
apply when there are transformers
* Adds a new light weight (private) {{RTGResultContext}} class for wrapping 
realtime {{SolrIndexSearcher}}s for use with transformers.
* Fixes {{ValueSourceAugmenter.setContext}} to use 
{{ResultContext.getSearcher()}} instead of the one that comes from the request
** Independent of anything else, this seems like a bug (it certainly violates 
the javadocs of {{ResultContext.getRequest()}}, even if it was getting the same 
SolrQueryRequest via a more round about way)


This patch fixes all existing {{@AwaitsFix(SOLR-9285)}} tests, and doesn't seem 
to cause any new test failures in the code base, but i don't think we should 
commit it as is for the simple reason that it seems like it is overkill: Simple 
transformers like RenameFieldTransformer can operate just fine on docs from the 
UpdateLog w/o any need for a (realtime) SolrIndexSearcher.

As noted in the patch with a {{nocommit}} I think what would be best is to 
enahnce this patch by adding a {{boolean needsSolrIndexSearcher()}} method to 
the {{DocTransformer}} API, which defaults to {{false}} but any transformers 
like {{ValueSourceAugmenter}} that actually _need_ access to a 
{{SolrIndexSearcher}} in their {{setContext}} method could ask for it, and the 
code in this patch that sets {{mustUseRealtimeSearcher = true}} anytime there 
is a transformer could be modified to instead use 
{{transformer.needsSolrIndexSearcher()}}

Questions/comments/concerns?



Reviewing the RTG Component code also makes me realize that in general we 
should have more RTG+transformer tests which:

* use {{fq}} params
* ask for uncommited ids both immediately after adding them (before a realtime 
searcher may have been opened) *AND* "after" a realtime searcher has been 
opened (for whatever reason) 
* ask for multiple ids in a single request (to ensure nothing broken in the 
looping logic)
** mixing commited, uncommited, and uncommited but in an already re-opened 
realtime searcher
** mixing them in various orders, since that affects *when* a realtime searcher 
might be opened



> ArrayIndexOutOfBoundsException when ValueSourceAugmenter used with RTG on 
> uncommitted doc
> -
>
> Key: SOLR-9285
> URL: https://issues.apache.org/jira/browse/SOLR-9285
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Hoss Man
> Attachments: SOLR-9285.patch
>
>
> Found in SOLR-9180 testing.
> Even in single node solr envs, doing an RTG for an uncommitted doc that uses 
> ValueSourceAugmenter (ie: simple field aliasing, or functions in fl) causes 
> an ArrayIndexOutOfBoundsException



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9252) Feature selection and logistic regression on text

[Cao Manh Dat (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15372012#comment-15372012
 ] 

Cao Manh Dat commented on SOLR-9252:


Thanks for the review, I will upload updated patch shortly.

Because in ML we deal a lot with number, array of numbers. So I think we will 
use dynamic field to define a standard schema. 
For example :
{code}
*_i : int
*_is : array of int
...
{code}

> Feature selection and logistic regression on text
> -
>
> Key: SOLR-9252
> URL: https://issues.apache.org/jira/browse/SOLR-9252
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Cao Manh Dat
>Assignee: Joel Bernstein
> Attachments: SOLR-9252.patch, SOLR-9252.patch, enron1.zip
>
>
> SOLR-9186 come up with a challenges that for each iterative we have to 
> rebuild the tf-idf vector for each documents. It is costly computation if we 
> represent doc by a lot of terms. Features selection can help reducing the 
> computation.
> Due to its computational efficiency and simple interpretation, information 
> gain is one of the most popular feature selection methods. It is used to 
> measure the dependence between features and labels and calculates the 
> information gain between the i-th feature and the class labels 
> (http://www.jiliang.xyz/publication/feature_selection_for_classification.pdf).
> I confirmed that by running logistics regressions on enron mail dataset (in 
> which each email is represented by top 100 terms that have highest 
> information gain) and got the accuracy by 92% and precision by 82%.
> This ticket will create two new streaming expression. Both of them use the 
> same *parallel iterative framework* as SOLR-8492.
> {code}
> featuresSelection(collection1, q="*:*",  field="tv_text", outcome="out_i", 
> positiveLabel=1, numTerms=100)
> {code}
> featuresSelection will emit top terms that have highest information gain 
> scores. It can be combined with new tlogit stream.
> {code}
> tlogit(collection1, q="*:*",
>  featuresSelection(collection1, 
>   q="*:*",  
>   field="tv_text", 
>   outcome="out_i", 
>   positiveLabel=1, 
>   numTerms=100),
>  field="tv_text",
>  outcome="out_i",
>  maxIterations=100)
> {code}
> In the iteration n, the text logistics regression will emit nth model, and 
> compute the error of (n-1)th model. Because the error will be wrong if we 
> compute the error dynamically in each iteration. 
> In each iteration tlogit will change learning rate based on error of previous 
> iteration. It will increase the learning rate by 5% if error is going down 
> and It will decrease the learning rate by 50% if error is going up.
> This will support use cases such as building models for spam detection, 
> sentiment analysis and threat detection. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9295) Remove Unicode BOM (U+FEFF) from text files in codebase

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371994#comment-15371994
 ] 

ASF subversion and git services commented on SOLR-9295:
---

Commit 6d6aacbc764891f25e7221a7cb6374606f0ad920 in lucene-solr's branch 
refs/heads/branch_6x from [~steve_rowe]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=6d6aacb ]

SOLR-9295: Added beginning-of-file UTF-8 BOM to the list of disallowed patterns 
in -validate-source-patterns


> Remove Unicode BOM (U+FEFF) from text files in codebase
> ---
>
> Key: SOLR-9295
> URL: https://issues.apache.org/jira/browse/SOLR-9295
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: scripts and tools
>Affects Versions: master (7.0)
>Reporter: Shawn Heisey
>Priority: Trivial
> Fix For: 6.2, master (7.0)
>
>
> When starting Solr built from the master branch on Windows, this is what you 
> see:
> {noformat}
> C:\Users\elyograg\git\lucene-solr\solr>bin\solr start
> C:\Users\elyograg\git\lucene-solr\solr>@REM
> '@REM' is not recognized as an internal or external command,
> operable program or batch file.
> {noformat}
> The three extra characters, found at the very beginning of the solr.cmd 
> script, are a Unicode BOM, and are invisible to vim, notepad, and notepad++.  
> The problem does not exist in 6.1.0, but IS present in branch_6x and master.
> Using grep to find this character in the entire codebase, I found one other 
> relevant file with a BOM.  All others were binary (images, jars, git pack 
> files, etc):
> ./solr/webapp/web/js/lib/jquery.blockUI.js



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9295) Remove Unicode BOM (U+FEFF) from text files in codebase

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371995#comment-15371995
 ] 

ASF subversion and git services commented on SOLR-9295:
---

Commit 39356bf64cfb7aa143e9f2203adf15bb7d4ff8f5 in lucene-solr's branch 
refs/heads/master from [~steve_rowe]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=39356bf ]

SOLR-9295: Added beginning-of-file UTF-8 BOM to the list of disallowed patterns 
in -validate-source-patterns


> Remove Unicode BOM (U+FEFF) from text files in codebase
> ---
>
> Key: SOLR-9295
> URL: https://issues.apache.org/jira/browse/SOLR-9295
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: scripts and tools
>Affects Versions: master (7.0)
>Reporter: Shawn Heisey
>Priority: Trivial
> Fix For: 6.2, master (7.0)
>
>
> When starting Solr built from the master branch on Windows, this is what you 
> see:
> {noformat}
> C:\Users\elyograg\git\lucene-solr\solr>bin\solr start
> C:\Users\elyograg\git\lucene-solr\solr>@REM
> '@REM' is not recognized as an internal or external command,
> operable program or batch file.
> {noformat}
> The three extra characters, found at the very beginning of the solr.cmd 
> script, are a Unicode BOM, and are invisible to vim, notepad, and notepad++.  
> The problem does not exist in 6.1.0, but IS present in branch_6x and master.
> Using grep to find this character in the entire codebase, I found one other 
> relevant file with a BOM.  All others were binary (images, jars, git pack 
> files, etc):
> ./solr/webapp/web/js/lib/jquery.blockUI.js



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9295) Remove Unicode BOM (U+FEFF) from text files in codebase

[Steve Rowe (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371985#comment-15371985
 ] 

Steve Rowe commented on SOLR-9295:
--

The following pattern added to the {{invalidPatterns}} array in the 
{{source-patterns}} groovy script included with the top-level ant target 
{{-validate-source-patterns}} successfully found the two files [~elyograg] 
found (after I checked out the commit just before his):

{{(~$/^\uFEFF/$) : 'UTF-8 byte order mark'}}

Here's the output:

{noformat}
-validate-source-patterns:
[source-patterns] UTF-8 byte order mark: solr/bin/solr.cmd
[source-patterns] UTF-8 byte order mark: 
solr/webapp/web/js/lib/jquery.blockUI.js

BUILD FAILED
/Users/sarowe/git/lucene-solr-3/build.xml:132: Found 2 violations in source 
files (UTF-8 byte order mark).
{noformat}

I tested that it only flags beginning-of-file occurrences by prepending junk to 
{{solr/bin/solr.cmd}} - {{od -c}} shows the BOM was still present, just not at 
the beginning - and {{-validate-source-patterns}} didn't complain.

Committing shortly.

> Remove Unicode BOM (U+FEFF) from text files in codebase
> ---
>
> Key: SOLR-9295
> URL: https://issues.apache.org/jira/browse/SOLR-9295
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: scripts and tools
>Affects Versions: master (7.0)
>Reporter: Shawn Heisey
>Priority: Trivial
> Fix For: 6.2, master (7.0)
>
>
> When starting Solr built from the master branch on Windows, this is what you 
> see:
> {noformat}
> C:\Users\elyograg\git\lucene-solr\solr>bin\solr start
> C:\Users\elyograg\git\lucene-solr\solr>@REM
> '@REM' is not recognized as an internal or external command,
> operable program or batch file.
> {noformat}
> The three extra characters, found at the very beginning of the solr.cmd 
> script, are a Unicode BOM, and are invisible to vim, notepad, and notepad++.  
> The problem does not exist in 6.1.0, but IS present in branch_6x and master.
> Using grep to find this character in the entire codebase, I found one other 
> relevant file with a BOM.  All others were binary (images, jars, git pack 
> files, etc):
> ./solr/webapp/web/js/lib/jquery.blockUI.js



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Comment Edited] (SOLR-9240) Support running the topic() Streaming Expression in parallel mode.

[Joel Bernstein (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371958#comment-15371958
 ] 

Joel Bernstein edited comment on SOLR-9240 at 7/12/16 12:33 AM:


This ticket is looking fairly good. I did a round of manual testing with the 
expression below which worked as expected.

{code}
parallel(
 workerCollection, 
 workers="2", 
 sort="_version_ desc", 
 daemon(
  update(
updateCollection, 
batchSize=200, 
topic(
checkpointCollection,
topicCollection, 
q=*:*, 
 id="topic40",
 fl="id, to , from", 
 partitionKeys="id",
 initialCheckpoint="0")), 
   runInterval="1000", 
   id="test3"))
{code}

What this expression does is send a daemon expression to two worker nodes. The 
daemon is wrapping an update expression which is wrapping a topic() expression. 
The topic has the new initialCheckpoint parameter so it starts pulling records 
from checkpoint 0, which includes every record that matches the topic query in 
the index. The topic also has the partitionKeys parameter set so each worker 
pulls a partition of records that match the topic query.

The daemon function will run the update() function iteratively. Each run will 
update the topic checkpoints for each worker.

The effect of this is that each worker will iterate though it's partition of 
the topic query, reindexing all the records that match the topic in another 
collection.



was (Author: joel.bernstein):
This ticket is looking fairly good. I did a round of manual testing which works 
as expected.

{code}
parallel(
 workerCollection, 
 workers="2", 
 sort="_version_ desc", 
 daemon(
  update(
updateCollection, 
batchSize=200, 
topic(
checkpointCollection,
topicCollection, 
q=*:*, 
 id="topic40",
 fl="id, to , from", 
 partitionKeys="id",
 initialCheckpoint="0")), 
   runInterval="1000", 
   id="test3"))
{code}



> Support running the topic() Streaming Expression in parallel mode.
> --
>
> Key: SOLR-9240
> URL: https://issues.apache.org/jira/browse/SOLR-9240
> Project: Solr
>  Issue Type: Improvement
>Reporter: Joel Bernstein
>Assignee: Joel Bernstein
> Attachments: SOLR-9240.patch, SOLR-9240.patch
>
>
> Currently the topic() function won't run in parallel mode because each worker 
> needs to maintain a separate set of checkpoints. The proposed solution for 
> this is to append the worker ID to the topic ID, which will cause each worker 
> to have it's own checkpoints.
> It would be useful to support parallelizing the topic function because it 
> will provide a general purpose approach for processing text in parallel 
> across worker nodes.
> For example this would allow a classify() function to be wrapped around a 
> topic() function to classify documents in parallel across worker nodes. 
> Sample syntax:
> {code}
> parallel(daemon(update(classify(topic(..., partitionKeys="id")
> {code}
> The example above would send a daemon to worker nodes that would classify all 
> documents returned by the topic() function. The update function would send 
> the output of classify() to a SolrCloud collection for indexing.
> The partitionKeys parameter would ensure that each worker would receive a 
> partition of the results returned by the topic() function. This allows the 
> classify() function to be run in parallel.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Comment Edited] (SOLR-9240) Support running the topic() Streaming Expression in parallel mode.

[Joel Bernstein (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371958#comment-15371958
 ] 

Joel Bernstein edited comment on SOLR-9240 at 7/12/16 12:22 AM:


This ticket is looking fairly good. I did a round of manual testing which works 
as expected.

{code}
parallel(
 workerCollection, 
 workers="2", 
 sort="_version_ desc", 
 daemon(
  update(
updateCollection, 
batchSize=200, 
topic(
checkpointCollection,
topicCollection, 
q=*:*, 
 id="topic40",
 fl="id, to , from", 
 partitionKeys="id",
 initialCheckpoint="0")), 
   runInterval="1000", 
   id="test3"))
{code}




was (Author: joel.bernstein):
This ticket is looking fairly good. I did a round of manual testing which works 
as expected.

{code}
parallel(workerCollection, 
 workers="2", 
 sort="_version_ desc", 
 daemon(update(updateCollection, 
   batchSize=200, 
   topic(checkpointCollection,
topicCollection, 
q=*:*, 
id="topic40",
fl="id, to , from", 
partitionKeys="id",
initialCheckpoint="0")), 
runInterval="1000", 
id="test3"))
{code}



> Support running the topic() Streaming Expression in parallel mode.
> --
>
> Key: SOLR-9240
> URL: https://issues.apache.org/jira/browse/SOLR-9240
> Project: Solr
>  Issue Type: Improvement
>Reporter: Joel Bernstein
>Assignee: Joel Bernstein
> Attachments: SOLR-9240.patch, SOLR-9240.patch
>
>
> Currently the topic() function won't run in parallel mode because each worker 
> needs to maintain a separate set of checkpoints. The proposed solution for 
> this is to append the worker ID to the topic ID, which will cause each worker 
> to have it's own checkpoints.
> It would be useful to support parallelizing the topic function because it 
> will provide a general purpose approach for processing text in parallel 
> across worker nodes.
> For example this would allow a classify() function to be wrapped around a 
> topic() function to classify documents in parallel across worker nodes. 
> Sample syntax:
> {code}
> parallel(daemon(update(classify(topic(..., partitionKeys="id")
> {code}
> The example above would send a daemon to worker nodes that would classify all 
> documents returned by the topic() function. The update function would send 
> the output of classify() to a SolrCloud collection for indexing.
> The partitionKeys parameter would ensure that each worker would receive a 
> partition of the results returned by the topic() function. This allows the 
> classify() function to be run in parallel.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9240) Support running the topic() Streaming Expression in parallel mode.

[Joel Bernstein (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371958#comment-15371958
 ] 

Joel Bernstein commented on SOLR-9240:
--

This ticket is looking fairly good. I did a round of manual testing which works 
as expected.

{code}
parallel(workerCollection, 
 workers="2", 
 sort="_version_ desc", 
 daemon(update(updateCollection, 
   batchSize=200, 
   topic(checkpointCollection,
topicCollection, 
q=*:*, 
id="topic40",
fl="id, to , from", 
partitionKeys="id",
initialCheckpoint="0")), 
runInterval="1000", 
id="test3"))
{code}



> Support running the topic() Streaming Expression in parallel mode.
> --
>
> Key: SOLR-9240
> URL: https://issues.apache.org/jira/browse/SOLR-9240
> Project: Solr
>  Issue Type: Improvement
>Reporter: Joel Bernstein
>Assignee: Joel Bernstein
> Attachments: SOLR-9240.patch, SOLR-9240.patch
>
>
> Currently the topic() function won't run in parallel mode because each worker 
> needs to maintain a separate set of checkpoints. The proposed solution for 
> this is to append the worker ID to the topic ID, which will cause each worker 
> to have it's own checkpoints.
> It would be useful to support parallelizing the topic function because it 
> will provide a general purpose approach for processing text in parallel 
> across worker nodes.
> For example this would allow a classify() function to be wrapped around a 
> topic() function to classify documents in parallel across worker nodes. 
> Sample syntax:
> {code}
> parallel(daemon(update(classify(topic(..., partitionKeys="id")
> {code}
> The example above would send a daemon to worker nodes that would classify all 
> documents returned by the topic() function. The update function would send 
> the output of classify() to a SolrCloud collection for indexing.
> The partitionKeys parameter would ensure that each worker would receive a 
> partition of the results returned by the topic() function. This allows the 
> classify() function to be run in parallel.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-6.x-Solaris (64bit/jdk1.8.0) - Build # 262 - Failure!

[Policeman Jenkins Server]

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-6.x-Solaris/262/
Java: 64bit/jdk1.8.0 -XX:+UseCompressedOops -XX:+UseConcMarkSweepGC

2 tests failed.
FAILED:  
junit.framework.TestSuite.org.apache.solr.cloud.overseer.ZkStateWriterTest

Error Message:
1 thread leaked from SUITE scope at 
org.apache.solr.cloud.overseer.ZkStateWriterTest: 1) Thread[id=11317, 
name=watches-1789-thread-1, state=TIMED_WAITING, group=TGRP-ZkStateWriterTest]  
   at sun.misc.Unsafe.park(Native Method) at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) 
at 
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
 at 
java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
 at 
java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) 
at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)   
  at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) 
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 
at java.lang.Thread.run(Thread.java:745)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE 
scope at org.apache.solr.cloud.overseer.ZkStateWriterTest: 
   1) Thread[id=11317, name=watches-1789-thread-1, state=TIMED_WAITING, 
group=TGRP-ZkStateWriterTest]
at sun.misc.Unsafe.park(Native Method)
at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at 
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at 
java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
at __randomizedtesting.SeedInfo.seed([7A2FB98AEE1A95BB]:0)


FAILED:  
junit.framework.TestSuite.org.apache.solr.cloud.overseer.ZkStateWriterTest

Error Message:
There are still zombie threads that couldn't be terminated:1) 
Thread[id=11317, name=watches-1789-thread-1, state=TIMED_WAITING, 
group=TGRP-ZkStateWriterTest] at sun.misc.Unsafe.park(Native Method)
 at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)  
   at 
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
 at 
java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
 at 
java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) 
at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)   
  at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) 
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 
at java.lang.Thread.run(Thread.java:745)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie 
threads that couldn't be terminated:
   1) Thread[id=11317, name=watches-1789-thread-1, state=TIMED_WAITING, 
group=TGRP-ZkStateWriterTest]
at sun.misc.Unsafe.park(Native Method)
at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at 
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at 
java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
at __randomizedtesting.SeedInfo.seed([7A2FB98AEE1A95BB]:0)




Build Log:
[...truncated 11499 lines...]
   [junit4] Suite: org.apache.solr.cloud.overseer.ZkStateWriterTest
   [junit4]   2> Creating dataDir: 
/export/home/jenkins/workspace/Lucene-Solr-6.x-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.overseer.ZkStateWriterTest_7A2FB98AEE1A95BB-001/init-core-data-001
   [junit4]   2> 1550508 INFO  
(SUITE-ZkStateWriterTest-seed#[7A2FB98AEE1A95BB]-worker) [] 
o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: 
@org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN, clientAuth=NaN)
   [junit4]   2> 1550510 INFO  
(TEST-ZkStateWriterTest.testSingleExternalCollection-seed#[7A2FB98AEE1A95BB]) [ 
   ] 

[jira] [Commented] (SOLR-8593) Integrate Apache Calcite into the SQLHandler

[Julian Hyde (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-8593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371912#comment-15371912
 ] 

Julian Hyde commented on SOLR-8593:
---

Hi everyone! I'm VP of Apache Calcite. I only just noticed this JIRA case. I am 
excited that you are considering using Calcite. Please let me know if I can 
help.

> Integrate Apache Calcite into the SQLHandler
> 
>
> Key: SOLR-8593
> URL: https://issues.apache.org/jira/browse/SOLR-8593
> Project: Solr
>  Issue Type: Improvement
>Reporter: Joel Bernstein
>
> The Presto SQL Parser was perfect for phase one of the SQLHandler. It was 
> nicely split off from the larger Presto project and it did everything that 
> was needed for the initial implementation.
> Phase two of the SQL work though will require an optimizer. Here is where 
> Apache Calcite comes into play. It has a battle tested cost based optimizer 
> and has been integrated into Apache Drill and Hive.
> This work can begin in trunk following the 6.0 release. The final query plans 
> will continue to be translated to Streaming API objects (TupleStreams), so 
> continued work on the JDBC driver should plug in nicely with the Calcite work.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-master-Solaris (64bit/jdk1.8.0) - Build # 712 - Still Failing!

[Policeman Jenkins Server]

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-master-Solaris/712/
Java: 64bit/jdk1.8.0 -XX:-UseCompressedOops -XX:+UseG1GC

1 tests failed.
FAILED:  org.apache.solr.cloud.TestLocalFSCloudBackupRestore.test

Error Message:
Error from server at http://127.0.0.1:38609/solr: 'location' is not specified 
as a query parameter or as a default repository property or as a cluster 
property.

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error 
from server at http://127.0.0.1:38609/solr: 'location' is not specified as a 
query parameter or as a default repository property or as a cluster property.
at 
__randomizedtesting.SeedInfo.seed([33943AED05A69349:BBC00537AB5AFEB1]:0)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:606)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:259)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:248)
at 
org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:413)
at 
org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:366)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1270)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:1040)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:976)
at 
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:149)
at 
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:166)
at 
org.apache.solr.cloud.AbstractCloudBackupRestoreTestCase.testInvalidPath(AbstractCloudBackupRestoreTestCase.java:149)
at 
org.apache.solr.cloud.AbstractCloudBackupRestoreTestCase.test(AbstractCloudBackupRestoreTestCase.java:128)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 

[jira] [Commented] (LUCENE-2899) Add OpenNLP Analysis capabilities as a module

[Steve Rowe (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-2899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371843#comment-15371843
 ] 

Steve Rowe commented on LUCENE-2899:


bq. Steve Rowe per our conversation yesterday.. Would be interesting to store 
the PoS and entity information as stacked tokens vs (or in addition to the) 
payload... such that you could do "bob @person"~0 or "house @verb"~0 type 
queries.. or things like "@person @ceo"~10

[~sbower], I agree, that possibility would be nice.  I checked for the 
existence of a token type->synonym filter, and don't see one, but I think it 
would be fairly easy to add.

Which reminds me: the lemmatization filter I added here should have the ability 
(like some stemmers, indirectly) to emit lemmas as synonyms - this is possible, 
as in the PorterStemmer implementaiton, simply by not processing any tokens 
with the keyword attribute set to true, and preceding with the 
KeywordRepeatFilter. 

> Add OpenNLP Analysis capabilities as a module
> -
>
> Key: LUCENE-2899
> URL: https://issues.apache.org/jira/browse/LUCENE-2899
> Project: Lucene - Core
>  Issue Type: New Feature
>  Components: modules/analysis
>Reporter: Grant Ingersoll
>Assignee: Grant Ingersoll
>Priority: Minor
> Fix For: 4.9, 6.0
>
> Attachments: LUCENE-2899-6.1.0.patch, LUCENE-2899-RJN.patch, 
> LUCENE-2899.patch, LUCENE-2899.patch, LUCENE-2899.patch, OpenNLPFilter.java, 
> OpenNLPTokenizer.java
>
>
> Now that OpenNLP is an ASF project and has a nice license, it would be nice 
> to have a submodule (under analysis) that exposed capabilities for it. Drew 
> Farris, Tom Morton and I have code that does:
> * Sentence Detection as a Tokenizer (could also be a TokenFilter, although it 
> would have to change slightly to buffer tokens)
> * NamedEntity recognition as a TokenFilter
> We are also planning a Tokenizer/TokenFilter that can put parts of speech as 
> either payloads (PartOfSpeechAttribute?) on a token or at the same position.
> I'd propose it go under:
> modules/analysis/opennlp



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-2899) Add OpenNLP Analysis capabilities as a module

[Steve Rowe (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-2899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Steve Rowe updated LUCENE-2899:
---
Attachment: LUCENE-2899-6.1.0.patch

bq. Can this patch be now used with latest version of SOLR .

The patch named 
[LUCENE-2899.patch|https://issues.apache.org/jira/secure/attachment/12817033/LUCENE-2899.patch]
 is against the (unreleased) master branch of Lucene/Solr.

I tried applying it to the 6.1.0 source code (from a git clone: {{git checkout 
releases/lucene-solr/6.1.0}}), and it failed in a couple places.

So I made a 6.1.0-specific patch named {{LUCENE-2899-6.1.0.patch}} and am 
attaching it to the issue.  Compilation and tests succeeded after I ran {{ant 
train-test-models}} from {{lucene/analysis/opennlp/}}.

> Add OpenNLP Analysis capabilities as a module
> -
>
> Key: LUCENE-2899
> URL: https://issues.apache.org/jira/browse/LUCENE-2899
> Project: Lucene - Core
>  Issue Type: New Feature
>  Components: modules/analysis
>Reporter: Grant Ingersoll
>Assignee: Grant Ingersoll
>Priority: Minor
> Fix For: 4.9, 6.0
>
> Attachments: LUCENE-2899-6.1.0.patch, LUCENE-2899-RJN.patch, 
> LUCENE-2899.patch, LUCENE-2899.patch, LUCENE-2899.patch, OpenNLPFilter.java, 
> OpenNLPTokenizer.java
>
>
> Now that OpenNLP is an ASF project and has a nice license, it would be nice 
> to have a submodule (under analysis) that exposed capabilities for it. Drew 
> Farris, Tom Morton and I have code that does:
> * Sentence Detection as a Tokenizer (could also be a TokenFilter, although it 
> would have to change slightly to buffer tokens)
> * NamedEntity recognition as a TokenFilter
> We are also planning a Tokenizer/TokenFilter that can put parts of speech as 
> either payloads (PartOfSpeechAttribute?) on a token or at the same position.
> I'd propose it go under:
> modules/analysis/opennlp



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Lucene Index accelerator interface

[Beercandyman]

I've been working on writing an indexer that runs in an FPGA. I find FPGAs
have a hard time doing functions that are very easy on a CPU. When I think
about it the same is true of GPUs. I want to propose some kind of
accelerator interface into the indexing code.  

 

Here is a start to a proposal.

http://bit.ly/29EOSnW

 

I'm wondering if anyone thinks this idea has merit and if anyone want to
work on it with me?

 

Cheers

 

Steve

 

 

[jira] [Resolved] (SOLR-9287) single node RTG: NPE if score is requested

[Hoss Man (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-9287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Hoss Man resolved SOLR-9287.

   Resolution: Fixed
 Assignee: Hoss Man
Fix Version/s: master (7.0)
   6.2

thanks for the help Ishan

> single node RTG: NPE if score is requested
> --
>
> Key: SOLR-9287
> URL: https://issues.apache.org/jira/browse/SOLR-9287
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Hoss Man
>Assignee: Hoss Man
> Fix For: 6.2, master (7.0)
>
> Attachments: SOLR-9287.patch, SOLR-9287.patch
>
>
> Found in SOLR-9180 testing.
> In single node solr setups, if an RTG request is made that includes "score" 
> in the fl, then there is an NPE from ResultContext.wantsScores.
> This does *not* happen if the same request happens in a SolrCloud setup - in 
> that case the request for "score" is silently ignored -- this seems to me 
> like the optimal behavior  (similarly: using the {{\[explain\]}} transformer 
> in the fl for an RTG is currently silently ignored for both single node and 
> solr cloud envs)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9290) TCP-connections in CLOSE_WAIT spikes during heavy indexing when SSL is enabled

[Hoss Man (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371758#comment-15371758
 ] 

Hoss Man commented on SOLR-9290:


questions specifically for [~shaie] followng up on comments made in the mailing 
list thread mentioned in the isue summary...

{quote}
When it does happen, the number of CLOSE_WAITS climb very high, to the order of 
30K+ entries in 'netstat'.
...
When I say it does not reproduce on 5.4.1 I really mean the numbers
don't go as high as they do in 5.5.1. Meaning, when running without
SSL, the number of CLOSE_WAITs is smallish, usually less than a 10 (I
would separately like to understand why we have any in that state at
all). When running with SSL and 5.4.1, they stay low at the order of
hundreds the most.
{quote}

* Does this only reproduce in your application, with your customized configs of 
Solr, or can you reproduce it using something trivial like "modify 
bin/solr.in.sh to point at an SSL cert, then run; {{bin/solr -noprompt 
-cloud}}." ?
* Does the problem only manifest solely with indexing, or with queries as well? 
ie...
** assuming a pre-built collection, and then all nodes restarted, does 
hammering the cluster with read only queries manifest the problem?
** assuming a virgin cluster with no docs, does hammering the cluster w/updates 
but never any queries, manifest the problem?
* Assuming you start by bringing up a virgin cluster and then begin hammering 
it with whatever sequences of requests are needed to manifest the problem, how 
long do you have to wait before the number of CLOSE_WAITS spikes high enough 
that you are reasonably confident the problem has occured?

The last question being a pre-req to wondering if we can just git bisect to 
identify where/when the problem originated.  

Even if writing a (reliable) bash automation script (to start the cluster, 
_and_ triggering requests, _and_ monitoring the CLOSE_WAITS to see if they go 
over a specified threshold in under a specified timelimit, _and_ shut 
everything down cleanly) is too cumbersome to have faith in running an 
automated {{git bisect run test.sh}}, we could still consider doing some 
manually driven git bisection to try and track this down, as long as each 
iteration doesn't take very long.

Specifically: {{git merge-base}} says ffadf9715c4a511178183fc1411b18c1701b9f1d 
is the common ancestor for 5.4.1 and 5.5.1, and {{git log}} says there are 487 
commits between that point and the 5.5.1 tag.  Statistically speaking it should 
only take 
~10 iterations to do a binary search of those commits to find the first 
problematic one.

Assuming there is a manual process someone can run on a clean git checkout of 
5.4.1 that takes under 10 minutes to get from "ant clean server" to an obvious 
splke in CLOSE_WAITS, someone with some CPU cycles to spare who doesn't mind a 
lot of context switching while they do their day job could be...
# running a command to spin up the cluster & client hammering code
# setting a 10 minute timer
# when the timer goes off, check the results of another command to count the 
CLOSE_WAITS
# {{git bisect good/bad}}
# repeat
...and within ~2-3 hours should almost certainly have tracked down when/where 
the problem started.



> TCP-connections in CLOSE_WAIT spikes during heavy indexing when SSL is enabled
> --
>
> Key: SOLR-9290
> URL: https://issues.apache.org/jira/browse/SOLR-9290
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Affects Versions: 5.5.1, 5.5.2
>Reporter: Anshum Gupta
>Priority: Critical
>
> Heavy indexing on Solr with SSL leads to a lot of connections in CLOSE_WAIT 
> state. 
> At my workplace, we have seen this issue only with 5.5.1 and could not 
> reproduce it with 5.4.1 but from my conversation with Shalin, he knows of 
> users with 5.3.1 running into this issue too. 
> Here's an excerpt from the email [~shaie] sent to the mailing list  (about 
> what we see:
> {quote}
> 1) It consistently reproduces on 5.5.1, but *does not* reproduce on 5.4.1
> 2) It does not reproduce when SSL is disabled
> 3) Restarting the Solr process (sometimes both need to be restarted), the
> count drops to 0, but if indexing continues, they climb up again
> When it does happen, Solr seems stuck. The leader cannot talk to the
> replica, or vice versa, the replica is usually put in DOWN state and
> there's no way to fix it besides restarting the JVM.
> {quote}
> Here's the mail thread: 
> http://mail-archives.apache.org/mod_mbox/lucene-solr-user/201607.mbox/%3c46cc66220a8143dc903fa34e79205...@vp-exc01.dips.local%3E
> Creating this issue so we could track this and have more people comment on 
> what they see. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (SOLR-9287) single node RTG: NPE if score is requested

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371743#comment-15371743
 ] 

ASF subversion and git services commented on SOLR-9287:
---

Commit b3cc4b320c43747a60177389ff9ad7a5a840 in lucene-solr's branch 
refs/heads/branch_6x from Chris Hostetter
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=b3cc4b3 ]

SOLR-9287: Including 'score' in the 'fl' param when doing an RTG no longer 
causes an NPE

(cherry picked from commit 462dc04cb6aaf3a876b56f27c6b511b00e25e85a)


> single node RTG: NPE if score is requested
> --
>
> Key: SOLR-9287
> URL: https://issues.apache.org/jira/browse/SOLR-9287
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Hoss Man
> Attachments: SOLR-9287.patch, SOLR-9287.patch
>
>
> Found in SOLR-9180 testing.
> In single node solr setups, if an RTG request is made that includes "score" 
> in the fl, then there is an NPE from ResultContext.wantsScores.
> This does *not* happen if the same request happens in a SolrCloud setup - in 
> that case the request for "score" is silently ignored -- this seems to me 
> like the optimal behavior  (similarly: using the {{\[explain\]}} transformer 
> in the fl for an RTG is currently silently ignored for both single node and 
> solr cloud envs)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9287) single node RTG: NPE if score is requested

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371744#comment-15371744
 ] 

ASF subversion and git services commented on SOLR-9287:
---

Commit 462dc04cb6aaf3a876b56f27c6b511b00e25e85a in lucene-solr's branch 
refs/heads/master from Chris Hostetter
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=462dc04 ]

SOLR-9287: Including 'score' in the 'fl' param when doing an RTG no longer 
causes an NPE


> single node RTG: NPE if score is requested
> --
>
> Key: SOLR-9287
> URL: https://issues.apache.org/jira/browse/SOLR-9287
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Hoss Man
> Attachments: SOLR-9287.patch, SOLR-9287.patch
>
>
> Found in SOLR-9180 testing.
> In single node solr setups, if an RTG request is made that includes "score" 
> in the fl, then there is an NPE from ResultContext.wantsScores.
> This does *not* happen if the same request happens in a SolrCloud setup - in 
> that case the request for "score" is silently ignored -- this seems to me 
> like the optimal behavior  (similarly: using the {{\[explain\]}} transformer 
> in the fl for an RTG is currently silently ignored for both single node and 
> solr cloud envs)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-6.x-MacOSX (64bit/jdk1.8.0) - Build # 273 - Failure!

[Policeman Jenkins Server]

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-6.x-MacOSX/273/
Java: 64bit/jdk1.8.0 -XX:+UseCompressedOops -XX:+UseParallelGC

1 tests failed.
FAILED:  org.apache.solr.handler.TestReqParamsAPI.test

Error Message:
Could not get expected value  'CY val' for path 'params/c' full output: {   
"responseHeader":{ "status":0, "QTime":0},   "params":{ "a":"A 
val", "b":"B val", "wt":"json", "useParams":""},   "context":{ 
"webapp":"/if_yvi/ff", "path":"/dump1", "httpMethod":"GET"}},  from 
server:  http://127.0.0.1:54333/if_yvi/ff/collection1

Stack Trace:
java.lang.AssertionError: Could not get expected value  'CY val' for path 
'params/c' full output: {
  "responseHeader":{
"status":0,
"QTime":0},
  "params":{
"a":"A val",
"b":"B val",
"wt":"json",
"useParams":""},
  "context":{
"webapp":"/if_yvi/ff",
"path":"/dump1",
"httpMethod":"GET"}},  from server:  
http://127.0.0.1:54333/if_yvi/ff/collection1
at 
__randomizedtesting.SeedInfo.seed([61AAE02E53228E7C:E9FEDFF4FDDEE384]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at 
org.apache.solr.core.TestSolrConfigHandler.testForResponseElement(TestSolrConfigHandler.java:481)
at 
org.apache.solr.handler.TestReqParamsAPI.testReqParams(TestReqParamsAPI.java:171)
at 
org.apache.solr.handler.TestReqParamsAPI.test(TestReqParamsAPI.java:61)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 

[jira] [Commented] (SOLR-9295) Remove Unicode BOM (U+FEFF) from text files in codebase

[Erick Erickson (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371549#comment-15371549
 ] 

Erick Erickson commented on SOLR-9295:
--

Thanks for catching this!

> Remove Unicode BOM (U+FEFF) from text files in codebase
> ---
>
> Key: SOLR-9295
> URL: https://issues.apache.org/jira/browse/SOLR-9295
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: scripts and tools
>Affects Versions: master (7.0)
>Reporter: Shawn Heisey
>Priority: Trivial
> Fix For: 6.2, master (7.0)
>
>
> When starting Solr built from the master branch on Windows, this is what you 
> see:
> {noformat}
> C:\Users\elyograg\git\lucene-solr\solr>bin\solr start
> C:\Users\elyograg\git\lucene-solr\solr>@REM
> '@REM' is not recognized as an internal or external command,
> operable program or batch file.
> {noformat}
> The three extra characters, found at the very beginning of the solr.cmd 
> script, are a Unicode BOM, and are invisible to vim, notepad, and notepad++.  
> The problem does not exist in 6.1.0, but IS present in branch_6x and master.
> Using grep to find this character in the entire codebase, I found one other 
> relevant file with a BOM.  All others were binary (images, jars, git pack 
> files, etc):
> ./solr/webapp/web/js/lib/jquery.blockUI.js



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-master-MacOSX (64bit/jdk1.8.0) - Build # 3405 - Still Failing!

[Policeman Jenkins Server]

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-master-MacOSX/3405/
Java: 64bit/jdk1.8.0 -XX:-UseCompressedOops -XX:+UseConcMarkSweepGC

1 tests failed.
FAILED:  org.apache.solr.cloud.TestLocalFSCloudBackupRestore.test

Error Message:
Error from server at https://127.0.0.1:52747/solr: 'location' is not specified 
as a query parameter or as a default repository property or as a cluster 
property.

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error 
from server at https://127.0.0.1:52747/solr: 'location' is not specified as a 
query parameter or as a default repository property or as a cluster property.
at 
__randomizedtesting.SeedInfo.seed([B01D1BC172A320FF:3849241BDC5F4D07]:0)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:606)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:259)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:248)
at 
org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:413)
at 
org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:366)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1270)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:1040)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:976)
at 
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:149)
at 
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:166)
at 
org.apache.solr.cloud.AbstractCloudBackupRestoreTestCase.testInvalidPath(AbstractCloudBackupRestoreTestCase.java:149)
at 
org.apache.solr.cloud.AbstractCloudBackupRestoreTestCase.test(AbstractCloudBackupRestoreTestCase.java:128)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 

[jira] [Created] (SOLR-9296) Examine SortingResponseWriter with an eye towards removing extra object creation

[Erick Erickson (JIRA)]

Erick Erickson created SOLR-9296:


 Summary: Examine SortingResponseWriter with an eye towards 
removing extra object creation
 Key: SOLR-9296
 URL: https://issues.apache.org/jira/browse/SOLR-9296
 Project: Solr
  Issue Type: Improvement
  Security Level: Public (Default Security Level. Issues are Public)
Affects Versions: 6.2, master (7.0)
Reporter: Erick Erickson
Assignee: Erick Erickson


Assigning to myself just to keep from losing track it. Anyone who wants to take 
it, please feel free!

While looking at SOLR-9166 I noticed that SortingResponseWriter does a toString 
for each field it writes out. At a _very_ preliminary examination it seems like 
we create a lot of String objects that need to be GC'd. Could we reduce this by 
using some kind of CharsRef/ByteBuffer/Whatever?

I've only looked at this briefly, not quite sure what the gotchas are but 
throwing it out for discussion.

Some initial thoughts:
1> for the fixed types (numerics, dates, booleans) there's a strict upper limit 
on the size of each value so we can allocate something up-front.
2> for string fields, we already get a chars ref so just pass that through?
3> must make sure that whatever does the actual writing transfers all the bytes 
before returning.

I'm sure I won't get to this for a week or perhaps more, so grab it if you have 
the bandwidth.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-6.x-Windows (32bit/jdk1.8.0_92) - Build # 314 - Still Failing!

[Policeman Jenkins Server]

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-6.x-Windows/314/
Java: 32bit/jdk1.8.0_92 -client -XX:+UseG1GC

1 tests failed.
FAILED:  org.apache.solr.cloud.TestLocalFSCloudBackupRestore.test

Error Message:
Error from server at http://127.0.0.1:54725/solr: The backup directory already 
exists: 
file:///C:/Users/jenkins/workspace/Lucene-Solr-6.x-Windows/solr/build/solr-core/test/J1/temp/solr.cloud.TestLocalFSCloudBackupRestore_FBF9FDDCFD620298-001/tempDir-002/mytestbackup/

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error 
from server at http://127.0.0.1:54725/solr: The backup directory already 
exists: 
file:///C:/Users/jenkins/workspace/Lucene-Solr-6.x-Windows/solr/build/solr-core/test/J1/temp/solr.cloud.TestLocalFSCloudBackupRestore_FBF9FDDCFD620298-001/tempDir-002/mytestbackup/
at 
__randomizedtesting.SeedInfo.seed([FBF9FDDCFD620298:73ADC206539E6F60]:0)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:590)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:259)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:248)
at 
org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:403)
at 
org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:356)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1228)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:998)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:934)
at 
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:149)
at 
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:166)
at 
org.apache.solr.cloud.AbstractCloudBackupRestoreTestCase.testBackupAndRestore(AbstractCloudBackupRestoreTestCase.java:207)
at 
org.apache.solr.cloud.AbstractCloudBackupRestoreTestCase.test(AbstractCloudBackupRestoreTestCase.java:127)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 

[jira] [Updated] (SOLR-9295) Remove Unicode BOM (U+FEFF) from text files in codebase

[Shawn Heisey (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-9295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shawn Heisey updated SOLR-9295:
---
Description: 
When starting Solr built from the master branch on Windows, this is what you 
see:

{noformat}
C:\Users\elyograg\git\lucene-solr\solr>bin\solr start

C:\Users\elyograg\git\lucene-solr\solr>@REM
'@REM' is not recognized as an internal or external command,
operable program or batch file.
{noformat}

The three extra characters, found at the very beginning of the solr.cmd script, 
are a Unicode BOM, and are invisible to vim, notepad, and notepad++.  

The problem does not exist in 6.1.0, but IS present in branch_6x and master.

Using grep to find this character in the entire codebase, I found one other 
relevant file with a BOM.  All others were binary (images, jars, git pack 
files, etc):

./solr/webapp/web/js/lib/jquery.blockUI.js


  was:
When starting Solr built from the master branch on Windows, this is what you 
see:

{noformat}
C:\Users\elyograg\git\lucene-solr\solr>bin\solr start

C:\Users\elyograg\git\lucene-solr\solr>@REM
'@REM' is not recognized as an internal or external command,
operable program or batch file.
{noformat}

The three extra characters, found at the very beginning of the solr.cmd script, 
are a Unicode BOM, and are invisible to vim, notepad, and notepad++.  

The problem does not exist in 6.1.0, but IS present in branch_6x and master.

Using grep to find this character in the entire codebase, I found one other 
file with a BOM:

./solr/webapp/web/js/lib/jquery.blockUI.js



> Remove Unicode BOM (U+FEFF) from text files in codebase
> ---
>
> Key: SOLR-9295
> URL: https://issues.apache.org/jira/browse/SOLR-9295
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: scripts and tools
>Affects Versions: master (7.0)
>Reporter: Shawn Heisey
>Priority: Trivial
> Fix For: 6.2, master (7.0)
>
>
> When starting Solr built from the master branch on Windows, this is what you 
> see:
> {noformat}
> C:\Users\elyograg\git\lucene-solr\solr>bin\solr start
> C:\Users\elyograg\git\lucene-solr\solr>@REM
> '@REM' is not recognized as an internal or external command,
> operable program or batch file.
> {noformat}
> The three extra characters, found at the very beginning of the solr.cmd 
> script, are a Unicode BOM, and are invisible to vim, notepad, and notepad++.  
> The problem does not exist in 6.1.0, but IS present in branch_6x and master.
> Using grep to find this character in the entire codebase, I found one other 
> relevant file with a BOM.  All others were binary (images, jars, git pack 
> files, etc):
> ./solr/webapp/web/js/lib/jquery.blockUI.js



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-9295) Remove Unicode BOM (U+FEFF) from text files in codebase

[Shawn Heisey (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-9295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shawn Heisey updated SOLR-9295:
---
Fix Version/s: master (7.0)
   6.2

> Remove Unicode BOM (U+FEFF) from text files in codebase
> ---
>
> Key: SOLR-9295
> URL: https://issues.apache.org/jira/browse/SOLR-9295
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: scripts and tools
>Affects Versions: master (7.0)
>Reporter: Shawn Heisey
>Priority: Trivial
> Fix For: 6.2, master (7.0)
>
>
> When starting Solr built from the master branch on Windows, this is what you 
> see:
> {noformat}
> C:\Users\elyograg\git\lucene-solr\solr>bin\solr start
> C:\Users\elyograg\git\lucene-solr\solr>@REM
> '@REM' is not recognized as an internal or external command,
> operable program or batch file.
> {noformat}
> The three extra characters, found at the very beginning of the solr.cmd 
> script, are a Unicode BOM, and are invisible to vim, notepad, and notepad++.  
> The problem does not exist in 6.1.0, but IS present in branch_6x and master.
> Using grep to find this character in the entire codebase, I found one other 
> file with a BOM:
> ./solr/webapp/web/js/lib/jquery.blockUI.js



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9295) Remove Unicode BOM (U+FEFF) from text files in codebase

[Shawn Heisey (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371370#comment-15371370
 ] 

Shawn Heisey commented on SOLR-9295:


Immediate visible problem fixed.

> Remove Unicode BOM (U+FEFF) from text files in codebase
> ---
>
> Key: SOLR-9295
> URL: https://issues.apache.org/jira/browse/SOLR-9295
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: scripts and tools
>Affects Versions: master (7.0)
>Reporter: Shawn Heisey
>Priority: Trivial
> Fix For: 6.2, master (7.0)
>
>
> When starting Solr built from the master branch on Windows, this is what you 
> see:
> {noformat}
> C:\Users\elyograg\git\lucene-solr\solr>bin\solr start
> C:\Users\elyograg\git\lucene-solr\solr>@REM
> '@REM' is not recognized as an internal or external command,
> operable program or batch file.
> {noformat}
> The three extra characters, found at the very beginning of the solr.cmd 
> script, are a Unicode BOM, and are invisible to vim, notepad, and notepad++.  
> The problem does not exist in 6.1.0, but IS present in branch_6x and master.
> Using grep to find this character in the entire codebase, I found one other 
> file with a BOM:
> ./solr/webapp/web/js/lib/jquery.blockUI.js



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9295) Remove Unicode BOM (U+FEFF) from text files in codebase

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371364#comment-15371364
 ] 

ASF subversion and git services commented on SOLR-9295:
---

Commit 268021af9c84aebb7ec290623a7a252e9dbbbebd in lucene-solr's branch 
refs/heads/branch_6x from elyograg
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=268021a ]

SOLR-9295: Remove BOM (unicode Byte Order Mark) from text files.


> Remove Unicode BOM (U+FEFF) from text files in codebase
> ---
>
> Key: SOLR-9295
> URL: https://issues.apache.org/jira/browse/SOLR-9295
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: scripts and tools
>Affects Versions: master (7.0)
>Reporter: Shawn Heisey
>Priority: Trivial
>
> When starting Solr built from the master branch on Windows, this is what you 
> see:
> {noformat}
> C:\Users\elyograg\git\lucene-solr\solr>bin\solr start
> C:\Users\elyograg\git\lucene-solr\solr>@REM
> '@REM' is not recognized as an internal or external command,
> operable program or batch file.
> {noformat}
> The three extra characters, found at the very beginning of the solr.cmd 
> script, are a Unicode BOM, and are invisible to vim, notepad, and notepad++.  
> The problem does not exist in 6.1.0, but IS present in branch_6x and master.
> Using grep to find this character in the entire codebase, I found one other 
> file with a BOM:
> ./solr/webapp/web/js/lib/jquery.blockUI.js



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9294) DateMathParser Milliseconds separator

[Hoss Man (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371352#comment-15371352
 ] 

Hoss Man commented on SOLR-9294:


For updates it's already possible to configure your update processor chain to 
accept a wide variety of date formats...

https://lucene.apache.org/solr/6_1_0/solr-core/org/apache/solr/update/processor/ParseDateFieldUpdateProcessorFactory.html

...as a general purpose feature of date parsing/formatting in all aspects of 
request handling (ie: query parsing, DateRangeField, xml/json response 
writting, etc...) the scope of the problem gets much bigger.  

I personally think it adds a lot of complexity (both internally and for 
clients) and would prefer to stick to one canonical date representation.

> DateMathParser Milliseconds separator
> -
>
> Key: SOLR-9294
> URL: https://issues.apache.org/jira/browse/SOLR-9294
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: query parsers
>Affects Versions: 6.1
>Reporter: Curtis Fehr
>Priority: Minor
>
> DateMathParser is no longer accepting the format -xx-xxTxx:xx:xx:xxxZ 
> whereas it did in the past (5.3 works).  Changing the milliseconds separator 
> to "." fixes the issue, but this is a time consuming change for organizations 
> that use many data consumers.  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9295) Remove Unicode BOM (U+FEFF) from text files in codebase

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371324#comment-15371324
 ] 

ASF subversion and git services commented on SOLR-9295:
---

Commit e66ff585dda61fa8b3ebe83c69434d4f35bba0bb in lucene-solr's branch 
refs/heads/master from [~elyograg]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=e66ff58 ]

SOLR-9295: Remove BOM (unicode Byte Order Mark) from text files.


> Remove Unicode BOM (U+FEFF) from text files in codebase
> ---
>
> Key: SOLR-9295
> URL: https://issues.apache.org/jira/browse/SOLR-9295
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: scripts and tools
>Affects Versions: master (7.0)
>Reporter: Shawn Heisey
>Priority: Trivial
>
> When starting Solr built from the master branch on Windows, this is what you 
> see:
> {noformat}
> C:\Users\elyograg\git\lucene-solr\solr>bin\solr start
> C:\Users\elyograg\git\lucene-solr\solr>@REM
> '@REM' is not recognized as an internal or external command,
> operable program or batch file.
> {noformat}
> The three extra characters, found at the very beginning of the solr.cmd 
> script, are a Unicode BOM, and are invisible to vim, notepad, and notepad++.  
> The problem does not exist in 6.1.0, but IS present in branch_6x and master.
> Using grep to find this character in the entire codebase, I found one other 
> file with a BOM:
> ./solr/webapp/web/js/lib/jquery.blockUI.js



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9295) Remove Unicode BOM (U+FEFF) from text files in codebase

[Shawn Heisey (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371309#comment-15371309
 ] 

Shawn Heisey commented on SOLR-9295:


I wonder how difficult it would be put a check for BOM into the precommit 
target, and whether such a thing might be worthwhile.

> Remove Unicode BOM (U+FEFF) from text files in codebase
> ---
>
> Key: SOLR-9295
> URL: https://issues.apache.org/jira/browse/SOLR-9295
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: scripts and tools
>Affects Versions: master (7.0)
>Reporter: Shawn Heisey
>Priority: Trivial
>
> When starting Solr built from the master branch on Windows, this is what you 
> see:
> {noformat}
> C:\Users\elyograg\git\lucene-solr\solr>bin\solr start
> C:\Users\elyograg\git\lucene-solr\solr>@REM
> '@REM' is not recognized as an internal or external command,
> operable program or batch file.
> {noformat}
> The three extra characters, found at the very beginning of the solr.cmd 
> script, are a Unicode BOM, and are invisible to vim, notepad, and notepad++.  
> The problem does not exist in 6.1.0, but IS present in branch_6x and master.
> Using grep to find this character in the entire codebase, I found one other 
> file with a BOM:
> ./solr/webapp/web/js/lib/jquery.blockUI.js



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9294) DateMathParser Milliseconds separator

[Curtis Fehr (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371308#comment-15371308
 ] 

Curtis Fehr commented on SOLR-9294:
---

Hoss Man's answer is satisfactory and makes perfect sense, however, would it be 
beneficial to add additional format support?

> DateMathParser Milliseconds separator
> -
>
> Key: SOLR-9294
> URL: https://issues.apache.org/jira/browse/SOLR-9294
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: query parsers
>Affects Versions: 6.1
>Reporter: Curtis Fehr
>Priority: Minor
>
> DateMathParser is no longer accepting the format -xx-xxTxx:xx:xx:xxxZ 
> whereas it did in the past (5.3 works).  Changing the milliseconds separator 
> to "." fixes the issue, but this is a time consuming change for organizations 
> that use many data consumers.  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-9295) Remove Unicode BOM (U+FEFF) from text files in codebase

[Shawn Heisey (JIRA)]

Shawn Heisey created SOLR-9295:
--

 Summary: Remove Unicode BOM (U+FEFF) from text files in codebase
 Key: SOLR-9295
 URL: https://issues.apache.org/jira/browse/SOLR-9295
 Project: Solr
  Issue Type: Bug
  Security Level: Public (Default Security Level. Issues are Public)
  Components: scripts and tools
Affects Versions: master (7.0)
Reporter: Shawn Heisey
Priority: Trivial


When starting Solr built from the master branch on Windows, this is what you 
see:

{noformat}
C:\Users\elyograg\git\lucene-solr\solr>bin\solr start

C:\Users\elyograg\git\lucene-solr\solr>@REM
'@REM' is not recognized as an internal or external command,
operable program or batch file.
{noformat}

The three extra characters, found at the very beginning of the solr.cmd script, 
are a Unicode BOM, and are invisible to vim, notepad, and notepad++.  

The problem does not exist in 6.1.0, but IS present in branch_6x and master.

Using grep to find this character in the entire codebase, I found one other 
file with a BOM:

./solr/webapp/web/js/lib/jquery.blockUI.js




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-9240) Support running the topic() Streaming Expression in parallel mode.

[Joel Bernstein (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-9240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joel Bernstein updated SOLR-9240:
-
Attachment: SOLR-9240.patch

New patch that fixes some tests that were broken in the initial patch.

> Support running the topic() Streaming Expression in parallel mode.
> --
>
> Key: SOLR-9240
> URL: https://issues.apache.org/jira/browse/SOLR-9240
> Project: Solr
>  Issue Type: Improvement
>Reporter: Joel Bernstein
>Assignee: Joel Bernstein
> Attachments: SOLR-9240.patch, SOLR-9240.patch
>
>
> Currently the topic() function won't run in parallel mode because each worker 
> needs to maintain a separate set of checkpoints. The proposed solution for 
> this is to append the worker ID to the topic ID, which will cause each worker 
> to have it's own checkpoints.
> It would be useful to support parallelizing the topic function because it 
> will provide a general purpose approach for processing text in parallel 
> across worker nodes.
> For example this would allow a classify() function to be wrapped around a 
> topic() function to classify documents in parallel across worker nodes. 
> Sample syntax:
> {code}
> parallel(daemon(update(classify(topic(..., partitionKeys="id")
> {code}
> The example above would send a daemon to worker nodes that would classify all 
> documents returned by the topic() function. The update function would send 
> the output of classify() to a SolrCloud collection for indexing.
> The partitionKeys parameter would ensure that each worker would receive a 
> partition of the results returned by the topic() function. This allows the 
> classify() function to be run in parallel.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-master-Windows (64bit/jdk1.8.0_92) - Build # 5977 - Still Failing!

[Policeman Jenkins Server]

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-master-Windows/5977/
Java: 64bit/jdk1.8.0_92 -XX:-UseCompressedOops -XX:+UseSerialGC

1 tests failed.
FAILED:  org.apache.solr.cloud.TestLocalFSCloudBackupRestore.test

Error Message:
Error from server at http://127.0.0.1:54574/solr: The backup directory already 
exists: 
file:///C:/Users/jenkins/workspace/Lucene-Solr-master-Windows/solr/build/solr-core/test/J1/temp/solr.cloud.TestLocalFSCloudBackupRestore_7ED6AC07826583A5-001/tempDir-002/mytestbackup/

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error 
from server at http://127.0.0.1:54574/solr: The backup directory already 
exists: 
file:///C:/Users/jenkins/workspace/Lucene-Solr-master-Windows/solr/build/solr-core/test/J1/temp/solr.cloud.TestLocalFSCloudBackupRestore_7ED6AC07826583A5-001/tempDir-002/mytestbackup/
at 
__randomizedtesting.SeedInfo.seed([7ED6AC07826583A5:F68293DD2C99EE5D]:0)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:606)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:259)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:248)
at 
org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:413)
at 
org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:366)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1270)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:1040)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:976)
at 
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:149)
at 
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:166)
at 
org.apache.solr.cloud.AbstractCloudBackupRestoreTestCase.testBackupAndRestore(AbstractCloudBackupRestoreTestCase.java:207)
at 
org.apache.solr.cloud.AbstractCloudBackupRestoreTestCase.test(AbstractCloudBackupRestoreTestCase.java:127)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 

[jira] [Commented] (SOLR-9232) Can't Swap Cores in New Admin UI

[Shawn Heisey (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371246#comment-15371246
 ] 

Shawn Heisey commented on SOLR-9232:


Same problem in the master branch.

> Can't Swap Cores in New Admin UI
> 
>
> Key: SOLR-9232
> URL: https://issues.apache.org/jira/browse/SOLR-9232
> Project: Solr
>  Issue Type: Bug
>  Components: UI
>Affects Versions: 6.0, 6.1
>Reporter: Nikitas Tampakis
>Priority: Minor
> Attachments: swap-cores-ui-screenshot.png
>
>
> Using the Core Admin page on the new Admin UI, there is no way to submit a 
> Swap Cores request. After choosing the Swap action for an existing core and 
> selecting a core to swap with, clicking the Swap Cores button does nothing.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-9287) single node RTG: NPE if score is requested

[Hoss Man (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-9287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Hoss Man updated SOLR-9287:
---
Attachment: SOLR-9287.patch

updated patch modifying all 3 tests that were marked {{@AwaitFix(SOLR-9287)}} 
... 2 are now enabled, the other has been updated to be 
{{@AwaitsFix(SOLR-9285)}} since that issue still prevents the test from passing.

> single node RTG: NPE if score is requested
> --
>
> Key: SOLR-9287
> URL: https://issues.apache.org/jira/browse/SOLR-9287
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Hoss Man
> Attachments: SOLR-9287.patch, SOLR-9287.patch
>
>
> Found in SOLR-9180 testing.
> In single node solr setups, if an RTG request is made that includes "score" 
> in the fl, then there is an NPE from ResultContext.wantsScores.
> This does *not* happen if the same request happens in a SolrCloud setup - in 
> that case the request for "score" is silently ignored -- this seems to me 
> like the optimal behavior  (similarly: using the {{\[explain\]}} transformer 
> in the fl for an RTG is currently silently ignored for both single node and 
> solr cloud envs)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9232) Can't Swap Cores in New Admin UI

[Erick Erickson (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371225#comment-15371225
 ] 

Erick Erickson commented on SOLR-9232:
--

Hmmm, Anyone with UI and Angular JS skills would be _very_ welcome. The new UI 
has a lot of possibilities for expanding its functionality and the burden 
shouldn't be on a single person who's already done yeoman's duty with it. I'm 
totally clueless here...

> Can't Swap Cores in New Admin UI
> 
>
> Key: SOLR-9232
> URL: https://issues.apache.org/jira/browse/SOLR-9232
> Project: Solr
>  Issue Type: Bug
>  Components: UI
>Affects Versions: 6.0, 6.1
>Reporter: Nikitas Tampakis
>Priority: Minor
> Attachments: swap-cores-ui-screenshot.png
>
>
> Using the Core Admin page on the new Admin UI, there is no way to submit a 
> Swap Cores request. After choosing the Swap action for an existing core and 
> selecting a core to swap with, clicking the Swap Cores button does nothing.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9232) Can't Swap Cores in New Admin UI

[Shawn Heisey (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371184#comment-15371184
 ] 

Shawn Heisey commented on SOLR-9232:


Just tried it myself with a newly downloaded Solr 6.1.0.

The old admin UI works.  The new one doesn't.  When Nikitas says that the 
"Swap" button does nothing ... he means that *absolutely* nothing happens.  
It's not just that the swap doesn't happen ... the UI literally doesn't change 
at all.

> Can't Swap Cores in New Admin UI
> 
>
> Key: SOLR-9232
> URL: https://issues.apache.org/jira/browse/SOLR-9232
> Project: Solr
>  Issue Type: Bug
>  Components: UI
>Affects Versions: 6.0, 6.1
>Reporter: Nikitas Tampakis
>Priority: Minor
> Attachments: swap-cores-ui-screenshot.png
>
>
> Using the Core Admin page on the new Admin UI, there is no way to submit a 
> Swap Cores request. After choosing the Swap action for an existing core and 
> selecting a core to swap with, clicking the Swap Cores button does nothing.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-9240) Support running the topic() Streaming Expression in parallel mode.

[Joel Bernstein (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-9240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joel Bernstein updated SOLR-9240:
-
Attachment: SOLR-9240.patch

Initial patch.

> Support running the topic() Streaming Expression in parallel mode.
> --
>
> Key: SOLR-9240
> URL: https://issues.apache.org/jira/browse/SOLR-9240
> Project: Solr
>  Issue Type: Improvement
>Reporter: Joel Bernstein
>Assignee: Joel Bernstein
> Attachments: SOLR-9240.patch
>
>
> Currently the topic() function won't run in parallel mode because each worker 
> needs to maintain a separate set of checkpoints. The proposed solution for 
> this is to append the worker ID to the topic ID, which will cause each worker 
> to have it's own checkpoints.
> It would be useful to support parallelizing the topic function because it 
> will provide a general purpose approach for processing text in parallel 
> across worker nodes.
> For example this would allow a classify() function to be wrapped around a 
> topic() function to classify documents in parallel across worker nodes. 
> Sample syntax:
> {code}
> parallel(daemon(update(classify(topic(..., partitionKeys="id")
> {code}
> The example above would send a daemon to worker nodes that would classify all 
> documents returned by the topic() function. The update function would send 
> the output of classify() to a SolrCloud collection for indexing.
> The partitionKeys parameter would ensure that each worker would receive a 
> partition of the results returned by the topic() function. This allows the 
> classify() function to be run in parallel.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9232) Can't Swap Cores in New Admin UI

[Shawn Heisey (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371132#comment-15371132
 ] 

Shawn Heisey commented on SOLR-9232:


Core swapping would do very bad things if you're in cloud mode.  So I need to 
make sure you're not in cloud mode.  If you are, then it would be correct for 
it to not work, but you should get an error message instead of what's happening 
now.

If you're not in cloud mode, then this is a bigger problem.  Note that you 
should be able to use the CoreAdmin API directly (in non-cloud mode) to 
accomplish the swap.  I would like to know whether this works, or if it's just 
the admin UI.

https://cwiki.apache.org/confluence/display/solr/CoreAdmin+API


> Can't Swap Cores in New Admin UI
> 
>
> Key: SOLR-9232
> URL: https://issues.apache.org/jira/browse/SOLR-9232
> Project: Solr
>  Issue Type: Bug
>  Components: UI
>Affects Versions: 6.0, 6.1
>Reporter: Nikitas Tampakis
>Priority: Minor
> Attachments: swap-cores-ui-screenshot.png
>
>
> Using the Core Admin page on the new Admin UI, there is no way to submit a 
> Swap Cores request. After choosing the Swap action for an existing core and 
> selecting a core to swap with, clicking the Swap Cores button does nothing.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9252) Feature selection and logistic regression on text

[Joel Bernstein (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371129#comment-15371129
 ] 

Joel Bernstein commented on SOLR-9252:
--

Ok I reviewed the TextLogitStream and it looks great! The 
ClassificationEvaluation is really nice.

Really the whole patch looks very good.

What I need to do now is test with a few different data sets. This will verify 
the results that [~caomanhdat] has been getting. It will also test out the 
mechanics of running the functions and storing and retrieving the features and 
models.


> Feature selection and logistic regression on text
> -
>
> Key: SOLR-9252
> URL: https://issues.apache.org/jira/browse/SOLR-9252
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Cao Manh Dat
>Assignee: Joel Bernstein
> Attachments: SOLR-9252.patch, SOLR-9252.patch, enron1.zip
>
>
> SOLR-9186 come up with a challenges that for each iterative we have to 
> rebuild the tf-idf vector for each documents. It is costly computation if we 
> represent doc by a lot of terms. Features selection can help reducing the 
> computation.
> Due to its computational efficiency and simple interpretation, information 
> gain is one of the most popular feature selection methods. It is used to 
> measure the dependence between features and labels and calculates the 
> information gain between the i-th feature and the class labels 
> (http://www.jiliang.xyz/publication/feature_selection_for_classification.pdf).
> I confirmed that by running logistics regressions on enron mail dataset (in 
> which each email is represented by top 100 terms that have highest 
> information gain) and got the accuracy by 92% and precision by 82%.
> This ticket will create two new streaming expression. Both of them use the 
> same *parallel iterative framework* as SOLR-8492.
> {code}
> featuresSelection(collection1, q="*:*",  field="tv_text", outcome="out_i", 
> positiveLabel=1, numTerms=100)
> {code}
> featuresSelection will emit top terms that have highest information gain 
> scores. It can be combined with new tlogit stream.
> {code}
> tlogit(collection1, q="*:*",
>  featuresSelection(collection1, 
>   q="*:*",  
>   field="tv_text", 
>   outcome="out_i", 
>   positiveLabel=1, 
>   numTerms=100),
>  field="tv_text",
>  outcome="out_i",
>  maxIterations=100)
> {code}
> In the iteration n, the text logistics regression will emit nth model, and 
> compute the error of (n-1)th model. Because the error will be wrong if we 
> compute the error dynamically in each iteration. 
> In each iteration tlogit will change learning rate based on error of previous 
> iteration. It will increase the learning rate by 5% if error is going down 
> and It will decrease the learning rate by 50% if error is going up.
> This will support use cases such as building models for spam detection, 
> sentiment analysis and threat detection. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9200) Add Delegation Token Support to Solr

[Anshum Gupta (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371126#comment-15371126
 ] 

Anshum Gupta commented on SOLR-9200:


Don't think I can get to this one today or tomorrow but I'll try and take a 
look whenever I get time.

> Add Delegation Token Support to Solr
> 
>
> Key: SOLR-9200
> URL: https://issues.apache.org/jira/browse/SOLR-9200
> Project: Solr
>  Issue Type: New Feature
>  Components: security
>Reporter: Gregory Chanan
>Assignee: Gregory Chanan
> Attachments: SOLR-9200.patch, SOLR-9200.patch
>
>
> SOLR-7468 added support for kerberos authentication via the hadoop 
> authentication filter.  Hadoop also has support for an authentication filter 
> that supports delegation tokens, which allow authenticated users the ability 
> to grab/renew/delete a token that can be used to bypass the normal 
> authentication path for a time.  This is useful in a variety of use cases:
> 1) distributed clients (e.g. MapReduce) where each client may not have access 
> to the user's kerberos credentials.  Instead, the job runner can grab a 
> delegation token and use that during task execution.
> 2) If the load on the kerberos server is too high, delegation tokens can 
> avoid hitting the kerberos server after the first request
> 3) If requests/permissions need to be delegated to another user: the more 
> privileged user can request a delegation token that can be passed to the less 
> privileged user.
> Note to self:
> In 
> https://issues.apache.org/jira/browse/SOLR-7468?focusedCommentId=14579636=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14579636
>  I made the following comment which I need to investigate further, since I 
> don't know if anything changed in this area:
> {quote}3) I'm a little concerned with the "NoContext" code in KerberosPlugin 
> moving forward (I understand this is more a generic auth question than 
> kerberos specific). For example, in the latest version of the filter we are 
> using at Cloudera, we play around with the ServletContext in order to pass 
> information around 
> (https://github.com/cloudera/lucene-solr/blob/cdh5-4.10.3_5.4.2/solr/core/src/java/org/apache/solr/servlet/SolrHadoopAuthenticationFilter.java#L106).
>  Is there any way we can get the actual ServletContext in a plugin?{quote}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9294) DateMathParser Milliseconds separator

[Hoss Man (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371108#comment-15371108
 ] 

Hoss Man commented on SOLR-9294:


That (ie: ":" between seconds and milliseconds) was never an officially 
supported format .. it was certainly never documented to be valid.

6.0 changed a lot of internal date parsing/formatting code as part of SOLR-8904 
to fix numerious bugs, fixing this sloppy parsing logic was evidently a side 
effect of that issue.

> DateMathParser Milliseconds separator
> -
>
> Key: SOLR-9294
> URL: https://issues.apache.org/jira/browse/SOLR-9294
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: query parsers
>Affects Versions: 6.1
>Reporter: Curtis Fehr
>Priority: Minor
>
> DateMathParser is no longer accepting the format -xx-xxTxx:xx:xx:xxxZ 
> whereas it did in the past (5.3 works).  Changing the milliseconds separator 
> to "." fixes the issue, but this is a time consuming change for organizations 
> that use many data consumers.  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9292) Suggester not working for more than 200K records

[Erick Erickson (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371083#comment-15371083
 ] 

Erick Erickson commented on SOLR-9292:
--

govind:

Have you tracked any possible code issues? If not, it's usually best to ask 
these kinds of questions on the user's list first, it'll be seen by a _lot_ 
more eyes then raise a JIRA if it's confirmed as something to be addressed in 
code.

As you can tell I haven't a clue about the underlying issue...



> Suggester not working for more than 200K records
> 
>
> Key: SOLR-9292
> URL: https://issues.apache.org/jira/browse/SOLR-9292
> Project: Solr
>  Issue Type: Test
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Suggester
>Affects Versions: 5.5
> Environment: ubuntu 14.04, open jdk 1.8 
>Reporter: govind
>
> Using lookup suggester and fuzzy lookup suggester in combination. Data 
> uploaded to solr through json. Up to 200K records, both suggesters are 
> working fine, but after words they are not working.
> Pushing Data as: 
> curl -X POST -H 'Content-Type: application/json' 
> 'http://localhost:8983/solr/demo_autocomplete/update?versions=false' 
> --data-binary @data.json
> Reload as: curl 
> "http://localhost:8983/solr/admin/cores?wt=json=RELOAD=demo_autocomplete;
> and after that building and testing the suggester as:
> url 
> "http://localhost:8983/solr/demo_autocomplete/suggest?suggest.dictionary=fuzzySuggester=true=true=test=json;
> When I insert more than 200K records, it gives error as:
> at 
> org.apache.lucene.util.automaton.Operations.topoSortStatesRecurse(Operations.java:1311)
> ---
> Is there any config I am missing or Do I need to modify some default 
> limitations ?
> Regards,
> Govind



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9294) DateMathParser Milliseconds separator

[Erick Erickson (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371077#comment-15371077
 ] 

Erick Erickson commented on SOLR-9294:
--

That might be true of comma too, i.e.
-xx-xxTxx:xx:xx,xxxZ

At least a program I'm pretty sure I used to use to populate test indexes 
suddenly threw a parsing error, I kind of assumed I'd inadvertently changed ti 
but this JIRA makes me wonder.

Just FYI, I don't really have a strong opinion either way.

> DateMathParser Milliseconds separator
> -
>
> Key: SOLR-9294
> URL: https://issues.apache.org/jira/browse/SOLR-9294
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: query parsers
>Affects Versions: 6.1
>Reporter: Curtis Fehr
>Priority: Minor
>
> DateMathParser is no longer accepting the format -xx-xxTxx:xx:xx:xxxZ 
> whereas it did in the past (5.3 works).  Changing the milliseconds separator 
> to "." fixes the issue, but this is a time consuming change for organizations 
> that use many data consumers.  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7373) Break out Directory.syncMetaData from FSDirectory.renameFile

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371007#comment-15371007
 ] 

ASF subversion and git services commented on LUCENE-7373:
-

Commit df9efb8b6da5195d6c8454cb7fd9b91147cb93fd in lucene-solr's branch 
refs/heads/master from Mike McCandless
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=df9efb8 ]

LUCENE-7373: deprecate Directory.renameFile, which both renamed and fsync'd the 
directory, replacing it with separate rename and syncMetaData methods


> Break out Directory.syncMetaData from FSDirectory.renameFile
> 
>
> Key: LUCENE-7373
> URL: https://issues.apache.org/jira/browse/LUCENE-7373
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Michael McCandless
>Assignee: Michael McCandless
> Fix For: master (7.0), 6.2
>
> Attachments: LUCENE-7373.patch
>
>
> Today, when you call {{FSDirectory.renameFile}} it also calls fsync on
> the directory.
> This is OK for Lucene's current usage of this method, to rename just
> the one {{segments_N}} file on commit.
> But in playing with adding NRT replication (LUCENE-5438) to the simple
> demo Lucene server (LUCENE-5376) I found that, on spinning disks, that
> fsync is very costly, because when copying over an NRT point, we write
> to N .tmp files and then rename many files (taking seconds) in the
> end.
> I think we should just deprecate/remove the existing method, and make a new
> {{rename}} method that does only renaming, and a separate
> {{syncMetaData}} to call fsync on the directory?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-NightlyTests-master - Build # 1068 - Still Failing

[Apache Jenkins Server]

Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-master/1068/

12 tests failed.
FAILED:  
org.apache.solr.cloud.CdcrReplicationDistributedZkTest.testReplicationAfterLeaderChange

Error Message:
Timeout while trying to assert number of documents @ target_collection

Stack Trace:
java.lang.AssertionError: Timeout while trying to assert number of documents @ 
target_collection
at 
__randomizedtesting.SeedInfo.seed([6C380F77C37EF848:BEC843949DD15E7A]:0)
at 
org.apache.solr.cloud.BaseCdcrDistributedZkTest.assertNumDocs(BaseCdcrDistributedZkTest.java:268)
at 
org.apache.solr.cloud.CdcrReplicationDistributedZkTest.testReplicationAfterLeaderChange(CdcrReplicationDistributedZkTest.java:291)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:985)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:960)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 

[jira] [Created] (SOLR-9294) DateMathParser Milliseconds separator

[Curtis Fehr (JIRA)]

Curtis Fehr created SOLR-9294:
-

 Summary: DateMathParser Milliseconds separator
 Key: SOLR-9294
 URL: https://issues.apache.org/jira/browse/SOLR-9294
 Project: Solr
  Issue Type: Bug
  Security Level: Public (Default Security Level. Issues are Public)
  Components: query parsers
Affects Versions: 6.1
Reporter: Curtis Fehr
Priority: Minor


DateMathParser is no longer accepting the format -xx-xxTxx:xx:xx:xxxZ 
whereas it did in the past (5.3 works).  Changing the milliseconds separator to 
"." fixes the issue, but this is a time consuming change for organizations that 
use many data consumers.  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-master-Linux (32bit/jdk1.8.0_92) - Build # 17218 - Still Failing!

[Policeman Jenkins Server]

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/17218/
Java: 32bit/jdk1.8.0_92 -client -XX:+UseG1GC

1 tests failed.
FAILED:  org.apache.solr.cloud.LeaderFailoverAfterPartitionTest.test

Error Message:
No live SolrServers available to handle this 
request:[http://127.0.0.1:41386/c8n_1x3_lf_shard1_replica3]

Stack Trace:
org.apache.solr.client.solrj.impl.CloudSolrClient$RouteException: No live 
SolrServers available to handle this 
request:[http://127.0.0.1:41386/c8n_1x3_lf_shard1_replica3]
at 
__randomizedtesting.SeedInfo.seed([7645D36716A0BF0B:FE11ECBDB85CD2F3]:0)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.directUpdate(CloudSolrClient.java:739)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1151)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:1040)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:976)
at 
org.apache.solr.cloud.AbstractFullDistribZkTestBase.sendDocsWithRetry(AbstractFullDistribZkTestBase.java:753)
at 
org.apache.solr.cloud.AbstractFullDistribZkTestBase.sendDocsWithRetry(AbstractFullDistribZkTestBase.java:741)
at 
org.apache.solr.cloud.LeaderFailoverAfterPartitionTest.testRf3WithLeaderFailover(LeaderFailoverAfterPartitionTest.java:178)
at 
org.apache.solr.cloud.LeaderFailoverAfterPartitionTest.test(LeaderFailoverAfterPartitionTest.java:57)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:985)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:960)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 

[jira] [Resolved] (LUCENE-7373) Break out Directory.syncMetaData from FSDirectory.renameFile

[Michael McCandless (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-7373?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael McCandless resolved LUCENE-7373.

Resolution: Fixed

> Break out Directory.syncMetaData from FSDirectory.renameFile
> 
>
> Key: LUCENE-7373
> URL: https://issues.apache.org/jira/browse/LUCENE-7373
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Michael McCandless
>Assignee: Michael McCandless
> Fix For: master (7.0), 6.2
>
> Attachments: LUCENE-7373.patch
>
>
> Today, when you call {{FSDirectory.renameFile}} it also calls fsync on
> the directory.
> This is OK for Lucene's current usage of this method, to rename just
> the one {{segments_N}} file on commit.
> But in playing with adding NRT replication (LUCENE-5438) to the simple
> demo Lucene server (LUCENE-5376) I found that, on spinning disks, that
> fsync is very costly, because when copying over an NRT point, we write
> to N .tmp files and then rename many files (taking seconds) in the
> end.
> I think we should just deprecate/remove the existing method, and make a new
> {{rename}} method that does only renaming, and a separate
> {{syncMetaData}} to call fsync on the directory?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7373) Break out Directory.syncMetaData from FSDirectory.renameFile

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371008#comment-15371008
 ] 

ASF subversion and git services commented on LUCENE-7373:
-

Commit 3125fccd8dde211567c49a6fe508f80ace3b053b in lucene-solr's branch 
refs/heads/master from Mike McCandless
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=3125fcc ]

LUCENE-7373: remove deprecated API


> Break out Directory.syncMetaData from FSDirectory.renameFile
> 
>
> Key: LUCENE-7373
> URL: https://issues.apache.org/jira/browse/LUCENE-7373
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Michael McCandless
>Assignee: Michael McCandless
> Fix For: master (7.0), 6.2
>
> Attachments: LUCENE-7373.patch
>
>
> Today, when you call {{FSDirectory.renameFile}} it also calls fsync on
> the directory.
> This is OK for Lucene's current usage of this method, to rename just
> the one {{segments_N}} file on commit.
> But in playing with adding NRT replication (LUCENE-5438) to the simple
> demo Lucene server (LUCENE-5376) I found that, on spinning disks, that
> fsync is very costly, because when copying over an NRT point, we write
> to N .tmp files and then rename many files (taking seconds) in the
> end.
> I think we should just deprecate/remove the existing method, and make a new
> {{rename}} method that does only renaming, and a separate
> {{syncMetaData}} to call fsync on the directory?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9252) Feature selection and logistic regression on text

[Joel Bernstein (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15370987#comment-15370987
 ] 

Joel Bernstein commented on SOLR-9252:
--

I think we should postfix the textlogiststream also.

A standard schema would be nice, but I don't know if it will be possible. For 
example the logit() model is pretty different then the tlogit() model.

I'll provide some more feedback on textlogitstream shortly.



> Feature selection and logistic regression on text
> -
>
> Key: SOLR-9252
> URL: https://issues.apache.org/jira/browse/SOLR-9252
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Cao Manh Dat
>Assignee: Joel Bernstein
> Attachments: SOLR-9252.patch, SOLR-9252.patch, enron1.zip
>
>
> SOLR-9186 come up with a challenges that for each iterative we have to 
> rebuild the tf-idf vector for each documents. It is costly computation if we 
> represent doc by a lot of terms. Features selection can help reducing the 
> computation.
> Due to its computational efficiency and simple interpretation, information 
> gain is one of the most popular feature selection methods. It is used to 
> measure the dependence between features and labels and calculates the 
> information gain between the i-th feature and the class labels 
> (http://www.jiliang.xyz/publication/feature_selection_for_classification.pdf).
> I confirmed that by running logistics regressions on enron mail dataset (in 
> which each email is represented by top 100 terms that have highest 
> information gain) and got the accuracy by 92% and precision by 82%.
> This ticket will create two new streaming expression. Both of them use the 
> same *parallel iterative framework* as SOLR-8492.
> {code}
> featuresSelection(collection1, q="*:*",  field="tv_text", outcome="out_i", 
> positiveLabel=1, numTerms=100)
> {code}
> featuresSelection will emit top terms that have highest information gain 
> scores. It can be combined with new tlogit stream.
> {code}
> tlogit(collection1, q="*:*",
>  featuresSelection(collection1, 
>   q="*:*",  
>   field="tv_text", 
>   outcome="out_i", 
>   positiveLabel=1, 
>   numTerms=100),
>  field="tv_text",
>  outcome="out_i",
>  maxIterations=100)
> {code}
> In the iteration n, the text logistics regression will emit nth model, and 
> compute the error of (n-1)th model. Because the error will be wrong if we 
> compute the error dynamically in each iteration. 
> In each iteration tlogit will change learning rate based on error of previous 
> iteration. It will increase the learning rate by 5% if error is going down 
> and It will decrease the learning rate by 50% if error is going up.
> This will support use cases such as building models for spam detection, 
> sentiment analysis and threat detection. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7373) Break out Directory.syncMetaData from FSDirectory.renameFile

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15370950#comment-15370950
 ] 

ASF subversion and git services commented on LUCENE-7373:
-

Commit 621a98faa29f11d59d1f9312f7d6674519db38f3 in lucene-solr's branch 
refs/heads/branch_6x from Mike McCandless
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=621a98f ]

LUCENE-7373: deprecate Directory.renameFile, which both renamed and fsync'd the 
directory, replacing it with separate rename and syncMetaData methods


> Break out Directory.syncMetaData from FSDirectory.renameFile
> 
>
> Key: LUCENE-7373
> URL: https://issues.apache.org/jira/browse/LUCENE-7373
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Michael McCandless
>Assignee: Michael McCandless
> Fix For: master (7.0), 6.2
>
> Attachments: LUCENE-7373.patch
>
>
> Today, when you call {{FSDirectory.renameFile}} it also calls fsync on
> the directory.
> This is OK for Lucene's current usage of this method, to rename just
> the one {{segments_N}} file on commit.
> But in playing with adding NRT replication (LUCENE-5438) to the simple
> demo Lucene server (LUCENE-5376) I found that, on spinning disks, that
> fsync is very costly, because when copying over an NRT point, we write
> to N .tmp files and then rename many files (taking seconds) in the
> end.
> I think we should just deprecate/remove the existing method, and make a new
> {{rename}} method that does only renaming, and a separate
> {{syncMetaData}} to call fsync on the directory?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9252) Feature selection and logistic regression on text

[Cao Manh Dat (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15370914#comment-15370914
 ] 

Cao Manh Dat commented on SOLR-9252:


Hi Joel,

Should we add postfix for textlogitstream output too? Do you think we should 
define a standard schema for all ML stream output?

> Feature selection and logistic regression on text
> -
>
> Key: SOLR-9252
> URL: https://issues.apache.org/jira/browse/SOLR-9252
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Cao Manh Dat
>Assignee: Joel Bernstein
> Attachments: SOLR-9252.patch, SOLR-9252.patch, enron1.zip
>
>
> SOLR-9186 come up with a challenges that for each iterative we have to 
> rebuild the tf-idf vector for each documents. It is costly computation if we 
> represent doc by a lot of terms. Features selection can help reducing the 
> computation.
> Due to its computational efficiency and simple interpretation, information 
> gain is one of the most popular feature selection methods. It is used to 
> measure the dependence between features and labels and calculates the 
> information gain between the i-th feature and the class labels 
> (http://www.jiliang.xyz/publication/feature_selection_for_classification.pdf).
> I confirmed that by running logistics regressions on enron mail dataset (in 
> which each email is represented by top 100 terms that have highest 
> information gain) and got the accuracy by 92% and precision by 82%.
> This ticket will create two new streaming expression. Both of them use the 
> same *parallel iterative framework* as SOLR-8492.
> {code}
> featuresSelection(collection1, q="*:*",  field="tv_text", outcome="out_i", 
> positiveLabel=1, numTerms=100)
> {code}
> featuresSelection will emit top terms that have highest information gain 
> scores. It can be combined with new tlogit stream.
> {code}
> tlogit(collection1, q="*:*",
>  featuresSelection(collection1, 
>   q="*:*",  
>   field="tv_text", 
>   outcome="out_i", 
>   positiveLabel=1, 
>   numTerms=100),
>  field="tv_text",
>  outcome="out_i",
>  maxIterations=100)
> {code}
> In the iteration n, the text logistics regression will emit nth model, and 
> compute the error of (n-1)th model. Because the error will be wrong if we 
> compute the error dynamically in each iteration. 
> In each iteration tlogit will change learning rate based on error of previous 
> iteration. It will increase the learning rate by 5% if error is going down 
> and It will decrease the learning rate by 50% if error is going up.
> This will support use cases such as building models for spam detection, 
> sentiment analysis and threat detection. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS-EA] Lucene-Solr-6.x-Linux (64bit/jdk-9-ea+126) - Build # 1122 - Failure!

[Policeman Jenkins Server]

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-6.x-Linux/1122/
Java: 64bit/jdk-9-ea+126 -XX:+UseCompressedOops -XX:+UseG1GC

1 tests failed.
FAILED:  org.apache.solr.cloud.SyncSliceTest.test

Error Message:
timeout waiting to see all nodes active

Stack Trace:
java.lang.AssertionError: timeout waiting to see all nodes active
at 
__randomizedtesting.SeedInfo.seed([2DA2F81F935571BB:A5F6C7C53DA91C43]:0)
at org.junit.Assert.fail(Assert.java:93)
at 
org.apache.solr.cloud.SyncSliceTest.waitTillAllNodesActive(SyncSliceTest.java:235)
at org.apache.solr.cloud.SyncSliceTest.test(SyncSliceTest.java:164)
at 
jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(java.base@9-ea/Native 
Method)
at 
jdk.internal.reflect.NativeMethodAccessorImpl.invoke(java.base@9-ea/NativeMethodAccessorImpl.java:62)
at 
jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(java.base@9-ea/DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(java.base@9-ea/Method.java:533)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 

[JENKINS] Lucene-Solr-master-Solaris (64bit/jdk1.8.0) - Build # 711 - Still Failing!

[Policeman Jenkins Server]

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-master-Solaris/711/
Java: 64bit/jdk1.8.0 -XX:+UseCompressedOops -XX:+UseConcMarkSweepGC

2 tests failed.
FAILED:  
org.apache.solr.cloud.TestTolerantUpdateProcessorCloud.testVariousAddsViaNoCollectionClient

Error Message:
Could not load collection from ZK: test_col

Stack Trace:
org.apache.solr.common.SolrException: Could not load collection from ZK: 
test_col
at 
__randomizedtesting.SeedInfo.seed([20873478A28CAA56:64C9B64C8825C358]:0)
at 
org.apache.solr.common.cloud.ZkStateReader.getCollectionLive(ZkStateReader.java:1072)
at 
org.apache.solr.common.cloud.ZkStateReader$LazyCollectionRef.get(ZkStateReader.java:638)
at 
org.apache.solr.common.cloud.ClusterState.getCollectionOrNull(ClusterState.java:211)
at 
org.apache.solr.common.cloud.ClusterState.hasCollection(ClusterState.java:113)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.getCollectionNames(CloudSolrClient.java:1281)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:1003)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:976)
at 
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:149)
at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:942)
at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:957)
at 
org.apache.solr.cloud.TestTolerantUpdateProcessorCloud.assertQueryDocIds(TestTolerantUpdateProcessorCloud.java:1020)
at 
org.apache.solr.cloud.TestTolerantUpdateProcessorCloud.testVariousAdds(TestTolerantUpdateProcessorCloud.java:431)
at 
org.apache.solr.cloud.TestTolerantUpdateProcessorCloud.testVariousAddsViaNoCollectionClient(TestTolerantUpdateProcessorCloud.java:411)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 

[jira] [Commented] (LUCENE-7375) Can we allow FSDirectory subclasses to customize whether the ctor does a mkdir?

[Robert Muir (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15370826#comment-15370826
 ] 

Robert Muir commented on LUCENE-7375:
-

Also, think about any external use of `getDirectory()` and so on (this is even 
used by toString). If someone calls this method, e.g. to key into a map of 
their own, we've brought the old java.io.File trap back :)

So the Files.createDirectories isn't useless in that sense either, it returns 
{{toRealPath}} today, which means such traps are impossible.

> Can we allow FSDirectory subclasses to customize whether the ctor does a 
> mkdir?
> ---
>
> Key: LUCENE-7375
> URL: https://issues.apache.org/jira/browse/LUCENE-7375
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Michael McCandless
>
> Today, just instantiating an {{FSDirectory}} always brings the directory into 
> existence, even if you will do no writes ({{createOutput}}).
> We have gone back and forth on this, over the ages.  E.g. see LUCENE-16 (only 
> 2 digits there!!), LUCENE-773 (only 3 digits there!!), LUCENE-1464.  At one 
> point we created the directory lazily, on the first write ({{createOutput}}) 
> attempt, but now we always create it when you instantiate {{FSDirectory}}.
> This causes some hassle for consumers, e.g. in 
> https://github.com/elastic/elasticsearch/pull/19338 ES is forking 
> {{SimpleFSDirectory}} in order to have a read-only directory impl.
> Maybe we can do the {{Files.createDirectories}} in protected method that a 
> subclass could override?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-8995) Minimize code with Lambdas

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-8995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15370822#comment-15370822
 ] 

ASF subversion and git services commented on SOLR-8995:
---

Commit 8db04e3457ea6c9a047cf61936ed8cfec3b07fb4 in lucene-solr's branch 
refs/heads/branch_6x from [~noble.paul]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=8db04e3 ]

SOLR-8995: Use Lamdas for Thread/Runnable


> Minimize code with Lambdas
> --
>
> Key: SOLR-8995
> URL: https://issues.apache.org/jira/browse/SOLR-8995
> Project: Solr
>  Issue Type: Task
>Reporter: Noble Paul
>Assignee: Noble Paul
> Attachments: SOLR-8995.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-8995) Minimize code with Lambdas

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-8995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15370820#comment-15370820
 ] 

ASF subversion and git services commented on SOLR-8995:
---

Commit 74de196565eb91e14e584c0a44569091ec3513fd in lucene-solr's branch 
refs/heads/master from [~noble.paul]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=74de196 ]

SOLR-8995: Use Lamdas for Thread/Runnable


> Minimize code with Lambdas
> --
>
> Key: SOLR-8995
> URL: https://issues.apache.org/jira/browse/SOLR-8995
> Project: Solr
>  Issue Type: Task
>Reporter: Noble Paul
>Assignee: Noble Paul
> Attachments: SOLR-8995.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7375) Can we allow FSDirectory subclasses to customize whether the ctor does a mkdir?

[Robert Muir (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15370815#comment-15370815
 ] 

Robert Muir commented on LUCENE-7375:
-

Well, another option is that NativeFSLockFactory etc already do their own 
createDirectories + toRealPath (defensively). This came after the move to 
nio.2, and maybe the one in FSDir can now just be removed.

This would require a bit of hair though: I think the only clean way to do it, 
would be to say that the lockfactory is responsible for creating the directory 
if it doesnt exist. That has implications for NO_LOCK_FACTORY and other crazy 
options we have :)

> Can we allow FSDirectory subclasses to customize whether the ctor does a 
> mkdir?
> ---
>
> Key: LUCENE-7375
> URL: https://issues.apache.org/jira/browse/LUCENE-7375
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Michael McCandless
>
> Today, just instantiating an {{FSDirectory}} always brings the directory into 
> existence, even if you will do no writes ({{createOutput}}).
> We have gone back and forth on this, over the ages.  E.g. see LUCENE-16 (only 
> 2 digits there!!), LUCENE-773 (only 3 digits there!!), LUCENE-1464.  At one 
> point we created the directory lazily, on the first write ({{createOutput}}) 
> attempt, but now we always create it when you instantiate {{FSDirectory}}.
> This causes some hassle for consumers, e.g. in 
> https://github.com/elastic/elasticsearch/pull/19338 ES is forking 
> {{SimpleFSDirectory}} in order to have a read-only directory impl.
> Maybe we can do the {{Files.createDirectories}} in protected method that a 
> subclass could override?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7373) Break out Directory.syncMetaData from FSDirectory.renameFile

[Uwe Schindler (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15370806#comment-15370806
 ] 

Uwe Schindler commented on LUCENE-7373:
---

Stromg +1
I discussed that a while ago with [~rcmuir], but at that time we said "keep it 
simple because we only use renameFile for committing". But now this changes. 
LGTM.

> Break out Directory.syncMetaData from FSDirectory.renameFile
> 
>
> Key: LUCENE-7373
> URL: https://issues.apache.org/jira/browse/LUCENE-7373
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Michael McCandless
>Assignee: Michael McCandless
> Fix For: master (7.0), 6.2
>
> Attachments: LUCENE-7373.patch
>
>
> Today, when you call {{FSDirectory.renameFile}} it also calls fsync on
> the directory.
> This is OK for Lucene's current usage of this method, to rename just
> the one {{segments_N}} file on commit.
> But in playing with adding NRT replication (LUCENE-5438) to the simple
> demo Lucene server (LUCENE-5376) I found that, on spinning disks, that
> fsync is very costly, because when copying over an NRT point, we write
> to N .tmp files and then rename many files (taking seconds) in the
> end.
> I think we should just deprecate/remove the existing method, and make a new
> {{rename}} method that does only renaming, and a separate
> {{syncMetaData}} to call fsync on the directory?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7375) Can we allow FSDirectory subclasses to customize whether the ctor does a mkdir?

[Uwe Schindler (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15370801#comment-15370801
 ] 

Uwe Schindler commented on LUCENE-7375:
---

Maybe allow to pass "read-only" to FSDirectory ctor and FSDirectory.open()? In 
the RO case we can fail early if the directory does not exist. In addition, all 
the write methods fail with UOE.

> Can we allow FSDirectory subclasses to customize whether the ctor does a 
> mkdir?
> ---
>
> Key: LUCENE-7375
> URL: https://issues.apache.org/jira/browse/LUCENE-7375
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Michael McCandless
>
> Today, just instantiating an {{FSDirectory}} always brings the directory into 
> existence, even if you will do no writes ({{createOutput}}).
> We have gone back and forth on this, over the ages.  E.g. see LUCENE-16 (only 
> 2 digits there!!), LUCENE-773 (only 3 digits there!!), LUCENE-1464.  At one 
> point we created the directory lazily, on the first write ({{createOutput}}) 
> attempt, but now we always create it when you instantiate {{FSDirectory}}.
> This causes some hassle for consumers, e.g. in 
> https://github.com/elastic/elasticsearch/pull/19338 ES is forking 
> {{SimpleFSDirectory}} in order to have a read-only directory impl.
> Maybe we can do the {{Files.createDirectories}} in protected method that a 
> subclass could override?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: [POLL] What should happen to PyLucene now?

[Oliver Frietsch]

Hi,

(Sorry, just joined the list and so I'm unable to use a "real reply" in 
my mail client)


As for my company, our answers are:

[X] Still mostly happy with the 3.6 release, planning to upgrade to 4.x 
some time in future (please don't ask when, also requires a lot of 
changes in our product)...


[X] Have developed/updated Python3 Port of PyLucene3.6 and would be 
happy to contribute this


Please note: Two tests are still failing, seem to be some encoding issues.

[X] Please, a new 6.x release (but we can’t contribute and are uncertain 
if we'll upgrade soon)


Our PyLucene python 3 port is currently pushed forward by some 
"almost-retired" (but still having fun to code!) partner of us, so there 
is not too much manpower behind and not much more to expect in near 
future. But if that helps, we're very happy to contribute that existing 
patch.


Also, a huge thank you to all PyLucene committers so far! You did a 
great job!


Oliver

[jira] [Commented] (LUCENE-2899) Add OpenNLP Analysis capabilities as a module

[Steven Bower (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-2899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15370786#comment-15370786
 ] 

Steven Bower commented on LUCENE-2899:
--

[~steve_rowe] per our conversation yesterday.. Would be interesting to store 
the PoS and entity information as stacked tokens vs (or in addition to the) 
payload... such that you could do {{"bob @person"~0}} or {{"house @verb"~0}} 
type queries.. or things like {{"@person @ceo"~10}}

> Add OpenNLP Analysis capabilities as a module
> -
>
> Key: LUCENE-2899
> URL: https://issues.apache.org/jira/browse/LUCENE-2899
> Project: Lucene - Core
>  Issue Type: New Feature
>  Components: modules/analysis
>Reporter: Grant Ingersoll
>Assignee: Grant Ingersoll
>Priority: Minor
> Fix For: 4.9, 6.0
>
> Attachments: LUCENE-2899-RJN.patch, LUCENE-2899.patch, 
> LUCENE-2899.patch, LUCENE-2899.patch, OpenNLPFilter.java, 
> OpenNLPTokenizer.java
>
>
> Now that OpenNLP is an ASF project and has a nice license, it would be nice 
> to have a submodule (under analysis) that exposed capabilities for it. Drew 
> Farris, Tom Morton and I have code that does:
> * Sentence Detection as a Tokenizer (could also be a TokenFilter, although it 
> would have to change slightly to buffer tokens)
> * NamedEntity recognition as a TokenFilter
> We are also planning a Tokenizer/TokenFilter that can put parts of speech as 
> either payloads (PartOfSpeechAttribute?) on a token or at the same position.
> I'd propose it go under:
> modules/analysis/opennlp



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-master-Linux (64bit/jdk1.8.0_92) - Build # 17217 - Still Failing!

[Policeman Jenkins Server]

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/17217/
Java: 64bit/jdk1.8.0_92 -XX:+UseCompressedOops -XX:+UseParallelGC

All tests passed

Build Log:
[...truncated 12229 lines...]
   [junit4] JVM J1: stdout was not empty, see: 
/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/temp/junit4-J1-20160711_130012_973.sysout
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) 
   [junit4] java.lang.OutOfMemoryError: GC overhead limit exceeded
   [junit4] Dumping heap to 
/home/jenkins/workspace/Lucene-Solr-master-Linux/heapdumps/java_pid30892.hprof 
...
   [junit4] Heap dump file created [480482955 bytes in 1.607 secs]
   [junit4] <<< JVM J1: EOF 

   [junit4] JVM J1: stderr was not empty, see: 
/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/temp/junit4-J1-20160711_130012_973.syserr
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) 
   [junit4] 
   [junit4] Exception: java.lang.OutOfMemoryError thrown from the 
UncaughtExceptionHandler in thread 
"OverseerStateUpdate-96222810499645448-127.0.0.1:38693_solr-n_00"
   [junit4] WARN: Unhandled exception in event serialization. -> 
java.lang.OutOfMemoryError: GC overhead limit exceeded
   [junit4] <<< JVM J1: EOF 

[...truncated 340 lines...]
   [junit4] ERROR: JVM J1 ended with an exception, command line: 
/home/jenkins/tools/java/64bit/jdk1.8.0_92/jre/bin/java -XX:+UseCompressedOops 
-XX:+UseParallelGC -XX:+HeapDumpOnOutOfMemoryError 
-XX:HeapDumpPath=/home/jenkins/workspace/Lucene-Solr-master-Linux/heapdumps -ea 
-esa -Dtests.prefix=tests -Dtests.seed=4A33129F16A8BE7F -Xmx512M -Dtests.iters= 
-Dtests.verbose=false -Dtests.infostream=false -Dtests.codec=random 
-Dtests.postingsformat=random -Dtests.docvaluesformat=random 
-Dtests.locale=random -Dtests.timezone=random -Dtests.directory=random 
-Dtests.linedocsfile=europarl.lines.txt.gz -Dtests.luceneMatchVersion=7.0.0 
-Dtests.cleanthreads=perClass 
-Djava.util.logging.config.file=/home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/tools/junit4/logging.properties
 -Dtests.nightly=false -Dtests.weekly=false -Dtests.monster=false 
-Dtests.slow=true -Dtests.asserts=true -Dtests.multiplier=3 -DtempDir=./temp 
-Djava.io.tmpdir=./temp 
-Djunit4.tempDir=/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/temp
 -Dcommon.dir=/home/jenkins/workspace/Lucene-Solr-master-Linux/lucene 
-Dclover.db.dir=/home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/build/clover/db
 
-Djava.security.policy=/home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/tools/junit4/solr-tests.policy
 -Dtests.LUCENE_VERSION=7.0.0 -Djetty.testMode=1 -Djetty.insecurerandom=1 
-Dsolr.directoryFactory=org.apache.solr.core.MockDirectoryFactory 
-Djava.awt.headless=true -Djdk.map.althashing.threshold=0 
-Djunit4.childvm.cwd=/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J1
 -Djunit4.childvm.id=1 -Djunit4.childvm.count=3 -Dtests.leaveTemporary=false 
-Dtests.filterstacks=true -Dtests.disableHdfs=true 
-Djava.security.manager=org.apache.lucene.util.TestSecurityManager -classpath 

[jira] [Commented] (LUCENE-7375) Can we allow FSDirectory subclasses to customize whether the ctor does a mkdir?

[Robert Muir (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15370752#comment-15370752
 ] 

Robert Muir commented on LUCENE-7375:
-

the mkdir is necessary if the Directory will ever be used to perform any 
writes, otherwise locking is broken.

Lucene was broken here forever, because it used getCanonicalPath on a directory 
that may or may not exist: that would then be used by e.g. NativeFSLock as the 
key for the in-memory map and so on.

This was just one of the many ways that locking was broken.

You can see this by reading the javadocs:
https://docs.oracle.com/javase/7/docs/api/java/io/File.html#getCanonicalPath()

{quote}
 Every pathname that denotes an existing file or directory has a unique 
canonical form. Every pathname that denotes a nonexistent file or directory 
also has a unique canonical form. The canonical form of the pathname of a 
nonexistent file or directory may be different from the canonical form of the 
same pathname after the file or directory is created. Similarly, the canonical 
form of the pathname of an existing file or directory may be different from the 
canonical form of the same pathname after the file or directory is deleted.
{quote}

So, in order for locking to work, its critical that we create the file.

> Can we allow FSDirectory subclasses to customize whether the ctor does a 
> mkdir?
> ---
>
> Key: LUCENE-7375
> URL: https://issues.apache.org/jira/browse/LUCENE-7375
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Michael McCandless
>
> Today, just instantiating an {{FSDirectory}} always brings the directory into 
> existence, even if you will do no writes ({{createOutput}}).
> We have gone back and forth on this, over the ages.  E.g. see LUCENE-16 (only 
> 2 digits there!!), LUCENE-773 (only 3 digits there!!), LUCENE-1464.  At one 
> point we created the directory lazily, on the first write ({{createOutput}}) 
> attempt, but now we always create it when you instantiate {{FSDirectory}}.
> This causes some hassle for consumers, e.g. in 
> https://github.com/elastic/elasticsearch/pull/19338 ES is forking 
> {{SimpleFSDirectory}} in order to have a read-only directory impl.
> Maybe we can do the {{Files.createDirectories}} in protected method that a 
> subclass could override?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7373) Break out Directory.syncMetaData from FSDirectory.renameFile

[Robert Muir (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15370740#comment-15370740
 ] 

Robert Muir commented on LUCENE-7373:
-

+1

> Break out Directory.syncMetaData from FSDirectory.renameFile
> 
>
> Key: LUCENE-7373
> URL: https://issues.apache.org/jira/browse/LUCENE-7373
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Michael McCandless
>Assignee: Michael McCandless
> Fix For: master (7.0), 6.2
>
> Attachments: LUCENE-7373.patch
>
>
> Today, when you call {{FSDirectory.renameFile}} it also calls fsync on
> the directory.
> This is OK for Lucene's current usage of this method, to rename just
> the one {{segments_N}} file on commit.
> But in playing with adding NRT replication (LUCENE-5438) to the simple
> demo Lucene server (LUCENE-5376) I found that, on spinning disks, that
> fsync is very costly, because when copying over an NRT point, we write
> to N .tmp files and then rename many files (taking seconds) in the
> end.
> I think we should just deprecate/remove the existing method, and make a new
> {{rename}} method that does only renaming, and a separate
> {{syncMetaData}} to call fsync on the directory?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (LUCENE-7375) Can we allow FSDirectory subclasses to customize whether the ctor does a mkdir?

[Michael McCandless (JIRA)]

Michael McCandless created LUCENE-7375:
--

 Summary: Can we allow FSDirectory subclasses to customize whether 
the ctor does a mkdir?
 Key: LUCENE-7375
 URL: https://issues.apache.org/jira/browse/LUCENE-7375
 Project: Lucene - Core
  Issue Type: Improvement
Reporter: Michael McCandless


Today, just instantiating an {{FSDirectory}} always brings the directory into 
existence, even if you will do no writes ({{createOutput}}).

We have gone back and forth on this, over the ages.  E.g. see LUCENE-16 (only 2 
digits there!!), LUCENE-773 (only 3 digits there!!), LUCENE-1464.  At one point 
we created the directory lazily, on the first write ({{createOutput}}) attempt, 
but now we always create it when you instantiate {{FSDirectory}}.

This causes some hassle for consumers, e.g. in 
https://github.com/elastic/elasticsearch/pull/19338 ES is forking 
{{SimpleFSDirectory}} in order to have a read-only directory impl.

Maybe we can do the {{Files.createDirectories}} in protected method that a 
subclass could override?




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-9284) The HDFS BlockDirectoryCache should not let it's keysToRelease or names maps grow indefinitely.

[Mark Miller (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-9284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mark Miller updated SOLR-9284:
--
Attachment: SOLR-9284.patch

Just a quick patch, needs a bit of review and polish, but this is along the 
lines I was thinking of for a fix.

Not super satisfied with the 'names' issue, but not sure what we should do at 
the moment. We might want a higher upper limit and/or to make it configurable. 
Or perhaps there should be some kind of cleaner thread that prunes occasionally?

> The HDFS BlockDirectoryCache should not let it's keysToRelease or names maps 
> grow indefinitely.
> ---
>
> Key: SOLR-9284
> URL: https://issues.apache.org/jira/browse/SOLR-9284
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: hdfs
>Reporter: Mark Miller
>Assignee: Mark Miller
> Fix For: 6.2, master (7.0)
>
> Attachments: SOLR-9284.patch
>
>
> https://issues.apache.org/jira/browse/SOLR-9284



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-9267) Cloud MLT field boost not working

[Brian Feldman (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-9267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brian Feldman updated SOLR-9267:

Affects Version/s: 5.5.3
   6.2
   6.1.1
   6.0.2
   5.6

> Cloud MLT field boost not working
> -
>
> Key: SOLR-9267
> URL: https://issues.apache.org/jira/browse/SOLR-9267
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: MoreLikeThis
>Affects Versions: 5.0, 5.1, 5.2, 5.2.1, 5.3, 5.3.1, 5.3.2, 5.4, 5.4.1, 
> 5.5, 5.5.1, 5.5.2, 5.6, 6.0, 6.0.1, 6.0.2, 6.1, 6.1.1, 6.2, 5.5.3
>Reporter: Brian Feldman
>
> When boosting by field "fieldname otherFieldName^4.0" the boost is not 
> stripped from the field name when adding to fieldNames ArrayList.  So on line 
> 133 of CloudMLTQParser when adding field content to the filteredDocument the 
> field is not found (incorrectly trying to find 'otherFieldName^4.0').
> The easiest but perhaps hackiest solution is to overwrite qf:
> {code}
> if (localParams.get("boost") != null) {
>   mlt.setBoost(localParams.getBool("boost"));
>   boostFields = SolrPluginUtils.parseFieldBoosts(qf);
>   qf = boostFields.keySet().toArray(qf);
> }
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Reopened] (SOLR-9242) Collection level backup/restore should provide a param for specifying the repository implementation it should use

[Varun Thacker (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-9242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Varun Thacker reopened SOLR-9242:
-

> Collection level backup/restore should provide a param for specifying the 
> repository implementation it should use
> -
>
> Key: SOLR-9242
> URL: https://issues.apache.org/jira/browse/SOLR-9242
> Project: Solr
>  Issue Type: Improvement
>Reporter: Hrishikesh Gadre
>Assignee: Varun Thacker
> Fix For: 6.2
>
> Attachments: SOLR-9242.patch, SOLR-9242.patch, SOLR-9242.patch, 
> SOLR-9242.patch, SOLR-9242.patch, SOLR-9242_followup.patch
>
>
> SOLR-7374 provides BackupRepository interface to enable storing Solr index 
> data to a configured file-system (e.g. HDFS, local file-system etc.). This 
> JIRA is to track the work required to extend this functionality at the 
> collection level.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-9242) Collection level backup/restore should provide a param for specifying the repository implementation it should use

[Varun Thacker (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-9242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Varun Thacker updated SOLR-9242:

Attachment: SOLR-9242_followup.patch

- Moved ASF source headers at the very top
- ForceUpdate cluster props to make sure we are reading the latest value. This 
tackles Jenkins failures like  
https://builds.apache.org/job/Lucene-Solr-Tests-master/1270/

I'm looking into the windows path failure as well 

> Collection level backup/restore should provide a param for specifying the 
> repository implementation it should use
> -
>
> Key: SOLR-9242
> URL: https://issues.apache.org/jira/browse/SOLR-9242
> Project: Solr
>  Issue Type: Improvement
>Reporter: Hrishikesh Gadre
>Assignee: Varun Thacker
> Fix For: 6.2
>
> Attachments: SOLR-9242.patch, SOLR-9242.patch, SOLR-9242.patch, 
> SOLR-9242.patch, SOLR-9242.patch, SOLR-9242_followup.patch
>
>
> SOLR-7374 provides BackupRepository interface to enable storing Solr index 
> data to a configured file-system (e.g. HDFS, local file-system etc.). This 
> JIRA is to track the work required to extend this functionality at the 
> collection level.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (LUCENE-7374) Solrj client should support and parse hierarchical clusters

[Dawid Weiss (JIRA)]

Dawid Weiss created LUCENE-7374:
---

 Summary: Solrj client should support and parse hierarchical 
clusters
 Key: LUCENE-7374
 URL: https://issues.apache.org/jira/browse/LUCENE-7374
 Project: Lucene - Core
  Issue Type: Improvement
Reporter: Dawid Weiss
Assignee: Dawid Weiss
Priority: Trivial
 Fix For: 6.x, master (7.0)






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Moved] (SOLR-9293) Solrj client should support and parse hierarchical clusters

[Dawid Weiss (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-9293?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dawid Weiss moved LUCENE-7374 to SOLR-9293:
---

Fix Version/s: (was: master (7.0))
   (was: 6.x)
   master (7.0)
 Security: Public
Lucene Fields:   (was: New)
  Key: SOLR-9293  (was: LUCENE-7374)
  Project: Solr  (was: Lucene - Core)

> Solrj client should support and parse hierarchical clusters
> ---
>
> Key: SOLR-9293
> URL: https://issues.apache.org/jira/browse/SOLR-9293
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Dawid Weiss
>Assignee: Dawid Weiss
>Priority: Trivial
> Fix For: master (7.0)
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-master-Linux (32bit/jdk1.8.0_92) - Build # 17216 - Failure!

[Policeman Jenkins Server]

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/17216/
Java: 32bit/jdk1.8.0_92 -client -XX:+UseSerialGC

1 tests failed.
FAILED:  
org.apache.solr.handler.TestReplicationHandler.doTestIndexAndConfigAliasReplication

Error Message:
expected:<1> but was:<0>

Stack Trace:
java.lang.AssertionError: expected:<1> but was:<0>
at 
__randomizedtesting.SeedInfo.seed([8C0E53D76E09E42E:7B7DBD8FA8E14BC8]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.junit.Assert.assertEquals(Assert.java:456)
at 
org.apache.solr.handler.TestReplicationHandler.doTestIndexAndConfigAliasReplication(TestReplicationHandler.java:1327)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 11952 lines...]
   [junit4] Suite: 

[JENKINS] Lucene-Solr-master-MacOSX (64bit/jdk1.8.0) - Build # 3404 - Still Failing!

[Policeman Jenkins Server]

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-master-MacOSX/3404/
Java: 64bit/jdk1.8.0 -XX:+UseCompressedOops -XX:+UseSerialGC

2 tests failed.
FAILED:  org.apache.solr.cloud.LeaderFailoverAfterPartitionTest.test

Error Message:
Expected 2 of 3 replicas to be active but only found 1; 
[core_node3:{"core":"c8n_1x3_lf_shard1_replica2","base_url":"http://127.0.0.1:62255","node_name":"127.0.0.1:62255_","state":"active","leader":"true"}];
 clusterState: DocCollection(c8n_1x3_lf//clusterstate.json/33)={   
"replicationFactor":"3",   "shards":{"shard1":{   
"range":"8000-7fff",   "state":"active",   "replicas":{ 
"core_node1":{   "state":"down",   
"base_url":"http://127.0.0.1:62239;,   
"core":"c8n_1x3_lf_shard1_replica1",   "node_name":"127.0.0.1:62239_"}, 
"core_node2":{   "core":"c8n_1x3_lf_shard1_replica3",   
"base_url":"http://127.0.0.1:62245;,   "node_name":"127.0.0.1:62245_",  
 "state":"down"}, "core_node3":{   
"core":"c8n_1x3_lf_shard1_replica2",   
"base_url":"http://127.0.0.1:62255;,   "node_name":"127.0.0.1:62255_",  
 "state":"active",   "leader":"true",   
"router":{"name":"compositeId"},   "maxShardsPerNode":"1",   
"autoAddReplicas":"false"}

Stack Trace:
java.lang.AssertionError: Expected 2 of 3 replicas to be active but only found 
1; 
[core_node3:{"core":"c8n_1x3_lf_shard1_replica2","base_url":"http://127.0.0.1:62255","node_name":"127.0.0.1:62255_","state":"active","leader":"true"}];
 clusterState: DocCollection(c8n_1x3_lf//clusterstate.json/33)={
  "replicationFactor":"3",
  "shards":{"shard1":{
  "range":"8000-7fff",
  "state":"active",
  "replicas":{
"core_node1":{
  "state":"down",
  "base_url":"http://127.0.0.1:62239;,
  "core":"c8n_1x3_lf_shard1_replica1",
  "node_name":"127.0.0.1:62239_"},
"core_node2":{
  "core":"c8n_1x3_lf_shard1_replica3",
  "base_url":"http://127.0.0.1:62245;,
  "node_name":"127.0.0.1:62245_",
  "state":"down"},
"core_node3":{
  "core":"c8n_1x3_lf_shard1_replica2",
  "base_url":"http://127.0.0.1:62255;,
  "node_name":"127.0.0.1:62255_",
  "state":"active",
  "leader":"true",
  "router":{"name":"compositeId"},
  "maxShardsPerNode":"1",
  "autoAddReplicas":"false"}
at 
__randomizedtesting.SeedInfo.seed([437A5AFCDA8A6D57:CB2E6526747600AF]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at 
org.apache.solr.cloud.LeaderFailoverAfterPartitionTest.testRf3WithLeaderFailover(LeaderFailoverAfterPartitionTest.java:170)
at 
org.apache.solr.cloud.LeaderFailoverAfterPartitionTest.test(LeaderFailoverAfterPartitionTest.java:57)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:985)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:960)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
at 

[jira] [Closed] (SOLR-9292) Suggester not working for more than 200K records

[govind (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-9292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

govind closed SOLR-9292.

Resolution: Fixed

After 200K records, one of the records was messed up with the schema.
isn't a bug.

> Suggester not working for more than 200K records
> 
>
> Key: SOLR-9292
> URL: https://issues.apache.org/jira/browse/SOLR-9292
> Project: Solr
>  Issue Type: Test
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Suggester
>Affects Versions: 5.5
> Environment: ubuntu 14.04, open jdk 1.8 
>Reporter: govind
>
> Using lookup suggester and fuzzy lookup suggester in combination. Data 
> uploaded to solr through json. Up to 200K records, both suggesters are 
> working fine, but after words they are not working.
> Pushing Data as: 
> curl -X POST -H 'Content-Type: application/json' 
> 'http://localhost:8983/solr/demo_autocomplete/update?versions=false' 
> --data-binary @data.json
> Reload as: curl 
> "http://localhost:8983/solr/admin/cores?wt=json=RELOAD=demo_autocomplete;
> and after that building and testing the suggester as:
> url 
> "http://localhost:8983/solr/demo_autocomplete/suggest?suggest.dictionary=fuzzySuggester=true=true=test=json;
> When I insert more than 200K records, it gives error as:
> at 
> org.apache.lucene.util.automaton.Operations.topoSortStatesRecurse(Operations.java:1311)
> ---
> Is there any config I am missing or Do I need to modify some default 
> limitations ?
> Regards,
> Govind



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-6.x-Windows (64bit/jdk1.8.0_92) - Build # 313 - Still Failing!

[Policeman Jenkins Server]

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-6.x-Windows/313/
Java: 64bit/jdk1.8.0_92 -XX:+UseCompressedOops -XX:+UseParallelGC

2 tests failed.
FAILED:  org.apache.solr.core.TestDynamicLoading.testDynamicLoading

Error Message:
Could not get expected value  'X val' for path 'x' full output: {   
"responseHeader":{ "status":0, "QTime":0},   "params":{"wt":"json"},   
"context":{ "webapp":"", "path":"/test1", "httpMethod":"GET"},   
"class":"org.apache.solr.core.BlobStoreTestRequestHandler",   "x":null},  from 
server:  null

Stack Trace:
java.lang.AssertionError: Could not get expected value  'X val' for path 'x' 
full output: {
  "responseHeader":{
"status":0,
"QTime":0},
  "params":{"wt":"json"},
  "context":{
"webapp":"",
"path":"/test1",
"httpMethod":"GET"},
  "class":"org.apache.solr.core.BlobStoreTestRequestHandler",
  "x":null},  from server:  null
at 
__randomizedtesting.SeedInfo.seed([EE6E6C5D3EAC9E54:3623410AC9713BF4]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at 
org.apache.solr.core.TestSolrConfigHandler.testForResponseElement(TestSolrConfigHandler.java:481)
at 
org.apache.solr.core.TestDynamicLoading.testDynamicLoading(TestDynamicLoading.java:232)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 

[JENKINS] Lucene-Solr-6.x-Linux (64bit/jdk1.8.0_92) - Build # 1120 - Still Failing!

[Policeman Jenkins Server]

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-6.x-Linux/1120/
Java: 64bit/jdk1.8.0_92 -XX:-UseCompressedOops -XX:+UseConcMarkSweepGC

1 tests failed.
FAILED:  
org.apache.lucene.queries.mlt.TestMoreLikeThis.testMultiFieldShouldReturnPerFieldBooleanQuery

Error Message:


Stack Trace:
java.lang.AssertionError
at 
__randomizedtesting.SeedInfo.seed([5CBE08C63EECE7F4:380D50008302610F]:0)
at org.junit.Assert.fail(Assert.java:92)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertTrue(Assert.java:54)
at 
org.apache.lucene.queries.mlt.TestMoreLikeThis.testMultiFieldShouldReturnPerFieldBooleanQuery(TestMoreLikeThis.java:319)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 1617 lines...]
   [junit4] JVM J2: stdout was not empty, see: 
/home/jenkins/workspace/Lucene-Solr-6.x-Linux/lucene/build/core/test/temp/junit4-J2-20160711_093804_071.sysout
   [junit4] >>> JVM J2 emitted unexpected output (verbatim) 
   [junit4] Default case invoked for: 
   [junit4]opcode  = 0, "Node"
   [junit4] Default case invoked for: 
   [junit4]opcode  = 0, "Node"
   [junit4] Default case invoked for: 
   [junit4]opcode  = 200, "Phi"
   [junit4] <<< JVM J2: EOF 

[...truncated 6146 lines...]
   [junit4] Suite: 

[jira] [Commented] (LUCENE-2899) Add OpenNLP Analysis capabilities as a module

[Ramesh Kumar Thirumalaisamy (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-2899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15370477#comment-15370477
 ] 

Ramesh Kumar Thirumalaisamy commented on LUCENE-2899:
-

Glad to see some update on this page. Hope this code gets committed soon.
I am trying to use this patch to do some information extraction using SOLR and 
Open NLP.
Can this patch be now used with latest version of SOLR .


> Add OpenNLP Analysis capabilities as a module
> -
>
> Key: LUCENE-2899
> URL: https://issues.apache.org/jira/browse/LUCENE-2899
> Project: Lucene - Core
>  Issue Type: New Feature
>  Components: modules/analysis
>Reporter: Grant Ingersoll
>Assignee: Grant Ingersoll
>Priority: Minor
> Fix For: 4.9, 6.0
>
> Attachments: LUCENE-2899-RJN.patch, LUCENE-2899.patch, 
> LUCENE-2899.patch, LUCENE-2899.patch, OpenNLPFilter.java, 
> OpenNLPTokenizer.java
>
>
> Now that OpenNLP is an ASF project and has a nice license, it would be nice 
> to have a submodule (under analysis) that exposed capabilities for it. Drew 
> Farris, Tom Morton and I have code that does:
> * Sentence Detection as a Tokenizer (could also be a TokenFilter, although it 
> would have to change slightly to buffer tokens)
> * NamedEntity recognition as a TokenFilter
> We are also planning a Tokenizer/TokenFilter that can put parts of speech as 
> either payloads (PartOfSpeechAttribute?) on a token or at the same position.
> I'd propose it go under:
> modules/analysis/opennlp



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-9292) Suggester not working for more than 200K records

[govind (JIRA)]

govind created SOLR-9292:


 Summary: Suggester not working for more than 200K records
 Key: SOLR-9292
 URL: https://issues.apache.org/jira/browse/SOLR-9292
 Project: Solr
  Issue Type: Test
  Security Level: Public (Default Security Level. Issues are Public)
  Components: Suggester
Affects Versions: 5.5
 Environment: ubuntu 14.04, open jdk 1.8 
Reporter: govind


Using lookup suggester and fuzzy lookup suggester in combination. Data uploaded 
to solr through json. Up to 200K records, both suggesters are working fine, but 
after words they are not working.

Pushing Data as: 
curl -X POST -H 'Content-Type: application/json' 
'http://localhost:8983/solr/demo_autocomplete/update?versions=false' 
--data-binary @data.json

Reload as: curl 
"http://localhost:8983/solr/admin/cores?wt=json=RELOAD=demo_autocomplete;

and after that building and testing the suggester as:
url 
"http://localhost:8983/solr/demo_autocomplete/suggest?suggest.dictionary=fuzzySuggester=true=true=test=json;


When I insert more than 200K records, it gives error as:
at 
org.apache.lucene.util.automaton.Operations.topoSortStatesRecurse(Operations.java:1311)
---


Is there any config I am missing or Do I need to modify some default 
limitations ?

Regards,
Govind






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9200) Add Delegation Token Support to Solr

[Ishan Chattopadhyaya (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15370426#comment-15370426
 ] 

Ishan Chattopadhyaya commented on SOLR-9200:


Hi Greg. I'm reviewing the patch and shall post my comments soon. :-) However, 
I just thought I'll share my thoughts on the following:

bq. 3) AuthenticationPlugin has an incompatible change that means this should 
only go into the next major version.
Since AuthenticationPlugin is marked @lucene.experimental, I think we should be 
good to go with the change within 6x. Afair, we've done such incompatible 
changes in the past within 5x (when we were working on the BasicAuth 
functionality. Sorry, I don't have the exact JIRAs handy at the moment). Any 
3rd party authentication plugins would suffer, but it should be immediately 
obvious to the developers how to fix it.

> Add Delegation Token Support to Solr
> 
>
> Key: SOLR-9200
> URL: https://issues.apache.org/jira/browse/SOLR-9200
> Project: Solr
>  Issue Type: New Feature
>  Components: security
>Reporter: Gregory Chanan
>Assignee: Gregory Chanan
> Attachments: SOLR-9200.patch, SOLR-9200.patch
>
>
> SOLR-7468 added support for kerberos authentication via the hadoop 
> authentication filter.  Hadoop also has support for an authentication filter 
> that supports delegation tokens, which allow authenticated users the ability 
> to grab/renew/delete a token that can be used to bypass the normal 
> authentication path for a time.  This is useful in a variety of use cases:
> 1) distributed clients (e.g. MapReduce) where each client may not have access 
> to the user's kerberos credentials.  Instead, the job runner can grab a 
> delegation token and use that during task execution.
> 2) If the load on the kerberos server is too high, delegation tokens can 
> avoid hitting the kerberos server after the first request
> 3) If requests/permissions need to be delegated to another user: the more 
> privileged user can request a delegation token that can be passed to the less 
> privileged user.
> Note to self:
> In 
> https://issues.apache.org/jira/browse/SOLR-7468?focusedCommentId=14579636=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14579636
>  I made the following comment which I need to investigate further, since I 
> don't know if anything changed in this area:
> {quote}3) I'm a little concerned with the "NoContext" code in KerberosPlugin 
> moving forward (I understand this is more a generic auth question than 
> kerberos specific). For example, in the latest version of the filter we are 
> using at Cloudera, we play around with the ServletContext in order to pass 
> information around 
> (https://github.com/cloudera/lucene-solr/blob/cdh5-4.10.3_5.4.2/solr/core/src/java/org/apache/solr/servlet/SolrHadoopAuthenticationFilter.java#L106).
>  Is there any way we can get the actual ServletContext in a plugin?{quote}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-master-Windows (64bit/jdk1.8.0_92) - Build # 5976 - Still Failing!

[Policeman Jenkins Server]

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-master-Windows/5976/
Java: 64bit/jdk1.8.0_92 -XX:+UseCompressedOops -XX:+UseParallelGC

3 tests failed.
FAILED:  
org.apache.solr.handler.TestReplicationHandler.doTestIndexAndConfigAliasReplication

Error Message:
expected:<1> but was:<0>

Stack Trace:
java.lang.AssertionError: expected:<1> but was:<0>
at 
__randomizedtesting.SeedInfo.seed([BFEB3AEC36B689AA:4898D4B4F05E264C]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.junit.Assert.assertEquals(Assert.java:456)
at 
org.apache.solr.handler.TestReplicationHandler.doTestIndexAndConfigAliasReplication(TestReplicationHandler.java:1327)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.security.BasicAuthIntegrationTest.testBasics

[jira] [Commented] (SOLR-9106) Cache cluster properties in ZkStateReader

[Noble Paul (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15370348#comment-15370348
 ] 

Noble Paul commented on SOLR-9106:
--

Hi [~romseygeek] I missed this. Not watching clusterprops was done 
deliberately. it is read rarely and watching is expensive.

> Cache cluster properties in ZkStateReader
> -
>
> Key: SOLR-9106
> URL: https://issues.apache.org/jira/browse/SOLR-9106
> Project: Solr
>  Issue Type: Improvement
>Affects Versions: master (7.0)
>Reporter: Alan Woodward
>Assignee: Alan Woodward
> Fix For: 6.1
>
> Attachments: SOLR-9106.patch, SOLR-9106.patch
>
>
> ZkStateReader currently makes calls into ZK every time getClusterProps() is 
> called.  Instead we should keep the data locally and use a Watcher to update 
> it.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-6.x-Linux (32bit/jdk1.8.0_92) - Build # 1119 - Failure!

[Policeman Jenkins Server]

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-6.x-Linux/1119/
Java: 32bit/jdk1.8.0_92 -server -XX:+UseParallelGC

1 tests failed.
FAILED:  org.apache.solr.search.TestStressLucene.testStressLuceneNRT

Error Message:
MockDirectoryWrapper: cannot close: there are still open files: {_1.fdx=1, 
_1.fdt=1}

Stack Trace:
java.lang.RuntimeException: MockDirectoryWrapper: cannot close: there are still 
open files: {_1.fdx=1, _1.fdt=1}
at 
org.apache.lucene.store.MockDirectoryWrapper.close(MockDirectoryWrapper.java:827)
at 
org.apache.solr.search.TestStressLucene.testStressLuceneNRT(TestStressLucene.java:370)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException: unclosed IndexOutput: _1.fdx
at 
org.apache.lucene.store.MockDirectoryWrapper.addFileHandle(MockDirectoryWrapper.java:718)
at 
org.apache.lucene.store.MockDirectoryWrapper.createOutput(MockDirectoryWrapper.java:653)
at 

[JENKINS] Lucene-Solr-master-Solaris (64bit/jdk1.8.0) - Build # 710 - Still Failing!

[Policeman Jenkins Server]

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-master-Solaris/710/
Java: 64bit/jdk1.8.0 -XX:+UseCompressedOops -XX:+UseConcMarkSweepGC

2 tests failed.
FAILED:  org.apache.solr.cloud.LeaderFailoverAfterPartitionTest.test

Error Message:
org.apache.solr.client.solrj.SolrServerException: No live SolrServers available 
to handle this request:[http://127.0.0.1:42170/c8n_1x3_lf_shard1_replica2]

Stack Trace:
org.apache.solr.client.solrj.SolrServerException: 
org.apache.solr.client.solrj.SolrServerException: No live SolrServers available 
to handle this request:[http://127.0.0.1:42170/c8n_1x3_lf_shard1_replica2]
at 
__randomizedtesting.SeedInfo.seed([B657F228A8FFA627:3E03CDF20603CBDF]:0)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.directUpdate(CloudSolrClient.java:753)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1151)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:1040)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:976)
at 
org.apache.solr.cloud.AbstractFullDistribZkTestBase.sendDocsWithRetry(AbstractFullDistribZkTestBase.java:753)
at 
org.apache.solr.cloud.AbstractFullDistribZkTestBase.sendDocsWithRetry(AbstractFullDistribZkTestBase.java:741)
at 
org.apache.solr.cloud.LeaderFailoverAfterPartitionTest.testRf3WithLeaderFailover(LeaderFailoverAfterPartitionTest.java:178)
at 
org.apache.solr.cloud.LeaderFailoverAfterPartitionTest.test(LeaderFailoverAfterPartitionTest.java:57)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:985)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:960)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 

[jira] [Commented] (SOLR-9242) Collection level backup/restore should provide a param for specifying the repository implementation it should use

[Varun Thacker (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15370234#comment-15370234
 ] 

Varun Thacker commented on SOLR-9242:
-

There's another type of test failure : 
https://builds.apache.org/job/Lucene-Solr-Tests-master/1270/console

In this case the ZK watch for updating the clusterprop occurs after the backup 
command has been invoked . Hence it doesn't find the value set in the 
cluster-prop api

{code}
[junit4]   2> 819240 INFO  (qtp102693249-7831) [n:127.0.0.1:52710_solr] 
o.a.s.h.a.CollectionsHandler Invoked Collection Action :clusterprop with params 
val=/location/does/not/exist=location=CLUSTERPROP=javabin=2
 and sendToOCPQueue=true
   [junit4]   2> 819241 INFO  (qtp102693249-7831) [n:127.0.0.1:52710_solr] 
o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections 
params={val=/location/does/not/exist=location=CLUSTERPROP=javabin=2}
 status=0 QTime=0
   [junit4]   2> 819242 INFO  (qtp102693249-7825) [n:127.0.0.1:52710_solr] 
o.a.s.h.a.CollectionsHandler Invoked Collection Action :backup with params 
name=invalidbackuprequest=BACKUP=backuprestore=javabin=2
 and sendToOCPQueue=true
   [junit4]   2> 819242 INFO  (zkCallback-1166-thread-1) [] 
o.a.s.c.c.ZkStateReader Loaded cluster properties: 
{location=/location/does/not/exist}
   [junit4]   2> 819242 INFO  
(zkCallback-1160-thread-2-processing-n:127.0.0.1:41141_solr) 
[n:127.0.0.1:41141_solr] o.a.s.c.c.ZkStateReader Loaded cluster properties: 
{location=/location/does/not/exist}
   [junit4]   2> 819242 ERROR (qtp102693249-7825) [n:127.0.0.1:52710_solr] 
o.a.s.h.RequestHandlerBase org.apache.solr.common.SolrException: 'location' is 
not specified as a query parameter or as a default repository property or as a 
cluster property.
   [junit4]   2>at 
org.apache.solr.handler.admin.CollectionsHandler$CollectionOperation$28.call(CollectionsHandler.java:822)
   [junit4]   2>at 
org.apache.solr.handler.admin.CollectionsHandler.handleRequestBody(CollectionsHandler.java:207)
   [junit4]   2>at 
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:154)
   [junit4]   2>at 
org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:659)
   [junit4]   2>at 
org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:441)
   [junit4]   2>at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:257)
   [junit4]   2>at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:208)
   [junit4]   2>at 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1676)
   [junit4]   2>at 
org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:111)
   [junit4]   2>at 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1676)
   [junit4]   2>at 
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:581)
   [junit4]   2>at 
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
   [junit4]   2>at 
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1160)
   [junit4]   2>at 
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:511)
   [junit4]   2>at 
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
   [junit4]   2>at 
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1092)
   [junit4]   2>at 
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
   [junit4]   2>at 
org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:462)
   [junit4]   2>at 
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
   [junit4]   2>at 
org.eclipse.jetty.server.Server.handle(Server.java:518)
   [junit4]   2>at 
org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:308)
   [junit4]   2>at 
org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:244)
   [junit4]   2>at 
org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
   [junit4]   2>at 
org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
   [junit4]   2>at 
org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
   [junit4]   2>at 
org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceAndRun(ExecuteProduceConsume.java:246)
   [junit4]   2>at 
org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:156)
   [junit4]   2>at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:654)
   [junit4]   2>at