Newbie: Interested in contributing to Lucene source code

[Rajath Agasthya]

Hi,

My name is Rajath Agasthya. I am a software engineer at Cisco. I would like
to contribute to Lucene's source code. I am aware of the newdev issues on
JIRA. But any other suggestions on how to get started is appreciated.

Thanks,
Rajath

[JENKINS] Lucene-Solr-trunk-MacOSX (64bit/jdk1.8.0) - Build # 1994 - Failure!

[Policeman Jenkins Server]

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-MacOSX/1994/
Java: 64bit/jdk1.8.0 -XX:+UseCompressedOops -XX:+UseSerialGC

86 tests failed.
FAILED:  org.apache.solr.cloud.AsyncMigrateRouteKeyTest.test

Error Message:
Could not get the port for ZooKeeper server

Stack Trace:
java.lang.RuntimeException: Could not get the port for ZooKeeper server
at org.apache.solr.cloud.ZkTestServer.run(ZkTestServer.java:482)
at 
org.apache.solr.cloud.AbstractDistribZkTestBase.distribSetUp(AbstractDistribZkTestBase.java:62)
at 
org.apache.solr.cloud.AbstractFullDistribZkTestBase.distribSetUp(AbstractFullDistribZkTestBase.java:198)
at 
org.apache.solr.cloud.BasicDistributedZkTest.distribSetUp(BasicDistributedZkTest.java:112)
at 
org.apache.solr.cloud.MigrateRouteKeyTest.distribSetUp(MigrateRouteKeyTest.java:55)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:910)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.cloud.BasicDistributedZk2Test.test

Error Message:
Could not get the port for ZooKeeper server

Stack Trace:
java.lang.RuntimeException: Could not get the port for ZooKeeper server
at org.apache.solr.cloud.ZkTestServer.run(ZkTestServer.java:482)
at 
org.apache.solr.cloud.AbstractDistribZkTestBase.distribSetUp(AbstractDistribZkTestBase.java:62)
at 
org.apache.solr.cloud.AbstractFullDistribZkTestBase.distribSetUp(AbstractFullDistribZkTestBase.

[JENKINS] Lucene-Solr-Tests-5.x-Java7 - Build # 2636 - Still Failing

[Apache Jenkins Server]

Build: https://builds.apache.org/job/Lucene-Solr-Tests-5.x-Java7/2636/

5 tests failed.
FAILED:  org.apache.solr.cloud.HttpPartitionTest.test

Error Message:
org.apache.http.NoHttpResponseException: The target server failed to respond

Stack Trace:
org.apache.solr.client.solrj.SolrServerException: 
org.apache.http.NoHttpResponseException: The target server failed to respond
at 
__randomizedtesting.SeedInfo.seed([2AFCF9B335AC8633:A2A8C6699B50EBCB]:0)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:865)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:730)
at 
org.apache.solr.cloud.HttpPartitionTest.doSendDoc(HttpPartitionTest.java:484)
at 
org.apache.solr.cloud.HttpPartitionTest.sendDoc(HttpPartitionTest.java:501)
at 
org.apache.solr.cloud.HttpPartitionTest.testRf2(HttpPartitionTest.java:193)
at 
org.apache.solr.cloud.HttpPartitionTest.test(HttpPartitionTest.java:106)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:940)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:915)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
at 
org.apache.lucene.ut

[JENKINS] Lucene-Solr-5.x-Linux (64bit/jdk1.8.0_40-ea-b22) - Build # 11628 - Failure!

[Policeman Jenkins Server]

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-5.x-Linux/11628/
Java: 64bit/jdk1.8.0_40-ea-b22 -XX:-UseCompressedOops -XX:+UseSerialGC

1 tests failed.
FAILED:  org.apache.solr.cloud.DeleteInactiveReplicaTest.deleteLiveReplicaTest

Error Message:
Should have had a good message here

Stack Trace:
java.lang.AssertionError: Should have had a good message here
at 
__randomizedtesting.SeedInfo.seed([E27AFA625D3D168A:4F1A4E694002BEFF]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at 
org.apache.solr.cloud.DeleteReplicaTest.deleteLiveReplicaTest(DeleteReplicaTest.java:125)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:940)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:915)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(Statement

[jira] [Comment Edited] (SOLR-5722) Add catenateShingles option to WordDelimiterFilter

[Greg Pendlebury (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-5722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13903781#comment-13903781
 ] 

Greg Pendlebury edited comment on SOLR-5722 at 2/13/15 3:23 AM:


The link to the doco is working for me today so I took a quick look. I think 
the other reason that the HyphenatedWordsFilter is not suitable is that it 
removes the hyphen from the material assuming that it can only have one 
meaning. The specific circumstances I am considering is when the hyphen is part 
of a legitimately hyphenated word that just happen to break across a line wrap. 
eg. 'up-\{\n\}to-date'

The HyphenatedWordsFilter would turn this into 'upto-date', and cause user 
searches of 'up to date' to not match, since no filters later in the chain can 
really pull 'upto' apart again. Whereas the 'catenateShingles' option is 
intended to preserve the word delimiter and provide all the permutations a user 
might type to find that term: "up to date", "upto date", "up todate", "uptodate"


was (Author: gpendleb):
The link to the doco is working for me today so I took a quick look. I think 
the other reason that the HyphenatedWordsFilter is not suitable is that it 
removes the hyphen from the material assuming that it can only have one 
meaning. The specific circumstances I am considering is when the hyphen is part 
of a legitimately hyphenated word that just happen to break across a line wrap. 
eg. 'up-\{\n\}to-date'

The HyphenatedWordsFilter would turn this into 'upto-date', and cause user 
searches of 'up to date' to not match, since no filters later in the change can 
really pull 'upto' apart again. Whereas the 'catenateShingles' option is 
intended to preserve the word delimiter and provide all the permutations a user 
might type to find that term: "up to date", "upto date", "up todate", "uptodate"

> Add catenateShingles option to WordDelimiterFilter
> --
>
> Key: SOLR-5722
> URL: https://issues.apache.org/jira/browse/SOLR-5722
> Project: Solr
>  Issue Type: Improvement
>Reporter: Greg Pendlebury
>Priority: Minor
>  Labels: filter, newbie, patch
> Attachments: WDFconcatShingles.patch
>
>
> Apologies if I put this in the wrong spot. I'm attaching a patch (against 
> current trunk) that adds support for a 'catenateShingles' option to the 
> WordDelimiterFilter. 
> We (National Library of Australia - NLA) are currently maintaining this as an 
> internal modification to the Filter, but I believe it is generic enough to 
> contribute upstream.
> Description:
> =
> {code}
> /**
>  * NLA Modification to the standard word delimiter to support various
>  * hyphenation use cases. Primarily driven by requirements for
>  * newspapers where words are often broken across line endings.
>  *
>  *  eg. "hyphenated-surname" is printed printed across a line ending and
>  * turns out like "hyphen-ated-surname" or "hyphenated-sur-name".
>  *
>  *  In this scenario the stock filter, with 'catenateAll' turned on, will
>  *  generate individual tokens plus one combined token, but not
>  *  sub-tokens like "hyphenated surname" and "hyphenatedsur name".
>  *
>  *  So we add a new 'catenateShingles' to achieve this.
> */
> {code}
> Includes unit tests, and as is noted in one of them CATENATE_WORDS and 
> CATENATE_SHINGLES are logically considered mutually exclusive for sensible 
> usage and can cause duplicate tokens (although they should have the same 
> positions etc).
> I'm happy to work on it more if anyone finds problems with it.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Progress on SOLR-2649 (MM ignored in edismax queries with operators)

[Greg Pendlebury]

I would like to try and get the proposed fix on SOLR-2649 (
https://issues.apache.org/jira/browse/SOLR-2649) into the codebase if
possible. We have been running a patched v4.7.2 (SOLR-2649-with-Qop.patch)
in production since May 2014, and are currently planning an upgrade to
either Solr 5 or the latest v4.X (time will tell which).

I would love to be able to drop this patch out of our build procedures, and
I am happy to help with any work involved, but not sure where to start (in
terms of procedure, not code). The wiki doco on contributing which I read
last time just said to put patches in Jira, but I guess this one needs a
little more of a push. Given the potentially disruptive nature of the
change inside edismax I think more feedback from other edismax users would
be nice, but the Jira ticket doesn't get a lot of comment normally.

Ta,
Greg

[JENKINS] Lucene-Solr-5.x-Windows (32bit/jdk1.8.0_31) - Build # 4378 - Still Failing!

[Policeman Jenkins Server]

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-5.x-Windows/4378/
Java: 32bit/jdk1.8.0_31 -client -XX:+UseParallelGC

4 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.solr.core.TestSolrConfigHandler

Error Message:
Could not remove the following files (in the order of attempts):
C:\Users\JenkinsSlave\workspace\Lucene-Solr-5.x-Windows\solr\build\solr-core\test\J0\temp\solr.core.TestSolrConfigHandler
 F6361BB219E1C4D5-001\tempDir-010\collection1\conf\configoverlay.json: 
java.nio.file.FileSystemException: 
C:\Users\JenkinsSlave\workspace\Lucene-Solr-5.x-Windows\solr\build\solr-core\test\J0\temp\solr.core.TestSolrConfigHandler
 F6361BB219E1C4D5-001\tempDir-010\collection1\conf\configoverlay.json: The 
process cannot access the file because it is being used by another process. 
C:\Users\JenkinsSlave\workspace\Lucene-Solr-5.x-Windows\solr\build\solr-core\test\J0\temp\solr.core.TestSolrConfigHandler
 F6361BB219E1C4D5-001\tempDir-010\collection1\conf: 
java.nio.file.DirectoryNotEmptyException: 
C:\Users\JenkinsSlave\workspace\Lucene-Solr-5.x-Windows\solr\build\solr-core\test\J0\temp\solr.core.TestSolrConfigHandler
 F6361BB219E1C4D5-001\tempDir-010\collection1\conf
C:\Users\JenkinsSlave\workspace\Lucene-Solr-5.x-Windows\solr\build\solr-core\test\J0\temp\solr.core.TestSolrConfigHandler
 F6361BB219E1C4D5-001\tempDir-010\collection1: 
java.nio.file.DirectoryNotEmptyException: 
C:\Users\JenkinsSlave\workspace\Lucene-Solr-5.x-Windows\solr\build\solr-core\test\J0\temp\solr.core.TestSolrConfigHandler
 F6361BB219E1C4D5-001\tempDir-010\collection1
C:\Users\JenkinsSlave\workspace\Lucene-Solr-5.x-Windows\solr\build\solr-core\test\J0\temp\solr.core.TestSolrConfigHandler
 F6361BB219E1C4D5-001\tempDir-010: java.nio.file.DirectoryNotEmptyException: 
C:\Users\JenkinsSlave\workspace\Lucene-Solr-5.x-Windows\solr\build\solr-core\test\J0\temp\solr.core.TestSolrConfigHandler
 F6361BB219E1C4D5-001\tempDir-010 

Stack Trace:
java.io.IOException: Could not remove the following files (in the order of 
attempts):
   
C:\Users\JenkinsSlave\workspace\Lucene-Solr-5.x-Windows\solr\build\solr-core\test\J0\temp\solr.core.TestSolrConfigHandler
 F6361BB219E1C4D5-001\tempDir-010\collection1\conf\configoverlay.json: 
java.nio.file.FileSystemException: 
C:\Users\JenkinsSlave\workspace\Lucene-Solr-5.x-Windows\solr\build\solr-core\test\J0\temp\solr.core.TestSolrConfigHandler
 F6361BB219E1C4D5-001\tempDir-010\collection1\conf\configoverlay.json: The 
process cannot access the file because it is being used by another process.

   
C:\Users\JenkinsSlave\workspace\Lucene-Solr-5.x-Windows\solr\build\solr-core\test\J0\temp\solr.core.TestSolrConfigHandler
 F6361BB219E1C4D5-001\tempDir-010\collection1\conf: 
java.nio.file.DirectoryNotEmptyException: 
C:\Users\JenkinsSlave\workspace\Lucene-Solr-5.x-Windows\solr\build\solr-core\test\J0\temp\solr.core.TestSolrConfigHandler
 F6361BB219E1C4D5-001\tempDir-010\collection1\conf
   
C:\Users\JenkinsSlave\workspace\Lucene-Solr-5.x-Windows\solr\build\solr-core\test\J0\temp\solr.core.TestSolrConfigHandler
 F6361BB219E1C4D5-001\tempDir-010\collection1: 
java.nio.file.DirectoryNotEmptyException: 
C:\Users\JenkinsSlave\workspace\Lucene-Solr-5.x-Windows\solr\build\solr-core\test\J0\temp\solr.core.TestSolrConfigHandler
 F6361BB219E1C4D5-001\tempDir-010\collection1
   
C:\Users\JenkinsSlave\workspace\Lucene-Solr-5.x-Windows\solr\build\solr-core\test\J0\temp\solr.core.TestSolrConfigHandler
 F6361BB219E1C4D5-001\tempDir-010: java.nio.file.DirectoryNotEmptyException: 
C:\Users\JenkinsSlave\workspace\Lucene-Solr-5.x-Windows\solr\build\solr-core\test\J0\temp\solr.core.TestSolrConfigHandler
 F6361BB219E1C4D5-001\tempDir-010

at __randomizedtesting.SeedInfo.seed([F6361BB219E1C4D5]:0)
at org.apache.lucene.util.IOUtils.rm(IOUtils.java:294)
at 
org.apache.lucene.util.TestRuleTemporaryFilesCleanup.afterAlways(TestRuleTemporaryFilesCleanup.java:170)
at 
com.carrotsearch.randomizedtesting.rules.TestRuleAdapter$1.afterAlways(TestRuleAdapter.java:31)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:43)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
at java.lang.Thread.run(Thread.java:745)


FAILED:  
junit.framework.TestSuite.org.apa

[JENKINS] Lucene-Solr-5.0-Linux (64bit/jdk1.9.0-ea-b47) - Build # 134 - Failure!

[Policeman Jenkins Server]

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-5.0-Linux/134/
Java: 64bit/jdk1.9.0-ea-b47 -XX:+UseCompressedOops -XX:+UseParallelGC

1 tests failed.
FAILED:  org.apache.solr.client.solrj.TestLBHttpSolrClient.testReliability

Error Message:
IOException occured when talking to server at: 
https://127.0.0.1:35777/solr/collection1

Stack Trace:
org.apache.solr.client.solrj.SolrServerException: IOException occured when 
talking to server at: https://127.0.0.1:35777/solr/collection1
at 
__randomizedtesting.SeedInfo.seed([3C1427A1EC912D9D:FDDCFAE74DF7FC34]:0)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:573)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:214)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:210)
at 
org.apache.solr.client.solrj.request.AbstractUpdateRequest.process(AbstractUpdateRequest.java:124)
at org.apache.solr.client.solrj.SolrClient.commit(SolrClient.java:169)
at org.apache.solr.client.solrj.SolrClient.commit(SolrClient.java:147)
at 
org.apache.solr.client.solrj.TestLBHttpSolrClient.addDocs(TestLBHttpSolrClient.java:113)
at 
org.apache.solr.client.solrj.TestLBHttpSolrClient.setUp(TestLBHttpSolrClient.java:96)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:861)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at 
org.apache.lucene

Re: hello solr devs

[Erick Erickson]

Scott:

Please be bold here ;). _Nobody_ will be upset with you putting a
patch up, and you can't check it in and
"muck up the system" since a committer has to take it on and actually,
you know, commit it to the code line.
s/he really takes ultimate responsibility for sanity checking it.

Two especially interesting targets for the ant build are:
1> 'ant precommit'
2> 'ant test'

Please indicate what version of Solr you built the patch against. The
"usual" (but not mandatory) way
is to check out trunk and build the patch against that. Whoever
commits it will merge it into the 5x
code line as part of the checkin process.

If you add unit tests that's even better! It's often easiest to use
one of the pre-existing test files for
SolrJ and put your tests there. Of course you can add a new file if
that makes more sense, but
if something already exists just adding a test to something that runs
already can be simpler.

Best,
Erick

On Thu, Feb 12, 2015 at 3:47 PM, Anshum Gupta  wrote:
> Hi Scott,
>
> The best (and recommended way) to do that would be to just upload your patch
> to https://issues.apache.org/jira/browse/SOLR-949 (the existing issue).
>
>
> On Thu, Feb 12, 2015 at 2:59 PM, Scott C. Cote  wrote:
>>
>> Yes.
>>
>> I looked at the issue and saw that it was unresolved.  Because I wasn’t
>> sure if my company would let me open source my “workaround”, i wrote my code
>> as a  utility to add on to solj.   Fortunately, they said go ahead and
>> release it.
>>
>> I’m a newbie in the “contributor” realm so guidance would be appreciated.
>>
>> The classes as they stand now have the usual corp boilerplate that I will
>> remove.
>>
>> Should I go ahead and attach them as messages in this list so you guys can
>> provide me guidance on how to give it back.   For all i know, you guys might
>> hate it  :)  I would like guidance on what package to place it in as well as
>> naming ….  Will do my best to read the committers guide, but I wanted to
>> give this before I get bogged down at work with other stuff …..
>>
>> Looking forward to your response.
>>
>> Regards,
>>
>> SCott
>>
>> On Feb 12, 2015, at 4:38 PM, Walter Underwood 
>> wrote:
>>
>> It looks like some work on this stalled out a few years ago:
>>
>> https://issues.apache.org/jira/browse/SOLR-949
>>
>> wunder
>> Walter Underwood
>> wun...@wunderwood.org
>> http://observer.wunderwood.org/  (my blog)
>>
>>
>> On Feb 12, 2015, at 2:24 PM, Scott C. Cote  wrote:
>>
>> I have been an on again and off again user of SOLR and had a need to use
>> the Term Vectors that SOLR creates.  Discovered that I could not get them
>> via the Solrj client, so I wrote a utility class to extract the data from
>> the QueryResponse object.  By virtue of how the utility class was authored,
>> it is an “add-on” to the existing SOLR code, not an update.  Additionally, I
>> have permission from my company to give it back to the community  :)
>>
>> For all i know, there is already a solution for what I have done, but I
>> could not find it.
>>
>> So do I need to submit this as a patch, or do I send the “five java source
>> files” to one of you “commiters” for review (which I would actually prefer)?
>> I have a similar utility for the analysis data (discovered the analysis
>> request object afterwards).
>>
>> Am nervous about trying to create a patch and muck up the system.  Would
>> like to take baby steps.
>>
>> Help with helping you help me would be helpful.
>>
>> :)
>>
>> Regards,
>>
>> SCott
>> -
>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
>> For additional commands, e-mail: dev-h...@lucene.apache.org
>>
>>
>>
>
>
>
> --
> Anshum Gupta
> http://about.me/anshumgupta

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: [VOTE] 5.0.0 RC2

[Anshum Gupta]

Thank you everyone! This vote has passed and I'll start the process later
tonight.


On Mon, Feb 9, 2015 at 3:16 PM, Anshum Gupta  wrote:

> Please vote for the second release candidate for Lucene/Solr 5.0.0.
>
> The artifacts can be downloaded here:
>
> http://people.apache.org/~anshum/staging_area/lucene-solr-5.0.0-RC2-rev1658469
>
> Or you can run the smoke tester directly with this command:
> python3.2 dev-tools/scripts/smokeTestRelease.py
> http://people.apache.org/~anshum/staging_area/lucene-solr-5.0.0-RC2-rev1658469
>
>
> I could not get the above command to work as downloading some file or the
> other timed out for me (over 6 attempts) so I instead downloaded the entire
> RC as a tgz. I still have it here:
>
>
> http://people.apache.org/~anshum/staging_area/lucene-solr-5.0.0-RC2-rev1658469.tgz
>
> Untar the above folder at a location of choice. Do not change the name of
> the folder as the smokeTestRelease.py extracts information from that.
>
> and then instead of using http, used file://. Here's the command:
>
> python3.2 dev-tools/scripts/smokeTestRelease.py
> file://
>
> and finally, here's my +1:
>
> > SUCCESS! [0:30:50.246761]
>
>
> --
> Anshum Gupta
> http://about.me/anshumgupta
>



-- 
Anshum Gupta
http://about.me/anshumgupta

[JENKINS] Lucene-Solr-Tests-5.x-Java7 - Build # 2635 - Still Failing

[Apache Jenkins Server]

Build: https://builds.apache.org/job/Lucene-Solr-Tests-5.x-Java7/2635/

7 tests failed.
REGRESSION:  org.apache.solr.handler.TestBlobHandler.doBlobHandlerTest

Error Message:
{responseHeader={status=0, QTime=1}, response={numFound=0, start=0, docs=[]}}

Stack Trace:
java.lang.AssertionError: {responseHeader={status=0, QTime=1}, 
response={numFound=0, start=0, docs=[]}}
at 
__randomizedtesting.SeedInfo.seed([852427C7A664FD20:65E505951D888BD2]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at 
org.apache.solr.handler.TestBlobHandler.doBlobHandlerTest(TestBlobHandler.java:96)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:940)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:915)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(Statem

[JENKINS] Lucene-Solr-trunk-Windows (32bit/jdk1.8.0_31) - Build # 4482 - Still Failing!

[Policeman Jenkins Server]

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Windows/4482/
Java: 32bit/jdk1.8.0_31 -client -XX:+UseParallelGC

3 tests failed.
FAILED:  
junit.framework.TestSuite.org.apache.solr.handler.TestReplicationHandlerBackup

Error Message:
Could not remove the following files (in the order of attempts):
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.handler.TestReplicationHandlerBackup
 32ADD2C64A3E1608-001\solr-instance-002\collection1: 
java.nio.file.DirectoryNotEmptyException: 
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.handler.TestReplicationHandlerBackup
 32ADD2C64A3E1608-001\solr-instance-002\collection1
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.handler.TestReplicationHandlerBackup
 32ADD2C64A3E1608-001\solr-instance-002: 
java.nio.file.DirectoryNotEmptyException: 
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.handler.TestReplicationHandlerBackup
 32ADD2C64A3E1608-001\solr-instance-002
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.handler.TestReplicationHandlerBackup
 32ADD2C64A3E1608-001\solr-instance-001\collection1: 
java.nio.file.DirectoryNotEmptyException: 
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.handler.TestReplicationHandlerBackup
 32ADD2C64A3E1608-001\solr-instance-001\collection1
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.handler.TestReplicationHandlerBackup
 32ADD2C64A3E1608-001\solr-instance-001: 
java.nio.file.DirectoryNotEmptyException: 
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.handler.TestReplicationHandlerBackup
 32ADD2C64A3E1608-001\solr-instance-001
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.handler.TestReplicationHandlerBackup
 32ADD2C64A3E1608-001\solr-instance-001\collection1\data: 
java.nio.file.AccessDeniedException: 
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.handler.TestReplicationHandlerBackup
 32ADD2C64A3E1608-001\solr-instance-001\collection1\data
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.handler.TestReplicationHandlerBackup
 32ADD2C64A3E1608-001\solr-instance-001: 
java.nio.file.DirectoryNotEmptyException: 
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.handler.TestReplicationHandlerBackup
 32ADD2C64A3E1608-001\solr-instance-001
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.handler.TestReplicationHandlerBackup
 32ADD2C64A3E1608-001\solr-instance-002\collection1\data: 
java.nio.file.AccessDeniedException: 
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.handler.TestReplicationHandlerBackup
 32ADD2C64A3E1608-001\solr-instance-002\collection1\data
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.handler.TestReplicationHandlerBackup
 32ADD2C64A3E1608-001\solr-instance-002: 
java.nio.file.DirectoryNotEmptyException: 
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.handler.TestReplicationHandlerBackup
 32ADD2C64A3E1608-001\solr-instance-002
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.handler.TestReplicationHandlerBackup
 32ADD2C64A3E1608-001: java.nio.file.DirectoryNotEmptyException: 
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.handler.TestReplicationHandlerBackup
 32ADD2C64A3E1608-001 

Stack Trace:
java.io.IOException: Could not remove the following files (in the order of 
attempts):
   
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.handler.TestReplicationHandlerBackup
 32ADD2C64A3E1608-001\solr-instance-002\collection1: 
java.nio.file.DirectoryNotEmptyException: 
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.handler.TestReplicationHandlerBackup
 32ADD2C64A3E1608-001\solr-instance-002\collection1
   
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.handler.TestReplicationHandlerBackup
 32ADD2C64A3E1608-001\solr-instance-002: 
java.nio.file.DirectoryNotEmptyException: 
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.handler.TestReplicationHandlerBackup
 32ADD2C64A3E1608-001\solr-instance-002
   
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.handler.TestReplicationHandl

Re: hello solr devs

[Anshum Gupta]

Hi Scott,

The best (and recommended way) to do that would be to just upload your
patch to https://issues.apache.org/jira/browse/SOLR-949 (the existing
issue).


On Thu, Feb 12, 2015 at 2:59 PM, Scott C. Cote  wrote:

> Yes.
>
> I looked at the issue and saw that it was unresolved.  Because I wasn’t
> sure if my company would let me open source my “workaround”, i wrote my
> code as a  utility to add on to solj.   Fortunately, they said go ahead and
> release it.
>
> I’m a newbie in the “contributor” realm so guidance would be appreciated.
>
> The classes as they stand now have the usual corp boilerplate that I will
> remove.
>
> Should I go ahead and attach them as messages in this list so you guys can
> provide me guidance on how to give it back.   For all i know, you guys
> might hate it  :)  I would like guidance on what package to place it in as
> well as naming ….  Will do my best to read the committers guide, but I
> wanted to give this before I get bogged down at work with other stuff …..
>
> Looking forward to your response.
>
> Regards,
>
> SCott
>
> On Feb 12, 2015, at 4:38 PM, Walter Underwood 
> wrote:
>
> It looks like some work on this stalled out a few years ago:
>
> https://issues.apache.org/jira/browse/SOLR-949
>
> wunder
> Walter Underwood
> wun...@wunderwood.org
> http://observer.wunderwood.org/  (my blog)
>
>
> On Feb 12, 2015, at 2:24 PM, Scott C. Cote  wrote:
>
> I have been an on again and off again user of SOLR and had a need to use
> the Term Vectors that SOLR creates.  Discovered that I could not get them
> via the Solrj client, so I wrote a utility class to extract the data from
> the QueryResponse object.  By virtue of how the utility class was authored,
> it is an “add-on” to the existing SOLR code, not an update.  Additionally,
> I have permission from my company to give it back to the community  :)
>
> For all i know, there is already a solution for what I have done, but I
> could not find it.
>
> So do I need to submit this as a patch, or do I send the “five java source
> files” to one of you “commiters” for review (which I would actually
> prefer)?  I have a similar utility for the analysis data (discovered the
> analysis request object afterwards).
>
> Am nervous about trying to create a patch and muck up the system.  Would
> like to take baby steps.
>
> Help with helping you help me would be helpful.
>
> :)
>
> Regards,
>
> SCott
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org
>
>
>
>


-- 
Anshum Gupta
http://about.me/anshumgupta

[JENKINS] Lucene-Solr-trunk-Linux (64bit/jdk1.8.0_40-ea-b22) - Build # 11788 - Failure!

[Policeman Jenkins Server]

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/11788/
Java: 64bit/jdk1.8.0_40-ea-b22 -XX:-UseCompressedOops -XX:+UseParallelGC

1 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.MultiThreadedOCPTest

Error Message:
1 thread leaked from SUITE scope at org.apache.solr.cloud.MultiThreadedOCPTest: 
1) Thread[id=248, name=OverseerThreadFactory-21-thread-5, 
state=TIMED_WAITING, group=Overseer collection creation process.] at 
java.lang.Thread.sleep(Native Method) at 
org.apache.solr.cloud.OverseerCollectionProcessor.waitForCoreNodeName(OverseerCollectionProcessor.java:1823)
 at 
org.apache.solr.cloud.OverseerCollectionProcessor.splitShard(OverseerCollectionProcessor.java:1710)
 at 
org.apache.solr.cloud.OverseerCollectionProcessor.processMessage(OverseerCollectionProcessor.java:622)
 at 
org.apache.solr.cloud.OverseerCollectionProcessor$Runner.run(OverseerCollectionProcessor.java:2862)
 at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) 
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 
at java.lang.Thread.run(Thread.java:745)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE 
scope at org.apache.solr.cloud.MultiThreadedOCPTest: 
   1) Thread[id=248, name=OverseerThreadFactory-21-thread-5, 
state=TIMED_WAITING, group=Overseer collection creation process.]
at java.lang.Thread.sleep(Native Method)
at 
org.apache.solr.cloud.OverseerCollectionProcessor.waitForCoreNodeName(OverseerCollectionProcessor.java:1823)
at 
org.apache.solr.cloud.OverseerCollectionProcessor.splitShard(OverseerCollectionProcessor.java:1710)
at 
org.apache.solr.cloud.OverseerCollectionProcessor.processMessage(OverseerCollectionProcessor.java:622)
at 
org.apache.solr.cloud.OverseerCollectionProcessor$Runner.run(OverseerCollectionProcessor.java:2862)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
at __randomizedtesting.SeedInfo.seed([9C5E364B93B93CCE]:0)




Build Log:
[...truncated 9152 lines...]
   [junit4] Suite: org.apache.solr.cloud.MultiThreadedOCPTest
   [junit4]   2> Creating dataDir: 
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest
 9C5E364B93B93CCE-001/init-core-data-001
   [junit4]   2> 6620 T23 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl 
(true) and clientAuth (false)
   [junit4]   2> 6621 T23 oas.BaseDistributedSearchTestCase.initHostContext 
Setting hostContext system property: /
   [junit4]   2> 6655 T23 oasc.ZkTestServer.run STARTING ZK TEST SERVER
   [junit4]   1> client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 6659 T24 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting 
server
   [junit4]   2> 6758 T23 oasc.ZkTestServer.run start zk server on port:41612
   [junit4]   2> 6778 T23 
oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default 
ZkCredentialsProvider
   [junit4]   2> 6849 T23 oascc.ConnectionManager.waitForConnected Waiting for 
client to connect to ZooKeeper
   [junit4]   2> 6966 T31 oascc.ConnectionManager.process Watcher 
org.apache.solr.common.cloud.ConnectionManager@152e9334 
name:ZooKeeperConnection Watcher:127.0.0.1:41612 got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 6967 T23 oascc.ConnectionManager.waitForConnected Client is 
connected to ZooKeeper
   [junit4]   2> 6969 T23 oascc.SolrZkClient.createZkACLProvider Using default 
ZkACLProvider
   [junit4]   2> 6972 T23 oascc.SolrZkClient.makePath makePath: /solr
   [junit4]   2> 6998 T23 
oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default 
ZkCredentialsProvider
   [junit4]   2> 7000 T23 oascc.ConnectionManager.waitForConnected Waiting for 
client to connect to ZooKeeper
   [junit4]   2> 7002 T34 oascc.ConnectionManager.process Watcher 
org.apache.solr.common.cloud.ConnectionManager@32c91dd name:ZooKeeperConnection 
Watcher:127.0.0.1:41612/solr got event WatchedEvent state:SyncConnected 
type:None path:null path:null type:None
   [junit4]   2> 7002 T23 oascc.ConnectionManager.waitForConnected Client is 
connected to ZooKeeper
   [junit4]   2> 7002 T23 oascc.SolrZkClient.createZkACLProvider Using default 
ZkACLProvider
   [junit4]   2> 7007 T23 oascc.SolrZkClient.makePath makePath: 
/collections/collection1
   [junit4]   2> 7012 T23 oascc.SolrZkClient.makePath makePath: 
/collections/collection1/shards
   [junit4]   2> 7016 T23 oascc.SolrZkClient.makePath makePath: 
/collections/control_collection
   [junit4]   2> 7018 T23 oascc.SolrZkClient.makePath makePath: 
/collections/control_collection/shards
   [junit4]   2> 7022 T23 oasc.AbstractZkTestCase.putConfig put 

[jira] [Commented] (SOLR-6865) Upgrade HttpClient to 4.4

[Shawn Heisey (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-6865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14319197#comment-14319197
 ] 

Shawn Heisey commented on SOLR-6865:


After thinking about this, I think it might be a good idea to wait for 4.4.1 to 
be released, and monitor the httpclient mailing list for a week or so, before 
upgrading.  I think that's the best way to assure stability for this important 
component.

If a serious problem in the current 4.3 release crops up that's fixed in 4.4, I 
can do the upgrade immediately.


> Upgrade HttpClient to 4.4
> -
>
> Key: SOLR-6865
> URL: https://issues.apache.org/jira/browse/SOLR-6865
> Project: Solr
>  Issue Type: Task
>Affects Versions: 5.0
>Reporter: Shawn Heisey
>Priority: Minor
> Fix For: Trunk, 5.1
>
> Attachments: SOLR-6865.patch
>
>
> HttpClient 4.4 has been released.  5.0 seems like a good time to upgrade.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: hello solr devs

[Scott C. Cote]

Thank you.

SCott
> On Feb 12, 2015, at 4:59 PM, Shawn Heisey  wrote:
> 
> On 2/12/2015 3:24 PM, Scott C. Cote wrote:
>> I have been an on again and off again user of SOLR and had a need to use the 
>> Term Vectors that SOLR creates.  Discovered that I could not get them via 
>> the Solrj client, so I wrote a utility class to extract the data from the 
>> QueryResponse object.  By virtue of how the utility class was authored, it 
>> is an “add-on” to the existing SOLR code, not an update.  Additionally, I 
>> have permission from my company to give it back to the community  :)
>> 
>> For all i know, there is already a solution for what I have done, but I 
>> could not find it.
>> 
>> So do I need to submit this as a patch, or do I send the “five java source 
>> files” to one of you “commiters” for review (which I would actually prefer)? 
>>  I have a similar utility for the analysis data (discovered the analysis 
>> request object afterwards).
>> 
>> Am nervous about trying to create a patch and muck up the system.  Would 
>> like to take baby steps.
>> 
>> Help with helping you help me would be helpful.
> 
> I'm also unsure whether there is existing functionality, but I don't
> recall seeing anything for accessing termvectors with SolrJ.
> 
> The first step on this journey is to file an issue in Jira.  If you
> don't already have an account on Apache's jira install, you will need to
> create one:
> 
> https://issues.apache.org/jira/browse/SOLR 
> 
> 
> The next step would be to obtain either branch_5x or trunk from the
> apache subversion repository.  Then you can include your changes in that
> downloaded source repository.
> 
> http://wiki.apache.org/solr/HowToContribute#Getting_the_source_code 
> 
> 
> To create the patch, first run "svn stat" in the root of the checkout to
> see if any files need to be added to your local copy.  Use "svn add" to
> add any that need adding (the output lines will likely start with the !
> character).  Finally, run "svn diff > ../SOLR-.patch" (use a
> backslash if on windows) to create the patch file, and upload that file
> to the issue that you created.
> 
> Someone will review the patch, and if it is acceptable or that person
> knows how to modify it so it's acceptable, it can be cleaned up and
> committed.
> 
> If your patch *only* consists of files added, you could zip those files
> up and upload them to the issue instead of building a patch.  A patch is
> preferred, but not necessarily required.
> 
> Thanks,
> Shawn
> 
> 
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org 
> 
> For additional commands, e-mail: dev-h...@lucene.apache.org 
> 

Re: hello solr devs

[Scott C. Cote]

Yes.

I looked at the issue and saw that it was unresolved.  Because I wasn’t sure if 
my company would let me open source my “workaround”, i wrote my code as a  
utility to add on to solj.   Fortunately, they said go ahead and release it.  

I’m a newbie in the “contributor” realm so guidance would be appreciated.

The classes as they stand now have the usual corp boilerplate that I will 
remove.

Should I go ahead and attach them as messages in this list so you guys can 
provide me guidance on how to give it back.   For all i know, you guys might 
hate it  :)  I would like guidance on what package to place it in as well as 
naming ….  Will do my best to read the committers guide, but I wanted to give 
this before I get bogged down at work with other stuff …..

Looking forward to your response.

Regards,

SCott
> On Feb 12, 2015, at 4:38 PM, Walter Underwood  wrote:
> 
> It looks like some work on this stalled out a few years ago:
> 
> https://issues.apache.org/jira/browse/SOLR-949 
> 
> 
> wunder
> Walter Underwood
> wun...@wunderwood.org 
> http://observer.wunderwood.org/  (my blog)
> 
> 
> On Feb 12, 2015, at 2:24 PM, Scott C. Cote  > wrote:
> 
>> I have been an on again and off again user of SOLR and had a need to use the 
>> Term Vectors that SOLR creates.  Discovered that I could not get them via 
>> the Solrj client, so I wrote a utility class to extract the data from the 
>> QueryResponse object.  By virtue of how the utility class was authored, it 
>> is an “add-on” to the existing SOLR code, not an update.  Additionally, I 
>> have permission from my company to give it back to the community  :)
>> 
>> For all i know, there is already a solution for what I have done, but I 
>> could not find it.
>> 
>> So do I need to submit this as a patch, or do I send the “five java source 
>> files” to one of you “commiters” for review (which I would actually prefer)? 
>>  I have a similar utility for the analysis data (discovered the analysis 
>> request object afterwards).
>> 
>> Am nervous about trying to create a patch and muck up the system.  Would 
>> like to take baby steps.
>> 
>> Help with helping you help me would be helpful.
>> 
>> :)
>> 
>> Regards,
>> 
>> SCott
>> -
>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org 
>> 
>> For additional commands, e-mail: dev-h...@lucene.apache.org 
>> 
>> 
> 

Re: hello solr devs

[Shawn Heisey]

On 2/12/2015 3:24 PM, Scott C. Cote wrote:
> I have been an on again and off again user of SOLR and had a need to use the 
> Term Vectors that SOLR creates.  Discovered that I could not get them via the 
> Solrj client, so I wrote a utility class to extract the data from the 
> QueryResponse object.  By virtue of how the utility class was authored, it is 
> an “add-on” to the existing SOLR code, not an update.  Additionally, I have 
> permission from my company to give it back to the community  :)
>
> For all i know, there is already a solution for what I have done, but I could 
> not find it.
>
> So do I need to submit this as a patch, or do I send the “five java source 
> files” to one of you “commiters” for review (which I would actually prefer)?  
> I have a similar utility for the analysis data (discovered the analysis 
> request object afterwards).
>
> Am nervous about trying to create a patch and muck up the system.  Would like 
> to take baby steps.
>
> Help with helping you help me would be helpful.

I'm also unsure whether there is existing functionality, but I don't
recall seeing anything for accessing termvectors with SolrJ.

The first step on this journey is to file an issue in Jira.  If you
don't already have an account on Apache's jira install, you will need to
create one:

https://issues.apache.org/jira/browse/SOLR

The next step would be to obtain either branch_5x or trunk from the
apache subversion repository.  Then you can include your changes in that
downloaded source repository.

http://wiki.apache.org/solr/HowToContribute#Getting_the_source_code

To create the patch, first run "svn stat" in the root of the checkout to
see if any files need to be added to your local copy.  Use "svn add" to
add any that need adding (the output lines will likely start with the !
character).  Finally, run "svn diff > ../SOLR-.patch" (use a
backslash if on windows) to create the patch file, and upload that file
to the issue that you created.

Someone will review the patch, and if it is acceptable or that person
knows how to modify it so it's acceptable, it can be cleaned up and
committed.

If your patch *only* consists of files added, you could zip those files
up and upload them to the issue instead of building a patch.  A patch is
preferred, but not necessarily required.

Thanks,
Shawn


-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

VOTE: RC0 Release apache-solr-ref-guide-5.0.pdf

[Chris Hostetter]

Please vote to release the following artifact as the "Apache Solr 
Reference Guide for Solr 5.0" ...


https://dist.apache.org/repos/dist/dev/lucene/solr/ref-guide/apache-solr-ref-guide-5.0-RC0/



-Hoss
http://www.lucidworks.com/

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: hello solr devs

[Walter Underwood]

It looks like some work on this stalled out a few years ago:

https://issues.apache.org/jira/browse/SOLR-949

wunder
Walter Underwood
wun...@wunderwood.org
http://observer.wunderwood.org/  (my blog)


On Feb 12, 2015, at 2:24 PM, Scott C. Cote  wrote:

> I have been an on again and off again user of SOLR and had a need to use the 
> Term Vectors that SOLR creates.  Discovered that I could not get them via the 
> Solrj client, so I wrote a utility class to extract the data from the 
> QueryResponse object.  By virtue of how the utility class was authored, it is 
> an “add-on” to the existing SOLR code, not an update.  Additionally, I have 
> permission from my company to give it back to the community  :)
> 
> For all i know, there is already a solution for what I have done, but I could 
> not find it.
> 
> So do I need to submit this as a patch, or do I send the “five java source 
> files” to one of you “commiters” for review (which I would actually prefer)?  
> I have a similar utility for the analysis data (discovered the analysis 
> request object afterwards).
> 
> Am nervous about trying to create a patch and muck up the system.  Would like 
> to take baby steps.
> 
> Help with helping you help me would be helpful.
> 
> :)
> 
> Regards,
> 
> SCott
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org
> 

hello solr devs

[Scott C. Cote]

I have been an on again and off again user of SOLR and had a need to use the 
Term Vectors that SOLR creates.  Discovered that I could not get them via the 
Solrj client, so I wrote a utility class to extract the data from the 
QueryResponse object.  By virtue of how the utility class was authored, it is 
an “add-on” to the existing SOLR code, not an update.  Additionally, I have 
permission from my company to give it back to the community  :)

For all i know, there is already a solution for what I have done, but I could 
not find it.

So do I need to submit this as a patch, or do I send the “five java source 
files” to one of you “commiters” for review (which I would actually prefer)?  I 
have a similar utility for the analysis data (discovered the analysis request 
object afterwards).

Am nervous about trying to create a patch and muck up the system.  Would like 
to take baby steps.

Help with helping you help me would be helpful.

:)

Regards,

SCott
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-6232) Replace ValueSource context Map with a more concrete data type

[Mike Drob (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-6232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mike Drob updated LUCENE-6232:
--
Attachment: LUCENE-6232.patch

Attaching a patch that doesn't clean up all of the raw type warnings, but 
certainly gets a lot of them with a new class {{ValueSourceContext}}

CollapseScore stuff in here would introduce a circular dependency, so maybe 
that gets subclassed.

By my count there are still 7 calls left to {{put()}}, so it's at least a 
manageable number. At this point we might be able to add docs that say this is 
obviously unsafe, and supress the warnings if we can't think of anything better.

> Replace ValueSource context Map with a more concrete data type
> --
>
> Key: LUCENE-6232
> URL: https://issues.apache.org/jira/browse/LUCENE-6232
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Mike Drob
> Attachments: LUCENE-6232.patch
>
>
> Inspired by LUCENE-3973
> The context object used by ValueSource and friends is a raw Map that provides 
> no type safety guarantees. In our current state, there are lots of warnings 
> about unchecked casts, raw types, and generally unsafe code from the 
> compiler's perspective.
> There are several common patterns and types of Objects that we store in the 
> context. It would be beneficial to instead use a class with typed methods for 
> get/set of Scorer, Weights, etc.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-7043) Refactor SolrCLI, bin\solr, bin\solr.cmd to be more unit-testable and less OS specific

[JIRA]

[ 
https://issues.apache.org/jira/browse/SOLR-7043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14319019#comment-14319019
 ] 

Jan Høydahl commented on SOLR-7043:
---

The bin/solr shell/batch script only needs to locate (any) JAVA to start 
SolrCLI, and that logic is already in place. The SolrCLI process itself do not 
need all the JVM flags etc, as its only job would be to parse cmdline opts and 
config, then assemble the complete solr command line and return it to the 
script for execution (to avoid another child process).

> Refactor SolrCLI, bin\solr, bin\solr.cmd to be more unit-testable and less OS 
> specific
> --
>
> Key: SOLR-7043
> URL: https://issues.apache.org/jira/browse/SOLR-7043
> Project: Solr
>  Issue Type: Improvement
>  Components: scripts and tools
>Reporter: Timothy Potter
>Assignee: Timothy Potter
>
> With the 5.0 release, we've reached critical mass with the bin/solr script 
> interface, but we've picked up some cruft along the way. Specifically, 
> there's too much OS-specific constructs in the scripts and they are quite 
> complex overall. They also require extensive manual testing. Moreover, 
> SolrCLI (provides support for the scripts) needs to be refactored to use the 
> Collections API support added to SolrJ instead of using low-level JSON / HTTP 
> constructs. SolrCLI is also in desperate need of a unit test. The overall 
> goal of this ticket is to move as much as possible out of the shell scripts 
> and into SolrCLI, thus increasing test coverage.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-7105) Running Solr as a windows service

[Uwe Schindler (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-7105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14318999#comment-14318999
 ] 

Uwe Schindler commented on SOLR-7105:
-

Yeah, we should then also use the Apache Code Signing Service to sign the 
daemon exe files. Tomcat is also doing this.

> Running Solr as a windows service
> -
>
> Key: SOLR-7105
> URL: https://issues.apache.org/jira/browse/SOLR-7105
> Project: Solr
>  Issue Type: Improvement
>Reporter: Varun Thacker
> Fix For: Trunk, 5.1
>
>
> Since we moved away from shipping a war, it's useful to have scripts to start 
> Solr as a service.
> In 5.0 we already added a script for unix systems, we should also add one for 
> windows.
> The Commons Daemon project seems like a good way to implement it - 
> http://commons.apache.org/proper/commons-daemon/procrun.html



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-7043) Refactor SolrCLI, bin\solr, bin\solr.cmd to be more unit-testable and less OS specific

[Hoss Man (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-7043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14318985#comment-14318985
 ] 

Hoss Man commented on SOLR-7043:


bq. Also, should we deprecate the OS-dependent solr.in.cmd and solr.in.sh in 
favor of some new solr.yml or something? Then SolrCLI would find and parse the 
file.

the scripts have to parse that file because it contains things that need to be 
determined before java can be invoked (ie: JAVA_HOME, gc options, misc java 
command line flags, logging properties, etc...)

> Refactor SolrCLI, bin\solr, bin\solr.cmd to be more unit-testable and less OS 
> specific
> --
>
> Key: SOLR-7043
> URL: https://issues.apache.org/jira/browse/SOLR-7043
> Project: Solr
>  Issue Type: Improvement
>  Components: scripts and tools
>Reporter: Timothy Potter
>Assignee: Timothy Potter
>
> With the 5.0 release, we've reached critical mass with the bin/solr script 
> interface, but we've picked up some cruft along the way. Specifically, 
> there's too much OS-specific constructs in the scripts and they are quite 
> complex overall. They also require extensive manual testing. Moreover, 
> SolrCLI (provides support for the scripts) needs to be refactored to use the 
> Collections API support added to SolrJ instead of using low-level JSON / HTTP 
> constructs. SolrCLI is also in desperate need of a unit test. The overall 
> goal of this ticket is to move as much as possible out of the shell scripts 
> and into SolrCLI, thus increasing test coverage.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-7043) Refactor SolrCLI, bin\solr, bin\solr.cmd to be more unit-testable and less OS specific

[JIRA]

[ 
https://issues.apache.org/jira/browse/SOLR-7043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14318981#comment-14318981
 ] 

Jan Høydahl commented on SOLR-7043:
---

+1

Also, should we deprecate the OS-dependent {{solr.in.cmd}} and {{solr.in.sh}} 
in favor of some new {{solr.yml}} or something? Then SolrCLI would find and 
parse the file.

> Refactor SolrCLI, bin\solr, bin\solr.cmd to be more unit-testable and less OS 
> specific
> --
>
> Key: SOLR-7043
> URL: https://issues.apache.org/jira/browse/SOLR-7043
> Project: Solr
>  Issue Type: Improvement
>  Components: scripts and tools
>Reporter: Timothy Potter
>Assignee: Timothy Potter
>
> With the 5.0 release, we've reached critical mass with the bin/solr script 
> interface, but we've picked up some cruft along the way. Specifically, 
> there's too much OS-specific constructs in the scripts and they are quite 
> complex overall. They also require extensive manual testing. Moreover, 
> SolrCLI (provides support for the scripts) needs to be refactored to use the 
> Collections API support added to SolrJ instead of using low-level JSON / HTTP 
> constructs. SolrCLI is also in desperate need of a unit test. The overall 
> goal of this ticket is to move as much as possible out of the shell scripts 
> and into SolrCLI, thus increasing test coverage.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-Tests-5.x-Java7 - Build # 2634 - Still Failing

[Apache Jenkins Server]

Build: https://builds.apache.org/job/Lucene-Solr-Tests-5.x-Java7/2634/

6 tests failed.
REGRESSION:  org.apache.solr.handler.component.DistributedMLTComponentTest.test

Error Message:
Timeout occured while waiting response from server at: 
http://127.0.0.1:20195//collection1

Stack Trace:
org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting 
response from server at: http://127.0.0.1:20195//collection1
at 
__randomizedtesting.SeedInfo.seed([62E94C4332136D9B:EABD73999CEF0063]:0)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:568)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:214)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:210)
at 
org.apache.solr.client.solrj.request.QueryRequest.process(QueryRequest.java:91)
at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:309)
at 
org.apache.solr.BaseDistributedSearchTestCase.queryServer(BaseDistributedSearchTestCase.java:538)
at 
org.apache.solr.BaseDistributedSearchTestCase.query(BaseDistributedSearchTestCase.java:586)
at 
org.apache.solr.BaseDistributedSearchTestCase.query(BaseDistributedSearchTestCase.java:568)
at 
org.apache.solr.BaseDistributedSearchTestCase.query(BaseDistributedSearchTestCase.java:547)
at 
org.apache.solr.handler.component.DistributedMLTComponentTest.test(DistributedMLTComponentTest.java:126)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:940)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:915)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOr

Re: Enforce "reasonable" field names in Solr?

[Jan Høydahl]

+1 to formally defining field naming

There would be a few absolutes, like
* No spaces (or a convention that spaces will be replaced by _)
* Not start with + or - since they already have special meaning in q parsers
* etc

A list of reserved chars would also be helpful and a well defined way to escape
those to use them. It would however be sad if we disallow "." since it is quite
nice to be able to have dots in field names. Can the f..whatever try
looking for the longest possible string? Should probably disallow "," since
it is used as separator in fl etc. Or is it acceptable with escaping? e.g.
  fl=name\,last,name\,first

+1 to a way to relax the rules for back-compat

--
Jan Høydahl, search solution architect
Cominvent AS - www.cominvent.com

> 12. feb. 2015 kl. 20.26 skrev Alexandre Rafalovitch :
> 
> I wonder if the people who are using dynamic schema care about having
> the fields indexed without _them_ doing pre-processing, but don't mind
> if they have to use cleaned-up names during search. Like, when you
> index from Tika and you just have no clue what possible metadata names
> are in various files. So, you just want to throw the whole lot in,
> prefixed.
> 
> In which case, this could be solved with a specialized
> UpdateRequestProcessor step that will normalize the field names in a
> consistent fashion.
> 
> Regards,
>   Alex.
> 
> Sign up for my Solr resources newsletter at http://www.solr-start.com/
> 
> 
> On 12 February 2015 at 13:02, Erick Erickson  wrote:
>> Jack:
>> 
>> re: your "little gotcha". I suspect there are enough of these lying
>> around that it'd be a rat-hole to formally support them, and as a
>> developer I'd at least like the choice to "fail early fail often".
>> 
>> Your point about dynamic field names is well taken, sometimes there
>> isn't total control of the field names. Which is why I suggested that
>> the strict mode be the default, but overridable.
>> 
>> So not only does the bit about verifying the field names need to take
>> managed schema into account, but also dynamic field definition...
>> Siiih...
>> 
>> That is, if we do anything about it.
>> 
>> 
>> On Thu, Feb 12, 2015 at 8:35 AM, Jack Krupansky
>>  wrote:
>>> I used to be 100% in favor of strict names (well, plus the hyphen!), and in
>>> general it is fine for statically declared fields. But then I started
>>> encountering uses of numbers, spaces, slashes, and other punctuation, but
>>> always in the context of dynamic fields. For example, somebody wants to
>>> support a map-like field using dynamic fields with a dynamic field for each
>>> map key, but their map keys are application-defined and not restricted to
>>> Java name rules, such as a date with punctuation, or something that looks
>>> like a part number with numbers and dashes, or a product name or person or
>>> place name that has spaces and dashes and slashes and commas and periods and
>>> parentheses.
>>> 
>>> The big question is how might Solr depend on strict names, and then how to
>>> properly escape improper field names. There are a lot of spaces that use
>>> field names within some larger syntax, but no consistent escaping rules. For
>>> example, the fl and qf parameters, and fielded queries.
>>> 
>>> Maybe the real bottom line is to assure that the issue of field naming needs
>>> to be clearly documented early on in tutorials and upfront in the doc,
>>> rather than some relatively hidden fine print.
>>> 
>>> Hmmm... what does Elasticsearch do? As long as the field name is simply a
>>> single quoted string, then there is no issue.
>>> 
>>> Oh, here's a great little gotcha: field names embedded in parameters that
>>> are field-specific, like f..facet. URL escaping would be needed,
>>> but are names with embedded dots supported? And does the URL query parameter
>>> syntax support escaping of an equal sign in a query parameter name?
>>> 
>>> 
>>> -- Jack Krupansky
>>> 
>>> On Thu, Feb 12, 2015 at 10:30 AM, Erick Erickson 
>>> wrote:
 
 I was commenting on SOLR-6997 about allowing hyphens in field names
 and started to wonder about whether we should try to push people to
 "good" names. The ref guide states:
 
 "Field names should consist of alphanumeric or underscore characters
 only and not start with a digit"
 
 and SOLR-6997 is a good example of why. I am _not_ at all interested
 in supporting the hyphen BTW.
 
 I realize we can't suddenly start enforcing this rule b/c it would
 break existing installations. What do people think about defaulting to
 throwing an error? Or posting a fat warning with a "deprecation"
 message?
 
 I'm envisioning a "strict_field_name" tag or some such that defaults
 to true, but could be set to false for back compat and just checking
 when parsing a schema.
 
 I'm not at all sure how that plays with the managed schema stuff though.
 
 Erick
 
 --

Re: [VOTE] 5.0.0 RC2

[Shawn Heisey]

I ran into the SOLR-6915 test failure that Shalin mentioned when I ran
it the first time, but a second run (also testing java 8) gave me this:

SUCCESS! [2:36:35.504046]

+1 to a new era for the project!

I hope I can get this running on my dev server very soon.


-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: [VOTE] 5.0.0 RC2

[Erik Hatcher]

I personally would never start with that particular configuration nor recommend 
anyone do (I use the data driven one these days), so I’m +1 with this being a 
non-blocker.

Erik

> On Feb 12, 2015, at 3:04 PM, Anshum Gupta  wrote:
> 
> The basic_configs do not have any field defined in it i.e. no "text" field. 
> This means that a collection created using this configset would throw an 
> exception on a ping request.
> The only way to add a field here is to manually edit the schema and upload 
> (if using it in cloud mode) as this schema isn't managed.
> 
> It'd be good to know what everyone else thinks though. I personally am fine 
> with it as we don't really advertise this much or suggest using this in the 
> ref guide too much.
> 
> On Mon, Feb 9, 2015 at 3:16 PM, Anshum Gupta  > wrote:
> Please vote for the second release candidate for Lucene/Solr 5.0.0.
> 
> The artifacts can be downloaded here:
> http://people.apache.org/~anshum/staging_area/lucene-solr-5.0.0-RC2-rev1658469
>  
> 
> 
> Or you can run the smoke tester directly with this command:
> python3.2 dev-tools/scripts/smokeTestRelease.py 
> http://people.apache.org/~anshum/staging_area/lucene-solr-5.0.0-RC2-rev1658469
>  
> 
> 
> 
> I could not get the above command to work as downloading some file or the 
> other timed out for me (over 6 attempts) so I instead downloaded the entire 
> RC as a tgz. I still have it here:
> 
> http://people.apache.org/~anshum/staging_area/lucene-solr-5.0.0-RC2-rev1658469.tgz
>  
> 
> 
> Untar the above folder at a location of choice. Do not change the name of the 
> folder as the smokeTestRelease.py extracts information from that.
> 
> and then instead of using http, used file://. Here's the command:
> 
> python3.2 dev-tools/scripts/smokeTestRelease.py 
> file://
> 
> and finally, here's my +1:
> 
> > SUCCESS! [0:30:50.246761]
> 
> 
> -- 
> Anshum Gupta
> http://about.me/anshumgupta 
> 
> 
> 
> -- 
> Anshum Gupta
> http://about.me/anshumgupta 

Re: [VOTE] 5.0.0 RC2

[Anshum Gupta]

The basic_configs do not have any field defined in it i.e. no "text" field.
This means that a collection created using this configset would throw an
exception on a ping request.
The only way to add a field here is to manually edit the schema and upload
(if using it in cloud mode) as this schema isn't managed.

It'd be good to know what everyone else thinks though. I personally am fine
with it as we don't really advertise this much or suggest using this in the
ref guide too much.

On Mon, Feb 9, 2015 at 3:16 PM, Anshum Gupta  wrote:

> Please vote for the second release candidate for Lucene/Solr 5.0.0.
>
> The artifacts can be downloaded here:
>
> http://people.apache.org/~anshum/staging_area/lucene-solr-5.0.0-RC2-rev1658469
>
> Or you can run the smoke tester directly with this command:
> python3.2 dev-tools/scripts/smokeTestRelease.py
> http://people.apache.org/~anshum/staging_area/lucene-solr-5.0.0-RC2-rev1658469
>
>
> I could not get the above command to work as downloading some file or the
> other timed out for me (over 6 attempts) so I instead downloaded the entire
> RC as a tgz. I still have it here:
>
>
> http://people.apache.org/~anshum/staging_area/lucene-solr-5.0.0-RC2-rev1658469.tgz
>
> Untar the above folder at a location of choice. Do not change the name of
> the folder as the smokeTestRelease.py extracts information from that.
>
> and then instead of using http, used file://. Here's the command:
>
> python3.2 dev-tools/scripts/smokeTestRelease.py
> file://
>
> and finally, here's my +1:
>
> > SUCCESS! [0:30:50.246761]
>
>
> --
> Anshum Gupta
> http://about.me/anshumgupta
>



-- 
Anshum Gupta
http://about.me/anshumgupta

[jira] [Commented] (SOLR-6671) Introduce a solr.data.root as root dir for all data

[JIRA]

[ 
https://issues.apache.org/jira/browse/SOLR-6671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14318863#comment-14318863
 ] 

Jan Høydahl commented on SOLR-6671:
---

But that would change the directory Solr looks for the core config as well, if 
you're using a plain old cores layout.

My patch is aiming for the possibility to simply add 
{{-Dsolr.data.dir=/mnt/bigdisk}} when starting Solr (eventually bin/solr 
option) to indicate that all data should live at that location, regardless of 
where coreRoot or SOLR_HOME is, and regardless of using config sets, cloud or 
not. I'll whip up a better patch soon.

> Introduce a solr.data.root as root dir for all data
> ---
>
> Key: SOLR-6671
> URL: https://issues.apache.org/jira/browse/SOLR-6671
> Project: Solr
>  Issue Type: New Feature
>  Components: SolrCloud
>Affects Versions: 4.10.1
>Reporter: Jan Høydahl
>Assignee: Jan Høydahl
> Fix For: Trunk, 5.1
>
> Attachments: SOLR-6671.patch
>
>
> Many users prefer to deploy code, config and data on separate disk locations, 
> so the default of placing the indexes under 
> {{$\{solr.solr.home\}/$\{solr.core.name\}/data}} is not always wanted.
> In a multi-core/collection system, there is not much help in the 
> {{solr.data.dir}} option, as it would set the {{dataDir}} to the same folder 
> for all collections. One workaround, if you don't want to hardcode paths in 
> your {{solrconfig.xml}}, is to specify the {{dataDir}} property in each 
> {{solr.properties}} file.
> A more elegant solution would be to introduce a new Java-option 
> {{solr.data.root}} which would be to data the same as {{solr.solr.home}} is 
> for config. If set, all collections would default their {{dataDir}} as 
> {{$\{solr.data.root\)/$\{solr.core.name\}/data}}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-6736) A collections-like request handler to manage solr configurations on zookeeper

[Varun Rajput (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-6736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14318834#comment-14318834
 ] 

Varun Rajput commented on SOLR-6736:


Hey [~noble.paul], I meant to say the ability to upload configurations and not 
the implementation of ZkCli specifically. Doing better IS the aim in this 
ticket and I will try to address the security concerns if possible with your 
suggestions and input.

I am not asking for expanding the scope and am in agreement with you on keeping 
this ticket simple. 



> A collections-like request handler to manage solr configurations on zookeeper
> -
>
> Key: SOLR-6736
> URL: https://issues.apache.org/jira/browse/SOLR-6736
> Project: Solr
>  Issue Type: New Feature
>  Components: SolrCloud
>Reporter: Varun Rajput
>Assignee: Anshum Gupta
>Priority: Minor
> Fix For: 5.0, Trunk
>
> Attachments: SOLR-6736.patch, SOLR-6736.patch
>
>
> Managing Solr configuration files on zookeeper becomes cumbersome while using 
> solr in cloud mode, especially while trying out changes in the 
> configurations. 
> It will be great if there is a request handler that can provide an API to 
> manage the configurations similar to the collections handler that would allow 
> actions like uploading new configurations, linking them to a collection, 
> deleting configurations, etc.
> example : 
> {code}
> #use the following command to upload a new configset called mynewconf. This 
> will fail if there is alredy a conf called 'mynewconf'. The file could be a 
> jar , zip or a tar file which contains all the files for the this conf.
> curl -X POST -H 'Content-Type: application/octet-stream' --data-binary 
> @testconf.zip http://localhost:8983/solr/admin/configs/mynewconf
> {code}
> A GET to http://localhost:8983/solr/admin/configs will give a list of configs 
> available
> A GET to http://localhost:8983/solr/admin/configs/mynewconf would give the 
> list of files in mynewconf



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Enforce "reasonable" field names in Solr?

[Alexandre Rafalovitch]

I wonder if the people who are using dynamic schema care about having
the fields indexed without _them_ doing pre-processing, but don't mind
if they have to use cleaned-up names during search. Like, when you
index from Tika and you just have no clue what possible metadata names
are in various files. So, you just want to throw the whole lot in,
prefixed.

In which case, this could be solved with a specialized
UpdateRequestProcessor step that will normalize the field names in a
consistent fashion.

Regards,
   Alex.

Sign up for my Solr resources newsletter at http://www.solr-start.com/


On 12 February 2015 at 13:02, Erick Erickson  wrote:
> Jack:
>
> re: your "little gotcha". I suspect there are enough of these lying
> around that it'd be a rat-hole to formally support them, and as a
> developer I'd at least like the choice to "fail early fail often".
>
> Your point about dynamic field names is well taken, sometimes there
> isn't total control of the field names. Which is why I suggested that
> the strict mode be the default, but overridable.
>
> So not only does the bit about verifying the field names need to take
> managed schema into account, but also dynamic field definition...
> Siiih...
>
> That is, if we do anything about it.
>
>
> On Thu, Feb 12, 2015 at 8:35 AM, Jack Krupansky
>  wrote:
>> I used to be 100% in favor of strict names (well, plus the hyphen!), and in
>> general it is fine for statically declared fields. But then I started
>> encountering uses of numbers, spaces, slashes, and other punctuation, but
>> always in the context of dynamic fields. For example, somebody wants to
>> support a map-like field using dynamic fields with a dynamic field for each
>> map key, but their map keys are application-defined and not restricted to
>> Java name rules, such as a date with punctuation, or something that looks
>> like a part number with numbers and dashes, or a product name or person or
>> place name that has spaces and dashes and slashes and commas and periods and
>> parentheses.
>>
>> The big question is how might Solr depend on strict names, and then how to
>> properly escape improper field names. There are a lot of spaces that use
>> field names within some larger syntax, but no consistent escaping rules. For
>> example, the fl and qf parameters, and fielded queries.
>>
>> Maybe the real bottom line is to assure that the issue of field naming needs
>> to be clearly documented early on in tutorials and upfront in the doc,
>> rather than some relatively hidden fine print.
>>
>> Hmmm... what does Elasticsearch do? As long as the field name is simply a
>> single quoted string, then there is no issue.
>>
>> Oh, here's a great little gotcha: field names embedded in parameters that
>> are field-specific, like f..facet. URL escaping would be needed,
>> but are names with embedded dots supported? And does the URL query parameter
>> syntax support escaping of an equal sign in a query parameter name?
>>
>>
>> -- Jack Krupansky
>>
>> On Thu, Feb 12, 2015 at 10:30 AM, Erick Erickson 
>> wrote:
>>>
>>> I was commenting on SOLR-6997 about allowing hyphens in field names
>>> and started to wonder about whether we should try to push people to
>>> "good" names. The ref guide states:
>>>
>>> "Field names should consist of alphanumeric or underscore characters
>>> only and not start with a digit"
>>>
>>> and SOLR-6997 is a good example of why. I am _not_ at all interested
>>> in supporting the hyphen BTW.
>>>
>>> I realize we can't suddenly start enforcing this rule b/c it would
>>> break existing installations. What do people think about defaulting to
>>> throwing an error? Or posting a fat warning with a "deprecation"
>>> message?
>>>
>>> I'm envisioning a "strict_field_name" tag or some such that defaults
>>> to true, but could be set to false for back compat and just checking
>>> when parsing a schema.
>>>
>>> I'm not at all sure how that plays with the managed schema stuff though.
>>>
>>> Erick
>>>
>>> -
>>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
>>> For additional commands, e-mail: dev-h...@lucene.apache.org
>>>
>>
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org
>

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-7105) Running Solr as a windows service

[Varun Thacker (JIRA)]

Varun Thacker created SOLR-7105:
---

 Summary: Running Solr as a windows service
 Key: SOLR-7105
 URL: https://issues.apache.org/jira/browse/SOLR-7105
 Project: Solr
  Issue Type: Improvement
Reporter: Varun Thacker
 Fix For: Trunk, 5.1


Since we moved away from shipping a war, it's useful to have scripts to start 
Solr as a service.

In 5.0 we already added a script for unix systems, we should also add one for 
windows.

The Commons Daemon project seems like a good way to implement it - 
http://commons.apache.org/proper/commons-daemon/procrun.html



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-6069) compile with compact profiles

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-6069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14318704#comment-14318704
 ] 

ASF subversion and git services commented on LUCENE-6069:
-

Commit 1659349 from [~thetaphi] in branch 'dev/branches/branch_5x'
[ https://svn.apache.org/r1659349 ]

LUCENE-6069: Backport the test changes to 5.x

> compile with compact profiles
> -
>
> Key: LUCENE-6069
> URL: https://issues.apache.org/jira/browse/LUCENE-6069
> Project: Lucene - Core
>  Issue Type: Task
>  Components: general/build
>Affects Versions: Trunk
>Reporter: Robert Muir
>Assignee: Robert Muir
> Fix For: Trunk
>
> Attachments: LUCENE-6069.patch, LUCENE-6069.patch, LUCENE-6069.patch, 
> LUCENE-6069.patch, LUCENE-6069.patch
>
>
> If we clean up the 'alignment' calculator in RamUsageEstimator, we can 
> compile core with compact1, and the rest of lucene (except tests) with 
> compact2.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (LUCENE-6069) compile with compact profiles

[Uwe Schindler (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-6069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Uwe Schindler resolved LUCENE-6069.
---
Resolution: Fixed

Thanks Robert!
I also backported the test changes to get rid of crazy stuff.

> compile with compact profiles
> -
>
> Key: LUCENE-6069
> URL: https://issues.apache.org/jira/browse/LUCENE-6069
> Project: Lucene - Core
>  Issue Type: Task
>  Components: general/build
>Affects Versions: Trunk
>Reporter: Robert Muir
>Assignee: Robert Muir
> Fix For: Trunk
>
> Attachments: LUCENE-6069.patch, LUCENE-6069.patch, LUCENE-6069.patch, 
> LUCENE-6069.patch, LUCENE-6069.patch
>
>
> If we clean up the 'alignment' calculator in RamUsageEstimator, we can 
> compile core with compact1, and the rest of lucene (except tests) with 
> compact2.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-6069) compile with compact profiles

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-6069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14318690#comment-14318690
 ] 

ASF subversion and git services commented on LUCENE-6069:
-

Commit 1659347 from [~thetaphi] in branch 'dev/trunk'
[ https://svn.apache.org/r1659347 ]

LUCENE-6069: Lucene Core now gets compiled with Java 8 "compact1" profile, all 
other modules with "compact2".

> compile with compact profiles
> -
>
> Key: LUCENE-6069
> URL: https://issues.apache.org/jira/browse/LUCENE-6069
> Project: Lucene - Core
>  Issue Type: Task
>  Components: general/build
>Affects Versions: Trunk
>Reporter: Robert Muir
>Assignee: Robert Muir
> Fix For: Trunk
>
> Attachments: LUCENE-6069.patch, LUCENE-6069.patch, LUCENE-6069.patch, 
> LUCENE-6069.patch, LUCENE-6069.patch
>
>
> If we clean up the 'alignment' calculator in RamUsageEstimator, we can 
> compile core with compact1, and the rest of lucene (except tests) with 
> compact2.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-6069) compile with compact profiles

[Uwe Schindler (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-6069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14318674#comment-14318674
 ] 

Uwe Schindler commented on LUCENE-6069:
---

OK, will do this. I will now run the whole build, to validate all is fine with 
Java 8.

> compile with compact profiles
> -
>
> Key: LUCENE-6069
> URL: https://issues.apache.org/jira/browse/LUCENE-6069
> Project: Lucene - Core
>  Issue Type: Task
>  Components: general/build
>Affects Versions: Trunk
>Reporter: Robert Muir
>Assignee: Robert Muir
> Fix For: Trunk
>
> Attachments: LUCENE-6069.patch, LUCENE-6069.patch, LUCENE-6069.patch, 
> LUCENE-6069.patch, LUCENE-6069.patch
>
>
> If we clean up the 'alignment' calculator in RamUsageEstimator, we can 
> compile core with compact1, and the rest of lucene (except tests) with 
> compact2.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Enforce "reasonable" field names in Solr?

[Erick Erickson]

Jack:

re: your "little gotcha". I suspect there are enough of these lying
around that it'd be a rat-hole to formally support them, and as a
developer I'd at least like the choice to "fail early fail often".

Your point about dynamic field names is well taken, sometimes there
isn't total control of the field names. Which is why I suggested that
the strict mode be the default, but overridable.

So not only does the bit about verifying the field names need to take
managed schema into account, but also dynamic field definition...
Siiih...

That is, if we do anything about it.


On Thu, Feb 12, 2015 at 8:35 AM, Jack Krupansky
 wrote:
> I used to be 100% in favor of strict names (well, plus the hyphen!), and in
> general it is fine for statically declared fields. But then I started
> encountering uses of numbers, spaces, slashes, and other punctuation, but
> always in the context of dynamic fields. For example, somebody wants to
> support a map-like field using dynamic fields with a dynamic field for each
> map key, but their map keys are application-defined and not restricted to
> Java name rules, such as a date with punctuation, or something that looks
> like a part number with numbers and dashes, or a product name or person or
> place name that has spaces and dashes and slashes and commas and periods and
> parentheses.
>
> The big question is how might Solr depend on strict names, and then how to
> properly escape improper field names. There are a lot of spaces that use
> field names within some larger syntax, but no consistent escaping rules. For
> example, the fl and qf parameters, and fielded queries.
>
> Maybe the real bottom line is to assure that the issue of field naming needs
> to be clearly documented early on in tutorials and upfront in the doc,
> rather than some relatively hidden fine print.
>
> Hmmm... what does Elasticsearch do? As long as the field name is simply a
> single quoted string, then there is no issue.
>
> Oh, here's a great little gotcha: field names embedded in parameters that
> are field-specific, like f..facet. URL escaping would be needed,
> but are names with embedded dots supported? And does the URL query parameter
> syntax support escaping of an equal sign in a query parameter name?
>
>
> -- Jack Krupansky
>
> On Thu, Feb 12, 2015 at 10:30 AM, Erick Erickson 
> wrote:
>>
>> I was commenting on SOLR-6997 about allowing hyphens in field names
>> and started to wonder about whether we should try to push people to
>> "good" names. The ref guide states:
>>
>> "Field names should consist of alphanumeric or underscore characters
>> only and not start with a digit"
>>
>> and SOLR-6997 is a good example of why. I am _not_ at all interested
>> in supporting the hyphen BTW.
>>
>> I realize we can't suddenly start enforcing this rule b/c it would
>> break existing installations. What do people think about defaulting to
>> throwing an error? Or posting a fat warning with a "deprecation"
>> message?
>>
>> I'm envisioning a "strict_field_name" tag or some such that defaults
>> to true, but could be set to false for back compat and just checking
>> when parsing a schema.
>>
>> I'm not at all sure how that plays with the managed schema stuff though.
>>
>> Erick
>>
>> -
>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
>> For additional commands, e-mail: dev-h...@lucene.apache.org
>>
>

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: [VOTE] 5.0.0 RC2

[Mark Miller]

+1.

Had a lot of download issues too it looks, but the retries got me through
the smoke tester.

SUCCESS! [1:32:36.658396]


On Thu Feb 12 2015 at 3:12:25 AM Shalin Shekhar Mangar <
shalinman...@gmail.com> wrote:

> +1 to release
>
> Java 7: SUCCESS! [0:27:14.138849]
>
> And with Java8 too: SUCCESS! [0:46:26.948995]
>
>
> On Tue, Feb 10, 2015 at 4:46 AM, Anshum Gupta 
> wrote:
>
>> Please vote for the second release candidate for Lucene/Solr 5.0.0.
>>
>> The artifacts can be downloaded here:
>>
>> http://people.apache.org/~anshum/staging_area/lucene-solr-5.0.0-RC2-rev1658469
>>
>> Or you can run the smoke tester directly with this command:
>> python3.2 dev-tools/scripts/smokeTestRelease.py
>> http://people.apache.org/~anshum/staging_area/lucene-solr-5.0.0-RC2-rev1658469
>>
>>
>> I could not get the above command to work as downloading some file or the
>> other timed out for me (over 6 attempts) so I instead downloaded the entire
>> RC as a tgz. I still have it here:
>>
>>
>> http://people.apache.org/~anshum/staging_area/lucene-solr-5.0.0-RC2-rev1658469.tgz
>>
>> Untar the above folder at a location of choice. Do not change the name of
>> the folder as the smokeTestRelease.py extracts information from that.
>>
>> and then instead of using http, used file://. Here's the command:
>>
>> python3.2 dev-tools/scripts/smokeTestRelease.py
>> file://
>>
>> and finally, here's my +1:
>>
>> > SUCCESS! [0:30:50.246761]
>>
>>
>> --
>> Anshum Gupta
>> http://about.me/anshumgupta
>>
>
>
>
> --
> Regards,
> Shalin Shekhar Mangar.
>

RE: VOTE: Remove sun.misc.Unsafe calls in RamUsageEstimator

[Uwe Schindler]

This was already solved. So no vote needed anymore.

-
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: u...@thetaphi.de


> -Original Message-
> From: Robert Muir [mailto:rcm...@gmail.com]
> Sent: Wednesday, February 11, 2015 1:01 PM
> To: dev@lucene.apache.org
> Subject: VOTE: Remove sun.misc.Unsafe calls in RamUsageEstimator
> 
> See issue: LUCENE-6239
> 
> in my opinion, the objections here are not technical, instead similar to
> objections on https://issues.apache.org/jira/browse/LUCENE-6069.
> "Its my code, don't try to fix it".
> 
> The problem is, all this sheisty, platform-dependent, unsafe code, well, its
> actually totally unnecessary.
> 
> Here is my +1 to remove it.
> 
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional
> commands, e-mail: dev-h...@lucene.apache.org


-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-Tests-5.x-Java7 - Build # 2633 - Still Failing

[Apache Jenkins Server]

Build: https://builds.apache.org/job/Lucene-Solr-Tests-5.x-Java7/2633/

5 tests failed.
FAILED:  org.apache.solr.cloud.HttpPartitionTest.test

Error Message:
org.apache.solr.client.solrj.SolrServerException: IOException occured when 
talking to server at: http://127.0.0.1:14110/c8n_1x2_shard1_replica2

Stack Trace:
org.apache.solr.client.solrj.SolrServerException: 
org.apache.solr.client.solrj.SolrServerException: IOException occured when 
talking to server at: http://127.0.0.1:14110/c8n_1x2_shard1_replica2
at 
__randomizedtesting.SeedInfo.seed([4CE47E402A2256AC:C4B0419A84DE3B54]:0)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.directUpdate(CloudSolrClient.java:575)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:884)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:787)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:730)
at 
org.apache.solr.cloud.HttpPartitionTest.doSendDoc(HttpPartitionTest.java:484)
at 
org.apache.solr.cloud.HttpPartitionTest.sendDoc(HttpPartitionTest.java:501)
at 
org.apache.solr.cloud.HttpPartitionTest.testRf2(HttpPartitionTest.java:193)
at 
org.apache.solr.cloud.HttpPartitionTest.test(HttpPartitionTest.java:106)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:940)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:915)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
co

[jira] [Commented] (SOLR-6736) A collections-like request handler to manage solr configurations on zookeeper

[Noble Paul (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-6736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14318578#comment-14318578
 ] 

Noble Paul commented on SOLR-6736:
--

[~varunrajput]
We DON'T want to mimic ZkCli. We want to do better. There are various security 
implications we need to consider before we provide our API. 

I wish to keep this ticket simple. 

-1 for expanding the scope. 
I'm all for separate tickets if that is the case

> A collections-like request handler to manage solr configurations on zookeeper
> -
>
> Key: SOLR-6736
> URL: https://issues.apache.org/jira/browse/SOLR-6736
> Project: Solr
>  Issue Type: New Feature
>  Components: SolrCloud
>Reporter: Varun Rajput
>Assignee: Anshum Gupta
>Priority: Minor
> Fix For: 5.0, Trunk
>
> Attachments: SOLR-6736.patch, SOLR-6736.patch
>
>
> Managing Solr configuration files on zookeeper becomes cumbersome while using 
> solr in cloud mode, especially while trying out changes in the 
> configurations. 
> It will be great if there is a request handler that can provide an API to 
> manage the configurations similar to the collections handler that would allow 
> actions like uploading new configurations, linking them to a collection, 
> deleting configurations, etc.
> example : 
> {code}
> #use the following command to upload a new configset called mynewconf. This 
> will fail if there is alredy a conf called 'mynewconf'. The file could be a 
> jar , zip or a tar file which contains all the files for the this conf.
> curl -X POST -H 'Content-Type: application/octet-stream' --data-binary 
> @testconf.zip http://localhost:8983/solr/admin/configs/mynewconf
> {code}
> A GET to http://localhost:8983/solr/admin/configs will give a list of configs 
> available
> A GET to http://localhost:8983/solr/admin/configs/mynewconf would give the 
> list of files in mynewconf



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-6736) A collections-like request handler to manage solr configurations on zookeeper

[Varun Rajput (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-6736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14318568#comment-14318568
 ] 

Varun Rajput commented on SOLR-6736:


Thanks for clarifying that [~noble.paul]. This handler mimicks the 
functionality provided by ZkCli with the advantage of it being packaged as a 
handler.

I can go ahead and modify the syntax/semantics described by Noble and remove 
the ability to upload individual files and linking collections once everyone is 
in agreement.

> A collections-like request handler to manage solr configurations on zookeeper
> -
>
> Key: SOLR-6736
> URL: https://issues.apache.org/jira/browse/SOLR-6736
> Project: Solr
>  Issue Type: New Feature
>  Components: SolrCloud
>Reporter: Varun Rajput
>Assignee: Anshum Gupta
>Priority: Minor
> Fix For: 5.0, Trunk
>
> Attachments: SOLR-6736.patch, SOLR-6736.patch
>
>
> Managing Solr configuration files on zookeeper becomes cumbersome while using 
> solr in cloud mode, especially while trying out changes in the 
> configurations. 
> It will be great if there is a request handler that can provide an API to 
> manage the configurations similar to the collections handler that would allow 
> actions like uploading new configurations, linking them to a collection, 
> deleting configurations, etc.
> example : 
> {code}
> #use the following command to upload a new configset called mynewconf. This 
> will fail if there is alredy a conf called 'mynewconf'. The file could be a 
> jar , zip or a tar file which contains all the files for the this conf.
> curl -X POST -H 'Content-Type: application/octet-stream' --data-binary 
> @testconf.zip http://localhost:8983/solr/admin/configs/mynewconf
> {code}
> A GET to http://localhost:8983/solr/admin/configs will give a list of configs 
> available
> A GET to http://localhost:8983/solr/admin/configs/mynewconf would give the 
> list of files in mynewconf



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[GitHub] lucene-solr pull request: Branch 3x

[hossman]

Github user hossman commented on the pull request:

https://github.com/apache/lucene-solr/pull/128#issuecomment-74110665
  
This PR doesn't even remotley make sense ... i suspectit was created by 
mistake?

please close and if you have some specific suggestions for changes to 
lucene that this PR was intended to represent, write & send an email to the 
dev@lucene (apache.org) mailing list procposing your suggestions.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-6198) two phase intersection

[Robert Muir (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-6198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14318557#comment-14318557
 ] 

Robert Muir commented on LUCENE-6198:
-

Yes, I think thats good to show it has nice speedups for positions queries even 
in the smallish case (1KB docs). I think if you ran this on the full documents, 
e.g. wikibig, it would be more dramatic.

> two phase intersection
> --
>
> Key: LUCENE-6198
> URL: https://issues.apache.org/jira/browse/LUCENE-6198
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Robert Muir
> Attachments: LUCENE-6198.patch, LUCENE-6198.patch, LUCENE-6198.patch, 
> LUCENE-6198.patch, phrase_intersections.tasks
>
>
> Currently some scorers have to do a lot of per-document work to determine if 
> a document is a match. The simplest example is a phrase scorer, but there are 
> others (spans, sloppy phrase, geospatial, etc).
> Imagine a conjunction with two MUST clauses, one that is a term that matches 
> all odd documents, another that is a phrase matching all even documents. 
> Today this conjunction will be very expensive, because the zig-zag 
> intersection is reading a ton of useless positions.
> The same problem happens with filteredQuery and anything else that acts like 
> a conjunction.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-6069) compile with compact profiles

[Robert Muir (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-6069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14318548#comment-14318548
 ] 

Robert Muir commented on LUCENE-6069:
-

You can commit, if you want. its is your latest patch. Can we backport the 
tests changes only to branch_5x?

> compile with compact profiles
> -
>
> Key: LUCENE-6069
> URL: https://issues.apache.org/jira/browse/LUCENE-6069
> Project: Lucene - Core
>  Issue Type: Task
>  Components: general/build
>Affects Versions: Trunk
>Reporter: Robert Muir
>Assignee: Robert Muir
> Fix For: Trunk
>
> Attachments: LUCENE-6069.patch, LUCENE-6069.patch, LUCENE-6069.patch, 
> LUCENE-6069.patch, LUCENE-6069.patch
>
>
> If we clean up the 'alignment' calculator in RamUsageEstimator, we can 
> compile core with compact1, and the rest of lucene (except tests) with 
> compact2.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-6069) compile with compact profiles

[Uwe Schindler (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-6069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14318542#comment-14318542
 ] 

Uwe Schindler commented on LUCENE-6069:
---

+1 to commit in trunk! Maybe add some good CHANGES.txt so its public like 
"Lucene Core is Compact profile"

> compile with compact profiles
> -
>
> Key: LUCENE-6069
> URL: https://issues.apache.org/jira/browse/LUCENE-6069
> Project: Lucene - Core
>  Issue Type: Task
>  Components: general/build
>Affects Versions: Trunk
>Reporter: Robert Muir
>Assignee: Robert Muir
> Fix For: Trunk
>
> Attachments: LUCENE-6069.patch, LUCENE-6069.patch, LUCENE-6069.patch, 
> LUCENE-6069.patch, LUCENE-6069.patch
>
>
> If we clean up the 'alignment' calculator in RamUsageEstimator, we can 
> compile core with compact1, and the rest of lucene (except tests) with 
> compact2.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-6069) compile with compact profiles

[Robert Muir (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-6069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14318535#comment-14318535
 ] 

Robert Muir commented on LUCENE-6069:
-

+1 to commit!

> compile with compact profiles
> -
>
> Key: LUCENE-6069
> URL: https://issues.apache.org/jira/browse/LUCENE-6069
> Project: Lucene - Core
>  Issue Type: Task
>  Components: general/build
>Affects Versions: Trunk
>Reporter: Robert Muir
>Assignee: Robert Muir
> Fix For: Trunk
>
> Attachments: LUCENE-6069.patch, LUCENE-6069.patch, LUCENE-6069.patch, 
> LUCENE-6069.patch, LUCENE-6069.patch
>
>
> If we clean up the 'alignment' calculator in RamUsageEstimator, we can 
> compile core with compact1, and the rest of lucene (except tests) with 
> compact2.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS-MAVEN] Lucene-Solr-Maven-trunk #1349: POMs out of sync

[Apache Jenkins Server]

Build: https://builds.apache.org/job/Lucene-Solr-Maven-trunk/1349/

No tests ran.

Build Log:
[...truncated 37045 lines...]
-validate-maven-dependencies:
[artifact:dependencies] [INFO] snapshot 
org.apache.lucene:lucene-test-framework:6.0.0-SNAPSHOT: checking for updates 
from maven-restlet
[artifact:dependencies] [INFO] snapshot 
org.apache.lucene:lucene-test-framework:6.0.0-SNAPSHOT: checking for updates 
from releases.cloudera.com
[artifact:dependencies] [INFO] snapshot 
org.apache.lucene:lucene-parent:6.0.0-SNAPSHOT: checking for updates from 
sonatype.releases
[artifact:dependencies] [INFO] snapshot 
org.apache.lucene:lucene-parent:6.0.0-SNAPSHOT: checking for updates from 
maven-restlet
[artifact:dependencies] [INFO] snapshot 
org.apache.lucene:lucene-parent:6.0.0-SNAPSHOT: checking for updates from 
releases.cloudera.com
[artifact:dependencies] [INFO] snapshot 
org.apache.lucene:lucene-analyzers-common:6.0.0-SNAPSHOT: checking for updates 
from maven-restlet
[artifact:dependencies] [INFO] snapshot 
org.apache.lucene:lucene-analyzers-common:6.0.0-SNAPSHOT: checking for updates 
from releases.cloudera.com
[artifact:dependencies] [INFO] snapshot 
org.apache.lucene:lucene-analyzers-kuromoji:6.0.0-SNAPSHOT: checking for 
updates from maven-restlet
[artifact:dependencies] [INFO] snapshot 
org.apache.lucene:lucene-analyzers-kuromoji:6.0.0-SNAPSHOT: checking for 
updates from releases.cloudera.com
[artifact:dependencies] [INFO] snapshot 
org.apache.lucene:lucene-analyzers-phonetic:6.0.0-SNAPSHOT: checking for 
updates from maven-restlet
[artifact:dependencies] [INFO] snapshot 
org.apache.lucene:lucene-analyzers-phonetic:6.0.0-SNAPSHOT: checking for 
updates from releases.cloudera.com
[artifact:dependencies] [INFO] snapshot 
org.apache.lucene:lucene-backward-codecs:6.0.0-SNAPSHOT: checking for updates 
from maven-restlet
[artifact:dependencies] [INFO] snapshot 
org.apache.lucene:lucene-backward-codecs:6.0.0-SNAPSHOT: checking for updates 
from releases.cloudera.com
[artifact:dependencies] [INFO] snapshot 
org.apache.lucene:lucene-codecs:6.0.0-SNAPSHOT: checking for updates from 
maven-restlet
[artifact:dependencies] [INFO] snapshot 
org.apache.lucene:lucene-codecs:6.0.0-SNAPSHOT: checking for updates from 
releases.cloudera.com
[artifact:dependencies] [INFO] snapshot 
org.apache.lucene:lucene-core:6.0.0-SNAPSHOT: checking for updates from 
maven-restlet
[artifact:dependencies] [INFO] snapshot 
org.apache.lucene:lucene-core:6.0.0-SNAPSHOT: checking for updates from 
releases.cloudera.com
[artifact:dependencies] [INFO] snapshot 
org.apache.lucene:lucene-expressions:6.0.0-SNAPSHOT: checking for updates from 
maven-restlet
[artifact:dependencies] [INFO] snapshot 
org.apache.lucene:lucene-expressions:6.0.0-SNAPSHOT: checking for updates from 
releases.cloudera.com
[artifact:dependencies] [INFO] snapshot 
org.apache.lucene:lucene-grouping:6.0.0-SNAPSHOT: checking for updates from 
maven-restlet
[artifact:dependencies] [INFO] snapshot 
org.apache.lucene:lucene-grouping:6.0.0-SNAPSHOT: checking for updates from 
releases.cloudera.com
[artifact:dependencies] [INFO] snapshot 
org.apache.lucene:lucene-highlighter:6.0.0-SNAPSHOT: checking for updates from 
maven-restlet
[artifact:dependencies] [INFO] snapshot 
org.apache.lucene:lucene-highlighter:6.0.0-SNAPSHOT: checking for updates from 
releases.cloudera.com
[artifact:dependencies] [INFO] snapshot 
org.apache.lucene:lucene-join:6.0.0-SNAPSHOT: checking for updates from 
maven-restlet
[artifact:dependencies] [INFO] snapshot 
org.apache.lucene:lucene-join:6.0.0-SNAPSHOT: checking for updates from 
releases.cloudera.com
[artifact:dependencies] [INFO] snapshot 
org.apache.lucene:lucene-memory:6.0.0-SNAPSHOT: checking for updates from 
maven-restlet
[artifact:dependencies] [INFO] snapshot 
org.apache.lucene:lucene-memory:6.0.0-SNAPSHOT: checking for updates from 
releases.cloudera.com
[artifact:dependencies] [INFO] snapshot 
org.apache.lucene:lucene-misc:6.0.0-SNAPSHOT: checking for updates from 
maven-restlet
[artifact:dependencies] [INFO] snapshot 
org.apache.lucene:lucene-misc:6.0.0-SNAPSHOT: checking for updates from 
releases.cloudera.com
[artifact:dependencies] [INFO] snapshot 
org.apache.lucene:lucene-queries:6.0.0-SNAPSHOT: checking for updates from 
maven-restlet
[artifact:dependencies] [INFO] snapshot 
org.apache.lucene:lucene-queries:6.0.0-SNAPSHOT: checking for updates from 
releases.cloudera.com
[artifact:dependencies] [INFO] snapshot 
org.apache.lucene:lucene-queryparser:6.0.0-SNAPSHOT: checking for updates from 
maven-restlet
[artifact:dependencies] [INFO] snapshot 
org.apache.lucene:lucene-queryparser:6.0.0-SNAPSHOT: checking for updates from 
releases.cloudera.com
[artifact:dependencies] [INFO] snapshot 
org.apache.lucene:lucene-spatial:6.0.0-SNAPSHOT: checking for updates from 
maven-restlet
[artifact:dependencies] [INFO] snapshot 
org.apache.lucene:lucene-spatial:6.0.0-SNAPSHOT: checking for updates from 
releases.cloudera.com
[artifact:dependencies] [IN

Re: Enforce "reasonable" field names in Solr?

[Jack Krupansky]

I used to be 100% in favor of strict names (well, plus the hyphen!), and in
general it is fine for statically declared fields. But then I started
encountering uses of numbers, spaces, slashes, and other punctuation, but
always in the context of dynamic fields. For example, somebody wants to
support a map-like field using dynamic fields with a dynamic field for each
map key, but their map keys are application-defined and not restricted to
Java name rules, such as a date with punctuation, or something that looks
like a part number with numbers and dashes, or a product name or person or
place name that has spaces and dashes and slashes and commas and periods
and parentheses.

The big question is how might Solr depend on strict names, and then how to
properly escape improper field names. There are a lot of spaces that use
field names within some larger syntax, but no consistent escaping rules.
For example, the fl and qf parameters, and fielded queries.

Maybe the real bottom line is to assure that the issue of field naming
needs to be clearly documented early on in tutorials and upfront in the
doc, rather than some relatively hidden fine print.

Hmmm... what does Elasticsearch do? As long as the field name is simply a
single quoted string, then there is no issue.

Oh, here's a great little gotcha: field names embedded in parameters that
are field-specific, like f..facet. URL escaping would be
needed, but are names with embedded dots supported? And does the URL query
parameter syntax support escaping of an equal sign in a query parameter
name?


-- Jack Krupansky

On Thu, Feb 12, 2015 at 10:30 AM, Erick Erickson 
wrote:

> I was commenting on SOLR-6997 about allowing hyphens in field names
> and started to wonder about whether we should try to push people to
> "good" names. The ref guide states:
>
> "Field names should consist of alphanumeric or underscore characters
> only and not start with a digit"
>
> and SOLR-6997 is a good example of why. I am _not_ at all interested
> in supporting the hyphen BTW.
>
> I realize we can't suddenly start enforcing this rule b/c it would
> break existing installations. What do people think about defaulting to
> throwing an error? Or posting a fat warning with a "deprecation"
> message?
>
> I'm envisioning a "strict_field_name" tag or some such that defaults
> to true, but could be set to false for back compat and just checking
> when parsing a schema.
>
> I'm not at all sure how that plays with the managed schema stuff though.
>
> Erick
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org
>
>

[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 761 - Still Failing

[Apache Jenkins Server]

Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/761/

5 tests failed.
FAILED:  org.apache.solr.cloud.HttpPartitionTest.test

Error Message:
org.apache.solr.client.solrj.SolrServerException: IOException occured when 
talking to server at: http://127.0.0.1:33382/hn/n/c8n_1x2_shard1_replica2

Stack Trace:
org.apache.solr.client.solrj.SolrServerException: 
org.apache.solr.client.solrj.SolrServerException: IOException occured when 
talking to server at: http://127.0.0.1:33382/hn/n/c8n_1x2_shard1_replica2
at 
__randomizedtesting.SeedInfo.seed([95E8F46B89D049DF:1DBCCBB1272C2427]:0)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.directUpdate(CloudSolrClient.java:575)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:884)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:787)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:730)
at 
org.apache.solr.cloud.HttpPartitionTest.doSendDoc(HttpPartitionTest.java:484)
at 
org.apache.solr.cloud.HttpPartitionTest.sendDoc(HttpPartitionTest.java:501)
at 
org.apache.solr.cloud.HttpPartitionTest.testRf2(HttpPartitionTest.java:193)
at 
org.apache.solr.cloud.HttpPartitionTest.test(HttpPartitionTest.java:106)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:940)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:915)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)

[jira] [Updated] (LUCENE-6069) compile with compact profiles

[Uwe Schindler (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-6069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Uwe Schindler updated LUCENE-6069:
--
Attachment: LUCENE-6069.patch

Patch with the conflicting changes in RUE removed (already done in LUCENE-6239).

> compile with compact profiles
> -
>
> Key: LUCENE-6069
> URL: https://issues.apache.org/jira/browse/LUCENE-6069
> Project: Lucene - Core
>  Issue Type: Task
>  Components: general/build
>Affects Versions: Trunk
>Reporter: Robert Muir
>Assignee: Robert Muir
> Fix For: Trunk
>
> Attachments: LUCENE-6069.patch, LUCENE-6069.patch, LUCENE-6069.patch, 
> LUCENE-6069.patch, LUCENE-6069.patch
>
>
> If we clean up the 'alignment' calculator in RamUsageEstimator, we can 
> compile core with compact1, and the rest of lucene (except tests) with 
> compact2.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-6239) Remove RAMUsageEstimator Unsafe calls

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-6239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14318497#comment-14318497
 ] 

ASF subversion and git services commented on LUCENE-6239:
-

Commit 1659305 from [~thetaphi] in branch 'dev/branches/branch_5x'
[ https://svn.apache.org/r1659305 ]

Merged revision(s) 1659303 from lucene/dev/trunk:
LUCENE-6239: Removed RAMUsageEstimator's sun.misc.Unsafe calls

> Remove RAMUsageEstimator Unsafe calls
> -
>
> Key: LUCENE-6239
> URL: https://issues.apache.org/jira/browse/LUCENE-6239
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Robert Muir
>Assignee: Uwe Schindler
> Fix For: Trunk, 5.x, 5.1
>
> Attachments: LUCENE-6239.patch, LUCENE-6239.patch, LUCENE-6239.patch
>
>
> This is unnecessary risk. We should remove this stuff, it is not needed here. 
> We are a search engine, not a ram calculator.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (LUCENE-6239) Remove RAMUsageEstimator Unsafe calls

[Uwe Schindler (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-6239?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Uwe Schindler resolved LUCENE-6239.
---
   Resolution: Fixed
Fix Version/s: 5.1
   5.x
   Trunk
 Assignee: Uwe Schindler

> Remove RAMUsageEstimator Unsafe calls
> -
>
> Key: LUCENE-6239
> URL: https://issues.apache.org/jira/browse/LUCENE-6239
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Robert Muir
>Assignee: Uwe Schindler
> Fix For: Trunk, 5.x, 5.1
>
> Attachments: LUCENE-6239.patch, LUCENE-6239.patch, LUCENE-6239.patch
>
>
> This is unnecessary risk. We should remove this stuff, it is not needed here. 
> We are a search engine, not a ram calculator.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-6239) Remove RAMUsageEstimator Unsafe calls

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-6239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14318480#comment-14318480
 ] 

ASF subversion and git services commented on LUCENE-6239:
-

Commit 1659303 from [~thetaphi] in branch 'dev/trunk'
[ https://svn.apache.org/r1659303 ]

LUCENE-6239: Removed RAMUsageEstimator's sun.misc.Unsafe calls

> Remove RAMUsageEstimator Unsafe calls
> -
>
> Key: LUCENE-6239
> URL: https://issues.apache.org/jira/browse/LUCENE-6239
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Robert Muir
> Attachments: LUCENE-6239.patch, LUCENE-6239.patch, LUCENE-6239.patch
>
>
> This is unnecessary risk. We should remove this stuff, it is not needed here. 
> We are a search engine, not a ram calculator.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-6241) don't filter subdirectories in listAll()

[Robert Muir (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-6241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14318471#comment-14318471
 ] 

Robert Muir commented on LUCENE-6241:
-

And for the record, the exception may change in future JDK versions. Its 
actually related to the fsync-directory issue... today you are allowed to open 
a directory as a file and you dont get exception until you try to read.

> don't filter subdirectories in listAll()
> 
>
> Key: LUCENE-6241
> URL: https://issues.apache.org/jira/browse/LUCENE-6241
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Robert Muir
> Attachments: LUCENE-6241.patch, LUCENE-6241.patch
>
>
> The issue is, today this means listAll() is always slow, sometimes MUCH 
> slower, because it must do the fstat()-equivalent of each file to check if 
> its a directory to filter it out.
> When i benchmarked this on a fast filesystem, doing all these filesystem 
> metadata calls only makes listAll() 2.6x slower, but on a non-ssd, slower 
> i/o, it can be more than 60x slower.
> Lucene doesn't make subdirectories, so hiding these for abuse cases just 
> makes real use cases slower.
> To add insult to injury, most code (e.g. all of lucene except for where 
> RAMDir copies from an FSDir) does not actually care if extraneous files are 
> directories or not.
> Finally it sucks the name is listAll() when it is doing anything but that.
> I really hate to add a method here to deal with this abusive stuff, but I'd 
> rather add isDirectory(String) for the rare code that wants to filter out, 
> than just let stuff always be slow.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Comment Edited] (LUCENE-6241) don't filter subdirectories in listAll()

[Robert Muir (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-6241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14318443#comment-14318443
 ] 

Robert Muir edited comment on LUCENE-6241 at 2/12/15 4:10 PM:
--

{quote}
Maybe we should simply remove RAMDir's copy ctor? That method seems 
abusive/trappy to me too!
{quote}

Well, I think there is a real use case for this? Despite how bad it might 
perform, some people want to load their entire index into memory.

I would greatly prefer if we had an option to mmap that called 
MappedByteBuffer.load() or something for these users, this one will even do the 
correct madvise() call and read one byte for each page and do this essentially 
as efficiently as the OS can. I think its a better solution for that use case.

In general, for this issue, I wanted to avoid controversies or larger changes 
like this, my problem is one with operations on Directory API not being 
transparent, I want to make some progress on this and I think they should map 
directly to what is happening on the filesystem. Thats easiest to understand 
and the least trappy.
* listAll() should just list files, not readdir() + fstat()
* rename() should just rename() and not rename() + fsync(dir)
* createOutput should just createOutput, not delete() + create()

edit: i meant load() not force()


was (Author: rcmuir):
{quote}
Maybe we should simply remove RAMDir's copy ctor? That method seems 
abusive/trappy to me too!
{quote}

Well, I think there is a real use case for this? Despite how bad it might 
perform, some people want to load their entire index into memory.

I would greatly prefer if we had an option to mmap that called 
MappedByteBuffer.force() or something for these users, this one will even do 
the correct madvise() call and read one byte for each page and do this 
essentially as efficiently as the OS can. I think its a better solution for 
that use case.

In general, for this issue, I wanted to avoid controversies or larger changes 
like this, my problem is one with operations on Directory API not being 
transparent, I want to make some progress on this and I think they should map 
directly to what is happening on the filesystem. Thats easiest to understand 
and the least trappy.
* listAll() should just list files, not readdir() + fstat()
* rename() should just rename() and not rename() + fsync(dir)
* createOutput should just createOutput, not delete() + create()


> don't filter subdirectories in listAll()
> 
>
> Key: LUCENE-6241
> URL: https://issues.apache.org/jira/browse/LUCENE-6241
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Robert Muir
> Attachments: LUCENE-6241.patch, LUCENE-6241.patch
>
>
> The issue is, today this means listAll() is always slow, sometimes MUCH 
> slower, because it must do the fstat()-equivalent of each file to check if 
> its a directory to filter it out.
> When i benchmarked this on a fast filesystem, doing all these filesystem 
> metadata calls only makes listAll() 2.6x slower, but on a non-ssd, slower 
> i/o, it can be more than 60x slower.
> Lucene doesn't make subdirectories, so hiding these for abuse cases just 
> makes real use cases slower.
> To add insult to injury, most code (e.g. all of lucene except for where 
> RAMDir copies from an FSDir) does not actually care if extraneous files are 
> directories or not.
> Finally it sucks the name is listAll() when it is doing anything but that.
> I really hate to add a method here to deal with this abusive stuff, but I'd 
> rather add isDirectory(String) for the rare code that wants to filter out, 
> than just let stuff always be slow.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-6241) don't filter subdirectories in listAll()

[Michael McCandless (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-6241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14318448#comment-14318448
 ] 

Michael McCandless commented on LUCENE-6241:


OK then +1 to keep RAMDir's copy ctor, have it take FSDirectory, have it check 
itself for directories and skip them.

> don't filter subdirectories in listAll()
> 
>
> Key: LUCENE-6241
> URL: https://issues.apache.org/jira/browse/LUCENE-6241
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Robert Muir
> Attachments: LUCENE-6241.patch, LUCENE-6241.patch
>
>
> The issue is, today this means listAll() is always slow, sometimes MUCH 
> slower, because it must do the fstat()-equivalent of each file to check if 
> its a directory to filter it out.
> When i benchmarked this on a fast filesystem, doing all these filesystem 
> metadata calls only makes listAll() 2.6x slower, but on a non-ssd, slower 
> i/o, it can be more than 60x slower.
> Lucene doesn't make subdirectories, so hiding these for abuse cases just 
> makes real use cases slower.
> To add insult to injury, most code (e.g. all of lucene except for where 
> RAMDir copies from an FSDir) does not actually care if extraneous files are 
> directories or not.
> Finally it sucks the name is listAll() when it is doing anything but that.
> I really hate to add a method here to deal with this abusive stuff, but I'd 
> rather add isDirectory(String) for the rare code that wants to filter out, 
> than just let stuff always be slow.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-6241) don't filter subdirectories in listAll()

[Robert Muir (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-6241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14318443#comment-14318443
 ] 

Robert Muir commented on LUCENE-6241:
-

{quote}
Maybe we should simply remove RAMDir's copy ctor? That method seems 
abusive/trappy to me too!
{quote}

Well, I think there is a real use case for this? Despite how bad it might 
perform, some people want to load their entire index into memory.

I would greatly prefer if we had an option to mmap that called 
MappedByteBuffer.force() or something for these users, this one will even do 
the correct madvise() call and read one byte for each page and do this 
essentially as efficiently as the OS can. I think its a better solution for 
that use case.

In general, for this issue, I wanted to avoid controversies or larger changes 
like this, my problem is one with operations on Directory API not being 
transparent, I want to make some progress on this and I think they should map 
directly to what is happening on the filesystem. Thats easiest to understand 
and the least trappy.
* listAll() should just list files, not readdir() + fstat()
* rename() should just rename() and not rename() + fsync(dir)
* createOutput should just createOutput, not delete() + create()


> don't filter subdirectories in listAll()
> 
>
> Key: LUCENE-6241
> URL: https://issues.apache.org/jira/browse/LUCENE-6241
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Robert Muir
> Attachments: LUCENE-6241.patch, LUCENE-6241.patch
>
>
> The issue is, today this means listAll() is always slow, sometimes MUCH 
> slower, because it must do the fstat()-equivalent of each file to check if 
> its a directory to filter it out.
> When i benchmarked this on a fast filesystem, doing all these filesystem 
> metadata calls only makes listAll() 2.6x slower, but on a non-ssd, slower 
> i/o, it can be more than 60x slower.
> Lucene doesn't make subdirectories, so hiding these for abuse cases just 
> makes real use cases slower.
> To add insult to injury, most code (e.g. all of lucene except for where 
> RAMDir copies from an FSDir) does not actually care if extraneous files are 
> directories or not.
> Finally it sucks the name is listAll() when it is doing anything but that.
> I really hate to add a method here to deal with this abusive stuff, but I'd 
> rather add isDirectory(String) for the rare code that wants to filter out, 
> than just let stuff always be slow.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-6241) don't filter subdirectories in listAll()

[Robert Muir (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-6241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14318434#comment-14318434
 ] 

Robert Muir commented on LUCENE-6241:
-

Just illustration: remove the isDirectory() call in my patch and run 
TestRAMDirectory, the exception would look like this:

{noformat}
java.io.IOException: Is a directory: 
NIOFSIndexInput(path="/tmp/lucene.store.TestRAMDirectory 
F412F4510E9B7B4-001/testsubdir-001/subdir")
at 
__randomizedtesting.SeedInfo.seed([F412F4510E9B7B4:3E7CBDE940612694]:0)
at 
org.apache.lucene.store.NIOFSDirectory$NIOFSIndexInput.readInternal(NIOFSDirectory.java:190)
at 
org.apache.lucene.store.BufferedIndexInput.readBytes(BufferedIndexInput.java:160)
at 
org.apache.lucene.store.BufferedIndexInput.readBytes(BufferedIndexInput.java:116)
at 
org.apache.lucene.store.MockIndexInputWrapper.readBytes(MockIndexInputWrapper.java:140)
at org.apache.lucene.store.DataOutput.copyBytes(DataOutput.java:275)
at org.apache.lucene.store.Directory.copyFrom(Directory.java:156)
at org.apache.lucene.store.RAMDirectory.(RAMDirectory.java:102)
at org.apache.lucene.store.RAMDirectory.(RAMDirectory.java:96)
at 
org.apache.lucene.store.TestRAMDirectory.testCopySubdir(TestRAMDirectory.java:78)
{noformat}

Its not something we can easily "upgrade" to a better exception.


> don't filter subdirectories in listAll()
> 
>
> Key: LUCENE-6241
> URL: https://issues.apache.org/jira/browse/LUCENE-6241
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Robert Muir
> Attachments: LUCENE-6241.patch, LUCENE-6241.patch
>
>
> The issue is, today this means listAll() is always slow, sometimes MUCH 
> slower, because it must do the fstat()-equivalent of each file to check if 
> its a directory to filter it out.
> When i benchmarked this on a fast filesystem, doing all these filesystem 
> metadata calls only makes listAll() 2.6x slower, but on a non-ssd, slower 
> i/o, it can be more than 60x slower.
> Lucene doesn't make subdirectories, so hiding these for abuse cases just 
> makes real use cases slower.
> To add insult to injury, most code (e.g. all of lucene except for where 
> RAMDir copies from an FSDir) does not actually care if extraneous files are 
> directories or not.
> Finally it sucks the name is listAll() when it is doing anything but that.
> I really hate to add a method here to deal with this abusive stuff, but I'd 
> rather add isDirectory(String) for the rare code that wants to filter out, 
> than just let stuff always be slow.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-6241) don't filter subdirectories in listAll()

[Michael McCandless (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-6241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14318431#comment-14318431
 ] 

Michael McCandless commented on LUCENE-6241:


bq. The user will get IOException if we don't do something, and the IOException 
will make it look like a lucene bug to them.

Well, I think that's OK?  The IOExc will specific abusing directory they had 
added to Lucene's index directory?  Plus I think we are talking about very few 
actual occurrences of this: it's abusers who also use RAMDir's copy ctor.

Maybe we should simply remove RAMDir's copy ctor?  That method seems 
abusive/trappy to me too!

> don't filter subdirectories in listAll()
> 
>
> Key: LUCENE-6241
> URL: https://issues.apache.org/jira/browse/LUCENE-6241
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Robert Muir
> Attachments: LUCENE-6241.patch, LUCENE-6241.patch
>
>
> The issue is, today this means listAll() is always slow, sometimes MUCH 
> slower, because it must do the fstat()-equivalent of each file to check if 
> its a directory to filter it out.
> When i benchmarked this on a fast filesystem, doing all these filesystem 
> metadata calls only makes listAll() 2.6x slower, but on a non-ssd, slower 
> i/o, it can be more than 60x slower.
> Lucene doesn't make subdirectories, so hiding these for abuse cases just 
> makes real use cases slower.
> To add insult to injury, most code (e.g. all of lucene except for where 
> RAMDir copies from an FSDir) does not actually care if extraneous files are 
> directories or not.
> Finally it sucks the name is listAll() when it is doing anything but that.
> I really hate to add a method here to deal with this abusive stuff, but I'd 
> rather add isDirectory(String) for the rare code that wants to filter out, 
> than just let stuff always be slow.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-6241) don't filter subdirectories in listAll()

[Robert Muir (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-6241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14318423#comment-14318423
 ] 

Robert Muir commented on LUCENE-6241:
-

{quote}
+1 to this, or to simply disallow the abuse case to RAMDir's copy ctor.
{quote}

What do you mean by disallow the abuse case? The user will get IOException if 
we don't do something, and the IOException will make it look like a lucene bug 
to them.


> don't filter subdirectories in listAll()
> 
>
> Key: LUCENE-6241
> URL: https://issues.apache.org/jira/browse/LUCENE-6241
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Robert Muir
> Attachments: LUCENE-6241.patch, LUCENE-6241.patch
>
>
> The issue is, today this means listAll() is always slow, sometimes MUCH 
> slower, because it must do the fstat()-equivalent of each file to check if 
> its a directory to filter it out.
> When i benchmarked this on a fast filesystem, doing all these filesystem 
> metadata calls only makes listAll() 2.6x slower, but on a non-ssd, slower 
> i/o, it can be more than 60x slower.
> Lucene doesn't make subdirectories, so hiding these for abuse cases just 
> makes real use cases slower.
> To add insult to injury, most code (e.g. all of lucene except for where 
> RAMDir copies from an FSDir) does not actually care if extraneous files are 
> directories or not.
> Finally it sucks the name is listAll() when it is doing anything but that.
> I really hate to add a method here to deal with this abusive stuff, but I'd 
> rather add isDirectory(String) for the rare code that wants to filter out, 
> than just let stuff always be slow.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-6241) don't filter subdirectories in listAll()

[Michael McCandless (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-6241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14318420#comment-14318420
 ] 

Michael McCandless commented on LUCENE-6241:


bq. Another option would be to change this signature from 
RAMDirectory(Directory other) to RAMDirectory(FSDirectory other). FSDirectory 
already has 'Path getDirectory()' so RAMDirectory could use this directly, to 
ignore subdirectories.

+1 to this, or to simply disallow the abuse case to RAMDir's copy ctor.

This way we don't add APIs solely for abuse cases.

> don't filter subdirectories in listAll()
> 
>
> Key: LUCENE-6241
> URL: https://issues.apache.org/jira/browse/LUCENE-6241
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Robert Muir
> Attachments: LUCENE-6241.patch, LUCENE-6241.patch
>
>
> The issue is, today this means listAll() is always slow, sometimes MUCH 
> slower, because it must do the fstat()-equivalent of each file to check if 
> its a directory to filter it out.
> When i benchmarked this on a fast filesystem, doing all these filesystem 
> metadata calls only makes listAll() 2.6x slower, but on a non-ssd, slower 
> i/o, it can be more than 60x slower.
> Lucene doesn't make subdirectories, so hiding these for abuse cases just 
> makes real use cases slower.
> To add insult to injury, most code (e.g. all of lucene except for where 
> RAMDir copies from an FSDir) does not actually care if extraneous files are 
> directories or not.
> Finally it sucks the name is listAll() when it is doing anything but that.
> I really hate to add a method here to deal with this abusive stuff, but I'd 
> rather add isDirectory(String) for the rare code that wants to filter out, 
> than just let stuff always be slow.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-5.x-Windows (64bit/jdk1.7.0_76) - Build # 4377 - Still Failing!

[Policeman Jenkins Server]

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-5.x-Windows/4377/
Java: 64bit/jdk1.7.0_76 -XX:+UseCompressedOops -XX:+UseSerialGC

3 tests failed.
FAILED:  
junit.framework.TestSuite.org.apache.solr.handler.TestReplicationHandlerBackup

Error Message:
Could not remove the following files (in the order of attempts):
C:\Users\JenkinsSlave\workspace\Lucene-Solr-5.x-Windows\solr\build\solr-core\test\J0\temp\solr.handler.TestReplicationHandlerBackup
 D80B5F13B5BE9E5-001\solr-instance-002\collection1: 
java.nio.file.DirectoryNotEmptyException: 
C:\Users\JenkinsSlave\workspace\Lucene-Solr-5.x-Windows\solr\build\solr-core\test\J0\temp\solr.handler.TestReplicationHandlerBackup
 D80B5F13B5BE9E5-001\solr-instance-002\collection1
C:\Users\JenkinsSlave\workspace\Lucene-Solr-5.x-Windows\solr\build\solr-core\test\J0\temp\solr.handler.TestReplicationHandlerBackup
 D80B5F13B5BE9E5-001\solr-instance-002: 
java.nio.file.DirectoryNotEmptyException: 
C:\Users\JenkinsSlave\workspace\Lucene-Solr-5.x-Windows\solr\build\solr-core\test\J0\temp\solr.handler.TestReplicationHandlerBackup
 D80B5F13B5BE9E5-001\solr-instance-002
C:\Users\JenkinsSlave\workspace\Lucene-Solr-5.x-Windows\solr\build\solr-core\test\J0\temp\solr.handler.TestReplicationHandlerBackup
 D80B5F13B5BE9E5-001\solr-instance-001\collection1: 
java.nio.file.DirectoryNotEmptyException: 
C:\Users\JenkinsSlave\workspace\Lucene-Solr-5.x-Windows\solr\build\solr-core\test\J0\temp\solr.handler.TestReplicationHandlerBackup
 D80B5F13B5BE9E5-001\solr-instance-001\collection1
C:\Users\JenkinsSlave\workspace\Lucene-Solr-5.x-Windows\solr\build\solr-core\test\J0\temp\solr.handler.TestReplicationHandlerBackup
 D80B5F13B5BE9E5-001\solr-instance-001: 
java.nio.file.DirectoryNotEmptyException: 
C:\Users\JenkinsSlave\workspace\Lucene-Solr-5.x-Windows\solr\build\solr-core\test\J0\temp\solr.handler.TestReplicationHandlerBackup
 D80B5F13B5BE9E5-001\solr-instance-001
C:\Users\JenkinsSlave\workspace\Lucene-Solr-5.x-Windows\solr\build\solr-core\test\J0\temp\solr.handler.TestReplicationHandlerBackup
 D80B5F13B5BE9E5-001\solr-instance-001\collection1\data: 
java.nio.file.AccessDeniedException: 
C:\Users\JenkinsSlave\workspace\Lucene-Solr-5.x-Windows\solr\build\solr-core\test\J0\temp\solr.handler.TestReplicationHandlerBackup
 D80B5F13B5BE9E5-001\solr-instance-001\collection1\data
C:\Users\JenkinsSlave\workspace\Lucene-Solr-5.x-Windows\solr\build\solr-core\test\J0\temp\solr.handler.TestReplicationHandlerBackup
 D80B5F13B5BE9E5-001\solr-instance-001: 
java.nio.file.DirectoryNotEmptyException: 
C:\Users\JenkinsSlave\workspace\Lucene-Solr-5.x-Windows\solr\build\solr-core\test\J0\temp\solr.handler.TestReplicationHandlerBackup
 D80B5F13B5BE9E5-001\solr-instance-001
C:\Users\JenkinsSlave\workspace\Lucene-Solr-5.x-Windows\solr\build\solr-core\test\J0\temp\solr.handler.TestReplicationHandlerBackup
 D80B5F13B5BE9E5-001\solr-instance-002\collection1\data: 
java.nio.file.AccessDeniedException: 
C:\Users\JenkinsSlave\workspace\Lucene-Solr-5.x-Windows\solr\build\solr-core\test\J0\temp\solr.handler.TestReplicationHandlerBackup
 D80B5F13B5BE9E5-001\solr-instance-002\collection1\data
C:\Users\JenkinsSlave\workspace\Lucene-Solr-5.x-Windows\solr\build\solr-core\test\J0\temp\solr.handler.TestReplicationHandlerBackup
 D80B5F13B5BE9E5-001\solr-instance-002: 
java.nio.file.DirectoryNotEmptyException: 
C:\Users\JenkinsSlave\workspace\Lucene-Solr-5.x-Windows\solr\build\solr-core\test\J0\temp\solr.handler.TestReplicationHandlerBackup
 D80B5F13B5BE9E5-001\solr-instance-002
C:\Users\JenkinsSlave\workspace\Lucene-Solr-5.x-Windows\solr\build\solr-core\test\J0\temp\solr.handler.TestReplicationHandlerBackup
 D80B5F13B5BE9E5-001: java.nio.file.DirectoryNotEmptyException: 
C:\Users\JenkinsSlave\workspace\Lucene-Solr-5.x-Windows\solr\build\solr-core\test\J0\temp\solr.handler.TestReplicationHandlerBackup
 D80B5F13B5BE9E5-001 

Stack Trace:
java.io.IOException: Could not remove the following files (in the order of 
attempts):
   
C:\Users\JenkinsSlave\workspace\Lucene-Solr-5.x-Windows\solr\build\solr-core\test\J0\temp\solr.handler.TestReplicationHandlerBackup
 D80B5F13B5BE9E5-001\solr-instance-002\collection1: 
java.nio.file.DirectoryNotEmptyException: 
C:\Users\JenkinsSlave\workspace\Lucene-Solr-5.x-Windows\solr\build\solr-core\test\J0\temp\solr.handler.TestReplicationHandlerBackup
 D80B5F13B5BE9E5-001\solr-instance-002\collection1
   
C:\Users\JenkinsSlave\workspace\Lucene-Solr-5.x-Windows\solr\build\solr-core\test\J0\temp\solr.handler.TestReplicationHandlerBackup
 D80B5F13B5BE9E5-001\solr-instance-002: 
java.nio.file.DirectoryNotEmptyException: 
C:\Users\JenkinsSlave\workspace\Lucene-Solr-5.x-Windows\solr\build\solr-core\test\J0\temp\solr.handler.TestReplicationHandlerBackup
 D80B5F13B5BE9E5-001\solr-instance-002
   
C:\Users\JenkinsSlave\workspace\Lucene-Solr-5.x-Windows\solr\build\solr-core\test\J0\temp\solr.handler.TestReplicationHandlerBackup
 D80B5F13B5BE9E5-001\solr-instance-001\collectio

[jira] [Created] (SOLR-7104) AddReplica Collection API call discards all the property prefix parameters

[Varun Thacker (JIRA)]

Varun Thacker created SOLR-7104:
---

 Summary: AddReplica Collection API call discards all the property 
prefix parameters
 Key: SOLR-7104
 URL: https://issues.apache.org/jira/browse/SOLR-7104
 Project: Solr
  Issue Type: Bug
Reporter: Varun Thacker
Priority: Minor


{{CollectionsHandler.handleAddReplica}} does not call 
{{copyPropertiesIfNotNull}} . This means all the property prefix values are 
discarded. 

It's behaviour is documented in the ref guide 
https://cwiki.apache.org/confluence/display/solr/Collections+API#CollectionsAPI-api_addreplica
 .



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-6241) don't filter subdirectories in listAll()

[Robert Muir (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-6241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14318413#comment-14318413
 ] 

Robert Muir commented on LUCENE-6241:
-

Another option would be to change this signature from RAMDirectory(Directory 
other) to RAMDirectory(FSDirectory other). FSDirectory already has 'Path 
getDirectory()' so RAMDirectory could use this directly, to ignore 
subdirectories.

> don't filter subdirectories in listAll()
> 
>
> Key: LUCENE-6241
> URL: https://issues.apache.org/jira/browse/LUCENE-6241
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Robert Muir
> Attachments: LUCENE-6241.patch, LUCENE-6241.patch
>
>
> The issue is, today this means listAll() is always slow, sometimes MUCH 
> slower, because it must do the fstat()-equivalent of each file to check if 
> its a directory to filter it out.
> When i benchmarked this on a fast filesystem, doing all these filesystem 
> metadata calls only makes listAll() 2.6x slower, but on a non-ssd, slower 
> i/o, it can be more than 60x slower.
> Lucene doesn't make subdirectories, so hiding these for abuse cases just 
> makes real use cases slower.
> To add insult to injury, most code (e.g. all of lucene except for where 
> RAMDir copies from an FSDir) does not actually care if extraneous files are 
> directories or not.
> Finally it sucks the name is listAll() when it is doing anything but that.
> I really hate to add a method here to deal with this abusive stuff, but I'd 
> rather add isDirectory(String) for the rare code that wants to filter out, 
> than just let stuff always be slow.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Enforce "reasonable" field names in Solr?

[Erick Erickson]

I was commenting on SOLR-6997 about allowing hyphens in field names
and started to wonder about whether we should try to push people to
"good" names. The ref guide states:

"Field names should consist of alphanumeric or underscore characters
only and not start with a digit"

and SOLR-6997 is a good example of why. I am _not_ at all interested
in supporting the hyphen BTW.

I realize we can't suddenly start enforcing this rule b/c it would
break existing installations. What do people think about defaulting to
throwing an error? Or posting a fat warning with a "deprecation"
message?

I'm envisioning a "strict_field_name" tag or some such that defaults
to true, but could be set to false for back compat and just checking
when parsing a schema.

I'm not at all sure how that plays with the managed schema stuff though.

Erick

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-6997) Support distributed 'facet on same field different ways'

[Erick Erickson (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-6997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14318367#comment-14318367
 ] 

Erick Erickson commented on SOLR-6997:
--

While this fix is easy enough, I'm -1 on it. The fact that Solr allows it is 
not the same as supporting it. From the ref guide:

"Field names should consist of alphanumeric or underscore characters only and 
not start with a digit"

I would vastly prefer to go the opposite way, fail any names that didn't follow 
the rule above. Unfortunately, that would break existing installations and 
require re-indexing, which is much too onerous. Historically I think supporting 
"non-standard" field names was more an accident than intentional.

As far as query parsing is concerned, I'd be a bit afraid of this popping out 
somewhere and messing up the NOT operator.

> Support distributed 'facet on same field different ways'
> 
>
> Key: SOLR-6997
> URL: https://issues.apache.org/jira/browse/SOLR-6997
> Project: Solr
>  Issue Type: Bug
>  Components: faceting
>Affects Versions: 4.10.3
>Reporter: Pablo Queixalos
>Priority: Minor
>  Labels: distributed_search
> Fix For: 4.10.4, 5.0
>
> Attachments: 
> 0001-SEA-1284-field-facet-local-param-parsing-fails-on-fi.patch, 
> 0001-honor-facet-field-local-params.patch
>
>
> SOLR-1351 brought the ability to facet on the same field differently using 
> local parameters but distributed process ({{FacetComponent::FieldFacet}}) 
> does not honor local params from _facet string_.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-trunk-Linux (64bit/jdk1.8.0_31) - Build # 11786 - Failure!

[Policeman Jenkins Server]

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/11786/
Java: 64bit/jdk1.8.0_31 -XX:-UseCompressedOops -XX:+UseSerialGC

1 tests failed.
FAILED:  org.apache.solr.handler.TestSolrConfigHandlerConcurrent.test

Error Message:


Stack Trace:
java.lang.NullPointerException
at 
__randomizedtesting.SeedInfo.seed([38F834C69ED466FA:B0AC0B1C30280B02]:0)
at 
org.apache.solr.handler.TestSolrConfigHandlerConcurrent.test(TestSolrConfigHandlerConcurrent.java:117)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:940)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:915)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
at java.lang.Thread.run(Thread.ja

[jira] [Commented] (LUCENE-6241) don't filter subdirectories in listAll()

[Robert Muir (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-6241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14318316#comment-14318316
 ] 

Robert Muir commented on LUCENE-6241:
-

Well, or its copy filters by CODEC_PATTERN, so it only copies in lucene index 
files and not extraneous junk. Thats an alternative here too.

But yes, you get the problem, its only non-lucene usage and lucene in general 
does not care here, so i really hate adding any method to the Directory api.

> don't filter subdirectories in listAll()
> 
>
> Key: LUCENE-6241
> URL: https://issues.apache.org/jira/browse/LUCENE-6241
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Robert Muir
> Attachments: LUCENE-6241.patch, LUCENE-6241.patch
>
>
> The issue is, today this means listAll() is always slow, sometimes MUCH 
> slower, because it must do the fstat()-equivalent of each file to check if 
> its a directory to filter it out.
> When i benchmarked this on a fast filesystem, doing all these filesystem 
> metadata calls only makes listAll() 2.6x slower, but on a non-ssd, slower 
> i/o, it can be more than 60x slower.
> Lucene doesn't make subdirectories, so hiding these for abuse cases just 
> makes real use cases slower.
> To add insult to injury, most code (e.g. all of lucene except for where 
> RAMDir copies from an FSDir) does not actually care if extraneous files are 
> directories or not.
> Finally it sucks the name is listAll() when it is doing anything but that.
> I really hate to add a method here to deal with this abusive stuff, but I'd 
> rather add isDirectory(String) for the rare code that wants to filter out, 
> than just let stuff always be slow.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-5944) Support updates of numeric DocValues

[Ishan Chattopadhyaya (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-5944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ishan Chattopadhyaya updated SOLR-5944:
---
Attachment: SOLR-5944.patch

Bringing the last patch up to trunk, so that further work can be done on this 
issue. This still suffers from the potential consistency issue in replicas if 
updates are reordered, as Yonik mentioned.

> Support updates of numeric DocValues
> 
>
> Key: SOLR-5944
> URL: https://issues.apache.org/jira/browse/SOLR-5944
> Project: Solr
>  Issue Type: New Feature
>Reporter: Ishan Chattopadhyaya
>Assignee: Shalin Shekhar Mangar
> Attachments: SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch
>
>
> LUCENE-5189 introduced support for updates to numeric docvalues. It would be 
> really nice to have Solr support this.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-6229) Remove Scorer.getChildren?

[Uwe Schindler (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-6229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14318298#comment-14318298
 ] 

Uwe Schindler commented on LUCENE-6229:
---

I agree,
actually only one of my customers really used this. And at the time they did 
this, the API was already not perfect, and as Robert said - incomplete and not 
guaranteed to work (Lucene 3 times) - e.g. it only worked if bulk scoring was 
not enabled (the customer just passed correct booleans to weight, so it thought 
that it was not top-level query). The API was really cool in Lucene 4+, but I 
was never using it for real use cases.

The other (current) customer split ther query up already before the scorer was 
invoved (they are interested in counts on sub querys) - so removal is not an 
issue. The code used was actually written by [~simonw], I just inherited it :-)

> Remove Scorer.getChildren?
> --
>
> Key: LUCENE-6229
> URL: https://issues.apache.org/jira/browse/LUCENE-6229
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Adrien Grand
>Priority: Minor
>
> This API is used in a single place in our code base: 
> ToParentBlockJoinCollector. In addition, the usage is a bit buggy given that 
> using this API from a collector only works if setScorer is called with an 
> actual Scorer (and not eg. FakeScorer or BooleanScorer like you would get in 
> disjunctions) so it needs a custom IndexSearcher that does not use the 
> BulkScorer API.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-6241) don't filter subdirectories in listAll()

[Michael McCandless (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-6241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14318296#comment-14318296
 ] 

Michael McCandless commented on LUCENE-6241:


bq.  but how will ramdirs copy-ctor work?

I think RAMDir's copy ctor shouldn't handle this abuse case?  If you are an 
abuser, you do your own copying to RAMDir ...

> don't filter subdirectories in listAll()
> 
>
> Key: LUCENE-6241
> URL: https://issues.apache.org/jira/browse/LUCENE-6241
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Robert Muir
> Attachments: LUCENE-6241.patch, LUCENE-6241.patch
>
>
> The issue is, today this means listAll() is always slow, sometimes MUCH 
> slower, because it must do the fstat()-equivalent of each file to check if 
> its a directory to filter it out.
> When i benchmarked this on a fast filesystem, doing all these filesystem 
> metadata calls only makes listAll() 2.6x slower, but on a non-ssd, slower 
> i/o, it can be more than 60x slower.
> Lucene doesn't make subdirectories, so hiding these for abuse cases just 
> makes real use cases slower.
> To add insult to injury, most code (e.g. all of lucene except for where 
> RAMDir copies from an FSDir) does not actually care if extraneous files are 
> directories or not.
> Finally it sucks the name is listAll() when it is doing anything but that.
> I really hate to add a method here to deal with this abusive stuff, but I'd 
> rather add isDirectory(String) for the rare code that wants to filter out, 
> than just let stuff always be slow.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-6229) Remove Scorer.getChildren?

[Robert Muir (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-6229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14318287#comment-14318287
 ] 

Robert Muir commented on LUCENE-6229:
-

I dont think this is guaranteed. Lets be clear about how this went down.

The functionality was added on LUCENE-2590, but was hardcoded to only work with 
booleanscorer.  Just look at the patch, you dont have to believe me. There was 
also a lot of inconsistencies regarding various scorers. For example issues 
like LUCENE-2686.

But we didnt slow down our core functionality, instead we waited until there 
was a way to clean this stuff up non-intrusively. I added the getChildren API 
to more scorers and fixed lots of bugs like this in issues like LUCENE-3505.

However, this api is *always* best effort. The one and only purpose of scorers 
is to score hits. If someone can make disjunctionscorer 5% faster, and 
completely break this getChildren API, I will be +1.


> Remove Scorer.getChildren?
> --
>
> Key: LUCENE-6229
> URL: https://issues.apache.org/jira/browse/LUCENE-6229
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Adrien Grand
>Priority: Minor
>
> This API is used in a single place in our code base: 
> ToParentBlockJoinCollector. In addition, the usage is a bit buggy given that 
> using this API from a collector only works if setScorer is called with an 
> actual Scorer (and not eg. FakeScorer or BooleanScorer like you would get in 
> disjunctions) so it needs a custom IndexSearcher that does not use the 
> BulkScorer API.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-6242) SparseFixedBitDocIdSet.ramBytesUsed() reports wrong size if alignment of JVM is not 8

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-6242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14318280#comment-14318280
 ] 

ASF subversion and git services commented on LUCENE-6242:
-

Commit 1659278 from [~jpountz] in branch 'dev/branches/branch_5x'
[ https://svn.apache.org/r1659278 ]

LUCENE-6242: Fix SparseFixedBitSet ram usage estimation when object alignment 
is different from 8.

> SparseFixedBitDocIdSet.ramBytesUsed() reports wrong size if alignment of JVM 
> is not 8
> -
>
> Key: LUCENE-6242
> URL: https://issues.apache.org/jira/browse/LUCENE-6242
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Uwe Schindler
> Attachments: LUCENE-6242.patch
>
>
> There seems to be a bug in SparseFixedBitDocIdSet's ramBytesUsed. To me this 
> is a bit crazy implemented, so I have not yet found the issue. To me it looks 
> like some of the summing up breaks if the alignment of the JVM is not 8:
> {noformat}
>[junit4] Suite: org.apache.lucene.util.TestSparseFixedBitDocIdSet
>[junit4]   2> NOTE: reproduce with: ant test  
> -Dtestcase=TestSparseFixedBitDocIdSet -Dtests.method=testRamBytesUsed 
> -Dtests.seed=
> C1F3B881CB1C5E8A -Dtests.locale=es_BO -Dtests.timezone=America/Detroit 
> -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
>[junit4] FAILURE 0.06s J1 | TestSparseFixedBitDocIdSet.testRamBytesUsed <<<
>[junit4]> Throwable #1: java.lang.AssertionError: expected:<264> but 
> was:<256>
>[junit4]>at 
> __randomizedtesting.SeedInfo.seed([C1F3B881CB1C5E8A:3350AAC1016341DC]:0)
>[junit4]>at 
> org.apache.lucene.util.BaseDocIdSetTestCase.testRamBytesUsed(BaseDocIdSetTestCase.java:104)
>[junit4]>at java.lang.Thread.run(Thread.java:745)
> {noformat}
> To reproduce this failure, run inside core (with 64 bits JVM):
> {noformat}
> ant test -Dtestcase=TestSparseFixedBitDocIdSet 
> -Dtests.method=testRamBytesUsed -Dargs="-XX:ObjectAlignmentInBytes=16" 
> -Dtests.seed=C1F3B881CB1C5E8A
> {noformat}
> The default works:
> {noformat}
> ant test -Dtestcase=TestSparseFixedBitDocIdSet 
> -Dtests.method=testRamBytesUsed -Dargs="-XX:ObjectAlignmentInBytes=8" 
> -Dtests.seed=C1F3B881CB1C5E8A
> {noformat}
> I think we should randomly also specify the ObjectAlignmentInBytes for test 
> runs on Policeman Jenkins. Any Power of 2 is fine.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-6229) Remove Scorer.getChildren?

[Uwe Schindler (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-6229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14318279#comment-14318279
 ] 

Uwe Schindler commented on LUCENE-6229:
---

I had/have 2 customers using that... But I don't think it is really in wide use.

> Remove Scorer.getChildren?
> --
>
> Key: LUCENE-6229
> URL: https://issues.apache.org/jira/browse/LUCENE-6229
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Adrien Grand
>Priority: Minor
>
> This API is used in a single place in our code base: 
> ToParentBlockJoinCollector. In addition, the usage is a bit buggy given that 
> using this API from a collector only works if setScorer is called with an 
> actual Scorer (and not eg. FakeScorer or BooleanScorer like you would get in 
> disjunctions) so it needs a custom IndexSearcher that does not use the 
> BulkScorer API.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-6229) Remove Scorer.getChildren?

[Terry Smith (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-6229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14318272#comment-14318272
 ] 

Terry Smith commented on LUCENE-6229:
-

[~jpountz] - I'm going to split the freq() vs score() thing into a separate 
ticket so it doesn't hijack this one. I intend to take the unit test I 
previously pasted and extend it to create some randomized BooleanQuerys to try 
and locate possibly broken edge cases and give a safety blanket for future 
refactoring.

I'll make these assumptions, shout out if they are incorrect.

For a BooleanQuery I should be able to perform doc-at-a-time scoring, meaning 
that in a Collector or Scorer I can

1. Find all Scorers from the child clauses of the BooleanQuery
2. Have those Scorers be positioned for me by calling score() or freq()


> Remove Scorer.getChildren?
> --
>
> Key: LUCENE-6229
> URL: https://issues.apache.org/jira/browse/LUCENE-6229
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Adrien Grand
>Priority: Minor
>
> This API is used in a single place in our code base: 
> ToParentBlockJoinCollector. In addition, the usage is a bit buggy given that 
> using this API from a collector only works if setScorer is called with an 
> actual Scorer (and not eg. FakeScorer or BooleanScorer like you would get in 
> disjunctions) so it needs a custom IndexSearcher that does not use the 
> BulkScorer API.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-6242) SparseFixedBitDocIdSet.ramBytesUsed() reports wrong size if alignment of JVM is not 8

[Uwe Schindler (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-6242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14318271#comment-14318271
 ] 

Uwe Schindler commented on LUCENE-6242:
---

I think we should still run the G1GC tests to be able to show to people, that 
it is currently a bad idea to run with G1GC.

> SparseFixedBitDocIdSet.ramBytesUsed() reports wrong size if alignment of JVM 
> is not 8
> -
>
> Key: LUCENE-6242
> URL: https://issues.apache.org/jira/browse/LUCENE-6242
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Uwe Schindler
> Attachments: LUCENE-6242.patch
>
>
> There seems to be a bug in SparseFixedBitDocIdSet's ramBytesUsed. To me this 
> is a bit crazy implemented, so I have not yet found the issue. To me it looks 
> like some of the summing up breaks if the alignment of the JVM is not 8:
> {noformat}
>[junit4] Suite: org.apache.lucene.util.TestSparseFixedBitDocIdSet
>[junit4]   2> NOTE: reproduce with: ant test  
> -Dtestcase=TestSparseFixedBitDocIdSet -Dtests.method=testRamBytesUsed 
> -Dtests.seed=
> C1F3B881CB1C5E8A -Dtests.locale=es_BO -Dtests.timezone=America/Detroit 
> -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
>[junit4] FAILURE 0.06s J1 | TestSparseFixedBitDocIdSet.testRamBytesUsed <<<
>[junit4]> Throwable #1: java.lang.AssertionError: expected:<264> but 
> was:<256>
>[junit4]>at 
> __randomizedtesting.SeedInfo.seed([C1F3B881CB1C5E8A:3350AAC1016341DC]:0)
>[junit4]>at 
> org.apache.lucene.util.BaseDocIdSetTestCase.testRamBytesUsed(BaseDocIdSetTestCase.java:104)
>[junit4]>at java.lang.Thread.run(Thread.java:745)
> {noformat}
> To reproduce this failure, run inside core (with 64 bits JVM):
> {noformat}
> ant test -Dtestcase=TestSparseFixedBitDocIdSet 
> -Dtests.method=testRamBytesUsed -Dargs="-XX:ObjectAlignmentInBytes=16" 
> -Dtests.seed=C1F3B881CB1C5E8A
> {noformat}
> The default works:
> {noformat}
> ant test -Dtestcase=TestSparseFixedBitDocIdSet 
> -Dtests.method=testRamBytesUsed -Dargs="-XX:ObjectAlignmentInBytes=8" 
> -Dtests.seed=C1F3B881CB1C5E8A
> {noformat}
> I think we should randomly also specify the ObjectAlignmentInBytes for test 
> runs on Policeman Jenkins. Any Power of 2 is fine.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-6671) Introduce a solr.data.root as root dir for all data

[Alan Woodward (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-6671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14318270#comment-14318270
 ] 

Alan Woodward commented on SOLR-6671:
-

I think this can be achieved now using a combination of configsets (or 
SolrCloud) and setting coreRootDirectory in solr.xml?

> Introduce a solr.data.root as root dir for all data
> ---
>
> Key: SOLR-6671
> URL: https://issues.apache.org/jira/browse/SOLR-6671
> Project: Solr
>  Issue Type: New Feature
>  Components: SolrCloud
>Affects Versions: 4.10.1
>Reporter: Jan Høydahl
>Assignee: Jan Høydahl
> Fix For: Trunk, 5.1
>
> Attachments: SOLR-6671.patch
>
>
> Many users prefer to deploy code, config and data on separate disk locations, 
> so the default of placing the indexes under 
> {{$\{solr.solr.home\}/$\{solr.core.name\}/data}} is not always wanted.
> In a multi-core/collection system, there is not much help in the 
> {{solr.data.dir}} option, as it would set the {{dataDir}} to the same folder 
> for all collections. One workaround, if you don't want to hardcode paths in 
> your {{solrconfig.xml}}, is to specify the {{dataDir}} property in each 
> {{solr.properties}} file.
> A more elegant solution would be to introduce a new Java-option 
> {{solr.data.root}} which would be to data the same as {{solr.solr.home}} is 
> for config. If set, all collections would default their {{dataDir}} as 
> {{$\{solr.data.root\)/$\{solr.core.name\}/data}}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-6242) SparseFixedBitDocIdSet.ramBytesUsed() reports wrong size if alignment of JVM is not 8

[Robert Muir (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-6242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14318267#comment-14318267
 ] 

Robert Muir commented on LUCENE-6242:
-

{quote}
 I am already annoyed by G1GC tests!
{quote}

Yes, thats exactly my problem. When looking at any test failure that has G1GC, 
i know there is a 50% chance (at least) that I am completely wasting my time. 
Many days, i see G1GC was in use, and just move on with my day, because I 
simply dont have the time to waste.


> SparseFixedBitDocIdSet.ramBytesUsed() reports wrong size if alignment of JVM 
> is not 8
> -
>
> Key: LUCENE-6242
> URL: https://issues.apache.org/jira/browse/LUCENE-6242
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Uwe Schindler
> Attachments: LUCENE-6242.patch
>
>
> There seems to be a bug in SparseFixedBitDocIdSet's ramBytesUsed. To me this 
> is a bit crazy implemented, so I have not yet found the issue. To me it looks 
> like some of the summing up breaks if the alignment of the JVM is not 8:
> {noformat}
>[junit4] Suite: org.apache.lucene.util.TestSparseFixedBitDocIdSet
>[junit4]   2> NOTE: reproduce with: ant test  
> -Dtestcase=TestSparseFixedBitDocIdSet -Dtests.method=testRamBytesUsed 
> -Dtests.seed=
> C1F3B881CB1C5E8A -Dtests.locale=es_BO -Dtests.timezone=America/Detroit 
> -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
>[junit4] FAILURE 0.06s J1 | TestSparseFixedBitDocIdSet.testRamBytesUsed <<<
>[junit4]> Throwable #1: java.lang.AssertionError: expected:<264> but 
> was:<256>
>[junit4]>at 
> __randomizedtesting.SeedInfo.seed([C1F3B881CB1C5E8A:3350AAC1016341DC]:0)
>[junit4]>at 
> org.apache.lucene.util.BaseDocIdSetTestCase.testRamBytesUsed(BaseDocIdSetTestCase.java:104)
>[junit4]>at java.lang.Thread.run(Thread.java:745)
> {noformat}
> To reproduce this failure, run inside core (with 64 bits JVM):
> {noformat}
> ant test -Dtestcase=TestSparseFixedBitDocIdSet 
> -Dtests.method=testRamBytesUsed -Dargs="-XX:ObjectAlignmentInBytes=16" 
> -Dtests.seed=C1F3B881CB1C5E8A
> {noformat}
> The default works:
> {noformat}
> ant test -Dtestcase=TestSparseFixedBitDocIdSet 
> -Dtests.method=testRamBytesUsed -Dargs="-XX:ObjectAlignmentInBytes=8" 
> -Dtests.seed=C1F3B881CB1C5E8A
> {noformat}
> I think we should randomly also specify the ObjectAlignmentInBytes for test 
> runs on Policeman Jenkins. Any Power of 2 is fine.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-6242) SparseFixedBitDocIdSet.ramBytesUsed() reports wrong size if alignment of JVM is not 8

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-6242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14318264#comment-14318264
 ] 

ASF subversion and git services commented on LUCENE-6242:
-

Commit 1659273 from [~jpountz] in branch 'dev/trunk'
[ https://svn.apache.org/r1659273 ]

LUCENE-6242: Fix SparseFixedBitSet ram usage estimation when object alignment 
is different from 8.

> SparseFixedBitDocIdSet.ramBytesUsed() reports wrong size if alignment of JVM 
> is not 8
> -
>
> Key: LUCENE-6242
> URL: https://issues.apache.org/jira/browse/LUCENE-6242
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Uwe Schindler
> Attachments: LUCENE-6242.patch
>
>
> There seems to be a bug in SparseFixedBitDocIdSet's ramBytesUsed. To me this 
> is a bit crazy implemented, so I have not yet found the issue. To me it looks 
> like some of the summing up breaks if the alignment of the JVM is not 8:
> {noformat}
>[junit4] Suite: org.apache.lucene.util.TestSparseFixedBitDocIdSet
>[junit4]   2> NOTE: reproduce with: ant test  
> -Dtestcase=TestSparseFixedBitDocIdSet -Dtests.method=testRamBytesUsed 
> -Dtests.seed=
> C1F3B881CB1C5E8A -Dtests.locale=es_BO -Dtests.timezone=America/Detroit 
> -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
>[junit4] FAILURE 0.06s J1 | TestSparseFixedBitDocIdSet.testRamBytesUsed <<<
>[junit4]> Throwable #1: java.lang.AssertionError: expected:<264> but 
> was:<256>
>[junit4]>at 
> __randomizedtesting.SeedInfo.seed([C1F3B881CB1C5E8A:3350AAC1016341DC]:0)
>[junit4]>at 
> org.apache.lucene.util.BaseDocIdSetTestCase.testRamBytesUsed(BaseDocIdSetTestCase.java:104)
>[junit4]>at java.lang.Thread.run(Thread.java:745)
> {noformat}
> To reproduce this failure, run inside core (with 64 bits JVM):
> {noformat}
> ant test -Dtestcase=TestSparseFixedBitDocIdSet 
> -Dtests.method=testRamBytesUsed -Dargs="-XX:ObjectAlignmentInBytes=16" 
> -Dtests.seed=C1F3B881CB1C5E8A
> {noformat}
> The default works:
> {noformat}
> ant test -Dtestcase=TestSparseFixedBitDocIdSet 
> -Dtests.method=testRamBytesUsed -Dargs="-XX:ObjectAlignmentInBytes=8" 
> -Dtests.seed=C1F3B881CB1C5E8A
> {noformat}
> I think we should randomly also specify the ObjectAlignmentInBytes for test 
> runs on Policeman Jenkins. Any Power of 2 is fine.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-6242) SparseFixedBitDocIdSet.ramBytesUsed() reports wrong size if alignment of JVM is not 8

[Uwe Schindler (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-6242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14318230#comment-14318230
 ] 

Uwe Schindler commented on LUCENE-6242:
---

I would only look at them if they are related to ramBytesUsed calculations :-) 
This was my intention in running those tests.

I agree, we should not really randomize those. I am already annoyed by G1GC 
tests!

> SparseFixedBitDocIdSet.ramBytesUsed() reports wrong size if alignment of JVM 
> is not 8
> -
>
> Key: LUCENE-6242
> URL: https://issues.apache.org/jira/browse/LUCENE-6242
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Uwe Schindler
> Attachments: LUCENE-6242.patch
>
>
> There seems to be a bug in SparseFixedBitDocIdSet's ramBytesUsed. To me this 
> is a bit crazy implemented, so I have not yet found the issue. To me it looks 
> like some of the summing up breaks if the alignment of the JVM is not 8:
> {noformat}
>[junit4] Suite: org.apache.lucene.util.TestSparseFixedBitDocIdSet
>[junit4]   2> NOTE: reproduce with: ant test  
> -Dtestcase=TestSparseFixedBitDocIdSet -Dtests.method=testRamBytesUsed 
> -Dtests.seed=
> C1F3B881CB1C5E8A -Dtests.locale=es_BO -Dtests.timezone=America/Detroit 
> -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
>[junit4] FAILURE 0.06s J1 | TestSparseFixedBitDocIdSet.testRamBytesUsed <<<
>[junit4]> Throwable #1: java.lang.AssertionError: expected:<264> but 
> was:<256>
>[junit4]>at 
> __randomizedtesting.SeedInfo.seed([C1F3B881CB1C5E8A:3350AAC1016341DC]:0)
>[junit4]>at 
> org.apache.lucene.util.BaseDocIdSetTestCase.testRamBytesUsed(BaseDocIdSetTestCase.java:104)
>[junit4]>at java.lang.Thread.run(Thread.java:745)
> {noformat}
> To reproduce this failure, run inside core (with 64 bits JVM):
> {noformat}
> ant test -Dtestcase=TestSparseFixedBitDocIdSet 
> -Dtests.method=testRamBytesUsed -Dargs="-XX:ObjectAlignmentInBytes=16" 
> -Dtests.seed=C1F3B881CB1C5E8A
> {noformat}
> The default works:
> {noformat}
> ant test -Dtestcase=TestSparseFixedBitDocIdSet 
> -Dtests.method=testRamBytesUsed -Dargs="-XX:ObjectAlignmentInBytes=8" 
> -Dtests.seed=C1F3B881CB1C5E8A
> {noformat}
> I think we should randomly also specify the ObjectAlignmentInBytes for test 
> runs on Policeman Jenkins. Any Power of 2 is fine.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-6242) SparseFixedBitDocIdSet.ramBytesUsed() reports wrong size if alignment of JVM is not 8

[Robert Muir (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-6242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14318227#comment-14318227
 ] 

Robert Muir commented on LUCENE-6242:
-

{quote}
At least we should fix this issue here. If we enable random alignments could be 
decided after some local tests. In any case I would enable this only very 
seldom with values 8 <= x <= 128.
{quote}

You can do what you like with your jenkins, but if the failures aren't using 
the default alignment, I won't even look at them.

I don't think we should test this.

> SparseFixedBitDocIdSet.ramBytesUsed() reports wrong size if alignment of JVM 
> is not 8
> -
>
> Key: LUCENE-6242
> URL: https://issues.apache.org/jira/browse/LUCENE-6242
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Uwe Schindler
> Attachments: LUCENE-6242.patch
>
>
> There seems to be a bug in SparseFixedBitDocIdSet's ramBytesUsed. To me this 
> is a bit crazy implemented, so I have not yet found the issue. To me it looks 
> like some of the summing up breaks if the alignment of the JVM is not 8:
> {noformat}
>[junit4] Suite: org.apache.lucene.util.TestSparseFixedBitDocIdSet
>[junit4]   2> NOTE: reproduce with: ant test  
> -Dtestcase=TestSparseFixedBitDocIdSet -Dtests.method=testRamBytesUsed 
> -Dtests.seed=
> C1F3B881CB1C5E8A -Dtests.locale=es_BO -Dtests.timezone=America/Detroit 
> -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
>[junit4] FAILURE 0.06s J1 | TestSparseFixedBitDocIdSet.testRamBytesUsed <<<
>[junit4]> Throwable #1: java.lang.AssertionError: expected:<264> but 
> was:<256>
>[junit4]>at 
> __randomizedtesting.SeedInfo.seed([C1F3B881CB1C5E8A:3350AAC1016341DC]:0)
>[junit4]>at 
> org.apache.lucene.util.BaseDocIdSetTestCase.testRamBytesUsed(BaseDocIdSetTestCase.java:104)
>[junit4]>at java.lang.Thread.run(Thread.java:745)
> {noformat}
> To reproduce this failure, run inside core (with 64 bits JVM):
> {noformat}
> ant test -Dtestcase=TestSparseFixedBitDocIdSet 
> -Dtests.method=testRamBytesUsed -Dargs="-XX:ObjectAlignmentInBytes=16" 
> -Dtests.seed=C1F3B881CB1C5E8A
> {noformat}
> The default works:
> {noformat}
> ant test -Dtestcase=TestSparseFixedBitDocIdSet 
> -Dtests.method=testRamBytesUsed -Dargs="-XX:ObjectAlignmentInBytes=8" 
> -Dtests.seed=C1F3B881CB1C5E8A
> {noformat}
> I think we should randomly also specify the ObjectAlignmentInBytes for test 
> runs on Policeman Jenkins. Any Power of 2 is fine.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-6242) SparseFixedBitDocIdSet.ramBytesUsed() reports wrong size if alignment of JVM is not 8

[Uwe Schindler (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-6242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14318222#comment-14318222
 ] 

Uwe Schindler commented on LUCENE-6242:
---

Yeah, I think I understand the logic now. Looks good to me!
If you tested already I am more than fine with this patch!

> SparseFixedBitDocIdSet.ramBytesUsed() reports wrong size if alignment of JVM 
> is not 8
> -
>
> Key: LUCENE-6242
> URL: https://issues.apache.org/jira/browse/LUCENE-6242
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Uwe Schindler
> Attachments: LUCENE-6242.patch
>
>
> There seems to be a bug in SparseFixedBitDocIdSet's ramBytesUsed. To me this 
> is a bit crazy implemented, so I have not yet found the issue. To me it looks 
> like some of the summing up breaks if the alignment of the JVM is not 8:
> {noformat}
>[junit4] Suite: org.apache.lucene.util.TestSparseFixedBitDocIdSet
>[junit4]   2> NOTE: reproduce with: ant test  
> -Dtestcase=TestSparseFixedBitDocIdSet -Dtests.method=testRamBytesUsed 
> -Dtests.seed=
> C1F3B881CB1C5E8A -Dtests.locale=es_BO -Dtests.timezone=America/Detroit 
> -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
>[junit4] FAILURE 0.06s J1 | TestSparseFixedBitDocIdSet.testRamBytesUsed <<<
>[junit4]> Throwable #1: java.lang.AssertionError: expected:<264> but 
> was:<256>
>[junit4]>at 
> __randomizedtesting.SeedInfo.seed([C1F3B881CB1C5E8A:3350AAC1016341DC]:0)
>[junit4]>at 
> org.apache.lucene.util.BaseDocIdSetTestCase.testRamBytesUsed(BaseDocIdSetTestCase.java:104)
>[junit4]>at java.lang.Thread.run(Thread.java:745)
> {noformat}
> To reproduce this failure, run inside core (with 64 bits JVM):
> {noformat}
> ant test -Dtestcase=TestSparseFixedBitDocIdSet 
> -Dtests.method=testRamBytesUsed -Dargs="-XX:ObjectAlignmentInBytes=16" 
> -Dtests.seed=C1F3B881CB1C5E8A
> {noformat}
> The default works:
> {noformat}
> ant test -Dtestcase=TestSparseFixedBitDocIdSet 
> -Dtests.method=testRamBytesUsed -Dargs="-XX:ObjectAlignmentInBytes=8" 
> -Dtests.seed=C1F3B881CB1C5E8A
> {noformat}
> I think we should randomly also specify the ObjectAlignmentInBytes for test 
> runs on Policeman Jenkins. Any Power of 2 is fine.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-6242) SparseFixedBitDocIdSet.ramBytesUsed() reports wrong size if alignment of JVM is not 8

[Adrien Grand (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-6242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14318217#comment-14318217
 ] 

Adrien Grand commented on LUCENE-6242:
--

I just beasted (50 iters) with alignements of 8, 16, 32, 64 and 128 and tests 
passed.

> SparseFixedBitDocIdSet.ramBytesUsed() reports wrong size if alignment of JVM 
> is not 8
> -
>
> Key: LUCENE-6242
> URL: https://issues.apache.org/jira/browse/LUCENE-6242
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Uwe Schindler
> Attachments: LUCENE-6242.patch
>
>
> There seems to be a bug in SparseFixedBitDocIdSet's ramBytesUsed. To me this 
> is a bit crazy implemented, so I have not yet found the issue. To me it looks 
> like some of the summing up breaks if the alignment of the JVM is not 8:
> {noformat}
>[junit4] Suite: org.apache.lucene.util.TestSparseFixedBitDocIdSet
>[junit4]   2> NOTE: reproduce with: ant test  
> -Dtestcase=TestSparseFixedBitDocIdSet -Dtests.method=testRamBytesUsed 
> -Dtests.seed=
> C1F3B881CB1C5E8A -Dtests.locale=es_BO -Dtests.timezone=America/Detroit 
> -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
>[junit4] FAILURE 0.06s J1 | TestSparseFixedBitDocIdSet.testRamBytesUsed <<<
>[junit4]> Throwable #1: java.lang.AssertionError: expected:<264> but 
> was:<256>
>[junit4]>at 
> __randomizedtesting.SeedInfo.seed([C1F3B881CB1C5E8A:3350AAC1016341DC]:0)
>[junit4]>at 
> org.apache.lucene.util.BaseDocIdSetTestCase.testRamBytesUsed(BaseDocIdSetTestCase.java:104)
>[junit4]>at java.lang.Thread.run(Thread.java:745)
> {noformat}
> To reproduce this failure, run inside core (with 64 bits JVM):
> {noformat}
> ant test -Dtestcase=TestSparseFixedBitDocIdSet 
> -Dtests.method=testRamBytesUsed -Dargs="-XX:ObjectAlignmentInBytes=16" 
> -Dtests.seed=C1F3B881CB1C5E8A
> {noformat}
> The default works:
> {noformat}
> ant test -Dtestcase=TestSparseFixedBitDocIdSet 
> -Dtests.method=testRamBytesUsed -Dargs="-XX:ObjectAlignmentInBytes=8" 
> -Dtests.seed=C1F3B881CB1C5E8A
> {noformat}
> I think we should randomly also specify the ObjectAlignmentInBytes for test 
> runs on Policeman Jenkins. Any Power of 2 is fine.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-6242) SparseFixedBitDocIdSet.ramBytesUsed() reports wrong size if alignment of JVM is not 8

[Uwe Schindler (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-6242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14318216#comment-14318216
 ] 

Uwe Schindler commented on LUCENE-6242:
---

Hehe, I was expecting that something like this is missing, I assumed that the 
Length-1-Longs are the bad ones. I have no idea if this fix is correct, because 
I have no idea how the ramBytesUsed are calculated :-)

Did you check some values of the alignment and ran it with "test beast" on 
several values of alignment?

> SparseFixedBitDocIdSet.ramBytesUsed() reports wrong size if alignment of JVM 
> is not 8
> -
>
> Key: LUCENE-6242
> URL: https://issues.apache.org/jira/browse/LUCENE-6242
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Uwe Schindler
> Attachments: LUCENE-6242.patch
>
>
> There seems to be a bug in SparseFixedBitDocIdSet's ramBytesUsed. To me this 
> is a bit crazy implemented, so I have not yet found the issue. To me it looks 
> like some of the summing up breaks if the alignment of the JVM is not 8:
> {noformat}
>[junit4] Suite: org.apache.lucene.util.TestSparseFixedBitDocIdSet
>[junit4]   2> NOTE: reproduce with: ant test  
> -Dtestcase=TestSparseFixedBitDocIdSet -Dtests.method=testRamBytesUsed 
> -Dtests.seed=
> C1F3B881CB1C5E8A -Dtests.locale=es_BO -Dtests.timezone=America/Detroit 
> -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
>[junit4] FAILURE 0.06s J1 | TestSparseFixedBitDocIdSet.testRamBytesUsed <<<
>[junit4]> Throwable #1: java.lang.AssertionError: expected:<264> but 
> was:<256>
>[junit4]>at 
> __randomizedtesting.SeedInfo.seed([C1F3B881CB1C5E8A:3350AAC1016341DC]:0)
>[junit4]>at 
> org.apache.lucene.util.BaseDocIdSetTestCase.testRamBytesUsed(BaseDocIdSetTestCase.java:104)
>[junit4]>at java.lang.Thread.run(Thread.java:745)
> {noformat}
> To reproduce this failure, run inside core (with 64 bits JVM):
> {noformat}
> ant test -Dtestcase=TestSparseFixedBitDocIdSet 
> -Dtests.method=testRamBytesUsed -Dargs="-XX:ObjectAlignmentInBytes=16" 
> -Dtests.seed=C1F3B881CB1C5E8A
> {noformat}
> The default works:
> {noformat}
> ant test -Dtestcase=TestSparseFixedBitDocIdSet 
> -Dtests.method=testRamBytesUsed -Dargs="-XX:ObjectAlignmentInBytes=8" 
> -Dtests.seed=C1F3B881CB1C5E8A
> {noformat}
> I think we should randomly also specify the ObjectAlignmentInBytes for test 
> runs on Policeman Jenkins. Any Power of 2 is fine.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[GitHub] lucene-solr pull request: Branch 3x

[elyograg]

Github user elyograg commented on the pull request:

https://github.com/apache/lucene-solr/pull/128#issuecomment-74073810
  
I don't think this is going to happen, for a few reasons:

1) branch_3x no longer exists in the current repositories.  Neither does 
branch_4x.  The current stable development branch is branch_5x.

2) The 3x branch was deleted a LONG time ago.  The amount of change in 
trunk since then is *enormous*, so the merging will NOT be clean.  Conflict 
resolution will be manual and very tedious.

3) Almost all commits on a stable branch like branch_3x are already in 
trunk, because that's how we do development - changes are applied to trunk and 
then backported to whatever the stable branch is at the time.  I would be 
willing to bet that almost all of the changes you have listed here are already 
in trunk.  This fact probably causes even more merge conflicts than the age of 
branch_3x, and means that most of these changes are completely unnecessary.

4) This patch is HUGE.  Nobody will agree to merge anything unless it makes 
a discrete change that can be easily analyzed to determine what's being done 
and how it benefits the project.  Very large changes must be broken up into 
much smaller changes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-6242) SparseFixedBitDocIdSet.ramBytesUsed() reports wrong size if alignment of JVM is not 8

[Adrien Grand (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-6242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Adrien Grand updated LUCENE-6242:
-
Attachment: LUCENE-6242.patch

Here is a patch.

> SparseFixedBitDocIdSet.ramBytesUsed() reports wrong size if alignment of JVM 
> is not 8
> -
>
> Key: LUCENE-6242
> URL: https://issues.apache.org/jira/browse/LUCENE-6242
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Uwe Schindler
> Attachments: LUCENE-6242.patch
>
>
> There seems to be a bug in SparseFixedBitDocIdSet's ramBytesUsed. To me this 
> is a bit crazy implemented, so I have not yet found the issue. To me it looks 
> like some of the summing up breaks if the alignment of the JVM is not 8:
> {noformat}
>[junit4] Suite: org.apache.lucene.util.TestSparseFixedBitDocIdSet
>[junit4]   2> NOTE: reproduce with: ant test  
> -Dtestcase=TestSparseFixedBitDocIdSet -Dtests.method=testRamBytesUsed 
> -Dtests.seed=
> C1F3B881CB1C5E8A -Dtests.locale=es_BO -Dtests.timezone=America/Detroit 
> -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
>[junit4] FAILURE 0.06s J1 | TestSparseFixedBitDocIdSet.testRamBytesUsed <<<
>[junit4]> Throwable #1: java.lang.AssertionError: expected:<264> but 
> was:<256>
>[junit4]>at 
> __randomizedtesting.SeedInfo.seed([C1F3B881CB1C5E8A:3350AAC1016341DC]:0)
>[junit4]>at 
> org.apache.lucene.util.BaseDocIdSetTestCase.testRamBytesUsed(BaseDocIdSetTestCase.java:104)
>[junit4]>at java.lang.Thread.run(Thread.java:745)
> {noformat}
> To reproduce this failure, run inside core (with 64 bits JVM):
> {noformat}
> ant test -Dtestcase=TestSparseFixedBitDocIdSet 
> -Dtests.method=testRamBytesUsed -Dargs="-XX:ObjectAlignmentInBytes=16" 
> -Dtests.seed=C1F3B881CB1C5E8A
> {noformat}
> The default works:
> {noformat}
> ant test -Dtestcase=TestSparseFixedBitDocIdSet 
> -Dtests.method=testRamBytesUsed -Dargs="-XX:ObjectAlignmentInBytes=8" 
> -Dtests.seed=C1F3B881CB1C5E8A
> {noformat}
> I think we should randomly also specify the ObjectAlignmentInBytes for test 
> runs on Policeman Jenkins. Any Power of 2 is fine.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-6242) SparseFixedBitDocIdSet.ramBytesUsed() reports wrong size if alignment of JVM is not 8

[Adrien Grand (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-6242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14318194#comment-14318194
 ] 

Adrien Grand commented on LUCENE-6242:
--

Thanks Uwe, I will look into this issue.

> SparseFixedBitDocIdSet.ramBytesUsed() reports wrong size if alignment of JVM 
> is not 8
> -
>
> Key: LUCENE-6242
> URL: https://issues.apache.org/jira/browse/LUCENE-6242
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Uwe Schindler
>
> There seems to be a bug in SparseFixedBitDocIdSet's ramBytesUsed. To me this 
> is a bit crazy implemented, so I have not yet found the issue. To me it looks 
> like some of the summing up breaks if the alignment of the JVM is not 8:
> {noformat}
>[junit4] Suite: org.apache.lucene.util.TestSparseFixedBitDocIdSet
>[junit4]   2> NOTE: reproduce with: ant test  
> -Dtestcase=TestSparseFixedBitDocIdSet -Dtests.method=testRamBytesUsed 
> -Dtests.seed=
> C1F3B881CB1C5E8A -Dtests.locale=es_BO -Dtests.timezone=America/Detroit 
> -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
>[junit4] FAILURE 0.06s J1 | TestSparseFixedBitDocIdSet.testRamBytesUsed <<<
>[junit4]> Throwable #1: java.lang.AssertionError: expected:<264> but 
> was:<256>
>[junit4]>at 
> __randomizedtesting.SeedInfo.seed([C1F3B881CB1C5E8A:3350AAC1016341DC]:0)
>[junit4]>at 
> org.apache.lucene.util.BaseDocIdSetTestCase.testRamBytesUsed(BaseDocIdSetTestCase.java:104)
>[junit4]>at java.lang.Thread.run(Thread.java:745)
> {noformat}
> To reproduce this failure, run inside core (with 64 bits JVM):
> {noformat}
> ant test -Dtestcase=TestSparseFixedBitDocIdSet 
> -Dtests.method=testRamBytesUsed -Dargs="-XX:ObjectAlignmentInBytes=16" 
> -Dtests.seed=C1F3B881CB1C5E8A
> {noformat}
> The default works:
> {noformat}
> ant test -Dtestcase=TestSparseFixedBitDocIdSet 
> -Dtests.method=testRamBytesUsed -Dargs="-XX:ObjectAlignmentInBytes=8" 
> -Dtests.seed=C1F3B881CB1C5E8A
> {noformat}
> I think we should randomly also specify the ObjectAlignmentInBytes for test 
> runs on Policeman Jenkins. Any Power of 2 is fine.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-6239) Remove RAMUsageEstimator Unsafe calls

[Uwe Schindler (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-6239?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Uwe Schindler updated LUCENE-6239:
--
Attachment: LUCENE-6239.patch

New patch with some documentation improvements and a testcase that checks if 
Hotspot VM is used and validates that everything is sane. This test would fail 
if you corrupt the stringified method names.

> Remove RAMUsageEstimator Unsafe calls
> -
>
> Key: LUCENE-6239
> URL: https://issues.apache.org/jira/browse/LUCENE-6239
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Robert Muir
> Attachments: LUCENE-6239.patch, LUCENE-6239.patch, LUCENE-6239.patch
>
>
> This is unnecessary risk. We should remove this stuff, it is not needed here. 
> We are a search engine, not a ram calculator.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-6241) don't filter subdirectories in listAll()

[Robert Muir (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-6241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14318184#comment-14318184
 ] 

Robert Muir commented on LUCENE-6241:
-

I'd love to, but how will ramdirs copy-ctor work?

Maybe it doesnt need to take Directory 'other' and can take FSDirectory 'other' 
?

> don't filter subdirectories in listAll()
> 
>
> Key: LUCENE-6241
> URL: https://issues.apache.org/jira/browse/LUCENE-6241
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Robert Muir
> Attachments: LUCENE-6241.patch, LUCENE-6241.patch
>
>
> The issue is, today this means listAll() is always slow, sometimes MUCH 
> slower, because it must do the fstat()-equivalent of each file to check if 
> its a directory to filter it out.
> When i benchmarked this on a fast filesystem, doing all these filesystem 
> metadata calls only makes listAll() 2.6x slower, but on a non-ssd, slower 
> i/o, it can be more than 60x slower.
> Lucene doesn't make subdirectories, so hiding these for abuse cases just 
> makes real use cases slower.
> To add insult to injury, most code (e.g. all of lucene except for where 
> RAMDir copies from an FSDir) does not actually care if extraneous files are 
> directories or not.
> Finally it sucks the name is listAll() when it is doing anything but that.
> I really hate to add a method here to deal with this abusive stuff, but I'd 
> rather add isDirectory(String) for the rare code that wants to filter out, 
> than just let stuff always be slow.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-6242) SparseFixedBitDocIdSet.ramBytesUsed() reports wrong size if alignment of JVM is not 8

[Dawid Weiss (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-6242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14318180#comment-14318180
 ] 

Dawid Weiss commented on LUCENE-6242:
-

Definitely the larger ones (> 32 from what I recall).

> SparseFixedBitDocIdSet.ramBytesUsed() reports wrong size if alignment of JVM 
> is not 8
> -
>
> Key: LUCENE-6242
> URL: https://issues.apache.org/jira/browse/LUCENE-6242
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Uwe Schindler
>
> There seems to be a bug in SparseFixedBitDocIdSet's ramBytesUsed. To me this 
> is a bit crazy implemented, so I have not yet found the issue. To me it looks 
> like some of the summing up breaks if the alignment of the JVM is not 8:
> {noformat}
>[junit4] Suite: org.apache.lucene.util.TestSparseFixedBitDocIdSet
>[junit4]   2> NOTE: reproduce with: ant test  
> -Dtestcase=TestSparseFixedBitDocIdSet -Dtests.method=testRamBytesUsed 
> -Dtests.seed=
> C1F3B881CB1C5E8A -Dtests.locale=es_BO -Dtests.timezone=America/Detroit 
> -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
>[junit4] FAILURE 0.06s J1 | TestSparseFixedBitDocIdSet.testRamBytesUsed <<<
>[junit4]> Throwable #1: java.lang.AssertionError: expected:<264> but 
> was:<256>
>[junit4]>at 
> __randomizedtesting.SeedInfo.seed([C1F3B881CB1C5E8A:3350AAC1016341DC]:0)
>[junit4]>at 
> org.apache.lucene.util.BaseDocIdSetTestCase.testRamBytesUsed(BaseDocIdSetTestCase.java:104)
>[junit4]>at java.lang.Thread.run(Thread.java:745)
> {noformat}
> To reproduce this failure, run inside core (with 64 bits JVM):
> {noformat}
> ant test -Dtestcase=TestSparseFixedBitDocIdSet 
> -Dtests.method=testRamBytesUsed -Dargs="-XX:ObjectAlignmentInBytes=16" 
> -Dtests.seed=C1F3B881CB1C5E8A
> {noformat}
> The default works:
> {noformat}
> ant test -Dtestcase=TestSparseFixedBitDocIdSet 
> -Dtests.method=testRamBytesUsed -Dargs="-XX:ObjectAlignmentInBytes=8" 
> -Dtests.seed=C1F3B881CB1C5E8A
> {noformat}
> I think we should randomly also specify the ObjectAlignmentInBytes for test 
> runs on Policeman Jenkins. Any Power of 2 is fine.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-6242) SparseFixedBitDocIdSet.ramBytesUsed() reports wrong size if alignment of JVM is not 8

[Uwe Schindler (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-6242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14318179#comment-14318179
 ] 

Uwe Schindler commented on LUCENE-6242:
---

Oha :-) I assume for large ones or also with stuff like 16?

At least we should fix this issue here. If we enable random alignments could be 
decided after some local tests. In any case I would enable this only very 
seldom with values 8 <= x <= 128.

> SparseFixedBitDocIdSet.ramBytesUsed() reports wrong size if alignment of JVM 
> is not 8
> -
>
> Key: LUCENE-6242
> URL: https://issues.apache.org/jira/browse/LUCENE-6242
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Uwe Schindler
>
> There seems to be a bug in SparseFixedBitDocIdSet's ramBytesUsed. To me this 
> is a bit crazy implemented, so I have not yet found the issue. To me it looks 
> like some of the summing up breaks if the alignment of the JVM is not 8:
> {noformat}
>[junit4] Suite: org.apache.lucene.util.TestSparseFixedBitDocIdSet
>[junit4]   2> NOTE: reproduce with: ant test  
> -Dtestcase=TestSparseFixedBitDocIdSet -Dtests.method=testRamBytesUsed 
> -Dtests.seed=
> C1F3B881CB1C5E8A -Dtests.locale=es_BO -Dtests.timezone=America/Detroit 
> -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
>[junit4] FAILURE 0.06s J1 | TestSparseFixedBitDocIdSet.testRamBytesUsed <<<
>[junit4]> Throwable #1: java.lang.AssertionError: expected:<264> but 
> was:<256>
>[junit4]>at 
> __randomizedtesting.SeedInfo.seed([C1F3B881CB1C5E8A:3350AAC1016341DC]:0)
>[junit4]>at 
> org.apache.lucene.util.BaseDocIdSetTestCase.testRamBytesUsed(BaseDocIdSetTestCase.java:104)
>[junit4]>at java.lang.Thread.run(Thread.java:745)
> {noformat}
> To reproduce this failure, run inside core (with 64 bits JVM):
> {noformat}
> ant test -Dtestcase=TestSparseFixedBitDocIdSet 
> -Dtests.method=testRamBytesUsed -Dargs="-XX:ObjectAlignmentInBytes=16" 
> -Dtests.seed=C1F3B881CB1C5E8A
> {noformat}
> The default works:
> {noformat}
> ant test -Dtestcase=TestSparseFixedBitDocIdSet 
> -Dtests.method=testRamBytesUsed -Dargs="-XX:ObjectAlignmentInBytes=8" 
> -Dtests.seed=C1F3B881CB1C5E8A
> {noformat}
> I think we should randomly also specify the ObjectAlignmentInBytes for test 
> runs on Policeman Jenkins. Any Power of 2 is fine.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-trunk-Windows (32bit/jdk1.8.0_31) - Build # 4481 - Still Failing!

[Policeman Jenkins Server]

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Windows/4481/
Java: 32bit/jdk1.8.0_31 -client -XX:+UseParallelGC

3 tests failed.
FAILED:  
junit.framework.TestSuite.org.apache.solr.handler.TestReplicationHandlerBackup

Error Message:
Could not remove the following files (in the order of attempts):
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.handler.TestReplicationHandlerBackup
 705E031ACF617A8D-001\solr-instance-002\collection1: 
java.nio.file.DirectoryNotEmptyException: 
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.handler.TestReplicationHandlerBackup
 705E031ACF617A8D-001\solr-instance-002\collection1
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.handler.TestReplicationHandlerBackup
 705E031ACF617A8D-001\solr-instance-002: 
java.nio.file.DirectoryNotEmptyException: 
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.handler.TestReplicationHandlerBackup
 705E031ACF617A8D-001\solr-instance-002
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.handler.TestReplicationHandlerBackup
 705E031ACF617A8D-001\solr-instance-001\collection1: 
java.nio.file.DirectoryNotEmptyException: 
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.handler.TestReplicationHandlerBackup
 705E031ACF617A8D-001\solr-instance-001\collection1
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.handler.TestReplicationHandlerBackup
 705E031ACF617A8D-001\solr-instance-001: 
java.nio.file.DirectoryNotEmptyException: 
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.handler.TestReplicationHandlerBackup
 705E031ACF617A8D-001\solr-instance-001
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.handler.TestReplicationHandlerBackup
 705E031ACF617A8D-001\solr-instance-001\collection1\data: 
java.nio.file.AccessDeniedException: 
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.handler.TestReplicationHandlerBackup
 705E031ACF617A8D-001\solr-instance-001\collection1\data
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.handler.TestReplicationHandlerBackup
 705E031ACF617A8D-001\solr-instance-001: 
java.nio.file.DirectoryNotEmptyException: 
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.handler.TestReplicationHandlerBackup
 705E031ACF617A8D-001\solr-instance-001
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.handler.TestReplicationHandlerBackup
 705E031ACF617A8D-001\solr-instance-002\collection1\data: 
java.nio.file.AccessDeniedException: 
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.handler.TestReplicationHandlerBackup
 705E031ACF617A8D-001\solr-instance-002\collection1\data
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.handler.TestReplicationHandlerBackup
 705E031ACF617A8D-001\solr-instance-002: 
java.nio.file.DirectoryNotEmptyException: 
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.handler.TestReplicationHandlerBackup
 705E031ACF617A8D-001\solr-instance-002
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.handler.TestReplicationHandlerBackup
 705E031ACF617A8D-001: java.nio.file.DirectoryNotEmptyException: 
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.handler.TestReplicationHandlerBackup
 705E031ACF617A8D-001 

Stack Trace:
java.io.IOException: Could not remove the following files (in the order of 
attempts):
   
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.handler.TestReplicationHandlerBackup
 705E031ACF617A8D-001\solr-instance-002\collection1: 
java.nio.file.DirectoryNotEmptyException: 
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.handler.TestReplicationHandlerBackup
 705E031ACF617A8D-001\solr-instance-002\collection1
   
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.handler.TestReplicationHandlerBackup
 705E031ACF617A8D-001\solr-instance-002: 
java.nio.file.DirectoryNotEmptyException: 
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.handler.TestReplicationHandlerBackup
 705E031ACF617A8D-001\solr-instance-002
   
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.handler.TestReplicationHandl

[jira] [Commented] (LUCENE-6242) SparseFixedBitDocIdSet.ramBytesUsed() reports wrong size if alignment of JVM is not 8

[Dawid Weiss (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-6242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14318170#comment-14318170
 ] 

Dawid Weiss commented on LUCENE-6242:
-

Just a note -- I was toying with alignments a while ago and I had random JVM 
crashes.

> SparseFixedBitDocIdSet.ramBytesUsed() reports wrong size if alignment of JVM 
> is not 8
> -
>
> Key: LUCENE-6242
> URL: https://issues.apache.org/jira/browse/LUCENE-6242
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Uwe Schindler
>
> There seems to be a bug in SparseFixedBitDocIdSet's ramBytesUsed. To me this 
> is a bit crazy implemented, so I have not yet found the issue. To me it looks 
> like some of the summing up breaks if the alignment of the JVM is not 8:
> {noformat}
>[junit4] Suite: org.apache.lucene.util.TestSparseFixedBitDocIdSet
>[junit4]   2> NOTE: reproduce with: ant test  
> -Dtestcase=TestSparseFixedBitDocIdSet -Dtests.method=testRamBytesUsed 
> -Dtests.seed=
> C1F3B881CB1C5E8A -Dtests.locale=es_BO -Dtests.timezone=America/Detroit 
> -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
>[junit4] FAILURE 0.06s J1 | TestSparseFixedBitDocIdSet.testRamBytesUsed <<<
>[junit4]> Throwable #1: java.lang.AssertionError: expected:<264> but 
> was:<256>
>[junit4]>at 
> __randomizedtesting.SeedInfo.seed([C1F3B881CB1C5E8A:3350AAC1016341DC]:0)
>[junit4]>at 
> org.apache.lucene.util.BaseDocIdSetTestCase.testRamBytesUsed(BaseDocIdSetTestCase.java:104)
>[junit4]>at java.lang.Thread.run(Thread.java:745)
> {noformat}
> To reproduce this failure, run inside core (with 64 bits JVM):
> {noformat}
> ant test -Dtestcase=TestSparseFixedBitDocIdSet 
> -Dtests.method=testRamBytesUsed -Dargs="-XX:ObjectAlignmentInBytes=16" 
> -Dtests.seed=C1F3B881CB1C5E8A
> {noformat}
> The default works:
> {noformat}
> ant test -Dtestcase=TestSparseFixedBitDocIdSet 
> -Dtests.method=testRamBytesUsed -Dargs="-XX:ObjectAlignmentInBytes=8" 
> -Dtests.seed=C1F3B881CB1C5E8A
> {noformat}
> I think we should randomly also specify the ObjectAlignmentInBytes for test 
> runs on Policeman Jenkins. Any Power of 2 is fine.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-6242) SparseFixedBitDocIdSet.ramBytesUsed() reports wrong size if alignment of JVM is not 8

[Uwe Schindler (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-6242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Uwe Schindler updated LUCENE-6242:
--
Description: 
There seems to be a bug in SparseFixedBitDocIdSet's ramBytesUsed. To me this is 
a bit crazy implemented, so I have not yet found the issue. To me it looks like 
some of the summing up breaks if the alignment of the JVM is not 8:

{noformat}
   [junit4] Suite: org.apache.lucene.util.TestSparseFixedBitDocIdSet
   [junit4]   2> NOTE: reproduce with: ant test  
-Dtestcase=TestSparseFixedBitDocIdSet -Dtests.method=testRamBytesUsed 
-Dtests.seed=
C1F3B881CB1C5E8A -Dtests.locale=es_BO -Dtests.timezone=America/Detroit 
-Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
   [junit4] FAILURE 0.06s J1 | TestSparseFixedBitDocIdSet.testRamBytesUsed <<<
   [junit4]> Throwable #1: java.lang.AssertionError: expected:<264> but 
was:<256>
   [junit4]>at 
__randomizedtesting.SeedInfo.seed([C1F3B881CB1C5E8A:3350AAC1016341DC]:0)
   [junit4]>at 
org.apache.lucene.util.BaseDocIdSetTestCase.testRamBytesUsed(BaseDocIdSetTestCase.java:104)
   [junit4]>at java.lang.Thread.run(Thread.java:745)
{noformat}

To reproduce this failure, run inside core (with 64 bits JVM):

{noformat}
ant test -Dtestcase=TestSparseFixedBitDocIdSet -Dtests.method=testRamBytesUsed 
-Dargs="-XX:ObjectAlignmentInBytes=16" -Dtests.seed=C1F3B881CB1C5E8A
{noformat}

The default works:

{noformat}
ant test -Dtestcase=TestSparseFixedBitDocIdSet -Dtests.method=testRamBytesUsed 
-Dargs="-XX:ObjectAlignmentInBytes=8" -Dtests.seed=C1F3B881CB1C5E8A
{noformat}

I think we should randomly also specify the ObjectAlignmentInBytes for test 
runs on Policeman Jenkins. Any Power of 2 is fine.

  was:
There seems to be a bug in SparseFixedBitDocIdSet's ramBytesUsed. To me this is 
a bit crazy implemented, so I have not yet found the issue. To me it looks like 
some of the summing up breaks if the alignment of the JVM is not 8:

{noformat}
   [junit4] Suite: org.apache.lucene.util.TestSparseFixedBitDocIdSet
   [junit4]   2> NOTE: reproduce with: ant test  
-Dtestcase=TestSparseFixedBitDocIdSet -Dtests.method=testRamBytesUsed 
-Dtests.seed=
C1F3B881CB1C5E8A -Dtests.locale=es_BO -Dtests.timezone=America/Detroit 
-Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
   [junit4] FAILURE 0.06s J1 | TestSparseFixedBitDocIdSet.testRamBytesUsed <<<
   [junit4]> Throwable #1: java.lang.AssertionError: expected:<264> but 
was:<256>
   [junit4]>at 
__randomizedtesting.SeedInfo.seed([C1F3B881CB1C5E8A:3350AAC1016341DC]:0)
   [junit4]>at 
org.apache.lucene.util.BaseDocIdSetTestCase.testRamBytesUsed(BaseDocIdSetTestCase.java:104)
   [junit4]>at java.lang.Thread.run(Thread.java:745)
{noformat}

To reproduce this failure, run inside core:

{noformat}
ant test -Dtestcase=TestSparseFixedBitDocIdSet -Dtests.method=testRamBytesUsed 
-Dargs="-XX:ObjectAlignmentInBytes=16" -Dtests.seed=C1F3B881CB1C5E8A
{noformat}

The default works:

{noformat}
ant test -Dtestcase=TestSparseFixedBitDocIdSet -Dtests.method=testRamBytesUsed 
-Dargs="-XX:ObjectAlignmentInBytes=8" -Dtests.seed=C1F3B881CB1C5E8A
{noformat}

I think we should randomly also specify the ObjectAlignmentInBytes for test 
runs on Policeman Jenkins. Any Power of 2 is fine.


> SparseFixedBitDocIdSet.ramBytesUsed() reports wrong size if alignment of JVM 
> is not 8
> -
>
> Key: LUCENE-6242
> URL: https://issues.apache.org/jira/browse/LUCENE-6242
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Uwe Schindler
>
> There seems to be a bug in SparseFixedBitDocIdSet's ramBytesUsed. To me this 
> is a bit crazy implemented, so I have not yet found the issue. To me it looks 
> like some of the summing up breaks if the alignment of the JVM is not 8:
> {noformat}
>[junit4] Suite: org.apache.lucene.util.TestSparseFixedBitDocIdSet
>[junit4]   2> NOTE: reproduce with: ant test  
> -Dtestcase=TestSparseFixedBitDocIdSet -Dtests.method=testRamBytesUsed 
> -Dtests.seed=
> C1F3B881CB1C5E8A -Dtests.locale=es_BO -Dtests.timezone=America/Detroit 
> -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
>[junit4] FAILURE 0.06s J1 | TestSparseFixedBitDocIdSet.testRamBytesUsed <<<
>[junit4]> Throwable #1: java.lang.AssertionError: expected:<264> but 
> was:<256>
>[junit4]>at 
> __randomizedtesting.SeedInfo.seed([C1F3B881CB1C5E8A:3350AAC1016341DC]:0)
>[junit4]>at 
> org.apache.lucene.util.BaseDocIdSetTestCase.testRamBytesUsed(BaseDocIdSetTestCase.java:104)
>[junit4]>at java.lang.Thread.run(Thread.java:745)
> {noformat}
> To reproduce this failure, run inside core (with 64 bits JVM):
> {noformat}
> ant test -Dtestcase=TestSparseFixedBitDocIdSet 
> -Dtests.method=

[jira] [Created] (LUCENE-6242) SparseFixedBitDocIdSet.ramBytesUsed() reports wrong size if alignment of JVM is not 8

[Uwe Schindler (JIRA)]

Uwe Schindler created LUCENE-6242:
-

 Summary: SparseFixedBitDocIdSet.ramBytesUsed() reports wrong size 
if alignment of JVM is not 8
 Key: LUCENE-6242
 URL: https://issues.apache.org/jira/browse/LUCENE-6242
 Project: Lucene - Core
  Issue Type: Bug
Reporter: Uwe Schindler


There seems to be a bug in SparseFixedBitDocIdSet's ramBytesUsed. To me this is 
a bit crazy implemented, so I have not yet found the issue. To me it looks like 
some of the summing up breaks if the alignment of the JVM is not 8:

{noformat}
   [junit4] Suite: org.apache.lucene.util.TestSparseFixedBitDocIdSet
   [junit4]   2> NOTE: reproduce with: ant test  
-Dtestcase=TestSparseFixedBitDocIdSet -Dtests.method=testRamBytesUsed 
-Dtests.seed=
C1F3B881CB1C5E8A -Dtests.locale=es_BO -Dtests.timezone=America/Detroit 
-Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
   [junit4] FAILURE 0.06s J1 | TestSparseFixedBitDocIdSet.testRamBytesUsed <<<
   [junit4]> Throwable #1: java.lang.AssertionError: expected:<264> but 
was:<256>
   [junit4]>at 
__randomizedtesting.SeedInfo.seed([C1F3B881CB1C5E8A:3350AAC1016341DC]:0)
   [junit4]>at 
org.apache.lucene.util.BaseDocIdSetTestCase.testRamBytesUsed(BaseDocIdSetTestCase.java:104)
   [junit4]>at java.lang.Thread.run(Thread.java:745)
{noformat}

To reproduce this failure, run inside core:

{noformat}
ant test -Dtestcase=TestSparseFixedBitDocIdSet -Dtests.method=testRamBytesUsed 
-Dargs="-XX:ObjectAlignmentInBytes=16" -Dtests.seed=C1F3B881CB1C5E8A
{noformat}

The default works:

{noformat}
ant test -Dtestcase=TestSparseFixedBitDocIdSet -Dtests.method=testRamBytesUsed 
-Dargs="-XX:ObjectAlignmentInBytes=8" -Dtests.seed=C1F3B881CB1C5E8A
{noformat}

I think we should randomly also specify the ObjectAlignmentInBytes for test 
runs on Policeman Jenkins. Any Power of 2 is fine.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Comment Edited] (SOLR-6997) Support distributed 'facet on same field different ways'

[Pablo Queixalos (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-6997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14315777#comment-14315777
 ] 

Pablo Queixalos edited comment on SOLR-6997 at 2/12/15 1:15 PM:


Also, as solr/lucene allows naming fields using hyphen character, attached 
patch for identifier parsing in QueryParsing.java, which is basically :
{code}
-  if (!Character.isJavaIdentifierPart(ch) && ch != '.') {
+  if (!Character.isJavaIdentifierPart(ch) && ch != '.' && ch != '-' ) {
{code}


was (Author: pqueixalos):
Also, as solr/lucene allows naming fields using dash character, attached patch 
for identifier parsing in QueryParsing.java, which is basically :
{code}
-  if (!Character.isJavaIdentifierPart(ch) && ch != '.') {
+  if (!Character.isJavaIdentifierPart(ch) && ch != '.' && ch != '-' ) {
{code}

> Support distributed 'facet on same field different ways'
> 
>
> Key: SOLR-6997
> URL: https://issues.apache.org/jira/browse/SOLR-6997
> Project: Solr
>  Issue Type: Bug
>  Components: faceting
>Affects Versions: 4.10.3
>Reporter: Pablo Queixalos
>Priority: Minor
>  Labels: distributed_search
> Fix For: 4.10.4, 5.0
>
> Attachments: 
> 0001-SEA-1284-field-facet-local-param-parsing-fails-on-fi.patch, 
> 0001-honor-facet-field-local-params.patch
>
>
> SOLR-1351 brought the ability to facet on the same field differently using 
> local parameters but distributed process ({{FacetComponent::FieldFacet}}) 
> does not honor local params from _facet string_.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org