[jira] [Commented] (SOLR-10130) Serious performance degradation in Solr 6.4.1 due to the new metrics collection

[Ere Maijala (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15867402#comment-15867402
 ] 

Ere Maijala commented on SOLR-10130:


I don't have proper benchmarks at hand, but I can support others' findings 
about the serious query performance degradation. I suppose it's easily 
overlooked when testing with light concurrency, but with enough concurrent 
queries being served it gets CPU-heavy. We use queries with a lot of filters so 
that may play a role too. I'll see if I came come up with a reproducible-enough 
test results from our actual queries.

> Serious performance degradation in Solr 6.4.1 due to the new metrics 
> collection
> ---
>
> Key: SOLR-10130
> URL: https://issues.apache.org/jira/browse/SOLR-10130
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: metrics
>Affects Versions: 6.4.1
> Environment: Centos 7, OpenJDK 1.8.0 update 111
>Reporter: Ere Maijala
>Assignee: Andrzej Bialecki 
>Priority: Blocker
>  Labels: perfomance
> Fix For: master (7.0), 6.4.2
>
> Attachments: SOLR-10130.patch, solr-8983-console-f1.log
>
>
> We've stumbled on serious performance issues after upgrading to Solr 6.4.1. 
> Looks like the new metrics collection system in MetricsDirectoryFactory is 
> causing a major slowdown. This happens with an index configuration that, as 
> far as I can see, has no metrics specific configuration and uses 
> luceneMatchVersion 5.5.0. In practice a moderate load will completely bog 
> down the server with Solr threads constantly using up all CPU (600% on 6 core 
> machine) capacity with a load that normally  where we normally see an average 
> load of < 50%.
> I took stack traces (I'll attach them) and noticed that the threads are 
> spending time in com.codahale.metrics.Meter.mark. I tested building Solr 
> 6.4.1 with the metrics collection disabled in MetricsDirectoryFactory getByte 
> and getBytes methods and was unable to reproduce the issue.
> As far as I can see there are several issues:
> 1. Collecting metrics on every single byte read is slow.
> 2. Having it enabled by default is not a good idea.
> 3. The comment "enable coarse-grained metrics by default" at 
> https://github.com/apache/lucene-solr/blob/branch_6x/solr/core/src/java/org/apache/solr/update/SolrIndexConfig.java#L104
>  implies that only coarse-grained metrics should be enabled by default, and 
> this contradicts with collecting metrics on every single byte read.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10117) Big docs and the DocumentCache; umbrella issue

[David Smiley (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15867396#comment-15867396
 ] 

David Smiley commented on SOLR-10117:
-

Another idea that I'm starting to like even more I think about it is to put 
large fields into BinaryDocValues, with compression (either at DocValuesFormat 
(codec) layer, or at Solr layer).  For very large fields, I think column stored 
(hence docValues) actually makes more sense than the stored field codec (row 
stored).  Then at the Solr layer we add docValues support to TextField (as 
BinaryDocValues), and then also at the Solr layer enable 
SolrIndexSearcher.doc() to see {{useDocValuesAsStored}} fields thus enabling 
highlighting to see it.  I wish I had thought of this earlier.

> Big docs and the DocumentCache; umbrella issue
> --
>
> Key: SOLR-10117
> URL: https://issues.apache.org/jira/browse/SOLR-10117
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: David Smiley
>Assignee: David Smiley
> Attachments: SOLR_10117_large_fields.patch
>
>
> This is an umbrella issue for improved handling of large documents (large 
> stored fields), generally related to the DocumentCache or SolrIndexSearcher's 
> doc() methods.  Highlighting is affected as it's the primary consumer of this 
> data.  "Large" here is multi-megabyte, especially tens even hundreds of 
> megabytes. We'd like to support such users without forcing them to choose 
> between no DocumentCache (bad performance), or having one but hitting OOM due 
> to massive Strings winding up in there.  I've contemplated this for longer 
> than I'd like to admit and it's a complicated issue with difference concerns 
> to balance.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: First stumble

[Toke Eskildsen]

On Wed, 2017-02-15 at 12:35 +0530, Ishan Chattopadhyaya wrote:

> I see that your name has been added to
> https://lucene.apache.org/whoweare.html. Maybe, you could slot it
> into the right place as per alphabetical order of last name.

Of course. Poor form to miss that.

I wonder how the page got published? I never got past the "Publish
lucene site"-page and my current sort-correction is still in staging.
Maybe someone else OK'ed the change?


Thank you,
Toke Eskildsen, Danish Royal Library

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: First stumble

[Ishan Chattopadhyaya]

Hi Toke,
I see that your name has been added to
https://lucene.apache.org/whoweare.html. Maybe, you could slot it into the
right place as per alphabetical order of last name.

Welcome and many congrats!
Regards,
Ishan

On Wed, Feb 15, 2017 at 12:14 PM, Toke Eskildsen  wrote:

> That did not take long...
>
> The initiation rite of adding my name to the committers list went well
> until it was time to publish. The Publish lucene site at
> https://cms.apache.org/lucene/publish
> shows nothing under "Authors:" and when I press "View Diff", the
> browser waits until I close the tab. I tried it three times yesterday
> and the same this morning. Sometimes I did not even get the main page.
>
> I guess I should contact the apache-intra-team? Or is this some known
> problem with an also known workaround? I'll read around and see if I
> can find some information.
>
> - Toke
>
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org
>
>

[JENKINS] Lucene-Solr-master-Solaris (64bit/jdk1.8.0) - Build # 1133 - Unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Solaris/1133/
Java: 64bit/jdk1.8.0 -XX:+UseCompressedOops -XX:+UseParallelGC

1 tests failed.
FAILED:  
org.apache.solr.cloud.CdcrBootstrapTest.testBootstrapWithContinousIndexingOnSourceCluster

Error Message:
Document mismatch on target after sync expected:<2000> but was:<1100>

Stack Trace:
java.lang.AssertionError: Document mismatch on target after sync 
expected:<2000> but was:<1100>
at 
__randomizedtesting.SeedInfo.seed([43B83B17D56C633:D07EC8E89A0075C8]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at 
org.apache.solr.cloud.CdcrBootstrapTest.testBootstrapWithContinousIndexingOnSourceCluster(CdcrBootstrapTest.java:309)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 10987 

First stumble

[Toke Eskildsen]

That did not take long...

The initiation rite of adding my name to the committers list went well
until it was time to publish. The Publish lucene site at
https://cms.apache.org/lucene/publish
shows nothing under "Authors:" and when I press "View Diff", the
browser waits until I close the tab. I tried it three times yesterday
and the same this morning. Sometimes I did not even get the main page.

I guess I should contact the apache-intra-team? Or is this some known
problem with an also known workaround? I'll read around and see if I
can find some information.

- Toke


-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS-EA] Lucene-Solr-6.x-Linux (32bit/jdk-9-ea+155) - Build # 2858 - Still Unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-6.x-Linux/2858/
Java: 32bit/jdk-9-ea+155 -client -XX:+UseConcMarkSweepGC

1 tests failed.
FAILED:  org.apache.solr.handler.admin.TestApiFramework.testFramework

Error Message:


Stack Trace:
java.lang.ExceptionInInitializerError
at 
__randomizedtesting.SeedInfo.seed([E4131904D330B839:F365D323D5E45404]:0)
at 
net.sf.cglib.core.KeyFactory$Generator.generateClass(KeyFactory.java:166)
at 
net.sf.cglib.core.DefaultGeneratorStrategy.generate(DefaultGeneratorStrategy.java:25)
at 
net.sf.cglib.core.AbstractClassGenerator.create(AbstractClassGenerator.java:216)
at net.sf.cglib.core.KeyFactory$Generator.create(KeyFactory.java:144)
at net.sf.cglib.core.KeyFactory.create(KeyFactory.java:116)
at net.sf.cglib.core.KeyFactory.create(KeyFactory.java:108)
at net.sf.cglib.core.KeyFactory.create(KeyFactory.java:104)
at net.sf.cglib.proxy.Enhancer.(Enhancer.java:69)
at 
org.easymock.internal.ClassProxyFactory.createEnhancer(ClassProxyFactory.java:259)
at 
org.easymock.internal.ClassProxyFactory.createProxy(ClassProxyFactory.java:174)
at org.easymock.internal.MocksControl.createMock(MocksControl.java:60)
at org.easymock.EasyMock.createMock(EasyMock.java:104)
at 
org.apache.solr.handler.admin.TestCoreAdminApis.getCoreContainerMock(TestCoreAdminApis.java:83)
at 
org.apache.solr.handler.admin.TestApiFramework.testFramework(TestApiFramework.java:59)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:543)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 

[JENKINS] Lucene-Solr-SmokeRelease-5.5 - Build # 14 - Still Failing

[Apache Jenkins Server]

Build: https://builds.apache.org/job/Lucene-Solr-SmokeRelease-5.5/14/

No tests ran.

Build Log:
[...truncated 39721 lines...]
prepare-release-no-sign:
[mkdir] Created dir: 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-5.5/lucene/build/smokeTestRelease/dist
 [copy] Copying 461 files to 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-5.5/lucene/build/smokeTestRelease/dist/lucene
 [copy] Copying 245 files to 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-5.5/lucene/build/smokeTestRelease/dist/solr
   [smoker] Java 1.7 JAVA_HOME=/home/jenkins/tools/java/latest1.7
   [smoker] Java 1.8 JAVA_HOME=/home/jenkins/tools/java/latest1.8
   [smoker] NOTE: output encoding is UTF-8
   [smoker] 
   [smoker] Load release URL 
"file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-5.5/lucene/build/smokeTestRelease/dist/"...
   [smoker] 
   [smoker] Test Lucene...
   [smoker]   test basics...
   [smoker]   get KEYS
   [smoker] 0.2 MB in 0.01 sec (29.8 MB/sec)
   [smoker]   check changes HTML...
   [smoker]   download lucene-5.5.4-src.tgz...
   [smoker] 28.8 MB in 0.02 sec (1186.9 MB/sec)
   [smoker] verify md5/sha1 digests
   [smoker]   download lucene-5.5.4.tgz...
   [smoker] 63.3 MB in 0.05 sec (1168.3 MB/sec)
   [smoker] verify md5/sha1 digests
   [smoker]   download lucene-5.5.4.zip...
   [smoker] 73.2 MB in 0.06 sec (1179.0 MB/sec)
   [smoker] verify md5/sha1 digests
   [smoker]   unpack lucene-5.5.4.tgz...
   [smoker] verify JAR metadata/identity/no javax.* or java.* classes...
   [smoker] test demo with 1.7...
   [smoker]   got 6190 hits for query "lucene"
   [smoker] checkindex with 1.7...
   [smoker] test demo with 1.8...
   [smoker]   got 6190 hits for query "lucene"
   [smoker] checkindex with 1.8...
   [smoker] check Lucene's javadoc JAR
   [smoker]   unpack lucene-5.5.4.zip...
   [smoker] verify JAR metadata/identity/no javax.* or java.* classes...
   [smoker] test demo with 1.7...
   [smoker]   got 6190 hits for query "lucene"
   [smoker] checkindex with 1.7...
   [smoker] test demo with 1.8...
   [smoker]   got 6190 hits for query "lucene"
   [smoker] checkindex with 1.8...
   [smoker] check Lucene's javadoc JAR
   [smoker]   unpack lucene-5.5.4-src.tgz...
   [smoker] make sure no JARs/WARs in src dist...
   [smoker] run "ant validate"
   [smoker] run tests w/ Java 7 and testArgs='-Dtests.slow=false'...
   [smoker] test demo with 1.7...
   [smoker]   got 221 hits for query "lucene"
   [smoker] checkindex with 1.7...
   [smoker] generate javadocs w/ Java 7...
   [smoker] 
   [smoker] Crawl/parse...
   [smoker] 
   [smoker] Verify...
   [smoker] run tests w/ Java 8 and testArgs='-Dtests.slow=false'...
   [smoker] test demo with 1.8...
   [smoker]   got 221 hits for query "lucene"
   [smoker] checkindex with 1.8...
   [smoker] generate javadocs w/ Java 8...
   [smoker] 
   [smoker] Crawl/parse...
   [smoker] 
   [smoker] Verify...
   [smoker]   confirm all releases have coverage in TestBackwardsCompatibility
   [smoker] find all past Lucene releases...
   [smoker] run TestBackwardsCompatibility..
   [smoker]   Backcompat testing not required for release 6.0.1 because 
it's not less than 5.5.4
   [smoker]   Backcompat testing not required for release 6.0.0 because 
it's not less than 5.5.4
   [smoker]   Backcompat testing not required for release 6.4.1 because 
it's not less than 5.5.4
   [smoker]   Backcompat testing not required for release 6.4.0 because 
it's not less than 5.5.4
   [smoker]   Backcompat testing not required for release 5.5.4 because 
it's not less than 5.5.4
   [smoker]   Backcompat testing not required for release 6.1.0 because 
it's not less than 5.5.4
   [smoker]   Backcompat testing not required for release 6.2.1 because 
it's not less than 5.5.4
   [smoker]   Backcompat testing not required for release 6.2.0 because 
it's not less than 5.5.4
   [smoker]   Backcompat testing not required for release 6.3.0 because 
it's not less than 5.5.4
   [smoker] success!
   [smoker] 
   [smoker] Test Solr...
   [smoker]   test basics...
   [smoker]   get KEYS
   [smoker] 0.2 MB in 0.00 sec (109.2 MB/sec)
   [smoker]   check changes HTML...
   [smoker]   download solr-5.5.4-src.tgz...
   [smoker] 37.7 MB in 0.04 sec (1067.3 MB/sec)
   [smoker] verify md5/sha1 digests
   [smoker]   download solr-5.5.4.tgz...
   [smoker] 130.4 MB in 0.12 sec (1075.2 MB/sec)
   [smoker] verify md5/sha1 digests
   [smoker]   download solr-5.5.4.zip...
   [smoker] 138.0 MB in 0.13 sec (1098.9 MB/sec)
   [smoker] verify md5/sha1 digests
   [smoker]   unpack solr-5.5.4.tgz...
   [smoker] verify JAR metadata/identity/no javax.* or java.* classes...
   [smoker] unpack lucene-5.5.4.tgz...
   [smoker]   **WARNING**: skipping check of 

[JENKINS] Lucene-Solr-6.x-MacOSX (64bit/jdk1.8.0) - Build # 698 - Failure!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-6.x-MacOSX/698/
Java: 64bit/jdk1.8.0 -XX:-UseCompressedOops -XX:+UseSerialGC

No tests ran.

Build Log:
[...truncated 17 lines...]
FATAL: Exception caught during execution of reset command. Cannot lock 
/Users/jenkins/workspace/Lucene-Solr-6.x-MacOSX/.git/index
org.eclipse.jgit.errors.LockFailedException: Cannot lock 
/Users/jenkins/workspace/Lucene-Solr-6.x-MacOSX/.git/index
at org.eclipse.jgit.dircache.DirCache.lock(DirCache.java:250)
at org.eclipse.jgit.dircache.DirCache.lock(DirCache.java:327)
at org.eclipse.jgit.dircache.DirCache.lock(DirCache.java:293)
at org.eclipse.jgit.lib.Repository.lockDirCache(Repository.java:1183)
at 
org.eclipse.jgit.api.ResetCommand.checkoutIndex(ResetCommand.java:404)
at org.eclipse.jgit.api.ResetCommand.call(ResetCommand.java:207)
at 
org.jenkinsci.plugins.gitclient.JGitAPIImpl.clean(JGitAPIImpl.java:1189)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
hudson.remoting.RemoteInvocationHandler$RPCRequest.perform(RemoteInvocationHandler.java:894)
at 
hudson.remoting.RemoteInvocationHandler$RPCRequest.call(RemoteInvocationHandler.java:869)
at 
hudson.remoting.RemoteInvocationHandler$RPCRequest.call(RemoteInvocationHandler.java:828)
at hudson.remoting.UserRequest.perform(UserRequest.java:153)
at hudson.remoting.UserRequest.perform(UserRequest.java:50)
at hudson.remoting.Request$2.run(Request.java:336)
at 
hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:68)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused: org.eclipse.jgit.api.errors.JGitInternalException: Exception caught 
during execution of reset command. Cannot lock 
/Users/jenkins/workspace/Lucene-Solr-6.x-MacOSX/.git/index
at org.eclipse.jgit.api.ResetCommand.call(ResetCommand.java:234)
at 
org.jenkinsci.plugins.gitclient.JGitAPIImpl.clean(JGitAPIImpl.java:1189)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
hudson.remoting.RemoteInvocationHandler$RPCRequest.perform(RemoteInvocationHandler.java:894)
at 
hudson.remoting.RemoteInvocationHandler$RPCRequest.call(RemoteInvocationHandler.java:869)
at 
hudson.remoting.RemoteInvocationHandler$RPCRequest.call(RemoteInvocationHandler.java:828)
at hudson.remoting.UserRequest.perform(UserRequest.java:153)
at hudson.remoting.UserRequest.perform(UserRequest.java:50)
at hudson.remoting.Request$2.run(Request.java:336)
at 
hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:68)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
at ..remote call to MacOSX VBOX(Native Method)
at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1537)
at hudson.remoting.UserResponse.retrieve(UserRequest.java:253)
at hudson.remoting.Channel.call(Channel.java:822)
at 
hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:256)
at com.sun.proxy.$Proxy53.clean(Unknown Source)
at 
org.jenkinsci.plugins.gitclient.RemoteGitImpl.clean(RemoteGitImpl.java:449)
at 
hudson.plugins.git.extensions.impl.CleanBeforeCheckout.decorateFetchCommand(CleanBeforeCheckout.java:32)
at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:802)
at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1066)
at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1097)
at hudson.scm.SCM.checkout(SCM.java:496)
at hudson.model.AbstractProject.checkout(AbstractProject.java:1278)
at 
hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:604)
at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
at 
hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:529)
at 

[JENKINS] Lucene-Solr-master-MacOSX (64bit/jdk1.8.0) - Build # 3833 - Still Unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-MacOSX/3833/
Java: 64bit/jdk1.8.0 -XX:+UseCompressedOops -XX:+UseSerialGC

11 tests failed.
FAILED:  org.apache.solr.cloud.CustomCollectionTest.testCreateShardRepFactor

Error Message:
Error from server at https://127.0.0.1:54548/solr: Could not find collection : 
testCreateShardRepFactor

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error 
from server at https://127.0.0.1:54548/solr: Could not find collection : 
testCreateShardRepFactor
at 
__randomizedtesting.SeedInfo.seed([BA3F445A78D6:4ED1CEDE608C9D46]:0)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:627)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:279)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:268)
at 
org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:439)
at 
org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:391)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1358)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:1109)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:1042)
at 
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:160)
at 
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:177)
at 
org.apache.solr.cloud.CustomCollectionTest.testCreateShardRepFactor(CustomCollectionTest.java:190)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 

[JENKINS-EA] Lucene-Solr-master-Linux (32bit/jdk-9-ea+155) - Build # 18966 - Unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/18966/
Java: 32bit/jdk-9-ea+155 -server -XX:+UseConcMarkSweepGC

1 tests failed.
FAILED:  org.apache.solr.handler.admin.TestApiFramework.testFramework

Error Message:


Stack Trace:
java.lang.ExceptionInInitializerError
at 
__randomizedtesting.SeedInfo.seed([B4937BB91AC59930:A3E5B19E1C11750D]:0)
at 
net.sf.cglib.core.KeyFactory$Generator.generateClass(KeyFactory.java:166)
at 
net.sf.cglib.core.DefaultGeneratorStrategy.generate(DefaultGeneratorStrategy.java:25)
at 
net.sf.cglib.core.AbstractClassGenerator.create(AbstractClassGenerator.java:216)
at net.sf.cglib.core.KeyFactory$Generator.create(KeyFactory.java:144)
at net.sf.cglib.core.KeyFactory.create(KeyFactory.java:116)
at net.sf.cglib.core.KeyFactory.create(KeyFactory.java:108)
at net.sf.cglib.core.KeyFactory.create(KeyFactory.java:104)
at net.sf.cglib.proxy.Enhancer.(Enhancer.java:69)
at 
org.easymock.internal.ClassProxyFactory.createEnhancer(ClassProxyFactory.java:259)
at 
org.easymock.internal.ClassProxyFactory.createProxy(ClassProxyFactory.java:174)
at org.easymock.internal.MocksControl.createMock(MocksControl.java:60)
at org.easymock.EasyMock.createMock(EasyMock.java:104)
at 
org.apache.solr.handler.admin.TestCoreAdminApis.getCoreContainerMock(TestCoreAdminApis.java:83)
at 
org.apache.solr.handler.admin.TestApiFramework.testFramework(TestApiFramework.java:59)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:543)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 

[jira] [Commented] (SOLR-9530) Add an Atomic Update Processor

[Ishan Chattopadhyaya (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15867251#comment-15867251
 ] 

Ishan Chattopadhyaya commented on SOLR-9530:


To solve the problem of multiple clients concurrently updating the same 
document, I think the URP should add the current version to the updated 
document. If two or more such documents hit DUP with the same version on 
update, all except the first one will fail with a version conflict and the 
client could retry that document. Only doing so, i.e. leveraging the optimistic 
concurrency for atomic updates, would make this URP a truly "atomic" update 
processor.

> Add an Atomic Update Processor 
> ---
>
> Key: SOLR-9530
> URL: https://issues.apache.org/jira/browse/SOLR-9530
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Varun Thacker
> Attachments: SOLR-9530.patch
>
>
> I'd like to explore the idea of adding a new update processor to help ingest 
> partial updates.
> Example use-case - There are two datasets with a common id field. How can I 
> merge both of them at index time?
> Proposed Solution: 
> {code}
> 
>   
> add
>   
>   
>   
> 
> {code}
> So the first JSON dump could be ingested against 
> {{http://localhost:8983/solr/gettingstarted/update/json}}
> And then the second JSON could be ingested against
> {{http://localhost:8983/solr/gettingstarted/update/json?processor=atomic}}
> The Atomic Update Processor could support all the atomic update operations 
> currently supported.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-10139) ShardSplitTest needs to be hardened.

[Mark Miller (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-10139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mark Miller updated SOLR-10139:
---
Attachment: logs.tar.gz

> ShardSplitTest needs to be hardened.
> 
>
> Key: SOLR-10139
> URL: https://issues.apache.org/jira/browse/SOLR-10139
> Project: Solr
>  Issue Type: Test
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Mark Miller
> Attachments: logs.tar.gz
>
>
> {noformat}
>[junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=ShardSplitTest 
> -Dtests.method=testSplitAfterFailedSplit -Dtests.seed=4D4F48964A9DD277 
> -Dtests.slow=true -Dtests.locale=zh-CN -Dtests.timezone=Australia/Queensland 
> -Dtests.asserts=true -Dtests.file.encoding=UTF-8
>[junit4] FAILURE 35.7s | ShardSplitTest.testSplitAfterFailedSplit <<<
>[junit4]> Throwable #1: java.lang.AssertionError: expected:<1> but 
> was:<2>
>[junit4]>  at 
> __randomizedtesting.SeedInfo.seed([4D4F48964A9DD277:B402DB3976E89FFD]:0)
>[junit4]>  at 
> org.apache.solr.cloud.ShardSplitTest.testSplitAfterFailedSplit(ShardSplitTest.java:279)
>[junit4]>  at 
> org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:985)
>[junit4]>  at 
> org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:960)
>[junit4]>  at java.lang.Thread.run(Thread.java:745)
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-10139) ShardSplitTest needs to be hardened.

[Mark Miller (JIRA)]

Mark Miller created SOLR-10139:
--

 Summary: ShardSplitTest needs to be hardened.
 Key: SOLR-10139
 URL: https://issues.apache.org/jira/browse/SOLR-10139
 Project: Solr
  Issue Type: Test
  Security Level: Public (Default Security Level. Issues are Public)
Reporter: Mark Miller


{noformat}
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=ShardSplitTest 
-Dtests.method=testSplitAfterFailedSplit -Dtests.seed=4D4F48964A9DD277 
-Dtests.slow=true -Dtests.locale=zh-CN -Dtests.timezone=Australia/Queensland 
-Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] FAILURE 35.7s | ShardSplitTest.testSplitAfterFailedSplit <<<
   [junit4]> Throwable #1: java.lang.AssertionError: expected:<1> but 
was:<2>
   [junit4]>at 
__randomizedtesting.SeedInfo.seed([4D4F48964A9DD277:B402DB3976E89FFD]:0)
   [junit4]>at 
org.apache.solr.cloud.ShardSplitTest.testSplitAfterFailedSplit(ShardSplitTest.java:279)
   [junit4]>at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:985)
   [junit4]>at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:960)
   [junit4]>at java.lang.Thread.run(Thread.java:745)
{noformat}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10068) The Nightly test SharedFSAutoReplicaFailoverTest appears to be too fragile.

[Mark Miller (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15867188#comment-15867188
 ] 

Mark Miller commented on SOLR-10068:


One issue with this test is that we need to lower the hdfs block cache size.

Another issue I'm still trying to figure out. I see a replica for shard 2 
recover and register as active but then we fail as a search keeps saying that 
same shard has no one serving it.

> The Nightly test SharedFSAutoReplicaFailoverTest appears to be too fragile.
> ---
>
> Key: SOLR-10068
> URL: https://issues.apache.org/jira/browse/SOLR-10068
> Project: Solr
>  Issue Type: Test
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Mark Miller
>Assignee: Mark Miller
>
> SharedFSAutoReplicaFailoverTest 37.00% screwy 30.00 273.97 @Nightly



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9530) Add an Atomic Update Processor

[AMRIT SARKAR (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15867163#comment-15867163
 ] 

AMRIT SARKAR commented on SOLR-9530:


Meanwhile as [~noble.paul] mentioned, if two threads try to update the same 
SolrInputDocument via AtomicUpdateProcessor, it may fail or give abrupt 
results. Allow me some time to review this, test it out and make thread-safe if 
deem necessary. Thank you.

> Add an Atomic Update Processor 
> ---
>
> Key: SOLR-9530
> URL: https://issues.apache.org/jira/browse/SOLR-9530
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Varun Thacker
> Attachments: SOLR-9530.patch
>
>
> I'd like to explore the idea of adding a new update processor to help ingest 
> partial updates.
> Example use-case - There are two datasets with a common id field. How can I 
> merge both of them at index time?
> Proposed Solution: 
> {code}
> 
>   
> add
>   
>   
>   
> 
> {code}
> So the first JSON dump could be ingested against 
> {{http://localhost:8983/solr/gettingstarted/update/json}}
> And then the second JSON could be ingested against
> {{http://localhost:8983/solr/gettingstarted/update/json?processor=atomic}}
> The Atomic Update Processor could support all the atomic update operations 
> currently supported.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-10136) TestReqParamsAPI regularly fails on Policeman Jenkins

[Mark Miller (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-10136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mark Miller updated SOLR-10136:
---
Priority: Major  (was: Minor)

> TestReqParamsAPI regularly fails on Policeman Jenkins
> -
>
> Key: SOLR-10136
> URL: https://issues.apache.org/jira/browse/SOLR-10136
> Project: Solr
>  Issue Type: Test
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Christine Poerschke
>Assignee: Noble Paul
> Attachments: logs.tar.gz
>
>
> {{org.apache.solr.handler.TestReqParamsAPI.test}} regularly fails though 
> interestly only on Policeman Jenkins and not on Apache Jenkins e.g. 
> https://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/18962/testReport/junit/org.apache.solr.handler/TestReqParamsAPI/test/
> The Feb 9th SOLR-10032 report categorised the test as _flakey_.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-10136) TestReqParamsAPI regularly fails on Policeman Jenkins

[Mark Miller (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-10136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mark Miller updated SOLR-10136:
---
Attachment: logs.tar.gz

Fail logs attached.

> TestReqParamsAPI regularly fails on Policeman Jenkins
> -
>
> Key: SOLR-10136
> URL: https://issues.apache.org/jira/browse/SOLR-10136
> Project: Solr
>  Issue Type: Test
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Christine Poerschke
>Assignee: Noble Paul
>Priority: Minor
> Attachments: logs.tar.gz
>
>
> {{org.apache.solr.handler.TestReqParamsAPI.test}} regularly fails though 
> interestly only on Policeman Jenkins and not on Apache Jenkins e.g. 
> https://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/18962/testReport/junit/org.apache.solr.handler/TestReqParamsAPI/test/
> The Feb 9th SOLR-10032 report categorised the test as _flakey_.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Assigned] (SOLR-10136) TestReqParamsAPI regularly fails on Policeman Jenkins

[Noble Paul (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-10136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Noble Paul reassigned SOLR-10136:
-

Assignee: Noble Paul

> TestReqParamsAPI regularly fails on Policeman Jenkins
> -
>
> Key: SOLR-10136
> URL: https://issues.apache.org/jira/browse/SOLR-10136
> Project: Solr
>  Issue Type: Test
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Christine Poerschke
>Assignee: Noble Paul
>Priority: Minor
>
> {{org.apache.solr.handler.TestReqParamsAPI.test}} regularly fails though 
> interestly only on Policeman Jenkins and not on Apache Jenkins e.g. 
> https://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/18962/testReport/junit/org.apache.solr.handler/TestReqParamsAPI/test/
> The Feb 9th SOLR-10032 report categorised the test as _flakey_.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Comment Edited] (SOLR-9530) Add an Atomic Update Processor

[AMRIT SARKAR (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15867142#comment-15867142
 ] 

AMRIT SARKAR edited comment on SOLR-9530 at 2/15/17 3:41 AM:
-

[~noble.paul] [~ichattopadhyaya] Varun,
thank you for looking into the patch.

[~noble.paul], the patch is not thread safe as the processor is thread-specific 
and is mutually independent of other threads. This processor is somehow doing 
in-place conversion of documents (more like a plugin) and will be passed 
ultimately to DistributedUpdateProcessor (which is thread-safe), where the 
actual "atomic" update to the documents in index takes place. 

If multiple threads try to update the same document in parallel via 
AtomicUpdateProcessor, that case will be similar to multiple threads carrying 
"atomic-style" update of same document, which is already happening in our 
latest Solr. DistributedUpdateProcessor, being thread-safe, handles the 
resources well and don't let the versions conflict each other.


was (Author: sarkaramr...@gmail.com):
[~noble.paul] [~ichattopadhyaya] Varun,
thank you for looking into the patch.

[~noble.paul], the patch is not thread safe as the processor is thread-specific 
and is mutually independent of other threads. This processor is somehow doing 
in-place conversion of documents (more like a plugin) and will be passed 
ultimately to DistributedUpdateProcessor (which is thread-safe), where the 
actual "atomic" update to the documents in index takes place. 

If multiple threads try to update the same document in parallel via 
AtomicUpdateProcessor, that case will be similar to multiple threads carrying 
"atomic-style" update of same document, which is already happening in our 
latest Solr. DistributedUpdateProcessor, being thread-safe, handles the 
resources well and don't let the versions conflict each other.

Let me know if I am missing out anything in this regard.

> Add an Atomic Update Processor 
> ---
>
> Key: SOLR-9530
> URL: https://issues.apache.org/jira/browse/SOLR-9530
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Varun Thacker
> Attachments: SOLR-9530.patch
>
>
> I'd like to explore the idea of adding a new update processor to help ingest 
> partial updates.
> Example use-case - There are two datasets with a common id field. How can I 
> merge both of them at index time?
> Proposed Solution: 
> {code}
> 
>   
> add
>   
>   
>   
> 
> {code}
> So the first JSON dump could be ingested against 
> {{http://localhost:8983/solr/gettingstarted/update/json}}
> And then the second JSON could be ingested against
> {{http://localhost:8983/solr/gettingstarted/update/json?processor=atomic}}
> The Atomic Update Processor could support all the atomic update operations 
> currently supported.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10136) TestReqParamsAPI regularly fails on Policeman Jenkins

[Mark Miller (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15867156#comment-15867156
 ] 

Mark Miller commented on SOLR-10136:


On the last run this report went up to 'unreliable'. In my most recent test 
report run, this is currently the worst non nightly test.

> TestReqParamsAPI regularly fails on Policeman Jenkins
> -
>
> Key: SOLR-10136
> URL: https://issues.apache.org/jira/browse/SOLR-10136
> Project: Solr
>  Issue Type: Test
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Christine Poerschke
>Priority: Minor
>
> {{org.apache.solr.handler.TestReqParamsAPI.test}} regularly fails though 
> interestly only on Policeman Jenkins and not on Apache Jenkins e.g. 
> https://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/18962/testReport/junit/org.apache.solr.handler/TestReqParamsAPI/test/
> The Feb 9th SOLR-10032 report categorised the test as _flakey_.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10138) Transaction log replay can hit an NPE due to new Metrics code.

[Mark Miller (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15867151#comment-15867151
 ] 

Mark Miller commented on SOLR-10138:


{noformat}
   [junit4]   2> java.lang.NullPointerException
   [junit4]   2>at 
org.apache.solr.update.UpdateLog$LogReplayer.doReplay(UpdateLog.java:1671)
   [junit4]   2>at 
org.apache.solr.update.UpdateLog$LogReplayer.run(UpdateLog.java:1510)
   [junit4]   2>at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
   [junit4]   2>at 
java.util.concurrent.FutureTask.run(FutureTask.java:266)
   [junit4]   2>at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
   [junit4]   2>at 
java.util.concurrent.FutureTask.run(FutureTask.java:266)
   [junit4]   2>at 
org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)
   [junit4]   2>at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
   [junit4]   2>at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]   2>at java.lang.Thread.run(Thread.java:745)
{noformat}

{noformat}
if (state == State.REPLAYING) {
  replayOpsMeter.mark();
{noformat}

> Transaction log replay can hit an NPE due to new Metrics code.
> --
>
> Key: SOLR-10138
> URL: https://issues.apache.org/jira/browse/SOLR-10138
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Mark Miller
>




--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-10138) Transaction log replay can hit an NPE due to new Metrics code.

[Mark Miller (JIRA)]

Mark Miller created SOLR-10138:
--

 Summary: Transaction log replay can hit an NPE due to new Metrics 
code.
 Key: SOLR-10138
 URL: https://issues.apache.org/jira/browse/SOLR-10138
 Project: Solr
  Issue Type: Bug
  Security Level: Public (Default Security Level. Issues are Public)
Reporter: Mark Miller






--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10068) The Nightly test SharedFSAutoReplicaFailoverTest appears to be too fragile.

[Mark Miller (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15867147#comment-15867147
 ] 

Mark Miller commented on SOLR-10068:


This seems to be because of Metrics:
{noformat}
if (state == State.REPLAYING) {
  replayOpsMeter.mark();
{noformat}

> The Nightly test SharedFSAutoReplicaFailoverTest appears to be too fragile.
> ---
>
> Key: SOLR-10068
> URL: https://issues.apache.org/jira/browse/SOLR-10068
> Project: Solr
>  Issue Type: Test
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Mark Miller
>Assignee: Mark Miller
>
> SharedFSAutoReplicaFailoverTest 37.00% screwy 30.00 273.97 @Nightly



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-6.x-Windows (32bit/jdk1.8.0_121) - Build # 729 - Unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-6.x-Windows/729/
Java: 32bit/jdk1.8.0_121 -server -XX:+UseParallelGC

1 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.TestCloudRecovery

Error Message:
Could not remove the following files (in the order of attempts):
C:\Users\jenkins\workspace\Lucene-Solr-6.x-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.TestCloudRecovery_88997E5D35A07074-001\tempDir-001\node2\collection1_shard2_replica1\data\tlog\tlog.001:
 java.nio.file.FileSystemException: 
C:\Users\jenkins\workspace\Lucene-Solr-6.x-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.TestCloudRecovery_88997E5D35A07074-001\tempDir-001\node2\collection1_shard2_replica1\data\tlog\tlog.001:
 The process cannot access the file because it is being used by another 
process. 
C:\Users\jenkins\workspace\Lucene-Solr-6.x-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.TestCloudRecovery_88997E5D35A07074-001\tempDir-001\node2\collection1_shard2_replica1\data\tlog:
 java.nio.file.DirectoryNotEmptyException: 
C:\Users\jenkins\workspace\Lucene-Solr-6.x-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.TestCloudRecovery_88997E5D35A07074-001\tempDir-001\node2\collection1_shard2_replica1\data\tlog

C:\Users\jenkins\workspace\Lucene-Solr-6.x-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.TestCloudRecovery_88997E5D35A07074-001\tempDir-001\node2\collection1_shard2_replica1\data:
 java.nio.file.DirectoryNotEmptyException: 
C:\Users\jenkins\workspace\Lucene-Solr-6.x-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.TestCloudRecovery_88997E5D35A07074-001\tempDir-001\node2\collection1_shard2_replica1\data

C:\Users\jenkins\workspace\Lucene-Solr-6.x-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.TestCloudRecovery_88997E5D35A07074-001\tempDir-001\node2\collection1_shard2_replica1:
 java.nio.file.DirectoryNotEmptyException: 
C:\Users\jenkins\workspace\Lucene-Solr-6.x-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.TestCloudRecovery_88997E5D35A07074-001\tempDir-001\node2\collection1_shard2_replica1

C:\Users\jenkins\workspace\Lucene-Solr-6.x-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.TestCloudRecovery_88997E5D35A07074-001\tempDir-001\node2\collection1_shard1_replica1\data\tlog\tlog.001:
 java.nio.file.FileSystemException: 
C:\Users\jenkins\workspace\Lucene-Solr-6.x-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.TestCloudRecovery_88997E5D35A07074-001\tempDir-001\node2\collection1_shard1_replica1\data\tlog\tlog.001:
 The process cannot access the file because it is being used by another 
process. 
C:\Users\jenkins\workspace\Lucene-Solr-6.x-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.TestCloudRecovery_88997E5D35A07074-001\tempDir-001\node2\collection1_shard1_replica1\data\tlog:
 java.nio.file.DirectoryNotEmptyException: 
C:\Users\jenkins\workspace\Lucene-Solr-6.x-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.TestCloudRecovery_88997E5D35A07074-001\tempDir-001\node2\collection1_shard1_replica1\data\tlog

C:\Users\jenkins\workspace\Lucene-Solr-6.x-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.TestCloudRecovery_88997E5D35A07074-001\tempDir-001\node2\collection1_shard1_replica1\data:
 java.nio.file.DirectoryNotEmptyException: 
C:\Users\jenkins\workspace\Lucene-Solr-6.x-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.TestCloudRecovery_88997E5D35A07074-001\tempDir-001\node2\collection1_shard1_replica1\data

C:\Users\jenkins\workspace\Lucene-Solr-6.x-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.TestCloudRecovery_88997E5D35A07074-001\tempDir-001\node2\collection1_shard1_replica1:
 java.nio.file.DirectoryNotEmptyException: 
C:\Users\jenkins\workspace\Lucene-Solr-6.x-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.TestCloudRecovery_88997E5D35A07074-001\tempDir-001\node2\collection1_shard1_replica1

C:\Users\jenkins\workspace\Lucene-Solr-6.x-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.TestCloudRecovery_88997E5D35A07074-001\tempDir-001\node2:
 java.nio.file.DirectoryNotEmptyException: 
C:\Users\jenkins\workspace\Lucene-Solr-6.x-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.TestCloudRecovery_88997E5D35A07074-001\tempDir-001\node2

C:\Users\jenkins\workspace\Lucene-Solr-6.x-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.TestCloudRecovery_88997E5D35A07074-001\tempDir-001\node1\collection1_shard2_replica2\data\tlog\tlog.001:
 java.nio.file.FileSystemException: 
C:\Users\jenkins\workspace\Lucene-Solr-6.x-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.TestCloudRecovery_88997E5D35A07074-001\tempDir-001\node1\collection1_shard2_replica2\data\tlog\tlog.001:
 The process cannot access the file because it is being used by another 
process. 

[jira] [Commented] (SOLR-10068) The Nightly test SharedFSAutoReplicaFailoverTest appears to be too fragile.

[Mark Miller (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15867145#comment-15867145
 ] 

Mark Miller commented on SOLR-10068:


I see a bunch of this in the last failed log I looked at:

{noformat}
   [junit4]   2> java.lang.NullPointerException
   [junit4]   2>at 
org.apache.solr.update.UpdateLog$LogReplayer.doReplay(UpdateLog.java:1671)
   [junit4]   2>at 
org.apache.solr.update.UpdateLog$LogReplayer.run(UpdateLog.java:1510)
   [junit4]   2>at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
   [junit4]   2>at 
java.util.concurrent.FutureTask.run(FutureTask.java:266)
   [junit4]   2>at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
   [junit4]   2>at 
java.util.concurrent.FutureTask.run(FutureTask.java:266)
   [junit4]   2>at 
org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)
   [junit4]   2>at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
   [junit4]   2>at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]   2>at java.lang.Thread.run(Thread.java:745)
{noformat}

> The Nightly test SharedFSAutoReplicaFailoverTest appears to be too fragile.
> ---
>
> Key: SOLR-10068
> URL: https://issues.apache.org/jira/browse/SOLR-10068
> Project: Solr
>  Issue Type: Test
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Mark Miller
>Assignee: Mark Miller
>
> SharedFSAutoReplicaFailoverTest 37.00% screwy 30.00 273.97 @Nightly



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-SmokeRelease-master - Build # 699 - Still Failing

[Apache Jenkins Server]

Build: https://builds.apache.org/job/Lucene-Solr-SmokeRelease-master/699/

No tests ran.

Build Log:
[...truncated 42019 lines...]
prepare-release-no-sign:
[mkdir] Created dir: 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/lucene/build/smokeTestRelease/dist
 [copy] Copying 476 files to 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/lucene/build/smokeTestRelease/dist/lucene
 [copy] Copying 260 files to 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/lucene/build/smokeTestRelease/dist/solr
   [smoker] Java 1.8 JAVA_HOME=/home/jenkins/tools/java/latest1.8
   [smoker] NOTE: output encoding is UTF-8
   [smoker] 
   [smoker] Load release URL 
"file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/lucene/build/smokeTestRelease/dist/"...
   [smoker] 
   [smoker] Test Lucene...
   [smoker]   test basics...
   [smoker]   get KEYS
   [smoker] 0.2 MB in 0.01 sec (24.8 MB/sec)
   [smoker]   check changes HTML...
   [smoker]   download lucene-7.0.0-src.tgz...
   [smoker] 30.6 MB in 0.03 sec (1042.5 MB/sec)
   [smoker] verify md5/sha1 digests
   [smoker]   download lucene-7.0.0.tgz...
   [smoker] 65.0 MB in 0.06 sec (1134.6 MB/sec)
   [smoker] verify md5/sha1 digests
   [smoker]   download lucene-7.0.0.zip...
   [smoker] 75.4 MB in 0.07 sec (1073.7 MB/sec)
   [smoker] verify md5/sha1 digests
   [smoker]   unpack lucene-7.0.0.tgz...
   [smoker] verify JAR metadata/identity/no javax.* or java.* classes...
   [smoker] test demo with 1.8...
   [smoker]   got 6204 hits for query "lucene"
   [smoker] checkindex with 1.8...
   [smoker] check Lucene's javadoc JAR
   [smoker]   unpack lucene-7.0.0.zip...
   [smoker] verify JAR metadata/identity/no javax.* or java.* classes...
   [smoker] test demo with 1.8...
   [smoker]   got 6204 hits for query "lucene"
   [smoker] checkindex with 1.8...
   [smoker] check Lucene's javadoc JAR
   [smoker]   unpack lucene-7.0.0-src.tgz...
   [smoker] make sure no JARs/WARs in src dist...
   [smoker] run "ant validate"
   [smoker] run tests w/ Java 8 and testArgs='-Dtests.slow=false'...
   [smoker] test demo with 1.8...
   [smoker]   got 215 hits for query "lucene"
   [smoker] checkindex with 1.8...
   [smoker] generate javadocs w/ Java 8...
   [smoker] 
   [smoker] Crawl/parse...
   [smoker] 
   [smoker] Verify...
   [smoker]   confirm all releases have coverage in TestBackwardsCompatibility
   [smoker] find all past Lucene releases...
   [smoker] run TestBackwardsCompatibility..
   [smoker] Releases that don't seem to be tested:
   [smoker]   5.5.4
   [smoker] Traceback (most recent call last):
   [smoker]   File 
"/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/dev-tools/scripts/smokeTestRelease.py",
 line 1474, in 
   [smoker] main()
   [smoker]   File 
"/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/dev-tools/scripts/smokeTestRelease.py",
 line 1418, in main
   [smoker] smokeTest(c.java, c.url, c.revision, c.version, c.tmp_dir, 
c.is_signed, ' '.join(c.test_args))
   [smoker]   File 
"/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/dev-tools/scripts/smokeTestRelease.py",
 line 1456, in smokeTest
   [smoker] unpackAndVerify(java, 'lucene', tmpDir, 'lucene-%s-src.tgz' % 
version, gitRevision, version, testArgs, baseURL)
   [smoker]   File 
"/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/dev-tools/scripts/smokeTestRelease.py",
 line 622, in unpackAndVerify
   [smoker] verifyUnpacked(java, project, artifact, unpackPath, 
gitRevision, version, testArgs, tmpDir, baseURL)
   [smoker]   File 
"/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/dev-tools/scripts/smokeTestRelease.py",
 line 768, in verifyUnpacked
   [smoker] confirmAllReleasesAreTestedForBackCompat(version, unpackPath)
   [smoker]   File 
"/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/dev-tools/scripts/smokeTestRelease.py",
 line 1394, in confirmAllReleasesAreTestedForBackCompat
   [smoker] raise RuntimeError('some releases are not tested by 
TestBackwardsCompatibility?')
   [smoker] RuntimeError: some releases are not tested by 
TestBackwardsCompatibility?

BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/build.xml:571:
 exec returned: 1

Total time: 39 minutes 50 seconds
Build step 'Invoke Ant' marked build as failure
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any




-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9530) Add an Atomic Update Processor

[AMRIT SARKAR (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15867142#comment-15867142
 ] 

AMRIT SARKAR commented on SOLR-9530:


[~noble.paul] [~ichattopadhyaya] Varun,
thank you for looking into the patch.

[~noble.paul], the patch is not thread safe as the processor is thread-specific 
and is mutually independent of other threads. This processor is somehow doing 
in-place conversion of documents (more like a plugin) and will be passed 
ultimately to DistributedUpdateProcessor (which is thread-safe), where the 
actual "atomic" update to the documents in index takes place. 

If multiple threads try to update the same document in parallel via 
AtomicUpdateProcessor, that case will be similar to multiple threads carrying 
"atomic-style" update of same document, which is already happening in our 
latest Solr. DistributedUpdateProcessor, being thread-safe, handles the 
resources well and don't let the versions conflict each other.

Let me know if I am missing out anything in this regard.

> Add an Atomic Update Processor 
> ---
>
> Key: SOLR-9530
> URL: https://issues.apache.org/jira/browse/SOLR-9530
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Varun Thacker
> Attachments: SOLR-9530.patch
>
>
> I'd like to explore the idea of adding a new update processor to help ingest 
> partial updates.
> Example use-case - There are two datasets with a common id field. How can I 
> merge both of them at index time?
> Proposed Solution: 
> {code}
> 
>   
> add
>   
>   
>   
> 
> {code}
> So the first JSON dump could be ingested against 
> {{http://localhost:8983/solr/gettingstarted/update/json}}
> And then the second JSON could be ingested against
> {{http://localhost:8983/solr/gettingstarted/update/json?processor=atomic}}
> The Atomic Update Processor could support all the atomic update operations 
> currently supported.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS-EA] Lucene-Solr-6.x-Linux (32bit/jdk-9-ea+155) - Build # 2857 - Unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-6.x-Linux/2857/
Java: 32bit/jdk-9-ea+155 -server -XX:+UseSerialGC

1 tests failed.
FAILED:  org.apache.solr.handler.admin.TestApiFramework.testFramework

Error Message:


Stack Trace:
java.lang.ExceptionInInitializerError
at 
__randomizedtesting.SeedInfo.seed([D98FD5C37F011EF5:CEF91FE479D5F2C8]:0)
at 
net.sf.cglib.core.KeyFactory$Generator.generateClass(KeyFactory.java:166)
at 
net.sf.cglib.core.DefaultGeneratorStrategy.generate(DefaultGeneratorStrategy.java:25)
at 
net.sf.cglib.core.AbstractClassGenerator.create(AbstractClassGenerator.java:216)
at net.sf.cglib.core.KeyFactory$Generator.create(KeyFactory.java:144)
at net.sf.cglib.core.KeyFactory.create(KeyFactory.java:116)
at net.sf.cglib.core.KeyFactory.create(KeyFactory.java:108)
at net.sf.cglib.core.KeyFactory.create(KeyFactory.java:104)
at net.sf.cglib.proxy.Enhancer.(Enhancer.java:69)
at 
org.easymock.internal.ClassProxyFactory.createEnhancer(ClassProxyFactory.java:259)
at 
org.easymock.internal.ClassProxyFactory.createProxy(ClassProxyFactory.java:174)
at org.easymock.internal.MocksControl.createMock(MocksControl.java:60)
at org.easymock.EasyMock.createMock(EasyMock.java:104)
at 
org.apache.solr.handler.admin.TestCoreAdminApis.getCoreContainerMock(TestCoreAdminApis.java:83)
at 
org.apache.solr.handler.admin.TestApiFramework.testFramework(TestApiFramework.java:59)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:543)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 

[JENKINS] Lucene-Solr-Tests-5.5 - Build # 28 - Failure

[Apache Jenkins Server]

Build: https://builds.apache.org/job/Lucene-Solr-Tests-5.5/28/

1 tests failed.
FAILED:  
org.apache.solr.cloud.overseer.ZkStateReaderTest.testStateFormatUpdateWithExplicitRefresh

Error Message:
Could not find collection : c1

Stack Trace:
org.apache.solr.common.SolrException: Could not find collection : c1
at 
__randomizedtesting.SeedInfo.seed([6C99E32F0C188684:732392D8DC784041]:0)
at 
org.apache.solr.common.cloud.ClusterState.getCollection(ClusterState.java:170)
at 
org.apache.solr.cloud.overseer.ZkStateReaderTest.testStateFormatUpdate(ZkStateReaderTest.java:136)
at 
org.apache.solr.cloud.overseer.ZkStateReaderTest.testStateFormatUpdateWithExplicitRefresh(ZkStateReaderTest.java:42)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 10588 lines...]
   [junit4] Suite: org.apache.solr.cloud.overseer.ZkStateReaderTest
   [junit4]   2> Creating dataDir: 

[jira] [Commented] (SOLR-8029) Modernize and standardize Solr APIs

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-8029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15867081#comment-15867081
 ] 

ASF subversion and git services commented on SOLR-8029:
---

Commit 2470fbaa0ee4d1a211957bc47242bd081a31234d in lucene-solr's branch 
refs/heads/master from [~noble.paul]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=2470fba ]

SOLR-8029: introspect was not showing for certain collection apis


> Modernize and standardize Solr APIs
> ---
>
> Key: SOLR-8029
> URL: https://issues.apache.org/jira/browse/SOLR-8029
> Project: Solr
>  Issue Type: Improvement
>Affects Versions: 6.0
>Reporter: Noble Paul
>Assignee: Noble Paul
>  Labels: API, EaseOfUse
> Fix For: 6.0
>
> Attachments: SOLR-8029.patch, SOLR-8029.patch, SOLR-8029.patch, 
> SOLR-8029.patch, SOLR-8029.patch
>
>
> Solr APIs have organically evolved and they are sometimes inconsistent with 
> each other or not in sync with the widely followed conventions of HTTP 
> protocol. Trying to make incremental changes to make them modern is like 
> applying band-aid. So, we have done a complete rethink of what the APIs 
> should be. The most notable aspects of the API are as follows:
> The new set of APIs will be placed under a new path {{/solr2}}. The legacy 
> APIs will continue to work under the {{/solr}} path as they used to and they 
> will be eventually deprecated.
> There are 4 types of requests in the new API 
> * {{/v2//*}} : Hit a collection directly or manage 
> collections/shards/replicas 
> * {{/v2//*}} : Hit a core directly or manage cores 
> * {{/v2/cluster/*}} : Operations on cluster not pertaining to any collection 
> or core. e.g: security, overseer ops etc
> This will be released as part of a major release. Check the link given below 
> for the full specification.  Your comments are welcome
> [Solr API version 2 Specification | http://bit.ly/1JYsBMQ]



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-8029) Modernize and standardize Solr APIs

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-8029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15867078#comment-15867078
 ] 

ASF subversion and git services commented on SOLR-8029:
---

Commit 9a1702a8f5b5496893a99c4e1f39cd58520786ae in lucene-solr's branch 
refs/heads/master from [~ichattopadhyaya]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=9a1702a ]

SOLR-8029: Reverting the previous commit and the merge


> Modernize and standardize Solr APIs
> ---
>
> Key: SOLR-8029
> URL: https://issues.apache.org/jira/browse/SOLR-8029
> Project: Solr
>  Issue Type: Improvement
>Affects Versions: 6.0
>Reporter: Noble Paul
>Assignee: Noble Paul
>  Labels: API, EaseOfUse
> Fix For: 6.0
>
> Attachments: SOLR-8029.patch, SOLR-8029.patch, SOLR-8029.patch, 
> SOLR-8029.patch, SOLR-8029.patch
>
>
> Solr APIs have organically evolved and they are sometimes inconsistent with 
> each other or not in sync with the widely followed conventions of HTTP 
> protocol. Trying to make incremental changes to make them modern is like 
> applying band-aid. So, we have done a complete rethink of what the APIs 
> should be. The most notable aspects of the API are as follows:
> The new set of APIs will be placed under a new path {{/solr2}}. The legacy 
> APIs will continue to work under the {{/solr}} path as they used to and they 
> will be eventually deprecated.
> There are 4 types of requests in the new API 
> * {{/v2//*}} : Hit a collection directly or manage 
> collections/shards/replicas 
> * {{/v2//*}} : Hit a core directly or manage cores 
> * {{/v2/cluster/*}} : Operations on cluster not pertaining to any collection 
> or core. e.g: security, overseer ops etc
> This will be released as part of a major release. Check the link given below 
> for the full specification.  Your comments are welcome
> [Solr API version 2 Specification | http://bit.ly/1JYsBMQ]



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10129) Expose lucene PointRange fields in Solr

[Yonik Seeley (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15867059#comment-15867059
 ] 

Yonik Seeley commented on SOLR-10129:
-

Other range queries use * as open endpoints... perhaps that syntax should be 
reused here?

> Expose lucene PointRange fields in Solr
> ---
>
> Key: SOLR-10129
> URL: https://issues.apache.org/jira/browse/SOLR-10129
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Alan Woodward
>Assignee: Alan Woodward
> Attachments: SOLR-10129.patch
>
>
> Follow up to SOLR-8396, it would be nice to expose the sandbox PointRange 
> fields in Solr.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[ANNOUNCE] Apache PyLucene 6.4.1

[Andi Vajda]

I am pleased to announce the availability of Apache PyLucene 6.4.1.

Apache PyLucene, a subproject of Apache Lucene, is a Python extension for
accessing Apache Lucene Core. Its goal is to allow you to use Lucene's text
indexing and searching capabilities from Python. It is API compatible with
the latest version of Lucene 6.x Core, 6.4.1.

This release contains many changes since the last PyLucene release was 
compatible with Lucene 4.x. Details can be found in the changes files:


http://svn.apache.org/repos/asf/lucene/pylucene/tags/pylucene_6_4_1/CHANGES
http://svn.apache.org/repos/asf/lucene/pylucene/tags/pylucene_6_4_1/jcc/CHANGES
http://lucene.apache.org/core/6_4_1/changes/Changes.html

Apache PyLucene is available from the following download page:
http://www.apache.org/dyn/closer.cgi/lucene/pylucene/pylucene-6.4.1-src.tar.gz

When downloading from a mirror site, please remember to verify the downloads 
using signatures found on the Apache site:

https://dist.apache.org/repos/dist/release/lucene/pylucene/KEYS

For more information on Apache PyLucene, visit the project home page:
  http://lucene.apache.org/pylucene

Andi..

Question on Solr

[Prathib Kumar]

Hi,

We are evaluating solr to see if it can help to do a search of the xml
snippets from the whole xml doc.

For Ex:
Document-1:


   Prathib
   Java
   san jose
CA


Document-2:

   Joe
   C++
   chennai
TN


Document-3:

   Ramu
   Python
   LosAngeles
CA



My Search string is another XML doc which could be like.

Query-1:

 san jose


Query-2:

   CA


I have broken this down for simplicity, in reality our xmls are nested and
have many attributes on each tag.

To continue the evaluation of solr, can you please help me from where I
could start the analysis ?

Note : currently our xml document doesnt adhere to any schema but we could
create a schema if required.



Regards
Prathib Kumar.

[jira] [Commented] (SOLR-8029) Modernize and standardize Solr APIs

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-8029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15866993#comment-15866993
 ] 

ASF subversion and git services commented on SOLR-8029:
---

Commit 65c6c576b720b19029a10bf14f81d4de23302863 in lucene-solr's branch 
refs/heads/master from [~noble.paul]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=65c6c57 ]

SOLR-8029: introspect was not showing for certain collection apis


> Modernize and standardize Solr APIs
> ---
>
> Key: SOLR-8029
> URL: https://issues.apache.org/jira/browse/SOLR-8029
> Project: Solr
>  Issue Type: Improvement
>Affects Versions: 6.0
>Reporter: Noble Paul
>Assignee: Noble Paul
>  Labels: API, EaseOfUse
> Fix For: 6.0
>
> Attachments: SOLR-8029.patch, SOLR-8029.patch, SOLR-8029.patch, 
> SOLR-8029.patch, SOLR-8029.patch
>
>
> Solr APIs have organically evolved and they are sometimes inconsistent with 
> each other or not in sync with the widely followed conventions of HTTP 
> protocol. Trying to make incremental changes to make them modern is like 
> applying band-aid. So, we have done a complete rethink of what the APIs 
> should be. The most notable aspects of the API are as follows:
> The new set of APIs will be placed under a new path {{/solr2}}. The legacy 
> APIs will continue to work under the {{/solr}} path as they used to and they 
> will be eventually deprecated.
> There are 4 types of requests in the new API 
> * {{/v2//*}} : Hit a collection directly or manage 
> collections/shards/replicas 
> * {{/v2//*}} : Hit a core directly or manage cores 
> * {{/v2/cluster/*}} : Operations on cluster not pertaining to any collection 
> or core. e.g: security, overseer ops etc
> This will be released as part of a major release. Check the link given below 
> for the full specification.  Your comments are welcome
> [Solr API version 2 Specification | http://bit.ly/1JYsBMQ]



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-8029) Modernize and standardize Solr APIs

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-8029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15866988#comment-15866988
 ] 

ASF subversion and git services commented on SOLR-8029:
---

Commit 5c4ee10683fab338f3f4619a850d509c716ebe33 in lucene-solr's branch 
refs/heads/branch_6x from [~noble.paul]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=5c4ee10 ]

SOLR-8029: introspect was not showing for certain collection apis


> Modernize and standardize Solr APIs
> ---
>
> Key: SOLR-8029
> URL: https://issues.apache.org/jira/browse/SOLR-8029
> Project: Solr
>  Issue Type: Improvement
>Affects Versions: 6.0
>Reporter: Noble Paul
>Assignee: Noble Paul
>  Labels: API, EaseOfUse
> Fix For: 6.0
>
> Attachments: SOLR-8029.patch, SOLR-8029.patch, SOLR-8029.patch, 
> SOLR-8029.patch, SOLR-8029.patch
>
>
> Solr APIs have organically evolved and they are sometimes inconsistent with 
> each other or not in sync with the widely followed conventions of HTTP 
> protocol. Trying to make incremental changes to make them modern is like 
> applying band-aid. So, we have done a complete rethink of what the APIs 
> should be. The most notable aspects of the API are as follows:
> The new set of APIs will be placed under a new path {{/solr2}}. The legacy 
> APIs will continue to work under the {{/solr}} path as they used to and they 
> will be eventually deprecated.
> There are 4 types of requests in the new API 
> * {{/v2//*}} : Hit a collection directly or manage 
> collections/shards/replicas 
> * {{/v2//*}} : Hit a core directly or manage cores 
> * {{/v2/cluster/*}} : Operations on cluster not pertaining to any collection 
> or core. e.g: security, overseer ops etc
> This will be released as part of a major release. Check the link given below 
> for the full specification.  Your comments are welcome
> [Solr API version 2 Specification | http://bit.ly/1JYsBMQ]



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-10117) Big docs and the DocumentCache; umbrella issue

[David Smiley (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-10117?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

David Smiley updated SOLR-10117:

Attachment: SOLR_10117_large_fields.patch

Here's a patch that adds a "large" boolean property to schema fields.  Such 
fields, in conjunction with lazy field loading, will get their own separate 
lazy document.  _This is one piece of a larger puzzle._ 

It would be awesome to _instead_ have large-ness detected dynamically without 
having to declare them as large as I'm doing here but that has some issues.  
First if the lazy field loading wanted to know the size then it has to incur 
the cost of loading it, which defeats the point.  A possible solution I'm in 
favor of (yet might be controversial?) would be Solr's DocumentBuilder 
detecting large string values and then if there is one then adding the name to 
a proposed docValues {{\_largeFields\_}} field.  Then lazy field loading could 
examine the values.  An alternative variation on this is to save it as a stored 
value that comes first in the stored document, since lazy field loading has to 
go to disk for this any way. I actually like that better than using docValues.  
It might even go further and place the largest field value(s) last to benefit 
from a Lucene level optimization I got in a while back.

Notice this LargeFieldsTest uses some testing techniques I want to popularize 
and refine.  There are no new schema/solrconfig files despite that this test 
defines fields, field types, and makes solrconfig changes.  And it doesn't copy 
configs to do this.

> Big docs and the DocumentCache; umbrella issue
> --
>
> Key: SOLR-10117
> URL: https://issues.apache.org/jira/browse/SOLR-10117
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: David Smiley
>Assignee: David Smiley
> Attachments: SOLR_10117_large_fields.patch
>
>
> This is an umbrella issue for improved handling of large documents (large 
> stored fields), generally related to the DocumentCache or SolrIndexSearcher's 
> doc() methods.  Highlighting is affected as it's the primary consumer of this 
> data.  "Large" here is multi-megabyte, especially tens even hundreds of 
> megabytes. We'd like to support such users without forcing them to choose 
> between no DocumentCache (bad performance), or having one but hitting OOM due 
> to massive Strings winding up in there.  I've contemplated this for longer 
> than I'd like to admit and it's a complicated issue with difference concerns 
> to balance.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-NightlyTests-5.5 - Build # 12 - Failure

[Apache Jenkins Server]

Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.5/12/

All tests passed

Build Log:
[...truncated 11830 lines...]
   [junit4] JVM J1: stdout was not empty, see: 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.5/checkout/solr/build/solr-core/test/temp/junit4-J1-20170214_222805_842.sysout
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) 
   [junit4] java.lang.OutOfMemoryError: GC overhead limit exceeded
   [junit4] Dumping heap to 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.5/checkout/heapdumps/java_pid18214.hprof
 ...
   [junit4] Heap dump file created [662086582 bytes in 22.765 secs]
   [junit4] <<< JVM J1: EOF 

   [junit4] JVM J1: stderr was not empty, see: 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.5/checkout/solr/build/solr-core/test/temp/junit4-J1-20170214_222805_842.syserr
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) 
   [junit4] WARN: Unhandled exception in event serialization. -> 
java.lang.OutOfMemoryError: GC overhead limit exceeded
   [junit4] <<< JVM J1: EOF 

[...truncated 893 lines...]
   [junit4] ERROR: JVM J1 ended with an exception, command line: 
/usr/local/asfpackages/java/jdk1.7.0_80/jre/bin/java 
-XX:+HeapDumpOnOutOfMemoryError 
-XX:HeapDumpPath=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.5/checkout/heapdumps
 -XX:MaxPermSize=192m -ea -esa -Dtests.prefix=tests 
-Dtests.seed=BA153CB3E9BA2EC2 -Xmx512M -Dtests.iters= -Dtests.verbose=false 
-Dtests.infostream=false -Dtests.codec=random -Dtests.postingsformat=random 
-Dtests.docvaluesformat=random -Dtests.locale=random -Dtests.timezone=random 
-Dtests.directory=random 
-Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.5/test-data/enwiki.random.lines.txt
 -Dtests.luceneMatchVersion=5.5.4 -Dtests.cleanthreads=perClass 
-Djava.util.logging.config.file=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.5/checkout/lucene/tools/junit4/logging.properties
 -Dtests.nightly=true -Dtests.weekly=false -Dtests.monster=false 
-Dtests.slow=true -Dtests.asserts=true -Dtests.multiplier=2 -DtempDir=./temp 
-Djava.io.tmpdir=./temp 
-Djunit4.tempDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.5/checkout/solr/build/solr-core/test/temp
 
-Dcommon.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.5/checkout/lucene
 
-Dclover.db.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.5/checkout/lucene/build/clover/db
 
-Djava.security.policy=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.5/checkout/lucene/tools/junit4/solr-tests.policy
 -Dtests.LUCENE_VERSION=5.5.4 -Djetty.testMode=1 -Djetty.insecurerandom=1 
-Dsolr.directoryFactory=org.apache.solr.core.MockDirectoryFactory 
-Djava.awt.headless=true -Djdk.map.althashing.threshold=0 
-Djunit4.childvm.cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.5/checkout/solr/build/solr-core/test/J1
 -Djunit4.childvm.id=1 -Djunit4.childvm.count=3 -Dtests.leaveTemporary=false 
-Dtests.filterstacks=true -Dfile.encoding=ISO-8859-1 -classpath 

[JENKINS] Lucene-Solr-6.x-Solaris (64bit/jdk1.8.0) - Build # 668 - Still Unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-6.x-Solaris/668/
Java: 64bit/jdk1.8.0 -XX:+UseCompressedOops -XX:+UseG1GC

4 tests failed.
FAILED:  org.apache.solr.core.TestLazyCores.testNoCommit

Error Message:
Exception during query

Stack Trace:
java.lang.RuntimeException: Exception during query
at 
__randomizedtesting.SeedInfo.seed([C8CCD3E337D0508F:17AC7232FCF7332A]:0)
at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:918)
at org.apache.solr.core.TestLazyCores.check10(TestLazyCores.java:794)
at 
org.apache.solr.core.TestLazyCores.testNoCommit(TestLazyCores.java:776)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException: REQUEST FAILED: 
xpath=//result[@numFound='10']
xml response was: 



  0
  0
  
*:*
  





request was:q=*:*
at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:911)
... 41 more


FAILED:  

[jira] [Commented] (SOLR-10081) Web UI (Generate Training Data ) for LTR plugin

[qfdk (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15866916#comment-15866916
 ] 

qfdk commented on SOLR-10081:
-

[~cpoerschke]
Thanks for your advice :) I think if we could developer an interface for ltr 
plugin is a good idea. 

exp:
- if there is "visit_number" column in a database(solr) we could use this filed 
as "click_logs" by adding a button or an input filed in web ui. 
- A "train" button to execute the python code or javascript pour upload the 
feature and model. Then users could understand how ltr work.

training data(WEB UI) -> feature -> model -> ltr query 

Regards,
LI

> Web UI (Generate Training Data ) for LTR plugin
> ---
>
> Key: SOLR-10081
> URL: https://issues.apache.org/jira/browse/SOLR-10081
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: qfdk
>
> This is a New Feature for LTR plugin. It will help you to generate the 
> training data (HUMAN_JUDGEMENT) by using score between 0-4 and generate 
> (Click_logs). Live demo http://ltr.qfdk.me



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5143) rm or formalize dealing with "general" KEYS files in our dist dir

[JIRA]

[ 
https://issues.apache.org/jira/browse/LUCENE-5143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15866911#comment-15866911
 ] 

Jan Høydahl commented on LUCENE-5143:
-

To bring closure, I propose the following

* We delete these: {{lucene/java/KEYS}}, {{lucene/solr/KEYS}}
* We update https://archive.apache.org/dist/lucene/KEYS and make sure it 
contains all keys ever used for signing a release
* We stop publishing KEYS to every release dir. Instead the RM will only need 
to make sure that his/her *own* key is in the file. Never remove keys, only add.
* https://wiki.apache.org/lucene-java/ReleaseTodo should probably be updated to 
clarify

I think this is in-line with 
https://www.apache.org/dev/release-signing.html#keys-policy

> rm or formalize dealing with "general" KEYS files in our dist dir
> -
>
> Key: LUCENE-5143
> URL: https://issues.apache.org/jira/browse/LUCENE-5143
> Project: Lucene - Core
>  Issue Type: Task
>Reporter: Hoss Man
>
> At some point in the past, we started creating a snapshots of KEYS (taken 
> from the auto-generated data from id.apache.org) in the release dir of each 
> release...
> http://www.apache.org/dist/lucene/solr/4.4.0/KEYS
> http://www.apache.org/dist/lucene/java/4.4.0/KEYS
> http://archive.apache.org/dist/lucene/java/4.3.0/KEYS
> http://archive.apache.org/dist/lucene/solr/4.3.0/KEYS
> etc...
> But we also still have some "general" KEYS files...
> https://www.apache.org/dist/lucene/KEYS
> https://www.apache.org/dist/lucene/java/KEYS
> https://www.apache.org/dist/lucene/solr/KEYS
> ...which (as i discovered when i went to add my key to them today) are stale 
> and don't seem to be getting updated.
> I vaguely remember someone (rmuir?) explaining to me at one point the reason 
> we started creating a fresh copy of KEYS in each release dir, but i no longer 
> remember what they said, and i can't find any mention of a reason in any of 
> the release docs, or in any sort of comment in buildAndPushRelease.py
> we probably do one of the following:
>  * remove these "general" KEYS files
>  * add a disclaimer to the top of these files that they are legacy files for 
> verifying old releases and are no longer used for new releases
>  * ensure these files are up to date stop generating per-release KEYS file 
> copies
>  * update our release process to ensure that the general files get updated on 
> each release as well



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-10081) Web UI (Generate Training Data ) for LTR plugin

[qfdk (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-10081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

qfdk updated SOLR-10081:

Description: This is a New Feature for LTR plugin. It will help you to 
generate the training data (HUMAN_JUDGEMENT) by using score between 0-4 and 
generate (Click_logs). Live demo http://ltr.qfdk.me  (was: This is a New 
Feature for LTR plugin. It will help you to generate the training data 
(HUMAN_JUDGEMENT) by using score between 0-4. Live demo http://ltr.qfdk.me)

> Web UI (Generate Training Data ) for LTR plugin
> ---
>
> Key: SOLR-10081
> URL: https://issues.apache.org/jira/browse/SOLR-10081
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: qfdk
>
> This is a New Feature for LTR plugin. It will help you to generate the 
> training data (HUMAN_JUDGEMENT) by using score between 0-4 and generate 
> (Click_logs). Live demo http://ltr.qfdk.me



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10129) Expose lucene PointRange fields in Solr

[JIRA]

[ 
https://issues.apache.org/jira/browse/SOLR-10129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15866895#comment-15866895
 ] 

Tomás Fernández Löbbe commented on SOLR-10129:
--

Looks good. I think we should be more defensive while parsing the ranges, be 
able to handle and error on invalid inputs. 
I'm not sure we should call the query parser {{PointRangeQParserPlugin}}, the 
"points" part is just an implementation detail, these are just ranges, right? 
Why not just  {{RangeQParserPlugin}}?

> Expose lucene PointRange fields in Solr
> ---
>
> Key: SOLR-10129
> URL: https://issues.apache.org/jira/browse/SOLR-10129
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Alan Woodward
>Assignee: Alan Woodward
> Attachments: SOLR-10129.patch
>
>
> Follow up to SOLR-8396, it would be nice to expose the sandbox PointRange 
> fields in Solr.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10130) Serious performance degradation in Solr 6.4.1 due to the new metrics collection

[Ishan Chattopadhyaya (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15866888#comment-15866888
 ] 

Ishan Chattopadhyaya commented on SOLR-10130:
-

{code}
Prefix query times for 6.4.1 with SOLR-10130 patch
-
java -cp 
target/org.apache.solr.tests.upgradetests-0.0.1-SNAPSHOT-jar-with-dependencies.jar:.
 org.apache.solr.tests.upgradetests.SimpleBenchmarks -v 
72f75b2503fa0aa4f0aff76d439874feb923bb0e -patchUrl 
https://issues.apache.org/jira/secure/attachment/12852444/SOLR-10130.patch 
-Nodes 1 -Shards 1 -Replicas 1 -numDocs 1 -threads 4 -benchmarkType 
generalQuerying

Got results for prefix queries: 1
Max time (prefix queries): 1716
Total time (prefix queries): 852266
{code}

> Serious performance degradation in Solr 6.4.1 due to the new metrics 
> collection
> ---
>
> Key: SOLR-10130
> URL: https://issues.apache.org/jira/browse/SOLR-10130
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: metrics
>Affects Versions: 6.4.1
> Environment: Centos 7, OpenJDK 1.8.0 update 111
>Reporter: Ere Maijala
>Assignee: Andrzej Bialecki 
>Priority: Blocker
>  Labels: perfomance
> Fix For: master (7.0), 6.4.2
>
> Attachments: SOLR-10130.patch, solr-8983-console-f1.log
>
>
> We've stumbled on serious performance issues after upgrading to Solr 6.4.1. 
> Looks like the new metrics collection system in MetricsDirectoryFactory is 
> causing a major slowdown. This happens with an index configuration that, as 
> far as I can see, has no metrics specific configuration and uses 
> luceneMatchVersion 5.5.0. In practice a moderate load will completely bog 
> down the server with Solr threads constantly using up all CPU (600% on 6 core 
> machine) capacity with a load that normally  where we normally see an average 
> load of < 50%.
> I took stack traces (I'll attach them) and noticed that the threads are 
> spending time in com.codahale.metrics.Meter.mark. I tested building Solr 
> 6.4.1 with the metrics collection disabled in MetricsDirectoryFactory getByte 
> and getBytes methods and was unable to reproduce the issue.
> As far as I can see there are several issues:
> 1. Collecting metrics on every single byte read is slow.
> 2. Having it enabled by default is not a good idea.
> 3. The comment "enable coarse-grained metrics by default" at 
> https://github.com/apache/lucene-solr/blob/branch_6x/solr/core/src/java/org/apache/solr/update/SolrIndexConfig.java#L104
>  implies that only coarse-grained metrics should be enabled by default, and 
> this contradicts with collecting metrics on every single byte read.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7465) Add a PatternTokenizer that uses Lucene's RegExp implementation

[Michael McCandless (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15866854#comment-15866854
 ] 

Michael McCandless commented on LUCENE-7465:


Thanks [~steve_rowe], I'll dig.

> Add a PatternTokenizer that uses Lucene's RegExp implementation
> ---
>
> Key: LUCENE-7465
> URL: https://issues.apache.org/jira/browse/LUCENE-7465
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Michael McCandless
>Assignee: Michael McCandless
> Fix For: master (7.0), 6.5
>
> Attachments: LUCENE-7465.patch, LUCENE-7465.patch
>
>
> I think there are some nice benefits to a version of PatternTokenizer that 
> uses Lucene's RegExp impl instead of the JDK's:
>   * Lucene's RegExp is compiled to a DFA up front, so if a "too hard" RegExp 
> is attempted the user discovers it up front instead of later on when a 
> "lucky" document arrives
>   * It processes the incoming characters as a stream, only pulling 128 
> characters at a time, vs the existing {{PatternTokenizer}} which currently 
> reads the entire string up front (this has caused heap problems in the past)
>   * It should be fast.
> I named it {{SimplePatternTokenizer}}, and it still needs a factory and 
> improved tests, but I think it's otherwise close.
> It currently does not take a {{group}} parameter because Lucene's RegExps 
> don't yet implement sub group capture.  I think we could add that at some 
> point, but it's a bit tricky.
> This doesn't even have group=-1 support (like String.split) ... I think if we 
> did that we should maybe name it differently 
> ({{SimplePatternSplitTokenizer}}?).



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10130) Serious performance degradation in Solr 6.4.1 due to the new metrics collection

[Ishan Chattopadhyaya (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15866806#comment-15866806
 ] 

Ishan Chattopadhyaya commented on SOLR-10130:
-

I could reproduce a 1.6x slowdown for prefix queries.

Benchmarking suite: 
https://github.com/chatman/solr-upgrade-tests/blob/master/BENCHMARKS.md
Environment: packet.net, Type 0 server 
(https://www.packet.net/bare-metal/servers/type-0/)
{code}
Prefix query times for 6.4.1

java -cp 
target/org.apache.solr.tests.upgradetests-0.0.1-SNAPSHOT-jar-with-dependencies.jar:.
 org.apache.solr.tests.upgradetests.SimpleBenchmarks -v 
72f75b2503fa0aa4f0aff76d439874feb923bb0e -Nodes 1 -Shards 1 -Replicas 1 
-numDocs 1 -threads 4 -benchmarkType generalQuerying

Got results for prefix queries: 1
Max time (prefix queries): 2156ms
Total time (prefix queries): 1324856ms

Prefix query times for 6.3.0

java -cp 
target/org.apache.solr.tests.upgradetests-0.0.1-SNAPSHOT-jar-with-dependencies.jar:.
 org.apache.solr.tests.upgradetests.SimpleBenchmarks -v 
6fa26fe8553b7b65dee96da741f2c1adf4cb6216 -patchUrl 
http://147.75.108.131/LUCENE-7651.patch -Nodes 1 -Shards 1 -Replicas 1 -numDocs 
1 -threads 4 -benchmarkType generalQuerying

Got results for prefix queries: 1
Max time (prefix queries): 1358ms
Total time (prefix queries): 839534ms
{code}

Notes:
1. The -threads parameter here is for no. of indexing threads, and number of 
querying threads is 4 times that, i.e. 16 in this case.
2. Total time is the sum of all times, as reported in the response header's 
"QTime". Max time is the QTime for the worst performing query.

> Serious performance degradation in Solr 6.4.1 due to the new metrics 
> collection
> ---
>
> Key: SOLR-10130
> URL: https://issues.apache.org/jira/browse/SOLR-10130
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: metrics
>Affects Versions: 6.4.1
> Environment: Centos 7, OpenJDK 1.8.0 update 111
>Reporter: Ere Maijala
>Assignee: Andrzej Bialecki 
>Priority: Blocker
>  Labels: perfomance
> Fix For: master (7.0), 6.4.2
>
> Attachments: SOLR-10130.patch, solr-8983-console-f1.log
>
>
> We've stumbled on serious performance issues after upgrading to Solr 6.4.1. 
> Looks like the new metrics collection system in MetricsDirectoryFactory is 
> causing a major slowdown. This happens with an index configuration that, as 
> far as I can see, has no metrics specific configuration and uses 
> luceneMatchVersion 5.5.0. In practice a moderate load will completely bog 
> down the server with Solr threads constantly using up all CPU (600% on 6 core 
> machine) capacity with a load that normally  where we normally see an average 
> load of < 50%.
> I took stack traces (I'll attach them) and noticed that the threads are 
> spending time in com.codahale.metrics.Meter.mark. I tested building Solr 
> 6.4.1 with the metrics collection disabled in MetricsDirectoryFactory getByte 
> and getBytes methods and was unable to reproduce the issue.
> As far as I can see there are several issues:
> 1. Collecting metrics on every single byte read is slow.
> 2. Having it enabled by default is not a good idea.
> 3. The comment "enable coarse-grained metrics by default" at 
> https://github.com/apache/lucene-solr/blob/branch_6x/solr/core/src/java/org/apache/solr/update/SolrIndexConfig.java#L104
>  implies that only coarse-grained metrics should be enabled by default, and 
> this contradicts with collecting metrics on every single byte read.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Closed] (LUCENE-7132) BooleanQuery scores can be diff for same docs+sim when using coord (disagree with Explanation which doesn't change)

[Adrien Grand (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-7132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Adrien Grand closed LUCENE-7132.


> BooleanQuery scores can be diff for same docs+sim when using coord (disagree 
> with Explanation which doesn't change)
> ---
>
> Key: LUCENE-7132
> URL: https://issues.apache.org/jira/browse/LUCENE-7132
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: core/search
>Affects Versions: 5.5
>Reporter: Ahmet Arslan
>Assignee: Steve Rowe
>Priority: Blocker
> Fix For: 6.1, 5.5.2, master (7.0)
>
> Attachments: debug.xml, LUCENE-7132.patch, LUCENE-7132.patch, 
> LUCENE-7132.patch, LUCENE-7132.patch, LUCENE-7132.patch, LUCENE-7132.patch, 
> LUCENE-7132.patch, SOLR-8884.patch, SOLR-8884.patch
>
>
> Some of the folks 
> [reported|http://find.searchhub.org/document/80666f5c3b86ddda] that sometimes 
> explain's score can be different than the score requested by fields 
> parameter. Interestingly, Explain's scores would create a different ranking 
> than the original result list. This is something users experience, but it 
> cannot be re-produced deterministically.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (LUCENE-7132) BooleanQuery scores can be diff for same docs+sim when using coord (disagree with Explanation which doesn't change)

[Adrien Grand (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-7132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Adrien Grand resolved LUCENE-7132.
--
Resolution: Fixed

> BooleanQuery scores can be diff for same docs+sim when using coord (disagree 
> with Explanation which doesn't change)
> ---
>
> Key: LUCENE-7132
> URL: https://issues.apache.org/jira/browse/LUCENE-7132
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: core/search
>Affects Versions: 5.5
>Reporter: Ahmet Arslan
>Assignee: Steve Rowe
>Priority: Blocker
> Fix For: master (7.0), 5.5.2, 6.1
>
> Attachments: debug.xml, LUCENE-7132.patch, LUCENE-7132.patch, 
> LUCENE-7132.patch, LUCENE-7132.patch, LUCENE-7132.patch, LUCENE-7132.patch, 
> LUCENE-7132.patch, SOLR-8884.patch, SOLR-8884.patch
>
>
> Some of the folks 
> [reported|http://find.searchhub.org/document/80666f5c3b86ddda] that sometimes 
> explain's score can be different than the score requested by fields 
> parameter. Interestingly, Explain's scores would create a different ranking 
> than the original result list. This is something users experience, but it 
> cannot be re-produced deterministically.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Reopened] (LUCENE-7132) BooleanQuery scores can be diff for same docs+sim when using coord (disagree with Explanation which doesn't change)

[Adrien Grand (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-7132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Adrien Grand reopened LUCENE-7132:
--

Reopening to fix the fixVersion.

> BooleanQuery scores can be diff for same docs+sim when using coord (disagree 
> with Explanation which doesn't change)
> ---
>
> Key: LUCENE-7132
> URL: https://issues.apache.org/jira/browse/LUCENE-7132
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: core/search
>Affects Versions: 5.5
>Reporter: Ahmet Arslan
>Assignee: Steve Rowe
>Priority: Blocker
> Fix For: 6.1, 5.5.2, master (7.0)
>
> Attachments: debug.xml, LUCENE-7132.patch, LUCENE-7132.patch, 
> LUCENE-7132.patch, LUCENE-7132.patch, LUCENE-7132.patch, LUCENE-7132.patch, 
> LUCENE-7132.patch, SOLR-8884.patch, SOLR-8884.patch
>
>
> Some of the folks 
> [reported|http://find.searchhub.org/document/80666f5c3b86ddda] that sometimes 
> explain's score can be different than the score requested by fields 
> parameter. Interestingly, Explain's scores would create a different ranking 
> than the original result list. This is something users experience, but it 
> cannot be re-produced deterministically.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-7132) BooleanQuery scores can be diff for same docs+sim when using coord (disagree with Explanation which doesn't change)

[Adrien Grand (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-7132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Adrien Grand updated LUCENE-7132:
-
Fix Version/s: 5.5.2

> BooleanQuery scores can be diff for same docs+sim when using coord (disagree 
> with Explanation which doesn't change)
> ---
>
> Key: LUCENE-7132
> URL: https://issues.apache.org/jira/browse/LUCENE-7132
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: core/search
>Affects Versions: 5.5
>Reporter: Ahmet Arslan
>Assignee: Steve Rowe
>Priority: Blocker
> Fix For: 6.1, 5.5.2, master (7.0)
>
> Attachments: debug.xml, LUCENE-7132.patch, LUCENE-7132.patch, 
> LUCENE-7132.patch, LUCENE-7132.patch, LUCENE-7132.patch, LUCENE-7132.patch, 
> LUCENE-7132.patch, SOLR-8884.patch, SOLR-8884.patch
>
>
> Some of the folks 
> [reported|http://find.searchhub.org/document/80666f5c3b86ddda] that sometimes 
> explain's score can be different than the score requested by fields 
> parameter. Interestingly, Explain's scores would create a different ranking 
> than the original result list. This is something users experience, but it 
> cannot be re-produced deterministically.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Welcome Toke Eskildsen as a Lucene/Solr committer

[Stefan Matheis]

Welcome and congrats Toke!

-Stefan

On Feb 14, 2017 1:09 PM, "Jan Høydahl"  wrote:

> I'm pleased to announce that Toke Eskildsen has accepted the Lucene
> PMC's invitation to become a committer.
>
> Toke, it's tradition that you introduce yourself with a brief bio.
>
> Your account "toke" has been added to the “lucene" LDAP group, so you
> now have commit privileges. Please test this by adding yourself to the
> committers section of the Who We Are page on the website:
>  (instructions here
> ).
>
> The ASF dev page also has lots of useful links: <
> http://www.apache.org/dev/>.
>
> --
> Jan Høydahl, search solution architect
> Cominvent AS - www.cominvent.com
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org
>
>

[jira] [Resolved] (SOLR-8873) Enforce dataDir/instanceDir/ulogDir to be paths that contain only a controlled subset of characters

[JIRA]

 [ 
https://issues.apache.org/jira/browse/SOLR-8873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tomás Fernández Löbbe resolved SOLR-8873.
-
Resolution: Won't Fix

> Enforce dataDir/instanceDir/ulogDir to be paths that contain only a 
> controlled subset of characters
> ---
>
> Key: SOLR-8873
> URL: https://issues.apache.org/jira/browse/SOLR-8873
> Project: Solr
>  Issue Type: Improvement
>Reporter: Tomás Fernández Löbbe
> Attachments: SOLR-8873.patch
>
>
> We currently support any valid path for dataDir/instanceDir/ulogDir. I think 
> we should prevent special characters and restrict to a subset that is 
> commonly used and tested.
> My initial proposals it to allow the Java pattern: 
> {code:java}"^[a-zA-Z0-9\\.\\ \\-_/\"':]+$"{code} but I'm open to 
> suggestions. I'm not sure if there can be issues with HDFS paths (this 
> pattern does pass the tests we currently have), or some other use case I'm 
> not considering.
> I also think our tests should use all those characters randomly. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Welcome Toke Eskildsen as a Lucene/Solr committer

[Shawn Heisey]

On 2/14/2017 11:11 AM, Toke Eskildsen wrote:
> Thank you for the invitation and the warm welcome.
>
> I am a 43 year old Danish man, with a family and a job at the Royal Danish 
> Library, where I have been working mostly with search-related technology for 
> 10 years.
>
> I have done a fair bit of Lucene/Solr hacking during the years, with focus on 
> speed- and memory-optimizations. Implementing bit-packing structures, 
> eliminating steps in calculations and in general making more things possible 
> on less hardware is a bit of an obsession. I hope to continue in that 
> direction as a committer and am looking forward to a more controlled and 
> community-oriented way of writing code: The one-man-show is a lot of fun and 
> can work well for specific use cases, but it tends to get a bit out of 
> control and the result might not be that usable elsewhere.

Congratulations, welcome, and thanks for all your patience on the IRC
channel!

Thanks,
Shawn


-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-6.x-MacOSX (64bit/jdk1.8.0) - Build # 697 - Failure!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-6.x-MacOSX/697/
Java: 64bit/jdk1.8.0 -XX:+UseCompressedOops -XX:+UseG1GC

No tests ran.

Build Log:
[...truncated 17 lines...]
FATAL: java.io.IOException: Unexpected termination of the channel
java.io.EOFException
at 
java.io.ObjectInputStream$PeekInputStream.readFully(ObjectInputStream.java:2624)
at 
java.io.ObjectInputStream$BlockDataInputStream.readShort(ObjectInputStream.java:3099)
at 
java.io.ObjectInputStream.readStreamHeader(ObjectInputStream.java:853)
at java.io.ObjectInputStream.(ObjectInputStream.java:349)
at 
hudson.remoting.ObjectInputStreamEx.(ObjectInputStreamEx.java:48)
at 
hudson.remoting.AbstractSynchronousByteArrayCommandTransport.read(AbstractSynchronousByteArrayCommandTransport.java:34)
at 
hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:59)
Caused: java.io.IOException: Unexpected termination of the channel
at 
hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:73)
Caused: hudson.remoting.RequestAbortedException
at hudson.remoting.Request.abort(Request.java:307)
at hudson.remoting.Channel.terminate(Channel.java:888)
at 
hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:92)
at ..remote call to MacOSX VBOX(Native Method)
at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1537)
at hudson.remoting.Request.call(Request.java:172)
at hudson.remoting.Channel.call(Channel.java:821)
at 
hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:256)
at com.sun.proxy.$Proxy53.clean(Unknown Source)
at 
org.jenkinsci.plugins.gitclient.RemoteGitImpl.clean(RemoteGitImpl.java:449)
at 
hudson.plugins.git.extensions.impl.CleanBeforeCheckout.decorateFetchCommand(CleanBeforeCheckout.java:32)
at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:802)
at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1066)
at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1097)
at hudson.scm.SCM.checkout(SCM.java:496)
at hudson.model.AbstractProject.checkout(AbstractProject.java:1278)
at 
hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:604)
at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
at 
hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:529)
at hudson.model.Run.execute(Run.java:1728)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:98)
at hudson.model.Executor.run(Executor.java:405)
ERROR: Step ‘Archive the artifacts’ failed: no workspace for 
Lucene-Solr-6.x-MacOSX #697
ERROR: Step ‘Scan for compiler warnings’ failed: no workspace for 
Lucene-Solr-6.x-MacOSX #697
ERROR: Step ‘Publish JUnit test result report’ failed: no workspace for 
Lucene-Solr-6.x-MacOSX #697
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-master-MacOSX (64bit/jdk1.8.0) - Build # 3832 - Failure!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-MacOSX/3832/
Java: 64bit/jdk1.8.0 -XX:-UseCompressedOops -XX:+UseG1GC

No tests ran.

Build Log:
[...truncated 595 lines...]
ERROR: command execution failed.
ERROR: Step ‘Archive the artifacts’ failed: no workspace for 
Lucene-Solr-master-MacOSX #3832
ERROR: Step ‘Scan for compiler warnings’ failed: no workspace for 
Lucene-Solr-master-MacOSX #3832
ERROR: Step ‘Publish JUnit test result report’ failed: no workspace for 
Lucene-Solr-master-MacOSX #3832
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7465) Add a PatternTokenizer that uses Lucene's RegExp implementation

[Steve Rowe (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15866514#comment-15866514
 ] 

Steve Rowe commented on LUCENE-7465:


My Jenkins found a reproducing seed on master for a TestRandomChains failure 
that implicates the new tokenizer:

{noformat}
   [junit4] Suite: org.apache.lucene.analysis.core.TestRandomChains
   [junit4]   2> TEST FAIL: useCharFilter=false text='puzoh 
\u6a8b\u59e2\u96aa\u85f0\u614a\u9010\u7782\u5547 
\uef27\uda09\uddd2\u9b9c\u056e\u33f0 W\udb24\udce6> 
\u2d12\u2d23\u2d05\u2d1c\u2d23 *\ud9f0\udc74\uea94\ub9c6 pev trjrbvcwb tzzntfd 
y|)]){1  gmabf'
   [junit4]   2> TEST FAIL: useCharFilter=false text='puzoh 
\u6a8b\u59e2\u96aa\u85f0\u614a\u9010\u7782\u5547 
\uef27\uda09\uddd2\u9b9c\u056e\u33f0 W\udb24\udce6> 
\u2d12\u2d23\u2d05\u2d1c\u2d23 *\ud9f0\udc74\uea94\ub9c6 pev trjrbvcwb tzzntfd 
y|)]){1  gmabf'
   [junit4]   2> TEST FAIL: useCharFilter=false text='puzoh 
\u6a8b\u59e2\u96aa\u85f0\u614a\u9010\u7782\u5547 
\uef27\uda09\uddd2\u9b9c\u056e\u33f0 W\udb24\udce6> 
\u2d12\u2d23\u2d05\u2d1c\u2d23 *\ud9f0\udc74\uea94\ub9c6 pev trjrbvcwb tzzntfd 
y|)]){1  gmabf'
   [junit4]   2> feb 14, 2017 2:13:13 P.M. 
com.carrotsearch.randomizedtesting.RandomizedRunner$QueueUncaughtExceptionsHandler
 uncaughtException
   [junit4]   2> WARNING: Uncaught exception in thread: 
Thread[Thread-17,5,TGRP-TestRandomChains]
   [junit4]   2> java.lang.AssertionError: finalOffset expected:<79> but 
was:<65>
   [junit4]   2>at 
__randomizedtesting.SeedInfo.seed([3ABEF2F287EE4968]:0)
   [junit4]   2>at org.junit.Assert.fail(Assert.java:93)
   [junit4]   2>at org.junit.Assert.failNotEquals(Assert.java:647)
   [junit4]   2>at org.junit.Assert.assertEquals(Assert.java:128)
   [junit4]   2>at org.junit.Assert.assertEquals(Assert.java:472)
   [junit4]   2>at 
org.apache.lucene.analysis.BaseTokenStreamTestCase.assertTokenStreamContents(BaseTokenStreamTestCase.java:293)
   [junit4]   2>at 
org.apache.lucene.analysis.BaseTokenStreamTestCase.assertTokenStreamContents(BaseTokenStreamTestCase.java:308)
   [junit4]   2>at 
org.apache.lucene.analysis.BaseTokenStreamTestCase.assertTokenStreamContents(BaseTokenStreamTestCase.java:312)
   [junit4]   2>at 
org.apache.lucene.analysis.BaseTokenStreamTestCase.checkAnalysisConsistency(BaseTokenStreamTestCase.java:843)
   [junit4]   2>at 
org.apache.lucene.analysis.BaseTokenStreamTestCase.checkRandomData(BaseTokenStreamTestCase.java:642)
   [junit4]   2>at 
org.apache.lucene.analysis.BaseTokenStreamTestCase.access$000(BaseTokenStreamTestCase.java:66)
   [junit4]   2>at 
org.apache.lucene.analysis.BaseTokenStreamTestCase$AnalysisThread.run(BaseTokenStreamTestCase.java:510)
   [junit4]   2> 
   [junit4]   2> feb 14, 2017 2:13:13 P.M. 
com.carrotsearch.randomizedtesting.RandomizedRunner$QueueUncaughtExceptionsHandler
 uncaughtException
   [junit4]   2> WARNING: Uncaught exception in thread: 
Thread[Thread-18,5,TGRP-TestRandomChains]
   [junit4]   2> java.lang.AssertionError: finalOffset expected:<79> but 
was:<65>
   [junit4]   2>at 
__randomizedtesting.SeedInfo.seed([3ABEF2F287EE4968]:0)
   [junit4]   2>at org.junit.Assert.fail(Assert.java:93)
   [junit4]   2>at org.junit.Assert.failNotEquals(Assert.java:647)
   [junit4]   2>at org.junit.Assert.assertEquals(Assert.java:128)
   [junit4]   2>at org.junit.Assert.assertEquals(Assert.java:472)
   [junit4]   2>at 
org.apache.lucene.analysis.BaseTokenStreamTestCase.assertTokenStreamContents(BaseTokenStreamTestCase.java:293)
   [junit4]   2>at 
org.apache.lucene.analysis.BaseTokenStreamTestCase.assertTokenStreamContents(BaseTokenStreamTestCase.java:308)
   [junit4]   2>at 
org.apache.lucene.analysis.BaseTokenStreamTestCase.assertTokenStreamContents(BaseTokenStreamTestCase.java:312)
   [junit4]   2>at 
org.apache.lucene.analysis.BaseTokenStreamTestCase.checkAnalysisConsistency(BaseTokenStreamTestCase.java:843)
   [junit4]   2>at 
org.apache.lucene.analysis.BaseTokenStreamTestCase.checkRandomData(BaseTokenStreamTestCase.java:642)
   [junit4]   2>at 
org.apache.lucene.analysis.BaseTokenStreamTestCase.access$000(BaseTokenStreamTestCase.java:66)
   [junit4]   2>at 
org.apache.lucene.analysis.BaseTokenStreamTestCase$AnalysisThread.run(BaseTokenStreamTestCase.java:510)
   [junit4]   2> 
   [junit4]   2> feb 14, 2017 2:13:13 P.M. 
com.carrotsearch.randomizedtesting.RandomizedRunner$QueueUncaughtExceptionsHandler
 uncaughtException
   [junit4]   2> WARNING: Uncaught exception in thread: 
Thread[Thread-19,5,TGRP-TestRandomChains]
   [junit4]   2> java.lang.AssertionError: finalOffset expected:<79> but 
was:<65>
   [junit4]   2>at 
__randomizedtesting.SeedInfo.seed([3ABEF2F287EE4968]:0)
   [junit4]   2>at org.junit.Assert.fail(Assert.java:93)
 

Re: VOTE: Apache Solr Ref Guide for 6.4 (RC1)

[Cassandra Targett]

Because Confluence does that with pages that have "special" characters
in the page name.

On Tue, Feb 14, 2017 at 12:56 PM, Alexandre Rafalovitch
 wrote:
> Just noticed this. Is there any reason why CDCR page is missing a page
> alias and uses a number instead:
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=62687462
>
> Regards,
>Alex.
> 
> http://www.solr-start.com/ - Resources for Solr users, new and experienced
>
>
> On 14 February 2017 at 12:32, Cassandra Targett  wrote:
>> Thanks everyone, the vote passed.
>>
>> I was traveling all last week, so my apologies for being really behind
>> on this. I'll start the release process now.
>>
>> On Mon, Feb 6, 2017 at 8:20 PM, Steve Rowe  wrote:
>>> +1
>>>
>>> --
>>> Steve
>>> www.lucidowrks.com
>>>
 On Feb 6, 2017, at 12:24 PM, David Smiley  wrote:

 +1

 On Mon, Feb 6, 2017 at 1:59 PM Cassandra Targett  
 wrote:
 Please vote for the release of the Apache Solr Reference Guide for 6.4.

 Artifacts are available from:
 https://dist.apache.org/repos/dist/dev/lucene/solr/ref-guide/apache-solr-ref-guide-6.4-RC1

 $ cat apache-solr-ref-guide-6.4-RC1/apache-solr-ref-guide-6.4.pdf.sha1
 4396f8f7a26e55cb920b3b6ac82627ca96814014  apache-solr-ref-guide-6.4.pdf

 Here's my +1.

 Thanks,
 Cassandra

 -
 To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
 For additional commands, e-mail: dev-h...@lucene.apache.org

 --
 Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker
 LinkedIn: http://linkedin.com/in/davidwsmiley | Book: 
 http://www.solrenterprisesearchserver.com
>>>
>>>
>>> -
>>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
>>> For additional commands, e-mail: dev-h...@lucene.apache.org
>>>
>>
>> -
>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
>> For additional commands, e-mail: dev-h...@lucene.apache.org
>>
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org
>

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: VOTE: Apache Solr Ref Guide for 6.4 (RC1)

[Alexandre Rafalovitch]

Just noticed this. Is there any reason why CDCR page is missing a page
alias and uses a number instead:
https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=62687462

Regards,
   Alex.

http://www.solr-start.com/ - Resources for Solr users, new and experienced


On 14 February 2017 at 12:32, Cassandra Targett  wrote:
> Thanks everyone, the vote passed.
>
> I was traveling all last week, so my apologies for being really behind
> on this. I'll start the release process now.
>
> On Mon, Feb 6, 2017 at 8:20 PM, Steve Rowe  wrote:
>> +1
>>
>> --
>> Steve
>> www.lucidowrks.com
>>
>>> On Feb 6, 2017, at 12:24 PM, David Smiley  wrote:
>>>
>>> +1
>>>
>>> On Mon, Feb 6, 2017 at 1:59 PM Cassandra Targett  
>>> wrote:
>>> Please vote for the release of the Apache Solr Reference Guide for 6.4.
>>>
>>> Artifacts are available from:
>>> https://dist.apache.org/repos/dist/dev/lucene/solr/ref-guide/apache-solr-ref-guide-6.4-RC1
>>>
>>> $ cat apache-solr-ref-guide-6.4-RC1/apache-solr-ref-guide-6.4.pdf.sha1
>>> 4396f8f7a26e55cb920b3b6ac82627ca96814014  apache-solr-ref-guide-6.4.pdf
>>>
>>> Here's my +1.
>>>
>>> Thanks,
>>> Cassandra
>>>
>>> -
>>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
>>> For additional commands, e-mail: dev-h...@lucene.apache.org
>>>
>>> --
>>> Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker
>>> LinkedIn: http://linkedin.com/in/davidwsmiley | Book: 
>>> http://www.solrenterprisesearchserver.com
>>
>>
>> -
>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
>> For additional commands, e-mail: dev-h...@lucene.apache.org
>>
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org
>

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: immutable configsets in SOLR

[Hrishikesh Gadre]

Thanks David! I filed SOLR-10137 to fix this issue.

-Hrishikesh


On Fri, Feb 10, 2017 at 9:21 PM, David Smiley 
wrote:

> Makes sense to me -- new configsets _should_ be mutable by default, even
> if it's from a copy that's immutable.
>
> On Thu, Feb 9, 2017 at 2:38 PM Hrishikesh Gadre 
> wrote:
>
>> Hi,
>>
>> SOLR-7742  introduced
>> support for "immutable" config-sets in SOLR. This allows system
>> administrators to define config-set templates a-priori (e.g. based on
>> managed schema vs schemaless etc.) and then users can use configset APIs to
>> define new configurations for the collections they want to create.
>>
>> https://cwiki.apache.org/confluence/display/solr/ConfigSets+API
>>
>> One problem I notice here (based on Solr 6.3) is that when a user creates
>> a new config-set by referring to an existing "immutable" config-set, the
>> new config-set is also defined as "immutable" unless user explicitly
>> specify immutable=false as part of the create operation.
>>
>> This is a bit problematic since Solr does not allow deleting an immutable
>> config-set via an API.
>> https://github.com/apache/lucene-solr/blob/0b817e6e495c40496b7cedc6f06060
>> e43e5e2afc/solr/core/src/java/org/apache/solr/cloud/
>> OverseerConfigSetMessageHandler.java#L369-L371
>>
>> I think we should change this behavior so that when a user creates a new
>> config set using an API, it is always mutable (i.e. immutable=false). This
>> will enable users to delete the config sets they created using the API.
>>
>> What do you think? I can file a jira if this make sense.
>>
>> Thanks
>> Hrishikesh
>>
>>
>> --
> Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker
> LinkedIn: http://linkedin.com/in/davidwsmiley | Book: http://www.
> solrenterprisesearchserver.com
>

[jira] [Created] (SOLR-10137) Configsets created via API should always be mutable

[Hrishikesh Gadre (JIRA)]

Hrishikesh Gadre created SOLR-10137:
---

 Summary: Configsets created via API should always be mutable
 Key: SOLR-10137
 URL: https://issues.apache.org/jira/browse/SOLR-10137
 Project: Solr
  Issue Type: Bug
  Security Level: Public (Default Security Level. Issues are Public)
Reporter: Hrishikesh Gadre


Please refer to this discussion for details,

https://marc.info/?l=solr-dev=148679049516375=4



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-10136) TestReqParamsAPI regularly fails on Policeman Jenkins

[Christine Poerschke (JIRA)]

Christine Poerschke created SOLR-10136:
--

 Summary: TestReqParamsAPI regularly fails on Policeman Jenkins
 Key: SOLR-10136
 URL: https://issues.apache.org/jira/browse/SOLR-10136
 Project: Solr
  Issue Type: Test
  Security Level: Public (Default Security Level. Issues are Public)
Reporter: Christine Poerschke
Priority: Minor


{{org.apache.solr.handler.TestReqParamsAPI.test}} regularly fails though 
interestly only on Policeman Jenkins and not on Apache Jenkins e.g. 
https://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/18962/testReport/junit/org.apache.solr.handler/TestReqParamsAPI/test/

The Feb 9th SOLR-10032 report categorised the test as _flakey_.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10136) TestReqParamsAPI regularly fails on Policeman Jenkins

[Christine Poerschke (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15866359#comment-15866359
 ] 

Christine Poerschke commented on SOLR-10136:


{code}
ant beast -Dbeast.iters=10 -Dtests.dups=10 -Dtestcase=TestReqParamsAPI
{code}
locally for me fails though interestingly without the {{-Dtests.dups=10}} it 
passes.

Had a bit of a look at the [TestReqParamsAPI.java 
code|https://github.com/apache/lucene-solr/blob/master/solr/core/src/test/org/apache/solr/handler/TestReqParamsAPI.java]
 but couldn't locate a clear point at which to explore making changes.

> TestReqParamsAPI regularly fails on Policeman Jenkins
> -
>
> Key: SOLR-10136
> URL: https://issues.apache.org/jira/browse/SOLR-10136
> Project: Solr
>  Issue Type: Test
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Christine Poerschke
>Priority: Minor
>
> {{org.apache.solr.handler.TestReqParamsAPI.test}} regularly fails though 
> interestly only on Policeman Jenkins and not on Apache Jenkins e.g. 
> https://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/18962/testReport/junit/org.apache.solr.handler/TestReqParamsAPI/test/
> The Feb 9th SOLR-10032 report categorised the test as _flakey_.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (SOLR-8396) Add support for PointFields in Solr

[JIRA]

 [ 
https://issues.apache.org/jira/browse/SOLR-8396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tomás Fernández Löbbe resolved SOLR-8396.
-
   Resolution: Fixed
Fix Version/s: master (7.0)
   6.5

Resolving this Jira since the main work is done. Followup work can be done in 
related Jiras

> Add support for PointFields in Solr
> ---
>
> Key: SOLR-8396
> URL: https://issues.apache.org/jira/browse/SOLR-8396
> Project: Solr
>  Issue Type: Improvement
>Reporter: Ishan Chattopadhyaya
>Assignee: Tomás Fernández Löbbe
> Fix For: 6.5, master (7.0)
>
> Attachments: SOLR-8396.patch, SOLR-8396.patch, SOLR-8396.patch, 
> SOLR-8396.patch, SOLR-8396.patch, SOLR-8396.patch, SOLR-8396.patch, 
> SOLR-8396.patch, SOLR-8396.patch, SOLR-8396.patch
>
>
> In LUCENE-6917, [~mikemccand] mentioned that DimensionalValues are better 
> than NumericFields in most respects. We should explore the benefits of using 
> it in Solr and hence, if appropriate, switch over to using them.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Welcome Toke Eskildsen as a Lucene/Solr committer

[Tomás Fernández Löbbe]

Welcome Toke!

On Tue, Feb 14, 2017 at 10:11 AM, Toke Eskildsen  wrote:

> Thank you for the invitation and the warm welcome.
>
>
> I am a 43 year old Danish man, with a family and a job at the Royal Danish
> Library, where I have been working mostly with search-related technology
> for 10 years.
>
> I have done a fair bit of Lucene/Solr hacking during the years, with focus
> on speed- and memory-optimizations. Implementing bit-packing structures,
> eliminating steps in calculations and in general making more things
> possible on less hardware is a bit of an obsession. I hope to continue in
> that direction as a committer and am looking forward to a more controlled
> and community-oriented way of writing code: The one-man-show is a lot of
> fun and can work well for specific use cases, but it tends to get a bit out
> of control and the result might not be that usable elsewhere.
>
> Happy to be here,
> Toke
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org
>
>

[jira] [Updated] (SOLR-10011) Refactor PointField & TrieField to share common code

[JIRA]

 [ 
https://issues.apache.org/jira/browse/SOLR-10011?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tomás Fernández Löbbe updated SOLR-10011:
-
Fix Version/s: (was: 6.4)
   6.5

> Refactor PointField & TrieField to share common code
> 
>
> Key: SOLR-10011
> URL: https://issues.apache.org/jira/browse/SOLR-10011
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Ishan Chattopadhyaya
>Assignee: Ishan Chattopadhyaya
> Fix For: 6.5, master (7.0)
>
> Attachments: SOLR-10011.patch, SOLR-10011.patch, SOLR-10011.patch, 
> SOLR-10011.patch, SOLR-10011.patch
>
>
> We should eliminate PointTypes and TrieTypes enum to have a common enum for 
> both. That would enable us to share a lot of code between the two field types.
> In the process, fix the bug:
> PointFields with indexed=false, docValues=true seem to be using 
> (Int|Double|Float|Long)Point.newRangeQuery() for performing exact matches and 
> range queries. However, they should instead be using DocValues based range 
> query.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (SOLR-9996) Unstored PointFields return types are wrong

[JIRA]

 [ 
https://issues.apache.org/jira/browse/SOLR-9996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tomás Fernández Löbbe resolved SOLR-9996.
-
   Resolution: Fixed
Fix Version/s: master (7.0)
   6.5

Backported the fix. Resolving

> Unstored PointFields return types are wrong
> ---
>
> Key: SOLR-9996
> URL: https://issues.apache.org/jira/browse/SOLR-9996
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Ishan Chattopadhyaya
>Assignee: Ishan Chattopadhyaya
> Fix For: 6.5, master (7.0)
>
> Attachments: SOLR-9996.patch, SOLR-9996.patch, SOLR-9996.patch
>
>
> Seems like unstored PointFields return Long types, ignoring the actual type.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (SOLR-10011) Refactor PointField & TrieField to share common code

[JIRA]

 [ 
https://issues.apache.org/jira/browse/SOLR-10011?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tomás Fernández Löbbe resolved SOLR-10011.
--
   Resolution: Fixed
Fix Version/s: master (7.0)
   6.4

Backported the changes. resolving

> Refactor PointField & TrieField to share common code
> 
>
> Key: SOLR-10011
> URL: https://issues.apache.org/jira/browse/SOLR-10011
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Ishan Chattopadhyaya
>Assignee: Ishan Chattopadhyaya
> Fix For: 6.4, master (7.0)
>
> Attachments: SOLR-10011.patch, SOLR-10011.patch, SOLR-10011.patch, 
> SOLR-10011.patch, SOLR-10011.patch
>
>
> We should eliminate PointTypes and TrieTypes enum to have a common enum for 
> both. That would enable us to share a lot of code between the two field types.
> In the process, fix the bug:
> PointFields with indexed=false, docValues=true seem to be using 
> (Int|Double|Float|Long)Point.newRangeQuery() for performing exact matches and 
> range queries. However, they should instead be using DocValues based range 
> query.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (SOLR-9987) Implement support for multi-valued DocValues in PointFields

[JIRA]

 [ 
https://issues.apache.org/jira/browse/SOLR-9987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tomás Fernández Löbbe resolved SOLR-9987.
-
   Resolution: Fixed
Fix Version/s: master (7.0)
   6.5

> Implement support for multi-valued DocValues in PointFields
> ---
>
> Key: SOLR-9987
> URL: https://issues.apache.org/jira/browse/SOLR-9987
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Tomás Fernández Löbbe
>Assignee: Tomás Fernández Löbbe
> Fix For: 6.5, master (7.0)
>
> Attachments: SOLR-9987.patch, SOLR-9987.patch
>
>
> This is not currently supported, and since PointFields can't use FieldCache, 
> faceting, stats, etc is not supported on multi-valued point fields. Followup 
> task of SOLR-8396



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Welcome Toke Eskildsen as a Lucene/Solr committer

[Toke Eskildsen]

Thank you for the invitation and the warm welcome.


I am a 43 year old Danish man, with a family and a job at the Royal Danish 
Library, where I have been working mostly with search-related technology for 10 
years.

I have done a fair bit of Lucene/Solr hacking during the years, with focus on 
speed- and memory-optimizations. Implementing bit-packing structures, 
eliminating steps in calculations and in general making more things possible on 
less hardware is a bit of an obsession. I hope to continue in that direction as 
a committer and am looking forward to a more controlled and community-oriented 
way of writing code: The one-man-show is a lot of fun and can work well for 
specific use cases, but it tends to get a bit out of control and the result 
might not be that usable elsewhere.

Happy to be here,
Toke

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-7449) Add CROSSES query support to RangeField

[Nicholas Knize (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-7449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nicholas Knize updated LUCENE-7449:
---
Attachment: (was: LUCENE-7449.patch)

> Add CROSSES query support to RangeField
> ---
>
> Key: LUCENE-7449
> URL: https://issues.apache.org/jira/browse/LUCENE-7449
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Nicholas Knize
> Attachments: LUCENE-7449.patch, LUCENE-7449.patch
>
>
> {{RangeField}} currently supports {{INTERSECTS}}, {{WITHIN}}, and 
> {{CONTAINS}} query behavior. This feature adds support for an explicit 
> {{CROSSES}} query. Unlike {{INTERSECT}} and {{OVERLAP}} queries the 
> {{CROSSES}} query finds any indexed ranges whose interior (within range) 
> intersect the interior AND exterior (outside range) of the query range.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-7449) Add CROSSES query support to RangeField

[Nicholas Knize (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-7449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nicholas Knize updated LUCENE-7449:
---
Attachment: LUCENE-7449.patch

Attached wrong patch... here's the correct one

> Add CROSSES query support to RangeField
> ---
>
> Key: LUCENE-7449
> URL: https://issues.apache.org/jira/browse/LUCENE-7449
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Nicholas Knize
> Attachments: LUCENE-7449.patch, LUCENE-7449.patch
>
>
> {{RangeField}} currently supports {{INTERSECTS}}, {{WITHIN}}, and 
> {{CONTAINS}} query behavior. This feature adds support for an explicit 
> {{CROSSES}} query. Unlike {{INTERSECT}} and {{OVERLAP}} queries the 
> {{CROSSES}} query finds any indexed ranges whose interior (within range) 
> intersect the interior AND exterior (outside range) of the query range.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-10121) BlockCache corruption with high concurrency

[Yonik Seeley (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-10121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yonik Seeley updated SOLR-10121:

Attachment: SOLR-10121.patch

Here's a draft patch to fix the currently identified concurrency issues in 
BlockCache.
- fetch() checked isRemoved() before the read from the block, but the block 
could be reused after that point (i.e. before or during the read), causing the 
wrong data to be returned.
- store() allowed existing blocks to be updated, but resulted in corruption.  
The reason is that if one retrieves an existing block, one does not know when 
that block may stop being used for one key and start being used for another 
key.  To safely do this, one would require write locks.  Since we don't need 
the functionality, I simply failed the case of trying to cache/update a block 
already in the cache.

> BlockCache corruption with high concurrency
> ---
>
> Key: SOLR-10121
> URL: https://issues.apache.org/jira/browse/SOLR-10121
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Yonik Seeley
>Assignee: Yonik Seeley
> Attachments: SOLR-10121.patch
>
>
> Improving the tests of the BlockCache in SOLR-10116 uncovered a corruption 
> bug (either that or the test is flawed... TBD).
> The failing test is TestBlockCache.testBlockCacheConcurrent()



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-7449) Add CROSSES query support to RangeField

[Nicholas Knize (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-7449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nicholas Knize updated LUCENE-7449:
---
Attachment: LUCENE-7449.patch

Thanks for the suggestion [~dsmiley]! And my apologies for the looong delay on 
this issue. I had to put it on the back burner but was recently able to get 
back to it. Your suggestion made it much cleaner, and easier to read, I think. 
I have attached an updated patch for review. Let me know if you have any 
thoughts and I'll get this closed up as soon as possible.

> Add CROSSES query support to RangeField
> ---
>
> Key: LUCENE-7449
> URL: https://issues.apache.org/jira/browse/LUCENE-7449
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Nicholas Knize
> Attachments: LUCENE-7449.patch, LUCENE-7449.patch
>
>
> {{RangeField}} currently supports {{INTERSECTS}}, {{WITHIN}}, and 
> {{CONTAINS}} query behavior. This feature adds support for an explicit 
> {{CROSSES}} query. Unlike {{INTERSECT}} and {{OVERLAP}} queries the 
> {{CROSSES}} query finds any indexed ranges whose interior (within range) 
> intersect the interior AND exterior (outside range) of the query range.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-10114) child documents lack _version_, susceptible to reordered delete-by-query

[Mano Kovacs (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-10114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mano Kovacs updated SOLR-10114:
---
Attachment: SOLR-10114.patch

Adding patch with
- fix by adding version for childdocs if there is
- fix by using same insert-or-update logic when handling reordered DBQ
- recovery and peersync tests.


> child documents lack _version_, susceptible to reordered delete-by-query 
> -
>
> Key: SOLR-10114
> URL: https://issues.apache.org/jira/browse/SOLR-10114
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Yonik Seeley
> Attachments: SOLR-10114.patch, SOLR-10114-validation.patch
>
>
> It looks like when a block of documents is indexed, child documents get no 
> \_version\_ field.  This means (among other potential issues) that a 
> delete-by-query that is reordered will cause matching child documents to be 
> deleted.  DBQ normally prevents deleting newer docs by including a 
> restriction on \_version\_, which doesn't work for anything lacking that 
> field.  Re-ordered delete-by-term of any child docs would also be affected 
> (although it should be a much rarer issue.)
> The leading candidate for a fix is to use the exact same \_version\_ for all 
> child docs.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (SOLR-10104) BlockDirectoryCache release hooks do not work with multiple directories

[Mark Miller (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-10104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mark Miller resolved SOLR-10104.

   Resolution: Fixed
Fix Version/s: master (7.0)
   6.5

Thanks Mike!

> BlockDirectoryCache release hooks do not work with multiple directories
> ---
>
> Key: SOLR-10104
> URL: https://issues.apache.org/jira/browse/SOLR-10104
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: hdfs
>Affects Versions: 6.4
>Reporter: Mike Drob
>Assignee: Mark Miller
> Fix For: 6.5, master (7.0)
>
>
> https://github.com/apache/lucene-solr/blob/5738c293f0c3f346b3e3e52c937183060d59cba1/solr/core/src/java/org/apache/solr/store/blockcache/BlockDirectoryCache.java#L53
> {code}
> if (releaseBlocks) {
>   keysToRelease = Collections.newSetFromMap(new 
> ConcurrentHashMap(1024, 0.75f, 512));
>   blockCache.setOnRelease(new OnRelease() {
> 
> @Override
> public void release(BlockCacheKey key) {
>   keysToRelease.remove(key);
> }
>   });
> }
> {code}
> If we're using the global block cache option and create multiple directories 
> using the same factory, we will lose the release hook for the first 
> directory. I think we can verify that by creating a server with multiple 
> cores.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: VOTE: Apache Solr Ref Guide for 6.4 (RC1)

[Cassandra Targett]

Thanks everyone, the vote passed.

I was traveling all last week, so my apologies for being really behind
on this. I'll start the release process now.

On Mon, Feb 6, 2017 at 8:20 PM, Steve Rowe  wrote:
> +1
>
> --
> Steve
> www.lucidowrks.com
>
>> On Feb 6, 2017, at 12:24 PM, David Smiley  wrote:
>>
>> +1
>>
>> On Mon, Feb 6, 2017 at 1:59 PM Cassandra Targett  wrote:
>> Please vote for the release of the Apache Solr Reference Guide for 6.4.
>>
>> Artifacts are available from:
>> https://dist.apache.org/repos/dist/dev/lucene/solr/ref-guide/apache-solr-ref-guide-6.4-RC1
>>
>> $ cat apache-solr-ref-guide-6.4-RC1/apache-solr-ref-guide-6.4.pdf.sha1
>> 4396f8f7a26e55cb920b3b6ac82627ca96814014  apache-solr-ref-guide-6.4.pdf
>>
>> Here's my +1.
>>
>> Thanks,
>> Cassandra
>>
>> -
>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
>> For additional commands, e-mail: dev-h...@lucene.apache.org
>>
>> --
>> Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker
>> LinkedIn: http://linkedin.com/in/davidwsmiley | Book: 
>> http://www.solrenterprisesearchserver.com
>
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org
>

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9846) OverseerAutoReplicaFailoverThread can leak out of unit tests.

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9846?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15866195#comment-15866195
 ] 

ASF subversion and git services commented on SOLR-9846:
---

Commit 17f441e8e263ce29e2b1da0aa11506a9541e3d3a in lucene-solr's branch 
refs/heads/branch_6x from markrmiller
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=17f441e ]

SOLR-9846: Track Overseer close better.


> OverseerAutoReplicaFailoverThread can leak out of unit tests.
> -
>
> Key: SOLR-9846
> URL: https://issues.apache.org/jira/browse/SOLR-9846
> Project: Solr
>  Issue Type: Test
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Mark Miller
>Assignee: Mark Miller
> Fix For: 6.4, master (7.0)
>
>
> We should interrupt it on close.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10104) BlockDirectoryCache release hooks do not work with multiple directories

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15866196#comment-15866196
 ] 

ASF subversion and git services commented on SOLR-10104:


Commit 7f77f71201250c143c0438d5bb34a59ee964d41b in lucene-solr's branch 
refs/heads/branch_6x from markrmiller
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=7f77f71 ]

SOLR-10104: BlockDirectoryCache release hooks do not work with multiple 
directories.


> BlockDirectoryCache release hooks do not work with multiple directories
> ---
>
> Key: SOLR-10104
> URL: https://issues.apache.org/jira/browse/SOLR-10104
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: hdfs
>Affects Versions: 6.4
>Reporter: Mike Drob
>Assignee: Mark Miller
>
> https://github.com/apache/lucene-solr/blob/5738c293f0c3f346b3e3e52c937183060d59cba1/solr/core/src/java/org/apache/solr/store/blockcache/BlockDirectoryCache.java#L53
> {code}
> if (releaseBlocks) {
>   keysToRelease = Collections.newSetFromMap(new 
> ConcurrentHashMap(1024, 0.75f, 512));
>   blockCache.setOnRelease(new OnRelease() {
> 
> @Override
> public void release(BlockCacheKey key) {
>   keysToRelease.remove(key);
> }
>   });
> }
> {code}
> If we're using the global block cache option and create multiple directories 
> using the same factory, we will lose the release hook for the first 
> directory. I think we can verify that by creating a server with multiple 
> cores.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Closed] (SOLR-10111) MBeansHandlerTest.testDiff regularly fails

[Christine Poerschke (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-10111?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Christine Poerschke closed SOLR-10111.
--
Resolution: Duplicate

> MBeansHandlerTest.testDiff regularly fails
> --
>
> Key: SOLR-10111
> URL: https://issues.apache.org/jira/browse/SOLR-10111
> Project: Solr
>  Issue Type: Test
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Christine Poerschke
>Assignee: Christine Poerschke
>Priority: Minor
> Attachments: SOLR-10111.patch
>
>
> {{org.apache.solr.handler.admin.MBeansHandlerTest.testDiff}} regularly fails, 
> both for me locally and on the Jenkins instances that email the dev mailing 
> list e.g. 
> https://builds.apache.org/job/Lucene-Solr-Tests-6.x/708/testReport/junit/org.apache.solr.handler.admin/MBeansHandlerTest/testDiff/
> This appears to be an issue with the test itself, patch to follow.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10104) BlockDirectoryCache release hooks do not work with multiple directories

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15866189#comment-15866189
 ] 

ASF subversion and git services commented on SOLR-10104:


Commit a1f114f70f3800292c25be08213edf39b3e37f6a in lucene-solr's branch 
refs/heads/master from markrmiller
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=a1f114f ]

SOLR-10104: BlockDirectoryCache release hooks do not work with multiple 
directories.

# Conflicts:
#   solr/CHANGES.txt


> BlockDirectoryCache release hooks do not work with multiple directories
> ---
>
> Key: SOLR-10104
> URL: https://issues.apache.org/jira/browse/SOLR-10104
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: hdfs
>Affects Versions: 6.4
>Reporter: Mike Drob
>Assignee: Mark Miller
>
> https://github.com/apache/lucene-solr/blob/5738c293f0c3f346b3e3e52c937183060d59cba1/solr/core/src/java/org/apache/solr/store/blockcache/BlockDirectoryCache.java#L53
> {code}
> if (releaseBlocks) {
>   keysToRelease = Collections.newSetFromMap(new 
> ConcurrentHashMap(1024, 0.75f, 512));
>   blockCache.setOnRelease(new OnRelease() {
> 
> @Override
> public void release(BlockCacheKey key) {
>   keysToRelease.remove(key);
> }
>   });
> }
> {code}
> If we're using the global block cache option and create multiple directories 
> using the same factory, we will lose the release hook for the first 
> directory. I think we can verify that by creating a server with multiple 
> cores.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9846) OverseerAutoReplicaFailoverThread can leak out of unit tests.

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9846?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15866187#comment-15866187
 ] 

ASF subversion and git services commented on SOLR-9846:
---

Commit a243befdbb4c011c33c27b1b864d4a202b401675 in lucene-solr's branch 
refs/heads/master from markrmiller
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=a243bef ]

SOLR-9846: Track Overseer close better.


> OverseerAutoReplicaFailoverThread can leak out of unit tests.
> -
>
> Key: SOLR-9846
> URL: https://issues.apache.org/jira/browse/SOLR-9846
> Project: Solr
>  Issue Type: Test
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Mark Miller
>Assignee: Mark Miller
> Fix For: 6.4, master (7.0)
>
>
> We should interrupt it on close.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10081) Web UI (Generate Training Data ) for LTR plugin

[Christine Poerschke (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15866180#comment-15866180
 ] 

Christine Poerschke commented on SOLR-10081:


Hello [~qfdk],

Thank you for your interest in the Apache Solr Learning-to-Rank (LTR) plugin 
and thanks for opening this ticket.

My initial thoughts would be that your GenTrainingDataSolr web UI is 
conceptually independent of any particular search engine and that as such the 
web UI would best be maintained and shared independently e.g. via a repo such 
as your [GenTrainingDataSolr|https://github.com/qfdk/GenTrainingDataSolr] repo.

In your repo seeing the _Apache Solr_ vs. _Apache Solr with LTR_ comparison in 
your [second 
screenshot|https://github.com/qfdk/GenTrainingDataSolr/blob/master/imgs/2.png] 
reminded me of the [Quepid|http://quepid.com/] tool. If you haven't come across 
Quepid before then [this blog 
post|http://www.flax.co.uk/blog/2016/07/08/setting-first-quepid-test-case/] is 
great to get started.

Regards,

Christine



PS: [~upayavira] and [~romseygeek] - would you have any thoughts to share here 
perhaps e.g. based on your work with the Solr Admin UI and/or your experience 
with Quepid?


> Web UI (Generate Training Data ) for LTR plugin
> ---
>
> Key: SOLR-10081
> URL: https://issues.apache.org/jira/browse/SOLR-10081
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: qfdk
>
> This is a New Feature for LTR plugin. It will help you to generate the 
> training data (HUMAN_JUDGEMENT) by using score between 0-4. Live demo 
> http://ltr.qfdk.me



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re:Welcome Toke Eskildsen as a Lucene/Solr committer

[Christine Poerschke (BLOOMBERG/ LONDON)]

Welcome Toke!

- Original Message -
From: dev@lucene.apache.org
To: dev@lucene.apache.org
At: 02/14/17 12:09:32

I'm pleased to announce that Toke Eskildsen has accepted the Lucene
PMC's invitation to become a committer.

Toke, it's tradition that you introduce yourself with a brief bio.

Your account "toke" has been added to the “lucene" LDAP group, so you
now have commit privileges. Please test this by adding yourself to the
committers section of the Who We Are page on the website:
 (instructions here
).

The ASF dev page also has lots of useful links: .

--
Jan Høydahl, search solution architect
Cominvent AS - www.cominvent.com

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-6.x-Solaris (64bit/jdk1.8.0) - Build # 667 - Unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-6.x-Solaris/667/
Java: 64bit/jdk1.8.0 -XX:+UseCompressedOops -XX:+UseConcMarkSweepGC

1 tests failed.
FAILED:  
org.apache.solr.security.hadoop.TestDelegationWithHadoopAuth.testDelegationTokenRenew

Error Message:
expected:<200> but was:<403>

Stack Trace:
java.lang.AssertionError: expected:<200> but was:<403>
at 
__randomizedtesting.SeedInfo.seed([4F6D40E378A205CD:78F6B4FD406ED869]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.junit.Assert.assertEquals(Assert.java:456)
at 
org.apache.solr.security.hadoop.TestDelegationWithHadoopAuth.renewDelegationToken(TestDelegationWithHadoopAuth.java:118)
at 
org.apache.solr.security.hadoop.TestDelegationWithHadoopAuth.verifyDelegationTokenRenew(TestDelegationWithHadoopAuth.java:302)
at 
org.apache.solr.security.hadoop.TestDelegationWithHadoopAuth.testDelegationTokenRenew(TestDelegationWithHadoopAuth.java:319)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 

[JENKINS] Lucene-Solr-NightlyTests-6.x - Build # 283 - Still unstable

[Apache Jenkins Server]

Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-6.x/283/

3 tests failed.
FAILED:  org.apache.solr.cloud.hdfs.StressHdfsTest.test

Error Message:
Timeout occured while waiting response from server at: http://127.0.0.1:45877

Stack Trace:
org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting 
response from server at: http://127.0.0.1:45877
at 
__randomizedtesting.SeedInfo.seed([142663C043B89C34:9C725C1AED44F1CC]:0)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:621)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:279)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:268)
at 
org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:435)
at 
org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:387)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1358)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:1109)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:1042)
at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
at 
org.apache.solr.cloud.hdfs.StressHdfsTest.createAndDeleteCollection(StressHdfsTest.java:220)
at 
org.apache.solr.cloud.hdfs.StressHdfsTest.test(StressHdfsTest.java:103)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 

Re: Welcome Toke Eskildsen as a Lucene/Solr committer

[Alan Woodward]

Welcome Toke!

Alan Woodward
www.flax.co.uk


> On 14 Feb 2017, at 12:09, Jan Høydahl  wrote:
> 
> I'm pleased to announce that Toke Eskildsen has accepted the Lucene
> PMC's invitation to become a committer.
> 
> Toke, it's tradition that you introduce yourself with a brief bio.
> 
> Your account "toke" has been added to the “lucene" LDAP group, so you
> now have commit privileges. Please test this by adding yourself to the
> committers section of the Who We Are page on the website:
>  (instructions here
> ).
> 
> The ASF dev page also has lots of useful links: .
> 
> --
> Jan Høydahl, search solution architect
> Cominvent AS - www.cominvent.com
> 
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org
> 

[JENKINS-EA] Lucene-Solr-master-Linux (64bit/jdk-9-ea+155) - Build # 18962 - Unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/18962/
Java: 64bit/jdk-9-ea+155 -XX:+UseCompressedOops -XX:+UseParallelGC

2 tests failed.
FAILED:  org.apache.solr.handler.TestReqParamsAPI.test

Error Message:
Could not get expected value  'A val' for path 'params/a' full output: {   
"responseHeader":{ "status":0, "QTime":0},   "params":{"wt":"json"},   
"context":{ "webapp":"/solr", "path":"/dump1", 
"httpMethod":"GET"}},  from server:  
https://127.0.0.1:45318/solr/collection1_shard1_replica2

Stack Trace:
java.lang.AssertionError: Could not get expected value  'A val' for path 
'params/a' full output: {
  "responseHeader":{
"status":0,
"QTime":0},
  "params":{"wt":"json"},
  "context":{
"webapp":"/solr",
"path":"/dump1",
"httpMethod":"GET"}},  from server:  
https://127.0.0.1:45318/solr/collection1_shard1_replica2
at 
__randomizedtesting.SeedInfo.seed([3CBDF6197C68E2DD:B4E9C9C3D2948F25]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at 
org.apache.solr.core.TestSolrConfigHandler.testForResponseElement(TestSolrConfigHandler.java:556)
at 
org.apache.solr.handler.TestReqParamsAPI.testReqParams(TestReqParamsAPI.java:152)
at 
org.apache.solr.handler.TestReqParamsAPI.test(TestReqParamsAPI.java:69)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:543)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 

[jira] [Created] (SOLR-10135) A Regex URP that can extract multiple fields from a given field

[Noble Paul (JIRA)]

Noble Paul created SOLR-10135:
-

 Summary: A Regex URP that can extract multiple fields from a given 
field 
 Key: SOLR-10135
 URL: https://issues.apache.org/jira/browse/SOLR-10135
 Project: Solr
  Issue Type: New Feature
  Security Level: Public (Default Security Level. Issues are Public)
Reporter: Noble Paul


example: 
{code}
processor=Regex=fullName[firstName,lastName]:Mr(\w*)\b(.*)
{code}
This would apply the regex on the field {{fullName}} and extract two extra 
fields  {{firstName}} and {{lastName}} from {{group(1)}} and {{group(2)}} 
respectively 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10130) Serious performance degradation in Solr 6.4.1 due to the new metrics collection

[Walter Underwood (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15866092#comment-15866092
 ] 

Walter Underwood commented on SOLR-10130:
-

I have a JMeter-based load script I can share. It replays access logs. I reload 
the collection to clear caches, run warming queries, then run queries at a 
controlled rate. After, it calculates percentiles.

This was a test of 6.4.1. Really slow. The errors are usually log lines with 
queries so long that they are truncated and end up with bad syntax. There is 
one column per request handler, so these results are for /auto, /mobile, 
/select, and /srp.

Mon Feb 13 12:01:29 PST 2017 ; INFO testing solr-cloud.test.cheggnet.com:8983
Mon Feb 13 12:01:29 PST 2017 ; INFO testing with 2000 requests/min
Mon Feb 13 12:01:29 PST 2017 ; INFO testing with 24 requests
Mon Feb 13 12:01:29 PST 2017 : splitting log into cache warming (first 2000 
lines) and benchmark for 
/home/wunder/2016-12-12-peak-questions-traffic-clean.log
Mon Feb 13 12:01:36 PST 2017 : starting cache warming to 
solr-cloud.test.cheggnet.com:8983
Mon Feb 13 12:24:29 PST 2017 : starting benchmarking to 
solr-cloud.test.cheggnet.com:8983
Mon Feb 13 12:24:29 PST 2017 : benchmark should run for 120 minutes
Mon Feb 13 12:24:29 PST 2017 : to get a count of requests sent so far, use "wc 
-l out-32688.jtl"
Mon Feb 13 14:55:01 PST 2017 : WARNING 207 error responses from 
solr-cloud.test.cheggnet.com
Mon Feb 13 14:55:01 PST 2017 : INFO Removing 207 error responses from JMeter 
output file before analysis
Mon Feb 13 14:55:01 PST 2017 : analyzing results
/home/wunder/search-test/load-test
Mon Feb 13 14:55:04 PST 2017 : 25th percentiles are 3151.0,3389.0,9329.0,5647.0
Mon Feb 13 14:55:04 PST 2017 : medians are 6101.0,10579.0,18692.0,8780.0
Mon Feb 13 14:55:04 PST 2017 : 75th percentiles are 
6871.0,12499.0,25000.0,12580.0
Mon Feb 13 14:55:04 PST 2017 : 90th percentiles are 
7593.0,13481.0,27623.0,14068.0
Mon Feb 13 14:55:04 PST 2017 : 95th percentiles are 
8079.0,14039.0,28566.0,16606.0
Mon Feb 13 14:55:04 PST 2017 : full results are in test.csv

> Serious performance degradation in Solr 6.4.1 due to the new metrics 
> collection
> ---
>
> Key: SOLR-10130
> URL: https://issues.apache.org/jira/browse/SOLR-10130
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: metrics
>Affects Versions: 6.4.1
> Environment: Centos 7, OpenJDK 1.8.0 update 111
>Reporter: Ere Maijala
>Assignee: Andrzej Bialecki 
>Priority: Blocker
>  Labels: perfomance
> Fix For: master (7.0), 6.4.2
>
> Attachments: SOLR-10130.patch, solr-8983-console-f1.log
>
>
> We've stumbled on serious performance issues after upgrading to Solr 6.4.1. 
> Looks like the new metrics collection system in MetricsDirectoryFactory is 
> causing a major slowdown. This happens with an index configuration that, as 
> far as I can see, has no metrics specific configuration and uses 
> luceneMatchVersion 5.5.0. In practice a moderate load will completely bog 
> down the server with Solr threads constantly using up all CPU (600% on 6 core 
> machine) capacity with a load that normally  where we normally see an average 
> load of < 50%.
> I took stack traces (I'll attach them) and noticed that the threads are 
> spending time in com.codahale.metrics.Meter.mark. I tested building Solr 
> 6.4.1 with the metrics collection disabled in MetricsDirectoryFactory getByte 
> and getBytes methods and was unable to reproduce the issue.
> As far as I can see there are several issues:
> 1. Collecting metrics on every single byte read is slow.
> 2. Having it enabled by default is not a good idea.
> 3. The comment "enable coarse-grained metrics by default" at 
> https://github.com/apache/lucene-solr/blob/branch_6x/solr/core/src/java/org/apache/solr/update/SolrIndexConfig.java#L104
>  implies that only coarse-grained metrics should be enabled by default, and 
> this contradicts with collecting metrics on every single byte read.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10130) Serious performance degradation in Solr 6.4.1 due to the new metrics collection

[Ishan Chattopadhyaya (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15866077#comment-15866077
 ] 

Ishan Chattopadhyaya commented on SOLR-10130:
-

Thanks Yonik. I'm working on query performance benchmarks for this.

> Serious performance degradation in Solr 6.4.1 due to the new metrics 
> collection
> ---
>
> Key: SOLR-10130
> URL: https://issues.apache.org/jira/browse/SOLR-10130
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: metrics
>Affects Versions: 6.4.1
> Environment: Centos 7, OpenJDK 1.8.0 update 111
>Reporter: Ere Maijala
>Assignee: Andrzej Bialecki 
>Priority: Blocker
>  Labels: perfomance
> Fix For: master (7.0), 6.4.2
>
> Attachments: SOLR-10130.patch, solr-8983-console-f1.log
>
>
> We've stumbled on serious performance issues after upgrading to Solr 6.4.1. 
> Looks like the new metrics collection system in MetricsDirectoryFactory is 
> causing a major slowdown. This happens with an index configuration that, as 
> far as I can see, has no metrics specific configuration and uses 
> luceneMatchVersion 5.5.0. In practice a moderate load will completely bog 
> down the server with Solr threads constantly using up all CPU (600% on 6 core 
> machine) capacity with a load that normally  where we normally see an average 
> load of < 50%.
> I took stack traces (I'll attach them) and noticed that the threads are 
> spending time in com.codahale.metrics.Meter.mark. I tested building Solr 
> 6.4.1 with the metrics collection disabled in MetricsDirectoryFactory getByte 
> and getBytes methods and was unable to reproduce the issue.
> As far as I can see there are several issues:
> 1. Collecting metrics on every single byte read is slow.
> 2. Having it enabled by default is not a good idea.
> 3. The comment "enable coarse-grained metrics by default" at 
> https://github.com/apache/lucene-solr/blob/branch_6x/solr/core/src/java/org/apache/solr/update/SolrIndexConfig.java#L104
>  implies that only coarse-grained metrics should be enabled by default, and 
> this contradicts with collecting metrics on every single byte read.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9530) Add an Atomic Update Processor

[Noble Paul (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15866064#comment-15866064
 ] 

Noble Paul commented on SOLR-9530:
--

The description is not clear. This is relevant when the user does not have the 
choice of using a SET command . for example {{/update/csv}} or 
{{/update/json/docs}} . 

The patch is not thread-safe. What if multiple threads try to update the same 
document in parallel?

> Add an Atomic Update Processor 
> ---
>
> Key: SOLR-9530
> URL: https://issues.apache.org/jira/browse/SOLR-9530
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Varun Thacker
> Attachments: SOLR-9530.patch
>
>
> I'd like to explore the idea of adding a new update processor to help ingest 
> partial updates.
> Example use-case - There are two datasets with a common id field. How can I 
> merge both of them at index time?
> Proposed Solution: 
> {code}
> 
>   
> add
>   
>   
>   
> 
> {code}
> So the first JSON dump could be ingested against 
> {{http://localhost:8983/solr/gettingstarted/update/json}}
> And then the second JSON could be ingested against
> {{http://localhost:8983/solr/gettingstarted/update/json?processor=atomic}}
> The Atomic Update Processor could support all the atomic update operations 
> currently supported.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10130) Serious performance degradation in Solr 6.4.1 due to the new metrics collection

[Yonik Seeley (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15866067#comment-15866067
 ] 

Yonik Seeley commented on SOLR-10130:
-

Just a matter of how many little IOs are involved in your request.
I was easily able to reproduce a 5x slowdown with a prefix query that matches 
many terms.

> Serious performance degradation in Solr 6.4.1 due to the new metrics 
> collection
> ---
>
> Key: SOLR-10130
> URL: https://issues.apache.org/jira/browse/SOLR-10130
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: metrics
>Affects Versions: 6.4.1
> Environment: Centos 7, OpenJDK 1.8.0 update 111
>Reporter: Ere Maijala
>Assignee: Andrzej Bialecki 
>Priority: Blocker
>  Labels: perfomance
> Fix For: master (7.0), 6.4.2
>
> Attachments: SOLR-10130.patch, solr-8983-console-f1.log
>
>
> We've stumbled on serious performance issues after upgrading to Solr 6.4.1. 
> Looks like the new metrics collection system in MetricsDirectoryFactory is 
> causing a major slowdown. This happens with an index configuration that, as 
> far as I can see, has no metrics specific configuration and uses 
> luceneMatchVersion 5.5.0. In practice a moderate load will completely bog 
> down the server with Solr threads constantly using up all CPU (600% on 6 core 
> machine) capacity with a load that normally  where we normally see an average 
> load of < 50%.
> I took stack traces (I'll attach them) and noticed that the threads are 
> spending time in com.codahale.metrics.Meter.mark. I tested building Solr 
> 6.4.1 with the metrics collection disabled in MetricsDirectoryFactory getByte 
> and getBytes methods and was unable to reproduce the issue.
> As far as I can see there are several issues:
> 1. Collecting metrics on every single byte read is slow.
> 2. Having it enabled by default is not a good idea.
> 3. The comment "enable coarse-grained metrics by default" at 
> https://github.com/apache/lucene-solr/blob/branch_6x/solr/core/src/java/org/apache/solr/update/SolrIndexConfig.java#L104
>  implies that only coarse-grained metrics should be enabled by default, and 
> this contradicts with collecting metrics on every single byte read.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Comment Edited] (SOLR-9530) Add an Atomic Update Processor

[Noble Paul (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15865992#comment-15865992
 ] 

Noble Paul edited comment on SOLR-9530 at 2/14/17 4:23 PM:
---

IIUC , you can use this URP and keep calling ADD instead of SET and achieve the 
same ?


was (Author: noble.paul):
IIUC , you can use this URP and keep calling ADD instead of UPDATE and achieve 
the same ?

> Add an Atomic Update Processor 
> ---
>
> Key: SOLR-9530
> URL: https://issues.apache.org/jira/browse/SOLR-9530
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Varun Thacker
> Attachments: SOLR-9530.patch
>
>
> I'd like to explore the idea of adding a new update processor to help ingest 
> partial updates.
> Example use-case - There are two datasets with a common id field. How can I 
> merge both of them at index time?
> Proposed Solution: 
> {code}
> 
>   
> add
>   
>   
>   
> 
> {code}
> So the first JSON dump could be ingested against 
> {{http://localhost:8983/solr/gettingstarted/update/json}}
> And then the second JSON could be ingested against
> {{http://localhost:8983/solr/gettingstarted/update/json?processor=atomic}}
> The Atomic Update Processor could support all the atomic update operations 
> currently supported.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10006) Cannot do a full sync (fetchindex) if the replica can't open a searcher

[Mike Drob (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15866013#comment-15866013
 ] 

Mike Drob commented on SOLR-10006:
--

Yea, I'm imagining the force option to indicate the operator saying "yes, I 
know your checksums match and you think this is an unnecessary action, but do 
it anyway." Maybe bypass the searcher, not totally clear on implementation 
myself yet either.

The other open question to verify is whether you remembered to set the 
SOLR-9836 property the latest time you ran the test. If not, maybe we should 
have made that the default, I don't know. Regardless, we should still be able 
to handle things gracefully.

> Cannot do a full sync (fetchindex) if the replica can't open a searcher
> ---
>
> Key: SOLR-10006
> URL: https://issues.apache.org/jira/browse/SOLR-10006
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Affects Versions: 5.3.1, 6.4
>Reporter: Erick Erickson
> Attachments: SOLR-10006.patch, SOLR-10006.patch, solr.log, solr.log
>
>
> Doing a full sync or fetchindex requires an open searcher and if you can't 
> open the searcher those operations fail.
> For discussion. I've seen a situation in the field where a replica's index 
> became corrupt. When the node was restarted, the replica tried to do a full 
> sync but fails because the core can't open a searcher. The replica went into 
> an endless sync/fail/sync cycle.
> I couldn't reproduce that exact scenario, but it's easy enough to get into a 
> similar situation. Create a 2x2 collection and index some docs. Then stop one 
> of the instances and go in and remove a couple of segments files and restart.
> The replica stays in the "down" state, fine so far.
> Manually issue a fetchindex. That fails because the replica can't open a 
> searcher. Sure, issuing a fetchindex is abusive but I think it's the same 
> underlying issue: why should we care about the state of a replica's current 
> index when we're going to completely replace it anyway?



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9530) Add an Atomic Update Processor

[Noble Paul (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15865992#comment-15865992
 ] 

Noble Paul commented on SOLR-9530:
--

IIUC , you can use this URP and keep calling ADD instead of UPDATE and achieve 
the same ?

> Add an Atomic Update Processor 
> ---
>
> Key: SOLR-9530
> URL: https://issues.apache.org/jira/browse/SOLR-9530
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Varun Thacker
> Attachments: SOLR-9530.patch
>
>
> I'd like to explore the idea of adding a new update processor to help ingest 
> partial updates.
> Example use-case - There are two datasets with a common id field. How can I 
> merge both of them at index time?
> Proposed Solution: 
> {code}
> 
>   
> add
>   
>   
>   
> 
> {code}
> So the first JSON dump could be ingested against 
> {{http://localhost:8983/solr/gettingstarted/update/json}}
> And then the second JSON could be ingested against
> {{http://localhost:8983/solr/gettingstarted/update/json?processor=atomic}}
> The Atomic Update Processor could support all the atomic update operations 
> currently supported.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10006) Cannot do a full sync (fetchindex) if the replica can't open a searcher

[Erick Erickson (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15865990#comment-15865990
 ] 

Erick Erickson commented on SOLR-10006:
---

MIke:

As for the admin UI, I agree that's a separate issue. No good reason to let 
this JIRA sprawl.

As far as the force option. I'm a little fuzzy on when it'd apply. I completely 
agree that adding expense to fetchindex (and by implication polling?) to handle 
this case isn't a good tradeoff.

Are you thinking that the force would bypass the necessity of having an open 
searcher? Well, if it works ;)

The scenario here is that somehow the index got corrupt _and_ the instance got 
restarted. There's no way to recover the replica without bouncing the server 
again, and usually that would mean going into the file system and deleting the 
data directory. Which for some installations is prohibitively expensive, and/or 
requires filling out forms and the like ;) If we can force the fetchindex to 
happen on a core that's never successfully opened a searcher that's the end 
point I'm looking for. The rest is gravy.

Thanks!
Erick

> Cannot do a full sync (fetchindex) if the replica can't open a searcher
> ---
>
> Key: SOLR-10006
> URL: https://issues.apache.org/jira/browse/SOLR-10006
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Affects Versions: 5.3.1, 6.4
>Reporter: Erick Erickson
> Attachments: SOLR-10006.patch, SOLR-10006.patch, solr.log, solr.log
>
>
> Doing a full sync or fetchindex requires an open searcher and if you can't 
> open the searcher those operations fail.
> For discussion. I've seen a situation in the field where a replica's index 
> became corrupt. When the node was restarted, the replica tried to do a full 
> sync but fails because the core can't open a searcher. The replica went into 
> an endless sync/fail/sync cycle.
> I couldn't reproduce that exact scenario, but it's easy enough to get into a 
> similar situation. Create a 2x2 collection and index some docs. Then stop one 
> of the instances and go in and remove a couple of segments files and restart.
> The replica stays in the "down" state, fine so far.
> Manually issue a fetchindex. That fails because the replica can't open a 
> searcher. Sure, issuing a fetchindex is abusive but I think it's the same 
> underlying issue: why should we care about the state of a replica's current 
> index when we're going to completely replace it anyway?



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10130) Serious performance degradation in Solr 6.4.1 due to the new metrics collection

[Walter Underwood (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15865980#comment-15865980
 ] 

Walter Underwood commented on SOLR-10130:
-

Sorry, the 6000 rpm was with 6.2.1, not 6.4.0.

I've backrev'ed the cluster to 6.3.0 and I'll be running load benchmarks today.


> Serious performance degradation in Solr 6.4.1 due to the new metrics 
> collection
> ---
>
> Key: SOLR-10130
> URL: https://issues.apache.org/jira/browse/SOLR-10130
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: metrics
>Affects Versions: 6.4.1
> Environment: Centos 7, OpenJDK 1.8.0 update 111
>Reporter: Ere Maijala
>Assignee: Andrzej Bialecki 
>Priority: Blocker
>  Labels: perfomance
> Fix For: master (7.0), 6.4.2
>
> Attachments: SOLR-10130.patch, solr-8983-console-f1.log
>
>
> We've stumbled on serious performance issues after upgrading to Solr 6.4.1. 
> Looks like the new metrics collection system in MetricsDirectoryFactory is 
> causing a major slowdown. This happens with an index configuration that, as 
> far as I can see, has no metrics specific configuration and uses 
> luceneMatchVersion 5.5.0. In practice a moderate load will completely bog 
> down the server with Solr threads constantly using up all CPU (600% on 6 core 
> machine) capacity with a load that normally  where we normally see an average 
> load of < 50%.
> I took stack traces (I'll attach them) and noticed that the threads are 
> spending time in com.codahale.metrics.Meter.mark. I tested building Solr 
> 6.4.1 with the metrics collection disabled in MetricsDirectoryFactory getByte 
> and getBytes methods and was unable to reproduce the issue.
> As far as I can see there are several issues:
> 1. Collecting metrics on every single byte read is slow.
> 2. Having it enabled by default is not a good idea.
> 3. The comment "enable coarse-grained metrics by default" at 
> https://github.com/apache/lucene-solr/blob/branch_6x/solr/core/src/java/org/apache/solr/update/SolrIndexConfig.java#L104
>  implies that only coarse-grained metrics should be enabled by default, and 
> this contradicts with collecting metrics on every single byte read.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Welcome Toke Eskildsen as a Lucene/Solr committer

[Varun Thacker]

Welcome Toke!

On Tue, Feb 14, 2017 at 7:10 AM, David Smiley 
wrote:

> Welcome Toke!
>
> On Tue, Feb 14, 2017 at 4:09 AM Jan Høydahl  wrote:
>
>> I'm pleased to announce that Toke Eskildsen has accepted the Lucene
>> PMC's invitation to become a committer.
>>
>> Toke, it's tradition that you introduce yourself with a brief bio.
>>
>> Your account "toke" has been added to the “lucene" LDAP group, so you
>> now have commit privileges. Please test this by adding yourself to the
>> committers section of the Who We Are page on the website:
>>  (instructions here
>> ).
>>
>> The ASF dev page also has lots of useful links: <
>> http://www.apache.org/dev/>.
>>
>> --
>> Jan Høydahl, search solution architect
>> Cominvent AS - www.cominvent.com
>>
>> -
>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
>> For additional commands, e-mail: dev-h...@lucene.apache.org
>>
>> --
> Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker
> LinkedIn: http://linkedin.com/in/davidwsmiley | Book: http://www.
> solrenterprisesearchserver.com
>

[jira] [Commented] (SOLR-9530) Add an Atomic Update Processor

[Varun Thacker (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15865972#comment-15865972
 ] 

Varun Thacker commented on SOLR-9530:
-

Ishan - Exactly.. That was my motivation behind creating this Jira.

> Add an Atomic Update Processor 
> ---
>
> Key: SOLR-9530
> URL: https://issues.apache.org/jira/browse/SOLR-9530
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Varun Thacker
> Attachments: SOLR-9530.patch
>
>
> I'd like to explore the idea of adding a new update processor to help ingest 
> partial updates.
> Example use-case - There are two datasets with a common id field. How can I 
> merge both of them at index time?
> Proposed Solution: 
> {code}
> 
>   
> add
>   
>   
>   
> 
> {code}
> So the first JSON dump could be ingested against 
> {{http://localhost:8983/solr/gettingstarted/update/json}}
> And then the second JSON could be ingested against
> {{http://localhost:8983/solr/gettingstarted/update/json?processor=atomic}}
> The Atomic Update Processor could support all the atomic update operations 
> currently supported.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org