[JENKINS] Lucene-Solr-4.x-Linux (32bit/jdk1.8.0_05) - Build # 10103 - Failure!
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-4.x-Linux/10103/ Java: 32bit/jdk1.8.0_05 -server -XX:+UseParallelGC 1 tests failed. REGRESSION: org.apache.lucene.analysis.core.TestRandomChains.testRandomChains Error Message: startOffset must be non-negative, and endOffset must be >= startOffset, startOffset=2,endOffset=1 Stack Trace: java.lang.IllegalArgumentException: startOffset must be non-negative, and endOffset must be >= startOffset, startOffset=2,endOffset=1 at __randomizedtesting.SeedInfo.seed([490A5E65E84449EE:74EB7704AF56542E]:0) at org.apache.lucene.analysis.tokenattributes.OffsetAttributeImpl.setOffset(OffsetAttributeImpl.java:45) at org.apache.lucene.analysis.shingle.ShingleFilter.incrementToken(ShingleFilter.java:345) at org.apache.lucene.analysis.ValidatingTokenFilter.incrementToken(ValidatingTokenFilter.java:68) at org.apache.lucene.analysis.BaseTokenStreamTestCase.checkAnalysisConsistency(BaseTokenStreamTestCase.java:701) at org.apache.lucene.analysis.BaseTokenStreamTestCase.checkRandomData(BaseTokenStreamTestCase.java:612) at org.apache.lucene.analysis.BaseTokenStreamTestCase.checkRandomData(BaseTokenStreamTestCase.java:511) at org.apache.lucene.analysis.core.TestRandomChains.testRandomChains(TestRandomChains.java:925) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:483) at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618) at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827) at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863) at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877) at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50) at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55) at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:360) at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:793) at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:453) at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836) at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738) at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772) at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65) at org.apache.lucene.util.TestRuleIgnoreTestSuit
Re: CJKBigramFilter - position bug with outputUnigrams?
On 4/21/2014 12:47 PM, Robert Muir wrote: > I think you misunderstand what the filter does. It does not "output unigrams". > > In the case you choose this option, the positions are from the > unigrams omitted by your tokenizer (StandardTokenizer or whatever), > and it just adds bigrams as synonyms to those. It cannot safely do > anything else. > > There can be only one "n". I took a quick look at the code. I'm sure it's easy to grasp once you're really familiar with everything, but I'm having a hard time decoding exactly how the filter works. I don't have any more time to plow through it tonight. Would it be possible to implement an option with a name similar to "lastUnigramAtPreviousPosition" so that I can optionally get the behavior I'm after when the input is two or more characters, without changing current behavior for anyone else? This would completely solve my current problem. Thanks, Shawn - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5611) Simplify the default indexing chain
[ https://issues.apache.org/jira/browse/LUCENE-5611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13987283#comment-13987283 ] ASF subversion and git services commented on LUCENE-5611: - Commit 1591808 from [~rcmuir] in branch 'dev/branches/branch_4x' [ https://svn.apache.org/r1591808 ] LUCENE-5611: lazy-init the stored fields writer like before to prevent leaks > Simplify the default indexing chain > --- > > Key: LUCENE-5611 > URL: https://issues.apache.org/jira/browse/LUCENE-5611 > Project: Lucene - Core > Issue Type: Improvement > Components: core/index >Reporter: Michael McCandless >Assignee: Michael McCandless > Fix For: 4.9, 5.0 > > Attachments: LUCENE-5611.patch, LUCENE-5611.patch > > > I think Lucene's current indexing chain has too many classes / > hierarchy / abstractions, making it look much more complex than it > really should be, and discouraging users from experimenting/innovating > with their own indexing chains. > Also, if it were easier to understand/approach, then new developers > would more likely try to improve it ... it really should be simpler. > So I'm exploring a pared back indexing chain, and have a starting patch > that I think is looking ok: it seems more approachable than the > current indexing chain, or at least has fewer strange classes. > I also thought this could give some speedup for tiny documents (a more > common use of Lucene lately), and it looks like, with the evil > optimizations, this is a ~25% speedup for Geonames docs. Even without > those evil optos it's a bit faster. > This is very much a work in progress / nocommits, and there are some > behavior changes e.g. the new chain requires all fields to have the > same TV options (rather than auto-upgrading all fields by the same > name that the current chain does)... -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5611) Simplify the default indexing chain
[ https://issues.apache.org/jira/browse/LUCENE-5611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13987282#comment-13987282 ] ASF subversion and git services commented on LUCENE-5611: - Commit 1591807 from [~rcmuir] in branch 'dev/trunk' [ https://svn.apache.org/r1591807 ] LUCENE-5611: lazy-init the stored fields writer like before to prevent leaks > Simplify the default indexing chain > --- > > Key: LUCENE-5611 > URL: https://issues.apache.org/jira/browse/LUCENE-5611 > Project: Lucene - Core > Issue Type: Improvement > Components: core/index >Reporter: Michael McCandless >Assignee: Michael McCandless > Fix For: 4.9, 5.0 > > Attachments: LUCENE-5611.patch, LUCENE-5611.patch > > > I think Lucene's current indexing chain has too many classes / > hierarchy / abstractions, making it look much more complex than it > really should be, and discouraging users from experimenting/innovating > with their own indexing chains. > Also, if it were easier to understand/approach, then new developers > would more likely try to improve it ... it really should be simpler. > So I'm exploring a pared back indexing chain, and have a starting patch > that I think is looking ok: it seems more approachable than the > current indexing chain, or at least has fewer strange classes. > I also thought this could give some speedup for tiny documents (a more > common use of Lucene lately), and it looks like, with the evil > optimizations, this is a ~25% speedup for Geonames docs. Even without > those evil optos it's a bit faster. > This is very much a work in progress / nocommits, and there are some > behavior changes e.g. the new chain requires all fields to have the > same TV options (rather than auto-upgrading all fields by the same > name that the current chain does)... -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-6039) debug=track causes debug=query info to be suprsedd when no results found
[
https://issues.apache.org/jira/browse/SOLR-6039?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tomás Fernández Löbbe updated SOLR-6039:
Attachment: SOLR-6039.patch
I'm looking into this issue. This patch would fix the problem described in the
users list with the query explain, but not yet the with the timing.
I kind of see where the timing issue is too, but I'm wondering if this is
expected. Seems like the timing info should be merged in
DebugComponent.finishStage(...), but as it is now, it would only consider the
time spent in the phases with GET_DEBUG purpose. This purpose is only included
in in the GET_FIELDS phase, we would not be considering time in parts of the
request?
In addition, once it gets the timing information from the shards it adds it. I
think it would be better if it get the max value instead? after all, the time
in each phase will be the time the slowest shard took, right?
> debug=track causes debug=query info to be suprsedd when no results found
>
>
> Key: SOLR-6039
> URL: https://issues.apache.org/jira/browse/SOLR-6039
> Project: Solr
> Issue Type: Bug
>Affects Versions: 4.7
>Reporter: Hoss Man
> Attachments: SOLR-6039.patch
>
>
> Shamik Bandopadhyay noted on the mailing list that debugQuery=true wasn't
> returning info about how the query was being parsed in SolrCloud.
> Steps to reproduce...
> * startup a simple 2 shard solr cluster using the example configs
> * Load this URL:
> ** http://localhost:8983/solr/select?q=Foo&debug=query
> ** note that the debug=query causes a debug block including "parsedquery"
> showing "title:foo"
> * Load either of these URLs:
> ** http://localhost:8983/solr/select?q=Foo&debug=query&debug=track
> ** http://localhost:8983/solr/select?q=Foo&debugQuery=true (legacy short hand
> option for enabling all debug options)
> ** Note that the debug block exists, but *only* includes the distributed
> "track" options - the query parsing debugging info is not available
> * index the sample data (java -jar post.jar *.xml)
> * Load either of these URLs:
> ** http://localhost:8983/solr/select?q=Solr&debugQuery=true&fl=id
> ** http://localhost:8983/solr/select?q=Solr&debug=query&debug=track&fl=id
> ** Note that now we have at least one matching doc, and the parsedquery info
> is included in the debug block along with the tracking info
> * Load either of these URLs:
> ** http://localhost:8983/solr/select?q=Solr&debug=query&debug=track&rows=0
> ** http://localhost:8983/solr/select?q=Solr&debug=query&debug=track&rows=0
> ** Note: even though we have a matching doc, since rows=0 prevents it from
> being returned, the parsedquery debug info again no longer works - just the
> track debug info
>
> The work around, for people who want don't care about the newer "debug
> tracking" and what the same debug information as pre-4.7, is to enumerate the
> debug options (ie: {{debug=query&debug=timing&debug=results}}) instead of
> relying on the shorthand: {{debugQuery=true}}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Solr-trunk-Linux (64bit/jdk1.8.0_20-ea-b05) - Build # 10212 - Failure!
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/10212/
Java: 64bit/jdk1.8.0_20-ea-b05 -XX:+UseCompressedOops -XX:+UseParallelGC
All tests passed
Build Log:
[...truncated 58121 lines...]
-documentation-lint:
[jtidy] Checking for broken html (such as invalid tags)...
[delete] Deleting directory
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build/jtidy_tmp
[echo] Checking for broken links...
[exec]
[exec] Crawl/parse...
[exec]
[exec]
file:///mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/docs/changes/Changes.html
[exec] WARNING: failed to parse
file:///mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/docs/changes/Changes.html:
[exec] Traceback (most recent call last):
[exec] File
"/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/dev-tools/scripts/checkJavadocLinks.py",
line 111, in parse
[exec] parser.close()
[exec] File "/usr/lib/python3.2/html/parser.py", line 146, in close
[exec] self.goahead(1)
[exec] File "/usr/lib/python3.2/html/parser.py", line 207, in goahead
[exec] self.error("EOF in middle of construct")
[exec] File "/usr/lib/python3.2/html/parser.py", line 149, in error
[exec] raise HTMLParseError(message, self.getpos())
[exec] html.parser.HTMLParseError: EOF in middle of construct, at line
8739, column 51
[exec]
[exec] Verify...
[exec]
[exec] Broken javadocs links were found!
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:467: The following
error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:63: The following
error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build.xml:542: The
following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build.xml:555: The
following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:2330:
exec returned: 1
Total time: 67 minutes 55 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 64bit/jdk1.8.0_20-ea-b05 -XX:+UseCompressedOops
-XX:+UseParallelGC
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Resolved] (SOLR-6037) Stats Component Reports Incorrect max/sum/stddev for date fields in Distributed Data
[
https://issues.apache.org/jira/browse/SOLR-6037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hoss Man resolved SOLR-6037.
Resolution: Fixed
Fix Version/s: 5.0
4.9
> Stats Component Reports Incorrect max/sum/stddev for date fields in
> Distributed Data
>
>
> Key: SOLR-6037
> URL: https://issues.apache.org/jira/browse/SOLR-6037
> Project: Solr
> Issue Type: Bug
> Components: SearchComponents - other
>Reporter: Brett Lucey
>Assignee: Hoss Man
>Priority: Minor
> Fix For: 4.9, 5.0
>
> Attachments: SOLR-6037.patch, SOLR-6037.patch, SOLR-6037.patch
>
>
> When computing statistics using the statistics component, the value for max
> may be reported incorrectly if the maximum value does not lie in the first
> shard.
> This is where the problem is:
> {code:title=StatsValuesFactory.java|borderStyle=solid}
> @Override
> protected void updateMinMax(Date min, Date max) {
> if(this.min==null || this.min.after(min)) {
> this.min = min;
> }
> if(this.max==null || this.max.before(min)) {
> this.max = max;
> }
> }
> {code}
> Attached is a one-line patch which fixes this issue.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-6037) Stats Component Reports Incorrect max/sum/stddev for date fields in Distributed Data
[
https://issues.apache.org/jira/browse/SOLR-6037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13987227#comment-13987227
]
ASF subversion and git services commented on SOLR-6037:
---
Commit 1591801 from hoss...@apache.org in branch 'dev/branches/branch_4x'
[ https://svn.apache.org/r1591801 ]
SOLR-6037: Fixed incorrect max/sum/stddev for Date fields in StatsComponent
(merge r1591800)
> Stats Component Reports Incorrect max/sum/stddev for date fields in
> Distributed Data
>
>
> Key: SOLR-6037
> URL: https://issues.apache.org/jira/browse/SOLR-6037
> Project: Solr
> Issue Type: Bug
> Components: SearchComponents - other
>Reporter: Brett Lucey
>Assignee: Hoss Man
>Priority: Minor
> Attachments: SOLR-6037.patch, SOLR-6037.patch, SOLR-6037.patch
>
>
> When computing statistics using the statistics component, the value for max
> may be reported incorrectly if the maximum value does not lie in the first
> shard.
> This is where the problem is:
> {code:title=StatsValuesFactory.java|borderStyle=solid}
> @Override
> protected void updateMinMax(Date min, Date max) {
> if(this.min==null || this.min.after(min)) {
> this.min = min;
> }
> if(this.max==null || this.max.before(min)) {
> this.max = max;
> }
> }
> {code}
> Attached is a one-line patch which fixes this issue.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-6037) Stats Component Reports Incorrect max/sum/stddev for date fields in Distributed Data
[
https://issues.apache.org/jira/browse/SOLR-6037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13987213#comment-13987213
]
ASF subversion and git services commented on SOLR-6037:
---
Commit 1591800 from hoss...@apache.org in branch 'dev/trunk'
[ https://svn.apache.org/r1591800 ]
SOLR-6037: Fixed incorrect max/sum/stddev for Date fields in StatsComponent
> Stats Component Reports Incorrect max/sum/stddev for date fields in
> Distributed Data
>
>
> Key: SOLR-6037
> URL: https://issues.apache.org/jira/browse/SOLR-6037
> Project: Solr
> Issue Type: Bug
> Components: SearchComponents - other
>Reporter: Brett Lucey
>Assignee: Hoss Man
>Priority: Minor
> Attachments: SOLR-6037.patch, SOLR-6037.patch, SOLR-6037.patch
>
>
> When computing statistics using the statistics component, the value for max
> may be reported incorrectly if the maximum value does not lie in the first
> shard.
> This is where the problem is:
> {code:title=StatsValuesFactory.java|borderStyle=solid}
> @Override
> protected void updateMinMax(Date min, Date max) {
> if(this.min==null || this.min.after(min)) {
> this.min = min;
> }
> if(this.max==null || this.max.before(min)) {
> this.max = max;
> }
> }
> {code}
> Attached is a one-line patch which fixes this issue.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-6037) Stats Component Reports Incorrect max/sum/stddev for date fields in Distributed Data
[
https://issues.apache.org/jira/browse/SOLR-6037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hoss Man updated SOLR-6037:
---
Attachment: SOLR-6037.patch
updated patch with a bit more test coverage
> Stats Component Reports Incorrect max/sum/stddev for date fields in
> Distributed Data
>
>
> Key: SOLR-6037
> URL: https://issues.apache.org/jira/browse/SOLR-6037
> Project: Solr
> Issue Type: Bug
> Components: SearchComponents - other
>Reporter: Brett Lucey
>Assignee: Hoss Man
>Priority: Minor
> Attachments: SOLR-6037.patch, SOLR-6037.patch, SOLR-6037.patch
>
>
> When computing statistics using the statistics component, the value for max
> may be reported incorrectly if the maximum value does not lie in the first
> shard.
> This is where the problem is:
> {code:title=StatsValuesFactory.java|borderStyle=solid}
> @Override
> protected void updateMinMax(Date min, Date max) {
> if(this.min==null || this.min.after(min)) {
> this.min = min;
> }
> if(this.max==null || this.max.before(min)) {
> this.max = max;
> }
> }
> {code}
> Attached is a one-line patch which fixes this issue.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-6037) Stats Component Reports Incorrect Max in Distributed Data
[
https://issues.apache.org/jira/browse/SOLR-6037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hoss Man updated SOLR-6037:
---
Attachment: SOLR-6037.patch
Ugh...
Brett: thanks for reporting this.
I've updated your patch to account for the fact that min/max might be null
coming back from a shard. When updating the existing distributed stats test to
also include date fields to trigger this problem, I uncovered another glitch in
how the sum/stddev are computed (sum was inited to "-1" for some bizare reason)
which is also fixed in this updated patch.
> Stats Component Reports Incorrect Max in Distributed Data
> -
>
> Key: SOLR-6037
> URL: https://issues.apache.org/jira/browse/SOLR-6037
> Project: Solr
> Issue Type: Bug
> Components: SearchComponents - other
>Reporter: Brett Lucey
>Priority: Minor
> Attachments: SOLR-6037.patch, SOLR-6037.patch
>
>
> When computing statistics using the statistics component, the value for max
> may be reported incorrectly if the maximum value does not lie in the first
> shard.
> This is where the problem is:
> {code:title=StatsValuesFactory.java|borderStyle=solid}
> @Override
> protected void updateMinMax(Date min, Date max) {
> if(this.min==null || this.min.after(min)) {
> this.min = min;
> }
> if(this.max==null || this.max.before(min)) {
> this.max = max;
> }
> }
> {code}
> Attached is a one-line patch which fixes this issue.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-6037) Stats Component Reports Incorrect max/sum/stddev for date fields in Distributed Data
[
https://issues.apache.org/jira/browse/SOLR-6037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hoss Man updated SOLR-6037:
---
Affects Version/s: (was: 5.0)
Assignee: Hoss Man
Summary: Stats Component Reports Incorrect max/sum/stddev for
date fields in Distributed Data (was: Stats Component Reports Incorrect Max in
Distributed Data)
updating summary
> Stats Component Reports Incorrect max/sum/stddev for date fields in
> Distributed Data
>
>
> Key: SOLR-6037
> URL: https://issues.apache.org/jira/browse/SOLR-6037
> Project: Solr
> Issue Type: Bug
> Components: SearchComponents - other
>Reporter: Brett Lucey
>Assignee: Hoss Man
>Priority: Minor
> Attachments: SOLR-6037.patch, SOLR-6037.patch
>
>
> When computing statistics using the statistics component, the value for max
> may be reported incorrectly if the maximum value does not lie in the first
> shard.
> This is where the problem is:
> {code:title=StatsValuesFactory.java|borderStyle=solid}
> @Override
> protected void updateMinMax(Date min, Date max) {
> if(this.min==null || this.min.after(min)) {
> this.min = min;
> }
> if(this.max==null || this.max.before(min)) {
> this.max = max;
> }
> }
> {code}
> Attached is a one-line patch which fixes this issue.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-6039) debug=track causes debug=query info to be suprsedd when no results found
[
https://issues.apache.org/jira/browse/SOLR-6039?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13987159#comment-13987159
]
Shawn Heisey commented on SOLR-6039:
I had noticed this with a non-cloud distributed search on Solr 4.7. It was
causing problems when trying to figure out why a search wasn't returning
results. I sent the query directly to a shard to get the info, and put the
problem in the mental category of "Report it later when there's time."
> debug=track causes debug=query info to be suprsedd when no results found
>
>
> Key: SOLR-6039
> URL: https://issues.apache.org/jira/browse/SOLR-6039
> Project: Solr
> Issue Type: Bug
>Affects Versions: 4.7
>Reporter: Hoss Man
>
> Shamik Bandopadhyay noted on the mailing list that debugQuery=true wasn't
> returning info about how the query was being parsed in SolrCloud.
> Steps to reproduce...
> * startup a simple 2 shard solr cluster using the example configs
> * Load this URL:
> ** http://localhost:8983/solr/select?q=Foo&debug=query
> ** note that the debug=query causes a debug block including "parsedquery"
> showing "title:foo"
> * Load either of these URLs:
> ** http://localhost:8983/solr/select?q=Foo&debug=query&debug=track
> ** http://localhost:8983/solr/select?q=Foo&debugQuery=true (legacy short hand
> option for enabling all debug options)
> ** Note that the debug block exists, but *only* includes the distributed
> "track" options - the query parsing debugging info is not available
> * index the sample data (java -jar post.jar *.xml)
> * Load either of these URLs:
> ** http://localhost:8983/solr/select?q=Solr&debugQuery=true&fl=id
> ** http://localhost:8983/solr/select?q=Solr&debug=query&debug=track&fl=id
> ** Note that now we have at least one matching doc, and the parsedquery info
> is included in the debug block along with the tracking info
> * Load either of these URLs:
> ** http://localhost:8983/solr/select?q=Solr&debug=query&debug=track&rows=0
> ** http://localhost:8983/solr/select?q=Solr&debug=query&debug=track&rows=0
> ** Note: even though we have a matching doc, since rows=0 prevents it from
> being returned, the parsedquery debug info again no longer works - just the
> track debug info
>
> The work around, for people who want don't care about the newer "debug
> tracking" and what the same debug information as pre-4.7, is to enumerate the
> debug options (ie: {{debug=query&debug=timing&debug=results}}) instead of
> relying on the shorthand: {{debugQuery=true}}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (SOLR-6039) debug=track causes debug=query info to be suprsedd when no results found
Hoss Man created SOLR-6039:
--
Summary: debug=track causes debug=query info to be suprsedd when
no results found
Key: SOLR-6039
URL: https://issues.apache.org/jira/browse/SOLR-6039
Project: Solr
Issue Type: Bug
Affects Versions: 4.7
Reporter: Hoss Man
Shamik Bandopadhyay noted on the mailing list that debugQuery=true wasn't
returning info about how the query was being parsed in SolrCloud.
Steps to reproduce...
* startup a simple 2 shard solr cluster using the example configs
* Load this URL:
** http://localhost:8983/solr/select?q=Foo&debug=query
** note that the debug=query causes a debug block including "parsedquery"
showing "title:foo"
* Load either of these URLs:
** http://localhost:8983/solr/select?q=Foo&debug=query&debug=track
** http://localhost:8983/solr/select?q=Foo&debugQuery=true (legacy short hand
option for enabling all debug options)
** Note that the debug block exists, but *only* includes the distributed
"track" options - the query parsing debugging info is not available
* index the sample data (java -jar post.jar *.xml)
* Load either of these URLs:
** http://localhost:8983/solr/select?q=Solr&debugQuery=true&fl=id
** http://localhost:8983/solr/select?q=Solr&debug=query&debug=track&fl=id
** Note that now we have at least one matching doc, and the parsedquery info is
included in the debug block along with the tracking info
* Load either of these URLs:
** http://localhost:8983/solr/select?q=Solr&debug=query&debug=track&rows=0
** http://localhost:8983/solr/select?q=Solr&debug=query&debug=track&rows=0
** Note: even though we have a matching doc, since rows=0 prevents it from
being returned, the parsedquery debug info again no longer works - just the
track debug info
The work around, for people who want don't care about the newer "debug
tracking" and what the same debug information as pre-4.7, is to enumerate the
debug options (ie: {{debug=query&debug=timing&debug=results}}) instead of
relying on the shorthand: {{debugQuery=true}}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Resolved] (SOLR-6012) Dutch language stemming issues
[ https://issues.apache.org/jira/browse/SOLR-6012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hoss Man resolved SOLR-6012. Resolution: Invalid There doesn't appear to be an actually bug report here: just an annecdoteal example o a single failure to match on a term w/o any specifics on configuration/analyser used, the data indexed, etc... Ashokkumar: please use the solr-user@lucene mailing list for questions about using Solr and/or concerns about bahavior that you are getting that doesn't match your expectations. Bugs should only be filed once there is confirmation of incorrect behavior with correct usage. See the wiki for details on what types of information you should provide when asking for help on the user list... https://wiki.apache.org/solr/UsingMailingLists > Dutch language stemming issues > -- > > Key: SOLR-6012 > URL: https://issues.apache.org/jira/browse/SOLR-6012 > Project: Solr > Issue Type: Bug > Components: search >Affects Versions: 3.5 > Environment: Linux >Reporter: Ashokkumar Balasubramanian > Labels: easyfix, newbie > > I am trying to search a word in dutch language with the word Brievenbussen. > Normally this is the proper dutch word and it should result in some matches > but it results in 0 matches. The dutch word Brievenbusen (Letter 's' is > removed) returns matches. > The problem is it doesn't take the last word 'bus' vowel character into > account. If a vowel is found in the last before character (in this case, it > is 'U'), then the proper dutch word should be Brievenbussen. > Can you please confirm if this is a problem with 3.5 version. > Please let me know if you need more information. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-3610) Revamp spatial APIs that use primitives (or arrays of primitives) in their args/results so that they use strongly typed objects
[
https://issues.apache.org/jira/browse/LUCENE-3610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Smiley updated LUCENE-3610:
-
Priority: Minor (was: Major)
Fix Version/s: (was: 4.9)
(was: 5.0)
> Revamp spatial APIs that use primitives (or arrays of primitives) in their
> args/results so that they use strongly typed objects
> ---
>
> Key: LUCENE-3610
> URL: https://issues.apache.org/jira/browse/LUCENE-3610
> Project: Lucene - Core
> Issue Type: Improvement
> Components: modules/spatial
>Reporter: Hoss Man
>Priority: Minor
>
> My "spatial awareness" is pretty meek, but LUCENE-3599 seems like a prime
> example of the types of mistakes that are probably really easy to make with
> all of the Spatial related APIs that deal with arrays (or sequences) of
> doubles where specific indexes of those arrays (or sequences) have
> significant meaning: mainly latitude vs longitude.
> We should probably reconsider any method that takes in double[] or multiple
> doubles to express latlon pairs and rewrite them to use the existing LatLng
> class -- or if people think that class is too heavyweight, then add a new
> lightweight class to handle the strong typing of a basic latlon point instead
> of just passing around a double[2] or two doubles called "x" and "y" ...
> {code}
> public static final class SimpleLatLonPointInRadians {
> public double latitude;
> public double longitude;
> }
> {code}
> ...then all those various methods that expect lat+lon pairs in radians (like
> DistanceUtils.haversine, DistanceUtils.normLat, DistanceUtils.normLng,
> DistanceUtils.pointOnBearing, DistanceUtils.latLonCorner, etc...) can start
> having APIs that don't make your eyes bleed when you start trying to
> understand what order the args go in.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5831) Scale score PostFilter
[
https://issues.apache.org/jira/browse/SOLR-5831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13987037#comment-13987037
]
Chris Russell commented on SOLR-5831:
-
Hi. I have uploaded a patch that is designed to improve the performance of the
scale function by allowing it to only score documents that match the filters.
I am curious how my patch compares with your posted benchmarks from March 11th.
Would you be willing to apply it and run them?
LUCENE-5637
> Scale score PostFilter
> --
>
> Key: SOLR-5831
> URL: https://issues.apache.org/jira/browse/SOLR-5831
> Project: Solr
> Issue Type: Improvement
> Components: search
>Affects Versions: 4.7
>Reporter: Peter Keegan
>Assignee: Joel Bernstein
>Priority: Minor
> Fix For: 4.9
>
> Attachments: SOLR-5831.patch, SOLR-5831.patch, SOLR-5831.patch,
> TestScaleScoreQParserPlugin.patch
>
>
> The ScaleScoreQParserPlugin is a PostFilter that performs score scaling.
> This is an alternative to using a function query wrapping a scale() wrapping
> a query(). For example:
> select?qq={!edismax v='news' qf='title^2
> body'}&scaledQ=scale(product(query($qq),1),0,1)&q={!func}sum(product(0.75,$scaledQ),product(0.25,field(myfield)))&fq={!query
> v=$qq}
> The problem with this query is that it has to scale every hit. Usually, only
> the returned hits need to be scaled,
> but there may be use cases where the number of hits to be scaled is greater
> than the returned hit count,
> but less than or equal to the total hit count.
> Sample syntax:
> fq={!scalescore+l=0.0 u=1.0 maxscalehits=1
> func=sum(product(sscore(),0.75),product(field(myfield),0.25))}
> l=0.0 u=1.0 //Scale scores to values between 0-1, inclusive
> maxscalehits=1//The maximum number of result scores to scale (-1 =
> all hits, 0 = results 'page' size)
> func=... //Apply the composite function to each hit. The
> scaled score value is accessed by the 'score()' value source
> All parameters are optional. The defaults are:
> l=0.0 u=1.0
> maxscalehits=0 (result window size)
> func=(null)
>
> Note: this patch is not complete, as it contains no test cases and may not
> conform
> to all the guidelines in http://wiki.apache.org/solr/HowToContribute.
>
> I would appreciate any feedback on the usability and implementation.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
Issue with functions that require metadata, and LeafCollectors
Hi.
I have opened an issue on Jira about improving the scale() function:
https://issues.apache.org/jira/browse/LUCENE-5637
I was able to improve the performance of the scale function quite a bit, but
this required me to refactor some code in IndexSearcher.Search
There is a loop where scorers are created for each AtomicReaderContext, and
then used to score documents. It looks like this in 4.8:
for (AtomicReaderContext ctx : leaves) { // search each subreader
try {
collector.setNextReader(ctx);
[...]
BulkScorer scorer = weight.bulkScorer(ctx,
!collector.acceptsDocsOutOfOrder(), ctx.reader().getLiveDocs());
if (scorer != null) {
try {
scorer.score(collector);
[...]
}
I was able to break this up into two for-loops, and this was necessary because
the scale function needed to see each AtomicReaderContext before being asked to
score any documents, in order to determine the scale constant without doing
something like grabbing the top level reader and looking at every document in
the index (previous behavior)
So, new loops like this in 4.8:
ArrayList scorers = new ArrayList();
for (AtomicReaderContext ctx : leaves) { // search each subreader
BulkScorer scorer = weight.bulkScorer(ctx,
!collector.acceptsDocsOutOfOrder(), ctx.reader().getLiveDocs());
scorers.add(scorer);
}
for(int i = 0; i < leaves.size(); i++) {
BulkScorer scorer = scorers.get(i);
AtomicReaderContext ctx = leaves.get(i);
try {
collector.setNextReader(ctx);
[...]
if (scorer != null) {
try {
scorer.score(collector);
[...]
}
This seems to work fine and allows the function to gather the metadata it needs.
When trying to bring my code to trunk, I ran into an issue with the recently
introduced LeafCollector interface.
It seems like setNextReader no longer exists, and scorer.score takes in a
LeafCollector now.
In trunk, when I try to break this for-loop into two for-loops, it breaks a ton
of unit tests.
I need the LeafCollectors in the first loop where I am making the scorers
because LeafCollector now has the acceptDocsOutOfOrder method.
I need them in the second loop because that is what .score takes now.
So I tried keeping track of the LeafCollectors I created in the first loop and
using them in the second, which did not work.
I also tried asking the collector for new LeafCollectors in each of the two
loops, and that did not work.
I think this is all because setNextReader went away and there is some side
effect I am encountering related to making a LeafCollector and not immediately
scoring with it? Does asking the passed-in collector for another LeafCollector
for some other context do something to the previous LeafCollector?
All I am trying to do is create all scorers before using them, which seems like
it should be possible logically. This is especially useful for functions that
require metadata.
Any assistance would be appreciated.
-Chris
[jira] [Commented] (SOLR-5831) Scale score PostFilter
[
https://issues.apache.org/jira/browse/SOLR-5831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13987005#comment-13987005
]
Joel Bernstein commented on SOLR-5831:
--
I went through the latest patch and it looks really good.
The main pitfalls with the CollapsingQParserPlugin dealt with how the
DummyScorer behaved with the different types of ranking collectors. You
mentioned a secondary sort issue, that was fixed in the latest patch. I'm not
sure if this was related to the use of the DummyScorer.
To account for this lets add some tests that confirm proper result ordering
under the different sorting conditions. I should have some time to install the
patch and work with it next week.
> Scale score PostFilter
> --
>
> Key: SOLR-5831
> URL: https://issues.apache.org/jira/browse/SOLR-5831
> Project: Solr
> Issue Type: Improvement
> Components: search
>Affects Versions: 4.7
>Reporter: Peter Keegan
>Assignee: Joel Bernstein
>Priority: Minor
> Fix For: 4.9
>
> Attachments: SOLR-5831.patch, SOLR-5831.patch, SOLR-5831.patch,
> TestScaleScoreQParserPlugin.patch
>
>
> The ScaleScoreQParserPlugin is a PostFilter that performs score scaling.
> This is an alternative to using a function query wrapping a scale() wrapping
> a query(). For example:
> select?qq={!edismax v='news' qf='title^2
> body'}&scaledQ=scale(product(query($qq),1),0,1)&q={!func}sum(product(0.75,$scaledQ),product(0.25,field(myfield)))&fq={!query
> v=$qq}
> The problem with this query is that it has to scale every hit. Usually, only
> the returned hits need to be scaled,
> but there may be use cases where the number of hits to be scaled is greater
> than the returned hit count,
> but less than or equal to the total hit count.
> Sample syntax:
> fq={!scalescore+l=0.0 u=1.0 maxscalehits=1
> func=sum(product(sscore(),0.75),product(field(myfield),0.25))}
> l=0.0 u=1.0 //Scale scores to values between 0-1, inclusive
> maxscalehits=1//The maximum number of result scores to scale (-1 =
> all hits, 0 = results 'page' size)
> func=... //Apply the composite function to each hit. The
> scaled score value is accessed by the 'score()' value source
> All parameters are optional. The defaults are:
> l=0.0 u=1.0
> maxscalehits=0 (result window size)
> func=(null)
>
> Note: this patch is not complete, as it contains no test cases and may not
> conform
> to all the guidelines in http://wiki.apache.org/solr/HowToContribute.
>
> I would appreciate any feedback on the usability and implementation.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-5637) Scaling scale function
[ https://issues.apache.org/jira/browse/LUCENE-5637?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Russell updated LUCENE-5637: -- Description: The existing scale() function examines the scores of all documents in the index in order to calculate its scale constant. This does not perform well in solr on very large indexes or with costly scoring mechanisms such as geo distance. I have developed a patch that allows the scale function to only score documents that match the given filters, thus improving performance of the scale function. For test queries involving two scale operations where one was scaling the result of keyword scoring and the other was scaling the result of geo distance scoring on an index with ~2 million documents, query time was improved from ~400 ms with vanilla scale to ~190 ms with new scale. A similar query using no scaling ran in ~90 ms. (Each enhanced scale function added to the query appeared to add about 50 ms of processing) e.g. scaled query - q = scale(keywords, 0, 90) and scale(geo, 0, 10) e.g. unscaled query - q = keywords and geo In both cases fq includes keywords and geo. In order to accomplish this goal I had to introduce a couple of changes: 1) In the indexsearcher.search method where scorers are created and then used to score on a per-atomicreadercontext basis I had to make it so that all scorers would be created before any scoring was done. This was so that the scale function would have an opportunity to observe the entire index before being asked to score something. 2) Introduced a new property to the Bits interface that indicates whether or not the bits provide constant-time access. Why? Read on. 3) FilterSet used to return Null when asked for its bits because it did not have any, it had an iterator. This was an issue when trying to make it so that scale would only score documents matching the filter. Thus a new bits implementation was added (LazyIteratorBackedBits) that could expose an iterator as a Bits implementation. It advances the iterator on-demand when asked about a document and uses an OpenBitSet to keep track of what it has advanced beyond. Thus once the iterator is exhausted it provides constant-time answers like any other Bits. 4) Introduced a function on the ValueSource interface to allow a Bits to be passed in for filtering purposes. This was originally developed against Solr 4.2 but I have ported it to Solr 4.8. There is one failing unit test related to code that has been added in the interim, AnalyzingInfixSuggesterTest.testRandomNRT. I have not been able to figure out why this test fails. All other tests pass. In relation to implementation detail 1) above, the introduction of LeafCollectors in trunk has caused somewhat of an issue. ( LUCENE-5527 ) It seems to no longer be possible to create multiple scorers without immediately scoring on that LeafCollector. This may be related to the encapsulation of the Collector.setNextReader() method which was very useful for this purpose. was: The existing scale() function examines the scores of all documents in the index in order to calculate its scale constant. This does not perform well in solr on very large indexes or with costly scoring mechanisms such as geo distance. I have developed a patch that allows the scale function to only score documents that match the given filters, thus improving performance of the scale function. For test queries involving two scale operations where one was scaling the result of keyword scoring and the other was scaling the result of geo distance scoring on an index with ~2 million documents, query time was improved from ~400 ms with vanilla scale to ~190 ms with new scale. A similar query using no scaling ran in ~90 ms. (Each enhanced scale function added to the query appeared to add about 50 ms of processing) e.g. scaled query - q = scale(keywords, 0, 90) and scale(geo, 0, 10) e.g. unscaled query - q = keywords and geo In both cases fq includes keywords and geo. In order to accomplish this goal I had to introduce a couple of changes: 1) In the indexsearcher.search method where scorers are created and then used to score on a per-atomicreadercontext basis I had to make it so that all scorers would be created before any scoring was done. This was so that the scale function would have an opportunity to observe the entire index before being asked to score something. 2) Introduced a new property to the Bits interface that indicates whether or not the bits provide constant-time access. Why? Read on. 3) FilterSet used to return Null when asked for its bits because it did not have any, it had an iterator. This was an issue when trying to make it so that scale would only score documents matching the filter. Thus a new bits implementation was added (LazyIteratorBackedBits) that could expose an iterator as a Bits implementation. It advance
[jira] [Commented] (SOLR-6036) Can't create collection with replicationFactor=0
[
https://issues.apache.org/jira/browse/SOLR-6036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13986967#comment-13986967
]
Jack Krupansky commented on SOLR-6036:
--
But I can sympathize - the term "copies of the data" is ambiguous and vague,
unless you have seriously taken the mantra "there is no master!" to heart and
etched it into your arms with acid. Maybe "instances of the data" would be a
little less ambiguous.
> Can't create collection with replicationFactor=0
>
>
> Key: SOLR-6036
> URL: https://issues.apache.org/jira/browse/SOLR-6036
> Project: Solr
> Issue Type: Bug
> Components: SolrCloud
>Affects Versions: 4.3.1, 4.8
>Reporter: John Wong
>Priority: Trivial
>
> solrcloud$ curl
> 'http://localhost:8983/solr/admin/collections?action=CREATE&name=collection&numShards=2&replicationFactor=0'
>
>
> 400 name="QTime">60052org.apache.solr.common.SolrException:org.apache.solr.common.SolrException:
> replicationFactor must be greater than or equal to 0 name="exception">replicationFactor must be greater than or
> equal to 0400 name="msg">replicationFactor must be greater than or equal to 0 name="code">400
>
> I am using solr 4.3.1, but I peeked into the source up to 4.8 and the problem
> still persists, but in 4.8, the exception message now is changed to be
> greater than 0.
> The code snippet in OverseerCollectionProcessor.java:
> if (repFactor <= 0) {
> throw new SolrException(ErrorCode.BAD_REQUEST, REPLICATION_FACTOR + "
> must be greater than 0");
> }
> I believe the <= should just be < as it won't allow 0. It may have been
> legacy from when replicationFactor of 1 included the leader/master copy,
> whereas in solr 4.x, replicationFactor is defined by additional replicas on
> top of the leader.
> http://wiki.apache.org/solr/SolrCloud
> replicationFactor: The number of copies of each document (or, the number of
> physical replicas to be created for each logical shard of the collection.) A
> replicationFactor of 3 means that there will be 3 replicas (one of which is
> normally designated to be the leader) for each logical shard. NOTE: in Solr
> 4.0, replicationFactor was the number of *additional* copies as opposed to
> the total number of copies.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-5637) Scaling scale function
[ https://issues.apache.org/jira/browse/LUCENE-5637?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Russell updated LUCENE-5637: -- Attachment: Lucene-5637.patch Patch for Solr 4.8, with one unit test failure I haven't figured out yet. > Scaling scale function > -- > > Key: LUCENE-5637 > URL: https://issues.apache.org/jira/browse/LUCENE-5637 > Project: Lucene - Core > Issue Type: Improvement >Reporter: Chris Russell >Priority: Minor > Labels: patch, performance > Fix For: 4.8 > > Attachments: Lucene-5637.patch > > > The existing scale() function examines the scores of all documents in the > index in order to calculate its scale constant. This does not perform well > in solr on very large indexes or with costly scoring mechanisms such as geo > distance. > I have developed a patch that allows the scale function to only score > documents that match the given filters, thus improving performance of the > scale function. > For test queries involving two scale operations where one was scaling the > result of keyword scoring and the other was scaling the result of geo > distance scoring on an index with ~2 million documents, query time was > improved from ~400 ms with vanilla scale to ~190 ms with new scale. A > similar query using no scaling ran in ~90 ms. (Each enhanced scale function > added to the query appeared to add about 50 ms of processing) > e.g. scaled query - q = scale(keywords, 0, 90) and scale(geo, 0, 10) > e.g. unscaled query - q = keywords and geo > In both cases fq includes keywords and geo. > In order to accomplish this goal I had to introduce a couple of changes: > 1) In the indexsearcher.search method where scorers are created and then used > to score on a per-atomicreadercontext basis I had to make it so that all > scorers would be created before any scoring was done. This was so that the > scale function would have an opportunity to observe the entire index before > being asked to score something. > 2) Introduced a new property to the Bits interface that indicates whether or > not the bits provide constant-time access. Why? Read on. > 3) FilterSet used to return Null when asked for its bits because it did not > have any, it had an iterator. This was an issue when trying to make it so > that scale would only score documents matching the filter. Thus a new bits > implementation was added (LazyIteratorBackedBits) that could expose an > iterator as a Bits implementation. It advances the iterator on-demand when > asked about a document and uses an OpenBitSet to keep track of what it has > advanced beyond. Thus once the iterator is exhausted it provides > constant-time answers like any other Bits. > 4) Introduced a function on the ValueSource interface to allow a Bits to be > passed in for filtering purposes. > This was originally developed against Solr 4.2 but I have ported it to Solr > 4.8. There is one failing unit test related to code that has been added in > the interim, AnalyzingInfixSuggesterTest.testRandomNRT. I have not been able > to figure out why this test fails. All other tests pass. > In relation to implementation detail 1) above, the introduction of > LeafCollectors in trunk has caused somewhat of an issue. It seems to no > longer be possible to create multiple scorers without immediately scoring on > that LeafCollector. This may be related to the encapsulation of the > Collector.setNextReader() method which was very useful for this purpose. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (LUCENE-5637) Scaling scale function
Chris Russell created LUCENE-5637: - Summary: Scaling scale function Key: LUCENE-5637 URL: https://issues.apache.org/jira/browse/LUCENE-5637 Project: Lucene - Core Issue Type: Improvement Reporter: Chris Russell Priority: Minor Fix For: 4.8 The existing scale() function examines the scores of all documents in the index in order to calculate its scale constant. This does not perform well in solr on very large indexes or with costly scoring mechanisms such as geo distance. I have developed a patch that allows the scale function to only score documents that match the given filters, thus improving performance of the scale function. For test queries involving two scale operations where one was scaling the result of keyword scoring and the other was scaling the result of geo distance scoring on an index with ~2 million documents, query time was improved from ~400 ms with vanilla scale to ~190 ms with new scale. A similar query using no scaling ran in ~90 ms. (Each enhanced scale function added to the query appeared to add about 50 ms of processing) e.g. scaled query - q = scale(keywords, 0, 90) and scale(geo, 0, 10) e.g. unscaled query - q = keywords and geo In both cases fq includes keywords and geo. In order to accomplish this goal I had to introduce a couple of changes: 1) In the indexsearcher.search method where scorers are created and then used to score on a per-atomicreadercontext basis I had to make it so that all scorers would be created before any scoring was done. This was so that the scale function would have an opportunity to observe the entire index before being asked to score something. 2) Introduced a new property to the Bits interface that indicates whether or not the bits provide constant-time access. Why? Read on. 3) FilterSet used to return Null when asked for its bits because it did not have any, it had an iterator. This was an issue when trying to make it so that scale would only score documents matching the filter. Thus a new bits implementation was added (LazyIteratorBackedBits) that could expose an iterator as a Bits implementation. It advances the iterator on-demand when asked about a document and uses an OpenBitSet to keep track of what it has advanced beyond. Thus once the iterator is exhausted it provides constant-time answers like any other Bits. 4) Introduced a function on the ValueSource interface to allow a Bits to be passed in for filtering purposes. This was originally developed against Solr 4.2 but I have ported it to Solr 4.8. There is one failing unit test related to code that has been added in the interim, AnalyzingInfixSuggesterTest.testRandomNRT. I have not been able to figure out why this test fails. All other tests pass. In relation to implementation detail 1) above, the introduction of LeafCollectors in trunk has caused somewhat of an issue. It seems to no longer be possible to create multiple scorers without immediately scoring on that LeafCollector. This may be related to the encapsulation of the Collector.setNextReader() method which was very useful for this purpose. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5376) Add a demo search server
[ https://issues.apache.org/jira/browse/LUCENE-5376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13986944#comment-13986944 ] ASF subversion and git services commented on LUCENE-5376: - Commit 1591764 from jd...@apache.org in branch 'dev/branches/lucene5376_2' [ https://svn.apache.org/r1591764 ] LUCENE-5376: HelpHandler fixes > Add a demo search server > > > Key: LUCENE-5376 > URL: https://issues.apache.org/jira/browse/LUCENE-5376 > Project: Lucene - Core > Issue Type: Improvement >Reporter: Michael McCandless >Assignee: Michael McCandless > Attachments: lucene-demo-server.tgz > > > I think it'd be useful to have a "demo" search server for Lucene. > Rather than being fully featured, like Solr, it would be minimal, just > wrapping the existing Lucene modules to show how you can make use of these > features in a server setting. > The purpose is to demonstrate how one can build a minimal search server on > top of APIs like SearchManager, SearcherLifetimeManager, etc. > This is also useful for finding rough edges / issues in Lucene's APIs that > make building a server unnecessarily hard. > I don't think it should have back compatibility promises (except Lucene's > index back compatibility), so it's free to improve as Lucene's APIs change. > As a starting point, I'll post what I built for the "eating your own dog > food" search app for Lucene's & Solr's jira issues > http://jirasearch.mikemccandless.com (blog: > http://blog.mikemccandless.com/2013/05/eating-dog-food-with-lucene.html ). It > uses Netty to expose basic indexing & searching APIs via JSON, but it's very > rough (lots nocommits). -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5376) Add a demo search server
[ https://issues.apache.org/jira/browse/LUCENE-5376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13986940#comment-13986940 ] ASF subversion and git services commented on LUCENE-5376: - Commit 1591762 from jd...@apache.org in branch 'dev/branches/lucene5376_2' [ https://svn.apache.org/r1591762 ] LUCENE-5376: add HelpHandler to aid new users and provide quick reference > Add a demo search server > > > Key: LUCENE-5376 > URL: https://issues.apache.org/jira/browse/LUCENE-5376 > Project: Lucene - Core > Issue Type: Improvement >Reporter: Michael McCandless >Assignee: Michael McCandless > Attachments: lucene-demo-server.tgz > > > I think it'd be useful to have a "demo" search server for Lucene. > Rather than being fully featured, like Solr, it would be minimal, just > wrapping the existing Lucene modules to show how you can make use of these > features in a server setting. > The purpose is to demonstrate how one can build a minimal search server on > top of APIs like SearchManager, SearcherLifetimeManager, etc. > This is also useful for finding rough edges / issues in Lucene's APIs that > make building a server unnecessarily hard. > I don't think it should have back compatibility promises (except Lucene's > index back compatibility), so it's free to improve as Lucene's APIs change. > As a starting point, I'll post what I built for the "eating your own dog > food" search app for Lucene's & Solr's jira issues > http://jirasearch.mikemccandless.com (blog: > http://blog.mikemccandless.com/2013/05/eating-dog-food-with-lucene.html ). It > uses Netty to expose basic indexing & searching APIs via JSON, but it's very > rough (lots nocommits). -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Closed] (SOLR-6036) Can't create collection with replicationFactor=0
[
https://issues.apache.org/jira/browse/SOLR-6036?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shawn Heisey closed SOLR-6036.
--
Resolution: Invalid
{quote}
I believe the <= should just be < as it won't allow 0. It may have been legacy
from when replicationFactor of 1 included the leader/master copy, whereas in
solr 4.x, replicationFactor is defined by additional replicas on top of the
leader.
http://wiki.apache.org/solr/SolrCloud
replicationFactor: The number of copies of each document (or, the number of
physical replicas to be created for each logical shard of the collection.) A
replicationFactor of 3 means that there will be 3 replicas (one of which is
normally designated to be the leader) for each logical shard. NOTE: in Solr
4.0, replicationFactor was the number of additional copies as opposed to the
total number of copies
{quote}
The "NOTE" above refers specifically to one specific release -- Solr 4.0. In
4.1 and later (including the just-released 4.8), if you specify
replicationFactor=2, you will get a total of two replicas. One of them will be
elected the leader. There will not be a leader and two additional copies --
there will be exactly two. To create a collection that has no redundancy, use
replicationFactor=1.
I have created the collections on my small SolrCloud install with
replicationFactor=2, running Solr 4.2.1. I have exactly two copies of every
shard. I have not seen anything in any SOLR- issue or any code commit to
change this meaning.
The "if" statement in the code is correct. If you specified a replication
factor of zero, there would be no index.
> Can't create collection with replicationFactor=0
>
>
> Key: SOLR-6036
> URL: https://issues.apache.org/jira/browse/SOLR-6036
> Project: Solr
> Issue Type: Bug
> Components: SolrCloud
>Affects Versions: 4.3.1, 4.8
>Reporter: John Wong
>Priority: Trivial
>
> solrcloud$ curl
> 'http://localhost:8983/solr/admin/collections?action=CREATE&name=collection&numShards=2&replicationFactor=0'
>
>
> 400 name="QTime">60052org.apache.solr.common.SolrException:org.apache.solr.common.SolrException:
> replicationFactor must be greater than or equal to 0 name="exception">replicationFactor must be greater than or
> equal to 0400 name="msg">replicationFactor must be greater than or equal to 0 name="code">400
>
> I am using solr 4.3.1, but I peeked into the source up to 4.8 and the problem
> still persists, but in 4.8, the exception message now is changed to be
> greater than 0.
> The code snippet in OverseerCollectionProcessor.java:
> if (repFactor <= 0) {
> throw new SolrException(ErrorCode.BAD_REQUEST, REPLICATION_FACTOR + "
> must be greater than 0");
> }
> I believe the <= should just be < as it won't allow 0. It may have been
> legacy from when replicationFactor of 1 included the leader/master copy,
> whereas in solr 4.x, replicationFactor is defined by additional replicas on
> top of the leader.
> http://wiki.apache.org/solr/SolrCloud
> replicationFactor: The number of copies of each document (or, the number of
> physical replicas to be created for each logical shard of the collection.) A
> replicationFactor of 3 means that there will be 3 replicas (one of which is
> normally designated to be the leader) for each logical shard. NOTE: in Solr
> 4.0, replicationFactor was the number of *additional* copies as opposed to
> the total number of copies.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5635) Better exception testing for indexwriter
[ https://issues.apache.org/jira/browse/LUCENE-5635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13986902#comment-13986902 ] ASF subversion and git services commented on LUCENE-5635: - Commit 1591751 from [~rcmuir] in branch 'dev/branches/lucene_solr_4_8' [ https://svn.apache.org/r1591751 ] LUCENE-5635: make this test more efficient at finding bugs > Better exception testing for indexwriter > > > Key: LUCENE-5635 > URL: https://issues.apache.org/jira/browse/LUCENE-5635 > Project: Lucene - Core > Issue Type: Bug >Reporter: Robert Muir > Fix For: 4.8.1, 4.9, 5.0 > > > Currently we only catch exc-handling bugs depending on where a codec does i/o. > Instead we should add CrankyCodec and CrankyAnalyzer, which throw random > exceptions from any method. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5635) Better exception testing for indexwriter
[ https://issues.apache.org/jira/browse/LUCENE-5635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13986900#comment-13986900 ] ASF subversion and git services commented on LUCENE-5635: - Commit 1591749 from [~rcmuir] in branch 'dev/branches/branch_4x' [ https://svn.apache.org/r1591749 ] LUCENE-5635: make this test more efficient at finding bugs > Better exception testing for indexwriter > > > Key: LUCENE-5635 > URL: https://issues.apache.org/jira/browse/LUCENE-5635 > Project: Lucene - Core > Issue Type: Bug >Reporter: Robert Muir > Fix For: 4.8.1, 4.9, 5.0 > > > Currently we only catch exc-handling bugs depending on where a codec does i/o. > Instead we should add CrankyCodec and CrankyAnalyzer, which throw random > exceptions from any method. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5635) Better exception testing for indexwriter
[ https://issues.apache.org/jira/browse/LUCENE-5635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13986898#comment-13986898 ] ASF subversion and git services commented on LUCENE-5635: - Commit 1591748 from [~rcmuir] in branch 'dev/trunk' [ https://svn.apache.org/r1591748 ] LUCENE-5635: make this test more efficient at finding bugs > Better exception testing for indexwriter > > > Key: LUCENE-5635 > URL: https://issues.apache.org/jira/browse/LUCENE-5635 > Project: Lucene - Core > Issue Type: Bug >Reporter: Robert Muir > Fix For: 4.8.1, 4.9, 5.0 > > > Currently we only catch exc-handling bugs depending on where a codec does i/o. > Instead we should add CrankyCodec and CrankyAnalyzer, which throw random > exceptions from any method. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5635) Better exception testing for indexwriter
[ https://issues.apache.org/jira/browse/LUCENE-5635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13986871#comment-13986871 ] ASF subversion and git services commented on LUCENE-5635: - Commit 1591736 from [~rcmuir] in branch 'dev/branches/lucene_solr_4_8' [ https://svn.apache.org/r1591736 ] LUCENE-5635: fold in dv updates > Better exception testing for indexwriter > > > Key: LUCENE-5635 > URL: https://issues.apache.org/jira/browse/LUCENE-5635 > Project: Lucene - Core > Issue Type: Bug >Reporter: Robert Muir > Fix For: 4.8.1, 4.9, 5.0 > > > Currently we only catch exc-handling bugs depending on where a codec does i/o. > Instead we should add CrankyCodec and CrankyAnalyzer, which throw random > exceptions from any method. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5635) Better exception testing for indexwriter
[ https://issues.apache.org/jira/browse/LUCENE-5635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13986870#comment-13986870 ] ASF subversion and git services commented on LUCENE-5635: - Commit 1591735 from [~rcmuir] in branch 'dev/branches/branch_4x' [ https://svn.apache.org/r1591735 ] LUCENE-5635: fold in dv updates > Better exception testing for indexwriter > > > Key: LUCENE-5635 > URL: https://issues.apache.org/jira/browse/LUCENE-5635 > Project: Lucene - Core > Issue Type: Bug >Reporter: Robert Muir > Fix For: 4.8.1, 4.9, 5.0 > > > Currently we only catch exc-handling bugs depending on where a codec does i/o. > Instead we should add CrankyCodec and CrankyAnalyzer, which throw random > exceptions from any method. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5635) Better exception testing for indexwriter
[ https://issues.apache.org/jira/browse/LUCENE-5635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13986868#comment-13986868 ] ASF subversion and git services commented on LUCENE-5635: - Commit 1591733 from [~rcmuir] in branch 'dev/trunk' [ https://svn.apache.org/r1591733 ] LUCENE-5635: fold in dv updates > Better exception testing for indexwriter > > > Key: LUCENE-5635 > URL: https://issues.apache.org/jira/browse/LUCENE-5635 > Project: Lucene - Core > Issue Type: Bug >Reporter: Robert Muir > Fix For: 4.8.1, 4.9, 5.0 > > > Currently we only catch exc-handling bugs depending on where a codec does i/o. > Instead we should add CrankyCodec and CrankyAnalyzer, which throw random > exceptions from any method. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5636) SegmentCommitInfo continues to list unneeded gen'd files
[ https://issues.apache.org/jira/browse/LUCENE-5636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13986859#comment-13986859 ] Shai Erera commented on LUCENE-5636: Hmm, actually I think I can fix it w/o breaking index back-compat, by manually removing the old .fnm files in SegCommitInfo (I'll also improve how this is done in general). Assuming very little apps, if at all, write their own FieldInfosFormat, I think that's acceptable? I'll work on a patch later. > SegmentCommitInfo continues to list unneeded gen'd files > > > Key: LUCENE-5636 > URL: https://issues.apache.org/jira/browse/LUCENE-5636 > Project: Lucene - Core > Issue Type: Bug > Components: core/index >Reporter: Shai Erera >Assignee: Shai Erera >Priority: Critical > Attachments: LUCENE-5636.patch > > > I thought I handled it in LUCENE-5246, but turns out I didn't handle it > fully. I'll upload a patch which improves the test to expose the bug. I know > where it is, but I'm not sure how to fix it without breaking index > back-compat. Can we do that on experimental features? > The problem is that if you update different fields in different gens, the > FieldInfos files of older gens remain referenced (still!!). I open a new > issue since LUCENE-5246 is already resolved and released, so don't want to > mess up our JIRA... > The severity of the bug is that unneeded files are still referenced in the > index. Everything still works correctly, it's just that .fnm files are still > there. But as I wrote, I'm still not sure how to solve it without requiring > apps that use dv updates to reindex. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5635) Better exception testing for indexwriter
[ https://issues.apache.org/jira/browse/LUCENE-5635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13986855#comment-13986855 ] ASF subversion and git services commented on LUCENE-5635: - Commit 1591728 from [~rcmuir] in branch 'dev/branches/lucene_solr_4_8' [ https://svn.apache.org/r1591728 ] LUCENE-5635: speed up and improve test > Better exception testing for indexwriter > > > Key: LUCENE-5635 > URL: https://issues.apache.org/jira/browse/LUCENE-5635 > Project: Lucene - Core > Issue Type: Bug >Reporter: Robert Muir > Fix For: 4.8.1, 4.9, 5.0 > > > Currently we only catch exc-handling bugs depending on where a codec does i/o. > Instead we should add CrankyCodec and CrankyAnalyzer, which throw random > exceptions from any method. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-5636) SegmentCommitInfo continues to list unneeded gen'd files
[ https://issues.apache.org/jira/browse/LUCENE-5636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shai Erera updated LUCENE-5636: --- Attachment: LUCENE-5636.patch Patch improves the test to expose the bug. > SegmentCommitInfo continues to list unneeded gen'd files > > > Key: LUCENE-5636 > URL: https://issues.apache.org/jira/browse/LUCENE-5636 > Project: Lucene - Core > Issue Type: Bug > Components: core/index >Reporter: Shai Erera >Assignee: Shai Erera >Priority: Critical > Attachments: LUCENE-5636.patch > > > I thought I handled it in LUCENE-5246, but turns out I didn't handle it > fully. I'll upload a patch which improves the test to expose the bug. I know > where it is, but I'm not sure how to fix it without breaking index > back-compat. Can we do that on experimental features? > The problem is that if you update different fields in different gens, the > FieldInfos files of older gens remain referenced (still!!). I open a new > issue since LUCENE-5246 is already resolved and released, so don't want to > mess up our JIRA... > The severity of the bug is that unneeded files are still referenced in the > index. Everything still works correctly, it's just that .fnm files are still > there. But as I wrote, I'm still not sure how to solve it without requiring > apps that use dv updates to reindex. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5635) Better exception testing for indexwriter
[ https://issues.apache.org/jira/browse/LUCENE-5635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13986850#comment-13986850 ] ASF subversion and git services commented on LUCENE-5635: - Commit 1591727 from [~rcmuir] in branch 'dev/branches/branch_4x' [ https://svn.apache.org/r1591727 ] LUCENE-5635: speed up and improve test > Better exception testing for indexwriter > > > Key: LUCENE-5635 > URL: https://issues.apache.org/jira/browse/LUCENE-5635 > Project: Lucene - Core > Issue Type: Bug >Reporter: Robert Muir > Fix For: 4.8.1, 4.9, 5.0 > > > Currently we only catch exc-handling bugs depending on where a codec does i/o. > Instead we should add CrankyCodec and CrankyAnalyzer, which throw random > exceptions from any method. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5635) Better exception testing for indexwriter
[ https://issues.apache.org/jira/browse/LUCENE-5635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13986848#comment-13986848 ] ASF subversion and git services commented on LUCENE-5635: - Commit 1591726 from [~rcmuir] in branch 'dev/trunk' [ https://svn.apache.org/r1591726 ] LUCENE-5635: speed up and improve test > Better exception testing for indexwriter > > > Key: LUCENE-5635 > URL: https://issues.apache.org/jira/browse/LUCENE-5635 > Project: Lucene - Core > Issue Type: Bug >Reporter: Robert Muir > Fix For: 4.8.1, 4.9, 5.0 > > > Currently we only catch exc-handling bugs depending on where a codec does i/o. > Instead we should add CrankyCodec and CrankyAnalyzer, which throw random > exceptions from any method. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (LUCENE-5636) SegmentCommitInfo continues to list unneeded gen'd files
Shai Erera created LUCENE-5636: -- Summary: SegmentCommitInfo continues to list unneeded gen'd files Key: LUCENE-5636 URL: https://issues.apache.org/jira/browse/LUCENE-5636 Project: Lucene - Core Issue Type: Bug Components: core/index Reporter: Shai Erera Assignee: Shai Erera Priority: Critical I thought I handled it in LUCENE-5246, but turns out I didn't handle it fully. I'll upload a patch which improves the test to expose the bug. I know where it is, but I'm not sure how to fix it without breaking index back-compat. Can we do that on experimental features? The problem is that if you update different fields in different gens, the FieldInfos files of older gens remain referenced (still!!). I open a new issue since LUCENE-5246 is already resolved and released, so don't want to mess up our JIRA... The severity of the bug is that unneeded files are still referenced in the index. Everything still works correctly, it's just that .fnm files are still there. But as I wrote, I'm still not sure how to solve it without requiring apps that use dv updates to reindex. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-5495) Recovery strategy for leader partitioned from replica case.
[ https://issues.apache.org/jira/browse/SOLR-5495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Timothy Potter updated SOLR-5495: - Attachment: SOLR-5495.patch Updated patch to fix compilation error after backing out SOLR-5473 and tighten up the unit test, such as reducing the amount of time it waits before healing a partition. Also cleaned up a few minor issues in the LeaderInitiatedRecoveryThread loop. > Recovery strategy for leader partitioned from replica case. > --- > > Key: SOLR-5495 > URL: https://issues.apache.org/jira/browse/SOLR-5495 > Project: Solr > Issue Type: Bug >Reporter: Mark Miller >Assignee: Timothy Potter > Attachments: SOLR-5495.patch, SOLR-5495.patch, SOLR-5495.patch > > > We need to work out a strategy for the case of: > Leader and replicas can still talk to ZooKeeper, Leader cannot talk to > replica. > We punted on this in initial design, but I'd like to get something in. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5635) Better exception testing for indexwriter
[ https://issues.apache.org/jira/browse/LUCENE-5635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13986762#comment-13986762 ] ASF subversion and git services commented on LUCENE-5635: - Commit 1591710 from [~rcmuir] in branch 'dev/branches/lucene_solr_4_8' [ https://svn.apache.org/r1591710 ] LUCENE-5635: add some deletes and open reader from writer > Better exception testing for indexwriter > > > Key: LUCENE-5635 > URL: https://issues.apache.org/jira/browse/LUCENE-5635 > Project: Lucene - Core > Issue Type: Bug >Reporter: Robert Muir > Fix For: 4.8.1, 4.9, 5.0 > > > Currently we only catch exc-handling bugs depending on where a codec does i/o. > Instead we should add CrankyCodec and CrankyAnalyzer, which throw random > exceptions from any method. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5635) Better exception testing for indexwriter
[ https://issues.apache.org/jira/browse/LUCENE-5635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13986759#comment-13986759 ] ASF subversion and git services commented on LUCENE-5635: - Commit 1591709 from [~rcmuir] in branch 'dev/branches/branch_4x' [ https://svn.apache.org/r1591709 ] LUCENE-5635: add some deletes and open reader from writer > Better exception testing for indexwriter > > > Key: LUCENE-5635 > URL: https://issues.apache.org/jira/browse/LUCENE-5635 > Project: Lucene - Core > Issue Type: Bug >Reporter: Robert Muir > Fix For: 4.8.1, 4.9, 5.0 > > > Currently we only catch exc-handling bugs depending on where a codec does i/o. > Instead we should add CrankyCodec and CrankyAnalyzer, which throw random > exceptions from any method. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5635) Better exception testing for indexwriter
[ https://issues.apache.org/jira/browse/LUCENE-5635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13986757#comment-13986757 ] ASF subversion and git services commented on LUCENE-5635: - Commit 1591708 from [~rcmuir] in branch 'dev/trunk' [ https://svn.apache.org/r1591708 ] LUCENE-5635: add some deletes and open reader from writer > Better exception testing for indexwriter > > > Key: LUCENE-5635 > URL: https://issues.apache.org/jira/browse/LUCENE-5635 > Project: Lucene - Core > Issue Type: Bug >Reporter: Robert Muir > Fix For: 4.8.1, 4.9, 5.0 > > > Currently we only catch exc-handling bugs depending on where a codec does i/o. > Instead we should add CrankyCodec and CrankyAnalyzer, which throw random > exceptions from any method. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5611) Simplify the default indexing chain
[ https://issues.apache.org/jira/browse/LUCENE-5611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13986748#comment-13986748 ] ASF subversion and git services commented on LUCENE-5611: - Commit 1591706 from [~mikemccand] in branch 'dev/branches/branch_4x' [ https://svn.apache.org/r1591706 ] LUCENE-5611: don't abort segment if term vector settings are wrong > Simplify the default indexing chain > --- > > Key: LUCENE-5611 > URL: https://issues.apache.org/jira/browse/LUCENE-5611 > Project: Lucene - Core > Issue Type: Improvement > Components: core/index >Reporter: Michael McCandless >Assignee: Michael McCandless > Fix For: 4.9, 5.0 > > Attachments: LUCENE-5611.patch, LUCENE-5611.patch > > > I think Lucene's current indexing chain has too many classes / > hierarchy / abstractions, making it look much more complex than it > really should be, and discouraging users from experimenting/innovating > with their own indexing chains. > Also, if it were easier to understand/approach, then new developers > would more likely try to improve it ... it really should be simpler. > So I'm exploring a pared back indexing chain, and have a starting patch > that I think is looking ok: it seems more approachable than the > current indexing chain, or at least has fewer strange classes. > I also thought this could give some speedup for tiny documents (a more > common use of Lucene lately), and it looks like, with the evil > optimizations, this is a ~25% speedup for Geonames docs. Even without > those evil optos it's a bit faster. > This is very much a work in progress / nocommits, and there are some > behavior changes e.g. the new chain requires all fields to have the > same TV options (rather than auto-upgrading all fields by the same > name that the current chain does)... -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5611) Simplify the default indexing chain
[ https://issues.apache.org/jira/browse/LUCENE-5611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13986735#comment-13986735 ] ASF subversion and git services commented on LUCENE-5611: - Commit 1591702 from [~rcmuir] in branch 'dev/branches/branch_4x' [ https://svn.apache.org/r1591702 ] LUCENE-5611: nuke dup buggy test > Simplify the default indexing chain > --- > > Key: LUCENE-5611 > URL: https://issues.apache.org/jira/browse/LUCENE-5611 > Project: Lucene - Core > Issue Type: Improvement > Components: core/index >Reporter: Michael McCandless >Assignee: Michael McCandless > Fix For: 4.9, 5.0 > > Attachments: LUCENE-5611.patch, LUCENE-5611.patch > > > I think Lucene's current indexing chain has too many classes / > hierarchy / abstractions, making it look much more complex than it > really should be, and discouraging users from experimenting/innovating > with their own indexing chains. > Also, if it were easier to understand/approach, then new developers > would more likely try to improve it ... it really should be simpler. > So I'm exploring a pared back indexing chain, and have a starting patch > that I think is looking ok: it seems more approachable than the > current indexing chain, or at least has fewer strange classes. > I also thought this could give some speedup for tiny documents (a more > common use of Lucene lately), and it looks like, with the evil > optimizations, this is a ~25% speedup for Geonames docs. Even without > those evil optos it's a bit faster. > This is very much a work in progress / nocommits, and there are some > behavior changes e.g. the new chain requires all fields to have the > same TV options (rather than auto-upgrading all fields by the same > name that the current chain does)... -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5611) Simplify the default indexing chain
[ https://issues.apache.org/jira/browse/LUCENE-5611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13986731#comment-13986731 ] ASF subversion and git services commented on LUCENE-5611: - Commit 1591701 from [~rcmuir] in branch 'dev/trunk' [ https://svn.apache.org/r1591701 ] LUCENE-5611: nuke dup buggy test > Simplify the default indexing chain > --- > > Key: LUCENE-5611 > URL: https://issues.apache.org/jira/browse/LUCENE-5611 > Project: Lucene - Core > Issue Type: Improvement > Components: core/index >Reporter: Michael McCandless >Assignee: Michael McCandless > Fix For: 4.9, 5.0 > > Attachments: LUCENE-5611.patch, LUCENE-5611.patch > > > I think Lucene's current indexing chain has too many classes / > hierarchy / abstractions, making it look much more complex than it > really should be, and discouraging users from experimenting/innovating > with their own indexing chains. > Also, if it were easier to understand/approach, then new developers > would more likely try to improve it ... it really should be simpler. > So I'm exploring a pared back indexing chain, and have a starting patch > that I think is looking ok: it seems more approachable than the > current indexing chain, or at least has fewer strange classes. > I also thought this could give some speedup for tiny documents (a more > common use of Lucene lately), and it looks like, with the evil > optimizations, this is a ~25% speedup for Geonames docs. Even without > those evil optos it's a bit faster. > This is very much a work in progress / nocommits, and there are some > behavior changes e.g. the new chain requires all fields to have the > same TV options (rather than auto-upgrading all fields by the same > name that the current chain does)... -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5611) Simplify the default indexing chain
[ https://issues.apache.org/jira/browse/LUCENE-5611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13986730#comment-13986730 ] ASF subversion and git services commented on LUCENE-5611: - Commit 1591700 from [~mikemccand] in branch 'dev/trunk' [ https://svn.apache.org/r1591700 ] LUCENE-5611: don't abort segment if term vector settings are wrong > Simplify the default indexing chain > --- > > Key: LUCENE-5611 > URL: https://issues.apache.org/jira/browse/LUCENE-5611 > Project: Lucene - Core > Issue Type: Improvement > Components: core/index >Reporter: Michael McCandless >Assignee: Michael McCandless > Fix For: 4.9, 5.0 > > Attachments: LUCENE-5611.patch, LUCENE-5611.patch > > > I think Lucene's current indexing chain has too many classes / > hierarchy / abstractions, making it look much more complex than it > really should be, and discouraging users from experimenting/innovating > with their own indexing chains. > Also, if it were easier to understand/approach, then new developers > would more likely try to improve it ... it really should be simpler. > So I'm exploring a pared back indexing chain, and have a starting patch > that I think is looking ok: it seems more approachable than the > current indexing chain, or at least has fewer strange classes. > I also thought this could give some speedup for tiny documents (a more > common use of Lucene lately), and it looks like, with the evil > optimizations, this is a ~25% speedup for Geonames docs. Even without > those evil optos it's a bit faster. > This is very much a work in progress / nocommits, and there are some > behavior changes e.g. the new chain requires all fields to have the > same TV options (rather than auto-upgrading all fields by the same > name that the current chain does)... -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: [JENKINS] Lucene-trunk-Linux-Java7-64-test-only - Build # 83995 - Failure!
This looks like a clone of the same buggy test as
TestTermVectors.testMixedVectrosVectors
I rewrote that test as TestTermVectorsWriter.testInconsistentTermVectorOptions
I will see if this one really has something different to offer, if it
doesnt i will delete the test.
On Thu, May 1, 2014 at 12:13 PM, wrote:
> Build: builds.flonkings.com/job/Lucene-trunk-Linux-Java7-64-test-only/83995/
>
> 1 tests failed.
> REGRESSION:
> org.apache.lucene.index.TestDocumentWriter.testMixedTermVectorSettingsSameField
>
> Error Message:
> MockDirectoryWrapper: cannot close: there are still open files: {_0.fdt=1,
> _0.fdx=1}
>
> Stack Trace:
> java.lang.RuntimeException: MockDirectoryWrapper: cannot close: there are
> still open files: {_0.fdt=1, _0.fdx=1}
> at
> org.apache.lucene.store.MockDirectoryWrapper.close(MockDirectoryWrapper.java:663)
> at
> org.apache.lucene.index.TestDocumentWriter.tearDown(TestDocumentWriter.java:50)
> at sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:885)
> at
> org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
> at
> org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
> at
> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
> at
> com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
> at
> org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
> at
> org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
> at
> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:360)
> at
> com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:793)
> at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:453)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
> at
> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
> at
> org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
> at
> com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
> at
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
> at
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
> at
> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
> at
> org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
> at
> org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:360)
> at java.lang.Thread.run(Thread.java:724)
> Caused by: java.lang.RuntimeException: unclosed IndexOutput: _0.fdx
> at
> org.apache.lucene.store.MockDirectoryWrapper.addFileHandle(MockDirectoryWrapper.java:550)
> at
> org.apache.lucene.store.MockDirectoryWra
[jira] [Updated] (SOLR-5468) Option to enforce a majority quorum approach to accepting updates in SolrCloud
[ https://issues.apache.org/jira/browse/SOLR-5468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Timothy Potter updated SOLR-5468: - Attachment: SOLR-5468.patch Here is a patch that should be classified as exploratory / discovery into this topic. It has a little overlap with the patch I posted for SOLR-5495, but not to worry on that as I plan to commit SOLR-5495 first so the overlap will get worked out shortly. As I dug into this idea in more detail, it became pretty clear that what we can accomplish in the area of providing stronger enforcement of replication during update processing is fairly limited by our architecture. Of course this is not a criticism of the current architecture as I feel it’s fundamentally sound. The underlying concept in this feature is a client application wants to ensure a write succeeds on more than just the leader. For instance in a collection with RF=3, the client may want to say don’t consider an update request is successful unless it succeeds on 2 of the 3 replicas, vs. how it works today is an update request is considered successful if it succeeds on the leader only. The problem is that there’s no consistent way to “back out” an update without some sort of distributed transaction coordination among the replicas, which I’m pretty sure we don’t want to even go down that road. Backing out an update seems doable (potentially) if we’re just talking about the leader but what happens when the client wants minRF=3 and the update only works on the leader and one of the replicas? Now we’re needing to back out an update from the leader and one of the replicas. Gets ugly fast … So what is accomplished in this patch? First, a client application has the ability to request information back from the cluster on what replication factor was achieved for an update request by sending the min_rf parameter in the request. This is the hint to the leader to keep track of the success or failure of the request on each replica. As that implies some waiting to see the result, the client can also send the max_rfwait parameter that tells the leader how long it should wait to collect the results from the replicas (default is 5 seconds). This is captured in the ReplicationFactorTest class. This can be useful for client applications that have idempotent updates and thus decide to retry the updates if the desired replication factor was not achieved. What we can’t do is fail the request if the desired min_rf is not achieved as that leads to the aforementioned backing out issues. There is one case where we can fail the request and avoid the backing out issue is if we know the min_rf can’t be achieved before we do the write locally on the leader first. This patch doesn’t have that solution in place as I wasn’t sure if that’s desired? If so, it will be easy to add that. Also, this patch doesn’t have anything in place for batch processing, ie. only works with single update requests as I wanted to get some feedback before going down that path any further. Moreover, there’s a pretty high cost in terms of slowing down update request processing in SolrCloud by having the leader block until it knows the result of the request on the replicas. In other words, this functionality is not for free but may still be useful for some applications? To me, at the end of the day, what’s really needed is to ensure that any update requests that were ack’d back to the client are not lost. This could happen under the current architecture if the leader is the only replica that has a write and then fails and doesn’t recover before another replica recovers and resumes the leadership role (after waiting too long to see the previous leader come back). Thus, from where I sit, our efforts are better spent on continuing to harden the leader failover and recovery processes and applications needing stronger guarantees should have more replicas. SolrCloud should then just focus on only allowing in-sync replicas to become the leader using strategies like what was provided with SOLR-5495. > Option to enforce a majority quorum approach to accepting updates in SolrCloud > -- > > Key: SOLR-5468 > URL: https://issues.apache.org/jira/browse/SOLR-5468 > Project: Solr > Issue Type: New Feature > Components: SolrCloud >Affects Versions: 4.5 > Environment: All >Reporter: Timothy Potter >Assignee: Timothy Potter >Priority: Minor > Attachments: SOLR-5468.patch > > > I've been thinking about how SolrCloud deals with write-availability using > in-sync replica sets, in which writes will continue to be accepted so long as > there is at least one healthy node per shard. > For a little background (and to verify my understanding of the process is > correc
[JENKINS] Lucene-trunk-Linux-Java7-64-test-only - Build # 83995 - Failure!
Build: builds.flonkings.com/job/Lucene-trunk-Linux-Java7-64-test-only/83995/
1 tests failed.
REGRESSION:
org.apache.lucene.index.TestDocumentWriter.testMixedTermVectorSettingsSameField
Error Message:
MockDirectoryWrapper: cannot close: there are still open files: {_0.fdt=1,
_0.fdx=1}
Stack Trace:
java.lang.RuntimeException: MockDirectoryWrapper: cannot close: there are still
open files: {_0.fdt=1, _0.fdx=1}
at
org.apache.lucene.store.MockDirectoryWrapper.close(MockDirectoryWrapper.java:663)
at
org.apache.lucene.index.TestDocumentWriter.tearDown(TestDocumentWriter.java:50)
at sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:885)
at
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at
org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at
com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:360)
at
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:793)
at
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:453)
at
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
at
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at
com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:360)
at java.lang.Thread.run(Thread.java:724)
Caused by: java.lang.RuntimeException: unclosed IndexOutput: _0.fdx
at
org.apache.lucene.store.MockDirectoryWrapper.addFileHandle(MockDirectoryWrapper.java:550)
at
org.apache.lucene.store.MockDirectoryWrapper.createOutput(MockDirectoryWrapper.java:522)
at
org.apache.lucene.store.TrackingDirectoryWrapper.createOutput(TrackingDirectoryWrapper.java:44)
at
org.apache.lucene.codecs.compressing.CompressingStoredFieldsWriter.(CompressingStoredFieldsWriter.java:110)
at
org.apache.lucene.codecs.compressing.CompressingStoredFieldsFormat.fieldsWriter(CompressingStoredFieldsFormat.java:120)
at
org.apache.lucene.index.DefaultIndexingChain.(DefaultIndexingChain.ja
[jira] [Commented] (LUCENE-5635) Better exception testing for indexwriter
[ https://issues.apache.org/jira/browse/LUCENE-5635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13986706#comment-13986706 ] ASF subversion and git services commented on LUCENE-5635: - Commit 1591698 from [~rcmuir] in branch 'dev/branches/lucene_solr_4_8' [ https://svn.apache.org/r1591698 ] LUCENE-5635: Better exception testing for indexwriter > Better exception testing for indexwriter > > > Key: LUCENE-5635 > URL: https://issues.apache.org/jira/browse/LUCENE-5635 > Project: Lucene - Core > Issue Type: Bug >Reporter: Robert Muir > Fix For: 4.8.1, 4.9, 5.0 > > > Currently we only catch exc-handling bugs depending on where a codec does i/o. > Instead we should add CrankyCodec and CrankyAnalyzer, which throw random > exceptions from any method. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Resolved] (LUCENE-5635) Better exception testing for indexwriter
[ https://issues.apache.org/jira/browse/LUCENE-5635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Muir resolved LUCENE-5635. - Resolution: Fixed Fix Version/s: 5.0 4.9 4.8.1 > Better exception testing for indexwriter > > > Key: LUCENE-5635 > URL: https://issues.apache.org/jira/browse/LUCENE-5635 > Project: Lucene - Core > Issue Type: Bug >Reporter: Robert Muir > Fix For: 4.8.1, 4.9, 5.0 > > > Currently we only catch exc-handling bugs depending on where a codec does i/o. > Instead we should add CrankyCodec and CrankyAnalyzer, which throw random > exceptions from any method. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5635) Better exception testing for indexwriter
[ https://issues.apache.org/jira/browse/LUCENE-5635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13986695#comment-13986695 ] ASF subversion and git services commented on LUCENE-5635: - Commit 1591694 from [~rcmuir] in branch 'dev/branches/branch_4x' [ https://svn.apache.org/r1591694 ] LUCENE-5635: consistent with trunk > Better exception testing for indexwriter > > > Key: LUCENE-5635 > URL: https://issues.apache.org/jira/browse/LUCENE-5635 > Project: Lucene - Core > Issue Type: Bug >Reporter: Robert Muir > > Currently we only catch exc-handling bugs depending on where a codec does i/o. > Instead we should add CrankyCodec and CrankyAnalyzer, which throw random > exceptions from any method. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5635) Better exception testing for indexwriter
[ https://issues.apache.org/jira/browse/LUCENE-5635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13986690#comment-13986690 ] ASF subversion and git services commented on LUCENE-5635: - Commit 1591691 from [~rcmuir] in branch 'dev/branches/branch_4x' [ https://svn.apache.org/r1591691 ] LUCENE-5635: Better exception testing for indexwriter > Better exception testing for indexwriter > > > Key: LUCENE-5635 > URL: https://issues.apache.org/jira/browse/LUCENE-5635 > Project: Lucene - Core > Issue Type: Bug >Reporter: Robert Muir > > Currently we only catch exc-handling bugs depending on where a codec does i/o. > Instead we should add CrankyCodec and CrankyAnalyzer, which throw random > exceptions from any method. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5635) Better exception testing for indexwriter
[ https://issues.apache.org/jira/browse/LUCENE-5635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13986660#comment-13986660 ] ASF subversion and git services commented on LUCENE-5635: - Commit 1591676 from [~mikemccand] in branch 'dev/trunk' [ https://svn.apache.org/r1591676 ] LUCENE-5635: enable this nice new test; make all FieldsConsumers impl Closeable and move closing responsibility to caller > Better exception testing for indexwriter > > > Key: LUCENE-5635 > URL: https://issues.apache.org/jira/browse/LUCENE-5635 > Project: Lucene - Core > Issue Type: Bug >Reporter: Robert Muir > > Currently we only catch exc-handling bugs depending on where a codec does i/o. > Instead we should add CrankyCodec and CrankyAnalyzer, which throw random > exceptions from any method. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-6035) CloudSolrServer directUpdate routing should use getCoreUrl
[ https://issues.apache.org/jira/browse/SOLR-6035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joel Bernstein updated SOLR-6035: - Fix Version/s: 4.9 > CloudSolrServer directUpdate routing should use getCoreUrl > --- > > Key: SOLR-6035 > URL: https://issues.apache.org/jira/browse/SOLR-6035 > Project: Solr > Issue Type: Improvement > Components: SolrCloud >Reporter: Marvin Justice >Assignee: Joel Bernstein >Priority: Minor > Labels: performance > Fix For: 4.9 > > Attachments: SOLR-6035.patch, SOLR-6035.patch > > > In a multisharded node environment we were seeing forward-to-leader hops when > using CloudSolrServer directUpdate (with the hop being on the same node) . > Consequently, there was no improvement in indexing performance over > non-directUpdate. > Changing buildUrlMap to use getCoreUrl eliminated the extra hop and now we > see a dramatic improvement in performance. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (SOLR-6035) CloudSolrServer directUpdate routing should use getCoreUrl
[ https://issues.apache.org/jira/browse/SOLR-6035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13986654#comment-13986654 ] Joel Bernstein edited comment on SOLR-6035 at 5/1/14 3:04 PM: -- A review of the DistributedUpdateProcessor looks like it should handle going directly to the core. The SolrDispatchFilter does attempt to pick out the leader core for a collection. If there are two leaders from the same collection in the same Solr instance then it could pick the wrong leader and this would cause the routing on the server side. All tests are passing with the patch. I'll do some manual testing, but so far so good. Mark, let me know if you think I'm missing something. was (Author: joel.bernstein): A review of the DistributedUpdateProcessor looks like it should handle going directly to the core. The SolrDispatchFilter does attempt to pick out the leader core for a collection. If there are two leaders from the same collection in the same Solr instance then it could pick the wrong leader and this would cause the routing on the server side. All tests are passing with patch. I'll do some manual testing, but so far so good. Mark, let me know if you think I'm missing something. > CloudSolrServer directUpdate routing should use getCoreUrl > --- > > Key: SOLR-6035 > URL: https://issues.apache.org/jira/browse/SOLR-6035 > Project: Solr > Issue Type: Improvement > Components: SolrCloud >Reporter: Marvin Justice >Assignee: Joel Bernstein >Priority: Minor > Labels: performance > Attachments: SOLR-6035.patch, SOLR-6035.patch > > > In a multisharded node environment we were seeing forward-to-leader hops when > using CloudSolrServer directUpdate (with the hop being on the same node) . > Consequently, there was no improvement in indexing performance over > non-directUpdate. > Changing buildUrlMap to use getCoreUrl eliminated the extra hop and now we > see a dramatic improvement in performance. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-6035) CloudSolrServer directUpdate routing should use getCoreUrl
[ https://issues.apache.org/jira/browse/SOLR-6035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13986654#comment-13986654 ] Joel Bernstein commented on SOLR-6035: -- A review of the DistributedUpdateProcessor looks like it should handle going directly to the core. The SolrDispatchFilter does attempt to pick out the leader core for a collection. If there are two leaders from the same collection in the same Solr instance then it could pick the wrong leader and this would cause the routing on the server side. All tests are passing with patch. I'll do some manual testing, but so far so good. Mark, let me know if you think I'm missing something. > CloudSolrServer directUpdate routing should use getCoreUrl > --- > > Key: SOLR-6035 > URL: https://issues.apache.org/jira/browse/SOLR-6035 > Project: Solr > Issue Type: Improvement > Components: SolrCloud >Reporter: Marvin Justice >Assignee: Joel Bernstein >Priority: Minor > Labels: performance > Attachments: SOLR-6035.patch, SOLR-6035.patch > > > In a multisharded node environment we were seeing forward-to-leader hops when > using CloudSolrServer directUpdate (with the hop being on the same node) . > Consequently, there was no improvement in indexing performance over > non-directUpdate. > Changing buildUrlMap to use getCoreUrl eliminated the extra hop and now we > see a dramatic improvement in performance. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-6037) Stats Component Reports Incorrect Max in Distributed Data
[
https://issues.apache.org/jira/browse/SOLR-6037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brett Lucey updated SOLR-6037:
--
Description:
When computing statistics using the statistics component, the value for max may
be reported incorrectly if the maximum value does not lie in the first shard.
This is where the problem is:
{code:title=StatsValuesFactory.java|borderStyle=solid}
@Override
protected void updateMinMax(Date min, Date max) {
if(this.min==null || this.min.after(min)) {
this.min = min;
}
if(this.max==null || this.max.before(min)) {
this.max = max;
}
}
{code}
Attached is a one-line patch which fixes this issue.
was:
When computing statistics using the statistics component, the value for max may
be reported incorrectly if the maximum value does not lie in the first shard.
This is where the problem is (in StatsValuesFactory.java):
{{
@Override
protected void updateMinMax(Date min, Date max) {
if(this.min==null || this.min.after(min)) {
this.min = min;
}
if(this.max==null || this.max.before(min)) {
this.max = max;
}
}
}}
Attached is a one-line patch which fixes this issue.
> Stats Component Reports Incorrect Max in Distributed Data
> -
>
> Key: SOLR-6037
> URL: https://issues.apache.org/jira/browse/SOLR-6037
> Project: Solr
> Issue Type: Bug
> Components: SearchComponents - other
>Affects Versions: 5.0
>Reporter: Brett Lucey
>Priority: Minor
> Attachments: SOLR-6037.patch
>
>
> When computing statistics using the statistics component, the value for max
> may be reported incorrectly if the maximum value does not lie in the first
> shard.
> This is where the problem is:
> {code:title=StatsValuesFactory.java|borderStyle=solid}
> @Override
> protected void updateMinMax(Date min, Date max) {
> if(this.min==null || this.min.after(min)) {
> this.min = min;
> }
> if(this.max==null || this.max.before(min)) {
> this.max = max;
> }
> }
> {code}
> Attached is a one-line patch which fixes this issue.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-6037) Stats Component Reports Incorrect Max in Distributed Data
[
https://issues.apache.org/jira/browse/SOLR-6037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brett Lucey updated SOLR-6037:
--
Description:
When computing statistics using the statistics component, the value for max may
be reported incorrectly if the maximum value does not lie in the first shard.
This is where the problem is (in StatsValuesFactory.java):
{{
@Override
protected void updateMinMax(Date min, Date max) {
if(this.min==null || this.min.after(min)) {
this.min = min;
}
if(this.max==null || this.max.before(min)) {
this.max = max;
}
}
}}
Attached is a one-line patch which fixes this issue.
was:
When computing statistics using the statistics component, the value for max may
be reported incorrectly if the maximum value does not lie in the first shard.
This is where the problem is (in StatsValueFactory.java):
@Override
protected void updateMinMax(Date min, Date max) {
if(this.min==null || this.min.after(min)) {
this.min = min;
}
if(this.max==null || this.max.before(min)) {
this.max = max;
}
}
Attached is a one-line patch which fixes this issue.
> Stats Component Reports Incorrect Max in Distributed Data
> -
>
> Key: SOLR-6037
> URL: https://issues.apache.org/jira/browse/SOLR-6037
> Project: Solr
> Issue Type: Bug
> Components: SearchComponents - other
>Affects Versions: 5.0
>Reporter: Brett Lucey
>Priority: Minor
> Attachments: SOLR-6037.patch
>
>
> When computing statistics using the statistics component, the value for max
> may be reported incorrectly if the maximum value does not lie in the first
> shard.
> This is where the problem is (in StatsValuesFactory.java):
> {{
> @Override
> protected void updateMinMax(Date min, Date max) {
> if(this.min==null || this.min.after(min)) {
> this.min = min;
> }
> if(this.max==null || this.max.before(min)) {
> this.max = max;
> }
> }
> }}
> Attached is a one-line patch which fixes this issue.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-6037) Stats Component Reports Incorrect Max in Distributed Data
[
https://issues.apache.org/jira/browse/SOLR-6037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brett Lucey updated SOLR-6037:
--
Description:
When computing statistics using the statistics component, the value for max may
be reported incorrectly if the maximum value does not lie in the first shard.
This is where the problem is (in StatsValueFactory.java):
@Override
protected void updateMinMax(Date min, Date max) {
if(this.min==null || this.min.after(min)) {
this.min = min;
}
if(this.max==null || this.max.before(min)) {
this.max = max;
}
}
Attached is a one-line patch which fixes this issue.
> Stats Component Reports Incorrect Max in Distributed Data
> -
>
> Key: SOLR-6037
> URL: https://issues.apache.org/jira/browse/SOLR-6037
> Project: Solr
> Issue Type: Bug
> Components: SearchComponents - other
>Affects Versions: 5.0
>Reporter: Brett Lucey
>Priority: Minor
>
> When computing statistics using the statistics component, the value for max
> may be reported incorrectly if the maximum value does not lie in the first
> shard.
> This is where the problem is (in StatsValueFactory.java):
> @Override
> protected void updateMinMax(Date min, Date max) {
> if(this.min==null || this.min.after(min)) {
> this.min = min;
> }
> if(this.max==null || this.max.before(min)) {
> this.max = max;
> }
> }
> Attached is a one-line patch which fixes this issue.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-6037) Stats Component Reports Incorrect Max in Distributed Data
[
https://issues.apache.org/jira/browse/SOLR-6037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brett Lucey updated SOLR-6037:
--
Attachment: SOLR-6037.patch
> Stats Component Reports Incorrect Max in Distributed Data
> -
>
> Key: SOLR-6037
> URL: https://issues.apache.org/jira/browse/SOLR-6037
> Project: Solr
> Issue Type: Bug
> Components: SearchComponents - other
>Affects Versions: 5.0
>Reporter: Brett Lucey
>Priority: Minor
> Attachments: SOLR-6037.patch
>
>
> When computing statistics using the statistics component, the value for max
> may be reported incorrectly if the maximum value does not lie in the first
> shard.
> This is where the problem is (in StatsValueFactory.java):
> @Override
> protected void updateMinMax(Date min, Date max) {
> if(this.min==null || this.min.after(min)) {
> this.min = min;
> }
> if(this.max==null || this.max.before(min)) {
> this.max = max;
> }
> }
> Attached is a one-line patch which fixes this issue.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (SOLR-6037) Stats Component Reports Incorrect Max in Distributed Data
Brett Lucey created SOLR-6037: - Summary: Stats Component Reports Incorrect Max in Distributed Data Key: SOLR-6037 URL: https://issues.apache.org/jira/browse/SOLR-6037 Project: Solr Issue Type: Bug Components: SearchComponents - other Affects Versions: 5.0 Reporter: Brett Lucey Priority: Minor -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (SOLR-6038) Errors on core start are not properly propagated to Solr clients
Oleg Alexeyev created SOLR-6038: --- Summary: Errors on core start are not properly propagated to Solr clients Key: SOLR-6038 URL: https://issues.apache.org/jira/browse/SOLR-6038 Project: Solr Issue Type: Bug Affects Versions: 4.8 Environment: Standalone Solr - solr.war deployed to Tomcat 7. Reporter: Oleg Alexeyev Priority: Minor If a core cannot start (in our case index was manually corrupted), and client tries to query, Solr throws org.apache.solr.common.SolrException: Unable to create core: , which is not handled by Solr itself and therefore handled by Tomcat - it renders an HTML page with stack trace as the response. SolrJ, in turn, first checks for the content type (HttpSolrServer.executeMethod() - switch (httpStatus):default is skipped because processor != null). And since response is not application/octet-stream as expected for javabin, but text/html instead, throws org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: Expected mime type application/octet-stream but got text/html. Apache Tomcat/7.0.50 - Error report... - printing the whole HTML page into the message. As result, our logs contain unreadable huge exceptions with HTML, Solr server side stack traces and unclear reason that content type doesn't match. While it would perfectly OK to simply print "internal server error" meaning we should check Solr logs. Currently we're working around that as follows: - add error page reference to Tomcat's conf/web.xml: java.lang.Throwable /errors/throwable-javabin.jsp - add throwable-javabin.jsp to webapps/solr/errors/: <%@ page contentType="application/octet-stream" %><% new org.apache.solr.common.util.JavaBinCodec().marshal(new org.apache.solr.common.util.NamedList(), response.getOutputStream()); %> This way, when an exception happens, Solr responds with an empty valid javabin named list. And SolrJ passes content type check and response parsing. Then it properly fails on further http status check (HttpSolrServer.executeMethod() - httpStatus != HttpStatus.SC_OK) and we get a nice message like org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: Internal Server Error request: http://localhost:8983/solr/core/select?q=... -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-6035) CloudSolrServer directUpdate routing should use getCoreUrl
[ https://issues.apache.org/jira/browse/SOLR-6035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joel Bernstein updated SOLR-6035: - Attachment: SOLR-6035.patch New patch using svn diff against trunk. > CloudSolrServer directUpdate routing should use getCoreUrl > --- > > Key: SOLR-6035 > URL: https://issues.apache.org/jira/browse/SOLR-6035 > Project: Solr > Issue Type: Improvement > Components: SolrCloud >Reporter: Marvin Justice >Assignee: Joel Bernstein >Priority: Minor > Labels: performance > Attachments: SOLR-6035.patch, SOLR-6035.patch > > > In a multisharded node environment we were seeing forward-to-leader hops when > using CloudSolrServer directUpdate (with the hop being on the same node) . > Consequently, there was no improvement in indexing performance over > non-directUpdate. > Changing buildUrlMap to use getCoreUrl eliminated the extra hop and now we > see a dramatic improvement in performance. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5635) Better exception testing for indexwriter
[ https://issues.apache.org/jira/browse/LUCENE-5635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13986585#comment-13986585 ] ASF subversion and git services commented on LUCENE-5635: - Commit 1591665 from [~rcmuir] in branch 'dev/trunk' [ https://svn.apache.org/r1591665 ] LUCENE-5635: properly shutdown IW > Better exception testing for indexwriter > > > Key: LUCENE-5635 > URL: https://issues.apache.org/jira/browse/LUCENE-5635 > Project: Lucene - Core > Issue Type: Bug >Reporter: Robert Muir > > Currently we only catch exc-handling bugs depending on where a codec does i/o. > Instead we should add CrankyCodec and CrankyAnalyzer, which throw random > exceptions from any method. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5635) Better exception testing for indexwriter
[ https://issues.apache.org/jira/browse/LUCENE-5635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13986574#comment-13986574 ] ASF subversion and git services commented on LUCENE-5635: - Commit 1591662 from [~rcmuir] in branch 'dev/trunk' [ https://svn.apache.org/r1591662 ] LUCENE-5635: make CrankyCodec crankier > Better exception testing for indexwriter > > > Key: LUCENE-5635 > URL: https://issues.apache.org/jira/browse/LUCENE-5635 > Project: Lucene - Core > Issue Type: Bug >Reporter: Robert Muir > > Currently we only catch exc-handling bugs depending on where a codec does i/o. > Instead we should add CrankyCodec and CrankyAnalyzer, which throw random > exceptions from any method. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5635) Better exception testing for indexwriter
[ https://issues.apache.org/jira/browse/LUCENE-5635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13986572#comment-13986572 ] ASF subversion and git services commented on LUCENE-5635: - Commit 1591661 from [~rcmuir] in branch 'dev/trunk' [ https://svn.apache.org/r1591661 ] LUCENE-5635: make these usable > Better exception testing for indexwriter > > > Key: LUCENE-5635 > URL: https://issues.apache.org/jira/browse/LUCENE-5635 > Project: Lucene - Core > Issue Type: Bug >Reporter: Robert Muir > > Currently we only catch exc-handling bugs depending on where a codec does i/o. > Instead we should add CrankyCodec and CrankyAnalyzer, which throw random > exceptions from any method. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5611) Simplify the default indexing chain
[ https://issues.apache.org/jira/browse/LUCENE-5611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13986568#comment-13986568 ] ASF subversion and git services commented on LUCENE-5611: - Commit 1591660 from [~mikemccand] in branch 'dev/branches/branch_4x' [ https://svn.apache.org/r1591660 ] LUCENE-5611: increment lastStoredDocID in one plase; add Asserting.toString > Simplify the default indexing chain > --- > > Key: LUCENE-5611 > URL: https://issues.apache.org/jira/browse/LUCENE-5611 > Project: Lucene - Core > Issue Type: Improvement > Components: core/index >Reporter: Michael McCandless >Assignee: Michael McCandless > Fix For: 4.9, 5.0 > > Attachments: LUCENE-5611.patch, LUCENE-5611.patch > > > I think Lucene's current indexing chain has too many classes / > hierarchy / abstractions, making it look much more complex than it > really should be, and discouraging users from experimenting/innovating > with their own indexing chains. > Also, if it were easier to understand/approach, then new developers > would more likely try to improve it ... it really should be simpler. > So I'm exploring a pared back indexing chain, and have a starting patch > that I think is looking ok: it seems more approachable than the > current indexing chain, or at least has fewer strange classes. > I also thought this could give some speedup for tiny documents (a more > common use of Lucene lately), and it looks like, with the evil > optimizations, this is a ~25% speedup for Geonames docs. Even without > those evil optos it's a bit faster. > This is very much a work in progress / nocommits, and there are some > behavior changes e.g. the new chain requires all fields to have the > same TV options (rather than auto-upgrading all fields by the same > name that the current chain does)... -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5611) Simplify the default indexing chain
[ https://issues.apache.org/jira/browse/LUCENE-5611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13986566#comment-13986566 ] ASF subversion and git services commented on LUCENE-5611: - Commit 1591657 from [~mikemccand] in branch 'dev/trunk' [ https://svn.apache.org/r1591657 ] LUCENE-5611: always abort if we hit exc in StoredFieldsWriter.start/FinishDocument > Simplify the default indexing chain > --- > > Key: LUCENE-5611 > URL: https://issues.apache.org/jira/browse/LUCENE-5611 > Project: Lucene - Core > Issue Type: Improvement > Components: core/index >Reporter: Michael McCandless >Assignee: Michael McCandless > Fix For: 4.9, 5.0 > > Attachments: LUCENE-5611.patch, LUCENE-5611.patch > > > I think Lucene's current indexing chain has too many classes / > hierarchy / abstractions, making it look much more complex than it > really should be, and discouraging users from experimenting/innovating > with their own indexing chains. > Also, if it were easier to understand/approach, then new developers > would more likely try to improve it ... it really should be simpler. > So I'm exploring a pared back indexing chain, and have a starting patch > that I think is looking ok: it seems more approachable than the > current indexing chain, or at least has fewer strange classes. > I also thought this could give some speedup for tiny documents (a more > common use of Lucene lately), and it looks like, with the evil > optimizations, this is a ~25% speedup for Geonames docs. Even without > those evil optos it's a bit faster. > This is very much a work in progress / nocommits, and there are some > behavior changes e.g. the new chain requires all fields to have the > same TV options (rather than auto-upgrading all fields by the same > name that the current chain does)... -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5635) Better exception testing for indexwriter
[ https://issues.apache.org/jira/browse/LUCENE-5635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13986559#comment-13986559 ] ASF subversion and git services commented on LUCENE-5635: - Commit 1591652 from [~rcmuir] in branch 'dev/trunk' [ https://svn.apache.org/r1591652 ] LUCENE-5635: add a prototype test (currently disabled with AwaitsFix) > Better exception testing for indexwriter > > > Key: LUCENE-5635 > URL: https://issues.apache.org/jira/browse/LUCENE-5635 > Project: Lucene - Core > Issue Type: Bug >Reporter: Robert Muir > > Currently we only catch exc-handling bugs depending on where a codec does i/o. > Instead we should add CrankyCodec and CrankyAnalyzer, which throw random > exceptions from any method. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Solr-4.x-Linux (64bit/jdk1.8.0_05) - Build # 10096 - Still Failing!
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-4.x-Linux/10096/ Java: 64bit/jdk1.8.0_05 -XX:+UseCompressedOops -XX:+UseSerialGC 1 tests failed. REGRESSION: org.apache.lucene.analysis.core.TestRandomChains.testRandomChainsWithLargeStrings Error Message: some thread(s) failed Stack Trace: java.lang.RuntimeException: some thread(s) failed at org.apache.lucene.analysis.BaseTokenStreamTestCase.checkRandomData(BaseTokenStreamTestCase.java:533) at org.apache.lucene.analysis.core.TestRandomChains.testRandomChainsWithLargeStrings(TestRandomChains.java:941) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:483) at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618) at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827) at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863) at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877) at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50) at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55) at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:360) at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:793) at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:453) at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836) at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738) at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772) at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65) at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:360) at java.lang.Thread.run(Thread.java:745) Build Log: [...truncated 5627 lines...] [junit4] Suite: org.apache.lucene.analysis.core.TestRandomChains [junit4] 2> TEST FAIL: useCharFilter=false text='determinazione a cui perverr\u00e0 il Parlamento europeo in merito ai due regolamenti' [junit4] 2> TEST FAIL: useCharFilter=false text='determinazione a cui perverr\u00e0 il Parlamento europeo in merito ai due regolamenti' [junit4] 2> TEST FAIL: useCharFilter=fals
[jira] [Commented] (LUCENE-5635) Better exception testing for indexwriter
[ https://issues.apache.org/jira/browse/LUCENE-5635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13986530#comment-13986530 ] ASF subversion and git services commented on LUCENE-5635: - Commit 1591630 from [~rcmuir] in branch 'dev/branches/branch_4x' [ https://svn.apache.org/r1591630 ] LUCENE-5635: don't try to use CrankyTokenFilter in TestRandomChains > Better exception testing for indexwriter > > > Key: LUCENE-5635 > URL: https://issues.apache.org/jira/browse/LUCENE-5635 > Project: Lucene - Core > Issue Type: Bug >Reporter: Robert Muir > > Currently we only catch exc-handling bugs depending on where a codec does i/o. > Instead we should add CrankyCodec and CrankyAnalyzer, which throw random > exceptions from any method. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5635) Better exception testing for indexwriter
[ https://issues.apache.org/jira/browse/LUCENE-5635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13986529#comment-13986529 ] ASF subversion and git services commented on LUCENE-5635: - Commit 1591629 from [~rcmuir] in branch 'dev/trunk' [ https://svn.apache.org/r1591629 ] LUCENE-5635: don't try to use CrankyTokenFilter in TestRandomChains > Better exception testing for indexwriter > > > Key: LUCENE-5635 > URL: https://issues.apache.org/jira/browse/LUCENE-5635 > Project: Lucene - Core > Issue Type: Bug >Reporter: Robert Muir > > Currently we only catch exc-handling bugs depending on where a codec does i/o. > Instead we should add CrankyCodec and CrankyAnalyzer, which throw random > exceptions from any method. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: [JENKINS] Lucene-trunk-Linux-java7-64-analyzers - Build # 4120 - Failure!
Ooops: I committed a fix On Thu, May 1, 2014 at 7:59 AM, wrote: > Build: builds.flonkings.com/job/Lucene-trunk-Linux-java7-64-analyzers/4120/ > > 1 tests failed. > REGRESSION: > org.apache.lucene.analysis.core.TestRandomChains.testRandomChainsWithLargeStrings > > Error Message: > TokenStream implementation classes or at least their incrementToken() > implementation must be final > > Stack Trace: > java.lang.AssertionError: TokenStream implementation classes or at least > their incrementToken() implementation must be final > at > __randomizedtesting.SeedInfo.seed([CA0D432C2AA2AE3:66FB6B239BE40A10]:0) > at > org.apache.lucene.analysis.TokenStream.assertFinal(TokenStream.java:120) > at org.apache.lucene.analysis.TokenStream.(TokenStream.java:101) > at org.apache.lucene.analysis.TokenFilter.(TokenFilter.java:33) > at > org.apache.lucene.analysis.CrankyTokenFilter.(CrankyTokenFilter.java:35) > at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native > Method) > at > sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57) > at > sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) > at java.lang.reflect.Constructor.newInstance(Constructor.java:526) > at > org.apache.lucene.analysis.core.TestRandomChains$MockRandomAnalyzer.createComponent(TestRandomChains.java:681) > at > org.apache.lucene.analysis.core.TestRandomChains$MockRandomAnalyzer.newFilterChain(TestRandomChains.java:793) > at > org.apache.lucene.analysis.core.TestRandomChains$MockRandomAnalyzer.toString(TestRandomChains.java:670) > at java.lang.String.valueOf(String.java:2854) > at java.lang.StringBuilder.append(StringBuilder.java:128) > at > org.apache.lucene.analysis.core.TestRandomChains.testRandomChainsWithLargeStrings(TestRandomChains.java:923) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at > com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618) > at > com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827) > at > com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863) > at > com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877) > at > org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50) > at > org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51) > at > org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) > at > com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55) > at > org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49) > at > org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65) > at > org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) > at > com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) > at > com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:360) > at > com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:793) > at > com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:453) > at > com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836) > at > com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738) > at > com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772) > at > com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783) > at > org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) > at > org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42) > at > com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55) > at > com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) > at > com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShad
[JENKINS] Lucene-trunk-Linux-java7-64-analyzers - Build # 4120 - Failure!
Build: builds.flonkings.com/job/Lucene-trunk-Linux-java7-64-analyzers/4120/ 1 tests failed. REGRESSION: org.apache.lucene.analysis.core.TestRandomChains.testRandomChainsWithLargeStrings Error Message: TokenStream implementation classes or at least their incrementToken() implementation must be final Stack Trace: java.lang.AssertionError: TokenStream implementation classes or at least their incrementToken() implementation must be final at __randomizedtesting.SeedInfo.seed([CA0D432C2AA2AE3:66FB6B239BE40A10]:0) at org.apache.lucene.analysis.TokenStream.assertFinal(TokenStream.java:120) at org.apache.lucene.analysis.TokenStream.(TokenStream.java:101) at org.apache.lucene.analysis.TokenFilter.(TokenFilter.java:33) at org.apache.lucene.analysis.CrankyTokenFilter.(CrankyTokenFilter.java:35) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:526) at org.apache.lucene.analysis.core.TestRandomChains$MockRandomAnalyzer.createComponent(TestRandomChains.java:681) at org.apache.lucene.analysis.core.TestRandomChains$MockRandomAnalyzer.newFilterChain(TestRandomChains.java:793) at org.apache.lucene.analysis.core.TestRandomChains$MockRandomAnalyzer.toString(TestRandomChains.java:670) at java.lang.String.valueOf(String.java:2854) at java.lang.StringBuilder.append(StringBuilder.java:128) at org.apache.lucene.analysis.core.TestRandomChains.testRandomChainsWithLargeStrings(TestRandomChains.java:923) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618) at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827) at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863) at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877) at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50) at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55) at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:360) at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:793) at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:453) at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836) at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738) at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772) at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[JENKINS] Lucene-Solr-trunk-Linux (64bit/jdk1.7.0_60-ea-b14) - Build # 10206 - Failure!
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/10206/ Java: 64bit/jdk1.7.0_60-ea-b14 -XX:-UseCompressedOops -XX:+UseConcMarkSweepGC 3 tests failed. REGRESSION: org.apache.lucene.analysis.core.TestAllAnalyzersHaveFactories.test Error Message: A SPI class of type org.apache.lucene.analysis.util.TokenFilterFactory with name 'Cranky' does not exist. You need to add the corresponding JAR file supporting this SPI to your classpath. The current classpath supports the following names: [apostrophe, arabicnormalization, arabicstem, bulgarianstem, brazilianstem, cjkbigram, cjkwidth, soraninormalization, soranistem, commongrams, commongramsquery, dictionarycompoundword, hyphenationcompoundword, lowercase, stop, type, uppercase, czechstem, germanlightstem, germanminimalstem, germannormalization, germanstem, greeklowercase, greekstem, englishminimalstem, englishpossessive, kstem, porterstem, spanishlightstem, persiannormalization, finnishlightstem, frenchlightstem, frenchminimalstem, irishlowercase, galicianminimalstem, galicianstem, hindinormalization, hindistem, hungarianlightstem, hunspellstem, indonesianstem, indicnormalization, italianlightstem, latvianstem, asciifolding, capitalization, codepointcount, hyphenatedwords, keepword, keywordmarker, keywordrepeat, length, limittokencount, limittokenposition, removeduplicates, stemmeroverride, trim, truncate, worddelimiter, scandinavianfolding, scandinaviannormalization, edgengram, ngram, norwegianlightstem, norwegianminimalstem, patternreplace, patterncapturegroup, delimitedpayload, numericpayload, tokenoffsetpayload, typeaspayload, portugueselightstem, portugueseminimalstem, portuguesestem, reversestring, russianlightstem, shingle, snowballporter, classic, standard, swedishlightstem, synonym, thaiword, turkishlowercase, elision] Stack Trace: java.lang.IllegalArgumentException: A SPI class of type org.apache.lucene.analysis.util.TokenFilterFactory with name 'Cranky' does not exist. You need to add the corresponding JAR file supporting this SPI to your classpath. The current classpath supports the following names: [apostrophe, arabicnormalization, arabicstem, bulgarianstem, brazilianstem, cjkbigram, cjkwidth, soraninormalization, soranistem, commongrams, commongramsquery, dictionarycompoundword, hyphenationcompoundword, lowercase, stop, type, uppercase, czechstem, germanlightstem, germanminimalstem, germannormalization, germanstem, greeklowercase, greekstem, englishminimalstem, englishpossessive, kstem, porterstem, spanishlightstem, persiannormalization, finnishlightstem, frenchlightstem, frenchminimalstem, irishlowercase, galicianminimalstem, galicianstem, hindinormalization, hindistem, hungarianlightstem, hunspellstem, indonesianstem, indicnormalization, italianlightstem, latvianstem, asciifolding, capitalization, codepointcount, hyphenatedwords, keepword, keywordmarker, keywordrepeat, length, limittokencount, limittokenposition, removeduplicates, stemmeroverride, trim, truncate, worddelimiter, scandinavianfolding, scandinaviannormalization, edgengram, ngram, norwegianlightstem, norwegianminimalstem, patternreplace, patterncapturegroup, delimitedpayload, numericpayload, tokenoffsetpayload, typeaspayload, portugueselightstem, portugueseminimalstem, portuguesestem, reversestring, russianlightstem, shingle, snowballporter, classic, standard, swedishlightstem, synonym, thaiword, turkishlowercase, elision] at __randomizedtesting.SeedInfo.seed([358F86B7E9419656:BDDBB96D47BDFBAE]:0) at org.apache.lucene.analysis.util.AnalysisSPILoader.lookupClass(AnalysisSPILoader.java:122) at org.apache.lucene.analysis.util.TokenFilterFactory.lookupClass(TokenFilterFactory.java:42) at org.apache.lucene.analysis.core.TestAllAnalyzersHaveFactories.test(TestAllAnalyzersHaveFactories.java:148) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618) at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827) at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863) at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877) at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50) at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$
[jira] [Commented] (LUCENE-5611) Simplify the default indexing chain
[ https://issues.apache.org/jira/browse/LUCENE-5611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13986517#comment-13986517 ] ASF subversion and git services commented on LUCENE-5611: - Commit 1591616 from [~mikemccand] in branch 'dev/branches/branch_4x' [ https://svn.apache.org/r1591616 ] LUCENE-5611: always abort if we hit exc in StoredFieldsWriter.start/FinishDocument > Simplify the default indexing chain > --- > > Key: LUCENE-5611 > URL: https://issues.apache.org/jira/browse/LUCENE-5611 > Project: Lucene - Core > Issue Type: Improvement > Components: core/index >Reporter: Michael McCandless >Assignee: Michael McCandless > Fix For: 4.9, 5.0 > > Attachments: LUCENE-5611.patch, LUCENE-5611.patch > > > I think Lucene's current indexing chain has too many classes / > hierarchy / abstractions, making it look much more complex than it > really should be, and discouraging users from experimenting/innovating > with their own indexing chains. > Also, if it were easier to understand/approach, then new developers > would more likely try to improve it ... it really should be simpler. > So I'm exploring a pared back indexing chain, and have a starting patch > that I think is looking ok: it seems more approachable than the > current indexing chain, or at least has fewer strange classes. > I also thought this could give some speedup for tiny documents (a more > common use of Lucene lately), and it looks like, with the evil > optimizations, this is a ~25% speedup for Geonames docs. Even without > those evil optos it's a bit faster. > This is very much a work in progress / nocommits, and there are some > behavior changes e.g. the new chain requires all fields to have the > same TV options (rather than auto-upgrading all fields by the same > name that the current chain does)... -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5635) Better exception testing for indexwriter
[ https://issues.apache.org/jira/browse/LUCENE-5635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13986515#comment-13986515 ] ASF subversion and git services commented on LUCENE-5635: - Commit 1591613 from [~rcmuir] in branch 'dev/branches/branch_4x' [ https://svn.apache.org/r1591613 ] LUCENE-5635: no spi for crankytokenfilter > Better exception testing for indexwriter > > > Key: LUCENE-5635 > URL: https://issues.apache.org/jira/browse/LUCENE-5635 > Project: Lucene - Core > Issue Type: Bug >Reporter: Robert Muir > > Currently we only catch exc-handling bugs depending on where a codec does i/o. > Instead we should add CrankyCodec and CrankyAnalyzer, which throw random > exceptions from any method. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5635) Better exception testing for indexwriter
[ https://issues.apache.org/jira/browse/LUCENE-5635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13986514#comment-13986514 ] ASF subversion and git services commented on LUCENE-5635: - Commit 1591611 from [~rcmuir] in branch 'dev/trunk' [ https://svn.apache.org/r1591611 ] LUCENE-5635: no spi for crankytokenfilter > Better exception testing for indexwriter > > > Key: LUCENE-5635 > URL: https://issues.apache.org/jira/browse/LUCENE-5635 > Project: Lucene - Core > Issue Type: Bug >Reporter: Robert Muir > > Currently we only catch exc-handling bugs depending on where a codec does i/o. > Instead we should add CrankyCodec and CrankyAnalyzer, which throw random > exceptions from any method. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: [JENKINS] Lucene-trunk-Linux-java7-64-analyzers - Build # 4118 - Failure!
oops: i committed a fix On Thu, May 1, 2014 at 7:26 AM, wrote: > Build: builds.flonkings.com/job/Lucene-trunk-Linux-java7-64-analyzers/4118/ > > 2 tests failed. > REGRESSION: > org.apache.lucene.analysis.core.TestAllAnalyzersHaveFactories.test > > Error Message: > A SPI class of type org.apache.lucene.analysis.util.TokenFilterFactory with > name 'Cranky' does not exist. You need to add the corresponding JAR file > supporting this SPI to your classpath. The current classpath supports the > following names: [apostrophe, arabicnormalization, arabicstem, bulgarianstem, > brazilianstem, cjkbigram, cjkwidth, soraninormalization, soranistem, > commongrams, commongramsquery, dictionarycompoundword, > hyphenationcompoundword, lowercase, stop, type, uppercase, czechstem, > germanlightstem, germanminimalstem, germannormalization, germanstem, > greeklowercase, greekstem, englishminimalstem, englishpossessive, kstem, > porterstem, spanishlightstem, persiannormalization, finnishlightstem, > frenchlightstem, frenchminimalstem, irishlowercase, galicianminimalstem, > galicianstem, hindinormalization, hindistem, hungarianlightstem, > hunspellstem, indonesianstem, indicnormalization, italianlightstem, > latvianstem, asciifolding, capitalization, codepointcount, hyphenatedwords, > keepword, keywordmarker, keywordrepeat, length, limittokencount, > limittokenposition, removeduplicates, stemmeroverride, trim, truncate, > worddelimiter, scandinavianfolding, scandinaviannormalization, edgengram, > ngram, norwegianlightstem, norwegianminimalstem, patternreplace, > patterncapturegroup, delimitedpayload, numericpayload, tokenoffsetpayload, > typeaspayload, portugueselightstem, portugueseminimalstem, portuguesestem, > reversestring, russianlightstem, shingle, snowballporter, classic, standard, > swedishlightstem, synonym, thaiword, turkishlowercase, elision] > > Stack Trace: > java.lang.IllegalArgumentException: A SPI class of type > org.apache.lucene.analysis.util.TokenFilterFactory with name 'Cranky' does > not exist. You need to add the corresponding JAR file supporting this SPI to > your classpath. The current classpath supports the following names: > [apostrophe, arabicnormalization, arabicstem, bulgarianstem, brazilianstem, > cjkbigram, cjkwidth, soraninormalization, soranistem, commongrams, > commongramsquery, dictionarycompoundword, hyphenationcompoundword, lowercase, > stop, type, uppercase, czechstem, germanlightstem, germanminimalstem, > germannormalization, germanstem, greeklowercase, greekstem, > englishminimalstem, englishpossessive, kstem, porterstem, spanishlightstem, > persiannormalization, finnishlightstem, frenchlightstem, frenchminimalstem, > irishlowercase, galicianminimalstem, galicianstem, hindinormalization, > hindistem, hungarianlightstem, hunspellstem, indonesianstem, > indicnormalization, italianlightstem, latvianstem, asciifolding, > capitalization, codepointcount, hyphenatedwords, keepword, keywordmarker, > keywordrepeat, length, limittokencount, limittokenposition, removeduplicates, > stemmeroverride, trim, truncate, worddelimiter, scandinavianfolding, > scandinaviannormalization, edgengram, ngram, norwegianlightstem, > norwegianminimalstem, patternreplace, patterncapturegroup, delimitedpayload, > numericpayload, tokenoffsetpayload, typeaspayload, portugueselightstem, > portugueseminimalstem, portuguesestem, reversestring, russianlightstem, > shingle, snowballporter, classic, standard, swedishlightstem, synonym, > thaiword, turkishlowercase, elision] > at > __randomizedtesting.SeedInfo.seed([15B8EB55456878E0:9DECD48FEB941518]:0) > at > org.apache.lucene.analysis.util.AnalysisSPILoader.lookupClass(AnalysisSPILoader.java:122) > at > org.apache.lucene.analysis.util.TokenFilterFactory.lookupClass(TokenFilterFactory.java:42) > at > org.apache.lucene.analysis.core.TestAllAnalyzersHaveFactories.test(TestAllAnalyzersHaveFactories.java:148) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at > com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618) > at > com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827) > at > com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863) > at > com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877) > at > org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50) > at > org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51) > at > org.apache.lucen
[JENKINS] Lucene-trunk-Linux-java7-64-analyzers - Build # 4118 - Failure!
Build: builds.flonkings.com/job/Lucene-trunk-Linux-java7-64-analyzers/4118/ 2 tests failed. REGRESSION: org.apache.lucene.analysis.core.TestAllAnalyzersHaveFactories.test Error Message: A SPI class of type org.apache.lucene.analysis.util.TokenFilterFactory with name 'Cranky' does not exist. You need to add the corresponding JAR file supporting this SPI to your classpath. The current classpath supports the following names: [apostrophe, arabicnormalization, arabicstem, bulgarianstem, brazilianstem, cjkbigram, cjkwidth, soraninormalization, soranistem, commongrams, commongramsquery, dictionarycompoundword, hyphenationcompoundword, lowercase, stop, type, uppercase, czechstem, germanlightstem, germanminimalstem, germannormalization, germanstem, greeklowercase, greekstem, englishminimalstem, englishpossessive, kstem, porterstem, spanishlightstem, persiannormalization, finnishlightstem, frenchlightstem, frenchminimalstem, irishlowercase, galicianminimalstem, galicianstem, hindinormalization, hindistem, hungarianlightstem, hunspellstem, indonesianstem, indicnormalization, italianlightstem, latvianstem, asciifolding, capitalization, codepointcount, hyphenatedwords, keepword, keywordmarker, keywordrepeat, length, limittokencount, limittokenposition, removeduplicates, stemmeroverride, trim, truncate, worddelimiter, scandinavianfolding, scandinaviannormalization, edgengram, ngram, norwegianlightstem, norwegianminimalstem, patternreplace, patterncapturegroup, delimitedpayload, numericpayload, tokenoffsetpayload, typeaspayload, portugueselightstem, portugueseminimalstem, portuguesestem, reversestring, russianlightstem, shingle, snowballporter, classic, standard, swedishlightstem, synonym, thaiword, turkishlowercase, elision] Stack Trace: java.lang.IllegalArgumentException: A SPI class of type org.apache.lucene.analysis.util.TokenFilterFactory with name 'Cranky' does not exist. You need to add the corresponding JAR file supporting this SPI to your classpath. The current classpath supports the following names: [apostrophe, arabicnormalization, arabicstem, bulgarianstem, brazilianstem, cjkbigram, cjkwidth, soraninormalization, soranistem, commongrams, commongramsquery, dictionarycompoundword, hyphenationcompoundword, lowercase, stop, type, uppercase, czechstem, germanlightstem, germanminimalstem, germannormalization, germanstem, greeklowercase, greekstem, englishminimalstem, englishpossessive, kstem, porterstem, spanishlightstem, persiannormalization, finnishlightstem, frenchlightstem, frenchminimalstem, irishlowercase, galicianminimalstem, galicianstem, hindinormalization, hindistem, hungarianlightstem, hunspellstem, indonesianstem, indicnormalization, italianlightstem, latvianstem, asciifolding, capitalization, codepointcount, hyphenatedwords, keepword, keywordmarker, keywordrepeat, length, limittokencount, limittokenposition, removeduplicates, stemmeroverride, trim, truncate, worddelimiter, scandinavianfolding, scandinaviannormalization, edgengram, ngram, norwegianlightstem, norwegianminimalstem, patternreplace, patterncapturegroup, delimitedpayload, numericpayload, tokenoffsetpayload, typeaspayload, portugueselightstem, portugueseminimalstem, portuguesestem, reversestring, russianlightstem, shingle, snowballporter, classic, standard, swedishlightstem, synonym, thaiword, turkishlowercase, elision] at __randomizedtesting.SeedInfo.seed([15B8EB55456878E0:9DECD48FEB941518]:0) at org.apache.lucene.analysis.util.AnalysisSPILoader.lookupClass(AnalysisSPILoader.java:122) at org.apache.lucene.analysis.util.TokenFilterFactory.lookupClass(TokenFilterFactory.java:42) at org.apache.lucene.analysis.core.TestAllAnalyzersHaveFactories.test(TestAllAnalyzersHaveFactories.java:148) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618) at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827) at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863) at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877) at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50) at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55) at org.apach
[jira] [Commented] (LUCENE-5635) Better exception testing for indexwriter
[ https://issues.apache.org/jira/browse/LUCENE-5635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13986506#comment-13986506 ] ASF subversion and git services commented on LUCENE-5635: - Commit 1591606 from [~rcmuir] in branch 'dev/branches/branch_4x' [ https://svn.apache.org/r1591606 ] LUCENE-5635: add CrankyTokenFilter > Better exception testing for indexwriter > > > Key: LUCENE-5635 > URL: https://issues.apache.org/jira/browse/LUCENE-5635 > Project: Lucene - Core > Issue Type: Bug >Reporter: Robert Muir > > Currently we only catch exc-handling bugs depending on where a codec does i/o. > Instead we should add CrankyCodec and CrankyAnalyzer, which throw random > exceptions from any method. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5635) Better exception testing for indexwriter
[ https://issues.apache.org/jira/browse/LUCENE-5635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13986505#comment-13986505 ] ASF subversion and git services commented on LUCENE-5635: - Commit 1591605 from [~rcmuir] in branch 'dev/trunk' [ https://svn.apache.org/r1591605 ] LUCENE-5635: add CrankyTokenFilter > Better exception testing for indexwriter > > > Key: LUCENE-5635 > URL: https://issues.apache.org/jira/browse/LUCENE-5635 > Project: Lucene - Core > Issue Type: Bug >Reporter: Robert Muir > > Currently we only catch exc-handling bugs depending on where a codec does i/o. > Instead we should add CrankyCodec and CrankyAnalyzer, which throw random > exceptions from any method. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: [JENKINS] Lucene-4x-Linux-Java7-64-test-only - Build # 20243 - Failure!
I think the issue is this particular codec (4.0 uncompressed stored fields writer) might do i/o in places like startDocument() / finishDocument() where the others dont. I don't think testing exception handling this way is very effective, we should test the api instead. I opened an issue: https://issues.apache.org/jira/browse/LUCENE-5635 On Thu, May 1, 2014 at 5:24 AM, Robert Muir wrote: > I did the refactoring (4.x: im porting forward to trunk now). > > Thing is it still doesnt fix the bug. but i think its easier to work with. > > On Wed, Apr 30, 2014 at 4:44 PM, Robert Muir wrote: >> The exception handling in processDocument() is too complex. it must do >> try+finally with two separate cases: aborting exception and >> non-aborting exception, depending on where it happens. >> >> I think since the whole thing loops through fields, if we factor out >> processField it will be easier... >> >> On Wed, Apr 30, 2014 at 4:21 PM, Robert Muir wrote: >>> reproduces if you tack on -Dtests.dups=100 ... looking >>> >>> On Wed, Apr 30, 2014 at 4:04 PM, wrote: Build: builds.flonkings.com/job/Lucene-4x-Linux-Java7-64-test-only/20243/ 1 tests failed. REGRESSION: org.apache.lucene.index.TestIndexWriterWithThreads.testImmediateDiskFullWithThreads Error Message: fdx size mismatch: docCount is 12 but fdx file size is 133 file=MockIndexOutputWrapper(org.apache.lucene.store.RAMOutputStream@3afabf87); now aborting this merge to prevent index corruption Stack Trace: java.lang.RuntimeException: fdx size mismatch: docCount is 12 but fdx file size is 133 file=MockIndexOutputWrapper(org.apache.lucene.store.RAMOutputStream@3afabf87); now aborting this merge to prevent index corruption at __randomizedtesting.SeedInfo.seed([70486912DEA23EAA:FF309021F56884D9]:0) at org.apache.lucene.codecs.lucene40.Lucene40StoredFieldsWriter.finish(Lucene40StoredFieldsWriter.java:233) at org.apache.lucene.index.DefaultIndexingChain.flush(DefaultIndexingChain.java:96) at org.apache.lucene.index.DocumentsWriterPerThread.flush(DocumentsWriterPerThread.java:415) at org.apache.lucene.index.DocumentsWriter.doFlush(DocumentsWriter.java:512) at org.apache.lucene.index.DocumentsWriter.flushAllThreads(DocumentsWriter.java:622) at org.apache.lucene.index.IndexWriter.doFlush(IndexWriter.java:3203) at org.apache.lucene.index.IndexWriter.flush(IndexWriter.java:3179) at org.apache.lucene.index.IndexWriter.closeInternal(IndexWriter.java:995) at org.apache.lucene.index.IndexWriter.close(IndexWriter.java:939) at org.apache.lucene.index.TestIndexWriterWithThreads.testImmediateDiskFullWithThreads(TestIndexWriterWithThreads.java:168) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618) at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827) at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863) at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877) at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50) at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55) at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:360) at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:793) at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakCont
[jira] [Created] (LUCENE-5635) Better exception testing for indexwriter
Robert Muir created LUCENE-5635: --- Summary: Better exception testing for indexwriter Key: LUCENE-5635 URL: https://issues.apache.org/jira/browse/LUCENE-5635 Project: Lucene - Core Issue Type: Bug Reporter: Robert Muir Currently we only catch exc-handling bugs depending on where a codec does i/o. Instead we should add CrankyCodec and CrankyAnalyzer, which throw random exceptions from any method. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5611) Simplify the default indexing chain
[ https://issues.apache.org/jira/browse/LUCENE-5611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13986476#comment-13986476 ] ASF subversion and git services commented on LUCENE-5611: - Commit 1591588 from [~rcmuir] in branch 'dev/branches/branch_4x' [ https://svn.apache.org/r1591588 ] LUCENE-5611: test that dv exceptions are non-aborting > Simplify the default indexing chain > --- > > Key: LUCENE-5611 > URL: https://issues.apache.org/jira/browse/LUCENE-5611 > Project: Lucene - Core > Issue Type: Improvement > Components: core/index >Reporter: Michael McCandless >Assignee: Michael McCandless > Fix For: 4.9, 5.0 > > Attachments: LUCENE-5611.patch, LUCENE-5611.patch > > > I think Lucene's current indexing chain has too many classes / > hierarchy / abstractions, making it look much more complex than it > really should be, and discouraging users from experimenting/innovating > with their own indexing chains. > Also, if it were easier to understand/approach, then new developers > would more likely try to improve it ... it really should be simpler. > So I'm exploring a pared back indexing chain, and have a starting patch > that I think is looking ok: it seems more approachable than the > current indexing chain, or at least has fewer strange classes. > I also thought this could give some speedup for tiny documents (a more > common use of Lucene lately), and it looks like, with the evil > optimizations, this is a ~25% speedup for Geonames docs. Even without > those evil optos it's a bit faster. > This is very much a work in progress / nocommits, and there are some > behavior changes e.g. the new chain requires all fields to have the > same TV options (rather than auto-upgrading all fields by the same > name that the current chain does)... -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
RE: [JENKINS] Lucene-Solr-trunk-MacOSX (64bit/jdk1.8.0) - Build # 1541 - Failure!
Hi Robert, I found the message on hotspot mailing list: http://mail.openjdk.java.net/pipermail/hotspot-dev/2013-May/009496.html The problem is System.nanoTime() falls back to gettimeofday, because some fast and monotonic POSIX API is not available on OSX, which is horrible on the mac, because it can go backwards. > Here's draft patch that uses mach_absolute_time() to provide fast and > monotonic System.nanoTime() for OS X: > https://gist.github.com/denofevil/db57fa547510b2dd87fd For now we should just ignore those failures on Jenkins, its related to this brokenness and the fact that we are running OSX in a VirtualBOX, which makes its clock very non-monotonic...! Uwe - Uwe Schindler H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de eMail: u...@thetaphi.de > -Original Message- > From: Uwe Schindler [mailto:u...@thetaphi.de] > Sent: Thursday, May 01, 2014 12:01 PM > To: dev@lucene.apache.org > Subject: RE: [JENKINS] Lucene-Solr-trunk-MacOSX (64bit/jdk1.8.0) - Build # > 1541 - Failure! > > See also: http://mail-archives.apache.org/mod_mbox/lucene- > dev/201301.mbox/%3C030701cdea9b$f2334850$d699d8f0$@thetaphi.de%3E > > - > Uwe Schindler > H.-H.-Meier-Allee 63, D-28213 Bremen > http://www.thetaphi.de > eMail: u...@thetaphi.de > > > > -Original Message- > > From: Uwe Schindler [mailto:u...@thetaphi.de] > > Sent: Thursday, May 01, 2014 12:00 PM > > To: dev@lucene.apache.org > > Subject: RE: [JENKINS] Lucene-Solr-trunk-MacOSX (64bit/jdk1.8.0) - > > Build # > > 1541 - Failure! > > > > Uwe, > > > > This happens from time to time. It looks like wrong usage of syscalls > > in the MacOSX libc, because it seems to happen if the clock on Mac > > goes backwards. This happens more often in virtual machines, because > > time synchronization of virtual machine and host clock happens more > > often because of virtual processor scheduling issues, so clocks differ > > more often and need to be corrected by NTP. As far as I remember, > > there was already a patch on hotspot mailing list to use more modern APIs. > > > > Uwe > > > > - > > Uwe Schindler > > H.-H.-Meier-Allee 63, D-28213 Bremen > > http://www.thetaphi.de > > eMail: u...@thetaphi.de > > > > > > > -Original Message- > > > From: Robert Muir [mailto:rcm...@gmail.com] > > > Sent: Thursday, May 01, 2014 10:52 AM > > > To: dev@lucene.apache.org > > > Subject: Re: [JENKINS] Lucene-Solr-trunk-MacOSX (64bit/jdk1.8.0) - > > > Build # > > > 1541 - Failure! > > > > > > I downloaded this jvm on my mac and tried the master seed a few > > > times, no SIGFPE. > > > > > > I can't reproduce it. > > > > > > On Thu, May 1, 2014 at 2:54 AM, Policeman Jenkins Server > > > wrote: > > > > Build: > > > > http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-MacOSX/1541/ > > > > Java: 64bit/jdk1.8.0 -XX:+UseCompressedOops -XX:+UseParallelGC > > > > > > > > No tests ran. > > > > > > > > Build Log: > > > > [...truncated 8349 lines...] > > > >[junit4] JVM J0: stdout was not empty, see: > > > /Users/jenkins/workspace/Lucene-Solr-trunk- > > > MacOSX/lucene/build/join/test/temp/junit4-J0- > > > 20140501_065329_348.sysout > > > >[junit4] >>> JVM J0: stdout (verbatim) > > > >[junit4] # > > > >[junit4] # A fatal error has been detected by the Java Runtime > > > Environment: > > > >[junit4] # > > > >[junit4] # SIGFPE (0x8) at pc=0x7fff8a0d6b63, pid=203, tid=21251 > > > >[junit4] # > > > >[junit4] # JRE version: Java(TM) SE Runtime Environment > > > > (8.0_05-b13) > > > (build 1.8.0_05-b13) > > > >[junit4] # Java VM: Java HotSpot(TM) 64-Bit Server VM (25.5-b02 > > > > mixed > > > mode bsd-amd64 compressed oops) > > > >[junit4] # Problematic frame: > > > >[junit4] # C [libsystem_c.dylib+0x2b63] > > > __commpage_gettimeofday+0x43 > > > >[junit4] # > > > >[junit4] # Failed to write core dump. Core dumps have been > > > > disabled. To > > > enable core dumping, try "ulimit -c unlimited" before starting Java > > > again > > > >[junit4] # > > > >[junit4] # An error report file with more information is saved as: > > > >[junit4] # /Users/jenkins/workspace/Lucene-Solr-trunk- > > > MacOSX/lucene/build/join/test/J0/hs_err_pid203.log > > > >[junit4] # > > > >[junit4] # Compiler replay data is saved as: > > > >[junit4] # /Users/jenkins/workspace/Lucene-Solr-trunk- > > > MacOSX/lucene/build/join/test/J0/replay_pid203.log > > > >[junit4] # > > > >[junit4] # If you would like to submit a bug report, please visit: > > > >[junit4] # http://bugreport.sun.com/bugreport/crash.jsp > > > >[junit4] # > > > >[junit4] <<< JVM J0: EOF > > > > > > > > [...truncated 1 lines...] > > > >[junit4] ERROR: JVM J0 ended with an exception, command line: > > > /Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/jre/ > > > bi n/ java -XX:+UseCompressedOops -XX:+UseParallelGC - > > > XX:+HeapDumpOnOutOfMemoryError - > > > XX:HeapDumpPath=/Users/jenkins/works
[jira] [Commented] (LUCENE-5611) Simplify the default indexing chain
[ https://issues.apache.org/jira/browse/LUCENE-5611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13986472#comment-13986472 ] ASF subversion and git services commented on LUCENE-5611: - Commit 1591587 from [~rcmuir] in branch 'dev/trunk' [ https://svn.apache.org/r1591587 ] LUCENE-5611: fix dv exceptions to be non-aborting > Simplify the default indexing chain > --- > > Key: LUCENE-5611 > URL: https://issues.apache.org/jira/browse/LUCENE-5611 > Project: Lucene - Core > Issue Type: Improvement > Components: core/index >Reporter: Michael McCandless >Assignee: Michael McCandless > Fix For: 4.9, 5.0 > > Attachments: LUCENE-5611.patch, LUCENE-5611.patch > > > I think Lucene's current indexing chain has too many classes / > hierarchy / abstractions, making it look much more complex than it > really should be, and discouraging users from experimenting/innovating > with their own indexing chains. > Also, if it were easier to understand/approach, then new developers > would more likely try to improve it ... it really should be simpler. > So I'm exploring a pared back indexing chain, and have a starting patch > that I think is looking ok: it seems more approachable than the > current indexing chain, or at least has fewer strange classes. > I also thought this could give some speedup for tiny documents (a more > common use of Lucene lately), and it looks like, with the evil > optimizations, this is a ~25% speedup for Geonames docs. Even without > those evil optos it's a bit faster. > This is very much a work in progress / nocommits, and there are some > behavior changes e.g. the new chain requires all fields to have the > same TV options (rather than auto-upgrading all fields by the same > name that the current chain does)... -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
RE: [JENKINS] Lucene-Solr-trunk-MacOSX (64bit/jdk1.8.0) - Build # 1541 - Failure!
See also: http://mail-archives.apache.org/mod_mbox/lucene-dev/201301.mbox/%3C030701cdea9b$f2334850$d699d8f0$@thetaphi.de%3E - Uwe Schindler H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de eMail: u...@thetaphi.de > -Original Message- > From: Uwe Schindler [mailto:u...@thetaphi.de] > Sent: Thursday, May 01, 2014 12:00 PM > To: dev@lucene.apache.org > Subject: RE: [JENKINS] Lucene-Solr-trunk-MacOSX (64bit/jdk1.8.0) - Build # > 1541 - Failure! > > Uwe, > > This happens from time to time. It looks like wrong usage of syscalls in the > MacOSX libc, because it seems to happen if the clock on Mac goes > backwards. This happens more often in virtual machines, because time > synchronization of virtual machine and host clock happens more often > because of virtual processor scheduling issues, so clocks differ more often > and need to be corrected by NTP. As far as I remember, there was already a > patch on hotspot mailing list to use more modern APIs. > > Uwe > > - > Uwe Schindler > H.-H.-Meier-Allee 63, D-28213 Bremen > http://www.thetaphi.de > eMail: u...@thetaphi.de > > > > -Original Message- > > From: Robert Muir [mailto:rcm...@gmail.com] > > Sent: Thursday, May 01, 2014 10:52 AM > > To: dev@lucene.apache.org > > Subject: Re: [JENKINS] Lucene-Solr-trunk-MacOSX (64bit/jdk1.8.0) - > > Build # > > 1541 - Failure! > > > > I downloaded this jvm on my mac and tried the master seed a few times, > > no SIGFPE. > > > > I can't reproduce it. > > > > On Thu, May 1, 2014 at 2:54 AM, Policeman Jenkins Server > > wrote: > > > Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-MacOSX/1541/ > > > Java: 64bit/jdk1.8.0 -XX:+UseCompressedOops -XX:+UseParallelGC > > > > > > No tests ran. > > > > > > Build Log: > > > [...truncated 8349 lines...] > > >[junit4] JVM J0: stdout was not empty, see: > > /Users/jenkins/workspace/Lucene-Solr-trunk- > > MacOSX/lucene/build/join/test/temp/junit4-J0- > > 20140501_065329_348.sysout > > >[junit4] >>> JVM J0: stdout (verbatim) > > >[junit4] # > > >[junit4] # A fatal error has been detected by the Java Runtime > > Environment: > > >[junit4] # > > >[junit4] # SIGFPE (0x8) at pc=0x7fff8a0d6b63, pid=203, tid=21251 > > >[junit4] # > > >[junit4] # JRE version: Java(TM) SE Runtime Environment > > > (8.0_05-b13) > > (build 1.8.0_05-b13) > > >[junit4] # Java VM: Java HotSpot(TM) 64-Bit Server VM (25.5-b02 > > > mixed > > mode bsd-amd64 compressed oops) > > >[junit4] # Problematic frame: > > >[junit4] # C [libsystem_c.dylib+0x2b63] > > __commpage_gettimeofday+0x43 > > >[junit4] # > > >[junit4] # Failed to write core dump. Core dumps have been > > > disabled. To > > enable core dumping, try "ulimit -c unlimited" before starting Java > > again > > >[junit4] # > > >[junit4] # An error report file with more information is saved as: > > >[junit4] # /Users/jenkins/workspace/Lucene-Solr-trunk- > > MacOSX/lucene/build/join/test/J0/hs_err_pid203.log > > >[junit4] # > > >[junit4] # Compiler replay data is saved as: > > >[junit4] # /Users/jenkins/workspace/Lucene-Solr-trunk- > > MacOSX/lucene/build/join/test/J0/replay_pid203.log > > >[junit4] # > > >[junit4] # If you would like to submit a bug report, please visit: > > >[junit4] # http://bugreport.sun.com/bugreport/crash.jsp > > >[junit4] # > > >[junit4] <<< JVM J0: EOF > > > > > > [...truncated 1 lines...] > > >[junit4] ERROR: JVM J0 ended with an exception, command line: > > /Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/jre/bi > > n/ java -XX:+UseCompressedOops -XX:+UseParallelGC - > > XX:+HeapDumpOnOutOfMemoryError - > > XX:HeapDumpPath=/Users/jenkins/workspace/Lucene-Solr-trunk- > > MacOSX/heapdumps -Dtests.prefix=tests - > Dtests.seed=D104BAEF7A7EA4D6 > > -Xmx512M -Dtests.iters= -Dtests.verbose=false -Dtests.infostream=false > > - Dtests.codec=random -Dtests.postingsformat=random - > > Dtests.docvaluesformat=random -Dtests.locale=random - > > Dtests.timezone=random -Dtests.directory=random - > > Dtests.linedocsfile=europarl.lines.txt.gz > > -Dtests.luceneMatchVersion=5.0 - Dtests.cleanthreads=perMethod - > > Djava.util.logging.config.file=/Users/jenkins/workspace/Lucene-Solr-tr > > unk- MacOSX/lucene/tools/junit4/logging.properties > > -Dtests.nightly=false - Dtests.weekly=false -Dtests.monster=false > > -Dtests.slow=true - Dtests.asserts.gracious=false -Dtests.multiplier=1 > > -DtempDir=. - Djava.io.tmpdir=. > > -Djunit4.tempDir=/Users/jenkins/workspace/Lucene-Solr- > > trunk-MacOSX/lucene/build/join/test/temp - > > Dclover.db.dir=/Users/jenkins/workspace/Lucene-Solr-trunk- > > MacOSX/lucene/build/clover/db - > > Djava.security.manager=org.apache.lucene.util.TestSecurityManager - > > Djava.security.policy=/Users/jenkins/workspace/Lucene-Solr-trunk- > > MacOSX/lucene/tools/junit4/tests.policy -Dlucene.version=5.0-SNAPSHOT > > - > > Djetty.testMode=1 -Djetty.insecurerandom=1 - >
RE: [JENKINS] Lucene-Solr-trunk-MacOSX (64bit/jdk1.8.0) - Build # 1541 - Failure!
Uwe, This happens from time to time. It looks like wrong usage of syscalls in the MacOSX libc, because it seems to happen if the clock on Mac goes backwards. This happens more often in virtual machines, because time synchronization of virtual machine and host clock happens more often because of virtual processor scheduling issues, so clocks differ more often and need to be corrected by NTP. As far as I remember, there was already a patch on hotspot mailing list to use more modern APIs. Uwe - Uwe Schindler H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de eMail: u...@thetaphi.de > -Original Message- > From: Robert Muir [mailto:rcm...@gmail.com] > Sent: Thursday, May 01, 2014 10:52 AM > To: dev@lucene.apache.org > Subject: Re: [JENKINS] Lucene-Solr-trunk-MacOSX (64bit/jdk1.8.0) - Build # > 1541 - Failure! > > I downloaded this jvm on my mac and tried the master seed a few times, no > SIGFPE. > > I can't reproduce it. > > On Thu, May 1, 2014 at 2:54 AM, Policeman Jenkins Server > wrote: > > Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-MacOSX/1541/ > > Java: 64bit/jdk1.8.0 -XX:+UseCompressedOops -XX:+UseParallelGC > > > > No tests ran. > > > > Build Log: > > [...truncated 8349 lines...] > >[junit4] JVM J0: stdout was not empty, see: > /Users/jenkins/workspace/Lucene-Solr-trunk- > MacOSX/lucene/build/join/test/temp/junit4-J0- > 20140501_065329_348.sysout > >[junit4] >>> JVM J0: stdout (verbatim) > >[junit4] # > >[junit4] # A fatal error has been detected by the Java Runtime > Environment: > >[junit4] # > >[junit4] # SIGFPE (0x8) at pc=0x7fff8a0d6b63, pid=203, tid=21251 > >[junit4] # > >[junit4] # JRE version: Java(TM) SE Runtime Environment (8.0_05-b13) > (build 1.8.0_05-b13) > >[junit4] # Java VM: Java HotSpot(TM) 64-Bit Server VM (25.5-b02 mixed > mode bsd-amd64 compressed oops) > >[junit4] # Problematic frame: > >[junit4] # C [libsystem_c.dylib+0x2b63] > __commpage_gettimeofday+0x43 > >[junit4] # > >[junit4] # Failed to write core dump. Core dumps have been disabled. To > enable core dumping, try "ulimit -c unlimited" before starting Java again > >[junit4] # > >[junit4] # An error report file with more information is saved as: > >[junit4] # /Users/jenkins/workspace/Lucene-Solr-trunk- > MacOSX/lucene/build/join/test/J0/hs_err_pid203.log > >[junit4] # > >[junit4] # Compiler replay data is saved as: > >[junit4] # /Users/jenkins/workspace/Lucene-Solr-trunk- > MacOSX/lucene/build/join/test/J0/replay_pid203.log > >[junit4] # > >[junit4] # If you would like to submit a bug report, please visit: > >[junit4] # http://bugreport.sun.com/bugreport/crash.jsp > >[junit4] # > >[junit4] <<< JVM J0: EOF > > > > [...truncated 1 lines...] > >[junit4] ERROR: JVM J0 ended with an exception, command line: > /Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/jre/bin/ > java -XX:+UseCompressedOops -XX:+UseParallelGC - > XX:+HeapDumpOnOutOfMemoryError - > XX:HeapDumpPath=/Users/jenkins/workspace/Lucene-Solr-trunk- > MacOSX/heapdumps -Dtests.prefix=tests -Dtests.seed=D104BAEF7A7EA4D6 > -Xmx512M -Dtests.iters= -Dtests.verbose=false -Dtests.infostream=false - > Dtests.codec=random -Dtests.postingsformat=random - > Dtests.docvaluesformat=random -Dtests.locale=random - > Dtests.timezone=random -Dtests.directory=random - > Dtests.linedocsfile=europarl.lines.txt.gz -Dtests.luceneMatchVersion=5.0 - > Dtests.cleanthreads=perMethod - > Djava.util.logging.config.file=/Users/jenkins/workspace/Lucene-Solr-trunk- > MacOSX/lucene/tools/junit4/logging.properties -Dtests.nightly=false - > Dtests.weekly=false -Dtests.monster=false -Dtests.slow=true - > Dtests.asserts.gracious=false -Dtests.multiplier=1 -DtempDir=. - > Djava.io.tmpdir=. -Djunit4.tempDir=/Users/jenkins/workspace/Lucene-Solr- > trunk-MacOSX/lucene/build/join/test/temp - > Dclover.db.dir=/Users/jenkins/workspace/Lucene-Solr-trunk- > MacOSX/lucene/build/clover/db - > Djava.security.manager=org.apache.lucene.util.TestSecurityManager - > Djava.security.policy=/Users/jenkins/workspace/Lucene-Solr-trunk- > MacOSX/lucene/tools/junit4/tests.policy -Dlucene.version=5.0-SNAPSHOT - > Djetty.testMode=1 -Djetty.insecurerandom=1 - > Dsolr.directoryFactory=org.apache.solr.core.MockDirectoryFactory - > Djava.awt.headless=true -Djdk.map.althashing.threshold=0 - > Dtests.leaveTemporary=false -Dtests.filterstacks=true - > Dtests.disableHdfs=true -Dfile.encoding=UTF-8 -classpath > /Users/jenkins/workspace/Lucene-Solr-trunk- > MacOSX/lucene/build/join/classes/test:/Users/jenkins/workspace/Lucene- > Solr-trunk-MacOSX/lucene/build/test- > framework/classes/java:/Users/jenkins/workspace/Lucene-Solr-trunk- > MacOSX/lucene/build/codecs/classes/java:/Users/jenkins/workspace/Lucen > e-Solr-trunk-MacOSX/lucene/build/grouping/lucene-grouping-5.0- > SNAPSHOT.jar:/Users/jenkins/workspace/Lucene-Solr-trunk- > MacOSX/lucene/build/core/classes/j
[jira] [Commented] (SOLR-5514) atomic update throws exception if the schema contains uuid fields: Invalid UUID String: 'java.util.UUID:e26c4d56-e98d-41de-9b7f-f63192089670'
[
https://issues.apache.org/jira/browse/SOLR-5514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13986465#comment-13986465
]
Elran Dvir commented on SOLR-5514:
--
It happens in my real time environment all the time. After the fix, it seems it
stopped happening.
I tried to create a test case, but I couldn't reproduce the problem.
I think it happens when there is some time between updates, so the older
version of the document is fetched by a searcher and by Update Log.
I think the update shouldn't contain the uuid field. The uuid field comes from
the original document and is merged.
> atomic update throws exception if the schema contains uuid fields: Invalid
> UUID String: 'java.util.UUID:e26c4d56-e98d-41de-9b7f-f63192089670'
> -
>
> Key: SOLR-5514
> URL: https://issues.apache.org/jira/browse/SOLR-5514
> Project: Solr
> Issue Type: Bug
>Affects Versions: 4.5.1
> Environment: unix and windows
>Reporter: Dirk Reuss
>Assignee: Shalin Shekhar Mangar
> Attachments: SOLR-5514.patch
>
>
> I am updating an exiting document with the statement
> newvalue
> All fields are stored and I have several UUID fields. About 10-20% of the
> update commands will fail with the message: (example)
> Invalid UUID String: 'java.util.UUID:532c9353-d391-4a04-8618-dc2fa1ef8b35'
> the point is that java.util.UUID seems to be prepended to the original uuid
> stored in the field and when the value is written this error occours.
> I tried to check if this specific uuid field was the problem and
> added the uuid field in the update xml with( update='set'>...). But the error simply moved to an other uuid field.
> here is the original exception:
> 500 name="QTime">34Error while
> creating field
> 'MyUUIDField{type=uuid,properties=indexed,stored,omitTermFreqAndPositions,required,
> required=true}' from value
> 'java.util.UUID:e26c4d56-e98d-41de-9b7f-f63192089670' name="trace">org.apache.solr.common.SolrException: Error while creating field
> 'MyUUIDField{type=uuid,properties=indexed,stored,omitTermFreqAndPositions,required,
> required=true}' from value
> 'java.util.UUID:e26c4d56-e98d-41de-9b7f-f63192089670'
> at org.apache.solr.schema.FieldType.createField(FieldType.java:259)
> at org.apache.solr.schema.StrField.createFields(StrField.java:56)
> at
> org.apache.solr.update.DocumentBuilder.addField(DocumentBuilder.java:47)
> at
> org.apache.solr.update.DocumentBuilder.toDocument(DocumentBuilder.java:118)
> at
> org.apache.solr.update.AddUpdateCommand.getLuceneDocument(AddUpdateCommand.java:77)
> at
> org.apache.solr.update.DirectUpdateHandler2.addDoc(DirectUpdateHandler2.java:215)
> at
> org.apache.solr.update.processor.RunUpdateProcessor.processAdd(RunUpdateProcessorFactory.java:69)
> at
> org.apache.solr.update.processor.UpdateRequestProcessor.processAdd(UpdateRequestProcessor.java:51)
> at
> org.apache.solr.update.processor.DistributedUpdateProcessor.doLocalAdd(DistributedUpdateProcessor.java:556)
> at
> org.apache.solr.update.processor.DistributedUpdateProcessor.versionAdd(DistributedUpdateProcessor.java:692)
> at
> org.apache.solr.update.processor.DistributedUpdateProcessor.processAdd(DistributedUpdateProcessor.java:435)
> at
> org.apache.solr.update.processor.LogUpdateProcessor.processAdd(LogUpdateProcessorFactory.java:100)
> at
> org.apache.solr.handler.loader.XMLLoader.processUpdate(XMLLoader.java:247)
> at org.apache.solr.handler.loader.XMLLoader.load(XMLLoader.java:174)
> at
> org.apache.solr.handler.UpdateRequestHandler$1.load(UpdateRequestHandler.java:92)
> at
> org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:74)
> at
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
> at org.apache.solr.core.SolrCore.execute(SolrCore.java:1859)
> at
> org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:703)
> at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:406)
> at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:195)
> at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
> at
> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
> at
> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
> at
> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java
Re: [JENKINS] Lucene-4x-Linux-Java7-64-test-only - Build # 20243 - Failure!
I did the refactoring (4.x: im porting forward to trunk now). Thing is it still doesnt fix the bug. but i think its easier to work with. On Wed, Apr 30, 2014 at 4:44 PM, Robert Muir wrote: > The exception handling in processDocument() is too complex. it must do > try+finally with two separate cases: aborting exception and > non-aborting exception, depending on where it happens. > > I think since the whole thing loops through fields, if we factor out > processField it will be easier... > > On Wed, Apr 30, 2014 at 4:21 PM, Robert Muir wrote: >> reproduces if you tack on -Dtests.dups=100 ... looking >> >> On Wed, Apr 30, 2014 at 4:04 PM, wrote: >>> Build: builds.flonkings.com/job/Lucene-4x-Linux-Java7-64-test-only/20243/ >>> >>> 1 tests failed. >>> REGRESSION: >>> org.apache.lucene.index.TestIndexWriterWithThreads.testImmediateDiskFullWithThreads >>> >>> Error Message: >>> fdx size mismatch: docCount is 12 but fdx file size is 133 >>> file=MockIndexOutputWrapper(org.apache.lucene.store.RAMOutputStream@3afabf87); >>> now aborting this merge to prevent index corruption >>> >>> Stack Trace: >>> java.lang.RuntimeException: fdx size mismatch: docCount is 12 but fdx file >>> size is 133 >>> file=MockIndexOutputWrapper(org.apache.lucene.store.RAMOutputStream@3afabf87); >>> now aborting this merge to prevent index corruption >>> at >>> __randomizedtesting.SeedInfo.seed([70486912DEA23EAA:FF309021F56884D9]:0) >>> at >>> org.apache.lucene.codecs.lucene40.Lucene40StoredFieldsWriter.finish(Lucene40StoredFieldsWriter.java:233) >>> at >>> org.apache.lucene.index.DefaultIndexingChain.flush(DefaultIndexingChain.java:96) >>> at >>> org.apache.lucene.index.DocumentsWriterPerThread.flush(DocumentsWriterPerThread.java:415) >>> at >>> org.apache.lucene.index.DocumentsWriter.doFlush(DocumentsWriter.java:512) >>> at >>> org.apache.lucene.index.DocumentsWriter.flushAllThreads(DocumentsWriter.java:622) >>> at >>> org.apache.lucene.index.IndexWriter.doFlush(IndexWriter.java:3203) >>> at org.apache.lucene.index.IndexWriter.flush(IndexWriter.java:3179) >>> at >>> org.apache.lucene.index.IndexWriter.closeInternal(IndexWriter.java:995) >>> at org.apache.lucene.index.IndexWriter.close(IndexWriter.java:939) >>> at >>> org.apache.lucene.index.TestIndexWriterWithThreads.testImmediateDiskFullWithThreads(TestIndexWriterWithThreads.java:168) >>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >>> at >>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) >>> at >>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >>> at java.lang.reflect.Method.invoke(Method.java:606) >>> at >>> com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618) >>> at >>> com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827) >>> at >>> com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863) >>> at >>> com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877) >>> at >>> org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50) >>> at >>> org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51) >>> at >>> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) >>> at >>> com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55) >>> at >>> org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49) >>> at >>> org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65) >>> at >>> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) >>> at >>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) >>> at >>> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:360) >>> at >>> com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:793) >>> at >>> com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:453) >>> at >>> com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836) >>> at >>> com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738) >>> at >>> com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772) >>> at >>> com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783) >>> at >>> org.apac
[jira] [Commented] (SOLR-5514) atomic update throws exception if the schema contains uuid fields: Invalid UUID String: 'java.util.UUID:e26c4d56-e98d-41de-9b7f-f63192089670'
[
https://issues.apache.org/jira/browse/SOLR-5514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13986463#comment-13986463
]
Shalin Shekhar Mangar commented on SOLR-5514:
-
Elran, can you create a test case which demonstrates the problem? Or at least
provide the steps to reproduce the problem? I have not been able to reproduce
the problem the last time I tried but I admit that I did not spend too much
time on it.
Your patch adds UUID as a known type which means that we'd need to bump the
javabin version which brings us to the same discussion that happened in
SOLR-807.
> atomic update throws exception if the schema contains uuid fields: Invalid
> UUID String: 'java.util.UUID:e26c4d56-e98d-41de-9b7f-f63192089670'
> -
>
> Key: SOLR-5514
> URL: https://issues.apache.org/jira/browse/SOLR-5514
> Project: Solr
> Issue Type: Bug
>Affects Versions: 4.5.1
> Environment: unix and windows
>Reporter: Dirk Reuss
>Assignee: Shalin Shekhar Mangar
> Attachments: SOLR-5514.patch
>
>
> I am updating an exiting document with the statement
> newvalue
> All fields are stored and I have several UUID fields. About 10-20% of the
> update commands will fail with the message: (example)
> Invalid UUID String: 'java.util.UUID:532c9353-d391-4a04-8618-dc2fa1ef8b35'
> the point is that java.util.UUID seems to be prepended to the original uuid
> stored in the field and when the value is written this error occours.
> I tried to check if this specific uuid field was the problem and
> added the uuid field in the update xml with( update='set'>...). But the error simply moved to an other uuid field.
> here is the original exception:
> 500 name="QTime">34Error while
> creating field
> 'MyUUIDField{type=uuid,properties=indexed,stored,omitTermFreqAndPositions,required,
> required=true}' from value
> 'java.util.UUID:e26c4d56-e98d-41de-9b7f-f63192089670' name="trace">org.apache.solr.common.SolrException: Error while creating field
> 'MyUUIDField{type=uuid,properties=indexed,stored,omitTermFreqAndPositions,required,
> required=true}' from value
> 'java.util.UUID:e26c4d56-e98d-41de-9b7f-f63192089670'
> at org.apache.solr.schema.FieldType.createField(FieldType.java:259)
> at org.apache.solr.schema.StrField.createFields(StrField.java:56)
> at
> org.apache.solr.update.DocumentBuilder.addField(DocumentBuilder.java:47)
> at
> org.apache.solr.update.DocumentBuilder.toDocument(DocumentBuilder.java:118)
> at
> org.apache.solr.update.AddUpdateCommand.getLuceneDocument(AddUpdateCommand.java:77)
> at
> org.apache.solr.update.DirectUpdateHandler2.addDoc(DirectUpdateHandler2.java:215)
> at
> org.apache.solr.update.processor.RunUpdateProcessor.processAdd(RunUpdateProcessorFactory.java:69)
> at
> org.apache.solr.update.processor.UpdateRequestProcessor.processAdd(UpdateRequestProcessor.java:51)
> at
> org.apache.solr.update.processor.DistributedUpdateProcessor.doLocalAdd(DistributedUpdateProcessor.java:556)
> at
> org.apache.solr.update.processor.DistributedUpdateProcessor.versionAdd(DistributedUpdateProcessor.java:692)
> at
> org.apache.solr.update.processor.DistributedUpdateProcessor.processAdd(DistributedUpdateProcessor.java:435)
> at
> org.apache.solr.update.processor.LogUpdateProcessor.processAdd(LogUpdateProcessorFactory.java:100)
> at
> org.apache.solr.handler.loader.XMLLoader.processUpdate(XMLLoader.java:247)
> at org.apache.solr.handler.loader.XMLLoader.load(XMLLoader.java:174)
> at
> org.apache.solr.handler.UpdateRequestHandler$1.load(UpdateRequestHandler.java:92)
> at
> org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:74)
> at
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
> at org.apache.solr.core.SolrCore.execute(SolrCore.java:1859)
> at
> org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:703)
> at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:406)
> at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:195)
> at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
> at
> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
> at
> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
> at
> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.jav
[jira] [Commented] (SOLR-6013) Fix method visibility of Evaluator, refactor DateFormatEvaluator for extensibility
[ https://issues.apache.org/jira/browse/SOLR-6013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13986460#comment-13986460 ] ASF subversion and git services commented on SOLR-6013: --- Commit 1591574 from sha...@apache.org in branch 'dev/branches/branch_4x' [ https://svn.apache.org/r1591574 ] SOLR-6013: Fix method visibility of Evaluator, refactor DateFormatEvaluator for extensibility > Fix method visibility of Evaluator, refactor DateFormatEvaluator for > extensibility > -- > > Key: SOLR-6013 > URL: https://issues.apache.org/jira/browse/SOLR-6013 > Project: Solr > Issue Type: Improvement > Components: contrib - DataImportHandler >Affects Versions: 4.7 >Reporter: Aaron LaBella >Assignee: Shalin Shekhar Mangar > Fix For: 4.9, 5.0 > > Attachments: 0001-add-getters-for-datemathparser.patch, > 0001-change-method-access-to-protected.patch, > 0001-change-method-variable-visibility-and-refactor-for-extensibility.patch, > SOLR-6013.patch > > Original Estimate: 1h > Remaining Estimate: 1h > > This is similar to issue 5981, the Evaluator class is declared as abstract, > yet the parseParams method is package private? Surely this is an oversight, > as I wouldn't expect everyone writing their own evaluators to have to deal > with parsing the parameters. > Similarly, I needed to refactor DateFormatEvaluator because I need to do some > custom date math/parsing and it wasn't written in a way that I can extend it. > Please review/apply my attached patch to the next version of Solr, ie: 4.8 or > 4.9 if I must wait. > Thanks! -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Resolved] (SOLR-6013) Fix method visibility of Evaluator, refactor DateFormatEvaluator for extensibility
[ https://issues.apache.org/jira/browse/SOLR-6013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shalin Shekhar Mangar resolved SOLR-6013. - Resolution: Fixed Fix Version/s: 5.0 Thanks Aaron! > Fix method visibility of Evaluator, refactor DateFormatEvaluator for > extensibility > -- > > Key: SOLR-6013 > URL: https://issues.apache.org/jira/browse/SOLR-6013 > Project: Solr > Issue Type: Improvement > Components: contrib - DataImportHandler >Affects Versions: 4.7 >Reporter: Aaron LaBella >Assignee: Shalin Shekhar Mangar > Fix For: 4.9, 5.0 > > Attachments: 0001-add-getters-for-datemathparser.patch, > 0001-change-method-access-to-protected.patch, > 0001-change-method-variable-visibility-and-refactor-for-extensibility.patch, > SOLR-6013.patch > > Original Estimate: 1h > Remaining Estimate: 1h > > This is similar to issue 5981, the Evaluator class is declared as abstract, > yet the parseParams method is package private? Surely this is an oversight, > as I wouldn't expect everyone writing their own evaluators to have to deal > with parsing the parameters. > Similarly, I needed to refactor DateFormatEvaluator because I need to do some > custom date math/parsing and it wasn't written in a way that I can extend it. > Please review/apply my attached patch to the next version of Solr, ie: 4.8 or > 4.9 if I must wait. > Thanks! -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-6013) Fix method visibility of Evaluator, refactor DateFormatEvaluator for extensibility
[ https://issues.apache.org/jira/browse/SOLR-6013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13986459#comment-13986459 ] ASF subversion and git services commented on SOLR-6013: --- Commit 1591573 from sha...@apache.org in branch 'dev/trunk' [ https://svn.apache.org/r1591573 ] SOLR-6013: Fix method visibility of Evaluator, refactor DateFormatEvaluator for extensibility > Fix method visibility of Evaluator, refactor DateFormatEvaluator for > extensibility > -- > > Key: SOLR-6013 > URL: https://issues.apache.org/jira/browse/SOLR-6013 > Project: Solr > Issue Type: Improvement > Components: contrib - DataImportHandler >Affects Versions: 4.7 >Reporter: Aaron LaBella >Assignee: Shalin Shekhar Mangar > Fix For: 4.9 > > Attachments: 0001-add-getters-for-datemathparser.patch, > 0001-change-method-access-to-protected.patch, > 0001-change-method-variable-visibility-and-refactor-for-extensibility.patch, > SOLR-6013.patch > > Original Estimate: 1h > Remaining Estimate: 1h > > This is similar to issue 5981, the Evaluator class is declared as abstract, > yet the parseParams method is package private? Surely this is an oversight, > as I wouldn't expect everyone writing their own evaluators to have to deal > with parsing the parameters. > Similarly, I needed to refactor DateFormatEvaluator because I need to do some > custom date math/parsing and it wasn't written in a way that I can extend it. > Please review/apply my attached patch to the next version of Solr, ie: 4.8 or > 4.9 if I must wait. > Thanks! -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Assigned] (SOLR-6013) Fix method visibility of Evaluator, refactor DateFormatEvaluator for extensibility
[ https://issues.apache.org/jira/browse/SOLR-6013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shalin Shekhar Mangar reassigned SOLR-6013: --- Assignee: Shalin Shekhar Mangar > Fix method visibility of Evaluator, refactor DateFormatEvaluator for > extensibility > -- > > Key: SOLR-6013 > URL: https://issues.apache.org/jira/browse/SOLR-6013 > Project: Solr > Issue Type: Improvement > Components: contrib - DataImportHandler >Affects Versions: 4.7 >Reporter: Aaron LaBella >Assignee: Shalin Shekhar Mangar > Fix For: 4.9 > > Attachments: 0001-add-getters-for-datemathparser.patch, > 0001-change-method-access-to-protected.patch, > 0001-change-method-variable-visibility-and-refactor-for-extensibility.patch, > SOLR-6013.patch > > Original Estimate: 1h > Remaining Estimate: 1h > > This is similar to issue 5981, the Evaluator class is declared as abstract, > yet the parseParams method is package private? Surely this is an oversight, > as I wouldn't expect everyone writing their own evaluators to have to deal > with parsing the parameters. > Similarly, I needed to refactor DateFormatEvaluator because I need to do some > custom date math/parsing and it wasn't written in a way that I can extend it. > Please review/apply my attached patch to the next version of Solr, ie: 4.8 or > 4.9 if I must wait. > Thanks! -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-6013) Fix method visibility of Evaluator, refactor DateFormatEvaluator for extensibility
[ https://issues.apache.org/jira/browse/SOLR-6013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shalin Shekhar Mangar updated SOLR-6013: Attachment: SOLR-6013.patch Thanks Aaron. Here's a single patch with all your changes. We typically attach a patch named as SOLR-xxx.patch and if/when we need to revise, we upload patch with the same name again. Jira will automatically gray out the older patches in the UI. This makes it easier to know the latest patch. I made three changes: # FileListEntityProcessor.getDate was missing a return statement # I removed the getters for VariableWrapper and made the attributes public final # Changed DateFormatCacheKey to static protected. I will commit this shortly. > Fix method visibility of Evaluator, refactor DateFormatEvaluator for > extensibility > -- > > Key: SOLR-6013 > URL: https://issues.apache.org/jira/browse/SOLR-6013 > Project: Solr > Issue Type: Improvement > Components: contrib - DataImportHandler >Affects Versions: 4.7 >Reporter: Aaron LaBella > Fix For: 4.9 > > Attachments: 0001-add-getters-for-datemathparser.patch, > 0001-change-method-access-to-protected.patch, > 0001-change-method-variable-visibility-and-refactor-for-extensibility.patch, > SOLR-6013.patch > > Original Estimate: 1h > Remaining Estimate: 1h > > This is similar to issue 5981, the Evaluator class is declared as abstract, > yet the parseParams method is package private? Surely this is an oversight, > as I wouldn't expect everyone writing their own evaluators to have to deal > with parsing the parameters. > Similarly, I needed to refactor DateFormatEvaluator because I need to do some > custom date math/parsing and it wasn't written in a way that I can extend it. > Please review/apply my attached patch to the next version of Solr, ie: 4.8 or > 4.9 if I must wait. > Thanks! -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: [JENKINS] Lucene-Solr-trunk-MacOSX (64bit/jdk1.8.0) - Build # 1541 - Failure!
I downloaded this jvm on my mac and tried the master seed a few times, no SIGFPE. I can't reproduce it. On Thu, May 1, 2014 at 2:54 AM, Policeman Jenkins Server wrote: > Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-MacOSX/1541/ > Java: 64bit/jdk1.8.0 -XX:+UseCompressedOops -XX:+UseParallelGC > > No tests ran. > > Build Log: > [...truncated 8349 lines...] >[junit4] JVM J0: stdout was not empty, see: > /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/lucene/build/join/test/temp/junit4-J0-20140501_065329_348.sysout >[junit4] >>> JVM J0: stdout (verbatim) >[junit4] # >[junit4] # A fatal error has been detected by the Java Runtime Environment: >[junit4] # >[junit4] # SIGFPE (0x8) at pc=0x7fff8a0d6b63, pid=203, tid=21251 >[junit4] # >[junit4] # JRE version: Java(TM) SE Runtime Environment (8.0_05-b13) > (build 1.8.0_05-b13) >[junit4] # Java VM: Java HotSpot(TM) 64-Bit Server VM (25.5-b02 mixed mode > bsd-amd64 compressed oops) >[junit4] # Problematic frame: >[junit4] # C [libsystem_c.dylib+0x2b63] __commpage_gettimeofday+0x43 >[junit4] # >[junit4] # Failed to write core dump. Core dumps have been disabled. To > enable core dumping, try "ulimit -c unlimited" before starting Java again >[junit4] # >[junit4] # An error report file with more information is saved as: >[junit4] # > /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/lucene/build/join/test/J0/hs_err_pid203.log >[junit4] # >[junit4] # Compiler replay data is saved as: >[junit4] # > /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/lucene/build/join/test/J0/replay_pid203.log >[junit4] # >[junit4] # If you would like to submit a bug report, please visit: >[junit4] # http://bugreport.sun.com/bugreport/crash.jsp >[junit4] # >[junit4] <<< JVM J0: EOF > > [...truncated 1 lines...] >[junit4] ERROR: JVM J0 ended with an exception, command line: > /Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/jre/bin/java > -XX:+UseCompressedOops -XX:+UseParallelGC -XX:+HeapDumpOnOutOfMemoryError > -XX:HeapDumpPath=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/heapdumps > -Dtests.prefix=tests -Dtests.seed=D104BAEF7A7EA4D6 -Xmx512M -Dtests.iters= > -Dtests.verbose=false -Dtests.infostream=false -Dtests.codec=random > -Dtests.postingsformat=random -Dtests.docvaluesformat=random > -Dtests.locale=random -Dtests.timezone=random -Dtests.directory=random > -Dtests.linedocsfile=europarl.lines.txt.gz -Dtests.luceneMatchVersion=5.0 > -Dtests.cleanthreads=perMethod > -Djava.util.logging.config.file=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/lucene/tools/junit4/logging.properties > -Dtests.nightly=false -Dtests.weekly=false -Dtests.monster=false > -Dtests.slow=true -Dtests.asserts.gracious=false -Dtests.multiplier=1 > -DtempDir=. -Djava.io.tmpdir=. > -Djunit4.tempDir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/lucene/build/join/test/temp > > -Dclover.db.dir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/lucene/build/clover/db > -Djava.security.manager=org.apache.lucene.util.TestSecurityManager > -Djava.security.policy=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/lucene/tools/junit4/tests.policy > -Dlucene.version=5.0-SNAPSHOT -Djetty.testMode=1 -Djetty.insecurerandom=1 > -Dsolr.directoryFactory=org.apache.solr.core.MockDirectoryFactory > -Djava.awt.headless=true -Djdk.map.althashing.threshold=0 > -Dtests.leaveTemporary=false -Dtests.filterstacks=true > -Dtests.disableHdfs=true -Dfile.encoding=UTF-8 -classpath > /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/lucene/build/join/classes/test:/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/lucene/build/test-framework/classes/java:/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/lucene/build/codecs/classes/java:/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/lucene/build/grouping/lucene-grouping-5.0-SNAPSHOT.jar:/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/lucene/build/core/classes/java:/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/lucene/test-framework/lib/junit-4.10.jar:/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/lucene/test-framework/lib/randomizedtesting-runner-2.1.3.jar:/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/lucene/build/join/classes/java:/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-launcher.jar:/Users/jenkins/.ant/lib/ivy-2.3.0.jar:/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-antlr.jar:/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-bcel.jar:/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-bsf.jar:/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-log4j.jar:/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-oro.jar:/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-regexp.jar:/Users/jenkins/tools/hudson.tasks.A
Re: [JENKINS] Lucene-4x-Linux-Java7-64-test-only - Build # 20274 - Failure!
I committed a fix. On Thu, May 1, 2014 at 12:44 AM, wrote: > Build: builds.flonkings.com/job/Lucene-4x-Linux-Java7-64-test-only/20274/ > > 1 tests failed. > REGRESSION: org.apache.lucene.index.TestIndexWriter.testManyFields > > Error Message: > 3.x codec does not support payloads on vectors! > > Stack Trace: > java.lang.UnsupportedOperationException: 3.x codec does not support payloads > on vectors! > at > __randomizedtesting.SeedInfo.seed([A7DEAB3AF9A18B1F:8FE0FDA4EBDC8889]:0) > at > org.apache.lucene.codecs.lucene3x.PreFlexRWTermVectorsWriter.startField(PreFlexRWTermVectorsWriter.java:82) > at > org.apache.lucene.index.TermVectorsConsumerPerField.finishDocument(TermVectorsConsumerPerField.java:83) > at > org.apache.lucene.index.TermVectorsConsumer.finishDocument(TermVectorsConsumer.java:112) > at org.apache.lucene.index.TermsHash.finishDocument(TermsHash.java:93) > at > org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:345) > at > org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:222) > at > org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:459) > at > org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1541) > at > org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1211) > at > org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1192) > at > org.apache.lucene.index.TestIndexWriter.testManyFields(TestIndexWriter.java:275) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at > com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618) > at > com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827) > at > com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863) > at > com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877) > at > org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50) > at > org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51) > at > org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) > at > com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55) > at > org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49) > at > org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65) > at > org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) > at > com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) > at > com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:360) > at > com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:793) > at > com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:453) > at > com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836) > at > com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738) > at > com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772) > at > com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783) > at > org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) > at > org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42) > at > com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55) > at > com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) > at > com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) > at > com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) > at > com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) > at > com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdap
