[jira] [Updated] (LUCENE-6616) IndexWriter should list files once on init, and IFD should not suppress FNFE
[ https://issues.apache.org/jira/browse/LUCENE-6616?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael McCandless updated LUCENE-6616: --- Attachment: LUCENE-6616.patch New patch, it's closer I think. Tests passed once, but I still have one nocommit ... I made some drastic changes to IndexFileDeleter (thanks to [~rcmuir] for this idea!). All places that used to directly delete a file now instead make three passes: * First just gather up all files wanting to be deleted, adding them to the deletable HashSet. * Second, try to delete all the segments_N files in that set. * Finally, delete all non-segments files, only if 2nd pass succeeded. This ensures that even in the presence of a virus checker, the index is never left in a state where a segments_N is referencing a non-existent file. I also fixed all file deletion done by IW to use IFD's methods, and made IFD.deleteFile private. IndexWriter should list files once on init, and IFD should not suppress FNFE Key: LUCENE-6616 URL: https://issues.apache.org/jira/browse/LUCENE-6616 Project: Lucene - Core Issue Type: Improvement Reporter: Michael McCandless Assignee: Michael McCandless Fix For: 5.3, Trunk Attachments: LUCENE-6616.patch, LUCENE-6616.patch Some nice ideas [~rcmuir] had for cleaning up IW/IFD on init ... -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-6595) CharFilter offsets correction is wonky
[
https://issues.apache.org/jira/browse/LUCENE-6595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14604614#comment-14604614
]
Michael McCandless commented on LUCENE-6595:
I think we'll also need to conditionalize this behavior change by version for
back compat ...
CharFilter offsets correction is wonky
--
Key: LUCENE-6595
URL: https://issues.apache.org/jira/browse/LUCENE-6595
Project: Lucene - Core
Issue Type: Bug
Reporter: Michael McCandless
Attachments: LUCENE-6595.patch, LUCENE-6595.patch
Spinoff from this original Elasticsearch issue:
https://github.com/elastic/elasticsearch/issues/11726
If I make a MappingCharFilter with these mappings:
{noformat}
( -
) -
{noformat}
i.e., just erase left and right paren, then tokenizing the string
(F31) with e.g. WhitespaceTokenizer, produces a single token F31,
with start offset 1 (good).
But for its end offset, I would expect/want 4, but it produces 5
today.
This can be easily explained given how the mapping works: each time a
mapping rule matches, we update the cumulative offset difference,
conceptually as an array like this (it's encoded more compactly):
{noformat}
Output offset: 0 1 2 3
Input offset: 1 2 3 5
{noformat}
When the tokenizer produces F31, it assigns it startOffset=0 and
endOffset=3 based on the characters it sees (F, 3, 1). It then asks
the CharFilter to correct those offsets, mapping them backwards
through the above arrays, which creates startOffset=1 (good) and
endOffset=5 (bad).
At first, to fix this, I thought this is an off-by-1 and when
correcting the endOffset we really should return
1+correct(outputEndOffset-1), which would return the correct value (4)
here.
But that's too naive, e.g. here's another example:
{noformat}
- cc
{noformat}
If I then tokenize , today we produce the correct offsets (0, 4)
but if we do this off-by-1 fix for endOffset, we would get the wrong
endOffset (2).
I'm not sure what to do here...
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-6629) Random 7200 seconds build timeouts / infinite loops in Lucene tests?
[
https://issues.apache.org/jira/browse/LUCENE-6629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14604651#comment-14604651
]
Michael McCandless commented on LUCENE-6629:
Here's another possible case, from
http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/12746 (but now deleted):
{noformat}
[junit4] 21) Thread[id=149,
name=TEST-TestSnowballVocab.testStemmers-seed#[7FBAD445ED608074],
state=RUNNABLE, group=TGRP-TestSnowballVocab]
[junit4] 2 at
org.tartarus.snowball.ext.FrenchStemmer.stem(FrenchStemmer.java:1379)
[junit4] 2 at
org.apache.lucene.analysis.snowball.SnowballFilter.incrementToken(SnowballFilter.java:95)
[junit4] 2 at
org.apache.lucene.analysis.BaseTokenStreamTestCase.checkResetException(BaseTokenStreamTestCase.java:400)
[junit4] 2 at
org.apache.lucene.analysis.BaseTokenStreamTestCase.assertAnalyzesTo(BaseTokenStreamTestCase.java:352)
[junit4] 2 at
org.apache.lucene.analysis.BaseTokenStreamTestCase.assertAnalyzesTo(BaseTokenStreamTestCase.java:362)
[junit4] 2 at
org.apache.lucene.analysis.BaseTokenStreamTestCase.checkOneTerm(BaseTokenStreamTestCase.java:423)
[junit4] 2 at
org.apache.lucene.analysis.VocabularyAssert.assertVocabulary(VocabularyAssert.java:47)
[junit4] 2 at
org.apache.lucene.analysis.VocabularyAssert.assertVocabulary(VocabularyAssert.java:73)
[junit4] 2 at
org.apache.lucene.analysis.snowball.TestSnowballVocab.assertCorrectOutput(TestSnowballVocab.java:80)
[junit4] 2 at
org.apache.lucene.analysis.snowball.TestSnowballVocab.testStemmers(TestSnowballVocab.java:45)
{noformat}
Random 7200 seconds build timeouts / infinite loops in Lucene tests?
Key: LUCENE-6629
URL: https://issues.apache.org/jira/browse/LUCENE-6629
Project: Lucene - Core
Issue Type: Bug
Reporter: Michael McCandless
I'm not sure what's going on here, but every so often a Jenkins run will fail
with a build timeout (7200 seconds) with stack traces that do not look like
deadlock. They never reproduce for me.
We really need to improve test infra here, so that each HEARTBEAT we also got
1) full thread stacks and 2) total CPU usage of the process as reported by
the ManagementBean APIs ... this would shed more light on whether the JVM is
somehow hung vs our bug (infinite loop). But infinite loop seems most likely
... the stacks always seem to be somewhere spooky.
We should try to gather recent Jenkins runs where this is happening, here, to
see if there are patterns / we can get to the root cause.
Anyway, this happened to me on my old beast box, which runs the nightly ant
test times graphs:
http://people.apache.org/~mikemccand/lucenebench/antcleantest.html
The 2015/06/27 data point is missing because it failed with this timeout:
{noformat}
[junit4] Suite: org.apache.lucene.search.TestDocValuesRewriteMethod
[junit4] 2 ??? 28, 2015 7:01:29 ?
com.carrotsearch.randomizedtesting.ThreadLeakControl$2 evaluate
[junit4] 2 WARNING: Suite execution timed out:
org.apache.lucene.search.TestDocValuesRewriteMethod
[junit4] 21) Thread[id=1, name=main, state=WAITING, group=main]
[junit4] 2 at java.lang.Object.wait(Native Method)
[junit4] 2 at java.lang.Thread.join(Thread.java:1245)
[junit4] 2 at java.lang.Thread.join(Thread.java:1319)
[junit4] 2 at
com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:578)
[junit4] 2 at
com.carrotsearch.randomizedtesting.RandomizedRunner.run(RandomizedRunner.java:444)
[junit4] 2 at
com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.execute(SlaveMain.java:199)
[junit4] 2 at
com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.main(SlaveMain.java:310)
[junit4] 2 at
com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe.main(SlaveMainSafe.java:12)
[junit4] 22) Thread[id=213,
name=TEST-TestDocValuesRewriteMethod.testRegexps-seed#[C2DDF486BB909D8],
state=RUNNABLE, group=TGRP-TestDocValuesRewriteMethod]
[junit4] 2 at
org.apache.lucene.util.automaton.Operations.getLiveStates(Operations.java:900)
[junit4] 2 at
org.apache.lucene.util.automaton.Operations.hasDeadStates(Operations.java:389)
[junit4] 2 at
org.apache.lucene.util.automaton.Automata.makeString(Automata.java:517)
[junit4] 2 at
org.apache.lucene.util.automaton.RegExp.toAutomatonInternal(RegExp.java:579)
[junit4] 2 at
org.apache.lucene.util.automaton.RegExp.findLeaves(RegExp.java:617)
[junit4] 2 at
[jira] [Commented] (LUCENE-6607) Move geo3d to Lucene's sandbox module
[ https://issues.apache.org/jira/browse/LUCENE-6607?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14604666#comment-14604666 ] Karl Wright commented on LUCENE-6607: - My 2 cents: I would love to get geo3d in Mike's hands so he can do BKD tree development on it. But I don't want to sacrifice the functional spatial4j integration either. I don't know whether lucene modules have the capability for cross-dependency or not. If not, maybe we have to think of another way. Move geo3d to Lucene's sandbox module - Key: LUCENE-6607 URL: https://issues.apache.org/jira/browse/LUCENE-6607 Project: Lucene - Core Issue Type: Improvement Reporter: Michael McCandless Assignee: Michael McCandless Fix For: 5.3, Trunk Geo3d is a powerful low-level geo API, recording places on the earth's surface in the index in three dimensions (as 3 separate numbers) and offering fast shape intersection/distance testing at search time. [~daddywri] originally contributed this in LUCENE-6196, and we put it in spatial module, but I think a more natural place for it, for now anyway, is Lucene's sandbox module: it's very new, its APIs/abstractions are very much in flux (and the higher standards for abstractions in the spatial module cause disagreements: LUCENE-6578), [~daddywri] and others could iterate faster on changes in sandbox, etc. This would also un-block issues like LUCENE-6480, allowing GeoPointField and BKD trees to also use geo3d. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Solr-trunk-Linux (32bit/jdk1.8.0_60-ea-b12) - Build # 13239 - Failure!
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/13239/
Java: 32bit/jdk1.8.0_60-ea-b12 -server -XX:+UseConcMarkSweepGC
1 tests failed.
FAILED: org.apache.solr.search.TestSearcherReuse.test
Error Message:
expected same:Searcher@818ee2[collection1]
main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(6.0.0):c2)
Uninverting(_1(6.0.0):c1)))} was not:Searcher@183ef4f[collection1]
main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(6.0.0):c2)
Uninverting(_1(6.0.0):c1)))}
Stack Trace:
java.lang.AssertionError: expected same:Searcher@818ee2[collection1]
main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(6.0.0):c2)
Uninverting(_1(6.0.0):c1)))} was not:Searcher@183ef4f[collection1]
main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(6.0.0):c2)
Uninverting(_1(6.0.0):c1)))}
at
__randomizedtesting.SeedInfo.seed([8568DF22D9215D40:D3CE0F877DD30B8]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotSame(Assert.java:641)
at org.junit.Assert.assertSame(Assert.java:580)
at org.junit.Assert.assertSame(Assert.java:593)
at
org.apache.solr.search.TestSearcherReuse.assertSearcherHasNotChanged(TestSearcherReuse.java:247)
at
org.apache.solr.search.TestSearcherReuse.test(TestSearcherReuse.java:117)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
at
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
at
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
at
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
at
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
at
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at
[jira] [Commented] (LUCENE-6607) Move geo3d to Lucene's sandbox module
[ https://issues.apache.org/jira/browse/LUCENE-6607?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14604617#comment-14604617 ] Michael McCandless commented on LUCENE-6607: bq. Mike, why in your view is it not sufficient to mark APIs that are particularly subject to change with @lucene.experimental ? I think there are several compelling reasons to have large, new features in sandbox first, regardless of whether their eventual destination is core or misc or spatial or wherever. First, it sends a message to users that this is very new functionality, very subject to change, much more strongly than @lucene.experimental. Second, we are free to make drastic changes / iterations, and to have a lower bar that the API/abstractions are correct. When I see discussions like LUCENE-6578, where the standards for contributing new features to the spatial module are too high (in my opinion), sandbox is the perfect answer: we can't and shouldn't try get all abstractions right from the get-go. Third, it keeps our modules/functions better separated. If geo3d were in sandbox from the start, it would not need the external dependencies in the spatial module (spatial4j). Forth, for this particular case, I think it's interesting to explore integration of BKD tree and GeoPointField with Geo3d, which are also already in sandbox. Net/net I think it's a big win if we move geo3d to sandbox. Move geo3d to Lucene's sandbox module - Key: LUCENE-6607 URL: https://issues.apache.org/jira/browse/LUCENE-6607 Project: Lucene - Core Issue Type: Improvement Reporter: Michael McCandless Assignee: Michael McCandless Fix For: 5.3, Trunk Geo3d is a powerful low-level geo API, recording places on the earth's surface in the index in three dimensions (as 3 separate numbers) and offering fast shape intersection/distance testing at search time. [~daddywri] originally contributed this in LUCENE-6196, and we put it in spatial module, but I think a more natural place for it, for now anyway, is Lucene's sandbox module: it's very new, its APIs/abstractions are very much in flux (and the higher standards for abstractions in the spatial module cause disagreements: LUCENE-6578), [~daddywri] and others could iterate faster on changes in sandbox, etc. This would also un-block issues like LUCENE-6480, allowing GeoPointField and BKD trees to also use geo3d. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: FreeText Auto-suggest
Which documentation are you reading?
The analyzer you send to FreeTextSuggester should not make shingles
itself: the suggester does this internally, based on the grams
parameter.
Maybe look at the TestFreeTextSuggester unit test as an example?
Mike McCandless
http://blog.mikemccandless.com
On Sat, Jun 27, 2015 at 6:52 PM, Alessandro Benedetti
benedetti.ale...@gmail.com wrote:
Hi guys,
after reading the documentation for the FreetextSuggester I have some doubts
:
Actually the documentation is not clear enough.
Let's try to understand this suggester.
Building
This suggester build a FST that it will use to provide the autocomplete
feature running prefix searches on it .
The terms it uses to generate the FST are the tokens produced by the
suggestFreeTextAnalyzerFieldType .
And this should be correct.
So if we have a shingle token filter[1-3] ( we produce unigrams as well) in
our analysis to keep it simple , from these original field values :
mp3 ipod
mp3 player
mp3 player ipod
player of Real
- we produce these list of possible suggestions in our FST :
mp3
player
ipod
real
of
mp3 ipod
mp3 player
player ipod
player of
of real
mp3 player ipod
player of real
From the documentation I read :
ngrams: The max number of tokens out of which singles will be make the
dictionary. The default value is 2. Increasing this would mean you want more
than the previous 2 tokens to be taken into consideration when making the
suggestions.
This makes me confused, as I was not expecting this param to affect the
suggestion dictionary.
So I would like a clarification here from our masters :)
At this point let's see what happens at query time .
Query Time
As my understanding the ngrams params will consider the last N-1 tokens the
user put separated by the space separator.
Builds an ngram model from the text sent to {@link
* #build} and predicts based on the last grams-1 tokens in
* the request sent to {@link #lookup}. This tries to
* handle the long tail of suggestions for when the
* incoming query is a never before seen query string.
Example , grams=3 should consider only the last 2 tokens
special mp3 p - mp3 p
Then this query is analysed using the suggestFreeTextAnalyzerFieldType .
We produce 3 tokens :
mp3
p
mp3 p
And we run the prefix matching on the FST .
Conclusion
My understanding is wrong for sure at some point, as the behaviour I get is
different.
Can we discuss this , clarify this and eventually put it in the official
documentation ?
Cheers
--
--
Benedetti Alessandro
Visiting card : http://about.me/alessandro_benedetti
Tyger, tyger burning bright
In the forests of the night,
What immortal hand or eye
Could frame thy fearful symmetry?
William Blake - Songs of Experience -1794 England
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-6578) Geo3d: arcDistanceToShape() method may be useful
[ https://issues.apache.org/jira/browse/LUCENE-6578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14604668#comment-14604668 ] David Smiley commented on LUCENE-6578: -- Right. I'll post an update tonight hopefully. I worked on it last night some. Geo3d: arcDistanceToShape() method may be useful Key: LUCENE-6578 URL: https://issues.apache.org/jira/browse/LUCENE-6578 Project: Lucene - Core Issue Type: Bug Components: modules/spatial Reporter: Karl Wright Attachments: LUCENE-6578.patch, LUCENE-6578.patch, LUCENE-6578.patch I've got an application that seems like it may need the ability to compute a new kind of arc distance, from a GeoPoint to the nearest edge/point of a GeoShape. Adding this method to the interface, and corresponding implementations, would increase the utility of the package for ranking purposes. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-6595) CharFilter offsets correction is wonky
[
https://issues.apache.org/jira/browse/LUCENE-6595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14604599#comment-14604599
]
Michael McCandless commented on LUCENE-6595:
I think the API change here is necessary, but maybe we can minimize it?
E.g., can we fix the existing BaseCharFilter.addOffCorrectMap method to forward
to the new one that now takes an inputOffset? And can it just pass {{off}} as
the inputOffset (instead of filling with 0)?
I think we may not need the new method BaseCharFilter.correctEnd, but we do
need Tokenizer.correctEndOffset, but can we just implement it as LUCENE-5734
proposed ({{correctOffset(endOffset-1)+1}})?
CharFilter offsets correction is wonky
--
Key: LUCENE-6595
URL: https://issues.apache.org/jira/browse/LUCENE-6595
Project: Lucene - Core
Issue Type: Bug
Reporter: Michael McCandless
Attachments: LUCENE-6595.patch, LUCENE-6595.patch
Spinoff from this original Elasticsearch issue:
https://github.com/elastic/elasticsearch/issues/11726
If I make a MappingCharFilter with these mappings:
{noformat}
( -
) -
{noformat}
i.e., just erase left and right paren, then tokenizing the string
(F31) with e.g. WhitespaceTokenizer, produces a single token F31,
with start offset 1 (good).
But for its end offset, I would expect/want 4, but it produces 5
today.
This can be easily explained given how the mapping works: each time a
mapping rule matches, we update the cumulative offset difference,
conceptually as an array like this (it's encoded more compactly):
{noformat}
Output offset: 0 1 2 3
Input offset: 1 2 3 5
{noformat}
When the tokenizer produces F31, it assigns it startOffset=0 and
endOffset=3 based on the characters it sees (F, 3, 1). It then asks
the CharFilter to correct those offsets, mapping them backwards
through the above arrays, which creates startOffset=1 (good) and
endOffset=5 (bad).
At first, to fix this, I thought this is an off-by-1 and when
correcting the endOffset we really should return
1+correct(outputEndOffset-1), which would return the correct value (4)
here.
But that's too naive, e.g. here's another example:
{noformat}
- cc
{noformat}
If I then tokenize , today we produce the correct offsets (0, 4)
but if we do this off-by-1 fix for endOffset, we would get the wrong
endOffset (2).
I'm not sure what to do here...
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Solr-trunk-Windows (32bit/jdk1.8.0_45) - Build # 4975 - Failure!
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Windows/4975/ Java: 32bit/jdk1.8.0_45 -client -XX:+UseSerialGC 1 tests failed. FAILED: org.apache.solr.schema.TestCloudManagedSchemaConcurrent.test Error Message: Captured an uncaught exception in thread: Thread[id=5882, name=Thread-1343, state=RUNNABLE, group=TGRP-TestCloudManagedSchemaConcurrent] Stack Trace: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=5882, name=Thread-1343, state=RUNNABLE, group=TGRP-TestCloudManagedSchemaConcurrent] Caused by: java.lang.RuntimeException: org.apache.http.NoHttpResponseException: 127.0.0.1:58055 failed to respond at __randomizedtesting.SeedInfo.seed([77B84679C0006905]:0) at org.apache.solr.schema.TestCloudManagedSchemaConcurrent$PutFieldTypeThread.run(TestCloudManagedSchemaConcurrent.java:566) Caused by: org.apache.http.NoHttpResponseException: 127.0.0.1:58055 failed to respond at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:143) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261) at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:283) at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:251) at org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:197) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124) at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:685) at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:487) at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107) at org.apache.solr.util.RestTestHarness.getResponse(RestTestHarness.java:208) at org.apache.solr.util.RestTestHarness.put(RestTestHarness.java:120) at org.apache.solr.schema.TestCloudManagedSchemaConcurrent.addFieldTypePut(TestCloudManagedSchemaConcurrent.java:161) at org.apache.solr.schema.TestCloudManagedSchemaConcurrent.access$600(TestCloudManagedSchemaConcurrent.java:45) at org.apache.solr.schema.TestCloudManagedSchemaConcurrent$PutFieldTypeThread.run(TestCloudManagedSchemaConcurrent.java:563) Build Log: [...truncated 10004 lines...] [junit4] Suite: org.apache.solr.schema.TestCloudManagedSchemaConcurrent [junit4] 2 Creating dataDir: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.schema.TestCloudManagedSchemaConcurrent_77B84679C0006905-001\init-core-data-001 [junit4] 2 577089 INFO (SUITE-TestCloudManagedSchemaConcurrent-seed#[77B84679C0006905]-worker) [] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) [junit4] 2 577089 INFO (SUITE-TestCloudManagedSchemaConcurrent-seed#[77B84679C0006905]-worker) [] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /t_ck/r [junit4] 2 577093 INFO (TEST-TestCloudManagedSchemaConcurrent.test-seed#[77B84679C0006905]) [] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER [junit4] 2 577093 INFO (Thread-1180) [] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0 [junit4] 2 577093 INFO (Thread-1180) [] o.a.s.c.ZkTestServer Starting server [junit4] 2 577188 INFO (TEST-TestCloudManagedSchemaConcurrent.test-seed#[77B84679C0006905]) [] o.a.s.c.ZkTestServer start zk server on port:58024 [junit4] 2 577188 INFO (TEST-TestCloudManagedSchemaConcurrent.test-seed#[77B84679C0006905]) [] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider [junit4] 2 577189 INFO (TEST-TestCloudManagedSchemaConcurrent.test-seed#[77B84679C0006905]) [] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper [junit4] 2 577209 INFO (zkCallback-876-thread-1) [] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@2c7d80 name:ZooKeeperConnection Watcher:127.0.0.1:58024 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2 577209 INFO (TEST-TestCloudManagedSchemaConcurrent.test-seed#[77B84679C0006905]) [] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper [junit4] 2 577209 INFO
[JENKINS] Lucene-Solr-trunk-MacOSX (64bit/jdk1.8.0) - Build # 2463 - Still Failing!
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-MacOSX/2463/ Java: 64bit/jdk1.8.0 -XX:+UseCompressedOops -XX:+UseConcMarkSweepGC 1 tests failed. FAILED: org.apache.solr.cloud.CollectionsAPIAsyncDistributedZkTest.testSolrJAPICalls Error Message: Shard split did not complete. Last recorded state: running expected:[completed] but was:[running] Stack Trace: org.junit.ComparisonFailure: Shard split did not complete. Last recorded state: running expected:[completed] but was:[running] at __randomizedtesting.SeedInfo.seed([B9D166BF96B55A52:E1B5EADE90DFF286]:0) at org.junit.Assert.assertEquals(Assert.java:125) at org.apache.solr.cloud.CollectionsAPIAsyncDistributedZkTest.testSolrJAPICalls(CollectionsAPIAsyncDistributedZkTest.java:83) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:497) at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627) at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836) at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872) at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886) at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:960) at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:935) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57) at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365) at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798) at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458) at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845) at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747) at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781) at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65) at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at
[JENKINS] Lucene-Solr-NightlyTests-trunk - Build # 725 - Still Failing
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-trunk/725/ 2 tests failed. FAILED: org.apache.solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest.test Error Message: shard2 is not consistent. Got 804 from http://127.0.0.1:47293/g_/e/collection1lastClient and got 234 from http://127.0.0.1:54250/g_/e/collection1 Stack Trace: java.lang.AssertionError: shard2 is not consistent. Got 804 from http://127.0.0.1:47293/g_/e/collection1lastClient and got 234 from http://127.0.0.1:54250/g_/e/collection1 at __randomizedtesting.SeedInfo.seed([FA71FEA7F6F8C869:7225C17D5804A591]:0) at org.junit.Assert.fail(Assert.java:93) at org.apache.solr.cloud.AbstractFullDistribZkTestBase.checkShardConsistency(AbstractFullDistribZkTestBase.java:1244) at org.apache.solr.cloud.AbstractFullDistribZkTestBase.checkShardConsistency(AbstractFullDistribZkTestBase.java:1223) at org.apache.solr.cloud.ChaosMonkeySafeLeaderTest.test(ChaosMonkeySafeLeaderTest.java:165) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:483) at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627) at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836) at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872) at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886) at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:960) at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:935) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57) at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365) at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798) at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458) at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845) at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747) at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781) at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
[jira] [Resolved] (LUCENE-6524) Create an IndexWriter from an already opened NRT or non-NRT reader
[ https://issues.apache.org/jira/browse/LUCENE-6524?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael McCandless resolved LUCENE-6524. Resolution: Fixed Create an IndexWriter from an already opened NRT or non-NRT reader -- Key: LUCENE-6524 URL: https://issues.apache.org/jira/browse/LUCENE-6524 Project: Lucene - Core Issue Type: Improvement Reporter: Michael McCandless Assignee: Michael McCandless Fix For: 5.3, Trunk Attachments: LUCENE-6524.patch, LUCENE-6524.patch, LUCENE-6524.patch I'd like to add a new ctor to IndexWriter, letting you start from an already opened NRT or non-NRT DirectoryReader. I think this is a long missing API in Lucene today, and we've talked in the past about different ways to fix it e.g. factoring out a shared reader pool between writer and reader. One use-case, which I hit in LUCENE-5376: if you have a read-only index, so you've opened a non-NRT DirectoryReader to search it, and then you want to upgrade to a read/write index, we don't handle that very gracefully now because you are forced to open 2X the SegmentReaders. But with this API, IW populates its reader pool with the incoming SegmentReaders so they are shared on any subsequent NRT reopens / segment merging / deletes applying, etc. Another (more expert) use case is allowing rollback to an NRT-point. Today, you can only rollback to a commit point (segments_N). But an NRT reader also reflects a valid point in time view of the index (it just doesn't have a segments_N file, and its ref'd files are not fsync'd), so with this change you can close your old writer, open a new one from this NRT point, and revert all changes that had been done after the NRT reader was opened from the old writer. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-7733) remove/rename optimize references in the UI.
[ https://issues.apache.org/jira/browse/SOLR-7733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erick Erickson updated SOLR-7733: - Attachment: SOLR-7733.patch Here's a patch. Given how novice-like I am here, it's simply patterning on other elements in the two files, no clue whether this is correct but it _looks_ OK. remove/rename optimize references in the UI. -- Key: SOLR-7733 URL: https://issues.apache.org/jira/browse/SOLR-7733 Project: Solr Issue Type: Improvement Components: UI Affects Versions: 5.3, Trunk Reporter: Erick Erickson Priority: Minor Attachments: SOLR-7733.patch Since optimizing indexes is kind of a special circumstance thing, what do we think about removing (or renaming) optimize-related stuff on the core admin and core overview pages? The optimize button is already gone from the core admin screen (was this intentional?). My personal feeling is that we should remove this entirely as it's too easy to think Of course I want my index optimized and look, this screen says my index isn't optimized, that must mean I should optimize it. The core admin screen and the core overview page both have an optimized checkmark, I propose just removing it from the overview page and on the core admin page changing it to Segment Count #. NOTE: the overview page already has a Segment Count entry. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-7714) reduce SearchHandler's overall use of ShardHandler objects (from N+1+x to just 1)
[ https://issues.apache.org/jira/browse/SOLR-7714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14604749#comment-14604749 ] Ramkumar Aiyengar commented on SOLR-7714: - If there are no other objections, I will add a note to CHANGES.txt mentioning this in the upgrade notes, hopefully that should help.. reduce SearchHandler's overall use of ShardHandler objects (from N+1+x to just 1) - Key: SOLR-7714 URL: https://issues.apache.org/jira/browse/SOLR-7714 Project: Solr Issue Type: Improvement Reporter: Christine Poerschke Assignee: Ramkumar Aiyengar Priority: Minor Fix For: 5.3 github pull request with proposed change to follow. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-6568) rat-sources-typedef doesn't respect ivy configuration
[
https://issues.apache.org/jira/browse/LUCENE-6568?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14604759#comment-14604759
]
ASF subversion and git services commented on LUCENE-6568:
-
Commit 1688023 from [~andyetitmoves] in branch 'dev/branches/branch_5x'
[ https://svn.apache.org/r1688023 ]
LUCENE-6568: Make rat invocation depend on ivy configuration being set up
rat-sources-typedef doesn't respect ivy configuration
-
Key: LUCENE-6568
URL: https://issues.apache.org/jira/browse/LUCENE-6568
Project: Lucene - Core
Issue Type: Bug
Reporter: Ramkumar Aiyengar
Assignee: Ramkumar Aiyengar
Priority: Minor
rat-sources-typedef invoked as a part of precommit doesn't respect settings
in {{lucene/ivy-settings.xml}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-6568) rat-sources-typedef doesn't respect ivy configuration
[
https://issues.apache.org/jira/browse/LUCENE-6568?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14604758#comment-14604758
]
ASF subversion and git services commented on LUCENE-6568:
-
Commit 1688022 from [~andyetitmoves] in branch 'dev/trunk'
[ https://svn.apache.org/r1688022 ]
LUCENE-6568: Make rat invocation depend on ivy configuration being set up
rat-sources-typedef doesn't respect ivy configuration
-
Key: LUCENE-6568
URL: https://issues.apache.org/jira/browse/LUCENE-6568
Project: Lucene - Core
Issue Type: Bug
Reporter: Ramkumar Aiyengar
Assignee: Ramkumar Aiyengar
Priority: Minor
rat-sources-typedef invoked as a part of precommit doesn't respect settings
in {{lucene/ivy-settings.xml}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-6595) CharFilter offsets correction is wonky
[
https://issues.apache.org/jira/browse/LUCENE-6595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14604729#comment-14604729
]
Cao Manh Dat commented on LUCENE-6595:
--
{quote}
Any HTML entity that maps to empty string (e.g. em, /em, b, etc., I
think?) would not be included within the output token's start/endOffset, unless
that entity was inside a token.
{quote}
I think it will not a problem because we only ask for start/end offset of a
token.
CharFilter offsets correction is wonky
--
Key: LUCENE-6595
URL: https://issues.apache.org/jira/browse/LUCENE-6595
Project: Lucene - Core
Issue Type: Bug
Reporter: Michael McCandless
Attachments: LUCENE-6595.patch, LUCENE-6595.patch
Spinoff from this original Elasticsearch issue:
https://github.com/elastic/elasticsearch/issues/11726
If I make a MappingCharFilter with these mappings:
{noformat}
( -
) -
{noformat}
i.e., just erase left and right paren, then tokenizing the string
(F31) with e.g. WhitespaceTokenizer, produces a single token F31,
with start offset 1 (good).
But for its end offset, I would expect/want 4, but it produces 5
today.
This can be easily explained given how the mapping works: each time a
mapping rule matches, we update the cumulative offset difference,
conceptually as an array like this (it's encoded more compactly):
{noformat}
Output offset: 0 1 2 3
Input offset: 1 2 3 5
{noformat}
When the tokenizer produces F31, it assigns it startOffset=0 and
endOffset=3 based on the characters it sees (F, 3, 1). It then asks
the CharFilter to correct those offsets, mapping them backwards
through the above arrays, which creates startOffset=1 (good) and
endOffset=5 (bad).
At first, to fix this, I thought this is an off-by-1 and when
correcting the endOffset we really should return
1+correct(outputEndOffset-1), which would return the correct value (4)
here.
But that's too naive, e.g. here's another example:
{noformat}
- cc
{noformat}
If I then tokenize , today we produce the correct offsets (0, 4)
but if we do this off-by-1 fix for endOffset, we would get the wrong
endOffset (2).
I'm not sure what to do here...
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (SOLR-7733) remove/rename optimize references in the UI.
Erick Erickson created SOLR-7733: Summary: remove/rename optimize references in the UI. Key: SOLR-7733 URL: https://issues.apache.org/jira/browse/SOLR-7733 Project: Solr Issue Type: Improvement Components: UI Affects Versions: 5.3, Trunk Reporter: Erick Erickson Priority: Minor Since optimizing indexes is kind of a special circumstance thing, what do we think about removing (or renaming) optimize-related stuff on the core admin and core overview pages? The optimize button is already gone from the core admin screen (was this intentional?). My personal feeling is that we should remove this entirely as it's too easy to think Of course I want my index optimized and look, this screen says my index isn't optimized, that must mean I should optimize it. The core admin screen and the core overview page both have an optimized checkmark, I propose just removing it from the overview page and on the core admin page changing it to Segment Count #. NOTE: the overview page already has a Segment Count entry. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-7733) remove/rename optimize references in the UI.
[ https://issues.apache.org/jira/browse/SOLR-7733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14604762#comment-14604762 ] Shawn Heisey commented on SOLR-7733: +1. I also think that the red color when it's not optimized tends to reinforce the notion that the index is in a bad state that the user must correct. I still optimize my indexes on a relatively infrequent basis, and would hate to lose the ability to do that, but it doesn't belong in the admin UI. Sometimes the performance consequences from clicking that optimize button are severe. With a large enough SolrCloud installation, the performance impact from a seemingly innocent button click could last for hours. remove/rename optimize references in the UI. -- Key: SOLR-7733 URL: https://issues.apache.org/jira/browse/SOLR-7733 Project: Solr Issue Type: Improvement Components: UI Affects Versions: 5.3, Trunk Reporter: Erick Erickson Priority: Minor Since optimizing indexes is kind of a special circumstance thing, what do we think about removing (or renaming) optimize-related stuff on the core admin and core overview pages? The optimize button is already gone from the core admin screen (was this intentional?). My personal feeling is that we should remove this entirely as it's too easy to think Of course I want my index optimized and look, this screen says my index isn't optimized, that must mean I should optimize it. The core admin screen and the core overview page both have an optimized checkmark, I propose just removing it from the overview page and on the core admin page changing it to Segment Count #. NOTE: the overview page already has a Segment Count entry. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (LUCENE-6629) Random 7200 seconds build timeouts / infinite loops in Lucene tests?
Michael McCandless created LUCENE-6629:
--
Summary: Random 7200 seconds build timeouts / infinite loops in
Lucene tests?
Key: LUCENE-6629
URL: https://issues.apache.org/jira/browse/LUCENE-6629
Project: Lucene - Core
Issue Type: Bug
Reporter: Michael McCandless
I'm not sure what's going on here, but every so often a Jenkins run will fail
with a build timeout (7200 seconds) with stack traces that do not look like
deadlock. They never reproduce for me.
We really need to improve test infra here, so that each HEARTBEAT we also got
1) full thread stacks and 2) total CPU usage of the process as reported by the
ManagementBean APIs ... this would shed more light on whether the JVM is
somehow hung vs our bug (infinite loop). But infinite loop seems most likely
... the stacks always seem to be somewhere spooky.
We should try to gather recent Jenkins runs where this is happening, here, to
see if there are patterns / we can get to the root cause.
Anyway, this happened to me on my old beast box, which runs the nightly ant
test times graphs:
http://people.apache.org/~mikemccand/lucenebench/antcleantest.html
The 2015/06/27 data point is missing because it failed with this timeout:
{noformat}
[junit4] Suite: org.apache.lucene.search.TestDocValuesRewriteMethod
[junit4] 2 ??? 28, 2015 7:01:29 ?
com.carrotsearch.randomizedtesting.ThreadLeakControl$2 evaluate
[junit4] 2 WARNING: Suite execution timed out:
org.apache.lucene.search.TestDocValuesRewriteMethod
[junit4] 21) Thread[id=1, name=main, state=WAITING, group=main]
[junit4] 2 at java.lang.Object.wait(Native Method)
[junit4] 2 at java.lang.Thread.join(Thread.java:1245)
[junit4] 2 at java.lang.Thread.join(Thread.java:1319)
[junit4] 2 at
com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:578)
[junit4] 2 at
com.carrotsearch.randomizedtesting.RandomizedRunner.run(RandomizedRunner.java:444)
[junit4] 2 at
com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.execute(SlaveMain.java:199)
[junit4] 2 at
com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.main(SlaveMain.java:310)
[junit4] 2 at
com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe.main(SlaveMainSafe.java:12)
[junit4] 22) Thread[id=213,
name=TEST-TestDocValuesRewriteMethod.testRegexps-seed#[C2DDF486BB909D8],
state=RUNNABLE, group=TGRP-TestDocValuesRewriteMethod]
[junit4] 2 at
org.apache.lucene.util.automaton.Operations.getLiveStates(Operations.java:900)
[junit4] 2 at
org.apache.lucene.util.automaton.Operations.hasDeadStates(Operations.java:389)
[junit4] 2 at
org.apache.lucene.util.automaton.Automata.makeString(Automata.java:517)
[junit4] 2 at
org.apache.lucene.util.automaton.RegExp.toAutomatonInternal(RegExp.java:579)
[junit4] 2 at
org.apache.lucene.util.automaton.RegExp.findLeaves(RegExp.java:617)
[junit4] 2 at
org.apache.lucene.util.automaton.RegExp.toAutomatonInternal(RegExp.java:519)
[junit4] 2 at
org.apache.lucene.util.automaton.RegExp.findLeaves(RegExp.java:617)
[junit4] 2 at
org.apache.lucene.util.automaton.RegExp.toAutomatonInternal(RegExp.java:510)
[junit4] 2 at
org.apache.lucene.util.automaton.RegExp.toAutomaton(RegExp.java:495)
[junit4] 2 at
org.apache.lucene.util.automaton.RegExp.toAutomaton(RegExp.java:466)
[junit4] 2 at
org.apache.lucene.search.RegexpQuery.init(RegexpQuery.java:109)
[junit4] 2 at
org.apache.lucene.search.RegexpQuery.init(RegexpQuery.java:79)
[junit4] 2 at
org.apache.lucene.search.TestDocValuesRewriteMethod.assertSame(TestDocValuesRewriteMethod.java:117)
[junit4] 2 at
org.apache.lucene.search.TestDocValuesRewriteMethod.testRegexps(TestDocValuesRewriteMethod.java:109)
[junit4] 2 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
Method)
[junit4] 2 at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[junit4] 2 at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4] 2 at java.lang.reflect.Method.invoke(Method.java:497)
[junit4] 2 at
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
[junit4] 2 at
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
[junit4] 2 at
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
[junit4] 2 at
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
[junit4] 2 at
[jira] [Commented] (LUCENE-6629) Random 7200 seconds build timeouts / infinite loops in Lucene tests?
[
https://issues.apache.org/jira/browse/LUCENE-6629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14604653#comment-14604653
]
Robert Muir commented on LUCENE-6629:
-
I don't think that one is us. Those vocabulary asserts just stem the same .zip
files always. its deterministic and a rather small (maybe 100k) amount of
strings, same ones tested every time.
Random 7200 seconds build timeouts / infinite loops in Lucene tests?
Key: LUCENE-6629
URL: https://issues.apache.org/jira/browse/LUCENE-6629
Project: Lucene - Core
Issue Type: Bug
Reporter: Michael McCandless
I'm not sure what's going on here, but every so often a Jenkins run will fail
with a build timeout (7200 seconds) with stack traces that do not look like
deadlock. They never reproduce for me.
We really need to improve test infra here, so that each HEARTBEAT we also got
1) full thread stacks and 2) total CPU usage of the process as reported by
the ManagementBean APIs ... this would shed more light on whether the JVM is
somehow hung vs our bug (infinite loop). But infinite loop seems most likely
... the stacks always seem to be somewhere spooky.
We should try to gather recent Jenkins runs where this is happening, here, to
see if there are patterns / we can get to the root cause.
Anyway, this happened to me on my old beast box, which runs the nightly ant
test times graphs:
http://people.apache.org/~mikemccand/lucenebench/antcleantest.html
The 2015/06/27 data point is missing because it failed with this timeout:
{noformat}
[junit4] Suite: org.apache.lucene.search.TestDocValuesRewriteMethod
[junit4] 2 ??? 28, 2015 7:01:29 ?
com.carrotsearch.randomizedtesting.ThreadLeakControl$2 evaluate
[junit4] 2 WARNING: Suite execution timed out:
org.apache.lucene.search.TestDocValuesRewriteMethod
[junit4] 21) Thread[id=1, name=main, state=WAITING, group=main]
[junit4] 2 at java.lang.Object.wait(Native Method)
[junit4] 2 at java.lang.Thread.join(Thread.java:1245)
[junit4] 2 at java.lang.Thread.join(Thread.java:1319)
[junit4] 2 at
com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:578)
[junit4] 2 at
com.carrotsearch.randomizedtesting.RandomizedRunner.run(RandomizedRunner.java:444)
[junit4] 2 at
com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.execute(SlaveMain.java:199)
[junit4] 2 at
com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.main(SlaveMain.java:310)
[junit4] 2 at
com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe.main(SlaveMainSafe.java:12)
[junit4] 22) Thread[id=213,
name=TEST-TestDocValuesRewriteMethod.testRegexps-seed#[C2DDF486BB909D8],
state=RUNNABLE, group=TGRP-TestDocValuesRewriteMethod]
[junit4] 2 at
org.apache.lucene.util.automaton.Operations.getLiveStates(Operations.java:900)
[junit4] 2 at
org.apache.lucene.util.automaton.Operations.hasDeadStates(Operations.java:389)
[junit4] 2 at
org.apache.lucene.util.automaton.Automata.makeString(Automata.java:517)
[junit4] 2 at
org.apache.lucene.util.automaton.RegExp.toAutomatonInternal(RegExp.java:579)
[junit4] 2 at
org.apache.lucene.util.automaton.RegExp.findLeaves(RegExp.java:617)
[junit4] 2 at
org.apache.lucene.util.automaton.RegExp.toAutomatonInternal(RegExp.java:519)
[junit4] 2 at
org.apache.lucene.util.automaton.RegExp.findLeaves(RegExp.java:617)
[junit4] 2 at
org.apache.lucene.util.automaton.RegExp.toAutomatonInternal(RegExp.java:510)
[junit4] 2 at
org.apache.lucene.util.automaton.RegExp.toAutomaton(RegExp.java:495)
[junit4] 2 at
org.apache.lucene.util.automaton.RegExp.toAutomaton(RegExp.java:466)
[junit4] 2 at
org.apache.lucene.search.RegexpQuery.init(RegexpQuery.java:109)
[junit4] 2 at
org.apache.lucene.search.RegexpQuery.init(RegexpQuery.java:79)
[junit4] 2 at
org.apache.lucene.search.TestDocValuesRewriteMethod.assertSame(TestDocValuesRewriteMethod.java:117)
[junit4] 2 at
org.apache.lucene.search.TestDocValuesRewriteMethod.testRegexps(TestDocValuesRewriteMethod.java:109)
[junit4] 2 at
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4] 2 at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[junit4] 2 at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4] 2 at java.lang.reflect.Method.invoke(Method.java:497)
[junit4] 2 at
[JENKINS] Lucene-Solr-trunk-Linux (64bit/jdk1.8.0_45) - Build # 13241 - Failure!
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/13241/ Java: 64bit/jdk1.8.0_45 -XX:-UseCompressedOops -XX:+UseParallelGC 3 tests failed. FAILED: junit.framework.TestSuite.org.apache.solr.cloud.SolrCloudExampleTest Error Message: ERROR: SolrIndexSearcher opens=26 closes=25 Stack Trace: java.lang.AssertionError: ERROR: SolrIndexSearcher opens=26 closes=25 at __randomizedtesting.SeedInfo.seed([BE6514A6D5D48B5F]:0) at org.junit.Assert.fail(Assert.java:93) at org.apache.solr.SolrTestCaseJ4.endTrackingSearchers(SolrTestCaseJ4.java:472) at org.apache.solr.SolrTestCaseJ4.afterClass(SolrTestCaseJ4.java:232) at sun.reflect.GeneratedMethodAccessor39.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:497) at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627) at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:799) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65) at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365) at java.lang.Thread.run(Thread.java:745) FAILED: junit.framework.TestSuite.org.apache.solr.cloud.SolrCloudExampleTest Error Message: 1 thread leaked from SUITE scope at org.apache.solr.cloud.SolrCloudExampleTest: 1) Thread[id=6717, name=searcherExecutor-3292-thread-1, state=WAITING, group=TGRP-SolrCloudExampleTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Stack Trace: com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.cloud.SolrCloudExampleTest: 1) Thread[id=6717, name=searcherExecutor-3292-thread-1, state=WAITING, group=TGRP-SolrCloudExampleTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) at __randomizedtesting.SeedInfo.seed([BE6514A6D5D48B5F]:0) FAILED: junit.framework.TestSuite.org.apache.solr.cloud.SolrCloudExampleTest Error Message: There are still zombie threads that
[jira] [Commented] (LUCENE-6578) Geo3d: arcDistanceToShape() method may be useful
[ https://issues.apache.org/jira/browse/LUCENE-6578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14604664#comment-14604664 ] Karl Wright commented on LUCENE-6578: - So no need to submit another patch? Geo3d: arcDistanceToShape() method may be useful Key: LUCENE-6578 URL: https://issues.apache.org/jira/browse/LUCENE-6578 Project: Lucene - Core Issue Type: Bug Components: modules/spatial Reporter: Karl Wright Attachments: LUCENE-6578.patch, LUCENE-6578.patch, LUCENE-6578.patch I've got an application that seems like it may need the ability to compute a new kind of arc distance, from a GeoPoint to the nearest edge/point of a GeoShape. Adding this method to the interface, and corresponding implementations, would increase the utility of the package for ranking purposes. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-6630) Further improve memory of FieldInfos
[
https://issues.apache.org/jira/browse/LUCENE-6630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14604864#comment-14604864
]
Robert Muir commented on LUCENE-6630:
-
{quote}
Maybe we should always just be sparse, unless it's complete dense?
{quote}
-1
Don't even think about it.
When I said I wouldn't let abuse cases hurt real use cases, did you think I was
joking? I'm not going to let real users have shitty performance here.
Further improve memory of FieldInfos
Key: LUCENE-6630
URL: https://issues.apache.org/jira/browse/LUCENE-6630
Project: Lucene - Core
Issue Type: Bug
Reporter: Ryan Ernst
Attachments: LUCENE-6630.patch
In LUCENE-6325, FieldInfos were improved to use an array except in very
sparse cases. In the sparse cases, a TreeMap is still used. However, even in
the sparse case, we can still use an array, and then use binary search. The
perf will be equivalent, but memory use should be much less.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-7733) remove/rename optimize references in the UI.
[ https://issues.apache.org/jira/browse/SOLR-7733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14604873#comment-14604873 ] Upayavira commented on SOLR-7733: - If you were gonna go a little further, you could look in the controller and see if there's any logic preparing data for the view that isn't needed anymore. remove/rename optimize references in the UI. -- Key: SOLR-7733 URL: https://issues.apache.org/jira/browse/SOLR-7733 Project: Solr Issue Type: Improvement Components: UI Affects Versions: 5.3, Trunk Reporter: Erick Erickson Priority: Minor Attachments: SOLR-7733.patch Since optimizing indexes is kind of a special circumstance thing, what do we think about removing (or renaming) optimize-related stuff on the core admin and core overview pages? The optimize button is already gone from the core admin screen (was this intentional?). My personal feeling is that we should remove this entirely as it's too easy to think Of course I want my index optimized and look, this screen says my index isn't optimized, that must mean I should optimize it. The core admin screen and the core overview page both have an optimized checkmark, I propose just removing it from the overview page and on the core admin page changing it to Segment Count #. NOTE: the overview page already has a Segment Count entry. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-6607) Move geo3d to Lucene's sandbox module
[ https://issues.apache.org/jira/browse/LUCENE-6607?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14604903#comment-14604903 ] Karl Wright commented on LUCENE-6607: - Fine; I just wasn't sure that was allowed. Move geo3d to Lucene's sandbox module - Key: LUCENE-6607 URL: https://issues.apache.org/jira/browse/LUCENE-6607 Project: Lucene - Core Issue Type: Improvement Reporter: Michael McCandless Assignee: Michael McCandless Fix For: 5.3, Trunk Geo3d is a powerful low-level geo API, recording places on the earth's surface in the index in three dimensions (as 3 separate numbers) and offering fast shape intersection/distance testing at search time. [~daddywri] originally contributed this in LUCENE-6196, and we put it in spatial module, but I think a more natural place for it, for now anyway, is Lucene's sandbox module: it's very new, its APIs/abstractions are very much in flux (and the higher standards for abstractions in the spatial module cause disagreements: LUCENE-6578), [~daddywri] and others could iterate faster on changes in sandbox, etc. This would also un-block issues like LUCENE-6480, allowing GeoPointField and BKD trees to also use geo3d. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: FreeText Auto-suggest
Thanks Mike !
Tomorrow I will read the details,
I was taking as guide the Solr Guide, that was not so clear in relation of
the type of Analyzer to provide the Suggester with.
Thanks for the support,
Tomorrow I will do some experiment and let you know!
Cheers
2015-06-28 11:48 GMT+01:00 Michael McCandless luc...@mikemccandless.com:
Which documentation are you reading?
The analyzer you send to FreeTextSuggester should not make shingles
itself: the suggester does this internally, based on the grams
parameter.
Maybe look at the TestFreeTextSuggester unit test as an example?
Mike McCandless
http://blog.mikemccandless.com
On Sat, Jun 27, 2015 at 6:52 PM, Alessandro Benedetti
benedetti.ale...@gmail.com wrote:
Hi guys,
after reading the documentation for the FreetextSuggester I have some
doubts
:
Actually the documentation is not clear enough.
Let's try to understand this suggester.
Building
This suggester build a FST that it will use to provide the autocomplete
feature running prefix searches on it .
The terms it uses to generate the FST are the tokens produced by the
suggestFreeTextAnalyzerFieldType .
And this should be correct.
So if we have a shingle token filter[1-3] ( we produce unigrams as well)
in
our analysis to keep it simple , from these original field values :
mp3 ipod
mp3 player
mp3 player ipod
player of Real
- we produce these list of possible suggestions in our FST :
mp3
player
ipod
real
of
mp3 ipod
mp3 player
player ipod
player of
of real
mp3 player ipod
player of real
From the documentation I read :
ngrams: The max number of tokens out of which singles will be make the
dictionary. The default value is 2. Increasing this would mean you want
more
than the previous 2 tokens to be taken into consideration when making
the
suggestions.
This makes me confused, as I was not expecting this param to affect the
suggestion dictionary.
So I would like a clarification here from our masters :)
At this point let's see what happens at query time .
Query Time
As my understanding the ngrams params will consider the last N-1 tokens
the
user put separated by the space separator.
Builds an ngram model from the text sent to {@link
* #build} and predicts based on the last grams-1 tokens in
* the request sent to {@link #lookup}. This tries to
* handle the long tail of suggestions for when the
* incoming query is a never before seen query string.
Example , grams=3 should consider only the last 2 tokens
special mp3 p - mp3 p
Then this query is analysed using the suggestFreeTextAnalyzerFieldType
.
We produce 3 tokens :
mp3
p
mp3 p
And we run the prefix matching on the FST .
Conclusion
My understanding is wrong for sure at some point, as the behaviour I get
is
different.
Can we discuss this , clarify this and eventually put it in the official
documentation ?
Cheers
--
--
Benedetti Alessandro
Visiting card : http://about.me/alessandro_benedetti
Tyger, tyger burning bright
In the forests of the night,
What immortal hand or eye
Could frame thy fearful symmetry?
William Blake - Songs of Experience -1794 England
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
--
--
Benedetti Alessandro
Visiting card : http://about.me/alessandro_benedetti
Tyger, tyger burning bright
In the forests of the night,
What immortal hand or eye
Could frame thy fearful symmetry?
William Blake - Songs of Experience -1794 England
[jira] [Commented] (SOLR-7733) remove/rename optimize references in the UI.
[ https://issues.apache.org/jira/browse/SOLR-7733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14604872#comment-14604872 ] Upayavira commented on SOLR-7733: - Patch looks good to me, go for it!! remove/rename optimize references in the UI. -- Key: SOLR-7733 URL: https://issues.apache.org/jira/browse/SOLR-7733 Project: Solr Issue Type: Improvement Components: UI Affects Versions: 5.3, Trunk Reporter: Erick Erickson Priority: Minor Attachments: SOLR-7733.patch Since optimizing indexes is kind of a special circumstance thing, what do we think about removing (or renaming) optimize-related stuff on the core admin and core overview pages? The optimize button is already gone from the core admin screen (was this intentional?). My personal feeling is that we should remove this entirely as it's too easy to think Of course I want my index optimized and look, this screen says my index isn't optimized, that must mean I should optimize it. The core admin screen and the core overview page both have an optimized checkmark, I propose just removing it from the overview page and on the core admin page changing it to Segment Count #. NOTE: the overview page already has a Segment Count entry. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-6630) Further improve memory of FieldInfos
[ https://issues.apache.org/jira/browse/LUCENE-6630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14604884#comment-14604884 ] Robert Muir commented on LUCENE-6630: - Also, the patch should: * Just use Integer's compare method to sort the array, no custom comparator needed. * Use arrays.binarysearch, not a handcoded binarysearch with overflow bugs. Use of the ternary operator is confusing here. Can the logic be simplified? Further improve memory of FieldInfos Key: LUCENE-6630 URL: https://issues.apache.org/jira/browse/LUCENE-6630 Project: Lucene - Core Issue Type: Bug Reporter: Ryan Ernst Attachments: LUCENE-6630.patch In LUCENE-6325, FieldInfos were improved to use an array except in very sparse cases. In the sparse cases, a TreeMap is still used. However, even in the sparse case, we can still use an array, and then use binary search. The perf will be equivalent, but memory use should be much less. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Solr-5.x-Linux (64bit/jdk1.8.0_45) - Build # 13068 - Failure!
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-5.x-Linux/13068/ Java: 64bit/jdk1.8.0_45 -XX:-UseCompressedOops -XX:+UseParallelGC 1 tests failed. FAILED: org.apache.lucene.analysis.charfilter.HTMLStripCharFilterTest.testUTF16Surrogates Error Message: unpaired high surrogate: d86c, followed by: e28f Stack Trace: java.lang.IllegalStateException: unpaired high surrogate: d86c, followed by: e28f at __randomizedtesting.SeedInfo.seed([4E6534FD881A4783:12258E280AF4A8B9]:0) at org.apache.lucene.analysis.MockTokenizer.failAlways(MockTokenizer.java:131) at org.apache.lucene.analysis.MockTokenizer.readCodePoint(MockTokenizer.java:216) at org.apache.lucene.analysis.MockTokenizer.incrementToken(MockTokenizer.java:150) at org.apache.lucene.analysis.BaseTokenStreamTestCase.checkResetException(BaseTokenStreamTestCase.java:400) at org.apache.lucene.analysis.BaseTokenStreamTestCase.assertAnalyzesTo(BaseTokenStreamTestCase.java:352) at org.apache.lucene.analysis.BaseTokenStreamTestCase.assertAnalyzesTo(BaseTokenStreamTestCase.java:362) at org.apache.lucene.analysis.charfilter.HTMLStripCharFilterTest.testUTF16Surrogates(HTMLStripCharFilterTest.java:607) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:497) at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627) at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836) at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872) at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886) at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365) at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798) at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458) at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845) at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747) at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781) at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65) at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365) at java.lang.Thread.run(Thread.java:745) Build Log: [...truncated 2774 lines...]
[jira] [Commented] (LUCENE-6607) Move geo3d to Lucene's sandbox module
[ https://issues.apache.org/jira/browse/LUCENE-6607?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14604900#comment-14604900 ] Michael McCandless commented on LUCENE-6607: bq. But I don't want to sacrifice the functional spatial4j integration either. The spatial module can depend on sandbox and use geo3d from there? Move geo3d to Lucene's sandbox module - Key: LUCENE-6607 URL: https://issues.apache.org/jira/browse/LUCENE-6607 Project: Lucene - Core Issue Type: Improvement Reporter: Michael McCandless Assignee: Michael McCandless Fix For: 5.3, Trunk Geo3d is a powerful low-level geo API, recording places on the earth's surface in the index in three dimensions (as 3 separate numbers) and offering fast shape intersection/distance testing at search time. [~daddywri] originally contributed this in LUCENE-6196, and we put it in spatial module, but I think a more natural place for it, for now anyway, is Lucene's sandbox module: it's very new, its APIs/abstractions are very much in flux (and the higher standards for abstractions in the spatial module cause disagreements: LUCENE-6578), [~daddywri] and others could iterate faster on changes in sandbox, etc. This would also un-block issues like LUCENE-6480, allowing GeoPointField and BKD trees to also use geo3d. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Solr-5.x-MacOSX (64bit/jdk1.7.0) - Build # 2416 - Failure!
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-5.x-MacOSX/2416/ Java: 64bit/jdk1.7.0 -XX:-UseCompressedOops -XX:+UseG1GC 1 tests failed. FAILED: org.apache.solr.cloud.ChaosMonkeySafeLeaderTest.test Error Message: shard1 is not consistent. Got 203 from https://127.0.0.1:60584/collection1lastClient and got 96 from https://127.0.0.1:60608/collection1 Stack Trace: java.lang.AssertionError: shard1 is not consistent. Got 203 from https://127.0.0.1:60584/collection1lastClient and got 96 from https://127.0.0.1:60608/collection1 at __randomizedtesting.SeedInfo.seed([E5D67A84E70E48F6:6D82455E49F2250E]:0) at org.junit.Assert.fail(Assert.java:93) at org.apache.solr.cloud.AbstractFullDistribZkTestBase.checkShardConsistency(AbstractFullDistribZkTestBase.java:1244) at org.apache.solr.cloud.AbstractFullDistribZkTestBase.checkShardConsistency(AbstractFullDistribZkTestBase.java:1223) at org.apache.solr.cloud.ChaosMonkeySafeLeaderTest.test(ChaosMonkeySafeLeaderTest.java:165) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627) at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836) at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872) at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886) at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:960) at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:935) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57) at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365) at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798) at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458) at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845) at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747) at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781) at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) at
[jira] [Commented] (SOLR-7727) bin/solr script incorrectly requires solr.xml in $SOLR_HOME
[
https://issues.apache.org/jira/browse/SOLR-7727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14604918#comment-14604918
]
Jan Høydahl commented on SOLR-7727:
---
Thinking more about this, an alternative solution is to change
{{SolrDispatchFilter}} to look for {{solr.xml}} in ZK by default in cloud mode,
and fallback to {{SOLR_HOME}} for back compat, getting rid of
{{-Dsolr.solrxml.location}}. Do you see any problems with such a solution?
Aside: Currently, if you start Solr in non-cloud mode with solrxml.location
specified, e.g. {{bin/solr start -Dsolr.solrxml.location=zookeeper}}, Jetty
starts up but in a bad state, instead of failing fast, but no error is given in
the start script. Trying to access Solr gives 500 error with nullpointer. This
would also be fixed with the code-change above, since loading solr.xml from ZK
would only be attempted when in ZK mode.
bin/solr script incorrectly requires solr.xml in $SOLR_HOME
---
Key: SOLR-7727
URL: https://issues.apache.org/jira/browse/SOLR-7727
Project: Solr
Issue Type: Bug
Components: scripts and tools
Affects Versions: 5.2.1
Reporter: Crawdaddy
Assignee: Jan Høydahl
Fix For: 5.3, Trunk
Attachments: SOLR-7727.patch
The bin/solr script currently requires solr.xml to exist in $SOLR_HOME, going
against SOLR-4718 which allows it to be hosted in Zookeeper.
Line 1219:
if [ ! -e $SOLR_HOME/solr.xml ]; then
echo -e \nSolr home directory $SOLR_HOME must contain a solr.xml file!\n
exit 1
fi
Commenting this out, and specifying the following in one's solr.in.sh, allows
solr.xml to once again be hosted in ZK:
SOLR_OPTS=$SOLR_OPTS -Dsolr.solrxml.location=zookeeper
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Solr-trunk-Windows (32bit/jdk1.8.0_45) - Build # 4977 - Failure!
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Windows/4977/ Java: 32bit/jdk1.8.0_45 -client -XX:+UseParallelGC 3 tests failed. FAILED: junit.framework.TestSuite.org.apache.solr.core.TestLazyCores Error Message: ERROR: SolrIndexSearcher opens=51 closes=50 Stack Trace: java.lang.AssertionError: ERROR: SolrIndexSearcher opens=51 closes=50 at __randomizedtesting.SeedInfo.seed([22E8D954893199FB]:0) at org.junit.Assert.fail(Assert.java:93) at org.apache.solr.SolrTestCaseJ4.endTrackingSearchers(SolrTestCaseJ4.java:472) at org.apache.solr.SolrTestCaseJ4.afterClass(SolrTestCaseJ4.java:232) at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:497) at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627) at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:799) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65) at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365) at java.lang.Thread.run(Thread.java:745) FAILED: junit.framework.TestSuite.org.apache.solr.core.TestLazyCores Error Message: 1 thread leaked from SUITE scope at org.apache.solr.core.TestLazyCores: 1) Thread[id=3477, name=searcherExecutor-1044-thread-1, state=WAITING, group=TGRP-TestLazyCores] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Stack Trace: com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.core.TestLazyCores: 1) Thread[id=3477, name=searcherExecutor-1044-thread-1, state=WAITING, group=TGRP-TestLazyCores] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) at __randomizedtesting.SeedInfo.seed([22E8D954893199FB]:0) FAILED: junit.framework.TestSuite.org.apache.solr.core.TestLazyCores Error Message: There are still zombie threads that couldn't be terminated:1) Thread[id=3477,
[jira] [Commented] (SOLR-7485) replace shards.info with ShardParams.SHARDS_INFO in TestTolerantSearch.java and CloudSolrClientTest.java, add ShardParamsTest.java
[ https://issues.apache.org/jira/browse/SOLR-7485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14604789#comment-14604789 ] ASF subversion and git services commented on SOLR-7485: --- Commit 1688027 from [~andyetitmoves] in branch 'dev/branches/branch_5x' [ https://svn.apache.org/r1688027 ] SOLR-7485: Replace shards.info occurrences with ShardParams.SHARDS_INFO replace shards.info with ShardParams.SHARDS_INFO in TestTolerantSearch.java and CloudSolrClientTest.java, add ShardParamsTest.java Key: SOLR-7485 URL: https://issues.apache.org/jira/browse/SOLR-7485 Project: Solr Issue Type: Improvement Reporter: Christine Poerschke Assignee: Ramkumar Aiyengar Priority: Minor various other tests already use ShardParams.SHARDS_INFO e.g. TestDistributedSearch.java -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-7710) replace various async with CommonAdminParams.ASYNC
[
https://issues.apache.org/jira/browse/SOLR-7710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14604837#comment-14604837
]
ASF subversion and git services commented on SOLR-7710:
---
Commit 1688039 from [~andyetitmoves] in branch 'dev/trunk'
[ https://svn.apache.org/r1688039 ]
SOLR-7710: Replace async occurrences with CommonAdminParams.ASYNC
replace various async with CommonAdminParams.ASYNC
Key: SOLR-7710
URL: https://issues.apache.org/jira/browse/SOLR-7710
Project: Solr
Issue Type: Wish
Reporter: Christine Poerschke
Assignee: Ramkumar Aiyengar
Priority: Minor
Attachments: SOLR-7710.patch
* github pull request with proposed changes to follow shortly.
* Motivation is addition of a test case for an unrelated change (not yet
uploaded) and not wishing to add literally {{async}} in that test case.
Using {{CommonAdminParams.ASYNC}} instead of async can also make code base
'navigation' a little easier e.g. locating of existing test cases.
* The proposed change also adds a little {{CommonAdminParamsTest}} to test
the backwards compatibility of CommonAdminParams parameter constants i.e. if
someone accidentally changes those constants then the test will flag that up.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-7485) replace shards.info with ShardParams.SHARDS_INFO in TestTolerantSearch.java and CloudSolrClientTest.java, add ShardParamsTest.java
[ https://issues.apache.org/jira/browse/SOLR-7485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14604788#comment-14604788 ] ASF subversion and git services commented on SOLR-7485: --- Commit 1688026 from [~andyetitmoves] in branch 'dev/trunk' [ https://svn.apache.org/r1688026 ] SOLR-7485: Add CHANGES.txt replace shards.info with ShardParams.SHARDS_INFO in TestTolerantSearch.java and CloudSolrClientTest.java, add ShardParamsTest.java Key: SOLR-7485 URL: https://issues.apache.org/jira/browse/SOLR-7485 Project: Solr Issue Type: Improvement Reporter: Christine Poerschke Assignee: Ramkumar Aiyengar Priority: Minor various other tests already use ShardParams.SHARDS_INFO e.g. TestDistributedSearch.java -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-7485) replace shards.info with ShardParams.SHARDS_INFO in TestTolerantSearch.java and CloudSolrClientTest.java, add ShardParamsTest.java
[ https://issues.apache.org/jira/browse/SOLR-7485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14604785#comment-14604785 ] ASF subversion and git services commented on SOLR-7485: --- Commit 1688025 from [~andyetitmoves] in branch 'dev/trunk' [ https://svn.apache.org/r1688025 ] SOLR-7485: Replace shards.info occurrences with ShardParams.SHARDS_INFO replace shards.info with ShardParams.SHARDS_INFO in TestTolerantSearch.java and CloudSolrClientTest.java, add ShardParamsTest.java Key: SOLR-7485 URL: https://issues.apache.org/jira/browse/SOLR-7485 Project: Solr Issue Type: Improvement Reporter: Christine Poerschke Assignee: Ramkumar Aiyengar Priority: Minor various other tests already use ShardParams.SHARDS_INFO e.g. TestDistributedSearch.java -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Solr-5.x-Linux (32bit/jdk1.8.0_45) - Build # 13066 - Failure!
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-5.x-Linux/13066/ Java: 32bit/jdk1.8.0_45 -server -XX:+UseParallelGC 1 tests failed. FAILED: org.apache.solr.client.solrj.impl.CloudSolrClientTest.test Error Message: Error from server at http://127.0.0.1:40960/_rgk/v/checkStateVerCol: no servers hosting shard: shard2 Stack Trace: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:40960/_rgk/v/checkStateVerCol: no servers hosting shard: shard2 at __randomizedtesting.SeedInfo.seed([43ABF7BDC95D9998:CBFFC86767A1F460]:0) at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:560) at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:234) at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:226) at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:135) at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:943) at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:958) at org.apache.solr.client.solrj.impl.CloudSolrClientTest.stateVersionParamTest(CloudSolrClientTest.java:543) at org.apache.solr.client.solrj.impl.CloudSolrClientTest.test(CloudSolrClientTest.java:116) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:497) at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627) at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836) at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872) at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886) at org.junit.rules.ExpectedException$ExpectedExceptionStatement.evaluate(ExpectedException.java:110) at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:960) at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:935) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57) at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365) at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798) at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458) at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845) at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747) at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781) at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at
[jira] [Commented] (LUCENE-6630) Further improve memory of FieldInfos
[
https://issues.apache.org/jira/browse/LUCENE-6630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14604810#comment-14604810
]
Michael McCandless commented on LUCENE-6630:
+1, this looks nice!
Maybe add a comment above {{ return o1.number - o2.number;}} that this is safe
(can't overflow int) because {{number}} is non-negative?
We can increase the 1/16th heuristic now? We picked that because that was the
cutover to minimize RAM, but with this change it suddenly becomes a time/space
tradeoff because the sparse representation is now always = the non-sparse one.
Maybe we should always just be sparse, unless it's complete dense?
Further improve memory of FieldInfos
Key: LUCENE-6630
URL: https://issues.apache.org/jira/browse/LUCENE-6630
Project: Lucene - Core
Issue Type: Bug
Reporter: Ryan Ernst
Attachments: LUCENE-6630.patch
In LUCENE-6325, FieldInfos were improved to use an array except in very
sparse cases. In the sparse cases, a TreeMap is still used. However, even in
the sparse case, we can still use an array, and then use binary search. The
perf will be equivalent, but memory use should be much less.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-7710) replace various async with CommonAdminParams.ASYNC
[
https://issues.apache.org/jira/browse/SOLR-7710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ramkumar Aiyengar updated SOLR-7710:
Attachment: SOLR-7710.patch
Slightly modified patch to use static imports where they are already used
elsewhere in the file..
replace various async with CommonAdminParams.ASYNC
Key: SOLR-7710
URL: https://issues.apache.org/jira/browse/SOLR-7710
Project: Solr
Issue Type: Wish
Reporter: Christine Poerschke
Assignee: Ramkumar Aiyengar
Priority: Minor
Attachments: SOLR-7710.patch
* github pull request with proposed changes to follow shortly.
* Motivation is addition of a test case for an unrelated change (not yet
uploaded) and not wishing to add literally {{async}} in that test case.
Using {{CommonAdminParams.ASYNC}} instead of async can also make code base
'navigation' a little easier e.g. locating of existing test cases.
* The proposed change also adds a little {{CommonAdminParamsTest}} to test
the backwards compatibility of CommonAdminParams parameter constants i.e. if
someone accidentally changes those constants then the test will flag that up.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-7710) replace various async with CommonAdminParams.ASYNC
[
https://issues.apache.org/jira/browse/SOLR-7710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14604838#comment-14604838
]
ASF subversion and git services commented on SOLR-7710:
---
Commit 1688040 from [~andyetitmoves] in branch 'dev/branches/branch_5x'
[ https://svn.apache.org/r1688040 ]
SOLR-7710: Replace async occurrences with CommonAdminParams.ASYNC
replace various async with CommonAdminParams.ASYNC
Key: SOLR-7710
URL: https://issues.apache.org/jira/browse/SOLR-7710
Project: Solr
Issue Type: Wish
Reporter: Christine Poerschke
Assignee: Ramkumar Aiyengar
Priority: Minor
Attachments: SOLR-7710.patch
* github pull request with proposed changes to follow shortly.
* Motivation is addition of a test case for an unrelated change (not yet
uploaded) and not wishing to add literally {{async}} in that test case.
Using {{CommonAdminParams.ASYNC}} instead of async can also make code base
'navigation' a little easier e.g. locating of existing test cases.
* The proposed change also adds a little {{CommonAdminParamsTest}} to test
the backwards compatibility of CommonAdminParams parameter constants i.e. if
someone accidentally changes those constants then the test will flag that up.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-6631) Lucene Document Classification
[ https://issues.apache.org/jira/browse/LUCENE-6631?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14604850#comment-14604850 ] Alessandro Benedetti commented on LUCENE-6631: -- I will provide the patch in a couple of days. it's almost ready, just few modifications needed. The algorithms supported for Document Classification will be : - KNearestNeighborClassifier - SimpleNaiveBayesClassifier Lucene Document Classification -- Key: LUCENE-6631 URL: https://issues.apache.org/jira/browse/LUCENE-6631 Project: Lucene - Core Issue Type: Improvement Components: modules/classification Affects Versions: 5.2.1 Reporter: Alessandro Benedetti Labels: classification Currently the Lucene Classification module supports the classification for an input text using the Lucene index as a trained model. This improvement is adding to the module a set of components to provide Document classification ( where the Document is a Lucene document ). All selected fields from the Document will have their part in the classification ( including the use of the proper Analyzer per field). -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (LUCENE-6631) Lucene Document Classification
Alessandro Benedetti created LUCENE-6631: Summary: Lucene Document Classification Key: LUCENE-6631 URL: https://issues.apache.org/jira/browse/LUCENE-6631 Project: Lucene - Core Issue Type: Improvement Components: modules/classification Affects Versions: 5.2.1 Reporter: Alessandro Benedetti Currently the Lucene Classification module supports the classification for an input text using the Lucene index as a trained model. This improvement is adding to the module a set of components to provide Document classification ( where the Document is a Lucene document ). All selected fields from the Document will have their part in the classification ( including the use of the proper Analyzer per field). -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-6630) Further improve memory of FieldInfos
[ https://issues.apache.org/jira/browse/LUCENE-6630?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ryan Ernst updated LUCENE-6630: --- Attachment: LUCENE-6630.patch Patch. Further improve memory of FieldInfos Key: LUCENE-6630 URL: https://issues.apache.org/jira/browse/LUCENE-6630 Project: Lucene - Core Issue Type: Bug Reporter: Ryan Ernst Attachments: LUCENE-6630.patch In LUCENE-6325, FieldInfos were improved to use an array except in very sparse cases. In the sparse cases, a TreeMap is still used. However, even in the sparse case, we can still use an array, and then use binary search. The perf will be equivalent, but memory use should be much less. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (LUCENE-6630) Further improve memory of FieldInfos
Ryan Ernst created LUCENE-6630: -- Summary: Further improve memory of FieldInfos Key: LUCENE-6630 URL: https://issues.apache.org/jira/browse/LUCENE-6630 Project: Lucene - Core Issue Type: Bug Reporter: Ryan Ernst In LUCENE-6325, FieldInfos were improved to use an array except in very sparse cases. In the sparse cases, a TreeMap is still used. However, even in the sparse case, we can still use an array, and then use binary search. The perf will be equivalent, but memory use should be much less. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Assigned] (SOLR-7710) replace various async with CommonAdminParams.ASYNC
[
https://issues.apache.org/jira/browse/SOLR-7710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ramkumar Aiyengar reassigned SOLR-7710:
---
Assignee: Ramkumar Aiyengar
replace various async with CommonAdminParams.ASYNC
Key: SOLR-7710
URL: https://issues.apache.org/jira/browse/SOLR-7710
Project: Solr
Issue Type: Wish
Reporter: Christine Poerschke
Assignee: Ramkumar Aiyengar
Priority: Minor
* github pull request with proposed changes to follow shortly.
* Motivation is addition of a test case for an unrelated change (not yet
uploaded) and not wishing to add literally {{async}} in that test case.
Using {{CommonAdminParams.ASYNC}} instead of async can also make code base
'navigation' a little easier e.g. locating of existing test cases.
* The proposed change also adds a little {{CommonAdminParamsTest}} to test
the backwards compatibility of CommonAdminParams parameter constants i.e. if
someone accidentally changes those constants then the test will flag that up.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Solr-trunk-MacOSX (64bit/jdk1.8.0) - Build # 2465 - Failure!
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-MacOSX/2465/ Java: 64bit/jdk1.8.0 -XX:+UseCompressedOops -XX:+UseConcMarkSweepGC 3 tests failed. FAILED: org.apache.solr.cloud.TestMiniSolrCloudCluster.testBasics Error Message: Stack Trace: java.lang.NullPointerException at __randomizedtesting.SeedInfo.seed([F753CFA19A53DD30:CA8B618DA2BD8340]:0) at java.util.LinkedList$ListItr.next(LinkedList.java:893) at org.apache.solr.cloud.MiniSolrCloudCluster.shutdown(MiniSolrCloudCluster.java:349) at org.apache.solr.cloud.TestMiniSolrCloudCluster.testCollectionCreateSearchDelete(TestMiniSolrCloudCluster.java:199) at org.apache.solr.cloud.TestMiniSolrCloudCluster.testBasics(TestMiniSolrCloudCluster.java:86) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:497) at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627) at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836) at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872) at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57) at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365) at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798) at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458) at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845) at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747) at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781) at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65) at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365) at java.lang.Thread.run(Thread.java:745) FAILED: junit.framework.TestSuite.org.apache.solr.cloud.TestMiniSolrCloudCluster Error Message: 88 threads leaked from SUITE scope at
[jira] [Commented] (SOLR-6992) ShowFileRequestHandler is hiding dynamic schema file even in read-only mode
[
https://issues.apache.org/jira/browse/SOLR-6992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14604862#comment-14604862
]
Alexandre Rafalovitch commented on SOLR-6992:
-
Could someone please review this.
It really present inconsistent Files view in Admin UI when comparing
data-driven and schema.xml style cores.
ShowFileRequestHandler is hiding dynamic schema file even in read-only mode
---
Key: SOLR-6992
URL: https://issues.apache.org/jira/browse/SOLR-6992
Project: Solr
Issue Type: Bug
Affects Versions: 5.0
Reporter: Alexandre Rafalovitch
Priority: Minor
Fix For: 5.0
When using dynamic schema, the schema file is not shown in the admin UI
(Files tab) by default. It is hidden because (as per [the section
comment|https://github.com/apache/lucene-solr/blob/db33df44c037e04fea9ac3e391ef79c0d6ae04f4/solr/core/src/java/org/apache/solr/handler/admin/ShowFileRequestHandler.java#L279]):
{quote}
// Make sure that if the schema is managed, we don't allow editing.
{quote}
But we don't have editing interface for those files (SOLR-5287 was backed
out), so the reason is not valid and makes reviewing schema just that bit
harder.
The fix is probably just editing out that section, unless there are tests
specifically for that.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[CI] Lucene 5x Linux 64 Test Only - Build # 53450 - Failure!
BUILD FAILURE Build URLhttp://build-eu-00.elastic.co/job/lucene_linux_java8_64_test_only/53450/ Project:lucene_linux_java8_64_test_only Randomization: JDKEA8,local,heap[685m],-server +UseParallelGC +UseCompressedOops,sec manager on Date of build:Sun, 28 Jun 2015 23:38:47 +0200 Build duration:2 min 31 sec CHANGES No Changes BUILD ARTIFACTS checkout/lucene/build/core/test/temp/junit4-J0-20150628_233907_257.events checkout/lucene/build/core/test/temp/junit4-J1-20150628_233907_257.events checkout/lucene/build/core/test/temp/junit4-J2-20150628_233907_257.events checkout/lucene/build/core/test/temp/junit4-J3-20150628_233907_257.events checkout/lucene/build/core/test/temp/junit4-J4-20150628_233907_257.events checkout/lucene/build/core/test/temp/junit4-J5-20150628_233907_259.events checkout/lucene/build/core/test/temp/junit4-J6-20150628_233907_259.events checkout/lucene/build/core/test/temp/junit4-J7-20150628_233907_259.events FAILED JUNIT TESTS Name: org.apache.lucene.index Failed: 1 test(s), Passed: 788 test(s), Skipped: 22 test(s), Total: 811 test(s) Failed: org.apache.lucene.index.TestAddIndexes.testAddIndexesWithCloseNoWait CONSOLE OUTPUT [...truncated 1786 lines...] [junit4] Tests with failures: [junit4] - org.apache.lucene.index.TestAddIndexes.testAddIndexesWithCloseNoWait [junit4] [junit4] [junit4] JVM J0: 0.86 .. 121.14 = 120.28s [junit4] JVM J1: 0.87 .. 120.96 = 120.09s [junit4] JVM J2: 0.88 .. 123.18 = 122.30s [junit4] JVM J3: 1.10 .. 120.18 = 119.08s [junit4] JVM J4: 0.96 .. 120.69 = 119.73s [junit4] JVM J5: 0.88 .. 122.60 = 121.72s [junit4] JVM J6: 0.88 .. 120.15 = 119.28s [junit4] JVM J7: 0.88 .. 130.97 = 130.09s [junit4] Execution time total: 2 minutes 10 seconds [junit4] Tests summary: 402 suites, 3269 tests, 1 failure, 49 ignored (45 assumptions) BUILD FAILED /home/jenkins/workspace/lucene_linux_java8_64_test_only/checkout/lucene/build.xml:50: The following error occurred while executing this line: /home/jenkins/workspace/lucene_linux_java8_64_test_only/checkout/lucene/common-build.xml:1444: The following error occurred while executing this line: /home/jenkins/workspace/lucene_linux_java8_64_test_only/checkout/lucene/common-build.xml:999: There were test failures: 402 suites, 3269 tests, 1 failure, 49 ignored (45 assumptions) Total time: 2 minutes 19 seconds Build step 'Invoke Ant' marked build as failure Archiving artifacts Recording test results [description-setter] Description set: JDKEA8,local,heap[685m],-server +UseParallelGC +UseCompressedOops,sec manager on Email was triggered for: Failure - 1st Trigger Failure - Any was overridden by another trigger and will not send an email. Trigger Failure - Still was overridden by another trigger and will not send an email. Sending email for trigger: Failure - 1st - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 888 - Still Failing
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/888/ 1 tests failed. FAILED: org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test Error Message: Captured an uncaught exception in thread: Thread[id=12636, name=collection0, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest] Stack Trace: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=12636, name=collection0, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest] Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:59125/ex/eo: collection already exists: awholynewstresscollection_collection0_1 at __randomizedtesting.SeedInfo.seed([2B6FFE80E3F0FAE8]:0) at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:560) at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:234) at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:226) at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:376) at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:328) at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1086) at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:856) at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:799) at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220) at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1572) at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1593) at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:887) Build Log: [...truncated 10938 lines...] [junit4] Suite: org.apache.solr.cloud.CollectionsAPIDistributedZkTest [junit4] 2 Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_2B6FFE80E3F0FAE8-001/init-core-data-001 [junit4] 2 1661405 INFO (SUITE-CollectionsAPIDistributedZkTest-seed#[2B6FFE80E3F0FAE8]-worker) [] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (false) [junit4] 2 1661406 INFO (SUITE-CollectionsAPIDistributedZkTest-seed#[2B6FFE80E3F0FAE8]-worker) [] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /ex/eo [junit4] 2 1661411 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[2B6FFE80E3F0FAE8]) [] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER [junit4] 2 1661411 INFO (Thread-7498) [] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0 [junit4] 2 1661412 INFO (Thread-7498) [] o.a.s.c.ZkTestServer Starting server [junit4] 2 1661511 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[2B6FFE80E3F0FAE8]) [] o.a.s.c.ZkTestServer start zk server on port:54104 [junit4] 2 1661526 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[2B6FFE80E3F0FAE8]) [] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml [junit4] 2 1661528 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[2B6FFE80E3F0FAE8]) [] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml [junit4] 2 1661531 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[2B6FFE80E3F0FAE8]) [] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml [junit4] 2 1661532 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[2B6FFE80E3F0FAE8]) [] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt [junit4] 2 1661534 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[2B6FFE80E3F0FAE8]) [] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt [junit4] 2 1661536 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[2B6FFE80E3F0FAE8]) [] o.a.s.c.AbstractZkTestCase put
