[jira] [Commented] (LUCENE-6066) New remove method in PriorityQueue
[ https://issues.apache.org/jira/browse/LUCENE-6066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14251364#comment-14251364 ] Adrien Grand commented on LUCENE-6066: -- bq. the abstract method returns NumericDocValues, which is confusing: how does beatles become a number? Why not e.g. SortedDVs? I think that ideally it should work with both numerics and strings, but then I'm afraid we would end-up with a complicated FieldComparator-like API ? One idea I had (that might be bad) is that if we use NumericDocValues as a way to get keys, then we could also make it work for strings by having a NumericDocValues instance that returns an OrdinalMap's global ordinals as keys. Another option would be to have something that produces an object key (with equals/hashcode) for every document but for numerics it would probably mean lots of boxing, and for strings it would imply value lookups in every collect() call, so it's not very appealing ? New remove method in PriorityQueue Key: LUCENE-6066 URL: https://issues.apache.org/jira/browse/LUCENE-6066 Project: Lucene - Core Issue Type: Improvement Components: core/query/scoring Reporter: Mark Harwood Priority: Minor Fix For: 5.0 Attachments: LUCENE-PQRemoveV3.patch It would be useful to be able to remove existing elements from a PriorityQueue. The proposal is that a linear scan is performed to find the element being removed and then the end element in heap[size] is swapped into this position to perform the delete. The method downHeap() is then called to shuffle the replacement element back down the array but the existing downHeap method must be modified to allow picking up an entry from any point in the array rather than always assuming the first element (which is its only current mode of operation). A working javascript model of the proposal with animation is available here: http://jsfiddle.net/grcmquf2/22/ In tests the modified version of downHeap produces the same results as the existing impl but adds the ability to push down from any point. An example use case that requires remove is where a client doesn't want more than N matches for any given key (e.g. no more than 5 products from any one retailer in a marketplace). In these circumstances a document that was previously thought of as competitive has to be removed from the final PQ and replaced with another doc (eg a retailer who already has 5 matches in the PQ receives a 6th match which is better than his previous ones). This particular process is managed by a special DiversifyingPriorityQueue which wraps the main PriorityQueue and could be contributed as part of another issue if there is interest in that. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-6365) specify appends, defaults, invariants outside of the component
[
https://issues.apache.org/jira/browse/SOLR-6365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14251368#comment-14251368
]
ASF subversion and git services commented on SOLR-6365:
---
Commit 1646401 from [~noble.paul] in branch 'dev/trunk'
[ https://svn.apache.org/r1646401 ]
SOLR-6365 lazy loaded components did not have initParams applied
specify appends, defaults, invariants outside of the component
---
Key: SOLR-6365
URL: https://issues.apache.org/jira/browse/SOLR-6365
Project: Solr
Issue Type: Improvement
Reporter: Noble Paul
Assignee: Noble Paul
Fix For: 5.0, Trunk
Attachments: SOLR-6365-crappy-test.patch, SOLR-6365.patch,
SOLR-6365.patch, SOLR-6365.patch
The components are configured in solrconfig.xml mostly for specifying these
extra parameters. If we separate these out, we can avoid specifying the
components altogether and make solrconfig much simpler. Eventually we want
users to see all functions as paths instead of components and control these
params from outside , through an API and persisted in ZK
objectives :
* define standard components implicitly and let users override some params
only
* reuse standard params across components
* define multiple param sets and mix and match these params at request time
example
{code:xml}
!-- use json for all paths and _txt as the default search field--
initParams name=global path=/**
lst name=defaults
str name=wtjson/str
str name=df_txt/str
/lst
/initParams
{code}
other examples
{code:xml}
initParams name=a path=/dump3,/root/*,/root1/**
lst name=defaults
str name=aA/str
/lst
lst name=invariants
str name=bB/str
/lst
lst name=appends
str name=cC/str
/lst
/initParams
requestHandler name=/dump3 class=DumpRequestHandler/
requestHandler name=/dump4 class=DumpRequestHandler/
requestHandler name=/root/dump5 class=DumpRequestHandler/
requestHandler name=/root1/anotherlevel/dump6
class=DumpRequestHandler/
requestHandler name=/dump1 class=DumpRequestHandler initParams=a/
requestHandler name=/dump2 class=DumpRequestHandler initParams=a
lst name=defaults
str name=aA1/str
/lst
lst name=invariants
str name=bB1/str
/lst
lst name=appends
str name=cC1/str
/lst
/requestHandler
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-6365) specify appends, defaults, invariants outside of the component
[
https://issues.apache.org/jira/browse/SOLR-6365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14251372#comment-14251372
]
ASF subversion and git services commented on SOLR-6365:
---
Commit 1646402 from [~noble.paul] in branch 'dev/branches/branch_5x'
[ https://svn.apache.org/r1646402 ]
SOLR-6365 lazy loaded components did not have initParams applied
specify appends, defaults, invariants outside of the component
---
Key: SOLR-6365
URL: https://issues.apache.org/jira/browse/SOLR-6365
Project: Solr
Issue Type: Improvement
Reporter: Noble Paul
Assignee: Noble Paul
Fix For: 5.0, Trunk
Attachments: SOLR-6365-crappy-test.patch, SOLR-6365.patch,
SOLR-6365.patch, SOLR-6365.patch
The components are configured in solrconfig.xml mostly for specifying these
extra parameters. If we separate these out, we can avoid specifying the
components altogether and make solrconfig much simpler. Eventually we want
users to see all functions as paths instead of components and control these
params from outside , through an API and persisted in ZK
objectives :
* define standard components implicitly and let users override some params
only
* reuse standard params across components
* define multiple param sets and mix and match these params at request time
example
{code:xml}
!-- use json for all paths and _txt as the default search field--
initParams name=global path=/**
lst name=defaults
str name=wtjson/str
str name=df_txt/str
/lst
/initParams
{code}
other examples
{code:xml}
initParams name=a path=/dump3,/root/*,/root1/**
lst name=defaults
str name=aA/str
/lst
lst name=invariants
str name=bB/str
/lst
lst name=appends
str name=cC/str
/lst
/initParams
requestHandler name=/dump3 class=DumpRequestHandler/
requestHandler name=/dump4 class=DumpRequestHandler/
requestHandler name=/root/dump5 class=DumpRequestHandler/
requestHandler name=/root1/anotherlevel/dump6
class=DumpRequestHandler/
requestHandler name=/dump1 class=DumpRequestHandler initParams=a/
requestHandler name=/dump2 class=DumpRequestHandler initParams=a
lst name=defaults
str name=aA1/str
/lst
lst name=invariants
str name=bB1/str
/lst
lst name=appends
str name=cC1/str
/lst
/requestHandler
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Resolved] (SOLR-6365) specify appends, defaults, invariants outside of the component
[
https://issues.apache.org/jira/browse/SOLR-6365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Noble Paul resolved SOLR-6365.
--
Resolution: Fixed
when startup=lazy was specified , initParams were not applied
specify appends, defaults, invariants outside of the component
---
Key: SOLR-6365
URL: https://issues.apache.org/jira/browse/SOLR-6365
Project: Solr
Issue Type: Improvement
Reporter: Noble Paul
Assignee: Noble Paul
Fix For: 5.0, Trunk
Attachments: SOLR-6365-crappy-test.patch, SOLR-6365.patch,
SOLR-6365.patch, SOLR-6365.patch
The components are configured in solrconfig.xml mostly for specifying these
extra parameters. If we separate these out, we can avoid specifying the
components altogether and make solrconfig much simpler. Eventually we want
users to see all functions as paths instead of components and control these
params from outside , through an API and persisted in ZK
objectives :
* define standard components implicitly and let users override some params
only
* reuse standard params across components
* define multiple param sets and mix and match these params at request time
example
{code:xml}
!-- use json for all paths and _txt as the default search field--
initParams name=global path=/**
lst name=defaults
str name=wtjson/str
str name=df_txt/str
/lst
/initParams
{code}
other examples
{code:xml}
initParams name=a path=/dump3,/root/*,/root1/**
lst name=defaults
str name=aA/str
/lst
lst name=invariants
str name=bB/str
/lst
lst name=appends
str name=cC/str
/lst
/initParams
requestHandler name=/dump3 class=DumpRequestHandler/
requestHandler name=/dump4 class=DumpRequestHandler/
requestHandler name=/root/dump5 class=DumpRequestHandler/
requestHandler name=/root1/anotherlevel/dump6
class=DumpRequestHandler/
requestHandler name=/dump1 class=DumpRequestHandler initParams=a/
requestHandler name=/dump2 class=DumpRequestHandler initParams=a
lst name=defaults
str name=aA1/str
/lst
lst name=invariants
str name=bB1/str
/lst
lst name=appends
str name=cC1/str
/lst
/requestHandler
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-6118) Improve efficiency of the history structure for filter caching
[
https://issues.apache.org/jira/browse/LUCENE-6118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Adrien Grand updated LUCENE-6118:
-
Attachment: LUCENE-6118.patch
It wouldn't work unfortunately, let's imagine the following example: you have
four keys A, B, C and D. Given their hash codes, A and B should go to slot 3
while C and D should go to slot 4. It is possible to have them store at the
following slots in the hash table:
{noformat}
slot | 3 | 4 | 5 | 6 |
key | A | C | B | D |
{noformat}
Now if we remove A, the end of the chain is D but its slot is 4 not 3 so we
cannot move it into slot 3. Instead, we need to move B in the slot of A and D
in the slot of B to come back to a consistent state:
{noformat}
slot | 3 | 4 | 5 | 6 |
key | B | C | D | ∅ |
{noformat}
But you're right that the algorithm is not optimal here... I updated the patch
with something better (and hopefully still readable?).
Improve efficiency of the history structure for filter caching
--
Key: LUCENE-6118
URL: https://issues.apache.org/jira/browse/LUCENE-6118
Project: Lucene - Core
Issue Type: Improvement
Reporter: Adrien Grand
Assignee: Adrien Grand
Priority: Minor
Attachments: LUCENE-6118.patch, LUCENE-6118.patch
The filter caching uses a ring buffer that tracks frequencies of the
hashcodes of the most-recently used filters. However it is based on an
ArrayDequeInteger and a HashMapInteger which keep on (un)wrapping ints.
Since the data-structure is very simple, we could try to do something
better...
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-6115) Add getMergeInstance to CompressingStoredFieldsReader
[ https://issues.apache.org/jira/browse/LUCENE-6115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14251410#comment-14251410 ] ASF subversion and git services commented on LUCENE-6115: - Commit 1646413 from [~jpountz] in branch 'dev/trunk' [ https://svn.apache.org/r1646413 ] LUCENE-6115: Add getMergeInstance to CompressingStoredFieldsReader. Add getMergeInstance to CompressingStoredFieldsReader - Key: LUCENE-6115 URL: https://issues.apache.org/jira/browse/LUCENE-6115 Project: Lucene - Core Issue Type: Improvement Reporter: Adrien Grand Assignee: Adrien Grand Priority: Minor Attachments: LUCENE-6115.patch, LUCENE-6115.patch CompressingStoredFieldsReader is currently terrible at merging with different codecs or wrapped readers since it does not keep state. So if you want to get 5 documents that come from the same block, it means that you will have to decode the block header and decompress 5 times. It has some optimizations so that if you want to get the 2nd doc of the block then it will stop decompressing soon after the 2nd document, but it doesn't help much with merging since we want all documents. We should implement getMergeInstance and have a different behaviour when merging by decompressing everything up-front and then reusing for all documents of the block. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-6115) Add getMergeInstance to CompressingStoredFieldsReader
[ https://issues.apache.org/jira/browse/LUCENE-6115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14251416#comment-14251416 ] ASF subversion and git services commented on LUCENE-6115: - Commit 1646415 from [~jpountz] in branch 'dev/branches/branch_5x' [ https://svn.apache.org/r1646415 ] LUCENE-6115: Add getMergeInstance to CompressingStoredFieldsReader. Add getMergeInstance to CompressingStoredFieldsReader - Key: LUCENE-6115 URL: https://issues.apache.org/jira/browse/LUCENE-6115 Project: Lucene - Core Issue Type: Improvement Reporter: Adrien Grand Assignee: Adrien Grand Priority: Minor Attachments: LUCENE-6115.patch, LUCENE-6115.patch CompressingStoredFieldsReader is currently terrible at merging with different codecs or wrapped readers since it does not keep state. So if you want to get 5 documents that come from the same block, it means that you will have to decode the block header and decompress 5 times. It has some optimizations so that if you want to get the 2nd doc of the block then it will stop decompressing soon after the 2nd document, but it doesn't help much with merging since we want all documents. We should implement getMergeInstance and have a different behaviour when merging by decompressing everything up-front and then reusing for all documents of the block. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Solr-Tests-5.x-Java7 - Build # 2344 - Failure
Build: https://builds.apache.org/job/Lucene-Solr-Tests-5.x-Java7/2344/ 1 tests failed. REGRESSION: org.apache.solr.client.solrj.TestLBHttpSolrServer.testReliability Error Message: No live SolrServers available to handle this request Stack Trace: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request at __randomizedtesting.SeedInfo.seed([4FB89F475089CD33:8E704201F1EF1C9A]:0) at org.apache.solr.client.solrj.impl.LBHttpSolrServer.request(LBHttpSolrServer.java:539) at org.apache.solr.client.solrj.request.QueryRequest.process(QueryRequest.java:91) at org.apache.solr.client.solrj.SolrServer.query(SolrServer.java:301) at org.apache.solr.client.solrj.TestLBHttpSolrServer.testReliability(TestLBHttpSolrServer.java:223) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618) at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827) at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863) at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53) at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55) at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365) at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798) at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458) at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836) at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738) at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772) at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65) at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at
[jira] [Commented] (LUCENE-6019) IndexWriter allows to add same field with different docvlaues type
[ https://issues.apache.org/jira/browse/LUCENE-6019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14251418#comment-14251418 ] ASF subversion and git services commented on LUCENE-6019: - Commit 1646416 from [~mikemccand] in branch 'dev/branches/lucene_solr_4_10' [ https://svn.apache.org/r1646416 ] LUCENE-6019, LUCENE-6117: remove -Dtests.assert: this is too big a change for a bug-fix release (and it introduced a bug) IndexWriter allows to add same field with different docvlaues type --- Key: LUCENE-6019 URL: https://issues.apache.org/jira/browse/LUCENE-6019 Project: Lucene - Core Issue Type: Bug Components: core/index Affects Versions: 4.10.1 Reporter: Simon Willnauer Assignee: Michael McCandless Priority: Critical Fix For: 4.10.2, 5.0, Trunk Attachments: LUCENE-6019.patch, LUCENE-6019.patch IndexWriter checks if the DV types are consitent in multiple places but if due to some problems in Elasticsearch users where able to add the same field with different DV types causing merges to fail. Yet I was able to reduce this to a lucene testcase but I was puzzled since it always failed. Yet, I had to run it without assertions and that cause the bug to happen. I can add field foo with BINARY and SORTED_SET causing a merge to fail. Here is a gist https://gist.github.com/s1monw/8707f924b76ba40ee5f3 / https://github.com/elasticsearch/elasticsearch/issues/8009 While this is certainly a problem in Elasticsearch Lucene also allows to corrupt an index due to user error which I think should be prevented. NOTE: this only fails if you run without assertions which I think lucene should do in CI once in a while too. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Resolved] (LUCENE-6019) IndexWriter allows to add same field with different docvlaues type
[ https://issues.apache.org/jira/browse/LUCENE-6019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael McCandless resolved LUCENE-6019. Resolution: Fixed IndexWriter allows to add same field with different docvlaues type --- Key: LUCENE-6019 URL: https://issues.apache.org/jira/browse/LUCENE-6019 Project: Lucene - Core Issue Type: Bug Components: core/index Affects Versions: 4.10.1 Reporter: Simon Willnauer Assignee: Michael McCandless Priority: Critical Fix For: 5.0, Trunk, 4.10.2 Attachments: LUCENE-6019.patch, LUCENE-6019.patch IndexWriter checks if the DV types are consitent in multiple places but if due to some problems in Elasticsearch users where able to add the same field with different DV types causing merges to fail. Yet I was able to reduce this to a lucene testcase but I was puzzled since it always failed. Yet, I had to run it without assertions and that cause the bug to happen. I can add field foo with BINARY and SORTED_SET causing a merge to fail. Here is a gist https://gist.github.com/s1monw/8707f924b76ba40ee5f3 / https://github.com/elasticsearch/elasticsearch/issues/8009 While this is certainly a problem in Elasticsearch Lucene also allows to corrupt an index due to user error which I think should be prevented. NOTE: this only fails if you run without assertions which I think lucene should do in CI once in a while too. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-6117) infostream is currently unusable out of box
[
https://issues.apache.org/jira/browse/LUCENE-6117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14251419#comment-14251419
]
ASF subversion and git services commented on LUCENE-6117:
-
Commit 1646416 from [~mikemccand] in branch 'dev/branches/lucene_solr_4_10'
[ https://svn.apache.org/r1646416 ]
LUCENE-6019, LUCENE-6117: remove -Dtests.assert: this is too big a change for a
bug-fix release (and it introduced a bug)
infostream is currently unusable out of box
---
Key: LUCENE-6117
URL: https://issues.apache.org/jira/browse/LUCENE-6117
Project: Lucene - Core
Issue Type: Bug
Reporter: Robert Muir
Fix For: 5.0, Trunk
Attachments: LUCENE-6117.patch
testpoints used to only be emitted by assertions (still sketchy), but now are
emitted always. I assume this is due to the change to support running tests
with assertions disabled.
we should try to clean this up, simple stuff like this is now useless:
{code}
indexWriterConfig.setInfoStream(System.out);
// causes massive flooding like this:
// TP 0 [Tue Dec 16 20:19:37 EST 2014; Thread-0]: DocumentsWriterPerThread
addDocument start
// TP 0 [Tue Dec 16 20:19:37 EST 2014; Thread-0]: DocumentsWriterPerThread
addDocument start
// TP 0 [Tue Dec 16 20:19:37 EST 2014; Thread-0]: DocumentsWriterPerThread
addDocument start
{code}
I hit this several times today just trying to do benchmarks and debugging.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Resolved] (LUCENE-6115) Add getMergeInstance to CompressingStoredFieldsReader
[ https://issues.apache.org/jira/browse/LUCENE-6115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adrien Grand resolved LUCENE-6115. -- Resolution: Fixed Fix Version/s: Trunk 5.0 Add getMergeInstance to CompressingStoredFieldsReader - Key: LUCENE-6115 URL: https://issues.apache.org/jira/browse/LUCENE-6115 Project: Lucene - Core Issue Type: Improvement Reporter: Adrien Grand Assignee: Adrien Grand Priority: Minor Fix For: 5.0, Trunk Attachments: LUCENE-6115.patch, LUCENE-6115.patch CompressingStoredFieldsReader is currently terrible at merging with different codecs or wrapped readers since it does not keep state. So if you want to get 5 documents that come from the same block, it means that you will have to decode the block header and decompress 5 times. It has some optimizations so that if you want to get the 2nd doc of the block then it will stop decompressing soon after the 2nd document, but it doesn't help much with merging since we want all documents. We should implement getMergeInstance and have a different behaviour when merging by decompressing everything up-front and then reusing for all documents of the block. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-6787) API to manage blobs in Solr
[
https://issues.apache.org/jira/browse/SOLR-6787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14251427#comment-14251427
]
ASF subversion and git services commented on SOLR-6787:
---
Commit 1646418 from [~noble.paul] in branch 'dev/trunk'
[ https://svn.apache.org/r1646418 ]
SOLR-6787 refactored a method out
API to manage blobs in Solr
Key: SOLR-6787
URL: https://issues.apache.org/jira/browse/SOLR-6787
Project: Solr
Issue Type: Sub-task
Reporter: Noble Paul
Assignee: Noble Paul
Fix For: 5.0, Trunk
Attachments: SOLR-6787.patch, SOLR-6787.patch
A special collection called .system needs to be created by the user to
store/manage blobs. The schema/solrconfig of that collection need to be
automatically supplied by the system so that there are no errors
APIs need to be created to manage the content of that collection
{code}
#create your .system collection first
http://localhost:8983/solr/admin/collections?action=CREATEname=.systemreplicationFactor=2
#The config for this collection is automatically created . numShards for this
collection is hardcoded to 1
#create a new jar or add a new version of a jar
curl -X POST -H 'Content-Type: application/octet-stream' --data-binary
@mycomponent.jar http://localhost:8983/solr/.system/blob/mycomponent
# GET on the end point would give a list of jars and other details
curl http://localhost:8983/solr/.system/blob
# GET on the end point with jar name would give details of various versions
of the available jars
curl http://localhost:8983/solr/.system/blob/mycomponent
# GET on the end point with jar name and version with a wt=filestream to get
the actual file
curl http://localhost:8983/solr/.system/blob/mycomponent/1?wt=filestream
mycomponent.1.jar
# GET on the end point with jar name and wt=filestream to get the latest
version of the file
curl http://localhost:8983/solr/.system/blob/mycomponent?wt=filestream
mycomponent.jar
{code}
Please note that the jars are never deleted. a new version is added to the
system everytime a new jar is posted for the name. You must use the standard
delete commands to delete the old entries
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-6787) API to manage blobs in Solr
[
https://issues.apache.org/jira/browse/SOLR-6787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14251434#comment-14251434
]
ASF subversion and git services commented on SOLR-6787:
---
Commit 1646419 from [~noble.paul] in branch 'dev/branches/branch_5x'
[ https://svn.apache.org/r1646419 ]
SOLR-6787 refactored a method out
API to manage blobs in Solr
Key: SOLR-6787
URL: https://issues.apache.org/jira/browse/SOLR-6787
Project: Solr
Issue Type: Sub-task
Reporter: Noble Paul
Assignee: Noble Paul
Fix For: 5.0, Trunk
Attachments: SOLR-6787.patch, SOLR-6787.patch
A special collection called .system needs to be created by the user to
store/manage blobs. The schema/solrconfig of that collection need to be
automatically supplied by the system so that there are no errors
APIs need to be created to manage the content of that collection
{code}
#create your .system collection first
http://localhost:8983/solr/admin/collections?action=CREATEname=.systemreplicationFactor=2
#The config for this collection is automatically created . numShards for this
collection is hardcoded to 1
#create a new jar or add a new version of a jar
curl -X POST -H 'Content-Type: application/octet-stream' --data-binary
@mycomponent.jar http://localhost:8983/solr/.system/blob/mycomponent
# GET on the end point would give a list of jars and other details
curl http://localhost:8983/solr/.system/blob
# GET on the end point with jar name would give details of various versions
of the available jars
curl http://localhost:8983/solr/.system/blob/mycomponent
# GET on the end point with jar name and version with a wt=filestream to get
the actual file
curl http://localhost:8983/solr/.system/blob/mycomponent/1?wt=filestream
mycomponent.1.jar
# GET on the end point with jar name and wt=filestream to get the latest
version of the file
curl http://localhost:8983/solr/.system/blob/mycomponent?wt=filestream
mycomponent.jar
{code}
Please note that the jars are never deleted. a new version is added to the
system everytime a new jar is posted for the name. You must use the standard
delete commands to delete the old entries
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-6718) relative coreRootDirectory evaluated against CWD, not Solr Home
[
https://issues.apache.org/jira/browse/SOLR-6718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alan Woodward updated SOLR-6718:
Attachment: SOLR-6718.patch
Better patch, using NIO2 to resolve paths.
relative coreRootDirectory evaluated against CWD, not Solr Home
---
Key: SOLR-6718
URL: https://issues.apache.org/jira/browse/SOLR-6718
Project: Solr
Issue Type: Bug
Affects Versions: 4.10
Reporter: Hoss Man
Assignee: Alan Woodward
Priority: Blocker
Fix For: 5.0
Attachments: SOLR-6718.patch, SOLR-6718.patch
as reported by Andreas Hubold on solr-user...
{noformat}
I'd like to just set it to a subdirectory of solr home (a cores directory to
avoid confusion with configsets and other directories). I tried
str name=coreRootDirectorycores/str
but that's interpreted relative to the current working directory. Other paths
such as sharedLib are interpreted relative to Solr Home and I had expected
this here too.
{noformat}
This is definitely non-intuitive and we should try to fix it in 5.0 since it
will represent a back-compat change for anyone already using a relative path.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Solr-5.x-MacOSX (64bit/jdk1.8.0) - Build # 1957 - Failure!
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-5.x-MacOSX/1957/ Java: 64bit/jdk1.8.0 -XX:+UseCompressedOops -XX:+UseParallelGC (asserts: true) 1 tests failed. FAILED: org.apache.solr.cloud.OverseerStatusTest.testDistribSearch Error Message: Error from server at http://127.0.0.1:61049/_: reload the collection time out:180s Stack Trace: org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: Error from server at http://127.0.0.1:61049/_: reload the collection time out:180s at __randomizedtesting.SeedInfo.seed([592B05954C680F06:D8CD8B8D3B376F3A]:0) at org.apache.solr.client.solrj.impl.HttpSolrServer.executeMethod(HttpSolrServer.java:566) at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:213) at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:209) at org.apache.solr.cloud.AbstractFullDistribZkTestBase.invokeCollectionApi(AbstractFullDistribZkTestBase.java:1867) at org.apache.solr.cloud.OverseerStatusTest.doTest(OverseerStatusTest.java:103) at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:869) at sun.reflect.GeneratedMethodAccessor41.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:483) at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618) at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827) at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863) at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53) at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55) at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365) at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798) at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458) at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836) at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738) at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772) at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65) at
[jira] [Updated] (SOLR-6770) Add/edit param sets and use them in Requests
[
https://issues.apache.org/jira/browse/SOLR-6770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Noble Paul updated SOLR-6770:
-
Description:
Make it possible to define paramsets and use them directly in requests
example
{code}
curl http://localhost:8983/solr/collection1/config -H
'Content-type:application/json' -d '{
create-paramset : {name ,x
val: {
a:A val,
b: B val}
},
update-paramset : {name ,y
val: {
x:X val,
Y: Y val}
},
delete-paramset : z
}'
{code}
This data will be stored in conf/paramsets.json
example usage http://localhost/solr/collection/select?useParams=x
or it can be directly configured with a request handler as follows
{code}
requestHandler name=/dump1 class=DumpRequestHandler useParams=x/
{code}
was:
Make it possible to define paramsets and use them directly in requests
example
{code}
curl http://localhost:8983/solr/collection1/config -H
'Content-type:application/json' -d '{
create-paramset : {name ,x
val: {
a:A val,
b: B val}
},
update-paramset : {name ,y
val: {
x:X val,
Y: Y val}
},
delete-paramset : z
}'
{code}
This data will be stored in conf/paramsets.json
example usage http://localhost/solr/collection/select?paramset=x
Add/edit param sets and use them in Requests
Key: SOLR-6770
URL: https://issues.apache.org/jira/browse/SOLR-6770
Project: Solr
Issue Type: Sub-task
Reporter: Noble Paul
Assignee: Noble Paul
Make it possible to define paramsets and use them directly in requests
example
{code}
curl http://localhost:8983/solr/collection1/config -H
'Content-type:application/json' -d '{
create-paramset : {name ,x
val: {
a:A val,
b: B val}
},
update-paramset : {name ,y
val: {
x:X val,
Y: Y val}
},
delete-paramset : z
}'
{code}
This data will be stored in conf/paramsets.json
example usage http://localhost/solr/collection/select?useParams=x
or it can be directly configured with a request handler as follows
{code}
requestHandler name=/dump1 class=DumpRequestHandler useParams=x/
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-6770) Add/edit param sets and use them in Requests
[
https://issues.apache.org/jira/browse/SOLR-6770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Noble Paul updated SOLR-6770:
-
Description:
Make it possible to define paramsets and use them directly in requests
example
{code}
curl http://localhost:8983/solr/collection1/config/params -H
'Content-type:application/json' -d '{
create-paramset : {name ,x
val: {
a:A val,
b: B val}
},
update-paramset : {name ,y
val: {
x:X val,
Y: Y val}
},
delete-paramset : z
}'
{code}
This data will be stored in conf/paramsets.json
Any changes to these are only applied at the request time and no core reload is
required but , all nodes will get notified of the changes and all of them will
have the latest version of this all the time
example usage http://localhost/solr/collection/select?useParams=x
or it can be directly configured with a request handler as follows
{code}
requestHandler name=/dump1 class=DumpRequestHandler useParams=x/
{code}
was:
Make it possible to define paramsets and use them directly in requests
example
{code}
curl http://localhost:8983/solr/collection1/config -H
'Content-type:application/json' -d '{
create-paramset : {name ,x
val: {
a:A val,
b: B val}
},
update-paramset : {name ,y
val: {
x:X val,
Y: Y val}
},
delete-paramset : z
}'
{code}
This data will be stored in conf/paramsets.json
example usage http://localhost/solr/collection/select?useParams=x
or it can be directly configured with a request handler as follows
{code}
requestHandler name=/dump1 class=DumpRequestHandler useParams=x/
{code}
Add/edit param sets and use them in Requests
Key: SOLR-6770
URL: https://issues.apache.org/jira/browse/SOLR-6770
Project: Solr
Issue Type: Sub-task
Reporter: Noble Paul
Assignee: Noble Paul
Make it possible to define paramsets and use them directly in requests
example
{code}
curl http://localhost:8983/solr/collection1/config/params -H
'Content-type:application/json' -d '{
create-paramset : {name ,x
val: {
a:A val,
b: B val}
},
update-paramset : {name ,y
val: {
x:X val,
Y: Y val}
},
delete-paramset : z
}'
{code}
This data will be stored in conf/paramsets.json
Any changes to these are only applied at the request time and no core reload
is required but , all nodes will get notified of the changes and all of them
will have the latest version of this all the time
example usage http://localhost/solr/collection/select?useParams=x
or it can be directly configured with a request handler as follows
{code}
requestHandler name=/dump1 class=DumpRequestHandler useParams=x/
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-6066) New remove method in PriorityQueue
[ https://issues.apache.org/jira/browse/LUCENE-6066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14251540#comment-14251540 ] Michael McCandless commented on LUCENE-6066: bq. I think that ideally it should work with both numerics and strings, but then I'm afraid we would end-up with a complicated FieldComparator-like API That's a great and scary point. We do NOT want another FieldComparator. And we don't need the generic Object... I agree: let's use NumericDV, but can we add comments / maybe examples in the test case showing how the beatles example can work w/ this API? New remove method in PriorityQueue Key: LUCENE-6066 URL: https://issues.apache.org/jira/browse/LUCENE-6066 Project: Lucene - Core Issue Type: Improvement Components: core/query/scoring Reporter: Mark Harwood Priority: Minor Fix For: 5.0 Attachments: LUCENE-PQRemoveV3.patch It would be useful to be able to remove existing elements from a PriorityQueue. The proposal is that a linear scan is performed to find the element being removed and then the end element in heap[size] is swapped into this position to perform the delete. The method downHeap() is then called to shuffle the replacement element back down the array but the existing downHeap method must be modified to allow picking up an entry from any point in the array rather than always assuming the first element (which is its only current mode of operation). A working javascript model of the proposal with animation is available here: http://jsfiddle.net/grcmquf2/22/ In tests the modified version of downHeap produces the same results as the existing impl but adds the ability to push down from any point. An example use case that requires remove is where a client doesn't want more than N matches for any given key (e.g. no more than 5 products from any one retailer in a marketplace). In these circumstances a document that was previously thought of as competitive has to be removed from the final PQ and replaced with another doc (eg a retailer who already has 5 matches in the PQ receives a 6th match which is better than his previous ones). This particular process is managed by a special DiversifyingPriorityQueue which wraps the main PriorityQueue and could be contributed as part of another issue if there is interest in that. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-6770) Add/edit param sets and use them in Requests
[
https://issues.apache.org/jira/browse/SOLR-6770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Noble Paul updated SOLR-6770:
-
Description:
Make it possible to define paramsets and use them directly in requests
example
{code}
curl http://localhost:8983/solr/collection1/config/params -H
'Content-type:application/json' -d '{
create : {name ,x
val: {
a:A val,
b: B val}
},
update : {name ,y
val: {
x:X val,
Y: Y val}
},
delete : z
}'
{code}
This data will be stored in conf/paramsets.json
example usage http://localhost/solr/collection/select?useParams=x
or it can be directly configured with a request handler as follows
{code}
requestHandler name=/dump1 class=DumpRequestHandler useParams=x/
{code}
was:
Make it possible to define paramsets and use them directly in requests
example
{code}
curl http://localhost:8983/solr/collection1/config/params -H
'Content-type:application/json' -d '{
create-paramset : {name ,x
val: {
a:A val,
b: B val}
},
update-paramset : {name ,y
val: {
x:X val,
Y: Y val}
},
delete-paramset : z
}'
{code}
This data will be stored in conf/paramsets.json
Any changes to these are only applied at the request time and no core reload is
required but , all nodes will get notified of the changes and all of them will
have the latest version of this all the time
example usage http://localhost/solr/collection/select?useParams=x
or it can be directly configured with a request handler as follows
{code}
requestHandler name=/dump1 class=DumpRequestHandler useParams=x/
{code}
Add/edit param sets and use them in Requests
Key: SOLR-6770
URL: https://issues.apache.org/jira/browse/SOLR-6770
Project: Solr
Issue Type: Sub-task
Reporter: Noble Paul
Assignee: Noble Paul
Make it possible to define paramsets and use them directly in requests
example
{code}
curl http://localhost:8983/solr/collection1/config/params -H
'Content-type:application/json' -d '{
create : {name ,x
val: {
a:A val,
b: B val}
},
update : {name ,y
val: {
x:X val,
Y: Y val}
},
delete : z
}'
{code}
This data will be stored in conf/paramsets.json
example usage http://localhost/solr/collection/select?useParams=x
or it can be directly configured with a request handler as follows
{code}
requestHandler name=/dump1 class=DumpRequestHandler useParams=x/
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-6770) Add/edit param sets and use them in Requests
[
https://issues.apache.org/jira/browse/SOLR-6770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Noble Paul updated SOLR-6770:
-
Description:
Make it possible to define paramsets and use them directly in requests
example
{code}
curl http://localhost:8983/solr/collection1/config/params -H
'Content-type:application/json' -d '{
create : {name ,x
val: {
a:A val,
b: B val}
},
update : {name ,y
val: {
x:X val,
Y: Y val}
},
delete : z
}'
do a GET to view all the configured params
curl http://localhost:8983/solr/collection1/config/params
or GET with a specific name to get only one set of params
curl http://localhost:8983/solr/collection1/config/params/x
{code}
This data will be stored in conf/paramsets.json
This is used requesttime and adding/editing params will not result in core
reload and it will have no impact on the performance
example usage http://localhost/solr/collection/select?useParams=x
or it can be directly configured with a request handler as follows
{code}
requestHandler name=/dump1 class=DumpRequestHandler useParams=x/
{code}
was:
Make it possible to define paramsets and use them directly in requests
example
{code}
curl http://localhost:8983/solr/collection1/config/params -H
'Content-type:application/json' -d '{
create : {name ,x
val: {
a:A val,
b: B val}
},
update : {name ,y
val: {
x:X val,
Y: Y val}
},
delete : z
}'
{code}
This data will be stored in conf/paramsets.json
example usage http://localhost/solr/collection/select?useParams=x
or it can be directly configured with a request handler as follows
{code}
requestHandler name=/dump1 class=DumpRequestHandler useParams=x/
{code}
Add/edit param sets and use them in Requests
Key: SOLR-6770
URL: https://issues.apache.org/jira/browse/SOLR-6770
Project: Solr
Issue Type: Sub-task
Reporter: Noble Paul
Assignee: Noble Paul
Make it possible to define paramsets and use them directly in requests
example
{code}
curl http://localhost:8983/solr/collection1/config/params -H
'Content-type:application/json' -d '{
create : {name ,x
val: {
a:A val,
b: B val}
},
update : {name ,y
val: {
x:X val,
Y: Y val}
},
delete : z
}'
do a GET to view all the configured params
curl http://localhost:8983/solr/collection1/config/params
or GET with a specific name to get only one set of params
curl http://localhost:8983/solr/collection1/config/params/x
{code}
This data will be stored in conf/paramsets.json
This is used requesttime and adding/editing params will not result in core
reload and it will have no impact on the performance
example usage http://localhost/solr/collection/select?useParams=x
or it can be directly configured with a request handler as follows
{code}
requestHandler name=/dump1 class=DumpRequestHandler useParams=x/
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-6770) Add/edit param sets and use them in Requests
[
https://issues.apache.org/jira/browse/SOLR-6770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Noble Paul updated SOLR-6770:
-
Description:
Make it possible to define paramsets and use them directly in requests
example
{code}
curl http://localhost:8983/solr/collection1/config/params -H
'Content-type:application/json' -d '{
create : {name ,x
val: {
a:A val,
b: B val}
},
update : {name ,y
val: {
x:X val,
Y: Y val}
},
delete : z
}'
#do a GET to view all the configured params
curl http://localhost:8983/solr/collection1/config/params
#or GET with a specific name to get only one set of params
curl http://localhost:8983/solr/collection1/config/params/x
{code}
This data will be stored in conf/paramsets.json
This is used requesttime and adding/editing params will not result in core
reload and it will have no impact on the performance
example usage http://localhost/solr/collection/select?useParams=x
or it can be directly configured with a request handler as follows
{code}
requestHandler name=/dump1 class=DumpRequestHandler useParams=x/
{code}
was:
Make it possible to define paramsets and use them directly in requests
example
{code}
curl http://localhost:8983/solr/collection1/config/params -H
'Content-type:application/json' -d '{
create : {name ,x
val: {
a:A val,
b: B val}
},
update : {name ,y
val: {
x:X val,
Y: Y val}
},
delete : z
}'
do a GET to view all the configured params
curl http://localhost:8983/solr/collection1/config/params
or GET with a specific name to get only one set of params
curl http://localhost:8983/solr/collection1/config/params/x
{code}
This data will be stored in conf/paramsets.json
This is used requesttime and adding/editing params will not result in core
reload and it will have no impact on the performance
example usage http://localhost/solr/collection/select?useParams=x
or it can be directly configured with a request handler as follows
{code}
requestHandler name=/dump1 class=DumpRequestHandler useParams=x/
{code}
Add/edit param sets and use them in Requests
Key: SOLR-6770
URL: https://issues.apache.org/jira/browse/SOLR-6770
Project: Solr
Issue Type: Sub-task
Reporter: Noble Paul
Assignee: Noble Paul
Make it possible to define paramsets and use them directly in requests
example
{code}
curl http://localhost:8983/solr/collection1/config/params -H
'Content-type:application/json' -d '{
create : {name ,x
val: {
a:A val,
b: B val}
},
update : {name ,y
val: {
x:X val,
Y: Y val}
},
delete : z
}'
#do a GET to view all the configured params
curl http://localhost:8983/solr/collection1/config/params
#or GET with a specific name to get only one set of params
curl http://localhost:8983/solr/collection1/config/params/x
{code}
This data will be stored in conf/paramsets.json
This is used requesttime and adding/editing params will not result in core
reload and it will have no impact on the performance
example usage http://localhost/solr/collection/select?useParams=x
or it can be directly configured with a request handler as follows
{code}
requestHandler name=/dump1 class=DumpRequestHandler useParams=x/
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5951) Detect when index is on SSD and set dynamic defaults
[ https://issues.apache.org/jira/browse/LUCENE-5951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14251556#comment-14251556 ] Michael McCandless commented on LUCENE-5951: Thanks Rob, I love the new tests :) I'll revert my over-zealous changes to CreateIndexTask (just fix to use the new default) and commit soon ... Detect when index is on SSD and set dynamic defaults Key: LUCENE-5951 URL: https://issues.apache.org/jira/browse/LUCENE-5951 Project: Lucene - Core Issue Type: Improvement Reporter: Michael McCandless Assignee: Michael McCandless Attachments: LUCENE-5951.patch, LUCENE-5951.patch, LUCENE-5951.patch, LUCENE-5951.patch, LUCENE-5951.patch, LUCENE-5951.patch E.g. ConcurrentMergeScheduler should default maxMergeThreads to 3 if it's on SSD and 1 if it's on spinning disks. I think the new NIO2 APIs can let us figure out which device we are mounted on, and from there maybe we can do os-specific stuff e.g. look at /sys/block/dev/queue/rotational to see if it's spinning storage or not ... -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-6718) relative coreRootDirectory evaluated against CWD, not Solr Home
[
https://issues.apache.org/jira/browse/SOLR-6718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alan Woodward updated SOLR-6718:
Attachment: SOLR-6718.patch
Final patch, all tests passing.
relative coreRootDirectory evaluated against CWD, not Solr Home
---
Key: SOLR-6718
URL: https://issues.apache.org/jira/browse/SOLR-6718
Project: Solr
Issue Type: Bug
Affects Versions: 4.10
Reporter: Hoss Man
Assignee: Alan Woodward
Priority: Blocker
Fix For: 5.0
Attachments: SOLR-6718.patch, SOLR-6718.patch, SOLR-6718.patch
as reported by Andreas Hubold on solr-user...
{noformat}
I'd like to just set it to a subdirectory of solr home (a cores directory to
avoid confusion with configsets and other directories). I tried
str name=coreRootDirectorycores/str
but that's interpreted relative to the current working directory. Other paths
such as sharedLib are interpreted relative to Solr Home and I had expected
this here too.
{noformat}
This is definitely non-intuitive and we should try to fix it in 5.0 since it
will represent a back-compat change for anyone already using a relative path.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-6119) Add IndexWriter.getTotalNewBytesWritten
[ https://issues.apache.org/jira/browse/LUCENE-6119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael McCandless updated LUCENE-6119: --- Attachment: LUCENE-6119.patch OK new patch with a completely different approach, moving the tracking under Directory. I added a new AdaptiveRateLimitedDirectoryWrapper: it watches the average bytes/sec written by non-merges, and then (based on a multiplier) sets the merge throttling accordingly. It uses a rolling timestamps window of the last 1 GB of writes with 1 MB resolution, and lets you set min/max on the merge throttle. Also, I removed RateLimitedDirectoryWrapper: I think it's dangerous how it encourages you to throttle anything except merges, and encourages you to just set a fixed rate (one size does NOT fit all...). If you want similar behavior you can use AdaptiveRateLimitedDirectoryWrapper but set min and max to the same value. Add IndexWriter.getTotalNewBytesWritten --- Key: LUCENE-6119 URL: https://issues.apache.org/jira/browse/LUCENE-6119 Project: Lucene - Core Issue Type: Improvement Reporter: Michael McCandless Assignee: Michael McCandless Fix For: 5.0, Trunk Attachments: LUCENE-6119.patch, LUCENE-6119.patch This method returns number of incoming bytes IW has written since it was opened, excluding merging. It tracks flushed segments, new commits (segments_N), incoming files/segments by addIndexes, newly written live docs / doc values updates files. It's an easy statistic for IW to track and should be useful to help applications more intelligently set defaults for IO throttling (RateLimiter). For example, an application that does hardly any indexing but finally triggered a large merge can afford to heavily throttle that large merge so it won't interfere with ongoing searches. But an application that's causing IW to write new bytes at 50 MB/sec must set a correspondingly higher IO throttling otherwise merges will clearly fall behind. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-6119) Add AdaptiveRateLimitedDirectoryWrapper
[ https://issues.apache.org/jira/browse/LUCENE-6119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael McCandless updated LUCENE-6119: --- Summary: Add AdaptiveRateLimitedDirectoryWrapper (was: Add IndexWriter.getTotalNewBytesWritten) Add AdaptiveRateLimitedDirectoryWrapper --- Key: LUCENE-6119 URL: https://issues.apache.org/jira/browse/LUCENE-6119 Project: Lucene - Core Issue Type: Improvement Reporter: Michael McCandless Assignee: Michael McCandless Fix For: 5.0, Trunk Attachments: LUCENE-6119.patch, LUCENE-6119.patch This method returns number of incoming bytes IW has written since it was opened, excluding merging. It tracks flushed segments, new commits (segments_N), incoming files/segments by addIndexes, newly written live docs / doc values updates files. It's an easy statistic for IW to track and should be useful to help applications more intelligently set defaults for IO throttling (RateLimiter). For example, an application that does hardly any indexing but finally triggered a large merge can afford to heavily throttle that large merge so it won't interfere with ongoing searches. But an application that's causing IW to write new bytes at 50 MB/sec must set a correspondingly higher IO throttling otherwise merges will clearly fall behind. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-6324) Set finite default timeouts for select and update
[
https://issues.apache.org/jira/browse/SOLR-6324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14251594#comment-14251594
]
Ramkumar Aiyengar commented on SOLR-6324:
-
I did look at seeing what I can do about optimize, but couldn't come up with
any approach short of rearchitecting optimize to be async which I didn't have
time for. The other possibility here was that we apply a different timeout on a
per request basis (which btw is generally useful and something i wished for),
but that ran to a dead end as well -- I don't recall why now..
The stale check removal does force an idle timeout, but the connection timeouts
still need to be made finite?
Set finite default timeouts for select and update
-
Key: SOLR-6324
URL: https://issues.apache.org/jira/browse/SOLR-6324
Project: Solr
Issue Type: Improvement
Components: search, update
Reporter: Ramkumar Aiyengar
Assignee: Mark Miller
Priority: Minor
Fix For: 5.0, Trunk
Currently {{HttpShardHandlerFactory}} and {{UpdateShardHandler}} default to
infinite timeouts for socket connection and read. This can lead to
undesirable behaviour, for example, if a machine crashes, then searches in
progress will wait forever for a result to come back and end up using threads
which will only get terminated at shutdown.
We should have some finite default, however conservative it might be. These
parameters are already configurable, so for expert uses, they can be
increased if necessary anyway.
Will attach a patch to set connection timeout to 60s and read timeout to
600s, but I am not too concerned about the actual value as long as there is
one.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5951) Detect when index is on SSD and set dynamic defaults
[
https://issues.apache.org/jira/browse/LUCENE-5951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14251626#comment-14251626
]
Uwe Schindler commented on LUCENE-5951:
---
I am interested to see, if the detection works correctly on Policeman Jenkins.
This machine has an SSD, so what is the best way to see from test output if it
detected an SSD? To me the algorithm looks correct!
{noformat}
serv1:/sys/block/sdc# mount
/dev/md1 on / type ext3 (rw)
proc on /proc type proc (rw)
sysfs on /sys type sysfs (rw,noexec,nosuid,nodev)
none on /sys/fs/fuse/connections type fusectl (rw)
none on /sys/kernel/debug type debugfs (rw)
none on /sys/kernel/security type securityfs (rw)
udev on /dev type devtmpfs (rw,mode=0755)
devpts on /dev/pts type devpts (rw,noexec,nosuid,gid=5,mode=0620)
tmpfs on /run type tmpfs (rw,noexec,nosuid,size=10%,mode=0755)
none on /run/lock type tmpfs (rw,noexec,nosuid,nodev,size=5242880)
none on /run/shm type tmpfs (rw,nosuid,nodev)
/dev/md0 on /boot type ext2 (rw)
/dev/sdc1 on /mnt/ssd type ext4 (rw,noatime,discard)
serv1:/sys/block/sdc# cat /sys/block/sda/queue/rotational
1
serv1:/sys/block/sdc# cat /sys/block/sdb/queue/rotational
1
serv1:/sys/block/sdc# cat /sys/block/sdc/queue/rotational
0
serv1:/sys/block/sdc#
{noformat}
Detect when index is on SSD and set dynamic defaults
Key: LUCENE-5951
URL: https://issues.apache.org/jira/browse/LUCENE-5951
Project: Lucene - Core
Issue Type: Improvement
Reporter: Michael McCandless
Assignee: Michael McCandless
Attachments: LUCENE-5951.patch, LUCENE-5951.patch, LUCENE-5951.patch,
LUCENE-5951.patch, LUCENE-5951.patch, LUCENE-5951.patch
E.g. ConcurrentMergeScheduler should default maxMergeThreads to 3 if it's on
SSD and 1 if it's on spinning disks.
I think the new NIO2 APIs can let us figure out which device we are mounted
on, and from there maybe we can do os-specific stuff e.g. look at
/sys/block/dev/queue/rotational to see if it's spinning storage or not ...
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (LUCENE-5951) Detect when index is on SSD and set dynamic defaults
[
https://issues.apache.org/jira/browse/LUCENE-5951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14251626#comment-14251626
]
Uwe Schindler edited comment on LUCENE-5951 at 12/18/14 1:11 PM:
-
I am interested to see, if the detection works correctly on Policeman Jenkins.
This machine has an SSD, so what is the best way to see from test output if it
detected an SSD? To me the algorithm looks correct!
{noformat}
serv1:/sys/block/sdc# mount
/dev/md1 on / type ext3 (rw)
proc on /proc type proc (rw)
sysfs on /sys type sysfs (rw,noexec,nosuid,nodev)
none on /sys/fs/fuse/connections type fusectl (rw)
none on /sys/kernel/debug type debugfs (rw)
none on /sys/kernel/security type securityfs (rw)
udev on /dev type devtmpfs (rw,mode=0755)
devpts on /dev/pts type devpts (rw,noexec,nosuid,gid=5,mode=0620)
tmpfs on /run type tmpfs (rw,noexec,nosuid,size=10%,mode=0755)
none on /run/lock type tmpfs (rw,noexec,nosuid,nodev,size=5242880)
none on /run/shm type tmpfs (rw,nosuid,nodev)
/dev/md0 on /boot type ext2 (rw)
/dev/sdc1 on /mnt/ssd type ext4 (rw,noatime,discard)
serv1:/sys/block/sdc# cat /sys/block/sda/queue/rotational
1
serv1:/sys/block/sdc# cat /sys/block/sdb/queue/rotational
1
serv1:/sys/block/sdc# cat /sys/block/sdc/queue/rotational
0
serv1:/sys/block/sdc#
{noformat}
DYI: The Worksspace is on /mnt/ssd.
was (Author: thetaphi):
I am interested to see, if the detection works correctly on Policeman Jenkins.
This machine has an SSD, so what is the best way to see from test output if it
detected an SSD? To me the algorithm looks correct!
{noformat}
serv1:/sys/block/sdc# mount
/dev/md1 on / type ext3 (rw)
proc on /proc type proc (rw)
sysfs on /sys type sysfs (rw,noexec,nosuid,nodev)
none on /sys/fs/fuse/connections type fusectl (rw)
none on /sys/kernel/debug type debugfs (rw)
none on /sys/kernel/security type securityfs (rw)
udev on /dev type devtmpfs (rw,mode=0755)
devpts on /dev/pts type devpts (rw,noexec,nosuid,gid=5,mode=0620)
tmpfs on /run type tmpfs (rw,noexec,nosuid,size=10%,mode=0755)
none on /run/lock type tmpfs (rw,noexec,nosuid,nodev,size=5242880)
none on /run/shm type tmpfs (rw,nosuid,nodev)
/dev/md0 on /boot type ext2 (rw)
/dev/sdc1 on /mnt/ssd type ext4 (rw,noatime,discard)
serv1:/sys/block/sdc# cat /sys/block/sda/queue/rotational
1
serv1:/sys/block/sdc# cat /sys/block/sdb/queue/rotational
1
serv1:/sys/block/sdc# cat /sys/block/sdc/queue/rotational
0
serv1:/sys/block/sdc#
{noformat}
Detect when index is on SSD and set dynamic defaults
Key: LUCENE-5951
URL: https://issues.apache.org/jira/browse/LUCENE-5951
Project: Lucene - Core
Issue Type: Improvement
Reporter: Michael McCandless
Assignee: Michael McCandless
Attachments: LUCENE-5951.patch, LUCENE-5951.patch, LUCENE-5951.patch,
LUCENE-5951.patch, LUCENE-5951.patch, LUCENE-5951.patch
E.g. ConcurrentMergeScheduler should default maxMergeThreads to 3 if it's on
SSD and 1 if it's on spinning disks.
I think the new NIO2 APIs can let us figure out which device we are mounted
on, and from there maybe we can do os-specific stuff e.g. look at
/sys/block/dev/queue/rotational to see if it's spinning storage or not ...
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5951) Detect when index is on SSD and set dynamic defaults
[ https://issues.apache.org/jira/browse/LUCENE-5951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14251642#comment-14251642 ] Uwe Schindler commented on LUCENE-5951: --- I only found out that our detection *may* not work with LVM or MD devices, unfortunately I cannot try it out: http://lkml.iu.edu/hypermail/linux/kernel/1202.2/01578.html Theoretically, the software RAID should pass this flag down unmodified, if *all* devices are rotational or not. But it seems, it doesn't. For the policeman machine, sda and sdb (both rotational) together create an md0 mirror RAID device, which reports rotational=1. This is good, but I am not sure if this works if there are really SSDs as both raid devives (some people do this using RAID0 devices to speed up sequential reads). Maybe somebody else can report back, but I think the linux kernel puts rotational=1 for RAID devices as a fixed value or this has changed in the meantime. Detect when index is on SSD and set dynamic defaults Key: LUCENE-5951 URL: https://issues.apache.org/jira/browse/LUCENE-5951 Project: Lucene - Core Issue Type: Improvement Reporter: Michael McCandless Assignee: Michael McCandless Attachments: LUCENE-5951.patch, LUCENE-5951.patch, LUCENE-5951.patch, LUCENE-5951.patch, LUCENE-5951.patch, LUCENE-5951.patch E.g. ConcurrentMergeScheduler should default maxMergeThreads to 3 if it's on SSD and 1 if it's on spinning disks. I think the new NIO2 APIs can let us figure out which device we are mounted on, and from there maybe we can do os-specific stuff e.g. look at /sys/block/dev/queue/rotational to see if it's spinning storage or not ... -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (LUCENE-5951) Detect when index is on SSD and set dynamic defaults
[
https://issues.apache.org/jira/browse/LUCENE-5951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14251626#comment-14251626
]
Uwe Schindler edited comment on LUCENE-5951 at 12/18/14 1:25 PM:
-
I am interested to see, if the detection works correctly on Policeman Jenkins.
This machine has an SSD, so what is the best way to see from test output if it
detected an SSD? To me the algorithm looks correct!
{noformat}
serv1:~# mount
/dev/md1 on / type ext3 (rw)
proc on /proc type proc (rw)
sysfs on /sys type sysfs (rw,noexec,nosuid,nodev)
none on /sys/fs/fuse/connections type fusectl (rw)
none on /sys/kernel/debug type debugfs (rw)
none on /sys/kernel/security type securityfs (rw)
udev on /dev type devtmpfs (rw,mode=0755)
devpts on /dev/pts type devpts (rw,noexec,nosuid,gid=5,mode=0620)
tmpfs on /run type tmpfs (rw,noexec,nosuid,size=10%,mode=0755)
none on /run/lock type tmpfs (rw,noexec,nosuid,nodev,size=5242880)
none on /run/shm type tmpfs (rw,nosuid,nodev)
/dev/md0 on /boot type ext2 (rw)
/dev/sdc1 on /mnt/ssd type ext4 (rw,noatime,discard)
serv1:~# cat /sys/block/sda/queue/rotational
1
serv1:~# cat /sys/block/sdb/queue/rotational
1
serv1:~# cat /sys/block/sdc/queue/rotational
0
serv1:~#
{noformat}
DYI: The Worksspace is on /mnt/ssd.
was (Author: thetaphi):
I am interested to see, if the detection works correctly on Policeman Jenkins.
This machine has an SSD, so what is the best way to see from test output if it
detected an SSD? To me the algorithm looks correct!
{noformat}
serv1:/sys/block/sdc# mount
/dev/md1 on / type ext3 (rw)
proc on /proc type proc (rw)
sysfs on /sys type sysfs (rw,noexec,nosuid,nodev)
none on /sys/fs/fuse/connections type fusectl (rw)
none on /sys/kernel/debug type debugfs (rw)
none on /sys/kernel/security type securityfs (rw)
udev on /dev type devtmpfs (rw,mode=0755)
devpts on /dev/pts type devpts (rw,noexec,nosuid,gid=5,mode=0620)
tmpfs on /run type tmpfs (rw,noexec,nosuid,size=10%,mode=0755)
none on /run/lock type tmpfs (rw,noexec,nosuid,nodev,size=5242880)
none on /run/shm type tmpfs (rw,nosuid,nodev)
/dev/md0 on /boot type ext2 (rw)
/dev/sdc1 on /mnt/ssd type ext4 (rw,noatime,discard)
serv1:/sys/block/sdc# cat /sys/block/sda/queue/rotational
1
serv1:/sys/block/sdc# cat /sys/block/sdb/queue/rotational
1
serv1:/sys/block/sdc# cat /sys/block/sdc/queue/rotational
0
serv1:/sys/block/sdc#
{noformat}
DYI: The Worksspace is on /mnt/ssd.
Detect when index is on SSD and set dynamic defaults
Key: LUCENE-5951
URL: https://issues.apache.org/jira/browse/LUCENE-5951
Project: Lucene - Core
Issue Type: Improvement
Reporter: Michael McCandless
Assignee: Michael McCandless
Attachments: LUCENE-5951.patch, LUCENE-5951.patch, LUCENE-5951.patch,
LUCENE-5951.patch, LUCENE-5951.patch, LUCENE-5951.patch
E.g. ConcurrentMergeScheduler should default maxMergeThreads to 3 if it's on
SSD and 1 if it's on spinning disks.
I think the new NIO2 APIs can let us figure out which device we are mounted
on, and from there maybe we can do os-specific stuff e.g. look at
/sys/block/dev/queue/rotational to see if it's spinning storage or not ...
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-6121) Fix CachingTokenFilter to propagate reset() the first time
[ https://issues.apache.org/jira/browse/LUCENE-6121?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14251653#comment-14251653 ] David Smiley commented on LUCENE-6121: -- The main challenge with doing exactly what the subject calls for, propagating reset(), is backwards compatibility. Any user of this would have to know, starting with Lucene 5, that they should call reset() on this filter not what it contains. That's easy to overlook and will compile and could lead to some head-scratching periods when an exception first triggers. [~rcmuir] thought maybe a new class is a solution. I think that'd be a shame when the name of this one is perfectly good, and already known to people. I propose instead that the existing constructor be made private, that a new protected one appear (for subclassing) with a boolean resetInput, and a public static wrap(input) method appear to be used instead of the old constructor. Someone upgrading to Lucene 5 will see a compile error forcing them to look at the new API, and we'll spell out that they should now call reset() on this filter, not the input. Fix CachingTokenFilter to propagate reset() the first time -- Key: LUCENE-6121 URL: https://issues.apache.org/jira/browse/LUCENE-6121 Project: Lucene - Core Issue Type: Improvement Reporter: David Smiley Assignee: David Smiley Fix For: 5.0, Trunk CachingTokenFilter should have been propagating reset() _but only the first time_ and thus you would then use CachingTokenFilter in a more normal way – wrap it and call reset() then increment in a loop, etc., instead of knowing you need to reset() on what it wraps but not this token filter itself. That's weird. It's ab-normal for a TokenFilter to never propagate reset, so every user of CachingTokenFilter to date has worked around this by calling reset() on the underlying input instead of the final wrapping token filter (CachingTokenFilter in this case). -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-6770) Add/edit param sets and use them in Requests
[
https://issues.apache.org/jira/browse/SOLR-6770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14251655#comment-14251655
]
David Smiley commented on SOLR-6770:
+1 Nice! Looking forward to this.
Add/edit param sets and use them in Requests
Key: SOLR-6770
URL: https://issues.apache.org/jira/browse/SOLR-6770
Project: Solr
Issue Type: Sub-task
Reporter: Noble Paul
Assignee: Noble Paul
Make it possible to define paramsets and use them directly in requests
example
{code}
curl http://localhost:8983/solr/collection1/config/params -H
'Content-type:application/json' -d '{
create : {name ,x
val: {
a:A val,
b: B val}
},
update : {name ,y
val: {
x:X val,
Y: Y val}
},
delete : z
}'
#do a GET to view all the configured params
curl http://localhost:8983/solr/collection1/config/params
#or GET with a specific name to get only one set of params
curl http://localhost:8983/solr/collection1/config/params/x
{code}
This data will be stored in conf/paramsets.json
This is used requesttime and adding/editing params will not result in core
reload and it will have no impact on the performance
example usage http://localhost/solr/collection/select?useParams=x
or it can be directly configured with a request handler as follows
{code}
requestHandler name=/dump1 class=DumpRequestHandler useParams=x/
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-6554) Speed up overseer operations for collections with stateFormat 1
[ https://issues.apache.org/jira/browse/SOLR-6554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shalin Shekhar Mangar updated SOLR-6554: Attachment: SOLR-6554-workqueue-fixes.patch Refactored a bit more. ZkWriteListener is called ZkWriteCallback instead. There was no need of a onFlushBefore and onFlushAfter methods. A single onWrite method is enough. I am not very happy with this callback API but this is the best I could come up. The other option would be to move the batching logic back to Overseer. Speed up overseer operations for collections with stateFormat 1 - Key: SOLR-6554 URL: https://issues.apache.org/jira/browse/SOLR-6554 Project: Solr Issue Type: Improvement Components: SolrCloud Affects Versions: 5.0, Trunk Reporter: Shalin Shekhar Mangar Attachments: SOLR-6554-batching-refactor.patch, SOLR-6554-batching-refactor.patch, SOLR-6554-batching-refactor.patch, SOLR-6554-batching-refactor.patch, SOLR-6554-workqueue-fixes.patch, SOLR-6554-workqueue-fixes.patch, SOLR-6554-workqueue-fixes.patch, SOLR-6554.patch, SOLR-6554.patch, SOLR-6554.patch, SOLR-6554.patch, SOLR-6554.patch, SOLR-6554.patch, SOLR-6554.patch, SOLR-6554.patch Right now (after SOLR-5473 was committed), a node watches a collection only if stateFormat=1 or if that node hosts at least one core belonging to that collection. This means that a node which is the overseer operates on all collections but watches only a few. So any read goes directly to zookeeper which slows down overseer operations. Let's have the overseer node watch all collections always and never remove those watches (except when the collection itself is deleted). -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-6293) Solr tests have gotten much slower.
[ https://issues.apache.org/jira/browse/SOLR-6293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14251669#comment-14251669 ] Mark Miller commented on SOLR-6293: --- It seems a lot of this may have come from SSL. See my recent comments in SOLR-4509. Solr tests have gotten much slower. --- Key: SOLR-6293 URL: https://issues.apache.org/jira/browse/SOLR-6293 Project: Solr Issue Type: Test Reporter: Mark Miller Assignee: Mark Miller I run tests with 8 jvms before I took a 2-3 month hiatus from Lucene / Solr, Solr tests ran in about 9-10 minutes on my machine. Now it's 16 minutes. This issue is a top level issue to track looking into what has caused the additional time, if it makes sense, or if there is stuff we can move to nightly. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5951) Detect when index is on SSD and set dynamic defaults
[ https://issues.apache.org/jira/browse/LUCENE-5951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14251676#comment-14251676 ] Robert Muir commented on LUCENE-5951: - Then those are bugs in the linux kernel. its not our problem. the worst that happens, is you get the same behavior as today. People seem to have difficulty understanding this. Detect when index is on SSD and set dynamic defaults Key: LUCENE-5951 URL: https://issues.apache.org/jira/browse/LUCENE-5951 Project: Lucene - Core Issue Type: Improvement Reporter: Michael McCandless Assignee: Michael McCandless Attachments: LUCENE-5951.patch, LUCENE-5951.patch, LUCENE-5951.patch, LUCENE-5951.patch, LUCENE-5951.patch, LUCENE-5951.patch E.g. ConcurrentMergeScheduler should default maxMergeThreads to 3 if it's on SSD and 1 if it's on spinning disks. I think the new NIO2 APIs can let us figure out which device we are mounted on, and from there maybe we can do os-specific stuff e.g. look at /sys/block/dev/queue/rotational to see if it's spinning storage or not ... -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-6121) Fix CachingTokenFilter to propagate reset() the first time
[ https://issues.apache.org/jira/browse/LUCENE-6121?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14251678#comment-14251678 ] Robert Muir commented on LUCENE-6121: - As i said on the previous issue, I am -1 to this boolean. Just fix the behavior of the class. Its a major release, we can break anything. Fix CachingTokenFilter to propagate reset() the first time -- Key: LUCENE-6121 URL: https://issues.apache.org/jira/browse/LUCENE-6121 Project: Lucene - Core Issue Type: Improvement Reporter: David Smiley Assignee: David Smiley Fix For: 5.0, Trunk CachingTokenFilter should have been propagating reset() _but only the first time_ and thus you would then use CachingTokenFilter in a more normal way – wrap it and call reset() then increment in a loop, etc., instead of knowing you need to reset() on what it wraps but not this token filter itself. That's weird. It's ab-normal for a TokenFilter to never propagate reset, so every user of CachingTokenFilter to date has worked around this by calling reset() on the underlying input instead of the final wrapping token filter (CachingTokenFilter in this case). -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5951) Detect when index is on SSD and set dynamic defaults
[ https://issues.apache.org/jira/browse/LUCENE-5951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14251690#comment-14251690 ] Uwe Schindler commented on LUCENE-5951: --- [~rcmuir]: I just wanted to be sure that the inverse does not happen: So a RAID device of spinning disk suddenly returning non-spinning because of another bug in linux. My test has verified that it returns rotational=1 for my example - so I am happy. I just wanted to write this down here to have a reference that someone looked at it. There are other things to mention: the ssd/rotational flag also does not work correctly in VMware's VShere or VirtualBOX unless the creator of the virtual machine selects SSD as virtual device type (in VirtualBox you can do this). I created here a virtual machine with an virtual SSD sitting on a spinning disk... So we should document this that the whole detection only works correct, if you use raw disks on metal hardware. People should also be aware that their VSphere infrastructure is configured correctly. :-) Detect when index is on SSD and set dynamic defaults Key: LUCENE-5951 URL: https://issues.apache.org/jira/browse/LUCENE-5951 Project: Lucene - Core Issue Type: Improvement Reporter: Michael McCandless Assignee: Michael McCandless Attachments: LUCENE-5951.patch, LUCENE-5951.patch, LUCENE-5951.patch, LUCENE-5951.patch, LUCENE-5951.patch, LUCENE-5951.patch E.g. ConcurrentMergeScheduler should default maxMergeThreads to 3 if it's on SSD and 1 if it's on spinning disks. I think the new NIO2 APIs can let us figure out which device we are mounted on, and from there maybe we can do os-specific stuff e.g. look at /sys/block/dev/queue/rotational to see if it's spinning storage or not ... -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (LUCENE-6122) explicitly always set CMS values in LuceneTestCase.newIndexWriterConfig
Robert Muir created LUCENE-6122: --- Summary: explicitly always set CMS values in LuceneTestCase.newIndexWriterConfig Key: LUCENE-6122 URL: https://issues.apache.org/jira/browse/LUCENE-6122 Project: Lucene - Core Issue Type: Test Reporter: Robert Muir This is just like not using FSDirectory.open in tests. The same seed should do the same stuff on different machines. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (SOLR-6865) Upgrade HttpClient to 4.4
Shawn Heisey created SOLR-6865: -- Summary: Upgrade HttpClient to 4.4 Key: SOLR-6865 URL: https://issues.apache.org/jira/browse/SOLR-6865 Project: Solr Issue Type: Task Affects Versions: 5.0 Reporter: Shawn Heisey Priority: Minor Fix For: 5.0, Trunk HttpClient 4.4 has been released. 5.0 seems like a good time to upgrade. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-6865) Upgrade HttpClient to 4.4
[ https://issues.apache.org/jira/browse/SOLR-6865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14251694#comment-14251694 ] Shawn Heisey commented on SOLR-6865: It will take me a couple of days to do the required testing to see if any code changes are required. If anyone else wants to swipe this issue, feel free. Upgrade HttpClient to 4.4 - Key: SOLR-6865 URL: https://issues.apache.org/jira/browse/SOLR-6865 Project: Solr Issue Type: Task Affects Versions: 5.0 Reporter: Shawn Heisey Priority: Minor Fix For: 5.0, Trunk HttpClient 4.4 has been released. 5.0 seems like a good time to upgrade. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5951) Detect when index is on SSD and set dynamic defaults
[ https://issues.apache.org/jira/browse/LUCENE-5951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14251695#comment-14251695 ] Robert Muir commented on LUCENE-5951: - the worst that happens, is you get the same behavior as today. Detect when index is on SSD and set dynamic defaults Key: LUCENE-5951 URL: https://issues.apache.org/jira/browse/LUCENE-5951 Project: Lucene - Core Issue Type: Improvement Reporter: Michael McCandless Assignee: Michael McCandless Attachments: LUCENE-5951.patch, LUCENE-5951.patch, LUCENE-5951.patch, LUCENE-5951.patch, LUCENE-5951.patch, LUCENE-5951.patch E.g. ConcurrentMergeScheduler should default maxMergeThreads to 3 if it's on SSD and 1 if it's on spinning disks. I think the new NIO2 APIs can let us figure out which device we are mounted on, and from there maybe we can do os-specific stuff e.g. look at /sys/block/dev/queue/rotational to see if it's spinning storage or not ... -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-6121) Fix CachingTokenFilter to propagate reset() the first time
[ https://issues.apache.org/jira/browse/LUCENE-6121?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14251736#comment-14251736 ] David Smiley commented on LUCENE-6121: -- Ok, nevermind the protected constructor with the boolean. How do you feel about wrap(input)? I like it. This ensures people get a compile error triggering them to look at the API. The existing constructor could be protected. Fix CachingTokenFilter to propagate reset() the first time -- Key: LUCENE-6121 URL: https://issues.apache.org/jira/browse/LUCENE-6121 Project: Lucene - Core Issue Type: Improvement Reporter: David Smiley Assignee: David Smiley Fix For: 5.0, Trunk CachingTokenFilter should have been propagating reset() _but only the first time_ and thus you would then use CachingTokenFilter in a more normal way – wrap it and call reset() then increment in a loop, etc., instead of knowing you need to reset() on what it wraps but not this token filter itself. That's weird. It's ab-normal for a TokenFilter to never propagate reset, so every user of CachingTokenFilter to date has worked around this by calling reset() on the underlying input instead of the final wrapping token filter (CachingTokenFilter in this case). -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-6787) API to manage blobs in Solr
[
https://issues.apache.org/jira/browse/SOLR-6787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14251750#comment-14251750
]
Alexandre Rafalovitch commented on SOLR-6787:
-
Why do you need a user to create the collection first? Especially since the
config/etc is hardcoded and it needs to be present on every node? Wouldn't it
make sense for collection to be just auto-created on first access? Or is there
a specific flexibility that only user can decide on?
Also, if you have heterogeneous environment, the current implementation would
require a union of all blobs, right? As in, every node will need to have all
possible blobs/jars, not just the jars the collections on that node have.
Finally, does the _.system_ collection show up on the Admin UI? Are there
implications/restrictions of that?
API to manage blobs in Solr
Key: SOLR-6787
URL: https://issues.apache.org/jira/browse/SOLR-6787
Project: Solr
Issue Type: Sub-task
Reporter: Noble Paul
Assignee: Noble Paul
Fix For: 5.0, Trunk
Attachments: SOLR-6787.patch, SOLR-6787.patch
A special collection called .system needs to be created by the user to
store/manage blobs. The schema/solrconfig of that collection need to be
automatically supplied by the system so that there are no errors
APIs need to be created to manage the content of that collection
{code}
#create your .system collection first
http://localhost:8983/solr/admin/collections?action=CREATEname=.systemreplicationFactor=2
#The config for this collection is automatically created . numShards for this
collection is hardcoded to 1
#create a new jar or add a new version of a jar
curl -X POST -H 'Content-Type: application/octet-stream' --data-binary
@mycomponent.jar http://localhost:8983/solr/.system/blob/mycomponent
# GET on the end point would give a list of jars and other details
curl http://localhost:8983/solr/.system/blob
# GET on the end point with jar name would give details of various versions
of the available jars
curl http://localhost:8983/solr/.system/blob/mycomponent
# GET on the end point with jar name and version with a wt=filestream to get
the actual file
curl http://localhost:8983/solr/.system/blob/mycomponent/1?wt=filestream
mycomponent.1.jar
# GET on the end point with jar name and wt=filestream to get the latest
version of the file
curl http://localhost:8983/solr/.system/blob/mycomponent?wt=filestream
mycomponent.jar
{code}
Please note that the jars are never deleted. a new version is added to the
system everytime a new jar is posted for the name. You must use the standard
delete commands to delete the old entries
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-6121) Fix CachingTokenFilter to propagate reset() the first time
[ https://issues.apache.org/jira/browse/LUCENE-6121?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14251761#comment-14251761 ] Robert Muir commented on LUCENE-6121: - If this one has the name you want, then we just change the behavior and thats it. Please, no booleans, factories, builders, or other unnecessary abstractions. Static factory methods are terrible. You cannot tell what they are doing... we should always avoid this kind of garbage. Fix CachingTokenFilter to propagate reset() the first time -- Key: LUCENE-6121 URL: https://issues.apache.org/jira/browse/LUCENE-6121 Project: Lucene - Core Issue Type: Improvement Reporter: David Smiley Assignee: David Smiley Fix For: 5.0, Trunk CachingTokenFilter should have been propagating reset() _but only the first time_ and thus you would then use CachingTokenFilter in a more normal way – wrap it and call reset() then increment in a loop, etc., instead of knowing you need to reset() on what it wraps but not this token filter itself. That's weird. It's ab-normal for a TokenFilter to never propagate reset, so every user of CachingTokenFilter to date has worked around this by calling reset() on the underlying input instead of the final wrapping token filter (CachingTokenFilter in this case). -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-6554) Speed up overseer operations for collections with stateFormat 1
[ https://issues.apache.org/jira/browse/SOLR-6554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14251776#comment-14251776 ] ASF subversion and git services commented on SOLR-6554: --- Commit 1646474 from sha...@apache.org in branch 'dev/trunk' [ https://svn.apache.org/r1646474 ] SOLR-6554: Fix work queue handling Speed up overseer operations for collections with stateFormat 1 - Key: SOLR-6554 URL: https://issues.apache.org/jira/browse/SOLR-6554 Project: Solr Issue Type: Improvement Components: SolrCloud Affects Versions: 5.0, Trunk Reporter: Shalin Shekhar Mangar Attachments: SOLR-6554-batching-refactor.patch, SOLR-6554-batching-refactor.patch, SOLR-6554-batching-refactor.patch, SOLR-6554-batching-refactor.patch, SOLR-6554-workqueue-fixes.patch, SOLR-6554-workqueue-fixes.patch, SOLR-6554-workqueue-fixes.patch, SOLR-6554.patch, SOLR-6554.patch, SOLR-6554.patch, SOLR-6554.patch, SOLR-6554.patch, SOLR-6554.patch, SOLR-6554.patch, SOLR-6554.patch Right now (after SOLR-5473 was committed), a node watches a collection only if stateFormat=1 or if that node hosts at least one core belonging to that collection. This means that a node which is the overseer operates on all collections but watches only a few. So any read goes directly to zookeeper which slows down overseer operations. Let's have the overseer node watch all collections always and never remove those watches (except when the collection itself is deleted). -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5951) Detect when index is on SSD and set dynamic defaults
[ https://issues.apache.org/jira/browse/LUCENE-5951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14251783#comment-14251783 ] Uwe Schindler commented on LUCENE-5951: --- If you have a spinning disk and you falsefully detect it as an SSD then its a problem... Detect when index is on SSD and set dynamic defaults Key: LUCENE-5951 URL: https://issues.apache.org/jira/browse/LUCENE-5951 Project: Lucene - Core Issue Type: Improvement Reporter: Michael McCandless Assignee: Michael McCandless Attachments: LUCENE-5951.patch, LUCENE-5951.patch, LUCENE-5951.patch, LUCENE-5951.patch, LUCENE-5951.patch, LUCENE-5951.patch E.g. ConcurrentMergeScheduler should default maxMergeThreads to 3 if it's on SSD and 1 if it's on spinning disks. I think the new NIO2 APIs can let us figure out which device we are mounted on, and from there maybe we can do os-specific stuff e.g. look at /sys/block/dev/queue/rotational to see if it's spinning storage or not ... -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Solr-trunk-Linux (32bit/jdk1.8.0_20) - Build # 11768 - Failure!
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/11768/ Java: 32bit/jdk1.8.0_20 -client -XX:+UseSerialGC (asserts: false) 1 tests failed. FAILED: org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testDistribSearch Error Message: expected:...9942E65A94EC1-001/te[mpDir-003/solrj_test_core_props_shard1_replica1] but was:...9942E65A94EC1-001/te[stPropertyParamsForCreate-001/instanceDir-fmchl] Stack Trace: org.junit.ComparisonFailure: expected:...9942E65A94EC1-001/te[mpDir-003/solrj_test_core_props_shard1_replica1] but was:...9942E65A94EC1-001/te[stPropertyParamsForCreate-001/instanceDir-fmchl] at __randomizedtesting.SeedInfo.seed([8219942E65A94EC1:3FF1A3612F62EFD]:0) at org.junit.Assert.assertEquals(Assert.java:125) at org.junit.Assert.assertEquals(Assert.java:147) at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.checkInstanceDirs(CollectionsAPIDistributedZkTest.java:1135) at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testCollectionsAPI(CollectionsAPIDistributedZkTest.java:883) at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.doTest(CollectionsAPIDistributedZkTest.java:204) at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:869) at sun.reflect.GeneratedMethodAccessor48.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:483) at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618) at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827) at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863) at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53) at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55) at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365) at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798) at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458) at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836) at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738) at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772) at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) at
[jira] [Commented] (SOLR-4509) Disable HttpClient stale check for performance and fewer spurious connection errors.
[ https://issues.apache.org/jira/browse/SOLR-4509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14251815#comment-14251815 ] Mark Miller commented on SOLR-4509: --- bq. There is a rather interesting affect from this change - the 'close idle' thread will trump the socket timeout. I gathered this because when I run the close idle connection thread at a lower interval than the socket timeout, slow SSL tests where hitting a socket error (no response from server) rather than the socket timeout they would hit if I raised the close idle connection thread interval. Looking at the code though, I think this idle connection close should only apply to unused connections in the pool. I have to look closer at what is happening. Disable HttpClient stale check for performance and fewer spurious connection errors. Key: SOLR-4509 URL: https://issues.apache.org/jira/browse/SOLR-4509 Project: Solr Issue Type: Improvement Components: search Environment: 5 node SmartOS cluster (all nodes living in same global zone - i.e. same physical machine) Reporter: Ryan Zezeski Assignee: Mark Miller Priority: Minor Fix For: 5.0, Trunk Attachments: IsStaleTime.java, SOLR-4509-4_4_0.patch, SOLR-4509.patch, SOLR-4509.patch, SOLR-4509.patch, SOLR-4509.patch, SOLR-4509.patch, SOLR-4509.patch, baremetal-stale-nostale-med-latency.dat, baremetal-stale-nostale-med-latency.svg, baremetal-stale-nostale-throughput.dat, baremetal-stale-nostale-throughput.svg By disabling the Apache HTTP Client stale check I've witnessed a 2-4x increase in throughput and reduction of over 100ms. This patch was made in the context of a project I'm leading, called Yokozuna, which relies on distributed search. Here's the patch on Yokozuna: https://github.com/rzezeski/yokozuna/pull/26 Here's a write-up I did on my findings: http://www.zinascii.com/2013/solr-distributed-search-and-the-stale-check.html I'm happy to answer any questions or make changes to the patch to make it acceptable. ReviewBoard: https://reviews.apache.org/r/28393/ -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5951) Detect when index is on SSD and set dynamic defaults
[ https://issues.apache.org/jira/browse/LUCENE-5951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14251833#comment-14251833 ] Robert Muir commented on LUCENE-5951: - I dont think its really a problem at all. its a heuristic for defaults. If there are bugs in the linux kernel, or virtualizers, or device drivers, its not our duty to fix that. Please, complain on the linux kernel list instead. today its a far bigger problem that we always falsely assume you have a spinning disk, and hurt performance on any modern hardware. Users can always set their merge threads etc explicitly. Detect when index is on SSD and set dynamic defaults Key: LUCENE-5951 URL: https://issues.apache.org/jira/browse/LUCENE-5951 Project: Lucene - Core Issue Type: Improvement Reporter: Michael McCandless Assignee: Michael McCandless Attachments: LUCENE-5951.patch, LUCENE-5951.patch, LUCENE-5951.patch, LUCENE-5951.patch, LUCENE-5951.patch, LUCENE-5951.patch E.g. ConcurrentMergeScheduler should default maxMergeThreads to 3 if it's on SSD and 1 if it's on spinning disks. I think the new NIO2 APIs can let us figure out which device we are mounted on, and from there maybe we can do os-specific stuff e.g. look at /sys/block/dev/queue/rotational to see if it's spinning storage or not ... -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5951) Detect when index is on SSD and set dynamic defaults
[ https://issues.apache.org/jira/browse/LUCENE-5951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14251878#comment-14251878 ] Uwe Schindler commented on LUCENE-5951: --- Sorry Robert, it is my personal decision to comment on this. It was not a complaint, just a notice, so anybody who wants to lookup more on this issue, to get the relevant information. My problem was just that some user with a misconfigured system could get sudenly the SSD optimization on a spinning disk and then his IO system gives up :-) So it should be documented and I just ask for some hints in the documentation, that one should take care to configure his virtual machines correctly. I DON'T COMPLAIN! But now I complain: Why are you attacking me? I just bring in here useful items that might help others. Detect when index is on SSD and set dynamic defaults Key: LUCENE-5951 URL: https://issues.apache.org/jira/browse/LUCENE-5951 Project: Lucene - Core Issue Type: Improvement Reporter: Michael McCandless Assignee: Michael McCandless Attachments: LUCENE-5951.patch, LUCENE-5951.patch, LUCENE-5951.patch, LUCENE-5951.patch, LUCENE-5951.patch, LUCENE-5951.patch E.g. ConcurrentMergeScheduler should default maxMergeThreads to 3 if it's on SSD and 1 if it's on spinning disks. I think the new NIO2 APIs can let us figure out which device we are mounted on, and from there maybe we can do os-specific stuff e.g. look at /sys/block/dev/queue/rotational to see if it's spinning storage or not ... -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS-MAVEN] Lucene-Solr-Maven-5.x #791: POMs out of sync
Build: https://builds.apache.org/job/Lucene-Solr-Maven-5.x/791/ 2 tests failed. FAILED: org.apache.solr.hadoop.MorphlineBasicMiniMRTest.testPathParts Error Message: Test abandoned because suite timeout was reached. Stack Trace: java.lang.Exception: Test abandoned because suite timeout was reached. at __randomizedtesting.SeedInfo.seed([965D5543C663DC82]:0) FAILED: org.apache.solr.hadoop.MorphlineBasicMiniMRTest.org.apache.solr.hadoop.MorphlineBasicMiniMRTest Error Message: Suite timeout exceeded (= 720 msec). Stack Trace: java.lang.Exception: Suite timeout exceeded (= 720 msec). at __randomizedtesting.SeedInfo.seed([965D5543C663DC82]:0) Build Log: [...truncated 53824 lines...] BUILD FAILED /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Maven-5.x/build.xml:552: The following error occurred while executing this line: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Maven-5.x/build.xml:204: The following error occurred while executing this line: : Java returned: 1 Total time: 390 minutes 59 seconds Build step 'Invoke Ant' marked build as failure Recording test results Email was triggered for: Failure Sending email for trigger: Failure - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-6554) Speed up overseer operations for collections with stateFormat 1
[ https://issues.apache.org/jira/browse/SOLR-6554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14251913#comment-14251913 ] ASF subversion and git services commented on SOLR-6554: --- Commit 1646493 from sha...@apache.org in branch 'dev/branches/branch_5x' [ https://svn.apache.org/r1646493 ] SOLR-6554: Speed up overseer operations Speed up overseer operations for collections with stateFormat 1 - Key: SOLR-6554 URL: https://issues.apache.org/jira/browse/SOLR-6554 Project: Solr Issue Type: Improvement Components: SolrCloud Affects Versions: 5.0, Trunk Reporter: Shalin Shekhar Mangar Attachments: SOLR-6554-batching-refactor.patch, SOLR-6554-batching-refactor.patch, SOLR-6554-batching-refactor.patch, SOLR-6554-batching-refactor.patch, SOLR-6554-workqueue-fixes.patch, SOLR-6554-workqueue-fixes.patch, SOLR-6554-workqueue-fixes.patch, SOLR-6554.patch, SOLR-6554.patch, SOLR-6554.patch, SOLR-6554.patch, SOLR-6554.patch, SOLR-6554.patch, SOLR-6554.patch, SOLR-6554.patch Right now (after SOLR-5473 was committed), a node watches a collection only if stateFormat=1 or if that node hosts at least one core belonging to that collection. This means that a node which is the overseer operates on all collections but watches only a few. So any read goes directly to zookeeper which slows down overseer operations. Let's have the overseer node watch all collections always and never remove those watches (except when the collection itself is deleted). -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-6787) API to manage blobs in Solr
[
https://issues.apache.org/jira/browse/SOLR-6787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14251925#comment-14251925
]
Yonik Seeley commented on SOLR-6787:
{quote}
why there would be a separate/different API to get/set the value of such a
field.
Tell me how can I do that today.
{quote}
That's my entire point... if it's hard to do X, look first into making X easier
rather than creating a different method.
At the end of the day, there may still be some little things that warrant
separate API calls to make it easier. But if anything is identified to make
generic APIs more usable, that's a win.
Another way to look at it: It appears you've created an API specific to
.system... but why should it be specific? If it's easier to manage blobs
this way, shouldn't users be able to use this on whatever collections / fields
they want?
API to manage blobs in Solr
Key: SOLR-6787
URL: https://issues.apache.org/jira/browse/SOLR-6787
Project: Solr
Issue Type: Sub-task
Reporter: Noble Paul
Assignee: Noble Paul
Fix For: 5.0, Trunk
Attachments: SOLR-6787.patch, SOLR-6787.patch
A special collection called .system needs to be created by the user to
store/manage blobs. The schema/solrconfig of that collection need to be
automatically supplied by the system so that there are no errors
APIs need to be created to manage the content of that collection
{code}
#create your .system collection first
http://localhost:8983/solr/admin/collections?action=CREATEname=.systemreplicationFactor=2
#The config for this collection is automatically created . numShards for this
collection is hardcoded to 1
#create a new jar or add a new version of a jar
curl -X POST -H 'Content-Type: application/octet-stream' --data-binary
@mycomponent.jar http://localhost:8983/solr/.system/blob/mycomponent
# GET on the end point would give a list of jars and other details
curl http://localhost:8983/solr/.system/blob
# GET on the end point with jar name would give details of various versions
of the available jars
curl http://localhost:8983/solr/.system/blob/mycomponent
# GET on the end point with jar name and version with a wt=filestream to get
the actual file
curl http://localhost:8983/solr/.system/blob/mycomponent/1?wt=filestream
mycomponent.1.jar
# GET on the end point with jar name and wt=filestream to get the latest
version of the file
curl http://localhost:8983/solr/.system/blob/mycomponent?wt=filestream
mycomponent.jar
{code}
Please note that the jars are never deleted. a new version is added to the
system everytime a new jar is posted for the name. You must use the standard
delete commands to delete the old entries
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (LUCENE-6123) BitDocIdSet.Builder.andNot is wrong
Adrien Grand created LUCENE-6123: Summary: BitDocIdSet.Builder.andNot is wrong Key: LUCENE-6123 URL: https://issues.apache.org/jira/browse/LUCENE-6123 Project: Lucene - Core Issue Type: Bug Reporter: Adrien Grand Assignee: Adrien Grand The 2nd part of the body references sparseSet instead of denseSet. We should test it better. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Resolved] (SOLR-6554) Speed up overseer operations for collections with stateFormat 1
[ https://issues.apache.org/jira/browse/SOLR-6554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shalin Shekhar Mangar resolved SOLR-6554. - Resolution: Fixed Fix Version/s: Trunk 5.0 Assignee: Shalin Shekhar Mangar This is fixed. Thanks everyone. Let's discuss stateFormat=2 as default for 5.0 in it's own separate issue. Speed up overseer operations for collections with stateFormat 1 - Key: SOLR-6554 URL: https://issues.apache.org/jira/browse/SOLR-6554 Project: Solr Issue Type: Improvement Components: SolrCloud Affects Versions: 5.0, Trunk Reporter: Shalin Shekhar Mangar Assignee: Shalin Shekhar Mangar Fix For: 5.0, Trunk Attachments: SOLR-6554-batching-refactor.patch, SOLR-6554-batching-refactor.patch, SOLR-6554-batching-refactor.patch, SOLR-6554-batching-refactor.patch, SOLR-6554-workqueue-fixes.patch, SOLR-6554-workqueue-fixes.patch, SOLR-6554-workqueue-fixes.patch, SOLR-6554.patch, SOLR-6554.patch, SOLR-6554.patch, SOLR-6554.patch, SOLR-6554.patch, SOLR-6554.patch, SOLR-6554.patch, SOLR-6554.patch Right now (after SOLR-5473 was committed), a node watches a collection only if stateFormat=1 or if that node hosts at least one core belonging to that collection. This means that a node which is the overseer operates on all collections but watches only a few. So any read goes directly to zookeeper which slows down overseer operations. Let's have the overseer node watch all collections always and never remove those watches (except when the collection itself is deleted). -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (LUCENE-6124) Fix broken close() methods
Robert Muir created LUCENE-6124:
---
Summary: Fix broken close() methods
Key: LUCENE-6124
URL: https://issues.apache.org/jira/browse/LUCENE-6124
Project: Lucene - Core
Issue Type: Bug
Reporter: Robert Muir
Closeable.close() says If the stream is already closed then invoking this
method has no effect..
But a lot of our code does not really respect that. If i add an extra close()
call in assertingcodec, it finds all kinds of bugs in codec code, for example:
{noformat}
[junit4] Tests with failures (first 10 out of 59):
[junit4] -
org.apache.lucene.index.TestCrashCausesCorruptIndex.testCrashCorruptsIndexing
[junit4] -
org.apache.lucene.codecs.asserting.TestAssertingPostingsFormat.testDocsOnly
[junit4] -
org.apache.lucene.codecs.asserting.TestAssertingPostingsFormat.testDocsAndFreqsAndPositionsAndOffsetsAndPayloads
[junit4] -
org.apache.lucene.codecs.asserting.TestAssertingPostingsFormat.testDocsAndFreqs
[junit4] -
org.apache.lucene.codecs.asserting.TestAssertingPostingsFormat.testDocsAndFreqsAndPositionsAndOffsets
[junit4] -
org.apache.lucene.codecs.asserting.TestAssertingPostingsFormat.testRandom
[junit4] -
org.apache.lucene.codecs.asserting.TestAssertingPostingsFormat.testDocsAndFreqsAndPositionsAndPayloads
[junit4] -
org.apache.lucene.codecs.asserting.TestAssertingPostingsFormat.testDocsAndFreqsAndPositions
[junit4] - org.apache.lucene.index.TestDirectoryReader.testFilesOpenClose
[junit4] -
org.apache.lucene.index.TestIndexWriterDelete.testIndexingThenDeleting
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-6124) Fix broken close() methods
[
https://issues.apache.org/jira/browse/LUCENE-6124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Muir updated LUCENE-6124:
Attachment: LUCENE-6124_asserts.patch
Here is the simple change i added to asserting codec... these are all bugs. And
we should think about other things we could test too.
Maybe BaseXXXTestCase can do it, to have better coverage across all codecs (and
backwards ones). We should also think about BaseDirectoryTestCase for .store
apis, and similar.
Fix broken close() methods
--
Key: LUCENE-6124
URL: https://issues.apache.org/jira/browse/LUCENE-6124
Project: Lucene - Core
Issue Type: Bug
Reporter: Robert Muir
Attachments: LUCENE-6124_asserts.patch
Closeable.close() says If the stream is already closed then invoking this
method has no effect..
But a lot of our code does not really respect that. If i add an extra
close() call in assertingcodec, it finds all kinds of bugs in codec code, for
example:
{noformat}
[junit4] Tests with failures (first 10 out of 59):
[junit4] -
org.apache.lucene.index.TestCrashCausesCorruptIndex.testCrashCorruptsIndexing
[junit4] -
org.apache.lucene.codecs.asserting.TestAssertingPostingsFormat.testDocsOnly
[junit4] -
org.apache.lucene.codecs.asserting.TestAssertingPostingsFormat.testDocsAndFreqsAndPositionsAndOffsetsAndPayloads
[junit4] -
org.apache.lucene.codecs.asserting.TestAssertingPostingsFormat.testDocsAndFreqs
[junit4] -
org.apache.lucene.codecs.asserting.TestAssertingPostingsFormat.testDocsAndFreqsAndPositionsAndOffsets
[junit4] -
org.apache.lucene.codecs.asserting.TestAssertingPostingsFormat.testRandom
[junit4] -
org.apache.lucene.codecs.asserting.TestAssertingPostingsFormat.testDocsAndFreqsAndPositionsAndPayloads
[junit4] -
org.apache.lucene.codecs.asserting.TestAssertingPostingsFormat.testDocsAndFreqsAndPositions
[junit4] - org.apache.lucene.index.TestDirectoryReader.testFilesOpenClose
[junit4] -
org.apache.lucene.index.TestIndexWriterDelete.testIndexingThenDeleting
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-6124) Fix broken close() methods
[
https://issues.apache.org/jira/browse/LUCENE-6124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Muir updated LUCENE-6124:
Attachment: LUCENE-6124_asserts.patch
I added tests to BaseDirectoryTestCase.
Some .store code has bugs too, e.g.:
{noformat}
org.apache.lucene.store.AlreadyClosedException: this Directory is closed
at org.apache.lucene.store.BaseDirectory.ensureOpen(BaseDirectory.java:50)
at org.apache.lucene.store.RAMDirectory.listAll(RAMDirectory.java:106)
at
org.apache.lucene.store.NRTCachingDirectory.close(NRTCachingDirectory.java:203)
at
org.apache.lucene.store.BaseDirectoryTestCase.testDoubleCloseDirectory(BaseDirectoryTestCase.java:1050)
{noformat}
Fix broken close() methods
--
Key: LUCENE-6124
URL: https://issues.apache.org/jira/browse/LUCENE-6124
Project: Lucene - Core
Issue Type: Bug
Reporter: Robert Muir
Attachments: LUCENE-6124_asserts.patch, LUCENE-6124_asserts.patch
Closeable.close() says If the stream is already closed then invoking this
method has no effect..
But a lot of our code does not really respect that. If i add an extra
close() call in assertingcodec, it finds all kinds of bugs in codec code, for
example:
{noformat}
[junit4] Tests with failures (first 10 out of 59):
[junit4] -
org.apache.lucene.index.TestCrashCausesCorruptIndex.testCrashCorruptsIndexing
[junit4] -
org.apache.lucene.codecs.asserting.TestAssertingPostingsFormat.testDocsOnly
[junit4] -
org.apache.lucene.codecs.asserting.TestAssertingPostingsFormat.testDocsAndFreqsAndPositionsAndOffsetsAndPayloads
[junit4] -
org.apache.lucene.codecs.asserting.TestAssertingPostingsFormat.testDocsAndFreqs
[junit4] -
org.apache.lucene.codecs.asserting.TestAssertingPostingsFormat.testDocsAndFreqsAndPositionsAndOffsets
[junit4] -
org.apache.lucene.codecs.asserting.TestAssertingPostingsFormat.testRandom
[junit4] -
org.apache.lucene.codecs.asserting.TestAssertingPostingsFormat.testDocsAndFreqsAndPositionsAndPayloads
[junit4] -
org.apache.lucene.codecs.asserting.TestAssertingPostingsFormat.testDocsAndFreqsAndPositions
[junit4] - org.apache.lucene.index.TestDirectoryReader.testFilesOpenClose
[junit4] -
org.apache.lucene.index.TestIndexWriterDelete.testIndexingThenDeleting
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-6787) API to manage blobs in Solr
[
https://issues.apache.org/jira/browse/SOLR-6787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14251967#comment-14251967
]
Yonik Seeley commented on SOLR-6787:
{quote}
We should stop building ready to cook stuff and start making ready to eat
stuff if we want to survive in this space.
So. I am starting with the user and ask the question , how would they use it.
{quote}
I've always had that philosophy and I think the majority of other committers do
also.
Trying to reuse and improve existing APIs before creating new ones, or making
new APIs as generic as possible does not run counter to that (and produces a
better system over the long haul).
{quote}
I'm going solo. So , there are not many discussions. I would like others to
start discussions in JIRA and I can participate
{quote}
Another option would have been to do it in a branch first, esp if it's
exploratory.
API to manage blobs in Solr
Key: SOLR-6787
URL: https://issues.apache.org/jira/browse/SOLR-6787
Project: Solr
Issue Type: Sub-task
Reporter: Noble Paul
Assignee: Noble Paul
Fix For: 5.0, Trunk
Attachments: SOLR-6787.patch, SOLR-6787.patch
A special collection called .system needs to be created by the user to
store/manage blobs. The schema/solrconfig of that collection need to be
automatically supplied by the system so that there are no errors
APIs need to be created to manage the content of that collection
{code}
#create your .system collection first
http://localhost:8983/solr/admin/collections?action=CREATEname=.systemreplicationFactor=2
#The config for this collection is automatically created . numShards for this
collection is hardcoded to 1
#create a new jar or add a new version of a jar
curl -X POST -H 'Content-Type: application/octet-stream' --data-binary
@mycomponent.jar http://localhost:8983/solr/.system/blob/mycomponent
# GET on the end point would give a list of jars and other details
curl http://localhost:8983/solr/.system/blob
# GET on the end point with jar name would give details of various versions
of the available jars
curl http://localhost:8983/solr/.system/blob/mycomponent
# GET on the end point with jar name and version with a wt=filestream to get
the actual file
curl http://localhost:8983/solr/.system/blob/mycomponent/1?wt=filestream
mycomponent.1.jar
# GET on the end point with jar name and wt=filestream to get the latest
version of the file
curl http://localhost:8983/solr/.system/blob/mycomponent?wt=filestream
mycomponent.jar
{code}
Please note that the jars are never deleted. a new version is added to the
system everytime a new jar is posted for the name. You must use the standard
delete commands to delete the old entries
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-6124) Fix broken close() methods
[
https://issues.apache.org/jira/browse/LUCENE-6124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Muir updated LUCENE-6124:
Attachment: LUCENE-6124.patch
Here is current patch, with those tests passing. BlockTree, NRTCachingDir,
MockDir, etc had bugs.
We need to add checks in e.g. BasePostingsFormatTestCase and all those kinds of
tests too, because I know there are other term dictionaries with the same
pattern blocktree has.
Fix broken close() methods
--
Key: LUCENE-6124
URL: https://issues.apache.org/jira/browse/LUCENE-6124
Project: Lucene - Core
Issue Type: Bug
Reporter: Robert Muir
Attachments: LUCENE-6124.patch, LUCENE-6124_asserts.patch,
LUCENE-6124_asserts.patch
Closeable.close() says If the stream is already closed then invoking this
method has no effect..
But a lot of our code does not really respect that. If i add an extra
close() call in assertingcodec, it finds all kinds of bugs in codec code, for
example:
{noformat}
[junit4] Tests with failures (first 10 out of 59):
[junit4] -
org.apache.lucene.index.TestCrashCausesCorruptIndex.testCrashCorruptsIndexing
[junit4] -
org.apache.lucene.codecs.asserting.TestAssertingPostingsFormat.testDocsOnly
[junit4] -
org.apache.lucene.codecs.asserting.TestAssertingPostingsFormat.testDocsAndFreqsAndPositionsAndOffsetsAndPayloads
[junit4] -
org.apache.lucene.codecs.asserting.TestAssertingPostingsFormat.testDocsAndFreqs
[junit4] -
org.apache.lucene.codecs.asserting.TestAssertingPostingsFormat.testDocsAndFreqsAndPositionsAndOffsets
[junit4] -
org.apache.lucene.codecs.asserting.TestAssertingPostingsFormat.testRandom
[junit4] -
org.apache.lucene.codecs.asserting.TestAssertingPostingsFormat.testDocsAndFreqsAndPositionsAndPayloads
[junit4] -
org.apache.lucene.codecs.asserting.TestAssertingPostingsFormat.testDocsAndFreqsAndPositions
[junit4] - org.apache.lucene.index.TestDirectoryReader.testFilesOpenClose
[junit4] -
org.apache.lucene.index.TestIndexWriterDelete.testIndexingThenDeleting
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Solr-trunk-Windows (64bit/jdk1.8.0_40-ea-b09) - Build # 4498 - Failure!
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Windows/4498/ Java: 64bit/jdk1.8.0_40-ea-b09 -XX:+UseCompressedOops -XX:+UseG1GC (asserts: true) 3 tests failed. FAILED: junit.framework.TestSuite.org.apache.solr.core.TestSolrConfigHandler Error Message: Could not remove the following files (in the order of attempts): C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.core.TestSolrConfigHandler-51368B885BE2F0FD-001\tempDir-005\collection1\conf\configoverlay.json: java.nio.file.FileSystemException: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.core.TestSolrConfigHandler-51368B885BE2F0FD-001\tempDir-005\collection1\conf\configoverlay.json: The process cannot access the file because it is being used by another process. C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.core.TestSolrConfigHandler-51368B885BE2F0FD-001\tempDir-005\collection1\conf: java.nio.file.DirectoryNotEmptyException: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.core.TestSolrConfigHandler-51368B885BE2F0FD-001\tempDir-005\collection1\conf C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.core.TestSolrConfigHandler-51368B885BE2F0FD-001\tempDir-005\collection1: java.nio.file.DirectoryNotEmptyException: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.core.TestSolrConfigHandler-51368B885BE2F0FD-001\tempDir-005\collection1 C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.core.TestSolrConfigHandler-51368B885BE2F0FD-001\tempDir-005: java.nio.file.DirectoryNotEmptyException: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.core.TestSolrConfigHandler-51368B885BE2F0FD-001\tempDir-005 C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.core.TestSolrConfigHandler-51368B885BE2F0FD-001\tempDir-005: java.nio.file.DirectoryNotEmptyException: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.core.TestSolrConfigHandler-51368B885BE2F0FD-001\tempDir-005 C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.core.TestSolrConfigHandler-51368B885BE2F0FD-001: java.nio.file.DirectoryNotEmptyException: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.core.TestSolrConfigHandler-51368B885BE2F0FD-001 Stack Trace: java.io.IOException: Could not remove the following files (in the order of attempts): C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.core.TestSolrConfigHandler-51368B885BE2F0FD-001\tempDir-005\collection1\conf\configoverlay.json: java.nio.file.FileSystemException: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.core.TestSolrConfigHandler-51368B885BE2F0FD-001\tempDir-005\collection1\conf\configoverlay.json: The process cannot access the file because it is being used by another process. C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.core.TestSolrConfigHandler-51368B885BE2F0FD-001\tempDir-005\collection1\conf: java.nio.file.DirectoryNotEmptyException: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.core.TestSolrConfigHandler-51368B885BE2F0FD-001\tempDir-005\collection1\conf C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.core.TestSolrConfigHandler-51368B885BE2F0FD-001\tempDir-005\collection1: java.nio.file.DirectoryNotEmptyException: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.core.TestSolrConfigHandler-51368B885BE2F0FD-001\tempDir-005\collection1 C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.core.TestSolrConfigHandler-51368B885BE2F0FD-001\tempDir-005: java.nio.file.DirectoryNotEmptyException: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.core.TestSolrConfigHandler-51368B885BE2F0FD-001\tempDir-005 C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.core.TestSolrConfigHandler-51368B885BE2F0FD-001\tempDir-005: java.nio.file.DirectoryNotEmptyException: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.core.TestSolrConfigHandler-51368B885BE2F0FD-001\tempDir-005 C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.core.TestSolrConfigHandler-51368B885BE2F0FD-001:
[jira] [Created] (SOLR-6866) the collection created by bin/solr -e schemaless should be named gettingstarted
Hoss Man created SOLR-6866: -- Summary: the collection created by bin/solr -e schemaless should be named gettingstarted Key: SOLR-6866 URL: https://issues.apache.org/jira/browse/SOLR-6866 Project: Solr Issue Type: Improvement Reporter: Hoss Man Fix For: 5.0 at the moment, if you run bin/solr -e schemaless the collection created for you is named schemaless this might confuse users -- in particularly it makes some of the URLs in the docs on the types of the Schema API commands kind of confusing... https://cwiki.apache.org/confluence/display/solr/Schemaless+Mode http://localhost:8983/solr/schemaless/schema/fields schemaless/schema and all that. It would probably be better to follow the example set by bin/solr -e cloud and give the new collection the name gettingstarted -- that way every example that uses data_driven_schema_configs would create gettingstarted and every gettingstarted collection created by an example would be using data_driven_schema_configs -- and all of the example URLs in docs should be interchangable. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-6787) API to manage blobs in Solr
[
https://issues.apache.org/jira/browse/SOLR-6787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14252097#comment-14252097
]
Noble Paul commented on SOLR-6787:
--
bq.That's my entire point... if it's hard to do X, look first into making X
easier
We should do that too.
But it does not solve the problem
bq.If it's easier to manage blobs this way
It won't be. Why would anyone want to store a blob and all the past versions of
it normally?
I'm trying to dedupe a blob here , is it required normally ?
I'm trying to ensure that there is always a name for a blob , is it required
normally ?
bq.Another option would have been to do it in a branch first, esp if it's
exploratory.
It is not exploratory . This is a feature we need
bq.Trying to reuse and improve existing APIs before creating new ones, or
making new APIs as generic as possible does not run counter to that
As I said earlier I'm fully convinced about the need of this API. At the same
time I'm open to suggestions to improve it.
We do an exploratory branch when we have a doubt on how to implement it. In
this case , the question is about the interface which requires discussion not
another branch
API to manage blobs in Solr
Key: SOLR-6787
URL: https://issues.apache.org/jira/browse/SOLR-6787
Project: Solr
Issue Type: Sub-task
Reporter: Noble Paul
Assignee: Noble Paul
Fix For: 5.0, Trunk
Attachments: SOLR-6787.patch, SOLR-6787.patch
A special collection called .system needs to be created by the user to
store/manage blobs. The schema/solrconfig of that collection need to be
automatically supplied by the system so that there are no errors
APIs need to be created to manage the content of that collection
{code}
#create your .system collection first
http://localhost:8983/solr/admin/collections?action=CREATEname=.systemreplicationFactor=2
#The config for this collection is automatically created . numShards for this
collection is hardcoded to 1
#create a new jar or add a new version of a jar
curl -X POST -H 'Content-Type: application/octet-stream' --data-binary
@mycomponent.jar http://localhost:8983/solr/.system/blob/mycomponent
# GET on the end point would give a list of jars and other details
curl http://localhost:8983/solr/.system/blob
# GET on the end point with jar name would give details of various versions
of the available jars
curl http://localhost:8983/solr/.system/blob/mycomponent
# GET on the end point with jar name and version with a wt=filestream to get
the actual file
curl http://localhost:8983/solr/.system/blob/mycomponent/1?wt=filestream
mycomponent.1.jar
# GET on the end point with jar name and wt=filestream to get the latest
version of the file
curl http://localhost:8983/solr/.system/blob/mycomponent?wt=filestream
mycomponent.jar
{code}
Please note that the jars are never deleted. a new version is added to the
system everytime a new jar is posted for the name. You must use the standard
delete commands to delete the old entries
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (SOLR-6787) API to manage blobs in Solr
[
https://issues.apache.org/jira/browse/SOLR-6787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14252097#comment-14252097
]
Noble Paul edited comment on SOLR-6787 at 12/18/14 7:34 PM:
bq.That's my entire point... if it's hard to do X, look first into making X
easier
We should do that too.
But it does not solve the problem
bq.If it's easier to manage blobs this way
It won't be. Why would anyone want to store a blob and all the past versions of
it normally?
I'm trying to dedupe a blob here , is it required normally ?
I'm trying to ensure that there is always a name for a blob , is it required
normally ?
bq.Another option would have been to do it in a branch first, esp if it's
exploratory.
It is not exploratory . This is a feature we need
bq.Trying to reuse and improve existing APIs before creating new ones, or
making new APIs as generic as possible does not run counter to that
As I said earlier I'm fully convinced about the need of this API. At the same
time I'm open to suggestions to improve it.
We do an exploratory branch when we have a doubt on how to implement it. In
this case , the question is about the interface which requires discussion not
another branch
I guess you are confused by the title of the ticket. It is not an API to jus
manage binary fields . I need certain semantics for storing large objects in
a special collection. There are more use cases , such as large synonyms,
stopwords etc and other storage for the system where ZK is not suitable
was (Author: noble.paul):
bq.That's my entire point... if it's hard to do X, look first into making X
easier
We should do that too.
But it does not solve the problem
bq.If it's easier to manage blobs this way
It won't be. Why would anyone want to store a blob and all the past versions of
it normally?
I'm trying to dedupe a blob here , is it required normally ?
I'm trying to ensure that there is always a name for a blob , is it required
normally ?
bq.Another option would have been to do it in a branch first, esp if it's
exploratory.
It is not exploratory . This is a feature we need
bq.Trying to reuse and improve existing APIs before creating new ones, or
making new APIs as generic as possible does not run counter to that
As I said earlier I'm fully convinced about the need of this API. At the same
time I'm open to suggestions to improve it.
We do an exploratory branch when we have a doubt on how to implement it. In
this case , the question is about the interface which requires discussion not
another branch
API to manage blobs in Solr
Key: SOLR-6787
URL: https://issues.apache.org/jira/browse/SOLR-6787
Project: Solr
Issue Type: Sub-task
Reporter: Noble Paul
Assignee: Noble Paul
Fix For: 5.0, Trunk
Attachments: SOLR-6787.patch, SOLR-6787.patch
A special collection called .system needs to be created by the user to
store/manage blobs. The schema/solrconfig of that collection need to be
automatically supplied by the system so that there are no errors
APIs need to be created to manage the content of that collection
{code}
#create your .system collection first
http://localhost:8983/solr/admin/collections?action=CREATEname=.systemreplicationFactor=2
#The config for this collection is automatically created . numShards for this
collection is hardcoded to 1
#create a new jar or add a new version of a jar
curl -X POST -H 'Content-Type: application/octet-stream' --data-binary
@mycomponent.jar http://localhost:8983/solr/.system/blob/mycomponent
# GET on the end point would give a list of jars and other details
curl http://localhost:8983/solr/.system/blob
# GET on the end point with jar name would give details of various versions
of the available jars
curl http://localhost:8983/solr/.system/blob/mycomponent
# GET on the end point with jar name and version with a wt=filestream to get
the actual file
curl http://localhost:8983/solr/.system/blob/mycomponent/1?wt=filestream
mycomponent.1.jar
# GET on the end point with jar name and wt=filestream to get the latest
version of the file
curl http://localhost:8983/solr/.system/blob/mycomponent?wt=filestream
mycomponent.jar
{code}
Please note that the jars are never deleted. a new version is added to the
system everytime a new jar is posted for the name. You must use the standard
delete commands to delete the old entries
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-6787) API to manage blobs in Solr
[
https://issues.apache.org/jira/browse/SOLR-6787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14252121#comment-14252121
]
Noble Paul commented on SOLR-6787:
--
hi [~arafalov] I postyed a reply to you , but it got lost
bq.Why do you need a user to create the collection first? Especially since the
config/etc is hardcoded and it needs to be present on every node?
Most users will not use this feature . When it becomes common place we should
create it y default.
bq.Also, if you have heterogeneous environment, the current implementation
would require a union of all blobs, right? As in, every node will need to have
all possible blobs/jars, not just the jars the collections on that node have.
That is not true. The jars are fetched over the http API and used on demand .
It is never fetched from a local collection
bq.Finally, does the .system collection show up on the Admin UI? Are there
implications/restrictions of that?
It is a normal collection where we have a predefined schema/config and it shows
up in the admin UI
API to manage blobs in Solr
Key: SOLR-6787
URL: https://issues.apache.org/jira/browse/SOLR-6787
Project: Solr
Issue Type: Sub-task
Reporter: Noble Paul
Assignee: Noble Paul
Fix For: 5.0, Trunk
Attachments: SOLR-6787.patch, SOLR-6787.patch
A special collection called .system needs to be created by the user to
store/manage blobs. The schema/solrconfig of that collection need to be
automatically supplied by the system so that there are no errors
APIs need to be created to manage the content of that collection
{code}
#create your .system collection first
http://localhost:8983/solr/admin/collections?action=CREATEname=.systemreplicationFactor=2
#The config for this collection is automatically created . numShards for this
collection is hardcoded to 1
#create a new jar or add a new version of a jar
curl -X POST -H 'Content-Type: application/octet-stream' --data-binary
@mycomponent.jar http://localhost:8983/solr/.system/blob/mycomponent
# GET on the end point would give a list of jars and other details
curl http://localhost:8983/solr/.system/blob
# GET on the end point with jar name would give details of various versions
of the available jars
curl http://localhost:8983/solr/.system/blob/mycomponent
# GET on the end point with jar name and version with a wt=filestream to get
the actual file
curl http://localhost:8983/solr/.system/blob/mycomponent/1?wt=filestream
mycomponent.1.jar
# GET on the end point with jar name and wt=filestream to get the latest
version of the file
curl http://localhost:8983/solr/.system/blob/mycomponent?wt=filestream
mycomponent.jar
{code}
Please note that the jars are never deleted. a new version is added to the
system everytime a new jar is posted for the name. You must use the standard
delete commands to delete the old entries
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-6124) Fix broken close() methods
[
https://issues.apache.org/jira/browse/LUCENE-6124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Muir updated LUCENE-6124:
Attachment: LUCENE-6124.patch
added asserts to BasePostingsFormatTestCase. Now lots of codecs are angry :)
Fix broken close() methods
--
Key: LUCENE-6124
URL: https://issues.apache.org/jira/browse/LUCENE-6124
Project: Lucene - Core
Issue Type: Bug
Reporter: Robert Muir
Attachments: LUCENE-6124.patch, LUCENE-6124.patch,
LUCENE-6124_asserts.patch, LUCENE-6124_asserts.patch
Closeable.close() says If the stream is already closed then invoking this
method has no effect..
But a lot of our code does not really respect that. If i add an extra
close() call in assertingcodec, it finds all kinds of bugs in codec code, for
example:
{noformat}
[junit4] Tests with failures (first 10 out of 59):
[junit4] -
org.apache.lucene.index.TestCrashCausesCorruptIndex.testCrashCorruptsIndexing
[junit4] -
org.apache.lucene.codecs.asserting.TestAssertingPostingsFormat.testDocsOnly
[junit4] -
org.apache.lucene.codecs.asserting.TestAssertingPostingsFormat.testDocsAndFreqsAndPositionsAndOffsetsAndPayloads
[junit4] -
org.apache.lucene.codecs.asserting.TestAssertingPostingsFormat.testDocsAndFreqs
[junit4] -
org.apache.lucene.codecs.asserting.TestAssertingPostingsFormat.testDocsAndFreqsAndPositionsAndOffsets
[junit4] -
org.apache.lucene.codecs.asserting.TestAssertingPostingsFormat.testRandom
[junit4] -
org.apache.lucene.codecs.asserting.TestAssertingPostingsFormat.testDocsAndFreqsAndPositionsAndPayloads
[junit4] -
org.apache.lucene.codecs.asserting.TestAssertingPostingsFormat.testDocsAndFreqsAndPositions
[junit4] - org.apache.lucene.index.TestDirectoryReader.testFilesOpenClose
[junit4] -
org.apache.lucene.index.TestIndexWriterDelete.testIndexingThenDeleting
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (SOLR-6867) bin/solr -e foo should use collection/cores api to check if collection/core exists before trying to create it
Hoss Man created SOLR-6867:
--
Summary: bin/solr -e foo should use collection/cores api to check
if collection/core exists before trying to create it
Key: SOLR-6867
URL: https://issues.apache.org/jira/browse/SOLR-6867
Project: Solr
Issue Type: Improvement
Reporter: Hoss Man
currently if you run an example command and the core/collection being created
already exists you get an error message in the console, ut then the example
keeps working (see below).
I think it would be better if the script used the cores api (or collection api
in the 'cloud' example) to check if the collection already existed, and if so
just output a note about it, something like...
{noformat}
Core API indicates a core named 'techproducts' already exists...
http://localhost:8983/solr/admin/cores?action=statuscore=techproducts
To remove this core, use the following command...
bin/solr ...etc...
And then you can recreate it with...
bin/solr ...etc...
{noformat}
Current behavior...
{noformat}
...
Creating new core 'techproducts' using command:
http://localhost:8983/solr/admin/cores?action=CREATEname=techproductsinstanceDir=techproducts
Exception in thread main org.apache.solr.client.solrj.SolrServerException:
Request to
http://localhost:8983/solr/admin/cores?action=CREATEname=techproductsinstanceDir=techproducts
failed due to: Core with name 'techproducts' already exists.
at org.apache.solr.util.SolrCLI.getJson(SolrCLI.java:473)
at org.apache.solr.util.SolrCLI.getJson(SolrCLI.java:416)
at org.apache.solr.util.SolrCLI.getJson(SolrCLI.java:402)
at
org.apache.solr.util.SolrCLI$CreateCoreTool.runTool(SolrCLI.java:1274)
at org.apache.solr.util.SolrCLI.main(SolrCLI.java:185)
Indexing tech product example docs from
/home/hossman/lucene/5x_dev/solr/example/exampledocs
SimplePostTool version 1.5
Posting files to base url http://localhost:8983/solr/techproducts/update using
content-type application/xml..
...
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5951) Detect when index is on SSD and set dynamic defaults
[
https://issues.apache.org/jira/browse/LUCENE-5951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14252157#comment-14252157
]
Michael McCandless commented on LUCENE-5951:
[~thetaphi] how about this disclaimer in CMS's top javadocs?
{noformat}
* pThis class attempts to detect whether the index is
* on rotational storage (traditional hard drive) or not
* (e.g. solid-state disk) and changes the default max merge
* and thread count accordingly. This detection is currently
* Linux-only, and relies on the OS to put the right value
* into /sys/block/lt;devgt;/block/rotational./p
{noformat}
Detect when index is on SSD and set dynamic defaults
Key: LUCENE-5951
URL: https://issues.apache.org/jira/browse/LUCENE-5951
Project: Lucene - Core
Issue Type: Improvement
Reporter: Michael McCandless
Assignee: Michael McCandless
Attachments: LUCENE-5951.patch, LUCENE-5951.patch, LUCENE-5951.patch,
LUCENE-5951.patch, LUCENE-5951.patch, LUCENE-5951.patch
E.g. ConcurrentMergeScheduler should default maxMergeThreads to 3 if it's on
SSD and 1 if it's on spinning disks.
I think the new NIO2 APIs can let us figure out which device we are mounted
on, and from there maybe we can do os-specific stuff e.g. look at
/sys/block/dev/queue/rotational to see if it's spinning storage or not ...
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-6866) the collection created by bin/solr -e schemaless should be named gettingstarted
[ https://issues.apache.org/jira/browse/SOLR-6866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14252246#comment-14252246 ] Steve Rowe commented on SOLR-6866: -- +1 the collection created by bin/solr -e schemaless should be named gettingstarted --- Key: SOLR-6866 URL: https://issues.apache.org/jira/browse/SOLR-6866 Project: Solr Issue Type: Improvement Reporter: Hoss Man Fix For: 5.0 at the moment, if you run bin/solr -e schemaless the collection created for you is named schemaless this might confuse users -- in particularly it makes some of the URLs in the docs on the types of the Schema API commands kind of confusing... https://cwiki.apache.org/confluence/display/solr/Schemaless+Mode http://localhost:8983/solr/schemaless/schema/fields schemaless/schema and all that. It would probably be better to follow the example set by bin/solr -e cloud and give the new collection the name gettingstarted -- that way every example that uses data_driven_schema_configs would create gettingstarted and every gettingstarted collection created by an example would be using data_driven_schema_configs -- and all of the example URLs in docs should be interchangable. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Resolved] (SOLR-5922) Add Properties and other parameters to SolrJ Collection Admin Request calls
[ https://issues.apache.org/jira/browse/SOLR-5922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anshum Gupta resolved SOLR-5922. Resolution: Fixed Add Properties and other parameters to SolrJ Collection Admin Request calls --- Key: SOLR-5922 URL: https://issues.apache.org/jira/browse/SOLR-5922 Project: Solr Issue Type: Improvement Components: SolrJ Reporter: Varun Thacker Assignee: Anshum Gupta Fix For: 5.0, Trunk Attachments: SOLR-5922.patch, SOLR-5922.patch, SOLR-5922.patch, SOLR-5922.patch, SOLR-5922.patch, SOLR-5922.patch, SOLR-5922.patch SOLR-5208 added functionality for the setting of core.properties key/values at create-time on Collections API. We should allow the same behaviour for SolrJ API calls as well. Also I want to add get and set methods to be able to add 'instanceDir', 'dataDir', 'ulogDir' for a create colleciton call. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 707 - Still Failing
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/707/ 2 tests failed. REGRESSION: org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest.testDistribSearch Error Message: document count mismatch. control=1502 sum(shards)=1503 cloudClient=1503 Stack Trace: java.lang.AssertionError: document count mismatch. control=1502 sum(shards)=1503 cloudClient=1503 at __randomizedtesting.SeedInfo.seed([AFE8A0D51025B96:8B180415265D3BAA]:0) at org.junit.Assert.fail(Assert.java:93) at org.apache.solr.cloud.AbstractFullDistribZkTestBase.checkShardConsistency(AbstractFullDistribZkTestBase.java:1308) at org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest.doTest(ChaosMonkeyNothingIsSafeTest.java:232) at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:869) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618) at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827) at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863) at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53) at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55) at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365) at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798) at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458) at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836) at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738) at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772) at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65) at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at
[jira] [Commented] (SOLR-6719) Collection API: CREATE ignores 'property.name' when creating individual cores
[ https://issues.apache.org/jira/browse/SOLR-6719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14252317#comment-14252317 ] Martin Grotzke commented on SOLR-6719: -- We experienced the same with ADDREPLICA and property.solr.common.data.dir (running Solr 4.10.2). Collection API: CREATE ignores 'property.name' when creating individual cores - Key: SOLR-6719 URL: https://issues.apache.org/jira/browse/SOLR-6719 Project: Solr Issue Type: Bug Reporter: Hoss Man Yashveer Rana pointed this out in the ref guide comments... https://cwiki.apache.org/confluence/display/solr/Collections+API?focusedCommentId=47382851#comment-47382851 * Collection CREATE is documented to support property._name_=_value_ (where 'name' and 'property' are italics placeholders for user supplied key=val) as Set core property _name_ to _value_. See core.properties file contents. * The [docs for core.properties|https://cwiki.apache.org/confluence/display/solr/Format+of+solr.xml#Formatofsolr.xml-core.properties_files] include a list of supported property values, including name (literal) as The name of the SolrCore. You'll use this name to reference the SolrCore when running commands with the CoreAdminHandler. From these docs, it's reasonable to assume that using a URL like this... http://localhost:8983/solr/admin/collections?action=CREATEname=my_collectionnumShards=2configSet=data_driven_schema_configsproperty.name=my_corename ...should cause my_collection to be created, with the core name used for every replica being my_corename ... but that doesn't happen. instead the replicas get core names like my_collection_shard1_replica1 This is either a bug, or (my suspicion) it's intentional that the user specific core name is not being used -- if it's intentional, then the Collection CREATE command should fail with a clear error if a user does try to use property.name rather then silently ignoring it and the Collection CREATE docs should be updated to make it clear that name is an exception to the general property.foo - foo in core.properties support. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Solr-5.x-Windows (64bit/jdk1.8.0_20) - Build # 4393 - Failure!
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-5.x-Windows/4393/ Java: 64bit/jdk1.8.0_20 -XX:-UseCompressedOops -XX:+UseSerialGC (asserts: false) 1 tests failed. FAILED: junit.framework.TestSuite.org.apache.solr.client.solrj.embedded.LargeVolumeEmbeddedTest Error Message: Could not remove the following files (in the order of attempts): C:\Users\JenkinsSlave\workspace\Lucene-Solr-5.x-Windows\solr\build\solr-solrj\test\J0\temp\solr.client.solrj.embedded.LargeVolumeEmbeddedTest-C6AF373F8E6FDC52-001\init-core-data-001\tlog\tlog.002: java.nio.file.FileSystemException: C:\Users\JenkinsSlave\workspace\Lucene-Solr-5.x-Windows\solr\build\solr-solrj\test\J0\temp\solr.client.solrj.embedded.LargeVolumeEmbeddedTest-C6AF373F8E6FDC52-001\init-core-data-001\tlog\tlog.002: The process cannot access the file because it is being used by another process. C:\Users\JenkinsSlave\workspace\Lucene-Solr-5.x-Windows\solr\build\solr-solrj\test\J0\temp\solr.client.solrj.embedded.LargeVolumeEmbeddedTest-C6AF373F8E6FDC52-001\init-core-data-001\tlog: java.nio.file.DirectoryNotEmptyException: C:\Users\JenkinsSlave\workspace\Lucene-Solr-5.x-Windows\solr\build\solr-solrj\test\J0\temp\solr.client.solrj.embedded.LargeVolumeEmbeddedTest-C6AF373F8E6FDC52-001\init-core-data-001\tlog C:\Users\JenkinsSlave\workspace\Lucene-Solr-5.x-Windows\solr\build\solr-solrj\test\J0\temp\solr.client.solrj.embedded.LargeVolumeEmbeddedTest-C6AF373F8E6FDC52-001\init-core-data-001: java.nio.file.DirectoryNotEmptyException: C:\Users\JenkinsSlave\workspace\Lucene-Solr-5.x-Windows\solr\build\solr-solrj\test\J0\temp\solr.client.solrj.embedded.LargeVolumeEmbeddedTest-C6AF373F8E6FDC52-001\init-core-data-001 C:\Users\JenkinsSlave\workspace\Lucene-Solr-5.x-Windows\solr\build\solr-solrj\test\J0\temp\solr.client.solrj.embedded.LargeVolumeEmbeddedTest-C6AF373F8E6FDC52-001\init-core-data-001: java.nio.file.DirectoryNotEmptyException: C:\Users\JenkinsSlave\workspace\Lucene-Solr-5.x-Windows\solr\build\solr-solrj\test\J0\temp\solr.client.solrj.embedded.LargeVolumeEmbeddedTest-C6AF373F8E6FDC52-001\init-core-data-001 C:\Users\JenkinsSlave\workspace\Lucene-Solr-5.x-Windows\solr\build\solr-solrj\test\J0\temp\solr.client.solrj.embedded.LargeVolumeEmbeddedTest-C6AF373F8E6FDC52-001: java.nio.file.DirectoryNotEmptyException: C:\Users\JenkinsSlave\workspace\Lucene-Solr-5.x-Windows\solr\build\solr-solrj\test\J0\temp\solr.client.solrj.embedded.LargeVolumeEmbeddedTest-C6AF373F8E6FDC52-001 Stack Trace: java.io.IOException: Could not remove the following files (in the order of attempts): C:\Users\JenkinsSlave\workspace\Lucene-Solr-5.x-Windows\solr\build\solr-solrj\test\J0\temp\solr.client.solrj.embedded.LargeVolumeEmbeddedTest-C6AF373F8E6FDC52-001\init-core-data-001\tlog\tlog.002: java.nio.file.FileSystemException: C:\Users\JenkinsSlave\workspace\Lucene-Solr-5.x-Windows\solr\build\solr-solrj\test\J0\temp\solr.client.solrj.embedded.LargeVolumeEmbeddedTest-C6AF373F8E6FDC52-001\init-core-data-001\tlog\tlog.002: The process cannot access the file because it is being used by another process. C:\Users\JenkinsSlave\workspace\Lucene-Solr-5.x-Windows\solr\build\solr-solrj\test\J0\temp\solr.client.solrj.embedded.LargeVolumeEmbeddedTest-C6AF373F8E6FDC52-001\init-core-data-001\tlog: java.nio.file.DirectoryNotEmptyException: C:\Users\JenkinsSlave\workspace\Lucene-Solr-5.x-Windows\solr\build\solr-solrj\test\J0\temp\solr.client.solrj.embedded.LargeVolumeEmbeddedTest-C6AF373F8E6FDC52-001\init-core-data-001\tlog C:\Users\JenkinsSlave\workspace\Lucene-Solr-5.x-Windows\solr\build\solr-solrj\test\J0\temp\solr.client.solrj.embedded.LargeVolumeEmbeddedTest-C6AF373F8E6FDC52-001\init-core-data-001: java.nio.file.DirectoryNotEmptyException: C:\Users\JenkinsSlave\workspace\Lucene-Solr-5.x-Windows\solr\build\solr-solrj\test\J0\temp\solr.client.solrj.embedded.LargeVolumeEmbeddedTest-C6AF373F8E6FDC52-001\init-core-data-001 C:\Users\JenkinsSlave\workspace\Lucene-Solr-5.x-Windows\solr\build\solr-solrj\test\J0\temp\solr.client.solrj.embedded.LargeVolumeEmbeddedTest-C6AF373F8E6FDC52-001\init-core-data-001: java.nio.file.DirectoryNotEmptyException: C:\Users\JenkinsSlave\workspace\Lucene-Solr-5.x-Windows\solr\build\solr-solrj\test\J0\temp\solr.client.solrj.embedded.LargeVolumeEmbeddedTest-C6AF373F8E6FDC52-001\init-core-data-001 C:\Users\JenkinsSlave\workspace\Lucene-Solr-5.x-Windows\solr\build\solr-solrj\test\J0\temp\solr.client.solrj.embedded.LargeVolumeEmbeddedTest-C6AF373F8E6FDC52-001: java.nio.file.DirectoryNotEmptyException: C:\Users\JenkinsSlave\workspace\Lucene-Solr-5.x-Windows\solr\build\solr-solrj\test\J0\temp\solr.client.solrj.embedded.LargeVolumeEmbeddedTest-C6AF373F8E6FDC52-001 at __randomizedtesting.SeedInfo.seed([C6AF373F8E6FDC52]:0) at org.apache.lucene.util.IOUtils.rm(IOUtils.java:289) at
[jira] [Updated] (LUCENE-6121) Fix CachingTokenFilter to propagate reset() the first time
[
https://issues.apache.org/jira/browse/LUCENE-6121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Smiley updated LUCENE-6121:
-
Attachment:
LUCENE-6121_CachingTokenFilter_reset_propagates_reset_if_not_cached.patch
The attached patch propagates reset() on reset() only when the stream hasn't
been cached yet. This is good/intuitive behavior, simple, and actually isn't
as much of a change to existing users; many might not even notice. If you
still call reset on the input (but not in CachingTokenFilter) before then
calling incrementToken, it'll all work out fine. You shouldn't to follow the
spirit of our API, but you can. In fact there's one query builder ('flexible'
AnalyzerQueryNodeProcessor) that I didn't change to move what it calls reset()
on and it still works.
In the patch you may notice I moved the reset() to be before incrementToken()
-- I find that flow clearest.
I did have to make a change to the default highlighter.
WeightedSpanTermExtractor handed off it's stream to MemoryIndex, and when done,
it called reset() as the last thing it did. That's bad behavior, IMO but it
turned out to (previously) be necessary because Highlighter called reset()
_before_ passing the tokenStream to QueryScorer/WSTE. I fixed this so that
WSTE doesn't call reset (it doesn't call incrementToken itself, after all), and
moved Highlighter's invocation of reset() to the last possible moment, right
before the loop of incrementToken(). I think this is best practice in general
-- always call reset() as close to incrementToken() as you can.
In CHANGES.txt I'll say this:
bq. CachingTokenFilter now propagates reset() to its input if incrementToken()
hasn't been called yet. You should generally call reset() now on this token
filter instead of doing it a-priori on its input (which previously didn't work).
Fix CachingTokenFilter to propagate reset() the first time
--
Key: LUCENE-6121
URL: https://issues.apache.org/jira/browse/LUCENE-6121
Project: Lucene - Core
Issue Type: Improvement
Reporter: David Smiley
Assignee: David Smiley
Fix For: 5.0, Trunk
Attachments:
LUCENE-6121_CachingTokenFilter_reset_propagates_reset_if_not_cached.patch
CachingTokenFilter should have been propagating reset() _but only the first
time_ and thus you would then use CachingTokenFilter in a more normal way –
wrap it and call reset() then increment in a loop, etc., instead of knowing
you need to reset() on what it wraps but not this token filter itself. That's
weird. It's ab-normal for a TokenFilter to never propagate reset, so every
user of CachingTokenFilter to date has worked around this by calling reset()
on the underlying input instead of the final wrapping token filter
(CachingTokenFilter in this case).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-6540) strdist() causes NPE if doc is missing field
[
https://issues.apache.org/jira/browse/SOLR-6540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hoss Man updated SOLR-6540:
---
Description:
If you try to use the strdist function on a field which is missing in some
docs, you'll get a NullPointerException
A workarround in some contexts can be to wrap the strdist function in an if
that checks exists(fieldname) and returns some suitable default if it's not
found.
{code}
THIS: if(exists(foo),strdist(literal,field_name_s,edit),0)
INSTEAD OF: strdist(literal,field_name_s,edit)
{code}
was:If you try to use the strdist function on a field which is missing in
some docs, you'll get a NullPointerException
strdist() causes NPE if doc is missing field
Key: SOLR-6540
URL: https://issues.apache.org/jira/browse/SOLR-6540
Project: Solr
Issue Type: Bug
Reporter: Hoss Man
Fix For: 5.0, Trunk
Attachments: SOLR-6540.patch
If you try to use the strdist function on a field which is missing in some
docs, you'll get a NullPointerException
A workarround in some contexts can be to wrap the strdist function in an if
that checks exists(fieldname) and returns some suitable default if it's not
found.
{code}
THIS: if(exists(foo),strdist(literal,field_name_s,edit),0)
INSTEAD OF: strdist(literal,field_name_s,edit)
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-6523) Provide SolrJ support for specifying stateFormat while creating Collections
[ https://issues.apache.org/jira/browse/SOLR-6523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anshum Gupta updated SOLR-6523: --- Attachment: SOLR-6523.patch Patch + test. Provide SolrJ support for specifying stateFormat while creating Collections --- Key: SOLR-6523 URL: https://issues.apache.org/jira/browse/SOLR-6523 Project: Solr Issue Type: Improvement Components: SolrJ Reporter: Anshum Gupta Assignee: Anshum Gupta Attachments: SOLR-6523.patch, SOLR-6523.patch The new zk state format (as introduced by SOLR-5473) should be supported by CollectionsAdminRequest/SolrJ Create() class. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Solr-5.x-Linux (64bit/jdk1.8.0_20) - Build # 11601 - Failure!
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-5.x-Linux/11601/ Java: 64bit/jdk1.8.0_20 -XX:+UseCompressedOops -XX:+UseConcMarkSweepGC (asserts: true) All tests passed Build Log: [...truncated 59715 lines...] BUILD FAILED /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/build.xml:529: The following error occurred while executing this line: /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/build.xml:436: The following error occurred while executing this line: /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/extra-targets.xml:105: The following error occurred while executing this line: /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/extra-targets.xml:206: The following files have the svn:keywords property set: * ./solr/core/src/test/org/apache/solr/cloud/overseer/ZkStateWriterTest.java Total time: 102 minutes 17 seconds Build step 'Invoke Ant' marked build as failure [description-setter] Description set: Java: 64bit/jdk1.8.0_20 -XX:+UseCompressedOops -XX:+UseConcMarkSweepGC (asserts: true) Archiving artifacts Recording test results Email was triggered for: Failure - Any Sending email for trigger: Failure - Any - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-6523) Provide SolrJ support for specifying stateFormat while creating Collections
[ https://issues.apache.org/jira/browse/SOLR-6523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anshum Gupta updated SOLR-6523: --- Attachment: SOLR-6523.patch Patch that messes around less with the imports (didn't know idea by default optimized the imports *on the fly*). Provide SolrJ support for specifying stateFormat while creating Collections --- Key: SOLR-6523 URL: https://issues.apache.org/jira/browse/SOLR-6523 Project: Solr Issue Type: Improvement Components: SolrJ Reporter: Anshum Gupta Assignee: Anshum Gupta Attachments: SOLR-6523.patch, SOLR-6523.patch, SOLR-6523.patch The new zk state format (as introduced by SOLR-5473) should be supported by CollectionsAdminRequest/SolrJ Create() class. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-6523) Provide SolrJ support for specifying stateFormat while creating Collections
[ https://issues.apache.org/jira/browse/SOLR-6523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14252422#comment-14252422 ] ASF subversion and git services commented on SOLR-6523: --- Commit 1646558 from [~anshumg] in branch 'dev/trunk' [ https://svn.apache.org/r1646558 ] SOLR-6523: Provide SolrJ support for specifying stateFormat while creating Collections Provide SolrJ support for specifying stateFormat while creating Collections --- Key: SOLR-6523 URL: https://issues.apache.org/jira/browse/SOLR-6523 Project: Solr Issue Type: Improvement Components: SolrJ Reporter: Anshum Gupta Assignee: Anshum Gupta Attachments: SOLR-6523.patch, SOLR-6523.patch, SOLR-6523.patch The new zk state format (as introduced by SOLR-5473) should be supported by CollectionsAdminRequest/SolrJ Create() class. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-6859) Disable REBALANCELEADERS for 5.0
[ https://issues.apache.org/jira/browse/SOLR-6859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14252430#comment-14252430 ] Hoss Man commented on SOLR-6859: Erick: please remove from the ref guide as well. you can always cut/paste to this page to keep it arround for the future when it's ready again... https://cwiki.apache.org/confluence/display/solr/Internal+-+Trunk+Changes+to+Document Disable REBALANCELEADERS for 5.0 Key: SOLR-6859 URL: https://issues.apache.org/jira/browse/SOLR-6859 Project: Solr Issue Type: Bug Reporter: Erick Erickson Assignee: Erick Erickson Priority: Blocker This is flat dangerous with it's current implementation and should not get into the wild. The (I hope) proper fix is in SOLR-6691. I want to let that code bake for a while post 5.0 before committing though. So this will just comment the handling of REBALANCELEADERS from the collections API for the time being. Marked as blocker, but I should be able to take care of this ASAP so it shouldn't stand in the way of 5.0. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-6769) Election bug
[ https://issues.apache.org/jira/browse/SOLR-6769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14252440#comment-14252440 ] Alexander S. commented on SOLR-6769: This might be related: http://lucene.472066.n3.nabble.com/Endless-100-CPU-usage-on-searcherExecutor-thread-td4175088.html Election bug Key: SOLR-6769 URL: https://issues.apache.org/jira/browse/SOLR-6769 Project: Solr Issue Type: Bug Reporter: Alexander S. Attachments: Screenshot 876.png Hello, I have a very simple set up: 2 shards and 2 replicas (4 nodes in total). What I did is just stopped the shards, but if first shard stopped immediately the second one took about 5 minutes to stop. You can see on the screenshot what happened next. In short: 1. Shard 1 stopped normally 3. Replica 1 became a leader 2. Shard 2 still was performing some job but wasn't accepting connection 4. Replica 2 did not became a leader because Shard 2 is still there but doesn't work 5. Entire cluster went down until Shard 2 stopped and Replica 2 became a leader Marked as critical because this shuts down the entire cluster. Please adjust if I am wrong. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-6124) Fix broken close() methods
[
https://issues.apache.org/jira/browse/LUCENE-6124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Muir updated LUCENE-6124:
Attachment: LUCENE-6124.patch
I generalized the method to test all Closeables in
BaseIndexFileFormatTestCase then fixed all the problems.
This is ready, i dont like some of the failures to be honest, because if you
call close() again it will write extra data and create a corrupt file today.
Fix broken close() methods
--
Key: LUCENE-6124
URL: https://issues.apache.org/jira/browse/LUCENE-6124
Project: Lucene - Core
Issue Type: Bug
Reporter: Robert Muir
Attachments: LUCENE-6124.patch, LUCENE-6124.patch, LUCENE-6124.patch,
LUCENE-6124_asserts.patch, LUCENE-6124_asserts.patch
Closeable.close() says If the stream is already closed then invoking this
method has no effect..
But a lot of our code does not really respect that. If i add an extra
close() call in assertingcodec, it finds all kinds of bugs in codec code, for
example:
{noformat}
[junit4] Tests with failures (first 10 out of 59):
[junit4] -
org.apache.lucene.index.TestCrashCausesCorruptIndex.testCrashCorruptsIndexing
[junit4] -
org.apache.lucene.codecs.asserting.TestAssertingPostingsFormat.testDocsOnly
[junit4] -
org.apache.lucene.codecs.asserting.TestAssertingPostingsFormat.testDocsAndFreqsAndPositionsAndOffsetsAndPayloads
[junit4] -
org.apache.lucene.codecs.asserting.TestAssertingPostingsFormat.testDocsAndFreqs
[junit4] -
org.apache.lucene.codecs.asserting.TestAssertingPostingsFormat.testDocsAndFreqsAndPositionsAndOffsets
[junit4] -
org.apache.lucene.codecs.asserting.TestAssertingPostingsFormat.testRandom
[junit4] -
org.apache.lucene.codecs.asserting.TestAssertingPostingsFormat.testDocsAndFreqsAndPositionsAndPayloads
[junit4] -
org.apache.lucene.codecs.asserting.TestAssertingPostingsFormat.testDocsAndFreqsAndPositions
[junit4] - org.apache.lucene.index.TestDirectoryReader.testFilesOpenClose
[junit4] -
org.apache.lucene.index.TestIndexWriterDelete.testIndexingThenDeleting
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-6121) Fix CachingTokenFilter to propagate reset() the first time
[
https://issues.apache.org/jira/browse/LUCENE-6121?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14252471#comment-14252471
]
Robert Muir commented on LUCENE-6121:
-
{quote}
If you still call reset on the input (but not in CachingTokenFilter) before
then calling incrementToken, it'll all work out fine.
{quote}
How is that? MockTokenizer should be throwing an exception here. its not ok to
reset twice. Please change the text to you must.
{quote}
In fact there's one query builder ('flexible' AnalyzerQueryNodeProcessor) that
I didn't change to move what it calls reset() on and it still works.
{quote}
Then it should be fixed, ideally to use MockAnalyzer so the double-reset causes
a test failure first.
We dont need an API that has flexibility, consumers must follow the normal
contract and it will work:
reset() .. incrementToken() end(), close()
I am also ok if we only forward close() a single time for now. I do not trust
that all TokenStreams obey Closeable yet and will ignore the second invocation.
But technically, CachingTokenFilter shouldn't need to do anything special there.
Fix CachingTokenFilter to propagate reset() the first time
--
Key: LUCENE-6121
URL: https://issues.apache.org/jira/browse/LUCENE-6121
Project: Lucene - Core
Issue Type: Improvement
Reporter: David Smiley
Assignee: David Smiley
Fix For: 5.0, Trunk
Attachments:
LUCENE-6121_CachingTokenFilter_reset_propagates_reset_if_not_cached.patch
CachingTokenFilter should have been propagating reset() _but only the first
time_ and thus you would then use CachingTokenFilter in a more normal way –
wrap it and call reset() then increment in a loop, etc., instead of knowing
you need to reset() on what it wraps but not this token filter itself. That's
weird. It's ab-normal for a TokenFilter to never propagate reset, so every
user of CachingTokenFilter to date has worked around this by calling reset()
on the underlying input instead of the final wrapping token filter
(CachingTokenFilter in this case).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
Re: [JENKINS] Lucene-Solr-5.x-Linux (64bit/jdk1.8.0_20) - Build # 11601 - Failure!
My bad. I committed a fix. On Thu, Dec 18, 2014 at 10:11 PM, Policeman Jenkins Server jenk...@thetaphi.de wrote: Build: http://jenkins.thetaphi.de/job/Lucene-Solr-5.x-Linux/11601/ Java: 64bit/jdk1.8.0_20 -XX:+UseCompressedOops -XX:+UseConcMarkSweepGC (asserts: true) All tests passed Build Log: [...truncated 59715 lines...] BUILD FAILED /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/build.xml:529: The following error occurred while executing this line: /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/build.xml:436: The following error occurred while executing this line: /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/extra-targets.xml:105: The following error occurred while executing this line: /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/extra-targets.xml:206: The following files have the svn:keywords property set: * ./solr/core/src/test/org/apache/solr/cloud/overseer/ZkStateWriterTest.java Total time: 102 minutes 17 seconds Build step 'Invoke Ant' marked build as failure [description-setter] Description set: Java: 64bit/jdk1.8.0_20 -XX:+UseCompressedOops -XX:+UseConcMarkSweepGC (asserts: true) Archiving artifacts Recording test results Email was triggered for: Failure - Any Sending email for trigger: Failure - Any -- Regards, Shalin Shekhar Mangar.
[jira] [Commented] (SOLR-6523) Provide SolrJ support for specifying stateFormat while creating Collections
[ https://issues.apache.org/jira/browse/SOLR-6523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14252500#comment-14252500 ] ASF subversion and git services commented on SOLR-6523: --- Commit 1646567 from [~anshumg] in branch 'dev/trunk' [ https://svn.apache.org/r1646567 ] SOLR-6523: Fixing the test. Removing the unwanted check and calls Provide SolrJ support for specifying stateFormat while creating Collections --- Key: SOLR-6523 URL: https://issues.apache.org/jira/browse/SOLR-6523 Project: Solr Issue Type: Improvement Components: SolrJ Reporter: Anshum Gupta Assignee: Anshum Gupta Attachments: SOLR-6523.patch, SOLR-6523.patch, SOLR-6523.patch The new zk state format (as introduced by SOLR-5473) should be supported by CollectionsAdminRequest/SolrJ Create() class. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-6523) Provide SolrJ support for specifying stateFormat while creating Collections
[ https://issues.apache.org/jira/browse/SOLR-6523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14252503#comment-14252503 ] ASF subversion and git services commented on SOLR-6523: --- Commit 1646569 from [~anshumg] in branch 'dev/branches/branch_5x' [ https://svn.apache.org/r1646569 ] SOLR-6523: Provide SolrJ support for specifying stateFormat while creating Collections (merge from trunk) Provide SolrJ support for specifying stateFormat while creating Collections --- Key: SOLR-6523 URL: https://issues.apache.org/jira/browse/SOLR-6523 Project: Solr Issue Type: Improvement Components: SolrJ Reporter: Anshum Gupta Assignee: Anshum Gupta Attachments: SOLR-6523.patch, SOLR-6523.patch, SOLR-6523.patch The new zk state format (as introduced by SOLR-5473) should be supported by CollectionsAdminRequest/SolrJ Create() class. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-6864) Support registering searcher listeners in SolrCoreAware.inform(SolrCore) method
[
https://issues.apache.org/jira/browse/SOLR-6864?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14252536#comment-14252536
]
ASF subversion and git services commented on SOLR-6864:
---
Commit 1646577 from [~tomasflobbe] in branch 'dev/trunk'
[ https://svn.apache.org/r1646577 ]
SOLR-6864: Support registering searcher listeners in
SolrCoreAware.inform(SolrCore) method
Support registering searcher listeners in SolrCoreAware.inform(SolrCore)
method
---
Key: SOLR-6864
URL: https://issues.apache.org/jira/browse/SOLR-6864
Project: Solr
Issue Type: Bug
Affects Versions: 5.0, Trunk
Reporter: Tomás Fernández Löbbe
Assignee: Tomás Fernández Löbbe
Attachments: SOLR-6864.patch
I'm marking this Jira as Bug because we already have components that do this
(SuggestComponent and SpellcheckComponent), however, listeners registered at
this stage not always work.
From
https://issues.apache.org/jira/browse/SOLR-6845?focusedCommentId=14250350page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14250350
{quote}
Trying to add some unit tests to this feature I found another issue.
SuggestComponent and SpellcheckComponent rely on a {{firstSearcherListener}}
to load (and in this case, also build) some structures. These
firstSearcherListeners are registered on {{SolrCoreAware.inform()}}, however
the first searcher listener task is only added to the queue of warming tasks
if there is at least one listener registered at the time of the first
searcher creation (before SolrCoreAware.inform() is ever called). See
{code:title=SolrCore.java}
if (currSearcher == null firstSearcherListeners.size() 0) {
future = searcherExecutor.submit(new Callable() {
@Override
public Object call() throws Exception {
try {
for (SolrEventListener listener : firstSearcherListeners) {
listener.newSearcher(newSearcher, null);
}
} catch (Throwable e) {
SolrException.log(log, null, e);
if (e instanceof Error) {
throw (Error) e;
}
}
return null;
}
});
}
{code}
I'll create a new Jira for this
{quote}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Solr-Tests-5.x-Java7 - Build # 2346 - Failure
Build: https://builds.apache.org/job/Lucene-Solr-Tests-5.x-Java7/2346/ All tests passed Build Log: [...truncated 59412 lines...] BUILD FAILED /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/build.xml:529: The following error occurred while executing this line: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/build.xml:436: The following error occurred while executing this line: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/extra-targets.xml:105: The following error occurred while executing this line: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/extra-targets.xml:206: The following files have the svn:keywords property set: * ./solr/core/src/test/org/apache/solr/cloud/overseer/ZkStateWriterTest.java Total time: 106 minutes 10 seconds Build step 'Invoke Ant' marked build as failure Archiving artifacts Sending artifact delta relative to Lucene-Solr-Tests-5.x-Java7 #2345 Archived 1 artifacts Archive block size is 32768 Received 0 blocks and 464 bytes Compression is 0.0% Took 24 ms Recording test results Email was triggered for: Failure Sending email for trigger: Failure - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-6864) Support registering searcher listeners in SolrCoreAware.inform(SolrCore) method
[
https://issues.apache.org/jira/browse/SOLR-6864?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14252550#comment-14252550
]
ASF subversion and git services commented on SOLR-6864:
---
Commit 1646581 from [~tomasflobbe] in branch 'dev/branches/branch_5x'
[ https://svn.apache.org/r1646581 ]
SOLR-6864: Support registering searcher listeners in
SolrCoreAware.inform(SolrCore) method
Support registering searcher listeners in SolrCoreAware.inform(SolrCore)
method
---
Key: SOLR-6864
URL: https://issues.apache.org/jira/browse/SOLR-6864
Project: Solr
Issue Type: Bug
Affects Versions: 5.0, Trunk
Reporter: Tomás Fernández Löbbe
Assignee: Tomás Fernández Löbbe
Attachments: SOLR-6864.patch
I'm marking this Jira as Bug because we already have components that do this
(SuggestComponent and SpellcheckComponent), however, listeners registered at
this stage not always work.
From
https://issues.apache.org/jira/browse/SOLR-6845?focusedCommentId=14250350page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14250350
{quote}
Trying to add some unit tests to this feature I found another issue.
SuggestComponent and SpellcheckComponent rely on a {{firstSearcherListener}}
to load (and in this case, also build) some structures. These
firstSearcherListeners are registered on {{SolrCoreAware.inform()}}, however
the first searcher listener task is only added to the queue of warming tasks
if there is at least one listener registered at the time of the first
searcher creation (before SolrCoreAware.inform() is ever called). See
{code:title=SolrCore.java}
if (currSearcher == null firstSearcherListeners.size() 0) {
future = searcherExecutor.submit(new Callable() {
@Override
public Object call() throws Exception {
try {
for (SolrEventListener listener : firstSearcherListeners) {
listener.newSearcher(newSearcher, null);
}
} catch (Throwable e) {
SolrException.log(log, null, e);
if (e instanceof Error) {
throw (Error) e;
}
}
return null;
}
});
}
{code}
I'll create a new Jira for this
{quote}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-6855) bin/solr -e dih launches, but has some path cruft issues preventing some of the imports don't work
[
https://issues.apache.org/jira/browse/SOLR-6855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hoss Man updated SOLR-6855:
---
Attachment: SOLR-6855.patch
first start at a patch...
* fixes the READEME.txt
* fixes the hsqldb tika paths by using \$\{solr.install.dir\}
* removes the (unused) managed_en fieldType from all the schemas
When commiting this:
* these files can be removed and added to svn:ignores for this dir (should have
never been commited in the first place):
** solr/example/example-DIH/hsqldb/ex.properties
** solr/example/example-DIH/hsqldb/ex.log
* all of the ./conf dirs should have dataimport.properties added t o
svn:ignore
Things that still need work:
* {{_rest_managed.json}} files are still created for every collection - i
thought these were only created if/when something wanted to use the resource
manager, but i may be wrong -- if there is no way to avoid them, we should
commit empty copies of them
Probably should be defered to a distinct issue but...
these example configs are still big and bloated and carry over a lot of stuff
that isn't needed to make them work - i really think that the dih
solrconfig.xml and schema.xml should be as small as freaking possible to still
demonstrate the dih features they use, with comments directing people to
basic_configs the sample_techproducts_configs to see other features of solr.
ie: we don't bother having dih configured in the techproducs example because
it's not needed -- why do we have a bunch of tech product cruft that's not
needed configured in the dih examples?
bin/solr -e dih launches, but has some path cruft issues preventing some of
the imports don't work
--
Key: SOLR-6855
URL: https://issues.apache.org/jira/browse/SOLR-6855
Project: Solr
Issue Type: Bug
Reporter: Hoss Man
Priority: Blocker
Fix For: 5.0
Attachments: SOLR-6855.patch
While trying to update this ref guide page...
https://cwiki.apache.org/confluence/display/solr/Uploading+Structured+Data+Store+Data+with+the+Data+Import+Handler
I encountered a bunch of problems when running {{bin/solr -e dih}}:
# every collection in {{example/example-DIH/solr}} started getting
\_rest_managed\* and \_schema_analysis\* files created in it
#* either we should commit these empty files into the example, or pair down
the schema's in these collections not to use these fieldTypes
# a {{server/example-DIH}} directory got created containing some hsqldb logs
properties
# at least 2 of the full import commands don't seem to work
#* the DB probably isn't working because the path to the hsqldb may not be
correct anymore - hence the problem above as well (JDBC probably relative to
CWD at the moment? need sys prop to be relative to solr home?)
#* the tika import doesn't seem to work either - probably another relative
path problem
# the {{example/example-DIH/README.txt}} file still needs updated to refer to
{{bin/solr -e dih}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Resolved] (SOLR-6864) Support registering searcher listeners in SolrCoreAware.inform(SolrCore) method
[
https://issues.apache.org/jira/browse/SOLR-6864?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tomás Fernández Löbbe resolved SOLR-6864.
-
Resolution: Fixed
Fix Version/s: Trunk
5.0
Support registering searcher listeners in SolrCoreAware.inform(SolrCore)
method
---
Key: SOLR-6864
URL: https://issues.apache.org/jira/browse/SOLR-6864
Project: Solr
Issue Type: Bug
Affects Versions: 5.0, Trunk
Reporter: Tomás Fernández Löbbe
Assignee: Tomás Fernández Löbbe
Fix For: 5.0, Trunk
Attachments: SOLR-6864.patch
I'm marking this Jira as Bug because we already have components that do this
(SuggestComponent and SpellcheckComponent), however, listeners registered at
this stage not always work.
From
https://issues.apache.org/jira/browse/SOLR-6845?focusedCommentId=14250350page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14250350
{quote}
Trying to add some unit tests to this feature I found another issue.
SuggestComponent and SpellcheckComponent rely on a {{firstSearcherListener}}
to load (and in this case, also build) some structures. These
firstSearcherListeners are registered on {{SolrCoreAware.inform()}}, however
the first searcher listener task is only added to the queue of warming tasks
if there is at least one listener registered at the time of the first
searcher creation (before SolrCoreAware.inform() is ever called). See
{code:title=SolrCore.java}
if (currSearcher == null firstSearcherListeners.size() 0) {
future = searcherExecutor.submit(new Callable() {
@Override
public Object call() throws Exception {
try {
for (SolrEventListener listener : firstSearcherListeners) {
listener.newSearcher(newSearcher, null);
}
} catch (Throwable e) {
SolrException.log(log, null, e);
if (e instanceof Error) {
throw (Error) e;
}
}
return null;
}
});
}
{code}
I'll create a new Jira for this
{quote}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-6540) strdist() causes NPE if doc is missing field
[
https://issues.apache.org/jira/browse/SOLR-6540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hoss Man updated SOLR-6540:
---
Description:
If you try to use the strdist function on a field which is missing in some
docs, you'll get a NullPointerException
A workarround in some contexts can be to wrap the strdist function in an if
that checks exists(fieldname) and returns some suitable default if it's not
found.
{code}
THIS: if(exists(field_name_s),strdist(literal,field_name_s,edit),0)
INSTEAD OF: strdist(literal,field_name_s,edit)
{code}
was:
If you try to use the strdist function on a field which is missing in some
docs, you'll get a NullPointerException
A workarround in some contexts can be to wrap the strdist function in an if
that checks exists(fieldname) and returns some suitable default if it's not
found.
{code}
THIS: if(exists(foo),strdist(literal,field_name_s,edit),0)
INSTEAD OF: strdist(literal,field_name_s,edit)
{code}
strdist() causes NPE if doc is missing field
Key: SOLR-6540
URL: https://issues.apache.org/jira/browse/SOLR-6540
Project: Solr
Issue Type: Bug
Reporter: Hoss Man
Fix For: 5.0, Trunk
Attachments: SOLR-6540.patch
If you try to use the strdist function on a field which is missing in some
docs, you'll get a NullPointerException
A workarround in some contexts can be to wrap the strdist function in an if
that checks exists(fieldname) and returns some suitable default if it's not
found.
{code}
THIS:
if(exists(field_name_s),strdist(literal,field_name_s,edit),0)
INSTEAD OF: strdist(literal,field_name_s,edit)
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-6523) Provide SolrJ support for specifying stateFormat while creating Collections
[ https://issues.apache.org/jira/browse/SOLR-6523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anshum Gupta updated SOLR-6523: --- Fix Version/s: Trunk 5.0 Provide SolrJ support for specifying stateFormat while creating Collections --- Key: SOLR-6523 URL: https://issues.apache.org/jira/browse/SOLR-6523 Project: Solr Issue Type: Improvement Components: SolrJ Reporter: Anshum Gupta Assignee: Anshum Gupta Fix For: 5.0, Trunk Attachments: SOLR-6523.patch, SOLR-6523.patch, SOLR-6523.patch The new zk state format (as introduced by SOLR-5473) should be supported by CollectionsAdminRequest/SolrJ Create() class. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Resolved] (SOLR-6523) Provide SolrJ support for specifying stateFormat while creating Collections
[ https://issues.apache.org/jira/browse/SOLR-6523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anshum Gupta resolved SOLR-6523. Resolution: Fixed Provide SolrJ support for specifying stateFormat while creating Collections --- Key: SOLR-6523 URL: https://issues.apache.org/jira/browse/SOLR-6523 Project: Solr Issue Type: Improvement Components: SolrJ Reporter: Anshum Gupta Assignee: Anshum Gupta Attachments: SOLR-6523.patch, SOLR-6523.patch, SOLR-6523.patch The new zk state format (as introduced by SOLR-5473) should be supported by CollectionsAdminRequest/SolrJ Create() class. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (SOLR-6868) Investigate failing CollectionsAPIDistributedZkTest
Anshum Gupta created SOLR-6868:
--
Summary: Investigate failing CollectionsAPIDistributedZkTest
Key: SOLR-6868
URL: https://issues.apache.org/jira/browse/SOLR-6868
Project: Solr
Issue Type: Bug
Components: Tests
Reporter: Anshum Gupta
CollectionsAPIDistributedZkTest has been failing (on trunk and 5x) off late.
Running the following
bq. ant test -Dtestcase=CollectionsAPIDistributedZkTest
-Dtests.method=testDistribSearch -Dtests.seed=8219942E65A94EC1
-Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=nl
-Dtests.timezone=America/Halifax -Dtests.asserts=false
-Dtests.file.encoding=ISO-8859-1
causes the following errors/exceptions:
{code}
[junit4] Throwable #1: org.junit.ComparisonFailure:
expected:...9942E65A94EC1-002/te[mpDir-003/solrj_test_core_props_shard1_replica1]
but
was:...9942E65A94EC1-002/te[stPropertyParamsForCreate-001/instanceDir-fmchl]
[junit4]at
__randomizedtesting.SeedInfo.seed([8219942E65A94EC1:3FF1A3612F62EFD]:0)
[junit4]at
org.apache.solr.cloud.CollectionsAPIDistributedZkTest.checkInstanceDirs(CollectionsAPIDistributedZkTest.java:1154)
[junit4]at
org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testCollectionsAPI(CollectionsAPIDistributedZkTest.java:901)
[junit4]at
org.apache.solr.cloud.CollectionsAPIDistributedZkTest.doTest(CollectionsAPIDistributedZkTest.java:205)
[junit4]at
org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:869)
[junit4]at java.lang.Thread.run(Thread.java:745)
{code}
and running it without that seed (and may be with the seed too) often leads to :
{code}
oasu.SolrIndexWriter.rollback ERROR Exception rolling back IndexWriter
org.apache.lucene.store.AlreadyClosedException: refusing to delete any files:
this IndexWriter hit an unrecoverable exception
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Assigned] (SOLR-6848) Typing incorrect configuration name for cloud example aborts the script, leaves nodes running
[ https://issues.apache.org/jira/browse/SOLR-6848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Timothy Potter reassigned SOLR-6848: Assignee: Timothy Potter Typing incorrect configuration name for cloud example aborts the script, leaves nodes running - Key: SOLR-6848 URL: https://issues.apache.org/jira/browse/SOLR-6848 Project: Solr Issue Type: Bug Components: scripts and tools Affects Versions: 5.0 Reporter: Alexandre Rafalovitch Assignee: Timothy Potter Priority: Minor Labels: example Please choose a configuration for the gettingstarted collection, available options are: *basic_configs*, data_driven_schema_configs, or sample_techproducts_configs \[data_ddriven_schema_configs\] *basic_config* Connecting to ZooKeeper at localhost:9983 Exception in thread main java.io.FileNotFoundException: Specified config basic_config not found in .../solr-5.0.0-SNAPSHOT/server/solr/configsets at org.apache.solr.util.SolrCLI$CreateCollectionTool.runCloudTool(SolrCLI.java:867) at org.apache.solr.util.SolrCLI$CreateCollectionTool.runTool(SolrCLI.java:824) at org.apache.solr.util.SolrCLI.main(SolrCLI.java:185) SolrCloud example running, please visit http://localhost:8983/solr $ -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Resolved] (SOLR-6833) bin/solr -e foo should not use server/solr as the SOLR_HOME
[ https://issues.apache.org/jira/browse/SOLR-6833?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Timothy Potter resolved SOLR-6833. -- Resolution: Fixed bin/solr -e foo should not use server/solr as the SOLR_HOME --- Key: SOLR-6833 URL: https://issues.apache.org/jira/browse/SOLR-6833 Project: Solr Issue Type: Bug Reporter: Hoss Man Assignee: Timothy Potter Fix For: 5.0 Attachments: SOLR-6833.patch i think it's weird right now that running bin/solr with the -e (example) option causes it to create example solr instances inside the server directory. i think that's fine for running solr normally (ie: start) but if you use -e that seems like the solr.solr.home for those example should instead be created under $SOLR_TIP/example. I would even go so far as to suggest that the *log* files created should live in that directory as well. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-6848) Typing incorrect configuration name for cloud example aborts the script, leaves nodes running
[ https://issues.apache.org/jira/browse/SOLR-6848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14252606#comment-14252606 ] Timothy Potter commented on SOLR-6848: -- There's some other hardening I want to do around selecting configsets, such as allowing the user to put their own custom configs in the configsets directory and have the script be able to use it correctly. Typing incorrect configuration name for cloud example aborts the script, leaves nodes running - Key: SOLR-6848 URL: https://issues.apache.org/jira/browse/SOLR-6848 Project: Solr Issue Type: Bug Components: scripts and tools Affects Versions: 5.0 Reporter: Alexandre Rafalovitch Assignee: Timothy Potter Priority: Minor Labels: example Please choose a configuration for the gettingstarted collection, available options are: *basic_configs*, data_driven_schema_configs, or sample_techproducts_configs \[data_ddriven_schema_configs\] *basic_config* Connecting to ZooKeeper at localhost:9983 Exception in thread main java.io.FileNotFoundException: Specified config basic_config not found in .../solr-5.0.0-SNAPSHOT/server/solr/configsets at org.apache.solr.util.SolrCLI$CreateCollectionTool.runCloudTool(SolrCLI.java:867) at org.apache.solr.util.SolrCLI$CreateCollectionTool.runTool(SolrCLI.java:824) at org.apache.solr.util.SolrCLI.main(SolrCLI.java:185) SolrCloud example running, please visit http://localhost:8983/solr $ -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-6851) oom_solr.sh problems and other issues that make it hard to run Solr as a service on *nix
[
https://issues.apache.org/jira/browse/SOLR-6851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Timothy Potter updated SOLR-6851:
-
Summary: oom_solr.sh problems and other issues that make it hard to run
Solr as a service on *nix (was: oom_solr.sh problems)
oom_solr.sh problems and other issues that make it hard to run Solr as a
service on *nix
Key: SOLR-6851
URL: https://issues.apache.org/jira/browse/SOLR-6851
Project: Solr
Issue Type: Bug
Reporter: Hoss Man
Assignee: Timothy Potter
Fix For: 5.0
noticed 2 problems with the oom_solr.sh script...
1) the script is only being run with the port of hte solr instance to
terminate, so the log messages aren't getting writen to the correct directory
-- if we change hte script to take a log dir/file as an argument, we can
ensure the logs are written to the correct place
2) on my ubuntu linux machine (where /bin/sh is a symlink to /bin/dash),
the console log is recording a script error when java runs oom_solr.sh...
{noformat}
#
# java.lang.OutOfMemoryError: Java heap space
# -XX:OnOutOfMemoryError=/home/hossman/lucene/5x_dev/solr/bin/oom_solr.sh
8983
# Executing /bin/sh -c /home/hossman/lucene/5x_dev/solr/bin/oom_solr.sh
8983...
/home/hossman/lucene/5x_dev/solr/bin/oom_solr.sh: 20: [: 14305: unexpected
operator
Running OOM killer script for process 14305 for Solr on port 8983
Killed process 14305
{noformat}
steps to reproduce: {{bin/solr -e techproducts -m 10m}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
