[
https://issues.apache.org/jira/browse/SOLR-13534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hoss Man reopened SOLR-13534:
-----------------------------
Noble: the new TestDynamicLoadingUrl can fail easily on heavily loaded machines
when loading the "new" SolrCore instances (that result from reloading all cores
after a config change) may not complete prior to the subsequent REST calls that
depend on them
In many recent jenkins failures this even happens as a result of the first
SolrCore reload needed when executing the {{'create-requesthandler'}} config
command to register the {{/jarhandler}} used by the test to "fake" a remote
server.
Here's an example of the order of the "going to send config command" logging
from the test compared to the "CLOSING SolrCore" logging (indicating that the
_new_ reloaded version of the core is already live and ready for requests) in a
recent jenkins failure...
{noformat}
$ sed -n -e '/o.a.s.c.TestSolrConfigHandler/,+1p' -e '/config update listener
called/p' -e '/CLOSING SolrCore/p'
thetaphi_Lucene-Solr-master-MacOSX_5259.log.txt
[junit4] 2> 549681 INFO
(TEST-TestDynamicLoadingUrl.testDynamicLoadingUrl-seed#[B56AB6A699945C60]) [
] o.a.s.c.TestSolrConfigHandler going to send config command. path /config ,
payload: {
[junit4] 2> 'create-requesthandler' : { 'name' : '/jarhandler', 'class':
org.apache.solr.core.TestDynamicLoadingUrl$JarHandler, registerPath:
'/solr,/v2' }
[junit4] 2> 549688 INFO (Thread-2015) [ ] o.a.s.c.SolrCore config
update listener called for core collection1_shard2_replica_n1
[junit4] 2> 549688 INFO (Thread-2014) [ ] o.a.s.c.SolrCore config
update listener called for core collection1_shard1_replica_n2
[junit4] 2> 549689 INFO (Thread-2013) [ ] o.a.s.c.SolrCore config
update listener called for core control_collection_shard1_replica_n1
[junit4] 2> 549689 INFO (Thread-2017) [ ] o.a.s.c.SolrCore config
update listener called for core collection1_shard2_replica_n5
[junit4] 2> 549689 INFO (Thread-2016) [ ] o.a.s.c.SolrCore config
update listener called for core collection1_shard1_replica_n6
[junit4] 2> 550916 INFO (Thread-2013) [n:127.0.0.1:56493_m
c:control_collection s:shard1 r:core_node2
x:control_collection_shard1_replica_n1 ] o.a.s.c.SolrCore
[control_collection_shard1_replica_n1] CLOSING SolrCore
org.apache.solr.core.SolrCore@3bceb4b6
[junit4] 2> 550921 INFO (Thread-2015) [n:127.0.0.1:56521_m c:collection1
s:shard2 r:core_node3 x:collection1_shard2_replica_n1 ] o.a.s.c.SolrCore
[collection1_shard2_replica_n1] CLOSING SolrCore
org.apache.solr.core.SolrCore@724097d0
[junit4] 2> 550986 INFO (qtp1160063522-12178) [n:127.0.0.1:56525_m
c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n2 ]
o.a.s.c.SolrCore [collection1_shard1_replica_n2] CLOSING SolrCore
org.apache.solr.core.SolrCore@6972fc14
[junit4] 2> 551100 INFO
(TEST-TestDynamicLoadingUrl.testDynamicLoadingUrl-seed#[B56AB6A699945C60]) [
] o.a.s.c.TestSolrConfigHandler going to send config command. path /config ,
payload: {
[junit4] 2> 'add-runtimelib' : { 'name' : 'urljar', url :
'http://127.0.0.1:56525/m/collection1/jarhandler?wt=filestream'
'sha512':'e01b51de67ae1680a84a813983b1de3b592fc32f1a22b662fc9057da5953abd1b72476388ba342cad21671cd0b805503c78ab9075ff2f3951fdf75fa16981420'}}
[junit4] 2> 551111 INFO
(TEST-TestDynamicLoadingUrl.testDynamicLoadingUrl-seed#[B56AB6A699945C60]) [
] o.a.s.c.TestSolrConfigHandler going to send config command. path /config ,
payload: {
[junit4] 2> 'add-runtimelib' : { 'name' : 'urljar', url :
'http://127.0.0.1:56531/m/collection1/jarhandler?wt=filestream'
'sha512':'d01b51de67ae1680a84a813983b1de3b592fc32f1a22b662fc9057da5953abd1b72476388ba342cad21671cd0b805503c78ab9075ff2f3951fdf75fa16981420'}}
[junit4] 2> 551126 INFO (Thread-2016) [n:127.0.0.1:56536_m c:collection1
s:shard1 r:core_node8 x:collection1_shard1_replica_n6 ] o.a.s.c.SolrCore
[collection1_shard1_replica_n6] CLOSING SolrCore
org.apache.solr.core.SolrCore@3ff8c0a0
[junit4] 2> 551145 INFO (Thread-2017) [n:127.0.0.1:56531_m c:collection1
s:shard2 r:core_node7 x:collection1_shard2_replica_n5 ] o.a.s.c.SolrCore
[collection1_shard2_replica_n5] CLOSING SolrCore
org.apache.solr.core.SolrCore@11bde630
[junit4] 2> 551232 INFO (coreCloseExecutor-4027-thread-1)
[n:127.0.0.1:56493_m c:control_collection s:shard1 r:core_node2
x:control_collection_shard1_replica_n1 ] o.a.s.c.SolrCore
[control_collection_shard1_replica_n1] CLOSING SolrCore
org.apache.solr.core.SolrCore@3c0a3773
[junit4] 2> 551234 INFO (coreCloseExecutor-4026-thread-1)
[n:127.0.0.1:56521_m c:collection1 s:shard2 r:core_node3
x:collection1_shard2_replica_n1 ] o.a.s.c.SolrCore
[collection1_shard2_replica_n1] CLOSING SolrCore
org.apache.solr.core.SolrCore@735e18b7
[junit4] 2> 551235 INFO (coreCloseExecutor-4030-thread-1)
[n:127.0.0.1:56525_m c:collection1 s:shard1 r:core_node4
x:collection1_shard1_replica_n2 ] o.a.s.c.SolrCore
[collection1_shard1_replica_n2] CLOSING SolrCore
org.apache.solr.core.SolrCore@2afc2ed1
[junit4] 2> 551236 INFO (coreCloseExecutor-4029-thread-1)
[n:127.0.0.1:56536_m c:collection1 s:shard1 r:core_node8
x:collection1_shard1_replica_n6 ] o.a.s.c.SolrCore
[collection1_shard1_replica_n6] CLOSING SolrCore
org.apache.solr.core.SolrCore@7c1b3d2c
[junit4] 2> 551237 INFO (coreCloseExecutor-4028-thread-1)
[n:127.0.0.1:56531_m c:collection1 s:shard2 r:core_node7
x:collection1_shard2_replica_n5 ] o.a.s.c.SolrCore
[collection1_shard2_replica_n5] CLOSING SolrCore
org.apache.solr.core.SolrCore@13862a67
{noformat}
Note that the first call to the {{add-runtimelib}} command (where the test
explicitly sends the incorrect sha512) points to
{{[http://127.0.0.1:56525/m/collection1/]...}} which has already finished
reloading – so that call passes. but the second call to {{add-runtimelib}}
(with the correct sha512) points to
{{[http://127.0.0.1:56531/m/collection1/]...}} and the SolrCore running on port
#56531 has *NOT* finished reloading yet – so the test fails because the "old"
SolrCore the processes that request doesn't have a {{/jarhandler}} registered.
----
The test reliability could potentially be improved slightly by using a "mock" /
standalone HTTP server to host the jar, instead of faking it via a request
handler (and randomly picking a node to hit for that request handler every
time) so we would know for certain it was alive and ready when sending the
{{'add-runtimelib'}} ... but i'm not sure if that would really resolve all the
potential problems or just kick the can down the road a bit ...
In particular i'm not sure if the subsequent {{'create-requesthandler'}} call
(attempting to use the new runtime lib) would be garunteed to succeed, w/o some
sort of hook to first ensure that every SolrCore has reloaded?
(likewise, i'm not sure if the subsequent
{{TestSolrConfigHandler.testForResponseElement}} calls are garunteed to work
... it looks like they have some built in retry mechanism but i didn't dig that
deep)
> Dynamic loading of jars from a url
> ----------------------------------
>
> Key: SOLR-13534
> URL: https://issues.apache.org/jira/browse/SOLR-13534
> Project: Solr
> Issue Type: Improvement
> Reporter: Noble Paul
> Assignee: Noble Paul
> Priority: Major
> Fix For: 8.2
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> Dynamic loading is possible from {{.system}} collection. It's much easier to
> host the jars on a remote service and load it from there. This way the user
> should have no problem in loading jars when the {{.system}} collection is not
> available for some reason.
> The steps should look as follows
> # get the hash of your jar file. {{openssl dgst -sha512 <jar>}}
> # upload it your hosting service . say the location is
> {{[http://host:port/my-jar/location|http://hostport/]}}
> # create a runtime lib entry for the collection as follows
> {code:java}
> curl http://localhost:8983/solr/techproducts/config -H
> 'Content-type:application/json' -d '{
> "add-runtimelib": { "name":"jarblobname",
> "sha512":"e94bb3990b39aacdabaa3eef7ca6102d96fa46766048da50269f25fd41164440a4e024d7a7fb0d5ec328cd8322bb65f5ba7886e076a8f224f78cb310fd45896d"
> , "url" : "http://host:port/my-jar/loaction"}
> }'
> {code}
> to update the jar, just repeat the steps and use the {{update-runtimelib}} to
> update the sha512 hash
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
