[
https://issues.apache.org/jira/browse/SOLR-7147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14334171#comment-14334171
]
Hoss Man commented on SOLR-7147:
--------------------------------
bq. I created a TrackingShardHandlerFactory which can record shard requests
sent from any node. There are a few helper methods to get requests by shard and
by purpose.
I like the idea ... hooking into the ShardHandlerFactory seems like a good
solution to these types of problems...
bq. I will likely move the TrackingShardHandlerFactory into its own issue
because it is helpful for other distributed tests as well.
Yeah, a top level class in the test-framework would be good, but...
I'm not sure if a {{Map<String, List<ShardRequestAndParams>>}} is really the
best solution here?
It's good enough for this small test case, but as a general solution ...
consider Test Classes were folks want to setup their distrib cluster, do a
handfull of specific queries to monitor what gets sent to each shard, and then
in other test methods crank out thousands of distrib queries. the way this is
setup, that's either going to eat up a ton of ram, or the test is going to eat
up a ton of ram, or the test is going to need to be sprinkled with tons of
calls to some clearOutShardHandlerTrackers() utility method
perhaps instead, the TrackingShardHandlerFactory should have a
setTrackingQueue/getTrackingQueue methods for a
{{Queue<ShardRequestAndParams>}} (and ShardRequestAndParams should also have a
{{String shard}} property - although isn't this already in the ShardRequest?).
By default, the Queue is null and TrackingShardHandlerFactory doesn't track
anything -- but if it's non-null, then the ShardHandler calls {{add(new
ShardRequestAndParams(...)}} just before passing executing it.
this way, there is almost no overhead to using TrackingShardHandlerFactory
unless/until a test explicitly specifies a Queue. Test classes, for the
portions of the test when they want to monitor shard requests, could call
setTrackingQueue on a custom subclass of AbstractQueue that directly asserts
things like...
{code}
if (req.sreq.purpose & PURPOSE_GET_TOP_IDS) {
assertEquals(expected, req.sreq.getParams(FL));
}
{code}
.. alternatively, a LinkedBlockingQueue could be specified, and the test class
could spin up an Executor that was would poll that Queue continuously
inspecting them & recording info for processing/assert at the end of the test.
at any point when the test case no longer cares about tracking,
setTrackingQueue(null) again.
The simplier use case of what you're doing here could still pretty easily be
supported, just by providing a simple special case Queue class that has a
{{Map<String, List<ShardRequestAndParams>> getAll()}} method on it, and tests
could do something like...
{code}
TrackAllQueue myQueue =
TrackingShardHandlerFactory.setupTrackAllQueue(myMiniSolrCloudCluster,
collectionName);
// .. do test stuff ..
TrackingShardHandlerFactory.setAllTrackingQueues(myMiniSolrCloudCluster, null)
List<ShardRequestAndParams> list = myQueye.getAll().get(coreUrl);
// ...
{code}
bq. I will likely move the TrackingShardHandlerFactory into its own issue
because it is helpful for other distributed tests as well. I also need to
decouple it from the MiniSolrCloudCluster abstraction.
I think the main thing would be some static methods in
TrackingShardHandlerFactory to get/set the TrackingQueue for all
JettySolrRunners in a list, or on a MiniSolrCloudCluster (in which case it just
pulls out the JettySolrRunners from there)
what do you think?
> Intoduce new TrackingShardHandlerFactory for monitoring what requests are
> sent to shards during tests
> -----------------------------------------------------------------------------------------------------
>
> Key: SOLR-7147
> URL: https://issues.apache.org/jira/browse/SOLR-7147
> Project: Solr
> Issue Type: Improvement
> Reporter: Hoss Man
>
> this is an idea shalin proposed as part of the testing for SOLR-7128...
> bq. I created a TrackingShardHandlerFactory which can record shard requests
> sent from any node. There are a few helper methods to get requests by shard
> and by purpose.
> ...
> bq. I will likely move the TrackingShardHandlerFactory into its own issue
> because it is helpful for other distributed tests as well. I also need to
> decouple it from the MiniSolrCloudCluster abstraction.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
