[
https://issues.apache.org/jira/browse/SOLR-12791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16668423#comment-16668423
]
Jan Høydahl commented on SOLR-12791:
------------------------------------
I worked a bit to improve the tests, and discovered that when beasting the
tests the assertion on metric count gets wrong. We assert the counts by talking
to the cluster's Jetty Runners, getting core container and metrics manager from
there. So if another test runs in parallel the numbers will be wrong.
With Mark's push towards less flaky tests, I'd like this one to be beast-able
:) Here are the alternatives I see:
* A: Initialize the cluster inside the test method instead of in BeforeClass.
Then each test instance will have their own cluster with their own metrics.
This works for test classes having only one test method.
* B: Do not assert any metrics in the various Auth tests. Instead create a new
{{TestAuthMetrics}} class with exactly one test method, spinning up a cluster
inside the test. Disadvantage is we duplicate much test logic in two places.
* C: Mock. Spin up Solr with e.g. a {{MockBasicAuthPlugin}} in security.json,
which somehow manages to keep metrics per test-method. To be honest I don't see
clearly what that would look like..
* D: Do not expect a certain count for various metrics. Instead do *one*
single assert at the end of each auth plugin's integration test where we simply
check that all counters that the test is supposed to increment are non-null.
That assertion will pass even if another test also increments the same counter
in another thread.
> Add Metrics reporting for AuthenticationPlugin
> ----------------------------------------------
>
> Key: SOLR-12791
> URL: https://issues.apache.org/jira/browse/SOLR-12791
> Project: Solr
> Issue Type: New Feature
> Security Level: Public(Default Security Level. Issues are Public)
> Components: Authentication, metrics
> Reporter: Jan Høydahl
> Assignee: Jan Høydahl
> Priority: Major
> Time Spent: 2h 10m
> Remaining Estimate: 0h
>
> Propose to add Metrics support for all Auth plugins. Will let abstract
> {{AuthenticationPlugin}} base class implement {{SolrMetricProducer}} and keep
> the counters, such as:
> * requests
> * req authenticated
> * req pass-through (no credentials and blockUnknown false)
> * req with auth failures due to wrong or malformed credentials
> * req auth failures due to missing credentials
> * errors
> * timeouts
> * timing stats
> Each implementation still needs to increment the counters etc.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
