Hi, Not merged in yet but this is an example pr that is mocking metrics and checking they are properly updated: https://github.com/apache/flink/pull/4725
On Fri, Oct 13, 2017 at 1:49 PM, Aljoscha Krettek <aljos...@apache.org> wrote: > I think we could add this functionality to the (operator) test harnesses. > I.e. add a mock MetricGroup thingy in there that you can query to check the > state of metrics. > > > On 13. Oct 2017, at 13:50, Chesnay Schepler <ches...@apache.org> wrote: > > I meant that you could unit-test the behavior of the function in > isolation. You could create a dummy metric group that > verifies that the correct counters are being registered (based on names i > guess), as well as provide access to them. > Mock some input and observe whether the counter value is being modified. > > Whether this is a viable option depends a bit on the complexity of the > function of course, that is how much how mocking > you would have to do. > > On 13.10.2017 11:18, Piotr Nowojski wrote: > > For testing Link applications in general you can read > https://ci.apache.org/projects/flink/flink-docs-release-1.4/dev/stream/ > testing.html > > However as we said before, testing metrics would require using custom or a > imx reporter. > > Yes, please report this bug in Jira. > > Thanks, Piotrek > > On 13 Oct 2017, at 04:31, Colin Williams <colin.williams.seat...@gmail.com> > wrote: > > Team wants an integration test, I'm not sure what unit test you had in > mind. Actually feel that I've been trying to avoid the reporter method but > that would be more end to end. > > The documentation for metrics and Scala are missing with the exception of > Gauge: https://ci.apache.org/projects/flink/flink-docs- > release-1.3/monitoring/metrics.html . Should I file a issue against that? > > Then it leaves you guessing a little bit how to implement Counters. One > approach tried was using objects > > object PointFilter extends RichMapFunction[... > > @transient lazy val someCounter = > getRuntimeContext.getMetricGroup.counter(...) > > > This allowed access to the counter before and after execution . However > between the unit tests the Counter kept its value also and that's a no for > the test. Think that might be an issue with ScalaTest. > > I've tried to get at the counter from some other directions like trying to > find a way to inject a reporter to get it's state. But don't see a way to > do it. So probably the best thing to do is fire up something to collect the > metrics from the reporter. > > On Thu, Oct 12, 2017 at 5:29 AM, Chesnay Schepler <ches...@apache.org> > wrote: > >> Well damn, i should've read the second part of the initial mail. >> >> I'm wondering though, could you not unit-test this behavior? >> >> >> On 12.10.2017 14:25, Chesnay Schepler wrote: >> >>> You could also write a custom reporter that opens a socket or similar >>> for communication purposes. >>> >>> You can then either query it for the metrics, or even just trigger the >>> verification in the reporter, >>> and fail with an error if the reporter returns an error. >>> >>> On 12.10.2017 14:02, Piotr Nowojski wrote: >>> >>>> Hi, >>>> >>>> Doing as you proposed using JMXReporter (or custom reporter) should >>>> work. I think there is no easier way to do this at the moment. >>>> >>>> Piotrek >>>> >>>> On 12 Oct 2017, at 04:58, Colin Williams <colin.williams.seattle@gmail. >>>>> com> wrote: >>>>> >>>>> I have a RichMapFunction and I'd like to ensure Meter fields are >>>>> properly incremented. I've been trying to think of the best way to do >>>>> this. >>>>> Currently I think that I'd need to either implement my own reporter (or >>>>> use >>>>> JMX) and write to a socket, create a listener and wait for the reporter to >>>>> send the message. >>>>> >>>>> Is this a good approach for writing the test, or should I be >>>>> considering something else? >>>>> >>>> >>>> >>> >>> >> > > > >
