Hi Rohit, Just for clarification, as stated in the java doc, metricChange "is called whenever a metric is updated or added". It is not the function when a metric is recorded; in fact, the metrics collection is in the "pull" model, where the implemented reporters can fetch the current calculated values based on its metric type (rate, min, max, histogram, etc). You may want to first take a look at the default JMXReporter implementation with the " getAttribute" function which does the pulling to make sure your customized reporter does the right behavior.
Guozhang On Tue, Aug 30, 2016 at 6:01 PM, Guozhang Wang <wangg...@gmail.com> wrote: > Hello Rohit, > > As you are already aware, monitoring kafka streams is no difference than > monitoring kafka producers / consumers. So you can just monitor on its > embedded consumer's "records-lag-max" metric, which gets recorded > whenever the consumer received the fetch response. > > As for your application, your way of passing the class name of your > implemented MetricsReporter through the StreamsConfig is correct. Have > you seen records being processing by your streams application, meaning that > there are indeed some fetched records from the fetch response? > > Guozhang > > > On Mon, Aug 29, 2016 at 10:48 AM, Rohit Valsakumar <rvalsaku...@tivo.com> > wrote: > >> Hi all, >> >> Any opinions about monitoring the records-lag-max for a kafka streams job? >> >> Thanks, >> Rohit >> >> On 8/26/16, 2:53 PM, "Rohit Valsakumar" <rvalsaku...@tivo.com> wrote: >> >> >Hi all, >> > >> >I want to monitor the max lag of a kafka streams job which is consuming >> >from three topics and to do that I have implemented the MetricsReporter >> >interface which I pass through the Streams Configuration to the >> >KafkaStreams object. In the implementation¹s metricChange() method I >> >have logging for the metric with group consumer-fetch-manager-metrics and >> >name records-lag-max. However, the implementation never receives a >> >metricChange notification for 'records-lag-max¹. >> > >> >I would like to know if what I am doing makes sense, is the correct >> >approach and whether the implementation should be getting notifications >> >for 'records-lag-max¹ periodically or only when the max lag is changing >> >i.e. Increasing/decreasing. >> > >> >Thanks, >> >Rohit >> > >> > >> >________________________________ >> > >> >This email and any attachments may contain confidential and privileged >> >material for the sole use of the intended recipient. Any review, copying, >> >or distribution of this email (or any attachments) by others is >> >prohibited. If you are not the intended recipient, please contact the >> >sender immediately and permanently delete this email and any attachments. >> >No employee or agent of TiVo Inc. is authorized to conclude any binding >> >agreement on behalf of TiVo Inc. by email. Binding agreements with TiVo >> >Inc. may only be made by a signed written agreement. >> >> >> ________________________________ >> >> This email and any attachments may contain confidential and privileged >> material for the sole use of the intended recipient. Any review, copying, >> or distribution of this email (or any attachments) by others is prohibited. >> If you are not the intended recipient, please contact the sender >> immediately and permanently delete this email and any attachments. No >> employee or agent of TiVo Inc. is authorized to conclude any binding >> agreement on behalf of TiVo Inc. by email. Binding agreements with TiVo >> Inc. may only be made by a signed written agreement. >> > > > > -- > -- Guozhang > -- -- Guozhang
