That's what I had in mind, just need to figure out how to do it with CodaHale. Want to avoid CodaHale MetricRegistry class floating around in code. 1st step would be to migrate to CodaHale, I am yet to figure out how managing the Reporter part. Let me try to put a patch together.
On Wed, Nov 12, 2014 at 9:29 AM, Hari Shreedharan <hshreedha...@cloudera.com> wrote: > Currently the metrics exposed via Sources/sinks/channels are hardcoded. We > could add a method to add custom metrics (I think it is already there in > MonitoredCounterGroup, just not in the *Counter classes). > > > Thanks, > Hari > > On Tue, Nov 11, 2014 at 6:34 PM, Ashish <paliwalash...@gmail.com> wrote: > >> I was just playing around so piggy-backed it. The changes have to be >> within MonitoredCounterGroup and its sub-classes, as its used across >> the implementations. Let me raise a JIRA for this. >> One thing that I would like to have in future is to allow >> Sinks/Channels/Sources to be able to have custom counters/Meters and >> be available to have those Metrics available via the framework. I >> think CounterGroup was created for the same purpose. >> Let me create a JIRA for this and work on it. >> thanks >> ashish >> On Wed, Nov 12, 2014 at 1:11 AM, Hari Shreedharan >> <hshreedha...@cloudera.com> wrote: >>> I think it is best to replace the current system rather than having >>> multiple metrics systems. We should implement it such that it is >>> transparent to user code, and the changes only go into >>> MonitoredCounterGroup (the CounterGroup class is obsolete - dont bother). >>> >>> >>> Thanks, >>> Hari >>> >>> On Mon, Nov 10, 2014 at 5:36 PM, Ashish <paliwalash...@gmail.com> wrote: >>> >>>> Codahale does have lot more features like the rate/sec one, which is >>>> most needed metrics. It provides a lot of reporters out of box >>>> (Nagios, HTTP, Ganglia, Graphite, Console etc) so we just need to >>>> enable it, rather than writing custom components. >>>> As of now we have to hide it within MonitoredCounterGroup and its >>>> sub-classes, that's what I did for running it. It was running parallel >>>> to existing system. Let me work more on it and see if I can simplify >>>> the same. >>>> On Tue, Nov 11, 2014 at 4:50 AM, Hari Shreedharan >>>> <hshreedha...@cloudera.com> wrote: >>>>> Is it easier to use than the current one and/or does it give better >>>>> performance? You’d need to support the current metrics API >>>>> (MonitoredCounterGroup, SourceCounter, SinkCounter, ChannelCounter etc). >>>>> >>>>> >>>>> Thanks, >>>>> Hari >>>>> >>>>> On Sat, Nov 8, 2014 at 8:21 PM, Ashish <paliwalash...@gmail.com> wrote: >>>>> >>>>>> Hi, >>>>>> Have hacked a bit into our existing instrumentation package and piggy >>>>>> backed cohahale metrics package. Here is one sample for Spooled >>>>>> Directory source (with instrumentation only for Source and Channel ), >>>>>> using console reporter >>>>>> -- Gauges >>>>>> ---------------------------------------------------------------------- >>>>>> org.apache.flume.instrumentation.ChannelCounter.channel.current.size >>>>>> value = 200 >>>>>> org.apache.flume.instrumentation.ChannelCounter.channel.fill.percentage >>>>>> value = 2.0 >>>>>> org.apache.flume.instrumentation.SourceCounter.src.open-connection.count >>>>>> value = 0 >>>>>> -- Counters >>>>>> -------------------------------------------------------------------- >>>>>> org.apache.flume.instrumentation.ChannelCounter.channel.event.put.attempt >>>>>> count = 1138800 >>>>>> org.apache.flume.instrumentation.ChannelCounter.channel.event.put.success >>>>>> count = 1138800 >>>>>> org.apache.flume.instrumentation.ChannelCounter.channel.event.take.attempt >>>>>> count = 1138601 >>>>>> org.apache.flume.instrumentation.ChannelCounter.channel.event.take.success >>>>>> count = 1138600 >>>>>> org.apache.flume.instrumentation.SourceCounter.src.events.accepted >>>>>> count = 1138800 >>>>>> org.apache.flume.instrumentation.SourceCounter.src.events.received >>>>>> count = 1138800 >>>>>> src.append-batch.accepted >>>>>> count = 11388 >>>>>> src.append-batch.received >>>>>> count = 11388 >>>>>> src.append.accepted >>>>>> count = 0 >>>>>> src.append.received >>>>>> count = 0 >>>>>> -- Meters >>>>>> ---------------------------------------------------------------------- >>>>>> eventAcceptedMeter >>>>>> count = 1138800 >>>>>> mean rate = 106478.56 events/second >>>>>> 1-minute rate = 93420.18 events/second >>>>>> 5-minute rate = 91389.24 events/second >>>>>> 15-minute rate = 91037.40 events/second >>>>>> eventReceivedMeter >>>>>> count = 1138800 >>>>>> mean rate = 106462.14 events/second >>>>>> 1-minute rate = 93420.18 events/second >>>>>> 5-minute rate = 91389.24 events/second >>>>>> 15-minute rate = 91037.40 events/second >>>>>> If there is interest in the community, can raise a jira and continue >>>>>> to work on it. >>>>>> -- >>>>>> thanks >>>>>> ashish >>>>>> Blog: http://www.ashishpaliwal.com/blog >>>>>> My Photo Galleries: http://www.pbase.com/ashishpaliwal >>>> -- >>>> thanks >>>> ashish >>>> Blog: http://www.ashishpaliwal.com/blog >>>> My Photo Galleries: http://www.pbase.com/ashishpaliwal >> -- >> thanks >> ashish >> Blog: http://www.ashishpaliwal.com/blog >> My Photo Galleries: http://www.pbase.com/ashishpaliwal -- thanks ashish Blog: http://www.ashishpaliwal.com/blog My Photo Galleries: http://www.pbase.com/ashishpaliwal
