Re: Using CodaHale metrics for monitoring

2014-11-11 Thread Hari Shreedharan
I think it is best to replace the current system rather than having multiple 
metrics systems. We should implement it such that it is transparent to user 
code, and the changes only go into MonitoredCounterGroup (the CounterGroup 
class is obsolete - dont bother).


Thanks,
Hari

On Mon, Nov 10, 2014 at 5:36 PM, Ashish paliwalash...@gmail.com wrote:

 Codahale does have lot more features like the rate/sec one, which is
 most needed metrics. It provides a lot of reporters out of box
 (Nagios, HTTP, Ganglia, Graphite, Console etc) so we just need to
 enable it, rather than writing custom components.
 As of now we have to hide it within MonitoredCounterGroup and its
 sub-classes, that's what I did for running it. It was running parallel
 to existing system. Let me work more on it and see if I can simplify
 the same.
 On Tue, Nov 11, 2014 at 4:50 AM, Hari Shreedharan
 hshreedha...@cloudera.com wrote:
 Is it easier to use than the current one and/or does it give better 
 performance? You’d need to support the current metrics API 
 (MonitoredCounterGroup, SourceCounter, SinkCounter, ChannelCounter etc).


 Thanks,
 Hari

 On Sat, Nov 8, 2014 at 8:21 PM, Ashish paliwalash...@gmail.com wrote:

 Hi,
 Have hacked a bit into our existing instrumentation package and piggy
 backed cohahale metrics package. Here is one sample for Spooled
 Directory source (with instrumentation only for Source and Channel ),
 using console reporter
 -- Gauges 
 --
 org.apache.flume.instrumentation.ChannelCounter.channel.current.size
  value = 200
 org.apache.flume.instrumentation.ChannelCounter.channel.fill.percentage
  value = 2.0
 org.apache.flume.instrumentation.SourceCounter.src.open-connection.count
  value = 0
 -- Counters 
 
 org.apache.flume.instrumentation.ChannelCounter.channel.event.put.attempt
  count = 1138800
 org.apache.flume.instrumentation.ChannelCounter.channel.event.put.success
  count = 1138800
 org.apache.flume.instrumentation.ChannelCounter.channel.event.take.attempt
  count = 1138601
 org.apache.flume.instrumentation.ChannelCounter.channel.event.take.success
  count = 1138600
 org.apache.flume.instrumentation.SourceCounter.src.events.accepted
  count = 1138800
 org.apache.flume.instrumentation.SourceCounter.src.events.received
  count = 1138800
 src.append-batch.accepted
  count = 11388
 src.append-batch.received
  count = 11388
 src.append.accepted
  count = 0
 src.append.received
  count = 0
 -- Meters 
 --
 eventAcceptedMeter
  count = 1138800
  mean rate = 106478.56 events/second
  1-minute rate = 93420.18 events/second
  5-minute rate = 91389.24 events/second
 15-minute rate = 91037.40 events/second
 eventReceivedMeter
  count = 1138800
  mean rate = 106462.14 events/second
  1-minute rate = 93420.18 events/second
  5-minute rate = 91389.24 events/second
 15-minute rate = 91037.40 events/second
 If there is interest in the community, can raise a jira and continue
 to work on it.
 --
 thanks
 ashish
 Blog: http://www.ashishpaliwal.com/blog
 My Photo Galleries: http://www.pbase.com/ashishpaliwal
 -- 
 thanks
 ashish
 Blog: http://www.ashishpaliwal.com/blog
 My Photo Galleries: http://www.pbase.com/ashishpaliwal

Re: Using CodaHale metrics for monitoring

2014-11-11 Thread Ashish
I was just playing around so piggy-backed it. The changes have to be
within MonitoredCounterGroup and its sub-classes, as its used across
the implementations. Let me raise a JIRA for this.

One thing that I would like to have in future is to allow
Sinks/Channels/Sources to be able to have custom counters/Meters and
be available to have those Metrics available via the framework. I
think CounterGroup was created for the same purpose.

Let me create a JIRA for this and work on it.

thanks
ashish

On Wed, Nov 12, 2014 at 1:11 AM, Hari Shreedharan
hshreedha...@cloudera.com wrote:
 I think it is best to replace the current system rather than having multiple 
 metrics systems. We should implement it such that it is transparent to user 
 code, and the changes only go into MonitoredCounterGroup (the CounterGroup 
 class is obsolete - dont bother).


 Thanks,
 Hari

 On Mon, Nov 10, 2014 at 5:36 PM, Ashish paliwalash...@gmail.com wrote:

 Codahale does have lot more features like the rate/sec one, which is
 most needed metrics. It provides a lot of reporters out of box
 (Nagios, HTTP, Ganglia, Graphite, Console etc) so we just need to
 enable it, rather than writing custom components.
 As of now we have to hide it within MonitoredCounterGroup and its
 sub-classes, that's what I did for running it. It was running parallel
 to existing system. Let me work more on it and see if I can simplify
 the same.
 On Tue, Nov 11, 2014 at 4:50 AM, Hari Shreedharan
 hshreedha...@cloudera.com wrote:
 Is it easier to use than the current one and/or does it give better 
 performance? You’d need to support the current metrics API 
 (MonitoredCounterGroup, SourceCounter, SinkCounter, ChannelCounter etc).


 Thanks,
 Hari

 On Sat, Nov 8, 2014 at 8:21 PM, Ashish paliwalash...@gmail.com wrote:

 Hi,
 Have hacked a bit into our existing instrumentation package and piggy
 backed cohahale metrics package. Here is one sample for Spooled
 Directory source (with instrumentation only for Source and Channel ),
 using console reporter
 -- Gauges 
 --
 org.apache.flume.instrumentation.ChannelCounter.channel.current.size
  value = 200
 org.apache.flume.instrumentation.ChannelCounter.channel.fill.percentage
  value = 2.0
 org.apache.flume.instrumentation.SourceCounter.src.open-connection.count
  value = 0
 -- Counters 
 
 org.apache.flume.instrumentation.ChannelCounter.channel.event.put.attempt
  count = 1138800
 org.apache.flume.instrumentation.ChannelCounter.channel.event.put.success
  count = 1138800
 org.apache.flume.instrumentation.ChannelCounter.channel.event.take.attempt
  count = 1138601
 org.apache.flume.instrumentation.ChannelCounter.channel.event.take.success
  count = 1138600
 org.apache.flume.instrumentation.SourceCounter.src.events.accepted
  count = 1138800
 org.apache.flume.instrumentation.SourceCounter.src.events.received
  count = 1138800
 src.append-batch.accepted
  count = 11388
 src.append-batch.received
  count = 11388
 src.append.accepted
  count = 0
 src.append.received
  count = 0
 -- Meters 
 --
 eventAcceptedMeter
  count = 1138800
  mean rate = 106478.56 events/second
  1-minute rate = 93420.18 events/second
  5-minute rate = 91389.24 events/second
 15-minute rate = 91037.40 events/second
 eventReceivedMeter
  count = 1138800
  mean rate = 106462.14 events/second
  1-minute rate = 93420.18 events/second
  5-minute rate = 91389.24 events/second
 15-minute rate = 91037.40 events/second
 If there is interest in the community, can raise a jira and continue
 to work on it.
 --
 thanks
 ashish
 Blog: http://www.ashishpaliwal.com/blog
 My Photo Galleries: http://www.pbase.com/ashishpaliwal
 --
 thanks
 ashish
 Blog: http://www.ashishpaliwal.com/blog
 My Photo Galleries: http://www.pbase.com/ashishpaliwal



-- 
thanks
ashish

Blog: http://www.ashishpaliwal.com/blog
My Photo Galleries: http://www.pbase.com/ashishpaliwal


Re: Using CodaHale metrics for monitoring

2014-11-11 Thread Hari Shreedharan
Currently the metrics exposed via Sources/sinks/channels are hardcoded. We 
could add a method to add custom metrics (I think it is already there in 
MonitoredCounterGroup, just not in the *Counter classes).


Thanks,
Hari

On Tue, Nov 11, 2014 at 6:34 PM, Ashish paliwalash...@gmail.com wrote:

 I was just playing around so piggy-backed it. The changes have to be
 within MonitoredCounterGroup and its sub-classes, as its used across
 the implementations. Let me raise a JIRA for this.
 One thing that I would like to have in future is to allow
 Sinks/Channels/Sources to be able to have custom counters/Meters and
 be available to have those Metrics available via the framework. I
 think CounterGroup was created for the same purpose.
 Let me create a JIRA for this and work on it.
 thanks
 ashish
 On Wed, Nov 12, 2014 at 1:11 AM, Hari Shreedharan
 hshreedha...@cloudera.com wrote:
 I think it is best to replace the current system rather than having multiple 
 metrics systems. We should implement it such that it is transparent to user 
 code, and the changes only go into MonitoredCounterGroup (the CounterGroup 
 class is obsolete - dont bother).


 Thanks,
 Hari

 On Mon, Nov 10, 2014 at 5:36 PM, Ashish paliwalash...@gmail.com wrote:

 Codahale does have lot more features like the rate/sec one, which is
 most needed metrics. It provides a lot of reporters out of box
 (Nagios, HTTP, Ganglia, Graphite, Console etc) so we just need to
 enable it, rather than writing custom components.
 As of now we have to hide it within MonitoredCounterGroup and its
 sub-classes, that's what I did for running it. It was running parallel
 to existing system. Let me work more on it and see if I can simplify
 the same.
 On Tue, Nov 11, 2014 at 4:50 AM, Hari Shreedharan
 hshreedha...@cloudera.com wrote:
 Is it easier to use than the current one and/or does it give better 
 performance? You’d need to support the current metrics API 
 (MonitoredCounterGroup, SourceCounter, SinkCounter, ChannelCounter etc).


 Thanks,
 Hari

 On Sat, Nov 8, 2014 at 8:21 PM, Ashish paliwalash...@gmail.com wrote:

 Hi,
 Have hacked a bit into our existing instrumentation package and piggy
 backed cohahale metrics package. Here is one sample for Spooled
 Directory source (with instrumentation only for Source and Channel ),
 using console reporter
 -- Gauges 
 --
 org.apache.flume.instrumentation.ChannelCounter.channel.current.size
  value = 200
 org.apache.flume.instrumentation.ChannelCounter.channel.fill.percentage
  value = 2.0
 org.apache.flume.instrumentation.SourceCounter.src.open-connection.count
  value = 0
 -- Counters 
 
 org.apache.flume.instrumentation.ChannelCounter.channel.event.put.attempt
  count = 1138800
 org.apache.flume.instrumentation.ChannelCounter.channel.event.put.success
  count = 1138800
 org.apache.flume.instrumentation.ChannelCounter.channel.event.take.attempt
  count = 1138601
 org.apache.flume.instrumentation.ChannelCounter.channel.event.take.success
  count = 1138600
 org.apache.flume.instrumentation.SourceCounter.src.events.accepted
  count = 1138800
 org.apache.flume.instrumentation.SourceCounter.src.events.received
  count = 1138800
 src.append-batch.accepted
  count = 11388
 src.append-batch.received
  count = 11388
 src.append.accepted
  count = 0
 src.append.received
  count = 0
 -- Meters 
 --
 eventAcceptedMeter
  count = 1138800
  mean rate = 106478.56 events/second
  1-minute rate = 93420.18 events/second
  5-minute rate = 91389.24 events/second
 15-minute rate = 91037.40 events/second
 eventReceivedMeter
  count = 1138800
  mean rate = 106462.14 events/second
  1-minute rate = 93420.18 events/second
  5-minute rate = 91389.24 events/second
 15-minute rate = 91037.40 events/second
 If there is interest in the community, can raise a jira and continue
 to work on it.
 --
 thanks
 ashish
 Blog: http://www.ashishpaliwal.com/blog
 My Photo Galleries: http://www.pbase.com/ashishpaliwal
 --
 thanks
 ashish
 Blog: http://www.ashishpaliwal.com/blog
 My Photo Galleries: http://www.pbase.com/ashishpaliwal
 -- 
 thanks
 ashish
 Blog: http://www.ashishpaliwal.com/blog
 My Photo Galleries: http://www.pbase.com/ashishpaliwal

Re: Using CodaHale metrics for monitoring

2014-11-11 Thread Ashish
That's what I had in mind, just need to figure out how to do it with
CodaHale. Want to avoid CodaHale MetricRegistry class floating around
in code. 1st step would be to migrate to CodaHale, I am yet to figure
out how managing the Reporter part. Let me try to put a patch
together.

On Wed, Nov 12, 2014 at 9:29 AM, Hari Shreedharan
hshreedha...@cloudera.com wrote:
 Currently the metrics exposed via Sources/sinks/channels are hardcoded. We 
 could add a method to add custom metrics (I think it is already there in 
 MonitoredCounterGroup, just not in the *Counter classes).


 Thanks,
 Hari

 On Tue, Nov 11, 2014 at 6:34 PM, Ashish paliwalash...@gmail.com wrote:

 I was just playing around so piggy-backed it. The changes have to be
 within MonitoredCounterGroup and its sub-classes, as its used across
 the implementations. Let me raise a JIRA for this.
 One thing that I would like to have in future is to allow
 Sinks/Channels/Sources to be able to have custom counters/Meters and
 be available to have those Metrics available via the framework. I
 think CounterGroup was created for the same purpose.
 Let me create a JIRA for this and work on it.
 thanks
 ashish
 On Wed, Nov 12, 2014 at 1:11 AM, Hari Shreedharan
 hshreedha...@cloudera.com wrote:
 I think it is best to replace the current system rather than having 
 multiple metrics systems. We should implement it such that it is 
 transparent to user code, and the changes only go into 
 MonitoredCounterGroup (the CounterGroup class is obsolete - dont bother).


 Thanks,
 Hari

 On Mon, Nov 10, 2014 at 5:36 PM, Ashish paliwalash...@gmail.com wrote:

 Codahale does have lot more features like the rate/sec one, which is
 most needed metrics. It provides a lot of reporters out of box
 (Nagios, HTTP, Ganglia, Graphite, Console etc) so we just need to
 enable it, rather than writing custom components.
 As of now we have to hide it within MonitoredCounterGroup and its
 sub-classes, that's what I did for running it. It was running parallel
 to existing system. Let me work more on it and see if I can simplify
 the same.
 On Tue, Nov 11, 2014 at 4:50 AM, Hari Shreedharan
 hshreedha...@cloudera.com wrote:
 Is it easier to use than the current one and/or does it give better 
 performance? You’d need to support the current metrics API 
 (MonitoredCounterGroup, SourceCounter, SinkCounter, ChannelCounter etc).


 Thanks,
 Hari

 On Sat, Nov 8, 2014 at 8:21 PM, Ashish paliwalash...@gmail.com wrote:

 Hi,
 Have hacked a bit into our existing instrumentation package and piggy
 backed cohahale metrics package. Here is one sample for Spooled
 Directory source (with instrumentation only for Source and Channel ),
 using console reporter
 -- Gauges 
 --
 org.apache.flume.instrumentation.ChannelCounter.channel.current.size
  value = 200
 org.apache.flume.instrumentation.ChannelCounter.channel.fill.percentage
  value = 2.0
 org.apache.flume.instrumentation.SourceCounter.src.open-connection.count
  value = 0
 -- Counters 
 
 org.apache.flume.instrumentation.ChannelCounter.channel.event.put.attempt
  count = 1138800
 org.apache.flume.instrumentation.ChannelCounter.channel.event.put.success
  count = 1138800
 org.apache.flume.instrumentation.ChannelCounter.channel.event.take.attempt
  count = 1138601
 org.apache.flume.instrumentation.ChannelCounter.channel.event.take.success
  count = 1138600
 org.apache.flume.instrumentation.SourceCounter.src.events.accepted
  count = 1138800
 org.apache.flume.instrumentation.SourceCounter.src.events.received
  count = 1138800
 src.append-batch.accepted
  count = 11388
 src.append-batch.received
  count = 11388
 src.append.accepted
  count = 0
 src.append.received
  count = 0
 -- Meters 
 --
 eventAcceptedMeter
  count = 1138800
  mean rate = 106478.56 events/second
  1-minute rate = 93420.18 events/second
  5-minute rate = 91389.24 events/second
 15-minute rate = 91037.40 events/second
 eventReceivedMeter
  count = 1138800
  mean rate = 106462.14 events/second
  1-minute rate = 93420.18 events/second
  5-minute rate = 91389.24 events/second
 15-minute rate = 91037.40 events/second
 If there is interest in the community, can raise a jira and continue
 to work on it.
 --
 thanks
 ashish
 Blog: http://www.ashishpaliwal.com/blog
 My Photo Galleries: http://www.pbase.com/ashishpaliwal
 --
 thanks
 ashish
 Blog: http://www.ashishpaliwal.com/blog
 My Photo Galleries: http://www.pbase.com/ashishpaliwal
 --
 thanks
 ashish
 Blog: http://www.ashishpaliwal.com/blog
 My Photo Galleries: http://www.pbase.com/ashishpaliwal



-- 
thanks
ashish

Blog: 

Re: Using CodaHale metrics for monitoring

2014-11-11 Thread Hari Shreedharan
Makes sense.


Thanks,
Hari

On Tue, Nov 11, 2014 at 9:51 PM, Ashish paliwalash...@gmail.com wrote:

 That's what I had in mind, just need to figure out how to do it with
 CodaHale. Want to avoid CodaHale MetricRegistry class floating around
 in code. 1st step would be to migrate to CodaHale, I am yet to figure
 out how managing the Reporter part. Let me try to put a patch
 together.
 On Wed, Nov 12, 2014 at 9:29 AM, Hari Shreedharan
 hshreedha...@cloudera.com wrote:
 Currently the metrics exposed via Sources/sinks/channels are hardcoded. We 
 could add a method to add custom metrics (I think it is already there in 
 MonitoredCounterGroup, just not in the *Counter classes).


 Thanks,
 Hari

 On Tue, Nov 11, 2014 at 6:34 PM, Ashish paliwalash...@gmail.com wrote:

 I was just playing around so piggy-backed it. The changes have to be
 within MonitoredCounterGroup and its sub-classes, as its used across
 the implementations. Let me raise a JIRA for this.
 One thing that I would like to have in future is to allow
 Sinks/Channels/Sources to be able to have custom counters/Meters and
 be available to have those Metrics available via the framework. I
 think CounterGroup was created for the same purpose.
 Let me create a JIRA for this and work on it.
 thanks
 ashish
 On Wed, Nov 12, 2014 at 1:11 AM, Hari Shreedharan
 hshreedha...@cloudera.com wrote:
 I think it is best to replace the current system rather than having 
 multiple metrics systems. We should implement it such that it is 
 transparent to user code, and the changes only go into 
 MonitoredCounterGroup (the CounterGroup class is obsolete - dont bother).


 Thanks,
 Hari

 On Mon, Nov 10, 2014 at 5:36 PM, Ashish paliwalash...@gmail.com wrote:

 Codahale does have lot more features like the rate/sec one, which is
 most needed metrics. It provides a lot of reporters out of box
 (Nagios, HTTP, Ganglia, Graphite, Console etc) so we just need to
 enable it, rather than writing custom components.
 As of now we have to hide it within MonitoredCounterGroup and its
 sub-classes, that's what I did for running it. It was running parallel
 to existing system. Let me work more on it and see if I can simplify
 the same.
 On Tue, Nov 11, 2014 at 4:50 AM, Hari Shreedharan
 hshreedha...@cloudera.com wrote:
 Is it easier to use than the current one and/or does it give better 
 performance? You’d need to support the current metrics API 
 (MonitoredCounterGroup, SourceCounter, SinkCounter, ChannelCounter etc).


 Thanks,
 Hari

 On Sat, Nov 8, 2014 at 8:21 PM, Ashish paliwalash...@gmail.com wrote:

 Hi,
 Have hacked a bit into our existing instrumentation package and piggy
 backed cohahale metrics package. Here is one sample for Spooled
 Directory source (with instrumentation only for Source and Channel ),
 using console reporter
 -- Gauges 
 --
 org.apache.flume.instrumentation.ChannelCounter.channel.current.size
  value = 200
 org.apache.flume.instrumentation.ChannelCounter.channel.fill.percentage
  value = 2.0
 org.apache.flume.instrumentation.SourceCounter.src.open-connection.count
  value = 0
 -- Counters 
 
 org.apache.flume.instrumentation.ChannelCounter.channel.event.put.attempt
  count = 1138800
 org.apache.flume.instrumentation.ChannelCounter.channel.event.put.success
  count = 1138800
 org.apache.flume.instrumentation.ChannelCounter.channel.event.take.attempt
  count = 1138601
 org.apache.flume.instrumentation.ChannelCounter.channel.event.take.success
  count = 1138600
 org.apache.flume.instrumentation.SourceCounter.src.events.accepted
  count = 1138800
 org.apache.flume.instrumentation.SourceCounter.src.events.received
  count = 1138800
 src.append-batch.accepted
  count = 11388
 src.append-batch.received
  count = 11388
 src.append.accepted
  count = 0
 src.append.received
  count = 0
 -- Meters 
 --
 eventAcceptedMeter
  count = 1138800
  mean rate = 106478.56 events/second
  1-minute rate = 93420.18 events/second
  5-minute rate = 91389.24 events/second
 15-minute rate = 91037.40 events/second
 eventReceivedMeter
  count = 1138800
  mean rate = 106462.14 events/second
  1-minute rate = 93420.18 events/second
  5-minute rate = 91389.24 events/second
 15-minute rate = 91037.40 events/second
 If there is interest in the community, can raise a jira and continue
 to work on it.
 --
 thanks
 ashish
 Blog: http://www.ashishpaliwal.com/blog
 My Photo Galleries: http://www.pbase.com/ashishpaliwal
 --
 thanks
 ashish
 Blog: http://www.ashishpaliwal.com/blog
 My Photo Galleries: http://www.pbase.com/ashishpaliwal
 --
 thanks
 ashish
 Blog: 

Re: Using CodaHale metrics for monitoring

2014-11-10 Thread Hari Shreedharan
Is it easier to use than the current one and/or does it give better 
performance? You’d need to support the current metrics API 
(MonitoredCounterGroup, SourceCounter, SinkCounter, ChannelCounter etc).


Thanks,
Hari

On Sat, Nov 8, 2014 at 8:21 PM, Ashish paliwalash...@gmail.com wrote:

 Hi,
 Have hacked a bit into our existing instrumentation package and piggy
 backed cohahale metrics package. Here is one sample for Spooled
 Directory source (with instrumentation only for Source and Channel ),
 using console reporter
 -- Gauges 
 --
 org.apache.flume.instrumentation.ChannelCounter.channel.current.size
  value = 200
 org.apache.flume.instrumentation.ChannelCounter.channel.fill.percentage
  value = 2.0
 org.apache.flume.instrumentation.SourceCounter.src.open-connection.count
  value = 0
 -- Counters 
 
 org.apache.flume.instrumentation.ChannelCounter.channel.event.put.attempt
  count = 1138800
 org.apache.flume.instrumentation.ChannelCounter.channel.event.put.success
  count = 1138800
 org.apache.flume.instrumentation.ChannelCounter.channel.event.take.attempt
  count = 1138601
 org.apache.flume.instrumentation.ChannelCounter.channel.event.take.success
  count = 1138600
 org.apache.flume.instrumentation.SourceCounter.src.events.accepted
  count = 1138800
 org.apache.flume.instrumentation.SourceCounter.src.events.received
  count = 1138800
 src.append-batch.accepted
  count = 11388
 src.append-batch.received
  count = 11388
 src.append.accepted
  count = 0
 src.append.received
  count = 0
 -- Meters 
 --
 eventAcceptedMeter
  count = 1138800
  mean rate = 106478.56 events/second
  1-minute rate = 93420.18 events/second
  5-minute rate = 91389.24 events/second
 15-minute rate = 91037.40 events/second
 eventReceivedMeter
  count = 1138800
  mean rate = 106462.14 events/second
  1-minute rate = 93420.18 events/second
  5-minute rate = 91389.24 events/second
 15-minute rate = 91037.40 events/second
 If there is interest in the community, can raise a jira and continue
 to work on it.
 -- 
 thanks
 ashish
 Blog: http://www.ashishpaliwal.com/blog
 My Photo Galleries: http://www.pbase.com/ashishpaliwal

Re: Using CodaHale metrics for monitoring

2014-11-10 Thread Ashish
Codahale does have lot more features like the rate/sec one, which is
most needed metrics. It provides a lot of reporters out of box
(Nagios, HTTP, Ganglia, Graphite, Console etc) so we just need to
enable it, rather than writing custom components.

As of now we have to hide it within MonitoredCounterGroup and its
sub-classes, that's what I did for running it. It was running parallel
to existing system. Let me work more on it and see if I can simplify
the same.

On Tue, Nov 11, 2014 at 4:50 AM, Hari Shreedharan
hshreedha...@cloudera.com wrote:
 Is it easier to use than the current one and/or does it give better 
 performance? You’d need to support the current metrics API 
 (MonitoredCounterGroup, SourceCounter, SinkCounter, ChannelCounter etc).


 Thanks,
 Hari

 On Sat, Nov 8, 2014 at 8:21 PM, Ashish paliwalash...@gmail.com wrote:

 Hi,
 Have hacked a bit into our existing instrumentation package and piggy
 backed cohahale metrics package. Here is one sample for Spooled
 Directory source (with instrumentation only for Source and Channel ),
 using console reporter
 -- Gauges 
 --
 org.apache.flume.instrumentation.ChannelCounter.channel.current.size
  value = 200
 org.apache.flume.instrumentation.ChannelCounter.channel.fill.percentage
  value = 2.0
 org.apache.flume.instrumentation.SourceCounter.src.open-connection.count
  value = 0
 -- Counters 
 
 org.apache.flume.instrumentation.ChannelCounter.channel.event.put.attempt
  count = 1138800
 org.apache.flume.instrumentation.ChannelCounter.channel.event.put.success
  count = 1138800
 org.apache.flume.instrumentation.ChannelCounter.channel.event.take.attempt
  count = 1138601
 org.apache.flume.instrumentation.ChannelCounter.channel.event.take.success
  count = 1138600
 org.apache.flume.instrumentation.SourceCounter.src.events.accepted
  count = 1138800
 org.apache.flume.instrumentation.SourceCounter.src.events.received
  count = 1138800
 src.append-batch.accepted
  count = 11388
 src.append-batch.received
  count = 11388
 src.append.accepted
  count = 0
 src.append.received
  count = 0
 -- Meters 
 --
 eventAcceptedMeter
  count = 1138800
  mean rate = 106478.56 events/second
  1-minute rate = 93420.18 events/second
  5-minute rate = 91389.24 events/second
 15-minute rate = 91037.40 events/second
 eventReceivedMeter
  count = 1138800
  mean rate = 106462.14 events/second
  1-minute rate = 93420.18 events/second
  5-minute rate = 91389.24 events/second
 15-minute rate = 91037.40 events/second
 If there is interest in the community, can raise a jira and continue
 to work on it.
 --
 thanks
 ashish
 Blog: http://www.ashishpaliwal.com/blog
 My Photo Galleries: http://www.pbase.com/ashishpaliwal



-- 
thanks
ashish

Blog: http://www.ashishpaliwal.com/blog
My Photo Galleries: http://www.pbase.com/ashishpaliwal


Using CodaHale metrics for monitoring

2014-11-08 Thread Ashish
Hi,

Have hacked a bit into our existing instrumentation package and piggy
backed cohahale metrics package. Here is one sample for Spooled
Directory source (with instrumentation only for Source and Channel ),
using console reporter

-- Gauges --

org.apache.flume.instrumentation.ChannelCounter.channel.current.size

 value = 200

org.apache.flume.instrumentation.ChannelCounter.channel.fill.percentage

 value = 2.0

org.apache.flume.instrumentation.SourceCounter.src.open-connection.count

 value = 0


-- Counters 

org.apache.flume.instrumentation.ChannelCounter.channel.event.put.attempt

 count = 1138800

org.apache.flume.instrumentation.ChannelCounter.channel.event.put.success

 count = 1138800

org.apache.flume.instrumentation.ChannelCounter.channel.event.take.attempt

 count = 1138601

org.apache.flume.instrumentation.ChannelCounter.channel.event.take.success

 count = 1138600

org.apache.flume.instrumentation.SourceCounter.src.events.accepted

 count = 1138800

org.apache.flume.instrumentation.SourceCounter.src.events.received

 count = 1138800

src.append-batch.accepted

 count = 11388

src.append-batch.received

 count = 11388

src.append.accepted

 count = 0

src.append.received

 count = 0


-- Meters --

eventAcceptedMeter

 count = 1138800

 mean rate = 106478.56 events/second

 1-minute rate = 93420.18 events/second

 5-minute rate = 91389.24 events/second

15-minute rate = 91037.40 events/second

eventReceivedMeter

 count = 1138800

 mean rate = 106462.14 events/second

 1-minute rate = 93420.18 events/second

 5-minute rate = 91389.24 events/second

15-minute rate = 91037.40 events/second


If there is interest in the community, can raise a jira and continue
to work on it.


-- 
thanks
ashish

Blog: http://www.ashishpaliwal.com/blog
My Photo Galleries: http://www.pbase.com/ashishpaliwal