Hi Himanshu,
To enable ganglia monitoring for flume agent, you need to modify flume-en.sh
file. Add the following in JAVA_OPTS, where monitoring host is headnode of
ganglia. After configuring, restart flume agent. You can have the graphs at
ganglia web ui for this host.
JAVA_OPTS= -Dflume.moni
Hi,
I have a multi agent flume flow running.
I was interested in monitoring the channels, so I enabled the JSON reporting.
Now I want to view these metrics in Graphs.
Can anyone help me with this?
This is a sample screenshot of the graphs I picked up somewhere from the web. I
am interested i
Yes the file channel is designed to handle this and is what you should be
using.
You are also on the right track regarding sizing your file channel to
account for the number of events that could accumulate in the event that
your terminal sink is unable to complete transactions. With the amount of
d
Ok, I just realized that I am missing a 0 on the rollSize, and it is
probably doing exactly what it is supposed to since I told it close the
file at 3 KB not 3 MB...
Sorry everyone!
Thanks!
*Devin Suiter*
Jr. Data Solutions Software Engineer
100 Sandusky Street | 2nd Floor | Pittsburgh, PA 15212
Hi, this may be a problem with our understanding, or my configuration.
I am trying to take data from rsyslog via remote forwarding over TCP into a
syslogTCP source, collect it as an avro sink, connect the avro sink to an
avro source, and then into an HDFS sink.
Everything is connected and the dat
Thanks for the explanation Hari.
Unfortunately our flume boxes are virtualized so multi disk file channel is
not an option for us.
However we can bump up the memory from current 16GB to say 32GB, and our
/tmp are tmpfs, so wen can use file channel with channel directories under
/tmp.
That way we
I am using a AMQP Souce, so I don't know how changing to a JMS source would
have any difference.
I am concerned about the volume of data and the file channel. Even if I
switched to JMS, my question would be the same.
On Fri, Oct 4, 2013 at 4:46 PM, Hari Shreedharan
wrote:
> Have you tried the
Hi Hari,
Finally I got it working.I am not sure what was missing in my earlier
configuration. My configuration might not have used by the instance agent
because of the way CDH uses inheritances on agent configuration. Thanks for
all your help.
tier1.sinks.sink1.type = hdfs
tier1.sinks.sink1.hdf
