Hi,
How can I write custom serializer to write event body and header to HDFS now I
am getting only log messages which are written on HDFS. Timestamp and other
information is not coming.
Regards,
Chhaya Vishwakarma
The contents of this e-mail and any
I think you can implement your custome sink, where you can take event body
and header if any to HDFS...
Regardsm
Som
On Fri, Jan 11, 2013 at 2:53 PM, Chhaya Vishwakarma
chhaya.vishwaka...@lntinfotech.com wrote:
Hi,
** **
How can I write custom serializer to write event body and
Thanks Jeff,
Clear and detailed explanations. These deserve to be on the wiki, as these
parameters have direct implications on the performance of flume nodes.
thanks
Bhaskar
On Tue, Jan 8, 2013 at 9:40 PM, Jeff Lord jl...@cloudera.com wrote:
Hi Bashkar,
1) Batch Size
1.a) When configured
There was recently added a HeaderAndText serializer to the Flume Core,
which is available in the latest Git snapshot (git clone
http://github.com/apache/flume). You will need to follow directions in
the developer
documentationhttp://flume.apache.org/FlumeDeveloperGuide.html#compile-test-flumeto
Hi folks,
I just posted to the Apache blog on how to do performance tuning with
Flume. I plan on following it up with a post about using the Flume
monitoring capabilities while tuning. Feedback is welcome.
https://blogs.apache.org/flume/entry/flume_performance_tuning_part_1
Regards,
Mike
Nice post!
On Fri, Jan 11, 2013 at 12:13 PM, Mike Percy mpe...@apache.org wrote:
Hi folks,
I just posted to the Apache blog on how to do performance tuning with Flume.
I plan on following it up with a post about using the Flume monitoring
capabilities while tuning. Feedback is welcome.
Thanks Brock! I've been working on this, off and on, for a while. :)
On Fri, Jan 11, 2013 at 12:18 PM, Brock Noland br...@cloudera.com wrote:
Nice post!
On Fri, Jan 11, 2013 at 12:13 PM, Mike Percy mpe...@apache.org wrote:
Hi folks,
I just posted to the Apache blog on how to do
Great post, Mike!
One question if you can either address via mailing list or future posts...
I am curious about how to remove duplicated messages in this flow. For
example, when I set up a switch/router to send syslog messages, I'd
like to send two syslog collectors or two flume agents. In this
Hi Simon,
There is no good way that I am aware of for Flume to dedup messages. This
is because there is no abstraction for doing pairwise comparison of events,
and, as you scale up, maintaining some kind of hash table of processed
events generally becomes prohibitive or makes it not worth the
