There are plenty of log aggregation tools both open source and commercial
off the shelf. Here's some
http://devopsangle.com/2012/04/19/8-splunk-alternatives/
My personal recommendation is LogStash.
On Thu, Oct 10, 2013 at 10:38 PM, Raymond Tay raymondtay1...@gmail.comwrote:
You can try Chukwa
I've used Splunk in the past for log aggregation. It's commercial/proprietary,
but I think there's a free version.
http://www.splunk.com/
From: Raymond Tay [mailto:raymondtay1...@gmail.com]
Sent: Friday, October 11, 2013 1:39 AM
To: user@hadoop.apache.org
Subject: Re: State of Art in Hadoop Log
Hi,
http://flume.apache.org
- Alex
On Oct 11, 2013, at 7:36 AM, Sagar Mehta sagarme...@gmail.com wrote:
Hi Guys,
We have fairly decent sized Hadoop cluster of about 200 nodes and was
wondering what is the state of art if I want to aggregate and visualize
Hadoop ecosystem logs,
Sagar,
It sounds like you want a management console. We are using Cloudera
Manager, but for 200 nodes you would need to license it, it is only free up
to 50 nodes.
The FOSS version of this is Ambari, iirc.
http://incubator.apache.org/ambari/
Flume will provide a Hadoop-integrated pipeline for
Just a clarification: Cloudera Manager is now free for any number of nodes.
Ref:
http://www.cloudera.com/content/cloudera/en/products/cloudera-manager.html
-Sandy
On Fri, Oct 11, 2013 at 7:05 AM, DSuiter RDX dsui...@rdx.com wrote:
Sagar,
It sounds like you want a management console. We are
Hi Guys,
We have fairly decent sized Hadoop cluster of about 200 nodes and was
wondering what is the state of art if I want to aggregate and visualize
Hadoop ecosystem logs, particularly
1. Tasktracker logs
2. Datanode logs
3. Hbase RegionServer logs
One way is to use something like a
You can try Chukwa which is part of the incubating projects under Apache. Tried
it before and liked it for aggregating logs.
On 11 Oct, 2013, at 1:36 PM, Sagar Mehta sagarme...@gmail.com wrote:
Hi Guys,
We have fairly decent sized Hadoop cluster of about 200 nodes and was
wondering what
