State of Art in Hadoop Log aggregation

2013-10-10 Thread Sagar Mehta
Hi Guys, We have fairly decent sized Hadoop cluster of about 200 nodes and was wondering what is the state of art if I want to aggregate and visualize Hadoop ecosystem logs, particularly 1. Tasktracker logs 2. Datanode logs 3. Hbase RegionServer logs One way is to use something like a

Re: Automatically mapping a job submitted by a particular user to a specific hadoop map-reduce queue

2013-04-26 Thread Sagar Mehta
at 6:22 PM, Sagar Mehta sagarme...@gmail.com wrote: Hi Guys, We have a general purpose Hive cluster [about 200 nodes] which is used for various jobs like - Production - Experimental/Research - Adhoc queries We are using the fair-share scheduler to schedule them and for this we

Re: Automatically mapping a job submitted by a particular user to a specific hadoop map-reduce queue

2013-04-26 Thread Sagar Mehta
), but a JIRA has been filed recently that addresses a similar need. Would https://issues.apache.org/jira/browse/MAPREDUCE-5132 work for what you're trying to do? If not, would you mind filing a new JIRA for the functionality you'd want? -Sandy On Wed, Apr 24, 2013 at 6:22 PM, Sagar Mehta sagarme

Re: Automatically mapping a job submitted by a particular user to a specific hadoop map-reduce queue

2013-04-26 Thread Sagar Mehta
Vavilapalli Hortonworks Inc. http://hortonworks.com/ On Apr 24, 2013, at 6:22 PM, Sagar Mehta wrote: Hi Guys, We have a general purpose Hive cluster [about 200 nodes] which is used for various jobs like - Production - Experimental/Research - Adhoc queries We are using the fair

Automatically mapping a job submitted by a particular user to a specific hadoop map-reduce queue

2013-04-24 Thread Sagar Mehta
Hi Guys, We have a general purpose Hive cluster [about 200 nodes] which is used for various jobs like - Production - Experimental/Research - Adhoc queries We are using the fair-share scheduler to schedule them and for this we have corresponding 3 pools in the scheduler. *Here is what