Re: job priorities

2010-12-12 Thread Sriram Ramachandrasekaran
Nitin, You should read about the queues concept in hadoop job scheduler. Each scheduler works with a certain type of Queue. The default scheduler uses a FIFO queue. However, hadoop is quite flexible, in that, you can plug-in any scheduler that's been contributed. Right now, capacity scheduler & fai

Re: job priorities

2010-12-12 Thread Harsh J
Hi, On Sun, Dec 12, 2010 at 11:21 PM, nitin reddy wrote: > Hi > > I want to induce dynamic priority to the jobs being submitted like > rather than the default FIFO i want to set a variable that decides the > priority of it a situation may be even the job may be first in the > queue it might have

job priorities

2010-12-12 Thread nitin reddy
Hi I want to induce dynamic priority to the jobs being submitted like rather than the default FIFO i want to set a variable that decides the priority of it a situation may be even the job may be first in the queue it might have less priority than the job that is behind in the queue . just wanted t

Re: task statistics

2010-12-12 Thread rahul patodi
Hi nitin, you can get all the information about task (submitted, running, failed etc) , jobs on the web interface given by hadoop for jobtrackerhttp://IP-ADDRESS:50030/ for tasktracker http://IP-ADDRESS:50060/ -- *Regards*, Rahul Patodi Associate Software Engineer, Impetus Infotech (

Re: task tracker statistics

2010-12-12 Thread rahul patodi
Hi nitin, you can get following information from web based interface provided by hadoop *Jobtracker* http://IP-ADDRESS:50030/jobtracker.jsp: 1. cluster summary 2.. scheduling information 3. running jobs 4. completed jobs 5. retired jobs 6. local logs *taskt

Increasing MR job execution time.

2010-12-12 Thread Sriram Ramachandrasekaran
Hi, I have been using Hadoop's MapReduce only for the past few months. I use it for data mining purposes. I use a very small cluster, 4 nodes. 1- Name node, 3 - Datanodes and the Job tracker runs on the Name node itself. My requirements are to process ~10-12G of compressed data, which amounts to 40