hi, Thanks for the reply.
The problem i want to solve is, I am writing a scheduler plugin in which, i want to divide a job into tasks and assign to TaskTrackers(Obvious). Then, i want to assign the tasks on to which ever TaskTracker i want and assign it whenever i want. I am using org.apache.hadoop.mapred.TaskScheduler.assignTasks(TaskTrackerStatus taskTrackerStatus) which would return list of Tasks, on every Heart Beat. In the above method, i am calling org.apache.hadoop.mapred.JobInProgress.obtainNewMapTask method to get Task for a job. But everytime it is called, i would only get a null as return. Initially i thought that JobStatus.status is not set to RUNNING. But the problem is not that. Is there any other way that i could assign tasks on to a TaskTracker apart from above method. hoping a solution from you. Thank you Radheshyam Nanduri On Fri, Sep 10, 2010 at 6:46 PM, Hemanth Yamijala <yhema...@gmail.com>wrote: > [Moving to mapreduce-dev, copying common-dev] > > Hi, > > On Thu, Sep 9, 2010 at 11:30 AM, radheshyam nanduri > <radheshyam.nand...@gmail.com> wrote: > > Hi, > > > > I am working on writing a scheduler plugin for Hadoop. > > Currently, the model supported to plug-in schedulers to Hadoop is to > extend the TaskScheduler class in o.a.h.mapred package. Primarily what > a 'plug-in' scheduler can do is that given a set of jobs and a > tasktracker, it can assign one or more suitable tasks to the > tasktracker. The scheduler will have flexibility in choosing the job > and the tasks it wants to schedule. You can take a look at some of the > existing schedulers like CapacityTaskScheduler or FairScheduler to see > what they do and how. > > > I have divided the job received into number of tasks. > > This is already done in the framework when a job is submitted. Are you > overriding this ? Can you explain what you are doing in some more > detail ? > > > My task now is to assign a task on to a particular TaskTracker. > > I want to start the Task right away with a method which accepts the Task > and > > TaskTracker as arguments. > > I am not sure I am following this. A task needs to be sent via > Hadoop's RPC mechanisms to a tasktracker where it should be executed. > So, conceptually, it is the tasktracker that has an RPC method which > accepts tasks to launch. The task is launched typically straight-away, > but in case of certain scheduling choices, it could have to wait for a > short while to get a free slot to execute. > > > Could you guide me on doing this. > > It may be better if you can describe what you want the plug-in > scheduler to achieve. > > Thanks > Hemanth > > > > > > > Thanks in advance. > > > > > > -- > > Radheshyam Nanduri > > > -- Radheshyam Nanduri
