@Devaraj K *I tried explicitly calling incomingJob.initTasks()*and it gave me a pleasant INFO org.apache.hadoop.mapred.JobInProgress: Job job_201307111248_0002 initialized successfully with 1 map tasks and 1 reduce tasks
*Yet the **Cannot create task split for <my-job-id>** is still there*. About errors relating to the JobID, is this significant? --- INFO org.apache.hadoop.mapred.JobInProgress: job_201307111248_0002: nMaps=1 nReduces=1 max=-1 *P.S: In the log file I shared, anything with "sreejith == " were logged by me. Not system-generated* On Thu, Jul 11, 2013 at 1:18 PM, Sreejith Ramakrishnan < [email protected]> wrote: > I've attached the relevant portion of the log. > > For easy testing, *I've attached the maven project* for the > EDFJobScheduler. Just make a JAR from it and put it in $HADOOP_HOME/lib and > add this property to *mapred-site.xml*: > > *<property> > * > * <name>mapred.jobtracker.taskScheduler</name> > <value>org.apache.hadoop.mapred.EDFJobScheduler</value> > * > *</property>* > > > On Thu, Jul 11, 2013 at 12:19 PM, Sreejith Ramakrishnan < > [email protected]> wrote: > >> I'm making my own scheduler. Since I had errors, I rewrote the scheduler >> to do just a simple thing. When I give it a job, it allocates 1 map() and 1 >> reduce() to it. But that isn't happening. I used a barebones WordCount pgm. >> I'm stuck at map: 0% reduce 0%. And in the jobtracker logs, I see: >> >> INFO org.apache.hadoop.mapred.JobInProgress: Cannot create task split for >> <my-job-id> >> >> DETAILED INFO: >> >> Here's some snippets of code. The scheduler logic is inside the >> assignTasks(): >> >> We get a JobInProgressListener in the constructor- >> this.jobQueueJobInProgressListener = new JobQueueJobInProgressListener(); >> >> In start() - >> >> this.taskTrackerManager.addJobInProgressListener(jobQueueJobInProgressListener); >> >> Then I just take the last element in that queue (I assume its the latest >> submitted job) and assign to incomingJob (of type JobInProgress). >> >> Then I obtain a new map task as follows and append this task to a list >> which is the return value of the assignTasks(): >> Task createdMap = incomingJob.obtainNewMapTask(taskTrackerStatus, >> numTaskTrackers, taskTrackerManager.getNumberOfUniqueHosts()); >> >> >> I've attached the scheduler. Can any expert point out if there's any >> ignorant mistakes from my side? >> > >
