Hi Hadoop Users, I am wondering if it is possible to use any other schedulers like SLURM/PBS Pro as default schedulers to scheduler MR Jobs
Regards Dhvanika -----Original Message----- From: Harsh J [mailto:ha...@cloudera.com] Sent: Sunday, April 13, 2014 10:54 PM To: <user@hadoop.apache.org> Cc: ektacloudst...@gmail.com; common-u...@hadoop.apache.org; hdfs-u...@hadoop.apache.org Subject: Re: Changing default scheduler in hadoop Hi, On Sun, Apr 13, 2014 at 10:47 AM, Mahesh Khandewal <mahesh.k....@gmail.com> wrote: > Sir i am using Hadoop 1.1.2 > I don't know where is the code residing of default scheduler? Doing a simple 'find' in the source checkout for name pattern 'Scheduler' should reveal pretty relevant hits. We do name our Java classes seriously :) https://github.com/apache/hadoop-common/blob/release-1.1.2/src/mapred/org/apache/hadoop/mapred/JobQueueTaskScheduler.java > I want to change the default scheduler to fair how can i do this?? You can override the mapred-site.xml placed property 'mapred.jobtracker.taskScheduler' to specify a custom scheduler (or a supplied one, such as Fair [http://hadoop.apache.org/docs/r1.1.2/fair_scheduler.html] or Capacity [http://hadoop.apache.org/docs/r1.1.2/capacity_scheduler.html] Schedulers). > And if i want to get back to default scheduler how can i do this? Remove the configuration override, and it will always go back to the default FIFO based scheduler, the same whose source has been linked above. > I am struggling since 4 months to get help on Apache Hadoop?? Are you unsure about this? -- Harsh J The information contained in this electronic message and any attachments to this message are intended for the exclusive use of the addressee(s) and may contain proprietary, confidential or privileged information. If you are not the intended recipient, you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately and destroy all copies of this message and any attachments. WARNING: Computer viruses can be transmitted via email. The recipient should check this email and any attachments for the presence of viruses. The company accepts no liability for any damage caused by any virus transmitted by this email. www.wipro.com
