On Wed, May 20, 2009 at 3:39 AM, Chuck Lam <chuck....@gmail.com> wrote: > Can you set the number of reducers to zero and see if it becomes a map only > job? If it does, then it's able to read in the mapred.reduce.tasks property > correctly but just refuse to have 2 reducers. In that case, it's most likely > you're running in local mode, which doesn't allow more than 1 reducer.
As I have already mentioned in my original mail, I am not running it in local mode. Quoting from my original mail: "My configuration file is set as follows: mapred.map.tasks = 2 mapred.reduce.tasks = 2 However, the description of these properties mention that these settings would be ignored if mapred.job.tracker is set as 'local'. Mine is set properly with IP address, port number." > > If setting zero doesn't change anything, then your config file is not being > read, or it's being overridden. > > As an aside, if you use ToolRunner in your Hadoop program, then it will > support generic options such that you can run your program with the option > -D mapred.reduce.tasks=2 > to tell it to use 2 reducers. This allows you to set the number of reducers > on a per-job basis. > > I understand that it is being overridden by something else. What I want to know is which file is overriding it. Also, please note that I have these settings only in the conf/hadoop-site.xml of job tracker node. Is that enough?
