Thanks for your response Rasit. You may have missed a portion of my post. > On a different note, when I attempt to pass params via -D I get a usage message; when I use > -jobconf the command goes through (and works in the case of mapred.reduce.tasks=0 for > example) but I get a deprecation warning).
I'm using Hadoop 0.19.0 and -D is not working. Are you using version 0.19.0 as well? John On Wed, Feb 18, 2009 at 9:14 AM, Rasit OZDAS <rasitoz...@gmail.com> wrote: > John, did you try -D option instead of -jobconf, > > I had -D option in my code, I changed it with -jobconf, this is what I get: > > ... > ... > Options: > -input <path> DFS input file(s) for the Map step > -output <path> DFS output directory for the Reduce step > -mapper <cmd|JavaClassName> The streaming command to run > -combiner <JavaClassName> Combiner has to be a Java class > -reducer <cmd|JavaClassName> The streaming command to run > -file <file> File/dir to be shipped in the Job jar file > -inputformat > TextInputFormat(default)|SequenceFileAsTextInputFormat|JavaClassName > Optional. > -outputformat TextOutputFormat(default)|JavaClassName Optional. > -partitioner JavaClassName Optional. > -numReduceTasks <num> Optional. > -inputreader <spec> Optional. > -cmdenv <n>=<v> Optional. Pass env.var to streaming commands > -mapdebug <path> Optional. To run this script when a map task fails > -reducedebug <path> Optional. To run this script when a reduce task fails > > -verbose > > Generic options supported are > -conf <configuration file> specify an application configuration file > -D <property=value> use value for given property > -fs <local|namenode:port> specify a namenode > -jt <local|jobtracker:port> specify a job tracker > -files <comma separated list of files> specify comma separated files to > be copied to the map reduce cluster > -libjars <comma separated list of jars> specify comma separated jar > files > to include in the classpath. > -archives <comma separated list of archives> specify comma separated > archives to be unarchived on the compute machines. > > The general command line syntax is > bin/hadoop command [genericOptions] [commandOptions] > > For more details about these options: > Use $HADOOP_HOME/bin/hadoop jar build/hadoop-streaming.jar -info > > > > I think -jobconf is not used in v.0.19 . > > 2009/2/18 S D <sd.codewarr...@gmail.com> > > > I'm having trouble overriding the maximum number of map tasks that run on > a > > given machine in my cluster. The default value of > > mapred.tasktracker.map.tasks.maximum is set to 2 in hadoop-default.xml. > > When > > running my job I passed > > > > -jobconf mapred.tasktracker.map.tasks.maximum=1 > > > > to limit map tasks to one per machine but each machine was still > allocated > > 2 > > map tasks (simultaneously). The only way I was able to guarantee a > maximum > > of one map task per machine was to change the value of the property in > > hadoop-site.xml. This is unsatisfactory since I'll often be changing the > > maximum on a per job basis. Any hints? > > > > On a different note, when I attempt to pass params via -D I get a usage > > message; when I use -jobconf the command goes through (and works in the > > case > > of mapred.reduce.tasks=0 for example) but I get a deprecation warning). > > > > Thanks, > > John > > > > > > -- > M. Raşit ÖZDAŞ >
