On 2/18/10 11:24 AM, Rob Stewart wrote:
Hi Eric, thanks.
It appears not:
----------------
JobConf jobConf = new JobConf(getConf(), Sort.class);
jobConf.setJobName("join");
jobConf.setMapperClass(IdentityMapper.class);
jobConf.setReducerClass(IdentityReducer.class);
JobClient client = new JobClient(jobConf);
ClusterStatus cluster = client.getClusterStatus();
int num_maps = cluster.getTaskTrackers() *
jobConf.getInt("test.sort.maps_per_host", 10);
int num_reduces = (int) (cluster.getMaxReduceTasks() * 0.9);
String sort_reduces = jobConf.get("test.sort.reduces_per_host");
if (sort_reduces != null) {
num_reduces = cluster.getTaskTrackers() *
Integer.parseInt(sort_reduces);
}
jobConf.setNumReduceTasks(num_reduces);
-----------
Any idea why my parameter for reduce tasks is being ignored ?
Rob:
It is setting the number of reducers itself. See the line:
jobConf.setNumReduceTasks(num_reduces)
In short, you can't control the number of reducers this code uses from
the command line.
--
Eric Sammer
e...@lifeless.net
http://esammer.blogspot.com
