Hello everyone,
I was reading the tutorial for Google Web dataset (local mode, pseudo
distributed cluser) at http://wiki.apache.org/hama/WriteHamaGraphFile.
I downloaded a Graph (~1GB) and uploaded it to hdfs. The file was splitted in
17 hdfs chunks. (I remind that I am trying to run in pseudo distributed mode, I
have everything in my laptop).
ls -l /tmp/hadoop/dfs/data/current/ | grep -v 'meta' | wc -l
17
Then I tried to run a code I wrote based on the example but:
# tail -f hama-my-bspmaster-my.local.log
2013-03-09 20:09:47,060 INFO org.apache.hama.bsp.JobInProgress: num BSPTasks: 17
2013-03-09 20:09:47,100 INFO org.apache.hama.bsp.JobInProgress: Job is
initialized.
2013-03-09 20:09:47,103 ERROR org.apache.hama.bsp.SimpleTaskScheduler: Could
not schedule all tasks!
2013-03-09 20:09:47,103 ERROR org.apache.hama.bsp.SimpleTaskScheduler:
Scheduling of job Hama Graph Loader could not be done successfully. Killing it!
My configuration is:
HamaConfiguration conf = new HamaConfiguration(new Configuration());
conf.set("bsp.local.tasks.maximum", "1");
GraphJob graphJob = new GraphJob(conf, HamaGraphLoader.class);
graphJob.setNumBspTask(1);
I also tried to change the values of:
conf.set("bsp.tasks.maximum", "1");
conf.set("bsp.max.tasks.per.job", "1");
conf.set("mapred.map.tasks", "1");
conf.set("mapred.min.split.size", String.valueOf(Long.MAX_VALUE));
I even changed the according variables from hama-default.xml but nothing seems
to be working. The number of BSPTasks remain to 17.
I attach my full configuration just in case.
Thank you in advance,
Anastasis
