Ok, so the problem is you have a machine with 16 CPUs and a computation job that uses at most 3 of them. Is this right?
What is the Mahout task? Do you know that it has good multi-Hadoop performance and tuning? What matters is that the Partitioner for the Mahout code can separate the computations. 2010/11/30 Matthew Foley <ma...@yahoo-inc.com>: > Here is a "recipe" for how to run multiple datanodes on a single server, > posted to this list on Sept. 15: > > http://mail-archives.apache.org/mod_mbox/hadoop-common-user/201009.mbox/%3c8a898c33-dc4e-418c-adc0-5689d434b...@yahoo-inc.com%3e > > If you're having trouble getting multiple cores utilized, this should help, > assuming the computation is well expressed in Map/Reduce. > There is no point in running VMs in your situation, as it just adds overhead > without changing the amount of physical processor power > available. > > --Matt > > On Nov 30, 2010, at 1:21 AM, beneo_7 wrote: > >> If you want to just use one machine, why do you want to use hadoop? Hadoop's >> power lies in distributed computing. That being said, it is possible to use >> hadoop on a single machine by using the pseudo-distributed mode (Read >> http://hadoop.apache.org/common/docs/current/single_node_setup.html and >> http://www.michael-noll.com/tutorials/running-hadoop-on-ubuntu-linux-single-node-cluster/). >> If you are using just one machine, at least ensure that your machine has >> lots of cores (8 core/16 cores would be great) to get benefit out of hadoop. >> >> I am not sure, but using virtual machines won't be helpful here as a virtual >> machine is just an abstraction and not real hardware. > > > thanks very much, i use the hadoop because the apache Mahout project need it > for clustering. > > my machine is only one and powerful, 16cores and 32g mem, but i have only > one, so i need configuration about multi nodes on one machine. > > i have used the "pseudo-distributed mode", however, the project always used 1 > core, the cpu freq always 100% ~ 103%, the time the execution is 4 hours, > it's too slow. > > i can not change the mahout project source code, the trunk always update > sometime, it's difficulty to solute confliction. > > > is there any way to conf some slaves in one machine? > > At 2010-11-30 17:07:49，"Hari Sreekumar" <hsreeku...@clickable.com> wrote: > >> Hi beneo, >> >> If you want to just use one machine, why do you want to use hadoop? Hadoop's >> power lies in distributed computing. That being said, it is possible to use >> hadoop on a single machine by using the pseudo-distributed mode (Read >> http://hadoop.apache.org/common/docs/current/single_node_setup.html and >> http://www.michael-noll.com/tutorials/running-hadoop-on-ubuntu-linux-single-node-cluster/). >> If you are using just one machine, at least ensure that your machine has >> lots of cores (8 core/16 cores would be great) to get benefit out of hadoop. >> >> I am not sure, but using virtual machines won't be helpful here as a virtual >> machine is just an abstraction and not real hardware. >> >> Cheers, >> Hari >> >> 2010/11/30 beneo_7 <bene...@163.com> >> >>> i'm sorry, but, are you sure?? >>> At 2010-11-30 15:53:58，"rahul patodi" <patodira...@gmail.com> wrote: >>>> you can create virtual machines on your single machine: >>>> for you have to install sun virtual box(other tools are also available >>> like >>>> VMware) >>>> now you can create as many virtual machine as you want >>>> then you can create one master and all slaves >>>> >>>> -Thanks and Regards, >>>> Rahul Patodi >>>> Associate Software Engineer, >>>> Impetus Infotech (India) Private Limited, >>>> www.impetus.com >>>> Mob:09907074413 >>>> >>>> 2010/11/30 beneo_7 <bene...@163.com> >>>> >>>>> i have only one machine and it's powerful. >>>>> so, i want the all the slaves and master on one machine? >>>>> >>>>> thx in advanced >>>>> >>>> >>>> >>>> >>>> -- >>> >>> > > -- Lance Norskog goks...@gmail.com
