Ok, so the problem is you have a machine with 16 CPUs and a
computation job that uses at most 3 of them. Is this right?
What is the Mahout task? Do you know that it has good multi-Hadoop
performance and tuning? What matters is that the Partitioner for the
Mahout code can separate the computations.
Here is a "recipe" for how to run multiple datanodes on a single server, posted
to this list on Sept. 15:
http://mail-archives.apache.org/mod_mbox/hadoop-common-user/201009.mbox/%3c8a898c33-dc4e-418c-adc0-5689d434b...@yahoo-inc.com%3e
If you're having trouble getting multiple cores utili
On 30/11/10 10:32, Adarsh Sharma wrote:
Is it possible to run Hadoop in VMs on Production Clusters so that we
have 1s of nodes on 100s of servers to achieve high performance
through Cloud Computing.
you don't achieve performance that way. You are better off with 1VM per
physical host, and
Machines is certainly better than VMs. If you are running 4 VMs on top of
one machine with 128 GB RAM, each gets 32 GB. But the cost of 4 machines
with 32 gigs RAM would be less than the cost of one machine with 128 GB, so
then there's no point of going to hadoop right? Plus all the VMs would
compe
Is it possible to run Hadoop in VMs on Production Clusters so that we
have 1s of nodes on 100s of servers to achieve high performance
through Cloud Computing.
or
We have to simply configure Hadoop on 1s of commodity machines.
Which i
Try tweaking the mapred-site.xml config parameters.. these 2 parameters
could help.. if you haven't tried already:
mapred.job.reuse.jvm.num.tasks
-1
mapred.tasktracker.map.tasks.maximum
32
mapred.tasktracker.reduce.tasks.maximum
16
mapred.child.java.opts
last option i gave was to run hadoop in fully distributed mode
but you can run hadoop in pseudo distributed mode:
http://hadoop-tutorial.blogspot.com/2010/11/running-hadoop-in-pseudo-distributed.html
or
standalone mode:
http://hadoop-tutorial.blogspot.com/2010/11/running-hadoop-in-standalone-mode.
>If you want to just use one machine, why do you want to use hadoop? Hadoop's
>power lies in distributed computing. That being said, it is possible to use
>hadoop on a single machine by using the pseudo-distributed mode (Read
>http://hadoop.apache.org/common/docs/current/single_node_setup.html and
Hi beneo,
If you want to just use one machine, why do you want to use hadoop? Hadoop's
power lies in distributed computing. That being said, it is possible to use
hadoop on a single machine by using the pseudo-distributed mode (Read
http://hadoop.apache.org/common/docs/current/single_node_setup.ht
i'm sorry, but, are you sure??
At 2010-11-30 15:53:58，"rahul patodi" wrote:
>you can create virtual machines on your single machine:
>for you have to install sun virtual box(other tools are also available like
>VMware)
>now you can create as many virtual machine as you want
>then you can create on
you can create virtual machines on your single machine:
for you have to install sun virtual box(other tools are also available like
VMware)
now you can create as many virtual machine as you want
then you can create one master and all slaves
-Thanks and Regards,
Rahul Patodi
Associate Software Engi
i have only one machine and it's powerful.
so, i want the all the slaves and master on one machine?
thx in advanced
12 matches
Mail list logo
