Correction, they are 100Mbps NIC's...
iperf shows that we're getting about 95 Mbits/sec from one node to another.
On Apr 12, 2010, at 1:05 PM, Andrew Nguyen wrote:
@Todd:
I do need the sorting behavior, eventually. However, I'll try it with zero
reduce jobs to see.
@Alex:
Yes, I
Andrew,
here are some tips for hadoop runtime config:
http://cloudepr.blogspot.com/2009/09/cluster-facilities-hardware-and.html
also
here are some results from my cluster (using 1GE NICs, Fiber), Dell 5500,
24GB, 8-core (16 hypervised), JBOD, i saw slightly better numbers on a
different 4-nodes
Good to know... The problem is that I'm in an academic environment that
needs a lot of convincing regarding new computational technologies. I need
to show proven benefit before getting the funds to actually implement
anything. These servers were the best I could come up with for this
Hello,
I recently setup a 5 node cluster (1 master, 4 slaves) and am looking to use it
to process high volumes of patient physiologic data. As an initial exercise to
gain a better understanding, I have attempted to run the following problem
(which isn't the type of problem that Hadoop was
Andrew,
I would also suggest to run DFSIO benchmark to isolate io related issues
hadoop jar hadoop-0.20.2-test.jar TestDFSIO -write -nrFiles 10 -fileSize
1000
hadoop jar hadoop-0.20.2-test.jar TestDFSIO -read -nrFiles 10 -fileSize 1000
there are additional tests specific for mapreduce - run
I don't think you can :-). Sorry, they are 100Mbps NIC's... I get 95Mbit/sec
from one node to another with iperf.
Should I still be expecting such dismal performance with just 100Mbps?
On Apr 12, 2010, at 1:31 PM, Todd Lipcon wrote:
On Mon, Apr 12, 2010 at 1:05 PM, Andrew Nguyen
I guess my question below can be rephrased as, What is the absolute minimum hw
requirements for me to still see 'better-than-a-single-machine' performance?
Thanks!
On Apr 12, 2010, at 1:45 PM, Andrew Nguyen wrote:
I don't think you can :-). Sorry, they are 100Mbps NIC's... I get
