Re: hadoop on unstable nodes

2010-08-03 Thread He Chen
Condor has a hadoop subproject in UW-Madison, and there are also some scientists from VT. They worked on security Hadoop MapReduce on Internet. In my opinion, Alex is correct, Hadoop MR is communication intensive especially in the map and shuffle stage. In the map stage, every mapper needs input d

Re: hadoop on unstable nodes

2010-08-03 Thread Alex Loddengaard
I don't know of any research, but such a scenario is likely not going to turn out so well. Hadoop is very network hungry and is designed to be run in a datacenter. Sorry I don't have more information for you. Alex On Mon, Aug 2, 2010 at 9:14 PM, Rahul.V. wrote: > Hi, > Is there any research cu

hadoop on unstable nodes

2010-08-02 Thread Rahul.V.
Hi, Is there any research currently going on where map reduce is applied to nodes in normal internet scenarios?.In environments where network bandwidth is at premium what are the tweaks applied to hadoop? I would be very thankful if you can post me links in this direction. -- Regards, R.V.