code for finding where the map outputs are transferred to file.

2010-08-17 Thread Rahul.V.
Hi, Ive read that the intermediate map output is written to the disk at the regular intervals. Infact Ive read that there are background threads which spill the data onto disk whenever it crosses the threshold.[Source:Hadoop: The Definitive Guide.] Ive tried to dig into the code a couple of times

Error with Heap Space.

2010-08-16 Thread Rahul.V.
Hi, Am not sure if this is the right place to post this doubt.We tried implementing M/R on a different distributed file system developed in house. Every time I run more than 30 threads I get an error saying Java.lang.outofmemory: Heap exception Is the heap error due to the number of threads or

Re: Error with Heap Space.

2010-08-16 Thread Rahul.V.
--- Regards, Vitaliy S On Mon, Aug 16, 2010 at 11:04 AM, Rahul.V. greatness.hardn...@gmail.com wrote: Hi, Am not sure if this is the right place to post this doubt.We tried implementing M/R on a different distributed file system

hadoop on unstable nodes

2010-08-02 Thread Rahul.V.
Hi, Is there any research currently going on where map reduce is applied to nodes in normal internet scenarios?.In environments where network bandwidth is at premium what are the tweaks applied to hadoop? I would be very thankful if you can post me links in this direction. -- Regards, R.V.

a benchmarking software in linux

2010-07-19 Thread Rahul.V.
Hi, while working on hadoop I realised the need for a benchmarking software to compare various nodes. Can you let me know some popular benchmarking programs which are used to compare cpus. -- Regards, R.V.

help with hadoop source code

2010-07-11 Thread Rahul.V.
Hi, Am trying to implement a small map reduce framework on another file system. A very elementary one at that. So I am trying to access the Hadoop source code for that. Where do I find it? -- Regards, R.V.

Re: help with hadoop source code

2010-07-11 Thread Rahul.V.
Thanks a lot.. On Sun, Jul 11, 2010 at 9:13 PM, Ted Yu yuzhih...@gmail.com wrote: http://hadoop.apache.org/common/version_control.html#Anonymous+Access+%28read-only%29 On Sat, Jul 10, 2010 at 11:13 PM, Rahul.V. greatness.hardn...@gmail.com wrote: Hi, Am trying to implement a small map