Hi,
We are using hadoop installation over 6 nodes, that looks like:
172.19.0.10 (master)
172.19.0.11
172.19.0.12
172.19.0.13
172.19.0.14
172.19.0.15
Each node has (4x1.5 TB) hard disk (hdparm -t shows ~125 MB/sec for each
other) and 12 GB memory. Connected to
100 Mb switch.
I get following resul
I had a similar issue and found the profiling information to be helpful:
http://hadoop.apache.org/common/docs/current/mapred_tutorial.html#Profiling
It should tell you where it's spending most of the time.
On Wed, Dec 9, 2009 at 8:51 AM, Dmitriy Lyfar wrote:
> Hi,
>
> We are using hadoop instal
2009/12/15 William Kinney
> I had a similar issue and found the profiling information to be helpful:
> http://hadoop.apache.org/common/docs/current/mapred_tutorial.html#Profiling
>
> It should tell you where it's spending most of the time.
>
>
Thank you William, which problem did you have if you
Our machines suffered from bad memcpy performance, which became
apparent after profiling (a lot of time in System.arraycopy() for our
inputformat reader).
On 12/16/09, Dmitriy Lyfar wrote:
> 2009/12/15 William Kinney
>
>> I had a similar issue and found the profiling information to be helpful:
>
