common-user@hadoop.apache.org
Sent: Monday, October 3, 2011 12:31 PM
Subject: Re: pointing mapred.local.dir to a ramdisk
Joey
Thanks. Will try and uppgrade to a newer version and check. I will also change
the logs to debug and see if more information is available.
Raj
mapred.local.dir to a ramdisk
Hi all
I have been trying some experiments to improve performance. One of the
experiments involved pointing mapred.local.dir to a RAM disk. To this end I
created a 128MB RAM disk ( each of my map outputs are smaller than this) but I
have not been able to get the task tracker
and not related to Cloudera distribution.
Raj
- Forwarded Message -
From: Raj V rajv...@yahoo.com
To: CDH Users cdh-u...@cloudera.org
Sent: Friday, September 30, 2011 5:21 PM
Subject: pointing mapred.local.dir to a ramdisk
Hi all
I have been trying some experiments to improve
eric.casp...@amd.com
To: common-user@hadoop.apache.org
Sent: Monday, October 3, 2011 8:44 AM
Subject: Re: pointing mapred.local.dir to a ramdisk
Are you sure you have chown'd/chmod'd the ramdisk directory to be writeable by
your hadoop user? I have played with this in the past and it should basically
@hadoop.apache.org
Sent: Monday, October 3, 2011 8:44 AM
Subject: Re: pointing mapred.local.dir to a ramdisk
Are you sure you have chown'd/chmod'd the ramdisk directory to be
writeable by your hadoop user? I have played with this in the past and it
should basically work.
On Oct 3, 2011
Caspole eric.casp...@amd.com
To: common-user@hadoop.apache.org
Sent: Monday, October 3, 2011 8:44 AM
Subject: Re: pointing mapred.local.dir to a ramdisk
Are you sure you have chown'd/chmod'd the ramdisk directory to be
writeable by your hadoop user? I have played with this in the past
everything else the same.
Raj
From: Vinod Kumar Vavilapalli vino...@hortonworks.com
To: common-user@hadoop.apache.org; Raj V rajv...@yahoo.com
Sent: Monday, October 3, 2011 9:07 AM
Subject: Re: pointing mapred.local.dir to a ramdisk
Must be related to some kind
claim given that I can't get it to run.
-best regards
Raj
From: Edward Capriolo edlinuxg...@gmail.com
To: common-user@hadoop.apache.org
Cc: Raj V rajv...@yahoo.com
Sent: Monday, October 3, 2011 10:36 AM
Subject: Re: pointing mapred.local.dir to a ramdisk
: Monday, October 3, 2011 10:36 AM
Subject: Re: pointing mapred.local.dir to a ramdisk
This directory can get very large, in many cases I doubt it would fit on a
ram disk.
Also RAM Disks tend to help most with random read/write, since hadoop is
doing mostly linear IO you may not see a great benefit from
, October 3, 2011 11:49 AM
Subject: Re: pointing mapred.local.dir to a ramdisk
Raj,
I just tried this on my CHD3u1 VM, and the ramdisk worked the first
time. So, it's possible you've hit a bug in CDH3b3 that was later
fixed. Can you enable debug logging in log4j.properties and then
repost your
10 matches
Mail list logo