Hi Vikas,
You might want to check your logs. MR can generate huge logs depending
on what you are logging, and they are not on the DFS. The are on
non-dfs. If it's comming from there, you can change the loglevel to
reduce the size of the output.
On my own cluster, I turned the logs to debug and
What is the amount of data you are attempting to crunch in one MR job? Note
that Map intermediate outputs are written to disk before being sent to
reducers and this counts for non-DFS usage. So to say grossly, if your
input is 14 GB, you surely need more than 2 or 3 x 14G free space overall
to do
Ha, you joke, but we're planning on running with no local OS. If it works
as planned I'll post a nice summary of our approach. :)
On Wed, Jan 16, 2013 at 2:53 AM, Harsh J ha...@cloudera.com wrote:
kidding Wipe your OS out. /kidding
Please read: http://search-hadoop.com/m/9Qwi9UgMOe
On
I think you can still run with the OS on another drive, or on a live
USB drive, or even on the memory only, loaded from the network while
the server is booting from the network drive, etc. No?
JM
2013/1/16, Mohammad Tariq donta...@gmail.com:
That would be really cool Chris.
+1 for that.
Warm
kidding Wipe your OS out. /kidding
Please read: http://search-hadoop.com/m/9Qwi9UgMOe
On Wed, Jan 16, 2013 at 1:16 PM, Vikas Jadhav vikascjadha...@gmail.comwrote:
how to remove non dfs space from hadoop cluster
--
*
*
*
Thanx and Regards*
* Vikas Jadhav*
--
Harsh J
:)
Check whats consuming space other then core OS files.
Check temp spaces and other areas
On Wed, Jan 16, 2013 at 6:53 PM, Harsh J ha...@cloudera.com wrote:
kidding Wipe your OS out. /kidding
Please read: http://search-hadoop.com/m/9Qwi9UgMOe
On Wed, Jan 16, 2013 at 1:16 PM, Vikas
