Hoot, these are big numbers - some thoughts
1) does your machine have 1000GB to spare for each java child thread (each
mapper + each reducer)? mapred.child.java.opts / -Xmx1048576m
2) does each of your daemons need / have 10G? HADOOP_HEAPSIZE=1
hth
EF
> # The maximum amount of heap to use, i
hanks for your time
>
> Marco Didonna
>
> PS: I use both locally and on the cloud latest version of cloudera
> distribution for hadoop
>
--
*Eric Fiala*
*Fiala Consulting*
T: 403.828.1117
E: e...@fiala.ca
http://www.fiala.ca
