Re: Setting the heap size

2010-10-29 Thread Sean Bigdatafun
Why would you only run 9 RS and leave 3 mapreduce-only nodes? I can't see
any benefit of doing that.

Sean

On Thu, Oct 28, 2010 at 2:52 AM, Tim Robertson timrobertson...@gmail.comwrote:

 Hi all,

 We are setting up a small Hadoop 13 node cluster running 1 HDFS
 master, 9 region severs for HBase and 3 map reduce nodes, and are just
 installing zookeeper to perform the HBase coordination and to manage a
 few simple process locks for other tasks we run.

 Could someone please advise what kind on heap we should give to our
 single ZK node and also (ahem) how does one actually set this? It's
 not immediately obvious in the docs or config.

 Thanks,
 Tim



Re: Setting the heap size

2010-10-29 Thread Tim Robertson
Great - thanks Patrick!


On Thu, Oct 28, 2010 at 6:13 PM, Patrick Hunt ph...@apache.org wrote:
 Tim, one other thing you might want to be aware of:
 http://hadoop.apache.org/zookeeper/docs/current/zookeeperAdmin.html#sc_supervision

 Patrick

 On Thu, Oct 28, 2010 at 9:11 AM, Patrick Hunt ph...@apache.org wrote:
 On Thu, Oct 28, 2010 at 2:52 AM, Tim Robertson
 timrobertson...@gmail.com wrote:
 We are setting up a small Hadoop 13 node cluster running 1 HDFS
 master, 9 region severs for HBase and 3 map reduce nodes, and are just
 installing zookeeper to perform the HBase coordination and to manage a
 few simple process locks for other tasks we run.

 Could someone please advise what kind on heap we should give to our
 single ZK node and also (ahem) how does one actually set this? It's
 not immediately obvious in the docs or config.

 The amount of heap necessary will be dependent on the application(s)
 using ZK, also configuration of the heap is dependent on what
 packaging you are using to start ZK.

 Are you using zkServer.sh from our distribution? If so then you
 probably want to set JVMFLAGS env variable. We pass this through to
 the jvm, see -Xmx in the man page
 (http://www.manpagez.com/man/1/java/)

 Given this is Hbase (which I'm reasonably familiar with) the default
 heap should be fine. However you might want to check with the Hbase
 team on that.

 I'd also encourage you to enter a JIRA on the (lack of) doc issue you
 highlighted:  https://issues.apache.org/jira/browse/ZOOKEEPER

 Regards,

 Patrick