FWIW, we actually push a completely separate config to the name node, jt, etc, because of some of the other settings (like saves and dfs.[in|ex]cludes). But if you wanted to do an all-in-one, well...
Hmm. Looking at the code, this worked differently than I always thought it did (at least in 0.18). Like Amogh, I thought that HADOOP_NAMENODE_OPTS (or at least HADOOP_NAMENODE_HEAPSIZE) would override, but that clearly isn't the case. I've filed HADOOP-6168 and appropriately bonked some of my local Hadoop committers on the head. :) On 7/22/09 7:50 AM, "Fernando Padilla" <f...@alum.mit.edu> wrote: > But right now the script forcefully adds and extra -Xmx1000m even if you > don't want it.. > > I guess I'll be submitting a patch for hadoop-daemon.sh later. :) :) > > thank you all > > > On 7/22/09 2:25 AM, Amogh Vasekar wrote: >> I haven't played a lot with it, but you may want to check if setting >> HADOOP_NAMENODE_OPTS, HADOOP_TASKTRACKER_OPTS help. Let me know if you find a >> way to do this :) >> >> Cheers! >> Amogh >> >> -----Original Message----- >> From: Fernando Padilla [mailto:f...@alum.mit.edu] >> Sent: Wednesday, July 22, 2009 9:47 AM >> To: common-user@hadoop.apache.org >> Subject: Re: best way to set memory >> >> I was thinking not for M/R, but for the actual daemons: >> >> When I go and start up a daemon (like below). They all use the same >> hadoop-env.sh. Which allows you to only set the HADOOP_HEAPSIZE once.. >> not differently for each daemon-type.. >> >> bin/hadoop-daemon.sh start namenode >> bin/hadoop-daemon.sh start datanode >> bin/hadoop-daemon.sh start secondarynamenode >> bin/hadoop-daemon.sh start jobtracker >> bin/hadoop-daemon.sh start tasktracker >> >> >> >> Amogh Vasekar wrote: >>> If you need to set the java_options for mem., you can do this via configure >>> in your MR job. >>> >>> -----Original Message----- >>> From: Fernando Padilla [mailto:f...@alum.mit.edu] >>> Sent: Wednesday, July 22, 2009 9:11 AM >>> To: common-user@hadoop.apache.org >>> Subject: best way to set memory >>> >>> So.. I want to have different memory profiles for >>> NameNode/DataNode/JobTracker/TaskTracker. >>> >>> But it looks like I only have one environment variable to modify, >>> HADOOP_HEAPSIZE, but I might be running more than one on a single >>> box/deployment/conf directory. >>> >>> Is there a proper way to set the memory for each kind of server? Or has >>> an issue been created to document this bug/deficiency??
