Author: shv Date: Fri Jan 28 22:40:42 2011 New Revision: 1064917 URL: http://svn.apache.org/viewvc?rev=1064917&view=rev Log: HADOOP-6812. Change documentation for correct placement of configuration variables: mapreduce.reduce.input.buffer.percent, mapreduce.task.io.sort.factor, mapreduce.task.io.sort.mb. Contributed by Chris Douglas.
Modified: hadoop/common/trunk/CHANGES.txt hadoop/common/trunk/src/docs/src/documentation/content/xdocs/cluster_setup.xml hadoop/common/trunk/src/docs/src/documentation/content/xdocs/hod_scheduler.xml Modified: hadoop/common/trunk/CHANGES.txt URL: http://svn.apache.org/viewvc/hadoop/common/trunk/CHANGES.txt?rev=1064917&r1=1064916&r2=1064917&view=diff ============================================================================== --- hadoop/common/trunk/CHANGES.txt (original) +++ hadoop/common/trunk/CHANGES.txt Fri Jan 28 22:40:42 2011 @@ -268,6 +268,11 @@ Release 0.22.0 - Unreleased HADOOP-7110. Implement chmod with JNI. (todd) + HADOOP-6812. Change documentation for correct placement of configuration + variables: mapreduce.reduce.input.buffer.percent, + mapreduce.task.io.sort.factor, mapreduce.task.io.sort.mb + (Chris Douglas via shv) + OPTIMIZATIONS HADOOP-6884. Add LOG.isDebugEnabled() guard for each LOG.debug(..). Modified: hadoop/common/trunk/src/docs/src/documentation/content/xdocs/cluster_setup.xml URL: http://svn.apache.org/viewvc/hadoop/common/trunk/src/docs/src/documentation/content/xdocs/cluster_setup.xml?rev=1064917&r1=1064916&r2=1064917&view=diff ============================================================================== --- hadoop/common/trunk/src/docs/src/documentation/content/xdocs/cluster_setup.xml (original) +++ hadoop/common/trunk/src/docs/src/documentation/content/xdocs/cluster_setup.xml Fri Jan 28 22:40:42 2011 @@ -666,22 +666,33 @@ </td> </tr> <tr> - <td>conf/core-site.xml</td> - <td>fs.inmemory.size.mb</td> - <td>200</td> + <td>conf/mapred-site.xml</td> + <td>mapreduce.reduce.shuffle.input.buffer.percent</td> + <td>0.80</td> <td> - Larger amount of memory allocated for the in-memory - file-system used to merge map-outputs at the reduces. + Larger amount of memory allocated for merging map output + in memory during the shuffle. Expressed as a fraction of + the total heap. </td> </tr> <tr> - <td>conf/core-site.xml</td> + <td>conf/mapred-site.xml</td> + <td>mapreduce.reduce.input.buffer.percent</td> + <td>0.80</td> + <td> + Larger amount of memory allocated for retaining map output + in memory during the reduce. Expressed as a fraction of + the total heap. + </td> + </tr> + <tr> + <td>conf/mapred-site.xml</td> <td>mapreduce.task.io.sort.factor</td> <td>100</td> <td>More streams merged at once while sorting files.</td> </tr> <tr> - <td>conf/core-site.xml</td> + <td>conf/mapred-site.xml</td> <td>mapreduce.task.io.sort.mb</td> <td>200</td> <td>Higher memory-limit while sorting data.</td> Modified: hadoop/common/trunk/src/docs/src/documentation/content/xdocs/hod_scheduler.xml URL: http://svn.apache.org/viewvc/hadoop/common/trunk/src/docs/src/documentation/content/xdocs/hod_scheduler.xml?rev=1064917&r1=1064916&r2=1064917&view=diff ============================================================================== --- hadoop/common/trunk/src/docs/src/documentation/content/xdocs/hod_scheduler.xml (original) +++ hadoop/common/trunk/src/docs/src/documentation/content/xdocs/hod_scheduler.xml Fri Jan 28 22:40:42 2011 @@ -250,7 +250,7 @@ info_port = Port number of the HDFS Name <p>For example:</p> <source> server-params = mapred.reduce.parallel.copies=20,io.sort.factor=100,io.sort.mb=128,io.file.buffer.size=131072 -final-server-params = mapred.child.java.opts=-Xmx512m,dfs.block.size=134217728,fs.inmemory.size.mb=128 +final-server-params = mapred.child.java.opts=-Xmx512m,dfs.block.size=134217728 </source> <p>In order to provide the options from command line, you can use the following syntax:</p> <p>For configuring the MapReduce daemons use:</p>