This link may help you with the configuration http://hortonworks.com/blog/how-to-plan-and-configure-yarn-in-hdp-2-0/
Thanks, Jian On Mon, Dec 2, 2013 at 7:53 PM, ch huang <justlo...@gmail.com> wrote: > 13/12/03 11:46:56 INFO mapreduce.Job: map 100% reduce 19% > 13/12/03 11:47:33 INFO mapreduce.Job: map 100% reduce 20% > 13/12/03 11:47:54 INFO mapreduce.Job: map 100% reduce 21% > 13/12/03 11:48:06 INFO mapreduce.Job: map 100% reduce 22% > 13/12/03 11:48:17 INFO mapreduce.Job: map 100% reduce 23% > 13/12/03 11:48:29 INFO mapreduce.Job: map 100% reduce 24% > 13/12/03 11:49:23 INFO mapreduce.Job: map 83% reduce 25% > 13/12/03 11:49:39 INFO mapreduce.Job: map 84% reduce 25% > 13/12/03 11:49:52 INFO mapreduce.Job: map 85% reduce 25% > 13/12/03 11:50:08 INFO mapreduce.Job: map 86% reduce 25% > 13/12/03 11:50:16 INFO mapreduce.Job: map 87% reduce 25% > 13/12/03 11:50:21 INFO mapreduce.Job: map 88% reduce 25% > 13/12/03 11:50:30 INFO mapreduce.Job: map 89% reduce 25% > 13/12/03 11:50:42 INFO mapreduce.Job: map 90% reduce 25% > 13/12/03 11:50:57 INFO mapreduce.Job: map 91% reduce 25% > 13/12/03 11:51:10 INFO mapreduce.Job: map 92% reduce 25% > 13/12/03 11:51:18 INFO mapreduce.Job: map 92% reduce 26% > 13/12/03 11:51:20 INFO mapreduce.Job: map 93% reduce 26% > 13/12/03 11:51:25 INFO mapreduce.Job: map 94% reduce 26% > 13/12/03 11:51:31 INFO mapreduce.Job: map 95% reduce 26% > 13/12/03 11:51:43 INFO mapreduce.Job: map 96% reduce 26% > 13/12/03 11:51:50 INFO mapreduce.Job: map 97% reduce 26% > 13/12/03 11:51:59 INFO mapreduce.Job: map 98% reduce 26% > 13/12/03 11:52:19 INFO mapreduce.Job: map 99% reduce 26% > 13/12/03 11:52:29 INFO mapreduce.Job: map 100% reduce 26% > > > > On Tue, Dec 3, 2013 at 10:25 AM, ch huang <justlo...@gmail.com> wrote: > >> another question is why the map process progress will back when it reach >> 100%? >> >> >> >> >> On Tue, Dec 3, 2013 at 10:07 AM, ch huang <justlo...@gmail.com> wrote: >> >>> hi,maillist: >>> i run a job on my CDH4.4 yarn framework ,it's map task >>> finished very fast,but reduce is very slow, i check it use ps command >>> find it's work heap size is 200m,so i try to increase heap size used by >>> reduce task,i add "YARN_OPTS="$YARN_OPTS >>> -Dmapreduce.reduce.java.opts=-Xmx1024m -verbose:gc -XX:+PrintGCDetails >>> -XX:+PrintGCDateStamps >>> -Xloggc:$YARN_LOG_DIR/gc-$(hostname)-resourcemanager.log >>> -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=5 -XX:GCLogFileSize=15M >>> -XX:-UseGCOverheadLimit" in yarn-env.sh file ,but when i restart the >>> nodemanager ,i find new reduce task still use 200m heap ,why? >>> >>> # jps >>> 2853 DataNode >>> 19533 Jps >>> 10949 YarnChild >>> 10661 NodeManager >>> 15130 HRegionServer >>> # ps -ef|grep 10949 >>> yarn 10949 10661 99 09:52 ? 00:19:31 >>> /usr/java/jdk1.7.0_45/bin/java -Djava.net.preferIPv4Stack=true >>> -Dhadoop.metrics.log.level=WARN -Xmx200m >>> -Djava.io.tmpdir=/data/1/mrlocal/yarn/local/usercache/hdfs/appcache/application_1385983958793_0022/container_1385983958793_0022_01_005650/tmp >>> -Dlog4j.configuration=container-log4j.properties >>> -Dyarn.app.mapreduce.container.log.dir=/data/2/mrlocal/yarn/logs/application_1385983958793_0022/container_1385983958793_0022_01_005650 >>> -Dyarn.app.mapreduce.container.log.filesize=0 -Dhadoop.root.logger=INFO,CLA >>> org.apache.hadoop.mapred.YarnChild 192.168.11.10 48936 >>> attempt_1385983958793_0022_r_000000_14 5650 >>> >>> >>> >>> >>> >> >> > -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
