Re: performance about writing data to HDFS

Hello, Here is an example. You can set an initial low replication like this code does:

Re: Questions About Event Listener For Yarn RM

Jessica, Thank you for raising this. Currently there is no event listener, you can only poll the REST API. Please feel free to open a jira for this. Best Regards, Miklos On Thu, Jan 4, 2018 at 3:41 PM, Herier chen wrote: > Hi Hadoop Experts, > > Happy New Years!! > > I

Re: Creating mapred-site.xml file

Gary, It is there in Hadoop 3.0 as well. It is just named simply as mapred-site.xml. # find . -name mapred-site* ./etc/hadoop/mapred-site.xml # find . -name mapred-site* | xargs cat Thank you, Miklos On Mon, Dec 25, 2017 at 2:04 PM, Gary Beckler wrote: > I am

Re: How to monitor YARN application memory per container?

Hello, MAPREDUCE-6829 was about showing the peak memory usage for mapreduce. Here are some of the new counters: [root@42e243b8cf16 hadoop]# bin/yarn jar ./share/hadoop/mapreduce/hadoop-mapreduce-examples-jar pi 1 1000 Number of Maps = 1 Samples per Map = 1000 ... Peak Map Physical

Re: Yarn not picking up the Resource manager configurations

Hi, I usually get this error when the yarn-site.xml is not on the class path and it picks up a default 0.0.0.0. I would try: 1. bin/yarn jar ./share/hadoop/mapreduce/hadoop-mapreduce-examples-2.7.3.jar pi 1 1000 2. explicitly adding $HADOOP_HOME/etc/hadoop to your client classpath 3. make sure

Re: Physical memory (bytes) snapshot counter question - how to get maximum memory used in reduce task

There are two new counters, MAP_PHYSICAL_MEMORY_BYTES_MAX and REDUCE_PHYSICAL_MEMORY_BYTES_MAX that give you the max value for map and reduce respectively. Thanks, Miklos On Wed, Apr 5, 2017 at 6:37 PM, Aaron Eng wrote: > An important consideration is the difference between the