Hi, Thanks for your response.
The /metrics API returns a blank page on our RM. The /jmx API has some metrics, but these are the same metrics we are already loading into data-dog. It's not good enough, because it doesn't break down the memory use by container. I need the by-container breakdown because resource allocation is per container and I would like to se if my job is really using up all the allocated memory. Shmuel On Tue, Jun 13, 2017 at 6:05 PM, Sidharth Kumar <sidharthkumar2...@gmail.com > wrote: > Hi, > > I guess you can get it from http://<resourcemanager-host>:<rm-port>/jmx > or /metrics > > Regards > Sidharth > LinkedIn: www.linkedin.com/in/sidharthkumar2792 > > On 13-Jun-2017 6:26 PM, "Shmuel Blitz" <shmuel.bl...@similarweb.com> > wrote: > >> (This question has also been published on StackOveflow >> <https://stackoverflow.com/q/44484940/416300>) >> >> I am looking for a way to monitor memory usage of YARN containers over >> time. >> >> Specifically - given a YARN application-id, how can you get a graph, >> showing the memory usage of each of its containers over time? >> >> The main goal is to better fit memory allocation requirements for our >> YARN applications (Spark / Map-Reduce), to avoid over allocation and >> cluster resource waste. A side goal would be the ability to debug memory >> issues when developing our jobs and attempting to pick reasonable resource >> allocations. >> >> We've tried using the Data-Dog integration, But it doesn't break down the >> metrics by container. >> >> Another approach was to parse the hadoop-yarn logs. These logs have >> messages like: >> >> Memory usage of ProcessTree 57251 for container-id >> container_e116_1495951495692_35134_01_000001: 1.9 GB of 11 GB physical >> memory used; 14.4 GB of 23.1 GB virtual memory used >> Parsing the logs correctly can yield data that can be used to plot a >> graph of memory usage over time. >> >> That's exactly what we want, but there are two downsides: >> >> It involves reading human-readable log lines and parsing them into >> numeric data. We'd love to avoid that. >> If this data can be consumed otherwise, we're hoping it'll have more >> information that we might be interest in in the future. We wouldn't want to >> put the time into parsing the logs just to realize we need something else. >> Is there any other way to extract these metrics, either by plugging in to >> an existing producer or by writing a simple listener? >> >> Perhaps a whole other approach? >> >> -- >> [image: Logo] >> <https://www.similarweb.com/?utm_source=WiseStamp&utm_medium=email&utm_term=&utm_content=&utm_campaign=signature> >> Shmuel Blitz >> *Big Data Developer* >> www.similarweb.com >> <http://www.similarweb.com?utm_source=WiseStamp&utm_medium=email&utm_term=&utm_content=&utm_campaign=signature> >> >> <https://www.facebook.com/SimilarWeb/?fref=ts&utm_source=WiseStamp&utm_medium=email&utm_term=&utm_content=&utm_campaign=signature> >> Like >> Us >> <https://www.facebook.com/SimilarWeb/?fref=ts&utm_source=WiseStamp&utm_medium=email&utm_term=&utm_content=&utm_campaign=signature> >> >> <https://twitter.com/SimilarWeb?utm_source=WiseStamp&utm_medium=email&utm_term=&utm_content=&utm_campaign=signature> >> Follow >> Us >> <https://twitter.com/SimilarWeb?utm_source=WiseStamp&utm_medium=email&utm_term=&utm_content=&utm_campaign=signature> >> >> <https://www.youtube.com/watch?v=Sb09jaZYY7s&utm_source=WiseStamp&utm_medium=email&utm_term=&utm_content=&utm_campaign=signature> >> Watch >> Us >> <https://www.youtube.com/watch?v=Sb09jaZYY7s&utm_source=WiseStamp&utm_medium=email&utm_term=&utm_content=&utm_campaign=signature> >> >> <https://www.similarweb.com/blog/?utm_source=WiseStamp&utm_medium=email&utm_term=&utm_content=&utm_campaign=signature> >> Read >> Us >> <https://www.similarweb.com/blog/?utm_source=WiseStamp&utm_medium=email&utm_term=&utm_content=&utm_campaign=signature> >> > -- [image: Logo] <https://www.similarweb.com/?utm_source=WiseStamp&utm_medium=email&utm_term=&utm_content=&utm_campaign=signature> Shmuel Blitz *Big Data Developer* www.similarweb.com <http://www.similarweb.com?utm_source=WiseStamp&utm_medium=email&utm_term=&utm_content=&utm_campaign=signature> <https://www.facebook.com/SimilarWeb/?fref=ts&utm_source=WiseStamp&utm_medium=email&utm_term=&utm_content=&utm_campaign=signature> Like Us <https://www.facebook.com/SimilarWeb/?fref=ts&utm_source=WiseStamp&utm_medium=email&utm_term=&utm_content=&utm_campaign=signature> <https://twitter.com/SimilarWeb?utm_source=WiseStamp&utm_medium=email&utm_term=&utm_content=&utm_campaign=signature> Follow Us <https://twitter.com/SimilarWeb?utm_source=WiseStamp&utm_medium=email&utm_term=&utm_content=&utm_campaign=signature> <https://www.youtube.com/watch?v=Sb09jaZYY7s&utm_source=WiseStamp&utm_medium=email&utm_term=&utm_content=&utm_campaign=signature> Watch Us <https://www.youtube.com/watch?v=Sb09jaZYY7s&utm_source=WiseStamp&utm_medium=email&utm_term=&utm_content=&utm_campaign=signature> <https://www.similarweb.com/blog/?utm_source=WiseStamp&utm_medium=email&utm_term=&utm_content=&utm_campaign=signature> Read Us <https://www.similarweb.com/blog/?utm_source=WiseStamp&utm_medium=email&utm_term=&utm_content=&utm_campaign=signature>
