Hi, We are starting to use https://github.com/banzaicloud/spark-metrics . Keep in mind that their solution is for Spark for K8s, to make it work for Spark on Yarn you have to copy the dependencies of the spark-metrics into Spark Jars folders on all the Spark machines (took me a while to figure).
Thanks, Alex On Fri, Sep 13, 2019 at 7:58 AM raman gugnani <ramangugnani....@gmail.com> wrote: > Hi Team, > > I am new to spark. I am using spark on hortonworks dataplatform with > amazon EC2 machines. I am running spark in cluster mode with yarn. > > I need to monitor individual JVMs and other Spark metrics with > *prometheus*. > > Can anyone suggest the solution to do the same. > > -- > Raman Gugnani >
