I think you can try to use ” spark.metrics.conf” to manually specify the path of metrics.properties, but the prerequisite is that each container should find this file in their local FS because this file is loaded locally.
Besides I think this might be a kind of workaround, a better solution is to fix this by some other solutions. Thanks Jerry From: Vladimir Tretyakov [mailto:[email protected]] Sent: Thursday, September 11, 2014 10:08 PM Cc: [email protected] Subject: Re: JMXSink for YARN deployment Hi Shao, thx for explanation, any ideas how to fix it? Where should I put metrics.properties file? On Thu, Sep 11, 2014 at 4:18 PM, Shao, Saisai <[email protected]<mailto:[email protected]>> wrote: Hi， I’m guessing the problem is that driver or executor cannot get the metrics.properties configuration file in the yarn container, so metrics system cannot load the right sinks. Thanks Jerry From: Vladimir Tretyakov [mailto:[email protected]<mailto:[email protected]>] Sent: Thursday, September 11, 2014 7:30 PM To: [email protected]<mailto:[email protected]> Subject: JMXSink for YARN deployment Hello, we are in Sematext (https://apps.sematext.com/) are writing Monitoring tool for Spark and we came across one question: How to enable JMX metrics for YARN deployment? We put "*.sink.jmx.class=org.apache.spark.metrics.sink.JmxSink" to file $SPARK_HOME/conf/metrics.properties but it doesn't work. Everything works in Standalone mode, but not in YARN mode. Can somebody help? Thx! PS: I've found also https://stackoverflow.com/questions/23529404/spark-on-yarn-how-to-send-metrics-to-graphite-sink/25786112 without answer.
