Hi again, yeah , I've tried to use ” spark.metrics.conf” before my question in ML, had no luck:( Any other ideas from somebody? Seems nobody use metrics in YARN deployment mode. How about Mesos? I didn't try but maybe Spark has the same difficulties on Mesos?
PS: Spark is great thing in general, will be nice to see metrics in YARN/Mesos mode, not only in Standalone:) On Thu, Sep 11, 2014 at 5:25 PM, Shao, Saisai <saisai.s...@intel.com> wrote: > I think you can try to use ” spark.metrics.conf” to manually specify the > path of metrics.properties, but the prerequisite is that each container > should find this file in their local FS because this file is loaded locally. > > > > Besides I think this might be a kind of workaround, a better solution is > to fix this by some other solutions. > > > > Thanks > > Jerry > > > > *From:* Vladimir Tretyakov [mailto:vladimir.tretya...@sematext.com] > *Sent:* Thursday, September 11, 2014 10:08 PM > *Cc:* user@spark.apache.org > *Subject:* Re: JMXSink for YARN deployment > > > > Hi Shao, thx for explanation, any ideas how to fix it? Where should I put > metrics.properties file? > > > > On Thu, Sep 11, 2014 at 4:18 PM, Shao, Saisai <saisai.s...@intel.com> > wrote: > > Hi， > > > > I’m guessing the problem is that driver or executor cannot get the > metrics.properties configuration file in the yarn container, so metrics > system cannot load the right sinks. > > > > Thanks > > Jerry > > > > *From:* Vladimir Tretyakov [mailto:vladimir.tretya...@sematext.com] > *Sent:* Thursday, September 11, 2014 7:30 PM > *To:* user@spark.apache.org > *Subject:* JMXSink for YARN deployment > > > > Hello, we are in Sematext (https://apps.sematext.com/) are writing > Monitoring tool for Spark and we came across one question: > > > > How to enable JMX metrics for YARN deployment? > > > > We put "*.sink.jmx.class=org.apache.spark.metrics.sink.JmxSink" > > to file $SPARK_HOME/conf/metrics.properties but it doesn't work. > > > > Everything works in Standalone mode, but not in YARN mode. > > > > Can somebody help? > > > > Thx! > > > > PS: I've found also > https://stackoverflow.com/questions/23529404/spark-on-yarn-how-to-send-metrics-to-graphite-sink/25786112 > without answer. > > >
