Hi, Kousuke, Can you please explain a bit detailed what do you mean, I am new in Spark, looked at https://spark.apache.org/docs/latest/submitting-applications.html seems there is no '--files' option.
I just have to add '--files /path-to-metrics.properties' ? Undocumented ability? Thx for answer. On Thu, Sep 11, 2014 at 5:55 PM, Kousuke Saruta <saru...@oss.nttdata.co.jp> wrote: > Hi Vladimir > > How about use --files option with spark-submit? > > - Kousuke > > > (2014/09/11 23:43), Vladimir Tretyakov wrote: > > Hi again, yeah , I've tried to use ” spark.metrics.conf” before my > question in ML, had no luck:( > Any other ideas from somebody? > Seems nobody use metrics in YARN deployment mode. > How about Mesos? I didn't try but maybe Spark has the same difficulties on > Mesos? > > PS: Spark is great thing in general, will be nice to see metrics in > YARN/Mesos mode, not only in Standalone:) > > > On Thu, Sep 11, 2014 at 5:25 PM, Shao, Saisai <saisai.s...@intel.com> > wrote: > >> I think you can try to use ” spark.metrics.conf” to manually specify >> the path of metrics.properties, but the prerequisite is that each container >> should find this file in their local FS because this file is loaded locally. >> >> >> >> Besides I think this might be a kind of workaround, a better solution is >> to fix this by some other solutions. >> >> >> >> Thanks >> >> Jerry >> >> >> >> *From:* Vladimir Tretyakov [mailto:vladimir.tretya...@sematext.com] >> *Sent:* Thursday, September 11, 2014 10:08 PM >> *Cc:* user@spark.apache.org >> *Subject:* Re: JMXSink for YARN deployment >> >> >> >> Hi Shao, thx for explanation, any ideas how to fix it? Where should I put >> metrics.properties file? >> >> >> >> On Thu, Sep 11, 2014 at 4:18 PM, Shao, Saisai <saisai.s...@intel.com> >> wrote: >> >> Hi， >> >> >> >> I’m guessing the problem is that driver or executor cannot get the >> metrics.properties configuration file in the yarn container, so metrics >> system cannot load the right sinks. >> >> >> >> Thanks >> >> Jerry >> >> >> >> *From:* Vladimir Tretyakov [mailto:vladimir.tretya...@sematext.com] >> *Sent:* Thursday, September 11, 2014 7:30 PM >> *To:* user@spark.apache.org >> *Subject:* JMXSink for YARN deployment >> >> >> >> Hello, we are in Sematext (https://apps.sematext.com/) are writing >> Monitoring tool for Spark and we came across one question: >> >> >> >> How to enable JMX metrics for YARN deployment? >> >> >> >> We put "*.sink.jmx.class=org.apache.spark.metrics.sink.JmxSink" >> >> to file $SPARK_HOME/conf/metrics.properties but it doesn't work. >> >> >> >> Everything works in Standalone mode, but not in YARN mode. >> >> >> >> Can somebody help? >> >> >> >> Thx! >> >> >> >> PS: I've found also >> https://stackoverflow.com/questions/23529404/spark-on-yarn-how-to-send-metrics-to-graphite-sink/25786112 >> without answer. >> >> >> > > >
