Hello, we are in Sematext (https://apps.sematext.com/) are writing
Monitoring tool for Spark and we came across one question:
How to enable JMX metrics for YARN deployment?
We put *.sink.jmx.class=org.apache.spark.metrics.sink.JmxSink
to file $SPARK_HOME/conf/metrics.properties but it doesn't
in the yarn container, so metrics
system cannot load the right sinks.
Thanks
Jerry
*From:* Vladimir Tretyakov [mailto:vladimir.tretya...@sematext.com]
*Sent:* Thursday, September 11, 2014 7:30 PM
*To:* user@spark.apache.org
*Subject:* JMXSink for YARN deployment
Hello, we are in Sematext
should find this file in their local FS because this file is loaded locally.
Besides I think this might be a kind of workaround, a better solution is
to fix this by some other solutions.
Thanks
Jerry
*From:* Vladimir Tretyakov [mailto:vladimir.tretya...@sematext.com]
*Sent
.
On Thu, Sep 11, 2014 at 5:55 PM, Kousuke Saruta saru...@oss.nttdata.co.jp
wrote:
Hi Vladimir
How about use --files option with spark-submit?
- Kousuke
(2014/09/11 23:43), Vladimir Tretyakov wrote:
Hi again, yeah , I've tried to use ” spark.metrics.conf” before my
question in ML
/spm-monitor-spark.jar=myValue-*2*
:spark-executor:default
...
...
...
Can I do something like that in Spark for executor? If not maybe it can be
done in the future? Will be useful.
Thx, best redgards, Vladimir Tretyakov.
> Hi,
>
> That's correct. One app one web UI. Open 4041 and you'll see the other
> app.
>
> Jacek
>
> On 9 Sep 2016 11:53 a.m., "Vladimir Tretyakov" <
> vladimir.tretya...@sematext.com> wrote:
>
>> Hello again.
>>
>> I am trying to pl
https://medium.com/@jaceklaskowski/
>> Mastering Apache Spark 2.0 http://bit.ly/mastering-apache-spark
>> Follow me at https://twitter.com/jaceklaskowski
>>
>>
>> On Sun, Sep 11, 2016 at 11:18 AM, Vladimir Tretyakov
>> <vladimir.tretya...@sematext.com> wrote:
Hi,
When I start Spark v1.6 (cdh5.8.0) in Yarn client mode I see that 4040
port is avaiable, but UI shows nothing and API returns not full information.
I started Spark application like this:
spark-submit --master yarn-client --class
org.apache.spark.examples.SparkPi
could try running simple Spark applications
> on Standalone and Yarn.
>
> On Fri, Sep 16, 2016 at 10:32 PM, Vladimir Tretyakov <
> vladimir.tretya...@sematext.com> wrote:
>
>> Hello.
>>
>> Found that there is also Spark metric Sink like MetricsServlet.
>> which is e
other than Standalone?
Why are there 2 ways to get information, REST API and this Sink?
Best regards, Vladimir.
On Mon, Sep 12, 2016 at 3:53 PM, Vladimir Tretyakov <
vladimir.tretya...@sematext.com> wrote:
> Hello Saisai Shao, Jacek Laskowski , thx for information.
>
>
10 matches
Mail list logo