Just curious - is this HttpSink your own custom sink or Dropwizard
configuration?
If your own custom code, I would suggest looking/trying out the Dropwizard.
See
http://spark.apache.org/docs/latest/monitoring.html#metrics
https://metrics.dropwizard.io/4.0.0/
Also, from what I know, the metrics
Thanks a lot for the explanation
Spark declare the Sink trait with package private, that's why the package looks
weird, the metric system seems not intent to be extended
package org.apache.spark.metrics.sink
private[spark] trait Sink
Make the custom sink class available on every executor
First, it's really weird to use "org.apache.spark" for a class that is
not in Spark.
For executors, the jar file of the sink needs to be in the system
classpath; the application jar is not in the system classpath, so that
does not work. There are different ways for you to get it there, most
of
Hi, Spark Users
I'm play with spark metric monitoring, and want to add a custom sink which is
HttpSink that send the metric through Restful API
A subclass of Sink "org.apache.spark.metrics.sink.HttpSink" is created and
packaged within application jar
It works for driver instance, but once
