Hi, anyone has any ideas please?
On Mon, Sep 5, 2016 at 8:30 PM, map reduced wrote:
> Hi,
>
> I've written my custom metrics source/sink for my Spark streaming app and
> I am trying to initialize it from metrics.properties - but that doesn't
> work from executors. I don't have control on the mac
We use Graphite/Grafana for custom metrics. We found Spark’s metrics not to be
customizable. So, we write directly using Graphite’s API, which was very easy
to do using Java’s socket library in Scala. It works great for us, and we are
going one step further using Sensu to alert us if there is an
Thanks for the reply, I wish it did. We have an internal metrics system
where we need to submit to. I am sure that the ways I've tried work with
yarn deployment, but not with standalone.
Thanks,
KP
On Tue, Sep 6, 2016 at 11:36 PM, Benjamin Kim wrote:
> We use Graphite/Grafana for custom metrics
Can this be listed as an issue on JIRA?
On Wed, Sep 7, 2016 at 10:19 AM, map reduced wrote:
> Thanks for the reply, I wish it did. We have an internal metrics system
> where we need to submit to. I am sure that the ways I've tried work with
> yarn deployment, but not with standalone.
>
> Thanks,
I just stumbled upon this issue as well in Spark 1.6.2 when trying to write
my own custom Sink. For anyone else who runs into this issue, there are
two relevant JIRAs that I found, but no solution as of yet:
- https://issues.apache.org/jira/browse/SPARK-14151 - Propose to refactor
and expose Metri