Re: Spark Metrics: custom source/sink configurations not getting recognized

I just stumbled upon this issue as well in Spark 1.6.2 when trying to write my own custom Sink. For anyone else who runs into this issue, there are two relevant JIRAs that I found, but no solution as of yet: - https://issues.apache.org/jira/browse/SPARK-14151 - Propose to refactor and expose

Re: Spark Metrics: custom source/sink configurations not getting recognized

Can this be listed as an issue on JIRA? On Wed, Sep 7, 2016 at 10:19 AM, map reduced wrote: > Thanks for the reply, I wish it did. We have an internal metrics system > where we need to submit to. I am sure that the ways I've tried work with > yarn deployment, but not with

Re: Spark Metrics: custom source/sink configurations not getting recognized

Thanks for the reply, I wish it did. We have an internal metrics system where we need to submit to. I am sure that the ways I've tried work with yarn deployment, but not with standalone. Thanks, KP On Tue, Sep 6, 2016 at 11:36 PM, Benjamin Kim wrote: > We use

Re: Spark Metrics: custom source/sink configurations not getting recognized

We use Graphite/Grafana for custom metrics. We found Spark’s metrics not to be customizable. So, we write directly using Graphite’s API, which was very easy to do using Java’s socket library in Scala. It works great for us, and we are going one step further using Sensu to alert us if there is

Re: Spark Metrics: custom source/sink configurations not getting recognized

Hi, anyone has any ideas please? On Mon, Sep 5, 2016 at 8:30 PM, map reduced wrote: > Hi, > > I've written my custom metrics source/sink for my Spark streaming app and > I am trying to initialize it from metrics.properties - but that doesn't > work from executors. I don't