Re: Spark Metrics Framework?

2016-04-01 Thread Yiannis Gkoufas
t which is required to read the >> counters. Then it could publish the counters to the Spark metrics system. >> >> I would suggest going ahead and submitting a JIRA request if there isn’t >> one already. >> >> Thanks, >> Silvio >> >> From: Mike Sukmanowsk

Re: Spark Metrics Framework?

2016-04-01 Thread Mike Sukmanowsky
gt; > Thanks, > Silvio > > From: Mike Sukmanowsky > Date: Friday, March 25, 2016 at 10:48 AM > > To: Silvio Fiorito , "user@spark.apache.org" > > Subject: Re: Spark Metrics Framework? > > Pinging again - any thoughts? > > On Wed, 23 Mar 2016 at 0

Re: Spark Metrics Framework?

2016-03-25 Thread Silvio Fiorito
.apache.org<mailto:user@spark.apache.org>" mailto:user@spark.apache.org>> Subject: Re: Spark Metrics Framework? Pinging again - any thoughts? On Wed, 23 Mar 2016 at 09:17 Mike Sukmanowsky mailto:mike.sukmanow...@gmail.com>> wrote: Thanks Ted and Silvio. I think I'll

Re: Spark Metrics Framework?

2016-03-25 Thread Mike Sukmanowsky
> wrote: > >> Hi Mike, >> >> It’s been a while since I worked on a custom Source but I think all you >> need to do is make your Source in the org.apache.spark package. >> >> Thanks, >> Silvio >> >> From: Mike Sukmanowsky >> Date: Tu

Re: Spark Metrics Framework?

2016-03-23 Thread Mike Sukmanowsky
hile since I worked on a custom Source but I think all you > need to do is make your Source in the org.apache.spark package. > > Thanks, > Silvio > > From: Mike Sukmanowsky > Date: Tuesday, March 22, 2016 at 3:13 PM > To: Silvio Fiorito , "user@spark.apache.org&qu

Re: Spark Metrics Framework?

2016-03-22 Thread Silvio Fiorito
iorito mailto:silvio.fior...@granturing.com>>, "user@spark.apache.org<mailto:user@spark.apache.org>" mailto:user@spark.apache.org>> Subject: Re: Spark Metrics Framework? The Source class is private<https://github.com/apache/spark/blob/v1.4.1/core/src/main/scala/org/apache/spar

Re: Spark Metrics Framework?

2016-03-22 Thread Ted Yu
l#metrics >> >> If you want to push the metrics to another system you can define a custom >> sink. You can also extend the metrics by defining a custom source. >> >> From: Mike Sukmanowsky >> Date: Monday, March 21, 2016 at 11:54 AM >> To: "user@spa

Re: Spark Metrics Framework?

2016-03-22 Thread Mike Sukmanowsky
2016 at 11:54 AM > To: "user@spark.apache.org" > Subject: Spark Metrics Framework? > > We make extensive use of the elasticsearch-hadoop library for > Hadoop/Spark. In trying to troubleshoot our Spark applications, it'd be > very handy to have access to some of t

Re: Spark Metrics Framework?

2016-03-21 Thread Silvio Fiorito
mailto:mike.sukmanow...@gmail.com>> Date: Monday, March 21, 2016 at 11:54 AM To: "user@spark.apache.org<mailto:user@spark.apache.org>" mailto:user@spark.apache.org>> Subject: Spark Metrics Framework? We make extensive use of the elasticsearch-hadoop library for Hadoop/Spark. In

Spark Metrics Framework?

2016-03-21 Thread Mike Sukmanowsky
We make extensive use of the elasticsearch-hadoop library for Hadoop/Spark. In trying to troubleshoot our Spark applications, it'd be very handy to have access to some of the many metrics that the library makes available wh