Github user raajay commented on the issue:
https://github.com/apache/spark/pull/18690
I understand. My previous comment was just a clarification to your
question: "I'm not sure how does this code work in your changes?". I will close
this PR. The JIRA is already closed.
---
If your
Github user jerryshao commented on the issue:
https://github.com/apache/spark/pull/18690
So I think if you want to connect your custom sink to Spark Metrics System,
then you should at least follow what Spark and codahale metrics library did.
Adding a feature in Spark specifically
Github user raajay commented on the issue:
https://github.com/apache/spark/pull/18690
@jerryshao My CustomSInk has the report function defined. What I did not
have was an equivalent of JmxReporter defined in my CustomSink. The reporter
essentially periodically invokes the report
Github user jerryshao commented on the issue:
https://github.com/apache/spark/pull/18690
If your custom sink doesn't have a "reporter", then I'm not sure how does
this code work in your changes? Since `MetricsSystem#report` will call report
API of each sink. I'm still not clearly
Github user raajay commented on the issue:
https://github.com/apache/spark/pull/18690
We were using a custom sink rather than the JmxSink for gathering metrics.
The sink did NOT have a "reporter" like the ones JmxSink or CsvSink have. I
guess a cleaner design is to implement a
Github user jerryshao commented on the issue:
https://github.com/apache/spark/pull/18690
I think it is because you don't configure metrics sink here, so by default
none of the metrics are reported.
For example if I enabled jmx sink
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/18690
Can one of the admins verify this patch?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this
