szhem commented on a change in pull request #24556: [SPARK-27641][CORE] Fix
MetricsSystem to remove unregistered source correctly
URL: https://github.com/apache/spark/pull/24556#discussion_r284139036
##########
File path:
core/src/main/scala/org/apache/spark/metrics/source/AccumulatorSource.scala
##########
@@ -64,10 +64,11 @@ class DoubleAccumulatorSource extends AccumulatorSource
*/
@Experimental
object LongAccumulatorSource {
- def register(sc: SparkContext, accumulators: Map[String, LongAccumulator]):
Unit = {
+ def register(sc: SparkContext, accumulators: Map[String, LongAccumulator]):
Source = {
Review comment:
`AccumulatorSource` is package private as well. So to extend it, it will be
necessary to place your code within the same package. So there is no need to
expose package private `Source` and make it a part of the public contract.
To implement tests without changing the API you could use something like the
following
```scala
val source = new LongAccumulatorSource
source.register(accumulators)
sc.env.metricsSystem.registerSource(source)
```
... instead of
```scala
LongAccumulatorSource.register(mockContext, Map("laF" -> laFirst, "laS" ->
laSecond))
```
So the necessary source to delete during the tests will be already stored in
the local variable.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
