[
https://issues.apache.org/jira/browse/SPARK-12958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15113665#comment-15113665
]
Sean Owen commented on SPARK-12958:
-----------------------------------
Meaning, an accumulator that sums Long values for keys? (It need not, and
probably should not, be specific to String keys.)
See https://cwiki.apache.org/confluence/display/SPARK/Contributing+to+Spark for
how to handle PRs.
The change itself looks reasonable; I'm just wondering if it's worth adding to
core.
> Map accumulator in spark
> ------------------------
>
> Key: SPARK-12958
> URL: https://issues.apache.org/jira/browse/SPARK-12958
> Project: Spark
> Issue Type: Wish
> Components: Spark Core
> Reporter: Souri
> Priority: Minor
>
> Spark by default supports accumulators of Int,Long,Double,Float.
> It would be good if we can have a Map accumulator where each executor can
> just add key->value pairs and driver can have access to the aggregated value
> for each key in the map.
> In this way, it would also be easier to use accumulators for various metrics.
> We can define metrics at runtime as the map can take any string key.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
