[
https://issues.apache.org/jira/browse/SPARK-2868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15827519#comment-15827519
]
Kyle Heath commented on SPARK-2868:
---
Short version: Is there anything I can do to help bring this feature to
pyspark?
Long version: I've been implementing large jobs in pyspark for about 6 months.
The ability to monitor named accumulators in the web-ui seems really important.
Running complex jobs at scale has been a bit like flying blind. I'm new to
this community, but want to help if I can.
> Support named accumulators in Python
>
>
> Key: SPARK-2868
> URL: https://issues.apache.org/jira/browse/SPARK-2868
> Project: Spark
> Issue Type: New Feature
> Components: PySpark
>Reporter: Patrick Wendell
>
> SPARK-2380 added this for Java/Scala. To allow this in Python we'll need to
> make some additional changes. One potential path is to have a 1:1
> correspondence with Scala accumulators (instead of a one-to-many). A
> challenge is exposing the stringified values of the accumulators to the Scala
> code.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
