[
https://issues.apache.org/jira/browse/SPARK-14654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15255840#comment-15255840
]
holdenk commented on SPARK-14654:
-
Ah yes, in DAGScheduler right now we cast it to [Any, Any] when we get
[
https://issues.apache.org/jira/browse/SPARK-14654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15255820#comment-15255820
]
Reynold Xin commented on SPARK-14654:
-
But Holden the call sites in Spark are already erasing the
[
https://issues.apache.org/jira/browse/SPARK-14654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15255819#comment-15255819
]
holdenk commented on SPARK-14654:
-
Even if merge isn't supposed to be called by the end user it would be
[
https://issues.apache.org/jira/browse/SPARK-14654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15255363#comment-15255363
]
Reynold Xin commented on SPARK-14654:
-
I see. The merge function isn't supposed to be called by the
[
https://issues.apache.org/jira/browse/SPARK-14654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15255359#comment-15255359
]
holdenk commented on SPARK-14654:
-
So ACC isn't the internal buffer type, rather its the type of the
[
https://issues.apache.org/jira/browse/SPARK-14654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15255357#comment-15255357
]
Reynold Xin commented on SPARK-14654:
-
No it can't be private spark because it needs to be
[
https://issues.apache.org/jira/browse/SPARK-14654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15255356#comment-15255356
]
holdenk commented on SPARK-14654:
-
Since we're talking about average accumulators anyways - what would
[
https://issues.apache.org/jira/browse/SPARK-14654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15255355#comment-15255355
]
holdenk commented on SPARK-14654:
-
If were only going to do Long and Double for the easy creation on the
[
https://issues.apache.org/jira/browse/SPARK-14654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15255312#comment-15255312
]
Reynold Xin commented on SPARK-14654:
-
I don't get what you are trying to accomplish. It seems like
[
https://issues.apache.org/jira/browse/SPARK-14654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15255291#comment-15255291
]
holdenk commented on SPARK-14654:
-
You wouldn't know if you want a counter or overage, but the same
[
https://issues.apache.org/jira/browse/SPARK-14654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15255169#comment-15255169
]
Reynold Xin commented on SPARK-14654:
-
How do you know for double whether we'd want a counter or an
[
https://issues.apache.org/jira/browse/SPARK-14654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15255163#comment-15255163
]
holdenk commented on SPARK-14654:
-
[~rxin] its true, but compared to the current proposed new API we
[
https://issues.apache.org/jira/browse/SPARK-14654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15255159#comment-15255159
]
Reynold Xin commented on SPARK-14654:
-
That only works for the built-in ones and can't work if there
[
https://issues.apache.org/jira/browse/SPARK-14654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15255123#comment-15255123
]
holdenk commented on SPARK-14654:
-
Giving it a bit of thought on the flight, is there a reason why we
[
https://issues.apache.org/jira/browse/SPARK-14654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15254053#comment-15254053
]
Apache Spark commented on SPARK-14654:
--
User 'cloud-fan' has created a pull request for this issue:
[
https://issues.apache.org/jira/browse/SPARK-14654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15253000#comment-15253000
]
holdenk commented on SPARK-14654:
-
Looks like a nice clean API, out of interest what is `dedup` for? Is
[
https://issues.apache.org/jira/browse/SPARK-14654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15242540#comment-15242540
]
Reynold Xin commented on SPARK-14654:
-
cc [~holdenk] and [~imranr]
> New accumulator API
>
[
https://issues.apache.org/jira/browse/SPARK-14654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15242536#comment-15242536
]
Reynold Xin commented on SPARK-14654:
-
Note that one challenge is that we most likely don't want to
18 matches
Mail list logo