[
https://issues.apache.org/jira/browse/CRUNCH-384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Josh Wills updated CRUNCH-384:
------------------------------
Attachment: CRUNCH-384.patch
This patch upgrades Crunch-on-Spark to use Spark 0.9.1 and fixes a number of
limitations in the old version: this version works w/Spark in standalone mode
on top of HDFS and also makes using Counters transparently map to using Spark's
accumulators (even storing them in the same place in the PipelineResult object.)
[~mkwhitacre] want to take a look at this one?
> Upgrade Spark to 0.9.1
> ----------------------
>
> Key: CRUNCH-384
> URL: https://issues.apache.org/jira/browse/CRUNCH-384
> Project: Crunch
> Issue Type: Improvement
> Components: Core
> Reporter: Josh Wills
> Assignee: Josh Wills
> Attachments: CRUNCH-384.patch
>
>
> Spark 0.9.x has better compatibility with Hadoop 2.x along with a bunch of
> other fixes. It also requires Scala 2.10, which we should upgrade to anyway.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
