Josh Wills created CRUNCH-435:
---------------------------------
Summary: Cleaner controls for configuring and initializing the
SparkContext and runtime Configuration
Key: CRUNCH-435
URL: https://issues.apache.org/jira/browse/CRUNCH-435
Project: Crunch
Issue Type: Bug
Reporter: Josh Wills
I spent a good chunk of the week working with the SparkPipeline implementation,
and I came up with a few improvements to the flow that controls how the
JavaSparkContext gets created when we're not given one directly.
The main idea is to re-use the Pipeline's Configuration object to specify any
of the critical spark-specific configuration options (e.g., memory allocations)
in a SparkConf instance before constructing the JavaSparkContext.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
