[ https://issues.apache.org/jira/browse/SPARK-1565?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Patrick Wendell updated SPARK-1565: ----------------------------------- Description: Example: https://github.com/pwendell/spark/commit/3fdad8ada28c1cef9b3b367993057de22295b8ed Now that we have the spark-submit script we should make some changes to the examples: 1. We should always create a SparkConf that sets the app name and use the SparkContext constructor that accepts a conf. 2. We shouldn't set the master based on a command line argument, instead users can do this on their own with spark-submit. 3. The examples projects should mark spark-core/streaming as a "provided" dependency, so that the examples assembly jar does not include spark. Then users can launch examples like this: {code} ./bin/spark-submit examples/target/scala-2.10/spark-examples-assembly-1.0.0-SNAPSHOT.jar \ --class org.apache.spark.examples.SparkPi \ --arg 1000 {code} was: https://github.com/pwendell/spark/commit/3fdad8ada28c1cef9b3b367993057de22295b8ed Now that we have the spark-submit script we should make some changes to the examples: 1. We should always create a SparkConf that sets the app name and use the SparkContext constructor that accepts a conf. 2. We shouldn't set the master based on a command line argument, instead users can do this on their own with spark-submit. 3. The examples projects should mark spark-core/streaming as a "provided" dependency, so that the examples assembly jar does not include spark. Then users can launch examples like this: {code} ./bin/spark-submit examples/target/scala-2.10/spark-examples-assembly-1.0.0-SNAPSHOT.jar \ --class org.apache.spark.examples.SparkPi \ --arg 1000 {code} > Spark examples should be changed given spark-submit > --------------------------------------------------- > > Key: SPARK-1565 > URL: https://issues.apache.org/jira/browse/SPARK-1565 > Project: Spark > Issue Type: Improvement > Components: Examples > Reporter: Patrick Wendell > Assignee: Prashant Sharma > Fix For: 1.0.0 > > > Example: > https://github.com/pwendell/spark/commit/3fdad8ada28c1cef9b3b367993057de22295b8ed > Now that we have the spark-submit script we should make some changes to the > examples: > 1. We should always create a SparkConf that sets the app name and use the > SparkContext constructor that accepts a conf. > 2. We shouldn't set the master based on a command line argument, instead > users can do this on their own with spark-submit. > 3. The examples projects should mark spark-core/streaming as a "provided" > dependency, so that the examples assembly jar does not include spark. > Then users can launch examples like this: > {code} > ./bin/spark-submit > examples/target/scala-2.10/spark-examples-assembly-1.0.0-SNAPSHOT.jar \ > --class org.apache.spark.examples.SparkPi \ > --arg 1000 > {code} -- This message was sent by Atlassian JIRA (v6.2#6252)