Repository: spark Updated Branches: refs/heads/branch-1.4 51d98b0e9 -> d014a447a
[SPARK-3674] YARN support in Spark EC2 This corresponds to https://github.com/mesos/spark-ec2/pull/116 in the spark-ec2 repo. The only changes required on the spark_ec2.py script is to open the RM port. cc andrewor14 Author: Shivaram Venkataraman <shiva...@cs.berkeley.edu> Closes #6376 from shivaram/spark-ec2-yarn and squashes the following commits: 961504a [Shivaram Venkataraman] Merge branch 'master' of https://github.com/apache/spark into spark-ec2-yarn 152c94c [Shivaram Venkataraman] Open 8088 for YARN in EC2 (cherry picked from commit 2e9a5f229e1a2ccffa74fa59fa6a55b2704d9c1a) Signed-off-by: Andrew Or <and...@databricks.com> Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/d014a447 Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/d014a447 Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/d014a447 Branch: refs/heads/branch-1.4 Commit: d014a447a361f65db8986e839e2f9a7a7a356f57 Parents: 51d98b0 Author: Shivaram Venkataraman <shiva...@cs.berkeley.edu> Authored: Tue May 26 15:01:27 2015 -0700 Committer: Andrew Or <and...@databricks.com> Committed: Tue May 26 15:01:44 2015 -0700 ---------------------------------------------------------------------- ec2/spark_ec2.py | 2 ++ 1 file changed, 2 insertions(+) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/spark/blob/d014a447/ec2/spark_ec2.py ---------------------------------------------------------------------- diff --git a/ec2/spark_ec2.py b/ec2/spark_ec2.py index 2fab5c4..2154046 100755 --- a/ec2/spark_ec2.py +++ b/ec2/spark_ec2.py @@ -492,6 +492,8 @@ def launch_cluster(conn, opts, cluster_name): master_group.authorize('udp', 2049, 2049, authorized_address) master_group.authorize('tcp', 4242, 4242, authorized_address) master_group.authorize('udp', 4242, 4242, authorized_address) + # RM in YARN mode uses 8088 + master_group.authorize('tcp', 8088, 8088, authorized_address) if opts.ganglia: master_group.authorize('tcp', 5080, 5080, authorized_address) if slave_group.rules == []: # Group was just now created --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
