[ https://issues.apache.org/jira/browse/SPARK-3405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Josh Rosen resolved SPARK-3405. ------------------------------- Resolution: Fixed Fix Version/s: 1.3.0 Issue resolved by pull request 2872 [https://github.com/apache/spark/pull/2872] > EC2 cluster creation on VPC > --------------------------- > > Key: SPARK-3405 > URL: https://issues.apache.org/jira/browse/SPARK-3405 > Project: Spark > Issue Type: New Feature > Components: EC2 > Affects Versions: 1.0.2 > Environment: Ubuntu 12.04 > Reporter: Dawson Reid > Priority: Minor > Fix For: 1.3.0 > > > It would be very useful to be able to specify the EC2 VPC in which the Spark > cluster should be created. > When creating a Spark cluster on AWS via the spark-ec2 script there is no way > to specify a VPC id of the VPC you would like the cluster to be created in. > The script always creates the cluster in the default VPC. > In my case I have deleted the default VPC and the spark-ec2 script errors out > with the following : > Setting up security groups... > Creating security group test-master > ERROR:boto:400 Bad Request > ERROR:boto:<?xml version="1.0" encoding="UTF-8"?> > <Response><Errors><Error><Code>VPCIdNotSpecified</Code><Message>No default > VPC for this > user</Message></Error></Errors><RequestID>312a2281-81a1-4d3c-ba10-0593a886779d</RequestID></Response> > Traceback (most recent call last): > File "./spark_ec2.py", line 860, in <module> > main() > File "./spark_ec2.py", line 852, in main > real_main() > File "./spark_ec2.py", line 735, in real_main > conn, opts, cluster_name) > File "./spark_ec2.py", line 247, in launch_cluster > master_group = get_or_make_group(conn, cluster_name + "-master") > File "./spark_ec2.py", line 143, in get_or_make_group > return conn.create_security_group(name, "Spark EC2 group") > File > "/home/dawson/Develop/spark-1.0.2/ec2/third_party/boto-2.4.1.zip/boto-2.4.1/boto/ec2/connection.py", > line 2011, in create_security_group > File > "/home/dawson/Develop/spark-1.0.2/ec2/third_party/boto-2.4.1.zip/boto-2.4.1/boto/connection.py", > line 925, in get_object > boto.exception.EC2ResponseError: EC2ResponseError: 400 Bad Request > <?xml version="1.0" encoding="UTF-8"?> > <Response><Errors><Error><Code>VPCIdNotSpecified</Code><Message>No default > VPC for this > user</Message></Error></Errors><RequestID>312a2281-81a1-4d3c-ba10-0593a886779d</RequestID></Response> -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org