Re: Trouble launching EC2 Cluster with Spark

2014-06-04 Thread Sam Taylor Steyer 
Thanks you! The regions advice solved the problem for my friend who was getting 
the key pair does not exist problem. I am still getting the error:

ERROR:boto:400 Bad Request
ERROR:boto:?xml version=1.0 encoding=UTF-8?
ResponseErrorsErrorCodeInvalidParameterValue/CodeMessageInvalid 
value 'null' for protocol. VPC security group rules must specify protocols 
explicitly./Message/Error/ErrorsRequestID7ff92687-b95a-4a39-94cb-e2d00a6928fd/RequestID/Response

This sounds like it could have to do with the access settings of the security 
group, but I don't know how to change. Any advice would be much appreciated!

Sam

- Original Message -
From: Krishna Sankar ksanka...@gmail.com
To: user@spark.apache.org
Sent: Wednesday, June 4, 2014 8:52:59 AM
Subject: Re: Trouble launching EC2 Cluster with Spark

One reason could be that the keys are in a different region. Need to create
the keys in us-east-1-North Virginia.
Cheers
k/


On Wed, Jun 4, 2014 at 7:45 AM, Sam Taylor Steyer sste...@stanford.edu
wrote:

 Hi,

 I am trying to launch an EC2 cluster from spark using the following
 command:

 ./spark-ec2 -k HackerPair -i [path]/HackerPair.pem -s 2 launch
 HackerCluster

 I set my access key id and secret access key. I have been getting an error
 in the setting up security groups... phase:

 Invalid value 'null' for protocol. VPC security groups must specify
 protocols explicitly.

 My project partner gets one step further and then gets the error

 The key pair 'JamesAndSamTest' does not exist.

 Any thoughts as to how we could fix these problems? Thanks a lot!
 Sam

Re: Trouble launching EC2 Cluster with Spark

2014-06-04 Thread Krishna Sankar 
chmod 600 path/FinalKey.pem

Cheers

k/


On Wed, Jun 4, 2014 at 12:49 PM, Sam Taylor Steyer sste...@stanford.edu
wrote:

 Also, once my friend logged in to his cluster he received the error
 Permissions 0644 for 'FinalKey.pem' are too open. This sounds like the
 other problem described. How do we make the permissions more private?

 Thanks very much,
 Sam

 - Original Message -
 From: Sam Taylor Steyer sste...@stanford.edu
 To: user@spark.apache.org
 Sent: Wednesday, June 4, 2014 12:42:04 PM
 Subject: Re: Trouble launching EC2 Cluster with Spark

 Thanks you! The regions advice solved the problem for my friend who was
 getting the key pair does not exist problem. I am still getting the error:

 ERROR:boto:400 Bad Request
 ERROR:boto:?xml version=1.0 encoding=UTF-8?
 ResponseErrorsErrorCodeInvalidParameterValue/CodeMessageInvalid
 value 'null' for protocol. VPC security group rules must specify protocols
 explicitly./Message/Error/ErrorsRequestID7ff92687-b95a-4a39-94cb-e2d00a6928fd/RequestID/Response

 This sounds like it could have to do with the access settings of the
 security group, but I don't know how to change. Any advice would be much
 appreciated!

 Sam

 - Original Message -
 From: Krishna Sankar ksanka...@gmail.com
 To: user@spark.apache.org
 Sent: Wednesday, June 4, 2014 8:52:59 AM
 Subject: Re: Trouble launching EC2 Cluster with Spark

 One reason could be that the keys are in a different region. Need to create
 the keys in us-east-1-North Virginia.
 Cheers
 k/


 On Wed, Jun 4, 2014 at 7:45 AM, Sam Taylor Steyer sste...@stanford.edu
 wrote:

  Hi,
 
  I am trying to launch an EC2 cluster from spark using the following
  command:
 
  ./spark-ec2 -k HackerPair -i [path]/HackerPair.pem -s 2 launch
  HackerCluster
 
  I set my access key id and secret access key. I have been getting an
 error
  in the setting up security groups... phase:
 
  Invalid value 'null' for protocol. VPC security groups must specify
  protocols explicitly.
 
  My project partner gets one step further and then gets the error
 
  The key pair 'JamesAndSamTest' does not exist.
 
  Any thoughts as to how we could fix these problems? Thanks a lot!
  Sam

Re: Trouble launching EC2 Cluster with Spark

2014-06-04 Thread Sam Taylor Steyer 
Awesome, that worked. Thank you!

- Original Message -
From: Krishna Sankar ksanka...@gmail.com
To: user@spark.apache.org
Sent: Wednesday, June 4, 2014 12:52:00 PM
Subject: Re: Trouble launching EC2 Cluster with Spark

chmod 600 path/FinalKey.pem

Cheers

k/


On Wed, Jun 4, 2014 at 12:49 PM, Sam Taylor Steyer sste...@stanford.edu
wrote:

 Also, once my friend logged in to his cluster he received the error
 Permissions 0644 for 'FinalKey.pem' are too open. This sounds like the
 other problem described. How do we make the permissions more private?

 Thanks very much,
 Sam

 - Original Message -
 From: Sam Taylor Steyer sste...@stanford.edu
 To: user@spark.apache.org
 Sent: Wednesday, June 4, 2014 12:42:04 PM
 Subject: Re: Trouble launching EC2 Cluster with Spark

 Thanks you! The regions advice solved the problem for my friend who was
 getting the key pair does not exist problem. I am still getting the error:

 ERROR:boto:400 Bad Request
 ERROR:boto:?xml version=1.0 encoding=UTF-8?
 ResponseErrorsErrorCodeInvalidParameterValue/CodeMessageInvalid
 value 'null' for protocol. VPC security group rules must specify protocols
 explicitly./Message/Error/ErrorsRequestID7ff92687-b95a-4a39-94cb-e2d00a6928fd/RequestID/Response

 This sounds like it could have to do with the access settings of the
 security group, but I don't know how to change. Any advice would be much
 appreciated!

 Sam

 - Original Message -
 From: Krishna Sankar ksanka...@gmail.com
 To: user@spark.apache.org
 Sent: Wednesday, June 4, 2014 8:52:59 AM
 Subject: Re: Trouble launching EC2 Cluster with Spark

 One reason could be that the keys are in a different region. Need to create
 the keys in us-east-1-North Virginia.
 Cheers
 k/


 On Wed, Jun 4, 2014 at 7:45 AM, Sam Taylor Steyer sste...@stanford.edu
 wrote:

  Hi,
 
  I am trying to launch an EC2 cluster from spark using the following
  command:
 
  ./spark-ec2 -k HackerPair -i [path]/HackerPair.pem -s 2 launch
  HackerCluster
 
  I set my access key id and secret access key. I have been getting an
 error
  in the setting up security groups... phase:
 
  Invalid value 'null' for protocol. VPC security groups must specify
  protocols explicitly.
 
  My project partner gets one step further and then gets the error
 
  The key pair 'JamesAndSamTest' does not exist.
 
  Any thoughts as to how we could fix these problems? Thanks a lot!
  Sam

Re: Trouble launching EC2 Cluster with Spark

2014-06-04 Thread Patrick Wendell 
Hey Sam,

You mentioned two problems here, did your VPC error message get fixed
or only the key permissions problem?

I noticed we had some report a similar issue with the VPC stuff a long
time back (but there is no real resolution here):
https://spark-project.atlassian.net/browse/SPARK-1166

If that's still an issue, one thing to try is just changing the name
of the cluster. We create groups that are identified with the cluster
name, and there might be something that just got screwed up with the
original group creation and AWS isn't happy.

- Patrick



On Wed, Jun 4, 2014 at 12:55 PM, Sam Taylor Steyer sste...@stanford.edu wrote:
 Awesome, that worked. Thank you!

 - Original Message -
 From: Krishna Sankar ksanka...@gmail.com
 To: user@spark.apache.org
 Sent: Wednesday, June 4, 2014 12:52:00 PM
 Subject: Re: Trouble launching EC2 Cluster with Spark

 chmod 600 path/FinalKey.pem

 Cheers

 k/


 On Wed, Jun 4, 2014 at 12:49 PM, Sam Taylor Steyer sste...@stanford.edu
 wrote:

 Also, once my friend logged in to his cluster he received the error
 Permissions 0644 for 'FinalKey.pem' are too open. This sounds like the
 other problem described. How do we make the permissions more private?

 Thanks very much,
 Sam

 - Original Message -
 From: Sam Taylor Steyer sste...@stanford.edu
 To: user@spark.apache.org
 Sent: Wednesday, June 4, 2014 12:42:04 PM
 Subject: Re: Trouble launching EC2 Cluster with Spark

 Thanks you! The regions advice solved the problem for my friend who was
 getting the key pair does not exist problem. I am still getting the error:

 ERROR:boto:400 Bad Request
 ERROR:boto:?xml version=1.0 encoding=UTF-8?
 ResponseErrorsErrorCodeInvalidParameterValue/CodeMessageInvalid
 value 'null' for protocol. VPC security group rules must specify protocols
 explicitly./Message/Error/ErrorsRequestID7ff92687-b95a-4a39-94cb-e2d00a6928fd/RequestID/Response

 This sounds like it could have to do with the access settings of the
 security group, but I don't know how to change. Any advice would be much
 appreciated!

 Sam

 - Original Message -
 From: Krishna Sankar ksanka...@gmail.com
 To: user@spark.apache.org
 Sent: Wednesday, June 4, 2014 8:52:59 AM
 Subject: Re: Trouble launching EC2 Cluster with Spark

 One reason could be that the keys are in a different region. Need to create
 the keys in us-east-1-North Virginia.
 Cheers
 k/


 On Wed, Jun 4, 2014 at 7:45 AM, Sam Taylor Steyer sste...@stanford.edu
 wrote:

  Hi,
 
  I am trying to launch an EC2 cluster from spark using the following
  command:
 
  ./spark-ec2 -k HackerPair -i [path]/HackerPair.pem -s 2 launch
  HackerCluster
 
  I set my access key id and secret access key. I have been getting an
 error
  in the setting up security groups... phase:
 
  Invalid value 'null' for protocol. VPC security groups must specify
  protocols explicitly.
 
  My project partner gets one step further and then gets the error
 
  The key pair 'JamesAndSamTest' does not exist.
 
  Any thoughts as to how we could fix these problems? Thanks a lot!
  Sam