I'm trying to use the spark-ec2 script to launch a spark cluster within a virtual private cloud (VPC) but I don't see an option for that. Is there a way to specify the VPC while using the spark-ec2 script?
I found an old spark-incubator mailing list comment which claims to have added that support, but none of the links from the thread are still active. http://mail-archives.apache.org/mod_mbox/incubator-spark-dev/201402.mbox/%3cgit-pr-620-incubator-sp...@git.apache.org%3E The exact error is ssh 255 while generating the cluster's SSH key on master. The error suggests checking the identity file and key pairs, but I confirmed on another instance that these are correct. Please check that you have provided the correct --identity-file and > --key-pair parameters and try again. Any help?
