Thanks a lot for reporting this. I think we just missed installing xfsprogs on the AMI. I have a fix for this at https://github.com/mesos/spark-ec2/pull/59.
After the pull request is merged, any new clusters launched should have mkfs.xfs Thanks Shivaram On Fri, Jul 18, 2014 at 4:56 PM, Ben Horner <ben.hor...@atigeo.com> wrote: > Hello all, > > There is a bug in the spark-ec2 script (perhaps due to a change in the > Amazon AMI). > > The --ebs-vol-size option directs the spark-ec2 script to add an EBS volume > of the specified size, and mount it at /vol for a persistent HDFS. To do > this, it uses mkfs.xfs which is not available (though mkfs is). > > To work around this, I was able to run "yum install xfsprogs" on the master > and each slave, and then use the --resume option with the script, and the > persistent HDFS actually worked! > > This has been a frustrating experience, but I've used the spark-ec2 script > for several months now, and it's incredibly helpful. I hope this post > helps > towards fixing the problem! > > Thanks, > -Ben > > P.S. This is the full initial command I used, in case this is isolated to > particular instance types or anything: > ec2/spark-ec2 -k ... -i ... -z us-east-1d -s 4 -t m3.2xlarge > --ebs-vol-size=250 -m r3.2xlarge launch ... > > P.P.S. Ganglia is still broken, and has been for a while... > > > > -- > View this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/BUG-in-spark-ec2-script-ebs-vol-size-and-workaround-tp10217.html > Sent from the Apache Spark User List mailing list archive at Nabble.com. >
