improved by removing unnecessary
> > copies 3. We could make less frequently used modules like Tachyon,
> > persistent
> hdfs
> > not a part of the default setup.
> >
> > [1] https://github.com/mesos/spark-ec2/blob/v3/copy-dir.sh#L42
> >
> > Thanks
>
You are partially correct.
It's not terribly complex, but also not easy to accomplish. Sounds like you
want to manage some partially/fully baked AMI's with the core spark libs and
dependencies already on the image. Main issues that crop up are:
1) image sprawl, as libs/config/defaults/etc cha
