[
https://issues.apache.org/jira/browse/SPARK-5637?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14321811#comment-14321811
]
Nicholas Chammas commented on SPARK-5637:
-----------------------------------------
OK, so I watched [the intro video for
StarCluster|https://www.youtube.com/watch?v=vC3lJcPq1FY] and it seems like a
neat tool.
In fact, it seems very similar to [Ubuntu Juju|https://juju.ubuntu.com/],
though less general. Speaking of which, there already seems to be a [Juju Charm
for launching Spark
clusters|http://manage.jujucharms.com/~asanjar/trusty/spark], which would
probably offer a similar UX to what I saw in the StartCluster video.
That all said, I'm not sure I see the value in changing {{spark-ec2}} to be a
plugin to StarCluster. It would probably be a lot of work (or am I mistaken
about that?), and it would tie us to a project external to Spark. Furthermore,
we already have a very functional and fully featured tool in {{spark-ec2}} and
I don't see a big return on investing in StarCluster integration. The only
thing I saw in the video that made me think "Oh, we should have that in
spark-ec2!" is the idea of persisting configs to a file, which we are tracking
in [SPARK-925].
Finally, and most importantly, I don't see why we can't just leave this to the
community to implement independently. Just as some people have already created
[Juju Charms for
Spark|http://manage.jujucharms.com/search?search_text=spark&op=], wouldn't it
be better if we left the community to develop a plugin for StarCluster?
If that were to happen and the plugin gained some traction, it would be a lot
easier to reconsider integrating it and officially supporting it via
{{spark-ec2}}. For one, it would be easier to judge the maintenance burden and
project fit of taking that on.
Until then, this doesn't seem like something we should take on.
Does that seem reasonable, [~agrothberg]?
> Expose spark_ec2 as as StarCluster Plugin
> -----------------------------------------
>
> Key: SPARK-5637
> URL: https://issues.apache.org/jira/browse/SPARK-5637
> Project: Spark
> Issue Type: Improvement
> Components: EC2
> Reporter: Alex Rothberg
> Priority: Minor
>
> Starcluster has a lot features in place for stating EC2 instances and it
> would be great to have an option to leverage that as a plugin.
> See: http://star.mit.edu/cluster/docs/latest/index.html
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
