[
https://issues.apache.org/jira/browse/SPARK-5627?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nicholas Chammas updated SPARK-5627:
------------------------------------
Description:
There are some cases where users may want to programmatically invoke
{{spark-ec2}} to manage clusters. For example, we might want to
programmatically launch clusters as part of a {{spark-perf}} run and then
destroy them once the performance testing is done.
We should support some of these use cases, perhaps with the explicit caveat
that we are not offering API stability for this access.
A good way to offer this might be to follow [the Packer
model|https://www.packer.io/docs/command-line/machine-readable.html] and add a
{{--machine-readable}} option to spark-ec2.
It would be a lot of work to support such an option for everything that
spark-ec2 does, and it probably isn't relevant for most things anyway.
Still, we can phase it in for select things like [returning the
version|SPARK-5628] and [describing clusters|SPARK-5629].
was:
There are some cases where users may want to programmatically invoke
{{spark-ec2}} to manage clusters. For example, we might want to
programmatically launch clusters as part of a {{spark-perf}} run and then
destroy them once the performance testing is done.
We should support some of these use cases, perhaps with the explicit caveat
that we are not offering API stability for this access.
> Enhance spark-ec2 for some programmatic use cases
> -------------------------------------------------
>
> Key: SPARK-5627
> URL: https://issues.apache.org/jira/browse/SPARK-5627
> Project: Spark
> Issue Type: Improvement
> Components: EC2
> Reporter: Nicholas Chammas
> Priority: Minor
>
> There are some cases where users may want to programmatically invoke
> {{spark-ec2}} to manage clusters. For example, we might want to
> programmatically launch clusters as part of a {{spark-perf}} run and then
> destroy them once the performance testing is done.
> We should support some of these use cases, perhaps with the explicit caveat
> that we are not offering API stability for this access.
> A good way to offer this might be to follow [the Packer
> model|https://www.packer.io/docs/command-line/machine-readable.html] and add
> a {{--machine-readable}} option to spark-ec2.
> It would be a lot of work to support such an option for everything that
> spark-ec2 does, and it probably isn't relevant for most things anyway.
> Still, we can phase it in for select things like [returning the
> version|SPARK-5628] and [describing clusters|SPARK-5629].
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
