Repository: spark Updated Branches: refs/heads/branch-1.1 5ac55c841 -> d01fdd3bb
[SPARK-4745] Fix get_existing_cluster() function with multiple security groups The current get_existing_cluster() function would only find an instance belonged to a cluster if the instance's security groups == cluster_name + "-master" (or "-slaves"). This fix allows for multiple security groups by checking if the cluster_name + "-master" security group is in the list of groups for a particular instance. Author: alexdebrie <alexdebr...@gmail.com> Closes #3596 from alexdebrie/master and squashes the following commits: 9d51232 [alexdebrie] Fix get_existing_cluster() function with multiple security groups (cherry picked from commit 794f3aec24acb578e258532ad0590554d07958ba) Signed-off-by: Josh Rosen <joshro...@databricks.com> Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/d01fdd3b Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/d01fdd3b Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/d01fdd3b Branch: refs/heads/branch-1.1 Commit: d01fdd3bbc8e223895b98e9ef408e28206892c24 Parents: 5ac55c8 Author: alexdebrie <alexdebr...@gmail.com> Authored: Thu Dec 4 14:13:59 2014 -0800 Committer: Josh Rosen <joshro...@databricks.com> Committed: Thu Dec 4 14:16:56 2014 -0800 ---------------------------------------------------------------------- ec2/spark_ec2.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/spark/blob/d01fdd3b/ec2/spark_ec2.py ---------------------------------------------------------------------- diff --git a/ec2/spark_ec2.py b/ec2/spark_ec2.py index 374d3a6..fef95bb 100755 --- a/ec2/spark_ec2.py +++ b/ec2/spark_ec2.py @@ -475,9 +475,9 @@ def get_existing_cluster(conn, opts, cluster_name, die_on_error=True): active = [i for i in res.instances if is_active(i)] for inst in active: group_names = [g.name for g in inst.groups] - if group_names == [cluster_name + "-master"]: + if (cluster_name + "-master") in group_names: master_nodes.append(inst) - elif group_names == [cluster_name + "-slaves"]: + elif (cluster_name + "-slaves") in group_names: slave_nodes.append(inst) if any((master_nodes, slave_nodes)): print ("Found %d master(s), %d slaves" % (len(master_nodes), len(slave_nodes))) --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
