Here's answer from Mitch. ---------- Forwarded message ---------- From: Mitch Garnaat <[email protected]> Date: Thu, Jul 7, 2011 at 7:19 AM Subject: Re: AuthorizeSecurityGroupIngress To: Sang-Min Park <[email protected]>
Hi Sang-Min - It is true that you can specify both a group and a cidr on the AuthorizeSecurityGroupIngress request for EC2. However, EC2 silently ignores the cidr block and simply authorizes the group. To prove this, if you do a DescribeGroups request after the authorize request, you will see no mention of the cidr block in the group description. On Thu, Jul 7, 2011 at 7:01 AM, Sang-Min Park <[email protected]>wrote: > It uses soap interface. Euca2ools (boto underneath) uses query interface > and quite popular. > I'll ask our tool master (Mitch who manages boto) if he's aware of this. > > -- SM > > > On Thu, Jul 7, 2011 at 6:52 AM, [email protected] <[email protected]>wrote: > >> On 07/07/11 16:48, Sang-Min Park wrote: >> >>> I tried 'ec2-authorize' in the latest ec2-api-tools, and it doesn't allow >>> the mix. >>> Here's the output: >>> Specify either source groups or source CIDRs, not both. (-h for >>> usage) >>> >>> I could create a rule with multiple groups or a rule with multiple ip >>> range. >>> But I couldn't give -s [cidr] -o [group] option together. >>> >>> -- SM >>> >> >> thanks for trying that - well, that's very interesting indeed. I wonder >> why AWS allows this via the REST API but not via their own command line >> tools? This might be one for investigation for when I next get some idle >> time (e.g. what calls is the ec2-api-tools making? which interface is it >> using etc) >> >> marios >> >> >> >>> >>> On Thu, Jul 7, 2011 at 5:37 AM, >>> [email protected]<mandreou@**redhat.com<[email protected]> >>> >wrote: >>> >>> On 07/07/11 15:02, Sang-Min Park wrote: >>>> >>>> I haven't tried this mix and it was purely speculation from the API. >>>>> >>>>> >>>> ok >>>> >>>> >>>> Here's the specific line in the link: >>>> >>>>> >>>>> "Name of the source security group. Cannot be used when specifying a >>>>> CIDR >>>>> IP >>>>> address." >>>>> "CIDR range. Cannot be used when specifying a source security group." >>>>> >>>>> >>>>> yeah, ok i see now. This was also a source of confusion/frustration >>>> for me >>>> when i was writing the firewalls code. In the end I think this can just >>>> be >>>> taken literally - e.g. for 'name of the source group' : "you cannot use >>>> this >>>> parameter when specifying an IP address, since this refers to groups, >>>> not >>>> addresses". >>>> >>>> >>>> Also, ec2-authorize ( >>>> >>>>> http://docs.amazonwebservices.****com/AWSEC2/2011-01-01/** >>>>> CommandLineReference/index.****html?ApiReference-cmd-** >>>>> AuthorizeSecurityGroupIngress.****html<http://docs.** >>>>> amazonwebservices.com/AWSEC2/**2011-01-01/** >>>>> CommandLineReference/index.**html?ApiReference-cmd-** >>>>> AuthorizeSecurityGroupIngress.**html<http://docs.amazonwebservices.com/AWSEC2/2011-01-01/CommandLineReference/index.html?ApiReference-cmd-AuthorizeSecurityGroupIngress.html> >>>>> > >>>>> >>>>> ) >>>>> doesn't allow the mix. >>>>> >>>>> >>>>> OK, this is interesting. First off, did you try this? What was the >>>> result? >>>> This line: >>>> >>>> " >>>> For EC2 security groups and ingress rules: This command either gives one >>>> or >>>> more CIDR IP address ranges permission to access a security group in >>>> your >>>> account, or it gives one or more security groups ( >>>> " >>>> >>>> seems to suggest what you're saying. However, this line: >>>> >>>> " >>>> Each rule consists of the protocol (e.g., TCP), plus either a CIDR >>>> range, >>>> or a source group (for ingress rules) or destination group (for egress >>>> rules). >>>> " >>>> >>>> Note the use of 'a CIDR range' or 'a source group' - the plural is >>>> dropped; >>>> also, all the examples on that page seem to specify just a single >>>> source, >>>> either an address, or a group. *This* might be the reason that you can >>>> specify address OR ip, since you only specify one. Of course, this is >>>> all >>>> speculation, why don't you try this out and let us know the answer? >>>> >>>> >>>> If you made the call without an error, it's maybe wrong documentation. >>>> >>>>> >>>>> >>>> Of course I made the call - many, many times. I would never submit >>>> something for inclusion without it being tested first, not least because >>>> this code was a contribution to a another project (the appoxy aws gem). >>>> The >>>> question however should be turned on its head - did you make the call, >>>> and >>>> did it cause the error that you are reporting here? >>>> >>>> marios >>>> >>>> >>>> >>>> >>>> -- SM >>>>> >>>>> On Wed, Jul 6, 2011 at 11:23 PM, [email protected]<mandreou@**r** >>>>> edhat.com <http://redhat.com><[email protected]> >>>>> >>>>> wrote: >>>>>> >>>>> >>>>> On 07/07/11 00:16, Sang-Min Park wrote: >>>>> >>>>>> >>>>>> Marios, >>>>>> >>>>>>> >>>>>>> I looked at your aws patch (manage_security_group_******ingress) and >>>>>>> wonder >>>>>>> if you >>>>>>> considered the limitation that either a cidr IP range or user-group >>>>>>> pair >>>>>>> can >>>>>>> be given, not both. >>>>>>> >>>>>>> >>>>>>> why? Or rather, how have you arrived at this conclusion? >>>>>> >>>>>> >>>>>> >>>>>> In other words, >>>>>> >>>>>> >>>>>> source_ip_ranges.each_index do |i| >>>>>>> call_params.merge!({"******IpPermissions.1.IpRanges.#{i+*** >>>>>>> ** >>>>>>> *1}.CidrIp" >>>>>>> => >>>>>>> source_ip_ranges[i].to_s}) >>>>>>> end >>>>>>> source_groups.each_index do |i| >>>>>>> call_params.merge!({"******IpPermissions.1.Groups.#{i+1}.**** >>>>>>> **GroupName" >>>>>>> => >>>>>>> source_groups[i]['group_name']******.to_s, >>>>>>> "IpPermissions.1.Groups.#{i+1}**** >>>>>>> **.UserId"=> >>>>>>> source_groups[i]['owner'].to_******s.gsub(/-/,'')}) >>>>>>> end >>>>>>> >>>>>>> If a user gives both source ip range and groups to the method, it may >>>>>>> end >>>>>>> up >>>>>>> a request like this: >>>>>>> >>>>>>> &IpPermissions.1.IpProtocol=******tcp >>>>>>> &IpPermissions.1.FromPort=80 >>>>>>> &IpPermissions.1.ToPort=80 >>>>>>> &IpPermissions.1.Groups.1.******GroupName=OtherAccountGroup >>>>>>> &IpPermissions.1.Groups.1.******UserId=999988887777 >>>>>>> &IpPermissions.1.IpRanges.1.******CidrIp=192.0.2.0/24 >>>>>>> >>>>>>> >>>>>>> Looks ok to me. Did you try this code and have trouble making a call >>>>>>> >>>>>> with >>>>>> these parameters (or were the effects of making such a call >>>>>> unexpected?) >>>>>> >>>>>> >>>>>> >>>>>> It doesn't look like a valid request according to the AWS definition >>>>>> ( >>>>>> >>>>>> http://docs.amazonwebservices.******com/AWSEC2/2011-01-01/** >>>>>>> APIReference/index.html?******ApiReference-query-** >>>>>>> AuthorizeSecurityGroupIngress.******html<http://docs.** >>>>>>> amazonwebservices.com/AWSEC2/****2011-01-01/APIReference/**index.**<http://amazonwebservices.com/AWSEC2/**2011-01-01/APIReference/index.**> >>>>>>> html?ApiReference-query-****AuthorizeSecurityGroupIngress.****html< >>>>>>> http://docs.**amazonwebservices.com/AWSEC2/** >>>>>>> 2011-01-01/APIReference/index.**html?ApiReference-query-** >>>>>>> AuthorizeSecurityGroupIngress.**html<http://docs.amazonwebservices.com/AWSEC2/2011-01-01/APIReference/index.html?ApiReference-query-AuthorizeSecurityGroupIngress.html> >>>>>>> > >>>>>>> >>>>>>> >>>>>>>> ) >>>>>>> >>>>>>> >>>>>>> Interesting - but am unable to determine which part of the aws >>>>>>> >>>>>> definition >>>>>> makes the above a non-valid request? Can you be a little more specific >>>>>> please? >>>>>> >>>>>> marios >>>>>> >>>>>> >>>>>> Maybe I'm wrong? >>>>>> >>>>>> >>>>>>> -- SM >>>>>>> >>>>>>> >>>>>>> >>>>>>> On Tue, Jun 28, 2011 at 4:24 PM, Sang-Min >>>>>>> Park<sang-min.park@eucalyptus. >>>>>>> *** >>>>>>> *com<sang-min.park@eucalyptus.****com<sang-min.park@**eucalyptus.com<[email protected]> >>>>>>> >> >>>>>>> >>>>>>> >>>>>>> >>>>>>> wrote: >>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> Sorry marios for my late response. What I meant was not that >>>>>>> Eucalyptus >>>>>>> >>>>>>> driver supports firewall now, but the issue is there for me to >>>>>>>> implement >>>>>>>> it. >>>>>>>> Thanks for pointing out that AWS still has the old code. I'll use >>>>>>>> that >>>>>>>> old >>>>>>>> methods to implement firewalls against Eucalyptus. It will be uglier >>>>>>>> than >>>>>>>> your code though (multiple AWS invocation as you imagine). >>>>>>>> >>>>>>>> -- SM >>>>>>>> >>>>>>>> >>>>>>>> On Sun, Jun 26, 2011 at 11:06 PM, [email protected]<mandreou@** >>>>>>>> redhat.com<[email protected]****>>wrote: >>>>>>>> >>>>>>>> Hi Sang-Min, >>>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>>> On 25/06/11 15:49, Sang-Min Park wrote: >>>>>>>>> >>>>>>>>> FYI, I looked at the firewall implementation and found that >>>>>>>>> there's >>>>>>>>> an >>>>>>>>> >>>>>>>>> issue >>>>>>>>>> with Eucalyptus driver. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> I was confused at first by what you meant as looking at Euca >>>>>>>>>> driver >>>>>>>>>> >>>>>>>>> there's no mention of firewalls yet (besides the declaration of >>>>>>>>> 'feature >>>>>>>>> :instances, :firewalls') >>>>>>>>> >>>>>>>>> >>>>>>>>> Eucalyptus supports the old parameter convention in >>>>>>>>> >>>>>>>>> '********AuthorizeSecurityGroupIngress' action. I'll try if I can >>>>>>>>> >>>>>>>>>> patch >>>>>>>>>> AWS to >>>>>>>>>> generate the old parameters as well as the new one. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> OK: you shouldn't need to patch aws at all - my additions to aws >>>>>>>>>> >>>>>>>>> added >>>>>>>>> the >>>>>>>>> new 'manage_security_group_********ingress' method, BUT did not >>>>>>>>> remove >>>>>>>>> any >>>>>>>>> of >>>>>>>>> the old code. I'm not sure if thats what you meant by 'generate the >>>>>>>>> old >>>>>>>>> parameters as well as the new one' but if you mean that your Euca >>>>>>>>> setup >>>>>>>>> (or >>>>>>>>> Euca in general) relies on the old appoxy/aws interface then your >>>>>>>>> existing >>>>>>>>> code should be fine - appoxy/aws gem has not removed any of those >>>>>>>>> earlier >>>>>>>>> methods: >>>>>>>>> >>>>>>>>> 1. authorize_security_group_********named_ingress >>>>>>>>> 2. revoke_security_group_named_********ingress >>>>>>>>> 3. authorize_security_group_IP_********ingress >>>>>>>>> 4. revoke_security_group_IP_********ingress >>>>>>>>> >>>>>>>>> That being said, here's why I implemented the new >>>>>>>>> 'manage_security_groups' >>>>>>>>> method in the appoxy/aws gem: >>>>>>>>> (https://github.com/appoxy/********<https://github.com/appoxy/******> >>>>>>>>> <https://github.com/appoxy/****** <https://github.com/appoxy/****> >>>>>>>>> > >>>>>>>>> >>>>>>>>> aws/pull/91<https://github.****com/appoxy/**aws/pull/91<https** >>>>>>>>> ://github.com/appoxy/**aws/**pull/91<https://github.com/appoxy/**aws/pull/91> >>>>>>>>> > >>>>>>>>> >>>>>>>>>> <http**s://github.<https://**github <https://github>.>** >>>>>>>>>> >>>>>>>>> >>>>>>>>> com/appoxy/aws/pull/91<https:/****/github.com/appoxy/aws/pull/**91<http://github.com/appoxy/aws/pull/91> >>>>>>>>> <https://github.com/appoxy/**aws/pull/91<https://github.com/appoxy/aws/pull/91> >>>>>>>>> > >>>>>>>>> **>>): >>>>>>>>> >>>>>>>>> methods 1/2 above don't allow you to specify fine-grained control >>>>>>>>> over >>>>>>>>> group >>>>>>>>> access - i.e. you can specify which groups to authorize, but not >>>>>>>>> which >>>>>>>>> protocols/ports to allow for those groups. Also, 3/4 only allow you >>>>>>>>> to >>>>>>>>> specify a single IP range at a time - thus if a given firewall rule >>>>>>>>> has >>>>>>>>> a >>>>>>>>> large number of address ranges then this operation will need to be >>>>>>>>> done >>>>>>>>> for >>>>>>>>> each of those. Similarly, you can't specify both groups AND IP >>>>>>>>> addresses >>>>>>>>> in >>>>>>>>> a single call (thus defining an entire firewall rule with a single >>>>>>>>> call). >>>>>>>>> >>>>>>>>> The earlier implementations of appoxy/aws were based on the 2009 >>>>>>>>> version >>>>>>>>> of AWS API http://docs.amazonwebservices.**** >>>>>>>>> ****com/AWSEC2/2009-07-15/** >>>>>>>>> APIReference/ApiReference-********query-****** >>>>>>>>> AuthorizeSecurityGroupIngress.***** >>>>>>>>> ***html<http://docs.****amazonwe**bservices.com/**AWSEC2/**<http://bservices.com/AWSEC2/**> >>>>>>>>> <http://**amazonwebservices.com/AWSEC2/****<http://amazonwebservices.com/AWSEC2/**> >>>>>>>>> > >>>>>>>>> >>>>>>>>> 2009-07-15/APIReference/******ApiReference-query-** >>>>>>>>> AuthorizeSecurityGroupIngress.******html<http://docs.** >>>>>>>>> amazonwebservices.com/AWSEC2/****2009-07-15/APIReference/**<http://amazonwebservices.com/AWSEC2/**2009-07-15/APIReference/**> >>>>>>>>> ApiReference-query-****AuthorizeSecurityGroupIngress.****html< >>>>>>>>> http://docs.**amazonwebservices.com/AWSEC2/** >>>>>>>>> 2009-07-15/APIReference/**ApiReference-query-** >>>>>>>>> AuthorizeSecurityGroupIngress.**html<http://docs.amazonwebservices.com/AWSEC2/2009-07-15/APIReference/ApiReference-query-AuthorizeSecurityGroupIngress.html> >>>>>>>>> > >>>>>>>>> >>>>>>>>>> . >>>>>>>>>>> >>>>>>>>>> In the latest version of API, you can specify a number of IP >>>>>>>>> addresses, or >>>>>>>>> >>>>>>>>> groups, or mix of both, for which the specified rule will apply. >>>>>>>>> You >>>>>>>>> can >>>>>>>>> now >>>>>>>>> also specify 'from_port' 'to_port' and 'protocol' for ingress >>>>>>>>> groups >>>>>>>>> in >>>>>>>>> a >>>>>>>>> rule, >>>>>>>>> >>>>>>>>> marios >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> Sang-min >>>>>>>>> >>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Fri, Jun 17, 2011 at 8:06 AM,<[email protected]> wrote: >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> This patch uses the new 'Firewalls' collection (I pushed that to >>>>>>>>>> trunk >>>>>>>>>> >>>>>>>>>> today). >>>>>>>>>>> The create_instance operation for the ec2 driver takes an array >>>>>>>>>>> of >>>>>>>>>>> firewall >>>>>>>>>>> names >>>>>>>>>>> for the instance to be 'launched into'. Patch includes: >>>>>>>>>>> >>>>>>>>>>> * necessary modifications to server.rb >>>>>>>>>>> * addition of 'firewalls' to the Instance model >>>>>>>>>>> * modification of the haml views: html for the create operation, >>>>>>>>>>> html/xml >>>>>>>>>>> for showing >>>>>>>>>>> firewalls when inspecting a given instance. >>>>>>>>>>> >>>>>>>>>>> If you aren't using the html interface to create an instance, you >>>>>>>>>>> can >>>>>>>>>>> specify >>>>>>>>>>> firewalls using form input : 'firewall#=name' where '#' is any >>>>>>>>>>> digit. >>>>>>>>>>> For >>>>>>>>>>> example: >>>>>>>>>>> >>>>>>>>>>> curl -F 'image_id=ami-48aa4921' -F 'firewalls1=default' -F >>>>>>>>>>> 'firewalls2=test' >>>>>>>>>>> --user 'ec2_key:ec2_password' >>>>>>>>>>> http://localhost:3001/api/********instances?format=xml<http://localhost:3001/api/******instances?format=xml> >>>>>>>>>>> <http://**localhost:3001/api/******instances?format=xml<http://localhost:3001/api/****instances?format=xml> >>>>>>>>>>> > >>>>>>>>>>> <http://**localhost:3001/api/******instances?format=xml<http:/** >>>>>>>>>>> /localhost:3001/api/****instances?format=xml<http://localhost:3001/api/**instances?format=xml> >>>>>>>>>>> > >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> <http://**localhost:3001/api/****instances?**format=xml<http:/ >>>>>>>>>>> **/** >>>>>>>>>>> localhost:3001/api/instances?****format=xml<http://localhost:** >>>>>>>>>>> 3001/api/instances?format=xml<http://localhost:3001/api/instances?format=xml> >>>>>>>>>>> > >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> will create an instance from ami-48aa4921 and place it into >>>>>>>>>>> firewalls >>>>>>>>>>> 'default' >>>>>>>>>>> and 'test'. EC2 does not support 'moving' an instance between >>>>>>>>>>> firewalls >>>>>>>>>>> once it's >>>>>>>>>>> launched so this functionality was not implemented >>>>>>>>>>> (http://aws.amazon.com/********articles/1145?_encoding=UTF8&*** >>>>>>>>>>> *****<http://aws.amazon.com/******articles/1145?_encoding=UTF8&******> >>>>>>>>>>> <http://aws.amazon.com/******articles/1145?_encoding=**UTF8&****<http://aws.amazon.com/****articles/1145?_encoding=UTF8&****> >>>>>>>>>>> > >>>>>>>>>>> >>>>>>>>>>> <http://aws.amazon.com/******articles/1145?_encoding=UTF8&******<http://aws.amazon.com/****articles/1145?_encoding=UTF8&****> >>>>>>>>>>> <http://aws.amazon.com/****articles/1145?_encoding=UTF8&****<http://aws.amazon.com/**articles/1145?_encoding=UTF8&**> >>>>>>>>>>> > >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> jiveRedirect=1#13<http://aws.******amazon.com/articles/1145?_*** >>>>>>>>>>> * <http://amazon.com/articles/1145?_**> >>>>>>>>>>> encoding=UTF8&jiveRedirect=1#******13<http://aws.amazon.com/** >>>>>>>>>>> articles/1145?_encoding=UTF8&****jiveRedirect=1#13<http://aws.** >>>>>>>>>>> amazon.com/articles/1145?_**encoding=UTF8&jiveRedirect=1#**13<http://aws.amazon.com/articles/1145?_encoding=UTF8&jiveRedirect=1#13> >>>>>>>>>>> > >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> ) >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> marios >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>>> >>>>>> >>>>> >>>>> >>>> >>> >>> >> > > > -- > > ---------------------------------------------------- > Sang-Min Park > Engineer > Eucalyptus Systems > > -- ---------------------------------------------------- Sang-Min Park Engineer Eucalyptus Systems
