[openstack-dev] [neutron][security-groups] Neutron default security groups
Folks, I have had discussions with some folks individually about this but I would like bring this to a broader audience. I have been playing with security groups and I see the notion of 'default' security group seems to create some nuisance/issues. There are list of things I have noticed so far: - Tenant for OpenStack services (normally named service/services) also ends up having default security group. - Port create operation ends up ensuring default security groups for all the tenants as this completely seems out of the context of the tenant the port operation takes place. (bug?) - Race conditions where if system is stressed and Neutron tries to ensure the first default security group and in parallel another call comes, Neutron ends up trying to create multiple default security groups as the checks for duplicate groups are invalidated as soon as the call make past a certain point in code. - API performance where orchestration chooses to spawn 1000 tenants and we see unnecessary overhead. - For plugins that use RESTful proxy backends require the backend systems to be up at the time neutron starts. [Minor, but affects some packaging solutions] To summarize, is there a way to disable default security groups? Expected answer is no; can we introduce a way to disable it? If that does not make sense, should we go ahead and fix the issues around it? I am sure some of you must have seen some of these issues and solved them already. Please do share how do tackle these issues? Thanks, Fawad Khaliq ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [neutron][security-groups] Neutron default security groups
The similar problem has been discussed before. There is no definitive answer, and currently seems we cannot simply disable it since G version. However, we can add some ALLOW rules to bypass the rules inside the iptables chains. Hope there be more flexibility to controller the security groups in the future release. On Tue, Sep 16, 2014 at 4:00 PM, Fawad Khaliq wrote: > Folks, > > I have had discussions with some folks individually about this but I would > like bring this to a broader audience. > > I have been playing with security groups and I see the notion of 'default' > security group seems to create some nuisance/issues. > > There are list of things I have noticed so far: > >- Tenant for OpenStack services (normally named service/services) also >ends up having default security group. >- Port create operation ends up ensuring default security groups for >all the tenants as this completely seems out of the context of the tenant >the port operation takes place. (bug?) >- Race conditions where if system is stressed and Neutron tries to >ensure the first default security group and in parallel another call comes, >Neutron ends up trying to create multiple default security groups as the >checks for duplicate groups are invalidated as soon as the call make past a >certain point in code. >- API performance where orchestration chooses to spawn 1000 tenants >and we see unnecessary overhead. >- For plugins that use RESTful proxy backends require the backend >systems to be up at the time neutron starts. [Minor, but affects some >packaging solutions] > > > To summarize, is there a way to disable default security groups? Expected > answer is no; can we introduce a way to disable it? If that does not make > sense, should we go ahead and fix the issues around it? > > I am sure some of you must have seen some of these issues and solved them > already. Please do share how do tackle these issues? > > Thanks, > Fawad Khaliq > > > ___ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > -- Best wishes! Baohua ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [neutron][security-groups] Neutron default security groups
Hi Boahua, Thanks for sharing your thoughts. The issues seen are not related to "access", they are all related to API layer, so having ALLOW all etc does not fix/workaround the problems I mentioned. Please do share if you have something more to add. Fawad Khaliq On Tue, Sep 16, 2014 at 7:28 PM, Baohua Yang wrote: > The similar problem has been discussed before. > There is no definitive answer, and currently seems we cannot simply > disable it since G version. > However, we can add some ALLOW rules to bypass the rules inside the > iptables chains. > Hope there be more flexibility to controller the security groups in the > future release. > > > On Tue, Sep 16, 2014 at 4:00 PM, Fawad Khaliq wrote: > >> Folks, >> >> I have had discussions with some folks individually about this but I >> would like bring this to a broader audience. >> >> I have been playing with security groups and I see the notion of >> 'default' security group seems to create some nuisance/issues. >> >> There are list of things I have noticed so far: >> >>- Tenant for OpenStack services (normally named service/services) >>also ends up having default security group. >>- Port create operation ends up ensuring default security groups for >>all the tenants as this completely seems out of the context of the tenant >>the port operation takes place. (bug?) >>- Race conditions where if system is stressed and Neutron tries to >>ensure the first default security group and in parallel another call >> comes, >>Neutron ends up trying to create multiple default security groups as the >>checks for duplicate groups are invalidated as soon as the call make past >> a >>certain point in code. >>- API performance where orchestration chooses to spawn 1000 tenants >>and we see unnecessary overhead. >>- For plugins that use RESTful proxy backends require the backend >>systems to be up at the time neutron starts. [Minor, but affects some >>packaging solutions] >> >> >> To summarize, is there a way to disable default security groups? Expected >> answer is no; can we introduce a way to disable it? If that does not make >> sense, should we go ahead and fix the issues around it? >> >> I am sure some of you must have seen some of these issues and solved them >> already. Please do share how do tackle these issues? >> >> Thanks, >> Fawad Khaliq >> >> >> ___ >> OpenStack-dev mailing list >> OpenStack-dev@lists.openstack.org >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> >> > > > -- > Best wishes! > Baohua > > ___ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [neutron][security-groups] Neutron default security groups
Hi fawad Yes, you're right. I mentioned that not to answer the exact question, but think to drop some line around it. I do hope we can provide the capacity in the API layer, and let the security group become more intuitive for users. On Tue, Sep 16, 2014 at 10:45 PM, Fawad Khaliq wrote: > Hi Boahua, > > Thanks for sharing your thoughts. The issues seen are not related to > "access", they are all related to API layer, so having ALLOW all etc does > not fix/workaround the problems I mentioned. > > Please do share if you have something more to add. > > Fawad Khaliq > > On Tue, Sep 16, 2014 at 7:28 PM, Baohua Yang wrote: > >> The similar problem has been discussed before. >> There is no definitive answer, and currently seems we cannot simply >> disable it since G version. >> However, we can add some ALLOW rules to bypass the rules inside the >> iptables chains. >> Hope there be more flexibility to controller the security groups in the >> future release. >> >> >> On Tue, Sep 16, 2014 at 4:00 PM, Fawad Khaliq wrote: >> >>> Folks, >>> >>> I have had discussions with some folks individually about this but I >>> would like bring this to a broader audience. >>> >>> I have been playing with security groups and I see the notion of >>> 'default' security group seems to create some nuisance/issues. >>> >>> There are list of things I have noticed so far: >>> >>>- Tenant for OpenStack services (normally named service/services) >>>also ends up having default security group. >>>- Port create operation ends up ensuring default security groups for >>>all the tenants as this completely seems out of the context of the tenant >>>the port operation takes place. (bug?) >>>- Race conditions where if system is stressed and Neutron tries to >>>ensure the first default security group and in parallel another call >>> comes, >>>Neutron ends up trying to create multiple default security groups as the >>>checks for duplicate groups are invalidated as soon as the call make >>> past a >>>certain point in code. >>>- API performance where orchestration chooses to spawn 1000 tenants >>>and we see unnecessary overhead. >>>- For plugins that use RESTful proxy backends require the backend >>>systems to be up at the time neutron starts. [Minor, but affects some >>>packaging solutions] >>> >>> >>> To summarize, is there a way to disable default security groups? >>> Expected answer is no; can we introduce a way to disable it? If that does >>> not make sense, should we go ahead and fix the issues around it? >>> >>> I am sure some of you must have seen some of these issues and solved >>> them already. Please do share how do tackle these issues? >>> >>> Thanks, >>> Fawad Khaliq >>> >>> >>> ___ >>> OpenStack-dev mailing list >>> OpenStack-dev@lists.openstack.org >>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >>> >>> >> >> >> -- >> Best wishes! >> Baohua >> >> ___ >> OpenStack-dev mailing list >> OpenStack-dev@lists.openstack.org >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> >> > > ___ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > -- Best wishes! Baohua ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [neutron][security-groups] Neutron default security groups
Hi, Inline: On Tue, Sep 16, 2014 at 1:00 AM, Fawad Khaliq wrote: > Folks, > > I have had discussions with some folks individually about this but I would > like bring this to a broader audience. > > I have been playing with security groups and I see the notion of 'default' > security group seems to create some nuisance/issues. > > There are list of things I have noticed so far: > >- Tenant for OpenStack services (normally named service/services) also >ends up having default security group. >- Port create operation ends up ensuring default security groups for >all the tenants as this completely seems out of the context of the tenant >the port operation takes place. (bug?) >- Race conditions where if system is stressed and Neutron tries to >ensure the first default security group and in parallel another call comes, >Neutron ends up trying to create multiple default security groups as the >checks for duplicate groups are invalidated as soon as the call make past a >certain point in code. > > Right this is a bug. We should catch this foreign key constraint exception here and pass as the default security group was already created. We do something similar here when ports are created as there is a similar race for ip_allocation. > >- >- API performance where orchestration chooses to spawn 1000 tenants >and we see unnecessary overhead > > >- For plugins that use RESTful proxy backends require the backend >systems to be up at the time neutron starts. [Minor, but affects some >packaging solutions] > > This is probably always a requirement for neutron to work so I don't think it's related. > > To summarize, is there a way to disable default security groups? Expected > answer is no; can we introduce a way to disable it? If that does not make > sense, should we go ahead and fix the issues around it? > I think we should fix these bugs you pointed out. > > I am sure some of you must have seen some of these issues and solved them > already. Please do share how do tackle these issues? > > Thanks, > Fawad Khaliq > > > ___ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [neutron][security-groups] Neutron default security groups
As I know there is no a way to disable default security groups, but I think this BP can solve this problem: https://blueprints.launchpad.net/neutron/+spec/default-rules-for-default-security-group 在 2014-09-17 07:44:42，"Aaron Rosen" 写道： Hi, Inline: On Tue, Sep 16, 2014 at 1:00 AM, Fawad Khaliq wrote: Folks, I have had discussions with some folks individually about this but I would like bring this to a broader audience. I have been playing with security groups and I see the notion of 'default' security group seems to create some nuisance/issues. There are list of things I have noticed so far: Tenant for OpenStack services (normally named service/services) also ends up having default security group. Port create operation ends up ensuring default security groups for all the tenants as this completely seems out of the context of the tenant the port operation takes place. (bug?) Race conditions where if system is stressed and Neutron tries to ensure the first default security group and in parallel another call comes, Neutron ends up trying to create multiple default security groups as the checks for duplicate groups are invalidated as soon as the call make past a certain point in code. Right this is a bug. We should catch this foreign key constraint exception here and pass as the default security group was already created. We do something similar here when ports are created as there is a similar race for ip_allocation. API performance where orchestration chooses to spawn 1000 tenants and we see unnecessary overhead For plugins that use RESTful proxy backends require the backend systems to be up at the time neutron starts. [Minor, but affects some packaging solutions] This is probably always a requirement for neutron to work so I don't think it's related. To summarize, is there a way to disable default security groups? Expected answer is no; can we introduce a way to disable it? If that does not make sense, should we go ahead and fix the issues around it? I think we should fix these bugs you pointed out. I am sure some of you must have seen some of these issues and solved them already. Please do share how do tackle these issues? Thanks, Fawad Khaliq ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [neutron][security-groups] Neutron default security groups
Hi, shihanzhang: Thanks for bringing this up, again. As I said before, this blueprint will solve the problems that the 'hard-coded' rules related to the default security group we are suffering from, which I do think will give Fawad an anser. So, I really hope that we can particapate all together to make this happen in Kilo. 2014-09-17 10:11 GMT+08:00 shihanzhang : > As I know there is no a way to disable default security groups, but I think > this BP can solve this problem: > https://blueprints.launchpad.net/neutron/+spec/default-rules-for-default-security-group > > > 在 2014-09-17 07:44:42，"Aaron Rosen" 写道： > > Hi, > > Inline: > > On Tue, Sep 16, 2014 at 1:00 AM, Fawad Khaliq wrote: >> >> Folks, >> >> I have had discussions with some folks individually about this but I would >> like bring this to a broader audience. >> >> I have been playing with security groups and I see the notion of 'default' >> security group seems to create some nuisance/issues. >> >> There are list of things I have noticed so far: >> >> Tenant for OpenStack services (normally named service/services) also ends >> up having default security group. >> Port create operation ends up ensuring default security groups for all the >> tenants as this completely seems out of the context of the tenant the port >> operation takes place. (bug?) >> Race conditions where if system is stressed and Neutron tries to ensure >> the first default security group and in parallel another call comes, Neutron >> ends up trying to create multiple default security groups as the checks for >> duplicate groups are invalidated as soon as the call make past a certain >> point in code. > > Right this is a bug. We should catch this foreign key constraint exception > here and pass as the default security group was already created. We do > something similar here when ports are created as there is a similar race for > ip_allocation. >> >> >> API performance where orchestration chooses to spawn 1000 tenants and we >> see unnecessary overhead >> >> For plugins that use RESTful proxy backends require the backend systems to >> be up at the time neutron starts. [Minor, but affects some packaging >> solutions] > > > This is probably always a requirement for neutron to work so I don't think > it's related. >> >> >> To summarize, is there a way to disable default security groups? Expected >> answer is no; can we introduce a way to disable it? If that does not make >> sense, should we go ahead and fix the issues around it? > > > I think we should fix these bugs you pointed out. >> >> >> I am sure some of you must have seen some of these issues and solved them >> already. Please do share how do tackle these issues? >> >> Thanks, >> Fawad Khaliq >> >> >> ___ >> OpenStack-dev mailing list >> OpenStack-dev@lists.openstack.org >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> > > > > > ___ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > -- Regards! --- Lingxian Kong ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [neutron][security-groups] Neutron default security groups
Agree! +1 On Thu, Sep 18, 2014 at 1:47 PM, Lingxian Kong wrote: > Hi, shihanzhang: > > Thanks for bringing this up, again. > > As I said before, this blueprint will solve the problems that the > 'hard-coded' rules related to the default security group we are > suffering from, which I do think will give Fawad an anser. So, I > really hope that we can particapate all together to make this happen > in Kilo. > > 2014-09-17 10:11 GMT+08:00 shihanzhang : > > As I know there is no a way to disable default security groups, but I > think > > this BP can solve this problem: > > > https://blueprints.launchpad.net/neutron/+spec/default-rules-for-default-security-group > > > > > > 在 2014-09-17 07:44:42，"Aaron Rosen" 写道： > > > > Hi, > > > > Inline: > > > > On Tue, Sep 16, 2014 at 1:00 AM, Fawad Khaliq > wrote: > >> > >> Folks, > >> > >> I have had discussions with some folks individually about this but I > would > >> like bring this to a broader audience. > >> > >> I have been playing with security groups and I see the notion of > 'default' > >> security group seems to create some nuisance/issues. > >> > >> There are list of things I have noticed so far: > >> > >> Tenant for OpenStack services (normally named service/services) also > ends > >> up having default security group. > >> Port create operation ends up ensuring default security groups for all > the > >> tenants as this completely seems out of the context of the tenant the > port > >> operation takes place. (bug?) > >> Race conditions where if system is stressed and Neutron tries to ensure > >> the first default security group and in parallel another call comes, > Neutron > >> ends up trying to create multiple default security groups as the checks > for > >> duplicate groups are invalidated as soon as the call make past a certain > >> point in code. > > > > Right this is a bug. We should catch this foreign key constraint > exception > > here and pass as the default security group was already created. We do > > something similar here when ports are created as there is a similar race > for > > ip_allocation. > >> > >> > >> API performance where orchestration chooses to spawn 1000 tenants and we > >> see unnecessary overhead > >> > >> For plugins that use RESTful proxy backends require the backend systems > to > >> be up at the time neutron starts. [Minor, but affects some packaging > >> solutions] > > > > > > This is probably always a requirement for neutron to work so I don't > think > > it's related. > >> > >> > >> To summarize, is there a way to disable default security groups? > Expected > >> answer is no; can we introduce a way to disable it? If that does not > make > >> sense, should we go ahead and fix the issues around it? > > > > > > I think we should fix these bugs you pointed out. > >> > >> > >> I am sure some of you must have seen some of these issues and solved > them > >> already. Please do share how do tackle these issues? > >> > >> Thanks, > >> Fawad Khaliq > >> > >> > >> ___ > >> OpenStack-dev mailing list > >> OpenStack-dev@lists.openstack.org > >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > >> > > > > > > > > > > ___ > > OpenStack-dev mailing list > > OpenStack-dev@lists.openstack.org > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > > > > -- > Regards! > --- > Lingxian Kong > > ___ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > -- Best wishes! Baohua ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
