Re: [openstack-dev] [keystone][qa] PCI-DSS (security compliance tests)
On Fri, Oct 14, 2016 at 2:23 PM, Rodrigo Duarte wrote: > > > On Fri, Oct 14, 2016 at 1:10 PM, Clark Boylan > wrote: > >> On Fri, Oct 14, 2016, at 08:21 AM, Rodrigo Duarte wrote: >> > Hi all, >> > >> > Recently in keystone we got merged the PCI-DSS feature [1]. Basically, >> we >> > have new settings that enforce password security practices. For example, >> > if >> > we set the password history setting to 2, a user won't be able to update >> > its password to one of the last 2 that have been set in the past. >> > >> > The issue is that, this settings, can break a couple of tests in >> Tempest. >> > Assuming the non-admin users in this tests don't affect any other test, >> > I've inserted a "security_compliance" feature flag and skipped the >> > portion >> > of the tests that can break when the PCI-DSS settings are enabled [2]. >> > >> > With that, I've pushed another patch that sets these settings upon >> > DevStack >> > deployment [3] and added the actual tests for the feature at [4]. So we >> > have a "tempest -> devstack -> tempest" chain of patches dependencies. >> >> Curious why the tests for this wouldn't go into the keystone functional >> job [5] which appear to run as a tempest plugin? Testing of these >> features shouldn't require any other service, just keystone right >> (though this job does start and run other services)? One big reason I >> ask is it could help constrain the testing of non default behaviors of >> keystone to a single job rather than needing to edit a bunch of other >> jobs or create new jobs just to toggle the non default behavior. >> > > That was the plan initially, but we considered two things: > > 1 - The users API is a pretty widely used API, so having it running in > Tempest makes sense > 2 - We need to add new settings in Tempest's config - I know that we can > "inherit" the Tempest config's in the plugin, but looks strange having some > stuff not actually used in Tempest but set there. > > But... If the both things above are acceptable, or even preferable, we can > change the approach. > Turns out there were smarter ways to restore the user credentials after the tests run, we won't need the first tempest patch after all. Submitted the new changes at [4]. > > >> >> > >> > I want your feedback regarding this, if this approach is acceptable and, >> > if >> > not, what are the options. >> >> I don't really know which approach is more preferable but I think that >> functional testing is an option too. > > >> > >> > Thanks! >> > >> > [1] https://blueprints.launchpad.net/keystone/+spec/pci-dss >> > [2] https://review.openstack.org/#/c/382018/ >> > [3] https://review.openstack.org/#/c/377004/ >> > [4] https://review.openstack.org/#/c/378624/ >> > >> >> [5] >> http://logs.openstack.org/56/371856/5/gate/gate-keystone-dsv >> m-functional-ubuntu-xenial/c9f937c/ >> >> Clark >> >> >> __ >> OpenStack Development Mailing List (not for usage questions) >> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscrib >> e >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> > > > > -- > Rodrigo Duarte Sousa > Senior Quality Engineer @ Red Hat > MSc in Computer Science > http://rodrigods.com > -- Rodrigo Duarte Sousa Senior Quality Engineer @ Red Hat MSc in Computer Science http://rodrigods.com __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [keystone][qa] PCI-DSS (security compliance tests)
On Fri, Oct 14, 2016 at 1:10 PM, Clark Boylan wrote: > On Fri, Oct 14, 2016, at 08:21 AM, Rodrigo Duarte wrote: > > Hi all, > > > > Recently in keystone we got merged the PCI-DSS feature [1]. Basically, we > > have new settings that enforce password security practices. For example, > > if > > we set the password history setting to 2, a user won't be able to update > > its password to one of the last 2 that have been set in the past. > > > > The issue is that, this settings, can break a couple of tests in Tempest. > > Assuming the non-admin users in this tests don't affect any other test, > > I've inserted a "security_compliance" feature flag and skipped the > > portion > > of the tests that can break when the PCI-DSS settings are enabled [2]. > > > > With that, I've pushed another patch that sets these settings upon > > DevStack > > deployment [3] and added the actual tests for the feature at [4]. So we > > have a "tempest -> devstack -> tempest" chain of patches dependencies. > > Curious why the tests for this wouldn't go into the keystone functional > job [5] which appear to run as a tempest plugin? Testing of these > features shouldn't require any other service, just keystone right > (though this job does start and run other services)? One big reason I > ask is it could help constrain the testing of non default behaviors of > keystone to a single job rather than needing to edit a bunch of other > jobs or create new jobs just to toggle the non default behavior. > That was the plan initially, but we considered two things: 1 - The users API is a pretty widely used API, so having it running in Tempest makes sense 2 - We need to add new settings in Tempest's config - I know that we can "inherit" the Tempest config's in the plugin, but looks strange having some stuff not actually used in Tempest but set there. But... If the both things above are acceptable, or even preferable, we can change the approach. > > > > > I want your feedback regarding this, if this approach is acceptable and, > > if > > not, what are the options. > > I don't really know which approach is more preferable but I think that > functional testing is an option too. > > > > Thanks! > > > > [1] https://blueprints.launchpad.net/keystone/+spec/pci-dss > > [2] https://review.openstack.org/#/c/382018/ > > [3] https://review.openstack.org/#/c/377004/ > > [4] https://review.openstack.org/#/c/378624/ > > > > [5] > http://logs.openstack.org/56/371856/5/gate/gate-keystone- > dsvm-functional-ubuntu-xenial/c9f937c/ > > Clark > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > -- Rodrigo Duarte Sousa Senior Quality Engineer @ Red Hat MSc in Computer Science http://rodrigods.com __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [keystone][qa] PCI-DSS (security compliance tests)
On Fri, Oct 14, 2016, at 08:21 AM, Rodrigo Duarte wrote: > Hi all, > > Recently in keystone we got merged the PCI-DSS feature [1]. Basically, we > have new settings that enforce password security practices. For example, > if > we set the password history setting to 2, a user won't be able to update > its password to one of the last 2 that have been set in the past. > > The issue is that, this settings, can break a couple of tests in Tempest. > Assuming the non-admin users in this tests don't affect any other test, > I've inserted a "security_compliance" feature flag and skipped the > portion > of the tests that can break when the PCI-DSS settings are enabled [2]. > > With that, I've pushed another patch that sets these settings upon > DevStack > deployment [3] and added the actual tests for the feature at [4]. So we > have a "tempest -> devstack -> tempest" chain of patches dependencies. Curious why the tests for this wouldn't go into the keystone functional job [5] which appear to run as a tempest plugin? Testing of these features shouldn't require any other service, just keystone right (though this job does start and run other services)? One big reason I ask is it could help constrain the testing of non default behaviors of keystone to a single job rather than needing to edit a bunch of other jobs or create new jobs just to toggle the non default behavior. > > I want your feedback regarding this, if this approach is acceptable and, > if > not, what are the options. I don't really know which approach is more preferable but I think that functional testing is an option too. > > Thanks! > > [1] https://blueprints.launchpad.net/keystone/+spec/pci-dss > [2] https://review.openstack.org/#/c/382018/ > [3] https://review.openstack.org/#/c/377004/ > [4] https://review.openstack.org/#/c/378624/ > [5] http://logs.openstack.org/56/371856/5/gate/gate-keystone-dsvm-functional-ubuntu-xenial/c9f937c/ Clark __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [keystone][qa] PCI-DSS (security compliance tests)
Hi all, Recently in keystone we got merged the PCI-DSS feature [1]. Basically, we have new settings that enforce password security practices. For example, if we set the password history setting to 2, a user won't be able to update its password to one of the last 2 that have been set in the past. The issue is that, this settings, can break a couple of tests in Tempest. Assuming the non-admin users in this tests don't affect any other test, I've inserted a "security_compliance" feature flag and skipped the portion of the tests that can break when the PCI-DSS settings are enabled [2]. With that, I've pushed another patch that sets these settings upon DevStack deployment [3] and added the actual tests for the feature at [4]. So we have a "tempest -> devstack -> tempest" chain of patches dependencies. I want your feedback regarding this, if this approach is acceptable and, if not, what are the options. Thanks! [1] https://blueprints.launchpad.net/keystone/+spec/pci-dss [2] https://review.openstack.org/#/c/382018/ [3] https://review.openstack.org/#/c/377004/ [4] https://review.openstack.org/#/c/378624/ -- Rodrigo Duarte Sousa Senior Quality Engineer @ Red Hat MSc in Computer Science http://rodrigods.com __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
