Re: [openstack-dev] [puppet] Moving forward with puppet-keystone CI (beaker tests)
On 04/22/2015 08:03 PM, Emilien Macchi wrote: * shell testing: yes because it's the way we wrote our providers. We don't necessary need to shell out, but instead implement resources for Serverspec which will use openstackclient to test Keystone/Nova/... resources. I'm currently in process of merging Serverspec tests I made and will submit patches as soon as I will have something working. What I have currently is just Serverspec resource for testing INI file content, but openstackclient Serverspec resources are next on my TODO list. Regards, Martin -- Martin Mágr IRC nick: mmagr / para Freenode channels: #openstack-dev, #packstack-dev, #puppet-openstack, #rdo, #rdo-puppet __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [puppet] Moving forward with puppet-keystone CI (beaker tests)
Hi, Some important work is being done on Keystone v3 API support in puppet-keystone. We've clearly seen there is a lack of review and I think we all worry about breaking something. Spencer I are working on beaker tests lately and the jobs are non-voting for now. I propose: * to review (and eventually merge) the beaker-tests patches [1] [2] for Keystone openstacklib. * to patch project-config [3] to make vote Beaker jobs in Puppet OpenStack gate for puppet-keystone puppet-openstacklib. Why voting? Because otherwise I'm not sure people will notice the failure and some patches will be merged while beaker is red. So we can have a good set of tests that will help us to detect some issues in the future. I don't think we will catch all mistakes we can do, but this is a good start. To vote this proposal, you can use the gerrit patches and let any feedback. Thanks, [1] puppet-keystone: https://review.openstack.org/#/c/155873/ [2] puppet-openstacklib: https://review.openstack.org/#/c/176098/ [3] project-config: https://review.openstack.org/176343 -- Emilien Macchi signature.asc Description: OpenPGP digital signature __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [puppet] Moving forward with puppet-keystone CI (beaker tests)
Emillen, Do you see shelling out to openstackclient in the keystone test to verify that the keystone resources have been created? Do you see trying to hit the api from something like aviator? Ultimately I'd like to see us spin up an entire openstack in one test then hit it with tempest. It may be possible to use a very narrow version of tempest to validate just keystone. Thanks, Spencer On Wed, Apr 22, 2015 at 7:51 AM, Emilien Macchi emil...@redhat.com wrote: Hi, Some important work is being done on Keystone v3 API support in puppet-keystone. We've clearly seen there is a lack of review and I think we all worry about breaking something. Spencer I are working on beaker tests lately and the jobs are non-voting for now. I propose: * to review (and eventually merge) the beaker-tests patches [1] [2] for Keystone openstacklib. * to patch project-config [3] to make vote Beaker jobs in Puppet OpenStack gate for puppet-keystone puppet-openstacklib. Why voting? Because otherwise I'm not sure people will notice the failure and some patches will be merged while beaker is red. So we can have a good set of tests that will help us to detect some issues in the future. I don't think we will catch all mistakes we can do, but this is a good start. To vote this proposal, you can use the gerrit patches and let any feedback. Thanks, [1] puppet-keystone: https://review.openstack.org/#/c/155873/ [2] puppet-openstacklib: https://review.openstack.org/#/c/176098/ [3] project-config: https://review.openstack.org/176343 -- Emilien Macchi -- To unsubscribe from this group and stop receiving emails from it, send an email to puppet-openstack+unsubscr...@puppetlabs.com. -- Spencer Krum (619)-980-7820 __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [puppet] Moving forward with puppet-keystone CI (beaker tests)
On 04/22/2015 11:53 AM, Spencer Krum wrote: Emillen, Do you see shelling out to openstackclient in the keystone test to verify that the keystone resources have been created? Do you see trying to hit the api from something like aviator? Ultimately I'd like to see us spin up an entire openstack in one test then hit it with tempest. * shell testing: yes because it's the way we wrote our providers. * aviator: no because we gave up some months ago in favor of using openstackclient. I'm not in favor of using aviator which would add yet another dependency and complexity. Maybe I'm wrong though. * tempest: well... tempest aims to test API features while we only want to check Keystone is actually running. I think serverspec + some shelling could help. Having tempest is (to me) overkill and could slow down our CI if something's wrong in Tempest. It may be possible to use a very narrow version of tempest to validate just keystone. Like, only a small set of tests that we would run with testr? Thanks, Spencer On Wed, Apr 22, 2015 at 7:51 AM, Emilien Macchi emil...@redhat.com mailto:emil...@redhat.com wrote: Hi, Some important work is being done on Keystone v3 API support in puppet-keystone. We've clearly seen there is a lack of review and I think we all worry about breaking something. Spencer I are working on beaker tests lately and the jobs are non-voting for now. I propose: * to review (and eventually merge) the beaker-tests patches [1] [2] for Keystone openstacklib. * to patch project-config [3] to make vote Beaker jobs in Puppet OpenStack gate for puppet-keystone puppet-openstacklib. Why voting? Because otherwise I'm not sure people will notice the failure and some patches will be merged while beaker is red. So we can have a good set of tests that will help us to detect some issues in the future. I don't think we will catch all mistakes we can do, but this is a good start. To vote this proposal, you can use the gerrit patches and let any feedback. Thanks, [1] puppet-keystone: https://review.openstack.org/#/c/155873/ [2] puppet-openstacklib: https://review.openstack.org/#/c/176098/ [3] project-config: https://review.openstack.org/176343 -- Emilien Macchi -- To unsubscribe from this group and stop receiving emails from it, send an email to puppet-openstack+unsubscr...@puppetlabs.com mailto:puppet-openstack%2bunsubscr...@puppetlabs.com. -- Spencer Krum (619)-980-7820 -- To unsubscribe from this group and stop receiving emails from it, send an email to puppet-openstack+unsubscr...@puppetlabs.com mailto:puppet-openstack+unsubscr...@puppetlabs.com. -- Emilien Macchi signature.asc Description: OpenPGP digital signature __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [puppet] Moving forward with puppet-keystone CI (beaker tests)
On Wed, Apr 22, 2015 at 02:03:13PM -0400, Emilien Macchi wrote: On 04/22/2015 11:53 AM, Spencer Krum wrote: Emillen, Do you see shelling out to openstackclient in the keystone test to verify that the keystone resources have been created? Do you see trying to hit the api from something like aviator? Ultimately I'd like to see us spin up an entire openstack in one test then hit it with tempest. * shell testing: yes because it's the way we wrote our providers. * aviator: no because we gave up some months ago in favor of using openstackclient. I'm not in favor of using aviator which would add yet another dependency and complexity. Maybe I'm wrong though. * tempest: well... tempest aims to test API features while we only want to check Keystone is actually running. I think serverspec + some shelling could help. Having tempest is (to me) overkill and could slow down our CI if something's wrong in Tempest. I kinda take issue with this comment, more likely than not when there are issues with running tempest it's either a config or service problem. That's not to say there aren't tempest bugs, I just don't think tempest breaking your CI should be a primary concern. (especially how often it's used for this exact purpose) If it did break something it's not like fixing it is a complicated procedure. We've been running tempest as the gating tests for some time so we've got some experience fixing issues with fixing the test suite when things really go haywire. I actually think running tempest against against a deployment using the puppet modules would probably be very useful. It'll let you test that everything comes together and actually works. It's used for gating devstack commits to verify basically the same thing. I'm also interested in doing this because I think it'll help us improve tempest by having direct feedback loop from running it against a non-devstack environment. That's something I'd really like to have because it's something I think we're missing right now. If you need any help with setting something like this up, I'm definitely available to give a hand with it. It may be possible to use a very narrow version of tempest to validate just keystone. Like, only a small set of tests that we would run with testr? So this is the intent of the smoke tag in tempest, to provide a small set of tests to do a quick sanity check that a deployment is working. Unfortunately right now the tag doesn't do that because it got conflated as part of the neutron testing bring up. We just need to spend some time sitting down and rationalizing the use of the tag to provide this. Also, if you want to just test service specific testing tempest already does service tagging of tests. So you can run a subset by using the service name as the filter. (ie identity for tests which talk to keystone or compute for nova tests) -Matt Treinish On Wed, Apr 22, 2015 at 7:51 AM, Emilien Macchi emil...@redhat.com mailto:emil...@redhat.com wrote: Hi, Some important work is being done on Keystone v3 API support in puppet-keystone. We've clearly seen there is a lack of review and I think we all worry about breaking something. Spencer I are working on beaker tests lately and the jobs are non-voting for now. I propose: * to review (and eventually merge) the beaker-tests patches [1] [2] for Keystone openstacklib. * to patch project-config [3] to make vote Beaker jobs in Puppet OpenStack gate for puppet-keystone puppet-openstacklib. Why voting? Because otherwise I'm not sure people will notice the failure and some patches will be merged while beaker is red. So we can have a good set of tests that will help us to detect some issues in the future. I don't think we will catch all mistakes we can do, but this is a good start. To vote this proposal, you can use the gerrit patches and let any feedback. Thanks, [1] puppet-keystone: https://review.openstack.org/#/c/155873/ [2] puppet-openstacklib: https://review.openstack.org/#/c/176098/ [3] project-config: https://review.openstack.org/176343 -- Emilien Macchi -- To unsubscribe from this group and stop receiving emails from it, send an email to puppet-openstack+unsubscr...@puppetlabs.com mailto:puppet-openstack%2bunsubscr...@puppetlabs.com. -- Spencer Krum (619)-980-7820 -- To unsubscribe from this group and stop receiving emails from it, send an email to puppet-openstack+unsubscr...@puppetlabs.com mailto:puppet-openstack+unsubscr...@puppetlabs.com. -- Emilien Macchi __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
Re: [openstack-dev] [puppet] Moving forward with puppet-keystone CI (beaker tests)
On 04/22/2015 10:51 AM, Emilien Macchi wrote: Hi, Some important work is being done on Keystone v3 API support in puppet-keystone. We've clearly seen there is a lack of review and I think we all worry about breaking something. Spencer I are working on beaker tests lately and the jobs are non-voting for now. I propose: * to review (and eventually merge) the beaker-tests patches [1] [2] for Keystone openstacklib. * to patch project-config [3] to make vote Beaker jobs in Puppet OpenStack gate for puppet-keystone puppet-openstacklib. Why voting? Because otherwise I'm not sure people will notice the failure and some patches will be merged while beaker is red. So we can have a good set of tests that will help us to detect some issues in the future. I don't think we will catch all mistakes we can do, but this is a good start. To vote this proposal, you can use the gerrit patches and let any feedback. Thanks, [1] puppet-keystone: https://review.openstack.org/#/c/155873/ [2] puppet-openstacklib: https://review.openstack.org/#/c/176098/ [3] project-config: https://review.openstack.org/176343 I added Keystone-core (13 People) to each of these reviews. If things are critical, and concert Keystone, we really should be notified so someone on the Keystone side can voice an opinion. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
