Hi, all I'd like to discuss the topic about how do we configure tempest in CI jobs for TripleO. I have currently two patches: support for tempest: https://review.openstack.org/#/c/295844/ actually run of tests: https://review.openstack.org/#/c/297038/
Right now there is no upstream tool to configure tempest, so everybody use their own tools. However it's planned and David Mellado is working on it AFAIK. Till then everybody use their own tools for tempest configuration. I'd review two of them: 1) Puppet configurations that is used in puppet modules CI 2) Using configure_tempest.py script from https://github.com/redhat-openstack/tempest/blob/master/tools/config_tempest.py Unfortunately there is no ready puppet module or script, that configures tempest, you need to create your own. On other hand the config_tempest.py script provides full configuration, support for tempest-deployer-input.conf and possibility to add any config options in the command line when running it: python config_tempest.py \ --out etc/tempest.conf \ --debug \ --create \ --deployer-input ~/tempest-deployer-input.conf \ identity.uri $OS_AUTH_URL \ compute.allow_tenant_isolation true \ identity.admin_password $OS_PASSWORD \ compute.build_timeout 500 \ compute.image_ssh_user cirros Also it uploads images, creates necessary roles, etc. The only thing it requires - existence of public network. So finally all tempest configuration from scratch will be like: CONFIGURE_TEMPEST_DIR="$(ls /usr/share/openstack-tempest-*/tools/configure-tempest-directory)" $CONFIGURE_TEMPEST_DIR neutron net-create nova --shared --router:external=True --provider:network_type flat --provider:physical_network datacentre; neutron subnet-create --name ext-subnet --allocation-pool start=$FLOATING_IP_START,end=$FLOATING_IP_END --disable-dhcp --gateway $EXTERNAL_NETWORK_GATEWAY nova $FLOATING_IP_CIDR; python tempest/tools/install_venv.py python config_tempest.py \ --out etc/tempest.conf \ --debug \ --create \ --deployer-input ~/tempest-deployer-input.conf \ identity.uri $OS_AUTH_URL \ compute.allow_tenant_isolation true \ identity.admin_password $OS_PASSWORD \ compute.build_timeout 500 \ compute.image_ssh_user cirros testr init; testr run In my patch [1] I have proposed it with little changes to TripleO CI repo [2] Like I wrote before there is an option to use puppet for this, I spent a time to investigate how to do it and would like share the results with your in order to compare it with config_tempest.py approach. First of all it's surprising that puppet-tempest actually doesn't know to do almost anything. All it knows - it's to set IDs of public network (but not router) and images. That's all. All the rest you need to configure manually. Then comes another problem - you can use it only on overcloud controller node, where are all service configurations and hiera data. Most of values are taken directly from /etc/{service}/service.conf files, so doing it on undercloud you will configure undercloud itself (instead of overcloud) So first of all you need to upload this manifest to controller node of overcloud. Let's write this puppet manifest, I wrote everything in one file for saving a time, but of course it should be a module with usual puppet module structure: module_name/manifests/init.pp with module_name class. Manual configurations: class testt::config { $os_username = 'admin' $os_tenant_name = hiera(keystone::roles::admin::admin_tenant) $os_password = hiera(admin_password) $os_auth_url = hiera(keystone::endpoint::public_url) $keystone_auth_uri = regsubst($os_auth_url, '/v2.0', '') $floating_range = "192.0.2.0/24" $gateway_ip = "192.0.2.1" $floating_pool = 'start=192.0.2.50,end=192.0.2.99' $fixed_range = '10.0.0.0/24' $router_name = 'router1' $ca_bundle_cert_path = '/etc/ssl/certs/ca-bundle.crt' $cert_path = '/etc/pki/ca-trust/source/anchors/puppet_openstack.pem' $update_ca_certs_cmd = '/usr/bin/update-ca-trust force-enable && /usr/bin/update-ca-trust extract' $host_url = regsubst($keystone_auth_uri, ':5000', '') } Most of data is taken from hiera on the controller host. (/etc/hieradata) Then we start actually the tempest configuration. Surprisingly it doesn't have resource type to work with flavors, so all its configuration is done by "exec"s. We run puppet with bash to run bash within a puppet, what gives pretty big overhead. class testt::provision { include testt::config $os_auth_options = "--os-username ${config::os_username} --os-password ${config::os_password} --os-tenant-name ${config::os_tenant_name} --os-auth-url ${config::os_auth_url}/v2.0" exec { 'manage_m1.nano_nova_flavor': path => '/usr/bin:/bin:/usr/sbin:/sbin', provider => shell, command => "nova ${os_auth_options} flavor-delete m1.nano ||: ; nova ${os_auth_options} flavor-create m1.nano pup_tempest_custom_nano 128 0 1", unless => "nova ${os_auth_options} flavor-list | grep pup_tempest_custom_nano", } exec { 'manage_m1.micro_nova_flavor': path => '/usr/bin:/bin:/usr/sbin:/sbin', provider => shell, command => "nova ${os_auth_options} flavor-delete m1.micro ||: ;nova ${os_auth_options} flavor-create m1.micro pup_tempest_custom_micro 128 0 1", unless => "nova ${os_auth_options} flavor-list | grep pup_tempest_custom_micro", } Then we create public and private networks, and router between them: $neutron_deps = [Neutron_network['nova']] neutron_network { 'nova': ensure => 'present', router_external => true, tenant_name => "${config::os_tenant_name}", } neutron_subnet { 'ext-subnet': ensure => 'present', cidr => "${config::floating_range}", enable_dhcp => false, allocation_pools => ["${config::floating_pool}"], gateway_ip => "${config::gateway_ip}", network_name => 'nova', tenant_name => "${config::os_tenant_name}", } neutron_network { 'private': ensure => 'present', tenant_name => "${config::os_tenant_name}", } neutron_subnet { 'private_subnet': ensure => 'present', cidr => "${config::fixed_range}", network_name => 'private', tenant_name => "${config::os_tenant_name}", } neutron_router { "${config::router_name}": ensure => 'present', tenant_name => "${config::os_tenant_name}", gateway_network_name => 'nova', require => Neutron_subnet['ext-subnet'], } neutron_router_interface { "${config::router_name}:private_subnet": ensure => 'present', } After this it's a time to upload images: glance_image { 'cirros': ensure => present, container_format => 'bare', disk_format => 'qcow2', is_public => 'yes', source => ' http://download.cirros-cloud.net/0.3.4/cirros-0.3.4-x86_64-disk.img', } glance_image { 'cirros_alt': ensure => present, container_format => 'bare', disk_format => 'qcow2', is_public => 'yes', source => ' http://download.cirros-cloud.net/0.3.4/cirros-0.3.4-x86_64-disk.img', } And then you run actually puppet tempest with configuration values, most of them you set by yourself: class { '::tempest': debug => true, use_stderr => false, log_file => 'tempest.log', tempest_clone_owner => $::id, git_clone => true, setup_venv => true, tempest_clone_path => '/tmp/openstack/tempest', lock_path => '/tmp/openstack/tempest', tempest_config_file => '/tmp/openstack/tempest/etc/tempest.conf', configure_images => true, configure_networks => true, allow_tenant_isolation => true, identity_uri => "${testt::config::keystone_auth_uri}/v2.0", identity_uri_v3 => "${testt::config::keystone_auth_uri}/v3", admin_username => "${testt::config::os_username}", admin_tenant_name => "${testt::config::os_tenant_name}", admin_password => "${testt::config::os_password}", admin_domain_name => 'Default', auth_version => 'v3', image_name => 'cirros', image_name_alt => 'cirros_alt', cinder_available => true, glance_available => true, horizon_available => $horizon, nova_available => true, neutron_available => true, ceilometer_available => $ceilometer, aodh_available => $aodh, trove_available => $trove, sahara_available => $sahara, heat_available => $heat, swift_available => true, ironic_available => $ironic, public_network_name => 'nova', dashboard_url => "${testt::config::host_url}", flavor_ref => 'pup_tempest_custom_nano', flavor_ref_alt => 'pup_tempest_custom_micro', image_ssh_user => 'cirros', image_alt_ssh_user => 'cirros', img_file => 'cirros-0.3.4-x86_64-disk.img', compute_build_interval => 10, ca_certificates_file => "${testt::config::ca_bundle_cert_path}", img_dir => '/tmp/openstack/tempest', } But it's not enough, you need also to make some workarounds and additional configurations, for example: tempest_config { 'object-storage/operator_role': value => 'SwiftOperator', path => "${tempest_clone_path}/etc/tempest.conf", } } After this run puppet on controller node: sudo puppet apply --verbose --debug --detailed-exitcodes -e "include ::testt" | tee ~/puppet_run.log After everything is finished, you need to copy the folder with tempest to your node: scp -r -heat-admin@${CONTROLLER}:/tmp/openstack /tmp/ After this run within this directory testr init and run tests: /tmp/tempest/tools/with_venv.sh testr init /tmp/tempest/tools/with_venv.sh testr run There are still holes in this configuration and most likely you'd fix it by another workarounds and tempest_config runs, because it's still a few of skipped tests, so configuration is not full as it would be done with config_tempest.py. You don't have also any possibility to add custom configuration in running the manifest, for each config change you need to change the manifest itself which makes it maintenance harder and more complex. I would say that conclusion is quite obvious for me and it's much easier even to write tempest.conf manually from scratch or simple template and use 5 bash lines, then use puppet for things it's completely not fit to. P.S. In this script I used ideas from puppet-openstack-integration and packstack projects. [1] https://review.openstack.org/#/c/295844/ [2] https://git.openstack.org/openstack-infra/tripleo-ci -- Best regards Sagi Shnaidman
__________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
