Blueprint changed by James Page: Whiteboard changed: - Work items: - Base framework to support complex deployment testing: TODO - Configurations for complex deployment scenarios (may overlap with formulas and recipes): TODO - Update ISO testing to deal with server ISO refactoring work: TODO - ec2 testing configuration for kernel team patch testing work: TODO + Scope: Deployment and Testing of Ubuntu Server to support complex requirements i.e. OpenStack + others + Coverage: + OpenStack + + + Method: orchestra+juju AND cloud+juju + + james-page proposed a method for testing juju charms on juju@l.u.c - see + https://lists.ubuntu.com/archives/juju/2011-October/000837.html + + Broadley this involves a new charm when can test other charms - results are then collated and can be reported on. + This is then executed by a framework (TBN) which set's up the juju environment and services, executes the charm testing and then can execute tests on the deployed environment. + I think this is the right way todo this testing - it can be used with any juju provider so can be applied in both physical and cloud environments. + + Frequency: TBC + + Daily Test with current development release + + for OpenStack: + + Daily Test of upstream trunk? + + + Questions/Proposals: + [james-page] We need to verify the list of deployments and get volunteers to develop charms for them where gaps exist. + + Work Items: + Develop juju charm testing framework to support all requirements: TODO + Writing testing hooks for all charms which we want to test this way: TODO + Setup OpenStack test lab: TODO + Identify OpenStack testing requirements on the deployed service: TODO + + >>>>>>>>>>>>>>>>>>>>>>> UDS-O Session Notes: 1) Complex Deployment Testing Alot of Ubuntu server deployments are much more than an ISO install on a single node; however this is pretty much the limit of the automated ISO testing that we currently complete. We should aim to target some complex Ubuntu server deployment scenarios for regular automated testing during the release cycle. This could be implemented either in the cloud or using virtualization; ensemble and cobbler+puppet might form part of the solution. 2) Automated Boot testing As we make changes to the boot/shutdown sequence of the server, and migrate init.d scripts to upstart jobs, we need some continuous integration testing to make sure it continues to work. We should take advantage of the automated testing framework to install, reboot, shutdown, start, remove, and repeat with all packages that are changed. We should also automate this testing with all of the available tasks on the install CDs. In addition, we should collect metrics on boot time and how each package affects it. hggdh: * we need to test iso install * want to do a single install and create gold copy * need test do-release-upgrade High Level Test Objectives: ------------------------------------- a) Deployment Testing: i) ISO Image Proving RAID testing (N.B. For raid testing also see: https://blueprints.launchpad.net/ubuntu/+spec/foundations-o-software-raid and/or etherpad until blueprint updated) LVM testing Multipath and encryption Existing ISO testing framework should deal with this. ii) AMI Image Proving Existing ec2 testing framework should deal with this. b) Package Removal Testing: - install services - start services - check services - stop services - uninstall services - re-install same services - start & re-check services c) Phase 2 - Service Proving Deployment of service and pluggable testing Deploy through orchestra and/or ensemble - which would allow testing of recipes and formulas as well (+1) Framework required for executing tests once deployment complete; mcollective could be used to distribute and execute tests; maybe still use python unittests; d) Automated Boot Testing: Minimal install - startup and shutdown OK Full install - startup and shutdown OK Complete upgrade and then re-test. e) Upgrade Testing: Lucid->Maverick->Natty for example. Need to catchup with mvo in this area - suspect some sort of gold-copy type setup might make sense. f) Recovery Testing: Recovery Mode Testing: Complex Deployment Scenarios: Wiki + Database + Memcache + Load Balancer Bridging and bonding of network interfaces Networking File Systems Openstack Deployment test Launchpad Samba + LDAP or Kerberos Backend Backup solutions. Ruby on Rails stack? Requirements: Remote testing of services. Possibly ask community to donate resources for optional tests
-- Automated Complex Server Deployment Testing https://blueprints.launchpad.net/ubuntu/+spec/servercloud-p-complex-deployment-testing -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
