Blueprint changed by Dave Walker: Whiteboard changed: Work items: - Base framework to support complex deployment testing: TODO - Configurations for complex deployment scenarios (may overlap with formulas and recipes): TODO - Update ISO testing to deal with server ISO refactoring work: TODO - ec2 testing configuration for kernel team patch testing work: TODO + Base framework to support complex deployment testing: POSTPONED + Configurations for complex deployment scenarios (may overlap with formulas and recipes): POSTPONED + Update ISO testing to deal with server ISO refactoring work: POSTPONED + ec2 testing configuration for kernel team patch testing work: POSTPONED UDS-O Session Notes: 1) Complex Deployment Testing Alot of Ubuntu server deployments are much more than an ISO install on a single node; however this is pretty much the limit of the automated ISO testing that we currently complete. We should aim to target some complex Ubuntu server deployment scenarios for regular automated testing during the release cycle. This could be implemented either in the cloud or using virtualization; ensemble and cobbler+puppet might form part of the solution. 2) Automated Boot testing As we make changes to the boot/shutdown sequence of the server, and migrate init.d scripts to upstart jobs, we need some continuous integration testing to make sure it continues to work. We should take advantage of the automated testing framework to install, reboot, shutdown, start, remove, and repeat with all packages that are changed. We should also automate this testing with all of the available tasks on the install CDs. In addition, we should collect metrics on boot time and how each package affects it. hggdh: * we need to test iso install * want to do a single install and create gold copy * need test do-release-upgrade High Level Test Objectives: ------------------------------------- a) Deployment Testing: i) ISO Image Proving RAID testing (N.B. For raid testing also see: https://blueprints.launchpad.net/ubuntu/+spec/foundations-o-software-raid and/or etherpad until blueprint updated) LVM testing Multipath and encryption Existing ISO testing framework should deal with this. ii) AMI Image Proving Existing ec2 testing framework should deal with this. b) Package Removal Testing: - install services - start services - check services - stop services - uninstall services - re-install same services - start & re-check services c) Phase 2 - Service Proving Deployment of service and pluggable testing Deploy through orchestra and/or ensemble - which would allow testing of recipes and formulas as well (+1) Framework required for executing tests once deployment complete; mcollective could be used to distribute and execute tests; maybe still use python unittests; d) Automated Boot Testing: Minimal install - startup and shutdown OK Full install - startup and shutdown OK Complete upgrade and then re-test. e) Upgrade Testing: Lucid->Maverick->Natty for example. Need to catchup with mvo in this area - suspect some sort of gold-copy type setup might make sense. f) Recovery Testing: Recovery Mode Testing: Complex Deployment Scenarios: Wiki + Database + Memcache + Load Balancer Bridging and bonding of network interfaces Networking File Systems Openstack Deployment test Launchpad Samba + LDAP or Kerberos Backend Backup solutions. Ruby on Rails stack? Requirements: Remote testing of services. Possibly ask community to donate resources for optional tests
-- Automated Complex Server Deployment Testing https://blueprints.launchpad.net/ubuntu/+spec/server-o-complex-deployment-testing -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs