Blueprint changed by Antonio Rosales: Whiteboard changed: charm unit tests recap... we automatically run: - installation - graph test we'd like to additionally run - charm tests read the spec lp:juju in the "docs" dir... look for something like charmtests.rst http://bazaar.launchpad.net/~clint-fewbar/juju/charm-tests-spec/view/head:/source/charm-tests.rst policy stuff? tests required? just encouraged for now additional testing plugins possible: - nessus, etc against the running service - app-armor complain against running service - lint-type 'charm proof' - upgrade-charm tests - testing spread with add-unit - coverage ??? - watching relation settings?????? please put all projects in one of: charm-tools juju-jitsu charms juju environments - include all of them, including MAAS support MAAS, other distro needs enable manual kicks from ~charmers (charmbot, but we prob wanna do that through openid over the web) - parameterized jenkins tests, both official and against specific distro needs charmtester is a *charm* in the charm store - test my custom stack of services is readily doable (contributions are welcome as always) jenkins file bugs when charms fail (close them when they pass again?) try for "maintainer" notification through launchpad how do long-failing tests relate to "series" freeze??? coverage tools possible? check out next cycle load testing for individual charms? for example, siege all charms with http interface scale testing? weekly large runs? really a _juju_ test and less a _charm_ test --- User Stories: - *Levi has just submitted and gotten his new Charm approved for inclusion in the Juju Charm store. Levi and other Juju community members would like to ensure his new Charm is staying functional with updates, and properly interacting with related services. Furthermore, Levi would like additional tools to give him further insights on how to improve his Charm. - *Andrew has submitted his charm and gotten it accepted in a previous Ubuntu release. Andrew would like to regularly confirm his charm is running appropriately in the current release under development. - *Naomi is interested in using MAAS/OpenStack/EC2 to deploy services via Juju. She would like to review charm testing results for a given Ubuntu release and have insights into how charms are performing in that release. - *Sarah is are regular charm and has a few charms she is maintaining. She would like to get Launchpad notifications when one of her charms fails in CharmTester. Sarah would like to easily look at a testing reporting interface and drill down to the logs to start the debugging process. + *Levi has just submitted and gotten his new Charm approved for inclusion in the Juju Charm store. Levi and other Juju community members would like to ensure his new Charm is staying functional with updates, and properly interacting with related services. Furthermore, Levi would like additional tools to give him further insights on how to improve his Charm. + *Andrew has submitted his charm and gotten it accepted in a previous Ubuntu release. Andrew would like to regularly confirm his charm is running appropriately in the current release under development. + *Naomi is interested in using MAAS/OpenStack/EC2 to deploy services via Juju. She would like to review charm testing results for a given Ubuntu release and have insights into how charms are performing in that release. + *Sarah is are regular charm and has a few charms she is maintaining. She would like to get Launchpad notifications when one of her charms fails in CharmTester. Sarah would like to easily look at a testing reporting interface and drill down to the logs to start the debugging process. Test Plans: - *need formal plan [a.rosales] + *Run charm tester against new and current charms in supported Ubuntu releases against supported Juju providers. Assumptions: - * Juju providers are functional to test in + * Juju providers are functional to test in Release Notes: - * Add availability notice and information on CharmTester. + * Add availability notice and information on CharmTester.
-- Juju Charm Unit Tests https://blueprints.launchpad.net/ubuntu/+spec/servercloud-q-juju-charm-unit-tests -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
