On (28/04/17 17:07), Adam Williamson wrote: >Hi folks! I thought this might be of interest to the FreeIPA community, >so I thought I'd write it up here in case anyone missed it elsewhere. > >I work on the Fedora QA team, and we have been using the openQA >automated test system (developed by our friends at SUSE) to run various >functional tests on Fedora composes for the last couple of years. > >As FreeIPA is considered a critical part of Fedora Server, we run a few >tests that exercise FreeIPA. The tests set up a FreeIPA server, run >some basic checks on it, and also enrol two systems as clients of the >domain, one using the 'realm join' command directly, one using Cockpit. >The client tests do some basic client functionality testing (getent, >logging in as a domain user, changing passwords, etc.) and also test >the web UI to some extent. > >Until recently we ran these tests only on Fedora's nightly development >release distribution composes. Recently, though, we deployed some >enhancements to our openQA setup that let us run tests on Fedora >distribution updates as well, and have the results made visible through >the Fedora update system (Bodhi). The tests are automatically run on >any critical path package, and as of today, they are also run on any >update containing any of a manually-tended list of FreeIPA-related >packages: > >389-ds >389-ds-base >bind >bind-dyndb-ldap >certmonger >ding-libs >freeipa >krb5-server >pki-core >sssd >tomcat >cockpit > >This means that for any Fedora update containing one of these or any >critical path package, Fedora's openQA FreeIPA tests should run, and >you should see the results in the Fedora update system (Bodhi). You can >see the results in Bodhi by clicking the Automated Updates tab for any >update. For instance, here's a recent 389-ds-base update for Fedora 26: > >https://bodhi.fedoraproject.org/updates/FEDORA-2017-15e2a038b2 > >If you look at the Automated Tests tab, you can see passes for: > >update.server_role_deploy_domain_controller >update.realmd_join_cockpit >update.realmd_join_sssd > >indicating that this update didn't cause any problems for FreeIPA. >Clicking on any test result will take you to the openQA page for the >test, where you can diagnose failures and so on (explaining how to do >this is a bit beyond the scope of this mail, please do ask me if you're >interested!) > >I hope this stuff will help us avoid shipping updates that break >FreeIPA (and other key components). If you have any questions, >concerns, comments, or suggestions, please do ask! > >To anticipate one question: you can cause *all* the tests for an update >to be re-run by editing the update in any way (you don't have to change >the package loadout, just changing a single character in the >description or something will do). If you think just one test result is >bogus and want it re-run, currently, you'll have to ask someone with >the necessary power - either me or Jan Sedlak (garretraziel on IRC). >I'm in North America and he's in Europe, so we should have most >timezones covered between us. We're hoping to set up a better mechanism >for this in future. > >Note, if you're interested in the results for the nightly Fedora >distribution composes, an email summary of the results for those is >sent each time they're run to the Fedora test@ and devel@ lists, look >for mails with "compose check report" in the subject. Any time any of >the FreeIPA tests fails, the failure will be listed in the mail (passed >tests are not specifically listed, just a count of them). I usually >keep an eye on those results and analyze failures and file bugs, >though.
Tested with sssd and it passed as well. https://bodhi.fedoraproject.org/updates/FEDORA-2017-8addfc0188 freeIPA has also upstream integration tests packaged in python{2,3}-ipatests. They use pytest + python-pytest-multihost. Will it be possible to run some of them in openQA? e.g. test_installation.py ( LS -- Manage your subscription for the Freeipa-devel mailing list: https://www.redhat.com/mailman/listinfo/freeipa-devel Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code