-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 22.10.2014 13:43, Honza Horak wrote: > Fedora lacks integration testing (unit testing done during build > is not enough). Taskotron will be able to fill some gaps in the > future, so maintainers will be able to set-up various tasks after > their component is built. But even before this works we can > benefit from having the tests already available (and run them > manually if needed). > > Hereby, I'd like to get ideas and figure out answers for how and > where to keep the tests. A similar discussion already took place > before, which I'd like to continue in: > https://lists.fedoraproject.org/pipermail/devel/2014-January/193498.html > > > And some short discussion already took place here as well: > https://lists.fedoraproject.org/pipermail/env-and-stacks/2014-October/000570.html > > > > > Some high level requirements: * tests will be written by > maintainers or broader community, not a dedicated team * tests > will be easy to run on anybody's computer (but might be > potentially destructive; some secure environment will not be part > of tests) * tests will be run automatically after related > components get built (probably by Taskotron) > > Where to keep tests? a/ in current dist-git for related components > (problem with sharing parts of code, problem where to keep tests > related for more components) b/ in separate git with similar > functionality as dist-git (needs new infrastructure, components > are not directly connected with tests, won't make mess in current > dist-git) c/ in current dist-git but as ordinary components (no > new infrastructure needed but components are not directly > connected with tests) > > How to deliver tests? a/ just use them directly from git (we need > to keep some metadata for dependencies anyway) b/ package them as > RPMs (we can keep metadata there; e.g. Taskotron will run only > tests that have "Provides: ci-tests(mariadb)" after mariadb is > built; we also might automate packaging tests to RPMs) > > Structure for tests? a/ similar to what components use (branches > for Fedora versions) b/ only one branch Test maintainers should be > allowed to behave the same as package maintainers do -- one likes > keeping branches the same and uses "%if %fedora" macros, someone > else likes specs clean and rather maintain more different > branches) -- we won't find one structure that would fit all, so > allowing both ways seems better. > > Which framework to use? People have no time to learn new things, > so we should let them to write the tests in any language and just > define some conventions how to run them.
TAP (Test Anything Protocol) FTW. It really makes sense when you're trying to combine tests from multiple different languages, testing frameworks, etc. Stef -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iEYEARECAAYFAlRJWOMACgkQe/sRCNknZa/ltQCfcTBBPOIl3fISqjm0j3YUw+TU eSAAoIMo+NSOg/iWf27VQuq0J2rTebT/ =L9uQ -----END PGP SIGNATURE----- -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
