On Tue, 19 Jan 2016 07:44:32 -0500 (EST)
Kamil Paral wrote:
> > I'm preparing for an automation workshop at DevConf in February and
> > I'm trying to think of good example cases to show off what
> > Taskotron can do.
> >
> > At the moment, I figure that I'll do at least 1 easy task and 1 not
> > so easy but not crazy task.
> >
> > Rpmlint is usually my go-to for a trivial/example task but I was
> > hoping to use something that's a bit more package specific. The
> > things I've thought of so far are:
> >
> > - Put together some package specific stuff for a package that we
> > have some control over (testcloud, yourls etc.)
> >
> > - Find some other guinea pig package that just works well for
> >package-specific tasks since the triggering will be artificial
> > for the demos
>
> Do you have any ideas what those package specific checks could be?
> Because all I can think of is running some test suite which can't be
> used during build time in Koji, but we have nothing like that in our
> own packages, we would have to find and use some other package (and
> study and write a wrapper for that suite).
Yeah, I don't have too many brilliant ideas either. I figured that
writing one to run testcloud's test suite would work well enough - it's
trivial but it runs quickly and would get the idea across well enough.
The only thing I've thought of beyond the list of testcases you
listed below is the Docker test suite that we're likely to run before
long anyways.
> >
> > - See if there are any existing test cases which would automate
> > easily.
>
> Not sure if you mean release validation test cases, but we can most
> probably rule them out, because they all require testing with that
> specific TC/RC (or an installed system that matches those package
> versions), which we can't offer at the moment - not without nested
> virt or some heavy modifications in the environment setup process.
>
> So I guess you mean package test cases (everything that starts with
> "Package "): https://fedoraproject.org/wiki/Category:Test_Cases
>
> And this is really a great idea. I have went through all of those
> test cases and the following seem simple enough and easily automated:
> https://fedoraproject.org/wiki/QA:Testcase_ClamAV
> https://fedoraproject.org/wiki/QA:Testcase_Django_smoketest
> https://fedoraproject.org/wiki/QA:Testcase_HTTPd
> https://fedoraproject.org/wiki/QA:Testcase_MySQL
> https://fedoraproject.org/wiki/QA:Testcase_clone_distgit_repository
> https://fedoraproject.org/wiki/QA:Testcase_download_upstream_sources
> https://fedoraproject.org/wiki/QA:Testcase_build_from_distgit_with_mock
> https://fedoraproject.org/wiki/QA:Testcase_Rkhunter
>
> I think that picking one of them could be a nice demonstration of
> what we are aiming for with Taskotron. The only downside is that we
> would be showing them something they still can't do themselves (since
> we're not still open to arbitrary third-party task submissions, nor
> we support storing tasks in distgit-like system). But as a
> demonstration of what is hopefully coming in a reasonably close
> future, I think this would serve well.
Yeah, that'd be good to at least investigate.
Before DevConf, I want to at least have a scheme for supporting tasks
in git and it'd be great if that was in dev or stg before I do the
workshop. The other things that I want to have on our roadmap before
DevConf and be able to say more than "I don't know" about when asked
when they'll land are:
- Report Generation with jinja or something similar
(I thought we had a ticket for this, will write one up)
- Generic test runner
https://phab.qadevel.cloud.fedoraproject.org/T589
Hopefully we'll get disposable clients into prod this week and then get
back to a saner planning method than what we seem to have slipped back
into. Mostly my fault and I apologize for that :(
Tim
pgp3uGJwj19Qs.pgp
Description: OpenPGP digital signature
___
qa-devel mailing list
qa-devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/qa-devel@lists.fedoraproject.org
