On Tue, Jul 12, 2016 at 11:52:45PM -0700, Adam Williamson wrote: > FWIW, as someone who is working on this, I don't think we can > realistically aim to do distribution-level automated testing with per- > package granularity. We actually have all the bits in place to do > something like that if we wanted to - I could have some kind of PoC > using existing openQA tests in a week or so - but I just kinda don't > think it's the way to go. > > I think a more profitable angle at the distribution level is to define > what it is we actually think a distribution should do, and test whether > updates change *those* things. That's an appropriate and manageable > level of automated testing that we can actually achieve.
Agreed, the automated tests are an ideal that we have set for ourselves as the glibc team and it doesn't actually plug into the distribution's goals for QE. The only real relationship with the distribution is that we want to leverage the infrastructure to do our automated tests so that we don't have to set up our own on sourceware or wherever else. > But from the perspective of the Fedora QA team, I don't think the best > thing we could do with our time is, you know, draw up a big list of > packages and start working down it, writing automated tests for one > package at a time. If we started doing that we might make some sort of > vaguely noticeable dent by, oh, say, 2026 or so. ;) Heh, no, the 'someone' I talked about would be one of us glibc maintainers or maybe some poor soul who thinks it would be a great project to dive into, not Fedora QE. We have been a bit lazy about it so far because (in addition to the fact that we don't have volunteers yet) a lot of the distribution level tests are reasonable sanity checks for glibc. Siddhesh -- devel mailing list devel@lists.fedoraproject.org https://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org
