Stefano Zacchiroli writes ("Re: Bug#693691: gzip: Enable autopkgtest, add
simple gzip test"):
> On Mon, Nov 19, 2012 at 10:07:40AM -0700, Bdale Garbee wrote:
> > So, I'm not interested in merging this patch as-is. If it can be
> > reworked to run the existing test suite on an autopkgtest testbed, then
> > I'd be willing to merge such a patch. If that's not possible, then I
> > suggest the design of autopkgtest be revisited so that we don't need to
> > replicate and/or diverge from upstream test suite creation and management.
>
> It's definitely possible. autopkgtest is an interface to run a testsuite
> and declare its presence. But a specific autopkgtest test can do
> whatever it wants, including chain-running upstream test suites. In
> fact, while I can't speak for Ian :-), I think autopkgtest has been
> designed precisely with that flexibility in mind.
Yes, exactly.
In general I think the right way to add autopkgtest support to a
package with an existing upstream test suite is normally to find a way
to cause the upstream tests to be run against the installed version.
Writing new tests should be a last resort for if the upstream test
driver framework is too hard to persuade to run things from PATH and
simply pick up the installed libraries, etc.. Or, I guess, if the
upstream test suite is a unit test for a small portion of the software
and really what's needed is something different.
> In this specific case, while I don't see anything wrong with adding some
> ad-hoc autopkgtest tests (which seem better than no testing at all!), I
> agree with you that running the upstream test suite would be even
> better.
However, like Stefano, I do think there might be some value even in
simple ad-hoc autopkgtest tests. This is because upstream tests are
typically focused on finding regressions in corner cases in the code,
whereas the most value can be got out of autopkgtest by having tests
which look for installation problems - which can be much narrower in
terms of coverage of the core upstream codebase, but conversely
typically want to be wider in other respects such as provoking plugin
loading, testing frequently-used "convenience wrapper" scripts,
interaction with other programs and the filesystem, etc.
I haven't looked at gzip in particular, so I don't know what its
upstream test suite is like.
Ian.
--
To UNSUBSCRIBE, email to [email protected]
with a subject of "unsubscribe". Trouble? Contact [email protected]
