Hi Werner, Werner LEMBERG wrote on Sun, Jun 22, 2014 at 09:01:46AM +0200:
> What we are discussing right now is not bug handling per se, > but checking the current integrity of the git groff repository, > ensuring that it simply compiles and install. Actually, to be honest, i am more worried about potential subtle issues being introduced into the groff code base than about outright build system crashes. Lately, we are seing a considerable number of large commits shuffling stuff around that are inherently hard to review. At the same time, we are seing that a large fraction of commits contains blatant issues, so they were apparently insufficiently tested before commit. Some of these issues are fixed shortly afterwards (anybody remember the commit exchanging the arguments of Perl push()?). But what about those that aren't? Or what makes you think that no subtle issues are being introduced, at a rate of the same order of magnitude as the obvious, blatant issues we are all seeing? That frequent build system breakage is just annoying (in particular when i'm the first one to run into it ;-) but it is 100% sure all build system breakage will be found before people actually run the software in production, because, well, because the build breaks. What we should really worry about is the integrity of groff code at places where malfunction is less obvious. > Ingo has automated this process with his packaging script, and he > is already sending patches. What i'm automating is mostly the application of OpenBSD-specific patches. I'm doing the following by hand: * From a git checkout: - make distclean - ./configure - export AUTOMAKE_VERSION=1.14 - export AUTOCONF_VERSION=2.69 - make dist - cp groff*tgz somewhere else * From the port directory: - make clean='all dist' - rm distinfo - make fetch - make makesum - make package # which effectively just runs the steps patch, # configure, build, fake install, package - sudo pkg_add -r the_new_package Not much automation is needed to do basic QA. Of course, i'm not at all opposed to unit testing. It can be a very valuable supplement to manual testing. However, no kind of unit testing or even automated integration testing can replace careful development practices and careful manual testing. You see, the Zope project once released a fully automatically tested stable production release where everything worked perfectly - except that, well, if you tried to start the server program *outside* the testing framework, like you would in production, it crashed instantly, every time. Yours, Ingo