Hello, Thanks again for your feedback.
On Tue, Aug 23, 2016 at 11:30:07AM +0200, Santiago Vila wrote: > On Mon, 22 Aug 2016, Sean Whitton wrote: > > > 1. flycheck tends to FTBFS because the test suite contains many tests > > which encode expected output from ~30 command line tools. When one of > > those tools sees a new upstream version which tweaks its output, the > > test breaks. > > Perhaps encoding the expected output (to the letter) is not a good > idea, then. Assuming those are standard Unix tools, they usually follow > some standard, which tells them how the output should be. A well > designed test should be prepared to check any output which follow this > standard, not the exact stream of bytes produced by a certain fixed > version. I don't think any of them would count as "standard unix tools": they're language-specific linters whose output does not follow any defined standard. I don't personally have any experience designing a test suite. If you can see some better way to ensure that flycheck is working as intended, I'd encourage you to start a discussion on upstream's bug tracker[1] -- if there's a better way to test flycheck, I'd like to know what it would look like. > > 2. The test suite also fails under certain autobuilder setups. In > > particular, the reproducible builds team's automated builders fail > > with mysterious permission errors. I think it is something to do > > with their tmpfs setup -- though I use tmpfs with sbuild and can't > > reproduce the error. > > We are working with computers, which are designed to do that we tell > them to do. There are not really "mysterious errors", only errors > which nobody has investigated yet. > > In particular, a build failure is usually one of two things: Either a > bug in the program (even if it does not always happen), or a bug in > the building software (I use sbuild, they use pbuilder). > > So, I would encourage you to contact them to see if there are any bugs > in pbuilder (or their setup in general). Yes, I've been in touch with them (in relation to a different package of mine whose test suite was failing in a similar way), but we haven't got to the bottom of it.[2] [1] https://github.com/flycheck/flycheck/issues [2] http://lists.alioth.debian.org/pipermail/reproducible-builds/Week-of-Mon-20160425/005352.html -- Sean Whitton
signature.asc
Description: PGP signature
