Hi Branden, > My personal test procedures, I think, adequately do this for man(7); > every time I'm about to push I render all of our man pages (about 60 > source documents) to text and compare them to my cache of the ones I > rendered the last time I pushed.
Yes, that's good as a lone developer. Making it a hurdle for others could be nice to have. > I think there is a risk here of confounding macro package and > formatter problems with output driver problems. All should be tested, > but not necessarily together except for inputs designed as integration > tests. I think a distinction between us is I'm not talking about designed inputs. ‘.DS .bp .DE’ isn't a typical designed input. > With the benefit of a few years experience, I would claim that our > defect rate in output drivers is pretty low compared to that in the > formatter and (particularly) macro packages. Yes, I'd have thought that likely. Though fuzzing device drivers would be fun. :-) > That is why the tests I've written have demonstrated an increasing > bent toward use of "groff -Z" and groff "-a"; these produce > device-independent output and "an abstract preview of output", > respectively. troff's output is device dependent, as I just mentioned in another thread, but I know what you mean. $ sdiff -s -w64 <(troff <<<foo) <(troff -TX100 <<<bar) x T ps | x T X100 x res 72000 1 1 | x res 100 1 1 s10000 | s10 V12000 | V16 H72000 | H100 tfoo | cb07a07rh5 n12000 0 | n16 0 V792000 | V1100 $ I agree such formats are useful for written tests. > I reiterate though, that the bugs we tend to encounter are detectable > before getting to the output driver. The encountered bugs are, yes. There are the bugs unseen. The aim of formatting a corpus to pixels would be to quickly test a growing set of real-world documents. It would be cheap to add another document. The output of a preprocessor, troff, or a device driver may change intentionally. Eyeballing those changes for the corpus would be tedious and error prone. The pixels intentionally change less often. And eyeballing pixels to see the nature of the change tends to be quick compared to comprehending what a diff at a stage of the pipeline represents. So a corpus diffed as pixels serves a different purpose to hand-written coverage or regression tests. Just as fuzzing attacks from yet another angle. > Several ways to skin this cat. :) Yes. I'd be tempted to have a standard encoding which gives a readable rendering but compresses two or more blank lines. awk ' !length { b++; next } b == 1 { print "" } b > 1 { print "-" b } { b = 0 } /^-[0-9]/ { printf "-" } 1 ' One could also highlight or encode tabs or end-of-line white-space to make it obvious to the reader and protect it from incorrect change. -- Cheers, Ralph.