On Sun, Mar 26, 2017 at 11:25 PM, Peter Goldstein <pe...@valimail.com> wrote:
> Second, the premise that "it's not hard to write test code..." is simply > not true. What would be required to actually write such code would be > either a) pick a preferred ordering/formatting for these tags, and only > have the tests pass for that ordering or b) include a complete ARC > verifying system in the test code. The first is what OpenDKIM did for its > specs (see below). The latter is possible, but seems ill-advised for a > test suite. > OpenDKIM does both. At some point when it was still "dkim-milter", when it came time to write tests (because at one point there were none), I mainly wanted to assure that commits made didn't cause any regressions; correctness could be affirmed in other ways like at interop events or through other implementers' auto-responders. So I picked a tag ordering I liked, generated a bunch of signatures with it, and then wrote tests that did two things with them: 1) Confirmed that the signing code produces those specific signatures given fixed inputs (timestamp, key, selector, domain, canonicalization, etc.). This was inherently a tautology at the time I generated them of course, but my goal was just to ensure no regressions were introduced. I didn't know for sure that the output was right, but I knew it would be consistent. 2) Since OpenDKIM conveniently includes a verifying capability, there are some tests that instantiate the library to produce a signature, then re-instantiate the library to verify it. Obviously this doesn't prove that its signatures are correct, but they do prove that its code is consistent with itself, and again that changes made later don't break verifying. None of this was done with an eye toward making my test suite applicable to other implementations. It's only for my own. What we're talking about here is different: coming up with tests that could be applied to any implementation, not to a specific one, so long as they agree on fixed syntax and inputs. To see that this is the case, it's useful to look at two earlier email > standards and their corresponding test suites. SPF has a language-agnostic > test suite (http://www.openspf.org/Test_Suite) that makes it easy to test > implementations in any language. I've done it myself, and it requires > minimal effort - https://github.com/ValiMail/ > coppertone/tree/master/spec/open_spf. I can tell you from personal > experience that the test suite was really helpful in ensuring the > implementation was correct in all cases. > An inherent difference between SPF and DKIM is that the order of the tokens in SPF is significant. In DKIM, reordering tags changes nothing about the meaning of the header field, though obviously it alters the resulting header hash. In contrast, the order of tags in SPF is necessarily specified. -MSK
_______________________________________________ dmarc mailing list dmarc@ietf.org https://www.ietf.org/mailman/listinfo/dmarc
