dan nessett <dness...@yahoo.com> wrote:
> I am investigating how to write a comprehensive parser regression test. What
> I mean by this is something you wouldn't normally run frequently, but rather
> something that we could use to get past the "known to fail" tests now
> disabled. The problem is no one understands the parser well enough to have
> confidence that if you fix one of these tests that you will not break
> something else.
> So, I thought, how about using the guts of DumpHTML to create a comprehensive
> parser regression test. The idea is to have two versions of phase3 +
> extensions, one without the change you make to the parser to fix a
> known-to-fail test (call this Base) and one with the change (call this
> Current). Modify DumpHTML to first visit a page through Base, saving the HTML
> then visit the same page through Current and compare the two results. Do this
> for every page in the database. If there are no differences, the change in
> Current works.
> [...]
I use a similar approach on a toolserver script in addition
to smaller tests: I saved several revisions of "interesting"
wiki pages and the respective output of the then-current
script version to the subversion repository. Before commit-
ting changes, I run a test whether the current script produ-
ces the same results. If the results are different, either a
bug needs to be fixed or the expected output be amended (in
the same commit).
Tim
P. S.: I find your dedication to QA very laudable; I think
though that more people would read and embrace your
thoughts if you would find a more concise way to put
them across :-).
_______________________________________________
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
