There are a bunch of CSS test suites available that we ought to be testing against Gecko. In many cases, it would be quite a bit of work to automate them (and in some cases they may not be licensed appropriately for modification), but it would still be very useful to run them occasionally so that we find and fix the bugs these tests show. In some ways, fixing these bugs is more important than fixing bugs we find ourselves, since these are tests that other browser vendors are likely to be using as well, and therefore when all browsers fix the bugs in them, authors can depend on what these tests are testing. (That said, we should also be trying to share our tests better.)
Going through existing CSS test suites is actually a good way to get started in testing CSS. (It's actually how I got started.) The eventual goal of this activity is to get bugs filed on all the test failures that are legitimate bugs. The question is how to do that efficiently without creating tons of duplicates, given that test suites can be repetitive, and in many cases the bugs are filed already. (It's also good to avoid invalid bugs where the tests are wrong or out-of-date, which is pretty common... especially filing the same invalid bugs multiple times.) So I'd suggest something like the following approach would be most useful: To start, go through some set of tests to find the ones that fail, and write a Web page, somewhat like one of the following: http://lists.w3.org/Archives/Public/public-css-testsuite/2008Mar/0002.html http://dbaron.org/css/browsers/nglay that lists which tests fail and groups the failures together if they seem to be caused by the same bug. Then try to find existing bug reports that correspond to those failures and list those on the page if you can find them (or, if the tests are incorrect, say so in the page). Then post the page to this list and bug somebody like me to look at it and see if it seems correct, and once you get comments on it, file new bug reports for those failures that aren't covered by existing bug reports and which are valid bugs (and then add the bug reports to the page). It might be useful to do this somewhere on wiki.mozilla.org so multiple people can edit. Existing CSS test suites that we should be doing this on include: http://www.hixie.ch/tests/adhoc/ (covers more than just CSS) http://www.w3.org/Style/CSS/Test/CSS2.1/current/ (make sure to include the big Microsoft contribution which might not be in the current "build" of the test suite, and other not-yet-vetted tests) http://annevankesteren.nl/test/ (includes more than just CSS) http://dbaron.org/css/test/ I think the first two are the highest priority. -David -- L. David Baron http://dbaron.org/ Mozilla Corporation http://www.mozilla.com/ _______________________________________________ dev-tech-layout mailing list [email protected] https://lists.mozilla.org/listinfo/dev-tech-layout
