-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Titus Brown wrote: | Other than that, I'm interested in figuring out how to encourage the | OLPC project to build some automated tests without doing it all myself | :). Thoughts welcome. | | Basically I think it's incredibly important for the OLPC to invest in | automated tests.
There are many people in and around OLPC who feel the same. There are also a few who are exceptionally familiar with Tinderbox, and for a time we had a tinderbox running tests on every build, hooked up to an actual XO to maximize realism. The closest I'm aware of to actual GUI testing was ensuring that Sugar started successfully. See http://wiki.laptop.org/go/Tinderbox_Testing . | As you can guess from the "death spiral" post, I'm not terribly | interested in *debating* the usefulness of automated tests, and I don't | buy into the notion that GUI automation is impossible, either. I would | very much like to try to introduce systematic and robust GUI testing | into the OLPC and I will be working towards that end as time and | resources permit. Constructive comments and genuine interest are | welcome! You will get a warmer welcome if you take a more positive tone. Personally, I am skeptical, for several reasons: 0. We already know what the bugs are. The problem is not new bugs, and it's not regressions. The problem is bugs that we've known about for a long time that are just not easy to fix. 1. Regressions are rare, and they are not the main problem. At least 90% of OLPC's problems fall into the category of unimplemented functionality, not really bugs at all. Recently, a number of features have been removed because we finally did enough testing to discover underlying bugs, but these were not regressions. 2. Many, and perhaps most, of OLPC's remaining difficult bugs are related to the network. They are most commonly related to the closed wireless firmware, which is buggy and lacks key features regarding mesh routing and multicast. 3. Almost all of OLPC's major bugs are Heisenbugs. They often don't appear at all with only one laptop, and appear rarely until one has 12 or more laptops sharing a wireless mesh. One way to prove me wrong would be to filter the ~6800 bugs in Trac into "Regression", "Non-regression", and "Other". I think you will find that there are very few regressions. Another perfectly good way to prove me wrong is to build a complete testing suite and start spotting regressions, but that's more difficult. -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.7 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFH6vYXUJT6e6HFtqQRAv5/AKCgG/nTudD44HmCVkXSgPy76Ds+KgCfTqer lncEU/e0rFBDXiXkG8EW1UY= =zpnk -----END PGP SIGNATURE----- _______________________________________________ Testing mailing list [email protected] http://lists.laptop.org/listinfo/testing
