We held a QA team meeting on 11/5/08. Here is the summary.
Why AP tests? Q: Why are we spending so much time running AP tests when the subject of APs for our international deployments is up to government bids, and not at our recommendation? A: So that we have system level test results, and also so that we can make recommendations that the deployments can choose whether or not to take. Comments: - We should start writing down test cases for these situations so that those deployments can do their own testing for AP selection. - So we know what test was for, how it was run, and why it was created. - So we can repeat the test in the future. - So we can log results and bug reports. - For AP testing, it sounds like a great idea for us to be able to provide test cases for any interested deployments so they can run the tests themselves to ensure the AP they are considering will work for them. Testing at 1CC is done so we can make recommendations and we can say we tested with x, y, and z access points. Backup/Restore testing This testing involves a 24-hour wait; trying to reproduce the bugs. Still in the middle of these tests. Comment: Backup testing should no longer involve a 24 hour wait. Scripts We have now a working "Deregister from school server" script that does the following: - Removes XS line from config file. - Deletes the network.cfg file. - Sets the .xsession login. - Sets the ds backup logging. - Copies itself into /usr/bin so that it'll be permanently on your XO, and you can run it without the USB stick. We will make a wiki section for these scripts. Wiki reconstruction Don't start it yet; collect ideas, but don't implement. Make sure we have an archive when the new architecture gets implemented. Fedora on XO It would be fine with one or two people joining the fedora-olpc test group and formally contributing a few hours/week on this testing as the olpc representative(s). Automation VNC: has been set up and running, but here are some issues: - No frame control. - Some Activities don't behave the same way when you start them from a remote machine vs. locally. Ssh scripts: Let's first simplify and focus on running just the ps command and getting results back from it. Tinderbox: has been on the back burner. Tinderbox and the ssh-scripts will let us run basically the same commands, but the tradeoff is that Tinderbox takes a long time to setup on an individual XO because you have to open it up and plug a special device into the motherboard, whereas the ssh-scripts can quickly be run on anything we have an IP for. However, Tinderbox doesn't rely on good connectivity to work. Let's put together a small laptop testbed (1 or 2 to start) with one server that can run the tinderbox scripts. We can experiment on that bed just as we are experimenting with VNC, Monitor, etc. We need to explore uses of these tools to really understand the benefits. Community test We should try to get more volunteers involved. One of the outstanding questions from the test community is how to prioritize the Activities for testing. Different opinions: - Volunteers should test activities we don't test at 1cc. We test the behavior of Activities in a particular system situation. - Volunteers should focus on testing activities that OLPC directly works on because we consider it core functionality, like Browse and Write, because we have more direct contact with the developers that make them. Browse, Write, Record, and Chat. - Connect to jabber.laptop.org [so you can test Activity behavior independent of the mesh network] because it tests both collaboration and the XS. - People with one laptop can create test cases and help write up the 'documentation' of how an activity works. Large testbed testing with only AP (no school server) connectivity We're trying to see how many machines we can get up; have just started this, running into issues but finding ways to get past them. Side note: today we experienced a 2 hr email delay because our email server was overrun by kids in Uruguay accessing our wiki. Email processing and OLPC wiki should run on different machines. New OFW firmware testing: first finish smoke tests, then move to a different testbed. _______________________________________________ Testing mailing list [email protected] http://lists.laptop.org/listinfo/testing
