-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 W dniu 03.11.2010 17:22, Marc Tardif pisze: > Hi folks, > > I would appreciate your feedback about interpreting test results across > multiple test runs. For example, consider the following test results for > the first test run of a given project: > > test1: Pass > test2: Pass > test3: Fail > > Then, consider the following partial test results for a second test run > of the same project: > > test1: Pass > > The second test run only executed the first test and, for the sake of > this example, we can safely assume that this was deliberate and the > test runner process did not crash and burn. > > My question is: can we make any reasonable assumptions about the tests > that were not run? This can be a matter of opinion where one extreme > might not make any assumptions at all, whereas another extreme might > assume that test results remain the same until proven otherwise. So, > I'm calling for your opinions on what you consider is reasonable.
Hi Marc Unless you have reliable information on how to handle such condition in the test case meta data *and* can sufficiently guarantee that the meda-data is accurate and up-to-date then you should do very little more than notify the user that the particular test was not run (or not present in the test result data, I don't know how you handle that part). Perhaps if you can explain the kind of analysis you are doing then we can come up with some reasonable set of guidelines for such tests. Thanks ZK -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iQEcBAEBAgAGBQJM3HXxAAoJEKkR4hQBRI+l9sQH/iF0VD48RO35qCiLny0fHa+p jLnmfYaiEm5eTRYYVRf7Tkhen/cPH0zF0Fd6PqP/IfSQikxKMPnqRYGDM8bW+Qms DcNuEzgVNEj4fdVOTGfVbodFGcEmb4gf9Bw7pKkv5XUo068/+cs9W1TITeUOr78T 2BPVk7AHTtzC3OpSVO3KDpwP5Gf4d/PBIBot44BfVZC3IExAmg/JtiWjA/TY/Pey S5RJag0l4GYGBh5H96gkxHa90HfES5gF7i0OeLG9puNWW/PbF7SpDKIAtLO8ZJW/ 9J9A4m2pTPSWdBGpeoFmB+5/Sznucgrv3R2y+3o3UpGYbAp1NC668RmuPnaZE7E= =Wswr -----END PGP SIGNATURE----- -- Ubuntu-qa mailing list Ubuntu-qa@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-qa
