From: Uri Guttman >>>>> "BM" == Bob McConnell <r...@cbord.com> writes:
> BM> From: Philip Potter > >> On 6 April 2010 16:52, Bob McConnell <r...@cbord.com> wrote: > >>> I have a test harness set up with a series of Selenium test scripts. > >>> Each script tests a specific scenario on my web site. But I have some > >>> scenarios that I want to test multiple times with different data > BM> entered > >>> each time. Currently I am using environment parameters to pass the > BM> test > >>> data into each script. But that doesn't work for multiple passes, > BM> since > >>> I have no way to save the base data and restore it after modifying > BM> it. > >>> > >> [snip] > >>> > >>> How can I make this loop through some tests multiple times with > >>> different sets of data each time? > > BM> After a great deal of reading and speculation, it looks like YAML is the > BM> way to go. I will have to move the loop inside each test script, and > BM> iterate thorough the records read. The biggest problem is that the plan > BM> is no longer fixed, so I do lose some auditability, but I don't see a > BM> cleaner way to do this. > > i call this table driven testing. i have done it for several of my > modules which required many variations on a theme. check out the test > code in Sort::Maker. it does a double level loop on each test with > different data and different arguments (sort methods). it doesn't need > external files as the args are in a perl data structure. the concept is > very simple and easy to implement. it needs to be customized for your > testing but you can steal the overall design from there. Working examples are good, I'll take a look at that. But this is for integration test, not a unit test. > BM> I will end up with a test.yml file (which I didn't even know existed > BM> when I got the referenced message) defining an array of hashes, each > BM> with input for one iteration of the test. I'll loop over the whole > BM> scenario, stepping through the array until it is gone. > > you can even use perl itself. just write a perl data structure and > call do on it. it can be an anon array or hash which is the returned > value. no need to learn yaml or load another module. you can also > slurp/eval the file (which is what do does anyhow). or do what i did and > put the data in the test file, write a shared driver module which you > load and then you write a simple wrapper to run the tests. also you can > still do a plan. all you need to do is prescan the data to count all the > tests (and variations) and send that to the test plan. i do that too. But QA techs and managers won't be able to read or extend the data in Perl. What attracted me to YAML specifically was that other less technical folks should be able to modify and add test cases. Once I am done with this, it will get checked into Perforce and may eventually become part of a Hudson managed regression test suite. I will add more test scripts, but they will need to maintain the individual cases. In some situations, I expect they will want to completely rewrite the input data set using data sets exported from other related systems. > BM> Then the next step will be to figure out how to do fuzz testing on those > BM> forms. The response to submitting a form will change with each incorrect > BM> input, which will present a challenge in how to detect which response is > BM> correct and how to react to each one. > > you write pairs of inputs and expected outputs in your data. not tricky > at all. in my structures i have input data, golden sort code (which is > supposed to generate expected results) or expected results. you then > compare the expected to actual results for the ok() call. this is very > easy for almost any kind of test where you could call ok(). The problem is that I can't define a one to one pairing. There are multiple fields in the forms that interact, so one value in a field may cause different results depending on the values in one or more of the other fields. For example, if I enter a valid VISA card number but select Discover from a drop down, whether everything else is correct or not, I should get a specific error. When they match correctly and I enter a valid but incorrect zip code for that card, I get a different error. Even with the correct zip code, an incorrect street address will produce another error. (No, I haven't figured out what happens if both the zip and address are wrong. I assume one of them takes precedence, but the vendor's document doesn't say.) I just can't see any logical way to map all of these interactions. But thank you for the suggestions and pointers. I have a lot more to learn before I resolve these problems. Bob McConnell -- To unsubscribe, e-mail: beginners-unsubscr...@perl.org For additional commands, e-mail: beginners-h...@perl.org http://learn.perl.org/
