Rob Cozens wrote:
I'd be interested in what Jacque Gay might have to say on the subject. I recall a very interesting conversation we had at a Macworld regarding the work she did for Apple to automate HyperCard testing.
It was a long time ago so I've forgotten a lot of the details, but this testing engine was the reason that HyperCard consistently shipped with virtually no significant bugs. The HC team would start the test engine running and leave it until it crashed, at which point they'd fix the problem and start it up again. When the engine ran all the way to completion, they figured HC was okay to ship. And it almost always was. Any bugs that surfaced later on were always small things that were well outside the range of normal operations.
There were two test engines. The main one automated the testing of every language token, and ran unsupervised. The second one was a manual test engine that required a human to execute specified tasks. The manual test stack was much smaller and tested things like manually resizing graphics, choosing tools from the palette, using keyboard shortcuts, and other behaviors that couldn't be simulated via scripts.
I didn't write the original test engine, which someone on the HC team wrote way back at version 1.0. As the HC language grew, more tests were added for new syntax and behaviors. By the time I became involved with the test engine it was massive and had been worked on over the years by many people.
Exactly how it worked is both proprietary (and protected by NDA) and vague in my memory now, but the basic idea was this. The stack script contained a handler that used a "do" statement to execute the parameters that were sent to it. Each card used this handler to execute various tests. The handler then returned a result that the calling handler could check and log to an external text file. Each card in the stack tested one token (command, function, etc) of the language. Each card had a separate handler for each variation of the token. For example, a card to test the "go" command would have handlers for going to a different stack and back, going to a stack in a new window, going to a stack with screen locked, and so forth.
Each card had an "opencard" handler with a repeat loop that ran through all the permutations on that card. A main handler in the stack script also had a repeat loop that went to each card in sequence (which triggered each opencard handler.) As each test was performed, "the result" was checked and logged to the text file. At the completion of the opencard handler, control was returned to the stack script which then moved on to the next test card.
The scripts were complex and more elaborate than I've described. Each card test had to follow a particular format and contain handlers with predefined names. There were as many provisions as possible to bypass errors without bringing the testing to a halt; errors were logged, the specific handler was exited, and the next handler in the sequence began. The logging routines listed the name of the test, the card it came from, and the final result so that a human could read through it and check for unanticipated results. If the engine crashed, the last entry in the log showed what was executing at the time so it could be tracked and fixed.
The first card of the stack had a "Start" button that was clicked to begin the test sequence. From there, it ran on its own. If not stopped, it would begin over at the first card and retest everything all over again, running until a certain keyboard sequence stopped it. I heard they sometimes ran it for weeks at a time to test memory usage.
Funny this should come up, as I was recently talking to the person I shared the work with, and we wondered whether Apple would object to us converting the test engine to Revolution for the RR team. We don't know the status of our Apple NDA at this point, since HC is long gone. Conversion, however, would be a massive job; not only does Rev has three or four times as many features as HC, but many of the existing handlers would have to be rewritten to accomodate differences in how the engines work. It would be a project that could easily take a couple of years to finish. Just writing new tests for HC 2.3/2.4 took three of us some months, and we didn't have to change any of the existing card tests, we only had to add new ones. So, the project is probably outside the realm of possibility at this point.
-- Jacqueline Landman Gay | [EMAIL PROTECTED] HyperActive Software | http://www.hyperactivesw.com _______________________________________________ use-revolution mailing list use-revolution@lists.runrev.com Please visit this url to subscribe, unsubscribe and manage your subscription preferences: http://lists.runrev.com/mailman/listinfo/use-revolution
