Re: Test::Harness Conditional Test Execution
On Mon, 25 Mar 2002, Michael G Schwern wrote: 2. Skip test of code where dependencies have been tested and found to be failing. For example, if the test for the database connection module fails, nothing that uses that module should have its test run. (The tests should of course be reported as skipped.) Not really a Test::Harness thing, it's up to your test program to do this. Test::More has facilities to skip blocks of tests or whole test files. They're explained in the Test::Tutorial. Hmm. So you're saying that I should modify my tests to skip themselves when they get information that a module they depend on has failed its tests? How do you propose I get that information to the test? You can use the callback interface, but it's not stable and is subject to change. In fact, it probably will. I still encourage people to use it. Well, since I need it for my current method of working, I'll just use it and be aware that it will change. cjs -- Curt Sampson [EMAIL PROTECTED] +81 90 7737 2974 http://www.netbsd.org Don't you know, in this new Dark Age, we're all light. --XTC
Re: Test::Harness Conditional Test Execution
On Tue, Mar 26, 2002 at 01:47:38PM +0900, Curt Sampson wrote: On Mon, 25 Mar 2002, Michael G Schwern wrote: 2. Skip test of code where dependencies have been tested and found to be failing. For example, if the test for the database connection module fails, nothing that uses that module should have its test run. (The tests should of course be reported as skipped.) Not really a Test::Harness thing, it's up to your test program to do this. Test::More has facilities to skip blocks of tests or whole test files. They're explained in the Test::Tutorial. Hmm. So you're saying that I should modify my tests to skip themselves when they get information that a module they depend on has failed its tests? How do you propose I get that information to the test? You can't. Not easily. About all you can do is abort the whole suite of tests using the Bail Out feature of Test::Harness should a Really Important Thing fail. print Bail out! Database connection failed\n; and Test::Harness won't run any further test scripts. Anything of a finer granularity rapidly gets difficult. You can use the callback interface, but it's not stable and is subject to change. In fact, it probably will. I still encourage people to use it. Well, since I need it for my current method of working, I'll just use it and be aware that it will change. Make sure to give me any feedback about it to I can expand it according to people's needs. I'll try to at least document it next release. -- Michael G. Schwern [EMAIL PROTECTED]http://www.pobox.com/~schwern/ Perl Quality Assurance [EMAIL PROTECTED] Kwalitee Is Job One GOD made us funky!
Re: Test::Harness Conditional Test Execution
On Mon, Mar 25, 2002 at 04:35:52PM +0900, Curt Sampson wrote: I'm setting a unit testing system for a pile of code right now which is going to expand into hundreds of tests. I've currently got a simple program that finds all of the tests in the system and hands this list to Test::Harness to be run. However, I'd like to be able to do two things: 1. Run the tests in dependency order. That is, if module A uses module B, the tests for module B should be executed before those for module A. Test::Harness will run the files in whatever order you give it to them. So assuming a single module's sanity tests are in a single file you can simply feed them in in a rough dependency order. 2. Skip test of code where dependencies have been tested and found to be failing. For example, if the test for the database connection module fails, nothing that uses that module should have its test run. (The tests should of course be reported as skipped.) Not really a Test::Harness thing, it's up to your test program to do this. Test::More has facilities to skip blocks of tests or whole test files. They're explained in the Test::Tutorial. This should make it easier to fix problems because we'll see right away where we need to start fixing things, and we won't have to go through reams of test failures to find the one low level module failure that caused them all. I'm not very familiar with Test::Harness usage, and I'm wondering if someone could suggest a good way of doing this. I've looked at examples/mini_harness.plx; is using those private methods really the suggested way to go about things? No. It uses an undocumented, experimental callback interface and some private formatting functions from Test::Harness which I'm not quite sure yet how to move into Straps. mini_harness is just a proof-of-concept to see if I could reimplement most of Test::Harness's interface. You can use the callback interface, but it's not stable and is subject to change. In fact, it probably will. I still encourage people to use it. Also, from looking at the code, I get the impression that analyze_file is supposed to run the perl file given to it as an argument. But the docs say otherwise: Like analyze, but it reads from the given $test_file. Well, analyze reads test results, so presumbly the documentation is saying that this is supposed to be a file containing the test results, not the test code. Hmmm. Nope. Your original impression is correct. Docs are just poorly worded. Just touched it up. -- Michael G. Schwern [EMAIL PROTECTED]http://www.pobox.com/~schwern/ Perl Quality Assurance [EMAIL PROTECTED] Kwalitee Is Job One If you have to shoot, shoot! Don't talk. -- Tuco, The Good, The Bad And The Ugly
