MySQL + TAP == MyTAP
Stumbled across this while finding an alternative to libtap for testing C (it has some sort of issue linking with this hairy project I'm working on). Apparently MySQL wrote their own TAP library for C. From http://dev.mysql.com/doc/mysqltest/en/unit-test.html The unit-testing facility is based on the Test Anything Protocol (TAP) which is mainly used when developing Perl and PHP modules. To write unit tests for C/C++ code, MySQL has developed a library for generating TAP output from C/C++ files. Each unit test is written as a separate source file that is compiled to produce an executable. For the unit test to be recognized as a unit test, the executable file has to be of the format mytext-t. For example, you can create a source file named mytest-t.c the compiles to produce an executable mytest-t. The executable will be found and run when you execute make test or make test-unit in the distribution top-level directory. Here's the docs. http://www.kindahl.net/mytap/doc/index.html -- Defender of Lexical Encapsulation
Re: MySQL + TAP == MyTAP
Hi Michael, * Michael G Schwern [EMAIL PROTECTED] [2008-03-29 11:35]: Stumbled across this while finding an alternative to libtap for testing C (it has some sort of issue linking with this hairy project I'm working on). Apparently MySQL wrote their own TAP library for C. From http://dev.mysql.com/doc/mysqltest/en/unit-test.html The unit-testing facility is based on the Test Anything Protocol (TAP) which is mainly used when developing Perl and PHP modules. To write unit tests for C/C++ code, MySQL has developed a library for generating TAP output from C/C++ files. Each unit test is written as a separate source file that is compiled to produce an executable. For the unit test to be recognized as a unit test, the executable file has to be of the format mytext-t. For example, you can create a source file named mytest-t.c the compiles to produce an executable mytest-t. The executable will be found and run when you execute make test or make test-unit in the distribution top-level directory. Here's the docs. http://www.kindahl.net/mytap/doc/index.html I think you wanted to send this to the TAP list. :-) Regards, -- Aristotle Pagaltzis // http://plasmasturm.org/
An alternate view on deferred plans
I debated whether or not to post this here for a long time, because I gather that deferred plans are somewhat of a hot topic on this group. But finally I decided that I just needed to understand some history and make sure there's nothing I'm missing here. First, if you'll bear with me, a little background: When I first discovered test-first programming, via the original XP book, I thought it was brilliant, and I started employing it on many of my projects. However, I confess that (somewhat foolishly) I did not use Test::More and friends to do my tests. Instead, I rolled my own system. Since I have never (yet) released any of my code to CPAN, using Test::More (and friends) was not already a habit for me. And I thought to myself that the testing process for installation of a module was quite different from the testing process for developing code. Now that I'm considering finally getting off my lazy ass and putting some stuff out to CPAN, I realized that I had to deal with installation testing, and I started reading everything I could about the testing frameworks of standard Perl, TAP, etc. My conclusions are that I still think that installation testing is a _little_ different from development testing, but certainly not enough to justify having my own little world of testing libraries, most of which do exactly what Test::More et al do, only without the benefit of nearly as much aggregated programmer time put into making sure they're correct. So I set out to convert my stuff to the standard stuff. And I only hit two real snags. The first is that _during development_, I want to stop my tests on the first failure. This is the minor difference, IMHO, between installation testing and development testing. For installation, it obviously makes no sense to do this. But for development testing, it makes sense to me. (I understand that others may not feel so, but I've also noticed that some on this list agree.) When I first began this effort, there wasn't really a solution for me on CPAN, but now that Ovid has released the excellent Test::Most, that particular itch is scratched, so I don't really want to open that can of worms again. The second sticking point is the concept of plans, and here is where I really want to understand the reasoning behind it. My understanding is that a plan has one primary function: to insure that if a test script dies before all the tests are run, that is recognized as an error. Now, in my own test scripts, I achieved this via a very simple methodology: all my scripts ended with a simple print statement: ALL DONE. I ran the script, and if I saw my ALL DONE at the end, the script was fine. If I didn't, I obviously had a problem. Very simple, very clean. My implementation was a bit simplistic, perhaps, but no matter how much I've read about testing in Perl, I still can't see why the idea itself isn't sound. This idea is similar to what Ovid has been calling deferred plans, except that he has been suggesting that the _number_ of tests be submitted to the test harness at the end, whereas I'm just talking about a success flag. Now, in the original discussion back in November, Adrian Howard asked: I don't get this. Why is saying I know this test script outputs test results at the start better than saying it at the end? And Aristotle answered: So you'd defer the plan in cases where the number of tests is predetermined but maybe hard to precompute, or where it's variable. So in both cases you are calculating the number at run time, which is immediately subject to more bugs than providing a constant. Additionally, it's more likely for a bug in the calculation to line up with a bug in the corresponding test code, so that you end up with a plan that matches the number of tests run even though you *intended* to run fewer/more tests. And lastly, even a runtime-calculated predeclared plan separates the test code and calculation code at least in time (while running) and probably also in space (in the source code). Therefore it seems to me that bugs are somewhat less likely to line up. Now, I have a great respect for Aristotle (actually, I have a great respect for all the people on this list with whose work I have any previous experience), but it seems to me that all these objections relate to presenting a number at the end of the tests. My method inherently avoids those issues. And I don't want to defer the plan because the number of tests is hard to predetermine, or because it's indeterminate. I just think that having code where every time you change things in one place, you have to make sure you change something somewhere else isn't a good thing. In any other programming scenario, I think most everyone would agree with me. But when it comes to testing, doing this in terms of tests is not only okay, it's considered best practice. So I guess I'm looking for enlightenment. I can't help but feel like there's something I'm missing. If the only function
Re: An alternate view on deferred plans
On Saturday 29 March 2008 12:51:34 Buddy Burden wrote: The first is that _during development_, I want to stop my tests on the first failure. Get a better harness. The second sticking point is the concept of plans, and here is where I really want to understand the reasoning behind it. My understanding is that a plan has one primary function: to insure that if a test script dies before all the tests are run, that is recognized as an error. Now, in my own test scripts, I achieved this via a very simple methodology: all my scripts ended with a simple print statement: ALL DONE. I ran the script, and if I saw my ALL DONE at the end, the script was fine. If I didn't, I obviously had a problem. Very simple, very clean. My implementation was a bit simplistic, perhaps, but no matter how much I've read about testing in Perl, I still can't see why the idea itself isn't sound. Any solution which requires a human being to read and think about the output beyond It's all okay! or Something fell!* is not a long-term solution. In particular, you lose the separation between producing TAP and interpreting TAP, as well as the automation benefits of both. So I guess I'm looking for enlightenment. I can't help but feel like there's something I'm missing. If the only function of a test plan is to make sure all the tests are run, my method handles that, and it's simpler and requires less bookkeeping. The other function of a test plan is to make sure that you aren't running *more* tests than you intended. If you don't know why that's important, try patching File::Find sometime. I have. Not criticizing, not claiming my method is better, just looking for any reasons why this wouldn't work. And, JIC there's some agreement that it _would_ work, I've already put together a patch for Test::Most that does it. That is, at the top of your script, you put this: use Test::Most 'defer_plan'; and at the bottom of your script, you put this: all_done(); and it just DTRT. The implementation is perhaps not as clean as I'd like it to be, but it's not a horrific hack either. I'm going to forward it to Ovid through normal CPAN channels. Thoughts? Suppose you take advantage of the existence of Test::Builder and roll in several other testing modules, per good programming practice of re-using working and well-tested code. Whose responsibility is it then to emit the single canonical all_done() call? I realize that this suggests that multiple test libraries should be able to add to the plan, and I'm sort of okay with that as well. -- c * I stopped reading Cerebus at Guys, partly because it was so difficult to find new phone books and partly because the second ascension ended so
Re: An alternate view on deferred plans
chromatic, Any solution which requires a human being to read and think about the output beyond It's all okay! or Something fell!* is not a long-term solution. I don't think that's true of this implementation. If the script doesn't reach the all_done() call, there is a very obvious error. If it does, it internally produces a plan at the end--acceptable according to TAP specs--and it's very obviously okay. In particular, you lose the separation between producing TAP and interpreting TAP, as well as the automation benefits of both. I'm not sure I see what you're saying here. The other function of a test plan is to make sure that you aren't running *more* tests than you intended. If you don't know why that's important, try patching File::Find sometime. I have. Well, that is admittedly a very real disadvantage of my method, and definitely one I hadn't considered. Suppose you take advantage of the existence of Test::Builder and roll in several other testing modules, per good programming practice of re-using working and well-tested code. Whose responsibility is it then to emit the single canonical all_done() call? The all_done() is handled by each individual test script. It should never be called in a library. Test::Aggregate might have to perform some magic to make sure it only gets called once, but my impression is that Test::Aggregate is already fairly magical. Thanx for taking the time to reply. -- Buddy
Re: An alternate view on deferred plans
Buddy Burden wrote: Not criticizing, not claiming my method is better, just looking for any reasons why this wouldn't work. And, JIC there's some agreement that it _would_ work, I've already put together a patch for Test::Most that does it. That is, at the top of your script, you put this: use Test::Most 'defer_plan'; and at the bottom of your script, you put this: all_done(); and it just DTRT. The implementation is perhaps not as clean as I'd like it to be, but it's not a horrific hack either. I'm going to forward it to Ovid through normal CPAN channels. Thoughts? This method is fine and has been suggested several times before. http://rt.cpan.org/Public/Bug/Display.html?id=20959 I've just been dragging my feet on it. I finally had occasion to make use of it when I had to hand roll a C TAP library (libtap tried to be nice about threads and wound up losing). I hate writing plans, and C makes everything 3x more annoying, so I wrote a done_testing() function and just didn't bother requiring a plan at all. It was all very convenient. So maybe I'll get around to implementing it. -- Insulting our readers is part of our business model. http://somethingpositive.net/sp07122005.shtml
