Re: Test Suite Slowing Down My Development
Tels ha scritto: -BEGIN PGP SIGNED MESSAGE- Moin, On Friday 28 October 2005 22:34, Christopher H. Laco wrote: Ovid wrote: The code is designed well enough that adding new features is quick and easy. Unfortunately, whenever I need to change my code I fire up a Web server and view the results in the browser and then write the tests after I've written the code (this is closely related to When test-driven development just won't do). This is because XML and XHTML are just text. I need to see the output. I've been finding more and more that small changes in my code are making huge changes in the output and trying to continuously update the tests to exactly match the XML, XSLT and XHTML using Test::XML and XML::XPath has led to a serious productivity drop. I'm considering just using Test::WWW::Mechanize to do integration testing through a Web server I run in the tests. This will be much faster and allow me to get my development speed back up. However, I'd be skipping the unit testing of the output. I'll catch the bugs but it will likely take me longer to track them down. I feel your pain. The test suite for Handel has xml/tt output tests for its AxKit and Template Toolkit plugins. I've got oodles of template pages using the components whos output I compare to static .out files that contain the expected output. Everytime I write a new plugin, or a new tag in the plugin, I waste tons of time just writing the tests for them. So far, I've been good about writing the tests before I write the code, but it takes forever and I rarely get the tests right the first time. I'm curious to see what comes out of your question. I'm in the same boat. I am somewhat in the same boat with Graph::Easy - the t/ascii.t script tests rendering of graphs in ASCII, ala: [ A ] - [ B ] is transformed into: # echo [Test] - [This] .. [ Now ] | perl examples/as_ascii +--+ +--+ +-+ | Test | -- | This | .. | Now | +--+ +--+ +-+ While this works mostly fine for ASCII, the HTML/SVG is undertested because the text/code output can change quite radically, while still rendering/representing the same graph. And of course I do want to test that the end result is the right one, not that the generated SVG/HTML code is a specific example. If the output is valid xml, couldn't one semantically compare the expected output and the actual output with something like XML::SemanticDiff ? Just my 0.2 eurocents... So everytime I change the HTML template/output, I need to fiddle back a host of tests... Best wishes, Tels - -- Signed on Fri Oct 28 23:33:31 2005 with key 0x93B84C15. Visit my photo gallery at http://bloodgate.com/photos/ PGP key on http://bloodgate.com/tels.asc or per email. Ok, a shiny copper piece to the first person who can tell me why Elan is running around naked. - Giant moth attack? - Maybe a bad guy casted 'Dispel clothes'? - The Order of the Stick issue #26 -BEGIN PGP SIGNATURE- Version: GnuPG v1.2.4 (GNU/Linux) iQEVAwUBQ2KZ9ncLPEOTuEwVAQEZxAf/Rz/gxvA3w2bbD7D1CfT2HaFe+glj37kT WkJUQ7pj9bEZA/o74WJwpdZyL7MfTe3K6MhqohstrNDL9XxvN0C0ScSQiHlagHny zVl6gdZ9DDhxWBOzHZc5oEdrsV6qhkHkdvusADC3mTb0UZEqgE50RbgERw1Wlu1v 08O5NWmqJTI+KDujqdY5FZ6pBSGUR72eiaWw8UxN7HkSi42DP8/dmnw+23FsVTmc z3sJhNuhaNsVorMHdPHxUs9n5WuXRP/h1cemrrzpx2o3SfP4DBiDoxt0GKMdeRiB s29F8bWWQ4CLUsQqAgk9L4P7fOp00+tqDBLr/9t/ECLiVxWDFTxcWg== =b8rX -END PGP SIGNATURE- -- Marcello Romani Developer Spin s.r.l. Reggio Emilia http://www.spinsoft.it
Re: Test Suite Slowing Down My Development
-BEGIN PGP SIGNED MESSAGE- Moin, On Wednesday 02 November 2005 16:45, Marcello wrote: Tels ha scritto: -BEGIN PGP SIGNED MESSAGE- [snip] I'm considering just using Test::WWW::Mechanize to do integration testing through a Web server I run in the tests. This will be much faster and allow me to get my development speed back up. However, I'd be skipping the unit testing of the output. I'll catch the bugs but it will likely take me longer to track them down. I feel your pain. The test suite for Handel has xml/tt output tests for its AxKit and Template Toolkit plugins. I've got oodles of template pages using the components whos output I compare to static .out files that contain the expected output. Everytime I write a new plugin, or a new tag in the plugin, I waste tons of time just writing the tests for them. So far, I've been good about writing the tests before I write the code, but it takes forever and I rarely get the tests right the first time. I'm curious to see what comes out of your question. I'm in the same boat. I am somewhat in the same boat with Graph::Easy - the t/ascii.t script tests rendering of graphs in ASCII, ala: [ A ] - [ B ] is transformed into: # echo [Test] - [This] .. [ Now ] | perl examples/as_ascii +--+ +--+ +-+ | Test | -- | This | .. | Now | +--+ +--+ +-+ While this works mostly fine for ASCII, the HTML/SVG is undertested because the text/code output can change quite radically, while still rendering/representing the same graph. And of course I do want to test that the end result is the right one, not that the generated SVG/HTML code is a specific example. If the output is valid xml, couldn't one semantically compare the expected output and the actual output with something like XML::SemanticDiff ? Might be, but I am not generting XML, but HTML... And there are many ways to write code that looks visually the same, and yet is wildly different. More so for, lets say SVG. Sometimes I resort to soft tests, like looking if I find certain strings in the output, like for [ Bonn ] - [ Berlin ], you expect Bonn and Berlin in there somewhere... Best wishes, Tels - -- Signed on Wed Nov 2 17:17:05 2005 with key 0x93B84C15. Visit my photo gallery at http://bloodgate.com/photos/ PGP key on http://bloodgate.com/tels.asc or per email. Build a man a fire, and he'll be warm for a day. Set a man on fire, and he'll be warm for the rest of his life. -- Terry Pratchett -BEGIN PGP SIGNATURE- Version: GnuPG v1.2.4 (GNU/Linux) iQEVAwUBQ2jm5HcLPEOTuEwVAQFI1Af+J5xwJopUDR/eLJ/AtyeVPrAxiwcF5Mt+ ikOj8jmLI0uwQ5bqrj74Ad+XNQxTOIoiu7ItecF0aAjQRMBWBm4wp+/bsSIgFdBr 6substw3TSNYaCeQTU95IVXQV1CNcerpnuC5SVdmIE9l+PJOUAZHWSa3QJLcLD7e bFg9Fj4WlJsdc1FY9+l+qzXnfFwTSpZoF9ExvVQlBsilEJNOfcUgbHzXnqE/Kt0i GTuNHrxPse+Lt+j25Kv2vBygQ94opiH/MEVM2jIED68nUESYhGJGNPZm23gvXRy7 CQw2qdmB6mHfXw3fEYB5E99nQn0rLP45KCcdm4AkEjBuSNNkBnuzpw== =5jrY -END PGP SIGNATURE-
Re: Test Suite Slowing Down My Development
Tels ha scritto: -BEGIN PGP SIGNED MESSAGE- Moin, On Wednesday 02 November 2005 16:45, Marcello wrote: Tels ha scritto: -BEGIN PGP SIGNED MESSAGE- [snip] I'm considering just using Test::WWW::Mechanize to do integration testing through a Web server I run in the tests. This will be much faster and allow me to get my development speed back up. However, I'd be skipping the unit testing of the output. I'll catch the bugs but it will likely take me longer to track them down. I feel your pain. The test suite for Handel has xml/tt output tests for its AxKit and Template Toolkit plugins. I've got oodles of template pages using the components whos output I compare to static .out files that contain the expected output. Everytime I write a new plugin, or a new tag in the plugin, I waste tons of time just writing the tests for them. So far, I've been good about writing the tests before I write the code, but it takes forever and I rarely get the tests right the first time. I'm curious to see what comes out of your question. I'm in the same boat. I am somewhat in the same boat with Graph::Easy - the t/ascii.t script tests rendering of graphs in ASCII, ala: [ A ] - [ B ] is transformed into: # echo [Test] - [This] .. [ Now ] | perl examples/as_ascii +--+ +--+ +-+ | Test | -- | This | .. | Now | +--+ +--+ +-+ While this works mostly fine for ASCII, the HTML/SVG is undertested because the text/code output can change quite radically, while still rendering/representing the same graph. And of course I do want to test that the end result is the right one, not that the generated SVG/HTML code is a specific example. If the output is valid xml, couldn't one semantically compare the expected output and the actual output with something like XML::SemanticDiff ? Might be, but I am not generting XML, but HTML... Valid XHTML is also valid XML. And there are many ways to write code that looks visually the same, and yet is wildly different. More so for, lets say SVG. My point was that 'semantically' comparing XML data structures instead of their ascii representations is an improvement. Of course this doesn't take into account specific cases like SVG, where you have 'semantically' different xml documents producing the same (visual) output. To address this specific case one would need a way to test the 'visual' equivalence of two svg documents. Sometimes I resort to soft tests, like looking if I find certain strings in the output, like for [ Bonn ] - [ Berlin ], you expect Bonn and Berlin in there somewhere... Best wishes, Tels - -- Signed on Wed Nov 2 17:17:05 2005 with key 0x93B84C15. Visit my photo gallery at http://bloodgate.com/photos/ PGP key on http://bloodgate.com/tels.asc or per email. Build a man a fire, and he'll be warm for a day. Set a man on fire, and he'll be warm for the rest of his life. -- Terry Pratchett -BEGIN PGP SIGNATURE- Version: GnuPG v1.2.4 (GNU/Linux) iQEVAwUBQ2jm5HcLPEOTuEwVAQFI1Af+J5xwJopUDR/eLJ/AtyeVPrAxiwcF5Mt+ ikOj8jmLI0uwQ5bqrj74Ad+XNQxTOIoiu7ItecF0aAjQRMBWBm4wp+/bsSIgFdBr 6substw3TSNYaCeQTU95IVXQV1CNcerpnuC5SVdmIE9l+PJOUAZHWSa3QJLcLD7e bFg9Fj4WlJsdc1FY9+l+qzXnfFwTSpZoF9ExvVQlBsilEJNOfcUgbHzXnqE/Kt0i GTuNHrxPse+Lt+j25Kv2vBygQ94opiH/MEVM2jIED68nUESYhGJGNPZm23gvXRy7 CQw2qdmB6mHfXw3fEYB5E99nQn0rLP45KCcdm4AkEjBuSNNkBnuzpw== =5jrY -END PGP SIGNATURE- -- Marcello Romani Developer Spin s.r.l. Reggio Emilia http://www.spinsoft.it
Re: Test Suite Slowing Down My Development
On 10/28/05, Ovid [EMAIL PROTECTED] wrote: The code is designed well enough that adding new features is quick and easy. Unfortunately, whenever I need to change my code I fire up a Web server and view the results in the browser and then write the tests after I've written the code (this is closely related to When test-driven development just won't do). This is because XML and XHTML are just text. I need to see the output. I've been finding more and more that small changes in my code are making huge changes in the output and trying to continuously update the tests to exactly match the XML, XSLT and XHTML using Test::XML and XML::XPath has led to a serious productivity drop. I'm in a similar boat, and I do the same thing. It's pretty easy to the idiot tests -- XML well-formed-ness and XHTML validity -- but testing the content of the output is obviously more difficult. At the very least, you need a mockup of your expectations, and creating that mockup by hand is *much* more error-prone and tedious than grabbing the initial output. Especially so when things like whitespace sensitivity enter into the discussion. While this isn't test-first development, it works well with the test-early-test-often mindset. Instead of testing your expectations (which are quickly met or dashed by visual inspection) you refocus on regression testing. So while you're not proving that your initial output is exactly as you expect, you *are* proving that your tests are the canary in the coal mine when you mess up. The other legs of this three-legged stool are version control and brute force. The project I'm working on at the moment involves parsing queries and converting them into SQL (involving a bizarre and baroque set of business rules). The only way I can keep my sanity here is to md5^Wcheck each and every input and every output to make sure that nothing changes. The only way for these tests to be meaningful is to be rigorous -- identify all possible things to test, and generate all possible combinations and permutations. When you break a corner case, you'll know. (HOP really helps here -- iterators, odometers and such really help when writing a few dozen lines of Perl that generate a few thousand inputs.) Keep in mind that all of your sources and tests are in subversion, so when things break, the regression two-step is a quick 'diff -b' and 'diff' to see if the changes are meaningful and/or expected. When those checks fail, there's a bug. (Alternatively, you may want to run a constant pre-processor on your test data to normalize it, but that's both trickier and riskier.) Of course, be ready to throw out previous test results when the expectations change radically. ;-) HTH, Z.
Re: Test Suite Slowing Down My Development
-BEGIN PGP SIGNED MESSAGE- Moin, On Wednesday 02 November 2005 18:02, Marcello wrote: Tels ha scritto: -BEGIN PGP SIGNED MESSAGE- While this works mostly fine for ASCII, the HTML/SVG is undertested because the text/code output can change quite radically, while still rendering/representing the same graph. And of course I do want to test that the end result is the right one, not that the generated SVG/HTML code is a specific example. If the output is valid xml, couldn't one semantically compare the expected output and the actual output with something like XML::SemanticDiff ? Might be, but I am not generting XML, but HTML... Valid XHTML is also valid XML. But valid XHTML doesn't work well in the praxis. (It offers nothing what HTML can't do, but creates more problems) And it still doesn't solve the problem, because a: * div with 1px border * table cell with 1px border look exactly in the browser, but have much different (X)HTML source. And there are many ways to write code that looks visually the same, and yet is wildly different. More so for, lets say SVG. My point was that 'semantically' comparing XML data structures instead of their ascii representations is an improvement. Of course this doesn't take into account specific cases like SVG, where you have 'semantically' different xml documents producing the same (visual) output. To address this specific case one would need a way to test the 'visual' equivalence of two svg documents. The same for HTML documents. Hm. Maybe one could convert them to PNG and compare the outputs. But this is also hard. *thinks* Best wishes, Tels - -- Signed on Wed Nov 2 18:06:15 2005 with key 0x93B84C15. Visit my photo gallery at http://bloodgate.com/photos/ PGP key on http://bloodgate.com/tels.asc or per email. We're confident that DNF will be one of the greatest, if not the greatest, game of 1998. And this confidence is not misplaced. -Scott Miller, 1997 (http://tinyurl.com/6m8nh) -BEGIN PGP SIGNATURE- Version: GnuPG v1.2.4 (GNU/Linux) iQEVAwUBQ2jy+ncLPEOTuEwVAQGOTQf9FDcLaYVVoYdgfq41pcqQK9avi1oqaqmp GKKMVnYT1PsDkhYxxgCj2MYJd7+nVZ+v5n+nW0PN2pjfHzPvDOq5bIHTbqqZaJ4s k14oUBk0JRLx78k9jQORdWhGbgWR2vAtcRtN6thI1IqANRGNX9kMQscSPl1gVaEG Wwz/NYY6JJJ5+DWZORa3xOPp3daE4lFY+pJl2ENSmcpqNBLlMBPm2/ajw0aa6Zkv Iz0TOh3pyXxzPwT7c4lP3kwdySc1yikUftan753yvAErCEaFmXEFbNYZE8L2xal5 o/d+aQ1uBh1r2ycCG918y6z04CL9Y5QA0GRW1ztKFbLLhSXA+5Sq4A== =M0v6 -END PGP SIGNATURE-
Re: Test Suite Slowing Down My Development
Tels ha scritto: -BEGIN PGP SIGNED MESSAGE- Moin, On Wednesday 02 November 2005 18:02, Marcello wrote: Tels ha scritto: -BEGIN PGP SIGNED MESSAGE- While this works mostly fine for ASCII, the HTML/SVG is undertested because the text/code output can change quite radically, while still rendering/representing the same graph. And of course I do want to test that the end result is the right one, not that the generated SVG/HTML code is a specific example. If the output is valid xml, couldn't one semantically compare the expected output and the actual output with something like XML::SemanticDiff ? Might be, but I am not generting XML, but HTML... Valid XHTML is also valid XML. But valid XHTML doesn't work well in the praxis. (It offers nothing what HTML can't do, but creates more problems) And it still doesn't solve the problem, because a: * div with 1px border * table cell with 1px border look exactly in the browser, but have much different (X)HTML source. Testing 'visual equivalency' of two ([x]html|svg|whatever) documents is unfeasible IMHO. We need a compromise, like deciding to use a div or a table to produce some visual effect. This is required anyway, because while it's true that there are a dozen ways to generate the same page, they will differ wildly on some relevant aspects (e.g. browser compatibility, table- or css- based layout, etc.), so you have to make a decision at some point and choose one. And there are many ways to write code that looks visually the same, and yet is wildly different. More so for, lets say SVG. My point was that 'semantically' comparing XML data structures instead of their ascii representations is an improvement. Of course this doesn't take into account specific cases like SVG, where you have 'semantically' different xml documents producing the same (visual) output. To address this specific case one would need a way to test the 'visual' equivalence of two svg documents. The same for HTML documents. Hm. Maybe one could convert them to PNG and compare the outputs. But this is also hard. *thinks* Interesting... Best wishes, Tels - -- Signed on Wed Nov 2 18:06:15 2005 with key 0x93B84C15. Visit my photo gallery at http://bloodgate.com/photos/ PGP key on http://bloodgate.com/tels.asc or per email. We're confident that DNF will be one of the greatest, if not the greatest, game of 1998. And this confidence is not misplaced. -Scott Miller, 1997 (http://tinyurl.com/6m8nh) -BEGIN PGP SIGNATURE- Version: GnuPG v1.2.4 (GNU/Linux) iQEVAwUBQ2jy+ncLPEOTuEwVAQGOTQf9FDcLaYVVoYdgfq41pcqQK9avi1oqaqmp GKKMVnYT1PsDkhYxxgCj2MYJd7+nVZ+v5n+nW0PN2pjfHzPvDOq5bIHTbqqZaJ4s k14oUBk0JRLx78k9jQORdWhGbgWR2vAtcRtN6thI1IqANRGNX9kMQscSPl1gVaEG Wwz/NYY6JJJ5+DWZORa3xOPp3daE4lFY+pJl2ENSmcpqNBLlMBPm2/ajw0aa6Zkv Iz0TOh3pyXxzPwT7c4lP3kwdySc1yikUftan753yvAErCEaFmXEFbNYZE8L2xal5 o/d+aQ1uBh1r2ycCG918y6z04CL9Y5QA0GRW1ztKFbLLhSXA+5Sq4A== =M0v6 -END PGP SIGNATURE- -- Marcello Romani Developer Spin s.r.l. Reggio Emilia http://www.spinsoft.it
Re: Test Suite Slowing Down My Development
-BEGIN PGP SIGNED MESSAGE- Moin Marcello, On Wednesday 02 November 2005 18:36, Marcello wrote: Tels ha scritto: -BEGIN PGP SIGNED MESSAGE- Moin, On Wednesday 02 November 2005 18:02, Marcello wrote: Tels ha scritto: [snip] Might be, but I am not generting XML, but HTML... Valid XHTML is also valid XML. But valid XHTML doesn't work well in the praxis. (It offers nothing what HTML can't do, but creates more problems) And it still doesn't solve the problem, because a: * div with 1px border * table cell with 1px border look exactly in the browser, but have much different (X)HTML source. Testing 'visual equivalency' of two ([x]html|svg|whatever) documents is unfeasible IMHO. We need a compromise, like deciding to use a div or a table to produce some visual effect. This is required anyway, because while it's true that there are a dozen ways to generate the same page, they will differ wildly on some relevant aspects (e.g. browser compatibility, table- or css- based layout, etc.), so you have to make a decision at some point and choose one. Well, yes I did. :) The problem arose when I switched from one table cell per node to 3x3 table cells per node, and again when I switched theway CSS classes were used and again when I changed them again etc. Each time the source changed, but the basic output remained the same - and yet I had to adapt dozends of test. That was the original problem, and so far I haven't found a way to really avoid it. But the discussion has been interesting :) best wishes, Tels - -- Signed on Wed Nov 2 20:07:47 2005 with key 0x93B84C15. Visit my photo gallery at http://bloodgate.com/photos/ PGP key on http://bloodgate.com/tels.asc or per email. To be beautiful is enough! If a woman can do that well who should demand more from her? You don't want a rose to sing. -- Thackeray -BEGIN PGP SIGNATURE- Version: GnuPG v1.2.4 (GNU/Linux) iQEVAwUBQ2kPEHcLPEOTuEwVAQH4Mwf+J8zZ/e7IEOVl0qwRRPCWhQT0UA+NgH4Y P0+WY8cowWbQx23+p6/HAZTuMFOygH8aAfeCioXlbV7OskeI0vD+Zevp0iPhT1uk xVf7McV2yu1qOVDko9zgLr+LrrkMgqUeINPiyqBUCemobKjhyr7c3jX6AfXvz5l5 0UukfGc89y9sSsuxlgVMAMr1RgbekQmcA3sctPiDYVkpHLbvM/I/Gzpi2+hEuDCa JDASjcKKxYHO8NMD25ba4Wxttu/0C8Gh+h15XY+zgNFAP/p5pRFeF62qXz1m5jcg thYUO+QjBLd5hmuusoH6SNNGhX7rv6ltZD2wtomuY5GGbO0VVrW/XQ== =JwwY -END PGP SIGNATURE-
Re: Test Suite Slowing Down My Development
Tels ha scritto: -BEGIN PGP SIGNED MESSAGE- Moin Marcello, On Wednesday 02 November 2005 18:36, Marcello wrote: Tels ha scritto: -BEGIN PGP SIGNED MESSAGE- Moin, On Wednesday 02 November 2005 18:02, Marcello wrote: Tels ha scritto: [snip] Might be, but I am not generting XML, but HTML... Valid XHTML is also valid XML. But valid XHTML doesn't work well in the praxis. (It offers nothing what HTML can't do, but creates more problems) And it still doesn't solve the problem, because a: * div with 1px border * table cell with 1px border look exactly in the browser, but have much different (X)HTML source. Testing 'visual equivalency' of two ([x]html|svg|whatever) documents is unfeasible IMHO. We need a compromise, like deciding to use a div or a table to produce some visual effect. This is required anyway, because while it's true that there are a dozen ways to generate the same page, they will differ wildly on some relevant aspects (e.g. browser compatibility, table- or css- based layout, etc.), so you have to make a decision at some point and choose one. Well, yes I did. :) The problem arose when I switched from one table cell per node to 3x3 table cells per node, and again when I switched theway CSS classes were used and again when I changed them again etc. Each time the source changed, but the basic output remained the same - and yet I had to adapt dozends of test. Ehm... did I say make a choice ? ;-) That was the original problem, and so far I haven't found a way to really avoid it. But the discussion has been interesting :) It's been interesting for me too. Thanks. :-) best wishes, Tels - -- Signed on Wed Nov 2 20:07:47 2005 with key 0x93B84C15. Visit my photo gallery at http://bloodgate.com/photos/ PGP key on http://bloodgate.com/tels.asc or per email. To be beautiful is enough! If a woman can do that well who should demand more from her? You don't want a rose to sing. -- Thackeray -BEGIN PGP SIGNATURE- Version: GnuPG v1.2.4 (GNU/Linux) iQEVAwUBQ2kPEHcLPEOTuEwVAQH4Mwf+J8zZ/e7IEOVl0qwRRPCWhQT0UA+NgH4Y P0+WY8cowWbQx23+p6/HAZTuMFOygH8aAfeCioXlbV7OskeI0vD+Zevp0iPhT1uk xVf7McV2yu1qOVDko9zgLr+LrrkMgqUeINPiyqBUCemobKjhyr7c3jX6AfXvz5l5 0UukfGc89y9sSsuxlgVMAMr1RgbekQmcA3sctPiDYVkpHLbvM/I/Gzpi2+hEuDCa JDASjcKKxYHO8NMD25ba4Wxttu/0C8Gh+h15XY+zgNFAP/p5pRFeF62qXz1m5jcg thYUO+QjBLd5hmuusoH6SNNGhX7rv6ltZD2wtomuY5GGbO0VVrW/XQ== =JwwY -END PGP SIGNATURE- -- Marcello Romani Developer Spin s.r.l. Reggio Emilia http://www.spinsoft.it
Test Suite Slowing Down My Development
Hi all, I suspect that most of us who do test-driven development will probably argue that tests do not slow down our development or, better still, will speed up our development. Usually I find this is true for myself, but not now. I could really use feedback on http://perlmonks.org/index.pl?node_id=503758. In short, I need to use a different strategy to get around the problems outlined in that node (reproduced below). Cheers, Ovid -- One of the less appreciated benefits of test suites is how they speed up development. It seems counter-intuitive as you're writing twice the amount of code, but the experience of many programmers has been that test suites lead to better interfaces, less time spent tracking down bugs and, due to easier refactoring, a smaller code base that's easier to work with. I personally find that I'm getting much more work done now that I write solid test suites. Until now. The code is designed well enough that adding new features is quick and easy. Unfortunately, whenever I need to change my code I fire up a Web server and view the results in the browser and then write the tests after I've written the code (this is closely related to When test-driven development just won't do). This is because XML and XHTML are just text. I need to see the output. I've been finding more and more that small changes in my code are making huge changes in the output and trying to continuously update the tests to exactly match the XML, XSLT and XHTML using Test::XML and XML::XPath has led to a serious productivity drop. I'm considering just using Test::WWW::Mechanize to do integration testing through a Web server I run in the tests. This will be much faster and allow me to get my development speed back up. However, I'd be skipping the unit testing of the output. I'll catch the bugs but it will likely take me longer to track them down. -- If this message is a response to a question on a mailing list, please send follow up questions to the list. Web Programming with Perl -- http://users.easystreet.com/ovid/cgi_course/
Re: Test Suite Slowing Down My Development
Ovid wrote: The code is designed well enough that adding new features is quick and easy. Unfortunately, whenever I need to change my code I fire up a Web server and view the results in the browser and then write the tests after I've written the code (this is closely related to When test-driven development just won't do). This is because XML and XHTML are just text. I need to see the output. I've been finding more and more that small changes in my code are making huge changes in the output and trying to continuously update the tests to exactly match the XML, XSLT and XHTML using Test::XML and XML::XPath has led to a serious productivity drop. I'm considering just using Test::WWW::Mechanize to do integration testing through a Web server I run in the tests. This will be much faster and allow me to get my development speed back up. However, I'd be skipping the unit testing of the output. I'll catch the bugs but it will likely take me longer to track them down. I feel your pain. The test suite for Handel has xml/tt output tests for its AxKit and Template Toolkit plugins. I've got oodles of template pages using the components whos output I compare to static .out files that contain the expected output. Everytime I write a new plugin, or a new tag in the plugin, I waste tons of time just writing the tests for them. So far, I've been good about writing the tests before I write the code, but it takes forever and I rarely get the tests right the first time. I'm curious to see what comes out of your question. I'm in the same boat. -=Chris smime.p7s Description: S/MIME Cryptographic Signature
Re: Test Suite Slowing Down My Development
--- Christopher H. Laco [EMAIL PROTECTED] wrote: I feel your pain. The test suite for Handel has xml/tt output tests for its AxKit and Template Toolkit plugins. I've got oodles of template pages using the components whos output I compare to static .out files that contain the expected output. Everytime I write a new plugin, or a new tag in the plugin, I waste tons of time just writing the tests for them. So far, I've been good about writing the tests before I write the code, but it takes forever and I rarely get the tests right the first time. It might be nice if you posted this comment to http://perlmonks.org/index.pl?node_id=503758 If others can see that this is not an unusual problem it might help. Cheers, Ovid -- If this message is a response to a question on a mailing list, please send follow up questions to the list. Web Programming with Perl -- http://users.easystreet.com/ovid/cgi_course/
Re: Test Suite Slowing Down My Development
-BEGIN PGP SIGNED MESSAGE- Moin, On Friday 28 October 2005 22:34, Christopher H. Laco wrote: Ovid wrote: The code is designed well enough that adding new features is quick and easy. Unfortunately, whenever I need to change my code I fire up a Web server and view the results in the browser and then write the tests after I've written the code (this is closely related to When test-driven development just won't do). This is because XML and XHTML are just text. I need to see the output. I've been finding more and more that small changes in my code are making huge changes in the output and trying to continuously update the tests to exactly match the XML, XSLT and XHTML using Test::XML and XML::XPath has led to a serious productivity drop. I'm considering just using Test::WWW::Mechanize to do integration testing through a Web server I run in the tests. This will be much faster and allow me to get my development speed back up. However, I'd be skipping the unit testing of the output. I'll catch the bugs but it will likely take me longer to track them down. I feel your pain. The test suite for Handel has xml/tt output tests for its AxKit and Template Toolkit plugins. I've got oodles of template pages using the components whos output I compare to static .out files that contain the expected output. Everytime I write a new plugin, or a new tag in the plugin, I waste tons of time just writing the tests for them. So far, I've been good about writing the tests before I write the code, but it takes forever and I rarely get the tests right the first time. I'm curious to see what comes out of your question. I'm in the same boat. I am somewhat in the same boat with Graph::Easy - the t/ascii.t script tests rendering of graphs in ASCII, ala: [ A ] - [ B ] is transformed into: # echo [Test] - [This] .. [ Now ] | perl examples/as_ascii +--+ +--+ +-+ | Test | -- | This | .. | Now | +--+ +--+ +-+ While this works mostly fine for ASCII, the HTML/SVG is undertested because the text/code output can change quite radically, while still rendering/representing the same graph. And of course I do want to test that the end result is the right one, not that the generated SVG/HTML code is a specific example. So everytime I change the HTML template/output, I need to fiddle back a host of tests... Best wishes, Tels - -- Signed on Fri Oct 28 23:33:31 2005 with key 0x93B84C15. Visit my photo gallery at http://bloodgate.com/photos/ PGP key on http://bloodgate.com/tels.asc or per email. Ok, a shiny copper piece to the first person who can tell me why Elan is running around naked. - Giant moth attack? - Maybe a bad guy casted 'Dispel clothes'? - The Order of the Stick issue #26 -BEGIN PGP SIGNATURE- Version: GnuPG v1.2.4 (GNU/Linux) iQEVAwUBQ2KZ9ncLPEOTuEwVAQEZxAf/Rz/gxvA3w2bbD7D1CfT2HaFe+glj37kT WkJUQ7pj9bEZA/o74WJwpdZyL7MfTe3K6MhqohstrNDL9XxvN0C0ScSQiHlagHny zVl6gdZ9DDhxWBOzHZc5oEdrsV6qhkHkdvusADC3mTb0UZEqgE50RbgERw1Wlu1v 08O5NWmqJTI+KDujqdY5FZ6pBSGUR72eiaWw8UxN7HkSi42DP8/dmnw+23FsVTmc z3sJhNuhaNsVorMHdPHxUs9n5WuXRP/h1cemrrzpx2o3SfP4DBiDoxt0GKMdeRiB s29F8bWWQ4CLUsQqAgk9L4P7fOp00+tqDBLr/9t/ECLiVxWDFTxcWg== =b8rX -END PGP SIGNATURE-
