Re: TAP and Bail out!
Eric Wilhelm wrote: # from Geoffrey Young # on Tuesday 15 January 2008 11:49: what I think many are asking for is for Test::Builder to be able to issue the appropriate TAP on $condition. Where $condition is a run-time option set at the command-line, not hardcoded into the test. Writing or BAIL_OUT(...) in the test is a different thing than saying I'm going to start a test run, but I want it to bail if anything fails. The former is in effect for both the developer and the end-user/ installer. One typically wants a full report from an end-user, so you don't want to ship it with bail everywhere. This is not a command-line conditional bail, but rather the module won't even load for some reason and thus testing anything else is pointless. The latter is for developer convenience while adding some tests or changing some code. The error is often an obvious typo or something that can be fixed with only the data from the first fail. This is not really a bail out so much as a ok, I get it. Everything should stop now. Well, possibly. The bail outs you'd keep long term are in big test suites where you have tests that run early in the order (assuming non-shuffle) that suffer a failure that violates the integrity of the test suite itself. For example, if your compile test can't compile your mail module, then running the rest of the test suite is normally pointless. If the environment of configuration test can't locate configuration resources that MUST exist to run sanely (configuration management is a big deal in very large applications) then just forget about running. Or if the configuration resources exist, but the tests for your error-checking configuration module find that the config is invalid, etc etc. Adam K
Re: TAP and Bail out!
# from Adam Kennedy # on Friday 18 January 2008 00:23: Eric Wilhelm wrote: This is not a command-line conditional bail, but rather the module won't even load for some reason and thus testing anything else is pointless. ... Well, possibly. The bail outs you'd keep long term are in big test suites where you have tests that run early in the order (assuming non-shuffle) that suffer a failure that violates the integrity of the test suite itself. For example, if your compile test can't compile your mail module, then running the rest of the test suite is normally pointless. Well, yes. That's basically what I said, but I was trying to generalize to things other than mail modules ;-) Still, BAIL_OUT() is not the droid we're looking for. --Eric -- The first rule about Debian is you don't talk about Debian --- http://scratchcomputing.com ---
Re: is_deeply and qr// content on 5.11
Ian Malpass wrote: I got a failure message from CPAN testers for Pod::Extract::URI for 5.11.0 patch 33001 on Linux 2.6.22-3-amd64 (x86_64-linux-thread-multi-ld)[0] The failures were where I was testing to see if an arrayref of qr// patterns was the array I was expecting. is_deeply() has heretofore worked perfectly well. The test line in question is: is_deeply( $peu-stop_uris, [ qr/foo/ ] ); And the failure is: # Failed test at t/new.t line 42. # Structures begin differing at: # $got-[0] = (?i-xsm:foo) # $expected-[0] = (?-xism:foo) # Looks like you failed 1 test of 24. So, is this 5.11 brokenness, is_deeply() brokenness, or are my tests naive (or wrong)? (?i-xsm:foo) is equivalent to qr/foo/i. (?-xism:foo) is qr/foo/. They are not equivalent. So it's possible that 5.11 fixed/broke something, but it's inside stop_uris(). There have been issues with how is_deeply() compares regexes in the past (the ordering of the xism operators, for example) but in this case it appears to be working. -- On error resume stupid
is_deeply and qr// content on 5.11
I got a failure message from CPAN testers for Pod::Extract::URI for 5.11.0 patch 33001 on Linux 2.6.22-3-amd64 (x86_64-linux-thread-multi-ld)[0] The failures were where I was testing to see if an arrayref of qr// patterns was the array I was expecting. is_deeply() has heretofore worked perfectly well. The test line in question is: is_deeply( $peu-stop_uris, [ qr/foo/ ] ); And the failure is: # Failed test at t/new.t line 42. # Structures begin differing at: # $got-[0] = (?i-xsm:foo) # $expected-[0] = (?-xism:foo) # Looks like you failed 1 test of 24. So, is this 5.11 brokenness, is_deeply() brokenness, or are my tests naive (or wrong)? Is there a better/preferred way to compare qr// patterns? Thanks, Ian [0] http://www.nntp.perl.org/group/perl.cpan.testers/2008/01/msg969390.html
Re: is_deeply and qr// content on 5.11
On Fri, 2008-01-18 at 17:22 -0600, Jonathan Rockway wrote: On Fri, 2008-01-18 at 16:57 -0600, Ian Malpass wrote: I got a failure message from CPAN testers for Pod::Extract::URI for 5.11.0 patch 33001 on Linux 2.6.22-3-amd64 (x86_64-linux-thread-multi-ld)[0] The failures were where I was testing to see if an arrayref of qr// patterns was the array I was expecting. is_deeply() has heretofore worked perfectly well. The test line in question is: is_deeply( $peu-stop_uris, [ qr/foo/ ] ); And the failure is: # Failed test at t/new.t line 42. # Structures begin differing at: # $got-[0] = (?i-xsm:foo) # $expected-[0] = (?-xism:foo) # Looks like you failed 1 test of 24. Are you positive you didn't mean to say qr/foo/i ? Are you sure that $peu-stop_uris should be returning qr/foo/i instead of qr/foo/? Sorry to reply to myself (it's Friday, what can I say)... but under 5.8.8 and 5.10.0, this test passes: is_deeply qr/foo/i, qr/foo/ # pass Honestly, I think it's a bug in the older versions and 5.11.0 gets it right. Those two regexes aren't the same. Since there's been some discussion on p5p about regex object internals, I'll forward the message over there so they know the behavior has changed. Regards, Jonathan Rockway signature.asc Description: This is a digitally signed message part
Re: is_deeply and qr// content on 5.11
Jonathan Rockway wrote: # Failed test at t/new.t line 42. # Structures begin differing at: # $got-[0] = (?i-xsm:foo) # $expected-[0] = (?-xism:foo) # Looks like you failed 1 test of 24. Are you positive you didn't mean to say qr/foo/i ? Are you sure that $peu-stop_uris should be returning qr/foo/i instead of qr/foo/? I know this sounds obvious... but I just want to make sure. Good heavens! stop_uris() *does* make the pattern case-insensitive. How did I come to write that? Sorry, shouldn't have assumed I remembered the code I wrote, but I could have sworn it was case-sensitive. Well, well done 5.11 I suppose Now I have to decide if the case insensitivity is a feature, or a bug Ian
