Re: Should Test2 maintain $! and $@?

[Kent Fredric]

On 12 January 2016 at 13:53, Chad Granum  wrote:
> $! and $@ are altered by many many things, adding { local ... } around all
> of them is a pain

As much as I agree, and as much as this is a "thing in all perl, so we
should expect this problem from every module"

If I was to offer a counter example, consider the effort of a `*.t`
file needing to preserve these things instead.

> ok(do_something_scary());
> is($!, 0, "expected $! val");
> is($@, undef, '$@ not changed');

vs

> my ( $error, $exception );
> ok(do {
>   local $@;
>   local $!;
>   my $ret  = do_something_scary());
>   ( $error, $exception ) = ($!, $@);
>   $ret
> });
> is($error, 0, "expected $! val");
> is($exception, undef, '$@ not changed);

I'm not sure I'd consider the second of these an "improvement".


-- 
Kent

KENTNL - https://metacpan.org/author/KENTNL

Should Test2 maintain $! and $@?

[Chad Granum]

Test::More/Test::Builder work VERY hard to ensure nothing inside them
alters $! or $@. This is for thing like this:

ok(do_something_scary());
> is($!, 0, "expected $! val");
> is($@, undef, '$@ not changed');


Without Test::More/Builder being careful to support this, the second 2
assertions could fail because something inside ok() modifies $! or $@.

*I cannot change Test::Builder/More* they must continue to protect $! and
$@, otherwise things break (I have a few downstream samples to show that it
does).

However. It is easy enough to have Test::Builder/More protect $! and $@
without requiring Test2 to do so.

Since Test2 is new, and nothing depends on any of its behaviors yet, I am
considering having Test2 not care about modifying $! and $@. So far I have
been taking care to preserve $! and $@, but it does add significant
complexity (and minor, but noticeable slowdown) to Test2.

*Reasons for dropping this promise from Test2:*

   - Simplifies code
   - $! and $@ are altered by many many things, adding { local ... } around
   all of them is a pain
   - Sometimes internals you don't expect to set $! will
   - Perl itself documents that you cannot depend on $! and $@ being
   unchanged past the immediate line after you set it.
   - Test::Builder will continue to protect $! and $@, so nothing will break
   - Test2 is new, nothing depends on it preserving these

*Reasons not to drop it:*

   - It is helpful to people who might not realize $! and $@ are often
   altered unexpectedly.

I am asking for input in case I missed any reasons/arguments for either
side. In either case backwards compatibility will be preserved.

-Chad

Re: Should Test2 maintain $! and $@?

[Ricardo Signes]

* Chad Granum  [2016-01-11T19:53:15]
> However. It is easy enough to have Test::Builder/More protect $! and $@
> without requiring Test2 to do so.

I don't have really strong feelings here, but the general reaction on p5p over
the years to "some core library changes $! needlessly" is:  $! is only reliable
(a) *immediately* after a routine that uses $! to communicate and (b) if that
routine returns in error.  In some rare cases it's worth maintaining that.
Mostly, we say "too bad."

-- 
rjbs


signature.asc
Description: Digital signature

Re: Should Test2 maintain $! and $@?

[Karen Etheridge]

> Test2, the dist, is just internals. It provides no tools. It does not
have ok(), is(), etc.

Um, so... what *is* Test2 then? (And the second question would be: and what
does it have to do with Test::More?)  Without context, your first question
is equivalent to "should Foo::Bar maintain $! and $@?".


On Mon, Jan 11, 2016 at 7:14 PM, Chad Granum  wrote:

> Some things I forgot to mention:
>
> Test2, the dist, is just internals. It provides no tools. It does not have
> ok(), is(), etc. What I am talking about is not thr simple task of putting
> local $! In some exports. This is localizing $! Around any call to use,
> require, open, close, etc. Anything that can alter a handle, write a file,
> etc.
>
> The reason for these extreme measures is because that is what
> Test::Builder does. Test::More does not localize $! In ok() and is(), it
> instead has protections scattered about its internals.
>
> So my proposal is not to write tools that modify $!, it is to spare the
> internals from all these hurdles and let the tools do the $! Protection if
> they need it.
>
> Now, lots of things depend on Test::Builder jumping through these hoops,
> so I will ensure Test::Builder still protects $!.
>
> The question is, do I accomplish this by wrapping everything that
> canchange $! In all the Test2 internals, or do I do it by having
> Test::Builder protect them when it calls out to Test2?  The latter option
> would reduce the complexity of Test2, but not break things.
>
> As for tools, yes, preserving $! Is a valuable behavior, but I think it
> belongs at the tool level, not the internals.
>
> That said, it just occured to me that this can possibly be accomplished by
> having a context store $! And $@ when it is obtained, then restore them
> when it is released, which would avoid needing to use local everywhere, and
> still preserve them for all tools automatically...
> On Jan 11, 2016 4:53 PM, "Chad Granum"  wrote:
>
>> Test::More/Test::Builder work VERY hard to ensure nothing inside them
>> alters $! or $@. This is for thing like this:
>>
>> ok(do_something_scary());
>>> is($!, 0, "expected $! val");
>>> is($@, undef, '$@ not changed');
>>
>>
>> Without Test::More/Builder being careful to support this, the second 2
>> assertions could fail because something inside ok() modifies $! or $@.
>>
>> *I cannot change Test::Builder/More* they must continue to protect $!
>> and $@, otherwise things break (I have a few downstream samples to show
>> that it does).
>>
>> However. It is easy enough to have Test::Builder/More protect $! and $@
>> without requiring Test2 to do so.
>>
>> Since Test2 is new, and nothing depends on any of its behaviors yet, I am
>> considering having Test2 not care about modifying $! and $@. So far I have
>> been taking care to preserve $! and $@, but it does add significant
>> complexity (and minor, but noticeable slowdown) to Test2.
>>
>> *Reasons for dropping this promise from Test2:*
>>
>>- Simplifies code
>>- $! and $@ are altered by many many things, adding { local ... }
>>around all of them is a pain
>>- Sometimes internals you don't expect to set $! will
>>- Perl itself documents that you cannot depend on $! and $@ being
>>unchanged past the immediate line after you set it.
>>- Test::Builder will continue to protect $! and $@, so nothing will
>>break
>>- Test2 is new, nothing depends on it preserving these
>>
>> *Reasons not to drop it:*
>>
>>- It is helpful to people who might not realize $! and $@ are often
>>altered unexpectedly.
>>
>> I am asking for input in case I missed any reasons/arguments for either
>> side. In either case backwards compatibility will be preserved.
>>
>> -Chad
>>
>>
>>

Re: Should Test2 maintain $! and $@?

[Chad Granum]

The magic is already there, but it is there by localizing the vars in a ton
of non obvious places. Between that and having it done in one obvious and
documented place seems like a Much better option.
On Jan 11, 2016 7:21 PM, "Kent Fredric"  wrote:

> On 12 January 2016 at 16:14, Chad Granum  wrote:
> > That said, it just occured to me that this can possibly be accomplished
> by
> > having a context store $! And $@ when it is obtained, then restore them
> when
> > it is released, which would avoid needing to use local everywhere, and
> still
> > preserve them for all tools automatically...
>
>
> As written, that suggestion scares me slightly, because its behaviour
> wouldn't be entirely obvious that it does that.
>
> I'd be fine with a system like:
>
> my $error_state = save_errors();
> ...
> restore_errors($error_state);
> return $value;
>
> Or
>
>return preserve_errors(sub{
>/* code that can generate errors that shouldn't leak */
>});
>
> Just lumping it in with the "Context" object seems too magical.
>
> --
> Kent
>
> KENTNL - https://metacpan.org/author/KENTNL
>

Re: Should Test2 maintain $! and $@?

[Chad Granum]

Some things I forgot to mention:

Test2, the dist, is just internals. It provides no tools. It does not have
ok(), is(), etc. What I am talking about is not thr simple task of putting
local $! In some exports. This is localizing $! Around any call to use,
require, open, close, etc. Anything that can alter a handle, write a file,
etc.

The reason for these extreme measures is because that is what Test::Builder
does. Test::More does not localize $! In ok() and is(), it instead has
protections scattered about its internals.

So my proposal is not to write tools that modify $!, it is to spare the
internals from all these hurdles and let the tools do the $! Protection if
they need it.

Now, lots of things depend on Test::Builder jumping through these hoops, so
I will ensure Test::Builder still protects $!.

The question is, do I accomplish this by wrapping everything that canchange
$! In all the Test2 internals, or do I do it by having Test::Builder
protect them when it calls out to Test2?  The latter option would reduce
the complexity of Test2, but not break things.

As for tools, yes, preserving $! Is a valuable behavior, but I think it
belongs at the tool level, not the internals.

That said, it just occured to me that this can possibly be accomplished by
having a context store $! And $@ when it is obtained, then restore them
when it is released, which would avoid needing to use local everywhere, and
still preserve them for all tools automatically...
On Jan 11, 2016 4:53 PM, "Chad Granum"  wrote:

> Test::More/Test::Builder work VERY hard to ensure nothing inside them
> alters $! or $@. This is for thing like this:
>
> ok(do_something_scary());
>> is($!, 0, "expected $! val");
>> is($@, undef, '$@ not changed');
>
>
> Without Test::More/Builder being careful to support this, the second 2
> assertions could fail because something inside ok() modifies $! or $@.
>
> *I cannot change Test::Builder/More* they must continue to protect $! and
> $@, otherwise things break (I have a few downstream samples to show that it
> does).
>
> However. It is easy enough to have Test::Builder/More protect $! and $@
> without requiring Test2 to do so.
>
> Since Test2 is new, and nothing depends on any of its behaviors yet, I am
> considering having Test2 not care about modifying $! and $@. So far I have
> been taking care to preserve $! and $@, but it does add significant
> complexity (and minor, but noticeable slowdown) to Test2.
>
> *Reasons for dropping this promise from Test2:*
>
>- Simplifies code
>- $! and $@ are altered by many many things, adding { local ... }
>around all of them is a pain
>- Sometimes internals you don't expect to set $! will
>- Perl itself documents that you cannot depend on $! and $@ being
>unchanged past the immediate line after you set it.
>- Test::Builder will continue to protect $! and $@, so nothing will
>break
>- Test2 is new, nothing depends on it preserving these
>
> *Reasons not to drop it:*
>
>- It is helpful to people who might not realize $! and $@ are often
>altered unexpectedly.
>
> I am asking for input in case I missed any reasons/arguments for either
> side. In either case backwards compatibility will be preserved.
>
> -Chad
>
>
>

Re: Should Test2 maintain $! and $@?

[Chad Granum]

https://github.com/Test-More/Test2/issues/9

Issue created to do this the easy/efficient way.
On Jan 11, 2016 7:14 PM, "Chad Granum"  wrote:

> Some things I forgot to mention:
>
> Test2, the dist, is just internals. It provides no tools. It does not have
> ok(), is(), etc. What I am talking about is not thr simple task of putting
> local $! In some exports. This is localizing $! Around any call to use,
> require, open, close, etc. Anything that can alter a handle, write a file,
> etc.
>
> The reason for these extreme measures is because that is what
> Test::Builder does. Test::More does not localize $! In ok() and is(), it
> instead has protections scattered about its internals.
>
> So my proposal is not to write tools that modify $!, it is to spare the
> internals from all these hurdles and let the tools do the $! Protection if
> they need it.
>
> Now, lots of things depend on Test::Builder jumping through these hoops,
> so I will ensure Test::Builder still protects $!.
>
> The question is, do I accomplish this by wrapping everything that
> canchange $! In all the Test2 internals, or do I do it by having
> Test::Builder protect them when it calls out to Test2?  The latter option
> would reduce the complexity of Test2, but not break things.
>
> As for tools, yes, preserving $! Is a valuable behavior, but I think it
> belongs at the tool level, not the internals.
>
> That said, it just occured to me that this can possibly be accomplished by
> having a context store $! And $@ when it is obtained, then restore them
> when it is released, which would avoid needing to use local everywhere, and
> still preserve them for all tools automatically...
> On Jan 11, 2016 4:53 PM, "Chad Granum"  wrote:
>
>> Test::More/Test::Builder work VERY hard to ensure nothing inside them
>> alters $! or $@. This is for thing like this:
>>
>> ok(do_something_scary());
>>> is($!, 0, "expected $! val");
>>> is($@, undef, '$@ not changed');
>>
>>
>> Without Test::More/Builder being careful to support this, the second 2
>> assertions could fail because something inside ok() modifies $! or $@.
>>
>> *I cannot change Test::Builder/More* they must continue to protect $!
>> and $@, otherwise things break (I have a few downstream samples to show
>> that it does).
>>
>> However. It is easy enough to have Test::Builder/More protect $! and $@
>> without requiring Test2 to do so.
>>
>> Since Test2 is new, and nothing depends on any of its behaviors yet, I am
>> considering having Test2 not care about modifying $! and $@. So far I have
>> been taking care to preserve $! and $@, but it does add significant
>> complexity (and minor, but noticeable slowdown) to Test2.
>>
>> *Reasons for dropping this promise from Test2:*
>>
>>- Simplifies code
>>- $! and $@ are altered by many many things, adding { local ... }
>>around all of them is a pain
>>- Sometimes internals you don't expect to set $! will
>>- Perl itself documents that you cannot depend on $! and $@ being
>>unchanged past the immediate line after you set it.
>>- Test::Builder will continue to protect $! and $@, so nothing will
>>break
>>- Test2 is new, nothing depends on it preserving these
>>
>> *Reasons not to drop it:*
>>
>>- It is helpful to people who might not realize $! and $@ are often
>>altered unexpectedly.
>>
>> I am asking for input in case I missed any reasons/arguments for either
>> side. In either case backwards compatibility will be preserved.
>>
>> -Chad
>>
>>
>>

Re: Should Test2 maintain $! and $@?

[Kent Fredric]

On 12 January 2016 at 16:14, Chad Granum  wrote:
> That said, it just occured to me that this can possibly be accomplished by
> having a context store $! And $@ when it is obtained, then restore them when
> it is released, which would avoid needing to use local everywhere, and still
> preserve them for all tools automatically...


As written, that suggestion scares me slightly, because its behaviour
wouldn't be entirely obvious that it does that.

I'd be fine with a system like:

my $error_state = save_errors();
...
restore_errors($error_state);
return $value;

Or

   return preserve_errors(sub{
   /* code that can generate errors that shouldn't leak */
   });

Just lumping it in with the "Context" object seems too magical.

-- 
Kent

KENTNL - https://metacpan.org/author/KENTNL

Re: Should Test2 maintain $! and $@?

[Chris Nandor]

I agree, Kent.  Shouldn't it be simple to wrap ok() and is() accordingly,
to simply save-and-restore $! and $@, without going deep into anything?
There's already a working example of how to do that right here.

sub ok {
  my($err, $exception) = ($!, $@);
  # ...
  ($!, $@) = ($err, $exception);
  return $ret;
}

And if ok() accepts a code reference to execute, fine ... same principle,
for whenever you execute the code.  It doesn't seem like it's a lot of
work, though I imagine I am probably missing something.  That said, I
generally do something like you have, except I don't put it in a do(), I
just execute it external to the test function:

  local($@, $!);
  my $ret  = do_something_scary();
  ($error, $exception) = ($!, $@);
  ok($ret);
  is($err, 0);
  is($exception, undef);

It's not as nice, but it's safe.  And if I call it multiple times, I
usually put it in my own wrapper:

  sub my_test {
local($@, $!);
my($ref) = @_;
my $ret  = $ref->();
($error, $exception) = ($!, $@);
ok($ret);
is($err, 0);
is($exception, undef);
  }

or whatever.  Again ... it's not ideal, but if I am calling it multiple
times, and I want to check for no errors, even if $@ and $! are protected,
I'd still want to have a separate function ... and then the overhead isn't
a big deal.  So, I guess my point is "whatever."  :-)


On Mon, Jan 11, 2016 at 5:34 PM, Kent Fredric  wrote:

> On 12 January 2016 at 13:53, Chad Granum  wrote:
> > $! and $@ are altered by many many things, adding { local ... } around
> all
> > of them is a pain
>
> As much as I agree, and as much as this is a "thing in all perl, so we
> should expect this problem from every module"
>
> If I was to offer a counter example, consider the effort of a `*.t`
> file needing to preserve these things instead.
>
> > ok(do_something_scary());
> > is($!, 0, "expected $! val");
> > is($@, undef, '$@ not changed');
>
> vs
>
> > my ( $error, $exception );
> > ok(do {
> >   local $@;
> >   local $!;
> >   my $ret  = do_something_scary());
> >   ( $error, $exception ) = ($!, $@);
> >   $ret
> > });
> > is($error, 0, "expected $! val");
> > is($exception, undef, '$@ not changed);
>
> I'm not sure I'd consider the second of these an "improvement".
>
>
> --
> Kent
>
> KENTNL - https://metacpan.org/author/KENTNL
>