Re: Proposed (optional) kwalitee metric; use re 'taint'

2008-06-24 Thread chromatic
On Monday 23 June 2008 22:57:26 Paul Fenwick wrote:

 As such, I'd like to propose a new (optional) kwalitee metric.  Like using
 strict and warnings, modules should use re 'taint' at the top of the code
 as well.  Even modules that exist to untaint data *should* be doing this,
 as it means that regexps that do the untainting are more clearly marked. 
 For modules that don't mean to be doing untainting, this is an extra
 safeguard to make sure they don't.

 So:

   * Does anyone think this is a bad idea?

Absolutely.

I'm not a fan of having to write code to do nothing in C++ (so that the C++ 
compiler doesn't do the wrong thing by default), and I'd hate to see that bad 
habit enter Perl, even if the reason is so that someone gets to move ahead 
one slot on a scoreboard somewhere.

I also hate to get bug reports saying You should add this line of code even 
if it does nothing useful to all of your modules, because it's not there!

I'm happy to get bug reports and test reports which say Your distribution 
doesn't extract or Your permissions are wrong or Your MANIFEST doesn't 
match your packing list because those are real problems and I can correct 
them.

I'm very much not happy to get bug reports and test failures and big red bars 
against my distributions because an automated heuristic which applies only to 
some cases decided that it knows better about how to write code than I do.  
If the heuristic is so smart about what makes good code, maybe it should 
maintain my code and free me up for a hobby with fewer stupid automated 
distractions.

(Want a heuristic which finds *actual* bugs in almost every module ever 
written?  Check the use of eval and subsequent $@ testing, or the use of 
ref(), or SUPER::whatever(), for example.)

 Is there someplace this should be going besides from CPANTS?
 It's definitely a common mistake that module authors can easily fix.

Perl::Critic?

-- c


Re: Proposed (optional) kwalitee metric; use re 'taint'

2008-06-24 Thread Thomas Klausner
Hi!

On Tue, Jun 24, 2008 at 03:57:26PM +1000, Paul Fenwick wrote:

   * What's the procedure of getting this added to the CPANTS list
 of kwalitee metrics?

Either you wait until I have tuits (but to tell the truth there are some 
more issues to fix with CPANTS before I can add more metrics), or:

* ask me for a comit bit for http://code.google.com/p/cpants/
* add the new metric to Module::CPANTS::Analyse
  I would put it into Module::CPANTS::Kwalitee::Uses (although the 
  current code does not report what's in the import list, i.e you can 
  see that 're' is uses, but not that it's used with 'taint'...)
* commit (or send a patch)
* kick me to deploy the new metric on the server and to do a --force run 
  (testing all CPAN dists)

(yay, and another slide for my YAPC::Europe talk on CPANTS is nearly done :-)


-- 
#!/usr/bin/perl  http://domm.plix.at
for(ref bless{},just'another'perl'hacker){s-:+-$-gprint$_.$/}


Re: Proposed (optional) kwalitee metric; use re 'taint'

2008-06-24 Thread Paul Fenwick

G'day chromatic / QA,

chromatic wrote:

I also hate to get bug reports saying You should add this line of code even 
if it does nothing useful to all of your modules, because it's not there!


Unfortunately while the code may do nothing useful to *your* modules per se, 
it's extremely useful should any be using your modules in a program that 
uses taint mode and wants to be careful about their data.  It's rather 
awkward that Perl has ended up with the same code that commonly means parse 
this data to implicitly mean this data is safe.


There's definitely a place for something that changes the default behaviour 
of perl to *not* automatically untaint when using regexps, but I haven't 
even looked into how difficult such a piece of code would be.


I'm very much not happy to get bug reports and test failures and big red bars 
against my distributions because an automated heuristic which applies only to 
some cases decided that it knows better about how to write code than I do. 


One could argue this is the case for all the CPANTS tests.  Why should your 
code use strict, or warnings, or Test::Pod::Coverage, or have its tests in 
t/ ?  These are all just automated heuristics telling me how to write my code.


However I certainly take your point that we can go overboard with CPANTS 
heuristics.



Is there someplace this should be going besides from CPANTS?
It's definitely a common mistake that module authors can easily fix.


Perl::Critic?


Perl::Critic is an excellent suggestion, if nothing else because I'd love to 
apply the critic tests to my own code, even if they're not inflicted on the 
rest of the world.  ;)


Thanks for the excellent feedback,

Paul

--
Paul Fenwick [EMAIL PROTECTED] | http://perltraining.com.au/
Director of Training   | Ph:  +61 3 9354 6001
Perl Training Australia| Fax: +61 3 9354 2681


Re: Proposed (optional) kwalitee metric; use re 'taint'

2008-06-24 Thread chromatic
On Tuesday 24 June 2008 00:50:27 Paul Fenwick wrote:

  I also hate to get bug reports saying You should add this line of code
  even if it does nothing useful to all of your modules, because it's not
  there!

 Unfortunately while the code may do nothing useful to *your* modules per
 se, it's extremely useful should any be using your modules in a program
 that uses taint mode and wants to be careful about their data.  It's rather
 awkward that Perl has ended up with the same code that commonly means
 parse this data to implicitly mean this data is safe.

I find it difficult to believe that you've audited all of my 
publicly-available code in the past 90 minutes to know exactly how much even 
uses regular expressions, let alone on data that could possibly come from 
tainted sources.  I believe I write from a position of authority about code 
where I'm the, well, author.

Quality (or even Kwalitee) suggests *removing* dead code, not adding it.

-- c


Re: Proposed (optional) kwalitee metric; use re 'taint'

2008-06-24 Thread chromatic
On Tuesday 24 June 2008 00:50:27 Paul Fenwick wrote:

I forgot to respond to another portion of this post; I apologize for double 
responses.

  I'm very much not happy to get bug reports and test failures and big red
  bars against my distributions because an automated heuristic which
  applies only to some cases decided that it knows better about how to
  write code than I do.

 One could argue this is the case for all the CPANTS tests.  Why should your
 code use strict,

Because it helps me as an author.

 or warnings,

Because it helps me as an author, and some of my code uses lexical warnings, 
which help my users.

 or Test::Pod::Coverage,

It doesn't (and soon won't), because it's almost always useless for the code I 
write.  When I spend more time tweaking the exception list to skip internal 
subroutines which are not and never will be part of the public API but help 
me write cleaner, more maintainable, more understandable code than I spend 
actually writing the code, Test::Pod::Coverage is actually *harmful*, and it 
must go away (from the code I write).

If a heuristic says I should use it, the heuristic is wrong.

(I'm aware of the argument which says You can work around a wrong heuristic 
by piling hack upon hack and writing unnatural, ugly code but I hope the 
problem with that fallacy is superlatively obvious.  I could waste my time 
and clutter my source code with workarounds trying to make flawed heuristics 
at least not do the wrong thing, or I could use my brain and experience to 
look at and reason about shorter, clearer code and spend my time doing 
something more valuable.  It seems a matter of bad policy to encourage other 
authors to waste their time and energy on minutiae.)

 or have its tests in t/ ?

Because it helps me as an author and it helps my users.

 These are all just automated heuristics telling me how to write my 
 code.

Some of them are even *useful*.  I like those.  The useless distracty ones and 
the actively harmful ones -- not so much.

-- c


Re: Proposed (optional) kwalitee metric; use re 'taint'

2008-06-24 Thread Aristotle Pagaltzis
* chromatic [EMAIL PROTECTED] [2008-06-24 08:30]:
 On Monday 23 June 2008 22:57:26 Paul Fenwick wrote:
  * Does anyone think this is a bad idea?
 
 Absolutely.
 
  Is there someplace this should be going besides from CPANTS?
  It's definitely a common mistake that module authors can
  easily fix.
 
 Perl::Critic?

Took the words out of my, well, fingers.

Regards,
-- 
Aristotle Pagaltzis // http://plasmasturm.org/


Re: Proposed (optional) kwalitee metric; use re 'taint' / Per-author tests?

2008-06-24 Thread Paul Fenwick

G'day chromatic,

chromatic wrote:

I find it difficult to believe that you've audited all of my 
publicly-available code in the past 90 minutes to know exactly how much even 
uses regular expressions,


My original suggestion clearly should have come with some qualifications.

Obviously modules which don't use regular expressions at all aren't going to 
be untainting data by accident through regexp matches.  Under such a 
theoretical test, these modules can be excluded by default.


 let alone on data that could possibly come from tainted sources.

This has me confused.

As the user of a module, it's possible for me to pass in tainted data.  The 
module doesn't know from where it's been sourced.  However, unless the 
*intent* of the module is to untaint this data, anything derived from that 
data should probably remain tainted.  Likewise, unless it's the purpose of 
the module is untaint incoming data, anything the module reads from an 
external source should probably also remain tainted.


At the moment, the default behaviour of regular expressions makes is very 
easy to untaint by accident, which is unfortunate.  I even remember (a very 
long time ago) when CGI.pm used to untaint by accident, which was extremely 
unfortunate.  Most module authors don't even think about taint mode when 
writing their code, because most module authors don't which makes catching 
this sort of behaviour that much more difficult.


Yes, taint mode isn't an iron-clad guarantee of security, and if you don't 
trust a module, don't use it.  However taint mode can be a useful safety 
net, and for me it would be nice if more people were aware of it and how it 
interacts with their code.


Some of them are even *useful*.  I like those.  The useless distracty ones and 
the actively harmful ones -- not so much.


This is the core of the whole automated kwalitee metrics debate, isn't it? 
The metrics are subjective, one person's useful heuristic is another 
person's painful annoyance.


It's clear that my idea is going to easily fall into the painful annoyance 
category, so I'm withdrawing the proposal to add it as an optional CPANTS 
test for now.


On that note, surely we could save a lot of anguish with regards to many of 
the CPAN tests just by making the optional ones[1] actually optional?  As a 
completely off-the-bat suggestion that could be controlled by META.yml:


cpants:
disable:
- has_test_pod_coverage
- uses_no_re_taint
- valid_gpg_siganture

enable:
- included_in_slackware
- won_poetry_competition
- includes_Tolkein_quote

My apologies if this has already been discussed and I've missed it.

Cheerio,

Paul

[1] Or even all of them?

--
Paul Fenwick [EMAIL PROTECTED] | http://perltraining.com.au/
Director of Training   | Ph:  +61 3 9354 6001
Perl Training Australia| Fax: +61 3 9354 2681


Re: Proposed (optional) kwalitee metric; use re 'taint'

2008-06-24 Thread David Golden
On Tue, Jun 24, 2008 at 3:50 AM, Paul Fenwick [EMAIL PROTECTED] wrote:
 Unfortunately while the code may do nothing useful to *your* modules per se,
 it's extremely useful should any be using your modules in a program that
 uses taint mode and wants to be careful about their data.  It's rather
 awkward that Perl has ended up with the same code that commonly means parse
 this data to implicitly mean this data is safe.

Except use re q/taint/ is lexical.  So if some module isn't itself
reading data from a potentially tainted source, then it really doesn't
need to bother with this.  That's not the same as strict and warnings,
which always apply to my code.

I'm kind of against encouraging more cargo-culted incantations, even
if they are a good practice when appropriate.  And there are a lot of
applications of perl where security *isn't* a primary requirement and
I don't think that we should set the bar of Kwalitee that high.

I like the suggestion of a Perl::Critic module, as then people can
decide whether they are working on something that requires a higher
security standard and switch it on or off as needed.

-- David


Re: Proposed (optional) kwalitee metric; use re 'taint' / Per-author tests?

2008-06-24 Thread David Golden
On Tue, Jun 24, 2008 at 5:08 AM, Paul Fenwick [EMAIL PROTECTED] wrote:
 As the user of a module, it's possible for me to pass in tainted data.  The
 module doesn't know from where it's been sourced.  However, unless the
 *intent* of the module is to untaint this data, anything derived from that
 data should probably remain tainted.  Likewise, unless it's the purpose of
 the module is untaint incoming data, anything the module reads from an
 external source should probably also remain tainted.

I think I disagree with this. (Though perhaps could be argued out of
it.)  It seems to me that data should be validated at the time it is
collected and untainted once validated.  I don't see why some
subroutine N levels down the call stack in some utility module should
be expected to preserve taint on data you didn't check when you
received it.

-- David


Re: Proposed (optional) kwalitee metric; use re 'taint'

2008-06-24 Thread Paul Fenwick

G'day David,

David Golden wrote:


Except use re q/taint/ is lexical.  So if some module isn't itself
reading data from a potentially tainted source, then it really doesn't
need to bother with this.  That's not the same as strict and warnings,
which always apply to my code.


Ah!  But this is not about reading data!  Let's pretend we have the 
following (naive) code in a module that I use:


sub extract_sentence {
my ($str) = @_;

# Sentences end with a dot, followed by a space.
my ($title) = $str =~ /(.*)\.\s/;

return $title;
}

However if I pass in a tainted string like:

It was dark; very dark.  The moon was hidden by clouds

The subroutine returns an *UNTAINTED*:

It was dark; very dark.

Having just extracted the first sentence doesn't mean that it's been checked 
for safety in any way whatsoever.  And here's the problem, as the developer 
USING this module, I can't easily stop extract_sentence from untainting my data.


Note that this has nothing to do with *reading* data, and everything to do 
with regular expressions.  The example code above was not written with 
untainting data in mind, and yet it untaints data by default.



I don't see why some subroutine N levels down the call stack in some
utility module should be expected to preserve taint on data you didn't
check when you received it.


For the same reasons I don't want some subroutine N levels down to overwrite 
$_, screw around with my $/, or make $@ mysteriously disappear or change.


If I had wanted my data untainted, I would have done it explicitly.  In 
fact, I want to *keep* most of my data tainted, so I don't accidentally do 
something foolish with it.



I think I disagree with this. (Though perhaps could be argued out of
it.)  It seems to me that data should be validated at the time it is
collected and untainted once validated.


I agree with your first sentiment, but not your second.  Data should be 
validated when you first collect it, but untainting does not necessarily 
follow from that.  Just because I've read a valid hunk of HTML from a 
webpage doesn't mean that I'd ever intend to use that HTML as a filename, 
put it anywhere near a shell, or use it in any other taint-aware method I 
may use[1].


This is a hard problem, which can be summarised as:

* Perl provides a very good mechanism for tracking untrusted
  data.

* Not many people use that mechanism, or even think about it.

* One of Perl's most commonly used language features (regexps)
  marks data as trusted by default.

* Therefore, most code that uses regexps will untaint by accident.

Unfortunately, adding a metric to CPANTS is going to solve it, although it 
would certainly increase awareness of the issue.  Changing how Perl untaints 
data isn't possible, since we'll break old code, and it won't help older 
Perls anyway.


Having a module that changes the default behaviour and can be loaded into an 
application that cares probably will work[2], and is much more dependable 
than relying upon every CPAN author to have been doing the right thing for 
something they may never use.


David, chromatic, thank-you both for letting me bounce ideas off you, I 
really do appreciate it a great deal.


Many thanks!

Paul

[1] DBI, for example, provides a lovely interface for checking data is 
untainted before being used in a statement.


[2] For some definition of work.

--
Paul Fenwick [EMAIL PROTECTED] | http://perltraining.com.au/
Director of Training   | Ph:  +61 3 9354 6001
Perl Training Australia| Fax: +61 3 9354 2681


Re: Proposed (optional) kwalitee metric; use re 'taint'

2008-06-24 Thread David Golden
On Tue, Jun 24, 2008 at 8:05 AM, Paul Fenwick [EMAIL PROTECTED] wrote:
 I don't see why some subroutine N levels down the call stack in some
 utility module should be expected to preserve taint on data you didn't
 check when you received it.

I think this is the crux of our disagreement and for me it's a
practical issue, not a principles issue.  I think that since Perl
doesn't allow one to turn off regex untainting globally, that it's
unreasonable (unrealistic?) to expect every module in your
prerequisite tree to do it for you.  Therefore, practically speaking,
you can't make any assumptions about taint status so you should manage
it locally where you have control.

 For the same reasons I don't want some subroutine N levels down to overwrite
 $_, screw around with my $/, or make $@ mysteriously disappear or change.

Those are globals (which is a whole separate issue).  An argument to a
function and the return values aren't.

* One of Perl's most commonly used language features (regexps)
  marks data as trusted by default.

* Therefore, most code that uses regexps will untaint by accident.


 Unfortunately, adding a metric to CPANTS is going to solve it, although it
 would certainly increase awareness of the issue.  Changing how Perl untaints
 data isn't possible, since we'll break old code, and it won't help older
 Perls anyway.

I'm not sure it's not worth putting to p5p anyway.  Maybe there should
be a global don't untaint via regex option not unlike the -w and
-W command line switches.

 Having a module that changes the default behaviour and can be loaded into an
 application that cares probably will work[2], and is much more dependable
 than relying upon every CPAN author to have been doing the right thing for
 something they may never use.

I've thought about several approaches.  Wrapping the subroutines to
taint results might work, but no guarantee everything that needs to be
wrapped get wrapped.  My best idea so far is overriding
CORE::GLOBAL::require to source filter each module on load and add
use re 'taint' at the top:

If you load the module first on the command line with
-MRequire::re::taint or whatever, that might be enough for what you
need.

 David, chromatic, thank-you both for letting me bounce ideas off you, I
 really do appreciate it a great deal.

My pleasure.  Always fun to think about the hard problems.  ;-)

David


Re: Proposed (optional) kwalitee metric; use re 'taint'

2008-06-24 Thread brian d foy
In article [EMAIL PROTECTED], Paul Fenwick
[EMAIL PROTECTED] wrote:

  * Does anyone think this is a bad idea?

I think it's an exceedingly bad idea. 

The more things I have to pay attention to in CPANTS, the less likely
that I'm going to do anything about it. You might like using it, but
I'm not going to need it in almost any of my code, and I would have to
ignore this metric.

CPANTS gaming made my distributions better, but only because there were
a limited number of things I had to fix or change. For a long time, I
was the guy on the top of the list. I've been ignoring CPANTS for
several months because of the explosion of metrics, although I was
convinced last week to look again.

If I start ignoring metrics, I'll just start ignoring CPANTS again.

  * Is there someplace this should be going besides from CPANTS?
It's definitely a common mistake that module authors can
easily fix.

I don't know about common. I don't even think taint checking is common,
even if it should be.


Re: Proposed (optional) kwalitee metric; use re 'taint'

2008-06-24 Thread brian d foy
In article [EMAIL PROTECTED], Paul Fenwick
[EMAIL PROTECTED] wrote:


  I'm very much not happy to get bug reports and test failures and big red
  bars 
  against my distributions because an automated heuristic which applies only
  to 
  some cases decided that it knows better about how to write code than I do. 
 
 One could argue this is the case for all the CPANTS tests.  Why should your 
 code use strict, or warnings, or Test::Pod::Coverage, or have its tests in 
 t/ ?  These are all just automated heuristics telling me how to write my code.

Yes, those metrics are telling you how to write your code and I wish
they weren't part of CPANTS either.


Re: Proposed (optional) kwalitee metric; use re 'taint' / Per-author tests?

2008-06-24 Thread brian d foy
In article [EMAIL PROTECTED], Paul Fenwick
[EMAIL PROTECTED] wrote:

 On that note, surely we could save a lot of anguish with regards to many of 
 the CPAN tests just by making the optional ones[1] actually optional?  As a 
 completely off-the-bat suggestion that could be controlled by META.yml:

Why should I have to work to disable something I don't want and doesn't
apply to me? Why make me do something that distracts me from the real
issue of writing better modules and packaging distributions wisely?


Re: Per-author tests?

2008-06-24 Thread Paul Fenwick

G'day brian / pqa,

brian d foy wrote:

In article [EMAIL PROTECTED], Paul Fenwick
[EMAIL PROTECTED] wrote:

On that note, surely we could save a lot of anguish with regards to many of 
the CPAN tests just by making the optional ones[1] actually optional?  As a 
completely off-the-bat suggestion that could be controlled by META.yml:


Why should I have to work to disable something I don't want and doesn't
apply to me? Why make me do something that distracts me from the real
issue of writing better modules and packaging distributions wisely?


In which case let me revise the idea.  Why not have it so that you can go to 
extra work to enable something you DO want and which DOES apply to you? 
There were recently a number of short-lived tests added to CPANTS, I'd love 
to be able to say that I do want them run for (some of) my distributions.


Presently there seems to be a cycle of adding new tests (which I imagine 
takes considerable work), which is then followed by those tests being 
removed due to general outcry.  That seems to mean that CPANTS is stuck 
where it is without much space to wiggle.  Having even the most 
coarse-grained way of disabling/enabling extra tests I think has the 
potential to get around this.


Authors who consider Test::Pod::Coverage (or any other metric) to be harmful 
can switch it off, and have it disabled for ALL their modules.  Authors who 
think their module isn't complete until it's been packaged and shipped in 
Debian can turn on those extra tests.  Authors who want to make exception to 
their rules can put extra data in their META.yml.  Authors who don't know 
about CPANTS still won't know and probably won't care.


I've only been on perl-qa for a couple of months, so maybe this discussion 
has already happened and I missed it.  If so, let me know, and I'll go back 
to lurking in the corner.


Cheerio,

Paul

--
Paul Fenwick [EMAIL PROTECTED] | http://perltraining.com.au/
Director of Training   | Ph:  +61 3 9354 6001
Perl Training Australia| Fax: +61 3 9354 2681


Re: Proposed (optional) kwalitee metric; use re 'taint' / Per-author tests?

2008-06-24 Thread chromatic
On Tuesday 24 June 2008 02:08:00 Paul Fenwick wrote:

 As the user of a module, it's possible for me to pass in tainted data.  The
 module doesn't know from where it's been sourced.  However, unless the
 *intent* of the module is to untaint this data, anything derived from that
 data should probably remain tainted.  Likewise, unless it's the purpose of
 the module is untaint incoming data, anything the module reads from an
 external source should probably also remain tainted.

That's my point.  Most of the uses of regexes in most of the distributions 
I've written never act on data that could possibly be tainted (unless you 
somehow named subroutines or classes with tainted strings, in which case you 
have worse things to worry about than my code).

-- c


Re: Proposed (optional) kwalitee metric; use re 'taint'

2008-06-24 Thread Andy Lester


On Jun 24, 2008, at 11:41 AM, brian d foy wrote:


Yes, those metrics are telling you how to write your code and I wish
they weren't part of CPANTS either.



And all of this again points out that CPANTS is not really aimed at  
anyone but the CPANTS test crew.


You know what'd be swell?  If CPANTS let us create profiles so we  
could say I care about this, I don't care about that.  To me, the re  
'taint' data point is a pretty cool idea, but that's just 'cause I  
love taint.


--
Andy Lester = [EMAIL PROTECTED] = www.petdance.com = AIM:petdance






common errors?

2008-06-24 Thread Hilary Holz
On 6/23/08 11:24 PM, chromatic [EMAIL PROTECTED] wrote:

 (Want a heuristic which finds *actual* bugs in almost every module ever
 written?  Check the use of eval and subsequent $@ testing, or the use of
 ref(), or SUPER::whatever(), for example.)
 
Could you expand a bit, please (or provide pointers to an existing reference
or references)? Sounds like you have particular common errors in mind for
each of these, and off the top of my head I'm not coming up with any such
references. Unfortunately, the combination of schedule and lack of funding
means that I don't get to go to techie conferences since heading off into
academia, so there may be slide sets, etc

Hilary

-- 
Hilary Holz, D.Sc. 
Co-chair, SIGCSE Committee on Teaching Computing Research Methods
Director, Laboratory for Adaptive Hypermedia and Assistive Technology
Associate Professor of Computer Science
Department of Mathematics and Computer Science, Cal State, East Bay
http://acc.csueastbay.edu/  [EMAIL PROTECTED]





Re: common errors?

2008-06-24 Thread David Golden
On Tue, Jun 24, 2008 at 4:06 PM, Hilary Holz [EMAIL PROTECTED] wrote:
 On 6/23/08 11:24 PM, chromatic [EMAIL PROTECTED] wrote:

 (Want a heuristic which finds *actual* bugs in almost every module ever
 written?  Check the use of eval and subsequent $@ testing, or the use of
 ref(), or SUPER::whatever(), for example.)

 Could you expand a bit, please (or provide pointers to an existing reference
 or references)? Sounds like you have particular common errors in mind for
 each of these, and off the top of my head I'm not coming up with any such
 references. Unfortunately, the combination of schedule and lack of funding
 means that I don't get to go to techie conferences since heading off into
 academia, so there may be slide sets, etc

Not sure if those are referenced from here:
http://www.perlfoundation.org/perl5/index.cgi?mistakes_to_avoid

If not, they probably should be.

-- David


Re: Test::Most 0.02 (Adds Test::Warn and deferred plans)

2008-06-24 Thread Buddy Burden
Ovid,

 Are you on the latest version of Test::Deep?

Ah, yes.  That fixed it.  Thanx!

  There are issues with
 previous versions having an isa() sub causing strange failures.

Yeah, I saw something about that on the list, but it didn't occur to
me that having a strange isa() could cause this problem.  But that's
exactly what it was, so everything is hunky-dory now.  Thanx again.


 -- Buddy


use re 'taint' experiment

2008-06-24 Thread David Golden
Just because it's a bizarre Perl challenge (and I tend to like those),
I took a hack at what a command line '-MRequires::re::taint would
need to do in order to force use re 'taint' in all modules
subsequently loaded.

I've attached a sample module to do just that and some sample code.
To try it, untar the attachment, cd into the resulting directory, and
run this:

  $ perl -T -I. -MRequire::re::taint taint-test.pl

Type in some filename, e.g. foo.txt and hit return to watch it die
of a taint error.

And despite having inserted use re 'taint' into wrapper.pm, it even
dies with the correct line number.  :-)

This is so awful that I'm certainly not planning to release it as a
module (I wouldn't want to even pretend to support it) -- or at least
not outside the Acme namespace, but if anyone wants to take it on,
they're welcome to take this code as a starting point.

-- David


require-taint-wrap.tar.gz
Description: GNU Zip compressed data


Re: common errors?

2008-06-24 Thread Paul Fenwick

G'day Hilary,

chromatic wrote:


(Want a heuristic which finds *actual* bugs in almost every module ever
written?  Check the use of eval and subsequent $@ testing, or the use of
ref(), or SUPER::whatever(), for example.)


Hilary Holz requested:


Could you expand a bit, please (or provide pointers to an existing reference
or references)?


There's a discussion of eval/$@ happening (or finishing) on p5p right now. 
It starts at [1].


It doesn't list all the eval/$@ pitfalls, but the ones that immediately 
bring to mind include:


* Exception-handling code that directly or indirectly uses eval,
  which then results in $@ being cleared.  Copying $@
  before doing anything with it is recommended.

* Related to that, any code that uses an eval without
  localizing $@ can clobber a value in $@ that the caller
  may have cared about.

* Anything that does localizing $@ can result in problems
  trying to re-throw errors unless extra care is taken.
  See [2].

* Objects can use eval inside their DESTROY blocks, which
  can result in $@ being cleared when the eval exits,
  even though an exception occured!

* Signals can clobber [EMAIL PROTECTED]  See [3].

The recommended way to use eval looks like this:

eval {

# code that may die goes here...

1;

} or do {
my $err = $@;
print Oh no!  Something went wrong\n;

# Use $err to hopefully figure out what happened.

};

Eval returns false if it captures an exception, or the last value of the 
block (in our case, 1) if successful.  We then hope that $@ is still around 
by the time we get to it.


I'm not exactly sure which pitfalls chromatic is alluding to with regards to 
ref() and SUPER:: .


All the best,

Paul

[1] http://www.xray.mpe.mpg.de/mailing-lists/perl5-porters/2008-06/msg00507.html

[2] http://rt.perl.org/rt3/Ticket/Display.html?id=29696

[3] http://rt.perl.org/rt3//Ticket/Display.html?id=47928

--
Paul Fenwick [EMAIL PROTECTED] | http://perltraining.com.au/
Director of Training   | Ph:  +61 3 9354 6001
Perl Training Australia| Fax: +61 3 9354 2681


Re: use re 'taint' experiment

2008-06-24 Thread Paul Fenwick

G'day David / PQA,

David Golden wrote:

Just because it's a bizarre Perl challenge (and I tend to like those),
I took a hack at what a command line '-MRequires::re::taint would
need to do in order to force use re 'taint' in all modules
subsequently loaded.


Gosh.  That's a lot less code than the XS I was writing to intercept require 
and set bits in $^H appropriately.



This is so awful that I'm certainly not planning to release it as a
module (I wouldn't want to even pretend to support it) -- or at least
not outside the Acme namespace, but if anyone wants to take it on,
they're welcome to take this code as a starting point.


Taken!  I assume I can use this under the same terms as Perl itself?

Cheerio,

Paul

--
Paul Fenwick [EMAIL PROTECTED] | http://perltraining.com.au/
Director of Training   | Ph:  +61 3 9354 6001
Perl Training Australia| Fax: +61 3 9354 2681


Re: Proposed (optional) kwalitee metric; use re 'taint'

2008-06-24 Thread Chris Dolan

On Jun 24, 2008, at 9:59 AM, David Golden wrote:

On Tue, Jun 24, 2008 at 8:05 AM, Paul Fenwick  
[EMAIL PROTECTED] wrote:

I don't see why some subroutine N levels down the call stack in some
utility module should be expected to preserve taint on data you  
didn't

check when you received it.


I think this is the crux of our disagreement and for me it's a
practical issue, not a principles issue.  I think that since Perl
doesn't allow one to turn off regex untainting globally, that it's
unreasonable (unrealistic?) to expect every module in your
prerequisite tree to do it for you.  Therefore, practically speaking,
you can't make any assumptions about taint status so you should manage
it locally where you have control.


This is the most important point of the conversation so far.  Paul's  
use re 'taint' solution only works if *everyone* participates.   
That's simply not going to happen, because CPAN is too anarchic to  
hope for that.  All you need is one module, N levels deep to not use  
re 'taint' who should have, and it's suddenly your weakest link.   
That's not a good situation.


I'll make a blanket statement that I think is pretty non- 
controversial: technological change that requires broad participation  
is doomed to fail unless there is a big incentive for that change.   
CPANTS and Perl::Critic scores are not big incentives.  To many,  
they're not even small incentives.


I'm perfectly willing to accept a use re 'taint' policy in P::C,  
but I'd push for it to be a low severity policy -- P::C is designed  
to be inclusive without being pushy (except when it's important to be  
pushy).  Instead, the obviously correct solution is to fix untainting  
to be deliberate instead of a side-effect of a common operation.  5.12?


Chris