On Sep 11, 2006, at 7:08 PM, Randy W. Sims wrote:
Ovid wrote:
Last week I was at a testing conference with Acme and he came up
with the idea of installing tests. He looked into hacking
Module::Build and ExtUtils::MakeMaker. He also considered hacking
CPAN.pm and CPANPLUS.pm. While I don't know if he plans to
continue working on this idea, he said he didn't mind me posting
his idea here for others to consider.
Basically, installing tests would be good because then you can run
your full test suite against *installed* modules. That would be
nice because then you could install a module and rerun your tests
for your entire installation and see what broke.
I love this idea, but here are some issues that we spotted:
1. How does one install tests for modules already installed?
2. If you install a module with already failing tests, you need
to track what the failures are so you can note different failures
when you run the test suite in the future.
3. What's the best way to install them? Should a separate tool
just for this be built?
Anyone want to take a crack at this?
I'm also going to post this to Perlmonks.
IIRC, I think this is something Ken has had in mind for a long time
for Module::Build. He might have some ideas about how it might be
done.
Yes, I've been thinking about this for a long time. In fact, in the
most recent M::B beta I made some steps toward it, by adding a
'retest' action that's just like 'test' except that it doesn't look
in blib/, just in @INC. Functionally that actually covers a lot of
the same ground you're after.
What I like about the 'retest' approach is that it's very easy and
it's much more likely to work. It also makes it possible to run old
tests against a new installation, or vice versa. What I don't like
about it is that the user has to find the tarball again that they
previously installed. I can imagine that in some situations that
wouldn't be trivial. In other situations, when people can plan
ahead, it's probably not a big deal.
I think there are some larger issues than 1,2,3 above that you might
have missed, too:
4) Many distributions, including many of the most crucial and well-
used ones, have some extra set-up steps in their build/install
sequences. Others make assumptions in their test suites about where
various files are located relative to the test code or relative to
the current working directory. It's quite possible that in order for
"installed" tests to work correctly it could take some serious
coöperation by modules' authors.
5) Where should tests be installed? Where would any other
supporting materials be installed?
For your #1 above, I'd say just perform a reinstall. For #2, maybe
just punt - is there much of a need for that? For #3, I think we can
work it into Module::Build as an action or flag(s) to the 'install'
action. For EU::MM-based modules I'm not sure what the best approach
would be, but probably I don't have to think about it. =)
-Ken
