As seen in a recent thread here (cross-posted to
module-auth...@perl.org), I was thinking of moving the Makefile.PL
"C++ guess" logic to a separate module to decouple development from
the core Inline::CPP module. The discussion that followed led me to
find ExtUtils::CppGuess, which looks like a really nice alternative.
EU::CG seems to have a strong 'PASS' rate with the CPAN testers.
Nevertheless, I don't want to give up any hard-won ground by blindly
shifting over. It would be a shame to take two steps forward and one
backward, when some care in transitioning could bring us three steps
forward instead.
Here's a strategy I've come up with:
First step: scrape all the CPAN smoke test reports for the latest
Inline::CPP distribution. Parse them for the various $Config{...}
stats that we currently use as a basis for our C++ compiler guess
logic, and assemble those stats into a csv file, along with a column
for 'pass/fail'.
Second step: Import our existing Makefile.PL logic into a test
script. Within that test script, run each of the configuration
profiles against our existing logic, and compare it to results from
ExtUtils::CppGuess. Spit out a log of any differences between our
guesses, and EU::CG's guesses along with whether or not that
particular profile resulted in an Inline::CPP pass or fail.
Now, ideally everywhere I currently have a FAIL, EU::CG will have a
different guess, and everywhere I have a PASS the existing logic will
agree with EU::CG. That would be ideal. Drop in EU::CG and relax.
Ideal, but highly unlikely.
The worst case would be different guesses for each profile, and no
relationship to PASS/FAIL. Give up on the idea, create my own module
instead. Also an unlikely scenario.
What I'm actually hoping to find will be that most of the time we're
guessing identically, and that in those few cases where the guesses
disagree I'll be able to assess whether the winner should be EU::CG,
or my existing logic. Hopefully there would be at least some
correlation between FAILS and differences between our guesses. After
working through those discrepancies, in those (hopefully few) places
where Inline::CPP's Makefile.PL logic is guessing better than EU::CG,
I could submit bug reports (hopefully with patches) to EU::CG. That
could lead to EU::CG gaining additional ground, making it the obvious
solution to Inline::CPP's need. Otherwise, I could special-case a few
profiles in Makefile.PL, and still shift to EU::CG.
Does anyone see a better approach to regression testing this sort of transition?
Dave
