Testsuite fails
Hello! The CVS version of Automake fails in libtool2.test. There is no mostlyclean-libtool in sub/Makefile.in. I suggest that if a failing test is submitted, it should be added to XFAIL_TESTS in the same commit. It should be removed from XFAIL_TESTS in the same commit that fixes it. This way we would reserve the true failures for the problems nobody is aware of, so that they can be spotted and reported immediately. -- Regards, Pavel Roskin
Re: Testsuite fails
Pavel == Pavel Roskin [EMAIL PROTECTED] writes: Pavel I suggest that if a failing test is submitted, it should be Pavel added to XFAIL_TESTS in the same commit. It should be removed Pavel from XFAIL_TESTS in the same commit that fixes it. I agree. Sorry about that. I checked in the fix. Tom
Re: Testsuite fails
"adl" == Alexandre Duret-Lutz [EMAIL PROTECTED] writes: adl Is it an "ok" for my proposal or a "no, keep it as it is now"? adl Right now, I would have to call AC_CHECK_TOOL([STRIP],[strip]) adl in my configure.in *before* calling AM_INIT_AUTOMAKE to make adl thinks work. Yeah. I think we should just change AM_PROG_INSTALL_STRIP to always use install-sh. I don't think it is very important to try to optimize install-strip. adl Basically I'm asking the same thing for strip. Except that adl since Autoconf 2.13 doesn't ship with an AC_PROG_STRIP macro adl that Automake can check for, this macro would be supplied by adl automake (unfortunately). I don't want to supply it in automake. We're moving out of the macro business now that autoconf is maintained. Tom
Re: Testsuite fails
"Tom" == Tom Tromey [EMAIL PROTECTED] writes: "adl" == Alexandre Duret-Lutz [EMAIL PROTECTED] writes: [...] adl Otherwise, as install-strip is concerned, another idea is to: adl 1) not call AM_PROG_INSTALL_STRIP from AM_INIT_AUTOMAKE adl 2) revert all `commenting' that has been done in AM_PROG_INSTALL_STRIP adl 3) ask the user to call AM_PROG_INSTALL_STRIP if she uses adl AC_CANONICAL_HOST or AC_CANONICAL_TARGET (automake can adl check for this, I guess). Tom One reason I knew it was "ok" to disable this entirely is that Tom packages which are set up for cross-compilation still have to do a lot Tom of work themselves. For instance they must search for the cross Tom ranlib themselves. That is, this is a general problem in autoconf Tom that I prefer we solve there, not in automake. Is it an "ok" for my proposal or a "no, keep it as it is now"? Right now, I would have to call AC_CHECK_TOOL([STRIP],[strip]) in my configure.in *before* calling AM_INIT_AUTOMAKE to make thinks work. My package does cross-build successfully and I had no additional work to do appart from calling AC_CANONICAL_HOST (only install-strip fails, of course). AC_PROG_RANLIB works for cross-compilation too (at least in CVS autoconf, I have never used the old), and Automake does check whether this macro is called when it needs ranlib; this is great. Basically I'm asking the same thing for strip. Except that since Autoconf 2.13 doesn't ship with an AC_PROG_STRIP macro that Automake can check for, this macro would be supplied by automake (unfortunately). Yes, I'm asking a lot. -- Alexandre Duret-Lutz
Re: Testsuite fails
"adl" == Alexandre Duret-Lutz [EMAIL PROTECTED] writes: Akim Your explanation is very correct. AC_CHECK_TOOL only needs Akim host_alias, hence there was no reason to require AC_CANONICAL_*. adl Has anyone a plan to change this in Automake? Eventually when we move to traces this will change. For now we must be conservative. I believe the last released autoconf still has the old behavior. We must work with that. adl Otherwise, as install-strip is concerned, another idea is to: adl 1) not call AM_PROG_INSTALL_STRIP from AM_INIT_AUTOMAKE adl 2) revert all `commenting' that has been done in AM_PROG_INSTALL_STRIP adl 3) ask the user to call AM_PROG_INSTALL_STRIP if she uses adl AC_CANONICAL_HOST or AC_CANONICAL_TARGET (automake can adl check for this, I guess). One reason I knew it was "ok" to disable this entirely is that packages which are set up for cross-compilation still have to do a lot of work themselves. For instance they must search for the cross ranlib themselves. That is, this is a general problem in autoconf that I prefer we solve there, not in automake. Tom
Re: Testsuite fails
Akim Demaille wrote: Pavel Roskin [EMAIL PROTECTED] writes: We probably need a special macro, e.g. AC_REQUIRE_FILE, so that the macros will be able to indicate what files they need. This is what Derek and I are working on :) But I doubt 2.50 will be the good starting point, 2.51 will. Incidentally, I think we're going to need AM_ALL_LINGUAS or the like to replace the simple ALL_LINGUAS variable set. Derek -- Derek Price CVS Solutions Architect ( http://CVSHome.org ) mailto:[EMAIL PROTECTED] OpenAvenue ( http://OpenAvenue.com ) -- Since English is a mess, it maps well onto the problem space, which is also a mess, which we call reality. - Larry Wall
Re: Testsuite fails
"Derek" == Derek R Price [EMAIL PROTECTED] writes: I imagine we'll have to radically revamp the test suite when we move to using autoconf's tracing feature. The current test suite is very dumb and doesn't usually generate a correct configure.in. Derek I'm working on this so I can test my conversion. I've been Derek reading the puropse lines and I haven't noticed anything that Derek looks like it's going to ba a major problem yet, but can Derek anybody else think of anything in particular I should watch out Derek for? In the past I've had to make systematic changes to the automake test suite. It sucks. I'd like this to be the last major test suite rewrite. That means trying to modularize somehow. Maybe rewriting the tests in some other language with better functionality (eg Perl) would make this simpler. Akim: I still haven't looked at autotest. Tom
Re: Testsuite fails
"Akim" == Akim Demaille [EMAIL PROTECTED] writes:
Ideally we'd have some way to tell autoconf, "if this package can
be cross-compiled, then check for a cross `strip' as well". Maybe
there's an ugly way to do this.
Akim But I think your request is precisely AC_CHECK_TOOL. What
Akim exactly do you mean?
Suppose I have AM_INIT_AUTOMAKE call AC_CHECK_TOOL. Then every
package needs config.{guess,sub}. (Or at least that was true in the
past.)
The idea is something like:
* the user declares a package suitable for cross compilation:
AC_CAN_CROSS_COMPILE
* AM_INIT_AUTOMAKE conditionalizes on this:
AC_IF_CROSS([AC_CHECK_TOOL([STRIP],[strip])])
Akim Also, my number one grief against the test suite is that it is
Akim classified by alphabetical order, which looks cute, but is a
Akim stupid an order as the random order. We need to order them from
Akim the ``innermost tests'' to the outermost, so that it gives us
Akim some good intuition of *where* it fails, and what failures are
Akim actually the same only one.
That would be nice, but the automake tests are really just a bunch of
regression tests. Mostly they test for fairly specific bugs. This
isn't a great quality to have in a test suite.
If someone wants to reorder them, that's fine, I guess. I don't want
to do it since it seems like a lot of work for no particular reason.
One problem with reordering is deciding where to put a new test.
Tom
Re: Testsuite fails
Tom Tromey [EMAIL PROTECTED] writes: "Pavel" == Pavel Roskin [EMAIL PROTECTED] writes: Pavel AC_CHECK_TOOL([STRIP],[strip]) Yes, I wasn't thinking. I've removed this. Ideally we'd have some way to tell autoconf, "if this package can be cross-compiled, then check for a cross `strip' as well". Maybe there's an ugly way to do this. There is still plenty to do to clean up the interface Autoconf provides to its users :( But I think your request is precisely AC_CHECK_TOOL. What exactly do you mean? - Macro: AC_CHECK_TOOL (VARIABLE, PROG-TO-CHECK-FOR, [VALUE-IF-NOT-FOUND], [PATH]) Like `AC_CHECK_PROG', but first looks for PROG-TO-CHECK-FOR with a prefix of the host type as determined by `AC_CANONICAL_HOST', followed by a dash (*note Canonicalizing::). For example, if the user runs `configure --host=i386-gnu', then this call: AC_CHECK_TOOL(RANLIB, ranlib, :) sets `RANLIB' to `i386-gnu-ranlib' if that program exists in `PATH', or to `ranlib' if that program exists in `PATH', or to `:' if neither program exists. or do you mean if $cross_compiling; then AC_CHECK_TOOL([STRIP], [strip]) fi ? Pavel But what bothers me is the fact that very serious patches were Pavel committed without running the testsuite. Yeah, that's my fault. Sometimes I forget to do this. That's bad. Yeah, you're a bad guy :) gcc uses a nice regression framework that sends email to people who committed patches in the window where a new bug could have been introduced. That seems like overkill for automake. Well, if it's easy to install, I would do it! Yep, this is overkill, but some comfort through more automation is somewhat attractive. I imagine we'll have to radically revamp the test suite when we move to using autoconf's tracing feature. The current test suite is very dumb and doesn't usually generate a correct configure.in. Well, not to say `never'. I would really enjoy seeing Automake move to Autotest. Also, my number one grief against the test suite is that it is classified by alphabetical order, which looks cute, but is a stupid an order as the random order. We need to order them from the ``innermost tests'' to the outermost, so that it gives us some good intuition of *where* it fails, and what failures are actually the same only one. Perhaps we could also rewrite the test suite to be faster somehow. Maybe writing most of it in Perl would help. I considered several times having several Autotest target languages, and Perl indeed seems a good one. But let's not invent yet another testing framework, let's go to Autotest!
Re: Testsuite fails
"Pavel" == Pavel Roskin [EMAIL PROTECTED] writes: Pavel AC_CHECK_TOOL([STRIP],[strip]) Yes, I wasn't thinking. I've removed this. Ideally we'd have some way to tell autoconf, "if this package can be cross-compiled, then check for a cross `strip' as well". Maybe there's an ugly way to do this. Pavel But what bothers me is the fact that very serious patches were Pavel committed without running the testsuite. Yeah, that's my fault. Sometimes I forget to do this. That's bad. cvs doesn't provide a way to force regression tests to be run on the client system at commit time. We could set it up to do this on the server, but I'd prefer not to. gcc uses a nice regression framework that sends email to people who committed patches in the window where a new bug could have been introduced. That seems like overkill for automake. Pavel Some people are offended by this condition because it's Pavel _implied_. But now we have to think how to fix the testsuite Pavel that already fails. I'm just directly removing the bug. I imagine we'll have to radically revamp the test suite when we move to using autoconf's tracing feature. The current test suite is very dumb and doesn't usually generate a correct configure.in. Perhaps we could also rewrite the test suite to be faster somehow. Maybe writing most of it in Perl would help. Tom
