Re: [Patch] Don't ignore testsuite errors in Makefile

2011-10-09 Thread Jakub Jelinek
On Sun, Oct 09, 2011 at 04:32:12PM +0200, Mikael Morin wrote:
 currently, the testsuite return value is ignored by make. It is a little 
 annoying if one wants to check automatically for regressions as we have to 
 parse the testsuite output.
 This patch reverts to the normal make behaviour, which is to not ignore 
 commands' return values. Note: As a result the -k flag has to be added to the 
 make command line if one wants the tests to continue after one failure.
 
 OK for trunk?

Please no.  This is a very bad idea, most of the testsuites on many
architectures contain some FAILs and a failure from check-parallel-% would
mean the *.log/*.sum files would be never merged in that case.

If you really need to propagate the return value (I fail to see how it is
useful), then you should e.g. store the $? value from $(RUNTEST) in
check-parallel-% into some file in that directory and have the
parallelization goal after the merging collect those from the individual
files and or them all together into the final return value.

 2011-10-09  Mikael Morin  mikael.mo...@sfr.fr
 
   * Makefile.in (check-parallel-%): Don't ignore testsuite errors.

 Index: Makefile.in
 ===
 --- Makefile.in   (révision 179710)
 +++ Makefile.in   (copie de travail)
 @@ -5116,10 +5124,10 @@ $(patsubst %,%-subtargets,$(lang_checks_paralleliz
  # Otherwise check-$lang isn't parallelized and runtest is invoked just with
  # the $(RUNTESTFLAGS) arguments.
  check-parallel-% : site.exp
 - -test -d plugin || mkdir plugin
 - -test -d $(TESTSUITEDIR) || mkdir $(TESTSUITEDIR)
 + test -d plugin || mkdir plugin
 + test -d $(TESTSUITEDIR) || mkdir $(TESTSUITEDIR)
   test -d $(TESTSUITEDIR)/$(check_p_subdir) || mkdir 
 $(TESTSUITEDIR)/$(check_p_subdir)
 - -(rootme=`${PWD_COMMAND}`; export rootme; \
 + (rootme=`${PWD_COMMAND}`; export rootme; \
   srcdir=`cd ${srcdir}; ${PWD_COMMAND}` ; export srcdir ; \
   cd $(TESTSUITEDIR)/$(check_p_subdir); \
   rm -f tmp-site.exp; \


Jakub


Re: [Patch] Don't ignore testsuite errors in Makefile

2011-10-09 Thread Mikael Morin
On Sunday 09 October 2011 21:12:12 Jakub Jelinek wrote:
 On Sun, Oct 09, 2011 at 04:32:12PM +0200, Mikael Morin wrote:
  currently, the testsuite return value is ignored by make. It is a little
  annoying if one wants to check automatically for regressions as we have
  to parse the testsuite output.
  This patch reverts to the normal make behaviour, which is to not ignore
  commands' return values. Note: As a result the -k flag has to be added to
  the make command line if one wants the tests to continue after one
  failure.
  
  OK for trunk?
 
 Please no.  This is a very bad idea, most of the testsuites on many
 architectures contain some FAILs and a failure from check-parallel-% would
 mean the *.log/*.sum files would be never merged in that case.
 
 If you really need to propagate the return value (I fail to see how it is
 useful), then you should e.g. store the $? value from $(RUNTEST) in
 check-parallel-% into some file in that directory and have the
 parallelization goal after the merging collect those from the individual
 files and or them all together into the final return value.
Thanks for the tips. I will just keep the patch locally for now.
I don't use parallel testing anyway.

Mikael