Re: Failure on test parallel-tests5.test

2010-10-31 Thread Lluís Batlle i Rossell
On Sun, Oct 31, 2010 at 10:48:10AM +0100, Lluís Batlle i Rossell wrote:
 Hello Ralf,
 
 On Sun, Oct 31, 2010 at 10:01:09AM +0100, Ralf Wildenhues wrote:
  * Lluís Batlle i Rossell wrote on Sun, Oct 31, 2010 at 12:02:21AM CEST:
   sorry to provide this little information, but building automake 1.11.1
   on mips linux n32 today failed to me in a single test:
   parallel-tests5.test
  
  cd tests
  (ulimit -t 300; sh -c 'while make check TESTS=parallel-tests5.test; do :; 
  done')
  echo $?
 I will let the loop run for long.
  
  Which make implementation and version do you use?  Which distro is this?
 
 It's NixOS.
 gnu make 3.82.

After many retries, the test failed again. The part of the log about the failure
says:

..
gcc -DPACKAGE_NAME=\parallel-tests5\ -DPACKAGE_TARNAME=\parallel-tests5\ 
-DPACKAGE_VERSION=\1.0\ -DPACKAGE_STRING=\parallel-tests5\ 1.0\ 
-DPACKAGE_BUGREPORT=\\ -DPACKAGE_URL=\\ -DPACKAGE=\parallel-tests5\ 
-DVERSION=\1.0\ -I. -g -O2 -MT foo4.o -MD -MP -MF .deps/foo4.Tpo -c -o 
foo4.o foo4.c
building foo5.o
mv -f .deps/foo2.Tpo .deps/foo2.Po
make[2]: vfork: Function not implemented
make[2]: *** Deleting file `foo2.o'
mv -f .deps/foo4.Tpo .deps/foo4.Po
gcc -DPACKAGE_NAME=\parallel-tests5\ -DPACKAGE_TARNAME=\parallel-tests5\ 
-DPACKAGE_VERSION=\1.0\ -DPACKAGE_STRING=\parallel-tests5\ 1.0\ 
-DPACKAGE_BUGREPORT=\\ -DPACKAGE_URL=\\ -DPACKAGE=\parallel-tests5\ 
-DVERSION=\1.0\ -I. -g -O2 -MT foo5.o -MD -MP -MF .deps/foo5.Tpo -c -o 
foo5.o foo5.c
.
PASS: foo2.test
PASS: foo3.test
PASS: foo1.test
PASS: foo4.test
building foo6.log
building foo7.log
building foo8.log
PASS: foo5.test
PASS: foo6.test
PASS: foo7.test
PASS: foo8.test
building foo1.prog
gcc  -g -O2   -o foo1.prog foo1.o
building foo1.prog.log
PASS: foo1.prog
make[2]: Leaving directory 
`/tmp/nix-build-v732qfq81afcilibmp48al3gqw1avpvz-automake-1.11.1.drv-0/automake-1.11.1/tests/parallel-tests5.dir'
make[1]: *** [check-TESTS] Error 2
make[1]: Leaving directory 
`/tmp/nix-build-v732qfq81afcilibmp48al3gqw1avpvz-automake-1.11.1.drv-0/automake-1.11.1/tests/parallel-tests5.dir'
make: *** [check-am] Error 2

It looks like a 'make' or glibc problem, right?

If you want more details of the test results, I have all the files. Request 
whatever you need.

Regards,
Lluís.



Re: Copy files in VPATH build

2010-10-31 Thread Ralf Wildenhues
* Paul J. Davis wrote on Sun, Oct 31, 2010 at 12:10:01AM CEST:
 On Oct 30, 2010, at 5:17 PM, Jason Curl jcurln...@arcor.de wrote:
  When building my package with ./configure everything works well as
  some datafiles that are required for tests are in the correct place
  (my tests don't need to be installed with make install). However,
  when I use VPATH builds with ../configure, I don't know how to get
  my data files copied from the source to the current build path.
  
  I've just read up on VPATH, and I think gmake is assuming, that
  while the datafiles are in the source, the dependencies are
  satisfied, even though they aren't in the build directory.
  
  The following rules don't work:

Right.  If @srcdir@ is '.', then GNU make will error out here
(non-GNU make would destroy your data, except that it will barf over the
use of $ outside of inference rules first).

  test.ini: @srcdir@/test.ini
   cp $ $@
  
  ./test.ini: @srcdir@/test.ini
   cp $ $@
  
  But the rule:
  test2.ini: @srcdir@/test.ini
   cp $ $@
  
  does work, but I don't need to rename the file in the copy, and when
  not using VPATH, it just performs a useless copy.

Correct as well.

 The only thing I've way I've overcome this is to have a test runner
 that gets built by configure so it can have access to both the build
 and source directory paths.

Good idea.

An alternative can be to use AC_CONFIG_LINKS with identical source and
target name for files: the link-or-copy will the only be made if in a
VPATH build.  You need Autoconf = 2.62 and Automake = 1.11 for this to
work correctly (please set AC_PREREQ and Automake Option to this end,
because the older versions had bugs that could cause source file
removal.)  Please report any remaining bugs.

Hope that helps.

Cheers,
Ralf



ylwrap - flex filename weirdness

2010-10-31 Thread Jan Engelhardt
Hi,


While trying to utilize Autotools in a preexisting project that 
previously just relied on Makefiles, I came across ylwrap failing for 
some absurd, unknown reason. I couldn't help but run sh -x and debug it 
piecewise.

So what's ultimately done is running flex inside a temp directory,

flex /home/jengelh/code/iproute2/tc/emp_ematch_lex.l

It turns out flex produces a file called lex.ematch_.c, which is 
something ylwrap so does not expect at all - it only checks for 
what's in pairlist=yy.lex.c emp_ematch_lex.c.

I can use AM_LFLAGS = -o yy.lex.c to work around it, but that feels like 
an ugly hack. What's up with flex here?


thanks,
Jan



Bison and automake together

2010-10-31 Thread Paulo J. Matos
Hi,

I am slightly confused as to whether I should be posting this to the
bison or this mailing list but here it goes.

I have a bison based project which has the following definitions:
,
| BUILT_SOURCES = scgparser.h
| AM_YFLAGS = -d
| AM_LDFLAGS = -lgmpxx -lgmp
| 
| bin_PROGRAMS = scgc
| 
| scgc_SOURCES = scgc.cc scgparser-driver.cc scgparser.yy scglexer.l
`

When I ran this with ./configure  make I get:
,
| /bin/bash ./ylwrap scgparser.yy y.tab.c scgparser.cc y.tab.h \
|   scgparser.h y.output scgparser.output -- bison -y  -d
`

This is ok because bison generates y.tab.c and y.tab.h that are renamed
by ylwrap as scgparser.cc and scgparser.h respectively.

However, bison generated parser scgparser.cc includes y.tab.h (which was
the name known to bison when the file was generated). This results in
an error during compilation since there is no y.tab.h anymore when the
scgparser.cc is compiled. What am I missing here? 

Feel free to redirect me to the bison mailing list if this is not an
automake config problem.

Cheers,

-- 
PMatos




Re: Copy files in VPATH build

2010-10-31 Thread Jason Curl

On 31/10/2010 10:20, Ralf Wildenhues wrote:

* Paul J. Davis wrote on Sun, Oct 31, 2010 at 12:10:01AM CEST:

On Oct 30, 2010, at 5:17 PM, Jason Curljcurln...@arcor.de  wrote:

When building my package with ./configure everything works well as
some datafiles that are required for tests are in the correct place
(my tests don't need to be installed with make install). However,
when I use VPATH builds with ../configure, I don't know how to get
my data files copied from the source to the current build path.

I've just read up on VPATH, and I think gmake is assuming, that
while the datafiles are in the source, the dependencies are
satisfied, even though they aren't in the build directory.

The following rules don't work:

Right.  If @srcdir@ is '.', then GNU make will error out here
(non-GNU make would destroy your data, except that it will barf over the
use of $  outside of inference rules first).


test.ini: @srcdir@/test.ini
  cp $  $@

./test.ini: @srcdir@/test.ini
  cp $  $@


An alternative can be to use AC_CONFIG_LINKS with identical source and
target name for files: the link-or-copy will the only be made if in a
VPATH build.  You need Autoconf= 2.62 and Automake= 1.11 for this to
work correctly (please set AC_PREREQ and Automake Option to this end,
because the older versions had bugs that could cause source file
removal.)  Please report any remaining bugs.

It's unfortunate that I can't use this functionality, one of my build 
machines is still ubuntu 8.04 and this still has autoconf 2.61 and 
automake 1.10.1.


I've implemented a shell script to do the copies for me based on a phony 
rule (moving the check out of make and into a script). Tested for VPATH 
and non-VPATH. It allows me to keep all changes local to the directory 
where the tests are made. The phony rules are a dependency to 
all-local so they get copied at the end of the process. I didn't use 
myprog_DEPENDENCIES as this would cause a link


e.g.
Makefile.am

all-local: cpreadini cpoptionini

cpreadini:
@srcdir@/cpfiles.sh @srcdir@/read.ini ./read.ini

cpoptionini:
@srcdir@/cpfiles.sh @srcdir@/option1.ini ./option1.ini
@srcdir@/cpfiles.sh @srcdir@/option2.ini ./option2.ini

.PHONY: cpreadini cpoptionini

Feedback welcome.

Thanks for your help Ralf  Paul.
Jason.

#!/bin/sh

SRC=$1
DST=$2

# Check input parameters
if [ $# -lt 2 ]; then
  echo $0 src dest
  exit 2
fi

# Check that the source file exists
if [ ! -e $SRC ]; then
  echo $SRC not found
  exit 2
fi
 
if [ ! -e $DST ]; then
  echo cp $SRC $DST
  cp $SRC $DST
else
  # Check that we have the 'stat' binary. If not, we don't do the copy.
  which stat  /dev/null

  if [ $? -eq 1 ]; then
echo warning: stat not found, no comparison can be made
exit 0
  fi

  SRCT=$(stat -c %Y $SRC)
  DSTT=$(stat -c %Y $DST)
  if [ $DSTT -lt $SRCT ]; then
echo cp $SRC (time=$SRCT) $DST (time=$DSTT)
cp -f $SRC $DST
  fi
fi



Re: Automake and AR

2010-10-31 Thread NightStrike
On Fri, Oct 22, 2010 at 1:07 PM, NightStrike nightstr...@gmail.com wrote:
 On Wed, Mar 3, 2010 at 3:51 PM, Ralf Wildenhues ralf.wildenh...@gmx.de 
 wrote:
 * NightStrike wrote on Wed, Mar 03, 2010 at 06:59:53PM CET:
 Automake somehow defines AR to 'ar'.  I'm not sure where this comes
 from, but I do know that it's definitely not $host-ar, as I would
 expect.

 Is this an automake bug, or user error?

 Looks like an automake bug to me.  Just putting
  AC_CHECK_TOOL([AR], [ar], [false])

 somewhere in configure.ac should serve as a workaround though.

 Thanks for the report, will fix,
 Ralf


 Was this ever fixed?  What version of automake will it be in?


Ping



Re: Bison and automake together

2010-10-31 Thread Philip Herron
On 30 October 2010 11:46, Paulo J. Matos pocma...@gmail.com wrote:
 Hi,

 I am slightly confused as to whether I should be posting this to the
 bison or this mailing list but here it goes.

 I have a bison based project which has the following definitions:
 ,
 | BUILT_SOURCES = scgparser.h
 | AM_YFLAGS = -d
 | AM_LDFLAGS = -lgmpxx -lgmp
 |
 | bin_PROGRAMS = scgc
 |
 | scgc_SOURCES = scgc.cc scgparser-driver.cc scgparser.yy scglexer.l
 `

 When I ran this with ./configure  make I get:
 ,
 | /bin/bash ./ylwrap scgparser.yy y.tab.c scgparser.cc y.tab.h \
 |           scgparser.h y.output scgparser.output -- bison -y  -d
 `

 This is ok because bison generates y.tab.c and y.tab.h that are renamed
 by ylwrap as scgparser.cc and scgparser.h respectively.

 However, bison generated parser scgparser.cc includes y.tab.h (which was
 the name known to bison when the file was generated). This results in
 an error during compilation since there is no y.tab.h anymore when the
 scgparser.cc is compiled. What am I missing here?

 Feel free to redirect me to the bison mailing list if this is not an
 automake config problem.


Your bison generated header also gets renamed by ylwrap, and yeah this
is probably the best place to ask about this problem your bison header
(y.tab.h) will get renamed to the same name as your bla.y but changed
to bla.h.

Notice the make line you posted was scgparser.cc y.tab.h scgparser.h
See the y.tab.h then scgparser.h.

Hope this helps!

--Phil



Re: ylwrap - flex filename weirdness

2010-10-31 Thread Philip Herron
On 31 October 2010 10:54, Jan Engelhardt jeng...@medozas.de wrote:
 Hi,


 While trying to utilize Autotools in a preexisting project that
 previously just relied on Makefiles, I came across ylwrap failing for
 some absurd, unknown reason. I couldn't help but run sh -x and debug it
 piecewise.

 So what's ultimately done is running flex inside a temp directory,

        flex /home/jengelh/code/iproute2/tc/emp_ematch_lex.l

 It turns out flex produces a file called lex.ematch_.c, which is
 something ylwrap so does not expect at all - it only checks for
 what's in pairlist=yy.lex.c emp_ematch_lex.c.

 I can use AM_LFLAGS = -o yy.lex.c to work around it, but that feels like
 an ugly hack. What's up with flex here?


Ylwrap renames your Lex and Yacc files to their respective source
names.{c,h} So if your lexer was foo.l it would output foo.c not
lex.yy.c which is what your expecting. Here is a Makefile.am snippet
of a basic compiler i write a while back:

bin_PROGRAMS= cmod$(EXEEXT)
noinst_LIBRARIES = libparser.a

AM_CPPFLAGS = -I$(top_srcdir)/include
AM_YFLAGS = -d

cmod_SOURCES = \
mm-cmod.c \
dd-util.c \
dd-vec.c \
ii-toplev.c \
ii-mod-dot.c \
bb-linux-i386.c

cmod_LDADD  = libparser.a

libcparser_a_YFLAGS = -d
libparser_a_CFLAGS = -DMOD -DPARSER
libparser_a_SOURCES = ss-parser.y ss-lexer.l

Hope this helps.

--Phil



Re: Bison and automake together

2010-10-31 Thread Pippijn van Steenhoven
Hi,

On Sat, Oct 30, 2010 at 11:46:25AM +0100, Paulo J. Matos wrote:
 However, bison generated parser scgparser.cc includes y.tab.h (which was
 the name known to bison when the file was generated). This results in
 an error during compilation since there is no y.tab.h anymore when the
 scgparser.cc is compiled. What am I missing here? 

I solved this problem by adding my own y.tab.h which does nothing but
#include scgparser.h.

-- 
Pippijn van Steenhoven


signature.asc
Description: Digital signature


Re: ylwrap - flex filename weirdness

2010-10-31 Thread Jan Engelhardt

On Sunday 2010-10-31 16:42, Philip Herron wrote:

 While trying to utilize Autotools in a preexisting project that
 previously just relied on Makefiles, I came across ylwrap failing for
 some absurd, unknown reason. I couldn't help but run sh -x and debug it
 piecewise.

 So what's ultimately done is running flex inside a temp directory,

        flex /home/jengelh/code/iproute2/tc/emp_ematch_lex.l

 It turns out flex produces a file called lex.ematch_.c, which is
 something ylwrap so does not expect at all - it only checks for
 what's in pairlist=yy.lex.c emp_ematch_lex.c.

 I can use AM_LFLAGS = -o yy.lex.c to work around it, but that feels like
 an ugly hack. What's up with flex here?

Ylwrap renames your Lex and Yacc files to their respective source
names.{c,h} So if your lexer was foo.l it would output foo.c not
lex.yy.c which is what your expecting.

I'm not expecting anything, ylwrap is. For this very rename that
ylwrap is supposed to do, it is expecting to see lex.yy.c or
source_name.c from the flex subcommand it calls, but never gets
that.



Re: ylwrap - flex filename weirdness

2010-10-31 Thread Pippijn van Steenhoven
On Sun, Oct 31, 2010 at 11:54:04AM +0100, Jan Engelhardt wrote:
 Hi,
 
 
 While trying to utilize Autotools in a preexisting project that 
 previously just relied on Makefiles, I came across ylwrap failing for 
 some absurd, unknown reason. I couldn't help but run sh -x and debug it 
 piecewise.
 
 So what's ultimately done is running flex inside a temp directory,
 
   flex /home/jengelh/code/iproute2/tc/emp_ematch_lex.l
 
 It turns out flex produces a file called lex.ematch_.c, which is 
 something ylwrap so does not expect at all - it only checks for 
 what's in pairlist=yy.lex.c emp_ematch_lex.c.
 
 I can use AM_LFLAGS = -o yy.lex.c to work around it, but that feels like 
 an ugly hack. What's up with flex here?

You added a prefix ematch_ in your flex source file. ylwrap doesn't
like that. In fact, ylwrap doesn't like multiple parsers in one project,
at all... and it doesn't bother renaming flex-generated header files,
either.

-- 
Pippijn van Steenhoven


signature.asc
Description: Digital signature


Re: Using -MMD instead of -MD for depndency generation

2010-10-31 Thread Paul Smith
On Thu, 2010-10-28 at 12:18 -0400, Paul Smith wrote:
  Is there anything I can help with? bisect GNU make 3.81 - 3.82 to see when 
  it
  got slower? Or anything on our GNU make/automake usage? E.g. we use a non
  recursive makefile
 
 I'll try to look into this more this afternoon/tonight.  Until then I
 wouldn't bother doing too much work on it.  Let me try to reproduce your
 results then we can see where we are.

I was able to install the proper packages and get a complete build of
WebKit.

After this I see that GNU make 3.82 is twice as slow as 3.81, but even
3.82 only takes about 30 seconds to run (a do-nothing build) on my
system.  Taking 5+ minutes is crazy.  There must be something odd about
your system or setup causing this.

As for why 3.82 is slower, unfortunately I'm having problems figuring it
out.  I compiled with gprof but the cumulative profiled code in GNU make
only took 6 seconds or so, so I suppose the other 24 seconds must be in
libc or someplace... but trying to install a profile-enabled version of
libc on my system did not succeed (or rather the package install worked
but linking with the profiled libc did not work).


All I can assume is there's some difference in heap management in 3.82
which is causing this, because that's the only difference I can think of
that would not show up in the GNU make code itself.  I'll try using
valgrind and see what I get.  If anyone has other suggestions for
profiling tools let me know.

Cheers!

-- 
---
 Paul D. Smith psm...@gnu.org  Find some GNU make tips at:
 http://www.gnu.org  http://make.mad-scientist.net
 Please remain calm...I may be mad, but I am a professional. --Mad Scientist




Re: Bison and automake together

2010-10-31 Thread Paulo J. Matos
Philip Herron redbr...@gcc.gnu.org writes:

 Your bison generated header also gets renamed by ylwrap, and yeah this
 is probably the best place to ask about this problem your bison header
 (y.tab.h) will get renamed to the same name as your bla.y but changed
 to bla.h.

 Notice the make line you posted was scgparser.cc y.tab.h scgparser.h
 See the y.tab.h then scgparser.h.


I know, but that is not the problem. The problem is that the bison
generated file includes y.tab.h but this doesn't exist since, as you
said, ylwrap renames it.

-- 
PMatos




Re: Bison and automake together

2010-10-31 Thread Paulo J. Matos
Pippijn van Steenhoven pip8...@gmail.com writes:


 I solved this problem by adding my own y.tab.h which does nothing but
 #include scgparser.h.

Thanks. This sounds like a good workaround but it's far from a perfect
solution. 

-- 
PMatos




Re: Using -MMD instead of -MD for depndency generation

2010-10-31 Thread Holger Freyther
On 10/31/2010 06:24 PM, Paul Smith wrote:

 All I can assume is there's some difference in heap management in 3.82
 which is causing this, because that's the only difference I can think of
 that would not show up in the GNU make code itself.  I'll try using
 valgrind and see what I get.  If anyone has other suggestions for
 profiling tools let me know.

Hi,

I can use oprofile on my system and send you the oparchive? Or if perf
supports something like this use that. My guess is that limited RAM + dm-crypt
on faster but still slow laptop disks made make a difference.




Re: Using -MMD instead of -MD for depndency generation

2010-10-31 Thread Miles Bader
Paul Smith psm...@gnu.org writes:
 As for why 3.82 is slower, unfortunately I'm having problems figuring it
 out.  I compiled with gprof but the cumulative profiled code in GNU make
 only took 6 seconds or so, so I suppose the other 24 seconds must be in
 libc or someplace... but trying to install a profile-enabled version of
 libc on my system did not succeed (or rather the package install worked
 but linking with the profiled libc did not work).

The prof program distributed with recent linux kernels is extremely
useful too -- it's much easier and less intrusive to use than gprof in
many cases.

[The annoyance is that it's still not packaged in debian for whatever
reason... you can fetch the linux sources and just build in the prof
directory, but you need to make sure the source versions match the your
kernel you're using, since the kernel interfaces have changed around at
various points.]

-Miles

-- 
Suppose He doesn't give a shit?  Suppose there is a God but He
just doesn't give a shit?  [George Carlin]




Re: Bison and automake together

2010-10-31 Thread Philip Herron
On 31 October 2010 18:49, Paulo J. Matos pocma...@gmail.com wrote:
 Philip Herron redbr...@gcc.gnu.org writes:

 Your bison generated header also gets renamed by ylwrap, and yeah this
 is probably the best place to ask about this problem your bison header
 (y.tab.h) will get renamed to the same name as your bla.y but changed
 to bla.h.

 Notice the make line you posted was scgparser.cc y.tab.h scgparser.h
 See the y.tab.h then scgparser.h.


 I know, but that is not the problem. The problem is that the bison
 generated file includes y.tab.h but this doesn't exist since, as you
 said, ylwrap renames it.


Your bison file shouldnt generate any code with includes y.tab.h. You
must have it in your field delcarations for bison. Its only your lexer
needs to see these bison definitions.

Heres some example:

http://code.redbrain.co.uk/cgit.cgi/qub-mod-c/tree/src/ss-lexer.l
http://code.redbrain.co.uk/cgit.cgi/qub-mod-c/tree/src/ss-parser.y

--Phil



Re: Bison and automake together

2010-10-31 Thread Pippijn van Steenhoven
On Sun, Oct 31, 2010 at 10:55:53PM +, Philip Herron wrote:
 Your bison file shouldnt generate any code with includes y.tab.h. You
 must have it in your field delcarations for bison. Its only your lexer
 needs to see these bison definitions.

It does for GLR parsers.

-- 
Pippijn van Steenhoven


signature.asc
Description: Digital signature


Re: Bison and automake together

2010-10-31 Thread Philip Herron
Then thats probably a bug although i havent played with GLR prarsers
in bison you may want to post this to bison-help and see what they
say.

--Phil

On 31 October 2010 23:03, Pippijn van Steenhoven pip8...@gmail.com wrote:
 On Sun, Oct 31, 2010 at 10:55:53PM +, Philip Herron wrote:
 Your bison file shouldnt generate any code with includes y.tab.h. You
 must have it in your field delcarations for bison. Its only your lexer
 needs to see these bison definitions.

 It does for GLR parsers.

 --
 Pippijn van Steenhoven

 -BEGIN PGP SIGNATURE-
 Version: GnuPG v1.4.9 (GNU/Linux)

 iEYEARECAAYFAkzN9cwACgkQJc+zqGNdDgrhqwCg4kJPoABYHKjEgSXjPd5M+5QT
 xwYAoLpUNUFTcQyVNk3U2S9uy+L7pt0P
 =rQxd
 -END PGP SIGNATURE-