Bug#756123: chained implicit rule can fail to make a prerequisite

[Alex Vong]

Hi everyone,

The bug appears to be fixed in GNU Make 4.0 and 4.1 by some unknown commits.

According to ,

Hi Manoj; I ran the test with both 3.81 and 3.82 and it failed almost
immediately: I ran it 100 times with 4.0 and 4.1 and couldn't reproduce the
failure. So, some change between 3.82 and 4.0 fixed the issue; unfortunately
I'm not sure which one so I can't help with that. It probably wouldn't be 
too
difficult to use git bisect to find the particular change if you wanted to
backport it.

I am going to close this bug report after some time. Feel free to ping me if 
you have any problems.

Cheers,
Alex

Bug#756123: chained implicit rule can fail to make a prerequisite [and 1 more messages]

[Ian Jackson]

Manoj Srivastava writes (Re: Bug#756123: chained implicit rule can fail to 
make a prerequisite [and 1 more messages]):
 In that case, I shall forward this bug report upstream, to see
  if we can get help in identifying the patch that fixed the issue (there
  ight not be a simpe set of commits that I might be able to backport,
  though). Even if we do find the fixes, we would still need to talk to
  the SRM team to see if this can be backported into stable.

Right.  I am prepared to try to properly debug this in 3.81, but I
haven't ever looked at the code inside make so I would need some help
for that to be a viable strategy.

I could also bisect if there's a bisectable git history somewhere.

 I am traveling until Friday, and so will deal with this when I
  get back.

Thanks.  To be honest, while this bug seems pretty strange, I doubt it
can be very critical in the grand scheme of things.  From the point of
view of a stable update, it might be better to leave everyone with the
bugs they have rather than try to patch this - unless the fix is very
very obviously correct.

So I won't be upset if you don't want to put more effort into this
bug.

Regards,
Ian.


-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org

Bug#756123: chained implicit rule can fail to make a prerequisite [and 1 more messages]

[Manoj Srivastava]

On Mon, Aug 04 2014, Ian Jackson wrote:

 Manoj Srivastava writes (Re: Bug#756123: chained implicit rule can
 fail to make a prerequisite [and 1 more messages]):
 In that case, I shall forward this bug report upstream, to see
  if we can get help in identifying the patch that fixed the issue (there
  ight not be a simpe set of commits that I might be able to backport,
  though). Even if we do find the fixes, we would still need to talk to
  the SRM team to see if this can be backported into stable.

 Right.  I am prepared to try to properly debug this in 3.81, but I
 haven't ever looked at the code inside make so I would need some help
 for that to be a viable strategy.

 I could also bisect if there's a bisectable git history somewhere.

The upstream git repository is at:
 git://git.sv.gnu.org/make.git

The debian tree is atL
 git.debian.org/~srivasta/public_git/debian/make-dfsg.git

I use the upstream git repo locally to cherry-pick fxes, so I
 think that bisecting might work.

 I am traveling until Friday, and so will deal with this when I
  get back.

 Thanks.  To be honest, while this bug seems pretty strange, I doubt it
 can be very critical in the grand scheme of things.  From the point of
 view of a stable update, it might be better to leave everyone with the
 bugs they have rather than try to patch this - unless the fix is very
 very obviously correct.

 So I won't be upset if you don't want to put more effort into this
 bug.

I am going to forward this upstream anyway, and I suspect it
 will be closed, but perhaps with a pointer to the issue or patch that
 fixed the race condition. The new upstream report is at:
  https://savannah.gnu.org/bugs/index.php?42910

manoj
-- 
Your boss is thinking about you.
Manoj Srivastava sriva...@acm.org http://www.golden-gryphon.com/  
4096R/C5779A1C E37E 5EC5 2A01 DA25 AD20  05B6 CF48 9438 C577 9A1C


signature.asc
Description: PGP signature

Bug#756123: chained implicit rule can fail to make a prerequisite [and 1 more messages]

[Ian Jackson]

Manoj Srivastava writes (Bug#756123: chained implicit rule can fail to make a 
prerequisite [and 1 more messages]):
 for i in $(seq 1 10); do
 for  set -e ; touch ../src/addrfam.c  rm -f adnstest_c.o  make -j4 
 adnstest_record; set + e
 for done

I tried make 4.0-8 in my sid chroot and it worked.  I.e., I wasn't
able to repro the bug with gnu make 4.

Ian.


-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org

Bug#756123: chained implicit rule can fail to make a prerequisite [and 1 more messages]

[Manoj Srivastava]

On Tue, Jul 29 2014, Ian Jackson wrote:

 Manoj Srivastava writes (Bug#756123: chained implicit rule can fail
 to make a prerequisite [and 1 more messages]):
 for i in $(seq 1 10); do
 for  set -e ; touch ../src/addrfam.c  rm -f adnstest_c.o  make -j4 
 adnstest_record; set + e
 for done

 I tried make 4.0-8 in my sid chroot and it worked.  I.e., I wasn't
 able to repro the bug with gnu make 4.

In that case, I shall forward this bug report upstream, to see
 if we can get help in identifying the patch that fixed the issue (there
 ight not be a simpe set of commits that I might be able to backport,
 though). Even if we do find the fixes, we would still need to talk to
 the SRM team to see if this can be backported into stable.

I am traveling until Friday, and so will deal with this when I
 get back.

manoj
-- 
I never let my schooling get in the way of my education. Mark Twain
Manoj Srivastava sriva...@acm.org http://www.golden-gryphon.com/  
4096R/C5779A1C E37E 5EC5 2A01 DA25 AD20  05B6 CF48 9438 C577 9A1C


signature.asc
Description: PGP signature

Bug#756123: chained implicit rule can fail to make a prerequisite

[Ian Jackson]

Package: make
Version: 3.81-8.2

I have a Makefile where make appears to decide to use the 2nd of a
pair of chained implicit rules to construct a file, despite having
earlier rejected this same pair of rules when deciding whether to
build the necessasry intermediate file.

As a result, make can run the rules for the 2nd implicit rule despite
the fact that one of its prerequisites has not been built.  The
commands for the 2nd rule then fail.


To reproduce:

Firstly, preparation:

 $ git clone git://git.chiark.greenend.org.uk/~ianmdlvl/adns.git
   [ usual output from git clone ]
 $ cd adns
 $ git checkout make-bug.2014-07-26
   [ warning about git about detached HEAD ]
 $ ./configure
 $ make
   [ build should complete successfully ]
 $ cd regress

Now repeat the following steps until you lose the race like this:

 $ touch ../src/addrfam.c 
 $ make -j2
 gcc -g -O2 -Wall -Wmissing-prototypes -Wwrite-strings -Wstrict-prototypes 
-Wcast-qual -Wpointer-arith -Wno-pointer-sign  -I../src -I. -I./../src  
-DADNS_REGRESS_TEST -I. -c -g -o addrfam_d.o ../src/addrfam.c
 gccadnstest_c.o hrecord.o hcommon.o types_d.o event_d.o query_d.o 
reply_d.o general_d.o setup_d.o transmit_d.o parse_d.o poll_d.o check_d.o 
addrfam_d.o  -o adnstest_record
 gcc: error: adnstest_c.o: No such file or directory
 make: *** [adnstest_record] Error 1
 $

(If you win the race it rebuilds adnstest_c.o again.)


Detailed consideration of the manual node `Implicit Rule Search'
suggests that the Makefile is correct by virtue of para.6: while the
intermediate file adnstest_c.o does not exist, and is not mentioned as
a target or explicit prerequisite, it can be made.

So make is correct to try to use this rule.  But of course it ought to
construct the file first.

The bug occurs only with nontrivial -j options.  I have reproduced it
with -j4 and (as here) -j2.

I have also, once, seen it do this:

 regress$ touch ../src/addrfam.c; make -j adnstest_record
 gcc -g -O2 -Wall -Wmissing-prototypes -Wwrite-strings -Wstrict-prototypes 
-Wcast-qual -Wpointer-arith -Wno-pointer-sign -I../src -I. -I./../src  
-DADNS_REGRESS_TEST -I. -c -g -o addrfam_d.o ../src/addrfam.c
 regress$

I.e., it built one of the necessary prerequisites and then exited.
(I'm afraid I don't have the exit status.)


I searched the upstream bug database and found this:
  bug #29620: Implicit rule search is still sensitive to random side effects
  http://savannah.gnu.org/bugs/index.php?29620

That seems possibly related but not exactly my problem.


I have found that it is possible to work around this problem by adding
  $(addsuffix _c.o, $(filter-out adnshost,$(CLIENTS))):
or
  .SECONDARY: $(addsuffix _c.o, $(CLIENTS))
to the Makefile.


Thanks,
Ian.


-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org

Bug#756123: chained implicit rule can fail to make a prerequisite [and 1 more messages]

[Ian Jackson]

I wrote:
 I have found that it is possible to work around this problem by adding
   $(addsuffix _c.o, $(filter-out adnshost,$(CLIENTS))):
 or
   .SECONDARY: $(addsuffix _c.o, $(CLIENTS))
 to the Makefile.

In fact, this is not completely correct.  Adding those things to the
makefile causes make to not automatically delete %_c.o (eg,
adnstest_c.o).  That makes it much less likely that with this
particular Makefile the user will trip over it.

Even with those things in the Makefile, I can reproduce the problem by
running
  touch ../src/addrfam.c; rm adnstest_c.o; make -j adnstest_record

Thanks,
Ian.


-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org

Bug#756123: chained implicit rule can fail to make a prerequisite

[Manoj Srivastava]

Hi,

I can't seem to bve able to reproduce this on
--8---cut here---start-8---
 make --version
GNU Make 4.0
Built for x86_64-pc-linux-gnu
Copyright (C) 1988-2013 Free Software Foundation, Inc.
--8---cut here---end---8---

How often must one run the make in ./regress until one loses the
 race?  I gave up after 50 tries (half of them with -j4). This is a quad
 core box.

I could not see anything in the changelog to indicate that this
 issue would have been addressed, and the upstream bug mentioned in the
 report is still open.  However, being unable to reproduce the errors
 does put a crimp in my ability to investigate the error.

manoj

--8---cut here---start-8---
 touch ../src/addrfam.c  make -j4
gcc -g -O2 -Wall -Wmissing-prototypes -Wwrite-strings -Wstrict-prototypes 
-Wcast-qual -Wpointer-arith -Wno-pointer-sign  -I../src -I. -I./../src  
-DADNS_REGRESS_TEST -I. -c -g -o addrfam_d.o ../src/addrfam.c
gcc -g -O2 -Wall -Wmissing-prototypes -Wwrite-strings -Wstrict-prototypes 
-Wcast-qual -Wpointer-arith -Wno-pointer-sign  -I../src -I. -I./../src  
-DADNS_REGRESS_TEST -I. -I ./../src -c -g -o adnstest_c.o ../client/adnstest.c
gccadh-main_c.o adh-opts_c.o adh-query_c.o hrecord.o hcommon.o types_d.o 
event_d.o query_d.o reply_d.o general_d.o setup_d.o transmit_d.o parse_d.o 
poll_d.o check_d.o addrfam_d.o  -o adnshost_record
gcc -g -O2 -Wall -Wmissing-prototypes -Wwrite-strings -Wstrict-prototypes 
-Wcast-qual -Wpointer-arith -Wno-pointer-sign  -I../src -I. -I./../src  
-DADNS_REGRESS_TEST -I. -I ./../src -c -g -o adnslogres_c.o 
../client/adnslogres.c
gcc -g -O2 -Wall -Wmissing-prototypes -Wwrite-strings -Wstrict-prototypes 
-Wcast-qual -Wpointer-arith -Wno-pointer-sign  -I../src -I. -I./../src  
-DADNS_REGRESS_TEST -I. -I ./../src -c -g -o adnsresfilter_c.o 
../client/adnsresfilter.c
gccadh-main_c.o adh-opts_c.o adh-query_c.o hplayback.o hcommon.o types_d.o 
event_d.o query_d.o reply_d.o general_d.o setup_d.o transmit_d.o parse_d.o 
poll_d.o check_d.o addrfam_d.o  -o adnshost_playback
gccadnslogres_c.o hplayback.o hcommon.o types_d.o event_d.o query_d.o 
reply_d.o general_d.o setup_d.o transmit_d.o parse_d.o poll_d.o check_d.o 
addrfam_d.o  -o adnslogres_playback
gccadnstest_c.o hrecord.o hcommon.o types_d.o event_d.o query_d.o reply_d.o 
general_d.o setup_d.o transmit_d.o parse_d.o poll_d.o check_d.o addrfam_d.o  -o 
adnstest_record
gccadnslogres_c.o hrecord.o hcommon.o types_d.o event_d.o query_d.o 
reply_d.o general_d.o setup_d.o transmit_d.o parse_d.o poll_d.o check_d.o 
addrfam_d.o  -o adnslogres_record
gccadnsresfilter_c.o hrecord.o hcommon.o types_d.o event_d.o query_d.o 
reply_d.o general_d.o setup_d.o transmit_d.o parse_d.o poll_d.o check_d.o 
addrfam_d.o  -o adnsresfilter_record
gccadnstest_c.o hplayback.o hcommon.o types_d.o event_d.o query_d.o 
reply_d.o general_d.o setup_d.o transmit_d.o parse_d.o poll_d.o check_d.o 
addrfam_d.o  -o adnstest_playback
gccadnsresfilter_c.o hplayback.o hcommon.o types_d.o event_d.o query_d.o 
reply_d.o general_d.o setup_d.o transmit_d.o parse_d.o poll_d.o check_d.o 
addrfam_d.o  -o adnsresfilter_playback
rm adnstest_c.o adnsresfilter_c.o adnslogres_c.o
--8---cut here---end---8---

-- 
Unix: a moment of convenience, a lifetime of regret.  old ITS hacker
saying
Manoj Srivastava sriva...@acm.org 
4096R/C5779A1C E37E 5EC5 2A01 DA25 AD20  05B6 CF48 9438 C577 9A1C


signature.asc
Description: PGP signature

Bug#756123: chained implicit rule can fail to make a prerequisite [and 1 more messages]

[Manoj Srivastava]

for i in $(seq 1 10); do
for  set -e ; touch ../src/addrfam.c  rm -f adnstest_c.o  make -j4 
adnstest_record; set + e
for done

 

gcc -g -O2 -Wall -Wmissing-prototypes -Wwrite-strings -Wstrict-prototypes 
-Wcast-qual -Wpointer-arith -Wno-pointer-sign  -I../src -I. -I./../src  
-DADNS_REGRESS_TEST -I. -c -g -o addrfam_d.o ../src/addrfam.c
gcc -g -O2 -Wall -Wmissing-prototypes -Wwrite-strings -Wstrict-prototypes 
-Wcast-qual -Wpointer-arith -Wno-pointer-sign  -I../src -I. -I./../src  
-DADNS_REGRESS_TEST -I. -I ./../src -c -g -o adnstest_c.o ../client/adnstest.c
gccadnstest_c.o hrecord.o hcommon.o types_d.o event_d.o query_d.o reply_d.o 
general_d.o setup_d.o transmit_d.o parse_d.o poll_d.o check_d.o addrfam_d.o  -o 
adnstest_record
rm adnstest_c.o



Still no go. :-(


manoj
-- 
Never eat at a place called Mom's.  -- Nelson Algren
Manoj Srivastava sriva...@acm.org 
4096R/C5779A1C E37E 5EC5 2A01 DA25 AD20  05B6 CF48 9438 C577 9A1C


signature.asc
Description: PGP signature

Bug#756123: chained implicit rule can fail to make a prerequisite

[Ian Jackson]

Manoj Srivastava writes (Bug#756123: chained implicit rule can fail to make a 
prerequisite):
 --8---cut here---start-8---
  make --version
 GNU Make 4.0
 Built for x86_64-pc-linux-gnu
 Copyright (C) 1988-2013 Free Software Foundation, Inc.
 --8---cut here---end---8---
 
 How often must one run the make in ./regress until one loses the
  race?  I gave up after 50 tries (half of them with -j4). This is a quad
  core box.

On my 2-core 4-HT Atom netbook I lose the race around 50% of the time
with make 3.81.

 I could not see anything in the changelog to indicate that this
  issue would have been addressed, and the upstream bug mentioned in the
  report is still open.  However, being unable to reproduce the errors
  does put a crimp in my ability to investigate the error.

I managed to repro the race under strace and with -d and -p.  You
could try that, or I could send you the output.

Or maybe I can try to make a recipe to force it by putting sleeps in
the right place, although I realy don't know well enough how make
works to do that.

Ian.


-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org