[bug #27609] Stupid inference rule for yacc files can clobber C sources!

[Paul D. Smith]

Follow-up Comment #12, bug #27609 (project make):

It's hard to diagnose problems based on general descriptions.  Please provide
specific examples of what you did and what the result was.

However, it's definitely true that if you're relying on some of the built-in
rules then turning them all off with .SUFFIXES: will break that.

Please see the GNU make manual section on suffix rules.  First, when defining
suffixes you add them one suffix at a time, not as a group (use ".o" and ".c",
not ".o.c", for example).  Second, suffixes as .SUFFIXES: prerequisites only
_adds_ to the existing suffix list, it doesn't _replace_ the existing suffix
list.

If you're trying to keep only some of the rules then first you have to turn
them all off, then add back just the ones you want:


.SUFFIXES:
.SUFFIXES: .c .o




___

Reply to this item at:

  

___
  Message sent via/by Savannah
  http://savannah.gnu.org/


___
Bug-make mailing list
Bug-make@gnu.org
https://lists.gnu.org/mailman/listinfo/bug-make

[bug #27609] Stupid inference rule for yacc files can clobber C sources!

[Kaz Kylheku]

Follow-up Comment #11, bug #27609 (project make):

I tried .SUFFIXES: but now make tried to build the program even though
lex.yy.o is a prerequisite, and is missing, without any complaints. This is,
of course, because I have a rule which gives a dependency from lex.yy.o to
lex.yy.c, but which has no recipe.

I then added .o.c to suffixes, in hopes that  this would turn on rules
involving .o and .c files only.

The lex.yy.o: lex.yy.c rule now started to work again.

But alas, the bad rule now kicked in!

$ make
lex -i -8 hc.l
cc -g -Wall -W -ansi -D_XOPEN_SOURCE=500-c -o lex.yy.o lex.yy.c
lex.yy.c:4548:17: warning: ‘yyunput’ defined but not used
[-Wunused-function]
lex.yy.c:4589:16: warning: ‘input’ defined but not used
[-Wunused-function]
lex  -t hc.l > hc.c   <--- HUH?

Is that a bug? This is make 3.81 on Ubuntu 11.04.

If not, what is the real way to put the through the heart of this beast (and
anything like it, known or unknown), without losing the useful implicit
rules?

How can I say, "I want only the implicit rules which involve only these file
types and do not involve any other file types."

___

Reply to this item at:

  

___
  Message sent via/by Savannah
  http://savannah.gnu.org/


___
Bug-make mailing list
Bug-make@gnu.org
https://lists.gnu.org/mailman/listinfo/bug-make

[bug #27609] Stupid inference rule for yacc files can clobber C sources!

[Paul D. Smith]

Follow-up Comment #10, bug #27609 (project make):

Yes, both .y.c and .l.c suffix rules are required by the POSIX standard. 
However, they've been present in GNU make for over 20 years, since before
POSIX standardized make, so while POSIX requirements are one reason to keep
them, their existence cannot be blamed on POSIX.

I recommend getting into the habit of adding a ".SUFFIXES:" target to your
makefiles to disable the built-in suffix rules, so that the worst that can
happen if your rules fail to apply for some reason is that make complains it
doesn't know how to build the target.

___

Reply to this item at:

  

___
  Message sent via/by Savannah
  http://savannah.gnu.org/


___
Bug-make mailing list
Bug-make@gnu.org
https://lists.gnu.org/mailman/listinfo/bug-make

[bug #27609] Stupid inference rule for yacc files can clobber C sources!

[Kaz Kylheku]

Follow-up Comment #9, bug #27609 (project make):

What if these dangerous implicit rules made a rotating backup of their victim?
What is the downside?

mv foo.c.keep.1 foo.c.keep.2
mv foo.c foo.c.keep.1
mv y.tab.c foo.c


___

Reply to this item at:

  

___
  Message sent via/by Savannah
  http://savannah.gnu.org/


___
Bug-make mailing list
Bug-make@gnu.org
https://lists.gnu.org/mailman/listinfo/bug-make

[bug #27609] Stupid inference rule for yacc files can clobber C sources!

[Kaz Kylheku]

Follow-up Comment #8, bug #27609 (project make):

I was just burned by this again! I have a small project with "hc.l" lexer, and
a "hc.c" source file.  It's been working fine. 

My Makefile was generating lex.yy.c from hc.l, compiling hc.c to hc.o, and
lex.yy.c to lex.yy.o, and linking everything.

All of a sudden though, for some reason, an internal rule in make kicked in
and did this:

   lex -t hc.l > hc.c

oops, my source file is gone; all I have is the most recent git repo copy
which doesn't have all my recent work.

Is this is another POSIX-required misfeature?

This is the sort of thing for which Stallman invented --posix-me-harder.


___

Reply to this item at:

  

___
  Message sent via/by Savannah
  http://savannah.gnu.org/


___
Bug-make mailing list
Bug-make@gnu.org
https://lists.gnu.org/mailman/listinfo/bug-make

[bug #27609] Stupid inference rule for yacc files can clobber C sources!

[Paul D. Smith]

Update of bug #27609 (project make):

  Item Group: Bug => Enhancement


___

Reply to this item at:

  

___
  Message sent via/by Savannah
  http://savannah.gnu.org/


___
Bug-make mailing list
Bug-make@gnu.org
https://lists.gnu.org/mailman/listinfo/bug-make

[bug #27609] Stupid inference rule for yacc files can clobber C sources!

[Kaz Kylheku]

Follow-up Comment #7, bug #27609 (project make):

Also, another consideration is that the prerequisite .y is involved in an
explicit rule.   If an implicit rule matches some file as a prerequisite, but
that file is already used as a prerequisite in some rule, then those rules are
competing. Maybe make should assume that the explicit rule takes precedence.

I'm not so confident about this because there are situations where something
is a prerequisite to more than one rule. (Like the obvious case of a header
file being a dependency for many objects).

Nevertheless, there are situations where prerequisites are typically
understood to be processed by just one rule. You probably would not want to
pass a .y file through yacc twice in two different rules. If such a situation
exists, and one of the rules is generated from an implicit template, but the
other one is explicit, somehow the explicit one should win.

Or some syntax could denote ``weak'' rules. 

You know how there are order-only prerequisites; there could be a similar
extension to denote weak-exclusive prerequisites, and another extension to
denote clobbers.

For the sake of concrete discussion, let's use the characters @ and & for
clobbers and exclusive prerequisites. The rule could then look like:

  %.c: & %.y @ y.tab.c
 $(YACC) ...
 mv ...


So now, if the rule matches some ``foo'' stem, we have a target foo.c, a
exclusive prerequisite foo.y, and a clobber y.tab.c.

The exclusive prerequisite tells make that this rule must be exclusive for
that prerequisite. If two explicit rules are exlusive for the same
prerequisite, then the situation is erroneous: a diagnostic is emitted and the
make fails. If the exclusivity conflict is between an implicit and explicit
rule (like the above case), then the implicit rule is treated as weak: it is
resolved in favor of the explicit rule.

The clobber element y.tab.c informs make that the rule body destroys the file
y.tab.c, which is neither a target nor prerequisite.  Make will suppress this
rule if y.tab.c appears as a target or prerequisite in any other rule
(possibly with some warning diagnostic?). Moreover, if two or more rules both
specify y.tab.c as a clobber, the execution of those rules will be serialized
(in either order, just not parallel).

Note that this allows phony clobber files to be used as mutexes for
serializing rules, which could be handy! Suppose that two rules exist which,
say, do something destructive in the same CVS sandbox (like an update) which
can modify the CVS/Entries and whatnot. They could have a clobber called
``cvs-mutex'', and make would thus not run those rules in parallel. This may
be easier than inserting fake dependencies to enforce an order. I know there
is .NOTPARALLEL, but that's a big hammer.




___

Reply to this item at:

  

___
  Message sent via/by Savannah
  http://savannah.gnu.org/



___
Bug-make mailing list
Bug-make@gnu.org
http://lists.gnu.org/mailman/listinfo/bug-make

[bug #27609] Stupid inference rule for yacc files can clobber C sources!

[Kaz Kylheku]

Follow-up Comment #6, bug #27609 (project make):

I've thought about this a little bit more. Basically, here is what may be the
real issue: the rule interferes with objects which are already entangle din
other rules.

Be it as it may that it's required by POSIX and let's take it for granted
that we want that rule because people find it useful, existing projects depend
on it, et cetera.

The problem is that if I have a makefile in which the file ``y.tab.c'' is
already involved in an explicit rule.

If the makefile has this rule:

   y.tab.c: foo.y
$(YACC) ...

make should not be interfering with this makefile by launching a ``mv
y.tab.c. xxx'' command!

This problem would mostly go away if make was smart enough to know that
target X appears on the left hand side of a rule, then it must suppress any
hidden rules which use X as a temporary file.

Surely, it's okay not to try to conform to the POSIX rule, when the makefile
explicitly /wants/ to make a target called y.tab.c.

Remmeber, the rule which makes y.tab.c could be making that y.tab.c from any
one of several Yacc files, not necessarily the same Yacc file which the
implicit rule wants to use.

In my case, it was my custom rule which made the y.tab.c, but then the
implicit rule moved it to the target (my C source which got clobbered).
Neither the source file y.tab.c, nor the target, were appropriate to the
implicit rule.

And what about parallelized builds? What if the implicit rule applies more
than one way; will it work correctly under parallel make?  Suppose that a
foo.c is to be implicitly made from a foo.y, and a bar.c is to be impliclitly
made from a bar.y. Both actions will use y.tab.c as an intermediate file.  Are
there some hidden dependencies established which serialize the execution of
the actions?

See, this is why Stallman came up with ``posix me harder''.


___

Reply to this item at:

  

___
  Message sent via/by Savannah
  http://savannah.gnu.org/



___
Bug-make mailing list
Bug-make@gnu.org
http://lists.gnu.org/mailman/listinfo/bug-make

[bug #27609] Stupid inference rule for yacc files can clobber C sources!

[Paul D. Smith]

Follow-up Comment #5, bug #27609 (project make):

First let me point out that the behavior you're concerned about is actually
required by the POSIX standard for make; the standard requires this as a built
in rule to all conforming implementations:


.y.c:
$(YACC) $(YFLAGS) $<
mv y.tab.c $@


So we can continue to discuss this to see if there's some way to alleviate
the potential bad effects, but ultimately I'm not going to remove this rule or
modify it in a way that breaks the standard (and also many makefiles out there
which are written to expect this behavior).

The heuristics you suggest for detecting yacc-generated files are not
acceptable.  Any builtin rule MUST be completely portable to all platforms
that make runs on, which includes not only all POSIX platforms with just
/bin/sh, not bash, as shells, but also VMS, Windows, DOS, OS/2, and Amiga
platforms (currently) where other tools like "diff" might not even exist. 
Furthermore, different implementations of yacc may well generate different
output in the first 3 lines.  This is simply not robust OR portable enough to
be included as the builtin rule for GNU make.

The complaint about make updating foo.c from foo.y even though foo.c already
exists is a primary feature of GNU make (see the manual discussion of implicit
rule chaining).  Even if I wanted to, which I don't, there's no way to change
this without breaking the vast majority of GNU makefiles, which rely on this
very useful behavior in some form.

A builtin rule that builds a .o directly from a .y doesn't help your
situation unless the current rule to build a .c from a .y were also removed
completely, and that's not going to happen.

You can, of course, disable these rules in your own makefiles in various
ways.

___

Reply to this item at:

  

___
  Message sent via/by Savannah
  http://savannah.gnu.org/



___
Bug-make mailing list
Bug-make@gnu.org
http://lists.gnu.org/mailman/listinfo/bug-make

[bug #27609] Stupid inference rule for yacc files can clobber C sources!

[Kaz Kylheku]

Follow-up Comment #4, bug #27609 (project make):

By the way, this is quite bad. The behavior happens even if I have an
explicit rule like this:

  foo.o: foo.c

If there is a foo.y, foo.c is clobbered, even though foo.c exists.

This is so retarded.

The search for the prerequisite of foo.o must stop when foo.c is found.

I can understand that the search, upon not finding foo.c, might go on to try
other suffixes like foo.y.

That is to say, the rule should be about making a foo.o from a foo.y, not
making a foo.c from a foo.y!!!

Yes, sure. Okay. If I have a Makefile like this:

  program: foo.o

and the only file in the directory is foo.y (and no foo.c), then go ahead and
run yacc to make foo.c from foo.y, and compile foo.c to foo.o.

Actually we don't need foo.c in that case. We just need some temporary name.
The rule has to handle the complete job of translating .y to .o:

   $(YACC) $(YACCFLAGS) foo.y
   mv y.tab.c 
   $(CC) $(CFLAGS) -c  -o foo.o

Now this actually makes sense. If there is no .c file, why not infer that .y
is the prerequisite for a .o?

Just don't involve a foo.c file in the inference chain.

 can be based on foo.y somehow (just not in a naive way like
foo.c!)  Even if it was foo.y.tab.c, or foo.y.out, or whatever.

There could even be an implicit rule which, given a foo.o, finds the
prerequisite foo.y.tab.c.

It's less likely that someone would write a file by hand called foo.y.tab.c,
than foo.c.










___

Reply to this item at:

  

___
  Message sent via/by Savannah
  http://savannah.gnu.org/



___
Bug-make mailing list
Bug-make@gnu.org
http://lists.gnu.org/mailman/listinfo/bug-make

[bug #27609] Stupid inference rule for yacc files can clobber C sources!

[Kaz Kylheku]

Follow-up Comment #3, bug #27609 (project make):

Re: what can be done about it.

Enough silly venting. How about the question of what can be done about it? I
have an idea. How about a check to see whether the .c file to be clobbered is
actually a yacc-generated file?

If the file already exists, only do the move if, say, the first three lines
of the the file are different from the first three lines of y.tab.c.

Otherwise emit a diagnostic.

If you have GNU bash, or some other shell which supports process
substitution, the logic can be expressed as:

  # Do the rename if $target does not exist, or if it
  # matches source in the first three lines.
  
  if ![ -e "$target" ] || !diff <(head -3 "$source") <(head -3 "$target");
then
mv -- "$source" "$target"
  else
echo "refusing update $target by replacing it with $source"
echo "target seems to have contents unrelated to $source"
  fi

Of course, it has to be done according to whatever portability guidelines
apply to the ruleset.

The downside would be that the rule refuses to do the move when it should:
there is a false difference in the first three lines on every incremental
rebuild. This might happen if the yacc implementation does something different
in the first three lines (like adding a date stamp). Or (probably very
unlikely) if edits in the actual yacc spec are reflected in an actual change
in the first three lines.

The output of Bison does have a date stamp in the first three lines, but it's
not generated; it's part of the boiler plate. For example:

#ifndef lint
static char const
yyrcsid[] = "$FreeBSD: src/usr.bin/yacc/skeleton.c,v 1.28 2000/01/17 02:04:06
bde Exp $";

This stamp won't change unless someone checks in the generated parser under
version control which not only does RCS keyword expansion, but recognizes and
expands the custom FreeBSD keyword.

Suppose your CVS does that. So you'd be doing a ``cvs up'', and then a
``make'', and ... oops! The parser.c got refreshed, so did parser.y, and now
GNU make refuses to rename y.tab.c to parser.c, because CVS modified the
$FreeBSD: ... $ text.

You would have to "rm parser.c" and run make again.


___

Reply to this item at:

  

___
  Message sent via/by Savannah
  http://savannah.gnu.org/



___
Bug-make mailing list
Bug-make@gnu.org
http://lists.gnu.org/mailman/listinfo/bug-make

[bug #27609] Stupid inference rule for yacc files can clobber C sources!

[Kaz Kylheku]

Follow-up Comment #2, bug #27609 (project make):

If I have two different yacc files that generate different parsers in the
same directory, then I'm going to be aware of that situation and resolve it.
If I don't resolve it, then they will both clobber y.tab.c, which is my fault.
I'm bitten by my own folly, and not by some hidden rule that destroys source
code.

That the yacc program clobbers y.tab.c is not the result of some secret rule
which only emerges when you've created an important file (not by means of
yacc) and named it y.tab.c! Everyone knows that yacc makes a y.tab.c file. 
Not everyone knows that make will clobber your foo.c, because a foo.y exists.

There are going to be other issues anyway. If I have two parsers that go into
the same link, there will be clashes on all the YY stuff. It's not enough to
just rename the files.

If I don't like the name y.tab.c, I can easily write a rule like this:

 myparser.c: myparser.y
$(YACC) $(YFLAGS) -o $@ $<

If I want that to be an implicit rule for general .y -> .c conversion, I can
do it like this:

 %.c: %.y
$(YACC) $(YFLAGS) -o $@ $<

Now I have a rule which clobbers .c files. I wrote it; it's spelled out in my
Makefile. If that overwrites a non-generated .c file, it serves me right!


___

Reply to this item at:

  

___
  Message sent via/by Savannah
  http://savannah.gnu.org/



___
Bug-make mailing list
Bug-make@gnu.org
http://lists.gnu.org/mailman/listinfo/bug-make

[bug #27609] Stupid inference rule for yacc files can clobber C sources!

[Paul D. Smith]

Follow-up Comment #1, bug #27609 (project make):

This is how the default rule for building YACC files in GNU make has always
been, for 20-odd years.  From the GNU make manual section "Catalog of Rules":


Yacc for C programs
n.c is made automatically from n.y by running Yacc with the command
`$(YACC) $(YFLAGS)'.
Lex for C programs
n.c is made automatically from n.l by running Lex. The actual command is
`$(LEX) $(LFLAGS)'.


Creating a statically-named "y.tab.c" from "foo.y" was an inane idea in the
first place: what happens if you have two different YACC files that generate
different parsers in the same directory?  Ditto, of course, for lex.

I'm not sure what we can do about this.

___

Reply to this item at:

  

___
  Message sent via/by Savannah
  http://savannah.gnu.org/



___
Bug-make mailing list
Bug-make@gnu.org
http://lists.gnu.org/mailman/listinfo/bug-make

[bug #27609] Stupid inference rule for yacc files can clobber C sources!

[Kaz Kylheku]

URL:
  

 Summary: Stupid inference rule for yacc files can clobber C
sources!
 Project: make
Submitted by: kkylheku
Submitted on: Mon 05 Oct 2009 12:36:09 PM PDT
Severity: 3 - Normal
  Item Group: Bug
  Status: None
 Privacy: Public
 Assigned to: None
 Open/Closed: Open
 Discussion Lock: Any
   Component Version: 3.80
Operating System: Any
   Fixed Release: None
   Triage Status: None

___

Details:

I have a project in which there is a lex grammar and yacc grammar. Let's call
them foo.l and foo.y.   It's somewhat of a tradition to use the same name for
both. There is no interference because one is translated to a lex.yy.c, and
the other to y.tab.c.

So, silly me, I decided to add a third file, a main driver program, which I
called foo.c, thinking that all three files foo.l, foo.y and foo.c would be
independent.

In my Makefile, I added the dependency, by naming foo.o as a prerequisite of
the main program. But I did not add a rule for how foo.o is obtained; all my
other .o files have implicitly deduced .c prerequisites.

Lo and behold, when I ran make, this happened:

  mv -f y.tab.c foo.c

There goes my foo.c file! Apparently, foo.c is connected to foo.y by
prerequisite inference.

What kind of moron writes implicit rules, to be shipped in the default rule
set of a make program, which guess up .c file names and proceed to clobber
them?

And since when has foo foo.c ever been related to foo.y as a derived object?

If I want my y.tab.c to be called something else, I will write the rule for
it.

Luckily the contents of foo.c came from another file which is under version
control. I lost only two very minor edits.





___

Reply to this item at:

  

___
  Message sent via/by Savannah
  http://savannah.gnu.org/



___
Bug-make mailing list
Bug-make@gnu.org
http://lists.gnu.org/mailman/listinfo/bug-make