bug#55025: Automake should allow one to enable POSIX make behavior

[Vincent Lefevre]

https://pubs.opengroup.org/onlinepubs/9699919799/utilities/make.html
says about the target rules:

  .POSIX
The application shall ensure that this special target is specified
without prerequisites or commands. If it appears as the first
non-comment line in the makefile, /make/ shall process the makefile
as specified by this section; otherwise, the behavior of /make/ is
unspecified.

But even though one may add a .POSIX target as the first non-comment
line in one's Makefile.am file, Automake will add various non-comment
lines before this target in the generated Makefile. I received a
remark about that for GNU MPFR. Though GNU make does not require
this target to be the first non-comment line, this may matter with
other make implementations.

This could be done either by detecting a .POSIX target in Makefile.am
or with some AM_* macro in the configure.ac file.

-- 
Vincent Lefèvre  - Web: 
100% accessible validated (X)HTML - Blog: 
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

bug#55025: Automake should allow one to enable POSIX make behavior

[Mike Frysinger]

On 19 Apr 2022 17:33, Vincent Lefevre wrote:
> https://pubs.opengroup.org/onlinepubs/9699919799/utilities/make.html
> says about the target rules:
> 
>   .POSIX
> The application shall ensure that this special target is specified
> without prerequisites or commands. If it appears as the first
> non-comment line in the makefile, /make/ shall process the makefile
> as specified by this section; otherwise, the behavior of /make/ is
> unspecified.
> 
> But even though one may add a .POSIX target as the first non-comment
> line in one's Makefile.am file, Automake will add various non-comment
> lines before this target in the generated Makefile. I received a
> remark about that for GNU MPFR. Though GNU make does not require
> this target to be the first non-comment line, this may matter with
> other make implementations.
> 
> This could be done either by detecting a .POSIX target in Makefile.am
> or with some AM_* macro in the configure.ac file.

any reason we don't just define it ourselves unconditionally ?  seems
like the whole point of Automake is for devs to not worry about these
kind of nitty details.
-mike

--- a/lib/am/header-vars.am
+++ b/lib/am/header-vars.am
@@ -14,6 +14,8 @@
 ## You should have received a copy of the GNU General Public License
 ## along with this program.  If not, see .
 
+.POSIX:
+
 VPATH = @srcdir@
 
 @SET_MAKE@


signature.asc
Description: PGP signature

bug#55025: Automake should allow one to enable POSIX make behavior

[Vincent Lefevre]

On 2023-01-13 01:19:39 -0500, Mike Frysinger wrote:
> any reason we don't just define it ourselves unconditionally ?  seems
> like the whole point of Automake is for devs to not worry about these
> kind of nitty details.

Probably. Note, however, that .POSIX affects GNU "make" on a few
points (the default is obviously not portable, but some developers
might require GNU "make", though probably for other reasons):

   If the '.POSIX' special target is defined then backslash/newline
handling is modified slightly to conform to POSIX.2: first, whitespace
preceding a backslash is not removed and second, consecutive
backslash/newlines are not condensed.

 In particular, if this target is mentioned then recipes will be
 invoked as if the shell had been passed the '-e' flag: the first
 failing command in a recipe will cause the recipe to fail
 immediately.

'warning: ignoring prerequisites on suffix rule definition'
 According to POSIX, a suffix rule cannot contain prerequisites.  If
 a rule that could be a suffix rule has prerequisites it is
 interpreted as a simple explicit rule, with an odd target name.
 This requirement is obeyed when POSIX-conforming mode is enabled
 (the '.POSIX' target is defined).  In versions of GNU 'make' prior
 to 4.3, no warning was emitted and a suffix rule was created,
 however all prerequisites were ignored and were not part of the
 suffix rule.  Starting with GNU 'make' 4.3 the behavior is the
 same, and in addition this warning is generated.  In a future
 version the POSIX-conforming behavior will be the only behavior: no
 rule with a prerequisite can be suffix rule and this warning will
 be removed.

-- 
Vincent Lefèvre  - Web: 
100% accessible validated (X)HTML - Blog: 
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

bug#55025: Automake should allow one to enable POSIX make behavior

[Karl Berry]

I am doubtful about blithely defining .POSIX unconditionally. I feel
sure that will break existing Makefiles. I don't think we should do that.

Detecting .POSIX in an existing Makefile.am and moving it to the front
sounds desirable, since that is clearly what the developer intended.

Another (not mutually exclusive) possibility is to add an Automake
option (say, "make-posix") that outputs the .POSIX line. --thanks, karl.

bug#55025: Automake should allow one to enable POSIX make behavior

[Mike Frysinger]

On 13 Jan 2023 16:01, Karl Berry wrote:
> I am doubtful about blithely defining .POSIX unconditionally. I feel
> sure that will break existing Makefiles. I don't think we should do that.
> 
> Detecting .POSIX in an existing Makefile.am and moving it to the front
> sounds desirable, since that is clearly what the developer intended.
> 
> Another (not mutually exclusive) possibility is to add an Automake
> option (say, "make-posix") that outputs the .POSIX line. --thanks, karl.

i'd be amenable to a `no-make-posix` option, but imo forcing people to
opt-in to the right behavior is the wrong mental model.

i'm also a bit skeptical of the idea that we're breaking makefiles.  if
an implementation is POSIX compliant, then it already behaves as POSIX
defines make.  as it stands now, we, and our users, are generating files
that target an undefined standard that no one agrees on, cannot be tested
or asserted, and maybe they happen to work on the developer's desktop, but
not on random users of their own.  by defining .POSIX, we turn the state
from "usually works, but sometimes fails" to "either works or fails, but
it's the same everywhere".

i grok that not everyone is POSIX-compliant, and we sometimes try to add
support for such systems where reasonable, but that's a case-by-case, and
we can't keep bending over backwards to support decades old dead software
that no one uses.
-mike


signature.asc
Description: PGP signature

bug#55025: Automake should allow one to enable POSIX make behavior

[Nick Bowler]

On 2023-01-13, Mike Frysinger  wrote:
> On 13 Jan 2023 16:01, Karl Berry wrote:
>> I am doubtful about blithely defining .POSIX unconditionally. I feel
>> sure that will break existing Makefiles. I don't think we should do that.
>>
>> Detecting .POSIX in an existing Makefile.am and moving it to the front
>> sounds desirable, since that is clearly what the developer intended.
>>
>> Another (not mutually exclusive) possibility is to add an Automake
>> option (say, "make-posix") that outputs the .POSIX line. --thanks, karl.
>
> i'd be amenable to a `no-make-posix` option, but imo forcing people to
> opt-in to the right behavior is the wrong mental model.

Does adding .POSIX: to a Makefile actually solve any real-world
portability problem with Automake generated makefiles, or is all of
this just hypothetical?

Because we shouldn't go changing default behaviour to solve imaginary
problems ...

> i'm also a bit skeptical of the idea that we're breaking makefiles.

... and adding this to a Makefile really does change the behaviour
significantly on at least one popular make implementation (GNU make).

I imagine a lot of people who are unfamiliar with traditional UNIX
implementations would be very surprised to see gmake suddenly start
running their rules using /bin/sh -e, which is probably the most
obvious effect that the .POSIX special target has on this particular
implementation.

And it doesn't even seem to work to get GNU make to follow the POSIX
behaviour, as POSIX says -e is NOT used whenever errors are being
suppressed (that is, when running commands that start with a -, or
if the .IGNORE: special target is present in the makefile, or if
make is run with the -i option).  GNU make appears to just always
run the shell with -e if you include the .POSIX special target.

Even then, portable makefiles can't rely on the shell being run
with -e even normally, as NetBSD make does not do this.

I tried several other implementations and they follow the POSIX
behaviour by default, adding -e only when errors are not suppressed.

> if an implementation is POSIX compliant, then it already behaves as
> POSIX defines make.  as it stands now, we, and our users, are
> generating files that target an undefined standard that no one agrees
> on, cannot be tested or asserted, and maybe they happen to work on the
> developer's desktop, but not on random users of their own.  by
> defining .POSIX, we turn the state from "usually works, but sometimes
> fails" to "either works or fails, but it's the same everywhere".

Unfortunately, reality is messy.  GNU's behaviour means that adding
.POSIX: actually means that with this specific behaviour (whether or
not the shell is run with "-e"), you now have to consider three
possibilities instead of just two.

Consider these two Makefiles, identical except for the presence of
the .POSIX special target:

  % cat >noposix.mk <<'EOF'
  craziness:
@-false; echo hello
@false; echo hello
EOF

  % cat >posix.mk <<'EOF'
  .POSIX:
  craziness:
@-false; echo hello
@false; echo hello
EOF

GNU:

  % gmake -f noposix.mk
  hello
  hello

  % gmake -f posix.mk
  gmake: [posix.mk:3: craziness] Error 1 (ignored)
  gmake: *** [posix.mk:4: craziness] Error 1

NetBSD:

  % make -f noposix.mk
  hello
  hello

  % make -f posix.mk
  hello
  hello

HP-UX 11:

  (this is the actual POSIX-specified behaviour which I also observed on
  multiple other systems):

  % make -f noposix.mk
  hello
  *** Error exit code 1

  Stop.

  % make -f posix.mk
  hello
  *** Error exit code 1

Cheers,
  Nick

bug#55025: Automake should allow one to enable POSIX make behavior

[Mike Frysinger]

On 14 Jan 2023 01:30, Nick Bowler wrote:
> On 2023-01-13, Mike Frysinger  wrote:
> > On 13 Jan 2023 16:01, Karl Berry wrote:
> >> I am doubtful about blithely defining .POSIX unconditionally. I feel
> >> sure that will break existing Makefiles. I don't think we should do that.
> >>
> >> Detecting .POSIX in an existing Makefile.am and moving it to the front
> >> sounds desirable, since that is clearly what the developer intended.
> >>
> >> Another (not mutually exclusive) possibility is to add an Automake
> >> option (say, "make-posix") that outputs the .POSIX line. --thanks, karl.
> >
> > i'd be amenable to a `no-make-posix` option, but imo forcing people to
> > opt-in to the right behavior is the wrong mental model.
> 
> Does adding .POSIX: to a Makefile actually solve any real-world
> portability problem with Automake generated makefiles, or is all of
> this just hypothetical?
> 
> Because we shouldn't go changing default behaviour to solve imaginary
> problems ...
> 
> > i'm also a bit skeptical of the idea that we're breaking makefiles.
> 
> ... and adding this to a Makefile really does change the behaviour
> significantly on at least one popular make implementation (GNU make).
> 
> I imagine a lot of people who are unfamiliar with traditional UNIX
> implementations would be very surprised to see gmake suddenly start
> running their rules using /bin/sh -e, which is probably the most
> obvious effect that the .POSIX special target has on this particular
> implementation.
> 
> And it doesn't even seem to work to get GNU make to follow the POSIX
> behaviour, as POSIX says -e is NOT used whenever errors are being
> suppressed (that is, when running commands that start with a -, or
> if the .IGNORE: special target is present in the makefile, or if
> make is run with the -i option).  GNU make appears to just always
> run the shell with -e if you include the .POSIX special target.

if this is the case, then it sounds like a bug.  at a glance, i don't see a
report in the upstream tracker, so can you throw one up there if you have a
test case that shows the problem ?

> Even then, portable makefiles can't rely on the shell being run
> with -e even normally, as NetBSD make does not do this.
> 
> I tried several other implementations and they follow the POSIX
> behaviour by default, adding -e only when errors are not suppressed.

this is exactly my point.  if i'm developing a project with automake and i'm
using gnu make, i can easily produce bad code that is not portable.  but the
insidious part is that it doesn't fail for me, it fails for my users who are
not using gnu make, but using a POSIX-compliant implementation that respects
the `set -e` behavior.

the reason i use automake in my projects in the first place is to produce a
build env that is reasonably portable, and if i have to write custom hooks,
to help steer me away from non-POSIX (i.e. things likely to not be portable)
constructs.
-mike


signature.asc
Description: PGP signature

bug#55025: Automake should allow one to enable POSIX make behavior

[Nick Bowler]

On 14/01/2023, Mike Frysinger  wrote:
[...]
>> I tried several other implementations and they follow the POSIX
>> behaviour by default, adding -e only when errors are not suppressed.
>
> this is exactly my point.  if i'm developing a project with automake
> and i'm using gnu make, i can easily produce bad code that is not
> portable.  but the insidious part is that it doesn't fail for me, it
> fails for my users who are not using gnu make, but using a
> POSIX-compliant implementation that respects the `set -e` behavior.

But as I demonstrated, adding .POSIX doesn't actually make GNU make
work the same as these "POSIX-by-default" implementations.  It just
gives it a different, GNU-specific behaviour, since GNU make apparently
enables -e regardless of whether or not errors are suppressed.

Does changing the behaviour in this way really improve anything for a
maintainer that is only testing with GNU make?

Without putting .POSIX in the Makefile, GNU is at least consistent with
NetBSD here (which never uses -e) and probably also other modern BSD
derivatives, as I think many have these days adopted NetBSD make.

Cheers,
  Nick

bug#55025: Automake should allow one to enable POSIX make behavior

[Karl Berry]

Mike, if you want to make an option that lets people have .POSIX if they
want it (and/or recognize .POSIX in the Makefile.am), fine. Please don't
foist it on the rest of us. As Nick has shown, I feel pretty sure the
change would break a lot of existing projects, which IMHO is something
to avoid at all costs. The benefit here does not seem nearly worth
it to me. --thanks, karl.

bug#55025: Automake should allow one to enable POSIX make behavior

[Karl Berry]

P.S. I note that GNU make just made an alpha release. If anyone wants to
report the purported .POSIX bug, might be a good time.

bug#55025: Automake should allow one to enable POSIX make behavior

[Vincent Lefevre]

On 2023-01-14 01:30:24 -0500, Nick Bowler wrote:
> Does adding .POSIX: to a Makefile actually solve any real-world
> portability problem with Automake generated makefiles, or is all of
> this just hypothetical?

For MPFR, when I initially added it, this was just in case.

The real issue at that time was that I was using $< in a case
where it was unspecified by POSIX, and this was failing on FreeBSD.
Unfortunately there doesn't seem to be a way to get a failure
on non-portable features (which would ease maintenance). Though
I test the tarball on various systems before a release candidate,
this is not the case of the development branch (and the Makefiles
would typically be more affected than the C code, on which various
portability checks are done under Linux).

Well, I use autoreconf with --warnings=all,error (passed to automake),
but I didn't get any portability error.

Later I noted:

# The ".POSIX" line is needed in particular for GNU "make", so that
# recipes are invoked as if the shell had been passed the -e flag.

as this is really what we want, both for consistency with other
"make" implementations and to be able to detect errors.

-- 
Vincent Lefèvre  - Web: 
100% accessible validated (X)HTML - Blog: 
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

bug#55025: Automake should allow one to enable POSIX make behavior

[Vincent Lefevre]

On 2023-01-14 15:12:00 -0700, Karl Berry wrote:
> P.S. I note that GNU make just made an alpha release. If anyone
> wants to report the purported .POSIX bug, might be a good time.

I've just reported it:

  https://savannah.gnu.org/bugs/index.php?63667

-- 
Vincent Lefèvre  - Web: 
100% accessible validated (X)HTML - Blog: 
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

bug#55025: Automake should allow one to enable POSIX make behavior

[Vincent Lefevre]

On 2023-01-14 01:30:24 -0500, Nick Bowler wrote:
> Consider these two Makefiles, identical except for the presence of
> the .POSIX special target:
> 
>   % cat >noposix.mk <<'EOF'
>   craziness:
>   @-false; echo hello
>   @false; echo hello
> EOF
> 
>   % cat >posix.mk <<'EOF'
>   .POSIX:
>   craziness:
>   @-false; echo hello
>   @false; echo hello
> EOF
[...]
> HP-UX 11:
> 
>   (this is the actual POSIX-specified behaviour which I also observed on
>   multiple other systems):
> 
>   % make -f noposix.mk
>   hello
>   *** Error exit code 1
> 
>   Stop.
> 
>   % make -f posix.mk
>   hello
>   *** Error exit code 1

Same behavior on AIX, i.e. the .POSIX special target is useless for
*this* example. However, if it is not the first non-comment line, I
get a warning:

  warning: .POSIX directive is not first non-comment line.

(found during my test of GNU MPFR on cfarm119.cfarm.net).

Note that its "make" man page says:

  .POSIX
  Causes the make command to use a different default rules file. The
  file, /usr/ccs/lib/posix.mk, provides the default rules as
  specified in the POSIX standard.

but I don't know whether this changes the behavior with Automake.

If /usr/ccs/lib/posix.mk is included when the .POSIX directive is read,
the fact that it is not the first non-comment line might cause issues.
For instance, this file (re)defines some variables:

MAKE=make
AR=ar
ARFLAGS=-rv
YACC=yacc
YFLAGS=
LEX=lex
LFLAGS=
LDFLAGS=
CC=c89
CFLAGS=-O
FC=fort77
FFLAGS=-O 1
GET=get
GFLAGS=

So this is expected to be found early in the makefile.

-- 
Vincent Lefèvre  - Web: 
100% accessible validated (X)HTML - Blog: 
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

bug#55025: Automake should allow one to enable POSIX make behavior

[Karl Berry]

Sorry, I don't understand what you want Automake to do.

Right now, as far as I can tell, Automake does nothing with .POSIX. It's
not mentioned in the manual nor, as far as I can grep, the code. Maybe
that's the issue, and you want a leading .POSIX in your Makefile.am to
be specially copied at the beginning of the output Makefile.in?
(Or an option, or something.)

But ... have you tried to run a Makefile originating from Automake with
.POSIX in effect?  I feel doubtful that it will work, although I don't
know it for a fact.  -k

bug#55025: Automake should allow one to enable POSIX make behavior

[Vincent Lefevre]

On 2023-10-02 15:17:13 -0600, Karl Berry wrote:
> Sorry, I don't understand what you want Automake to do.
> 
> Right now, as far as I can tell, Automake does nothing with .POSIX. It's
> not mentioned in the manual nor, as far as I can grep, the code. Maybe
> that's the issue, and you want a leading .POSIX in your Makefile.am to
> be specially copied at the beginning of the output Makefile.in?
> (Or an option, or something.)

Yes, Automake should either detect a ".POSIX:" in the Makefile.am
file (at least if it is the first non-comment line) and place it at
the beginning of Makefile.in, or provide an option to be used with
AM_INIT_AUTOMAKE (or AUTOMAKE_OPTIONS) in order to have a ".POSIX:"
at the beginning of Makefile.in.

> But ... have you tried to run a Makefile originating from Automake with
> .POSIX in effect?  I feel doubtful that it will work, although I don't
> know it for a fact.  -k

The Makefile.am files of GNU MPFR have a ".POSIX:" target as the
first non-comment line: the shell for recipes is expected to be
run with the -e flag. So, the ".POSIX:" is necessary for GNU Make
in order to detect errors[*] (fortunately, GNU Make does not seem
to care about its position in Makefile).

[*] It is not needed for a successful build, but its absence might
yield successful builds that are actually broken.

-- 
Vincent Lefèvre  - Web: 
100% accessible validated (X)HTML - Blog: 
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

bug#55025: Automake should allow one to enable POSIX make behavior

[Karl Berry]

Yes, Automake should either detect a ".POSIX:" in the Makefile.am
file (at least if it is the first non-comment line) and place it at
the beginning of Makefile.in, or provide an option to be used with
AM_INIT_AUTOMAKE (or AUTOMAKE_OPTIONS) in order to have a ".POSIX:"
at the beginning of Makefile.in.

Ok. I welcome patches for either or both. It's not something I'll be
implementing myself any time soon (regrettably).

The Makefile.am files of GNU MPFR have a ".POSIX:" target 

Glad to know it works in practice for you. I'm just surprised that
nothing in Automake's make output breaks under POSIX. --thanks, karl.

bug#55025: Automake should allow one to enable POSIX make behavior

[Vincent Lefevre]

On 2023-10-03 15:08:42 -0600, Karl Berry wrote:
> Yes, Automake should either detect a ".POSIX:" in the Makefile.am
> file (at least if it is the first non-comment line) and place it at
> the beginning of Makefile.in, or provide an option to be used with
> AM_INIT_AUTOMAKE (or AUTOMAKE_OPTIONS) in order to have a ".POSIX:"
> at the beginning of Makefile.in.
> 
> Ok. I welcome patches for either or both. It's not something I'll be
> implementing myself any time soon (regrettably).

I think that a "posix" option is cleaner and simpler to implement.
I have some idea of a patch. I'll try to look at it more closely
late tonight.

> The Makefile.am files of GNU MPFR have a ".POSIX:" target 
> 
> Glad to know it works in practice for you. I'm just surprised that
> nothing in Automake's make output breaks under POSIX. --thanks, karl.

On platforms with a POSIX "make", the generated makefile needs to
work as expected. This limits the risk of breakage when using a
".POSIX:" target.

-- 
Vincent Lefèvre  - Web: 
100% accessible validated (X)HTML - Blog: 
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)