maintainer-makefile: catch uses of $< in non-implicit rules
Bruno made a bug report some time ago: make check failed for Bison on Solaris.
The problem was the use of $< in some non-implicit rules. This syntax-check
caught them.
See
http://lists.gnu.org/archive/html/bug-bison/2019-05/msg9.html
and
https://lists.gnu.org/archive/html/bison-patches/2019-05/msg00017.html
Ok to install?
commit e8a5869f2936459762cc48ab370c35b69f86d5e0
Author: Akim Demaille
Date: Sat May 18 08:46:00 2019 +0200
maintainer-makefile: catch uses of $< in non-implicit rules
* top/maint.mk (sc_prohibit_magic_number_exit): New.
diff --git a/ChangeLog b/ChangeLog
index 1918041f0..8d5a4ca6e 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,8 @@
+2019-05-18 Akim Demaille
+
+ maintainer-makefile: catch uses of $< in non-implicit rules
+ * top/maint.mk (sc_prohibit_magic_number_exit): New.
+
2019-05-14 Paul Eggert
close-stream, closein, closeout: simplify
diff --git a/top/maint.mk b/top/maint.mk
index e9d5ee7d4..05ee7661c 100644
--- a/top/maint.mk
+++ b/top/maint.mk
@@ -408,6 +408,41 @@ sc_prohibit_magic_number_exit:
halt='use EXIT_* values rather than magic number' \
$(_sc_search_regexp)
+# Check that we don't use $< in non-implicit Makefile rules.
+#
+# To find the Makefiles, trace AC_CONFIG_FILES. Using VC_LIST would
+# miss the Makefiles that are not under VC control (e.g., symlinks
+# installed for gettext). "Parsing" (recursive) uses of SUBDIRS seems
+# too delicate.
+#
+# Use GNU Make's --print-data-base to normalize the rules into some
+# easy to parse format: they are separated by two \n. Look for the
+# "section" about non-pattern rules (marked with "# Files") inside
+# which there are still the POSIX Make like implicit rules (".c.o").
+sc_prohibit_gnu_make_extensions_awk_ = \
+ BEGIN { \
+ RS = "\n\n"; \
+ in_rules = 0;\
+ }; \
+ /^\# Files/ {
\
+ in_rules = 1;\
+ }; \
+ /\$$/dev/null | \
+ awk -v file=$$m -e '$($@_awk_)' || exit 1;\
+ done
+
# Using EXIT_SUCCESS as the first argument to error is misleading,
# since when that parameter is 0, error does not exit. Use '0' instead.
sc_error_exit_success:
Re: maintainer-makefile: catch uses of $< in non-implicit rules
Akim Demaille wrote: + }; \ Thanks, that looks good except I don't see why some of the lines like the above have semicolons while others don't.
Re: maintainer-makefile: catch uses of $< in non-implicit rules
Hi Paul!
> Le 18 mai 2019 à 09:52, Paul Eggert a écrit :
>
> Akim Demaille wrote:
>> + };
>> \
>
> Thanks, that looks good except I don't see why some of the lines like the
> above have semicolons while others don't.
You're right, more consistency wouldn't hurt. Because everything ends in a
single line, some of them are needed (separators of statements of compound
statements), others aren't.
How about this?
commit 2e801e81bb362429d0e252d076233bfdac20e367
Author: Akim Demaille
Date: Sat May 18 08:46:00 2019 +0200
maintainer-makefile: catch uses of $< in non-implicit rules
* top/maint.mk (sc_prohibit_magic_number_exit): New.
diff --git a/ChangeLog b/ChangeLog
index 1918041f0..8d5a4ca6e 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,8 @@
+2019-05-18 Akim Demaille
+
+ maintainer-makefile: catch uses of $< in non-implicit rules
+ * top/maint.mk (sc_prohibit_magic_number_exit): New.
+
2019-05-14 Paul Eggert
close-stream, closein, closeout: simplify
diff --git a/top/maint.mk b/top/maint.mk
index e9d5ee7d4..3dbe9c378 100644
--- a/top/maint.mk
+++ b/top/maint.mk
@@ -408,6 +408,41 @@ sc_prohibit_magic_number_exit:
halt='use EXIT_* values rather than magic number' \
$(_sc_search_regexp)
+# Check that we don't use $< in non-implicit Makefile rules.
+#
+# To find the Makefiles, trace AC_CONFIG_FILES. Using VC_LIST would
+# miss the Makefiles that are not under VC control (e.g., symlinks
+# installed for gettext). "Parsing" (recursive) uses of SUBDIRS seems
+# too delicate.
+#
+# Use GNU Make's --print-data-base to normalize the rules into some
+# easy to parse format: they are separated by two \n. Look for the
+# "section" about non-pattern rules (marked with "# Files") inside
+# which there are still the POSIX Make like implicit rules (".c.o").
+sc_prohibit_gnu_make_extensions_awk_ = \
+ BEGIN { \
+ RS = "\n\n"; \
+ in_rules = 0;\
+ }\
+ /^\# Files/ {
\
+ in_rules = 1;\
+ }\
+ /\$$/dev/null | \
+ awk -v file=$$m -e '$($@_awk_)' || exit 1;\
+ done
+
# Using EXIT_SUCCESS as the first argument to error is misleading,
# since when that parameter is 0, error does not exit. Use '0' instead.
sc_error_exit_success:
Re: maintainer-makefile: catch uses of $< in non-implicit rules
Akim Demaille wrote: How about this? Looks OK to me; thanks.
Re: maintainer-makefile: catch uses of $< in non-implicit rules
> Le 18 mai 2019 à 20:21, Paul Eggert a écrit : > > Akim Demaille wrote: > >> How about this? > > Looks OK to me; thanks. Installed. Thanks!
Re: maintainer-makefile: catch uses of $< in non-implicit rules
Hi Tim, > Le 17 juin 2019 à 11:57, Tim Rühsen a écrit : > > Hi Akim, > > The patch uses awk -e which is understood only by GNU awk. This breaks > all Debian CI tests here since Debian installs 'mawk' by default (I > wasn't aware of that before). > > It's not a big deal to install package 'gawk' everywhere, but I just > wanted to mention it. I can see no warning/hint in any of the patch's > comments. That was not my intention. I expect that maintainers have gawk installed, so I think this check should be skipped if awk is not gawk. WDYT?
Re: maintainer-makefile: catch uses of $< in non-implicit rules
Hi Akim, On 5/19/19 7:42 AM, Akim Demaille wrote: > >> Le 18 mai 2019 à 20:21, Paul Eggert a écrit : >> >> Akim Demaille wrote: >> >>> How about this? >> >> Looks OK to me; thanks. > > Installed. Thanks! > The patch uses awk -e which is understood only by GNU awk. This breaks all Debian CI tests here since Debian installs 'mawk' by default (I wasn't aware of that before). It's not a big deal to install package 'gawk' everywhere, but I just wanted to mention it. I can see no warning/hint in any of the patch's comments. Hmmm, this also implies documenting a new dependency in all projects that use gnulib or disabling sc_prohibit_gnu_make_extensions. Regards, Tim signature.asc Description: OpenPGP digital signature
Re: maintainer-makefile: catch uses of $< in non-implicit rules
Hi Akim, On 6/17/19 11:59 AM, Akim Demaille wrote: > Hi Tim, > >> Le 17 juin 2019 à 11:57, Tim Rühsen a écrit : >> >> Hi Akim, >> >> The patch uses awk -e which is understood only by GNU awk. This breaks >> all Debian CI tests here since Debian installs 'mawk' by default (I >> wasn't aware of that before). >> >> It's not a big deal to install package 'gawk' everywhere, but I just >> wanted to mention it. I can see no warning/hint in any of the patch's >> comments. > > That was not my intention. > > I expect that maintainers have gawk installed, so I think this check > should be skipped if awk is not gawk. WDYT? Thanks, sounds reasonable to me. Regards, Tim signature.asc Description: OpenPGP digital signature
Re: maintainer-makefile: catch uses of $< in non-implicit rules
Hi Tim,
> Le 17 juin 2019 à 12:04, Tim Rühsen a écrit :
>
> Hi Akim,
>
> On 6/17/19 11:59 AM, Akim Demaille wrote:
>> Hi Tim,
>>
>>> Le 17 juin 2019 à 11:57, Tim Rühsen a écrit :
>>>
>>> Hi Akim,
>>>
>>> The patch uses awk -e which is understood only by GNU awk. This breaks
>>> all Debian CI tests here since Debian installs 'mawk' by default (I
>>> wasn't aware of that before).
>>>
>>> It's not a big deal to install package 'gawk' everywhere, but I just
>>> wanted to mention it. I can see no warning/hint in any of the patch's
>>> comments.
>>
>> That was not my intention.
>>
>> I expect that maintainers have gawk installed, so I think this check
>> should be skipped if awk is not gawk. WDYT?
>
> Thanks, sounds reasonable to me.
Here is my proposal. Worked properly with the non GNU awk sitting on my
machine.
commit b70c97f3ecf45a5c98b7f32189c1a4ae72a60788
Author: Akim Demaille
Date: Sat Jun 22 17:52:16 2019 +0200
maintainer-makefile: restore portability to non-GNU awks
Reported by Tim Rühsen.
* top/maint.mk (AWK): New variable. Use it.
(sc_prohibit_gnu_make_extensions): Skip if $(AWK) is not gawk.
diff --git a/ChangeLog b/ChangeLog
index cdc2b3d7a..dc7e52c0a 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,10 @@
+2019-06-22 Akim Demaille
+
+ maintainer-makefile: restore portability to non-GNU awks
+ Reported by Tim Rühsen.
+ * top/maint.mk (AWK): New variable. Use it.
+ (sc_prohibit_gnu_make_extensions): Skip if $(AWK) is not gawk.
+
2019-06-22 Akim Demaille
argmatch: put all the docs member last.
diff --git a/top/maint.mk b/top/maint.mk
index 3dbe9c378..16e936022 100644
--- a/top/maint.mk
+++ b/top/maint.mk
@@ -24,6 +24,7 @@ ME := maint.mk
# These variables ought to be defined through the configure.ac section
# of the module description. But some packages import this file directly,
# ignoring the module description.
+AWK ?= awk
GREP ?= grep
SED ?= sed
@@ -190,7 +191,7 @@ $(sc_z_rules_): %.z: %
@end=$$(date +%s.%N); \
start=$$(cat .sc-start-$*); \
rm -f .sc-start-$*; \
- awk -v s=$$start -v e=$$end \
+ $(AWK) -v s=$$start -v e=$$end \
'END {printf "%.2f $(patsubst sc_%,%,$*)\n", e - s}' < /dev/null
# The patsubst here is to replace each sc_% rule with its sc_%.z wrapper
@@ -435,13 +436,15 @@ sc_prohibit_gnu_make_extensions_awk_ =
\
exit status; \
}
sc_prohibit_gnu_make_extensions:
- (cd $(srcdir) && autoconf --trace AC_CONFIG_FILES:'$$1') | \
- tr ' ' '\n' | \
- $(SED) -ne '/Makefile/{s/\.in$$//;p;}' | \
- while read m; do \
- $(MAKE) -qp -f $$m .DUMMY-TARGET 2>/dev/null | \
- awk -v file=$$m -e '$($@_awk_)' || exit 1;\
- done
+ @if $(AWK) --version | grep GNU >/dev/null 2>&1; then \
+ (cd $(srcdir) && autoconf --trace AC_CONFIG_FILES:'$$1') |\
+ tr ' ' '\n' | \
+ $(SED) -ne '/Makefile/{s/\.in$$//;p;}' |\
+ while read m; do\
+ $(MAKE) -qp -f $$m .DUMMY-TARGET 2>/dev/null |\
+ $(AWK) -v file=$$m -e '$($@_awk_)' || exit 1; \
+ done; \
+ fi
# Using EXIT_SUCCESS as the first argument to error is misleading,
# since when that parameter is 0, error does not exit. Use '0' instead.
@@ -1383,7 +1386,7 @@ gpg_key_ID ?=
\
$$(cd $(srcdir) \
&& git cat-file tag v$(VERSION) \
| $(gpgv) --status-fd 1 --keyring /dev/null - - 2>/dev/null\
-| awk '/^\[GNUPG:\] ERRSIG / {print $$3; exit}')
+| $(AWK) '/^\[GNUPG:\] ERRSIG / {print $$3; exit}')
translation_project_ ?= coordina...@translationproject.org
Re: maintainer-makefile: catch uses of $< in non-implicit rules
Hi Akim,
On 6/22/19 5:53 PM, Akim Demaille wrote:
> Hi Tim,
>
>> Le 17 juin 2019 à 12:04, Tim Rühsen a écrit :
>>
>> Hi Akim,
>>
>> On 6/17/19 11:59 AM, Akim Demaille wrote:
>>> Hi Tim,
>>>
Le 17 juin 2019 à 11:57, Tim Rühsen a écrit :
Hi Akim,
The patch uses awk -e which is understood only by GNU awk. This breaks
all Debian CI tests here since Debian installs 'mawk' by default (I
wasn't aware of that before).
It's not a big deal to install package 'gawk' everywhere, but I just
wanted to mention it. I can see no warning/hint in any of the patch's
comments.
>>>
>>> That was not my intention.
>>>
>>> I expect that maintainers have gawk installed, so I think this check
>>> should be skipped if awk is not gawk. WDYT?
>>
>> Thanks, sounds reasonable to me.
>
> Here is my proposal. Worked properly with the non GNU awk sitting on my
> machine.
Your proposal works here on Debian after I switched to mawk via
'update-alternatives --config awk', selecting mawk. It detects two
occurrences of $< in our (project) Makefiles correctly.
Switching back to the old maint.mk, brings back the error message about
'-e'. I just did this to make sure.
Thanks for working on it !
Regards, Tim
>
> commit b70c97f3ecf45a5c98b7f32189c1a4ae72a60788
> Author: Akim Demaille
> Date: Sat Jun 22 17:52:16 2019 +0200
>
> maintainer-makefile: restore portability to non-GNU awks
>
> Reported by Tim Rühsen.
> * top/maint.mk (AWK): New variable. Use it.
> (sc_prohibit_gnu_make_extensions): Skip if $(AWK) is not gawk.
>
> diff --git a/ChangeLog b/ChangeLog
> index cdc2b3d7a..dc7e52c0a 100644
> --- a/ChangeLog
> +++ b/ChangeLog
> @@ -1,3 +1,10 @@
> +2019-06-22 Akim Demaille
> +
> + maintainer-makefile: restore portability to non-GNU awks
> + Reported by Tim Rühsen.
> + * top/maint.mk (AWK): New variable. Use it.
> + (sc_prohibit_gnu_make_extensions): Skip if $(AWK) is not gawk.
> +
> 2019-06-22 Akim Demaille
>
> argmatch: put all the docs member last.
> diff --git a/top/maint.mk b/top/maint.mk
> index 3dbe9c378..16e936022 100644
> --- a/top/maint.mk
> +++ b/top/maint.mk
> @@ -24,6 +24,7 @@ ME := maint.mk
> # These variables ought to be defined through the configure.ac section
> # of the module description. But some packages import this file directly,
> # ignoring the module description.
> +AWK ?= awk
> GREP ?= grep
> SED ?= sed
>
> @@ -190,7 +191,7 @@ $(sc_z_rules_): %.z: %
> @end=$$(date +%s.%N); \
> start=$$(cat .sc-start-$*); \
> rm -f .sc-start-$*; \
> - awk -v s=$$start -v e=$$end \
> + $(AWK) -v s=$$start -v e=$$end \
> 'END {printf "%.2f $(patsubst sc_%,%,$*)\n", e - s}' < /dev/null
>
> # The patsubst here is to replace each sc_% rule with its sc_%.z wrapper
> @@ -435,13 +436,15 @@ sc_prohibit_gnu_make_extensions_awk_ =
> \
> exit status;\
>}
> sc_prohibit_gnu_make_extensions:
> - (cd $(srcdir) && autoconf --trace AC_CONFIG_FILES:'$$1') | \
> - tr ' ' '\n' | \
> - $(SED) -ne '/Makefile/{s/\.in$$//;p;}' | \
> - while read m; do \
> - $(MAKE) -qp -f $$m .DUMMY-TARGET 2>/dev/null | \
> - awk -v file=$$m -e '$($@_awk_)' || exit 1;\
> - done
> + @if $(AWK) --version | grep GNU >/dev/null 2>&1; then \
> + (cd $(srcdir) && autoconf --trace AC_CONFIG_FILES:'$$1') |\
> + tr ' ' '\n' | \
> + $(SED) -ne '/Makefile/{s/\.in$$//;p;}' |\
> + while read m; do\
> + $(MAKE) -qp -f $$m .DUMMY-TARGET 2>/dev/null |\
> + $(AWK) -v file=$$m -e '$($@_awk_)' || exit 1; \
> + done; \
> + fi
>
> # Using EXIT_SUCCESS as the first argument to error is misleading,
> # since when that parameter is 0, error does not exit. Use '0' instead.
> @@ -1383,7 +1386,7 @@ gpg_key_ID ?=
> \
>$$(cd $(srcdir)\
> && git cat-file tag v$(VERSION) \
> | $(gpgv) --status-fd 1 --keyring /dev/null - - 2>/dev/null \
> -| awk '/^\[GNUPG:\] ERRSIG / {print $$3; exit}')
> +| $(AWK) '/^\[GNUPG:\] ERRSIG / {print $$3; exit}')
>
> translation_project_ ?= coordina...@translationproject.org
>
>
>
signature.asc
Re: maintainer-makefile: catch uses of $< in non-implicit rules
Hi Tim, > Le 24 juin 2019 à 11:42, Tim Rühsen a écrit : > > Hi Akim, > > Your proposal works here on Debian after I switched to mawk via > 'update-alternatives --config awk', selecting mawk. It detects two > occurrences of $< in our (project) Makefiles correctly. > > Switching back to the old maint.mk, brings back the error message about > '-e'. I just did this to make sure. Perfect, thanks. Installed.
Re: maintainer-makefile: catch uses of $< in non-implicit rules
Hi Akim, On 6/24/19 5:56 PM, Akim Demaille wrote: > Hi Tim, > >> Le 24 juin 2019 à 11:42, Tim Rühsen a écrit : >> >> Hi Akim, >> >> Your proposal works here on Debian after I switched to mawk via >> 'update-alternatives --config awk', selecting mawk. It detects two >> occurrences of $< in our (project) Makefiles correctly. >> >> Switching back to the old maint.mk, brings back the error message about >> '-e'. I just did this to make sure. > > Perfect, thanks. Installed. I updated gnulib to your commit (47405621b3066c035b4b98d2db934d550aaed1ad) and - sorry to say - two CI runners broke. That are Arch Linux and Fedora 30. Output on Fedora 30 from 'make syntax-check': prohibit_gnu_make_extensions Error: lib/Makefile: $< in a non implicit rule # makefile (from 'lib/Makefile', line 1310) NEXT_STRING_H = # makefile (from 'lib/Makefile', line 1409) REPLACE_LOCALECONV = 0 # makefile (from 'lib/Makefile', line 1231) LTLIBICONV = # makefile (from 'lib/Makefile', line 476) ETAGS = etags # environment halt = # makefile (from 'lib/Makefile', line 1450) REPLACE_PTSNAME = 0 # automatic ?F = $(notdir $?) # makefile (from 'lib/Makefile', line 784) GNULIB_STRTOD = 0 # makefile (from 'lib/Makefile', line 1410) REPLACE_LOCALTIME = 0 ... (many more lines like these) ... MAKE_VERSION := 4.2.1 # makefile (from 'lib/Makefile', line 540) GNULIB_ATOLL = 0 # variable set hash-table stats: # Load=1368/2048=67%, Rehash=1, Collisions=6067/4772=127% Error: lib/Makefile: $< in a non implicit rule %.lo: %.c # recipe to execute (from 'lib/Makefile', line 2150): $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.lo$$||'`;\ $(LTCOMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ $< &&\ $(am__mv) $$depbase.Tpo $$depbase.Plo Error: lib/Makefile: $< in a non implicit rule %.o: %.c # recipe to execute (from 'lib/Makefile', line 2134): $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.o$$||'`;\ $(COMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ $< &&\ $(am__mv) $$depbase.Tpo $$depbase.Po Error: lib/Makefile: $< in a non implicit rule %.obj: %.c # recipe to execute (from 'lib/Makefile', line 2142): $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.obj$$||'`;\ $(COMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ `$(CYGPATH_W) '$<'` &&\ $(am__mv) $$depbase.Tpo $$depbase.Po Error: lib/Makefile: $< in a non implicit rule (%): % # recipe to execute (built-in): $(AR) $(ARFLAGS) $@ $< Error: lib/Makefile: $< in a non implicit rule %.out: % # recipe to execute (built-in): @rm -f $@ cp $< $@ Error: lib/Makefile: $< in a non implicit rule %:: s.% # recipe to execute (built-in): $(GET) $(GFLAGS) $(SCCS_OUTPUT_OPTION) $< Error: lib/Makefile: $< in a non implicit rule %:: SCCS/s.% # recipe to execute (built-in): $(GET) $(GFLAGS) $(SCCS_OUTPUT_OPTION) $< make: *** [maint.mk:439: sc_prohibit_gnu_make_extensions] Error 1 [root@61f4306ca485 wget2]# cd gnulib [root@61f4306ca485 gnulib]# git branch * (HEAD detached at 47405621b) master [root@61f4306ca485 gnulib]# git log commit 47405621b3066c035b4b98d2db934d550aaed1ad (HEAD, origin/master, origin/HEAD, master) Author: Akim Demaille Date: Tue Jun 25 08:11:34 2019 +0200 [root@61f4306ca485 gnulib]# awk --version GNU Awk 4.2.1, API: 2.0 (GNU MPFR 3.1.6-p2, GNU MP 6.1.2) Regards, Tim signature.asc Description: OpenPGP digital signature
Re: maintainer-makefile: catch uses of $< in non-implicit rules
Hi Akim,
the command expands to
if gawk --version | grep GNU >/dev/null 2>&1; then \
(cd . && autoconf --trace AC_CONFIG_FILES:'$1') | \
tr ' ' '\n' | \
/usr/bin/sed -ne '/Makefile/{s/\.in$//;p;}' | \
while read m; do\
make -qp -f $m .DUMMY-TARGET 2>/dev/null |\
gawk -v file=$m -e 'BEGIN { RS = "\n\n"; in_rules = 0; } /^#
Files/ { in_rules = 1; } /\$
signature.asc
Description: OpenPGP digital signature
Re: maintainer-makefile: catch uses of $< in non-implicit rules
Hi Tim,
> the command expands to
>
> if gawk --version | grep GNU >/dev/null 2>&1; then \
> (cd . && autoconf --trace AC_CONFIG_FILES:'$1') | \
> tr ' ' '\n' | \
> /usr/bin/sed -ne '/Makefile/{s/\.in$//;p;}' | \
> while read m; do\
> make -qp -f $m .DUMMY-TARGET 2>/dev/null |\
> gawk -v file=$m -e 'BEGIN { RS = "\n\n"; in_rules = 0; } /^#
> Files/ { in_rules = 1; } /\$ \.\w+(\.\w+)?:/ { print "Error: " file ": $< in a non implicit rule\n"
Does the attached patch fix it?
Bruno
diff --git a/top/maint.mk b/top/maint.mk
index 16e9360..cb52631 100644
--- a/top/maint.mk
+++ b/top/maint.mk
@@ -429,7 +429,7 @@ sc_prohibit_gnu_make_extensions_awk_ = \
in_rules = 1; \
} \
/\$$
Re: maintainer-makefile: catch uses of $< in non-implicit rules
Hi Bruno,
sorry, it doesn't.
On 6/25/19 4:37 PM, Bruno Haible wrote:
> Hi Tim,
>
>> the command expands to
>>
>> if gawk --version | grep GNU >/dev/null 2>&1; then \
>> (cd . && autoconf --trace AC_CONFIG_FILES:'$1') | \
>> tr ' ' '\n' | \
>> /usr/bin/sed -ne '/Makefile/{s/\.in$//;p;}' | \
>> while read m; do\
>> make -qp -f $m .DUMMY-TARGET 2>/dev/null |\
>> gawk -v file=$m -e 'BEGIN { RS = "\n\n"; in_rules = 0; } /^#
>> Files/ { in_rules = 1; } /\$> \.\w+(\.\w+)?:/ { print "Error: " file ": $< in a non implicit rule\n"
>
> Does the attached patch fix it?
>
> Bruno
>
signature.asc
Description: OpenPGP digital signature
Re: maintainer-makefile: catch uses of $< in non-implicit rules
Here is a Dockerfile for reproducing the issue on Fedora 30. docker build ... docker run ... git clone https://gitlab.com/gnuwget/wget2.git cd wget2 git checkout tmp-stuff ./bootstrap ./configure make syntax-check Regards, Tim FROM fedora:latest LABEL maintainer "Tim Rühsen " WORKDIR /usr/local RUN dnf update -y RUN dnf install -y --allowerasing \ git \ make \ gcc \ coreutils \ autoconf \ libtool \ gettext-devel \ automake \ autogen \ python \ valgrind \ libunistring-devel \ flex \ gnutls-devel \ libpsl-devel \ libnghttp2-devel \ zlib-devel \ libidn2-devel \ bzip2-devel \ xz-devel \ libmicrohttpd-devel \ gperf \ lzip \ rsync \ pandoc \ texinfo \ perl-IO-Socket-SSL \ perl-HTTP-Daemon \ gpgme-devel \ ccache \ which RUN git clone https://git.savannah.gnu.org/git/gnulib.git ENV GNULIB_SRCDIR /usr/local/gnulib signature.asc Description: OpenPGP digital signature
Re: maintainer-makefile: catch uses of $< in non-implicit rules
On 6/25/19 3:24 PM, Tim Rühsen wrote:
> Hi Akim,
>
> the command expands to
>
> if gawk --version | grep GNU >/dev/null 2>&1; then \
> (cd . && autoconf --trace AC_CONFIG_FILES:'$1') | \
> tr ' ' '\n' | \
> /usr/bin/sed -ne '/Makefile/{s/\.in$//;p;}' | \
> while read m; do\
> make -qp -f $m .DUMMY-TARGET 2>/dev/null |\
> gawk -v file=$m -e 'BEGIN { RS = "\n\n"; in_rules = 0; } /^#
> Files/ { in_rules = 1; } /\$ \.\w+(\.\w+)?:/ { print "Error: " file ": $< in a non implicit rule\n"
> $0; status = 1; } END { exit status; }' || exit 1;\
> done; \
> fi
>
> # make --version
> GNU Make 4.2.1
> Built for x86_64-redhat-linux-gnu
>
> # gawk --version
> GNU Awk 4.2.1, API: 2.0 (GNU MPFR 3.1.6-p2, GNU MP 6.1.2)
>
> # sed --version
> sed (GNU sed) 4.5
>
> # tr --version
> tr (GNU coreutils) 8.31
>
> # autoconf --version
> autoconf (GNU Autoconf) 2.69
Testing the above directly in bash works.
So after some testing I found a work-around:
LC_ALL=en_US.UTF-8 make syntax-check
The issue is seen on Fedora 30 and Arch Linux.
Arch `locale` output:
LANG=en_US.UTF-8
LC_CTYPE="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_PAPER="en_US.UTF-8"
LC_NAME="en_US.UTF-8"
LC_ADDRESS="en_US.UTF-8"
LC_TELEPHONE="en_US.UTF-8"
LC_MEASUREMENT="en_US.UTF-8"
LC_IDENTIFICATION="en_US.UTF-8"
LC_ALL=
The difference to a 'working' environment is that LC_ALL isn't set.
So the outcome of 'make syntax-check' (concrete:
sc_prohibit_gnu_make_extensions rule) is locale dependent.
Is that wanted behavior ?
Fun fact: Once you run `LC_ALL=en_US.UTF-8 make syntax-check`, all
following 'make syntax-check' succeed while LC_ALL still is empty. This
is on Arch.
I have no idea how to fix this properly.
Regards, Tim
signature.asc
Description: OpenPGP digital signature
Re: maintainer-makefile: catch uses of $< in non-implicit rules
On 7/10/19 11:07 AM, Tim Rühsen wrote:
> On 6/25/19 3:24 PM, Tim Rühsen wrote:
>> Hi Akim,
>>
>> the command expands to
>>
>> if gawk --version | grep GNU >/dev/null 2>&1; then \
>> (cd . && autoconf --trace AC_CONFIG_FILES:'$1') | \
>> tr ' ' '\n' | \
>> /usr/bin/sed -ne '/Makefile/{s/\.in$//;p;}' | \
>> while read m; do\
>> make -qp -f $m .DUMMY-TARGET 2>/dev/null |\
>> gawk -v file=$m -e 'BEGIN { RS = "\n\n"; in_rules = 0; } /^#
>> Files/ { in_rules = 1; } /\$> \.\w+(\.\w+)?:/ { print "Error: " file ": $< in a non implicit rule\n"
>> $0; status = 1; } END { exit status; }' || exit 1;\
>> done; \
>> fi
>>
>> # make --version
>> GNU Make 4.2.1
>> Built for x86_64-redhat-linux-gnu
>>
>> # gawk --version
>> GNU Awk 4.2.1, API: 2.0 (GNU MPFR 3.1.6-p2, GNU MP 6.1.2)
>>
>> # sed --version
>> sed (GNU sed) 4.5
>>
>> # tr --version
>> tr (GNU coreutils) 8.31
>>
>> # autoconf --version
>> autoconf (GNU Autoconf) 2.69
>
> Testing the above directly in bash works.
> So after some testing I found a work-around:
>
> LC_ALL=en_US.UTF-8 make syntax-check
>
> The issue is seen on Fedora 30 and Arch Linux.
>
> Arch `locale` output:
> LANG=en_US.UTF-8
> LC_CTYPE="en_US.UTF-8"
> LC_NUMERIC="en_US.UTF-8"
> LC_TIME="en_US.UTF-8"
> LC_COLLATE="en_US.UTF-8"
> LC_MONETARY="en_US.UTF-8"
> LC_MESSAGES="en_US.UTF-8"
> LC_PAPER="en_US.UTF-8"
> LC_NAME="en_US.UTF-8"
> LC_ADDRESS="en_US.UTF-8"
> LC_TELEPHONE="en_US.UTF-8"
> LC_MEASUREMENT="en_US.UTF-8"
> LC_IDENTIFICATION="en_US.UTF-8"
> LC_ALL=
>
> The difference to a 'working' environment is that LC_ALL isn't set.
>
> So the outcome of 'make syntax-check' (concrete:
> sc_prohibit_gnu_make_extensions rule) is locale dependent.
>
> Is that wanted behavior ?
>
> Fun fact: Once you run `LC_ALL=en_US.UTF-8 make syntax-check`, all
> following 'make syntax-check' succeed while LC_ALL still is empty. This
> is on Arch.
>
> I have no idea how to fix this properly.
Sorry, I have to backpaddle. After more tests with a fresh docker
container it seems not LC_ALL related. Instead after several 'make
syntax-check' those suddenly succeed.
All that changes my project directory is the files in '.deps/'. Removing
that directory, I am back at failing 'make syntax-check' (after several
invocations it then succeeds again).
It looks like the 'make' in the sc_prohibit_gnu_make_extensions rule
creates .Plo and .Po files. One group of files for each SUBDIR/Makefile
per 'make check'. If all files for all SUBDIRs have been created, make
'syntax-check' succeeds.
Knowing this, the issue is reproducible on Debian (unstable) as well.
I report more once tracked down.
Regards, Tim
signature.asc
Description: OpenPGP digital signature
Re: maintainer-makefile: catch uses of $< in non-implicit rules
On 7/10/19 11:38 AM, Tim Rühsen wrote:
>
>
> On 7/10/19 11:07 AM, Tim Rühsen wrote:
>> On 6/25/19 3:24 PM, Tim Rühsen wrote:
>>> Hi Akim,
>>>
>>> the command expands to
>>>
>>> if gawk --version | grep GNU >/dev/null 2>&1; then \
>>> (cd . && autoconf --trace AC_CONFIG_FILES:'$1') | \
>>> tr ' ' '\n' | \
>>> /usr/bin/sed -ne '/Makefile/{s/\.in$//;p;}' | \
>>> while read m; do\
>>> make -qp -f $m .DUMMY-TARGET 2>/dev/null |\
>>> gawk -v file=$m -e 'BEGIN { RS = "\n\n"; in_rules = 0; } /^#
>>> Files/ { in_rules = 1; } /\$>> \.\w+(\.\w+)?:/ { print "Error: " file ": $< in a non implicit rule\n"
>>> $0; status = 1; } END { exit status; }' || exit 1;\
>>> done; \
>>> fi
>>>
>>> # make --version
>>> GNU Make 4.2.1
>>> Built for x86_64-redhat-linux-gnu
>>>
>>> # gawk --version
>>> GNU Awk 4.2.1, API: 2.0 (GNU MPFR 3.1.6-p2, GNU MP 6.1.2)
>>>
>>> # sed --version
>>> sed (GNU sed) 4.5
>>>
>>> # tr --version
>>> tr (GNU coreutils) 8.31
>>>
>>> # autoconf --version
>>> autoconf (GNU Autoconf) 2.69
>>
>> Testing the above directly in bash works.
>> So after some testing I found a work-around:
>>
>> LC_ALL=en_US.UTF-8 make syntax-check
>>
>> The issue is seen on Fedora 30 and Arch Linux.
>>
>> Arch `locale` output:
>> LANG=en_US.UTF-8
>> LC_CTYPE="en_US.UTF-8"
>> LC_NUMERIC="en_US.UTF-8"
>> LC_TIME="en_US.UTF-8"
>> LC_COLLATE="en_US.UTF-8"
>> LC_MONETARY="en_US.UTF-8"
>> LC_MESSAGES="en_US.UTF-8"
>> LC_PAPER="en_US.UTF-8"
>> LC_NAME="en_US.UTF-8"
>> LC_ADDRESS="en_US.UTF-8"
>> LC_TELEPHONE="en_US.UTF-8"
>> LC_MEASUREMENT="en_US.UTF-8"
>> LC_IDENTIFICATION="en_US.UTF-8"
>> LC_ALL=
>>
>> The difference to a 'working' environment is that LC_ALL isn't set.
>>
>> So the outcome of 'make syntax-check' (concrete:
>> sc_prohibit_gnu_make_extensions rule) is locale dependent.
>>
>> Is that wanted behavior ?
>>
>> Fun fact: Once you run `LC_ALL=en_US.UTF-8 make syntax-check`, all
>> following 'make syntax-check' succeed while LC_ALL still is empty. This
>> is on Arch.
>>
>> I have no idea how to fix this properly.
>
> Sorry, I have to backpaddle. After more tests with a fresh docker
> container it seems not LC_ALL related. Instead after several 'make
> syntax-check' those suddenly succeed.
>
> All that changes my project directory is the files in '.deps/'. Removing
> that directory, I am back at failing 'make syntax-check' (after several
> invocations it then succeeds again).
>
> It looks like the 'make' in the sc_prohibit_gnu_make_extensions rule
> creates .Plo and .Po files. One group of files for each SUBDIR/Makefile
> per 'make check'. If all files for all SUBDIRs have been created, make
> 'syntax-check' succeeds.
>
> Knowing this, the issue is reproducible on Debian (unstable) as well.
>
> I report more once tracked down.
Reproducible everywhere (needs gawk being installed, else the
sc_prohibit_gnu_make_extensions is a no-op).
Akim, at least with GNU make 4.2.1 the combination of -q and -p doesn't
do what you expect. From the make man page, I would say that both
options contradict. -q: don't print anything; -p: print the database
For now, I have to disable sc_prohibit_gnu_make_extensions in my projects.
Regards, Tim
signature.asc
Description: OpenPGP digital signature
Re: maintainer-makefile: catch uses of $< in non-implicit rules
Hi Tim, Sorry I dropped the ball... > Reproducible everywhere (needs gawk being installed, else the > sc_prohibit_gnu_make_extensions is a no-op). Which is what I meant. So are you saying it work as (I) expected? > Akim, at least with GNU make 4.2.1 the combination of -q and -p doesn't > do what you expect. From the make man page, I would say that both > options contradict. -q: don't print anything; -p: print the database I'm using 4.2.1, and it does what I meant: -p prints the rules, and -q (which is --question, not --quiet) avoids that we fired a rule (i.e., "make -q" does not run "make all"). So I'm just clueless here. I don't know what to do to address your issue. Are you running "make syntax-check" on a configured builddir?
Re: maintainer-makefile: catch uses of $< in non-implicit rules
Hi Akim, On 7/10/19 9:32 PM, Akim Demaille wrote: > Hi Tim, > > Sorry I dropped the ball... NP, I dropped it as well ;-) >> Reproducible everywhere (needs gawk being installed, else the >> sc_prohibit_gnu_make_extensions is a no-op). > > Which is what I meant. So are you saying it work as (I) expected? It works as expected for mawk which effectively means the test is skipped. >> Akim, at least with GNU make 4.2.1 the combination of -q and -p doesn't >> do what you expect. From the make man page, I would say that both >> options contradict. -q: don't print anything; -p: print the database > > I'm using 4.2.1, and it does what I meant: -p prints the rules, > and -q (which is --question, not --quiet) avoids that we > fired a rule (i.e., "make -q" does not run "make all"). I just summed up the man page and picked up the 'Do not ... print anything': -q, --question ``Question mode''. Do not run any commands, or print anything; just return an exit status that is zero if the specified targets are already up to date, nonzero otherwise. > So I'm just clueless here. I don't know what to do to address > your issue. It looks like this is not *my* issue. The issue shows up on different installations which have gawk and make 4.2.1 in common. I will check if the issue pops up with older make versions or with other projects as well. > Are you running "make syntax-check" on a configured builddir? No sure what you mean. The command sequence to trigger the issue is straight forward: ./bootstrap ./configure make make check make syntax-check <- bang What (C) project + environment are you using successfully ? Regards, Tim signature.asc Description: OpenPGP digital signature
Re: maintainer-makefile: catch uses of $< in non-implicit rules
On 7/10/19 9:32 PM, Akim Demaille wrote: > I'm using 4.2.1, and it does what I meant: -p prints the rules, > and -q (which is --question, not --quiet) avoids that we > fired a rule (i.e., "make -q" does not run "make all"). > > So I'm just clueless here. I don't know what to do to address > your issue. I can *NOT* reproduce the issue on Debian stretch (gawk 4.1.4, make 4.1). Also, here the .deps/ directory is not being generated by make -qp. Back to Debian unstable / Arch / Fedora 30. I tested with good old wget and see the same issue. The command sequence is git clone https://gitlab.com/gnuwget/wget.git cd wget ./bootstrap cd gnulib git checkout master git pull cd .. git commit -m "update gnulib" gnulib ./bootstrap ./configure make syntax-check This also creates the .deps/ directory. So maybe make 4.2.1 has a bug as it creates files though -q is given. But then, why can't you reproduce with the same version of make ? Earlier in this thread I attached a Dockerfile that makes it pretty easy to reproduce. If you don't use docker, I could send you the complete make -qp output. Regards, Tim signature.asc Description: OpenPGP digital signature
Re: maintainer-makefile: catch uses of $< in non-implicit rules
Hi Akim,
On 5/18/19 11:51 AM, Akim Demaille wrote:
> commit 2e801e81bb362429d0e252d076233bfdac20e367
> Author: Akim Demaille
> Date: Sat May 18 08:46:00 2019 +0200
>
> maintainer-makefile: catch uses of $< in non-implicit rules
>
> * top/maint.mk (sc_prohibit_magic_number_exit): New.
Wrong sc rule - it should have been this:
* top/maint.mk (sc_prohibit_gnu_make_extensions_awk_): New.
(sc_prohibit_gnu_make_extensions): New.
Do we fix the ChangeLog in cases like this?
Thanks & have a nice day,
Berny
>
> diff --git a/ChangeLog b/ChangeLog
> index 1918041f0..8d5a4ca6e 100644
> --- a/ChangeLog
> +++ b/ChangeLog
> @@ -1,3 +1,8 @@
> +2019-05-18 Akim Demaille
> +
> + maintainer-makefile: catch uses of $< in non-implicit rules
> + * top/maint.mk (sc_prohibit_magic_number_exit): New.
> +
> 2019-05-14 Paul Eggert
>
> close-stream, closein, closeout: simplify
> diff --git a/top/maint.mk b/top/maint.mk
> index e9d5ee7d4..3dbe9c378 100644
> --- a/top/maint.mk
> +++ b/top/maint.mk
> @@ -408,6 +408,41 @@ sc_prohibit_magic_number_exit:
> halt='use EXIT_* values rather than magic number' \
> $(_sc_search_regexp)
>
> +# Check that we don't use $< in non-implicit Makefile rules.
> +#
> +# To find the Makefiles, trace AC_CONFIG_FILES. Using VC_LIST would
> +# miss the Makefiles that are not under VC control (e.g., symlinks
> +# installed for gettext). "Parsing" (recursive) uses of SUBDIRS seems
> +# too delicate.
> +#
> +# Use GNU Make's --print-data-base to normalize the rules into some
> +# easy to parse format: they are separated by two \n. Look for the
> +# "section" about non-pattern rules (marked with "# Files") inside
> +# which there are still the POSIX Make like implicit rules (".c.o").
> +sc_prohibit_gnu_make_extensions_awk_ =
> \
> + BEGIN {\
> + RS = "\n\n"; \
> + in_rules = 0; \
> + } \
> + /^\# Files/ {
> \
> + in_rules = 1; \
> + } \
> + /\$$ + print "Error: " file ": $$< in a non implicit rule\n" $$0; \
> + status = 1;\
> + } \
> + END {
> \
> + exit status;\
> + }
> +sc_prohibit_gnu_make_extensions:
> + (cd $(srcdir) && autoconf --trace AC_CONFIG_FILES:'$$1') | \
> + tr ' ' '\n' | \
> + $(SED) -ne '/Makefile/{s/\.in$$//;p;}' | \
> + while read m; do \
> + $(MAKE) -qp -f $$m .DUMMY-TARGET 2>/dev/null | \
> + awk -v file=$$m -e '$($@_awk_)' || exit 1;\
> + done
> +
> # Using EXIT_SUCCESS as the first argument to error is misleading,
> # since when that parameter is 0, error does not exit. Use '0' instead.
> sc_error_exit_success:
Re: maintainer-makefile: catch uses of $< in non-implicit rules
On 7/11/19 10:46 AM, Tim Rühsen wrote: > I can *NOT* reproduce the issue on Debian stretch (gawk 4.1.4, make 4.1). > Also, here the .deps/ directory is not being generated by make -qp. > > > Back to Debian unstable / Arch / Fedora 30. > > I tested with good old wget and see the same issue. > The command sequence is > git clone https://gitlab.com/gnuwget/wget.git > cd wget > ./bootstrap > cd gnulib > git checkout master > git pull > cd .. > git commit -m "update gnulib" gnulib > ./bootstrap > ./configure > make syntax-check I'm getting the same syntax-check error on openSUSE:Tumbleweed (GNU Awk 4.2.1, GNU Make 4.2.1) for the GNU coreutils when trying to update gnulib to latest (similar command sequence as above): $ make syntax-check ... prohibit_gnu_make_extensions Error: gnulib-tests/Makefile: $< in a non implicit rule Have a nice day, Berny
