Re: I: adjust test suite for upcoming GNU Make 3.83
pds == Paul D Smith [EMAIL PROTECTED] writes: [...] adl So any solution that require a syntax that is different in adl prerequisites than it is in the commands seems wrong to me. pds I definitely agree that this incongruity is disturbing, but that's the pds way it is. Don't read me wrong, that wasn't a judgement of the way things are but an attempt to characterize an acceptable solution for Automake. Automake is meant to produce portable Makefiles so I don't want to introduce non-portable constructs in its generated output, even conditionally (it's already hard enough to test). The only place we could tolerate non-portable constructs is in the user input (*.am) that is output almost verbatim. Hence my statement: since users writes the variables, I'm looking for a solution that involves only changing these variables, and since these variables are used both in prerequisite lists and commands... pds Hopefully the next version of GNU make will have some more generic pds mechanism for quoting that will handle ALL different special pds characters that might appear in filenames, including colons and pds whitespace as well as dollar signs. Excellent! That seems to be the ticket, doesn't it? -- Alexandre Duret-Lutz
Re: I: adjust test suite for upcoming GNU Make 3.83
%% Alexandre Duret-Lutz [EMAIL PROTECTED] writes: adl So any solution that require a syntax that is different in adl prerequisites than it is in the commands seems wrong to me. pds I definitely agree that this incongruity is disturbing, but pds that's the way it is. adl Don't read me wrong, that wasn't a judgement of the way things adl are but an attempt to characterize an acceptable solution for adl Automake. Automake is meant to produce portable Makefiles so I adl don't want to introduce non-portable constructs in its generated adl output, even conditionally (it's already hard enough to test). adl The only place we could tolerate non-portable constructs is in adl the user input (*.am) that is output almost verbatim. Hence my adl statement: since users writes the variables, I'm looking for a adl solution that involves only changing these variables, and since adl these variables are used both in prerequisite lists and adl commands... I definitely understand your concern. All I can say is that filenames with $ in them (and so, in effect, any support of Java IIRC) is already completely non-portable. I guess the problem is that it used to be non-portable in that it wouldn't work correctly, but at least the output makefile was a valid makefile for all versions of make. Now, if we want to support backward-compatibility, you'd need to put some code into the makefile which is valid make syntax only in GNU make... which means you'd have to forgo even printing it into a makefile unless you knew it was needed. Hm. Or, do you? This actually should be valid make syntax in all versions of make: PRE_D = $(if $(filter second-expansion,$(.FEATURES)),,$$) FILENAMES = foo$$bar all: $(subst $$,$(PRE_D),$(FILENAMES)) @echo 'FILENAMES = $(FILENAMES)' @echo '$@ : $^' foo$$bar: @echo 'touch $@' Now, obviously, it won't WORK properly with any make other than GNU make, but at least the syntax is valid POSIX make syntax. The other problem of course, is that you can't use the subst function in every makefile for every prerequisite: you'd need to figure out which makefiles might have $ in the prerequisite lists and only use it there. So, it's still gross. The other alternative is to do away with backward compatibility and just say that as of release automake XXX you need GNU make 3.81 if your makefiles have $ in the names. The nice(?) thing about this is that these makefiles should also, then, work with other SysV make versions. Or, you can have an automake option that selects which the user has. Or test for it, although of course that's not very reliable. Bleah. I see no good solution to this. pds Hopefully the next version of GNU make will have some more generic pds mechanism for quoting that will handle ALL different special pds characters that might appear in filenames, including colons and pds whitespace as well as dollar signs. adl Excellent! That seems to be the ticket, doesn't it? Well, sure, but that won't help us between the time 3.81 is released and the time the version with the quoting enhancement is released. I'd really hate to have to tell people they couldn't use 3.81 with automake. I guess it would only impact users with $ in filenames but still... And, of course, that syntax won't be portable either :-/. -- --- Paul D. Smith [EMAIL PROTECTED] Find some GNU make tips at: http://www.gnu.org http://make.paulandlesley.org Please remain calm...I may be mad, but I am a professional. --Mad Scientist
Re: I: adjust test suite for upcoming GNU Make 3.83
However, apparently there is no choice but to support them since Java requires it. It really confuses me how a company like Sun could create a language that is so difficult to use with standard UNIX tools. If you're going to invent a language, why not make it easy to work with? I may be able to help you on this. Java guys have a very different view of what portability is: try to avoid any tools which are not written in Java. Wow! now all you need to do to cover a new platform, is to port the JVM. No need to port each individual tool separately, nor to learn portable functionality subsets out of fear that a recent tool version may not run on the target platform. In fact, you may even extend most tools (in the sense of writing a make plug-in in Java to *save* labor) if you subscribe to this line of thinking, and it is frequently done. Thus, while most C/C++ projects with wide portability do use make, most Java projects with wide portability use ant in its place. Today, C/C++, Java, Unix, make etc. are so unbelievably common things that you may certainly want to support Java in make (for example for large-scale integration of legacy Java with C/C++, esp. in UNIX environment.) But it was not so when Sun was working on the support for inner classes (i.e. before Java 1.1). At that time, many people still saw Java as a (web) client-side programming language, and the dollars were thus seen to be primarily introduced into bytecode class names, rather than into filenames.) Jirka
Re: I: adjust test suite for upcoming GNU Make 3.83
Hi, On Sun, Jul 10, 2005 at 12:19:03PM -0400, Paul D. Smith wrote: The other alternative is to do away with backward compatibility and just say that as of release automake XXX you need GNU make 3.81 if your makefiles have $ in the names. that sounds reasonable. Perhaps there should be an AM_* macro to ensure this, to be called in configure.ac for such projects. But as you might have noticed, Alexandre has much more experience. His opinion is what matters. The nice(?) thing about this is that these makefiles should also, then, work with other SysV make versions. Do other SysV make versions support $(subst ...)? Without it, the makefile might have to contain two copies of each variable containing a '$' (and all variables which use this variable, and recursively). This won't be convenient. And moreover, if some of the source files are substituted by configure, you cannot fix the syntax. Or, you can have an automake option that selects which the user has. I'm afraid this is not possiblle: Automake options are set by the maintainer of the project, long before the tarball is generated. But I guess you don't want to have different tarballs for different make flavours. Have a nice day, Stepan Kasal
Re: I: adjust test suite for upcoming GNU Make 3.83
adl == Alexandre Duret-Lutz [EMAIL PROTECTED] writes: [...] pds I don't know whether automake currently works correctly pds for Java with any other version of make, but this would pds definitely cause that not to be the case going forward. adl I believe $-filenames works only with GNU make. Scratch that, I was mistaken by the test case. I restricted the test case to GNU make when I discovered it was failing with Solaris make. But $$ works fine with other make implementations such as BSD make and Tru64 make. Maybe that is another argument in favor of a way to disable second-expansion? I really think this is the best solution so far. It sounds simple to me because it is localized (just don't run the second expansion in make, as opposed to edit every single rule that other tools like Automake output or that people write), and it's generally useful (allowing to write Makefiles that are portable to the aforementioned make implementations). -- Alexandre Duret-Lutz
Re: I: adjust test suite for upcoming GNU Make 3.83
Hi, On Tue, Jul 05, 2005 at 10:24:55AM -0400, [EMAIL PROTECTED] wrote: [...] For example the version of SysV make that comes with Solaris (and probably other UNIX systems) works identically to the new behavior of GNU make: the prerequisites list is evaluated twice, so any $ that appears on the prerequisites list must be double-quoted (). sk Alternatively, Automake could simply state that $'s in filename are sk not portable. In fact, that is true already, and has always been true :-/. thank you for teaching me. All things being equal I prefer not to allow whole features like this to be enabled/disabled as it makes testing, etc. more difficult. However, I'll consider doing it if there's a plausible argument to be made that the old behavior was inherently more portable than the current behavior. Anyway, with the new behaviour, the $-in-filenames is broken for current Automake. If future versions of Automake wanted to restore that support, and if the feature cannot be switched off, Makefile.in would have to contain the two alternatives I mentioned in my previous mail. This would not only require some work from Automake developpers, it also makes Makefile.in slightly less readable. And yes, there are many people who peek into Makefile.in when they want to understand what's going on, or when they are debugging a bug in their autotools scripts. Have a nice day, Stepan Kasal
Re: I: adjust test suite for upcoming GNU Make 3.83
%% Alexandre Duret-Lutz [EMAIL PROTECTED] writes: adl Is there a way to disable this feature? That might be the adl simplest solution. No, there is no way to disable this at the moment. pds I don't know whether automake currently works correctly pds for Java with any other version of make, but this would pds definitely cause that not to be the case going forward. adl I believe $-filenames works only with GNU make. If so then perhaps it's not as large of a problem. The snippet I posted earlier is portable to all versions of GNU make AFAIK, and will allow $-in-prerequisites to work the same for both. -- --- Paul D. Smith [EMAIL PROTECTED] Find some GNU make tips at: http://www.gnu.org http://make.paulandlesley.org Please remain calm...I may be mad, but I am a professional. --Mad Scientist
Re: I: adjust test suite for upcoming GNU Make 3.83
%% Alexandre Duret-Lutz [EMAIL PROTECTED] writes: adl The user lists filenames in variables, and Automake uses these adl variables in both prerequisite lists or commands. For instance adl see $(dist_my_DATA) in the last excerpt of adl http://lists.gnu.org/archive/html/automake/2005-03/msg00067.html Well, if you can assume GNU make syntax you can easily convert my example into a subst call so that the contents of the variables don't have to change, like this: Same as before: ifeq (,$(filter second-expansion,$(.FEATURES))) # GNU make 3.81 PRE_D := $$ else # GNU make =3.81 PRE_D := endif Then whenever you have prerequisites that might contain $ you would wrap it in a subst call like this: install-dist_myDATA: $(subst $$,$(PRE_D),$(dist_my_DATA)) I've tested this by hand and it works. The trick will be knowing when to generate this GNU make-specific content into the makefile, and when you can avoid it... I assume you'd only want to generate it when you have filenames containing $. adl So any solution that require a syntax that is different in adl prerequisites than it is in the commands seems wrong to me. I definitely agree that this incongruity is disturbing, but that's the way it is. Hopefully the next version of GNU make will have some more generic mechanism for quoting that will handle ALL different special characters that might appear in filenames, including colons and whitespace as well as dollar signs. -- --- Paul D. Smith [EMAIL PROTECTED] Find some GNU make tips at: http://www.gnu.org http://make.paulandlesley.org Please remain calm...I may be mad, but I am a professional. --Mad Scientist
Re: I: adjust test suite for upcoming GNU Make 3.83
Hello, On Tue, Jul 05, 2005 at 09:31:10AM -0400, [EMAIL PROTECTED] wrote: ifeq (,$(filter second-expansion,$(.FEATURES))) # GNU make 3.81 PRE_D := $$ else # GNU make =3.81 PRE_D := endif ... install-dist_myDATA: $(subst $$,$(PRE_D),$(dist_my_DATA)) The trick will be knowing when to generate this GNU make-specific content into the makefile, and when you can avoid it... I assume you'd only want to generate it when you have filenames containing $. Or, as Alexandre pointed out, when you _might_ have them, for example when some filenames are substituted by configure. This means that Automake has to place both alternatives into the generated Makefile.in. When building the package, configure would detect which make is used, and select the right alternative when generating the Makefile, by a mechanism similar to AM_CONDITIONAL. Actually, this can be reduced: Makefile.in would contain the following two alternatives: install-dist_myDATA: $(dist_my_DATA) install-dist_myDATA: $(subst $$,,$(dist_my_DATA)) And configure would select the latter variant, iff GNU make 3.81 is detected. Actually, it follows the standard teplate: - test for a vendor bug in configure - activate a workaround if the bug is detected But it's sad that the vendor of the non-portable implementation here is GNU. Alternatively, Automake could simply state that $'s in filename are not portable. I'd really like to ask you to introduce a way to switch this unusual feature off, preferably with a dot-target, so that portable Makefiles can be generated more easily. Have a nice day, Stepan
Re: I: adjust test suite for upcoming GNU Make 3.83
%% Stepan Kasal [EMAIL PROTECTED] writes: sk But it's sad that the vendor of the non-portable implementation sk here is GNU. It's NOT non-portable. There are other versions of make which do this sort of thing already. For example the version of SysV make that comes with Solaris (and probably other UNIX systems) works identically to the new behavior of GNU make: the prerequisites list is evaluated twice, so any $ that appears on the prerequisites list must be double-quoted (). sk Alternatively, Automake could simply state that $'s in filename are sk not portable. In fact, that is true already, and has always been true :-/. However, apparently there is no choice but to support them since Java requires it. It really confuses me how a company like Sun could create a language that is so difficult to use with standard UNIX tools. If you're going to invent a language, why not make it easy to work with? sk I'd really like to ask you to introduce a way to switch this sk unusual feature off, preferably with a dot-target, so that sk portable Makefiles can be generated more easily. All things being equal I prefer not to allow whole features like this to be enabled/disabled as it makes testing, etc. more difficult. However, I'll consider doing it if there's a plausible argument to be made that the old behavior was inherently more portable than the current behavior. The new behavior is not backward-compatible so that poses its own set of problems. But, I don't think the old behavior was particularly portable either. -- --- Paul D. Smith [EMAIL PROTECTED] Find some GNU make tips at: http://www.gnu.org http://make.paulandlesley.org Please remain calm...I may be mad, but I am a professional. --Mad Scientist
Re: I: adjust test suite for upcoming GNU Make 3.83
Hi all; I got no real response to this. This release of GNU make (3.81) is coming up (one hopes!) and this seems like a real problem that we need to come up with a solution for. If we don't it seems like automake and GNU make 3.81 and Java simply will not be usable together. Anyone have any thoughts or opinions on this? Reminding you; from the NEWS file for 3.81beta3: ftp://alpha.gnu.org/gnu/make/make-3.81beta3.tar.bz2 * WARNING: Backward-incompatibility! GNU make now implements a generic second expansion feature on the prerequisites of both explicit and implicit (pattern) targets. After the rule has been parsed, the prerequisites are expanded a second time, this time with all the automatic variables in scope. This means that in addition to using standard SysV $$@ in prerequisites lists, you can also use complex functions such as $$(notdir $$@) etc. This behavior applies to implicit rules, as well, where the second expansion occurs after the rule is matched. However, this means that you need to double-quote any $ in your filenames; instead of foo: boo$$bar you must write foo: foobar. Since we last spoke, however, there has been a new feature added which may help somewhat: * New special variables available in this release: - .FEATURES: Contains a list of special features available in this version of GNU make. One of the words in the .FEATURES list will be second-expansion if this feature is present. However, I see no way to take advantage of this that is compatible with non-GNU versions of make. I don't know whether automake currently works correctly for Java with any other version of make, but this would definitely cause that not to be the case going forward. -- --- Paul D. Smith [EMAIL PROTECTED] Find some GNU make tips at: http://www.gnu.org http://make.paulandlesley.org Please remain calm...I may be mad, but I am a professional. --Mad Scientist
Re: I: adjust test suite for upcoming GNU Make 3.83
FYI, here's one way to handle it (obviously requires GNU make): ifeq (,$(filter second-expansion,$(.FEATURES))) # GNU make 3.81 PRE_D := $$ else # GNU make =3.81 PRE_D := endif # Now use PRE_D in prerequisites all: foo$(PRE_D)bar ; @echo '$@: $' foo$$bar: ; @echo 'building $@' -- --- Paul D. Smith [EMAIL PROTECTED] Find some GNU make tips at: http://www.gnu.org http://make.paulandlesley.org Please remain calm...I may be mad, but I am a professional. --Mad Scientist
Re: I: adjust test suite for upcoming GNU Make 3.83
pds == Paul D Smith [EMAIL PROTECTED] writes: pds Hi all; pds I got no real response to this. This release of GNU make (3.81) is pds coming up (one hopes!) and this seems like a real problem that we need pds to come up with a solution for. If we don't it seems like automake and pds GNU make 3.81 and Java simply will not be usable together. I Paul. Your mail came in right when I was browsing my TODO list for things to do before Automake 1.9.6. One entry is ping about dollar.test. My last mail about this is here: http://lists.gnu.org/archive/html/automake/2005-03/msg00067.html [...] pds One of the words in the .FEATURES list will be second-expansion if this pds feature is present. pds However, I see no way to take advantage of this that is compatible with pds non-GNU versions of make. I can't see any way either. Is there a way to disable this feature? That might be the simplest solution. pds I don't know whether automake currently works correctly pds for Java with any other version of make, but this would pds definitely cause that not to be the case going forward. I believe $-filenames works only with GNU make. -- Alexandre Duret-Lutz
Re: I: adjust test suite for upcoming GNU Make 3.83
pds == Paul D Smith [EMAIL PROTECTED] writes: pds FYI, here's one way to handle it (obviously requires GNU make): pds ifeq (,$(filter second-expansion,$(.FEATURES))) pds # GNU make 3.81 pds PRE_D := $$ pds else pds # GNU make =3.81 pds PRE_D := pds endif pds # Now use PRE_D in prerequisites pds all: foo$(PRE_D)bar ; @echo '$@: $' pds foo$$bar: ; @echo 'building $@' The user lists filenames in variables, and Automake uses these variables in both prerequisite lists or commands. For instance see $(dist_my_DATA) in the last excerpt of http://lists.gnu.org/archive/html/automake/2005-03/msg00067.html So any solution that require a syntax that is different in prerequisites than it is in the commands seems wrong to me. Besides such a variable might as well defined from configure via a substitution, so Automake cannot do any preprocessing on its contents. -- Alexandre Duret-Lutz
Re: I: adjust test suite for upcoming GNU Make 3.83
pds == Paul D Smith [EMAIL PROTECTED] writes: [...] pds I'm interested in discussing the issue and possible solutions. The one pds I added to the bug report involves using :=, since simply-expanded pds variables in GNU make are always only expanded one time, no matter how pds many times they're referenced (unless within an eval function). But, pds obviously, using := does not yield a portable makefile. Actually it does not appear to work here. I compiled CVS make, and patched Automake as follows. --- tests/dollar.test 14 Nov 2003 21:25:58 - 1.7 +++ tests/dollar.test 16 Mar 2005 22:32:49 - @@ -33,7 +33,7 @@ cat Makefile.am 'EOF' mydir = $(prefix)/my -dist_my_DATA = hello$$world +dist_my_DATA := hello$$world check-dist: distdir test -f '$(distdir)/hello$$world' @@ -43,7 +43,7 @@ $ACLOCAL $AUTOCONF -$AUTOMAKE +$AUTOMAKE -Wno-portability ./configure --prefix `pwd`/inst $MAKE install test -f 'inst/my/hello$world' This test still fails with | ~/projs/cvs/automake/HEAD3/tests % ./dollar.test | /home/adl/projs/cvs/automake/HEAD3/tests:/home/adl/usr/bin:/usr/local/bin:/usr/bin:/bin:/usr/bin/X11:/usr/games | dollar: running make --version -v | grep GNU | GNU Make 3.81beta3 | === Running test ./dollar.test | ++ pwd | /home/adl/projs/cvs/automake/HEAD3/tests/testSubDir | + set -e | + echo AC_OUTPUT | + cat | + : | + aclocal-1.9a -Werror | + autoconf | + automake-1.9a --foreign -Werror -Wall -Wno-portability | ++ pwd | + ./configure --prefix /home/adl/projs/cvs/automake/HEAD3/tests/testSubDir/inst | checking for a BSD-compatible install... /usr/bin/install -c | checking whether build environment is sane... yes | checking for gawk... gawk | checking whether make sets $(MAKE)... yes | configure: creating ./config.status | config.status: creating Makefile | + make install | make: *** No rule to make target `helloorld', needed by `all-am'. Stop. the generated Makefile contains | dist_my_DATA := hello$$world | | DATA = $(dist_my_DATA) | | all-am: Makefile $(DATA) | | install-dist_myDATA: $(dist_my_DATA) | @$(NORMAL_INSTALL) | test -z $(mydir) || $(mkdir_p) $(DESTDIR)$(mydir) | @list='$(dist_my_DATA)'; for p in $$list; do \ | if test -f $$p; then d=; else d=$(srcdir)/; fi; \ | f=$(am__strip_dir) \ | echo $(dist_myDATA_INSTALL) '$$d$$p' '$(DESTDIR)$(mydir)/$$f'; \ | $(dist_myDATA_INSTALL) $$d$$p $(DESTDIR)$(mydir)/$$f; \ | done this seems to imply that simply-expanded variables are re-expanded when they appear inside deferred variables. If I manually change `DATA = $(dist_my_DATA)' (this is not a real solution, because this line is generated by Automake) to `DATA := $(dist_my_DATA)', then the all-am rule passes, but the test fails later with make[1]: *** No rule to make target `helloorld', needed by `install-dist_myDATA'. Stop. -- Alexandre Duret-Lutz
I: adjust test suite for upcoming GNU Make 3.83
Hi, Upcoming version of GNU Make introduces new incompatibility. The NEWS file says: * WARNING: Backward-incompatibility! GNU make now implements a generic second expansion feature on the prerequisites of both explicit and implicit (pattern) targets. After the rule has been parsed, the prerequisites are expanded a second time, this time with all the automatic variables in scope. This means that in addition to using standard SysV $$@ in prerequisites lists, you can also use complex functions such as $$(patsubst f%r,b%o,$$@) etc. This behavior applies to implicit rules, as well, where the second expansion occurs after the rule is matched. However, this means that you need to double-quote any $ in your filenames; instead of foo: boo$$bar you must write foo: foobar GNU Make developers suggest to adjust dollar.test, to make it work with both the old and new versions of GNU make, see http://savannah.gnu.org/bugs/?func=detailitemitem_id=12260 However, automake-1.9 --foreign -Werror -Wall issues warning when :=-style assignment is used, so dollar.test fails in the adjusted form, too. -- ldv pgp2pYYYnZ5SF.pgp Description: PGP signature
Re: I: adjust test suite for upcoming GNU Make 3.83
%% Dmitry V. Levin [EMAIL PROTECTED] writes: dvl Upcoming version of GNU Make introduces new incompatibility. dvl The NEWS file says: dvl * WARNING: Backward-incompatibility! dvl GNU make now implements a generic second expansion feature on the dvl prerequisites of both explicit and implicit (pattern) targets. After dvl the rule has been parsed, the prerequisites are expanded a second dvl time, this time with all the automatic variables in scope. This means dvl that in addition to using standard SysV $$@ in prerequisites lists, dvl you can also use complex functions such as $$(patsubst f%r,b%o,$$@) etc. dvl This behavior applies to implicit rules, as well, where the second dvl expansion occurs after the rule is matched. dvl However, this means that you need to double-quote any $ in your dvl filenames; instead of foo: boo$$bar you must write foo: foobar dvl GNU Make developers suggest to adjust dollar.test, to make it work with dvl both the old and new versions of GNU make, see dvl http://savannah.gnu.org/bugs/?func=detailitemitem_id=12260 Hi Dmitry; I was going to post something here about this when I saw your message :-). I'm interested in discussing the issue and possible solutions. The one I added to the bug report involves using :=, since simply-expanded variables in GNU make are always only expanded one time, no matter how many times they're referenced (unless within an eval function). But, obviously, using := does not yield a portable makefile. I'm not actually sure whether there's any way to write a makefile that doesn't rely on any GNU make features, that handles $ in filenames properly. -- --- Paul D. Smith [EMAIL PROTECTED] Find some GNU make tips at: http://www.gnu.org http://make.paulandlesley.org Please remain calm...I may be mad, but I am a professional. --Mad Scientist
