RE: suggestion: new make function
On Tue, 2011-10-11 at 11:03 -0700, Lawrence Ibarria wrote: -Original Message- From: bug-make-bounces+libarria=nvidia@gnu.org [mailto:bug-make- bounces+libarria=nvidia@gnu.org] On Behalf Of David Boyce Sent: Sunday, September 25, 2011 2:17 PM To: Tim Murphy Cc: bug-make@gnu.org; make-...@gnu.org Subject: Re: suggestion: new make function I agree with Tim and with Tim's suggested API, in particular withfile. Involving the shell in something as basic and inherently portable as writing to a file opens up a world of portability issues, in addition to the performance and readability problems mentioned. Given that some platforms (Windows) have ARG_MAX limits and have adopted command files as the official workaround, a command line generator (which is fundamentally all make does) should have the native capability to create those kinds of command lines. IMHO. My main reason for being reticent about this particular issue is it would be nice to include some kind of @-operator capability in GNU make which would mean you wouldn't need to write files directly yourself for the purposes of splitting very long command lines. However, I am not adverse to adding a new function to write to a file. I'm not super-jazzed about the 3-argument withfile proposal though: I don't like using C fopen() option strings in a make interface. Also I think reading from a file is something that needs a lot more thought; probably best to split that into a separate proposal. ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
RE: suggestion: new make function
I do agree that writing and reading are two different things (since reading would involve some kind of parsing and character handling). I'd table reading from a file until later or until there is enough support for that feature. I would also like to make the writefile interface independent of the fopen interface. Your ideas on how to do this will be the best approach, surely. -- Lawrence -Original Message- From: Paul Smith [mailto:psm...@gnu.org] Sent: Wednesday, October 12, 2011 8:17 AM To: Lawrence Ibarria Cc: 'David Boyce'; Tim Murphy; bug-make@gnu.org; make-...@gnu.org Subject: RE: suggestion: new make function On Tue, 2011-10-11 at 11:03 -0700, Lawrence Ibarria wrote: -Original Message- From: bug-make-bounces+libarria=nvidia@gnu.org [mailto:bug- make- bounces+libarria=nvidia@gnu.org] On Behalf Of David Boyce Sent: Sunday, September 25, 2011 2:17 PM To: Tim Murphy Cc: bug-make@gnu.org; make-...@gnu.org Subject: Re: suggestion: new make function I agree with Tim and with Tim's suggested API, in particular withfile. Involving the shell in something as basic and inherently portable as writing to a file opens up a world of portability issues, in addition to the performance and readability problems mentioned. Given that some platforms (Windows) have ARG_MAX limits and have adopted command files as the official workaround, a command line generator (which is fundamentally all make does) should have the native capability to create those kinds of command lines. IMHO. My main reason for being reticent about this particular issue is it would be nice to include some kind of @-operator capability in GNU make which would mean you wouldn't need to write files directly yourself for the purposes of splitting very long command lines. However, I am not adverse to adding a new function to write to a file. I'm not super-jazzed about the 3-argument withfile proposal though: I don't like using C fopen() option strings in a make interface. Also I think reading from a file is something that needs a lot more thought; probably best to split that into a separate proposal. ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: suggestion: new make function
I have limited sympathy for this type of situation, multifile compilation is against the general idea of make. On the other hand, running ar once per .o file cost time (at least) quadratic in the number of files, when I tried it. So multifile archiving is perfectly standard - and it's usually driven by make. Eddy. ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: suggestion: new make function
At Tue, 20 Sep 2011 12:09:42 -0700,
Lawrence Ibarria wrote:
I have often hit problems with the limit of command line lengths
many shells have (CMD.EXE in Windows in my case).
This is a common case with few solutions, I have searched around:
http://www.makelinux.net/make3/make3-CHP-5-SECT-6
I would like to propose a couple new functions in make that can help
work around this problem.
$(dumpOneLine filename, $(list))
$(dumpOnePerLine filename, $(list))
The idea here is to ask make to dump into a file (either on a single
line or each element of the list in a new line) all the contents of
a list.
This function specifically does not use the shell, I expect make to
implement it with fopen directly. Once all the contents of a long
list in make are in a file, programs (such as linker, or compiler)
can use that file as input.
I do not know how to address specifics such as variable expansion on
the list (I'd suggest no variable expansion at all, create a new
list for that).
What do you think?
-- Lawrence
I vote 'no'. This can easily be implemented in your
Makefile. (assuming no single list item breaks the limit)
dumpOneLine = $(foreach item,$(2),echo -n '$(item) ' '$(1)')
dumpOnePerLine = $(foreach item,$(2),echo '$(item)' '$(1)')
then use
$(call dumpOnePerLine filename, $(list))
I don't know if '' works the same in CMD.EXE as it does in Bash
(concatentate stdout of the command on the left to the file on the
right), or if echo behaves similarly ('-n' to make it ommit the
trailing newline), but something like it should work.
Further, this is frequently the WRONG way to do it. The example on the
page you linked to is terrible (surprising, I've come to expect better
from O'Reilly). The proper way to do that in make is:
java_source_files = $(wildcard $(addsuffix /*.java,$(source_dirs)))
java_class_files = $(patsubst %.java,%.class,$(java_source_files))
compile_all: $(java_class_files)
%.class: %.java
$(JAVAC) $
I'm sure someone can find an example of when the command length is an
actuall problem, but generally, it means that you're doing it wrong.
You're trying to write your makefile like a scripting language,
defining functions for doing tasks. Make is about defining
relationships between files.
~ Luke Shumaker
http://lukeshu.ath.cx
___
Bug-make mailing list
Bug-make@gnu.org
https://lists.gnu.org/mailman/listinfo/bug-make
Fwd: Re: suggestion: new make function
You didn't CC the list
---BeginMessage---
I use the foreach trick and it utterly sucks. It's complicated and
tends to execute the shell 10x more than it needs to. For small
makefiles it doesn't matter but doing this for compiles (e.g. the ARM
RVCT's multifile compilation option) in a large makefile is very
unfortunate. I have wanted this feature in other situations too e.g.
one which involved being able to create a persistent list of files
seen in the build of a named component no matter what platform it was
built for and the only way to get what was needed turned out to be
much more mindbending thanks to not having a function that appends to
a file I find the code hard to understand every time I look at it and
I know that anyone else who does is at risk of breaking it.
There are obviously a multitude of other uses like keeping debug
information out of the main log - you name it.
Although you can try using $(shell), you simply introduce a
performance hit again which adds an hour onto some of my builds.
I second the original poster of the message except I'd prefer
something more like:
$(writefile filename,x)
$(appendfile filename,x)
I'd get the one-item-per-line effect by inserting '\n' into the spaces
in advance.
or perhaps:
$(withfile filename,w,$(info some stuff))
$(withfile filename,a,$(foreach item,$(stuff),$(info $(item
$(withfile filename,r+,$(info GET /)$(read RESPONSE))
... acting as a redirector for stdout to the file and allowing one to
perform quite complicated actions such as writing and reading from
sockets although it is perhaps not all that functional.
Regards,
Tim
On 25 September 2011 18:41, Luke Shumaker luke...@sbcglobal.net wrote:
At Tue, 20 Sep 2011 12:09:42 -0700,
Lawrence Ibarria wrote:
I have often hit problems with the limit of command line lengths
many shells have (CMD.EXE in Windows in my case).
This is a common case with few solutions, I have searched around:
http://www.makelinux.net/make3/make3-CHP-5-SECT-6
I would like to propose a couple new functions in make that can help
work around this problem.
$(dumpOneLine filename, $(list))
$(dumpOnePerLine filename, $(list))
The idea here is to ask make to dump into a file (either on a single
line or each element of the list in a new line) all the contents of
a list.
This function specifically does not use the shell, I expect make to
implement it with fopen directly. Once all the contents of a long
list in make are in a file, programs (such as linker, or compiler)
can use that file as input.
I do not know how to address specifics such as variable expansion on
the list (I'd suggest no variable expansion at all, create a new
list for that).
What do you think?
-- Lawrence
I vote 'no'. This can easily be implemented in your
Makefile. (assuming no single list item breaks the limit)
dumpOneLine = $(foreach item,$(2),echo -n '$(item) ' '$(1)')
dumpOnePerLine = $(foreach item,$(2),echo '$(item)' '$(1)')
then use
$(call dumpOnePerLine filename, $(list))
I don't know if '' works the same in CMD.EXE as it does in Bash
(concatentate stdout of the command on the left to the file on the
right), or if echo behaves similarly ('-n' to make it ommit the
trailing newline), but something like it should work.
Further, this is frequently the WRONG way to do it. The example on the
page you linked to is terrible (surprising, I've come to expect better
from O'Reilly). The proper way to do that in make is:
java_source_files = $(wildcard $(addsuffix /*.java,$(source_dirs)))
java_class_files = $(patsubst %.java,%.class,$(java_source_files))
compile_all: $(java_class_files)
%.class: %.java
$(JAVAC) $
I'm sure someone can find an example of when the command length is an
actuall problem, but generally, it means that you're doing it wrong.
You're trying to write your makefile like a scripting language,
defining functions for doing tasks. Make is about defining
relationships between files.
~ Luke Shumaker
http://lukeshu.ath.cx
___
Bug-make mailing list
Bug-make@gnu.org
https://lists.gnu.org/mailman/listinfo/bug-make
--
You could help some brave and decent people to have access to
uncensored news by making a donation at:
http://www.thezimbabwean.co.uk/friends/
---End Message---
___
Bug-make mailing list
Bug-make@gnu.org
https://lists.gnu.org/mailman/listinfo/bug-make
Re: suggestion: new make function
At Sun, 25 Sep 2011 19:36:26 +0100, Tim Murphy wrote: My apologies to Luke for forgetting reply to all and for forgetting my manners by using the word sucks when I should not have. Now it's my turn to apologize, I just forwarded your original message to me to the list. Sorry. dumpOneLine = $(foreach item,$(2),echo -n '$(item) ' '$(1)') dumpOnePerLine = $(foreach item,$(2),echo '$(item)' '$(1)') then use $(call dumpOnePerLine filename, $(list)) I use the foreach trick and I dislike it. It's complicated and tends to execute the shell 10x more than it needs to. For small makefiles it doesn't matter but doing this for compiles (e.g. the ARM RVCT's multifile compilation option) in a large makefile is very unfortunate. I have limited sympathy for this type of situation, multifile compilation is against the general idea of make. Certainly it does offer a speed improvement when compiling the whole thing for the first time, but (in my experience) ends up costing the developer more time as they wait for the entire thing to recompile every time. I suppose there is something to be said for the end-user compiling it, but having the end-user compile something theirself is uncommon. I have wanted this feature in other situations too e.g. one which involved being able to create a persistent list of files across multiple builds for a named component no matter what platform it was built for and no matter whether any recipe actions happened or not. The only way to get what was needed turned out to be much more mindbending thanks to not having a function that appends to a file I find the code hard to understand every time I look at it and I know that anyone else who does is at risk of breaking it. There are obviously a multitude of other uses like keeping debug information out of the main log - you name it. Although you can try using $(shell), you simply introduce a performance hit again which adds an hour onto some of my builds. I second the original poster of the message except I'd prefer something more like: $(writefile filename,x) $(appendfile filename,x) I'd get the one-item-per-line effect by inserting '\n' into the spaces in advance. or perhaps: $(withfile filename,w,$(info some stuff)) $(withfile filename,a,$(foreach item,$(stuff),$(info $(item $(withfile filename,r+,$(info GET /)$(read RESPONSE)) ... acting as a redirector for stdout to the file and allowing one to perform quite complicated actions such as writing and reading from sockets although it is perhaps not all that functional. I find your suggestions more elagant than the original, they seem more general purpose, and less let's introduce this feature to get around a specific problem. Of your two suggestions, I'd favor the withfile option, it would be both more powerful, and doesn't introduce the problem of trying to play with whitespace in make vairables. I'm sure more discussion will need to happen before anything gets merged, but I do like the idea of make-level IO redirection, but mostly for debugging/logging, and storing persistant information. ~ Luke Shumaker http://lukeshu.ath.cx ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: suggestion: new make function
I find your suggestions more elagant than the original, they seem more general purpose, and less let's introduce this feature to get around a specific problem. Of your two suggestions, I'd favor the withfile option, it would be both more powerful, and doesn't introduce the problem of trying to play with whitespace in make vairables. I'm sure more discussion will need to happen before anything gets merged, but I do like the idea of make-level IO redirection, but mostly for debugging/logging, and storing persistant information. Libreoffice could use that too: right now we use a ugly hack to work around command line size limitation: # Write string to temporary file by chopping into pieces that # fit the commandline # parameters: filename, maxitems (for one write), string # returns: filename define var2file $(strip $(1) $(eval gb_var2file_helpervar := $$(shell printf %s $(1) ))\ $(foreach item,$(3),$(eval gb_var2file_curblock += $(item) ifeq ($$(words $$(gb_var2file_curblock)),$(2)) gb_var2file_helpervar := $$(shell printf %s $$(gb_var2file_curblock) $(1) ) gb_var2file_curblock := endif ))\ $(eval gb_var2file_helpervar := $(shell echo $(gb_var2file_curblock) $(1) ) gb_var2file_curblock := )) endef being able to do make-level-io would certainly improve that a bit... Norbert ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
