Re: Optimization for reading *.d files
On Sun, Mar 19, 2017 at 11:54 AM, brenorg wrote: > Hi Norbert, > > You are absolutely right. There is much more redundancy than I expected. I > joined all .d files in one single file and after running make on it and > printing the database, it's actually 10x smaller. And I know there must be > more duplication since my files names are not all relative to the same root > yet. > > I'll continue this path and see if I can get acceptable performance take a look at https://cgit.freedesktop.org/libreoffice/core/tree/solenv/bin/concat-deps.c it is specific to our project.. but still the general gist of it is there. Norbert ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: Optimization for reading *.d files
On Sat, Mar 18, 2017 at 9:25 PM, brenorg wrote: > > Anyway, I want to continue to use GNU Make, and not fallback to CMake/Ninja. > After some profiling, what's killing me is parsing the "*.d" files generated > by the compiler. > > The time to include all dependency files of my project in one single > makefile (as I want to get rid of recursive make), is 4 seconds. > have you looked in how much redundancy you have in all these dep ? For LibreOffice, which use one big make to build it all (60k files or so) we wrote a step to combine and de-duplicate all these .d file reducing the amount that need to be parsed by make by quite a bit iow instead of getting a faster process, reduce the amount to be processed. Norbert ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: Difficulties from the combination of functions "call" and "eval"
On Sun, Jan 18, 2015 at 6:31 AM, SF Markus Elfring wrote: > > Yes. - The evaluated variable should produce shell commands for recipes. > I try to reuse it as a subfunction. then use $(call -- or possibly $$(call, if you want to defer the call to when the recipe get invoked) not $(eval >> It's just _part_ of a recipe? > > It is a fragment for the reuse by another variable expansion. > > >> Then $(eval) is the Wrong Thing. > > I hope not … hope is not relevant... $(eval) _is_ the Wrong Thing(tm) for what you seems to want to do ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: Dynamic adjustments of build dependencies for the number of available processors
On Mon, Jan 5, 2015 at 1:23 PM, Paul Smith wrote: > On Mon, 2015-01-05 at 20:06 +0100, SF Markus Elfring wrote: >> > So you have in your toolbox $(shell) and $(eval). >> >> I am not familiar enough with the second make function. >> http://www.gnu.org/software/make/manual/html_node/Eval-Function.html > > I wrote some blog posts about eval and other metaprogramming techniques > in make that you might find interesting: > > http://make.mad-scientist.net/category/metaprogramming/ > For a real-life large scale use and abuse of these techniques see: implementation: http://cgit.freedesktop.org/libreoffice/core/tree/solenv/gbuild example of use: http://cgit.freedesktop.org/libreoffice/core/tree/svl/Library_svl.mk ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: Add "makefile.mak" to list of default makefiles
On Mon, Jul 7, 2014 at 4:59 PM, Eli Zaretskii wrote: >> From: Paul Smith >> Cc: Eli Zaretskii , bug-make@gnu.org >> Date: Mon, 07 Jul 2014 02:06:54 -0400 >> >> On Mon, 2014-06-30 at 21:28 +0100, Jonny Grant wrote: >> > I have a few, but triggered by make -f makefile.mak. So it would be >> > quite useful GNU Make could pick up the Windows makefile extension >> > .MAK >> >> If Eli feels this is useful for Windows implementations he can add it; >> however, I don't want this added for the non-Windows ports. I've never >> seen anyone name a makefile like this on any UNIX/POSIX system, ever. >> >> As Reinier points out, on UNIX/POSIX systems you often see ".mk" used as >> an extension, but never (IME) "Makefile.mk"; UNIX/POSIX environments >> don't rely (solely) on extensions and (again IME) have no problem >> understanding that files named "Makefile" or "makefile" are makefiles, >> even without extensions. > > Should we add "Makefile.mk" for Posix hosts and "makefile.mak" for > Windows, then? Please do not (for Posix) For instance we use a lot of include of piece for our Makefile(s).. all these pieces are indeed call .mk but these files are fragment and are not to be run directly. Makefile.mk is at best a pleonasm, and if someone really, really likes that, it is already supported via -f Makefile.mk > >> Renaming the Windows README files is also fine with me if it's fine with >> Eli. > > My only doubt about this request is that README files for other > platforms will still be called README.. Should we rename them > all? Why inflict some vestigial CP/M 'feature' on the rest of the world ? Norbert ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: [PATCH1/2] Use spawn() on Cygwin
On Tue, Jul 30, 2013 at 10:39 AM, Eli Zaretskii wrote: >> From: Pavel Fedin >> Date: Tue, 30 Jul 2013 14:42:23 +0400 >> >> Please take this patch, Cygwin team told that they would like to integrate >> with upstream. I have already posted it some time ago but got no reply. >> The patch significantly improves performance of Make under Cygwin. > > Thanks. > > Is there any discussion we could read about that with the details of > the problem and how/why does the proposed patch solves it? fork() is a very expensive operation in cygwin. we are seeing that in LibreOffice build a lot. I will try to pick up that patch and report numbers on it (we have a pretty big make, with 125K+ targets http://skyfromme.wordpress.com/2013/02/28/one/ ) > > In general, I feel it's wrong to do this: Cygwin is a Posix platform, > so it should be using the Posix code, to be as compatible with other > Posix platforms as possible. EMX is not a Posix platform, so using > its code will likely make the Cygwin Make deviate from Posix behavior > at times. in theory you are right... in practice Cygwin maybe be posix, but it's underlying OS is not and the emulation layer can be very costly. Norbert ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: [bug #39028] [patch] fix and uniformize four error messages
On Tue, May 21, 2013 at 12:01 AM, Edward Welbourne wrote: > > In paticular, in the case of "integral", there is a second meaning of > the word (go on, check that dictionary, in case you've forgotten your > calculus) which creates a (completely wanton, given that there's a > plainer term for a number being whole) potential for confusing readers. but you would also need to add some other words to make the sentence grammatically correct again s/integral/integer/ or s/integral/whole number/ does not produce a grammatically correct sentence. so you would need , at the very least, replace 'integral' by 'whole number as' > Even transient confusion disrupts the flow of reading (as does having to > look a word up in a dictionary, for that matter). Plain language aids > all readers, That is a stretch why aren't you complaining about 'argument' which also has multiple meaning. In human languages, context matter. Surely if you are aware of the 'calculus' meaning of integral, you must be aware of the 'being or related to integer' meaning. and in context you can't be more confused here than you'd be confused between the different possible meaning of 'argument' that follows it. Beside, you are aware that we are talking about an error message right ? The flow has already been disrupted. Anyway, I'll stop here on that particular bikesheed. I was just a bit disappointed with the tendency to want to dumb down everything... oh well. > Eddy, a mathematician. Norbert, an engineer who has done his fair share of Math. ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: [bug #39028] [patch] fix and uniformize four error messages
On Mon, May 20, 2013 at 5:53 PM, Edward Welbourne wrote: >>> I think "integer" is meant instead of "integral". 'integral' is the adjective form of 'integer', so, in context, it is correctly used. > >> Eg C99 uses "integral" as an adjective meaning "of integers", C99 and Merriam Webster, Oxford, etc... > > How about using plain language and calling it a "whole number" > instead of using jargon ? How about not catering to the lowest common denominator and devolving to baby-speech for fear that someone may be intimidated by a dictionary ? Norbert ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: Some potential bugs in make-3.82
On Tue, May 7, 2013 at 10:24 PM, Zhenbo Xu wrote: > 2. Out of bound > file: function.c > function: func_sort > At line 1150: words = xmalloc (wordi * sizeof (char *)); > The value of wordi may be 1. > > At line 1171: if (i == wordi - 1 || strlen (words[i + 1]) != len > words[i + 1] may cause overflow. no because i < wordi - 1 ( i < wordi because of the for() condition, and if i == wordi - 1 then the || evaluation is shortcut-ed by i == wordi - 1, hence strlen() is not evaluated) also note that the wordi in the alloc is a 1-based index whereas the wordi in the for loop is a 0-based index ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: suggestion: new make function
> > I find your suggestions more elagant than the original, they seem more > general purpose, and less "let's introduce this feature to get around > a specific problem". Of your two suggestions, I'd favor the "withfile" > option, it would be both more powerful, and doesn't introduce the > problem of trying to play with whitespace in make vairables. > > I'm sure more discussion will need to happen before anything gets > merged, but I do like the idea of make-level IO redirection, but > mostly for debugging/logging, and storing persistant information. Libreoffice could use that too: right now we use a ugly hack to work around command line size limitation: # Write string to temporary file by chopping into pieces that # fit the commandline # parameters: filename, maxitems (for one write), string # returns: filename define var2file $(strip $(1) $(eval gb_var2file_helpervar := $$(shell printf "%s" "" > $(1) ))\ $(foreach item,$(3),$(eval gb_var2file_curblock += $(item) ifeq ($$(words $$(gb_var2file_curblock)),$(2)) gb_var2file_helpervar := $$(shell printf "%s" "$$(gb_var2file_curblock)" >> $(1) ) gb_var2file_curblock := endif ))\ $(eval gb_var2file_helpervar := $(shell echo " "$(gb_var2file_curblock) >> $(1) ) gb_var2file_curblock := )) endef being able to do make-level-io would certainly improve that a bit... Norbert ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
[PATCH] expand debugging options to provide tracing of call to $(eval and $(call
At LibreOffice we are migrating our build system from a customized dmake to gmake. The design of the build system used involved a fairly intensive use of $(eval and $(call to implement an 'Object-Oriented' kind of system. This is nice when things works, but it is pretty hard to find bug in the build system itself... So, to help internally to debug these problems I wrote a patch based on make 3.82, to add --debug=c and --debug=e which activate tracing for $(call and $(eval respectively the patch can be found at http://cgit.freedesktop.org/libreoffice/contrib/dev-tools/plain/misc_patches/make-3.82.lo_trace.patch?id=a4f03f17f42ded70e6a3c49cf4e9a90eaf3c12ca In the spirit of 'pushing upstream', I submit this patch here, under GPLv3+ terms. Regards, Norbert Thiebaud Below is an example of output produced by this patch: [...] ### call $(gb_Library_set_include) --> ### arg 0 for call $(gb_Library_set_include) is 'gb_Library_set_include' ### arg 1 for call $(gb_Library_set_include) is 'msword' ### arg 2 for call $(gb_Library_set_include) is ' -I/lo/libo/clone/writer/sw/source/core/inc -I/lo/libo/clone/writer/sw/source/ui/inc -I/lo/libo/clone/writer/sw/source/filter/inc -I/lo/libo/clone/writer/sw/inc/pch -I/lo/libo/clone/writer/sw/inc -I/lo/libo/solver/340/unxlngx6.pro/workdir/inc/sw/sdi -I/lo/libo/solver/340/unxlngx6.pro/workdir/Misc/sw/ $(INCLUDE) -I/lo/libo/solver/340/unxlngx6.pro/inc/offuh -I/lo/libo/solver/340/unxlngx6.pro/inc/sw ' ### call $(gb_Library_get_linktargetname) --> ### arg 0 for call $(gb_Library_get_linktargetname) is 'gb_Library_get_linktargetname' ### arg 1 for call $(gb_Library_get_linktargetname) is 'msword' ### arg 2 for call $(gb_Library_get_linktargetname) is implicit ### call $(gb_Library_get_filename) --> ### arg 0 for call $(gb_Library_get_filename) is 'gb_Library_get_filename' ### arg 1 for call $(gb_Library_get_filename) is 'msword' ### arg 2 for call $(gb_Library_get_filename) is implicit ### call to $(gb_Library_get_filename) expended into libmswordlo.so ### call $(gb_Library_get_filename) <-- ### call to $(gb_Library_get_linktargetname) expended into Library/libmswordlo.so ### call $(gb_Library_get_linktargetname) <-- ### call $(gb_LinkTarget_set_include) --> ### arg 0 for call $(gb_LinkTarget_set_include) is 'gb_LinkTarget_set_include' ### arg 1 for call $(gb_LinkTarget_set_include) is 'Library/libmswordlo.so' ### arg 2 for call $(gb_LinkTarget_set_include) is ' -I/lo/libo/clone/writer/sw/source/core/inc -I/lo/libo/clone/writer/sw/source/ui/inc -I/lo/libo/clone/writer/sw/source/filter/inc -I/lo/libo/clone/writer/sw/inc/pch -I/lo/libo/clone/writer/sw/inc -I/lo/libo/solver/340/unxlngx6.pro/workdir/inc/sw/sdi -I/lo/libo/solver/340/unxlngx6.pro/workdir/Misc/sw/ $(INCLUDE) -I/lo/libo/solver/340/unxlngx6.pro/inc/offuh -I/lo/libo/solver/340/unxlngx6.pro/inc/sw ' ### arg 3 for call $(gb_LinkTarget_set_include) is '' ### call $(gb_LinkTarget_get_headers_target) --> ### arg 0 for call $(gb_LinkTarget_get_headers_target) is 'gb_LinkTarget_get_headers_target' ### arg 1 for call $(gb_LinkTarget_get_headers_target) is 'Library/libmswordlo.so' ### arg 2 for call $(gb_LinkTarget_get_headers_target) is implicit ### arg 3 for call $(gb_LinkTarget_get_headers_target) is implicit ### call to $(gb_LinkTarget_get_headers_target) expended into /lo/libo/solver/340/unxlngx6.pro/workdir/Headers/Library/libmswordlo.so ### call $(gb_LinkTarget_get_headers_target) <-- ### call $(gb_LinkTarget_get_target) --> ### arg 0 for call $(gb_LinkTarget_get_target) is 'gb_LinkTarget_get_target' ### arg 1 for call $(gb_LinkTarget_get_target) is 'Library/libmswordlo.so' ### arg 2 for call $(gb_LinkTarget_get_target) is implicit ### arg 3 for call $(gb_LinkTarget_get_target) is implicit ### call to $(gb_LinkTarget_get_target) expended into /lo/libo/solver/340/unxlngx6.pro/workdir/LinkTarget/Library/libmswordlo.so ### call $(gb_LinkTarget_get_target) <-- ### call $(gb_LinkTarget_get_dep_target) --> ### arg 0 for call $(gb_LinkTarget_get_dep_target) is 'gb_LinkTarget_get_dep_target' ### arg 1 for call $(gb_LinkTarget_get_dep_target) is 'Library/libmswordlo.so' ### arg 2 for call $(gb_LinkTarget_get_dep_target) is implicit ### arg 3 for call $(gb_LinkTarget_get_dep_target) is implicit ### call to $(gb_LinkTarget_get_dep_target) expended into /lo/libo/solver/340/unxlngx6.pro/workdir/Dep/LinkTarget/Library/libmswordlo.so.d ### call $(gb_LinkTarget_get_dep_target) <-- ### call to $(gb_LinkTarget_set_include) expended into /lo/libo/solver/340/unxlngx6.pro/workdir/Headers/Library/libmswordlo.so /lo/libo/solver/340/unxlngx6.pro/workdir/LinkTarget/Libra
