Re: Converting make rules into other file formats
> Are the chances better to extend the tool “remake” then? … > Probably. Can it be that our software development resources are also too limited for mentioned tasks? > My intention was to provide profiling information by default … This is generally fine. Do any collaboration challenges hinder to achieve further improvements in this direction? > The dependency graphing feature was also present in the branches > I was planning to upstream since it allowed me to check > if the dependencies in our build system were incorrect from a logical PoV > (started with a sequential build) This information sounds promising. Did you clarify any remaining open issues with involved maintainers (in the background)? Regards, Markus ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: Converting make rules into other file formats
> Before talking about formats and graphical representation, > I advise you to ask if Paul agrees with this kind thing. Are the chances better to extend the tool “remake” then? https://github.com/rocky/remake/wiki Regards, Markus ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: Converting make rules into other file formats
> A Makefile is basically a human-readable text file with instructions > on how to create executables with a compiler for example. How often do you look for a more visual representation of this content? > It has pretty much nothing to do with creating graphics. I got an other opinion. * Do you care to display constructed directed acyclic graphs by other tools? * Are you interested in further improvements for data export variants? Regards, Markus ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Converting make rules into other file formats
Hello, How does the current software situation look like for the conversion of data from make scripts into file formats with which other users are more familiar? Examples: * Outlines * Vector graphics Would you like to point any special development tools out? Regards, Markus ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Checking unique recipe execution during parallel software builds
Hello, I am testing my recent adjustments for a few build scripts. Now I wonder again about an error message. But I guess that it should only be taken into account when an other issue will be checked once more and resolved then hopefully. elfring@Sonne:~/Projekte/Bau/OCamlbuild> LANG=C make -r -j4 byte … ocamlc.opt -pack -o 'o/ocamlbuild_pack.cmo' … … make: *** [GNUmakefile:1221: o/ocamlbuild_pack.cmi] Error 2 This compilation command is started twice in the shown software situation while I would expect that the affected recipe should be executed only once at the end of the attempted build process. Do I observe further challenges for parallel software builds here? Regards, Markus ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: Switching from relative paths to absolute directory specifications in make scripts?
>> This test result seems to show that it can make a significant difference >> if such a target was specified by a relative or absolute path. >> (I would expect that these specifications will refer to the same file.) > > Keep in mind that targets are opaque strings, I would prefer a more convenient handling of build goals in some use cases. I noticed once more how relevant implicit make rules can become in this test case. > not references to files as such, and hence the exact path matters. The make software manages also a bit of extra background information. How interesting can it become to filter targets if they are phony (or not)? > E.g. "foo", "./foo", and "bar/../foo" are all different targets as far > as make is concerned, even if they all resolve to the same file on disk. Can the software situation be improved any more there? >> Do I need to extend the software build rules a bit more here? > > Yep, you need to fix your makefile if you want it to work with both > relative and absolute paths. I can adjust specific make scripts to some degree. Will it be better to avoid the duplication of corresponding build rules? > If you are doing complicated things, especially if you have > a modular build system with makefiles in different sub-directories > referring to the same files, you need to be careful how you use and > construct target paths so they are consistent. * Do you specify any extra explicit (non-pattern) make rules then? * How are the chances to reduce any inconveniences there? Regards, Markus ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Switching from relative paths to absolute directory specifications in make scripts?
Hello, I am fiddling with a few build scripts for another free software. elfring@Sonne:~/Projekte/Bau/OCamlbuild> rm -f src/glob_lexer.ml && LANG=C make --no-builtin-rules V=1 src/glob_lexer.ml; LANG=C make --no-builtin-rules V=1 ~/Projekte/Bau/OCamlbuild/src/glob_lexer.ml make: *** No rule to make target 'src/glob_lexer.ml'. Stop. ocamllex.opt -o /home/elfring/Projekte/Bau/OCamlbuild/src/glob_lexer.ml /home/elfring/Projekte/OCaml/OCamlbuild/lokal/src/glob_lexer.mll 55 states, 419 transitions, table size 2006 bytes This test result seems to show that it can make a significant difference if such a target was specified by a relative or absolute path. (I would expect that these specifications will refer to the same file.) Do I need to extend the software build rules a bit more here? Regards, Markus ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: Ensuring existance of directories before writing files into them
> Here's one way to solve the problem: > http://www.chaos.org.uk/~eddy/craft/make.html#AutoDir Thanks for your link for information around the topic “Autogenerating (needed) directories”. Will it become better recognised as another design/construction pattern? Regards, Markus ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: [bug #51311] Checking search retries for implicit make rules
> At the risk of stating the obvious, note that there is a trivial > work-around for this - there is nothing special about dot in patterns, > so you could merge the dot into what % matches to make it non-empty: It might be acceptable to give up a character from either end of the prefix (or suffix) for my toy example. Will the pattern match be a bit more flexible with this trick than what was originally intended? I imagine that direct support would be nice for a placeholder which can be treated as optional in such an use case. Regards, Markus ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Ensuring existance of directories before writing files into them
Hello, Files are stored within directories. So there is a general dependency that the directory should exist (together with an identifier) before a file can be changed. Prerequisites for make rules can be specified in the way that their file names reference a specific build directory. I have got a few approaches in mind to convert such an implicit dependency to an explicit one for the make software. * I could check if the source file folder is different from the selected build directory. * If the desired build subdirectory did not exist, I could create it automatically. There is a choice if such a directory creation should be performed by a call of the make function “shell” or by a command in a recipe. Which design variant would you prefer for this use case? Regards, Markus ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: [bug #51309] Determination of a file list from a single folder without changing the working directory
> Are you able to provide any example proving how it matters under some more > or less rare circumstances? I guess not at the moment. > Or do you find the current software good enough I dared to point some change possibilities out recently. > and no changes needed? It depends on your views for software developments which your prefer. I imagine that algorithm comparisons can lead to related improvements, can't they? Regards, Markus ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Trying to avoid a repeated search for a prerequisite (or target)
Hello, The variable “VPATH” has got a special meaning for the make software. https://www.gnu.org/software/make/manual/html_node/General-Search.html “… Thus, if a file that is listed as a target or prerequisite does not exist in the current directory, make searches … …” I would interpret this information also in the way that it indicates a few optimisation possibilities to avoid the second search try there. * A used path can eventually never fit to the current directory. Can this only happen with absolute paths? * The relevant files refer to a separate build directory (instead of a source file directory). So I imagine that the execution of make rules can be influenced by the detail if an unique path was specified. Will any efforts for the construction of such unique paths pay off? I got another software development concern in this context: How much can the usage of absolute paths influence the desired name resolution between targets and prerequisites in make (static) pattern rules? Regards, Markus ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: Improving dependency specifications in make rules?
> I am also curious on how other involved contributors will react to > a pull request for recent software developments around the discussed > build scripts. Would you like to add any constructive comments there? https://github.com/coccinelle/coccinelle/pull/108 How do you think about to clarify related ideas? Regards, Markus ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: Checking application of dependencies from make rules without recipes
> I can only assume you are referring to this pattern rule: > >> %.cmo: %.ml %.cmi >> $(o_compilation) '$<' > $@ > > and you're saying that if the .cmi file cannot be created, that make > should not consider this an error. I became also curious on how the clarification will evolve for the request “Checking search retries for implicit make rules”. https://savannah.gnu.org/bugs/?51311 I began to reconsider my software development approach once more after the provided feedback. > It is generally not possible to represent this behavior in makefile syntax. Will further software extensions become relevant here? > The best you can do is remove the %.cmi from the pattern rule and > declare explicit prerequisite rules for those targets that you'd like to > have a .cmi file prerequisite. I tried to increase the relevance for compiled interface descriptions there. Yesterday I published changes for my intermediate development results. https://github.com/elfring/Coccinelle-20160205/commit/13fd24aff54fd86ded5a3093f927f0fed53d1f99 I started to use static pattern rules at another place. This change triggers a need to clarify the following situation. elfring@Sonne:~/Projekte/Coccinelle/20160205/commons> LANG=C make --no-builtin-rules V=1 Makefile:215: warning: overriding recipe for target 'commands.cmo' /home/elfring/Projekte/Coccinelle/20160205/common_rules.make:9: warning: ignoring old recipe for target 'commands.cmo' Makefile:218: warning: overriding recipe for target 'commands.cmx' /home/elfring/Projekte/Coccinelle/20160205/common_rules.make:12: warning: ignoring old recipe for target 'commands.cmx' make: *** No rule to make target 'commands.cmi', needed by 'commands.cmo'. Stop. 1. This error message is appropriate at the moment while it indicates related improvement opportunities. The mentioned OCaml module “commands” should be compilable without a cmi file so far. https://lists.gnu.org/archive/html/bug-make/2017-06/msg00092.html I specified therefore a corresponding make rule pair. https://github.com/elfring/Coccinelle-20160205/blob/9597598779f3604272621f6d42ead79eaafad2a8/commons/Makefile#L214 I hoped then that the build process would still succeed because I tried to lift the dependency on a special file type. Now it seems that my expectations do not fit to the current software behaviour again. Can this approach work ever in such a design direction? 2. The shown warnings will be displayed so often that I would prefer to suppress them occasionally. Is it possible to deactivate them for such cases? Regards, Markus ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: [bug #51309] Determination of a file list from a single folder without changing the working directory
> MY_FILES=$(notdir $(wildcard somedir/*.ext)) I agree that this algorithm can work. But it does not fit to my imaginations for efficient data processing. If the desired directory on which a file filter should be applied could be selected before, the file names would not contain references to this directory and another function call is therefore unnecessary to remove it from the returned list elements. The difference can eventually better seen by an approach like the following. ~/Projekte> ls Bau/*txt Bau/MOTD.txt ~/Projekte> cd Bau && ls *txt MOTD.txt I would like to achieve the second list result without using the command “cd” before. Regards, Markus ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: [bug #51311] Checking search retries for implicit make rules
> To quote the make info pages, section 10.5.1 "Introduction to Pattern Rules": > - > A pattern rule contains the character `%' (exactly one of them) I interpreted this description in the way that one placeholder would be supported for each element in the list of prerequisites. > …; the `%' matches any nonempty substring, while other characters match only > themselves. > - How do you think about to mention the requirement “non-empty string” at another place? https://www.gnu.org/software/make/manual/html_node/Pattern-Match.html > There's no restriction in the documention on the number of prequisites on > the rule and they may contain any mix of pattern and non-pattern names. I find that this wording can be only partly appropriate. * Is there a current software restriction that only a single pattern is supported within the prerequisites list for a specific make rule? * Will any more developers become interested in related software extensions? Regards, Markus ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: Replacing suffix rules in make scripts?
> Why do I get such an error message for a parallel software build attempt > when the specified dependencies worked as expected for the non-parallel > build try before? How do you think about the corresponding update suggestion “Improve some dependency specifications in make rules”? https://github.com/elfring/Coccinelle-20160205/commit/9597598779f3604272621f6d42ead79eaafad2a8 Regards, Markus ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: Replacing suffix rules in make scripts?
> The OCaml makefile specifies an order in which the files should be > compiled in the project. If you are sidestepping that ordering, then you > are not doing things in the intended manner. I hope that some details were clarified to some degree in the meantime. > There is only an intent to support running make or make opt in the > subdirectories or at the root. Perhaps Thierry will put a new make file > that has some other properties (I don't know which ones) in the future. > But there is no intent to support anything else at the moment. I suggest to take also another look around the discussed build scripts. Example: https://github.com/coccinelle/coccinelle/blob/cd539cd12ad0674d5a8f4c33f6c21ae00aec8e57/globals/Makefile#L50 Now I wonder about the following test result for my current “developer rules”. elfring@Sonne:~/Projekte/Coccinelle/20160205/globals> make clean && LANG=C make --no-builtin-rules V=1 && make clean && LANG=C make --no-builtin-rules -j4 V=1 rm -f … /usr/bin/ocamldep -I ../commons -I /home/elfring/Projekte/Coccinelle/20160205/bundles/pcre/ *.mli *.ml > .depend … /usr/bin/ocamlc.opt -unsafe -I ../commons -I /home/elfring/Projekte/Coccinelle/20160205/bundles/pcre/ -c flag.ml … /usr/bin/ocamlc.opt -unsafe -I ../commons -I /home/elfring/Projekte/Coccinelle/20160205/bundles/pcre/ -a -o globals.cma config.cmo flag.cmo regexp_pcre.cmo regexp.cmo iteration.cmo rm -f … /usr/bin/ocamldep -I ../commons -I /home/elfring/Projekte/Coccinelle/20160205/bundles/pcre/ *.mli *.ml > .depend … /usr/bin/ocamlc.opt -unsafe -I ../commons -I /home/elfring/Projekte/Coccinelle/20160205/bundles/pcre/ -c flag.ml … /usr/bin/ocamlc.opt -unsafe -I ../commons -I /home/elfring/Projekte/Coccinelle/20160205/bundles/pcre/ -c iteration.mli make: *** No rule to make target 'flag.cmi', needed by 'iteration.cmo'. Stop. … Why do I get such an error message for a parallel software build attempt when the specified dependencies worked as expected for the non-parallel build try before? Do you get any more ideas on how to explain such a test surprise? Regards, Markus ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: How to avoid the double execution of a make info call?
> If the makefile, or any of the files it includes, is updated by make > then make will re-execute itself and re-read all the makefiles from > scratch so it can see the updated content. When that happens, > obviously all the $(info ...) functions etc. will be re-expanded. Thanks for your repeated explanation. I'm sorry that I needed also another reminder. >> Or is it more desirable to deactivate the message “modules without >> mli files”? > > That's up to you; we can't decide for you. I am curious on how the support for consistent interface descriptions will evolve also for a programming language like OCaml. > If you decide you want to test whether or not this makefile is being > read due to the initial parse versus a re-execution of make you can > check the MAKE_RESTARTS variable: I have added this one. > Or you can just remove these $(info ...) statements. I am also curious on how other involved contributors will react to a pull request for recent software developments around the discussed build scripts. Regards, Markus ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: How to avoid the double execution of a make info call?
> Your Makefile looks likely to rebuild .depend, a file that it includes. > So, like Paul said: >>> or else make has re-executed itself >>> because this makefile or one of its >>> included files was rebuilt; Yes. - This can happen at the moment. > I wonder about the information “Re-executing[1]: make --no-builtin-rules -d” > there. > Unfortunately, I do not recognise so far which rule triggers this action. > > Which condition checks should I add? I imagine that my extra information display might be useful. Should I check anyhow then if the included dependency file was modified? Or is it more desirable to deactivate the message “modules without mli files”? Regards, Markus ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: How to avoid the double execution of a make info call?
> If you are seeing these values printed multiple times it means that the > makefile is being parsed multiple times. How can such additional parsing be clarified further? Will it help to display extra data from special make variables? > Why that might be, we can't determine from the information provided. I hope then that the following test data can be better explained. elfring@Sonne:~/Projekte/Coccinelle/20160205/parsing_c> make clean … LEVEL 0: FLAGS : 7 modules without mli files rm -f parsing_c.top … elfring@Sonne:~/Projekte/Coccinelle/20160205/parsing_c> LANG=C make --no-builtin-rules V=1 … LEVEL 0: FLAGS r: 8 modules without mli files /usr/bin/ocamllex.opt lexer_c.mll 549 states, 26116 transitions, table size 107758 bytes 15973 additional bytes used for bindings /usr/bin/ocamlyacc -v parser_c.mly /usr/bin/ocamldep -one-line -I ../commons -I ../commons/ocamlextra -I ../globals -I ../parsing_cocci *.mli *.ml > .depend … LEVEL 0: FLAGS r: 7 modules without mli files /usr/bin/ocamlc.opt -unsafe -I ../commons -I ../commons/ocamlextra -I ../globals -I ../parsing_cocci -c token_annot.mli … It seems that a recursive make is not performed in this use case. https://github.com/coccinelle/coccinelle/blob/07d5a85c2234521238820f7bf3b0ec304839c414/parsing_c/Makefile#L67 > As always, if you can't figure out what's going on running "make -d" > will help you more than we can. I wonder about the information “Re-executing[1]: make --no-builtin-rules -d” there. Unfortunately, I do not recognise so far which rule triggers this action. Which condition checks should I add? Regards, Markus ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: How to avoid the double execution of a make info call?
> modules_without_mli::=$(filter-out $(basename $(mli_sources)),$(basename > $(ml_sources))) > $(info modules_without_mli) > $(info $(sort $(modules_without_mli))) It seems that I got this design approach working to some degree by the addition of a few calls of make functions. I am not completely pleased with this intermediate work result. Now I observe that the test output occurs twice while I am trying the evolving script “~/Projekte/Coccinelle/20160205/parsing_c/Makefile” out. I expect that such data processing should be performed only once in this subdirectory. So I am curios to find out how I can avoid the unwanted double execution there. Regards, Markus ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: Checking application of dependencies from make rules without recipes
> Your claim was that either explicit rules should come before implicit rules > or vice versa. You are more free with your choices for placement of explicit rules in make scripts (because you can not and will therefore not cancel them). You should be careful then which of your explicit rules will be configured for the use as the default make rule. This distinction has got important consequences on how the different make rule types can and should be ordered relative to each other. > > I still find that to be one of your clearer sentences, even if it left us > wondering > what these important consequences are. Bigger freedom for positioning of explicit make rules. Have you got any preferences there? > Still, if all you were saying was that implicit rule ordering can be > important, > then I think we can happily close this thread. The clarification for the mentioned distinction might be finished in principle for now. But I guess that the discussion will be continued around the handling of “optional” input files (and their effect on the usage of static pattern rules and suffix rules). Regards, Markus ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: Checking application of dependencies from make rules without recipes
> I don't think that's relevant: a graph's topology isn't dependent > on the order in which its arcs are enumerated. I have tried to stress the construction sequence. > When you submit a test case that demonstrates a bug, I stumbled on two special cases depending on your view of the corresponding software situation. I am unsure if any more evolution will happen there. > then I'm sure someone will look at it. Would you like to take another look at a detail from my clarification request on the topic “Checking file generation for a test script”? https://lists.gnu.org/archive/html/bug-make/2017-06/msg00126.html > Expressions calculating prerequisites are a notable case, causing "variables > before rules" > to be one guideline for writing makefiles. Thanks for your information. > You claimed that the inability to cancel an explicit rule meant something > important I got an acknowledgement that the handling of explicit rules is generally different in comparison to implicit rules in specific cases. > about whether explicit rules or implicit rules should come first. > Paul prompted you to explain what that was, but you haven't. I thought that I have tried this. Now again the documented detail for an ordering constraint: Implicit make rules can be cancelled. So it is expected that something did exist to be switched off on concrete demand. Regards, Markus ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: [bug #51269] Reusing data from targets for prerequisites
> It's important to realize that even though the unfortunate choice of > naming implies otherwise, static pattern rules are in all ways > _explicit_ rules. They are not and should never be considered > implicit rules. Thanks for your explanation. This terminology can occasionally trigger corresponding understanding difficulties. > A static pattern rule is merely a shorthand for writing a bunch of > explicit rules. I am struggling with advanced usage of placeholder indicators to extract further data from such targets. > OBJS = foo.o bar.o biz.o baz.o > > $(OBJS): %.o : %.c > $(CC) $(CFLAGS) -c -o $@ $< Can the description for this construct be improved a bit more in the documentation? I hope that I can avoid to introduce another make rule generator for some use cases. Regards, Markus ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: Checking application of dependencies from make rules without recipes
> I don't see why it makes a difference in how rule types should be ordered A dependency graph is constructed by make scripts. > (I assume that by "ordered" you mean which rules are defined > first in the makefile). Yes. The possibility is supported to cancel implicit rules (which refer to targets that contain the placeholder “%”) on demand during a specific construction sequence. > Only if an explicit rule does not exist will an implicit rule search > be performed. I got the impression that this general handling might need another look for one of my test cases. > Makefiles are not like the shell or other interpreted languages where > commands are executed as they are parsed. They correspond to a special category of programming languages. I find that there are also functions supported which are immediately executed. Regards, Markus ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: Checking application of dependencies from make rules without recipes
>> How much can make scripts help to manage (optional) programming >> interface descriptions? > > Are you asking, how can a makefile represent a prerequisite which may be > built, but is not required to be built and it's not an error if it > cannot be built? How do you think about the following approach to determine which source files can be tried to compile even when interface descriptions are (temporarily or intentionally) not available? source_dir::=$(abs_top_srcdir)/$(TARGET)/ ml_sources::=$(addprefix $(source_dir),$(SRC)) mli_sources::=$(wildcard $(source_dir)*.mli) modules_without_mli::=$(filter-out $(basename $(mli_sources)),$(basename $(ml_sources))) $(info modules_without_mli) $(info $(sort $(modules_without_mli))) $(modules_without_mli): %.cmo: %.ml $$(OCAMLC_CMD) -c $$< Unfortunately, corresponding messages seem to indicate that my knowledge and understanding is still incomplete for the application of static pattern rules. elfring@Sonne:~/Projekte/Coccinelle/20160205/parsing_c> LANG=C make --no-builtin-rules V=1 modules_without_mli /home/elfring/Projekte/Coccinelle/20160205/parsing_c/ast_c … … Makefile:145: target '/home/elfring/Projekte/Coccinelle/20160205/parsing_c/ast_c' doesn't match the target pattern … Makefile:149: warning: overriding recipe for target '/home/elfring/Projekte/Coccinelle/20160205/parsing_c/token_c' Makefile:146: warning: ignoring old recipe for target '/home/elfring/Projekte/Coccinelle/20160205/parsing_c/token_c' … Regards, Markus ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: [bug #51269] Reusing data from targets for prerequisites
>> * Can it eventually happen that dependencies will not be resolved if >> target names do not contain the percent character? > > I can't remember, but a simple experiment should answer that ! I guess that it will be noticed once more that implicit rules will work in an other way than static pattern rules. Does the support for placeholder syntax differ there just by the software design which was intended? >> * Can another link help in the section “How patterns match” of the >> documentation? > > Be specific: to which page (give its URL) do you consider adding a link, > to where, using which part of the page's text as anchor. https://www.gnu.org/software/make/manual/html_node/Pattern-Match.html Are you interested in a better connection between manual sections? >> I am also interested in data processing for several fields from >> targets (and not only a single “%” as the usual placeholder). Will it >> make sense to capture any more data into corresponding variables? > > There's little point going beyond basic substitution unless we go for > full regex-based matching and replacement, > OBJ = $(SRC:s!(.*)/srcroot/(.*)\.c!$1/buildroot/$2.o!) > or similar, and I doubt make shall aim for that any time soon. How are the chances to support regular expressions by the make programming language without invoking external shell commands? Would you like to use named capturing groups? Regards, Markus ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: Checking application of dependencies from make rules without recipes
> This applies only to implicit rules, not explicit rules. > >> Can an explicit make rule also be cancelled anyhow? > > No. Thanks for this clarification. This distinction has got important consequences on how the different make rule types can and should be ordered relative to each other. Would you like to point any further details out for the combination (or distribution) of make rules with the same targets? Regards, Markus ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: [bug #51269] Reusing data from targets for prerequisites
>> Would it make sense to allow that another special syntax will denote >> additional placeholders (similar to the handling of the percent character)? > > Take a look at > https://www.gnu.org/software/make/manual/html_node/Static-Usage.html > > It may enable what you want already. This functionality fits to my advanced software build needs partly. I find the section “Static pattern rules versus implicit rules” also interesting in the documentation. https://www.gnu.org/software/make/manual/html_node/Static-versus-Implicit.html But it seems that another important detail is missing in this comparison. * How do you choose the desired pre-requirements? * Which file names would you specify for the parameter “prereq-patterns”? Regards, Markus ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: [bug #51269] Reusing data from targets for prerequisites
>> Would it make sense to allow that another special syntax will denote >> additional placeholders (similar to the handling of the percent character)? > > Take a look at > https://www.gnu.org/software/make/manual/html_node/Static-Usage.html Thanks for your link. > It may enable what you want already. The provided example looks promising. bigoutput littleoutput : %output : text.g generate text.g -$* > $@ I got another software development concern for such an use case. * Can it eventually happen that dependencies will not be resolved if target names do not contain the percent character? * Should the determination of prefixes and suffixes just work here? * Can another link help in the section “How patterns match” of the documentation? > Otherwise, give a specific example of what it is you can't do > with the existing syntax. I am also interested in data processing for several fields from targets (and not only a single “%” as the usual placeholder). Will it make sense to capture any more data into corresponding variables? Regards, Markus ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: Checking file generation for a test script
>> My pattern example does not work with the current make software in the >> way I hoped would be occasionally convenient. > > No it does not. Did you read my entire previous answer? Yes. - I replied to it twice with different information. Regards, Markus ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: Checking application of dependencies from make rules without recipes
>> Would you like to add any more advice to this aspect? > > Only if you have a specific question. I have got another software development concern. The section “Canceling implicit rules” of the documentation mentions something about the omission of recipes (or do they become “empty”?). https://www.gnu.org/software/make/manual/html_node/Canceling-Rules.html The wording “… override a built-in implicit rule (or one you have defined yourself) …” might need another clarification. Does the distinction with explicit make rules result in special consequences here? https://www.gnu.org/software/make/manual/html_node/Makefile-Contents.html#index-explicit-rule_002c-definition-of Can an explicit make rule also be cancelled anyhow? Regards, Markus ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: Checking file generation for a test script
> It would have been generated if you would have called make with a command > like: elfring@Sonne:~/Projekte/Bau> LANG=C make --no-builtin-rules -f ../rule-check2.make MOTD.log make: *** No rule to make target 'MOTD.log'. Stop. > LANG=C make --no-builtin-rules -f ../rule-check2.make MOTD.log > and if you had a rule to build MOTD.log My pattern example does not work with the current make software in the way I hoped would be occasionally convenient. Regards, Markus ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: Checking application of dependencies from make rules without recipes
>> Does such a feedback indicate that would like to look also into the >> corresponding development repository? > > No. At least I don't have time to review an entire development environment. Such a choice is also fine. > If you post the set of rules responsible for creating those files > in the previous makefile configuration we can probably tell > you why they behave differently, if you don't understand it. But I find that such general feedback indicates also one of the usual challenges for handling advanced (free) software support. Another code example might help to achieve a bit more progress in the desired clarification of implementation details. But it could have been reduced too much with the consequence to look at other (unwanted) side effects. > I didn't recognize any other requests for clarification among your comments. This is a pity. I hoped somehow that your trained eyes might spot more issues from software design patterns around the mentioned details. >> How much can make scripts help to manage (optional) programming >> interface descriptions? > > Are you asking, how can a makefile represent a prerequisite which > may be built, but is not required to be built and it's not an error > if it cannot be built? Yes. > I can only assume you are referring to this pattern rule: > >> %.cmo: %.ml %.cmi >> $(o_compilation) '$<' > $@ Yes. And the original suffix rule: https://github.com/coccinelle/coccinelle/blob/1c4474890c065711b16c7d02089a56da5a07e13a/Makefile#L512 … .mli.cmi: $(OCAMLC_CMD) -c $< … > and you're saying that if the .cmi file cannot be created, > that make should not consider this an error. There are cases where this can and should happen. I guess that this aspect affects an organisational issue. > It is generally not possible to represent this behavior in makefile syntax. I find this information only partly appropriate. > The best you can do is remove the %.cmi from the pattern rule This suggestion is only one possibility. > and declare explicit prerequisite rules for those targets that you'd like to > have a .cmi file prerequisite. You referenced my approach for the conversion of a specific suffix rule to a similar pattern rule. The Coccinelle software can be successfully built by the published make scripts with the effect of special limitations (and the usual remaining update candidates). I am trying to adjust some details there. But I reached a point after some tweaks where I began to stumble on unexpected error messages because of the published dependency specifications. I wondered about them for a while. It took also another while until the handling of optional software data was identified as a corresponding big factor for the discussed use case. It is a recurring topic to clarify circumstances under which something can be really optional (or not). The development status is evolving as usual also for the mentioned concrete software. Now I imagine that there is a need to decide on this design detail once more in an automatic way. I am looking for further solutions besides the approaches which I applied for a few subdirectories in one of my own development repositories. How do you think about to extend information sources like the manual section “Generating prerequisites automatically”? https://www.gnu.org/software/make/manual/html_node/Automatic-Prerequisites.html Regards, Markus ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: Checking application of dependencies from make rules without recipes
>> It seems then that the original make scripts could treat more >> interface descriptions as optional somehow. > > Without know what your original makefiles said I can't comment on that. Does such a feedback indicate that would like to look also into the corresponding development repository? Example: https://github.com/coccinelle/coccinelle/blob/07d5a85c2234521238820f7bf3b0ec304839c414/parsing_c/Makefile#L1 >> Would you like to add any more advice to this aspect? > > Only if you have a specific question. I indicated further clarification opportunities already. > I've stated the algorithm make is using; This description is generally fine. > I don't know what else to say. How much can make scripts help to manage (optional) programming interface descriptions? > Certainly the more files you have the more output is generated. I guess that this fact can grow to scalability problem. > So it shouldn't be a problem to search the output for the target that is > not being built correctly then examine the steps make follows to try to > build it. It's the same set of steps whether it's the only target in > the makefile, or one of a thousand targets in the makefile. I imagine that there are related challenges and software development tools to consider for the safer handling of dependency outlines. Regards, Markus ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: Checking application of dependencies from make rules without recipes
> Could you provide examples of the Makefiles rules you tried? A rough approximation for further discussion: i_compilation?=echo o_compilation?=echo a_generation?=$(o_compilation) 'Checked modules: ' parsing_c.cma: ast_c.cmo token_annot.cmo $(a_generation) '$<' > $@ %.cmi: %.mli $(i_compilation) '$<' > $@ %.cmo: %.ml %.cmi $(o_compilation) '$<' > $@ includes.cmi: ast_c.cmo elfring@Sonne:~/Projekte/Coccinelle/20160205/parsing_c> LANG=C make --no-builtin-rules -f parsing-rule-check1.make make: *** No rule to make target 'ast_c.cmo', needed by 'parsing_c.cma'. Stop. How do you think about such a test result? See also the discussion “Replacing suffix rules in make scripts?” for relevant background information: https://systeme.lip6.fr/pipermail/cocci/2017-June/004181.html Regards, Markus ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: Checking file generation for a test script
> It would have been generated if you would have called make with a > command like: > LANG=C make --no-builtin-rules -f ../rule-check2.make MOTD.log I hoped that I do not need to specify another file name for such command variant just to test a special default setting. > and if you had a rule to build MOTD.log I thought that the shown pattern rule should be sufficient. > In the documentation at > https://www.gnu.org/software/make/manual/html_node/Rules.html it says: > > "a target that defines a pattern rule has no effect on the default goal." Thanks for your link. My expectations for the test example did not fit to this information then. Regards, Markus ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Checking file generation for a test script
Hello,
I have tried the following small script out together with the program
“GNU Make 4.2.1-1.7” on my openSUSE Tumbleweed system.
my_compilation?=echo
my_preparation?=cat
footer?=MOTD.txt
prepared_file?=MOTD.in
MOTD%.log: MOTD%.txt MOTD%.in
${my_compilation} "$<: $$(cat ${prepared_file} ${footer})" > $@
${prepared_file}: MOTD.draft
${my_preparation} $< > $@
elfring@Sonne:~/Projekte/Bau> my_message=MOTD.log && rm -f ${my_message}; touch
MOTD.draft MOTD.txt && LANG=C make --no-builtin-rules -f ../rule-check2.make &&
LANG=C ls -l MOTD.in ${my_message}
cat MOTD.draft > MOTD.in
ls: cannot access 'MOTD.log': No such file or directory
-rw-r--r-- 1 elfring users 6 Jun 18 12:56 MOTD.in
Now I wonder why the log file is not generated by this build approach at the
end.
Where is my knowledge and understanding incomplete for this software situation?
Regards,
Markus
___
Bug-make mailing list
Bug-make@gnu.org
https://lists.gnu.org/mailman/listinfo/bug-make
Re: Checking alternatives for a dynamic make rule construction
> Sure, it could be made clear in the documentation that either of the
> sides of the "=" could be empty.
In how many source files should corresponding information be integrated
to make these special cases better known for substitution references?
> However, there are plenty of ways to do this same thing:
>
> In your situation where there's only one word in the ${1} variable,
> "${1:=.cmo}" is the same as writing "${1}.cmo" which is even simpler and
> easier to understand.
>
> In a situation where there may be multiple words in the variable that
> you want to modify, you can consider using the "$(addsuffix .cmo,${1})"
> function which is arguably more clear than using substitution references
> and does the same thing as using an empty left-hand side value.
* Will another link help between documentation sections?
* Can a shorthand become relevant for such a function?
>> 2. Can the distinction between appending suffixes and replacing them become
>>occasionally more relevant for better software build characteristics?
>
> I don't know how to respond to this.
I imagine that the mentioned function call variants have got a different
impact on the run time behaviour.
How often do you stumble on the need to improve software build performance?
>> Another software extension:
>> How are the chances to assign aliases to numbered temporary variables?
>
> At this time I don't see the need to add such a thing. At the moment I
> don't see a reasonable way to extend the existing "call" syntax to
> provide aliases.
Would it make sense to configure aliases only from within (longer)
function implementations?
Regards,
Markus
___
Bug-make mailing list
Bug-make@gnu.org
https://lists.gnu.org/mailman/listinfo/bug-make
Re: Checking alternatives for a dynamic make rule construction
>> my_test_command?=cat
>>
>> define my_rule_demo=
>> name::=${1}
>> $${name:.in=.txt}: ${1}
>> $${my_test_command} $$< > $$@
>> endef
>>
>> $(eval $(call my_rule_demo,MOTD.in))
>
> Is there something wrong with it?
The answer depends on our views about approaches how customised
make functions should be safely written.
I sent it just as another follow-up in the hope to reduce a few
of our communication difficulties.
Can such tiny code examples help in this dialogue at all?
>> How would you like to clarify remaining details from other requests?
>
> I'm sorry but I've lost track of what issues have been resolved versus
> which are still outstanding. If you can provide an example and a
> specific question, we can attempt to respond.
I hope that there will no confusion occur with subjects from
my other recent discussion threads.
I am still curious about information I presented at 10:28.
http://lists.gnu.org/archive/html/bug-make/2017-06/msg00093.html
Repetition:
1. Will it make sense to extend any documentation for “substitution references”?
2. Can the distinction between appending suffixes and replacing them become
occasionally more relevant for better software build characteristics?
Another software extension:
How are the chances to assign aliases to numbered temporary variables?
Regards,
Markus
___
Bug-make mailing list
Bug-make@gnu.org
https://lists.gnu.org/mailman/listinfo/bug-make
Re: Checking alternatives for a dynamic make rule construction
>> define my_broken_rule=
>> name::=${1}
>> ${name:.in=.txt}: ${1}
>> $${my_test_command} $$< > $$@
>> endef
>>
>> $(eval $(call my_broken_rule,MOTD.in))
>
> Yes, this will not work.
Thanks for your quick answer.
It might also be expected that this simple example just can not work
as intended because of a special typo (one dollar sign missing).
Corrected example:
my_test_command?=cat
define my_rule_demo=
name::=${1}
$${name:.in=.txt}: ${1}
$${my_test_command} $$< > $$@
endef
$(eval $(call my_rule_demo,MOTD.in))
> Generally I recommend that people DO NOT try to create local variables
It can look a bit easier to understand when a numbered temporary
variable gets a reasonable alphabetic name as an alias.
> inside define'd makefile sections and instead just write out the
> reference you want each time.
I switched to an alphabetic variable name because of an unclear
aspect with “substitution references” and the impression that something
did not work as expected for the notation “${1}” at one place
while I was using the correct double dollar signs for the other variable.
> It's more verbose, but has many fewer side-effects.
Customised make functions trigger their own special development challenges
because of the risk to miscount dollar signs there.
How would you like to clarify remaining details from other requests?
Regards,
Markus
___
Bug-make mailing list
Bug-make@gnu.org
https://lists.gnu.org/mailman/listinfo/bug-make
Re: Checking alternatives for a dynamic make rule construction
> What Philip is saying (and I agree 100%) is that you provide the above
> example which is excellent and helpful and allows us to see exactly what
> you're doing. Thank you for that.
I was also curious in this case how the interest will evolve to clarify
the shown construction of make rules.
> But then, you ask questions which are abstract and require a lot of
> effort on our part to understand.
I can follow this concern to some degree.
> For example:
>
>> 2. The specification “$$(name)” is working. Would it make sense to avoid
>>such a variable reference there?
>>Unfortunately, the simple specification “$(1)” does not work in the way
>>I would expect for the selection of a single file as a dependency.
>>Would you like to explain this software behaviour?
>
> This seems to say that when you tried to use something different than
> the $$(name) reference above, it didn't work the way you expected.
This interpretation is correct.
> However, you provide no information on what the "something different"
> that you tried was, how it worked, or how you expected it to work.
It seems then I chose a wording from which you would not pick
the essential information up directly.
> In programming (and makefiles) everything is about the details.
I am used to this view as I am trying to be a professional software
developer for a while.
> The statement "I would expect for the selection of a single file as a
> dependency" is, at best, hard to understand (personally I didn't
> understand it).
So I try again to express my needs a bit more in make code.
Example MOTD.make:
my_test_command?=cat
define my_rule=
${1:.in=.txt}: ${1}
$${my_test_command} $$< > $$@
endef
$(eval $(call my_rule,MOTD.in))
elfring@Sonne:~/Projekte/Bau> rm MOTD.txt; echo 'Test!' > MOTD.in && make -f
../MOTD.make && cat MOTD.txt
cat MOTD.in > MOTD.txt
Test!
This result is fine. Please try the following approach out again.
Example MOTD2.make:
my_test_command?=cat
define my_broken_rule=
name::=${1}
${name:.in=.txt}: ${1}
$${my_test_command} $$< > $$@
endef
$(eval $(call my_broken_rule,MOTD.in))
elfring@Sonne:~/Projekte/Bau> rm MOTD.txt && LANG=C make -f
~/Projekte/MOTD2.make
make: *** No targets. Stop.
How do you think about this test result?
Regards,
Markus
___
Bug-make mailing list
Bug-make@gnu.org
https://lists.gnu.org/mailman/listinfo/bug-make
Re: Checking application of dependencies from make rules without recipes
> Could you provide examples of the Makefiles rules you tried? It depends on the detail level you prefer and my tendency to increase corresponding efforts only incrementally. > It's very difficult to provide feedback without complete information. Would you like to add any advice to a discussion around a topic like “Replacing suffix rules in make scripts?”? https://systeme.lip6.fr/pipermail/cocci/2017-June/004181.html > Does ocamldep generate output suitable for inclusion by make? It is expected that this tool “Dependency generator for OCaml” provides appropriate dependency specifications. Regards, Markus ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: Checking alternatives for a dynamic make rule construction
> Using ::= in a makefile which is already dependent on GNU make is, IMO,
> pointless.
I am trying to use portable make specifications to some degree.
But it seems to be challenging to avoid the usage of all the nice
functionality which is provided as extensions also by this software
development tool.
> You can substitution references with ${1} directly, ala ${1:.ml=.cmo}
Thanks for this advice.
> I *think* you're asking whether rule_pair could take just the bare
> basename (ala "commands" instead of "commands.ml").
Your view is appropriate.
> If so, then sure: a substitution reference could have an empty replacement
> part:
> ${1:=.cmo}
>
> will expand to "commands.cmo" if $1 is "commands"
I was not used to this possibility.
* Would it make sense to extend any documentation for such an use case?
* Can the distinction between appending suffixes and replacing them
become occasionally more relevant for better software build characteristics?
> I started to write a long explanation of how $(eval) works but then
> realized it didn't explain when $(1) didn't work for you.
Interesting …
> So: nope, can't explain the error message you saw, because it makes *no
> sense* to me why it didn't work based on what you wrote.
Does this kind of feedback mean also that there are specific details
worth for further clarifications?
> In case the theme of my replies isn't 100% clear: it would be a *lot*
> easier to assist you if you included the complete input you tried,
> the complete output you got, and *what you expected/desired*.
I fiddle with bigger make scripts once more where I try to exclude
some code at the beginning of another clarification attempt.
I would like to avoid efforts for a dedicated test script (for a moment).
> Or we can guess,
This can happen.
> and then give up and go away.
This is another usual possibility.
But I find your constructive feedback very promising and helpful.
I am curious on how such a dialogue will be continued.
Regards,
Markus
___
Bug-make mailing list
Bug-make@gnu.org
https://lists.gnu.org/mailman/listinfo/bug-make
Re: Checking software build tries for “commands.cmo”
> I believe the point of Martin's rhetorical query is "why should make treat > the failure of *this* pattern rule any different than the failure of the > many other pattern rules that would have permitted this compilation > to complete?" It seems that I need to try harder for the desired clarification of consequences when a specific source file (like an interface description file for the programming language “OCaml”) can be occasionally treated as an optional item in some software areas. There will not be any make rule specified for a file type which was intentionally left out for a specific source item. The general challenge is to find out for how long something can be omitted (while its use can be recommended in several situations). Regards, Markus ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: Checking software build tries for “commands.cmo”
>> I needed another moment to become really aware that this software module >> is a bit special. It seems that it is intended that it will be compiled >> without a corresponding interface description file (suffix “mli”). > > Well, I'm totally lost. Here's why: > > You told make ".cmo files can be built from *THE COMBINATION OF* the > matching .cmi and .ml files, but *only* if both the files exist or can be > built via some other rule". That is what this rule says: > > %.cmo: %.ml %.cmi >$(OCAMLC_CMD) -c $< Your repetition is correct according to the development situation where I started another try to update build scripts for an other evolving software. > Now you're saying that commands.cmo could be built from *just* > commands.cmi without commands.ml! My previous attempt was incomplete for the description of the dependency chain “(mli ⇒ cmi ⇒)? ml ⇒ cmo”. I suggest to take another look. The OCaml programming language is also working with interface description files. These should be compiled when they are available at all. But they can be optional. I added a corresponding make rule then. commands.cmo: commands.ml $(OCAMLC_CMD) -c $< This pattern corresponds to a general development challenge: For which software modules can you depend on interface descriptions in a safe way? Regards, Markus ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Checking application of dependencies from make rules without recipes
Hello, I got some special software development experiences by the usage of the program “GNU Make 4.2.1-1.7” on my openSUSE Tumbleweed system. The functionality “Generating prerequisites automatically” is desribed in the manual. I would like to check again how good this works at the moment. I am trying also a software out where such build dependencies are provided by the tool “ocamldep”. Unfortunately, I observe still broken build tries because of missing files which should have been built before. I started to “fix” this for my needs by the addition of small make functions which construct extra rules. But it seems that such special stuff should usually not be needed according to the documentation. I would appreciate further advice for this issue. Are there any difficulties (or open issues) to consider for the handling of make rules with empty recipes? Regards, Markus ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: Checking software build tries for “commands.cmo”
> Did you tell make, then, to disable all of its default rules for creating > commands.mli or, > for that matter, commands.cmo, by checking them out of version control? Not directly so far. - The source file “commands.mli” is not needed for the generation of the object module “commands.cmo” in the final use case. But I used make rule specifications for a moment to express the simple dependency chain “mli ⇒ cmi ⇒ ml ⇒ cmo” before I became more aware about related software constraints. I would have found an error message like “The target “commands.cmo” could not be built because the specified dependency “commands.mli” did not exist.” nicer in a specific development situation. Regards, Markus ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Checking alternatives for a dynamic make rule construction
Hello, I have constructed the following small script to be executed by the program “GNU Make 4.2.1-1.7” on my openSUSE Tumbleweed system. define rule_pair = name::=$(1) $$(name:.ml=.cmo): $$(name) $$(OCAMLC_CMD) -c $$< $$(name:.ml=.cmx): $$(name) $$(OCAMLOPT_CMD) -c $$< endef $(foreach x,$(SRC_without_mli),$(eval $(call rule_pair,$(x I guess that the functionality from this approach is similar to the usage of two specific suffix rules. But I would like to distinguish the involved software dependencies better. I imagine that it would be nice to improve the shown implementation details. 1. I am using the extra variable “name” just for the reuse of the make functionality “substitution references”. I would prefer to append desired file name suffixes directly instead of replacing a previous one. Can the passed text list become a bit smaller eventually? 2. The specification “$$(name)” is working. Would it make sense to avoid such a variable reference there? Unfortunately, the simple specification “$(1)” does not work in the way I would expect for the selection of a single file as a dependency. Would you like to explain this software behaviour? Regards, Markus ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: Checking software build tries for “commands.cmo”
>> I have noticed a moment ago that an interface description file was missing >> somehow for the OCaml source file in this compilation attempt. > ... >> I wonder then that the make tool did not give me a direct clue for a failed >> software dependency as I am used to in other cases. > Would you have it complain that commands.mli,v, RCS/commands.mli,v, > RCS/commands.mli, s.commands.mli and SCCS/s.commands.mli were missing? Not really in this use case. > The existence of any of those would have let make build commands.mli, hence > commands.cmo. This interface description file could also be generated by a command variant of the OCaml compiler. But it seems that it is finally not needed for the build scripts which I am trying to update. > Can you come up with a specification for finding which of the possible > missing dependencies is the one that you actually care about? Can an error message like “The target “commands.cmo” could not be built because the specified dependency “commands.mli” did not exist.” be more appropriate instead of the message “make: Nothing to be done for 'commands.cmo'.”? Regards, Markus ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: Checking software build tries for “commands.cmo”
> (I would guess that most of the people on the bug-make mailing list have > never worked with ocaml and therefore don't know what needs to be done to > build it.) Yesterday I realised that one source source file needed a special handling in the mentioned subdirectory. > The snippet of Makefile you provided didn't include any rules for > building "commands.ml", so I don't understand why you expect make to build it. The suffix “ml” denotes a source file in this use case. I needed another moment to become really aware that this software module is a bit special. It seems that it is intended that it will be compiled without a corresponding interface description file (suffix “mli”). Regards, Markus ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: Checking software build tries for “commands.cmo”
> elfring@Sonne:~/Projekte/Coccinelle/20160205/commons> LANG=C make -d > commands.cmo V=1 > … > No implicit rule found for 'commands.cmo'. > Finished prerequisites of target file 'commands.cmo'. > Must remake target 'commands.cmo'. > Successfully remade target file 'commands.cmo'. > make: Nothing to be done for 'commands.cmo'. > > > Now I wonder why I do not get the desired software generation results for > this test case. Why is the source file “commands.ml” not compiled again > in the way as the other one? I have noticed a moment ago that an interface description file was missing somehow for the OCaml source file in this compilation attempt. I have added the following extra rules as a special handling. commands.cmo: commands.ml $(OCAMLC_CMD) -c $< commands.cmx: commands.ml $(OCAMLOPT_CMD) -c $< I wonder then that the make tool did not give me a direct clue for a failed software dependency as I am used to in other cases. Regards, Markus ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Checking software build tries for “commands.cmo”
Hello, I am testing the following pattern rules with the program “GNU Make 4.2.1-1.7” on my openSUSE Tumbleweed system as I would like to adjust some areas in affected make scripts for another evolving software tool. … %.cmi: %.mli $(OCAMLC_CMD) -c $< %.cmo: %.ml %.cmi $(OCAMLC_CMD) -c $< %.cmx: %.ml %.cmi $(OCAMLOPT_CMD) -c $< … I have tried a specific command out. elfring@Sonne:~/Projekte/Coccinelle/20160205/commons> LANG=C make -d common.cmo V=1 … Must remake target 'common.cmo'. /usr/bin/ocamlc.opt … … Removing child 0x557e31ae64e0 PID 2693 from chain. Successfully remade target file 'common.cmo'. This test showed the expected results. I have tried another command variant out. elfring@Sonne:~/Projekte/Coccinelle/20160205/commons> LANG=C make -d commands.cmo V=1 … No implicit rule found for 'commands.cmo'. Finished prerequisites of target file 'commands.cmo'. Must remake target 'commands.cmo'. Successfully remade target file 'commands.cmo'. make: Nothing to be done for 'commands.cmo'. Now I wonder why I do not get the desired software generation results for this test case. Why is the source file “commands.ml” not compiled again in the way as the other one? I am curious for your advices to improve the involved system dependencies. Regards, Markus ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: Management for relative paths in global build parameters
> Read up on the differences between builddir and srcdir, current and top_, > relative and abs_. > These are all preset output variables in Autoconf. Thanks for your reminder. I submitted a similar clarification request also for this software area. “Management for relative paths in global build system settings” http://lists.gnu.org/archive/html/autoconf/2017-06/msg3.html I am curious to know which function (or macro) determines the values for the “absolute” variants of these configuration variables. > Top and subprojects each have their own separate build system. I am contributing a bit also to a special tool which deals with software bundles to some degree. There are various software development challenges to consider, aren't there? Regards, Markus ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: Management for relative paths in global build parameters
> Relative paths are fine when including makefiles from sub-directories. > What matters is that targets use *consistent* paths. Thanks for your feedback. I would like to clarify dependencies for other build system details a bit more. > This assumes your make system is non-recursive, of course. I am more interested in consequences around recursive make scripts. It seems to be simple to ensure access to an included file from a well-known level in the directory hierarchy. But how do you think about the usual case that the build configuration contains also a bit of input data where relative paths are provided. How would you try to construct the desired referential integrity for involved subdirectories then? Regards, Markus ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Management for relative paths in global build parameters
Hello, Software development projects can grow in the way that corresponding components will be stored in several subdirectories. Such a project organisation has got consequences on the management of some dependencies. Now I am looking for advanced support possibilities by the make software once more. Components in these subdirectories will usually need some information also from the global build system settings. A portion of the involved configuration variables will eventually store relative paths. I imagine that desired data processing for them can trigger special development challenges then. How do you prefer to achieve consistent and safe access for paths across the directory hierarchy? I looked at the possibility to convert such data into absolute paths (by the function “abspath” for example). But make variables do not directly support higher level data structures for the transformation of passed build parameters in the way I would find convenient. How portable can extensions become for the provided function library? Regards, Markus ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: Fine-tuning for parallel building with GNU make
> Visiting folders non PARALLEL while compiling itself inside of the folders is > IN PARALLEL. How do you think about to traverse the affected directory hierarchy by a dedicated make recipe? * Have you got any expectations about the amount of parallel work which should be triggered by scripts in subdirectories? * Did you look at any possibilities around a variable like "MAKEFLAGS"? https://www.gnu.org/software/make/manual/html_node/Options_002fRecursion.html Regards, Markus ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: Parallel build consequences from GNU make
> I tried to use the following command for parallel build but it is not working. > *Make **–**j 7 **–**f makefile.mak build* I find your impression strange. > When I execute the above command, I can see that the GNU make is utilizing > 40%-60% of CPU. Which tools did you use to check the run time situation? > When I execute the same command using *clearmake**(clearmake **–**j 7 > makefile.mak build)***process, Do you suggest a software comparison with a tool like the ClearCase build utility? > I can see that it is always utilizing 80-100% of CPU. Are there any more details to inspect? > Which informs that the GNU make process is not using the parallelism. Would you like to reconsider your conclusion once more? Do you distinguish how many computation resources are needed by the build tool itself in comparison what is really required for the execution of commands from make recipes? Regards, Markus ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: Parallel make with distributed systems
I have nothing against doing more than that in theory, but before I'd agree to add something complex I'd need to understand where the existing method fails, and the new method would provide significant benefits. I guess that there are some software development challenges around the specification which tasks should be performed on a local computer and which should be executed in a more powerful remote build environment. The corresponding work forwarding will also need some preparation steps. There are more constraints to consider for tasks besides elements like distributed compilation. Would you like to make the build tool capable for the automatic selection of the preferred data processing system? Regards, Markus ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: Parallel make
GNU make has no built-in capability to use multiple machines: How are the chances to integrate additional job submission systems? conceptually it may be a straightforward extension but the effort needed to communicate between multiple systems over a network, send and receive results reliably, kill jobs when someone stops the main make process with ^C, etc. is FAR out of GNU make's current wheelhouse. Would you like to reuse any software from other approaches like Makeflow? http://www.cse.nd.edu/~ccl/software/makeflow/ Regards, Markus ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: Conditional recipe execution
But a quick check of your link above shows no comments or anything
in the makefile describing what it does,
Thanks for your look at my example.
and I don't really understand the purpose behind all these recipes
which consist of if-statements where the body of the recipe is made empty
if variables are not set.
It is my approach for recipe execution guards. I hope that such conditions
are sufficient to let the affected recipes only run for a single target
despite of the specification of several output files there.
Some targets will not usually work together with the make command-line
option -j if a corresponding value was passed that is greater than one
for such a parameter.
I will also mention that I see you using various bash-specific shell
features (e.g., 'XY=$${XY/|/ }'), but I don't see any setting of SHELL
= /bin/bash in your makefile; unless you do this then your makefiles
will fail on any system where /bin/sh (which is the default setting for
SHELL in make) is not bash;
Thanks for your reminder.
Are there any more programs which support text replacement within content
from shell variables directly?
Regards,
Markus
___
Bug-make mailing list
Bug-make@gnu.org
https://lists.gnu.org/mailman/listinfo/bug-make
Re: Challenges around the combination of functions call and eval
$(eval ...) is a (relatively) recently implemented, very advanced feature that exists only in GNU make: no other implementation of make has anything like it (that I'm aware of). Since which program version are the functions call and eval generally available? So you really just don't need $(eval ...), except in rare situations where you really do need it. Nothing you've shown us so far really needs it. Would you like to adjust your view a bit if you look at my working intermediate build scripts again? https://build.opensuse.org/package/view_file/home:elfring:semantic_patching:Deletion_of_checks_before_specific_function_calls/Deletion_of_checks_before_specific_function_calls_in_Linux_source_files/parallel-inc.make?expand=1 Regards, Markus ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: Conditional recipe execution
I hoped somehow that another level of data indirection could be avoided. To be honest, based on what I've seen, you could do with a few more levels of indirection. My knowledge around GNU make usage is evolving. The rules you've presented here are (to me) almost impossible to read. Interesting … Abstracting some of these longer and more complex contents into other variable assignments might help make the structure of the makefile more clear. I guess that this view can also become a matter of taste. I could develop a complete make function library to achieve a better design. Can it eventually become harder then to understand the relationships within a bigger build script collection? Would to like to point to any examples which demonstrate the kind of software abstraction level that you would prefer? It prints the generated recipe at least after the addition of the backslash you recommended, doesn't it? It prints the recipe, I needed this debug output for a moment. but cannot run the recipe, because the result of expanding the $(info ...) function is the empty string. I know that. I found that it would help a bit in our clarification of implementation details. Regards, Markus ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: Extensions for make file parsing?
As I've said, it is not possible for a single variable or function reference to span multiple logical lines. Are there any chances to lift this restriction for make functions? It's very unlikely. The entire implementation of the parser in make at its most basic level is completely line-oriented. Thanks for your explanation. It would be quite a bit of code change to change this behavior. Would it be generally acceptable to change the parser in the way that also function parameters which would like not to pass several logical lines in a single variable with multi-line content will be supported? Does the current software provide a function library for corresponding parsing of text components? Regards, Markus ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: Conditional recipe execution
You need to describe your situation. Use words, but with detail. I have adjusted my build scripts. Now I wonder about another error message for a recipe. parallel-inc.make:95: *** unterminated call to function 'info': missing ')'. Stop. Corresponding excerpt Lines 91 - 101: … $(RESULT_FUNCTIONS_WITH_PREFIX1_PARALLEL) \ $(RESULT_SQL_DATABASE_NOTIFICATION8): \ initialise_database \ $(RESULT_SQL_DATABASE_NOTIFICATION4) $(info $(if $(LET_RUN_DATA_EXPORT1_PARALLEL),\ $(eval LET_RUN_DATA_EXPORT1_PARALLEL::=)@$(RM) '$(RESULT_SQL_DATABASE_NOTIFICATION8)' $(DATABASE_CLI) -c \ '$(SQL_SELECT_START)_parallel1 where static = 0 $(SQL_SELECT_END)' \ '$(DATABASE_NAME)' \ '$(RESULT_FUNCTIONS_WITH_PREFIX_PARALLEL)' \ $(TOUCH) '$(RESULT_SQL_DATABASE_NOTIFICATION8)')) … How should such a test rule be improved? Regards, Markus ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: Conditional recipe execution
parallel-inc.make:95: *** unterminated call to function 'info': missing ')'. Stop. I think the message is pretty clear: you're missing the end parenthesis ) to the info function which starts at line 95: I read my make rule in the way that the closing parentheses is really not on the line 95. But it should be found on the line 101. …_NOTIFICATION8)')) You can see that you open both an $(info ... and $(if ... function here, and you don't close either of them (missing end-parens). The editor from the tool KDevelop 4.7.0-2.1 gives me an other indication for the affected file. Regards, Markus ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: Conditional recipe execution
The error doesn't mean that the closing parentheses is missing from line 95, I understand this information part. it means that the open parenthesis that it didn't match was the info function on line 95. I find this detail hard to believe. I hope that the current KDevelop editor does not misguide me. The parentheses pairs are found there as expected. Do you read the shown rule example eventually in the way that its recipe does not end with …_NOTIFICATION8)'))? If you want that then you need a backslash at the end of line 96, to continue the logical line: I find that a backslash would not be needed at this place if would like to keep the RM command on a separate logical line. $(info $(if $(LET_RUN_DATA_EXPORT1_PARALLEL),\ $(eval LET_RUN_DATA_EXPORT1_PARALLEL::=)@$(RM) '$(RESULT_SQL_DATABASE_NOTIFICATION8)' Make never looks across logical lines to complete variable or function so you have to combine them into a single logical line with backslashes. Do I expect more functionality here than what is actually provided so far? Regards, Markus ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: Conditional recipe execution
It appears that KDevelop's makefile editing mode doesn't properly handle backslash continuations. This tool might still have got few open issues around improved handling of the make file syntax. But I find that its visual feedback was appropriate for parentheses pairs at least. Do you read the shown rule example eventually in the way that its recipe does not end with …_NOTIFICATION8)'))? The _RULE_ does end with it, because the rule includes all the recipe lines that appear after the target. Thanks for your acknowledgement. But variable/function invocations cannot span different lines of a recipe. They all must be contained within the same logical line, as I mentioned before. My knowledge was incomplete on this detail. I did not get the information from the description (in the manual) for the used conditional function that it does not support the inclusion of several logical lines so far. I am sorry if did not pay enough attention on this aspect. I find that a backslash would not be needed at this place if would like to keep the RM command on a separate logical line. It is definitely needed. As I've said, it is not possible for a single variable or function reference to span multiple logical lines. Are there any chances to lift this restriction for make functions? If you want to do this you have to use define/endef to create a variable that contains newlines, then use the variable in the recipe; see example below. I hoped somehow that another level of data indirection could be avoided. I'm really not at all sure why you're using $(info ...) here... The error message looked a bit nicer than a complaint on a shorter if. it won't actually do anything if you do that. It prints the generated recipe at least after the addition of the backslash you recommended, doesn't it? It seems that I had just another inappropriate expectation for the handling of a line continuation here. I thank you very much for this quick clarification. Regards, Markus ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: Formatting for longer function parameters
Will the line continuation work also with the backslash character there? http://www.gnu.org/software/make/manual/html_node/Splitting-Lines.html I have found out that it works generally. But I do not like the documented behaviour at some places of my make scripts. … Outside of recipe lines, backslash/newlines are converted into a single space character. … Does this detail make the splitting of a logical line inappropriate occasionally? Regards, Markus ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: Conditional recipe execution
Would use cases around the standard command split be concrete enough for further clarification? https://en.wikipedia.org/wiki/Split_%28Unix%29 No. I am bit surprised that you find such a tool as an unclear example at the moment. I am using my Python script which I developed as a contribution for a discussion around the topic 'Splitting search results from a find -print0'. http://lists.gnu.org/archive/html/coreutils/2015-01/msg00023.html You need to describe your situation. Use words, but with detail. I would like to perform some file searches. The found files should be analysed by dedicated programs. A couple of programs which I use for corresponding data processing are still single-threaded. But I want to get all processor cores working on the search result. So I split it into smaller files which I can give to each core as a work unit. Is this approach just an ordinary parallel work flow? It sounds to me like you what you mean is you have a single recipe which generates multiple output files and you want those files to be prerequisites of other targets. Yes, exactly. There are two possibilities. The best case is that the generated files and the input file(s) are all related to each other through some aspect of the filename: I have found that there are some software development challenges to consider for safe parallel data processing. in that case you can use pattern rules and everything is simple. I noticed a situation where such a special case needs to be detected. A recipe was executed multiple times by the tool make for each passed target. So for example the bison parser generator takes a file foo.y and generates foo.tab.h and foo.tab.c: the names of the output files are related to the name of the input file through the foo prefix. This is another well-known use case. Now listing foo.tab.c or foo.tab.h as a prerequisite will cause this rule to be run one time, with foo.y as the prerequisite and make knows both files are generated by a single invocation. Are there any cases left over where the make tool will interpret a rule specification in the way that a single invocation is not sufficient? The other possibility is that the names of the generated files have no relationship to the names of the input files. There is a relationship of course. But make file pattern operators can be inappropriate in some cases of my build scripts. In this case you'll have to use a sentinel file: output list: .sentinel ; I would interpret this specification as a rule with an empty recipe. How does this approach help? Regards, Markus ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: Difficulties from the combination of functions call and eval
Try changing eval to info This introspection approach will not work in the moment when I stumble on the mentioned error messages. and then inspect the output to see if it makes sense as a makefile. The result should become components for recipes in my use case. Write small test makefiles that do small aspects of what you want and build them up step by step until you get the result you want when you introduce an error you will know exactly what change made it I can also try this approach once more. Most of all, don't dump the full complexity of your problem on other people because makefiles are brain twisting and hard to understand even when they are simplified. I hope that there are more experienced software developers around who can spot programming mistakes within make statements eventually a bit easier than me. Simplifying the example will help you and help us help you too. :) Should I explain the involved design patterns a bit more so that a common understanding can be achieved easier? Regards, Markus ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: Difficulties from the combination of functions call and eval
The result should become components for recipes in my use case. As described before, this is not right. $(eval ...) is used to internalize make syntax. The result of expanding $(eval ...) is the _empty string_, so putting it in a recipe results in NOTHING being added to the recipe. Thanks for your reminder. I am going restructure the affected build rules. Regards, Markus ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: Conditional recipe execution
* I need to split an input file into several smaller files. I don't see any way that $(eval ...) is needed or helpful for any of those things. Can a make variable be set by this function within a recipe? I imagine that conditional evaluation of recipes will be possible somehow, wont it? I don't know what you mean by this question; if you mean that a recipe should make a decision and do different things, then typically you would use _shell_ conditional statements for this: How should it be achieved that a recipe will only be executed for its first run despite of the detail that multiple targets were specified for a specific rule? There are other possibilities, but without a specific use-case we can't say which one(s) are appropriate. Would use cases around the standard command split be concrete enough for further clarification? https://en.wikipedia.org/wiki/Split_%28Unix%29 Regards, Markus ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: Difficulties from the combination of functions call and eval
Without knowing exactly what you mean by handling of rules with multiple output files I can't say for sure, but I think it's unlikely that $(eval ...) can help with this. A lot of rules can come into the situation to fiddle with multiple output files easily. Examples: * Specification of standard output files and standard error channels for shell commands * The database SQLite can create its storage file while also exporting data to another file. * I need to split an input file into several smaller files. Just to be clear, you cannot create new rules, or add to the targets or prerequisites of existing rules, using $(eval ...) _inside a recipe_. I'm sorry that I needed a bit longer to realise this fact. This means the ONLY time setting a variable from within a recipe is helpful is if the only place you need to use that variable is INSIDE some other recipe that has not been run yet. I imagine that conditional evaluation of recipes will be possible somehow, wont it? Regards, Markus ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Questionable line number in a message missing separator?
Hello, I can use some phony targets from my build script as expected. But the software tool make 4.1-2.2 does not like a specific place in this file at the moment. … parallel-inc.make:247: *** missing separator. Stop. The mentioned line has got a leading tab character for the beginning of a recipe. How can this syntax difficulty be resolved now? Regards, Markus ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Formatting for longer function parameters
Hello, Several parameters can be passed to user-defined make functions. http://www.gnu.org/software/make/manual/html_node/Syntax-of-Functions.html Some strings can become longer. So I see a need occasionally to distribute the parameter list over more lines for better build script readability. I would like to be especially careful with passing of no additional white-space characters at this place. Will the line continuation work also with the backslash character there? http://www.gnu.org/software/make/manual/html_node/Splitting-Lines.html Regards, Markus ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Escaping of dollar signs
Hello, The rule syntax description mentions an use case where it will be needed to work with four dollar signs. The description of the eval function points also out to be careful with the $ character. Can it happen that even more dollars are needed when several variables are called for evaluation in a recursive/nested way? Regards, Markus ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: Dynamic evaluation of build scripts
Make implementations are notorious for having lots of incompatible extensions. That's because the POSIX standard for make (which all implementations typically adhere to) is very limited in what it requires, so implementations have added their own features in addition to the standard, and they rarely are compatible with each other. I thank you very much for your detailed explanations. Regards, Markus ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: Dynamic evaluation of build scripts
GNU make is the only implementation of make that supports any GNU make functions, including $(eval ...), if that's what you mean. I hope that more software tools can cope with make file syntax and processing of corresponding GNU extensions. In fact, recursive variable expansion is about the only one of the techniques I discuss above which is widely implemented in other versions of make. I find it interesting which limitations from other approaches need to be also considered for the development of reasonably portable build scripts. No other make, that I know of, supports regenerating makefiles and re-exec'ing. And definitely no other make supports $(eval ...). Does GNU software provide some promising functionality here? Are there any chances that it will spread out to further implementations? The $(call ...) function is what's normally used to create a user-defined function that takes multiple arguments in GNU make... Would it help to mention explicitly that the built-in function eval takes script code as a single input parameter? Regards, Markus ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: Visualisation of make files as graphs
You can use gmake --print-database -f Makefile and then write a script to convert that into .dot format which gephi will load up for you. Do you know a few scripts (or other software tools) which provide such a data format conversion? Will it become easier to present and edit an overview of the involved build rule dependencies? Regards, Markus ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Visualisation of make files as graphs
Hello, Make files can grow to a size (with all their build rules and various script code) where it becomes harder to follow the involved data processing structures. Do you know any software tools which provide graphical visualisations for work flows that were developed with the make file syntax? Regards, Markus ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: Dynamic evaluation of build scripts
So you have in your toolbox $(shell) and $(eval). I am not familiar enough with the second make function. http://www.gnu.org/software/make/manual/html_node/Eval-Function.html I wrote some blog posts about eval and other metaprogramming techniques in make that you might find interesting: http://make.mad-scientist.net/category/metaprogramming/ How many software implementations support the eval function in make files? # Then you might use eval to generate rules dynamically: $(foreach N,$(NPROC),$(eval # YOUR DYNAMIC RULE MYTARGET_$(N): TABmytool --dosomething -o$@ ) ) How do you think about to mention in the manual how several parameters can be passed? When should parameters be used as further function arguments? Regards, Markus ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: Cooperation with distributed job processing systems
How do you think about to delegate such an implementation detail to a configurable job submission system? That could be done, but the current implementation of the jobserver software isn't really separable like that. It would be a nice thing to make a more abstract API which could be replaced with alternatives. Do you find information about the software Makeflow interesting for improved management of build work flows? http://www.cse.nd.edu/~ccl/software/makeflow/ Regards, Markus ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: Better description for define?
The define construct is already in the index. I am missing an explanation there for the aspect when the construct define … endef should be used instead of the other variants for variable assignments. http://www.gnu.org/software/make/manual/html_node/Reading-Makefiles.html#index-define_002c-expansion A corresponding implementation detail might not be as obvious here for me as for other make users. Regards, Markus ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: Better description for define?
Would it help a bit to mention this information explicitly in the manual? I'm not sure what you mean. If you look up define in the GNU make manual index: http://www.gnu.org/software/make/manual/make.html#Name-Index_fn_letter-D it will send you here: http://www.gnu.org/software/make/manual/make.html#Multi_002dLine which is an entire section describing this feature with examples, and titled Defining Multi-Line Variables. Now I notice that the indexes are different between the manual versions one web page and page per node. Is there any more synchronisation needed for this documentation? Would a link be nice besides the paragraph Variable definitions are parsed as follows: from the section 3.7 How make Reads a Makefile to the section 6.8 Defining Multi-Line Variables? Regards, Markus ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Extracting structured data from file names
Hello, I can generate a few file names with an approach like the following. MY_FILES::= $(foreach index,$(shell seq 2 -1 0),$(eval MY_FILES+=X$(index).txt)) all: $(MY_FILES) @echo '|$^|' $(MY_FILES): ; Now I am looking for a way to extract the number from the shown prerequisite. Can I get such encoded data by standard make and shell functions? Regards, Markus ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Aborting script evaluation for targets without parallel build support
Hello, I would like to abort the evaluation of a make script if data processing was requested for a target which does not support a parallel software build process so far. I do not want to use the .NOTPARALLEL pseudo-target for some of my applications when I can call a make function like error on concrete demand. I have got a few difficulties there. 1. The list of targets which can be specified as command line parameters is not provided by the tool make 4.1-2.1 in the variable MAKEFLAGS. How should the target name be checked then? 2. It seems that the situation for a safe handling of the parameter -j is unclear within MAKEFLAGS. http://www.gnu.org/software/make/manual/html_node/Parallel.html I would appreciate your ideas and further advices. Regards, Markus ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: Dynamic adjustments of build dependencies for the number of available processors
So I need to be flexible with the specification of input and output parameters for these programs. But I see some software development challenges for this use case. I'm sorry but I don't really understand what you're trying to do. How do you think about to share any more software development experiences for the application of the command getconf _NPROCESSORS_ONLN together with make tools? Perhaps if you provided a concrete example it would help. I imagine that another example will not help better because I am more interested in the clarification of a general issue. How should build rules be automatically adjusted for a varying processor number? Regards, Markus ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: Cooperation with distributed job processing systems
Jobserver is a method of communicating how many jobs make thinks are running between different instances (parent/child) of the make program itself, so it knows that no more than N jobs are invoked between all instances. How do you think about to delegate such an implementation detail to a configurable job submission system? What you seem to be suggesting is something different: a facility for make to start jobs on remote systems. Yes. - My suggestion goes into this direction. There are programming interfaces available which support transparent data exchange between local and remote information systems. There are various methods for this: the most common is to use distcc in conjunction with make: https://code.google.com/p/distcc/ although this is really restricted to building C/C++ programs It is nice when remote compilation can be used with dedicated tools already. so not generally applicable. I would appreciate when this limitation can be dropped eventually. There is a nascent facility in the GNU make code that provides for make invoking jobs through a separate facility: the remote capability. I am curious to know a bit more about such a possibility. Currently there is a remote-stub.c which defines the interface, and a remote-cstms.c which attempts to use the Customs facility for starting remote jobs. However, this is obsolete and I'm not sure if anyone has tried to use it for years. Interesting background information ... Nevertheless if one were to try to add native facilities for starting jobs on remote systems into GNU make itself, rather than doing it through use of a special SHELL variable setting, etc. then remote would likely be the best way to do it, not jobserver. Does the software use a clean internal API already? I urge you to explore these other methods first, though, as they're a lot less effort all the way around. If you can write a small program that will forward a command to a remote system, then you can just replace SHELL in your makefile with that command: SHELL = /bin/submit-job all: run-a-job I find that this approach would apply one step too late because the make program manages corresponding process parallelisation by the parameter -j. Is an alternative job submission needed earlier? Regards, Markus ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: Dynamic adjustments of build dependencies for the number of available processors
I wrote some blog posts about eval and other metaprogramming techniques in make that you might find interesting: http://make.mad-scientist.net/category/metaprogramming/ I find this article also useful and helpful for my software development tasks. You mention a construct define … endef there. It seems that you call it as a macro in the section N-ary Evaluation. I wonder where these terms are explained in make's manual. I would also appreciate an addition for such key words to the documentation index. Regards, Markus ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: Dynamic adjustments of build dependencies for the number of available processors
I assume that you want something different from this but you need to explain better, sorry :-) I hope that an other wording will be clearer. Can make rules be extended on demand while a build script is evaluated? How much can a command like getconf _NPROCESSORS_ONLN influence rule evaluation directly during a make run? Regards, Markus ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: Dynamic adjustments of build dependencies for the number of available processors
It seems like you want to automatically expand or contract the number of jobs that make will run in parallel, Yes, exactly. I can not use the parameter -j directly for the start of the main make process in my case so far. based on some processing of the recipe. One of the variables (or make rules) will call the command getconf _NPROCESSORS_ONLN. The corresponding processes should be efficiently started for parallel background execution after a serial data preparation was finished. I would appreciate if I can avoid the evaluation of additional make files in a recursive way. Regards, Markus ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: Dynamic adjustments of build dependencies for the number of available processors
So you have in your toolbox $(shell) and $(eval). I am not familiar enough with the second make function. http://www.gnu.org/software/make/manual/html_node/Eval-Function.html $(eval) lets you generate rules dynamically. Does it really support the generation of completely new build rules (instead of adjusting only recipes)? NPROC:=$(shell grep processor /proc/cpuinfo | cut -b12- ) # this returns 0 1 2 3 etc # Then you might use eval to generate rules dynamically: $(foreach N,$(NPROC),$(eval # YOUR DYNAMIC RULE MYTARGET_$(N): TABmytool --dosomething -o$@ ) ) Would it make sense to stress such functionality a bit more in the documentation? I am probably guessing wrong but perhaps this gives you some ideas. I thank you very much for your constructive feedback. Regards, Markus ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: Cooperation with distributed job processing systems
There are two completely separate things: one thing is make deciding how many jobs can be run in parallel and when more jobs can be started. Thanks for your explanation. That's controlled by the jobserver and currently there's no way to replace the built-in jobserver implementation with something different (such as allowing a remote facility to track this). I hope that this situation can be improved somehow. The second thing is invoking a single job once make believes that it's OK to do so. That is reasonable. The remote API can make it possible to build make so that it always uses this method, This might be also interesting for specific software applications. but I think it's become obsolete The software status might evolve again a bit more. because it's simpler and more flexible to implement a separate program and reset SHELL, How many computer resources can be eventually saved with an alternative approach? as I describe above, than implement a bunch of new code inside GNU make itself. How are the chances to improve internal configuration interfaces? Regards, Markus ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: Dynamic adjustments of build dependencies for the number of available processors
I guess I don't really see what the purpose of this is. If you have 2 CPUs you get targets MYTARGET_0 and MYTARGET_1. If you have 4 CPUs you get targets MYTARGET_0, MYTARGET_1, MYTARGET_2, MYTARGET_3. So... then what? I need to find some source files before I can start further processes for parallel work on the search results. I want to distribute the found work amount between available processors only after the initial job data preparation. Regards, Markus ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: Dynamic adjustments of build dependencies for the number of available processors
The corresponding processes should be efficiently started for parallel background execution after a serial data preparation was finished. This makes very little sense. This issue depends on work flow ordering. In a properly written Makefile, your serial preparation step will remain serial no matter what -j arguments are used, Yes, that is usual. - The initialisation part works as expected. The dependency situation changes immediately after that step. so it should be safe to just invoke make -j xx at the beginning and let things run serially or in parallel as necessary. I have got an other software development experience here. Are you aware of open issues around topics like the following? * Rules With Multiple Outputs in GNU Make by Eric Melski http://www.cmcrossroads.com/article/rules-multiple-outputs-gnu-make * a syntax for multitarget rules by Reinier Post http://lists.gnu.org/archive/html/help-make/2014-11/msg00013.html Which tricks do you use to avoid that make tries to call a specific build rule in parallel because several output files should be regenerated by the same recipe? Regards, Markus ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Dynamic adjustments of build dependencies for the number of available processors
Hello, I would like to use all processor cores for a software build. So I try to reuse a corresponding system setting by a command like getconf _NPROCESSORS_ONLN in recipes for a make file. A bit of build preparation needs to be performed in a serial way before I can begin the desired parallel data processing with starting of some single-threaded programs for background execution. Such a system setting can vary during the run time of the affected computing environment. So I need to be flexible with the specification of input and output parameters for these programs. But I see some software development challenges for this use case. 1. If I would use a specific file name pattern here, I guess that they will also be needed for the dependency list of make targets. How should I create make targets in a consistent way on demand? 2. The return/exit code needs to recorded in dedicated files for the background processes so that their results can be safely checked in recipes later on. But I need to fiddle with several output files as a consequence for a specific target. How should I handle corresponding open issues then? * a syntax for multitarget rules by Reinier Post http://lists.gnu.org/archive/html/help-make/2014-11/msg00013.html * Rules With Multiple Outputs in GNU Make by Eric Melski http://www.cmcrossroads.com/article/rules-multiple-outputs-gnu-make I would appreciate your ideas and further advices. Regards, Markus ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
