Sorry you asked for an example. Here's an overall one from the Symbian Raptor build system that uses a shell wrapper to implement a structured output format:
<recipe name='tools2linkexe' target='/home/tim/epocroot/epoc32/release/tools2/linux-i386-libc2_17/rel/tool_exe' host='tsuro' layer='' component='' bldinf='/home/tim/src/raptor/test/smoke_suite/test_resources/tools2/bld.inf' mmp='/home/tim/src/rapto r/test/smoke_suite/test_resources/tools2/tool_exe.mmp' config='tools2_rel' platform='tools2' phase='ALL' source='' prereqs=' '> <![CDATA[ + /usr/bin/g++ -static-libgcc -O1 -Wall -Wno-ctor-dtor-privacy -Wno-unknown-pragmas -m32 -s -o /home/tim/epocroot/epoc32/release/tools2/linux-i386-libc2_17/rel/tool_exe /home/tim/epocroot/epoc32/build/tools2/c_1ccf7d25a61817e1/tool_exe_exe/tool_exe_exe/tools2/rel/linux-i386-libc2_17/tool_exe_a.o /home/tim/epocroot/epoc32/build/tools2/c_1ccf7d25a61817e1/tool_exe_exe/tool_exe_exe/tools2/rel/linux-i386-libc2_17/tool_exe_b.o /home/tim/epocroot/epoc32/release/tools2/linux-i386-libc2_17/rel/libtool_lib1.a /home/tim/epocroot/epoc32/release/tools2/linux-i386-libc2_17/rel/libtool_lib2.a /home/tim/epocroot/epoc32/release/tools2/linux-i386-libc2_17/rel/libstlport.5.1.a -lpthread g++: error: /home/tim/epocroot/epoc32/release/tools2/linux-i386-libc2_17/rel/libstlport.5.1.a: No such file or directory ]]><time start='1387115629.02783' elapsed='0.013' /> <status exit='failed' code='1' attempt='1' /> </recipe> Unfortunately I don't have the right toolchain installed hence the error. Note the time flag. This was used in a very large build system and it was easy to parse and produced a lot of useful data for us which would have been exceedingly difficult to do without a structured output format. Cheers, Tim On 15 December 2013 13:38, Tim Murphy <tnmur...@gmail.com> wrote: > I suppose I'm skirting around saying that I think gnu make needs an > output format in the same way that valgrind has "--xml=yes". I'm not > an XML fan really - JSON might be an alternative. > > It isn't your problem to provide such a mechanism and I realise it's > unfair of me to give you any sort of hard time about it. This feature > is just a small example of how gnu make will evolve an irregular > output format that's not easy to change once its "finalised" because > it's not designed to be extendable. > > Extra information like timing could be a new tag and any parser that > didn't understand the tag could just ignore it and would be able to do > so without ever making a mistake. Also the tag would be within the > tags of the job it referred to so it would be easy to see which target > it related to and any other statistical information that one wanted to > record about a job would also be there so you would have a guaranteed > way to tie information together. e.g. one might want to record how > much IO some job did or it might be important for the recipe itself to > be able to report metadata about the file like that it's meant to be > placed in the output ROFS image in a certain directory. > > If you're timing things then you might think about timing out (I have > needed that) and also about retries and this might lead you to add new > output fields e.g. "num_tries=2". With JSON or XML new attributes can > be easily and reliably ignored by a parser and what's more the > nastiest parts of the parse job can be done by any number of libraries > out there. > > I like the idea of external files because it means that log output is > unaffected so nobody's parsing will be upset and thus we are not > committed in the future to maintaining (or maintaining parser for) > something that isn't really the final answer to make output. > > I also think that floating point output is great - much easier to > parse. Even if your time storage mechanism can't be a double on some > platform, it would be great if the output could be 1.xyxyyx seconds > where the precision might vary depending on the architecture and OS. > > Anyhow it's just my 2p. > > Cheers and merry christmas, > > Tim > > On 15 December 2013 09:00, Eddy Petrișor <eddy.petri...@gmail.com> wrote: >> >> Pe 29.11.2013 12:30, "Tim Murphy" <tnmur...@gmail.com> a scris: >> >> >>> >>> When I did something similar (which I am not allowed to post) I made a >>> new file for each submake and the output filename "base" was in an >>> environment variable. I realise that nobody ever wants to change the >> >> Does it make sense to separate the profiling info into one file per process >> and then try to merge the information back? >> >> The second proposal I sent prints the pid for the 'short' and 'long' >> formats, so if one wants to focus on a specific submake, one can grep for >> the pid in the whole profile listing. The first proposal had a flexible >> output format, so one could choose to print the pid info then filter that. >> >> >> >>> way they implemented something but I thought I'd mention it. >>> >>> Also, having written a few external tools, I always appreciate it when >>> the creator of some information source makes it easy for me to recover >>> data and the relationships between each item. I also like it when >> >> Not sure what you are suggesting here. What relation would you want and >> think is not available? >> >>> there's not really any way for my parser to potentially make some >>> mistake just because a tool prints out something similar to what I'm >> >> The first version allowed a custom format which allowed any literal string >> to be printed anywhere, alongside with the target name, start and end >> timestamps, pid duration and recursivity level. >> >> The second proposal allows a custom prefix before the profiling info. >> >> So in both cases a simple grep for a specific prefix which can be chosen by >> the user is possible. Do you think that would be enough for your >> hypothetical parser? >> >>> looking for. I also like it when the format is such that it can >>> change a bit without necessarily forcing me to rewrite my parser. >> >> Not sure what would you think could change in the output without informing >> the parser at all. >> >> Currently the prefix and the field separators are customisable. >> >> Maybe an example could help. >> >>> >>> Again, this is only an opinion - it carries no weight. >>> >>> Regards, >>> >>> Tim >>> >>> On 29 November 2013 02:43, Eddy Petrișor <eddy.petri...@gmail.com> wrote: >>> > >>> > Pe 27.11.2013 13:12, "Tim Murphy" <tnmur...@gmail.com> a scris: >>> > >>> > >>> >> >>> >> FWIW >>> >> >>> >> As for profiling output, this should probably go to a file (possibly >>> >> with a .PID on the end) , not stdout .....unless..... you start to >>> >> embrace the idea of structured output for everything that make >>> >> produces. >>> > >>> > The profiling info is printed on stderr, not on stdout. >>> > I also initially thought I should isolate the information in a different >>> > file, but the problem is that for recursive make that file descriptor >>> > must >>> > be passed on every fork, which is a very intrusive change to make and >>> > very >>> > wasteful (passing an extra file descriptor, even when unnecessary). >>> > >>> > So I opted for the next best thing, prefixing the output with something >>> > a >>> > grep could anchor to. >>> > >>> >> >>> >> I have used XML before and it has advantages, not the least of which >>> >> is that it is somewhat robust in the face of corruption and that does >>> >> happen. I prefer JSON though because it's more readable and it's very >>> >> easy to parse. >>> > >>> > As I said before, I think that's the job of an external tool. >>> > >>> >> >>> >> On 27 November 2013 07:56, Eddy Petrișor <eddy.petri...@gmail.com> >>> >> wrote: >>> >> > >>> >> > Pe 25.11.2013 11:09, "Reinier Post" <reinp...@win.tue.nl> a scris: >>> >> > >>> >> > >>> >> >> > > Can't this functionality be provided by a wrapper $SHELL? >>> >> >> > > Sure, it's an extra exec(), >>> >> >> > > but it will keep the make code base simpler. >>> >> >> > >>> >> >> > I'm not sure if I understand what you mean. Do you mean wrapping >>> >> >> > all >>> >> >> > target invokes in a $SHELL? >>> >> >> >>> >> >> Yes; something line /usr/bin/time. >>> >> > >>> >> > Not on Windows. >>> >> > >>> >> >> > If that's the case, you don't get the actual/real-time >>> >> >> > measurement, >>> >> >> > and >>> >> >> > that might skew your profiling information. >>> >> >> >>> >> >> True. Hnow how much of a difference does it make? >>> >> > >>> >> > Depends on how badly the build system is written. And that's what the >>> >> > target >>> >> > of the profiling is, finding out how bad is it. I feel that >>> >> > penalising >>> >> > the >>> >> > most exactly the build systems that need the least the slow down it's >>> >> > bad >>> >> > design. >>> >> > >>> >> >> > I have some build systems that can take easily over 12 hours to do >>> >> >> > a >>> >> >> > full build (several thousands of elfs in recursive >>> >> >> > make calls - I know that's bad, I am working on it), so any extra >>> >> >> > time >>> >> >> > waste just for the measurement can mean >>> >> >> > significant delays in getting the results. >>> >> >> >>> >> >> If it's 15 minutes, they are not the thing you want to optimize >>> >> >> first. >>> >> >> The build is probably running overnight anyway. >>> >> > >>> >> > If it's a live/production system, then any unknown slow down time >>> >> > might >>> >> > not >>> >> > be acceptable. >>> >> > >>> >> > >>> >> > _______________________________________________ >>> >> > Bug-make mailing list >>> >> > Bug-make@gnu.org >>> >> > https://lists.gnu.org/mailman/listinfo/bug-make >>> >> > >>> >> >>> >> >>> >> >>> >> -- >>> >> You could help some brave and decent people to have access to >>> >> uncensored news by making a donation at: >>> >> >>> >> http://www.thezimbabwean.co.uk/friends/ >>> >>> >>> >>> -- >>> You could help some brave and decent people to have access to >>> uncensored news by making a donation at: >>> >>> http://www.thezimbabwean.co.uk/friends/ > > > > -- > You could help some brave and decent people to have access to > uncensored news by making a donation at: > > http://www.thezimbabwean.co.uk/friends/ -- You could help some brave and decent people to have access to uncensored news by making a donation at: http://www.thezimbabwean.co.uk/friends/ _______________________________________________ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make