Report to stdout like Linux kernel compilation does
Hi automakers, probably this is an oftne asked and therefore annoying question ... but is there a simple way to get automake to produce a similar output as the Linux kernel makefiles? Instead of this verbose confusing output like automake makefiles, the kernels make produces something like [ compile object1 ] [ compile object2 ] ... [ building module ] and so on. For me it would be nice to have something like a switch in the configure script to enable/ disbale verbose/ minimalistic information. E.g. first try with min. info, run into an error, therefore rerun with extensive info, search for the min. statemnt on which the error occured and voila! Like this: ./configure --disable-verbose make [ compile object1 ] [ compile object2 ] error on building object 2! ./configure make | grep [ compile object1 ] if gcc ... and so on [ compile object2 ] if gcc ... and so on error... I hope somebody can understand what I want to do. greetings, Stefan -- Stefan Bienert Zentrum fuer Bioinformatik Universitaet Hamburg Bundesstrasse 43 20146 Hamburg Germany Email: [EMAIL PROTECTED] Phone: +49 (40) 42838 7345 Fax:+49 (40) 42838 7332
Re: Report to stdout like Linux kernel compilation does
Stefan, I asked this very question a few years ago on this list. Interestingly, my examples came not from the Linux kernel build process, but from Windows builds, which use a similar output format. I love this format because warnings and errors are obvious, and yet you get enough output per file to tell you that something's going on. The real benefit of this output format is that WARNINGS are obvious. Often, in standard GNU/Unix/Linux build processes, warnings just zip right by without much notice. My question to the list was: How can I get Autotools builds to be quiet, so I can see the warnings? The response that I got was that I should just redirect stdout to /dev/null on the make command line. For the next couple of years, I very was frustrated with this response. I thought it was a cop-out for just not wanting to provide the functionality. Then I realized that it was really the Unix way. You want to see everything so that you know what's going on. When you want to clean up the warnings (usually something done near the end of a development cycle), you simply build with stdout redirected to /dev/null when you run make a few times, and you'll see the warnings appear, because they're output to STDERR, not STDOUT. Now -- that said, I really see nothing wrong with my original request, in the form of an Automake switch. It would be nice to be able to tell Automake to build Makefiles that generate this sort of output. Unfortunately, you and I aren't going to get much agreement, I think. Perhaps, if you were to write a patch to Autoconf, providing a macro or switch that generates such Makefiles... This also the GNU way. :) Regards, John On Fri, Apr 11, 2008 at 5:42 AM, Stefan Bienert [EMAIL PROTECTED] wrote: Hi automakers, probably this is an oftne asked and therefore annoying question ... but is there a simple way to get automake to produce a similar output as the Linux kernel makefiles? Instead of this verbose confusing output like automake makefiles, the kernels make produces something like [ compile object1 ] [ compile object2 ] ... [ building module ] and so on. For me it would be nice to have something like a switch in the configure script to enable/ disbale verbose/ minimalistic information. E.g. first try with min. info, run into an error, therefore rerun with extensive info, search for the min. statemnt on which the error occured and voila! Like this: ./configure --disable-verbose make [ compile object1 ] [ compile object2 ] error on building object 2! ./configure make | grep [ compile object1 ] if gcc ... and so on [ compile object2 ] if gcc ... and so on error... I hope somebody can understand what I want to do. greetings, Stefan -- Stefan Bienert Zentrum fuer Bioinformatik Universitaet Hamburg Bundesstrasse 43 20146 Hamburg Germany Email: [EMAIL PROTECTED] Phone: +49 (40) 42838 7345 Fax:+49 (40) 42838 7332
Re: Report to stdout like Linux kernel compilation does
Some work on improving make output (pretty-printing) has been done for Xerces-C http://xerces.apache.org/xerces-c/ See also this bugreport http://issues.apache.org/jira/browse/XERCESC-1440 You may want to ask about it on one of the mailing lists http://xerces.apache.org/xerces-c/mailing-lists.html Jeroen. Stefan, I asked this very question a few years ago on this list. Interestingly, my examples came not from the Linux kernel build process, but from Windows builds, which use a similar output format. I love this format because warnings and errors are obvious, and yet you get enough output per file to tell you that something's going on. The real benefit of this output format is that WARNINGS are obvious. Often, in standard GNU/Unix/Linux build processes, warnings just zip right by without much notice. My question to the list was: How can I get Autotools builds to be quiet, so I can see the warnings? The response that I got was that I should just redirect stdout to /dev/null on the make command line. For the next couple of years, I very was frustrated with this response. I thought it was a cop-out for just not wanting to provide the functionality. Then I realized that it was really the Unix way. You want to see everything so that you know what's going on. When you want to clean up the warnings (usually something done near the end of a development cycle), you simply build with stdout redirected to /dev/null when you run make a few times, and you'll see the warnings appear, because they're output to STDERR, not STDOUT. Now -- that said, I really see nothing wrong with my original request, in the form of an Automake switch. It would be nice to be able to tell Automake to build Makefiles that generate this sort of output. Unfortunately, you and I aren't going to get much agreement, I think. Perhaps, if you were to write a patch to Autoconf, providing a macro or switch that generates such Makefiles... This also the GNU way. :) Regards, John On Fri, Apr 11, 2008 at 5:42 AM, Stefan Bienert [EMAIL PROTECTED] wrote: Hi automakers, probably this is an oftne asked and therefore annoying question ... but is there a simple way to get automake to produce a similar output as the Linux kernel makefiles? Instead of this verbose confusing output like automake makefiles, the kernels make produces something like [ compile object1 ] [ compile object2 ] ... [ building module ] and so on. For me it would be nice to have something like a switch in the configure script to enable/ disbale verbose/ minimalistic information. E.g. first try with min. info, run into an error, therefore rerun with extensive info, search for the min. statemnt on which the error occured and voila! Like this: ./configure --disable-verbose make [ compile object1 ] [ compile object2 ] error on building object 2! ./configure make | grep [ compile object1 ] if gcc ... and so on [ compile object2 ] if gcc ... and so on error... I hope somebody can understand what I want to do. greetings, Stefan -- Stefan Bienert Zentrum fuer Bioinformatik Universitaet Hamburg Bundesstrasse 43 20146 Hamburg Germany Email: [EMAIL PROTECTED] Phone: +49 (40) 42838 7345 Fax:+49 (40) 42838 7332
Re: Report to stdout like Linux kernel compilation does
On 2008-04-11 16:28, John Calcote wrote: Stefan, I asked this very question a few years ago on this list. Interestingly, my examples came not from the Linux kernel build process, but from Windows builds, which use a similar output format. I love this format because warnings and errors are obvious, and yet you get enough output per file to tell you that something's going on. The real benefit of this output format is that WARNINGS are obvious. Often, in standard GNU/Unix/Linux build processes, warnings just zip right by without much notice. Part of that is that I expect a significant number of open source developers use emacs or similar environments that clearly highlight errors/warnings. So the full compilation commands don't normally get in the way, and are always there to look at for detailed info, without having to rebuild with a special switch (or regenerate the makefile using a special switch). Now -- that said, I really see nothing wrong with my original request, in the form of an Automake switch. It would be nice to be able to tell Automake to build Makefiles that generate this sort of output. Unfortunately, you and I aren't going to get much agreement, I think. This would mean that if I wanted to get the full commands back for debugging, I'd have to rerun automake with a new option, potentially triggering lots of rebuilds (if there are rules with Makefile as a dependency). In addition, I would expect such an option to be quite invasive (most likely requiring changes to all .am fragments). So it would be hard to maintain, with only a small set of actual users. Perhaps, if you were to write a patch to Autoconf, providing a macro or switch that generates such Makefiles... This also the GNU way. :) Of course, build the feature without breaking anything else, and it will exist for as long as someone is willing to maintain it :D
Re: Report to stdout like Linux kernel compilation does
John Calcote wrote: I love this format because warnings and errors are obvious, and yet you get enough output per file to tell you that something's going on. To give you a different perspective, I *hate* that format because it hides problems and *makes debugging harder*. I want to see exactly the command that was executed. I want to see the entire command. I don't want to see an abbreviation of the command. The real benefit of this output format is that WARNINGS are obvious. Again, from my perspective the disadvantage of this is that it *hides* the commands that produced the errors and warnings. Hiding this information makes the build harder to debug. Often, in standard GNU/Unix/Linux build processes, warnings just zip right by without much notice. Using the eye to scan through build output to find problems isn't a reliable method. It is too easy to miss something when tired or hurried. If you are counting on using your eye to find these then statistically I believe you will have missed some error or warning at some time in the past. (I know this from human spam filtering which has a non-zero error rate.) It is much better to use a tool to scan through build output. Tools are much more reliable than humans for this type of tedious and repetitive task. For me I use my editor for this. I tell my editor to open the file and position my cursor on the next error or warning. Since the editor is walking each warning and error I can't miss any. Most programming editors do this these days. However even without editor assistance these messages can be extracted using 'grep' or other process. My question to the list was: How can I get Autotools builds to be quiet, so I can see the warnings? The response that I got was that I should just redirect stdout to /dev/null on the make command line. That is effectively what you would be getting if you threw away the information of the command that produced the problem. If you want to throw it away then okay but please leave it there for the rest of us who need it. For the next couple of years, I very was frustrated with this response. I thought it was a cop-out for just not wanting to provide the functionality. It isn't a cop-out. Instead it is that your requested behavior *actively hurts* the rest of us who need that information. If things were reversed then I would be writing the list begging to please provide the full build information. It is easy to throw useful information away. After having been thrown away it can't be gotten back. It is not symmetrical. Then I realized that it was really the Unix way. You want to see everything so that you know what's going on. Yes! When you want to clean up the warnings (usually something done near the end of a development cycle), you simply build with stdout redirected to /dev/null when you run make a few times, and you'll see the warnings appear, because they're output to STDERR, not STDOUT. No! I never build with output redirected to /dev/null. I always build with full warnings enabled and clean them up as I go. Trying to do a big cleanup at the very end is bad. It is better to keep it clean as it goes. Always build with full warnings enabled. Always clean up warnings as they are introduced. Always keep a warning free build. Now -- that said, I really see nothing wrong with my original request, in the form of an Automake switch. It would be nice to be able to tell Automake to build Makefiles that generate this sort of output. Unfortunately, you and I aren't going to get much agreement, I think. As an option I think it is fine too. But someone would need to do the work. Perhaps, if you were to write a patch to Autoconf, providing a macro or switch that generates such Makefiles... This also the GNU way. :) Yes. Bob
Re: Report to stdout like Linux kernel compilation does
Bob Proulx wrote: To give you a different perspective, I *hate* that format because it hides problems and *makes debugging harder*. I want to see exactly the command that was executed. I want to see the entire command. I don't want to see an abbreviation of the command. Fortunately, no one is advocating the current formatting be removed or even altered, only that a less spammy alternative exist. Always build with full warnings enabled. Always clean up warnings as they are introduced. Always keep a warning free build. Given the wide leeway the C and C++ standards give implementors in respect to what is warning-worthy, I think this is crazy. There are far more compilers in the world than just GCC, and trying to make a warning-free build on /every/ platform is just far too much torment-of-the-damned for far too little benefit.
Re: Report to stdout like Linux kernel compilation does
On Fri, Apr 11, 2008 at 10:36 AM, Robert J. Hansen [EMAIL PROTECTED] wrote: Bob Proulx wrote: Always build with full warnings enabled. Always clean up warnings as they are introduced. Always keep a warning free build. Given the wide leeway the C and C++ standards give implementors in respect to what is warning-worthy, I think this is crazy. There are far more compilers in the world than just GCC, and trying to make a warning-free build on /every/ platform is just far too much torment-of-the-damned for far too little benefit. Hmmm. I'd have to disagree here. I carefully consider every warning I see, and evaluate whether or not it represents a real problem. While there have been versions of various compilers in the past that like to warn about stupid things, compiler writers understand the language better than most people. If they warn me about something, then I at least give it the credit it's due. Often, these warnings indicate a real or future problem in my code. Interestingly, as my proficiency with the language (C or C++) has grown, the number of warnings I deal with has decreased. I still get errors, from stupid type-o's, but I don't get that many warnings anymore. John
Re: Report to stdout like Linux kernel compilation does
* Robert J. Hansen wrote on Fri, Apr 11, 2008 at 06:36:56PM CEST: Bob Proulx wrote: To give you a different perspective, I *hate* that format because it hides problems and *makes debugging harder*. I want to see exactly the command that was executed. I want to see the entire command. I don't want to see an abbreviation of the command. Fortunately, no one is advocating the current formatting be removed or even altered, only that a less spammy alternative exist. When Alexandre last commented on a request like this, one comment that wasn't already mentioned in this thread was this issue: a patch for nicer output should not cause hugely increased Makefile.in files. Makefile.in size is an issue for some larger projects. There have been several patches, and a couple of add-on projects, posted on either this list or automake-patches. At least one of them was ready to use IIRC. Please look them up. I might consider looking at one eventually, but right now that isn't one of my priorities. Cheers, Ralf
Re: Report to stdout like Linux kernel compilation does
On Fri, 11 Apr 2008, Ralf Wildenhues wrote: When Alexandre last commented on a request like this, one comment that wasn't already mentioned in this thread was this issue: a patch for nicer output should not cause hugely increased Makefile.in files. Makefile.in size is an issue for some larger projects. Since libtool is going to be even faster than it is now, it seems that this should free up some CPU time for Automake to implement reporting to stdout like the Linux kernel compilation does. I see little need to give back this CPU time to the community since CPUs spend most of their time executing the kernel idle task and autotool developers are more qualified than mere users to decide how to make use of the CPU time which has already been acquired. We can simply wire-transfer it from the libtool CPU time account into Automake's CPU time account. If the next Automake and libtool are released at the same time, no one will notice that the transfer took place. Once libtool is faster, people will start to complain that the large size of the Automake makefile causes 'make' to take a long time to start doing anything. A shim layer called 'nmake' can be used to capture the output of 'make -n' and execute these commands more efficiently via the bash shell. Bob == Bob Friesenhahn [EMAIL PROTECTED], http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer,http://www.GraphicsMagick.org/
Re: Report to stdout like Linux kernel compilation does
Robert J. Hansen wrote: John Calcote wrote: Hmmm. I'd have to disagree here. I carefully consider every warning I see, and evaluate whether or not it represents a real problem. Yes. This strikes me as perfectly sane behavior. I also agree with this. Using reasonable judgement is a good thing and I apologize if I gave any impression otherwise. Insisting on warning-free builds is not sane behavior, especially given just how many compilers there are out there and just how many warnings can be reached. On my main two development environments it is quite easy to achieve warning free output. My main compiler for years was the native HP ANSI C compiler for 64-bit HP-UX. I almost reacted to the There are far more compilers in the world than just GCC comment with GCC? What's that? I have actually found gcc to generate less warnings about things the HP compiler would warn about. But gcc moves forward quite quickly and probably many of those are already improved by now. I am not a single compiler advocate. Insistence on perfection is not reasonable. Not given the staggering diversity of platforms, not given the staggering diversity of compilers, the expense of getting access to these environments, etc. Developing with a clean build and requiring a clean build on other systems it is built upon are two different things. I was talking about development. I think it is important to keep things clean as you go along during development. Thinking that you will fix them all later at some other point in time IMNHO is bad. This is very much like the fixing broken windows theory. http://en.wikipedia.org/wiki/Fixing_Broken_Windows I have worked on projects with so many problems that when new ones came along people weren't motivated to fix them. But if the build is clean and someone adds a new problem to it then they feel pressure to clean it up quickly. Valid warnings frequently point to subtle bugs that are easily avoided but otherwise consume a large amount of debug time. Worse is when people don't actually fix the source of the problem but apply a layer of workaround elsewhere because they didn't understand the root source of a problem. Consider Bob's original statement, the one I was disagreeing with: Always build with full warnings enabled. Always clean up warnings as they are introduced. Always keep a warning free build. I agree with the first. I disagree with the second--some warnings are erroneous--and I think the third is not practical, given the number of different compilers and OSes in use. Consider the statement to which I was replying: When you want to clean up the warnings (usually something done near the end of a development cycle), you simply build with stdout redirected to /dev/null when you run make a few times, and you'll see the warnings appear, because they're output to STDERR, not STDOUT. Since you disagree with my statement challenging this then does that mean that you agree with the strategy I was challenging? That is, don't check the project build warning status until you are near the end of a development cycle and _then_ start to address warnings? I am sorry but that type of strategy triggers an immune response in me. I strongly believe this is not a good strategy and am not shy about stating my opinion on it. Don't let broken windows accumulate. Fix broken windows as you go along so that the project is always in as good of a state as possible all of the time. Every time I compile on a new platform I look at the warnings generated. If my native compile environment isn't smart enough to to generate a valid warning but another platform is smart and generates a previously unknown valid warning then I am not going to ignore it. Every new environment that my code reaches almost always teaches me something useful about my code. Bob
Re: Report to stdout like Linux kernel compilation does
On Fri, Apr 11, 2008 at 3:00 PM, Bob Proulx [EMAIL PROTECTED] wrote: When you want to clean up the warnings (usually something done near the end of a development cycle), you simply build with stdout redirected to /dev/null when you run make a few times, and you'll see the warnings appear, because they're output to STDERR, not STDOUT. Since you disagree with my statement challenging this then does that mean that you agree with the strategy I was challenging? That is, don't check the project build warning status until you are near the end of a development cycle and _then_ start to address warnings? I am sorry but that type of strategy triggers an immune response in me. I strongly believe this is not a good strategy and am not shy about stating my opinion on it. Don't let broken windows accumulate. Fix broken windows as you go along so that the project is always in as good of a state as possible all of the time. Every time I compile on a new platform I look at the warnings generated. If my native compile environment isn't smart enough to to generate a valid warning but another platform is smart and generates a previously unknown valid warning then I am not going to ignore it. Every new environment that my code reaches almost always teaches me something useful about my code. I'm sorry - I was a little confusing here. I don't actually work this way either. I'm a stickler for details, and I can't really abide warnings being left around for weeks or months - it's the Type-A'ed-ness in me or something. Anyway, what I meant was that my development cycles are fairly short - I might not get around to checking for warnings (by redirecting stdout to /dev/null) for a few days, so I might let a few warnings go for a week (or less), if I'm busy working on a task. But for this very reason, I think it's a good idea to have clean output. I've heard all the arguments before about not being able to see the cause of the warning because your command line is obscured by a pretty-printing build system. I like to see full output, AFTER I've discovered there's a problem. In which case, I'd then go to the build log file to see the FULL output. I just don't see why we have to watch complete build information go by with a build that has no warnings or errors - which is supposed to be the usual case - it's the exceptional case where you need to see full output. As far as setting up your editor to take you to the next warning or error on command - now that's a cool idea - I love how this works in the M$ IDE on my Windows projects. John
Re: Report to stdout like Linux kernel compilation does
* John Calcote wrote on Fri, Apr 11, 2008 at 11:18:48PM CEST: Anyway, what I meant was that my development cycles are fairly short - I might not get around to checking for warnings (by redirecting stdout to /dev/null) for a few days, so I might let a few warnings go for a week (or less), if I'm busy working on a task. Redirecting stdout to /dev/null typically does not hide the warnings, they typically show up on stderr. As far as setting up your editor to take you to the next warning or error on command - now that's a cool idea - I love how this works in the M$ IDE on my Windows projects. I can't understand why people do edit-compile-test cycles without having compiler output be post-processed by their editor. It works with all decent unix editors, and it's even more comfortable than not having to go through the verbose output with your eyes. The other problem I have with Linux style build output is that if I ever get a bug report showing a build failure using that style, i.e., the reporter does not post the failing command line, then that bug report just lost half its value, by requiring yet another email roundtrip of questions. Cheers, Ralf
Re: Report to stdout like Linux kernel compilation does
On Fri, Apr 11, 2008 at 5:28 PM, Ralf Wildenhues [EMAIL PROTECTED] wrote: I can't understand why people do edit-compile-test cycles without having compiler output be post-processed by their editor. It works with all decent unix editors, and it's even more comfortable than not having to go through the verbose output with your eyes. Can you explain how to do that? ie, the post-processing in an editor thing.
