"Mark Knecht" <[EMAIL PROTECTED]> posted [EMAIL PROTECTED], excerpted below, on Thu, 14 Sep 2006 17:43:19 -0700:
> Now, you are very adept at this. You're explanations make sense to > the level I've considered them. (Not very far right now...) Main > questions: Adept, perhaps, but don't take my observations as being from God or anything! =8^) I try to be fairly cautious with my CFLAGS, but if anything quits working, I know how to undo them and try with a more generic set, and in fact do so from time to time on individual packages, before filing bugs on them. Sometimes it's my CFLAGS, tho usually my config doesn't matter a whit to the bug, as I've been reasonably cautious in my choices to begin with and don't tend to enable stuff like the unsafe floating-point math options that give folks problems from time to time. In particular, as you can see from the -ftree-vectorize subthread, I tend to stay with the defaults when I can't explain with some degree of confidence exactly what the effect of a flag might be and why I might or might not want it. I don't know enough about that area to do that, so I've stayed well away from it in my CFLAGS. > 1) What can be done to test this out at my end without making a 2-day > commitment to rebuild the complete machine. Is it possibly to rebuild > only portions of the machine using a different set of flags or is it a > system wide commitment requiring that I rebuild 575 packages as I did > last weekend? In general, you /can/ rebuild only a part of your system and test that, before making further changes. However, it's important to use a bit of (un?)common sense when doing so, or your results won't be worth much. Basically, in ordered to see how an optimization affects something, you must have some awareness of the shared libraries it uses and to what extent it uses them, recompiling enough of the heavily used dependencies that the critical parts of your test applications (including the libraries they load) are using the new optimizations. One lib that all applications make some use of is glibc, so it can be worth recompiling. It's a big recompile on its own, but of course nowhere near as big as recompiling the entire system. =8^) However, glibc is a special case in some aspects for a number of reasons. The glibc ebuild is pretty conservative with the flags it allows, and actually replaces -Os with -O2, due to problems -Os had mainly on x86, back in the gcc-3.2 and 3.3 era. Since the system is pretty horribly broken if glibc breaks, to the point you are likely to have to boot to a backup or liveCD to fix it, this isn't an unreasonable policy at all. None-the-less, after making doubly sure I had tested-working backups, I decided to see just what the effect of taking out that -Os -> -O2 replace in the glibc ebuild might be. For awhile I actually ran a glibc I had built after having removed that replace. The system continued to work just fine with a -Os compiled glibc, it didn't break or anything, but it didn't seem to be much better either and in some cases seemed worse. It turns out that glibc is built in a much more modular fashion than many libraries, so an app will only load the parts of it it needs, not the parts it doesn't, and that -Os doesn't work so well with this rather extreme (compared to most libs) modularization. As well, as I said, glibc is used by everything on the system, which meant that having bypassed one of the safeties in the glibc ebuild, I could never be sure whether a bug I was experiencing was due to my strange glibc, or to some problem with the package the bug was showing up in or one of its other dependencies. I concluded that it simply wasn't worth bypassing the safeties in the ebuild, and since then, have left them there. Thus, with glibc anyway, simply switching to -Os in your CFLAGS won't make any difference, since the ebuild replaces that with -O2 anyway. The /other/ CFLAGS might make a difference, but -Os it self won't, unless you bypass the replace in the ebuild, and as my experimentation demonstrated well enough for me, that's really not worth the trouble. As I said, the other CFLAGS may make a bit of difference tho, so you might consider it anyway, if you decide to try them. For X users, another library that's going to be commonly used is libX11. You'll probably want to recompile xorg-server (assuming modular-X) as well, plus whatever xf86-video-* driver you use, and libXcomposite if you use the composite extension (transparent windows and the like). Together, those will be pretty critical for performance of any X app. For OpenGL accelerated apps, mesa is likely to be critical to performance as well, for any functions not handled by hardware. For anything written in C++, almost anything KDE among other packages, gcc libstdc++, a part of gcc, will be critical. Other than for C++ apps/libraries, recompiling gcc with new CFLAGS shouldn't make that much difference in how the app runs, tho it might make some difference in how fast compiles the app. (Of course, note that compile speed can be dramatically affected by the optimizations being compiled, since many of them cause additional passes in the process, to catch that little bit of extra optimization you are telling gcc to enable. Tell gcc to do more work and of course it'll take longer doing it!) You mentioned gnome. For anything gnome related, you'll want to recompile glib, gtk, libgnome, possibly orbit, and maybe others (your window manager). I don't know enough about gnome internals to go further with it. With KDE, it'd be qt and kdelibs (plus as I mentioned gcc, for libstdc++), kwin, probably kicker and konqueror, etc, plus the packages behind various kparts as used in whatever app you are testing, as well as the app itself. > 2) What about building the kernel? How do the standard > > make && make modules_install > > command make any use of the flags in /etc/make.conf? They normally don't. However, as is commonly the case with manually compiled packages, the kernel build process makes use of CFLAGS if it finds that environment variable set. Thus, you can source make.conf, export the desired variables, and go from there. In fact, I have a small utility script installed as /usr/local/bin/buildflags.sh that contains the following: # Source this previous to doing a manual build to import portage buildflags [ -z "$CFLAGS" ] && CFLAGS="`. /etc/make.conf 2>/dev/null; echo $CFLAGS`" [ -z "$CXXFLAGS" ] && CXXFLAGS="`. /etc/make.conf 2>/dev/null; echo $CXXFLAGS`" [ -z "$LDFLAGS" ] && LDFLAGS="`. /etc/make.conf 2>/dev/null; echo $LDFLAGS`" export CFLAGS CXXFLAGS LDFLAGS As you can see, that gathers and exports the three build variables CFLAGS, CXXFLAGS, and LDFLAGS. If I source that into my running shell before I build anything (including the kernel) manually, the build should see my existing environment and compile accordingly. As it happens, I use a script to compile and install the kernel, as well, and it sources buildflags.sh before it does the kernel compile. /usr/local/sbin/ki (for kernel install: #!/bin/bash # kernel compile and install echo echo sourcing buildflags ki_buildflagfile=/usr/local/bin/buildflags.sh if [ -f $ki_buildflagfile ] ; then . $ki_buildflagfile else pause 10 x \\a$ki_buildflagfile doesn\'t exist fi unset buildflagfile echo echo cd-ing to /usr/src/linux cd /usr/src/linux echo Next: make sleep 1 echo make || exit 1 echo echo Next: make modules_install sleep 1 echo make modules_install || exit 2 echo echo Next: mount /boot sleep 1 echo mount /boot &>/dev/null echo echo Next: make install sleep 1 echo make install || exit 3 echo echo Next: umount /boot sleep 1 echo umount /boot &>/dev/null echo Done. echo In addition to using buildflags.sh, that script also uses a utility script I've written called pause, installed as /usr/local/bin/pause (and also as /bin/pause here, so I can use it to debug initscripts before /usr/local is mounted). #!/bin/bash # putting everything in a function to allow local scoped vars function doit { echo local PROMPT="Pause for char" case $# in 0) read -n1 -p "$PROMPT:";; 1) if [ $1 == "--help" -o $1 == "-h" -o $1 == "-?" ] then echo $PROMPT script utility. echo echo Usage: pause [timeoutsec [defaultchar [promptstring]]] echo \ \ \ \ \ \ \ pause \(--help\|-h\|-?\) echo echo Source pause to have \$REPLY set to the returned char. echo echo A timeoutsec of 0 indicates no timeout. Otherwise, echo on timeout, defaultchar if provided is returned in \$REPLY. echo echo With timeout and defaultchar, promptstring defaults to: echo $PROMPT \(default defaultchar in timeoutsec\): echo With just a timeout, prompt default is: echo $PROMPT \(timeout timeoutsec\): echo Without a timeout, the default is: echo $PROMPT: echo echo elif [ $1 == 0 ] then read -n1 -p "$PROMPT:" else read -n1 -t$1 -p "$PROMPT (timeout $1):"; echo fi;; 2) if [ $1 == 0 ] then read -n1 -p "$PROMPT:" else read -n1 -t$1 -p "$PROMPT (default $2 in $1):"; echo REPLY=${REPLY:-$2} fi;; *) if [ $1 == 0 ] then shift 2 echo -e $* read -n1 else local TIMEOUT=$1 DEFAULTCHAR=$2; shift 2 echo -e $* read -n1 -t$TIMEOUT; echo REPLY=${REPLY:-$DEFAULTCHAR} fi;; esac echo } doit $* Obviously, I've taken a bit more time to properly document pause, so you can run pause -h or --help to get the usual usage summary. The other two scripts aren't so nicely --help equipped; you have to look at the source to see what they are doing. I've found pause to be very useful over the years. =8^) As you can see, I routinely compile the kernel with my chosen CFLAGS, and it works fine here. Of course, your kernel config will be different and I can't vouch for every single driver, but the concept is sane and my CFLAGS are demonstrated to work with the core kernel and the drivers I use, anyway. As Greg mentions, there's also a compile for size option in the kernel config, and I have it enabled as well. It enables -Os, but I think it does a few other kernel specific things as well. I've never looked into it in detail, but I know I've had it enabled every since its introduction some kernels ago, with the accompanying coverage in LWN. That coverage in fact was what got me started on the whole -Os thing, before I'd even switched to Gentoo with 2004.1 and was still on Mandrake IIRC, so it has been a kernel option and I've been using it for over two years now. -- Duncan - List replies preferred. No HTML msgs. "Every nonfree program has a lord, a master -- and if you use the program, he is your master." Richard Stallman -- gentoo-amd64@gentoo.org mailing list