Re: pros/cons of installing from source
On 5/3/07, Greg Folkert [EMAIL PROTECTED] wrote: On Thu, 2007-05-03 at 22:38 -0600, Javier Vasquez wrote: On 5/3/07, Greg Folkert [EMAIL PROTECTED] wrote: ... ... Nope, aptitude offers you the dependencies the distro developer specifies (not just the application developer), some of them are recommendations, some of them are strictly required. When you are to compile the application yourself, you can find that even things strictly required by a binary distro are really not. The reason is that the distro developer compiled using a particular library for example, when he/she could have used another or none. So on binary distros one has 2 levels of non optional dependencies I believe, the ones set by the original package developer, and the ones set by the distro developer for the package. This is not true on sourceMage, not sure on gentoo (it looks like people immediately thinks of gentoo when talking about source based distros) since I don't know about it, and it's just because the only really required dependencies on sourceMage by policy are the ones set by the original package developer. Whether this makes a difference or not, it depends on the system one wants to get. I specifically picked gd for that very reason. It supports eleventy options. The reason I picked it, is because the linked set of libraries for Debian pulls in some xlibs on even cursor based systems. Basically the changelog said something like: the linking of the code against xlibs, only slightly increases the pull in of files amounting to 72KiB, these days this small amount of disk space does not matter. The performance is not affected in any way, but allows for 98% coverage and reduces package count by 12 flavors. If you must have no xlibs, compile it yourself without it. Which, to be honest, is the exact same reason people restore or rebuild classic cars or heavily customize the ricers they own, or build thier own house or hand craft the Linux Distro of their choice. I might be wrong though about how debian package developers compile things though, I'm not one, and it might be that there's a policy to keep as required dependencies only the ones set by upstream, but I'm not aware of it. There is the Debian Developers Guide and the Debian Free software Guide. These BOTH have an effect on the Original Source code. BTW, you do know that *EXCEPT* for non-free pieces (like non-source firmware and binary blobs) that Debian include *.orig.tar.gz for everything? They also have a *.diff.tar.gz... so following your comment about keeping upstream untouched as much as possible is not-genuine. Debian does this, but at the same time folowing the DFSG. As a side note, something I liked from sourceMage was its policy of keeping upstream code untouched as much as possible. I don't know of any binary distro trying to keep up with that. However this is beyond the discussion since there's a lot to talk about that, just something to mention, :). I mentioned Debian Policy (Set forth by the Debian Free Software Guide) as being the BEST reason to run Debian Linux... or Debian FreeBSD or Debian period. For anything else I agree. Just wanted to clarify a bit further about the dependencies comment. For not compiling the kernel as a suggestion, well, again it depends (I don't totally agree). For a regular user with 40GB of HD or more, there's no problem on having a blotted set of modules he/she will never use. If you have limited HD, you'd like to compile only what you need, and not everything so far supported by the kernel (besides you get more tunned configuration at the same time for free if you want, I provided the pentium M example, but I bet there are more, like the kind of pre-emption, the frequency, etc, not that one gets better performance, but that one gets the right tunned configuration for the system, and not just a blotted generic one). Same thing applies to other packages. One might want to remove any gnome/QT dependency as much as possible, one might not support some graphics libraries although required for the general purpose, etc. Good enough, I could pick, but won't. :-P ... I am fully up on Gentoo. I like its handling, its tools for helping in dealing with packaging and other features... specifically not using upstart (at the moment) and other pieces that traditional UNIX systems have more in common with it. Gentoo is very friendly, it is just picky about its friends. Please, if I'm completely wrong about my comments on dependencies, let me know. Maybe there's a debian policiy talking about this (is there a pointer?) that I'm not aware of, and I was just talking non sense, :). The Debian Social Contract and the DFSG is located: http://www.debian.org/social_contract The Developer's Guide is located: http://www.debian.org/devel/ Specifically though if you want to really read on policy:
Re: pros/cons of installing from source
On Fri, May 04, 2007 at 10:34:27AM -0600, Javier Vasquez wrote: On 5/3/07, Greg Folkert [EMAIL PROTECTED] wrote: On Thu, 2007-05-03 at 22:38 -0600, Javier Vasquez wrote: On 5/3/07, Greg Folkert [EMAIL PROTECTED] wrote: ... ... Nope, aptitude offers you the dependencies the distro developer specifies (not just the application developer), some of them are recommendations, some of them are strictly required. When you are to compile the application yourself, you can find that even things strictly required by a binary distro are really not. The reason is that the distro developer compiled using a particular library for example, when he/she could have used another or none. So on binary distros one has 2 levels of non optional dependencies I believe, the ones set by the original package developer, and the ones set by the distro developer for the package. This is not true on sourceMage, not sure on gentoo (it looks like people immediately thinks of gentoo when talking about source based distros) since I don't know about it, and it's just because the only really required dependencies on sourceMage by policy are the ones set by the original package developer. Whether this makes a difference or not, it depends on the system one wants to get. I specifically picked gd for that very reason. It supports eleventy options. The reason I picked it, is because the linked set of libraries for Debian pulls in some xlibs on even cursor based systems. Basically the changelog said something like: the linking of the code against xlibs, only slightly increases the pull in of files amounting to 72KiB, these days this small amount of disk space does not matter. The performance is not affected in any way, but allows for 98% coverage and reduces package count by 12 flavors. If you must have no xlibs, compile it yourself without it. Which, to be honest, is the exact same reason people restore or rebuild classic cars or heavily customize the ricers they own, or build thier own house or hand craft the Linux Distro of their choice. I might be wrong though about how debian package developers compile things though, I'm not one, and it might be that there's a policy to keep as required dependencies only the ones set by upstream, but I'm not aware of it. There is the Debian Developers Guide and the Debian Free software Guide. These BOTH have an effect on the Original Source code. BTW, you do know that *EXCEPT* for non-free pieces (like non-source firmware and binary blobs) that Debian include *.orig.tar.gz for everything? They also have a *.diff.tar.gz... so following your comment about keeping upstream untouched as much as possible is not-genuine. Debian does this, but at the same time folowing the DFSG. As a side note, something I liked from sourceMage was its policy of keeping upstream code untouched as much as possible. I don't know of any binary distro trying to keep up with that. However this is beyond the discussion since there's a lot to talk about that, just something to mention, :). I mentioned Debian Policy (Set forth by the Debian Free Software Guide) as being the BEST reason to run Debian Linux... or Debian FreeBSD or Debian period. For anything else I agree. Just wanted to clarify a bit further about the dependencies comment. For not compiling the kernel as a suggestion, well, again it depends (I don't totally agree). For a regular user with 40GB of HD or more, there's no problem on having a blotted set of modules he/she will never use. If you have limited HD, you'd like to compile only what you need, and not everything so far supported by the kernel (besides you get more tunned configuration at the same time for free if you want, I provided the pentium M example, but I bet there are more, like the kind of pre-emption, the frequency, etc, not that one gets better performance, but that one gets the right tunned configuration for the system, and not just a blotted generic one). Same thing applies to other packages. One might want to remove any gnome/QT dependency as much as possible, one might not support some graphics libraries although required for the general purpose, etc. Good enough, I could pick, but won't. :-P ... I am fully up on Gentoo. I like its handling, its tools for helping in dealing with packaging and other features... specifically not using upstart (at the moment) and other pieces that traditional UNIX systems have more in common with it. Gentoo is very friendly, it is just picky about its friends. Please, if I'm completely wrong about my comments on dependencies, let me know. Maybe there's a debian policiy talking about this (is there a pointer?) that I'm not aware of, and I was just talking non sense, :). The Debian Social Contract and the DFSG is located: http://www.debian.org/social_contract
Re: pros/cons of installing from source
On Fri, May 04, 2007 at 12:42:40PM -0600, Paul E Condon wrote: On Fri, May 04, 2007 at 10:34:27AM -0600, Javier Vasquez wrote: [heavy snippage dude] You mentioned debian commitment to FSF and its social contract, as very good reasons by themselves to run debian. I totally agree. However debian is not the only distro with such commitment. Actually sourceMage picked debian social contract and modified it a bit... snip... I understand Greg's comments to be about Debian's commitment to enforcing a packaging policy, i.e. a policy on where and how things are installed. To me is quite a different thing than a social policy. In Debian, if the install scripts of a package to not put things where the policy says they should be _that_ is a bug in the package. It may also be considered a bug in some other distro.s. I've not kept track of this sort of policy issue in any other distro. since I discovered Debian. The Social Policy is also good. But I think it is easy to feel good about a Social Policy, and it is hard work to implement a packaging policy. I think that the packaging policy is what really sets debian apart. THat's why everything just works... because dev's can count on things being a certain way and if its not, they can count on it being fixed. A signature.asc Description: Digital signature
Re: pros/cons of installing from source
On Friday 04 May 2007 05:36, Greg Folkert wrote: like encoders and decoders. Along with the entire Going to be very machine specific... one of the biggest groans against GCC is that is supports so many target platforms but doesn't do any of them particularly well. Intel's compiler generates very quick code but IIRC only targets their processors. Encoders and decoders, and things like emulators that use image scaling, generally contain code to take advantage of processor specific features at run time. For example if you build mplayer correctly it will detect and utilise the correct SIMD extensions for your hardware at runtime. emerge is very simple tool to use, especially if you just hunker down and use it. Selecting the right architecture and listings (I used current snapshots) is a big factor. I had everything up and running as soon as it stopped compiling and installing. I reckon if we did the maths.. most ricers spend more time compiling stuff than the time they save by their system being subjectively faster. The only platform I've ever seen a real benefit from compiling everything with arch specific GCC flags is the 32bit SPARCs. Old SPARCs didn't have a hardware multiply or divide (I forget which) so GCC built code that didn't use it even on SPARCs that had the instruction. There again switching from a Linux based system to NetBSD on that hardware increased real world performance by a big big margin without it rumbling away for days building single large packages like perl. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: pros/cons of installing from source
On Fri, May 04, 2007 at 12:42:40PM -0600, Paul E Condon wrote: I understand Greg's comments to be about Debian's commitment to enforcing a packaging policy, i.e. a policy on where and how things are installed. To me is quite a different thing than a social AOL policy. In Debian, if the install scripts of a package to not put things where the policy says they should be _that_ is a bug in the package. It may also be considered a bug in some other distro.s. I've not kept track of this sort of policy issue in any other distro. since I discovered Debian. It's not only a bug, it's a Release Critical bug. This means it is either fixed or the package will not make the stable release! How many distro's have this? Regards, Andrei -- If you can't explain it simply, you don't understand it well enough. (Albert Einstein) signature.asc Description: Digital signature
Re: pros/cons of installing from source
On 5/4/07, Andrew Sackville-West [EMAIL PROTECTED] wrote: On Fri, May 04, 2007 at 12:42:40PM -0600, Paul E Condon wrote: On Fri, May 04, 2007 at 10:34:27AM -0600, Javier Vasquez wrote: [heavy snippage dude] You mentioned debian commitment to FSF and its social contract, as very good reasons by themselves to run debian. I totally agree. However debian is not the only distro with such commitment. Actually sourceMage picked debian social contract and modified it a bit... snip... I understand Greg's comments to be about Debian's commitment to enforcing a packaging policy, i.e. a policy on where and how things are installed. To me is quite a different thing than a social policy. In Debian, if the install scripts of a package to not put things where the policy says they should be _that_ is a bug in the package. It may also be considered a bug in some other distro.s. I've not kept track of this sort of policy issue in any other distro. since I discovered Debian. The Social Policy is also good. But I think it is easy to feel good about a Social Policy, and it is hard work to implement a packaging policy. I think that the packaging policy is what really sets debian apart. THat's why everything just works... because dev's can count on things being a certain way and if its not, they can count on it being fixed. A Hmm, OK, we're changing the original topic now, but it's OK. I didn't want to comment more, but I think there's a confusion here. On a binary distribution you required a packaging policy, since you have different package developers, and in order to keep a coherent functional and robust system (dependencies, etc), you need to enforce a packaging policy. Debian packaging policy has demonstrated to me by far to be the best (personal opinion here), and not now, almost from the very beginning. However on a source based distribution, there's no different package developers, the admin of the system is the developer at the same time, and he/she is the one deciding what to compile against (libraries, dependencies whether strict or optional, etc). Furthermore, sourceMage, and probably other source based distros also have their own packaging policies. In sourceMage for example the spells, include a section for dependencies, just like in debian, and the required dependencies by upstream are included there. Beyond that there are 2 release branches, one stable, and the other testing, plus a development environment. Nothing goes to stable if the testing community is not satisfied about it. I think there's no way to compare packaging policy between a binary distro and a source based one. The philosophy is completely different. On a binary distro the policy is enforced to the distro package developers, while on a source based one the developer is oneself, and even considering that, there are policies enforced by the original application developer which are enforced... Remember that it's not entirely correct to compare oranges against apples. -- Javier -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: pros/cons of installing from source
On Fri, 2007-05-04 at 13:22 -0600, Javier Vasquez wrote: On 5/4/07, Andrew Sackville-West [EMAIL PROTECTED] wrote: On Fri, May 04, 2007 at 12:42:40PM -0600, Paul E Condon wrote: On Fri, May 04, 2007 at 10:34:27AM -0600, Javier Vasquez wrote: [heavy snippage dude] You mentioned debian commitment to FSF and its social contract, as very good reasons by themselves to run debian. I totally agree. However debian is not the only distro with such commitment. Actually sourceMage picked debian social contract and modified it a bit... snip... I understand Greg's comments to be about Debian's commitment to enforcing a packaging policy, i.e. a policy on where and how things are installed. To me is quite a different thing than a social policy. In Debian, if the install scripts of a package to not put things where the policy says they should be _that_ is a bug in the package. It may also be considered a bug in some other distro.s. I've not kept track of this sort of policy issue in any other distro. since I discovered Debian. The Social Policy is also good. But I think it is easy to feel good about a Social Policy, and it is hard work to implement a packaging policy. I think that the packaging policy is what really sets debian apart. THat's why everything just works... because dev's can count on things being a certain way and if its not, they can count on it being fixed. A Hmm, OK, we're changing the original topic now, but it's OK. I didn't want to comment more, but I think there's a confusion here. On a binary distribution you required a packaging policy, since you have different package developers, and in order to keep a coherent functional and robust system (dependencies, etc), you need to enforce a packaging policy. Debian packaging policy has demonstrated to me by far to be the best (personal opinion here), and not now, almost from the very beginning. Yes, only Debian does the Policy and enforcing of it. It is my whole point about policy. However on a source based distribution, there's no different package developers, the admin of the system is the developer at the same time, and he/she is the one deciding what to compile against (libraries, dependencies whether strict or optional, etc). Okay then, so how do you work through all the problems? Like a fail to compile due to a slight change in libfoozle that is incompatible with program blarfengangle's method of factoring? Furthermore, sourceMage, and probably other source based distros also have their own packaging policies. In sourceMage for example the spells, include a section for dependencies, just like in debian, and the required dependencies by upstream are included there. Beyond that there are 2 release branches, one stable, and the other testing, plus a development environment. Nothing goes to stable if the testing community is not satisfied about it. Again, just because something is compiled against something else does not needlessly mean that it is slower or faster or even better or worse. I'd like to know how you delve deep into the deps on things for HUGE packages such as X.org or OpenOffice or GNOME or KDE. Seem there has to be a packaging policy being foisted on you by someone. Which if you REALLY want to build you own Linux, you need to boot a LiveCD, Download source for the base libraries, compile them for a chroot. Install them in the chroot. Then compile the compilers and install them in the chroot. Then configure and compile your compilation tools (like autoconf, automake, m4, awk... etc) and install them. Finally chroot into the adhoc area. Then download and build the Kernel and other critical packages. Then a bootloader (grub/lilo/elilo/whetever) compile it and install it, then write the MBR for it. Once that is done, setup the booting stanzas and re-run the updating or what have you script to get it possible to boot from the disk. Once booted, you get to rebuild everything you just built, so you get native builds (and don't forget to heavily optimize) and not cross/non-native builds. From that point forward, you get to download HUGE tarballs and then configure them... grab pieces you forgot/missed to for things you want enabled, like DNS resolution is a good thing, recompile everything again to use DNS resolving libraries... Then restart the X compilation, finding yet another core thing usually caught... compression... egads yet another make world and the list goes on and on and on. I think there's no way to compare packaging policy between a binary distro and a source based one. Yes there is. Gentoo has one, SourceMage has one. Unless you force things into NON default locations, you are following packaging policy. Sort of like ports on *BSD. The philosophy is completely different. It actually is not very different at all. sourceMage requires things in particular places and to use specific calls. Just like
Re: pros/cons of installing from source
On Fri, May 04, 2007 at 01:22:03PM -0600, Javier Vasquez wrote: On 5/4/07, Andrew Sackville-West [EMAIL PROTECTED] wrote: On Fri, May 04, 2007 at 12:42:40PM -0600, Paul E Condon wrote: On Fri, May 04, 2007 at 10:34:27AM -0600, Javier Vasquez wrote: [heavy snippage dude] You mentioned debian commitment to FSF and its social contract, as very good reasons by themselves to run debian. I totally agree. However debian is not the only distro with such commitment. Actually sourceMage picked debian social contract and modified it a bit... snip... I understand Greg's comments to be about Debian's commitment to enforcing a packaging policy, i.e. a policy on where and how things are installed. To me is quite a different thing than a social policy. In Debian, if the install scripts of a package to not put things where the policy says they should be _that_ is a bug in the package. It may also be considered a bug in some other distro.s. I've not kept track of this sort of policy issue in any other distro. since I discovered Debian. The Social Policy is also good. But I think it is easy to feel good about a Social Policy, and it is hard work to implement a packaging policy. I think that the packaging policy is what really sets debian apart. THat's why everything just works... because dev's can count on things being a certain way and if its not, they can count on it being fixed. Hmm, OK, we're changing the original topic now, but it's OK. I didn't want to comment more, but I think there's a confusion here. On a binary distribution you required a packaging policy, since you have different package developers, and in order to keep a coherent functional and robust system (dependencies, etc), you need to enforce a packaging policy. Debian packaging policy has demonstrated to me by far to be the best (personal opinion here), and not now, almost from the very beginning. However on a source based distribution, there's no different package developers, the admin of the system is the developer at the same time, and he/she is the one deciding what to compile against (libraries, dependencies whether strict or optional, etc). Furthermore, sourceMage, and probably other source based distros also have their own packaging policies. In sourceMage for example the spells, include a section for dependencies, just like in debian, and the required dependencies by upstream are included there. Beyond that there are 2 release branches, one stable, and the other testing, plus a development environment. Nothing goes to stable if the testing community is not satisfied about it. are there not rules for make install that determine where things go? and what about standardization of config files and their locations? I think that a lot of the policy can be applied to source distributions just like a binary one. Its, of course, fairly easy as the end user to change things in a source based distro A signature.asc Description: Digital signature
Re: pros/cons of installing from source
Daniel Graham Palmer [EMAIL PROTECTED] writes: The only platform I've ever seen a real benefit from compiling everything with arch specific GCC flags is the 32bit SPARCs. You've never done any floating-point-intensive programming, apparently... Gcc may not be as good as icc in extreme cases, but it's certainly capable of quite a bit. -miles -- I'm beginning to think that life is just one long Yoko Ono album; no rhyme or reason, just a lot of incoherent shrieks and then it's over. --Ian Wolff -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
pros/cons of installing from source
Hi I would like to know whether installing from source rather than from the repositories has any advantage in terms of performance or something else. I didn't notice any difference when comparing mplayer's behavior, although compiling it myself gave me the choice of fine tuning the available codecs as well as installing the very latest version, for instance. Thanks -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: pros/cons of installing from source
On Thu, May 03, 2007 at 01:59:32PM +0100, yag wrote: Hi I would like to know whether installing from source rather than from the repositories has any advantage in terms of performance or something else. I didn't notice any difference when comparing mplayer's behavior, although compiling it myself gave me the choice of fine tuning the available codecs as well as installing the very latest version, for instance. I'm not sure mplayer is the right program to use for this purpose (comparing self-compiled vs. repository) unless you were to do a big transcoding job. Regardless, my understanding is the results are likely to be mixed. Its true that you can tweak compile-time options and possibily get better performance, but that's at the cost of convenience and possibly stability. THere are those who claim it really makes a difference and all that compile time is worth the few seconds you save later, but they mostly run gentoo ;) A signature.asc Description: Digital signature
Re: pros/cons of installing from source
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 yag wrote: Hi I would like to know whether installing from source rather than from the repositories has any advantage in terms of performance or something else. I didn't notice any difference when comparing mplayer's behavior, although compiling it myself gave me the choice of fine tuning the available codecs as well as installing the very latest version, for instance. In my experience, it is usually a lot simpler to install the Debian packages. They have already been formatted to work properly with Debian, some of which are tweaked a bit for better compatibility. Sure, you can compile things yourself, but then you have to do all the dependency checking yourself (although most software comes with a configure script to do this, they don't install the missing dependencies) so sometimes you have to STFW for the right files and may need to compile those first. That is the main reason APT exists, to save people those headaches. IMO, in most cases the small performance gain of self-compiling is not worth the effort. Joe - -- Registerd Linux user #443289 at http://counter.li.org/ -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFGOfX0iXBCVWpc5J4RAvxlAKC8gciA/csGzHZNMJcYB6aapp0m8gCfY0GJ YVUjYApsfjrjC+nBfk0xxW4= =MBV0 -END PGP SIGNATURE- -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: pros/cons of installing from source
On Thu, May 03, 2007 at 04:47:16PM +0200, Joe Hart wrote: yag wrote: Hi I would like to know whether installing from source rather than from the repositories has any advantage in terms of performance or something else. I didn't notice any difference when comparing mplayer's behavior, although compiling it myself gave me the choice of fine tuning the available codecs as well as installing the very latest version, for instance. In my experience, it is usually a lot simpler to install the Debian packages. They have already been formatted to work properly with Debian, some of which are tweaked a bit for better compatibility. Sure, you can compile things yourself, but then you have to do all the dependency checking yourself (although most software comes with a configure script to do this, they don't install the missing dependencies) so sometimes you have to STFW for the right files and may need to compile those first. apt-get build-dep package A signature.asc Description: Digital signature
Re: pros/cons of installing from source
Joe writes: Sure, you can compile things yourself, but then you have to do all the dependency checking yourself Download the Debian source package. Run 'apt-get build-dep package' to install the build dependencies. Edit the source to taste. Edit debian/changelog and up the version number. Run 'dpkg-buildpackage -rfakeroot -us -uc' to rebuild the package. Install your optimized package and be happy. -- John Hasler -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: pros/cons of installing from source
On 5/3/07, John Hasler [EMAIL PROTECTED] wrote: Joe writes: Sure, you can compile things yourself, but then you have to do all the dependency checking yourself Download the Debian source package. Run 'apt-get build-dep package' to install the build dependencies. Edit the source to taste. Edit debian/changelog and up the version number. Run 'dpkg-buildpackage -rfakeroot -us -uc' to rebuild the package. Install your optimized package and be happy. -- John Hasler This is useful if you're planning to have just a few packages compiled by yourself. If you plan to have most of the applications with custom configuration/compilation, then a binary distribution might not be the right thing, and maybe a source based distribution might be of a better taste (eg: gentoo or sourceMage). -- Javier -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: pros/cons of installing from source
On Thu, 2007-05-03 at 13:59 +0100, yag wrote: Hi I would like to know whether installing from source rather than from the repositories has any advantage in terms of performance or something else. I didn't notice any difference when comparing mplayer's behavior, although compiling it myself gave me the choice of fine tuning the available codecs as well as installing the very latest version, for instance. This is not Gentoo. Gentoo's vision of maximum performance is a great effort, but in reality is far from optimal. For every report of Woot! Compiling from source kicks butt. Why didn't I do this earlier, I can find 1 that disagrees with you and 1 that says maybe it is worth it for max performance, but WOW, 196 hours to get a workable complete system, I'm not so sure The statistics I find important are the massive amount of testing some have done (I leave that upto the reader to find, should you need help finding it, please ask the list to help find it). These people have done installs of LFS/Gentoo and other source distributions and highly optimized the compilation process. Following multiple best guides to compile by. In the end, it really depends on WHAT you want to accomplish. Given that most of the things that typically matter like word processing and surfing the internet and listening to music... playing cards, etc. I would hazard to say that: No it is not worth the time and effort to install from source The reason I say this, is even if you get 1%-5% improvement in performance, are you really going to see (really and truly feel) it? The answer is: no. Now, if you are doing nothing but ripping DVD or encoding MPEG files or doing full CGI animation renderings which sometimes take WEEKS to finish the sequence, then even 1% improvement may in fact be worth it. But then again, recompiling everything takes a very high amount of time and then you are just competing for processor time from the rendering. In business terms it would be cheaper to just ad a few more machines. One last point, if compiling from source is so great, why does Gentoo supply pre-compiled binaries for about 95% of the available packages that can be emerge'd on any one system? The answer is: Because compiling take a very long time and people are impatient. They want Gentoo for the elitism aspect of Gentoo, but none of the waiting. So, in summary, there are a few situations where compiling from source is desirable. But in general, you will not notice the difference. The only thing you will have done is add to the eventual heat death of the universe. -- greg, [EMAIL PROTECTED] Novell's Directory Services is a competitive product to Microsoft's Active Directory in much the same way that the Saturn V is a competitive product to those dinky little model rockets that kids light off down at the playfield. -- Thane Walkup signature.asc Description: This is a digitally signed message part
Re: pros/cons of installing from source
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Greg Folkert wrote: On Thu, 2007-05-03 at 13:59 +0100, yag wrote: Hi I would like to know whether installing from source rather than from the repositories has any advantage in terms of performance or something else. I didn't notice any difference when comparing mplayer's behavior, although compiling it myself gave me the choice of fine tuning the available codecs as well as installing the very latest version, for instance. This is not Gentoo. Gentoo's vision of maximum performance is a great effort, but in reality is far from optimal. For every report of Woot! Compiling from source kicks butt. Why didn't I do this earlier, I can find 1 that disagrees with you and 1 that says maybe it is worth it for max performance, but WOW, 196 hours to get a workable complete system, I'm not so sure The statistics I find important are the massive amount of testing some have done (I leave that upto the reader to find, should you need help finding it, please ask the list to help find it). These people have done installs of LFS/Gentoo and other source distributions and highly optimized the compilation process. Following multiple best guides to compile by. In the end, it really depends on WHAT you want to accomplish. Given that most of the things that typically matter like word processing and surfing the internet and listening to music... playing cards, etc. I would hazard to say that: No it is not worth the time and effort to install from source The reason I say this, is even if you get 1%-5% improvement in performance, are you really going to see (really and truly feel) it? The answer is: no. Now, if you are doing nothing but ripping DVD or encoding MPEG files or doing full CGI animation renderings which sometimes take WEEKS to finish the sequence, then even 1% improvement may in fact be worth it. But then again, recompiling everything takes a very high amount of time and then you are just competing for processor time from the rendering. In business terms it would be cheaper to just ad a few more machines. One last point, if compiling from source is so great, why does Gentoo supply pre-compiled binaries for about 95% of the available packages that can be emerge'd on any one system? The answer is: Because compiling take a very long time and people are impatient. They want Gentoo for the elitism aspect of Gentoo, but none of the waiting. So, in summary, there are a few situations where compiling from source is desirable. But in general, you will not notice the difference. The only thing you will have done is add to the eventual heat death of the universe. Exactly the point I was trying to make, but you said much better than I did. Greg, you really are an eloquent GNU/Linux guru. Kudos. Joe - -- Registerd Linux user #443289 at http://counter.li.org/ -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFGOhmliXBCVWpc5J4RAr94AJ9PYd8Eig0V4o1QTUuBnNqdpkkHiwCggleY qmMP/JvgPF42vSe5n1nyLQ4= =P3as -END PGP SIGNATURE- -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: pros/cons of installing from source
Biggest contenders for optimisation are the kernel and libc. Most applications are mainly a series of calls to these two so won't directly benefit from by being compiled with *magical* flags. Debian supply both optimised kernels and to a lesser degree optimised libc packages. Use of SIMD extensions etc is something totally different, and compiler flags aren't going to do all that much there really... Think of all the dolphins that will die because of the electricity wasted pointlessly compiling packages. ^^ -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: pros/cons of installing from source
For every report of Woot! Compiling from source kicks butt. Why didn't I do this earlier, I can find 1 that disagrees with you and 1 that says maybe it is worth it for max performance, but WOW, 196 hours to get a workable complete system, I'm not so sure ... The reason I say this, is even if you get 1%-5% improvement in performance, are you really going to see (really and truly feel) it? The answer is: no. ... Biggest contenders for optimisation are the kernel and libc. Most applications are mainly a series of calls to these two so won't directly benefit from by being compiled with *magical* flags. Debian supply both optimised kernels and to a lesser degree optimised libc packages. Use of SIMD extensions etc is something totally different, and compiler flags aren't going to do all that much there really... Think of all the dolphins that will die because of the electricity wasted pointlessly compiling packages. ^^ Well, I have no experience whatsoever with gentoo, but I have experimented with sourceMage. And I have most things compiled for the systems I need, and it doesn't' take that much time as I've heard about gentoo (no discussion it requires quiet more time to build a system than to install it). The most time I've spent is to learn about the new packaging system for the sources (taking care of dependencies, automatic download, testing/stable, etc, not that much though), and also about what is really required and what is not for a package (this requires the most time for me, since I'm used to just aptitude install what I need). BTW, the kernel is a good example not for optimization flags, neither dependencies, but for configuration. The stuck kernel supplied by debian is compiled to work for most systems, and supports a lot of hardware one might not need. A bit of optimizations can be granted by correctly selecting the cpu also, and by tuning the configurations as well (example, one might want to select pentium M for cpu architecture and the like). Of course this doesn't require a source based distro though, debian provides pretty good kernel compiling/packaging/installing tools. Also It's not only compiler optimizations what you get from source base distros, it's dependencies control. There are things one might compile against, that others might think as irrelevant. On binary based distros one can't control how the packages are compiled, thus one need to comply with the dependencies set by the developer. Which is OK for the most part, except if you want customized systems (whether lighter or even more blotted). It all depends on one's tastes and necessities. I for example am trying sourceMage in some machines, and found it amazingly easy and fast enough for the building part, and still I have debian installed in some others. I have to recognize that things given by granted on debian, need to be sometimes carefully looked on sourceMage though. Any ways, it's true that if you're planning to try a source based distro, you need to save time for that purpose, not just because of compilation time though, but also for learning about some packages and their lots of optional dependencies offered. If you still have doubts, you need to try, and then generate a personal criteria. You asked for pros and cons. I think the cons are very clear, but I wanted to complement the pros, because IMHO it's not just the optimizations flags what gets provided... BTW, you might still use a binary distro, and compile some critical things, as implicitly suggested by a previous post. The best example for such thing, as stated in a previous post is the kernel. Maybe others... -- Javier -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: pros/cons of installing from source
On Thu, 2007-05-03 at 13:10 -0600, Javier Vasquez wrote: For every report of Woot! Compiling from source kicks butt. Why didn't I do this earlier, I can find 1 that disagrees with you and 1 that says maybe it is worth it for max performance, but WOW, 196 hours to get a workable complete system, I'm not so sure ... The reason I say this, is even if you get 1%-5% improvement in performance, are you really going to see (really and truly feel) it? The answer is: no. ... Biggest contenders for optimisation are the kernel and libc. Most applications are mainly a series of calls to these two so won't directly benefit from by being compiled with *magical* flags. Debian supply both optimised kernels and to a lesser degree optimised libc packages. Use of SIMD extensions etc is something totally different, and compiler flags aren't going to do all that much there really... Think of all the dolphins that will die because of the electricity wasted pointlessly compiling packages. ^^ Well, I have no experience whatsoever with gentoo, but I have experimented with sourceMage. well, I just got done doing a FULL install starting with the 50MB install disk. Doing a full emerge of the kernel source (inside the chroot). Then did a full configure for a non-initrd kernel and specifically built it for this ONE machine. An XP3200+ Athlon with 2GB of memory and SATA-150 drives and DVD+-RW and a DVD-Reader, an nVidia 5950, a VIA chipsets (KT880 for AthlonXP chips and not Athlon64s). I did a full on build of the entire system. Libraries, Kernel, Desktop Environment (GNOME) with FireFox, OpenOffice.org 2.2, many tools I use everyday, plus some things I just like to have available should I need them. I used the full optimizations for ultimate power in the right places, like libc and the kernel and a few other areas that are sensitive to tweak, like encoders and decoders. Along with the entire two phase build process and the final stripping and other such niceties. The machine finally was done after 166 hours. And I have most things compiled for the systems I need, and it doesn't' take that much time as I've heard about gentoo (no discussion it requires quiet more time to build a system than to install it). The most time I've spent is to learn about the new packaging system for the sources (taking care of dependencies, automatic download, testing/stable, etc, not that much though), and also about what is really required and what is not for a package (this requires the most time for me, since I'm used to just aptitude install what I need). emerge is very simple tool to use, especially if you just hunker down and use it. Selecting the right architecture and listings (I used current snapshots) is a big factor. I had everything up and running as soon as it stopped compiling and installing. BTW, the kernel is a good example not for optimization flags, neither dependencies, but for configuration. The stuck kernel supplied by debian is compiled to work for most systems, and supports a lot of hardware one might not need. I will bet you that my Stock Debian kernel will run *JUST* as fast as your kernel will on the same exact hardware, once booted. You know why? I have Debian on the same machine I have Gentoo on. Both are reasonably close to each other in versions of most things. The only places I've seen REAL performance enhancement is in encoding of files. A bit of optimizations can be granted by correctly selecting the cpu also, and by tuning the configurations as well (example, one might want to select pentium M for cpu architecture and the like). Of course this doesn't require a source based distro though, debian provides pretty good kernel compiling/packaging/installing tools. Again, specifics pointed out. Pentium M is still a sub-arch of another arch. K7 is effective an i686, but slightly better instruction set for a few things Also It's not only compiler optimizations what you get from source base distros, it's dependencies control. Dependency controls... like what apt or aptitude does? There are things one might compile against, that others might think as irrelevant. On binary based distros one can't control how the packages are compiled, thus one need to comply with the dependencies set by the developer. Which is OK for the most part, except if you want customized systems (whether lighter or even more blotted). The way things are done and the speeds at which processors run, you will not feel the difference on a package compiled additionally against xlibs as ones without that compiled option. I dare you to REALLY show me qualitative and empirical proof that you get 3 more cycles per second of processing time for something like the gd package. You'd have to be doing MOUNTAINS of processing to even being to show the difference. But then, an apparent 3 more cycles might be coming from your recent change in Apache to cache more... so more IO is
Re: pros/cons of installing from source
On 5/3/07, Greg Folkert [EMAIL PROTECTED] wrote: ... Also It's not only compiler optimizations what you get from source base distros, it's dependencies control. Dependency controls... like what apt or aptitude does? There are things one might compile against, that others might think as irrelevant. On binary based distros one can't control how the packages are compiled, thus one need to comply with the dependencies set by the developer. Which is OK for the most part, except if you want customized systems (whether lighter or even more blotted). Nope, aptitude offers you the dependencies the distro developer specifies (not just the application developer), some of them are recommendations, some of them are strictly required. When you are to compile the application yourself, you can find that even things strictly required by a binary distro are really not. The reason is that the distro developer compiled using a particular library for example, when he/she could have used another or none. So on binary distros one has 2 levels of non optional dependencies I believe, the ones set by the original package developer, and the ones set by the distro developer for the package. This is not true on sourceMage, not sure on gentoo (it looks like people immediately thinks of gentoo when talking about source based distros) since I don't know about it, and it's just because the only really required dependencies on sourceMage by policy are the ones set by the original package developer. Whether this makes a difference or not, it depends on the system one wants to get. I might be wrong though about how debian package developers compile things though, I'm not one, and it might be that there's a policy to keep as required dependencies only the ones set by upstream, but I'm not aware of it. As a side note, something I liked from sourceMage was its policy of keeping upstream code untouched as much as possible. I don't know of any binary distro trying to keep up with that. However this is beyond the discussion since there's a lot to talk about that, just something to mention, :). For anything else I agree. Just wanted to clarify a bit further about the dependencies comment. For not compiling the kernel as a suggestion, well, again it depends (I don't totally agree). For a regular user with 40GB of HD or more, there's no problem on having a blotted set of modules he/she will never use. If you have limited HD, you'd like to compile only what you need, and not everything so far supported by the kernel (besides you get more tunned configuration at the same time for free if you want, I provided the pentium M example, but I bet there are more, like the kind of pre-emption, the frequency, etc, not that one gets better performance, but that one gets the right tunned configuration for the system, and not just a blotted generic one). Same thing applies to other packages. One might want to remove any gnome/QT dependency as much as possible, one might not support some graphics libraries although required for the general purpose, etc. So far I'm living with both, debian, which I'm fond of (just a user though for several years now), and just started sourceMage, as I mentioned before, and I wouldn't just demerit source based distros just because of the build time if that's what one needs, and I wouldn't suggest source based distros just because of performance either (the arguments about this are very clear, and there's nothing to discuss about that). Again, I think it's a matter of tastes and necessities. But again, I might have the concepts about the whole thing completely twisted, since I'm just an user of both distros, :). Please, if I'm completely wrong about my comments on dependencies, let me know. Maybe there's a debian policiy talking about this (is there a pointer?) that I'm not aware of, and I was just talking non sense, :). -- Javier -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: pros/cons of installing from source
On Thu, 2007-05-03 at 22:38 -0600, Javier Vasquez wrote: On 5/3/07, Greg Folkert [EMAIL PROTECTED] wrote: ... Also It's not only compiler optimizations what you get from source base distros, it's dependencies control. Dependency controls... like what apt or aptitude does? There are things one might compile against, that others might think as irrelevant. On binary based distros one can't control how the packages are compiled, thus one need to comply with the dependencies set by the developer. Which is OK for the most part, except if you want customized systems (whether lighter or even more blotted). Nope, aptitude offers you the dependencies the distro developer specifies (not just the application developer), some of them are recommendations, some of them are strictly required. When you are to compile the application yourself, you can find that even things strictly required by a binary distro are really not. The reason is that the distro developer compiled using a particular library for example, when he/she could have used another or none. So on binary distros one has 2 levels of non optional dependencies I believe, the ones set by the original package developer, and the ones set by the distro developer for the package. This is not true on sourceMage, not sure on gentoo (it looks like people immediately thinks of gentoo when talking about source based distros) since I don't know about it, and it's just because the only really required dependencies on sourceMage by policy are the ones set by the original package developer. Whether this makes a difference or not, it depends on the system one wants to get. I specifically picked gd for that very reason. It supports eleventy options. The reason I picked it, is because the linked set of libraries for Debian pulls in some xlibs on even cursor based systems. Basically the changelog said something like: the linking of the code against xlibs, only slightly increases the pull in of files amounting to 72KiB, these days this small amount of disk space does not matter. The performance is not affected in any way, but allows for 98% coverage and reduces package count by 12 flavors. If you must have no xlibs, compile it yourself without it. Which, to be honest, is the exact same reason people restore or rebuild classic cars or heavily customize the ricers they own, or build thier own house or hand craft the Linux Distro of their choice. I might be wrong though about how debian package developers compile things though, I'm not one, and it might be that there's a policy to keep as required dependencies only the ones set by upstream, but I'm not aware of it. There is the Debian Developers Guide and the Debian Free software Guide. These BOTH have an effect on the Original Source code. BTW, you do know that *EXCEPT* for non-free pieces (like non-source firmware and binary blobs) that Debian include *.orig.tar.gz for everything? They also have a *.diff.tar.gz... so following your comment about keeping upstream untouched as much as possible is not-genuine. Debian does this, but at the same time folowing the DFSG. As a side note, something I liked from sourceMage was its policy of keeping upstream code untouched as much as possible. I don't know of any binary distro trying to keep up with that. However this is beyond the discussion since there's a lot to talk about that, just something to mention, :). I mentioned Debian Policy (Set forth by the Debian Free Software Guide) as being the BEST reason to run Debian Linux... or Debian FreeBSD or Debian period. For anything else I agree. Just wanted to clarify a bit further about the dependencies comment. For not compiling the kernel as a suggestion, well, again it depends (I don't totally agree). For a regular user with 40GB of HD or more, there's no problem on having a blotted set of modules he/she will never use. If you have limited HD, you'd like to compile only what you need, and not everything so far supported by the kernel (besides you get more tunned configuration at the same time for free if you want, I provided the pentium M example, but I bet there are more, like the kind of pre-emption, the frequency, etc, not that one gets better performance, but that one gets the right tunned configuration for the system, and not just a blotted generic one). Same thing applies to other packages. One might want to remove any gnome/QT dependency as much as possible, one might not support some graphics libraries although required for the general purpose, etc. Good enough, I could pick, but won't. :-P So far I'm living with both, debian, which I'm fond of (just a user though for several years now), and just started sourceMage, as I mentioned before, and I wouldn't just demerit source based distros just because of the build time if that's what one needs, and I wouldn't suggest source based
