Re: [gentoo-user] DistCC + Portage Questions
Nick Rout wrote: On Thu, 03 Feb 2005 23:27:59 +0100 Philip Lawatsch [EMAIL PROTECTED] wrote: This will for sure work but is definitely not what I intend. Simply overriding gcc by symlinking it to distcc is not what I want. Thats exactly what distcc does, it puts a gcc wrapper in the PATH ahead of gcc. I was merely trying to establish if distcc was working. OTOH if it works, then it will solve your problem until we find the real answer. Really? I do have the impressions that distcc should be used instead of gcc and use kind of a patch attack against gcc :) Anyway, it works if i make sure that the gcc in $PATH being used is a symlink to gcc. Its just strange that emerge does not do this for me. It looks to me like a problem with emerge not setting the PATH properly when compiling, so it gets to the real gcc before the wrapper. Take a look in /etc/env.d and see if it all looks fine. -8 # This file is managed by distcc-config; use it to change these settings. DISTCC_LOG= DCCC_PATH=/usr/lib/distcc/bin DISTCC_VERBOSE=0 8 Is in 02distcc. Starting to wonder why its called DCCC_PATH and not DISTCC_PATH ... I'd like to let emerge do this work for me. Look I don't have a magic answer, i am just trying to help work out the problem. Perhaps you should email Lisa Seelye direct - she is the gentoo distcc maintainer, or else the gentoo-dev list may be able to offer some help. Will give it a try, and I do know that you're trying to help and I also do appreciate it! As I said, if i manually use distcc it works. Also I doubt that a package can disable distcc if it does not use something advanced to check for the real gcc. If you just disable parallel make you wont automagically stop distcc from being used afair. Yes I guess that is true, but then how do you know distcc is being used with -j1, with the -j1 being set by the package (or perhaps the ebuild). An ebuild could also presumably over ride FEATURES Yes, it could override it but none of the ones I tried did override it. I'd like to know why emerge itself does not use distcc even though I tell it to by setting features to distcc. me too. what versions of distcc and gcc? gcc [ I] 3.3.5-r1 (3.3) sys-devel/distcc : [ I] 2.16-r2 (0) -ph- -- gentoo-user@gentoo.org mailing list
Re: [gentoo-user] DistCC + Portage Questions
Philip Lawatsch wrote: The other machines are in /etc/distcc/hosts and I cant ping / ssh to them. tv1 / # distcc-config --get-hosts localhost icebear icebird Still emerge does not use distcc to compile for instance xine-lib or bison or perl. I'm not sure what I missed to do, perhaps someone here has a hint for me. And, another far worse problem I have is that in my case I'm having at least 8 parallel compile jobs running on my machine which really criples it. As far as I understood if the Makefile does support parallel building I should also be able to use distcc. But it looks like all the autoconf scripts try to use gcc / g++ instead of distcc for as the compiler. Some packages can't be built in parallel, your best bet to not cripple the localhost is the put a limit on how many jobs it should allow Example: distcc-config --set-hosts localhost/2 icebear icebird Will limit localhost to 2 active compile jobs at a time... Take a look at the distcc manual and the nice document Lisa has put together on the Gentoo web /Andreas -- gentoo-user@gentoo.org mailing list
Re: [gentoo-user] DistCC + Portage Questions
Andreas Vinsander wrote: Philip Lawatsch wrote: The other machines are in /etc/distcc/hosts and I cant ping / ssh to them. Sorry, this is a typo, of course i _can_ ping / ssh to them, otherwise this would not make much sense ... tv1 / # distcc-config --get-hosts localhost icebear icebird Still emerge does not use distcc to compile for instance xine-lib or bison or perl. I'm not sure what I missed to do, perhaps someone here has a hint for me. And, another far worse problem I have is that in my case I'm having at least 8 parallel compile jobs running on my machine which really criples it. As far as I understood if the Makefile does support parallel building I should also be able to use distcc. But it looks like all the autoconf scripts try to use gcc / g++ instead of distcc for as the compiler. Some packages can't be built in parallel, your best bet to not cripple the localhost is the put a limit on how many jobs it should allow Example: distcc-config --set-hosts localhost/2 icebear icebird Will limit localhost to 2 active compile jobs at a time... Take a look at the distcc manual and the nice document Lisa has put together on the Gentoo web I know all of that and I've already set up the distccd on the local host in this way (not via distcc_hosts but by configuring distccd on localhost). BUT, for some reason portage does _not_ use distcc at all but the normal gcc and this will result in -jN jobs running on localhost. kind regards Philip -- gentoo-user@gentoo.org mailing list
Re: [gentoo-user] DistCC + Portage Questions
Philip Lawatsch wrote: I know all of that and I've already set up the distccd on the local host in this way (not via distcc_hosts but by configuring distccd on localhost). BUT, for some reason portage does _not_ use distcc at all but the normal gcc and this will result in -jN jobs running on localhost. Ok, then what do u have in /etc/conf.d/distccd on the different hosts U need to configure it to listen to connections from your subnet or specific hosts /Andreas -- gentoo-user@gentoo.org mailing list
RE: [gentoo-user] DistCC + Portage Questions
I know all of that and I've already set up the distccd on the local host in this way (not via distcc_hosts but by configuring distccd on localhost). BUT, for some reason portage does _not_ use distcc at all but the normal gcc and this will result in -jN jobs running on localhost. Phil, I've noticed the same lack of distcc usage that you've encountered. I haven't isolated the cause (i.e. I think portage has been updated twice since it started to fail for me), but I can assure you that it's not necessarily anything you're doing... -- gentoo-user@gentoo.org mailing list
Re: [gentoo-user] DistCC + Portage Questions
Andreas Vinsander wrote: Philip Lawatsch wrote: I know all of that and I've already set up the distccd on the local host in this way (not via distcc_hosts but by configuring distccd on localhost). BUT, for some reason portage does _not_ use distcc at all but the normal gcc and this will result in -jN jobs running on localhost. Ok, then what do u have in /etc/conf.d/distccd on the different hosts U need to configure it to listen to connections from your subnet or specific hosts My distcc setup is verified to work (compiling something else on the machines right now). My problem is that emerge simply does ignore FEATURES=distcc and instead uses gcc / g++ directly. Either emerge fails to tell the packages autoconf script to use distcc as a compiler or something else. In case my distcc setup would be broken I'd get complaints from distcc about it, or get at least something in the distccd log on the local machine. But I get neither. kind regards Philip -- gentoo-user@gentoo.org mailing list
Re: [gentoo-user] DistCC + Portage Questions
Philip Lawatsch wrote: My distcc setup is verified to work (compiling something else on the machines right now). My problem is that emerge simply does ignore FEATURES=distcc and instead uses gcc / g++ directly. Either emerge fails to tell the packages autoconf script to use distcc as a compiler or something else. Yep, that sounds strange... What about environment variables? On my system the distcc ebuild has installed the file /etc/env.d/02distcc Are the variables mentioned in there available in your actual environment? env-update source /etc/profile might be helpful... Looking at /usr/lib/portage/bin/ebuild.sh there are some references to distcc, take look and try to find out if some of the tests found might cripple your portage's use of distcc I have run out of ideas... sorry! /Andreas -- gentoo-user@gentoo.org mailing list
Re: [gentoo-user] DistCC + Portage Questions
are you setting up distcc using distcc-config?
what does the logfile say?
distcc-config --set-hosts host list
distcc-config set-verbose 1
distcc-config set-log /var/log/distcc
touch /var/log/distcc
emerge blah
tools to analyse:
tail -f /var/log/distcc (on the compiling machine)
DISTCC_DIR=/var/tmp/portage/.distcc distccmon-text 1 (ditto)
tcpdump port 3632 (on either machine)
On Thu, 03 Feb 2005 18:02:06 +0100
Philip Lawatsch [EMAIL PROTECTED] wrote:
Hi,
I'm trying to use distcc for building my system.
I've yet encountered a rather strange problem, I've added
CFLAGS=-O2 -pipe -march=athlon-xp -fomit-frame-pointer
CHOST=i686-pc-linux-gnu
CXXFLAGS=${CFLAGS}
MAKEOPTS=-j8
FEATURES=distcc
to my make.conf.
I've got the daemons running on the other machines (and also the local
host).
The other machines are in /etc/distcc/hosts and I cant ping / ssh to them.
tv1 / # distcc-config --get-hosts
localhost icebear icebird
Still emerge does not use distcc to compile for instance xine-lib or
bison or perl.
I'm not sure what I missed to do, perhaps someone here has a hint for me.
And, another far worse problem I have is that in my case I'm having at
least 8 parallel compile jobs running on my machine which really criples
it. As far as I understood if the Makefile does support parallel
building I should also be able to use distcc. But it looks like all the
autoconf scripts try to use gcc / g++ instead of distcc for as the compiler.
Any help appreciated.
kind regards Philip
--
gentoo-user@gentoo.org mailing list
--
Nick Rout [EMAIL PROTECTED]
--
gentoo-user@gentoo.org mailing list
Re: [gentoo-user] DistCC + Portage Questions
Nick Rout wrote: are you setting up distcc using distcc-config? what does the logfile say? distcc-config --set-hosts host list distcc-config set-verbose 1 distcc-config set-log /var/log/distcc touch /var/log/distcc emerge blah tools to analyse: tail -f /var/log/distcc (on the compiling machine) DISTCC_DIR=/var/tmp/portage/.distcc distccmon-text 1 (ditto) tcpdump port 3632 (on either machine) Nothing since distcc is not used by emerge but plain old gcc. -ph- -- gentoo-user@gentoo.org mailing list
Re: [gentoo-user] DistCC + Portage Questions
On Thu, 03 Feb 2005 22:02:36 +0100 Philip Lawatsch [EMAIL PROTECTED] wrote: Nick Rout wrote: are you setting up distcc using distcc-config? what does the logfile say? distcc-config --set-hosts host list distcc-config set-verbose 1 distcc-config set-log /var/log/distcc touch /var/log/distcc emerge blah tools to analyse: tail -f /var/log/distcc (on the compiling machine) DISTCC_DIR=/var/tmp/portage/.distcc distccmon-text 1 (ditto) tcpdump port 3632 (on either machine) Nothing since distcc is not used by emerge but plain old gcc. can i suggest you try setting PATH on the emerge command line and see what happens? PATH=/usr/lib/distcc/bin;$PATH emerge blah make sure blah is a package that does not disable distcc by disabling parallel makes, I know for sure mc will use distcc so i often test with that (if you don't want mc you can ctrl-c it before it finishes compiling). I also know Xorg-x11 disables parallel makes. with other packages YMMV -ph- -- gentoo-user@gentoo.org mailing list -- Nick Rout [EMAIL PROTECTED] -- gentoo-user@gentoo.org mailing list
Re: [gentoo-user] DistCC + Portage Questions
Nick Rout wrote: are you setting up distcc using distcc-config? what does the logfile say? Nothing since distcc is not used by emerge but plain old gcc. can i suggest you try setting PATH on the emerge command line and see what happens? PATH=/usr/lib/distcc/bin;$PATH emerge blah make sure blah is a package that does not disable distcc by disabling parallel makes, I know for sure mc will use distcc so i often test with that (if you don't want mc you can ctrl-c it before it finishes compiling). I also know Xorg-x11 disables parallel makes. with other packages YMMV This will for sure work but is definitely not what I intend. Simply overriding gcc by symlinking it to distcc is not what I want. I'd like to let emerge do this work for me. As I said, if i manually use distcc it works. Also I doubt that a package can disable distcc if it does not use something advanced to check for the real gcc. If you just disable parallel make you wont automagically stop distcc from being used afair. I'd like to know why emerge itself does not use distcc even though I tell it to by setting features to distcc. kind regards Philip -- gentoo-user@gentoo.org mailing list
Re: [gentoo-user] DistCC + Portage Questions
On Thu, 03 Feb 2005 23:27:59 +0100 Philip Lawatsch [EMAIL PROTECTED] wrote: This will for sure work but is definitely not what I intend. Simply overriding gcc by symlinking it to distcc is not what I want. Thats exactly what distcc does, it puts a gcc wrapper in the PATH ahead of gcc. I was merely trying to establish if distcc was working. OTOH if it works, then it will solve your problem until we find the real answer. It looks to me like a problem with emerge not setting the PATH properly when compiling, so it gets to the real gcc before the wrapper. Take a look in /etc/env.d and see if it all looks fine. I'd like to let emerge do this work for me. Look I don't have a magic answer, i am just trying to help work out the problem. Perhaps you should email Lisa Seelye direct - she is the gentoo distcc maintainer, or else the gentoo-dev list may be able to offer some help. As I said, if i manually use distcc it works. Also I doubt that a package can disable distcc if it does not use something advanced to check for the real gcc. If you just disable parallel make you wont automagically stop distcc from being used afair. Yes I guess that is true, but then how do you know distcc is being used with -j1, with the -j1 being set by the package (or perhaps the ebuild). An ebuild could also presumably over ride FEATURES I'd like to know why emerge itself does not use distcc even though I tell it to by setting features to distcc. me too. what versions of distcc and gcc? kind regards Philip -- Nick Rout [EMAIL PROTECTED] -- gentoo-user@gentoo.org mailing list
Re: [gentoo-user] DistCC + Portage Questions
It actually doesnt stop distcc - just runs a single make job on the other machine. Very useful for low powered machines as you dont specify localhost so all the makes are remote, even if just one machine (should be, doesnt always happen tho!) To stop distcc use FEATURES=-distcc However, this thread is of use to me as I also have a laptop that plain refuses to distcc anything. Havent had time trace it yet, but the symptoms are similar to what others are reporting. Only difference with my usual inastall method is that I tried to set up distcc according to the gentoo guide so it would be used during bootstrap - it didnt work there either. Think were looking at some kind of configuration bug that some systems trigger? BillK On Fri, 2005-02-04 at 11:55 +1300, Nick Rout wrote: On Thu, 03 Feb 2005 23:27:59 +0100 Philip Lawatsch [EMAIL PROTECTED] wrote: Also I doubt that a package can disable distcc if it does not use something advanced to check for the real gcc. If you just disable parallel make you wont automagically stop distcc from being used afair. Yes I guess that is true, but then how do you know distcc is being used with -j1, with the -j1 being set by the package (or perhaps the ebuild). An ebuild could also presumably over ride FEATURES -- William Kenworthy [EMAIL PROTECTED] Home! -- gentoo-user@gentoo.org mailing list
