Re: make -j4 vs -j8... 4 works, but 8 does not
On Sat, Sep 15, 2001 at 06:06:10PM -0400, Garance A Drosihn wrote: > [aside on those multiple builds that I did: It was interesting that > even though it was on a dual-processor system, there was not much of > a speed improvement (on 4.3-stable) when going from -j4 to -j10. Big > improvement going from -j1 to -j4, but after that it didn't help much] That sounds exactly like what I would expect. More than 2 (or maybe 3) jobs per CPU is normally not going to make things go faster but might actually slow it down. Increasing the number of jobs running in parallell up to the number of CPUs will naturally increase performance. (Since otherwise you have some unused processors.) The reason why running more than one job/CPU might be good is that the processes uses a mixture of I/O and CPU. If you run two jobs in parallell then one can use the CPU while the other waits for I/O. This increases the efficiency in total resource usage. If you run more than that things will not improve much since you can only have one process using a CPU at a particular time. And if you have several processes trying to access a disk at the same time the disk will spend a lot of time seeking back and forth which is a sure way of making a system slow. So for a system with P processors the speed improvements for using -jN with N > 2*P is normally going to be fairly small. (While the speed improvments when increasing N from 1 up to 2*P can often be noticable, especially when going from 1 to P.) -- Erik Trulsson [EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-stable" in the body of the message
Re: make -j4 vs -j8... 4 works, but 8 does not
On Sat, Sep 15, 2001 at 06:06:10PM -0400, Garance A Drosihn wrote: > At 12:38 PM -0400 9/15/01, Mike Tancsa wrote: > >Should a parallel build always work ? I was just trying to stress > >a new series of MB we are evaluating and to my suprise, -j4 works, > >but not -j8 > > Well, in a philosophical sense, yes it "should" always work. Bugs > creep into the process from time-to-time. I can not say that I > know how to pin down such problems when they occur. I do know > that earlier this year I had some problem I was checking, and as > a tangent to that problem I did several fresh 'make buildworld's, > going from -j2 to -j10 on my dual 650-MHz pentium machine. I then > did md5 comparisons of the resulting obj-tree results, and they all > came out the same. Of course, I wasn't getting any errors at build > time, either. In my experience with a 2-CPU box, the problems with -j 5 or bigger are exclusively with module (or include file, etc.) A not being built by the time that process B needs it to complete a compile or link or whatever. These failures, in general, don't recur if the make is rerun from the failure point -- even with -j 8 or more. I frequently run with -j 16 when I'm doing kernel builds and buildworlds, and I haven't had any of those fail in a longish time. -- Mike Andrews [EMAIL PROTECTED] Tired old sysadmin since 1964 To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-stable" in the body of the message
Re: make -j4 vs -j8... 4 works, but 8 does not
At 06:06 PM 9/15/2001 -0400, Garance A Drosihn wrote: What happens if you remove all of /usr/obj/usr/src before trying >to build with -j8? I did each time. One poster pointed out, the CPU type was wrong. It should be i686, not 686. This is on a DURON 900 with AOPen Duron specific MB. I just did another build/installworld with the correct CPU type. I will reboot and try again to see if that was the problem. BTW, what CPUTYPE are people using for the AMD DURON ? ---Mike Mike Tancsa, tel +1 519 651 3400 Sentex Communications,[EMAIL PROTECTED] Providing Internet since 1994www.sentex.net Cambridge, Ontario Canada www.sentex.net/mike To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-stable" in the body of the message
Re: make -j4 vs -j8... 4 works, but 8 does not
At 12:38 PM -0400 9/15/01, Mike Tancsa wrote: >Should a parallel build always work ? I was just trying to stress >a new series of MB we are evaluating and to my suprise, -j4 works, >but not -j8 Well, in a philosophical sense, yes it "should" always work. Bugs creep into the process from time-to-time. I can not say that I know how to pin down such problems when they occur. I do know that earlier this year I had some problem I was checking, and as a tangent to that problem I did several fresh 'make buildworld's, going from -j2 to -j10 on my dual 650-MHz pentium machine. I then did md5 comparisons of the resulting obj-tree results, and they all came out the same. Of course, I wasn't getting any errors at build time, either. What happens if you remove all of /usr/obj/usr/src before trying to build with -j8? >===> usr.sbin/boot0cfg >rm -f .depend >mkdep -f .depend -a-I/usr/obj/usr/src/i386/usr/include >/usr/src/usr.sbin/boot0cfg/boot0cfg.c >cd /usr/src/usr.sbin/boot0cfg; make _EXTRADEPEND >echo boot0cfg: /usr/obj/usr/src/i386/usr/lib/libc.a >> .depend >1 error >*** Error code 2 >1 error >*** Error code 2 the thing which interests me about the above is that 'make' aborts with "error code 2", but I don't see any error message as to what the error was. I wonder if this is another case in "/bin/sh -e" where an error return is NOT being ignored when it should be ignored. I fixed two similar bugs last summer last summer. [aside on those multiple builds that I did: It was interesting that even though it was on a dual-processor system, there was not much of a speed improvement (on 4.3-stable) when going from -j4 to -j10. Big improvement going from -j1 to -j4, but after that it didn't help much] -- Garance Alistair Drosehn= [EMAIL PROTECTED] Senior Systems Programmer or [EMAIL PROTECTED] Rensselaer Polytechnic Instituteor [EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-stable" in the body of the message
