Re: Clarification needed on Handbook: Tracking for Multiple Machines
On Tue, 24 Feb 2004, Tony Frank wrote: [snip] TF> > > : If "True", for optimized code across all machines, the code should TF> > > : just be built on each machine, right? TF> > > That would give slightly better performance. However, it can be more TF> > > pain than it is worth if the number of machine types is high. TF> > Consuming considerably more time and disk space, a shell script to TF> > alter make.conf and rename /usr/obj between the build for each machine TF> > is doable, though pro'lly not worth it. The install at each each box TF> > would just have to mount it's corresponding /usr/obj tree. TF> TF> Just need to set __MAKE_CONF and MAKEOBJDIRPREFIX before running the build. Unfortunately not, as some utilities (from phase III) will be linked against existing system libraries. Or, you should follow the process like 'make release': build world with lower CPU optimization, intall it to the sandbox, chroot to it and build world again. This was explained by ru@ some months ago [1,2] [1]: http://www.geocrawler.com/archives/3/152/2002/5/0/8811539/ [2]: http://groups.google.com/groups?hl=ru&lr=&ie=UTF-8&inlang=ru&threadm=fa.f0qca6v.f7g49v%40ifi.uio.no&rnum=4&prev=/groups%3Fq%3Dfreebsd%2Bruslan%2Bermilov%2BCPUTYPE%2Bworld%2Bproblems%2Bwhen%2Brunning%2Bmake%26hl%3Dru%26lr%3D%26ie%3DUTF-8%26inlang%3Dru%26selm%3Dfa.f0qca6v.f7g49v%2540ifi.uio.no%26rnum%3D4 Sincerely, D.Marck [DM5020, MCK-RIPE, DM3-RIPN] *** Dmitry Morozovsky --- D.Marck --- Wild Woozle --- [EMAIL PROTECTED] *** ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: Clarification needed on Handbook: Tracking for Multiple Machines
Hi, On Mon, Feb 23, 2004 at 10:46:07AM -0600, D J Hawkey Jr wrote: > On Feb 23, at 09:12 AM, M. Warner Losh wrote: > > > > In message: <[EMAIL PROTECTED]> > > D J Hawkey Jr <[EMAIL PROTECTED]> writes: > > : True or False: Setting CPUTYPE to the lowest target CPU ("p2") in > > : a build machine's make.conf will cripple the performance of target > > : machines with higher CPUs ("p3", "p4", "i586", "i686", etc.). > > > > False. It might have a minor impact on performance, but not a major > > one. At least in my experience. Minor here means < 10% for something > > like the world stone. Cripple to me implies > 25%. > > OK, thanks. Just to satisfy my anal-retentive side, would that ~10% > degradation be a higher level of performance on a PIII (or higher) > with no CPUTYPE specified at all, given the same *FLAGS? Perhaps the easiest option is to actually try it and see what the results are? > > : If "True", for optimized code across all machines, the code should > > : just be built on each machine, right? > > That would give slightly better performance. However, it can be more > > pain than it is worth if the number of machine types is high. > Consuming considerably more time and disk space, a shell script to > alter make.conf and rename /usr/obj between the build for each machine > is doable, though pro'lly not worth it. The install at each each box > would just have to mount it's corresponding /usr/obj tree. Just need to set __MAKE_CONF and MAKEOBJDIRPREFIX before running the build. Regards, Tony ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: Clarification needed on Handbook: Tracking for Multiple Machines
On Feb 23, at 09:12 AM, M. Warner Losh wrote: > > In message: <[EMAIL PROTECTED]> > D J Hawkey Jr <[EMAIL PROTECTED]> writes: > : True or False: Setting CPUTYPE to the lowest target CPU ("p2") in > : a build machine's make.conf will cripple the performance of target > : machines with higher CPUs ("p3", "p4", "i586", "i686", etc.). > > False. It might have a minor impact on performance, but not a major > one. At least in my experience. Minor here means < 10% for something > like the world stone. Cripple to me implies > 25%. OK, thanks. Just to satisfy my anal-retentive side, would that ~10% degradation be a higher level of performance on a PIII (or higher) with no CPUTYPE specified at all, given the same *FLAGS? > : If "True", for optimized code across all machines, the code should > : just be built on each machine, right? > > That would give slightly better performance. However, it can be more > pain than it is worth if the number of machine types is high. Consuming considerably more time and disk space, a shell script to alter make.conf and rename /usr/obj between the build for each machine is doable, though pro'lly not worth it. The install at each each box would just have to mount it's corresponding /usr/obj tree. Thanks, Warner, Dave -- __ __ \__ \D. J. HAWKEY JR. / __/ \/\ [EMAIL PROTECTED]/\/ http://www.visi.com/~hawkeyd/ ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: Clarification needed on Handbook: Tracking for Multiple Machines
In message: <[EMAIL PROTECTED]> D J Hawkey Jr <[EMAIL PROTECTED]> writes: : True or False: Setting CPUTYPE to the lowest target CPU ("p2") in : a build machine's make.conf will cripple the performance of target : machines with higher CPUs ("p3", "p4", "i586", "i686", etc.). False. It might have a minor impact on performance, but not a major one. At least in my experience. Minor here means < 10% for something like the world stone. Cripple to me implies > 25%. : If "True", for optimized code across all machines, the code should : just be built on each machine, right? That would give slightly better performance. However, it can be more pain than it is worth if the number of machine types is high. Warner ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: Clarification needed on Handbook: Tracking for Multiple Machines
OK, I've cross-posted this message to -hackers, to see if we can get some sort of definitive [to me] answer. Please forgive if it's considered bad form. -hackers: There is a thread in -questions in response to my query as to building the world and kernels for a variety of Intel CPUs on one machine. For brevity's sake, I won't reproduce the entire thread here. OK, I guess my question boils down to these, then: True or False: Setting CPUTYPE to the lowest target CPU ("p2") in a build machine's make.conf will cripple the performance of target machines with higher CPUs ("p3", "p4", "i586", "i686", etc.). If "True", for optimized code across all machines, the code should just be built on each machine, right? Thanks, Dave -- __ __ \__ \D. J. HAWKEY JR. / __/ \/\ [EMAIL PROTECTED]/\/ http://www.visi.com/~hawkeyd/ ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: Clarification needed on Handbook: Tracking for Multiple Machines
Hi, On Sun, Feb 22, 2004 at 09:07:23AM -0600, D J Hawkey Jr wrote: > On Feb 23, at 01:05 AM, Tony Frank wrote: > > > > On Sat, Feb 21, 2004 at 11:23:28AM -0600, D J Hawkey Jr wrote: > > > On Feb 21, at 05:56 PM, Gabriel Ambuehl wrote: > > > > > > > > DJHJ> Second, two machines are of the same architecture, but they have > > > > different > > > > DJHJ> CPUs: One is an Intel PIII, but the other is a PII. Will the world built > > > > DJHJ> on a PIII be correct for a PII? Similarly, will the kernel for the PII > > > > DJHJ> built on a PIII be correct for the PII, given the different variables and > > > > DJHJ> settings in the two kernel configuration files? > > > > > > > > Just make sure you build for 686. If that doesn't work, make it 586 (I > > > > think the PI qualifies as 686 but I'm not entirely sure). I think the > > > > extensions such as > > > > SSE etc are detected dynamically and shouldn't cause any problem. > > > > In all my years of messing with builds, I never run into this problem, > > > > so I guess it's pretty safe. > > > Yes, both [my] machines define I686_CPU. > > > > > > "Dynamically", as in "at runtime"? I think you're right, but I don't > > > know for certain, either. This is exactly what I'm wondering about; > > > the PII has only MMX, for instance, while the PIII has SSE and MMX2. > > > > > > I assume the world's codebase is CPU-agnostic within an architecture, > > > but I really don't want to assume this; I'd rather know this. > > > > I have PII, Celeron, PIII and P4 in my environment. > > All these use "I686_CPU" in my kernel configs (I got rid of the other I[345]_CPU > > types. > > In a desktop's kernel config: > machine i386 > cpu I686_CPU > ident SHEOL > > In my laptop's kernel config: > machine i386 > cpu I686_CPU > ident CHARON > > Getting difficult, isn't it? ;-, > > Boot messages on that desktop from kernel built on that desktop: > CPU: Pentium III/Pentium III Xeon/Celeron (764.35-MHz 686-class CPU) > Origin = "GenuineIntel" Id = 0x686 Stepping = 6 > > Features=0x383f9ff > > Boot messages on that laptop from kernel built on that laptop: > CPU: Pentium II/Pentium II Xeon/Celeron (233.87-MHz 686-class CPU) > Origin = "GenuineIntel" Id = 0x652 Stepping = 2 > > Features=0x183f9ff > > > In /etc/make.conf I include "CPUTYPE=p2" as the lowest common denominator if > > including a CPUTYPE flag. > > The resulting world & kernel run fine on all the systems. > > > > The higher flags p3 and p4 still just use -march=pentiumpro, however for SSE you > > will need p3 or p4 as MACHINE_CPU does not include SSE for p2 level. > > Check out /usr/share/mk/bsd.cpu.mk for specifics. > > If I read things right, the first chunk in bsd.cpu.mk results in > "CPUTYPE = i686"; the conditional-on-CPU_TYPE-defined assignment > sees "cpu I686_CPU" in the kernel configs, right?. I'm not sure the kernel config is used for this - beyond my current depth. :) > The second > chunk in bsd.cpu.mk also results in "CPUTYPE = i686" (fallthrough). > The third chunk results in "_CPUCFLAGS = -march=pentiumpro", as > you state. Finally, the last chunk in bsd.cpu.mk results in > "MACHINE_CPU = i686 i586 i486 i386". > > If I specify "CPUTYPE=p2" in make.conf, and I'm still reading > things right, the final result in bsd.cpu.mk would be the same > _CPUCFLAGS, and "MACHINE_CPU = i686 mmx i586 i486 i386". > > But based on the boot messages, it would seem that the features of > the chips are detected correctly without specifying a CPUTYPE in > make.conf. So... As things are right now (no CPUTYPE in make.conf), > it would appear that "CPU feature" code is built into the kernels > on the respective machines, and that code correctly detects the > features available. Therefore, I don't see where adding CPUTYPE > in make.conf would get me anything I'm not already getting? CPU feature detection is done elsewhere, the make.conf entry I think will just set the flag so if code elsewhere is written to check for those flags it will use the appropriate bits. > That is, why make things more specific than they have to be? > I don't see where adding "CPUTYPE=p2" on a PIII build machine > would change anything for PII and/or PIII target machines. What > have I missed? I believe there are one or two areas of code (openssl?) that do make use of these flags and enable some optimised routines. Perhaps you can try comparing output of 'openssl speed' after compiling with different options? (eg for CPUTYPE empty, i686 and p2) Someone more familiar with the kernel internals might be able to offer more insight here. > > > > DJHJ> /etc/defaults/make.conf doesn't mention KERNCONF; /usr/src/Makefile.inc1 > > > > DJHJ> does. Since /usr/share/mk/sys.mk sucks in /etc/make.conf, that should > > > > DJHJ> propogate KERNCONF to /usr/src/Makefile, right? > > > > To your original question, yes. > > Just add a KER
Re: Clarification needed on Handbook: Tracking for Multiple Machines
On Feb 23, at 01:05 AM, Tony Frank wrote: > > On Sat, Feb 21, 2004 at 11:23:28AM -0600, D J Hawkey Jr wrote: > > On Feb 21, at 05:56 PM, Gabriel Ambuehl wrote: > > > > > > DJHJ> Second, two machines are of the same architecture, but they have different > > > DJHJ> CPUs: One is an Intel PIII, but the other is a PII. Will the world built > > > DJHJ> on a PIII be correct for a PII? Similarly, will the kernel for the PII > > > DJHJ> built on a PIII be correct for the PII, given the different variables and > > > DJHJ> settings in the two kernel configuration files? > > > > > > Just make sure you build for 686. If that doesn't work, make it 586 (I > > > think the PI qualifies as 686 but I'm not entirely sure). I think the extensions > > > such as > > > SSE etc are detected dynamically and shouldn't cause any problem. > > > In all my years of messing with builds, I never run into this problem, > > > so I guess it's pretty safe. > > Yes, both [my] machines define I686_CPU. > > > > "Dynamically", as in "at runtime"? I think you're right, but I don't > > know for certain, either. This is exactly what I'm wondering about; > > the PII has only MMX, for instance, while the PIII has SSE and MMX2. > > > > I assume the world's codebase is CPU-agnostic within an architecture, > > but I really don't want to assume this; I'd rather know this. > > I have PII, Celeron, PIII and P4 in my environment. > All these use "I686_CPU" in my kernel configs (I got rid of the other I[345]_CPU > types. In a desktop's kernel config: machine i386 cpu I686_CPU ident SHEOL In my laptop's kernel config: machine i386 cpu I686_CPU ident CHARON Getting difficult, isn't it? ;-, Boot messages on that desktop from kernel built on that desktop: CPU: Pentium III/Pentium III Xeon/Celeron (764.35-MHz 686-class CPU) Origin = "GenuineIntel" Id = 0x686 Stepping = 6 Features=0x383f9ff Boot messages on that laptop from kernel built on that laptop: CPU: Pentium II/Pentium II Xeon/Celeron (233.87-MHz 686-class CPU) Origin = "GenuineIntel" Id = 0x652 Stepping = 2 Features=0x183f9ff > In /etc/make.conf I include "CPUTYPE=p2" as the lowest common denominator if > including a CPUTYPE flag. > The resulting world & kernel run fine on all the systems. > > The higher flags p3 and p4 still just use -march=pentiumpro, however for SSE you > will need p3 or p4 as MACHINE_CPU does not include SSE for p2 level. > Check out /usr/share/mk/bsd.cpu.mk for specifics. If I read things right, the first chunk in bsd.cpu.mk results in "CPUTYPE = i686"; the conditional-on-CPU_TYPE-defined assignment sees "cpu I686_CPU" in the kernel configs, right?. The second chunk in bsd.cpu.mk also results in "CPUTYPE = i686" (fallthrough). The third chunk results in "_CPUCFLAGS = -march=pentiumpro", as you state. Finally, the last chunk in bsd.cpu.mk results in "MACHINE_CPU = i686 i586 i486 i386". If I specify "CPUTYPE=p2" in make.conf, and I'm still reading things right, the final result in bsd.cpu.mk would be the same _CPUCFLAGS, and "MACHINE_CPU = i686 mmx i586 i486 i386". But based on the boot messages, it would seem that the features of the chips are detected correctly without specifying a CPUTYPE in make.conf. So... As things are right now (no CPUTYPE in make.conf), it would appear that "CPU feature" code is built into the kernels on the respective machines, and that code correctly detects the features available. Therefore, I don't see where adding CPUTYPE in make.conf would get me anything I'm not already getting? That is, why make things more specific than they have to be? I don't see where adding "CPUTYPE=p2" on a PIII build machine would change anything for PII and/or PIII target machines. What have I missed? > > > DJHJ> /etc/defaults/make.conf doesn't mention KERNCONF; /usr/src/Makefile.inc1 > > > DJHJ> does. Since /usr/share/mk/sys.mk sucks in /etc/make.conf, that should > > > DJHJ> propogate KERNCONF to /usr/src/Makefile, right? > > To your original question, yes. > Just add a KERNCONF= line to /etc/make.conf. > First entry should be the local machine kernel (to install) and > any subsequent entries will also be built during 'make buildkernel' > eg: KERNCONF= MARVIN RAIDER RAIDERI GENERIC > make buildkernel builds all four. > make installkernel installs MARVIN. Cool. So, the -CURRENT (or is that -STABLE?) Handbook does apply to 4.5-REL, at least where this stuff is concerned. Dave -- __ __ \__ \D. J. HAWKEY JR. / __/ \/\ [EMAIL PROTECTED]/\/ http://www.visi.com/~hawkeyd/ ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]
Re: Clarification needed on Handbook: Tracking for Multiple Machines
Hi, On Sat, Feb 21, 2004 at 11:23:28AM -0600, D J Hawkey Jr wrote: > On Feb 21, at 05:56 PM, Gabriel Ambuehl wrote: > > > > DJHJ> Second, two machines are of the same architecture, but they have different > > DJHJ> CPUs: One is an Intel PIII, but the other is a PII. Will the world built > > DJHJ> on a PIII be correct for a PII? Similarly, will the kernel for the PII > > DJHJ> built on a PIII be correct for the PII, given the different variables and > > DJHJ> settings in the two kernel configuration files? > > > > Just make sure you build for 686. If that doesn't work, make it 586 (I > > think the PI qualifies as 686 but I'm not entirely sure). I think the extensions > > such as > > SSE etc are detected dynamically and shouldn't cause any problem. > > In all my years of messing with builds, I never run into this problem, > > so I guess it's pretty safe. > Yes, both [my] machines define I686_CPU. > > "Dynamically", as in "at runtime"? I think you're right, but I don't > know for certain, either. This is exactly what I'm wondering about; > the PII has only MMX, for instance, while the PIII has SSE and MMX2. > > I assume the world's codebase is CPU-agnostic within an architecture, > but I really don't want to assume this; I'd rather know this. I have PII, Celeron, PIII and P4 in my environment. All these use "I686_CPU" in my kernel configs (I got rid of the other I[345]_CPU types. In /etc/make.conf I include "CPUTYPE=p2" as the lowest common denominator if including a CPUTYPE flag. The resulting world & kernel run fine on all the systems. The higher flags p3 and p4 still just use -march=pentiumpro, however for SSE you will need p3 or p4 as MACHINE_CPU does not include SSE for p2 level. Check out /usr/share/mk/bsd.cpu.mk for specifics. > > DJHJ> /etc/defaults/make.conf doesn't mention KERNCONF; /usr/src/Makefile.inc1 > > DJHJ> does. Since /usr/share/mk/sys.mk sucks in /etc/make.conf, that should > > DJHJ> propogate KERNCONF to /usr/src/Makefile, right? > > You can also > > just supply it on the command line when doing your make runs. > Yes, but this means individual commands for each machine's kernel, as > opposed to one command for all machines (think "issue command and go to > bed", or even an `at` command). Are you stating definitively that what > I saw in the makefile chain isn't what is really there? To your original question, yes. Just add a KERNCONF= line to /etc/make.conf. First entry should be the local machine kernel (to install) and any subsequent entries will also be built during 'make buildkernel' eg: KERNCONF= MARVIN RAIDER RAIDERI GENERIC make buildkernel builds all four. make installkernel installs MARVIN. Good luck, Tony ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: Clarification needed on Handbook: Tracking for Multiple Machines
On Feb 21, at 05:56 PM, Gabriel Ambuehl wrote: > > DJHJ> Second, two machines are of the same architecture, but they have different > DJHJ> CPUs: One is an Intel PIII, but the other is a PII. Will the world built > DJHJ> on a PIII be correct for a PII? Similarly, will the kernel for the PII > DJHJ> built on a PIII be correct for the PII, given the different variables and > DJHJ> settings in the two kernel configuration files? > > Just make sure you build for 686. If that doesn't work, make it 586 (I > think the PI qualifies as 686 but I'm not entirely sure). I think the extensions > such as > SSE etc are detected dynamically and shouldn't cause any problem. > In all my years of messing with builds, I never run into this problem, > so I guess it's pretty safe. Yes, both [my] machines define I686_CPU. "Dynamically", as in "at runtime"? I think you're right, but I don't know for certain, either. This is exactly what I'm wondering about; the PII has only MMX, for instance, while the PIII has SSE and MMX2. I assume the world's codebase is CPU-agnostic within an architecture, but I really don't want to assume this; I'd rather know this. > DJHJ> Finally, after briefly following the makefile chain, it looks as though > DJHJ> what is written for -CURRENT is backward-compatible to 4.5-RELEASE? > > I don't think it is. 4.5 is OLD. You might not even find 4.9 to be > backwards compatible to 4.5, much less CURRENT. Yes, 4.5-REL is old, but I have too much vested in my 4.5-REL systems to jump anytime soon. I have been maintaining my 4.5-REL systems WRT post-4.5 security updates (right up to SA-04:02), and for what these machines are and what they do, 4.5-REL is perfectly suitable. > DJHJ> /etc/defaults/make.conf doesn't mention KERNCONF; /usr/src/Makefile.inc1 > DJHJ> does. Since /usr/share/mk/sys.mk sucks in /etc/make.conf, that should > DJHJ> propogate KERNCONF to /usr/src/Makefile, right? > > You can also > just supply it on the command line when doing your make runs. Yes, but this means individual commands for each machine's kernel, as opposed to one command for all machines (think "issue command and go to bed", or even an `at` command). Are you stating definitively that what I saw in the makefile chain isn't what is really there? Dave -- __ __ \__ \D. J. HAWKEY JR. / __/ \/\ [EMAIL PROTECTED]/\/ http://www.visi.com/~hawkeyd/ ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: Clarification needed on Handbook: Tracking for Multiple Machines
Hi D J Hawkey Jr, you wrote. DJHJ> So, two machines use the same world, except that a laptop doesn't want DJHJ> profiled libraries or games. Since the install is separate from the build, DJHJ> the build machine's make.conf must _not_ define NOPROFILE nor NOGAMES, If that machine itself wants those, yes. If you don't care for them (I could never be bothered about either one), you don't need to build them, obviously. DJHJ> but the laptop's make.conf _must_ define NOPROFILE and NOGAMES, such that DJHJ> both are present when `make installworld` is run on the machine that wants DJHJ> them, but they won't be installed when `make installworld` is run on the DJHJ> laptop, right? That should work, building more than you're going to install should be ok, the other way round obviously won't work. DJHJ> Second, two machines are of the same architecture, but they have different DJHJ> CPUs: One is an Intel PIII, but the other is a PII. Will the world built DJHJ> on a PIII be correct for a PII? Similarly, will the kernel for the PII DJHJ> built on a PIII be correct for the PII, given the different variables and DJHJ> settings in the two kernel configuration files? Just make sure you build for 686. If that doesn't work, make it 586 (I think the PI qualifies as 686 but I'm not entirely sure). I think the extensions such as SSE etc are detected dynamically and shouldn't cause any problem. In all my years of messing with builds, I never run into this problem, so I guess it's pretty safe. DJHJ> Finally, after briefly following the makefile chain, it looks as though DJHJ> what is written for -CURRENT is backward-compatible to 4.5-RELEASE? I don't think it is. 4.5 is OLD. You might not even find 4.9 to be backwards compatible to 4.5, much less CURRENT. DJHJ> /etc/defaults/make.conf doesn't mention KERNCONF; /usr/src/Makefile.inc1 DJHJ> does. Since /usr/share/mk/sys.mk sucks in /etc/make.conf, that should DJHJ> propogate KERNCONF to /usr/src/Makefile, right? I suggest you go for 4.9 anyhow. Then KERNCONF is ok. You can also just supply it on the command line when doing your make runs. Regards, Gabriel ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"