Re: Upper limit on make -j ?
Brian wrote: Isn't 1000 an unrealistically high value? Brian On a system with only 2 cores and 2GB RAM - yes. I'm not sure that even with a huge number of cores you'd get much benefit from running such a massively parallel make, unless the build system is more intelligent than I think it is. -- Bruce ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Upper limit on make -j ?
Brian wrote: There's a thread on the -stable list about 8 cores being an issue in a very specific case. I couldn't imagine setting j to something more than 2x or 3x at the most of your available cores. So it's okay by you that freebsd utilities crash when you overload them? Not to me ;-) BTW, Aryeh, posting 800KB of your buildworld output to the mailing list is exceedingly lame and doesn't help anything. Please study the gdb trace provided by Bruce: that is the kind of debugging that was required here, not huge amounts of irrelevant text from your console. Kris ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Upper limit on make -j ?
Before I file a PR I just want to know if it is worth it to file a PR for: make -j1000 buildworld buildkernel installkernel seg faulting ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Upper limit on make -j ?
On Tue, Nov 27, 2007 at 03:25:52AM -0500, Aryeh Friedman wrote: Before I file a PR I just want to know if it is worth it to file a PR for: make -j1000 buildworld buildkernel installkernel seg faulting No. Where in the handbook or UPDATING does it tell you to build your system like that? -- Frank Contact info: http://www.esperance-linux.co.uk/misc/contact.html ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Upper limit on make -j ?
Frank Shute wrote: On Tue, Nov 27, 2007 at 03:25:52AM -0500, Aryeh Friedman wrote: Before I file a PR I just want to know if it is worth it to file a PR for: make -j1000 buildworld buildkernel installkernel seg faulting No. Where in the handbook or UPDATING does it tell you to build your system like that? It should not seg fault though. However, when I have done this in the past it has not seg faulted, either. The OP needs to obtain backtraces with his PR submission since it may not be reliably reproduceable. Kris ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Upper limit on make -j ?
Aryeh Friedman wrote: Before I file a PR I just want to know if it is worth it to file a PR for: make -j1000 buildworld buildkernel installkernel seg faulting I thought that the kernel builds couldn't be built using parallel jobs, that it might break something. Is that not true? ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED] ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Upper limit on make -j ?
On Tuesday 27 November 2007, Andy Greenwood wrote: Aryeh Friedman wrote: Before I file a PR I just want to know if it is worth it to file a PR for: make -j1000 buildworld buildkernel installkernel seg faulting I thought that the kernel builds couldn't be built using parallel jobs, that it might break something. Is that not true? In my experience parallel kernel builds are fine. I think (wild guess) the OP is running out of memory and somewhere in make or gcc the return value of malloc() isn't checked. - Pieter de Goeje ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Upper limit on make -j ?
Aryeh Friedman wrote: Before I file a PR I just want to know if it is worth it to file a PR for: make -j1000 buildworld buildkernel installkernel seg faulting Yes it is. Like others said, obtain as much information about the problem as you can (backtraces). Try to discover if it fails in make, gcc or somewhere else. The ridiculous setting should do the right thing which is work very slowly, but it *should* work. signature.asc Description: OpenPGP digital signature
Re: Upper limit on make -j ?
Andy Greenwood wrote: Aryeh Friedman wrote: Before I file a PR I just want to know if it is worth it to file a PR for: make -j1000 buildworld buildkernel installkernel seg faulting I thought that the kernel builds couldn't be built using parallel jobs, that it might break something. Is that not true? Not true for many years. Kris ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Upper limit on make -j ?
On Tue, Nov 27, 2007 at 08:02:44PM +0100, Kris Kennaway wrote: Andy Greenwood wrote: Aryeh Friedman wrote: Before I file a PR I just want to know if it is worth it to file a PR for: make -j1000 buildworld buildkernel installkernel seg faulting I thought that the kernel builds couldn't be built using parallel jobs, that it might break something. Is that not true? Not true for many years. From /usr/src/UPDATING: Avoid using make -j when upgrading. From time to time in the past there have been problems using -j with buildworld and/or installworld. This is especially true when upgrading between distant versions (eg one that cross a major release boundary or several minor releases, or when several months have passed on the -current branch) Should this be changed then? -- Frank Contact info: http://www.esperance-linux.co.uk/misc/contact.html ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Upper limit on make -j ?
Frank Shute wrote: On Tue, Nov 27, 2007 at 08:02:44PM +0100, Kris Kennaway wrote: Andy Greenwood wrote: Aryeh Friedman wrote: Before I file a PR I just want to know if it is worth it to file a PR for: make -j1000 buildworld buildkernel installkernel seg faulting I thought that the kernel builds couldn't be built using parallel jobs, that it might break something. Is that not true? Not true for many years. From /usr/src/UPDATING: Avoid using make -j when upgrading. From time to time in the past there have been problems using -j with buildworld and/or installworld. This is especially true when upgrading between distant versions (eg one that cross a major release boundary or several minor releases, or when several months have passed on the -current branch) Should this be changed then? Maybe. Kris ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Upper limit on make -j ?
Pieter de Goeje wrote: On Tuesday 27 November 2007, Andy Greenwood wrote: Aryeh Friedman wrote: Before I file a PR I just want to know if it is worth it to file a PR for: make -j1000 buildworld buildkernel installkernel seg faulting I thought that the kernel builds couldn't be built using parallel jobs, that it might break something. Is that not true? In my experience parallel kernel builds are fine. I think (wild guess) the OP is running out of memory and somewhere in make or gcc the return value of malloc() isn't checked. - Pieter de Goeje I just tried this on my amd64 7.0-BETA3 system with 2GB RAM and swap was never touched but make still segfaulted during 'make -j1000 buildworld': [...] rm -f .depend mkdep -f .depend -a [long list of files] echo libc.so.7: /usr/obj/usr/src/tmp/usr/lib/libgcc.a .depend Segmentation fault (core dumped) *** Error code 139 1 error *** Error code 2 1 error *** Error code 2 1 error *** Error code 2 1 error gdb -core /usr/obj/usr/src/lib/libc/make.core /usr/bin/make GNU gdb 6.1.1 [FreeBSD] [...] This GDB was configured as amd64-marcel-freebsd... Core was generated by `make'. Program terminated with signal 11, Segmentation fault. #0 0x00432067 in __vfprintf () (gdb) bt #0 0x00432067 in __vfprintf () #1 0x00435114 in vfprintf () #2 0x0042d676 in fprintf () #3 0x00406305 in JobExec (job=0x80123f000, argv=0x7fffdb10) at /usr/src/usr.bin/make/job.c:1321 #4 0x0040702d in JobStart (gn=0x8006d9120, flags=Variable flags is not available. ) at /usr/src/usr.bin/make/job.c:1843 #5 0x0040b14c in MakeStartJobs () at /usr/src/usr.bin/make/make.c:621 #6 0x0040b34c in Make_Run (targs=0x7fffe7f0) at /usr/src/usr.bin/make/make.c:793 #7 0x0040a081 in main (argc=0, argv=0x0) at /usr/src/usr.bin/make/main.c:1273 -- Bruce ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Upper limit on make -j ?
Isn't 1000 an unrealistically high value? Brian On Tue, 27 Nov 2007, Bruce Cran wrote: Pieter de Goeje wrote: On Tuesday 27 November 2007, Andy Greenwood wrote: Aryeh Friedman wrote: Before I file a PR I just want to know if it is worth it to file a PR for: make -j1000 buildworld buildkernel installkernel seg faulting I thought that the kernel builds couldn't be built using parallel jobs, that it might break something. Is that not true? In my experience parallel kernel builds are fine. I think (wild guess) the OP is running out of memory and somewhere in make or gcc the return value of malloc() isn't checked. - Pieter de Goeje I just tried this on my amd64 7.0-BETA3 system with 2GB RAM and swap was never touched but make still segfaulted during 'make -j1000 buildworld': [...] rm -f .depend mkdep -f .depend -a [long list of files] echo libc.so.7: /usr/obj/usr/src/tmp/usr/lib/libgcc.a .depend Segmentation fault (core dumped) *** Error code 139 1 error *** Error code 2 1 error *** Error code 2 1 error *** Error code 2 1 error gdb -core /usr/obj/usr/src/lib/libc/make.core /usr/bin/make GNU gdb 6.1.1 [FreeBSD] [...] This GDB was configured as amd64-marcel-freebsd... Core was generated by `make'. Program terminated with signal 11, Segmentation fault. #0 0x00432067 in __vfprintf () (gdb) bt #0 0x00432067 in __vfprintf () #1 0x00435114 in vfprintf () #2 0x0042d676 in fprintf () #3 0x00406305 in JobExec (job=0x80123f000, argv=0x7fffdb10) at /usr/src/usr.bin/make/job.c:1321 #4 0x0040702d in JobStart (gn=0x8006d9120, flags=Variable flags is not available. ) at /usr/src/usr.bin/make/job.c:1843 #5 0x0040b14c in MakeStartJobs () at /usr/src/usr.bin/make/make.c:621 #6 0x0040b34c in Make_Run (targs=0x7fffe7f0) at /usr/src/usr.bin/make/make.c:793 #7 0x0040a081 in main (argc=0, argv=0x0) at /usr/src/usr.bin/make/main.c:1273 -- Bruce ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED] ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Upper limit on make -j ?
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Brian wrote: Isn't 1000 an unrealistically high value? For the next few years maybe but with core counts going up I can easily see a 512 or 1024 core machine by say 2015.. i.e. Moore's law may not apply to single tasking chips any mores but it seems to be in full force for multi-core ones... this brings me to why I tried it in the first place I was exploring weither or not FreeBSD was ready for this kind of core count (specifically the idea of a multitasking OS scheduling by allocating one process per core for core counts greater then say 64)... I am in the process of considering the design requirements for a OS I plan to do from the ground up and like to know the limits of current ones. - -- Aryeh M. Friedman Developer, not business, friendly http://www.flosoft-systems.com -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.4 (FreeBSD) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFHTOJpJ9+1V27SttsRAhMCAJ902zwVIxCGecI2cAdIm2bywN383ACfU243 HMUWkzcO8hH87PecYBmkgLc= =5614 -END PGP SIGNATURE- ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Upper limit on make -j ?
There's a thread on the -stable list about 8 cores being an issue in a very specific case. I couldn't imagine setting j to something more than 2x or 3x at the most of your available cores. Brian On Tue, 27 Nov 2007, Aryeh M. Friedman wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Brian wrote: Isn't 1000 an unrealistically high value? For the next few years maybe but with core counts going up I can easily see a 512 or 1024 core machine by say 2015.. i.e. Moore's law may not apply to single tasking chips any mores but it seems to be in full force for multi-core ones... this brings me to why I tried it in the first place I was exploring weither or not FreeBSD was ready for this kind of core count (specifically the idea of a multitasking OS scheduling by allocating one process per core for core counts greater then say 64)... I am in the process of considering the design requirements for a OS I plan to do from the ground up and like to know the limits of current ones. - -- Aryeh M. Friedman Developer, not business, friendly http://www.flosoft-systems.com -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.4 (FreeBSD) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFHTOJpJ9+1V27SttsRAhMCAJ902zwVIxCGecI2cAdIm2bywN383ACfU243 HMUWkzcO8hH87PecYBmkgLc= =5614 -END PGP SIGNATURE- ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED] ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
