Re: [SOLVED] Re: should 'make -j8 build' work?
On Wed, 8 Feb 2012 11:27:14 -0500 "Joe Gidi" wrote: > I actually resolved this by pulling and reseating all the DIMMs. > > Oddly enough, prior to that, the box went through 3 complete runs of > memtest86+ without error, but continued to hang at random spots during > 'make -j8 build'. > By complete runs, do you mean 3 passes. I've read you need atleast 8 passes to be sure and I believe from observation but could be wrong that each pass tests different things as do the tests. I've only ever waited for 8 passes, once or twice however (just after I'd read about it), it took ages. > I pulled and reseated all the memory and then did 5 'make -j8 build' runs > successfully, no more hangs. > > Very strange, but this is why I like to stress-test new builds... Kc
Re: [SOLVED] Re: should 'make -j8 build' work?
On Wed, 8 Feb 2012 18:52:15 +0100 Norman Golisz wrote: > actually, it's the most reliable way to detect faulty hardware. Memory > testers, if at all, only find specific issues (mostly by writing and > reading bit patterns to RAM). They can't stimulate and stress the > hardware as a build process (e.g. of an operating system) does. > > Memtest and such only have limited use. Building certainly throws extra heat into the equation. -- Kc
Re: [SOLVED] Re: should 'make -j8 build' work?
Hi Joe, On Wed Feb 8 2012 11:27, Joe Gidi wrote: > On Wed, February 8, 2012 3:25 am, Stuart Henderson wrote: > > On 2012-02-07, Joe Gidi wrote: > >> In every case, when the box hangs, I'm unable to break into ddb. > > > > How long do you leave it when it hangs? There have been occasions > > where a box appears to hang but then recovers. > > > > Are you using softdep? > > I actually resolved this by pulling and reseating all the DIMMs. > > Oddly enough, prior to that, the box went through 3 complete runs of > memtest86+ without error, but continued to hang at random spots during > 'make -j8 build'. > > I pulled and reseated all the memory and then did 5 'make -j8 build' runs > successfully, no more hangs. > > Very strange, but this is why I like to stress-test new builds... actually, it's the most reliable way to detect faulty hardware. Memory testers, if at all, only find specific issues (mostly by writing and reading bit patterns to RAM). They can't stimulate and stress the hardware as a build process (e.g. of an operating system) does. Memtest and such only have limited use. Yours, Norman
[SOLVED] Re: should 'make -j8 build' work?
On Wed, February 8, 2012 3:25 am, Stuart Henderson wrote: > On 2012-02-07, Joe Gidi wrote: >> In every case, when the box hangs, I'm unable to break into ddb. > > How long do you leave it when it hangs? There have been occasions > where a box appears to hang but then recovers. > > Are you using softdep? I actually resolved this by pulling and reseating all the DIMMs. Oddly enough, prior to that, the box went through 3 complete runs of memtest86+ without error, but continued to hang at random spots during 'make -j8 build'. I pulled and reseated all the memory and then did 5 'make -j8 build' runs successfully, no more hangs. Very strange, but this is why I like to stress-test new builds... -- Joe Gidi j...@entropicblur.com "You cannot buy skill." -- Ross Seyfried
Re: should 'make -j8 build' work?
On 2012-02-07, Joe Gidi wrote: > In every case, when the box hangs, I'm unable to break into ddb. How long do you leave it when it hangs? There have been occasions where a box appears to hang but then recovers. Are you using softdep?
Re: should 'make -j8 build' work?
On Tue, 07 Feb 2012 19:01:42 +0100, Joe Gidi wrote: I just built a new box with one of AMD's FX-8120 8-core processors and wanted to stress-test it a bit. I installed the Feb 6 amd64 snapshot, checked out a src tree, and had no trouble compiling GENERIC.MP. I tried compiling userland with "make -j8 build" and the box hung. Before I go further in depth with troubleshooting, I'd just like to know if this "should" work properly on good hardware. If it's known not to work, I won't waste my (or the lists') time on figuring out why it's hanging. Also, is there a maximum number of make jobs that should work? I used to run 'make -j4 build" successfully on my previous 4-core system... Thanks for any feedback, A very good general stress test is a full build of the ports tree. You can build 8 different packages at time. http://openbsd.org/faq/faq15.html#dpb -- Juan Francisco Cantero Hurtado http://juanfra.info
Re: should 'make -j8 build' work?
On Tue, February 7, 2012 1:56 pm, Joe Gidi wrote:
> On Tue, February 7, 2012 1:45 pm, Kenneth R Westerback wrote:
>> On Tue, Feb 07, 2012 at 01:01:42PM -0500, Joe Gidi wrote:
>>> I just built a new box with one of AMD's FX-8120 8-core processors and
>>> wanted to stress-test it a bit. I installed the Feb 6 amd64 snapshot,
>>> checked out a src tree, and had no trouble compiling GENERIC.MP. I
>>> tried
>>> compiling userland with "make -j8 build" and the box hung.
>>>
>>> Before I go further in depth with troubleshooting, I'd just like to
>>> know
>>> if this "should" work properly on good hardware. If it's known not to
>>> work, I won't waste my (or the lists') time on figuring out why it's
>>> hanging.
>>>
>>> Also, is there a maximum number of make jobs that should work? I used
>>> to
>>> run 'make -j4 build" successfully on my previous 4-core system...
>>>
>>> Thanks for any feedback,
>>>
>>> --
>>> Joe Gidi
>>> j...@entropicblur.com
>>>
>>> "You cannot buy skill." -- Ross Seyfried
>>>
>>
>> Sometimes it works and sometimes not. There are parts of the tree that
>> have races.
>>
>> Ken
>
> Thanks. Should "not working" cause the box to hang, or just cause the
> build to fail at some point with an error?
>
> I'm running a bog-stock snapshot install; I didn't try to get cute or
> clever anywhere, no customization, no non-stock sysctl settings, no
> mk.conf. I just installed the snap, checked out a tree, and built per the
> FAQ. I only went "off the reservation" by running 'make -j8 build' instead
> of 'make build'.
>
> I have just enabled ddb.console=1 and am going to try a series of builds
> with progressively higher -j flags. A -j4 build just succeeded.
>
> I'll report back to the list with full details if/when I can reproduce the
> hang.
>
> Thanks again...
And after a bunch of testing with both the Feb 6 snapshot and 5.0-RELEASE,
I get inconsistent results. I've had a 'make -j8 build' succeed at times,
fail at others. Same with 'make -j6 build'. Seems like 'make -j4 build' is
stable. The hangs occur at different places within the build process, with
no pattern I've noticed.
In every case, when the box hangs, I'm unable to break into ddb. I tested
and confirmed that ddb DOES work during normal operation. So, I don't know
what more I can do to gather useful information.
The inconsistency makes me tend to believe I have some flaky hardware. I
have memtest86+ looping on the box right now, no errors so far.
I do wonder, though, if maybe I'm hitting some weird issue with the amd
15h architecture? Are any other users or developers running amd 15h
("Bulldozer") hardware yet?
--
Joe Gidi
j...@entropicblur.com
"You cannot buy skill." -- Ross Seyfried
Re: should 'make -j8 build' work?
You can still use -j8 while building the kernel though, no races there.
Re: should 'make -j8 build' work?
On Tue, February 7, 2012 1:56 pm, Joe Gidi wrote: > On Tue, February 7, 2012 1:45 pm, Kenneth R Westerback wrote: >> On Tue, Feb 07, 2012 at 01:01:42PM -0500, Joe Gidi wrote: >>> I just built a new box with one of AMD's FX-8120 8-core processors and >>> wanted to stress-test it a bit. I installed the Feb 6 amd64 snapshot, >>> checked out a src tree, and had no trouble compiling GENERIC.MP. I >>> tried >>> compiling userland with "make -j8 build" and the box hung. >>> >>> Before I go further in depth with troubleshooting, I'd just like to >>> know >>> if this "should" work properly on good hardware. If it's known not to >>> work, I won't waste my (or the lists') time on figuring out why it's >>> hanging. >>> >>> Also, is there a maximum number of make jobs that should work? I used >>> to >>> run 'make -j4 build" successfully on my previous 4-core system... >>> >>> Thanks for any feedback, >>> >>> -- >>> Joe Gidi >>> j...@entropicblur.com >>> >>> "You cannot buy skill." -- Ross Seyfried >>> >> >> Sometimes it works and sometimes not. There are parts of the tree that >> have races. >> >> Ken > > Thanks. Should "not working" cause the box to hang, or just cause the > build to fail at some point with an error? > > I'm running a bog-stock snapshot install; I didn't try to get cute or > clever anywhere, no customization, no non-stock sysctl settings, no > mk.conf. I just installed the snap, checked out a tree, and built per the > FAQ. I only went "off the reservation" by running 'make -j8 build' instead > of 'make build'. > > I have just enabled ddb.console=1 and am going to try a series of builds > with progressively higher -j flags. A -j4 build just succeeded. > > I'll report back to the list with full details if/when I can reproduce the > hang. > > Thanks again... Okay, 'make -j4 build' and 'make -j5 build' both completed successfully. 'make -j6 build' hung. The last line on the screen was: Running Makefile.PL in ext/I18N-Langinfo ../../miniperl Makefile.PL INSTALLDIRS=perl INSTALLMAN1DIR=none INSTALLMAN3DIR=none PERL_CORE=1 LIBPERL_A=libperl.so.12.0 LINKTYPE=dynamic Below that was a blank line with a blinking cursor. Could not switch to another virtual console. Could not get into ddb via Ctrl+Alt+Esc. I'm currently re-running the -j6 with a PS2 keyboard, to (A) see if it hangs in the same spot and (B) hopefully be able to get into ddb. dmesg: OpenBSD 5.1-beta (GENERIC.MP) #0: Tue Feb 7 12:57:00 EST 2012 r...@testbed.gateway.2wire.net:/usr/src/sys/arch/amd64/compile/GENERIC.MP real mem = 8533127168 (8137MB) avail mem = 8291811328 (7907MB) mainbus0 at root bios0 at mainbus0: SMBIOS rev. 2.7 @ 0xeed90 (55 entries) bios0: vendor American Megatrends Inc. version "0901" date 11/24/2011 bios0: ASUSTeK COMPUTER INC. M5A97 acpi0 at bios0: rev 2 acpi0: sleep states S0 S1 S3 S4 S5 acpi0: tables DSDT FACP APIC MCFG HPET SSDT acpi0: wakeup devices SBAZ(S4) PS2K(S3) PS2M(S3) UAR1(S4) P0PC(S4) UHC1(S4) UHC2(S4) USB3(S4) UHC4(S4) USB5(S4) UHC6(S4) UHC7(S4) PE20(S4) PE21(S4) PE22(S4) PE23(S4) PC02(S4) PC04(S4) PC05(S4) PC06(S4) PC07(S4) PC09(S4) PC0A(S4) PC0B(S4) PC0C(S4) PC0D(S4) PWRB(S4) acpitimer0 at acpi0: 3579545 Hz, 32 bits acpimadt0 at acpi0 addr 0xfee0: PC-AT compat cpu0 at mainbus0: apid 16 (boot processor) cpu0: AMD FX(tm)-8120 Eight-Core Processor , 3110.78 MHz cpu0: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUS H,MMX,FXSR,SSE,SSE2,HTT,SSE3,PCLMUL,MWAIT,SSSE3,CX16,SSE4.1,SSE4.2,POPCNT,AES ,XSAVE,AVX,NXE,MMXX,FFXSR,LONG,LAHF,CMPLEG,SVM,EAPICSP,AMCR8,ABM,SSE4A,MASSE, 3DNOWP,OSVW,IBS,XOP,SKINIT,WDT,FMA4,NODEID,TOPEXT cpu0: 64KB 64b/line 2-way I-cache, 16KB 64b/line 4-way D-cache, 2MB 64b/line 16-way L2 cache cpu0: ITLB 48 4KB entries fully associative, 24 4MB entries fully associative cpu0: DTLB 32 4KB entries fully associative, 32 4MB entries fully associative cpu0: apic clock running at 200MHz cpu1 at mainbus0: apid 17 (application processor) cpu1: AMD FX(tm)-8120 Eight-Core Processor , 556.23 MHz cpu1: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUS H,MMX,FXSR,SSE,SSE2,HTT,SSE3,PCLMUL,MWAIT,SSSE3,CX16,SSE4.1,SSE4.2,POPCNT,AES ,XSAVE,AVX,NXE,MMXX,FFXSR,LONG,LAHF,CMPLEG,SVM,EAPICSP,AMCR8,ABM,SSE4A,MASSE, 3DNOWP,OSVW,IBS,XOP,SKINIT,WDT,FMA4,NODEID,TOPEXT cpu1: 64KB 64b/line 2-way I-cache, 16KB 64b/line 4-way D-cache, 2MB 64b/line 16-way L2 cache cpu1: ITLB 48 4KB entries fully associative, 24 4MB entries fully associative cpu1: DTLB 32 4KB entries fully associative, 32 4MB entries fully associative cpu2 at mainbus0: apid 18 (application processor) cpu2: AMD FX(tm)-8120 Eight-Core Processor , 211.96 MHz cpu2: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUS H,MMX,FXSR,SSE,SSE2,HTT,SSE3,PCLMUL,MWAIT,SSSE3,CX16,SSE4.1,SSE4.2,POPCNT,AES ,XSAVE,AVX,NXE,MMXX,FFXSR,LONG,LAHF,CMPLEG,SVM,EAPICSP,AMCR8,ABM,SSE4A,MASSE, 3DNOWP,OSVW,IBS,XOP,SKINIT,WDT,FMA4,NODEID,TOPEXT cpu2: 64KB 64b/line 2-way I-cache, 16KB 64b/line 4-
Re: should 'make -j8 build' work?
On Tue, Feb 07, 2012 at 07:05:04PM +, Christian Weisgerber wrote: > Joe Gidi wrote: > > > I just built a new box with one of AMD's FX-8120 8-core processors and > > wanted to stress-test it a bit. I installed the Feb 6 amd64 snapshot, > > checked out a src tree, and had no trouble compiling GENERIC.MP. I tried > > compiling userland with "make -j8 build" and the box hung. > > > > Before I go further in depth with troubleshooting, I'd just like to know > > if this "should" work properly on good hardware. > > Yes, it should. I've certainly done it before. > It's possible that there are some build races left somewhere that > could cause the build to error out on occasion. But the box hanging > is not expected. Sorry, I should have said that too. Builds failing are possible. The box hanging is a problem. Ken > > > Also, is there a maximum number of make jobs that should work? > > High numbers might reveal some lingering problems in the Makefiles. > More importantly, contention for the big kernel lock will limit the > amount of concurrency that is achievable. make build scales > reasonably well to -j4, but with -j8 you're well into the range of > sadly diminished returns. > > -- > Christian "naddy" Weisgerber na...@mips.inka.de
Re: should 'make -j8 build' work?
On Tue, Feb 7, 2012 at 14:10, Joe Gidi wrote: > On Tue, February 7, 2012 2:00 pm, patrick keshishian wrote: > > On Tue, Feb 7, 2012 at 10:45 AM, Kenneth R Westerback > > wrote: > >> On Tue, Feb 07, 2012 at 01:01:42PM -0500, Joe Gidi wrote: > >>> I just built a new box with one of AMD's FX-8120 8-core processors and > >>> wanted to stress-test it a bit. I installed the Feb 6 amd64 snapshot, > >>> checked out a src tree, and had no trouble compiling GENERIC.MP. I > >>> tried > >>> compiling userland with "make -j8 build" and the box hung. > >>> > >>> Before I go further in depth with troubleshooting, I'd just like to > >>> know > >>> if this "should" work properly on good hardware. If it's known not to > >>> work, I won't waste my (or the lists') time on figuring out why it's > >>> hanging. > >>> > >>> Also, is there a maximum number of make jobs that should work? I used > >>> to > >>> run 'make -j4 build" successfully on my previous 4-core system... > >>> > >>> Thanks for any feedback, > >>> > >>> -- > >>> Joe Gidi > >>> j...@entropicblur.com > >>> > >>> "You cannot buy skill." -- Ross Seyfried > >>> > >> > >> Sometimes it works and sometimes not. There are parts of the tree that > >> have races. > > > Strange, as i I can run make -j8 without any problems, both when compiling src and xenocara. It also has a pretty noticeable improvement over -j4. I run this on a i7 2620M on my thinkpad t520. Have never had any issue with -j x before, besides an error in ports, that was fixed the next time time I updated my cvs sources...
Re: should 'make -j8 build' work?
On Tue, February 7, 2012 2:00 pm, patrick keshishian wrote: > On Tue, Feb 7, 2012 at 10:45 AM, Kenneth R Westerback > wrote: >> On Tue, Feb 07, 2012 at 01:01:42PM -0500, Joe Gidi wrote: >>> I just built a new box with one of AMD's FX-8120 8-core processors and >>> wanted to stress-test it a bit. I installed the Feb 6 amd64 snapshot, >>> checked out a src tree, and had no trouble compiling GENERIC.MP. I >>> tried >>> compiling userland with "make -j8 build" and the box hung. >>> >>> Before I go further in depth with troubleshooting, I'd just like to >>> know >>> if this "should" work properly on good hardware. If it's known not to >>> work, I won't waste my (or the lists') time on figuring out why it's >>> hanging. >>> >>> Also, is there a maximum number of make jobs that should work? I used >>> to >>> run 'make -j4 build" successfully on my previous 4-core system... >>> >>> Thanks for any feedback, >>> >>> -- >>> Joe Gidi >>> j...@entropicblur.com >>> >>> "You cannot buy skill." -- Ross Seyfried >>> >> >> Sometimes it works and sometimes not. There are parts of the tree that >> have races. > > it is one thing for the build to fail, but the guy said "the box > hung", which indicates something is amiss. Correct, it hung. Screen stopped updating, couldn't switch virtual consoles, had to reset via the power button. I had forgotten to enable ddb before the build, and I was watching 'systat sensors' on another virtual console at the time of the hang (watching CPU temps), so I couldn't see exactly where in the build the box hung. It did run for about 10 minutes before hanging, though. -- Joe Gidi j...@entropicblur.com "You cannot buy skill." -- Ross Seyfried
Re: should 'make -j8 build' work?
Joe Gidi wrote: > I just built a new box with one of AMD's FX-8120 8-core processors and > wanted to stress-test it a bit. I installed the Feb 6 amd64 snapshot, > checked out a src tree, and had no trouble compiling GENERIC.MP. I tried > compiling userland with "make -j8 build" and the box hung. > > Before I go further in depth with troubleshooting, I'd just like to know > if this "should" work properly on good hardware. Yes, it should. I've certainly done it before. It's possible that there are some build races left somewhere that could cause the build to error out on occasion. But the box hanging is not expected. > Also, is there a maximum number of make jobs that should work? High numbers might reveal some lingering problems in the Makefiles. More importantly, contention for the big kernel lock will limit the amount of concurrency that is achievable. make build scales reasonably well to -j4, but with -j8 you're well into the range of sadly diminished returns. -- Christian "naddy" Weisgerber na...@mips.inka.de
Re: should 'make -j8 build' work?
On Tue, Feb 7, 2012 at 10:45 AM, Kenneth R Westerback wrote: > On Tue, Feb 07, 2012 at 01:01:42PM -0500, Joe Gidi wrote: >> I just built a new box with one of AMD's FX-8120 8-core processors and >> wanted to stress-test it a bit. I installed the Feb 6 amd64 snapshot, >> checked out a src tree, and had no trouble compiling GENERIC.MP. I tried >> compiling userland with "make -j8 build" and the box hung. >> >> Before I go further in depth with troubleshooting, I'd just like to know >> if this "should" work properly on good hardware. If it's known not to >> work, I won't waste my (or the lists') time on figuring out why it's >> hanging. >> >> Also, is there a maximum number of make jobs that should work? I used to >> run 'make -j4 build" successfully on my previous 4-core system... >> >> Thanks for any feedback, >> >> -- >> Joe Gidi >> j...@entropicblur.com >> >> "You cannot buy skill." -- Ross Seyfried >> > > Sometimes it works and sometimes not. There are parts of the tree that > have races. it is one thing for the build to fail, but the guy said "the box hung", which indicates something is amiss.
Re: should 'make -j8 build' work?
On Tue, February 7, 2012 1:45 pm, Kenneth R Westerback wrote: > On Tue, Feb 07, 2012 at 01:01:42PM -0500, Joe Gidi wrote: >> I just built a new box with one of AMD's FX-8120 8-core processors and >> wanted to stress-test it a bit. I installed the Feb 6 amd64 snapshot, >> checked out a src tree, and had no trouble compiling GENERIC.MP. I tried >> compiling userland with "make -j8 build" and the box hung. >> >> Before I go further in depth with troubleshooting, I'd just like to know >> if this "should" work properly on good hardware. If it's known not to >> work, I won't waste my (or the lists') time on figuring out why it's >> hanging. >> >> Also, is there a maximum number of make jobs that should work? I used to >> run 'make -j4 build" successfully on my previous 4-core system... >> >> Thanks for any feedback, >> >> -- >> Joe Gidi >> j...@entropicblur.com >> >> "You cannot buy skill." -- Ross Seyfried >> > > Sometimes it works and sometimes not. There are parts of the tree that > have races. > > Ken Thanks. Should "not working" cause the box to hang, or just cause the build to fail at some point with an error? I'm running a bog-stock snapshot install; I didn't try to get cute or clever anywhere, no customization, no non-stock sysctl settings, no mk.conf. I just installed the snap, checked out a tree, and built per the FAQ. I only went "off the reservation" by running 'make -j8 build' instead of 'make build'. I have just enabled ddb.console=1 and am going to try a series of builds with progressively higher -j flags. A -j4 build just succeeded. I'll report back to the list with full details if/when I can reproduce the hang. Thanks again... -- Joe Gidi j...@entropicblur.com "You cannot buy skill." -- Ross Seyfried
Re: should 'make -j8 build' work?
On Tue, Feb 07, 2012 at 01:01:42PM -0500, Joe Gidi wrote: > I just built a new box with one of AMD's FX-8120 8-core processors and > wanted to stress-test it a bit. I installed the Feb 6 amd64 snapshot, > checked out a src tree, and had no trouble compiling GENERIC.MP. I tried > compiling userland with "make -j8 build" and the box hung. > > Before I go further in depth with troubleshooting, I'd just like to know > if this "should" work properly on good hardware. If it's known not to > work, I won't waste my (or the lists') time on figuring out why it's > hanging. > > Also, is there a maximum number of make jobs that should work? I used to > run 'make -j4 build" successfully on my previous 4-core system... > > Thanks for any feedback, > > -- > Joe Gidi > j...@entropicblur.com > > "You cannot buy skill." -- Ross Seyfried > Sometimes it works and sometimes not. There are parts of the tree that have races. Ken
