Re: high sys time, very very slow builds on new 24-core system
On Wed, Mar 23, 2011 at 05:24:12PM -0400, Thor Lancelot Simon wrote: All cores spend well over 50% time in 'sys', even when all or almost all are running cc1 processes. The kernel is amd64 -current GENERIC from about 1 week ago -- no DIAGNOSTIC, DEBUG, KMEMSTATS, LOCKDEBUG, etc. I am getting some lockstat output now; I will send it here as soon as I have it. While I wait, I can see two very curious things in systat vmstat: 1) There is a truly huge number of zfod pages, sometimes well over 100,000/sec, even with 10-15% CPU idle. I wonder if the idle loop page zeroer is for some reason not working efficiently on this system. 2) About 200 xfers/sec (about 3MB/sec worth) is still going on to the SSD for much of the build process. It's all writes. All build directories (obj, dest, tools, release) are on a separate tmpfs, /tmp is on tmpfs, the machine isn't using any swap, and I am building with -U -O. By the time I verified all this, the disk writes were done so I couldn't use fstat to see what files might be being written. I wonder if some tool has reverted to using /var/tmp for its intermediate files, or similar. Don't see an obvious culprit for that, however. Thor
filesystem writing inodes when it oughtn't?
On Thu, Mar 31, 2011 at 03:53:22PM -0400, Thor Lancelot Simon wrote: 2) About 200 xfers/sec (about 3MB/sec worth) is still going on to the SSD for much of the build process. It's all writes. All build directories (obj, dest, tools, release) are on a separate tmpfs, /tmp is on tmpfs, the machine isn't using any swap, and I am building with -U -O. By the time I verified all this, the disk writes were done so I couldn't use fstat to see what files might be being written. I cannot find anything relevant open in / for write while this is going on (/ is the SSD on this system). The filesystem is mounted noatime, nodevmtime, but I suspect it's having metadata written out when it shouldn't anyway. I note that others have reported laptop disks spinning up recently for no obvious reason... Thor
Re: high sys time, very very slow builds on new 24-core system
On Thu, Mar 31, 2011 at 8:53 PM, Thor Lancelot Simon t...@panix.com wrote: [...] I wonder if some tool has reverted to using /var/tmp for its intermediate files, or similar. Don't see an obvious culprit for that, however. gcc uses /var/tmp by default unless you provide the -pipe flag, doesn't it? -- Julio Merino / @jmmv
covering set of PCI kernels?
Today I am trying to answer the question, what is the smallest set of kernel configurations to build, in order to compile all machine-dependent PCI code and all PCI drivers at least once? The answer I have come up with so far is, All GENERIC kernels for architectures having a pci/ directory, that is, these GENERIC kernels: % ls -d sys/arch/*/pci | sed 's/\/pci$//' | sed 's/$/\/conf\/GENERIC/' | \ xargs ls -d ls: sys/arch/algor/conf/GENERIC: No such file or directory ls: sys/arch/atari/conf/GENERIC: No such file or directory ls: sys/arch/powerpc/conf/GENERIC: No such file or directory ls: sys/arch/sgimips/conf/GENERIC: No such file or directory ls: sys/arch/x86/conf/GENERIC: No such file or directory sys/arch/alpha/conf/GENERIC sys/arch/ibmnws/conf/GENERIC sys/arch/arc/conf/GENERIC sys/arch/macppc/conf/GENERIC sys/arch/bebox/conf/GENERIC sys/arch/mvmeppc/conf/GENERIC sys/arch/cats/conf/GENERIC sys/arch/netwinder/conf/GENERIC sys/arch/cobalt/conf/GENERICsys/arch/ofppc/conf/GENERIC sys/arch/i386/conf/GENERIC sys/arch/prep/conf/GENERIC sys/arch/ia64/conf/GENERIC sys/arch/sandpoint/conf/GENERIC Plus some subset of the kernel configuration files in sys/arch/algor/conf/ sys/arch/atari/conf/ sys/arch/sgimips/conf/ (Which ones?) And an unknown number of powerpc configurations: evbppc, rs6000, what else? Dave -- David Young OJC Technologies dyo...@ojctech.com Urbana, IL * (217) 344-0444 x24
Re: covering set of PCI kernels?
I would be surprised if there are not some drivers in i386/conf/ALL that are not in GENERIC... On Thu, 31 Mar 2011, David Young wrote: Today I am trying to answer the question, what is the smallest set of kernel configurations to build, in order to compile all machine-dependent PCI code and all PCI drivers at least once? The answer I have come up with so far is, All GENERIC kernels for architectures having a pci/ directory, that is, these GENERIC kernels: % ls -d sys/arch/*/pci | sed 's/\/pci$//' | sed 's/$/\/conf\/GENERIC/' | \ xargs ls -d ls: sys/arch/algor/conf/GENERIC: No such file or directory ls: sys/arch/atari/conf/GENERIC: No such file or directory ls: sys/arch/powerpc/conf/GENERIC: No such file or directory ls: sys/arch/sgimips/conf/GENERIC: No such file or directory ls: sys/arch/x86/conf/GENERIC: No such file or directory sys/arch/alpha/conf/GENERIC sys/arch/ibmnws/conf/GENERIC sys/arch/arc/conf/GENERIC sys/arch/macppc/conf/GENERIC sys/arch/bebox/conf/GENERIC sys/arch/mvmeppc/conf/GENERIC sys/arch/cats/conf/GENERIC sys/arch/netwinder/conf/GENERIC sys/arch/cobalt/conf/GENERICsys/arch/ofppc/conf/GENERIC sys/arch/i386/conf/GENERIC sys/arch/prep/conf/GENERIC sys/arch/ia64/conf/GENERIC sys/arch/sandpoint/conf/GENERIC Plus some subset of the kernel configuration files in sys/arch/algor/conf/ sys/arch/atari/conf/ sys/arch/sgimips/conf/ (Which ones?) And an unknown number of powerpc configurations: evbppc, rs6000, what else? Dave -- David Young OJC Technologies dyo...@ojctech.com Urbana, IL * (217) 344-0444 x24 !DSPAM:4d94ed852444056616303! - | Paul Goyette | PGP Key fingerprint: | E-mail addresses: | | Customer Service | FA29 0E3B 35AF E8AE 6651 | paul at whooppee.com| | Network Engineer | 0786 F758 55DE 53BA 7731 | pgoyette at juniper.net | | Kernel Developer | | pgoyette at netbsd.org | -
Re: covering set of PCI kernels?
On Mar 31, 2011, at 2:09 PM, David Young wrote: Today I am trying to answer the question, what is the smallest set of kernel configurations to build, in order to compile all machine-dependent PCI code and all PCI drivers at least once? The answer I have come up with so far is, All GENERIC kernels for architectures having a pci/ directory, that is, these GENERIC kernels: You are ignoring the evb* which has a lot of MD PCI which aren't referenced any place else.
Re: covering set of PCI kernels?
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hello, On Mar 31, 2011, at 5:09 PM, David Young wrote: Today I am trying to answer the question, what is the smallest set of kernel configurations to build, in order to compile all machine-dependent PCI code and all PCI drivers at least once? The answer I have come up with so far is, All GENERIC kernels for architectures having a pci/ directory, that is, these GENERIC kernels: % ls -d sys/arch/*/pci | sed 's/\/pci$//' | sed 's/$/\/conf\/ GENERIC/' | \ xargs ls -d ls: sys/arch/algor/conf/GENERIC: No such file or directory ls: sys/arch/atari/conf/GENERIC: No such file or directory ls: sys/arch/powerpc/conf/GENERIC: No such file or directory ls: sys/arch/sgimips/conf/GENERIC: No such file or directory ls: sys/arch/x86/conf/GENERIC: No such file or directory sys/arch/alpha/conf/GENERIC sys/arch/ibmnws/conf/GENERIC sys/arch/arc/conf/GENERIC sys/arch/macppc/conf/GENERIC sys/arch/bebox/conf/GENERIC sys/arch/mvmeppc/conf/GENERIC sys/arch/cats/conf/GENERIC sys/arch/netwinder/conf/ GENERIC sys/arch/cobalt/conf/GENERICsys/arch/ofppc/conf/GENERIC sys/arch/i386/conf/GENERIC sys/arch/prep/conf/GENERIC sys/arch/ia64/conf/GENERIC sys/arch/sandpoint/conf/ GENERIC Plus some subset of the kernel configuration files in sys/arch/algor/conf/ sys/arch/atari/conf/ sys/arch/sgimips/conf/ (Which ones?) And an unknown number of powerpc configurations: evbppc, rs6000, what else? sgimips doesn't have a GENERIC but the IP2x and IP3x kernels have PCI - - IP3x has it natively, IP2x has some GIO ethernet cards that contain PCI bridges so you'll need both. have fun Michael -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.7 (Darwin) iQEVAwUBTZUjXcpnzkX8Yg2nAQJ/swf9HHnNpqsldD0A9Nm7kjrLoMHIsnTTghjl xEuyaWFS40tJMuTd/9y87QixFW5FcjQAkF0EHmgbwHslSQZZgdgTMXejdwJUnnH9 Pc3nkUuPMULBVd/87rI72lrvcMLcFv2CmCivPTCGHVfEU8KXiAXkncJOn0i5PEsK h6/n5U2pLMRD45KSkndZg253ZMgFbmZs+WLNj5vm97RoZUHyvnYtoY46nEtSya+n yTrTXo7MHbdLQHFwxyOUZ8KbRl81HzhH30gxEOZBshddXbdml0mYqw7WEkQ9Io8h YF2StoZa3ZGfPpBvOMUJxwdSp5JEziXafjPj69jlLyeTAkY/oPcygw== =vzC/ -END PGP SIGNATURE-
