Re: make buildworld is now 50% slower
On Sun, Jul 07, 2013 at 11:50:29AM +0300, Daniel Braniss wrote: On Fri, Jul 05, 2013 at 02:39:00PM +0200, Dimitry Andric wrote: [redirecting to the correct mailing list, freebsd-stable@ ...] On Jul 5, 2013, at 10:53, Daniel Braniss da...@cs.huji.ac.il wrote: after today's update of 9.1-STABLE I noticed that make build[world|kernel] are taking conciderable more time, is it because the upgrade of clang? and if so, is the code produced any better? before: buildwordl:26m4.52s real 2h28m32.12s user 36m6.27s sys buildkernel: 7m29.42s real 23m22.22s user 4m26.26s sys today: buildwordl: 34m29.80s real 2h38m9.37s user 37m7.61s sys buildkernel:15m31.52s real 22m59.40s user 4m33.06s sys Ehm, your user and sys times are not that much different at all, they add up to about 5% slower for buildworld, and 1% faster for build kernel. Are you sure nothing else is running on that machine, eating up CPU time while you are building? :) But yes, clang 3.3 is of course somewhat larger than 3.2. You might especially notice that, if you are using gcc, which is very slow at compiling C++. In any case, if you do not care about clang, just set WITHOUT_CLANG= in your /etc/src.conf, and you can shave off some build time. I just built world/kernel (stable/9 r252769) 5 hours ago. Results: time make -j4 buildworld = roughly 21 minutes on my hardware time make -j4 buildkernel = roughly 8 minutes on my hardware It's been a long time since I saw such numbers, maybe it's time to see where time is being spent, I will run it without clang to compare with your numbers. These numbers are about the norm for me, meaning I do not see a substantial increase in build times. Key point: I do not use/build/grok clang, i.e. WITHOUT_CLANG=true is in my src.conf. But I am aware of the big clang change in r252723. If hardware details are wanted, ask, but I don't think it's relevant to what the root cause is. from what you are saying, I guess clang is not responsible. looking for my Sherlock Holmes hat. Some points to those numbers I stated above: - System is an Intel Q9550 with 8GB of RAM - Single SSD (UFS2+SU+TRIM) is used for root, /usr, /var, /tmp, and swap - /usr/src is on ZFS (raidz1 + 3 disks) -- however I got equally small numbers when it was on the SSD - /usr/src is using compression=lz4 (to folks from -fs: yeah, I'm trying it out to see how much of an impact it has on interactivity. I can still tell when it kicks in, but it's way, way better than lzjb. Rather not get into that here) - Contents of /etc/src.conf (to give you some idea of what I disable): WITHOUT_ATM=true WITHOUT_BLUETOOTH=true WITHOUT_CLANG=true WITHOUT_FLOPPY=true WITHOUT_FREEBSD_UPDATE=true WITHOUT_INET6=true WITHOUT_IPFILTER=true WITHOUT_IPX=true WITHOUT_KERBEROS=true WITHOUT_LIB32=true WITHOUT_LPR=true WITHOUT_NDIS=true WITHOUT_NETGRAPH=true WITHOUT_PAM_SUPPORT=true WITHOUT_PPP=true WITHOUT_SENDMAIL=true WITHOUT_WIRELESS=true WITH_OPENSSH_NONE_CIPHER=true It's WITHOUT_CLANG that cuts down the buildworld time by a *huge* amount (I remember when it got introduced, my buildworld jumped up to something like 40 minutes); the rest probably save a minute or two at most. - /etc/make.conf doesn't contain much that's relevant, other than: CPUTYPE?=core2 # For DTrace; also affects ports STRIP= CFLAGS+= -fno-omit-frame-pointer - I do some tweaks in /etc/sysctl.conf (mainly vfs.read_min and vfs.read_max), but I will admit I am not completely sure what those do quite yet (I just saw the commit from scottl@ a while back talking about how an increased vfs.read_min helps them at Netflix quite a lot). I also adjust kern.maxvnodes. - Some ZFS ARC settings are adjusted in /boot/loader.conf (I'm playing with some stuff I read in Andriy Gapon's ZFS PDF), but they definitely do not have a major impact on the numbers I listed off. - I do increase kern.maxdsiz, kern.dfldsiz, and kern.maxssiz in /boot/loader.conf to 2560M/2560M/256M respectively, but that was mainly from the days when I ran MySQL and needed a huge userland processes. All in all my numbers are low/small because of two things: the SSD, and WITHOUT_CLANG. Hope this gives you somewhere to start/stuff to ponder. indeed! on my pretty much standard dev machine, PowerEdge R710, X5550 @ 2.67GHz, 16384 MB, root on ufs, the rest is via ZFS, this is what I'm getting for buildworld: with clang 27m32.91s real 2h42m52.82s user 36m20.69s sys without clang 13m24.19s real 1h23m26.52s user 29m5.18s sys on a similar machine but with root via NFS, and /var on ZFS, the numbers are similar: with clang 23m30.92s real 2h9m8.85s user 29m27.84s sys without clang 12m7.53s real 1h7m54.24s user 23m9.78s sys (this host is newer, PowerEdge
Re: make buildworld is now 50% slower
On Fri, Jul 05, 2013 at 02:39:00PM +0200, Dimitry Andric wrote: [redirecting to the correct mailing list, freebsd-stable@ ...] On Jul 5, 2013, at 10:53, Daniel Braniss da...@cs.huji.ac.il wrote: after today's update of 9.1-STABLE I noticed that make build[world|kernel] are taking conciderable more time, is it because the upgrade of clang? and if so, is the code produced any better? before: buildwordl:26m4.52s real 2h28m32.12s user 36m6.27s sys buildkernel: 7m29.42s real 23m22.22s user 4m26.26s sys today: buildwordl: 34m29.80s real 2h38m9.37s user 37m7.61s sys buildkernel:15m31.52s real 22m59.40s user 4m33.06s sys Ehm, your user and sys times are not that much different at all, they add up to about 5% slower for buildworld, and 1% faster for build kernel. Are you sure nothing else is running on that machine, eating up CPU time while you are building? :) But yes, clang 3.3 is of course somewhat larger than 3.2. You might especially notice that, if you are using gcc, which is very slow at compiling C++. In any case, if you do not care about clang, just set WITHOUT_CLANG= in your /etc/src.conf, and you can shave off some build time. I just built world/kernel (stable/9 r252769) 5 hours ago. Results: time make -j4 buildworld = roughly 21 minutes on my hardware time make -j4 buildkernel = roughly 8 minutes on my hardware It's been a long time since I saw such numbers, maybe it's time to see where time is being spent, I will run it without clang to compare with your numbers. These numbers are about the norm for me, meaning I do not see a substantial increase in build times. Key point: I do not use/build/grok clang, i.e. WITHOUT_CLANG=true is in my src.conf. But I am aware of the big clang change in r252723. If hardware details are wanted, ask, but I don't think it's relevant to what the root cause is. from what you are saying, I guess clang is not responsible. looking for my Sherlock Holmes hat. thanks, danny -- | Jeremy Chadwick j...@koitsu.org | | UNIX Systems Administratorhttp://jdc.koitsu.org/ | | Making life hard for others since 1977. PGP 4BD6C0CB | ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: make buildworld is now 50% slower
On Sun, Jul 07, 2013 at 11:50:29AM +0300, Daniel Braniss wrote: On Fri, Jul 05, 2013 at 02:39:00PM +0200, Dimitry Andric wrote: [redirecting to the correct mailing list, freebsd-stable@ ...] On Jul 5, 2013, at 10:53, Daniel Braniss da...@cs.huji.ac.il wrote: after today's update of 9.1-STABLE I noticed that make build[world|kernel] are taking conciderable more time, is it because the upgrade of clang? and if so, is the code produced any better? before: buildwordl: 26m4.52s real 2h28m32.12s user 36m6.27s sys buildkernel: 7m29.42s real 23m22.22s user 4m26.26s sys today: buildwordl: 34m29.80s real 2h38m9.37s user 37m7.61s sys buildkernel:15m31.52s real 22m59.40s user 4m33.06s sys Ehm, your user and sys times are not that much different at all, they add up to about 5% slower for buildworld, and 1% faster for build kernel. Are you sure nothing else is running on that machine, eating up CPU time while you are building? :) But yes, clang 3.3 is of course somewhat larger than 3.2. You might especially notice that, if you are using gcc, which is very slow at compiling C++. In any case, if you do not care about clang, just set WITHOUT_CLANG= in your /etc/src.conf, and you can shave off some build time. I just built world/kernel (stable/9 r252769) 5 hours ago. Results: time make -j4 buildworld = roughly 21 minutes on my hardware time make -j4 buildkernel = roughly 8 minutes on my hardware It's been a long time since I saw such numbers, maybe it's time to see where time is being spent, I will run it without clang to compare with your numbers. These numbers are about the norm for me, meaning I do not see a substantial increase in build times. Key point: I do not use/build/grok clang, i.e. WITHOUT_CLANG=true is in my src.conf. But I am aware of the big clang change in r252723. If hardware details are wanted, ask, but I don't think it's relevant to what the root cause is. from what you are saying, I guess clang is not responsible. looking for my Sherlock Holmes hat. Some points to those numbers I stated above: - System is an Intel Q9550 with 8GB of RAM - Single SSD (UFS2+SU+TRIM) is used for root, /usr, /var, /tmp, and swap - /usr/src is on ZFS (raidz1 + 3 disks) -- however I got equally small numbers when it was on the SSD - /usr/src is using compression=lz4 (to folks from -fs: yeah, I'm trying it out to see how much of an impact it has on interactivity. I can still tell when it kicks in, but it's way, way better than lzjb. Rather not get into that here) - Contents of /etc/src.conf (to give you some idea of what I disable): WITHOUT_ATM=true WITHOUT_BLUETOOTH=true WITHOUT_CLANG=true WITHOUT_FLOPPY=true WITHOUT_FREEBSD_UPDATE=true WITHOUT_INET6=true WITHOUT_IPFILTER=true WITHOUT_IPX=true WITHOUT_KERBEROS=true WITHOUT_LIB32=true WITHOUT_LPR=true WITHOUT_NDIS=true WITHOUT_NETGRAPH=true WITHOUT_PAM_SUPPORT=true WITHOUT_PPP=true WITHOUT_SENDMAIL=true WITHOUT_WIRELESS=true WITH_OPENSSH_NONE_CIPHER=true It's WITHOUT_CLANG that cuts down the buildworld time by a *huge* amount (I remember when it got introduced, my buildworld jumped up to something like 40 minutes); the rest probably save a minute or two at most. - /etc/make.conf doesn't contain much that's relevant, other than: CPUTYPE?=core2 # For DTrace; also affects ports STRIP= CFLAGS+=-fno-omit-frame-pointer - I do some tweaks in /etc/sysctl.conf (mainly vfs.read_min and vfs.read_max), but I will admit I am not completely sure what those do quite yet (I just saw the commit from scottl@ a while back talking about how an increased vfs.read_min helps them at Netflix quite a lot). I also adjust kern.maxvnodes. - Some ZFS ARC settings are adjusted in /boot/loader.conf (I'm playing with some stuff I read in Andriy Gapon's ZFS PDF), but they definitely do not have a major impact on the numbers I listed off. - I do increase kern.maxdsiz, kern.dfldsiz, and kern.maxssiz in /boot/loader.conf to 2560M/2560M/256M respectively, but that was mainly from the days when I ran MySQL and needed a huge userland processes. All in all my numbers are low/small because of two things: the SSD, and WITHOUT_CLANG. Hope this gives you somewhere to start/stuff to ponder. -- | Jeremy Chadwick j...@koitsu.org | | UNIX Systems Administratorhttp://jdc.koitsu.org/ | | Making life hard for others since 1977. PGP 4BD6C0CB | ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: make buildworld is now 50% slower
Apropos of nothing, but...
On Sun, Jul 07, 2013 at 03:17:14AM -0700 I heard the voice of
Jeremy Chadwick, and lo! it spake thus:
WITHOUT_LIB32=true
suggests you're running amd64, which I'm pretty sure means
- I do increase kern.maxdsiz, kern.dfldsiz, and kern.maxssiz in
/boot/loader.conf to 2560M/2560M/256M respectively, but that was mainly
from the days when I ran MySQL and needed a huge userland processes.
are not necessarily _in_creases, and may well be mostly _de_creases.
e.g., on a RELENG_9 box with 8 gig of physical RAM:
% sysctl kern.{max{d,s},dfld}siz
kern.maxdsiz: 34359738368
kern.maxssiz: 536870912
kern.dfldsiz: 134217728
while a -CURRENT box with 16 has dfldsiz blown all the way up too. I
don't recall doing anything to change them at all recently, and a
glance over loader.conf, sysctl.conf, rc.local, and the kernel configs
doesn't turn up anything.
--
Matthew Fuller (MF4839) | fulle...@over-yonder.net
Systems/Network Administrator | http://www.over-yonder.net/~fullermd/
On the Internet, nobody can hear you scream.
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: make buildworld is now 50% slower
On Sun, Jul 07, 2013 at 05:47:31AM -0500, Matthew D. Fuller wrote:
Apropos of nothing, but...
On Sun, Jul 07, 2013 at 03:17:14AM -0700 I heard the voice of
Jeremy Chadwick, and lo! it spake thus:
WITHOUT_LIB32=true
suggests you're running amd64, which I'm pretty sure means
- I do increase kern.maxdsiz, kern.dfldsiz, and kern.maxssiz in
/boot/loader.conf to 2560M/2560M/256M respectively, but that was mainly
from the days when I ran MySQL and needed a huge userland processes.
are not necessarily _in_creases, and may well be mostly _de_creases.
e.g., on a RELENG_9 box with 8 gig of physical RAM:
% sysctl kern.{max{d,s},dfld}siz
kern.maxdsiz: 34359738368
kern.maxssiz: 536870912
kern.dfldsiz: 134217728
while a -CURRENT box with 16 has dfldsiz blown all the way up too. I
don't recall doing anything to change them at all recently, and a
glance over loader.conf, sysctl.conf, rc.local, and the kernel configs
doesn't turn up anything.
Thanks!
The settings I mention are from ancient times -- specifically RELENG_6
on i386 (I know because I found an old mailing list post of mine
discussing the settings with a user).
The problem as I said was that mysqld would crap itself (crash and be
quite loud about it) if the process allocated too much memory/became too
large. I am fairly certain the issue related to the data size, **not**
the stack size (but I didn't see the harm in increasing that either).
It's good to know I can remove these on amd64. Yay, one less thing in
loader.conf I have to deal with... :-) Thanks again!
--
| Jeremy Chadwick j...@koitsu.org |
| UNIX Systems Administratorhttp://jdc.koitsu.org/ |
| Making life hard for others since 1977. PGP 4BD6C0CB |
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: make buildworld is now 50% slower
[redirecting to the correct mailing list, freebsd-stable@ ...] On Jul 5, 2013, at 10:53, Daniel Braniss da...@cs.huji.ac.il wrote: after today's update of 9.1-STABLE I noticed that make build[world|kernel] are taking conciderable more time, is it because the upgrade of clang? and if so, is the code produced any better? before: buildwordl:26m4.52s real 2h28m32.12s user 36m6.27s sys buildkernel: 7m29.42s real 23m22.22s user 4m26.26s sys today: buildwordl: 34m29.80s real 2h38m9.37s user 37m7.61s sys buildkernel:15m31.52s real 22m59.40s user 4m33.06s sys Ehm, your user and sys times are not that much different at all, they add up to about 5% slower for buildworld, and 1% faster for build kernel. Are you sure nothing else is running on that machine, eating up CPU time while you are building? :) But yes, clang 3.3 is of course somewhat larger than 3.2. You might especially notice that, if you are using gcc, which is very slow at compiling C++. In any case, if you do not care about clang, just set WITHOUT_CLANG= in your /etc/src.conf, and you can shave off some build time. -Dimitry ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: make buildworld is now 50% slower
On Fri, Jul 05, 2013 at 02:39:00PM +0200, Dimitry Andric wrote: [redirecting to the correct mailing list, freebsd-stable@ ...] On Jul 5, 2013, at 10:53, Daniel Braniss da...@cs.huji.ac.il wrote: after today's update of 9.1-STABLE I noticed that make build[world|kernel] are taking conciderable more time, is it because the upgrade of clang? and if so, is the code produced any better? before: buildwordl: 26m4.52s real 2h28m32.12s user 36m6.27s sys buildkernel: 7m29.42s real 23m22.22s user 4m26.26s sys today: buildwordl: 34m29.80s real 2h38m9.37s user 37m7.61s sys buildkernel:15m31.52s real 22m59.40s user 4m33.06s sys Ehm, your user and sys times are not that much different at all, they add up to about 5% slower for buildworld, and 1% faster for build kernel. Are you sure nothing else is running on that machine, eating up CPU time while you are building? :) But yes, clang 3.3 is of course somewhat larger than 3.2. You might especially notice that, if you are using gcc, which is very slow at compiling C++. In any case, if you do not care about clang, just set WITHOUT_CLANG= in your /etc/src.conf, and you can shave off some build time. I just built world/kernel (stable/9 r252769) 5 hours ago. Results: time make -j4 buildworld = roughly 21 minutes on my hardware time make -j4 buildkernel = roughly 8 minutes on my hardware These numbers are about the norm for me, meaning I do not see a substantial increase in build times. Key point: I do not use/build/grok clang, i.e. WITHOUT_CLANG=true is in my src.conf. But I am aware of the big clang change in r252723. If hardware details are wanted, ask, but I don't think it's relevant to what the root cause is. -- | Jeremy Chadwick j...@koitsu.org | | UNIX Systems Administratorhttp://jdc.koitsu.org/ | | Making life hard for others since 1977. PGP 4BD6C0CB | ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
