Re: Concurrent execution of rc-scripts with rcorder(8)
2011/8/24 Vitaly Magerya :
>> the idea to start services concurrently during boot isn't new and the
>> question why FreeBSD doesn't do it has popped up on the forum and
>> mailing list occasionally. So, why not give it a shot?
>
> As someone who uses FreeBSD on hist laptop and is constantly annoyed
> by the lack of suspend-to-disk, every second trimmed of from boot
> time is a win.
>
> In line of the recent FreeBSD problems & solutions discussion, would
> any commiter take time to review and commit this? "FreeBSD 9.1
> introduces concurrent startup, improves boot speed" is the kind of
> buzz we're after.
>
>> Any ideas and feedback are very welcome!
>
> One thing to try is to attach a diagnostics feature that will produce
> data about rc script dependencies and execution times, which can
> be used to visualize which scripts take most time, and how to
> reorganize dependencies to improve boot time (one example I noticed
> is moused: it is only started after network is up, which is a shame,
> since it could easily start while DHCP negotiation is in progress).
Perhaps background_dhclient="YES" should solve it ? I think
background approach (which is current archlinux one [1] ) is not so
bad. It's clearly less powerful than automagic parallelization but
it's maybe less invasive and more flexible for sysadmins.
I gave it a try with a little patch for rc.subr that introduces a
background keyword (eg: moused_enable="background"). It's surly buggy
with some variables like rc_quiet. I have to check more.
[1] https://wiki.archlinux.org/index.php/DAEMONS
Joris
> ___
> freebsd-hackers@freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
> To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
>
--- /etc/rc.subr 2011-05-02 08:49:11.0 +0200
+++ rc.subr 2011-08-25 13:50:29.300275783 +0200
@@ -142,8 +142,9 @@
debug "checkyesno: $1 is set to $_value."
case $_value in
- # "yes", "true", "on", or "1"
- [Yy][Ee][Ss]|[Tt][Rr][Uu][Ee]|[Oo][Nn]|1)
+ # "yes", "true", "on", "1", "bg" or "background"
+ [Yy][Ee][Ss]|[Tt][Rr][Uu][Ee]|[Oo][Nn]|1|[Bb][Gg]|\
+[Bb][Aa][Cc][Kk][Gg][Rr][Oo][Uu][Nn][Dd])
return 0
;;
@@ -159,6 +160,25 @@
}
#
+# checkbg var
+# Test $1 variable, and return 0 if it's background or bg.
+# Return nonzero otherwise.
+#
+checkbg()
+{
+ eval _value=\$${1}
+ debug "checkbg: $1 is set to $_value."
+ case $_value in
+ [Bb][Gg]|[Bb][Aa][Cc][Kk][Gg][Rr][Oo][Uu][Nn][Dd])
+ return 0
+ ;;
+ *)
+ return 1
+ ;;
+ esac
+}
+
+#
# reverse_list list
# print the list in reverse order
#
@@ -735,54 +755,11 @@
;;
start)
- if [ -z "$rc_fast" -a -n "$rc_pid" ]; then
-echo 1>&2 "${name} already running? (pid=$rc_pid)."
-return 1
+ if checkbg ${rcvar}; then
+eval _run_rc_start &
+ else
+_run_rc_start
fi
-
- if [ ! -x ${_chroot}${_chroot:+"/"}${command} ]; then
-warn "run_rc_command: cannot run $command"
-return 1
- fi
-
- if ! _run_rc_precmd; then
-warn "failed precmd routine for ${name}"
-return 1
- fi
-
- # setup the full command to run
- #
- check_startmsgs && echo "Starting ${name}."
- if [ -n "$_chroot" ]; then
-_doit="\
-${_nice:+nice -n $_nice }\
-chroot ${_user:+-u $_user }${_group:+-g $_group }${_groups:+-G $_groups }\
-$_chroot $command $rc_flags $command_args"
- else
-_doit="\
-${_chdir:+cd $_chdir && }\
-$command $rc_flags $command_args"
-if [ -n "$_user" ]; then
-_doit="su -m $_user -c 'sh -c \"$_doit\"'"
-fi
-if [ -n "$_nice" ]; then
- if [ -z "$_user" ]; then
- _doit="sh -c \"$_doit\""
- fi
- _doit="nice -n $_nice $_doit"
-fi
- fi
-
- # run the full command
- #
- if ! _run_rc_doit "$_doit"; then
-warn "failed to start ${name}"
-return 1
- fi
-
- # finally, run postcmd
- #
- _run_rc_postcmd
;;
stop)
@@ -985,6 +962,59 @@
echo "$_cmd"
}
+_run_rc_start()
+{
+ if [ -z "$rc_fast" -a -n "$rc_pid" ]; then
+ echo 1>&2 "${name} already running? (pid=$rc_pid)."
+ return 1
+ fi
+
+ if [ ! -x ${_chroot}${_chroot:+"/"}${command} ]; then
+ warn "run_rc_command: cannot run $command"
+ return 1
+ fi
+
+ if ! _run_rc_precmd; then
+ warn "failed precmd routine for ${name}"
+ return 1
+ fi
+
+ # setup the full command to run
+ #
+ check_startmsgs && echo "Starting ${name}."
+ if [ -n "$_chroot" ]; then
+_doit="\
+${_nice:+nice -n $_nice }\
+chroot ${_user:+-u $_user }${_group:+-g $_group }${_groups:+-G $_groups }\
+$_chroot $command $rc_flags $command_args"
+ else
+_doit="\
+${_chdir:+cd $_chdir && }\
+$command $rc_flags $command_args"
+ if [ -n "$_user" ]; then
+ _doit="su -m $_user -c 'sh -c \"$_doit\"'"
+ fi
+ if [ -n "$_nice" ]; then
+ if [ -z "$_user" ]; then
+_doit="sh -c \"$_doit\""
+ fi
+ _doit="nice -n $_nice $_doit"
+ fi
+ fi
+
+ # run the full command
+ #
+ if ! _run_rc_doit "$_doit"; then
+ warn
Re: Concurrent execution of rc-scripts with rcorder(8)
Further optimization in rc.d: splitting non-networking DAEMON (eg. moused) Critical issue in rcexecr: stop_boot (in /etc/rc.subr) doesn't stop rcexecr it might need a check before involving /usr/local/etc/rc.d/*, or just kill child processes? --Buganini ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: Concurrent execution of rc-scripts with rcorder(8)
rcexecr generate time table like this: # env ARG0=\? ARG1=start rcexecr /etc/rc.d/* 1 beg /etc/rc.d/dhclient start 1 beg /etc/rc.d/rctl start 1 beg /etc/rc.d/natd start 1 beg /etc/rc.d/ugidfw start 1 beg /etc/rc.d/sysctl start 2 end /etc/rc.d/sysctl 3 beg /etc/rc.d/hostid start 4 end /etc/rc.d/hostid 5 beg /etc/rc.d/zvol start 6 end /etc/rc.d/zvol 7 beg /etc/rc.d/dumpon start 8 end /etc/rc.d/dumpon 9 beg /etc/rc.d/ddb start 10 end /etc/rc.d/ddb 11 beg /etc/rc.d/initrandom start 12 end /etc/rc.d/initrandom 13 beg /etc/rc.d/gbde start 13 beg /etc/rc.d/ccd start 13 beg /etc/rc.d/encswap start 13 beg /etc/rc.d/geli start 14 end /etc/rc.d/gbde 14 end /etc/rc.d/encswap 14 end /etc/rc.d/geli 14 end /etc/rc.d/ccd 15 beg /etc/rc.d/swap1 start 16 end /etc/rc.d/swap1 17 beg /etc/rc.d/fsck start 18 end /etc/rc.d/fsck 19 beg /etc/rc.d/root start 20 end /etc/rc.d/root 21 beg /etc/rc.d/serial start 21 beg /etc/rc.d/mdconfig start 21 beg /etc/rc.d/sppp start 21 beg /etc/rc.d/nsswitch start 21 beg /etc/rc.d/gssd start 21 beg /etc/rc.d/hostid_save start 21 beg /etc/rc.d/atm1 start 22 end /etc/rc.d/hostid_save 22 end /etc/rc.d/mdconfig 23 beg /etc/rc.d/mountcritlocal start 24 end /etc/rc.d/mountcritlocal 25 beg /etc/rc.d/zfs start 26 end /etc/rc.d/zfs 27 beg /etc/rc.d/FILESYSTEMS start 28 end /etc/rc.d/FILESYSTEMS 29 beg /etc/rc.d/hostname start 29 beg /etc/rc.d/ip6addrctl start 29 beg /etc/rc.d/geli2 start 29 beg /etc/rc.d/kldxref start 29 beg /etc/rc.d/kld start 29 beg /etc/rc.d/ipfilter start 29 beg /etc/rc.d/netoptions start 29 beg /etc/rc.d/addswap start 29 beg /etc/rc.d/ipsec start 30 end /etc/rc.d/ipfilter 30 end /etc/rc.d/kld 31 beg /etc/rc.d/ipnat start 31 beg /etc/rc.d/var start 32 end /etc/rc.d/ipnat 32 end /etc/rc.d/var 33 beg /etc/rc.d/ipfs start 33 beg /etc/rc.d/random start 33 beg /etc/rc.d/cleanvar start 34 end /etc/rc.d/hostname 34 end /etc/rc.d/random 35 beg /etc/rc.d/ipmon start 35 beg /etc/rc.d/adjkerntz start 35 beg /etc/rc.d/mixer start 36 end /etc/rc.d/atm1 36 end /etc/rc.d/serial 36 end /etc/rc.d/ip6addrctl 36 end /etc/rc.d/netoptions 36 end /etc/rc.d/adjkerntz 36 end /etc/rc.d/kldxref 36 end /etc/rc.d/sppp 36 end /etc/rc.d/ipfs 36 end /etc/rc.d/cleanvar 37 beg /etc/rc.d/netif start 38 end /etc/rc.d/netif 39 beg /etc/rc.d/ppp start 39 beg /etc/rc.d/rtsold start 39 beg /etc/rc.d/faith start 39 beg /etc/rc.d/pfsync start 39 beg /etc/rc.d/pflog start 39 beg /etc/rc.d/devd start 39 beg /etc/rc.d/static_arp start 39 beg /etc/rc.d/stf start 39 beg /etc/rc.d/bridge start 39 beg /etc/rc.d/resolv start 39 beg /etc/rc.d/atm2 start 40 end /etc/rc.d/devd 40 end /etc/rc.d/atm2 40 end /etc/rc.d/pfsync 40 end /etc/rc.d/pflog 41 beg /etc/rc.d/defaultroute start 41 beg /etc/rc.d/atm3 start 41 beg /etc/rc.d/pf start 41 beg /etc/rc.d/ipfw start 42 end /etc/rc.d/faith 42 end /etc/rc.d/ppp 42 end /etc/rc.d/stf 42 end /etc/rc.d/pf 43 beg /etc/rc.d/routing start 44 end /etc/rc.d/routing 45 beg /etc/rc.d/mroute6d start 45 beg /etc/rc.d/route6d start 45 beg /etc/rc.d/routed start 45 beg /etc/rc.d/mrouted start 46 end /etc/rc.d/routed 46 end /etc/rc.d/resolv 46 end /etc/rc.d/rtsold 46 end /etc/rc.d/ipfw 46 end /etc/rc.d/mroute6d 46 end /etc/rc.d/nsswitch 46 end /etc/rc.d/defaultroute 46 end /etc/rc.d/mrouted 46 end /etc/rc.d/route6d 47 beg /etc/rc.d/NETWORKING start 48 end /etc/rc.d/NETWORKING 49 beg /etc/rc.d/pppoed start 49 beg /etc/rc.d/kerberos start 49 beg /etc/rc.d/netwait start 49 beg /etc/rc.d/nfsuserd start 50 end /etc/rc.d/kerberos 50 end /etc/rc.d/ipsec 50 end /etc/rc.d/netwait 51 beg /etc/rc.d/mountcritremote start 51 beg /etc/rc.d/nfscbd start 51 beg /etc/rc.d/kadmind start 52 end /etc/rc.d/kadmind 52 end /etc/rc.d/mountcritremote 53 beg /etc/rc.d/motd start 53 beg /etc/rc.d/wpa_supplicant start 53 beg /etc/rc.d/tmp s
Re: Concurrent execution of rc-scripts with rcorder(8)
On 24 Aug 2011, at 10:08, Vitaly Magerya wrote: > One thing to try is to attach a diagnostics feature that will produce > data about rc script dependencies and execution times, which can > be used to visualize which scripts take most time, and how to > reorganize dependencies to improve boot time (one example I noticed > is moused: it is only started after network is up, which is a shame, > since it could easily start while DHCP negotiation is in progress). Perhaps https://github.com/buganini/rcexecr may be a good start. Dw ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: Concurrent execution of rc-scripts with rcorder(8)
> the idea to start services concurrently during boot isn't new and the > question why FreeBSD doesn't do it has popped up on the forum and > mailing list occasionally. So, why not give it a shot? As someone who uses FreeBSD on hist laptop and is constantly annoyed by the lack of suspend-to-disk, every second trimmed of from boot time is a win. In line of the recent FreeBSD problems & solutions discussion, would any commiter take time to review and commit this? "FreeBSD 9.1 introduces concurrent startup, improves boot speed" is the kind of buzz we're after. > Any ideas and feedback are very welcome! One thing to try is to attach a diagnostics feature that will produce data about rc script dependencies and execution times, which can be used to visualize which scripts take most time, and how to reorganize dependencies to improve boot time (one example I noticed is moused: it is only started after network is up, which is a shame, since it could easily start while DHCP negotiation is in progress). ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: Concurrent execution of rc-scripts with rcorder(8)
On Sun, Aug 21, 2011 at 02:54:15PM +0100, Chris Rees wrote: > On 21 Aug 2011 13:39, "kilian" wrote: > > > > Hello, > > > > the idea to start services concurrently during boot isn't new and the > > question why FreeBSD doesn't do it has popped up on the forum and > > mailing list occasionally. So, why not give it a shot? > > > > rcorder(8) is normally used during boot to bring the rc-scripts into a > > particular order, so when they are executed linearly by /etc/rc, all > > constraints will be satisfied. I modified rcorder(8) to enable it to > > run rc-scripts concurrently, while keeping track of the constraints as > > rc-scripts start and finish. You can find the code at > > https://github.com/kil/rcorder. As it works now, it will fall back to the > > current mode of execution if anything goes wrong. So, if worst comes to > > worst, booting takes a bit longer. > > > > If you feel brave, give it a try (Actually, not too much bravery is > needed: > > on all boots of my machine it worked perfectly every time.) > > > > I haven't done any measurements yet on how large the speedup is, but > booting > > feels a bit faster with it. Also, there probably is room for improvement. > > Any ideas and feedback are very welcome! > > > > -kilian > > > > I might suggest moving this to rc@. I'll try it later, looks interesting. > > Chris For anyone who is interested, updated the README[1] with some numbers, detailing the influence on booting time. [1] https://github.com/kil/rcorder/blob/master/README#L30 ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: Concurrent execution of rc-scripts with rcorder(8)
On Sun, Aug 21, 2011 at 04:14:02PM +0300, Vitaly Magerya wrote: > > I haven't done any measurements yet on how large the speedup is, but booting > > feels a bit faster with it. > > FWIW, my laptop's boot time (from pressing "power" to seeing login > prompt) is reduced from 44s to 37s (out of which, first 20-25s are > kernel booting). Thats about the same speedup I'm seeing. Also, today I found a bug in the /etc/rc patch that caused too many scripts to be run. I doubt it has a large impact but it maybe worth a try. > ___ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org" > ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: Concurrent execution of rc-scripts with rcorder(8)
On Sun, Aug 21, 2011 at 04:18:10PM +0200, joris dedieu wrote: > 2011/8/21 kilian : > > Hello, > > > > the idea to start services concurrently during boot isn't new and the > > question why FreeBSD doesn't do it has popped up on the forum and > > mailing list occasionally. So, why not give it a shot? > > > > rcorder(8) is normally used during boot to bring the rc-scripts into a > > particular order, so when they are executed linearly by /etc/rc, all > > constraints will be satisfied. I modified rcorder(8) to enable it to > > run rc-scripts concurrently, while keeping track of the constraints as > > rc-scripts start and finish. You can find the code at > > https://github.com/kil/rcorder. As it works now, it will fall back to the > > current mode of execution if anything goes wrong. So, if worst comes to > > worst, booting takes a bit longer. > > > > If you feel brave, give it a try (Actually, not too much bravery is needed: > > on all boots of my machine it worked perfectly every time.) > > > > I haven't done any measurements yet on how large the speedup is, but booting > > feels a bit faster with it. Also, there probably is room for improvement. > > Any ideas and feedback are very welcome! > > There is another implementation which had been posted here (maybe) > some times ago. Maybe a good thing to compare. > > https://github.com/buganini/rcexecr Didn't know someone else already implemented this. I had a (quick) look at the source code and tried it out, too. Seems to do its job quite well. I'm (obviously) biased in my opinion but what might be considered a drawback of the rcexecr implementation is that it requires modifications to some files in /etc/rc.d. The modifications to /etc/rc are also more invasive. rcexecr is a whole new program, based on rcorder. My implementation requires changes only to /etc/rc and adds features to rcorder (preserving compatibility, I might add). - kilian > > Joris > > > > -kilian > > ___ > > freebsd-hackers@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > > To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org" > > > ___ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org" > ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: Concurrent execution of rc-scripts with rcorder(8)
2011/8/21 kilian : > Hello, > > the idea to start services concurrently during boot isn't new and the > question why FreeBSD doesn't do it has popped up on the forum and > mailing list occasionally. So, why not give it a shot? > > rcorder(8) is normally used during boot to bring the rc-scripts into a > particular order, so when they are executed linearly by /etc/rc, all > constraints will be satisfied. I modified rcorder(8) to enable it to > run rc-scripts concurrently, while keeping track of the constraints as > rc-scripts start and finish. You can find the code at > https://github.com/kil/rcorder. As it works now, it will fall back to the > current mode of execution if anything goes wrong. So, if worst comes to > worst, booting takes a bit longer. > > If you feel brave, give it a try (Actually, not too much bravery is needed: > on all boots of my machine it worked perfectly every time.) > > I haven't done any measurements yet on how large the speedup is, but booting > feels a bit faster with it. Also, there probably is room for improvement. > Any ideas and feedback are very welcome! There is another implementation which had been posted here (maybe) some times ago. Maybe a good thing to compare. https://github.com/buganini/rcexecr Joris > > -kilian > ___ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org" > ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: Concurrent execution of rc-scripts with rcorder(8)
*** Sorry, emailed from wrong address and it bounced from hackers ***: I might suggest moving this to rc@. I'll try it later, looks interesting. Chris ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: Concurrent execution of rc-scripts with rcorder(8)
Here is another approach: https://github.com/buganini/rcexecr -- Buganini ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: Concurrent execution of rc-scripts with rcorder(8)
> I haven't done any measurements yet on how large the speedup is, but booting > feels a bit faster with it. FWIW, my laptop's boot time (from pressing "power" to seeing login prompt) is reduced from 44s to 37s (out of which, first 20-25s are kernel booting). ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Concurrent execution of rc-scripts with rcorder(8)
Hello, the idea to start services concurrently during boot isn't new and the question why FreeBSD doesn't do it has popped up on the forum and mailing list occasionally. So, why not give it a shot? rcorder(8) is normally used during boot to bring the rc-scripts into a particular order, so when they are executed linearly by /etc/rc, all constraints will be satisfied. I modified rcorder(8) to enable it to run rc-scripts concurrently, while keeping track of the constraints as rc-scripts start and finish. You can find the code at https://github.com/kil/rcorder. As it works now, it will fall back to the current mode of execution if anything goes wrong. So, if worst comes to worst, booting takes a bit longer. If you feel brave, give it a try (Actually, not too much bravery is needed: on all boots of my machine it worked perfectly every time.) I haven't done any measurements yet on how large the speedup is, but booting feels a bit faster with it. Also, there probably is room for improvement. Any ideas and feedback are very welcome! -kilian ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
