Re: can we get some interaction between halt/reboot and bhyve?
Hi Paul On Tue, May 26, 2015 at 11:18 PM, Paul Vixie wrote: > > > Craig Rodrigues wrote: >> ... >> >> I haven't tried them, but Michael Dexter has written more comprehensive >> rc.d scripts from bhyve: >> >> http://vmrc.bsd.lv/ > > these look good but i'm waiting for man pages, while meanwhile i bake my > own toolset. > >> rc.d scripts work well if you can live with the normal shutdown process >> invoking rc.d scripts. >> For my uses, I can live with shutdown and rc.d. > > so can i except in emergencies. > >> If you want to halt/reboot/poweroff, then you still have the original >> problem that Paul pointed out. > > reboot.c currently detects paging. is there a way to have it detect disk > i/o in general? > > note, 30 seconds isn't always enough. on a DCC server with 4GB mmap()'d, > i've run out of time. so, a sysctl for both the minimum (currently 5 > seconds) and maximum (currently 30 seconds) reboot i/o drain period > would be great. i would code these and submit them if these changes were > welcomed. > Yes, please do. best Neel > -- > Paul Vixie > ___ > freebsd-virtualization@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > To unsubscribe, send any mail to > "freebsd-virtualization-unsubscr...@freebsd.org" ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: can we get some interaction between halt/reboot and bhyve?
Craig Rodrigues wrote: > ... > > I haven't tried them, but Michael Dexter has written more comprehensive > rc.d scripts from bhyve: > > http://vmrc.bsd.lv/ these look good but i'm waiting for man pages, while meanwhile i bake my own toolset. > rc.d scripts work well if you can live with the normal shutdown process > invoking rc.d scripts. > For my uses, I can live with shutdown and rc.d. so can i except in emergencies. > If you want to halt/reboot/poweroff, then you still have the original > problem that Paul pointed out. reboot.c currently detects paging. is there a way to have it detect disk i/o in general? note, 30 seconds isn't always enough. on a DCC server with 4GB mmap()'d, i've run out of time. so, a sysctl for both the minimum (currently 5 seconds) and maximum (currently 30 seconds) reboot i/o drain period would be great. i would code these and submit them if these changes were welcomed. -- Paul Vixie ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: can we get some interaction between halt/reboot and bhyve?
On Sun, May 24, 2015 at 8:08 PM, John-Mark Gurney wrote: > > Shouldn't shutdown be used instead of reboot/halt? This allows > various rc.d scripts to run, and you can use that to make sure all > your bhyve instances are shutdown... > I wrote these rc.d scripts to start and stop bhyve VM's, so that on system shutdown, all bhyve instances started with these scripts are shutdown: https://github.com/freebsd/freebsd-ci/tree/master/vm/scripts/ I haven't tried them, but Michael Dexter has written more comprehensive rc.d scripts from bhyve: http://vmrc.bsd.lv/ rc.d scripts work well if you can live with the normal shutdown process invoking rc.d scripts. For my uses, I can live with shutdown and rc.d. If you want to halt/reboot/poweroff, then you still have the original problem that Paul pointed out. -- Craig ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: can we get some interaction between halt/reboot and bhyve?
On 5/25/15 2:01 PM, Allan Jude wrote: On 2015-05-25 00:32, Paul Vixie wrote: John-Mark Gurney wrote: Shouldn't shutdown be used instead of reboot/halt? This allows various rc.d scripts to run, and you can use that to make sure all your bhyve instances are shutdown... yes. but, if the reboot or poweroff or halt command is used, i'd like my bhyve's to have enough time to umount their ufs zvol's. the rc.d script should pass along some softer signal that can cause the bhyve to do its own clean shutdown. but where SIGTERM is used, which is passed on as ACPI power-off, i'd like busy (that is, not hung, trying hard to shut down cleanly) bhyve's to get more time. the current poweroff/halt/reboot command only looks for evidence of paging as a reason to extend the time between SIGTERM and SIGKILL. i'd like to give it some bhyve-relevant additional reason to delay that SIGKILL. One option might be a sysctl to control how much time processes are given between the SIGTERM and SIGKILL. Right now, it looks like, unless paging it detected, processes are only given 5 seconds, which is definitely not enough for bhyve and some other processes. A tunable gives people the option of how long they want to wait. On my laptop, I likely want to shutdown in a hurry, but on my virtualization server, I'm willing to wait a few minutes to keep the shutdown clean. you only want more tme if the processes in quetion are likely to respond. I'd suggest that we have a way for a process to set some sort of ' Yeah I got the signal, and I'd like some time to work on it" response. Since the original is a signal there is no back channel but maybe we can think of a suitable method. sysctls, process flags, ... timers? Maybe a per-process timer. You set it to a time by which you promise to be finished and the kernel promises to shoot you in the head if you were lying. Then the existence of the timer inhibits further signals and kicks. the timer could have a maximum setting that is set by sysctl.. so effectively an example might be. set max time to 45 seconds (or whatever) kick (HUP/TERM) all processes processes set their suicide timers to 20 secs and start shutting down. (sets process flag) after 5 seconds rekicks (-9) all processes without said timer running. (process flag?) waits for count of timers to go to 0 (a sysctl?.. decrements when process exits or it fires.) kill -9 all remaining processes.. wait 5 secs.. reboot ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: can we get some interaction between halt/reboot and bhyve?
On 2015-05-25 00:32, Paul Vixie wrote: > > > John-Mark Gurney wrote: >> Shouldn't shutdown be used instead of reboot/halt? This allows >> various rc.d scripts to run, and you can use that to make sure all >> your bhyve instances are shutdown... > > yes. but, if the reboot or poweroff or halt command is used, i'd like my > bhyve's to have enough time to umount their ufs zvol's. the rc.d script > should pass along some softer signal that can cause the bhyve to do its > own clean shutdown. but where SIGTERM is used, which is passed on as > ACPI power-off, i'd like busy (that is, not hung, trying hard to shut > down cleanly) bhyve's to get more time. the current poweroff/halt/reboot > command only looks for evidence of paging as a reason to extend the time > between SIGTERM and SIGKILL. i'd like to give it some bhyve-relevant > additional reason to delay that SIGKILL. > One option might be a sysctl to control how much time processes are given between the SIGTERM and SIGKILL. Right now, it looks like, unless paging it detected, processes are only given 5 seconds, which is definitely not enough for bhyve and some other processes. A tunable gives people the option of how long they want to wait. On my laptop, I likely want to shutdown in a hurry, but on my virtualization server, I'm willing to wait a few minutes to keep the shutdown clean. -- Allan Jude signature.asc Description: OpenPGP digital signature
Re: can we get some interaction between halt/reboot and bhyve?
John-Mark Gurney wrote: > Shouldn't shutdown be used instead of reboot/halt? This allows > various rc.d scripts to run, and you can use that to make sure all > your bhyve instances are shutdown... yes. but, if the reboot or poweroff or halt command is used, i'd like my bhyve's to have enough time to umount their ufs zvol's. the rc.d script should pass along some softer signal that can cause the bhyve to do its own clean shutdown. but where SIGTERM is used, which is passed on as ACPI power-off, i'd like busy (that is, not hung, trying hard to shut down cleanly) bhyve's to get more time. the current poweroff/halt/reboot command only looks for evidence of paging as a reason to extend the time between SIGTERM and SIGKILL. i'd like to give it some bhyve-relevant additional reason to delay that SIGKILL. -- Paul Vixie ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: can we get some interaction between halt/reboot and bhyve?
Paul Vixie wrote this message on Sun, May 24, 2015 at 18:05 -0700: > bhyve interprets SIGTERM as an ACPI poweroff, and usually the VM will > shutdown very quickly. but it may not happen in five seconds, which is > all halt(8) (and reboot(8)) promises. the code for reboot and halt > (/usr/src/sbin/reboot/reboot.c) is shown below. what's the right way to > change this so that if there are 50 bhyve processes running and they are > ACPI powerdowning as fast as they can, then the halt or reboot of the > bare metal will wait for them? Shouldn't shutdown be used instead of reboot/halt? This allows various rc.d scripts to run, and you can use that to make sure all your bhyve instances are shutdown... -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not." ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
can we get some interaction between halt/reboot and bhyve?
bhyve interprets SIGTERM as an ACPI poweroff, and usually the VM will
shutdown very quickly. but it may not happen in five seconds, which is
all halt(8) (and reboot(8)) promises. the code for reboot and halt
(/usr/src/sbin/reboot/reboot.c) is shown below. what's the right way to
change this so that if there are 50 bhyve processes running and they are
ACPI powerdowning as fast as they can, then the halt or reboot of the
bare metal will wait for them?
/* Send a SIGTERM first, a chance to save the buffers. */
if (kill(-1, SIGTERM) == -1 && errno != ESRCH)
err(1, "SIGTERM processes");
/*
* After the processes receive the signal, start the rest of the
* buffers on their way. Wait 5 seconds between the SIGTERM and
* the SIGKILL to give everybody a chance. If there is a lot of
* paging activity then wait longer, up to a maximum of approx
* 60 seconds.
*/
sleep(2);
for (i = 0; i < 20; i++) {
pageins = get_pageins();
if (!nflag)
sync();
sleep(3);
if (get_pageins() == pageins)
break;
}
for (i = 1;; ++i) {
if (kill(-1, SIGKILL) == -1) {
if (errno == ESRCH)
break;
goto restart;
}
if (i > 5) {
(void)fprintf(stderr,
"WARNING: some process(es) wouldn't die\n");
break;
}
(void)sleep(2 * i);
}
--
Paul Vixie
___
freebsd-virtualization@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to
"freebsd-virtualization-unsubscr...@freebsd.org"
