Hey all, I hope everyone is doing well. I wanted to give some final updates
since I feel I'm basically done with my research, set up, and testing. I'm
pleased to say that I'm officially calling my setup as "stable". For the past
week, I've been testing the VM by either playing some games on it for a few
hours, or leaving the VM on idle, and so far I haven't really experienced any
VM crashes. The games seem to be working smoothly when I'm playing them, and I
don't get any sudden VM crashes. I left the VM on idle for the past 1d20h and
no issues. I wanted to finish working on my rc.d start up/shutdown script so I
had to bring the VM down to do that :P. There were some instances where the VM
wouldn't really display any output and would effectively be frozen when I
rebooted the machine multiple consecutive times during my testing, so much so
that doing a "service vm_gaming restart" (or stop) would just leave the script
at the "Waiting for PIDs" message. In this case I had to do a "bhyvectl
--vm=gaming --force-poweroff" to kill it. The subsequent start of the VM then
seemed to work correctly and video was working again. This doesn't happen all
the time but it could happen so I wanted to mention that for the record. Once
the VM is up, it's pretty stable. I do see the windows start up screen always
say "Getting devices ready" and then it proceeds to complete the boot with no
issues. Not sure what that's about but there are no negative consequences.
In regards to my research regarding the graceful shutdown, given that
throughout all of many days of testing, my VM hasn't gotten any disk
corruption, I believe that the KEYWORD: shutdown (or KEYWORD: nojail shutdown)
have been working correctly. When doing my testing, I do see that if I do a
"service vm_gaming stop" explicitly, I see Windows 10 showing the light blue
screen with the "Shutting down" message. When I do "shutdown -r now" I
sometimes see the same light blue screen, but usually I don't see the screen,
what I do see though is that if I move my mouse cursor to the middle of the
screen, once I trigger the shutdown, I would see the light blue circle/loop
around the mouse cursor start to spin. This happens every single time I run the
"shutdown -r now" command. So this means that the shutdown signal is properly
being sent, and my theory is that the GPU/OS is just not reflecting that screen
update due to the speed at which things are happening. I also do see that the
time it takes to reboot after the shutdown -r now is called takes a little bit
of time, which I believe is also partly the time it takes for the system to
wait for the bhyve vm to shutdown and for resource to possibly be cleaned
(bhyvectl ... destroy). So I believe I'm done with the main research,
experimentation, and testing. I've updated my blog post as well, and below you
can find my updated rc.d script. I saw Corvin recently pushed some updates to
CURRENT to get NVIDIA passthrough working (PIN related work). I do have my
Razer Blade 15" (2020) that has an Intel proc and NVIDIA GTX 2060. In the
future I may try and install FreeBSD on there to test the Intel/NVIDIA side of
things, but that's not something I'm planning on doing any time soon.
Take care all and I hope all of this helps y'all have some fun and relax. Stay
safe, Jonathan!
---- updated script
#!/bin/sh
# PROVIDE: vm_gaming
# REQUIRE: LOGIN FILESYSTEMS
# KEYWORD: shutdown
. /etc/rc.subr
name="vm_gaming"
rcvar="vm_gaming_enable"
start_cmd="${name}_start"
stop_cmd="${name}_stop"
stop_postcmd="${name}_poststop"
restart_cmd="${name}_restart"
status_cmd="${name}_status"
pidfile="/var/run/${name}.pid"
vm_path="/atlantis/vms/gaming"
vm_name="gaming"
load_rc_config $name
: ${vm_gaming_enable:="NO"}
vm_gaming_start()
{
daemon \
-o /var/log/${name}.log \
-p "${pidfile}" \
bhyve -AHPSw -c sockets=1,cores=16,threads=1 -m 32G \
-s 0,hostbridge \
-s 1,nvme,${vm_path}/disk0.img \
-s 3:0,passthru,3/0/0 \
-s 3:1,passthru,3/0/1 \
-s 13:0,passthru,13/0/0 \
-s 30,xhci,tablet \
-s 31,lpc \
-l
bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd,fwcfg=qemu \
-o console=stdio \
${vm_name}
}
vm_gaming_stop()
{
if [ ! -f "${pidfile}" ]; then
echo "the gaming vm isn't running!"
return
fi
# Send SIGTERM twice to make sure Windows listens to
# the ACPI shutdown signal.
pid="$(cat ${pidfile})"
kill "${pid}"
kill "${pid}"
# Wait a bit for the guest to shutdown properly before
# we continue shutting down the host.
wait_for_pids "${pid}"
}
vm_gaming_poststop()
{
bhyvectl --vm="${vm_name}" --destroy
}
vm_gaming_restart()
{
# NOTE: AMD users will most likely experience the famous
# AMD Hardware Reset Bug. This means that after you reboot
# the guest, you most likely won't have video out. If this
# happens, you'll need to restart the host. Sometimes this
# command will work for AMD users though. So you can try a
# restart and see if it works.
vm_gaming_stop
vm_gaming_start
}
vm_gaming_status()
{
if [ -f "${pidfile}" ]; then
echo "$(cat ${pidfile})"
fi
}
run_rc_command "$1"
