On 8/31/25 02:10, Jonathan Vasquez wrote:
Hey all,
I've updated my blog post with more information and some useful
screenshots:
https://xyinn.org/blog/freebsd/freebsd_bhyve_gpu_passthrough_amd
The important stuff to mention is that I've also gotten a chance to
play with vm-bhyve, but for the purposes of making sure we get as
clear information as possible without having to unwrap the vm-bhyve
scripts, I'll continue to use bhyve directly. With that said, some
useful info:
* The vBIOS is not required when running on Windows as mentioned by
Corvin here:
https://lists.freebsd.org/archives/freebsd-virtualization/2022-June/000797.html,
thus I've stopped using it. I was however also able to
successfully use his "echo 1 ..." recommendation to dump the bios
from within Linux (Fedora 42). I actually failed to extract the
bios before (without using amdvbflash) because most likely my
Linux Mint 22.1's Kernel was too old to extract the ROM. When I
tried to do the same exact thing today with Fedora 42, it worked,
and I was able to see the "rom" file from Fedora 42, but not from
Linux Mint 22.1.
* I was able to get the GPU to load without a Code 43 error. But
it's still unusable. I was actually able to get to this state
before but it was inconsistent. Through more research I was able
to get it to be reproducible. There are a few things to mention here:
o The card will always start in a Code 43 failed state
regardless of what I mention below (including if I reboot the
host).
o If you use RDP, the card will always be in a Code 43 state.
o If I use VNC, the card will start in a failed state (as
mentioned above), HOWEVER, you will be able to go into the
Device Manager, Disable the card, and then Enable the card,
and it will be enabled successfully as you can see in my
screenshots. The OS will also give it resources from what we
can see in the Device's Properties' Resources Tab.
Given what I mentioned above, I would recommend (as some other people
recommended in other links I've found), to disable the card in the VM
before you shut it down, and then enable it from inside of Windows
when you turn it back on. Although I don't know how necessary this is
given that if I use VNC, even if it starts off in a failed state
(let's say from a fresh host reboot), doing the "device re-nabling
trick" will let it load successfully again. At this point since I've
gotten the card to say it worked, I'm not sure how to tell Windows to
use that as the main GPU when I run my apps. I've looked at some
examples online of using display settings to change the preferred
card, but I don't see it listed there. Am I suppose to see the card in
dxdiag if I'm using VNC? I'm just running the bhyve VM on my server,
and I VNC over from my laptop on the same LAN network. I'm guessing if
things are working correctly I should be able to do that right? I know
there are also some benchmarking apps that show you what GPUs its
detected and maybe allow you to pick one to run a benchmark on. I
tried opening FurMark but it fails saying that it only detects Open GL
1.1 and all of that... I actually tried installing Mesa for windows to
see what happens and that does help but it's slow and it's clearly not
using the card and most likely doing some sort of software rendering
(maybe like a Windows llvmpipe).
Jonathan Vasquez
PGP: 34DA 858C 1447 509E C77A D49F FB85 90B7 C4CA 5279
Sent with ProtonMail Secure Email
On Monday, August 25th, 2025 at 14:51, Jonathan Vasquez
<[email protected]> wrote:
No go for my testing inside a Linux Mint 22.1 VM. Fully updated, and
passing in both 3/0/0 and 3/0/1 for my AMD Radeon card. I was able to
extract some of the "dmesg" and "eglinfo" output. Running "glxinfo"
also showed that the OpenGL renderer was set to llvmpipe, so software
rendering :(. "lspci -k" detected it and showed that it was using the
"amdgpu" kernel driver. Since I'm using vm-bhyve atm I didn't test
this with the dumped rom, just directly passing them through. I've
uploaded some of the logs here for now in case anyone wants to take a
glance at them:
https://xyinn.org/freebsd/files/gpu_pass/2025-08-25/
Jonathan Vasquez
PGP: 34DA 858C 1447 509E C77A D49F FB85 90B7 C4CA 5279
Sent with ProtonMail Secure Email
On Monday, August 25th, 2025 at 14:37, Robert Cocker
<[email protected]> wrote:
Hey group has bhvye core dev baked out an update for this
Robert
------------------------------------------------------------------------
*From:* [email protected]
<[email protected]> on behalf of Jonathan
Vasquez <[email protected]>
*Sent:* Monday, August 25, 2025 11:35:31 AM
*To:* Jonathan Vasquez <[email protected]>
*Cc:* Stephan Althaus <[email protected]>;
[email protected] <[email protected]>
*Subject:* Re: GPU Passthrough on FreeBSD 14.3 (AMD Radeon RX 6900
XT and Windows 10 Pro)
I was able to get a fresh Win 10 Pro VM running through vm-bhyve but
no progress. Still same Code 43 error as before. I wasn't able to
figure out how to pass the |rom=bios.rom| through vm-bhyve, I
didn't see anything about it in the config.sample file and I also
tried to bypass that by adding it in bhyve_options but no go, the VM
wouldn't even start if I did that. I see there is a "bhyve_devices"
that contains all of the slot devices but I don't see that as an
adjustable variable based on the config.sample info. I've pasted my
current vm-bhyve template below. My next test is to try the same
thing inside a Linux guest to see what happens.
----------------
loader="uefi"
cpu="4"
cpu_sockets="1"
cpu_cores="4"
cpu_threads="1"
memory="16G"
wired_memory="yes"
utctime="no"
ahci_device_limit="8"
disk0_type="nvme"
disk0_name="disk0.img"
network0_type="e1000"
network0_switch="public"
passthru0="3/0/0=2:0"
passthru1="3/0/1=2:1"
#bhyve_options="-s 2:0,passthru,3/0/0,rom=bios.rom"
# Using RDP post-install.
graphics="no"
graphics_port="5900"
graphics_listen="0.0.0.0"
graphics_res="1920x1080"
graphics_wait="yes"
xhci_mouse="yes"
uuid="535d54b5-81cb-11f0-93a2-98b78501bda8"
network0_mac="58:9c:fc:04:5a:eb"
----------------
Jonathan Vasquez
PGP: 34DA 858C 1447 509E C77A D49F FB85 90B7 C4CA 5279
Sent with ProtonMail Secure Email
On Monday, August 25th, 2025 at 11:05, Jonathan Vasquez
<[email protected]> wrote:
Unfortunately I don't have an intel machine with an available PCIe
slot. However I'm trying another thing which is to passthrough the
card to a Linux guest. I need to run more experiments but a
preliminary boot of a Linux Mint ISO did show the AMD Radeon RX
6900 XT card in |lspci| and showed that it was using the |amdgpu|
driver. I then did a basic |glxgears| and it opened up in my VNC
window and showed the FPS counter at over 2000 fps. The only issue
I noticed was when shutting down the live CD environment there were
|amdgpu| failure messages where it was failing to write something
to it. I'm currently redoing my "basic bhyve script" and moving it
over to "vm-bhyve" since it seems a lot of people are using that
instead of the more manual approach. This should hopefully help
simplify things for debugging. If the card works on Linux via
passthrough this would also be useful info and can allow me to see
the performance of Cyperpunk 2077, and some other games (even if
I'm not getting 100% of the card's performance).
Jonathan Vasquez
PGP: 34DA 858C 1447 509E C77A D49F FB85 90B7 C4CA 5279
Sent with ProtonMail Secure Email
On Monday, August 25th, 2025 at 11:00, Stephan Althaus
<[email protected]> wrote:
On 8/25/25 16:56, Jonathan Vasquez wrote:
Hey Stephan,
Yup, I've tried passing both 3/0/0 and 3/0/1 and no go, and I
also make sure to reboot the host in between runs due to the amd
hardware issue. I don't believe 3/0/1 is necessary though based
on the examples by Corvin in his talk. He only passed in 3/0/0
(I'm guessing his card also has a 3/0/1 function that he isn't
using since that's the audio channel).
Jonathan Vasquez
PGP: 34DA 858C 1447 509E C77A D49F FB85 90B7 C4CA 5279
Sent with ProtonMail Secure Email
On Monday, August 25th, 2025 at 10:52, Stephan Althaus
<[email protected]>
<mailto:[email protected]> wrote:
On 8/25/25 16:47, Stephan Althaus wrote:
On 8/24/25 23:58, Jonathan Vasquez wrote:
Hey all,
I've been experimenting with trying to get my server's
dedicated AMD Radeon RX 6900 XT working on a bhyve Windows 10
Pro client for gaming purposes. I've been reading a bunch of
blog posts, sites, files, and videos (thanks Corvin) to try
and piece things together. I've been able to get the VM to
recognize the card, but still stuck on the Code 43 issue (with
or without using the vBIOS I've extracted from my dedicated
card and also ensuring any options needed to be on in the UEFI
settings are enabled). This machine is running on an AMD Ryzen
9 7950X 16-Core Processor, so it's a full AMD setup, no Intel
stuff. It technically should be relatively "easy" to get AMD
working from what I've read (excluding the known AMD Hardware
reset issue). I've documented everything I've done so far, and
will be documenting more and simplifying the instructions as I
make progress. Please let me know if you see any thing weird
or anything I could try based on what I've rewritten. I'll try
to give it a go and provide feedback. Thank you!
https://xyinn.org/blog/freebsd/freebsd_bhyve_gpu_passthrough_amd
<https://xyinn.org/blog/freebsd/freebsd_bhyve_gpu_passthrough_amd>
Jonathan Vasquez
PGP: 34DA 858C 1447 509E C77A D49F FB85 90B7 C4CA 5279
Sent with ProtonMail Secure Email
Hi!
i have a AMD RX6400 GPU setup running on 14.2 with win10 guest,
host is a intel system.
i think you have to passthrough the 3/0/1 device as well!
The bios.rom is extracted from this card in place ?
Regards,
Stephan
Ah, and be shure to reboot the whole setup, inclusive host, for
every new test-case,
at least as long at is not guaranteed that we can live without..
Regars,
Stephan
Hi!
My only next assumption would be that this only does not work on
AMD host.
You don't have an intel system to test, you would havedone it, if
so.. (?)
Stephan
Hello!
I have a successful GPU passthrough of an AMD RX6400 to Win10 on an
Intel host.
Just want to share some Infos
+ no problem with RDP here, that's my use case
+ i used GPU-Z within the vm to check if the gpu is actually useable, it
shows GPU frequency > 0 , and OpenGL >1.1 when it is.
+ i have thr ROM in my bhyve start script, the vm has no problem with
it. i don't use vm-bhyve so far.
- your win 10 vm is EFI install ?
+ yes, mesa is software rendering, sort of 'software GPU'
It seems to me that the only noticeable difference between our systems
is that mine is an intel host..
Regards,
Stephan
