RE: FreeBSD 12.1 i386 on Hyper-V Fails to Mount Root
Hi Harry, Can you test if this patch fixes the problems? https://reviews.freebsd.org/D30577 Thanks, Wei > -Original Message- > From: Harry Schmalzbauer > Sent: Friday, May 21, 2021 8:50 PM > To: Konstantin Belousov ; Wei Hu > Cc: freebsd-virtualization@freebsd.org > Subject: Re: FreeBSD 12.1 i386 on Hyper-V Fails to Mount Root > > Am 19.05.2021 um 13:26 schrieb Harry Schmalzbauer: > > ... > > I can just assure that I tracked it down to the single r332489 committ > > which makes vmbus(4) failing on i386. > > Verification done: > > - built/installed world&kernel on i386 HyV guest with r332488 > > reboot fine. > > - built/installed kernel on the same vm after svnlite up -r 332489 > > reboot fails with "vmbus0: cannot find free IDT vector" > > - loaded kernel.old/kernel, did revert 332489, by 'svnlite up -r > > 332488' and compiled/reinstalled kernel (again r332488) > > reboot fine. > > While having hard time catching up with x86 basics, I found another oddity: > On Hyper-V gen1, the maximum ammount of RAM I can assign a FreeBSD guest > beeing able to boot i386 is 3914MB, startring with r332489. > As soon as I configure the guest to have 3916MB or more, FreeBSD-i386 > r332489 booting leads to guest reset. Loader starts kernel, machine reboots. > Either reverting to r332488 or reducing guest-RAM solves this. > > The most interesting fact here is that I couldn't reproduce it on ESXi. > I ducplicated the Hyper-V HDD and started the exactly same code on > ESXi(v6) guest, where it doesn't crash! I can boot FreeBSD-i386 r332489 with > any ammount of guest-RAM on ESXi. > (speaking for GENERIC, non-PAE) > > Host CPU differs significantly though and I have no possibility to test this > influence. Ther HyperV is Dual-Socket E5-2630 while the ESXi host is a single > socket X3470 Xeon (pre SandyBridge). > > Maybe someone has an idea how this affects these symptoms. > > Thanks, > -harry > ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Windows 2022 Server Test 1
Looks like a different approch to WinServer . PLease let me know if you have succeeded. -- Member - Liberal International This is doctor@@nl2k.ab.ca Ici doctor@@nl2k.ab.ca Yahweh, Queen & country!Never Satan President Republic!Beware AntiChrist rising! Look at Psalms 14 and 53 on Atheism https://www.empire.kred/ROOTNK?t=94a1f39b People who show off their righteousness have none. -unknown ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: increasing bhyve VM_MAXCPU
On 18 May 2021, at 19:21, Rodney W. Grimes wrote: >> I'm experimenting with bhyve VMs with more than 16 vCPUs on FreeBSD >> I began working on a machine with only eight CPU cores/threads, so >> rather than trying to increase VM_MAXCPU, I tried to decrease it, which >> seemed to work fine. My procedure was: >> >>edit /usr/src/sys/amd64/include/vmm.h > > cp /usr/src/sys/amd64/include/vmm.h /usr/include/machine/vmm.h Thanks so much for this. Things are now working fine, e.g.: [root@fbsd1] # grep 'Multiprocessor System Detected' /var/run/dmesg.boot FreeBSD/SMP: Multiprocessor System Detected: 24 CPUs ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
increasing bhyve VM_MAXCPU
I'm experimenting with bhyve VMs with more than 16 vCPUs on FreeBSD 12.2-RELEASE. As I understand it, this limit is defined as VM_MAXCPU in /usr/src/sys/amd64/include/vmm.h. I also understand that as of about two years ago, it should be possible to increase it beyond 21[1]. I began working on a machine with only eight CPU cores/threads, so rather than trying to increase VM_MAXCPU, I tried to decrease it, which seemed to work fine. My procedure was: edit /usr/src/sys/amd64/include/vmm.h cd /usr/src/lib/libvmmapi ; make clean ; make ; make install cd /usr/src/usr.sbin/bhyve ; make clean ; make ; make install cd /usr/src/usr.sbin/bhyvectl ; make clean ; make ; make install cd /usr/src/usr.sbin/bhyveload ; make clean ; make ; make install cd /usr/src ; make buildkernel KERNCONF=JLD ; make installkernel KERNCONF=JLD reboot where JLD is a kernel config file that differs from GENERIC only in the "ident" line. I tried several values (1, 2, and 4) for VM_MAXCPU and verified that VMs with a number of vCPUs less than or equal to the max booted fine and that VMs with more didn't. So then I tried increasing VM_MAXCPU on a machine with two CPUs, 14 cores/28 threads each, using the same procedure. I set VM_MAXCPU to 28 and configured a VM with 24 vCPUs but that didn't boot. Also tried setting the max to 20 and booting a VM with 18 vCPUs but that didn't work either. In all cases, VMs with 16 or fewer vCPUs continued to boot fine. All the tests were done with VMs created before doing any of this stuff. I am using vm-bhyve to manage VMs. The VMs in these tests were running one of FreeBSD 12.2-RELEASE, alpine linux 3.13, and debian-testing. So it seems like I'm overlooking something but I don't know what, or maybe this just isn't going to work. Anybody have experience actually doing this? Thanks very much for any advice or further information. [1] <https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=212782> ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
RE: FreeBSD 12.1 i386 on Hyper-V Fails to Mount Root
> -Original Message- > From: Konstantin Belousov > Sent: Monday, May 17, 2021 1:47 PM > To: Wei Hu > Cc: Harry Schmalzbauer ; Chris Knight > ; freebsd-virtualization@freebsd.org > Subject: Re: FreeBSD 12.1 i386 on Hyper-V Fails to Mount Root > > On Mon, May 17, 2021 at 04:12:20AM +, Wei Hu wrote: > > Copy Kib who was the author of PTI and other relate patches. > > > > This looks to be caused by Meltdown and Spectre mitigation patches > > which were checked into head couple years ago. See this link for details > > about > these patches: > > https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwiki > > .freebsd.org%2FSpeculativeExecutionVulnerabilities&data=04%7C01%7C > > > weh%40microsoft.com%7Cd7b33cd5fc774c88310008d918f744c4%7C72f988bf > 86f14 > > > 1af91ab2d7cd011db47%7C1%7C0%7C637568272592647219%7CUnknown%7 > CTWFpbGZsb > > > 3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0% > 3D% > > > 7C1000&sdata=5HjRc67IiHlrpR5bgWwJq9opMwBKudlHJpElsDZkS4A%3D& > amp;re > > served=0 > > > > The amd64 patches also caused problems on Hyper-V (triple fault) when > > vmbus and other HyperV drivers (netvsc, storvsc etc) were loaded > > separately. The workaround for this on amd64 is either build these > > Hyper-V modules into the kernel or completely disable mitigation by > > setting vm.pmap.pti="0" in /boot/loader.conf in the affect FreeBSD guests. > > > For amd64 I posted the patch to supposedly fix the issue, you did not > responded. > I am sorry I might have missed your review request. Would you please let me know the link to the patch or the commit if you already have checked in? I will need to test and inform the parties who got affected. For the issue on i386, any plan to fix it. Thanks, Wei ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
RE: FreeBSD 12.1 i386 on Hyper-V Fails to Mount Root
Copy Kib who was the author of PTI and other relate patches.
This looks to be caused by Meltdown and Spectre mitigation patches which were
checked into head couple years ago. See this link for details about these
patches:
https://wiki.freebsd.org/SpeculativeExecutionVulnerabilities
The amd64 patches also caused problems
on Hyper-V (triple fault) when vmbus and other HyperV drivers (netvsc, storvsc
etc)
were loaded separately. The workaround for this on amd64 is either build these
Hyper-V modules into the kernel or completely disable mitigation by
setting vm.pmap.pti="0" in /boot/loader.conf in the affect FreeBSD guests.
It looks the problem on i386 guests is different. While multiple patches have
been
checked in for amd64, there was only one patch r332489 for i386. I am not sure
whether there is similar vm.pmap.pti option to disable the patch for i386 or
not.
I am not familiar to these patches.
@ Kib, will you be able to comment on this?
Thanks,
Wei
> -Original Message-
> From: Harry Schmalzbauer
> Sent: Sunday, May 16, 2021 2:55 AM
> To: Wei Hu ; Chris Knight ; freebsd-
> virtualizat...@freebsd.org
> Subject: Re: FreeBSD 12.1 i386 on Hyper-V Fails to Mount Root
>
> Am 14.05.2021 um 18:44 schrieb Harry Schmalzbauer:
> > Am 16.10.2020 um 11:35 schrieb Wei Hu via freebsd-virtualization:
> >>> :
> >>>
> >> Did you try 12.0? Maybe something changed in 12.x causes failure.
> >> I386 is not well tested on Hyper-V. I will try to reproduce and take
> >> a look.
> >
> > The regression happened before 12.0.
> > Today I stumbled across this problem too and tried 11.4 (/May 8th,
> > 2020) /sucessfully, while 12.0 (/Oct 19th, 2018)/ is failing with i386
> > releases.
> >
> > One commit which is in 12 but not in 11.4 is r332489.
> >
> :
> :
> :
> > Unfortunately beyond my skills.
>
> In sys/x86/x86/local_apic.c, I added this debug printf:
> for (idx = IPI_DYN_FIRST; idx <= IPI_DYN_LAST; idx++) {
> printf("DEBUG: native_lapic_ipi_alloc(), idx = %i\n", idx);
> ip = &idt[idx];
> func = (ip->gd_hioffset << 16) | ip->gd_looffset;
> if ((!pti && func == (uintptr_t)&IDTVEC(rsvd)) ||
> (pti && func == (uintptr_t)&IDTVEC(rsvd_pti))) {
> vector = idx;
> setidt(vector, ipifunc, SDT_APIC, SEL_KPL,
> GSEL_APIC);
> break;
> }
> }
>
> This is my latest attempt to see if at least the origin of my fault tracing
> matches.
> It does.
>
> From #define IPI_DYN_FIRST (APIC_IPI_INTS + 9) in sys/x86/include/apicvar.h
> I expected to see 252, 253 and 254, which I sucessfully verified.
>
> Now comes the part I cannot read: (uintptr_t)&IDTVEC(rsvd)
> IDTVEC(rsvd) seems to be a assembler makro from sys/i386/i386/exception.s
>
> But now it doesn't make any sense for me to dig further, I simply do not
> understand it.
>
> Unfortunately having broken i386 support on Hyper-V is not a cosmetic issue.
> Removing "device hyperv" from kernel gives access to the v-hdd via legacy
> IDE/ada, but then I have no NIC.
> Choosing between missing HDD and missing network support is hard
> nowadays ;-)
>
> Any help getting i386 up and running back on Hyper-V highly appreciated! 11.4
> is the latest working version, all i386-releases starting with 12.0 don't
> work!
>
> Thanks,
> -harry
___
freebsd-virtualization@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to
"freebsd-virtualization-unsubscr...@freebsd.org"
Re: bhyve current windows status
Thanks for the insight. It’s a shame that MS doesn’t bundle the RedHat drivers in the installation image. On Mon, Apr 12, 2021 at 00:54, Jason Tubnor wrote: > On Sun, 11 Apr 2021 at 20:07, Robert Crowston wrote: > >> nvme is faster than virtio-blk? It seems strange that a paravirtualized >> driver would be slower. Is that because of the regression you mention? > > Not saying it is slower, just not any faster than the NVMe presentation that > we have discovered in production. NVMe is the general direction storage is > taking, doesn't require the drivers to be injected at installation time and > lower usability friction. > > The regression was due to a bug in the virtio-stor code that would nuke your > storage as soon as the driver loaded after upgrading it the following boot. > That was discussed on a bhyve call last year. It is no longer an issue with > the latest driver if you wish to continue to use virtio-stor. > > Cheers, > > Jason. ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: bhyve current windows status
nvme is faster than virtio-blk? It seems strange that a paravirtualized driver would be slower. Is that because of the regression you mention? — RHC. ‐‐‐ Original Message ‐‐‐ On Sunday, 11 April 2021 04:49, Jason Tubnor wrote: > Hi Matt, > > Further to Peter's input below, I have added what we have in production for > Windows Server 2016/2019 > > On Fri, 9 Apr 2021 at 16:30, Peter Grehan gre...@freebsd.org wrote: > > > > What are the current recommended devices/options for Windows (2019 > > > server in my case) - especially with ZFS. Should I be specifying a > > > 512/4096 sector/block size via bhyve and/or zfs? I assume nvme & > > > virtio-net are the current best options but is there a preferred virtio > > > driver version. Are any of the other virtio drivers of any use to be > > > installed or just the network drivers? > > > > nvme - yes. > > If using 12.2 or greater, NVMe across the board for guests. We will be > switching over once we bring the fleet of appliances up to 13.0 upon > release. > > If you are using 11.4, virtio-stor is your only option if you are after > performance. While you can use ahci-hd, this is shockingly slow. About 2 > versions ago of the VirtIO stack users of the virtio-stor drivers saw a > regression in the driver take out whole virtual storage devices. Running > the latest one as at 11 April 2021 should be fine for you. > > > I'll leave the sector/block size issues to others. I don't touch any > > of those params but don't use enough Windows apps to make a qualified call. > > We set volblocksize=4k for all guests unless the guest is running MSSQL, in > which case, volblocksize=512. We have observed significant storage > consumption when using this smaller block size, likely due to the checksum > overhead for small amounts of committed data. > > No need for other virtio drivers. For virtio-net, the recommendation > > > is to use the latest one. > > > > > Are there any known problems with applications like AD/Exchange? I know > > > that SQL 2012 had massive storage overhead issues on ZFS due to 512 byte > > > writes, but I'm not sure if that still affects newer versions or other > > > applications? > > > > As above, I'll leave it up to others to chime in here. > > Yes, that still applies. It is clear that you have discovered what we have > (as also what I typed above for others to reference). I don't believe that > has been fixed by Microsoft yet. I may get around to testing against newer > versions over the next couple of months. > > Cheers, > > Jason. > > freebsd-virtualization@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > To unsubscribe, send any mail to > "freebsd-virtualization-unsubscr...@freebsd.org" ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: FreeBSD13/12 oddity
On Sat, Apr 03, 2021 at 11:32:15AM +1000, Peter Grehan wrote: > > I was testing AlmaLinux on bhyve. > > works on FreeBSD 13> > but is crashes > > on 12 upon install. > I tried on CURRENT and it also works (with UEFI). > > Were you using UEFI on 12 ? Same options as on 13 ? > AFAIK yes, b ut I could be incorrect. > later, > > Peter. -- Member - Liberal International This is doctor@@nl2k.ab.ca Ici doctor@@nl2k.ab.ca Yahweh, Queen & country!Never Satan President Republic!Beware AntiChrist rising! Look at Psalms 14 and 53 on Atheism https://www.empire.kred/ROOTNK?t=94a1f39b If they can dehumanize the most innocent, they can dehumanize anyone. -unknown ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
FreeBSD13/12 oddity
I was testing AlmaLinux on bhyve. works on FreeBSD 13 but is crashes on 12 upon install. Huh? -- Member - Liberal International This is doctor@@nl2k.ab.ca Ici doctor@@nl2k.ab.ca Yahweh, Queen & country!Never Satan President Republic!Beware AntiChrist rising! Look at Psalms 14 and 53 on Atheism https://www.empire.kred/ROOTNK?t=94a1f39b If they can dehumanize the most innocent, they can dehumanize anyone. -unknown ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
RE: FreeBSD on Hyper-V network driver negotiation
Hi Alex, “the hn0 interface is set to 10GbaseT/ full-duplex” à this should not matter, because the “hn0” interface in a FreeBSD VM on Hyper-V is a virtual NIC which doesn’t really have the concept of “1000baseT”, “10GbaseT”, .etc. I think we just happen to report 10GbaseT in the “hn” driver, but actually we can report anything -- probably we should change the driver to report 100Gbps, just in case people think the vNIC is limited to 10Gbps. :-) If the underlying physical NIC is a 100Gbps NIC, then theoretically the vNIC should be able to reach a speed near to 100Gbps. It would good if you can share your ‘dmesg’, ‘ifconfig -a’, “arp -n -a”, and “netstat -r”, and when you try to ping your default gateway, please also run “tcpdump -i hn0” and check if something is wrong (you can add “-w tcpdump.cap” to save the caputred packets into a file and share it if necessary). Can you try the vNIC with a standard FreeBSD VM, e.g. you can try the pre-generated BSD image here: https://download.freebsd.org/ftp/releases/VM-IMAGES/12.2-RELEASE/amd64/Latest/ https://download.freebsd.org/ftp/releases/VM-IMAGES/12.2-RELEASE/amd64/Latest/FreeBSD-12.2-RELEASE-amd64.vhd.xz Thanks, -- Dexuan From: Alex Leach Sent: Tuesday, March 23, 2021 2:13 PM To: freebsd-virtualization@freebsd.org Cc: Microsoft FreeBSD Team Subject: FreeBSD on Hyper-V network driver negotiation Hello, I've recently installed a virtualized instance of pfSense on Hyper-V and am having trouble with the network driver attached to an external switch. The physical network adapter is an Intel X550 10Gbps NIC, and it is physically connected to a 1Gbps port on a SonicWall router. There is no connectivity at all to the SonicWall router, I can't ping or route traffic through it at all, when the SonicWall is set as the default gateway. The problem as I see it, is that negotiation doesn't work. Under ifconfig, I can see the hn0 interface is set to 10GbaseT/ full-duplex, and no other media speeds are available. I cannot manually set the interface to 1000baseT, which is what I think should be automatically negotiated with the SonicWall. Maybe this is a bit of an edge case scenario, having a 10Gbps NIC connected to a 1Gbps switch, but I expected it to work out of the box. Please could you offer some advice as to how I can get this to work. Obviously if there is any logging or debugging information I can provide, please let me know. Thanks and kind regards, Alex PS. Apologies for not providing any specific error messages or output here, I thought I'd start with a summary overview of what I see as the issue. ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: Extra dependency of the uefi-edk2-bhyve-csm firmware port
On Wed, Mar 17, 2021 at 11:43:52AM +0700, Victor Sudakov wrote: > Dear Colleagues, > > Why do you think sysutils/uefi-edk2-bhyve-csm would have a runtime (!) > dependency on gcc48 ? > > PR https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=254257 > That is a bug! > -- > Victor Sudakov VAS4-RIPE > http://vas.tomsk.ru/ > 2:5005/49@fidonet -- Member - Liberal International This is doctor@@nl2k.ab.ca Ici doctor@@nl2k.ab.ca Yahweh, Queen & country!Never Satan President Republic!Beware AntiChrist rising! Look at Psalms 14 and 53 on Atheism https://www.empire.kred/ROOTNK?t=94a1f39b Light wants an end to war, but so does darkness. -unknown _______ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: vm-bhyve No disk(s) present to configure
Thanks to all who responded. For the record, the solution was to add this: disk0_type="virtio-blk" to my /vmm/fbsd1/fbsd1.conf file. I also added that to my /vmm/.templates/freebsd-zvol.conf. On 16 Mar 2021, at 8:01, John Doherty via freebsd-virtualization wrote: Hi, I am trying to get started with vm-bhyve and getting a little stuck. I am sure there are things I don't understand here so I hope you will bear with me. I am using a machine with an AMD FX-8370E eight-core CPU, running 12.2-RELEASE-p4. The machine boots from a ~60GB SATA SSD which is configured as the zroot zpool. I have run a few bhyve vms on this machine before, including a couple of linux variants, by just using the bhyve command directly and they worked fine. In those cases, I just used files created with truncate(1) for vm disks but I would like to use zvols. vm-bhyve looks pretty nice to me and I think I'd like using it also. I have this in /boot/loader.conf: vmm_load="YES" nmdm_load="YES" if_tap_load="YES" if_bridge_load="YES" and this in /etc/rc.conf: vm_enable="YES" vm_dir="zfs:zroot/vmm" vm_list="" vm_delay="5" And so I thus have this: [root@sand] # zfs list -r zroot/vmm NAMEUSED AVAIL REFER MOUNTPOINT zroot/vmm 1.43G 36.8G 1.43G /vmm and this: [root@sand] # vm datastore list NAMETYPEPATH ZFS DATASET default zfs /vmm zroot/vmm I have populated /vmm/.templates with the files from /usr/local/share/examples/vm-bhyve but made some changes to them. For example, the freebsd-zvol.conf template now looks like this: loader="bhyveload" cpu=1 memory=4096M network0_type="virtio-net" network0_switch="public" disk0_dev="sparse-zvol" disk0_name="disk0" I've also populated /vmm/.iso with some ISO images including FreeBSD-12.2-RELEASE-amd64-disc1.iso. So with that, I can for example do this: [root@sand] # vm create -t freebsd-zvol fbsd1 [root@sand] # vm list NAME DATASTORE LOADER CPU MEMORY VNC AUTOSTART STATE fbsd1 defaultbhyveload 14096M -No Stopped and zfs things then look like this: [root@sand] # zfs list -r zroot/vmm NAMEUSED AVAIL REFER MOUNTPOINT zroot/vmm 1.43G 36.8G 1.43G /vmm zroot/vmm/fbsd1 164K 36.8G 108K /vmm/fbsd1 zroot/vmm/fbsd1/disk056K 36.8G56K - So that all seems pretty good so far. Then I do this: vm install -f fbsd1 FreeBSD-12.2-RELEASE-amd64-disc1.iso The vm boots and I'm asked to choose a console type, I take the default vt100, and boom -- I get the familiar FreeBSD Installer screen. I proceed through that and try to do a Guided Root-on-ZFS installation, but when I choose Pool Type/Disks, I'm told "No disk(s) present to configure." So. Pretty clearly doing something wrong here but I'm not sure what. Happy to provide any further info that I might have neglected above. Any advice that points me in the right direction much appreciated. Thanks. _______ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org" _______ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
vm-bhyve No disk(s) present to configure
Hi, I am trying to get started with vm-bhyve and getting a little stuck.
I am sure there are things I don't understand here so I hope you will
bear with me.
I am using a machine with an AMD FX-8370E eight-core CPU, running
12.2-RELEASE-p4. The machine boots from a ~60GB SATA SSD which is
configured as the zroot zpool.
I have run a few bhyve vms on this machine before, including a couple of
linux variants, by just using the bhyve command directly and they worked
fine. In those cases, I just used files created with truncate(1) for vm
disks but I would like to use zvols. vm-bhyve looks pretty nice to me
and I think I'd like using it also.
I have this in /boot/loader.conf:
vmm_load="YES"
nmdm_load="YES"
if_tap_load="YES"
if_bridge_load="YES"
and this in /etc/rc.conf:
vm_enable="YES"
vm_dir="zfs:zroot/vmm"
vm_list=""
vm_delay="5"
And so I thus have this:
[root@sand] # zfs list -r zroot/vmm
NAMEUSED AVAIL REFER MOUNTPOINT
zroot/vmm 1.43G 36.8G 1.43G /vmm
and this:
[root@sand] # vm datastore list
NAMETYPEPATH ZFS DATASET
default zfs /vmm zroot/vmm
I have populated /vmm/.templates with the files from
/usr/local/share/examples/vm-bhyve but made some changes to them. For
example, the freebsd-zvol.conf template now looks like this:
loader="bhyveload"
cpu=1
memory=4096M
network0_type="virtio-net"
network0_switch="public"
disk0_dev="sparse-zvol"
disk0_name="disk0"
I've also populated /vmm/.iso with some ISO images including
FreeBSD-12.2-RELEASE-amd64-disc1.iso.
So with that, I can for example do this:
[root@sand] # vm create -t freebsd-zvol fbsd1
[root@sand] # vm list
NAME DATASTORE LOADER CPU MEMORY VNC AUTOSTART STATE
fbsd1 defaultbhyveload 14096M -No Stopped
and zfs things then look like this:
[root@sand] # zfs list -r zroot/vmm
NAMEUSED AVAIL REFER MOUNTPOINT
zroot/vmm 1.43G 36.8G 1.43G /vmm
zroot/vmm/fbsd1 164K 36.8G 108K /vmm/fbsd1
zroot/vmm/fbsd1/disk056K 36.8G56K -
So that all seems pretty good so far. Then I do this:
vm install -f fbsd1 FreeBSD-12.2-RELEASE-amd64-disc1.iso
The vm boots and I'm asked to choose a console type, I take the default
vt100, and boom -- I get the familiar FreeBSD Installer screen. I
proceed through that and try to do a Guided Root-on-ZFS installation,
but when I choose Pool Type/Disks, I'm told "No disk(s) present to
configure."
So. Pretty clearly doing something wrong here but I'm not sure what.
Happy to provide any further info that I might have neglected above. Any
advice that points me in the right direction much appreciated. Thanks.
_______
freebsd-virtualization@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to
"freebsd-virtualization-unsubscr...@freebsd.org"
Re: sysutils/uefi-edk2-bhyve isssue
On Tue, Feb 23, 2021 at 09:18:57PM -0700, Rebecca Cran wrote: > On 2/23/2021 5:58 PM, The Doctor wrote: > > Again, same issue. > > > > What is causing > > > > ld-elf.so.1: /usr/local/lib/compat/libstdc++.so.6: version CXXABI_1.3.8 > > required > > +by > > +/usr/ports/sysutils/uefi-edk2-bhyve/work/edk2-2e1e8c35f3178df95d79da81ac6deec24 > > +2da74c2/BaseTools/Source/C/bin/VfrCompile not found > > make[2]: *** [GNUmakefile:315: > > +/usr/ports/sysutils/uefi-edk2-bhyve/work/Build/BhyveX64/RELEASE_GCC5/X64/MdeMod > > +ulePkg/Library/BootMaintenanceManagerUiLib/BootMaintenanceManagerUiLib/DEBUG/Bo > > +otMaintenanceManager.c] Error 1 > > make[2]: Leaving directory > > +'/usr/ports/sysutils/uefi-edk2-bhyve/work/Build/BhyveX64/RELEASE_GCC5/X64/MdeMo > > +dulePkg/Library/BootMaintenanceManagerUiLib/BootMaintenanceManagerUiLib' > > > > To take place? > > I'm fairly sure it means you have _some_ sort of toolchain > installation/configuration that's messing things up. > > Could you run 'make clean' in the uefi-edk2-bhyve directory - then, run > 'script bhyve_build_log.txt make' to log all the output to the file > 'bhyve_build_log.txt'? > > And then upload the contents somewhere, for example https://pastebin.com/ . > Will do after I I implement FreeBSD-12.2-p4 > > -- > Rebecca Cran > > -- Member - Liberal International This is doctor@@nl2k.ab.ca Ici doctor@@nl2k.ab.ca Yahweh, Queen & country!Never Satan President Republic!Beware AntiChrist rising! Look at Psalms 14 and 53 on Atheism https://www.empire.kred/ROOTNK?t=94a1f39b NFLD on 13 Feb vote Liberal ! ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: sysutils/uefi-edk2-bhyve isssue
On Tue, Feb 23, 2021 at 08:31:18AM -0700, Rebecca Cran wrote: > On 2/23/21 7:49 AM, The Doctor wrote: > > On Mon, Feb 22, 2021 at 11:30:24PM -0700, Rebecca Cran wrote: > > >> Do you have anything in /etc/make.conf that could be affecting the build? > > > > DEFAULT_VERSIONS+=linux=c7_64 > > DEFAULT_VERSIONS+=ruby=2.7 > > DEFAULT_VERSIONS+=python=3.8 > > DEFAULT_VERSIONS+=pytho3=3.8 > > DEFAULT_VERSIONS+=bdb=5.3 > > DEFAULT_VERSIONS+=lua=5.3 > > DEFAULT_VERSIONS+=perl5=5.30 > > DEFAULT_VERSIONS+=ssl=openssl > > DEFAULT_VERSIONS+=gcc=10 > > HAVE_MOTIF=yes > > CC=/usr/local/bin/clang11 > > C++=/usr/local/bin/clang++11 > > CXX=/usr/local/bin/clang++11 > > CPP=/usr/local/bin/clang-cpp11 > > MAKE_JOBS_NUMBER=1 > > SUEXEC_DOCROOT=/usr/home > > SUEXEC_USERDIR=html > > SUEXEC_GID_MIN=100 > > SUEXEC_UID_MIN=100 > > > > Could you try removing or commenting out the CC, C++, CXX and CPP lines > please? > > -- > Rebecca Cran > Again, same issue. What is causing ld-elf.so.1: /usr/local/lib/compat/libstdc++.so.6: version CXXABI_1.3.8 required +by +/usr/ports/sysutils/uefi-edk2-bhyve/work/edk2-2e1e8c35f3178df95d79da81ac6deec24 +2da74c2/BaseTools/Source/C/bin/VfrCompile not found make[2]: *** [GNUmakefile:315: +/usr/ports/sysutils/uefi-edk2-bhyve/work/Build/BhyveX64/RELEASE_GCC5/X64/MdeMod +ulePkg/Library/BootMaintenanceManagerUiLib/BootMaintenanceManagerUiLib/DEBUG/Bo +otMaintenanceManager.c] Error 1 make[2]: Leaving directory +'/usr/ports/sysutils/uefi-edk2-bhyve/work/Build/BhyveX64/RELEASE_GCC5/X64/MdeMo +dulePkg/Library/BootMaintenanceManagerUiLib/BootMaintenanceManagerUiLib' To take place? -- Member - Liberal International This is doctor@@nl2k.ab.ca Ici doctor@@nl2k.ab.ca Yahweh, Queen & country!Never Satan President Republic!Beware AntiChrist rising! Look at Psalms 14 and 53 on Atheism https://www.empire.kred/ROOTNK?t=94a1f39b NFLD on 13 Feb vote Liberal ! _______ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: sysutils/uefi-edk2-bhyve isssue
On Tue, Feb 23, 2021 at 08:31:18AM -0700, Rebecca Cran wrote: > On 2/23/21 7:49 AM, The Doctor wrote: > > On Mon, Feb 22, 2021 at 11:30:24PM -0700, Rebecca Cran wrote: > > >> Do you have anything in /etc/make.conf that could be affecting the build? > > > > DEFAULT_VERSIONS+=linux=c7_64 > > DEFAULT_VERSIONS+=ruby=2.7 > > DEFAULT_VERSIONS+=python=3.8 > > DEFAULT_VERSIONS+=pytho3=3.8 > > DEFAULT_VERSIONS+=bdb=5.3 > > DEFAULT_VERSIONS+=lua=5.3 > > DEFAULT_VERSIONS+=perl5=5.30 > > DEFAULT_VERSIONS+=ssl=openssl > > DEFAULT_VERSIONS+=gcc=10 > > HAVE_MOTIF=yes > > CC=/usr/local/bin/clang11 > > C++=/usr/local/bin/clang++11 > > CXX=/usr/local/bin/clang++11 > > CPP=/usr/local/bin/clang-cpp11 > > MAKE_JOBS_NUMBER=1 > > SUEXEC_DOCROOT=/usr/home > > SUEXEC_USERDIR=html > > SUEXEC_GID_MIN=100 > > SUEXEC_UID_MIN=100 > > > > Could you try removing or commenting out the CC, C++, CXX and CPP lines > please? > > -- > Rebecca Cran > Same result. -- Member - Liberal International This is doctor@@nl2k.ab.ca Ici doctor@@nl2k.ab.ca Yahweh, Queen & country!Never Satan President Republic!Beware AntiChrist rising! Look at Psalms 14 and 53 on Atheism https://www.empire.kred/ROOTNK?t=94a1f39b NFLD on 13 Feb vote Liberal ! _______ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: sysutils/uefi-edk2-bhyve isssue
On Mon, Feb 22, 2021 at 11:30:24PM -0700, Rebecca Cran wrote: > On 2/22/2021 9:21 PM, Rebecca Cran wrote: > > On 2/22/2021 8:43 AM, The Doctor wrote: > > > >> Same result and now looking to reinstall gcc48 > >> in the csm version! > > > > I just installed a fresh copy of FreeBSD 12.2 and ran 'make' in > > /usr/ports/sysutils/uefi-edk2-bhyve and it completed without any errors. > > > > I'm trying using synth next. > > That worked too. > > Do you have anything in /etc/make.conf that could be affecting the build? > > > -- > Rebecca Cran > > DEFAULT_VERSIONS+=linux=c7_64 DEFAULT_VERSIONS+=ruby=2.7 DEFAULT_VERSIONS+=python=3.8 DEFAULT_VERSIONS+=pytho3=3.8 DEFAULT_VERSIONS+=bdb=5.3 DEFAULT_VERSIONS+=lua=5.3 DEFAULT_VERSIONS+=perl5=5.30 DEFAULT_VERSIONS+=ssl=openssl DEFAULT_VERSIONS+=gcc=10 HAVE_MOTIF=yes CC=/usr/local/bin/clang11 C++=/usr/local/bin/clang++11 CXX=/usr/local/bin/clang++11 CPP=/usr/local/bin/clang-cpp11 MAKE_JOBS_NUMBER=1 SUEXEC_DOCROOT=/usr/home SUEXEC_USERDIR=html SUEXEC_GID_MIN=100 SUEXEC_UID_MIN=100 -- Member - Liberal International This is doctor@@nl2k.ab.ca Ici doctor@@nl2k.ab.ca Yahweh, Queen & country!Never Satan President Republic!Beware AntiChrist rising! Look at Psalms 14 and 53 on Atheism https://www.empire.kred/ROOTNK?t=94a1f39b NFLD on 13 Feb vote Liberal ! _______ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: sysutils/uefi-edk2-bhyve isssue
On Mon, Feb 22, 2021 at 08:04:06AM -0700, The Doctor via freebsd-virtualization wrote: > On Mon, Feb 22, 2021 at 12:30:55AM -0700, Rebecca Cran wrote: > > Oh, yes - I???ve had problems building software with the gcc48 port > > installed too. Similar to the build error reported. > > > > Rebecca > > > > Sent from my iPhone > > > > > On Feb 21, 2021, at 11:40 PM, Dustin Marquess wrote: > > > > > > ???I had the same issue if the gcc48 port was installed, but removing > > > that somehow resolved it for me. > > > > > > -Dustin > > > > > >> On Mon, Feb 22, 2021 at 12:08 AM The Doctor via freebsd-virtualization > > >> wrote: > > >> > > >>> On Sun, Feb 21, 2021 at 09:38:29PM -0700, Rebecca Cran wrote: > > >>> On 2/18/2021 6:22 PM, The Doctor via freebsd-ports wrote: > > >>>> ld-elf.so.1: /usr/local/lib/compat/libstdc++.so.6: version > > >>>> CXXABI_1.3.8 required by > > >>>> /usr/ports/sysutils/uefi-edk2-bhyve/work/edk2-2e1e8c35f3178df95d79da81ac6deec242da74c2/BaseTools/Source/C/bin/VfrCompile > > >>>> not found > > >>>> make[2]: *** [GNUmakefile:315: > > >>>> /usr/ports/sysutils/uefi-edk2-bhyve/work/Build/BhyveX64/RELEASE_GCC5/X64/MdeModulePkg/Library/BootMaintenanceManagerUiLib/BootMaintenanceManagerUiLib/DEBUG/BootMaintenanceManager.c] > > >>>> Error 1 > > >>>> make[2]: Leaving directory > > >>>> '/usr/ports/sysutils/uefi-edk2-bhyve/work/Build/BhyveX64/RELEASE_GCC5/X64/MdeModulePkg/Library/BootMaintenanceManagerUiLib/BootMaintenanceManagerUiLib' > > >>>> > > >>>> > > >>>> build.py... > > >>>> : error 7000: Failed to execute command > > >>>> make tbuild > > >>>> [/usr/ports/sysutils/uefi-edk2-bhyve/work/Build/BhyveX64/RELEASE_GCC5/X64/MdeModulePkg/Library/BootMaintenanceManagerUiLib/BootMaintenanceManagerUiLib] > > >>>> BaseTools/BinWrappers/PosixLike/../../Source/Python/build/build.py:654: > > >>>> ResourceWarning: unclosed file <_io.BufferedReader name=13> > > >>>> ResourceWarning: Enable tracemalloc to get the object allocation > > >>>> traceback > > >>> > > >>> What version of FreeBSD are you running? And what version of GCC? > > >>> > > >> > > >> 12.2 and I tried both GCC 9 and 10 ! > > >> > > Just did a pkg delete in gcc48 > > > >>> > > >>> -- > > >>> Rebecca Cran > > >>> > > >>> > > >>> _______ > > >>> freebsd-po...@freebsd.org mailing list > > >>> https://lists.freebsd.org/mailman/listinfo/freebsd-ports > > >>> To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org" > > >> > > >> -- > > >> Member - Liberal International This is doctor@@nl2k.ab.ca Ici > > >> doctor@@nl2k.ab.ca > > >> Yahweh, Queen & country!Never Satan President Republic!Beware AntiChrist > > >> rising! > > >> Look at Psalms 14 and 53 on Atheism > > >> https://www.empire.kred/ROOTNK?t=94a1f39b > > >> NFLD on 13 Feb vote Liberal ! > > >> ___ > > >> freebsd-virtualization@freebsd.org mailing list > > >> https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > > >> To unsubscribe, send any mail to > > >> "freebsd-virtualization-unsubscr...@freebsd.org" > > > > > Same result and now looking to reinstall gcc48 in the csm version! > -- > Member - Liberal International This is doctor@@nl2k.ab.ca Ici > doctor@@nl2k.ab.ca > Yahweh, Queen & country!Never Satan President Republic!Beware AntiChrist > rising! > Look at Psalms 14 and 53 on Atheism https://www.empire.kred/ROOTNK?t=94a1f39b > > NFLD on 13 Feb vote Liberal ! > ___ > freebsd-virtualization@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > To unsubscribe, send any mail to > "freebsd-virtualization-unsubscr...@freebsd.org" -- Member - Liberal International This is doctor@@nl2k.ab.ca Ici doctor@@nl2k.ab.ca Yahweh, Queen & country!Never Satan President Republic!Beware AntiChrist rising! Look at Psalms 14 and 53 on Atheism https://www.empire.kred/ROOTNK?t=94a1f39b NFLD on 13 Feb vote Liberal ! ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: sysutils/uefi-edk2-bhyve isssue
On Mon, Feb 22, 2021 at 12:30:55AM -0700, Rebecca Cran wrote: > Oh, yes - I???ve had problems building software with the gcc48 port installed > too. Similar to the build error reported. > > Rebecca > > Sent from my iPhone > > > On Feb 21, 2021, at 11:40 PM, Dustin Marquess wrote: > > > > ???I had the same issue if the gcc48 port was installed, but removing > > that somehow resolved it for me. > > > > -Dustin > > > >> On Mon, Feb 22, 2021 at 12:08 AM The Doctor via freebsd-virtualization > >> wrote: > >> > >>> On Sun, Feb 21, 2021 at 09:38:29PM -0700, Rebecca Cran wrote: > >>> On 2/18/2021 6:22 PM, The Doctor via freebsd-ports wrote: > >>>> ld-elf.so.1: /usr/local/lib/compat/libstdc++.so.6: version CXXABI_1.3.8 > >>>> required by > >>>> /usr/ports/sysutils/uefi-edk2-bhyve/work/edk2-2e1e8c35f3178df95d79da81ac6deec242da74c2/BaseTools/Source/C/bin/VfrCompile > >>>> not found > >>>> make[2]: *** [GNUmakefile:315: > >>>> /usr/ports/sysutils/uefi-edk2-bhyve/work/Build/BhyveX64/RELEASE_GCC5/X64/MdeModulePkg/Library/BootMaintenanceManagerUiLib/BootMaintenanceManagerUiLib/DEBUG/BootMaintenanceManager.c] > >>>> Error 1 > >>>> make[2]: Leaving directory > >>>> '/usr/ports/sysutils/uefi-edk2-bhyve/work/Build/BhyveX64/RELEASE_GCC5/X64/MdeModulePkg/Library/BootMaintenanceManagerUiLib/BootMaintenanceManagerUiLib' > >>>> > >>>> > >>>> build.py... > >>>> : error 7000: Failed to execute command > >>>> make tbuild > >>>> [/usr/ports/sysutils/uefi-edk2-bhyve/work/Build/BhyveX64/RELEASE_GCC5/X64/MdeModulePkg/Library/BootMaintenanceManagerUiLib/BootMaintenanceManagerUiLib] > >>>> BaseTools/BinWrappers/PosixLike/../../Source/Python/build/build.py:654: > >>>> ResourceWarning: unclosed file <_io.BufferedReader name=13> > >>>> ResourceWarning: Enable tracemalloc to get the object allocation > >>>> traceback > >>> > >>> What version of FreeBSD are you running? And what version of GCC? > >>> > >> > >> 12.2 and I tried both GCC 9 and 10 ! > >> Just did a pkg delete in gcc48 > >>> > >>> -- > >>> Rebecca Cran > >>> > >>> > >>> ___ > >>> freebsd-po...@freebsd.org mailing list > >>> https://lists.freebsd.org/mailman/listinfo/freebsd-ports > >>> To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org" > >> > >> -- > >> Member - Liberal International This is doctor@@nl2k.ab.ca Ici > >> doctor@@nl2k.ab.ca > >> Yahweh, Queen & country!Never Satan President Republic!Beware AntiChrist > >> rising! > >> Look at Psalms 14 and 53 on Atheism > >> https://www.empire.kred/ROOTNK?t=94a1f39b > >> NFLD on 13 Feb vote Liberal ! > >> ___ > >> freebsd-virtualization@freebsd.org mailing list > >> https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > >> To unsubscribe, send any mail to > >> "freebsd-virtualization-unsubscr...@freebsd.org" > > -- Member - Liberal International This is doctor@@nl2k.ab.ca Ici doctor@@nl2k.ab.ca Yahweh, Queen & country!Never Satan President Republic!Beware AntiChrist rising! Look at Psalms 14 and 53 on Atheism https://www.empire.kred/ROOTNK?t=94a1f39b NFLD on 13 Feb vote Liberal ! ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: sysutils/uefi-edk2-bhyve isssue
On Sun, Feb 21, 2021 at 09:38:29PM -0700, Rebecca Cran wrote: > On 2/18/2021 6:22 PM, The Doctor via freebsd-ports wrote: > > ld-elf.so.1: /usr/local/lib/compat/libstdc++.so.6: version CXXABI_1.3.8 > > required by > > /usr/ports/sysutils/uefi-edk2-bhyve/work/edk2-2e1e8c35f3178df95d79da81ac6deec242da74c2/BaseTools/Source/C/bin/VfrCompile > > not found > > make[2]: *** [GNUmakefile:315: > > /usr/ports/sysutils/uefi-edk2-bhyve/work/Build/BhyveX64/RELEASE_GCC5/X64/MdeModulePkg/Library/BootMaintenanceManagerUiLib/BootMaintenanceManagerUiLib/DEBUG/BootMaintenanceManager.c] > > Error 1 > > make[2]: Leaving directory > > '/usr/ports/sysutils/uefi-edk2-bhyve/work/Build/BhyveX64/RELEASE_GCC5/X64/MdeModulePkg/Library/BootMaintenanceManagerUiLib/BootMaintenanceManagerUiLib' > > > > > > build.py... > > : error 7000: Failed to execute command > > make tbuild > > [/usr/ports/sysutils/uefi-edk2-bhyve/work/Build/BhyveX64/RELEASE_GCC5/X64/MdeModulePkg/Library/BootMaintenanceManagerUiLib/BootMaintenanceManagerUiLib] > > BaseTools/BinWrappers/PosixLike/../../Source/Python/build/build.py:654: > > ResourceWarning: unclosed file <_io.BufferedReader name=13> > > ResourceWarning: Enable tracemalloc to get the object allocation traceback > > What version of FreeBSD are you running? And what version of GCC? > 12.2 and I tried both GCC 9 and 10 ! > > -- > Rebecca Cran > > > _______ > freebsd-po...@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-ports > To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org" -- Member - Liberal International This is doctor@@nl2k.ab.ca Ici doctor@@nl2k.ab.ca Yahweh, Queen & country!Never Satan President Republic!Beware AntiChrist rising! Look at Psalms 14 and 53 on Atheism https://www.empire.kred/ROOTNK?t=94a1f39b NFLD on 13 Feb vote Liberal ! ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: Call for testing: sysutils/uefi-edk2-bhyve update for upcoming edk2-stable202102
On Sun, Feb 14, 2021 at 11:42:17PM -0700, Rebecca Cran wrote: > I've done some more work on the sysutils/uefi-edk2-bhyve* ports, and I > think they're about ready to be updated from 2014.SP1 to this year's > code. My current plan is to use a pretty random commit from today > (2021-02-14), and in a couple of weeks update the port again to use code > from the upcoming stable tag, edk2-stable202102. > > I've uploaded pre-built binaries to > https://people.freebsd.org/~bcran/bhyve/BhyveX64-20210214/ . > > As I said back in November: > > The diff can be downloaded from https://reviews.freebsd.org/D27230 > (click "Download raw diff" on the right-hand side). It deletes the > uefi-edk2-bhyve-devel port, copies the current uefi-edk2-bhyve port into > uefi-edk2-bhyve-csm and makes it only build with CSM support, and > updates the uefi-edk2-bhyve port to build with recent code from > https://github.com/tianocore/edk2/tree/master/OvmfPkg/Bhyve . > > > Both testing of the BhyveX64 firmware and review feedback on the port > changes would be welcome. There's an upstream [edited: edk2-stable202102 > tag coming up soon, and I'd like to fix any issues in time for that. > Would this work with Python3 and gcc9+ ? > -- > Rebecca Cran > > _______ > freebsd-virtualization@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > To unsubscribe, send any mail to > "freebsd-virtualization-unsubscr...@freebsd.org" -- Member - Liberal International This is doctor@@nl2k.ab.ca Ici doctor@@nl2k.ab.ca Yahweh, Queen & country!Never Satan President Republic!Beware AntiChrist rising! Look at Psalms 14 and 53 on Atheism https://www.empire.kred/ROOTNK?t=94a1f39b NFLD on 13 Feb vote Liberal ! ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: RHEL virtualization
Hello, I can't talk about RHEL but by using vm-bhyve and customized configuration (based on https://github.com/churchers/vm-bhyve/blob/master/sample-templates/centos7.conf), I have managed to successfully install Ubuntu 20.04 in bhyve last week. In general, both booting and installation were smooth and installed system is completely usable. Maybe you would like to try RHEL with centos7.conf? Regards, Nazim Can. On 24/01/2021 02:14, John Kennedy wrote: > At work, we have RHEL (-ish; some RHEL, some CentOS, some OEL). Mostly v7, > some v8. Since I'm doing the Covid work-from-home telecommute, I'm trying to > recreate some of my work infrastructure while trying to plan a bit towards > the future (migrating a lot of VMs to Azure). > > What I'd like to recreate is my existing kickstart infrastructure, where I > PXE boot the system, feed it anaconda goodness which dovetails into puppet > and I can generate a clean system from a template. Works great for VMWare > and HyperV, not so much for Azure but if I can generate a snapshot disk > image Azure can ingest, I'll be happy on that score. > > I've been very happy with bhyve for FreeBSD. I messed with VirtualBox for > a while (a long time ago), but with my tendency to track stable (think: > kernel modules) and keep very current on ports-from-source (frequent > package updates, upon which VirtualBox has MANY dependencies) made that a > poorer experience than I had with it on Windows. I've been very happy with > bhyve since it's basically baked right in. > > > That being said, RHEL on bhyve has been a pain to figure out. The best I've > done so far is using sysutils/grub2-bhyve to set up the boot CD, using > BHYVE_UEFI.fd as UEFI firmware (sysutils/bhyve-firmware I think) and then > getting at the console via net/tigervnc-viewer. > > Currently I'm fighting grub-bhyve's issue finding the kernel to load (if I'm > finding the right problem reports, it doesn't seem to like modern XFS or > ext4 partitions). I couldn't get net/ipxe to PXE boot anything, and I din't > manage to get very far with sysutils/uefi-edk2-bhyve. And of course some > of these are flagged with python2.7 isses. > > I'm not a fan of grub-bhyve, but that's mostly because you have to specify > the full kernel-with-version path (changes every kernel update), but I > figure I could make an expect-script that would figure it out if I could > find a /boot filesystem type that grub-bhyve could "ls" properly. > > > Ignoring my own setup details right now, what would someone currently > bhyving RHEL recommend that I be doing right now? > > There is so much old information/documentation out there that I'm really > second-guessing myself and probably chasing a bunch of dying ports. But > someone on here must be happy with what they've got going for them. > > ___ > freebsd-virtualization@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > To unsubscribe, send any mail to > "freebsd-virtualization-unsubscr...@freebsd.org" ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: RHEL virtualization
Hello, I can't talk about RHEL but by using vm-bhyve and customized configuration (based on https://github.com/churchers/vm-bhyve/blob/master/sample-templates/centos7.conf), I have managed to successfully install Ubuntu 20.04 in bhyve last week. In general, both booting and installation were smooth and installed system is completely usable. Maybe I would like to try RHEL with centos7.conf? Regards, Nazim Can. On 24/01/2021 02:14, John Kennedy wrote: > At work, we have RHEL (-ish; some RHEL, some CentOS, some OEL). Mostly v7, > some v8. Since I'm doing the Covid work-from-home telecommute, I'm trying to > recreate some of my work infrastructure while trying to plan a bit towards > the future (migrating a lot of VMs to Azure). > > What I'd like to recreate is my existing kickstart infrastructure, where I > PXE boot the system, feed it anaconda goodness which dovetails into puppet > and I can generate a clean system from a template. Works great for VMWare > and HyperV, not so much for Azure but if I can generate a snapshot disk > image Azure can ingest, I'll be happy on that score. > > I've been very happy with bhyve for FreeBSD. I messed with VirtualBox for > a while (a long time ago), but with my tendency to track stable (think: > kernel modules) and keep very current on ports-from-source (frequent > package updates, upon which VirtualBox has MANY dependencies) made that a > poorer experience than I had with it on Windows. I've been very happy with > bhyve since it's basically baked right in. > > > That being said, RHEL on bhyve has been a pain to figure out. The best I've > done so far is using sysutils/grub2-bhyve to set up the boot CD, using > BHYVE_UEFI.fd as UEFI firmware (sysutils/bhyve-firmware I think) and then > getting at the console via net/tigervnc-viewer. > > Currently I'm fighting grub-bhyve's issue finding the kernel to load (if I'm > finding the right problem reports, it doesn't seem to like modern XFS or > ext4 partitions). I couldn't get net/ipxe to PXE boot anything, and I din't > manage to get very far with sysutils/uefi-edk2-bhyve. And of course some > of these are flagged with python2.7 isses. > > I'm not a fan of grub-bhyve, but that's mostly because you have to specify > the full kernel-with-version path (changes every kernel update), but I > figure I could make an expect-script that would figure it out if I could > find a /boot filesystem type that grub-bhyve could "ls" properly. > > > Ignoring my own setup details right now, what would someone currently > bhyving RHEL recommend that I be doing right now? > > There is so much old information/documentation out there that I'm really > second-guessing myself and probably chasing a bunch of dying ports. But > someone on here must be happy with what they've got going for them. > > ___ > freebsd-virtualization@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > To unsubscribe, send any mail to > "freebsd-virtualization-unsubscr...@freebsd.org" ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: Zfs mount error on boot
On 09/01/2021 17:11, Brandon helsley wrote: I am using VirtualBox 6.1 and have had another problem booting after an error with the system. I have made this post about the failure to boot before with screenshots on virtualbix forums but they told me it's an error with FreeBSD and to search for help there. Since I ran into the problem again I thought I would make this post. Also, this occurred after canceling an unsafe build of apache24 with Cntrl c and then rebooting. Below I have included a copy paste of part of the last log file where I think it displays what went wrong. ZFS: I/o error - all block copies unavailable ZFS: can't read MOS of pool root gotzfsboot: failed to mount default pool root FreeBSD/x86 boot ZFS: I/O error - blocks larger than 16777216 are not supported ZFS: can't find dataset 0 Default: root/<0x0>: boot: _ Well either FreeBSD has flipped, or there's something wrong with your ZFS data. (obvs.) What I do at times like this is mount he zpool on something else and find out which it is. If your datasets normally end up mounted on / (likely if you're booting off it), make sure you import using the option altroot=/something/other/than/root or you'll end up with no OS afterwards. readonly=off is also a good one. Good luck, and if you're not familiar with ZFS to import volumes, please ask. The above advice on altroot was a timely reminder, not a tutorial! Frank. _______ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
bhyve crashing on guest restart
I don't remember if that has been reported. When I start my Win 10 guest by bhyve (upon its restart), bhyve crashes. It won't start unless I do `bhyvectl --destroy --vm=$my_vm_name`.In FreeBSD-12.2 this doesn't happen. With kindest regards, Kostya Berger _______ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
RE: resizing a FreeBSD-12.1 vm on Azure
Hello, You can use resize option on Azure portal to increase the CPU and RAM of your VM. I believe you don't need to shut down the VM. Once you start the resizing, Azure will take care of rebooting the VM for you. The public IP will not change after the resizing. If you want to grow the disk size, you will have to shut down the VM. Find the disk for your VM on the portal. It should have option to grow the size. You may need to increase the FreeBSD filesystem size manually once the guest boots up again. I have not done this. I just heard from someone who has done this for Linux VM on Azure. Thanks, Wei > -Original Message- > From: owner-freebsd-virtualizat...@freebsd.org virtualizat...@freebsd.org> On Behalf Of tech-lists > Sent: Tuesday, December 15, 2020 11:57 AM > To: freebsd-virtualization@freebsd.org > Subject: resizing a FreeBSD-12.1 vm on Azure > > Hello, > > Has anyone ever resized a freebsd vm on Azure? I don't mean just the disks. > Really the whole point is to have more CPU and RAM. I have the option to do > this via their portal, just select whats required and hit 'resize' but I'm a > bit > hesitant. I guess it'll work on a windows VM but anything else? > > If you did manage to do this, did you need to shut the VM down before resizing > it? Was your external IP preserved or did it force you to have another one? > Did > you have to run anything else, like growfs? > > The author of https://forums.freebsd.org/threads/azure-vm.62440/ tried to do > this in 2017 but wasn't able to at the time. > > thanks, > -- > J. _______ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Centos Stream
[8.613719] entry_SYSCALL_64_after_hwframe+0x65/0xca [8.615120] RIP: 0033:0x7f79f8b1b26e [8.616092] Code: 89 fa 41 b8 e7 00 00 00 be 3c 00 00 00 eb 14 0f 1f 44 00 00 89 d7 89 f0 0f 05 48 3d 00 f0 ff ff 77 1a f4 89 d7 44 89 c0 0f 05 <48> 3d 00 f0 ff ff 76 e2 f7 d8 89 05 82 de 20 00 eb d8 f7 d8 89 05 [8.621098] RSP: 002b:7ffdfc61d858 EFLAGS: 0206 ORIG_RAX: 00e7 [8.623152] RAX: ffda RBX: 7f79f8b23a50 RCX: 7f79f8b1b26e [8.625241] RDX: 007f RSI: 003c RDI: 007f [8.627361] RBP: 7f79f8d1d5a0 R08: 00e7 R09: 7ffdfc61d768 [8.629375] R10: R11: 0206 R12: 0002 [8.631348] R13: 0018 R14: 7f79f8d29150 R15: [8.633458] Kernel Offset: 0x1360 from 0x8100 (relocation range: 0x8000-0xbfff) [8.636355] ---[ end Kernel panic - not syncing: Attempted to kill init! exitcode=0x7f00 [8.636355] ]--- vm_run error -1, errno 6 Why is this happening? -- Member - Liberal International This is doctor@@nl2k.ab.ca Ici doctor@@nl2k.ab.ca Yahweh, Queen & country!Never Satan President Republic!Beware AntiChrist rising! Look at Psalms 14 and 53 on Atheism https://www.empire.kred/ROOTNK?t=94a1f39b Merry Christmas 2020 and Happy New Year 2021 ! _______ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
virtio_pci: Allow memory space for configuration
For some reasons, cloud service vendors in mainland China use modified qemu. This patch will affect whether 70% of cloud servers in mainland China can run FreeBSD. https://reviews.freebsd.org/D26915 ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: HD audio problem on FreeBSD 12.1 bhyve VM
Hi Peter, BTW, I got recently a T495 running current, same audio issue on FreeBSD bhyve guests, but audio works perfectly fine on OpenSUSE 15.2. Here is /dev/sndstat on T495 pcm0: (play) pcm1: (play) pcm2: (play) pcm3: (play/rec) default pcm4: (rec) Regards, Ali On 29.09.2020 13:01, Ali Abdallah wrote: > On 29.09.2020 12:28, Peter Grehan wrote: > > Hi Ali, > > Hi Peter, > > > > > > It doesn't work for me... See below. > > ... > > > I'm booting with the following script. > > > > > > VM=freebsd12.1 > > > ifconfig tap1 up > > > bhyve -AHP -s 0:0,hostbridge -s 1:0,lpc \ > > > -s 2:0,virtio-net,tap1 -s 3:0,virtio-blk,./disk.img \ > > > -c 1 -m 4096M \ > > > -s 19,xhci,tablet \ > > > -s 11,fbuf,tcp=0.0.0.0:5920,w=1280,h=1024,wait \ > > > -s 20,hda,play=/dev/dsp0,rec=/dev/dsp0 \ > > > -l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd \ > > > $VM > > > bhyvectl --destroy --vm=$VM > > > > Does removing the 'rec=/dev/dsp0' make any difference ? > > Already tried that, doesn't make any difference. > > > > > > > Audio on the Thinkpad host is: > > > > > > > > grehan@carbon-usb:/images $ cat /dev/sndstat > > > > Installed devices: > > > > pcm0: (play/rec) default > > > > pcm1: (rec) > > > > pcm2: (play) > > > > > > > > > > I have the issue on my daily driver, a Thinkpad T430. But also tested on > > > my x220, same issue, only noise. However, sound works perfectly fine > > > on OpenSUSE guest. > > > > What does 'cat /dev/sndstat' show on your 430/x220 ? > > Here is what I have on my T430 > > pcm0: (play/rec) > pcm1: (play) > pcm2: (play) > pcm3: (play) > pcm4: (play/rec) default > > Please note that I have the following in the device hints in order not > to have a separate pcm1 for the internal microphone > (pcm1: (rec)) > > hint.hdaa.0.nid18.config="as=3 seq=0 device=mic" > hint.hdaa.0.nid24.config="as=3 seq=15 device=mic" > > On my x220: > > pcm0: (play/rec) default > pcm1: (play) > pcm2: (play) > pcm3: (play) > > > later, > > > > Peter. > > > > Cheers, > Ali > > > ___ > freebsd-virtualization@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > To unsubscribe, send any mail to > "freebsd-virtualization-unsubscr...@freebsd.org" > ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: Using OpenBSD guest as PF firewall
Thank you. I didn't know they had never 're-synced'. Paul On Wednesday, November 4, 2020, 2:48:20 PM EST, Thomas Laus wrote: Paul Pathiakis [pathia...@yahoo.com] wrote: > Hi, > Is there a reason you would want to use OpenBSD versus FreeBSD? > FreeBSD has pf and I use it on my server at home. > > Are you exploring OpenBSD? Did you not know that pf is an > available firewall on FreeBSD? > The OpenBSD PF firewall is several revisions ahead and more inte- grated than one in FreeBSD. The PF versions diverged in OpenBSD 4.7 and the one in FreeBSD was left behind. I use them both on their respected OS. It was very recent in bhyve development that pci-passthru was finally operational with an OpenBSD guest and I was building a new server and wanted to test things out. Tom -- Public Keys: PGP KeyID = 0x5F22FDC1 GnuPG KeyID = 0x620836CF ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: Using OpenBSD guest as PF firewall
Hi, Is there a reason you would want to use OpenBSD versus FreeBSD? FreeBSD has pf and I use it on my server at home. Are you exploring OpenBSD? Did you not know that pf is an available firewall on FreeBSD? Paul On Wednesday, November 4, 2020, 11:32:33 AM EST, Thomas Laus wrote: Is there a How-To or a Handbook article for using an OpenBSD guest as a firewall for a FreeBSD host? I have enabled pci-passthru and the OpenBSD guest can use the functional FreeBSD NIC hardware and has a hostname.vio0 configured with an IP address and netmask. I have created a public switch on the FreeBSD side and have added tap0. I can connect both from and to the OpenBSD / FreeBSD host by their respective IP addresses. These addresses both use the same subnet. I can't connect anywhere else from the FreeBSD host. The OpenBSD guest has an open pf.conf file to pass all packets to from vio0 to my re0 NIC. The OpenBSD system is version 6.8 and the FreeBSD is Current r367054. It looks like I need to create a bridge somewhere, but can't find the proper commands to make one. I have read a few instructions, but none of them use commands from the vm-bhyve port. I found that is always a good idea to not 'mix and match' these methods. Tom -- Public Keys: PGP KeyID = 0x5F22FDC1 GnuPG KeyID = 0x620836CF ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org" _______ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Disinfectant
Good morning, looking for companies interested in raising additional capital by diversifying their offer in soaps, liquids and gels for hand disinfection and cosmetics for body and hair care. The distribution of innovative products corresponding to the current preferences of customers in the field of hygiene and preventive healthcare allows our partners to gain new markets and achieve better economic results. In addition to products with bactericidal action, our range includes shower gels, shampoos and hair conditioners, as well as efficient, concentrated detergents. The versatility (suitable for all skin types) combined with an affordable price means that customers make an informed choice of a product among others available on the market. Are you interested in cooperation? Ethan Smith ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: [Bug 250802] bhyve exited with status 134 when GPU PCI passthrough
Are you on AMD? Placing the PCI bars above 4 GB causes the processor to exit back to the vmm instead of handling the passthrough in the vm. But the vmm does not expect to handle memory passthrough, only i/o, tripping this assertion. There is a hardcoded max bar size where we decide whether to allocate the bar above or below the 4 GB mark. You can raise this limit and it may fix this error. This allows the GPU to work in Linux. However the GPU will not start in Windows, probably because we do not implement the ROM bar yet. — RHC. On Mon, Nov 2, 2020 at 09:10, wrote: > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=250802 > > Bug ID: 250802 > Summary: bhyve exited with status 134 when GPU PCI passthrough > Product: Base System > Version: 12.2-STABLE > Hardware: amd64 > OS: Any > Status: New > Severity: Affects Only Me > Priority: --- > Component: bhyve > Assignee: virtualizat...@freebsd.org > Reporter: mr.xa...@gmail.com > > Try to pass through GPU to a Windows 10 guest. > > /boot/loader.conf: > vmm_load="YES" > pptdevs="1/0/0 1/0/1" > > # pciconf -lv > ppt0@pci0:1:0:0: class=0x03 card=0x38991642 chip=0x0a6310de rev=0xa2 > hdr=0x00 > vendor = 'NVIDIA Corporation' > device = 'GT218 [GeForce 310]' > class = display > subclass = VGA > ppt1@pci0:1:0:1: class=0x040300 card=0x38991642 chip=0x0be310de rev=0xa1 > hdr=0x00 > vendor = 'NVIDIA Corporation' > device = 'High Definition Audio Controller' > class = multimedia > subclass = HDA > > bhyve options: > [bhyve options: -c 1 -m 3G -Hwl > bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd -U > 00dad61d-0d2e-11eb-936d-70f3951447ea -S] > [bhyve devices: -s 0,hostbridge -s 31,lpc -s > 4:0,ahci,hd:/mnt/Data/vm/win10/disk0.img -s > 5:0,virtio-net,tap0,mac=58:9c:fc:01:00:a8 -s 6:0,passthru,1/0/0 -s > 6:1,passthru,1/0/1 -s 7:0,xhci,tablet] > [bhyve console: -l com1,stdio] > [bhyve iso device: -s 3:0,ahci-cd,/mnt/Data/vm/.config/null.iso] > > Windows guest see NVidia GPU, try to install driver, and after reboot bhyve > stop working with error: > > bhyve exited with status 134 > Unhandled ps2 keyboard command 0x02 > Unhandled ps2 keyboard command 0x02 > Assertion failed: (pi->pi_bar[baridx].type == PCIBAR_IO), function > passthru_write, file /usr/src/usr.sbin/bhyve/pci_passthru.c, line 916. > > # uname -v > FreeBSD 12.2-STABLE r37 GENERIC > > -- > You are receiving this mail because: > You are the assignee for the bug. > ___ > freebsd-virtualization@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > To unsubscribe, send any mail to > "freebsd-virtualization-unsubscr...@freebsd.org" ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
RE: FreeBSD 12.1 i386 on Hyper-V Fails to Mount Root
> Howdy, > > Just started the upgrade process on some FreeBSD 11.4 VMs to FreeBSD 12.1 > using freebsd-update. VMs are running on multiple Hyper-V 2012 R2 and 2019 > systems, all systems with 2020-09 Cumulative Updates installed. > The amd64 VMs are upgrading happily. > The i386 VMs die a horrible death - the kernel throws the following error: > > vmbus0: cannot find free IDT vector > > shortly followed by mountroot failure due to the lack of storage device > initialisation. > Checked the FreeBSD 12.1 i386 ISO in case of environment pollution - same > vmbus error, same lack of storage device initialisation failure. Ditto with > the 12.2-RC2 i386 ISO. > Did you try 12.0? Maybe something changed in 12.x causes failure. I386 is not well tested on Hyper-V. I will try to reproduce and take a look. Thanks, Wei ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Szkolenie z angielskiego
Dzień dobry, czy byliby Państwo zainteresowani poszerzeniem umiejętności językowych wśród współpracowników? Reprezentuję szkołę językową, która już od 25 lat organizuje profesjonalne szkolenia z języka angielskiego dla firm. Już ponad 300 partnerów biznesowych korzysta z opracowanej przez nas strategii rozwoju językowego. Zajęcia prowadzą lektorzy z doświadczeniem w języku biznesowym zarówno w trybie stacjonarnym, jak i online. Zakres spotkań odpowiednio dopasowujemy do branży, stanowisk uczestników i zakresów ich obowiązków. Gwarantujemy elastyczność czasową oraz próbne spotkanie z lektorem, co pomoże ustalić potrzeby językowe i jasno sprecyzować cel. Jako autoryzowany ośrodek Cambridge zapewniamy możliwość certyfikacji kompetencji językowych w ramach Autoryzowanego Centrum Egzaminacyjnego Cambridge English Language Assessment. Jeśli są Państwo zainteresowani rozważeniem naszej propozycji, proszę o informację zwrotną. Z pozdrowieniami Aleksandra Musiał Project Manager ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Disinfectant
Good morning, looking for companies interested in raising additional capital by diversifying their offer in soaps, liquids and gels for hand disinfection and cosmetics for body and hair care. The distribution of innovative products corresponding to the current preferences of customers in the field of hygiene and preventive healthcare allows our partners to gain new markets and achieve better economic results. In addition to products with bactericidal action, our range includes shower gels, shampoos and hair conditioners, as well as efficient, concentrated detergents. The versatility (suitable for all skin types) combined with an affordable price means that customers make an informed choice of a product among others available on the market. Are you interested in cooperation? William Jones ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Doc #978432: faxed duplicate
Statement #28178 Best regards, Keller Anderson ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
MSR accesses that slows down the hypervisor/host
Hello, There are couple AMD processor related MSRs which are being accessed in FreeBSD. #define MSR_AMDK8_IPM 0xc0010055 #define MSR_LS_CFG 0xc0011020 We are seeing a lot of CPU time being spent in the host (Hyper-V) in handling traps when accessing these MSRs. Especially the first MRS is frequently being accessed in cpu_idle() so the performance impact to host is significant. We noted that Linux made some code changes in the 4.10 kernel to access the first MSR much less frequently. So we are wondering if there are similar changes in FreeBSD that might be in the plan. Microsoft Hyper-V team is also planning some work to speed up the accesses to these MSRs. However any suggestions or plan to improve in the FreeBSD guest are welcome. Thanks, Wei ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: grub-bhyve device map file usage
On 2020-09-14T16:28:15 -0700 Chuck Tuffli wrote: > Hi > > I'm working on an update to grub-bhyve and wanted to know if people's > map files differ from: > (hd0) /some/path/to/the/disk.img > Primarily, I'm interested if the map files use a device other than > 'hd', but I'd also be curious about use cases involving more than one > entry. TIA! From a production machine here: (hd0) /dev/zvol/storage/vm/66d99c52-9be0-434e-935e-7c7105952308/disk-0_1_0 (hd1) /dev/zvol/storage/vm/66d99c52-9be0-434e-935e-7c7105952308/disk-0_5_0 (cd0) /storage/images/debian-10.5.0-amd64-xfce-CD-1.iso hd0 is the boot device. hd1 is a larger disk image used to hold a database. cd0 is the installation media (and kept there in case we need to do some sort of rescue boot). -- Mark Raynsford | https://www.io7m.com pgpW9YPVXGc7Y.pgp Description: OpenPGP digital signature
Restricting IP ranges for guests over tap devices
Hello! Let's say I have a machine running a few dozen bhyve guests. Each bhyve guest gets its own tap device, and all of the tap devices are connected to a bridge. Everything works fine. I can write pf rules that control access between each guest, and between each guest and the world. I can't directly observe the IP addresses that the guests have assigned to the tap devices I gave them, but if I know the addresses beforehand, I can for example write pf rules that say things like: block log all pass in on tap23 proto tcp \ from any to $guest_23_ip port ssh modulate state That then means that even if the guest is compromised and tries to bind a server to another address, the pf rules won't allow anyone else to actually connect to it. The good thing about this is also the bad thing about this; I have to write specific rules that say "only allow access to this specific IP via this specific tap device". Over dozens of guests, that can multiply to hundreds of laboriously maintained rules. Is there some more general way I can supply a mapping between tap devices and allowed addresses? Remember that pf can't see the guest addresses on the host sides of the tap devices, so I can't use the (device) syntax to expand to "the address of a NIC called 'device'". I can generate rule sets, but perhaps there's something "better"[0]? The documentation isn't suggesting much. [0] Better in the sense that, for example, a table is usually better than a massive list of macros. :) -- Mark Raynsford | https://www.io7m.com pgpnnuwW7Yvvx.pgp Description: OpenPGP digital signature
no connection to bhyve guest
Hi,upgraded to 362292 via buildworld.Now I cannot connect to my bhyve guest as I used to: neither via VNC nor via RDP.VNC gets error: unable to connect the socket. Address family not supported by protocol family (47). Neither can I ping my bhyve IP (it uses a separate NIC and should have no problems) Internet connectivity is ok and I can ping other hosts on my network. The problem only concerned bhyve guest. Отправлено из Yahoo Почты для Android ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Single stepping a bhyve guest on AMD64
I am investigating using the -G flag to attach gdb to a running guest machine. It looks like breakpoints and single instruction stepping are not implemented in vmm/amd/svm.c. It seems that it was added to vmm/intel/vmx.c. I spent a little bit of time reading through the AMD64 Architecture Manual; it doesn't seem to provide a direct way to implement this in a way transparent to the guest? One way to implement single stepping that occurs to me is to set the trap flag in the guest's RFLAGS register, and then intercept each #DB exception in the vmm. Has anyone looked at this on the AMD side? Any advice? --- Rob Crowston. _______ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
RE: freebsd 12.0 azure and azure-agent (waagent)
Not running waagent on guest could cause problems in Azure. Fortunately Li-Wen Hsu from FreeBSD Foundation has recently updated the agent code to use Python 3.x. He is also working on publishing the 12.1 image on Azure Marketplace. You can check with him directly. You can enable the boot diagnostics feature on Azure portal and check the serial console output for the VMs running in Azure. Wei > -Original Message- > From: owner-freebsd-virtualizat...@freebsd.org virtualizat...@freebsd.org> On Behalf Of tech-lists > Sent: Sunday, May 24, 2020 8:04 PM > To: freebsd-virtualization@freebsd.org > Subject: freebsd 12.0 azure and azure-agent (waagent) > > Hello, > > I'm considering upgrading a freebsd instance originally installed onto azure > via > their marketplace. It's freebsd-12.0 and I'd like to bring it to 12.1 via > freebsd- > update. > > I notice that when it was provisioned, it seems azure-agent (waagent) was > installed manually, presumably by azure themselves. It was not installed via > the > pkg install method. It's dependent on python27 which is now obsolete. > Apart from that, it looks just like a standard basic vm. > > What I'd like to know: > > is waagent required for the VM to function? If I upgrade, and something goes > wrong with waagent, will the vm come back up? Or will it be stopped if > waagent is non-functional? > > As 12.0 -> 12.1 involves default python going 27 -> 3[6-7] what do I need to > look out for regarding waagent? I presume I'm going to have to install azure- > agent (python37) over the top. > > All this is an issue because there's no ILO functionality exposed to the > customer > in Azure. > > thanks, > -- > J. ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: Bhyve help
What are your network interfaces called on your machine? You probably have to change igb0 in the example to em0 (an Intel card) or re0 (a Realtek card). Type “ifconfig” to list all the interfaces, and look for ones with an ip address to figure it out. On Tue, May 5, 2020 at 10:10, Brandon helsley wrote: > I'm trying to set up bhyve like the example to learn and when doing the > physical interface interface get the result no such file or directory the > command is: > > ifconfig bridge0 addm igb0 addm tap0 > > "In this example, the physical interface is igb0" > > Sent from Outlook Mobile<https://aka.ms/blhgte> > ___ > freebsd-virtualization@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > To unsubscribe, send any mail to > "freebsd-virtualization-unsubscr...@freebsd.org" ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Quarantine notification - freebsd.org
___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: bhyve + AMD + PCI passthru
First, do these Jaguar processors support device passthrough? Is there anything interesting in dmesg? What does pciconf show? Maybe bhyve claiming the USB controller is interfering with SATA somehow. On my pcengines board the two devices are logical neighbours, and I guess your scenario has never been tested. Can you see ada0’s FAT file system in the virtual UEFI gui? If you can, what happens if you execute the boot loader manually? I normally point the virtual storage at a block file (or a zvol) rather than essentially pass through a real device like /dev/ada0. Can you try that?Maybe try a guest Linux installation first using a block file on your boot disk. The diagnostics will be better than a Windows guest. Finally which version of SeaBIOS are you using? (Also, too late if you already installed Windows, but why not use the paravirtualized virtual disk device instead of nvme?) On Thu, Apr 30, 2020 at 03:12, Dustin Marquess wrote: > I have this weird issue. I have a PC Engines box with a AMD SoC in it. > > I'm trying to pass through the USB 3.0 controller to a Windows VM. I added: > > hw.vmm.amdvi.enable="1" > pptdevs="0/16/0" > > to /boot/loader.conf, rebooted, installed Windows, etc. Everything > works great until I add: > > -s 5,passthru,0/16/0 > > to my bhyve command line. As soon as I add that (and don't change > anything else), TitanoCore suddenly can't boot from the virtual HD > anymore: > > Boot Failed. EFI Hard Drive > .PXE-E18: Server response timeout. > Boot Failed. EFI Network > UEFI Interactive Shell v2.1 > EDK II > UEFI v2.40 (BHYVE, 0x0001) > Mapping table > BLK0: Alias(s): > PciRoot(0x0)/Pci(0x3,0x0)/Sata(0x0,0x0,0x0) > Press ESC in 1 seconds to skip startup.nsh or any other key to continue. > Shell> > > I take the passthru back out and Windows works fine. > > Does this make sense? > > My full command line is: > > bhyve -A -H -P -w -S -u -c sockets=1,cores=3,threads=1 -m 3G \ > -s 0,amd_hostbridge \ > -s 3,nvme,/dev/ada0 \ > -s 5,passthru,0/16/0 \ > -s 10,virtio-net,tap0,mac=4a:ab:0f:25:4c:b7 \ > -s 20,virtio-rnd \ > -s 31,lpc \ > -l com1,/dev/nmdm0A -l > bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd win2019 > > Changing nvme to ahci-hd doesn't help. Only difference between the > two is that "Boot Failed. EFI Hard Drive" turns into "Boot Failed. EFI > Misc Device". > > This is really strange! > > Thanks, > -Dustin > ___ > freebsd-virtualization@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > To unsubscribe, send any mail to > "freebsd-virtualization-unsubscr...@freebsd.org" ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
grub-bhyve
Grub-bhyve is using Gcc 4.8.5 to copile with. I recall compiling with gcc 9.0 with a few updates needed to the code. Has anyone else tried this? -- Member - Liberal International This is doctor@@nl2k.ab.ca Ici doctor@@nl2k.ab.ca Yahweh, Queen & country!Never Satan President Republic!Beware AntiChrist rising! https://www.empire.kred/ROOTNK?t=94a1f39b Look at Psalms 14 and 53 on Atheism Marvelous Truth, confront us at every turn, in every guise. -Denise Levertov ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: [GPU pass-through] no compatible bridge window for claimed BAR
> This solved all the problems with BAR allocation. Nice! > Do we have already some code for ROM BAR somewhere? Henrik Gulbrandsen was working on expansion ROM support last July (CC'd). He posted his work up on https://www.gulbra.net/freebsd-bhyve/. ‐‐‐ Original Message ‐‐‐ On Thursday, 19 March 2020 03:52, Alex Erley wrote: > Hello, > > You are right about mapping 64-bit BARs. > > === (1) === > The initial value of PCI_EMUL_MEMBASE64 is 0xD0 and it > corresponds to the 64-bit mmio window set by QWordMemory(...) > in DSDT from pci_bus_write_dsdt(), so guest VM gets from > UEFI BIOS this 64-bit window: > 0x00D0-0x00DF > and it is ignored by guest kernel during boot: > ACPI: PCI Root Bridge [PC00] (domain [bus 00-ff]) > acpi PNP0A03:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM > Segments MSI] > acpi PNP0A03:00: _OSC failed (AE_NOT_FOUND); disabling ASPM > acpi PNP0A03:00: host bridge window [mem 0xd0-0xdf > window] (ignored, not CPU addressable) > acpi PNP0A03:00: host bridge window expanded to [io 0x-0x0cf7]; > [io 0x-0x0cf7 window] ignored > > 64-bit BAR code must be fixed to set proper memory region depending on > host and guest resources. > In general it makes sense only for huge BAR sizes, but in most cases all > the guest devices should fit in available 32-bit mmio window. > I decided to put it aside and concentrate my efforts on 32-bit BARs. > > So, I increased the max size in pci_emul_alloc_pbar() from 32 to 256 Mb > to have all BARs allocated in 32-bit mmio window (for me it still looks > like a hack). > As I already tried it before, BHyve failed on VM start and I turned to > debug what happened. > > Allocation goes in 32-bit mmio window 0xc000-0xdfff (512Mb). > Keeping in mind these two requirements from PCI standard: > > - size of each region must be a power of two (it is already OK), > > - base address must be aligned on a boundary equal to the region size, > BARs are allocated in the order of their indices: > BAR[0]: type=2, size=0x100 => addr=0xc000, nextaddr=0xc100 > BAR[1]: type=3, size=0x1000=> addr=0xd000, nextaddr=0xe000 > BAR[3]: type=3, size=0x200 => KO, 0xe000+size > 0xdfff > BAR[5]: type=1, size=0x80 > > I fixed cfginitbar() in pci_passthru.c to allocate BARs in different > order to make allocations more compact. They have to be allocated from > bigger claimed size to smaller (btw, as it is done on host system). > For this, in cfginitbar() I reordered BAR indices before calling > pci_emul_alloc_pbar(). > This solved all the problems with BAR allocation. > (will share my patch when review it a bit more) > > === (2) === > As you pointed before, now I faced the ROM mapping problem. > > All the ROM BARs are disabled in guest VM and quick look shows that > as PCI_BARMAX==5 (=PCIR_MAX_BAR_0 from ), > > > so only 6 BARs initialized in cfginitbar() at pci_passthru.c. > It corresponds to registers 10h, 14h, 18h, 1ch, 20h, 24h. > Expansion ROM Base Address register 30h is not initialized at all. > > I'd like to add missing code for it in pci_passthru.c. > Physical guest memory map described in pci_emul.c says all ROM BARs > must be allocated in 8Mb window 0xf000-0xf07f and it already > matches generated DSDT. > > From PCI documentation dealing with ROM doesn't seem to be very > complicated, but anyway, I'm interested in > > - What pitfalls will be on that way? > - Do we have already some code for ROM BAR somewhere? > > Any help is welcome. > > Have a nice day, > Alex > > On 3/15/20 3:20 PM, Robert Crowston wrote: > > > > I suggest you map the BAR into the 32 bit address space, unless you have so > > many PCI devices that this is not feasible. Just raise the limit of the > > special 64 bit handling to 1 GB or something big. > > > > - Many/most(?) consumer BIOS/UEFIs map 64 bit bars into the 32 bit > > address space by default, so this configuration is much more tested for > > device drivers and guest operating systems. > > - Passthrough doesn't work for me at all above the 4GB memory window on > > my recent AMD system. It exits back to the hypervisor, which then crashes > > with a failed assert because it doesn't expect to handle this. > > > > With this tweak it is possible to use the open source nVidia driver on > > Linux. > > However, your next problem---if you want to use proprietary drivers---is > > that access to the ROM BAR is not supported or emulated. If that could be > >
Re: bhyve: passthrough SMART info from host nvme controller
> Critical Warning State: 0x00 > Available spare: 0 > Temperature: 0 > Device reliability: 0 > Read only: 0 > Volatile memory backup: 0 > Temperature: 309 K, 35.85 C, 96.53 F > Available spare: 100 > Available spare threshold: 10 > Percentage used: 0 > Data units (512,000 byte) read: 173065 > Data units written: 9381970 > Host read commands: 4123220 > Host write commands: 846568092 > Controller busy time (minutes): 948 > Power cycles: 918 > Power on hours: 2631 > Unsafe shutdowns: 812 > Media errors: 0 > No. error info log entries: 2546 > Warning Temp Composite Time: 75 > Error Temp Composite Time: 58 > Temperature 1 Transition Count: 0 > Temperature 2 Transition Count: 0 > Total Time For Temperature 1: 0 > Total Time For Temperature 2: 0 > > But currently bhyve has Capsicum capability, I cannot > open /dev/nvme0 within pci_nvme.c without extra code. > (currently I just disable the Capsicum capability) > > any feedback? > > freebsd-virtualization@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > To unsubscribe, send any mail to > "freebsd-virtualization-unsubscr...@freebsd.org" ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: [GPU pass-through] no compatible bridge window for claimed BAR
-0xc1ff 16M gap > pci :00:01.0: reg 0x1c:[mem 0xc200-0xc3ff 64bit pref] 32M > pci :00:01.1: reg 0x10:[mem 0xc400-0xc4003fff]pci :00:02.0: reg > 0x14: [mem 0xc4004000-0xc4005fff] > pci :00:03.0: reg 0x14: [mem 0xc4006000-0xc4007fff] > pci :00:1d.0: reg 0x10: [mem 0xc4008000-0xc400807f] > ... 0xc4008080-0xc4ff <16M gap > pci :00:1d.0: reg 0x14: [mem 0xc500-0xc5ff] 16M > pci :00:1e.0: reg 0x10: [mem 0xc600-0xc6000fff] > ... 0xc6001000-0xd2ff <208M gap > pci :00:01.0: reg 0x30:[mem 0xd300-0xd307 pref] 512K > 0xd308-0xdfff <208M gap > pci :00:01.0: reg0x14:[mem 0x44000-0x44fff 64bit pref] 256M^^^- > this value is outside allowed range > > == (d) == > So, there is no window for 256M BAR, although there are 2 big gapes > of 208M in 512M space provided for BAR allocation by PCI bus. > > So, BAR reg 0x14 of size 256M for device 01.0 must be inside provisioned > 512M region 0xc000-0xdfff. > But refering to (1) above, setting base address to any value below > 0x44000 breaks bhyve on start. > According to (b), this value corresponds to upper addressable memory > limit in guest VM. > > So I'm blocked here at the moment: > > - Guest VM requires a value which BHyve doesn't like. > - Guest VM allocates BARs with huge gapes. > > I have little knowledge about PCI bus internals, although I already read > some articles on internet. > Could it be some ACPI trick to do? > I'd be happy to hear any ideas... > > PS > I suspect that if I take other OS as a guest VM or other pass-through > GPU model, it would probably allocate BARs properly. > But this is not what I want for this config. > There should be a way to allocate 256M BAR in guest Linux. > > Have a nice day, > Alex > > > freebsd-virtualization@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > To unsubscribe, send any mail to > "freebsd-virtualization-unsubscr...@freebsd.org" ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
RE: Cannot run FreeBSD 11 as a vm-bhyve guest under FreeBSD 12
Hello. I'd like to run a FreeBSD 11.3 VM under FreeBSD 12.1. As I'm already using vm-bhyve to run a Windows guest, I think the best (or possibly only) choice is to use bhyve for the FreeBSD guest too. I've got the following conf file: > guest="freebsd" > loader="bhyveload" > cpu=1 > memory=512M > network0_type="virtio-net" > network0_switch="public" > disk0_type="virtio-blk" > disk0_name="disk0" > disk0_dev="sparse-zvol" Running "vm install -f f11b FreeBSD-11.3-RELEASE-amd64-disc1.iso", will boot the guest, but it will hang after a while with: > /boot/kernel/kernel text=0x1564b08 data=0x145330+0x4cdf30 > syms=[0x8+0x16daf0+0x8+0x186a43] Booting... > | "vm stop" will stop it (through an ACIP shutdown) after a while. So I tried with UEFI; here's the config file: > guest="freebsd" > loader="uefi" > > cpu=1 > memory=512M > network0_type="virtio-net" > network0_switch="public" > disk0_type="virtio-blk" > disk0_name="disk0" > disk0_dev="sparse-zvol" > disk1_type="ahci-cd" > disk1_dev="custom" > disk1_name="/zroot/vm/.iso/FreeBSD-11.3-RELEASE-amd64-disc1.iso" > graphics="yes" > xhci_mouse="yes" > graphics_listen="192.168.xxx.1" Starting it with "vm start" and connecting through VNC, it goes through the same stages as above, but ends up with a blank screen with a white block cursor in the first column of the first row. >Any hint on how to get past this? This is likely caused by an issue in the way vm-bhyve handled stdio, which only worked correctly with older builds of bhyve. I would recommend installing vm-bhyve from ports (you should get version 1.4.2), which hopefully will boot using bhyveload with no issues. Matt > bye & Thanks > av. ___ >>freebsd-virtualization@freebsd.org mailing list >>https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization >To unsubscribe, send any mail to >"freebsd-virtualization-unsubscr...@freebsd.org" ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: upgrading 12.0-p12 to 12.1-RELEASE on Azure
I, I upgraded my 12.0 instance on azure to 12.1 with no issues whatever. It was the same old same old upgrade procedure that you’d normally follow. I did the upgrade on an Azure image with one nothing on the host and only one jail and experienced no issues. Let me know if you have any issues. Thanks, Sent from my iPhone > On Feb 11, 2020, at 9:58 AM, tech-lists wrote: > > Hi, > > Does upgrading 12.0 to 12.1 on Azure require anything azure-specific outside > what freebsd azure instance has as standard? > > The upgrade method being the standard freebsd-update method. > > thanks, > -- > J. ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
System (ps/top) & bhyvectl memory usage differences
Hello, Is there a reason why the resident memory used by a bhyve guest is quite different when comparing ps/top & bhyvectl? Does bhyvectl take in account something in kernel space that top/ps doesn't see? USER PID %CPU %MEM VSZRSS TT STAT STARTED TIME COMMAND root 12670 0.1 1.4 2120984 951764 1 SC+ 22Jan20 157:29.22 bhyve: smtp-a (bhyve) # bhyvectl --get-stats --vm=smtp-a | grep Res Resident memory 1439875072 1.4G vs 925M - Another thing I've just remembered, which is probably unrelated to the above but still concerning memory. I have a guest with 2G memory allocated, and dmesg lists 2048MB real memory. The real & avail figures are also quite close which resembles the output I generally expect on real hardware. Hypervisor: Origin = "bhyve bhyve " real memory = 2147483648 (2048 MB) avail memory = 2043318272 (1948 MB) However, I have a guest with 5G allocated, and get the following in dmesg - Hypervisor: Origin = "bhyve bhyve " real memory = 6442450944 (6144 MB) avail memory = 5141663744 (4903 MB) bhyveload -m 5G ... bhyve -c 2 -m 5G -AHP ... I haven't tested where it seems to change. My only theory is that it could possibly be something to do with crossing the old 32bit limit? Regards, Matt Churchyard _______ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
RE: bhyve issues on Dell C6220 node
> Hi, > >2016 is slow (even slower doing windows updates). 2019 is much better. A > >tip I have found is a minimum of 1 cpu, 2 cores and 4 threads to get decent > >speed from that OS (more CPUs in bhyve >tends to make performance worse - in > >my observations - in 2016). Also, use the Virtio collection from RedHat for > >vionet and viostor. We are currently using 0.1.171 without issue. The ahci > >>emulation in itself is extremely slow. NVMe and virtio is really the only > >way to go. > > Is there anything else I can check here? I haven’t got round to testing > networking yet but I’m using nvme for the disk. > It’s basically unusable and there is no way I could put anything production > on it. Just highlighting an icon on the desktop takes several seconds. >Windows is slow when running on Intel CPUs that don't support APICv. >That are (nearly?) all desktop CPUs, all Xeons before Sandy Bridge and some >Xeons after it. The problem is that Bhyve doesn't implement TPR shadowing. I'm >currently working on it. The review can >be found >here: https://reviews.freebsd.org/D22942 The speedup is about factor 6! >I've received some feedback in a private mail, a second version that adds TPR >thresholds can be found in my Github branch here: > https://github.com/Yamagi/freebsd/commits/wip/tpr_shadowing >A backport to 12.1 (the branch also includes the Intel SpeedShift patches from >https://reviews.freebsd.org/D18028) is here: >https://github.com/Yamagi/freebsd/commits/production/12.1 >I've applied it to 2 of my production servers about 4 hours ago. Looks good so >far. I'll update the review when I'm sure that it doesn't break anything, >maybe early next week. I've been running the patch for a few days now and not seen any problems. It's a complete difference compared to running stock 12.1. Thanks for your response and awesome work. Don't know if any more work on it is required but it would be nice if this made it into 12.2 as it's much easier for me to track the binary releases than have to rebuild the entire system. This fairly small patch makes a massive difference to Windows performance on something that doesn't have APICv I also found the fix for my original console issue (actually someone else fixed it, I just hadn't seen their pull request). I'm using vm-bhyve which originally redirected bhyve output to a log file to catch any error output. Changes to the way bhyve opens stdin & stderr caused this to stop working correctly. Regards, Matt >Regards, >Yamagi -- Homepage: https://www.yamagi.org Github: https://github.com/yamagi GPG: 0x1D502515 _______ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: bhyve issues on Dell C6220 node
> On 9 Jan 2020, at 18:58, Yamagi Burmeister wrote: > > Hi, > >>> 2016 is slow (even slower doing windows updates). 2019 is much better. A >>> tip I have found is a minimum of 1 cpu, 2 cores and 4 threads to get decent >>> speed from that OS (more CPUs in bhyve >tends to make performance worse - >>> in my observations - in 2016). Also, use the Virtio collection from RedHat >>> for vionet and viostor. We are currently using 0.1.171 without issue. The >>> ahci >emulation in itself is extremely slow. NVMe and virtio is really the >>> only way to go. >> Is there anything else I can check here? I haven’t got round to testing >> networking yet but I’m using nvme for the disk. >> It’s basically unusable and there is no way I could put anything production >> on it. Just highlighting an icon on the desktop takes several seconds. > > Windows is slow when running on Intel CPUs that don't support APICv. > That are (nearly?) all desktop CPUs, all Xeons before Sandy Bridge > and some Xeons after it. The problem is that Bhyve doesn't implement > TPR shadowing. I'm currently working on it. The review can be found > here: https://reviews.freebsd.org/D22942 The speedup is about factor > 6! > > I've received some feedback in a private mail, a second version that > adds TPR thresholds can be found in my Github branch here: > https://github.com/Yamagi/freebsd/commits/wip/tpr_shadowing > > A backport to 12.1 (the branch also includes the Intel SpeedShift > patches from https://reviews.freebsd.org/D18028) is here: > https://github.com/Yamagi/freebsd/commits/production/12.1 > > I've applied it to 2 of my production servers about 4 hours ago. Looks > good so far. I'll update the review when I'm sure that it doesn't break > anything, maybe early next week. I’ve had a system with the TPR changes applied for a few hours now and it seems to be ok. The install took less than 10 minutes compared to 40+ minutes before and is much more useable. I’ll leave it running over the weekend and see how it goes. Regards Matt > > Regards, > Yamagi > > -- > Homepage: https://www.yamagi.org > Github: https://github.com/yamagi > GPG: 0x1D502515 ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
RE: bhyve issues on Dell C6220 node
Hi, > >2016 is slow (even slower doing windows updates). 2019 is much better. A > >tip I have found is a minimum of 1 cpu, 2 cores and 4 threads to get decent > >speed from that OS (more CPUs in bhyve >tends to make performance worse - in > >my observations - in 2016). Also, use the Virtio collection from RedHat for > >vionet and viostor. We are currently using 0.1.171 without issue. The ahci > >>emulation in itself is extremely slow. NVMe and virtio is really the only > >way to go. > > Is there anything else I can check here? I haven’t got round to testing > networking yet but I’m using nvme for the disk. > It’s basically unusable and there is no way I could put anything production > on it. Just highlighting an icon on the desktop takes several seconds. >Windows is slow when running on Intel CPUs that don't support APICv. >That are (nearly?) all desktop CPUs, all Xeons before Sandy Bridge and some >Xeons after it. The problem is that Bhyve doesn't implement TPR shadowing. I'm >>currently working on it. The review can be found >here: https://reviews.freebsd.org/D22942 The speedup is about factor 6! >I've received some feedback in a private mail, a second version that adds TPR >thresholds can be found in my Github branch here: > https://github.com/Yamagi/freebsd/commits/wip/tpr_shadowing >A backport to 12.1 (the branch also includes the Intel SpeedShift patches from >https://reviews.freebsd.org/D18028) is here: >>https://github.com/Yamagi/freebsd/commits/production/12.1 >I've applied it to 2 of my production servers about 4 hours ago. Looks good so >far. I'll update the review when I'm sure that it doesn't break anything, >maybe >early next week. Thanks Yamagi, That's brilliant and provides me some hope that I may not have to completely abandon windows guests after all... These servers are just sat next to my desk in testing at the moment so I'll see if I can have a go at applying the patches and testing it in the morning. Regards, Matt _______ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
RE: bhyve issues on Dell C6220 node
On Thu, 9 Jan 2020 at 08:37, Rodney W. Grimes mailto:freebsd-...@gndrsh.dnsmgr.net>> wrote: > > Loading kernel... > /boot/kernel/kernel text=0x168fdf1 data=0x1d0a68+0x768d80 > syms=[0x8+0x178bc0+0x8+0x1969d5] > Loading configured modules... > can't find '/boot/entropy' > / > " > > I get either a "|" or "/" character, then nothing else. My experience has been when I see this it is a "wrong console" issue, ie the kernel has decided to use something else for a console and your output is going there. It might be your setup using a UEFI with a fb console up to the end of the loader, then the kernel decides it is using a serial console. Or vice versa. >Rod is correct. I came across this issue in 12.0 and probably is also in 11.3 >by now. See here: https://wiki.freebsd.org/bhyve/UEFI for some tips that may >help you work out where your console is >redirecting. >I use UEFI for all guest operating systems with a custom UEFI loader that has >been fixed to handle OpenBSD (as you know :-) ). Thanks guys. I have managed to get FreeBSD to boot using the UEFI loader with boot_serial=no. Shame that I can’t get it to boot with bhyveload as it seems a more streamlined boot process and allowed me to easily access the main console directly from ssh. One thing I have now noticed is that it will boot when accessing the console via an nmdm device on com1. However, if I try and boot bhyve in the foreground using stdio (I usually just run it in a tmux window as I find it far more flexible than nmdm/cu), it stops, so clearly does seem to be a console issue. I can’t however find any loader setting that will fix it. > > I've also tested a windows server 2016 guest, and while that will actually > install via UEFI, it is noticeably slow - over a minute to boot to the login > screen and everything crawls along > >2016 is slow (even slower doing windows updates). 2019 is much better. A tip >I have found is a minimum of 1 cpu, 2 cores and 4 threads to get decent speed >from that OS (more CPUs in bhyve >tends to make performance worse - in my >observations - in 2016). Also, use the Virtio collection from RedHat for >vionet and viostor. We are currently using 0.1.171 without issue. The ahci >>emulation in itself is extremely slow. NVMe and virtio is really the only >way to go. Is there anything else I can check here? I haven’t got round to testing networking yet but I’m using nvme for the disk. It’s basically unusable and there is no way I could put anything production on it. Just highlighting an icon on the desktop takes several seconds. Matt ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
bhyve issues on Dell C6220 node
Hello, I've recently got hold of some Dell C6220 systems (2 servers in a single chassis) while I was hoping to use for bhyve The basic spec of a single server is as follows - Xeon(R) CPU E5-2670 (VT-x: PAT,HLT,MTF,PAUSE,EPT,UG,VPID) 64GB DDR3 LSI MegaRaid 9265-8i FreeBSD 12.1 I had a bit of trouble getting the disks configured and booting (**probably unrelated but see below for the details). However, my main problem at the moment is that booting either 11.3 or 12.1 in a bhyve guest (using bhyveload) gets as far as starting to boot from the install ISO, then just hangs " Loading kernel... /boot/kernel/kernel text=0x168fdf1 data=0x1d0a68+0x768d80 syms=[0x8+0x178bc0+0x8+0x1969d5] Loading configured modules... can't find '/boot/entropy' / " I get either a "|" or "/" character, then nothing else. I've also tested a windows server 2016 guest, and while that will actually install via UEFI, it is noticeably slow - over a minute to boot to the login screen and everything crawls along I'm at a loss at the moment. My test machine for years has been an old i3-2100 system and that has always booted freebsd & windows guests fairly well I'd be grateful if anyone has any ideas or insight, or whether I just need to scrap the whole idea and either switch to a completely different hypervisor (which I'd really rather not do as I know FreeBSD very well and was planning on making use of ZFS for backups/migration/etc) or find some different systems **Regarding the disks, I didn't really want a raid controller and have ended up creating a jbod (RAID0) volume for each disk, which appears as mfidX. This seems to work fine, although at first I tried to create a single pool of 2 mirrored pairs (4 disks). This failed to boot due to zfs block i/o errors on boot. In the end I had to create a separate boot mirror across 2 partitions, the create a second pool for actual data storage. Regards, Matt Churchyard ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
RFC: in-guest CPU tick counters
I have written a patch that exposes to a sysctl the ticks spent by each CPU
executing guest code under bhyve.
This new feature differs from the existing vmm_stats code in that the existing
code expresses per-vm information on a per-virtual-cpu basis. This new patch
provides a per-physical-cpu counter of the aggregate number of ticks dedicated
to executing guest-mode code since the vmm.ko module was loaded. Following the
example of kern.cp_times, it is expressed as a new sysctl
("hw.vmm.stat.guest_ticks") rather than a system call, but I am not
particularly attached to that.
The diffs may be seen here:
https://github.com/freebsd/freebsd/compare/master...RobCrowston:vmm-host-stats
I am looking for comments. If the general idea meets with this list's approval,
I will submit it to phabricator.
The new data are intended to be used by utilities like htop. To that end, I
have written a separate patch for htop that colours the CPU bars to express the
time spent for guest execution. After re-compiling htop, enable it with F2
("Setup") -> "Display Options" -> "Detailed CPU time
(System/IO-Wait/Hard-IRQ/Soft-IRQ/Steal/Guest)". Presently, htop only supports
this feature for Linux hosts. If the above patch is accepted, I will submit the
following patch to htop separately:
https://github.com/hishamhm/htop/compare/master...RobCrowston:freebsd-vmm-counter-1
Happy New Year,
Rob Crowston.
___
freebsd-virtualization@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to
"freebsd-virtualization-unsubscr...@freebsd.org"
RE: Re: Need X-server to run in bhyve guest
___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: AMD Secure Encrypted Virtualization - FreeBSD Status?
Clay Daniels Jr. wrote: > Simon, please do elaborate more on your implementation. I suspect you are > talking about libsecureboot? I have played with the generation of certs > with OpenSSL & LibreSSL, but libsecureboot seems to take a different > approach. Please tell us more. Yes I meant libsecureboot. You should be able to create keys and certs with OpenSSL. That's all we use, but we keep all the private keys etc isolated in signing servers. The local.trust.mk in libsecureboot leverages the sign.py etc described at http://www.crufty.net/sjg/blog/signing-server.htm (which also contains a link to the src) But that does not alter the fact that the certs are simply those created by an OpenSSL based CA - there are a number of good tutorials on the net on how to setup such things. With all that said; you may find it more useful to use OpenPGP for signing we again use sign.py to retrieve OpenPGP public key, but you can do all you need using nothing more than gpg For an embedded vendor like Juniper X.509 makes a lot of sense. For an individual or small scale, OpenPGP is likely simpler. libsecureboot supports both, but you need to tailor local.trust.mk to suit. IIRC you can have local.trust.mk simply set TA_PEM_LIST etc to paths of pre-prepared pem files containing your trust anchors and ta.h and/or TA_ASC_LIST to a list of .asc files containing ascii armored openpgp trust anchors. BTW in current boot1.efi is no more, loader.efi is used. [I'm still mucking about trying to get a VM image booting using efi...] > > Clay > > On Mon, Oct 14, 2019 at 1:52 PM Simon J. Gerraty via freebsd-security < > freebsd-secur...@freebsd.org> wrote: > > > Tomasz CEDRO wrote: > > > > > would be really nice also to get UEFI BOOT compatible with SECURE BOOT > > :-) > > > > Unless you are using your own BIOS, the above means getting Microsoft > > to sign boot1.efi or similar. Shims that simply work around lack of > > acceptible signature don't help. > > > > That would need to then verify loader.efi - which can be built to > > to verify all the modules and kernel. > > > > In my implementation (uses the non efi loader) trust anchors are > > embedded in loader but there is code in current to lookup trust anchors > > in /efi I think which would be more generally useful - I've not looked > > at the attack vectors that introduces though. > > > > --sjg > > ___ > > freebsd-secur...@freebsd.org mailing list > > https://urldefense.com/v3/__https://lists.freebsd.org/mailman/listinfo/freebsd-security__;!8WoA6RjC81c!TLaVmP78NH0BviSHHV_3_V0-ispe2o0I7E59vmxZ_8XnbmOYxeHxemscoWsaXA$ > > > > To unsubscribe, send any mail to "freebsd-security-unsubscr...@freebsd.org > > " > > ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: AMD Secure Encrypted Virtualization - FreeBSD Status?
Tomasz CEDRO wrote: > would be really nice also to get UEFI BOOT compatible with SECURE BOOT :-) Unless you are using your own BIOS, the above means getting Microsoft to sign boot1.efi or similar. Shims that simply work around lack of acceptible signature don't help. That would need to then verify loader.efi - which can be built to to verify all the modules and kernel. In my implementation (uses the non efi loader) trust anchors are embedded in loader but there is code in current to lookup trust anchors in /efi I think which would be more generally useful - I've not looked at the attack vectors that introduces though. --sjg _______ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: Centos 8, Centos Stream and XFS
On Sun, Oct 06, 2019 at 12:12:13PM -0400, Allan Jude wrote: > On 2019-09-29 09:54, The Doctor via freebsd-virtualization wrote: > > Are we able to solve those 3 problems? > > > > If you boot using UEFI or UEFI-CSM you don't need bhyve-grub to support > the newer filesystem stuff, as it will use CentOS's verion of grub > installed inside the image. > Not the case. Oracle 8, Centos 8 and Centos Stream uses something called dracut pre-trigger hook wating to connct to lldpad Using #!/bin/sh /usr/sbin/bhyve -ASHP \ -s 0:0,hostbridge \ -s 31:0,lpc \ -s 2:0,virtio-net,tap2 \ -s 3:0,virtio-blk,/usr/vm/images/centos8 \ -s 4:0,ahci-cd,/usr/vm/.iso/CentOS-8-x86_64-1905-dvd1.iso \ -c 4 -m 512M \ -s 29,fbuf,tcp=0.0.0.0:5956,w=1024,h=768,wait \ -l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd \ centos8guest & To Install Centos 8 and #!/bin/sh /usr/sbin/bhyve -ASHP -s 0:0,hostbridge -s 31:0,lpc -s 2:0,virtio-net,tap5 -s 3:0,virtio-blk,/usr/vm/images/oracle -s 4:0,ahci-cd,/usr/vm/.iso/V983280-01.iso -s 29,fbuf,tcp=0.0.0.0:5961,w=1024,h=768,wait -c 4 -m 512M -l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd oracleguest & to install Oracle Linux 8.\\ So we need bhyve to address this and the gui issue posed by Debian. > -- > Allan Jude > -- Member - Liberal International This is doctor@@nl2k.ab.ca Ici doctor@@nl2k.ab.ca Yahweh, Queen & country!Never Satan President Republic!Beware AntiChrist rising! https://www.empire.kred/ROOTNK?t=94a1f39b Look at Psalms 14 and 53 on Atheism Canada - Choose Forward on 21 Oct 2019 ! ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Centos 8, Centos Stream and XFS
Are we able to solve those 3 problems? -- Member - Liberal International This is doctor@@nl2k.ab.ca Ici doctor@@nl2k.ab.ca Yahweh, Queen & country!Never Satan President Republic!Beware AntiChrist rising! https://www.empire.kred/ROOTNK?t=94a1f39b Look at Psalms 14 and 53 on Atheism Canada - Choose Forward on 21 Oct 2019 ! ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: Severely broken bhyve console
On 2019-08-20T14:34:02 -0700 "Rodney W. Grimes" wrote: > > What it looks as if is happened is you have disconnected from > a session while it was in grub and had sent specific control > sequences, possibly even expecting scroll regions and such > set up in the terminal. Now when you reconnect from a newly > initialzed terminal that has none of this setup things go > very wrong. Interesting. I don't remember this happening, but it's not inconceivable that it happened and I didn't notice it. What is kind of strange is that I appear to be partly in the grub prompt and partly in the Debian installer; I see output and responses from both interleaved. > One solution is to use the graphics console, that does not suffer > from these tty type issues, its a bit sluggish if you do not have > a good high speed network path to the bhyve host though. It's a 1Gbps LAN, so I assume that's fast enough. However: Is there some way I can reset the nmdm device? -- Mark Raynsford | http://www.io7m.com pgp2RegH4pIow.pgp Description: OpenPGP digital signature
Severely broken bhyve console
Hello! For reasons I'm unable to explain, my ability to access the nmdm consoles of bhyve guests has suddenly broken. It may have been caused by a recent 11.2 update - I keep this machine on 11.2-RELEASE plus freebsd-update patches but, as it's not a publically accessible machine, I rarely reboot for updates unless absolutely necessary. I can use "sudo cu -l /dev/nmdm*B" to connect to a particular bhyve guest's console, but as this video shows, the console is effectively unusable: https://ataxia.io7m.com/2019/08/20/broken_terminal.ogv In that video, I connect to /dev/nmdm57B when the guest is shut down. I start the guest in a separate terminal (not pictured) and wait for it to boot into the Debian installer. I don't touch the keys until the title bar becomes visible. At that point, touching the arrow keys or the return key results in chaos. It doesn't seem to be terminal-specific; I'm ssh'd into the host machine, and have tried various different terminals and $TERM settings to no effect. It doesn't seem to be guest-specific; I'm running FreeBSD, Debian, and OpenBSD guests and they now all show this behaviour. The guests themselves don't appear to be in any distress, I can ssh into the guests and everything is working correctly. The host: # uname -a FreeBSD cranberry.int.arc7.info 11.2-RELEASE-p13 FreeBSD 11.2-RELEASE-p13 #0: Tue Aug 6 06:41:33 UTC 2019 r...@amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC amd64 I'd appreciate any help on this. -- Mark Raynsford | http://www.io7m.com pgp8CdRuHOn5q.pgp Description: OpenPGP digital signature
Re: New video BIOS for bhyve
Great work Henrik On Thu, 1 Aug 2019 at 18:19, Rodney W. Grimes wrote: > > On 7/19/19 3:02 AM, Henrik Gulbrandsen wrote: > > > In the last few months, I have been extending the video support in > bhyve > > > to allow booting from live ISO images, which sometimes lack UEFI > > > loaders. > > > There was already a video BIOS with dormant support for VGA, but I > could > > > not use that, because the X11 int10 module runs a real-mode x86 > emulator > > > that aborts when it tries to enter the C code running in protected > mode. > > > > > > Instead, I added support for PCI Expansion ROMs to bhyve and wrote a > new > > > SVGA video BIOS from scratch in 8086 assembly code. The current > features > > > include VGA modes 3 and 12h, 24-bit and 32-bit VESA modes from 640x480 > > > to > > > 1920x1200, and enough BIOS functions to boot a typical Linux desktop. > > > > > > At this point, I think it's good enough for a preliminary release, so > if > > > you are interested, you will find all patch files at the following URL: > > > > > > https://www.gulbra.net/freebsd-bhyve/ > > > > > > My apologies for the 15-year-old web layout, which will probably break > > > horribly on modern smartphones. Also, I hope I have not duplicated the > > > work of someone else. I didn't want to announce this before it worked. > > > > This is pretty impressive work! Would you be able to post these patches > to > > a public repository, e.g. on github or gitlab or some such? That would > > really help with review and allow others to assist with the work as well. > > I let this fall through my cracks as well while busy with > other things. I too would like to see this work moved > forward. I see that your patches are against an 11.3 > system, and can appreciate that not everone has resources > to run -head, how can we assist in getting these patches > moved forward to head? > > Regards, > -- > Rod Grimes > rgri...@freebsd.org > _______ > freebsd-virtualization@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > To unsubscribe, send any mail to " > freebsd-virtualization-unsubscr...@freebsd.org" > ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: New video BIOS for bhyve
Fantastic! > Instead, I added support for PCI Expansion ROMs This was one of the limitation that prevents GPU pass through from working, so this is also a big step forwards. On Fri, Jul 19, 2019 at 11:02, Henrik Gulbrandsen wrote: > In the last few months, I have been extending the video support in bhyve > to allow booting from live ISO images, which sometimes lack UEFI > loaders. > There was already a video BIOS with dormant support for VGA, but I could > not use that, because the X11 int10 module runs a real-mode x86 emulator > that aborts when it tries to enter the C code running in protected mode. > > Instead, I added support for PCI Expansion ROMs to bhyve and wrote a new > SVGA video BIOS from scratch in 8086 assembly code. The current features > include VGA modes 3 and 12h, 24-bit and 32-bit VESA modes from 640x480 > to > 1920x1200, and enough BIOS functions to boot a typical Linux desktop. > > At this point, I think it's good enough for a preliminary release, so if > you are interested, you will find all patch files at the following URL: > > https://www.gulbra.net/freebsd-bhyve/ > > My apologies for the 15-year-old web layout, which will probably break > horribly on modern smartphones. Also, I hope I have not duplicated the > work of someone else. I didn't want to announce this before it worked. > > /Henrik > > ___ > freebsd-virtualization@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > To unsubscribe, send any mail to > "freebsd-virtualization-unsubscr...@freebsd.org" _______ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Debian 10, Current Kali and Current Parrot
Looks like Deb 10 uses a new grpahical driver that breaks bhyve. Same with Kali and PArrot. I have been able to get around Kali and Deb 10 via predefining the network, which does not work with Parrot. Anyone on top of this? -- Member - Liberal International This is doctor@@nl2k.ab.ca Ici doctor@@nl2k.ab.ca Yahweh, Queen & country!Never Satan President Republic!Beware AntiChrist rising! https://www.empire.kred/ROOTNK?t=94a1f39b Look at Psalms 14 and 53 on Atheism We don't know who we are until we see what we can do. -Martha Grimes _______ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
RE: VM Switch broken after update on 20190507
>I just updated my bhyve server running CURRENT up to r349133 and can't start >my network switch. I suspect that this problem has a root cause in the kernel >configuration change made on 20190507 that created if_tuntap to replace the >tunnel and tap devices. My vm switch has been in use for over a year and >worked fine. When booting I get the message: > /boot/kernel/if_tap.ko - unsupported file type. > I can't remove or destroy my vm switch to create a new one. I get the > message: > vm switch remove public re0 - unable to locate switch id vm destroy public - > failed to remove virtual switch > My vm_bhyve port is version 1.3.0 > I tried to locate any config files that were generated by the vm switch > create utility. I was not successful. I wanted to see if I could just > manually edit something to change if_tap to if_tuntap. What do I need to > make my vm switch work again? Hi Tom, I haven't tested this myself but there's a recent PR that was provided to fix this that hasn't made it into ports yet. https://github.com/churchers/vm-bhyve/pull/305/files Matt >Tom -- Public Keys: PGP KeyID = 0x5F22FDC1 GnuPG KeyID = 0x620836CF ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org" ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: Bhyve GPU passthrough
I think I am the most recent adventurer, but if you go back a couple of years I am only building on material I found discussed in this mailing list. I do have an nVidia GPU working on Linux under the nouveau drivet. It required some changes to the bhyve source; in particular I had to remap the 64 bit PCI BAR allocations into the lower 4 GB of the guest’s address space. I am not sure if that is a quirk of the AMD processor I’m using. I think there are a few things that prevent this from working in the general case. Probably the most serious is bhyve doesn’t support emulation or passthrough of PCI ROMs. I am working on a hack to make that work in my spare time. But I am not an official member of the project, just tinkering. :-) The nVidia Windows driver only checks the pending break enable bit on the feature flags register to see if it is virtualized (or at least that was the case about a year ago). You can readily change that to 0 in the bhyve source code. But we are some way from that yet. On Fri, Jun 14, 2019 at 20:53, Adam wrote: > There was a thread here a while back about some adventurer trying to > passthrough an Nvidia GPU which ultimately did not work. Unfortunately > that still seems to be the case. I'm using 12.x and I could not get it to > work in FreeBSD, Linux, or Windows guest. Win10 wouldn't even boot with it > passed through. Linux wouldn't load the driver(could have been me) and > FreeBSD guest loaded the driver, but I could not do anything useful with > the card. For example, nvidia-smi -q reported No devices were found. Here > is the guest view: > > vgapci0@pci0:0:1:0: class=0x03 card=0x77101558 chip=0x1f5010de > rev=0xa1 hdr=0x00 > vendor = 'NVIDIA Corporation' > device = 'TU106BM [GeForce RTX 2070 Mobile]' > class = display > subclass = VGA > bar [10] = type Memory, range 32, base 0xc000, size 16777216, > enabled > bar [14] = type Prefetchable Memory, range 64, base 0xd0, > size 268435456, enabled > bar [1c] = type Prefetchable Memory, range 64, base 0xc200, size > 33554432, enabled > bar [24] = type I/O Port, range 32, base 0x2000, size 128, enabled > cap 01[60] = powerspec 3 supports D0 D3 current D0 > cap 05[68] = MSI supports 1 message, 64 bit enabled with 1 message > cap 10[78] = PCI-Express 2 legacy endpoint max data 128(256) FLR RO NS > link x16(x16) speed 2.5(8.0) ASPM disabled(L0s/L1) > > I believe in earlier discussion there was objection to overcoming Nvidia's > virtualization restrictions, but I think since then both vmware and kvm > have implemented workarounds to allow for this. > > Is anyone one currently working on this or needs some testing done? > > -- > Adam > ___ > freebsd-virtualization@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > To unsubscribe, send any mail to > "freebsd-virtualization-unsubscr...@freebsd.org" _______ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: WebThings on bhyve and USB
> How to best approach Mozilla IoT Gateway? I cannot speak to the meat of this question but I'll respond to the bhyve-related questions. > I would need to somehow pass the USB ports to bhyve, so that Linux can > pass them to the docker container. I heard it's not possible to pass > specific USB ports but instead the whole PCI has to be passed Yes, you can pass through a whole USB controller, not its individual ports. In practice therefore you will probably need at least two USB controllers on your system, unless it is headless. > and it seems to be as complex as it sounds: I think the complexity is overstated in the article. A better (although slightly out-of-date) guide is https://wiki.freebsd.org/bhyve/pci_passthru. In particular I did not have to recompile the kernel or remove the xhci driver. As far as I remember all that is necessary was to add three lines to /boot/loader.conf: hw.vmm.amdvi.enable="1" # AMD only: delete this line if you have an Intel processor. pptdevs="10/0/3"# Change 10/0/3 to the address of your USB controller. vmm_load="YES" # Load the bhyve kernel module on boot. [There are some other caveats, for instance you may need to enable directed i/o virtualization in your BIOS.] ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: BHYVE: UEFI shims issue?
Hi, Sorry, in relation to this post, it turns out that it's the server my client has me on over 5 years old. I'm going to assume the UEFI is one of those with the 'limited memory space' allocated to UEFI. P. On Tuesday, June 4, 2019, 1:02:11 PM EDT, Paul Pathiakis via freebsd-virtualization wrote: Hi, I thought this would have been fixed by now, but I'm probably just not aware of the patch or even maybe this was never reported. I seem to be having the same UEFI shims issue with putting FreeBSD 12.0 RELEASE in BHyve on a FreeNAS 11.2-u4-1 system. On that system, I can't load anything the has a problem in the boot loader. This is both CentOS 7 1810 and later and FreeBSD 12.0. I can load and install both CentOS 7 1805 and FreeBSD 11.2 and everythign is fine. Nything produced after that on either has a problem with UEFI shims in /boot/UEFI . Is this being worked on? Is there a workaround? Thank you! P. _______ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org" ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
BHYVE: UEFI shims issue?
Hi, I thought this would have been fixed by now, but I'm probably just not aware of the patch or even maybe this was never reported. I seem to be having the same UEFI shims issue with putting FreeBSD 12.0 RELEASE in BHyve on a FreeNAS 11.2-u4-1 system. On that system, I can't load anything the has a problem in the boot loader. This is both CentOS 7 1810 and later and FreeBSD 12.0. I can load and install both CentOS 7 1805 and FreeBSD 11.2 and everythign is fine. Nything produced after that on either has a problem with UEFI shims in /boot/UEFI . Is this being worked on? Is there a workaround? Thank you! P. _______ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: Windows 10 guests fail to boot when attempting to passthrough network card
Does the card have multiple BARs? Can you send us the output of pciconf -lvb
for the card?
There is a bug in bhyve where if the OS assigns the 0x sentinel to more
than one BAR simultaneously it is not compatible with bhyve’s memory model for
the guest.
On Sat, May 18, 2019 at 21:14, Nick Principe wrote:
> Yeah, that triggers a barf with error 2:
> mmio_rb_lookup: barf ENOENTerror is 2
> mr.name is passthru-pci-5
> mr.flags is 0
> mr.arg2 is 0
> mr.base is 18446744073709027328
> mr.size is 524288
> Assertion failed: (error == 0), function modify_bar_registration, file
> /usr/src/usr.sbin/bhyve/pci_emul.c, line 510.
>
> Interestingly enough, after we talked I simply commented out the assert at
> 504 in pci_emul.c and Windows booted fine and can see the Chelsio card with
> both interfaces. I'm not sure if it would fall over once I started actually
> using it or not.
>
> `nap
>
> On Sat, May 18, 2019 at 4:06 PM Rodney W. Grimes <
> freebsd-...@gndrsh.dnsmgr.net> wrote:
>
>> > I have noticed that Windows 10 guests fail to boot when attempting to
>> pass
>> > through a network card. I believe I have observed this with both cxgbe
>> > (t580) and mlx5en cards, but only have a cxgbe to test with now. Without
>> > passthrough, the Windows 10 guest boots and operates normally.
>> >
>> > FreeBSD guests (12.0-RELEASE) have no issue when booting with the cxgbe
>> > card passed through - I can kldload cxgbe and I get both cxl ports
>> showing
>> > up in the FreeBSD guest.
>> >
>> > I have tested this with both 12.0-RELEASE and head (13-CURRENT r347883)
>> as
>> > the host OS with no change in behavior. The bhyve output is:
>> > Unhandled ps2 keyboard command 0x02
>> > Unhandled ps2 keyboard command 0x02
>> > Assertion failed: (error == 0), function modify_bar_registration, file
>> > /usr/src/usr.sbin/bhyve/pci_emul.c, line 504.
>> > fbuf frame buffer base: 0x94360 [sz 16777216]
>> >
>> > Two main suggestions from discussions at BSDCan this week were:
>> > - Capture pciconf -lvb from the FreeBSD guest
>> > - Add some printf to pci_emul.c to capture some values when there is an
>> > error
>> >
>> > I've captured the above, and a lot of other relevant info, in a Google
>> Doc
>> > here (too big to post directly):
>> >
>> https://docs.google.com/document/d/1t-UVIO9Aq0TPUFHyo1nVscqaW1LoPuNhfLPitL8oeTs/edit?usp=sharing
>> >
>> > `nap
>>
>> To confirm what I see from looking at your data could you tell me if this
>> patch triggers a barf?
>>
>> --- mem.c.orig 2019-05-18 20:04:26.707995000 +
>> +++ mem.c 2019-05-18 20:04:02.205119000 +
>> @@ -97,6 +97,7 @@
>> return (0);
>> }
>>
>> +printf("mmio_rb_lookup: barf ENOENT");
>> return (ENOENT);
>> }
>>
>> --
>> Rod Grimes
>> rgri...@freebsd.org
>>
>
> --
> Nick Principe
> Performance Engineering Supervisor
> iXsystems, Inc.
> Ph: (408) 943-4100 x341
> Fx: (408) 943-4101
> ___
> freebsd-virtualization@freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
> To unsubscribe, send any mail to
> "freebsd-virtualization-unsubscr...@freebsd.org"
___
freebsd-virtualization@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to
"freebsd-virtualization-unsubscr...@freebsd.org"
Unsubscribe
> On 9 May 2019, at 13:00, freebsd-virtualization-requ...@freebsd.org wrote: > > Send freebsd-virtualization mailing list submissions to >freebsd-virtualization@freebsd.org > > To subscribe or unsubscribe via the World Wide Web, visit >https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > or, via email, send a message with subject or body 'help' to >freebsd-virtualization-requ...@freebsd.org > > You can reach the person managing the list at >freebsd-virtualization-ow...@freebsd.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of freebsd-virtualization digest..." > > > Today's Topics: > > 1. [Bug 237636] bhyve guest ZFS filesystem freezes in > zcw->zcw_cv state (bugzilla-nore...@freebsd.org) > 2. [maybe OT] Strange behaviour when connecting to serial port > of a FreeBSD inside a VM (Joachim Durchholz) > 3. [Bug 231117] bhyve: I/O lockups inside guests > (bugzilla-nore...@freebsd.org) > 4. [Bug 231117] bhyve: I/O lockups inside guests > (bugzilla-nore...@freebsd.org) > > > -- > > Message: 1 > Date: Thu, 09 May 2019 01:53:57 + > From: bugzilla-nore...@freebsd.org > To: virtualizat...@freebsd.org > Subject: [Bug 237636] bhyve guest ZFS filesystem freezes in >zcw->zcw_cv state > Message-ID: > > Content-Type: text/plain; charset="UTF-8" > > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=237636 > > Mark Linimon changed: > > What|Removed |Added > > Assignee|b...@freebsd.org|virtualizat...@freebsd.org > > -- > You are receiving this mail because: > You are the assignee for the bug. > > -- > > Message: 2 > Date: Thu, 9 May 2019 08:52:42 +0200 > From: Joachim Durchholz > To: freebsd-virtualization@freebsd.org > Subject: [maybe OT] Strange behaviour when connecting to serial port >of a FreeBSD inside a VM > Message-ID: <2c875da4-741b-3601-dd38-b19145049...@durchholz.org> > Content-Type: text/plain; charset=utf-8; format=flowed > > Sorry if this off-topic, please direct me to the correct list. > > I have FreeBSD running inside a VirtualBox VM, and while FreeBSD boots > and works fine and I could successfully hook up a host's domain socket, > I don't see output on the socket. > SOMETHING is working: Some boot messages are redirected, though to a > place that I do not see. > > I have a shell script that recreates the entire situation from scratch, > I can describe everything in excruciating detail - where do I start? > > Regards, > Jo > > > -- > > Message: 3 > Date: Thu, 09 May 2019 07:31:01 + > From: bugzilla-nore...@freebsd.org > To: virtualizat...@freebsd.org > Subject: [Bug 231117] bhyve: I/O lockups inside guests > Message-ID: > > Content-Type: text/plain; charset="UTF-8" > > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=231117 > > --- Comment #24 from Mateusz Kwiatkowski --- > (In reply to Rodney W. Grimes from comment #23) > > Rodney, many thanks for your engagement in this matter. It seems that you were > right suspecting that arc_max is the culprit. After reducing its size virtual > machines are working stable for over a day. So, instead of stability issue > this > was more like user experience issue. As you see we both were tricked by top(1) > output which looked fine and didn't indicate problem. Is it possible to have a > notice printed in dmesg that happens (as we have for other kernel limits)? > > -- > You are receiving this mail because: > You are the assignee for the bug. > > -- > > Message: 4 > Date: Thu, 09 May 2019 09:58:01 + > From: bugzilla-nore...@freebsd.org > To: virtualizat...@freebsd.org > Subject: [Bug 231117] bhyve: I/O lockups inside guests > Message-ID: > > Content-Type: text/plain; charset="UTF-8" > > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=231117 > > --- Comment #25 from Mateusz Kwiatkowski --- > (In reply to Mateusz Kwiatkowski from comment #24) > I'm afraid I celebrated too soon. I/O in VM locked up - it just took more > time. > > Memory stats: > Mem: 3846M Active, 4806M Inact, 26G Wired, 27G Free > ARC: 7755M Total, 2419M MFU, 4208M MRU, 14M Anon, 478M Header, 635M Other > 6145M Compressed, 10G Uncom
RE: [vm-bhyve] Windows 2012 and 2016 servers guests would not stop
Victor Sudakov wrote on 2019-04-22 19:43:
...
>> And the implementation is pretty brutal:
>> # 'vm stopall'
>> # stop all bhyve instances
>> # note this will also stop instances not started by vm-bhyve #
>> core::stopall(){
>> local _pids=$(pgrep -f 'bhyve:')
>>
>> echo "Shutting down all bhyve virtual machines"
>> killall bhyve
>> sleep 1
>> killall bhyve
>> wait_for_pids ${_pids}
>> }
> yow.
>>
>> I wonder what the effect of the second kill is, that seems odd.
>
> Indeed.
> the first killall will cause each client OS to see a soft shutdown signal.
> the sleep 1 gives them some time to flush their buffers. the second killall
> says, time's up, just stop.
Both of these killall commands send a soft shutdown signal and I've never seen
an instance in my own testing where the second has caused an instant poweroff.
I've just double checked this, and a FreeBSD guest stopped with the existing
code fully shuts itself down, ending with "acpi0: Powering system off"
The main reason for having both is that in my initial testing, I could not get
Windows to respond to a single SIGINT. 100% of the time it would simply act
like nothing had happened. Sending two however triggered a shutdown. I only
have a single test machine though so maybe it's something strange about my
particular system that no one else experiences...
Matt
> i think this is worse than brutal, it's wrong. consider freebsd's own work
> flow when trying to comply with the first soft shutdown it got:
> https://github.com/freebsd/freebsd/blob/master/sbin/reboot/reboot.c#L220
> this has bitten me more than once, because using "pageins" as a proxy for "my
> server processes are busy trying to synchronize their user mode state" is
> inaccurate. i think _any_ continuing I/O should > be reason to wait the full
> 60 seconds.
> and so i think the "sleep 1" above should be a "sleep 65".
> What is needed in vm-bhyve is the feature that if ACPI does not stop
> the guest for a predefined period of time, the guest is powered off.
> i agree with this.
--
> P Vixie
_______
freebsd-virtualization@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to
"freebsd-virtualization-unsubscr...@freebsd.org"
___
freebsd-virtualization@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to
"freebsd-virtualization-unsubscr...@freebsd.org"
Centos and Debian issues
I am trying to install Centos and Debian on FreeBSD 12 . Memory warning turns up. -- Member - Liberal International This is doctor@@nl2k.ab.ca Ici doctor@@nl2k.ab.ca Yahweh, Queen & country!Never Satan President Republic!Beware AntiChrist rising! https://www.empire.kred/ROOTNK?t=94a1f39b Look at Psalms 14 and 53 on Atheism PEI on 23 April 2019, do not vote PC nor NDP! ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: Windows 2019 server
On Thu, Apr 11, 2019 at 10:10:03PM -0500, Dustin Marquess wrote:
> It's worth a shot at least to see if it works!
>
> But ya, basically the same as mine:
>
> bhyve -A -H -P -w -S -u -c sockets=1,cores=4,threads=2 -m ${RAM} \
> -s 0,hostbridge \
> -s 3,nvme,/dev/zvol/tank/vm/win2019 \
> -s 4,ahci-cd,${VMROOT}/win2019/empty.iso,nocache,ro \
> -s 5,passthru,130/0/0 \
> -s 10,virtio-net,tap2 \
> -s 20,virtio-rnd \
> -s 31,lpc \
> -l com1,stdio -l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd ${VM}
>
> -Dustin
I will try a bit of a mix. Also Do the same for the Linux emulations
such as Scientific, Fedora, Centos, Debian and Ubuntu and ORacle?
BTW rational for the -w , your -c content your passthru and the virtio-rnd?
>
> On Thu, Apr 11, 2019 at 5:54 PM The Doctor wrote:
> >
> > On Thu, Apr 11, 2019 at 05:18:10PM -0500, Dustin Marquess wrote:
> > > I'm not sure if 12.0 has bhyve nmve support, but 10 & 2019 both seem
> > > to run MUCH faster when using nvme compared to ahci-hd.
> > >
> > > -Dustin
> > >
> >
> > Here is the script I use to start the Windows server
> >
> >
> > #!/bin/sh
> > bhyve -c 4 -s 0,hostbridge -s 3,ahci-hd,/usr/vm/images/windows2019st -s
> > 10,virtio-net,tap3 -s 31,lpc -l com1,/dev/nmdm9A -l com2,/dev/nmdm10A -s
> > 29,fbuf,tcp=0.0.0.0:5958,w=1024,h=768,wait -l
> > bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd -m 4G -H -w
> > windows2019st &
> >
> > So I should change ahci-cd to nvme?
> >
> >
> > > On Thu, Apr 11, 2019 at 4:40 PM The Doctor via freebsd-virtualization
> > > wrote:
> > > >
> > > > Seems to be running slow on FreeBSD 12.0 p3.
> > > >
> > > > Just wondering if there are known issues.
> > > > --
> > > > Member - Liberal International This is doctor@@nl2k.ab.ca Ici
> > > > doctor@@nl2k.ab.ca
> > > > Yahweh, Queen & country!Never Satan President Republic!Beware
> > > > AntiChrist rising!
> > > > https://www.empire.kred/ROOTNK?t=94a1f39b Look at Psalms 14 and 53 on
> > > > Atheism
> > > > Alberta on 16 April 2019, do not vote UCP, FCP nor NDP!
> > > > ___
> > > > freebsd-virtualization@freebsd.org mailing list
> > > > https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
> > > > To unsubscribe, send any mail to
> > > > "freebsd-virtualization-unsubscr...@freebsd.org"
> >
> > --
> > Member - Liberal International This is doctor@@nl2k.ab.ca Ici
> > doctor@@nl2k.ab.ca
> > Yahweh, Queen & country!Never Satan President Republic!Beware AntiChrist
> > rising!
> > https://www.empire.kred/ROOTNK?t=94a1f39b Look at Psalms 14 and 53 on
> > Atheism
> > Alberta on 16 April 2019, do not vote UCP, FCP nor NDP!
--
Member - Liberal International This is doctor@@nl2k.ab.ca Ici doctor@@nl2k.ab.ca
Yahweh, Queen & country!Never Satan President Republic!Beware AntiChrist rising!
https://www.empire.kred/ROOTNK?t=94a1f39b Look at Psalms 14 and 53 on Atheism
Alberta on 16 April 2019, do not vote UCP, FCP nor NDP!
___
freebsd-virtualization@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to
"freebsd-virtualization-unsubscr...@freebsd.org"
Re: Windows 2019 server
On Thu, Apr 11, 2019 at 05:18:10PM -0500, Dustin Marquess wrote: > I'm not sure if 12.0 has bhyve nmve support, but 10 & 2019 both seem > to run MUCH faster when using nvme compared to ahci-hd. > > -Dustin > Here is the script I use to start the Windows server #!/bin/sh bhyve -c 4 -s 0,hostbridge -s 3,ahci-hd,/usr/vm/images/windows2019st -s 10,virtio-net,tap3 -s 31,lpc -l com1,/dev/nmdm9A -l com2,/dev/nmdm10A -s 29,fbuf,tcp=0.0.0.0:5958,w=1024,h=768,wait -l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd -m 4G -H -w windows2019st & So I should change ahci-cd to nvme? > On Thu, Apr 11, 2019 at 4:40 PM The Doctor via freebsd-virtualization > wrote: > > > > Seems to be running slow on FreeBSD 12.0 p3. > > > > Just wondering if there are known issues. > > -- > > Member - Liberal International This is doctor@@nl2k.ab.ca Ici > > doctor@@nl2k.ab.ca > > Yahweh, Queen & country!Never Satan President Republic!Beware AntiChrist > > rising! > > https://www.empire.kred/ROOTNK?t=94a1f39b Look at Psalms 14 and 53 on > > Atheism > > Alberta on 16 April 2019, do not vote UCP, FCP nor NDP! > > _______ > > freebsd-virtualization@freebsd.org mailing list > > https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > > To unsubscribe, send any mail to > > "freebsd-virtualization-unsubscr...@freebsd.org" -- Member - Liberal International This is doctor@@nl2k.ab.ca Ici doctor@@nl2k.ab.ca Yahweh, Queen & country!Never Satan President Republic!Beware AntiChrist rising! https://www.empire.kred/ROOTNK?t=94a1f39b Look at Psalms 14 and 53 on Atheism Alberta on 16 April 2019, do not vote UCP, FCP nor NDP! ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Windows 2019 server
Seems to be running slow on FreeBSD 12.0 p3. Just wondering if there are known issues. -- Member - Liberal International This is doctor@@nl2k.ab.ca Ici doctor@@nl2k.ab.ca Yahweh, Queen & country!Never Satan President Republic!Beware AntiChrist rising! https://www.empire.kred/ROOTNK?t=94a1f39b Look at Psalms 14 and 53 on Atheism Alberta on 16 April 2019, do not vote UCP, FCP nor NDP! ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: GPU passthrough: mixed success on Linux, not yet on Windows
That’s good to know, I wasn’t aware of that.
Separately I have found a number of GPUs taint their ROMs after they
initialization, preventing reinitialization. There is a related problem too
with video BIOS shadowing if you want to use the primary GPU for pass through.
I am not sure if the D3 reset can fix that.
I understand that qemu has an option to present a pre-captured pristine ROM
from file as the GPU’s ROM to the guest. I am looking at whether this is a
possible approach in bhyve.
On Tue, Apr 9, 2019 at 15:40, Nick Wolff wrote:
> Robert,
>
> I'm hoping that the set of commits done for
> https://reviews.freebsd.org/D19646 will fix the pcie reset problems.
> Apparently pcie wasn't auto retraining after reset. I don't know if connected
> to that review but wanted to let you know.
>
> On Wed, Apr 3, 2019 at 7:42 PM Robert Crowston via freebsd-virtualization
> wrote:
>
>> To get Windows to boot I think the only hacks I needed were in bhyve/mem.c
>> (this is not production ready!)
>>
>> Diff'd against 12.0-release
>>
>> -- /tmp//sh-np.vFXFJd 2019-04-04 00:29:32.75299 +0100
>> +++ mem.c 2019-03-02 22:27:14.500906000 +
>> @@ -101,20 +101,22 @@
>> }
>>
>> static int
>> -mmio_rb_add(struct mmio_rb_tree *rbt, struct mmio_rb_range *new)
>> +mmio_rb_add(struct mmio_rb_tree *rbt, struct mmio_rb_range *new_element)
>> {
>> struct mmio_rb_range *overlap;
>>
>> - overlap = RB_INSERT(mmio_rb_tree, rbt, new);
>> + overlap = RB_INSERT(mmio_rb_tree, rbt, new_element);
>>
>> + printf("mmio_rb_add: %lx:%lx ", new_element->mr_base,
>> new_element->mr_end);
>> +
>> if (overlap != NULL) {
>> -#ifdef RB_DEBUG
>> - printf("overlap detected: new %lx:%lx, tree %lx:%lx ",
>> - new->mr_base, new->mr_end,
>> +//#ifdef RB_DEBUG
>> + printf("overlap detected: new_element %lx:%lx, tree %lx:%lx
>> ",
>> + new_element->mr_base, new_element->mr_end,
>>overlap->mr_base, overlap->mr_end);
>> -#endif
>> +//#endif
>>
>> - return (EEXIST);
>> +// return (EEXIST);
>> }
>>
>> return (0);
>> @@ -336,6 +338,8 @@
>> assert((mr->flags & MEM_F_IMMUTABLE) == 0);
>> RB_REMOVE(mmio_rb_tree, &mmio_rb_root, entry);
>>
>> + printf("unregister: %lx:%lx ", mr->base, mr->base+mr->size);
>> +
>> /* flush Per-vCPU cache */
>> for (i=0; i < VM_MAXCPU; i++) {
>> if (mmio_hint[i] == entry)
>> @@ -348,7 +352,12 @@
>> if (entry)
>> free(entry);
>>
>> - return (err);
>> + if (err)
>> + fprintf( stderr, "Unregister mem errno %d for range %lx:%lx.
>> ", err,
>> + memp->base, memp->base + memp->size );
>> +
>> + return 0;
>> + //return (err);
>> }
>>
>> ‐‐‐ Original Message ‐‐‐
>> On Thursday, 28 March 2019 22:02, Ruslan Bukin wrote:
>>
>>> Hi Robert:
>>>
>>> On Sun, Mar 17, 2019 at 04:22:29PM +, Robert Crowston via
>>> freebsd-virtualization wrote:
>>>
>>> > Is it worth me continuing to hack away at these problems---of course I'm
>>> > happy to share anything I come up with---or is there an official solution
>>> > to GPU support in the pipe about to make my efforts redundant :)?
>>>
>>> Could you share your patch/hacks somewhere?
>>> I would like to try it with AMD graphics card and Windows.
>>>
>>> Ruslan
>>
>> ___
>> freebsd-virtualization@freebsd.org mailing list
>> https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
>> To unsubscribe, send any mail to
>> "freebsd-virtualization-unsubscr...@freebsd.org"
___
freebsd-virtualization@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to
"freebsd-virtualization-unsubscr...@freebsd.org"
unsubscribe
> On 7 Apr 2019, at 13:00, freebsd-virtualization-requ...@freebsd.org wrote: > > Send freebsd-virtualization mailing list submissions to > freebsd-virtualization@freebsd.org > > To subscribe or unsubscribe via the World Wide Web, visit > https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > or, via email, send a message with subject or body 'help' to > freebsd-virtualization-requ...@freebsd.org > > You can reach the person managing the list at > freebsd-virtualization-ow...@freebsd.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of freebsd-virtualization digest..." > > > Today's Topics: > > 1. Re: running FreePBX SNG7 Official Distro (Victor Sudakov) > 2. Re: running FreePBX SNG7 Official Distro (Victor Sudakov) > 3. Re: running FreePBX SNG7 Official Distro (Rodney W. Grimes) > 4. Re: running FreePBX SNG7 Official Distro (Rodney W. Grimes) > 5. Re: running FreePBX SNG7 Official Distro (Victor Sudakov) > 6. Re: running FreePBX SNG7 Official Distro (Victor Sudakov) > > > ------ > > Message: 1 > Date: Sun, 7 Apr 2019 09:37:43 +0700 > From: Victor Sudakov > To: freebsd-virtualization@freebsd.org > Subject: Re: running FreePBX SNG7 Official Distro > Message-ID: <20190407023743.gb99...@admin.sibptus.ru> > Content-Type: text/plain; charset="us-ascii" > > Rodney W. Grimes wrote: >>>> >>>> You can usually use the host by doing mdconfig -f >>> >>> Unfortunately mdconfig does not work with zvols: >>> >>> root@vas:~ # mdconfig -a -f /dev/zvol/d02/vm/freepbx/disk0 >>> mdconfig: /dev/zvol/d02/vm/freepbx/disk0 is not a regular file >> >> If its a zvol cant you just do >> gpart show /dev/zvol/d02/vm/freepbx/disk0 >> >> and >> mount -t msdosfs /dev/zvol/d02/vm/freepbx/disk0p2 > > No I can't if the zvol is in the "volmode=dev" mode which is the default. > > This is the default for a reason: it's no good exposing scores of always > coming and going guest geoms to the host system. I think you can even > get a conflict of labels or something like that one day. > >>>>> Moreover, I waited (for a long time!) for the EFI interactive shell >>>>> prompt and with a few commands: >>>> >>>> Yes, the timeout is very long, and I do not know that we >>>> document anyplace that if you wait long enough at a failed >>>> boot you do get a EFI shell prompt eventually. >>> >>> Can I press some key to escape to the EFI shell? >> Not that I am aware of. > > It's a major problem! There must be a well-known way to break the boot > sequence any time and enter the EFI shell. > >>>>> I can guess that it looks for a FAT16 partition in the GPT with the type >>>>> "efi" but the rest is a mystery for me. Why is it trying to find >>>>> "grubx64.efi" and not the default "boot64.efi" (which is present), for >>>>> example? >>>> >>>> I suspect that what ever guest you installed installed something >>>> else someplace, either within the eft partition, or possibly in >>>> the MBR? >>> >>> Do you mean to say, the guest installing something else someplace can >>> influence the boot sequence of bhyve efi? >> >> The guest created all of the bits on that zvol, >> it can influence many things. There is probably a tiny initial >> stub that efi loads that has this bath to grubx64.efi codded in >> it and that is what is causing this issue. > > It is very important to find and debug it because Oracle VirtualBox in > UEFI mode installs and runs this guest just fine. So it must be some > issue in bhyve itself. > > Here is the complete archive of everything the guest created in the EFI > partition: http://admin.sibptus.ru/~vas/freepbx.tar.gz > can you find those confusing bits? > > The standard procedure should be as follows: > > Automated detection relies on standardized file paths to the OS > loader, with the path varying depending on the computer architecture. > The format of the file path is defined as > /EFI/BOOT/BOOT.EFI; for > example, the file path to the OS loader on an x86-64 system is > /efi/BOOT/BOOTX64.EFI and efi\boot\bootaa64.efi on ARM64 architecture. > > Nothing about grub*.efi. But only bhyve is confused, VirtualBox is not. > > -- > Victor Sudakov, VAS4-RIPE, VAS47-RIPN > 2:5005/4
Bhyve with UEFI edk2-stable201903 and CSM support - debugging the crash
I've spent some time trying to figure out where and why the UEFI 2.70 firmware is crashing when built with CSM support under Bhyve. The first thing I noticed is that the code that builds the E820 table doesn't appear to be finding the location of the EBDA at 0x40E: the first entry should be something like 0x0 to 0x4cf00, but is 0-0 instead. E820[ 0]: 0x 0 0x 0, Type = 0x1 E820[ 1]: 0x 0 0x A, Type = 0x2 E820[ 2]: 0x E 0x 10, Type = 0x2 E820[ 3]: 0x 10 0x 406000, Type = 0x1 ... As an aside, I'm working on a patch to submit upstream to fix the output so it pads with zeros instead of spaces. I've also found that as was probably pretty obvious the system is crashing when trying to call into the CSM to initialize it. It gets as far as calling AsmThunk16 (AsmThunk.nasm), but never appears to reach CsmMain in BhyveCsm16.c. I'll see if I can work on it some more to narrow down the error. Another thing: the GNUmakefile in OvmfPkg/Csm/BhyveCsm16 has "CC ?= gcc", but it appears the build system already sets CC to "cc", which uses the system's clang compiler, which can't compile the assembly code. I don't know if we might want to either hard-code CC to "gcc" for now, or have users create a 'cc' symlinks in BaseTools/Bin/FreeBSD-amd64 ? -- Rebecca Cran ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: GPU passthrough: mixed success on Linux, not yet on Windows
To get Windows to boot I think the only hacks I needed were in bhyve/mem.c
(this is not production ready!)
Diff'd against 12.0-release
-- /tmp//sh-np.vFXFJd 2019-04-04 00:29:32.75299 +0100
+++ mem.c 2019-03-02 22:27:14.500906000 +
@@ -101,20 +101,22 @@
}
static int
-mmio_rb_add(struct mmio_rb_tree *rbt, struct mmio_rb_range *new)
+mmio_rb_add(struct mmio_rb_tree *rbt, struct mmio_rb_range *new_element)
{
struct mmio_rb_range *overlap;
- overlap = RB_INSERT(mmio_rb_tree, rbt, new);
+ overlap = RB_INSERT(mmio_rb_tree, rbt, new_element);
+ printf("mmio_rb_add: %lx:%lx\n", new_element->mr_base,
new_element->mr_end);
+
if (overlap != NULL) {
-#ifdef RB_DEBUG
- printf("overlap detected: new %lx:%lx, tree %lx:%lx\n",
- new->mr_base, new->mr_end,
+//#ifdef RB_DEBUG
+ printf("overlap detected: new_element %lx:%lx, tree %lx:%lx\n",
+ new_element->mr_base, new_element->mr_end,
overlap->mr_base, overlap->mr_end);
-#endif
+//#endif
- return (EEXIST);
+// return (EEXIST);
}
return (0);
@@ -336,6 +338,8 @@
assert((mr->flags & MEM_F_IMMUTABLE) == 0);
RB_REMOVE(mmio_rb_tree, &mmio_rb_root, entry);
+ printf("unregister: %lx:%lx\n", mr->base, mr->base+mr->size);
+
/* flush Per-vCPU cache */
for (i=0; i < VM_MAXCPU; i++) {
if (mmio_hint[i] == entry)
@@ -348,7 +352,12 @@
if (entry)
free(entry);
- return (err);
+ if (err)
+ fprintf( stderr, "Unregister mem errno %d for range
%lx:%lx.\n", err,
+ memp->base, memp->base + memp->size );
+
+ return 0;
+ //return (err);
}
‐‐‐ Original Message ‐‐‐
On Thursday, 28 March 2019 22:02, Ruslan Bukin wrote:
> Hi Robert:
>
> On Sun, Mar 17, 2019 at 04:22:29PM +, Robert Crowston via
> freebsd-virtualization wrote:
>
> > Is it worth me continuing to hack away at these problems---of course I'm
> > happy to share anything I come up with---or is there an official solution
> > to GPU support in the pipe about to make my efforts redundant :)?
>
> Could you share your patch/hacks somewhere?
> I would like to try it with AMD graphics card and Windows.
>
> Ruslan
_______
freebsd-virtualization@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to
"freebsd-virtualization-unsubscr...@freebsd.org"
Re: GPU passthrough: mixed success on Linux, not yet on Windows
> The hack I had to make: I found that many instructions to access
> memory-mapped PCI BARs are not being executed on the CPU in guest mode but
> are being passed back for emulation in the hypervisor.
Update on this: I found that by mapping the BARs within the lower 4 GB of the
guest's address space I am able to start X under Linux without other weird
hacks. The size of the BAR that was causing difficulty is 128 MB. The
vm_map_pptdev_mmio() call apparently succeeds in mapping the memory range but
the processor then faults on any guest instruction that accesses this range.
You can change the memory region either by bumping the size of a "small
request" (initially 32 MB) in bhyve/pci_emul.c around line 638, or by tweaking
the value of the macro constant PCI_EMUL_MEMBASE64 in the same file. I also
tried setting PCI_EMUL_MEMBASE64 to other low values (like 32 GB) but seems
like it has to be below 4 GB for Linux to be happy.
‐‐‐ Original Message ‐‐‐
On Sunday, 17 March 2019 16:22, Robert Crowston via freebsd-virtualization
wrote:
> Hi folks, this is my first post to the group. Apologies for length.
>
> I've been experimenting with GPU passthrough on bhyve. For background, the
> host system is FreeBSD 12.0-RELEASE on an AMD Ryzen 1700 CPU @ 3.8 GHz, 32 GB
> of ECC RAM, with two nVidia GPUs. I'm working with a Linux Debian 9 guest and
> a Windows Server 2019 (desktop experience installed) guest. I also have a USB
> controller passed-through for bluetooth and keyboard.
>
> With some unpleasant hacks I have succeeded in starting X on the Linux guest,
> passing-through an nVidia GT 710 under the nouveau driver. I can run the
> "mate" desktop and glxgears, both of which are smooth at 4K. The Unity Heaven
> benchmark runs at an embarrassing 0.1 fps, and 2160p x264 video in VLC runs
> at about 5 fps. Neither appears to be CPU-bound in the host or the guest.
>
> The hack I had to make: I found that many instructions to access
> memory-mapped PCI BARs are not being executed on the CPU in guest mode but
> are being passed back for emulation in the hypervisor. This causes an
> assertion to fail inside passthru_write() in pci_passthru.c
> ["pi->pi_bar[baridx].type == PCIBAR_IO"] because it does not expect to
> perform memory-mapped IO for the guest. Examining the to-be-emulated
> instructions in vmexit_inst_emul() {e.g., movl (%rdi), %eax}, they look
> benign to me, and I have no explanation for why the CPU refused to execute
> them in guest mode.
>
> As an amateur work-around, I removed the assertion and instead I obtain the
> desired offset into the guest's BAR, calculate what that guest address
> translates to in the host's address space, open(2) /dev/mem, mmap(2) over to
> that address, and perform the write directly. I do a similar trick in
> passthru_read(). Ugly, slow, but functional.
>
> This code path is accessed continuously whether or not X is running, with an
> increase in activity when running anything GPU-heavy. Always to bar 1, and
> mostly around the same offsets. I added some logging of this event. It runs
> at about 100 lines per second while playing video. An excerpt is:
> ...
> Unexpected out-of-vm passthrough write #492036 to bar 1 at offset 41100.
> Unexpected out-of-vm passthrough write #492037 to bar 1 at offset 41100.
> Unexpected out-of-vm passthrough read #276162 to bar 1 at offset 561280.
> Unexpected out-of-vm passthrough write #492038 to bar 1 at offset 38028.
> Unexpected out-of-vm passthrough write #492039 to bar 1 at offset 38028.
> Unexpected out-of-vm passthrough read #276163 to bar 1 at offset 561184.
> Unexpected out-of-vm passthrough read #276164 to bar 1 at offset 561184.
> Unexpected out-of-vm passthrough read #276165 to bar 1 at offset 561184.
> Unexpected out-of-vm passthrough read #276166 to bar 1 at offset 561184.
> ...
>
> So my question here is,
>
> 1. How do I diagnose why the instructions are not being executed in guest
> mode?
>
> Some other problems:
>
> 2. Once the virtual machine is shut down, the passed-through GPU doesn't get
> turned off. Whatever message was on the screen in the final throes of Linux's
> shutdown stays there. Maybe there is a specific detach command which bhyve or
> nouveau hasn't yet implemented? Alternatively, maybe I could exploit some
> power management feature to reset the card when bhyve exits.
> 3. It is not possible to reboot the guest and then start X again without an
> intervening host reboot. The text console works fine. Xorg.0.log has a
> message like
> (EE) [drm] Failed to open DRM device for pci::00:06.0: -19
> (EE) open /dev/dri/card0: No such file or directory
> dmesg is not very helpf
Debugging UEFI code in Bhyve
I was wondering, has anyone done any work on supporting debugging UEFI code under Bhyve? If not, I can take a look at adapting DebugPkg (https://code.bluestop.org/w/tianocore/debugging-with-gdb/). -- Rebecca Cran ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: Updating uefi-edk2-bhyve
On 3/30/19 2:37 PM, Rodney W. Grimes wrote: On 3/30/19 1:08 AM, Rodney W. Grimes wrote: So it is a edk2 implementation, and can you point me at the bhyve version of it? The Qemu/KVM version is rather not relavent to this discussion as that would not work due to core vmm implementation differences. Yes, OVMF is part of the TianoCore EDK2. OVMF/KVM is relevant in my opinion because the BHYVE UEFI firmware is a port of OVMF. In fact, with Scott's changes there's no longer a BhyvePkg, but our copy of OvmfPkg works with Bhyve. The code can be found at https://gitlab.com/scott-ph/edk2/tree/wip/2019-03/v2-bhyve-rebase-edk2-stable201903 . ^^^ This is the critical version, this is the version of the SMBIOS/dmitables implemented by the bios. I think you're confused: I wasn't asking about updating the version of _SMBIOS_ that Bhyve uses, just the BIOS version and release date that are shown in the BIOS section of the SMBIOS data. -- Rebecca Cran ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: Updating uefi-edk2-bhyve
On 3/30/19 1:08 AM, Rodney W. Grimes wrote: I do not know what the above is, could you elaborate for me? What is OVMF and what is its relation to bhyve? OVMF is the Open Virtual Machine Firmware: it's open source (BSD licensed) firmware for virtual machines. Currently, that means Qemu/KVM. It has been ported to run under Bhyve too. I have to assume this is with ed2k loaded, but I do not know you are showing me the SMBIOS string value or some other bios version value. THere are compliance levels associated with SMBIOS. I guess you mean edk2 (efi development kit 2), not ed2k? I'm showing you the SMBIOS values for the BIOS section. Another example: on my desktop system, 'dmidecode' shows: root@photon:/usr/home/bcran # dmidecode -t bios # dmidecode 3.2 # SMBIOS entry point at 0x7949f000 Found SMBIOS entry point in EFI, reading table from /dev/mem. SMBIOS 3.0 present. Handle 0x, DMI type 0, 24 bytes BIOS Information Vendor: American Megatrends Inc. Version: 1701 Release Date: 01/09/2019 Address: 0xF Runtime Size: 64 kB ROM Size: 16 MB Characteristics: PCI is supported APM is supported BIOS is upgradeable BIOS shadowing is allowed Boot from CD is supported Selectable boot is supported BIOS ROM is socketed EDD is supported 5.25"/1.2 MB floppy services are supported (int 13h) 3.5"/720 kB floppy services are supported (int 13h) 3.5"/2.88 MB floppy services are supported (int 13h) Print screen service is supported (int 5h) 8042 keyboard services are supported (int 9h) Serial services are supported (int 14h) Printer services are supported (int 17h) ACPI is supported USB legacy is supported BIOS boot specification is supported Targeted content distribution is supported UEFI is supported BIOS Revision: 5.13 -- Rebecca Cran _______ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: Updating uefi-edk2-bhyve
On 3/29/19 2:08 PM, Rebecca Cran via freebsd-virtualization wrote: I ran the SCT 2.6 against the updated Bhyve firmware and uploaded the results to https://bluestop.org/files/Summary.log . I'm not sure how the failures compare with either the current firmware or OVMF though, so I'm not sure how useful it is. I ran the SCT overnight on OVMF (on Linux, with Qemu from Git master), and some failures are the same but some are definitely unique to Bhyve. OVMF:Total ran: 7118, Failed: 99, Passed: 7019 BHYVE: Total ran: 6957, Failed: 34, Passed: 6923 The BHYVE specific failures were: RT.SetTime - SetTime() returns EFI_SUCCESS to update the Year at EFI_TPL_APPLICATION. RT.SetTime - SetTime() returns EFI_SUCCESS to update the Year at EFI_TPL_CALLBACK. RT.SetTime - SetTime() returns EFI_SUCCESS to update the Month at EFI_TPL_CALLBACK. SIMPLE_TEXT_INPUT_EX_PROTOCOL.ReadKeyStorkeEx - ReadKeyStorkeEx() returns EFI_NOT_READY with console just been reseted. SIMPLE_TEXT_INPUT_EX_PROTOCOL.Reset - Reset() returns EFI_SUCCESS and ReadKeyStrokeEx return EFI_NOT_READY SIMPLE_INPUT.Reset - ReadKeyStroke() returns EFI_NOT_READY when there is no key has been stroked. -- Rebecca Cran _______ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: Updating uefi-edk2-bhyve
On 3/29/19 9:29 PM, Rodney W. Grimes wrote: That, iirc, would be the SMBIOS version of ed2k, which yes should be updated if infact the newer ed2k has a new SBIOS implementation, if it is still 1.00 implementaton then this needs to be left alone. Under OVMF "smbiosview -t 0" shows: Vendor: EFI Development Kit II / OVMF BiosVersion: 0.0.0 BiosReleaseDate: 02/06/2015 Whereas, we have: Vendor: BHYVE BiosVersion: 1.00 BiosReleaseDate: 03/14/2014 -- Rebecca Cran _______ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: Updating uefi-edk2-bhyve
On 3/29/19 12:29 PM, Rebecca Cran via freebsd-virtualization wrote: On 3/25/19 3:59 PM, D Scott Phillips wrote: Yep, makes sense to me. For either of these changes we would want to get test converage on basically all functionality, so might as well take both changes at once. One thing I noticed is that the SMBIOS information claims it's BHYVE BIOS v1.00 released 3/14/2014. It would be nice if we could update that. I ran the SCT 2.6 against the updated Bhyve firmware and uploaded the results to https://bluestop.org/files/Summary.log . I'm not sure how the failures compare with either the current firmware or OVMF though, so I'm not sure how useful it is. I've tested FreeBSD, Ubuntu and Windows Server 2016 and all worked, though for some reason the Windows ISO kept hanging during boot for a while, until it just started working. -- Rebecca Cran ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: Updating uefi-edk2-bhyve
On 3/25/19 3:59 PM, D Scott Phillips wrote: Yep, makes sense to me. For either of these changes we would want to get test converage on basically all functionality, so might as well take both changes at once. One thing I noticed is that the SMBIOS information claims it's BHYVE BIOS v1.00 released 3/14/2014. It would be nice if we could update that. -- Rebecca Cran ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: GPU passthrough: mixed success on Linux, not yet on Windows
I added some logging in ppt_pci_reset() and I can confirm the GPU is indeed
cycled through the D3 state at the beginning and end of the VM session. AFAIK
that’s only D3 hot though; does FreeBSD have support for D3 cold?
On Wed, Mar 27, 2019 at 15:48, Nick Wolff wrote:
> Hi Robert,
>
> So for problem 2/3 you may want to look at the ppt_pci_reset(device_t dev)
> function in ppt.c. I'm not sure why this isn't dealing with both of your
> problems. Matt Macy may have some ideas.
>
> Great work by the way
>
> Thanks,
>
> Nick Wolff
>
> On Sun, Mar 17, 2019 at 12:23 PM Robert Crowston via freebsd-virtualization
> wrote:
>
>> Hi folks, this is my first post to the group. Apologies for length.
>>
>> I've been experimenting with GPU passthrough on bhyve. For background, the
>> host system is FreeBSD 12.0-RELEASE on an AMD Ryzen 1700 CPU @ 3.8 GHz, 32
>> GB of ECC RAM, with two nVidia GPUs. I'm working with a Linux Debian 9 guest
>> and a Windows Server 2019 (desktop experience installed) guest. I also have
>> a USB controller passed-through for bluetooth and keyboard.
>>
>> With some unpleasant hacks I have succeeded in starting X on the Linux
>> guest, passing-through an nVidia GT 710 under the nouveau driver. I can run
>> the "mate" desktop and glxgears, both of which are smooth at 4K. The Unity
>> Heaven benchmark runs at an embarrassing 0.1 fps, and 2160p x264 video in
>> VLC runs at about 5 fps. Neither appears to be CPU-bound in the host or the
>> guest.
>>
>> The hack I had to make: I found that many instructions to access
>> memory-mapped PCI BARs are not being executed on the CPU in guest mode but
>> are being passed back for emulation in the hypervisor. This causes an
>> assertion to fail inside passthru_write() in pci_passthru.c
>> ["pi->pi_bar[baridx].type == PCIBAR_IO"] because it does not expect to
>> perform memory-mapped IO for the guest. Examining the to-be-emulated
>> instructions in vmexit_inst_emul() {e.g., movl (%rdi), %eax}, they look
>> benign to me, and I have no explanation for why the CPU refused to execute
>> them in guest mode.
>>
>> As an amateur work-around, I removed the assertion and instead I obtain the
>> desired offset into the guest's BAR, calculate what that guest address
>> translates to in the host's address space, open(2) /dev/mem, mmap(2) over to
>> that address, and perform the write directly. I do a similar trick in
>> passthru_read(). Ugly, slow, but functional.
>>
>> This code path is accessed continuously whether or not X is running, with an
>> increase in activity when running anything GPU-heavy. Always to bar 1, and
>> mostly around the same offsets. I added some logging of this event. It runs
>> at about 100 lines per second while playing video. An excerpt is:
>> ...
>> Unexpected out-of-vm passthrough write #492036 to bar 1 at offset 41100.
>> Unexpected out-of-vm passthrough write #492037 to bar 1 at offset 41100.
>> Unexpected out-of-vm passthrough read #276162 to bar 1 at offset 561280.
>> Unexpected out-of-vm passthrough write #492038 to bar 1 at offset 38028.
>> Unexpected out-of-vm passthrough write #492039 to bar 1 at offset 38028.
>> Unexpected out-of-vm passthrough read #276163 to bar 1 at offset 561184.
>> Unexpected out-of-vm passthrough read #276164 to bar 1 at offset 561184.
>> Unexpected out-of-vm passthrough read #276165 to bar 1 at offset 561184.
>> Unexpected out-of-vm passthrough read #276166 to bar 1 at offset 561184.
>> ...
>>
>> So my question here is,
>> 1. How do I diagnose why the instructions are not being executed in guest
>> mode?
>>
>> Some other problems:
>>
>> 2. Once the virtual machine is shut down, the passed-through GPU doesn't get
>> turned off. Whatever message was on the screen in the final throes of
>> Linux's shutdown stays there. Maybe there is a specific detach command which
>> bhyve or nouveau hasn't yet implemented? Alternatively, maybe I could
>> exploit some power management feature to reset the card when bhyve exits.
>>
>> 3. It is not possible to reboot the guest and then start X again without an
>> intervening host reboot. The text console works fine. Xorg.0.log has a
>> message like
>> (EE) [drm] Failed to open DRM device for pci::00:06.0: -19
>> (EE) open /dev/dri/card0: No such file or directory
>> dmesg is not very helpful either.[0] I suspect that this is related to
>> problem (2).
>>
>> 4. There is a known bug in the version of the Xorg server tha
Re: Updating uefi-edk2-bhyve
On 3/25/19 3:59 PM, D Scott Phillips wrote: For either of these changes we would want to get test converage on basically all functionality, so might as well take both changes at once. What testing are you thinking is needed at this point? I guess installing and running FreeBSD, Linux and Windows, and probably running the SCT and FWTS. -- Rebecca Cran ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: Updating uefi-edk2-bhyve
On 3/22/19 4:29 PM, Rebecca Cran via freebsd-virtualization wrote: On 3/22/19 2:25 PM, D Scott Phillips wrote: Hmm, I guess it might be some diference in the code generation between gcc 4.8 and gcc 5. I've just tested switching from gcc 4.8 to 8.3.0 and everything seems to work fine - both build and runtime - so I think it may be more productive to upgrade and use that instead, since it's a supported version (gcc 4.8 is pretty old now). Would you agree? I've submitted a couple of merge requests to update your edk2 tree and port for gcc 8: https://gitlab.com/scott-ph/edk2/merge_requests/1 https://gitlab.com/scott-ph/freebsd-ports/merge_requests/1 -- Rebecca Cran _______ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: Updating uefi-edk2-bhyve
On 3/22/19 2:25 PM, D Scott Phillips wrote:
Hmm, I guess it might be some diference in the code generation between
gcc 4.8 and gcc 5.
I've just tested switching from gcc 4.8 to 8.3.0 and everything seems to
work fine - both build and runtime - so I think it may be more
productive to upgrade and use that instead, since it's a supported
version (gcc 4.8 is pretty old now). Would you agree?
The DEBUG build did complain about unused definitions and functions in
FeatureControl.c though, so I ended up with the following patch:
diff --git a/OvmfPkg/PlatformPei/FeatureControl.c
b/OvmfPkg/PlatformPei/FeatureControl.c
index 09f33fe5b1..051dbbdac0 100644
--- a/OvmfPkg/PlatformPei/FeatureControl.c
+++ b/OvmfPkg/PlatformPei/FeatureControl.c
@@ -24,7 +24,7 @@
//
// The value to be written to the Feature Control MSR, retrieved from
fw_cfg.
//
-STATIC UINT64 mFeatureControlValue;
+//STATIC UINT64 mFeatureControlValue;^M
/**
Write the Feature Control MSR on an Application Processor or the Boot
@@ -36,6 +36,7 @@ STATIC UINT64 mFeatureControlValue;
@param[in,out] WorkSpace Pointer to the input/output argument workspace
shared by all processors.
**/
+#if 0^M
STATIC
VOID
EFIAPI
@@ -45,7 +46,7 @@ WriteFeatureControl (
{
AsmWriteMsr64 (MSR_CORE2_FEATURE_CONTROL, mFeatureControlValue);
}
-
+#endif^M
/**
Notification function called when EFI_PEI_MP_SERVICES_PPI becomes
available.
@@ -57,6 +58,7 @@ WriteFeatureControl (
@return Status of the notification. The status code returned from this
function is ignored.
**/
+#if 0^M
STATIC
EFI_STATUS
EFIAPI
@@ -94,17 +96,17 @@ OnMpServicesAvailable (
WriteFeatureControl (NULL);
return EFI_SUCCESS;
}
-
+#endif^M
//
// Notification object for registering the callback, for when
// EFI_PEI_MP_SERVICES_PPI becomes available.
//
-STATIC CONST EFI_PEI_NOTIFY_DESCRIPTOR mMpServicesNotify = {
- EFI_PEI_PPI_DESCRIPTOR_NOTIFY_CALLBACK | // Flags
- EFI_PEI_PPI_DESCRIPTOR_TERMINATE_LIST,
- &gEfiPeiMpServicesPpiGuid, // Guid
- OnMpServicesAvailable // Notify
-};
+//STATIC CONST EFI_PEI_NOTIFY_DESCRIPTOR mMpServicesNotify = {^M
+// EFI_PEI_PPI_DESCRIPTOR_NOTIFY_CALLBACK | // Flags^M
+// EFI_PEI_PPI_DESCRIPTOR_TERMINATE_LIST,^M
+// &gEfiPeiMpServicesPpiGuid, // Guid^M
+// OnMpServicesAvailable // Notify^M
+//};^M
VOID
InstallFeatureControlCallback (
_______
freebsd-virtualization@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to
"freebsd-virtualization-unsubscr...@freebsd.org"
Re: Updating uefi-edk2-bhyve
5): /usr/local/bin/ld: warning: cannot find entry symbol _ModuleEntryPoint; not setting start address /usr/local/bin/ld: warning: cannot find entry symbol _ModuleEntryPoint; not setting start address "objcopy" /home/bcran/workspace/bhyveedk2/Build/OvmfX64/RELEASE_GCC48/X64/OvmfPkg/AmdSevDxe/AmdSevDxe/DEBUG/AmdSevDxe.dll "objcopy" /home/bcran/workspace/bhyveedk2/Build/OvmfX64/RELEASE_GCC48/X64/OvmfPkg/EmuVariableFvbRuntimeDxe/Fvb/DEBUG/EmuVariableFvbRuntimeDxe.dll objcopy: error: the input file '/home/bcran/workspace/bhyveedk2/Build/OvmfX64/RELEASE_GCC48/X64/OvmfPkg/EmuVariableFvbRuntimeDxe/Fvb/DEBUG/EmuVariableFvbRuntimeDxe.dll' has no sections objcopy: error: the input file '/home/bcran/workspace/bhyveedk2/Build/OvmfX64/RELEASE_GCC48/X64/OvmfPkg/AmdSevDxe/AmdSevDxe/DEBUG/AmdSevDxe.dll' has no sections make: *** [GNUmakefile:383: /home/bcran/workspace/bhyveedk2/Build/OvmfX64/RELEASE_GCC48/X64/OvmfPkg/AmdSevDxe/AmdSevDxe/DEBUG/AmdSevDxe.dll] Error 1 -- Rebecca Cran ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: Updating uefi-edk2-bhyve
On Mar 21, 2019, 9:54 PM -0600, Larry Rosenman , wrote: > out of curiosity, is there any work being done to move this to later > LLVM/CLANG? That’s something else I’m hoping to work on, since gcc 4.8 is very outdated now. It seems to build fine using gcc 7, but I need to test that it still runs fine. And that’s complicated by the fact that the DEBUG build generates a triple fault even with gcc 4.8. — Rebecca Cran ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
