10.2-RELEASE-p7 - bhyve crash ...
Hi all, I'm trying to install a Linux guest (debian and ubuntu) using bhyve. Upon installation on either of these guest installs bhyve crashes. OS: 10.2-RELEASE-p7 FreeBSD 10.2-RELEASE-p7 amd64 (GENERIC kernel) GUEST: ubuntu-14.10-server-amd64.iso GUEST: debian-8.2.0-amd64-CD-1.iso Hardware: IBM System x3550 M4 Server -[7914L2M] with the following VT-x extensions: VT-x: PAT,HLT,MTF,PAUSE,EPT,UG,VPID. Load + installation bootstrap: #grub-bhyve -r cd0 -m device.map -M 2048 ubuntu-14.10-server #bhyve -AI -H -P -s 0:0,hostbridge -s 1:0,lpc -s 2:0,virtio-net,tap0 -s 3:0,virtio-blk,./linux.img -s 4:0,ahci-cd,./ubuntu-14.10-server-amd64.iso -l com1,stdio -c 4 -m 2048M ubuntu-14.10-server The Crash Upon installation: Installing the system... ├───┐ │ │ │ 50% │ │ │ │ [ 158.064496] BUG: unable to handle kernel paging request at 1000 [ 158.065128] IP: [] __blk_bios_map_sg+0x1b1/0x3c0 │ [ 158.065668] PGD 7065b067 PUD 79126067 PMD 0 ─┘ [ 158.066071] Oops: [#1] SMP [ 158.066375] Modules linked in: squashfs xfs libcrc32c jfs btrfs raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq raid1 raid0 multipath linear ghash_clmulni_intel aesni_intel aes_x86_64 ablk_helper cryptd lrw gf128mul glue_helper nls_utf8(E) isofs(E) usb_storage(E) ahci(E) libahci(E) [ 158.069152] CPU: 0 PID: 136 Comm: kworker/u8:5 Tainted: GE 3.16.0-23-generic #31-Ubuntu [ 158.069905] Hardware name: BHYVE, BIOS 1.00 03/14/2014 [ 158.070353] Workqueue: writeback bdi_writeback_workfn (flush-252:0) [ 158.070907] task: 88007ba532f0 ti: 88007a60 task.ti: 88007a60 [ 158.071532] RIP: 0010:[] [] __blk_bios_map_sg+0x1b1/0x3c0 [ 158.072265] RSP: 0018:88007a603820 EFLAGS: 00010206 [ 158.072712] RAX: 1000 RBX: 4000 RCX: 88007a6038a8 [ 158.073306] RDX: 1001 RSI: 73cb1000 RDI: 88007bb1ea20 [ 158.073902] RBP: 88007a603898 R08: R09: 1600 [ 158.074496] R10: 001d R11: 73cb3000 R12: [ 158.075091] R13: 1000 R14: 001d R15: 88007a7ef420 [ 158.075688] FS: () GS:88007fc0() knlGS: [ 158.076360] CS: 0010 DS: ES: CR0: 80050033 [ 158.076843] CR2: 1000 CR3: 7065a000 CR4: 000406f0 [ 158.077439] Stack: [ 158.077617] 88007d001700 88007bb1e5a0 88093e00 001d [ 158.078290] 88007a6038a8 0125 ea0001cf2c40 1000 [ 158.078962] ea0001cf2c80 1000 88007bb1e400 88093e00 [ 158.079632] Call Trace: [ 158.079852] [] blk_rq_map_sg+0x35/0x170 [ 158.080316] [] virtio_queue_rq+0x9d/0x260 [ 158.080795] [] __blk_mq_run_hw_queue+0x1bf/0x320 [ 158.081321] [] blk_mq_run_hw_queue+0x65/0x80 [ 158.081821] [] blk_mq_insert_requests+0xcf/0x130 [ 158.082346] [] blk_mq_flush_plug_list+0x129/0x140 [ 158.082882] [] blk_flush_plug_list+0xd1/0x230 [ 158.083387] [] blk_finish_plug+0x14/0x50 [ 158.083859] [] ext4_writepages+0x3e9/0xcf0 [ 158.084343] [] ? global_dirtyable_memory+0x50/0x50 [ 158.084885] [] do_writepages+0x1e/0x30 [ 158.085341] [] __writeback_single_inode+0x40/0x210 [ 158.085883] [] ? wake_up_bit+0x25/0x30 [ 158.086338] [] writeback_sb_inodes+0x247/0x3b0 [ 158.086849] [] __writeback_inodes_wb+0x9f/0xd0 [ 158.087356] [] wb_writeback+0x223/0x2c0 [ 158.087820] [] bdi_writeback_workfn+0x1b9/0x430 [ 158.088337] [] process_one_work+0x182/0x4e0 [ 158.088829] [] worker_thread+0x6b/0x6a0 [ 158.089290] [] ? process_one_work+0x4e0/0x4e0 [ 158.089796] [] kthread+0xdb/0x100 [ 158.090215] [] ? kthread_create_on_node+0x1c0/0x1c0 [ 158.090764] [] ret_from_fork+0x7c/0xb0 [ 158.091218] [] ? kthread_create_on_node+0x1c0/0x1c0 [ 158.091765] Code: 39 f3 0f 84 4a 01 00 00 48 83 20 fd 4c 89 54 24 18 48 8b 39 48 89 4c 24 20 e8 5c a6 03 00 48 8b 4c 24 20 4c 8b 54 24 18 48 89 01 <48> 8b 30 48 8b 54 24 30 8b 7c 24 3c 83 e6 03 f6 c2 03 0f 85 07 [ 158.094199] RIP [] __blk_bios_map_sg+0x1b1/0x3c0 [ 158.094736] RSP [ 158.095035] CR2: 1000 [ 158.095320] ---[ end trace 86aa70b9ee9da8be ]--- [ 158.095900] BUG: unable to handle kernel paging request at ffd8 [ 158.096532] IP: [] kthread_data+0x10/0x20 [ 158.097041] PGD 1c16067 PUD 1c18067 PMD 0 [ 158.097447] Oops: [#2] SMP [ 158.097753] Modules linked in: squashfs xfs libcrc32c jfs btrfs raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq raid1 raid0 multipath linear ghash_clmulni_intel aesni_intel aes_x86_64 ablk_helper cryptd lrw gf128mul glue_helper nls_utf8(E) isofs(E) usb_storage(E) ahci(E) libahci(E) [ 158.100528] CPU: 0 PID: 136 Comm: kworker/u8:5 Tainted: G D E 3.16.0-23-generic #31-Ubuntu [ 158.101278] Hardw
Re: bhyve PCI pass-through to Linux guest
Excerpts from Peter Grehan's message from Tue 22-Dec-15 13:09: > > I pass trough a PCI device (USB controller) to a Linux guest. It works > > properly. Then I halt the VM, make sure that bhyve destroyed it and run > > Windows guest with the same PCI device passed-through. > > > > Windows device manager does show the device, however, e.g. a flash drive > > plugged in is not presented to Windows, instead it's being processed by > > FreeBSD. > > > > After that it does not work in Linux guest as well. Kernel module (vmm) > > unloading and reloading does not help. > > The flash drive being processed by FreeBSD would indicate that it has > ownership of the device. Would you be able to try a 'pciconf -vl' after > the Linux guest exists, and after the Windows guest exits ? First of all I found that I do not need to switch between guests. It is 100% reproducible with a single Linux VM. In freshly booted FreeBSD the USB ports (ones passed through) do not sense any device plugged in. When I boot Linux VM, it fully controls those ports, and I see the USB flash in Linux. As soon as I halt Linux, the flash drive appears in the host! Today I've checked out the latest sources and recompiled the whole system. Nothing has changed. Here is the corresponding output of pciconf iand devinfo after Linux exits: $ pciconf -vl ppt0@pci0:0:20:0: class=0x0c0330 card=0x21f317aa chip=0x1e318086 rev=0x04 hdr=0x00 vendor = 'Intel Corporation' device = '7 Series/C210 Series Chipset Family USB xHCI Host Controller' class = serial bus subclass = USB $ devinfo | grep -B15 mass pcib1 pci1 sdhci_pci0 pcib2 pci2 iwn0 pcib3 pci3 ehci1 usbus1 uhub1 uhub3 umass0 -- S. ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: bhyve PCI pass-through to Linux guest
Excerpts from Anish's message from Mon 21-Dec-15 21:48: > >ppt0@pci0:0:20:0: class=0x0c0330 card=0x21f317aa chip=0x1e318086 > Passthrough stub driver, part of vmm, controls the USB controller. > > Can you share output of /usr/sbin/devinfo from FreeBSD host, highlighting > the usb mass/flash device in tree? > Before I start the guest VM the flash drive is not in the list. Right after shuting down the guest OS it appears: pcib1 pci1 sdhci_pci0 pcib2 pci2 iwn0 pcib3 pci3 ehci1 usbus1 uhub1 uhub3 umass0 As you can see it's under "ehci", it looks that my USB3 controller is fully USB2 backward compatible and work with "ehci" driver. I tried (as Trent suggested in this thread) to recompile the kernel without "xhci", and nothing changed, still behaves exactly the same way.. S. ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Failed to start virtualbox natnetwork
Hello,
I am trying to use virtualbox natnetwork on a FreeBSD host. Natnetwork configuration seems to
complete ok, but actual natnetwork start fails with
NS_ERROR_FAILURE (0x80004005), more details below. Is there a way to get it
working?
Thanks
Irek.
$ uname -rs
FreeBSD 10.2-RELEASE-p7
$ VBoxManage --version
4.3.34_OSEr104062
$ VBoxManage list natnetworks
NetworkName:czajnet
IP: 10.0.22.1
Network:10.0.22.0/24
IPv6 Enabled: No
IPv6 Prefix:
DHCP Enabled: Yes
Enabled:Yes
loopback mappings (ipv4)
127.0.0.1=2
$ VBoxManage natnetwork start --netname czajnet
VBoxManage: error: Code NS_ERROR_FAILURE (0x80004005) - Operation failed
(extended info not available)
VBoxManage: error: Context: "Start(Bstr("whatever").raw())" at line 449 of file
VBoxManageNATNetwork.cpp
VBoxManage: error: Failed to start network
___
freebsd-virtualization@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to
"freebsd-virtualization-unsubscr...@freebsd.org"
Re: Windows/bhyve Bluescreen 0x50
>It appears to be related to the number of vCPUs. Using a single vCPU will result in this, wherease >= 2 seems to work Ok with 2k16 tp3. That did the trick! I still seem to be having issues with the firmware on the i5 machine getting Windows 2016 to boot. (Windows 2008 and 2012 work fine) bhyve exits with: >$ vm exit[0] >reason VMX >rip 0x017ca1d7 >inst_length 2 >status 0 >exit_reason 2 >qualification 0x >inst_type 0 >inst_error 0 >$ sysctl hw.model >hw.model: Intel(R) Core(TM) i5-2520M CPU @ 2.50GHz Thanks for all your help! ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: BAR and pci passthrough
On Saturday, December 12, 2015 02:04:17 PM G. Amanakis via freebsd-virtualization wrote: > Changing > #define PCIM_BAR_MEM_BASE 0xfff0ULL > to > #define PCIM_BAR_MEM_BASE 0xf000ULL > in > sys/dev/PCI/pcireg.h makes the min size of a memory BAR equal to 0x1000 or > 4096 bytes and it lets me passthrough the device. Any objections to this > strategy? This constant defines the spec as implemented in hardware. You will break real hardware on bare metal if you make this change. As Neel notes, this is hard to fix properly. The proper fix is to modify the ppt(4) driver so that it asks the PCI bus to allocate a full page for BARs that are smaller than a page. Unfortunately the PCI bus driver doesn't currently provide a way to do that. Even if it did it would not necessarily always work since the device may be behind a PCI-PCI bridge whose windows can't grow due to existing allocations of adjacent addresses. Alternatively the hypervisor could trap all accesses to this page and only permit accesses to the range that contains the BAR, but that would be quite slow. > On December 2, 2015 11:54:03 PM GMT+01:00, Neel Natu > wrote: > >On Wed, Dec 2, 2015 at 3:13 AM, G. Amanakis via freebsd-virtualization > > wrote: > >> I am facing the following problem: > >> on a X9SCM with an E3-1220Lv2 I am trying to passthrough the > >onboard usb controllers. I disable the usb module from the kernel > >config and using pptdev the controllers are assigned to ppt devices. > >However running bhyve on FreeBSD 10.2 with: > >> > >> sudo bhyve -AI -H -P -s 0:0,hostbridge -s 1:0,lpc -s > >2:0,virtio-net,tap0 -s 3:0,virtio-blk,./linux.img -s > >4:0,passthru,0/29/0 -l com1,stdio -c 2 -m 2048M linuxguest > >> > >> returns : > >> > >> passthru device 0/29/0 BAR 0: base 0xdf823000 or size 0x400 not > >page aligned > >> > >> The problem probably that the length of the bar is smaller and not > >aligned with the pagesize. Could the length of the BAR be modified in > >order to perform the pci passthrough? > > > >Yes, that's correct - the size of the BAR is not a multiple of the > >page size which leads to the error. If this BAR is mapped into the > >guest's address space then it will "leak" an additional 3K into the > >guest (since the minimum nested mapping is 4KB in size). > > > >It is hard to fix this in the general case if you want to truly > >passthrough the BAR to the guest. > > > >best > >Neel > > > >> -- > >> Sent from my Android device with K-9 Mail. Please excuse my brevity. > >> ___ > >> freebsd-virtualization@freebsd.org mailing list > >> https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > >> To unsubscribe, send any mail to > >"freebsd-virtualization-unsubscr...@freebsd.org" > > -- John Baldwin ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
