[CentOS-virt] PCI Passthrough not working
I am running Xen 4.6 on CentOS 7 in a Dell Poweredge T430 I need PCI Passthrough to get USB working. I am following the Xenproject Wiki I have enabled the Virtulasation in the BIOS. I have xen_pciback as a module I have issued the command: xl pci-assignable-add 00:1a0.0 and it shows up fine when I issue this xl pci-assignable-list I have pci=['00:1a.0'] on the DomU config file I have added this iommu=soft and swiotlb=force both together and separately to the kernel command line in the DomU (running Debian 8), but I get the same error each time. libxl: error: libxl_pci.c:999:do_pci_add: xc_assign_device failed: Operation not permitted libxl: error: libxl_create.c:1424:domcreate_attach_pci: libxl_device_pci_add failed: -3 Can anyone suggest a cure for this? Many thanks Francis ___ CentOS-virt mailing list CentOS-virt@centos.org https://lists.centos.org/mailman/listinfo/centos-virt
[CentOS-virt] PCI Passthrough not working
Further to my messages back in May I have at last got round to trying to get my DomU to recognise USB devices. I am using Xen 4.6 with CentOS kernel 3.18.34-20.el7.x86_64. I have to manually make the port available before creating the DomU by issuing the command: xl pci-assignable-add 00:1a.0 otherwise nothing shows in: xl pci-assignable-list I have added this to my .cfg file as per the May Emails: pci=['00:1a.0,rdm_policy=relaxed'] and in the DomU, which starts fine, I get the following information: lspci 00:00.0 USB controller: Intel Corporation Wellsburg USB Enhanced Host Controller #2 (rev 05) lsusb Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub However when I plug in a device to this USB port (Am sure it is the correct port in the Dom0) I see nothing in the DomU at all. What can I do now? Many thanks Francis ___ CentOS-virt mailing list CentOS-virt@centos.org https://lists.centos.org/mailman/listinfo/centos-virt
[CentOS-virt] PCI Passthrough not working
I am having trouble getting PCI Passthrough to work from Dom0 running CentOS 7 to DomU runnning Debian 8 I am using Xen 4.6 with CentOS kernel 3.18.34-20.el7.x86_64 on a Dell Poweredge T430. I think I have set it all up correctly, but I see no message when putting a USB device into any of the USB slots on the DomU There are three other DomUs running, but I have no need of PCI Passthrough set up for them. kernel command line on Dom0 from /etc/default/grub GRUB_CMDLINE_LINUX="crashkernel=auto intremap=no_x2apic_optout" GRUB_CMDLINE_XEN_DEFAULT="dom0_mem=13312M,max:14336M dom0_max_vcpus=6 dom0_vcpus_pin xen-pciback.passthrough=1 irqpoll GRUB_CMDLINE_LINUX_XEN_REPLACE_DEFAULT="console=hvc0 earlyprintk=xen" lspci on Dom0 filtered for USB 00:1a.0 USB controller: Intel Corporation C610/X99 series chipset USB Enhanced Host Controller #2 (rev 05) 00:1d.0 USB controller: Intel Corporation C610/X99 series chipset USB Enhanced Host Controller #1 (rev 05) lsmod on Dom0 filtered for xen xenfs 12978 1 xen_privcmd 13243 12 xenfs xen_pciback 52127 0 xen_netback 45777 6 xen_blkback 31807 0 xen_gntalloc 13144 0 xen_gntdev 17468 2 xen_evtchn 13033 5 config file for DomU == # Kernel command line options extra = "root=/dev/xvda1 swiotlb=force" memory = 2048 vcpus = 2 vif = ['mac=00:16:3E:00:00:35, bridge=xenbr5', 'mac=00:16:3E:00:00:36, bridge=xenbr6'] disk = ['phy:/dev/xen_vg/metsat_disk,xvda,w', 'phy:/dev/xen_vg/metsat_swap,xvdb,w', 'phy:/dev/xen_vg/metsat_receive,xvdc,w'] pci=['00:1a.0,rdm_policy=relaxed,permissive=1,seize=1'] on_poweroff = 'destroy' on_reboot = 'restart' on_crash = 'restart' # Run section == bootloader = "/usr/lib/xen/bin/pygrub" == In the Dom0 I get this when booting the DomU, even with irqpoll in the DomU kernel line xen_pciback: xen-pciback[:00:1a.0] IRQ line is not shared with other domains. Turning ISR off Jul 03 11:31:23 myDom0.mynetwork.net kernel: irq 18: nobody cared (try booting with the "irqpoll" option) Jul 03 11:31:23 myDom0.mynetwork.net kernel: CPU: 0 PID: 0 Comm: swapper/0 Not tainted 3.18.34-20.el7.x86_64 #1 Jul 03 11:31:23 myDom0.mynetwork.net kernel: Hardware name: Dell Inc. PowerEdge T430/0975F3, BIOS 1.5.4 10/05/2015 Jul 03 11:31:23 myDom0.mynetwork.net kernel: 8803bc603d88 81653783 8803b223e400 Jul 03 11:31:23 myDom0.mynetwork.net kernel: 8803b223e48c 8803bc603db8 810c6776 8803bc613340 Jul 03 11:31:23 myDom0.mynetwork.net kernel: 8803b223e400 0012 8803bc603e08 Jul 03 11:31:23 myDom0.mynetwork.net kernel: Call Trace: Jul 03 11:31:23 myDom0.mynetwork.net kernel: [] dump_stack+0x64/0x82 Jul 03 11:31:23 myDom0.mynetwork.net kernel: [] __report_bad_irq+0x36/0xd0 Jul 03 11:31:23 myDom0.mynetwork.net kernel: [] note_interrupt+0x226/0x270 Jul 03 11:31:23 myDom0.mynetwork.net kernel: [] ? add_interrupt_randomness+0x3c/0x1f0 Jul 03 11:31:23 myDom0.mynetwork.net kernel: [] handle_irq_event_percpu+0xcc/0x1e0 Jul 03 11:31:23 myDom0.mynetwork.net kernel: [] handle_irq_event+0x3b/0x60 Message from syslogd@myDom0 at Jul 3 11:31:23 ... kernelisabling IRQ #18 Jul 03 11:31:23 myDom0.mynetwork.net kernel: [] handle_fasteoi_irq+0x7a/0x130 Jul 03 11:31:23 myDom0.mynetwork.net kernel: [] generic_handle_irq+0x2b/0x40 Jul 03 11:31:23 myDom0.mynetwork.net kernel: [] evtchn_fifo_handle_events+0x162/0x170 Jul 03 11:31:23 myDom0.mynetwork.net kernel: [] __xen_evtchn_do_upcall+0x50/0x90 Jul 03 11:31:23 myDom0.mynetwork.net kernel: [] xen_evtchn_do_upcall+0x37/0x50 Jul 03 11:31:23 myDom0.mynetwork.net kernel: [] xen_do_hypervisor_callback+0x1e/0x40 Jul 03 11:31:23 myDom0.mynetwork.net kernel: [] ? xen_hypercall_sched_op+0xa/0x20 Jul 03 11:31:23 myDom0.mynetwork.net kernel: [] ? xen_hypercall_sched_op+0xa/0x20 Jul 03 11:31:23 myDom0.mynetwork.net kernel: [] ? xen_safe_halt+0x10/0x20 Jul 03 11:31:23 myDom0.mynetwork.net kernel: [] ? default_idle+0x24/0xf0 Jul 03 11:31:23 myDom0.mynetwork.net kernel: [] ? arch_cpu_idle+0xf/0x20 Jul 03 11:31:23 myDom0.mynetwork.net kernel: [] ? cpu_startup_entry+0x312/0x3e0 Jul 03 11:31:23 myDom0.mynetwork.net kernel: [] ? rest_init+0x77/0x80 Jul 03 11:31:23 myDom0.mynetwork.net kernel: [] ? start_kernel+0x4d0/0x4dd Jul 03 11:31:23 myDom0.mynetwork.net kernel: [] ? set_init_arg+0x55/0x55 Jul 03 11:31:23 myDom0.mynetwork.net kernel: [] ? x86_64_start_reservations+0x2a/0x2c Jul 03 11:31:23 myDom0.mynetwork.net kernel: [] ? xen_start_kernel+0x5a9/0x5b5 Jul 03 11:31:23 myDom0.mynetwork.net kernel: handlers: Jul 03 11:31:23 myDom0.mynetwork.net kernel: [] usb_hcd_irq Jul 03 11:31:23 myDom0.mynetwork.net kernel: [] xen_pcibk_guest_interrupt [xen_pciback] Jul 03 11:31:23 myDom0.mynetwork.net kernel: Di
Re: [CentOS-virt] PCI Passthrough not working
On Thu, May 12, 2016 at 12:11 PM, Francis Greaves wrote: > I am running Xen 4.6 on CentOS 7 in a Dell Poweredge T430 > I need PCI Passthrough to get USB working. I am following the Xenproject > Wiki > I have enabled the Virtulasation in the BIOS. > I have xen_pciback as a module > I have issued the command: > > xl pci-assignable-add 00:1a0.0 and it shows up fine when I issue this xl > pci-assignable-list > > I have pci=['00:1a.0'] on the DomU config file > I have added this iommu=soft and swiotlb=force both together and separately > to the kernel command line > in the DomU (running Debian 8), but I get the same error each time. > > libxl: error: libxl_pci.c:999:do_pci_add: xc_assign_device failed: Operation > not permitted > libxl: error: libxl_create.c:1424:domcreate_attach_pci: libxl_device_pci_add > failed: -3 Can you please attach the following: 1. The complete domU config file 2. A complete log of the command and all the output 3. The full output of "xl dmesg" just after you run the command Thanks, -George ___ CentOS-virt mailing list CentOS-virt@centos.org https://lists.centos.org/mailman/listinfo/centos-virt
Re: [CentOS-virt] PCI Passthrough not working
Dear George please find attached the three files as requested. I have used iommu=soft in the grub command line for the kernel in the domU as explained before. many thanks Francis From: "George Dunlap" To: "francis" , "centos-virt" Sent: Monday, 16 May, 2016 10:29:09 Subject: Re: [CentOS-virt] PCI Passthrough not working On Thu, May 12, 2016 at 12:11 PM, Francis Greaves wrote: > I am running Xen 4.6 on CentOS 7 in a Dell Poweredge T430 > I need PCI Passthrough to get USB working. I am following the Xenproject > Wiki > I have enabled the Virtulasation in the BIOS. > I have xen_pciback as a module > I have issued the command: > > xl pci-assignable-add 00:1a0.0 and it shows up fine when I issue this xl > pci-assignable-list > > I have pci=['00:1a.0'] on the DomU config file > I have added this iommu=soft and swiotlb=force both together and separately > to the kernel command line > in the DomU (running Debian 8), but I get the same error each time. > > libxl: error: libxl_pci.c:999:do_pci_add: xc_assign_device failed: Operation > not permitted > libxl: error: libxl_create.c:1424:domcreate_attach_pci: libxl_device_pci_add > failed: -3 Can you please attach the following: 1. The complete domU config file 2. A complete log of the command and all the output 3. The full output of "xl dmesg" just after you run the command Thanks, -George #xl create metsat.cfg -c the Grub menus selection is Debian GNU/Linux, with Linux 3.2.0-4-686-pae then after the Grub menu as the domU tries to boot you get this libxl: error: libxl_pci.c:999:do_pci_add: xc_assign_device failed: Operation not permitted libxl: error: libxl_create.c:1424:domcreate_attach_pci: libxl_device_pci_add failed: -3 libxl: error: libxl.c:1591:libxl__destroy_domid: non-existant domain 34 libxl: error: libxl.c:1549:domain_destroy_callback: unable to destroy guest with domid 34 libxl: error: libxl.c:1476:domain_destroy_cb: destruction of domain 34 failed and back to the dom0 prompt any help much appreciated metsat.cfg Description: Binary data Xen 4.6.1-6.el7 (XEN) Xen version 4.6.1-6.el7 (mockbu...@centos.org) (gcc (GCC) 4.8.5 20150623 (Red Hat 4.8.5-4)) debug=n Mon Apr 18 15:19:15 UTC 2016 (XEN) Latest ChangeSet: Wed Mar 16 16:51:27 2016 + git:ac630b2-dirty (XEN) Bootloader: GRUB 2.02~beta2 (XEN) Command line: placeholder dom0_mem=13312M,max:14336M dom0_max_vcpus=6 dom0_vcpus_pin (XEN) Video information: (XEN) VGA is text mode 80x25, font 8x16 (XEN) VBE/DDC methods: V2; EDID transfer time: 1 seconds (XEN) Disc information: (XEN) Found 4 MBR signatures (XEN) Found 4 EDD information structures (XEN) Xen-e820 RAM map: (XEN) - 0009c000 (usable) (XEN) 0009c000 - 000a (reserved) (XEN) 000e - 0010 (reserved) (XEN) 0010 - 67496000 (usable) (XEN) 67496000 - 6f49e000 (reserved) (XEN) 6f49e000 - 79f89000 (usable) (XEN) 79f89000 - 7ab0b000 (reserved) (XEN) 7ab0b000 - 7b53b000 (ACPI NVS) (XEN) 7b53b000 - 7b6c (ACPI data) (XEN) 7b6c - 7b6e9000 (usable) (XEN) 7b6e9000 - 7b6ff000 (ACPI data) (XEN) 7b6ff000 - 7b70 (usable) (XEN) 7b70 - 9000 (reserved) (XEN) feda8000 - fedac000 (reserved) (XEN) ff31 - 0001 (reserved) (XEN) 0001 - 00088000 (usable) (XEN) ACPI: RSDP 000FE320, 0024 (r2 DELL ) (XEN) ACPI: XSDT 7B6BE0E8, 00AC (r1 DELL PE_SC3 0 113) (XEN) ACPI: FACP 7B6BA000, 00F4 (r4 DELL PE_SC3 0 DELL1) (XEN) ACPI: DSDT 7B6A8000, DDAE (r2 DELL PE_SC3 3 DELL1) (XEN) ACPI: FACS 7B538000, 0040 (XEN) ACPI: MCEJ 7B6BD000, 0130 (r1 INTEL 1 INTL 10D) (XEN) ACPI: WD__ 7B6BC000, 0134 (r1 DELL PE_SC3 1 DELL1) (XEN) ACPI: SLIC 7B6BB000, 0024 (r1 DELL PE_SC3 1 DELL1) (XEN) ACPI: HPET 7B6B9000, 0038 (r1 DELL PE_SC3 1 DELL1) (XEN) ACPI: APIC 7B6B8000, 085C (r2 DELL PE_SC3 0 DELL1) (XEN) ACPI: MCFG 7B6B7000, 003C (r1 DELL PE_SC3 1 DELL1) (XEN) ACPI: PMCT 7B6B6000, 0064 (r1 DELL PE_SC3 0 DELL1) (XEN) ACPI: SSDT 7B5DC000, CBC9B (r2 DELL PE_SC3 4000 INTL 20121114) (XEN) ACPI: SSDT 7B5DA000, 1C94 (r2 DELL PE_SC3 2 INTL 20121114) (XEN) ACPI: SSDT 7B5D9000, 006E (r2 DELL PE_SC3 2 INTL 20121114) (XEN) ACPI: PRAD 7B5D8000, 0132 (r2 DELL PE_SC3 2 INTL 20121114) (XEN) ACPI: DMAR 7B6FE000, 0148 (r
Re: [CentOS-virt] PCI Passthrough not working
On Mon, May 16, 2016 at 12:00 PM, Francis Greaves wrote: > Dear George please find attached the three files as requested. > I have used > > iommu=soft > > in the grub command line for the kernel in the domU as explained before. > many thanks (Please reply in-line, like this, rather than top-posting.) Thanks -- as I suspected, your USB device has RMRRs, which are what's causing the problem. In this case, your USB controller's RMRRs are shared with another device. Theoretically, having two devices with the same RMRRs assigned to different domains could be a security issue, so Xen by default refuses to do it. The options are: 1. Figure out what the other device is and assign them both to the guest 2. Tell Xen that you don't mind the sharing. You should only do #2 if you're not using Xen for isolation -- i.e., if you trust the software in that VM not to attack dom0. I *think* you should be able to do #2 by adding 'rdm_policy=relaxed' to your pci stanza; i.e., it should look like this: pci=['00:1a.0,rdm_policy=relaxed'] Let me know if that works for you. -George ___ CentOS-virt mailing list CentOS-virt@centos.org https://lists.centos.org/mailman/listinfo/centos-virt
Re: [CentOS-virt] PCI Passthrough not working
>On Mon, May 16, 2016 at 12:00 PM, Francis Greaves wrote: >> Dear George please find attached the three files as requested. >> I have used >> >> iommu=soft >> >> in the grub command line for the kernel in the domU as explained before. >> many thanks >The options are: >1. Figure out what the other device is and assign them both to the guest >2. Tell Xen that you don't mind the sharing. > >You should only do #2 if you're not using Xen for isolation -- i.e., >if you trust the software in that VM not to attack dom0. > >I *think* you should be able to do #2 by adding 'rdm_policy=relaxed' >to your pci stanza; i.e., it should look like this: > >pci=['00:1a.0,rdm_policy=relaxed'] Dear George That worked, do I still need to add this to the cfg file? usb=1 usbdevice=['host:0529:0514'] or how do I assign the device without the 'rdm_policy=relaxed' option as per option 1? Thanks so much for your speedy advice. Francis ___ CentOS-virt mailing list CentOS-virt@centos.org https://lists.centos.org/mailman/listinfo/centos-virt
Re: [CentOS-virt] PCI Passthrough not working
On Mon, May 16, 2016 at 2:14 PM, Francis Greaves wrote: >>On Mon, May 16, 2016 at 12:00 PM, Francis Greaves wrote: >>> Dear George please find attached the three files as requested. >>> I have used >>> >>> iommu=soft >>> >>> in the grub command line for the kernel in the domU as explained before. >>> many thanks > >>The options are: >>1. Figure out what the other device is and assign them both to the guest >>2. Tell Xen that you don't mind the sharing. >> >>You should only do #2 if you're not using Xen for isolation -- i.e., >>if you trust the software in that VM not to attack dom0. >> >>I *think* you should be able to do #2 by adding 'rdm_policy=relaxed' >>to your pci stanza; i.e., it should look like this: >> >>pci=['00:1a.0,rdm_policy=relaxed'] > > Dear George > That worked, do I still need to add this to the cfg file? > > usb=1 > usbdevice=['host:0529:0514'] > > or how do I assign the device without the 'rdm_policy=relaxed' option as per > option 1? Sorry, I don't understand your question. In the pci case, you're passing through an entire USB controller. The guest will get access to anything plugged into that controller as though it were running on native. In the second case, you're presenting an emulated controller, and then having qemu pass commands through to the device. This should also work, but may be a bit slower. But for 4.6 it's only available in HVM mode -- and the config file you attached was set up to run in PV mode (if I remember correctly). > Thanks so much for your speedy advice. Responding right away means I can delete it and forget about it. ;-) -George ___ CentOS-virt mailing list CentOS-virt@centos.org https://lists.centos.org/mailman/listinfo/centos-virt
Re: [CentOS-virt] PCI Passthrough not working
More information... I have pcifront showing as a module in the DomU and the usb shows in dmesg as: [ 3.167543] usbcore: registered new interface driver usbfs [ 3.167563] usbcore: registered new interface driver hub [ 3.167585] usbcore: registered new device driver usb [ 3.196056] usb usb1: New USB device found, idVendor=1d6b, idProduct=0002 [ 3.196060] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1 [ 3.196064] usb usb1: Product: EHCI Host Controller [ 3.196068] usb usb1: Manufacturer: Linux 3.2.0-4-686-pae ehci_hcd [ 3.196071] usb usb1: SerialNumber: :00:00.0 [ 3.508036] usb 1-1: new high-speed USB device number 2 using ehci_hcd [ 19.064072] usb 1-1: device not accepting address 2, error -110 [ 19.176070] usb 1-1: new high-speed USB device number 3 using ehci_hcd [ 34.732067] usb 1-1: device not accepting address 3, error -110 [ 34.844082] usb 1-1: new high-speed USB device number 4 using ehci_hcd [ 45.280073] usb 1-1: device not accepting address 4, error -110 [ 45.392067] usb 1-1: new high-speed USB device number 5 using ehci_hcd [ 55.824112] usb 1-1: device not accepting address 5, error -110 I am looking at xl dmesg in Dom0 where there are some messages relating to the PCI usb: [VT-D] It's disallowed to assign :00:1a.0 with shared RMRR at 7b80 for Dom6. (XEN) XEN_DOMCTL_assign_device: assign :00:1a.0 to dom6 failed (-1) (XEN) [VT-D] It's risky to assign :00:1a.0 with shared RMRR at 7b80 for Dom7. (XEN) [VT-D] It's risky to assign :00:1a.0 with shared RMRR at 7b80 for Dom8. (XEN) [VT-D] It's risky to assign :00:1a.0 with shared RMRR at 7b80 for Dom9. (XEN) [VT-D] It's risky to assign :00:1a.0 with shared RMRR at 7b80 for Dom10. In the as an aside... I just get blocks on the screen after the scrubbing message, and no text. I see there is a message: (XEN) Xen is relinquishing VGA console. How can I prevent this? Is there something wrong with my /etc/default/grub ... GRUB_CMDLINE_LINUX="crashkernel=auto intremap=no_x2apic_optout" GRUB_CMDLINE_XEN_DEFAULT="dom0_mem=13312M,max:14336M dom0_max_vcpus=6 dom0_vcpus_pin" GRUB_GFXMODE=1024x768 GRUB_GFXPAYLOAD_LINUX=keep GRUB_CMDLINE_LINUX_XEN_REPLACE_DEFAULT="console=hvc0 earlyprintk=xen" Many thanks Francis From: "Francis Greaves" To: "centos-virt" Sent: Wednesday, 22 June, 2016 09:56:44 Subject: [CentOS-virt] PCI Passthrough not working Further to my messages back in May I have at last got round to trying to get my DomU to recognise USB devices. I am using Xen 4.6 with CentOS kernel 3.18.34-20.el7.x86_64. I have to manually make the port available before creating the DomU by issuing the command: xl pci-assignable-add 00:1a.0 otherwise nothing shows in: xl pci-assignable-list I have added this to my .cfg file as per the May Emails: pci=['00:1a.0,rdm_policy=relaxed'] and in the DomU, which starts fine, I get the following information: lspci 00:00.0 USB controller: Intel Corporation Wellsburg USB Enhanced Host Controller #2 (rev 05) lsusb Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub However when I plug in a device to this USB port (Am sure it is the correct port in the Dom0) I see nothing in the DomU at all. What can I do now? Many thanks Francis ___ CentOS-virt mailing list CentOS-virt@centos.org https://lists.centos.org/mailman/listinfo/centos-virt ___ CentOS-virt mailing list CentOS-virt@centos.org https://lists.centos.org/mailman/listinfo/centos-virt
Re: [CentOS-virt] PCI Passthrough not working
On Wed, Jun 22, 2016 at 10:56 AM, Francis Greaves wrote: > Further to my messages back in May I have at last got round to trying to > get my DomU to recognise USB devices. > > I am using Xen 4.6 with CentOS kernel 3.18.34-20.el7.x86_64. > I have to manually make the port available before creating the DomU by > issuing the command: > xl pci-assignable-add 00:1a.0 > otherwise nothing shows in: > xl pci-assignable-list > > I have added this to my .cfg file as per the May Emails: > pci=['00:1a.0,rdm_policy=relaxed'] The two-stage process for assigning pci devices (first pci-assignible-add, then pci-add) is a "seatbelt" to make sure that an accidental mis-type doesn't cause you to grab (say) your hard disk controller instead of your USB controller. You can add "seize=1" to your pci string to have xl automatically do both steps for you. Obviously, use this with care. :-) More on your next post... -George ___ CentOS-virt mailing list CentOS-virt@centos.org https://lists.centos.org/mailman/listinfo/centos-virt
Re: [CentOS-virt] PCI Passthrough not working
On Wed, Jun 22, 2016 at 11:49 AM, Francis Greaves wrote: > More information... > I have pcifront showing as a module in the DomU and the usb shows in dmesg > as: > [3.167543] usbcore: registered new interface driver usbfs > [3.167563] usbcore: registered new interface driver hub > [3.167585] usbcore: registered new device driver usb > [3.196056] usb usb1: New USB device found, idVendor=1d6b, idProduct=0002 > [3.196060] usb usb1: New USB device strings: Mfr=3, Product=2, > SerialNumber=1 > [3.196064] usb usb1: Product: EHCI Host Controller > [3.196068] usb usb1: Manufacturer: Linux 3.2.0-4-686-pae ehci_hcd > [3.196071] usb usb1: SerialNumber: :00:00.0 > [3.508036] usb 1-1: new high-speed USB device number 2 using ehci_hcd > [ 19.064072] usb 1-1: device not accepting address 2, error -110 > [ 19.176070] usb 1-1: new high-speed USB device number 3 using ehci_hcd > [ 34.732067] usb 1-1: device not accepting address 3, error -110 > [ 34.844082] usb 1-1: new high-speed USB device number 4 using ehci_hcd > [ 45.280073] usb 1-1: device not accepting address 4, error -110 > [ 45.392067] usb 1-1: new high-speed USB device number 5 using ehci_hcd > [ 55.824112] usb 1-1: device not accepting address 5, error -110 Can you post your question with your guest config file, lspci in dom0, and lspci in your domU to xen-users? There will be a lot more eyeballs there to help you get things sorted out. > I am looking at xl dmesg in Dom0 where there are some messages relating to > the PCI usb: > [VT-D] It's disallowed to assign :00:1a.0 with shared RMRR at 7b80 > for Dom6. > (XEN) XEN_DOMCTL_assign_device: assign :00:1a.0 to dom6 failed (-1) > (XEN) [VT-D] It's risky to assign :00:1a.0 with shared RMRR at 7b80 > for Dom7. > (XEN) [VT-D] It's risky to assign :00:1a.0 with shared RMRR at 7b80 > for Dom8. > (XEN) [VT-D] It's risky to assign :00:1a.0 with shared RMRR at 7b80 > for Dom9. > (XEN) [VT-D] It's risky to assign :00:1a.0 with shared RMRR at 7b80 > for Dom10. This looks like you tried once to start your guest without "rdm_policy=relaxed" (which failed), and then tried it four times with "rdm_policy=relaxed" (which succeeded). Other than warning that there's a shared RMRR (which could potentially be a security risk), everything here looks normal. There's a possibility that the shared RMRR is what's tripping things up, but it's not very likely. > In the > as an aside... I just get blocks on the screen after the scrubbing message, > and no text. I see there is a message: > (XEN) Xen is relinquishing VGA console. > How can I prevent this? Is there something wrong with my /etc/default/grub > > ... > GRUB_CMDLINE_LINUX="crashkernel=auto intremap=no_x2apic_optout" > GRUB_CMDLINE_XEN_DEFAULT="dom0_mem=13312M,max:14336M dom0_max_vcpus=6 > dom0_vcpus_pin" > GRUB_GFXMODE=1024x768 > GRUB_GFXPAYLOAD_LINUX=keep > GRUB_CMDLINE_LINUX_XEN_REPLACE_DEFAULT="console=hvc0 earlyprintk=xen" Could you post this as a separate message to xen-users? Thanks. -George ___ CentOS-virt mailing list CentOS-virt@centos.org https://lists.centos.org/mailman/listinfo/centos-virt
Re: [CentOS-virt] PCI Passthrough not working
Here is my post issued again from the beginning in some sort of logical order I hope, with additional information as suggested by George Dunlap. I am having trouble getting PCI Passthrough to work from Dom0 to DomU I am using Xen 4.6 with CentOS kernel 3.18.34-20.el7.x86_64 on a Dell Poweredge T430. When I plug in a device to the USB port, nothing happens. I am Watching /var/log/messages in the DomU. Nothing Here is my lspci on the Dom0 filtered to show USB and PCI devices 00:1a.0 USB controller: Intel Corporation C610/X99 series chipset USB Enhanced Host Controller #2 (rev 05) 00:1d.0 USB controller: Intel Corporation C610/X99 series chipset USB Enhanced Host Controller #1 (rev 05) 00:02.0 PCI bridge: Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 PCI Express Root Port 2 (rev 02) 00:03.0 PCI bridge: Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 PCI Express Root Port 3 (rev 02) 00:1c.0 PCI bridge: Intel Corporation C610/X99 series chipset PCI Express Root Port #1 (rev d5) 00:1c.1 PCI bridge: Intel Corporation C610/X99 series chipset PCI Express Root Port #2 (rev d5) 00:1c.2 PCI bridge: Intel Corporation C610/X99 series chipset PCI Express Root Port #3 (rev d5) 00:1c.4 PCI bridge: Intel Corporation C610/X99 series chipset PCI Express Root Port #5 (rev d5) 01:00.0 Ethernet controller: Broadcom Corporation NetXtreme BCM5720 Gigabit Ethernet PCIe 01:00.1 Ethernet controller: Broadcom Corporation NetXtreme BCM5720 Gigabit Ethernet PCIe 04:00.0 Ethernet controller: Broadcom Corporation NetXtreme BCM5720 Gigabit Ethernet PCIe 04:00.1 Ethernet controller: Broadcom Corporation NetXtreme BCM5720 Gigabit Ethernet PCIe 05:00.0 PCI bridge: Renesas Technology Corp. Device 001d 06:00.0 PCI bridge: Renesas Technology Corp. Device 001d 07:00.0 PCI bridge: Renesas Technology Corp. Device 001a 0a:00.0 Ethernet controller: Broadcom Corporation NetXtreme BCM5719 Gigabit Ethernet PCIe (rev 01) 0a:00.1 Ethernet controller: Broadcom Corporation NetXtreme BCM5719 Gigabit Ethernet PCIe (rev 01) 0a:00.2 Ethernet controller: Broadcom Corporation NetXtreme BCM5719 Gigabit Ethernet PCIe (rev 01) 0a:00.3 Ethernet controller: Broadcom Corporation NetXtreme BCM5719 Gigabit Ethernet PCIe (rev 01) 7f:10.0 System peripheral: Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 PCIe Ring Interface (rev 02) 7f:10.1 Performance counters: Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 PCIe Ring Interface (rev 02) 80:02.0 PCI bridge: Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 PCI Express Root Port 2 (rev 02) 80:02.2 PCI bridge: Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 PCI Express Root Port 2 (rev 02) ff:10.0 System peripheral: Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 PCIe Ring Interface (rev 02) ff:10.1 Performance counters: Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 PCIe Ring Interface (rev 02) Here is my lspci on the DomU 00:00.0 USB controller: Intel Corporation Wellsburg USB Enhanced Host Controller #2 (rev 05) Prior to starting the DomU I issue command: xl pci-assignable-add 00:1a.0 xl pci-assignable-list :00:1a.0 So this is OK Now for the config file for the DomU # Guest name == name = "metsat.fsoft.nnet" # Kernel command line options extra = "root=/dev/xvda1 swiotlb=force" # Initial memory allocation (MB) memory = 2048 # Number of VCPUS vcpus = 2 # two ethernet devices, one for the network, one for the Eumetcast receiver vif = ['mac=00:16:3E:00:00:35, bridge=xenbr5', 'mac=00:16:3E:00:00:36, bridge=xenbr6'] # Disk Devices disk = ['phy:/dev/xen_vg/metsat_disk,xvda,w', 'phy:/dev/xen_vg/metsat_swap,xvdb,w', 'phy:/dev/xen_vg/metsat_receive,xvdc,w'] # for Eumetcast Dongle pci=['00:1a.0,rdm_policy=relaxed,permissive=1'] on_poweroff = 'destroy' on_reboot = 'restart' on_crash = 'restart' # Run section == bootloader = "/usr/lib/xen/bin/pygrub" == I have pcifront showing as a module in the DomU and the usb shows in dmesg as: [ 3.167543] usbcore: registered new interface driver usbfs [ 3.167563] usbcore: registered new interface driver hub [ 3.167585] usbcore: registered new device driver usb [ 3.196056] usb usb1: New USB device found, idVendor=1d6b, idProduct=0002 [ 3.196060] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1 [ 3.196064] usb usb1: Product: EHCI Host Controller [ 3.196068] usb usb1: Manufacturer: Linux 3.2.0-4-686-pae ehci_hcd [ 3.196071] usb usb1: SerialNumber: :00:00.0 [ 3.508036] usb 1-1: new high-speed USB device number 2 using ehci_hcd [ 19.064072] usb 1-1: device not accepting address 2, error -110 [ 19.176070] usb 1-1: new high-speed USB device number 3 using ehci_hcd [ 34.732067] usb 1-1: device not accepting address 3, error -110 [ 34.844082] usb 1-1: new high-speed USB device number
Re: [CentOS-virt] PCI Passthrough not working
Further to my last post, I have removed the xen-pciback module from the Dom0 kernel, and reloaded it as modprobe xen-pciback passthrough=1 I now have the PCI device on the DomU matching the Dom0 Device usb usb1: SerialNumber: :00:1a.0 instead of :00:00.0 However I now have this error ehci_hcd :00:1a.0: Unlink after no-IRQ? Controller is probably using the wrong IRQ. does this give a clue as to what is going on? Many thanks Francis ___ CentOS-virt mailing list CentOS-virt@centos.org https://lists.centos.org/mailman/listinfo/centos-virt
Re: [CentOS-virt] PCI Passthrough not working
From: "Francis Greaves" To: "Francis Greaves" , "centos-virt" Sent: Sunday, 3 July, 2016 11:19:49 Subject: Re: [CentOS-virt] PCI Passthrough not working Further to my last post, I have removed the xen-pciback module from the Dom0 kernel, and reloaded it as modprobe xen-pciback passthrough=1 I now have the PCI device on the DomU matching the Dom0 Device usb usb1: SerialNumber: :00:1a.0 instead of :00:00.0 However I now have this error ehci_hcd :00:1a.0: Unlink after no-IRQ? Controller is probably using the wrong IRQ. does this give a clue as to what is going on? Many thanks Francis More information: In the Dom0 I get this when booting the DomU, even with irwpoll in the DomU kernel line xen_pciback: xen-pciback[:00:1a.0] IRQ line is not shared with other domains. Turning ISR off Jul 03 11:31:23 antares.fsoft.nnet kernel: irq 18: nobody cared (try booting with the "irqpoll" option) Jul 03 11:31:23 antares.fsoft.nnet kernel: CPU: 0 PID: 0 Comm: swapper/0 Not tainted 3.18.34-20.el7.x86_64 #1 Jul 03 11:31:23 antares.fsoft.nnet kernel: Hardware name: Dell Inc. PowerEdge T430/0975F3, BIOS 1.5.4 10/05/2015 Jul 03 11:31:23 antares.fsoft.nnet kernel: 8803bc603d88 81653783 8803b223e400 Jul 03 11:31:23 antares.fsoft.nnet kernel: 8803b223e48c 8803bc603db8 810c6776 8803bc613340 Jul 03 11:31:23 antares.fsoft.nnet kernel: 8803b223e400 0012 8803bc603e08 Jul 03 11:31:23 antares.fsoft.nnet kernel: Call Trace: Jul 03 11:31:23 antares.fsoft.nnet kernel: [] dump_stack+0x64/0x82 Jul 03 11:31:23 antares.fsoft.nnet kernel: [] __report_bad_irq+0x36/0xd0 Jul 03 11:31:23 antares.fsoft.nnet kernel: [] note_interrupt+0x226/0x270 Jul 03 11:31:23 antares.fsoft.nnet kernel: [] ? add_interrupt_randomness+0x3c/0x1f0 Jul 03 11:31:23 antares.fsoft.nnet kernel: [] handle_irq_event_percpu+0xcc/0x1e0 Jul 03 11:31:23 antares.fsoft.nnet kernel: [] handle_irq_event+0x3b/0x60 Message from syslogd@antares at Jul 3 11:31:23 ... kernel:Disabling IRQ #18 Jul 03 11:31:23 antares.fsoft.nnet kernel: [] handle_fasteoi_irq+0x7a/0x130 Jul 03 11:31:23 antares.fsoft.nnet kernel: [] generic_handle_irq+0x2b/0x40 Jul 03 11:31:23 antares.fsoft.nnet kernel: [] evtchn_fifo_handle_events+0x162/0x170 Jul 03 11:31:23 antares.fsoft.nnet kernel: [] __xen_evtchn_do_upcall+0x50/0x90 Jul 03 11:31:23 antares.fsoft.nnet kernel: [] xen_evtchn_do_upcall+0x37/0x50 Jul 03 11:31:23 antares.fsoft.nnet kernel: [] xen_do_hypervisor_callback+0x1e/0x40 Jul 03 11:31:23 antares.fsoft.nnet kernel: [] ? xen_hypercall_sched_op+0xa/0x20 Jul 03 11:31:23 antares.fsoft.nnet kernel: [] ? xen_hypercall_sched_op+0xa/0x20 Jul 03 11:31:23 antares.fsoft.nnet kernel: [] ? xen_safe_halt+0x10/0x20 Jul 03 11:31:23 antares.fsoft.nnet kernel: [] ? default_idle+0x24/0xf0 Jul 03 11:31:23 antares.fsoft.nnet kernel: [] ? arch_cpu_idle+0xf/0x20 Jul 03 11:31:23 antares.fsoft.nnet kernel: [] ? cpu_startup_entry+0x312/0x3e0 Jul 03 11:31:23 antares.fsoft.nnet kernel: [] ? rest_init+0x77/0x80 Jul 03 11:31:23 antares.fsoft.nnet kernel: [] ? start_kernel+0x4d0/0x4dd Jul 03 11:31:23 antares.fsoft.nnet kernel: [] ? set_init_arg+0x55/0x55 Jul 03 11:31:23 antares.fsoft.nnet kernel: [] ? x86_64_start_reservations+0x2a/0x2c Jul 03 11:31:23 antares.fsoft.nnet kernel: [] ? xen_start_kernel+0x5a9/0x5b5 Jul 03 11:31:23 antares.fsoft.nnet kernel: handlers: Jul 03 11:31:23 antares.fsoft.nnet kernel: [] usb_hcd_irq Jul 03 11:31:23 antares.fsoft.nnet kernel: [] xen_pcibk_guest_interrupt [xen_pciback] Jul 03 11:31:23 antares.fsoft.nnet kernel: Disabling IRQ #18 = does that help? Francis ___ CentOS-virt mailing list CentOS-virt@centos.org https://lists.centos.org/mailman/listinfo/centos-virt
Re: [CentOS-virt] PCI Passthrough not working
On Sun, Jul 3, 2016 at 12:34 PM, Francis Greaves wrote: > > From: "Francis Greaves" > To: "Francis Greaves" , "centos-virt" > > Sent: Sunday, 3 July, 2016 11:19:49 > Subject: Re: [CentOS-virt] PCI Passthrough not working > > Further to my last post, I have removed the xen-pciback module from the Dom0 > kernel, and reloaded it as > modprobe xen-pciback passthrough=1 > > I now have the PCI device on the DomU matching the Dom0 Device > usb usb1: SerialNumber: :00:1a.0 > instead of :00:00.0 > > However I now have this error > > ehci_hcd :00:1a.0: Unlink after no-IRQ? Controller is probably using > the wrong IRQ. > > does this give a clue as to what is going on? Francis, Thanks for posting with the extra info. Could you now also re-post it to the xen-users mailing list, as I asked, so that more people can get a look at what you're doing? Thanks. :-) -George ___ CentOS-virt mailing list CentOS-virt@centos.org https://lists.centos.org/mailman/listinfo/centos-virt
