Hello,
I tried to use the "macvtap" solution instead of the direct pci
passthrough. It works. But I soon as I try to attach the pci device in
direct mode into the VM it doesn't work,
I am wondering if my versions are too old maybe:
[root@peryn5 ~]# virsh version
Compiled against library: libvirt 1.2.8
Using library: libvirt 1.2.8
Using API: QEMU 1.2.8
Running hypervisor: QEMU 2.0.0
I tried to assign the PF0 where my VF are attached to and I have the
following errors:
Unable to complete install: 'internal error: process exited while
connecting to monitor: 2015-06-05T21:26:23.951795Z qemu-kvm: -device
vfio-pci,host=04:00.0,id=hostdev0,bus=pci.0,addr=0x7: vfio: error, group 14
is not viable, please ensure all devices within the iommu_group are bound
to their vfio bus driver.
2015-06-05T21:26:23.951831Z qemu-kvm: -device
vfio-pci,host=04:00.0,id=hostdev0,bus=pci.0,addr=0x7: vfio: failed to get
group 14
2015-06-05T21:26:23.951844Z qemu-kvm: -device
vfio-pci,host=04:00.0,id=hostdev0,bus=pci.0,addr=0x7: Device initialization
failed.
2015-06-05T21:26:23.951857Z qemu-kvm: -device
vfio-pci,host=04:00.0,id=hostdev0,bus=pci.0,addr=0x7: Device 'vfio-pci'
could not be initialized'
Same error for my PF1.
Unable to complete install: 'internal error: process exited while
connecting to monitor: 2015-06-05T21:28:48.689360Z qemu-kvm: -device
vfio-pci,host=04:00.1,id=hostdev0,bus=pci.0,addr=0x6: vfio: error, group 14
is not viable, please ensure all devices within the iommu_group are bound
to their vfio bus driver.
2015-06-05T21:28:48.689414Z qemu-kvm: -device
vfio-pci,host=04:00.1,id=hostdev0,bus=pci.0,addr=0x6: vfio: failed to get
group 14
2015-06-05T21:28:48.689428Z qemu-kvm: -device
vfio-pci,host=04:00.1,id=hostdev0,bus=pci.0,addr=0x6: Device initialization
failed.
2015-06-05T21:28:48.689442Z qemu-kvm: -device
vfio-pci,host=04:00.1,id=hostdev0,bus=pci.0,addr=0x6: Device 'vfio-pci'
could not be initialized
In my host my CPU is an Intel Xeon 5500 i7
lspci:
ff:00.0 Host bridge: Intel Corporation Xeon 5500/Core i7 QuickPath
Architecture Generic Non-Core Registers (rev 05)
ff:00.1 Host bridge: Intel Corporation Xeon 5500/Core i7 QuickPath
Architecture System Address Decoder (rev 05)
ff:02.0 Host bridge: Intel Corporation Xeon 5500/Core i7 QPI Link 0 (rev 05)
ff:02.1 Host bridge: Intel Corporation Xeon 5500/Core i7 QPI Physical 0
(rev 05)
ff:03.0 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory
Controller (rev 05)
ff:03.1 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory
Controller Target Address Decoder (rev 05)
ff:03.4 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory
Controller Test Registers (rev 05)
ff:04.0 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory
Controller Channel 0 Control Registers (rev 05)
ff:04.1 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory
Controller Channel 0 Address Registers (rev 05)
ff:04.2 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory
Controller Channel 0 Rank Registers (rev 05)
ff:04.3 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory
Controller Channel 0 Thermal Control Registers (rev 05)
ff:05.0 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory
Controller Channel 1 Control Registers (rev 05)
ff:05.1 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory
Controller Channel 1 Address Registers (rev 05)
ff:05.2 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory
Controller Channel 1 Rank Registers (rev 05)
ff:05.3 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory
Controller Channel 1 Thermal Control Registers (rev 05)
ff:06.0 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory
Controller Channel 2 Control Registers (rev 05)
ff:06.1 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory
Controller Channel 2 Address Registers (rev 05)
ff:06.2 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory
Controller Channel 2 Rank Registers (rev 05)
ff:06.3 Host bridge: Intel Corporation Xeon 5500/Core i7 Integrated Memory
Controller Channel 2 Thermal Control Registers (rev 05)
Using the virt-manager I tried to add the PCI device. I detached it from
the iommu group using
virsh nodedev-dettach pci_0000_04_02_4
Know the device appears in /dev/vfio/
I changed the right on the file to 666:
[root@peryn5 ~]# ls -la /dev/vfio/
crw-rw-rw- 1 root root 246, 1 Jun 5 14:40 24
I tried to boot my VM and I stil get that error:
Unable to complete install: 'internal error: process exited while
connecting to monitor: 2015-06-05T21:41:43.855542Z qemu-kvm: -device
vfio-pci,host=04:02.4,id=hostdev0,bus=pci.0,addr=0x7,rombar=0: vfio: failed
to open /dev/vfio/vfio: Operation not permitted
2015-06-05T21:41:43.855598Z qemu-kvm: -device
vfio-pci,host=04:02.4,id=hostdev0,bus=pci.0,addr=0x7,rombar=0: vfio: failed
to setup container for group 24
2015-06-05T21:41:43.855613Z qemu-kvm: -device
vfio-pci,host=04:02.4,id=hostdev0,bus=pci.0,addr=0x7,rombar=0: vfio: failed
to get group 24
2015-06-05T21:41:43.855632Z qemu-kvm: -device
vfio-pci,host=04:02.4,id=hostdev0,bus=pci.0,addr=0x7,rombar=0: Device
initialization failed.
2015-06-05T21:41:43.855652Z qemu-kvm: -device
vfio-pci,host=04:02.4,id=hostdev0,bus=pci.0,addr=0x7,rombar=0: Device
'vfio-pci' could not be initialized'
I am out of ideas :P
Thank you
2015-05-27 18:38 GMT-07:00 Bandan Das <b...@makefile.in>:
>
> > On May 22, 2015, at 4:50 PM, Gabriel Laupre <glau...@gmail.com> wrote:
> >
> > I have something new to investigate now :)
> >
> Sorry for not following up earlier. Did you investigate further ? Is the
> PF being used in the host ? Can you try assigning the PF instead ?
>
> Also, what processor are you trying this with ?
>
> Bandan
>
> > 2015-05-22 13:50 GMT-07:00 Gabriel Laupre <glau...@gmail.com>:
> > > You may need to detach all VFs if they are in the same iommu group.
> > All my VFs are in different iommu group
> > > Also, check if dmesg has any relevant messages when this error occurs.
> > I rebooted my machine and the "allow_unsafe_interrupts" was disable.
> > Therefore I enabled it and I tried to launch again the command. Now I
> have a different message:
> >
> > [root@peryn5 devstack]# /usr/libexec/qemu-kvm -device
> vfio-pci,host=04:01.4,id=hostdev0,bus=pci.0,addr=0x4 -monitor stdio
> > QEMU 1.5.3 monitor - type 'help' for more information
> > (qemu) qemu-kvm: -device
> vfio-pci,host=04:01.4,id=hostdev0,bus=pci.0,addr=0x4: vfio: msix_init failed
> > qemu-kvm: -device vfio-pci,host=04:01.4,id=hostdev0,bus=pci.0,addr=0x4:
> vfio: 0000:04:01.4 Error adding PCI capability 0x11[0x4f]@0xb0: -22
> > qemu-kvm: -device vfio-pci,host=04:01.4,id=hostdev0,bus=pci.0,addr=0x4:
> Device initialization failed.
> > qemu-kvm: -device vfio-pci,host=04:01.4,id=hostdev0,bus=pci.0,addr=0x4:
> Device 'vfio-pci' could not be initialized
> >
> > and in my dmesg I have the new entry:
> > [14346.555141] vfio-pci 0000:04:01.4: enabling device (0000 -> 0002)
> >
> >
> >
> > 2015-05-22 13:37 GMT-07:00 Bandan Das <b...@makefile.in>:
> >
> > Gabriel Laupre <glau...@gmail.com> writes:
> >
> > >> Just, try using a different slot number.
> > > I changed the addr from 0x3 to 0x4 with the existing device under
> 04.01.4
> > > (it is a VF from a NIC)/
> > > I get:
> > > [root@peryn5 devstack]# /usr/libexec/qemu-kvm -device
> > > vfio-pci,host=04:01.4,id=hostdev0,bus=pci.0,addr=0x4 -monitor stdio
> > > QEMU 1.5.3 monitor - type 'help' for more information
> > > (qemu) qemu-kvm: -device
> > > vfio-pci,host=04:01.4,id=hostdev0,bus=pci.0,addr=0x4: vfio: failed to
> set
> > > iommu for container: Operation not permitted
> > > qemu-kvm: -device vfio-pci,host=04:01.4,id=hostdev0,bus=pci.0,addr=0x4:
> > > vfio: failed to setup container for group 22
> > > qemu-kvm: -device vfio-pci,host=04:01.4,id=hostdev0,bus=pci.0,addr=0x4:
> > > vfio: failed to get group 22
> > > qemu-kvm: -device vfio-pci,host=04:01.4,id=hostdev0,bus=pci.0,addr=0x4:
> > > Device initialization failed.
> > > qemu-kvm: -device vfio-pci,host=04:01.4,id=hostdev0,bus=pci.0,addr=0x4:
> > > Device 'vfio-pci' could not be initialized
> > >
> > > under the group 22 I have the said device
> > > # /sys/kernel/iommu_groups/22/devices/0000:04:01.4
> > >
> > > I dettached it before using the command
> > > # virsh nodedev-detach pci_0000_04_01_4
> >
> > You may need to detach all VFs if they are in the same iommu group.
> >
> > Also, check if dmesg has any relevant messages when this error
> > occurs.
> >
> > > My goodness, what's going on :P Thank for you time Bandan, I really
> > > appreciate
> > >
> > > 2015-05-22 13:14 GMT-07:00 Bandan Das <b...@makefile.in>:
> > >
> > >> Gabriel Laupre <glau...@gmail.com> writes:
> > >>
> > >> > Actually, I get the same error if I enter a PCI address completely
> random
> > >> > (for example 04.01.5, which doesn't exist in my system)
> > >>
> > >> No, that has nothing to do with the pci address of the host device
> you are
> > >> trying to assign. It's telling you that that slot is already used up
> in
> > >> the guest.
> > >>
> > >> Just, try using a different slot number.
> > >>
> > >> > 2015-05-22 12:00 GMT-07:00 Bandan Das <b...@makefile.in>:
> > >> >
> > >> >> Gabriel Laupre <glau...@gmail.com> writes:
> > >> >>
> > >> >> >>To rule out if it's a problem with your guest xml. Let me know
> if you
> > >> are
> > >> >> > interested and I will write down the steps.
> > >> >> >
> > >> >> > I am way more than interested to try that :) Thanks a lot
> > >> >>
> > >> >> Ok, Based on your command line below, I am assuming that you have a
> > >> >> multifunction card. You might need to bind all functions to the
> > >> >> stub driver if they are in the same iommu group or else vfio will
> > >> complain.
> > >> >>
> > >> >> virsh nodedev-list | grep 04_10
> > >> >>
> > >> >> The output would be something like
> > >> >> pci_0000_04_10_0
> > >> >> pci_0000_04_10_1 and so on.
> > >> >>
> > >> >> For each, assuming they are the same card, you have to bind them
> to the
> > >> >> stub driver:
> > >> >> virsh nodedev-detach pci_0000_04_10_0
> > >> >> virsh nodedev-detach pci_0000_04_10_1 ...
> > >> >>
> > >> >> Shut down all guests and make sure no qemu processes are running
> > >> >> and then run qemu:
> > >> >> /usr/libexec/qemu-kvm -device
> > >> >> vfio-pci,host=04:10.4,id=hostdev0,bus=pci.0,addr=0x3 -monitor stdio
> > >> >>
> > >> >> I think this should be enough to spit out the error, there's no
> need to
> > >> >> actually
> > >> >> boot a guest image.
> > >> >>
> > >> >> > 2015-05-22 10:46 GMT-07:00 Bandan Das <b...@makefile.in>:
> > >> >> >
> > >> >> >> Gabriel Laupre <glau...@gmail.com> writes:
> > >> >> >>
> > >> >> >> >> Thanks for the update. Did you get a chance to run qemu
> standalone
> > >> >> >> > without libvirt?
> > >> >> >> > How can I do that, what are the purpose to do so and why to
> try
> > >> that?
> > >> >> >> > (thank you ^^)
> > >> >> >> To rule out if it's a problem with your guest xml. Let me know
> if you
> > >> >> are
> > >> >> >> interested and I will write down the steps.
> > >> >> >>
> > >> >> >> > 2015-05-21 13:32 GMT-07:00 Bandan Das <b...@makefile.in>:
> > >> >> >> >
> > >> >> >> >> Gabriel Laupre <glau...@gmail.com> writes:
> > >> >> >> >>
> > >> >> >> >> >> AppArmor is Debian based, but that doesn't stop you from
> > >> >> >> >> > running/installing it on other distros
> > >> >> >> >> >
> > >> >> >> >> >> $ dmesg | grep apparmor
> > >> >> >> >> >
> > >> >> >> >> > The service isn't started nor present in my machine. Thank
> you
> > >> for
> > >> >> the
> > >> >> >> >> hint
> > >> >> >> >> > though.
> > >> >> >> >> > I am still stuck with this error and it is not dependent
> of my
> > >> >> Intel
> > >> >> >> >> NIC, I
> > >> >> >> >> > used another one and I still got the same error message.
> > >> >> >> >> > ps: tks to remind me to send to the mailing list:)
> > >> >> >> >>
> > >> >> >> >> Thanks for the update. Did you get a chance to run qemu
> standalone
> > >> >> >> without
> > >> >> >> >> libvirt ?
> > >> >> >> >>
> > >> >> >> >> > 2015-05-21 12:32 GMT-07:00 Karl Apsite <
> > >> >> karl.aps...@dornerworks.com>:
> > >> >> >> >> >
> > >> >> >> >> >> AppArmor is Debian based, but that doesn't stop you from
> > >> >> >> >> >> running/installing it
> > >> >> >> >> >> on other distros
> > >> >> >> >> >>
> > >> >> >> >> >> $ dmesg | grep apparmor
> > >> >> >> >> >>
> > >> >> >> >> >> On 05/20/2015 02:56 PM, Bandan Das wrote:
> > >> >> >> >> >> > Gabriel Laupre <glau...@gmail.com> writes:
> > >> >> >> >> >> >
> > >> >> >> >> >> >> Mmmhh,
> > >> >> >> >> >> >> My SELinux is disabled. Is Apparmor not only on
> > >> debian/ubuntu
> > >> >> and
> > >> >> >> >> suse,
> > >> >> >> >> >> am
> > >> >> >> >> >> >> I wrong? I have no idea on that :)
> > >> >> >> >> >> >
> > >> >> >> >> >> > Yeah, me neither :) I am just trying to rule out all
> > >> >> possibilities.
> > >> >> >> >> >> >
> > >> >> >> >> >> >> 2015-05-20 11:23 GMT-07:00 Bandan Das <b...@makefile.in
> >:
> > >> >> >> >> >> >>
> > >> >> >> >> >> >>> Oh and one more thing! You already answered before
> but just
> > >> >> >> wanted
> > >> >> >> >> to
> > >> >> >> >> >> >>> confirm
> > >> >> >> >> >> >>> that you don't have apparmor running, right ?
> > >> >> >> >> >> >>>
> > >> >> >> >> >> >>> Bandan Das <b...@makefile.in> writes:
> > >> >> >> >> >> >>>
> > >> >> >> >> >> >>>> Gabriel Laupre <glau...@gmail.com> writes:
> > >> >> >> >> >> >>>>
> > >> >> >> >> >> >>>>>> Yes, indeed it is. What distro is this ? Do you
> have
> > >> >> SELinux
> > >> >> >> or
> > >> >> >> >> any
> > >> >> >> >> >> >>> other
> > >> >> >> >> >> >>>>> security feature enabled ? Can you please verify
> that the
> > >> >> file
> > >> >> >> >> has a
> > >> >> >> >> >> >>>>> appropriate label if SELinux is enabled ? (ls -lZ
> > >> >> >> /dev/vfio/vfio)
> > >> >> >> >> >> >>>>> My distrib:
> > >> >> >> >> >> >>>>> [root@peryn5 ~]# cat /proc/version
> > >> >> >> >> >> >>>>> Linux version 3.10.0-229.1.2.el7.x86_64 (
> > >> >> >> >> >> >>> buil...@kbuilder.dev.centos.org)
> > >> >> >> >> >> >>>>> (gcc version 4.8.2 20140120 (Red Hat 4.8.2-16)
> (GCC) ) #1
> > >> >> SMP
> > >> >> >> Fri
> > >> >> >> >> >> Mar 27
> > >> >> >> >> >> >>>>> 03:04:26 UTC 2015
> > >> >> >> >> >> >>>>> [root@peryn5 ~]# cat /etc/centos-release
> > >> >> >> >> >> >>>>> CentOS Linux release 7.1.1503 (Core)
> > >> >> >> >> >> >>>>>
> > >> >> >> >> >> >>>>> [root@peryn5 ~]# ls -lZ /dev/vfio/vfio
> > >> >> >> >> >> >>>>> crw-rw-rw- root root ?
> > >> >> >> >> /dev/vfio/vfio
> > >> >> >> >> >> >>>>>
> > >> >> >> >> >> >>>>> SELinux is disabled:
> > >> >> >> >> >> >>>>> [root@peryn5 ~]# getenforce
> > >> >> >> >> >> >>>>> Disabled
> > >> >> >> >> >> >>>>>
> > >> >> >> >> >> >>>>> I guess no other security feature is enabled that I
> am
> > >> aware
> > >> >> >> of. I
> > >> >> >> >> >> once
> > >> >> >> >> >> >>> had
> > >> >> >> >> >> >>>>> a message saying that it can be one of the following
> > >> issues
> > >> >> >> >> (listing
> > >> >> >> >> >> the
> > >> >> >> >> >> >>>>> 5). So I guess it can be any combination of those
> issues,
> > >> >> even
> > >> >> >> >> >> something
> > >> >> >> >> >> >>>>> completely different.
> > >> >> >> >> >> >>>>
> > >> >> >> >> >> >>>> Ugh, I am out of options! Can you please try a few
> more
> > >> >> things:
> > >> >> >> Can
> > >> >> >> >> >> you
> > >> >> >> >> >> >>> try
> > >> >> >> >> >> >>>> running qemu directly and see if you see the same
> > >> behavior ?
> > >> >> If
> > >> >> >> you
> > >> >> >> >> >> still
> > >> >> >> >> >> >>>> haven't tried running as root, please try that too.
> Also,
> > >> >> please
> > >> >> >> >> check
> > >> >> >> >> >> >>> dmesg
> > >> >> >> >> >> >>>> for any vfio related errors.
> > >> >> >> >> >> >>>>
> > >> >> >> >> >> >>>>> libvirtError: internal error: process exited while
> > >> >> connecting
> > >> >> >> to
> > >> >> >> >> >> >>>>> monitor: 2015-05-19T21:46:21.935043Z qemu-kvm:
> -device
> > >> >> >> >> >> >>>>>
> vfio-pci,host=04:10.4,id=hostdev0,bus=pci.0,addr=0x3:
> > >> vfio:
> > >> >> >> >> failed to
> > >> >> >> >> >> >>>>> open /dev/vfio/vfio: Operation not permitted
> > >> >> >> >> >> >>>> Well, this is the first error from
> > >> vfio_connect_container()
> > >> >> >> when it
> > >> >> >> >> >> does:
> > >> >> >> >> >> >>>> fd = qemu_open("/dev/vfio/vfio", O_RDWR);
> > >> >> >> >> >> >>>> if (fd < 0) {
> > >> >> >> >> >> >>>> error_report("vfio: failed to open
> /dev/vfio/vfio:
> > >> >> %m");
> > >> >> >> >> >> >>>> ret = -errno;
> > >> >> >> >> >> >>>> ...
> > >> >> >> >> >> >>>>
> > >> >> >> >> >> >>>> The rest are followup errors printed from the other
> > >> >> functions in
> > >> >> >> >> the
> > >> >> >> >> >> >>>> stack due to this error.
> > >> >> >> >> >> >>>>
> > >> >> >> >> >> >>>> Bandan
> > >> >> >> >> >> >>>>
> > >> >> >> >> >> >>>>> 2015-05-19T21:46:21.935091Z qemu-kvm: -device
> > >> >> >> >> >> >>>>>
> vfio-pci,host=04:10.4,id=hostdev0,bus=pci.0,addr=0x3:
> > >> vfio:
> > >> >> >> >> failed to
> > >> >> >> >> >> >>>>> setup container for group 24
> > >> >> >> >> >> >>>>> 2015-05-19T21:46:21.935107Z qemu-kvm: -device
> > >> >> >> >> >> >>>>>
> vfio-pci,host=04:10.4,id=hostdev0,bus=pci.0,addr=0x3:
> > >> vfio:
> > >> >> >> >> failed to
> > >> >> >> >> >> >>>>> get group 24
> > >> >> >> >> >> >>>>> 2015-05-19T21:46:21.935135Z qemu-kvm: -device
> > >> >> >> >> >> >>>>>
> vfio-pci,host=04:10.4,id=hostdev0,bus=pci.0,addr=0x3:
> > >> Device
> > >> >> >> >> >> >>>>> initialization failed.
> > >> >> >> >> >> >>>>> 2015-05-19T21:46:21.935157Z qemu-kvm: -device
> > >> >> >> >> >> >>>>>
> vfio-pci,host=04:10.4,id=hostdev0,bus=pci.0,addr=0x3:
> > >> Device
> > >> >> >> >> >> >>>>> 'vfio-pci' could not be initialized
> > >> >> >> >> >> >>>>>
> > >> >> >> >> >> >>>>>
> > >> >> >> >> >> >>>>>
> > >> >> >> >> >> >>>>>
> > >> >> >> >> >> >>>>> 2015-05-19 21:17 GMT-08:00 Bandan Das <
> b...@makefile.in>:
> > >> >> >> >> >> >>>>>
> > >> >> >> >> >> >>>>>>
> > >> >> >> >> >> >>>>>>> On May 20, 2015, at 12:29 AM, Gabriel Laupre <
> > >> >> >> glau...@gmail.com
> > >> >> >> >> >
> > >> >> >> >> >> >>> wrote:
> > >> >> >> >> >> >>>>>>>
> > >> >> >> >> >> >>>>>>> Thank Bandan,
> > >> >> >> >> >> >>>>>>>
> > >> >> >> >> >> >>>>>>>
> > >> >> >> >> >> >>>>>>>> Assuming you are on an intel box, have you
> booted your
> > >> >> >> kernel
> > >> >> >> >> with
> > >> >> >> >> >> >>>>>> intel_iommu=on ?
> > >> >> >> >> >> >>>>>>> Yes, I have booted my kernel with the
> intel_iommu=on.
> > >> (I
> > >> >> >> don't
> > >> >> >> >> >> >>> remember
> > >> >> >> >> >> >>>>>> how to check that now though ^^)
> > >> >> >> >> >> >>>>>>>
> > >> >> >> >> >> >>>>>>>> Please paste the output of dmesg | grep -e DMAR
> -e
> > >> IOMMU
> > >> >> ?
> > >> >> >> >> >> >>>>>>> [root@peryn5 ~]# dmesg | grep -e DMAR -e IOMMU
> > >> >> >> >> >> >>>>>>> [ 0.000000] ACPI: DMAR 00000000bf79e0c0 00118
> (v01
> > >> >> AMI
> > >> >> >> >> >> OEMDMAR
> > >> >> >> >> >> >>>>>> 00000001 MSFT 00000097)
> > >> >> >> >> >> >>>>>>> [ 0.000000] Intel-IOMMU: enabled
> > >> >> >> >> >> >>>>>>> [ 0.039149] dmar: IOMMU 0: reg_base_addr
> fbffe000
> > >> ver
> > >> >> 1:0
> > >> >> >> cap
> > >> >> >> >> >> >>>>>> c90780106f0462 ecap f020f6
> > >> >> >> >> >> >>>>>>> [ 0.550126] IOMMU 0 0xfbffe000: using Queued
> > >> >> invalidation
> > >> >> >> >> >> >>>>>>> [ 0.550131] IOMMU: Setting RMRR:
> > >> >> >> >> >> >>>>>>> [ 0.550149] IOMMU: Setting identity map for
> device
> > >> >> >> >> 0000:00:1a.0
> > >> >> >> >> >> >>>>>> [0xbf7ec000 - 0xbf7fffff]
> > >> >> >> >> >> >>>>>>> [ 0.550184] IOMMU: Setting identity map for
> device
> > >> >> >> >> 0000:00:1a.1
> > >> >> >> >> >> >>>>>> [0xbf7ec000 - 0xbf7fffff]
> > >> >> >> >> >> >>>>>>> [ 0.550211] IOMMU: Setting identity map for
> device
> > >> >> >> >> 0000:00:1a.2
> > >> >> >> >> >> >>>>>> [0xbf7ec000 - 0xbf7fffff]
> > >> >> >> >> >> >>>>>>> [ 0.550241] IOMMU: Setting identity map for
> device
> > >> >> >> >> 0000:00:1a.7
> > >> >> >> >> >> >>>>>> [0xbf7ec000 - 0xbf7fffff]
> > >> >> >> >> >> >>>>>>> [ 0.550272] IOMMU: Setting identity map for
> device
> > >> >> >> >> 0000:00:1d.0
> > >> >> >> >> >> >>>>>> [0xbf7ec000 - 0xbf7fffff]
> > >> >> >> >> >> >>>>>>> [ 0.550302] IOMMU: Setting identity map for
> device
> > >> >> >> >> 0000:00:1d.1
> > >> >> >> >> >> >>>>>> [0xbf7ec000 - 0xbf7fffff]
> > >> >> >> >> >> >>>>>>> [ 0.550329] IOMMU: Setting identity map for
> device
> > >> >> >> >> 0000:00:1d.2
> > >> >> >> >> >> >>>>>> [0xbf7ec000 - 0xbf7fffff]
> > >> >> >> >> >> >>>>>>> [ 0.550358] IOMMU: Setting identity map for
> device
> > >> >> >> >> 0000:00:1d.7
> > >> >> >> >> >> >>>>>> [0xbf7ec000 - 0xbf7fffff]
> > >> >> >> >> >> >>>>>>> [ 0.550375] IOMMU: Setting identity map for
> device
> > >> >> >> >> 0000:00:1a.0
> > >> >> >> >> >> >>>>>> [0xec000 - 0xeffff]
> > >> >> >> >> >> >>>>>>> [ 0.550387] IOMMU: Setting identity map for
> device
> > >> >> >> >> 0000:00:1a.1
> > >> >> >> >> >> >>>>>> [0xec000 - 0xeffff]
> > >> >> >> >> >> >>>>>>> [ 0.550399] IOMMU: Setting identity map for
> device
> > >> >> >> >> 0000:00:1a.2
> > >> >> >> >> >> >>>>>> [0xec000 - 0xeffff]
> > >> >> >> >> >> >>>>>>> [ 0.550410] IOMMU: Setting identity map for
> device
> > >> >> >> >> 0000:00:1a.7
> > >> >> >> >> >> >>>>>> [0xec000 - 0xeffff]
> > >> >> >> >> >> >>>>>>> [ 0.550421] IOMMU: Setting identity map for
> device
> > >> >> >> >> 0000:00:1d.0
> > >> >> >> >> >> >>>>>> [0xec000 - 0xeffff]
> > >> >> >> >> >> >>>>>>> [ 0.550433] IOMMU: Setting identity map for
> device
> > >> >> >> >> 0000:00:1d.1
> > >> >> >> >> >> >>>>>> [0xec000 - 0xeffff]
> > >> >> >> >> >> >>>>>>> [ 0.550444] IOMMU: Setting identity map for
> device
> > >> >> >> >> 0000:00:1d.2
> > >> >> >> >> >> >>>>>> [0xec000 - 0xeffff]
> > >> >> >> >> >> >>>>>>> [ 0.550458] IOMMU: Setting identity map for
> device
> > >> >> >> >> 0000:00:1d.7
> > >> >> >> >> >> >>>>>> [0xec000 - 0xeffff]
> > >> >> >> >> >> >>>>>>> [ 0.550471] IOMMU: Prepare 0-16MiB unity
> mapping for
> > >> >> LPC
> > >> >> >> >> >> >>>>>>> [ 0.550483] IOMMU: Setting identity map for
> device
> > >> >> >> >> 0000:00:1f.0
> > >> >> >> >> >> >>> [0x0
> > >> >> >> >> >> >>>>>> - 0xffffff]
> > >> >> >> >> >> >>>>>>>
> > >> >> >> >> >> >>>>>>
> > >> >> >> >> >> >>>>>> Yeah, this looks ok. Actually, taking a second
> look, I
> > >> >> can’t
> > >> >> >> >> think
> > >> >> >> >> >> of
> > >> >> >> >> >> >>>>>> anyway how this could be related to file
> permissions on
> > >> >> >> >> >> /dev/vfio/vfio.
> > >> >> >> >> >> >>>>>>
> > >> >> >> >> >> >>>>>>>> Why does opening /dev/vfio/vfio fail ? Can you
> please
> > >> >> >> confirm
> > >> >> >> >> that
> > >> >> >> >> >> >>> you
> > >> >> >> >> >> >>>>>> have read/write permissions as the user you are
> trying
> > >> to
> > >> >> run
> > >> >> >> ?
> > >> >> >> >> >> >>>>>>> [root@peryn5 ~]# cd /dev/vfio/
> > >> >> >> >> >> >>>>>>> [root@peryn5 vfio]# ls -la | grep vfio
> > >> >> >> >> >> >>>>>>> crw-rw-rw- 1 root root 10, 196 May 18 11:54 vfio
> > >> >> >> >> >> >>>>>>> The right should be okay I guess.
> > >> >> >> >> >> >>>>>>>
> > >> >> >> >> >> >>>>>> Yes, indeed it is. What distro is this ? Do you
> have
> > >> >> SELinux
> > >> >> >> or
> > >> >> >> >> any
> > >> >> >> >> >> >>> other
> > >> >> >> >> >> >>>>>> security feature enabled ? Can you please verify
> that
> > >> the
> > >> >> file
> > >> >> >> >> has a
> > >> >> >> >> >> >>>>>> appropriate label if SELinux is enabled ? (ls -lZ
> > >> >> >> /dev/vfio/vfio)
> > >> >> >> >> >> >>>>>>
> > >> >> >> >> >> >>>>>> Bandan
> > >> >> >> >> >> >>>>>>>
> > >> >> >> >> >> >>>>>>>
> > >> >> >> >> >> >>>>>>>
> > >> >> >> >> >> >>>>>>> 2015-05-19 18:54 GMT-08:00 Bandan Das <
> b...@makefile.in
> > >> >:
> > >> >> >> >> >> >>>>>>>
> > >> >> >> >> >> >>>>>>> Hello Gabriel,
> > >> >> >> >> >> >>>>>>>
> > >> >> >> >> >> >>>>>>>> On May 19, 2015, at 8:03 PM, Gabriel Laupre <
> > >> >> >> glau...@gmail.com
> > >> >> >> >> >
> > >> >> >> >> >> >>> wrote:
> > >> >> >> >> >> >>>>>>>>
> > >> >> >> >> >> >>>>>>>> Hello everyone,
> > >> >> >> >> >> >>>>>>>>
> > >> >> >> >> >> >>>>>>>> I am using a Centos 7.1 machine with the kernel
> > >> >> 3.10.229. I
> > >> >> >> >> want
> > >> >> >> >> >> to
> > >> >> >> >> >> >>>>>> use my host with SR-IOV to use a virtual function
> on my
> > >> >> NIC as
> > >> >> >> >> the
> > >> >> >> >> >> >>> vNIC in
> > >> >> >> >> >> >>>>>> my new VM.
> > >> >> >> >> >> >>>>>>>>
> > >> >> >> >> >> >>>>>>>> I have an instance started with a old NIC using
> > >> macvtap
> > >> >> >> that I
> > >> >> >> >> >> >>> want to
> > >> >> >> >> >> >>>>>> change. I am using the
> > >> >> >> >> >> >>>>>>>> virsh edit instance-00000034
> > >> >> >> >> >> >>>>>>>> command to edit the XML configuration to add the
> new
> > >> >> device
> > >> >> >> I
> > >> >> >> >> want
> > >> >> >> >> >> >>> to
> > >> >> >> >> >> >>>>>> attach.
> > >> >> >> >> >> >>>>>>> …
> > >> >> >> >> >> >>>>>>> Assuming you are on an intel box, have you booted
> your
> > >> >> kernel
> > >> >> >> >> with
> > >> >> >> >> >> >>>>>> intel_iommu=on ?
> > >> >> >> >> >> >>>>>>> Please paste the output of dmesg | grep -e DMAR -e
> > >> IOMMU ?
> > >> >> >> >> >> >>>>>>>
> > >> >> >> >> >> >>>>>>>> When I try to reboot the VM I get this error:
> > >> >> >> >> >> >>>>>>>> Error starting domain: internal error: process
> exited
> > >> >> while
> > >> >> >> >> >> >>> connecting
> > >> >> >> >> >> >>>>>> to monitor: 2015-05-19T21:46:21.935043Z qemu-kvm:
> > >> -device
> > >> >> >> >> >> >>>>>>
> vfio-pci,host=04:10.4,id=hostdev0,bus=pci.0,addr=0x3:
> > >> vfio:
> > >> >> >> >> failed
> > >> >> >> >> >> to
> > >> >> >> >> >> >>> open
> > >> >> >> >> >> >>>>>> /dev/vfio/vfio: Operation not permitted
> > >> >> >> >> >> >>>>>>> Why does opening /dev/vfio/vfio fail ? Can you
> please
> > >> >> confirm
> > >> >> >> >> that
> > >> >> >> >> >> >>> you
> > >> >> >> >> >> >>>>>> have read/write permissions as the user you are
> trying
> > >> to
> > >> >> run
> > >> >> >> ?
> > >> >> >> >> >> >>>>>>>
> > >> >> >> >> >> >>>>>>>> 2015-05-19T21:46:21.935091Z qemu-kvm: -device
> > >> >> >> >> >> >>>>>>
> vfio-pci,host=04:10.4,id=hostdev0,bus=pci.0,addr=0x3:
> > >> vfio:
> > >> >> >> >> failed
> > >> >> >> >> >> to
> > >> >> >> >> >> >>> setup
> > >> >> >> >> >> >>>>>> container for group 24
> > >> >> >> >> >> >>>>>>>> 2015-05-19T21:46:21.935107Z qemu-kvm: -device
> > >> >> >> >> >> >>>>>>
> vfio-pci,host=04:10.4,id=hostdev0,bus=pci.0,addr=0x3:
> > >> vfio:
> > >> >> >> >> failed
> > >> >> >> >> >> to
> > >> >> >> >> >> >>> get
> > >> >> >> >> >> >>>>>> group 24
> > >> >> >> >> >> >>>>>>>> 2015-05-19T21:46:21.935135Z qemu-kvm: -device
> > >> >> >> >> >> >>>>>>
> vfio-pci,host=04:10.4,id=hostdev0,bus=pci.0,addr=0x3:
> > >> >> Device
> > >> >> >> >> >> >>> initialization
> > >> >> >> >> >> >>>>>> failed.
> > >> >> >> >> >> >>>>>>>> 2015-05-19T21:46:21.935157Z qemu-kvm: -device
> > >> >> >> >> >> >>>>>>
> vfio-pci,host=04:10.4,id=hostdev0,bus=pci.0,addr=0x3:
> > >> >> Device
> > >> >> >> >> >> 'vfio-pci'
> > >> >> >> >> >> >>>>>> could not be initialized
> > >> >> >> >> >> >>>>>>>>
> > >> >> >> >> >> >>>>>>>> total Trace here: http://sprunge.us/XZFB
> > >> >> >> >> >> >>>>>>>>
> > >> >> >> >> >> >>>>>>>> Any idea how to fix that?
> > >> >> >> >> >> >>>>>>>>
> > >> >> >> >> >> >>>>>>>> Thank you very much :)
> > >> >> >> >> >> >>>>>>>>
> > >> >> >> >> >> >>>>>>>> Gabriel
> > >> >> >> >> >> >>>>>>>>
> > >> >> >> >> >> >>>>>>>
> > >> >> >> >> >> >>>>>>>
> > >> >> >> >> >> >>>>>>
> > >> >> >> >> >> >>>>>>
> > >> >> >> >> >> >>>
> > >> >> >> >> >> >
> > >> >> >> >> >>
> > >> >> >> >>
> > >> >> >>
> > >> >>
> > >>
> >
> >
>
>
