Hey Aaron, sorry for bugging you so often, but this is very odd...
I'm not sure what has changed, but using a new env, I am now getting
operation not permitted for /dev/vfio/noiommu-0.
Here's sestatus:
*[root@lago-network-suite-master-host-0 ~]# sestatus*
*SELinux status: enabled*
*SELinuxfs mount: /sys/fs/selinux*
*SELinux root directory: /etc/selinux*
*Loaded policy name: targeted*
*Current mode: permissive*
*Mode from config file: enforcing*
*Policy MLS status: enabled*
*Policy deny_unknown status: allowed*
*Max kernel policy version: 28*
Trying to add dpdk port:
*[root@lago-network-suite-master-host-0 ~]# ovs-vsctl add-port br0 dpdk-p0
-- set Interface dpdk-p0 type=dpdk options:dpdk-devargs=0000:00:04.0*
*ovs-vsctl: Error detected while setting up 'dpdk-p0': Error attaching
device '0000:00:04.0' to DPDK. See ovs-vswitchd log for details.*
*ovs-vsctl: The default log directory is "/var/log/openvswitch".*
*[root@lago-network-suite-master-host-0 ~]# cat
/var/log/openvswitch/ovs-vswitchd.log | grep permitted*
*2018-05-08T14:29:02.755Z|00227|dpdk|ERR|EAL: Cannot open
/dev/vfio/noiommu-0: Operation not permitted*
*2018-05-08T14:29:22.532Z|00239|dpdk|ERR|EAL: Cannot open
/dev/vfio/noiommu-0: Operation not permitted*
*2018-05-08T14:29:22.549Z|00250|dpdk|ERR|EAL: Cannot open
/dev/vfio/noiommu-0: Operation not permitted*
*2018-05-08T14:29:23.525Z|00023|dpdk|ERR|EAL: Cannot open
/dev/vfio/noiommu-0: Operation not permitted*
*2018-05-08T14:29:23.573Z|00082|dpdk|ERR|EAL: Cannot open
/dev/vfio/noiommu-0: Operation not permitted*
*2018-05-08T14:29:23.595Z|00105|dpdk|ERR|EAL: Cannot open
/dev/vfio/noiommu-0: Operation not permitted*
*2018-05-08T14:29:23.712Z|00124|dpdk|ERR|EAL: Cannot open
/dev/vfio/noiommu-0: Operation not permitted*
*2018-05-08T14:29:23.721Z|00135|dpdk|ERR|EAL: Cannot open
/dev/vfio/noiommu-0: Operation not permitted*
*2018-05-08T14:29:23.725Z|00146|dpdk|ERR|EAL: Cannot open
/dev/vfio/noiommu-0: Operation not permitted*
*2018-05-08T14:32:32.713Z|00160|dpdk|ERR|EAL: Cannot open
/dev/vfio/noiommu-0: Operation not permitted*
audit.log snippet:
*[root@lago-network-suite-master-host-0 ~]# tail /var/log/audit/audit.log |
grep openvswitch_t*
*type=AVC msg=audit(1525789763.711:1210): avc: denied { open } for
pid=22757 comm="ovs-vswitchd" path="/dev/vfio/noiommu-0" dev="devtmpfs"
ino=33747 scontext=system_u:system_r:openvswitch_t:s0
tcontext=system_u:object_r:device_t:s0 tclass=chr_file*
*type=SYSCALL msg=audit(1525789763.711:1210): arch=c000003e syscall=2
success=no exit=-1 a0=7fff0045bfa0 a1=2 a2=7fff0045bfb3 a3=0 items=0 ppid=1
pid=22757 auid=4294967295 uid=994 gid=1000 euid=994 suid=994 fsuid=994
egid=1000 sgid=1000 fsgid=1000 tty=(none) ses=4294967295
comm="ovs-vswitchd" exe="/usr/sbin/ovs-vswitchd"
subj=system_u:system_r:openvswitch_t:s0 key=(null)*
permissions:
*[root@lago-network-suite-master-host-0 ~]# ps aux | grep ovs-vswitchd*
*openvsw+ 22757 0.3 5.1 1122780 96568 ? S<Lsl 10:29 0:01
ovs-vswitchd unix:/var/run/openvswitch/db.sock -vconsole:emer -vsyslog:err
-vfile:info --mlockall --user openvswitch:hugetlbfs --no-chdir
--log-file=/var/log/openvswitch/ovs-vswitchd.log
--pidfile=/var/run/openvswitch/ovs-vswitchd.pid --detach*
*root 22933 0.0 0.0 112660 972 pts/0 S+ 10:34 0:00 grep
--color=auto ovs-vswitchd*
*[root@lago-network-suite-master-host-0 ~]# ls -lah /dev/vfio*
*total 0*
*drwxr-xr-x. 2 root root 80 May 8 06:34 .*
*drwxr-xr-x. 19 root root 3.2K May 8 06:34 ..*
*crw-rw----. 1 root hugetlbfs 244, 0 May 8 06:34 noiommu-0*
*crw-rw-rw-. 1 root root 10, 196 May 8 06:34 vfio*
*[root@lago-network-suite-master-host-0 ~]# *
On Tue, May 8, 2018 at 4:14 PM, Aaron Conole <acon...@redhat.com> wrote:
> Leon Goldberg <lgold...@redhat.com> writes:
>
> > On Mon, May 7, 2018 at 9:00 PM, Aaron Conole <acon...@redhat.com> wrote:
> >
> > Leon Goldberg <lgold...@redhat.com> writes:
> >
> > > I stand correct, I was not using permissive mode. With permissive
> mode, noiommu-0
> > issue seems to
> > > be resolved, however:
> >
> > Cool. I caught other issues turning on the -DB flag... although maybe
> > that was using enforcing mode as well.
> >
> > > type=AVC msg=audit(1525707587.009:447): avc: denied { remove_name }
> for
> > pid=4497
> > > comm="qemu-kvm" name="vhost-user-5" dev="vda3" ino=8742121
> > > scontext=system_u:system_r:svirt_t:s0:c794,c950
> > tcontext=unconfined_u:object_r:default_t:s0
> > > tclass=dir
> > > type=AVC msg=audit(1525707587.009:447): avc: denied { unlink } for
> pid=4497
> > > comm="qemu-kvm" name="vhost-user-5" dev="vda3" ino=8742121
> > > scontext=system_u:system_r:svirt_t:s0:c794,c950
> > tcontext=system_u:object_r:default_t:s0
> > > tclass=sock_file
> > > type=AVC msg=audit(1525707587.009:448): avc: denied { add_name }
> for
> > pid=4497
> > > comm="qemu-kvm" name="vhost-user-5"
> > scontext=system_u:system_r:svirt_t:s0:c794,c950
> > > tcontext=unconfined_u:object_r:default_t:s0 tclass=dir
> > > type=AVC msg=audit(1525707587.009:448): avc: denied { create } for
> pid=4497
> > > comm="qemu-kvm" name="vhost-user-5"
> > scontext=system_u:system_r:svirt_t:s0:c794,c950
> > > tcontext=system_u:object_r:default_t:s0 tclass=sock_file
> > >
> > > Still occurs.
> > >
> > > OVS log shows:
> > >
> > > 2018-05-07T15:27:15.059Z|00072|dpdk|INFO|VHOST_CONFIG: vhost-user
> client:
> > socket created, fd:
> > > 55
> > > 2018-05-07T15:27:15.059Z|00073|netdev_dpdk|INFO|vHost User device
> > 'dpdkvhostclient1' created
> > > in 'client' mode, using client socket '/vhostusers/vhost-user-5'
> > > 2018-05-07T15:27:15.062Z|00074|dpdk|WARN|VHOST_CONFIG: failed to
> connect to
> > > /vhostusers/vhost-user-5: Permission denied
> >
> > Does that directory exist? What are the permissions? What are the
> > permissions of the sock file that exist in that directory?
> >
> > [root@lago-network-suite-master-host-0 ~]# ll /vhostusers/
> > total 0
> > srwxrwxr-x. 1 qemu kvm 0 May 7 11:55 vhost-user-5
>
> That looks like a problem. Have a look at /etc/libvirt/qemu.conf and
> change the group to hugetlbfs, restart libvirtd, and see if that allows
> ovs+libvirt to proceed.
>
> More information could be available at:
> https://developers.redhat.com/blog/2018/03/23/non-root-open-vswitch-rhel/
>
> -Aaron
>
> >
> > > On Mon, May 7, 2018 at 6:21 PM, Leon Goldberg <lgold...@redhat.com>
> wrote:
> > >
> > > Aha, indeed, I see:
> > >
> > > type=AVC msg=audit(1525649015.102:1305): avc: denied { open } for
> > pid=12993
> > > comm="ovs-vswitchd" path="/dev/vfio/noiommu-0" dev="devtmpfs"
> ino=708920
> > > scontext=system_u:system_r:openvswitch_t:s0
> > tcontext=system_u:object_r:device_t:s0
> > > tclass=chr_file
> > > type=AVC msg=audit(1525649177.311:1326): avc: denied { open } for
> > pid=13241
> > > comm="ovs-vswitchd" path="/dev/vfio/noiommu-0" dev="devtmpfs"
> ino=708920
> > > scontext=system_u:system_r:openvswitch_t:s0
> > tcontext=system_u:object_r:device_t:s0
> > > tclass=chr_file
> > >
> > > and I'm using permissive mode.
> > >
> > > I also see:
> > >
> > > [root@lago-network-suite-master-host-0 ~]# cat
> /var/log/audit/audit.log | grep
> > vhost-user-5
> > > type=AVC msg=audit(1525636067.061:757): avc: denied { create }
> for pid=7533
> > > comm="qemu-kvm" name="vhost-user-5"
> > scontext=system_u:system_r:svirt_t:s0:c423,c510
> > > tcontext=system_u:object_r:tmp_t:s0 tclass=sock_file
> > > type=AVC msg=audit(1525648910.361:1276): avc: denied { add_name }
> for
> > pid=12734
> > > comm="qemu-kvm" name="vhost-user-5"
> > scontext=system_u:system_r:svirt_t:s0:c245,c301
> > > tcontext=unconfined_u:object_r:default_t:s0 tclass=dir
> > > type=AVC msg=audit(1525648910.361:1276): avc: denied { create }
> for
> > pid=12734
> > > comm="qemu-kvm" name="vhost-user-5"
> > scontext=system_u:system_r:svirt_t:s0:c245,c301
> > > tcontext=system_u:object_r:default_t:s0 tclass=sock_file
> > > type=AVC msg=audit(1525648979.442:1290): avc: denied {
> remove_name } for
> > pid=12822
> > > comm="qemu-kvm" name="vhost-user-5" dev="vda3" ino=8742121
> > > scontext=system_u:system_r:svirt_t:s0:c515,c819
> > tcontext=unconfined_u:object_r:default_t:s0
> > > tclass=dir
> > > type=AVC msg=audit(1525648979.442:1290): avc: denied { unlink }
> for
> > pid=12822
> > > comm="qemu-kvm" name="vhost-user-5" dev="vda3" ino=8742121
> > > scontext=system_u:system_r:svirt_t:s0:c515,c819
> > tcontext=system_u:object_r:default_t:s0
> > > tclass=sock_file
> > > type=AVC msg=audit(1525648979.442:1291): avc: denied { add_name }
> for
> > pid=12822
> > > comm="qemu-kvm" name="vhost-user-5"
> > scontext=system_u:system_r:svirt_t:s0:c515,c819
> > > tcontext=unconfined_u:object_r:default_t:s0 tclass=dir
> > > type=AVC msg=audit(1525648979.442:1291): avc: denied { create }
> for
> > pid=12822
> > > comm="qemu-kvm" name="vhost-user-5"
> > scontext=system_u:system_r:svirt_t:s0:c515,c819
> > > tcontext=system_u:object_r:default_t:s0 tclass=sock_file
> > >
> > > This is my vhostuser client.
> > >
> > > On Mon, May 7, 2018 at 4:39 PM, Aaron Conole <acon...@redhat.com>
> wrote:
> > >
> > > Leon Goldberg <lgold...@redhat.com> writes:
> > >
> > > > On Fri, May 4, 2018 at 10:19 PM, Aaron Conole <acon...@redhat.com>
> wrote:
> > > >
> > > > Leon Goldberg <lgold...@redhat.com> writes:
> > > >
> > > > > Hi list,
> > > > >
> > > > > I'm trying to integrate ovs-dpdk into oVirt. For testing
> purposes, I'm
> > > > > writing a test that looks to run a VM on top of a dpdk port.
> > > > >
> > > > > The testing environment consists of nested virtualization:
> > > > >
> > > > > Physical machine -> Jenkins CI VM -> Target VM
> > > > >
> > > > > The test merely looks to see that the various components are
> properly
> > > > > configured for the real world. For that purpose, I'm using
> NOIOMMU mode of
> > > > > VFIO.
> > > > >
> > > > > The select virtio device fails to to be attached to dpdk, and I
> suspect it
> > > > > is due to $subject.
> > > > >
> > > > > Here are the CI logs[1]. I see some other red lights, but
> $subject seems
> > > > > the brightest.
> > > >
> > > > Can you provide:
> > > >
> > > > $ ps aux | grep ovs-vswitchd
> > > > $ ls -lah /dev/vfio
> > > >
> > > > Hey Aaron,
> > > >
> > > > Here it is:
> > > >
> > > > [root@lago-network-suite-master-host-0 ~]# ps aux | grep
> ovs-vswitchd
> > > > openvsw+ 840 0.6 6.2 1273732 116716 ? S<Lsl 07:28 0:10
> ovs-vswitchd
> > > > unix:/var/run/openvswitch/db.sock -vconsole:emer -vsyslog:err
> -vfile:info --mlockall
> > --user
> > > > openvswitch:hugetlbfs --no-chdir --log-file=/var/log/
> openvswitch/ovs-vswitchd.log
> > > > --pidfile=/var/run/openvswitch/ovs-vswitchd.pid --detach
> > > > root 4425 0.0 0.0 112660 976 pts/0 R+ 07:55 0:00
> grep --color=auto
> > > ovs-vswitchd
> > > >
> > > > [root@lago-network-suite-master-host-0 ~]# ls -lah /dev/vfio
> > > > total 0
> > > > drwxr-xr-x. 2 root root 80 May 6 07:28 .
> > > > drwxr-xr-x. 19 root root 3.2K May 6 07:28 ..
> > > > crw-rw----. 1 root hugetlbfs 244, 0 May 6 07:28 noiommu-0
> > > > crw-rw-rw-. 1 root root 10, 196 May 6 07:28 vfio
> > >
> > > Okay - that looks like it should be okay.
> > >
> > > Can you check if there are any selinux violations in audit.log
> > > (specifically from the openvswitch_t domain)? Maybe there is a
> missing
> > > selinux policy directive.
> > >
> > > > Just want to see if there's a disconnect between the userid for
> ovs
> > > > and the permissions on the vfio file. If that's the case, we may
> need
> > > > to update the vfio rules.
> > > >
> > > > > Any tips will be greatly appreciated!
> > > > >
> > > > > Thanks,
> > > > > Leon
> > > > >
> > > > > [1]
> > > > >
> > > >
> > >
> > http://jenkins.ovirt.org/job/ovirt-system-tests_standard-
> check-patch/642/artifact/exported-artifacts/check-
> patch.network_suite_master.el7.x86_64/tests.test_dpdk/
> lago-network-suite-master-host-0/_var_log/openvswitch/ovs-vswitchd.log
> >
> > >
> > > >
> > > > >
> > > >
> > >
> > <http://jenkins.ovirt.org/job/ovirt-system-tests_standard-
> check-patch/642/artifact/exported-artifacts/check-
> patch.network_suite_master.el7.x86_64/tests.test_dpdk/
> lago-network-suite-master-host-0/_var_log/openvswitch/ovs-vswitchd.log>
> >
> > >
> > > >
> > > > > _______________________________________________
> > > > > dev mailing list
> > > > > d...@openvswitch.org
> > > > > https://mail.openvswitch.org/mailman/listinfo/ovs-dev
>
_______________________________________________
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev
