Re: [libvirt] [RFC PATCH v2 REBASE 00/18] Introduce vGPU mdev framework to libvirt

[Chen, Xiaoguang]

>-Original Message-
>From: Erik Skultety [mailto:eskul...@redhat.com]
>Sent: Thursday, March 16, 2017 10:41 PM
>To: Chen, Xiaoguang <xiaoguang.c...@intel.com>
>Cc: libvir-list@redhat.com; He, Yongli <yongli...@intel.com>
>Subject: Re: [libvirt] [RFC PATCH v2 REBASE 00/18] Introduce vGPU mdev
>framework to libvirt
>
>> [2] 2005
>> ubuntu@z-nuc-11:~/vgpu-meta/libvirt-stage$
>> ***
>> start libvirt-d
>> 2017-03-09 19:04:57.211+: 2059: info : libvirt version: 3.1.0
>> 2017-03-09 19:04:57.211+: 2059: info : hostname: z-nuc-11.maas
>> 2017-03-09 19:04:57.211+: 2059: error : qemuMonitorOpenUnix:367 :
>> failed to connect to monitor socket: No such process
>> 2017-03-09 19:04:57.213+: 2059: error :
>> virMediatedDeviceGetIOMMUGroupDev:153 : internal error: IOMMU group
>> file
>> /sys/bus/mdev/devices/894f3983-1a36-42b3-b52c-
>1024aca216be/iommu_group
>> is not a symlink
>
>When I saw this error message for the first time in the original thread, I got
>confused, since this just checks whether the symlink exists, if it doesn't, 
>the vfio
>device probably also doesn't exist (but take this with a grain of salt, I 
>haven't
>investigated that deep) and libvirt needs it to pass it onto qemu command 
>line. I
>hit this issue once by accident in the past and at that time I didn't 
>understand
>what caused it, but after a reboot it was gone.
>So seeing it here it caught my eye and I investigated it last week. What I 
>found
>out was that it's caused by the vfio-mdev module not being loaded automatically
>as a dependency. I solved it by autoloading the module on system boot. So this 
>is
>not a libvirt issue, but just for a reference, there is a BZ on this [1].
>
>> 2017-03-09 19:04:57.213+: 2003: info : libvirt version: 3.1.0
>> 2017-03-09 19:04:57.213+: 2003: info : hostname: z-nuc-11.maas
>> 2017-03-09 19:04:57.213+: 2003: error : virNetSocketReadWire:1800 :
>> End of file while reading data: Input/output error
>
>I suppose this corresponds to the problem above, do you hit this error if you 
>work
>around the vfio-mdev module problem described above?
Based on the KVMGT setup guide we should add kvmgt, vfio-iommu-type1 and 
vfio-mdev modules into initfamfs.
So I don't think it is the same problem. But we will double confirm it later. 
Yong li is on vacation this week when he come back we will do the test again.

>
>>
>> the screen call trace while start the VM (same for Ubuntu, Win10 etc)
>> ==
>>
>> ubuntu@z-nuc-11:~/vgpu-meta/libvirt-stage$ myvirsh start vgpu-ubuntu
>> 2017-03-09 19:06:50.483+: 2232: info : libvirt version: 3.1.0
>> 2017-03-09 19:06:50.483+: 2232: info : hostname: z-nuc-11.maas
>> 2017-03-09 19:06:50.483+: 2232: warning : qemuDomainObjTaint:4056 :
>> Domain id=1 name='vgpu-ubuntu'
>> uuid=972b5e38-0437-11e7-8f97-d36dba74552d
>> is tainted: high-privileges
>> 2017-03-09 19:06:50.819+: 2204: info : libvirt version: 3.1.0
>> 2017-03-09 19:06:50.819+: 2232: warning :
>> virDomainAuditHostdev:456
>> : Unexpected hostdev type while encoding audit message: 4
>
>This one's interesting, again, are you able to hit the error when you work 
>around
>the missing vfio-mdev module? I'll have a look at this if you actually can hit 
>the
>error, even if the XML is correct.
>
>I posted v3 of the series and also created a new branch 'mdev-next' on my 
>github
>[2]. I dropped the attribute 'type' from the source address element, so follow 
>the
>example in the updated docs.
Will test the v3 when yong li back from vacation.


>
>Thanks for giving it a try.
>Erik
>
>[1] https://bugzilla.redhat.com/show_bug.cgi?id=1420572
>[2] https://github.com/eskultety/libvirt/commits/mdev-next

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list

Re: [libvirt] [RFC PATCH v2 REBASE 00/18] Introduce vGPU mdev framework to libvirt

[Chen, Xiaoguang]

>-Original Message-
>From: sendmail [mailto:justsendmailnothinge...@gmail.com] On Behalf Of Laine
>Stump
>Sent: Thursday, March 16, 2017 10:01 PM
>To: libvir-list@redhat.com
>Cc: Chen, Xiaoguang <xiaoguang.c...@intel.com>; Erik Skultety
><eskul...@redhat.com>; He, Yongli <yongli...@intel.com>
>Subject: Re: [libvirt] [RFC PATCH v2 REBASE 00/18] Introduce vGPU mdev
>framework to libvirt
>
>On 03/16/2017 03:17 AM, Chen, Xiaoguang wrote:
>> the screen call trace while start the VM (same for Ubuntu, Win10 etc)
>> ==
>>
>> ubuntu@z-nuc-11:~/vgpu-meta/libvirt-stage$ myvirsh start vgpu-ubuntu
>> 2017-03-09 19:06:50.483+: 2232: info : libvirt version: 3.1.0
>> 2017-03-09 19:06:50.483+: 2232: info : hostname: z-nuc-11.maas
>> 2017-03-09 19:06:50.483+: 2232: warning : qemuDomainObjTaint:4056 :
>> Domain id=1 name='vgpu-ubuntu'
>> uuid=972b5e38-0437-11e7-8f97-d36dba74552d
>> is tainted: high-privileges
>
>I haven't considered any of the rest of the log yet, but this caught my eye on 
>a
>first pass - "high-privileges" means that you're running qemu as root, so your 
>test
>is bypassing several issues that could cause vfio device assignment to fail on 
>a
>"standard" system.
What do you mean for 'cause vfio device assignment to fail on a standard 
system'?

> It shouldn't be necessary to run qemu as root in order for
>device assignment to work. Is there some specific reason that you're doing it 
>this
>way? (I'm guessing that you've set "user = root" in /etc/libvirt/qemu.conf)
No. we will test the v3 using a non-root user.


--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list

Re: [libvirt] [RFC PATCH v2 REBASE 00/18] Introduce vGPU mdev framework to libvirt

[Chen, Xiaoguang]

On behalf of Yongli, He who tested these series with KVMGT vGPU mdev.

Hi, Erik

Here is the libvirt testing result. in general speaking,  it works well.  while 
start libvirt-d and starting the vm, there are some call traces, i attached 
them in the very end of this mail.  this mail will be the  test-by content 
later.


Test env summary
==
 0.  Test Bed hardware summary
cat /proc/cpuinfo
 vendor_id: GenuineIntel
 cpu family: 6
 model: 78
 model name: Intel(R) Core(TM) i5-6260U CPU @ 1.80GHz
 stepping: 3
 microcode: 0x84
 physical id: 0
 cpu cores: 2
 wp: yes
 flags: fpu vme de pse tsc msr pae mce cx8 apic 
sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe 
syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good 
nopl xtopology nonstop_tsc aperfmperf tsc_known_freq pni pclmulqdq dtes64 
monitor ds_cpl vmx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid sse4_1 sse4_2 
x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 
3dnowprefetch epb intel_pt tpr_shadow vnmi flexpriority ept vpid fsgsbase 
tsc_adjust bmi1 avx2 smep bmi2 erms invpcid mpx rdseed adx smap clflushopt 
xsaveopt xsavec xgetbv1 xsaves dtherm ida arat pln pts hwp hwp_notify 
hwp_act_window hwp_epp
bugs:


 1. test steps and software version following this guide, 
https://github.com/01org/gvt-linux/wiki/GVTg_Setup_Guide#322-build-kernel-source

 2. Host env
  uname -a
 Linux z-nuc-11 4.10.0-vgt #1 SMP Thu Mar 9 15:55:49 CST
2017 x86_64 x86_64 x86_64 GNU/Linux
 cat /etc/lsb-release
 DISTRIB_ID=Ubuntu
 DISTRIB_RELEASE=14.04
 DISTRIB_CODENAME=trusty
 DISTRIB_DESCRIPTION="Ubuntu 14.04.4 LTS"

  kernel build from source:
  git describe
  gvt-fixes-2017-01-25-1560-gcedbc34
 qemu build from source:
qemu-system-x86_64 --version

 QEMU emulator version 2.8.50 (v2.8.0-2059-geba44e9-dirty)
 Copyright (c) 2003-2017 Fabrice Bellard and the QEMU 
Project developers

   libvirt, use your branch https://github.com/eskultety/libvirt.git
  git describe
 v1.2.17-rc2-5530-g202402e
 git branch
   master
   * mdev-v3

   3. guest xml template, refer to attachment

   4. related scripts, refer to attachment
 myvirsh:  source build libvirt wrapper
 gdb-libvirtd:  libvirtd start up wrapper


the screen call trace while start the virtlogd 
==

ubuntu@z-nuc-11:~/vgpu-meta/libvirt-stage$  ./libvirt/gdb-libvirt &

***
start virtlogd

[2] 2005
ubuntu@z-nuc-11:~/vgpu-meta/libvirt-stage$
***
start libvirt-d
2017-03-09 19:04:57.211+: 2059: info : libvirt version: 3.1.0
2017-03-09 19:04:57.211+: 2059: info : hostname: z-nuc-11.maas
2017-03-09 19:04:57.211+: 2059: error : qemuMonitorOpenUnix:367 : 
failed to connect to monitor socket: No such process
2017-03-09 19:04:57.213+: 2059: error : 
virMediatedDeviceGetIOMMUGroupDev:153 : internal error: IOMMU group file 
/sys/bus/mdev/devices/894f3983-1a36-42b3-b52c-1024aca216be/iommu_group
is not a symlink
2017-03-09 19:04:57.213+: 2003: info : libvirt version: 3.1.0
2017-03-09 19:04:57.213+: 2003: info : hostname: z-nuc-11.maas
2017-03-09 19:04:57.213+: 2003: error : virNetSocketReadWire:1800 : 
End of file while reading data: Input/output error



the screen call trace while start the VM (same for Ubuntu, Win10 etc) 
==

ubuntu@z-nuc-11:~/vgpu-meta/libvirt-stage$ myvirsh start vgpu-ubuntu
2017-03-09 19:06:50.483+: 2232: info : libvirt version: 3.1.0
2017-03-09 19:06:50.483+: 2232: info : hostname: z-nuc-11.maas
2017-03-09 19:06:50.483+: 2232: warning : qemuDomainObjTaint:4056 : 
Domain id=1 name='vgpu-ubuntu' uuid=972b5e38-0437-11e7-8f97-d36dba74552d
is tainted: high-privileges
2017-03-09 19:06:50.819+: 2204: info : libvirt version: 3.1.0
2017-03-09 19:06:50.819+: 2232: warning : virDomainAuditHostdev:456
: Unexpected hostdev type while encoding audit message: 4
2017-03-09 19:06:50.819+: 2204: info : hostname: z-nuc-11.maas
2017-03-09 19:06:50.819+: 2204: error : virNetSocketReadWire:1800 : 
End of file while reading data: Input/output error Domain vgpu-ubuntu started

Tested-by: Yongli, He 


>-Original Message-
>From: libvir-list-boun...@redhat.com [mailto:libvir-list-boun...@redhat.com]
>On Behalf Of Erik Skultety
>Sent: Monday, February 20,