Re: [Xen-devel] PCI passthrough will cause unknown device appearance in device manager of Windows OS via xenstore keys

2017-04-13 Thread Konrad Rzeszutek Wilk
On April 13, 2017 4:41:03 AM EDT, Wei Liu  wrote:
>On Wed, Apr 12, 2017 at 04:21:42PM -0400, Konrad Rzeszutek Wilk wrote:
>> On Wed, Apr 12, 2017 at 04:46:33PM +0100, Wei Liu wrote:
>> > On Mon, Apr 10, 2017 at 09:43:13AM -0400, Konrad Rzeszutek Wilk
>wrote:
>> > [...]
>> > > 
>> > > .. Except that we need some way of doing FLR and Pciback
>> > > is the one doing it.
>> > > 
>> > > The right way would be to expand pciback to support the do_flr
>ioctl
>> > > and combine it with your patch.
>> > > 
>> > 
>> > OK. Does that mean the bug is in Linux and I should just take
>Ross's
>> 
>> Please don't.
>> 
>> Or if you would like I can do
>> 
>> Nacked-by: Konrad Rzeszutek Wilk 
>> 
>
>Please correct me if I'm wrong: current libxl code is broken in one way
>and
>taking that patch means to break it in another way?
>
>What is the way forward? Do you want me to wait until those kernel
>patches are upstreamed?

Yes, as we cannot allow PCI pass through devices to not have an FLR or proper 
configuration register reset when a passing a device to a guest (or from one 
guest to another).

I can figure something up next week on how to restructure the pciback module.
>
>Wei.
>
>> 
>> > patch as-is for 4.9?
>> > 
>> > Wei.


Thanks!

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel


Re: [Xen-devel] PCI passthrough will cause unknown device appearance in device manager of Windows OS via xenstore keys

2017-04-13 Thread Stefano Stabellini
On Thu, 13 Apr 2017, Roger Pau Monné wrote:
> On Thu, Apr 13, 2017 at 09:41:03AM +0100, Wei Liu wrote:
> > On Wed, Apr 12, 2017 at 04:21:42PM -0400, Konrad Rzeszutek Wilk wrote:
> > > On Wed, Apr 12, 2017 at 04:46:33PM +0100, Wei Liu wrote:
> > > > On Mon, Apr 10, 2017 at 09:43:13AM -0400, Konrad Rzeszutek Wilk wrote:
> > > > [...]
> > > > > 
> > > > > .. Except that we need some way of doing FLR and Pciback
> > > > > is the one doing it.
> > > > > 
> > > > > The right way would be to expand pciback to support the do_flr ioctl
> > > > > and combine it with your patch.
> > > > > 
> > > > 
> > > > OK. Does that mean the bug is in Linux and I should just take Ross's
> > > 
> > > Please don't.
> > > 
> > > Or if you would like I can do
> > > 
> > > Nacked-by: Konrad Rzeszutek Wilk 
> > > 
> > 
> > Please correct me if I'm wrong: current libxl code is broken in one way and
> > taking that patch means to break it in another way?
> > 
> > What is the way forward? Do you want me to wait until those kernel
> > patches are upstreamed?
> 
> Julien was looking into moving all this reset logic into the hypervisor 
> itself,
> so that ARM and PVHv2 passthrough doesn't use pciback at all. Maybe it could 
> be
> done now? AFAICT this would solve the issue under discussion here.
 
That is a medium to long term goal and I wouldn't rely on it to fix an
outstanding bug.___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel


Re: [Xen-devel] PCI passthrough will cause unknown device appearance in device manager of Windows OS via xenstore keys

2017-04-13 Thread Roger Pau Monné
On Thu, Apr 13, 2017 at 09:41:03AM +0100, Wei Liu wrote:
> On Wed, Apr 12, 2017 at 04:21:42PM -0400, Konrad Rzeszutek Wilk wrote:
> > On Wed, Apr 12, 2017 at 04:46:33PM +0100, Wei Liu wrote:
> > > On Mon, Apr 10, 2017 at 09:43:13AM -0400, Konrad Rzeszutek Wilk wrote:
> > > [...]
> > > > 
> > > > .. Except that we need some way of doing FLR and Pciback
> > > > is the one doing it.
> > > > 
> > > > The right way would be to expand pciback to support the do_flr ioctl
> > > > and combine it with your patch.
> > > > 
> > > 
> > > OK. Does that mean the bug is in Linux and I should just take Ross's
> > 
> > Please don't.
> > 
> > Or if you would like I can do
> > 
> > Nacked-by: Konrad Rzeszutek Wilk 
> > 
> 
> Please correct me if I'm wrong: current libxl code is broken in one way and
> taking that patch means to break it in another way?
> 
> What is the way forward? Do you want me to wait until those kernel
> patches are upstreamed?

Julien was looking into moving all this reset logic into the hypervisor itself,
so that ARM and PVHv2 passthrough doesn't use pciback at all. Maybe it could be
done now? AFAICT this would solve the issue under discussion here.

Roger.


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel


Re: [Xen-devel] PCI passthrough will cause unknown device appearance in device manager of Windows OS via xenstore keys

2017-04-13 Thread Wei Liu
On Wed, Apr 12, 2017 at 04:21:42PM -0400, Konrad Rzeszutek Wilk wrote:
> On Wed, Apr 12, 2017 at 04:46:33PM +0100, Wei Liu wrote:
> > On Mon, Apr 10, 2017 at 09:43:13AM -0400, Konrad Rzeszutek Wilk wrote:
> > [...]
> > > 
> > > .. Except that we need some way of doing FLR and Pciback
> > > is the one doing it.
> > > 
> > > The right way would be to expand pciback to support the do_flr ioctl
> > > and combine it with your patch.
> > > 
> > 
> > OK. Does that mean the bug is in Linux and I should just take Ross's
> 
> Please don't.
> 
> Or if you would like I can do
> 
> Nacked-by: Konrad Rzeszutek Wilk 
> 

Please correct me if I'm wrong: current libxl code is broken in one way and
taking that patch means to break it in another way?

What is the way forward? Do you want me to wait until those kernel
patches are upstreamed?

Wei.

> 
> > patch as-is for 4.9?
> > 
> > Wei.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel


Re: [Xen-devel] PCI passthrough will cause unknown device appearance in device manager of Windows OS via xenstore keys

2017-04-12 Thread Konrad Rzeszutek Wilk
On Wed, Apr 12, 2017 at 04:46:33PM +0100, Wei Liu wrote:
> On Mon, Apr 10, 2017 at 09:43:13AM -0400, Konrad Rzeszutek Wilk wrote:
> [...]
> > 
> > .. Except that we need some way of doing FLR and Pciback
> > is the one doing it.
> > 
> > The right way would be to expand pciback to support the do_flr ioctl
> > and combine it with your patch.
> > 
> 
> OK. Does that mean the bug is in Linux and I should just take Ross's

Please don't.

Or if you would like I can do

Nacked-by: Konrad Rzeszutek Wilk 


> patch as-is for 4.9?
> 
> Wei.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel


Re: [Xen-devel] PCI passthrough will cause unknown device appearance in device manager of Windows OS via xenstore keys

2017-04-12 Thread Wei Liu
On Mon, Apr 10, 2017 at 09:43:13AM -0400, Konrad Rzeszutek Wilk wrote:
[...]
> 
> .. Except that we need some way of doing FLR and Pciback
> is the one doing it.
> 
> The right way would be to expand pciback to support the do_flr ioctl
> and combine it with your patch.
> 

OK. Does that mean the bug is in Linux and I should just take Ross's
patch as-is for 4.9?

Wei.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel


Re: [Xen-devel] PCI passthrough will cause unknown device appearance in device manager of Windows OS via xenstore keys

2017-04-10 Thread Sander Eikelenboom

Monday, April 10, 2017, 4:44:47 PM, you wrote:

> On Mon, Apr 10, 2017 at 03:21:41PM +0100, Andrew Cooper wrote:
>> On 10/04/17 15:12, Konrad Rzeszutek Wilk wrote:
>> > On Mon, Apr 10, 2017 at 02:28:58PM +0100, Wei Liu wrote:
>> >> On Mon, Apr 10, 2017 at 01:46:20PM +0100, Andrew Cooper wrote:
>> >>> On 10/04/17 12:24, lidonglin wrote:
>>  Hi all:
>> 
>>  I have one question about PCI passthrough. I found
>>  that if I created a VM with host pci device(cfg file as below), there
>>  were some xenstore keys exsiting in /local/domain/0 and
>>  /local/domain/DomID/. Besides I found one unknown device in device
>>  manager of Windows OS with Class ID FF80 from vendor ID 5853. My
>>  questions  are as below:
>> 
>>   
>> 
>>  1.   Why do we create frontend and backend key pairs in xenstore?
>>   I think  Passthrough is not PV,  was it possible that we just used
>>  these keys to record something?
>> 
>>  2.After review the code, I think backend keys are used to
>>  record state of hostdev, some codes will query something using these
>>  keys. But for frontend keys, can we delete them?
>> 
>>  3.   if frontend keys exist in xenstore, then unknown device will
>>  appear in device manager. Can we fix this?  For  an obsessive,  he
>>  can’t stand for this.
>> 
>> >>> This is a libxl bug which has been reported before, but nothing happened.
>> >>>
>> >>> libxl must not start a PV pci-back when doing HVM PCI passthrough using
>> >>> qemu, because absolutely nothing good can come of having two different
>> >>> ways of poking at PCI state.
>> >>>
>> >>> The patch, unchanged from before, is
>> >>> https://github.com/xenserver/xen-4.7.pg/blob/master/master/0001-libxl-Don-t-insert-PCI-device-into-xenstore-for-HVM-.patch
>> >>>
>> >> ISTR Konrad made some comments about it. CC Konrad.
>> > Aye. My primary beef was that you need some FLR mechanism and pciback
>> > does that. The 'do_flr' was the solution to that - as you could
>> > effectively do it via an ioctl and all would be good.
>> >
>> > Except I am probably missing some edge case (like guest is killed
>> > and we need to do an FLR again, or there are AERs to deal with, etc) so
>> > some pciback -> xenstore -> libxl needs to be in place even for HVM guests.
>> >
>> > Just disabling pciback for HVM does not solve the problem that:
>> >
>> >  - We need FLR
>> >  - We need AER support (CC-ing Venu who is working on this for 
>> > libxl/pciback)
>> 
>> The problem is that pciback is two distinct pieces of functionality.  It
>> should be split in half, so the "steal a device from its real driver and

> Yes sure.

>> provide reset functionality" is purposefully separate from "be the
>> reverse half the PV interface".

>> 
>> Qemu has no problems with resetting the device when necessary, though,

> It does? The sysfs 'reset' is does not do everything you expect.

I can concur, i use this "do_flr" patchset from Konrad for quite some time now.
Without it i can't shutdown and restart HVM guest which use (secondary) GPU 
passthrough (without rebooting the host), with this patchset that works fine!

With other PCI devices there seems to be less of a problem, but for what i have 
as a test case these all function with this patchset as well.

In the past the patchset was blocked from upstreaming to the kernel 
by David Vrabel on the naming.
Since "FLR" is an actual existing (but not widely spread i guess) hardware 
reset function 
from the PCI spec. This patchset does however do something that is more like a 
"bus 
reset" and is not limited to devices having the hardware FLR option. 

Since the libxl code that runs the sysfs do_flr thing is kind of dead code now,
perhaps it's best to rename it to something generic, so it could be extended in 
the future if support for more reset-methods will be bolted on here ?
(so writing some other value to the ioctl could specify which reset methods 
should be tried on this device) 

--
Sander

>> which is why this patch functions perfectly well in a real system.
>> 
>> ~Andrew




___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel


Re: [Xen-devel] PCI passthrough will cause unknown device appearance in device manager of Windows OS via xenstore keys

2017-04-10 Thread Konrad Rzeszutek Wilk
On Mon, Apr 10, 2017 at 03:21:41PM +0100, Andrew Cooper wrote:
> On 10/04/17 15:12, Konrad Rzeszutek Wilk wrote:
> > On Mon, Apr 10, 2017 at 02:28:58PM +0100, Wei Liu wrote:
> >> On Mon, Apr 10, 2017 at 01:46:20PM +0100, Andrew Cooper wrote:
> >>> On 10/04/17 12:24, lidonglin wrote:
>  Hi all:
> 
>  I have one question about PCI passthrough. I found
>  that if I created a VM with host pci device(cfg file as below), there
>  were some xenstore keys exsiting in /local/domain/0 and
>  /local/domain/DomID/. Besides I found one unknown device in device
>  manager of Windows OS with Class ID FF80 from vendor ID 5853. My
>  questions  are as below:
> 
>   
> 
>  1.   Why do we create frontend and backend key pairs in xenstore?
>   I think  Passthrough is not PV,  was it possible that we just used
>  these keys to record something?
> 
>  2.After review the code, I think backend keys are used to
>  record state of hostdev, some codes will query something using these
>  keys. But for frontend keys, can we delete them?
> 
>  3.   if frontend keys exist in xenstore, then unknown device will
>  appear in device manager. Can we fix this?  For  an obsessive,  he
>  can’t stand for this.
> 
> >>> This is a libxl bug which has been reported before, but nothing happened.
> >>>
> >>> libxl must not start a PV pci-back when doing HVM PCI passthrough using
> >>> qemu, because absolutely nothing good can come of having two different
> >>> ways of poking at PCI state.
> >>>
> >>> The patch, unchanged from before, is
> >>> https://github.com/xenserver/xen-4.7.pg/blob/master/master/0001-libxl-Don-t-insert-PCI-device-into-xenstore-for-HVM-.patch
> >>>
> >> ISTR Konrad made some comments about it. CC Konrad.
> > Aye. My primary beef was that you need some FLR mechanism and pciback
> > does that. The 'do_flr' was the solution to that - as you could
> > effectively do it via an ioctl and all would be good.
> >
> > Except I am probably missing some edge case (like guest is killed
> > and we need to do an FLR again, or there are AERs to deal with, etc) so
> > some pciback -> xenstore -> libxl needs to be in place even for HVM guests.
> >
> > Just disabling pciback for HVM does not solve the problem that:
> >
> >  - We need FLR
> >  - We need AER support (CC-ing Venu who is working on this for 
> > libxl/pciback)
> 
> The problem is that pciback is two distinct pieces of functionality.  It
> should be split in half, so the "steal a device from its real driver and

Yes sure.

> provide reset functionality" is purposefully separate from "be the
> reverse half the PV interface".

> 
> Qemu has no problems with resetting the device when necessary, though,

It does? The sysfs 'reset' is does not do everything you expect.

> which is why this patch functions perfectly well in a real system.
> 
> ~Andrew

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel


Re: [Xen-devel] PCI passthrough will cause unknown device appearance in device manager of Windows OS via xenstore keys

2017-04-10 Thread Andrew Cooper
On 10/04/17 15:12, Konrad Rzeszutek Wilk wrote:
> On Mon, Apr 10, 2017 at 02:28:58PM +0100, Wei Liu wrote:
>> On Mon, Apr 10, 2017 at 01:46:20PM +0100, Andrew Cooper wrote:
>>> On 10/04/17 12:24, lidonglin wrote:
 Hi all:

 I have one question about PCI passthrough. I found
 that if I created a VM with host pci device(cfg file as below), there
 were some xenstore keys exsiting in /local/domain/0 and
 /local/domain/DomID/. Besides I found one unknown device in device
 manager of Windows OS with Class ID FF80 from vendor ID 5853. My
 questions  are as below:

  

 1.   Why do we create frontend and backend key pairs in xenstore?
  I think  Passthrough is not PV,  was it possible that we just used
 these keys to record something?

 2.After review the code, I think backend keys are used to
 record state of hostdev, some codes will query something using these
 keys. But for frontend keys, can we delete them?

 3.   if frontend keys exist in xenstore, then unknown device will
 appear in device manager. Can we fix this?  For  an obsessive,  he
 can’t stand for this.

>>> This is a libxl bug which has been reported before, but nothing happened.
>>>
>>> libxl must not start a PV pci-back when doing HVM PCI passthrough using
>>> qemu, because absolutely nothing good can come of having two different
>>> ways of poking at PCI state.
>>>
>>> The patch, unchanged from before, is
>>> https://github.com/xenserver/xen-4.7.pg/blob/master/master/0001-libxl-Don-t-insert-PCI-device-into-xenstore-for-HVM-.patch
>>>
>> ISTR Konrad made some comments about it. CC Konrad.
> Aye. My primary beef was that you need some FLR mechanism and pciback
> does that. The 'do_flr' was the solution to that - as you could
> effectively do it via an ioctl and all would be good.
>
> Except I am probably missing some edge case (like guest is killed
> and we need to do an FLR again, or there are AERs to deal with, etc) so
> some pciback -> xenstore -> libxl needs to be in place even for HVM guests.
>
> Just disabling pciback for HVM does not solve the problem that:
>
>  - We need FLR
>  - We need AER support (CC-ing Venu who is working on this for libxl/pciback)

The problem is that pciback is two distinct pieces of functionality.  It
should be split in half, so the "steal a device from its real driver and
provide reset functionality" is purposefully separate from "be the
reverse half the PV interface".

Qemu has no problems with resetting the device when necessary, though,
which is why this patch functions perfectly well in a real system.

~Andrew

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel


Re: [Xen-devel] PCI passthrough will cause unknown device appearance in device manager of Windows OS via xenstore keys

2017-04-10 Thread Konrad Rzeszutek Wilk
On Mon, Apr 10, 2017 at 02:28:58PM +0100, Wei Liu wrote:
> On Mon, Apr 10, 2017 at 01:46:20PM +0100, Andrew Cooper wrote:
> > On 10/04/17 12:24, lidonglin wrote:
> > >
> > > Hi all:
> > >
> > > I have one question about PCI passthrough. I found
> > > that if I created a VM with host pci device(cfg file as below), there
> > > were some xenstore keys exsiting in /local/domain/0 and
> > > /local/domain/DomID/. Besides I found one unknown device in device
> > > manager of Windows OS with Class ID FF80 from vendor ID 5853. My
> > > questions  are as below:
> > >
> > >  
> > >
> > > 1.   Why do we create frontend and backend key pairs in xenstore?
> > >  I think  Passthrough is not PV,  was it possible that we just used
> > > these keys to record something?
> > >
> > > 2.After review the code, I think backend keys are used to
> > > record state of hostdev, some codes will query something using these
> > > keys. But for frontend keys, can we delete them?
> > >
> > > 3.   if frontend keys exist in xenstore, then unknown device will
> > > appear in device manager. Can we fix this?  For  an obsessive,  he
> > > can’t stand for this.
> > >
> > 
> > This is a libxl bug which has been reported before, but nothing happened.
> > 
> > libxl must not start a PV pci-back when doing HVM PCI passthrough using
> > qemu, because absolutely nothing good can come of having two different
> > ways of poking at PCI state.
> > 
> > The patch, unchanged from before, is
> > https://github.com/xenserver/xen-4.7.pg/blob/master/master/0001-libxl-Don-t-insert-PCI-device-into-xenstore-for-HVM-.patch
> > 
> 
> ISTR Konrad made some comments about it. CC Konrad.

Aye. My primary beef was that you need some FLR mechanism and pciback
does that. The 'do_flr' was the solution to that - as you could
effectively do it via an ioctl and all would be good.

Except I am probably missing some edge case (like guest is killed
and we need to do an FLR again, or there are AERs to deal with, etc) so
some pciback -> xenstore -> libxl needs to be in place even for HVM guests.

Just disabling pciback for HVM does not solve the problem that:

 - We need FLR
 - We need AER support (CC-ing Venu who is working on this for libxl/pciback)

> 
> > ~Andrew

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel


Re: [Xen-devel] PCI passthrough will cause unknown device appearance in device manager of Windows OS via xenstore keys

2017-04-10 Thread Wei Liu
On Mon, Apr 10, 2017 at 01:46:20PM +0100, Andrew Cooper wrote:
> On 10/04/17 12:24, lidonglin wrote:
> >
> > Hi all:
> >
> > I have one question about PCI passthrough. I found
> > that if I created a VM with host pci device(cfg file as below), there
> > were some xenstore keys exsiting in /local/domain/0 and
> > /local/domain/DomID/. Besides I found one unknown device in device
> > manager of Windows OS with Class ID FF80 from vendor ID 5853. My
> > questions  are as below:
> >
> >  
> >
> > 1.   Why do we create frontend and backend key pairs in xenstore?
> >  I think  Passthrough is not PV,  was it possible that we just used
> > these keys to record something?
> >
> > 2.After review the code, I think backend keys are used to
> > record state of hostdev, some codes will query something using these
> > keys. But for frontend keys, can we delete them?
> >
> > 3.   if frontend keys exist in xenstore, then unknown device will
> > appear in device manager. Can we fix this?  For  an obsessive,  he
> > can’t stand for this.
> >
> 
> This is a libxl bug which has been reported before, but nothing happened.
> 
> libxl must not start a PV pci-back when doing HVM PCI passthrough using
> qemu, because absolutely nothing good can come of having two different
> ways of poking at PCI state.
> 
> The patch, unchanged from before, is
> https://github.com/xenserver/xen-4.7.pg/blob/master/master/0001-libxl-Don-t-insert-PCI-device-into-xenstore-for-HVM-.patch
> 

ISTR Konrad made some comments about it. CC Konrad.

> ~Andrew

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel


Re: [Xen-devel] PCI passthrough will cause unknown device appearance in device manager of Windows OS via xenstore keys

2017-04-10 Thread Konrad Rzeszutek Wilk
On Mon, Apr 10, 2017 at 01:46:20PM +0100, Andrew Cooper wrote:
> On 10/04/17 12:24, lidonglin wrote:
> >
> > Hi all:
> >
> > I have one question about PCI passthrough. I found
> > that if I created a VM with host pci device(cfg file as below), there
> > were some xenstore keys exsiting in /local/domain/0 and
> > /local/domain/DomID/. Besides I found one unknown device in device
> > manager of Windows OS with Class ID FF80 from vendor ID 5853. My
> > questions  are as below:
> >
> >  
> >
> > 1.   Why do we create frontend and backend key pairs in xenstore?
> >  I think  Passthrough is not PV,  was it possible that we just used
> > these keys to record something?
> >
> > 2.After review the code, I think backend keys are used to
> > record state of hostdev, some codes will query something using these
> > keys. But for frontend keys, can we delete them?
> >
> > 3.   if frontend keys exist in xenstore, then unknown device will
> > appear in device manager. Can we fix this?  For  an obsessive,  he
> > can’t stand for this.
> >
> 
> This is a libxl bug which has been reported before, but nothing happened.
> 
> libxl must not start a PV pci-back when doing HVM PCI passthrough using
> qemu, because absolutely nothing good can come of having two different
> ways of poking at PCI state.

.. Except that we need some way of doing FLR and Pciback
is the one doing it.

The right way would be to expand pciback to support the do_flr ioctl
and combine it with your patch.

> 
> The patch, unchanged from before, is
> https://github.com/xenserver/xen-4.7.pg/blob/master/master/0001-libxl-Don-t-insert-PCI-device-into-xenstore-for-HVM-.patch

Attaching the flr one (inline) and attachemnt


>From 314a0ff66af9380de4cd4c8a7f4f648c6f593bb8 Mon Sep 17 00:00:00 2001
From: Konrad Rzeszutek Wilk 
Date: Wed, 3 Dec 2014 15:47:02 -0500
Subject: [PATCH] xen/pciback: Implement PCI reset slot or bus with 'do_flr'
 SysFS attribute

The life-cycle of a PCI device in Xen pciback is complex
and is constrained by the PCI generic locking mechanism.

It starts with the device being binded to us - for which
we do a device function reset (and done via SysFS
so the PCI lock is held)

If the device is unbinded from us - we also do a function
reset (also done via SysFS so the PCI lock is held).

If the device is un-assigned from a guest - we do a function
reset (no PCI lock).

All on the individual PCI function level (so bus:device:function).

Unfortunatly a function reset is not adequate for certain
PCIe devices. The reset for an individual PCI function "means
device must support FLR (PCIe or AF), PM reset on D3hot->D0
device specific reset, or be a singleton device on a bus
a secondary bus reset.  FLR does not have widespread support,
reset is not very reliable, and bus topology is dictated by the
and device design.  We need to provide a means for a user to
a bus reset in cases where the existing mechanisms are not
 or not reliable. " (Adam Williamson, 'vfio-pci: PCI hot reset
interface' commit 8b27ee60bfd6bbb84d2df28fa706c5c5081066ca).

As such to do a slot or a bus reset is we need another mechanism.
This is not exposed SysFS as there is no good way of exposing
a bus topology there.

This is due to the complexity - we MUST know that the different
functions off a PCIe device are not in use by other drivers, or
if they are in use (say one of them is assigned to a guest
and the other is idle) - it is still OK to reset the slot
(assuming both of them are owned by Xen pciback).

This patch does that by doing an slot or bus reset (if
slot not supported) if all of the functions of a PCIe
device belong to Xen PCIback. We do not care if the device is
in-use as we depend on the toolstack to be aware of this -
however if it is we will WARN the user.

Due to the complexity with the PCI lock we cannot do
the reset when a device is binded ('echo $BDF > bind')
or when unbinded ('echo $BDF > unbind') as the pci_[slot|bus]_reset
also take the same lock resulting in a dead-lock.

Putting the reset function in a workqueue or thread
won't work either - as we have to do the reset function
outside the 'unbind' context (it holds the PCI lock).
But once you 'unbind' a device the device is no longer
under the ownership of Xen pciback and the pci_set_drvdata
has been reset so we cannot use a thread for this.

Instead of doing all this complex dance, we depend on the toolstack
doing the right thing. As such implement the 'do_flr' SysFS attribute
which 'xl' uses when a device is detached or attached from/to a guest.
It bypasses the need to worry about the PCI lock.

To not inadvertly do a bus reset that would affect devices that
are in use by other drivers (other than Xen pciback) prior
to the reset we check that all of the devices under the bridge
are owned by Xen pciback. If they are not we do not do
the bus (or slot) reset.

We also warn the user if the device is in use - but still
continue with the reset. This should not happen as the toolsta

Re: [Xen-devel] PCI passthrough will cause unknown device appearance in device manager of Windows OS via xenstore keys

2017-04-10 Thread Andrew Cooper
On 10/04/17 12:24, lidonglin wrote:
>
> Hi all:
>
> I have one question about PCI passthrough. I found
> that if I created a VM with host pci device(cfg file as below), there
> were some xenstore keys exsiting in /local/domain/0 and
> /local/domain/DomID/. Besides I found one unknown device in device
> manager of Windows OS with Class ID FF80 from vendor ID 5853. My
> questions  are as below:
>
>  
>
> 1.   Why do we create frontend and backend key pairs in xenstore?
>  I think  Passthrough is not PV,  was it possible that we just used
> these keys to record something?
>
> 2.After review the code, I think backend keys are used to
> record state of hostdev, some codes will query something using these
> keys. But for frontend keys, can we delete them?
>
> 3.   if frontend keys exist in xenstore, then unknown device will
> appear in device manager. Can we fix this?  For  an obsessive,  he
> can’t stand for this.
>

This is a libxl bug which has been reported before, but nothing happened.

libxl must not start a PV pci-back when doing HVM PCI passthrough using
qemu, because absolutely nothing good can come of having two different
ways of poking at PCI state.

The patch, unchanged from before, is
https://github.com/xenserver/xen-4.7.pg/blob/master/master/0001-libxl-Don-t-insert-PCI-device-into-xenstore-for-HVM-.patch

~Andrew
___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel


[Xen-devel] PCI passthrough will cause unknown device appearance in device manager of Windows OS via xenstore keys

2017-04-10 Thread lidonglin
Hi all:
I have one question about PCI passthrough. I found that if I 
created a VM with host pci device(cfg file as below), there were some xenstore 
keys exsiting in /local/domain/0 and /local/domain/DomID/. Besides I found one 
unknown device in device manager of Windows OS with Class ID FF80 from vendor 
ID 5853. My questions  are as below:


1.   Why do we create frontend and backend key pairs in xenstore?  I think  
Passthrough is not PV,  was it possible that we just used these keys to record 
something?

2.After review the code, I think backend keys are used to record state 
of hostdev, some codes will query something using these keys. But for frontend 
keys, can we delete them?

3.   if frontend keys exist in xenstore, then unknown device will appear in 
device manager. Can we fix this?  For  an obsessive,  he can't stand for this.

Cfg file as below:
builder = "hvm"
name = "win2008_server_r2_sp1_64"
viridian = 1
memory = 8192
maxmem = 8192
vcpus = 4
disk = [ '/mnt /win2008_server_ent_r2_sp1_64_gputhrough_vhd,vhd,xvda,rw' ]
vnc = 1
vnclisten = 'xxx.xxx.xxx.xxx'
vncdisplay = 0
usb = 1
usbdevice = ['tablet']
pci = [":06:00.0"]

I use xen-4.8.0 version.

Xenstore keys
For domU
/local/domain/5/device/vkbd/0/backend = "/local/domain/0/backend/vkbd/5/0"
/local/domain/5/device/vkbd/0/backend-id = "0"
/local/domain/5/device/vkbd/0/state = "1"
/local/domain/5/device/pci = ""
/local/domain/5/device/pci/0 = ""
/local/domain/5/device/pci/0/backend = "/local/domain/0/backend/pci/5/0"
/local/domain/5/device/pci/0/backend-id = "0"
/local/domain/5/device/pci/0/state = "1"
/local/domain/5/control = ""

For dom0
/local/domain/0/backend/vkbd/5/0/feature-abs-pointer = "1"
/local/domain/0/backend/vkbd/5/0/hotplug-status = "connected"
/local/domain/0/backend/pci = ""
/local/domain/0/backend/pci/5 = ""
/local/domain/0/backend/pci/5/0 = ""
/local/domain/0/backend/pci/5/0/frontend = "/local/domain/5/device/pci/0"
/local/domain/0/backend/pci/5/0/frontend-id = "5"
/local/domain/0/backend/pci/5/0/online = "1"
/local/domain/0/backend/pci/5/0/state = "3"
/local/domain/0/backend/pci/5/0/domain = "win2008_server_r2_sp1_64"
/local/domain/0/backend/pci/5/0/key-0 = ":06:00.0"
/local/domain/0/backend/pci/5/0/dev-0 = ":06:00.0"
/local/domain/0/backend/pci/5/0/vdevfn-0 = "20"
/local/domain/0/backend/pci/5/0/opts-0 = 
"msitranslate=0,power_mgmt=0,permissive=0"
/local/domain/0/backend/pci/5/0/state-0 = "3"
/local/domain/0/backend/pci/5/0/num_devs = "1"
/local/domain/0/backend/pci/5/0/vdev-0 = ":00:00.00"
/local/domain/0/backend/pci/5/0/root-0 = ":00"
/local/domain/0/backend/pci/5/0/root_num = "1"
/local/domain/5 = ""
/local/domain/5/vm = "/vm/81296e36-1576-4b58-bb6e-17dc2b4ea1f6"


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel