Re: [RFC PATCH V2 0/3] IXGBE/VFIO: Add live migration support for SRIOV NIC

2015-12-09 Thread Lan, Tianyu
On 12/9/2015 7:28 PM, Michael S. Tsirkin wrote: I remember reading that it's possible to implement a bus driver on windows if required. But basically I don't see how windows can be relevant to discussing guest driver patches. That discussion probably belongs on the qemu maling list, not on lkml.

Re: [RFC PATCH V2 0/3] IXGBE/VFIO: Add live migration support for SRIOV NIC

2015-12-09 Thread Lan, Tianyu
On 12/9/2015 6:37 PM, Michael S. Tsirkin wrote: On Sat, Dec 05, 2015 at 12:32:00AM +0800, Lan, Tianyu wrote: Hi Michael & Alexander: Thanks a lot for your comments and suggestions. It's nice that it's appreciated, but you then go on and ignore all that I have written here: ht

Re: [RFC PATCH V2 0/3] IXGBE/VFIO: Add live migration support for SRIOV NIC

2015-12-09 Thread Lan, Tianyu
On 12/8/2015 1:12 AM, Alexander Duyck wrote: On Mon, Dec 7, 2015 at 7:40 AM, Lan, Tianyu wrote: On 12/5/2015 1:07 AM, Alexander Duyck wrote: We still need to support Windows guest for migration and this is why our patches keep all changes in the driver since it's impossible to c

Re: [RFC PATCH V2 0/3] IXGBE/VFIO: Add live migration support for SRIOV NIC

2015-12-07 Thread Lan, Tianyu
On 12/5/2015 1:07 AM, Alexander Duyck wrote: We still need to support Windows guest for migration and this is why our patches keep all changes in the driver since it's impossible to change Windows kernel. That is a poor argument. I highly doubt Microsoft is interested in having to modify all

Re: [RFC PATCH V2 0/3] IXGBE/VFIO: Add live migration support for SRIOV NIC

2015-12-04 Thread Lan, Tianyu
Hi Michael & Alexander: Thanks a lot for your comments and suggestions. We still need to support Windows guest for migration and this is why our patches keep all changes in the driver since it's impossible to change Windows kernel. Following is my idea to do DMA tracking. Inject event to VF

Re: [RFC PATCH V2 0/3] IXGBE/VFIO: Add live migration support for SRIOV NIC

2015-12-01 Thread Lan, Tianyu
On 12/1/2015 12:07 AM, Alexander Duyck wrote: They can only be corrected if the underlying assumptions are correct and they aren't. Your solution would have never worked correctly. The problem is you assume you can keep the device running when you are migrating and you simply cannot. At some

Re: [RFC PATCH V2 0/3] IXGBE/VFIO: Add live migration support for SRIOV NIC

2015-11-29 Thread Lan, Tianyu
On 11/26/2015 11:56 AM, Alexander Duyck wrote: > I am not saying you cannot modify the drivers, however what you are doing is far too invasive. Do you seriously plan on modifying all of the PCI device drivers out there in order to allow any device that might be direct assigned to a port to suppo

Re: [RFC PATCH V2 3/3] Ixgbevf: Add migration support for ixgbevf driver

2015-11-25 Thread Lan, Tianyu
On 11/25/2015 8:28 PM, Michael S. Tsirkin wrote: Frankly, I don't really see what this short term hack buys us, and if it goes in, we'll have to maintain it forever. The framework of how to notify VF about migration status won't be changed regardless of stopping VF or not before doing migratio

Re: [RFC PATCH V2 0/3] IXGBE/VFIO: Add live migration support for SRIOV NIC

2015-11-25 Thread Lan Tianyu
On 2015年11月25日 13:30, Alexander Duyck wrote: > No, what I am getting at is that you can't go around and modify the > configuration space for every possible device out there. This > solution won't scale. PCI config space regs are emulation by Qemu and so We can find the free PCI config space regs

Re: [RFC PATCH V2 3/3] Ixgbevf: Add migration support for ixgbevf driver

2015-11-24 Thread Lan Tianyu
On 2015年11月25日 05:20, Michael S. Tsirkin wrote: > I have to say, I was much more interested in the idea > of tracking dirty memory. I have some thoughts about > that one - did you give up on it then? No, our finial target is to keep VF active before doing migration and tracking dirty memory is ess

Re: [RFC PATCH V2 0/3] IXGBE/VFIO: Add live migration support for SRIOV NIC

2015-11-24 Thread Lan Tianyu
On 2015年11月24日 22:20, Alexander Duyck wrote: > I'm still not a fan of this approach. I really feel like this is > something that should be resolved by extending the existing PCI hot-plug > rather than trying to instrument this per driver. Then you will get the > goodness for multiple drivers and

[RFC PATCH V2 2/3] PCI: Add macros for faked PCI migration capability

2015-11-24 Thread Lan Tianyu
migration status in the reg *PCI_VF_MIGRATION_VF_STATUS VF driver tells Qemu ready for migration *PCI_VF_MIGRATION_IRQ VF driver stores mailbox interrupt vector in the reg for Qemu to trigger during migration. Signed-off-by: Lan Tianyu --- include/uapi/linux/pci_regs.h | 18 +- 1 file

[RFC PATCH V2 1/3] VFIO: Add new ioctl cmd VFIO_GET_PCI_CAP_INFO

2015-11-24 Thread Lan Tianyu
This patch is to add new ioctl cmd VFIO_GET_PCI_CAP_INFO to get PCI cap table size and get free PCI config space regs according pos and size. Qemu will add faked PCI capability for migration and need such info. Signed-off-by: Lan Tianyu --- drivers/vfio/pci/vfio_pci.c | 21

[RFC PATCH V2 3/3] Ixgbevf: Add migration support for ixgbevf driver

2015-11-24 Thread Lan Tianyu
), mailbox irq won't be triggered and VF driver will disable it via PCI_VF_MIGRATION_CAP reg. These case will be resolved later. Signed-off-by: Lan Tianyu --- drivers/net/ethernet/intel/ixgbevf/ixgbevf.h | 5 ++ drivers/net/ethernet/intel/ixgbevf/ixgbevf_main.c | 102

[RFC PATCH V2 0/3] IXGBE/VFIO: Add live migration support for SRIOV NIC

2015-11-24 Thread Lan Tianyu
n nic before migration and put up again on the target machine. Lan Tianyu (3): VFIO: Add new ioctl cmd VFIO_GET_PCI_CAP_INFO PCI: Add macros for faked PCI migration capability Ixgbevf: Add migration support for ixgbevf driver drivers/net/ethernet/intel/ixgbevf/ixgbevf.h | 5 ++ driver

Re: [RFC Patch 00/12] IXGBE: Add live migration support for SRIOV NIC

2015-10-29 Thread Lan Tianyu
On 2015年10月30日 00:17, Alexander Duyck wrote: > On 10/29/2015 01:33 AM, Lan Tianyu wrote: >> On 2015年10月29日 14:58, Alexander Duyck wrote: >>> Your code was having to do a bunch of shuffling in order to get things >>> set up so that you could bring the interface back up.

Re: [RFC Patch 00/12] IXGBE: Add live migration support for SRIOV NIC

2015-10-29 Thread Lan Tianyu
On 2015年10月29日 14:58, Alexander Duyck wrote: > > Your code was having to do a bunch of shuffling in order to get things > set up so that you could bring the interface back up. I would argue > that it may actually be faster at least on the bring-up to just drop the > old rings and start over since

Re: [RFC Patch 00/12] IXGBE: Add live migration support for SRIOV NIC

2015-10-28 Thread Lan Tianyu
On 2015年10月26日 23:03, Alexander Duyck wrote: > No. I think you are missing the fact that there are 256 descriptors per > page. As such if you dirty just 1 you will be pulling in 255 more, of > which you may or may not have pulled in the receive buffer for. > > So for example if you have the desc

Re: [RFC Patch 00/12] IXGBE: Add live migration support for SRIOV NIC

2015-10-25 Thread Lan Tianyu
On 2015年10月24日 02:36, Alexander Duyck wrote: > I was thinking about it and I am pretty sure the dummy write approach is > problematic at best. Specifically the issue is that while you are > performing a dummy write you risk pulling in descriptors for data that > hasn't been dummy written to yet.

Re: [RFC Patch 03/12] IXGBE: Add sysfs interface for Qemu to migrate VF status in the PF driver

2015-10-25 Thread Lan, Tianyu
On 10/22/2015 4:45 AM, Alexander Duyck wrote: +/* Record states hold by PF */ +memcpy(&state->vf_data, &adapter->vfinfo[vfn], sizeof(struct vf_data_storage)); + +vf_shift = vfn % 32; +reg_offset = vfn / 32; + +reg = IXGBE_READ_REG(hw, IXGBE_VFTE(reg_offset)); +reg &= ~(1

Re: [RFC Patch 05/12] IXGBE: Add new sysfs interface of "notify_vf"

2015-10-24 Thread Lan, Tianyu
On 10/25/2015 2:03 PM, Alexander Duyck wrote: On 10/24/2015 08:43 AM, Lan, Tianyu wrote: On 10/22/2015 4:52 AM, Alexander Duyck wrote: Also have you even considered the MSI-X configuration on the VF? I haven't seen anything anywhere that would have migrated the VF's MSI-X con

Re: [RFC Patch 08/12] IXGBEVF: Rework code of finding the end transmit desc of package

2015-10-24 Thread Lan, Tianyu
On 10/22/2015 5:14 AM, Alexander Duyck wrote: Where is i being initialized? It was here but you removed it. Are you using i without initializing it? Sorry, the initialization was put into patch 10 by mistake. "i" is assigned with "tx_ring->next_to_clean". -- To unsubscribe from this list: se

Re: [RFC Patch 08/12] IXGBEVF: Rework code of finding the end transmit desc of package

2015-10-24 Thread Lan, Tianyu
On 10/22/2015 8:58 PM, Michael S. Tsirkin wrote: Do you really need to play the shifting games? Can't you just reset everything and re-initialize the rings? It's slower but way less intrusive. Also removes the need to track writes into rings. Shift ring is to avoid losing those packets in the

Re: [RFC Patch 05/12] IXGBE: Add new sysfs interface of "notify_vf"

2015-10-24 Thread Lan, Tianyu
On 10/22/2015 4:52 AM, Alexander Duyck wrote: Also have you even considered the MSI-X configuration on the VF? I haven't seen anything anywhere that would have migrated the VF's MSI-X configuration from BAR 3 on one system to the new system. MSI-X migration is done by Hypervisor(Qemu). Follow

Re: [RFC Patch 01/12] PCI: Add virtfn_index for struct pci_device

2015-10-24 Thread Lan, Tianyu
On 10/22/2015 2:07 AM, Alexander Duyck wrote: On 10/21/2015 09:37 AM, Lan Tianyu wrote: Add "virtfn_index" member in the struct pci_device to record VF sequence of PF. This will be used in the VF sysfs node handle. Signed-off-by: Lan Tianyu --- drivers/pci/iov.c | 1 + inc

[RFC Patch 03/12] IXGBE: Add sysfs interface for Qemu to migrate VF status in the PF driver

2015-10-21 Thread Lan Tianyu
This patch is to add sysfs interface state_in_pf under sysfs directory of VF PCI device for Qemu to get and put VF status in the PF driver during migration. Signed-off-by: Lan Tianyu --- drivers/net/ethernet/intel/ixgbe/ixgbe_sriov.c | 156 - 1 file changed, 155

[RFC Patch 09/12] IXGBEVF: Add live migration support for VF driver

2015-10-21 Thread Lan Tianyu
irst entry of the ring and then enable tx/rx rings. VF restarts to receive and transmit from original head desc. Signed-off-by: Lan Tianyu --- drivers/net/ethernet/intel/ixgbevf/defines.h | 6 ++ drivers/net/ethernet/intel/ixgbevf/ixgbevf.h | 7 +- driver

[RFC Patch 07/12] IXGBEVF: Add new mail box event for migration

2015-10-21 Thread Lan Tianyu
VF status in the PF driver needs to be restored after migration and reset VF hardware. This patch is to add a new event for VF driver to notify PF driver to restore status. Signed-off-by: Lan Tianyu --- drivers/net/ethernet/intel/ixgbevf/mbx.h | 3 +++ drivers/net/ethernet/intel/ixgbevf/vf.c

[RFC Patch 05/12] IXGBE: Add new sysfs interface of "notify_vf"

2015-10-21 Thread Lan Tianyu
This patch is to add new sysfs interface of "notify_vf" under sysfs directory of VF PCI device for Qemu to notify VF when migration status is changed. Signed-off-by: Lan Tianyu --- drivers/net/ethernet/intel/ixgbe/ixgbe_sriov.c | 30 ++ drivers/net/ethernet/i

[RFC Patch 04/12] IXGBE: Add ixgbe_ping_vf() to notify a specified VF via mailbox msg.

2015-10-21 Thread Lan Tianyu
This patch is to add ixgbe_ping_vf() to notify a specified VF. When migration status is changed, it's necessary to notify VF the change. VF driver will check the migrate status when it gets mailbox msg. Signed-off-by: Lan Tianyu --- drivers/net/ethernet/intel/ixgbe/ixgbe_sriov.c

[RFC Patch 00/12] IXGBE: Add live migration support for SRIOV NIC

2015-10-21 Thread Lan Tianyu
= So far, the patchset isn't perfect. VF net interface can't be open, closed, down and up during migration. Will prevent such operation during migration in the future job. Very appreciate for your comments. Lan Tianyu (12): PCI: Add virtfn_index for struct pci_device

[RFC Patch 08/12] IXGBEVF: Rework code of finding the end transmit desc of package

2015-10-21 Thread Lan Tianyu
is to replace recording pointer with recording the desc number of the package and find the end decs via the first desc and desc number. Signed-off-by: Lan Tianyu --- drivers/net/ethernet/intel/ixgbevf/ixgbevf.h | 1 + drivers/net/ethernet/intel/ixgbevf/ixgbevf_main.c | 19 -

[RFC Patch 01/12] PCI: Add virtfn_index for struct pci_device

2015-10-21 Thread Lan Tianyu
Add "virtfn_index" member in the struct pci_device to record VF sequence of PF. This will be used in the VF sysfs node handle. Signed-off-by: Lan Tianyu --- drivers/pci/iov.c | 1 + include/linux/pci.h | 1 + 2 files changed, 2 insertions(+) diff --git a/drivers/pci/iov.c b/drivers

[RFC Patch 11/12] IXGBEVF: Migrate VF statistic data

2015-10-21 Thread Lan Tianyu
t interface is up or open. Call existed function to update base and saved_reset data to keep statistic data continual during migration. Signed-off-by: Lan Tianyu --- drivers/net/ethernet/intel/ixgbevf/ixgbevf_main.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/drivers/net/ethernet/int

[RFC Patch 12/12] IXGBEVF: Track dma dirty pages

2015-10-21 Thread Lan Tianyu
e and write it back) during receive and transmit data. Signed-off-by: Lan Tianyu --- drivers/net/ethernet/intel/ixgbevf/ixgbevf_main.c | 14 +++--- 1 file changed, 11 insertions(+), 3 deletions(-) diff --git a/drivers/net/ethernet/intel/ixgbevf/ixgbevf_main.c b/drivers/net/ethernet/

[RFC Patch 10/12] IXGBEVF: Add lock to protect tx/rx ring operation

2015-10-21 Thread Lan Tianyu
Ring shifting during restoring VF function maybe race with original ring operation(transmit/receive package). This patch is to add tx/rx lock to protect ring related data. Signed-off-by: Lan Tianyu --- drivers/net/ethernet/intel/ixgbevf/ixgbevf.h | 2 ++ drivers/net/ethernet/intel/ixgbevf

[RFC Patch 06/12] IXGBEVF: Add self emulation layer

2015-10-21 Thread Lan Tianyu
In order to restore VF function after migration, add self emulation layer to record regs' values during accessing regs. Signed-off-by: Lan Tianyu --- drivers/net/ethernet/intel/ixgbevf/Makefile| 3 ++- drivers/net/ethernet/intel/ixgbevf/ixgbevf_main.c | 2 +- .../net/ethernet/

[RFC Patch 02/12] IXGBE: Add new mail box event to restore VF status in the PF driver

2015-10-21 Thread Lan Tianyu
This patch is to restore VF status in the PF driver when get event from VF. Signed-off-by: Lan Tianyu --- drivers/net/ethernet/intel/ixgbe/ixgbe.h | 1 + drivers/net/ethernet/intel/ixgbe/ixgbe_mbx.h | 1 + drivers/net/ethernet/intel/ixgbe/ixgbe_sriov.c | 40