On 9/18/2017 7:49 AM, Shijith Thotton wrote: > On Mon, Sep 18, 2017 at 03:50:30AM +0000, Yang, Qiming wrote: >> Sorry, Gregory and Shijith >> >> I misunderstand your advice. >> If we only remove PCI reset at igbuio_pci_open, not include >> igbuio_pci_release, I think it is a good solution. >> It reserve the functional of this patch and we have test on FVL 10G/40G/25G, >> all work fine. >> > > Thanks Qiming. Glad to hear that removing pci_reset_function from > igbuio_pci_open > fixed the issue for Intel NICs. > > Harish, > > Please let us know the behavior with QEDE. Reverting won't be necessary if the > fix works for all.
Hi Shijith, Would you mind submitting the patch, so this becomes easier to discuss and test. Thanks, ferruh > > Thanks, > Shijith > >>> -----Original Message----- >>> From: Zhang, Helin >>> Sent: Monday, September 18, 2017 10:39 AM >>> To: Yang, Qiming <qiming.y...@intel.com>; Gregory Etelson >>> <greg...@weka.io>; tho...@monjalon.net >>> Cc: dev@dpdk.org; Tan, Jianfeng <jianfeng....@intel.com>; Thotton, Shijith >>> <shijith.thot...@cavium.com>; Hu, Xuekun <xuekun...@intel.com>; Yigit, >>> Ferruh <ferruh.yi...@intel.com>; Patil, Harish <harish.pa...@cavium.com>; >>> Li, >>> Xiaoyun <xiaoyun...@intel.com> >>> Subject: RE: [dpdk-dev] vf init issue with patch igb_uio: issue FLR during >>> open >>> and release of device file >>> >>> Hi guys >>> >>> May I suggest to revert the patch first, as it blocks existing >>> functionalities. >>> Then the original authors can add it back with a fix? >>> >>> Regards, >>> Helin >>> >>> -----Original Message----- >>> From: dev [mailto:dev-boun...@dpdk.org] On Behalf Of Yang, Qiming >>> Sent: Monday, September 18, 2017 10:21 AM >>> To: Gregory Etelson <greg...@weka.io> >>> Cc: tho...@monjalon.net; dev@dpdk.org; Tan, Jianfeng >>> <jianfeng....@intel.com>; Thotton, Shijith <shijith.thot...@cavium.com>; Hu, >>> Xuekun <xuekun...@intel.com>; Yigit, Ferruh <ferruh.yi...@intel.com>; Patil, >>> Harish <harish.pa...@cavium.com>; Li, Xiaoyun <xiaoyun...@intel.com> >>> Subject: Re: [dpdk-dev] vf init issue with patch igb_uio: issue FLR during >>> open >>> and release of device file >>> >>> Yes, I have tried, and it works. But I think removing this function also >>> makes the >>> patch meaningless. >>> >>> Qiming >>> From: Gregory Etelson [mailto:greg...@weka.io] >>> Sent: Sunday, September 17, 2017 10:50 AM >>> To: Yang, Qiming <qiming.y...@intel.com> >>> Cc: tho...@monjalon.net; dev@dpdk.org; Tan, Jianfeng >>> <jianfeng....@intel.com>; Thotton, Shijith <shijith.thot...@cavium.com>; Hu, >>> Xuekun <xuekun...@intel.com>; Yigit, Ferruh <ferruh.yi...@intel.com>; Patil, >>> Harish <harish.pa...@cavium.com> >>> Subject: Re: [dpdk-dev] vf init issue with patch igb_uio: issue FLR during >>> open >>> and release of device file >>> >>> >>> Hello Qiming, >>> >>> >>> >>> Did you try to remove PCI reset at igbuio_pci_open ? >>> >>> Reverting the patch entirely will get us back to VF issues >>> >>> >>> >>> Regards, >>> >>> Gregory >>> >>> >>> >>> >>> >>> On Friday, 15 September 2017 11:04:16 IDT Yang, Qiming wrote: >>> >>>> Hi, Thomas >>> >>>> We meet a urgent issue because of Shijith's patch. >>> >>>> When use DPDK version 17.08, DPDK VF driver and kernel PF driver, i40e, >>>> ixgbe >>> and qede(Harish said) VF all meet initial failure. >>> >>>> This issue only occurred when use 17.08 igb_uio, vfio-pci and 17.05 >>>> igb_uio all >>> works well. >>> >>>> Considering this issue will block our vf driver develop and test work, >>>> could we >>> revert this patch first and find another appropriate way? >>> >>>> >>> >>>> Qiming >>> >>>> >>> >>>>> -----Original Message----- >>> >>>>> From: Patil, Harish [mailto:harish.pa...@cavium.com] >>> >>>>> Sent: Thursday, September 14, 2017 3:45 AM >>> >>>>> To: Yigit, Ferruh <ferruh.yi...@intel.com<mailto:ferruh.yi...@intel.com>>; >>> Hu, Xuekun <xuekun...@intel.com<mailto:xuekun...@intel.com>>; >>> >>>>> Thotton, Shijith >>> <shijith.thot...@cavium.com<mailto:shijith.thot...@cavium.com>>; Yang, >>> Qiming >>> >>>>> <qiming.y...@intel.com<mailto:qiming.y...@intel.com>>; Gregory Etelson >>> <greg...@weka.io<mailto:greg...@weka.io>> >>> >>>>> Cc: dev@dpdk.org<mailto:dev@dpdk.org>; Tan, Jianfeng >>> <jianfeng....@intel.com<mailto:jianfeng....@intel.com>> >>> >>>>> Subject: Re: [dpdk-dev] vf init issue with patch igb_uio: issue FLR >>>>> during open >>> >>>>> and release of device file >>> >>>>> >>> >>>>> >>> >>>>> >>> >>>>> -----Original Message----- >>> >>>>> From: dev <dev-boun...@dpdk.org<mailto:dev-boun...@dpdk.org>> on >>> behalf of Ferruh Yigit >>> >>>>> <ferruh.yi...@intel.com<mailto:ferruh.yi...@intel.com>> >>> >>>>> Date: Wednesday, September 13, 2017 at 10:06 AM >>> >>>>> To: "Hu, Xuekun" <xuekun...@intel.com<mailto:xuekun...@intel.com>>, >>> "Thotton, Shijith" >>> >>>>> <shijith.thot...@cavium.com<mailto:shijith.thot...@cavium.com>>, >>> "Yang, Qiming" <qiming.y...@intel.com<mailto:qiming.y...@intel.com>>, >>> >>>>> Gregory Etelson <greg...@weka.io<mailto:greg...@weka.io>> >>> >>>>> Cc: "dev@dpdk.org<mailto:dev@dpdk.org>" >>> <dev@dpdk.org<mailto:dev@dpdk.org>>, "Tan, Jianfeng" >>> <jianfeng....@intel.com<mailto:jianfeng....@intel.com>> >>> >>>>> Subject: Re: [dpdk-dev] vf init issue with patch igb_uio: issue FLR >>>>> during open >>> >>>>> and release of device file >>> >>>>> >>> >>>>>> On 9/13/2017 3:25 PM, Hu, Xuekun wrote: >>> >>>>>>> I met the same issue too, only with i40e 2.1.26 PF kernel driver. >>> >>>>>>> >>> >>>>>>> -----Original Message----- >>> >>>>>>> From: dev [mailto:dev-boun...@dpdk.org] On Behalf Of Ferruh Yigit >>> >>>>>>> Sent: Wednesday, September 13, 2017 7:04 PM >>> >>>>>>> To: Shijith Thotton >>> <shijith.thot...@caviumnetworks.com<mailto:shijith.thotton@caviumnetworks >>> .com>>; Yang, >>> >>>>>>> Qiming <qiming.y...@intel.com<mailto:qiming.y...@intel.com>>; >>> Gregory Etelson <greg...@weka.io<mailto:greg...@weka.io>> >>> >>>>>>> Cc: dev@dpdk.org<mailto:dev@dpdk.org>; Tan, Jianfeng >>> <jianfeng....@intel.com<mailto:jianfeng....@intel.com>> >>> >>>>>>> Subject: Re: [dpdk-dev] vf init issue with patch igb_uio: issue FLR >>> >>>>>>> during open and release of device file >>> >>>>>>> >>> >>>>>>> On 9/13/2017 11:48 AM, Shijith Thotton wrote: >>> >>>>>>>> On Wed, Sep 13, 2017 at 07:51:30AM +0000, Yang, Qiming wrote: >>> >>>>>>>>> Hi, Shijith >>> >>>>>>>>> >>> >>>>>>>>> >>> >>>>>>>>> >>> >>>>>>>>> VF init error will happen after apply your patch, error log as >>> >>>>>>>>> below. If >>> >>>>>>>>> revert your commit, all things work well. And this issue is not >>> >>>>>>>>> only occur >>> >>>>>>>>> in i40 VF but also ixgbe. >>> >>>>>> >>> >>>>>> Hi Qiming, >>> >>>>>> >>> >>>>>> I can reproduce the issue for the case: >>> >>>>>> >>> >>>>>> 1- i40e 2.1.26 PF kernel driver (NOT reproduced with 1.6.27-k) >>> >>>>>> 2- DPDK app run on host, so both PF and VF are in host. Linux driver >>> >>>>>> for PF and DPDK for VF. (When VF used in a VM, Linux PF on host and >>> >>>>>> DPDK VF on guest, this works fine) >>> >>>>>> >>> >>>>>> And I confirm the function igbuio_pci_open() added with below patch is >>> >>>>>> causing it. >>> >>>>>> >>> >>>>>> igbuio_pci_open() does: >>> >>>>>> pci_reset_function(dev); >>> >>>>>> pci_set_master(dev); >>> >>>>>> >>> >>>>>> Don't know yet root cause of the error, will dig more. >>> >>>>>> >>> >>>>>>>>> >>> >>>>>>>>> Could you help to check it soon? >>> >>>>>>>>> >>> >>>>>>>>> >>> >>>>>>>>> >>> >>>>>>>>> [root@localhost app]# ./testpmd -c 7 -n 4 -- -i >>> >>>>>>>>> >>> >>>>>>>>> EAL: Detected 10 lcore(s) >>> >>>>>>>>> >>> >>>>>>>>> EAL: No free hugepages reported in hugepages-1048576kB >>> >>>>>>>>> >>> >>>>>>>>> EAL: Probing VFIO support... >>> >>>>>>>>> >>> >>>>>>>>> EAL: WARNING: cpu flags constant_tsc=yes nonstop_tsc=no -> using >>> >>>>>>>>> unreliable clock cycles ! >>> >>>>>>>>> >>> >>>>>>>>> EAL: PCI device 0000:00:03.0 on NUMA socket -1 >>> >>>>>>>>> >>> >>>>>>>>> EAL: Invalid NUMA socket, default to 0 >>> >>>>>>>>> >>> >>>>>>>>> EAL: probe driver: 8086:154c net_i40e_vf >>> >>>>>>>>> >>> >>>>>>>>> i40evf_init_vf(): init_adminq failed: -53 >>> >>>>>>>>> >>> >>>>>>>>> i40evf_dev_init(): Init vf failed >>> >>>>>>>>> >>> >>>>>>>>> EAL: Requested device 0000:00:03.0 cannot be used >>> >>>>>>>>> >>> >>>>>>>>> >>> >>>>>>>>> >>> >>>>>>>>> commit b58eedfc7dd57eef6d12e2c654a52c834f36084a >>> >>>>>>>>> Author: Shijith Thotton >>> <shijith.thot...@caviumnetworks.com<mailto:shijith.thotton@caviumnetworks >>> .com>> >>> >>>>>>>>> Date: Fri Jul 7 16:43:51 2017 +0530 >>> >>>>>>>>> >>> >>>>>>>>> igb_uio: issue FLR during open and release of device file >>> >>>>>>>>> >>> >>>>>>>>> Set UIO info device file operations open and release. Call pci >>> >>>>>>>>> reset >>> >>>>>>>>> function inside open and release to clear device state at start >>> >>>>>>>>> and end. >>> >>>>>>>>> Copied this behaviour from vfio_pci kernel module code. With >>> >>>>>>>>> this patch, >>> >>>>>>>>> it is not mandatory to issue FLR by PMD's during init and close. >>> >>>>>>>>> >>> >>>>>>>>> Bus master enable and disable are added in open and release >>> >>>>>>>>> respectively >>> >>>>>>>>> to take care of device DMA. >>> >>>>>>>>> >>> >>>>>>>>> Signed-off-by: Shijith Thotton >>> <shijith.thot...@caviumnetworks.com<mailto:shijith.thotton@caviumnetworks >>> .com>> >>> >>>>>>>>> Reviewed-by: Jianfeng Tan >>> <jianfeng....@intel.com<mailto:jianfeng....@intel.com>> >>> >>>>>>>>> Acked-by: Ferruh Yigit >>> <ferruh.yi...@intel.com<mailto:ferruh.yi...@intel.com>> >>> >>>>>>>>> Acked-by: Gregory Etelson >>> <greg...@weka.io<mailto:greg...@weka.io>> >>> >>>>>>>>> >>> >>>>>>>>> >>> >>>>>>>>> >>> >>>>>>>>> Best Regard, >>> >>>>>>>>> >>> >>>>>>>>> Yang Qiming >>> >>>>>>>>> >>> >>>>>>>>> >>> >>>>>>>> >>> >>>>>>>> Hi Yang Qiming, >>> >>>>>>>> >>> >>>>>>>> We didn't face this issue while preparing patch. I think, Gregory >>> >>>>>>>> tested it for Intel NICs. Adding more people to the loop for help. >>> >>>>>>>> >>> >>>>>>> >>> >>>>>>> I will try to reproduce the issue. >>> >>>>>>> >>> >>>>>>>> Shijith >>> >>>>> >>> >>>>> [Harish] We are seeing similar issue with the bringup of qede VF devices >>>>> due >>> to >>> >>>>> this patch. >>> >>>>> Could we revert this and leave it to individual PMDs to manage it? >>> >>>>>>> >>> >>>> >>> >>>> >>> >>
