> -----Original Message----- > From: Stojaczyk, Dariusz <dariusz.stojac...@intel.com> > Sent: Tuesday, July 23, 2019 10:24 AM > To: Jerin Jacob Kollanukkaran <jer...@marvell.com>; Thomas Monjalon > <tho...@monjalon.net>; David Marchand <david.march...@redhat.com>; > Burakov, Anatoly <anatoly.bura...@intel.com> > Cc: dev@dpdk.org > Subject: [EXT] RE: [dpdk-dev] [PATCH v4 0/4] Fixes on IOVA mode selection > > > -----Original Message----- > > From: Jerin Jacob Kollanukkaran [mailto:jer...@marvell.com] > > Sent: Tuesday, July 23, 2019 6:19 AM > > > > > -----Original Message----- > > > From: Stojaczyk, Dariusz <dariusz.stojac...@intel.com> > > > Sent: Tuesday, July 23, 2019 9:06 AM > > > To: Thomas Monjalon <tho...@monjalon.net>; David Marchand > > > <david.march...@redhat.com>; Burakov, Anatoly > > > <anatoly.bura...@intel.com>; Jerin Jacob Kollanukkaran > > > <jer...@marvell.com> > > > Cc: dev@dpdk.org > > > Subject: [EXT] RE: [dpdk-dev] [PATCH v4 0/4] Fixes on IOVA mode > > > selection > > > > > > This introduces a regression where uio-bound devies are attached to > > > a > > DPDK > > > app at runtime. > > > > Just to understand the requirements; > > # Is this requirement for SPDK? > > # Is brand new PCI device scanned and attached to DPDK at runtime? > > # Any specific reason for using uio vs vfio? > > Jerin,
Stojaczyk, There reason to choose VA incase if bus detects DC is following: - All drivers are expected to work in RTE_IOVA_VA mode, irrespective of physical address availability. - By default, the mempool, first asks for IOVA-contiguous memory using ``RTE_MEMZONE_IOVA_CONTIG``. This is slow in RTE_IOVA_PA mode and it may affect the application boot time. - It is easy to enable large amount of IOVA-contiguous memory use-cases with IOVA in VA mode. > > It came up in SPDK tests, but it's certainly nothing SPDK-specific, I can't > give > you the steps but it should be reproducible even with testpmd. > > The PCI device could have been simply hotplugged to the system after DPDK > app start. DPDK didn't know about it at initialization, so it picked > RTE_IOVA_VA and then would fail to attach any UIO-bound device ever > after: > > EAL: Expecting 'PA' IOVA mode but current mode is 'VA', not initializing We have RTE_PCI_DRV_NEED_IOVA_AS_VA devices in DPDK, Which can work Only on VA. If we default 'PA' incase of DC, then what do with hotplugging on those devices? > EAL: Driver cannot attach the device (0000:00:09.0) > EAL: Failed to attach device on primary process > > UIO is commonly used on systems without IOMMU- including VMs. The latest machines has IOMMU. Which machines you are testing against, Can we detect the machines without IOMMU and switch to PA? > > > > > If it is for SPDK, > > # How about introducing rte_eal_init_with_mode(enum rte_iova_mode)? > > # How about adding dummy bus which returns RTE_IOVA_PA in the > > bus_get_iommus_class() in SPDK code base? > > There's already an --iova=mode option in DPDK that forces the iova mode. > I'm not concerned about configurability, but the regression in the default > behavior. > > I can add workarounds to SPDK, sure, but that wouldn't be a very healthy > approach. Nothing like workaround, I am looking for the options for expressing The requirements for PA? > > D. > > > > > > > > > When there are no devices attached at initialization, the only safe > > > default should be RTE_IOVA_PA. With RTE_IOVA_VA we just won't be > > > able to do any DMA to uio-bound PCI devices. > > > > > > Can we revert this patch? > > > > > > D. > > > > > > > -----Original Message----- > > > > From: dev [mailto:dev-boun...@dpdk.org] On Behalf Of Thomas > > Monjalon > > > > Sent: Monday, July 22, 2019 5:53 PM > > > > To: David Marchand <david.march...@redhat.com>; Burakov, Anatoly > > > > <anatoly.bura...@intel.com>; jer...@marvell.com > > > > Cc: dev@dpdk.org > > > > Subject: Re: [dpdk-dev] [PATCH v4 0/4] Fixes on IOVA mode > > > > selection > > > > > > > > 22/07/2019 14:56, David Marchand: > > > > > Following the issues reported by Jerin and the discussion that > > > > > emerged from it, here are fixes to restore and document the > > > > > behavior of the EAL and the pci bus driver. > > > > > > > > > > I pondered all the arguments and tried to have the less changes > > > > > possible. > > > > > I can't find a need for a flag to just announce support of > > > > > physical addresses from the pmd point of view. > > > > > So it ended up with something really close to what Jerin had > suggested. > > > > > > > > > > But the problem is that this is still unfinished wrt the > > > > > documentation. > > > > > I will be offline for 10 days and we need this to move forward, > > > > > so sending anyway. > > > > > > > > > > Changelog since v3: > > > > > - fixed typos in patch 2, > > > > > - updated patch 3 title, > > > > > - moved and reworded comments in the note section in patch 4, > > > > > > > > > > Changelog since v2 (Jerin): > > > > > - Patch 2/4 - Remove personal appeals in log messages(Anatoly) > > > > > - Patch 4/4 - Added documentation (Anatoly) > > > > > > > > > > Changelog since v1 (Jerin): > > > > > - Changed RTE_PCI_DRV_IOVA_AS_VA flag name as > > > > RTE_PCI_DRV_NEED_IOVA_AS_VA > > > > > (patch 3/4) > > > > > - Changed IOVA mode as VA for default case(patch 4/4) with > > > > documentation > > > > > - Tested the patch series on octeontx2 platform > > > > > > > > Applied, thanks Jerin, Anatoly and David for converging on a > > > > documented solution together. > > > > > > > >