On 7/22/2019 1:56 PM, David Marchand wrote:
> The incriminated commit broke the use of RTE_PCI_DRV_IOVA_AS_VA which
> was intended to mean "driver only supports VA" but had been understood
> as "driver supports both PA and VA" by most net drivers and used to let
> dpdk processes to run as non root (which do not have access to physical
> addresses on recent kernels).
>
> The check on physical addresses actually closed the gap for those
> drivers. We don't need to mark them with RTE_PCI_DRV_IOVA_AS_VA and this
> flag can retain its intended meaning.
> Document explicitly its meaning.
>
> We can check that a driver requirement wrt to IOVA mode is fulfilled
> before trying to probe a device.
>
> Finally, document the heuristic used to select the IOVA mode and hope
> that we won't break it again.
>
> Fixes: 703458e19c16 ("bus/pci: consider only usable devices for IOVA mode")
>
> Signed-off-by: David Marchand <[email protected]>
> Reviewed-by: Jerin Jacob <[email protected]>
> Tested-by: Jerin Jacob <[email protected]>
> Acked-by: Anatoly Burakov <[email protected]>
<...>
> diff --git a/drivers/bus/pci/pci_common.c b/drivers/bus/pci/pci_common.c
> index d2af472..9794552 100644
> --- a/drivers/bus/pci/pci_common.c
> +++ b/drivers/bus/pci/pci_common.c
> @@ -169,8 +169,22 @@ static struct rte_devargs *pci_devargs_lookup(struct
> rte_pci_device *dev)
> * This needs to be before rte_pci_map_device(), as it enables to use
> * driver flags for adjusting configuration.
> */
> - if (!already_probed)
> + if (!already_probed) {
> + enum rte_iova_mode dev_iova_mode;
> + enum rte_iova_mode iova_mode;
> +
> + dev_iova_mode = pci_device_iova_mode(dr, dev);
> + iova_mode = rte_eal_iova_mode();
> + if (dev_iova_mode != RTE_IOVA_DC &&
> + dev_iova_mode != iova_mode) {
> + RTE_LOG(ERR, EAL, " Expecting '%s' IOVA mode but
> current mode is '%s', not initializing\n",
> + dev_iova_mode == RTE_IOVA_PA ? "PA" : "VA",
> + iova_mode == RTE_IOVA_PA ? "PA" : "VA");
> + return -EINVAL;
> + }
> +
OvS reported an error while hotplugging a device.
It looks like DPDK application initialized as IOVA=VA, and the new device is
bound to 'igb_uio' which forces it to PA, fails on above check.
I would like to get your comment on the issue.
For the OvS mode, hopefully binding the device to 'vfio-pci' can be a solution,
but for the cases we don't have that option, can/should we force the DPDK to PA
mode after initialization?
