On Wed, Apr 14, 2021 at 04:33:10PM +0100, Burakov, Anatoly wrote:
> On 14-Apr-21 2:45 PM, Liang Ma wrote:
> > On Wed, Apr 14, 2021 at 02:21:53PM +0100, Bruce Richardson wrote:
> > > On Wed, Apr 14, 2021 at 01:52:14PM +0100, Liang Ma wrote:
> > > > On Wed, Apr 14, 2021 at 11:48:03AM +0100, Bruce Richardson wrote:
> > > > > On Wed, Apr 14, 2021 at 11:37:48AM +0100, Liang Ma wrote:
> > > > > > On Tue, Apr 13, 2021 at 07:24:41PM +0100, Bruce Richardson wrote:
> > > > > > > On Tue, Apr 13, 2021 at 06:36:38PM +0100, Liang Ma wrote:
> > > > > > > > Hi Bruce,
> > > > > > > > I look into current IOAT PMD. I found that looks only
> > > > > > > > support the
> > > > > > > > latest DSA device which is part of SPR(Please correct me if
> > > > > > > > I'm wrong).
> > > > > > > > However, there still is lots existing CPU that support
> > > > > > > > previous CBDMA
> > > > > > > > Intel Quick-DATA technology. SPDK enable those device with
> > > > > > > > uio.
> > > > > > > > Any hint to enable those CBDMA device with IOAT PMD ?
> > > > > > >
> > > > > > > Hi Liang,
> > > > > > >
> > > > > > > These devices are still supported by the driver. In the latest
> > > > > > > releases the
> > > > > > > new device support was added but no older device support was
> > > > > > > removed.
> > > > > > > Please let us know if you do hit any problems with running it on
> > > > > > > the CBDMA
> > > > > > > HW devices, because it should all still be working as before.
> > > > > > >
> > > > > > > Regards,
> > > > > > > /Bruce
> > > > > > Hi Bruce,
> > > > > > I hit a problem with CBDMA enabling.
> > > > > >
> > > > > > 1. I follow the DPDK IOAT driver Doc, I bind 0000.04.0 device
> > > > > > with
> > > > > > dpdk-devbind.py
> > > > > > 2. I run dpdk-test without any parameter then dpdk-test
> > > > > > complain with
> > > > > >
> > > > > > EAL: VFIO support initialized
> > > > > > EAL: 0000:00:04.0 VFIO group is not viable! Not all devices
> > > > > > in IOMMU
> > > > > > group bound to VFIO or unbound
> > > > > > Requested device 0000:00:04.0 cannot be used
> > > > > >
> > > > > > 3. kernel boot parameter include the "intel_iommu=on"
> > > > > >
> > > > > > If I still need some extra iommu settings for CBDMA device ?
> > > > > >
> > > > > It's an IOMMU limitation, not just for CBDMA but for other devices
> > > > > like
> > > > > NICs too - you need to ensure all subfunctions on a device are all
> > > > > bound to
> > > > > the same IOMMU group. So for the CBDMA instance, to use one channel
> > > > > with
> > > > > vfio, the other channels on the device also need to be bound to vfio,
> > > > > or
> > > > > else not bound to any driver. If bound to the kernel driver, there
> > > > > will be
> > > > > IOMMU conflicts as we can't split the channels between kernel and
> > > > > userspace
> > > > > address spaces.
> > > > >
> > > > > /Bruce
> > > > I bind whole device from 00:04.0 to 00:04.7. ioat PMD complain
> > > > with "ioat_rawdev_create(): ioat_rawdev_create: Channel appears
> > > > locked". Any hint?
> > > > Regards
> > > > Liang
> > >
> > > That should be a warning only, I think and the device should still be
> > > available. Is that not the case, or is there a later error?
> > >
> > > /Bruce
> > Good news, that's a warning only message, I checked the code.
> > after probe the device, rawdev_autotest all passed !
> > Many thanks for your help !
> > I strongly recommend to add the IOMMU hint of CBDMA(bind whole device)
> > into the Doc. I think DSA device don't need to do that right ?
> > Best Regards
> > Liang
> >
>
> This is documented in our VFIO docs [1].
>
> [1]
> http://doc.dpdk.org/guides/linux_gsg/linux_drivers.html#binding-and-unbinding-network-ports-to-from-the-kernel-modules
Thanks Anatoly.
