On 10/28/2015 07:28 AM, Sagi Grimberg wrote: > mlx4 devices (ConnectX-2, ConnectX-3) has a limitation > where rdma read work queue entries cannot exceed 512 bytes. > A rdma_read wqe needs to fit in 512 bytes: > - wqe control segment (16 bytes) > - rdma segment (16 bytes) > - scatter elements (16 bytes each) > > So max_sge_rd should be: (512 - 16 - 16) / 16 = 30. > > Signed-off-by: Sagi Grimberg <sa...@mellanox.com> > Reviewed-by: Steve Wise <sw...@opengridcomputing.com> > --- > drivers/infiniband/hw/mlx4/main.c | 2 +- > include/linux/mlx4/device.h | 11 +++++++++++ > 2 files changed, 12 insertions(+), 1 deletions(-) > > diff --git a/drivers/infiniband/hw/mlx4/main.c > b/drivers/infiniband/hw/mlx4/main.c > index 7026580..e0232a2 100644 > --- a/drivers/infiniband/hw/mlx4/main.c > +++ b/drivers/infiniband/hw/mlx4/main.c > @@ -499,7 +499,7 @@ static int mlx4_ib_init_device_flags(struct ib_device > *ibdev) > ibdev->max_qp_wr = dev->dev->caps.max_wqes - > MLX4_IB_SQ_MAX_SPARE; > ibdev->max_sge = min(dev->dev->caps.max_sq_sg, > dev->dev->caps.max_rq_sg); > - ibdev->max_sge_rd = ibdev->max_sge; > + ibdev->max_sge_rd = MLX4_MAX_SGE_RD; > ibdev->max_cq = dev->dev->quotas.cq; > ibdev->max_cqe = dev->dev->caps.max_cqes; > ibdev->max_mr = dev->dev->quotas.mpt;
What code tree did you make this patch from? Because in an upstream 4.4
kernel, there is no function mlx4_ib_init_device_flags(), but there is
mlx4_ib_query_device() that does much the same thing. But in there,
there is no ibdev->max_sge_rd, there is props->max_sge_rd. It's like
this came from an entirely different source base somewhere.
> diff --git a/include/linux/mlx4/device.h b/include/linux/mlx4/device.h
> index dac6872..3dcae41 100644
> --- a/include/linux/mlx4/device.h
> +++ b/include/linux/mlx4/device.h
> @@ -427,6 +427,17 @@ enum {
> };
>
> enum {
> + /*
> + * Max wqe size for rdma read is 512 bytes, so this
> + * limits our max_sge_rd as the wqe needs to fit:
> + * - ctrl segment (16 bytes)
> + * - rdma segment (16 bytes)
> + * - scatter elements (16 bytes each)
> + */
> + MLX4_MAX_SGE_RD = (512 - 16 - 16) / 16
> +};
> +
> +enum {
> MLX4_DEV_PMC_SUBTYPE_GUID_INFO = 0x14,
> MLX4_DEV_PMC_SUBTYPE_PORT_INFO = 0x15,
> MLX4_DEV_PMC_SUBTYPE_PKEY_TABLE = 0x16,
>
--
Doug Ledford <dledf...@redhat.com>
GPG KeyID: 0E572FDD
signature.asc
Description: OpenPGP digital signature
