Re: [PATCH] RDMA/cxgb3: fail get_dma_mr if the memory footprint can exceed 32b
On 07/24/2015 10:50 AM, Steve Wise wrote: > > >> -Original Message- >> From: linux-rdma-ow...@vger.kernel.org >> [mailto:linux-rdma-ow...@vger.kernel.org] On Behalf Of Doug Ledford >> Sent: Friday, July 24, 2015 9:45 AM >> To: Steve Wise >> Cc: linux-rdma@vger.kernel.org >> Subject: Re: [PATCH] RDMA/cxgb3: fail get_dma_mr if the memory footprint can >> exceed 32b >> >> On 07/23/2015 06:47 PM, Steve Wise wrote: >>>> -Original Message- From: Doug Ledford >>>> Should this be a static check of the pointer size versus installed >>>> memory? Would it be possible to have this work for machines with >>>> less than 4GB of physical memory even if they have 64bit pointers, >>>> or are you concerned that hotplug memory could take us over the >>>> limit after registration and cause problems? >>> >>> NFSRDMA doesn't need dma-mrs for T3 since it has FRMR + local dma >>> lkey support. And since the deficiency really can cause problems on >>> 64b systems if the memory grows > 4GB after dma-mr allocation, I >>> decided to just not allow them for potential large memory systems. >> >> Ok. I've pulled this for 4.2-rc then. >> > > The problem has been there since day one, so it doesn't represent a > regression. It is your call, but I think 4.3 is fine. It's a long standing problem, but it's a potential memory wrap issue, so might as well fix it (especially given the simple nature of the fix). I've got a few other things heading in for 4.2-rc, so no biggie. -- Doug Ledford GPG KeyID: 0E572FDD signature.asc Description: OpenPGP digital signature
RE: [PATCH] RDMA/cxgb3: fail get_dma_mr if the memory footprint can exceed 32b
> -Original Message- > From: linux-rdma-ow...@vger.kernel.org > [mailto:linux-rdma-ow...@vger.kernel.org] On Behalf Of Doug Ledford > Sent: Friday, July 24, 2015 9:45 AM > To: Steve Wise > Cc: linux-rdma@vger.kernel.org > Subject: Re: [PATCH] RDMA/cxgb3: fail get_dma_mr if the memory footprint can > exceed 32b > > On 07/23/2015 06:47 PM, Steve Wise wrote: > >> -Original Message- From: Doug Ledford > >> Should this be a static check of the pointer size versus installed > >> memory? Would it be possible to have this work for machines with > >> less than 4GB of physical memory even if they have 64bit pointers, > >> or are you concerned that hotplug memory could take us over the > >> limit after registration and cause problems? > > > > NFSRDMA doesn't need dma-mrs for T3 since it has FRMR + local dma > > lkey support. And since the deficiency really can cause problems on > > 64b systems if the memory grows > 4GB after dma-mr allocation, I > > decided to just not allow them for potential large memory systems. > > Ok. I've pulled this for 4.2-rc then. > The problem has been there since day one, so it doesn't represent a regression. It is your call, but I think 4.3 is fine. Thanks, Steve. -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] RDMA/cxgb3: fail get_dma_mr if the memory footprint can exceed 32b
On 07/23/2015 06:47 PM, Steve Wise wrote: >> -Original Message- From: Doug Ledford >> Should this be a static check of the pointer size versus installed >> memory? Would it be possible to have this work for machines with >> less than 4GB of physical memory even if they have 64bit pointers, >> or are you concerned that hotplug memory could take us over the >> limit after registration and cause problems? > > NFSRDMA doesn't need dma-mrs for T3 since it has FRMR + local dma > lkey support. And since the deficiency really can cause problems on > 64b systems if the memory grows > 4GB after dma-mr allocation, I > decided to just not allow them for potential large memory systems. Ok. I've pulled this for 4.2-rc then. -- Doug Ledford GPG KeyID: 0E572FDD signature.asc Description: OpenPGP digital signature
RE: [PATCH] RDMA/cxgb3: fail get_dma_mr if the memory footprint can exceed 32b
> -Original Message-
> From: Doug Ledford [mailto:dledf...@redhat.com]
> Sent: Thursday, July 23, 2015 4:33 PM
> To: Steve Wise
> Cc: linux-rdma@vger.kernel.org
> Subject: Re: [PATCH] RDMA/cxgb3: fail get_dma_mr if the memory footprint can
> exceed 32b
>
> On 07/22/2015 03:14 PM, Steve Wise wrote:
> > T3 HW only supports MRs of length < 4GB. If the system can have more
> > than that we need to fail dma mr allocation so we con't create a MR that
> > cannot span the entire possible memory space.
> >
> > Signed-off-by: Steve Wise
> > ---
> >
> > drivers/infiniband/hw/cxgb3/iwch_provider.c |4
> > 1 files changed, 4 insertions(+), 0 deletions(-)
> >
> > diff --git a/drivers/infiniband/hw/cxgb3/iwch_provider.c
> > b/drivers/infiniband/hw/cxgb3/iwch_provider.c
> > index b1b7323..bbbe018 100644
> > --- a/drivers/infiniband/hw/cxgb3/iwch_provider.c
> > +++ b/drivers/infiniband/hw/cxgb3/iwch_provider.c
> > @@ -736,6 +736,10 @@ static struct ib_mr *iwch_get_dma_mr(struct ib_pd *pd,
> > int acc)
> > /*
> > * T3 only supports 32 bits of size.
> > */
> > + if (sizeof(phys_addr_t) > 4) {
> > + pr_warn_once(MOD "Cannot support dma_mrs on this platform.\n");
> > + return ERR_PTR(-ENOTSUPP);
> > + }
> > bl.size = 0x;
> > bl.addr = 0;
> > kva = 0;
>
> Should this be a static check of the pointer size versus installed
> memory? Would it be possible to have this work for machines with less
> than 4GB of physical memory even if they have 64bit pointers, or are you
> concerned that hotplug memory could take us over the limit after
> registration and cause problems?
NFSRDMA doesn't need dma-mrs for T3 since it has FRMR + local dma lkey support.
And since the deficiency really can cause problems on 64b systems if the
memory grows > 4GB after dma-mr allocation, I decided to just not allow them
for potential large memory systems.
Steve
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] RDMA/cxgb3: fail get_dma_mr if the memory footprint can exceed 32b
On 07/22/2015 03:14 PM, Steve Wise wrote:
> T3 HW only supports MRs of length < 4GB. If the system can have more
> than that we need to fail dma mr allocation so we con't create a MR that
> cannot span the entire possible memory space.
>
> Signed-off-by: Steve Wise
> ---
>
> drivers/infiniband/hw/cxgb3/iwch_provider.c |4
> 1 files changed, 4 insertions(+), 0 deletions(-)
>
> diff --git a/drivers/infiniband/hw/cxgb3/iwch_provider.c
> b/drivers/infiniband/hw/cxgb3/iwch_provider.c
> index b1b7323..bbbe018 100644
> --- a/drivers/infiniband/hw/cxgb3/iwch_provider.c
> +++ b/drivers/infiniband/hw/cxgb3/iwch_provider.c
> @@ -736,6 +736,10 @@ static struct ib_mr *iwch_get_dma_mr(struct ib_pd *pd,
> int acc)
> /*
>* T3 only supports 32 bits of size.
>*/
> + if (sizeof(phys_addr_t) > 4) {
> + pr_warn_once(MOD "Cannot support dma_mrs on this platform.\n");
> + return ERR_PTR(-ENOTSUPP);
> + }
> bl.size = 0x;
> bl.addr = 0;
> kva = 0;
Should this be a static check of the pointer size versus installed
memory? Would it be possible to have this work for machines with less
than 4GB of physical memory even if they have 64bit pointers, or are you
concerned that hotplug memory could take us over the limit after
registration and cause problems?
--
Doug Ledford
GPG KeyID: 0E572FDD
signature.asc
Description: OpenPGP digital signature
[PATCH] RDMA/cxgb3: fail get_dma_mr if the memory footprint can exceed 32b
T3 HW only supports MRs of length < 4GB. If the system can have more
than that we need to fail dma mr allocation so we con't create a MR that
cannot span the entire possible memory space.
Signed-off-by: Steve Wise
---
drivers/infiniband/hw/cxgb3/iwch_provider.c |4
1 files changed, 4 insertions(+), 0 deletions(-)
diff --git a/drivers/infiniband/hw/cxgb3/iwch_provider.c
b/drivers/infiniband/hw/cxgb3/iwch_provider.c
index b1b7323..bbbe018 100644
--- a/drivers/infiniband/hw/cxgb3/iwch_provider.c
+++ b/drivers/infiniband/hw/cxgb3/iwch_provider.c
@@ -736,6 +736,10 @@ static struct ib_mr *iwch_get_dma_mr(struct ib_pd *pd, int
acc)
/*
* T3 only supports 32 bits of size.
*/
+ if (sizeof(phys_addr_t) > 4) {
+ pr_warn_once(MOD "Cannot support dma_mrs on this platform.\n");
+ return ERR_PTR(-ENOTSUPP);
+ }
bl.size = 0x;
bl.addr = 0;
kva = 0;
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
