Re: [openib-general] [PATCH/RFC 2.6.21] ehca: ehca_uverbs.c: refactor ehca_mmap() for better readability
> > Just want to make sure I understand this properly: > > One day, if eHCA supports srq, will I have to set qp pointer with a > > valid address or can I still leave it to NULL? > I though ehca hardware can't do SRQ? Yes, current ehca hardware can't SRQ. Our next generation eHCA will support SRQ. ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] [PATCH/RFC 2.6.21] ehca: ehca_uverbs.c: refactor ehca_mmap() for better readability
> > actually the IB spec requires QPN in the work completion if either SRQ > > or base queue management extensions are supported. I'm not sure why > > though. It seems at least for current code it is fine for ehca to > > report wc->qp as NULL. > > I've update the patch in my for-2.6.21 branch to do that, and pushed > > it out. > Just want to make sure I understand this properly: > One day, if eHCA supports srq, will I have to set qp pointer with a > valid address or can I still leave it to NULL? I though ehca hardware can't do SRQ? But anyway, the idea behind this API change is that for SRQ receive completions, then the ib_wc will have the QP pointer that the receive happened on rather than just the QPN. > Roland, you are talking about QPN. By that you mean also qp pointer? Well, I was quoting the IB spec which talks about QPN. But we're replacing that (fairly useless) info with a pointer to the actual QP, so that the consumer can get to qp_context. -R. ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] [PATCH/RFC 2.6.21] ehca: ehca_uverbs.c: refactor ehca_mmap() for better readability
[EMAIL PROTECTED] wrote on 18.01.2007 16:26:58: > > qpnum is actually only required from poll cq if srq is supported. > > so ehca can just fill in NULL. > actually the IB spec requires QPN in the work completion if either SRQ > or base queue management extensions are supported. I'm not sure why > though. It seems at least for current code it is fine for ehca to > report wc->qp as NULL. > I've update the patch in my for-2.6.21 branch to do that, and pushed > it out. Just want to make sure I understand this properly: One day, if eHCA supports srq, will I have to set qp pointer with a valid address or can I still leave it to NULL? Roland, you are talking about QPN. By that you mean also qp pointer? Regards Nam ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] [PATCH/RFC 2.6.21] ehca: ehca_uverbs.c: refactor ehca_mmap() for better readability
On Thu, Jan 18, 2007 at 10:56:01AM -0800, Roland Dreier wrote: > I've kind of lost the plot here. How does this patch fit in with the > previous series of patches you posted? Does it replace them or go on > top of them? It's a cleanup ontop of the actual fix. ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] [PATCH/RFC 2.6.21] ehca: ehca_uverbs.c: refactor ehca_mmap() for better readability
> qpnum is actually only required from poll cq if srq is supported. > so ehca can just fill in NULL. actually the IB spec requires QPN in the work completion if either SRQ or base queue management extensions are supported. I'm not sure why though. It seems at least for current code it is fine for ehca to report wc->qp as NULL. I've update the patch in my for-2.6.21 branch to do that, and pushed it out. ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] [PATCH/RFC 2.6.21] ehca: ehca_uverbs.c: refactor ehca_mmap() for better readability
> Quoting Roland Dreier <[EMAIL PROTECTED]>: > Subject: Re: [PATCH/RFC 2.6.21] ehca: ehca_uverbs.c: refactor ehca_mmap() for > better readability > > > Roland, please note that I applied the previous patches to > > your git tree for-2.6.21 before creating this patch. I also > > realized a compile issue with the patch from Michael T. in > > ehca_reqs.c regarding "return qp pointer in ib_wc". For this > > I'll send another patch. > > Michael -- I didn't notice before but you do seem to have misread the > ehca CQ polling code. The ehca CQ poll operation doesn't actually > have the qp pointer readily available -- unless I'm mistaken, ehca > will have to do an expensive search by QPN in a hash table to get the > qp pointer. That doesn't seem acceptable for every CQ poll, so do you > have any ideas of how to salvage this API change? qpnum is actually only required from poll cq if srq is supported. so ehca can just fill in NULL. -- MST ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] [PATCH/RFC 2.6.21] ehca: ehca_uverbs.c: refactor ehca_mmap() for better readability
> > Roland, please note that I applied the previous patches to > > your git tree for-2.6.21 before creating this patch. I also > > realized a compile issue with the patch from Michael T. in > > ehca_reqs.c regarding "return qp pointer in ib_wc". For this > > I'll send another patch. > Michael -- I didn't notice before but you do seem to have misread the > ehca CQ polling code. The ehca CQ poll operation doesn't actually > have the qp pointer readily available -- unless I'm mistaken, ehca > will have to do an expensive search by QPN in a hash table to get the > qp pointer. That doesn't seem acceptable for every CQ poll, so do you > have any ideas of how to salvage this API change? Yes, your concern is correct. ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] [PATCH/RFC 2.6.21] ehca: ehca_uverbs.c: refactor ehca_mmap() for better readability
No problem. Will resend the full patch set for 2.6.21. Thanks Nam [EMAIL PROTECTED] wrote on 18.01.2007 13:56:01: > I've kind of lost the plot here. How does this patch fit in with the > previous series of patches you posted? Does it replace them or go on > top of them? > > Can please you resend me the full series of patch that remove the use > of do_mmap(), with all cleanups and bug fixes included? And please > roll up the fixes, I don't want one patch that adds a yield() inside a > spinlock and then a later patch to fix it -- there's no sense in > adding landmines for people potentially doing git bisect in the > future. > > And also please try to split the patches so that they don't mix > together two things -- please try to make the "remove obsolete > prototypes" patch separate from the mmap fixes. > > Thanks... > ___ > Linuxppc-dev mailing list > [EMAIL PROTECTED] > https://ozlabs.org/mailman/listinfo/linuxppc-dev ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] [PATCH/RFC 2.6.21] ehca: ehca_uverbs.c: refactor ehca_mmap() for better readability
> Roland, please note that I applied the previous patches to > your git tree for-2.6.21 before creating this patch. I also > realized a compile issue with the patch from Michael T. in > ehca_reqs.c regarding "return qp pointer in ib_wc". For this > I'll send another patch. Michael -- I didn't notice before but you do seem to have misread the ehca CQ polling code. The ehca CQ poll operation doesn't actually have the qp pointer readily available -- unless I'm mistaken, ehca will have to do an expensive search by QPN in a hash table to get the qp pointer. That doesn't seem acceptable for every CQ poll, so do you have any ideas of how to salvage this API change? - R. ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] [PATCH/RFC 2.6.21] ehca: ehca_uverbs.c: refactor ehca_mmap() for better readability
I've kind of lost the plot here. How does this patch fit in with the previous series of patches you posted? Does it replace them or go on top of them? Can please you resend me the full series of patch that remove the use of do_mmap(), with all cleanups and bug fixes included? And please roll up the fixes, I don't want one patch that adds a yield() inside a spinlock and then a later patch to fix it -- there's no sense in adding landmines for people potentially doing git bisect in the future. And also please try to split the patches so that they don't mix together two things -- please try to make the "remove obsolete prototypes" patch separate from the mmap fixes. Thanks... ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
[openib-general] [PATCH/RFC 2.6.21] ehca: ehca_uverbs.c: refactor ehca_mmap() for better readability
Hello,
here is a patch for ehca_uverbs.c with the following changes:
- Rename mm_open/close() to ehca_mm_open/close() respectively
- Refactor ehca_mmap() into sub-functions ehca_mmap_cq/qp(),
which then call the new common sub-functions ehca_mmap_fw()
and ehca_mmap_queue() to register firmware memory block and
queue pages respectively
Roland, please note that I applied the previous patches to
your git tree for-2.6.21 before creating this patch. I also
realized a compile issue with the patch from Michael T. in
ehca_reqs.c regarding "return qp pointer in ib_wc". For this
I'll send another patch.
Thanks!
Nam
Signed-off-by Hoang-Nam Nguyen <[EMAIL PROTECTED]>
---
ehca_uverbs.c | 266 +++---
1 file changed, 146 insertions(+), 120 deletions(-)
diff -Nurp infiniband/drivers/infiniband/hw/ehca/ehca_uverbs.c
infiniband_work/drivers/infiniband/hw/ehca/ehca_uverbs.c
--- infiniband/drivers/infiniband/hw/ehca/ehca_uverbs.c 2007-01-17
21:39:01.0 +0100
+++ infiniband_work/drivers/infiniband/hw/ehca/ehca_uverbs.c2007-01-17
21:17:00.0 +0100
@@ -68,7 +68,7 @@ int ehca_dealloc_ucontext(struct ib_ucon
return 0;
}
-static void mm_open(struct vm_area_struct *vma)
+static void ehca_mm_open(struct vm_area_struct *vma)
{
u32 *count = (u32*)vma->vm_private_data;
if (!count) {
@@ -84,7 +84,7 @@ static void mm_open(struct vm_area_struc
vma->vm_start, vma->vm_end, *count);
}
-static void mm_close(struct vm_area_struct *vma)
+static void ehca_mm_close(struct vm_area_struct *vma)
{
u32 *count = (u32*)vma->vm_private_data;
if (!count) {
@@ -98,26 +98,150 @@ static void mm_close(struct vm_area_stru
}
static struct vm_operations_struct vm_ops = {
- .open = mm_open,
- .close = mm_close,
+ .open = ehca_mm_open,
+ .close = ehca_mm_close,
};
-static int ehca_mmap_qpages(struct vm_area_struct *vma, struct ipz_queue
*queue)
+static int ehca_mmap_fw(struct vm_area_struct *vma, struct h_galpas *galpas,
+ u32 *mm_count)
{
+ int ret;
+ u64 vsize, physical;
+
+ vsize = vma->vm_end - vma->vm_start;
+ if (vsize != EHCA_PAGESIZE) {
+ ehca_gen_err("invalid vsize=%lx", vma->vm_end - vma->vm_start);
+ return -EINVAL;
+ }
+
+ physical = galpas->user.fw_handle;
+ vma->vm_page_prot = pgprot_noncached(vma->vm_page_prot);
+ ehca_gen_dbg("vsize=%lx physical=%lx", vsize, physical);
+ /* VM_IO | VM_RESERVED are set by remap_pfn_range() */
+ ret = remap_pfn_range(vma, vma->vm_start, physical >> PAGE_SHIFT,
+ vsize, vma->vm_page_prot);
+ if (unlikely(ret)) {
+ ehca_gen_err("remap_pfn_range() failed ret=%x", ret);
+ return -ENOMEM;
+ }
+
+ vma->vm_private_data = mm_count;
+ (*mm_count)++;
+ vma->vm_ops = &vm_ops;
+
+ return 0;
+}
+
+static int ehca_mmap_queue(struct vm_area_struct *vma, struct ipz_queue *queue,
+ u32 *mm_count)
+{
+ int ret;
u64 start, ofs;
struct page *page;
- int rc = 0;
+
+ vma->vm_flags |= VM_RESERVED;
start = vma->vm_start;
for (ofs = 0; ofs < queue->queue_length; ofs += PAGE_SIZE) {
u64 virt_addr = (u64)ipz_qeit_calc(queue, ofs);
page = virt_to_page(virt_addr);
- rc = vm_insert_page(vma, start, page);
- if (unlikely(rc)) {
- ehca_gen_err("vm_insert_page() failed rc=%x", rc);
- return rc;
+ ret = vm_insert_page(vma, start, page);
+ if (unlikely(ret)) {
+ ehca_gen_err("vm_insert_page() failed rc=%x", ret);
+ return ret;
}
start += PAGE_SIZE;
}
+ vma->vm_private_data = mm_count;
+ (*mm_count)++;
+ vma->vm_ops = &vm_ops;
+
+ return 0;
+}
+
+static int ehca_mmap_cq(struct vm_area_struct *vma, struct ehca_cq *cq,
+ u32 rsrc_type)
+{
+ int ret;
+
+ switch (rsrc_type) {
+ case 1: /* galpa fw handle */
+ ehca_dbg(cq->ib_cq.device, "cq_num=%x fw", cq->cq_number);
+ ret = ehca_mmap_fw(vma, &cq->galpas, &cq->mm_count_galpa);
+ if (unlikely(ret)) {
+ ehca_err(cq->ib_cq.device,
+"ehca_mmap_fw() failed rc=%x cq_num=%x",
+ret, cq->cq_number);
+ return ret;
+ }
+ break;
+
+ case 2: /* cq queue_addr */
+ ehca_dbg(cq->ib_cq.device, "cq_num=%x queue", cq->cq_number);
+ ret = ehca_mmap_queue(vma, &cq->ipz_queue, &cq->mm_count_queue);
+ if (unlikely(ret)) {
+ ehca_err(cq->ib_cq.device,
+
