Re: [ewg] [ANNOUNCE] RHEL5.3 support added to OFED-1.4 (latest daily build)
Woodruff, Robert J wrote: I tried this on my systems, It seems to install and work fine on X86_64 platforms, but I get this compiler error on Itanium. woody ckport/2.6.18-EL5.3/include/linux/inetdevice.h:4, from /var/tmp/OFED_topdir/BUILD/ofa_kernel-1.4/drivers/infiniband/core/addr.c:37: /var/tmp/OFED_topdir/BUILD/ofa_kernel-1.4/kernel_addons/backport/2.6.18-EL5.3/include/asm/checksum.h:9: error: expected '=', ',', ';', 'asm' or '__attribute__' before 'backport_csum_tcpudp_nofold' make[4]: *** [/var/tmp/OFED_topdir/BUILD/ofa_kernel-1.4/drivers/infiniband/core/addr.o] Error 1 make[3]: *** [/var/tmp/OFED_topdir/BUILD/ofa_kernel-1.4/drivers/infiniband/core] Error 2 We don't have Itanium system with this OS here Can someone from your team solve this and send us the relevant backport pathces Tziporet ___ ewg mailing list ewg@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
Re: [ofa-general] Re: [ewg] RE: RHEL 5.3 and OFED 1.4.x
Steve Wise wrote: One way to alleviate this is to ship both 1.2.8 and 1.3 in ofed-1.4.1 and mark 1.3 as experimental. Then remove 1.2.8 in ofed-1.5 and make 1.3.x the production version for ofed-1.5. I suggested this in the last conf call but folks didn't like the thought of testing both. But perhaps marking it experimental resolves this issue? So the iWARP vendors will test 1.3 and little to no testing is required for 1.2.8 since it has been qualified with ofed-1.4 QA. We do not wish to have two MPI revisions in the same OFED package We talked about it several times and declined this option. Tziporet ___ ewg mailing list ewg@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
[ewg] [PATCH v3] mlx4_ib: Optimize hugetlab pages support
Since Linux may not merge adjacent pages into a single scatter entry through
calls to dma_map_sg(), we check the special case of hugetlb pages which are
likely to be mapped to coniguous dma addresses and if they are, take advantage
of this. This will result in a significantly lower number of MTT segments used
for registering hugetlb memory regions.
Signed-off-by: Eli Cohen e...@mellanox.co.il
---
drivers/infiniband/hw/mlx4/mr.c | 81 ++
1 files changed, 72 insertions(+), 9 deletions(-)
diff --git a/drivers/infiniband/hw/mlx4/mr.c b/drivers/infiniband/hw/mlx4/mr.c
index 8e4d26d..6e898a9 100644
--- a/drivers/infiniband/hw/mlx4/mr.c
+++ b/drivers/infiniband/hw/mlx4/mr.c
@@ -119,6 +119,66 @@ out:
return err;
}
+static int handle_hugetlb_user_mr(struct ib_pd *pd, struct mlx4_ib_mr *mr,
+ u64 start, u64 virt_addr, int access_flags)
+{
+#if defined(CONFIG_HUGETLB_PAGE) !defined(__powerpc__) !defined(__ia64__)
+ struct mlx4_ib_dev *dev = to_mdev(pd-device);
+ struct ib_umem_chunk *chunk;
+ unsigned dsize;
+ dma_addr_t daddr;
+ unsigned cur_size = 0;
+ dma_addr_t uninitialized_var(cur_addr);
+ int n;
+ struct ib_umem *umem = mr-umem;
+ u64 *arr;
+ int err = 0;
+ int i;
+ int j = 0;
+ int off = start (HPAGE_SIZE - 1);
+
+ n = DIV_ROUND_UP(off + umem-length, HPAGE_SIZE);
+ arr = kmalloc(n * sizeof *arr, GFP_KERNEL);
+ if (!arr)
+ return -ENOMEM;
+
+ list_for_each_entry(chunk, umem-chunk_list, list)
+ for (i = 0; i chunk-nmap; ++i) {
+ daddr = sg_dma_address(chunk-page_list[i]);
+ dsize = sg_dma_len(chunk-page_list[i]);
+ if (!cur_size) {
+ cur_addr = daddr;
+ cur_size = dsize;
+ } else if (cur_addr + cur_size != daddr) {
+ err = -EINVAL;
+ goto out;
+ } else
+ cur_size += dsize;
+
+ if (cur_size HPAGE_SIZE) {
+ err = -EINVAL;
+ goto out;
+ } else if (cur_size == HPAGE_SIZE) {
+ cur_size = 0;
+ arr[j++] = cur_addr;
+ }
+ }
+
+ err = mlx4_mr_alloc(dev-dev, to_mpd(pd)-pdn, virt_addr, umem-length,
+ convert_access(access_flags), n, HPAGE_SHIFT,
mr-mmr);
+ if (err)
+ goto out;
+
+ err = mlx4_write_mtt(dev-dev, mr-mmr.mtt, 0, n, arr);
+
+out:
+ kfree(arr);
+ return err;
+#else
+ return -ENOSYS;
+#endif
+}
+
struct ib_mr *mlx4_ib_reg_user_mr(struct ib_pd *pd, u64 start, u64 length,
u64 virt_addr, int access_flags,
struct ib_udata *udata)
@@ -140,17 +200,20 @@ struct ib_mr *mlx4_ib_reg_user_mr(struct ib_pd *pd, u64
start, u64 length,
goto err_free;
}
- n = ib_umem_page_count(mr-umem);
- shift = ilog2(mr-umem-page_size);
+ if (!mr-umem-hugetlb ||
+ handle_hugetlb_user_mr(pd, mr, start, virt_addr, access_flags)) {
+ n = ib_umem_page_count(mr-umem);
+ shift = ilog2(mr-umem-page_size);
- err = mlx4_mr_alloc(dev-dev, to_mpd(pd)-pdn, virt_addr, length,
- convert_access(access_flags), n, shift, mr-mmr);
- if (err)
- goto err_umem;
+ err = mlx4_mr_alloc(dev-dev, to_mpd(pd)-pdn, virt_addr,
length,
+ convert_access(access_flags), n, shift,
mr-mmr);
+ if (err)
+ goto err_umem;
- err = mlx4_ib_umem_write_mtt(dev, mr-mmr.mtt, mr-umem);
- if (err)
- goto err_mr;
+ err = mlx4_ib_umem_write_mtt(dev, mr-mmr.mtt, mr-umem);
+ if (err)
+ goto err_mr;
+ }
err = mlx4_mr_enable(dev-dev, mr-mmr);
if (err)
--
1.6.1
___
ewg mailing list
ewg@lists.openfabrics.org
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
Re: [ofa-general] Re: [ewg] [PATCH] ib_core: save process's virtual address in struct ib_umem
I see... you're right - no need to stick the address into struct ib_umem. Following this email is a new patch for mlx4_ib only. I excluded support for both powerpc and ia64 since I could not find a way to get HPAGE_SIZE (or HPAGE_SHIFT) for them. #include asm/page.h ? - R. ___ ewg mailing list ewg@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
Re: [ofa-general] Re: [ewg] [PATCH] ib_core: save process's virtual address in struct ib_umem
On Thu, Jan 29, 2009 at 07:33:22AM -0800, Roland Dreier wrote: I see... you're right - no need to stick the address into struct ib_umem. Following this email is a new patch for mlx4_ib only. I excluded support for both powerpc and ia64 since I could not find a way to get HPAGE_SIZE (or HPAGE_SHIFT) for them. #include asm/page.h ? It does not help. The problem with powerpc is that HPAGE_SHIFT is an unexported variable and for ia64 it's hpage_shift. ___ ewg mailing list ewg@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
Re: [ofa-general] Re: [ewg] [PATCH] ib_core: save process's virtual address in struct ib_umem
It does not help. The problem with powerpc is that HPAGE_SHIFT is an unexported variable and for ia64 it's hpage_shift. I see. hpage_shift is exported on ia64, so that should be OK. And I guess for powerpc it is just a matter of adding an export so we can use it in a module. - R. ___ ewg mailing list ewg@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
