> And anyway, this is all moot I believe because the code already does:
>
> for (i = 0; i < MTHCA_RES_NUM; ++i) {
> profile[i].type = i;
> profile[i].log_num = max(ffs(profile[i].num) - 1, 0);
> profile[i].size *= profile[i].num;
> if (mthca_is_memfree(dev))
> profile[i].size = max(profile[i].size, (u64) PAGE_SIZE);
> }
>
> and I'm comfortable assuming that there is no architecture where
> dma_get_cache_alignment() is bigger than PAGE_SIZE...
OK. Missed that. Good, so there's no issue.
we can drop this chunk then. So we are left with this:
[PATCHv3 5 of 5] IB/mthca: reserved MTTs issues
This fixes several issues related to reserved MTTs
1. MTTs are allocated in non-cache-coherent memory, so we must give
reserved MTTs their own cache line, to prevent both device and
CPU from writing into the same cache line at the same time.
2. reserved_mtts field has different meaning in Tavor and Arbel,
so we are wasting mtt entries on memfree. fix the Arbel case to match
Tavor semantics.
Signed-off-by: Michael S. Tsirkin <[EMAIL PROTECTED]>
---
Index: linux-2.6/drivers/infiniband/hw/mthca/mthca_main.c
===================================================================
--- linux-2.6.orig/drivers/infiniband/hw/mthca/mthca_main.c
+++ linux-2.6/drivers/infiniband/hw/mthca/mthca_main.c
@@ -464,6 +464,10 @@ static int mthca_init_icm(struct mthca_d
goto err_unmap_aux;
}
+ /* CPU writes to non-reserved MTTs, while HCA might DMA to reserved
mtts */
+ mdev->limits.reserved_mtts = ALIGN(mdev->limits.reserved_mtts *
MTHCA_MTT_SEG_SIZE,
+ dma_get_cache_alignment()) /
MTHCA_MTT_SEG_SIZE;
+
mdev->mr_table.mtt_table = mthca_alloc_icm_table(mdev,
init_hca->mtt_base,
MTHCA_MTT_SEG_SIZE,
mdev->limits.num_mtt_segs,
Index: linux-2.6/drivers/infiniband/hw/mthca/mthca_cmd.c
===================================================================
--- linux-2.6.orig/drivers/infiniband/hw/mthca/mthca_cmd.c
+++ linux-2.6/drivers/infiniband/hw/mthca/mthca_cmd.c
@@ -1051,7 +1051,11 @@ int mthca_QUERY_DEV_LIM(struct mthca_dev
MTHCA_GET(field, outbox, QUERY_DEV_LIM_MAX_EQ_OFFSET);
dev_lim->max_eqs = 1 << (field & 0x7);
MTHCA_GET(field, outbox, QUERY_DEV_LIM_RSVD_MTT_OFFSET);
- dev_lim->reserved_mtts = 1 << (field >> 4);
+ if (mthca_is_memfree(dev))
+ dev_lim->reserved_mtts = ALIGN(1 << (field >> 4),
+ MTHCA_MTT_SEG_SIZE /
sizeof(u64));
+ else
+ dev_lim->reserved_mtts = 1 << (field >> 4);
MTHCA_GET(field, outbox, QUERY_DEV_LIM_MAX_MRW_SZ_OFFSET);
dev_lim->max_mrw_sz = 1 << field;
MTHCA_GET(field, outbox, QUERY_DEV_LIM_RSVD_MRW_OFFSET);
--
MST
_______________________________________________
openib-general mailing list
openib-general@openib.org
http://openib.org/mailman/listinfo/openib-general
To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
