We have the following HCAs in one of the machines:
InfiniBand: Mellanox Technologies MT25204 [InfiniHost III Lx HCA] (rev 20)
There are two adapters in the machine and from various experiments we
find that we are limited to being able to register
(ibv_reg_mr()) 28 GB. My calculations show one should be able to
register up to 56 GB. I would like to understand the difference.
In /sys/module/ib_mthca/parameters I see the following values:
fmr_reserved_mtts = 262144
log_mtts_per_seg = 3
num_mpt = 131072
num_mtt = 2097152
Therefore max memory that one can register should be (2097152 -262144) *
4K * 2^(log_mtts_per_seg)
i.e. 1.75 MB * 4K * 8 = 56 GB.
Since ibv_reg_mr() ties one to a pd and thus a particular device, is the
56 GB split between the 2 HCAs in the machine?
How does one explain the difference?
Thanks
Pradeep
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
