On 7/29/10 3:13 PM, Jason Gunthorpe wrote:
On Thu, Jul 29, 2010 at 02:07:11PM -0500, Tom Tucker wrote:
Add an ib_iomem_get service that converts a vma to an array of
physical addresses. This makes it easier for each device driver to
add support for the reg_io_mr provider method.
Tom, could you say a few words on what this is for?
The proposed interface is there to allow a user-mode application to mmap
kernel memory and/or device memory as a data source or sync for RDMA.
This is useful for a number of applications, for example:
- An application that wishes to provide kernel statistics to a
monitoring host with zero overhead. For example, a set of compute nodes
that are being monitored by a statistics gathering application (Ovis),
but don't want to introduce scheduler jitter into the compute node. The
stats are provided in a kernel allocated buffer and the monitoring app
'gathers' those statistics with RDMA_READ at zero cost to the monitored
host.
- An application that wishes to provide network (RDMA) access to a large
static memory hung off of it's PCI-e bus, thereby creating a distributed
and persistent memory
Those are the only two that I've written so far that use these verbs.
Also, I'd like to see a strong defence of this new user space API
particularly:
1) Why can't this be done with the existing ibv_reg_mr, like huge
pages are.
The ibv_reg_mr API assumes that the memory being registered was
allocated in user mode and is part of the current->mm VMA. It uses
get_user_pages which will scoff and jeer at kernel memory.
2) How is it possible for userspace to know when it should use
ibv_reg_mr vs ibv_reg_io_mr?
By virtue of the device that it is mmap'ing. If I mmap my_vmstat_driver,
I know that the memory I am mapping is a kernel buffer.
3) Why can't ibv_dereg_io_mr be entirely handled by ibv_dereg_mr?
Probably could.
On first glance, this seems like a hugely bad API to me :)
Well hopefully now that it's purpose is revealed you will change your
view and we can collaboratively make it better :-)
Also, if new user space APIs are really needed, they do need man pages
too ...
Of course.
Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
