Hello, all. For the past few years, we (QLogic) have been using the IB user space SA module that was originally written by Sean Hefty. We think it would be great if this module were to become part of the official Linux Infiniband subsystem - but rather than simply dumping this on Roland as a surprise, I thought it might be a good idea to talk about it, first, and find out what the best approach would be.
What is ib_usa? --------------- The ib_usa module adds two features to the Infiniband Subsystem. It allows user applications to subscribe to SA notifications (remote port up, remote port down, etc.) and it allows user applications to subscribe to multicast groups. It does this via a device file, /dev/infiniband/ib_usa, similar to /dev/infiniband/uverbs and /dev/infiniband/umad. Why is this needed? ------------------- The SA protocol allows notices and multicast membership to be managed at a CA port level. As a result, if multiple processes want to receive notices or join multicast groups, that registration must be coordinated per CA port. Existing code in the Linux kernel exists for this purpose, coordinating the needs of various Infiniband kernel modules and presenting a single "per server CA port" perspective to the SA. The IB user space SA module provides a mechanism for user space applications to take advantage of this existing kernel code for managing SA registrations. By providing this mechanism, ib_usa provides a convenient and useful way for user applications to detect when nodes enter or leave the fabric. This is useful for fabric monitoring applications (like a system admin dashboard) or, conceivably, to allow a cluster application to detect problems and to dynamically balance a computational load across the fabric. In addition, the multicast functionality will allow user applications to use properly verbs to receive multicast traffic. How will it integrate into the existing Infiniband subsystem? ------------------------------------------------------------- We recommend including Sean's original module, and we can provide a version of it that works fine with the latest versions of OFED. This will add a new kernel module to the ofa_kernel package, ib_usa.ko. In addition, there will be a need for a shared library to provide an API to provide access to ib_usa. I would suggest adding the needed functions to OFED management module, as part of the libibumad library, or to their own libibusa library. In either case, I would provide the needed code and a sample/test application. So, how about it? What's the best way forward here? -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
