Hi Mike,
On 7/12/06, Michael Krause <[EMAIL PROTECTED]> wrote:
At 09:48 AM 7/12/2006, Jeff Broughton wrote:
Modifying the sockets API is just defining yet another RDMA API, and we have
so many already....
I disagree. This effort has distilled the API to basically one for RDMA
developers. Applications are supported over this via either MPI or Sockets.
There's been a lot of effort to make the RDMA verbs easy to use. With
the RDMA CM, socket-like connection semantics can be used to establish
the connection between QPs. The connection establishment is the hard
part - doing I/O is trivial in comparisson. This verbs and RDMA CM
have nothing to do with MPI.
If an application is going to be RDMA aware, I don't see any reason it
shouldn't just use the verbs directly and use the RDMA CM to establish
the connections.
What's your point? It seems you are in agreement that there is a single RDMA API that people can use.
It seems rather self limiting to think the traditional BSD synchronous
Sockets API is all the world should be able to use when it comes to Sockets.
Sockets developers could easily incorporate the extensions into their
applications providing them with improved designs and flexibility without
having to learn about RDMA itself.
Wait, you want applications to be able to register memory and issue
RDMA operations, but not have to learn about RDMA? How does that make
sense?
The Sockets API extensions allow developers to register memory. That has been a desire by many when it comes to SDP or copy avoidance technology as it optimizes the performance path by eliminating the need to do per op registration. For many applications which already known working sets, they can use this to enable the OS and underlying infrastructure to take advantage of this fact to improve performance and quality of the solution. The extensions provide the async communications and event collection mechanisms to also improve performance over the rather limiting select / poll supported by Sockets today.
It currently does not support explicit RDMA but it is rather trivial to add such calls and remove the need to interject SDP if desired. The benefits of such new API extensions are there for those that want to eliminate one more ULP with its unfortunate IP cloud over head.
If the couple of calls necessary to
extend this API to support direct RDMA would allow them to eliminate SDP
entirely, well, that has benefits that go beyond just its all Sockets;
For a socket implementation to support RDMA, the socket must have an
underlying RDMA QP. This means that if you want the application to
not have to be verbs-aware, you can't really get rid of SDP - you're
just extending SDP to let the application have a part in memory
registration and RDMA, while still supporting the traditional BSD
operations. This is IMO more complex than just letting applications
interface directly with verbs, especially since the SDP implementation
will size the QP for its own use, without a means for negotiating with
the user so that you don't cause buffer overruns.
Please take a look at the API extensions. I never stated that one gets rid of SDP unless one adds the RDMA-explicit calls.
As for complexity, well, the goal is to extend to Sockets developers the optimal communication paradigm already available on OS such as Windows without having to leave with the same unfortunate constraints imposed by the OS. The same logic applies to extending the benefits derived from MPI which supports async communications as well as put / get semantics which would be analogous to the additional RDMA interfaces I referenced.
I find it strange that people would argue against improving the Sockets developer's tool suite when the benefits are already proven elsewhere within the industry and even within this open source effort. Giving the millions of Sockets developers the choice of a set of extensions that work over both RDMA and traditional network stacks seems like a no brainer. Trying to force them to use a native RDMA API even if semantically similar to Sockets seems like a poor path to pursue. Leave the RDMA API to the middleware providers and those that need to be close the metal.
it also eliminates the IP cloud that hovers over SDP licensing. Something
that many developers and customers would appreciate.
I believe that Microsoft's IP claims only apply to SDP over IB -- I
don't believe SDP over iWarp is affected. I don't know how the RDMA
verbs moving towards a hardware independent (wrt IB vs. iWarp) affects
the IP claims, but it should certainly make things interesting if a
single SDP code base can work over both IB and iWarp.
SDP is SDP and it isn't just restricted to IB. I'll leave it to the lawyers to sort it out but having a single SDP with minor code execution path deltas for the IB-specifics isn't that hard to construct. It has been done on other OS already.
Mike
_______________________________________________ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general
To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
