On Wed, 13 Jul 2005, Michael Krause wrote:
At 06:39 AM 7/13/2005, James Lentini wrote:
kDAPL was designed specifically for RDMA networks with lots of features that allow you to control how the network is used. This is good if you are writing new code, but means that old code needs substantial porting.
Ideally, applications stay out of such decisions. Middleware's job is to handle application optimization, etc. so that the end consumer stays as ignorant as possible thus focused on their application's needs not the networks. The middleware API - whether DAPL, IT API, RNIC PI, whatever - can provide the hooks needed to manage the usage from a given endnode's perspective. But even here, the real network management, what routes are actually used, the arbitration for QoS, etc. should also be outside of the middleware's control. It simply manages a set of local resources and allows the fabric management to do the rest. There is more to this than that but that is how IB was constructed which is no different in many respects from how IP works as well.
Let me clarify: kDAPL users can specify exactly how data is transfered (SEND, RDMA write, RDMA read), completion events are processed, memory is registered, etc. This is the "network control" I was referring to. In retrospect, it would be more correct refer to this as "adapter control".
Just to nit-pick this a bit. The ULP determines what type of operation to use - SEND, RDMA Write, RDMA Read, or Atomic (where supported by the interconnect). The middleware API or the verbs API provide an interface to abstract the IHV hardware-specifics from the ULP allowing the ULP to be implemented across a variety of technologies. Thus, the API is just an abstraction of the underlying semantics and does not make decisions or provide much in the way of controls in any regard unless additional value-add beyond the underlying hardware semantics are transparently implemented within the API implementation itself For example, SDP defines specifically when to use a SEND for control operations, when to use a RDMA for a zcopy operation. SDP itself does not care what the underlying API is used to access the associated hardware resources but it does define what resources and associated services are used. SDP can be implemented directly on the verbs API (just like MPI) and operate quite nicely without the additional middleware API in the execution path.
Apologies of the nit pick but the API does not provide any type of control other than to act as an abstraction funnel between the ULP / application and the underlying hardware. There are opportunities to provide transparent value-add controls but I don't believe this open source effort is focused on these this time.
Mike
_______________________________________________ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general
To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
