Re: [Rdma-developers] Re: [openib-general] OpenIB and OpenRDMA: Convergence on common RDMAAPIs and ULPs for Linux
On Fri, May 27, 2005 at 03:56:58PM -0700, Bob Woodruff wrote: Caitlin wrote, Both uDAPL and kDAPL were designed for *application* use. Even kDAPL is more intended for use by a kernel daemon that is loaded separately from the kernel than for use within the kernel itself. kDAPL is intended as a kernel-level API for RDMA enabled fabrics. As it was initially written, it does not meet the Linux coding style and that is why it is being totally reworked as we speak to meet that goal. The codingstyle alone isn't the problem. The whole design philosophy is rather odd. An ideal API for use within the kernel would abstract as much as possible (without requiring emulation), and then have transport specific unions or enum values. It would hide no control options, merely provide common controls for common capabilities. So for every new RDMA device type that comes along, you need to add a new enum, and unions for device class specific stuff, etc. Seems rather static and not easily extended. Not to mention that testing nightmare when the thing has to support 20 different types of RDMA enabled devices. I think code like that could get pretty ugly pretty fast. read the _as much as possible_ above. There's a reason you'll need new ABIs for the socket interface aswell when adding new address families. ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [Rdma-developers] Re: [openib-general] OpenIB and OpenRDMA: Convergence on common RDMAAPIs and ULPs for Linux
On Fri, May 27, 2005 at 08:04:26PM -0700, Caitlin Bestler wrote: A commitment that after merger that the API will be transport neutral is essential. That is, IB HCAs do not have to emulate iWARP. iWARP HCAs do not have to emulate IB. You won't get any commitments from us at all. Is there anything in the general structure of RNIC-PI that you the API shows the usual signs of design by committee is is totally focused on tacking up separate propritary modules instead of focussing integrated stacks. It makes the usual mistake of mixing up user and kernel APIs, and the APIs are much too complicated. ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [Rdma-developers] Re: [openib-general] OpenIB and OpenRDMA: Convergence on common RDMAAPIs and ULPs for Linux
--- Christoph Hellwig [EMAIL PROTECTED] wrote: On Fri, May 27, 2005 at 06:24:46PM -0700, Caitlin Bestler wrote: Oh that's a great development plan for me to propose to my business people. Let's go write lots of code, and *after* we write it the IB people (our competitors) have agreed to *look* at it (but they're all busy, don't forget) and *then* we just might get merged in. fortunately we don't care about your business people. either you do something sane, or your hardware won't we supported by linux at all. tough life. That's a pretty bold statement. Linux grew up to be popular via mass acceptance. Seems like that charter has changed and a few have control over Linux and its future. The My way or the highway philosophy has gotten embedded in the Linux way of life. Life is getting tough. --- This SF.Net email is sponsored by Yahoo. Introducing Yahoo! Search Developer Network - Create apps using Yahoo! Search APIs Find out how you can build Yahoo! directly into your own Applications - visit http://developer.yahoo.net/?fr=offad-ysdn-ostg-q22005 ___ Rdma-developers mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/rdma-developers __ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] OpenSM crash
On Fri, 2005-05-27 at 17:37, Hal Rosenstock wrote: On Fri, 2005-05-27 at 17:33, Roland Dreier wrote: May 27 01:44:09 [43005960] - osm_vl15_post: 4294967295 MADs on wire, 2 MADs outstanding. Hal I take that back. That's just a lot of MADs have been sent Hal (on the IB wire). OpenSM was probably up and running for a Hal while... I find it hard to believe that OpenSM has sent 4 billion MADs -- that's more than 1000 MADs a second for a solid month. It also looks very suspicious that the value is equal to ((unsigned int) -1). ^^ on a 32 bit machine. Good point. The fact that it gets to -1 is significant as I think that is used as a magic value for some computations. I'm pretty sure that I see a way this could have gone negative in the vendor layer. I'm working on a patch for this. -- Hal ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [Rdma-developers] Re: [openib-general] OpenIB and OpenRDMA: Convergence on common RDMAAPIs and ULPs for Linux
On Sat, May 28, 2005 at 05:17:54AM -0700, Sukanta ganguly wrote: That's a pretty bold statement. Linux grew up to be popular via mass acceptance. Seems like that charter has changed and a few have control over Linux and its future. The My way or the highway philosophy has gotten embedded in the Linux way of life. Life is getting tough. You're totally missing the point. Linux is successfull exactly because it's lookinf for the right solution, not something the business people need short-term. ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [Rdma-developers] Re: [openib-general] OpenIB and OpenRDMA: Convergence on common RDMAAPIs and ULPs for Linux
On 5/28/05, Christoph Hellwig [EMAIL PROTECTED] wrote: On Sat, May 28, 2005 at 05:17:54AM -0700, Sukanta ganguly wrote: That's a pretty bold statement. Linux grew up to be popular via mass acceptance. Seems like that charter has changed and a few have control over Linux and its future. The My way or the highway philosophy has gotten embedded in the Linux way of life. Life is getting tough. You're totally missing the point. Linux is successfull exactly because it's lookinf for the right solution, not something the business people need short-term. Such myopic cheerleading gets annoying and accomplishes nothing. The topic under discussion is whether a low level API for RDMA is necessary (as opposed to a higher level API such as kDAPL) and if so what the best strategy for achieving it is (try to plan an IB/iWARP merge immediately or wait until there is an iWARP code base). Claiming that an InfiniBand-specific interface is somehow thinking long term is just plain ludicrous. Now it may be that the short term interest of the InfiniBand vendors is such that they cannot commit resources to helping build a transport neutral API. That is always a legitimate tradeoff, but it is short term corporate thinking. Last time I looked most of the commits being made to OpenIB (or sourceforge DAPL) were from being drawing paychecks from those evil corporations. ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [Rdma-developers] Re: [openib-general] OpenIB and OpenRDMA: Convergence on common RDMAAPIs and ULPs for Linux
On 5/27/05, Bob Woodruff [EMAIL PROTECTED] wrote:
So for every new RDMA device type that comes along, you need to add a new
enum, and unions for device class specific stuff, etc.
Seems rather static and not easily extended. Not
to mention that testing nightmare when the thing has to support
20 different types of RDMA enabled devices.
I think code like that could get pretty ugly pretty fast.
It would not be every new device, it would be every new RDMA
verbs layer API. Gen2 already covers the needs of InfiniBand.
RNIC-PI covers the needs of InfiniBand, iWARP/TCP and iWARP/SCTP.
I would not anticipate a high rate of additional RDMA verbs
appearing, particularly if the two existing verb APIs are unified.
Creating new verb APIs is something vendors do to enable new
markets, not because they have excess underutilized staff that
needs to be kept busy.
Well, who knows what vendors with excess underutilized staff do,
there haven't been sightings of such creatures in memory.
But in any event, that can be dealt with in one of two ways:
struct rdma_xyz_ib {
};
struct rdma_xyz_iwarp {
};
union rdma_xyz_xport {
struct rdma_xyz_ib ib;
struct rdma_xyz_iwarp iwarp;
};
struct rdma_xyz {
/* common fields */
union rdma_xyz_xport xport;
};
Verus..
struct rdma_xyz {
/* common fields */
};
struct rdma_xyz_ib {
struct rdma_xyz common;
/* ib fields */
};
struct rdma_xyz_iwarp {
struct rdma_xyz common;
/* iwarp fields */
};
The latter style is extensible, but makes it difficult to properly
allocate a buffer
that works for all variants.
The union style is also already in use in both IT-API and RNIC-PI.
I personally prefer sub-classing to unions, but I have found myself in the
minority on *most* projects where the issue has been discussed.
One reason is that sub-classing provides very little type-safety.
struct sockaddr is an example of this. It takes manual inspection
to ensure that the variants are properly differentiated and it is
still common for developers to pass in a plain struct sockaddr
without realizing that it is not large enough for a struct sockaddr_in6.
I'd rather see a registration mechanism like what we already have
with DAPL that does not require any code changes to add a new RDMA
device/provider. We have already proven that this works in DAPL
as I know if at least 3 providers, IB, Myrinet, and RNIC (Ammasso)
that were developed separately and were able to co-exist without
any changes (enums and device class unions) in the DAT mid-layer.
I assume that this can also be done with kDAPL in the kernel, but
I defer to the DAPL experts to answer that one.
There would be a registration method. And each device would load
a dynamic method table. What is in question is how many signatures
are supported.
___
openib-general mailing list
openib-general@openib.org
http://openib.org/mailman/listinfo/openib-general
To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
