[openib-general] cmpost: failure sending REQ: -22
Has anyone seen ib_send_cm_req() return -22? I seem to be stuck with this error today. I've seen it in the past; and it seems to have gone away without my being able to discern a conclusive reason. Usually rebooting the systems will cause it to work again. This go around, It's really refusing this time; I reverted to kernel 2.6.11.7 and infiniband sources from around the end of april (even binaries I built then that were working don't work now.) I rebooted the SM also, but now I'm going to actually powerdown the systems. Although it seems like it shouldn't be possible, this behavior really makes it look like the HCA can get in a broken state that persists after a reboot. Enjoy, Jeff ib1: no IPv6 routers present cmpost: starting client cmpost: connecting cmpost: failure sending REQ: -22 cmpost: test complete ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [Rdma-developers] Re: [openib-general] OpenIB and OpenRDMA: Convergence on common RDMAAPIs and ULPs for Linux
On 5/27/05, Bob Woodruff <[EMAIL PROTECTED]> wrote: > > So for every new RDMA device type that comes along, you need to add a new > enum, and unions for device class specific stuff, etc. > Seems rather static and not easily extended. Not > to mention that testing nightmare when the thing has to support > 20 different types of RDMA enabled devices. > I think code like that could get pretty ugly pretty fast. > It would not be every new device, it would be every new RDMA verbs layer API. Gen2 already covers the needs of InfiniBand. RNIC-PI covers the needs of InfiniBand, iWARP/TCP and iWARP/SCTP. I would not anticipate a high rate of additional RDMA verbs appearing, particularly if the two existing verb APIs are unified. Creating new verb APIs is something vendors do to enable new markets, not because they have excess underutilized staff that needs to be kept busy. Well, who knows what vendors with excess underutilized staff do, there haven't been sightings of such creatures in memory. But in any event, that can be dealt with in one of two ways: struct rdma_xyz_ib { }; struct rdma_xyz_iwarp { }; union rdma_xyz_xport { struct rdma_xyz_ib ib; struct rdma_xyz_iwarp iwarp; }; struct rdma_xyz { /* common fields */ union rdma_xyz_xport xport; }; Verus.. struct rdma_xyz { /* common fields */ }; struct rdma_xyz_ib { struct rdma_xyz common; /* ib fields */ }; struct rdma_xyz_iwarp { struct rdma_xyz common; /* iwarp fields */ }; The latter style is extensible, but makes it difficult to properly allocate a buffer that works for all variants. The union style is also already in use in both IT-API and RNIC-PI. I personally prefer sub-classing to unions, but I have found myself in the minority on *most* projects where the issue has been discussed. One reason is that sub-classing provides very little type-safety. struct sockaddr is an example of this. It takes manual inspection to ensure that the variants are properly differentiated and it is still common for developers to pass in a plain struct sockaddr without realizing that it is not large enough for a struct sockaddr_in6. > I'd rather see a registration mechanism like what we already have > with DAPL that does not require any code changes to add a new RDMA > device/provider. We have already proven that this works in DAPL > as I know if at least 3 providers, IB, Myrinet, and RNIC (Ammasso) > that were developed separately and were able to co-exist without > any changes (enums and device class unions) in the DAT mid-layer. > I assume that this can also be done with kDAPL in the kernel, but > I defer to the DAPL experts to answer that one. > There would be a registration method. And each device would load a dynamic method table. What is in question is how many signatures are supported. ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [Rdma-developers] Re: [openib-general] OpenIB and OpenRDMA: Convergence on common RDMAAPIs and ULPs for Linux
On 5/28/05, Christoph Hellwig <[EMAIL PROTECTED]> wrote: > On Sat, May 28, 2005 at 05:17:54AM -0700, Sukanta ganguly wrote: > > That's a pretty bold statement. Linux grew up to be > > popular via mass acceptance. Seems like that charter > > has changed and a few have control over Linux and its > > future. The "My way or the highway" philosophy has > > gotten embedded in the Linux way of life. > > Life is getting tough. > > You're totally missing the point. Linux is successfull exactly > because it's lookinf for the right solution, not something the > business people need short-term. Such myopic cheerleading gets annoying and accomplishes nothing. The topic under discussion is whether a low level API for RDMA is necessary (as opposed to a higher level API such as kDAPL) and if so what the best strategy for achieving it is (try to plan an IB/iWARP merge immediately or wait until there is an iWARP code base). Claiming that an InfiniBand-specific interface is somehow thinking "long term" is just plain ludicrous. Now it may be that the short term interest of the InfiniBand vendors is such that they cannot commit resources to helping build a transport neutral API. That is always a legitimate tradeoff, but it is "short term corporate thinking". Last time I looked most of the commits being made to OpenIB (or sourceforge DAPL) were from being drawing paychecks from those "evil corporations". ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [Rdma-developers] Re: [openib-general] OpenIB and OpenRDMA: Convergence on common RDMAAPIs and ULPs for Linux
On Sat, May 28, 2005 at 05:17:54AM -0700, Sukanta ganguly wrote: > That's a pretty bold statement. Linux grew up to be > popular via mass acceptance. Seems like that charter > has changed and a few have control over Linux and its > future. The "My way or the highway" philosophy has > gotten embedded in the Linux way of life. > Life is getting tough. You're totally missing the point. Linux is successfull exactly because it's lookinf for the right solution, not something the business people need short-term. ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [Rdma-developers] Re: [openib-general] OpenIB and OpenRDMA: Convergence on common RDMAAPIs and ULPs for Linux
--- Christoph Hellwig <[EMAIL PROTECTED]> wrote: > On Fri, May 27, 2005 at 06:24:46PM -0700, Caitlin > Bestler wrote: > > Oh that's a great development plan for me to > propose > > to my business people. Let's go write lots of > code, > > and *after* we write it the IB people (our > competitors) > > have agreed to *look* at it (but they're all busy, > don't > > forget) and *then* we just might get merged in. > > fortunately we don't care about your business > people. either > you do something sane, or your hardware won't we > supported by > linux at all. tough life. > > > > That's a pretty bold statement. Linux grew up to be popular via mass acceptance. Seems like that charter has changed and a few have control over Linux and its future. The "My way or the highway" philosophy has gotten embedded in the Linux way of life. Life is getting tough. --- > This SF.Net email is sponsored by Yahoo. > Introducing Yahoo! Search Developer Network - Create > apps using Yahoo! > Search APIs Find out how you can build Yahoo! > directly into your own > Applications - visit > http://developer.yahoo.net/?fr=offad-ysdn-ostg-q22005 > ___ > Rdma-developers mailing list > [EMAIL PROTECTED] > https://lists.sourceforge.net/lists/listinfo/rdma-developers > __ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] OpenSM crash
On Fri, 2005-05-27 at 17:37, Hal Rosenstock wrote: > On Fri, 2005-05-27 at 17:33, Roland Dreier wrote: > > > May 27 01:44:09 [43005960] -> osm_vl15_post: 4294967295 MADs on wire, > > 2 MADs outstanding. > > > > Hal> I take that back. That's just a lot of MADs have been sent > > Hal> (on the IB wire). OpenSM was probably up and running for a > > Hal> while... > > > > I find it hard to believe that OpenSM has sent 4 billion MADs -- > > that's more than 1000 MADs a second for a solid month. It also looks > > very suspicious that the value is equal to ((unsigned int) -1). > ^^ > on a 32 bit machine. > > Good point. The fact that it gets to -1 is significant as I think that > is used as a magic value for some computations. I'm pretty sure that I see a way this could have gone negative in the vendor layer. I'm working on a patch for this. -- Hal ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [Rdma-developers] Re: [openib-general] OpenIB and OpenRDMA: Convergence on common RDMAAPIs and ULPs for Linux
On Fri, May 27, 2005 at 08:04:26PM -0700, Caitlin Bestler wrote: > A commitment that after merger that the API will be transport > neutral is essential. That is, IB HCAs do not have to emulate > iWARP. iWARP HCAs do not have to emulate IB. You won't get any commitments from us at all. > Is there anything in the general structure of RNIC-PI that you the API shows the usual signs of design by committee is is totally focused on tacking up separate propritary modules instead of focussing integrated stacks. It makes the usual mistake of mixing up user and kernel APIs, and the APIs are much too complicated. ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [Rdma-developers] Re: [openib-general] OpenIB and OpenRDMA: Convergence on common RDMAAPIs and ULPs for Linux
On Fri, May 27, 2005 at 06:24:46PM -0700, Caitlin Bestler wrote: > Oh that's a great development plan for me to propose > to my business people. Let's go write lots of code, > and *after* we write it the IB people (our competitors) > have agreed to *look* at it (but they're all busy, don't > forget) and *then* we just might get merged in. fortunately we don't care about your business people. either you do something sane, or your hardware won't we supported by linux at all. tough life. ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [Rdma-developers] Re: [openib-general] OpenIB and OpenRDMA: Convergence on common RDMAAPIs and ULPs for Linux
On Fri, May 27, 2005 at 03:56:58PM -0700, Bob Woodruff wrote: > Caitlin wrote, > >Both uDAPL and kDAPL were designed for *application* use. > >Even kDAPL is more intended for use by a kernel daemon that > >is loaded separately from the kernel than for use within > >the kernel itself. > > kDAPL is intended as a kernel-level API > for RDMA enabled fabrics. As it was initially written, > it does not meet the Linux coding style and that is why > it is being totally reworked as we speak to meet that goal. The codingstyle alone isn't the problem. The whole design philosophy is rather odd. > >An ideal API for use within the kernel would abstract as > >much as possible (without requiring emulation), and then > >have transport specific unions or enum values. It would > >hide no control options, merely provide common controls > >for common capabilities. > > So for every new RDMA device type that comes along, you need to add a new > enum, and unions for device class specific stuff, etc. > Seems rather static and not easily extended. Not > to mention that testing nightmare when the thing has to support > 20 different types of RDMA enabled devices. > I think code like that could get pretty ugly pretty fast. read the _as much as possible_ above. There's a reason you'll need new ABIs for the socket interface aswell when adding new address families. ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general