> -----Original Message----- > From: Grant Grundler [mailto:[EMAIL PROTECTED] > Sent: Friday, May 27, 2005 7:15 PM > To: Caitlin Bestler > Cc: Bob Woodruff; Roland Dreier; James Lentini; > [EMAIL PROTECTED]; Venkata Jagana; > [EMAIL PROTECTED]; openib-general@openib.org > Subject: Re: [Rdma-developers] Re: [openib-general] OpenIB > and OpenRDMA: Convergence on common RDMAAPIs and ULPs for Linux > > On Fri, May 27, 2005 at 06:24:46PM -0700, Caitlin Bestler wrote: > > Oh that's a great development plan for me to propose to my business > > people. Let's go write lots of code, and *after* we write it the IB > > people (our competitors) have agreed to *look* at it (but > they're all > > busy, don't > > forget) and *then* we just might get merged in. > > You will never get a guarantee your code will go into > kernel.org regardless of which path you take. > I've seen three of the lead developers (a) promise to review > any proposed code changes and (b) indicate they are willing > to merge in support for RNICs. >
A commitment that after merger that the API will be transport neutral is essential. That is, IB HCAs do not have to emulate iWARP. iWARP HCAs do not have to emulate IB. I fail to see why an advance agreement about the goal cannot be reached. The iWARP vendors are willing to concede that current Gen2 verb modules still have to be supported in a merged solution. What is your definiton of a merged solution? Is there a commitment that "RDMA Services" does not mean "InfiniBand, and the iWARP vendors have to pretend to be InfiniBand but we'll add one or two fields for problems they *can't* solve." > > Look at the relevant openib.org header files (e.g. ib_verbs.h, > ib_user_verbs.h) and make a list of changes needed for it to > be useful to an RNIC. Just pointing at RNIC-PI isn't interesting. > Well since we are talking about what a final transport neutral API would look like, as a guide when writing new code that will be compatible with the final merger, then looking at an example of an API that attempts to be transport neutral strikes me as highly relevant. Is there anything in the general structure of RNIC-PI that you do not like? The IB-specific portions are different than Gen2, and that certainly would not be true of any final merged verbs. So far the only comments received are that RNIC-PI was not written to be inside the Linux Kernel. That's correct, it was designed to be an OS neutral API that could be supported on a wide variety of Operating systems. Changes *have* been made, such as always having struct/enum/union tags, precisely to guarantee that a Linux version of rnicpi.h can be written that will be interoperable with code written to the man pages. As for gen2, here's one minor example: struct ib_qp_attr { enum ib_qp_state qp_state; // States defined are IB specific enum ib_qp_state cur_qp_state; enum ib_mtu path_mtu; // The path MTU is not a QP attribute in iWARP, // The LLP maintains that. The maximum DDP Segment // size is exported to the DDP layer, but not to // the consumer. enum ib_mig_state path_mig_state; // Path migration is handled at L2 or L3, it is // not visisble to an iWARP QP. u32 qkey; // There is no qkey u32 rq_psn; // There is no psn. There is an MSN for send/recvs. // It is not normally exported to the consumer. u32 sq_psn; // There is no psn. u32 dest_qp_num; // The destination QP is not known. int qp_access_flags; // presumably mostly compatible, but its not documented // in the .h file struct ib_qp_cap cap; // different capacities are controllable between // iWARP and IB. RNIC-PI defined the union of the // two, and left it to the providers to simply replicate // counts that are redundant under their transport. // Example: RDMAC allows different sge capacities // depending on the type of work request (send vs. write). struct ib_ah_attr ah_attr; // entirely IB specific information. TCP equivalents are // not part of QP state. struct ib_ah_attr alt_ah_attr; // see ah_attr u16 pkey_index; // VLANs are the equivalent, and are managed below the // RDMA layer u16 alt_pkey_index; // ditto u8 en_sqd_async_notify; // There is no sqd state u8 sq_draining; u8 max_rd_atomic; // there are no atomics u8 max_dest_rd_atomic; u8 min_rnr_timer; // Retries and pauses are handled below the RDMA layer u8 port_num; // Egress ports are managed at the IP layer, not as // part of the RDMA layer. Egress ports can be // reassigned without changing the QP's state // at the RDMA layer u8 timeout; // managed by the LLP (TCP or SCTP) u8 retry_cnt; // managed by the LLP (TCP or SCTP) u8 rnr_retry; // managed by the LLP (TCP or SCTP) u8 alt_port_num; // see port num u8 alt_timeout; // see timeout }; In a merged transport neutral API virtually the entire current struct would become part of an ib union. Just adding comments that this field is "IB only" doesn't achieve neutrality, it makes iWARP developers wade through structs to determine which fields can be effected, and makes it all too likely that an application developer will set a field not realizing that an iWARP RNIC will not look at that field. If a transport neutral API based on unioning everything that is not transport neutral is not acceptable as the goal of eventual merger is not acceptable then the emphasis should be on kDAPL as the transport neutral API. _______________________________________________ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general