Kais,
Kais Belgaied wrote:
> - Case boundary question: since this marks a flag day for both TI and
> CI, can you list the components
> that are affected by this flag day?
Most of the IB modules in ON -
framework: IBTL
IB ULPs (TI): IPonIB, SDP, NFS/RDMA, uDAPL
HCA Drivers (CI): Tavor, Hermon
> - I am not clear on the consumer side of this new interface: What
> prompts a ULP to start using this interface?
> Is it expected to attempt ibt_alloc_io_mem() until it exhausts all
> resources?
> It would be easier to assess the completeness and the usefulness of the
> TI if you either extended the case's scope
> to include at least the changes on one transport consumer or gave a
> real example thereof.
We are in the process of fixing bugs in certain IB ULPs to
be good citizens in big SPARC platforms with memory DR where
we have to be careful about what is in/out of the cage.
The current plan for ULP usage (i.e. TI usage) is related
to this motivation.
> - mi_ibt_version seems to be an enumeration of apparently mutually
> exclusive values IBTI_V{1,2,3}
> yet the definition suggests a combination of independent (discrete)
> capabilities
> (FMR support, DMA wrapper support, etc.)
The features are examples of what was included at each version change.
More discussion of the relationship between ABI and features below ...
> . Is there any consumer of this interface that uses DMA wapper but
> not FMR?
Well to be honest FMR in the current form turned out to be a failure.
So no one uses FMR in ON right now.
But I think more generally what is going to happen is that
the features will be used independently of each other,
since they are generally not related to each other.
> . For future evolution, is the mi_ibt_version always intended to
> express a monotonically increasing set
> of capabilities (capabs of V(n+1) includes all capabs of N(n)) ?
Yes, that is the intent, but it is not guaranteed. However,
as you might imagine, it would involve a great deal of
discussion/agreement to remove anything and the ARC would be
in the loop.
> Basically I'm trying to see if information of different nature if
> being encoded in the same field. Without slipping
> in a design discussion you should consider if two fileds are more
> appropriate: 1 version (number or enum) and one capabs (bitmask).
There are in fact capability bitmasks elsewhere.
In IB there are a number of optional features. So the bitmasks
generally are for saying you have these optional features.
But the version number is more an ABI thing, and it is mistake
to conflate the too, though the reason we have to change ABI
is that new features demand more fields in the structs, etc.
> - Under what condition can the caller of
> ibt_alloc_io_mem()/ibt_free_io_mem() expect the following error
> to be returned?
> 59 IBT_MR_ACCESS_REQ_INVALID Invalid Access Control Specified.
> 60 Remote Write or Remote Atomic access is
> 61 requested without specifying Local
> Write.
It's a mistake and that text should be removed. That's only
something that could happen during an IB memory registration.
We will correct it.
> - in ibc_alloc_io_mem.9e
> these two sections are in conflict:
> 11 ibt_status_t prefix_ibc_alloc_io_mem(ibc_hca_hdl_t hca_hdl,
> 12 size_t size, ibt_mr_flags_t mr_flag, caddr_t *kaddrp,
> 13 ibc_mem_alloc_hdl_t *mem_alloc_hdl);
> and
> 23 hca_hdl IBTF channel Interface (TI) HCA Handle previously
> obtained
> 24 by calling ibt_open_hca(9F).
> 25
Yeah, it looks like a cut-n-paste mistake, since it mentioned the TI
stuff instead of CI. It should say the handle comes from the
ibc_hca_info_t returned via ibc_attach(). We will correct it.
-ted
--
Ted H. Kim
Sun Microsystems, Inc. ted.kim at sun.com
222 North Sepulveda Blvd., 10th Floor (310) 341-1116
El Segundo, CA 90245 (310) 341-1120 FAX
