On 12/8/2015 4:21 PM, Jason Gunthorpe wrote: > On Tue, Dec 08, 2015 at 12:33:02PM -0500, kaike....@intel.com wrote: >> From: Kaike Wan <kaike....@intel.com> >> >> In an insecure IB fabric, the default pkey in a port is 0xffff, where each >> node is allowed to talk to any other node in the fabric, including the SA >> node. However, in a secure fabric, to limit member access, not all nodes >> can have the full-member default pkey 0xffff. A typical configuration is >> to let SA node have pkey 0xffff while all other nodes have pkey 0x7fff; in >> addition, each node can be assigned some other full-member pkeys, such as >> 0x8001 and 0x8002, so that it can be assigned to different partitions. >> In this case, each node can access SA, and yet limits its other access to >> only those nodes in its assigned partitions. In such a secure fabric, >> however, ibacm will not work by interpreting "default" in its default >> address file as 0xffff. > > ipoib always uses the 0 pkey index to create the default ipoib > interface. (see eg, update_parent_pkey)
This is beyond IBA spec and is currently a linux convention for IPoIB. IMO it should be changed to search for this pkey rather than assume it's in index 0. There's no requirement that it be in index 0 other than at bootup with non volatile storage (C10-123). > When operating securely the SA should place the pkey for default ipoib > operation in pkey index 0, and place 0x7FFF in another index. I run > alot of networks exactly like this and it works very well. Yes, it can run that way but more secure is without the full default pkey. When full default pkey is in every port, the rest of the partitioning doesn't really matter... -- Hal -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
