On Fri, 12 Oct 2012 16:43:32 -0600 Jason Gunthorpe <jguntho...@obsidianresearch.com> wrote:
> On Thu, Oct 11, 2012 at 01:02:45PM +0200, Or Gerlitz wrote: > > > Thinking about this a bit, will it be possible to provide a udev > > rulethat can dictate to the IB core what name / suffix digit to > > assign for a device with certain BDF? > > FWIW, I reflected on this once (I belive it was for the netlink > discussion).. Allowing for a device to be renamed creates a serious > problem for RDMA since there isn't a stable 'if-index' like way for > software to refer to it. > > The udev rules all rely on a means to rename the device once the > kernel has created it. > > For this reason, all the software I've built uses the port GID to ID > the resource for bind/etc and strongly discourages use of the RDMA > device name. IMHO, from a user perspective using GID's or GUIDs is just too hard. While the port GID is unique across reboots in practice with 1000's of nodes in a cluster names are much more useful and, at least here at LLNL, nobody refers to HCA's by GUID. They refer to them by hostname/card name. This is especially true when HCA's get swapped out for maintenance. Furthermore, how would you propose MPI selects a card when the job is launched on different nodes, chosen by a scheduler, from run to run? Unfortunately, I don't know the details of the "serious problem" you describe above but it seems that a udev solution to renaming things would be worth trying to solve from a users perspective. Ira > > The ipoib devices should already be renamable via udev rules, though > has anyone checked that udev doesn't have a problem with the long 'MAC'? > > Jason > -- > To unsubscribe from this list: send the line "unsubscribe linux-rdma" in > the body of a message to majord...@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html -- Ira Weiny Member of Technical Staff Lawrence Livermore National Lab 925-423-8008 wei...@llnl.gov -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
