On 10/5/09 7:58 AM, "Aielli Roberto" <[email protected]> wrote:
> Hi, > I'm trying to configure Lustre 1.8.1 with a Voltaire infiniband network. > On the MGS, MDS and OSSs I have two interfaces: eth1 and ib0. I've > successfully completed a test using eth1 so I've mounted a filesystem > on client node. Now I want to do the same thing with Voltaire infiniband > (ib0) modifying the modprobe.conf on both servers an clients with the line: > > options lnet networks=tcp(ib0) This would use IPoIB, not native infiniband. Is that what you want? To do native infiniband, asssuming you are using OFED, you need to specify: options lnet networks=o2ib(ib0) Did you unload/reload the Lustre modules after changing modprobe.conf.local? If not, it would not recognize the changes in modprobe.conf.local. > > > When I try to mount the FS on the client node nothing happen and I find > the following error in the syslog: > > Oct 5 13:08:12 xc264 kernel: Lustre: > 5468:0:(linux-tcpip.c:688:libcfs_sock_connect()) Error -101 connecting > 0.0.0.0/1023 -> 172.31.1.25/988 > > Oct 5 13:08:12 xc264 kernel: Lustre: > 5468:0:(acceptor.c:95:lnet_connect_console_error()) Connection to > 172.31.1...@tcp at host 172.31.1.25 was unreachable: the network or that node > may be down, or > > Lustre may be misconfigured. > > Oct 5 13:08:12 xc264 kernel: Lustre: > 5468:0:(socklnd_cb.c:421:ksocknal_txlist_done()) Deleting packet type 1 len > 368 172.31.65...@tcp->172.31.1...@tcp > > Oct 5 13:08:17 xc264 kernel: Lustre: > 5474:0:(client.c:1383:ptlrpc_expire_one_request()) @@@ Request > x1315690795499541 sent from lustre-MDT0000-mdc-ffff81021f97e400 to NID > 172.31.1...@tcp 5s ago ha > > s timed out (limit 5s). > > Oct 5 13:08:17 xc264 kernel: r...@ffff81021828fc00 x1315690795499541/t0 > o38->[email protected]@tcp:12/10 lens 368/584 e 0 to 1 dl > 1254740897 ref 1 fl Rpc:N/0/0 rc 0/0 > > > The main problem is displayed on the first line. The MGS ib0 address is > 172.31.65.25 but as you can see the client always try to connect the > eth1 address (172.31.1.25) even shutting down eth1. > > Placing in modprobe.conf a line with vib(ib0) the problem is different > I've also tried to modify modprobe.conf by changing the options line to: > > options lnet networks=vib(ib0) > > > but in the syslog I've found: > > Oct 5 12:33:19 xc264 kernel: LustreError: > 4864:0:(api-ni.c:1043:lnet_startup_lndnis()) Can't load LND vib, module > kviblnd > > > Are there other configurations for IB that I forgot to make/modify? > > Thanks in advance for your help > _______________________________________________ > Lustre-discuss mailing list > [email protected] > http://lists.lustre.org/mailman/listinfo/lustre-discuss _______________________________________________ Lustre-discuss mailing list [email protected] http://lists.lustre.org/mailman/listinfo/lustre-discuss
