Hi All, I was able to get the lustre client mounted successfully. I was getting this error whenever i tried to mount client via tcp/o2ib network.
[867363.885584] LustreError: 162-5: Missing mount data: check that /sbin/mount.lustre isinstalled. [867363.885637] LustreError: 13214:0:(obd_mount.c:1325:lustre_fill_super()) Unable to mount (-22) [867363.885659] LustreError: 13214:0:(obd_mount.c:1325:lustre_fill_super()) Skipped 1 previous similar message [867364.107157] LustreError: 15c-8: MGC192.168.111.52@tcp: The configuration from log 'mylustre-client' failed (-2). This may be the result of communication errors between this node and the MGS, a bad configuration, or other errors. See the syslog for more information. [867364.107209] LustreError: 13215:0:(llite_lib.c:1046:ll_fill_super()) Unable to process log: -2 [867364.107729] Lustre: Unmounted mylustre-client MGC192.168.111.52@tcp but when checked on the mgs with cat /proc/fs/lustre/device, i could not see any entry for MGC192.168.111.52@tcp . Only MGC192.168.200.52@o2ib was present. Also some of the disks were missing. Which led me to look into my disks health. To my surprise the iscsi disks in zpool were "Degraded". Not sure what caused that. So i reconfigured the entire setup and was able to mount the client via o2ib without any issues. Couple of other tweaks i did this time is 1. Disabled the firewalld and ufw ( earlier i added an exception to allow the iscsi disks ) 2. used the following command to mount MGS and mdt ( both in single command). mkfs.lustre --fsname=lustre --mgs --mdt --backfstype=zfs mgs01/data --index 0 earlier i used below two commands mkfs.lustre --mgs --backfstype=zfs mds1_1/mgs mkfs.lustre --mdt --backfstype=zfs --fsname=mylustre --index=1 --mgsnode=192.168.200.52@o2ib mds1_1/mdt1 3. modified lustre.conf to have the preferred network as first parameter. Not sure if thats how its supposed to work. options lnet networks=o2ib(ib0),tcp0(eth0) if i want to connect over o2ib options lnet networks=tcp0(eth0),o2ib(ib0) if i want to connect over tcp Thanks all for your help. Hopefully my learning's will help others. On Mon, Aug 1, 2016 at 7:42 AM, Mohr Jr, Richard Frank (Rick Mohr) < rm...@utk.edu> wrote: > > > On Jul 28, 2016, at 9:54 PM, sohamm <soh...@gmail.com> wrote: > > > > Client is configured for IB interface. > > So it looks like there might be something wrong with the LNet config on > the client then. Based on the output from “lctl ping” that you ran from > the server, the client only reported a NID on the tcp network. > > > in my understanding i can specific the network of choice in the mount > command. tried both tcp and ib. > > That is true, but sometimes if the client and server both have interfaces > on two different networks (like ethernet and IB) there can be some subtle > issues. When you specify the NID for the MGS to mount the file system, the > client will retrieve information about the MDS/OSS servers from the MGS you > specified. This information includes the NIDS that the MDS/OSS servers > will listen for requests. If a client sees that a server has a NID on tcp0 > and a NID on o2ib0, and the client also has NIDs on tcp0 and o2ib0, then > the client sees that there are two paths to the same server and it will > just pick one of the paths (which might not be the one you want). And if > the path it chooses happens to be down, it won’t matter if the other path > is up. > > (Now, I should make a disclaimer about the above statements. I believe > that is how it worked on Lustre versions like 1.8 and 2.4. I have not > tried this with newer Lustre versions, so the behavior could be different. > I also have not experimented with anything like specifying weights for LNet > routes, so I don’t know if that could be used to prefer one interface over > another.) > > -- > Rick Mohr > Senior HPC System Administrator > National Institute for Computational Sciences > http://www.nics.tennessee.edu > >
_______________________________________________ lustre-discuss mailing list lustre-discuss@lists.lustre.org http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org