Dear all, I tried to get it working on centos 7.6 and am almost​ there...
I installed Mellanox OFED (MLNX_OFED_LINUX-4.9-4.0.8.0-rhel7.6-x86_64.tgz). It seemed to go through fine - it detected the connectx3 card and updated the firmware. I then went for a build from src. I had an issue that the out-of-the-box centos7.6 has kernel 3.10.0-957 but when I installed kernel-devl package it added the headers for 3.10.0-1062. In the end I had to get a package from the vault (https://stackoverflow.com/questions/58304570/no-package-kernel-devel-3-10-0-957-el7-x86-64-available) At this point using ./configure --disable-server --enable-client --with-o2ib=/usr/src/ofa_kernel/default failed and I had to tweak some files as per https://www.mail-archive.com/lustre-discuss@lists.lustre.org/msg15829.html<https://www.mail-archive.com/lustre-discuss@lists.lustre.org/msg15829.htmla> After that the build seemed to go to completion and I was left with some rpms. I did yum localinstall on these and tried to mount but this is where I'm stuck. mount -t lustre 10.3.11.32@o2ib:/scratchB /scratch mount.lustre: mount 10.3.11.32@o2ib:/scratchB at /scratch failed: No such file or directory Is the MGS specification correct? Is the filesystem name correct? If upgrading, is the copied client log valid? (see upgrade docs) I tried... lctl ping 10.3.11.32@o2ib failed to ping 10.3.11.32@o2ib: Input/output error On one of the current compute nodes with Centos6.7 and client 2.7 I get... lctl ping 10.3.11.32@o2ib 12345-0@lo 12345-10.3.11.32@o2ib 12345-10.4.11.32@o2ib1 On the new node... ping 10.3.11.32 PING 10.3.11.32 (10.3.11.32) 56(84) bytes of data. 64 bytes from 10.3.11.32: icmp_seq=1 ttl=64 time=2.41 ms 64 bytes from 10.3.11.32: icmp_seq=2 ttl=64 time=0.130 ms So it looks like ipoib works ok. I see this in /var/log/messages... Nov 1 21:23:44 glacier01 kernel: LustreError: 6045:0:(ldlm_lib.c:483:client_obd_setup()) can't add initial connection Nov 1 21:23:44 glacier01 kernel: LustreError: 6045:0:(obd_config.c:558:class_setup()) setup MGC10.3.11.32@o2ib failed (-2) Nov 1 21:23:44 glacier01 kernel: LustreError: 6045:0:(obd_mount.c:203:lustre_start_simple()) MGC10.3.11.32@o2ib setup error -2 Nov 1 21:23:44 glacier01 kernel: LustreError: 6045:0:(obd_mount.c:1582:lustre_fill_super()) Unable to mount (-2) Nov 1 21:24:13 glacier01 kernel: LNetError: 6054:0:(lib-move.c:1557:lnet_select_pathway()) no route to 10.3.11.32@o2ib I'm not sure if this is something to do with my lustre client build or if it's to do with the infiniband network on this new compute node. Anyone got any ideas? Many thanks, Emyr James Head of IT CRG - Centre for Genomic Regulation C/ Dr. Aiguader, 88 Edif. PRBB 08003 Barcelona, Spain Phone Ext: #1098 ________________________________ From: Emyr James <emyr.ja...@crg.eu> Sent: 21 October 2021 23:04 To: Patrick Farrell <pfarr...@ddn.com>; lustre-discuss@lists.lustre.org <lustre-discuss@lists.lustre.org> Subject: Re: Lustre Client on Ubuntu 20.04 Hi, Thanks for the info. Our server side will get upgraded to 2.12.7 LTS in a few months so looking for a more modern OS than redhat 6.7! Does the 2.14 client work with the 2.12.7 LTS...and even with 2.7? Many thanks, Emyr James Head of IT CRG - Centre for Genomic Regulation C/ Dr. Aiguader, 88 Edif. PRBB 08003 Barcelona, Spain Phone Ext: #1098 ________________________________ From: Patrick Farrell <pfarr...@ddn.com> Sent: 21 October 2021 22:57 To: Emyr James <emyr.ja...@crg.eu>; lustre-discuss@lists.lustre.org <lustre-discuss@lists.lustre.org> Subject: Re: Lustre Client on Ubuntu 20.04 Emyr, 2.10.8 is a fairly old version and I am pretty sure it won't build for Ubuntu 20.04. The most recent kernel it's known to build for is Linux 4.4.x, and Ubuntu 20.04 is at least 5.4.x. The only version which officially has support for 20.04 is Lustre 2.14 (and the upcoming 2.15). For Ubuntu builds, the instructions here give the basic steps: https://wiki.lustre.org/Compiling_Lustre And the other sections have instructions which will help you with adding in MOFED. Just bear in mind the comments about version compatibility. -Patrick ________________________________ From: lustre-discuss <lustre-discuss-boun...@lists.lustre.org> on behalf of Emyr James <emyr.ja...@crg.eu> Sent: Thursday, October 21, 2021 3:28 PM To: lustre-discuss@lists.lustre.org <lustre-discuss@lists.lustre.org> Subject: [lustre-discuss] Lustre Client on Ubuntu 20.04 Dear all, We currently have a lustre 2.7 system running on redhat 6.7. The 2.10.8 client has been tested on redhat 6.7 and works fine (using Mellanox OFED). I'd like to get a 2.10.8 client running on Ubuntu 20.04 which is compatible with Mellanox OFED. I haven't found any instructions on how to build just the client side on Ubuntu. Does anyone have instructions or can point me to a howto somewhere? Many thanks. Emyr James Head of IT CRG - Centre for Genomic Regulation C/ Dr. Aiguader, 88 Edif. PRBB 08003 Barcelona, Spain Phone Ext: #1098
_______________________________________________ lustre-discuss mailing list lustre-discuss@lists.lustre.org http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org