Re: [lustre-discuss] Does Lustre support RoCE?

2017-05-11 Thread Indivar Nair
Thanks for the advice. I had a hunch that the development will take time. Regards, Indivar Nair On Thu, May 11, 2017 at 11:28 PM, Oucharek, Doug S < doug.s.oucha...@intel.com> wrote: > As I write this, I am banging my head against this wall trying to figure > it out. It is related to the new

Re: [lustre-discuss] Does Lustre support RoCE?

2017-05-11 Thread Oucharek, Doug S
As I write this, I am banging my head against this wall trying to figure it out. It is related to the new memory region registration process used by mlx5 cards. I could really use the help of any Mellanox/RDMA experts out there. The API has virtually no documentation and without the source co

Re: [lustre-discuss] Does Lustre support RoCE?

2017-05-11 Thread Indivar Nair
Thanks a lot, Michael, Andreas, Simon, Doug, I have already installed MLNX OFED 4:-( I will now have to undo it and install the earlier version. Roughly, by when would the support for MLNX OFED 4 be available? Regards, Indivar Nair On Thu, May 11, 2017 at 9:35 PM, Oucharek, Doug S wrote: > T

Re: [lustre-discuss] lustre-discuss Digest, Vol 134, Issue 27

2017-05-11 Thread Ms. Megan Larko
ttelbosje 1 > >> 9747 AJ Groningen > >> Tel. 050 363 9276 > >> "God is hard, God is fair > >> some men he gave brains, others he gave hair" > >> > >> > >> ___ > >> lustre-discuss mailing l

Re: [lustre-discuss] Does Lustre support RoCE?

2017-05-11 Thread Oucharek, Doug S
The note regarding MOFED 4 not supported by Lustre: I’m working on it. MOFED 4 did not drop support of Lustre, but did make API/behaviour changes which Lustre has not fully adapted to yet. The ball is in the Lustre community’s court on this one now. Doug On May 11, 2017, at 8:47 AM, Simon Gui

Re: [lustre-discuss] Does Lustre support RoCE?

2017-05-11 Thread Simon Guilbault
Hi, your lnet.conf look fine, I tested lnet with RoCE V2 a while back with a pair of server using Connectx4 with a single 25Gb interface and RDMA was working with Centos 7.3, stock RHEL OFED and Lustre 2.9. The only settings that I had to use in lustre's config was this one: options lnet networks=

Re: [lustre-discuss] Does Lustre support RoCE?

2017-05-11 Thread Indivar Nair
So I should add something like this in lnet.conf - options lnet networks=o2ib0(p4p1) Thats it, right? Regards, Indivar Nair On Thu, May 11, 2017 at 8:39 PM, Dilger, Andreas wrote: > If you have RoCE cards and configure them with OFED, and configure Lustre > to use o2iblnd then it should use

Re: [lustre-discuss] Does Lustre support RoCE?

2017-05-11 Thread Dilger, Andreas
If you have RoCE cards and configure them with OFED, and configure Lustre to use o2iblnd then it should use RDMA for those interfaces. The fact that they are RoCE cards is hidden below OFED. Cheers, Andreas > On May 11, 2017, at 08:36, Indivar Nair wrote: > > Hi ..., > > I have read in diff

[lustre-discuss] Does Lustre support RoCE?

2017-05-11 Thread Indivar Nair
Hi ..., I have read in different forums and blogs that Lustre supports RoCE. But I cant find any documentation on it. I have a Lustre setup with 6 OSS and 2 SMB/NFS Gateways. They are all interconnected using Mellanox SN2700 100G Switch and Mellanox Connect-X4 100G NICs. I have installed the Mell

Re: [lustre-discuss] ost doesn't mount

2017-05-11 Thread Strikwerda, Ger
To answer my own question: # tunefs.lustre --mountfsoptions=" ,errors=remount-ro" /dev/dm-2 # tunefs.lustre --print /dev/dm-2 checking for existing Lustre data: found Reading CONFIGS/mountdata Read previous values: Target: umcgst08-OST0002 Index: 2 Lustre FS: umcgst08 Mount type: ld

Re: [lustre-discuss] ost doesn't mount

2017-05-11 Thread Strikwerda, Ger
Hi Colin, [root@umcg-storage03 ~]# tunefs.lustre --print /dev/dm-2 checking for existing Lustre data: found Reading CONFIGS/mountdata Read previous values: Target: umcgst08-OST0002 Index: 2 Lustre FS: umcgst08 Mount type: ldiskfs Flags: 0x2 (OST ) Persistent mount