Re: [lustre-discuss] Does Lustre support RoCE?

[Indivar Nair]

Thanks for the advice.
I had a hunch that the development will take time.

Regards,


Indivar Nair

On Thu, May 11, 2017 at 11:28 PM, Oucharek, Doug S <
doug.s.oucha...@intel.com> wrote:

> As I write this, I am banging my head against this wall trying to figure
> it out.  It is related to the new memory region registration process used
> by mlx5 cards.  I could really use the help of any Mellanox/RDMA experts
> out there.  The API has virtually no documentation and without the source
> code for MOFED 4, I am really in unable to do much more than guess at what
> is going on.
>
> So, expect this to take a long time to resolve and stick with MOFED 3.x.
>
> Doug
>
> On May 11, 2017, at 10:29 AM, Indivar Nair 
> wrote:
>
> Thanks a lot, Michael, Andreas, Simon, Doug,
> I have already installed MLNX OFED 4:-(
> I will now have to undo it and install the earlier version.
>
> Roughly, by when would the support for MLNX OFED 4 be available?
>
> Regards,
>
>
> Indivar Nair
>
> On Thu, May 11, 2017 at 9:35 PM, Oucharek, Doug S <
> doug.s.oucha...@intel.com> wrote:
>
>> The note regarding MOFED 4 not supported by Lustre: I’m working on it.
>> MOFED 4 did not drop support of Lustre, but did make API/behaviour changes
>> which Lustre has not fully adapted to yet.  The ball is in the Lustre
>> community’s court on this one now.
>>
>> Doug
>>
>> On May 11, 2017, at 8:47 AM, Simon Guilbault <
>> simon.guilba...@calculquebec.ca> wrote:
>>
>> Hi, your lnet.conf look fine, I tested lnet with RoCE V2 a while back
>> with a pair of server using Connectx4 with a single 25Gb interface and RDMA
>> was working with Centos 7.3, stock RHEL OFED and Lustre 2.9. The only
>> settings that I had to use in lustre's config was this one:
>>
>> options lnet networks=o2ib(ens2)
>>
>> The performance was about the same (1.9GB/s) without any tuning with the
>> lnet self-test but the CPU utilisation was a lot lower with RDMA than TCP
>> (3% vs 65% of a core).
>>
>> From my notes I took back then Lustre needed to be recompiled with MLNX
>> OFED 3.4 and MLNX OFED 4 dropped support of Lustre accordings to their
>> release notes.
>>
>> Ref 965588
>> https://www.mellanox.com/related-docs/prod_software/Mellanox
>> _OFED_Linux_Release_Notes_4_0-2_0_0_1.pdf
>> https://www.mellanox.com/related-docs/prod_software/Mellanox
>> _OFED_Linux_Release_Notes_4_0-2_0_2_0.pdf
>>
>>
>> On Thu, May 11, 2017 at 11:34 AM, Indivar Nair > > wrote:
>>
>>> So I should add something like this in lnet.conf -
>>>
>>> options lnet networks=o2ib0(p4p1)
>>>
>>> Thats it, right?
>>>
>>> Regards,
>>>
>>>
>>> Indivar Nair
>>>
>>> On Thu, May 11, 2017 at 8:39 PM, Dilger, Andreas >> com> wrote:
>>>
 If you have RoCE cards and configure them with OFED, and configure
 Lustre to use o2iblnd then it should use RDMA for those interfaces. The
 fact that they are RoCE cards is hidden below OFED.

 Cheers, Andreas

 > On May 11, 2017, at 08:36, Indivar Nair 
 wrote:
 >
 > Hi ...,
 >
 > I have read in different forums and blogs that Lustre supports RoCE.
 > But I cant find any documentation on it.
 >
 > I have a Lustre setup with 6 OSS and 2 SMB/NFS Gateways.
 > They are all interconnected using Mellanox SN2700 100G Switch and
 Mellanox Connect-X4 100G NICs.
 > I have installed the Mellanox OFED Drivers, but I cant find a way to
 tell Lustre / LNET to use RoCE.
 >
 > How do I go about?
 >
 > Regards,
 >
 >
 > Indivar Nair
 >
 >
 > ___
 > lustre-discuss mailing list
 > lustre-discuss@lists.lustre.org
 > http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org

>>>
>>>
>>> ___
>>> lustre-discuss mailing list
>>> lustre-discuss@lists.lustre.org
>>> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
>>>
>>>
>> ___
>> lustre-discuss mailing list
>> lustre-discuss@lists.lustre.org
>> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
>>
>>
>>
>
>
___
lustre-discuss mailing list
lustre-discuss@lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org

Re: [lustre-discuss] Does Lustre support RoCE?

[Oucharek, Doug S]

As I write this, I am banging my head against this wall trying to figure it 
out.  It is related to the new memory region registration process used by mlx5 
cards.  I could really use the help of any Mellanox/RDMA experts out there.  
The API has virtually no documentation and without the source code for MOFED 4, 
I am really in unable to do much more than guess at what is going on.

So, expect this to take a long time to resolve and stick with MOFED 3.x.

Doug

On May 11, 2017, at 10:29 AM, Indivar Nair 
> wrote:

Thanks a lot, Michael, Andreas, Simon, Doug,
I have already installed MLNX OFED 4:-(
I will now have to undo it and install the earlier version.

Roughly, by when would the support for MLNX OFED 4 be available?

Regards,


Indivar Nair

On Thu, May 11, 2017 at 9:35 PM, Oucharek, Doug S 
> wrote:
The note regarding MOFED 4 not supported by Lustre: I’m working on it. MOFED 4 
did not drop support of Lustre, but did make API/behaviour changes which Lustre 
has not fully adapted to yet.  The ball is in the Lustre community’s court on 
this one now.

Doug

On May 11, 2017, at 8:47 AM, Simon Guilbault 
> wrote:

Hi, your lnet.conf look fine, I tested lnet with RoCE V2 a while back with a 
pair of server using Connectx4 with a single 25Gb interface and RDMA was 
working with Centos 7.3, stock RHEL OFED and Lustre 2.9. The only settings that 
I had to use in lustre's config was this one:

options lnet networks=o2ib(ens2)

The performance was about the same (1.9GB/s) without any tuning with the lnet 
self-test but the CPU utilisation was a lot lower with RDMA than TCP (3% vs 65% 
of a core).

From my notes I took back then Lustre needed to be recompiled with MLNX OFED 
3.4 and MLNX OFED 4 dropped support of Lustre accordings to their release notes.

Ref 965588
https://www.mellanox.com/related-docs/prod_software/Mellanox_OFED_Linux_Release_Notes_4_0-2_0_0_1.pdf
https://www.mellanox.com/related-docs/prod_software/Mellanox_OFED_Linux_Release_Notes_4_0-2_0_2_0.pdf


On Thu, May 11, 2017 at 11:34 AM, Indivar Nair 
> wrote:
So I should add something like this in lnet.conf -

options lnet networks=o2ib0(p4p1)

Thats it, right?

Regards,


Indivar Nair

On Thu, May 11, 2017 at 8:39 PM, Dilger, Andreas 
> wrote:
If you have RoCE cards and configure them with OFED, and configure Lustre to 
use o2iblnd then it should use RDMA for those interfaces. The fact that they 
are RoCE cards is hidden below OFED.

Cheers, Andreas

> On May 11, 2017, at 08:36, Indivar Nair 
> > wrote:
>
> Hi ...,
>
> I have read in different forums and blogs that Lustre supports RoCE.
> But I cant find any documentation on it.
>
> I have a Lustre setup with 6 OSS and 2 SMB/NFS Gateways.
> They are all interconnected using Mellanox SN2700 100G Switch and Mellanox 
> Connect-X4 100G NICs.
> I have installed the Mellanox OFED Drivers, but I cant find a way to tell 
> Lustre / LNET to use RoCE.
>
> How do I go about?
>
> Regards,
>
>
> Indivar Nair
>
>
> ___
> lustre-discuss mailing list
> lustre-discuss@lists.lustre.org
> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org


___
lustre-discuss mailing list
lustre-discuss@lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org


___
lustre-discuss mailing list
lustre-discuss@lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org



___
lustre-discuss mailing list
lustre-discuss@lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org

Re: [lustre-discuss] Does Lustre support RoCE?

[Indivar Nair]

Thanks a lot, Michael, Andreas, Simon, Doug,
I have already installed MLNX OFED 4:-(
I will now have to undo it and install the earlier version.

Roughly, by when would the support for MLNX OFED 4 be available?

Regards,


Indivar Nair

On Thu, May 11, 2017 at 9:35 PM, Oucharek, Doug S  wrote:

> The note regarding MOFED 4 not supported by Lustre: I’m working on it.
> MOFED 4 did not drop support of Lustre, but did make API/behaviour changes
> which Lustre has not fully adapted to yet.  The ball is in the Lustre
> community’s court on this one now.
>
> Doug
>
> On May 11, 2017, at 8:47 AM, Simon Guilbault <
> simon.guilba...@calculquebec.ca> wrote:
>
> Hi, your lnet.conf look fine, I tested lnet with RoCE V2 a while back with
> a pair of server using Connectx4 with a single 25Gb interface and RDMA was
> working with Centos 7.3, stock RHEL OFED and Lustre 2.9. The only settings
> that I had to use in lustre's config was this one:
>
> options lnet networks=o2ib(ens2)
>
> The performance was about the same (1.9GB/s) without any tuning with the
> lnet self-test but the CPU utilisation was a lot lower with RDMA than TCP
> (3% vs 65% of a core).
>
> From my notes I took back then Lustre needed to be recompiled with MLNX
> OFED 3.4 and MLNX OFED 4 dropped support of Lustre accordings to their
> release notes.
>
> Ref 965588
> https://www.mellanox.com/related-docs/prod_software/
> Mellanox_OFED_Linux_Release_Notes_4_0-2_0_0_1.pdf
> https://www.mellanox.com/related-docs/prod_software/
> Mellanox_OFED_Linux_Release_Notes_4_0-2_0_2_0.pdf
>
>
> On Thu, May 11, 2017 at 11:34 AM, Indivar Nair 
>  wrote:
>
>> So I should add something like this in lnet.conf -
>>
>> options lnet networks=o2ib0(p4p1)
>>
>> Thats it, right?
>>
>> Regards,
>>
>>
>> Indivar Nair
>>
>> On Thu, May 11, 2017 at 8:39 PM, Dilger, Andreas > com> wrote:
>>
>>> If you have RoCE cards and configure them with OFED, and configure
>>> Lustre to use o2iblnd then it should use RDMA for those interfaces. The
>>> fact that they are RoCE cards is hidden below OFED.
>>>
>>> Cheers, Andreas
>>>
>>> > On May 11, 2017, at 08:36, Indivar Nair 
>>> wrote:
>>> >
>>> > Hi ...,
>>> >
>>> > I have read in different forums and blogs that Lustre supports RoCE.
>>> > But I cant find any documentation on it.
>>> >
>>> > I have a Lustre setup with 6 OSS and 2 SMB/NFS Gateways.
>>> > They are all interconnected using Mellanox SN2700 100G Switch and
>>> Mellanox Connect-X4 100G NICs.
>>> > I have installed the Mellanox OFED Drivers, but I cant find a way to
>>> tell Lustre / LNET to use RoCE.
>>> >
>>> > How do I go about?
>>> >
>>> > Regards,
>>> >
>>> >
>>> > Indivar Nair
>>> >
>>> >
>>> > ___
>>> > lustre-discuss mailing list
>>> > lustre-discuss@lists.lustre.org
>>> > http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
>>>
>>
>>
>> ___
>> lustre-discuss mailing list
>> lustre-discuss@lists.lustre.org
>> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
>>
>>
> ___
> lustre-discuss mailing list
> lustre-discuss@lists.lustre.org
> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
>
>
>
___
lustre-discuss mailing list
lustre-discuss@lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org

Re: [lustre-discuss] lustre-discuss Digest, Vol 134, Issue 27

[Ms. Megan Larko]

On the Subject of "ost doesn't mount", the Lustre community version 2.8.0
had a documented issue in that the SELinux context was appended at each
mount so the resulting behavior is that subsequent mounts would fail
because of multiple contexts which is an invalid configuration as noted
by  Strikwerda, Ger  in the "tunefs.lustre --print " command:

Persistent mount opts:
,errors=remount-ro,context=unconfined_u:object_r:user_
tmp_t:s0,context=unconfined_u:object_r:tmp_t:s0

A Solution:
The disabling of SELinux does certainly work, but my own solution was to
write a small script which ran on boot pror to the lustre file system mount
to re-write the parameters (saved from my initial printout from the
tunefs.lustre command) omitting the "context=.." section in the
'tunefs-lustre --mountfsoptions="" '  This action allowed me to
continue to use Lustre 2.8.0 with SELinux in "targeted" mode.

FYI, this is LU-7002 and was patched 14 March 2016 in newer versions of
Lustre.

Cheers,
megan

On Thu, May 11, 2017 at 5:19 AM, 
wrote:

> Send lustre-discuss mailing list submissions to
> lustre-discuss@lists.lustre.org
>
> To subscribe or unsubscribe via the World Wide Web, visit
> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
> or, via email, send a message with subject or body 'help' to
> lustre-discuss-requ...@lists.lustre.org
>
> You can reach the person managing the list at
> lustre-discuss-ow...@lists.lustre.org
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of lustre-discuss digest..."
>
>
> Today's Topics:
>
>1. Re: ost doesn't mount (Strikwerda, Ger)
>2. Re: ost doesn't mount (Strikwerda, Ger)
>
>
> --
>
> Message: 1
> Date: Thu, 11 May 2017 10:34:28 +0200
> From: "Strikwerda, Ger" 
> To: Colin Faber 
> Cc: Lustre discussion 
> Subject: Re: [lustre-discuss] ost doesn't mount
> Message-ID:
> 

Re: [lustre-discuss] Does Lustre support RoCE?

[Oucharek, Doug S]

The note regarding MOFED 4 not supported by Lustre: I’m working on it. MOFED 4 
did not drop support of Lustre, but did make API/behaviour changes which Lustre 
has not fully adapted to yet.  The ball is in the Lustre community’s court on 
this one now.

Doug

On May 11, 2017, at 8:47 AM, Simon Guilbault 
> wrote:

Hi, your lnet.conf look fine, I tested lnet with RoCE V2 a while back with a 
pair of server using Connectx4 with a single 25Gb interface and RDMA was 
working with Centos 7.3, stock RHEL OFED and Lustre 2.9. The only settings that 
I had to use in lustre's config was this one:

options lnet networks=o2ib(ens2)

The performance was about the same (1.9GB/s) without any tuning with the lnet 
self-test but the CPU utilisation was a lot lower with RDMA than TCP (3% vs 65% 
of a core).

From my notes I took back then Lustre needed to be recompiled with MLNX OFED 
3.4 and MLNX OFED 4 dropped support of Lustre accordings to their release notes.

Ref 965588
https://www.mellanox.com/related-docs/prod_software/Mellanox_OFED_Linux_Release_Notes_4_0-2_0_0_1.pdf
https://www.mellanox.com/related-docs/prod_software/Mellanox_OFED_Linux_Release_Notes_4_0-2_0_2_0.pdf


On Thu, May 11, 2017 at 11:34 AM, Indivar Nair 
> wrote:
So I should add something like this in lnet.conf -

options lnet networks=o2ib0(p4p1)

Thats it, right?

Regards,


Indivar Nair

On Thu, May 11, 2017 at 8:39 PM, Dilger, Andreas 
> wrote:
If you have RoCE cards and configure them with OFED, and configure Lustre to 
use o2iblnd then it should use RDMA for those interfaces. The fact that they 
are RoCE cards is hidden below OFED.

Cheers, Andreas

> On May 11, 2017, at 08:36, Indivar Nair 
> > wrote:
>
> Hi ...,
>
> I have read in different forums and blogs that Lustre supports RoCE.
> But I cant find any documentation on it.
>
> I have a Lustre setup with 6 OSS and 2 SMB/NFS Gateways.
> They are all interconnected using Mellanox SN2700 100G Switch and Mellanox 
> Connect-X4 100G NICs.
> I have installed the Mellanox OFED Drivers, but I cant find a way to tell 
> Lustre / LNET to use RoCE.
>
> How do I go about?
>
> Regards,
>
>
> Indivar Nair
>
>
> ___
> lustre-discuss mailing list
> lustre-discuss@lists.lustre.org
> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org


___
lustre-discuss mailing list
lustre-discuss@lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org


___
lustre-discuss mailing list
lustre-discuss@lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org

___
lustre-discuss mailing list
lustre-discuss@lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org

Re: [lustre-discuss] Does Lustre support RoCE?

[Simon Guilbault]

Hi, your lnet.conf look fine, I tested lnet with RoCE V2 a while back with
a pair of server using Connectx4 with a single 25Gb interface and RDMA was
working with Centos 7.3, stock RHEL OFED and Lustre 2.9. The only settings
that I had to use in lustre's config was this one:

options lnet networks=o2ib(ens2)

The performance was about the same (1.9GB/s) without any tuning with the
lnet self-test but the CPU utilisation was a lot lower with RDMA than TCP
(3% vs 65% of a core).

>From my notes I took back then Lustre needed to be recompiled with MLNX
OFED 3.4 and MLNX OFED 4 dropped support of Lustre accordings to their
release notes.

Ref 965588
https://www.mellanox.com/related-docs/prod_software/Mellanox_OFED_Linux_Release_Notes_4_0-2_0_0_1.pdf
https://www.mellanox.com/related-docs/prod_software/Mellanox_OFED_Linux_Release_Notes_4_0-2_0_2_0.pdf


On Thu, May 11, 2017 at 11:34 AM, Indivar Nair 
wrote:

> So I should add something like this in lnet.conf -
>
> options lnet networks=o2ib0(p4p1)
>
> Thats it, right?
>
> Regards,
>
>
> Indivar Nair
>
> On Thu, May 11, 2017 at 8:39 PM, Dilger, Andreas  > wrote:
>
>> If you have RoCE cards and configure them with OFED, and configure Lustre
>> to use o2iblnd then it should use RDMA for those interfaces. The fact that
>> they are RoCE cards is hidden below OFED.
>>
>> Cheers, Andreas
>>
>> > On May 11, 2017, at 08:36, Indivar Nair 
>> wrote:
>> >
>> > Hi ...,
>> >
>> > I have read in different forums and blogs that Lustre supports RoCE.
>> > But I cant find any documentation on it.
>> >
>> > I have a Lustre setup with 6 OSS and 2 SMB/NFS Gateways.
>> > They are all interconnected using Mellanox SN2700 100G Switch and
>> Mellanox Connect-X4 100G NICs.
>> > I have installed the Mellanox OFED Drivers, but I cant find a way to
>> tell Lustre / LNET to use RoCE.
>> >
>> > How do I go about?
>> >
>> > Regards,
>> >
>> >
>> > Indivar Nair
>> >
>> >
>> > ___
>> > lustre-discuss mailing list
>> > lustre-discuss@lists.lustre.org
>> > http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
>>
>
>
> ___
> lustre-discuss mailing list
> lustre-discuss@lists.lustre.org
> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
>
>
___
lustre-discuss mailing list
lustre-discuss@lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org

Re: [lustre-discuss] Does Lustre support RoCE?

[Indivar Nair]

So I should add something like this in lnet.conf -

options lnet networks=o2ib0(p4p1)

Thats it, right?

Regards,


Indivar Nair

On Thu, May 11, 2017 at 8:39 PM, Dilger, Andreas 
wrote:

> If you have RoCE cards and configure them with OFED, and configure Lustre
> to use o2iblnd then it should use RDMA for those interfaces. The fact that
> they are RoCE cards is hidden below OFED.
>
> Cheers, Andreas
>
> > On May 11, 2017, at 08:36, Indivar Nair 
> wrote:
> >
> > Hi ...,
> >
> > I have read in different forums and blogs that Lustre supports RoCE.
> > But I cant find any documentation on it.
> >
> > I have a Lustre setup with 6 OSS and 2 SMB/NFS Gateways.
> > They are all interconnected using Mellanox SN2700 100G Switch and
> Mellanox Connect-X4 100G NICs.
> > I have installed the Mellanox OFED Drivers, but I cant find a way to
> tell Lustre / LNET to use RoCE.
> >
> > How do I go about?
> >
> > Regards,
> >
> >
> > Indivar Nair
> >
> >
> > ___
> > lustre-discuss mailing list
> > lustre-discuss@lists.lustre.org
> > http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
>
___
lustre-discuss mailing list
lustre-discuss@lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org

Re: [lustre-discuss] Does Lustre support RoCE?

[Dilger, Andreas]

If you have RoCE cards and configure them with OFED, and configure Lustre to 
use o2iblnd then it should use RDMA for those interfaces. The fact that they 
are RoCE cards is hidden below OFED. 

Cheers, Andreas

> On May 11, 2017, at 08:36, Indivar Nair  wrote:
> 
> Hi ...,
> 
> I have read in different forums and blogs that Lustre supports RoCE.
> But I cant find any documentation on it.
> 
> I have a Lustre setup with 6 OSS and 2 SMB/NFS Gateways.
> They are all interconnected using Mellanox SN2700 100G Switch and Mellanox 
> Connect-X4 100G NICs.
> I have installed the Mellanox OFED Drivers, but I cant find a way to tell 
> Lustre / LNET to use RoCE.
> 
> How do I go about?
> 
> Regards,
> 
> 
> Indivar Nair 
> 
> 
> ___
> lustre-discuss mailing list
> lustre-discuss@lists.lustre.org
> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
___
lustre-discuss mailing list
lustre-discuss@lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org

[lustre-discuss] Does Lustre support RoCE?

[Indivar Nair]

Hi ...,

I have read in different forums and blogs that Lustre supports RoCE.
But I cant find any documentation on it.

I have a Lustre setup with 6 OSS and 2 SMB/NFS Gateways.
They are all interconnected using Mellanox SN2700 100G Switch and Mellanox
Connect-X4 100G NICs.
I have installed the Mellanox OFED Drivers, but I cant find a way to tell
Lustre / LNET to use RoCE.

How do I go about?

Regards,


Indivar Nair
___
lustre-discuss mailing list
lustre-discuss@lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org

Re: [lustre-discuss] ost doesn't mount

[Strikwerda, Ger]

To answer my own question:

# tunefs.lustre --mountfsoptions=" ,errors=remount-ro" /dev/dm-2

# tunefs.lustre --print /dev/dm-2
checking for existing Lustre data: found
Reading CONFIGS/mountdata

   Read previous values:
Target: umcgst08-OST0002
Index:  2
Lustre FS:  umcgst08
Mount type: ldiskfs
Flags:  0x2
  (OST )
Persistent mount opts: errors=remount-ro
Parameters:  mgsnode=172.23.34.214@tcp:172.23.34.213@tcp


   Permanent disk data:
Target: umcgst08-OST0002
Index:  2
Lustre FS:  umcgst08
Mount type: ldiskfs
Flags:  0x2
  (OST )
Persistent mount opts: errors=remount-ro
Parameters:  mgsnode=172.23.34.214@tcp:172.23.34.213@tcp

volumes are now able to mount, with SElinux disabled!




On Thu, May 11, 2017 at 10:34 AM, Strikwerda, Ger 
wrote:

> Hi Colin,
>
> [root@umcg-storage03 ~]# tunefs.lustre --print /dev/dm-2
> checking for existing Lustre data: found
> Reading CONFIGS/mountdata
>
>Read previous values:
> Target: umcgst08-OST0002
> Index:  2
> Lustre FS:  umcgst08
> Mount type: ldiskfs
> Flags:  0x2
>   (OST )
> Persistent mount opts: ,errors=remount-ro,context=
> unconfined_u:object_r:user_tmp_t:s0,context=unconfined_u:object_r:tmp_t:s0
> Parameters:  mgsnode=172.23.34.214@tcp:172.23.34.213@tcp
>
>
>
>
>
>
>Permanent disk data:
>
>
> Target: umcgst08-OST0002
>
> Index:  2
>
> Lustre FS:  umcgst08
>
> Mount type: ldiskfs
>
> Flags:  0x2
>
>   (OST )
>
>
> Persistent mount opts: ,errors=remount-ro,context=
> unconfined_u:object_r:user_tmp_t:s0,context=unconfined_u:
> object_r:tmp_t:s0
> Parameters:  mgsnode=172.23.34.214@tcp:172.23.34.213@tcp
>
>
>
>
> exiting before disk write.
>
> So it looks like the mount options are inside the filesystem? How do we
> get rid of those SELinux options? We have no mountoptions for this
> filesystem set in /etc/fstab
>
>
>
>
>
> On Wed, May 10, 2017 at 5:59 PM, Colin Faber 
> wrote:
>
>> 22 == invalid argument.
>>
>> Unrecognized mount option "context=unconfined_u:object_r:user_tmp_t:s0"
>> or missing value
>>
>> tunefs.lustre --print 
>>
>> Do you have this option set within your mount options list?
>>
>> -cf
>>
>>
>> On Wed, May 10, 2017 at 7:19 AM, Strikwerda, Ger > > wrote:
>>
>>> Hi all,
>>>
>>> On a OSS with SElinux disabled we get a strange probably selinux related
>>> error when we want to mount the OST:
>>>
>>> [root@umcg-storage03 /]# cat /etc/selinux/config
>>> SELINUX=disabled
>>>
>>> # mount -t lustre /dev/dm-3 /mnt/umcgst08-01
>>>
>>> dmesg log:
>>>
>>> Lustre: Lustre: Build Version: 2.8.0-RC5--PRISTINE-2.6.32-573
>>> .12.1.el6_lustre.x86_64
>>>
>>> LDISKFS-fs (dm-3): Unrecognized mount option
>>> "context=unconfined_u:object_r:user_tmp_t:s0" or missing value
>>> LustreError: 3266:0:(osd_handler.c:6305:osd_mount())
>>> umcgst08-OST0001-osd: can't mount 
>>> /dev/mapper/360080e50002d407603ca52240901:
>>> -22
>>> LustreError: 3266:0:(obd_config.c:578:class_setup()) setup
>>> umcgst08-OST0001-osd failed (-22)
>>> LustreError: 3266:0:(obd_mount.c:203:lustre_start_simple())
>>> umcgst08-OST0001-osd setup error -22
>>> LustreError: 3266:0:(obd_mount_server.c:1764:server_fill_super())
>>> Unable to start osd on /dev/mapper/360080e50002d407603ca52240901:
>>> -22
>>> LustreError: 3266:0:(obd_mount.c:1426:lustre_fill_super()) Unable to
>>> mount  (-22)
>>>
>>> What does a -22 Unable to mount exactly means? And how can we get rid of
>>> the unrecognized mount options?
>>>
>>>
>>> --
>>>
>>> Vriendelijke groet,
>>>
>>> Ger StrikwerdaChef Special
>>> Rijksuniversiteit Groningen
>>> Centrum voor Informatie Technologie
>>> Unit Pragmatisch Systeembeheer
>>>
>>> Smitsborg
>>> Nettelbosje 1
>>> 9747 AJ Groningen
>>> Tel. 050 363 9276
>>> "God is hard, God is fair
>>>  some men he gave brains, others he gave hair"
>>>
>>>
>>> ___
>>> lustre-discuss mailing list
>>> lustre-discuss@lists.lustre.org
>>> https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.lu
>>> stre.org_listinfo.cgi_lustre-2Ddiscuss-2Dlustre.org=DwICAg
>>> =IGDlg0lD0b-nebmJJ0Kp8A=x9pM59OqndbWw-lPPdr8w1Vud29EZi
>>> gcxcNkz0uw5oQ=QjWbp50LlCVPLX6V9kTVUlp20HagxUa85u2p35LjIO8&
>>> s=MwEH6wfs9JdiGbyrPBWzeGVdjSVk5BXb0m9A2UbNZh8=
>>>
>>>
>>
>
>
> --
>
> Vriendelijke groet,
>
> Ger StrikwerdaChef Special
> Rijksuniversiteit Groningen
> Centrum voor Informatie Technologie
> Unit Pragmatisch Systeembeheer
>
> Smitsborg
> Nettelbosje 1
> 9747 AJ Groningen
> Tel. 050 363 9276
> "God is hard, God is fair
>  some men he gave brains, others he gave hair"
>
>


-- 

Vriendelijke groet,

Ger StrikwerdaChef Special
Rijksuniversiteit Groningen
Centrum voor Informatie Technologie
Unit Pragmatisch Systeembeheer

Smitsborg
Nettelbosje 1
9747 AJ Groningen
Tel. 050 363 9276
"God is hard, God is fair
 some men he gave brains, others he gave hair"
___

Re: [lustre-discuss] ost doesn't mount

[Strikwerda, Ger]

Hi Colin,

[root@umcg-storage03 ~]# tunefs.lustre --print /dev/dm-2
checking for existing Lustre data: found
Reading CONFIGS/mountdata

   Read previous values:
Target: umcgst08-OST0002
Index:  2
Lustre FS:  umcgst08
Mount type: ldiskfs
Flags:  0x2
  (OST )
Persistent mount opts:
,errors=remount-ro,context=unconfined_u:object_r:user_tmp_t:s0,context=unconfined_u:object_r:tmp_t:s0
Parameters:  mgsnode=172.23.34.214@tcp:172.23.34.213@tcp




   Permanent disk
data:

Target:
umcgst08-OST0002

Index:
2

Lustre FS:
umcgst08

Mount type:
ldiskfs

Flags:
0x2

  (OST
)

Persistent mount opts:
,errors=remount-ro,context=unconfined_u:object_r:user_tmp_t:s0,context=unconfined_u:object_r:tmp_t:s0

Parameters:  mgsnode=172.23.34.214@tcp:172.23.34.213@tcp



exiting before disk write.

So it looks like the mount options are inside the filesystem? How do we get
rid of those SELinux options? We have no mountoptions for this filesystem
set in /etc/fstab





On Wed, May 10, 2017 at 5:59 PM, Colin Faber 
wrote:

> 22 == invalid argument.
>
> Unrecognized mount option "context=unconfined_u:object_r:user_tmp_t:s0"
> or missing value
>
> tunefs.lustre --print 
>
> Do you have this option set within your mount options list?
>
> -cf
>
>
> On Wed, May 10, 2017 at 7:19 AM, Strikwerda, Ger 
> wrote:
>
>> Hi all,
>>
>> On a OSS with SElinux disabled we get a strange probably selinux related
>> error when we want to mount the OST:
>>
>> [root@umcg-storage03 /]# cat /etc/selinux/config
>> SELINUX=disabled
>>
>> # mount -t lustre /dev/dm-3 /mnt/umcgst08-01
>>
>> dmesg log:
>>
>> Lustre: Lustre: Build Version: 2.8.0-RC5--PRISTINE-2.6.32-573
>> .12.1.el6_lustre.x86_64
>>
>> LDISKFS-fs (dm-3): Unrecognized mount option
>> "context=unconfined_u:object_r:user_tmp_t:s0" or missing value
>> LustreError: 3266:0:(osd_handler.c:6305:osd_mount())
>> umcgst08-OST0001-osd: can't mount 
>> /dev/mapper/360080e50002d407603ca52240901:
>> -22
>> LustreError: 3266:0:(obd_config.c:578:class_setup()) setup
>> umcgst08-OST0001-osd failed (-22)
>> LustreError: 3266:0:(obd_mount.c:203:lustre_start_simple())
>> umcgst08-OST0001-osd setup error -22
>> LustreError: 3266:0:(obd_mount_server.c:1764:server_fill_super()) Unable
>> to start osd on /dev/mapper/360080e50002d407603ca52240901: -22
>> LustreError: 3266:0:(obd_mount.c:1426:lustre_fill_super()) Unable to
>> mount  (-22)
>>
>> What does a -22 Unable to mount exactly means? And how can we get rid of
>> the unrecognized mount options?
>>
>>
>> --
>>
>> Vriendelijke groet,
>>
>> Ger StrikwerdaChef Special
>> Rijksuniversiteit Groningen
>> Centrum voor Informatie Technologie
>> Unit Pragmatisch Systeembeheer
>>
>> Smitsborg
>> Nettelbosje 1
>> 9747 AJ Groningen
>> Tel. 050 363 9276
>> "God is hard, God is fair
>>  some men he gave brains, others he gave hair"
>>
>>
>> ___
>> lustre-discuss mailing list
>> lustre-discuss@lists.lustre.org
>> https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.lu
>> stre.org_listinfo.cgi_lustre-2Ddiscuss-2Dlustre.org=
>> DwICAg=IGDlg0lD0b-nebmJJ0Kp8A=x9pM59OqndbWw-lPPdr8w1Vud2
>> 9EZigcxcNkz0uw5oQ=QjWbp50LlCVPLX6V9kTVUlp20HagxUa85u2p35Lj
>> IO8=MwEH6wfs9JdiGbyrPBWzeGVdjSVk5BXb0m9A2UbNZh8=
>>
>>
>


-- 

Vriendelijke groet,

Ger StrikwerdaChef Special
Rijksuniversiteit Groningen
Centrum voor Informatie Technologie
Unit Pragmatisch Systeembeheer

Smitsborg
Nettelbosje 1
9747 AJ Groningen
Tel. 050 363 9276
"God is hard, God is fair
 some men he gave brains, others he gave hair"
___
lustre-discuss mailing list
lustre-discuss@lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org