Re: [lustre-discuss] Which release to use?

[Riccardo Veraldi]

I am using Lustre 2.12.0 and seems workign pretty well, anyway I built 
it against zfs 0.7.12 libraries... was it a mistake ?

what's the zfs release that Lustre 2.12.0 is built/tested on ?


On 2/22/19 12:18 PM, Peter Jones wrote:


Nathan

Yes 2.12 is an LTS branch. We’re planning on putting out both 2.10.7 
and 2.12.1 this quarter but have been focusing on the former first to 
allow for more time to receive feedback from early adopters on 2.12.0. 
You can see the patches that will land starting to accumulate here - 
https://review.whamcloud.com/#/q/status:open+project:fs/lustre-release+branch:b2_12 
. I guess what I am trying to say is “be patient” ☺


Peter

*From: *Nathan R Crawford 
*Reply-To: *"nathan.crawf...@uci.edu" 
*Date: *Friday, February 22, 2019 at 11:31 AM
*To: *Peter Jones 
*Cc: *"lustre-discuss@lists.lustre.org" 
*Subject: *Re: [lustre-discuss] Which release to use?

Hi Peter,

  Somewhat related: where should we be looking for the commits leading 
up to 2.12.1? The b2_12 branch 
(https://git.whamcloud.com/?p=fs/lustre-release.git;a=shortlog;h=refs/heads/b2_12) 
has no activity since 2.12.0 was released. I assumed that if 2.12 is a 
LTS branch like 2.10, there would be something by now. Commits started 
appearing on b2_10 after a week or so.


  "Be patient" is an acceptable response :)

-Nate

On Fri, Feb 22, 2019 at 10:51 AM Peter Jones > wrote:


2.12.0 is relatively new. It does have some improvements over
2.10.x (notably Data on MDT) but if those are not an immediate
requirement then using 2.10.6 would be a proven and more
comnservative option. 2.12.51 is an interim development build for
2.13 and should absolutely not be used for production purposes.

On 2019-02-22, 10:07 AM, "lustre-discuss on behalf of Bernd
Melchers" mailto:lustre-discuss-boun...@lists.lustre.org> on behalf of
melch...@zedat.fu-berlin.de >
wrote:

    Hi all,
    in the git repository i find v2.10.6, v2.12.0 and v2.12.51.
Which version should
    i compile and use for my productive CentOS 7.6 System?

    Mit freundlichen Grüßen
    Bernd Melchers

    --
    Archiv- und Backup-Service | fab-serv...@zedat.fu-berlin.de

    Freie Universität Berlin   | Tel. +49-30-838-55905
    ___
    lustre-discuss mailing list
lustre-discuss@lists.lustre.org

http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org


___
lustre-discuss mailing list
lustre-discuss@lists.lustre.org

http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org


--

Dr. Nathan crawfordnathan.crawf...@uci.edu  
Modeling Facility Director
Department of Chemistry
1102 Natural Sciences II Office: 2101 Natural Sciences II
University of California, Irvine  Phone: 949-824-4508
Irvine, CA 92697-2025, USA

___
lustre-discuss mailing list
lustre-discuss@lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org



___
lustre-discuss mailing list
lustre-discuss@lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org

Re: [lustre-discuss] Which release to use?

[Patrick Farrell]

Please file bugs if anything does!

- Patrick

From: lustre-discuss  on behalf of 
Nathan R Crawford 
Sent: Friday, February 22, 2019 7:04:24 PM
To: Peter Jones
Cc: lustre-discuss@lists.lustre.org
Subject: Re: [lustre-discuss] Which release to use?

Thanks for the link! I'll spin up a test system with 2.12.0 and see what breaks.
-Nate

On Fri, Feb 22, 2019 at 12:20 PM Peter Jones 
mailto:pjo...@whamcloud.com>> wrote:

Nathan



Yes 2.12 is an LTS branch. We’re planning on putting out both 2.10.7 and 2.12.1 
this quarter but have been focusing on the former first to allow for more time 
to receive feedback from early adopters on 2.12.0. You can see the patches that 
will land starting to accumulate here - 
https://review.whamcloud.com/#/q/status:open+project:fs/lustre-release+branch:b2_12
 . I guess what I am trying to say is “be patient” ☺



Peter



From: Nathan R Crawford mailto:nrcra...@uci.edu>>
Reply-To: "nathan.crawf...@uci.edu" 
mailto:nathan.crawf...@uci.edu>>
Date: Friday, February 22, 2019 at 11:31 AM
To: Peter Jones mailto:pjo...@whamcloud.com>>
Cc: "lustre-discuss@lists.lustre.org" 
mailto:lustre-discuss@lists.lustre.org>>
Subject: Re: [lustre-discuss] Which release to use?



Hi Peter,



  Somewhat related: where should we be looking for the commits leading up to 
2.12.1? The b2_12 branch 
(https://git.whamcloud.com/?p=fs/lustre-release.git;a=shortlog;h=refs/heads/b2_12)
 has no activity since 2.12.0 was released. I assumed that if 2.12 is a LTS 
branch like 2.10, there would be something by now. Commits started appearing on 
b2_10 after a week or so.



  "Be patient" is an acceptable response :)



-Nate



On Fri, Feb 22, 2019 at 10:51 AM Peter Jones 
mailto:pjo...@whamcloud.com>> wrote:

2.12.0 is relatively new. It does have some improvements over 2.10.x (notably 
Data on MDT) but if those are not an immediate requirement then using 2.10.6 
would be a proven and more comnservative option. 2.12.51 is an interim 
development build for 2.13 and should absolutely not be used for production 
purposes.

On 2019-02-22, 10:07 AM, "lustre-discuss on behalf of Bernd Melchers" 
mailto:lustre-discuss-boun...@lists.lustre.org>
 on behalf of melch...@zedat.fu-berlin.de> 
wrote:

Hi all,
in the git repository i find v2.10.6, v2.12.0 and v2.12.51. Which version 
should
i compile and use for my productive CentOS 7.6 System?

Mit freundlichen Grüßen
Bernd Melchers

--
Archiv- und Backup-Service | 
fab-serv...@zedat.fu-berlin.de
Freie Universität Berlin   | Tel. +49-30-838-55905
___
lustre-discuss mailing list
lustre-discuss@lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org


___
lustre-discuss mailing list
lustre-discuss@lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org




--

Dr. Nathan Crawford  
nathan.crawf...@uci.edu

Modeling Facility Director

Department of Chemistry

1102 Natural Sciences II Office: 2101 Natural Sciences II

University of California, Irvine  Phone: 949-824-4508

Irvine, CA 92697-2025, USA


--

Dr. Nathan Crawford  
nathan.crawf...@uci.edu
Modeling Facility Director
Department of Chemistry
1102 Natural Sciences II Office: 2101 Natural Sciences II
University of California, Irvine  Phone: 949-824-4508
Irvine, CA 92697-2025, USA
___
lustre-discuss mailing list
lustre-discuss@lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org

Re: [lustre-discuss] Which release to use?

[Nathan R Crawford]

Thanks for the link! I'll spin up a test system with 2.12.0 and see what
breaks.
-Nate

On Fri, Feb 22, 2019 at 12:20 PM Peter Jones  wrote:

> Nathan
>
>
>
> Yes 2.12 is an LTS branch. We’re planning on putting out both 2.10.7 and
> 2.12.1 this quarter but have been focusing on the former first to allow for
> more time to receive feedback from early adopters on 2.12.0. You can see
> the patches that will land starting to accumulate here -
> https://review.whamcloud.com/#/q/status:open+project:fs/lustre-release+branch:b2_12
> . I guess what I am trying to say is “be patient” ☺
>
>
>
> Peter
>
>
>
> *From: *Nathan R Crawford 
> *Reply-To: *"nathan.crawf...@uci.edu" 
> *Date: *Friday, February 22, 2019 at 11:31 AM
> *To: *Peter Jones 
> *Cc: *"lustre-discuss@lists.lustre.org" 
> *Subject: *Re: [lustre-discuss] Which release to use?
>
>
>
> Hi Peter,
>
>
>
>   Somewhat related: where should we be looking for the commits leading up
> to 2.12.1? The b2_12 branch (
> https://git.whamcloud.com/?p=fs/lustre-release.git;a=shortlog;h=refs/heads/b2_12)
> has no activity since 2.12.0 was released. I assumed that if 2.12 is a LTS
> branch like 2.10, there would be something by now. Commits started
> appearing on b2_10 after a week or so.
>
>
>
>   "Be patient" is an acceptable response :)
>
>
>
> -Nate
>
>
>
> On Fri, Feb 22, 2019 at 10:51 AM Peter Jones  wrote:
>
> 2.12.0 is relatively new. It does have some improvements over 2.10.x
> (notably Data on MDT) but if those are not an immediate requirement then
> using 2.10.6 would be a proven and more comnservative option. 2.12.51 is an
> interim development build for 2.13 and should absolutely not be used for
> production purposes.
>
> On 2019-02-22, 10:07 AM, "lustre-discuss on behalf of Bernd Melchers" <
> lustre-discuss-boun...@lists.lustre.org on behalf of
> melch...@zedat.fu-berlin.de> wrote:
>
> Hi all,
> in the git repository i find v2.10.6, v2.12.0 and v2.12.51. Which
> version should
> i compile and use for my productive CentOS 7.6 System?
>
> Mit freundlichen Grüßen
> Bernd Melchers
>
> --
> Archiv- und Backup-Service | fab-serv...@zedat.fu-berlin.de
> Freie Universität Berlin   | Tel. +49-30-838-55905
> ___
> lustre-discuss mailing list
> lustre-discuss@lists.lustre.org
> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
>
>
> ___
> lustre-discuss mailing list
> lustre-discuss@lists.lustre.org
> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
>
>
>
>
> --
>
> Dr. Nathan Crawford  nathan.crawf...@uci.edu
>
> Modeling Facility Director
>
> Department of Chemistry
>
> 1102 Natural Sciences II Office: 2101 Natural Sciences II
>
> University of California, Irvine  Phone: 949-824-4508
>
> Irvine, CA 92697-2025, USA
>
>

-- 

Dr. Nathan Crawford  nathan.crawf...@uci.edu
Modeling Facility Director
Department of Chemistry
1102 Natural Sciences II Office: 2101 Natural Sciences II
University of California, Irvine  Phone: 949-824-4508
Irvine, CA 92697-2025, USA
___
lustre-discuss mailing list
lustre-discuss@lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org

Re: [lustre-discuss] Suspended jobs and rebooting lustre servers

[Andreas Dilger]

This is not really correct.

Lustre clients can handle the addition of OSTs to a running filesystem. The MGS 
will register the new OSTs, and the clients will be notified by the MGS that 
the OSTs have been added, so no need to unmount the clients during this process.


Cheers, Andreas

On Feb 21, 2019, at 19:23, Raj 
mailto:rajgau...@gmail.com>> wrote:

Hello Raj,
It’s best and safe to unmount from all the clients and then do the upgrade. 
Your FS is getting more OSTs and changing conf in the existing ones, your 
client needs to get the new layout by remounting it.
Also you mentioned about client eviction, during eviction the client has to 
drop it’s dirty pages and all the open file descriptors in the FS will be gone.

On Thu, Feb 21, 2019 at 12:25 PM Raj Ayyampalayam 
mailto:ans...@gmail.com>> wrote:
What can I expect to happen to the jobs that are suspended during the file 
system restart?
Will the processes holding an open file handle die when I unsuspend them after 
the filesystem restart?

Thanks!
-Raj


On Thu, Feb 21, 2019 at 12:52 PM Colin Faber 
mailto:cfa...@gmail.com>> wrote:
Ah yes,

If you're adding to an existing OSS, then you will need to reconfigure the file 
system which requires writeconf event.

On Thu, Feb 21, 2019 at 10:00 AM Raj Ayyampalayam 
mailto:ans...@gmail.com>> wrote:
The new OST's will be added to the existing file system (the OSS nodes are 
already part of the filesystem), I will have to re-configure the current HA 
resource configuration to tell it about the 4 new OST's.
Our exascaler's HA monitors the individual OST and I need to re-configure the 
HA on the existing filesystem.

Our vendor support has confirmed that we would have to restart the filesystem 
if we want to regenerate the HA configs to include the new OST's.

Thanks,
-Raj


On Thu, Feb 21, 2019 at 11:23 AM Colin Faber 
mailto:cfa...@gmail.com>> wrote:
It seems to me that steps may still be missing?

You're going to rack/stack and provision the OSS nodes with new OSTs'.

Then you're going to introduce failover options somewhere? new osts? existing 
system? etc?

If you're introducing failover with the new OST's and leaving the existing 
system in place, you should be able to accomplish this without bringing the 
system offline.

If you're going to be introducing failover to your existing system then you 
will need to reconfigure the file system to accommodate the new failover 
settings (failover nides, etc.)

-cf


On Thu, Feb 21, 2019 at 9:13 AM Raj Ayyampalayam 
mailto:ans...@gmail.com>> wrote:
Our upgrade strategy is as follows:

1) Load all disks into the storage array.
2) Create RAID pools and virtual disks.
3) Create lustre file system using mkfs.lustre command. (I still have to figure 
out all the parameters used on the existing OSTs).
4) Create mount points on all OSSs.
5) Mount the lustre OSTs.
6) Maybe rebalance the filesystem.
My understanding is that the above can be done without bringing the filesystem 
down. I want to create the HA configuration (corosync and pacemaker) for the 
new OSTs. This step requires the filesystem to be down. I want to know what 
would happen to the suspended processes across the cluster when I bring the 
filesystem down to re-generate the HA configs.

Thanks,
-Raj

On Thu, Feb 21, 2019 at 12:59 AM Colin Faber 
mailto:cfa...@gmail.com>> wrote:
Can you provide more details on your upgrade strategy? In some cases expanding 
your storage shouldn't impact client / job activity at all.

On Wed, Feb 20, 2019, 11:09 AM Raj Ayyampalayam 
mailto:ans...@gmail.com>> wrote:
Hello,

We are planning on expanding our storage by adding more OSTs to our lustre file 
system. It looks like it would be easier to expand if we bring the filesystem 
down and perform the necessary operations. We are planning to suspend all the 
jobs running on the cluster. We originally planned to add new OSTs to the live 
filesystem.

We are trying to determine the potential impact to the suspended jobs if we 
bring down the filesystem for the upgrade.
One of the questions we have is what would happen to the suspended processes 
that hold an open file handle in the lustre file system when the filesystem is 
brought down for the upgrade?
Will they recover from the client eviction?

We do have vendor support and have engaged them. I wanted to ask the community 
and get some feedback.

Thanks,
-Raj
___
lustre-discuss mailing list
lustre-discuss@lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
___
lustre-discuss mailing list
lustre-discuss@lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
___
lustre-discuss mailing list
lustre-discuss@lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustr

Re: [lustre-discuss] Which release to use?

[Peter Jones]

Nathan

Yes 2.12 is an LTS branch. We’re planning on putting out both 2.10.7 and 2.12.1 
this quarter but have been focusing on the former first to allow for more time 
to receive feedback from early adopters on 2.12.0. You can see the patches that 
will land starting to accumulate here - 
https://review.whamcloud.com/#/q/status:open+project:fs/lustre-release+branch:b2_12
 . I guess what I am trying to say is “be patient” ☺

Peter

From: Nathan R Crawford 
Reply-To: "nathan.crawf...@uci.edu" 
Date: Friday, February 22, 2019 at 11:31 AM
To: Peter Jones 
Cc: "lustre-discuss@lists.lustre.org" 
Subject: Re: [lustre-discuss] Which release to use?

Hi Peter,

  Somewhat related: where should we be looking for the commits leading up to 
2.12.1? The b2_12 branch 
(https://git.whamcloud.com/?p=fs/lustre-release.git;a=shortlog;h=refs/heads/b2_12)
 has no activity since 2.12.0 was released. I assumed that if 2.12 is a LTS 
branch like 2.10, there would be something by now. Commits started appearing on 
b2_10 after a week or so.

  "Be patient" is an acceptable response :)

-Nate

On Fri, Feb 22, 2019 at 10:51 AM Peter Jones 
mailto:pjo...@whamcloud.com>> wrote:
2.12.0 is relatively new. It does have some improvements over 2.10.x (notably 
Data on MDT) but if those are not an immediate requirement then using 2.10.6 
would be a proven and more comnservative option. 2.12.51 is an interim 
development build for 2.13 and should absolutely not be used for production 
purposes.

On 2019-02-22, 10:07 AM, "lustre-discuss on behalf of Bernd Melchers" 
mailto:lustre-discuss-boun...@lists.lustre.org>
 on behalf of melch...@zedat.fu-berlin.de> 
wrote:

Hi all,
in the git repository i find v2.10.6, v2.12.0 and v2.12.51. Which version 
should
i compile and use for my productive CentOS 7.6 System?

Mit freundlichen Grüßen
Bernd Melchers

--
Archiv- und Backup-Service | 
fab-serv...@zedat.fu-berlin.de
Freie Universität Berlin   | Tel. +49-30-838-55905
___
lustre-discuss mailing list
lustre-discuss@lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org


___
lustre-discuss mailing list
lustre-discuss@lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org


--

Dr. Nathan Crawford  
nathan.crawf...@uci.edu

Modeling Facility Director

Department of Chemistry

1102 Natural Sciences II Office: 2101 Natural Sciences II

University of California, Irvine  Phone: 949-824-4508

Irvine, CA 92697-2025, USA
___
lustre-discuss mailing list
lustre-discuss@lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org

Re: [lustre-discuss] Which release to use?

[Nathan R Crawford]

Hi Peter,

  Somewhat related: where should we be looking for the commits leading up
to 2.12.1? The b2_12 branch (
https://git.whamcloud.com/?p=fs/lustre-release.git;a=shortlog;h=refs/heads/b2_12)
has no activity since 2.12.0 was released. I assumed that if 2.12 is a LTS
branch like 2.10, there would be something by now. Commits started
appearing on b2_10 after a week or so.

  "Be patient" is an acceptable response :)

-Nate

On Fri, Feb 22, 2019 at 10:51 AM Peter Jones  wrote:

> 2.12.0 is relatively new. It does have some improvements over 2.10.x
> (notably Data on MDT) but if those are not an immediate requirement then
> using 2.10.6 would be a proven and more comnservative option. 2.12.51 is an
> interim development build for 2.13 and should absolutely not be used for
> production purposes.
>
> On 2019-02-22, 10:07 AM, "lustre-discuss on behalf of Bernd Melchers" <
> lustre-discuss-boun...@lists.lustre.org on behalf of
> melch...@zedat.fu-berlin.de> wrote:
>
> Hi all,
> in the git repository i find v2.10.6, v2.12.0 and v2.12.51. Which
> version should
> i compile and use for my productive CentOS 7.6 System?
>
> Mit freundlichen Grüßen
> Bernd Melchers
>
> --
> Archiv- und Backup-Service | fab-serv...@zedat.fu-berlin.de
> Freie Universität Berlin   | Tel. +49-30-838-55905
> ___
> lustre-discuss mailing list
> lustre-discuss@lists.lustre.org
> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
>
>
> ___
> lustre-discuss mailing list
> lustre-discuss@lists.lustre.org
> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
>


-- 

Dr. Nathan Crawford  nathan.crawf...@uci.edu
Modeling Facility Director
Department of Chemistry
1102 Natural Sciences II Office: 2101 Natural Sciences II
University of California, Irvine  Phone: 949-824-4508
Irvine, CA 92697-2025, USA
___
lustre-discuss mailing list
lustre-discuss@lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org

Re: [lustre-discuss] Which release to use?

[Peter Jones]

2.12.0 is relatively new. It does have some improvements over 2.10.x (notably 
Data on MDT) but if those are not an immediate requirement then using 2.10.6 
would be a proven and more comnservative option. 2.12.51 is an interim 
development build for 2.13 and should absolutely not be used for production 
purposes.

On 2019-02-22, 10:07 AM, "lustre-discuss on behalf of Bernd Melchers" 
 wrote:

Hi all,
in the git repository i find v2.10.6, v2.12.0 and v2.12.51. Which version 
should
i compile and use for my productive CentOS 7.6 System?

Mit freundlichen Grüßen
Bernd Melchers

-- 
Archiv- und Backup-Service | fab-serv...@zedat.fu-berlin.de
Freie Universität Berlin   | Tel. +49-30-838-55905
___
lustre-discuss mailing list
lustre-discuss@lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org


___
lustre-discuss mailing list
lustre-discuss@lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org

[lustre-discuss] Which release to use?

[Bernd Melchers]

Hi all,
in the git repository i find v2.10.6, v2.12.0 and v2.12.51. Which version should
i compile and use for my productive CentOS 7.6 System?

Mit freundlichen Grüßen
Bernd Melchers

-- 
Archiv- und Backup-Service | fab-serv...@zedat.fu-berlin.de
Freie Universität Berlin   | Tel. +49-30-838-55905
___
lustre-discuss mailing list
lustre-discuss@lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org

Re: [lustre-discuss] EINVAL error when writing to a PFL file (lustre 2.12.0)

[Patrick Farrell]

Thomas,


Yeah, that's possible, though I don't think people are regularly running in to 
this.  FWIW, I think the intention was to allow extension of layouts component 
by component.  An incomplete layout can have components added to it.


I can conceive of someone using an incomplete layout to enforce a file size 
limit, but that's about it.


- Patrick


From: LEIBOVICI Thomas 
Sent: Friday, February 22, 2019 11:09:03 AM
To: Patrick Farrell; lustre-discuss@lists.lustre.org
Subject: Re: [lustre-discuss] EINVAL error when writing to a PFL file (lustre 
2.12.0)

Hello Patrick,

Thank you for the quick reply.
No, I have no particular use-case in mind, I'm just playing around with PFL.

If this is currently not properly supported, a quick fix could be to prevent 
the user from creating such incomplete layouts?

Regards,
Thomas

On 2/22/19 5:33 PM, Patrick Farrell wrote:

Thomas,


This is expected, but it's also something we'd like to fix - See LU-9341.


Basically, append tries to instantiate the layout from 0 to infinity, and it 
fails because your layout is incomplete (ie doesn't go to infinity).

May I ask why you're creating a file with an incomplete layout?  Do you have a 
use case in mind?


- Patrick


From: lustre-discuss 

 on behalf of LEIBOVICI Thomas 

Sent: Friday, February 22, 2019 10:27:48 AM
To: lustre-discuss@lists.lustre.org
Subject: [lustre-discuss] EINVAL error when writing to a PFL file (lustre 
2.12.0)

Hello,

Is it expected to get an error when appending a PFL file made of 2
regions [0 - 1M] and [1M to 6M]
even if writing in this range?

I get an error when appending it, even when writting in the very first
bytes:

[root@vm0]# lfs setstripe  -E 1M -c 1 -E 6M -c 2 /mnt/lustre/m_fou3

[root@vm0]# lfs getstripe /mnt/lustre/m_fou3
/mnt/lustre/m_fou3
   lcm_layout_gen:2
   lcm_mirror_count:  1
   lcm_entry_count:   2
 lcme_id: 1
 lcme_mirror_id:  0
 lcme_flags:  init
 lcme_extent.e_start: 0
 lcme_extent.e_end:   1048576
   lmm_stripe_count:  1
   lmm_stripe_size:   1048576
   lmm_pattern:   raid0
   lmm_layout_gen:0
   lmm_stripe_offset: 3
   lmm_objects:
   - 0: { l_ost_idx: 3, l_fid: [0x10003:0x9cf:0x0] }

 lcme_id: 2
 lcme_mirror_id:  0
 lcme_flags:  0
 lcme_extent.e_start: 1048576
 lcme_extent.e_end:   6291456
   lmm_stripe_count:  2
   lmm_stripe_size:   1048576
   lmm_pattern:   raid0
   lmm_layout_gen:0
   lmm_stripe_offset: -1

[root@vm0]# stat -c %s /mnt/lustre/m_fou3
14

* append fails:

[root@vm0]# echo qsdkjqslkdjkj >> /mnt/lustre/m_fou3
bash: echo: write error: Invalid argument

# strace indicates that write() gets the error:

write(1, "qsdkjqslkdjkj\n", 14) = -1 EINVAL (Invalid argument)

* no error in case of an open/truncate:

[root@vm0]# echo qsdkjqslkdjkj > /mnt/lustre/m_fou3

OK

Is it expected or should I open a ticket?

Thomas

___
lustre-discuss mailing list
lustre-discuss@lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org

___
lustre-discuss mailing list
lustre-discuss@lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org

Re: [lustre-discuss] EINVAL error when writing to a PFL file (lustre 2.12.0)

[LEIBOVICI Thomas]

Hello Patrick,

Thank you for the quick reply.
No, I have no particular use-case in mind, I'm just playing around with PFL.

If this is currently not properly supported, a quick fix could be to 
prevent the user from creating such incomplete layouts?


Regards,
Thomas

On 2/22/19 5:33 PM, Patrick Farrell wrote:


Thomas,


This is expected, but it's also something we'd like to fix - See LU-9341.


Basically, append tries to instantiate the layout from 0 to infinity, 
and it fails because your layout is incomplete (ie doesn't go to 
infinity).



May I ask why you're creating a file with an incomplete layout?  Do 
you have a use case in mind?



- Patrick


*From:* lustre-discuss  on 
behalf of LEIBOVICI Thomas 

*Sent:* Friday, February 22, 2019 10:27:48 AM
*To:* lustre-discuss@lists.lustre.org
*Subject:* [lustre-discuss] EINVAL error when writing to a PFL file 
(lustre 2.12.0)

Hello,

Is it expected to get an error when appending a PFL file made of 2
regions [0 - 1M] and [1M to 6M]
even if writing in this range?

I get an error when appending it, even when writting in the very first
bytes:

[root@vm0]# lfs setstripe  -E 1M -c 1 -E 6M -c 2 /mnt/lustre/m_fou3

[root@vm0]# lfs getstripe /mnt/lustre/m_fou3
/mnt/lustre/m_fou3
   lcm_layout_gen:    2
   lcm_mirror_count:  1
   lcm_entry_count:   2
 lcme_id: 1
 lcme_mirror_id:  0
 lcme_flags:  init
 lcme_extent.e_start: 0
 lcme_extent.e_end:   1048576
   lmm_stripe_count:  1
   lmm_stripe_size:   1048576
   lmm_pattern:   raid0
   lmm_layout_gen:    0
   lmm_stripe_offset: 3
   lmm_objects:
   - 0: { l_ost_idx: 3, l_fid: [0x10003:0x9cf:0x0] }

 lcme_id: 2
 lcme_mirror_id:  0
 lcme_flags:  0
 lcme_extent.e_start: 1048576
 lcme_extent.e_end:   6291456
   lmm_stripe_count:  2
   lmm_stripe_size:   1048576
   lmm_pattern:   raid0
   lmm_layout_gen:    0
   lmm_stripe_offset: -1

[root@vm0]# stat -c %s /mnt/lustre/m_fou3
14

* append fails:

[root@vm0]# echo qsdkjqslkdjkj >> /mnt/lustre/m_fou3
bash: echo: write error: Invalid argument

# strace indicates that write() gets the error:

write(1, "qsdkjqslkdjkj\n", 14) = -1 EINVAL (Invalid argument)

* no error in case of an open/truncate:

[root@vm0]# echo qsdkjqslkdjkj > /mnt/lustre/m_fou3

OK

Is it expected or should I open a ticket?

Thomas

___
lustre-discuss mailing list
lustre-discuss@lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org



___
lustre-discuss mailing list
lustre-discuss@lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org

[lustre-discuss] Reminder - Lustre Usage Survey: Input Requested

[OpenSFS Administration]

Dear Lustre Community,

 

The OpenSFS Lustre Working Group has launched the eighth annual survey for
organizations using Lustre. We are looking for trends in Lustre usage to
assist with future planning on releases and will present the results at LUG.

 

Please complete this short survey ( 
https://www.surveymonkey.com/r/P8P6QL3) to make sure your organization's
voice is heard!

 

Response to the survey is due by February 28th. Note that all questions are
optional, so it is ok to submit a partially completed survey if you prefer
not to disclose some information.

 

Best regards,

OpenSFS Administration

__

OpenSFS Administration

3855 SW 153rd Drive Beaverton, OR 97003 USA

Phone: +1 503-619-0561 | Fax: +1 503-644-6708

Twitter:
 @OpenSFS 

Email:   ad...@opensfs.org | Website:
 www.opensfs.org 

 

___
lustre-discuss mailing list
lustre-discuss@lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org

Re: [lustre-discuss] EINVAL error when writing to a PFL file (lustre 2.12.0)

[Patrick Farrell]

Thomas,


This is expected, but it's also something we'd like to fix - See LU-9341.


Basically, append tries to instantiate the layout from 0 to infinity, and it 
fails because your layout is incomplete (ie doesn't go to infinity).

May I ask why you're creating a file with an incomplete layout?  Do you have a 
use case in mind?


- Patrick


From: lustre-discuss  on behalf of 
LEIBOVICI Thomas 
Sent: Friday, February 22, 2019 10:27:48 AM
To: lustre-discuss@lists.lustre.org
Subject: [lustre-discuss] EINVAL error when writing to a PFL file (lustre 
2.12.0)

Hello,

Is it expected to get an error when appending a PFL file made of 2
regions [0 - 1M] and [1M to 6M]
even if writing in this range?

I get an error when appending it, even when writting in the very first
bytes:

[root@vm0]# lfs setstripe  -E 1M -c 1 -E 6M -c 2 /mnt/lustre/m_fou3

[root@vm0]# lfs getstripe /mnt/lustre/m_fou3
/mnt/lustre/m_fou3
   lcm_layout_gen:2
   lcm_mirror_count:  1
   lcm_entry_count:   2
 lcme_id: 1
 lcme_mirror_id:  0
 lcme_flags:  init
 lcme_extent.e_start: 0
 lcme_extent.e_end:   1048576
   lmm_stripe_count:  1
   lmm_stripe_size:   1048576
   lmm_pattern:   raid0
   lmm_layout_gen:0
   lmm_stripe_offset: 3
   lmm_objects:
   - 0: { l_ost_idx: 3, l_fid: [0x10003:0x9cf:0x0] }

 lcme_id: 2
 lcme_mirror_id:  0
 lcme_flags:  0
 lcme_extent.e_start: 1048576
 lcme_extent.e_end:   6291456
   lmm_stripe_count:  2
   lmm_stripe_size:   1048576
   lmm_pattern:   raid0
   lmm_layout_gen:0
   lmm_stripe_offset: -1

[root@vm0]# stat -c %s /mnt/lustre/m_fou3
14

* append fails:

[root@vm0]# echo qsdkjqslkdjkj >> /mnt/lustre/m_fou3
bash: echo: write error: Invalid argument

# strace indicates that write() gets the error:

write(1, "qsdkjqslkdjkj\n", 14) = -1 EINVAL (Invalid argument)

* no error in case of an open/truncate:

[root@vm0]# echo qsdkjqslkdjkj > /mnt/lustre/m_fou3

OK

Is it expected or should I open a ticket?

Thomas

___
lustre-discuss mailing list
lustre-discuss@lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
___
lustre-discuss mailing list
lustre-discuss@lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org

[lustre-discuss] EINVAL error when writing to a PFL file (lustre 2.12.0)

[LEIBOVICI Thomas]

Hello,

Is it expected to get an error when appending a PFL file made of 2 
regions [0 - 1M] and [1M to 6M]

even if writing in this range?

I get an error when appending it, even when writting in the very first 
bytes:


[root@vm0]# lfs setstripe  -E 1M -c 1 -E 6M -c 2 /mnt/lustre/m_fou3

[root@vm0]# lfs getstripe /mnt/lustre/m_fou3
/mnt/lustre/m_fou3
  lcm_layout_gen:    2
  lcm_mirror_count:  1
  lcm_entry_count:   2
    lcme_id: 1
    lcme_mirror_id:  0
    lcme_flags:  init
    lcme_extent.e_start: 0
    lcme_extent.e_end:   1048576
  lmm_stripe_count:  1
  lmm_stripe_size:   1048576
  lmm_pattern:   raid0
  lmm_layout_gen:    0
  lmm_stripe_offset: 3
  lmm_objects:
  - 0: { l_ost_idx: 3, l_fid: [0x10003:0x9cf:0x0] }

    lcme_id: 2
    lcme_mirror_id:  0
    lcme_flags:  0
    lcme_extent.e_start: 1048576
    lcme_extent.e_end:   6291456
  lmm_stripe_count:  2
  lmm_stripe_size:   1048576
  lmm_pattern:   raid0
  lmm_layout_gen:    0
  lmm_stripe_offset: -1

[root@vm0]# stat -c %s /mnt/lustre/m_fou3
14

* append fails:

[root@vm0]# echo qsdkjqslkdjkj >> /mnt/lustre/m_fou3
bash: echo: write error: Invalid argument

# strace indicates that write() gets the error:

write(1, "qsdkjqslkdjkj\n", 14) = -1 EINVAL (Invalid argument)

* no error in case of an open/truncate:

[root@vm0]# echo qsdkjqslkdjkj > /mnt/lustre/m_fou3

OK

Is it expected or should I open a ticket?

Thomas

___
lustre-discuss mailing list
lustre-discuss@lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org

[lustre-discuss] systemd lnet start script has route config commented out

[Kurt Strosahl]

Good Morning,

I'm in the process of setting up a new lustre file system, one that will 
need to reach some nodes through a set of lnet routers.  Adding the routes by 
hand doesn't work but adding them to the lnet_routes.conf file doesn't get 
picked up on reboot.

I looked at the file and found that the line that would do the configuration 
automatically was commented out

cat /usr/lib/systemd/system/lnet.service
[Unit]
Description=lnet management

Requires=network-online.target
After=network-online.target openibd.service rdma.service

ConditionPathExists=!/proc/sys/lnet/

[Service]
Type=oneshot
RemainAfterExit=true
ExecStart=/sbin/modprobe lnet
#ExecStart=/usr/sbin/lctl network up
#ExecStart=/usr/sbin/lustre_routes_config /etc/lnet_routes.conf
ExecStart=/usr/sbin/lnetctl lnet configure
ExecStart=/usr/sbin/lnetctl import /etc/lnet.conf
ExecStop=/usr/sbin/lustre_rmmod ptlrpc
#ExecStop=/usr/sbin/lctl network down
ExecStop=/usr/sbin/lnetctl lnet unconfigure
ExecStop=/usr/sbin/lustre_rmmod libcfs ldiskfs

[Install]
WantedBy=multi-user.target


The RPMs I used were downloaded directly from whamcloud:
rpm -qa | grep lustre
lustre-zfs-dkms-2.10.6-1.el7.noarch
bpftool-3.10.0-957.el7_lustre.x86_64
perf-debuginfo-3.10.0-957.el7_lustre.x86_64
perf-3.10.0-957.el7_lustre.x86_64
lustre-osd-zfs-mount-2.10.6-1.el7.x86_64
lustre-2.10.6-1.el7.x86_64
lustre-iokit-2.10.6-1.el7.x86_64
lustre-resource-agents-2.10.6-1.el7.x86_64
kernel-debuginfo-common-x86_64-3.10.0-957.el7_lustre.x86_64

Is there a reason why it is commented out?

w/r,

Kurt J. Strosahl
System Administrator: Lustre, HPC
Scientific Computing Group, Thomas Jefferson National Accelerator Facility
___
lustre-discuss mailing list
lustre-discuss@lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org