Re: [lustre-discuss] How to speed up Lustre

2022-07-06 Thread Hans Henrik Happe via lustre-discuss
I haven't tried it, but the man page for setstripe  --pool explains it. Cheers, Hans Henrik On 06.07.2022 22.50, Thomas Roth via lustre-discuss wrote: Yes, I got it. But Marion states that they switched > to a PFL arrangement, where the first 64k lives on flash OST's (mounted on our metadata s

Re: [lustre-discuss] How to speed up Lustre

2022-07-06 Thread Andreas Dilger via lustre-discuss
On Jul 6, 2022, at 14:50, Thomas Roth mailto:t.r...@gsi.de>> wrote: Yes, I got it. But Marion states that they switched > to a PFL arrangement, where the first 64k lives on flash OST's (mounted on > our metadata servers), and the remainder of larger files lives on HDD OST's. So, how do you spec

[lustre-discuss] lustre mounts stuck in llog_process_or_fork

2022-07-06 Thread Nehring, Shane R [LAS] via lustre-discuss
Hello all, We recently rebuilt our filesystem on 2.15.0 due to issues we were seeing on our legacy volume. The mds and oss-es are rhel 8.6 The mdt is zfs as are all the osts. We built from b2_15 to get the rhel 8.6 compat commits. It's been working pretty well for the past week or so until we had

Re: [lustre-discuss] How to speed up Lustre

2022-07-06 Thread Marion Hakanson via lustre-discuss
Hi Thomas, I'll just clarify that on our system, the flash OSTs happen to be connected to the same servers which are our MDS's. But they are just OST's so in this case the MDS machines are also acting as OSS machines for our flash OST's. We do indeed have an nvme pool of OST's, and an hdd pool o

Re: [lustre-discuss] How to speed up Lustre

2022-07-06 Thread Thomas Roth via lustre-discuss
Yes, I got it. But Marion states that they switched > to a PFL arrangement, where the first 64k lives on flash OST's (mounted on our metadata servers), and the remainder of larger files lives on HDD OST's. So, how do you specify a particular OSTs (or group of OSTs) in a PFL? The OST-equivalent o

Re: [lustre-discuss] How to speed up Lustre

2022-07-06 Thread Andreas Dilger via lustre-discuss
Thomas, where the file data is stored depends entirely on the PFL layout used for the filesystem or parent directory. For DoM files, you need to specify a DoM component, like: lfs setstripe -E 64K -L mdt -E 1G -c 1 -E 16G -c 4 -E eof -c 32 so the first 64KB will be put onto the MDT where t

Re: [lustre-discuss] How to speed up Lustre

2022-07-06 Thread Thomas Roth via lustre-discuss
Hi Marion, I do not fully understand how to "mount flash OSTs on a metadata server" - You have a couple of SSDs, you assemble these into on block device and format it with "mkfs.lustre --ost ..." ? And then mount it just as any other OST? - PFL then puts the first 64k on these OSTs and the rest