Hi Andreas,
I have done some more testing in a virtualized environment. I took AlmaLinux
8.10, fully updated (kernel 4.18.0-553.74.1.el8_10.x86_64) and build zfs 2.1.16
and lustre 2.15.6 + 2.15.7
For lustre 2.15.6 compression works as expected. (see attached file)
For lustre 2.15.7, depending on the PFL layout, there is some compression, but
less than with 2.15.6. Or there is no compression on the OST's at all.
In the test where there is some compression, the PFL layout is simple, few
components and no SEL.
I have also tested lustre 2.15.5 on AlmaLinux 8.6, self compiled, and that also
worked as it should.
In my previous test with AlmaLinux 8.10 - Lustre 2.15.5 - ZFS 2.1.15,
compression did not work. That was on bare metal, where zfs was installed from
the zfsonlinux repo. Lustre was installed from the whamcloud repo.
No idea what can cause this, but there seems to be something not ok with lustre
2.15.7 in combination with zfs compression.
Best regards,
Martin Balvers
Danone Global Research & Innovation Center B.V.
Uppsalalaan 12, 3584CT Utrecht
________________________________
From: Andreas Dilger <[email protected]>
Sent: Friday, August 01, 2025 22:58
To: BALVERS Martin <[email protected]>
Cc: lustre-discuss <[email protected]>
Subject: Re: [lustre-discuss] No ZFS compression on OST's when using PFL
WARNING - EXTERNAL SENDER - BE CYBERSAFE
You are changing a number of different things with each of your tests, not just
Lustre. It could be ZFS 2.1.5->2.1.15 or Alma 8.6->8.10 is also causing
issues... Hard to say for sure without apples-to-apples comparisons.
It should be possible to build Lustre 2.15.5 on Alma 8.6 + ZFS 2.1.5 to help
isolate the issue.
There was
LU-19193<https://urldefense.com/v3/__https://jira.whamcloud.com/browse/LU-19193__;!!OUGTln_Lrg!Ugs-QNv1jVaQ0FZzMnQ9mLWiu0QW-B3iSDiiXoIU2XwGrg2ab0Uh-J-krG6w9pYI3ffwwwLgjxtjhLeNfA$>
filed recently about ZFS and blocksize detection, which might hurt ZFS
compression, but that was never landed on 2.15.
Cheers, Andreas
On Aug 1, 2025, at 06:33, BALVERS Martin via lustre-discuss
<[email protected]> wrote:
I ended up installing lustre 2.15.1 on the new servers. Now compression works
as expected when PFL is enabled.
What I have tested so far is:
AlmaLinux 9.4 - Lustre 2.16.1 - ZFS 2.1.16 - No compression with PFL
AlmaLinux 8.10 - Lustre 2.15.7 - ZFS 2.2.8 - No compression with PFL
AlmaLinux 8.10 - Lustre 2.15.5 - ZFS 2.1.15 - No compression with PFL
AlmaLinux 8.6 - Lustre 2.15.1 - ZFS 2.1.5 - PFL + compression works as expected
I tried installing 2.15.2, 2.15.3 and 2.15.4 but that failed with ksym errors.
It seems that somewhere between lustre 2.15.1 and 2.15.5 the PFL + compression
breaks.
gr,
Martin Balvers
Cheers, Andreas
—
Andreas Dilger
Lustre Principal Architect
Whamcloud/DDN
Ce message électronique et tous les fichiers attachés qu'il contient sont
confidentiels et destinés exclusivement à l'usage de la personne à laquelle ils
sont adressés. Si vous avez reçu ce message par erreur, merci de le retourner à
son émetteur. Les idées et opinions présentées dans ce message sont celles de
son auteur, et ne représentent pas nécessairement celles de DANONE ou d'une
quelconque de ses filiales. La publication, l'usage, la distribution,
l'impression ou la copie non autorisée de ce message et des attachements qu'il
contient sont strictement interdits.
This e-mail and any files transmitted with it are confidential and intended
solely for the use of the individual to whom it is addressed. If you have
received this email in error please send it back to the person that sent it to
you. Any views or opinions presented are solely those of its author and do not
necessarily represent those of DANONE or any of its subsidiary companies.
Unauthorized publication, use, dissemination, forwarding, printing or copying
of this email and its associated attachments is strictly prohibited.
## Test data
# ls -gGh
total 9.4G
-rw-rwxr-- 1 143M Oct 24 2024 file1.txt
-rw-rwxr-- 1 162M Oct 25 2024 file2.txt
-rw-rwxr-- 1 166M Oct 25 2024 file3.txt
-rw-rwxr-- 1 122M Oct 25 2024 file4.txt
-rwxr-xr-x 1 8.3G Mar 20 2024 file5.fastq
-rw-r--r-- 1 197M Oct 24 2024 file6.fastq
-rw-r--r-- 1 304M Sep 11 15:28 file7.txt
# Lustre 2.15.6, zfs 2.1.16, AlamLinux 8.10, 4.18.0-553.74.1.el8_10.x86_64
lfs setstripe -E 1M -L mdt -E -1 -c -1 -S 4M /mnt/test
LUSTRE:SVNAME COMPRESS NAME RATIO REFRATIO USED LUSED
lustre-MDT0000 zstd-fast mdt/mdt 1.84x 1.84x 9.59M 10.8M
lustre-OST0000 zstd lustre/ost1 5.61x 5.61x 572M 3.11G
lustre-OST0001 zstd lustre/ost1 5.62x 5.62x 571M 3.11G
lustre-OST0002 zstd lustre/ost1 5.61x 5.61x 571M 3.11G
lfs setstripe -E 1M -L mdt -E 128M -c 1 -S 4M -E 2G -c 2 -z 64M -E -1 -c -1 -z
256M /mnt/test
LUSTRE:SVNAME COMPRESS NAME RATIO REFRATIO USED LUSED
lustre-MDT0000 zstd-fast mdt/mdt 1.84x 1.84x 9.59M 10.8M
lustre-OST0000 zstd lustre/ost1 5.72x 5.72x 574M 3.19G
lustre-OST0001 zstd lustre/ost1 5.72x 5.72x 654M 3.64G
lustre-OST0002 zstd lustre/ost1 5.34x 5.34x 485M 2.51G
# Lustre 2.15.7, zfs 2.1.16, AlamLinux 8.10, 4.18.0-553.74.1.el8_10.x86_64
lfs setstripe -E 1M -L mdt -E -1 -c -1 -S 4M /mnt/test
LUSTRE:SVNAME COMPRESS NAME RATIO REFRATIO USED LUSED
lustre-MDT0000 zstd-fast mdt/mdt 1.84x 1.84x 9.61M 10.8M
lustre-OST0000 zstd lustre/ost1 4.56x 4.56x 704M 3.12G
lustre-OST0001 zstd lustre/ost1 1.06x 1.06x 2.97G 3.14G
lustre-OST0002 zstd lustre/ost1 4.91x 4.91x 653M 3.11G
lfs setstripe -E 1M -L mdt -E 128M -c 1 -S 4M -E 2G -c 2 -z 64M -E -1 -c -1 -z
256M /mnt/test
LUSTRE:SVNAME COMPRESS NAME RATIO REFRATIO USED LUSED
lustre-MDT0000 zstd-fast mdt/mdt 1.84x 1.84x 9.62M 10.8M
lustre-OST0000 zstd lustre/ost1 1.00x 1.00x 3.26G 3.22G
lustre-OST0001 zstd lustre/ost1 1.00x 1.00x 3.71G 3.67G
lustre-OST0002 zstd lustre/ost1 1.00x 1.00x 2.56G 2.53G
_______________________________________________
lustre-discuss mailing list
[email protected]
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
