Re: [lustre-discuss] lustre server build instructions for Rocky 8.10 & 9.4

/lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org Cheers, Andreas -- Andreas Dilger Lustre Principal Architect Whamcloud ___ lustre-discuss mailing list lustre-discuss@lists.lustre.org http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org

Re: [lustre-discuss] Read/Write on specific stripe of file via C api

eed as remote PCI network access because they are limited by the PCI bus bandwidth. You would notice a difference is if you have a large number of clients and they are completely IO-bound that overwhelm the storage. Cheers, Andreas -- Andreas Dilger Lustre Pri

Re: [lustre-discuss] lustre-2.14.0_ddn136 make fails with: error: passing argument 1 of ‘bio_alloc’ makes pointer from integer without a cast [-Werror=int-conversion]

ustre-discuss-lustre.org Cheers, Andreas -- Andreas Dilger Lustre Principal Architect Whamcloud ___ lustre-discuss mailing list lustre-discuss@lists.lustre.org http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org

Re: [lustre-discuss] How to activate an OST on a client ?

Hi Jan, There is "lctl --device recover" that will trigger a reconnect to the named OST device (per "lctl dl" output), but not sure if that will help. Cheers, Andreas On Aug 22, 2024, at 06:36, Haarst, Jan van via lustre-discuss wrote:  Hi, Probably the wording of the subject doesn’t

Re: [lustre-discuss] lfs find %LF

ers have files named "0" or "2" due to bad stderr redirection). All of the in-tree tools can handle FIDs without the [], but we can't remove them for compatibility if external tools expect to see them. It also makes the FIDs more visua

Re: [lustre-discuss] changelog record format

fortunately googl'ing 'lustre changelog' doesn't help :( Cheers, Andreas -- Andreas Dilger Lustre Principal Architect Whamcloud ___ lustre-discuss mailing list lustre-discuss@lists.lustre.org http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org

Re: [lustre-discuss] Rhel8.10 Lustre Kernel Performance Decrease

g 4.190 5.887 5.076 5.954 11.924 Avg IOPS 61 43 50 42 21 Bandwidth (MB/s) 64 45 53 45 23 IO (GB) 384 274 318 271 135 Thanks, Rashun Baucum Internal From: Andreas Dilger mailto:adil...@whamcloud.com>> Sent: Thursday, July 4, 2024 12:40 AM To: Baucum, Rashun mailto:rashun.bau...@td.c

Re: [lustre-discuss] question on usage of O_LOV_DELAY_CREATE

__ lustre-discuss mailing list lustre-discuss@lists.lustre.org<mailto:lustre-discuss@lists.lustre.org> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org Cheers, Andreas -- Andreas Dilger Lustre Principal Architect Whamcloud __

Re: [lustre-discuss] Trying to only build the lustreapi without sudo - make install error (Permission denied)

ers) It would make more sense to build and install the libraries together with the modules so that they are a consistent version. Cheers, Andreas -- Andreas Dilger Lustre Principal Architect Whamcloud ___ lustre-discuss mailing list lustre-di

Re: [lustre-discuss] lustreapi not found when compiling sample C program

Cheers, Apostolis On 8/6/24 21:19, Andreas Dilger wrote: On Jun 8, 2024, at 08:14, Apostolis Stamatis via lustre-discuss mailto:lustre-discuss@lists.lustre.org>> wrote: Hello everyone, I am trying to use the C api for lustre, using Ubuntu 22.04, kernel version 5.15.0-107 and lustre c

Re: [lustre-discuss] GitHub lustre-releases repo not in sync

git://git.whamcloud.com/fs/lustre-release.git does not work. I hope this can be fixed in some way. Cheers, Tom -- Andreas Dilger Lustre Principal Architect Whamcloud ___ lustre-discuss mailing list lustre-discuss@lists.lustre.org http://lists.lust

Re: [lustre-discuss] omnipath and lnet_selftest performance

rhel7 to rhel9, i'm unsure if there's a tunable not tuned. (unfortunately i don't have/can't seem to find previous numbers to compare) Cheers, Andreas -- Andreas Dilger Lustre Principal Architect Whamcloud ___ lustre-discuss mai

Re: [lustre-discuss] Rhel8.10 Lustre Kernel Performance Decrease

EL8.10? Which RHEL version are you comparing against, RHEL 8.9? Have you upgraded both the clients and servers to RHEL8.10, or only the clients? Cheers, Andreas -- Andreas Dilger Lustre Principal Architect Whamcloud ___ lustre-discuss mailin

Re: [lustre-discuss] Compiling client issue lustre 2.12.9

Chef d'Oeuvre égyptien, inaltérable! - Mais on vous demande pas de conserver, on vous demande de détruire! (Michel Audiard) ___ lustre-discuss mailing list lustre-discuss@lists.lustre.org<mailto:lustre-discuss@lists.lustre.org> http://

[lustre-discuss] Fwd: lustreapi not found when compiling sample C program

On Jun 8, 2024, at 08:14, Apostolis Stamatis via lustre-discuss wrote: Hello everyone, I am trying to use the C api for lustre, using Ubuntu 22.04, kernel version 5.15.0-107 and lustre client modules version 2.15.4 I am building lustre from source with the following steps (removed some junk

Re: [lustre-discuss] Unexpected used guard number

discuss@lists.lustre.org> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org Cheers, Andreas -- Andreas Dilger Lustre Principal Architect Whamcloud ___ lustre-discuss mailing list lustre-discuss@lists.lustre.org http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org

Re: [lustre-discuss] kernel threads for rpcs in flight

3893:1797634358961152:0@lo:16:dd.0> 0100:0010:2.0:1714503761.144172:0:23893:0:(client.c:2239:ptlrpc_check_set()) Completed RPC req@90c9a6ad3040 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:e81f3122-b1bc-4ac4-afcb-f6629a81e5bd:23893:1797634358961152:0@lo:16:dd.0&l

Re: [lustre-discuss] kernel threads for rpcs in flight

29a81e5bd:23893:1797634358961152:0@lo:16:dd.0 There are no stats files that aggregate information about ptlrpcd thread utilization. Cheers, Andreas -- Andreas Dilger Lustre Principal Architect Whamcloud ___ lustre-discuss mailing list lustre-discuss@lists.lustre.org http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org

Re: [lustre-discuss] [EXTERNAL] [BULK] Files created in append mode don't obey directory default stripe count

t;mailto:lustre-discuss@lists.lustre.org> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org ___ lustre-discuss mailing list lustre-discuss@lists.lustre.org<mailto:lustre-discuss@lists.lustre.org> http://lists.lustre.org/listinfo.cgi/lus

Re: [lustre-discuss] kernel threads for rpcs in flight

needed. Are there readily available statistics or tools for this scenario? What statistics are you looking for? There are "{osc,mdc}.*.stats" and "{osc,mdc}.*rpc_stats" that have aggregate information about RPC counts and latency. Cheers, Andreas -- Andreas Dilger Lustre Pri

Re: [lustre-discuss] ko2iblnd.conf

1324:0:(o2iblnd_cb.c:2612:kiblnd_passive_connect()) Can't accept conn from xxx.xxx.xxx.xxx@o2ib2, queue depth too large: 42 (<=32 wanted) — Dan Szkola FNAL On Apr 11, 2024, at 12:36 PM, Andreas Dilger mailto:adil...@whamcloud.com>> wrote: [EXTERNAL] – This message is from an external send

Re: [lustre-discuss] ko2iblnd.conf

ndreas -- Andreas Dilger Lustre Principal Architect Whamcloud ___ lustre-discuss mailing list lustre-discuss@lists.lustre.org http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org

Re: [lustre-discuss] Could not read from remote repository

/review.whamcloud.com/ and register your SSH public key there and then use: git clone ssh://review.whamcloud.com:29418/fs/lustre-release which you would want to do anyway if you are planning to submit any patches. Cheers, Andreas -- Andreas Dilger Lustre Principal Architect Whamcloud

Re: [lustre-discuss] Building Lustre against Mellanox OFED

ant to keep both RPMs installed (e.g. for different kernels) then you can probably just use "--force" since it looks like the .conf file would likely be the same from both packages. Cheers, Andreas -- Andreas Dilger Lustre Principal Architect Whamcloud ___

Re: [lustre-discuss] The confusion for mds hardware requirement

;100,000' files also an empirical parameter? Do I need to test it. Can I directly use the values '256' and '100,000'? Andreas Dilger mailto:adil...@whamcloud.com>> 于2024年3月11日周一 05:47写道： These numbers are just estimates, you can use values more suitable to your wor

Re: [lustre-discuss] The confusion for mds hardware requirement

es/core and 100,000 files, determined? Why? ___ lustre-discuss mailing list lustre-discuss@lists.lustre.org<mailto:lustre-discuss@lists.lustre.org> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org Cheers, Andreas -- Andreas Dilger

Re: [lustre-discuss] Issues draining OSTs for decommissioning

It's almost certainly just internal files. You could mount as ldiskfs and run "ls -lR" to check. Cheers, Andreas > On Mar 6, 2024, at 22:23, Scott Wood via lustre-discuss > wrote: > > Hi folks, > > Time to empty some OSTs to shut down some old arrays. I've been following > the docs from

Re: [lustre-discuss] lustre-client-dkms-2.15.4 is still checking for python2

dkms-rpm I can install the generated lustre-client-dkms-2.15.4-1.el9.noarch.rpm on AlmaLinux 9.3 I have no idea what that script does, or if it functions with python2 instead of python2 as env. Gr, Martin Balvers From: Andreas Dilger mailto:adil...@whamcloud.com>> Sent: Tuesday, January

Re: [lustre-discuss] ldiskfs / mdt size limits

Thomas, You are exactly correct that large MDTs can be useful for DoM if you have HDD OSTs. The benefit is relatively small if you have NVMe OSTs. If the MDT is larger than 16TB it must be formatted with the extents feature to address block numbers over 2^32. Unfortunately, this is _slightly_ l

Re: [lustre-discuss] Lustre github mirror out of sync

loud.com/fs/lustre-release.git and > https://github.com/lustre/lustre-release off on purpose? > > BR, > Tommi Cheers, Andreas -- Andreas Dilger Lustre Principal Architect Whamcloud ___ lustre-discuss mailing list lustre-discus

Re: [lustre-discuss] Odd behavior with tunefs.lustre and device index

_ lustre-discuss mailing list lustre-discuss@lists.lustre.org<mailto:lustre-discuss@lists.lustre.org> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org Cheers, Andreas -- Andreas Dilger Lustre Principal Architect Whamcloud ___ lustre-discuss mailing list lustre-discuss@lists.lustre.org http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org

Re: [lustre-discuss] OST still has inodes and size after deleting all files

parent FID for use with "lfs fid2path" on the client to see if there are any files related to these objects. You could also run "ll_decode_filter_fid" to do the same thing on the mounted ldiskfs filesystem. It is likely that there are a few stray objects from deleted files,

Re: [lustre-discuss] lustre-client-dkms-2.15.4 is still checking for python2

stre-client (try to add '--skip-broken' to skip uninstallable packages or '--nobest' to use not only best candidate packages) According to the changelog this should have been fixed (https://wiki.lustre.org/Lustre_2.15.4_Changelog). Regards, Martin Balvers Cheers, Andreas --

Re: [lustre-discuss] Lustre errors asking for help

ing them, or how to correct the issue. > > Any help would be greatly appreciated. Thanks a million for any suggestions > and solutions > > All the best > Roman > > > ___ > lustre-discuss mailing list > lustre-discuss@lists.lustre.org > http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org Cheers, Andreas -- Andreas Dilger Lustre Principal Architect Whamcloud ___ lustre-discuss mailing list lustre-discuss@lists.lustre.org http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org

Re: [lustre-discuss] LNet Multi-Rail config - with BODY!

of doing more than one connection per NID? Gwen ___ lustre-discuss mailing list lustre-discuss@lists.lustre.org<mailto:lustre-discuss@lists.lustre.org> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org Cheers, Andreas -- Andreas Dilger Lustre Principal Architect Whamcloud

Re: [lustre-discuss] Mixing ZFS and LDISKFS

mber of OSTs overall as the cluster being extended. On Fri, 12 Jan 2024 at 11:26, Andreas Dilger mailto:adil...@whamcloud.com>> wrote: Yes, some systems use ldiskfs for the MDT (for performance) and ZFS for the OSTs (for low-cost RAID). The IOPS performance of ZFS is low vs. ldiskfs, but

Re: [lustre-discuss] Recommendation on number of OSTs

f OSTs? In order to maximize throughput, go for more number of OSS with small OSTs. This means that it will end up with 1000s of OSTs. Any suggestions or recommendations? Thank you, Cheers, Andreas -- Andreas Dilger Lustre Principal Architect Wha

Re: [lustre-discuss] Mixing ZFS and LDISKFS

Yes, some systems use ldiskfs for the MDT (for performance) and ZFS for the OSTs (for low-cost RAID). The IOPS performance of ZFS is low vs. ldiskfs, but the streaming bandwidth is fine. Cheers, Andreas > On Jan 12, 2024, at 08:40, Backer via lustre-discuss > wrote: > >  > Hi, > > Could

Re: [lustre-discuss] Symbols not found in newly built lustre?

/lib/modules/4.18.0-513.9.1.el8_9.x86_64/extra/lustre/fs/lmv.ko needs "cfs_fail_loc": /lib/modules/4.18.0-513.9.1.el8_9.x86_64/extra/lustre/net/libcfs.ko /lib/modules/4.18.0-513.9.1.el8_9.x86_64/extra/lustre/fs/lmv.ko needs "LNetGetId": /lib/modules/4.18.0-513.9.1.el8_9.x86_64/extra/lustre/net/lnet.ko /lib/modules/4.18.0-513.9.1.el8_9.x86_64/extra/lustre/net/ksocklnd.ko needs "lnet_inet_enumerate": /lib/modules/4.18.0-513.9.1.el8_9.x86_64/extra/lustre/net/lnet.ko /lib/modules/4.18.0-513.9.1.el8_9.x86_64/extra/lustre/net/ksocklnd.ko needs "cfs_cpt_bind": /lib/modules/4.18.0-513.9.1.el8_9.x86_64/extra/lustre/net/libcfs.ko /lib/modules/4.18.0-513.9.1.el8_9.x86_64/extra/lustre/net/lnet_selftest.ko needs "lnet_cpt_of_nid": /lib/modules/4.18.0-513.9.1.el8_9.x86_64/extra/lustre/net/lnet.ko /lib/modules/4.18.0-513.9.1.el8_9.x86_64/extra/lustre/net/lnet_selftest.ko needs "cfs_wi_schedule": /lib/modules/4.18.0-513.9.1.el8_9.x86_64/extra/lustre/net/libcfs.ko ___ lustre-discuss mailing list lustre-discuss@lists.lustre.org<mailto:lustre-discuss@lists.lustre.org> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org Cheers, Andreas -- Andreas Dilger Lustre Principal Architect Whamcloud ___ lustre-discuss mailing list lustre-discuss@lists.lustre.org http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org

Re: [lustre-discuss] 2.15.4 o2iblnd on RoCEv2?

ng them... Something for the LNet folks to figure out. Cheers, Andreas On Jan 10, 2024, at 13:29, Jeff Johnson mailto:jeff.john...@aeoncomputing.com>> wrote: A LU ticket and patch for lnetctl or for me being an under-caffeinated idiot? ;-) On Wed, Jan 10, 2024 at 12:06 PM Andreas Dil

Re: [lustre-discuss] 2.15.4 o2iblnd on RoCEv2?

It would seem that the error message could be improved in this case? Could you file an LU ticket for that with the reproducer below, and ideally along with a patch? Cheers, Andreas > On Jan 10, 2024, at 11:37, Jeff Johnson > wrote: > > Man am I an idiot. Been up all night too many nights i

Re: [lustre-discuss] Extending Lustre file system

to 0 before) to some finite value and started file migration. As long as the migration is more effective, faster, than the users's file creations, the result should be evenly filled OSTs with a good mixture of files (file sizes, ages, types). Cheers Thomas On 1/8/24 19:07, Andreas Dilg

Re: [lustre-discuss] Extending Lustre file system

The need to rebalance depends on how full the existing OSTs are. My recommendation if you know that the data will continue to grow is to add new OSTs when the existing ones are at 60-70% full, and add them in larger groups rather than one at a time. Cheers, Andreas > On Jan 8, 2024, at 09:29,

Re: [lustre-discuss] Building lustre on rocky 8.8 fails?

_ lustre-discuss mailing list lustre-discuss@lists.lustre.org<mailto:lustre-discuss@lists.lustre.org> <mailto:lustre-discuss@lists.lustre.org> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org <http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org> ___ lustre-discuss mailing list lustre-discuss@lists.lustre.org<mailto:lustre-discuss@lists.lustre.org> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org Cheers, Andreas -- Andreas Dilger Lustre Principal Architect Whamcloud ___ lustre-discuss mailing list lustre-discuss@lists.lustre.org http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org

Re: [lustre-discuss] Error: GPG check FAILED when trying to install e2fsprogs

Sorry, those packages are not signed, you'll just have to install them without a signature. Cheers, Andreas > On Jan 3, 2024, at 09:10, Jan Andersen wrote: > > I have finally managed to build the lustre rpms, but when I try to install > them with: > > dnf install ./*.rpm > > I get a list

Re: [lustre-discuss] Building lustre on rocky 8.8 fails?

Try 2.15.4, as it may fix the EL8.8 build issue. Cheers, Andreas > On Jan 2, 2024, at 07:30, Jan Andersen wrote: > > I have installed Rocky 8.8 on a new server (Dell PowerEdge R640): > > [root@mds 4.18.0-513.9.1.el8_9.x86_64]# cat /etc/*release* > Rocky Linux release 8.8 (Green Obsidian) > N

Re: [lustre-discuss] Lustre server still try to recover the lnet reply to the depreciated clients

imilar messages > /var/log/messages:Dec 6 15:02:14 mds2 kernel: LNetError: > 3817248:0:(lib-move.c:4005:lnet_handle_recovery_reply()) peer NI > (10.67.176.25@tcp) recovery failed with -111 > > > Regards, > Qiulan > ___ > lustre-discuss m

Re: [lustre-discuss] Error messages (ex: not available for connect from 0@lo) on server boot with Lustre 2.15.3 and 2.15.4-RC1

shutdown clients cleanly well before the vlmf Lustre server is (also >> cleanly) shutdown. It is a sign of corruption ? How come this happen if >> shutdowns are clean ? >> >> Thanks (and sorry for the beginners questions), >> >> Martin >> >> Andreas

Re: [lustre-discuss] Lustre caching and NUMA nodes

ly 55 KB. Is this blocked by the list server? Cheers, Andreas -- Andreas Dilger Lustre Principal Architect Whamcloud ___ lustre-discuss mailing list lustre-discuss@lists.lustre.org http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org

Re: [lustre-discuss] Debian 11: configure fails

tre-discuss mailing list lustre-discuss@lists.lustre.org<mailto:lustre-discuss@lists.lustre.org> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org Cheers, Andreas -- Andreas Dilger Lustre Principal Architect Whamcloud ___ lustre-discu

Re: [lustre-discuss] Error messages (ex: not available for connect from 0@lo) on server boot with Lustre 2.15.3 and 2.15.4-RC1

It wasn't clear from your rail which message(s) are you concerned about? These look like normal mount message(s) to me. The "error" is pretty normal, it just means there were multiple services starting at once and one wasn't yet ready for the other. LustreError: 137-5: lustrevm-MDT000

Re: [lustre-discuss] OST is not mounting

The OST went read-only because that is what happens when the block device disappears underneath it. That is a behavior of ext4 and other local filesystems as well. If you look in the console logs you would see SCSI errors and the filesystem being remounted read-only. To have reliability in t

[lustre-discuss] Possible change to "lfs find -size" default units?

n the other hand, possibly this would be *less* confusing for users that are already used to the behavior of regular "find"? Cheers, Andreas -- Andreas Dilger Lustre Principal Architect Whamcloud ___ lustre-discuss mailing list lustr

Re: [lustre-discuss] Lustre-Manual on lfsck - non-existing entries?

econds average_speed_phase1: 6372 items/sec average_speed_phase2: 0 objs/sec real_time_speed_phase1: N/A real_time_speed_phase2: N/A current_position: N/A $ sudo ls /sys/kernel/debug/lustre/mdd/myth-MDT/ total 0 0 changelog_current_mask 0 changelog_users 0 lfsck_namespace

Re: [lustre-discuss] very slow mounts with OSS node down and peer discovery enabled

I can't comment on the LNet peer discovery part, but I would definitely not tecommend to leave the lnet_transaction_timeout that low for normal usage. This can cause messages to be dropped while the server is processing them and introduce failures needlessly. Cheers, Andreas > On Oct 26, 2023

Re: [lustre-discuss] re-registration of MDTs and OSTs

mmands for a particular MDT/OST or the "params" log for "set_param -P". The parameters can be restored from a file with "lctl set_param -F". See the lctl-set_param.8 and lctl-llog_print.8 man pages for details. Cheers, Andreas -- Andreas Dilger Lustre Princi

Re: [lustre-discuss] setting quotas from within a container

Hi Lisa, The first question to ask is which Lustre version you are using? Second, are you using subdirectory mounts or other UID/GID mapping for the container? That could happen at both the Lustre level or by the kernel itself. If you aren't sure, you could try creating a new file as root insid

Re: [lustre-discuss] mount not possible: "no server support"

ver support or similar Cheers, Andreas -- Andreas Dilger Lustre Principal Architect Whamcloud ___ lustre-discuss mailing list lustre-discuss@lists.lustre.org http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org

Re: [lustre-discuss] backup restore docs not quite accurate?

Removing the OI files is for ldiskfs backup/restore (eg. after tar/untar) when the inode numbers are changed. That is not needed for ZFS send/recv because the inode numbers stay the same after such an operation. If that isn't clear in the manual it should be fixed. Cheers, Andreas > On Oct 1

Re: [lustre-discuss] [EXTERNAL] [BULK] Re: Ongoing issues with quota

not > always, the blocks will have data in them. These are all zero-length. My > inclination is to see if I can delete them and be done with it, but I’m a bit > paranoid. > > — > Dan Szkola > FNAL > > > > > >> On Oct 17, 2023, at 4:23 PM, Andreas Di

Re: [lustre-discuss] [EXTERNAL] [BULK] Re: Ongoing issues with quota

e.org>> Reply-To: Daniel Szkola mailto:dszk...@fnal.gov> <mailto:dszk...@fnal.gov>> Date: Tuesday, October 10, 2023 at 2:30 PM To: Andreas Dilger mailto:adil...@whamcloud.com> <mailto:adil...@whamcloud.com>> Cc: lustre mailto:lustre-discuss@lists.lustre.org> <mailt

Re: [lustre-discuss] OSS on compute node

rformance due to the server contending with the client application. Cheers, Andreas -- Andreas Dilger Lustre Principal Architect Whamcloud ___ lustre-discuss mailing list lustre-discuss@lists.lustre.org http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org

Re: [lustre-discuss] Ongoing issues with quota

gt;>>> Am 09.10.2023 um 17:55 schrieb Daniel Szkola via lustre-discuss >>>> : >>>> >>>> Thanks, I will look into the ZFS quota since we are using ZFS for all >>>> storage, MDT and OSTs. >>>> >>>> In our case, there is a s

Re: [lustre-discuss] Ongoing issues with quota

The quota accounting is controlled by the backing filesystem of the OSTs and MDTs. For ldiskfs/ext4 you could run e2fsck to re-count all of the inode and block usage. For ZFS you would have to ask on the ZFS list to see if there is some way to re-count the quota usage. The "inode" quota is

Re: [lustre-discuss] OST went back in time: no(?) hardware issue

27;m not sure if the clients might try to preserve the next 55B RPCs in memory until the committed transno on the OST catches up, or if they just accept the new transno and get on with life? Cheers, Andreas -- Andreas Dilger Lustre Principal Architect Whamcloud _

Re: [lustre-discuss] Failing build of lustre client on Debian 12

ing the error and leaving it for the next person to fix. Cheers, Andreas -- Andreas Dilger Lustre Principal Architect Whamcloud ___ lustre-discuss mailing list lustre-discuss@lists.lustre.org http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org

Re: [lustre-discuss] Cannot mount MDT after upgrading from Lustre 2.12.6 to 2.15.3

.15.3 >>>>> >>>>> ***Attention*** This email originated from outside of the NRC. >>>>> ***Attention*** Ce courriel provient de l'extérieur du CNRC. >>>>> >>>>> Dear All, >>>>> >>>>&g

Re: [lustre-discuss] Adding lustre clients into the Debian

ild/test, or comments on specific lines in the patch. In some cases, Gerrit Janitor will also give negative code reviews in cases when a newly-added regression test added by a patch is failing regularly in its testing. Cheers, Andreas -- Andreas Dilger Lustre Principal Architect Whamclo

Re: [lustre-discuss] Adding lustre clients into the Debian

ific lines in the patch. In some cases, Gerrit Janitor will also give negative code reviews in cases when a newly-added regression test added by a patch is failing regularly in its testing. Cheers, Andreas -- Andreas Dilger Lustre Principal Architect Whamcloud __

Re: [lustre-discuss] Cannot mount MDT after upgrading from Lustre 2.12.6 to 2.15.3

;mailto:lustre-discuss@lists.lustre.org> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org Cheers, Andreas -- Andreas Dilger Lustre Principal Architect Whamcloud ___ lustre-discuss mailing list lustre-discuss@lists.lustre.org http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org

Re: [lustre-discuss] No port 988?

be started/stopped independently, even if they are running on the same server. Cheers, Andreas -- Andreas Dilger Lustre Principal Architect Whamcloud ___ lustre-discuss mailing list lustre-discuss@lists.lustre.org http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org

Re: [lustre-discuss] [BULK] Re: [EXTERNAL] Re: Data recovery with lost MDT data

ker, Darby J. (JSC-EG111)[Jacobs Technology, Inc.] via lustre-discuss" Reply-To: "Vicker, Darby J. (JSC-EG111)[Jacobs Technology, Inc.]" Date: Monday, September 25, 2023 at 8:56 AM To: Andreas Dilger Cc: "lustre-discuss@lists.lustre.org" Subject: Re: [lustre-discuss] [

Re: [lustre-discuss] [EXTERNAL EMAIL] Re: Lustre 2.15.3: patching the kernel fails

n ldiskfs, but can make it easier to use. That is up to you. Cheers, Andreas /jan On 21/09/2023 18:40, Andreas Dilger wrote: The first yes toon to ask is what is your end goal? If you just want to build only a client that is mounting to an existing server, then you can disable the server func

Re: [lustre-discuss] [EXTERNAL] Re: Data recovery with lost MDT data

foste2 mtfoste2 4096 Sep 19 11:35 mtfoste2/ 4 drwx-- 4 abeniniabenini 4096 Sep 19 15:33 abenini/ 4 drwx-- 9 pdetremp pdetremp 4096 Sep 19 16:49 pdetremp/ [dvicker@dvicker ~]$ From: Andreas Dilger mailt

Re: [lustre-discuss] Data recovery with lost MDT data

In the absence of backups, you could try LFSCK to link all of the orphan OST objects into .lustre/lost+found (see lctl-lfsck_start.8 man page for details). The data is still in the objects, and they should have UID/GID/PRJID assigned (if used) but they have no filenames. It would be up to you t

Re: [lustre-discuss] Lustre 2.15.3: patching the kernel fails

The first yes toon to ask is what is your end goal? If you just want to build only a client that is mounting to an existing server, then you can disable the server functionality: ./configure --disable-server and it should build fine. If you want to also build a server, and *really* want i

Re: [lustre-discuss] File size discrepancy on lustre

e, HPC Scientific Computing Group, Thomas Jefferson National Accelerator Facility ___ lustre-discuss mailing list lustre-discuss@lists.lustre.org<mailto:lustre-discuss@lists.lustre.org> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org Chee

Re: [lustre-discuss] Getting started with Lustre on RHEL 8.8

-agents [root@localhost ~]# On Wed, Sep 13, 2023 at 9:10 AM Cyberxstudio cxs mailto:cyberxstudio.cl...@gmail.com>> wrote: Thank you for the information. On Tue, Sep 12, 2023 at 8:40 PM Andreas Dilger mailto:adil...@whamcloud.com>> wrote: Hello, The preferred path to set up Lustre depends o

Re: [lustre-discuss] Getting started with Lustre on RHEL 8.8

Hello, The preferred path to set up Lustre depends on what you are planning to do with it? If for regular usage it is easiest to start with RPMs built for the distro from https://downloads.whamcloud.com/public/lustre/latest-release/

Re: [lustre-discuss] questions about group locks / LDLM_FL_NO_TIMEOUT flag

You can't directly dump the holders of a particular lock, but it is possible to dump the list of FIDs that each client has open. mds# lctl get_param mdt.*.exports.*.open_files | egrep "=|FID" | grep -B1 FID That should list all client NIDs that have FID open. It shouldn't be possible for cl

Re: [lustre-discuss] question about rename operation ?

Any directory renames where it is not just a simple name change (ie. parent directory is not the same for both source and target) the MDS thread doing the rename will take the LDLM "big filesystem lock" (BFL), which is a specific FID for global rename serialization. This ensures that there is o

Re: [lustre-discuss] getting without inodes

The t0 filesystem OSTs are formatted for an average file size of 70TB / 300M inodes = 240KB/inode. The t1 filesystem OSTs are formatted for an average file size of 500TB / 65M inodes = 7.7MB/inode. So not only are the t1 OSTs larger, but they have fewer inodes (by a factor of 32x). This must h

Re: [lustre-discuss] Pool_New Naming Error

tem name 'lustre' is not defined" or similar. A patch to fix this would be welcome. So your command should be: lctl pool_new lfs1.pool1 though I would suggest a more descriptive name than "pool1" (e.g. "flash" or "new_osts" or whatever),

Re: [lustre-discuss] how does lustre handle node failure

Shawn, Lustre handles the largest filesystems in the world, hundreds of PB in size, so there are definitely Lustre filesystems with hundreds of servers. In large storage clusters the servers failover in pairs or quads, since the storage is typically not on a single global SAN for all nodes to ac

Re: [lustre-discuss] File system global quota

Probably the closest that could be achieved like this would be to set the ldiskfs reserved space on the OSTs like: tune2fs -m 10 /dev/sdX That sets the root reserved space to 10% of the filesystem, and non-root users wouldn't be able to allocate blocks once the filesystem hits 90% full. This

Re: [lustre-discuss] Old Lustre Filesystem migrate to newer servers

nd advice will be highly appreciated. Thanks & regards, Richard. ___ lustre-discuss mailing list lustre-discuss@lists.lustre.org<mailto:lustre-discuss@lists.lustre.org> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org Chee

Re: [lustre-discuss] New client mounts fail after deactivating OSTs

Brian, Please file a ticket in LUDOC with details of how the manual should be updated. Ideally, including a patch. :-) Cheers, Andreas On Jul 11, 2023, at 15:39, Brad Merchant wrote:  We recreated the issue in a test cluster and it was definitely the llog_cancel steps that caused the issue.

Re: [lustre-discuss] Use of lazystatfs

ed to do anything with it anymore. The "lfs df" command will automatically skip unconfigured OSTs. Cheers, Andreas -- Andreas Dilger Lustre Principal Architect Whamcloud ___ lustre-discuss mailing list lustre-discuss@lists.lustre.org http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org

Re: [lustre-discuss] Rocky 9.2/lustre 2.15.3 client questions

Applying the LU-16626 patch locally should fix the issue, and has no risk since it is only fixing a build issue that affects an obscure diagnostic tool. That said, I've cherry-picked that patch back to b2_15, so it should be included into 2.15.4. https://review.whamcloud.com/51426 Cheers, Andr

Re: [lustre-discuss] CentOS Stream 8/9 support?

han EL kernels. [1] https://www.redhat.com/en/blog/furthering-evolution-centos-stream [2] https://wiki.whamcloud.com/display/PUB/Lustre+Support+Matrix Cheers, Will -- Dr Will Furnass | Research Platforms Engineer IT Services | University of Sheffield Cheers, Andreas -- Andreas Dilger Lustre Pri

Re: [lustre-discuss] No space left on device MDT DoM but not full nor run out of inodes

itter<http://twitter.com/CR_UK> [Description: CRI Logo]<http://www.cruk.cam.ac.uk/> ___ lustre-discuss mailing list lustre-discuss@lists.lustre.org<mailto:lustre-discuss@lists.lustre.org> http://lists.lustre.org/listinfo.cgi/lustre

Re: [lustre-discuss] Data stored in OST

Yes, the OSTs must provide internal redundancy - RAID-6 typically. There is File Level Redundancy (FLR = mirroring) possible in Lustre file layouts, but it is "unmanaged", so users or other system-level tools are required to resync FLR files if they are written after mirroring. Cheers, Andreas

Re: [lustre-discuss] mlx5 errors on oss

I can't comment on the specific network issue, but in general it is far better to use the MOFED drivers than the in-kernel ones. Cheers, Andreas > On May 18, 2023, at 09:08, Nehring, Shane R [LAS] via lustre-discuss > wrote: > > Hello all, > > We recently added infiniband to our cluster an

Re: [lustre-discuss] [EXTERNAL] Re: storing Lustre jobid in file xattrs: seeking feedback

> Sent: Friday, May 12, 2023 4:56 PM To: Andreas Dilger Cc: Bertschinger, Thomas Andrew Hjorth; lustre-discuss@lists.lustre.org<mailto:lustre-discuss@lists.lustre.org> Subject: [EXTERNAL] Re: [lustre-discuss] storing Lustre jobid in file xattrs: seeking feedback Just a thought, inst

Re: [lustre-discuss] storing Lustre jobid in file xattrs: seeking feedback

ave thoughts on how you could use this, please feel free to share them so that we design it in a way that meets your needs. Thanks! Tom Bertschinger LANL ___ lustre-discuss mailing list lustre-discuss@lists.lustre.org<mailto:lustre-discuss@lists.lust

Re: [lustre-discuss] Missing Files in /proc/fs/lustre after Upgrading to Lustre 2.15.X

change the path. The move of brw_stats to /sys/kernel/debug/lustre was mandated by the upstream kernel and only happened in 2.15.0. Cheers, Andreas -- Andreas Dilger Lustre Principal Architect Whamcloud ___ lustre-discuss mailing list lustre-discuss

Re: [lustre-discuss] question mark when listing file after the upgrade

This looks like https://jira.whamcloud.com/browse/LU-16655 causing problems after the upgrade from 2.12.x to 2.15.[012] breaking the Object Index files. A patch for this has already been landed to b2_15 and will be included in 2.15.3. If you've hit this issue, then you need to backup/delete the

Re: [lustre-discuss] Recovering MDT failure

there will not be any filenames associated with the files. Regards, Andreas -- Andreas Dilger Lustre Principal Architect Whamcloud ___ lustre-discuss mailing list lustre-discuss@lists.lustre.org http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org

Re: [lustre-discuss] [EXTERNAL] Mounting lustre on block device

061 4170 Morena Boulevard, Suite C - San Diego, CA 92117 High-Performance Computing / Lustre Filesystems / Scale-out Storage ___ lustre-discuss mailing list lustre-discuss@lists.lustre.org<mailto:lustre-discuss@lists.lustre.org> ht

Re: [lustre-discuss] Joining files

the necessary layout manipulation be possible in userspace? (I will > have a look into the implementations of `lfs migrate` and `lfs mirror > extend`). > > Thanks a lot! > Best, > Sven > > On Wed, Mar 29, 2023 at 07:41:56PM +, Andreas Dilger wrote: > [-- Type:

  1   2   3   4   5   6   7   8   9   10   >