Re: [lustre-discuss] Joining files

puting Lab (SCLab) Max Planck Institute for Meteorology Bundesstraße 53, D-20146 Hamburg, Germany ___ lustre-discuss mailing list lustre-discuss@lists.lustre.org<mailto:lustre-discuss@lists.lustre.org> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org Cheers, Andreas -- Andreas Dilger Lustre Principal Architect Whamcloud ___ lustre-discuss mailing list lustre-discuss@lists.lustre.org http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org

Re: [lustre-discuss] About Lustre small files performace(8k) improve

lustre-discuss mailing list lustre-discuss@lists.lustre.org<mailto:lustre-discuss@lists.lustre.org> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org Cheers, Andreas -- Andreas Dilger Lustre Principal Architect Whamcloud _

Re: [lustre-discuss] DNE v3 and directory inode changing

untar and then build/process files in that tree). That should help significantly with genomics and machine learning workloads that have this kind of usage pattern. Cheers, Andreas -- Andreas Dilger Lustre Principal Architect Whamcloud ___ lustre

Re: [lustre-discuss] DNE v3 and directory inode changing

.org<mailto:lustre-discuss@lists.lustre.org> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org Cheers, Andreas -- Andreas Dilger Lustre Principal Architect Whamcloud ___ lustre-discuss mailing list lustre-discuss@lists.lustre.org http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org

Re: [lustre-discuss] Lustre project quotas and project IDs

l DB, to keep track of the latest added ID, so that we could increment the highest value by 1 on new ID creation. The highest value could as well be looked up in: /proc/fs/lustre/osd-ldiskfs/myfs-MDT/quota_slave_dt/acct_project Regards, Marco Passerini F

Re: [lustre-discuss] Repeated ZFS panics on MDT

It's been a while since I've worked with ZFS servers, but one old chestnut that caused problems with ZFS 0.7 on the MDTs was the variable dnode size feature. I believe there was a tunable, something like "dnodesize=auto" that caused problems, and this could be changed to "dnodesize=1024" or sim

Re: [lustre-discuss] Lustre project quotas and project IDs

se e.g. PROJID=UID for user directories, PROJID=1M + UID for scratch, and PROJID=2M+N for independent projects, just to make the PROJIDs easily identified (at least until someone implements LU-13335 to do projid<->name mapping). How many IDs were you thinking of using? Cheers, Andre

Re: [lustre-discuss] Node Failure in Lustre

No, because the remote-attached SSDs are part of the ZFS pool and any drive failures a t that level are the responsibility of ZFS in that case to manage the failed drives (eg. with RAID) and for you to have system monitors in place to detect this case and alert you to the drive failures. This i

Re: [lustre-discuss] Slow Lustre traffic failover issue

s mostly used by native Lustre clients. Cheers, Andreas -- Andreas Dilger Lustre Principal Architect Whamcloud ___ lustre-discuss mailing list lustre-discuss@lists.lustre.org http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org

Re: [lustre-discuss] Renaming or Moving directories on Lustre?

quot;rename()" after the first EXDEV return, it creates the target directory and then tries to rename the files within the source directory to the target, before it does the file copy. It is likely that ext4 could also be patched to allow regular file renames without returning EXDEV. Che

Re: [lustre-discuss] Question about lustre deduplication?

Cheers, Andreas -- Andreas Dilger Lustre Principal Architect Whamcloud ___ lustre-discuss mailing list lustre-discuss@lists.lustre.org http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org

Re: [lustre-discuss] Access times for file (file heat)

plans to work further on this feature? I think of several use cases when knowing these stats. Cold data could be moved to archive like slow tape without relying on access time. Hot blocks could be replicated or moved to faster caches and lot more optimizations. Best regards Anna Am 18.02.20

Re: [lustre-discuss] lfs setstripe with stripe_count=0

stripe_offset: -1 pool: hdd-pool pfe24.jbauer2 1228> ___ lustre-discuss mailing list lustre-discuss@lists.lustre.org<mailto:lustre-discuss@lists.lustre.org> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org Cheers, Andreas -- Andreas Dilger Lustre Principal Arch

Re: [lustre-discuss] Access times for file (file heat)

Anna, there was a client-side file heat mechanism added a few years ago, but I don't know if it is fully functional today. lctl get_param llite.*.*heat* llite.myth-979380fc1800.file_heat=1 llite.myth-979380fc1800.heat_decay_percentage=80 llite.myth-979380fc1800.heat_period_second=60

Re: [lustre-discuss] Full List of Required Open Lustre Ports?

list lustre-discuss@lists.lustre.org http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org Cheers, Andreas -- Andreas Dilger Lustre Principal Architect Whamcloud ___ lustre-discuss mailing list lustre-discuss@lists.lustre.org http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org

Re: [lustre-discuss] Mistake while removing an OST

eed to keep. Regards, Martin From: Andreas Dilger mailto:adil...@whamcloud.com>> Sent: Wednesday, February 1, 2023 18:16 To: BALVERS Martin mailto:martin.balv...@danone.com>> Cc: lustre-discuss@lists.lustre.org<mailto:lustre-discuss@lists.lustre.org> Subject: Re: [lustre-discuss]

Re: [lustre-discuss] Mistake while removing an OST

You should just be able to run the "writeconf" process to regenerate the config logs. The removed OST will not re-register with the MGS, but all of the other servers will, so it should be fine. Cheers, Andreas On Feb 1, 2023, at 03:48, BALVERS Martin via lustre-discuss wrote:  Hi, I have a

Re: [lustre-discuss] Monitoring Lustre IOPS on OSTs

Yes, each RPC will increment these stats counters by one. Traditional "IOPS" are measured with 4KB read or write, but in this case the IO sizes are variable. Also, the client may aggregate multiple disjoint writes into a single RPC. This can be seen in the osd-ldiskfs.*.brw_stats as "discontiguo

Re: [lustre-discuss] User find out OST configuration

rom the file, rather than printing it out and then parsing the text again in userspace. Cheers, Andreas Am 21.01.2023 um 17:08 schrieb Andreas Dilger: Hi Anna, Beyond the number and size of OSTs and MDTs there isn't much information about the underlying storage available on the client. The &

Re: [lustre-discuss] User find out OST configuration

Hi Anna, Beyond the number and size of OSTs and MDTs there isn't much information about the underlying storage available on the client. The "lfs df -v" command will print a "f" at the end for flash (non-rotational) devices, if the storage is properly configured. The "osc*.imports " parameter f

Re: [lustre-discuss] Struggling with OSS mounts after a crash

You need to run writeconf on all targets at the same time, and mount in a specific order. That is documented in th Lustre Operations Manual. Cheers, Andreas On Jan 18, 2023, at 03:49, Edmondson, Edward via lustre-discuss wrote:  Hi all, I'm struggling to get my OSS mounts online after a les

Re: [lustre-discuss] Lustre Client

That would seem to be a Postgres problem and not Lustre? Cheers, Andreas On Jan 13, 2023, at 05:01, Nick dan via lustre-discuss wrote:  Hi Thank you for your help I am using postgres with lustre client, when I mount lustre client with -o ro the postgres service is not starting Can you help

Re: [lustre-discuss] Regarding Lustre with RDMA

low. We want to mount using the lustre filesystem and not ext4. Is there a need to change the lnet configuration? What else is need to be done? Cheers, Andreas -- Andreas Dilger Lustre Principal Architect Whamcloud ___ lustre-discuss mailing lis

Re: [lustre-discuss] Regarding Lustre with RDMA

stre can have network bandwidth comparable to locally attached NVMe devices, and can also scale far larger than directly-attached storage would allow. Cheers, Andreas -- Andreas Dilger Lustre Principal Architect Whamcloud ___ lustre-discuss mailing list

Re: [lustre-discuss] Project quota and project quota accounting

You should confirm that the "project" feature is enabled on all of the OSTs and MDTs with "dumpe2fs -h /dev/XXX" and checking the "features" line. Cheers, Andreas On Nov 29, 2022, at 20:29, David Cohen via lustre-discuss wrote:  Hi, We are running Lustre 2.12.7 (ldiskfs) both on the servers

Re: [lustre-discuss] liblustreapi.so llapi_layout_get_by_fd() taking a long time to complete

tre.lov", buf, buflen); llapi_layout_get_by_xattr(buf, buflen, 0); but then we wouldn't know what is making this slow and you couldn't submit a patch to fix it... Cheers, Andreas -- Andreas Dilger Lustre Principal Architect Whamcloud __

Re: [lustre-discuss] Restrict who can assign OST pools to directories

need something similar to what can be done for remote directories: lctl set_param mdt.*.enable_remote_dir_gid=1 Regards, Marco Passerini Cheers, Andreas -- Andreas Dilger Lustre Principal Architect Whamcloud ___ lustre-discuss mailing list lustr

Re: [lustre-discuss] l_getidentity

rn off l_getidentity writing to /var/log/secure. Our /var file-system keeps on running out of space due to this. Had a look at debugging parameters but have not found anything. Thanks Francois Cheers, Andreas -- Andreas Dilger Lustre Principal Architect Wha

Re: [lustre-discuss] Lustre recycle bin

No 1951/09/06. ___ lustre-discuss mailing list lustre-discuss@lists.lustre.org<mailto:lustre-discuss@lists.lustre.org> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org Cheers, Andreas -- Andreas Dilger Lustre Princi

Re: [lustre-discuss] Building Best Practices Guide

-What does the minimum configuration look like? Example Minimum MDS/MDT, OSS/OST, what does that look like from a minimum storage capacity perspective? Few people care about *minimum* limits. It is possible to format Lustre with a single 32MB MDT and 32MB OST, and mount both and a client on a

Re: [lustre-discuss] Lustre and ZFS ZIL

erformance. Our lustre OST underlying FS is ZFS and all disks are HDDs. I am planing to add NVMe ZIL device. Our Lustre version is 2.12 Some links says Lustre does not support ZIL. What does it mean by “support”? Cant ZFS use ZIL with Lustre? Best Regards, Cheers, Andreas -- Andreas Dilger

Re: [lustre-discuss] [Samba] Odd "File exists" behavior when copy-pasting many files to an SMB exported Lustre FS

written by kernel afs.* skip# AFS metadata and ACLs lustre.*skip Having Samba honor the xattr.conf (even if it is not using libattr) would at least make the behavior consistent between Samba and cp and other tools. Cheers, Andreas -- Andreas Dilger Lustre Principal Architect Whamcloud ___ lustre-discuss mailing list lustre-discuss@lists.lustre.org http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org

Re: [lustre-discuss] fio and lustre performance

=0 plot3 : https://www.dropbox.com/s/vk23vmufa388l7h/plot2.png?dl=0 ___ lustre-discuss mailing list lustre-discuss@lists.lustre.org<mailto:lustre-discuss@lists.lustre.org> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org Cheers,

Re: [lustre-discuss] lproc stats changed snapshot_time from unix-epoch to uptime/monotonic in 2.15

Ellis, thanks for reporting this. This looks like it was a mistake. The timestamps should definitely be in wallclock time, but this looks to have been changed unintentionally to reduce overhead, and use a u64 instead of dealing with timespec64 math, while losing the original intent (there are

Re: [lustre-discuss] fiemap, final chapter.

rastructure. * * Some portions copyright (C) 2007 Cluster File Systems, Inc * * Authors: Mark Fasheh <mailto:mfas...@suse.com> * Kalpak Shah <mailto:kalpak.s...@sun.com> * Andreas Dilger <mailto:adil...@sun.com> */ #ifndef _LINUX_FIEMAP_H #define _LINUX_F

Re: [lustre-discuss] fiemap

h filefrag /usr/sbin/filefrag John On 8/18/22 14:57, Andreas Dilger wrote: What version of Lustre are you using? Does "filefrag -v" from a newer Lustre e2fsprogs (1.45.6.wc3+) work properly? There was a small change to the Lustre FIEMAP handling in order to handle overstriped files

Re: [lustre-discuss] fiemap

-discuss mailing list lustre-discuss@lists.lustre.org<mailto:lustre-discuss@lists.lustre.org> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org Cheers, Andreas -- Andreas Dilger Lustre Principal Architect Whamcloud ___ lustre-discuss

Re: [lustre-discuss] Invalid jobid size

/listinfo.cgi/lustre-discuss-lustre.org Cheers, Andreas -- Andreas Dilger Lustre Principal Architect Whamcloud ___ lustre-discuss mailing list lustre-discuss@lists.lustre.org http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org

Re: [lustre-discuss] Changing default recovery window time settings

The maximum amount of time that recovery will run is controlled by "at_max". The default is 600s (10 mins), but on my 2-client home cluster (with a relatively light load) the recovery is usually finished in 10s or less. You can reduce the timeout based on what is your typical time. Note that t

Re: [lustre-discuss] llapi_layout_file_comp_del

raight forward for llapi to work with an fd than a pathname if a valid fd already exists. Am I missing an easier way to do this? Thanks, John On 7/27/22 16:25, Andreas Dilger wrote: The HLD document was written before the feature was implemented, and is outdated. The lustreapi

Re: [lustre-discuss] A project quota question

| 16-bit user), assuming UIDs can fit in 16 bits. Otherwise, assign an 8-bit PI and 24-bit UID or similar, or just allocate multiple 32-bit PROJIDs to the few users that are in multiple PI directories. Cheers, Andreas -- Andreas Dilger Lustre Principal Architect Whamcloud _

Re: [lustre-discuss] llapi_layout_file_comp_del

ss@lists.lustre.org> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org Cheers, Andreas -- Andreas Dilger Lustre Principal Architect Whamcloud ___ lustre-discuss mailing list lustre-discuss@lists.lustre.org http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org

Re: [lustre-discuss] How to speed up Lustre

pool name). Note that using a too-large PFL layout (more than 3-4 components) can be counter-productive as it may cause the layout for even small files to spill into an external xattr block and consume extra space and IOPS on the MDT. Cheers, Andreas Cheers Thomas On 7/6/22 21:42, Andreas Dilg

Re: [lustre-discuss] How to speed up Lustre

chtsrats: State Secretary / Staatssekretär Dr. Volkmar Dietz _______ lustre-discuss mailing list lustre-discuss@lists.lustre.org<mailto:lustre-discuss@lists.lustre.org> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org Cheers, Andreas -- Andreas Dilger Lustre Principal Architect Whamcloud ___ lustre-discuss mailing list lustre-discuss@lists.lustre.org http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org

Re: [lustre-discuss] Max Single OSS throughput is not crossing 7 GB/s Reads

-r -t ${xfersize} -b ${blksize} -O summaryFile=./${TESTDIR}/iorresult_SeqWrite_p${numthreads}_bs${blksize}_tf${xfersize}.json,summaryFormat=JSON please let me know what is the bottleneck in the above setup ? Also if you need more info , will be provide it right away . Cheers, Andreas -- Andreas

Re: [lustre-discuss] Installing 2.15 on rhel 8.5 fails

telists > > libselinux-devel libtool > > > > > > >> On 6/22/22 21:08, Jian Yu wrote: >> Hi Thomas, >> The issue is being fixed in https://jira.whamcloud.com/browse/LU-15962. >> A workaround is to build Lustre with "--with-o2ib=&quo

Re: [lustre-discuss] Help with recovery of data

mailing list lustre-discuss@lists.lustre.org<mailto:lustre-discuss@lists.lustre.org> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org Cheers, Andreas -- Andreas Dilger Lustre Principal Architect Whamcloud ___ lustre-discuss mailing list lustre-discuss@lists.lustre.org http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org

Re: [lustre-discuss] Installing 2.15 on rhel 8.5 fails

oot - no option either. Any hints how to proceed? The ko2iblnd module is built against the in-kernel OFED, so if you are using MOFED you will need to rebuild the kernel modules themselves. If you don't use IB at all you can ignore these depmod messages. Cheers, Andreas -- Andr

Re: [lustre-discuss] llapi documentation

-OST0082_UUID poollist_get() poolNames[7]=nbp17.4_ssd members[7]=nbp17-OST0083_UUID Pool: nbp17.hdd-pool poollist_get() poolNames[8]=nbp17.hdd-pool -75 members buffer=nbp17-OST_UUID Pool: nbp17.ssd-pool poollist_get() poolNames[9]=nbp17.ssd-pool -75 members buffer=nbp17-OST0064_UUID John On 6/1

Re: [lustre-discuss] llapi documentation

ome. The pool related functions should probably be moved into a new liblustreapi_pool.c file to reduce the size of liblustreapi.c. Cheers, Andreas -- Andreas Dilger Lustre Principal Architect Whamcloud ___ lustre-discuss mailing list lustr

Re: [lustre-discuss] need info regarding TCP ports for lustre

ame fabric/subnet. The Lustre-level MDS_CONNECT and OST_CONNECT are at a higher level and use "portal" numbers, which are totally different. Cheers, Andreas > On Jun 13, 2022, at 23:27, Åke Sandgren wrote: > >  > >> On 6/14/22 00:51, Andreas Dilger via lustre-di

Re: [lustre-discuss] need info regarding TCP ports for lustre

dropped, then server->client connections may be initiated to cancel a lock or similar. If this server->client connection cannot be established, then the client may be evicted. Cheers, Andreas -- Andreas Dilger Lustre Principal Architect Whamcloud ___

Re: [lustre-discuss] Removing stale files

5-835-7281 (BACK IN THE OFFICE!) Cell: 575-517-5668 (out of work hours) ___ lustre-discuss mailing list lustre-discuss@lists.lustre.org<mailto:lustre-discuss@lists.lustre.org> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustr

Re: [lustre-discuss] Misplaced position for two glibc checks

reateIssue!default.jspa (or just click the "Create" button at the top of https://jira.whamcloud.com/). Details of how to push patches to Gerrit for review and testing are at: https://wiki.lustre.org/Using_Gerrit Cheers, Andreas -- Andreas Dilger Lustre Principal Architect Whamcloud __

Re: [lustre-discuss] lustre_rsync with growing statuslog

t to totally discourage your usage of lustre_rsync since it has potential for further improvements (e.g. parallel copying, bug fixing, etc), otherwise it will never get better, but just wanted to make sure you know what the current state of this tool. Cheers, Andreas --

Re: [lustre-discuss] Avoiding system cache when using ssd pfl extent

wildcard "*" if all of the OSTs/MDTs are flash based. If you have a hybrid NVMe/HDD system, you can explicitly select a subset of OST/MDT devices to disable the caches. Cheers, Andreas On May 20, 2022, at 02:49, Åke Sandgren mailto:ake.sandg...@hpc2n.umu.se>> wrote: On 5/20/22

Re: [lustre-discuss] Avoiding system cache when using ssd pfl extent

_ >> lustre-discuss mailing list >> lustre-discuss@lists.lustre.org >> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org > ___ > lustre-discuss mailing list > lustre-discuss@lists.lustre.org > h

Re: [lustre-discuss] FLR Mirroring for read performance

class of storage, so they use either the "prefer" flag set on the flash mirror, or with LU-14996 it also checks the OS_STATFS_NONROT flag from the OSTs (if this is reported, check "lfs df -v" for the 'f' (flash) flag). Cheers, Andreas -- Andreas Dilger Lustre P

Re: [lustre-discuss] Anyone know why lustre-zfs-dkms-2.12.8_6_g5457c37-1.el7.noarch.rpm won't install?

ling list lustre-discuss@lists.lustre.org<mailto:lustre-discuss@lists.lustre.org> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org Cheers, Andreas -- Andreas Dilger Lustre Principal Architect Whamcloud ___ lustre-discuss mailing list l

Re: [lustre-discuss] Corrupted? MDT not mounting

isks are mounted over SRP from a DDN appliance. Would jumping to MOFED make a difference? Otherwise I'm open to suggestions as it's getting very tiring wrangling servers back to life MOFED is usually preferred over in-kernel OFED, it is just tested and fixed a lot more. Cheers, Andrea

Re: [lustre-discuss] Essential tools for Lustre

:40, Raj wrote:  Andreas, Is there any IO penalties in enabling project quota? Will I see the same throughput from the FS? Thanks -Raj On Fri, Apr 15, 2022 at 1:32 PM Andreas Dilger via lustre-discuss mailto:lustre-discuss@lists.lustre.org>> wrote: Note that in newer Lustre releases, if yo

Re: [lustre-discuss] Poor(?) Lustre performance

d Regards, Finn On Wed, 20 Apr 2022 at 09:24, Andreas Dilger mailto:adil...@whamcloud.com>> wrote: On Apr 16, 2022, at 22:51, Finn Rawles Malliagh via lustre-discuss mailto:lustre-discuss@lists.lustre.org>> wrote: Hi all, I have just set up a three-node Lustre configuration, and init

Re: [lustre-discuss] Poor(?) Lustre performance

y the performance of the local three disk ZFS is more performant than the lustre FS. I'm very new to this kind of benchmarking so it may also be that I am misinterpreting the results/ not applying the test correctly. Cheers, Andreas -- Andreas Dilger Lustre Principal Architect Whamcloud

Re: [lustre-discuss] Essential tools for Lustre

_ lustre-discuss mailing list lustre-discuss@lists.lustre.org<mailto:lustre-discuss@lists.lustre.org> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org Cheers, Andreas -- Andreas Dilger Lustre Principal Architect Whamcloud _

Re: [lustre-discuss] Cleanup of RPC-related Statistics at "import" Parameter on OSC Device

t? If yes, what is the command to do so? If not, then if I periodically reset the counters by modifying the Lustre source code, what kind of impact will it have for the client or the file system itself? Yes, you can clear the client stats like "lctl set_param osc.*.stats=0". Cheers, An

Re: [lustre-discuss] question regarding du vs df on lustre

Lustre is returning the file unlink from the MDS immediately, but deleting the objects from the OSTs asynchronously in the background. How many files are being deleted in this case? If you are running tests like IO500, where there are many millions of small files plus some huge files, then it ma

Re: [lustre-discuss] Target index choice

as it may unnecessarily put you on the bleeding edge of finding new bugs (or at least using more memory or less efficient processing than necessary in some places). Cheers, Andreas -- Andreas Dilger Lustre Principal Architect Whamcloud ___ lustre-disc

Re: [lustre-discuss] [EXTERNAL] Re: Write Performance is Abnormal for max_dirty_mb Value of 2047

__ lustre-discuss mailing list lustre-discuss@lists.lustre.org<mailto:lustre-discuss@lists.lustre.org> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org Cheers, Andreas -- Andreas Dilger Lustre Principal Architect Whamcloud __

Re: [lustre-discuss] Stripe Size for small file and OST query

s-lustre.org Cheers, Andreas -- Andreas Dilger Lustre Principal Architect Whamcloud ___ lustre-discuss mailing list lustre-discuss@lists.lustre.org http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org

Re: [lustre-discuss] Problem with standard fortran IO and lustre fs, sporadically slow io rate

, or Darshan if a parallel program, or other related tools, and you can get stats about IO size and alignment without having to understand the code first. This may surprise you if the application is doing e.g. 32-byte writes because you specified the array variables in the wrong order, or something.

Re: [lustre-discuss] Lustre Client Lockup Under Buffered I/O (2.14/2.15)

clients. Are you able to test with more RAM on the client? Have you tried with 2.12.8 installed on the client? Cheers, Andreas -- Andreas Dilger Lustre Principal Architect Whamcloud ___ lustre-discuss mailing list lustre-discuss@lists.lu

Re: [lustre-discuss] Jobstats Support with Singularity Container

r with 2.12 and the clients with 2.13? Best, Gabriele From: lustre-discuss mailto:lustre-discuss-boun...@lists.lustre.org>> on behalf of Iannetti, Gabriele mailto:g.ianne...@gsi.de>> Sent: Tuesday, December 14, 2021 11:14 To: Andreas Di

Re: [lustre-discuss] Jobstats Support with Singularity Container

See the Lustre Operations Manual for options setting the JobID. You can set it using fields like "%u" for UID, or you can set it per process group, or for the whole node. For containers, you could set it for the process group when it starts and it should be inherited by all processes in the con

Re: [lustre-discuss] Patched vs patchless server (again)

but that is unlikely for most users. Cheers, Andreas -- Andreas Dilger Lustre Principal Architect Whamcloud ___ lustre-discuss mailing list lustre-discuss@lists.lustre.org http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org

Re: [lustre-discuss] Lustre and server upgrade

ss-lustre.org ___________ lustre-discuss mailing list lustre-discuss@lists.lustre.org<mailto:lustre-discuss@lists.lustre.org> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org Cheers, Andreas -- Andreas Dilger Lustre Principal Architect Whamcloud ___ lustre-discuss mailing list lustre-discuss@lists.lustre.org http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org

Re: [lustre-discuss] OST "D" status - only 1 OSS mounting

10c8f483800 active. home-OST0002-osc-a10c8f483800 active. home-OST0003-osc-a10c8f483800 active. Should be 191TB... only shows 1 OST.. 10.140.93.42@o2ib:/home 48T 48T 414G 100% /home Where should I look? Cheers, Andreas -- Andreas Dilger

Re: [lustre-discuss] SLUB: Unable to allocate memory on node -1

.lustre.org<mailto:lustre-discuss@lists.lustre.org> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org Cheers, Andreas -- Andreas Dilger Lustre Principal Architect Whamcloud ___ lustre-discuss mailing list lustre-discuss@lists.

Re: [lustre-discuss] [EXTERNAL] Re: Fwd: RPCs in Flight are more than the max_rpcs_in_flight value

would be really helpful for me and highly appreciated if you can share your intuitions or reasons regarding why such improvement may have been observed. Thanks, Md. Hasanur Rashid On Thu, Oct 7, 2021 at 5:45 PM Andreas Dilger mailto:adil...@whamcloud.com>> wrote: [Caution: Email from

Re: [lustre-discuss] No read throughput shown for the sequential read write Filebench workload

I would guess that all of the reads are handled from the client page cache? Cheers, Andreas On Oct 13, 2021, at 06:37, Md Hasanur Rashid via lustre-discuss wrote:  Hello Everyone, I am running a Filebench workload which is provided below: define fileset name="testF",entries=100,filesize=1

Re: [lustre-discuss] dkms-2.8.6 breaks installation of lustre-zfs-dkms-2.12.7-1.el7.noarch

Riccardo, It would be great if you could submit your patch to Gerrit. Cheers, Andreas > On Oct 13, 2021, at 17:06, Riccardo Veraldi > wrote: > > yes, same problem for me, I Addressed this a few weeks go and I think I > Reported to the mailing list. > > This is my patch to make things works

Re: [lustre-discuss] Fwd: RPCs in Flight are more than the max_rpcs_in_flight value

say _why_ this off-by-one error is of interest? Definitely it seems like a bug that could be fixed, but it doesn't seem too critical to correct functionality. Cheers, Andreas -- Andreas Dilger Lustre Principal Architect Whamcloud ___

Re: [lustre-discuss] Question about max service threads

[mdt_rdpg00_003] Could I verify my assumption by counting the number of process mdt\d\d_\d*? Best regards, Houkun On 22. Sep 2021, at 21:21, Andreas Dilger mailto:adil...@whamcloud.com>> wrote: What version of Lustre are you running? I tested with 2.14.0 and observed that *.*.threads

Re: [lustre-discuss] Question about max service threads

I’ve always used ps, grep, and wc -l to answer that question :) From: lustre-discuss mailto:lustre-discuss-boun...@lists.lustre.org>> on behalf of Andreas Dilger via lustre-discuss mailto:lustre-discuss@lists.lustre.org>> Sent: Tuesday, September 2

Re: [lustre-discuss] Question about max service threads

Hello Houkun, There was patch https://review.whamcloud.com/34400 "LU-947 ptlrpc: allow stopping threads above threads_max" landed for the 2.13 release. You could apply this patch to your 2.12 release, or test with 2.14.0. Note that this patch only lazil

Re: [lustre-discuss] lustre file system installation issues

stem? Kind regards Nagmat ___ lustre-discuss mailing list lustre-discuss@lists.lustre.org<mailto:lustre-discuss@lists.lustre.org> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org Cheers, Andreas -- Andreas Dilger Lust

Re: [lustre-discuss] lru_size question

sized LRU the default is to use 100x core count on each node. The MGS is always static, since it doesn't need many locks, and less effort to manage LRU size. Cheers, Andreas -- Andreas Dilger Lustre Principal Architect Whamcloud ___ lustre-dis

Re: [lustre-discuss] Disabling multi-rail dynamic discovery

3520/t0(0) >>>> o400->scratch-MDT-mdc-98b0f1fc0800@192.52.98.31@tcp:12/10 lens >>>> 224/224 e 0 to 1 dl 1630699598 ref 2 fl Rpc:XN/0/ rc 0/-1 >>>> [ 739.832755] Lustre: 3526:0:(client.c:2146:ptlrpc_expire_one_request()) >>>> Skipped 5 pr

Re: [lustre-discuss] Disabling multi-rail dynamic discovery

tp%3A%2F%2Flists.lustre.org%2Flistinfo.cgi%2Flustre-discuss-lustre.org&data=04%7C01%7Cdarby.vicker-1%40nasa.gov%7C8943d25ba8254c75fded08d97795ee11%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637672308371242046%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVC

Re: [lustre-discuss] Full OST

tat" command above will also print the object creation time along with the normal timestamps. Is it safe to simply remove all these files, and then remount etc? How can we ensure that new files will be deleted from the OST in the future? If they are not referenced by any in-use file (per

Re: [lustre-discuss] Full OST

then check lost+found and/or a regular "find /mnt/ost -type f -size +1M" or similar to find where the files are. Cheers, Andreas -- Andreas Dilger Lustre Principal Architect Whamcloud ___ lustre-discuss mailing list lustre-dis

Re: [lustre-discuss] Full OST

> None of these files are on the dodgy OST. > > Any further suggestions? Essentially we have what seems to be a full OST > with nothing on it. > > Thanks, > Alastair. > >> On Sat, 4 Sep 2021, Andreas Dilger wrote: >> >> [EXTERNAL EMAIL] &

Re: [lustre-discuss] Full OST

7;s a problem of open files. Not sure which bit of this I need to use with lfs fid2path either... Cheers, Alastair. On Fri, 3 Sep 2021, Andreas Dilger wrote: [EXTERNAL EMAIL] You can also check "mdt.*.exports.*.open_files" on the MDTs for a list of FIDs open on each client, and use &quo

Re: [lustre-discuss] Full OST

:lustre-discuss@lists.lustre.org> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org Cheers, Andreas -- Andreas Dilger Lustre Principal Architect Whamcloud ___ lustre-discuss mailing list lustre-discuss@lists.lustre.org http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org

Re: [lustre-discuss] trimming flash-based external journal device

On Aug 5, 2021, at 17:44, Nathan Dauchy - NOAA Affiliate via lustre-discuss mailto:lustre-discuss@lists.lustre.org>> wrote: On Thu, Aug 5, 2021 at 3:23 PM Andreas Dilger mailto:adil...@whamcloud.com>> wrote: On Aug 5, 2021, at 13:29, Nathan Dauchy wrote: Andreas, thanks as alw

Re: [lustre-discuss] trimming flash-based external journal device

On Aug 5, 2021, at 13:29, Nathan Dauchy - NOAA Affiliate mailto:nathan.dau...@noaa.gov>> wrote: Andreas, thanks as always for your insight. Comments inline... On Thu, Aug 5, 2021 at 10:48 AM Andreas Dilger mailto:adil...@whamcloud.com>> wrote: On Aug 5, 2021, at 09:28, Natha

Re: [lustre-discuss] trimming flash-based external journal device

iskfs filesystem, reformat it as ext4 and mount locally, and then run benchmarks (e.g. "dd" would best match the JBD2 workload, or fio if you want random IOPS) against it. You could do this before/after trim (could use fstrim at this point) to see if it affects the

Re: [lustre-discuss] lustre client kernel compatibility

), and the sources are portable across a wide range of kernel versions. Cheers, Andreas -- Andreas Dilger Lustre Principal Architect Whamcloud ___ lustre-discuss mailing list lustre-discuss@lists.lustre.org http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org

Re: [lustre-discuss] Hardware advice for homelab

1MB/s, and backup of family laptops). I have a beefier x86 client (i5-6600 + 48GB RAM), formerly an Intel Atom, but that couldn't handle HD video decoding. I was also running a 32-bit Raspberry Pi 3 for a while, but it was very marginal for HD video and died after a year o

Re: [lustre-discuss] Unable to mount new OST

dex=81" (decimal). When trying to mount the with: mount.lustre /dev/mapper/OST0051 /Lustre/OST0051 The system stays on 100% CPU (one core) forever and the mount never completes, not even after a week. I tried tunefs.lustre --writeconf --erase-params on the MDS and all the other targets, but th

Re: [lustre-discuss] What's your favorite distributed filesystem benchmark?

datasets? Thanks, Vinayak From: Andreas Dilger Date: Monday, June 28, 2021 at 4:23 PM To: "Vinayak.Kamath" Cc: "lustre-discuss@lists.lustre.org" Subject: [EXTERNAL] Re: [lustre-discuss] What's your favorite distributed filesystem benchmark? On Jun 28, 2021, at

<    1   2   3   4   5   6   7   8   9   10   >