Re: [lustre-discuss] What's your favorite distributed filesystem benchmark?

OPS measurements, and testing non-POSIX IO interfaces like libaio, io_uring, mmap, etc. Cheers, Andreas -- Andreas Dilger Lustre Principal Architect Whamcloud ___ lustre-discuss mailing list lustre-discuss@lists.lustre.org http://lists.lustre.or

Re: [lustre-discuss] MDT filling up

ndreas -- Andreas Dilger Lustre Principal Architect Whamcloud ___ lustre-discuss mailing list lustre-discuss@lists.lustre.org http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org

Re: [lustre-discuss] kernel 5.10 on RHEL8 and lustre-client kind of working

g/listinfo.cgi/lustre-discuss-lustre.org Cheers, Andreas -- Andreas Dilger Principal Lustre Architect Whamcloud ___ lustre-discuss mailing list lustre-discuss@lists.lustre.org http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org

Re: [lustre-discuss] good ways to identify clients causing problems?

IML, lltop, Graphana that need agents installed, central monitoring, etc.), but having a simple "check load on the local node like 'top'" tool would still be helpful. Cheers, Andreas -- Andreas Dilger Principal Lustre Architect Whamcloud __

Re: [lustre-discuss] OST -> MDT migration and MDT -> OST migration

On Apr 15, 2021, at 00:52, Åke Sandgren wrote: > >  > >> On 4/15/21 5:12 AM, Andreas Dilger via lustre-discuss wrote: >> On Apr 14, 2021, at 18:42, Bill Anderson via lustre-discuss >> > <mailto:lustre-discuss@lists.lustre.org>> wrote: >>> >

Re: [lustre-discuss] OST -> MDT migration and MDT -> OST migration

T objects. However, OST-to-DoM migration isn't available until 2.13. Cheers, Andreas -- Andreas Dilger Principal Lustre Architect Whamcloud ___ lustre-discuss mailing list lustre-discuss@lists.lustre.org http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org

Re: [lustre-discuss] Experience with DDN AI400X

ng my own experience. Smile. Cheers, megan ___ lustre-discuss mailing list lustre-discuss@lists.lustre.org<mailto:lustre-discuss@lists.lustre.org> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org Cheers, Andreas -- Andrea

Re: [lustre-discuss] Viewing project quotas

On Feb 24, 2021, at 10:45, Peeples, Heath via lustre-discuss wrote: > > I am looking for a way to see all project quotas and their associated > directories for a file system.Is there an easy way to do that? Thanks > for the help. As yet, the "lfs project" command does not have support fo

Re: [lustre-discuss] Determine MDT object min, max and average size

Could you please explain further what you want? Typically there is no "MDT object size" since data is not usually stored on the MDT. Cheers, Andreas > On Feb 19, 2021, at 10:06, Sudheendra Sampath > wrote: > >  > Hi, > > I am trying to gather the MDT object size. Can someone please point

Re: [lustre-discuss] Lustre MDT with ZFS backend

ted if anyone could give us further suggestions. Cheers, Andreas -- Andreas Dilger Principal Lustre Architect Whamcloud ___ lustre-discuss mailing list lustre-discuss@lists.lustre.org http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org

Re: [lustre-discuss] Improving file create performance with larger create_count

tunable, which can help optimize OST creates if there is a lot of lock contention on the object directories for create vs. concurrent IO, but it is unlikely to be an issue under normal usage. If the problem is that the OSTs have huge numbers of objects and large object directories there are other potenti

Re: [lustre-discuss] Tuning for metadata performance

Probably the best single thing you could do for metadata performance would be to switch to SSD, or better NVMe, storage. ZFS is very sync and IOPS hungry, so using HDDs is killer for ZFS metadata performance. If you want to minimize the downtime, you could incrementally replace the HDDs in the zp

Re: [lustre-discuss] Kernel/OS/Lustre version matching

Note also that 2.12.6 was just released and it shows the EL8.2 kernel in the ChangeLog, among many others: https://git.whamcloud.com/?p=fs/lustre-release.git;a=blob;f=lustre/ChangeLog;h=beefe94e2199eaa571f7a796c8219384fa8364c0;hb=refs/heads/b2_12 4.18.0-147.3.1.el8 (RHEL8.1) 4

Re: [lustre-discuss] small typo in Lustre manual?

, Andreas On Nov 13, 2020, at 2:24 PM, Patrick Shopbell wrote: > > > Hi Andreas, > Sure thing - I can do that. > > I was mostly checking to make sure I was correct and not missing > something stupid. > > Thanks, > Patrick > > > On 11/13/20 1:22 PM, And

Re: [lustre-discuss] small typo in Lustre manual?

The best thing to do is file a LUDOC ticket at https://jira.whamcloud.com/ and push a fix patch against the doc/lustre-manual repo at https://review.whamcloud.com/ so the minor error can be fixed. Cheers, Andreas > On Nov 13, 2020, at 11:31, Patrick Shopbell wrote: > >  > Hello all, > Trivi

Re: [lustre-discuss] Cannot set ost.OSS.ost.thread_started

On Nov 3, 2020, at 9:25 AM, Youbiao He wrote: > I have installed lustre file system on CentOS. It works well when I write or > read files. But when I try to set the thread_started and thread_min for oss > services, I met the following problem. It is always important to include the Lustre releas

Re: [lustre-discuss] ZFS atime is it required?

es all access times? You don't really need atime enabled on the OSTs, but I also don't think "atime=off" will make any difference. That is a VFS/ZPL level option, and Lustre osd-zfs doesn't use any of the ZPL code, but rather handles atime internally. Cheers, Andr

Re: [lustre-discuss] Hidden QoS in Lustre ?

On Oct 8, 2020, at 10:37 AM, Tung-Han Hsieh wrote: > > Dear All, > > In the past months, we encountered several times of Lustre I/O abnormally > slowing down. It is quite mysterious that there seems no problem on the > network hardware, nor the lustre itself since there is no error message > at

Re: [lustre-discuss] Lustre optimize for spares data files ?

On Sep 8, 2020, at 9:13 PM, Tung-Han Hsieh wrote: > > I would like to ask whether Lustre file system has implemented the > function to optimize for large sparse data files ? > > For example, a 3GB data file but with more than 80% bytes zero, can > Lustre file system optimize the storage not act

Re: [lustre-discuss] Lustre 2.12 routing with MR and discovery off

On Aug 26, 2020, at 4:37 PM, Faaland, Olaf P. wrote: > > Does Lustre 2.12 require that routes for every intermediate network are > defined, on every node on a path? > > For example, given this Lustre network, where: > A-D are nodes and 1-6 are addresses > network tcp2 has only routers, no cli

Re: [lustre-discuss] Lustre 2.12 routing with MR and discovery off

ate with their peers, then _that_ should be sent back to A/D to indicate they cannot route packets to the target NID, but I wouldn't think A/D should get information about @tcp2 themselves? Cheers, Andreas -- Andreas Dilger Principal Lustre Architect Whamcloud ___

Re: [lustre-discuss] some clients dmesg filled up with "dirty page discard"

em becoming inconsistent. Cheers, Andreas -- Andreas Dilger Principal Lustre Architect Whamcloud ___ lustre-discuss mailing list lustre-discuss@lists.lustre.org http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org

Re: [lustre-discuss] import_set_state_nolock() with binary args in lctl debug_file output?

On Aug 14, 2020, at 3:44 PM, Sternberg, Michael G. wrote: > >  > In lctl debug_file output, for import_set_state_nolock(), I sometimes see > binary arguments (sample snippet at end of post), and figure that's not a > good sign. How can I get to the bottom of this? > > The only direct referenc

Re: [lustre-discuss] Can not mount ZFS-based device

On Aug 4, 2020, at 4:57 AM, yangshengwang2011 wrote: > > Hi, > > > I can not mount a ZFS-based device when install the lustre servers. > > > Information in syslog is, > > # kernel:osd_zfs: Unknown symbol zfs_refcount_add (0) > > # kernel:LustreError:158-c: Can't load module 'osd-zfs' > >

Re: [lustre-discuss] How to convert fids like /O/0/d19/115 to actual data ?

On Jul 9, 2020, at 3:52 AM, Zeeshan Ali Shah wrote: > > Dear All , > On zfs based lustre we are getting following > pool: ost2-xag > state: ONLINE > status: One or more devices has experienced an error resulting in data > corruption. Applications may be affected. > action: Restore the file in q

Re: [lustre-discuss] MGS+MDT migration to a new storage using LVM tools

On Jul 19, 2020, at 12:41 AM, David Cohen wrote: > > Hi, > We have a combined MGS+MDT and I'm looking for a migration to new storage > with a minimal disruption to the running jobs on the cluster. > > Can anyone find problems in the scenario below and/or suggest another > solution? > I would a

Re: [lustre-discuss] systemd lnet/rdma conflict

Rick, would you be able to put this in the form of a patch against lustre/scripts/systemd/lnet.service so that this is working well for everyone. You could use LU-9673 for this. > On Jul 16, 2020, at 2:34 PM, Mohr Jr, Richard Frank wrote: >> On Jul 16, 2020, at 2:46 PM, Christopher Benjamin C

Re: [lustre-discuss] Is there aceiling of lustre filesystem a client can mount

On Jul 15, 2020, at 8:39 PM, 肖正刚 wrote: > > Hi, Jongwoo & Andreas > > Sorry for the ambiguous description. > What I want to know is the number of lustre filesystems that a client can > mount on the same time. The number of filesystems a client can mount depends on how much RAM it has. I don

Re: [lustre-discuss] Is there aceiling of lustre filesystem a client can mount

On Jul 15, 2020, at 12:29 AM, 肖正刚 wrote: > > Hi, all > Is there a ceiling for a Lustre filesystem that can be mounted in a cluster? > If so, what's the number? > If not, how much is proper? > Does mount multiple filesystems can affect the stability of each file system > or cause other problems?

Re: [lustre-discuss] Can we re-index the lustre-discuss archive DB?

On Jul 15, 2020, at 6:07 PM, Cameron Harr wrote: > > To the person with the power, > > I've been trying to search the lustre-discuss > (http://lists.lustre.org/pipermail/lustre-discuss-lustre.org/) archives but > it seems only old (<= 2013 perhaps) messages are searchable with the "Search" >

Re: [lustre-discuss] Yahoo OpenID not working to log into gerrit.

Hi Arshad, I'm at least able to login to Gerrit, but I'm not using Yahoo for the authentication. Is it possible that Yahoo discontinued the OpenID login? That previously happened with Gmail accounts, which is why Gerrit no longer allows authentication with Gmail OpenID. > On Jul 4, 2020, at 11:

Re: [lustre-discuss] mlx4 and mxl5 mix environment

s.lustre.org<mailto:lustre-discuss@lists.lustre.org> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org Cheers, Andreas -- Andreas Dilger Principal Lustre Architect Whamcloud ___ lustre-discuss mailing list lustre-discuss@lists.lustre.org http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org

Re: [lustre-discuss] Questions about LU-13645

are), so upgrading to 2.12.5 doesn't affect this either way. Cheers, Andreas -- Andreas Dilger Principal Lustre Architect Whamcloud ___ lustre-discuss mailing list lustre-discuss@lists.lustre.org http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org

Re: [lustre-discuss] Permission denied on lfs getstripe

s02 ~]# l_getidentity -d 131364 uid=131364 gid=131364,130808,130817,131016,131237,249837,249945,249996 permissions: nid perm The relevant gid for the target directory is 130817. I verified that all 3 of our MDSs had the same view before and after the “ls”. Thanks; Chris From: Andreas

Re: [lustre-discuss] Permission denied on lfs getstripe

, Ph.D. Computational Scientist National Renewable Energy Laboratory 15013 Denver West Pkwy., MS ESIF301 Golden, CO 80401 ___ lustre-discuss mailing list lustre-discuss@lists.lustre.org<mailto:lustre-discuss@lists.lustre.org> http://lists.lustre.org

Re: [lustre-discuss] mlx4 and mxl5 mix environment

On Jun 22, 2020, at 2:13 AM, 肖正刚 wrote: > We setup up a cluster use mlx4 and mlx5 driver mixed，all things goes well. > Later I find something in wiki > http://wiki.lustre.org/Infiniband_Configuration_Howto and > http://lists.onebuilding.org/pipermail/lustre-devel-lustre.org/2016-May/003842.html

Re: [lustre-discuss] Client 2.12.5 on 4.18.0-193.6.3.el8_2.x86_64 does not load

On Jun 22, 2020, at 6:02 AM, Torsten Harenberg wrote: > > Dear all, > > due to the attacks to HPC centers, we were advised to update the kernels > to the newest version available. > > It seems that the Lustre 2.12.5 client does not load on the very recent > CentOS 8 kernel anymore: > > [root@

Re: [lustre-discuss] Do old clients ever go away?

for a short time. This doesn't mean that these clients are actively connected or part of the filesystem. Cheers, Andreas -- Andreas Dilger Principal Lustre Architect Whamcloud ___ lustre-discuss mailing list lustre-discuss@lists.lustre.org ht

Re: [lustre-discuss] compiling Lustre from source

ustre.org> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org Cheers, Andreas -- Andreas Dilger Principal Lustre Architect Whamcloud ___ lustre-discuss mailing list lustre-discuss@lists.lustre.org http://lists.lustre.org/listinf

Re: [lustre-discuss] how to mapping of RPC rate to bandwidth/IOPS?

lients will be able to get at most 40MB/s (assuming applications do relatively linear IO). If applications have small random IOPS then rpcrate=10 may get up to 256 4KB writes per RPC, or about 2560 IOPS = 10MB/s. Cheers, Andreas -- Andreas Dilger Principal Lustre Architec

Re: [lustre-discuss] patchless server vs. patched server

Pascal, Thanks for taking the time to update the wiki, every contribution helps. Note that the need for patches kernels for project quotas will also go away with newer kernels, but Red Hat just couldn't make that feature work with the RHEL7 kernel without breaking the ABI. Cheers, Andreas On Jun

[lustre-discuss] Fwd: IO500 ISC20 Call for Submission

> From: committee--- via IO-500 > Subject: [IO-500] IO500 ISC20 Call for Submission > Date: May 22, 2020 at 1:48:26 PM MDT > To: io-...@vi4io.org > Reply-To: commit...@io500.org > > Deadline: 08 June 2020 AoE > > The IO500 is now accepting and encouraging submissions > for t

Re: [lustre-discuss] NFS Client Attributes caching - equivalent feature/config in Lustre

anks, pinkesh valdria ___ lustre-discuss mailing list lustre-discuss@lists.lustre.org<mailto:lustre-discuss@lists.lustre.org> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org Cheers, Andreas -- Andreas Dilger Principal

Re: [lustre-discuss] confused about mdt space

o 1024 bytes in order to fit the more complex PFL layouts. That increased the total amount of space per inode from 2048 bytes to 2560 bytes. I guess the manual needs to be updated. Cheers, Andreas -- Andreas Dilger Principal Lustre Architect Whamcloud _

Re: [lustre-discuss] OST recovery

s there anyway I can rewrite the CONFIGS/mountdata back to its original values? Sorry for the lengthy messages and really appreciate your help! Best Regards, Grover On Mon, Mar 30, 2020 at 7:14 AM Andreas Dilger mailto:adil...@whamcloud.com>> wrote: It would be useful if you provided the

[lustre-discuss] CFP: First International CHAOSS International Workshop

s and databases - Optimized indexing techniques - Data organizations to support online workflows - Domain-specific data management solutions - Related experiences from users: what worked, what didn't? ## Program Committee - Gabriel Antoniu (INRIA) - Konstantinos Chasapis (DDN) - Andreas Dilger

Re: [lustre-discuss] OST recovery

lustre-discuss mailing list lustre-discuss@lists.lustre.org<mailto:lustre-discuss@lists.lustre.org> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org Cheers, Andreas -- Andreas Dilger Principal Lustre Architect Whamcloud __

Re: [lustre-discuss] DNE2 settings are not propagated?

ll directories to be striped, as that adds overhead without necessary improving performance. The current recommendation is that DNE should be used for e.g. a top-level directory to distribute files and subdirectories across MDTs, or in the case of large directories with millions of files. Cheer

Re: [lustre-discuss] "no space on device"

s mailing list lustre-discuss@lists.lustre.org<mailto:lustre-discuss@lists.lustre.org> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org Cheers, Andreas -- Andreas Dilger Principal Lustre Architect Whamcloud ___ lustre-discuss mailing list lustre-discuss@lists.lustre.org http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org

Re: [lustre-discuss] old Lustre 2.8.0 panic'ing continously

tre-discuss@lists.lustre.org<mailto:lustre-discuss@lists.lustre.org> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org Cheers, Andreas -- Andreas Dilger Principal Lustre Architect Whamcloud ___ lustre-discuss mailing list lustre-

Re: [lustre-discuss] old Lustre 2.8.0 panic'ing continously

ta tables are correct. Since the time taken by "tune2fs -O quota" is almost the same as running e2fsck, it is better to do it the right way. Cheers, Andreas -- Andreas Dilger Principal Lustre Architect Whamcloud ___ lustre-dis

Re: [lustre-discuss] project quota totals incorrect

ancy, you could verify that the "project" feature is on (dumpe2fs -h), and/or try running e2fsck on all the OSTs/MDTs to ensure that the files are properly accounted. Cheers, Andreas -- Andreas Dilger Principal Lustre Architect Whamcloud _

Re: [lustre-discuss] Lustre as VM backend

uch better flash performance than with 2.10 (there have been a few presentations about this recently). Cheers, Andreas -- Andreas Dilger Principal Lustre Architect Whamcloud ___ lustre-discuss mailing list lustre-discuss@lists.lustre.org http://

Re: [lustre-discuss] Jobstats harvesting

You don't mention which Lustre release you are using, but newer releases allow "complex JobIDs" that can contain both the SLURMJobID as well as other constant strings (e.g. cluster name), hostname, UID, GID, and process name. This is documented in the Lustre manual at: http://doc.lustre.org/lustr

Re: [lustre-discuss] Lustre 2.12.3 client can't mount filesystem

gt; http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org Cheers, Andreas -- Andreas Dilger Principal Lustre Architect Whamcloud ___ lustre-discuss mailing list lustre-discuss@lists.lustre.org http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org

Re: [lustre-discuss] pcc?

Cheers, Andreas -- Andreas Dilger Principal Lustre Architect Whamcloud ___ lustre-discuss mailing list lustre-discuss@lists.lustre.org http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org

Re: [lustre-discuss] Slow release of inodes on OST

o500 and doing nothing else. > > I set > osp.rsos-OST-osc-MDT.max_rpcs_in_flight=512 > osp.rsos-OST-osc-MDT.max_rpcs_in_progress=32768 > which severely reduced my waiting time between runs. > The in_progress being the one that actually affected things. > >>

Re: [lustre-discuss] Slow release of inodes on OST

ues. So is there any other tunable I can tweak on either OSS or MDS side? On 2/6/20 6:58 AM, Andreas Dilger wrote: On Feb 4, 2020, at 07:23, Åke Sandgren mailto:ake.sandg...@hpc2n.umu.se> <mailto:ake.sandg...@hpc2n.umu.se>> wrote: When I create a large number of files on an OST an

Re: [lustre-discuss] Slow release of inodes on OST

g osp.*.max_rpcs_in_flight on the MDS might speed this up? It should default to 32 per OST on the MDS vs. default 8 for clients Cheers, Andreas -- Andreas Dilger Principal Lustre Architect Whamcloud ___ lustre-discuss mailing list lustre-dis

Re: [lustre-discuss] Nodemap and multi-tenancy

ns want. You _might_ be able to use project quotas to handle this within the nodemap, but it isn't clear what you want to do in the end. Cheers, Andreas -- Andreas Dilger Principal Lustre Architect Whamcloud ___ lustre-discuss mailing list lu

Re: [lustre-discuss] mv / rename not working across directory boundary ("Invalid cross device link")

.lustre.org<mailto:lustre-discuss@lists.lustre.org> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org Cheers, Andreas -- Andreas Dilger Principal Lustre Architect Whamcloud ___ lustre-discuss mailing list lustre-discuss@lists.lustre.org http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org

Re: [lustre-discuss] Nodemap, ssk and mutiple fileset from one client

The main reason is that clients are identified for the nodemap by their NID, and optionally verified by crypto (Kerberos or SSK). That makes it difficult to separate two different mounts from the same client. It would potentially be possible to have the primary client identification be done by

Re: [lustre-discuss] Read performance bad, telepathy in Lustre

uss@lists.lustre.org> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org Cheers, Andreas -- Andreas Dilger Principal Lustre Architect Whamcloud ___ lustre-discuss mailing list lustre-discuss@lists.lustre.org http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org

Re: [lustre-discuss] [Lwg] SC19 bof slides

iscuss@lists.lustre.org<mailto:lustre-discuss@lists.lustre.org> http://lists.lustre.org/li<http://ustre.org/li>stinfo.cgi/lustre-discuss-lustre.org ___ lwg mailing list l...@lists.opensfs.org<

Re: [lustre-discuss] Lustre and Optane

s for any observations. Cheers, Dave -- ** Dave Holland ** Systems Support -- Informatics Systems Group ** ** 01223 496923 **Wellcome Sanger Institute, Hinxton, UK** Cheers, Andreas -- Andreas Dilger Principal Lustre Architect Whamcloud

Re: [lustre-discuss] reflecting state of underlying store in Lustre with HSM

lustre.org ___ lustre-discuss mailing list lustre-discuss@lists.lustre.org<mailto:lustre-discuss@lists.lustre.org> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org Cheers, Andreas -- Andreas Dilger Principal Lustre Architect Whamcloud

Re: [lustre-discuss] Lnet Self Test

The first thing to note is that lst reports results in binary units (MiB/s) while iperf reports results in decimal units (Gbps). If you do the conversion you get 2055.31 MiB/s = 2155 MB/s. The other thing to check is the CPU usage. For TCP the CPU usage can be high. You should try RoCE+o2iblnd in

Re: [lustre-discuss] one ost down

If the HDD has enough bad sectors that it is reporting errors to user space then it means that all of the remapping sectors are already consumed will typically continue to have more errors in the future. It should be replaced rather than continuing to be used. I would agree with Marek that mak

Re: [lustre-discuss] changing inode size on MDT

ONLINE 0 0 0 sdx ONLINE 0 0 0 sdy ONLINE 0 0 0 thanks Michael From: Andreas Dilger mailto:adil...@whamcloud.com>> Sent: Monday, November 11, 2019 14:42 To: Hebenstreit, Michael mailto:michael.hebenstr...@intel.com>>

Re: [lustre-discuss] changing inode size on MDT

dt # rpm -q -a | grep zfs libzfs2-0.7.9-1.el7.x86_64 lustre-osd-zfs-mount-2.10.8-1.el7.x86_64 lustre-zfs-dkms-2.10.8-1.el7.noarch zfs-0.7.9-1.el7.x86_64 zfs-dkms-0.7.9-1.el7.noarch From: Andreas Dilger mailto:adil...@whamcloud.com>> Sent: Thursday, October 03, 2019 20:38 To: Heb

Re: [lustre-discuss] Lustre client/server versions/compatibility

t and lfsck can clean things up for you, but because of this issue those old clients don’t seem to be used anywhere in production along with modern servers and there may be other problems to worry about. -Cory -- On 10/17/19, 12:12 AM, "lustre-discuss on behalf of Andreas Dilger" mailt

Re: [lustre-discuss] Lustre client/server versions/compatibility

pps in VMs/containers and use a newer kernel+Lustre underneath. Cheers, Andreas -- Andreas Dilger Principal Lustre Architect Whamcloud ___ lustre-discuss mailing list lustre-discuss@lists.lustre.org http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org

Re: [lustre-discuss] Limit to number of OSS?

On Oct 10, 2019, at 11:20, Michael Di Domenico mailto:mdidomeni...@gmail.com>> wrote: On Mon, Oct 7, 2019 at 6:33 PM Andreas Dilger mailto:adil...@whamcloud.com>> wrote: With socklnd there are 3 TCP connections per client-server pair. For IB there is no such connection limit that

Re: [lustre-discuss] Limit to number of OSS?

t; wrote: Thanks for this info. But actually I was really looking at the number of OSS, not OSTs :) This is really more how Lustre client nodes and MDT will cope with very large number of OSSes. De : Andreas Dilger mailto:adil...@whamcloud.com>> Date : vendredi 4 octobre 2019 à 04:

Re: [lustre-discuss] Lustre rpm install creating a file that breaks lustre

ows lnet to start properly. Cheers, Andreas -- Andreas Dilger Principal Lustre Architect Whamcloud ___ lustre-discuss mailing list lustre-discuss@lists.lustre.org http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org

Re: [lustre-discuss] Limit to number of OSS?

could be overcome with some changes. Given OSTs are starting to approach 1PB with large drives and declustered-parity RAID, this would get us in the range 8-65EB, which is over 2^64 bytes (16EB), so I don't think it is an immediate concern. Let me know if you have any trouble with a 9000-

Re: [lustre-discuss] changing inode size on MDT

MDT to the filesystem to increase the number of inodes available. Cheers, Andreas From: Andreas Dilger mailto:adil...@whamcloud.com>> Sent: Thursday, October 03, 2019 19:38 To: Hebenstreit, Michael mailto:michael.hebenstr...@intel.com>> Cc: Mohr Jr, Richard Frank mailto:rm...@utk.e

Re: [lustre-discuss] changing inode size on MDT

ode for files stored on the OST). If you use Data-on-MDT and only have a small files, then no OST object is needed for small files, but you consume correspondingly more space on the MDT. Cheers, Andreas From: Andreas Dilger mailto:adil...@whamcloud.com>> Sent: Wednes

Re: [lustre-discuss] changing inode size on MDT

or 1.5KB/inode, this can be an arbitrary number of bytes, it doesn't have to be an even multiple of anything) but it definitely isn't possible to have 1KB inode size and 1KB per inode ratio, as there wouldn't be *any* space left for directories, log files, journal,

Re: [lustre-discuss] Do not recreate OST objects on OST replacement

sure that if the MDS has previously allocated those objects there will be objects available for the clients to write to them. LFSCK can be used to clean up these orphan objects if they are not in use. Cheers, Andreas -- Andreas Dilger Principal Lustre Arc

Re: [lustre-discuss] find xdev?

On Sep 11, 2019, at 10:06, Michael Di Domenico mailto:mdidomeni...@gmail.com>> wrote: On Tue, Sep 10, 2019 at 5:48 PM Andreas Dilger mailto:adil...@whamcloud.com>> wrote: I don't think "lfs find -xdev" has never been a priority for Lustre, since it is rare f

Re: [lustre-discuss] find xdev?

uid, gid) then "lfs find" can be significantly faster (e.g 3-5x0. If it is uses file size, then they will be about the same unless there are other MDT-only parameters, or once LSOM support is landed (hopefully 2.13). Cheers, Andreas -- Andreas Dilger Principal Lustre Architect Whamc

Re: [lustre-discuss] Replacing ldiskfs MDT with larger disk

aster enough that we could weather the downtime. > > PFL and FLR aren't features they're planning for the file system and it may > be replaced next year so I suspect they'll opt for the DNE method. > > Thanks again, > Jesse Stroik > > On 7/31/19 3:11 PM

Re: [lustre-discuss] Replacing ldiskfs MDT with larger disk

Normally the easy answer would be that a "dd" copy of the MDT device from your HDDs to a larger SSD LUN, then resize2fs to increase the filesystem size would also increase the number of inodes proportionately to the LUN size. However, since you are *not* using 1024-byte inode size, only 512-byte

Re: [lustre-discuss] mdt: unhealthy - healthy

, somewhat later, lxmds20:~# cat /sys/fs/lustre/health_check healthy and all Lustre operations seem to be good, too. This means that some RPC has been stuck, but if the RPC eventually completes then there is no reason for the MDS to be "unhealthy" anymore. Cheers, Andreas -- Andre

Re: [lustre-discuss] Error in lfsck: "NOT IMPLEMETED YET"

If you are trying to delete MDT then that is definitely not implemented yet... Cheers, Andreas On Jul 22, 2019, at 16:08, João Carlos Mendes Luís mailto:jo...@corp.globo.com>> wrote: Hi, I'm running some lab tests with lustre 2.12.2 in Oracle Linux Server release 7.6. Last test I d

Re: [lustre-discuss] Lustre v2.12.3 Availability

erner, Andrew T" > wrote: >> >> What is the outlook for v2.12.3 availability? The release roadmap shows >> something around Q3 ’19. I’d like a more definitive target if possible. >> Thanks. Cheers, Andreas -- Andreas Dilger Principal Lustre Architect Whamcloud

Re: [lustre-discuss] A question about lctl lfsck

where I can get a list of what it did? Cheers, Andreas -- Andreas Dilger Principal Lustre Architect Whamcloud ___ lustre-discuss mailing list lustre-discuss@lists.lustre.org http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org

Re: [lustre-discuss] lfsck namespace doesn't stop and I cancel it

Fernando, please file an LUDOC ticket at https://jira.whamcloud.com/ to add this to the manual. Even better would be to submit a patch to add this to the manual (see http://wiki.lustre.org/Lustre_Manual_Changes for details). Cheers, Andreas On Jun 26, 2019, at 14:07, Fernando Pérez mailto:fpe.

Re: [lustre-discuss] Lustre metrics

documentation that gives a description of each metric and > what it represents? > > Thanks, > > Joe > ___ > lustre-discuss mailing list > lustre-discuss@lists.lustre.org > http://lists.lustre.org/listinfo.cgi/lustre-discuss-l

Re: [lustre-discuss] a question about max_create_count

t. >> >>I looked at that value on my system and the default seems to be 2, am >> I correct in thinking that this is the maximum number of simultaneous >> creates that can happen on an OST? >> > Cheers, Andreas -- Andreas Dilger Principal Lu

Re: [lustre-discuss] Unable to compile client in Debian

my kernel version 4.19.0. > > What can I do? Any help will be highly appreciated. > > Alejandro A. Sierra > National Earth Observation Laboratory, Mexico > http://www.lanot.unam.mx/ > ___ > lustre-discuss mailing list > lustre-discuss@lists.lustre.org > http://lists.

Re: [lustre-discuss] Lustre snapshots

ain fs and its > snapshot fs' are so separated that putting the mappings in the client > /proc structures of the main fs would become ugly. > > We will just communicate client mount name through another channel. > > Cheers, > Hans Henrik > > On 30/05/2019 1

Re: [lustre-discuss] Lustre snapshots

names like "last_month", "last_week", "yesterday", "6h_ago" and such and not have to update the automount map. The filesystem names could be mostly irrelevant if the snapshot mountpoints are chosen properly, like "

Re: [lustre-discuss] Lustre under autofs

t. If you are running a newer version, please file a ticket in Jira with details (/var/log/messages, Lustre kernel debug log when there is a problem). Cheers, Andreas -- Andreas Dilger Principal Lustre Architect Whamcloud ___ lustre-discuss mailing

Re: [lustre-discuss] lfs migrate of hard links

w handled more elegantly. If I were to upgrade the lustre > client to 2.12.1 on a dozen clients, would the new client side lfs_migrate in > 2.12.1 work with 2.10.3 servers, or will I need a system wide outage to > upgrade all clients and servers? > > Cheers! > S

Re: [lustre-discuss] FLR mirroring on 2.12.1-1

x10001:0x280a8:0x0] } >> >> lcme_id: 131074 >> lcme_mirror_id: 2 >> lcme_flags: init >> lcme_extent.e_start: 0 >> lcme_extent.e_end: EOF >> lmm_stripe_count: 1 >> lmm_stripe_size: 1048576 >>

Re: [lustre-discuss] Lustre kernel module compatibility

lly tested and is not recommended for use. > I’d really like to get all Lustre code to the 2.12 version but I’m not sure > that I can convince our cluster administrator of that. Thanks. Cheers, Andreas -- Andreas Dilger Principal Lustre Architect Whamcloud ___

Re: [lustre-discuss] Unable to mount client with 56 MDSes and beyond

gt; Scientist, HPC > Los Alamos National Laboratory > > _______ > lustre-discuss mailing list > lustre-discuss@lists.lustre.org > http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org Cheers, Andreas -- Andreas Dilger Principal Lustre A

Re: [lustre-discuss] Setting infinite grace period with soft quotas

7; doesn't have such a statement. > But, certainly, 2^48 sec. would be sufficient! Our workaround was just > to use a -t XXXw to specify a large number of weeks (largest granularity > I could see) for the grace period, but having a '-1' shortcut would be > preferable. &g

<    1   2   3   4   5   6   7   8   9   10   >