Hi Mark, All nodes are using ZFS. OSTs, MDT, and MGT are all ZFS-based, so there's really no way to fsck them. I could do a scrub, but that's not the same thing. Is there a Lustre/ZFS equivalent of 'tune2fs -O [^]quota' for ZFS?
I'm guessing that at some point, a large number of files was removed and somehow quota accounting missed this. There should be a simple way to reconcile or regenerate what quota has recorded vs what is actually on disk, which I have verified two different ways. -- Dan On Wed, 2023-10-04 at 15:01 +0100, Mark Dixon wrote: > Hi Dan, > > I think it gets corrected when you umount and fsck the OST's themselves > (not lfsck). At least I recall seeing such messages when fsck'ing on 2.12. > > Best, > > Mark > > On Wed, 4 Oct 2023, Daniel Szkola via lustre-discuss wrote: > > > [EXTERNAL EMAIL] > > > > No combination of lfsck runs has helped with this. > > > > Again, robinhood shows 1796104 files for the group, an 'lfs find -G gid' > > found 1796104 files as well. > > > > So why is the quota command showing over 3 million inodes used? > > > > There must be a way to force it to recount or clear all stale quota data > > and have it regenerate it? > > > > Anyone? > > > > — > > Dan Szkola > > FNAL > > > > > > > On Sep 27, 2023, at 9:42 AM, Daniel Szkola via lustre-discuss > > > <lustre-discuss@lists.lustre.org> wrote: > > > > > > We have a lustre filesystem that we just upgraded to 2.15.3, however > > > this problem has been going on for some time. > > > > > > The quota command shows this: > > > > > > Disk quotas for grp somegroup (gid 9544): > > > Filesystem used quota limit grace files quota > > > limit grace > > > /lustre1 13.38T 40T 45T - 3136761* 2621440 > > > 3670016 expired > > > > > > The group is not using nearly that many files. We have robinhood > > > installed and it show this: > > > > > > Using config file '/etc/robinhood.d/lustre1.conf'. > > > group, type, count, volume, spc_used, avg_size > > > somegroup, symlink, 59071, 5.12 MB, 103.16 MB, 91 > > > somegroup, dir, 426619, 5.24 GB, 5.24 GB, 12.87 KB > > > somegroup, file, 1310414, 16.24 TB, 13.37 TB, 13.00 MB > > > > > > Total: 1796104 entries, volume: 17866508365925 bytes (16.25 TB), space > > > used: 14704924899840 bytes (13.37 TB) > > > > > > Any ideas what is wrong here? > > > > > > — > > > Dan Szkola > > > FNAL > > > _______________________________________________ > > > lustre-discuss mailing list > > > lustre-discuss@lists.lustre.org > > > https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.lustre.org_listinfo.cgi_lustre-2Ddiscuss-2Dlustre.org&d=DwIGaQ&c=gRgGjJ3BkIsb5y6s49QqsA&r=e9DXjyTaQ786Tg7WH7oIVaQOA1YDRqyxHOUaYU2_LQw&m=Nk1MkSBTpT-KnrXzEvOOP5tZoVAKyHfPvB-o8_OhewuwHF6S0KelH_WPMLq8IRnR&s=JzAV0C2_CqaDUOG0wZr0mx5tiblBde6ZRUuIHZ2n9DI&e= > > > > _______________________________________________ > > lustre-discuss mailing list > > lustre-discuss@lists.lustre.org > > https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.lustre.org_listinfo.cgi_lustre-2Ddiscuss-2Dlustre.org&d=DwIDaQ&c=gRgGjJ3BkIsb5y6s49QqsA&r=e9DXjyTaQ786Tg7WH7oIVaQOA1YDRqyxHOUaYU2_LQw&m=BBVt50ufoqbL64MfSKVa87fK1B4Q0n91KVNJVmvb-9q9xOYwnzpZcOXWgUeM6fxQ&s=uTJ98MgxxcM61HIDJRBpfJpuLDt9Ug4ARh8P_Api3xQ&e= > > _______________________________________________ lustre-discuss mailing list lustre-discuss@lists.lustre.org http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org