Thank you for answering Andreas. Lustre version is 2.12.8
It is indeed when we delete io500 files when we discovered this, but we also see it when deleting other files, that the "df" lags 1-2 hours behind. We see it both on nvme and ssd drives. Haven't checkd hdd drives/osts yet. This is a new lustre setup, and benchmarks are good (in our opinion). For now it is just this annoyance bugging us. We didn't notice on previous lustre setup but will check if we see it there also. Einar ________________________________________ From: Andreas Dilger <adil...@whamcloud.com> Sent: Monday, April 11, 2022 18:01 To: Einar Næss Jensen Cc: lustre-discuss@lists.lustre.org Subject: Re: [lustre-discuss] question regarding du vs df on lustre Lustre is returning the file unlink from the MDS immediately, but deleting the objects from the OSTs asynchronously in the background. How many files are being deleted in this case? If you are running tests like IO500, where there are many millions of small files plus some huge files, then it may be that huge object deletion is behind small objects? That said, it probably shouldn't take hours to finish if the OST storage is NVMe based. Cheers, Andreas > On Apr 4, 2022, at 05:05, Einar Næss Jensen <einar.nass.jen...@ntnu.no> wrote: > > Hello lustre people. > > We are experimenting with lustre on nvme, and observe the following issue: > After running benchmarks and deleting benchmark files, we see that df and du > reports different sizes: > > [root@idun-02-27 ~]# du -hs /nvme/ > 38M /nvme/ > [root@idun-02-27 ~]# df -h|grep nvme > 10.3.1.2@o2ib:/nvme 5.5T 3.9T 1.3T 76% /nvme > > > It takes several hours before du and df agrees. > > What is causing this? > How can we get updated records for df immediately when deleting files? > > > Best REegards > Einar > _______________________________________________ > lustre-discuss mailing list > lustre-discuss@lists.lustre.org > http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org _______________________________________________ lustre-discuss mailing list lustre-discuss@lists.lustre.org http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
