About how many files are we talking here? Implementation detail on file deletion to understand why this might happen: deletion is async, deleting a file inserts it into the purge queue and the actual data will be removed in the background.
Paul -- Paul Emmerich Looking for help with your Ceph cluster? Contact us at https://croit.io croit GmbH Freseniusstr. 31h 81247 München www.croit.io Tel: +49 89 1896585 90 On Fri, May 10, 2019 at 1:04 PM Kenneth Waegeman <kenneth.waege...@ugent.be> wrote: > Hi all, > > I am seeing issues on cephfs running 13.2.5 when deleting files: > > [root@osd006 ~]# rm /mnt/ceph/backups/osd006.gigalith.os-2b5a3740.1326700 > rm: remove regular empty file > ‘/mnt/ceph/backups/osd006.gigalith.os-2b5a3740.1326700’? y > rm: cannot remove > ‘/mnt/ceph/backups/osd006.gigalith.os-2b5a3740.1326700’: No space left > on device > > few minutes later, I can remove it without problem. This happens > especially when there are a lot of files deleted somewhere on the > filesystem around the same time. > > We already have tuned our mds config: > > [mds] > mds_cache_memory_limit=10737418240 > mds_log_max_expiring=200 > mds_log_max_segments=200 > mds_max_purge_files=2560 > mds_max_purge_ops=327600 > mds_max_purge_ops_per_pg=20 > > ceph -s is reporting everything clean, and the file system space usage > is less than 50%, also no full osds or anything. > > Is there a way to further debug what the bottleneck is when removing > files that gives this 'no space left on device' error? > > > Thank you very much! > > Kenneth > > _______________________________________________ > ceph-users mailing list > ceph-users@lists.ceph.com > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com >
_______________________________________________ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
