Re: [ceph-users] MDS: obscene buffer_anon memory use when scanning lots of files

2020-01-22 Thread John Madden
> Couldn't John confirm that this is the issue by checking the heap stats and > triggering the release via > > ceph tell mds.mds1 heap stats > ceph tell mds.mds1 heap release > > (this would be much less disruptive than restarting the MDS) That was my first thought as well, but `release`

Re: [ceph-users] MDS: obscene buffer_anon memory use when scanning lots of files

2020-01-21 Thread Dan van der Ster
On Wed, Jan 22, 2020 at 12:24 AM Patrick Donnelly wrote: > On Tue, Jan 21, 2020 at 8:32 AM John Madden wrote: > > > > On 14.2.5 but also present in Luminous, buffer_anon memory use spirals > > out of control when scanning many thousands of files. The use case is > > more or less "look up this

Re: [ceph-users] MDS: obscene buffer_anon memory use when scanning lots of files

2020-01-21 Thread Patrick Donnelly
On Tue, Jan 21, 2020 at 8:32 AM John Madden wrote: > > On 14.2.5 but also present in Luminous, buffer_anon memory use spirals > out of control when scanning many thousands of files. The use case is > more or less "look up this file and if it exists append this chunk to > it, otherwise create it

[ceph-users] MDS: obscene buffer_anon memory use when scanning lots of files

2020-01-21 Thread John Madden
On 14.2.5 but also present in Luminous, buffer_anon memory use spirals out of control when scanning many thousands of files. The use case is more or less "look up this file and if it exists append this chunk to it, otherwise create it with this chunk." The memory is recovered as soon as the