[ceph-users] Re: ceph_leadership_team_meeting_s18e06.mkv

2023-09-07 Thread Rok Jaklič
Hi, we have also experienced several ceph-mgr oom kills on ceph v16.2.13 on 120T/200T data. Is there any tracker about the problem? Does upgrade to 17.x "solves" the problem? Kind regards, Rok On Wed, Sep 6, 2023 at 9:36 PM Ernesto Puerta wrote: > Dear Cephers, > > Today brought us an even

[ceph-users] Re: ceph_leadership_team_meeting_s18e06.mkv

2023-09-07 Thread Mark Nelson
Hi Rok, We're still try to catch what's causing the memory growth, so it's hard to guess at which releases are affected. We know it's happening intermittently on a live Pacific cluster at least. If you have the ability to catch it while it's happening, there are several approaches/tools tha

[ceph-users] Re: ceph_leadership_team_meeting_s18e06.mkv

2023-09-08 Thread Rok Jaklič
We do not use containers. Anything special for debugging or should we try something from previous email? - Enable profiling (Mark Nelson) - Try Bloomberg's Python mem profiler (Matthew Leonard) Profiling means instructions from https://docs.ceph.com/

[ceph-users] Re: ceph_leadership_team_meeting_s18e06.mkv

2023-09-08 Thread Loïc Tortay
On 07/09/2023 21:33, Mark Nelson wrote: Hi Rok, We're still try to catch what's causing the memory growth, so it's hard to guess at which releases are affected.  We know it's happening intermittently on a live Pacific cluster at least.  If you have the ability to catch it while it's happening

[ceph-users] Re: ceph_leadership_team_meeting_s18e06.mkv

2023-09-08 Thread David Orman
I would suggest updating: https://tracker.ceph.com/issues/59580 We did notice it with 16.2.13, as well, after upgrading from .10, so likely in-between those two releases. David On Fri, Sep 8, 2023, at 04:00, Loïc Tortay wrote: > On 07/09/2023 21:33, Mark Nelson wrote: >> Hi Rok, >> >> We're st

[ceph-users] Re: ceph_leadership_team_meeting_s18e06.mkv

2023-09-11 Thread Rok Jaklič
I can confirm this. ... as we did the upgrade from .10 also. Rok On Fri, Sep 8, 2023 at 5:26 PM David Orman wrote: > I would suggest updating: https://tracker.ceph.com/issues/59580 > > We did notice it with 16.2.13, as well, after upgrading from .10, so > likely in-between those two releases.