Hi,

> On 13 Aug 2021, at 14:10, Martin Traxl <martin.tr...@1und1.de> wrote:
> 
> yesterday evening one of my rgw nodes died again, radosgw was killed by the 
> kernel oom killer.
> 
> [Thu Aug 12 22:10:04 2021] Out of memory: Killed process 1376 (radosgw) 
> total-vm:70747176kB, anon-rss:63900544kB, file-rss:0kB, shmem-rss:0kB, 
> UID:167 pgtables:131008kB oom_score_adj:0
> [Thu Aug 12 22:10:09 2021] oom_reaper: reaped process 1376 (radosgw), now 
> anon-rss:0kB, file-rss:0kB, shmem-rss:0kB
> 
> The radosgw was eating up all the 64GB system memory.
> A few hours before this happened, mempool dump showed a total usage of only 
> 2.1 GB of ram, while in fact radosgw was using already 84.7% of 64GB.
> 
>        "total": {
>            "items": 88757980,
>            "bytes": 2147532284
> 
>    PID USER      PR  NI    VIRT    RES    SHR S  %CPU  %MEM     TIME+ COMMAND 
>                                                                               
>                                                                               
>                     
>   1376 ceph      20   0   58.8g  52.7g  17824 S  48.2  84.7  20158:04 radosgw
> 
> 
> It seems the radowgw loses track of some memory, like there is a memory leak.
> 
> Some additional information. I am running on CentOS 8.4, kernel 4.18. As 
> already mentioned, Ceph 14.2.22. radosgw is the only notable service running 
> on this machine.
> Any suggestions on this? Are there maybe any tuning settings? How could I 
> debug this further?

Please show your "config diff" from admin socket
Couple of days ago I was upgraded our RGW's to 14.2.21 to 14.2.22 and don't see 
increase memory consumption


Thanks,
k
_______________________________________________
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io

Reply via email to