Moin,

I have been dealing with memory-leaks on a host running an nginx
reverse proxy for some time. The host had been running 7.2 with nginx
1.23.1 (self compiled as i need some features not in the package) until
May, which was fine.

After upgrading to 7.3 and nginx-1.24.0, i started to see heavy memory
leakage over time. I initially attributed this to nginx, and solved the
issue by ignoring it/throwing a bit more memory at the box for some
time. However, I started debugging it now and could trace it to some
commit between nginx 1.23.3 and 1.23.4; Currently, I am going through
all commits to see with which version it first appears.

However, clicking around this morning, i noticed that my primary NS
shows a similar memory leakage for mysql/mariadb (runs powerdns with a
mysql backend, both from packages) since the upgrade to 7.3 in May as
well. One further host seems to show a significantly higher use of
inactive memory since 7.3. I found one more host with mariadb that now
shows a higher utilization of inactive memory (gitea+maria); However,
other maria+X instances on 7.3 run fine.

On the PowerDNS+Mysql host I also see gradually increasing CPU load
along with the memory leakage.

Restarting mysql and/or nginx resolves the issue for some time. I also
saw the issue on a host running maria+nextcloud, where i fixed it by
switching to postgres. Maria is still running there, but not receiving
any requests, and the memory leak is gone.

Images:

https://rincewind.home.aperture-labs.org/~tfiebig/pdns-maria-cpu-year.png
https://rincewind.home.aperture-labs.org/~tfiebig/pdns-maria-memory-year.png

https://rincewind.home.aperture-labs.org/~tfiebig/nginx-rev-proxy-memory-year.png

https://rincewind.home.aperture-labs.org/~tfiebig/gitea-maria-memory-year.png

https://rincewind.home.aperture-labs.org/~tfiebig/nextcloud-maria-switched-to-postgres-memory-year.png

All hosts are VMs on Linux/KVM hosts (proxmox), with a generic qemu-kvm
processor as CPU type, running 7.3. 

Hence, I am no longer 100% convinced that this is an nginx problem. To
help circling this in a bit closer (and maybe get it reproducible in a
first step): 

Did anyone else experience memory leakage on openbsd with mariadb or a
self-build >=nginx-1.23.4 (or other applications) since the upgrade to
7.3?

With best regards,
Tobias

Reply via email to