On Tue, Jul 02, 2019 at 09:57:44AM -0400, Brian Reichert wrote: > At $JOB, on some of our SLES12 boxes, our logs are getting swamped > with messages saying: > > "Too many concurrent connections, refusing"
Please always specify the systemd version in use. We're not all SLES users, and even if we were, I assume that there might be different package versions over time. > # ss -x | grep /run/systemd/private | wc -l > 4015 /run/systemd/private is used by systemctl and other systemd utilities when running as root. Those connections are expected to be short-lived. Generally, on a normal machine "ss -x | grep /run/systemd/private | wc -l" is expected to yield 0 or a very low number transiently. > But, despite the almost 4k connections, 'ss' shows that there are > no connected peers: > > # ss -x | grep /run/systemd/private | grep -v -e '* 0' | wc -l > 0 Interesting. ss output is not documented at all from what I can see, but indeed '* 0' seems to indicate that. It is possible that systemd has a leak and is not closing the private bus connections properly. > When we first spin up a new SLES12 host with our custom services, > the number of connections to /run/systemd/private numbers in the > mere hundreds. That sounds wrong already. Please figure out what those connections are. I'm afraid that you might have to do some debugging on your own, since this issue doesn't seem easily reproducible. (I installed systemd with CONNECTIONS_MAX set to 10, and I can easily saturate the number of available connections with for i in {1..11}; do systemctl status '*' & sleep 0.5; kill -STOP $!;done As soon as I allow the processes to continue or kill them, the connection count goes down. They never show up with '* 0'.) > Is my guess about CONNECTIONS_MAX's relationship to /run/systemd/private > correct? Yes. The number is hardcoded because it's expected to be "large enough". The connection count shouldn't be more than "a few" or maybe a dozen at any time. > I have a hypothesis that this may be some resource leak in systemd, > but I've not found a way to test that. Once you figure out what is creating the connection, it would be useful to attach strace to pid 1 and see what is happening there. Zbyszek _______________________________________________ systemd-devel mailing list systemd-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/systemd-devel