We have also noticed this problem. We did an strace on horizon and found that the connections made by horizon to keystone were not getting closed.
Digging deeper, we found 2 very similar commits that fix connection closure issues related to keystone. https://github.com/openstack/python- keystoneclient/commit/8fcacdc7c74f5ac68e8e55ea8c15918c452411fe and https://github.com/openstack/keystoneauth/commit/dbcbf414ac8423e97d77d0bda8157be5350530f0 I think that the commit in keystoneauth is incomplete, it's missing the destination of the move of the _FakeRequestSession class and so it's just a removal. In Newton, horizon is using keystoneclient to get keystone sessions and we don't see the connection leaks. Somewhere between Newton and Queens, horizon switched to keystoneauth and lost the _FakeRequestSession hack and so we are now seeing a regression where horizon is leaking connections again. I am attaching a patch that we are currently testing, it re-introduces the _FakeRequestSession class into keystoneauth. The patch is a little naive, I don't know what to do with the instance attribute "adapters" of _FakeRequestSession, I just set it to an empty list because it is being referenced by code elsewhere. It seems to be working for now. ** Patch added: "session.py.patch" https://bugs.launchpad.net/horizon/+bug/1780164/+attachment/5184962/+files/session.py.patch ** Also affects: keystoneauth Importance: Undecided Status: New -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Dashboard (Horizon). https://bugs.launchpad.net/bugs/1780164 Title: httpd leaks open files Status in OpenStack Dashboard (Horizon): New Status in keystoneauth: New Bug description: horizon version 13.0.0-1.el7 (Queens) on centos 7.4.1708 After some time working on the dashboard, it stops working and throw this error in the error log [Wed Jul 04 22:49:33.744241 2018] [:error] [pid 23924] [remote 10.144.187.237:52] IOError: [Errno 24] Too many open files: '/usr/share/openstack- dashboard/openstack_dashboard/templates/500.html' if we check the open files of this process 23924 ls -l /proc/23924/fd | wc -l 1023 even if we increase the nofile limit of this process, it doesn't help as the open files is always increasing the problem is cleared if we restart the httpd process, but then the open files will increase again how to reproduce 1- login to the dashboard 2- get the pid from the error_log file [Wed Jul 04 22:50:58.620832 2018] [:error] [pid 23924] INFO openstack_auth.forms Login successful for user "demo" using domain "default" 3- browse the dashboard different menus, specially the network topology tab 4- monitor the open files no. with ls -l /proc/<pid>/fd | wc -l observation the no. is always increasing To manage notifications about this bug go to: https://bugs.launchpad.net/horizon/+bug/1780164/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp