@Jiri Denemark <jdene...@redhat.com> @Eduardo Lima <etru...@redhat.com> can you please have a look on libvirt side? @Martin Perina <mper...@redhat.com> the host/stats part within vdsm was handled by people who are not working anymore on oVirt project, perhaps someone from infra can have a look?
Il giorno gio 9 dic 2021 alle ore 11:20 Victor Stinner <vstin...@redhat.com> ha scritto: > On Tue, Dec 7, 2021 at 6:12 PM Chris Adams <c...@cmadams.net> wrote: > > Top differences > > /usr/lib64/python3.6/site-packages/libvirt.py:442: size=295 MiB (+285 > MiB), count=5511282 (+5312311), average=56 B > > /usr/lib64/python3.6/json/decoder.py:355: size=73.9 MiB (+70.2 MiB), > count=736108 (+697450), average=105 B > > /usr/lib64/python3.6/logging/__init__.py:1630: size=44.2 MiB (+43.8 > MiB), count=345704 (+342481), average=134 B > > /usr/lib64/python3.6/site-packages/libvirt.py:5695: size=30.3 MiB (+30.0 > MiB), count=190449 (+188665), average=167 B > > /usr/lib/python3.6/site-packages/vdsm/host/stats.py:138: size=12.1 MiB > (+11.4 MiB), count=75366 (+70991), average=168 B > > /usr/lib/python3.6/site-packages/vdsm/utils.py:358: size=10.4 MiB (+9968 > KiB), count=70204 (+65272), average=156 B > > That's quite significant! > > > Top block > > 5511282 memory blocks: 302589.8 KiB > > File "/usr/lib64/python3.6/site-packages/libvirt.py", line 442 > > ret = libvirtmod.virEventRunDefaultImpl() > > File > "/usr/lib/python3.6/site-packages/vdsm/common/libvirtconnection.py", line 69 > > libvirt.virEventRunDefaultImpl() > > File "/usr/lib/python3.6/site-packages/vdsm/common/concurrent.py", > line 260 > > ret = func(*args, **kwargs) > > You should check where these "ret" objects (of libvirt.py:442) are > stored: 5,511,282 is a lot of small objects (average: 56 bytes)! Maybe > they are stored in a list and never destroyed. > > Maybe it's a reference leak in the libvirtmod.virEventRunDefaultImpl() > function of "libvirtmod" C extension: missing Py_DECREF() somewhere. > > Or something somehow prevents to delete these projects object. For > example, an exception is stored somewhere which keeps all variables > alive (in Python 3, an exception stores a traceback object which keeps > all variables of all frames alive). > > On GitHub and GitLab, I found the following code. Maybe there are > minor differences in the versions that you are using. > > https://gitlab.com/libvirt/libvirt-python > (I built the code locally to get build/libvirt.py) > > build/libvirt.c: > --- > PyObject * > libvirt_intWrap(int val) > { > return PyLong_FromLong((long) val); > } > > PyObject * > libvirt_virEventRunDefaultImpl(PyObject *self ATTRIBUTE_UNUSED, > PyObject *args ATTRIBUTE_UNUSED) { > PyObject *py_retval; > int c_retval; > LIBVIRT_BEGIN_ALLOW_THREADS; > c_retval = virEventRunDefaultImpl(); > LIBVIRT_END_ALLOW_THREADS; > py_retval = libvirt_intWrap((int) c_retval); > return py_retval; > } > > static PyMethodDef libvirtMethods[] = { > { (char *)"virEventRunDefaultImpl", > libvirt_virEventRunDefaultImpl, METH_VARARGS, NULL }, > ... > {NULL, NULL, 0, NULL} > }; > --- > > This code looks correct and straightforward. Is it possible that > internally virEventRunDefaultImpl() calls a Python memory allocator? > > build/libvirt.py: > --- > def virEventRunDefaultImpl(): > ret = libvirtmod.virEventRunDefaultImpl() > if ret == -1: > raise libvirtError('virEventRunDefaultImpl() failed') > return ret > --- > > Again, this code looks correct and straightforward. > > > https://github.com/oVirt/vdsm/blob/37ed5c279c2dd9c9bb06329d674882e0f98f34d6/lib/vdsm/common/libvirtconnection.py > > vdsm/common/libvirtconnection.py: > --- > def __run(self): > try: > libvirt.virEventRegisterDefaultImpl() > while self.run: > libvirt.virEventRunDefaultImpl() > finally: > self.run = False > --- > > libvirt.virEventRunDefaultImpl() result is ignored and so I don't see > anything obvious which would explain a leak. > > > Sometimes, looking at the top function is misleading since the > explanation can be found in one of the caller functions. > > For example, which function creates 70.2 MiB of objects from a JSON > document? What calls json/decoder.py:355? > > Victor > _______________________________________________ > Users mailing list -- users@ovirt.org > To unsubscribe send an email to users-le...@ovirt.org > Privacy Statement: https://www.ovirt.org/privacy-policy.html > oVirt Code of Conduct: > https://www.ovirt.org/community/about/community-guidelines/ > List Archives: > https://lists.ovirt.org/archives/list/users@ovirt.org/message/O5OAA6KNLINLRT2VYKNBI2PPH6UIYR4A/ > -- Sandro Bonazzola MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV Red Hat EMEA <https://www.redhat.com/> sbona...@redhat.com <https://www.redhat.com/> *Red Hat respects your work life balance. Therefore there is no need to answer this email out of your office hours.*
_______________________________________________ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-le...@ovirt.org Privacy Statement: https://www.ovirt.org/privacy-policy.html oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/JIZSOSY4JDYNXXPJM7IKJ4VU3SDRCJP3/
