[Bug 1844455] Re: Memory leak on libvirt 1.3.1
We validated that commit 38816336a5 ("node_device_conf: Don't leak
@physical_function in virNodeDeviceGetPCISRIOVCaps") [0] indeed fixes
the leak investigated. Although there are more definitely-lost memory
reports from Valgrind, they are ultimately glibc-related and given the
report was in Trusty and they are not considerable leaks (at most,
8K/24H) our focus will be to fix the PCI-related leak in all libvirt
releases.
SRU template and debdiffs will get added here soon.
Cheers,
Guilherme
[0] libvirt.org/git/?p=libvirt.git;a=commit;h=38816336a5
** Also affects: libvirt (Ubuntu Bionic)
Importance: Undecided
Status: New
** Also affects: libvirt (Ubuntu Focal)
Importance: High
Assignee: Guilherme G. Piccoli (gpiccoli)
Status: Confirmed
** Also affects: libvirt (Ubuntu Eoan)
Importance: Undecided
Status: New
** Changed in: libvirt (Ubuntu Eoan)
Status: New => Confirmed
** Changed in: libvirt (Ubuntu Bionic)
Status: New => Confirmed
** Changed in: libvirt (Ubuntu Xenial)
Status: New => Confirmed
** Changed in: libvirt (Ubuntu Eoan)
Importance: Undecided => High
** Changed in: libvirt (Ubuntu Bionic)
Importance: Undecided => High
** Changed in: libvirt (Ubuntu Xenial)
Importance: Undecided => High
** Changed in: libvirt (Ubuntu Bionic)
Assignee: (unassigned) => Guilherme G. Piccoli (gpiccoli)
** Changed in: libvirt (Ubuntu Eoan)
Assignee: (unassigned) => Guilherme G. Piccoli (gpiccoli)
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1844455
Title:
Memory leak on libvirt 1.3.1
To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-archive/+bug/1844455/+subscriptions
--
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1844455] Re: Memory leak on libvirt 1.3.1
Just a minor update here, we found one leak to be fixed by the following commit: libvirt.org/git/?p=libvirt.git;a=commit;h=38816336a5 . We continue the investigation of the other leaks. Cheers, Guilherme -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1844455 Title: Memory leak on libvirt 1.3.1 To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-archive/+bug/1844455/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1844455] Re: Memory leak on libvirt 1.3.1
Hi Christian, thanks for your comment! I didn't receive an email with that comment, unfortunately. I'm not sure if they happen on upstream version, I've tried already and saw different leaks, potentially correlated with the virErr ones. I couldn't see libxml2 leaks. But i continue the investigation, thanks! -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1844455 Title: Memory leak on libvirt 1.3.1 To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-archive/+bug/1844455/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1844455] Re: Memory leak on libvirt 1.3.1
** Changed in: cloud-archive Assignee: (unassigned) => Guilherme G. Piccoli (gpiccoli) ** Changed in: cloud-archive/mitaka Assignee: (unassigned) => Guilherme G. Piccoli (gpiccoli) ** Changed in: cloud-archive Status: New => Confirmed ** Changed in: cloud-archive/mitaka Status: New => Confirmed -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1844455 Title: Memory leak on libvirt 1.3.1 To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-archive/+bug/1844455/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1844455] Re: Memory leak on libvirt 1.3.1
Hi, thanks for the report and your work on it. Are you sure those issues are not happening on newer versions? Because a) if they are fixed in newer versions then I'd ask you to backport the respective fixes from there (can be partial if they were bigger reworks, but always better than starting from scratch). b) if they are not fixed in newer versions we should address and fix those as well (For SRU policy and long term viability) So after your further analysis of the case give the new versions a check in that regard please. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1844455 Title: Memory leak on libvirt 1.3.1 To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-archive/+bug/1844455/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1844455] Re: Memory leak on libvirt 1.3.1
** Also affects: cloud-archive Importance: Undecided Status: New ** Also affects: cloud-archive/mitaka Importance: Undecided Status: New -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1844455 Title: Memory leak on libvirt 1.3.1 To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-archive/+bug/1844455/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1844455] Re: Memory leak on libvirt 1.3.1
Leak #2: ==21823==385 (280 direct, 105 indirect) bytes in 1 blocks are definitely lost in loss record 88 of 106 ==21823==at 0x4C2CC70: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so) ==21823==by 0x50A5299: virAlloc (viralloc.c:144) ==21823==by 0x14B185: daemonConfigNew (libvirtd-config.c:242) This one is similar to the previous one, it's likely struct daemonConfig is leaked. This structure is allocated in daemonNew and freed by daemonConfigFree(), at the end of libvirtd main() function. The hypothesis here again is that we had either a process killed or we should correctly terminate Valgrind in order it collects the free calls for this object. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1844455 Title: Memory leak on libvirt 1.3.1 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1844455/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1844455] Re: Memory leak on libvirt 1.3.1
** Attachment added: "valgrind-definitely_only.txt" https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1844455/+attachment/5289284/+files/valgrind-definitely_only.txt -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1844455 Title: Memory leak on libvirt 1.3.1 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1844455/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1844455] Re: Memory leak on libvirt 1.3.1
Leak #3 968 bytes in 1 blocks are definitely lost in loss record 1,313 of 1,405 at 0x4C2AB80: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so) by 0x6D4BA21: xmlGetGlobalState (in /usr/lib/x86_64-linux-gnu/libxml2.so.2.9.3) by 0x6D4B214: __xmlIndentTreeOutput (in /usr/lib/x86_64-linux-gnu/libxml2.so.2.9.3) by 0x5149F7D: virDomainDefFormatInternal (domain_conf.c:21655) by 0x514D0F2: virDomainDefFormat (domain_conf.c:22517) by 0x515B93B: virDomainDefCopy (domain_conf.c:23041) by 0x515B9FF: virDomainObjSetDefTransient (domain_conf.c:2818) by 0x2B70CCD9: qemuProcessInit (qemu_process.c:4483) by 0x2B70CFEE: qemuProcessStart (qemu_process.c:5150) by 0x2B76D727: qemuDomainObjStart.constprop.47 (qemu_driver.c:7396) by 0x2B76DE65: qemuDomainCreateWithFlags (qemu_driver.c:7450) by 0x51CC7FB: virDomainCreate (libvirt-domain.c:6753) I've checked the code in libxml2, and what is leaking is the struct _xmlGlobalState, in function xmlNewGlobalState(), called from xmlGetGlobalState() - this is part of thread orchestration mechanism in libxml. Using pahole I could check the struct size, and it matches: $ pahole --hex -C _xmlGlobalState | grep size /* size: 968, cachelines: 16, members: 33 */ The struct is freed in xmlFreeGlobalState(), when thread exits. This is again a case when there's a cleanup routine that seems to be skipped; we need to understand really if the process is being killed without having chance to run these cleanups. I'm preparing a test package to validate the cleanups execution. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1844455 Title: Memory leak on libvirt 1.3.1 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1844455/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1844455] Re: Memory leak on libvirt 1.3.1
Leak #1:
==12891==80 bytes in 1 blocks are definitely lost in loss record 949 of
1,360
==12891==at 0x4C2CC70: calloc (in
/usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==12891==by 0x50A5299: virAlloc (viralloc.c:144)
==12891==by 0x50C1C63: virLastErrorObject (virerror.c:240)
==12891==by 0x50C4F88: virResetLastError (virerror.c:412)
==12891==by 0x51B786C: virConnectOpen (libvirt.c:1139)
>From pahole:
struct _virError {
intcode; /* 0 0x4 */
intdomain; /* 0x4 0x4 */
char * message; /* 0x8 0x8 */
virErrorLevel level;/* 0x10 0x4 */
/* XXX 4 bytes hole, try to pack */
virConnectPtr conn; /* 0x18 0x8 */
virDomainPtr dom; /* 0x20 0x8 */
char * str1; /* 0x28 0x8 */
char * str2; /* 0x30 0x8 */
char * str3; /* 0x38 0x8 */
/* --- cacheline 1 boundary (64 bytes) --- */
intint1; /* 0x40 0x4 */
intint2; /* 0x44 0x4 */
virNetworkPtr net; /* 0x48 0x8 */
/* size: 80, cachelines: 2, members: 12 */
/* sum members: 76, holes: 1, sum holes: 4 */
/* last cacheline: 16 bytes */
};
The _virError struct (in the form of virErrorPtr typedef) is expected to be
freed
in virLastErrFreeData(), which is a thread "destructor" set in the pthread
creation. It should be called when thread exists (by pthread_exit() or something
analog), but can be skipped if process main() function returns.
So, hypothesis here are:
1) There was a process exist that led to this thread data getting leaked
2) Valgrind should be ended (with SIGTERM) in order to collect the thread
destructor execution, so this is a false/temporary leak.
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1844455
Title:
Memory leak on libvirt 1.3.1
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1844455/+subscriptions
--
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
