notforwarded 462664 title 462664 evolution-data-server: multiple massive memory leaks in the caldav backend thanks
[ Hi Roland, how're ya doing? ] justification of the "notforwarded": upstream bug report is not related to caldav (although it is, indeed, related to other memory leaks within E-D-S). here follows my original bug report which by chance failed to be delivered to [EMAIL PROTECTED] before I had a chance to notice Roland's earlier report on the same topic. ------------------ Package: evolution-data-server Version: 2.22.3-1.1 Severity: important Justification: try running your system with an E-D-S gobbling up 5G of memory ;) After a day of use of an Evolution with a single CalDAV calendar, the memory consumption of E-D-S is very high. Running E-D-S through valgrind yields a few interesting tracks: ==10847== 505,964 (22,392 direct, 483,572 indirect) bytes in 311 blocks are definitely lost in loss record 68 of 113 ==10847== at 0x4C20FEB: malloc (vg_replace_malloc.c:207) ==10847== by 0x7B4092E: xmlXPathWrapNodeSet (in /usr/lib/libxml2.so.2.6.32) ==10847== by 0x7B4ED93: (within /usr/lib/libxml2.so.2.6.32) ==10847== by 0x7B4CEF0: (within /usr/lib/libxml2.so.2.6.32) ==10847== by 0x7B4D6AE: (within /usr/lib/libxml2.so.2.6.32) ==10847== by 0x7B4FC74: (within /usr/lib/libxml2.so.2.6.32) ==10847== by 0x7B529BB: xmlXPathEvalExpression (in /usr/lib/libxml2.so.2.6.32) ==10847== by 0xEEFFB8E: xpath_eval (e-cal-backend-caldav.c:497) ==10847== by 0xEEFFF83: synch_slave_loop (e-cal-backend-caldav.c:700) ==10847== by 0x900A4D3: (within /usr/lib/libglib-2.0.so.0.1600.5) ==10847== by 0x9AB8FC6: start_thread (in /lib/libpthread-2.7.so) ==10847== by 0x9D9D7CC: clone (in /lib/libc-2.7.so) it seems that in function static gboolean parse_report_response (SoupMessage *soup_message, CalDAVObject **objs, int *len) from e-cal-backend-caldav.c, no-one attempts to free the xmlXPathObjects that are returned by function xpath_eval() (the method to free is xmlXPathFreeObject()) It does not appear that parse_report_response() frees memory in SVN evolution either. ==10847== 251,005 (22,080 direct, 228,925 indirect) bytes in 184 blocks are definitely lost in loss record 69 of 113 ==10847== at 0x4C20FEB: malloc (vg_replace_malloc.c:207) ==10847== by 0x7B1222D: xmlNewNode (in /usr/lib/libxml2.so.2.6.32) ==10847== by 0x7B165B7: xmlNewDocNode (in /usr/lib/libxml2.so.2.6.32) ==10847== by 0x7B166A0: xmlNewDocRawNode (in /usr/lib/libxml2.so.2.6.32) ==10847== by 0x7B16744: xmlNewTextChild (in /usr/lib/libxml2.so.2.6.32) ==10847== by 0xEEFFD95: synch_slave_loop (e-cal-backend-caldav.c:944) ==10847== by 0x900A4D3: (within /usr/lib/libglib-2.0.so.0.1600.5) ==10847== by 0x9AB8FC6: start_thread (in /lib/libpthread-2.7.so) ==10847== by 0x9D9D7CC: clone (in /lib/libc-2.7.so) ==10847== ==10847== 49,102 (3,456 direct, 45,646 indirect) bytes in 36 blocks are definitely lost in loss record 75 of 113 ==10847== at 0x4C20FEB: malloc (vg_replace_malloc.c:207) ==10847== by 0x7B11DBA: (within /usr/lib/libxml2.so.2.6.32) ==10847== by 0x7B14D56: xmlSetProp (in /usr/lib/libxml2.so.2.6.32) ==10847== by 0xEEFFDC2: synch_slave_loop (e-cal-backend-caldav.c:946) ==10847== by 0x900A4D3: (within /usr/lib/libglib-2.0.so.0.1600.5) ==10847== by 0x9AB8FC6: start_thread (in /lib/libpthread-2.7.so) ==10847== by 0x9D9D7CC: clone (in /lib/libc-2.7.so) ==10847== ==10847== 38,102 (3,360 direct, 34,742 indirect) bytes in 28 blocks are definitely lost in loss record 84 of 113 ==10847== at 0x4C20FEB: malloc (vg_replace_malloc.c:207) ==10847== by 0x7B11CC6: xmlNewText (in /usr/lib/libxml2.so.2.6.32) ==10847== by 0x7B11D6B: xmlNewDocText (in /usr/lib/libxml2.so.2.6.32) ==10847== by 0x7B11E23: (within /usr/lib/libxml2.so.2.6.32) ==10847== by 0x7B14D56: xmlSetProp (in /usr/lib/libxml2.so.2.6.32) ==10847== by 0xEEFFDC2: synch_slave_loop (e-cal-backend-caldav.c:946) ==10847== by 0x900A4D3: (within /usr/lib/libglib-2.0.so.0.1600.5) ==10847== by 0x9AB8FC6: start_thread (in /lib/libpthread-2.7.so) ==10847== by 0x9D9D7CC: clone (in /lib/libc-2.7.so) These three stacks all involve the function caldav_server_list_objects() (apparently untouched in SVN). It seems to me that nothing ever ties the root xmlNodePtr (from which all child nodes and attributes are created) to the xmlDocPtr "doc"; hence when the document is freed using xmlFreeDoc(doc), the root node (and its children) are lost. line 934 might read instead root = xmlNewDocNode (doc, NULL, (xmlChar *)"calendar-query", (xmlChar*)NULL); ==10847== 3,455 (56 direct, 3,399 indirect) bytes in 1 blocks are definitely lost in loss record 85 of 113 ==10847== at 0x4C20FEB: malloc (vg_replace_malloc.c:207) ==10847== by 0x6C73079: icalproperty_new_impl (in /usr/lib/libecal-1.2.so.7.2.0) ==10847== by 0x6C5F28A: icalproperty_new_version (in //usr/lib/libecal-1.2.so.7.2.0) ==10847== by 0x6C59727: e_cal_util_new_top_level (in /usr/lib/libecal-1.2.so.7.2.0) ==10847== by 0xEEFE789: pack_cobj (e-cal-backend-caldav.c:1645) ==10847== by 0xEEFF82C: caldav_create_object (e-cal-backend-caldav.c:1702) ==10847== by 0x69EBD12: e_cal_backend_sync_create_object (in /usr/lib/libedata-cal-1.2.so.6.0.2) ==10847== by 0x69EBDDB: (within /usr/lib/libedata-cal-1.2.so.6.0.2) ==10847== by 0x82D9FA5: ORBit_small_invoke_adaptor (in /usr/lib/libORBit-2.so.0.1.0) ==10847== by 0x82E927F: (within /usr/lib/libORBit-2.so.0.1.0) ==10847== by 0x82E9839: (within /usr/lib/libORBit-2.so.0.1.0) ==10847== by 0x82D3374: giop_thread_queue_process (in /usr/lib/libORBit-2.so.0.1.0) the routine static char * pack_cobj (ECalBackendCalDAV *cbdav, ECalComponent *ecomp) carefully constructs an icalcomponent* (named "calcomp") and asks it to serialize itself into a string ("objstr"). The string is returned and the calcomp is dropped as if there was a JVM to pick it up. This seems untouched in SVN. pack_cobj(...) should delete the icalcomponent (calling icalcomponent_free(); perhaps first calling g_strdup() on the objstr, depending on the ownership rules of icalcomponent_as_ical_string(). ==10847== 1,282,280 (1,228,896 direct, 53,384 indirect) bytes in 489 blocks are definitely lost in loss record 100 of 113 ==10847== at 0x4C200FC: calloc (vg_replace_malloc.c:397) ==10847== by 0x8FE9BE9: g_malloc0 (in /usr/lib/libglib-2.0.so.0.1600.5) ==10847== by 0xEEFFFF6: synch_slave_loop (e-cal-backend-caldav.c:711) ==10847== by 0x900A4D3: (within /usr/lib/libglib-2.0.so.0.1600.5) ==10847== by 0x9AB8FC6: start_thread (in /lib/libpthread-2.7.so) ==10847== by 0x9D9D7CC: clone (in /lib/libc-2.7.so) ==10847== ==10847== in function synchronize_cache(), the result of calling caldav_server_list_objects (the array of CalDAVObject named "sobjs") is never freed. It ought to be freed at the end of the for loop "see if we have to upate or add some objects", line 1305. It is possible that any server objects found with object->status != 200 are dropped without destruction. This seems untouched in SVN. ******************************************** ******************************************** furthermore, this only addresses the 4-odd megabytes that were definitely lost in that session. Valgrind, ran through valgrind --leak-check=full \ /usr/lib/evolution/evolution-data-server-2.22 \ --oaf-activate-iid=OAFIID:GNOME_Evolution_DataServer_InterfaceCheck \ --oaf-for-fd=25 also reports 280MB of "still reachable" memory (which is still very probably a case of 'hot leak', a.k.a failure to unreference obsolete things). I will re-run valgrind with --leak-check=full --show-reachable=yes to further investigate. -- Cyrille -- System Information: Debian Release: lenny/sid APT prefers testing APT policy: (800, 'testing'), (600, 'unstable') Architecture: amd64 (x86_64) Kernel: Linux 2.6.25-2-amd64 (SMP w/2 CPU cores) Locale: LANG=fr_FR.UTF-8, LC_CTYPE=fr_FR.UTF-8 (charmap=UTF-8) Shell: /bin/sh linked to /bin/bash Versions of packages evolution-data-server depends on: ii evolution-data-server 2.22.3-1.1 architecture independent files for ii libbonobo2-0 2.22.0-1 Bonobo CORBA interfaces library ii libc6 2.7-13 GNU C Library: Shared libraries ii libcamel1.2-11 2.22.3-1 The Evolution MIME message handlin ii libcomerr2 1.41.0-3 common error description library ii libdb4.6 4.6.21-8 Berkeley v4.6 Database Libraries [ ii libebook1.2-9 2.22.3-1 Client library for evolution addre ii libecal1.2-7 2.22.3-1 Client library for evolution calen ii libedata-book1.2-2 2.22.3-1 Backend library for evolution addr ii libedata-cal1.2-6 2.22.3-1 Backend library for evolution cale ii libedataserver1.2-9 2.22.3-1 Utility library for evolution data ii libegroupwise1.2-13 2.22.3-1 Client library for accessing group ii libgconf2-4 2.22.0-1 GNOME configuration database syste ii libgdata-google1.2-1 2.22.3-1 Client library for accessing Googl ii libgdata1.2-1 2.22.3-1 Client library for accessing Googl ii libglib2.0-0 2.16.5-1 The GLib library of C routines ii libgnome2-0 2.20.1.1-1 The GNOME 2 library - runtime file ii libgnomevfs2-0 1:2.22.0-4 GNOME Virtual File System (runtime ii libkrb53 1.6.dfsg.4~beta1-4 MIT Kerberos runtime libraries ii libldap-2.4-2 2.4.7-6.1 OpenLDAP libraries ii libnspr4-0d 4.7.1-3 NetScape Portable Runtime Library ii libnss3-1d 3.12.0~rc3-3 Network Security Service libraries ii liborbit2 1:2.14.13-0.1 libraries for ORBit2 - a CORBA ORB ii libpopt0 1.14-4 lib for parsing cmdline parameters ii libsoup2.4-1 2.4.1-1 an HTTP library implementation in ii libxml2 2.6.32.dfsg-3 GNOME XML library ii zlib1g 1:1.2.3.3.dfsg-12 compression library - runtime evolution-data-server recommends no packages. Versions of packages evolution-data-server suggests: ii evolution 2.22.3.1-1 groupware suite with mail client a ii evolution-data-server-dbg 2.22.3-1.1 evolution database backend server -- no debconf information -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]