[valgrind] [Bug 520170] valgrind reports Mismatched free() / delete / delete [], but it doesn't appear to be mismatched
https://bugs.kde.org/show_bug.cgi?id=520170 --- Comment #15 from Michael Catanzaro --- Corrected link: https://gitlab.gnome.org/GNOME/gnome-builder/-/work_items/2403 -- You are receiving this mail because: You are watching all bug changes.
[valgrind] [Bug 520170] valgrind reports Mismatched free() / delete / delete [], but it doesn't appear to be mismatched
https://bugs.kde.org/show_bug.cgi?id=520170 --- Comment #14 from Michael Catanzaro --- Builder issue report: https://gitlab.gnome.org/GNOME/gnome-build-meta/-/work_items/1279 -- You are receiving this mail because: You are watching all bug changes.
[valgrind] [Bug 520170] valgrind reports Mismatched free() / delete / delete [], but it doesn't appear to be mismatched
https://bugs.kde.org/show_bug.cgi?id=520170 --- Comment #13 from Paul Floyd --- (In reply to Michael Catanzaro from comment #12) > Huh, it seems Builder actually does have an option to use tcmalloc. Sigh. > Sorry for wasting your time. No worry! I appreciate that it is hard to know every piece of a very large project. Don't hesistate to to open bugzilla items here if you think you have problems. You did a good job in reporting all of the details. -- You are receiving this mail because: You are watching all bug changes.
[valgrind] [Bug 520170] valgrind reports Mismatched free() / delete / delete [], but it doesn't appear to be mismatched
https://bugs.kde.org/show_bug.cgi?id=520170 --- Comment #12 from Michael Catanzaro --- Huh, it seems Builder actually does have an option to use tcmalloc. Sigh. Sorry for wasting your time. -- You are receiving this mail because: You are watching all bug changes.
[valgrind] [Bug 520170] valgrind reports Mismatched free() / delete / delete [], but it doesn't appear to be mismatched
https://bugs.kde.org/show_bug.cgi?id=520170 Paul Floyd changed: What|Removed |Added Status|REPORTED|RESOLVED Resolution|--- |NOT A BUG --- Comment #11 from Paul Floyd --- >From what I see, you are using TCMalloc: sym at 0x50209b0: prefer 'nallocx' to 'tc_nallocx' sym at 0x50214c0: prefer 'tc_malloc' to '__libc_malloc' sym at 0x50214c0: prefer 'malloc' to 'tc_malloc' sym at 0x5021580: prefer 'cfree' to 'tc_cfree' sym at 0x5021580: prefer 'cfree' to '__libc_cfree' sym at 0x5021580: prefer 'cfree' to '__libc_free' sym at 0x5021580: prefer 'cfree' to 'tc_free' sym at 0x5021580: prefer 'cfree' to '_ZdlPv' sym at 0x5021580: prefer 'cfree' to '_ZdaPv' sym at 0x5021580: prefer 'cfree' to 'tc_delete' sym at 0x5021580: prefer 'cfree' to 'tc_deletearray' sym at 0x5021580: prefer 'free' to 'cfree' sym at 0x50216d0: prefer '_ZN13TCMallocGuardC1Ev' to '_ZN13TCMallocGuardC2Ev' sym at 0x5021740: prefer '_ZdlPvm' to 'tc_free_sized' sym at 0x5021740: prefer '_ZdlPvm' to 'tc_delete_sized' sym at 0x5021740: prefer '_ZdaPvm' to '_ZdlPvm' sym at 0x5021740: prefer '_ZdaPvm' to 'free_sized' sym at 0x5021740: prefer '_ZdaPvm' to 'tc_deletearray_sized' sym at 0x50218b0: prefer 'calloc' to '__libc_calloc' sym at 0x50218b0: prefer 'calloc' to 'tc_calloc' sym at 0x5021aa0: prefer 'tc_realloc' to '__libc_realloc' sym at 0x5021aa0: prefer 'realloc' to 'tc_realloc' sym at 0x5022280: prefer '_Znwm' to 'tc_new' sym at 0x5022280: prefer '_Znwm' to 'tc_newarray' (all those tc_ prefix functions are from TCMalloc). That means you have 3 choices, as described in the FAQ 1. Build with a version of TCMalloc that does not optimize deallocation functions by using aliases (a debug byuld of TCMalloc or build it with CPPFLAGS=-DTCMALLOC_NO_ALIASES 2. Do not link with tcmalloc 3. Use --show-mismatched-frees=no. You will then miss any genuine mismatches in your code. It's not the most dangerous error, so that isn't too bad an option. -- You are receiving this mail because: You are watching all bug changes.
[valgrind] [Bug 520170] valgrind reports Mismatched free() / delete / delete [], but it doesn't appear to be mismatched
https://bugs.kde.org/show_bug.cgi?id=520170 --- Comment #10 from Michael Catanzaro --- You weren't kidding about it being large. I have temporarily hosted it here: https://gitlab.gnome.org/uploads/-/system/personal_snippet/7785/431a2cf0fa7c494c80b49ad4debe29e8/valgrind_output.tar.xz -- You are receiving this mail because: You are watching all bug changes.
[valgrind] [Bug 520170] valgrind reports Mismatched free() / delete / delete [], but it doesn't appear to be mismatched
https://bugs.kde.org/show_bug.cgi?id=520170 --- Comment #9 from Paul Floyd --- I still don't have much of an idea what is happening. Could you attach the output with --trace-redir=yes --trace-symtab=yes? The output might be very large. -- You are receiving this mail because: You are watching all bug changes.
[valgrind] [Bug 520170] valgrind reports Mismatched free() / delete / delete [], but it doesn't appear to be mismatched
https://bugs.kde.org/show_bug.cgi?id=520170 --- Comment #8 from Paul Floyd --- Not having free_sized is not a problem. glibc doesn't take any advantage of it even when it is present. -- You are receiving this mail because: You are watching all bug changes.
[valgrind] [Bug 520170] valgrind reports Mismatched free() / delete / delete [], but it doesn't appear to be mismatched
https://bugs.kde.org/show_bug.cgi?id=520170 --- Comment #7 from Michael Catanzaro --- Whoops, I meant to write: free_sized is new in glibc 2.43. -- You are receiving this mail because: You are watching all bug changes.
[valgrind] [Bug 520170] valgrind reports Mismatched free() / delete / delete [], but it doesn't appear to be mismatched
https://bugs.kde.org/show_bug.cgi?id=520170 --- Comment #6 from Michael Catanzaro --- (In reply to Paul Floyd from comment #5) > Could you send the output of > nm -D /lib64/libstdc++.so.6 | grep _Zn > nm -D /lib64/libstdc++.so.6 | grep _Zd > and > nm /lib64/libc.so.6 | grep " free" > For the last one there are several functions that start with free, I'm only > interest in free and free_sized. Please change the path if you are not using > the system libc.so and libstdc__.so. Sure. We need to use the libc.so.6 provided by freedesktop-sdk. It took me a while to figure out that I also have to always use 'nm -D' because freedesktop-sdk's libc.so.6 is stripped and contains no non-dynamic symbols. $ flatpak run -d --command=bash org.gnome.Builder.Devel [📦 org.gnome.Builder.Devel ~]$ nm /usr/lib/x86_64-linux-gnu/libc.so.6 nm: /usr/lib/x86_64-linux-gnu/libc.so.6: no symbols Using nm -D: [📦 org.gnome.Builder.Devel ~]$ nm -D /usr/lib/x86_64-linux-gnu/libc.so.6 | grep free 000ab9d0 T cfree@GLIBC_2.2.5 000ab9d0 T free@@GLIBC_2.2.5 0014d020 T freeaddrinfo@@GLIBC_2.2.5 001f71c8 V __free_hook@GLIBC_2.2.5 0012dd90 W freeifaddrs@@GLIBC_2.3 00037060 T __freelocale@@GLIBC_2.2.5 00037060 W freelocale@@GLIBC_2.3 000f33c0 T globfree@@GLIBC_2.2.5 000f33c0 W globfree64@@GLIBC_2.2.5 0012c710 W if_freenameindex@@GLIBC_2.2.5 00093a20 T _IO_free_backup_area@@GLIBC_2.2.5 00089dd0 T _IO_free_wbackup_area@@GLIBC_2.2.5 000ab9d0 T __libc_free@@GLIBC_2.2.5 000adb40 T __libc_freeres@@GLIBC_2.2.5 000ad2f0 T _obstack_free@@GLIBC_2.2.5 000ad2f0 T obstack_free@@GLIBC_2.2.5 0011f480 W pkey_free@@GLIBC_2.27 001046d0 W regfree@@GLIBC_2.2.5 001054f0 T __sched_cpufree@@GLIBC_2.7 0010ae90 T wordfree@@GLIBC_2.2.5 00167b80 T xdr_free@GLIBC_2.2.5 The free looks like yours. But notice ***there is no free_sized*** unlike on my Fedora 44 host system, where it's listed between freelocale and globfree. Fedora 44 has glibc 2.43, but freedesktop-sdk 25.08 has glibc 2.42, and free_sized is new in glibc 2.42, so that's actually expected. Is that bad? On to libstdc++: [📦 org.gnome.Builder.Devel ~]$ nm -D /usr/lib/x86_64-linux-gnu/libstdc++.so | grep _Zn 000ccbd0 T _Znam@@GLIBCXX_3.4 000ccbe0 T _ZnamRKSt9nothrow_t@@GLIBCXX_3.4 000cccd0 T _ZnamSt11align_val_t@@CXXABI_1.3.11 000ccce0 T _ZnamSt11align_val_tRKSt9nothrow_t@@CXXABI_1.3.11 000ccb60 T _Znwm@@GLIBCXX_3.4 000ccbb0 T _ZnwmRKSt9nothrow_t@@GLIBCXX_3.4 000ccc00 T _ZnwmSt11align_val_t@@CXXABI_1.3.11 000cccb0 T _ZnwmSt11align_val_tRKSt9nothrow_t@@CXXABI_1.3.11 [📦 org.gnome.Builder.Devel ~]$ nm -D /usr/lib/x86_64-linux-gnu/libstdc++.so | grep _Zd 000cab70 T _ZdaPv@@GLIBCXX_3.4 000cab80 T _ZdaPvm@@CXXABI_1.3.9 000ccd50 T _ZdaPvmSt11align_val_t@@CXXABI_1.3.11 000cab90 T _ZdaPvRKSt9nothrow_t@@GLIBCXX_3.4 000ccd30 T _ZdaPvSt11align_val_t@@CXXABI_1.3.11 000ccd40 T _ZdaPvSt11align_val_tRKSt9nothrow_t@@CXXABI_1.3.11 000cab40 T _ZdlPv@@GLIBCXX_3.4 000cab50 T _ZdlPvm@@CXXABI_1.3.9 000ccd20 T _ZdlPvmSt11align_val_t@@CXXABI_1.3.11 000cab60 T _ZdlPvRKSt9nothrow_t@@GLIBCXX_3.4 000ccd00 T _ZdlPvSt11align_val_t@@CXXABI_1.3.11 000ccd10 T _ZdlPvSt11align_val_tRKSt9nothrow_t@@CXXABI_1.3.11 This all looks the same as on my Fedora 44 host system (just with different offsets). -- You are receiving this mail because: You are watching all bug changes.
[valgrind] [Bug 520170] valgrind reports Mismatched free() / delete / delete [], but it doesn't appear to be mismatched
https://bugs.kde.org/show_bug.cgi?id=520170 --- Comment #5 from Paul Floyd --- For Claude's analysis, this is the interesting bit: === **Dealloc side:** vte emits `call _ZdlPvm@plt`. The thunk chain is `_ZdlPvm → _ZdlPv → free`. On fdo-sdk 25.08 glibc (≥ 2.41 exports `free_sized`), `free` and `free_sized` share an implementation; valgrind's scanner ends up tagging the call via the `free_sized` intercept → `MC_AllocMalloc`. This is the `free_sized (vg_replace_malloc.c:996)` frame in the report. **Alloc side:** report shows the call going through the `_Znam` intercept (line 717) even though the source calls `::operator new`. Most plausible cause: fdo-sdk libstdc++ is built with link-time ICF collapsing `_Znwm` and `_Znam` to one address; valgrind picks `_Znam` as the canonical redirect. On my Fedora 43 host they're at distinct addresses (0x1f3e0 vs 0x1f440) so I can't reproduce this side directly, but it's the only mechanism I can see that produces the symptom from a `call _Znwm@plt`. Result: alloc `MC_AllocNewVec` vs free `MC_AllocMalloc` → mismatch. === `free` and `free_sized` share an implementation I do not believe that is correct. Testing on Fedora 44 nm /lib64/libc.so.6 | grep " free" 000843f0 T free 00084c60 W free_sized This is just making a sibling call to free: 00084c60 : 84c60: f3 0f 1e fa endbr64 84c64: e9 97 bb f7 ff jmp800 84c69: 0f 1f 80 00 00 00 00nopl 0x0(%rax) In /lib64/libstdc++.so.6 all of the operator new and operator delete overloads are global T symbols - no aliases. nm -D /lib64/libstdc++.so.6 | grep _Zn 00021340 T _Znam@@GLIBCXX_3.4 00021350 T _ZnamRKSt9nothrow_t@@GLIBCXX_3.4 00021410 T _ZnamSt11align_val_t@@CXXABI_1.3.11 00021420 T _ZnamSt11align_val_tRKSt9nothrow_t@@CXXABI_1.3.11 000212e0 T _Znwm@@GLIBCXX_3.4 00021320 T _ZnwmRKSt9nothrow_t@@GLIBCXX_3.4 00021370 T _ZnwmSt11align_val_t@@CXXABI_1.3.11 000213f0 T _ZnwmSt11align_val_tRKSt9nothrow_t@@CXXABI_1.3.11 nm -D /lib64/libstdc++.so.6 | grep _Zd 0001f310 T _ZdaPv@@GLIBCXX_3.4 0001f320 T _ZdaPvm@@CXXABI_1.3.9 00021490 T _ZdaPvmSt11align_val_t@@CXXABI_1.3.11 0001f330 T _ZdaPvRKSt9nothrow_t@@GLIBCXX_3.4 00021470 T _ZdaPvSt11align_val_t@@CXXABI_1.3.11 00021480 T _ZdaPvSt11align_val_tRKSt9nothrow_t@@CXXABI_1.3.11 0001f2e0 T _ZdlPv@@GLIBCXX_3.4 0001f2f0 T _ZdlPvm@@CXXABI_1.3.9 00021460 T _ZdlPvmSt11align_val_t@@CXXABI_1.3.11 0001f300 T _ZdlPvRKSt9nothrow_t@@GLIBCXX_3.4 00021440 T _ZdlPvSt11align_val_t@@CXXABI_1.3.11 00021450 T _ZdlPvSt11align_val_tRKSt9nothrow_t@@CXXABI_1.3.11 The assembler for sized scalar delete: 8830 <_ZdlPvm@plt>: 8830: f3 0f 1e fa endbr64 8834: ff 25 1e 44 2a 00 jmp*0x2a441e(%rip)# 2acc58 <_ZdlPvm@@CXXABI_1.3.9+0x28d968> 883a: 66 0f 1f 44 00 00 nopw 0x0(%rax,%rax,1) that leads to a sibling call to plain scalar delete. and sized vector delete: 0001f320 <_ZdaPvm@@CXXABI_1.3.9>: 1f320: f3 0f 1e fa endbr64 1f324: e9 77 5c fe ff jmp4fa0 <_ZdaPv@plt> 1f329: 0f 1f 80 00 00 00 00nopl 0x0(%rax) You gdb output is what I would expect. I still don't understand how the call to _ZdlPvm (sized scalar operator delete) gets to free_sized. Valgrind should be redirecting _ZdlPvm and using its own allocator, it should never reach glibc free_sized. Could you send the output of nm -D /lib64/libstdc++.so.6 | grep _Zn nm -D /lib64/libstdc++.so.6 | grep _Zd and nm /lib64/libc.so.6 | grep " free" For the last one there are several functions that start with free, I'm only interest in free and free_sized. Please change the path if you are not using the system libc.so and libstdc__.so. -- You are receiving this mail because: You are watching all bug changes.
[valgrind] [Bug 520170] valgrind reports Mismatched free() / delete / delete [], but it doesn't appear to be mismatched
https://bugs.kde.org/show_bug.cgi?id=520170 --- Comment #4 from Michael Catanzaro --- Created attachment 192322 --> https://bugs.kde.org/attachment.cgi?id=192322&action=edit AI analysis Somebody fed this bug report to an AI program (I think it is Claude) as an experiment. It seems pretty firmly convinced that the problem is likely identical code folding in libstdc++, possibly because you mentioned the possibility in your comment above. Only one problem: I don't know how to evaluate whether that is actually correct or not. Also, the agent did not run in the affected environment, so it was not actually able to reproduce the problem. But perhaps its report will be useful nonetheless. It certainly *seems* to be more educated than me -- You are receiving this mail because: You are watching all bug changes.
[valgrind] [Bug 520170] valgrind reports Mismatched free() / delete / delete [], but it doesn't appear to be mismatched
https://bugs.kde.org/show_bug.cgi?id=520170 --- Comment #3 from Michael Catanzaro --- Created attachment 192315 --> https://bugs.kde.org/attachment.cgi?id=192315&action=edit disassembled function -- You are receiving this mail because: You are watching all bug changes.
[valgrind] [Bug 520170] valgrind reports Mismatched free() / delete / delete [], but it doesn't appear to be mismatched
https://bugs.kde.org/show_bug.cgi?id=520170 --- Comment #2 from Michael Catanzaro --- (In reply to Paul Floyd from comment #1) > Let's say your app links with liba and libb and they link > > liba -> dynamic libstdc++.so > libb -> static libstdc++.a The allocation and deallocation both occur in vte at termpropsregistry.cc:199, so at least there is only one library involved. > 2. If you use "identical code folding" then the linker might be seeing that > deallocation functions have identical machine code and only generating one > version. I don't think vte or freedesktop-sdk uses any special linker flags to enable such behavior, so if it's not something that ld.bfd performs by default, then that seems unlikely? > 3. If you use tcmalloc, that manually does identical code folding and only > generates one alias for free/delete/delete[]. No tcmalloc involved. Thanks for the tips. Looks like this is going to be pretty hard to track down. :( I don't have experience with reading assembly code, and the disassembled function where the allocation and deallocation occurs is pretty complicated at 2377 lines long, since it contains lots of inlined C++ functions. But at least it's the same function for both the allocation and deallocation, _ZN3vte8property8Registry12install_manyESt16initializer_listINS1_8PropertyEE. I'm going to attach the disassembly. Although I can only barely understand what I see, I do notice something interesting: there is a correspondence between the target of the call instructions and gobbledygook in vg_replace_malloc.c. So despite my inexperience, that seems clearly interesting. At least it *almost* explains why we wind up where we do. For example, the disassembly contains: 0x764c95f9 <+185>: call 0x7647bf40 <_ZdlPvm@plt> and the same at +1567 and +2203. And hey! that _ZdlPvm is a match for the /*--- C++14 delete sized ---*/ section of vg_replace_malloc.c: // operator delete(void*, unsigned long) #elif __SIZEOF_SIZE_T__ == 8 DELETE_SIZED(VG_Z_LIBSTDCXX_SONAME, _ZdlPvm, __builtin_delete, DeleteSized ); DELETE_SIZED(VG_Z_LIBCXX_SONAME, _ZdlPvm, __builtin_delete, DeleteSized ); DELETE_SIZED(VG_Z_LIBC_SONAME, _ZdlPvm, __builtin_delete, DeleteSized ); DELETE_SIZED(SO_SYN_MALLOC, _ZdlPvm, __builtin_delete, DeleteSized ); #endif I also see: 0x764c98ac <+876>: call 0x7647c050 <_Znwm@plt> and the same at +1066 and +1685. That matches: // operator new(unsigned long) #if VG_WORDSIZE == 8 ALLOC_or_BOMB(VG_Z_LIBSTDCXX_SONAME, _Znwm, __builtin_new); ALLOC_or_BOMB(VG_Z_LIBCXX_SONAME,_Znwm, __builtin_new); ALLOC_or_BOMB(VG_Z_LIBC_SONAME, _Znwm, __builtin_new); ALLOC_or_BOMB(SO_SYN_MALLOC, _Znwm, __builtin_new); #endif With the magic of gdb breakpoints, I see that a breakpoint for std::__new_allocator::allocate stops here: => 0x764c9958 <+1048>: lea(%rax,%rax,2),%rax 0x764c995c <+1052>: shl$0x4,%rax 0x764c9960 <+1056>: mov%rax,%rdi 0x764c9963 <+1059>: mov%rax,-0xc8(%rbp) 0x764c996a <+1066>: call 0x7647c050 <_Znwm@plt> where it's about to call operator new (not operator new[]). A breakpoint for std::__new_allocator::deallocate stops here: => 0x764c9b59 <+1561>: mov%rcx,%rdi 0x764c9b5c <+1564>: sub%rcx,%rsi 0x764c9b5f <+1567>: call 0x7647bf40 <_ZdlPvm@plt> before the call to operator delete. What's especially weird is what I do *not* see: there is no _Znam anywhere in the disassembly, which is very strange because there ought to be, because valgrind's stack trace shows that its operator new[] is being called: ==84== Address 0x1b841470 is 0 bytes inside a block of size 48 alloc'd ==84==at 0x4FE55F3: operator new[](unsigned long) (vg_replace_malloc.c:717) And the gobbledygook for operator new[] on 64-bit Linux is _Znam: // operator new[](unsigned long), #if VG_WORDSIZE == 8 ALLOC_or_BOMB(VG_Z_LIBSTDCXX_SONAME, _Znam, __builtin_vec_new ); ALLOC_or_BOMB(VG_Z_LIBCXX_SONAME,_Znam, __builtin_vec_new ); ALLOC_or_BOMB(VG_Z_LIBC_SONAME, _Znam, __builtin_vec_new ); ALLOC_or_BOMB(SO_SYN_MALLOC, _Znam, __builtin_vec_new ); #endif I suppose that's a good thing, because it sure looks like the libstdc++ source code wants operator new, not operator new[]. So the disassembly appears to correspond to the source code, but it does not correspond to what valgrind reports. That's pretty weird! -- You are receiving this mail because: You are watching all bug changes.
[valgrind] [Bug 520170] valgrind reports Mismatched free() / delete / delete [], but it doesn't appear to be mismatched
https://bugs.kde.org/show_bug.cgi?id=520170 --- Comment #1 from Paul Floyd --- (In reply to Michael Catanzaro from comment #0) > DESCRIPTION > > I'm trying to run GNOME Builder under valgrind 3.27.0 because I strongly > suspect there is a memory corruption issue somewhere in vte. I think > valgrind might be printing false positive complaints about mismatched > free/delete/delete[]. I hope not. > I've been doing some squinting, and it doesn't appear > to be a mismatch. The most common cause of issues like this is that some combination of the compiler, linker and library is presenting inconsistent information to Valgrind. Inconsistent as in what Valgrind sees in the machine code does not match what was written in the source code. Possible causes for this are 1. Mixing use of sibling calls between allocation and deallocation. I'd say that this is the most likely, at least for the reason that the deallocation is not seen as operator delete. For instance I've seen this in a large project like this, mixing static and dynamic versions of libstdc++ Let's say your app links with liba and libb and they link liba -> dynamic libstdc++.so libb -> static libstdc++.a In this configuration if you have an object allocated in liba then Valgrind will see all of the allocation function symbols. If that object is then deallocated in libb there may be some optimisations such is sibling calls that hide call information from valgrind. Specifically in this case that could be deallocate -> sibling call to sized operator delete[] -> normal call to free_sized. Valgrind does not see the sibling call to operator delete[] and so it reports is mismatch. 2. If you use "identical code folding" then the linker might be seeing that deallocation functions have identical machine code and only generating one version. 3. If you use tcmalloc, that manually does identical code folding and only generates one alias for free/delete/delete[]. The reason why Valgrind sees operator delete[] rather than operator delete could be identical code folding. To be certain of what is happening you need to examine the assembler to see what is really being called (ot, in the case of sibling calls, jumped to). > I also wondered: is it possible that the memory was allocated in code built > by one compiler, and deallocated in code built by a different compiler, such > that the definitions really do become mismatched? But in this example, both > allocation and deallocation occur in vte's termpropsregistry.cc, so it's the > same source code file. All the compilers that Valgrind supports use GCC name mangling so that should not be an issue. -- You are receiving this mail because: You are watching all bug changes.
[valgrind] [Bug 520170] valgrind reports Mismatched free() / delete / delete [], but it doesn't appear to be mismatched
https://bugs.kde.org/show_bug.cgi?id=520170 Paul Floyd changed: What|Removed |Added Assignee|[email protected] |[email protected] CC||[email protected] -- You are receiving this mail because: You are watching all bug changes.
