Re: [Development] CI failing due to ASan tst_selftest
On Tuesday, 5 September 2023 04:43:52 PDT Ahmad Samir wrote: > I see the same issue locally, so I used `git bisect`, which showed it's > because in `checkErrorOutput(const QString , const QByteArray > )` in tst_selftests.cpp, the code should return early for > "silent" test, because the latter calls qFatal, that doesn't seem to happen > on Linux any more. Indeed, but that only happens in ASan builds. Anyway, I've submitted changes to deal with that. -- Thiago Macieira - thiago (AT) macieira.info - thiago (AT) kde.org Software Architect - Intel DCAI Cloud Engineering smime.p7s Description: S/MIME cryptographic signature -- Development mailing list Development@qt-project.org https://lists.qt-project.org/listinfo/development
Re: [Development] CI failing due to ASan tst_selftest
On 27/8/23 05:36, Thiago Macieira wrote: See https://testresults.qt.io/coin/integration/qt/qtbase/tasks/1698516419 [...] Looking at this log: ==19313==ERROR: LeakSanitizer: detected memory leaks agent:2023/08/26 21:36:18 build.go:405: 3: agent:2023/08/26 21:36:18 build.go:405: 3: Direct leak of 258 byte(s) in 1 object(s) allocated from: agent:2023/08/26 21:36:18 build.go:405: 3: #0 0x7ffa505c8e48 in __interceptor_malloc (/usr/lib64/libasan.so.5+0x109e48) agent:2023/08/26 21:36:18 build.go:405: 3: #1 0x7ffa4f2d7ff9 (/home/qt/work/install/lib/libQt6Core.so.6+0x896ff9) agent:2023/08/26 21:36:18 build.go:405: 3: #2 0x7ffa4f2d834d in QArrayData::allocate(QArrayData**, long long, long long, long long, QArrayData::AllocationOption) (/home/qt/work/install/lib/libQt6Core.so.6+0x89734d) agent:2023/08/26 21:36:18 build.go:405: 3: #3 0x7ffa4f23b700 (/home/qt/work/install/lib/libQt6Core.so.6+0x7fa700) agent:2023/08/26 21:36:18 build.go:405: 3: #4 0x7ffa4f1f6cc8 in QString::reallocData(long long, QArrayData::AllocationOption) (/home/qt/work/install/lib/libQt6Core.so.6+0x7b5cc8) agent:2023/08/26 21:36:18 build.go:405: 3: #5 0x7ffa4f1f68a7 in QString::resize(long long) (/home/qt/work/install/lib/libQt6Core.so.6+0x7b58a7) agent:2023/08/26 21:36:18 build.go:405: 3: #6 0x7ffa4f2092ff (/home/qt/work/install/lib/libQt6Core.so.6+0x7c82ff) agent:2023/08/26 21:36:18 build.go:405: 3: #7 0x7ffa4f209e09 in QString::vasprintf(char const*, __va_list_tag*) (/home/qt/work/install/lib/libQt6Core.so.6+0x7c8e09) agent:2023/08/26 21:36:18 build.go:405: 3: #8 0x7ffa4ed0d83d (/home/qt/work/install/lib/libQt6Core.so.6+0x2cc83d) agent:2023/08/26 21:36:18 build.go:405: 3: #9 0x7ffa4ed114a9 in QMessageLogger::fatal(char const*, ...) const (/home/qt/work/install/lib/libQt6Core.so.6+0x2d04a9) agent:2023/08/26 21:36:18 build.go:405: 3: #10 0x5641d2604c40 in tst_Silent::messages() /home/qt/work/qt/qtbase/tests/auto/testlib/selftests/silent/tst_silent.cpp:77 I see the same issue locally, so I used `git bisect`, which showed it's because in `checkErrorOutput(const QString , const QByteArray )` in tst_selftests.cpp, the code should return early for "silent" test, because the latter calls qFatal, that doesn't seem to happen on Linux any more. Regards, Ahmad Samir OpenPGP_signature Description: OpenPGP digital signature -- Development mailing list Development@qt-project.org https://lists.qt-project.org/listinfo/development
Re: [Development] CI failing due to ASan tst_selftest
On Sonntag, 27. August 2023 17:42:58 CEST Thiago Macieira wrote: > On Sunday, 27 August 2023 02:35:39 PDT Ulf Hermann via Development wrote: > > I've seen this one before. I suspect certain versions of fontconfig have > > an internal memory leak. For me, it fixed itself after an upgrade of > > some packages. Can we install debug symbols for fontconfig on the > > respective platforms? Then we could see a proper stack trace. > > That rings a bell, indeed. > > But I'm not convinced it's a leak, as opposed to reachable memory that > wasn't freed before exit. LSAN doesn't do a still-reachable analysis. > I also don't think debug symbols will help here, since this is coming from > the dynamic symbols of the module in question, not from the debug info. If > you look at the qFatal() leak I fixed, you see that there are unknown > QtCore frames printed, meaning the stack trace didn't use debug symbols. By default, LSAN will rely purely on frame pointers. As such, if fontconfig is compiled without them, the backtraces will be unhelpful. You can force usage of .eh_frame et al by exporting LSAN_OPTIONS=fast_unwind_on_malloc=false But this option is really very slow, as the sanitizers don't use a caching unwinding library. I do not recommend running tests with this always enabled, but for one-off investigations it should be useful. Hope that helps -- Milian Wolff m...@milianw.de http://milianw.de signature.asc Description: This is a digitally signed message part. -- Development mailing list Development@qt-project.org https://lists.qt-project.org/listinfo/development
Re: [Development] CI failing due to ASan tst_selftest
On Sunday, 27 August 2023 02:35:39 PDT Ulf Hermann via Development wrote: > I've seen this one before. I suspect certain versions of fontconfig have > an internal memory leak. For me, it fixed itself after an upgrade of > some packages. Can we install debug symbols for fontconfig on the > respective platforms? Then we could see a proper stack trace. That rings a bell, indeed. But I'm not convinced it's a leak, as opposed to reachable memory that wasn't freed before exit. I also don't think debug symbols will help here, since this is coming from the dynamic symbols of the module in question, not from the debug info. If you look at the qFatal() leak I fixed, you see that there are unknown QtCore frames printed, meaning the stack trace didn't use debug symbols. -- Thiago Macieira - thiago.macieira (AT) intel.com Cloud Software Architect - Intel DCAI Cloud Engineering smime.p7s Description: S/MIME cryptographic signature -- Development mailing list Development@qt-project.org https://lists.qt-project.org/listinfo/development
Re: [Development] CI failing due to ASan tst_selftest
==19383==ERROR: LeakSanitizer: detected memory leaks Direct leak of 48 byte(s) in 2 object(s) allocated from: #0 0x7f26aec7ae48 in __interceptor_malloc (/usr/lib64/libasan.so. 5+0x109e48) #1 0x7f26a8bcbc50 (/usr/lib64/libfontconfig.so.1+0xbc50) #2 0x3d4c4c415f434bff () I've seen this one before. I suspect certain versions of fontconfig have an internal memory leak. For me, it fixed itself after an upgrade of some packages. Can we install debug symbols for fontconfig on the respective platforms? Then we could see a proper stack trace. best regards, Ulf -- Development mailing list Development@qt-project.org https://lists.qt-project.org/listinfo/development
[Development] CI failing due to ASan tst_selftest
See https://testresults.qt.io/coin/integration/qt/qtbase/tasks/1698516419 Two separate sets of changes are running into this problem, which means neither can be at fault. Tthis must be some other configuration that changed in the CI. The file in qt5.git that seems to control this build hasn't changed recently, so the change must not be in the Git repositories. It did find a real leak, which I fixed in https://codereview.qt-project.org/c/qt/qtbase/+/499608. All the rest seem to be false leaks (i.e., memory in use at the end of the program) and moreover we have no control over, like: ==19383==ERROR: LeakSanitizer: detected memory leaks Direct leak of 48 byte(s) in 2 object(s) allocated from: #0 0x7f26aec7ae48 in __interceptor_malloc (/usr/lib64/libasan.so. 5+0x109e48) #1 0x7f26a8bcbc50 (/usr/lib64/libfontconfig.so.1+0xbc50) #2 0x3d4c4c415f434bff () So it appears that whatever change this was, it made ASan's Leak Checker actually work, whereas before it was passing but not doing its job. Unfortunately, we're not clean of leaks, at least not of those in third-party libs. That means we have two possible actions to take: * turn off ASan * turn on the proper suppressions for ASan -- Thiago Macieira - thiago.macieira (AT) intel.com Cloud Software Architect - Intel DCAI Cloud Engineering smime.p7s Description: S/MIME cryptographic signature -- Development mailing list Development@qt-project.org https://lists.qt-project.org/listinfo/development
