[kdevelop] [Bug 367841] Crash during parsing of cmake/C++/Python files

[Kevin Funk via KDE Bugzilla]

https://bugs.kde.org/show_bug.cgi?id=367841

Kevin Funk  changed:

   What|Removed |Added

 Resolution|--- |DUPLICATE
 Status|UNCONFIRMED |RESOLVED

--- Comment #9 from Kevin Funk  ---
Thanks a ton for the analysis, Axel.

And yes, those two bug reports have the same root cause => I'll merge them.

*** This bug has been marked as a duplicate of bug 363269 ***

-- 
You are receiving this mail because:
You are watching all bug changes.

[kdevelop] [Bug 367841] Crash during parsing of cmake/C++/Python files

[via KDE Bugzilla]

https://bugs.kde.org/show_bug.cgi?id=367841

--- Comment #8 from axel.kellerm...@gmx.de ---
I found the problem, and it's actually similar, but not the same as bug 363269.
The code base being parsed contains a subfolder with example text in a couple
of different asian languages, all stored as .txt files in UTF16LE. The cmake
parser seems to not just read CMakeLists.txt and *.cmake, but also generally
all files with the extension .txt. Apparently the encoding of the example files
is handled correctly, but then the wide characters are given to the lexer that
expects chars, hence the one byte buffer overflow reported (just my guess,
didn't have a look at all the code involved).

If I delete all the UTF16 files, the whole code base is processed successfully.

So I guess the problem is kind of a duplicate of 363269, but more specific. The
cmake parser seems generally to greedy in selecting the files it processes.
Could that be narrowed down to only CMakeLists.txt and *.cmake?

Feel free to close this one if you think it's covered by 363269.

-- 
You are receiving this mail because:
You are watching all bug changes.

[kdevelop] [Bug 367841] Crash during parsing of cmake/C++/Python files

[Kevin Funk via KDE Bugzilla]

https://bugs.kde.org/show_bug.cgi?id=367841

--- Comment #7 from Kevin Funk  ---
Could that be related to bug 363269? Could you check the CMake file KDevelop is
parsing (see debug output). Is it a binary file?

-- 
You are receiving this mail because:
You are watching all bug changes.

[kdevelop] [Bug 367841] Crash during parsing of cmake/C++/Python files

[via KDE Bugzilla]

https://bugs.kde.org/show_bug.cgi?id=367841

--- Comment #6 from axel.kellerm...@gmx.de ---
Created attachment 100989
  --> https://bugs.kde.org/attachment.cgi?id=100989=edit
ASAN output of heap-buffer-overflow error during parsing of CMake file

-- 
You are receiving this mail because:
You are watching all bug changes.

[kdevelop] [Bug 367841] Crash during parsing of cmake/C++/Python files

[via KDE Bugzilla]

https://bugs.kde.org/show_bug.cgi?id=367841

--- Comment #5 from axel.kellerm...@gmx.de ---
I compiled kdevelop and kdevplatform with ASAN and tested again on two
platforms:
1. A laptop with only 4GB of RAM, where memory actually is kind of an issue.
kdevelop takes up ~3.5GB of RAM
and the system starts to heavyly swap out pages. But the limits of the swap
partition are never reached, so
parsing gets very slow, but never runs out of memory.
2. A PC with 16GB of RAM, where memory is definitely no issue.

On both machines ASAN reports the same heap-buffer-overflow error during
parsing of cmake files (attached as asan_readCMakeFile_20160908.txt).

-- 
You are receiving this mail because:
You are watching all bug changes.

[kdevelop] [Bug 367841] Crash during parsing of cmake/C++/Python files

[via KDE Bugzilla]

https://bugs.kde.org/show_bug.cgi?id=367841

--- Comment #2 from axel.kellerm...@gmx.de ---
And another one:

#0  0x00300147e4f5 in malloc_consolidate (av=av@entry=0x7fffa420) at
malloc.c:4184
#1  0x00300148155e in _int_malloc (av=av@entry=0x7fffa420,
bytes=bytes@entry=19684) at malloc.c:3451
#2  0x0030014835a4 in __GI___libc_malloc (bytes=19684) at malloc.c:2914
#3  0x75cf7eb8 in QArrayData::allocate(unsigned long, unsigned long,
unsigned long, QFlags) () from
/usr/lib/x86_64-linux-gnu/libQt5Core.so.5
#4  0x75cfa9e1 in QByteArray::resize(int) () from
/usr/lib/x86_64-linux-gnu/libQt5Core.so.5
#5  0x75e00a29 in QIODevice::readAll() () from
/usr/lib/x86_64-linux-gnu/libQt5Core.so.5
#6  0x73975690 in KDevelop::ParseJob::readContents (this=0x34641d0) at
/home/akellerm/Repos/kdesrc/kdevplatform/language/backgroundparser/parsejob.cpp:316
#7  0x7fffbab19fd0 in CMakeParseJob::run (this=0x34641d0) at
/home/akellerm/Repos/kdesrc/kdevelop/projectmanagers/cmake/duchain/cmakeparsejob.cpp:59
#8  0x7fffec1dc650 in
ThreadWeaver::IdDecorator::run(QSharedPointer,
ThreadWeaver::Thread*) () from
/usr/lib/x86_64-linux-gnu/libKF5ThreadWeaver.so.5
#9  0x7fffec1dcc90 in
ThreadWeaver::Executor::run(QSharedPointer const&,
ThreadWeaver::Thread*) () from
/usr/lib/x86_64-linux-gnu/libKF5ThreadWeaver.so.5
#10 0x7fffec1db7e0 in
ThreadWeaver::Job::execute(QSharedPointer const&,
ThreadWeaver::Thread*) () from
/usr/lib/x86_64-linux-gnu/libKF5ThreadWeaver.so.5
#11 0x7fffec1db28a in ThreadWeaver::Thread::run() () from
/usr/lib/x86_64-linux-gnu/libKF5ThreadWeaver.so.5
#12 0x75cf684e in ?? () from /usr/lib/x86_64-linux-gnu/libQt5Core.so.5
#13 0x0030024076fa in start_thread (arg=0x7fffa700) at
pthread_create.c:333
#14 0x003001506b5d in clone () at
../sysdeps/unix/sysv/linux/x86_64/clone.S:109

-- 
You are receiving this mail because:
You are watching all bug changes.

[kdevelop] [Bug 367841] Crash during parsing of cmake/C++/Python files

[via KDE Bugzilla]

https://bugs.kde.org/show_bug.cgi?id=367841

--- Comment #3 from axel.kellerm...@gmx.de ---
I'd be happy to help with debugging, but I need some pointers where to start,
and what to look for.

-- 
You are receiving this mail because:
You are watching all bug changes.

[kdevelop] [Bug 367841] Crash during parsing of cmake/C++/Python files

[via KDE Bugzilla]

https://bugs.kde.org/show_bug.cgi?id=367841

axel.kellerm...@gmx.de changed:

   What|Removed |Added

Summary|Crash during parsing of |Crash during parsing of
   |C++/Python files|cmake/C++/Python files

--- Comment #1 from axel.kellerm...@gmx.de ---
Encountered another crash during parsing of cmake files:

#0  0x00300147fbd6 in _int_free (av=0x7fffa820, p=,
have_lock=0) at malloc.c:4015
#1  0x003001483abc in __GI___libc_free (mem=) at
malloc.c:2969
#2  0x7fffc0cb2a74 in cmListFileLexer_yyfree (ptr=0x7fffa962ceb0,
yyscanner=0x7fffa962ceb0) at cmListFileLexer.c:2417
#3  0x7fffc0cb29ea in cmListFileLexer_yylex_destroy
(yyscanner=0x7fffa962ceb0) at cmListFileLexer.c:2359
#4  0x7fffc0cb2ee3 in cmListFileLexerDestroy (lexer=0x7fffa962c750) at
cmListFileLexer.in.l:421
#5  0x7fffc0cb31a1 in cmListFileLexer_SetFileName (lexer=0x7fffa962c750,
name=0x0, bom=0x0) at cmListFileLexer.in.l:504
#6  0x7fffc0cb2fc7 in cmListFileLexer_Delete (lexer=0x7fffa962c750) at
cmListFileLexer.in.l:454
#7  0x7fffc0cb4382 in CMakeListsParser::readCMakeFile (_fileName=...) at
/home/akellerm/Repos/kdesrc/kdevelop/projectmanagers/cmake/parser/cmakelistsparser.cpp:148
#8  0x7fffbab1a58e in CMakeParseJob::run (this=0x3522310) at
/home/akellerm/Repos/kdesrc/kdevelop/projectmanagers/cmake/duchain/cmakeparsejob.cpp:98
#9  0x7fffec1dc650 in
ThreadWeaver::IdDecorator::run(QSharedPointer,
ThreadWeaver::Thread*) () from
/usr/lib/x86_64-linux-gnu/libKF5ThreadWeaver.so.5
#10 0x7fffec1dcc90 in
ThreadWeaver::Executor::run(QSharedPointer const&,
ThreadWeaver::Thread*) () from
/usr/lib/x86_64-linux-gnu/libKF5ThreadWeaver.so.5
#11 0x7fffec1db7e0 in
ThreadWeaver::Job::execute(QSharedPointer const&,
ThreadWeaver::Thread*) () from
/usr/lib/x86_64-linux-gnu/libKF5ThreadWeaver.so.5
#12 0x7fffec1db28a in ThreadWeaver::Thread::run() () from
/usr/lib/x86_64-linux-gnu/libKF5ThreadWeaver.so.5
#13 0x75cf684e in ?? () from /usr/lib/x86_64-linux-gnu/libQt5Core.so.5
#14 0x0030024076fa in start_thread (arg=0x7fffb8cd4700) at
pthread_create.c:333
#15 0x003001506b5d in clone () at
../sysdeps/unix/sysv/linux/x86_64/clone.S:109

-- 
You are receiving this mail because:
You are watching all bug changes.