[kdev-python] [Bug 378827] Kevelop 4.7, 5.0.x, and 5.1.0 crashes when background parsing is turned on for Unreal Engine 4
https://bugs.kde.org/show_bug.cgi?id=378827 Francis Herne changed: What|Removed |Added Version Fixed In||5.1.2 -- You are receiving this mail because: You are watching all bug changes.
[kdev-python] [Bug 378827] Kevelop 4.7, 5.0.x, and 5.1.0 crashes when background parsing is turned on for Unreal Engine 4
https://bugs.kde.org/show_bug.cgi?id=378827 Francis Herne changed: What|Removed |Added Latest Commit|https://commits.kde.org/kde |https://commits.kde.org/kde |v-python/6043407c4ee2e1b3ee |vplatform/396dda6a43402cade |fac8e1f8c96a651b92a14e |fbcffa4473d62d34de14175 Resolution|--- |FIXED Status|REOPENED|RESOLVED --- Comment #13 from Francis Herne --- Git commit 396dda6a43402cadefbcffa4473d62d34de14175 by Francis Herne. Committed on 26/07/2017 at 19:09. Pushed by flherne into branch '5.1'. Normalize line endings when creating ParseJob from background file. When the file being parsed is open in the editor, `ParseJob::readContents()` uses `KTextEditor::Document::text()` to read the contents. At some point within KTextEditor, the line endings are normalized - even if the document uses \r or \r\n, the returned content is separated by \n only. Otherwise, the contents are read directly from the file. Until this patch, the line endings aren't normalized in this case; the ParseJob contents differ depending on whether the file is open in the editor. Several places in kdev-python (and, at a glance, other language plugins) assume that lines can be separated by, and end with, \n. Old-style Macintosh endings (\r) break this assumption, in the worst case causing a crash. Using a regex in each instance would be a pain, and one code path normalizes line endings already, so I think the best option is to do so for background files also. Differential Revision: https://phabricator.kde.org/D5707 M +4-0language/backgroundparser/parsejob.cpp M +15 -0util/kdevstringhandler.cpp M +5-0util/kdevstringhandler.h M +29 -1util/tests/test_stringhandler.cpp M +3-0util/tests/test_stringhandler.h https://commits.kde.org/kdevplatform/396dda6a43402cadefbcffa4473d62d34de14175 -- You are receiving this mail because: You are watching all bug changes.
[kdev-python] [Bug 378827] Kevelop 4.7, 5.0.x, and 5.1.0 crashes when background parsing is turned on for Unreal Engine 4
https://bugs.kde.org/show_bug.cgi?id=378827 --- Comment #12 from Francis Herne --- No, see https://phabricator.kde.org/D5707 People wanted (reasonable) changes to my patch, but I was a bit burned-out and never made them. Something like mwolff's suggestion should be ok, I'll try it out and write some tests. -- You are receiving this mail because: You are watching all bug changes.
[kdev-python] [Bug 378827] Kevelop 4.7, 5.0.x, and 5.1.0 crashes when background parsing is turned on for Unreal Engine 4
https://bugs.kde.org/show_bug.cgi?id=378827 --- Comment #11 from Sven Brauch --- I don't recall -- did we ever fix this issue? I know we had a discussion and a RR ... -- You are receiving this mail because: You are watching all bug changes.
[kdev-python] [Bug 378827] Kevelop 4.7, 5.0.x, and 5.1.0 crashes when background parsing is turned on for Unreal Engine 4
https://bugs.kde.org/show_bug.cgi?id=378827
--- Comment #10 from Francis Herne ---
Correction, it's related to the use of MacOS Classic (why?!) line endings -
i.e. just a carriage return but no newline.
Many places in kdev-python assume that lines can be split with \n, so we should
probably normalise these earlier.
Alternative patch that solves this particular case:
diff --git a/parser/astbuilder.cpp b/parser/astbuilder.cpp
index 0e19f367..15df40a9 100644
--- a/parser/astbuilder.cpp
+++ b/parser/astbuilder.cpp
@@ -98,7 +98,8 @@ private:
class RangeFixVisitor : public AstDefaultVisitor {
public:
RangeFixVisitor(const QString& contents)
-: lines(contents.split('\n')) { };
+// Split on all three variants - \r, \n or \r\n
+: lines(contents.split(QRegExp("\\r\\n?|\\n"))) { };
void visitNode(Ast* node) override {
AstDefaultVisitor::visitNode(node);
--
You are receiving this mail because:
You are watching all bug changes.
[kdev-python] [Bug 378827] Kevelop 4.7, 5.0.x, and 5.1.0 crashes when background parsing is turned on for Unreal Engine 4
https://bugs.kde.org/show_bug.cgi?id=378827 --- Comment #9 from Francis Herne --- In this case, it's related to the use of Windows line endings in the file. When the file is parsed in the background (but not while opened in the editor), we fail to normalise them properly and the `lines` QStringList is given a single line containing the whole document. fixAlias() doesn't check that AST node positions exist in the document before trying to access them, and this fails when the line doesn't exist. -- You are receiving this mail because: You are watching all bug changes.
[kdev-python] [Bug 378827] Kevelop 4.7, 5.0.x, and 5.1.0 crashes when background parsing is turned on for Unreal Engine 4
https://bugs.kde.org/show_bug.cgi?id=378827 Francis Herne changed: What|Removed |Added Resolution|FIXED |--- Status|RESOLVED|REOPENED --- Comment #7 from Francis Herne --- Not fixed, it crashes in visitAlias instead. -- You are receiving this mail because: You are watching all bug changes.
[kdev-python] [Bug 378827] Kevelop 4.7, 5.0.x, and 5.1.0 crashes when background parsing is turned on for Unreal Engine 4
https://bugs.kde.org/show_bug.cgi?id=378827 --- Comment #8 from Francis Herne --- Sorry, fixAlias -- You are receiving this mail because: You are watching all bug changes.
[kdev-python] [Bug 378827] Kevelop 4.7, 5.0.x, and 5.1.0 crashes when background parsing is turned on for Unreal Engine 4
https://bugs.kde.org/show_bug.cgi?id=378827 Sven Brauch changed: What|Removed |Added Resolution|--- |FIXED Status|CONFIRMED |RESOLVED Latest Commit||https://commits.kde.org/kde ||v-python/6043407c4ee2e1b3ee ||fac8e1f8c96a651b92a14e --- Comment #6 from Sven Brauch --- Git commit 6043407c4ee2e1b3eefac8e1f8c96a651b92a14e by Sven Brauch. Committed on 01/05/2017 at 21:44. Pushed by brauch into branch '5.1'. fix crash This loop is obviously wrong, it iterates over size()+1 indices 0..size(), but there are only size(). M +1-1parser/astbuilder.cpp https://commits.kde.org/kdev-python/6043407c4ee2e1b3eefac8e1f8c96a651b92a14e -- You are receiving this mail because: You are watching all bug changes.
[kdev-python] [Bug 378827] Kevelop 4.7, 5.0.x, and 5.1.0 crashes when background parsing is turned on for Unreal Engine 4
https://bugs.kde.org/show_bug.cgi?id=378827 Francis Herne changed: What|Removed |Added Ever confirmed|0 |1 Priority|NOR |HI Status|UNCONFIRMED |CONFIRMED CC||m...@flherne.uk --- Comment #5 from Francis Herne --- Reproducible with UE master and kdev-python 5.1-git. === > AST > building abstract syntax tree for "/home/flh/projects/UnrealEngine/Engine/Extras/Maya_AnimationRiggingTools/ArtToolsOSX/MayaTools/General/Scripts/Modules/ART_Core.py" Got syntax tree from python parser: 1 1 ASSERT: "uint(i) < uint(size())" in file /usr/include/qt/QtCore/qstring.h, line 897 Thread 7 "Queue(0x8ab900)" received signal SIGABRT, Aborted. [Switching to Thread 0x7fffc4c43700 (LWP 27501)] 0x7fffef245a10 in raise () from /usr/lib/libc.so.6 (gdb) bt #0 0x7fffef245a10 in raise () at /usr/lib/libc.so.6 #1 0x7fffef24713a in abort () at /usr/lib/libc.so.6 #2 0x004058d6 in messageOutput(QtMsgType, QMessageLogContext const&, QString const&) (type=, context=..., msg=...) at /home/flh/projects/kdevplatform/util/duchainify/main.cpp:79 #3 0x7fffefefa298 in () at /usr/lib/libQt5Core.so.5 #4 0x7fffefefbe09 in QMessageLogger::fatal(char const*, ...) const () at /usr/lib/libQt5Core.so.5 #5 0x7fffefef73de in () at /usr/lib/libQt5Core.so.5 #6 0x7fffc68d13bc in QString::at(int) const (this=0x7fffc4c41360, i=-1) at /usr/include/qt/QtCore/qstring.h:897 #7 0x7fffc68e20af in Python::RangeFixVisitor::whitespaceAtEnd(QString const&) (this=0x7fffc4c41670, line=...) at /home/flh/projects/kdev-python/parser/astbuilder.cpp:443 #8 0x7fffc68e1fd6 in Python::RangeFixVisitor::fixAlias(Python::Ast*, Python::Ast*, int, int) (this=0x7fffc4c41670, dotted=0x7fffb8042fb0, asname=0x7fffb8042ff0, startLine=1, aliasIndex=0) at /home/flh/projects/kdev-python/parser/astbuilder.cpp:433 #9 0x7fffc68e12c4 in Python::RangeFixVisitor::visitImport(Python::ImportAst*) (this=0x7fffc4c41670, node=0x7fffb8059db0) at /home/flh/projects/kdev-python/parser/astbuilder.cpp:209 #10 0x7fffc68dbddc in Python::AstVisitor::visitNode(Python::Ast*) (this=0x7fffc4c41670, node=0x7fffb8059db0) at /home/flh/projects/kdev-python/parser/astvisitor.cpp:57 #11 0x7fffc68e0c55 in Python::RangeFixVisitor::visitNode(Python::Ast*) (this=0x7fffc4c41670, node=0x7fffb8059db0) at /home/flh/projects/kdev-python/parser/astbuilder.cpp:104 #12 0x7fffc68d5f3b in Python::AstDefaultVisitor::visitCode(Python::CodeAst*) (this=0x7fffc4c41670, node=0x7fffb8051510) at /home/flh/projects/kdev-python/parser/astdefaultvisitor.cpp:94 #13 0x7fffc68dbba4 in Python::AstVisitor::visitNode(Python::Ast*) (this=0x7fffc4c41670, node=0x7fffb8051510) at /home/flh/projects/kdev-python/parser/astvisitor.cpp:41 #14 0x7fffc68e0c55 in Python::RangeFixVisitor::visitNode(Python::Ast*) (this=0x7fffc4c41670, node=0x7fffb8051510) at /home/flh/projects/kdev-python/parser/astbuilder.cpp:104 #15 0x7fffc68df46b in Python::AstBuilder::parse(QUrl const&, QString&) (this=0x7fffc4c41d90, filename=..., contents=...) at /home/flh/projects/kdev-python/parser/astbuilder.cpp:757 #16 0x7fffc68d3909 in Python::ParseSession::parse() (this=0x7fffb8003900) at /home/flh/projects/kdev-python/parser/parsesession.cpp:76 #17 0x7fffc6d6dc24 in Python::ParseJob::run(QSharedPointer, ThreadWeaver::Thread*) (this=0xb72e20) at /home/flh/projects/kdev-python/pythonparsejob.cpp:157 #18 0x760ee5d6 in ThreadWeaver::IdDecorator::run(QSharedPointer, ThreadWeaver::Thread*) () at /usr/lib/libKF5ThreadWeaver.so.5 #19 0x760eebd8 in ThreadWeaver::Executor::run(QSharedPointer const&, ThreadWeaver::Thread*) () at /usr/lib/libKF5ThreadWeaver.so.5 #20 0x760edaea in ThreadWeaver::Job::execute(QSharedPointer const&, ThreadWeaver::Thread*) () at /usr/lib/libKF5ThreadWeaver.so.5 #21 0x760ed190 in ThreadWeaver::Thread::run() () at /usr/lib/libKF5ThreadWeaver.so.5 #22 0x7fffeff156d8 in () at /usr/lib/libQt5Core.so.5 #23 0x7fffe72ba2e7 in start_thread () at /usr/lib/libpthread.so.0 #24 0x7fffef2fe54f in clone () at /usr/lib/libc.so.6 (gdb) bt full #0 0x7fffef245a10 in raise () at /usr/lib/libc.so.6 #1 0x7fffef24713a in abort () at /usr/lib/libc.so.6 #2 0x004058d6 in messageOutput(QtMsgType, QMessageLogContext const&, QString const&) (type=, context=..., msg=...) at /home/flh/projects/kdevplatform/util/duchainify/main.cpp:79 #3 0x7fffefefa298 in () at /usr/lib/libQt5Core.so.5 #4 0x7fffefefbe09 in QMessageLogger::fatal(char const*, ...) const () at /usr/lib/libQt5Core.so.5 #5 0x7fffefef73de in () at /usr/lib/libQt5Core.so.5 #6 0x7fffc68d13bc in QString::at(int) const (this=0x7fffc4c41360, i=-1) at /usr/include/qt/QtCore/qstring.h:89
[kdev-python] [Bug 378827] Kevelop 4.7, 5.0.x, and 5.1.0 crashes when background parsing is turned on for Unreal Engine 4
https://bugs.kde.org/show_bug.cgi?id=378827 --- Comment #4 from Sven Brauch --- Alright thanks, I'll try ASAP -- You are receiving this mail because: You are watching all bug changes.
[kdev-python] [Bug 378827] Kevelop 4.7, 5.0.x, and 5.1.0 crashes when background parsing is turned on for Unreal Engine 4
https://bugs.kde.org/show_bug.cgi?id=378827 --- Comment #3 from Michael G Hart --- Sorry but I don't have anything bug in symbols. However with all the versions of kdevelop I tried I got the same failure wedding background parsing was turned on So I would be very surprised if you cannot replicate the problem. On Apr 17, 2017 12:02 PM, "Sven Brauch" wrote: > https://bugs.kde.org/show_bug.cgi?id=378827 > > --- Comment #2 from Sven Brauch --- > Do you have debug symbols by any chance? Otherwise I can also try to > reproduce > this ... > > -- > You are receiving this mail because: > You reported the bug. -- You are receiving this mail because: You are watching all bug changes.
[kdev-python] [Bug 378827] Kevelop 4.7, 5.0.x, and 5.1.0 crashes when background parsing is turned on for Unreal Engine 4
https://bugs.kde.org/show_bug.cgi?id=378827 --- Comment #2 from Sven Brauch --- Do you have debug symbols by any chance? Otherwise I can also try to reproduce this ... -- You are receiving this mail because: You are watching all bug changes.
[kdev-python] [Bug 378827] Kevelop 4.7, 5.0.x, and 5.1.0 crashes when background parsing is turned on for Unreal Engine 4
https://bugs.kde.org/show_bug.cgi?id=378827 Kevin Funk changed: What|Removed |Added Product|kdevelop|kdev-python Component|general |general CC||m...@svenbrauch.de Assignee|kdevelop-bugs-n...@kde.org |m...@svenbrauch.de --- Comment #1 from Kevin Funk --- Crash in kdev-python, moving ticket. -- You are receiving this mail because: You are watching all bug changes.
