Bug#973659: qtdeclarative5-dev-tools: qmlcachegen segfaults on hppa

[Dmitry Shachnev]

On February 9, 2021 11:48:08 PM GMT+03:00, John David Anglin 
 wrote:
> Is there a public qtdeclarative repository that can be cloned with
> git?

Yes: https://code.qt.io/qt/qtdeclarative.git

--
Dmitry Shachnev

Bug#973659: qtdeclarative5-dev-tools: qmlcachegen segfaults on hppa

[John David Anglin]

On 2021-02-09 2:02 p.m., Dmitry Shachnev wrote:
> That's because the relevant code is no longer present in 5.12, see:
>
> https://codereview.qt-project.org/c/qt/qtdeclarative/+/254748
Is there a public qtdeclarative repository that can be cloned with git?

-- 
John David Anglin  dave.ang...@bell.net

Bug#973659: qtdeclarative5-dev-tools: qmlcachegen segfaults on hppa

[Dmitry Shachnev]

Hi again!

On Sat, Feb 06, 2021 at 12:28:42PM -0500, John David Anglin wrote:
> There was a crashing bug patch in 5.11.3-4 that was removed in 5.12.2-1.
> Code was changed.

That's because the relevant code is no longer present in 5.12, see:

https://codereview.qt-project.org/c/qt/qtdeclarative/+/254748

Also I checked the test that fails on s390x, and it turned out to be a
problem in the test. I submitted a patch for it:

https://codereview.qt-project.org/c/qt/qtdeclarative/+/333611

So the code is compatible with big endian, generally speaking. This means
issues on hppa are probably related to stack or other hppa specifics, not
to endianness.

--
Dmitry Shachnev


signature.asc
Description: PGP signature

Bug#973659: qtdeclarative5-dev-tools: qmlcachegen segfaults on hppa

[John David Anglin]

On 2021-02-05 7:41 a.m., Dmitry Shachnev wrote:
> For NaN, you probably need to check if these masks are correct:
>
> https://sources.debian.org/src/qtdeclarative-opensource-src/5.15.2+dfsg-4/src/qml/common/qv4staticvalue_p.h/#L285
> https://sources.debian.org/src/qtdeclarative-opensource-src/5.15.2+dfsg-4/src/qml/common/qv4staticvalue_p.h/#L305
>
> and this method:
>
> https://sources.debian.org/src/qtdeclarative-opensource-src/5.15.2+dfsg-4/src/qml/common/qv4staticvalue_p.h/#L354
>
I believe the isNaN method should check for both quiet and signalling NaN 
encodings (0x0004 and 0x0008).
As best I can tell, the quiet and signalling encodings will end up reversed on 
hppa due to the value returned by qt_qnan().

I believe the "| 0x0002u" bit is redundant in the definition of 
Immediate_Mask_32.

However, I don't believe the above issues are the main problem.  The build log 
history indicates that version 5.11.3-4 was
okay and 5.12.2-1 was bad.  This is indicated by a segmentation fault by 
tst_parserstress, the first test in the testsuite.

It would be nice to bisect changes between 5.11.3-4 and 5.12.2-1 to find the 
change that causes the problem.  However,
I'm not away of a suitable archive.

There are also issues building 5.11.3-4 on sid:

qml/qqmlmetatype.cpp: In member function 'void 
QQmlTypePrivate::insertEnums(const QMetaObject*) const':
qml/qqmlmetatype.cpp:811:60: error: cannot convert 'const 
QMetaObject::SuperData* const' to 'const QMetaObject* const*' in initialization
  811 | const QMetaObject * const *related = 
metaObject->d.relatedMetaObjects;
  |  ~~^
~
  |    |
  |    const 
QMetaObject::SuperData* const

There was a crashing bug patch in 5.11.3-4 that was removed in 5.12.2-1.  Code 
was changed.

Regards,
Dave

-- 
John David Anglin  dave.ang...@bell.net

Bug#973659: qtdeclarative5-dev-tools: qmlcachegen segfaults on hppa

[Dmitry Shachnev]

On Thu, Feb 04, 2021 at 01:39:04PM -0500, John David Anglin wrote:
> That's the dichotomy.  I know hppa but not Qt.  It painful to find the
> parts of Qt that depend on endianness, stack layout, and possibly the NaN
> representation.

I also don’t know the QML / V4 engine code well.

But the qv4staticvalue_p.h file you mentioned has an excellent comment which
describes how it treats different double values:

https://sources.debian.org/src/qtdeclarative-opensource-src/5.15.2+dfsg-4/src/qml/common/qv4staticvalue_p.h/#L140

For NaN, you probably need to check if these masks are correct:

https://sources.debian.org/src/qtdeclarative-opensource-src/5.15.2+dfsg-4/src/qml/common/qv4staticvalue_p.h/#L285
https://sources.debian.org/src/qtdeclarative-opensource-src/5.15.2+dfsg-4/src/qml/common/qv4staticvalue_p.h/#L305

and this method:

https://sources.debian.org/src/qtdeclarative-opensource-src/5.15.2+dfsg-4/src/qml/common/qv4staticvalue_p.h/#L354

From endianness point of view, Qt Declarative should be mostly fine. I looked
at s390x test logs and only few tests are failing:

- qmltestrunner::item-grabber::test_endresult* — for this I filed
  https://bugreports.qt.io/browse/QTBUG-56806

- tst_qmldiskcache::regenerateAfterChange() — this one I didn’t notice
  before, and I will investigate it. It regressed somewhere between 5.11.3
  and 5.12.2.

--
Dmitry Shachnev


signature.asc
Description: PGP signature

Bug#973659: qtdeclarative5-dev-tools: qmlcachegen segfaults on hppa

[John David Anglin]

On 2021-02-04 1:25 p.m., Dmitry Shachnev wrote:
> I know almost nothing about hppa, and I don’t have much time to debug this,
> but if you provide a patch that will make more tests pass on hppa (and does
> not break other architectures), I will be happy to apply it (and help with
> pushing it upstream).
That's the dichotomy.  I know hppa but not Qt.  It painful to find the parts of 
Qt that
depend on endianness, stack layout, and possibly the NaN representation.
>
> Quick search showed me #810859 which looks like a similar problem in a
> different package.
Yes.  That bug was caused by the different representation of quiet and 
signalling NaNs.

Regards,
Dave

-- 
John David Anglin  dave.ang...@bell.net

Bug#973659: qtdeclarative5-dev-tools: qmlcachegen segfaults on hppa

[Dmitry Shachnev]

Hi John!

On Tue, Feb 02, 2021 at 10:07:52AM -0500, John David Anglin wrote:
> In looking at the JS Value encoding in src/qml/common/qv4staticvalue_p.h,
> I suspect there might be an issue with NaN/Inf values on hppa.  hppa and
> early mips used a different representation for signalling and quiet NaNs.
> This would need to be taken into account in converting between JS and
> hardware values.

Thanks a lot for your investigation!

I know almost nothing about hppa, and I don’t have much time to debug this,
but if you provide a patch that will make more tests pass on hppa (and does
not break other architectures), I will be happy to apply it (and help with
pushing it upstream).

Quick search showed me #810859 which looks like a similar problem in a
different package.

--
Dmitry Shachnev


signature.asc
Description: PGP signature

Bug#973659: qtdeclarative5-dev-tools: qmlcachegen segfaults on hppa

[John David Anglin]

On 2021-02-01 4:47 p.m., John David Anglin wrote:
> Test tst_qjsvalueiterator fails as follows:
> dave@mx3210:~/debian/qtdeclarative-opensource-src$
> /home/dave/debian/qtdeclarative-opensource-src/qtdeclarative-opensource-src-5.15.2+dfsg/tests/auto/qml/qjsvalueiterator/tst_qjsvalueiterator
> * Start testing of tst_QJSValueIterator *
> Config: Using QtTest library 5.15.2, Qt 5.15.2 (unknown-big_endian-ilp32 
> shared (dynamic) release build; by GCC 10.2.1 20210110), debian unknown
> PASS   : tst_QJSValueIterator::initTestCase()
> PASS   : tst_QJSValueIterator::iterateForward(no properties)
> PASS   : tst_QJSValueIterator::iterateForward(foo=bar)
> PASS   : tst_QJSValueIterator::iterateForward(foo=bar, baz=123)
> PASS   : tst_QJSValueIterator::iterateForward(foo=bar, baz=123, rab=oof)
> PASS   : tst_QJSValueIterator::iterateArray(no elements)
> PASS   : tst_QJSValueIterator::iterateArray(0=foo, 1=barr)
> PASS   : tst_QJSValueIterator::iterateArray(0=foo, 3=barr)
>
> === Received signal at function time: 70ms, total time: 310ms, dumping stack 
> ===
In looking at the JS Value encoding in src/qml/common/qv4staticvalue_p.h, I 
suspect there might be
an issue with NaN/Inf values on hppa.  hppa and early mips used a different 
representation for signalling
and quiet NaNs.  This would need to be taken into account in converting between 
JS and hardware values.

It seems hppa is the only big endian ilp32 architecture.

Dave

-- 
John David Anglin  dave.ang...@bell.net

Bug#973659: qtdeclarative5-dev-tools: qmlcachegen segfaults on hppa

[John David Anglin]

On 2020-11-09 1:35 p.m., Dmitry Shachnev wrote:
> I only found one old bug report and it was fixed:
> https://bugreports.qt.io/browse/QTBUG-44268
I think we have to go back to the original backtrace and the test failures.

I thought that for a bit that we might have a problem similar to ia64 and 
sparc64 but
hppa is currently 32-bit.

Test tst_qjsvalueiterator fails as follows:
dave@mx3210:~/debian/qtdeclarative-opensource-src$
/home/dave/debian/qtdeclarative-opensource-src/qtdeclarative-opensource-src-5.15.2+dfsg/tests/auto/qml/qjsvalueiterator/tst_qjsvalueiterator
* Start testing of tst_QJSValueIterator *
Config: Using QtTest library 5.15.2, Qt 5.15.2 (unknown-big_endian-ilp32 shared 
(dynamic) release build; by GCC 10.2.1 20210110), debian unknown
PASS   : tst_QJSValueIterator::initTestCase()
PASS   : tst_QJSValueIterator::iterateForward(no properties)
PASS   : tst_QJSValueIterator::iterateForward(foo=bar)
PASS   : tst_QJSValueIterator::iterateForward(foo=bar, baz=123)
PASS   : tst_QJSValueIterator::iterateForward(foo=bar, baz=123, rab=oof)
PASS   : tst_QJSValueIterator::iterateArray(no elements)
PASS   : tst_QJSValueIterator::iterateArray(0=foo, 1=barr)
PASS   : tst_QJSValueIterator::iterateArray(0=foo, 3=barr)

=== Received signal at function time: 70ms, total time: 310ms, dumping stack ===
GNU gdb (Debian 10.1-1.7) 10.1.90.20210103-git
Copyright (C) 2021 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later 
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "hppa-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
.
Find the GDB manual and other documentation resources online at:
    .

For help, type "help".
Type "apropos word" to search for commands related to "word".
Attaching to process 2844
[New LWP 2845]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/hppa-linux-gnu/libthread_db.so.1".
clone () at ../sysdeps/unix/sysv/linux/hppa/clone.S:83
(gdb)
Thread 2 (Thread 0xef8b3400 (LWP 2845) "QThread"):
#0  0xf594183c in _int_malloc (av=0x0, bytes=4010488392) at malloc.c:4116
    p = 
    iters = 
    nb = 164456
    idx = 1
    bin = 
    victim = 0x28240
    size = 210
    victim_index = 
    remainder = 0x
    remainder_size = 4294803050
    block = 
    bit = 
    map = 0
    fwd = 
    bck = 
    tcache_unsorted_count = 164424
    tcache_nb = 4010488392
    tc_idx = 0
    return_cached = 
    __PRETTY_FUNCTION__ = "_int_malloc"
#1  0x in ?? ()
No symbol table info available.
Backtrace stopped: previous frame identical to this frame (corrupt stack?)

^CDetaching from program:
/home/dave/debian/qtdeclarative-opensource-src/qtdeclarative-opensource-src-5.15.2+dfsg/tests/auto/qml/qjsvalueiterator/tst_qjsvalueiterator,
process 2844
[Inferior 1 (process 2844) detached]
=== End of stack trace ===
QFATAL : tst_QJSValueIterator::iterateString() Received signal 11
 Function time: 70ms Total time: 310ms
FAIL!  : tst_QJSValueIterator::iterateString() Received a fatal error.
   Loc: [Unknown file(0)]
Totals: 8 passed, 1 failed, 0 skipped, 0 blacklisted, 258862ms
* Finished testing of tst_QJSValueIterator *
Aborted (core dumped)
dave@mx3210:~/debian/qtdeclarative-opensource-src$ gdb -c core
GNU gdb (Debian 10.1-1.7) 10.1.90.20210103-git
Copyright (C) 2021 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later 
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "hppa-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
.
Find the GDB manual and other documentation resources online at:
    .

For help, type "help".
Type "apropos word" to search for commands related to "word".

warning: Can't open file /memfd:JSGCHeap:QtQml (deleted) during file-backed 
mapping note processing

warning: Can't open file /memfd:JSVMStack:QtQml (deleted) during file-backed 
mapping note processing
[New LWP 2844]
[New LWP 2845]
Core was generated by 
`/home/dave/debian/qtdeclarative-opensource-src/qtdeclarative-opensource-src-5.1'.
Program terminated with signal SIGABRT, Aborted.
#0  0xf5932b5c in ?? ()
[Current thread is 1 (LWP 2844)]
(gdb) quit

The test again fails because of a stack overflow.

gdb -c tests/auto/qml/qjsvalueiterator/core 

Bug#973659: qtdeclarative5-dev-tools: qmlcachegen segfaults on hppa

[Dmitry Shachnev]

On Sat, Nov 07, 2020 at 06:07:02PM -0500, John David Anglin wrote:
> That's what I was afraid of.  If these assumptions are localized, it would
> help if you could point them out.

I can't point them out :(

But I can suggest you to contact upstream (via the mailing list or via
bugreports.qt.io) and ask them where such code may be present.

I only found one old bug report and it was fixed:
https://bugreports.qt.io/browse/QTBUG-44268

--
Dmitry Shachnev


signature.asc
Description: PGP signature

Bug#973659: qtdeclarative5-dev-tools: qmlcachegen segfaults on hppa

[John David Anglin]

Hi Dmitry,

On 2020-11-07 1:20 p.m., Dmitry Shachnev wrote:
> I think the problem is not that hppa is not detected, but rather that the code
> has some assumptions about the stack growing down. The QML engine has a lot of
> low-level code...
That's what I was afraid of.  If these assumptions are localized, it would help 
if you could
point them out.  The code seems to have its own stack implementation.
>
> Can you check if your patch actually makes some tests pass that failed without
> it?
I revised the patch somewhat but it doesn't help much.
>
>> The PA-RISC architecture is big endian.  Perhaps this needs to be defined
>> somewhere else as well?
> Qt uses this file from qtbase:
>
> https://code.qt.io/cgit/qt/qtbase.git/tree/src/corelib/global/qprocessordetection.h
>
> which falls back to gcc's __BYTE_ORDER__ macro for unknown architectures.
> So it should work fine if that macro is defined correctly on hppa.
As far as I know, the __BYTE_ORDER__ macro is defined correctly for hppa.

Regards,
Dave

-- 
John David Anglin  dave.ang...@bell.net

Bug#973659: qtdeclarative5-dev-tools: qmlcachegen segfaults on hppa

[Dmitry Shachnev]

Hi John!

On Thu, Nov 05, 2020 at 03:49:47PM -0500, John David Anglin wrote:
> The attached patch fixes
> qtdeclarative-opensource-src-5.15.1+dfsg.orig/src/3rdparty/masm/wtf/Platform.h
> for hppa.  That's the only platform configuration that I could find in the
> package.

I think the problem is not that hppa is not detected, but rather that the code
has some assumptions about the stack growing down. The QML engine has a lot of
low-level code...

Can you check if your patch actually makes some tests pass that failed without
it?

> The PA-RISC architecture is big endian.  Perhaps this needs to be defined
> somewhere else as well?

Qt uses this file from qtbase:

https://code.qt.io/cgit/qt/qtbase.git/tree/src/corelib/global/qprocessordetection.h

which falls back to gcc's __BYTE_ORDER__ macro for unknown architectures.
So it should work fine if that macro is defined correctly on hppa.

--
Dmitry Shachnev


signature.asc
Description: PGP signature

Bug#973659: qtdeclarative5-dev-tools: qmlcachegen segfaults on hppa

[John David Anglin]

On 2020-11-05 12:20 p.m., Dmitry Shachnev wrote:
>> Another possibility is doStoreOnStack doesn't work on hppa.
> I think the complete QML engine may be broken on hppa, not just qmlcachegen.
>
> A lot of tests are failing — search for FAIL! in the build log:
>
> https://buildd.debian.org/status/fetch.php?pkg=qtdeclarative-opensource-src=hppa=5.14.2%2Bdfsg-3=1595527978
The attached patch fixes 
qtdeclarative-opensource-src-5.15.1+dfsg.orig/src/3rdparty/masm/wtf/Platform.h 
for hppa.  That's
the only platform configuration that I could find in the package.

The PA-RISC architecture is big endian.  Perhaps this needs to be defined 
somewhere else as well?

Regards,
Dave Anglin

-- 
John David Anglin  dave.ang...@bell.net

Index: qtdeclarative-opensource-src-5.15.1+dfsg/src/3rdparty/masm/wtf/Platform.h
===
--- 
qtdeclarative-opensource-src-5.15.1+dfsg.orig/src/3rdparty/masm/wtf/Platform.h
+++ qtdeclarative-opensource-src-5.15.1+dfsg/src/3rdparty/masm/wtf/Platform.h
@@ -70,6 +70,16 @@
 #define WTF_CPU_ALPHA 1
 #endif
 
+/* CPU(HPPA) - PA-RISC */
+#if defined(__hppa__)
+#define WTF_CPU_BIG_ENDIAN 1
+#if defined(__LP64__)
+#define WTF_CPU_HPPA64 1
+#else
+#define WTF_CPU_HPPA 1
+#endif
+#endif
+
 /* CPU(IA64) - Itanium / IA-64 */
 #if defined(__ia64__)
 #define WTF_CPU_IA64 1

Bug#973659: qtdeclarative5-dev-tools: qmlcachegen segfaults on hppa

[Dmitry Shachnev]

Hi!

On Thu, Nov 05, 2020 at 11:25:37AM -0500, John David Anglin wrote:
> The segfault is caused by stack overflow.  On hppa, the kernel config
> parameter CONFIG_MAX_STACK_SIZE_MB sets the maximum stack size.  This is
> because the main stack is allocated at the top of memory and the stack grows
> up.  256 MB is not enough.
>
> [...]
>
> How much stack does qmlcachegen typically need?

No idea, but I guess it depends on the size of QML file being cached.

> Another possibility is doStoreOnStack doesn't work on hppa.

I think the complete QML engine may be broken on hppa, not just qmlcachegen.

A lot of tests are failing — search for FAIL! in the build log:

https://buildd.debian.org/status/fetch.php?pkg=qtdeclarative-opensource-src=hppa=5.14.2%2Bdfsg-3=1595527978

--
Dmitry Shachnev


signature.asc
Description: PGP signature

Bug#973659: qtdeclarative5-dev-tools: qmlcachegen segfaults on hppa

[John David Anglin]

Package: qtdeclarative5-dev-tools
Version: 5.14.2+dfsg-3
Severity: normal

Dear Maintainer,

The qtgraphicaleffects-opensource-src package fails to build on hppa
because qmlcachegen faults with out-of bounds references:

do_page_fault() command='qmlcachegen' type=15 address=0xf98c4020 in 
qmlcachegen[1+c5000]
trap #15: Data TLB miss fault, vm_start = 0xf90c4000, vm_end = 0xf98c4000

do_page_fault() command='qmlcachegen' type=15 address=0xfaed5000 in 
libQt5Core.so.5.15.1[f7f28000+5e6000]
trap #15: Data TLB miss fault, vm_start = 0xfa6d5000, vm_end = 0xfaed5000

do_page_fault() command='qmlcachegen' type=15 address=0xf99b9000 in 
libQt5Core.so.5.15.1[f7f28000+5e6000]
trap #15: Data TLB miss fault, vm_start = 0xf91b9000, vm_end = 0xf99b9000

See
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=973646
for details regarding the qtgraphicaleffects-opensource-src build.

Regards,
Dave Anglin

-- System Information:
Debian Release: bullseye/sid
  APT prefers buildd-unstable
  APT policy: (500, 'buildd-unstable'), (500, 'unstable')
Architecture: hppa (parisc64)

Kernel: Linux 5.8.18+ (SMP w/4 CPU threads)
Locale: LANG=C, LC_CTYPE=C.UTF-8 (charmap=UTF-8), LANGUAGE not set
Shell: /bin/sh linked to /bin/dash
Init: systemd (via /run/systemd/system)

Versions of packages qtdeclarative5-dev-tools depends on:
ii  libc6  2.31-4
ii  libgcc-s4  10.2.0-16
ii  libqt5core5a [qtbase-abi-5-14-2]   5.14.2+dfsg-6
ii  libqt5gui5 5.14.2+dfsg-6
ii  libqt5network5 5.14.2+dfsg-6
ii  libqt5qml5 [qtdeclarative-abi-5-14-2]  5.14.2+dfsg-3
ii  libqt5quick5   5.14.2+dfsg-3
ii  libqt5quicktest5   5.14.2+dfsg-3
ii  libqt5widgets5 5.14.2+dfsg-6
ii  libstdc++6 10.2.0-16
ii  qtchooser  66-2

qtdeclarative5-dev-tools recommends no packages.

qtdeclarative5-dev-tools suggests no packages.

-- no debconf information