Re: [Development] Item creation time in QML

[Andrew den Exter]

>
> Is creation expensive because of constructor code, making too many
> signal/slot connections, or something else?
>

First of all it's accumulative, any QML application is going to end up
constructing many items, hundreds, thousands even and costs, even small
ones add up.  Secondly yes constructing and interacting with QObjects can
be expensive and there are ways to mitigate some of those costs.   Thirdly
parsing, compiling, instantiating, and executing bindings is all extra work
that comes at a cost.

Robin's singled out the Text items so lets have a look at those.  A good
amount of work has already been put into these items though so TextEdit
aside there's not a lot of low hanging fruit.

I'll list some slightly more detailed benchmarks below, but my imperfect
measure on the same device a raw Item takes around 30us to construct and
Text about 50us.  That extra time is spent initializing many primitives,
setting a couple of QQuickItem flags, executing the default constructors
for a couple of QLists, a QUrl, a QString, two QFonts, and a QLayout, and
attempting to determine the default layout direction.  From memory if you
profiled you'd find most of that time was in the QFont constructors,
including those in QLayout.

TextInput is the next slowest at about 80us.  It mostly has the same
initialization as a Text except it calls QClipboard::supportsSelection(),
and connects to QClipboard's dataChanged() signal.  That surprisingly uses
the traditional connect method which is almost universally shunned in
QtDeclarative code.  If I change that to a qmlobject_connect that time goes
down to 65ms, 20% right there.

TextEdit is a behemoth at about 420us.  A lot of that is QTextDocument
which is also why Text is so much slower with RichText.  The rest is mostly
because a lot of the implementation is in QQuickTextControl (extra QObject
allocation) and there are many signal/slot connections between
QTextDocument, QQuickTextControl, and QQuickTextEdit.  If anyone's looking
for a project, folding QQuickTextControl into QQuickText would yield a
decent improvement in absolute construction time.

Note these are pure instantiation times, when the items have to do actual
work the numbers climb again. Just setting a property such as Text {
horizontalAlignment: Text.AlignLeft } gives a time closer to 65us, and a
setting some text so a layout is actually performed like Text { text:
"hello world" } brings it up to the 120us mark for example.  An then
slightly decoupled from that in the real world that item is a member of a
scene graph and constructing the scene graph node add costs on top of that.


As for what you can do, first and foremost measure and compare.  Item
(~30us) to ListView (~80us) represent what I think are the boundaries of
reasonable expectation for performance, if you're exceeding ListView
something's probably wrong, try and aim for Item though.  Otherwise there
are a few common traps to avoid:

Create fewer QObjects, if you don't absolutely need it you don't want it.
Minimise signal/slot connections, use a connect mechanism that caches the
signal index if you do
Another unexpected one I don't have demonstrative times for right now is
QObject::setParent() or even just passing the parent to the QObject
constructor I think.  That will send a ParentChange event which eats up a
noticeable amount of time usually to no effect, internally QtDeclarative
avoids this by calling QQml_setParent_noEvent instead.
Defer work caused by property writes until componentComplete() or a polish
event if possible, there's no point re-evaluating something if it's
immediately invalided by a subsequent property change.  Text doesn't do the
latter right now which means it can be more expensive to make multiple
changes to an already constructed Text item than create a new one.


Andrew


The times below were measured using
https://github.com/adenexter/qml-benchmarks on a 1.4GHz dual core arm
device, your results will vary.

Measuring construction time of Item over 100 iterations

 import QtQuick 2.0


Item {

}

 One time parse time (ns) 1,037,692

One time construction time (ns): 91,561

Average construction time (ns):  29,909

Average destruction time (ns):   10,682

 Measuring construction time of Text over 100 iterations

 import QtQuick 2.0


Text {

}

 One time parse time (ns) 1,800,702

One time construction time (ns): 15,016,023

Average construction time (ns):  47,306

Average destruction time (ns):   21,059

 Measuring construction time of TextInput over 100 iterations

 import QtQuick 2.0


TextInput {

}

 One time parse time (ns) 1,892,263

One time construction time (ns): 396,765

Average construction time (ns):  80,573

Average destruction time (ns):   15,565

 Measuring construction time of TextEdit over 100 iterations

 import QtQuick 2.0


TextEdit {

}

 One time parse time (ns) 1,739,662

One time construction time (ns): 1,495,498

Average construction time (ns):  449,870

Average des

Re: [Development] QSsl: finer-grained protocol selection

[Mikkel Krautz]

I've opened a code review for adding TlsV1_0OrLater and friends at:

https://codereview.qt-project.org/#/c/106080/

We obviously haven't reached agreement on the actual API for this, but
as I said, we're pretty happy with the 'OrLater' variant, which the
patchset implements.  Hopefully, the patchset gives a better overview
of the changes involved.

Rudimentary (as much as possible) support for WinRT is available in
the patchset.

SecureTransport should be easy to add, since it uses a (min, max)
protocol range...


On Fri, Jan 16, 2015 at 3:16 PM, Mikkel Krautz  wrote:
> On Sun, Dec 28, 2014 at 2:26 PM, Thiago Macieira
>  wrote:
>> On Sunday 28 December 2014 13:11:13 Richard Moore wrote:
>>> At the moment there are still a lot of SSL accelerators out there with
>>> these problems. We can probably stop worrying in around a year once all the
>>> browsers have got around to disabling SSL3 and thereby forcing things to be
>>> fixed. Currently we will already fail to connect to these servers, but the
>>> API we provide allows users to implement workarounds in their own code. If
>>> we change the meaning of the TLSv1 constant in this way then it would no
>>> longer be possible for them to do this.
>>
>> Ah, I see.
>>
>> Then we just add to the list:
>>
>> TlsV1_0OrLater,
>> TlsV1_1OrLater,
>> TlsV1_2OrLater
>>
>> When TLS 1.3 comes into existence, we add:
>>
>> TlsV1_3,
>> TlsV1_3OrLater
>
> We'd be fine with either of these proposals.
>
> However, I think this proposal would be less surprising to existing
> users of QSslSocket, so it's the one I'd prefer personally.
>
>> Alternatively, we can add a
>>
>> /// if major == 0, sets to "Secure Protocols"
>> void setMinimumTlsVersion(int major, int minor);
>> int sessionTlsMajorVersion() const;
>> int sessionTlsMinorVersion() const;
>>
>> And deprecate setProtocol.
>> --
>> Thiago Macieira - thiago.macieira (AT) intel.com
>>   Software Architect - Intel Open Source Technology Center
>>
>> ___
>> Development mailing list
>> Development@qt-project.org
>> http://lists.qt-project.org/mailman/listinfo/development
___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development

Re: [Development] Item creation time in QML

[Gunnar Roth]

Hello Gunnar
yes i meant y-axis. 
ok so it is operations/frame and i assume a modern pc is used for Robin 
Burchells measurements as you did with using you MacBook Pro. 
After my vacation I will try to do execute  your benchmarking measurements on 
our wec2013 imx6 device, using 5.4 and qt 5.5 git. so i can see if we all see 
improvements here.
( I will ask a colleague to do similar on our im6 and omap linux devices)

but how do i get 5.5 git ? 
can i simply use :

git clone https://git.gitorious.org/qt/qt5.git qt5
cd qt5
git checkout 5.5
perl init-repository

??

Thank you very much for your efforts in making qml better and faster (again) . 
We had asked digia a long time for benchmarks, but did not get any good 
information
, then i found your blog last year giving us want we needed, an insightful 
designed benchmarking suite.well for some testcase we got a crash fair but all 
in all it was a good start.
we get involved with dig again with our results but … we will see.

Best regards,
Gunnar Roth


> Am 12.02.2015 um 19:25 schrieb Gunnar Sletta :
> 
> 
>> On 12 Feb 2015, at 18:03, Gunnar Roth  wrote:
>> 
>> Hi,
>> after downloading it to my cell phone and picking it with usb connection, i 
>> was finally able to read pdf document.
>> I am curious what the unit of the x-axis is in your diagrams, i fugure 
>> higher is better, but what is measured there?
> 
> Hi Gunnar :)
> 
> The x-axis is Qt version, did you perhaps mean y-axis? :)
> 
> From my original mail: 
> 
> How the benchmark works is that the it tries to figure out how many 
> operations of a given type is possible each frame (not per second) while 
> sustaining a perfect 60 fps (or whatever other fps you target). For these 
> numbers, we’re looking at delegation creation and destruction. You can look 
> at the content of each specific benchmark here: 
> https://github.com/sletta/stuff/tree/master/qml/benchmarks/benchmark/creation.
>  
> 
> 
>> IMO QuickItemText is too complicated for the simple case, which is 
>> "displaying a single line of plain text of about 15 characters", as this is 
>> the most common case in a UI. So having a QuickSimpleText item, which can 
>> only do the simple case should be much faster to create. Not having a 
>> QTextDocument and all that many members for all this many features. Of 
>> course even with most of this feratures it was faster in 5.1, but still 
>> could be a lot more , i think for the simple case.
> 
> It still needs to do shaping and all that though, so some involvement with 
> the text engine is required. However, I believe there is huge room for 
> improvement there. There is https://bugreports.qt.io/browse/QTBUG-42853 which 
> covers some ideas around writing a faster implementation.
> 
>>  
>> I am very interested in solving issue as in our environment we are hit by 
>> the degration and our product has to stay at 5.1 for now.
>> So keep on the good work!
>>  
>> Best regards,
>> Gunnar Roth
>>  
>> Gesendet: Donnerstag, 12. Februar 2015 um 11:56 Uhr
>> Von: "Robin Burchell" 
>> An: "Rutledge Shawn" 
>> Cc: "development@qt-project.org" 
>> Betreff: Re: [Development] Item creation time in QML
>> On Wed, Feb 11, 2015 at 2:49 PM, Rutledge Shawn
>>  wrote:
>> >> Being able to do 500+items rectangles in one frame is decent, but not 
>> >> awesome. Being able todo 130 text items in one frame, is less than ideal, 
>> >> given that we often use several text items per cell in a list or table. 
>> >> Text is probably the most important UI element we have. Being able to do 
>> >> 10 buttons is, well, unfortunate :) If we look at Button, we see that it 
>> >> is a fairly complex QML beast. Hierarchy is
>> >>
>> >> Button -> BasicButton -> Control -> FocusScope
>> >>
>> >> and there are quite a bit of stuff on every level, including the dynamic 
>> >> style handling which will in turn create a StyleItem.
>> >>
>> >> And keep in mind that even though this isn’t the most high-end mac, it is 
>> >> sitll a pretty decent computer, Qt is supposed to run on much worse 
>> >> hardware than this.
>> >>
>> >> —
>> >>
>> >> There have been a few changes going into 5.5 which make these numbers 
>> >> better than 5.4, but I still think we got quite a ways to go.
>> >
>> > That’s very interesting.
>> >
>> > Do you think it was ever better in past versions?
>> 
>> I'm glad you asked: http://rburchell.com/QtQuickPerformance.pdf
>> 
>> > What do you think we should do about it? Mainly focus on trying to create 
>> > fewer objects, or is there still a lot of headroom for making the creation 
>> > more efficient? Is creation expensive because of constructor code, making 
>> > too many signal/slot connections, or something else?
>> 
>> I think there's plenty that can be done, because it *used* to be
>> better. We should aim for matching the performance we used to have at
>> the very least, ideally, for exceeding that.
>> 
>> I'd say that one very particular case that needs attention is text
>> performance. It's pretty bad.

Re: [Development] Item creation time in QML

[Gunnar Sletta]

> On 12 Feb 2015, at 18:03, Gunnar Roth  wrote:
> 
> Hi,
> after downloading it to my cell phone and picking it with usb connection, i 
> was finally able to read pdf document.
> I am curious what the unit of the x-axis is in your diagrams, i fugure higher 
> is better, but what is measured there?

Hi Gunnar :)

The x-axis is Qt version, did you perhaps mean y-axis? :)

From my original mail: 

How the benchmark works is that the it tries to figure out how many operations 
of a given type is possible each frame (not per second) while sustaining a 
perfect 60 fps (or whatever other fps you target). For these numbers, we’re 
looking at delegation creation and destruction. You can look at the content of 
each specific benchmark here: 
https://github.com/sletta/stuff/tree/master/qml/benchmarks/benchmark/creation 
.
 


> IMO QuickItemText is too complicated for the simple case, which is 
> "displaying a single line of plain text of about 15 characters", as this is 
> the most common case in a UI. So having a QuickSimpleText item, which can 
> only do the simple case should be much faster to create. Not having a 
> QTextDocument and all that many members for all this many features. Of course 
> even with most of this feratures it was faster in 5.1, but still could be a 
> lot more , i think for the simple case.

It still needs to do shaping and all that though, so some involvement with the 
text engine is required. However, I believe there is huge room for improvement 
there. There is https://bugreports.qt.io/browse/QTBUG-42853 
 which covers some ideas around 
writing a faster implementation.

>  
> I am very interested in solving issue as in our environment we are hit by the 
> degration and our product has to stay at 5.1 for now.
> So keep on the good work!
>  
> Best regards,
> Gunnar Roth
>  
> Gesendet: Donnerstag, 12. Februar 2015 um 11:56 Uhr
> Von: "Robin Burchell" mailto:robin...@viroteck.net>>
> An: "Rutledge Shawn"  >
> Cc: "development@qt-project.org " 
> mailto:development@qt-project.org>>
> Betreff: Re: [Development] Item creation time in QML
> On Wed, Feb 11, 2015 at 2:49 PM, Rutledge Shawn
> mailto:shawn.rutle...@theqtcompany.com>> 
> wrote:
> >> Being able to do 500+items rectangles in one frame is decent, but not 
> >> awesome. Being able todo 130 text items in one frame, is less than ideal, 
> >> given that we often use several text items per cell in a list or table. 
> >> Text is probably the most important UI element we have. Being able to do 
> >> 10 buttons is, well, unfortunate :) If we look at Button, we see that it 
> >> is a fairly complex QML beast. Hierarchy is
> >>
> >> Button -> BasicButton -> Control -> FocusScope
> >>
> >> and there are quite a bit of stuff on every level, including the dynamic 
> >> style handling which will in turn create a StyleItem.
> >>
> >> And keep in mind that even though this isn’t the most high-end mac, it is 
> >> sitll a pretty decent computer, Qt is supposed to run on much worse 
> >> hardware than this.
> >>
> >> —
> >>
> >> There have been a few changes going into 5.5 which make these numbers 
> >> better than 5.4, but I still think we got quite a ways to go.
> >
> > That’s very interesting.
> >
> > Do you think it was ever better in past versions?
> 
> I'm glad you asked: http://rburchell.com/QtQuickPerformance.pdf 
> 
> 
> > What do you think we should do about it? Mainly focus on trying to create 
> > fewer objects, or is there still a lot of headroom for making the creation 
> > more efficient? Is creation expensive because of constructor code, making 
> > too many signal/slot connections, or something else?
> 
> I think there's plenty that can be done, because it *used* to be
> better. We should aim for matching the performance we used to have at
> the very least, ideally, for exceeding that.
> 
> I'd say that one very particular case that needs attention is text
> performance. It's pretty bad.
> 
> As for why, specifically, it's slow, I did some profiling in the past,
> but I don't have those results now. I plan to spend some time on that,
> maybe on the weekend, and send some more information once that's done.
> ___
> Development mailing list
> Development@qt-project.org 
> http://lists.qt-project.org/mailman/listinfo/development 
> ___
> Development mailing list
> Development@qt-project.org
> http://lists.qt-project.org/mailman/listinfo/development

___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development

Re: [Development] Item creation time in QML

[Gunnar Roth]

Hi,
after downloading it to my cell phone and picking it with usb connection, i was 
finally able to read pdf document.
I am curious what the unit of the x-axis is in your diagrams, i fugure higher 
is better, but what is measured there?
 
IMO QuickItemText is too complicated for the simple case, which is "displaying 
a single line of plain text of about 15 characters", as this is the most common 
case in a UI. So having a QuickSimpleText item, which can only do the simple 
case should be much faster to create. Not having a QTextDocument and all that 
many members for all this many features. Of course even with most of this 
feratures it was faster in 5.1, but still could be a lot more , i think for the 
simple case.
 
I am very interested in solving issue as in our environment we are hit by the 
degration and our product has to stay at 5.1 for now.
So keep on the good work!
 
Best regards,
Gunnar Roth
 
Gesendet: Donnerstag, 12. Februar 2015 um 11:56 Uhr
Von: "Robin Burchell" 
An: "Rutledge Shawn" 
Cc: "development@qt-project.org" 
Betreff: Re: [Development] Item creation time in QML
On Wed, Feb 11, 2015 at 2:49 PM, Rutledge Shawn
 wrote:
>> Being able to do 500+items rectangles in one frame is decent, but not 
>> awesome. Being able todo 130 text items in one frame, is less than ideal, 
>> given that we often use several text items per cell in a list or table. Text 
>> is probably the most important UI element we have. Being able to do 10 
>> buttons is, well, unfortunate :) If we look at Button, we see that it is a 
>> fairly complex QML beast. Hierarchy is
>>
>> Button -> BasicButton -> Control -> FocusScope
>>
>> and there are quite a bit of stuff on every level, including the dynamic 
>> style handling which will in turn create a StyleItem.
>>
>> And keep in mind that even though this isn’t the most high-end mac, it is 
>> sitll a pretty decent computer, Qt is supposed to run on much worse hardware 
>> than this.
>>
>> —
>>
>> There have been a few changes going into 5.5 which make these numbers better 
>> than 5.4, but I still think we got quite a ways to go.
>
> That’s very interesting.
>
> Do you think it was ever better in past versions?

I'm glad you asked: http://rburchell.com/QtQuickPerformance.pdf 


> What do you think we should do about it? Mainly focus on trying to create 
> fewer objects, or is there still a lot of headroom for making the creation 
> more efficient? Is creation expensive because of constructor code, making too 
> many signal/slot connections, or something else?

I think there's plenty that can be done, because it *used* to be
better. We should aim for matching the performance we used to have at
the very least, ideally, for exceeding that.

I'd say that one very particular case that needs attention is text
performance. It's pretty bad.

As for why, specifically, it's slow, I did some profiling in the past,
but I don't have those results now. I plan to spend some time on that,
maybe on the weekend, and send some more information once that's done.
___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development 
___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development

Re: [Development] Why can't QString use UTF-8 internally?

[Knoll Lars]

On 12/02/15 10:11, "Rutledge Shawn" 
wrote:

>
>On 12 Feb 2015, at 08:55, Konstantin Ritt  wrote:
>
>> 2015-02-12 11:53 GMT+04:00 Konstantin Ritt :
>> 2015-02-12 11:39 GMT+04:00 Rutledge Shawn
>>:
>> 
>> On 11 Feb 2015, at 18:15, Konstantin Ritt  wrote:
>> 
>> > FYI: Unicode codepoint != character visual representation. Moreover,
>>a single character could be represented with  a sequence of glyps or
>>vice versa - a sequence of characters could be represented with a single
>>glyph.
>> > QString (and every other Unicode string class in the world)
>>represents a sequence of Unicode codepoints (in this or that UTF), not
>>characters or glyphs - always remember that!
>> 
>> Is it impossible to convert some of the possible multi-codepoint
>>sequences into single ones, or is it just that we prefer to preserve
>>them so that when you convert back to UTF you get the same bytes with
>>which you created the QString?
>> 
>> Not sure I understand your question in context of visual representation.
>> Assume you're talking about composing the input string (though the same
>>string, composed and decomposed, would be shaped into the same sequence
>>of glyphs).
>> A while ago we decided to not change the composition form of the input
>>text and let the user to (de)compose where he needs a fixed composition
>>form, so that QString(wellformed_unicode_text).toUnicode() ==
>>wellformed_unicode_text.
>> 
>> P.S. We could re-consider this or could introduce a macro that would
>>change the composition form of a QString input but…why?
>
>It might be almost within our power to index into a QString and get a
>single, complete, renderable glyph, which is practical at least for
>rendering, and maybe for editing too.  But if we did that by storing the
>unicode that way, we’d lose this feature of being able to reproduce the
>input text exactly:
>QString(wellformed_unicode_text).toUnicode() == wellformed_unicode_text
>Consequently we have to do conversion each time we need the renderable
>text, and/or cache the results to avoid converting repeatedly.  Right?

The TrueType/OpenType font defines how to convert a unicode string into
it’s visual representation. That representation can be very different from
font to font and there simply is no predefined mapping from unicode text
to glyphs. Not even for latin text given things such as optional ff/fi
ligatures or predefined combined glyphs for certain string combinations.

I believe QString is about the best abstraction we could make for a
Unicode string. Of course one can debate about the concrete encoding of
Unicode we chose (utf16 vs utf8 vs utf32), but it’s currently a purely
academic exercise. All have advantages and disadvantages, but the fact
remains for all of them that one Unicode code point does not necessarily
always represent what you would trivially think of as one character in
your writing system. Some other writing systems are word based (e.g.
Chinese) or syllable based (e.g. parts of Japanese, indian languages).

There’s a reason the unicode specification (without data tables and
appendices) has almost a thousand pages ;-)

>But it would also be possible to go the other direction: save only UTF-8
>form in memory, so by definition we can reproduce text that was given in
>UTF-8 form.  But if the QString was constructed from text in some other
>UTF form, can we simply remember which UTF it was, convert to UTF-8, and
>then be able to reproduce it exactly by converting back from UTF-8?  And
>we still need to be able to do conversion to renderable glyphs, and maybe
>cache them.
>
>I see that QTextBoundaryFinder has
> const QChar *chars;
>so that is nearly a cache of glyphs, right?  except that e.g. a soft
>hyphen could exist in that list, and yet may or may not be rendered
>depending where the line breaks fall?  So at the end, rendering must
>still be done iteratively by calling toNextBoundary repeatedly and
>pulling out substrings and rendering those.  (QTextBoundaryFinder doesn’t
>have a QChar grapheme() accessor.  I guess that’s done elsewhere.)  But
>some of the decisions have been made already, and embodied into that
>array of QChars.  I was wondering whether it’s worthwhile to do more work
>each time we iterate, by using UTF-8 form directly, instead of converting
>to an array of QChars first.  So the memory to store the string would be
>less, but the code to do the glyph-by-glyph iteration at rendering time
>would become more “branchy”, and that is also bad for CPU cache
>performance.
>
>Oh but there’s another way of storing glyphs: the list of QScriptItems in
>the text engine.  That looks kindof bulky too, depending how long we keep
>it around.
>
>So Unicode is a mini-language which has to be interpreted at some point
>on the way to rendering; there’s no pre-interpreted form we could store
>it in.  TrueType is also a mini-language.  Maybe it would be possible to
>write a compiler which reads UTF-8 and TrueType and writes (nearly)
>branch-free code to render a whole line or

Re: [Development] Item creation time in QML

[Robin Burchell]

On Wed, Feb 11, 2015 at 2:49 PM, Rutledge Shawn
 wrote:
>> Being able to do 500+items rectangles in one frame is decent, but not 
>> awesome. Being able todo 130 text items in one frame, is less than ideal, 
>> given that we often use several text items per cell in a list or table. Text 
>> is probably the most important UI element we have. Being able to do 10 
>> buttons is, well, unfortunate :) If we look at Button, we see that it is a 
>> fairly complex QML beast. Hierarchy is
>>
>> Button -> BasicButton -> Control -> FocusScope
>>
>> and there are quite a bit of stuff on every level, including the dynamic 
>> style handling which will in turn create a StyleItem.
>>
>> And keep in mind that even though this isn’t the most high-end mac, it is 
>> sitll a pretty decent computer, Qt is supposed to run on much worse hardware 
>> than this.
>>
>> —
>>
>> There have been a few changes going into 5.5 which make these numbers better 
>> than 5.4, but I still think we got quite a ways to go.
>
> That’s very interesting.
>
> Do you think it was ever better in past versions?

I'm glad you asked: http://rburchell.com/QtQuickPerformance.pdf

> What do you think we should do about it?  Mainly focus on trying to create 
> fewer objects, or is there still a lot of headroom for making the creation 
> more efficient?  Is creation expensive because of constructor code, making 
> too many signal/slot connections, or something else?

I think there's plenty that can be done, because it *used* to be
better. We should aim for matching the performance we used to have at
the very least, ideally, for exceeding that.

I'd say that one very particular case that needs attention is text
performance. It's pretty bad.

As for why, specifically, it's slow, I did some profiling in the past,
but I don't have those results now. I plan to spend some time on that,
maybe on the weekend, and send some more information once that's done.
___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development

Re: [Development] Why can't QString use UTF-8 internally?

[Konstantin Ritt]

2015-02-12 13:11 GMT+04:00 Rutledge Shawn :

> Consequently we have to do conversion each time we need the renderable
> text, and/or cache the results to avoid converting repeatedly.  Right?
>

Pnrftm... what? Cache what? And where? I've missed the point...


>   And we still need to be able to do conversion to renderable glyphs, and
> maybe cache them.
>

Glyphs? Where glyphs came from?


> So Unicode is a mini-language which has to be interpreted at some point on
> the way to rendering; there’s no pre-interpreted form we could store it
> in.  TrueType is also a mini-language.  Maybe it would be possible to write
> a compiler which reads UTF-8 and TrueType and writes (nearly) branch-free
> code to render a whole line or block of text, so we could cache code
> instead of data.  It could be more compact and CPU cache-friendly.  I
> imagine nobody has done that yet.  But then if you think about all the
> fancy stuff TeX can do, it could get even more complex than what Qt
> currently does.  And I don’t understand much about what Harfbuzz does yet,
> either.
>

TrueType doesn't define a codepoint to glyph mapping. In fact, glyph
indices for the same codepoints would be different for two arbitrary TT
fonts.
___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development

Re: [Development] Upgrading the sources to C++11 keywords (Q_NULLPTR, etc.)

[Ziller Eike]

> On Feb 10, 2015, at 10:51 AM, Marc Mutz  wrote:
> 
> On Tuesday 10 February 2015 08:41:47 Ziller Eike wrote:
>>> On Feb 9, 2015, at 3:40 PM, Marc Mutz  wrote:
>>> 
>>> On Monday 09 February 2015 09:54:12 Smith Martin wrote:
 This is the kind of thing we should add to the documentation, but can
 you elaborate? I mean, illustrate the meaning of "locality of
 reference," "wildly mixing insertions, lookups, and removals," and
 "batch updates and separate them from lookups, time-wise."
>>> 
>>> There is a book, Effective STL, and an online paper, "What every
>>> programmer needs to know about memory". I don't think that it's the job
>>> of the Qt docs to explain "what every programmer needs to know about
>>> memory", because it is not specific to Qt.
>>> 
>>> The Qt documentation also doesn't give many details on Unicode. It's
>>> assumed that people using it know the basics.
>>> 
>>> You will also not find in the Qt docs that signed integer overflow is
>>> undefined behaviour.
>>> 
>>> The Qt docs are not a CS text book :)
>> 
>> That QMap is implemented based on a RB tree is mentioned in a single
>> sentence in the beginning of the description of the class, which is the
>> sentence that nobody ever reads (because it tends to contain useful
>> information like "The QList class is a template class that provides
>> lists.”).
>> 
>> It is followed by a short (but longer) comparison between QMap and QHash,
>> which in turn is linked to a relatively long section that compares the
>> complexity of Qt’s different containers. So, fact is that the Qt docs *do*
>> explain more than “QMap is a RB tree” (and I think that is good so). And
>> since they talk a lot about algorithmic complexity, it would probably be
>> useful to mention a few other things that concern performance differences
>> as well. In ~5 sentences. I really can’t see how that could hurt.
>> 
>> Actually you sounded like you’d like to educate people about this also
>> “outside of CS courses” (by sending them to their manager etc), so do you
>> have a different, realistic proposal?
> 
> The topic of this thread is guidelines for Qt code, not code using Qt.

Qt code is code using Qt. Qt developers are Qt users.
Qt developers (read: contributors) also read Qt documentation to find out which 
data structures to use for solving which problem.

> We 
> don't want to impose writing Q_DECL_CONSTEXPR everywhere possible for users 
> of 
> Qt. Likewise, if they want to use QMap, then that's fine. A profiler will 
> tell 
> them if they made a mistake that matters.
> 
> The situation is different for Qt code:

Actually not much, since there are many other libraries than Qt out there that 
use Qt and therefore must also “optimize for everything”.

> Consider a recent example: QString::multiArg(). It used a QMap. Now it uses a 
> QVarLengthArray (introduced in 7b5ba56b0ab55fcaf79fbf9aad70bf767c938e15). A 
> very naïve benchmark (I'm allowed to say that, it was my choice). was sped up 
> by 30%. The naïviteé in the benchmark is that this includes repeated 
> allocation of a handful of QString arguments from short (C) string literals 
> each time through the loop, which probably dominates the result.
> 
> We don't know what Qt is being used for, so we need to optimize everything. 
> One user surely will run multiArg() in a tight loop and that will dominate 
> his 
> runtime. As every C++ library that's any good, we cannot be sloppy when it 
> comes to efficiency.
> 
> So, no, I don't think we should discuss everthing ever written about C++ 
> efficiency in the Qt docs.

Nobody has claimed that we should.

> But we need to point it out to each other in code 
> reviews and become better at not writing sloppy code.

Nobody has claimed that we shouldn’t.
Discussing in the documentation which classes are best used in which use cases, 
might reduce the need to educate people over and over again in code reviews 
though. And reviews could be abbreviated to “don’t use QMap in this case, read 
”, instead of discussing the same thing over and over 
again.

> IOW: We need to start thinking about our algorithms and data structures 
> again[1], but this time in the new world of caches and multithreading where 
> the only fast data structure is an array.[2]
> 
> Thanks,
> Marc
> 
> [1] http://www.amazon.com/Algorithms-Structures-Prentice-Hall-Automatic-
> Computation/dp/0130224189
> 
> [2] http://vimeo.com/97337258
> 
> -- 
> Marc Mutz  | Senior Software Engineer
> KDAB (Deutschland) GmbH & Co.KG, a KDAB Group Company
> www.kdab.com || Germany +49-30-521325470 || Sweden (HQ) +46-563-540090
> KDAB - Qt Experts - Platform-Independent Software Solutions

-- 
Eike Ziller, Senior Software Engineer - The Qt Company GmbH
 
The Qt Company GmbH, Rudower Chaussee 13, D-12489 Berlin
Geschäftsführer: Mika Pälsi, Juha Varelius, Tuula Haataja
Sitz der Gesellschaft: Berlin, Registergericht: Amtsgericht Charlottenburg, HRB 
144331 B

___

Re: [Development] Why can't QString use UTF-8 internally?

[Rutledge Shawn]

On 12 Feb 2015, at 08:55, Konstantin Ritt  wrote:

> 2015-02-12 11:53 GMT+04:00 Konstantin Ritt :
> 2015-02-12 11:39 GMT+04:00 Rutledge Shawn :
> 
> On 11 Feb 2015, at 18:15, Konstantin Ritt  wrote:
> 
> > FYI: Unicode codepoint != character visual representation. Moreover, a 
> > single character could be represented with  a sequence of glyps or vice 
> > versa - a sequence of characters could be represented with a single glyph.
> > QString (and every other Unicode string class in the world) represents a 
> > sequence of Unicode codepoints (in this or that UTF), not characters or 
> > glyphs - always remember that!
> 
> Is it impossible to convert some of the possible multi-codepoint sequences 
> into single ones, or is it just that we prefer to preserve them so that when 
> you convert back to UTF you get the same bytes with which you created the 
> QString?
> 
> Not sure I understand your question in context of visual representation.
> Assume you're talking about composing the input string (though the same 
> string, composed and decomposed, would be shaped into the same sequence of 
> glyphs).
> A while ago we decided to not change the composition form of the input text 
> and let the user to (de)compose where he needs a fixed composition form, so 
> that QString(wellformed_unicode_text).toUnicode() == wellformed_unicode_text.
> 
> P.S. We could re-consider this or could introduce a macro that would change 
> the composition form of a QString input but…why?

It might be almost within our power to index into a QString and get a single, 
complete, renderable glyph, which is practical at least for rendering, and 
maybe for editing too.  But if we did that by storing the unicode that way, 
we’d lose this feature of being able to reproduce the input text exactly:
QString(wellformed_unicode_text).toUnicode() == wellformed_unicode_text
Consequently we have to do conversion each time we need the renderable text, 
and/or cache the results to avoid converting repeatedly.  Right?

But it would also be possible to go the other direction: save only UTF-8 form 
in memory, so by definition we can reproduce text that was given in UTF-8 form. 
 But if the QString was constructed from text in some other UTF form, can we 
simply remember which UTF it was, convert to UTF-8, and then be able to 
reproduce it exactly by converting back from UTF-8?  And we still need to be 
able to do conversion to renderable glyphs, and maybe cache them.

I see that QTextBoundaryFinder has
 const QChar *chars;
so that is nearly a cache of glyphs, right?  except that e.g. a soft hyphen 
could exist in that list, and yet may or may not be rendered depending where 
the line breaks fall?  So at the end, rendering must still be done iteratively 
by calling toNextBoundary repeatedly and pulling out substrings and rendering 
those.  (QTextBoundaryFinder doesn’t have a QChar grapheme() accessor.  I guess 
that’s done elsewhere.)  But some of the decisions have been made already, and 
embodied into that array of QChars.  I was wondering whether it’s worthwhile to 
do more work each time we iterate, by using UTF-8 form directly, instead of 
converting to an array of QChars first.  So the memory to store the string 
would be less, but the code to do the glyph-by-glyph iteration at rendering 
time would become more “branchy”, and that is also bad for CPU cache 
performance.

Oh but there’s another way of storing glyphs: the list of QScriptItems in the 
text engine.  That looks kindof bulky too, depending how long we keep it around.

So Unicode is a mini-language which has to be interpreted at some point on the 
way to rendering; there’s no pre-interpreted form we could store it in.  
TrueType is also a mini-language.  Maybe it would be possible to write a 
compiler which reads UTF-8 and TrueType and writes (nearly) branch-free code to 
render a whole line or block of text, so we could cache code instead of data.  
It could be more compact and CPU cache-friendly.  I imagine nobody has done 
that yet.  But then if you think about all the fancy stuff TeX can do, it could 
get even more complex than what Qt currently does.  And I don’t understand much 
about what Harfbuzz does yet, either.

___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development