Re: [Development] Cut 7+ms of Qt5 startup time on Linux
Before going to a binary format, I’d first like to check whether we can further speed up the parsing. I’ve done some work in that area in spring, but I know there was still quite some room to improve it. Cheers, Lars From: Jiergir Ogoerg f35f22...@gmail.commailto:f35f22...@gmail.com Date: torsdag 31. oktober 2013 04:10 To: development@qt-project.orgmailto:development@qt-project.org development@qt-project.orgmailto:development@qt-project.org Subject: [Development] Cut 7+ms of Qt5 startup time on Linux Hi, Since each qt (gui) app on Linux parses the text file /usr/share/X11/locale/en_US.UTF-8/Compose line by line which on my computer is 549Kib (almost 6000 lines) it takes on my 64bit OS with Intel Core i5 about 7ms to parse, so the idea is to cache the locale file in binary format in the home/.kde directory (the binary file is also smaller than the other one), this new way takes less than 1ms to load since there's pretty much nothing to parse. That's the core idea. The user's locale changes (very) rarely if at all, so pretty much all the time he'll load the cached (quick) version, but when the locale does change or gets updated - the cache is updated accordingly. If this approach isn't good stop reading here and please let me know. I attached the 2 files in zip format. The source code is rather dirty cause it doesn't strictly adhere to the Qt coding style - my goal was a working solution. Since I don't represent any company and the changes are simple would somebody please take the changes and shape them into a mergeable commit or must I take all the steps to become an official contributor and polish the code to a mergeable state myself? Only 2 files affected: /qtbase/src/plugins/platforminputcontexts/compose/generator/qtablegenerator.cpp and /qtbase/src/plugins/platforminputcontexts/compose/generator/qtablegenerator.h ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] Add-ons: Removing obsolete API in the same major version
On Thu, Oct 31, 2013 at 1:42 AM, Thiago Macieira thiago.macie...@intel.comwrote: Should we consider so, how about undeprecating API before Qt 6.0 deprecated in Qt 5.X? Undeprecating API is fine. Thanks. ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] Qt 5.2 Beta, possible memory leak(s)
Hi, I dislike those because we're incurring overhead at shutdown to do something that doesn't need doing, except to keep tools happy. For the platform printer support, it makes sense since it can potentially have handles open or similar. Not sure about the ICU leak, though? Regards, Friedemann -- Friedemann Kleint Digia, Qt ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] Cut 7+ms of Qt5 startup time on Linux
There is already a caching mechanism in place for the Compose file, can we make use of that? - http://kdemonkey.blogspot.co.uk/2008/04/magic-trick.html On 31 October 2013 07:38, Knoll Lars lars.kn...@digia.com wrote: Before going to a binary format, I’d first like to check whether we can further speed up the parsing. I’ve done some work in that area in spring, but I know there was still quite some room to improve it. Cheers, Lars From: Jiergir Ogoerg f35f22...@gmail.com Date: torsdag 31. oktober 2013 04:10 To: development@qt-project.org development@qt-project.org Subject: [Development] Cut 7+ms of Qt5 startup time on Linux Hi, Since each qt (gui) app on Linux parses the text file /usr/share/X11/locale/en_US.UTF-8/Compose line by line which on my computer is 549Kib (almost 6000 lines) it takes on my 64bit OS with Intel Core i5 about 7ms to parse, so the idea is to cache the locale file in binary format in the home/.kde directory (the binary file is also smaller than the other one), this new way takes less than 1ms to load since there's pretty much nothing to parse. That's the core idea. The user's locale changes (very) rarely if at all, so pretty much all the time he'll load the cached (quick) version, but when the locale does change or gets updated - the cache is updated accordingly. If this approach isn't good stop reading here and please let me know. I attached the 2 files in zip format. The source code is rather dirty cause it doesn't strictly adhere to the Qt coding style - my goal was a working solution. Since I don't represent any company and the changes are simple would somebody please take the changes and shape them into a mergeable commit or must I take all the steps to become an official contributor and polish the code to a mergeable state myself? Only 2 files affected: /qtbase/src/plugins/platforminputcontexts/compose/generator/qtablegenerator.cpp and /qtbase/src/plugins/platforminputcontexts/compose/generator/qtablegenerator.h ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
[Development] Qt 5.2 Beta, possible memory leak(s)
Hi, i agree and for the ICU part. Calling u_cleanup will free resources owned by ICU libraries. Converters used by Qt are already properly released so this is not an Qt issue. From ICU Api help: The use of u_cleanup() just before an application terminates is optional, but it should be called only once for performance reasons. The primary benefit is to eliminate reports of memory or resource leaks originating in ICU code from the results generated by heap analysis tools. Regards, Luka Dne 31. okt. 2013 09:36 je oseba Friedemann Kleint friedemann.kle...@digia.com napisala: Hi, I dislike those because we're incurring overhead at shutdown to do something that doesn't need doing, except to keep tools happy. For the platform printer support, it makes sense since it can potentially have handles open or similar. Not sure about the ICU leak, though? Regards, Friedemann -- Friedemann Kleint Digia, Qt ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] Qt 5.2 Beta, possible memory leak(s)
-Original Message- From: development-bounces+kai.koehne=digia@qt-project.org [mailto:development-bounces+kai.koehne=digia@qt-project.org] On Behalf Of Luka Vilar Sent: Thursday, October 31, 2013 12:22 PM To: Kleint Friedemann Cc: development@qt-project.org Subject: [Development] Qt 5.2 Beta, possible memory leak(s) Hi, i agree and for the ICU part. Calling u_cleanup will free resources owned by ICU libraries. Converters used by Qt are already properly released so this is not an Qt issue. From ICU Api help: The use of u_cleanup() just before an application terminates is optional, but it should be called only once for performance reasons. The primary benefit is to eliminate reports of memory or resource leaks originating in ICU code from the results generated by heap analysis tools. I'm not completely opposed to calling u_cleanup(), but we've to keep in mind that ICU API might get called very late (e.g. even in destructors of static objects, when someone decides to add say a qDebug() there). What happens if I call ICU API after u_cleanup() is called? Will it just miss data, or crash? This would need to be thoroughly tested ... Regards Kai ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] Qt 5.2 Beta, possible memory leak(s)
Le 31/10/2013 12:29, Koehne Kai a écrit : I'm not completely opposed to calling u_cleanup(), but we've to keep in mind that ICU API might get called very late (e.g. even in destructors of static objects, when someone decides to add say a qDebug() there). What happens if I call ICU API after u_cleanup() is called? Will it just miss data, or crash? This would need to be thoroughly tested ... Seems a good candidate for a phoenix singleton[1], then. Wouldn’t it make sense to call u_cleanup only in debug mode ? Since it is just a matter of false positive in memory leaks analyzers…[2] Regards, Julien [1] as per Alexandrescu’s definition in modern c++ [2] the issue may not be as neglictible as it seems. Having no memory leaks reported is a sane check in a continuous integration system. ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] Cut 7+ms of Qt5 startup time on Linux
The .compose-cache file stays empty even after a few reboots. Using Fedora 20 KDE version, 64 bit. I noticed the post is 5+ years old, has it been tested on (modern) Fedora and does it still work with modern versions of Kubuntu? Particularly, what should I do to make it work on Fedora (20)? On Thu, Oct 31, 2013 at 12:09 PM, Robert Knight robertkni...@gmail.comwrote: There is already a caching mechanism in place for the Compose file, can we make use of that? - http://kdemonkey.blogspot.co.uk/2008/04/magic-trick.html On 31 October 2013 07:38, Knoll Lars lars.kn...@digia.com wrote: Before going to a binary format, I’d first like to check whether we can further speed up the parsing. I’ve done some work in that area in spring, but I know there was still quite some room to improve it. Cheers, Lars From: Jiergir Ogoerg f35f22...@gmail.com Date: torsdag 31. oktober 2013 04:10 To: development@qt-project.org development@qt-project.org Subject: [Development] Cut 7+ms of Qt5 startup time on Linux Hi, Since each qt (gui) app on Linux parses the text file /usr/share/X11/locale/en_US.UTF-8/Compose line by line which on my computer is 549Kib (almost 6000 lines) it takes on my 64bit OS with Intel Core i5 about 7ms to parse, so the idea is to cache the locale file in binary format in the home/.kde directory (the binary file is also smaller than the other one), this new way takes less than 1ms to load since there's pretty much nothing to parse. That's the core idea. The user's locale changes (very) rarely if at all, so pretty much all the time he'll load the cached (quick) version, but when the locale does change or gets updated - the cache is updated accordingly. If this approach isn't good stop reading here and please let me know. I attached the 2 files in zip format. The source code is rather dirty cause it doesn't strictly adhere to the Qt coding style - my goal was a working solution. Since I don't represent any company and the changes are simple would somebody please take the changes and shape them into a mergeable commit or must I take all the steps to become an official contributor and polish the code to a mergeable state myself? Only 2 files affected: /qtbase/src/plugins/platforminputcontexts/compose/generator/qtablegenerator.cpp and /qtbase/src/plugins/platforminputcontexts/compose/generator/qtablegenerator.h ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] Qt 5.2 Beta, possible memory leak(s)
On quinta-feira, 31 de outubro de 2013 12:21:46, Luka Vilar wrote: i agree and for the ICU part. Calling u_cleanup will free resources owned by ICU libraries. Converters used by Qt are already properly released so this is not an Qt issue. Let me ask you this: what happens if Qt was loaded by a plugin to the main application, this plugin gets unloaded (thus, the Q_GLOBAL_STATIC destructors get run), but ICU is in use by the application? How will ICU react to u_cleanup being run and then more functions get called? -- Thiago Macieira - thiago.macieira (AT) intel.com Software Architect - Intel Open Source Technology Center signature.asc Description: This is a digitally signed message part. ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] Cut 7+ms of Qt5 startup time on Linux
On quinta-feira, 31 de outubro de 2013 07:38:00, Knoll Lars wrote: Before going to a binary format, I’d first like to check whether we can further speed up the parsing. I’ve done some work in that area in spring, but I know there was still quite some room to improve it. I'm also improving the fromUtf8 code. My current benchmarks are showing an improvement of 2x in the new plain code and 12x with SIMD improvements, for mostly ASCII input. One further step is to detect an UTF-8 locale and use our optimised code, instead of letting ICU do the encoding. -- Thiago Macieira - thiago.macieira (AT) intel.com Software Architect - Intel Open Source Technology Center signature.asc Description: This is a digitally signed message part. ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] QSharedPointer - Enable shared from this
On quinta-feira, 31 de outubro de 2013 15:57:13, Cristian Pallarés wrote: Hi folks, https://codereview.qt-project.org/60346 I've waited quite a bit for this feature. C++11 already ships this implementation, which makes std::shared_ptr very powerful, but the review process for the Qt implementation looks frozen. QSharedPointer is limited to me for that missing feature. For example, I want to make a tree class that uses shared pointers. Without shared from this I have to write very ugly things such privatizing the class constructors and implementing a public static constructor that creates the shared pointer for me and stores a weak pointer to the class itself. I'm wondering if could someone take a look at that. That would be me. Sorry, I haven't looked at it in a while... The way I work is that I ignore the topmost reviews in my dashboard because they're old and useless. Looks like this one shouldn't have been ignored. -- Thiago Macieira - thiago.macieira (AT) intel.com Software Architect - Intel Open Source Technology Center signature.asc Description: This is a digitally signed message part. ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] Cut 7+ms of Qt5 startup time on Linux
Great, of course you're a much better/smarter programmer than me,
but how much time of the whole parsing and loading
(which is done line by line which is slow) does fromUtf8()
take? If it's only like 10-20% then even if it's optimized to work 10 times
faster in the end it's a much smaller improvement compared to
binary loading which almost doesn't do any parsing at all, just copying
from disk onto ready memory.
If the total parsing time after your optimizations
is still like 2x+ times slower than loading the binary cache then imo
your optimizations should be incorporated anyway for the
(rare) cases when there's no (binary) cache yet or when it must be updated.
The type of code (in a loop) I mentioned which features lots of other
decoding/operations
besides fromUtf8():
char l[1024];
// formating of compose.dir has some inconsistencies
while (!mappings.atEnd()) {
int read = mappings.readLine(l, sizeof(l));
if (read = 0)
break;
char *line = l;
if (*line = 'a' *line = 'z') {
// file name
while (*line *line != ':' *line != ' ' *line !=
'\t')
++line;
if (!*line)
continue;
const char * const composeFileNameEnd = line;
*line = '\0';
++line;
// locale name
while (*line (*line == ' ' || *line == '\t'))
++line;
const char * const lc = line;
while (*line *line != ' ' *line != '\t' *line !=
'\n')
++line;
*line = '\0';
if (localeNameLength == (line - lc) !strncasecmp(lc,
localeData, line - lc)) {
file = QString::fromUtf8(l, composeFileNameEnd - l);
break;
}
}
}
mappings.close();
I wonder if we could test your optimizations soon?
Is there a list of other startup optimizations you're working on
so that we don't dilute the efforts by working on the same thing?
On Thu, Oct 31, 2013 at 5:22 PM, Thiago Macieira
thiago.macie...@intel.comwrote:
On quinta-feira, 31 de outubro de 2013 07:38:00, Knoll Lars wrote:
Before going to a binary format, I’d first like to check whether we can
further speed up the parsing. I’ve done some work in that area in spring,
but I know there was still quite some room to improve it.
I'm also improving the fromUtf8 code. My current benchmarks are showing an
improvement of 2x in the new plain code and 12x with SIMD improvements, for
mostly ASCII input.
One further step is to detect an UTF-8 locale and use our optimised code,
instead of letting ICU do the encoding.
--
Thiago Macieira - thiago.macieira (AT) intel.com
Software Architect - Intel Open Source Technology Center
___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development
___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] Cut 7+ms of Qt5 startup time on Linux
I noticed the post is 5+ years old, has it been tested
on (modern) Fedora and does it still work with modern versions of Kubuntu?
I haven't tested it myself recently and I have no idea whether it
works with current versions of Ubuntu or whether
it ever worked on Fedora.
Regards,
Rob.
On 31 October 2013 15:52, Jiergir Ogoerg f35f22...@gmail.com wrote:
Great, of course you're a much better/smarter programmer than me,
but how much time of the whole parsing and loading
(which is done line by line which is slow) does fromUtf8()
take? If it's only like 10-20% then even if it's optimized to work 10 times
faster in the end it's a much smaller improvement compared to
binary loading which almost doesn't do any parsing at all, just copying
from disk onto ready memory.
If the total parsing time after your optimizations
is still like 2x+ times slower than loading the binary cache then imo
your optimizations should be incorporated anyway for the
(rare) cases when there's no (binary) cache yet or when it must be updated.
The type of code (in a loop) I mentioned which features lots of other
decoding/operations
besides fromUtf8():
char l[1024];
// formating of compose.dir has some inconsistencies
while (!mappings.atEnd()) {
int read = mappings.readLine(l, sizeof(l));
if (read = 0)
break;
char *line = l;
if (*line = 'a' *line = 'z') {
// file name
while (*line *line != ':' *line != ' ' *line !=
'\t')
++line;
if (!*line)
continue;
const char * const composeFileNameEnd = line;
*line = '\0';
++line;
// locale name
while (*line (*line == ' ' || *line == '\t'))
++line;
const char * const lc = line;
while (*line *line != ' ' *line != '\t' *line !=
'\n')
++line;
*line = '\0';
if (localeNameLength == (line - lc) !strncasecmp(lc,
localeData, line - lc)) {
file = QString::fromUtf8(l, composeFileNameEnd - l);
break;
}
}
}
mappings.close();
I wonder if we could test your optimizations soon?
Is there a list of other startup optimizations you're working on
so that we don't dilute the efforts by working on the same thing?
On Thu, Oct 31, 2013 at 5:22 PM, Thiago Macieira thiago.macie...@intel.com
wrote:
On quinta-feira, 31 de outubro de 2013 07:38:00, Knoll Lars wrote:
Before going to a binary format, I’d first like to check whether we can
further speed up the parsing. I’ve done some work in that area in
spring,
but I know there was still quite some room to improve it.
I'm also improving the fromUtf8 code. My current benchmarks are showing an
improvement of 2x in the new plain code and 12x with SIMD improvements,
for
mostly ASCII input.
One further step is to detect an UTF-8 locale and use our optimised code,
instead of letting ICU do the encoding.
--
Thiago Macieira - thiago.macieira (AT) intel.com
Software Architect - Intel Open Source Technology Center
___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development
___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development
___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] Cut 7+ms of Qt5 startup time on Linux
Am Donnerstag, 31. Oktober 2013, 17:52:28 schrieb Jiergir Ogoerg:
Great, of course you're a much better/smarter programmer than me,
but how much time of the whole parsing and loading
(which is done line by line which is slow) does fromUtf8()
take? If it's only like 10-20% then even if it's optimized to work 10 times
faster in the end it's a much smaller improvement compared to
binary loading which almost doesn't do any parsing at all, just copying
from disk onto ready memory.
Hey,
I was curious about performance bottlenecks here, too, this morning. So I had
a quick look.
Results:
Running the attached program (main.cpp) compiled with 'g++ -g -O2 ...' through
callgrind yielded the following results for me:
Just showing the significant measurements:
* ~50% time (incl.) spent in xkb_keysym_from_name
* ~21% time (incl.) spent in QString::fromUtf8_helper
So it seems there's indeed potential that optimizing QString::fromUtf8 would
help. Not sure if there's any benefit in looking into xkb_keysym_from_name
from xkb-common, which seems to be another bottleneck.
If the total parsing time after your optimizations
is still like 2x+ times slower than loading the binary cache then imo
your optimizations should be incorporated anyway for the
(rare) cases when there's no (binary) cache yet or when it must be updated.
The type of code (in a loop) I mentioned which features lots of other
decoding/operations
besides fromUtf8():
(snip)
While having a quick look at the code in qtablegenerator.cpp I found a minor
performance bottleneck, patch attached.
With this patch, some iterations of the for-loop inside
TableGenerator::parseKeySequence are saved. Given that the amount of keys
inside the compose text file is significant smaller than
QT_KEYSEQUENCE_MAX_LEN (== 6) in average.
Callgrind results (Instruction fetches):
* Without patch: ~34,350k
* Withpatch: ~34,200k
Minor improvement, yes.
If the patch makes sense (i.e. if it is correct) I can put it on Gerrit.
Greets
--
Kevin Funk#include qtablegenerator.h
#include ctime
#include iostream
using namespace std;
int main()
{
QFile file(/usr/share/X11/locale/en_US.UTF-8/Compose);
bool rc = file.open(QIODevice::ReadOnly);
Q_ASSERT(rc);
clock_t begin = clock();
TableGenerator gen;
gen.parseComposeFile(file);
clock_t end = clock();
double elapsed_secs = double(end - begin) / CLOCKS_PER_SEC;
cout elapsed_secs endl;
}
diff --git a/src/plugins/platforminputcontexts/compose/generator/qtablegenerator.cpp b/src/plugins/platforminputcontexts/compose/generator/qtablegenerator.cpp
index 4cfc2a6..38cf267 100644
--- a/src/plugins/platforminputcontexts/compose/generator/qtablegenerator.cpp
+++ b/src/plugins/platforminputcontexts/compose/generator/qtablegenerator.cpp
@@ -350,7 +350,7 @@ void TableGenerator::parseKeySequence(char *line)
if (!keysEnd)
return;
-QComposeTableElement elem;
+QComposeTableElement elem = {};
// find the composed value - strings may be direct text encoded in the locale
// for which the compose file is to be used, or an escaped octal or hexadecimal
// character code. Octal codes are specified as \123 and hexadecimal codes as \0x123a.
@@ -409,7 +409,7 @@ void TableGenerator::parseKeySequence(char *line)
qWarning() QString(Qt Warning - invalid keysym: %1).arg(sym);
}
} else {
-elem.keys[i] = 0;
+break;
}
}
m_composeTable.append(elem);___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development
[Development] QML, Widgets, and setParent
Hi, I'm trying to port KDAB's DeclarativeWidgets[1] to Qt5. I've got it running for trivial QML files, however as soon as there are child widgets the running program aborts. The abort is caused by QObject::setParent, which contains this little gem: Q_ASSERT(!d-isWidget); Apart from not understanding why there is a significant difference between QObjects and QWidgets in regards to parenting... There used to be code that handled this in qqmlvme.cpp[2], but it has been disabled by an #if 0 . Apparently to remove dependencies on QtWidgets from QtQml. I kind of ran out of ideas on how to solve this. Should I consider setParent, qqmlvme or the direct use of QWidgets as a bug? Konrad [1] It enables you to create widget based GUIs using QML. git://github.com/KDAB/DeclarativeWidgets [2] In 5.2 beta this is line 627 in qqmlvme.cpp. It's Halloween. Go ahead, look at scary code! ;-) signature.asc Description: This is a digitally signed message part. ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] QML, Widgets, and setParent
I'd argue that it's setParent with the bug. It's conceptually valid to have a QWidget with a QObject parent but no QWidget parent, but it won't really show up until you try to use them in QML. If you do this with non-QML QWidgets, then it won't show up on screen so the user will think they've made an error (but with QML there's an intermediate state it goes through even in a correct usage case). QML as a language requires that you can set the QObject::parent on instances of QML types, so if that QObject assert is valid it means you can't use widgets in QML without hacks (specifically, that exact hack you found already which doesn't allow intermingling). -- Alan Alpert On Thu, Oct 31, 2013 at 12:17 PM, Konrad Rosenbaum kon...@silmor.de wrote: Hi, I'm trying to port KDAB's DeclarativeWidgets[1] to Qt5. I've got it running for trivial QML files, however as soon as there are child widgets the running program aborts. The abort is caused by QObject::setParent, which contains this little gem: Q_ASSERT(!d-isWidget); Apart from not understanding why there is a significant difference between QObjects and QWidgets in regards to parenting... There used to be code that handled this in qqmlvme.cpp[2], but it has been disabled by an #if 0 . Apparently to remove dependencies on QtWidgets from QtQml. I kind of ran out of ideas on how to solve this. Should I consider setParent, qqmlvme or the direct use of QWidgets as a bug? Konrad [1] It enables you to create widget based GUIs using QML. git://github.com/KDAB/DeclarativeWidgets [2] In 5.2 beta this is line 627 in qqmlvme.cpp. It's Halloween. Go ahead, look at scary code! ;-) ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
