Re: [Development] QtCS2019 Notes: Clang-based cpp parser for lupdate
On Sunday, 24 November 2019 09:25:37 PST Olivier Goffart wrote: > So moc stays much faster than actually compiling, but it has been reported > that moc does not play nice with ccache or distributed compile farm. There's no reason why moc can't be ccache'd. As for a compile farm, the big problem is that moc usually isn't scheduled and the farm doesn't know that you're trying to run it. When you run make -jN with N being the number of slots in your farm, you end up getting N local mocs too. The solution for that is a build tool (make, ninja, etc.) that understands farming. I apply the following hack to force moc scheduling with icecream. $ grep QMAKE_TOOL mkspecs/linux-g++-optimised/qmake.conf QMAKE_TOOL_PREFIX=env ICECC_COLOR_DIAGNOSTICS=0 icecc -- Thiago Macieira - thiago.macieira (AT) intel.com Software Architect - Intel System Software Products ___ Development mailing list Development@qt-project.org https://lists.qt-project.org/listinfo/development
Re: [Development] QtCS2019 Notes: Clang-based cpp parser for lupdate
On 24.11.19 12:24, Giuseppe D'Angelo via Development wrote: Il 21/11/19 17:38, Joerg Bornemann ha scritto: It works and produces seemingly the correct output, able to consume modern C++ constructs. But it's sloow. Running on Qt Creator takes 1 min with the old parser, and 50 min with the new one. Related question: does also a clang-based moc suffer from this extreme slowness? Yes. (Although not as bad as that because there is still room for improvement in lupdate, also lupdate needs to parse function bodies and moc does not) I've tried it on my machine, extracting the moc command line from a build of one project i'm working on, from a random moc file: moc from Qt 5.13.2 0.16s user 0.05s system 99% cpu 0.212 total moc-ng from https://github.com/woboq/moc-ng 1.48s user 0.06s system 98% cpu 1.575 total This is not using pre-compiled header, so there might be some room for improvements. (but even then i wouldn't assume we can use precompiled header for all builds) Note for comparison, the build of the .cpp file that include the .moc with gcc9 3.84s user 0.24s system 99% cpu 4.098 total So moc stays much faster than actually compiling, but it has been reported that moc does not play nice with ccache or distributed compile farm. -- Olivier ___ Development mailing list Development@qt-project.org https://lists.qt-project.org/listinfo/development
Re: [Development] QtCS2019 Notes: Clang-based cpp parser for lupdate
Il 21/11/19 17:38, Joerg Bornemann ha scritto: It works and produces seemingly the correct output, able to consume modern C++ constructs. But it's sloow. Running on Qt Creator takes 1 min with the old parser, and 50 min with the new one. Related question: does also a clang-based moc suffer from this extreme slowness? Thanks, -- Giuseppe D'Angelo | giuseppe.dang...@kdab.com | Senior Software Engineer KDAB (France) S.A.S., a KDAB Group company Tel. France +33 (0)4 90 84 08 53, http://www.kdab.com KDAB - The Qt, C++ and OpenGL Experts smime.p7s Description: Firma crittografica S/MIME ___ Development mailing list Development@qt-project.org https://lists.qt-project.org/listinfo/development
Re: [Development] QtCS2019 Notes: Clang-based cpp parser for lupdate
Am 22.11.19 um 21:25 schrieb Kai Pastor, DG0YT:
Am 22.11.19 um 17:03 schrieb Kai Köhne:
Keep in mind that you need this context available both at runtime and
at lupdate time. That is, the only setup how this IMO would reliably work
is if we make the context a part of the tr() call - effectively calling
QCoreApplication::translate(context, string). This is already possible right
now.
I admit I didn't think about the runtime perspective. But
QCoreApplication::translate("Context", string)
is much more verbose then
Context::tr(string)
or, with a declared and defined Context for the translation unit, just
tr(string)
One more comment: Sometimes I already have to use what "is already
possible right now" due to lupdate's C++11 quirks, and in combination
with the requirement to use a namespace to separate "my" class names
from 3rd-party library identifiers, the verbose form actually spells:
QCoreApplication::translate("::MyNameSpace::Context", string)
Possible, but hardly bearable...
Kai.
___
Development mailing list
Development@qt-project.org
https://lists.qt-project.org/listinfo/development
Re: [Development] QtCS2019 Notes: Clang-based cpp parser for lupdate
On Fri, Nov 22, 2019 at 12:25:21PM +, Joerg Bornemann wrote: On the topic of switching to gettext: My searches yielded nothing. because our archives are incomplete and badly indexed, and clearly nobody bothered to make a jira task (at least a publicly visible one). It would be cool to know where this was discussed before. notes from the 1st qtcs: https://lists.qt-project.org/pipermail/qt5-feedback/2011-June/000632.html small followup: https://lists.qt-project.org/pipermail/qt5-feedback/2011-July/000683.html the real discussion happened on the kde side: http://kde.6490.n7.nabble.com/Translation-in-Qt5-td1135058.html and resulted in a master thesis level document: http://nedohodnik.net/gettextbis/ obviously, this is about a lot more than just switching from l* to gettext - gotta think big when you break everything anyway. ;) ___ Development mailing list Development@qt-project.org https://lists.qt-project.org/listinfo/development
Re: [Development] QtCS2019 Notes: Clang-based cpp parser for lupdate
Am 22.11.19 um 17:03 schrieb Kai Köhne:
From: Development On Behalf Of
Joerg Bornemann
On 11/22/19 7:21 AM, Kai Pastor, DG0YT wrote:
Given the Qt Creator example, would it be feasible to make explicit
declaration of a file's context mandatory, at least for the existing
("fast") parser? If not mandatory, an explicit declaration could at
least disable all automatic detection for the rest of the file, making
it possible to use modern C++ without surprises.
That's certainly a possibility. With an explicit context declaration (that's
easy
to detect) on file level we can use an even simpler parser/extractor that
doesn't have to know much C++ at all.
Keep in mind that you need this context available both at runtime and
at lupdate time. That is, the only setup how this IMO would reliably work
is if we make the context a part of the tr() call - effectively calling
QCoreApplication::translate(context, string). This is already possible right
now.
I admit I didn't think about the runtime perspective. But
QCoreApplication::translate("Context", string)
is much more verbose then
Context::tr(string)
or, with a declared and defined Context for the translation unit, just
tr(string)
I wonder if an anonymous namespace couldn't be used to safely implement
the latter for the proposed explicit declaration per translation unit.
Kai.
___
Development mailing list
Development@qt-project.org
https://lists.qt-project.org/listinfo/development
Re: [Development] QtCS2019 Notes: Clang-based cpp parser for lupdate
> -Original Message-
> From: Development On Behalf Of
> Joerg Bornemann
> Sent: Friday, November 22, 2019 1:25 PM
> To: Kai Pastor, DG0YT ; development@qt-project.org
> Subject: Re: [Development] QtCS2019 Notes: Clang-based cpp parser for
> lupdate
>
> On 11/22/19 7:21 AM, Kai Pastor, DG0YT wrote:
> > Am 21.11.19 um 20:23 schrieb André Pönitz:
> >> On Thu, Nov 21, 2019 at 07:48:41PM +0100, Oswald Buddenhagen wrote:
> >>> a more radical and much simpler approach would be switching to
> >>> gettext
> >> The most prominent difference is the (usually) per-class automatic
> >> context which I always found too clever to be sensible, and that's
> >> effectively also the only reason to need a C++ parser.
> >
> > Indeed, while 'context' is a useful concept, the (implementation)
> > class name often isn't the best 'context' from a more general point of
> > view (features, human translators, screenshots). And if a class does
> > not inherit from QObject, the context already needs to be declared
> explicitly.
> >
> > Given the Qt Creator example, would it be feasible to make explicit
> > declaration of a file's context mandatory, at least for the existing
> > ("fast") parser? If not mandatory, an explicit declaration could at
> > least disable all automatic detection for the rest of the file, making
> > it possible to use modern C++ without surprises.
> >
> > Maybe this could be complemented by a lconvert feature to rename a
> > context in an existing ts-file, for those who want to move away from
> > class names as contexts.
>
> That's certainly a possibility. With an explicit context declaration (that's
> easy
> to detect) on file level we can use an even simpler parser/extractor that
> doesn't have to know much C++ at all.
Keep in mind that you need this context available both at runtime and
at lupdate time. That is, the only setup how this IMO would reliably work
is if we make the context a part of the tr() call - effectively calling
QCoreApplication::translate(context, string). This is already possible right
now.
> On the topic of switching to gettext:
>
> It would be cool to know where this was discussed before. My searches
> yielded nothing.
I remember having a meeting about that in the Trolltech office 2008,
weren't you there?
> It's tempting to switch to an established solution.
> Of course we must make sure that gettext is properly supported on all target
> platforms. It seems to be available via vcpkg on Windows at least.
>
> There needs to be a migration path if we want to use gettext as the
> recommended i18n solution for Qt users (and maybe deprecate the
> lupdate/lrelease). We could provide a migration tool, based on lupdate's
> current parser, that transforms tr("foo") calls to _("foo"). Of course, that
> gets
> more complicated due to the mentioned context differences.
> And Linguist should probably be able to edit .po files.
Well, if people want to use gettext they can already do so now. But switching
to
gettext is a source incompatible change, so even with 'porting' tooling I don't
think
we can remove lupdate / tr() any time soon.
Regards
kai
___
Development mailing list
Development@qt-project.org
https://lists.qt-project.org/listinfo/development
Re: [Development] QtCS2019 Notes: Clang-based cpp parser for lupdate
On 11/22/19 7:21 AM, Kai Pastor, DG0YT wrote:
> Am 21.11.19 um 20:23 schrieb André Pönitz:
>> On Thu, Nov 21, 2019 at 07:48:41PM +0100, Oswald Buddenhagen wrote:
>>> a more radical and much simpler approach would be switching to gettext
>> The most prominent difference is the (usually) per-class automatic
>> context
>> which I always found too clever to be sensible, and that's effectively
>> also the only reason to need a C++ parser.
>
> Indeed, while 'context' is a useful concept, the (implementation) class
> name often isn't the best 'context' from a more general point of view
> (features, human translators, screenshots). And if a class does not
> inherit from QObject, the context already needs to be declared explicitly.
>
> Given the Qt Creator example, would it be feasible to make explicit
> declaration of a file's context mandatory, at least for the existing
> ("fast") parser? If not mandatory, an explicit declaration could at
> least disable all automatic detection for the rest of the file, making
> it possible to use modern C++ without surprises.
>
> Maybe this could be complemented by a lconvert feature to rename a
> context in an existing ts-file, for those who want to move away from
> class names as contexts.
That's certainly a possibility. With an explicit context declaration
(that's easy to detect) on file level we can use an even simpler
parser/extractor that doesn't have to know much C++ at all.
On the topic of switching to gettext:
It would be cool to know where this was discussed before. My searches
yielded nothing.
It's tempting to switch to an established solution.
Of course we must make sure that gettext is properly supported on all
target platforms. It seems to be available via vcpkg on Windows at least.
There needs to be a migration path if we want to use gettext as the
recommended i18n solution for Qt users (and maybe deprecate the
lupdate/lrelease). We could provide a migration tool, based on lupdate's
current parser, that transforms tr("foo") calls to _("foo"). Of course,
that gets more complicated due to the mentioned context differences.
And Linguist should probably be able to edit .po files.
BR,
Joerg
___
Development mailing list
Development@qt-project.org
https://lists.qt-project.org/listinfo/development
Re: [Development] QtCS2019 Notes: Clang-based cpp parser for lupdate
Am 21.11.19 um 20:23 schrieb André Pönitz:
On Thu, Nov 21, 2019 at 07:48:41PM +0100, Oswald Buddenhagen wrote:
a more radical and much simpler approach would be switching to gettext
The most prominent difference is the (usually) per-class automatic context
which I always found too clever to be sensible, and that's effectively
also the only reason to need a C++ parser.
Indeed, while 'context' is a useful concept, the (implementation) class
name often isn't the best 'context' from a more general point of view
(features, human translators, screenshots). And if a class does not
inherit from QObject, the context already needs to be declared explicitly.
Given the Qt Creator example, would it be feasible to make explicit
declaration of a file's context mandatory, at least for the existing
("fast") parser? If not mandatory, an explicit declaration could at
least disable all automatic detection for the rest of the file, making
it possible to use modern C++ without surprises.
Maybe this could be complemented by a lconvert feature to rename a
context in an existing ts-file, for those who want to move away from
class names as contexts.
___
Development mailing list
Development@qt-project.org
https://lists.qt-project.org/listinfo/development
Re: [Development] QtCS2019 Notes: Clang-based cpp parser for lupdate
On Thu, Nov 21, 2019 at 07:48:41PM +0100, Oswald Buddenhagen wrote: > On Thu, Nov 21, 2019 at 04:38:28PM +, Joerg Bornemann wrote: > > There is the ongoing effort to replace the old outdated parser with a > > clang, similar to what's been done in qdoc. The corresponding JIRA > > issue is QTBUG-71835. > > > a more radical and much simpler approach would be switching to gettext > (actually, or at least in fundamental functionality) - that doesn't need a > whole c++ parser to do a sensible job. > > oh, wait, that topic ain't new ... ;) I wasn't aware of a related discussion, and I actually think this could be a way forward. The most prominent difference is the (usually) per-class automatic context which I always found too clever to be sensible, and that's effectively also the only reason to need a C++ parser. Andre' ___ Development mailing list Development@qt-project.org https://lists.qt-project.org/listinfo/development
Re: [Development] QtCS2019 Notes: Clang-based cpp parser for lupdate
On Thu, Nov 21, 2019 at 04:38:28PM +, Joerg Bornemann wrote: There is the ongoing effort to replace the old outdated parser with a clang, similar to what's been done in qdoc. The corresponding JIRA issue is QTBUG-71835. a more radical and much simpler approach would be switching to gettext (actually, or at least in fundamental functionality) - that doesn't need a whole c++ parser to do a sensible job. oh, wait, that topic ain't new ... ;) ___ Development mailing list Development@qt-project.org https://lists.qt-project.org/listinfo/development
