Re: [Development] QMetaMethod in Qt 6
On Fri, May 29, 2020 at 9:52 AM Oswald Buddenhagen < oswald.buddenha...@gmx.de> wrote: > On Fri, May 29, 2020 at 06:30:07AM -0700, Adam Light wrote: > >I will note, however, that #including the moc output from the .cpp > >file (the "moc_myclass.cpp" form) is not compatible with my mocinclude > >trick posted at https://bugreports.qt.io/browse/QTBUG-81348. > > > the report doesn't mention 'mocinclude', so please be a bit more > specific. > if you actually mean 'moccombine', then the issue appears to be at worst > one of current implementation, not fundamental. Yes, sorry, I meant "moccombine". I agree that the problem is an implementation problem, and that it can be fixed. Adam ___ Development mailing list Development@qt-project.org https://lists.qt-project.org/listinfo/development
Re: [Development] QMetaMethod in Qt 6
On Fri, May 29, 2020 at 06:30:07AM -0700, Adam Light wrote: I will note, however, that #including the moc output from the .cpp file (the "moc_myclass.cpp" form) is not compatible with my mocinclude trick posted at https://bugreports.qt.io/browse/QTBUG-81348. the report doesn't mention 'mocinclude', so please be a bit more specific. if you actually mean 'moccombine', then the issue appears to be at worst one of current implementation, not fundamental. ___ Development mailing list Development@qt-project.org https://lists.qt-project.org/listinfo/development
Re: [Development] QMetaMethod in Qt 6
[Resending to the entire list this time] On Thu, May 28, 2020 at 3:12 PM Thiago Macieira wrote: > On Thursday, 28 May 2020 10:19:58 PDT Adam Light wrote: > > I'm including generating moc files in "build time". I'm not saying that > > compiling the .cpp files will take significantly longer, but if a .cpp > file > > has a #include "myclass.moc" type statement, that .cpp file has to be > > processed by moc > > You're talking about when myclass.cpp has a Q_OBJECT. That's the only > reason > when moc needs to parse that .cpp. > > But you also don't have a choice. You have to #include the .moc output. > Yes, that's right. > We're talking about #include "moc_myclass.cpp", which is when "myclass.h" > has > a Q_OBJECT. You don't have to have that #include. But if you do, then it's > one > fewer .cpp file that needs to be compiled. It generates better code and > enables more warnings. > > The only drawback is that the compilation needs to wait for moc to finish. > If > you have a cluster, it means the cluster can't be used until moc has > finished, > for that file. But that's only for the first file. > Right. I got confused about the earlier mention of ".moc", even though it clearly would not make sense for every class that uses Q_OBJECT to be declared in a .cpp file. I will note, however, that #including the moc output from the .cpp file (the "moc_myclass.cpp" form) is not compatible with my mocinclude trick posted at https://bugreports.qt.io/browse/QTBUG-81348. And we are not using a cluster or anything particularly fancy, just a regular (self-built) desktop machine with a 16 core/32 thread AMD Threadripper processor and otherwise high end consumer components. Adam ___ Development mailing list Development@qt-project.org https://lists.qt-project.org/listinfo/development
Re: [Development] QMetaMethod in Qt 6
On Thursday, 28 May 2020 10:19:58 PDT Adam Light wrote: > I'm including generating moc files in "build time". I'm not saying that > compiling the .cpp files will take significantly longer, but if a .cpp file > has a #include "myclass.moc" type statement, that .cpp file has to be > processed by moc You're talking about when myclass.cpp has a Q_OBJECT. That's the only reason when moc needs to parse that .cpp. But you also don't have a choice. You have to #include the .moc output. We're talking about #include "moc_myclass.cpp", which is when "myclass.h" has a Q_OBJECT. You don't have to have that #include. But if you do, then it's one fewer .cpp file that needs to be compiled. It generates better code and enables more warnings. The only drawback is that the compilation needs to wait for moc to finish. If you have a cluster, it means the cluster can't be used until moc has finished, for that file. But that's only for the first file. -- Thiago Macieira - thiago.macieira (AT) intel.com Software Architect - Intel System Software Products ___ Development mailing list Development@qt-project.org https://lists.qt-project.org/listinfo/development
Re: [Development] QMetaMethod in Qt 6
On 5/28/20 8:21 PM, Matthew Woehlke wrote: if a .cpp file has a #include "myclass.moc" type statement, that .cpp file has to be processed by moc Huh?*Why*? A direct use case of this is to support having Q_OBJECT classes defined in a .cpp file. That requires moc to parse foo.cpp file and produce foo.moc. The question is, does this happen unconditionally when a #include "foo.moc" appears a .cpp, even if that .cpp is not using Q_OBJECT & friends? Why would it be necessary in this case? My 2 c, -- Giuseppe D'Angelo | giuseppe.dang...@kdab.com | Senior Software Engineer KDAB (France) S.A.S., a KDAB Group company Tel. France +33 (0)4 90 84 08 53, http://www.kdab.com KDAB - The Qt, C++ and OpenGL Experts smime.p7s Description: S/MIME Cryptographic Signature ___ Development mailing list Development@qt-project.org https://lists.qt-project.org/listinfo/development
Re: [Development] QMetaMethod in Qt 6
On 28/05/2020 13.19, Adam Light wrote: if a .cpp file has a #include "myclass.moc" type statement, that .cpp file has to be processed by moc Huh? *Why*? AFAIK that's just... wrong. MOC needs to process files that do metaobject things (e.g. use Q_OBJECT). Including a .moc in a .cpp does not affect whether that .cpp does metoobject things. Either you already needed to process it, or you don't, and adding a .moc include doesn't change that. (And your *build system* should be, without doing *any* sort of resolution of includes, processing your input files to determine whether or not to even *invoke* MOC at all.) This sounds like a problem with your build setup. -- Matthew ___ Development mailing list Development@qt-project.org https://lists.qt-project.org/listinfo/development
Re: [Development] QMetaMethod in Qt 6
On Thu, May 28, 2020 at 9:55 AM Oswald Buddenhagen < oswald.buddenha...@gmx.de> wrote: > On Thu, May 28, 2020 at 08:43:14AM -0700, Adam Light wrote: > >> [include mocs] > >> > >Changing Qt in a way that would require #including the moc output from > >the .cpp file might cause noticeable increase in build times unless moc > >is also changed. > > > care to explain how _exactly_ that would be the case? your report > demonstrates an issue with _generating_ the moc files, not compiling > them. I'm including generating moc files in "build time". I'm not saying that compiling the .cpp files will take significantly longer, but if a .cpp file has a #include "myclass.moc" type statement, that .cpp file has to be processed by moc, which will require moc to satisfy all other #includes in that .cpp file, which could be significantly slower than if moc processed the class declaration in the .h file, since .h files typically forward declare to the extent possible to minimize the number of included files. As I pointed out in the issue, mocing all files in our application takes about 1/3 of the time of the entire build under the *best* circumstances (see first table in the Possible Solutions, bindflt OFF column). Under non-optimal circumstances (which can be tricky to avoid), mocing all files takes roughly 2.5 times the *rest of the build* combined (same table, compare bindflt OFF versus bindflt ON in Default moc row, also demonstrated in the last table in the Additional complication section, which shows Creator build times). Note that when comparing bindflt ON vs. bindflt OFF times in any of those tables, almost 100% of the difference between ON and OFF is spent in moc.exe. As I explained in Note 1 of the bug report, the VS compiler uses different system API calls to satisfy #includes than moc does, so the compiler itself isn't a bottleneck. Adam ___ Development mailing list Development@qt-project.org https://lists.qt-project.org/listinfo/development
Re: [Development] QMetaMethod in Qt 6
On Thu, May 28, 2020 at 08:43:14AM -0700, Adam Light wrote: [include mocs] Changing Qt in a way that would require #including the moc output from the .cpp file might cause noticeable increase in build times unless moc is also changed. care to explain how _exactly_ that would be the case? your report demonstrates an issue with _generating_ the moc files, not compiling them. ___ Development mailing list Development@qt-project.org https://lists.qt-project.org/listinfo/development
Re: [Development] QMetaMethod in Qt 6
On Wed, May 27, 2020 at 8:51 AM Thiago Macieira wrote: > On Wednesday, 27 May 2020 03:42:19 PDT Oswald Buddenhagen wrote: > > > this is not something we can subject our users to. > > > > orly? kde had been doing that for quite a while. > > And I fixed QtCore to do the same. > > The only reason not to include the moc output in your .cpp is if you don't > have one (a header-only class whose only non-inline methods are the moc- > generated ones). Otherwise, #include your mocs. > > That said, we shouldn't enforce this. > > On Windows, the way moc satisfies #includes is very slow, particularly when many (32 in my case) moc processes are running concurrently. Since .cpp files tend to result in many more included files, #including a .moc file from a .cpp file as a regular practice could significantly increase build times, and it also prevents an optimization I came up with that mitigates the poor moc performance (see https://bugreports.qt.io/browse/QTBUG-81348 for details). As core count increases, moc's performance might impact more users, though it may not be obvious to many of them. Changing Qt in a way that would require #including the moc output from the .cpp file might cause noticeable increase in build times unless moc is also changed. Adam ___ Development mailing list Development@qt-project.org https://lists.qt-project.org/listinfo/development
Re: [Development] QMetaMethod in Qt 6
On Thursday, 28 May 2020 02:06:01 PDT Shawn Rutledge wrote: > > On 2020 May 27, at 17:50, Thiago Macieira > > wrote:> > > On Wednesday, 27 May 2020 03:42:19 PDT Oswald Buddenhagen wrote: > >>> this is not something we can subject our users to. > >> > >> orly? kde had been doing that for quite a while. > > > > And I fixed QtCore to do the same. > > > > The only reason not to include the moc output in your .cpp is if you don't > > have one (a header-only class whose only non-inline methods are the moc- > > generated ones). Otherwise, #include your mocs. > > The reason is to speed up compilation, right? Is there another reason? Aside from that benefit and the reason for this thread, it enables some warnings in Clang that aren't otherwise. If it can see all members of a class, including non-inline, it can tell if you forgot to use or initialise some of them. Plus the benefit of more inlining, as there's more the compiler can see. -- Thiago Macieira - thiago.macieira (AT) intel.com Software Architect - Intel System Software Products ___ Development mailing list Development@qt-project.org https://lists.qt-project.org/listinfo/development
Re: [Development] QMetaMethod in Qt 6
this is not something we can subject our users to. On Wednesday, 27 May 2020 03:42:19 PDT Oswald Buddenhagen wrote: >>> orly? kde had been doing that for quite a while. On 2020 May 27, at 17:50, Thiago Macieira wrote: >> And I fixed QtCore to do the same. >> >> The only reason not to include the moc output in your .cpp is if you >> don't have one (a header-only class whose only non-inline methods are >> the moc- generated ones). Otherwise, #include your mocs. Shawn Rutledge (28 May 2020 11:06) > The reason is to speed up compilation, right? Is there another reason? Yes, see earlier in this thread: if you #include your .moc in your .cpp, your .h can get away with forward-declaring some classes whose headers it doesn't #include; the .moc may need to see the actual definition, rather than a forward declaration, and the .cpp shall do the needed #include, hence make the definition visible, which the .moc then benefits from by being #included in the .cpp. Eddy. ___ Development mailing list Development@qt-project.org https://lists.qt-project.org/listinfo/development
Re: [Development] QMetaMethod in Qt 6
> On 2020 May 27, at 17:50, Thiago Macieira wrote: > > On Wednesday, 27 May 2020 03:42:19 PDT Oswald Buddenhagen wrote: >>> this is not something we can subject our users to. >> >> orly? kde had been doing that for quite a while. > > And I fixed QtCore to do the same. > > The only reason not to include the moc output in your .cpp is if you don't > have one (a header-only class whose only non-inline methods are the moc- > generated ones). Otherwise, #include your mocs. The reason is to speed up compilation, right? Is there another reason? ___ Development mailing list Development@qt-project.org https://lists.qt-project.org/listinfo/development
Re: [Development] QMetaMethod in Qt 6
On Wednesday, 27 May 2020 03:42:19 PDT Oswald Buddenhagen wrote: > > this is not something we can subject our users to. > > orly? kde had been doing that for quite a while. And I fixed QtCore to do the same. The only reason not to include the moc output in your .cpp is if you don't have one (a header-only class whose only non-inline methods are the moc- generated ones). Otherwise, #include your mocs. That said, we shouldn't enforce this. -- Thiago Macieira - thiago.macieira (AT) intel.com Software Architect - Intel System Software Products ___ Development mailing list Development@qt-project.org https://lists.qt-project.org/listinfo/development
Re: [Development] QMetaMethod in Qt 6
On Wed, May 27, 2020 at 08:21:48AM +, Fabian Kosmale wrote: the only way to get this to work is to include the moc file in the same cpp file. While doing this inside of Qt is possible, this is not something we can subject our users to. orly? kde had been doing that for quite a while. also, it wouldn't be exactly rocket science to make the build system wrap each cpp with its corresponding moc file before passing it to the compiler. the matching would be heuristical, so it would still need manual includes in corner cases, but that's hardly a big deal. ___ Development mailing list Development@qt-project.org https://lists.qt-project.org/listinfo/development
[Development] QMetaMethod in Qt 6
The task: Currently, when using QMetaMethod::parameterType, we end up doing a costly string comparison. In order to get rid of the property cache in QML, we need faster way to obtain the metatype of method parameters and return types. This also has some benefits outside of QML, as it should speed up non-pointer-to-member function based connects, as well as QMetaMethod::invoke. The first solution: Thanks to Olivier's work, we already have a way to create the QMetaType [1] for properties at compile time. The straightforward solution was to extend this work to also cover the QMetaMethod use case, which is what the patch at https://codereview.qt-project.org/c/qt/qtbase/+/294774 does. The catch: Unfortunately, in order to create the necessary information, we use templates. And those templates require the type for which we create the QMetaType to be complete (and if the type is T& or T*, T must be complete). While this mostly works for properties (as those often are backed by a member), for methods those types are often only forward declared. Getting qtbase to work with this was possible, but required larger changes (https://codereview.qt-project.org/c/qt/qtbase/+/297108, in addition to the fixes already in the initial patch). In many cases, that meant using Q_MOC_INCLUDE to tell the moc to include the necessary header.[2] More annoyingly, if a type is forward declared in a header and only implemented in a cpp file, the only way to get this to work is to include the moc file in the same cpp file. While doing this inside of Qt is possible, this is not something we can subject our users to. The proposed solution: In order to not break all the existing code, I would propose the following: We generate the QMetaTypes at compile time if possible. If the type is however incomplete, we store QMetaType::Unknown, and, at runtime, fall back to the slower string check to get the actual metatype. This is implemented in https://codereview.qt-project.org/c/qt/qtbase/+/298231. For classes exported to QML, we would then either generate a warning or an error if they lack complete types. We might also want a build system option to enforce having complete types. I would appreciate any suggestions, questions or complaints about the proposed plan. [1] Strictly speaking, the QMetaTypeInterface, which QMetaType then wraps. [2] Using a regular include works too, most of the time. It can however negatively affect compile times and breaks completely in the case of circular dependencies. -- Fabian Kosmale Software Engineer The Qt Company GmbH Erich-Thilo-Str. 10 D-12489 Berlin fabian.kosm...@qt.io +49 1638686070 http://qt.io Geschäftsführer: Mika Pälsi, Juha Varelius, Mika Harjuaho Sitz der Gesellschaft: Berlin, Registergericht: Amtsgericht Charlottenburg, HRB 144331 B -- ___ Development mailing list Development@qt-project.org https://lists.qt-project.org/listinfo/development
