Re: big Deps ...
On 10/12/2012 06:57 PM, Michael Meeks wrote: On Thu, 2012-06-21 at 13:47 +0200, Stephan Bergmann wrote: On 06/21/2012 10:57 AM, Michael Meeks wrote: Fair enough; but we could presumably easily tweak that such that a .hdl and .hpp would be updated in lock-step; after all if the .hpp is not changed - it necessarily includes the .hdl - so we didn't really gain anything from not touching the .hpp :-) Yeah, could probably be done. Turns out that gnumake does that anyway so I pushed this to git. We miss a dependency on a single .hdl file - but that one seems so unlikely to change at all that it's worth the win I think. #include com/sun/star/uno/TypeClass.hdl in cppu/inc/com/sun/star/uno/Type.h? (Caused by the broken design of having TypeClass as a UNOIDL entity instead of as a language-binding--specific entity like the C++ Type class.) I'm not really a fan of fast vs. correct. Stephan ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: big Deps ...
Hi there, Jack - FYI, I got bored and did this easy hack myself ;-) On Thu, 2012-06-21 at 13:47 +0200, Stephan Bergmann wrote: On 06/21/2012 10:57 AM, Michael Meeks wrote: Fair enough; but we could presumably easily tweak that such that a .hdl and .hpp would be updated in lock-step; after all if the .hpp is not changed - it necessarily includes the .hdl - so we didn't really gain anything from not touching the .hpp :-) Yeah, could probably be done. Turns out that gnumake does that anyway so I pushed this to git. We miss a dependency on a single .hdl file - but that one seems so unlikely to change at all that it's worth the win I think. I also elided all boost headers, turning them into a single header - for the case that boost is internal. These two ~halve the size of the dependencies we parse for 'sw' at least vs. a 3.6 tree I have to hand: $ ls -lh workdir/unxlngi6.pro/Dep/LinkTarget/Library/libswlo.so.d # 3.6 -rw-r--r-- 1 michael users 58M Oct 12 ... $ ls -lh workdir/unxlngi6.pro/Dep/LinkTarget/Library/libswlo.so.d # master -rw-r--r-- 1 michael users 25M Oct 12 ... Sadly that only gives a speedup (for a clean do-nothing build) of 'sw' of: 14.5 - 11seconds, which is less than I had hoped. Hopefully it doesn't break things for anyone; IIRC there were a few more simple ideas for eliding sillies in there, but I forget what they were. Presumably given the reasonably small size of the win - we've done something else that confuses / slows down make substantially [ perhaps doing something with that unit test in sw is it ? ]. HTH, Michael. -- michael.me...@suse.com , Pseudo Engineer, itinerant idiot ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: big Deps ...
On 21/06/12 17:05, Stephan Bergmann wrote: On 06/21/2012 04:01 PM, Michael Stahl wrote: On 21/06/12 13:47, Stephan Bergmann wrote: On 06/21/2012 10:57 AM, Michael Meeks wrote: [ indeed, I'm sure there is one - but what is the benefit of separating the .hdl files out, if we are sure that each .hpp will include all of it's dependent .hpp's ? ]. Circular dependencies among headers. i wonder: is that the case for all generated headers, or just for some small subset that is necessary for bootstrapping (i.e. probably css.uno/css.lang)? Its not about bootstrapping, its about scenarios like an interface X method argument of struct type S, where S in turn has a member of type X (IIRC). hmm.. interesting... so it looks to me like there are 3 possibilities for cyclic dependencies: between structs, exceptions and interfaces. i couldn't get a forward declaration of a struct or an exception in UNO IDL, so the problematic cases seem to be interface vs. struct and interface vs. exception. it looks like using struct as parameter or return value in interface can be done with forward declaration, so that cycle can be avoided. however the exceptions are listed in the C++ throw specifications, and that requires the type to be defined, a forward declaration is not sufficient. in turn, the exception constructors need to be able to initialize ReferenceXInterface members. surprisingly i could get the exception constructor definitions compiled with GCC with the interface only forward declared (and i admit i don't understand why this works); however i believe it's likely MSVC would be stricter here and require a definition of the type parameter of Reference to initialize the interface member. ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
big Deps ...
On Tue, 2012-06-05 at 11:13 +0200, Bjoern Michaelsen wrote: On Mon, Jun 04, 2012 at 11:35:44PM +0200, Bjoern Michaelsen wrote: will confirm -- CxxObjects is rather large ... Im beginning to like the idea of gzipping those. :-) Finished a master build: 1.2G workdir/unxlngx6.pro/Dep/ Modulo compressing the CxxObject deps, I'm still trying to prototype some more speedups for the concatenated / big-library dependencies with a few local patches to concat-deps.pl [ the hyper-optimised C version is not as easy to casually hack ]. Anyhow - doing some more analysis; I see: 4823735 - lines of LinkTarget/ deps 1598180 - lines containing /boost/ 1178594 - offapi 608585 - udkapi So it seems the next big low-hanging fruit after boost is the IDL compilation. Having got down to 3.2 million lines - having another 1.7million (over 50%) of the dependency lines being compiled IDL files is slightly amazing (at least to me). I wonder where they all come from and why. Anyhow - one obvious oddness reading the code is the .hdl and .hpp duplication; my question is: $ grep -R 'api/.*\.hpp' * | wc -l 892769 $ grep -R 'api/.*\.hdl' * | wc -l 899179 So a few questions: if a .idl file is changed is there any circumstance where the .hpp and .hdl will not both be updated in lock-step. Also - do all IDL generated .hpp deps ultimately include all the .hdl files as well ? ie. we could simply elide \.hdl$ from all dependency files ? - which might knock 25% off our deps at a very trivial stroke. Then again a small number of files include .hdl files directly; is that a bug ? :-) Thanks, Michael. -- michael.me...@suse.com , Pseudo Engineer, itinerant idiot ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: big Deps ...
On 06/20/2012 05:15 PM, Michael Meeks wrote: if a .idl file is changed is there any circumstance where the .hpp and .hdl will not both be updated in lock-step. IIRC, the logic of cppumaker is to first generate temporary .hpp and .hdl files and only copy them over existing ones in solver if they are actually different. That way, it should easily happen that a .hdl file is updated while the corresponding .hpp file is not. (For example, when a method is added to an interface and cppumaker is not running in comprehensive type information mode; or when a bug in cppumaker is fixed affecting .hdl but not .hpp output). Also - do all IDL generated .hpp deps ultimately include all the .hdl Each .hpp file includes the corresponding .hdl file (if any), yes. files as well ? ie. we could simply elide \.hdl$ from all dependency files ? - which might knock 25% off our deps at a very trivial stroke. That would be unsound (see above). Then again a small number of files include .hdl files directly; is that a bug ? :-) Looks like a bug, yes. Care to share the list? Stephan ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: big Deps ...
On Wed, 2012-06-20 at 18:28 +0200, Stephan Bergmann wrote: IIRC, the logic of cppumaker is to first generate temporary .hpp and .hdl files and only copy them over existing ones in solver if they are actually different. That way, it should easily happen that a .hdl file is updated while the corresponding .hpp file is not. (For example, when Fair enough; but we could presumably easily tweak that such that a .hdl and .hpp would be updated in lock-step; after all if the .hpp is not changed - it necessarily includes the .hdl - so we didn't really gain anything from not touching the .hpp :-) [ indeed, I'm sure there is one - but what is the benefit of separating the .hdl files out, if we are sure that each .hpp will include all of it's dependent .hpp's ? ]. a method is added to an interface and cppumaker is not running in comprehensive type information mode; or when a bug in cppumaker is fixed affecting .hdl but not .hpp output). Sure sure - but we could conceptually tweak cppumaker to do the diff on the pair and update both timestamps if one changes - and loose 25% of our dependency rules with a trivial regexp :-) Also - do all IDL generated .hpp deps ultimately include all the .hdl Each .hpp file includes the corresponding .hdl file (if any), yes. Great. Then again a small number of files include .hdl files directly; is that a bug ? :-) Looks like a bug, yes. Care to share the list? Nice; it's easier to generate than send: git grep 'include.*\.hdl' only 127 instances luckily. ATB, Michael. -- michael.me...@suse.com , Pseudo Engineer, itinerant idiot ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: big Deps ...
On 06/21/2012 10:57 AM, Michael Meeks wrote: On Wed, 2012-06-20 at 18:28 +0200, Stephan Bergmann wrote: IIRC, the logic of cppumaker is to first generate temporary .hpp and .hdl files and only copy them over existing ones in solver if they are actually different. That way, it should easily happen that a .hdl file is updated while the corresponding .hpp file is not. (For example, when Fair enough; but we could presumably easily tweak that such that a .hdl and .hpp would be updated in lock-step; after all if the .hpp is not changed - it necessarily includes the .hdl - so we didn't really gain anything from not touching the .hpp :-) Yeah, could probably be done. [ indeed, I'm sure there is one - but what is the benefit of separating the .hdl files out, if we are sure that each .hpp will include all of it's dependent .hpp's ? ]. Circular dependencies among headers. Then again a small number of files include .hdl files directly; is that a bug ? :-) Looks like a bug, yes. Care to share the list? Nice; it's easier to generate than send: git grep 'include.*\.hdl' only 127 instances luckily. Feel free to make that into an EasyHack. Stephan ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: big Deps ...
On 21/06/12 13:47, Stephan Bergmann wrote: On 06/21/2012 10:57 AM, Michael Meeks wrote: On Wed, 2012-06-20 at 18:28 +0200, Stephan Bergmann wrote: IIRC, the logic of cppumaker is to first generate temporary .hpp and .hdl files and only copy them over existing ones in solver if they are actually different. That way, it should easily happen that a .hdl file is updated while the corresponding .hpp file is not. (For example, when Fair enough; but we could presumably easily tweak that such that a .hdl and .hpp would be updated in lock-step; after all if the .hpp is not changed - it necessarily includes the .hdl - so we didn't really gain anything from not touching the .hpp :-) Yeah, could probably be done. that sounds easy enough. [ indeed, I'm sure there is one - but what is the benefit of separating the .hdl files out, if we are sure that each .hpp will include all of it's dependent .hpp's ? ]. Circular dependencies among headers. i wonder: is that the case for all generated headers, or just for some small subset that is necessary for bootstrapping (i.e. probably css.uno/css.lang)? if the latter, would it be possible to add some option or something to only generate 2 headers when really necessary? ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: big Deps ...
On 06/21/2012 04:01 PM, Michael Stahl wrote: On 21/06/12 13:47, Stephan Bergmann wrote: On 06/21/2012 10:57 AM, Michael Meeks wrote: [ indeed, I'm sure there is one - but what is the benefit of separating the .hdl files out, if we are sure that each .hpp will include all of it's dependent .hpp's ? ]. Circular dependencies among headers. i wonder: is that the case for all generated headers, or just for some small subset that is necessary for bootstrapping (i.e. probably css.uno/css.lang)? Its not about bootstrapping, its about scenarios like an interface X method argument of struct type S, where S in turn has a member of type X (IIRC). if the latter, would it be possible to add some option or something to only generate 2 headers when really necessary? Might or might not be possible to do that more or less easily; never spent much thought on it. Stephan ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: big Deps ...
On Thu, 2012-06-21 at 13:47 +0200, Stephan Bergmann wrote: [ indeed, I'm sure there is one - but what is the benefit of separating the .hdl files out, if we are sure that each .hpp will include all of it's dependent .hpp's ? ]. Circular dependencies among headers. Interesting; it'd of course be great to reduce that wherever possible; presumably for interfaces we could generate forward definitions of them before the header / includes to avoid generating and including two files where one would do. For structures clearly it's not possible to have two in-line structures inside each other ;-) so presumably it works well enough there already (?) what other cases are there ? Then again a small number of files include .hdl files directly; is that a bug ? :-) Looks like a bug, yes. Care to share the list? ... Feel free to make that into an EasyHack. I fixed them all with a simple script; it seems only one was actually required in cppu/inc/com/sun/star/uno/Type.h ATB, Michael. -- michael.me...@suse.com , Pseudo Engineer, itinerant idiot ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
