Hi, On Mon, Dec 23, 2013 at 02:30:33PM +0100, Christian Lohmaier wrote: > Just compare > > mkdir foo ; cd foo > time touch {1..1000} > > with > > mkdir foo; cd foo > time for i in {1..1000}; do touch $i ; done > > While on linux there's no real difference, on windows this is more > than an order of magnitude...(and the build system does the equivalent > of the latter a lot)
Yes, AFAIK currently there are two main contributions to the build on windows being slower: - the MSVC compiler being slower itself - process creation on windows (or cygwin or shell in cygwin) being slow (see Clophs example about) There isnt so much we can do about the first thing in a clean way. We can do something about the second thing, e.g.: http://cgit.freedesktop.org/libreoffice/core/commit/?id=66a0713dc9c676182fcd7aa1e21f8dc25c05be5e improved Windows build time by 10%. Some other things to consider doing for Windows optimization: - IIRC MSVC can compile multiple object files with one call. Obviously all the compile flags etc. must be the same then for all objects to create, but for cxx files from the same dir and for the same library this is given. So one could e.g. always batch up 8 of such objects and compile them with one call. But I fear this is adding a huge load of complexity to the build, because: - it IIRC does not generate per object dependency files then - in incremental builds, you would only need to rebuild _some_ files -- so you cant do the grouping of objects in some simple or static way (as you would 'overrebuild' then) - it also collides with parallelization in the build sytem -- if you put e.g. always 8 object in one call and only have to rebuild 8 objects (a common developer change), you would get no parallelization at all - as noted in the comment above the touching of a file in cygwin took some ~250ms -- interestingly, this is not CPU-bound. A simplistic approach thus would be, to just allways overcommit the build by some factor and see if it improves the build time (or the CPU load). Of course, the real issue is: what is this 250ms idle wait to touch a file from cygwin about? If we know that we can likely workaround it. Maybe its cygwin polling some network share for the cygwin->windows path mapping? So any detective work on that might help: - Can one make the delay go away by e.g. disabling network shares/virus scanners/other services? - Does this delay also appear when doing network-IO instead of file-IO - If so, could we e.g. start a (native-Windows) Python demon that listens on a network socket and hand it the commands to execute as be faster? - Or, does it make sense to make the gbuild in cygwin dump e.g. output for the ninja (the minimalistic build backend) and then have a (native windows) ninja process execute this. Finally, Michael Stahl is considering moving gbuild to work on a native windows GNU make without the cygwin wrapper. But that likely is quite a bit of work. So, you see there is a good bit to research (where does the 250 ms delay on cygwin come from), to evaluate (batch builds, handing over to a native demon, creating output for ninja, using native GNU make). TBH though, I fear most of the hacks above -- with the exception of running native GNU make -- make the build system even more complex and harder to maintain, so I am not convinced that its an good idea to push for them without very well-funded research covering all cases. Best, Bjoern _______________________________________________ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice