Fantastic information set. Many thanks for all this digging.
I suspect, that our build infrastructure work may help, in that if we get rid
of the nested makes,
then I can only assume we will be doing fewer stat() calls, but I think we
still have a problem.
A year or so ago, I managed to reduce the fork/exec count by 50% thinking it
would improve windows
build times, it did very little, I was disappointed, and assumed the bigger
spike had to be related to I/O.
So I tried a fast SSD on a Windows machine, which sure made it quiet, but also
didn't help much.
So I'm with you on the stat() theory, makes a great deal of sense.
-kto
On Feb 14, 2012, at 6:59 AM, Volker Simonis wrote:
> On Tue, Feb 14, 2012 at 12:43 PM, Fredrik Öhrström
> <fredrik.ohrst...@oracle.com> wrote:
>> 2012-02-14 12:29, Volker Simonis skrev:
>>> To cut a long story short:
>>> - disabling "on access" scanning of *.{java,c,cpp,h,hpp} seems to
>>> resolve the file io problems (permission denied, access denied)
>>> - disabling ASLR seems to resolve the "fork" problems
>>
>> Great work! Do you know if disabling ASLR affects the fork performance
>> on 64 bit windows?
>>
>
> No, definitely not.
> The build time on my DualCore i7 @ 2.7 GHz notebook with 8GB Ram is
> constantly 1h43min for a full JDK8 opt build.
>
> Now that that the build seems stable I want to compare it with a Linux
> build in a VirtualBox VM on the same host.
> I'll let you know the results once I'm done, but I don't think that
> "fork" is the big problem - at least not the only one.
>
> But once you asked here's my main suspect since long time (you asked
> so you have to read the conspiracy:)
>
> stat (or lstat/fstat/stat64 ...)
> ---------------------------------
>
> I'm quite sure that the poor implementation of stat in Cygwin and its
> usage in make is one of the main reasons
> why the build Windows build is so terrible slow in Windows compared to Linux.
>
> I first realized this years ago when I was first building Windows
> inside a VMWare image with the sources on
> network shares. The build times were excessively long and it was not
> the compile times because SMB shares
> do a quite efficient caching - it was the times which make needed in
> order to check the build dependencies.
> The build times could be improved by factors if the sources were moved
> to a local disc.
>
> But even locallly, I think that the stat calls for large dependency
> lists as we have them in the OpenJDK build
> may still be one of the causes for the slow build. Running make inside
> strace will reveal the shear number
> of these calls and there is evidence that the stat performance is
> really poor in Cygwin.
>
> I'll attach some unsorted links to some of the resources which discuss
> this issue just in case somebody wants to
> deep dive into this topic. I'm definitely not a Windows/Cygwin expert
> and I can't promise I'll have the time
> to solve it 'real soon' but I'll definitely stick to this topic
> because it is a real PITA (especially if you want to
> change your build from MKS to Cygwin the developers who will first
> love you because freed them from of
> the MKS licensing night mare will very soon kill you because of the
> build times:)
>
> "performance issue with cgywin make"
> http://www.mail-archive.com/make-w32@gnu.org/msg01353.html
>
> - mailing list thread which compares the bad performance of Cygwin
> make with MS nmake and discusses
> how some of the problems have been solved in CMake by using direct
> windows sys-calls instead of stat.
>
> "make 3.82 performing more stat() calls than 3.81"
> http://lists.gnu.org/archive/html/bug-make/2011-09/msg00025.html
>
> - mailing list thread discussing a bug in make 3.82 which leads to
> even more calls to stat
> (and has been fixed already in the head revision of make)
>
>
> "Cygwin Performance and stat()"
> http://omgili.com/mailinglist/cygwin/cygwin/com/efe8a37b2e4466daa7b6eb1aa610c3d7squirrelwwwwebmailwingertorg.html
>
> - very long thrad, very ‘flamy’, no outcome, some interesting infos
> nevertheless
>
> "cygwin: Use native Win32 API for stat"
> http://marc.info/?l=git&m=122278284210941
>
> - partial patch for the problem by implementing a stripped down
> version of stat which does just enough...
>
> "_NutFastStat(), _NutFastStat64()"
> http://www.mkssoftware.com/docs/man3/_NutFastStat.3.asp
>
> - MKSs limited but faster version of stat (not sure if MKS make uses
> this, but that may be an
> explanation why builds under MKS are reported to be considerably
> faster compared to Cygwin builds
>
> "Managing Projects with GNU Make, 3rd Edition, The Power of GNU make
> for Building Anything, By Robert Mecklenburg"
> http://shop.oreilly.com/product/9780596006105.do
>
> - Chapter 10: "Improving the Performance of make" freely available as
> PDF download:
> http://reilly.com/catalog/make3/book/ch10.pdf
>
>
> "Compile time Local Cygwin vs. VMware session on same system"
> http://cygwin.com/ml/cygwin/2008-10/threads.html#00415
>
> - mail thread suggesting the usage of ‘dash’ instead of ‘bash’ for a
>> 10% performance improvement
>
>> //Fredrik
>>
