On 13-08-21 05:53 PM, Bill Myers wrote: > Have you considered the following "non-specific" quick fixes? > > 1. Build on a ramfs/ramdisk
Not I/O bound. > 2. Distribute compilations and tests across a cluster of machines (like > distcc) Already doing what we can here. The remainder doesn't parallelize without a huge amount of compiler rearchitecture. > 3. If non-parallelizable code is still the bottleneck, use the fastest > CPU possible (i.e. an overclocked Core i7 4770K, overclocked Core i7 > 4960X/3960X/4930K/3930K, dual Xeon E5 2687W or quad Xeon E5 4650 > depending on whether you need 4, 6, 16 or 32 cores) They're all quad xeons and i7s. We could push a little ways up this slope but it gets expensive and the returns are not great. > 4. Read metadata only once, in one of these ways: > 4a. Pass all files to a single compiler invocation (per machine or core) That's check-fast. Some tests don't like that, so we only do it on windows. We could do it on more machines but we don't want to lose coverage for the tests that don't work for check-fast. Someone's suggested doing check-fast everywhere + manually running the xfail-fast ones on the side. That might work, but it won't get us below 45min on a single bootstrap (because it's 30min to build straight-line) and 1h on an -all bootstrap. > 4b. Have a long-lived rustc "daemon" (per machine or core) that keeps > crate metadata in memory and gets passed files to compile by fd That would be a huge amount of rearchitecture. > 4c. Use CRIU suspend/restore (or the unexec from Emacs or whatever) to > suspend a rustc process after metadata is read and restore that image > for each file instead of spawning a new one The metadata is read in response to the source file being processed. By the time we could do this, we're committed to compiling a particular source file. Again, major work to redesign for this not to be true. > 4d. Allocate metadata using a special allocator that allocates it from a > block at a fixed memory address, then just dump the block into a file, > and read metadata with a single mmap system call at that same fixed > address (this is a security hole in general, so it needs to be optional > and off by default) Again, large amount of work to rearrange everything that uses metadata to work with this sort of thing. People keep looking for non-bug "quick fixes" while ignoring the bugs that are the source of the speed issues. Fixing the bugs _is_ the quick fix. It's not super quick, but it'll be faster than any other ideas we've got (except possibly sharding the testsuite, I'm going to wire that into buildbot when I get a free moment). -Graydon _______________________________________________ Rust-dev mailing list [email protected] https://mail.mozilla.org/listinfo/rust-dev
