Author: ken Date: Tue Oct 10 12:40:57 2017 New Revision: 19316 Log: Details of how to use only 4 cores when measuring a build on a many-core machine.
Modified: trunk/edguide/chapter06.xml Modified: trunk/edguide/chapter06.xml ============================================================================== --- trunk/edguide/chapter06.xml Tue Oct 10 10:51:25 2017 (r19315) +++ trunk/edguide/chapter06.xml Tue Oct 10 12:40:57 2017 (r19316) @@ -504,4 +504,99 @@ </sect1> + <sect1> + <title>Measurements when using multiple processors</title> + + <para>For most packages, timings using a single processor (<command>make + -j1</command>) are preferred. But many modern packages now take a very long + time to build. Most users building BLFS on a modern machine will have 4 cores, + so using <command>make -j4</command> is perfectly acceptable for timing these + packages. In one or two places <command>make -j8</command> has been used, do + not do that without good reason.</para> + + <para>Where a package uses <command>ninja</command> to build, it will by + default use N+2 jobs, so on a single core machine it will try to run 3 jobs at + a time (nasty - too much thrashing) and on a 4 core machine it will try to run + six. This can be adjusted by using a <literal>-j</literal> switch, e.g. + <command>ninja -j1</command> to only run one job. On a laptop, building for + long periods using all cores might lead to overheating.</para> + + <para>The rest of this section covers temporarily restricting the number of + available processors on machines with more than 4 cores.</para> + + <para>Now that packages in the book are starting to use + <command>rust</command>, measuring on many-core machines has got harder. Rust + will determine how many processors are available, and then attempt to use all + of them (in practice it will usually pull in cargo source packages and some + parts of the build might only use one processor while waiting on another file + compiling).</para> + + <para>For people who have more than 4 processors, the canonical way to limit + how many are available to a job is to use <command>tasksel</command>. The + <systemitem class="username">root</systemitem> user can alternatively take + processors offline, but that offers less flexibility for running other tasks at + the same time.</para> + + <para>Unless a process is constrained to a subset of the available CPUs, Linux + will rotate processes across them. This is to even wear on the cores and to + help with cooling. When limiting which processors are to be used, if a machine + uses multithreading (intel i3, i7, etc and in practice almost all modern AMD + processors) it is best to use the appropriate pairs of CPUs - so 0,1 2,3 etc + (on an 8-core i7, using logical cores 0,2,4,6 will run all physical cores at up + to 100%, with full production of heat). If you are using any approach to limit + the number of available cores you should use different cores from time to time + to spread the usage.</para> + + <para>The following example shows how to do a measured build using only 4 + cores on an intel i7 with 8 logical cores, running X. If for some reason you + wish to do this without X you will know which tty you intend to use, so no + need to identify a /dev/pts terminal.</para> + + <para>First decide which cores you wish to use for the measured build. In this + example, paired cores 2-3 and 4-5. If you have enough cores, you could also do + a different build on other cores - but if that build uses the same disks or + filesystems (including /tmp) they can interfere with each other's + measurements.</para> + + <para>Now start your preferred graphical term where you intend to build, and + run <command>top</command>.</para> + + <para>In a different term, use ps to find out where it is running (the space + in ' top' is to avoid false matches if qtwebengine has been invoked, one of + the many switches passed by <package>QupZilla</package> includes 'top') : + </para> + +<screen><userinput>ps aux | grep ' top'</userinput></screen> + + <para>This example returned:</para> + +<literallayout>ken 30391 0.1 0.0 125452 2868 pts/2 S+ 14:54 0:00 top +ken 30394 0.0 0.0 116112 2224 pts/1 S+ 14:57 0:00 grep top</literallayout> + + <para>So in this example the term used <literal>pts/2</literal>.</para> + + <para>I then quit <command>top</command> so that term was only running + <command>bash</command>. Then I looked for the PID of that bash:</para> + +<screen><userinput>ps aux | grep 'pts/2 '</userinput></screen> + +<literallayout>ken 30368 0.0 0.0 128152 3800 pts/2 Ss+ 14:54 0:00 bash +ken 30396 0.0 0.0 116112 2012 pts/1 S+ 14:58 0:00 grep pts/2 </literallayout> + + <para>In that example, bash was running as PID 30368. Now you know your term's + bash PID, and which cores you wish to restrict it to, you can invoke + <command>taskset</command>. To complete the example, I used:</para> + +<screen><userinput>taskset -pc 2-5 30368</userinput></screen> + + <para>Now you can return to the term where you will run the measured build. + Check that you now have only 4 cores available by entering + <command>nproc</command>.</para> + + <para>If you intend to run anything else which loads the CPUs while the + measured build is running, you can similarly assign that to other cores. + </para> + + </sect1> + </chapter> -- http://lists.linuxfromscratch.org/listinfo/blfs-book FAQ: http://www.linuxfromscratch.org/blfs/faq.html Unsubscribe: See the above information page
