Progress - Re: How to increase maximum user cpu usage allowed on a multi core system?

2015-03-21 Thread stan
On Mon, 9 Mar 2015 12:03:39 -0700 stan stanl-fedorau...@vfemail.net wrote: But, when I run a compile job with -j6, in order to allow all six cores to be used, it limits the total amount of usage to 100% of a *single* core. Booting from a Knoppix live DVD works. All six cores are utilized

Re: How to increase maximum user cpu usage allowed on a multi core system?

2015-03-19 Thread stan
On Wed, 18 Mar 2015 19:03:45 -0400 Tom H tomh0...@gmail.com wrote: You can also run (after doing the config and patching): CONCURRENCY_LEVEL=$(getconf _NPROCESSORS_ONLN) INSTALL_MOD_STRIP=1 make rpm-pkg sudo yum install ~/rpmbuild/RPMS/x86_64/kernel-4.0.0-rc4-1.x86_64.rpm Thanks. I'll try

Re: How to increase maximum user cpu usage allowed on a multi core system?

2015-03-19 Thread stan
On Thu, 19 Mar 2015 15:56:34 +0100 Heinz Diehl htd...@fritha.org wrote: That's really complicated, baah :-) It seems complicated, but with the screen template, it's almost habit. I don't even have to think about it. I tried the bfs patch on rc4 of the 4.0 kernel, but it got these errors:

Re: How to increase maximum user cpu usage allowed on a multi core system?

2015-03-19 Thread Heinz Diehl
On 18.03.2015, stan wrote: How do you remove the old kernels so that they don't pile up indefinitely? Manually? Yes. Just delete the related files in /boot and the sourcetree in /usr/src. Or does this automatically replace the last version that was compiled and installed this way? No.

Re: How to increase maximum user cpu usage allowed on a multi core system?

2015-03-19 Thread Heinz Diehl
On 18.03.2015, Tom H wrote: You don't need to run make as root. Yes, as I already stated. Only the install of the modules and the kernel itself has to be done as root. -- users mailing list users@lists.fedoraproject.org To unsubscribe or change subscription options:

Re: How to increase maximum user cpu usage allowed on a multi core system?

2015-03-18 Thread stan
On Tue, 17 Mar 2015 16:53:33 -0400 Tom H tomh0...@gmail.com wrote: On Tue, Mar 17, 2015 at 1:58 PM, Heinz Diehl htd...@fritha.org wrote: CFS is not required at all, and so are cgroups. Any kernel with the BFS patch applied will run just fine on Fedora. In fact, most of the time I run a

Re: How to increase maximum user cpu usage allowed on a multi core system?

2015-03-18 Thread stan
On Tue, 17 Mar 2015 18:52:23 +0100 Heinz Diehl htd...@fritha.org wrote: Lucky? The machine was fully unusable. Yes, but you have full control of your machine. Yes, this machine is on F21. [htd@chiara ~]$ uname -a Linux chiara.fritha.org 3.19.2-rc1-bfq #1 SMP PREEMPT Mon Mar 16 16:16:07

Re: How to increase maximum user cpu usage allowed on a multi core system?

2015-03-18 Thread stan
On Tue, 17 Mar 2015 18:52:23 +0100 Heinz Diehl htd...@fritha.org wrote: Directly from source. Would you be willing to give a recipe that you use? i.e. what steps do you perform to do this? There is a vanilla source tree included in the fedora src.rpm for the kernel, so maybe I could use

Re: How to increase maximum user cpu usage allowed on a multi core system?

2015-03-18 Thread stan
On Tue, 17 Mar 2015 19:34:08 +0100 poma pomidorabelis...@gmail.com wrote: Borislav, can you help explain the man why this is happening with his Piledriver. http://www.cpu-world.com/CPUs/Bulldozer/AMD-FX-Series%20FX-6300.html poma Thanks, poma. -- users mailing list

Re: How to increase maximum user cpu usage allowed on a multi core system?

2015-03-18 Thread stan
On Tue, 17 Mar 2015 18:52:23 +0100 Heinz Diehl htd...@fritha.org wrote: This is the top output when compiling a kernel with -j8 (4 cores/8 threads): top - 18:47:16 up 9:07, 4 users, load average: 1.77, 0.39, 0.13 Tasks: 263 total, 10 running, 253 sleeping, 0 stopped, 0 zombie %Cpu0

Re: How to increase maximum user cpu usage allowed on a multi core system?

2015-03-18 Thread stan
On Wed, 18 Mar 2015 08:01:35 -0700 stan stanl-fedorau...@vfemail.net wrote: Heinz, what does cat /proc/cgroups show? Should have included this in my response: $ cat /proc/cgroups #subsys_namehierarchy num_cgroups enabled cpuset 2 1 1 cpu 3 1 1

Re: How to increase maximum user cpu usage allowed on a multi core system?

2015-03-18 Thread Heinz Diehl
On 18.03.2015, stan wrote: Heinz, what does cat /proc/cgroups show? [htd@chiara ~]$ cat /proc/cgroups #subsys_namehierarchy num_cgroups enabled cpuset 2 1 1 memory 3 1 1 devices 4 74 1 freezer 5 1 1 net_cls 6 1 1 blkio

Re: How to increase maximum user cpu usage allowed on a multi core system?

2015-03-18 Thread Heinz Diehl
On 18.03.2015, stan wrote: An afterthought. I notice that you are compiling the kernel as root. I do my build in the rpmbuild system as a user, so the compile is run as a user. Do you think that would matter? For the kernel to get properly installed, you have to be root. Precisely, there

Re: How to increase maximum user cpu usage allowed on a multi core system?

2015-03-18 Thread Heinz Diehl
On 18.03.2015, stan wrote: Would you be willing to give a recipe that you use? i.e. what steps do you perform to do this? 1. Download a kernel tarball from kernel.org 2. Unpack it into /usr/src 3. Copy .config from the latest Fedora kernel into the kernel toplevel sourcedir (it is stored

Re: How to increase maximum user cpu usage allowed on a multi core system?

2015-03-18 Thread Tom H
On Wed, Mar 18, 2015 at 3:34 PM, stan stanl-fedorau...@vfemail.net wrote: Here's my Fedora rpmbuild procedure: I go to koji, the fedora central build repository for package maintainers, and download the src.rpm. http://koji.fedoraproject.org/koji/packageinfo?packageID=8 I use rpm to

Re: How to increase maximum user cpu usage allowed on a multi core system?

2015-03-18 Thread Tom H
On Wed, Mar 18, 2015 at 1:26 PM, Heinz Diehl htd...@fritha.org wrote: On 17.03.2015, Tom H wrote: I don't know what the difference is between not compiling cgroup suppport into the kernel and compiling it in but disabling all controllers but it looks like that your assumption that cgroup

Re: How to increase maximum user cpu usage allowed on a multi core system?

2015-03-18 Thread Tom H
On Wed, Mar 18, 2015 at 11:01 AM, stan stanl-fedorau...@vfemail.net wrote: On Tue, 17 Mar 2015 16:53:33 -0400, Tom H tomh0...@gmail.com wrote: CONFIG_CGROUPS (it is OK to disable all controllers) is listed under REQUIREMENTS in the systemd README. I don't know what the difference is between

Re: How to increase maximum user cpu usage allowed on a multi core system?

2015-03-18 Thread Tom H
On Wed, Mar 18, 2015 at 1:22 PM, Heinz Diehl htd...@fritha.org wrote: On 18.03.2015, stan wrote: An afterthought. I notice that you are compiling the kernel as root. I do my build in the rpmbuild system as a user, so the compile is run as a user. Do you think that would matter? For the

Re: How to increase maximum user cpu usage allowed on a multi core system?

2015-03-18 Thread stan
On Wed, 18 Mar 2015 18:19:54 +0100 Heinz Diehl htd...@fritha.org wrote: 1. Download a kernel tarball from kernel.org 2. Unpack it into /usr/src 3. Copy .config from the latest Fedora kernel into the kernel toplevel sourcedir (it is stored in /boot). 4. make oldconfig 5. make 6. make

Re: How to increase maximum user cpu usage allowed on a multi core system?

2015-03-18 Thread Heinz Diehl
On 17.03.2015, Tom H wrote: I don't know what the difference is between not compiling cgroup suppport into the kernel and compiling it in but disabling all controllers but it looks like that your assumption that cgroup support isn't required is wrong. Thanks for pinting this out! You are

Re: How to increase maximum user cpu usage allowed on a multi core system?

2015-03-18 Thread stan
On Wed, 18 Mar 2015 18:19:54 +0100 Heinz Diehl htd...@fritha.org wrote: 1. Download a kernel tarball from kernel.org 2. Unpack it into /usr/src 3. Copy .config from the latest Fedora kernel into the kernel toplevel sourcedir (it is stored in /boot). 4. make oldconfig 5. make 6. make

Re: How to increase maximum user cpu usage allowed on a multi core system?

2015-03-17 Thread Heinz Diehl
On 17.03.2015, stan wrote: Lucky you! Lucky? The machine was fully unusable. Are you using F21? Which kernel? Yes, this machine is on F21. [htd@chiara ~]$ uname -a Linux chiara.fritha.org 3.19.2-rc1-bfq #1 SMP PREEMPT Mon Mar 16 16:16:07 CET 2015 x86_64 x86_64 x86_64 GNU/Linux It's a

Re: How to increase maximum user cpu usage allowed on a multi core system?

2015-03-17 Thread Tom H
On Tue, Mar 17, 2015 at 1:58 PM, Heinz Diehl htd...@fritha.org wrote: On 17.03.2015, stan wrote: I think cgroups are integrated into Fedora, and so the CFS with cgroup is required. Thus, BFS will probably not work in Fedora, as it has no support for cgroups. CFS is not required at all, and

Re: How to increase maximum user cpu usage allowed on a multi core system?

2015-03-17 Thread Heinz Diehl
On 17.03.2015, stan wrote: I think cgroups are integrated into Fedora, and so the CFS with cgroup is required. Thus, BFS will probably not work in Fedora, as it has no support for cgroups. CFS is not required at all, and so are cgroups. Any kernel with the BFS patch applied will run just

Re: How to increase maximum user cpu usage allowed on a multi core system?

2015-03-17 Thread stan
On Wed, 11 Mar 2015 20:56:37 +0100 Heinz Diehl htd...@fritha.org wrote: http://ck.kolivas.org/patches/bfs/3.0/3.18/3.18-sched-bfs-460.patch (BFS is designed with latency in mind, not throughput). By significant workaround and patching in kernel/sched, I was able to compile a kernel without

Re: How to increase maximum user cpu usage allowed on a multi core system?

2015-03-17 Thread stan
On Mon, 16 Mar 2015 16:05:37 +0100 Heinz Diehl htd...@fritha.org wrote: On 09.03.2015, Martin Cigorraga wrote: Just a minor clarification: when compiling, the -j flag should point to a unit above your available cores in order to fully utilize all of them. Curious what would happen, I

Re: How to increase maximum user cpu usage allowed on a multi core system?

2015-03-16 Thread Heinz Diehl
On 09.03.2015, Martin Cigorraga wrote: Just a minor clarification: when compiling, the -j flag should point to a unit above your available cores in order to fully utilize all of them. Curious what would happen, I remembered this mail when compiling a new kernel today. A nice -n 19 make -j

Re: How to increase maximum user cpu usage allowed on a multi core system?

2015-03-13 Thread stan
On Thu, 12 Mar 2015 23:20:27 +0100 Heinz Diehl htd...@fritha.org wrote: It depends on the machine used and the amount of processes. While cgroups limit more than just CPU power, you could try with BFS (which does not use cgroups). http://ck.kolivas.org/patches/bfs/3.0/3.19/ Thanks for

Re: How to increase maximum user cpu usage allowed on a multi core system?

2015-03-13 Thread poma
On 12.03.2015 20:40, stan wrote: On Wed, 11 Mar 2015 18:21:06 +0100 poma pomidorabelis...@gmail.com wrote: Both: 1. diff -u /boot/config-3.18.8-201.fc21.x86_64 /boot/config-3.19.0-1.20150211.fc21.x86_64 2. dmidecode (as root) to http://fpaste.org s'il vous plaît. 1 week

Re: How to increase maximum user cpu usage allowed on a multi core system?

2015-03-12 Thread stan
On Wed, 11 Mar 2015 16:36:43 + Ian Malone ibmal...@gmail.com wrote: Since I can't reproduce this problem I'm not sure what's causing it. If you really are finding make subprocesses limited to 100% cpu across the lot then maybe have a look to see if there are any cgroups limits active

Re: How to increase maximum user cpu usage allowed on a multi core system?

2015-03-12 Thread stan
On Wed, 11 Mar 2015 18:21:06 +0100 poma pomidorabelis...@gmail.com wrote: The Devil is in the detail. Both: 1. diff -u /boot/config-3.18.8-201.fc21.x86_64 /boot/config-3.19.0-1.20150211.fc21.x86_64 2. dmidecode (as root) to http://fpaste.org s'il vous plaît. You seem to be the

Re: How to increase maximum user cpu usage allowed on a multi core system?

2015-03-12 Thread Kevin Cummings
On 03/12/2015 03:40 PM, stan wrote: Are you sure you don't want to know what I had for breakfast today? :-D No! That's what Facebook is for! B^) -- Kevin J. Cummings kjch...@verizon.net cummi...@kjchome.homeip.net cummi...@kjc386.framingham.ma.us Registered Linux User #1232

Re: How to increase maximum user cpu usage allowed on a multi core system?

2015-03-12 Thread Rick Stevens
On 03/12/2015 01:54 PM, Kevin Cummings wrote: On 03/12/2015 03:40 PM, stan wrote: Are you sure you don't want to know what I had for breakfast today? :-D No! That's what Facebook is for! B^) Article tagline from the Orange County Register this morning: A study co-authored by a

Re: How to increase maximum user cpu usage allowed on a multi core system?

2015-03-12 Thread Heinz Diehl
On 12.03.2015, stan wrote: When I build firefox nightly with -j6, just at the end of the export phase, and before the compile starts, I see all 6 cores maxed out. Once the compile starts, it is back to a single core equivalent. The Gentoo users seemed to suggest that this was a flaw in the

Re: How to increase maximum user cpu usage allowed on a multi core system?

2015-03-12 Thread stan
On Wed, 11 Mar 2015 18:21:06 +0100 poma pomidorabelis...@gmail.com wrote: Both: 1. diff -u /boot/config-3.18.8-201.fc21.x86_64 /boot/config-3.19.0-1.20150211.fc21.x86_64 2. dmidecode (as root) to http://fpaste.org s'il vous plaît. 1 week http://fpaste.org/197315/88936142/ The config

Re: How to increase maximum user cpu usage allowed on a multi core system?

2015-03-12 Thread stan
On Wed, 11 Mar 2015 15:14:39 -0400 Kevin Cummings cummi...@kjchome.homeip.net wrote: We're thinking in terms of one machine with multiple cores here. What about an environment with multiple machines (each possibly with multiple cores). Now you have *many* more possibilities of where to run

Re: How to increase maximum user cpu usage allowed on a multi core system?

2015-03-12 Thread stan
On Tue, 10 Mar 2015 18:01:07 -0400 (EDT) ergodic g...@embarqmail.com wrote: Frankly I never check the loading, just use -j with no argument, but I always do other processes in parallel with no problem. Then I think you must be having the same behavior as me. Because, as Ian found, if a

Re: How to increase maximum user cpu usage allowed on a multi core system?

2015-03-12 Thread stan
On Wed, 11 Mar 2015 16:36:43 + Ian Malone ibmal...@gmail.com wrote: You may want to check the .NOTPARALLEL directive is not present http://www.gnu.org/software/make/manual/make.html#Parallel though I think that would simply prevent multiple processes. This sounded like exactly the

Re: How to increase maximum user cpu usage allowed on a multi core system?

2015-03-12 Thread stan
On Wed, 11 Mar 2015 20:56:37 +0100 Heinz Diehl htd...@fritha.org wrote: Haven't looked into this for some time, but take a look into /usr/src/linux/kernel/sched/fair.c. (The CFS code is complex and difficult to understand, though - at least for me). Took a quick look at this. Only ~8000

Re: How to increase maximum user cpu usage allowed on a multi core system?

2015-03-12 Thread Heinz Diehl
On 12.03.2015, stan wrote: So, cgroups seem like another dead end. It depends on the machine used and the amount of processes. While cgroups limit more than just CPU power, you could try with BFS (which does not use cgroups). http://ck.kolivas.org/patches/bfs/3.0/3.19/ -- users mailing list

Re: How to increase maximum user cpu usage allowed on a multi core system?

2015-03-11 Thread stan
On Wed, 11 Mar 2015 13:32:09 +0100 poma pomidorabelis...@gmail.com wrote: 1. You still haven't provided basic information about the processor, thus 'lscpu' or a similar command. $ lscpu Architecture: x86_64 CPU op-mode(s):32-bit, 64-bit Byte Order:Little Endian

Re: How to increase maximum user cpu usage allowed on a multi core system?

2015-03-11 Thread stan
On Wed, 11 Mar 2015 11:34:25 +0100 Heinz Diehl htd...@fritha.org wrote: I just tried a simple make on an 8-core machine. There was exactly one compile process, and it's 100% load was distributed over 3 cores. So nothing wrong with that one. If you run 100% on one core or 100% distributed

Re: How to increase maximum user cpu usage allowed on a multi core system?

2015-03-11 Thread Ian Malone
On 9 March 2015 at 19:03, stan stanl-fedorau...@vfemail.net wrote: I'm running Fedora 21 with a custom compiled kernel, 3.19.0-1.20150211.fc21.x86_64. I have a multi core system with 6 cores. All are recognized by the kernel. But, when I run a compile job with -j6, in order to allow all

Re: How to increase maximum user cpu usage allowed on a multi core system?

2015-03-11 Thread Heinz Diehl
On 11.03.2015, stan wrote: What is the point of -j6 or -j8 if the make can't spawn additional processes with their own limits, and thus take advantage of more resources that are available? The point is simply that you can exactly determine how many processes should be used. What is it that

Re: How to increase maximum user cpu usage allowed on a multi core system?

2015-03-11 Thread Kevin Cummings
On 03/11/2015 11:49 AM, stan wrote: On Wed, 11 Mar 2015 11:34:25 +0100 Heinz Diehl htd...@fritha.org wrote: It's the limiting to one process which causes what you observe. 1 process can not get more resources that 100%. The CPU scheduler handles how they are distributed. I think this is

Re: How to increase maximum user cpu usage allowed on a multi core system?

2015-03-11 Thread Heinz Diehl
On 11.03.2015, stan wrote: I don't see why this is necessary. The system is showing 470% idle. So the kernel cpu scheduler shouldn't need to limit the job to a single core maximum usage. I just tried a simple make on an 8-core machine. There was exactly one compile process, and it's 100%

Re: How to increase maximum user cpu usage allowed on a multi core system?

2015-03-11 Thread poma
On 11.03.2015 16:32, stan wrote: On Wed, 11 Mar 2015 13:32:09 +0100 poma pomidorabelis...@gmail.com wrote: 1. You still haven't provided basic information about the processor, thus 'lscpu' or a similar command. $ lscpu Architecture: x86_64 CPU op-mode(s):32-bit, 64-bit

Re: How to increase maximum user cpu usage allowed on a multi core system?

2015-03-10 Thread Heinz Diehl
On 09.03.2015, stan wrote: But, when I run a compile job with -j6, in order to allow all six cores to be used, it limits the total amount of usage to 100% of a *single* core. So, it might use all six cores, but the sum of the percentages on those six cores is always around 100% of one core.

Re: How to increase maximum user cpu usage allowed on a multi core system?

2015-03-10 Thread ergodic
Wouldn't be better to use -j with no argument? The make manual states: If the -j option is given without an argument, make will not limit the number of jobs that can run simultaneously. - Original Message - On 09.03.2015, stan wrote: But, when I run a compile job with -j6, in

Re: How to increase maximum user cpu usage allowed on a multi core system?

2015-03-10 Thread Heinz Diehl
On 10.03.2015, ergodic wrote: Wouldn't be better to use -j with no argument? It's a matter of taste. I compile my kernels with a nice value of 19 (lowest priority), because mostly I have to do other work while compiling a new kernel. I've never tried -j. How many processes does it open on you

Re: How to increase maximum user cpu usage allowed on a multi core system?

2015-03-10 Thread stan
On Tue, 10 Mar 2015 11:35:24 +0100 Heinz Diehl htd...@fritha.org wrote: On 09.03.2015, stan wrote: But, when I run a compile job with -j6, in order to allow all six cores to be used, it limits the total amount of usage to 100% of a *single* core. So, it might use all six cores, but the

Re: How to increase maximum user cpu usage allowed on a multi core system?

2015-03-10 Thread ergodic
Frankly I never check the loading, just use -j with no argument, but I always do other processes in parallel with no problem. - Original Message - On 10.03.2015, ergodic wrote: Wouldn't be better to use -j with no argument? It's a matter of taste. I compile my kernels with a nice

Re: How to increase maximum user cpu usage allowed on a multi core system?

2015-03-09 Thread Martin Cigorraga
Ah... couldn't tell, but the Gentoo wiki and Arch's one (in a lesser extent), are excellent resources to learn everything about that! HTH On Mon, Mar 9, 2015 at 5:36 PM, stan stanl-fedorau...@vfemail.net wrote: On Mon, 9 Mar 2015 17:30:57 -0300 Martin Cigorraga martincigorr...@gmail.com wrote:

Re: How to increase maximum user cpu usage allowed on a multi core system?

2015-03-09 Thread stan
On Mon, 9 Mar 2015 12:03:39 -0700 stan stanl-fedorau...@vfemail.net wrote: This is from htop output. Correction. *atop* output. -- users mailing list users@lists.fedoraproject.org To unsubscribe or change subscription options: https://admin.fedoraproject.org/mailman/listinfo/users Fedora Code

Re: How to increase maximum user cpu usage allowed on a multi core system?

2015-03-09 Thread Martin Cigorraga
Hello Stan, Just a minor clarification: when compiling, the -j flag should point to a unit above your available cores in order to fully utilize all of them. I'm sure you already know this, but nevertheless beware when doing intensive compilation and using all your cores as you might end with a

Re: How to increase maximum user cpu usage allowed on a multi core system?

2015-03-09 Thread stan
On Mon, 9 Mar 2015 17:30:57 -0300 Martin Cigorraga martincigorr...@gmail.com wrote: Hello Stan, Just a minor clarification: when compiling, the -j flag should point to a unit above your available cores in order to fully utilize all of them. I'm sure you already know this, but nevertheless