MCVE Extract the 2 attachments in the same directory. Execute milti-processor-examples shell script. Needs sudo access to pull machine profile. Change path to linux dictionary in script as needed.
Script is straight forward as it forks “grep ‘string’ dict” processes for all
words in word_list.
Actual output from my 2 HP systems are below. Note execution time differences
between later and older kernels. Runtimes are quicker on an older machine with
older kernel (process scheduling?). Note runtimes of xargs and for-loop are
much quicker!
Thank you for your time,
George Groom
Comparing runtime of serial and 3 parallel processing methods...parallel, xargs
and a for-loop
Run mode: cat
Kernel
3.10.0-693.17.1.el7.x86_64
Machine profile
HTT (Multi-threading)
Version: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz
Core Count: 22
Core Enabled: 22
Thread Count: 44
Multi-Core
Hardware Thread
HTT (Multi-threading)
Version: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz
Core Count: 22
Core Enabled: 22
Thread Count: 44
Multi-Core
Hardware Thread
Product Name: Synergy 480 Gen9
Product Name: Synergy 480 Gen9 Compute Module
String 2: Product ID: 732350-B21
serially... in this mode processing takes too long...skipping
parallel...
real 0m36.438s
user 2m40.102s
sys 0m29.745s
xargs...
real 0m28.718s
user 2m26.768s
sys 0m13.546s
for-loop...
real 0m29.858s
user 2m25.937s
sys 0m9.881s
Comparing runtime of serial and 3 parallel processing methods...parallel, xargs
and a for-loop
Run mode: cat
Kernel
2.6.18-238.el5
Machine profile
Product Name: ProLiant DL360 G7
HTT (Multi-threading)
Version: Intel(R) Xeon(R) CPU E5649 @ 2.53GHz
Core Count: 6
Core Enabled: 6
Thread Count: 12
HTT (Multi-threading)
Version: Intel(R) Xeon(R) CPU E5649 @ 2.53GHz
Core Count: 6
Core Enabled: 6
Thread Count: 12
String 2: Product ID: 579237-B21
serially... in this mode processing takes too long...skipping
parallel...
real 0m23.732s
user 1m33.362s
sys 0m34.111s
xargs...
real 0m4.950s
user 1m16.905s
sys 0m16.935s
for-loop...
real 0m3.667s
user 1m2.713s
sys 0m12.475s
-----Original Message-----
From: Ole Tange [mailto:[email protected]]
Sent: Sunday, October 21, 2018 12:53 PM
To: Groom, George
Cc: parallel
Subject: [EXTERNAL] Re: Linux kernel question, pre 2.6.23 vs later kernel
versions
On Wed, Oct 17, 2018 at 10:25 PM Groom, George <[email protected]> wrote:
> I understand that kernel process scheduling changed at 2.6.23… run queue
> based to Completely Fair Scheduling (CFS). Has anyone experienced longer
> parallel script execute times on systems with later versions of the Linux
> kernel?
>
> I’ve noticed longer execute times where I’m generating several thousands of
> processes on later kernel machines. CFS seems to throttle my processes
> within a few seconds and then control execution until script completion. Are
> there known workarounds for this behavior? Depending on application needs
> the older method might be preferable?
I cannot reproduce this. Make an MCVE.
/Ole
multi-processor-examples
Description: multi-processor-examples
word_list
Description: word_list
