Re: High Load/Interrupts on Wheezy

[Will Platnick]

So since there seems to be a few of us having this issue, are there any
Debian or linux kernel engineers out there who are willing to help? Is this
the best place for that?

On Wed, Jul 3, 2013 at 3:50 PM, David Mckisick  wrote:

> Same issue here exactly and have noticed this since upgrading to Wheezy.
> We have also delayed upgrading the rest of our servers until this gets
> fixed.
>
>
> On Wed, Jul 3, 2013 at 10:45 AM, Scott Ferguson <
> scott.ferguson.debian.u...@gmail.com> wrote:
>
>> On 04/07/13 00:30, Will Platnick wrote:
>> > More troubleshooting steps:
>> >
>> > Built and installed latest 3.10 kernel, no change in interrupts
>> > Built and installed latest 2.6.32 kernel, and I am back to Squeeze level
>> > loads and no high timer, rescheduling, non-maskable or performance
>> > interrupts are present.
>> >
>> > So, does anybody have any idea what changed in the 3.2+ series that
>> > could cause this?
>>
>> Nope. But I've been experiencing the same thing so following your posts.
>> I've set up two identical LAMP servers hosting identical sites using
>> Virtualmin, one pure, standard (untweaked) Squeeze, the other pure,
>> standard (untweaked) Wheezy built from the same package list.  The
>> Squeeze one runs fine in 256MB of RAM, the Wheezy takes nearly three
>> times as long to boot, likewise to shutdown *even* when given 512MB of
>> RAM. Identical virtualmachine setups.
>>
>> I've logged the output of ps aux and will compare them tomorrow night -
>> if I find anything obvious I'll post them.
>>
>> Everything else is similar to your results.
>>
>> >
>> > On Wed, Jul 3, 2013 at 8:06 AM, Will Platnick > > > wrote:
>> >
>> > Something else I just noticed now that I'm on a screen high enough
>> > to show all of /proc/interrupts on one line:
>> >
>> 
>>
>> Kind regards
>>
>> --
>> Iceweasel/Firefox/Chrome/Chromium/Iceape/IE extensions for finding
>> answers to Debian questions:-
>>
>> https://addons.mozilla.org/en-US/firefox/collections/Scott_Ferguson/debian/
>>
>>
>

Re: High Load/Interrupts on Wheezy

[David Mckisick]

Same issue here exactly and have noticed this since upgrading to Wheezy. We
have also delayed upgrading the rest of our servers until this gets fixed.


On Wed, Jul 3, 2013 at 10:45 AM, Scott Ferguson <
scott.ferguson.debian.u...@gmail.com> wrote:

> On 04/07/13 00:30, Will Platnick wrote:
> > More troubleshooting steps:
> >
> > Built and installed latest 3.10 kernel, no change in interrupts
> > Built and installed latest 2.6.32 kernel, and I am back to Squeeze level
> > loads and no high timer, rescheduling, non-maskable or performance
> > interrupts are present.
> >
> > So, does anybody have any idea what changed in the 3.2+ series that
> > could cause this?
>
> Nope. But I've been experiencing the same thing so following your posts.
> I've set up two identical LAMP servers hosting identical sites using
> Virtualmin, one pure, standard (untweaked) Squeeze, the other pure,
> standard (untweaked) Wheezy built from the same package list.  The
> Squeeze one runs fine in 256MB of RAM, the Wheezy takes nearly three
> times as long to boot, likewise to shutdown *even* when given 512MB of
> RAM. Identical virtualmachine setups.
>
> I've logged the output of ps aux and will compare them tomorrow night -
> if I find anything obvious I'll post them.
>
> Everything else is similar to your results.
>
> >
> > On Wed, Jul 3, 2013 at 8:06 AM, Will Platnick  > > wrote:
> >
> > Something else I just noticed now that I'm on a screen high enough
> > to show all of /proc/interrupts on one line:
> >
> 
>
> Kind regards
>
> --
> Iceweasel/Firefox/Chrome/Chromium/Iceape/IE extensions for finding
> answers to Debian questions:-
> https://addons.mozilla.org/en-US/firefox/collections/Scott_Ferguson/debian/
>
>

Re: High Load/Interrupts on Wheezy

[Scott Ferguson]

On 04/07/13 00:30, Will Platnick wrote:
> More troubleshooting steps:
> 
> Built and installed latest 3.10 kernel, no change in interrupts
> Built and installed latest 2.6.32 kernel, and I am back to Squeeze level
> loads and no high timer, rescheduling, non-maskable or performance
> interrupts are present.
> 
> So, does anybody have any idea what changed in the 3.2+ series that
> could cause this?

Nope. But I've been experiencing the same thing so following your posts.
I've set up two identical LAMP servers hosting identical sites using
Virtualmin, one pure, standard (untweaked) Squeeze, the other pure,
standard (untweaked) Wheezy built from the same package list.  The
Squeeze one runs fine in 256MB of RAM, the Wheezy takes nearly three
times as long to boot, likewise to shutdown *even* when given 512MB of
RAM. Identical virtualmachine setups.

I've logged the output of ps aux and will compare them tomorrow night -
if I find anything obvious I'll post them.

Everything else is similar to your results.

> 
> On Wed, Jul 3, 2013 at 8:06 AM, Will Platnick  > wrote:
> 
> Something else I just noticed now that I'm on a screen high enough
> to show all of /proc/interrupts on one line:
> 


Kind regards

-- 
Iceweasel/Firefox/Chrome/Chromium/Iceape/IE extensions for finding
answers to Debian questions:-
https://addons.mozilla.org/en-US/firefox/collections/Scott_Ferguson/debian/



signature.asc
Description: OpenPGP digital signature

Re: High Load/Interrupts on Wheezy

[Will Platnick]

More troubleshooting steps:

Built and installed latest 3.10 kernel, no change in interrupts
Built and installed latest 2.6.32 kernel, and I am back to Squeeze level
loads and no high timer, rescheduling, non-maskable or performance
interrupts are present.

So, does anybody have any idea what changed in the 3.2+ series that could
cause this?

On Wed, Jul 3, 2013 at 8:06 AM, Will Platnick  wrote:

> Something else I just noticed now that I'm on a screen high enough to show
> all of /proc/interrupts on one line:
>
>
> Non-maskable interrupts are happening on Wheezy whereas they didn't on
> Squeeze. Additionally, it seems Non-maskable interrupts and Performance
> monitoring are the same value all the time.
>
>
> --
> Will Platnick
> Sent with Airmail 
>
> On July 3, 2013 at 7:17:04 AM, Will Platnick (wplatn...@gmail.com) wrote:
>
> I followed those. I got nothing.
> —
> Sent from Mailbox  for iPhone
>
>
> On Wed, Jul 3, 2013 at 5:24 AM, Darac Marjal wrote:
>
>> null
>> 
>
>
>

Re: High Load/Interrupts on Wheezy

[Will Platnick]

Something else I just noticed now that I'm on a screen high enough to show all of /proc/interrupts on one line:Non-maskable interrupts are happening on Wheezy whereas they didn't on Squeeze. Additionally, it seems Non-maskable interrupts and Performance monitoring are the same value all the time. -- Will PlatnickSent with Airmail On July 3, 2013 at 7:17:04 AM, Will Platnick (wplatn...@gmail.com) wrote: I followed those. I got nothing.—Sent from Mailbox for iPhoneOn Wed, Jul 3, 2013 at 5:24 AM, Darac Marjal  wrote:null

Re: High Load/Interrupts on Wheezy

[Will Platnick]

I followed those. I got nothing.
—
Sent from Mailbox for iPhone

On Wed, Jul 3, 2013 at 5:24 AM, Darac Marjal 
wrote:

> On Tue, Jul 02, 2013 at 08:54:06PM -0400, Will Platnick wrote:
>> I am experiencing some issues with load after upgrading some of my Squeeze 
>> boxes to Wheezy. I have 7 app servers, all with identical hardware with 
>> identical packages and code. I upgraded one of my boxes to wheezy, along 
>> with the custom packages we use for Python, PHP, etc… Same versions of the 
>> software, just built on Wheezy instead of Squeeze. My problem is that my 
>> Wheezy boxes have a load of over 3 and are not staying up during our peak 
>> time, whereas our squeeze boxes have a load of less than 1. 
>> The interesting part, is that despite the high load, my wheezy boxes are 
>> actually performing quite well, and are outperforming my squeeze boxes by 
>> 2-3 ms. Never the less, the high load is giving us cause for concern and is 
>> stopping us from migrating completely, and we're wondering if anybody else 
>> is seeing the same thing or can give us some assistance on where to go from 
>> here.
>> I believe I have tracked down the issue with our load to be an interrupt 
>> issue. My interrupts on wheezy are way higher. CPU, I/O, Memory and Context 
>> Switches are all the same (measured with top, atop, iotop, vmstat). It 
>> doesn't appear to be a hardware issue, as I deployed wheezy and our code 
>> base to a different and faster motherboard/cpu combo, and the issue remained.
>> The items that stands out is that my "Rescheduling Interrupts" and "timer" 
>> are interrupting like crazy on wheezy compared to squeeze. Here is my output 
>> of total interrupts on Squeeze vs Wheezy for two different machines, 
>> rebooted and placed into service at the exact same time, with traffic 
>> distributed to them via round robin, so it should be fairly equal.
>> Rescheduling Interrupts: 4109580 on Wheezy vs 67418 on Squeeze.
>> Timer: 504238 on Wheezy vs 50 on Squeeze.
>> 
>> Thoughts? Suggestions?
> This was the first search result for "Rescheduling Interrupts". The
> advice should apply to Debian equally well.
> https://help.ubuntu.com/community/ReschedulingInterrupts

Re: High Load/Interrupts on Wheezy

[Darac Marjal]

On Tue, Jul 02, 2013 at 08:54:06PM -0400, Will Platnick wrote:
> I am experiencing some issues with load after upgrading some of my Squeeze 
> boxes to Wheezy. I have 7 app servers, all with identical hardware with 
> identical packages and code. I upgraded one of my boxes to wheezy, along with 
> the custom packages we use for Python, PHP, etc… Same versions of the 
> software, just built on Wheezy instead of Squeeze. My problem is that my 
> Wheezy boxes have a load of over 3 and are not staying up during our peak 
> time, whereas our squeeze boxes have a load of less than 1. 
> The interesting part, is that despite the high load, my wheezy boxes are 
> actually performing quite well, and are outperforming my squeeze boxes by 2-3 
> ms. Never the less, the high load is giving us cause for concern and is 
> stopping us from migrating completely, and we're wondering if anybody else is 
> seeing the same thing or can give us some assistance on where to go from here.
> I believe I have tracked down the issue with our load to be an interrupt 
> issue. My interrupts on wheezy are way higher. CPU, I/O, Memory and Context 
> Switches are all the same (measured with top, atop, iotop, vmstat). It 
> doesn't appear to be a hardware issue, as I deployed wheezy and our code base 
> to a different and faster motherboard/cpu combo, and the issue remained.
> The items that stands out is that my "Rescheduling Interrupts" and "timer" 
> are interrupting like crazy on wheezy compared to squeeze. Here is my output 
> of total interrupts on Squeeze vs Wheezy for two different machines, rebooted 
> and placed into service at the exact same time, with traffic distributed to 
> them via round robin, so it should be fairly equal.
> Rescheduling Interrupts: 4109580 on Wheezy vs 67418 on Squeeze.
> Timer: 504238 on Wheezy vs 50 on Squeeze.
> 
> Thoughts? Suggestions?

This was the first search result for "Rescheduling Interrupts". The
advice should apply to Debian equally well.
https://help.ubuntu.com/community/ReschedulingInterrupts



signature.asc
Description: Digital signature

Re: high load but no cpu usage

[Shri Shrikumar]

On Sat, 2003-10-04 at 19:44, Rus Foster wrote:
> > Hi,
> >
> > I seem to have a strange problem. I have a server which is showing a
> > load average of around 1 but cpu usage of 0.6% over two cpus.
> 
> This would imply I/O wait for me. What sort of disks does it have?

Thats what I thought but this same machine has handles twice the load at
just 0.16 load average. Also, its running off 2 scsi disks which are
mirrored.

> Run vmstat 1 for a few minutes and post it here

I have attached the output of vmstat. The load average of the machine
was around 1 throughout. Please let me know if you want me to add a
longer vmstat run.

Best wishes,

Shri

-- 

Shri Shrikumar   U R Byte Solutions   Tel:   0845 644 4745
I.T. Consultant  Edinburgh, Scotland  Mob:   0773 980 3499
 Web: www.urbyte.com  Email: [EMAIL PROTECTED]


vmstat.gz
Description: GNU Zip compressed data


signature.asc
Description: This is a digitally signed message part

Re: high load but no cpu usage

[Rus Foster]

> Hi,
>
> I seem to have a strange problem. I have a server which is showing a
> load average of around 1 but cpu usage of 0.6% over two cpus.

This would imply I/O wait for me. What sort of disks does it have?

> What bothers me is that load average used to stay under 0.16 previously
> - nothing has changed. I have already tried to see if there are any
> processes blocking using ps auxwww but they all seem to be in State S
> with a few in SW and two in SWN.
>

Run vmstat 1 for a few minutes and post it here

Rgds

Rus
-- 
w: http://www.jvds.com  | Dedicated FreeBSD,Debian and RedHat Servers
e: [EMAIL PROTECTED]| Dontations made to Debian, FreeBSD
t: +44 7919 373537  | and Slackware
t: 1-888-327-6330   | email: [EMAIL PROTECTED]



-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED] 
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]

Re: high load average

[Jack O'Quin]

Bijan Soleymani <[EMAIL PROTECTED]> writes:

> > Is this normal?  I don't seem to remember having ide performance issues like 
> > this before (this is a new install).
> > 
> This is normal if dma is not enabled.
> It isn't enabled by default in Debian.
> To enable it install hdparm and then
> run hdparm -d1 /dev/hdx as root
> where x is either a,b,c,d depending on the
> ide device.
> 
> Hopefully that will work and your problem
> will be solved. If you're really lucky
> like me you can do something like
> hdparm -c3d1m16X66 /dev/hda to enable
> other options such as ATA-66. Just
> do man hdparm and check out the options.


This sounds like a problem I'm having.  I tried everything I could
figure out to enable DMA on my IDE drive, but it still won't take the
enable command...

[joq@sulphur] ~/ $ sudo hdparm -d 1 /dev/hda

/dev/hda:
 setting using_dma to 1 (on)
 HDIO_SET_DMA failed: Operation not permitted
 using_dma=  0 (off)



[joq@sulphur] ~/ $ grep IDEDMA /usr/src/kernel-source-2.4.18/.config
CONFIG_BLK_DEV_IDEDMA_PCI=y
CONFIG_IDEDMA_PCI_AUTO=y
CONFIG_BLK_DEV_IDEDMA=y
# CONFIG_IDEDMA_PCI_WIP is not set
# CONFIG_IDEDMA_NEW_DRIVE_LISTINGS is not set
CONFIG_IDEDMA_AUTO=y
# CONFIG_IDEDMA_IVB is not set



[joq@sulphur] ~/ $ sudo hdparm -i /dev/hda

/dev/hda:

 Model=IC35L040AVVA07-0, FwRev=VA2OA52A, SerialNo=VNC202A2L1SU7A
 Config={ HardSect NotMFM HdSw>15uSec Fixed DTR>10Mbs }
 RawCHS=16383/16/63, TrkSize=0, SectSize=0, ECCbytes=52
 BuffType=DualPortCache, BuffSize=1863kB, MaxMultSect=16, MultSect=16
 CurCHS=16383/16/63, CurSects=16514064, LBA=yes, LBAsects=80418240
 IORDY=on/off, tPIO={min:240,w/IORDY:120}, tDMA={min:120,rec:120}
 PIO modes: pio0 pio1 pio2 pio3 pio4 
 DMA modes: mdma0 mdma1 mdma2 udma0 udma1 udma2 udma3 udma4 *udma5 
 AdvancedPM=yes: disabled (255) WriteCache=enabled
 Drive Supports : ATA/ATAPI-5 T13 1321D revision 1 : ATA-2 ATA-3 ATA-4 ATA-5 

[joq@sulphur] ~/ $ sudo hdparm /dev/hda

/dev/hda:
 multcount= 16 (on)
 I/O support  =  1 (32-bit)
 unmaskirq=  1 (on)
 using_dma=  0 (off)
 keepsettings =  0 (off)
 nowerr   =  0 (off)
 readonly =  0 (off)
 readahead=  8 (on)
 geometry = 5005/255/63, sectors = 80418240, start = 0
 busstate =  1 (on)



[joq@sulphur] ~/ $ sudo lspci -v
00:00.0 Host bridge: VIA Technologies, Inc. VT8367 [KT266]
Subsystem: Asustek Computer, Inc.: Unknown device 807f
Flags: bus master, 66Mhz, medium devsel, latency 0
Memory at e000 (32-bit, prefetchable) [size=64M]
Capabilities: [a0] AGP version 2.0
Capabilities: [c0] Power Management version 2

00:01.0 PCI bridge: VIA Technologies, Inc. VT8367 [KT266 AGP] (prog-if 00 [Normal 
decode])
Flags: bus master, 66Mhz, medium devsel, latency 0
Bus: primary=00, secondary=01, subordinate=01, sec-latency=0
Memory behind bridge: dc80-dddf
Prefetchable memory behind bridge: ddf0-dfff
Capabilities: [80] Power Management version 2



00:11.1 IDE interface: VIA Technologies, Inc. Bus Master IDE (rev 06) (prog-if 8a 
[Master SecP PriP])
Subsystem: Asustek Computer, Inc.: Unknown device 808c
Flags: bus master, medium devsel, latency 32
I/O ports at b400 [size=16]
Capabilities: [c0] Power Management version 2



[joq@sulphur] ~/ $ cat /proc/ide/hda/driver 
ide-disk version 1.10
[joq@sulphur] ~/ $ cat /proc/ide/hda/model
IC35L040AVVA07-0
[joq@sulphur] ~/ $ sudo cat /proc/ide/hda/cache
1863
[joq@sulphur] ~/ $ sudo cat /proc/ide/hda/settings
namevalue   min max mode
-   --- --- 
bios_cyl50050   65535   rw
bios_head   255 0   255 rw
bios_sect   63  0   63  rw
breada_readahead4   0   127 rw
bswap   0   0   1   r
current_speed   0   0   69  rw
failures0   0   65535   rw
file_readahead  124 0   16384   rw
ide_scsi0   0   1   rw
init_speed  0   0   69  rw
io_32bit1   0   3   rw
keepsettings0   0   1   rw
lun 0   0   7   rw
max_failures1   0   65535   rw
max_kb_per_request  127 1   127 rw
multcount   8   0   8   rw
nice1   1   0   1   rw
nowerr  0  

Re: high load average

[Quenten Griffith]

Or just get hwtools it creates a basic init.d script where you put your 
hdparm flags

Bijan Soleymani wrote:

>>Is this normal?  I don't seem to remember having ide performance issues like 
>>this before (this is a new install).
>>
>>
>>
>This is normal if dma is not enabled.
>It isn't enabled by default in Debian.
>To enable it install hdparm and then
>run hdparm -d1 /dev/hdx as root
>where x is either a,b,c,d depending on the
>ide device.
>
>Hopefully that will work and your problem
>will be solved. If you're really lucky
>like me you can do something like
>hdparm -c3d1m16X66 /dev/hda to enable
>other options such as ATA-66. Just
>do man hdparm and check out the options.
>
>You might want to make a script to run
>hdparm on boot. You can put such
>a script in /etc/rc.boot
>it would look something like
>#!/bin/sh
>hdparm -options /dev/1stdevice
>hdparm -options /dev/2nddevice
>...
>
>Hope that helps,
>Bijan
>
>
>  
>



-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED] 
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]

Re: high load average

[Bijan Soleymani]

> Is this normal?  I don't seem to remember having ide performance issues like 
> this before (this is a new install).
> 
This is normal if dma is not enabled.
It isn't enabled by default in Debian.
To enable it install hdparm and then
run hdparm -d1 /dev/hdx as root
where x is either a,b,c,d depending on the
ide device.

Hopefully that will work and your problem
will be solved. If you're really lucky
like me you can do something like
hdparm -c3d1m16X66 /dev/hda to enable
other options such as ATA-66. Just
do man hdparm and check out the options.

You might want to make a script to run
hdparm on boot. You can put such
a script in /etc/rc.boot
it would look something like
#!/bin/sh
hdparm -options /dev/1stdevice
hdparm -options /dev/2nddevice
...

Hope that helps,
Bijan


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED] 
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]

Re: high load average

[nate]

Jason Pepas said:
> the other day I was moving several gigs of files from one ide drive to
> another on the same ide chain (the secondary channel is broken) and my
> load  average went up to around 7 (no, not 0.07).  The machine would
> become  unresponsive for several seconds at a time.  This is a
> uniprocessor machine,  both drives are ext2 filesystems.
>
> Is this normal?  I don't seem to remember having ide performance issues
> like  this before (this is a new install).


this is normal(in my experience) if DMA is not enabled on one or
more of the IDE drives in use.

some broken IDE chipsets(e.g. VIA) don't work well in DMA mode and
the driver may automatically revert to PIO mode(even if you told it to
use DMA) if it encounters problems in DMA mode(which prompted me to
start using promise IDE controllers on VIA boards a couple years ago)

nate




-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED] 
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]

Re: high load average

[Ramon Kagan]

Have you checked your dma settings?  hdparm/hwtools?

Ramon Kagan
York University, Computing and Network Services
Unix Team -  Intermediate System Administrator
(416)736-2100 #20263
[EMAIL PROTECTED]

-
I have not failed.  I have just
found 10,000 ways that don't work.
- Thomas Edison
-

On Mon, 23 Sep 2002, Jason Pepas wrote:

> the other day I was moving several gigs of files from one ide drive to
> another on the same ide chain (the secondary channel is broken) and my load
> average went up to around 7 (no, not 0.07).  The machine would become
> unresponsive for several seconds at a time.  This is a uniprocessor machine,
> both drives are ext2 filesystems.
>
> Is this normal?  I don't seem to remember having ide performance issues like
> this before (this is a new install).
>
> -jason
>
>
> --
> To UNSUBSCRIBE, email to [EMAIL PROTECTED]
> with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
>


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED] 
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]

Re: High Load Average

[Nate Amsden]

"Jordi S. Bunster" wrote:

> We JUST installed the server. I mean, there's nothing hand
> compiled, except for Amavis. But it doesn't eat that much CPU

amavis is VERY cpu intensive i run it on many systems. is there a lot
of mail going through the system? is there a lot of big attachments?
one of my mail servers didnt dip below load of 8 until i upgraded the
system's hardware. amavis is great..but if you got a lotta mail you
need more horsepower.

also if your using something like UW Imap that can be a cause for very
high load as well. i suggest switching to something else like CYRUS which
reduces load by a factor of 100-200 (it did for me anyways) im sure there
are other good IMAP servers like courier(sp?) but i haven't tried them.

same goes for POP3. if your using qpopper or ipop3d those can be causes
of high load as well(cyrus has a pop3 server as well, does not
cause high load)

if you have a lot of mail going through i suggest setting up a raid0
array(or raid 10) for /var/spool and have amavis scan mail off that
drive. get SCSI if you can for this.

sample mail server config:
Average KB/hour of mail: 873kB/H
Max KB/hour of mail: 9944.7kB/H
Average Mail/hour: 71
Max Mail/hour: 1081

Average System Load: 1.02
Max System Load: 4.09

(Statistics gathered from MRTG over the past ~6 weeks or so)

System config:
Dual P3-800Mhz
Dual 15k RPM Ultra160 SCSI drives raid1 /var/spool
single 15k RPM Ultra160 drive (no raid) /
256MB ram
256MB swap
Uptime: 74 days
Time spent idle: 87.0%
Linux 2.2.17+many patches (openwall included)
Debian GNU/Linux 2.2r3
sendmail 8.9.3 + amavis
Cyrus IMAP/POP
Apache
Apache+ssl
Squirrelmail (webmail front end)
Mcafee Antivirus 4.0.70

running amavis 0.2.1. hope this gives you an idea of what to expect
when using amavis as far as load goes.

nate

-- 
:::
ICQ: 75132336
http://www.aphroland.org/
http://www.linuxpowered.net/
[EMAIL PROTECTED]

Re: High Load Average

[Christoph Simon]

On Sun, 3 Jun 2001 23:18:41 -0300 (BRT)
"Jordi S. Bunster" <[EMAIL PROTECTED]> wrote:

> 
> > you have a run away process and/or a memory leak
> > 
> > ( amd and intel cpu behave slightly differently for 
> > ( the same code...
> 
> Mmm .. speaking about internal programs, we only have some perl
> scripts. Perl is the compiled one, right?
> 
> > what apps is running???
> 
> We JUST installed the server. I mean, there's nothing hand
> compiled, except for Amavis. But it doesn't eat that much CPU
> time. In fact, top reveals that everyone uses CPU all the time. A
> ipop3d session easily goes for 18%, and a apache or sendmail one
> goes for 47% ~ 56%. It is just like everyone is using the machine
> at its most.
> 
> Look:
> 
> 91 processes: 89 sleeping, 2 running, 0 zombie, 0 stopped
> CPU states: 68.7% user, 31.2% system,  0.0% nice,  0.0% idle
> Mem:  257856K av, 229104K used,  28752K free, 103600K shrd, 
> 73192K buff
> Swap: 128484K av,  0K used, 128484K free
> 86696K cached
> 
>   PID USER PRI  NI  SIZE  RSS SHARE STAT  LIB %CPU %MEM  
> TIME COMMAND
>   170 root   0   0   632  632   516 S   0  5.3  0.2  
> 6:00 syslogd
> 13533 root  10   0  1124 1120   780 S   0  4.3  0.4  
> 0:00 scanmails
> 12172 jsb8   0  1192 1192   688 R   0  4.1  0.4  
> 0:03 top
[...]
> 
> At this moment, Load is a little bit lower (about 4), but idle is
> still 0%. Quite weird uh?
> 
> If any command output is helpful, please let me know.

Your table isn't very meaningful, as it doesn't show even 20% of
load. It might take a while to see. But as you say that all are
usually high, maybe you've got a kernel problem, maybe due to a
hardware (IRQ?) conflict. Just a quick guess.

--
Christoph Simon
[EMAIL PROTECTED]
---
^X^C
q
quit
:q
^C
end
x
exit
ZZ
^D
?
help
shit
.

Re: High Load Average

[Noah L. Meyerhans]

On Sun, Jun 03, 2001 at 11:18:41PM -0300, Jordi S. Bunster wrote:
> 91 processes: 89 sleeping, 2 running, 0 zombie, 0 stopped
> CPU states: 68.7% user, 31.2% system,  0.0% nice,  0.0% idle
> Mem:  257856K av, 229104K used,  28752K free, 103600K shrd, 
> 73192K buff
> Swap: 128484K av,  0K used, 128484K free
> 86696K cached

This (along with the process list copied from 'top') is not enough.  I
suspect that you have several processes in state 'D' which is
"uninterruptable sleep".  Run 'ps auxwww' and search for any 'D's in the
STAT column.  They won't be using any CPU, but if they're hanging out in
that state they could indicate some other kind of problem, possibly
hardware related.

noah

-- 
 ___
| Web: http://web.morgul.net/~frodo/
| PGP Public Key: http://web.morgul.net/~frodo/mail.html 



pgpjVK6ydfMg8.pgp
Description: PGP signature

Re: High Load Average

[Petr \[Dingo\] Dvorak]

On Sun, 3 Jun 2001, Jordi S. Bunster wrote:

JSB> > you have a run away process and/or a memory leak
JSB> > 
JSB> > ( amd and intel cpu behave slightly differently for 
JSB> > ( the same code...
JSB> 
JSB> Mmm .. speaking about internal programs, we only have some perl
JSB> scripts. Perl is the compiled one, right?
JSB> 
JSB> > what apps is running???
JSB> 
JSB> We JUST installed the server. I mean, there's nothing hand
JSB> compiled, except for Amavis. But it doesn't eat that much CPU
JSB> time. In fact, top reveals that everyone uses CPU all the time. A
JSB> ipop3d session easily goes for 18%, and a apache or sendmail one
JSB> goes for 47% ~ 56%. It is just like everyone is using the machine
JSB> at its most.
JSB> 
JSB> Look:
JSB> 
JSB> 91 processes: 89 sleeping, 2 running, 0 zombie, 0 stopped
JSB> CPU states: 68.7% user, 31.2% system,  0.0% nice,  0.0% idle
JSB> Mem:  257856K av, 229104K used,  28752K free, 103600K shrd, 
JSB> 73192K buff
JSB> Swap: 128484K av,  0K used, 128484K free
JSB> 86696K cached

check your bios settings, it looks like you have disabled external or internal
cache .. they should be both enabled .. and all other memory
region shadowing/caching should be disabled.

Dingo.


  ).|.(
'.'___'.'
   ' '(>~<)' '
   -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-ooO-=(_)=-Ooo-=-=-=-=-=-=-=-=-=-=-=-=-=-
Petr [Dingo] Dvorak [EMAIL PROTECTED]
Coder - Purple Dragon MUD   pdragon.org port 
   -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-[ 369D93 ]=-=-
 Debian version 2.2.18pre21, up 4 days, 13 users, load average: 1.00
   -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-

Re: High Load Average

[Jordi S. Bunster]

> you have a run away process and/or a memory leak
> 
> ( amd and intel cpu behave slightly differently for 
> ( the same code...

Mmm .. speaking about internal programs, we only have some perl
scripts. Perl is the compiled one, right?

> what apps is running???

We JUST installed the server. I mean, there's nothing hand
compiled, except for Amavis. But it doesn't eat that much CPU
time. In fact, top reveals that everyone uses CPU all the time. A
ipop3d session easily goes for 18%, and a apache or sendmail one
goes for 47% ~ 56%. It is just like everyone is using the machine
at its most.

Look:

91 processes: 89 sleeping, 2 running, 0 zombie, 0 stopped
CPU states: 68.7% user, 31.2% system,  0.0% nice,  0.0% idle
Mem:  257856K av, 229104K used,  28752K free, 103600K shrd, 
73192K buff
Swap: 128484K av,  0K used, 128484K free
86696K cached

  PID USER PRI  NI  SIZE  RSS SHARE STAT  LIB %CPU %MEM  
TIME COMMAND
  170 root   0   0   632  632   516 S   0  5.3  0.2  
6:00 syslogd
13533 root  10   0  1124 1120   780 S   0  4.3  0.4  
0:00 scanmails
12172 jsb8   0  1192 1192   688 R   0  4.1  0.4  
0:03 top
13532 root   0   0  1552 1552  1200 S   0  0.7  0.6  
0:00 sendmail
  177 root   0   0  4160 4160   804 S   0  0.5  1.6  
2:30 named
11006 www-data   0   0  7632 7632  4776 S   0  0.5  2.9  
0:01 apache
13271 www-data   0   0  4804 4804  4656 S   0  0.5  1.8  
0:00 apache
13673 root  11   0   464  464   296 R   0  0.5  0.1  
0:00 file
11825 root   0   0  1480 1480  1220 S   0  0.3  0.5  
0:00 sshd
12136 rosanak1   0  1460 1460   972 S   0  0.3  0.5  
0:00 ipop3d
13529 root   0   0  1412 1412  1200 S   0  0.3  0.5  
0:00 sendmail
13627 root   0   0  1396 1396  1160 S   0  0.3  0.5  
0:00 sendmail
  357 root   0   0  1212 1212  1072 S   0  0.1  0.4  
0:06 sendmail
15976 thomas 0   0  5752 5752   972 S   0  0.1  2.2  
0:06 ipop3d
11525 www-data   0   0  4828 4828  4680 S   0  0.1  1.8  
0:00 apache
1 root   0   0   472  472   400 S   0  0.0  0.1  
0:12 init
2 root   0   0 00 0 SW  0  0.0  0.0  
0:00 kflushd
3 root   0   0 00 0 SW  0  0.0  0.0  
0:03 kupdate
4 root   0   0 00 0 SW  0  0.0  0.0  
0:02 kswapd
5 root   0   0 00 0 SW  0  0.0  0.0  
0:00 keventd
6 root -20 -20 00 0 SW< 0  0.0  0.0  
0:00 mdrecoveryd
  102 daemon 0   0   492  492   408 S   0  0.0  0.1  
0:00 portmap
  172 root   0   0   760  760   384 S   0  0.0  0.2  
0:00 klogd
  230 root   0   0   440  440   376 S   0  0.0  0.1  
0:00 gpm
  241 root   0   0   560  560   476 S   0  0.0  0.2  
0:00 lpd
  356 root   0   0  1188 1188   832 S   0  0.0  0.4  
0:02 nmbd
  371 root   0   0  1204 1204   532 S   0  0.0  0.4  
0:00 xfs
  380 root   0   0  1548 1548  1320 S   0  0.0  0.6  
0:00 ntpd
  398 root   0   0   848  844   684 S   0  0.0  0.3  
0:00 radwatch
  399 root   0   0   856  856   792 S   0  0.0  0.3  
0:02 radiusd
  438 root   0   0   844  844   788 S   0  0.0  0.3  
0:13 radiusd
  469 root   0   0   616  616   512 S   0  0.0  0.2  
0:00 cron
 6613 root   0   0   440  440   376 S   0  0.0  0.1  
0:00 getty
22159 root   0   0   584  584   500 S   0  0.0  0.2  
0:03 inetd
31116 root   0   0  1224 1224   644 S   0  0.0  0.4  
0:00 smbmount-2.2
31129 root   0   0  1216 1216   744 S   0  0.0  0.4  
0:00 smbmount-2.2
31141 root   0   0  1220 1220   744 S   0  0.0  0.4  
0:00 smbmount-2.2
31159 root   0   0  1220 1220   744 S   0  0.0  0.4  
0:00 smbmount-2.2
31172 root   0   0  1220 1220   744 S   0  0.0  0.4  
0:00 smbmount-2.2


At this moment, Load is a little bit lower (about 4), but idle is
still 0%. Quite weird uh?

If any command output is helpful, please let me know.

  Jordi S. Bunster
<[EMAIL PROTECTED]>

Re: High Load Average

[Alvin Oga]

hi ay

or you could have a hacker running an irc on your machine
-- if the rest of your lan/machines is fine...
   than probably not

c ya
alvin


On Sun, 3 Jun 2001, Alvin Oga wrote:

> 
> hi ya jordi
> 
> you have a run away process and/or a memory leak
> 
> ( amd and intel cpu behave slightly differently for 
> ( the same code...
> 
> what apps is running???
> 
> top -i
> ps axuw
> 
> > Just a question: Is there any reason in particular for a Debian
> > Box keep its load average always over 6?

Re: High Load Average

[Christoph Simon]

On Sun, 3 Jun 2001 22:51:51 -0300 (BRT)
"Jordi S. Bunster" <[EMAIL PROTECTED]> wrote:

> Just a question: Is there any reason in particular for a Debian
> Box keep its load average always over 6?

Not really. Did you try top to find out which processes are doing
that? Maybe you where running a Netscape/Mozilla client and some java
stuff keeps runnig after a crash...

--
Christoph Simon
[EMAIL PROTECTED]
---
^X^C
q
quit
:q
^C
end
x
exit
ZZ
^D
?
help
shit
.

Re: High Load Average

[Alvin Oga]

hi ya jordi

you have a run away process and/or a memory leak

( amd and intel cpu behave slightly differently for 
( the same code...

what apps is running???

top -i
ps axuw

c ya
alvin


On Sun, 3 Jun 2001, Jordi S. Bunster wrote:

> 
> Just a question: Is there any reason in particular for a Debian
> Box keep its load average always over 6?
> 
> It is a AMD Athlon 750 Mhz with 256 Megs of RAM, running potato
> and 2.2.19, compiled to run on i686. It has the Patches Debian
> puts on the stock kernel, and the new-style raid patches,
> although no RAIDs are set up yet.
> 
> Sometimes the Load Average goes over 10, making sendmail refuse
> connections. It is running sendmail, IMAP, POP3, apache+perl,
> Radius(cistron) and that's it. What can possibly be wrong?
> 
> Sidenote: We had another similar machine (processor was a PIII
> 550 Mhz) running the same stuff, but with Slackware. Load was
> never that high, and the machine swapped all the time, at least
> 25 Megs. The new Debian Box never swaps, but has a high load
> always.
> 

Re: High Load Average

[Forrest English]

what is running on it? have you checked top for processes?


--
Forrest English
http://truffula.net

"When we have nothing left to give
There will be no reason for us to live
But when we have nothing left to lose
You will have nothing left to use"
-Fugazi 

On Sun, 3 Jun 2001, Jordi S. Bunster wrote:

> 
> Just a question: Is there any reason in particular for a Debian
> Box keep its load average always over 6?
> 
> It is a AMD Athlon 750 Mhz with 256 Megs of RAM, running potato
> and 2.2.19, compiled to run on i686. It has the Patches Debian
> puts on the stock kernel, and the new-style raid patches,
> although no RAIDs are set up yet.
> 
> Sometimes the Load Average goes over 10, making sendmail refuse
> connections. It is running sendmail, IMAP, POP3, apache+perl,
> Radius(cistron) and that's it. What can possibly be wrong?
> 
> Sidenote: We had another similar machine (processor was a PIII
> 550 Mhz) running the same stuff, but with Slackware. Load was
> never that high, and the machine swapped all the time, at least
> 25 Megs. The new Debian Box never swaps, but has a high load
> always.
> 
> 
> Any thoughts?
> 
> 
>   Jordi S. Bunster
> <[EMAIL PROTECTED]>
> 
> 
> 
> -- 
> To UNSUBSCRIBE, email to [EMAIL PROTECTED] 
> with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
> 

Re: high load average

[Dave Sherohman]

On Fri, Mar 09, 2001 at 03:25:24PM -0800, kmself@ix.netcom.com wrote:
> The clarification is given in the O'Reilly citation.  Runnable
> processes, not waiting on other resources, I/O blocking excepted.

Excellent - thanks!

-- 
Linux will do for applications what the Internet did for networks. 
- IBM, "Peace, Love, and Linux"
Geek Code 3.1:  GCS d? s+: a- C++ UL++$ P++>+++ L+++> E- W--(++) N+ o+
!K w---$ O M- V? PS+ PE Y+ PGP t 5++ X+ R++ tv b+ DI D G e* h+ r y+

Re: high load average

[kmself]

on Fri, Mar 09, 2001 at 01:27:50AM -0600, Dave Sherohman ([EMAIL PROTECTED]) 
wrote:
> On Thu, Mar 08, 2001 at 10:55:10PM -0800, kmself@ix.netcom.com wrote:
> > on Tue, Mar 06, 2001 at 11:21:07AM -0600, Dave Sherohman ([EMAIL 
> > PROTECTED]) wrote:
> > > You have the notation correct, but load average and CPU
> > > utilization are not directly related.  Load average is the average
> > > number of processes that are waiting on system resources over a
> > > certain time period; they could be waiting for CPU, for I/O, or
> > > for other resources.
> 
> > It *is* CPU.  These are processes in the run queue.  A process
> > blocked for I/O or another resource is blocked, not runnable
> 
> OK, now I'm confused...

I'm also somewhat fallible.  So, we'll get to the source of the question
this time.

In particular, a job blocked for I/O *is* runnable.  My error.

> My statements were based on my memory of a thread from last May (was
> it that long ago?) on this very list titled "(ot) What is load
> average?".  Checking back on the messages I saved from that
> conversation, I see a one from kmself@ix.netcom.com stating that load
> average is
> 
> | Number of processes in the run queue, averaged over time.  Often
> | confused with CPU utilization, which it is not.
> 
> Load average either is CPU or it isn't, right?  

Percent of clock ticks being utilized is CPU utilization.  Number of
jobs in runnable state is load average.  Related, but not identical
metrics.

My own statement:

Load average is a measure of _average current requests for CPU
processing_ over some time interval.

While we're at it, let's pull in a more authoritative definition, this
from _System Performance Tuning_, by Mike Loukides, O'Reilly, 1990:

The _system load average_ provides a convenient way to summarize the
activity on a system.  It is the first statistic you should look at
when performance seems to be poor.  UNIX defines load average as the
average number of processes in the kernel's run queue during an
interval.  A _process_ is a single stream of instructions.  Most
programs run as a single process, but some sapwn (UNIX terminology:
_fork_) other processes as they run.  A process is in the run queue
if it is:

  * Not waiting for any external event (e.g., not waiting for
someone to type a character at a terminal).
  
  * Not waiting of its own accord (e.g., the job hasn't called 'wait'.)

  * Not stopped (e.g., the job hasn't been stopped by CTRL-Z).
Processes cannot be stopped on XENIX and versions of System V.2.
The ability to stop processes has been added to System V.4 and
some versions of V.3.

While the load average is convenient, it may not give you an
accurate picture of the system's load.  There are two primary
reasons for this innaccuracy:

  * The load average counts as runnable all jobs waiting for disk
I/O.  This includes processes that are waiting for disk
operations to complete across NFS.  If an NFS server is not
responding (e.g., if the network is faulty or the server has
crashed), a percoess can wait for hours for an NFS operation to
complete.  It is considered runnable the entire time even though
nothing is happening; therefore, the load average climbs when
NFS servers crash, even though the system isn't really doing any
more work.

  * The load average does not account for scheduling priority.  It
does not differentiate between jobs that have been niced (i.e.,
placed at a lower priority and therefore not consuming much CPU
time) or jobs that are running at a high priority.

Hopefully, that clarifies a few misperceptions and sloppy statements (my
own included).

Specific to GNU/Linux, the count of active tasks is computed in
kernel/sched.c as:

static unsigned long count_active_tasks(void)
{
struct task_struct *p;
unsigned long nr = 0;

read_lock(&tasklist_lock);
for_each_task(p) {
if ((p->state == TASK_RUNNING ||
 p->state == TASK_UNINTERRUPTIBLE ||
 p->state == TASK_SWAPPING))
nr += FIXED_1;
}
read_unlock(&tasklist_lock);
return nr;
}

> So you can't have been correct both times.  

No, I am.  You're just not reading me consistently ;-)

My admonition in the current thread that load average is a metric of CPU
utilization is just that:  load average is concerned with CPU, it is
*not* concerned with memory, disk I/O (though I/O blocking can effect it),
etc.  However, as I clarify in this current post, and my prior thread,
load average is not equivalent to CPU _utilization_.

To put it in different terms:

   - Load average is how often you're asking for it.
   - CPU utilization is how often you're getting it.

High load average means you've got

Re: high load average

[Dave Sherohman]

On Thu, Mar 08, 2001 at 10:55:10PM -0800, kmself@ix.netcom.com wrote:
> on Tue, Mar 06, 2001 at 11:21:07AM -0600, Dave Sherohman ([EMAIL PROTECTED]) 
> wrote:
> > You have the notation correct, but load average and CPU utilization are not
> > directly related.  Load average is the average number of processes that are
> > waiting on system resources over a certain time period; they could be 
> > waiting
> > for CPU, for I/O, or for other resources.

> It *is* CPU.  These are processes in the run queue.  A process blocked
> for I/O or another resource is blocked, not runnable

OK, now I'm confused...

My statements were based on my memory of a thread from last May (was it
that long ago?) on this very list titled "(ot) What is load average?".
Checking back on the messages I saved from that conversation, I see a
one from kmself@ix.netcom.com stating that load average is

| Number of processes in the run queue, averaged over time.  Often
| confused with CPU utilization, which it is not.

Load average either is CPU or it isn't, right?  So you can't have been
correct both times.  Now, you may have been wrong last year and since
realized that it's more CPU-related than you had thought, but (aside from
this thread's original question describing a situation with a long-term
consistent load average of 2.00 and low-to-no CPU utilization) last
May's thread also included a message from [EMAIL PROTECTED] stating that

] It is the average number of processes in the 'R' (running/runnable) state
] (or blocked on I/O).

and

] The load average is most directly related to CPU.  Two CPU-intensive
] processes running will result in a load average of 2, etc.  But I/O
] intensive processes spend so much time active that they can drive up the
] load average also.  In addition if more than one process is blocked on I/O
] then the load average will go up very quickly, as both processes count
] toward the load even if only one can access the disk at a time.

Based on my observations of load and CPU readings on my boxes and the
messages from last May that I quoted above, I'm inclined to maintain
my earlier statement that processes waiting on any resource (not just
CPU) contribute to load.  But, if that's not the case, I'm willing to
be corrected.

-- 
Linux will do for applications what the Internet did for networks. 
- IBM, "Peace, Love, and Linux"
Geek Code 3.1:  GCS d? s+: a- C++ UL++$ P++>+++ L+++> E- W--(++) N+ o+
!K w---$ O M- V? PS+ PE Y+ PGP t 5++ X+ R++ tv b+ DI D G e* h+ r y+

Re: high load average

[kmself]

on Tue, Mar 06, 2001 at 11:21:07AM -0600, Dave Sherohman ([EMAIL PROTECTED]) 
wrote:
> On Tue, Mar 06, 2001 at 06:09:41PM +0100, Joris Lambrecht wrote:
> > isn't 2.00 more like 2% ? It is US notation where . is a decimal separator.
> > Not ?
> 
> You have the notation correct, but load average and CPU utilization are not
> directly related.  Load average is the average number of processes that are
> waiting on system resources over a certain time period; they could be waiting
> for CPU, for I/O, or for other resources.  (CPU does tend to be the biggest
> bottleneck, though, so a basic rule of thumb is that you usually don't want
> load to be much greater than the number of CPUs in the box.  

It *is* CPU.  These are processes in the run queue.  A process blocked
for I/O or another resource is blocked, not runnable (I think, I'm not
positive, but I'll bet my morning coffee on it -- which I *really* like,
and you'll want to give it to me anyway if I don't get it).

The significance of load average is that if you have more runnable
processes than CPUs, you have identified a system bottleneck:  it's now
possible to increase total system throughput by providing either more
and/or faster processors. 

Excessive swapping indicates the system is memory bound.  This isn't to
say that having a large amount of swapped memory is bad (it may or may
not be), but having a large number of processes swapping in and out of
memory is bad.

Not sure what the metric for I/O bound is.  Under Solaris, top would
report on I/O wait.  I could crack the O'Reilly system performance
tuning book and see what it says.

If none of the above are evident and things are still too slow, then
start optimizing your program(s).

> The machine I'm using starts killing off processes if load exceeds 6
> or 7; I wouldn't want to see it hit 100...)

It may not be all bad.  In certain cases, I believe Apache will spawn
large numbers of processes which manage to count against load average.
However, total system performance isn't actually negatively effected too
much.  I once took my UMP PII/180 box to a load of about 30 by running
multiple instances of computer v. computer gnuches  That took a
while to clean up.

-- 
Karsten M. Self http://kmself.home.netcom.com/
 What part of "Gestalt" don't you understand?   There is no K5 cabal
  http://gestalt-system.sourceforge.net/ http://www.kuro5hin.org


pgp78fxrBeMql.pgp
Description: PGP signature

Re: high load average

[kmself]

on Mon, Mar 05, 2001 at 11:12:16PM -0500, MaD dUCK ([EMAIL PROTECTED]) wrote:
> [cc'ing this to PLUG because it seems interesting...]
> 
> also sprach kmself@ix.netcom.com (on Mon, 05 Mar 2001 08:02:51PM -0800):
> > It's not 200% loaded.  There are two processes in the run queue.  I'd do
> 
> huh? is that what 2.00 means? the average length of the run queue?

Yep.

> that would explain it because i found two STAT = D processes which i
> cannot kill (any hints what to do when kill -9 doesn't work and the
> /proc/`pidof` directory cannot be removed?). that's why 2.00.

Find their parents and kill them.  Easiest way IMO is to use pstree:

$ pstree -p

...search for the PIDs of the defunct processes, locate parent(s), kill
same.  Report back if problems.

-- 
Karsten M. Self http://kmself.home.netcom.com/
 What part of "Gestalt" don't you understand?   There is no K5 cabal
  http://gestalt-system.sourceforge.net/ http://www.kuro5hin.org


pgpvocfV4OJyu.pgp
Description: PGP signature

Re: high load average

[Dave Sherohman]

On Tue, Mar 06, 2001 at 06:09:41PM +0100, Joris Lambrecht wrote:
> isn't 2.00 more like 2% ? It is US notation where . is a decimal separator.
> Not ?

You have the notation correct, but load average and CPU utilization are not
directly related.  Load average is the average number of processes that are
waiting on system resources over a certain time period; they could be waiting
for CPU, for I/O, or for other resources.  (CPU does tend to be the biggest
bottleneck, though, so a basic rule of thumb is that you usually don't want
load to be much greater than the number of CPUs in the box.  The machine I'm
using starts killing off processes if load exceeds 6 or 7; I wouldn't want to
see it hit 100...)

-- 
Linux will do for applications what the Internet did for networks. 
- IBM, "Peace, Love, and Linux"
Geek Code 3.1:  GCS d? s+: a- C++ UL++$ P++>+++ L+++> E- W--(++) N+ o+
!K w---$ O M- V? PS+ PE Y+ PGP t 5++ X+ R++ tv b+ DI D G e* h+ r y+

RE: high load average

[Joris Lambrecht]

Dear dUCK,

isn't 2.00 more like 2% ? It is US notation where . is a decimal separator.
Not ?

-Original Message-
From: MaD dUCK [mailto:[EMAIL PROTECTED]
Sent: Tuesday, March 06, 2001 3:38 AM
To: debian users
Subject: high load average


someone explain this to me:

albatross:~$ uname -a
Linux albatross 2.2.17 #2 Mon Sep 04 20:49:27 CET 2000 i586 unknown

albatross:~$ uptime
  2:56am  up 174 days,  5:50,  1 user,  load average: 2.00, 2.05, 2.01

# processes sorted by decreasing cpu usage
albatross:~$ ps aux | head -1 && ps aux | sort -nrk3 | head -5
USER   PID %CPU %MEM   VSZ  RSS TTY  STAT START   TIME COMMAND
root 15889  0.2  1.6  2720 1536 ?S02:50   0:02
/usr/sbin/sshd
root  1646  0.1  0.9  1672  864 ?S 2000  32:01
/usr/sbin/diald -
xfs   1776  0.0  1.0  2060 1020 ?S 2000   0:00 xfs -droppriv
-da
squid 1748  0.0  0.3  1088  332 ?S 2000   0:06 (unlinkd)
squid 1742  0.0 19.2 20048 18440 ?   S 2000  15:01 (squid) -D
root 25890  0.0  0.7  1652  764 ?D00:01   0:00 sh
/etc/ppp/ip-up
root 25889  0.0  0.7  1644  752 ?S00:01   0:00 bash
/etc/ppp/ip-

the load average displayed by uptime has been very consistently above
2.00 and the output of ps aux has been pretty much the same for the
past two weeks. no hung jobs. no traffic. the server basically *isn't
being used*, especially not during the last 1, 5, or 15 minutes. and
cron isn't running, there are *only* 35 running jobs. why, oh why then
is it 200% loaded???

martin

[greetings from the heart of the sun]# echo [EMAIL PROTECTED]:1:[EMAIL 
PROTECTED]@@@.net
-- 
the web site you seek
cannot be located but
endless others exist.


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED] 
with a subject of "unsubscribe". Trouble? Contact
[EMAIL PROTECTED]

Re: high load average

[MaD dUCK]

[cc'ing this to PLUG because it seems interesting...]

also sprach kmself@ix.netcom.com (on Mon, 05 Mar 2001 08:02:51PM -0800):
> It's not 200% loaded.  There are two processes in the run queue.  I'd do

huh? is that what 2.00 means? the average length of the run queue?

that would explain it because i found two STAT = D processes which i
cannot kill (any hints what to do when kill -9 doesn't work and the
/proc/`pidof` directory cannot be removed?). that's why 2.00.

thanks,
martin

[greetings from the heart of the sun]# echo [EMAIL PROTECTED]:1:[EMAIL 
PROTECTED]@@@.net
-- 
micro$oft is to operating systems & security
what mcdonalds is to gourmet cuisine.

Re: high load average

[kmself]

on Mon, Mar 05, 2001 at 09:37:36PM -0500, MaD dUCK ([EMAIL PROTECTED]) wrote:
> someone explain this to me:
> 
> albatross:~$ uname -a
> Linux albatross 2.2.17 #2 Mon Sep 04 20:49:27 CET 2000 i586 unknown
> 
> albatross:~$ uptime
>   2:56am  up 174 days,  5:50,  1 user,  load average: 2.00, 2.05, 2.01
> 
> # processes sorted by decreasing cpu usage
> albatross:~$ ps aux | head -1 && ps aux | sort -nrk3 | head -5
> USER   PID %CPU %MEM   VSZ  RSS TTY  STAT START   TIME COMMAND
> root 15889  0.2  1.6  2720 1536 ?S02:50   0:02 /usr/sbin/sshd
> root  1646  0.1  0.9  1672  864 ?S 2000  32:01 
> /usr/sbin/diald -
> xfs   1776  0.0  1.0  2060 1020 ?S 2000   0:00 xfs -droppriv 
> -da
> squid 1748  0.0  0.3  1088  332 ?S 2000   0:06 (unlinkd)
> squid 1742  0.0 19.2 20048 18440 ?   S 2000  15:01 (squid) -D
> root 25890  0.0  0.7  1652  764 ?D00:01   0:00 sh 
> /etc/ppp/ip-up
> root 25889  0.0  0.7  1644  752 ?S00:01   0:00 bash 
> /etc/ppp/ip-
> 
> the load average displayed by uptime has been very consistently above
> 2.00 and the output of ps aux has been pretty much the same for the
> past two weeks. no hung jobs. no traffic. the server basically *isn't
> being used*, especially not during the last 1, 5, or 15 minutes. and
> cron isn't running, there are *only* 35 running jobs. why, oh why then
> is it 200% loaded???

It's not 200% loaded.  There are two processes in the run queue.  I'd do
a 'ps aux' and look at what's runnable (STAT = 'R').  You might have to
do this repeatedly to find out what's there.  If it's the same processes
consistently, you might look to see what they or their children are
doing.

Note that you list *no* runnable processes in your ps output.

-- 
Karsten M. Self http://kmself.home.netcom.com/
 What part of "Gestalt" don't you understand?   There is no K5 cabal
  http://gestalt-system.sourceforge.net/ http://www.kuro5hin.org


pgpVPI810tT06.pgp
Description: PGP signature

Re: high load average

[MaD dUCK]

also sprach Noah L. Meyerhans (on Mon, 05 Mar 2001 09:51:53PM -0500):
> Load average is not an indication of how busy the CPU is.  A busy CPU
> can *cause* a high load average, but so can other stuff.

good point. so i found two offending processes in state D:

root 24520  0.0  0.9  1652  904 ?DFeb25   0:00 /bin/gawk
root 25890  0.0  0.7  1652  764 ?D00:01   0:00 sh /etc/ppp/ip-up

however, a kill -9 on either one doesn't delete them, i cannot delete
it's directory in /proc (as works on solaris 2.6), and to the best of
my knowledge, these processes won't go away.

any tips, other than to reboot?

martin

[greetings from the heart of the sun]# echo [EMAIL PROTECTED]:1:[EMAIL 
PROTECTED]@@@.net
-- 
oxymoron: micro$oft works

Re: high load average

[Noah L. Meyerhans]

On Mon, Mar 05, 2001 at 09:37:36PM -0500, MaD dUCK wrote:
> the load average displayed by uptime has been very consistently above
> 2.00 and the output of ps aux has been pretty much the same for the
> past two weeks. no hung jobs. no traffic. the server basically *isn't
> being used*, especially not during the last 1, 5, or 15 minutes. and
> cron isn't running, there are *only* 35 running jobs. why, oh why then
> is it 200% loaded???

Load average is not an indication of how busy the CPU is.  A busy CPU
can *cause* a high load average, but so can other stuff.

In this case, I would guess that the high load is caused by processes
being blocked while waiting for IO routines to complete.  In the ps
output or in top, look for processes in state 'D'.  I suspect you'll
find 2 of them (one was a ppp process visible in the ps output you
posted).  Figure out why they're blocking and you'll be able to do
something to fix it.

noah

-- 
 ___
| Web: http://web.morgul.net/~frodo/
| PGP Public Key: http://web.morgul.net/~frodo/mail.html 



pgph2L7ZgFNFZ.pgp
Description: PGP signature

Re: High load

[Raghavendra Bhat]

Suresh Kumar posts:

> I have never seen load averages going above 2
> earlier with redhat installation. 
> 

On a similar setup while running Netscape ?  Please
install libc5 and libg++272 found in /oldlibs of the
Debian 'slink' CD.


ragOO, VU2RGU. Kochi, INDIA.
Keeping the Air-Waves FREE.Amateur Radio
Keeping the W W W FREE..Debian GNU/Linux

RE: High load

[Bryan Scaringe]

Recent versions of netscape will slow a 16Mb system to a crawl.  How does the
system respond when you aren't running netscape?  What window manager
are you using?  What else are you running at the time.  Check you netscape
memory cache size.

I would be wiling to bet the problem lies in the (lack of) RAM.

Bryan


On 28-Apr-2000 Suresh Kumar.R wrote:
> Hi,
> 
> I recently installed a debian 2.1 on my machine which was earlier running
> redhat 5.2. (pentium 100MHz, 16mb ram). The machine becomes very very slow
> and unusable when I run netscape. I have dialup connection. The load
> average goes 100 and more. I have never seen load averages going above 2
> earlier with redhat installation. 
> 
> I tried issuing top command to know who the culprit is. I could not find
> much sense from the listing. It showed multiple entries of syslogd.
> 
> Any ideas on how to make the system useful ?
> 
> Suresh
> -
> Suresh Kumar.REmail: [EMAIL PROTECTED]
> Dept of Electronics & Communication
> College of Engineering, Trivandrum - 695 016
> INDIA
> 
> 
> 
> 
> 
> 
> -- 
> Unsubscribe?  mail -s unsubscribe [EMAIL PROTECTED] <
> /dev/null

Re: high load but idle CPU

[Joey Hess]

George Bonser wrote:
> Any process involved with heavy net activity in an SMP system with 2.2.3
> will do this. I had problems with web servers doing it. 2.2.9 seems OK.
> 2.2.6/7 were disasters. 2.2.5 seemed to work, though.

Hm, could you expand on that? I've been using 2.2.7 for a while, what
problems does it have?

-- 
see shy jo

Re: high load but idle CPU

[Max]

* George Bonser <[EMAIL PROTECTED]> [05/26/99 18:59] wrote:
> Do a ps -ax and see how many processes you have stuck in D state ;). Then
> go and get 2.2.9

Yup, that explains it!  I have 5 sxid processes in D state.
Hmmmcould it have something to do with the fact that I installed
arla 5 days ago and sxid is trying to traverse the entire AFS tree? :)
I guess I'll have to wait till the next reboot to clear these D
processes out.  In the meantime, editing sxid.conf is a good idea. :)

Thanks,
Max

-- 
The hopeful depend on a world without end
Whatever the hopeless may say
 Neil Peart, 1985


pgp6JTuhkfWBY.pgp
Description: PGP signature