Re: [CentOS] swap memory crash

[nate]

John R Pierce wrote:

> and, for that matter, someone running Oracle in production should have
> an Oracle support contract, and should work through them.

Yeah both right, didn't even think about that.. I'd be pretty
scared to run an Oracle DB with a newbie system admin. Not
knowing what sort of data is in the DB, if it's even a little
important I'd strongly suggest the OP getting someone more
experienced to manage the system, Not knowing where the OP
is I recommend Blue Gecko, a U.S. Oracle/Linux consulting shop
based out of Seattle, used them for Oracle DBs I had at my
last company.

http://www.bluegecko.net/

Oracle is a big complex beast, really powerful, and a real
good product if you use it right. When I first started using
it I really didn't like it and wanted to get rid of it, but
it's grown on me a lot over the years and I've gained a lot
of respect for it as a product and the technologies behind
it. And while Enterprise edition is really expensive, Standard
edition is dirt cheap by comparison and still very capable.

nate

___
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos

Re: [CentOS] swap memory crash

[John R Pierce]

MHR wrote:

On Thu, Sep 4, 2008 at 5:53 AM, lingu <[EMAIL PROTECTED]> wrote:
  

Dear all,

   I am getting below syslog error on my oracle database server running on
RHEL4U5. During this time my swap shows zero on both available and  free in
the top command output , result in total system hang. can any one guide me
how to fix this issue of memory and exactly what  this error indicates??


It strikes me that someone running RH4U5 should be able to get
technical support from Red Hat, no?...
  



and, for that matter, someone running Oracle in production should have 
an Oracle support contract, and should work through them.



___
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos

Re: [CentOS] swap memory crash

[MHR]

On Thu, Sep 4, 2008 at 5:53 AM, lingu <[EMAIL PROTECTED]> wrote:
> Dear all,
>
>I am getting below syslog error on my oracle database server running on
> RHEL4U5. During this time my swap shows zero on both available and  free in
> the top command output , result in total system hang. can any one guide me
> how to fix this issue of memory and exactly what  this error indicates??
>

It strikes me that someone running RH4U5 should be able to get
technical support from Red Hat, no?...

I know they're virtually identical, but, as others with more weight
than I on this list have pointed out, this is for support on CentOS
issues for CentOS users on CentOS systems, or am I going too far here?

mhr
___
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos

Re: [CentOS] swap memory crash

[nate]

lingu wrote:
> Dear all,
>
>I am getting below syslog error on my oracle database server running on
> RHEL4U5. During this time my swap shows zero on both available and  free in
> the top command output , result in total system hang. can any one guide me
> how to fix this issue of memory and exactly what  this error indicates??

It indicates that you ran out of memory.

1) add more memory
and/or
2) monitor the system closely to determine what is using all of
the available memory and either configure it not to, kill it, or
don't run it at all.

The system sometimes(depending on kernel version) tries to
kill processes it thinks might be related to the excessive
memory usage, to prevent the total system hang, but it's not
always successful.

I use nagios for threshold alerting, and it has a handy check_swap
monitor, I use it to alert when swap is 5% utilized. For my
systems swap is for emergency purposes only.

If Oracle is the only real application on that box and if this
problem happened suddenly, look at the statspack information
(or whatever it's called these days) and try to track down the
query that caused the spike in memory usage.

I found on a couple of occasions with bad queries in Oracle it
could cause memory usage to suddenly spike. Fortunately the
system had something like 16gigs of swap on an 8 disk 15k RPM
RAID 1+0 that was otherwise not used, so while performance
was really slow, the box didn't go down. I think the system
had 16gigs of ram.

I hope you have at least 16GB of memory, with memory prices
these days it's cheap even to have 32GB. Even if you don't
allocate it all to Oracle it's useful to help protect
against situations like this.

nate


___
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos

Re: [CentOS] swap memory crash

[James Pearson]

lingu wrote:

Dear all,

   I am getting below syslog error on my oracle database server running on
RHEL4U5. During this time my swap shows zero on both available and  free in
the top command output , result in total system hang. can any one guide me
how to fix this issue of memory and exactly what  this error indicates??

ERROR FROM SYSLOG

#
Aug 30 15:46:20 crmdb kernel: cpu 7 cold: low 0, high 32, batch 16
Aug 30 15:46:20 crmdb kernel:
Aug 30 15:46:20 crmdb kernel: Free pages:   16000kB (2560kB HighMem)
Aug 30 15:46:20 crmdb kernel: Active:1433949 inactive:40363 dirty:1
writeback:0 unstable:0 free:4
000 slab:24910 mapped:1405036 pagetables:565097
Aug 30 15:46:20 crmdb kernel: DMA free:12544kB min:16kB low:32kB high:48kB
active:0kB inactive:0k
B present:16384kB pages_scanned:1878896 all_unreclaimable? yes
Aug 30 15:46:20 crmdb kernel: protections[]: 0 0 0
Aug 30 15:46:20 crmdb kernel: Normal free:896kB min:928kB low:1856kB
high:2784kB active:736kB ina
ctive:208kB present:901120kB pages_scanned:1889 all_unreclaimable? yes
Aug 30 15:46:20 crmdb kernel: protections[]: 0 0 0
Aug 30 15:46:20 crmdb kernel: HighMem free:2560kB min:512kB low:1024kB
high:1536kB active:5735060
kB inactive:161244kB present:8257532kB pages_scanned:0 all_unreclaimable? no
Aug 30 15:46:20 crmdb kernel: protections[]: 0 0 0
Aug 30 15:46:20 crmdb kernel: DMA: 2*4kB 5*8kB 3*16kB 3*32kB 3*64kB 3*128kB
2*256kB 0*512kB 1*102
4kB 1*2048kB 2*4096kB = 12544kB
Aug 30 15:46:20 crmdb kernel: Normal: 0*4kB 28*8kB 2*16kB 0*32kB 0*64kB
1*128kB 0*256kB 1*512kB 0
*1024kB 0*2048kB 0*4096kB = 896kB
Aug 30 15:46:20 crmdb kernel: HighMem: 512*4kB 8*8kB 2*16kB 1*32kB 0*64kB
1*128kB 1*256kB 0*512kB
 0*1024kB 0*2048kB 0*4096kB = 2560kB
Aug 30 15:46:20 crmdb kernel: Swap cache: add 26913978, delete 26700605,
find 10635368/14522876, race 848+1364
Aug 30 15:46:20 crmdb kernel: 0 bounce buffer pages
Aug 30 15:46:20 crmdb kernel: Free swap:   14593184kB
Aug 30 15:46:20 crmdb kernel: 2293759 pages of RAM
Aug 30 15:46:20 crmdb kernel: 1867351 pages of HIGHMEM
Aug 30 15:46:20 crmdb kernel: 216549 reserved pages
Aug 30 15:46:20 crmdb kernel: 44388024 pages shared
Aug 30 15:46:20 crmdb kernel: 213522 pages swap cached
Aug 30 15:46:20 crmdb kernel: Out of Memory: Killed process 22462 (oracle).
##


Not sure if this is connected, but I had what looks like a similar 
problem - see: 


The workaround I use is to use the following sysctl setting:

vm.lower_zone_protection = 100

James Pearson
___
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos

[CentOS] swap memory crash

[lingu]

Dear all,

   I am getting below syslog error on my oracle database server running on
RHEL4U5. During this time my swap shows zero on both available and  free in
the top command output , result in total system hang. can any one guide me
how to fix this issue of memory and exactly what  this error indicates??

ERROR FROM SYSLOG

#
Aug 30 15:46:20 crmdb kernel: cpu 7 cold: low 0, high 32, batch 16
Aug 30 15:46:20 crmdb kernel:
Aug 30 15:46:20 crmdb kernel: Free pages:   16000kB (2560kB HighMem)
Aug 30 15:46:20 crmdb kernel: Active:1433949 inactive:40363 dirty:1
writeback:0 unstable:0 free:4
000 slab:24910 mapped:1405036 pagetables:565097
Aug 30 15:46:20 crmdb kernel: DMA free:12544kB min:16kB low:32kB high:48kB
active:0kB inactive:0k
B present:16384kB pages_scanned:1878896 all_unreclaimable? yes
Aug 30 15:46:20 crmdb kernel: protections[]: 0 0 0
Aug 30 15:46:20 crmdb kernel: Normal free:896kB min:928kB low:1856kB
high:2784kB active:736kB ina
ctive:208kB present:901120kB pages_scanned:1889 all_unreclaimable? yes
Aug 30 15:46:20 crmdb kernel: protections[]: 0 0 0
Aug 30 15:46:20 crmdb kernel: HighMem free:2560kB min:512kB low:1024kB
high:1536kB active:5735060
kB inactive:161244kB present:8257532kB pages_scanned:0 all_unreclaimable? no
Aug 30 15:46:20 crmdb kernel: protections[]: 0 0 0
Aug 30 15:46:20 crmdb kernel: DMA: 2*4kB 5*8kB 3*16kB 3*32kB 3*64kB 3*128kB
2*256kB 0*512kB 1*102
4kB 1*2048kB 2*4096kB = 12544kB
Aug 30 15:46:20 crmdb kernel: Normal: 0*4kB 28*8kB 2*16kB 0*32kB 0*64kB
1*128kB 0*256kB 1*512kB 0
*1024kB 0*2048kB 0*4096kB = 896kB
Aug 30 15:46:20 crmdb kernel: HighMem: 512*4kB 8*8kB 2*16kB 1*32kB 0*64kB
1*128kB 1*256kB 0*512kB
 0*1024kB 0*2048kB 0*4096kB = 2560kB
Aug 30 15:46:20 crmdb kernel: Swap cache: add 26913978, delete 26700605,
find 10635368/14522876, race 848+1364
Aug 30 15:46:20 crmdb kernel: 0 bounce buffer pages
Aug 30 15:46:20 crmdb kernel: Free swap:   14593184kB
Aug 30 15:46:20 crmdb kernel: 2293759 pages of RAM
Aug 30 15:46:20 crmdb kernel: 1867351 pages of HIGHMEM
Aug 30 15:46:20 crmdb kernel: 216549 reserved pages
Aug 30 15:46:20 crmdb kernel: 44388024 pages shared
Aug 30 15:46:20 crmdb kernel: 213522 pages swap cached
Aug 30 15:46:20 crmdb kernel: Out of Memory: Killed process 22462 (oracle).
##


Regards
lingu
___
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos