urgent problem [7:14942]
i have router 3640 with 12m processorr memory and 4M I/o memory running ospf process suddenly i can not telnet soemtimes it works sometimes not and when i log to console i find that there is a problem in its memory allocation here are some show comands it might help please advice as soon as possible i want to know is it bug in IOS or memory leakage or harware failure %SYS-2-MALLOCFAIL:memory allocation of 128975 bytes failed from 0x601451F8, pool processor alignment 0 %Low-on-Memory R002#sh ver Cisco Internetwork Operating System Software IOS (tm) 3600 Software (C3640-I-M), Version 11.2(7a)P, SHARED PLATFORM, RELEASE SOFTWARE (fc1) Copyright (c) 1986-1997 by cisco Systems, Inc. Compiled Wed 02-Jul-97 07:24 by ccai Image text-base: 0x600088E0, data-base: 0x6044 ROM: System Bootstrap, Version 11.1(7)AX [kuong (7)AX], EARLY DEPLOYMENT RELEASE SOFTWARE (fc2) R002 uptime is 1 hour, 7 minutes System restarted by power-on System image file is "flash:c3640-i-mz.112-7a.P", booted via flash cisco 3640 (R4700) processor (revision 0x00) with 12288K/4096K bytes of memory. Processor board ID 05691894 R4700 processor, Implementation 33, Revision 1.0 Channelized E1, Version 1.0. Bridging software. X.25 software, Version 2.0, NET2, BFE and GOSIP compliant. Primary Rate ISDN software, Version 1.0. 2 Ethernet/IEEE 802.3 interface(s) 8 Serial network interface(s) 16 Low-speed serial(sync/async) network interface(s) 1 Channelized E1/PRI port(s) DRAM configuration is 64 bits wide with parity disabled. 125K bytes of non-volatile configuration memory. 4096K bytes of processor board System flash (Read/Write) Configuration register is 0x2102 R002#sh processes memory Total: 5984448, Used: 5887396, Free: 97052 PID TTY Allocated FreedHoldingGetbufsRetbufs Process 00 194856 12362121464 0 0 *Init* 00288 300560288 0 0 *Sched* 0062287161719112 14224 297060 0 *Dead* 10256256 3772 0 0 Load Meter 20 1216 80760 7804 0 0 OSPF Hello 30 0 0 6772 0 0 Check heaps 40 73128 0 49660 25736 0 Pool Manager 50256256 6772 0 0 Timers 60172 4068 6944 0 0 ARP Input 70 92 0 6864 0 0 SERIAL A'dete 80256256 6772 0 0 Environmental tatus checker 90 0 0 6772 0 0 SYNCD2430 Hel r 100 26652 3632 24896 0 0 Framer backgr nd 110 43720724 498283114308780 0 IP Input 120 86680 78880 14572 0 0 CDP Protocol 130 0 0 6772 0 0 Asy FS Helper 140 0 19428 6772 0 0 TCP Timer 150 358116 29920 15752 0 0 TCP Protocols 160232 0 7004 0 0 Probe Input 170 92 0 6864 0 0 RARP Input 180 6752 5752 7772 0 0 BOOTP Server 190 4028 11800 12172 0 0 IP Background 220 21500 0 8556 2268 0 Net Background 230348256 12864 0 0 Logger 240 13172256 6772 0 0 TTY Background 250 0820 6772 0 0 Per-Second Jobs 260 31472 31472 7140 0 0 Net Periodic 270184 0 6956 0 0 Net Input 280256256 3772 0 0 Compute load av gs 290 0 0 6772 0 0 Per-minute Jobs 300384256 12900 0 0 HyBridge Input 310584292 7064 0 0 CCP manager 320972256 13488 0 0 PPP manager 330488256 7004 0 0 Multilink PPP 340512256 7028 0 0 Multilink PPP o ut 350 0 0 6772 0 0 OSPF Scanner 36 202 316992 315132 14648 0 0 Virtual Exec 37010824081081676 13504 0 0 IP SNMP 380 84652 73000 11224 6804 0 SNMP Traps 390 0 0 6800 0 0 Spanning Tree 400 1100256 7652 0 0 Tbridge Monitor
Re: urgent problem [7:14942]
Yeah - I've had a similar problem with a 3640 running 11.x code. Up it to 12.0.18 if you can and it should go away. Mine has been rock solid ever since. Other things I noticed, besides the intermitant connectivity, would be that the whole box would hang while you were on it for periods of time. Then it come back like nothing ever happened. All related to memory allocation. Mike Mohammed Saro wrote: > i have router 3640 with 12m processorr memory and 4M I/o memory running ospf > process suddenly i can not telnet soemtimes it works sometimes not and when i > log to console i find that there is a problem in its memory allocation here > are some show comands it might help please advice as soon as possible i want > to know is it bug in IOS or memory leakage or harware failure > > %SYS-2-MALLOCFAIL:memory allocation of 128975 bytes failed from 0x601451F8, > pool processor alignment 0 > > %Low-on-Memory > > > > R002#sh ver > Cisco Internetwork Operating System Software > IOS (tm) 3600 Software (C3640-I-M), Version 11.2(7a)P, SHARED PLATFORM, > RELEASE > SOFTWARE (fc1) > Copyright (c) 1986-1997 by cisco Systems, Inc. > Compiled Wed 02-Jul-97 07:24 by ccai > Image text-base: 0x600088E0, data-base: 0x6044 > > ROM: System Bootstrap, Version 11.1(7)AX [kuong (7)AX], EARLY DEPLOYMENT > RELEASE > SOFTWARE (fc2) > > R002 uptime is 1 hour, 7 minutes > System restarted by power-on > System image file is "flash:c3640-i-mz.112-7a.P", booted via flash > > cisco 3640 (R4700) processor (revision 0x00) with 12288K/4096K bytes of > memory. > Processor board ID 05691894 > R4700 processor, Implementation 33, Revision 1.0 > Channelized E1, Version 1.0. > Bridging software. > X.25 software, Version 2.0, NET2, BFE and GOSIP compliant. > Primary Rate ISDN software, Version 1.0. > 2 Ethernet/IEEE 802.3 interface(s) > 8 Serial network interface(s) > 16 Low-speed serial(sync/async) network interface(s) > 1 Channelized E1/PRI port(s) > DRAM configuration is 64 bits wide with parity disabled. > 125K bytes of non-volatile configuration memory. > 4096K bytes of processor board System flash (Read/Write) > > Configuration register is 0x2102 > > > R002#sh processes memory > Total: 5984448, Used: 5887396, Free: 97052 > PID TTY Allocated FreedHoldingGetbufsRetbufs Process >00 194856 12362121464 0 0 *Init* >00288 300560288 0 0 *Sched* >0062287161719112 14224 297060 0 *Dead* >10256256 3772 0 0 Load Meter >20 1216 80760 7804 0 0 OSPF Hello >30 0 0 6772 0 0 Check heaps >40 73128 0 49660 25736 0 Pool Manager >50256256 6772 0 0 Timers >60172 4068 6944 0 0 ARP Input >70 92 0 6864 0 0 SERIAL > A'dete >80256256 6772 0 0 > Environmental > tatus checker >90 0 0 6772 0 0 SYNCD2430 > Hel > r > 100 26652 3632 24896 0 0 Framer > backgr > nd > 110 43720724 498283114308780 0 IP Input > 120 86680 78880 14572 0 0 CDP Protocol > 130 0 0 6772 0 0 Asy FS > Helper > 140 0 19428 6772 0 0 TCP Timer > 150 358116 29920 15752 0 0 TCP > Protocols > 160232 0 7004 0 0 Probe Input > 170 92 0 6864 0 0 RARP Input > 180 6752 5752 7772 0 0 BOOTP Server > 190 4028 11800 12172 0 0 IP > Background > 220 21500 0 8556 2268 0 Net > Background > 230348256 12864 0 0 Logger > 240 13172256 6772 0 0 TTY > Background > 250 0820 6772 0 0 Per-Second > Jobs > 260 31472 31472 7140 0 0 Net Periodic > 270184 0 6956 0 0 Net Input > 280256256 3772 0 0 Compute load > av > gs > 290 0 0 6772 0 0 Per-minute > Jobs > 300384256 12900 0 0 HyBridge > Input > 310584292 7064 0 0 CCP manager > 320972256 13488 0 0 PPP manager > 330
Re: urgent problem [7:14942]
yeah but i want urgent solution i do not have enough fash or RAM to upgrade IOS and i'm suspecting a hardware failure so how can i know is it hardware failure or just software hangs Best Regards, Mohammed Saro Network Engineer - Original Message - From: "Michael R. Eckhoff" To: Sent: Sunday, August 05, 2001 2:39 PM Subject: Re: urgent problem [7:14942] > Yeah - I've had a similar problem with a 3640 running 11.x code. Up it > to 12.0.18 if you can and it should go away. Mine has been rock solid > ever since. > > Other things I noticed, besides the intermitant connectivity, would be > that the whole box would hang while you were on it for periods of time. > Then it come back like nothing ever happened. > > All related to memory allocation. > > Mike > > Mohammed Saro wrote: > > i have router 3640 with 12m processorr memory and 4M I/o memory running > ospf > > process suddenly i can not telnet soemtimes it works sometimes not and > when i > > log to console i find that there is a problem in its memory allocation here > > are some show comands it might help please advice as soon as possible i > want > > to know is it bug in IOS or memory leakage or harware failure > > > > %SYS-2-MALLOCFAIL:memory allocation of 128975 bytes failed from 0x601451F8, > > pool processor alignment 0 > > > > %Low-on-Memory > > > > > > > > R002#sh ver > > Cisco Internetwork Operating System Software > > IOS (tm) 3600 Software (C3640-I-M), Version 11.2(7a)P, SHARED PLATFORM, > > RELEASE > > SOFTWARE (fc1) > > Copyright (c) 1986-1997 by cisco Systems, Inc. > > Compiled Wed 02-Jul-97 07:24 by ccai > > Image text-base: 0x600088E0, data-base: 0x6044 > > > > ROM: System Bootstrap, Version 11.1(7)AX [kuong (7)AX], EARLY DEPLOYMENT > > RELEASE > > SOFTWARE (fc2) > > > > R002 uptime is 1 hour, 7 minutes > > System restarted by power-on > > System image file is "flash:c3640-i-mz.112-7a.P", booted via flash > > > > cisco 3640 (R4700) processor (revision 0x00) with 12288K/4096K bytes of > > memory. > > Processor board ID 05691894 > > R4700 processor, Implementation 33, Revision 1.0 > > Channelized E1, Version 1.0. > > Bridging software. > > X.25 software, Version 2.0, NET2, BFE and GOSIP compliant. > > Primary Rate ISDN software, Version 1.0. > > 2 Ethernet/IEEE 802.3 interface(s) > > 8 Serial network interface(s) > > 16 Low-speed serial(sync/async) network interface(s) > > 1 Channelized E1/PRI port(s) > > DRAM configuration is 64 bits wide with parity disabled. > > 125K bytes of non-volatile configuration memory. > > 4096K bytes of processor board System flash (Read/Write) > > > > Configuration register is 0x2102 > > > > > > R002#sh processes memory > > Total: 5984448, Used: 5887396, Free: 97052 > > PID TTY Allocated FreedHoldingGetbufsRetbufs Process > >00 194856 12362121464 0 0 *Init* > >00288 300560288 0 0 *Sched* > >0062287161719112 14224 297060 0 *Dead* > >10256256 3772 0 0 Load Meter > >20 1216 80760 7804 0 0 OSPF Hello > >30 0 0 6772 0 0 Check > heaps > >40 73128 0 49660 25736 0 Pool > Manager > >50256256 6772 0 0 Timers > >60172 4068 6944 0 0 ARP Input > >70 92 0 6864 0 0 SERIAL > > A'dete > >80256256 6772 0 0 > > Environmental > > tatus checker > >90 0 0 6772 0 0 SYNCD2430 > > Hel > > r > > 100 26652 3632 24896 0 0 Framer > > backgr > > nd > > 110 43720724 498283114308780 0 IP Input > > 120 86680 78880 14572 0 0 CDP > Protocol > > 130 0 0 6772 0 0 Asy FS > > Helper > > 140 0 19428 6772 0 0 TCP Timer > > 150 358116 29920 15752 0 0 TCP > > Protocols > > 160232 0 7004 0 0 Probe > Input > > 170
Re: urgent problem [7:14942]
Generaly, memory leaks are not hardware related. They're code related. Either way, the only real way of isolating them is process of elimination. Which means, turn down extra services, update OS, etc. Since I doubt you can turn down any extra services without impacting your users, upgrading to see if it goes away is the next simplest without impacting everyone (other than for the reload of course). You could also just search CCO or call TAC and see if there is a known issue with that revision of code as well. Mike Mohammed Saro wrote: > yeah but i want urgent solution i do not have enough fash or RAM to upgrade > IOS and i'm suspecting a hardware failure so how can i know is it hardware > failure or just software hangs > > > > Best Regards, > Mohammed Saro > Network Engineer > > - Original Message - > From: "Michael R. Eckhoff" > To: > Sent: Sunday, August 05, 2001 2:39 PM > Subject: Re: urgent problem [7:14942] > > > >>Yeah - I've had a similar problem with a 3640 running 11.x code. Up it >>to 12.0.18 if you can and it should go away. Mine has been rock solid >>ever since. >> >>Other things I noticed, besides the intermitant connectivity, would be >>that the whole box would hang while you were on it for periods of time. >> Then it come back like nothing ever happened. >> >>All related to memory allocation. >> >>Mike >> >>Mohammed Saro wrote: >> >>>i have router 3640 with 12m processorr memory and 4M I/o memory running >>> >>ospf >> >>>process suddenly i can not telnet soemtimes it works sometimes not and >>> >>when i >> >>>log to console i find that there is a problem in its memory allocation >>> > here > >>>are some show comands it might help please advice as soon as possible i >>> >>want >> >>>to know is it bug in IOS or memory leakage or harware failure >>> >>>%SYS-2-MALLOCFAIL:memory allocation of 128975 bytes failed from >>> > 0x601451F8, > >>>pool processor alignment 0 >>> >>>%Low-on-Memory >>> >>> >>> >>>R002#sh ver >>>Cisco Internetwork Operating System Software >>>IOS (tm) 3600 Software (C3640-I-M), Version 11.2(7a)P, SHARED PLATFORM, >>>RELEASE >>>SOFTWARE (fc1) >>>Copyright (c) 1986-1997 by cisco Systems, Inc. >>>Compiled Wed 02-Jul-97 07:24 by ccai >>>Image text-base: 0x600088E0, data-base: 0x6044 >>> >>>ROM: System Bootstrap, Version 11.1(7)AX [kuong (7)AX], EARLY DEPLOYMENT >>>RELEASE >>> SOFTWARE (fc2) >>> >>>R002 uptime is 1 hour, 7 minutes >>>System restarted by power-on >>>System image file is "flash:c3640-i-mz.112-7a.P", booted via flash >>> >>>cisco 3640 (R4700) processor (revision 0x00) with 12288K/4096K bytes of >>>memory. >>>Processor board ID 05691894 >>>R4700 processor, Implementation 33, Revision 1.0 >>>Channelized E1, Version 1.0. >>>Bridging software. >>>X.25 software, Version 2.0, NET2, BFE and GOSIP compliant. >>>Primary Rate ISDN software, Version 1.0. >>>2 Ethernet/IEEE 802.3 interface(s) >>>8 Serial network interface(s) >>>16 Low-speed serial(sync/async) network interface(s) >>>1 Channelized E1/PRI port(s) >>>DRAM configuration is 64 bits wide with parity disabled. >>>125K bytes of non-volatile configuration memory. >>>4096K bytes of processor board System flash (Read/Write) >>> >>>Configuration register is 0x2102 >>> >>> >>>R002#sh processes memory >>>Total: 5984448, Used: 5887396, Free: 97052 >>> PID TTY Allocated FreedHoldingGetbufsRetbufs Process >>> 00 194856 12362121464 0 0 *Init* >>> 00288 300560288 0 0 *Sched* >>> 0062287161719112 14224 297060 0 *Dead* >>> 10256256 3772 0 0 Load >>> > Meter > >>> 20 1216 80760 7804 0 0 OSPF >>> > Hello > >>> 30 0 0 6772 0 0 Check >>> >>heaps >> >>> 40 73128 0 49660 25736 0 Pool >>> >>Manager >> >>> 50256256 6772 0 0 Timers >>> 60172 4068 694
Re: urgent problem [7:14942]
It appears that 16MB of RAM just isn't enough for your interface requirements. During you "sh mem" it states that 97K is free, the MALLOC is for 128K. This 128K of memory must be in a single contiguous chunk. The largest chunk of memory is 33K. The number of interfaces on the router is rather high, for the amount of memory installed. You need to take a look at the 2600/3600 Memory Calculator: http://www.cisco.com/cgi-bin/Support/MemCalc/mem-calc.pl I took a look and attempted to configure a 3640 with the Interfaces describe in the "sh ver" and got 24MB of DRAM required. David C Prall [EMAIL PROTECTED] http://dcp.dcptech.com - Original Message - From: "Mohammed Saro" To: Sent: Sunday, August 05, 2001 7:30 AM Subject: urgent problem [7:14942] > i have router 3640 with 12m processorr memory and 4M I/o memory running ospf > process suddenly i can not telnet soemtimes it works sometimes not and when i > log to console i find that there is a problem in its memory allocation here > are some show comands it might help please advice as soon as possible i want > to know is it bug in IOS or memory leakage or harware failure > > %SYS-2-MALLOCFAIL:memory allocation of 128975 bytes failed from 0x601451F8, > pool processor alignment 0 > > %Low-on-Memory > > > > R002#sh ver > Cisco Internetwork Operating System Software > IOS (tm) 3600 Software (C3640-I-M), Version 11.2(7a)P, SHARED PLATFORM, > RELEASE > SOFTWARE (fc1) > Copyright (c) 1986-1997 by cisco Systems, Inc. > Compiled Wed 02-Jul-97 07:24 by ccai > Image text-base: 0x600088E0, data-base: 0x6044 > > ROM: System Bootstrap, Version 11.1(7)AX [kuong (7)AX], EARLY DEPLOYMENT > RELEASE > SOFTWARE (fc2) > > R002 uptime is 1 hour, 7 minutes > System restarted by power-on > System image file is "flash:c3640-i-mz.112-7a.P", booted via flash > > cisco 3640 (R4700) processor (revision 0x00) with 12288K/4096K bytes of > memory. > Processor board ID 05691894 > R4700 processor, Implementation 33, Revision 1.0 > Channelized E1, Version 1.0. > Bridging software. > X.25 software, Version 2.0, NET2, BFE and GOSIP compliant. > Primary Rate ISDN software, Version 1.0. > 2 Ethernet/IEEE 802.3 interface(s) > 8 Serial network interface(s) > 16 Low-speed serial(sync/async) network interface(s) > 1 Channelized E1/PRI port(s) > DRAM configuration is 64 bits wide with parity disabled. > 125K bytes of non-volatile configuration memory. > 4096K bytes of processor board System flash (Read/Write) > > Configuration register is 0x2102 > > > R002#sh processes memory > Total: 5984448, Used: 5887396, Free: 97052 > PID TTY Allocated FreedHoldingGetbufsRetbufs Process >00 194856 12362121464 0 0 *Init* >00288 300560288 0 0 *Sched* >0062287161719112 14224 297060 0 *Dead* >10256256 3772 0 0 Load Meter >20 1216 80760 7804 0 0 OSPF Hello >30 0 0 6772 0 0 Check heaps >40 73128 0 49660 25736 0 Pool Manager >50256256 6772 0 0 Timers >60172 4068 6944 0 0 ARP Input >70 92 0 6864 0 0 SERIAL > A'dete >80256256 6772 0 0 > Environmental > tatus checker >90 0 0 6772 0 0 SYNCD2430 > Hel > r > 100 26652 3632 24896 0 0 Framer > backgr > nd > 110 43720724 498283114308780 0 IP Input > 120 86680 78880 14572 0 0 CDP Protocol > 130 0 0 6772 0 0 Asy FS > Helper > 140 0 19428 6772 0 0 TCP Timer > 150 358116 29920 15752 0 0 TCP > Protocols > 160232 0 7004 0 0 Probe Input > 170 92 0 6864 0 0 RARP Input > 180 6752 5752 7772 0 0 BOOTP Server > 190 4028 11800 12172 0 0 IP > Background > 220 21500 0 8556 2268 0 Net > Background > 230348256 12864 0 0 Logger > 240 13172256 6772 0 0 TTY > Background > 25