[asterisk-users] Asterisk spontaneous reboot
Hello, Today I restart the MySQL-DB (/sbin/service mysqld restart) and I could no longer connect to asterisk (/usr/sbin/asterisk -r) for a few seconds. There is now a core dump present in /tmp : -rw--- 1 root root 88M Aug 26 08:07 core.sip.pbx.tld-2011-08-26T08:07:35+0200 How can I get usefull information about what went wrong ? Because a spontaneous reboot of Asterisk has never happened before when just restarting the MySQL-deamon. Kind regards, Jonas. -- _ -- Bandwidth and Colocation Provided by http://www.api-digital.com -- New to Asterisk? Join us for a live introductory webinar every Thurs: http://www.asterisk.org/hello asterisk-users mailing list To UNSUBSCRIBE or update options visit: http://lists.digium.com/mailman/listinfo/asterisk-users
Re: [asterisk-users] Asterisk spontaneous reboot
use gdb (The GNU Project Debugger) to take a look into the core dump gdb asterisk core.sip.pbx.tld-2011-08-26T08:07:35+0200 Kristijan 2011/8/26 Jonas Kellens jonas.kell...@telenet.be: Hello, Today I restart the MySQL-DB (/sbin/service mysqld restart) and I could no longer connect to asterisk (/usr/sbin/asterisk -r) for a few seconds. There is now a core dump present in /tmp : -rw--- 1 root root 88M Aug 26 08:07 core.sip.pbx.tld-2011-08-26T08:07:35+0200 How can I get usefull information about what went wrong ? Because a spontaneous reboot of Asterisk has never happened before when just restarting the MySQL-deamon. Kind regards, Jonas. -- _ -- Bandwidth and Colocation Provided by http://www.api-digital.com -- New to Asterisk? Join us for a live introductory webinar every Thurs: http://www.asterisk.org/hello asterisk-users mailing list To UNSUBSCRIBE or update options visit: http://lists.digium.com/mailman/listinfo/asterisk-users -- _ -- Bandwidth and Colocation Provided by http://www.api-digital.com -- New to Asterisk? Join us for a live introductory webinar every Thurs: http://www.asterisk.org/hello asterisk-users mailing list To UNSUBSCRIBE or update options visit: http://lists.digium.com/mailman/listinfo/asterisk-users
Re: [asterisk-users] Asterisk spontaneous reboot
On 11/06/2010 09:18 PM, Sherwood McGowan wrote:
On Sat, Nov 6, 2010 at 2:45 PM, Jonas Kellensjonas.kell...@telenet.be
wrote:
On 11/06/2010 07:18 PM, Tilghman Lesher wrote:
On Saturday 06 November 2010 11:22:06 Jonas Kellens wrote:
Hello,
I just experienced a spontaneous reboot of Asterisk. This is my log file
/var/log/messages :
Nov 6 16:37:37 vps kernel: miniserv.pl invoked oom-killer:
First line. Your miniserv.pl allocated more memory than is allocated to
the system, so the dreaded OOM killer came into play and killed a selected
process. Have you considered enabling swap memory?
I have 512 MB real RAM and 1024 of swap.
bash-3.2# cat /proc/meminfo
MemTotal: 524288 kB
MemFree: 23760 kB
Buffers: 28564 kB
Cached: 348668 kB
SwapCached: 6536 kB
Active: 193972 kB
Inactive: 231216 kB
HighTotal: 0 kB
HighFree:0 kB
LowTotal: 524288 kB
LowFree: 23760 kB
SwapTotal: 1048568 kB
SwapFree: 949456 kB
Dirty: 768 kB
Writeback: 0 kB
AnonPages: 46652 kB
Mapped: 16884 kB
Slab:21000 kB
PageTables: 8084 kB
NFS_Unstable:0 kB
Bounce: 0 kB
CommitLimit: 1310712 kB
Committed_AS: 321288 kB
VmallocTotal: 34359738367 kB
VmallocUsed: 784 kB
VmallocChunk: 34359737535 kB
miniserv.pl... I have webmin running yes and it was stopped after the
restart of Asterisk...
So the bad one in this story is WebMin that was eating up all the memory ?
Jonas.
--
_
-- Bandwidth and Colocation Provided by http://www.api-digital.com --
New to Asterisk? Join us for a live introductory webinar every Thurs:
http://www.asterisk.org/hello
asterisk-users mailing list
To UNSUBSCRIBE or update options visit:
http://lists.digium.com/mailman/listinfo/asterisk-users
Yessir, that's the culprit in this case
Strange, today I saw this in the logs :
Nov 7 17:02:18 vps kernel: crond invoked oom-killer: gfp_mask=0x201d2,
order=0, oomkilladj=0
Nov 7 17:02:18 vps kernel:
Nov 7 17:02:18 vps kernel: Call Trace:
Nov 7 17:02:18 vps kernel: [802bf74e] out_of_memory+0x8b/0x203
Nov 7 17:02:18 vps kernel: [8020f947] __alloc_pages+0x27f/0x308
Nov 7 17:02:18 vps kernel: [802138db]
__do_page_cache_readahead+0xc6/0x1ab
Nov 7 17:02:18 vps kernel: [802141c7] filemap_nopage+0x14c/0x360
Nov 7 17:02:18 vps kernel: [80208e8c]
__handle_mm_fault+0x442/0x1445
Nov 7 17:02:18 vps kernel: [8028866d] deactivate_task+0x28/0x5f
Nov 7 17:02:18 vps kernel: [8026769a] do_page_fault+0xf7b/0x12e0
Nov 7 17:02:18 vps kernel: [8025c8ff] hrtimer_cancel+0xc/0x16
Nov 7 17:02:18 vps kernel: [80263b14] do_nanosleep+0x47/0x70
Nov 7 17:02:18 vps kernel: [8025c7ec]
hrtimer_nanosleep+0x58/0x118
Nov 7 17:02:18 vps kernel: [8026082b] error_exit+0x0/0x6e
Nov 7 17:02:18 vps kernel:
Nov 7 17:02:18 vps kernel: Mem-info:
snip
So this time it is crond that invoked oom-killer...
I've had this since I commented out this in /etc/asterisk/logger.conf :
exec_after_rotate=gzip -9 ${filename}.2
Whenever I do a logger rotate on the Asterisk CLI, the CLI hangs...
Kind regards,
Jonas.
--
_
-- Bandwidth and Colocation Provided by http://www.api-digital.com --
New to Asterisk? Join us for a live introductory webinar every Thurs:
http://www.asterisk.org/hello
asterisk-users mailing list
To UNSUBSCRIBE or update options visit:
http://lists.digium.com/mailman/listinfo/asterisk-users
Re: [asterisk-users] Asterisk spontaneous reboot
On Sun, Nov 7, 2010 at 3:58 PM, Jonas Kellens jonas.kell...@telenet.be wrote: On 11/06/2010 09:18 PM, Sherwood McGowan wrote: On Sat, Nov 6, 2010 at 2:45 PM, Jonas Kellensjonas.kell...@telenet.be wrote: On 11/06/2010 07:18 PM, Tilghman Lesher wrote: On Saturday 06 November 2010 11:22:06 Jonas Kellens wrote: Hello, I just experienced a spontaneous reboot of Asterisk. This is my log file /var/log/messages : Nov 6 16:37:37 vps kernel: miniserv.pl invoked oom-killer: First line. Your miniserv.pl allocated more memory than is allocated to the system, so the dreaded OOM killer came into play and killed a selected process. Have you considered enabling swap memory? I have 512 MB real RAM and 1024 of swap. bash-3.2# cat /proc/meminfo MemTotal: 524288 kB MemFree: 23760 kB Buffers: 28564 kB Cached: 348668 kB SwapCached: 6536 kB Active: 193972 kB Inactive: 231216 kB HighTotal: 0 kB HighFree: 0 kB LowTotal: 524288 kB LowFree: 23760 kB SwapTotal: 1048568 kB SwapFree: 949456 kB Dirty: 768 kB Writeback: 0 kB AnonPages: 46652 kB Mapped: 16884 kB Slab: 21000 kB PageTables: 8084 kB NFS_Unstable: 0 kB Bounce: 0 kB CommitLimit: 1310712 kB Committed_AS: 321288 kB VmallocTotal: 34359738367 kB VmallocUsed: 784 kB VmallocChunk: 34359737535 kB miniserv.pl... I have webmin running yes and it was stopped after the restart of Asterisk... So the bad one in this story is WebMin that was eating up all the memory ? Jonas. -- _ -- Bandwidth and Colocation Provided by http://www.api-digital.com -- New to Asterisk? Join us for a live introductory webinar every Thurs: http://www.asterisk.org/hello asterisk-users mailing list To UNSUBSCRIBE or update options visit: http://lists.digium.com/mailman/listinfo/asterisk-users Yessir, that's the culprit in this case Strange, today I saw this in the logs : Nov 7 17:02:18 vps kernel: crond invoked oom-killer: gfp_mask=0x201d2, order=0, oomkilladj=0 Nov 7 17:02:18 vps kernel: Nov 7 17:02:18 vps kernel: Call Trace: Nov 7 17:02:18 vps kernel: [802bf74e] out_of_memory+0x8b/0x203 Nov 7 17:02:18 vps kernel: [8020f947] __alloc_pages+0x27f/0x308 Nov 7 17:02:18 vps kernel: [802138db] __do_page_cache_readahead+0xc6/0x1ab Nov 7 17:02:18 vps kernel: [802141c7] filemap_nopage+0x14c/0x360 Nov 7 17:02:18 vps kernel: [80208e8c] __handle_mm_fault+0x442/0x1445 Nov 7 17:02:18 vps kernel: [8028866d] deactivate_task+0x28/0x5f Nov 7 17:02:18 vps kernel: [8026769a] do_page_fault+0xf7b/0x12e0 Nov 7 17:02:18 vps kernel: [8025c8ff] hrtimer_cancel+0xc/0x16 Nov 7 17:02:18 vps kernel: [80263b14] do_nanosleep+0x47/0x70 Nov 7 17:02:18 vps kernel: [8025c7ec] hrtimer_nanosleep+0x58/0x118 Nov 7 17:02:18 vps kernel: [8026082b] error_exit+0x0/0x6e Nov 7 17:02:18 vps kernel: Nov 7 17:02:18 vps kernel: Mem-info: snip So this time it is crond that invoked oom-killer... Please read up on how the oom killer works. crond didn't invoke anything, but was rather the unfortunate task chosen to be sacrificed. -- _ -- Bandwidth and Colocation Provided by http://www.api-digital.com -- New to Asterisk? Join us for a live introductory webinar every Thurs: http://www.asterisk.org/hello asterisk-users mailing list To UNSUBSCRIBE or update options visit: http://lists.digium.com/mailman/listinfo/asterisk-users
[asterisk-users] Asterisk spontaneous reboot
Hello, I just experienced a spontaneous reboot of Asterisk. This is my log file /var/log/messages : Nov 6 16:37:37 vps2301 kernel: miniserv.pl invoked oom-killer: gfp_mask=0x201d2, order=0, oomkilladj=0 Nov 6 16:37:37 vps2301 kernel: Nov 6 16:37:37 vps2301 kernel: Call Trace: Nov 6 16:37:37 vps2301 kernel: [802bf74e] out_of_memory+0x8b/0x203 Nov 6 16:37:37 vps2301 kernel: [8029c403] autoremove_wake_function+0x0/0x2e Nov 6 16:37:37 vps2301 kernel: [8020f947] __alloc_pages+0x27f/0x308 Nov 6 16:37:37 vps2301 kernel: [802138db] __do_page_cache_readahead+0xc6/0x1ab Nov 6 16:37:37 vps2301 kernel: [802141c7] filemap_nopage+0x14c/0x360 Nov 6 16:37:37 vps2301 kernel: [80208e8c] __handle_mm_fault+0x442/0x1445 Nov 6 16:37:37 vps2301 kernel: [8026769a] do_page_fault+0xf7b/0x12e0 Nov 6 16:37:37 vps2301 kernel: [8029c403] autoremove_wake_function+0x0/0x2e Nov 6 16:37:37 vps2301 kernel: [8020cc84] _atomic_dec_and_lock+0x39/0x57 Nov 6 16:37:37 vps2301 kernel: [8020da9c] dput+0x84/0x114 Nov 6 16:37:37 vps2301 kernel: [8026082b] error_exit+0x0/0x6e Nov 6 16:37:37 vps2301 kernel: Nov 6 16:37:37 vps2301 kernel: Mem-info: Nov 6 16:37:37 vps2301 kernel: DMA per-cpu: Nov 6 16:37:37 vps2301 kernel: cpu 0 hot: high 186, batch 31 used:14 Nov 6 16:37:37 vps2301 kernel: cpu 0 cold: high 62, batch 15 used:7 Nov 6 16:37:37 vps2301 kernel: DMA32 per-cpu: empty Nov 6 16:37:37 vps2301 kernel: Normal per-cpu: empty Nov 6 16:37:38 vps2301 kernel: HighMem per-cpu: empty Nov 6 16:37:38 vps2301 kernel: Free pages:3896kB (0kB HighMem) Nov 6 16:37:38 vps2301 kernel: Active:56596 inactive:56534 dirty:0 writeback:0 unstable:0 free:974 slab:2557 mapped-file:1000 mapped-anon:112059 pagetables :2615 Nov 6 16:37:38 vps2301 kernel: DMA free:3896kB min:4108kB low:5132kB high:6160kB active:226384kB inactive:226136kB present:1056768kB pages_scanned:721434 a ll_unreclaimable? yes Nov 6 16:37:38 vps2301 kernel: lowmem_reserve[]: 0 0 0 0 Nov 6 16:37:38 vps2301 kernel: DMA32 free:0kB min:0kB low:0kB high:0kB active:0kB inactive:0kB present:0kB pages_scanned:0 all_unreclaimable? no Nov 6 16:37:38 vps2301 kernel: lowmem_reserve[]: 0 0 0 0 Nov 6 16:37:38 vps2301 kernel: Normal free:0kB min:0kB low:0kB high:0kB active:0kB inactive:0kB present:0kB pages_scanned:0 all_unreclaimable? no Nov 6 16:37:38 vps2301 kernel: lowmem_reserve[]: 0 0 0 0 Nov 6 16:37:38 vps2301 kernel: HighMem free:0kB min:128kB low:128kB high:128kB active:0kB inactive:0kB present:0kB pages_scanned:0 all_unreclaimable? no Nov 6 16:37:38 vps2301 kernel: lowmem_reserve[]: 0 0 0 0 Nov 6 16:37:38 vps2301 kernel: DMA: 160*4kB 37*8kB 1*16kB 0*32kB 0*64kB 1*128kB 1*256kB 1*512kB 0*1024kB 1*2048kB 0*4096kB = 3896kB Nov 6 16:37:38 vps2301 kernel: DMA32: empty Nov 6 16:37:38 vps2301 kernel: Normal: empty Nov 6 16:37:38 vps2301 kernel: HighMem: empty Nov 6 16:37:38 vps2301 kernel: 1358 pagecache pages Nov 6 16:37:38 vps2301 kernel: Swap cache: add 1266970, delete 1266715, find 2278772/2425760, race 0+63 Nov 6 16:37:38 vps2301 kernel: Free swap = 0kB Nov 6 16:37:38 vps2301 kernel: Total swap = 1048568kB Nov 6 16:37:38 vps2301 kernel: Free swap:0kB Nov 6 16:37:38 vps2301 kernel: 264192 pages of RAM Nov 6 16:37:38 vps2301 kernel: 8332 reserved pages Nov 6 16:37:38 vps2301 kernel: 4839 pages shared Nov 6 16:37:38 vps2301 kernel: 255 pages swap cached Nov 6 16:37:38 vps2301 kernel: Out of memory: Killed process 2606, UID 0, (asterisk). Nov 6 16:40:45 vps2301 ntpd[1033]: synchronized to 88.159.82.127, stratum 2 Does the Asterisk proces gets out of memory ?! Kind regards, Jonas. -- _ -- Bandwidth and Colocation Provided by http://www.api-digital.com -- New to Asterisk? Join us for a live introductory webinar every Thurs: http://www.asterisk.org/hello asterisk-users mailing list To UNSUBSCRIBE or update options visit: http://lists.digium.com/mailman/listinfo/asterisk-users
Re: [asterisk-users] Asterisk spontaneous reboot
On 11/06/2010 07:18 PM, Tilghman Lesher wrote: On Saturday 06 November 2010 11:22:06 Jonas Kellens wrote: Hello, I just experienced a spontaneous reboot of Asterisk. This is my log file /var/log/messages : Nov 6 16:37:37 vps2301 kernel: miniserv.pl invoked oom-killer: First line. Your miniserv.pl allocated more memory than is allocated to the system, so the dreaded OOM killer came into play and killed a selected process. Have you considered enabling swap memory? I have 512 MB real RAM and 1024 of swap. bash-3.2# cat /proc/meminfo MemTotal: 524288 kB MemFree: 23760 kB Buffers: 28564 kB Cached: 348668 kB SwapCached: 6536 kB Active: 193972 kB Inactive: 231216 kB HighTotal: 0 kB HighFree:0 kB LowTotal: 524288 kB LowFree: 23760 kB SwapTotal: 1048568 kB SwapFree: 949456 kB Dirty: 768 kB Writeback: 0 kB AnonPages: 46652 kB Mapped: 16884 kB Slab:21000 kB PageTables: 8084 kB NFS_Unstable:0 kB Bounce: 0 kB CommitLimit: 1310712 kB Committed_AS: 321288 kB VmallocTotal: 34359738367 kB VmallocUsed: 784 kB VmallocChunk: 34359737535 kB miniserv.pl... I have webmin running yes and it was stopped after the restart of Asterisk... So the bad one in this story is WebMin that was eating up all the memory ? Jonas. -- _ -- Bandwidth and Colocation Provided by http://www.api-digital.com -- New to Asterisk? Join us for a live introductory webinar every Thurs: http://www.asterisk.org/hello asterisk-users mailing list To UNSUBSCRIBE or update options visit: http://lists.digium.com/mailman/listinfo/asterisk-users
Re: [asterisk-users] Asterisk spontaneous reboot
On Sat, Nov 6, 2010 at 2:45 PM, Jonas Kellens jonas.kell...@telenet.be wrote: On 11/06/2010 07:18 PM, Tilghman Lesher wrote: On Saturday 06 November 2010 11:22:06 Jonas Kellens wrote: Hello, I just experienced a spontaneous reboot of Asterisk. This is my log file /var/log/messages : Nov 6 16:37:37 vps2301 kernel: miniserv.pl invoked oom-killer: First line. Your miniserv.pl allocated more memory than is allocated to the system, so the dreaded OOM killer came into play and killed a selected process. Have you considered enabling swap memory? I have 512 MB real RAM and 1024 of swap. bash-3.2# cat /proc/meminfo MemTotal: 524288 kB MemFree: 23760 kB Buffers: 28564 kB Cached: 348668 kB SwapCached: 6536 kB Active: 193972 kB Inactive: 231216 kB HighTotal: 0 kB HighFree: 0 kB LowTotal: 524288 kB LowFree: 23760 kB SwapTotal: 1048568 kB SwapFree: 949456 kB Dirty: 768 kB Writeback: 0 kB AnonPages: 46652 kB Mapped: 16884 kB Slab: 21000 kB PageTables: 8084 kB NFS_Unstable: 0 kB Bounce: 0 kB CommitLimit: 1310712 kB Committed_AS: 321288 kB VmallocTotal: 34359738367 kB VmallocUsed: 784 kB VmallocChunk: 34359737535 kB miniserv.pl... I have webmin running yes and it was stopped after the restart of Asterisk... So the bad one in this story is WebMin that was eating up all the memory ? Jonas. -- _ -- Bandwidth and Colocation Provided by http://www.api-digital.com -- New to Asterisk? Join us for a live introductory webinar every Thurs: http://www.asterisk.org/hello asterisk-users mailing list To UNSUBSCRIBE or update options visit: http://lists.digium.com/mailman/listinfo/asterisk-users Yessir, that's the culprit in this case -- _ -- Bandwidth and Colocation Provided by http://www.api-digital.com -- New to Asterisk? Join us for a live introductory webinar every Thurs: http://www.asterisk.org/hello asterisk-users mailing list To UNSUBSCRIBE or update options visit: http://lists.digium.com/mailman/listinfo/asterisk-users
