am urmatorul dmesg (see below) Q: nu inteleg de ce s-a invocat oom cand era swap disponibil suficient de mult (300Mb consumati din 4GB? Am citit http://serverfault.com/questions/141988/avoid-linux-out-of-memory-application-teardownca abordare generala insa poate ma lumineaza si pe mine cineva cum s-a ajuns la oom in cazul concret de mai jos, si anume cum a decis kernelul ca e mai bine sa termine un proces in loc sa-l arunce in swap?
daca era doar cazul de mai sus era ok, insa a inceput sa taie in stanga si-n dreapta in nevinovati gen irqbalance sau getty sau atd pana cand sistemul a devenit unresponsive (login pe consola dadea timeout, serviciile raspundeau dupa foarte mult timp sau deloc/timeout). root@mailhost:~# grep Killed /var/log/syslog Oct 3 03:58:59 mailhost kernel: [2030572.242524] Killed process 1941 (bdscand) total-vm:777136kB, anon-rss:56kB, file-rss:0kB Oct 3 03:58:59 mailhost kernel: [2030573.483461] Killed process 5695 (apache2) total-vm:266560kB, anon-rss:0kB, file-rss:0kB Oct 3 03:59:00 mailhost kernel: [2030574.239418] Killed process 5687 (apache2) total-vm:266076kB, anon-rss:0kB, file-rss:0kB Oct 3 07:56:08 0f kernel: [2043733.645708] Killed process 562 (rsyslogd) total-vm:249480kB, anon-rss:0kB, file-rss:0kB Oct 3 07:56:08 0f kernel: [2043736.253973] Killed process 865 (upstart-socket-) total-vm:15188kB, anon-rss:0kB, file-rss:0kB Oct 3 07:56:08 0f kernel: [2043747.850336] Killed process 1094 (getty) total-vm:15784kB, anon-rss:0kB, file-rss:4kB Oct 3 07:56:08 0f kernel: [2043758.258872] Killed process 1098 (getty) total-vm:15784kB, anon-rss:0kB, file-rss:4kB Oct 3 07:56:08 0f kernel: [2043773.042769] Killed process 1102 (getty) total-vm:15784kB, anon-rss:0kB, file-rss:4kB Oct 3 07:56:08 0f kernel: [2043778.861222] Killed process 1103 (getty) total-vm:15784kB, anon-rss:0kB, file-rss:4kB Oct 3 07:56:08 0f kernel: [2043792.617732] Killed process 1105 (getty) total-vm:15784kB, anon-rss:0kB, file-rss:4kB Oct 3 07:56:08 0f kernel: [2043797.324890] Killed process 1124 (atd) total-vm:16908kB, anon-rss:0kB, file-rss:0kB Oct 3 07:56:08 0f kernel: [2043821.025732] Killed process 1132 (irqbalance) total-vm:15980kB, anon-rss:0kB, file-rss:0kB parametri sysctl (toti sunt la valorile implicite mai putin ce e in sysctld.d/*) toti parametrii: http://paste.ubuntu.com/6187537/ cei modificati: root@mailhost:~# cat /etc/sysctl.d/*|grep -v ^#|grep = kernel.printk = 4 4 1 7 net.ipv6.conf.all.use_tempaddr = 2 net.ipv6.conf.default.use_tempaddr = 2 kernel.kptr_restrict = 1 net.ipv4.conf.default.rp_filter=1 net.ipv4.conf.all.rp_filter=1 net.ipv4.tcp_syncookies=1 kernel.yama.ptrace_scope = 1 vm.mmap_min_addr = 65536 net.core.wmem_max=1048576 net.core.rmem_max=1048576 net.core.wmem_default=1048576 net.core.rmem_default=1048576 net.ipv4.tcp_mem=1048576 net.ipv4.tcp_rmem=1048576 net.ipv4.tcp_wmem=1048576 kernel.hung_task_timeout_secs=15 dmesg: Oct 3 03:58:53 mailhost kernel: [2030572.236904] irqbalance invoked oom-killer: gfp_mask=0x40d0, order=2, oom_adj=0, oom_score_adj=0 Oct 3 03:58:54 mailhost kernel: [2030572.236908] irqbalance cpuset=/ mems_allowed=0 Oct 3 03:58:56 mailhost kernel: [2030572.236911] Pid: 1132, comm: irqbalance Tainted: P O 3.2.0-53-generic #81-Ubuntu Oct 3 03:58:56 mailhost kernel: [2030572.236913] Call Trace: Oct 3 03:58:56 mailhost kernel: [2030572.236920] [<ffffffff8111c8e1>] dump_header+0x91/0xe0 Oct 3 03:58:56 mailhost kernel: [2030572.236923] [<ffffffff8111cc65>] oom_kill_process+0x85/0xb0 Oct 3 03:58:56 mailhost kernel: [2030572.236926] [<ffffffff8111d00a>] out_of_memory+0xfa/0x220 Oct 3 03:58:56 mailhost kernel: [2030572.236930] [<ffffffff81122a2c>] __alloc_pages_nodemask+0x8dc/0x8f0 Oct 3 03:58:56 mailhost kernel: [2030572.236934] [<ffffffff81118e00>] ? sleep_on_page+0x20/0x20 Oct 3 03:58:56 mailhost kernel: [2030572.236940] [<ffffffff81159cd6>] alloc_pages_current+0xb6/0x120 Oct 3 03:58:56 mailhost kernel: [2030572.236943] [<ffffffff8111dabe>] __get_free_pages+0xe/0x40 Oct 3 03:58:56 mailhost kernel: [2030572.236946] [<ffffffff811646df>] kmalloc_order_trace+0x3f/0xd0 Oct 3 03:58:56 mailhost kernel: [2030572.236950] [<ffffffff81165255>] __kmalloc+0x185/0x190 Oct 3 03:58:56 mailhost kernel: [2030572.236954] [<ffffffff8129fcbb>] ? security_d_instantiate+0x1b/0x30 Oct 3 03:58:56 mailhost kernel: [2030572.236958] [<ffffffff811e30c2>] stat_open+0x52/0xb0 Oct 3 03:58:56 mailhost kernel: [2030572.236961] [<ffffffff811d9b40>] proc_reg_open+0xb0/0x190 Oct 3 03:58:56 mailhost kernel: [2030572.236964] [<ffffffff8119c410>] ? seq_release_private+0x50/0x50 Oct 3 03:58:56 mailhost kernel: [2030572.236967] [<ffffffff811e3070>] ? meminfo_proc_show+0x480/0x480 Oct 3 03:58:56 mailhost kernel: [2030572.236971] [<ffffffff81178170>] __dentry_open+0x2b0/0x380 Oct 3 03:58:56 mailhost kernel: [2030572.236973] [<ffffffff811d9a90>] ? proc_alloc_inode+0xb0/0xb0 Oct 3 03:58:56 mailhost kernel: [2030572.236976] [<ffffffff8129fcec>] ? security_inode_permission+0x1c/0x30 Oct 3 03:58:56 mailhost kernel: [2030572.236979] [<ffffffff81185f2a>] ? inode_permission+0x4a/0x110 Oct 3 03:58:56 mailhost kernel: [2030572.236982] [<ffffffff811787ed>] vfs_open+0x3d/0x40 Oct 3 03:58:56 mailhost kernel: [2030572.236985] [<ffffffff811796f0>] nameidata_to_filp+0x40/0x50 Oct 3 03:58:56 mailhost kernel: [2030572.236988] [<ffffffff81188768>] do_last+0x3f8/0x730 Oct 3 03:58:56 mailhost kernel: [2030572.236991] [<ffffffff81189e41>] path_openat+0xd1/0x3f0 Oct 3 03:58:56 mailhost kernel: [2030572.236995] [<ffffffff81661435>] ? page_fault+0x25/0x30 Oct 3 03:58:56 mailhost kernel: [2030572.236998] [<ffffffff8118a282>] do_filp_open+0x42/0xa0 Oct 3 03:58:56 mailhost kernel: [2030572.237004] [<ffffffff8131c5e1>] ? strncpy_from_user+0x31/0x40 Oct 3 03:58:56 mailhost kernel: [2030572.237007] [<ffffffff811855ca>] ? do_getname+0x10a/0x180 Oct 3 03:58:56 mailhost kernel: [2030572.237010] [<ffffffff81660cce>] ? _raw_spin_lock+0xe/0x20 Oct 3 03:58:56 mailhost kernel: [2030572.237014] [<ffffffff811975a7>] ? alloc_fd+0xf7/0x150 Oct 3 03:58:56 mailhost kernel: [2030572.237016] [<ffffffff811797f8>] do_sys_open+0xf8/0x240 Oct 3 03:58:56 mailhost kernel: [2030572.237019] [<ffffffff81179960>] sys_open+0x20/0x30 Oct 3 03:58:56 mailhost kernel: [2030572.237022] [<ffffffff81669302>] system_call_fastpath+0x16/0x1b Oct 3 03:58:56 mailhost kernel: [2030572.237024] Mem-Info: Oct 3 03:58:56 mailhost kernel: [2030572.237025] Node 0 DMA per-cpu: Oct 3 03:58:56 mailhost kernel: [2030572.237028] CPU 0: hi: 0, btch: 1 usd: 0 Oct 3 03:58:56 mailhost kernel: [2030572.237030] CPU 1: hi: 0, btch: 1 usd: 0 Oct 3 03:58:56 mailhost kernel: [2030572.237032] CPU 2: hi: 0, btch: 1 usd: 0 Oct 3 03:58:56 mailhost kernel: [2030572.237034] CPU 3: hi: 0, btch: 1 usd: 0 Oct 3 03:58:56 mailhost kernel: [2030572.237035] Node 0 DMA32 per-cpu: Oct 3 03:58:56 mailhost kernel: [2030572.237037] CPU 0: hi: 186, btch: 31 usd: 30 Oct 3 03:58:56 mailhost kernel: [2030572.237039] CPU 1: hi: 186, btch: 31 usd: 44 Oct 3 03:58:56 mailhost kernel: [2030572.237041] CPU 2: hi: 186, btch: 31 usd: 35 Oct 3 03:58:56 mailhost kernel: [2030572.237042] CPU 3: hi: 186, btch: 31 usd: 145 Oct 3 03:58:56 mailhost kernel: [2030572.237046] active_anon:48 inactive_anon:111 isolated_anon:1 Oct 3 03:58:56 mailhost kernel: [2030572.237047] active_file:80 inactive_file:239 isolated_file:0 Oct 3 03:58:56 mailhost kernel: [2030572.237048] unevictable:0 dirty:0 writeback:46 unstable:0 Oct 3 03:58:56 mailhost kernel: [2030572.237049] free:201482 slab_reclaimable:5545 slab_unreclaimable:7628 Oct 3 03:58:56 mailhost kernel: [2030572.237050] mapped:36 shmem:2 pagetables:1927 bounce:0 Oct 3 03:58:56 mailhost kernel: [2030572.237052] Node 0 DMA free:8072kB min:344kB low:428kB high:516kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15608kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:768kB slab_unreclaimable:2160kB kernel_stack:8kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Oct 3 03:58:56 mailhost kernel: [2030572.237060] lowmem_reserve[]: 0 1966 1966 1966 Oct 3 03:58:56 mailhost kernel: [2030572.237063] Node 0 DMA32 free:797856kB min:44708kB low:55884kB high:67060kB active_anon:192kB inactive_anon:444kB active_file:320kB inactive_file:956kB unevictable:0kB isolated(anon):4kB isolated(file):0kB present:2014188kB mlocked:0kB dirty:0kB writeback:184kB mapped:144kB shmem:8kB slab_reclaimable:21412kB slab_unreclaimable:28352kB kernel_stack:2352kB pagetables:7708kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:6566 all_unreclaimable? yes Oct 3 03:58:56 mailhost kernel: [2030572.237073] lowmem_reserve[]: 0 0 0 0 Oct 3 03:58:56 mailhost kernel: [2030572.237076] Node 0 DMA: 0*4kB 19*8kB 5*16kB 13*32kB 18*64kB 7*128kB 7*256kB 1*512kB 1*1024kB 1*2048kB 0*4096kB = 8072kB Oct 3 03:58:56 mailhost kernel: [2030572.237084] Node 0 DMA32: 179180*4kB 9552*8kB 50*16kB 2*32kB 2*64kB 0*128kB 1*256kB 1*512kB 1*1024kB 1*2048kB 0*4096kB = 797968kB Oct 3 03:58:56 mailhost kernel: [2030572.237093] 567 total pagecache pages Oct 3 03:58:56 mailhost kernel: [2030572.237094] 189 pages in swap cache Oct 3 03:58:56 mailhost kernel: [2030572.237096] Swap cache stats: add 765781, delete 765592, find 27680684/27770790 Oct 3 03:58:56 mailhost kernel: [2030572.237098] Free swap = 3612968kB Oct 3 03:58:56 mailhost kernel: [2030572.237099] Total swap = 3905520kB Oct 3 03:58:56 mailhost kernel: [2030572.242278] 515620 pages RAM Oct 3 03:58:56 mailhost kernel: [2030572.242281] 13126 pages reserved Oct 3 03:58:56 mailhost kernel: [2030572.242282] 272 pages shared Oct 3 03:58:56 mailhost kernel: [2030572.242283] 298378 pages non-shared Oct 3 03:58:56 mailhost kernel: [2030572.242285] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name Oct 3 03:58:56 mailhost kernel: [2030572.242296] [ 562] 101 562 62370 0 0 0 0 rsyslogd Oct 3 03:58:56 mailhost kernel: [2030572.242299] [ 865] 0 865 3797 0 0 0 0 upstart-socket- Oct 3 03:58:56 mailhost kernel: [2030572.242303] [ 986] 0 986 12508 0 1 -17 -1000 sshd Oct 3 03:58:56 mailhost kernel: [2030572.242306] [ 1094] 0 1094 3946 1 2 0 0 getty Oct 3 03:58:56 mailhost kernel: [2030572.242309] [ 1098] 0 1098 3946 1 1 0 0 getty Oct 3 03:58:56 mailhost kernel: [2030572.242312] [ 1102] 0 1102 3946 1 0 0 0 getty Oct 3 03:58:56 mailhost kernel: [2030572.242314] [ 1103] 0 1103 3946 1 3 0 0 getty Oct 3 03:58:56 mailhost kernel: [2030572.242317] [ 1105] 0 1105 3946 1 1 0 0 getty Oct 3 03:58:56 mailhost kernel: [2030572.242319] [ 1124] 0 1124 4227 0 1 0 0 atd Oct 3 03:58:56 mailhost kernel: [2030572.242321] [ 1132] 0 1132 3995 0 2 0 0 irqbalance Oct 3 03:58:56 mailhost kernel: [2030572.242324] [ 1373] 108 1373 31952 0 3 -13 -900 postgres Oct 3 03:58:56 mailhost kernel: [2030572.242327] [ 1739] 108 1739 31947 11 2 0 0 postgres Oct 3 03:58:56 mailhost kernel: [2030572.242330] [ 1740] 108 1740 31947 3 0 0 0 postgres Oct 3 03:58:56 mailhost kernel: [2030572.242332] [ 1741] 108 1741 32164 0 3 0 0 postgres Oct 3 03:58:56 mailhost kernel: [2030572.242335] [ 1742] 108 1742 24076 0 2 0 0 postgres Oct 3 03:58:56 mailhost kernel: [2030572.242338] [ 1892] 1019 1892 27108 0 2 0 0 bdregd Oct 3 03:58:56 mailhost kernel: [2030572.242341] [ 1941] 1019 1941 194284 14 0 0 0 bdscand Oct 3 03:58:56 mailhost kernel: [2030572.242344] [ 2098] 112 2098 9444 0 1 0 0 ntpd Oct 3 03:58:56 mailhost kernel: [2030572.242347] [ 2411] 0 2411 71691 0 2 0 0 bdsnmpd Oct 3 03:58:56 mailhost kernel: [2030572.242349] [ 2435] 1019 2435 81646 0 1 0 0 bdmond Oct 3 03:58:56 mailhost kernel: [2030572.242352] [ 2458] 1019 2458 44230 0 1 0 0 bdlived Oct 3 03:58:56 mailhost kernel: [2030572.242355] [ 2482] 1019 2482 71975 0 2 0 0 bdsmtpd Oct 3 03:58:56 mailhost kernel: [2030572.242358] [ 2520] 106 2520 12295 1 0 0 0 cyrmaster Oct 3 03:58:56 mailhost kernel: [2030572.242361] [ 2610] 106 2610 18829 0 2 0 0 idled Oct 3 03:58:56 mailhost kernel: [2030572.242364] [ 2614] 0 2614 12422 0 2 0 0 munin-node Oct 3 03:58:56 mailhost kernel: [2030572.242366] [ 2777] 0 2777 6277 0 3 0 0 master Oct 3 03:58:56 mailhost kernel: [2030572.242369] [ 2821] 105 2821 6834 0 0 0 0 qmgr Oct 3 03:58:56 mailhost kernel: [2030572.242372] [ 2956] 0 2956 6277 0 1 0 0 master Oct 3 03:58:56 mailhost kernel: [2030572.242374] [ 2975] 105 2975 6834 0 0 0 0 qmgr Oct 3 03:58:56 mailhost kernel: [2030572.242377] [ 3085] 0 3085 6277 0 1 0 0 master Oct 3 03:58:56 mailhost kernel: [2030572.242380] [ 3101] 105 3101 6835 0 0 0 0 qmgr Oct 3 03:58:56 mailhost kernel: [2030572.242383] [ 3347] 0 3347 6277 1 0 0 0 master Oct 3 03:58:56 mailhost kernel: [2030572.242386] [ 3359] 105 3359 9454 0 0 0 0 tlsmgr Oct 3 03:58:56 mailhost kernel: [2030572.242389] [ 3382] 105 3382 6835 0 2 0 0 qmgr Oct 3 03:58:56 mailhost kernel: [2030572.242391] [ 3383] 0 3383 19685 1 3 0 0 saslauthd Oct 3 03:58:56 mailhost kernel: [2030572.242394] [ 3416] 0 3416 4969 0 0 0 0 smartd Oct 3 03:58:56 mailhost kernel: [2030572.242396] [ 3480] 0 3480 27881 0 3 0 0 apcupsd Oct 3 03:58:56 mailhost kernel: [2030572.242399] [ 3516] 0 3516 62708 0 1 0 0 apache2 Oct 3 03:58:56 mailhost kernel: [2030572.242402] [ 3522] 33 3522 27594 1 1 0 0 apache2 Oct 3 03:58:56 mailhost kernel: [2030572.242405] [ 3532] 33 3532 27888 0 0 0 0 apache2 Oct 3 03:58:56 mailhost kernel: [2030572.242408] [ 3548] 0 3548 3946 1 0 0 0 getty Oct 3 03:58:56 mailhost kernel: [2030572.242411] [ 3633] 106 3633 20425 0 0 0 0 notifyd Oct 3 03:58:56 mailhost kernel: [2030572.242413] [ 4074] 105 4074 9454 0 1 0 0 tlsmgr Oct 3 03:58:56 mailhost kernel: [2030572.242418] [23872] 106 23872 26460 0 2 0 0 imapd Oct 3 03:58:56 mailhost kernel: [2030572.242421] [24271] 106 24271 26384 0 2 0 0 imapd Oct 3 03:58:56 mailhost kernel: [2030572.242423] [ 5687] 33 5687 66519 0 2 0 0 apache2 Oct 3 03:58:56 mailhost kernel: [2030572.242427] [ 5695] 33 5695 66640 0 1 0 0 apache2 Oct 3 03:58:56 mailhost kernel: [2030572.242430] [60720] 0 60720 4308 0 2 0 0 upstart-udev-br Oct 3 03:58:56 mailhost kernel: [2030572.242433] [60722] 0 60722 5365 1 0 -17 -1000 udevd Oct 3 03:58:56 mailhost kernel: [2030572.242435] [60776] 0 60776 3345 0 2 0 0 mdadm Oct 3 03:58:56 mailhost kernel: [2030572.242450] [49941] 106 49941 29041 0 2 0 0 imapd Oct 3 03:58:56 mailhost kernel: [2030572.242453] [11420] 0 11420 5364 1 2 -17 -1000 udevd Oct 3 03:58:56 mailhost kernel: [2030572.242456] [11442] 0 11442 5364 0 1 -17 -1000 udevd Oct 3 03:58:56 mailhost kernel: [2030572.242458] [22658] 0 22658 4778 0 0 0 0 cron Oct 3 03:58:56 mailhost kernel: [2030572.242461] [27682] 1019 27682 61666 0 1 0 0 bdlogd Oct 3 03:58:58 mailhost kernel: [2030572.242464] [27688] 1019 27688 135302 0 2 0 0 bdmaild Oct 3 03:58:59 mailhost kernel: [2030572.242466] [28146] 105 28146 6793 0 1 0 0 pickup Oct 3 03:58:59 mailhost kernel: [2030572.242469] [28601] 105 28601 6793 0 0 0 0 pickup Oct 3 03:58:59 mailhost kernel: [2030572.242471] [28603] 105 28603 6793 0 0 0 0 pickup Oct 3 03:58:59 mailhost kernel: [2030572.242474] [30857] 105 30857 13061 0 0 0 0 smtpd Oct 3 03:58:59 mailhost kernel: [2030572.242477] [30858] 105 30858 6793 0 2 0 0 proxymap Oct 3 03:58:59 mailhost kernel: [2030572.242479] [30860] 105 30860 6793 0 1 0 0 anvil Oct 3 03:58:59 mailhost kernel: [2030572.242482] [30861] 105 30861 6896 0 1 0 0 trivial-rewrite Oct 3 03:58:59 mailhost kernel: [2030572.242484] Out of memory: Kill process 1941 (bdscand) score 35 or sacrifice child Oct 3 03:58:59 mailhost kernel: [2030572.242524] Killed process 1941 (bdscand) total-vm:777136kB, anon-rss:56kB, file-rss:0kB root@mailhost:~# uname -a Linux mailhost 3.2.0-54-generic #82-Ubuntu SMP Tue Sep 10 20:08:42 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux _______________________________________________ RLUG mailing list RLUG@lists.lug.ro http://lists.lug.ro/mailman/listinfo/rlug
