Bug#615969: linux-base: Tracking a moderate number of conenctions permanently kills network
Package: linux-base Version: 2.6.32-30 Severity: important Tags: upstream Starting up rtorrent on a number of torrents causes my VPS to loose network connectivity. Although conntrack only reports around 250 connections the kernel oops and the network will not respond any further. I can shell onto the machine via terminal but the only way to restore network connectivity so far is a reboot. Just before I loose connectivity I saw the following memory state: Slab Info nf_conntrack_expect 0 0168 241 : tunables000 : slabdata 0 0 0 nf_conntrack_c14c6a24262289240 171 : tunables000 : slabdata 17 17 0 Number of connections 252 Memory MemFree:9100 kB HighFree:300 kB LowFree:8800 kB SwapFree:1759612 kB HugePages_Free:0 The kernel shows a number of oops: Mar 1 10:01:45 socrates kernel: [ 1107.984836] swapper: page allocation failure. order:0, mode:0x20 Mar 1 10:01:45 socrates kernel: [ 1107.984846] Pid: 0, comm: swapper Not tainted 2.6.32-5-686 #1 Mar 1 10:01:45 socrates kernel: [ 1107.984848] Call Trace: Mar 1 10:01:45 socrates kernel: [ 1107.984879] [c108c07e] ? __alloc_pages_nodemask+0x484/0x4d9 Mar 1 10:01:45 socrates kernel: [ 1107.984895] [f7ce3cc9] ? try_fill_recv+0x72/0x117 [virtio_net] Mar 1 10:01:45 socrates kernel: [ 1107.984900] [f7ce4556] ? virtnet_poll+0x45b/0x4d1 [virtio_net] Mar 1 10:01:45 socrates kernel: [ 1107.984914] [c1046850] ? hrtimer_get_next_event+0x8c/0xa0 Mar 1 10:01:45 socrates kernel: [ 1107.984934] [c11d9081] ? net_rx_action+0x96/0x194 Mar 1 10:01:45 socrates kernel: [ 1107.984947] [c10354c8] ? __do_softirq+0xaa/0x151 Mar 1 10:01:45 socrates kernel: [ 1107.984951] [c10355a0] ? do_softirq+0x31/0x3c Mar 1 10:01:45 socrates kernel: [ 1107.984954] [c1035676] ? irq_exit+0x26/0x58 Mar 1 10:01:45 socrates kernel: [ 1107.984965] [c1004699] ? do_IRQ+0x78/0x89 Mar 1 10:01:45 socrates kernel: [ 1107.984969] [c10037f0] ? common_interrupt+0x30/0x38 Mar 1 10:01:45 socrates kernel: [ 1107.984980] [c101a7f4] ? native_safe_halt+0x2/0x3 Mar 1 10:01:45 socrates kernel: [ 1107.984985] [c1008593] ? default_idle+0x3c/0x5a Mar 1 10:01:45 socrates kernel: [ 1107.984988] [c1002388] ? cpu_idle+0x89/0xa5 Mar 1 10:01:45 socrates kernel: [ 1107.984999] [c13bd7fc] ? start_kernel+0x318/0x31d Mar 1 10:01:45 socrates kernel: [ 1107.985001] Mem-Info: Mar 1 10:01:45 socrates kernel: [ 1107.985006] DMA per-cpu: Mar 1 10:01:45 socrates kernel: [ 1107.985009] CPU0: hi:0, btch: 1 usd: 0 Mar 1 10:01:45 socrates kernel: [ 1107.985011] Normal per-cpu: Mar 1 10:01:45 socrates kernel: [ 1107.985013] CPU0: hi: 186, btch: 31 usd: 184 Mar 1 10:01:45 socrates kernel: [ 1107.985015] HighMem per-cpu: Mar 1 10:01:45 socrates kernel: [ 1107.985017] CPU0: hi: 18, btch: 3 usd: 17 Mar 1 10:01:45 socrates kernel: [ 1107.985023] active_anon:14275 inactive_anon:24343 isolated_anon:0 Mar 1 10:01:45 socrates kernel: [ 1107.985024] active_file:98037 inactive_file:97736 isolated_file:0 Mar 1 10:01:45 socrates kernel: [ 1107.985026] unevictable:0 dirty:24157 writeback:9 unstable:0 Mar 1 10:01:45 socrates kernel: [ 1107.985027] free:1323 slab_reclaimable:2077 slab_unreclaimable:1271 Mar 1 10:01:45 socrates kernel: [ 1107.985029] mapped:73828 shmem:3 pagetables:574 bounce:0 Mar 1 10:01:45 socrates kernel: [ 1107.985036] DMA free:3460kB min:64kB low:80kB high:96kB active_anon:864kB inactive_anon:1588kB active_file:4812kB inactive_file:4744kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15868kB mlocked:0kB dirty:16kB writeback:0kB mapped:132kB shmem:0kB slab_reclaimable:12kB slab_unreclaimable:60kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Mar 1 10:01:45 socrates kernel: [ 1107.985044] lowmem_reserve[]: 0 861 936 936 Mar 1 10:01:45 socrates kernel: [ 1107.985053] Normal free:1384kB min:3720kB low:4648kB high:5580kB active_anon:45276kB inactive_anon:83068kB active_file:361948kB inactive_file:360812kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:881880kB mlocked:0kB dirty:91924kB writeback:32kB mapped:279348kB shmem:0kB slab_reclaimable:8296kB slab_unreclaimable:5024kB kernel_stack:928kB pagetables:1264kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:2681 all_unreclaimable? no Mar 1 10:01:45 socrates kernel: [ 1107.985060] lowmem_reserve[]: 0 0 603 603 Mar 1 10:01:45 socrates kernel: [ 1107.985069] HighMem free:448kB min:128kB low:208kB high:288kB active_anon:10960kB inactive_anon:12716kB active_file:25388kB inactive_file:25388kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:77212kB mlocked:0kB dirty:4688kB writeback:4kB mapped:15832kB shmem:12kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:1032kB unstable:0kB bounce:0kB writeback_tmp:0kB
Bug#615969: linux-base: Tracking a moderate number of conenctions permanently kills network
On Tue, 2011-03-01 at 12:28 +0100, Alex Bennee wrote: Package: linux-base Version: 2.6.32-30 Severity: important Tags: upstream Starting up rtorrent on a number of torrents causes my VPS to loose network connectivity. [...] What virtualisation system are you using? Does the VPS use virtio_net? Ben. -- Ben Hutchings Once a job is fouled up, anything done to improve it makes it worse. signature.asc Description: This is a digitally signed message part
Bug#615969: linux-base: Tracking a moderate number of conenctions permanently kills network
On Tue, Mar 01, 2011 at 12:55:19PM +, Ben Hutchings wrote: On Tue, 2011-03-01 at 12:28 +0100, Alex Bennee wrote: Package: linux-base Version: 2.6.32-30 Severity: important Tags: upstream Starting up rtorrent on a number of torrents causes my VPS to loose network connectivity. [...] What virtualisation system are you using? Does the VPS use virtio_net? KVM and yes. I'm unsure if the problem is a result of conntrack fragmenting memory as it's tracking all the extra connections or just exhaustion of the virtio receive queue due to the peak in traffic as the torrents come up. Having said that even if virtio had to throw stuff away due to memory scarcity I would expect it to recover once memory returned. It's not as though the virtual machine hard OOOs. You can still log in via the serial console after the network has died. -- Alex, homepage: http://www.bennee.com/~alex/ Cleanliness is next to impossible. signature.asc Description: Digital signature
Bug#615969: linux-base: Tracking a moderate number of conenctions permanently kills network
On Tue, Mar 01, 2011 at 02:09:36PM +0100, Alex wrote: On Tue, Mar 01, 2011 at 12:55:19PM +, Ben Hutchings wrote: On Tue, 2011-03-01 at 12:28 +0100, Alex Bennee wrote: Package: linux-base Version: 2.6.32-30 Severity: important Tags: upstream Starting up rtorrent on a number of torrents causes my VPS to loose network connectivity. [...] What virtualisation system are you using? Does the VPS use virtio_net? KVM and yes. I'm unsure if the problem is a result of conntrack fragmenting memory as it's tracking all the extra connections or just exhaustion of the virtio receive queue due to the peak in traffic as the torrents come up. [...] This is a known bug and should be fixed soon in version 2.6.32-31 (Debian 6.0.1). Ben. -- Ben Hutchings We get into the habit of living before acquiring the habit of thinking. - Albert Camus -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
