Public bug reported: I've encountered the following kernel crash on a system that heavily uses overlayfs. While not necessarily an overlayfs problem, it (adding overlayfs) was the last major change to the system.
During the crash rsync was synchronizing non-overlayfs directory which was bind-mounted on an overlayfs filesystem (i.e., /home under schroot). May 19 14:03:54 maximk kernel: [438288.554691] general protection fault: 0000 [#1] SMP May 19 14:03:54 maximk kernel: [438288.554735] Modules linked in: overlayfs vhost_net macvtap macvlan vhost ip6table_filter ip6_tables ebtable_nat ebtables ipt_MASQUERADE iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_state nf_conntrack ipt_REJECT xt_CHECKSUM iptable_mangle xt_tcpudp iptable_filter ip_tables x_tables bridge stp llc kvm_intel kvm rfcomm bnep parport_pc ppdev bluetooth binfmt_misc snd_hda_codec_hdmi snd_hda_codec_realtek snd_hda_intel snd_hda_codec snd_hwdep snd_pcm snd_seq_midi snd_rawmidi snd_seq_midi_event snd_seq snd_timer snd_seq_device dm_multipath scsi_dh snd soundcore i915 snd_page_alloc mei_me mei drm_kms_helper drm serio_raw gpio_ich mac_hid lp i2c_algo_bit video parport lpc_ich ahci libahci r8169 mii May 19 14:03:54 maximk kernel: [438288.555233] CPU: 1 PID: 59 Comm: kswapd0 Not tainted 3.11.0-20-generic #35~precise1-Ubuntu May 19 14:03:54 maximk kernel: [438288.555279] Hardware name: Gigabyte Technology Co., Ltd. H67MA-D2H-B3/H67MA-D2H-B3, BIOS F6 03/27/2012 May 19 14:03:54 maximk kernel: [438288.555331] task: ffff8804089f1770 ti: ffff8804089ec000 task.ti: ffff8804089ec000 May 19 14:03:54 maximk kernel: [438288.555372] RIP: 0010:[<ffffffff811d06f6>] [<ffffffff811d06f6>] prune_icache_sb+0x1a6/0x340 May 19 14:03:54 maximk kernel: [438288.555428] RSP: 0018:ffff8804089edb98 EFLAGS: 00010206 May 19 14:03:54 maximk kernel: [438288.555458] RAX: 08fd15b8ffff8804 RBX: ffff8803e78269ac RCX: 0000000000010000 May 19 14:03:54 maximk kernel: [438288.555498] RDX: ffff88040529e508 RSI: 00000000000001c4 RDI: ffff8803e782693c May 19 14:03:54 maximk kernel: [438288.555537] RBP: ffff8804089edbf8 R08: 0000000000000000 R09: 0000000000000000 May 19 14:03:54 maximk kernel: [438288.555592] R10: 0000000000000001 R11: 0000000000000000 R12: ffff88040529e400 May 19 14:03:54 maximk kernel: [438288.555632] R13: ffff8803e782693c R14: 000000000000010d R15: ffff88040529e508 May 19 14:03:54 maximk kernel: [438288.555672] FS: 0000000000000000(0000) GS:ffff88041fa40000(0000) knlGS:0000000000000000 May 19 14:03:54 maximk kernel: [438288.555714] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 May 19 14:03:54 maximk kernel: [438288.556917] CR2: 00007f265ae7a000 CR3: 0000000001c0d000 CR4: 00000000000427e0 May 19 14:03:54 maximk kernel: [438288.558432] Stack: May 19 14:03:54 maximk kernel: [438288.559943] ffff88041fdedd80 ffff880319ff1800 0000000000000000 ffff88040529e500 May 19 14:03:54 maximk kernel: [438288.561481] ffff8804089edbb8 ffff8804089edbb8 ffff8804089edbd8 ffff88040529e7b0 May 19 14:03:54 maximk kernel: [438288.563010] ffff88040529e400 0000000000000000 00000000000001c4 ffff8804089edd18 May 19 14:03:54 maximk kernel: [438288.564545] Call Trace: May 19 14:03:54 maximk kernel: [438288.566054] [<ffffffff811b8683>] prune_super+0x193/0x1b0 May 19 14:03:54 maximk kernel: [438288.567552] [<ffffffff8115b804>] shrink_slab+0x154/0x300 May 19 14:03:54 maximk kernel: [438288.569018] [<ffffffff811adf66>] ? vmpressure+0x96/0xa0 May 19 14:03:54 maximk kernel: [438288.570491] [<ffffffff8115e03b>] kswapd_shrink_zone+0xfb/0x1a0 May 19 14:03:54 maximk kernel: [438288.571963] [<ffffffff8115ee0c>] balance_pgdat+0x32c/0x470 May 19 14:03:54 maximk kernel: [438288.573439] [<ffffffff81166f20>] ? vm_events_fold_cpu+0x50/0x50 May 19 14:03:54 maximk kernel: [438288.574922] [<ffffffff8115f073>] kswapd+0x123/0x240 May 19 14:03:54 maximk kernel: [438288.576398] [<ffffffff8115ef50>] ? balance_pgdat+0x470/0x470 May 19 14:03:54 maximk kernel: [438288.577868] [<ffffffff81089190>] kthread+0xc0/0xd0 May 19 14:03:54 maximk kernel: [438288.579330] [<ffffffff810890d0>] ? flush_kthread_worker+0xb0/0xb0 May 19 14:03:54 maximk kernel: [438288.580764] [<ffffffff817528ec>] ret_from_fork+0x7c/0xb0 May 19 14:03:54 maximk kernel: [438288.582190] [<ffffffff810890d0>] ? flush_kthread_worker+0xb0/0xb0 May 19 14:03:54 maximk kernel: [438288.583621] Code: 10 48 8d 7d c0 e8 cb ef ff ff 48 83 c4 38 5b 41 5c 41 5d 41 5e 41 5f 5d c3 0f 1f 40 00 48 8b 13 48 8b 43 08 4c 89 ef 48 89 42 08 <48> 89 10 48 89 1b 48 89 5b 08 e8 2b 09 e8 ff 66 90 41 83 ac 24 May 19 14:03:54 maximk kernel: [438288.586816] RIP [<ffffffff811d06f6>] prune_icache_sb+0x1a6/0x340 May 19 14:03:54 maximk kernel: [438288.588370] RSP <ffff8804089edb98> May 19 14:03:54 maximk kernel: [438288.595099] ---[ end trace b89f93acb0e5743d ]--- May 19 14:04:22 maximk kernel: [438316.982835] BUG: soft lockup - CPU#0 stuck for 22s! [rsync:21394] May 19 14:04:22 maximk kernel: [438316.982839] Modules linked in: overlayfs vhost_net macvtap macvlan vhost ip6table_filter ip6_tables ebtable_nat ebtables ipt_MASQUERADE iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_state nf_conntrack ipt_REJECT xt_CHECKSUM iptable_mangle xt_tcpudp iptable_filter ip_tables x_tables bridge stp llc kvm_intel kvm rfcomm bnep parport_pc ppdev bluetooth binfmt_misc snd_hda_codec_hdmi snd_hda_codec_realtek snd_hda_intel snd_hda_codec snd_hwdep snd_pcm snd_seq_midi snd_rawmidi snd_seq_midi_event snd_seq snd_timer snd_seq_device dm_multipath scsi_dh snd soundcore i915 snd_page_alloc mei_me mei drm_kms_helper drm serio_raw gpio_ich mac_hid lp i2c_algo_bit video parport lpc_ich ahci libahci r8169 mii May 19 14:04:22 maximk kernel: [438316.982902] CPU: 0 PID: 21394 Comm: rsync Tainted: G D 3.11.0-20-generic #35~precise1-Ubuntu May 19 14:04:22 maximk kernel: [438316.982904] Hardware name: Gigabyte Technology Co., Ltd. H67MA-D2H-B3/H67MA-D2H-B3, BIOS F6 03/27/2012 May 19 14:04:22 maximk kernel: [438316.982907] task: ffff880405bf2ee0 ti: ffff8803209e0000 task.ti: ffff8803209e0000 May 19 14:04:22 maximk kernel: [438316.982910] RIP: 0010:[<ffffffff81050ff5>] [<ffffffff81050ff5>] __ticket_spin_lock+0x25/0x30 May 19 14:04:22 maximk kernel: [438316.982917] RSP: 0018:ffff8803209e1578 EFLAGS: 00000297 May 19 14:04:22 maximk kernel: [438316.982919] RAX: 0000000000005a18 RBX: ffffea0000bc75e0 RCX: 0000000000000001 May 19 14:04:22 maximk kernel: [438316.982921] RDX: 0000000000005a19 RSI: ffff8802abcbb138 RDI: ffff88040529e500 May 19 14:04:22 maximk kernel: [438316.982923] RBP: ffff8803209e1578 R08: ffffc90000002000 R09: 0000000000000000 May 19 14:04:22 maximk kernel: [438316.982925] R10: 000000000007fe18 R11: 0000000000000001 R12: ffffffff811aac95 May 19 14:04:22 maximk kernel: [438316.982927] R13: ffff8803209e14f8 R14: 0000000000000000 R15: 00ff880300000000 May 19 14:04:22 maximk kernel: [438316.982930] FS: 00007fad8d03b700(0000) GS:ffff88041fa00000(0000) knlGS:0000000000000000 May 19 14:04:22 maximk kernel: [438316.982932] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 May 19 14:04:22 maximk kernel: [438316.982935] CR2: 00007f265ae7a000 CR3: 000000013cab2000 CR4: 00000000000427e0 May 19 14:04:22 maximk kernel: [438316.982937] Stack: May 19 14:04:22 maximk kernel: [438316.982938] ffff8803209e1588 ffffffff8174957e ffff8803209e15a8 ffffffff811cfd74 May 19 14:04:22 maximk kernel: [438316.982943] ffff8802329e86c0 ffff8802abcbb0b0 ffff8803209e15d8 ffffffff811cfef9 May 19 14:04:22 maximk kernel: [438316.982947] ffff8803209e15f8 ffff8802abcbb0b0 ffff8802abcbb0b0 ffff8803209e16a8 May 19 14:04:22 maximk kernel: [438316.982951] Call Trace: May 19 14:04:22 maximk kernel: [438316.982959] [<ffffffff8174957e>] _raw_spin_lock+0xe/0x20 May 19 14:04:22 maximk kernel: [438316.982964] [<ffffffff811cfd74>] inode_add_lru+0x44/0xb0 May 19 14:04:22 maximk kernel: [438316.982968] [<ffffffff811cfef9>] iput_final+0x119/0x170 May 19 14:04:22 maximk kernel: [438316.982973] [<ffffffff811cff8e>] iput+0x3e/0x50 May 19 14:04:22 maximk kernel: [438316.982977] [<ffffffff811c9ea8>] dentry_iput+0xa8/0xf0 May 19 14:04:22 maximk kernel: [438316.982981] [<ffffffff811cbc28>] shrink_dentry_list+0x158/0x2f0 May 19 14:04:22 maximk kernel: [438316.982985] [<ffffffff811cc76a>] prune_dcache_sb+0x15a/0x190 May 19 14:04:22 maximk kernel: [438316.982989] [<ffffffff811b8678>] prune_super+0x188/0x1b0 May 19 14:04:22 maximk kernel: [438316.982994] [<ffffffff8115b804>] shrink_slab+0x154/0x300 May 19 14:04:22 maximk kernel: [438316.982999] [<ffffffff8115e4f8>] do_try_to_free_pages+0x218/0x290 May 19 14:04:22 maximk kernel: [438316.983003] [<ffffffff8115e8a4>] try_to_free_pages+0xe4/0x1a0 May 19 14:04:22 maximk kernel: [438316.983009] [<ffffffff81152b48>] __alloc_pages_nodemask+0x618/0x9a0 May 19 14:04:22 maximk kernel: [438316.983016] [<ffffffff811907ea>] alloc_pages_current+0xba/0x170 May 19 14:04:22 maximk kernel: [438316.983020] [<ffffffff81149ad7>] __page_cache_alloc+0xb7/0xd0 May 19 14:04:22 maximk kernel: [438316.983024] [<ffffffff8114a309>] grab_cache_page_write_begin+0x79/0xe0 May 19 14:04:22 maximk kernel: [438316.983030] [<ffffffff812569b9>] ext4_da_write_begin+0x89/0x2e0 May 19 14:04:22 maximk kernel: [438316.983035] [<ffffffff812579ce>] ? ext4_da_write_end+0xbe/0x250 May 19 14:04:22 maximk kernel: [438316.983039] [<ffffffff811496da>] generic_perform_write+0xca/0x210 May 19 14:04:22 maximk kernel: [438316.983043] [<ffffffff811de538>] ? __mark_inode_dirty+0x48/0x350 May 19 14:04:22 maximk kernel: [438316.983047] [<ffffffff8114987d>] generic_file_buffered_write+0x5d/0x90 May 19 14:04:22 maximk kernel: [438316.983051] [<ffffffff8114b466>] __generic_file_aio_write+0x1b6/0x3b0 May 19 14:04:22 maximk kernel: [438316.983055] [<ffffffff8114b6c9>] generic_file_aio_write+0x69/0xd0 May 19 14:04:22 maximk kernel: [438316.983059] [<ffffffff8124d43b>] ext4_file_write+0x8b/0xd0 May 19 14:04:22 maximk kernel: [438316.983064] [<ffffffff811b421a>] do_sync_write+0x7a/0xb0 May 19 14:04:22 maximk kernel: [438316.983069] [<ffffffff811b505e>] vfs_write+0xce/0x200 May 19 14:04:22 maximk kernel: [438316.983073] [<ffffffff811b5562>] SyS_write+0x52/0xa0 May 19 14:04:22 maximk kernel: [438316.983078] [<ffffffff8175299d>] system_call_fastpath+0x1a/0x1f May 19 14:04:22 maximk kernel: [438316.983080] Code: 90 90 90 90 90 90 55 b8 00 00 01 00 48 89 e5 f0 0f c1 07 89 c2 c1 ea 10 66 39 c2 74 13 66 0f 1f 84 00 00 00 00 00 f3 90 0f b7 07 <66> 39 d0 75 f6 5d c3 0f 1f 40 00 8b 17 55 31 c0 48 89 e5 89 d1 ProblemType: Bug DistroRelease: Ubuntu 12.04 Package: linux-image-3.11.0-20-generic 3.11.0-20.35~precise1 ProcVersionSignature: Ubuntu 3.11.0-20.35~precise1-generic 3.11.10.6 Uname: Linux 3.11.0-20-generic x86_64 AlsaVersion: Advanced Linux Sound Architecture Driver Version k3.11.0-20-generic. AplayDevices: aplay: device_list:252: no soundcards found... ApportVersion: 2.0.1-0ubuntu17.6 Architecture: amd64 ArecordDevices: arecord: device_list:252: no soundcards found... AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/by-path', '/dev/snd/controlC0', '/dev/snd/hwC0D2', '/dev/snd/hwC0D3', '/dev/snd/pcmC0D0c', '/dev/snd/pcmC0D0p', '/dev/snd/pcmC0D1p', '/dev/snd/pcmC0D2c', '/dev/snd/pcmC0D3p', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1: CRDA: Error: [Errno 2] No such file or directory Date: Tue May 20 08:32:58 2014 HibernationDevice: RESUME=UUID=a59c565f-f8c4-4b0b-893a-6686368a7cdc InstallationMedia: Ubuntu 12.04.4 LTS "Precise Pangolin" - Release amd64 (20140204) MachineType: Gigabyte Technology Co., Ltd. H67MA-D2H-B3 MarkForUpload: True ProcEnviron: LANGUAGE=en_NZ:en TERM=xterm-256color PATH=(custom, no user) LANG=en_NZ.UTF-8 SHELL=/bin/bash ProcFB: 0 inteldrmfb ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-3.11.0-20-generic root=UUID=7d3da03b-d936-4184-a562-f2efcd31d6bf ro quiet splash vt.handoff=7 PulseList: Error: command ['pacmd', 'list'] failed with exit code 1: No PulseAudio daemon running, or not running as session daemon. RelatedPackageVersions: linux-restricted-modules-3.11.0-20-generic N/A linux-backports-modules-3.11.0-20-generic N/A linux-firmware 1.79.14 RfKill: SourcePackage: linux-lts-saucy UpgradeStatus: No upgrade log present (probably fresh install) dmi.bios.date: 03/27/2012 dmi.bios.vendor: Award Software International, Inc. dmi.bios.version: F6 dmi.board.name: H67MA-D2H-B3 dmi.board.vendor: Gigabyte Technology Co., Ltd. dmi.board.version: x.x dmi.chassis.type: 3 dmi.chassis.vendor: Gigabyte Technology Co., Ltd. dmi.modalias: dmi:bvnAwardSoftwareInternational,Inc.:bvrF6:bd03/27/2012:svnGigabyteTechnologyCo.,Ltd.:pnH67MA-D2H-B3:pvr:rvnGigabyteTechnologyCo.,Ltd.:rnH67MA-D2H-B3:rvrx.x:cvnGigabyteTechnologyCo.,Ltd.:ct3:cvr: dmi.product.name: H67MA-D2H-B3 dmi.sys.vendor: Gigabyte Technology Co., Ltd. ** Affects: linux-lts-saucy (Ubuntu) Importance: Undecided Status: New ** Tags: amd64 apport-bug precise qa-kernel-lts-testing -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1321005 Title: Kernel crash: kswapd, rsync, overlayfs To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux-lts-saucy/+bug/1321005/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs