On Sat, Mar 02, 2013 at 03:00:28AM -0800, Russ Dill wrote: > I'm seeing a race in fs/fcntl.c. I'm not sure exactly how the race is > occurring, but the following is my best guess. A kernel log is > attached.
[snip the analysis - it's a different lock anyway] The traces below are essentially sys_execve() getting to get_random_bytes(), to kill_fasync(), to send_sigio(), which spins on tasklist_lock. Could you rebuild it with lockdep enabled and try to reproduce that? I very much doubt that this execve() is a part of deadlock - it's getting caught on one, but it shouldn't be holding any locks that nest inside tasklist_lock at that point, so even it hadn't been there, the process holding tasklist_lock probably wouldn't have progressed any further... > I exercise this by running a UML instance which uses /dev/random. > After a day or so, I'll eventually get a crash or a hang. I captured > this with netconsole. > > > [172635.399438] ------------[ cut here ]------------ > [172635.399449] WARNING: at > /build/buildd/linux-3.8.0/kernel/watchdog.c:246 > watchdog_overflow_callback+0x9c/0xd0() > [172635.399451] Hardware name: VPCSE190X > [172635.399454] Watchdog detected hard LOCKUP on cpu 0 > [172635.399456] Modules linked in: ftdi_sio netconsole(F) configfs(F) > ext2(F) nls_iso8859_1(F) usb_storage(F) pl2303 usbserial snd > _usb_audio snd_usbmidi_lib parport_pc(F) ppdev(F) lp(F) parport(F) > bnep rfcomm bluetooth binfmt_misc(F) dm_crypt(F) uvcvideo arc4( > F) snd_hda_codec_hdmi iwldvm snd_hda_codec_realtek snd_hda_intel > videobuf2_vmalloc videobuf2_memops mac80211 videobuf2_core snd_hd > a_codec videodev snd_hwdep(F) coretemp snd_pcm(F) snd_page_alloc(F) > joydev(F) iwlwifi[172635.399521] ------------[ cut here ]------------ > [172635.399525] WARNING: at > /build/buildd/linux-3.8.0/net/core/skbuff.c:573 > skb_release_head_state+0xed/0x100() > [172635.399525] Hardware name: VPCSE190X > [172635.399526] Modules linked in: ftdi_sio netconsole(F) configfs(F) > ext2(F) nls_iso8859_1(F) usb_storage(F) pl2303 usbserial snd_usb_audio > snd_usbmidi_lib parport_pc(F) ppdev(F) lp(F) parport(F) bnep rfcomm > bluetooth binfmt_misc(F) dm_crypt(F) uvcvideo arc4(F) > snd_hda_codec_hdmi iwldvm snd_hda_codec_realtek snd_hda_intel > videobuf2_vmalloc videobuf2_memops mac80211 videobuf2_core > snd_hda_codec videodev snd_hwdep(F) coretemp snd_pcm(F) > snd_page_alloc(F) joydev(F) iwlwifi kvm_intel snd_seq_midi(F) > snd_seq_midi_event(F) snd_rawmidi(F) kvm cfg80211 dm_multipath(F) > snd_seq(F) snd_seq_device(F) tpm_infineon snd_timer(F) psmouse(F) > snd(F) soundcore(F) scsi_dh serio_raw(F) sony_laptop rtsx_pci_ms mei > microcode(F) memstick mac_hid pcspkr lpc_ich tpm_tis firewire_sbp2 > firewire_core crc_itu_t(F) btrfs(F) zlib_deflate(F) libcrc32c(F) > hid_generic usbhid hid rtsx_pci_sdmmc ghash_clmulni_intel(F) > aesni_intel(F) aes_x86_64(F) xts(F) lrw(F) gf128mul(F) ablk_helper(F) > cryptd(F) radeon ahci(F) libahci(F) r8169 i915 ttm i2c_algo_bit > drm_kms_helper video(F) drm rtsx_pci > [172635.399572] Pid: 12575, comm: flock Tainted: GF > 3.8.0-7-generic #15-Ubuntu > [172635.399572] Call Trace: > [172635.399573] <NMI> [<ffffffff810587cf>] warn_slowpath_common+0x7f/0xc0 > [172635.399577] [<ffffffff8105882a>] warn_slowpath_null+0x1a/0x20 > [172635.399579] [<ffffffff815b51bd>] skb_release_head_state+0xed/0x100 > [172635.399580] [<ffffffff815b4fb2>] __kfree_skb+0x12/0xa0 > [172635.399582] [<ffffffff815b551c>] consume_skb+0x2c/0x80 > [172635.399586] [<ffffffffa014cf47>] rtl8169_poll+0x4b7/0x6d0 [r8169] > [172635.399590] [<ffffffff815d96c2>] netpoll_poll_dev+0x162/0x580 > [172635.399593] [<ffffffff815b666b>] ? __alloc_skb+0x8b/0x2a0 > [172635.399595] [<ffffffff815d9c6c>] netpoll_send_skb_on_dev+0x18c/0x3a0 > [172635.399598] [<ffffffff815da198>] netpoll_send_udp+0x278/0x2a0 > [172635.399600] [<ffffffffa07c4967>] write_msg+0xc7/0x110 [netconsole] > [172635.399603] [<ffffffff810594f1>] > call_console_drivers.constprop.13+0x91/0x100 > [172635.399605] [<ffffffff81059e4b>] console_unlock+0x2db/0x420 > [172635.399606] [<ffffffff8105a7dd>] vprintk_emit+0x1fd/0x4e0 > [172635.399608] [<ffffffff810e8ffc>] ? watchdog_overflow_callback+0x9c/0xd0 > [172635.399610] [<ffffffff816b78c1>] printk+0x67/0x69 > [172635.399613] [<ffffffff810c0423>] print_modules+0xa3/0xd0 > [172635.399617] [<ffffffff810587ca>] warn_slowpath_common+0x7a/0xc0 > [172635.399618] [<ffffffff810588cc>] warn_slowpath_fmt+0x4c/0x50 > [172635.399620] [<ffffffff8109170d>] ? sched_clock_cpu+0xbd/0x110 > [172635.399623] [<ffffffff810e8f60>] ? touch_nmi_watchdog+0x80/0x80 > [172635.399625] [<ffffffff810e8ffc>] watchdog_overflow_callback+0x9c/0xd0 > [172635.399627] [<ffffffff8112609d>] __perf_event_overflow+0x9d/0x230 > [172635.399630] [<ffffffff81025277>] ? x86_perf_event_set_period+0xd7/0x160 > [172635.399632] [<ffffffff81126cb4>] perf_event_overflow+0x14/0x20 > [172635.399634] [<ffffffff8102ad8b>] intel_pmu_handle_irq+0x1ab/0x330 > [172635.399637] [<ffffffff816c5b9d>] perf_event_nmi_handler+0x1d/0x20 > [172635.399640] [<ffffffff816c5351>] nmi_handle.isra.0+0x51/0x80 > [172635.399641] [<ffffffff816c5460>] do_nmi+0xe0/0x360 > [172635.399643] [<ffffffff816c4981>] end_repeat_nmi+0x1e/0x2e > [172635.399645] [<ffffffff813536d9>] ? __read_lock_failed+0x9/0x20 > [172635.399648] [<ffffffff813536d9>] ? __read_lock_failed+0x9/0x20 > [172635.399649] [<ffffffff813536d9>] ? __read_lock_failed+0x9/0x20 > [172635.399651] <<EOE>> [<ffffffff816c3e13>] _raw_read_lock+0x13/0x20 > [172635.399654] [<ffffffff811a4532>] send_sigio+0x52/0xf0 > [172635.399656] [<ffffffff811a4631>] kill_fasync+0x61/0x90 > [172635.399658] [<ffffffff8143ad83>] account+0x113/0x1d0 > [172635.399661] [<ffffffff816c4618>] ? page_fault+0x28/0x30 > [172635.399663] [<ffffffff8143b335>] extract_entropy+0x65/0x140 > [172635.399664] [<ffffffff8143b650>] get_random_bytes+0x20/0x30 > [172635.399666] [<ffffffff816bb2a9>] create_elf_tables+0xaa/0x614 > [172635.399669] [<ffffffff811e8a84>] load_elf_binary+0xae4/0xe00 > [172635.399672] [<ffffffff811e7fa0>] ? load_elf_library+0x240/0x240 > [172635.399674] [<ffffffff8119999e>] search_binary_handler+0x19e/0x340 > [172635.399677] [<ffffffff8119ad15>] do_execve_common.isra.22+0x3c5/0x470 > [172635.399679] [<ffffffff8119add8>] do_execve+0x18/0x20 > [172635.399680] [<ffffffff8119b07d>] sys_execve+0x3d/0x60 > [172635.399682] [<ffffffff816ccb49>] stub_execve+0x69/0xc0 > [172635.399685] ---[ end trace 5ccb38c703860d58 ]--- > [172635.399687] ------------[ cut here ]------------ > [172635.399688] WARNING: at > /build/buildd/linux-3.8.0/net/core/skbuff.c:573 > skb_release_head_state+0xed/0x100() > [172635.399689] Hardware name: VPCSE190X > [172635.399689] Modules linked in: ftdi_sio netconsole(F) configfs(F) > ext2(F) nls_iso8859_1(F) usb_storage(F) pl2303 usbserial snd_usb_audio > snd_usbmidi_lib parport_pc(F) ppdev(F) lp(F) parport(F) bnep rfcomm > bluetooth binfmt_misc(F) dm_crypt(F) uvcvideo arc4(F) > snd_hda_codec_hdmi iwldvm snd_hda_codec_realtek snd_hda_intel > videobuf2_vmalloc videobuf2_memops mac80211 videobuf2_core > snd_hda_codec videodev snd_hwdep(F) coretemp snd_pcm(F) > snd_page_alloc(F) joydev(F) iwlwifi kvm_intel snd_seq_midi(F) > snd_seq_midi_event(F) snd_rawmidi(F) kvm cfg80211 dm_multipath(F) > snd_seq(F) snd_seq_device(F) tpm_infineon snd_timer(F) psmouse(F) > snd(F) soundcore(F) scsi_dh serio_raw(F) sony_laptop rtsx_pci_ms mei > microcode(F) memstick mac_hid pcspkr lpc_ich tpm_tis firewire_sbp2 > firewire_core crc_itu_t(F) btrfs(F) zlib_deflate(F) libcrc32c(F) > hid_generic usbhid hid rtsx_pci_sdmmc ghash_clmulni_intel(F) > aesni_intel(F) aes_x86_64(F) xts(F) lrw(F) gf128mul(F) ablk_helper(F) > cryptd(F) radeon ahci(F) libahci(F) r8169 i915 ttm i2c_algo_bit > drm_kms_helper video(F) drm rtsx_pci > [172635.399725] Pid: 12575, comm: flock Tainted: GF W > 3.8.0-7-generic #15-Ubuntu > [172635.399726] Call Trace: > [172635.399726] <NMI> [<ffffffff810587cf>] warn_slowpath_common+0x7f/0xc0 > [172635.399728] [<ffffffff8105882a>] warn_slowpath_null+0x1a/0x20 > [172635.399729] [<ffffffff815b51bd>] skb_release_head_state+0xed/0x100 > [172635.399731] [<ffffffff815b4fb2>] __kfree_skb+0x12/0xa0 > [172635.399732] [<ffffffff815b551c>] consume_skb+0x2c/0x80 > [172635.399735] [<ffffffffa014cf47>] rtl8169_poll+0x4b7/0x6d0 [r8169] > [172635.399738] [<ffffffff815d96c2>] netpoll_poll_dev+0x162/0x580 > [172635.399740] [<ffffffff815b666b>] ? __alloc_skb+0x8b/0x2a0 > [172635.399742] [<ffffffff815d9c6c>] netpoll_send_skb_on_dev+0x18c/0x3a0 > [172635.399744] [<ffffffff815da198>] netpoll_send_udp+0x278/0x2a0 > [172635.399746] [<ffffffffa07c4967>] write_msg+0xc7/0x110 [netconsole] > [172635.399749] [<ffffffff810594f1>] > call_console_drivers.constprop.13+0x91/0x100 > [172635.399750] [<ffffffff81059e4b>] console_unlock+0x2db/0x420 > [172635.399752] [<ffffffff8105a7dd>] vprintk_emit+0x1fd/0x4e0 > [172635.399754] [<ffffffff810e8ffc>] ? watchdog_overflow_callback+0x9c/0xd0 > [172635.399756] [<ffffffff816b78c1>] printk+0x67/0x69 > [172635.399757] [<ffffffff810c0423>] print_modules+0xa3/0xd0 > [172635.399760] [<ffffffff810587ca>] warn_slowpath_common+0x7a/0xc0 > [172635.399761] [<ffffffff810588cc>] warn_slowpath_fmt+0x4c/0x50 > [172635.399763] [<ffffffff8109170d>] ? sched_clock_cpu+0xbd/0x110 > [172635.399765] [<ffffffff810e8f60>] ? touch_nmi_watchdog+0x80/0x80 > [172635.399766] [<ffffffff816c4981>] end_repeat_nmi+0x1e/0x2e > [<ffffffff811a4631>] kill_fasync+0x61/0x90 > [172635.399793] [<ffffffff8143ad83>] account+0x113/0x1d0 > [172635.399794] [<ffffffff816c4618>] ? page_fault+0x28/0x30 > [172635.399796] [172635.399800] [<ffffffff816bb2a9>] > create_elf_tables+0xaa/0x614 > [172635.399802] [<ffffffff811e8a84>] load_elf_binary+0xae4/0xe00 > [172635.399804] [<ffffffff811e7fa0>] ? load_elf_library+0x240/0x240 > [<ffffffff8119ad15>] do_execve_common.isra.22+0x3c5/0x470 > [172635.399810] [<ffffffff8119add8>] do_execve+0x18/0x20 > [172635.399812] [<ffffffff8119b07d>] sys_execve+0x3d/0x60 > [172635.399813] netconsole(F) configfs(F) ext2(F) bnep rfcomm > bluetooth binfmt_misc(F) dm_crypt(F) uvcvideo arc4(F) > snd_hda_codec_hdmi iwldvm snd_hwdep(F) coretemp snd_pcm(F) > snd_page_alloc(F) joydev(F) iwlwifi kvm_intel snd_seq_midi(F) scsi_dh > serio_raw(F) sony_laptop rtsx_pci_ms mei aes_x86_64(F) xts(F) lrw(F) > gf128mul(F) ablk_helper(F) i2c_algo_bit drm_kms_helper[172635.399858] > [<ffffffff815b51bd>] skb_release_head_state+0xed/0x100 > [172635.399860] [<ffffffff815b4fb2>] __kfree_skb+0x12/0xa0 > [172635.399861] [<ffffffff815b551c>] consume_skb+0x2c/0x80 > [<ffffffff815b666b>] ? __alloc_skb+0x8b/0x2a0 > [172635.399870] [<ffffffff815d9c6c>] netpoll_send_skb_on_dev+0x18c/0x3a0 > [<ffffffff810e8ffc>] ? watchdog_overflow_callback+0x9c/0xd0 > [172635.399884] [<ffffffff816b78c1>] printk+0x67/0x69 > [<ffffffff810588cc>] warn_slowpath_fmt+0x4c/0x50 > [172635.399891] [<ffffffff8109170d>] ? sched_clock_cpu+0xbd/0x110 > [172635.399893] [<ffffffff810e8f60>] ? touch_nmi_watchdog+0x80/0x80 > [172635.399894] [<ffffffff810e8ffc>] watchdog_overflow_callback+0x9c/0xd0 > [172635.399896] [172635.399903] [172635.399907] [172635.399912] > [<ffffffff813536d9>] ? __read_lock_failed+0x9/0x20 > [172635.399914] [<ffffffff813536d9>] ? __read_lock_failed+0x9/0x20 > [172635.399916] <<EOE>> [<ffffffff816c3e13>] _raw_read_lock+0x13/0x20 > [172635.399918] [<ffffffff811a4532>] send_sigio+0x52/0xf0 > [<ffffffff816bb2a9>] create_elf_tables+0xaa/0x614 > [172635.399929] [172635.399939] [<ffffffff8119b07d>] sys_execve+0x3d/0x60 > [172635.399941] [<ffffffff816ccb49>] stub_execve+0x69/0xc0 > [172635.399943] ---[ end trace 5ccb38c703860d5a ]--- > [172635.399944] ------------[ cut here ]------------ > nls_iso8859_1(F) usb_storage(F) parport_pc(F) ppdev(F) bluetooth > binfmt_misc(F) snd_hda_intel videobuf2_vmalloc videobuf2_memops > mac80211 videobuf2_core snd_hda_codec videodev snd_hwdep(F) snd_pcm(F) > snd_page_alloc(F) joydev(F) iwlwifi kvm_intel snd_seq_midi(F) > snd_seq_midi_event(F) dm_multipath(F) snd_seq(F) snd_seq_device(F) > tpm_infineon snd_timer(F) > -- > To unsubscribe from this list: send the line "unsubscribe linux-kernel" in > the body of a message to majord...@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html > Please read the FAQ at http://www.tux.org/lkml/ -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/