On 10/08/2013 10:18, Robert Mustacchi wrote:
On 10/7/13 22:32 , Alexander Pyhalov wrote:
Hello.
While working on rtorrent OpenIndiana component I've seen stange
sporadic application hangs. Application becomes unresponsive and can't
be killed without reboot.
After some debugging I've found that it actively uses mmap/munmap
functions and sometimes hangs in munmap system call.
It enters in munmap() with m_ptr=FD213000 m_end=FD413800 and never
leaves it.
Dtracing also showed that in this case munmap::entry probe is hitted,
but munmap::return is never reached. How can I debug this further?
There are a few things to do, but the first case is to simply use mdb to
get a sense of where the threads in question are. As a privileged user
you'll want to run 'mdb -k'. Figure out the process id that you care
about and at the prompt you'll want to run the following:
0t<pid>::pid2proc | ::walk thread | ::findstack -v
... output ...
$q
# mdb -k
Loading modules: [ unix genunix specfs dtrace mac cpu.generic uppc
pcplusmp scsi_vhci zfs qlc fctl fcp sd ip hook neti sockfs arp usba uhci
stmf stmf_sbd md lofs random idm crypto sppp cpc fcip ufs logindmux ptm
smbsrv nfs nsmb ipc ]
> 0t27270::pid2proc | ::walk thread | ::findstack -v
stack pointer for thread ffffff04fdac17a0: ffffff002065bcc0
[ ffffff002065bcc0 _resume_from_idle+0xf4() ]
ffffff002065bcf0 swtch+0x141()
ffffff002065bd30 cv_wait+0x70(ffffff04fa658ec2, ffffff04fa658eb8)
ffffff002065bde0 as_unmap+0x223(ffffff04fa658eb8, fd213000, 200800)
ffffff002065be20 munmap+0x83(fd213000, 200800)
ffffff002065bec0 dtrace_systrace_syscall32+0xe4(fd213000, 200800,
4dd6c00, fdac17a0, 1, 0)
ffffff002065bf10 _sys_sysenter_post_swapgs+0x149()
stack pointer for thread ffffff04f6326420: ffffff0020d5dc60
[ ffffff0020d5dc60 _resume_from_idle+0xf4() ]
ffffff0020d5dc90 swtch+0x141()
ffffff0020d5dd30 cv_timedwait_sig_hires+0x39d(ffffff04f632660e,
ffffff04f6326610, 27448c396e3e, 1, 3)
ffffff0020d5dd60 cv_timedwait_sig_hrtime+0x2a(ffffff04f632660e,
ffffff04f6326610, 27448c396e3e)
ffffff0020d5de20 poll_common+0x43c(fe4beb00, 0, ffffff0020d5de40, 0)
ffffff0020d5dec0 pollsys+0xe3(fe4beb00, 0, fe4beba8, 0)
ffffff0020d5df10 _sys_sysenter_post_swapgs+0x149()
stack pointer for thread ffffff0505c6c060: ffffff001eabdc60
[ ffffff001eabdc60 _resume_from_idle+0xf4() ]
ffffff001eabdc90 swtch+0x141()
ffffff001eabdd30 cv_timedwait_sig_hires+0x39d(ffffff0505c6c24e,
ffffff0505c6c250, 276e690ddc46, 1, 3)
ffffff001eabdd60 cv_timedwait_sig_hrtime+0x2a(ffffff0505c6c24e,
ffffff0505c6c250, 276e690ddc46)
ffffff001eabde20 poll_common+0x43c(fe13eb00, 0, ffffff001eabde40, 0)
ffffff001eabdec0 pollsys+0xe3(fe13eb00, 0, fe13eba8, 0)
ffffff001eabdf10 _sys_sysenter_post_swapgs+0x149()
I'm just a bit confused: is munmap supposed to wait for something in any
case? I thought it just have to return with error if something is wrong...
It seems that application also uses madvise with MADV_WILLNEED on its
mapped memory. Can it influence the munmap behavior?
--
Best regards,
Alexander Pyhalov,
system administrator of Computer Center of Southern Federal University
-------------------------------------------
illumos-discuss
Archives: https://www.listbox.com/member/archive/182180/=now
RSS Feed: https://www.listbox.com/member/archive/rss/182180/21175430-2e6923be
Modify Your Subscription:
https://www.listbox.com/member/?member_id=21175430&id_secret=21175430-6a77cda4
Powered by Listbox: http://www.listbox.com
