Re: How to troubleshoot stuck distributed function calls

2015-12-16 Thread Barry Oglesby
> *Sent:* Wednesday, December 16, 2015 12:25 AM > *To:* user@geode.incubator.apache.org > *Subject:* Re: How to troubleshoot stuck distributed function calls > > I think it depends on how the function is being invoked. Below is an > example with two peers using the onMembers API. I

Re: How to troubleshoot stuck distributed function calls

2015-12-15 Thread Hovhannes Antonyan
From: Barry Oglesby mailto:bogle...@pivotal.io>> Sent: Tuesday, December 15, 2015 11:37 PM To: user@geode.incubator.apache.org<mailto:user@geode.incubator.apache.org> Subject: Re: How to troubleshoot stuck distributed function calls You'll want to take

Re: How to troubleshoot stuck distributed function calls

2015-12-15 Thread Barry Oglesby
; *Sent:* Tuesday, December 15, 2015 11:37 PM > *To:* user@geode.incubator.apache.org > *Subject:* Re: How to troubleshoot stuck distributed function calls > > You'll want to take thread dumps (not heap dumps) in the members > especially the one that initiated the function call and the one th

Re: How to troubleshoot stuck distributed function calls

2015-12-15 Thread Hovhannes Antonyan
? From: Barry Oglesby Sent: Tuesday, December 15, 2015 11:37 PM To: user@geode.incubator.apache.org Subject: Re: How to troubleshoot stuck distributed function calls You'll want to take thread dumps (not heap dumps) in the members especially the one that initiated the fun

Re: How to troubleshoot stuck distributed function calls

2015-12-15 Thread Barry Oglesby
You'll want to take thread dumps (not heap dumps) in the members especially the one that initiated the function call and the one that didn't send a response. Those will tell you whether the thread processing the function or the thread processing the reply is stuck and if so, where. Barry Oglesby G

Re: How to troubleshoot stuck distributed function calls

2015-12-15 Thread Hovhannes Antonyan
I was looking at the heapdump and identified the node which didn't sent the response. But the question now is why didn't it send it, did it run the function or not yet...? From: Darrel Schneider Sent: Tuesday, December 15, 2015 9:58 PM To: user@geode.incubator.

Re: How to troubleshoot stuck distributed function calls

2015-12-15 Thread Darrel Schneider
Usually the member waiting for a response logs a warning that it has been waiting for longer than 15 seconds from a particular member. Use that member id to identify the member that is not responding. Get a stack dump on that member and look for a thread that is processing the unresponsive message.