On 10/02/2016 11:00, Mark Thomas wrote: > On 10/02/2016 09:20, Mark Thomas wrote: >> On 09/02/2016 16:19, Jeroen van Ooststroom wrote: >> >>>> Hello Mark, >>>> >>>> I have been trying to get an isolated test case reproducing the issue, >>>> but so far without luck. I'll try to reproduce it again with the >>>> original application and get a thread dump for you at the time it got >>>> stuck. Is there any Tomcat-level logging you want to see as well >>>> leading up to the stuck thread issue? >>>> >>>> Thanks, >>>> Jeroen... >>> Hello Mark, >>> >>> See the thread dump attached. >> >> Thanks. Is it edited? There are some "..." sections in the thread dump >> that don't look normal. >> >> A further piece of information that would be helpful is the version of >> IceFaces you are using. > > I've been looking at the source code for IcePush 4.1.0 and I have a > theory. Keep in mind I don't know the IcePush code at all. > > 1. Container Thread A ends up in AsyncAdaptingServlet.service() > This calls request.startAsync() followed by pushServer.service(). > This moves the async state to STARTING which will only change to > STARTED once the container thread exists the > AsyncAdaptingServlet.service() > > 2. Assuming (as it appears from the stack traces) that pushServer is an > instance of BlockingConnectionServer, this calls activeServer.service > > 3. activeServer is an instance of RunningServer. The serviceMethod calls > pendingRequest.put(pushRequest). Call this push request A. > > 4. Meanwhile, non-container thread B is running the "Notification Queue > Consumer". It holds a lock on the BlockingConnectionServer instance. > It tries to process push request A and calls complete(). > This blocks because the AsyncContext is still in state STARTING for > that request. > > 5. Container thread A continues and reaches sendNotifications(). This > blocks because that method is synchronized and non-container thread B > currently holds the lock. > > At this point deadlock occurs. > > The async request is in state STARTING. > > That state can't change until container thread A exits the service() method. > > Container Thread A is blocked in the service() method waiting for a lock > on BlockingConnectionServer. > > Container Thread B holds the lock on BlockingConnectionServer and won't > release it until the the async request changes state to STARTED. > > > My guess is that the deadlock only occurs if the "Notification Queue > Consumer" happens to try processing pending requests in the short period > of time between container thread A calling > pendingRequest.put(pushRequest) and sendNotifications(). > > The Servlet spec is clear that any call to AsyncContext.complete() will > not complete until the associated container thread has completed > processing the original request. On that basis it is my current view > that this is an ICEpush bug. > > You should be able to confirm this fairly easily is you use a debugge
Bah. Hit send too soon. You should be able to confirm this fairly easily if you use a debugger to pause a container thread between the calls to pendingRequest.put(pushRequest) and sendNotifications(). That should then trigger the problem. Mark --------------------------------------------------------------------- To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
