On Wednesday, August 24, 2011 1:57:59 PM Aki Yoshida wrote: > 2011/8/22 Daniel Kulp <dk...@apache.org>: > > On Monday, August 22, 2011 10:26:36 AM Aki Yoshida wrote: > >> Hi Dan, > >> I was confused about the original intention of this wait(20) code. I > >> didn't understand why it was important to wait 20msec and let the > >> executor thread start instead of not waiting. The comment in the code > >> says the intention is to try to keep the ordering of the messages from > >> the client. But I had an impression that this code would not be > >> helping in this matter. So I initially thought that we could just get > >> rid of this wait to avoid the reported blocking situation. If keeping > >> the ordering is important, shouldn't we introduce this mechanism > >> optionally instead of trying to put this in there implicitly? > > > > Basically, if a client proxy sends a one-way request, and then > > immediately follows it up with a two-way request, we were seeing (even > > in our unit tests) where, in most situations, the two-way would get > > dispatched into the impl before the one-way. While perfectly valid, > > this was kind of confusing to users. Thus we added a very short wait > > there to make sure the runnable has been started on a thread which > > usually allows it to proceed something closer to in order. If the > > queue is full or all threads are busy, the wait will timeout and the > > one-way will get there eventually, but in most cases this works fairly > > well. > > > > Basically, it's a "best effort, but no guarantees". If they need to > > make sure it's there in order, they would need two ways or use WS-RM. > > Thanks for the explanation. I am sorry to hear that some people are > not happy when a oneway call gets passed by a later invoked req-resp > call. I personally think we could get rid of this wait(20) to make a > oneway call more efficient.
In MOST cases, it doesn't wait for 20 ms, it's usually on the order of nano- seconds. It's only when the queue starts filling with lots of one-ways does it ever wait that long. That said, I'd definitely support dropping this down to a wait(5) or so. I'm really not sure why 20 was chosen. Even a "Thread.yield()" may work, but the wait with the lock is more accurate, especially on multi-core machines. > If someone wants to keep the invocation ordering of oneway and > req-resp calls from a single client and while keeping it as it is and > simple, could he also use the USE_ORIGINAL_THREAD option of this > interceptor or was this option intended to be a workaround for the > ordering problem? Well, that's actually there for things like transactions. If using JMS with transactions or even http with Spring transactions, we need to keep on the original thread so the transaction is properly associated with the request. If we jump threads and the original thread unwinds, the transaction may close. However, with USE_ORIGINAL_THREAD, it does consume the main thread which when running in a Sevlet container would consume a thread for http processing. For a long running process, that's likely not a good thing. Thus, it's really not a "good" option for long running one-ways. Dan > > thanks. > > regards, aki > > > Dan > > > >> Thanks. > >> regards, aki > >> > >> 2011/8/19 Daniel Kulp <dk...@apache.org>: > >> > On Friday, August 19, 2011 2:32:03 PM xuhb wrote: > >> >> Maybe it is becuase before chain.wait() return, the sync of > >> >> chain > >> >> will be re-locked, so it will block untill chain.resume() > >> >> finished; > >> > > >> > Thanks for the test case. There definitely is an issue here. > >> > At this point, the thread has already synchronized on the chain > >> > (due to the doIntercept method on the PhaseInterceptorChain being > >> > synchronized). Thus, some of the synchronized blocks in there > >> > don't look correct. If I create a simple: > >> > > >> > final Object lock = new Object(); > >> > > >> > in there, then that problem goes away. > >> > > >> > That said, I hit another couple of issues with your testcase as > >> > well > >> > that I'm looking into. :-( > >> > > >> > Dan > >> > > >> >> ----- Original Message ----- > >> >> From: "xuhb" <x...@tongtech.com> > >> >> To: <users@cxf.apache.org> > >> >> Sent: Friday, August 19, 2011 1:39 PM > >> >> Subject: Re: A mysteriously deadlock of CXF > >> >> OnewayProcessorInterceptor>> >> > >> >> > Sorry, I foget post the issue link: > >> >> > > >> >> > https://issues.apache.org/jira/browse/CXF-3750 > >> >> > > >> >> > ----- Original Message ----- > >> >> > From: "xuhb" <x...@tongtech.com> > >> >> > To: <users@cxf.apache.org> > >> >> > Sent: Friday, August 19, 2011 1:34 PM > >> >> > Subject: A mysteriously deadlock of CXF > >> >> > OnewayProcessorInterceptor > >> >> > > >> >> >> Hi: > >> >> >> > >> >> >> Recently when I am checking/testing CXF , there is a > >> >> >> mysteriously deadlock of CXF Oneway Process; Normally > >> >> >> CXF > >> >> >> engine will invoke the one way bussiness logical > >> >> >> asynchronized > >> >> >> ,, so the servlet handle will finished and return back > >> >> >> to > >> >> >> servlet engine immediately; > >> >> >> > >> >> >> > >> >> >> > >> >> >> But sometime, I noticed that the servlet > >> >> >> handle(JettyHTTPHandler) at server side doesn't return > >> >> >> back > >> >> >> to > >> >> >> servlet engine(Jetty) immediately , it will waiting > >> >> >> until the > >> >> >> asynchrouse business logical finished; > >> > > >> > After dig source of > >> > > >> >> >> CXF, I find it 's relate to > >> >> >> OnewayProcessorInterceptor;But > >> >> >> until now I can only show when will the deadlock > >> >> >> occurs, but > >> >> >> I > >> >> >> still can not explain why;>> > >> >> >> > >> >> >> Following is details: > >> >> >> OnewayProcessInterceptor.handleMessage{ > >> >> >> synchronized (chain) { > >> >> >> message.getExchange().get(Bus.class).getExtension(WorkQueu > >> >> >> eManag > >> >> >> er.cla ss) > >> > > >> > .getAutomaticWorkQueue().execute(new Runnable() { > >> > > >> >> >> public void run() { > >> >> >> synchronized (chain) { > >> >> >> > >> >> >> System.out.println("--notify all"); > >> >> >> chain.notifyAll(); > >> >> >> > >> >> >> } > >> >> >> > >> >> >> chain.resume(); //if chain.resume is called before > >> >> >> chain.wait > >> >> >> finished , the dead lock will occurs; It seems as > >> >> >> chain.resume is synchronized, so it will relock on > >> >> >> chain > >> >> >> object, so the chain.wait() will deadlocked (... I feel > >> >> >> confused for this, because jdk doesn't say so...) > >> >> >> ;After > >> >> >> chain.resume finished, locking on chain is released, > >> >> >> deadlock > >> >> >> of chain.wait() is also released; but I don't think > >> >> >> this is > >> >> >> problem of CXF , maybe it's jdk's problem ?? I feels > >> >> >> confused; > >> >> >> > >> >> >> } > >> >> >> > >> >> >> }); > >> >> >> > >> >> >> System.out.println("--wait begin"); > >> >> >> chain.wait(20); > >> >> >> System.out.println("--wait end"); > >> >> >> > >> >> >> } > >> >> >> } > >> >> >> syncrhonized PhaseInterceptorChain.resume(){ > >> >> >> > >> >> >> System.out.println("--api chain resume"); > >> >> >> > >> >> >> ... > >> >> >> > >> >> >> } > >> >> >> > >> >> >> if the execute sequence as following, every thing is ok. > >> >> >> there > >> >> >> is no dead lock; > >> >> >> > >> >> >> chain.wait enter > >> >> >> chian.notify invoked > >> >> >> chain.wait return; > >> >> >> chain.resume(); //resume also synchronzed on chain > >> >> >> object; > >> >> >> > >> >> >> > >> >> >> if the execute sequence as following , dead lock will > >> >> >> occurs: > >> >> >> > >> >> >> chain.wait enter > >> >> >> chain.notify > >> >> >> chain.resume// ..now waiting on chain will blocked > >> >> >> until > >> >> >> chain.resume finished(release sync on chain) > >> > > >> > chain.wait > >> > > >> >> >> return; > >> >> >> > >> >> >> > >> >> >> following dump on console indicate the above sequence: > >> >> >> > >> >> >> No DeadLock dump : > >> >> >> --wait begin > >> >> >> --notify all > >> >> >> --wait end > >> >> >> --api chain resume > >> >> >> product service begin Fri Aug 19 12:10:28 CST 2011 //a > >> >> >> lone > >> >> >> time(10 > >> >> >> seconds) one way business logical begin > >> > > >> > product service end Fri Aug > >> > > >> >> >> 19 12:10:38 CST 2011 .//a lone time(10 seconds) one way > >> >> >> business > >> >> >> logical end; > >> >> >> > >> >> >> DeadLock Dump: > >> >> >> --wait begin > >> >> >> --notify all > >> >> >> --api chain resume > >> >> >> product service begin Fri Aug 19 12:10:40 CST 2011 > >> >> >> product service end Fri Aug 19 12:10:50 CST 2011 > >> >> >> --wait end > >> >> >> > >> >> >> > >> >> >> Until now I am not sure if problem is CXF's or JDK's, or > >> >> >> something > >> >> >> which I don't know cause such a deadlock; > >> > > >> > I also wrote a simple > >> > > >> >> >> program to simulate the execute sequnce which causeddead > >> >> >> lock > >> >> >> in > >> >> >> CXF, but the simple program never dead lock; > >> >> >> I tried CXF 2.3.3 version && Jetty transport && (JDK1.5_22 > >> >> >> || > >> >> >> JDK > >> >> >> 1.6_17) && Windows XP system; > >> > > >> >> >> I also post this question as a JIRA issue: > >> > -- > >> > Daniel Kulp > >> > dk...@apache.org > >> > http://dankulp.com/blog > >> > Talend - http://www.talend.com > > > > -- > > Daniel Kulp > > dk...@apache.org > > http://dankulp.com/blog > > Talend - http://www.talend.com -- Daniel Kulp dk...@apache.org http://dankulp.com/blog Talend - http://www.talend.com
