Pierre, I applied a fix last week (
https://issues.apache.org/jira/browse/UIMA-2105) that enables recovery of
temp reply queue listener after broker failure and restart. With the fix,
I've tried to create a failing scenario but so far I dont see the hang. In
my testcase I have an aggregate UIMA AS service with one remote delegate.
The timeout on the remote delegate is configured to be 3 secs. The remote
delegate is configured with 1 CAS, no scaleup. The aggregate has a simple CM
producing 1 CAS per each input CAS. The application client is essentially
this code:
Map<String, Object> appCtx =
buildContext("tcp://localhost:61616", "TopLevelTaeQueue");
appCtx.put(UimaAsynchronousEngine.Timeout, 3000);
appCtx.put(UimaAsynchronousEngine.CasPoolSize, 3);
deployService(eeUimaEngine, relativePath + "/Deploy_NoOpAnnotator.xml");
// delegate
deployService(eeUimaEngine, relativePath +
"/Deploy_AggregateAnnotator.xml");
initialize(eeUimaEngine, appCtx);
waitUntilInitialized();
for (int i = 0; i < 3000; i++) {
CAS cas = eeUimaEngine.getCAS();
eeUimaEngine.sendCAS(cas);
}
While the app client is running I kill the broker, wait a few seconds and
start a new broker. I see timeouts both in the aggregate and the client. I
see that the aggregate fails to send replies (Timeout Exceptions) to the app
client due to temp queue being bad. After awhile the connections are
re-established, new listeners created on the remote services and processing
continues without errors.
JC
On Thu, Mar 31, 2011 at 1:55 PM, Jaroslaw Cwiklik <uim...@gmail.com> wrote:
> Pierre, thanks I will try to replicate the scenario to see what is
> preventing CASes from timing out in the application client.
>
> JC
>
>
> On Thu, Mar 31, 2011 at 11:03 AM, Pierre Pujalon <ppuja...@yahoo.fr>wrote:
>
>> JC, I have this timeout setted already then. The callback only receives
>> the timeout exception for the first CAS.
>> This only occurs when I have a delegated annotator.
>> When I configure my application with aggregates without any delegate
>> annotator, the callback receives timeout from all CASes.
>>
>> Pierre
>>
>>
>> Le 31/03/2011 16:40, Jaroslaw Cwiklik a écrit :
>>
>> Pierre, to set a timeout on the aggregate do the following in the
>>> application client:
>>>
>>> //create Asynchronous Client API
>>> uimaAsEngine = new BaseUIMAAsynchronousEngine_impl();
>>> //create Map to pass server URI and Endpoint parameters
>>> Map<String,Object> appCtx = new HashMap<String,Object>();
>>> ...
>>> appCtx.put(UimaAsynchronousEngine.Timeout, 500); // 500ms timeout on
>>> Process
>>> CAS
>>> uimaAsEngine.initialize(appCtx);
>>> ...
>>>
>>> This should help with recovery of the CASes with invalid reply queue.
>>> NOTE: if you listener plugged in to the uimaASEngine you will receive
>>> callback with Timeout exception.
>>> In your code you need to decide whether to retry CAS or not.
>>>
>>> JC
>>>
>>>
>>> On Thu, Mar 31, 2011 at 9:55 AM, Pierre Pujalon<ppuja...@yahoo.fr>
>>> wrote:
>>>
>>> JC
>>>>
>>>> I have a 15000ms process timeout for the UIMAEngine and I have a 15000ms
>>>> timeout for the remoteDelegate
>>>>
>>>> I haven't found how to set a timeout for the aggregate to be honest.
>>>>
>>>> You are right I have this exception in the console
>>>> javax.jms.InvalidDestinationException: Cannot publish to a deleted
>>>> Destination: temp-queue://ID:${brokerUrl}-55450-1301578320924-0:0:1
>>>> at
>>>> org.apache.activemq.ActiveMQSession.send(ActiveMQSession.java:1696)
>>>> at
>>>>
>>>> org.apache.activemq.ActiveMQMessageProducer.send(ActiveMQMessageProducer.java:231)
>>>> at
>>>>
>>>> org.apache.activemq.ActiveMQMessageProducerSupport.send(ActiveMQMessageProducerSupport.java:300)
>>>> at
>>>>
>>>> org.apache.uima.adapter.jms.activemq.JmsEndpointConnection_impl.send(JmsEndpointConnection_impl.java:574)
>>>>
>>>> at
>>>>
>>>> org.apache.uima.adapter.jms.activemq.JmsEndpointConnection_impl.send(JmsEndpointConnection_impl.java:507)
>>>> at
>>>>
>>>> org.apache.uima.adapter.jms.activemq.JmsOutputChannel.dispatch(JmsOutputChannel.java:1366)
>>>> at
>>>>
>>>> org.apache.uima.adapter.jms.activemq.JmsOutputChannel.sendCasToRemoteEndpoint(JmsOutputChannel.java:1527)
>>>> at
>>>>
>>>> org.apache.uima.adapter.jms.activemq.JmsOutputChannel.sendReply(JmsOutputChannel.java:689)
>>>> at
>>>>
>>>> org.apache.uima.aae.controller.AggregateAnalysisEngineController_impl.sendReplyToRemoteClient(AggregateAnalysisEngineController_impl.java:2033)
>>>> at
>>>>
>>>> org.apache.uima.aae.controller.AggregateAnalysisEngineController_impl.replyToClient(AggregateAnalysisEngineController_impl.java:2170)
>>>> at
>>>>
>>>> org.apache.uima.aae.controller.AggregateAnalysisEngineController_impl.finalStep(AggregateAnalysisEngineController_impl.java:1734)
>>>> at
>>>>
>>>> org.apache.uima.aae.controller.AggregateAnalysisEngineController_impl.executeFlowStep(AggregateAnalysisEngineController_impl.java:2317)
>>>>
>>>> at
>>>>
>>>> org.apache.uima.aae.controller.AggregateAnalysisEngineController_impl.process(AggregateAnalysisEngineController_impl.java:1229)
>>>> at
>>>>
>>>> org.apache.uima.aae.handler.HandlerBase.invokeProcess(HandlerBase.java:118)
>>>> at
>>>>
>>>> org.apache.uima.aae.handler.input.ProcessResponseHandler.cancelTimerAndProcess(ProcessResponseHandler.java:108)
>>>> at
>>>>
>>>> org.apache.uima.aae.handler.input.ProcessResponseHandler.handleProcessResponseWithCASReference(ProcessResponseHandler.java:387)
>>>> at
>>>>
>>>> org.apache.uima.aae.handler.input.ProcessResponseHandler.handle(ProcessResponseHandler.java:656)
>>>> at
>>>> org.apache.uima.aae.handler.HandlerBase.delegate(HandlerBase.java:149)
>>>> at
>>>>
>>>> org.apache.uima.aae.handler.input.ProcessRequestHandler_impl.handle(ProcessRequestHandler_impl.java:973)
>>>> at
>>>>
>>>> org.apache.uima.aae.spi.transport.vm.UimaVmMessageListener.onMessage(UimaVmMessageListener.java:107)
>>>> at
>>>>
>>>> org.apache.uima.aae.spi.transport.vm.UimaVmMessageDispatcher$1.run(UimaVmMessageDispatcher.java:70)
>>>> at
>>>>
>>>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>>>> at
>>>>
>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>>>> at java.lang.Thread.run(Thread.java:619)
>>>>
>>>>
>>>> Thanks
>>>> Pierre
>>>>
>>>> Le 31/03/2011 15:43, Jaroslaw Cwiklik a écrit :
>>>>
>>>> Pierre, since the aggregate received the three CASes and these are
>>>> never
>>>>
>>>>> returned to the application client I suspect that a temp reply queue
>>>>> associated with the three CASes is no longer valid. Check your
>>>>> aggregate
>>>>> log
>>>>> for exceptions. Perhaps the aggregate is trying to return the CASes but
>>>>> it
>>>>> fails on send since the temp reply queue no longer exists. Just to be
>>>>> clear
>>>>> do you define Process timeout in your application client? It wasnt
>>>>> clear
>>>>> to
>>>>> me if the timeout of 15000ms is in the aggregate client or the
>>>>> application
>>>>> client.
>>>>>
>>>>> JC
>>>>>
>>>>> On Thu, Mar 31, 2011 at 9:24 AM, Pierre Pujalon<ppuja...@yahoo.fr>
>>>>> wrote:
>>>>>
>>>>> JC,
>>>>>
>>>>>> I had to work on this topic and add instrumentation a bit to try to
>>>>>> have
>>>>>> a
>>>>>> clearer vision. The basic idea is to test robustness of the processing
>>>>>> when
>>>>>> the ActiveMQ have a downtime (generally only for milliseconds). This
>>>>>> scenario is a real life scenario.
>>>>>>
>>>>>> I have an application client using the aggregate client with
>>>>>> UimaAsynchronousEngine. I have a pool of 4 CASes, a timeout set to
>>>>>> 15000ms
>>>>>> in the client and not CASMultiplier in the aggregate.
>>>>>>
>>>>>> When JMS gets down and up. I receive in my CallbackListener only one
>>>>>> exception out of the four CASes (the 15000ms timeout) . The three
>>>>>> others
>>>>>> never trigger any timeout.
>>>>>> Actually, it seems that thoses CASes are being processed in the
>>>>>> aggregate
>>>>>> as I have the trace of the annotators in the aggregate log. But they
>>>>>> never
>>>>>> go back to the reply queue.
>>>>>>
>>>>>> I traced the CASId that the engine sends and receives in the listener.
>>>>>> Let's say I have id1, Id2, id3 and id4.
>>>>>>
>>>>>> As the JMS gets down and up, one of the case (id1) is released . The
>>>>>> application client send another CAS . As this very point, the method
>>>>>> sendCas
>>>>>> returns a CASId id5 but in the CallbackListener the
>>>>>> onBeforeMessageSend
>>>>>> status tells me the CASid is id4. It is confirmed with the aggregate
>>>>>> which
>>>>>> logs the message :
>>>>>> Duplicate Request With Cas Reference Id: (id4) Received. Ignoring
>>>>>> Duplicate.
>>>>>>
>>>>>> The CasPool in the UIMAEngines is full and the client stalls.
>>>>>> Restarting the UIMAAsynchronousEngine is the only solution to continue
>>>>>> processing, the ag
>>>>>> My guess is that it comes from the UIMAAsynchronousEngine.
>>>>>> I hope it is clear
>>>>>>
>>>>>> Pierre
>>>>>> PS : I set the timeout to 5 seconds, and I have a total different
>>>>>> scenario
>>>>>> which is no better :)
>>>>>>
>>>>>>
>>>>>>
>>>>>> Le 29/03/2011 18:56, Jaroslaw Cwiklik a écrit :
>>>>>>
>>>>>> Pierre, do you declare a timeout on Process in your aggregate
>>>>>> deployment
>>>>>>
>>>>>> descriptor for the remote delegate? If not, perhaps CASes were sent
>>>>>>> to
>>>>>>> the
>>>>>>> remote delegate and subsequently dropped because the delegate was not
>>>>>>> unable
>>>>>>> to deliver them to the aggregate client. A timeout would help to
>>>>>>> recover
>>>>>>> these CASes.
>>>>>>>
>>>>>>> Do you know where you are running out of CASes? Is it in application
>>>>>>> client
>>>>>>> or aggregate client? Use jConsole to attach to each jvm and examine
>>>>>>> the
>>>>>>> threads. Do you use a Cas Multiplier in the aggregate? How big is its
>>>>>>> pool?
>>>>>>>
>>>>>>> -JC
>>>>>>>
>>>>>>> On Tue, Mar 29, 2011 at 12:03 PM, Pierre Pujalon<ppuja...@yahoo.fr>
>>>>>>> wrote:
>>>>>>>
>>>>>>> JC,
>>>>>>>
>>>>>>> I tested the work around and a listener to the delegate is indeed
>>>>>>>> recreated. The CAS that were being processed pass through all
>>>>>>>> annotators.
>>>>>>>> But now I have other issues, but it is not the same each time I
>>>>>>>> tried
>>>>>>>> .Either the client is losing connection, or get block on the getCAS
>>>>>>>> method
>>>>>>>> as if cas process during the JMS down time were not release or the
>>>>>>>> aggregate
>>>>>>>> or the aggregate client get stalled...
>>>>>>>> I need to dig a little bit into this because it does not make many
>>>>>>>> sense
>>>>>>>> right now...
>>>>>>>>
>>>>>>>> What bother me is that if I don't make the annotator delegated, the
>>>>>>>> aggregate client always recover clean with this scenario.
>>>>>>>>
>>>>>>>> Anyway Thanks
>>>>>>>> Pierre
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> Le 28/03/2011 18:07, Jaroslaw Cwiklik a écrit :
>>>>>>>>
>>>>>>>> Pierre, in my last posting I meant maxRetries=1, not retryCount=1.
>>>>>>>>
>>>>>>>> JC
>>>>>>>>
>>>>>>>>> On Mon, Mar 28, 2011 at 12:05 PM, Jaroslaw Cwiklik<
>>>>>>>>> uim...@gmail.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>> Pierre, I've identified the bug in UIMA AS code that causes the
>>>>>>>>> problem
>>>>>>>>>
>>>>>>>>> described. As a workaround, set the retryCount=1 in the error
>>>>>>>>>
>>>>>>>>>> handling
>>>>>>>>>> configuration for the delegate defined in your deployment
>>>>>>>>>> descriptor.
>>>>>>>>>> Here
>>>>>>>>>> is an example:
>>>>>>>>>>
>>>>>>>>>> <remoteAnalysisEngine key="NoOp">
>>>>>>>>>> <inputQueue endpoint="NoOpAnnotatorQueue"
>>>>>>>>>> brokerURL="${BrokerURL}"/>
>>>>>>>>>> <serializer method="xmi"/>
>>>>>>>>>> <asyncAggregateErrorConfiguration>
>>>>>>>>>> <processCasErrors maxRetries="1"
>>>>>>>>>> timeout="15000" thresholdCount="1" thresholdAction="terminate" />
>>>>>>>>>> </asyncAggregateErrorConfiguration>
>>>>>>>>>> </remoteAnalysisEngine>
>>>>>>>>>>
>>>>>>>>>> This should force creation of a new listener on the delegate reply
>>>>>>>>>> queue.
>>>>>>>>>>
>>>>>>>>>> The listener recovery should be done with no retry being set and I
>>>>>>>>>> will
>>>>>>>>>> make necessary changes in the code to support that use case.
>>>>>>>>>>
>>>>>>>>>> Thanks for finding the bug!
>>>>>>>>>>
>>>>>>>>>> JC
>>>>>>>>>>
>>>>>>>>>> On Tue, Mar 22, 2011 at 1:12 PM, Jaroslaw Cwiklik<
>>>>>>>>>> uim...@gmail.com
>>>>>>>>>>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> Pierre, this appears to be a bug. In the scenario you've
>>>>>>>>>>> described
>>>>>>>>>>>
>>>>>>>>>> the
>>>>>>>>>>
>>>>>>>>>> uima as aggregate client should detect a lost connection (which
>>>>>>>>>> it
>>>>>>>>>>
>>>>>>>>>>> does)
>>>>>>>>>>> and
>>>>>>>>>>> create a new one on a subsequent process CAS request destined to
>>>>>>>>>>> the
>>>>>>>>>>> remote
>>>>>>>>>>> delegate. I will attempt to re-create your scenario and see if
>>>>>>>>>>> this
>>>>>>>>>>> is
>>>>>>>>>>> in
>>>>>>>>>>> fact a bug.
>>>>>>>>>>>
>>>>>>>>>>> Thanks, JC
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Tue, Mar 22, 2011 at 7:11 AM, Pierre Pujalon<
>>>>>>>>>>> ppuja...@yahoo.fr
>>>>>>>>>>>
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Hi,
>>>>>>>>>>>>
>>>>>>>>>>> I'm currently using UIMA-AS 2.3.1rc5 and I am having issues
>>>>>>>>>>> with
>>>>>>>>>>>
>>>>>>>>>>>> asynchronous delegated ..
>>>>>>>>>>>>
>>>>>>>>>>>> I have a chain of several annotators. One of them is an
>>>>>>>>>>>> asynchronous
>>>>>>>>>>>> delegate as it is very resource consuming.
>>>>>>>>>>>> So I have one UIMA service with a simpleChainDescriptor and
>>>>>>>>>>>> several
>>>>>>>>>>>> UIMA
>>>>>>>>>>>> services with myDelegateAnnotatorDescriptor.
>>>>>>>>>>>>
>>>>>>>>>>>> When the activeMQ broker fails and restart, all my UIMA
>>>>>>>>>>>> services
>>>>>>>>>>>> resfresh succesfully their JMS connection. But it appears that
>>>>>>>>>>>> the
>>>>>>>>>>>> service
>>>>>>>>>>>> holding the processing chain (simpleChainDescriptor) itself
>>>>>>>>>>>> does
>>>>>>>>>>>> not
>>>>>>>>>>>> refresh the JMS connection to the delegate JMS queue.
>>>>>>>>>>>> When a new CAS is processed, I have the following exception in
>>>>>>>>>>>> the
>>>>>>>>>>>> console !!!! and the processing of the cas stop silently and
>>>>>>>>>>>> passes
>>>>>>>>>>>> to
>>>>>>>>>>>> the
>>>>>>>>>>>> next cas.
>>>>>>>>>>>> If I restart the service holding the chain, everything go back
>>>>>>>>>>>> to
>>>>>>>>>>>> normal.
>>>>>>>>>>>>
>>>>>>>>>>>> Is this a bug or am I missing something?
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks,
>>>>>>>>>>>> Pierre
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> ATTENTION: Uima AS Service:simpleChainDescriptor Listener
>>>>>>>>>>>> Established
>>>>>>>>>>>> Connection to Broker: tcp://brokerurl:61616
>>>>>>>>>>>> 22 mars 2011 11:11:00
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerContainer
>>>>>>>>>>>> handleListenerSetupFailure
>>>>>>>>>>>> ATTENTION: Uima AS Service:simpleChainDescriptor Listener
>>>>>>>>>>>> Established
>>>>>>>>>>>> Connection to Broker: tcp://brokerurl:61616
>>>>>>>>>>>> 22 mars 2011 11:11:00
>>>>>>>>>>>> org.apache.uima.adapter.jms.activemq.JmsOutputChannel
>>>>>>>>>>>> invalidateConnectionAndEndpoints
>>>>>>>>>>>> INFO: Controller simpleChainDescriptor Invalidating JMS
>>>>>>>>>>>> Connection
>>>>>>>>>>>> To
>>>>>>>>>>>> Broker tcp://brokerurl:61616 and Closing Sessions To Delegates
>>>>>>>>>>>> 22 mars 2011 11:11:01
>>>>>>>>>>>> org.apache.uima.adapter.jms.activemq.JmsEndpointConnection_impl
>>>>>>>>>>>> send
>>>>>>>>>>>> ATTENTION: Service: simpleChainDescriptor Runtime Exception
>>>>>>>>>>>> 22 mars 2011 11:11:01
>>>>>>>>>>>> org.apache.uima.adapter.jms.activemq.JmsEndpointConnection_impl
>>>>>>>>>>>> send
>>>>>>>>>>>> ATTENTION:
>>>>>>>>>>>> org.apache.uima.aae.error.DelegateConnectionLostException:
>>>>>>>>>>>> Controller:simpleChainDescriptor Lost Connection to
>>>>>>>>>>>> Delegate:myDelegateAnnotatorDescriptor
>>>>>>>>>>>> at
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> org.apache.uima.adapter.jms.activemq.JmsEndpointConnection_impl.send(JmsEndpointConnection_impl.java:536)
>>>>>>>>>>>> at
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> org.apache.uima.adapter.jms.activemq.JmsEndpointConnection_impl.send(JmsEndpointConnection_impl.java:507)
>>>>>>>>>>>> at
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> org.apache.uima.adapter.jms.activemq.JmsOutputChannel.dispatch(JmsOutputChannel.java:1366)
>>>>>>>>>>>> at
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> org.apache.uima.adapter.jms.activemq.JmsOutputChannel.sendCasToRemoteEndpoint(JmsOutputChannel.java:1527)
>>>>>>>>>>>> at
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> org.apache.uima.adapter.jms.activemq.JmsOutputChannel.serializeCasAndSend(JmsOutputChannel.java:658)
>>>>>>>>>>>> at
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> org.apache.uima.adapter.jms.activemq.JmsOutputChannel.sendRequest(JmsOutputChannel.java:610)
>>>>>>>>>>>> at
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> org.apache.uima.aae.controller.AggregateAnalysisEngineController_impl.dispatch(AggregateAnalysisEngineController_impl.java:2381)
>>>>>>>>>>>> at
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> org.apache.uima.aae.controller.AggregateAnalysisEngineController_impl.dispatchProcessRequest(AggregateAnalysisEngineController_impl.java:2421)
>>>>>>>>>>>> at
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> org.apache.uima.aae.controller.AggregateAnalysisEngineController_impl.simpleStep(AggregateAnalysisEngineController_impl.java:1294)
>>>>>>>>>>>> at
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> org.apache.uima.aae.controller.AggregateAnalysisEngineController_impl.executeFlowStep(AggregateAnalysisEngineController_impl.java:2302)
>>>>>>>>>>>> at
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> org.apache.uima.aae.controller.AggregateAnalysisEngineController_impl.process(AggregateAnalysisEngineController_impl.java:1229)
>>>>>>>>>>>> at
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> org.apache.uima.aae.handler.HandlerBase.invokeProcess(HandlerBase.java:118)
>>>>>>>>>>>> at
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> org.apache.uima.aae.handler.input.ProcessResponseHandler.cancelTimerAndProcess(ProcessResponseHandler.java:108)
>>>>>>>>>>>> at
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> org.apache.uima.aae.handler.input.ProcessResponseHandler.handleProcessResponseWithCASReference(ProcessResponseHandler.java:387)
>>>>>>>>>>>> at
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> org.apache.uima.aae.handler.input.ProcessResponseHandler.handle(ProcessResponseHandler.java:656)
>>>>>>>>>>>> at
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> org.apache.uima.aae.handler.HandlerBase.delegate(HandlerBase.java:149)
>>>>>>>>>>>> at
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> org.apache.uima.aae.handler.input.ProcessRequestHandler_impl.handle(ProcessRequestHandler_impl.java:973)
>>>>>>>>>>>> at
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> org.apache.uima.aae.spi.transport.vm.UimaVmMessageListener.onMessage(UimaVmMessageListener.java:107)
>>>>>>>>>>>> at
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> org.apache.uima.aae.spi.transport.vm.UimaVmMessageDispatcher$1.run(UimaVmMessageDispatcher.java:70)
>>>>>>>>>>>> at
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>>>>>>>>>>>> at
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>>>>>>>>>>>> at java.lang.Thread.run(Thread.java:619)
>>>>>>>>>>>> 22 mars 2011 11:11:01
>>>>>>>>>>>> org.apache.uima.aae.error.handler.ProcessCasErrorHandler
>>>>>>>>>>>> handleError
>>>>>>>>>>>> ATTENTION: Service: simpleChainDescriptor Runtime Exception
>>>>>>>>>>>> 22 mars 2011 11:11:01
>>>>>>>>>>>> org.apache.uima.aae.error.handler.ProcessCasErrorHandler
>>>>>>>>>>>> handleError
>>>>>>>>>>>> ATTENTION:
>>>>>>>>>>>> org.apache.uima.aae.error.DelegateConnectionLostException:
>>>>>>>>>>>> Controller:simpleChainDescriptor Lost Connection to
>>>>>>>>>>>> Delegate:myDelegateAnnotatorDescriptor
>>>>>>>>>>>> at
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> org.apache.uima.adapter.jms.activemq.JmsEndpointConnection_impl.send(JmsEndpointConnection_impl.java:536)
>>>>>>>>>>>> at
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> org.apache.uima.adapter.jms.activemq.JmsEndpointConnection_impl.send(JmsEndpointConnection_impl.java:507)
>>>>>>>>>>>> at
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> org.apache.uima.adapter.jms.activemq.JmsOutputChannel.dispatch(JmsOutputChannel.java:1366)
>>>>>>>>>>>> at
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> org.apache.uima.adapter.jms.activemq.JmsOutputChannel.sendCasToRemoteEndpoint(JmsOutputChannel.java:1527)
>>>>>>>>>>>> at
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> org.apache.uima.adapter.jms.activemq.JmsOutputChannel.serializeCasAndSend(JmsOutputChannel.java:658)
>>>>>>>>>>>> at
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> org.apache.uima.adapter.jms.activemq.JmsOutputChannel.sendRequest(JmsOutputChannel.java:610)
>>>>>>>>>>>> at
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> org.apache.uima.aae.controller.AggregateAnalysisEngineController_impl.dispatch(AggregateAnalysisEngineController_impl.java:2381)
>>>>>>>>>>>> at
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> org.apache.uima.aae.controller.AggregateAnalysisEngineController_impl.dispatchProcessRequest(AggregateAnalysisEngineController_impl.java:2421)
>>>>>>>>>>>> at
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> org.apache.uima.aae.controller.AggregateAnalysisEngineController_impl.simpleStep(AggregateAnalysisEngineController_impl.java:1294)
>>>>>>>>>>>> at
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> org.apache.uima.aae.controller.AggregateAnalysisEngineController_impl.executeFlowStep(AggregateAnalysisEngineController_impl.java:2302)
>>>>>>>>>>>> at
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> org.apache.uima.aae.controller.AggregateAnalysisEngineController_impl.process(AggregateAnalysisEngineController_impl.java:1229)
>>>>>>>>>>>> at
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> org.apache.uima.aae.handler.HandlerBase.invokeProcess(HandlerBase.java:118)
>>>>>>>>>>>> at
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> org.apache.uima.aae.handler.input.ProcessResponseHandler.cancelTimerAndProcess(ProcessResponseHandler.java:108)
>>>>>>>>>>>> at
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> org.apache.uima.aae.handler.input.ProcessResponseHandler.handleProcessResponseWithCASReference(ProcessResponseHandler.java:387)
>>>>>>>>>>>> at
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> org.apache.uima.aae.handler.input.ProcessResponseHandler.handle(ProcessResponseHandler.java:656)
>>>>>>>>>>>> at
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> org.apache.uima.aae.handler.HandlerBase.delegate(HandlerBase.java:149)
>>>>>>>>>>>> at
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> org.apache.uima.aae.handler.input.ProcessRequestHandler_impl.handle(ProcessRequestHandler_impl.java:973)
>>>>>>>>>>>> at
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> org.apache.uima.aae.spi.transport.vm.UimaVmMessageListener.onMessage(UimaVmMessageListener.java:107)
>>>>>>>>>>>> at
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> org.apache.uima.aae.spi.transport.vm.UimaVmMessageDispatcher$1.run(UimaVmMessageDispatcher.java:70)
>>>>>>>>>>>> at
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>>>>>>>>>>>> at
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>>>>>>>>>>>> at java.lang.Thread.run(Thread.java:619)
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>
