[jira] Created: (UIMA-1660) uimaj-as-camel should support failover in configruation string
uimaj-as-camel should support failover in configruation string -- Key: UIMA-1660 URL: https://issues.apache.org/jira/browse/UIMA-1660 Project: UIMA Issue Type: Improvement Components: Async Scaleout Affects Versions: 2.3AS Reporter: Jörn Kottmann Assignee: Jörn Kottmann Fix For: 2.3AS It should be possible to use a failover broker URI in the configuration string. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Closed: (UIMA-1660) uimaj-as-camel should support failover in configruation string
[ https://issues.apache.org/jira/browse/UIMA-1660?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jörn Kottmann closed UIMA-1660. --- Resolution: Fixed > uimaj-as-camel should support failover in configruation string > -- > > Key: UIMA-1660 > URL: https://issues.apache.org/jira/browse/UIMA-1660 > Project: UIMA > Issue Type: Improvement > Components: Async Scaleout >Affects Versions: 2.3AS >Reporter: Jörn Kottmann >Assignee: Jörn Kottmann > Fix For: 2.3AS > > > It should be possible to use a failover broker URI in the configuration > string. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Closed: (UIMA-1657) UIMA AS client fails if a failover broker uri with multiple endpoints is used
[
https://issues.apache.org/jira/browse/UIMA-1657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jörn Kottmann closed UIMA-1657.
---
Resolution: Fixed
Thanks :-)
It works in our setup as expected.
> UIMA AS client fails if a failover broker uri with multiple endpoints is used
> -
>
> Key: UIMA-1657
> URL: https://issues.apache.org/jira/browse/UIMA-1657
> Project: UIMA
> Issue Type: Bug
> Components: Async Scaleout
>Affects Versions: 2.3AS
>Reporter: Jörn Kottmann
> Fix For: 2.3AS
>
>
> The initialize time out if a failover broker uri with multiple endpoints is
> used e.g.:
> failover:(tcp://XXX1:61616,tcp://XXX2:61616)?randomize=false
> On the client side it results in a time out exception:
> Caused by: org.apache.uima.resource.ResourceInitializationException
>at
> org.apache.uima.adapter.jms.client.BaseUIMAAsynchronousEngine_impl.initialize(BaseUIMAAsynchronousEngine_impl.java:637)
>at org.apache.uima.camel.UimaAsProducer.(UimaAsProducer.java:142)
>at
> org.apache.uima.camel.UimaAsEndpoint.createProducer(UimaAsEndpoint.java:89)
>at org.apache.camel.processor.SendProcessor.doStart(SendProcessor.java:93)
>at org.apache.camel.impl.ServiceSupport.start(ServiceSupport.java:49)
>at org.apache.camel.util.ServiceHelper.startService(ServiceHelper.java:42)
>at org.apache.camel.util.ServiceHelper.startServices(ServiceHelper.java:53)
>at
> org.apache.camel.processor.DelegateProcessor.doStart(DelegateProcessor.java:68)
>at org.apache.camel.impl.ServiceSupport.start(ServiceSupport.java:49)
>at org.apache.camel.util.ServiceHelper.startService(ServiceHelper.java:42)
>at org.apache.camel.util.ServiceHelper.startServices(ServiceHelper.java:53)
>at
> org.apache.camel.processor.DelegateProcessor.doStart(DelegateProcessor.java:68)
>at org.apache.camel.impl.ServiceSupport.start(ServiceSupport.java:49)
>at org.apache.camel.util.ServiceHelper.startService(ServiceHelper.java:42)
>at org.apache.camel.util.ServiceHelper.startServices(ServiceHelper.java:53)
>at
> org.apache.camel.processor.DelegateProcessor.doStart(DelegateProcessor.java:68)
>at org.apache.camel.impl.ServiceSupport.start(ServiceSupport.java:49)
>at org.apache.camel.util.ServiceHelper.startService(ServiceHelper.java:42)
>at org.apache.camel.util.ServiceHelper.startServices(ServiceHelper.java:53)
>at
> org.apache.camel.processor.DelegateAsyncProcessor.doStart(DelegateAsyncProcessor.java:54)
>at org.apache.camel.impl.ServiceSupport.start(ServiceSupport.java:49)
>at org.apache.camel.util.ServiceHelper.startService(ServiceHelper.java:42)
>at org.apache.camel.util.ServiceHelper.startServices(ServiceHelper.java:53)
>at org.apache.camel.impl.DefaultConsumer.doStart(DefaultConsumer.java:85)
>at org.apache.camel.component.jms.JmsConsumer.doStart(JmsConsumer.java:58)
>at org.apache.camel.impl.ServiceSupport.start(ServiceSupport.java:49)
>at
> org.apache.camel.impl.DefaultCamelContext.addService(DefaultCamelContext.java:422)
>at
> org.apache.camel.impl.DefaultCamelContext.startRoutes(DefaultCamelContext.java:664)
>at
> org.apache.camel.impl.DefaultCamelContext.doStart(DefaultCamelContext.java:635)
>at
> org.apache.camel.spring.SpringCamelContext.maybeDoStart(SpringCamelContext.java:166)
>at
> org.apache.camel.spring.SpringCamelContext.doStart(SpringCamelContext.java:161)
>at org.apache.camel.impl.ServiceSupport.start(ServiceSupport.java:49)
>at
> org.apache.camel.spring.SpringCamelContext.maybeStart(SpringCamelContext.java:96)
>at
> org.apache.camel.spring.SpringCamelContext.onApplicationEvent(SpringCamelContext.java:115)
>... 12 more
> Caused by: org.apache.uima.aae.error.UimaASMetaRequestTimeout
>... 46 more
> And on the worker node it results in this exception:
> rg.apache.uima.aae.error.AsynchAEException: javax.jms.JMSException: Could not
> create Transport. Reason: java.lang.IllegalArgumentException: Invalid connect
> parameters: {randomize=false}
>at
> org.apache.uima.adapter.jms.activemq.JmsEndpointConnection_impl.openChannel(JmsEndpointConnection_impl.java:284)
>at
> org.apache.uima.adapter.jms.activemq.JmsEndpointConnection_impl.openChannel(JmsEndpointConnection_impl.java:147)
>at
> org.apache.uima.adapter.jms.activemq.JmsEndpointConnection_impl.open(JmsEndpointConnection_impl.java:301)
>at
> org.apache.uima.adapter.jms.activemq.JmsEndpointConnection_impl.open(JmsEndpointConnection_impl.java:290)
>at
> org.apache.uima.adapter.jms.activemq.JmsOutputChannel.getEndpointConnection(JmsOutputChannel.java:493)
>at
> org.apache.uima.adapter.jms.activemq.JmsOutputChannel.sendReply(JmsOutputChannel.java:1153)
>at
> org.apache.uima.aae.controller.BaseAnalysisE
Re: doing 2.3.0 RC1 - tomorrow morning?
Jörn Kottmann wrote: > Marshall Schor wrote: >> Unless I hear some objections, I would like to tag the first (and >> hopefully only :-) ) release candidate for 2.3.0 tomorrow morning. >> Please get any fixes finished up that you would like to get into the >> release before then, or ask for a short delay if needed... >> >> -Marshall >> > Would be nice to get some feedback on UIMA-1658 > and UIMA-1659 before we do the rc. These > are the last to issue I have with our UIMA AS installation > which are important to get fixed. 1657 is fixed, 1658 will need perhaps extensive investigation involving uima-as, spring, and activemq - any of which could be the underlying cause. Because this may be quite open-ended, I will go ahead with doing release candidates now, and we can work for as long as we need to on hardening the fail-over. Let's tentatively plan on a uima-as point release in a couple of months to pick up these changes (assuming the fail-over capability has stablized by then). -Marshall > > Jörn > >
Re: doing 2.3.0 RC1 - tomorrow morning?
Marshall Schor wrote: Jörn Kottmann wrote: Marshall Schor wrote: Unless I hear some objections, I would like to tag the first (and hopefully only :-) ) release candidate for 2.3.0 tomorrow morning. Please get any fixes finished up that you would like to get into the release before then, or ask for a short delay if needed... -Marshall Would be nice to get some feedback on UIMA-1658 and UIMA-1659 before we do the rc. These are the last to issue I have with our UIMA AS installation which are important to get fixed. 1657 is fixed, 1658 will need perhaps extensive investigation involving uima-as, spring, and activemq - any of which could be the underlying cause. Because this may be quite open-ended, I will go ahead with doing release candidates now, and we can work for as long as we need to on hardening the fail-over. Let's tentatively plan on a uima-as point release in a couple of months to pick up these changes (assuming the fail-over capability has stablized by then). +1, through UIMA-1657 the failover now works after the worker nodes and the clients are restarted. I agree with Marshall that it could take some time to find out whats wrong. Jörn
[jira] Commented: (UIMA-1658) UIMA AS worker does not respond to client initialize after failover
[ https://issues.apache.org/jira/browse/UIMA-1658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12777038#action_12777038 ] Jerry Cwiklik commented on UIMA-1658: - Jorn, I was able to recreate the scenario. When the primary broker is killed, I can see in jConsole that the UIMA AS service silently reconnects to the remaining broker. The number of consumers is incremented as expected. I can also see that the client sends GetMeta request to the queue in the remaining broker. The enqueue count is incremented, however the dispatch count and dequeue count are not incremented. This suggests that there is some problem either in AMQ or Spring. There is no indication in the UIMA AS service that there is a problem. No exception is thrown from Spring when the primary broker is killed (as expected). The getMeta request is not being delivered to the UIMA AS listener object. Can you confirm that this is happening with you as well? > UIMA AS worker does not respond to client initialize after failover > --- > > Key: UIMA-1658 > URL: https://issues.apache.org/jira/browse/UIMA-1658 > Project: UIMA > Issue Type: Bug > Components: Async Scaleout >Reporter: Jörn Kottmann > Fix For: 2.3AS > > > A Pure Master Slave Broker is used to increase availability of the broker. > More information about it can be found in the activemq documentation: > http://activemq.apache.org/pure-master-slave.html > In a test we simulated Master failure through killing the process with kill > -9. > Here is the log output from the worker node: > INFO FailoverTransport - Successfully connected to > tcp://XXX1:61616 > Here I stopped the master broker process with kill -9 > WARN FailoverTransport - Transport failed to tcp://XXX1:61616 , > attempting to automatically reconnect due to: java.io.EOFException > WARN FailoverTransport - Transport failed to tcp://XXX1:61616 , > attempting to automatically reconnect due to: java.io.EOFException > WARN FailoverTransport - Transport failed to tcp://XXX1:61616 , > attempting to automatically reconnect due to: java.io.EOFException > INFO FailoverTransport - Successfully reconnected to > tcp://XXX2:61616 > INFO FailoverTransport - Successfully reconnected to > tcp://XXX2:61616 > INFO FailoverTransport - Successfully reconnected to > tcp://XXX2:61616 > Afterwards the client was restarted but got a time out error during > initialize. > During initializes it sends a message to the worker nodes input queue, but > this > message if never retrieved. > I used the activemq web interface to get some information about the message: > Command 2001 > MessageFrom ID:-51032-1257865414664-0:1:1 > ServerURI failover:(tcp://XXX1:61616,tcp://XXX2:61616)?randomize=false > MessageType 3000 -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (UIMA-1658) UIMA AS worker does not respond to client initialize after failover
[ https://issues.apache.org/jira/browse/UIMA-1658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12777041#action_12777041 ] Jörn Kottmann commented on UIMA-1658: - Yes, I think the same thing is happening on our side too. I used a remote debugger to have a look at the stack of the threads on the worker node and it looked like that two threads are waiting to receive a message inside activemq. I tried to add the timeout=3000 parameter to the broker uri, but it seems not to help. > UIMA AS worker does not respond to client initialize after failover > --- > > Key: UIMA-1658 > URL: https://issues.apache.org/jira/browse/UIMA-1658 > Project: UIMA > Issue Type: Bug > Components: Async Scaleout >Reporter: Jörn Kottmann > Fix For: 2.3AS > > > A Pure Master Slave Broker is used to increase availability of the broker. > More information about it can be found in the activemq documentation: > http://activemq.apache.org/pure-master-slave.html > In a test we simulated Master failure through killing the process with kill > -9. > Here is the log output from the worker node: > INFO FailoverTransport - Successfully connected to > tcp://XXX1:61616 > Here I stopped the master broker process with kill -9 > WARN FailoverTransport - Transport failed to tcp://XXX1:61616 , > attempting to automatically reconnect due to: java.io.EOFException > WARN FailoverTransport - Transport failed to tcp://XXX1:61616 , > attempting to automatically reconnect due to: java.io.EOFException > WARN FailoverTransport - Transport failed to tcp://XXX1:61616 , > attempting to automatically reconnect due to: java.io.EOFException > INFO FailoverTransport - Successfully reconnected to > tcp://XXX2:61616 > INFO FailoverTransport - Successfully reconnected to > tcp://XXX2:61616 > INFO FailoverTransport - Successfully reconnected to > tcp://XXX2:61616 > Afterwards the client was restarted but got a time out error during > initialize. > During initializes it sends a message to the worker nodes input queue, but > this > message if never retrieved. > I used the activemq web interface to get some information about the message: > Command 2001 > MessageFrom ID:-51032-1257865414664-0:1:1 > ServerURI failover:(tcp://XXX1:61616,tcp://XXX2:61616)?randomize=false > MessageType 3000 -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (UIMA-1658) UIMA AS worker does not respond to client initialize after failover
[ https://issues.apache.org/jira/browse/UIMA-1658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jörn Kottmann updated UIMA-1658: Environment: Ubuntu 8.10 Server, Java 1.6 and ActiveMQ 5.3.0 > UIMA AS worker does not respond to client initialize after failover > --- > > Key: UIMA-1658 > URL: https://issues.apache.org/jira/browse/UIMA-1658 > Project: UIMA > Issue Type: Bug > Components: Async Scaleout > Environment: Ubuntu 8.10 Server, Java 1.6 and ActiveMQ 5.3.0 >Reporter: Jörn Kottmann > Fix For: 2.3AS > > > A Pure Master Slave Broker is used to increase availability of the broker. > More information about it can be found in the activemq documentation: > http://activemq.apache.org/pure-master-slave.html > In a test we simulated Master failure through killing the process with kill > -9. > Here is the log output from the worker node: > INFO FailoverTransport - Successfully connected to > tcp://XXX1:61616 > Here I stopped the master broker process with kill -9 > WARN FailoverTransport - Transport failed to tcp://XXX1:61616 , > attempting to automatically reconnect due to: java.io.EOFException > WARN FailoverTransport - Transport failed to tcp://XXX1:61616 , > attempting to automatically reconnect due to: java.io.EOFException > WARN FailoverTransport - Transport failed to tcp://XXX1:61616 , > attempting to automatically reconnect due to: java.io.EOFException > INFO FailoverTransport - Successfully reconnected to > tcp://XXX2:61616 > INFO FailoverTransport - Successfully reconnected to > tcp://XXX2:61616 > INFO FailoverTransport - Successfully reconnected to > tcp://XXX2:61616 > Afterwards the client was restarted but got a time out error during > initialize. > During initializes it sends a message to the worker nodes input queue, but > this > message if never retrieved. > I used the activemq web interface to get some information about the message: > Command 2001 > MessageFrom ID:-51032-1257865414664-0:1:1 > ServerURI failover:(tcp://XXX1:61616,tcp://XXX2:61616)?randomize=false > MessageType 3000 -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (UIMA-1661) tag 2.3.0 rc1
tag 2.3.0 rc1 - Key: UIMA-1661 URL: https://issues.apache.org/jira/browse/UIMA-1661 Project: UIMA Issue Type: Task Reporter: Marshall Schor Assignee: Marshall Schor Priority: Minor -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (UIMA-1662) uima-as extract and build script has copy/paste error - is building base uima instead of uima-as
uima-as extract and build script has copy/paste error - is building base uima instead of uima-as Key: UIMA-1662 URL: https://issues.apache.org/jira/browse/UIMA-1662 Project: UIMA Issue Type: Bug Components: Async Scaleout, Build, Packaging and Test Reporter: Marshall Schor Assignee: Marshall Schor Fix For: 2.3AS fixing in the trunk and also in the 01 tag -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Closed: (UIMA-1662) uima-as extract and build script has copy/paste error - is building base uima instead of uima-as
[ https://issues.apache.org/jira/browse/UIMA-1662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marshall Schor closed UIMA-1662. Resolution: Fixed fixed in trunk and merged into tag for uima-as - 01 > uima-as extract and build script has copy/paste error - is building base uima > instead of uima-as > > > Key: UIMA-1662 > URL: https://issues.apache.org/jira/browse/UIMA-1662 > Project: UIMA > Issue Type: Bug > Components: Async Scaleout, Build, Packaging and Test >Reporter: Marshall Schor >Assignee: Marshall Schor > Fix For: 2.3AS > > > fixing in the trunk and also in the 01 tag -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Closed: (UIMA-1661) tag 2.3.0 rc1
[ https://issues.apache.org/jira/browse/UIMA-1661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marshall Schor closed UIMA-1661. Resolution: Fixed > tag 2.3.0 rc1 > - > > Key: UIMA-1661 > URL: https://issues.apache.org/jira/browse/UIMA-1661 > Project: UIMA > Issue Type: Task >Reporter: Marshall Schor >Assignee: Marshall Schor >Priority: Minor > -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
Apache Uima 2.3.0-incubating release candidate 01
The release candidate 01 for 2.3.0-incubating is on http://people.apache.org/~schor/uima-release-candidates/2.3.0-RC1/ It has 4 parts: uimaj - the main uima release uima-as - the uima-as add on uima-sandbox - the sandbox components being released uima-cpp - the C++ release, including several binary builds. The Release Audit Reports (RAT) are included. I'll work on posting a test plan to our wiki, probably tomorrow; meanwhile, please try it out. -Marshall
