Jerry,
Testing a real analytic running 28 sync pipelines as a DUCC managed UIMA-AS
service, the DUCC pinger is reporting failures for an idle service 2 or 3
times a day.
sm.log, an error surrounded by good pings, the ping normally done every 2
minutes:
Dec 2017 07:49:42,740 INFO SM.PingDriver - 10668 T[3588945]
handleStatistics Ping ok: UIMA-AS:eae-r1.16:tcp://broker1:61616
Alive[true] Healthy[true] + Info: QDEPTH[0] AveNQ[0] Consum[3] Prod[1]
minNQ[0] maxNQ[0] expCnt[0] inFlt[0] DQ[1] NQ[1] NDisp[1]
MetaNode[192.168.7.101] MetaPid[71061:91]
Dec 2017 07:51:42,844 INFO SM.PingDriver - 10668 T[3588945]
handleStatistics Ping ok: UIMA-AS:eae-r1.16:tcp://broker1:61616
Alive[true] Healthy[true] + Info: QDEPTH[0] AveNQ[0] Consum[3] Prod[1]
minNQ[0] maxNQ[0] expCnt[0] inFlt[0] DQ[1] NQ[1] NDisp[1]
MetaNode[192.168.7.101] MetaPid[71061:91]
Dec 2017 07:54:02,912 ERROR SM.PingDriver - 10668 T[3588945]
handleStatistics Missed_pings 1 endpoint
UIMA-AS:eae-r1.16:tcp://broker1:61616 Alive[false] Healthy[false] + Info:
QDEPTH[0] AveNQ[0] Consum[3] Prod[1] minNQ[0] maxNQ[0] expCnt[0] inFlt[0]
DQ[1] NQ[1] NDisp[1] MetaNode[N/A] MetaPid[N/A]: Cannot issue getMeta to:
eae-r1.16:tcp://broker1:61616: java.util.concurrent.TimeoutException
Dec 2017 07:56:03,016 INFO SM.PingDriver - 10668 T[3588945]
handleStatistics Ping ok: UIMA-AS:eae-r1.16:tcp://broker1:61616
Alive[true] Healthy[true] + Info: QDEPTH[0] AveNQ[89570] Consum[3] Prod[1]
minNQ[89570] maxNQ[89570] expCnt[0] inFlt[0] DQ[1] NQ[1] NDisp[1]
MetaNode[192.168.7.101] MetaPid[71061:91]
The broker logfile, complaining about all three service listeners around
the same time (the service runs on 192.168.7.101):
2017-12-24 07:54:48,257 [yMonitor Worker] WARN Transport
- Transport Connection to: tcp://192.168.7.101:43895 failed:
org.apache.activemq.transport.InactivityIOException: Channel was inactive
for too (>30000) long: tcp://192.168.7.101:43895
2017-12-24 07:54:48,287 [yMonitor Worker] WARN Transport
- Transport Connection to: tcp://192.168.7.101:43898 failed:
org.apache.activemq.transport.InactivityIOException: Channel was inactive
for too (>30000) long: tcp://192.168.7.101:43898
2017-12-24 07:54:48,288 [yMonitor Worker] WARN Transport
- Transport Connection to: tcp://192.168.7.101:43899 failed:
org.apache.activemq.transport.InactivityIOException: Channel was inactive
for too (>30000) long: tcp://192.168.7.101:43899
The service logfile also reporting problems for all three listeners:
07:55:12.457 - 91:
org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerContainer.onException:
WARNING: Service: Processing2 Runtime Exception
07:55:12.457 - 91:
org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerContainer.onException:
WARNING: Jms Listener Failed. Endpoint: eae-r1.16 Managed By:
tcp://broker1:61616 Reason: javax.jms.JMSException: java.io.EOFException
07:55:12.457 - 91:
org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerContainer.handleListenerSetupFailure:
WARNING: Uima AS Service:Processing2 Listener Unable To Connect To Broker:
tcp://broker1:61616 Retrying Until Successful ...
07:55:12.457 - 2336:
org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerContainer.onException:
WARNING: Service: Processing2 Runtime Exception
07:55:12.457 - 2336:
org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerContainer.onException:
WARNING: Jms Listener Failed. Endpoint: eae-r1.16 Managed By:
tcp://broker1:61616 Reason: javax.jms.JMSException: java.io.EOFException
07:55:12.457 - 2335:
org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerContainer.onException:
WARNING: Service: Processing2 Runtime Exception
07:55:12.457 - 2335:
org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerContainer.onException:
WARNING: Jms Listener Failed. Endpoint: eae-r1.16 Managed By:
tcp://broker1:61616 Reason: javax.jms.JMSException: java.io.EOFException
07:55:12.457 - 2336:
org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerContainer.handleListenerSetupFailure:
WARNING: Uima AS Service:Processing2 Listener Unable To Connect To Broker:
tcp://broker1:61616 Retrying Until Successful ...
07:55:12.457 - 2335:
org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerContainer.handleListenerSetupFailure:
WARNING: Uima AS Service:Processing2 Listener Unable To Connect To Broker:
tcp://broker1:61616 Retrying Until Successful ...
sm.log is showing the same kind of ping failures for previous versions of
UIMA-AS as well, not all at the same time as this event, but spread out
widely. The clocks on all three machines are synchronized so it could be
meaningful that the pinger client registered the error before the broker or
the service.
Don't think this is a blocker, just wanted to check if there is a problem
in UIMA-AS.
Eddie
On Thu, Dec 14, 2017 at 4:52 PM, Jaroslaw Cwiklik <cwik...@apache.org>
wrote:
> Hi,
>
> the UIMA-AS 2.10.2 release candidate 5 is ready for voting.
>
> This version contains the following fixes:
> - fixed documentation
>
> The list of changes in Jira:
> http://issues.apache.org/jira/issues/?jql=fixVersion%20%3D%
> 202.10.2AS%20AND%20project%20%3D%20UIMA
>
> The source and binary zip/tars are staged to
> http://dist.apache.org/repos/dist/dev/uima/uima-as/2.10.2/RC5/
>
> The eclipse update subsite is here:
> http://dist.apache.org/repos/dist/dev/uima/eclipse-update-site/uima-as/
>
> The Maven artifacts are here:
> http://repository.apache.org/content/repositories/orgapacheuima-1177
>
> The SVN tags are here:
> http://svn.apache.org/repos/asf/uima/uima-as/tags/uima-as-2.10.2/
>
> See http://uima.apache.org/testing-builds.html for suggestions on how to
> test release candidates.
>
> Please vote on release:
>
> [ ] +1 OK to release
> [ ] 0 Don't care
> [ ] -1 Not OK to release, because ...
>
