It looks like there is something wrong with Epoll in your system. Did you try useEpoll=false on the acceptors of the broker? it will force the netty to use java NIO instead of the native linux epoll.
Howard On Thu, Sep 5, 2019 at 9:15 PM helenoster <no-re...@klimbingspider.com> wrote: > We have a spring-boot application that has to connect to a deployed Apache > Apache ActiveMQ Artemis 2.9.0 broker. The broker is running on a Red Hat > Enterprise Linux 7.6 and the client should run in a docker, but doing > running it without docker we have got the same issue. > > The broker has the default configuration listening to the 61616 port. > > We're not able to communicate to the broker from the spring-boot > applcation. > It looks like it is even to starting the call. > > > Here some more details about the code: > We've got a library where we've encapsulated the calls to the broker. The > pom.xml is as follows: > > <project xmlns="http://maven.apache.org/POM/4.0.0"; > xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"; > xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 > http://maven.apache.org/maven-v4_0_0.xsd";> > <modelVersion>4.0.0</modelVersion> > <groupId>com.xxx.ems.microservice.core</groupId> > <artifactId>ems-ms-core-library</artifactId> > <version>1.0-SNAPSHOT</version> > <packaging>jar</packaging> > <name>EMSMSCoreLibrary</name> > <url>http://maven.apache.org</url> > <parent> > <groupId>org.springframework.boot</groupId> > <artifactId>spring-boot-starter-parent</artifactId> > <version>2.1.6.RELEASE</version> > <relativePath/> > </parent> > <properties> > <maven.compiler.target>1.8</maven.compiler.target> > <maven.compiler.source>1.8</maven.compiler.source> > </properties> > <dependencies> > <dependency> > <groupId>org.springframework.boot</groupId> > <artifactId>spring-boot-starter</artifactId> > <exclusions> > <exclusion> > <groupId>org.springframework.boot</groupId> > <artifactId>spring-boot-starter-logging</artifactId> > </exclusion> > </exclusions> > </dependency> > > <dependency> > <groupId>org.springframework.boot</groupId> > <artifactId>spring-boot-starter-test</artifactId> > <scope>test</scope> > <exclusions> > <exclusion> > <groupId>org.springframework.boot</groupId> > <artifactId>spring-boot-starter-logging</artifactId> > </exclusion> > </exclusions> > </dependency> > > <dependency> > <groupId>org.springframework.boot</groupId> > <artifactId>spring-boot-starter-artemis</artifactId> > </dependency> > <dependency> > <groupId>org.apache.activemq</groupId> > <artifactId>artemis-jms-server</artifactId> > </dependency> > <dependency> > <groupId>com.fasterxml.jackson.core</groupId> > <artifactId>jackson-databind</artifactId> > </dependency> > <dependency> > <groupId>org.springframework.boot</groupId> > <artifactId>spring-boot-starter-log4j2</artifactId> > </dependency> > <dependency> > <groupId>com.fasterxml.jackson.dataformat</groupId> > <artifactId>jackson-dataformat-yaml</artifactId> > </dependency> > <dependency> > <groupId>org.projectlombok</groupId> > <artifactId>lombok</artifactId> > </dependency> > <dependency> > <groupId>junit</groupId> > <artifactId>junit</artifactId> > <scope>test</scope> > </dependency> > > <dependency> > <groupId>co.elastic.apm</groupId> > <artifactId>apm-agent-api</artifactId> > <version>1.7.0</version> > </dependency> > > <dependency> > <groupId>org.springframework.boot</groupId> > <artifactId>spring-boot-starter-aop</artifactId> > </dependency> > <dependency> > <groupId>com.jcabi</groupId> > <artifactId>jcabi-manifests</artifactId> > <version>1.1</version> > </dependency> > <dependency> > <groupId>commons-io</groupId> > <artifactId>commons-io</artifactId> > <version>2.6</version> > </dependency> > </dependencies> > </project> > > Our configuration file includes the following lines to connect to the > artemis broker: > > spring: > artemis: > mode: native > host: myhost > port: 61616 > # Spring JMS Settings > jms: > template: > qos-enabled: true > > Here we include the method we're using to send the messages > > package com.xxx.ems.microservice.core.queue; > > import javax.jms.JMSException; > import javax.jms.Message; > import javax.jms.Session; > > import org.springframework.beans.factory.annotation.Autowired; > import org.springframework.jms.UncategorizedJmsException; > import org.springframework.jms.annotation.EnableJms; > import org.springframework.jms.core.JmsTemplate; > import org.springframework.jms.core.MessageCreator; > import org.springframework.jms.support.converter.MessageConverter; > import org.springframework.stereotype.Component; > > // Apache Artemis Message Sender > @Component > @EnableJms > public class MessageSender { > @Autowired > JmsTemplate jmsTemplate; > > @Autowired > MessageConverter defaultMessageConverter; > > protected void send(String destinationQueue, EMSFileMessage > emsFileMessage, int ... priority) throws EMSQueueException{ > try{ > if (priority.length == 0){ > jmsTemplate.convertAndSend(destinationQueue, > emsFileMessage); > }else{ > jmsTemplate.setPriority(priority[0]); > jmsTemplate.send(destinationQueue, new MessageCreator() { > @Override > public Message createMessage(Session session) throws > JMSException { > return > defaultMessageConverter.toMessage(emsFileMessage, session); > } > }); > } > }catch(UncategorizedJmsException ex){ > throw new EMSQueueException("Fatall error, probably due to > connection with message broker, address that is trying to connect with > address "+destinationQueue, ex); > } > } > } > > Before calling the send message we're writing a log to check the > ArtemisProperties (host and port) and both have the proper values. The > destinationQueue has also the correct value. > > > > For some reason we've got the following exception: > > com.xxx.ems.microservice.core.queue.EMSQueueException: Fatall error, > probably due to connection with message broker, address that is trying to > connect with address ems.gnfr > at > > com.xxx.ems.microservice.core.queue.MessageSender.send(MessageSender.java:56) > ~[classes/:?] > at > > com.xxx.ems.microservice.core.queue.MessageDispatcher.sendFile(MessageDispatcher.java:41) > ~[classes/:?] > at > > com.xxx.ems.microservice.core.queue.MessageDispatcher.send(MessageDispatcher.java:33) > ~[classes/:?] > at > > com.xxx.ems.microservice.msdiskloader.controller.DiskLoaderController.processSingleFile(DiskLoaderController.java:64) > ~[classes/:?] > at > > com.xxx.ems.microservice.msdiskloader.controller.DiskLoaderController$$FastClassBySpringCGLIB$$7f37b3f5.invoke(<generated>) > ~[classes/:?] > at > org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) > ~[spring-core-5.1.8.RELEASE.jar:5.1.8.RELEASE] > at > > org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:749) > ~[spring-aop-5.1.8.RELEASE.jar:5.1.8.RELEASE] > at > > org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) > ~[spring-aop-5.1.8.RELEASE.jar:5.1.8.RELEASE] > at > > org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:88) > ~[spring-aop-5.1.8.RELEASE.jar:5.1.8.RELEASE] > at > > com.xxx.ems.microservice.core.apm.aspects.AspectTransactionStarter.doMessageReception(AspectTransactionStarter.java:24) > ~[classes/:?] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > ~[?:1.8.0_221] > at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) > ~[?:1.8.0_221] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) > ~[?:1.8.0_221] > at java.lang.reflect.Method.invoke(Unknown Source) ~[?:1.8.0_221] > at > > org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:644) > ~[spring-aop-5.1.8.RELEASE.jar:5.1.8.RELEASE] > at > > org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:633) > ~[spring-aop-5.1.8.RELEASE.jar:5.1.8.RELEASE] > at > > org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:70) > ~[spring-aop-5.1.8.RELEASE.jar:5.1.8.RELEASE] > at > > org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) > ~[spring-aop-5.1.8.RELEASE.jar:5.1.8.RELEASE] > at > > org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:93) > ~[spring-aop-5.1.8.RELEASE.jar:5.1.8.RELEASE] > at > > org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) > ~[spring-aop-5.1.8.RELEASE.jar:5.1.8.RELEASE] > at > > org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:688) > ~[spring-aop-5.1.8.RELEASE.jar:5.1.8.RELEASE] > at > > com.xxx.ems.microservice.msdiskloader.controller.DiskLoaderController$$EnhancerBySpringCGLIB$$eca14b7b.processSingleFile(<generated>) > ~[classes/:?] > at > > com.xxx.ems.microservice.msdiskloader.utils.DirectoryChangesWatcher.lambda$proccesExistingFilesInPath$2(DirectoryChangesWatcher.java:249) > ~[classes/:?] > at java.lang.Iterable.forEach(Unknown Source) ~[?:1.8.0_221] > at > > com.xxx.ems.microservice.msdiskloader.utils.DirectoryChangesWatcher.proccesExistingFilesInPath(DirectoryChangesWatcher.java:249) > ~[classes/:?] > at > > com.xxx.ems.microservice.msdiskloader.utils.DirectoryChangesWatcher.startWatchingFolder(DirectoryChangesWatcher.java:94) > ~[classes/:?] > at > > com.xxx.ems.microservice.msdiskloader.controller.DiskLoaderController.onApplicationStart(DiskLoaderController.java:46) > ~[classes/:?] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > ~[?:1.8.0_221] > at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) > ~[?:1.8.0_221] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) > ~[?:1.8.0_221] > at java.lang.reflect.Method.invoke(Unknown Source) ~[?:1.8.0_221] > at > > org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor$LifecycleElement.invoke(InitDestroyAnnotationBeanPostProcessor.java:363) > ~[spring-beans-5.1.8.RELEASE.jar:5.1.8.RELEASE] > at > > org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor$LifecycleMetadata.invokeInitMethods(InitDestroyAnnotationBeanPostProcessor.java:307) > ~[spring-beans-5.1.8.RELEASE.jar:5.1.8.RELEASE] > at > > org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor.postProcessBeforeInitialization(InitDestroyAnnotationBeanPostProcessor.java:136) > ~[spring-beans-5.1.8.RELEASE.jar:5.1.8.RELEASE] > at > > org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.applyBeanPostProcessorsBeforeInitialization(AbstractAutowireCapableBeanFactory.java:414) > ~[spring-beans-5.1.8.RELEASE.jar:5.1.8.RELEASE] > at > > org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1770) > ~[spring-beans-5.1.8.RELEASE.jar:5.1.8.RELEASE] > at > > org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:593) > ~[spring-beans-5.1.8.RELEASE.jar:5.1.8.RELEASE] > at > > org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:515) > ~[spring-beans-5.1.8.RELEASE.jar:5.1.8.RELEASE] > at > > org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:320) > ~[spring-beans-5.1.8.RELEASE.jar:5.1.8.RELEASE] > at > > org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:222) > [spring-beans-5.1.8.RELEASE.jar:5.1.8.RELEASE] > at > > org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:318) > [spring-beans-5.1.8.RELEASE.jar:5.1.8.RELEASE] > at > > org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:199) > [spring-beans-5.1.8.RELEASE.jar:5.1.8.RELEASE] > at > > org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:845) > [spring-beans-5.1.8.RELEASE.jar:5.1.8.RELEASE] > at > > org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:877) > [spring-context-5.1.8.RELEASE.jar:5.1.8.RELEASE] > at > > org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:549) > [spring-context-5.1.8.RELEASE.jar:5.1.8.RELEASE] > at > > org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.refresh(ServletWebServerApplicationContext.java:140) > [spring-boot-2.1.6.RELEASE.jar:2.1.6.RELEASE] > at > > org.springframework.boot.SpringApplication.refresh(SpringApplication.java:742) > [spring-boot-2.1.6.RELEASE.jar:2.1.6.RELEASE] > at > > org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:389) > [spring-boot-2.1.6.RELEASE.jar:2.1.6.RELEASE] > at > org.springframework.boot.SpringApplication.run(SpringApplication.java:311) > [spring-boot-2.1.6.RELEASE.jar:2.1.6.RELEASE] > at > org.springframework.boot.SpringApplication.run(SpringApplication.java:1213) > [spring-boot-2.1.6.RELEASE.jar:2.1.6.RELEASE] > at > org.springframework.boot.SpringApplication.run(SpringApplication.java:1202) > [spring-boot-2.1.6.RELEASE.jar:2.1.6.RELEASE] > at > > com.xxx.ems.microservice.core.app.EMSMicroserviceApplication.run(EMSMicroserviceApplication.java:10) > [classes/:?] > at > > com.xxx.ems.microservice.msdiskloader.MicroserviceApplication.main(MicroserviceApplication.java:18) > [classes/:?] > Caused by: org.springframework.jms.UncategorizedJmsException: Uncategorized > exception occurred during JMS processing; nested exception is > javax.jms.JMSException: Failed to create session factory; nested exception > is ActiveMQNotConnectedException[errorType=NOT_CONNECTED message=AMQ219007: > Cannot connect to server(s). Tried with all available servers.] > at > > org.springframework.jms.support.JmsUtils.convertJmsAccessException(JmsUtils.java:311) > ~[spring-jms-5.1.8.RELEASE.jar:5.1.8.RELEASE] > at > > org.springframework.jms.support.JmsAccessor.convertJmsAccessException(JmsAccessor.java:185) > ~[spring-jms-5.1.8.RELEASE.jar:5.1.8.RELEASE] > at > org.springframework.jms.core.JmsTemplate.execute(JmsTemplate.java:507) > ~[spring-jms-5.1.8.RELEASE.jar:5.1.8.RELEASE] > at org.springframework.jms.core.JmsTemplate.send(JmsTemplate.java:584) > ~[spring-jms-5.1.8.RELEASE.jar:5.1.8.RELEASE] > at > > org.springframework.jms.core.JmsTemplate.convertAndSend(JmsTemplate.java:661) > ~[spring-jms-5.1.8.RELEASE.jar:5.1.8.RELEASE] > at > > com.xxx.ems.microservice.core.queue.MessageSender.send(MessageSender.java:45) > ~[classes/:?] > ... 52 more > Caused by: javax.jms.JMSException: Failed to create session factory > at > > org.apache.activemq.artemis.jms.client.ActiveMQConnectionFactory.createConnectionInternal(ActiveMQConnectionFactory.java:846) > ~[artemis-jms-client-2.6.4.jar:2.6.4] > at > > org.apache.activemq.artemis.jms.client.ActiveMQConnectionFactory.createConnection(ActiveMQConnectionFactory.java:282) > ~[artemis-jms-client-2.6.4.jar:2.6.4] > at > > org.apache.activemq.artemis.jms.client.ActiveMQConnectionFactory.createConnection(ActiveMQConnectionFactory.java:277) > ~[artemis-jms-client-2.6.4.jar:2.6.4] > at > > org.springframework.jms.support.JmsAccessor.createConnection(JmsAccessor.java:196) > ~[spring-jms-5.1.8.RELEASE.jar:5.1.8.RELEASE] > at > org.springframework.jms.core.JmsTemplate.execute(JmsTemplate.java:494) > ~[spring-jms-5.1.8.RELEASE.jar:5.1.8.RELEASE] > at org.springframework.jms.core.JmsTemplate.send(JmsTemplate.java:584) > ~[spring-jms-5.1.8.RELEASE.jar:5.1.8.RELEASE] > at > > org.springframework.jms.core.JmsTemplate.convertAndSend(JmsTemplate.java:661) > ~[spring-jms-5.1.8.RELEASE.jar:5.1.8.RELEASE] > at > > com.xxx.ems.microservice.core.queue.MessageSender.send(MessageSender.java:45) > ~[classes/:?] > ... 52 more > Caused by: > org.apache.activemq.artemis.api.core.ActiveMQNotConnectedException: > AMQ219007: Cannot connect to server(s). Tried with all available servers. > at > > org.apache.activemq.artemis.core.client.impl.ServerLocatorImpl.createSessionFactory(ServerLocatorImpl.java:797) > ~[artemis-core-client-2.6.4.jar:2.6.4] > at > > org.apache.activemq.artemis.jms.client.ActiveMQConnectionFactory.createConnectionInternal(ActiveMQConnectionFactory.java:844) > ~[artemis-jms-client-2.6.4.jar:2.6.4] > at > > org.apache.activemq.artemis.jms.client.ActiveMQConnectionFactory.createConnection(ActiveMQConnectionFactory.java:282) > ~[artemis-jms-client-2.6.4.jar:2.6.4] > at > > org.apache.activemq.artemis.jms.client.ActiveMQConnectionFactory.createConnection(ActiveMQConnectionFactory.java:277) > ~[artemis-jms-client-2.6.4.jar:2.6.4] > at > > org.springframework.jms.support.JmsAccessor.createConnection(JmsAccessor.java:196) > ~[spring-jms-5.1.8.RELEASE.jar:5.1.8.RELEASE] > at > org.springframework.jms.core.JmsTemplate.execute(JmsTemplate.java:494) > ~[spring-jms-5.1.8.RELEASE.jar:5.1.8.RELEASE] > at org.springframework.jms.core.JmsTemplate.send(JmsTemplate.java:584) > ~[spring-jms-5.1.8.RELEASE.jar:5.1.8.RELEASE] > at > > org.springframework.jms.core.JmsTemplate.convertAndSend(JmsTemplate.java:661) > ~[spring-jms-5.1.8.RELEASE.jar:5.1.8.RELEASE] > at > > com.xxx.ems.microservice.core.queue.MessageSender.send(MessageSender.java:45) > ~[classes/:?] > ... 52 more > > We have seen another exception with debug level about > netty_transport_native_epoll_x86_64 > > Unable to load the library 'netty_transport_native_epoll_x86_64', > trying > other loading mechanism. > java.lang.UnsatisfiedLinkError: no > netty_transport_native_epoll_x86_64 in > java.library.path > > We've asked the netty team (https://github.com/netty/netty/issues/9537) > and > they commented that there is no relation between the > netty_transport_native_epoll_x86_64 and the communication from Artemis. The > application should be working. > We have captured the packets received in the network in the machine where > the Apache ActiveMQ Artemis is installed and we don't receive any packet. > However, a telnet to the port works and a couple of packets are captured. > So > the shouldn't be any firewall issues. > We have done a test creating a tunnel to the machine where the spring-boot > aplication should be running and forwarding the call to the broker machine. > Running the spring-boot application in windows works well and we see > messages in the broker dashboard. > > As commented above, the application should be running in a docker, we have > tested using koosiedemoer/netty-tcnative-alpine and openjdk:8-jdk-alpine > as base images. The docker is able to communicate with an elastic-apm > service installed in the same machine as the Apache ActiveMQ Artemis. > For some reason it is not able to establish communication when the software > runs Red Hat Enterprise Linux 7.6 and java 1.8.0_221, neither when we run > it > from a docker image. > > Is there anything that should be installed, another base-image that should > be used? > We're new with artemis and such kind of brokers and the logs doesn't give > us > any clue to solve the issue. Any help would be appreciated > > > > > -- > Sent from: > http://activemq.2283324.n4.nabble.com/ActiveMQ-User-f2341805.html >
