Thanks for following up!
Justin On Wed, Mar 15, 2023 at 12:56 PM prateekjai...@gmail.com < prateekjai...@gmail.com> wrote: > Hi All, > > I am able to fix this issue. Posting my solution here, just in case anyone > else faces a similar issue. I converted deployment to a statefulset (to get > predictable names). Then used headless services for each pod. This enabled > both master and slave pods to lookup each other just by pod name. In case > anyone is looking for more details, feel free to connect. > > Regards, > Prateek Jain > > -------------------------------------------------------------- > EXPECTATION : Causes all troubles...... > -------------------------------------------------------------- > > > On Wed, Mar 15, 2023 at 3:54 PM prateekjai...@gmail.com < > prateekjai...@gmail.com> wrote: > > > > > Hi Justin, > > > > Thanks for replying, I think artemis is failing to lookup POD by its > name > > because we cannot just get POD IP just by its name. Given below is the > > error which is getting repeated in logs: > > > > 2023-03-15 10:04:06,611 INFO [org.apache.activemq.artemis] AMQ241002: > > Artemis Jolokia REST API available at > > http://jms-01-deployment-6d7dcb8ccd-j7pxl:8161/console/jolokia > > 2023-03-15 10:04:06,611 INFO [org.apache.activemq.artemis] AMQ241004: > > Artemis Console available at > > http://jms-01-deployment-6d7dcb8ccd-j7pxl:8161/console > > 2023-03-15 10:04:10,868 ERROR [org.apache.activemq.artemis.core.client] > > AMQ214016: Failed to create netty connection > > java.net.UnknownHostException: jms-01-deployment-6d7dcb8ccd-2t9g2 > > at java.net.InetAddress$CachedAddresses.get(InetAddress.java:797) > > ~[?:?] > > at java.net.InetAddress.getAllByName0(InetAddress.java:1524) > ~[?:?] > > at java.net.InetAddress.getAllByName(InetAddress.java:1382) > ~[?:?] > > at java.net.InetAddress.getAllByName(InetAddress.java:1306) > ~[?:?] > > at java.net.InetAddress.getByName(InetAddress.java:1256) ~[?:?] > > at io.netty.util.internal.SocketUtils$8.run(SocketUtils.java:156) > > ~[netty-common-4.1.86.Final.jar:4.1.86.Final] > > at io.netty.util.internal.SocketUtils$8.run(SocketUtils.java:153) > > ~[netty-common-4.1.86.Final.jar:4.1.86.Final] > > at java.security.AccessController.doPrivileged(Native Method) > > ~[?:?] > > at > > io.netty.util.internal.SocketUtils.addressByName(SocketUtils.java:153) > > ~[netty-common-4.1.86.Final.jar:4.1.86.Final] > > at > > > io.netty.resolver.DefaultNameResolver.doResolve(DefaultNameResolver.java:41) > > ~[netty-resolver-4.1.86.Final.jar:4.1.86.Final] > > at > > io.netty.resolver.SimpleNameResolver.resolve(SimpleNameResolver.java:61) > > ~[netty-resolver-4.1.86.Final.jar:4.1.86.Final] > > at > > io.netty.resolver.SimpleNameResolver.resolve(SimpleNameResolver.java:53) > > ~[netty-resolver-4.1.86.Final.jar:4.1.86.Final] > > at > > > io.netty.resolver.InetSocketAddressResolver.doResolve(InetSocketAddressResolver.java:55) > > ~[netty-resolver-4.1.86.Final.jar:4.1.86.Final] > > at > > > io.netty.resolver.InetSocketAddressResolver.doResolve(InetSocketAddressResolver.java:31) > > ~[netty-resolver-4.1.86.Final.jar:4.1.86.Final] > > at > > > io.netty.resolver.AbstractAddressResolver.resolve(AbstractAddressResolver.java:106) > > ~[netty-resolver-4.1.86.Final.jar:4.1.86.Final] > > at > > io.netty.bootstrap.Bootstrap.doResolveAndConnect0(Bootstrap.java:206) > > ~[netty-transport-4.1.86.Final.jar:4.1.86.Final] > > at io.netty.bootstrap.Bootstrap.access$000(Bootstrap.java:46) > > ~[netty-transport-4.1.86.Final.jar:4.1.86.Final] > > at > > io.netty.bootstrap.Bootstrap$1.operationComplete(Bootstrap.java:180) > > ~[netty-transport-4.1.86.Final.jar:4.1.86.Final] > > at > > io.netty.bootstrap.Bootstrap$1.operationComplete(Bootstrap.java:166) > > ~[netty-transport-4.1.86.Final.jar:4.1.86.Final] > > at > > > io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) > > ~[netty-common-4.1.86.Final.jar:4.1.86.Final] > > at > > > io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:557) > > ~[netty-common-4.1.86.Final.jar:4.1.86.Final] > > at > > > io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) > > ~[netty-common-4.1.86.Final.jar:4.1.86.Final] > > at > > > io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) > > ~[netty-common-4.1.86.Final.jar:4.1.86.Final] > > at > > > io.netty.util.concurrent.DefaultPromise.setSuccess0(DefaultPromise.java:625) > > ~[netty-common-4.1.86.Final.jar:4.1.86.Final] > > at > > > io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:105) > > ~[netty-common-4.1.86.Final.jar:4.1.86.Final] > > at > > > io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:84) > > ~[netty-transport-4.1.86.Final.jar:4.1.86.Final] > > at > > > io.netty.channel.AbstractChannel$AbstractUnsafe.safeSetSuccess(AbstractChannel.java:990) > > ~[netty-transport-4.1.86.Final.jar:4.1.86.Final] > > at > > > io.netty.channel.AbstractChannel$AbstractUnsafe.register0(AbstractChannel.java:516) > > ~[netty-transport-4.1.86.Final.jar:4.1.86.Final] > > at > > > io.netty.channel.AbstractChannel$AbstractUnsafe.access$200(AbstractChannel.java:429) > > ~[netty-transport-4.1.86.Final.jar:4.1.86.Final] > > at > > > io.netty.channel.AbstractChannel$AbstractUnsafe$1.run(AbstractChannel.java:486) > > ~[netty-transport-4.1.86.Final.jar:4.1.86.Final] > > at > > > io.netty.util.concurrent.AbstractEventExecutor.runTask(AbstractEventExecutor.java:174) > > ~[netty-common-4.1.86.Final.jar:4.1.86.Final] > > at > > > io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:167) > > ~[netty-common-4.1.86.Final.jar:4.1.86.Final] > > at > > > io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:470) > > ~[netty-common-4.1.86.Final.jar:4.1.86.Final] > > at > > io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:406) > > ~[netty-transport-classes-epoll-4.1.86.Final.jar:4.1.86.Final] > > at > > > io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) > > ~[netty-common-4.1.86.Final.jar:4.1.86.Final] > > at > > io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) > > ~[netty-common-4.1.86.Final.jar:4.1.86.Final] > > at > > > org.apache.activemq.artemis.utils.ActiveMQThreadFactory$1.run(ActiveMQThreadFactory.java:118) > > ~[artemis-commons-2.28.0.jar:?] > > 2023-03-15 10:04:10,868 ERROR [org.apache.activemq.artemis.core.client] > > AMQ214016: Failed to create netty connection > > java.net.UnknownHostException: jms-01-deployment-6d7dcb8ccd-2t9g2 > > at java.net.InetAddress$CachedAddresses.get(InetAddress.java:797) > > ~[?:?] > > at java.net.InetAddress.getAllByName0(InetAddress.java:1524) > ~[?:?] > > at java.net.InetAddress.getAllByName(InetAddress.java:1382) > ~[?:?] > > at java.net.InetAddress.getAllByName(InetAddress.java:1306) > ~[?:?] > > at java.net.InetAddress.getByName(InetAddress.java:1256) ~[?:?] > > > > IMO, I need to find some way in which the pod name resolves to pod ip. If > > you have any recommendations, let me know. > > > > Regards, > > Prateek Jain > > > > > > -------------------------------------------------------------- > > EXPECTATION : Causes all troubles...... > > -------------------------------------------------------------- > > > > > > On Wed, Mar 15, 2023 at 2:23 PM Justin Bertram <jbert...@apache.org> > > wrote: > > > >> ActiveMQ Artemis offers JGroups integration for cloud use-cases such as > >> yours, but, generally speaking, we don't really provide support for > >> JGroups > >> in the ActiveMQ community since we don't maintain the JGroups code-base > >> and > >> don't know its inner workings. > >> > >> That said, I would suggest that the JGroups configuration using > FILE_PING > >> shipped in the examples may not be the best for your use-case. We chose > >> the > >> FILE_PING configuration because it was the simplest for the use-cases in > >> the examples (i.e. where all the nodes are running on the same machine > on > >> different ports with access to the same disk). However, something like > >> KUBE_PING [1] would probably be a better fit for your use-case since you > >> are, in fact, using Kubernetes. > >> > >> In any event, it's hard to comment on what may be happening without more > >> information. You've only provided a snippet of the stack-trace for the > >> UnknownHostException so it's not clear what class is actually attempting > >> that operation. > >> > >> > >> Justin > >> > >> [1] https://github.com/jgroups-extras/jgroups-kubernetes > >> > >> On Wed, Mar 15, 2023 at 7:27 AM prateekjai...@gmail.com < > >> prateekjai...@gmail.com> wrote: > >> > >> > Hi All, > >> > > >> > How are you? I am a bit confused about the errors reported by artemis > >> slave > >> > instance (running in clustered mode using jgroups). I might be missing > >> > something here so please bear with me. > >> > > >> > Deployment details: I am running 2 pods of artemis in master/slave > >> > configuration in k8s. It supports NFS v4. I am using jgroups > >> configuration > >> > (same file which is supplied in examples of artemis) and I can see > that > >> > correct pod names and IPs are updated in share directory > >> > > >> > -------- output from jgroups file ---------------- > >> > root@jms-01-deployment-6d7dcb8ccd-2t9g2:/opt# cat > >> > > >> > > >> > /usr/share/artemis-jgroups/active_broadcast_channel/ab62408d-6a7a-4894-9923-2649d8bf3b28.jms-01-deployment-6d7dcb8ccd-2t9g2-29095.list > >> > > >> > jms-01-deployment-6d7dcb8ccd-2t9g2-29095 > >> > ab62408d-6a7a-4894-9923-2649d8bf3b28 192.168.48.154:7800 T > >> > jms-01-deployment-6d7dcb8ccd-j7pxl-34353 > >> > 5b3d099b-8e5f-4dd0-ab43-604ad1ae87c3 192.168.199.2:7800 F > >> > -------------------------------------------------------- > >> > > >> > POD "jms-01-deployment-6d7dcb8ccd-2t9g2" is the master and another is > >> > slave. Correct POD ips are listed there. But in log/artemis.log file > of > >> > slave pod, I see these logs > >> > > >> > ------------------------------- > >> > 2023-03-15 11:29:51,519 ERROR > [org.apache.activemq.artemis.core.client] > >> > AMQ214016: Failed to create netty connection > >> > java.net.UnknownHostException: jms-01-deployment-6d7dcb8ccd-2t9g2 > >> > at java.net.InetAddress$CachedAddresses.get(InetAddress.java:797) > ~[?:?] > >> > at java.net.InetAddress.getAllByName0(InetAddress.java:1524) ~[?:?] > >> > at java.net.InetAddress.getAllByName(InetAddress.java:1382) ~[?:?] > >> > ----------------------------------- > >> > > >> > jgroups is correctly creating and updating files. IPs of master/slave > >> pods > >> > are visible from each other. I have tested them using command > >> > > >> > "./artemis check node". > >> > Right now I have created deployment objects. Is it necessary to create > >> > stateful sets? > >> > > >> > Am I missing something here? Let me know if you need more information > on > >> > this. > >> > > >> > > >> > Regards, > >> > Prateek Jain > >> > > >> > > >> > -------------------------------------------------------------- > >> > EXPECTATION : Causes all troubles...... > >> > -------------------------------------------------------------- > >> > > >> > > >
