[GitHub] [storm] bipinprasad commented on a diff in pull request #3536: [STORM-3911] fetch net.minidev:json-smart:2.3 artifact from maven repo before conjars.org repo is referenced
bipinprasad commented on code in PR #3536:
URL: https://github.com/apache/storm/pull/3536#discussion_r1213687480
##
storm-core/pom.xml:
##
@@ -137,6 +137,7 @@
org.mockito
mockito-core
+${mockito.version}
Review Comment:
removed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscr...@storm.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [storm] agresch commented on a diff in pull request #3536: [STORM-3911] fetch net.minidev:json-smart:2.3 artifact from maven repo before conjars.org repo is referenced
agresch commented on code in PR #3536:
URL: https://github.com/apache/storm/pull/3536#discussion_r1213655650
##
storm-core/pom.xml:
##
@@ -137,6 +137,7 @@
org.mockito
mockito-core
+${mockito.version}
Review Comment:
why was this line added?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscr...@storm.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: Does storm 2.4.0 supports IPv6 address
Hello Aishwarya, Are you using a helm chart for this deploy and/or k8s deployment/service files that you can share? I don't believe Storm has been extensively tested in IPv6. Thanks--Bipin On Thursday, June 1, 2023 at 12:20:45 PM PDT, Aishwarya Soni wrote: Hi, I have a storm 2.4.0 cluster with all the storm components such as Nimbus, UI, supervisors, and drpc. Recently we moved to a new GCP Kubernetes cluster where both traffic was coming from both IPv4 and IPv6 addresses. Previously, we were on a hashicorp nomad cluster where only IPv4 was supported. Since the move to GCP Kubernetes (v1.23.x), I started to see supervisor is unable to communicate with drpc on port 3773. This is the error snippet from one of the topology logs inside a supervisor. 2023-06-01 19:02:10.136 o.a.s.d.DRPCSpout Thread-28-spout-executor[194, 194] [ERROR] Failed to fetch DRPC request from DRPC server [search-storm-drpc-4.search-storm-drpc:3773] org.apache.storm.thrift.transport.TTransportException: Socket is closed by peer. at org.apache.storm.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:130) ~[storm-shaded-deps-2.4.0.jar:2.4.0] at org.apache.storm.thrift.transport.TTransport.readAll(TTransport.java:86) ~[storm-shaded-deps-2.4.0.jar:2.4.0] at org.apache.storm.thrift.transport.TFramedTransport.readFrame(TFramedTransport.java:141) ~[storm-shaded-deps-2.4.0.jar:2.4.0] at org.apache.storm.thrift.transport.TFramedTransport.read(TFramedTransport.java:109) ~[storm-shaded-deps-2.4.0.jar:2.4.0] at org.apache.storm.thrift.transport.TTransport.readAll(TTransport.java:86) ~[storm-shaded-deps-2.4.0.jar:2.4.0] at org.apache.storm.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:455) ~[storm-shaded-deps-2.4.0.jar:2.4.0] at org.apache.storm.thrift.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:354) ~[storm-shaded-deps-2.4.0.jar:2.4.0] at org.apache.storm.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:243) ~[storm-shaded-deps-2.4.0.jar:2.4.0] at org.apache.storm.thrift.TServiceClient.receiveBase(TServiceClient.java:77) ~[storm-shaded-deps-2.4.0.jar:2.4.0] at org.apache.storm.generated.DistributedRPCInvocations$Client.recv_fetchRequest(DistributedRPCInvocations.java:114) ~[storm-client-2.4.0.jar:2.4.0] at org.apache.storm.generated.DistributedRPCInvocations$Client.fetchRequest(DistributedRPCInvocations.java:101) ~[storm-client-2.4.0.jar:2.4.0] at org.apache.storm.drpc.DRPCInvocationsClient.fetchRequest(DRPCInvocationsClient.java:84) ~[storm-client-2.4.0.jar:2.4.0] at org.apache.storm.drpc.DRPCSpout.nextTuple(DRPCSpout.java:175) [storm-client-2.4.0.jar:2.4.0] at org.apache.storm.executor.spout.SpoutExecutor$2.call(SpoutExecutor.java:187) [storm-client-2.4.0.jar:2.4.0] at org.apache.storm.executor.spout.SpoutExecutor$2.call(SpoutExecutor.java:153) [storm-client-2.4.0.jar:2.4.0] at org.apache.storm.utils.Utils$1.run(Utils.java:396) [storm-client-2.4.0.jar:2.4.0] at java.lang.Thread.run(Thread.java:748) [?:1.8.0_211] On further debugging, I see all storm-drpc ports (3772, 3773, 3774) are listening to IPv6 addresses. I tried adding the JVM flag -Djava.net.preferIPv4Stack=true mentioned here https://storm.apache.org/releases/2.4.0/Troubleshooting.html but that also did not help. The weird thing is, telnet to the DRPC DNS names and ports work from all pods, but still, I see socket errors in supervisor pods while talking to DRPC. Any inputs on any difficulties or workaround to run Storm 2.4.0 in IPv6 addresses or make it successfully run by making ports listen to IPv4 addresses? Regards, Ash
Does storm 2.4.0 supports IPv6 address
Hi, I have a storm 2.4.0 cluster with all the storm components such as Nimbus, UI, supervisors, and drpc. Recently we moved to a new GCP Kubernetes cluster where both traffic was coming from both IPv4 and IPv6 addresses. Previously, we were on a hashicorp nomad cluster where only IPv4 was supported. Since the move to GCP Kubernetes (v1.23.x), I started to see supervisor is unable to communicate with drpc on port 3773. This is the error snippet from one of the topology logs inside a supervisor. 2023-06-01 19:02:10.136 o.a.s.d.DRPCSpout Thread-28-spout-executor[194, 194] [ERROR] Failed to fetch DRPC request from DRPC server [search-storm-drpc-4.search-storm-drpc:3773] org.apache.storm.thrift.transport.TTransportException: Socket is closed by peer. at org.apache.storm.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:130) ~[storm-shaded-deps-2.4.0.jar:2.4.0] at org.apache.storm.thrift.transport.TTransport.readAll(TTransport.java:86) ~[storm-shaded-deps-2.4.0.jar:2.4.0] at org.apache.storm.thrift.transport.TFramedTransport.readFrame(TFramedTransport.java:141) ~[storm-shaded-deps-2.4.0.jar:2.4.0] at org.apache.storm.thrift.transport.TFramedTransport.read(TFramedTransport.java:109) ~[storm-shaded-deps-2.4.0.jar:2.4.0] at org.apache.storm.thrift.transport.TTransport.readAll(TTransport.java:86) ~[storm-shaded-deps-2.4.0.jar:2.4.0] at org.apache.storm.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:455) ~[storm-shaded-deps-2.4.0.jar:2.4.0] at org.apache.storm.thrift.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:354) ~[storm-shaded-deps-2.4.0.jar:2.4.0] at org.apache.storm.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:243) ~[storm-shaded-deps-2.4.0.jar:2.4.0] at org.apache.storm.thrift.TServiceClient.receiveBase(TServiceClient.java:77) ~[storm-shaded-deps-2.4.0.jar:2.4.0] at org.apache.storm.generated.DistributedRPCInvocations$Client.recv_fetchRequest(DistributedRPCInvocations.java:114) ~[storm-client-2.4.0.jar:2.4.0] at org.apache.storm.generated.DistributedRPCInvocations$Client.fetchRequest(DistributedRPCInvocations.java:101) ~[storm-client-2.4.0.jar:2.4.0] at org.apache.storm.drpc.DRPCInvocationsClient.fetchRequest(DRPCInvocationsClient.java:84) ~[storm-client-2.4.0.jar:2.4.0] at org.apache.storm.drpc.DRPCSpout.nextTuple(DRPCSpout.java:175) [storm-client-2.4.0.jar:2.4.0] at org.apache.storm.executor.spout.SpoutExecutor$2.call(SpoutExecutor.java:187) [storm-client-2.4.0.jar:2.4.0] at org.apache.storm.executor.spout.SpoutExecutor$2.call(SpoutExecutor.java:153) [storm-client-2.4.0.jar:2.4.0] at org.apache.storm.utils.Utils$1.run(Utils.java:396) [storm-client-2.4.0.jar:2.4.0] at java.lang.Thread.run(Thread.java:748) [?:1.8.0_211] On further debugging, I see all storm-drpc ports (3772, 3773, 3774) are listening to IPv6 addresses. [image: image.png] I tried adding the JVM flag *-Djava.net.preferIPv4Stack=true *mentioned here https://storm.apache.org/releases/2.4.0/Troubleshooting.html but that also did not help. The weird thing is, telnet to the DRPC DNS names and ports work from all pods, but still, I see socket errors in supervisor pods while talking to DRPC. Any inputs on any difficulties or workaround to run Storm 2.4.0 in IPv6 addresses or make it successfully run by making ports listen to IPv4 addresses? Regards, Ash
Does storm 2.4.0 supports IPv6 address
Hi, I have a storm 2.4.0 cluster with all the storm components such as Nimbus, UI, supervisors, and drpc. Recently we moved to a new GCP Kubernetes cluster where both traffic was coming from both IPv4 and IPv6 addresses. Previously, we were on a hashicorp nomad cluster where only IPv4 was supported. Since the move to GCP Kubernetes (v1.23.x), I started to see supervisor is unable to communicate with drpc on port 3773. This is the error snippet from one of the topology logs inside a supervisor. 2023-06-01 19:02:10.136 o.a.s.d.DRPCSpout Thread-28-spout-executor[194, 194] [ERROR] Failed to fetch DRPC request from DRPC server [search-storm-drpc-4.search-storm-drpc:3773] org.apache.storm.thrift.transport.TTransportException: Socket is closed by peer. at org.apache.storm.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:130) ~[storm-shaded-deps-2.4.0.jar:2.4.0] at org.apache.storm.thrift.transport.TTransport.readAll(TTransport.java:86) ~[storm-shaded-deps-2.4.0.jar:2.4.0] at org.apache.storm.thrift.transport.TFramedTransport.readFrame(TFramedTransport.java:141) ~[storm-shaded-deps-2.4.0.jar:2.4.0] at org.apache.storm.thrift.transport.TFramedTransport.read(TFramedTransport.java:109) ~[storm-shaded-deps-2.4.0.jar:2.4.0] at org.apache.storm.thrift.transport.TTransport.readAll(TTransport.java:86) ~[storm-shaded-deps-2.4.0.jar:2.4.0] at org.apache.storm.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:455) ~[storm-shaded-deps-2.4.0.jar:2.4.0] at org.apache.storm.thrift.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:354) ~[storm-shaded-deps-2.4.0.jar:2.4.0] at org.apache.storm.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:243) ~[storm-shaded-deps-2.4.0.jar:2.4.0] at org.apache.storm.thrift.TServiceClient.receiveBase(TServiceClient.java:77) ~[storm-shaded-deps-2.4.0.jar:2.4.0] at org.apache.storm.generated.DistributedRPCInvocations$Client.recv_fetchRequest(DistributedRPCInvocations.java:114) ~[storm-client-2.4.0.jar:2.4.0] at org.apache.storm.generated.DistributedRPCInvocations$Client.fetchRequest(DistributedRPCInvocations.java:101) ~[storm-client-2.4.0.jar:2.4.0] at org.apache.storm.drpc.DRPCInvocationsClient.fetchRequest(DRPCInvocationsClient.java:84) ~[storm-client-2.4.0.jar:2.4.0] at org.apache.storm.drpc.DRPCSpout.nextTuple(DRPCSpout.java:175) [storm-client-2.4.0.jar:2.4.0] at org.apache.storm.executor.spout.SpoutExecutor$2.call(SpoutExecutor.java:187) [storm-client-2.4.0.jar:2.4.0] at org.apache.storm.executor.spout.SpoutExecutor$2.call(SpoutExecutor.java:153) [storm-client-2.4.0.jar:2.4.0] at org.apache.storm.utils.Utils$1.run(Utils.java:396) [storm-client-2.4.0.jar:2.4.0] at java.lang.Thread.run(Thread.java:748) [?:1.8.0_211] On further debugging, I see all storm-drpc ports (3772, 3773, 3774) are listening to IPv6 addresses. [image: image.png] I tried adding the JVM flag *-Djava.net.preferIPv4Stack=true *mentioned here https://storm.apache.org/releases/2.4.0/Troubleshooting.html but that also did not help. The weird thing is, telnet to the DRPC DNS names and ports work from all pods, but still, I see socket errors in supervisor pods while talking to DRPC. Any inputs on any difficulties or workaround to run Storm 2.4.0 in IPv6 addresses or make it successfully run by making ports listen to IPv4 addresses? Regards, Ash
[GitHub] [storm] bipinprasad merged pull request #3535: [STORM-3910] Enhanced logging for rocksdb metrics store.
bipinprasad merged PR #3535: URL: https://github.com/apache/storm/pull/3535 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@storm.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
