Re: [EXT] Datadog _ JMX Integration facing connection issues.
Sai Vamsi,
On 12/27/23 04:12, Bodavula, Sai Vamsi Mohan Krishna (TR Technology) wrote:
Hello chris,
thanks for adding a point and I may have missed your email.
below is some of the points, which showed me how tomcat is being started in my
application.
* Line 19: INFO [main][2023-12-27 09:00:38,983] -
org.springframework.boot.web.embedded.tomcat.TomcatWebServer.initialize(TomcatWebServer.java:108)
- Tomcat initialized with port(s): 9109 (http) 10109 (http)
Line 21: INFO [main][2023-12-27 09:00:39,008] -
org.apache.juli.logging.DirectJDKLog.log(DirectJDKLog.java:173) - Starting
service [Tomcat]
Line 22: INFO [main][2023-12-27 09:00:39,009] -
org.apache.juli.logging.DirectJDKLog.log(DirectJDKLog.java:173) - Starting
Servlet engine: [Apache Tomcat/9.0.75]
Line 134: INFO
[lab1workflow4scalsvc2zus1][version:1.7.99-alpha.1130][traceId:][spanId:][parentSpanId:][spanExportable:][main][2023-12-27
09:01:01,057]-
org.springframework.boot.web.embedded.tomcat.TomcatWebServer.start:Line 220 -
Tomcat started on port(s): 9109 (http) 10109 (http) with context path
'/lab1workflow4scalsvc2zus1
*
to launch tomcat, we are not using any command here, I am trying to build my
code to an image, making some helm chart out of it, using helm chart to install
in my AKS containers.
This may be the first time you mentioned Spring.
I don't know if Spring uses any of those environment variables for
anything at all.
You may need to consult the Spring documentation to see how best to set
all of those JVM parameters.
-chris
From: Christopher Schultz
Sent: Thursday, December 14, 2023 20:33
To: users@tomcat.apache.org
Subject: Re: [EXT] Datadog _ JMX Integration facing connection issues.
Sai Vamsi,
On 12/14/23 00:26, Bodavula, Sai Vamsi Mohan Krishna (TR Technology) wrote:
Hai Chuck.,
Thanks for jumping in.,
thats what my concern is.
as previously added by Chris, I have added my annotations in Catalina_opts .,
Which is preferred by Java_Process for adding the annotations., But Here by
using catalina_opts., they are not reflecting.
where as adding them in java-tool-options., Makes a process with Tomcat and a
process that listens on the mentioned port., but throws port is already
binded., as its an environment variable, its already looking for a port, where
as its already created., throws binding error.
Yesterday ., I thought of adding both the formats, adding java_tool_opts and
Catalina opts., inorder to utilize the previous thigns., as below
- name: JAVA_TOOL_OPTIONS
value: {{ .Values.deployment.javaVMMemoryArgument }}
- name: POD_IP
valueFrom:
fieldRef:
fieldPath: status.podIP
- name: CATALINA_OPTS
value: >-
-Dcom.sun.management.jmxremote
-Dcom.sun.management.jmxremote.port=49151
-Djava.rmi.server.hostname=$POD_IP
-Dcom.sun.management.jmxremote.authenticate=false
-Dcom.sun.management.jmxremote.ssl=false
-Dcom.sun.management.jmxremote.local.only=false
-Xms2048M -Xmx10240M
Java_tool_opts are as : -XX:+UseStringDeduplication -XX:+UseContainerSupport
-Dcom.sun.management.jmxremote
just added to initiate a JMX process.,
but nothing is working out.,
I would like to request you for further inputs
How are you launching Tomcat itself? Are you using catalina.sh or
something similar, or do you use some kind of embedded-launch?
It sounds to me like using CATALINA_OPTS is causing Tomcat not to start
at all. If you use JAVA_TOOL_OPTIONS (which Tomcat ignores), Tomcat will
start but won't initialize JMX in the way you want it to.
The solution is to look at the log files that are produced when Tomcat
launches (or fails to launch). Are you able to observe those?
It would be great to see the exact command Helm (or whatever) uses to
attempt to launch Tomcat, and then the stdout/stderr of that attempt.
The diagnostic messages shown when a JVM refuses to start are usually
pretty good. Same thing with Tomcat, though Tomcat usually starts even
if e.g. no connectors are successfully started because there is no
actual requirement for Tomcat to listen to any ports.
-chris
From: Chuck Caldarale
Sent: Thursday, December 14, 2023 00:28
To: Tomcat Users List
Subject: Re: [EXT] Datadog _ JMX Integration facing connection issues.
On Dec 13, 2023, at 10:36, Bodavula, Sai Vamsi Mohan Krishna (TR Technology)
wrote:
as you just asked .,
I do have a process with Catalina.
root@lab1workflow4scalsvc2zus1-deployment-659dd79df7-wg59g:/# netstat -tulpn
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State
PID/Program name
tcp6 0 0 :::34753:::*LIS
Re: [EXT] Datadog _ JMX Integration facing connection issues.
Hello chris,
thanks for adding a point and I may have missed your email.
below is some of the points, which showed me how tomcat is being started in my
application.
* Line 19: INFO [main][2023-12-27 09:00:38,983] -
org.springframework.boot.web.embedded.tomcat.TomcatWebServer.initialize(TomcatWebServer.java:108)
- Tomcat initialized with port(s): 9109 (http) 10109 (http)
Line 21: INFO [main][2023-12-27 09:00:39,008] -
org.apache.juli.logging.DirectJDKLog.log(DirectJDKLog.java:173) - Starting
service [Tomcat]
Line 22: INFO [main][2023-12-27 09:00:39,009] -
org.apache.juli.logging.DirectJDKLog.log(DirectJDKLog.java:173) - Starting
Servlet engine: [Apache Tomcat/9.0.75]
Line 134: INFO
[lab1workflow4scalsvc2zus1][version:1.7.99-alpha.1130][traceId:][spanId:][parentSpanId:][spanExportable:][main][2023-12-27
09:01:01,057]-
org.springframework.boot.web.embedded.tomcat.TomcatWebServer.start:Line 220 -
Tomcat started on port(s): 9109 (http) 10109 (http) with context path
'/lab1workflow4scalsvc2zus1
*
to launch tomcat, we are not using any command here, I am trying to build my
code to an image, making some helm chart out of it, using helm chart to install
in my AKS containers.
Thanks & Regards,
--
SAI VAMSI .B
Senior DevOps Engineer
From: Christopher Schultz
Sent: Thursday, December 14, 2023 20:33
To: users@tomcat.apache.org
Subject: Re: [EXT] Datadog _ JMX Integration facing connection issues.
Sai Vamsi,
On 12/14/23 00:26, Bodavula, Sai Vamsi Mohan Krishna (TR Technology) wrote:
> Hai Chuck.,
> Thanks for jumping in.,
>
> thats what my concern is.
> as previously added by Chris, I have added my annotations in Catalina_opts .,
> Which is preferred by Java_Process for adding the annotations., But Here by
> using catalina_opts., they are not reflecting.
>
> where as adding them in java-tool-options., Makes a process with Tomcat and a
> process that listens on the mentioned port., but throws port is already
> binded., as its an environment variable, its already looking for a port,
> where as its already created., throws binding error.
>
> Yesterday ., I thought of adding both the formats, adding java_tool_opts and
> Catalina opts., inorder to utilize the previous thigns., as below
>
> - name: JAVA_TOOL_OPTIONS
>value: {{ .Values.deployment.javaVMMemoryArgument }}
>
> - name: POD_IP
>valueFrom:
> fieldRef:
>fieldPath: status.podIP
> - name: CATALINA_OPTS
>value: >-
> -Dcom.sun.management.jmxremote
> -Dcom.sun.management.jmxremote.port=49151
> -Djava.rmi.server.hostname=$POD_IP
> -Dcom.sun.management.jmxremote.authenticate=false
> -Dcom.sun.management.jmxremote.ssl=false
> -Dcom.sun.management.jmxremote.local.only=false
> -Xms2048M -Xmx10240M
>
> Java_tool_opts are as : -XX:+UseStringDeduplication -XX:+UseContainerSupport
> -Dcom.sun.management.jmxremote
>
> just added to initiate a JMX process.,
>
> but nothing is working out.,
> I would like to request you for further inputs
How are you launching Tomcat itself? Are you using catalina.sh or
something similar, or do you use some kind of embedded-launch?
It sounds to me like using CATALINA_OPTS is causing Tomcat not to start
at all. If you use JAVA_TOOL_OPTIONS (which Tomcat ignores), Tomcat will
start but won't initialize JMX in the way you want it to.
The solution is to look at the log files that are produced when Tomcat
launches (or fails to launch). Are you able to observe those?
It would be great to see the exact command Helm (or whatever) uses to
attempt to launch Tomcat, and then the stdout/stderr of that attempt.
The diagnostic messages shown when a JVM refuses to start are usually
pretty good. Same thing with Tomcat, though Tomcat usually starts even
if e.g. no connectors are successfully started because there is no
actual requirement for Tomcat to listen to any ports.
-chris
> ____________________
> From: Chuck Caldarale
> Sent: Thursday, December 14, 2023 00:28
> To: Tomcat Users List
> Subject: Re: [EXT] Datadog _ JMX Integration facing connection issues.
>
>
>> On Dec 13, 2023, at 10:36, Bodavula, Sai Vamsi Mohan Krishna (TR Technology)
>> wrote:
>>
>> as you just asked .,
>> I do have a process with Catalina.
>>
>> root@lab1workflow4scalsvc2zus1-deployment-659dd79df7-wg59g:/# netstat -tulpn
>> Active Internet connections (only servers)
>> Proto Recv-Q Send-Q Local Address Foreign Address State
>>PID/Program name
>> tcp6 0 0 :::34753:::*
Re: [EXT] Datadog _ JMX Integration facing connection issues.
Sai Vamsi,
have you answered any of the questions Chris, Chuck and I asked?
Tomcat is NOT running, so please check "why" first.
We'd like to help, but can't see what is happening on your machine.
Kind regards
Peter
> Am 26.12.2023 um 12:02 schrieb Bodavula, Sai Vamsi Mohan Krishna (TR
> Technology) :
>
> by chance any findings on this.,
>
> Thanks & Regards,
> --
>
> SAI VAMSI .B
> Senior DevOps Engineer
>
>
> From: Bodavula, Sai Vamsi Mohan Krishna (TR Technology)
>
> Sent: Thursday, December 14, 2023 10:56
> To: Tomcat Users List
> Subject: Re: [EXT] Datadog _ JMX Integration facing connection issues.
>
> Hai Chuck.,
> Thanks for jumping in.,
>
> thats what my concern is.
> as previously added by Chris, I have added my annotations in Catalina_opts .,
> Which is preferred by Java_Process for adding the annotations., But Here by
> using catalina_opts., they are not reflecting.
>
> where as adding them in java-tool-options., Makes a process with Tomcat and a
> process that listens on the mentioned port., but throws port is already
> binded., as its an environment variable, its already looking for a port,
> where as its already created., throws binding error.
>
> Yesterday ., I thought of adding both the formats, adding java_tool_opts and
> Catalina opts., inorder to utilize the previous thigns., as below
>
> - name: JAVA_TOOL_OPTIONS
> value: {{ .Values.deployment.javaVMMemoryArgument }}
>
>- name: POD_IP
> valueFrom:
>fieldRef:
> fieldPath: status.podIP
>- name: CATALINA_OPTS
> value: >-
>-Dcom.sun.management.jmxremote
>-Dcom.sun.management.jmxremote.port=49151
>-Djava.rmi.server.hostname=$POD_IP
>-Dcom.sun.management.jmxremote.authenticate=false
>-Dcom.sun.management.jmxremote.ssl=false
>-Dcom.sun.management.jmxremote.local.only=false
>-Xms2048M -Xmx10240M
>
> Java_tool_opts are as : -XX:+UseStringDeduplication -XX:+UseContainerSupport
> -Dcom.sun.management.jmxremote
>
> just added to initiate a JMX process.,
>
> but nothing is working out.,
> I would like to request you for further inputs
>
>
> cc :chris
>
> Thanks & Regards,
> --
>
> SAI VAMSI .B
> Senior DevOps Engineer
>
>
> From: Chuck Caldarale
> Sent: Thursday, December 14, 2023 00:28
> To: Tomcat Users List
> Subject: Re: [EXT] Datadog _ JMX Integration facing connection issues.
>
>
>> On Dec 13, 2023, at 10:36, Bodavula, Sai Vamsi Mohan Krishna (TR Technology)
>> wrote:
>>
>> as you just asked .,
>> I do have a process with Catalina.
>>
>> root@lab1workflow4scalsvc2zus1-deployment-659dd79df7-wg59g:/# netstat -tulpn
>> Active Internet connections (only servers)
>> Proto Recv-Q Send-Q Local Address Foreign Address State
>>PID/Program name
>> tcp6 0 0 :::34753:::*LISTEN
>>1/java
>> tcp6 0 0 :::9109 :::*LISTEN
>>1/java
>> tcp6 0 0 :::10109:::*LISTEN
>>1/java
>> root@lab1workflow4scalsvc2zus1-deployment-659dd79df7-wg59g:/# ^C
>> root@lab1workflow4scalsvc2zus1-deployment-659dd79df7-wg59g:/# ps aux | grep
>> catalina
>> root 744 0.0 0.0 6460 680 pts/1S+ 11:47 0:00 grep
>> --color=auto catalina
>> root@lab1workflow4scalsvc2zus1-deployment-659dd79df7-wg59g:/#
>
>
> That shows only the grep process looking for catalina, not anything using
> catalina. If Tomcat were actually running, you’d see something like this
> (slightly reformatted for clarity):
>
> chuck@Chuck-MacBookPro apache-tomcat-9.0.83 > ps aux | grep catalina
> chuck16879 0.0 0.0 408626896 1376 s000 S+ 12:53PM
> 0:00.00 grep catalina
> chuck16874 0.0 0.9 415316912 153296 s000 S12:53PM
> 0:02.66
> /Library/Java/JavaVirtualMachines/temurin-21.jdk/Contents/Home/bin/java
> -Djava.util.logging.config.file=/Users/chuck/Downloads/apache-tomcat-9.0.83/conf/logging.properties
> -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager
> -Djdk.tls.ephemeralDHKeySize=2048
> -Djava.protocol.handler.pkgs=org.apache.catalina.webresources
> -Dorg.apache.catalina.security.SecurityListener.UMASK=0027
> -Dtest_port=9090
> -Dignore.endorsed.dirs=
> -classpath
> /Users/chuck/Downloads/apache-tomcat-
Re: [EXT] Datadog _ JMX Integration facing connection issues.
by chance any findings on this.,
Thanks & Regards,
--
SAI VAMSI .B
Senior DevOps Engineer
From: Bodavula, Sai Vamsi Mohan Krishna (TR Technology)
Sent: Thursday, December 14, 2023 10:56
To: Tomcat Users List
Subject: Re: [EXT] Datadog _ JMX Integration facing connection issues.
Hai Chuck.,
Thanks for jumping in.,
thats what my concern is.
as previously added by Chris, I have added my annotations in Catalina_opts .,
Which is preferred by Java_Process for adding the annotations., But Here by
using catalina_opts., they are not reflecting.
where as adding them in java-tool-options., Makes a process with Tomcat and a
process that listens on the mentioned port., but throws port is already
binded., as its an environment variable, its already looking for a port, where
as its already created., throws binding error.
Yesterday ., I thought of adding both the formats, adding java_tool_opts and
Catalina opts., inorder to utilize the previous thigns., as below
- name: JAVA_TOOL_OPTIONS
value: {{ .Values.deployment.javaVMMemoryArgument }}
- name: POD_IP
valueFrom:
fieldRef:
fieldPath: status.podIP
- name: CATALINA_OPTS
value: >-
-Dcom.sun.management.jmxremote
-Dcom.sun.management.jmxremote.port=49151
-Djava.rmi.server.hostname=$POD_IP
-Dcom.sun.management.jmxremote.authenticate=false
-Dcom.sun.management.jmxremote.ssl=false
-Dcom.sun.management.jmxremote.local.only=false
-Xms2048M -Xmx10240M
Java_tool_opts are as : -XX:+UseStringDeduplication -XX:+UseContainerSupport
-Dcom.sun.management.jmxremote
just added to initiate a JMX process.,
but nothing is working out.,
I would like to request you for further inputs
cc :chris
Thanks & Regards,
--
SAI VAMSI .B
Senior DevOps Engineer
From: Chuck Caldarale
Sent: Thursday, December 14, 2023 00:28
To: Tomcat Users List
Subject: Re: [EXT] Datadog _ JMX Integration facing connection issues.
> On Dec 13, 2023, at 10:36, Bodavula, Sai Vamsi Mohan Krishna (TR Technology)
> wrote:
>
> as you just asked .,
> I do have a process with Catalina.
>
> root@lab1workflow4scalsvc2zus1-deployment-659dd79df7-wg59g:/# netstat -tulpn
> Active Internet connections (only servers)
> Proto Recv-Q Send-Q Local Address Foreign Address State
> PID/Program name
> tcp6 0 0 :::34753:::*LISTEN
> 1/java
> tcp6 0 0 :::9109 :::*LISTEN
> 1/java
> tcp6 0 0 :::10109:::*LISTEN
> 1/java
> root@lab1workflow4scalsvc2zus1-deployment-659dd79df7-wg59g:/# ^C
> root@lab1workflow4scalsvc2zus1-deployment-659dd79df7-wg59g:/# ps aux | grep
> catalina
> root 744 0.0 0.0 6460 680 pts/1S+ 11:47 0:00 grep
> --color=auto catalina
> root@lab1workflow4scalsvc2zus1-deployment-659dd79df7-wg59g:/#
That shows only the grep process looking for catalina, not anything using
catalina. If Tomcat were actually running, you’d see something like this
(slightly reformatted for clarity):
chuck@Chuck-MacBookPro apache-tomcat-9.0.83 > ps aux | grep catalina
chuck16879 0.0 0.0 408626896 1376 s000 S+ 12:53PM 0:00.00
grep catalina
chuck16874 0.0 0.9 415316912 153296 s000 S12:53PM 0:02.66
/Library/Java/JavaVirtualMachines/temurin-21.jdk/Contents/Home/bin/java
-Djava.util.logging.config.file=/Users/chuck/Downloads/apache-tomcat-9.0.83/conf/logging.properties
-Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager
-Djdk.tls.ephemeralDHKeySize=2048
-Djava.protocol.handler.pkgs=org.apache.catalina.webresources
-Dorg.apache.catalina.security.SecurityListener.UMASK=0027
-Dtest_port=9090
-Dignore.endorsed.dirs=
-classpath
/Users/chuck/Downloads/apache-tomcat-9.0.83/bin/bootstrap.jar:/Users/chuck/Downloads/apache-tomcat-9.0.83/bin/tomcat-juli.jar
-Dcatalina.base=/Users/chuck/Downloads/apache-tomcat-9.0.83
-Dcatalina.home=/Users/chuck/Downloads/apache-tomcat-9.0.83
-Djava.io.tmpdir=/Users/chuck/Downloads/apache-tomcat-9.0.83/temp
org.apache.catalina.startup.Bootstrap
start
- Chuck
Re: [EXT] Datadog _ JMX Integration facing connection issues.
Sai Vamsi,
On 12/14/23 00:26, Bodavula, Sai Vamsi Mohan Krishna (TR Technology) wrote:
Hai Chuck.,
Thanks for jumping in.,
thats what my concern is.
as previously added by Chris, I have added my annotations in Catalina_opts .,
Which is preferred by Java_Process for adding the annotations., But Here by
using catalina_opts., they are not reflecting.
where as adding them in java-tool-options., Makes a process with Tomcat and a
process that listens on the mentioned port., but throws port is already
binded., as its an environment variable, its already looking for a port, where
as its already created., throws binding error.
Yesterday ., I thought of adding both the formats, adding java_tool_opts and
Catalina opts., inorder to utilize the previous thigns., as below
- name: JAVA_TOOL_OPTIONS
value: {{ .Values.deployment.javaVMMemoryArgument }}
- name: POD_IP
valueFrom:
fieldRef:
fieldPath: status.podIP
- name: CATALINA_OPTS
value: >-
-Dcom.sun.management.jmxremote
-Dcom.sun.management.jmxremote.port=49151
-Djava.rmi.server.hostname=$POD_IP
-Dcom.sun.management.jmxremote.authenticate=false
-Dcom.sun.management.jmxremote.ssl=false
-Dcom.sun.management.jmxremote.local.only=false
-Xms2048M -Xmx10240M
Java_tool_opts are as : -XX:+UseStringDeduplication -XX:+UseContainerSupport
-Dcom.sun.management.jmxremote
just added to initiate a JMX process.,
but nothing is working out.,
I would like to request you for further inputs
How are you launching Tomcat itself? Are you using catalina.sh or
something similar, or do you use some kind of embedded-launch?
It sounds to me like using CATALINA_OPTS is causing Tomcat not to start
at all. If you use JAVA_TOOL_OPTIONS (which Tomcat ignores), Tomcat will
start but won't initialize JMX in the way you want it to.
The solution is to look at the log files that are produced when Tomcat
launches (or fails to launch). Are you able to observe those?
It would be great to see the exact command Helm (or whatever) uses to
attempt to launch Tomcat, and then the stdout/stderr of that attempt.
The diagnostic messages shown when a JVM refuses to start are usually
pretty good. Same thing with Tomcat, though Tomcat usually starts even
if e.g. no connectors are successfully started because there is no
actual requirement for Tomcat to listen to any ports.
-chris
From: Chuck Caldarale
Sent: Thursday, December 14, 2023 00:28
To: Tomcat Users List
Subject: Re: [EXT] Datadog _ JMX Integration facing connection issues.
On Dec 13, 2023, at 10:36, Bodavula, Sai Vamsi Mohan Krishna (TR Technology)
wrote:
as you just asked .,
I do have a process with Catalina.
root@lab1workflow4scalsvc2zus1-deployment-659dd79df7-wg59g:/# netstat -tulpn
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State
PID/Program name
tcp6 0 0 :::34753:::*LISTEN
1/java
tcp6 0 0 :::9109 :::*LISTEN
1/java
tcp6 0 0 :::10109:::*LISTEN
1/java
root@lab1workflow4scalsvc2zus1-deployment-659dd79df7-wg59g:/# ^C
root@lab1workflow4scalsvc2zus1-deployment-659dd79df7-wg59g:/# ps aux | grep
catalina
root 744 0.0 0.0 6460 680 pts/1S+ 11:47 0:00 grep
--color=auto catalina
root@lab1workflow4scalsvc2zus1-deployment-659dd79df7-wg59g:/#
That shows only the grep process looking for catalina, not anything using
catalina. If Tomcat were actually running, you’d see something like this
(slightly reformatted for clarity):
chuck@Chuck-MacBookPro apache-tomcat-9.0.83 > ps aux | grep catalina
chuck16879 0.0 0.0 408626896 1376 s000 S+ 12:53PM 0:00.00
grep catalina
chuck16874 0.0 0.9 415316912 153296 s000 S12:53PM 0:02.66
/Library/Java/JavaVirtualMachines/temurin-21.jdk/Contents/Home/bin/java
-Djava.util.logging.config.file=/Users/chuck/Downloads/apache-tomcat-9.0.83/conf/logging.properties
-Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager
-Djdk.tls.ephemeralDHKeySize=2048
-Djava.protocol.handler.pkgs=org.apache.catalina.webresources
-Dorg.apache.catalina.security.SecurityListener.UMASK=0027
-Dtest_port=9090
-Dignore.endorsed.dirs=
-classpath
/Users/chuck/Downloads/apache-tomcat-9.0.83/bin/bootstrap.jar:/Users/chuck/Downloads/apache-tomcat-9.0.83/bin/tomcat-juli.jar
-Dcatalina.base=/Users/chuck/Downloads/apache-tomcat-9.0.83
-Dcatalina.home=/Users/chuck/Downloads/apache-tomcat-9.0.83
-Djava.io.tmpdir=/Users/chuck/Downloads/apache-tomcat-9.0.83/temp
org.apache.catalina.startup.Bootstrap
start
-
Re: [EXT] Datadog _ JMX Integration facing connection issues.
Sai Vamsi, > Am 13.12.2023 um 19:59 schrieb Chuck Caldarale : > > >> On Dec 13, 2023, at 10:36, Bodavula, Sai Vamsi Mohan Krishna (TR Technology) >> wrote: >> >> as you just asked ., >> I do have a process with Catalina. >> >> root@lab1workflow4scalsvc2zus1-deployment-659dd79df7-wg59g:/# netstat -tulpn >> Active Internet connections (only servers) >> Proto Recv-Q Send-Q Local Address Foreign Address State >>PID/Program name >> tcp6 0 0 :::34753:::*LISTEN >>1/java >> tcp6 0 0 :::9109 :::*LISTEN >>1/java >> tcp6 0 0 :::10109:::*LISTEN >>1/java >> root@lab1workflow4scalsvc2zus1-deployment-659dd79df7-wg59g:/# ^C >> root@lab1workflow4scalsvc2zus1-deployment-659dd79df7-wg59g:/# ps aux | grep >> catalina >> root 744 0.0 0.0 6460 680 pts/1S+ 11:47 0:00 grep >> --color=auto catalina >> root@lab1workflow4scalsvc2zus1-deployment-659dd79df7-wg59g:/# > > you have to figure out WHY tomcat is not starting! There should be log files or error messages on the console. It seems you have put an error somewhere in any of the configfiles. It's not at all a question of the ports not being allocated. Take a step back and make tomcat launch again. After that we figure out where you have to set the options... Please detail how you start tomcat and show the output of startup (the beginning and last lines should be enough). Again, don't put any java options for tomcat in any global environment options (JAVA_OPTS, CATALINA_OPTS) in your shell. Only in setenv.sh . Peter > That shows only the grep process looking for catalina, not anything using > catalina. If Tomcat were actually running, you’d see something like this > (slightly reformatted for clarity): > > chuck@Chuck-MacBookPro apache-tomcat-9.0.83 > ps aux | grep catalina > chuck16879 0.0 0.0 408626896 1376 s000 S+ 12:53PM > 0:00.00 grep catalina > chuck16874 0.0 0.9 415316912 153296 s000 S12:53PM > 0:02.66 > /Library/Java/JavaVirtualMachines/temurin-21.jdk/Contents/Home/bin/java > -Djava.util.logging.config.file=/Users/chuck/Downloads/apache-tomcat-9.0.83/conf/logging.properties > -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager > -Djdk.tls.ephemeralDHKeySize=2048 > -Djava.protocol.handler.pkgs=org.apache.catalina.webresources > -Dorg.apache.catalina.security.SecurityListener.UMASK=0027 > -Dtest_port=9090 > -Dignore.endorsed.dirs= > -classpath > /Users/chuck/Downloads/apache-tomcat-9.0.83/bin/bootstrap.jar:/Users/chuck/Downloads/apache-tomcat-9.0.83/bin/tomcat-juli.jar > -Dcatalina.base=/Users/chuck/Downloads/apache-tomcat-9.0.83 > -Dcatalina.home=/Users/chuck/Downloads/apache-tomcat-9.0.83 > -Djava.io.tmpdir=/Users/chuck/Downloads/apache-tomcat-9.0.83/temp > org.apache.catalina.startup.Bootstrap > start > > > - Chuck > - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: [EXT] Datadog _ JMX Integration facing connection issues.
Hai Chuck.,
Thanks for jumping in.,
thats what my concern is.
as previously added by Chris, I have added my annotations in Catalina_opts .,
Which is preferred by Java_Process for adding the annotations., But Here by
using catalina_opts., they are not reflecting.
where as adding them in java-tool-options., Makes a process with Tomcat and a
process that listens on the mentioned port., but throws port is already
binded., as its an environment variable, its already looking for a port, where
as its already created., throws binding error.
Yesterday ., I thought of adding both the formats, adding java_tool_opts and
Catalina opts., inorder to utilize the previous thigns., as below
- name: JAVA_TOOL_OPTIONS
value: {{ .Values.deployment.javaVMMemoryArgument }}
- name: POD_IP
valueFrom:
fieldRef:
fieldPath: status.podIP
- name: CATALINA_OPTS
value: >-
-Dcom.sun.management.jmxremote
-Dcom.sun.management.jmxremote.port=49151
-Djava.rmi.server.hostname=$POD_IP
-Dcom.sun.management.jmxremote.authenticate=false
-Dcom.sun.management.jmxremote.ssl=false
-Dcom.sun.management.jmxremote.local.only=false
-Xms2048M -Xmx10240M
Java_tool_opts are as : -XX:+UseStringDeduplication -XX:+UseContainerSupport
-Dcom.sun.management.jmxremote
just added to initiate a JMX process.,
but nothing is working out.,
I would like to request you for further inputs
cc :chris
Thanks & Regards,
--
SAI VAMSI .B
Senior DevOps Engineer
From: Chuck Caldarale
Sent: Thursday, December 14, 2023 00:28
To: Tomcat Users List
Subject: Re: [EXT] Datadog _ JMX Integration facing connection issues.
> On Dec 13, 2023, at 10:36, Bodavula, Sai Vamsi Mohan Krishna (TR Technology)
> wrote:
>
> as you just asked .,
> I do have a process with Catalina.
>
> root@lab1workflow4scalsvc2zus1-deployment-659dd79df7-wg59g:/# netstat -tulpn
> Active Internet connections (only servers)
> Proto Recv-Q Send-Q Local Address Foreign Address State
> PID/Program name
> tcp6 0 0 :::34753:::*LISTEN
> 1/java
> tcp6 0 0 :::9109 :::*LISTEN
> 1/java
> tcp6 0 0 :::10109:::*LISTEN
> 1/java
> root@lab1workflow4scalsvc2zus1-deployment-659dd79df7-wg59g:/# ^C
> root@lab1workflow4scalsvc2zus1-deployment-659dd79df7-wg59g:/# ps aux | grep
> catalina
> root 744 0.0 0.0 6460 680 pts/1S+ 11:47 0:00 grep
> --color=auto catalina
> root@lab1workflow4scalsvc2zus1-deployment-659dd79df7-wg59g:/#
That shows only the grep process looking for catalina, not anything using
catalina. If Tomcat were actually running, you’d see something like this
(slightly reformatted for clarity):
chuck@Chuck-MacBookPro apache-tomcat-9.0.83 > ps aux | grep catalina
chuck16879 0.0 0.0 408626896 1376 s000 S+ 12:53PM 0:00.00
grep catalina
chuck16874 0.0 0.9 415316912 153296 s000 S12:53PM 0:02.66
/Library/Java/JavaVirtualMachines/temurin-21.jdk/Contents/Home/bin/java
-Djava.util.logging.config.file=/Users/chuck/Downloads/apache-tomcat-9.0.83/conf/logging.properties
-Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager
-Djdk.tls.ephemeralDHKeySize=2048
-Djava.protocol.handler.pkgs=org.apache.catalina.webresources
-Dorg.apache.catalina.security.SecurityListener.UMASK=0027
-Dtest_port=9090
-Dignore.endorsed.dirs=
-classpath
/Users/chuck/Downloads/apache-tomcat-9.0.83/bin/bootstrap.jar:/Users/chuck/Downloads/apache-tomcat-9.0.83/bin/tomcat-juli.jar
-Dcatalina.base=/Users/chuck/Downloads/apache-tomcat-9.0.83
-Dcatalina.home=/Users/chuck/Downloads/apache-tomcat-9.0.83
-Djava.io.tmpdir=/Users/chuck/Downloads/apache-tomcat-9.0.83/temp
org.apache.catalina.startup.Bootstrap
start
- Chuck
Re: [EXT] Datadog _ JMX Integration facing connection issues.
> On Dec 13, 2023, at 10:36, Bodavula, Sai Vamsi Mohan Krishna (TR Technology) > wrote: > > as you just asked ., > I do have a process with Catalina. > > root@lab1workflow4scalsvc2zus1-deployment-659dd79df7-wg59g:/# netstat -tulpn > Active Internet connections (only servers) > Proto Recv-Q Send-Q Local Address Foreign Address State > PID/Program name > tcp6 0 0 :::34753:::*LISTEN > 1/java > tcp6 0 0 :::9109 :::*LISTEN > 1/java > tcp6 0 0 :::10109:::*LISTEN > 1/java > root@lab1workflow4scalsvc2zus1-deployment-659dd79df7-wg59g:/# ^C > root@lab1workflow4scalsvc2zus1-deployment-659dd79df7-wg59g:/# ps aux | grep > catalina > root 744 0.0 0.0 6460 680 pts/1S+ 11:47 0:00 grep > --color=auto catalina > root@lab1workflow4scalsvc2zus1-deployment-659dd79df7-wg59g:/# That shows only the grep process looking for catalina, not anything using catalina. If Tomcat were actually running, you’d see something like this (slightly reformatted for clarity): chuck@Chuck-MacBookPro apache-tomcat-9.0.83 > ps aux | grep catalina chuck16879 0.0 0.0 408626896 1376 s000 S+ 12:53PM 0:00.00 grep catalina chuck16874 0.0 0.9 415316912 153296 s000 S12:53PM 0:02.66 /Library/Java/JavaVirtualMachines/temurin-21.jdk/Contents/Home/bin/java -Djava.util.logging.config.file=/Users/chuck/Downloads/apache-tomcat-9.0.83/conf/logging.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Djdk.tls.ephemeralDHKeySize=2048 -Djava.protocol.handler.pkgs=org.apache.catalina.webresources -Dorg.apache.catalina.security.SecurityListener.UMASK=0027 -Dtest_port=9090 -Dignore.endorsed.dirs= -classpath /Users/chuck/Downloads/apache-tomcat-9.0.83/bin/bootstrap.jar:/Users/chuck/Downloads/apache-tomcat-9.0.83/bin/tomcat-juli.jar -Dcatalina.base=/Users/chuck/Downloads/apache-tomcat-9.0.83 -Dcatalina.home=/Users/chuck/Downloads/apache-tomcat-9.0.83 -Djava.io.tmpdir=/Users/chuck/Downloads/apache-tomcat-9.0.83/temp org.apache.catalina.startup.Bootstrap start - Chuck
Re: [EXT] Datadog _ JMX Integration facing connection issues.
Sai Vamsi,
> Am 08.12.2023 um 06:43 schrieb Bodavula, Sai Vamsi Mohan Krishna (TR
> Technology) :
>
> Hey Christopher.,
> Greetings of the day.
>
>
> 1.
> Might I have confused you with posting the arguments directly ., Yeah as i
> just shared you the annotations with comments , to state you the stuff i am
> using., But in my deployment ., I am using them in catalina opts., and trying
> to call them from values.yaml., which looks like this :
>
> javaVMMemoryArgument: "-Xms2048M -Xmx10240M -XX:+UseStringDeduplication
> -XX:+UseContainerSupport -Dcom.sun.management.jmxremote
> -Dcom.sun.management.jmxremote.authenticate=false
> -Dcom.sun.management.jmxremote.ssl=false
> -Dcom.sun.management.jmxremote.local.only=false
> -Dcom.sun.management.jmxremote.port=49151
> -Djava.rmi.server.hostname=lab1workflow4scalsvc2zus1-service.hqm-lab1.svc.cluster.local
> "
>
> and I am referring the word "javaVMMemoryArgument" from values yaml and
> calling it in Catalina_opts, so that it would fetch all these
> annotations as mentioned above, during the deployment. This is my deployment
> part., where I am referring to the above values from values.yaml
> env:
>- name: CATALINA_OPTS
> value: {{ .Values.deployment.javaVMMemoryArgument }}
>
>
I compare with my CATALINA_OPTS (and I noticed that some of my options were
still in JAVA_OPTS boooh!) :
export CATALINA_OPTS="${CATALINA_OPTS} -Dcom.sun.management.jmxremote
-Dcom.sun.management.jmxremote.port=10001
-Dcom.sun.management.jmxremote.rmi.port=10002
-Dcom.sun.management.jmxremote.authenticate=false
-Dcom.sun.management.jmxremote.ssl=false
-Djava.rmi.server.hostname=${HOSTNAME}
-Dcom.sun.management.jmxremote.local.only=false"
I see all my options in
root@tomcat:/usr/local/tomcat# ps -aef | less
UID PIDPPID C STIME TTY TIME CMD
tomcat 1 0 71 10:42 ?00:04:02 /opt/java/openjdk/bin/java
-Djava.util.logging.config.file=/opt/apache-tomcat.base/conf/logging.properties
-Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager
-Djdk.tls.ephemeralDHKeySize=2048
-Djava.protocol.handler.pkgs=org.apache.catalina.webresources
-Dorg.apache.catalina.security.SecurityListener.UMASK=0027
-agentlib:jdwp=transport=dt_socket,address=0.0.0.0:8000,server=y,suspend=n
-XX:NativeMemoryTracking=summary -Dhostname= -Djava.awt.headless=true
-Djavax.net.ssl.trustStore=/opt/apache-tomcat.base/conf/ssl/cacerts.jks
-Xlog:gc:/opt/apache-tomcat.base/logs/gc.log
-Djava.security.egd=file:/dev/urandom -Dsun.net.inetaddr.ttl=60
-Djava.library.path=/usr/local/tomcat/native-jni-lib
-Djdk.tls.ephemeralDHKeySize=2048
-Djdk.tls.rejectClientInitiatedRenegotiation=true
-Djdk.tls.server.enableStatusRequestExtension=true
-Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=10001
-Dcom.sun.management.jmxremote.rmi.port=10002
-Dcom.sun.management.jmxremote.authenticate=false
-Dcom.sun.management.jmxremote.ssl=false -Djava.rmi.server.hostname=
-Dcom.sun.management.jmxremote.local.only=false
-javaagent:/opt/apache-tomcat.base/bin/jmx_prometheus_javaagent-0.12.0.jar=8080:/opt/apache-tomcat.base/bin/tomcat.yaml
-Djava.net.debug=ssl -XX:+UnlockDiagnosticVMOptions -Dignore.endorsed.dirs=
-classpath
/usr/local/tomcat/bin/bootstrap.jar:/usr/local/tomcat/bin/tomcat-juli.jar
-Dcatalina.base=/opt/apache-tomcat.base -Dcatalina.home=/usr/local/tomcat
-Djava.io.tmpdir=/opt/apache-tomcat.base/temp
org.apache.catalina.startup.Bootstrap start
Ports are opened
root@tomcat:/usr/local/tomcat# netstat -tulpn
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address Foreign Address State
PID/Program name
...
tcp0 0 0.0.0.0:10002 0.0.0.0:* LISTEN
-
tcp0 0 0.0.0.0:10001 0.0.0.0:* LISTEN
-
...
tcp2 0 0.0.0.0:84430.0.0.0:* LISTEN
-
...
jconsole :10001
> JMX connect success
>
>
>
> 1.
> Coming to Process., I have searched for Java process that listens on my
> mentioned port ie., 49151, but none of the process is listening to that
> process.
>
> I even tried with
> root@lab1workflow4scalsvc2zus1-deployment-fd64ff775-cwzn6:/# netstat -tulpn |
> grep LISTEN
> tcp6 0 0 :::10109:::*LISTEN
> 1/java
> tcp6 0 0 :::9109 :::*LISTEN
> 1/java
> root@lab1workflow4scalsvc2zus1-deployment-fd64ff775-cwzn6:/# netstat -tulpn |
> more
> Active Internet connections (only servers)
> Proto Recv-Q Send-Q Local Address Foreign Address State
> PID/Program name
> tcp6 0 0 :::10109
