Re: [EXT] Datadog _ JMX Integration facing connection issues.
Sai Vamsi, On 12/27/23 04:12, Bodavula, Sai Vamsi Mohan Krishna (TR Technology) wrote: Hello chris, thanks for adding a point and I may have missed your email. below is some of the points, which showed me how tomcat is being started in my application. * Line 19: INFO [main][2023-12-27 09:00:38,983] - org.springframework.boot.web.embedded.tomcat.TomcatWebServer.initialize(TomcatWebServer.java:108) - Tomcat initialized with port(s): 9109 (http) 10109 (http) Line 21: INFO [main][2023-12-27 09:00:39,008] - org.apache.juli.logging.DirectJDKLog.log(DirectJDKLog.java:173) - Starting service [Tomcat] Line 22: INFO [main][2023-12-27 09:00:39,009] - org.apache.juli.logging.DirectJDKLog.log(DirectJDKLog.java:173) - Starting Servlet engine: [Apache Tomcat/9.0.75] Line 134: INFO [lab1workflow4scalsvc2zus1][version:1.7.99-alpha.1130][traceId:][spanId:][parentSpanId:][spanExportable:][main][2023-12-27 09:01:01,057]- org.springframework.boot.web.embedded.tomcat.TomcatWebServer.start:Line 220 - Tomcat started on port(s): 9109 (http) 10109 (http) with context path '/lab1workflow4scalsvc2zus1 * to launch tomcat, we are not using any command here, I am trying to build my code to an image, making some helm chart out of it, using helm chart to install in my AKS containers. This may be the first time you mentioned Spring. I don't know if Spring uses any of those environment variables for anything at all. You may need to consult the Spring documentation to see how best to set all of those JVM parameters. -chris From: Christopher Schultz Sent: Thursday, December 14, 2023 20:33 To: users@tomcat.apache.org Subject: Re: [EXT] Datadog _ JMX Integration facing connection issues. Sai Vamsi, On 12/14/23 00:26, Bodavula, Sai Vamsi Mohan Krishna (TR Technology) wrote: Hai Chuck., Thanks for jumping in., thats what my concern is. as previously added by Chris, I have added my annotations in Catalina_opts ., Which is preferred by Java_Process for adding the annotations., But Here by using catalina_opts., they are not reflecting. where as adding them in java-tool-options., Makes a process with Tomcat and a process that listens on the mentioned port., but throws port is already binded., as its an environment variable, its already looking for a port, where as its already created., throws binding error. Yesterday ., I thought of adding both the formats, adding java_tool_opts and Catalina opts., inorder to utilize the previous thigns., as below - name: JAVA_TOOL_OPTIONS value: {{ .Values.deployment.javaVMMemoryArgument }} - name: POD_IP valueFrom: fieldRef: fieldPath: status.podIP - name: CATALINA_OPTS value: >- -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=49151 -Djava.rmi.server.hostname=$POD_IP -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.local.only=false -Xms2048M -Xmx10240M Java_tool_opts are as : -XX:+UseStringDeduplication -XX:+UseContainerSupport -Dcom.sun.management.jmxremote just added to initiate a JMX process., but nothing is working out., I would like to request you for further inputs How are you launching Tomcat itself? Are you using catalina.sh or something similar, or do you use some kind of embedded-launch? It sounds to me like using CATALINA_OPTS is causing Tomcat not to start at all. If you use JAVA_TOOL_OPTIONS (which Tomcat ignores), Tomcat will start but won't initialize JMX in the way you want it to. The solution is to look at the log files that are produced when Tomcat launches (or fails to launch). Are you able to observe those? It would be great to see the exact command Helm (or whatever) uses to attempt to launch Tomcat, and then the stdout/stderr of that attempt. The diagnostic messages shown when a JVM refuses to start are usually pretty good. Same thing with Tomcat, though Tomcat usually starts even if e.g. no connectors are successfully started because there is no actual requirement for Tomcat to listen to any ports. -chris From: Chuck Caldarale Sent: Thursday, December 14, 2023 00:28 To: Tomcat Users List Subject: Re: [EXT] Datadog _ JMX Integration facing connection issues. On Dec 13, 2023, at 10:36, Bodavula, Sai Vamsi Mohan Krishna (TR Technology) wrote: as you just asked ., I do have a process with Catalina. root@lab1workflow4scalsvc2zus1-deployment-659dd79df7-wg59g:/# netstat -tulpn Active Internet connections (only servers) Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name tcp6 0 0 :::34753:::*LIS
Re: [EXT] Datadog _ JMX Integration facing connection issues.
Hello chris, thanks for adding a point and I may have missed your email. below is some of the points, which showed me how tomcat is being started in my application. * Line 19: INFO [main][2023-12-27 09:00:38,983] - org.springframework.boot.web.embedded.tomcat.TomcatWebServer.initialize(TomcatWebServer.java:108) - Tomcat initialized with port(s): 9109 (http) 10109 (http) Line 21: INFO [main][2023-12-27 09:00:39,008] - org.apache.juli.logging.DirectJDKLog.log(DirectJDKLog.java:173) - Starting service [Tomcat] Line 22: INFO [main][2023-12-27 09:00:39,009] - org.apache.juli.logging.DirectJDKLog.log(DirectJDKLog.java:173) - Starting Servlet engine: [Apache Tomcat/9.0.75] Line 134: INFO [lab1workflow4scalsvc2zus1][version:1.7.99-alpha.1130][traceId:][spanId:][parentSpanId:][spanExportable:][main][2023-12-27 09:01:01,057]- org.springframework.boot.web.embedded.tomcat.TomcatWebServer.start:Line 220 - Tomcat started on port(s): 9109 (http) 10109 (http) with context path '/lab1workflow4scalsvc2zus1 * to launch tomcat, we are not using any command here, I am trying to build my code to an image, making some helm chart out of it, using helm chart to install in my AKS containers. Thanks & Regards, -- SAI VAMSI .B Senior DevOps Engineer From: Christopher Schultz Sent: Thursday, December 14, 2023 20:33 To: users@tomcat.apache.org Subject: Re: [EXT] Datadog _ JMX Integration facing connection issues. Sai Vamsi, On 12/14/23 00:26, Bodavula, Sai Vamsi Mohan Krishna (TR Technology) wrote: > Hai Chuck., > Thanks for jumping in., > > thats what my concern is. > as previously added by Chris, I have added my annotations in Catalina_opts ., > Which is preferred by Java_Process for adding the annotations., But Here by > using catalina_opts., they are not reflecting. > > where as adding them in java-tool-options., Makes a process with Tomcat and a > process that listens on the mentioned port., but throws port is already > binded., as its an environment variable, its already looking for a port, > where as its already created., throws binding error. > > Yesterday ., I thought of adding both the formats, adding java_tool_opts and > Catalina opts., inorder to utilize the previous thigns., as below > > - name: JAVA_TOOL_OPTIONS >value: {{ .Values.deployment.javaVMMemoryArgument }} > > - name: POD_IP >valueFrom: > fieldRef: >fieldPath: status.podIP > - name: CATALINA_OPTS >value: >- > -Dcom.sun.management.jmxremote > -Dcom.sun.management.jmxremote.port=49151 > -Djava.rmi.server.hostname=$POD_IP > -Dcom.sun.management.jmxremote.authenticate=false > -Dcom.sun.management.jmxremote.ssl=false > -Dcom.sun.management.jmxremote.local.only=false > -Xms2048M -Xmx10240M > > Java_tool_opts are as : -XX:+UseStringDeduplication -XX:+UseContainerSupport > -Dcom.sun.management.jmxremote > > just added to initiate a JMX process., > > but nothing is working out., > I would like to request you for further inputs How are you launching Tomcat itself? Are you using catalina.sh or something similar, or do you use some kind of embedded-launch? It sounds to me like using CATALINA_OPTS is causing Tomcat not to start at all. If you use JAVA_TOOL_OPTIONS (which Tomcat ignores), Tomcat will start but won't initialize JMX in the way you want it to. The solution is to look at the log files that are produced when Tomcat launches (or fails to launch). Are you able to observe those? It would be great to see the exact command Helm (or whatever) uses to attempt to launch Tomcat, and then the stdout/stderr of that attempt. The diagnostic messages shown when a JVM refuses to start are usually pretty good. Same thing with Tomcat, though Tomcat usually starts even if e.g. no connectors are successfully started because there is no actual requirement for Tomcat to listen to any ports. -chris > ____________________ > From: Chuck Caldarale > Sent: Thursday, December 14, 2023 00:28 > To: Tomcat Users List > Subject: Re: [EXT] Datadog _ JMX Integration facing connection issues. > > >> On Dec 13, 2023, at 10:36, Bodavula, Sai Vamsi Mohan Krishna (TR Technology) >> wrote: >> >> as you just asked ., >> I do have a process with Catalina. >> >> root@lab1workflow4scalsvc2zus1-deployment-659dd79df7-wg59g:/# netstat -tulpn >> Active Internet connections (only servers) >> Proto Recv-Q Send-Q Local Address Foreign Address State >>PID/Program name >> tcp6 0 0 :::34753:::*
Re: [EXT] Datadog _ JMX Integration facing connection issues.
Sai Vamsi, have you answered any of the questions Chris, Chuck and I asked? Tomcat is NOT running, so please check "why" first. We'd like to help, but can't see what is happening on your machine. Kind regards Peter > Am 26.12.2023 um 12:02 schrieb Bodavula, Sai Vamsi Mohan Krishna (TR > Technology) : > > by chance any findings on this., > > Thanks & Regards, > -- > > SAI VAMSI .B > Senior DevOps Engineer > > > From: Bodavula, Sai Vamsi Mohan Krishna (TR Technology) > > Sent: Thursday, December 14, 2023 10:56 > To: Tomcat Users List > Subject: Re: [EXT] Datadog _ JMX Integration facing connection issues. > > Hai Chuck., > Thanks for jumping in., > > thats what my concern is. > as previously added by Chris, I have added my annotations in Catalina_opts ., > Which is preferred by Java_Process for adding the annotations., But Here by > using catalina_opts., they are not reflecting. > > where as adding them in java-tool-options., Makes a process with Tomcat and a > process that listens on the mentioned port., but throws port is already > binded., as its an environment variable, its already looking for a port, > where as its already created., throws binding error. > > Yesterday ., I thought of adding both the formats, adding java_tool_opts and > Catalina opts., inorder to utilize the previous thigns., as below > > - name: JAVA_TOOL_OPTIONS > value: {{ .Values.deployment.javaVMMemoryArgument }} > >- name: POD_IP > valueFrom: >fieldRef: > fieldPath: status.podIP >- name: CATALINA_OPTS > value: >- >-Dcom.sun.management.jmxremote >-Dcom.sun.management.jmxremote.port=49151 >-Djava.rmi.server.hostname=$POD_IP >-Dcom.sun.management.jmxremote.authenticate=false >-Dcom.sun.management.jmxremote.ssl=false >-Dcom.sun.management.jmxremote.local.only=false >-Xms2048M -Xmx10240M > > Java_tool_opts are as : -XX:+UseStringDeduplication -XX:+UseContainerSupport > -Dcom.sun.management.jmxremote > > just added to initiate a JMX process., > > but nothing is working out., > I would like to request you for further inputs > > > cc :chris > > Thanks & Regards, > -- > > SAI VAMSI .B > Senior DevOps Engineer > > > From: Chuck Caldarale > Sent: Thursday, December 14, 2023 00:28 > To: Tomcat Users List > Subject: Re: [EXT] Datadog _ JMX Integration facing connection issues. > > >> On Dec 13, 2023, at 10:36, Bodavula, Sai Vamsi Mohan Krishna (TR Technology) >> wrote: >> >> as you just asked ., >> I do have a process with Catalina. >> >> root@lab1workflow4scalsvc2zus1-deployment-659dd79df7-wg59g:/# netstat -tulpn >> Active Internet connections (only servers) >> Proto Recv-Q Send-Q Local Address Foreign Address State >>PID/Program name >> tcp6 0 0 :::34753:::*LISTEN >>1/java >> tcp6 0 0 :::9109 :::*LISTEN >>1/java >> tcp6 0 0 :::10109:::*LISTEN >>1/java >> root@lab1workflow4scalsvc2zus1-deployment-659dd79df7-wg59g:/# ^C >> root@lab1workflow4scalsvc2zus1-deployment-659dd79df7-wg59g:/# ps aux | grep >> catalina >> root 744 0.0 0.0 6460 680 pts/1S+ 11:47 0:00 grep >> --color=auto catalina >> root@lab1workflow4scalsvc2zus1-deployment-659dd79df7-wg59g:/# > > > That shows only the grep process looking for catalina, not anything using > catalina. If Tomcat were actually running, you’d see something like this > (slightly reformatted for clarity): > > chuck@Chuck-MacBookPro apache-tomcat-9.0.83 > ps aux | grep catalina > chuck16879 0.0 0.0 408626896 1376 s000 S+ 12:53PM > 0:00.00 grep catalina > chuck16874 0.0 0.9 415316912 153296 s000 S12:53PM > 0:02.66 > /Library/Java/JavaVirtualMachines/temurin-21.jdk/Contents/Home/bin/java > -Djava.util.logging.config.file=/Users/chuck/Downloads/apache-tomcat-9.0.83/conf/logging.properties > -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager > -Djdk.tls.ephemeralDHKeySize=2048 > -Djava.protocol.handler.pkgs=org.apache.catalina.webresources > -Dorg.apache.catalina.security.SecurityListener.UMASK=0027 > -Dtest_port=9090 > -Dignore.endorsed.dirs= > -classpath > /Users/chuck/Downloads/apache-tomcat-
Re: [EXT] Datadog _ JMX Integration facing connection issues.
by chance any findings on this., Thanks & Regards, -- SAI VAMSI .B Senior DevOps Engineer From: Bodavula, Sai Vamsi Mohan Krishna (TR Technology) Sent: Thursday, December 14, 2023 10:56 To: Tomcat Users List Subject: Re: [EXT] Datadog _ JMX Integration facing connection issues. Hai Chuck., Thanks for jumping in., thats what my concern is. as previously added by Chris, I have added my annotations in Catalina_opts ., Which is preferred by Java_Process for adding the annotations., But Here by using catalina_opts., they are not reflecting. where as adding them in java-tool-options., Makes a process with Tomcat and a process that listens on the mentioned port., but throws port is already binded., as its an environment variable, its already looking for a port, where as its already created., throws binding error. Yesterday ., I thought of adding both the formats, adding java_tool_opts and Catalina opts., inorder to utilize the previous thigns., as below - name: JAVA_TOOL_OPTIONS value: {{ .Values.deployment.javaVMMemoryArgument }} - name: POD_IP valueFrom: fieldRef: fieldPath: status.podIP - name: CATALINA_OPTS value: >- -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=49151 -Djava.rmi.server.hostname=$POD_IP -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.local.only=false -Xms2048M -Xmx10240M Java_tool_opts are as : -XX:+UseStringDeduplication -XX:+UseContainerSupport -Dcom.sun.management.jmxremote just added to initiate a JMX process., but nothing is working out., I would like to request you for further inputs cc :chris Thanks & Regards, -- SAI VAMSI .B Senior DevOps Engineer From: Chuck Caldarale Sent: Thursday, December 14, 2023 00:28 To: Tomcat Users List Subject: Re: [EXT] Datadog _ JMX Integration facing connection issues. > On Dec 13, 2023, at 10:36, Bodavula, Sai Vamsi Mohan Krishna (TR Technology) > wrote: > > as you just asked ., > I do have a process with Catalina. > > root@lab1workflow4scalsvc2zus1-deployment-659dd79df7-wg59g:/# netstat -tulpn > Active Internet connections (only servers) > Proto Recv-Q Send-Q Local Address Foreign Address State > PID/Program name > tcp6 0 0 :::34753:::*LISTEN > 1/java > tcp6 0 0 :::9109 :::*LISTEN > 1/java > tcp6 0 0 :::10109:::*LISTEN > 1/java > root@lab1workflow4scalsvc2zus1-deployment-659dd79df7-wg59g:/# ^C > root@lab1workflow4scalsvc2zus1-deployment-659dd79df7-wg59g:/# ps aux | grep > catalina > root 744 0.0 0.0 6460 680 pts/1S+ 11:47 0:00 grep > --color=auto catalina > root@lab1workflow4scalsvc2zus1-deployment-659dd79df7-wg59g:/# That shows only the grep process looking for catalina, not anything using catalina. If Tomcat were actually running, you’d see something like this (slightly reformatted for clarity): chuck@Chuck-MacBookPro apache-tomcat-9.0.83 > ps aux | grep catalina chuck16879 0.0 0.0 408626896 1376 s000 S+ 12:53PM 0:00.00 grep catalina chuck16874 0.0 0.9 415316912 153296 s000 S12:53PM 0:02.66 /Library/Java/JavaVirtualMachines/temurin-21.jdk/Contents/Home/bin/java -Djava.util.logging.config.file=/Users/chuck/Downloads/apache-tomcat-9.0.83/conf/logging.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Djdk.tls.ephemeralDHKeySize=2048 -Djava.protocol.handler.pkgs=org.apache.catalina.webresources -Dorg.apache.catalina.security.SecurityListener.UMASK=0027 -Dtest_port=9090 -Dignore.endorsed.dirs= -classpath /Users/chuck/Downloads/apache-tomcat-9.0.83/bin/bootstrap.jar:/Users/chuck/Downloads/apache-tomcat-9.0.83/bin/tomcat-juli.jar -Dcatalina.base=/Users/chuck/Downloads/apache-tomcat-9.0.83 -Dcatalina.home=/Users/chuck/Downloads/apache-tomcat-9.0.83 -Djava.io.tmpdir=/Users/chuck/Downloads/apache-tomcat-9.0.83/temp org.apache.catalina.startup.Bootstrap start - Chuck
Re: [EXT] Datadog _ JMX Integration facing connection issues.
Sai Vamsi, On 12/14/23 00:26, Bodavula, Sai Vamsi Mohan Krishna (TR Technology) wrote: Hai Chuck., Thanks for jumping in., thats what my concern is. as previously added by Chris, I have added my annotations in Catalina_opts ., Which is preferred by Java_Process for adding the annotations., But Here by using catalina_opts., they are not reflecting. where as adding them in java-tool-options., Makes a process with Tomcat and a process that listens on the mentioned port., but throws port is already binded., as its an environment variable, its already looking for a port, where as its already created., throws binding error. Yesterday ., I thought of adding both the formats, adding java_tool_opts and Catalina opts., inorder to utilize the previous thigns., as below - name: JAVA_TOOL_OPTIONS value: {{ .Values.deployment.javaVMMemoryArgument }} - name: POD_IP valueFrom: fieldRef: fieldPath: status.podIP - name: CATALINA_OPTS value: >- -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=49151 -Djava.rmi.server.hostname=$POD_IP -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.local.only=false -Xms2048M -Xmx10240M Java_tool_opts are as : -XX:+UseStringDeduplication -XX:+UseContainerSupport -Dcom.sun.management.jmxremote just added to initiate a JMX process., but nothing is working out., I would like to request you for further inputs How are you launching Tomcat itself? Are you using catalina.sh or something similar, or do you use some kind of embedded-launch? It sounds to me like using CATALINA_OPTS is causing Tomcat not to start at all. If you use JAVA_TOOL_OPTIONS (which Tomcat ignores), Tomcat will start but won't initialize JMX in the way you want it to. The solution is to look at the log files that are produced when Tomcat launches (or fails to launch). Are you able to observe those? It would be great to see the exact command Helm (or whatever) uses to attempt to launch Tomcat, and then the stdout/stderr of that attempt. The diagnostic messages shown when a JVM refuses to start are usually pretty good. Same thing with Tomcat, though Tomcat usually starts even if e.g. no connectors are successfully started because there is no actual requirement for Tomcat to listen to any ports. -chris From: Chuck Caldarale Sent: Thursday, December 14, 2023 00:28 To: Tomcat Users List Subject: Re: [EXT] Datadog _ JMX Integration facing connection issues. On Dec 13, 2023, at 10:36, Bodavula, Sai Vamsi Mohan Krishna (TR Technology) wrote: as you just asked ., I do have a process with Catalina. root@lab1workflow4scalsvc2zus1-deployment-659dd79df7-wg59g:/# netstat -tulpn Active Internet connections (only servers) Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name tcp6 0 0 :::34753:::*LISTEN 1/java tcp6 0 0 :::9109 :::*LISTEN 1/java tcp6 0 0 :::10109:::*LISTEN 1/java root@lab1workflow4scalsvc2zus1-deployment-659dd79df7-wg59g:/# ^C root@lab1workflow4scalsvc2zus1-deployment-659dd79df7-wg59g:/# ps aux | grep catalina root 744 0.0 0.0 6460 680 pts/1S+ 11:47 0:00 grep --color=auto catalina root@lab1workflow4scalsvc2zus1-deployment-659dd79df7-wg59g:/# That shows only the grep process looking for catalina, not anything using catalina. If Tomcat were actually running, you’d see something like this (slightly reformatted for clarity): chuck@Chuck-MacBookPro apache-tomcat-9.0.83 > ps aux | grep catalina chuck16879 0.0 0.0 408626896 1376 s000 S+ 12:53PM 0:00.00 grep catalina chuck16874 0.0 0.9 415316912 153296 s000 S12:53PM 0:02.66 /Library/Java/JavaVirtualMachines/temurin-21.jdk/Contents/Home/bin/java -Djava.util.logging.config.file=/Users/chuck/Downloads/apache-tomcat-9.0.83/conf/logging.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Djdk.tls.ephemeralDHKeySize=2048 -Djava.protocol.handler.pkgs=org.apache.catalina.webresources -Dorg.apache.catalina.security.SecurityListener.UMASK=0027 -Dtest_port=9090 -Dignore.endorsed.dirs= -classpath /Users/chuck/Downloads/apache-tomcat-9.0.83/bin/bootstrap.jar:/Users/chuck/Downloads/apache-tomcat-9.0.83/bin/tomcat-juli.jar -Dcatalina.base=/Users/chuck/Downloads/apache-tomcat-9.0.83 -Dcatalina.home=/Users/chuck/Downloads/apache-tomcat-9.0.83 -Djava.io.tmpdir=/Users/chuck/Downloads/apache-tomcat-9.0.83/temp org.apache.catalina.startup.Bootstrap start -
Re: [EXT] Datadog _ JMX Integration facing connection issues.
Sai Vamsi, > Am 13.12.2023 um 19:59 schrieb Chuck Caldarale : > > >> On Dec 13, 2023, at 10:36, Bodavula, Sai Vamsi Mohan Krishna (TR Technology) >> wrote: >> >> as you just asked ., >> I do have a process with Catalina. >> >> root@lab1workflow4scalsvc2zus1-deployment-659dd79df7-wg59g:/# netstat -tulpn >> Active Internet connections (only servers) >> Proto Recv-Q Send-Q Local Address Foreign Address State >>PID/Program name >> tcp6 0 0 :::34753:::*LISTEN >>1/java >> tcp6 0 0 :::9109 :::*LISTEN >>1/java >> tcp6 0 0 :::10109:::*LISTEN >>1/java >> root@lab1workflow4scalsvc2zus1-deployment-659dd79df7-wg59g:/# ^C >> root@lab1workflow4scalsvc2zus1-deployment-659dd79df7-wg59g:/# ps aux | grep >> catalina >> root 744 0.0 0.0 6460 680 pts/1S+ 11:47 0:00 grep >> --color=auto catalina >> root@lab1workflow4scalsvc2zus1-deployment-659dd79df7-wg59g:/# > > you have to figure out WHY tomcat is not starting! There should be log files or error messages on the console. It seems you have put an error somewhere in any of the configfiles. It's not at all a question of the ports not being allocated. Take a step back and make tomcat launch again. After that we figure out where you have to set the options... Please detail how you start tomcat and show the output of startup (the beginning and last lines should be enough). Again, don't put any java options for tomcat in any global environment options (JAVA_OPTS, CATALINA_OPTS) in your shell. Only in setenv.sh . Peter > That shows only the grep process looking for catalina, not anything using > catalina. If Tomcat were actually running, you’d see something like this > (slightly reformatted for clarity): > > chuck@Chuck-MacBookPro apache-tomcat-9.0.83 > ps aux | grep catalina > chuck16879 0.0 0.0 408626896 1376 s000 S+ 12:53PM > 0:00.00 grep catalina > chuck16874 0.0 0.9 415316912 153296 s000 S12:53PM > 0:02.66 > /Library/Java/JavaVirtualMachines/temurin-21.jdk/Contents/Home/bin/java > -Djava.util.logging.config.file=/Users/chuck/Downloads/apache-tomcat-9.0.83/conf/logging.properties > -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager > -Djdk.tls.ephemeralDHKeySize=2048 > -Djava.protocol.handler.pkgs=org.apache.catalina.webresources > -Dorg.apache.catalina.security.SecurityListener.UMASK=0027 > -Dtest_port=9090 > -Dignore.endorsed.dirs= > -classpath > /Users/chuck/Downloads/apache-tomcat-9.0.83/bin/bootstrap.jar:/Users/chuck/Downloads/apache-tomcat-9.0.83/bin/tomcat-juli.jar > -Dcatalina.base=/Users/chuck/Downloads/apache-tomcat-9.0.83 > -Dcatalina.home=/Users/chuck/Downloads/apache-tomcat-9.0.83 > -Djava.io.tmpdir=/Users/chuck/Downloads/apache-tomcat-9.0.83/temp > org.apache.catalina.startup.Bootstrap > start > > > - Chuck > - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: [EXT] Datadog _ JMX Integration facing connection issues.
Hai Chuck., Thanks for jumping in., thats what my concern is. as previously added by Chris, I have added my annotations in Catalina_opts ., Which is preferred by Java_Process for adding the annotations., But Here by using catalina_opts., they are not reflecting. where as adding them in java-tool-options., Makes a process with Tomcat and a process that listens on the mentioned port., but throws port is already binded., as its an environment variable, its already looking for a port, where as its already created., throws binding error. Yesterday ., I thought of adding both the formats, adding java_tool_opts and Catalina opts., inorder to utilize the previous thigns., as below - name: JAVA_TOOL_OPTIONS value: {{ .Values.deployment.javaVMMemoryArgument }} - name: POD_IP valueFrom: fieldRef: fieldPath: status.podIP - name: CATALINA_OPTS value: >- -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=49151 -Djava.rmi.server.hostname=$POD_IP -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.local.only=false -Xms2048M -Xmx10240M Java_tool_opts are as : -XX:+UseStringDeduplication -XX:+UseContainerSupport -Dcom.sun.management.jmxremote just added to initiate a JMX process., but nothing is working out., I would like to request you for further inputs cc :chris Thanks & Regards, -- SAI VAMSI .B Senior DevOps Engineer From: Chuck Caldarale Sent: Thursday, December 14, 2023 00:28 To: Tomcat Users List Subject: Re: [EXT] Datadog _ JMX Integration facing connection issues. > On Dec 13, 2023, at 10:36, Bodavula, Sai Vamsi Mohan Krishna (TR Technology) > wrote: > > as you just asked ., > I do have a process with Catalina. > > root@lab1workflow4scalsvc2zus1-deployment-659dd79df7-wg59g:/# netstat -tulpn > Active Internet connections (only servers) > Proto Recv-Q Send-Q Local Address Foreign Address State > PID/Program name > tcp6 0 0 :::34753:::*LISTEN > 1/java > tcp6 0 0 :::9109 :::*LISTEN > 1/java > tcp6 0 0 :::10109:::*LISTEN > 1/java > root@lab1workflow4scalsvc2zus1-deployment-659dd79df7-wg59g:/# ^C > root@lab1workflow4scalsvc2zus1-deployment-659dd79df7-wg59g:/# ps aux | grep > catalina > root 744 0.0 0.0 6460 680 pts/1S+ 11:47 0:00 grep > --color=auto catalina > root@lab1workflow4scalsvc2zus1-deployment-659dd79df7-wg59g:/# That shows only the grep process looking for catalina, not anything using catalina. If Tomcat were actually running, you’d see something like this (slightly reformatted for clarity): chuck@Chuck-MacBookPro apache-tomcat-9.0.83 > ps aux | grep catalina chuck16879 0.0 0.0 408626896 1376 s000 S+ 12:53PM 0:00.00 grep catalina chuck16874 0.0 0.9 415316912 153296 s000 S12:53PM 0:02.66 /Library/Java/JavaVirtualMachines/temurin-21.jdk/Contents/Home/bin/java -Djava.util.logging.config.file=/Users/chuck/Downloads/apache-tomcat-9.0.83/conf/logging.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Djdk.tls.ephemeralDHKeySize=2048 -Djava.protocol.handler.pkgs=org.apache.catalina.webresources -Dorg.apache.catalina.security.SecurityListener.UMASK=0027 -Dtest_port=9090 -Dignore.endorsed.dirs= -classpath /Users/chuck/Downloads/apache-tomcat-9.0.83/bin/bootstrap.jar:/Users/chuck/Downloads/apache-tomcat-9.0.83/bin/tomcat-juli.jar -Dcatalina.base=/Users/chuck/Downloads/apache-tomcat-9.0.83 -Dcatalina.home=/Users/chuck/Downloads/apache-tomcat-9.0.83 -Djava.io.tmpdir=/Users/chuck/Downloads/apache-tomcat-9.0.83/temp org.apache.catalina.startup.Bootstrap start - Chuck
Re: [EXT] Datadog _ JMX Integration facing connection issues.
> On Dec 13, 2023, at 10:36, Bodavula, Sai Vamsi Mohan Krishna (TR Technology) > wrote: > > as you just asked ., > I do have a process with Catalina. > > root@lab1workflow4scalsvc2zus1-deployment-659dd79df7-wg59g:/# netstat -tulpn > Active Internet connections (only servers) > Proto Recv-Q Send-Q Local Address Foreign Address State > PID/Program name > tcp6 0 0 :::34753:::*LISTEN > 1/java > tcp6 0 0 :::9109 :::*LISTEN > 1/java > tcp6 0 0 :::10109:::*LISTEN > 1/java > root@lab1workflow4scalsvc2zus1-deployment-659dd79df7-wg59g:/# ^C > root@lab1workflow4scalsvc2zus1-deployment-659dd79df7-wg59g:/# ps aux | grep > catalina > root 744 0.0 0.0 6460 680 pts/1S+ 11:47 0:00 grep > --color=auto catalina > root@lab1workflow4scalsvc2zus1-deployment-659dd79df7-wg59g:/# That shows only the grep process looking for catalina, not anything using catalina. If Tomcat were actually running, you’d see something like this (slightly reformatted for clarity): chuck@Chuck-MacBookPro apache-tomcat-9.0.83 > ps aux | grep catalina chuck16879 0.0 0.0 408626896 1376 s000 S+ 12:53PM 0:00.00 grep catalina chuck16874 0.0 0.9 415316912 153296 s000 S12:53PM 0:02.66 /Library/Java/JavaVirtualMachines/temurin-21.jdk/Contents/Home/bin/java -Djava.util.logging.config.file=/Users/chuck/Downloads/apache-tomcat-9.0.83/conf/logging.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Djdk.tls.ephemeralDHKeySize=2048 -Djava.protocol.handler.pkgs=org.apache.catalina.webresources -Dorg.apache.catalina.security.SecurityListener.UMASK=0027 -Dtest_port=9090 -Dignore.endorsed.dirs= -classpath /Users/chuck/Downloads/apache-tomcat-9.0.83/bin/bootstrap.jar:/Users/chuck/Downloads/apache-tomcat-9.0.83/bin/tomcat-juli.jar -Dcatalina.base=/Users/chuck/Downloads/apache-tomcat-9.0.83 -Dcatalina.home=/Users/chuck/Downloads/apache-tomcat-9.0.83 -Djava.io.tmpdir=/Users/chuck/Downloads/apache-tomcat-9.0.83/temp org.apache.catalina.startup.Bootstrap start - Chuck
Re: [EXT] Datadog _ JMX Integration facing connection issues.
Sai Vamsi, > Am 08.12.2023 um 06:43 schrieb Bodavula, Sai Vamsi Mohan Krishna (TR > Technology) : > > Hey Christopher., > Greetings of the day. > > > 1. > Might I have confused you with posting the arguments directly ., Yeah as i > just shared you the annotations with comments , to state you the stuff i am > using., But in my deployment ., I am using them in catalina opts., and trying > to call them from values.yaml., which looks like this : > > javaVMMemoryArgument: "-Xms2048M -Xmx10240M -XX:+UseStringDeduplication > -XX:+UseContainerSupport -Dcom.sun.management.jmxremote > -Dcom.sun.management.jmxremote.authenticate=false > -Dcom.sun.management.jmxremote.ssl=false > -Dcom.sun.management.jmxremote.local.only=false > -Dcom.sun.management.jmxremote.port=49151 > -Djava.rmi.server.hostname=lab1workflow4scalsvc2zus1-service.hqm-lab1.svc.cluster.local > " > > and I am referring the word "javaVMMemoryArgument" from values yaml and > calling it in Catalina_opts, so that it would fetch all these > annotations as mentioned above, during the deployment. This is my deployment > part., where I am referring to the above values from values.yaml > env: >- name: CATALINA_OPTS > value: {{ .Values.deployment.javaVMMemoryArgument }} > > I compare with my CATALINA_OPTS (and I noticed that some of my options were still in JAVA_OPTS boooh!) : export CATALINA_OPTS="${CATALINA_OPTS} -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=10001 -Dcom.sun.management.jmxremote.rmi.port=10002 -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Djava.rmi.server.hostname=${HOSTNAME} -Dcom.sun.management.jmxremote.local.only=false" I see all my options in root@tomcat:/usr/local/tomcat# ps -aef | less UID PIDPPID C STIME TTY TIME CMD tomcat 1 0 71 10:42 ?00:04:02 /opt/java/openjdk/bin/java -Djava.util.logging.config.file=/opt/apache-tomcat.base/conf/logging.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Djdk.tls.ephemeralDHKeySize=2048 -Djava.protocol.handler.pkgs=org.apache.catalina.webresources -Dorg.apache.catalina.security.SecurityListener.UMASK=0027 -agentlib:jdwp=transport=dt_socket,address=0.0.0.0:8000,server=y,suspend=n -XX:NativeMemoryTracking=summary -Dhostname= -Djava.awt.headless=true -Djavax.net.ssl.trustStore=/opt/apache-tomcat.base/conf/ssl/cacerts.jks -Xlog:gc:/opt/apache-tomcat.base/logs/gc.log -Djava.security.egd=file:/dev/urandom -Dsun.net.inetaddr.ttl=60 -Djava.library.path=/usr/local/tomcat/native-jni-lib -Djdk.tls.ephemeralDHKeySize=2048 -Djdk.tls.rejectClientInitiatedRenegotiation=true -Djdk.tls.server.enableStatusRequestExtension=true -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=10001 -Dcom.sun.management.jmxremote.rmi.port=10002 -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Djava.rmi.server.hostname= -Dcom.sun.management.jmxremote.local.only=false -javaagent:/opt/apache-tomcat.base/bin/jmx_prometheus_javaagent-0.12.0.jar=8080:/opt/apache-tomcat.base/bin/tomcat.yaml -Djava.net.debug=ssl -XX:+UnlockDiagnosticVMOptions -Dignore.endorsed.dirs= -classpath /usr/local/tomcat/bin/bootstrap.jar:/usr/local/tomcat/bin/tomcat-juli.jar -Dcatalina.base=/opt/apache-tomcat.base -Dcatalina.home=/usr/local/tomcat -Djava.io.tmpdir=/opt/apache-tomcat.base/temp org.apache.catalina.startup.Bootstrap start Ports are opened root@tomcat:/usr/local/tomcat# netstat -tulpn Active Internet connections (servers and established) Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name ... tcp0 0 0.0.0.0:10002 0.0.0.0:* LISTEN - tcp0 0 0.0.0.0:10001 0.0.0.0:* LISTEN - ... tcp2 0 0.0.0.0:84430.0.0.0:* LISTEN - ... jconsole :10001 > JMX connect success > > > > 1. > Coming to Process., I have searched for Java process that listens on my > mentioned port ie., 49151, but none of the process is listening to that > process. > > I even tried with > root@lab1workflow4scalsvc2zus1-deployment-fd64ff775-cwzn6:/# netstat -tulpn | > grep LISTEN > tcp6 0 0 :::10109:::*LISTEN > 1/java > tcp6 0 0 :::9109 :::*LISTEN > 1/java > root@lab1workflow4scalsvc2zus1-deployment-fd64ff775-cwzn6:/# netstat -tulpn | > more > Active Internet connections (only servers) > Proto Recv-Q Send-Q Local Address Foreign Address State > PID/Program name > tcp6 0 0 :::10109