On Thu, Dec 12, 2019 at 10:05 AM Christopher Schultz <ch...@christopherschultz.net> wrote: > > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA256 > > Chris, > > On 12/11/19 15:52, Chris Cheshire wrote: > > On Wed, Dec 11, 2019 at 12:24 PM Christopher Schultz > > <ch...@christopherschultz.net> wrote: > >> > >> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 > >> > >> > >> > >> On 12/10/19 12:59, Chris Cheshire wrote: > >>> On Tue, Dec 10, 2019 at 11:58 AM Chris Cheshire > >>> <yahoono...@gmail.com> wrote: > >>>> > >>>> On Tue, Dec 10, 2019 at 9:42 AM Christopher Schultz > >>>> <ch...@christopherschultz.net> wrote: > >>>>> > >>>>> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 > >>>>> > >>>>> Chris, > >>>>> > >>>>> On 12/9/19 17:10, Chris Cheshire wrote: > >>>>>> In CATALINA_BASE/bin/setenv.sh I have the following : > >>>>>> > >>>>>> CATALINA_OPTS="-Dcom.sun.management.jmxremote > >>>>>> -Dcom.sun.management.jmxremote.ssl=false > >>>>>> -Dcom.sun.management.jmxremote.authenticate=false" > >>>>> > >>>>> Okay. > >>>>> > >>>>>> In CATALINA_BASE/conf/server.xml I have a listener > >>>>>> configured : > >>>>>> > >>>>>> <Listener > >>>>>> className="org.apache.catalina.mbeans.JmxRemoteLifecycleListener" > >>>>>> > >>>>>> > >> > >>>>>> > rmiRegistryPortPlatform="10001" rmiServerPortPlatform="10002" > >>>>>> useLocalPorts="true" /> > >>>>>> > >>>>>> > >>>>>> Upon startup I see in logs : INFO [main] > >>>>>> org.apache.catalina.mbeans.JmxRemoteLifecycleListener.createServe > r > >>>>>> > >>>>>> > >> > >>>>>> > The JMX Remote Listener has configured the registry on port > >>>>>> [10001] and the server on port [10002] for the > >>>>>> [Platform] server > >>>>>> > >>>>>> > >>>>>> $ netstat -an | grep 10001 tcp4 0 0 > >>>>>> 127.0.0.1.10001 *.* LISTEN tcp6 > >>>>>> 0 0 ::1.10001 *.* LISTEN > >>>>>> > >>>>>> On my local machine I have a tunnel set up as follows : > >>>>>> ssh -N -L10001:localhost:10001 -L10002:localhost:10002 > >>>>>> user@remotehost > >>>>>> > >>>>>> (where user is the user tomcat is running under) > >>>>>> > >>>>>> When I try to add a remote JMX connection in VisualVM on > >>>>>> my client machine to localhost:10001 I get an error > >>>>>> dialog after a brief delay with the message "Cannot > >>>>>> connect to localhost:10001 using > >>>>>> service:jmx:rmi:///jndi/rmi://localhost:10001/jmxrmi". If > >>>>>> I change it to port 10002 I get the same error. On the > >>>>>> server at this time : $ netstat -an | grep 10001 tcp4 > >>>>>> 0 0 127.0.0.1.10001 *.* LISTEN > >>>>>> tcp6 0 0 ::1.10001 *.* LISTEN > >>>>>> tcp4 0 0 127.0.0.1.62637 127.0.0.1.10001 > >>>>>> TIME_WAIT > >>>>>> > >>>>>> > >>>>>> If I try to use jconsole connecting to port 10001 I get > >>>>>> the error "Connection failed: non-JRMP server at remote > >>>>>> endpoint". Connecting to port 10002 I get the error > >>>>>> "Connection failed: no such object in table" > >>>>> > >>>>> You should be using the port defined by > >>>>> rmiRegistryPortPlatform, so 10001 is the correct port to > >>>>> use. > >>>>> > >>>>>> I've been through the tomcat configuration documentation > >>>>>> a couple times but I can't see what else I need to > >>>>>> configure. > >>>>> > >>>>> What you have looks good to me without reproducing it > >>>>> myself. Can you do : > >>>>> > >>>>> $ netstat -an | grep 1000[0-9] > >>>>> > >>>>> ? > >>>>> > >>>>> Just to be sure about both ports? > >>>>> > >>>> > >>>> $ netstat -an | grep 1000[0-9] tcp6 0 0 :::10001 > >>>> :::* LISTEN tcp6 0 0 :::10002 > >>>> :::* LISTEN > >>>> > >>>> > >>>> Hmmmm. Tomcat is only listening on ipv6 ports, but my tunnel > >>>> is using ipv4. After digging around [1], I added this to > >>>> CATALINA_OPTS in setenv.sh > >>>> > >>>> -Djava.net.preferIPv4Stack=true > >>>> -Djava.net.preferIPv4Addresses=true > >>>> > >>>> $ netstat -an | grep 1000[0-9] tcp 0 0 > >>>> 0.0.0.0:10001 0.0.0.0:* LISTEN tcp 0 > >>>> 0 0.0.0.0:10002 0.0.0.0:* LISTEN > >>>> > >>>> When I try to connect with jconsole I get the same error > >>>> (non-JRMP server at remote endpoint), with the server > >>>> showing > >>>> > >>>> tcp 0 0 0.0.0.0:10001 0.0.0.0:* LISTEN > >>>> tcp 0 0 0.0.0.0:10002 0.0.0.0:* LISTEN > >>>> tcp 0 0 127.0.0.1:10001 127.0.0.1:43803 > >>>> TIME_WAIT tcp 0 0 127.0.0.1:10001 > >>>> 127.0.0.1:43815 TIME_WAIT > >>>> > >>>> > >>>> I have also updated sshd_config with > >>>> > >>>> PermitTunnel yes > >>>> > >>>> and restarted that. Still no change. > >>>> > >>>> Chris > >>>> > >>>> > >>>> [1] > >>>> https://serverfault.com/questions/390840/how-does-one-get-tomcat-to > - -b > >> > >>>> > ind-to-ipv4-address > >>> > >>>> > >>> > >>> As a followup to take the tunnel out of the equation I > >>> downloaded jmxterm [1] on the server and tried to connect > >>> > >>> > >>> $ java -jar jmxterm-1.0.0-uber.jar Welcome to JMX terminal. > >>> Type "help" for available commands. $>open localhost:10001 > >>> #RuntimeIOException: Runtime IO exception: Failed to retrieve > >>> RMIServer stub: javax.naming.CommunicationException [Root > >>> exception is java.rmi.ConnectIOException: non-JRMP server at > >>> remote endpoint] $> > >>> > >>> > >>> Back to the tomcat documentation, I added this to > >>> CATALINA_OPTS (based on listener config and assumed defaults) > >>> > >>> -Dcom.sun.management.jmxremote.registry.ssl=false > >>> > >>> and now I get a different error : $>open localhost:10001 > >>> #RuntimeIOException: Runtime IO exception: Failed to retrieve > >>> RMIServer stub: javax.naming.CommunicationException [Root > >>> exception is java.rmi.UnmarshalException: error unmarshalling > >>> return; nested exception is: java.lang.ClassNotFoundException: > >>> org/apache/catalina/mbeans/JmxRemoteLifecycleListener$RmiClientLocal > ho > >> > >>> > stSocketFactory > >>> > >>> > >> (no security manager: RMI class loader disabled)] > >>> > >>> > >>> So I enabled the security manager by adding to CATALINA_OPTS > >>> > >>> -Djava.security.manager > >>> -Djava.security.policy=$CATALINA_BASE/conf/catalina.policy > >>> > >>> And got a reminder why I turned it off in the first place. Now > >>> I have to figure out how to allow the mysql drivers to work > >>> (and probably everything else about the web app) so tomcat will > >>> start :/ > >>> > >>> Uggh. > >>> > >>> Chris > >> > >> There's always the JMXProxyServlet. > >> > >> JMX is such an ugly protocol. Why not use HTTP(S) which is much > >> easier to configure and connect to? It also means you don't need > >> a Java client :) > >> > >> - -chris > > > > I went this route because I thought it would be the quickest way > > to start poking around within the exposed mbeans without writing > > code to query them myself. > > > > So if tomcat is not jconsole/visualvm compatible, how do I access > > the exposed JMX mbeans? > > Oh, Tomcat most definitely is jconsole/visualvm compatible. I can > connect without any problems on any local environment. I've never > bothered to set it up remotely, because frankly Java clients are too > wasteful IMO to deploy. I use Perl and/or Python-based clients which > query the JMXProxyServlet. > > Have a look at > http://tomcat.apache.org/presentations.html#latest-monitoring-with-jmx > to see how you cann use the JMXProxyServlet with ... any client you'd > like. There are examples using curl in that presentation. > > You can also have a look at: > https://github.com/ChristopherSchultz/check-jmxproxy > or: > https://github.com/ChristopherSchultz/apache-tomcat-stuff/tree/master/bi > n/nagios > > (I have forgotten which of those is more up-to-date... looks like the > latest commit was on the latter.) > > - -chris
Thanks Chris, I'll look into this later. I definitely don't want to run jconsole/visualvm on the server because of the memory/cpu pressure it would add, which was why I was looking to run it on my end over a tunnel. I did take a look at your slides initially but wanted a quick entry into browsing through the mbeans rather than writing some scripts to do it. That said, your approach is far more manageable for ongoing maintenance/monitoring so I'll take another look at this soon. Chris --------------------------------------------------------------------- To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
