Hi everyone,
I am trying connect JMX exporter https://github.com/prometheus/jmx_exporter
httpserver to my Kafka connect docker instance
https://hub.docker.com/r/confluentinc/cp-kafka-connect. SSL with
authentication must be enabled on JMX port due to company policies.
All JMX kafka connect options:
KAFKA_JMX_PORT=5555
KAFKA_JMX_OPTS=-Dcom.sun.management.jmxremote=true
-Dcom.sun.management.jmxremote.registry.ssl=true
-Dcom.sun.management.jmxremote.authenticate=true
-Dcom.sun.management.jmxremote.ssl=true
-Djava.net.preferIPv4Stack=true
-Djava.rmi.server.hostname=kafka-connect-01.triviadata.local
-Dcom.sun.management.jmxremote.password.file=/tmp/kafka_connect/jmxremote.password
-Dcom.sun.management.jmxremote.access.file=/tmp/kafka_connect/jmxremote.access
-Djavax.net.ssl.trustStore=/root/pki/jssecacerts
-Djavax.net.ssl.trustStoreType=jks
-Djavax.net.ssl.trustStorePassword=trust_pass
-Djavax.net.ssl.keyStore=/root/pki/host.jks
-Djavax.net.ssl.keyStoreType=jks
-Djavax.net.ssl.keyStorePassword=key_pass
Execution of exporter httpserver:
java -Dcom.sun.management.jmxremote.ssl=false
-Dcom.sun.management.jmxremote.authenticate=false
-Djavax.net.ssl.trustStore=/root/pki/jssecacerts
-Djavax.net.ssl.trustStoreType=jks
-Djavax.net.ssl.trustStorePassword=trust_pass
-Djavax.net.ssl.keyStore=/root/pki/host.jks
-Djavax.net.ssl.keyStoreType=jks
-Djavax.net.ssl.keyStorePassword=key_pass
-jar
/opt/jmx_exporter/jmx_prometheus_httpserver-0.16.1-jar-with-dependencies.jar
kafka-connect-01.triviadata.local:9555
/opt/jmx_exporter/jmx_httpserver_config.yml
Content of jmx_httpserver_config.yml:
---
username: monitorRole
password: monitor_pass
jmxUrl:
service:jmx:rmi:///jndi/rmi://kafka-connect-01.triviadata.local:5555/jmxrmi
rules:
- pattern: ".*"
Error message:
Aug 24, 2021 1:29:29 PM io.prometheus.jmx.JmxCollector collect
SEVERE: JMX scrape failed: java.io.IOException: Failed to retrieve
RMIServer stub: javax.naming.CommunicationException [Root exception is
java.rmi.ConnectIOException: non-JRMP server at remote endpoint]
at
javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:369)
at
javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:270)
at io.prometheus.jmx.JmxScraper.doScrape(JmxScraper.java:95)
at io.prometheus.jmx.JmxCollector.collect(JmxCollector.java:547)
at
io.prometheus.client.CollectorRegistry$MetricFamilySamplesEnumeration.findNextElement(CollectorRegistry.java:207)
at
io.prometheus.client.CollectorRegistry$MetricFamilySamplesEnumeration.nextElement(CollectorRegistry.java:240)
at
io.prometheus.client.CollectorRegistry$MetricFamilySamplesEnumeration.nextElement(CollectorRegistry.java:161)
at
io.prometheus.client.exporter.common.TextFormat.writeOpenMetrics100(TextFormat.java:202)
at
io.prometheus.client.exporter.common.TextFormat.writeFormat(TextFormat.java:57)
at
io.prometheus.client.exporter.HTTPServer$HTTPMetricHandler.handle(HTTPServer.java:83)
at com.sun.net.httpserver.Filter$Chain.doFilter(Filter.java:79)
at sun.net.httpserver.AuthFilter.doFilter(AuthFilter.java:83)
at com.sun.net.httpserver.Filter$Chain.doFilter(Filter.java:82)
at
sun.net.httpserver.ServerImpl$Exchange$LinkHandler.handle(ServerImpl.java:675)
at com.sun.net.httpserver.Filter$Chain.doFilter(Filter.java:79)
at sun.net.httpserver.ServerImpl$Exchange.run(ServerImpl.java:647)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: javax.naming.CommunicationException [Root exception is
java.rmi.ConnectIOException: non-JRMP server at remote endpoint]
at
com.sun.jndi.rmi.registry.RegistryContext.lookup(RegistryContext.java:136)
at
com.sun.jndi.toolkit.url.GenericURLContext.lookup(GenericURLContext.java:205)
at javax.naming.InitialContext.lookup(InitialContext.java:417)
at
javax.management.remote.rmi.RMIConnector.findRMIServerJNDI(RMIConnector.java:1955)
at
javax.management.remote.rmi.RMIConnector.findRMIServer(RMIConnector.java:1922)
at
javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:287)
... 18 more
Caused by: java.rmi.ConnectIOException: non-JRMP server at remote endpoint
at
sun.rmi.transport.tcp.TCPChannel.createConnection(TCPChannel.java:248)
at
sun.rmi.transport.tcp.TCPChannel.newConnection(TCPChannel.java:202)
at sun.rmi.server.UnicastRef.newCall(UnicastRef.java:343)
at
sun.rmi.registry.RegistryImpl_Stub.lookup(RegistryImpl_Stub.java:116)
at
com.sun.jndi.rmi.registry.RegistryContext.lookup(RegistryContext.java:132)
... 23 more
When I disable SSL for kafka connect JMX port by setting
-Dcom.sun.management.jmxremote.registry.ssl=false and
-Dcom.sun.management.jmxremote.ssl=false, JMX exporter httpserver is
running and I am able to get metrics from
kafka-connect-01.triviadata.local:9555/metrics
Can someone help me how to connect JMX exporter httpserver to SSL secured
JMX?
--
You received this message because you are subscribed to the Google Groups
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/prometheus-users/0e488973-b4da-45a2-8d8e-dffdd468bbebn%40googlegroups.com.
