Hi Ricardo,
Thanks for your kind response.
As per your suggestion, I have enabled trace and PFB the content of the log
file.
Log File Content:
[2020-07-08 18:48:08,963] INFO Registered kafka:type=kafka.Log4jController
MBean (kafka.utils.Log4jControllerRegistration$)
[2020-07-08 18:48:09,244] TRACE Registered metric named MetricName [name=count,
group=kafka-metrics-count, description=total number of registered metrics,
tags={}] (org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,289] INFO Successfully logged in.
(org.apache.kafka.common.security.authenticator.AbstractLogin)
[2020-07-08 18:48:09,290] TRACE LoginManager(serviceName=kafka,
publicCredentials=[admin], refCount=1) acquired
(org.apache.kafka.common.security.authenticator.LoginManager)
[2020-07-08 18:48:09,292] DEBUG Updated cluster metadata version 1 to
Cluster(id = null, nodes = [broker1:2345 (id: -1 rack: null), broker2:2345 (id:
-2 rack: null), broker3:2345 (id: -3 rack: null)], partitions = [], controller
= null) (org.apache.kafka.clients.Metadata)
[2020-07-08 18:48:09,303] DEBUG Added sensor with name connections-closed:
(org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,305] TRACE Registered metric named MetricName
[name=connection-close-total, group=admin-metrics, description=The total number
of connections closed, tags={}] (org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,306] TRACE Registered metric named MetricName
[name=connection-close-rate, group=admin-metrics, description=The number of
connections closed per second, tags={}]
(org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,306] DEBUG Added sensor with name connections-created:
(org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,306] TRACE Registered metric named MetricName
[name=connection-creation-total, group=admin-metrics, description=The total
number of new connections established, tags={}]
(org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,306] TRACE Registered metric named MetricName
[name=connection-creation-rate, group=admin-metrics, description=The number of
new connections established per second, tags={}]
(org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,306] DEBUG Added sensor with name
successful-authentication: (org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,306] TRACE Registered metric named MetricName
[name=successful-authentication-total, group=admin-metrics, description=The
total number of connections with successful authentication, tags={}]
(org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,306] TRACE Registered metric named MetricName
[name=successful-authentication-rate, group=admin-metrics, description=The
number of connections with successful authentication per second, tags={}]
(org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,306] DEBUG Added sensor with name failed-authentication:
(org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,306] TRACE Registered metric named MetricName
[name=failed-authentication-total, group=admin-metrics, description=The total
number of connections with failed authentication, tags={}]
(org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,307] TRACE Registered metric named MetricName
[name=failed-authentication-rate, group=admin-metrics, description=The number
of connections with failed authentication per second, tags={}]
(org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,307] DEBUG Added sensor with name bytes-sent-received:
(org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,307] TRACE Registered metric named MetricName
[name=network-io-total, group=admin-metrics, description=The total number of
network operations (reads or writes) on all connections, tags={}]
(org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,307] TRACE Registered metric named MetricName
[name=network-io-rate, group=admin-metrics, description=The number of network
operations (reads or writes) on all connections per second, tags={}]
(org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,307] DEBUG Added sensor with name bytes-sent:
(org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,307] TRACE Registered metric named MetricName
[name=outgoing-byte-total, group=admin-metrics, description=The total number of
outgoing bytes sent to all servers, tags={}]
(org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,308] TRACE Registered metric named MetricName
[name=outgoing-byte-rate, group=admin-metrics, description=The number of
outgoing bytes sent to all servers per second, tags={}]
(org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,308] TRACE Registered metric named MetricName
[name=request-total, group=admin-metrics, description=The total number of
requests sent, tags={}] (org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,308] TRACE Registered metric named MetricName
[name=request-rate, group=admin-metrics, description=The number of requests
sent per second, tags={}] (org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,308] TRACE Registered metric named MetricName
[name=request-size-avg, group=admin-metrics, description=The average size of
requests sent., tags={}] (org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,309] TRACE Registered metric named MetricName
[name=request-size-max, group=admin-metrics, description=The maximum size of
any request sent., tags={}] (org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,309] DEBUG Added sensor with name bytes-received:
(org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,309] TRACE Registered metric named MetricName
[name=incoming-byte-total, group=admin-metrics, description=The total number of
bytes read off all sockets, tags={}] (org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,309] TRACE Registered metric named MetricName
[name=incoming-byte-rate, group=admin-metrics, description=The number of bytes
read off all sockets per second, tags={}]
(org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,309] TRACE Registered metric named MetricName
[name=response-total, group=admin-metrics, description=The total number of
responses received, tags={}] (org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,309] TRACE Registered metric named MetricName
[name=response-rate, group=admin-metrics, description=The number of responses
received per second, tags={}] (org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,309] DEBUG Added sensor with name select-time:
(org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,309] TRACE Registered metric named MetricName
[name=select-total, group=admin-metrics, description=The total number of times
the I/O layer checked for new I/O to perform, tags={}]
(org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,311] TRACE Registered metric named MetricName
[name=select-rate, group=admin-metrics, description=The number of times the I/O
layer checked for new I/O to perform per second, tags={}]
(org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,311] TRACE Registered metric named MetricName
[name=io-wait-time-ns-avg, group=admin-metrics, description=The average length
of time the I/O thread spent waiting for a socket ready for reads or writes in
nanoseconds., tags={}] (org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,311] TRACE Registered metric named MetricName
[name=io-waittime-total, group=admin-metrics, description=The total time the
I/O thread spent waiting, tags={}] (org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,311] TRACE Registered metric named MetricName
[name=io-wait-ratio, group=admin-metrics, description=The fraction of time the
I/O thread spent waiting, tags={}] (org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,311] DEBUG Added sensor with name io-time:
(org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,311] TRACE Registered metric named MetricName
[name=io-time-ns-avg, group=admin-metrics, description=The average length of
time for I/O per select call in nanoseconds., tags={}]
(org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,311] TRACE Registered metric named MetricName
[name=iotime-total, group=admin-metrics, description=The total time the I/O
thread spent doing I/O, tags={}] (org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,311] TRACE Registered metric named MetricName
[name=io-ratio, group=admin-metrics, description=The fraction of time the I/O
thread spent doing I/O, tags={}] (org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,312] TRACE Registered metric named MetricName
[name=connection-count, group=admin-metrics, description=The current number of
active connections., tags={}] (org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,340] TRACE [NetworkClient clientId=admin-1] Removing node
broker2:2345 (id: -2 rack: null) from least loaded node selection:
is-blacked-out: false, in-flight-requests: 0
(org.apache.kafka.clients.NetworkClient)
[2020-07-08 18:48:09,340] TRACE [NetworkClient clientId=admin-1] Removing node
broker3:2345 (id: -3 rack: null) from least loaded node selection:
is-blacked-out: false, in-flight-requests: 0
(org.apache.kafka.clients.NetworkClient)
[2020-07-08 18:48:09,340] TRACE [NetworkClient clientId=admin-1] Found least
loaded node broker1:2345 (id: -1 rack: null)
(org.apache.kafka.clients.NetworkClient)
[2020-07-08 18:48:09,340] DEBUG [NetworkClient clientId=admin-1] Initialize
connection to node broker1:2345 (id: -1 rack: null) for sending metadata
request (org.apache.kafka.clients.NetworkClient)
[2020-07-08 18:48:09,340] DEBUG [NetworkClient clientId=admin-1] Initiating
connection to node broker1:2345 (id: -1 rack: null)
(org.apache.kafka.clients.NetworkClient)
[2020-07-08 18:48:09,351] DEBUG Set SASL client state to
SEND_APIVERSIONS_REQUEST
(org.apache.kafka.common.security.authenticator.SaslClientAuthenticator)
[2020-07-08 18:48:09,353] DEBUG Creating SaslClient:
client=null;service=kafka;serviceHostname=broker1;mechs=[SCRAM-SHA-512]
(org.apache.kafka.common.security.authenticator.SaslClientAuthenticator)
[2020-07-08 18:48:09,375] DEBUG Setting SASL/SCRAM_SHA_512 client state to
SEND_CLIENT_FIRST_MESSAGE
(org.apache.kafka.common.security.scram.internals.ScramSaslClient)
[2020-07-08 18:48:09,378] DEBUG Added sensor with name node--1.bytes-sent
(org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,379] TRACE Registered metric named MetricName
[name=outgoing-byte-total, group=admin-node-metrics, description=The total
number of outgoing bytes, tags={node-id=node--1}]
(org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,379] TRACE Registered metric named MetricName
[name=outgoing-byte-rate, group=admin-node-metrics, description=The number of
outgoing bytes per second, tags={node-id=node--1}]
(org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,379] TRACE Registered metric named MetricName
[name=request-total, group=admin-node-metrics, description=The total number of
requests sent, tags={node-id=node--1}] (org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,379] TRACE Registered metric named MetricName
[name=request-rate, group=admin-node-metrics, description=The number of
requests sent per second, tags={node-id=node--1}]
(org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,379] TRACE Registered metric named MetricName
[name=request-size-avg, group=admin-node-metrics, description=The average size
of requests sent., tags={node-id=node--1}]
(org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,383] TRACE Registered metric named MetricName
[name=request-size-max, group=admin-node-metrics, description=The maximum size
of any request sent., tags={node-id=node--1}]
(org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,383] DEBUG Added sensor with name node--1.bytes-received
(org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,383] TRACE Registered metric named MetricName
[name=incoming-byte-total, group=admin-node-metrics, description=The total
number of incoming bytes, tags={node-id=node--1}]
(org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,383] TRACE Registered metric named MetricName
[name=incoming-byte-rate, group=admin-node-metrics, description=The number of
incoming bytes per second, tags={node-id=node--1}]
(org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,383] TRACE Registered metric named MetricName
[name=response-total, group=admin-node-metrics, description=The total number of
responses received, tags={node-id=node--1}]
(org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,383] TRACE Registered metric named MetricName
[name=response-rate, group=admin-node-metrics, description=The number of
responses received per second, tags={node-id=node--1}]
(org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,384] DEBUG Added sensor with name node--1.latency
(org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,384] TRACE Registered metric named MetricName
[name=request-latency-avg, group=admin-node-metrics, description=,
tags={node-id=node--1}] (org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,384] TRACE Registered metric named MetricName
[name=request-latency-max, group=admin-node-metrics, description=,
tags={node-id=node--1}] (org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,385] DEBUG [Producer clientId=admin-1] Created socket with
SO_RCVBUF = 32768, SO_SNDBUF = 131072, SO_TIMEOUT = 0 to node -1
(org.apache.kafka.common.network.Selector)
[2020-07-08 18:48:09,452] DEBUG Set SASL client state to
RECEIVE_APIVERSIONS_RESPONSE
(org.apache.kafka.common.security.authenticator.SaslClientAuthenticator)
[2020-07-08 18:48:09,452] DEBUG [NetworkClient clientId=admin-1] Completed
connection to node -1. Fetching API versions.
(org.apache.kafka.clients.NetworkClient)
[2020-07-08 18:48:09,453] TRACE [NetworkClient clientId=admin-1] Removing node
broker1:2345 (id: -1 rack: null) from least loaded node selection:
is-blacked-out: false, in-flight-requests: 0
(org.apache.kafka.clients.NetworkClient)
[2020-07-08 18:48:09,453] TRACE [NetworkClient clientId=admin-1] Removing node
broker2:2345 (id: -2 rack: null) from least loaded node selection:
is-blacked-out: false, in-flight-requests: 0
(org.apache.kafka.clients.NetworkClient)
[2020-07-08 18:48:09,453] TRACE [NetworkClient clientId=admin-1] Found least
loaded node broker3:2345 (id: -3 rack: null)
(org.apache.kafka.clients.NetworkClient)
[2020-07-08 18:48:09,453] DEBUG [NetworkClient clientId=admin-1] Initialize
connection to node broker3:2345 (id: -3 rack: null) for sending metadata
request (org.apache.kafka.clients.NetworkClient)
[2020-07-08 18:48:09,453] DEBUG [NetworkClient clientId=admin-1] Initiating
connection to node broker3:2345 (id: -3 rack: null)
(org.apache.kafka.clients.NetworkClient)
[2020-07-08 18:48:09,454] DEBUG Set SASL client state to
SEND_APIVERSIONS_REQUEST
(org.apache.kafka.common.security.authenticator.SaslClientAuthenticator)
[2020-07-08 18:48:09,454] DEBUG Creating SaslClient:
client=null;service=kafka;serviceHostname=broker3;mechs=[SCRAM-SHA-512]
(org.apache.kafka.common.security.authenticator.SaslClientAuthenticator)
[2020-07-08 18:48:09,455] DEBUG Setting SASL/SCRAM_SHA_512 client state to
SEND_CLIENT_FIRST_MESSAGE
(org.apache.kafka.common.security.scram.internals.ScramSaslClient)
[2020-07-08 18:48:09,455] DEBUG Added sensor with name node--3.bytes-sent
(org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,455] TRACE Registered metric named MetricName
[name=outgoing-byte-total, group=admin-node-metrics, description=The total
number of outgoing bytes, tags={node-id=node--3}]
(org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,455] TRACE Registered metric named MetricName
[name=outgoing-byte-rate, group=admin-node-metrics, description=The number of
outgoing bytes per second, tags={node-id=node--3}]
(org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,455] TRACE Registered metric named MetricName
[name=request-total, group=admin-node-metrics, description=The total number of
requests sent, tags={node-id=node--3}] (org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,456] TRACE Registered metric named MetricName
[name=request-rate, group=admin-node-metrics, description=The number of
requests sent per second, tags={node-id=node--3}]
(org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,456] TRACE Registered metric named MetricName
[name=request-size-avg, group=admin-node-metrics, description=The average size
of requests sent., tags={node-id=node--3}]
(org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,456] TRACE Registered metric named MetricName
[name=request-size-max, group=admin-node-metrics, description=The maximum size
of any request sent., tags={node-id=node--3}]
(org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,456] DEBUG Added sensor with name node--3.bytes-received
(org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,456] TRACE Registered metric named MetricName
[name=incoming-byte-total, group=admin-node-metrics, description=The total
number of incoming bytes, tags={node-id=node--3}]
(org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,456] TRACE Registered metric named MetricName
[name=incoming-byte-rate, group=admin-node-metrics, description=The number of
incoming bytes per second, tags={node-id=node--3}]
(org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,456] TRACE Registered metric named MetricName
[name=response-total, group=admin-node-metrics, description=The total number of
responses received, tags={node-id=node--3}]
(org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,456] TRACE Registered metric named MetricName
[name=response-rate, group=admin-node-metrics, description=The number of
responses received per second, tags={node-id=node--3}]
(org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,457] DEBUG Added sensor with name node--3.latency
(org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,457] TRACE Registered metric named MetricName
[name=request-latency-avg, group=admin-node-metrics, description=,
tags={node-id=node--3}] (org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,457] TRACE Registered metric named MetricName
[name=request-latency-max, group=admin-node-metrics, description=,
tags={node-id=node--3}] (org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,457] DEBUG [Producer clientId=admin-1] Created socket with
SO_RCVBUF = 32768, SO_SNDBUF = 131072, SO_TIMEOUT = 0 to node -3
(org.apache.kafka.common.network.Selector)
[2020-07-08 18:48:09,457] DEBUG Set SASL client state to
RECEIVE_APIVERSIONS_RESPONSE
(org.apache.kafka.common.security.authenticator.SaslClientAuthenticator)
[2020-07-08 18:48:09,461] DEBUG Set SASL client state to SEND_HANDSHAKE_REQUEST
(org.apache.kafka.common.security.authenticator.SaslClientAuthenticator)
[2020-07-08 18:48:09,461] DEBUG Set SASL client state to
RECEIVE_HANDSHAKE_RESPONSE
(org.apache.kafka.common.security.authenticator.SaslClientAuthenticator)
[2020-07-08 18:48:09,461] DEBUG [NetworkClient clientId=admin-1] Completed
connection to node -3. Fetching API versions.
(org.apache.kafka.clients.NetworkClient)
[2020-07-08 18:48:09,462] TRACE [NetworkClient clientId=admin-1] Removing node
broker3:2345 (id: -3 rack: null) from least loaded node selection:
is-blacked-out: false, in-flight-requests: 0
(org.apache.kafka.clients.NetworkClient)
[2020-07-08 18:48:09,462] TRACE [NetworkClient clientId=admin-1] Removing node
broker1:2345 (id: -1 rack: null) from least loaded node selection:
is-blacked-out: false, in-flight-requests: 0
(org.apache.kafka.clients.NetworkClient)
[2020-07-08 18:48:09,462] TRACE [NetworkClient clientId=admin-1] Found least
loaded node broker2:2345 (id: -2 rack: null)
(org.apache.kafka.clients.NetworkClient)
[2020-07-08 18:48:09,462] DEBUG [NetworkClient clientId=admin-1] Initialize
connection to node broker2:2345 (id: -2 rack: null) for sending metadata
request (org.apache.kafka.clients.NetworkClient)
[2020-07-08 18:48:09,462] DEBUG [NetworkClient clientId=admin-1] Initiating
connection to node broker2:2345 (id: -2 rack: null)
(org.apache.kafka.clients.NetworkClient)
[2020-07-08 18:48:09,462] DEBUG Set SASL client state to
SEND_APIVERSIONS_REQUEST
(org.apache.kafka.common.security.authenticator.SaslClientAuthenticator)
[2020-07-08 18:48:09,462] DEBUG Creating SaslClient:
client=null;service=kafka;serviceHostname=broker2;mechs=[SCRAM-SHA-512]
(org.apache.kafka.common.security.authenticator.SaslClientAuthenticator)
[2020-07-08 18:48:09,463] DEBUG Setting SASL/SCRAM_SHA_512 client state to
SEND_CLIENT_FIRST_MESSAGE
(org.apache.kafka.common.security.scram.internals.ScramSaslClient)
[2020-07-08 18:48:09,464] DEBUG Set SASL client state to SEND_HANDSHAKE_REQUEST
(org.apache.kafka.common.security.authenticator.SaslClientAuthenticator)
[2020-07-08 18:48:09,464] DEBUG Set SASL client state to
RECEIVE_HANDSHAKE_RESPONSE
(org.apache.kafka.common.security.authenticator.SaslClientAuthenticator)
[2020-07-08 18:48:09,464] DEBUG Set SASL client state to INITIAL
(org.apache.kafka.common.security.authenticator.SaslClientAuthenticator)
[2020-07-08 18:48:09,471] DEBUG Setting SASL/SCRAM_SHA_512 client state to
RECEIVE_SERVER_FIRST_MESSAGE
(org.apache.kafka.common.security.scram.internals.ScramSaslClient)
[2020-07-08 18:48:09,471] DEBUG Set SASL client state to INTERMEDIATE
(org.apache.kafka.common.security.authenticator.SaslClientAuthenticator)
[2020-07-08 18:48:09,472] TRACE [NetworkClient clientId=admin-1] Removing node
broker2:2345 (id: -2 rack: null) from least loaded node selection:
is-blacked-out: false, in-flight-requests: 0
(org.apache.kafka.clients.NetworkClient)
[2020-07-08 18:48:09,472] TRACE [NetworkClient clientId=admin-1] Removing node
broker3:2345 (id: -3 rack: null) from least loaded node selection:
is-blacked-out: false, in-flight-requests: 0
(org.apache.kafka.clients.NetworkClient)
[2020-07-08 18:48:09,472] TRACE [NetworkClient clientId=admin-1] Found least
loaded node broker1:2345 (id: -1 rack: null)
(org.apache.kafka.clients.NetworkClient)
[2020-07-08 18:48:09,472] DEBUG Set SASL client state to INITIAL
(org.apache.kafka.common.security.authenticator.SaslClientAuthenticator)
[2020-07-08 18:48:09,472] DEBUG Setting SASL/SCRAM_SHA_512 client state to
RECEIVE_SERVER_FIRST_MESSAGE
(org.apache.kafka.common.security.scram.internals.ScramSaslClient)
[2020-07-08 18:48:09,473] DEBUG Set SASL client state to INTERMEDIATE
(org.apache.kafka.common.security.authenticator.SaslClientAuthenticator)
[2020-07-08 18:48:09,473] DEBUG Added sensor with name node--2.bytes-sent
(org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,473] TRACE Registered metric named MetricName
[name=outgoing-byte-total, group=admin-node-metrics, description=The total
number of outgoing bytes, tags={node-id=node--2}]
(org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,473] TRACE Registered metric named MetricName
[name=outgoing-byte-rate, group=admin-node-metrics, description=The number of
outgoing bytes per second, tags={node-id=node--2}]
(org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,474] TRACE Registered metric named MetricName
[name=request-total, group=admin-node-metrics, description=The total number of
requests sent, tags={node-id=node--2}] (org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,474] TRACE Registered metric named MetricName
[name=request-rate, group=admin-node-metrics, description=The number of
requests sent per second, tags={node-id=node--2}]
(org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,474] TRACE Registered metric named MetricName
[name=request-size-avg, group=admin-node-metrics, description=The average size
of requests sent., tags={node-id=node--2}]
(org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,474] TRACE Registered metric named MetricName
[name=request-size-max, group=admin-node-metrics, description=The maximum size
of any request sent., tags={node-id=node--2}]
(org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,474] DEBUG Added sensor with name node--2.bytes-received
(org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,474] TRACE Registered metric named MetricName
[name=incoming-byte-total, group=admin-node-metrics, description=The total
number of incoming bytes, tags={node-id=node--2}]
(org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,474] TRACE Registered metric named MetricName
[name=incoming-byte-rate, group=admin-node-metrics, description=The number of
incoming bytes per second, tags={node-id=node--2}]
(org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,474] TRACE Registered metric named MetricName
[name=response-total, group=admin-node-metrics, description=The total number of
responses received, tags={node-id=node--2}]
(org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,474] TRACE Registered metric named MetricName
[name=response-rate, group=admin-node-metrics, description=The number of
responses received per second, tags={node-id=node--2}]
(org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,475] DEBUG Added sensor with name node--2.latency
(org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,475] TRACE Registered metric named MetricName
[name=request-latency-avg, group=admin-node-metrics, description=,
tags={node-id=node--2}] (org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,475] TRACE Registered metric named MetricName
[name=request-latency-max, group=admin-node-metrics, description=,
tags={node-id=node--2}] (org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,475] DEBUG [Producer clientId=admin-1] Created socket with
SO_RCVBUF = 32768, SO_SNDBUF = 131072, SO_TIMEOUT = 0 to node -2
(org.apache.kafka.common.network.Selector)
[2020-07-08 18:48:09,475] DEBUG Set SASL client state to
RECEIVE_APIVERSIONS_RESPONSE
(org.apache.kafka.common.security.authenticator.SaslClientAuthenticator)
[2020-07-08 18:48:09,547] DEBUG Setting SASL/SCRAM_SHA_512 client state to
RECEIVE_SERVER_FINAL_MESSAGE
(org.apache.kafka.common.security.scram.internals.ScramSaslClient)
[2020-07-08 18:48:09,548] DEBUG [NetworkClient clientId=admin-1] Completed
connection to node -2. Fetching API versions.
(org.apache.kafka.clients.NetworkClient)
[2020-07-08 18:48:09,548] TRACE [NetworkClient clientId=admin-1] Removing node
broker2:2345 (id: -2 rack: null) from least loaded node selection:
is-blacked-out: false, in-flight-requests: 0
(org.apache.kafka.clients.NetworkClient)
[2020-07-08 18:48:09,548] TRACE [NetworkClient clientId=admin-1] Removing node
broker3:2345 (id: -3 rack: null) from least loaded node selection:
is-blacked-out: false, in-flight-requests: 0
(org.apache.kafka.clients.NetworkClient)
[2020-07-08 18:48:09,548] TRACE [NetworkClient clientId=admin-1] Found least
loaded node broker1:2345 (id: -1 rack: null)
(org.apache.kafka.clients.NetworkClient)
[2020-07-08 18:48:09,583] DEBUG Setting SASL/SCRAM_SHA_512 client state to
RECEIVE_SERVER_FINAL_MESSAGE
(org.apache.kafka.common.security.scram.internals.ScramSaslClient)
[2020-07-08 18:48:09,584] DEBUG Set SASL client state to SEND_HANDSHAKE_REQUEST
(org.apache.kafka.common.security.authenticator.SaslClientAuthenticator)
[2020-07-08 18:48:09,584] DEBUG Set SASL client state to
RECEIVE_HANDSHAKE_RESPONSE
(org.apache.kafka.common.security.authenticator.SaslClientAuthenticator)
[2020-07-08 18:48:09,586] DEBUG Setting SASL/SCRAM_SHA_512 client state to
COMPLETE (org.apache.kafka.common.security.scram.internals.ScramSaslClient)
[2020-07-08 18:48:09,586] DEBUG Set SASL client state to COMPLETE
(org.apache.kafka.common.security.authenticator.SaslClientAuthenticator)
[2020-07-08 18:48:09,587] DEBUG [NetworkClient clientId=admin-1] Initiating API
versions fetch from node -1. (org.apache.kafka.clients.NetworkClient)
[2020-07-08 18:48:09,587] TRACE [NetworkClient clientId=admin-1] No version
information found when sending API_VERSIONS with correlation id 1 to node -1.
Assuming version 2. (org.apache.kafka.clients.NetworkClient)
[2020-07-08 18:48:09,587] TRACE [NetworkClient clientId=admin-1] Sending
API_VERSIONS {} with correlation id 1 to node -1
(org.apache.kafka.clients.NetworkClient)
[2020-07-08 18:48:09,589] TRACE [NetworkClient clientId=admin-1] Removing node
broker3:2345 (id: -3 rack: null) from least loaded node selection:
is-blacked-out: false, in-flight-requests: 0
(org.apache.kafka.clients.NetworkClient)
[2020-07-08 18:48:09,589] TRACE [NetworkClient clientId=admin-1] Found least
loaded node broker2:2345 (id: -2 rack: null)
(org.apache.kafka.clients.NetworkClient)
[2020-07-08 18:48:09,590] DEBUG Setting SASL/SCRAM_SHA_512 client state to
COMPLETE (org.apache.kafka.common.security.scram.internals.ScramSaslClient)
[2020-07-08 18:48:09,590] DEBUG Set SASL client state to COMPLETE
(org.apache.kafka.common.security.authenticator.SaslClientAuthenticator)
[2020-07-08 18:48:09,590] DEBUG Set SASL client state to INITIAL
(org.apache.kafka.common.security.authenticator.SaslClientAuthenticator)
[2020-07-08 18:48:09,591] DEBUG Setting SASL/SCRAM_SHA_512 client state to
RECEIVE_SERVER_FIRST_MESSAGE
(org.apache.kafka.common.security.scram.internals.ScramSaslClient)
[2020-07-08 18:48:09,591] DEBUG Set SASL client state to INTERMEDIATE
(org.apache.kafka.common.security.authenticator.SaslClientAuthenticator)
[2020-07-08 18:48:09,591] DEBUG [NetworkClient clientId=admin-1] Initiating API
versions fetch from node -3. (org.apache.kafka.clients.NetworkClient)
[2020-07-08 18:48:09,591] TRACE [NetworkClient clientId=admin-1] No version
information found when sending API_VERSIONS with correlation id 2 to node -3.
Assuming version 2. (org.apache.kafka.clients.NetworkClient)
[2020-07-08 18:48:09,592] TRACE [NetworkClient clientId=admin-1] Sending
API_VERSIONS {} with correlation id 2 to node -3
(org.apache.kafka.clients.NetworkClient)
[2020-07-08 18:48:09,592] TRACE [NetworkClient clientId=admin-1] Removing node
broker1:2345 (id: -1 rack: null) from least loaded node selection:
is-blacked-out: false, in-flight-requests: 1
(org.apache.kafka.clients.NetworkClient)
[2020-07-08 18:48:09,592] TRACE [NetworkClient clientId=admin-1] Found least
loaded node broker2:2345 (id: -2 rack: null)
(org.apache.kafka.clients.NetworkClient)
[2020-07-08 18:48:09,593] TRACE [NetworkClient clientId=admin-1] Completed
receive from node -1 for API_VERSIONS with correlation id 1, received
{error_code=0,api_versions=[{api_key=0,min_version=0,max_version=6},{api_key=1,min_version=0,max_version=8},{api_key=2,min_version=0,max_version=3},{api_key=3,min_version=0,max_version=6},{api_key=4,min_version=0,max_version=1},{api_key=5,min_version=0,max_version=0},{api_key=6,min_version=0,max_version=4},{api_key=7,min_version=0,max_version=1},{api_key=8,min_version=0,max_version=4},{api_key=9,min_version=0,max_version=4},{api_key=10,min_version=0,max_version=2},{api_key=11,min_version=0,max_version=3},{api_key=12,min_version=0,max_version=2},{api_key=13,min_version=0,max_version=2},{api_key=14,min_version=0,max_version=2},{api_key=15,min_version=0,max_version=2},{api_key=16,min_version=0,max_version=2},{api_key=17,min_version=0,max_version=1},{api_key=18,min_version=0,max_version=2},{api_key=19,min_version=0,max_version=3},{api_key=20,min_version=0,max_version=2},{api_key=21,min_version=0,max_version=1},{api_key=22,min_version=0,max_version=1},{api_key=23,min_version=0,max_version=1},{api_key=24,min_version=0,max_version=1},{api_key=25,min_version=0,max_version=1},{api_key=26,min_version=0,max_version=1},{api_key=27,min_version=0,max_version=0},{api_key=28,min_version=0,max_version=1},{api_key=29,min_version=0,max_version=1},{api_key=30,min_version=0,max_version=1},{api_key=31,min_version=0,max_version=1},{api_key=32,min_version=0,max_version=2},{api_key=33,min_version=0,max_version=1},{api_key=34,min_version=0,max_version=1},{api_key=35,min_version=0,max_version=1},{api_key=36,min_version=0,max_version=0},{api_key=37,min_version=0,max_version=1},{api_key=38,min_version=0,max_version=1},{api_key=39,min_version=0,max_version=1},{api_key=40,min_version=0,max_version=1},{api_key=41,min_version=0,max_version=1},{api_key=42,min_version=0,max_version=1}],throttle_time_ms=0}
(org.apache.kafka.clients.NetworkClient)
[2020-07-08 18:48:09,595] DEBUG [NetworkClient clientId=admin-1] Recorded API
versions for node -1: (Produce(0): 0 to 6 [usable: 6], Fetch(1): 0 to 8
[usable: 8], ListOffsets(2): 0 to 3 [usable: 3], Metadata(3): 0 to 6 [usable:
6], LeaderAndIsr(4): 0 to 1 [usable: 1], StopReplica(5): 0 [usable: 0],
UpdateMetadata(6): 0 to 4 [usable: 4], ControlledShutdown(7): 0 to 1 [usable:
1], OffsetCommit(8): 0 to 4 [usable: 4], OffsetFetch(9): 0 to 4 [usable: 4],
FindCoordinator(10): 0 to 2 [usable: 2], JoinGroup(11): 0 to 3 [usable: 3],
Heartbeat(12): 0 to 2 [usable: 2], LeaveGroup(13): 0 to 2 [usable: 2],
SyncGroup(14): 0 to 2 [usable: 2], DescribeGroups(15): 0 to 2 [usable: 2],
ListGroups(16): 0 to 2 [usable: 2], SaslHandshake(17): 0 to 1 [usable: 1],
ApiVersions(18): 0 to 2 [usable: 2], CreateTopics(19): 0 to 3 [usable: 3],
DeleteTopics(20): 0 to 2 [usable: 2], DeleteRecords(21): 0 to 1 [usable: 1],
InitProducerId(22): 0 to 1 [usable: 1], OffsetForLeaderEpoch(23): 0 to 1
[usable: 1], AddPartitionsToTxn(24): 0 to 1 [usable: 1], AddOffsetsToTxn(25): 0
to 1 [usable: 1], EndTxn(26): 0 to 1 [usable: 1], WriteTxnMarkers(27): 0
[usable: 0], TxnOffsetCommit(28): 0 to 1 [usable: 1], DescribeAcls(29): 0 to 1
[usable: 1], CreateAcls(30): 0 to 1 [usable: 1], DeleteAcls(31): 0 to 1
[usable: 1], DescribeConfigs(32): 0 to 2 [usable: 2], AlterConfigs(33): 0 to 1
[usable: 1], AlterReplicaLogDirs(34): 0 to 1 [usable: 1], DescribeLogDirs(35):
0 to 1 [usable: 1], SaslAuthenticate(36): 0 [usable: 0], CreatePartitions(37):
0 to 1 [usable: 1], CreateDelegationToken(38): 0 to 1 [usable: 1],
RenewDelegationToken(39): 0 to 1 [usable: 1], ExpireDelegationToken(40): 0 to 1
[usable: 1], DescribeDelegationToken(41): 0 to 1 [usable: 1], DeleteGroups(42):
0 to 1 [usable: 1]) (org.apache.kafka.clients.NetworkClient)
[2020-07-08 18:48:09,596] TRACE [NetworkClient clientId=admin-1] Removing node
broker3:2345 (id: -3 rack: null) from least loaded node selection:
is-blacked-out: false, in-flight-requests: 1
(org.apache.kafka.clients.NetworkClient)
[2020-07-08 18:48:09,596] TRACE [NetworkClient clientId=admin-1] Removing node
broker1:2345 (id: -1 rack: null) from least loaded node selection:
is-blacked-out: false, in-flight-requests: 0
(org.apache.kafka.clients.NetworkClient)
[2020-07-08 18:48:09,596] TRACE [NetworkClient clientId=admin-1] Found least
loaded node broker2:2345 (id: -2 rack: null)
(org.apache.kafka.clients.NetworkClient)
[2020-07-08 18:48:09,615] DEBUG Setting SASL/SCRAM_SHA_512 client state to
RECEIVE_SERVER_FINAL_MESSAGE
(org.apache.kafka.common.security.scram.internals.ScramSaslClient)
[2020-07-08 18:48:09,616] TRACE [NetworkClient clientId=admin-1] Completed
receive from node -3 for API_VERSIONS with correlation id 2, received
{error_code=0,api_versions=[{api_key=0,min_version=0,max_version=6},{api_key=1,min_version=0,max_version=8},{api_key=2,min_version=0,max_version=3},{api_key=3,min_version=0,max_version=6},{api_key=4,min_version=0,max_version=1},{api_key=5,min_version=0,max_version=0},{api_key=6,min_version=0,max_version=4},{api_key=7,min_version=0,max_version=1},{api_key=8,min_version=0,max_version=4},{api_key=9,min_version=0,max_version=4},{api_key=10,min_version=0,max_version=2},{api_key=11,min_version=0,max_version=3},{api_key=12,min_version=0,max_version=2},{api_key=13,min_version=0,max_version=2},{api_key=14,min_version=0,max_version=2},{api_key=15,min_version=0,max_version=2},{api_key=16,min_version=0,max_version=2},{api_key=17,min_version=0,max_version=1},{api_key=18,min_version=0,max_version=2},{api_key=19,min_version=0,max_version=3},{api_key=20,min_version=0,max_version=2},{api_key=21,min_version=0,max_version=1},{api_key=22,min_version=0,max_version=1},{api_key=23,min_version=0,max_version=1},{api_key=24,min_version=0,max_version=1},{api_key=25,min_version=0,max_version=1},{api_key=26,min_version=0,max_version=1},{api_key=27,min_version=0,max_version=0},{api_key=28,min_version=0,max_version=1},{api_key=29,min_version=0,max_version=1},{api_key=30,min_version=0,max_version=1},{api_key=31,min_version=0,max_version=1},{api_key=32,min_version=0,max_version=2},{api_key=33,min_version=0,max_version=1},{api_key=34,min_version=0,max_version=1},{api_key=35,min_version=0,max_version=1},{api_key=36,min_version=0,max_version=0},{api_key=37,min_version=0,max_version=1},{api_key=38,min_version=0,max_version=1},{api_key=39,min_version=0,max_version=1},{api_key=40,min_version=0,max_version=1},{api_key=41,min_version=0,max_version=1},{api_key=42,min_version=0,max_version=1}],throttle_time_ms=0}
(org.apache.kafka.clients.NetworkClient)
[2020-07-08 18:48:09,617] DEBUG [NetworkClient clientId=admin-1] Recorded API
versions for node -3: (Produce(0): 0 to 6 [usable: 6], Fetch(1): 0 to 8
[usable: 8], ListOffsets(2): 0 to 3 [usable: 3], Metadata(3): 0 to 6 [usable:
6], LeaderAndIsr(4): 0 to 1 [usable: 1], StopReplica(5): 0 [usable: 0],
UpdateMetadata(6): 0 to 4 [usable: 4], ControlledShutdown(7): 0 to 1 [usable:
1], OffsetCommit(8): 0 to 4 [usable: 4], OffsetFetch(9): 0 to 4 [usable: 4],
FindCoordinator(10): 0 to 2 [usable: 2], JoinGroup(11): 0 to 3 [usable: 3],
Heartbeat(12): 0 to 2 [usable: 2], LeaveGroup(13): 0 to 2 [usable: 2],
SyncGroup(14): 0 to 2 [usable: 2], DescribeGroups(15): 0 to 2 [usable: 2],
ListGroups(16): 0 to 2 [usable: 2], SaslHandshake(17): 0 to 1 [usable: 1],
ApiVersions(18): 0 to 2 [usable: 2], CreateTopics(19): 0 to 3 [usable: 3],
DeleteTopics(20): 0 to 2 [usable: 2], DeleteRecords(21): 0 to 1 [usable: 1],
InitProducerId(22): 0 to 1 [usable: 1], OffsetForLeaderEpoch(23): 0 to 1
[usable: 1], AddPartitionsToTxn(24): 0 to 1 [usable: 1], AddOffsetsToTxn(25): 0
to 1 [usable: 1], EndTxn(26): 0 to 1 [usable: 1], WriteTxnMarkers(27): 0
[usable: 0], TxnOffsetCommit(28): 0 to 1 [usable: 1], DescribeAcls(29): 0 to 1
[usable: 1], CreateAcls(30): 0 to 1 [usable: 1], DeleteAcls(31): 0 to 1
[usable: 1], DescribeConfigs(32): 0 to 2 [usable: 2], AlterConfigs(33): 0 to 1
[usable: 1], AlterReplicaLogDirs(34): 0 to 1 [usable: 1], DescribeLogDirs(35):
0 to 1 [usable: 1], SaslAuthenticate(36): 0 [usable: 0], CreatePartitions(37):
0 to 1 [usable: 1], CreateDelegationToken(38): 0 to 1 [usable: 1],
RenewDelegationToken(39): 0 to 1 [usable: 1], ExpireDelegationToken(40): 0 to 1
[usable: 1], DescribeDelegationToken(41): 0 to 1 [usable: 1], DeleteGroups(42):
0 to 1 [usable: 1]) (org.apache.kafka.clients.NetworkClient)
[2020-07-08 18:48:09,617] TRACE [NetworkClient clientId=admin-1] Removing node
broker3:2345 (id: -3 rack: null) from least loaded node selection:
is-blacked-out: false, in-flight-requests: 0
(org.apache.kafka.clients.NetworkClient)
[2020-07-08 18:48:09,617] TRACE [NetworkClient clientId=admin-1] Removing node
broker1:2345 (id: -1 rack: null) from least loaded node selection:
is-blacked-out: false, in-flight-requests: 0
(org.apache.kafka.clients.NetworkClient)
[2020-07-08 18:48:09,617] TRACE [NetworkClient clientId=admin-1] Found least
loaded node broker2:2345 (id: -2 rack: null)
(org.apache.kafka.clients.NetworkClient)
[2020-07-08 18:48:09,620] DEBUG Setting SASL/SCRAM_SHA_512 client state to
COMPLETE (org.apache.kafka.common.security.scram.internals.ScramSaslClient)
[2020-07-08 18:48:09,620] DEBUG Set SASL client state to COMPLETE
(org.apache.kafka.common.security.authenticator.SaslClientAuthenticator)
[2020-07-08 18:48:09,620] DEBUG [NetworkClient clientId=admin-1] Initiating API
versions fetch from node -2. (org.apache.kafka.clients.NetworkClient)
[2020-07-08 18:48:09,620] TRACE [NetworkClient clientId=admin-1] No version
information found when sending API_VERSIONS with correlation id 3 to node -2.
Assuming version 2. (org.apache.kafka.clients.NetworkClient)
[2020-07-08 18:48:09,620] TRACE [NetworkClient clientId=admin-1] Sending
API_VERSIONS {} with correlation id 3 to node -2
(org.apache.kafka.clients.NetworkClient)
[2020-07-08 18:48:09,621] TRACE [NetworkClient clientId=admin-1] Removing node
broker1:2345 (id: -1 rack: null) from least loaded node selection:
is-blacked-out: false, in-flight-requests: 0
(org.apache.kafka.clients.NetworkClient)
[2020-07-08 18:48:09,621] TRACE [NetworkClient clientId=admin-1] Removing node
broker2:2345 (id: -2 rack: null) from least loaded node selection:
is-blacked-out: false, in-flight-requests: 1
(org.apache.kafka.clients.NetworkClient)
[2020-07-08 18:48:09,621] TRACE [NetworkClient clientId=admin-1] Found least
loaded node broker3:2345 (id: -3 rack: null)
(org.apache.kafka.clients.NetworkClient)
[2020-07-08 18:48:09,621] DEBUG [NetworkClient clientId=admin-1] Sending
metadata request (type=MetadataRequest, topics=) to node broker3:2345 (id: -3
rack: null) (org.apache.kafka.clients.NetworkClient)
[2020-07-08 18:48:09,622] TRACE [NetworkClient clientId=admin-1] Sending
METADATA {topics=[],allow_auto_topic_creation=true} with correlation id 4 to
node -3 (org.apache.kafka.clients.NetworkClient)
[2020-07-08 18:48:09,623] TRACE [NetworkClient clientId=admin-1] Sending
FIND_COORDINATOR
{coordinator_key=spark-kafka-source-2890f4c5-6119-48d4-8f1c-1d911ec4f36e--101837091-driver-0,coordinator_type=0}
with correlation id 0 to node -1 (org.apache.kafka.clients.NetworkClient)
[2020-07-08 18:48:09,624] TRACE [NetworkClient clientId=admin-1] Completed
receive from node -3 for METADATA with correlation id 4, received
{throttle_time_ms=0,brokers=[{node_id=2,host=broker2,port=3411,rack=null},{node_id=1,host=broker1,port=3411,rack=null},{node_id=3,host=broker3,port=3411,rack=null}],cluster_id=EgYhY5ofQaOMSa3UOR-t_w,controller_id=2,topic_metadata=[]}
(org.apache.kafka.clients.NetworkClient)
[2020-07-08 18:48:09,630] INFO Cluster ID: EgYhY5ofQaOMSa3UOR-t_w
(org.apache.kafka.clients.Metadata)
[2020-07-08 18:48:09,631] DEBUG Updated cluster metadata version 2 to
Cluster(id = EgYhY5ofQaOMSa3UOR-t_w, nodes = [broker1:2345 (id: 1 rack: null),
broker3:2345 (id: 3 rack: null), broker2:2345 (id: 2 rack: null)], partitions =
[], controller = broker2:2345 (id: 2 rack: null))
(org.apache.kafka.clients.Metadata)
[2020-07-08 18:48:09,632] TRACE [NetworkClient clientId=admin-1] Completed
receive from node -1 for FIND_COORDINATOR with correlation id 0, received
{throttle_time_ms=0,error_code=0,error_message=null,coordinator={node_id=1,host=broker1,port=3411}}
(org.apache.kafka.clients.NetworkClient)
[2020-07-08 18:48:09,633] TRACE [NetworkClient clientId=admin-1] Completed
receive from node -2 for API_VERSIONS with correlation id 3, received
{error_code=0,api_versions=[{api_key=0,min_version=0,max_version=6},{api_key=1,min_version=0,max_version=8},{api_key=2,min_version=0,max_version=3},{api_key=3,min_version=0,max_version=6},{api_key=4,min_version=0,max_version=1},{api_key=5,min_version=0,max_version=0},{api_key=6,min_version=0,max_version=4},{api_key=7,min_version=0,max_version=1},{api_key=8,min_version=0,max_version=4},{api_key=9,min_version=0,max_version=4},{api_key=10,min_version=0,max_version=2},{api_key=11,min_version=0,max_version=3},{api_key=12,min_version=0,max_version=2},{api_key=13,min_version=0,max_version=2},{api_key=14,min_version=0,max_version=2},{api_key=15,min_version=0,max_version=2},{api_key=16,min_version=0,max_version=2},{api_key=17,min_version=0,max_version=1},{api_key=18,min_version=0,max_version=2},{api_key=19,min_version=0,max_version=3},{api_key=20,min_version=0,max_version=2},{api_key=21,min_version=0,max_version=1},{api_key=22,min_version=0,max_version=1},{api_key=23,min_version=0,max_version=1},{api_key=24,min_version=0,max_version=1},{api_key=25,min_version=0,max_version=1},{api_key=26,min_version=0,max_version=1},{api_key=27,min_version=0,max_version=0},{api_key=28,min_version=0,max_version=1},{api_key=29,min_version=0,max_version=1},{api_key=30,min_version=0,max_version=1},{api_key=31,min_version=0,max_version=1},{api_key=32,min_version=0,max_version=2},{api_key=33,min_version=0,max_version=1},{api_key=34,min_version=0,max_version=1},{api_key=35,min_version=0,max_version=1},{api_key=36,min_version=0,max_version=0},{api_key=37,min_version=0,max_version=1},{api_key=38,min_version=0,max_version=1},{api_key=39,min_version=0,max_version=1},{api_key=40,min_version=0,max_version=1},{api_key=41,min_version=0,max_version=1},{api_key=42,min_version=0,max_version=1}],throttle_time_ms=0}
(org.apache.kafka.clients.NetworkClient)
[2020-07-08 18:48:09,635] DEBUG [NetworkClient clientId=admin-1] Recorded API
versions for node -2: (Produce(0): 0 to 6 [usable: 6], Fetch(1): 0 to 8
[usable: 8], ListOffsets(2): 0 to 3 [usable: 3], Metadata(3): 0 to 6 [usable:
6], LeaderAndIsr(4): 0 to 1 [usable: 1], StopReplica(5): 0 [usable: 0],
UpdateMetadata(6): 0 to 4 [usable: 4], ControlledShutdown(7): 0 to 1 [usable:
1], OffsetCommit(8): 0 to 4 [usable: 4], OffsetFetch(9): 0 to 4 [usable: 4],
FindCoordinator(10): 0 to 2 [usable: 2], JoinGroup(11): 0 to 3 [usable: 3],
Heartbeat(12): 0 to 2 [usable: 2], LeaveGroup(13): 0 to 2 [usable: 2],
SyncGroup(14): 0 to 2 [usable: 2], DescribeGroups(15): 0 to 2 [usable: 2],
ListGroups(16): 0 to 2 [usable: 2], SaslHandshake(17): 0 to 1 [usable: 1],
ApiVersions(18): 0 to 2 [usable: 2], CreateTopics(19): 0 to 3 [usable: 3],
DeleteTopics(20): 0 to 2 [usable: 2], DeleteRecords(21): 0 to 1 [usable: 1],
InitProducerId(22): 0 to 1 [usable: 1], OffsetForLeaderEpoch(23): 0 to 1
[usable: 1], AddPartitionsToTxn(24): 0 to 1 [usable: 1], AddOffsetsToTxn(25): 0
to 1 [usable: 1], EndTxn(26): 0 to 1 [usable: 1], WriteTxnMarkers(27): 0
[usable: 0], TxnOffsetCommit(28): 0 to 1 [usable: 1], DescribeAcls(29): 0 to 1
[usable: 1], CreateAcls(30): 0 to 1 [usable: 1], DeleteAcls(31): 0 to 1
[usable: 1], DescribeConfigs(32): 0 to 2 [usable: 2], AlterConfigs(33): 0 to 1
[usable: 1], AlterReplicaLogDirs(34): 0 to 1 [usable: 1], DescribeLogDirs(35):
0 to 1 [usable: 1], SaslAuthenticate(36): 0 [usable: 0], CreatePartitions(37):
0 to 1 [usable: 1], CreateDelegationToken(38): 0 to 1 [usable: 1],
RenewDelegationToken(39): 0 to 1 [usable: 1], ExpireDelegationToken(40): 0 to 1
[usable: 1], DescribeDelegationToken(41): 0 to 1 [usable: 1], DeleteGroups(42):
0 to 1 [usable: 1]) (org.apache.kafka.clients.NetworkClient)
[2020-07-08 18:48:09,636] DEBUG [NetworkClient clientId=admin-1] Initiating
connection to node broker1:2345 (id: 1 rack: null)
(org.apache.kafka.clients.NetworkClient)
[2020-07-08 18:48:09,637] DEBUG Set SASL client state to
SEND_APIVERSIONS_REQUEST
(org.apache.kafka.common.security.authenticator.SaslClientAuthenticator)
[2020-07-08 18:48:09,637] DEBUG Creating SaslClient:
client=null;service=kafka;serviceHostname=broker1;mechs=[SCRAM-SHA-512]
(org.apache.kafka.common.security.authenticator.SaslClientAuthenticator)
[2020-07-08 18:48:09,638] DEBUG Setting SASL/SCRAM_SHA_512 client state to
SEND_CLIENT_FIRST_MESSAGE
(org.apache.kafka.common.security.scram.internals.ScramSaslClient)
[2020-07-08 18:48:09,639] DEBUG Added sensor with name node-1.bytes-sent
(org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,639] TRACE Registered metric named MetricName
[name=outgoing-byte-total, group=admin-node-metrics, description=The total
number of outgoing bytes, tags={node-id=node-1}]
(org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,639] TRACE Registered metric named MetricName
[name=outgoing-byte-rate, group=admin-node-metrics, description=The number of
outgoing bytes per second, tags={node-id=node-1}]
(org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,639] TRACE Registered metric named MetricName
[name=request-total, group=admin-node-metrics, description=The total number of
requests sent, tags={node-id=node-1}] (org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,639] TRACE Registered metric named MetricName
[name=request-rate, group=admin-node-metrics, description=The number of
requests sent per second, tags={node-id=node-1}]
(org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,640] TRACE Registered metric named MetricName
[name=request-size-avg, group=admin-node-metrics, description=The average size
of requests sent., tags={node-id=node-1}]
(org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,640] TRACE Registered metric named MetricName
[name=request-size-max, group=admin-node-metrics, description=The maximum size
of any request sent., tags={node-id=node-1}]
(org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,640] DEBUG Added sensor with name node-1.bytes-received
(org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,640] TRACE Registered metric named MetricName
[name=incoming-byte-total, group=admin-node-metrics, description=The total
number of incoming bytes, tags={node-id=node-1}]
(org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,640] TRACE Registered metric named MetricName
[name=incoming-byte-rate, group=admin-node-metrics, description=The number of
incoming bytes per second, tags={node-id=node-1}]
(org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,640] TRACE Registered metric named MetricName
[name=response-total, group=admin-node-metrics, description=The total number of
responses received, tags={node-id=node-1}]
(org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,640] TRACE Registered metric named MetricName
[name=response-rate, group=admin-node-metrics, description=The number of
responses received per second, tags={node-id=node-1}]
(org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,640] DEBUG Added sensor with name node-1.latency
(org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,641] TRACE Registered metric named MetricName
[name=request-latency-avg, group=admin-node-metrics, description=,
tags={node-id=node-1}] (org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,641] TRACE Registered metric named MetricName
[name=request-latency-max, group=admin-node-metrics, description=,
tags={node-id=node-1}] (org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,641] DEBUG [Producer clientId=admin-1] Created socket with
SO_RCVBUF = 32768, SO_SNDBUF = 131072, SO_TIMEOUT = 0 to node 1
(org.apache.kafka.common.network.Selector)
[2020-07-08 18:48:09,641] DEBUG Set SASL client state to
RECEIVE_APIVERSIONS_RESPONSE
(org.apache.kafka.common.security.authenticator.SaslClientAuthenticator)
[2020-07-08 18:48:09,641] DEBUG [NetworkClient clientId=admin-1] Completed
connection to node 1. Fetching API versions.
(org.apache.kafka.clients.NetworkClient)
[2020-07-08 18:48:09,642] DEBUG Set SASL client state to SEND_HANDSHAKE_REQUEST
(org.apache.kafka.common.security.authenticator.SaslClientAuthenticator)
[2020-07-08 18:48:09,642] DEBUG Set SASL client state to
RECEIVE_HANDSHAKE_RESPONSE
(org.apache.kafka.common.security.authenticator.SaslClientAuthenticator)
[2020-07-08 18:48:09,643] DEBUG Set SASL client state to INITIAL
(org.apache.kafka.common.security.authenticator.SaslClientAuthenticator)
[2020-07-08 18:48:09,644] DEBUG Setting SASL/SCRAM_SHA_512 client state to
RECEIVE_SERVER_FIRST_MESSAGE
(org.apache.kafka.common.security.scram.internals.ScramSaslClient)
[2020-07-08 18:48:09,644] DEBUG Set SASL client state to INTERMEDIATE
(org.apache.kafka.common.security.authenticator.SaslClientAuthenticator)
[2020-07-08 18:48:09,664] DEBUG Setting SASL/SCRAM_SHA_512 client state to
RECEIVE_SERVER_FINAL_MESSAGE
(org.apache.kafka.common.security.scram.internals.ScramSaslClient)
[2020-07-08 18:48:09,665] DEBUG Setting SASL/SCRAM_SHA_512 client state to
COMPLETE (org.apache.kafka.common.security.scram.internals.ScramSaslClient)
[2020-07-08 18:48:09,666] DEBUG Set SASL client state to COMPLETE
(org.apache.kafka.common.security.authenticator.SaslClientAuthenticator)
[2020-07-08 18:48:09,666] DEBUG [NetworkClient clientId=admin-1] Initiating API
versions fetch from node 1. (org.apache.kafka.clients.NetworkClient)
[2020-07-08 18:48:09,666] TRACE [NetworkClient clientId=admin-1] No version
information found when sending API_VERSIONS with correlation id 6 to node 1.
Assuming version 2. (org.apache.kafka.clients.NetworkClient)
[2020-07-08 18:48:09,666] TRACE [NetworkClient clientId=admin-1] Sending
API_VERSIONS {} with correlation id 6 to node 1
(org.apache.kafka.clients.NetworkClient)
[2020-07-08 18:48:09,668] TRACE [NetworkClient clientId=admin-1] Completed
receive from node 1 for API_VERSIONS with correlation id 6, received
{error_code=0,api_versions=[{api_key=0,min_version=0,max_version=6},{api_key=1,min_version=0,max_version=8},{api_key=2,min_version=0,max_version=3},{api_key=3,min_version=0,max_version=6},{api_key=4,min_version=0,max_version=1},{api_key=5,min_version=0,max_version=0},{api_key=6,min_version=0,max_version=4},{api_key=7,min_version=0,max_version=1},{api_key=8,min_version=0,max_version=4},{api_key=9,min_version=0,max_version=4},{api_key=10,min_version=0,max_version=2},{api_key=11,min_version=0,max_version=3},{api_key=12,min_version=0,max_version=2},{api_key=13,min_version=0,max_version=2},{api_key=14,min_version=0,max_version=2},{api_key=15,min_version=0,max_version=2},{api_key=16,min_version=0,max_version=2},{api_key=17,min_version=0,max_version=1},{api_key=18,min_version=0,max_version=2},{api_key=19,min_version=0,max_version=3},{api_key=20,min_version=0,max_version=2},{api_key=21,min_version=0,max_version=1},{api_key=22,min_version=0,max_version=1},{api_key=23,min_version=0,max_version=1},{api_key=24,min_version=0,max_version=1},{api_key=25,min_version=0,max_version=1},{api_key=26,min_version=0,max_version=1},{api_key=27,min_version=0,max_version=0},{api_key=28,min_version=0,max_version=1},{api_key=29,min_version=0,max_version=1},{api_key=30,min_version=0,max_version=1},{api_key=31,min_version=0,max_version=1},{api_key=32,min_version=0,max_version=2},{api_key=33,min_version=0,max_version=1},{api_key=34,min_version=0,max_version=1},{api_key=35,min_version=0,max_version=1},{api_key=36,min_version=0,max_version=0},{api_key=37,min_version=0,max_version=1},{api_key=38,min_version=0,max_version=1},{api_key=39,min_version=0,max_version=1},{api_key=40,min_version=0,max_version=1},{api_key=41,min_version=0,max_version=1},{api_key=42,min_version=0,max_version=1}],throttle_time_ms=0}
(org.apache.kafka.clients.NetworkClient)
[2020-07-08 18:48:09,669] DEBUG [NetworkClient clientId=admin-1] Recorded API
versions for node 1: (Produce(0): 0 to 6 [usable: 6], Fetch(1): 0 to 8 [usable:
8], ListOffsets(2): 0 to 3 [usable: 3], Metadata(3): 0 to 6 [usable: 6],
LeaderAndIsr(4): 0 to 1 [usable: 1], StopReplica(5): 0 [usable: 0],
UpdateMetadata(6): 0 to 4 [usable: 4], ControlledShutdown(7): 0 to 1 [usable:
1], OffsetCommit(8): 0 to 4 [usable: 4], OffsetFetch(9): 0 to 4 [usable: 4],
FindCoordinator(10): 0 to 2 [usable: 2], JoinGroup(11): 0 to 3 [usable: 3],
Heartbeat(12): 0 to 2 [usable: 2], LeaveGroup(13): 0 to 2 [usable: 2],
SyncGroup(14): 0 to 2 [usable: 2], DescribeGroups(15): 0 to 2 [usable: 2],
ListGroups(16): 0 to 2 [usable: 2], SaslHandshake(17): 0 to 1 [usable: 1],
ApiVersions(18): 0 to 2 [usable: 2], CreateTopics(19): 0 to 3 [usable: 3],
DeleteTopics(20): 0 to 2 [usable: 2], DeleteRecords(21): 0 to 1 [usable: 1],
InitProducerId(22): 0 to 1 [usable: 1], OffsetForLeaderEpoch(23): 0 to 1
[usable: 1], AddPartitionsToTxn(24): 0 to 1 [usable: 1], AddOffsetsToTxn(25): 0
to 1 [usable: 1], EndTxn(26): 0 to 1 [usable: 1], WriteTxnMarkers(27): 0
[usable: 0], TxnOffsetCommit(28): 0 to 1 [usable: 1], DescribeAcls(29): 0 to 1
[usable: 1], CreateAcls(30): 0 to 1 [usable: 1], DeleteAcls(31): 0 to 1
[usable: 1], DescribeConfigs(32): 0 to 2 [usable: 2], AlterConfigs(33): 0 to 1
[usable: 1], AlterReplicaLogDirs(34): 0 to 1 [usable: 1], DescribeLogDirs(35):
0 to 1 [usable: 1], SaslAuthenticate(36): 0 [usable: 0], CreatePartitions(37):
0 to 1 [usable: 1], CreateDelegationToken(38): 0 to 1 [usable: 1],
RenewDelegationToken(39): 0 to 1 [usable: 1], ExpireDelegationToken(40): 0 to 1
[usable: 1], DescribeDelegationToken(41): 0 to 1 [usable: 1], DeleteGroups(42):
0 to 1 [usable: 1]) (org.apache.kafka.clients.NetworkClient)
[2020-07-08 18:48:09,669] TRACE [NetworkClient clientId=admin-1] Sending
DESCRIBE_GROUPS
{group_ids=[spark-kafka-source-2890f4c5-6119-48d4-8f1c-1d911ec4f36e--101837091-driver-0]}
with correlation id 5 to node 1 (org.apache.kafka.clients.NetworkClient)
[2020-07-08 18:48:09,671] TRACE [NetworkClient clientId=admin-1] Completed
receive from node 1 for DESCRIBE_GROUPS with correlation id 5, received
{throttle_time_ms=0,groups=[{error_code=0,group_id=spark-kafka-source-2890f4c5-6119-48d4-8f1c-1d911ec4f36e--101837091-driver-0,state=Stable,protocol_type=consumer,protocol=range,members=[{member_id=consumer-1-81b2f357-e145-4806-b987-12122f4072f6,client_id=consumer-1,client_host=/10.0.0.1,member_metadata=java.nio.HeapByteBuffer[pos=0
lim=49 cap=110],member_assignment=java.nio.HeapByteBuffer[pos=0 lim=57
cap=57]}]}]} (org.apache.kafka.clients.NetworkClient)
[2020-07-08 18:48:09,683] TRACE [NetworkClient clientId=admin-1] Sending
FIND_COORDINATOR
{coordinator_key=spark-kafka-source-2890f4c5-6119-48d4-8f1c-1d911ec4f36e--101837091-driver-0,coordinator_type=0}
with correlation id 7 to node -1 (org.apache.kafka.clients.NetworkClient)
[2020-07-08 18:48:09,685] TRACE [NetworkClient clientId=admin-1] Completed
receive from node -1 for FIND_COORDINATOR with correlation id 7, received
{throttle_time_ms=0,error_code=0,error_message=null,coordinator={node_id=1,host=broker1,port=3411}}
(org.apache.kafka.clients.NetworkClient)
[2020-07-08 18:48:09,686] TRACE [NetworkClient clientId=admin-1] Sending
OFFSET_FETCH
{group_id=spark-kafka-source-2890f4c5-6119-48d4-8f1c-1d911ec4f36e--101837091-driver-0,topics=null}
with correlation id 8 to node 1 (org.apache.kafka.clients.NetworkClient)
[2020-07-08 18:48:09,689] TRACE [NetworkClient clientId=admin-1] Completed
receive from node 1 for OFFSET_FETCH with correlation id 8, received
{throttle_time_ms=0,responses=[],error_code=0}
(org.apache.kafka.clients.NetworkClient)
[2020-07-08 18:48:09,792] TRACE Removed metric named MetricName
[name=connection-count, group=admin-metrics, description=The current number of
active connections., tags={}] (org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,792] TRACE Removed metric named MetricName
[name=connection-close-total, group=admin-metrics, description=The total number
of connections closed, tags={}] (org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,792] TRACE Removed metric named MetricName
[name=connection-close-rate, group=admin-metrics, description=The number of
connections closed per second, tags={}]
(org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,792] DEBUG Removed sensor with name connections-closed:
(org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,792] TRACE Removed metric named MetricName
[name=connection-creation-total, group=admin-metrics, description=The total
number of new connections established, tags={}]
(org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,792] TRACE Removed metric named MetricName
[name=connection-creation-rate, group=admin-metrics, description=The number of
new connections established per second, tags={}]
(org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,792] DEBUG Removed sensor with name connections-created:
(org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,792] TRACE Removed metric named MetricName
[name=successful-authentication-total, group=admin-metrics, description=The
total number of connections with successful authentication, tags={}]
(org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,792] TRACE Removed metric named MetricName
[name=successful-authentication-rate, group=admin-metrics, description=The
number of connections with successful authentication per second, tags={}]
(org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,792] DEBUG Removed sensor with name
successful-authentication: (org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,792] TRACE Removed metric named MetricName
[name=failed-authentication-total, group=admin-metrics, description=The total
number of connections with failed authentication, tags={}]
(org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,793] TRACE Removed metric named MetricName
[name=failed-authentication-rate, group=admin-metrics, description=The number
of connections with failed authentication per second, tags={}]
(org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,793] DEBUG Removed sensor with name failed-authentication:
(org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,793] TRACE Removed metric named MetricName
[name=network-io-total, group=admin-metrics, description=The total number of
network operations (reads or writes) on all connections, tags={}]
(org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,793] TRACE Removed metric named MetricName
[name=network-io-rate, group=admin-metrics, description=The number of network
operations (reads or writes) on all connections per second, tags={}]
(org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,793] DEBUG Removed sensor with name bytes-sent-received:
(org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,793] TRACE Removed metric named MetricName
[name=outgoing-byte-total, group=admin-metrics, description=The total number of
outgoing bytes sent to all servers, tags={}]
(org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,793] TRACE Removed metric named MetricName
[name=outgoing-byte-rate, group=admin-metrics, description=The number of
outgoing bytes sent to all servers per second, tags={}]
(org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,793] TRACE Removed metric named MetricName
[name=request-total, group=admin-metrics, description=The total number of
requests sent, tags={}] (org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,793] TRACE Removed metric named MetricName
[name=request-rate, group=admin-metrics, description=The number of requests
sent per second, tags={}] (org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,793] TRACE Removed metric named MetricName
[name=request-size-avg, group=admin-metrics, description=The average size of
requests sent., tags={}] (org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,793] TRACE Removed metric named MetricName
[name=request-size-max, group=admin-metrics, description=The maximum size of
any request sent., tags={}] (org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,793] DEBUG Removed sensor with name bytes-sent:
(org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,793] TRACE Removed metric named MetricName
[name=incoming-byte-total, group=admin-metrics, description=The total number of
bytes read off all sockets, tags={}] (org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,793] TRACE Removed metric named MetricName
[name=incoming-byte-rate, group=admin-metrics, description=The number of bytes
read off all sockets per second, tags={}]
(org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,794] TRACE Removed metric named MetricName
[name=response-total, group=admin-metrics, description=The total number of
responses received, tags={}] (org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,794] TRACE Removed metric named MetricName
[name=response-rate, group=admin-metrics, description=The number of responses
received per second, tags={}] (org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,794] DEBUG Removed sensor with name bytes-received:
(org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,794] TRACE Removed metric named MetricName
[name=select-total, group=admin-metrics, description=The total number of times
the I/O layer checked for new I/O to perform, tags={}]
(org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,794] TRACE Removed metric named MetricName
[name=select-rate, group=admin-metrics, description=The number of times the I/O
layer checked for new I/O to perform per second, tags={}]
(org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,794] TRACE Removed metric named MetricName
[name=io-wait-time-ns-avg, group=admin-metrics, description=The average length
of time the I/O thread spent waiting for a socket ready for reads or writes in
nanoseconds., tags={}] (org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,794] TRACE Removed metric named MetricName
[name=io-waittime-total, group=admin-metrics, description=The total time the
I/O thread spent waiting, tags={}] (org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,794] TRACE Removed metric named MetricName
[name=io-wait-ratio, group=admin-metrics, description=The fraction of time the
I/O thread spent waiting, tags={}] (org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,794] DEBUG Removed sensor with name select-time:
(org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,795] TRACE Removed metric named MetricName
[name=io-time-ns-avg, group=admin-metrics, description=The average length of
time for I/O per select call in nanoseconds., tags={}]
(org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,795] TRACE Removed metric named MetricName
[name=iotime-total, group=admin-metrics, description=The total time the I/O
thread spent doing I/O, tags={}] (org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,795] TRACE Removed metric named MetricName [name=io-ratio,
group=admin-metrics, description=The fraction of time the I/O thread spent
doing I/O, tags={}] (org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,795] DEBUG Removed sensor with name io-time:
(org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,795] TRACE Removed metric named MetricName
[name=outgoing-byte-total, group=admin-node-metrics, description=The total
number of outgoing bytes, tags={node-id=node--1}]
(org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,795] TRACE Removed metric named MetricName
[name=outgoing-byte-rate, group=admin-node-metrics, description=The number of
outgoing bytes per second, tags={node-id=node--1}]
(org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,795] TRACE Removed metric named MetricName
[name=request-total, group=admin-node-metrics, description=The total number of
requests sent, tags={node-id=node--1}] (org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,795] TRACE Removed metric named MetricName
[name=request-rate, group=admin-node-metrics, description=The number of
requests sent per second, tags={node-id=node--1}]
(org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,795] TRACE Removed metric named MetricName
[name=request-size-avg, group=admin-node-metrics, description=The average size
of requests sent., tags={node-id=node--1}]
(org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,795] TRACE Removed metric named MetricName
[name=request-size-max, group=admin-node-metrics, description=The maximum size
of any request sent., tags={node-id=node--1}]
(org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,795] DEBUG Removed sensor with name node--1.bytes-sent
(org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,795] TRACE Removed metric named MetricName
[name=incoming-byte-total, group=admin-node-metrics, description=The total
number of incoming bytes, tags={node-id=node--1}]
(org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,795] TRACE Removed metric named MetricName
[name=incoming-byte-rate, group=admin-node-metrics, description=The number of
incoming bytes per second, tags={node-id=node--1}]
(org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,796] TRACE Removed metric named MetricName
[name=response-total, group=admin-node-metrics, description=The total number of
responses received, tags={node-id=node--1}]
(org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,796] TRACE Removed metric named MetricName
[name=response-rate, group=admin-node-metrics, description=The number of
responses received per second, tags={node-id=node--1}]
(org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,796] DEBUG Removed sensor with name node--1.bytes-received
(org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,796] TRACE Removed metric named MetricName
[name=request-latency-avg, group=admin-node-metrics, description=,
tags={node-id=node--1}] (org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,796] TRACE Removed metric named MetricName
[name=request-latency-max, group=admin-node-metrics, description=,
tags={node-id=node--1}] (org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,796] DEBUG Removed sensor with name node--1.latency
(org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,796] TRACE Removed metric named MetricName
[name=outgoing-byte-total, group=admin-node-metrics, description=The total
number of outgoing bytes, tags={node-id=node--3}]
(org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,796] TRACE Removed metric named MetricName
[name=outgoing-byte-rate, group=admin-node-metrics, description=The number of
outgoing bytes per second, tags={node-id=node--3}]
(org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,796] TRACE Removed metric named MetricName
[name=request-total, group=admin-node-metrics, description=The total number of
requests sent, tags={node-id=node--3}] (org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,796] TRACE Removed metric named MetricName
[name=request-rate, group=admin-node-metrics, description=The number of
requests sent per second, tags={node-id=node--3}]
(org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,796] TRACE Removed metric named MetricName
[name=request-size-avg, group=admin-node-metrics, description=The average size
of requests sent., tags={node-id=node--3}]
(org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,796] TRACE Removed metric named MetricName
[name=request-size-max, group=admin-node-metrics, description=The maximum size
of any request sent., tags={node-id=node--3}]
(org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,797] DEBUG Removed sensor with name node--3.bytes-sent
(org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,797] TRACE Removed metric named MetricName
[name=incoming-byte-total, group=admin-node-metrics, description=The total
number of incoming bytes, tags={node-id=node--3}]
(org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,797] TRACE Removed metric named MetricName
[name=incoming-byte-rate, group=admin-node-metrics, description=The number of
incoming bytes per second, tags={node-id=node--3}]
(org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,797] TRACE Removed metric named MetricName
[name=response-total, group=admin-node-metrics, description=The total number of
responses received, tags={node-id=node--3}]
(org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,797] TRACE Removed metric named MetricName
[name=response-rate, group=admin-node-metrics, description=The number of
responses received per second, tags={node-id=node--3}]
(org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,797] DEBUG Removed sensor with name node--3.bytes-received
(org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,797] TRACE Removed metric named MetricName
[name=request-latency-avg, group=admin-node-metrics, description=,
tags={node-id=node--3}] (org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,797] TRACE Removed metric named MetricName
[name=request-latency-max, group=admin-node-metrics, description=,
tags={node-id=node--3}] (org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,797] DEBUG Removed sensor with name node--3.latency
(org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,797] TRACE Removed metric named MetricName
[name=outgoing-byte-total, group=admin-node-metrics, description=The total
number of outgoing bytes, tags={node-id=node--2}]
(org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,797] TRACE Removed metric named MetricName
[name=outgoing-byte-rate, group=admin-node-metrics, description=The number of
outgoing bytes per second, tags={node-id=node--2}]
(org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,797] TRACE Removed metric named MetricName
[name=request-total, group=admin-node-metrics, description=The total number of
requests sent, tags={node-id=node--2}] (org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,798] TRACE Removed metric named MetricName
[name=request-rate, group=admin-node-metrics, description=The number of
requests sent per second, tags={node-id=node--2}]
(org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,798] TRACE Removed metric named MetricName
[name=request-size-avg, group=admin-node-metrics, description=The average size
of requests sent., tags={node-id=node--2}]
(org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,798] TRACE Removed metric named MetricName
[name=request-size-max, group=admin-node-metrics, description=The maximum size
of any request sent., tags={node-id=node--2}]
(org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,798] DEBUG Removed sensor with name node--2.bytes-sent
(org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,798] TRACE Removed metric named MetricName
[name=incoming-byte-total, group=admin-node-metrics, description=The total
number of incoming bytes, tags={node-id=node--2}]
(org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,798] TRACE Removed metric named MetricName
[name=incoming-byte-rate, group=admin-node-metrics, description=The number of
incoming bytes per second, tags={node-id=node--2}]
(org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,798] TRACE Removed metric named MetricName
[name=response-total, group=admin-node-metrics, description=The total number of
responses received, tags={node-id=node--2}]
(org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,798] TRACE Removed metric named MetricName
[name=response-rate, group=admin-node-metrics, description=The number of
responses received per second, tags={node-id=node--2}]
(org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,798] DEBUG Removed sensor with name node--2.bytes-received
(org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,798] TRACE Removed metric named MetricName
[name=request-latency-avg, group=admin-node-metrics, description=,
tags={node-id=node--2}] (org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,798] TRACE Removed metric named MetricName
[name=request-latency-max, group=admin-node-metrics, description=,
tags={node-id=node--2}] (org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,798] DEBUG Removed sensor with name node--2.latency
(org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,799] TRACE Removed metric named MetricName
[name=outgoing-byte-total, group=admin-node-metrics, description=The total
number of outgoing bytes, tags={node-id=node-1}]
(org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,799] TRACE Removed metric named MetricName
[name=outgoing-byte-rate, group=admin-node-metrics, description=The number of
outgoing bytes per second, tags={node-id=node-1}]
(org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,799] TRACE Removed metric named MetricName
[name=request-total, group=admin-node-metrics, description=The total number of
requests sent, tags={node-id=node-1}] (org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,799] TRACE Removed metric named MetricName
[name=request-rate, group=admin-node-metrics, description=The number of
requests sent per second, tags={node-id=node-1}]
(org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,799] TRACE Removed metric named MetricName
[name=request-size-avg, group=admin-node-metrics, description=The average size
of requests sent., tags={node-id=node-1}]
(org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,799] TRACE Removed metric named MetricName
[name=request-size-max, group=admin-node-metrics, description=The maximum size
of any request sent., tags={node-id=node-1}]
(org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,800] DEBUG Removed sensor with name node-1.bytes-sent
(org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,800] TRACE Removed metric named MetricName
[name=incoming-byte-total, group=admin-node-metrics, description=The total
number of incoming bytes, tags={node-id=node-1}]
(org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,800] TRACE Removed metric named MetricName
[name=incoming-byte-rate, group=admin-node-metrics, description=The number of
incoming bytes per second, tags={node-id=node-1}]
(org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,800] TRACE Removed metric named MetricName
[name=response-total, group=admin-node-metrics, description=The total number of
responses received, tags={node-id=node-1}]
(org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,807] TRACE Removed metric named MetricName
[name=response-rate, group=admin-node-metrics, description=The number of
responses received per second, tags={node-id=node-1}]
(org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,807] DEBUG Removed sensor with name node-1.bytes-received
(org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,808] TRACE Removed metric named MetricName
[name=request-latency-avg, group=admin-node-metrics, description=,
tags={node-id=node-1}] (org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,808] TRACE Removed metric named MetricName
[name=request-latency-max, group=admin-node-metrics, description=,
tags={node-id=node-1}] (org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,808] DEBUG Removed sensor with name node-1.latency
(org.apache.kafka.common.metrics.Metrics)
[2020-07-08 18:48:09,808] TRACE LoginManager(serviceName=kafka,
publicCredentials=[admin], refCount=0) released
(org.apache.kafka.common.security.authenticator.LoginManager)
Kindly check and let me know in case of any other details required.
Regards,
AnnPricksEdmund
From: Ricardo Ferreira <[email protected]>
Date: Wednesday, 8 July 2020 at 6:29 PM
To: "[email protected]" <[email protected]>, Ann Pricks <[email protected]>
Subject: Re: Consumer Groups Describe is not working
Ann,
You can try execute the CLI `kafka-consumer-groups` with TRACE enabled to dig a
little deeper in the problem. In order to do this you need to:
1. Make a copy of your `$KAFKA_HOME/etc/kafka/tools-log4j.properties` file
2. Set `root.logger=TRACE,console`
3. Run `export
KAFKA_OPTS="-Dlog4j.configuration=file:/tmp/tools-log4j.properties<file:///tmp/tools-log4j.properties>"`
before executing the `kafka-consumer-groups` CLI.
Thanks,
-- Ricardo
On 7/8/20 6:56 AM, Ann Pricks wrote:
Hi Team,
Any update on this.
Regards,
Pricks
From: Ann Pricks <[email protected]><mailto:[email protected]>
Date: Friday, 3 July 2020 at 4:10 PM
To: "[email protected]"<mailto:[email protected]>
<[email protected]><mailto:[email protected]>
Subject: Consumer Groups Describe is not working
Hi Team,
Today, In our production cluster, we faced an issue with Kafka (Old offsets was
getting pulled from spark streaming application) and couldn't debug the issue
using kafka_consumer_group.sh CLI.
Whenever we execute the below command to list the consumer groups, it is
working fine. However, whenever we try to describe the consumer group to get to
know the offset details, it didn't work (Nothing is getting displayed. Just
blank).
Command to list the consumer group (Working):
/opt/kafka/kafka_2.11-2.0.0/bin/kafka-consumer-groups.sh \
--bootstrap-server broker1:2345,broker2:2345,broker3:2345 \
--list \
--command-config /opt/kafka/kafka_2.11-2.0.0/config/jaas_config.conf
Command to list the consumer group (Not Working):
/opt/kafka/kafka_2.11-2.0.0/bin/kafka-consumer-groups.sh \
--bootstrap-server broker1:2345,broker2:2345,broker3:2345 \
--describe \
--group
spark-kafka-source-f8e218d5-16d2-4e63-a25c-2f96fabb2809-605351645-driver-0 \
--command-config /opt/kafka/kafka_2.11-2.0.0/config/jaas_config.conf
Jass Config File:
sasl.jaas.config=org.apache.kafka.common.security.scram.ScramLoginModule
required \
username="admin" \
password="123@admin";
security.protocol=SASL_PLAINTEXT
sasl.mechanism=SCRAM-SHA-512
exclude.internal.topics=false
Kindly help us to monitor our Kafka cluster in case of any issues.
Details:
Kafka Version: 2.0.0
Security:
sasl.enabled.mechanisms=SCRAM-SHA-512
sasl.mechanism.inter.broker.protocol=SCRAM-SHA-512
security.inter.broker.protocol=SASL_PLAINTEXT
Please let us know in case of any other details required from our end.
Regards,
AnnPricksEdmund
