[
https://issues.apache.org/jira/browse/ZOOKEEPER-4991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18102103#comment-18102103
]
Andor Molnar commented on ZOOKEEPER-4991:
-----------------------------------------
[~luyongchuan] , have you been able to pinpoint the bottleneck of SSL
processing in 3.9.3?
Have you tried to upgrade to latest 3.9.5 release and revalidate the issue?
> For environments of the same scale, zookeeper version 3.9.3 has SSL request
> processing blocking issues compared to version 3.8.1, which can lead to
> coredump
> ------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: ZOOKEEPER-4991
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-4991
> Project: ZooKeeper
> Issue Type: Bug
> Affects Versions: 3.9.3
> Environment: 1、Zookeeper operation log：
> roduct-MCZKService/26.455.146/rtsp/python/lib:/usr/local/lib:/usr/lib:/lib:/usr/java/packages/lib:/usr/lib64:/lib64:/lib:/usr/lib
> 2025-08-28 21:28:03.131+0800 INFO (main) (Environment:98) Server
> environment:java.io.tmpdir=/tmp
> 2025-08-28 21:28:03.131+0800 INFO (main) (Environment:98) Server
> environment:java.compiler=<NA>
> 2025-08-28 21:28:03.131+0800 INFO (main) (Environment:98) Server
> environment:os.name=Linux
> 2025-08-28 21:28:03.132+0800 INFO (main) (Environment:98) Server
> environment:os.arch=amd64
> 2025-08-28 21:28:03.132+0800 INFO (main) (Environment:98) Server
> environment:os.version=5.10.0-136.12.0.86.h2554.eulerosv2r12.x86_64
> 2025-08-28 21:28:03.132+0800 INFO (main) (Environment:98) Server
> environment:user.name=ossadm
> 2025-08-28 21:28:03.132+0800 INFO (main) (Environment:98) Server
> environment:user.home=/home/ossadm
> 2025-08-28 21:28:03.132+0800 INFO (main) (Environment:98) Server
> environment:user.dir=/opt/oss/envs/Product-MCZKService/26.455.146
> 2025-08-28 21:28:03.133+0800 INFO (main) (Environment:98) Server
> environment:os.memory.free=185MB
> 2025-08-28 21:28:03.133+0800 INFO (main) (Environment:98) Server
> environment:os.memory.max=256MB
> 2025-08-28 21:28:03.133+0800 INFO (main) (Environment:98) Server
> environment:os.memory.total=256MB
> 2025-08-28 21:28:03.133+0800 INFO (main) (ZooKeeperServer:151)
> zookeeper.enableEagerACLCheck = false
> 2025-08-28 21:28:03.133+0800 INFO (main) (ZooKeeperServer:164)
> zookeeper.digest.enabled = true
> 2025-08-28 21:28:03.134+0800 INFO (main) (ZooKeeperServer:168)
> zookeeper.closeSessionTxn.enabled = true
> 2025-08-28 21:28:03.134+0800 INFO (main) (ZooKeeperServer:2274)
> zookeeper.serializeLastProcessedZxid.enabled = true
> 2025-08-28 21:28:03.134+0800 INFO (main) (ZooKeeperServer:1587)
> zookeeper.flushDelay = 0 ms
> 2025-08-28 21:28:03.134+0800 INFO (main) (ZooKeeperServer:1596)
> zookeeper.maxWriteQueuePollTime = 0 ms
> 2025-08-28 21:28:03.135+0800 INFO (main) (ZooKeeperServer:1605)
> zookeeper.maxBatchSize=1000
> 2025-08-28 21:28:03.135+0800 INFO (main) (ZooKeeperServer:289)
> zookeeper.intBufferStartingSizeBytes = 1024
> 2025-08-28 21:28:03.149+0800 INFO (main) (WatchManagerFactory:42) Using
> org.apache.zookeeper.server.watch.WatchManager as watch manager
> 2025-08-28 21:28:03.149+0800 INFO (main) (WatchManagerFactory:42) Using
> org.apache.zookeeper.server.watch.WatchManager as watch manager
> 2025-08-28 21:28:03.150+0800 INFO (main) (ZKDatabase:134)
> zookeeper.snapshotSizeFactor = 0.33
> 2025-08-28 21:28:03.152+0800 INFO (main) (ZKDatabase:154)
> zookeeper.commitLogCount=500
> 2025-08-28 21:28:03.162+0800 INFO (main) (QuorumPeer:2103) Using TLS
> encrypted quorum communication
> 2025-08-28 21:28:03.163+0800 INFO (main) (QuorumPeer:2111) Port unification
> disabled
> 2025-08-28 21:28:03.163+0800 INFO (main) (QuorumPeer:184)
> multiAddress.enabled set to false
> 2025-08-28 21:28:03.163+0800 INFO (main) (QuorumPeer:209)
> multiAddress.reachabilityCheckEnabled set to true
> 2025-08-28 21:28:03.163+0800 INFO (main) (QuorumPeer:196)
> multiAddress.reachabilityCheckTimeoutMs set to 1000
> 2025-08-28 21:28:03.163+0800 INFO (main) (QuorumPeer:2573) QuorumPeer
> communication is not secured! (SASL auth disabled)
> 2025-08-28 21:28:03.164+0800 INFO (main) (SessionTpsLimit:45) maxSessionTps
> set to -1
> 2025-08-28 21:28:03.165+0800 INFO (main) (QuorumPeer:2598)
> quorum.cnxn.threads.size set to 20
> 2025-08-28 21:28:03.165+0800 INFO (main) (FileSnap:86) Reading snapshot
> /opt/oss/share/manager/MCZKService/data_v2/version-2/snapshot.14076a6378
> 2025-08-28 21:28:03.217+0800 INFO (main) (DataTree:1706) The digest in the
> snapshot has digest version of 2, with zxid as 0x14076a637e, and digest value
> as 12180512875172
> 2025-08-28 21:28:03.218+0800 WARN (main) (DataTree:1778) Got EOFException
> while reading the last processed zxid, likely due to reading an older
> snapshot.
> 2025-08-28 21:28:03.312+0800 INFO (main) (FileTxnSnapLog:373) 710 txns loaded
> in 42 ms
> 2025-08-28 21:28:03.313+0800 INFO (main) (ZKDatabase:291) Snapshot loaded in
> 148 ms, highest zxid is 0x14076a663e, digest is 12115086123366
> 2025-08-28 21:28:03.314+0800 INFO (main) (QuorumPeer:1161) currentEpoch not
> found! Creating with a reasonable default of 20. This should only happen when
> you are upgrading your installation
> 2025-08-28 21:28:03.317+0800 INFO (main) (QuorumPeer:1189) acceptedEpoch not
> found! Creating with a reasonable default of 20. This should only happen when
> you are upgrading your installation
> 2025-08-28 21:28:03.319+0800 INFO (main) (NettyServerCnxnFactory:733) binding
> to port /***:26310
> 2025-08-28 21:28:03.409+0800 INFO (main) (NettyServerCnxnFactory:738) bound
> to port 26310
> 2025-08-28 21:28:03.410+0800 INFO (main) (QuorumPeer:2615) Using 15000ms as
> the quorum cnxn socket timeout
> 2025-08-28 21:28:03.416+0800 INFO (main) (QuorumCnxManager$Listener:946)
> Election port bind maximum retries is 3
> 2025-08-28 21:28:03.420+0800 INFO (main) (FastLeaderElection:90)
> zookeeper.fastleader.minNotificationInterval = 200 ms
> 2025-08-28 21:28:03.420+0800 INFO (main) (FastLeaderElection:92)
> zookeeper.fastleader.maxNotificationInterval = 30000 ms
> 2025-08-28 21:28:03.421+0800 INFO (ListenerHandler-/***:26312)
> (QuorumCnxManager$Listener$ListenerHandler:1164) Creating TLS-only quorum
> server socket
> 2025-08-28 21:28:03.424+0800 INFO (main) (ZKAuditProvider:42) ZooKeeper audit
> is disabled.
> 2025-08-28 21:28:03.424+0800 INFO (ListenerHandler-/***:26312)
> (QuorumCnxManager$Listener$ListenerHandler:1093) 1 is accepting connections
> now, my election bind port: /***:26312
> 2025-08-28 21:28:03.430+0800 INFO
> (QuorumPeer[myid=1](plain=disabled)(secure=***:26310)) (QuorumPeer:1471)
> LOOKING
> 2025-08-28 21:28:03.432+0800 INFO
> (QuorumPeer[myid=1](plain=disabled)(secure=***:26310))
> (FastLeaderElection:947) New election. My id = 1, proposed zxid=0x14076a663e
> 2025-08-28 21:28:03.438+0800 INFO (WorkerReceiver[myid=1])
> (FastLeaderElection$Messenger$WorkerReceiver:391) Notification: my
> state:LOOKING; n.sid:1, n.state:LOOKING, n.leader:1, n.round:0x1,
> n.peerEpoch:0x14, n.zxid:0x14076a663e, message format v
> ersion:0x2, n.config version:0x0
> 2025-08-28 21:28:03.641+0800 INFO
> (QuorumPeer[myid=1](plain=disabled)(secure=***:26310))
> (FastLeaderElection:997) Notification time out: 400 ms
> 2025-08-28 21:28:03.722+0800 INFO (QuorumConnectionThread-[myid=1]-1)
> (QuorumCnxManager:410) SSL handshake complete with /***.191:26312 - TLSv1.3 -
> TLS_AES_128_GCM_SHA256
> 2025-08-28 21:28:03.722+0800 INFO (QuorumConnectionThread-[myid=1]-2)
> (QuorumCnxManager:410) SSL handshake complete with /***:26312 - TLSv1.3 -
> TLS_AES_128_GCM_SHA256
> 2025-08-28 21:28:03.724+0800 INFO (QuorumConnectionThread-[myid=1]-2)
> (QuorumCnxManager:536) Have smaller server identifier, so dropping the
> connection: (myId:1 --> sid:2)
> 2025-08-28 21:28:03.728+0800 INFO (WorkerReceiver[myid=1])
> (FastLeaderElection$Messenger$WorkerReceiver:391) Notification: my
> state:LOOKING; n.sid:0, n.state:LOOKING, n.leader:2, n.round:0x1,
> n.peerEpoch:0x14, n.zxid:0x14076a663e, message format v
> ersion:0x2, n.config version:0x0
> 2025-08-28 21:28:03.728+0800 INFO (ListenerHandler-/***:26312)
> (QuorumCnxManager$Listener$ListenerHandler:1098) Received connection request
> from /***:58296
> 2025-08-28 21:28:03.729+0800 INFO (WorkerReceiver[myid=1])
> (FastLeaderElection$Messenger$WorkerReceiver:391) Notification: my
> state:LOOKING; n.sid:0, n.state:FOLLOWING, n.leader:2, n.round:0x1,
> n.peerEpoch:0x15, n.zxid:0x14076a663e, message format
> version:0x2, n.config version:0x0
> 2025-08-28 21:28:03.730+0800 INFO (WorkerReceiver[myid=1])
> (FastLeaderElection$Messenger$WorkerReceiver:391) Notification: my
> state:LOOKING; n.sid:1, n.state:LOOKING, n.leader:2, n.round:0x1,
> n.peerEpoch:0x14, n.zxid:0x14076a663e, message format v
> ersion:0x2, n.config version:0x0
> 2025-08-28 21:28:03.742+0800 INFO (WorkerReceiver[myid=1])
> (FastLeaderElection$Messenger$WorkerReceiver:391) Notification: my
> state:LOOKING; n.sid:0, n.state:FOLLOWING, n.leader:2, n.round:0x1,
> n.peerEpoch:0x15, n.zxid:0x14076a663e, message format
> version:0x2, n.config version:0x0
> 2025-08-28 21:28:03.782+0800 INFO (QuorumConnectionThread-[myid=1]-3)
> (QuorumCnxManager:410) SSL handshake complete with /***:26312 - TLSv1.3 -
> TLS_AES_128_GCM_SHA256
> 2025-08-28 21:28:03.783+0800 INFO (QuorumConnectionThread-[myid=1]-3)
> (QuorumCnxManager:536) Have smaller server identifier, so dropping the
> connection: (myId:1 --> sid:2)
> 2025-08-28 21:28:03.794+0800 INFO (ListenerHandler-/***:26312)
> (UnifiedServerSocket$UnifiedSocket:266) Accepted TLS connection from
> /***:58296 - TLSv1.3 - TLS_AES_128_GCM_SHA256
> 2025-08-28 21:28:03.797+0800 INFO (WorkerReceiver[myid=1])
> (FastLeaderElection$Messenger$WorkerReceiver:391) Notification: my
> state:LOOKING; n.sid:2, n.state:LOOKING, n.leader:2, n.round:0x1,
> n.peerEpoch:0x14, n.zxid:0x14076a663e, message format v
> ersion:0x2, n.config version:0x0
> 2025-08-28 21:28:03.798+0800 INFO (WorkerReceiver[myid=1])
> (FastLeaderElection$Messenger$WorkerReceiver:391) Notification: my
> state:LOOKING; n.sid:2, n.state:LEADING, n.leader:2, n.round:0x1,
> n.peerEpoch:0x15, n.zxid:0x14076a663e, message format v
> ersion:0x2, n.config version:0x0
> 2025-08-28 21:28:03.799+0800 INFO
> (QuorumPeer[myid=1](plain=disabled)(secure=***:26310)) (QuorumPeer:906) Peer
> state changed: following
> 2025-08-28 21:28:03.800+0800 INFO
> (QuorumPeer[myid=1](plain=disabled)(secure=***:26310)) (QuorumPeer:1554)
> FOLLOWING
> 2025-08-28 21:28:03.805+0800 INFO
> (QuorumPeer[myid=1](plain=disabled)(secure=***:26310)) (Learner:133)
> leaderConnectDelayDuringRetryMs: 100
> 2025-08-28 21:28:03.805+0800 INFO
> (QuorumPeer[myid=1](plain=disabled)(secure=***:26310)) (Learner:134) TCP
> NoDelay set to: true
> 2025-08-28 21:28:03.806+0800 INFO
> (QuorumPeer[myid=1](plain=disabled)(secure=***:26310)) (Learner:135)
> zookeeper.learner.asyncSending = false
> 2025-08-28 21:28:03.806+0800 INFO
> (QuorumPeer[myid=1](plain=disabled)(secure=***:26310)) (Learner:136)
> zookeeper.learner.closeSocketAsync = false
> 2025-08-28 21:28:03.808+0800 INFO
> (QuorumPeer[myid=1](plain=disabled)(secure=***:26310)) (BlueThrottle:141)
> Weighed connection throttling is disabled
> 2025-08-28 21:28:03.808+0800 INFO
> (QuorumPeer[myid=1](plain=disabled)(secure=***:26310))
> (AuthenticationHelper:66) zookeeper.enforce.auth.enabled = false
> 2025-08-28 21:28:03.809+0800 INFO
> (QuorumPeer[myid=1](plain=disabled)(secure=***:26310))
> (AuthenticationHelper:67) zookeeper.enforce.auth.schemes = []
> 2025-08-28 21:28:03.810+0800 INFO
> (QuorumPeer[myid=1](plain=disabled)(secure=***:26310)) (ZooKeeperServer:1397)
> minSessionTimeout set to 6000 ms
> 2025-08-28 21:28:03.810+0800 INFO
> (QuorumPeer[myid=1](plain=disabled)(secure=***:26310)) (ZooKeeperServer:1406)
> maxSessionTimeout set to 60000 ms
> 2025-08-28 21:28:03.811+0800 INFO
> (QuorumPeer[myid=1](plain=disabled)(secure=***:26310)) (ResponseCache:45)
> getData response cache size is initialized with value 400.
> 2025-08-28 21:28:03.811+0800 INFO
> (QuorumPeer[myid=1](plain=disabled)(secure=***:26310)) (ResponseCache:45)
> getChildren response cache size is initialized with value 400.
> 2025-08-28 21:28:03.813+0800 INFO
> (QuorumPeer[myid=1](plain=disabled)(secure=***:26310))
> (RequestPathMetricsCollector:109) zookeeper.pathStats.slotCapacity = 60
> 2025-08-28 21:28:03.813+0800 INFO
> (QuorumPeer[myid=1](plain=disabled)(secure=***:26310))
> (RequestPathMetricsCollector:110) zookeeper.pathStats.slotDuration = 15
> 2025-08-28 21:28:03.813+0800 INFO
> (QuorumPeer[myid=1](plain=disabled)(secure=***:26310))
> (RequestPathMetricsCollector:111) zookeeper.pathStats.maxDepth = 6
> 2025-08-28 21:28:03.813+0800 INFO
> (QuorumPeer[myid=1](plain=disabled)(secure=***:26310))
> (RequestPathMetricsCollector:112) zookeeper.pathStats.initialDelay = 5
> 2025-08-28 21:28:03.813+0800 INFO
> (QuorumPeer[myid=1](plain=disabled)(secure=***:26310))
> (RequestPathMetricsCollector:113) zookeeper.pathStats.delay = 5
> 2025-08-28 21:28:03.813+0800 INFO
> (QuorumPeer[myid=1](plain=disabled)(secure=***:26310))
> (RequestPathMetricsCollector:114) zookeeper.pathStats.enabled = false
> 2025-08-28 21:28:03.814+0800 INFO
> (QuorumPeer[myid=1](plain=disabled)(secure=***:26310)) (ZooKeeperServer:1624)
> The max bytes for all large requests are set to 104857600
> 2025-08-28 21:28:03.815+0800 INFO
> (QuorumPeer[myid=1](plain=disabled)(secure=***:26310)) (ZooKeeperServer:1638)
> The large request threshold is set to -1
> 2025-08-28 21:28:03.815+0800 INFO
> (QuorumPeer[myid=1](plain=disabled)(secure=***:26310)) (ZooKeeperServer:387)
> Created server with tickTime 3000 ms minSessionTimeout 6000 ms
> maxSessionTimeout 60000 ms clientPortListenBacklog -1 dataLogdir
> /opt/oss/share/manager/MCZKService/datalog_v2/version-2 snapdir
> /opt/oss/share/manager/MCZKService/data_v2/version-2
> 2025-08-28 21:28:03.816+0800 INFO
> (QuorumPeer[myid=1](plain=disabled)(secure=***:26310)) (Follower:77)
> FOLLOWING - LEADER ELECTION TOOK - 384 MS
> 2025-08-28 21:28:03.819+0800 INFO
> (QuorumPeer[myid=1](plain=disabled)(secure=***:26310)) (QuorumPeer:920) Peer
> state changed: following - discovery
> 2025-08-28 21:28:03.874+0800 INFO (LeaderConnector-/***:26311)
> (Learner$LeaderConnector:393) Successfully connected to leader, using
> address: /***:26311
> 2025-08-28 21:28:03.886+0800 INFO
> (QuorumPeer[myid=1](plain=disabled)(secure=***:26310)) (QuorumPeer:920) Peer
> state changed: following - synchronization
> 2025-08-28 21:28:03.903+0800 INFO
> (QuorumPeer[myid=1](plain=disabled)(secure=***:26310)) (QuorumPeer:925) Peer
> state changed: following - synchronization - snap
> 2025-08-28 21:28:03.903+0800 INFO
> (QuorumPeer[myid=1](plain=disabled)(secure=***:26310)) (Learner:624) Getting
> a snapshot from leader 0x1500000000
> 2025-08-28 21:28:03.904+0800 INFO
> (QuorumPeer[myid=1](plain=disabled)(secure=***:26310))
> (WatchManagerFactory:42) Using org.apache.zookeeper.server.watch.WatchManager
> as watch manager
> 2025-08-28 21:28:03.905+0800 INFO
> (QuorumPeer[myid=1](plain=disabled)(secure=***:26310))
> (WatchManagerFactory:42) Using org.apache.zookeeper.server.watch.WatchManager
> as watch manager
> 2025-08-28 21:28:03.941+0800 INFO (zkNetty-NioEventLoopGroup-4-1)
> (NettyServerCnxnFactory$CnxnChannelHandler:269) use server side cipher suites
> order.
> 2025-08-28 21:28:03.961+0800 INFO (zkNetty-NioEventLoopGroup-4-32)
> (NettyServerCnxnFactory$CnxnChannelHandler:269) use server side cipher suites
> order.
> 2025-08-28 21:28:03.988+0800 INFO (zkNetty-NioEventLoopGroup-4-31)
> (NettyServerCnxnFactory$CnxnChannelHandler:269) use server side cipher suites
> order.
> 2025-08-28 21:28:04.005+0800 INFO
> (QuorumPeer[myid=1](plain=disabled)(secure=***:26310)) (Learner:785) Learner
> received NEWLEADER message
> 2025-08-28 21:28:04.005+0800 INFO
> (QuorumPeer[myid=1](plain=disabled)(secure=***:26310)) (QuorumPeer:1909)
> Dynamic reconfig is disabled, we don't store the last seen config.
> 2025-08-28 21:28:04.006+0800 INFO
> (QuorumPeer[myid=1](plain=disabled)(secure=***:26310)) (FileTxnSnapLog:481)
> Snapshotting: 0x1500000000 to
> /opt/oss/share/manager/MCZKService/data_v2/version-2/snapshot.1500000000
> 2025-08-28 21:28:04.016+0800 INFO (zkNetty-NioEventLoopGroup-4-30)
> (NettyServerCnxnFactory$CnxnChannelHandler:269) use server side cipher suites
> order.
> 2025-08-28 21:28:04.037+0800 INFO
> (QuorumPeer[myid=1](plain=disabled)(secure=***:26310)) (ZooKeeperServer:591)
> Snapshot taken in 32 ms
> 2025-08-28 21:28:04.038+0800 INFO
> (QuorumPeer[myid=1](plain=disabled)(secure=***:26310)) (Learner:850) Set the
> current epoch to 21
> 2025-08-28 21:28:04.038+0800 INFO
> (QuorumPeer[myid=1](plain=disabled)(secure=***:26310)) (QuorumPeer:925) Peer
> state changed: following - synchronization
> 2025-08-28 21:28:04.039+0800 INFO
> (QuorumPeer[myid=1](plain=disabled)(secure=***:26310)) (Learner:856) Sent
> NEWLEADER ack to leader with zxid 1500000000
> 2025-08-28 21:28:04.040+0800 INFO
> (QuorumPeer[myid=1](plain=disabled)(secure=***:26310)) (Learner:769) Learner
> received UPTODATE message
> 2025-08-28 21:28:04.042+0800 INFO (zkNetty-NioEventLoopGroup-4-29)
> (NettyServerCnxnFactory$CnxnChannelHandler:269) use server side cipher suites
> order.
> 2025-08-28 21:28:04.046+0800 INFO
> (QuorumPeer[myid=1](plain=disabled)(secure=***:26310)) (CommitProcessor:490)
> Configuring CommitProcessor with readBatchSize -1 commitBatchSize 1
> 2025-08-28 21:28:04.046+0800 INFO
> (QuorumPeer[myid=1](plain=disabled)(secure=***:26310)) (CommitProcessor:451)
> Configuring CommitProcessor with 16 worker threads.
> 2025-08-28 21:28:04.048+0800 INFO
> (QuorumPeer[myid=1](plain=disabled)(secure=***:26310))
> (FollowerRequestProcessor:59) Initialized FollowerRequestProcessor with
> zookeeper.follower.skipLearnerRequestToNextProcessor as false
> 2025-08-28 21:28:04.051+0800 INFO
> (QuorumPeer[myid=1](plain=disabled)(secure=***:26310)) (RequestThrottler:75)
> zookeeper.request_throttler.shutdownTimeout = 10000 ms
> 2025-08-28 21:28:04.081+0800 INFO
> (QuorumPeer[myid=1](plain=disabled)(secure=***:26310)) (Learner:879) 0 txns
> have been logged asynchronously
> 2025-08-28 21:28:04.082+0800 INFO
> (QuorumPeer[myid=1](plain=disabled)(secure=***:26310)) (Learner:884) 0 txns
> have been committed
> 2025-08-28 21:28:04.082+0800 INFO
> (QuorumPeer[myid=1](plain=disabled)(secure=***:26310)) (QuorumPeer:920) Peer
> state changed: following - broadcast
> 2025-08-28 21:28:04.087+0800 INFO (zkNetty-NioEventLoopGroup-4-28)
> (NettyServerCnxnFactory$CnxnChannelHandler:269) use server side cipher suites
> order.
> 2025-08-28 21:28:04.087+0800 INFO (zkNetty-NioEventLoopGroup-4-28)
> (NettyServerCnxnFactory$CnxnChannelHandler:269) use server side cipher suites
> order.
> 2025-08-28 21:28:04.114+0800 INFO (zkNetty-NioEventLoopGroup-4-28)
> (NettyServerCnxnFactory$CnxnChannelHandler:269) use server side cipher suites
> order.
> 2025-08-28 21:28:04.140+0800 INFO (zkNetty-NioEventLoopGroup-4-27)
> (NettyServerCnxnFactory$CnxnChannelHandler:269) use server side cipher suites
> order.
> 2025-08-28 21:28:04.165+0800 INFO (zkNetty-NioEventLoopGroup-4-26)
> (NettyServerCnxnFactory$CnxnChannelHandler:269) use server side cipher suites
> order.
> 2025-08-28 21:28:04.190+0800 INFO (zkNetty-NioEventLoopGroup-4-25)
> (NettyServerCnxnFactory$CnxnChannelHandler:269) use server side cipher suites
> order.
> 2025-08-28 21:28:04.216+0800 INFO (zkNetty-NioEventLoopGroup-4-24)
> (NettyServerCnxnFactory$CnxnChannelHandler:269) use server side cipher suites
> order.
> 2025-08-28 21:28:04.241+0800 INFO (zkNetty-NioEventLoopGroup-4-23)
> (NettyServerCnxnFactory$CnxnChannelHandler:269) use server side cipher suites
> order.
> 2025-08-28 21:28:04.272+0800 INFO (zkNetty-NioEventLoopGroup-4-22)
> (NettyServerCnxnFactory$CnxnChannelHandler:269) use server side cipher suites
> order.
> 2025-08-28 21:28:04.299+0800 INFO (zkNetty-NioEventLoopGroup-4-21)
> (NettyServerCnxnFactory$CnxnChannelHandler:269) use server side cipher suites
> order.
> 2025-08-28 21:28:04.325+0800 INFO (zkNetty-NioEventLoopGroup-4-20)
> (NettyServerCnxnFactory$CnxnChannelHandler:269) use server side cipher suites
> order.
> 2025-08-28 21:28:04.351+0800 INFO (zkNetty-NioEventLoopGroup-4-19)
> (NettyServerCnxnFactory$CnxnChannelHandler:269) use server side cipher suites
> order.
> 2025-08-28 21:28:04.375+0800 INFO (zkNetty-NioEventLoopGroup-4-18)
> (NettyServerCnxnFactory$CnxnChannelHandler:269) use server side cipher suites
> order.
> 2025-08-28 21:28:04.400+0800 INFO (zkNetty-NioEventLoopGroup-4-17)
> (NettyServerCnxnFactory$CnxnChannelHandler:269) use server side cipher suites
> order.
> 2025-08-28 21:28:04.426+0800 INFO (zkNetty-NioEventLoopGroup-4-16)
> (NettyServerCnxnFactory$CnxnChannelHandler:269) use server side cipher suites
> order.
> 2025-08-28 21:28:04.451+0800 INFO (zkNetty-NioEventLoopGroup-4-15)
> (NettyServerCnxnFactory$CnxnChannelHandler:269) use server side cipher suites
> order.
> 2025-08-28 21:28:04.477+0800 INFO (zkNetty-NioEventLoopGroup-4-14)
> (NettyServerCnxnFactory$CnxnChannelHandler:269) use server side cipher suites
> order.
> 2025-08-28 21:28:04.501+0800 INFO (zkNetty-NioEventLoopGroup-4-13)
> (NettyServerCnxnFactory$CnxnChannelHandler:269) use server side cipher suites
> order.
> 2025-08-28 21:28:52.776+0800 ERROR(zkNetty-NioEventLoopGroup-4-18)
> (NettyServerCnxnFactory$CertificateVerifier:478) Unsuccessful handshake with
> session 0x*
> 2025-08-28 21:28:52.782+0800 ERROR(zkNetty-NioEventLoopGroup-4-18)
> (NettyServerCnxnFactory$CertificateVerifier:478) Unsuccessful handshake with
> session 0x*
> 2025-08-28 21:28:52.783+0800 ERROR(zkNetty-NioEventLoopGroup-4-18)
> (NettyServerCnxnFactory$CertificateVerifier:478) Unsuccessful handshake with
> session 0x*
> 2025-08-28 21:28:52.784+0800 ERROR(zkNetty-NioEventLoopGroup-4-18)
> (NettyServerCnxnFactory$CertificateVerifier:478) Unsuccessful handshake with
> session 0x*
> 2025-08-28 21:28:52.784+0800 ERROR(zkNetty-NioEventLoopGroup-4-18)
> (NettyServerCnxnFactory$CertificateVerifier:478) Unsuccessful handshake with
> session 0x*
> 2025-08-28 21:28:52.784+0800 ERROR(zkNetty-NioEventLoopGroup-4-18)
> (NettyServerCnxnFactory$CertificateVerifier:478) Unsuccessful handshake with
> session 0x*
> 2025-08-28 21:28:52.785+0800 ERROR(zkNetty-NioEventLoopGroup-4-18)
> (NettyServerCnxnFactory$CertificateVerifier:478) Unsuccessful handshake with
> session 0x*
> 2025-08-28 21:28:52.785+0800 ERROR(zkNetty-NioEventLoopGroup-4-18)
> (NettyServerCnxnFactory$CertificateVerifier:478) Unsuccessful handshake with
> session 0x*
> 2025-08-28 21:28:52.786+0800 ERROR(zkNetty-NioEventLoopGroup-4-18)
> (NettyServerCnxnFactory$CertificateVerifier:478) Unsuccessful handshake with
> session 0x*
> 2025-08-28 21:28:52.786+0800 ERROR(zkNetty-NioEventLoopGroup-4-18)
> (NettyServerCnxnFactory$CertificateVerifier:478) Unsuccessful handshake with
> session 0x*
> 2025-08-28 21:28:52.786+0800 ERROR(zkNetty-NioEventLoopGroup-4-18)
> (NettyServerCnxnFactory$CertificateVerifier:478) Unsuccessful handshake with
> session 0x*
> 2、Leak Suspects
> 33 instances of “io.netty.channel.nio.NioEventLoop”, loaded by
> “jdk.internal.loader.ClassLoaders$AppClassLoader @ 0xf05039b8” occupy
> 176,554,128 (68.86%) bytes.
> Biggest instances:
> •io.netty.channel.nio.NioEventLoop @ 0xf0500810 - 7,194,368 (2.81%) bytes.
> •io.netty.channel.nio.NioEventLoop @ 0xf0502160 - 6,880,880 (2.68%) bytes.
> •io.netty.channel.nio.NioEventLoop @ 0xf0651698 - 6,248,152 (2.44%) bytes.
> •io.netty.channel.nio.NioEventLoop @ 0xf061acf0 - 6,204,544 (2.42%) bytes.
> •io.netty.channel.nio.NioEventLoop @ 0xf06134d8 - 5,957,280 (2.32%) bytes.
> •io.netty.channel.nio.NioEventLoop @ 0xf0500a90 - 5,918,808 (2.31%) bytes.
> •io.netty.channel.nio.NioEventLoop @ 0xf063db20 - 5,903,400 (2.30%) bytes.
> •io.netty.channel.nio.NioEventLoop @ 0xf066f048 - 5,886,720 (2.30%) bytes.
> •io.netty.channel.nio.NioEventLoop @ 0xf06537f8 - 5,771,424 (2.25%) bytes.
> •io.netty.channel.nio.NioEventLoop @ 0xf0501568 - 5,757,696 (2.25%) bytes.
> •io.netty.channel.nio.NioEventLoop @ 0xf0673140 - 5,707,144 (2.23%) bytes.
> •io.netty.channel.nio.NioEventLoop @ 0xf056ec28 - 5,698,920 (2.22%) bytes.
> •io.netty.channel.nio.NioEventLoop @ 0xf056a898 - 5,605,200 (2.19%) bytes.
> •io.netty.channel.nio.NioEventLoop @ 0xf066b888 - 5,565,080 (2.17%) bytes.
> •io.netty.channel.nio.NioEventLoop @ 0xf0651600 - 5,564,384 (2.17%) bytes.
> •io.netty.channel.nio.NioEventLoop @ 0xf056a800 - 5,433,544 (2.12%) bytes.
> •io.netty.channel.nio.NioEventLoop @ 0xf0682ca8 - 5,409,968 (2.11%) bytes.
> •io.netty.channel.nio.NioEventLoop @ 0xf05d0e88 - 5,382,984 (2.10%) bytes.
> •io.netty.channel.nio.NioEventLoop @ 0xf0613608 - 5,370,000 (2.09%) bytes.
> •io.netty.channel.nio.NioEventLoop @ 0xf056f440 - 5,320,024 (2.07%) bytes.
> •io.netty.channel.nio.NioEventLoop @ 0xf056f3a8 - 5,276,704 (2.06%) bytes.
> •io.netty.channel.nio.NioEventLoop @ 0xf06731d8 - 5,183,752 (2.02%) bytes.
> •io.netty.channel.nio.NioEventLoop @ 0xf066f7c8 - 5,118,216 (2.00%) bytes.
> •io.netty.channel.nio.NioEventLoop @ 0xf066f860 - 5,101,112 (1.99%) bytes.
> •io.netty.channel.nio.NioEventLoop @ 0xf0501c18 - 5,046,552 (1.97%) bytes.
> •io.netty.channel.nio.NioEventLoop @ 0xf0503178 - 5,033,544 (1.96%) bytes.
> •io.netty.channel.nio.NioEventLoop @ 0xf0682d40 - 4,983,992 (1.94%) bytes.
> •io.netty.channel.nio.NioEventLoop @ 0xf0613570 - 4,954,160 (1.93%) bytes.
> •io.netty.channel.nio.NioEventLoop @ 0xf066b3f0 - 4,908,112 (1.91%) bytes.
> •io.netty.channel.nio.NioEventLoop @ 0xf0683828 - 4,839,000 (1.89%) bytes.
> •io.netty.channel.nio.NioEventLoop @ 0xf0599588 - 4,823,304 (1.88%) bytes.
> •io.netty.channel.nio.NioEventLoop @ 0xf0599620 - 4,495,352 (1.75%) bytes.
> Keywords
> io.netty.channel.nio.NioEventLoop
> jdk.internal.loader.ClassLoaders$AppClassLoader @ 0xf05039b8
> Reporter: luyongchuan
> Priority: Critical
>
> The Apache ZooKeeper third-party component version 3.9.3 extensively
> refactored the SSL request processing module. After upgrading to version
> 3.9.3, large-scale environments (characterized by 200-1000 virtual machine
> nodes and a large volume of business requests) will experience a large
> accumulation and blocking of business requests, and the Zookeeper server will
> not be able to respond to requests properly.
> I once attempted to increase the stack memory from 160M to 1024M, but failed
> to solve the problem;
> After replacing the Zookeeper package on the environment with the 3.8.1
> package, the problem was resolved.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
