[jira] [Commented] (KAFKA-6522) Retrying leaderEpoch request for partition xxx as the leader reported an error: UNKNOWN_SERVER_ERROR
[
https://issues.apache.org/jira/browse/KAFKA-6522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16989975#comment-16989975
]
Cory commented on KAFKA-6522:
-
Our team had a similar issue.
# A developer had spun up a local kafka cluster using the same zookeeper path
as our dev environment.
# This caused the dev brokers to include the local broker in the cluster.
# When the local broker was removed through the zookeeper shell script, we
started to see the OP's error (UNKNOWN_SERVER_ERROR, etc.).
# We described the kafka topics (kafka-topics.sh --zookeeper --describe) and
noticed that one topic (with a single partition) had a Leader value of '-1'
# Based on this, we believe the topic leader was assigned to the now-deleted
broker
# I believe this leaves two ways of recovering
a. Delete the topic and have it recreated (we were able to use this option)
b. Reassigning the leader to a valid broker id (not sure what tool could be
used for this, maybe the partition-reassignment script?)
> Retrying leaderEpoch request for partition xxx as the leader reported an
> error: UNKNOWN_SERVER_ERROR
>
>
> Key: KAFKA-6522
> URL: https://issues.apache.org/jira/browse/KAFKA-6522
> Project: Kafka
> Issue Type: New Feature
> Components: core
>Affects Versions: 1.0.0
> Environment: Ubuntu 16.04 LTS 64bit-server
>Reporter: Wang Shuxiao
>Priority: Major
>
> we have 3 brokers in a kafka cluster(brokerid:401,402,403). The broker-403
> fails to fetch data from leader:
> {code:java}
> [2018-02-02 08:58:26,861] INFO [ReplicaFetcher replicaId=403, leaderId=401,
> fetcherId=0] Retrying leaderEpoch request for partition sub_payone1hour-0 as
> the leader reported an error: UNKNOWN_SERVER_ERROR
> (kafka.server.ReplicaFetcherThread)
> [2018-02-02 08:58:26,865] WARN [ReplicaFetcher replicaId=403, leaderId=401,
> fetcherId=3] Error when sending leader epoch request for
> Map(sub_myshardSinfo-3 -> -1, sub_myshardUinfo-1 -> -1,
> sub_videoOnlineResourceType8Test-0 -> -1, pub_videoReportEevent-1 -> 9,
> sub_StreamNofity-3 -> -1, pub_RsVideoInfo-1 -> -1, pub_lidaTopic3-15 -> -1,
> pub_lidaTopic3-3 -> -1, sub_zwbtest-1 -> -1, sub_svAdminTagging-5 -> -1,
> pub_channelinfoupdate-1 -> -1, pub_RsPlayInfo-4 -> -1, sub_tinyVideoWatch-4
> -> 14, __consumer_offsets-36 -> -1, pub_ybusAuditorChannel3-2 -> -1,
> pub_vipPush-4 -> -1, sub_LivingNotifyOnline-3 -> -1, sub_baseonline-4 -> -1,
> __consumer_offsets-24 -> -1, sub_lidaTopic-3 -> -1,
> sub_mobileGuessGameReward-0 -> -1, pub_lidaTopic-6 -> -1, sub_NewUserAlgo-0
> -> -1, __consumer_offsets-48 -> -1, pub_RsUserBehavior-3 -> -1,
> sub_channelinfoupdate-0 -> -1, pub_tinyVideoComment-1 -> -1, pub_bulletin-2
> -> -1, pub_RecordCompleteNotifition-6 -> -1, sub_lidaTopic2-3 -> -1,
> smsgateway-10 -> -1, __consumer_offsets-0 -> -1, pub_baseonlinetest-1 -> -1,
> __consumer_offsets-12 -> -1, pub_myshardUinfo-0 -> -1, pub_baseonline-3 ->
> -1, smsGatewayMarketDbInfo-6 -> -1, sub_tinyVideoComment-0 -> 14)
> (kafka.server.ReplicaFetcherThread)
> java.io.IOException: Connection to 401 was disconnected before the response
> was read
> at
> org.apache.kafka.clients.NetworkClientUtils.sendAndReceive(NetworkClientUtils.java:95)
> at
> kafka.server.ReplicaFetcherBlockingSend.sendRequest(ReplicaFetcherBlockingSend.scala:96)
> at
> kafka.server.ReplicaFetcherThread.fetchEpochsFromLeader(ReplicaFetcherThread.scala:312)
> at
> kafka.server.AbstractFetcherThread.maybeTruncate(AbstractFetcherThread.scala:130)
> at kafka.server.AbstractFetcherThread.doWork(AbstractFetcherThread.scala:102)
> at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:64){code}
>
> on the leader(broker-401) side, the log shows:
> {code:java}
> [2018-02-02 08:58:26,859] ERROR Closing socket for
> 192.168.100.101:9099-192.168.100.103:30476 because of error
> (kafka.network.Processor)
> org.apache.kafka.common.errors.InvalidRequestException: Error getting request
> for apiKey: 23 and apiVersion: 0
> Caused by: java.lang.IllegalArgumentException: Unexpected ApiKeys id `23`, it
> should be between `0` and `20` (inclusive)
> at org.apache.kafka.common.protocol.ApiKeys.forId(ApiKeys.java:73)
> at
> org.apache.kafka.common.requests.AbstractRequest.getRequest(AbstractRequest.java:39)
> at
> kafka.network.RequestChannel$Request.liftedTree2$1(RequestChannel.scala:96)
> at kafka.network.RequestChannel$Request.(RequestChannel.scala:91)
> at
> kafka.network.Processor$$anonfun$processCompletedReceives$1.apply(SocketServer.scala:492)
> at
> kafka.network.Processor$$anonfun$processCompletedReceives$1.apply(SocketServer.scala:487)
> at scala.collection.Iterator$class.foreach(Iterator.scala:893)
> at
[jira] [Commented] (KAFKA-6522) Retrying leaderEpoch request for partition xxx as the leader reported an error: UNKNOWN_SERVER_ERROR
[
https://issues.apache.org/jira/browse/KAFKA-6522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16712214#comment-16712214
]
Sravan Meruva commented on KAFKA-6522:
--
[~bookxiao], Is this issue resolved for you ? If so, could you share the
resolution?
> Retrying leaderEpoch request for partition xxx as the leader reported an
> error: UNKNOWN_SERVER_ERROR
>
>
> Key: KAFKA-6522
> URL: https://issues.apache.org/jira/browse/KAFKA-6522
> Project: Kafka
> Issue Type: New Feature
> Components: core
>Affects Versions: 1.0.0
> Environment: Ubuntu 16.04 LTS 64bit-server
>Reporter: Wang Shuxiao
>Priority: Major
>
> we have 3 brokers in a kafka cluster(brokerid:401,402,403). The broker-403
> fails to fetch data from leader:
> {code:java}
> [2018-02-02 08:58:26,861] INFO [ReplicaFetcher replicaId=403, leaderId=401,
> fetcherId=0] Retrying leaderEpoch request for partition sub_payone1hour-0 as
> the leader reported an error: UNKNOWN_SERVER_ERROR
> (kafka.server.ReplicaFetcherThread)
> [2018-02-02 08:58:26,865] WARN [ReplicaFetcher replicaId=403, leaderId=401,
> fetcherId=3] Error when sending leader epoch request for
> Map(sub_myshardSinfo-3 -> -1, sub_myshardUinfo-1 -> -1,
> sub_videoOnlineResourceType8Test-0 -> -1, pub_videoReportEevent-1 -> 9,
> sub_StreamNofity-3 -> -1, pub_RsVideoInfo-1 -> -1, pub_lidaTopic3-15 -> -1,
> pub_lidaTopic3-3 -> -1, sub_zwbtest-1 -> -1, sub_svAdminTagging-5 -> -1,
> pub_channelinfoupdate-1 -> -1, pub_RsPlayInfo-4 -> -1, sub_tinyVideoWatch-4
> -> 14, __consumer_offsets-36 -> -1, pub_ybusAuditorChannel3-2 -> -1,
> pub_vipPush-4 -> -1, sub_LivingNotifyOnline-3 -> -1, sub_baseonline-4 -> -1,
> __consumer_offsets-24 -> -1, sub_lidaTopic-3 -> -1,
> sub_mobileGuessGameReward-0 -> -1, pub_lidaTopic-6 -> -1, sub_NewUserAlgo-0
> -> -1, __consumer_offsets-48 -> -1, pub_RsUserBehavior-3 -> -1,
> sub_channelinfoupdate-0 -> -1, pub_tinyVideoComment-1 -> -1, pub_bulletin-2
> -> -1, pub_RecordCompleteNotifition-6 -> -1, sub_lidaTopic2-3 -> -1,
> smsgateway-10 -> -1, __consumer_offsets-0 -> -1, pub_baseonlinetest-1 -> -1,
> __consumer_offsets-12 -> -1, pub_myshardUinfo-0 -> -1, pub_baseonline-3 ->
> -1, smsGatewayMarketDbInfo-6 -> -1, sub_tinyVideoComment-0 -> 14)
> (kafka.server.ReplicaFetcherThread)
> java.io.IOException: Connection to 401 was disconnected before the response
> was read
> at
> org.apache.kafka.clients.NetworkClientUtils.sendAndReceive(NetworkClientUtils.java:95)
> at
> kafka.server.ReplicaFetcherBlockingSend.sendRequest(ReplicaFetcherBlockingSend.scala:96)
> at
> kafka.server.ReplicaFetcherThread.fetchEpochsFromLeader(ReplicaFetcherThread.scala:312)
> at
> kafka.server.AbstractFetcherThread.maybeTruncate(AbstractFetcherThread.scala:130)
> at kafka.server.AbstractFetcherThread.doWork(AbstractFetcherThread.scala:102)
> at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:64){code}
>
> on the leader(broker-401) side, the log shows:
> {code:java}
> [2018-02-02 08:58:26,859] ERROR Closing socket for
> 192.168.100.101:9099-192.168.100.103:30476 because of error
> (kafka.network.Processor)
> org.apache.kafka.common.errors.InvalidRequestException: Error getting request
> for apiKey: 23 and apiVersion: 0
> Caused by: java.lang.IllegalArgumentException: Unexpected ApiKeys id `23`, it
> should be between `0` and `20` (inclusive)
> at org.apache.kafka.common.protocol.ApiKeys.forId(ApiKeys.java:73)
> at
> org.apache.kafka.common.requests.AbstractRequest.getRequest(AbstractRequest.java:39)
> at
> kafka.network.RequestChannel$Request.liftedTree2$1(RequestChannel.scala:96)
> at kafka.network.RequestChannel$Request.(RequestChannel.scala:91)
> at
> kafka.network.Processor$$anonfun$processCompletedReceives$1.apply(SocketServer.scala:492)
> at
> kafka.network.Processor$$anonfun$processCompletedReceives$1.apply(SocketServer.scala:487)
> at scala.collection.Iterator$class.foreach(Iterator.scala:893)
> at scala.collection.AbstractIterator.foreach(Iterator.scala:1336)
> at scala.collection.IterableLike$class.foreach(IterableLike.scala:72)
> at scala.collection.AbstractIterable.foreach(Iterable.scala:54)
> at kafka.network.Processor.processCompletedReceives(SocketServer.scala:487)
> at kafka.network.Processor.run(SocketServer.scala:417)
> at java.lang.Thread.run(Thread.java:745){code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KAFKA-6522) Retrying leaderEpoch request for partition xxx as the leader reported an error: UNKNOWN_SERVER_ERROR
[
https://issues.apache.org/jira/browse/KAFKA-6522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16712213#comment-16712213
]
Sravan Meruva commented on KAFKA-6522:
--
[~huxi_2b], We are facing a smiliar problem. we tried the following command
bin/kafka-broker-api-versions.sh --bootstrap-server ***
we ended up with the following error: hostname (id: 1 rack: null) -> *ERROR:
org.apache.kafka.common.errors.DisconnectException*
We restarted the whole cluster, we even deleted the data directory, this error
still prevails making the broker inoperable.
> Retrying leaderEpoch request for partition xxx as the leader reported an
> error: UNKNOWN_SERVER_ERROR
>
>
> Key: KAFKA-6522
> URL: https://issues.apache.org/jira/browse/KAFKA-6522
> Project: Kafka
> Issue Type: New Feature
> Components: core
>Affects Versions: 1.0.0
> Environment: Ubuntu 16.04 LTS 64bit-server
>Reporter: Wang Shuxiao
>Priority: Major
>
> we have 3 brokers in a kafka cluster(brokerid:401,402,403). The broker-403
> fails to fetch data from leader:
> {code:java}
> [2018-02-02 08:58:26,861] INFO [ReplicaFetcher replicaId=403, leaderId=401,
> fetcherId=0] Retrying leaderEpoch request for partition sub_payone1hour-0 as
> the leader reported an error: UNKNOWN_SERVER_ERROR
> (kafka.server.ReplicaFetcherThread)
> [2018-02-02 08:58:26,865] WARN [ReplicaFetcher replicaId=403, leaderId=401,
> fetcherId=3] Error when sending leader epoch request for
> Map(sub_myshardSinfo-3 -> -1, sub_myshardUinfo-1 -> -1,
> sub_videoOnlineResourceType8Test-0 -> -1, pub_videoReportEevent-1 -> 9,
> sub_StreamNofity-3 -> -1, pub_RsVideoInfo-1 -> -1, pub_lidaTopic3-15 -> -1,
> pub_lidaTopic3-3 -> -1, sub_zwbtest-1 -> -1, sub_svAdminTagging-5 -> -1,
> pub_channelinfoupdate-1 -> -1, pub_RsPlayInfo-4 -> -1, sub_tinyVideoWatch-4
> -> 14, __consumer_offsets-36 -> -1, pub_ybusAuditorChannel3-2 -> -1,
> pub_vipPush-4 -> -1, sub_LivingNotifyOnline-3 -> -1, sub_baseonline-4 -> -1,
> __consumer_offsets-24 -> -1, sub_lidaTopic-3 -> -1,
> sub_mobileGuessGameReward-0 -> -1, pub_lidaTopic-6 -> -1, sub_NewUserAlgo-0
> -> -1, __consumer_offsets-48 -> -1, pub_RsUserBehavior-3 -> -1,
> sub_channelinfoupdate-0 -> -1, pub_tinyVideoComment-1 -> -1, pub_bulletin-2
> -> -1, pub_RecordCompleteNotifition-6 -> -1, sub_lidaTopic2-3 -> -1,
> smsgateway-10 -> -1, __consumer_offsets-0 -> -1, pub_baseonlinetest-1 -> -1,
> __consumer_offsets-12 -> -1, pub_myshardUinfo-0 -> -1, pub_baseonline-3 ->
> -1, smsGatewayMarketDbInfo-6 -> -1, sub_tinyVideoComment-0 -> 14)
> (kafka.server.ReplicaFetcherThread)
> java.io.IOException: Connection to 401 was disconnected before the response
> was read
> at
> org.apache.kafka.clients.NetworkClientUtils.sendAndReceive(NetworkClientUtils.java:95)
> at
> kafka.server.ReplicaFetcherBlockingSend.sendRequest(ReplicaFetcherBlockingSend.scala:96)
> at
> kafka.server.ReplicaFetcherThread.fetchEpochsFromLeader(ReplicaFetcherThread.scala:312)
> at
> kafka.server.AbstractFetcherThread.maybeTruncate(AbstractFetcherThread.scala:130)
> at kafka.server.AbstractFetcherThread.doWork(AbstractFetcherThread.scala:102)
> at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:64){code}
>
> on the leader(broker-401) side, the log shows:
> {code:java}
> [2018-02-02 08:58:26,859] ERROR Closing socket for
> 192.168.100.101:9099-192.168.100.103:30476 because of error
> (kafka.network.Processor)
> org.apache.kafka.common.errors.InvalidRequestException: Error getting request
> for apiKey: 23 and apiVersion: 0
> Caused by: java.lang.IllegalArgumentException: Unexpected ApiKeys id `23`, it
> should be between `0` and `20` (inclusive)
> at org.apache.kafka.common.protocol.ApiKeys.forId(ApiKeys.java:73)
> at
> org.apache.kafka.common.requests.AbstractRequest.getRequest(AbstractRequest.java:39)
> at
> kafka.network.RequestChannel$Request.liftedTree2$1(RequestChannel.scala:96)
> at kafka.network.RequestChannel$Request.(RequestChannel.scala:91)
> at
> kafka.network.Processor$$anonfun$processCompletedReceives$1.apply(SocketServer.scala:492)
> at
> kafka.network.Processor$$anonfun$processCompletedReceives$1.apply(SocketServer.scala:487)
> at scala.collection.Iterator$class.foreach(Iterator.scala:893)
> at scala.collection.AbstractIterator.foreach(Iterator.scala:1336)
> at scala.collection.IterableLike$class.foreach(IterableLike.scala:72)
> at scala.collection.AbstractIterable.foreach(Iterable.scala:54)
> at kafka.network.Processor.processCompletedReceives(SocketServer.scala:487)
> at kafka.network.Processor.run(SocketServer.scala:417)
> at java.lang.Thread.run(Thread.java:745){code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KAFKA-6522) Retrying leaderEpoch request for partition xxx as the leader reported an error: UNKNOWN_SERVER_ERROR
[
https://issues.apache.org/jira/browse/KAFKA-6522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16677302#comment-16677302
]
Jaume M commented on KAFKA-6522:
I believe this is happening to me because there were some kafa nodes with the
same zookeeper cluster that I wasn't aware of. Which would make sense with what
was said two comments before this one.
> Retrying leaderEpoch request for partition xxx as the leader reported an
> error: UNKNOWN_SERVER_ERROR
>
>
> Key: KAFKA-6522
> URL: https://issues.apache.org/jira/browse/KAFKA-6522
> Project: Kafka
> Issue Type: New Feature
> Components: core
>Affects Versions: 1.0.0
> Environment: Ubuntu 16.04 LTS 64bit-server
>Reporter: Wang Shuxiao
>Priority: Major
>
> we have 3 brokers in a kafka cluster(brokerid:401,402,403). The broker-403
> fails to fetch data from leader:
> {code:java}
> [2018-02-02 08:58:26,861] INFO [ReplicaFetcher replicaId=403, leaderId=401,
> fetcherId=0] Retrying leaderEpoch request for partition sub_payone1hour-0 as
> the leader reported an error: UNKNOWN_SERVER_ERROR
> (kafka.server.ReplicaFetcherThread)
> [2018-02-02 08:58:26,865] WARN [ReplicaFetcher replicaId=403, leaderId=401,
> fetcherId=3] Error when sending leader epoch request for
> Map(sub_myshardSinfo-3 -> -1, sub_myshardUinfo-1 -> -1,
> sub_videoOnlineResourceType8Test-0 -> -1, pub_videoReportEevent-1 -> 9,
> sub_StreamNofity-3 -> -1, pub_RsVideoInfo-1 -> -1, pub_lidaTopic3-15 -> -1,
> pub_lidaTopic3-3 -> -1, sub_zwbtest-1 -> -1, sub_svAdminTagging-5 -> -1,
> pub_channelinfoupdate-1 -> -1, pub_RsPlayInfo-4 -> -1, sub_tinyVideoWatch-4
> -> 14, __consumer_offsets-36 -> -1, pub_ybusAuditorChannel3-2 -> -1,
> pub_vipPush-4 -> -1, sub_LivingNotifyOnline-3 -> -1, sub_baseonline-4 -> -1,
> __consumer_offsets-24 -> -1, sub_lidaTopic-3 -> -1,
> sub_mobileGuessGameReward-0 -> -1, pub_lidaTopic-6 -> -1, sub_NewUserAlgo-0
> -> -1, __consumer_offsets-48 -> -1, pub_RsUserBehavior-3 -> -1,
> sub_channelinfoupdate-0 -> -1, pub_tinyVideoComment-1 -> -1, pub_bulletin-2
> -> -1, pub_RecordCompleteNotifition-6 -> -1, sub_lidaTopic2-3 -> -1,
> smsgateway-10 -> -1, __consumer_offsets-0 -> -1, pub_baseonlinetest-1 -> -1,
> __consumer_offsets-12 -> -1, pub_myshardUinfo-0 -> -1, pub_baseonline-3 ->
> -1, smsGatewayMarketDbInfo-6 -> -1, sub_tinyVideoComment-0 -> 14)
> (kafka.server.ReplicaFetcherThread)
> java.io.IOException: Connection to 401 was disconnected before the response
> was read
> at
> org.apache.kafka.clients.NetworkClientUtils.sendAndReceive(NetworkClientUtils.java:95)
> at
> kafka.server.ReplicaFetcherBlockingSend.sendRequest(ReplicaFetcherBlockingSend.scala:96)
> at
> kafka.server.ReplicaFetcherThread.fetchEpochsFromLeader(ReplicaFetcherThread.scala:312)
> at
> kafka.server.AbstractFetcherThread.maybeTruncate(AbstractFetcherThread.scala:130)
> at kafka.server.AbstractFetcherThread.doWork(AbstractFetcherThread.scala:102)
> at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:64){code}
>
> on the leader(broker-401) side, the log shows:
> {code:java}
> [2018-02-02 08:58:26,859] ERROR Closing socket for
> 192.168.100.101:9099-192.168.100.103:30476 because of error
> (kafka.network.Processor)
> org.apache.kafka.common.errors.InvalidRequestException: Error getting request
> for apiKey: 23 and apiVersion: 0
> Caused by: java.lang.IllegalArgumentException: Unexpected ApiKeys id `23`, it
> should be between `0` and `20` (inclusive)
> at org.apache.kafka.common.protocol.ApiKeys.forId(ApiKeys.java:73)
> at
> org.apache.kafka.common.requests.AbstractRequest.getRequest(AbstractRequest.java:39)
> at
> kafka.network.RequestChannel$Request.liftedTree2$1(RequestChannel.scala:96)
> at kafka.network.RequestChannel$Request.(RequestChannel.scala:91)
> at
> kafka.network.Processor$$anonfun$processCompletedReceives$1.apply(SocketServer.scala:492)
> at
> kafka.network.Processor$$anonfun$processCompletedReceives$1.apply(SocketServer.scala:487)
> at scala.collection.Iterator$class.foreach(Iterator.scala:893)
> at scala.collection.AbstractIterator.foreach(Iterator.scala:1336)
> at scala.collection.IterableLike$class.foreach(IterableLike.scala:72)
> at scala.collection.AbstractIterable.foreach(Iterable.scala:54)
> at kafka.network.Processor.processCompletedReceives(SocketServer.scala:487)
> at kafka.network.Processor.run(SocketServer.scala:417)
> at java.lang.Thread.run(Thread.java:745){code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KAFKA-6522) Retrying leaderEpoch request for partition xxx as the leader reported an error: UNKNOWN_SERVER_ERROR
[
https://issues.apache.org/jira/browse/KAFKA-6522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16677217#comment-16677217
]
Jaume M commented on KAFKA-6522:
Same happens to me by the end of this tutorial if I follow it without any
deviation: [https://data-flair.training/blogs/kafka-cluster/] . It goes away
when I restart the nodes.
> Retrying leaderEpoch request for partition xxx as the leader reported an
> error: UNKNOWN_SERVER_ERROR
>
>
> Key: KAFKA-6522
> URL: https://issues.apache.org/jira/browse/KAFKA-6522
> Project: Kafka
> Issue Type: New Feature
> Components: core
>Affects Versions: 1.0.0
> Environment: Ubuntu 16.04 LTS 64bit-server
>Reporter: Wang Shuxiao
>Priority: Major
>
> we have 3 brokers in a kafka cluster(brokerid:401,402,403). The broker-403
> fails to fetch data from leader:
> {code:java}
> [2018-02-02 08:58:26,861] INFO [ReplicaFetcher replicaId=403, leaderId=401,
> fetcherId=0] Retrying leaderEpoch request for partition sub_payone1hour-0 as
> the leader reported an error: UNKNOWN_SERVER_ERROR
> (kafka.server.ReplicaFetcherThread)
> [2018-02-02 08:58:26,865] WARN [ReplicaFetcher replicaId=403, leaderId=401,
> fetcherId=3] Error when sending leader epoch request for
> Map(sub_myshardSinfo-3 -> -1, sub_myshardUinfo-1 -> -1,
> sub_videoOnlineResourceType8Test-0 -> -1, pub_videoReportEevent-1 -> 9,
> sub_StreamNofity-3 -> -1, pub_RsVideoInfo-1 -> -1, pub_lidaTopic3-15 -> -1,
> pub_lidaTopic3-3 -> -1, sub_zwbtest-1 -> -1, sub_svAdminTagging-5 -> -1,
> pub_channelinfoupdate-1 -> -1, pub_RsPlayInfo-4 -> -1, sub_tinyVideoWatch-4
> -> 14, __consumer_offsets-36 -> -1, pub_ybusAuditorChannel3-2 -> -1,
> pub_vipPush-4 -> -1, sub_LivingNotifyOnline-3 -> -1, sub_baseonline-4 -> -1,
> __consumer_offsets-24 -> -1, sub_lidaTopic-3 -> -1,
> sub_mobileGuessGameReward-0 -> -1, pub_lidaTopic-6 -> -1, sub_NewUserAlgo-0
> -> -1, __consumer_offsets-48 -> -1, pub_RsUserBehavior-3 -> -1,
> sub_channelinfoupdate-0 -> -1, pub_tinyVideoComment-1 -> -1, pub_bulletin-2
> -> -1, pub_RecordCompleteNotifition-6 -> -1, sub_lidaTopic2-3 -> -1,
> smsgateway-10 -> -1, __consumer_offsets-0 -> -1, pub_baseonlinetest-1 -> -1,
> __consumer_offsets-12 -> -1, pub_myshardUinfo-0 -> -1, pub_baseonline-3 ->
> -1, smsGatewayMarketDbInfo-6 -> -1, sub_tinyVideoComment-0 -> 14)
> (kafka.server.ReplicaFetcherThread)
> java.io.IOException: Connection to 401 was disconnected before the response
> was read
> at
> org.apache.kafka.clients.NetworkClientUtils.sendAndReceive(NetworkClientUtils.java:95)
> at
> kafka.server.ReplicaFetcherBlockingSend.sendRequest(ReplicaFetcherBlockingSend.scala:96)
> at
> kafka.server.ReplicaFetcherThread.fetchEpochsFromLeader(ReplicaFetcherThread.scala:312)
> at
> kafka.server.AbstractFetcherThread.maybeTruncate(AbstractFetcherThread.scala:130)
> at kafka.server.AbstractFetcherThread.doWork(AbstractFetcherThread.scala:102)
> at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:64){code}
>
> on the leader(broker-401) side, the log shows:
> {code:java}
> [2018-02-02 08:58:26,859] ERROR Closing socket for
> 192.168.100.101:9099-192.168.100.103:30476 because of error
> (kafka.network.Processor)
> org.apache.kafka.common.errors.InvalidRequestException: Error getting request
> for apiKey: 23 and apiVersion: 0
> Caused by: java.lang.IllegalArgumentException: Unexpected ApiKeys id `23`, it
> should be between `0` and `20` (inclusive)
> at org.apache.kafka.common.protocol.ApiKeys.forId(ApiKeys.java:73)
> at
> org.apache.kafka.common.requests.AbstractRequest.getRequest(AbstractRequest.java:39)
> at
> kafka.network.RequestChannel$Request.liftedTree2$1(RequestChannel.scala:96)
> at kafka.network.RequestChannel$Request.(RequestChannel.scala:91)
> at
> kafka.network.Processor$$anonfun$processCompletedReceives$1.apply(SocketServer.scala:492)
> at
> kafka.network.Processor$$anonfun$processCompletedReceives$1.apply(SocketServer.scala:487)
> at scala.collection.Iterator$class.foreach(Iterator.scala:893)
> at scala.collection.AbstractIterator.foreach(Iterator.scala:1336)
> at scala.collection.IterableLike$class.foreach(IterableLike.scala:72)
> at scala.collection.AbstractIterable.foreach(Iterable.scala:54)
> at kafka.network.Processor.processCompletedReceives(SocketServer.scala:487)
> at kafka.network.Processor.run(SocketServer.scala:417)
> at java.lang.Thread.run(Thread.java:745){code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KAFKA-6522) Retrying leaderEpoch request for partition xxx as the leader reported an error: UNKNOWN_SERVER_ERROR
[
https://issues.apache.org/jira/browse/KAFKA-6522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16355229#comment-16355229
]
huxihx commented on KAFKA-6522:
---
[~bookxiao] hmmm Based on the stack trace, ApiKeys [Line
#73|https://github.com/apache/kafka/blob/f20e4b72d3f5af4539a8c280efcf51b92d6a06af/clients/src/main/java/org/apache/kafka/common/protocol/ApiKeys.java#L73]
threw the IllegalArgumentException, that proved it should be from 0.10.2.x
codebase. Try to run command below to see if how many apis broker 401 supported:
{code:java}
bin/kafka-broker-api-versions.sh --bootstrap-server ***{code}
> Retrying leaderEpoch request for partition xxx as the leader reported an
> error: UNKNOWN_SERVER_ERROR
>
>
> Key: KAFKA-6522
> URL: https://issues.apache.org/jira/browse/KAFKA-6522
> Project: Kafka
> Issue Type: New Feature
> Components: core
>Affects Versions: 1.0.0
> Environment: Ubuntu 16.04 LTS 64bit-server
>Reporter: Wang Shuxiao
>Priority: Major
>
> we have 3 brokers in a kafka cluster(brokerid:401,402,403). The broker-403
> fails to fetch data from leader:
> {code:java}
> [2018-02-02 08:58:26,861] INFO [ReplicaFetcher replicaId=403, leaderId=401,
> fetcherId=0] Retrying leaderEpoch request for partition sub_payone1hour-0 as
> the leader reported an error: UNKNOWN_SERVER_ERROR
> (kafka.server.ReplicaFetcherThread)
> [2018-02-02 08:58:26,865] WARN [ReplicaFetcher replicaId=403, leaderId=401,
> fetcherId=3] Error when sending leader epoch request for
> Map(sub_myshardSinfo-3 -> -1, sub_myshardUinfo-1 -> -1,
> sub_videoOnlineResourceType8Test-0 -> -1, pub_videoReportEevent-1 -> 9,
> sub_StreamNofity-3 -> -1, pub_RsVideoInfo-1 -> -1, pub_lidaTopic3-15 -> -1,
> pub_lidaTopic3-3 -> -1, sub_zwbtest-1 -> -1, sub_svAdminTagging-5 -> -1,
> pub_channelinfoupdate-1 -> -1, pub_RsPlayInfo-4 -> -1, sub_tinyVideoWatch-4
> -> 14, __consumer_offsets-36 -> -1, pub_ybusAuditorChannel3-2 -> -1,
> pub_vipPush-4 -> -1, sub_LivingNotifyOnline-3 -> -1, sub_baseonline-4 -> -1,
> __consumer_offsets-24 -> -1, sub_lidaTopic-3 -> -1,
> sub_mobileGuessGameReward-0 -> -1, pub_lidaTopic-6 -> -1, sub_NewUserAlgo-0
> -> -1, __consumer_offsets-48 -> -1, pub_RsUserBehavior-3 -> -1,
> sub_channelinfoupdate-0 -> -1, pub_tinyVideoComment-1 -> -1, pub_bulletin-2
> -> -1, pub_RecordCompleteNotifition-6 -> -1, sub_lidaTopic2-3 -> -1,
> smsgateway-10 -> -1, __consumer_offsets-0 -> -1, pub_baseonlinetest-1 -> -1,
> __consumer_offsets-12 -> -1, pub_myshardUinfo-0 -> -1, pub_baseonline-3 ->
> -1, smsGatewayMarketDbInfo-6 -> -1, sub_tinyVideoComment-0 -> 14)
> (kafka.server.ReplicaFetcherThread)
> java.io.IOException: Connection to 401 was disconnected before the response
> was read
> at
> org.apache.kafka.clients.NetworkClientUtils.sendAndReceive(NetworkClientUtils.java:95)
> at
> kafka.server.ReplicaFetcherBlockingSend.sendRequest(ReplicaFetcherBlockingSend.scala:96)
> at
> kafka.server.ReplicaFetcherThread.fetchEpochsFromLeader(ReplicaFetcherThread.scala:312)
> at
> kafka.server.AbstractFetcherThread.maybeTruncate(AbstractFetcherThread.scala:130)
> at kafka.server.AbstractFetcherThread.doWork(AbstractFetcherThread.scala:102)
> at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:64){code}
>
> on the leader(broker-401) side, the log shows:
> {code:java}
> [2018-02-02 08:58:26,859] ERROR Closing socket for
> 192.168.100.101:9099-192.168.100.103:30476 because of error
> (kafka.network.Processor)
> org.apache.kafka.common.errors.InvalidRequestException: Error getting request
> for apiKey: 23 and apiVersion: 0
> Caused by: java.lang.IllegalArgumentException: Unexpected ApiKeys id `23`, it
> should be between `0` and `20` (inclusive)
> at org.apache.kafka.common.protocol.ApiKeys.forId(ApiKeys.java:73)
> at
> org.apache.kafka.common.requests.AbstractRequest.getRequest(AbstractRequest.java:39)
> at
> kafka.network.RequestChannel$Request.liftedTree2$1(RequestChannel.scala:96)
> at kafka.network.RequestChannel$Request.(RequestChannel.scala:91)
> at
> kafka.network.Processor$$anonfun$processCompletedReceives$1.apply(SocketServer.scala:492)
> at
> kafka.network.Processor$$anonfun$processCompletedReceives$1.apply(SocketServer.scala:487)
> at scala.collection.Iterator$class.foreach(Iterator.scala:893)
> at scala.collection.AbstractIterator.foreach(Iterator.scala:1336)
> at scala.collection.IterableLike$class.foreach(IterableLike.scala:72)
> at scala.collection.AbstractIterable.foreach(Iterable.scala:54)
> at kafka.network.Processor.processCompletedReceives(SocketServer.scala:487)
> at kafka.network.Processor.run(SocketServer.scala:417)
> at java.lang.Thread.run(Thread.java:745){code}
--
This message was sent by Atlassian JIRA
[jira] [Commented] (KAFKA-6522) Retrying leaderEpoch request for partition xxx as the leader reported an error: UNKNOWN_SERVER_ERROR
[
https://issues.apache.org/jira/browse/KAFKA-6522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16350123#comment-16350123
]
Wang Shuxiao commented on KAFKA-6522:
-
all brokers use the same binary file (kafka_2.11-1.0.0.tgz). And it recoveries
after restarting the broker-403
> Retrying leaderEpoch request for partition xxx as the leader reported an
> error: UNKNOWN_SERVER_ERROR
>
>
> Key: KAFKA-6522
> URL: https://issues.apache.org/jira/browse/KAFKA-6522
> Project: Kafka
> Issue Type: New Feature
> Components: core
>Affects Versions: 1.0.0
> Environment: Ubuntu 16.04 LTS 64bit-server
>Reporter: Wang Shuxiao
>Priority: Major
>
> we have 3 brokers in a kafka cluster(brokerid:401,402,403). The broker-403
> fails to fetch data from leader:
> {code:java}
> [2018-02-02 08:58:26,861] INFO [ReplicaFetcher replicaId=403, leaderId=401,
> fetcherId=0] Retrying leaderEpoch request for partition sub_payone1hour-0 as
> the leader reported an error: UNKNOWN_SERVER_ERROR
> (kafka.server.ReplicaFetcherThread)
> [2018-02-02 08:58:26,865] WARN [ReplicaFetcher replicaId=403, leaderId=401,
> fetcherId=3] Error when sending leader epoch request for
> Map(sub_myshardSinfo-3 -> -1, sub_myshardUinfo-1 -> -1,
> sub_videoOnlineResourceType8Test-0 -> -1, pub_videoReportEevent-1 -> 9,
> sub_StreamNofity-3 -> -1, pub_RsVideoInfo-1 -> -1, pub_lidaTopic3-15 -> -1,
> pub_lidaTopic3-3 -> -1, sub_zwbtest-1 -> -1, sub_svAdminTagging-5 -> -1,
> pub_channelinfoupdate-1 -> -1, pub_RsPlayInfo-4 -> -1, sub_tinyVideoWatch-4
> -> 14, __consumer_offsets-36 -> -1, pub_ybusAuditorChannel3-2 -> -1,
> pub_vipPush-4 -> -1, sub_LivingNotifyOnline-3 -> -1, sub_baseonline-4 -> -1,
> __consumer_offsets-24 -> -1, sub_lidaTopic-3 -> -1,
> sub_mobileGuessGameReward-0 -> -1, pub_lidaTopic-6 -> -1, sub_NewUserAlgo-0
> -> -1, __consumer_offsets-48 -> -1, pub_RsUserBehavior-3 -> -1,
> sub_channelinfoupdate-0 -> -1, pub_tinyVideoComment-1 -> -1, pub_bulletin-2
> -> -1, pub_RecordCompleteNotifition-6 -> -1, sub_lidaTopic2-3 -> -1,
> smsgateway-10 -> -1, __consumer_offsets-0 -> -1, pub_baseonlinetest-1 -> -1,
> __consumer_offsets-12 -> -1, pub_myshardUinfo-0 -> -1, pub_baseonline-3 ->
> -1, smsGatewayMarketDbInfo-6 -> -1, sub_tinyVideoComment-0 -> 14)
> (kafka.server.ReplicaFetcherThread)
> java.io.IOException: Connection to 401 was disconnected before the response
> was read
> at
> org.apache.kafka.clients.NetworkClientUtils.sendAndReceive(NetworkClientUtils.java:95)
> at
> kafka.server.ReplicaFetcherBlockingSend.sendRequest(ReplicaFetcherBlockingSend.scala:96)
> at
> kafka.server.ReplicaFetcherThread.fetchEpochsFromLeader(ReplicaFetcherThread.scala:312)
> at
> kafka.server.AbstractFetcherThread.maybeTruncate(AbstractFetcherThread.scala:130)
> at kafka.server.AbstractFetcherThread.doWork(AbstractFetcherThread.scala:102)
> at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:64){code}
>
> on the leader(broker-401) side, the log shows:
> {code:java}
> [2018-02-02 08:58:26,859] ERROR Closing socket for
> 192.168.100.101:9099-192.168.100.103:30476 because of error
> (kafka.network.Processor)
> org.apache.kafka.common.errors.InvalidRequestException: Error getting request
> for apiKey: 23 and apiVersion: 0
> Caused by: java.lang.IllegalArgumentException: Unexpected ApiKeys id `23`, it
> should be between `0` and `20` (inclusive)
> at org.apache.kafka.common.protocol.ApiKeys.forId(ApiKeys.java:73)
> at
> org.apache.kafka.common.requests.AbstractRequest.getRequest(AbstractRequest.java:39)
> at
> kafka.network.RequestChannel$Request.liftedTree2$1(RequestChannel.scala:96)
> at kafka.network.RequestChannel$Request.(RequestChannel.scala:91)
> at
> kafka.network.Processor$$anonfun$processCompletedReceives$1.apply(SocketServer.scala:492)
> at
> kafka.network.Processor$$anonfun$processCompletedReceives$1.apply(SocketServer.scala:487)
> at scala.collection.Iterator$class.foreach(Iterator.scala:893)
> at scala.collection.AbstractIterator.foreach(Iterator.scala:1336)
> at scala.collection.IterableLike$class.foreach(IterableLike.scala:72)
> at scala.collection.AbstractIterable.foreach(Iterable.scala:54)
> at kafka.network.Processor.processCompletedReceives(SocketServer.scala:487)
> at kafka.network.Processor.run(SocketServer.scala:417)
> at java.lang.Thread.run(Thread.java:745){code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KAFKA-6522) Retrying leaderEpoch request for partition xxx as the leader reported an error: UNKNOWN_SERVER_ERROR
[
https://issues.apache.org/jira/browse/KAFKA-6522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16350061#comment-16350061
]
huxihx commented on KAFKA-6522:
---
Is it possible that one broker is actually 0.10.2.x?
> Retrying leaderEpoch request for partition xxx as the leader reported an
> error: UNKNOWN_SERVER_ERROR
>
>
> Key: KAFKA-6522
> URL: https://issues.apache.org/jira/browse/KAFKA-6522
> Project: Kafka
> Issue Type: New Feature
> Components: core
>Affects Versions: 1.0.0
> Environment: Ubuntu 16.04 LTS 64bit-server
>Reporter: Wang Shuxiao
>Priority: Major
>
> we have 3 brokers in a kafka cluster(brokerid:401,402,403). The broker-403
> fails to fetch data from leader:
> {code:java}
> [2018-02-02 08:58:26,861] INFO [ReplicaFetcher replicaId=403, leaderId=401,
> fetcherId=0] Retrying leaderEpoch request for partition sub_payone1hour-0 as
> the leader reported an error: UNKNOWN_SERVER_ERROR
> (kafka.server.ReplicaFetcherThread)
> [2018-02-02 08:58:26,865] WARN [ReplicaFetcher replicaId=403, leaderId=401,
> fetcherId=3] Error when sending leader epoch request for
> Map(sub_myshardSinfo-3 -> -1, sub_myshardUinfo-1 -> -1,
> sub_videoOnlineResourceType8Test-0 -> -1, pub_videoReportEevent-1 -> 9,
> sub_StreamNofity-3 -> -1, pub_RsVideoInfo-1 -> -1, pub_lidaTopic3-15 -> -1,
> pub_lidaTopic3-3 -> -1, sub_zwbtest-1 -> -1, sub_svAdminTagging-5 -> -1,
> pub_channelinfoupdate-1 -> -1, pub_RsPlayInfo-4 -> -1, sub_tinyVideoWatch-4
> -> 14, __consumer_offsets-36 -> -1, pub_ybusAuditorChannel3-2 -> -1,
> pub_vipPush-4 -> -1, sub_LivingNotifyOnline-3 -> -1, sub_baseonline-4 -> -1,
> __consumer_offsets-24 -> -1, sub_lidaTopic-3 -> -1,
> sub_mobileGuessGameReward-0 -> -1, pub_lidaTopic-6 -> -1, sub_NewUserAlgo-0
> -> -1, __consumer_offsets-48 -> -1, pub_RsUserBehavior-3 -> -1,
> sub_channelinfoupdate-0 -> -1, pub_tinyVideoComment-1 -> -1, pub_bulletin-2
> -> -1, pub_RecordCompleteNotifition-6 -> -1, sub_lidaTopic2-3 -> -1,
> smsgateway-10 -> -1, __consumer_offsets-0 -> -1, pub_baseonlinetest-1 -> -1,
> __consumer_offsets-12 -> -1, pub_myshardUinfo-0 -> -1, pub_baseonline-3 ->
> -1, smsGatewayMarketDbInfo-6 -> -1, sub_tinyVideoComment-0 -> 14)
> (kafka.server.ReplicaFetcherThread)
> java.io.IOException: Connection to 401 was disconnected before the response
> was read
> at
> org.apache.kafka.clients.NetworkClientUtils.sendAndReceive(NetworkClientUtils.java:95)
> at
> kafka.server.ReplicaFetcherBlockingSend.sendRequest(ReplicaFetcherBlockingSend.scala:96)
> at
> kafka.server.ReplicaFetcherThread.fetchEpochsFromLeader(ReplicaFetcherThread.scala:312)
> at
> kafka.server.AbstractFetcherThread.maybeTruncate(AbstractFetcherThread.scala:130)
> at kafka.server.AbstractFetcherThread.doWork(AbstractFetcherThread.scala:102)
> at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:64){code}
>
> on the leader(broker-401) side, the log shows:
> {code:java}
> [2018-02-02 08:58:26,859] ERROR Closing socket for
> 192.168.100.101:9099-192.168.100.103:30476 because of error
> (kafka.network.Processor)
> org.apache.kafka.common.errors.InvalidRequestException: Error getting request
> for apiKey: 23 and apiVersion: 0
> Caused by: java.lang.IllegalArgumentException: Unexpected ApiKeys id `23`, it
> should be between `0` and `20` (inclusive)
> at org.apache.kafka.common.protocol.ApiKeys.forId(ApiKeys.java:73)
> at
> org.apache.kafka.common.requests.AbstractRequest.getRequest(AbstractRequest.java:39)
> at
> kafka.network.RequestChannel$Request.liftedTree2$1(RequestChannel.scala:96)
> at kafka.network.RequestChannel$Request.(RequestChannel.scala:91)
> at
> kafka.network.Processor$$anonfun$processCompletedReceives$1.apply(SocketServer.scala:492)
> at
> kafka.network.Processor$$anonfun$processCompletedReceives$1.apply(SocketServer.scala:487)
> at scala.collection.Iterator$class.foreach(Iterator.scala:893)
> at scala.collection.AbstractIterator.foreach(Iterator.scala:1336)
> at scala.collection.IterableLike$class.foreach(IterableLike.scala:72)
> at scala.collection.AbstractIterable.foreach(Iterable.scala:54)
> at kafka.network.Processor.processCompletedReceives(SocketServer.scala:487)
> at kafka.network.Processor.run(SocketServer.scala:417)
> at java.lang.Thread.run(Thread.java:745){code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
